OraFAQ Forum: Server Administration » ASM rebalance operation never end

Home » RDBMS Server » Server Administration » ASM rebalance operation never end (Oracle DB 11.1.0.6 on Sun Solaris 10)

Show: Today's Messages :: Polls :: Message Navigator
E-mail to friend

ASM rebalance operation never end [message #523458]

Mon, 19 September 2011 00:19

luc_tran
Messages: 31
Registered: October 2010
Location: Viet Nam

Member

Hi all,

I had added a new disk into disk group DATA1 with rebalance power of 5, it ran as estimated for about 2 days long, I can see the estimated time down to zero but the process never end.

The command used:

SQL> alter diskgroup DATA1 add disk '<path>' rebalance power 5;

Below is the disk status:

SQL> select group_number, disk_number, total_mb, free_mb, mount_status, header_status, state 
from v$asm_disk where group_number=1 order by disk_number;

GROUP_NUMBER DISK_NUMBER   TOTAL_MB    FREE_MB MOUNT_S HEADER_STATU STATE
------------ ----------- ---------- ---------- ------- ------------ --------
           1           0    1191626     314236 CACHED  MEMBER       NORMAL
           1           1    1191626     314230 CACHED  MEMBER       NORMAL
           1           2    1191626     314232 CACHED  MEMBER       NORMAL
           1           3    1191602     314229 CACHED  MEMBER       NORMAL
           1           4    1191626     314210 CACHED  MEMBER       NORMAL
           1           5    1191626     314218 CACHED  MEMBER       NORMAL
           1           6    1191626     314218 CACHED  MEMBER       NORMAL
           1           7    1191602     314223 CACHED  MEMBER       NORMAL
           1           8    1191626     314216 CACHED  MEMBER       NORMAL
           1           9    1191626     314202 CACHED  MEMBER       NORMAL
           1          10    1191626     314230 CACHED  MEMBER       NORMAL

GROUP_NUMBER DISK_NUMBER   TOTAL_MB    FREE_MB MOUNT_S HEADER_STATU STATE
------------ ----------- ---------- ---------- ------- ------------ --------
           1          11    1191602     314237 CACHED  MEMBER       NORMAL
           1          12    1191626     314229 CACHED  MEMBER       NORMAL
           1          13    1191626     314221 CACHED  MEMBER       NORMAL
           1          14    1191626     314220 CACHED  MEMBER       NORMAL
           1          15    1191602     314225 CACHED  MEMBER       NORMAL

16 rows selected.

Disk 15 is the new one, current rebalance status:

SQL> select group_number, operation, state, power, est_minutes from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER EST_MINUTES
------------ ----- ---- ---------- -----------
           1 REBAL RUN           5           0

I've checked and follow the status for a few days more, it still running with EST_MINUTES=0. I think it was hang and never end.

During the next few weeks, I've manually restarted the rebalance process by modifying its power to 6, 8, 10

SQL> alter diskgroup DATA1 rebalance power 10;

After every restart, the diskgroup rebalance again but has the same issue, never end.

Here is the content in the alert log

NOTE: starting rebalance of group 1/0x9d2529a0 (DATA1) at power 5
Starting background process ARB0
Fri Sep 16 12:01:11 2011
ARB0 started with pid=15, OS id=21066
Starting background process ARB1
Starting background process ARB2
Fri Sep 16 12:01:11 2011
ARB1 started with pid=16, OS id=21068
Starting background process ARB3
Fri Sep 16 12:01:11 2011
ARB2 started with pid=17, OS id=21070
Starting background process ARB4
Fri Sep 16 12:01:11 2011
ARB3 started with pid=18, OS id=21072
Fri Sep 16 12:01:11 2011
ARB4 started with pid=19, OS id=21074
NOTE: assigning ARB0 to group 1/0x9d2529a0 (DATA1)
Fri Sep 16 12:21:23 2011
NOTE: assigning ARB1 to group 1/0x9d2529a0 (DATA1)
Fri Sep 16 12:21:41 2011
NOTE: assigning ARB2 to group 1/0x9d2529a0 (DATA1)
NOTE: assigning ARB3 to group 1/0x9d2529a0 (DATA1)
NOTE: assigning ARB4 to group 1/0x9d2529a0 (DATA1)

As you can see, the rebalance process started 3 days ago, hanging till now

SQL> select group_number, operation, state, power, est_minutes from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER EST_MINUTES
------------ ----- ---- ---------- -----------
           1 REBAL RUN           5           0

Could you please tell me how to solve this problem, thanks a lot!

Luc

[Updated on: Mon, 19 September 2011 00:45] by Moderator

Report message to a moderator