Re-sizing the ASM diskgroup, when one of the disks in DG in hung state

Recently came across this scenario.

I had a development server with ASM Normal redundancy diskgroup, with one of the disk in HUNG state for some reason. No spare disks available which could be added in the diskgroup. Requirement was to resize the surviving ASM disk.

If we try to resize the diskgroup, ASM won’t allow that. It needs both the disks online. 

SQL> select name, path, header_status, state, failgroup, total_mb, 
free_mb from v$asm_disk
/
NAME           HEADER_STATUS  STATE    FAILGROUP   TOTAL_MB    FREE_MB
-------------- ------------  -------- ----------- ---------- --------
DATADG1_0000    CANDIDATE     HUNG                        0 0
DATADG1_0001   MEMBER        NORMAL   DATADG1_FG2    51200 951

So the solution was to resize the failgroup, instead of whole diskgroup.  This is not the solution to be used in production though!!

ASM alert log, when tried resizing the diskgroup

SQL> alter diskgroup DATADG1 resize all
Wed Apr 13 03:41:33 2011
SQL> alter diskgroup  DATADG1 resize disk  DATADG1_0001 size 13239M
Wed Apr 13 03:41:33 2011
NOTE: requesting all-instance PST refresh for group=1
Wed Apr 13 03:41:33 2011
NOTE: PST refresh pending for group 1/0xadb30ab8 (DATADG1)
SUCCESS: refreshed PST for 1/0xadb30ab8 (DATADG1)
Wed Apr 13 03:41:36 2011
NOTE: starting rebalance of group 1/0xadb30ab8 (DATADG1) at power 1
Starting background process ARB0
ARB0 started with pid=12, OS id=28053
Wed Apr 13 03:41:36 2011
NOTE: assigning ARB0 to group 1/0xadb30ab8 (DATADG1)
Wed Apr 13 03:41:36 2011
Errors in file /u01/app/10.2/admin/+ASM/bdump/+asm_arb0_28053.trc:
ORA-15041: diskgroup space exhausted
NOTE: stopping process ARB0
Wed Apr 13 03:41:39 2011
WARNING: rebalance not completed for group 1/0xadb30ab8 (DATADG1)

Trace file  /u01/app/10.2/admin/+ASM/bdump/+asm_arb0_28053.trc

Oracle Database 10g Release 10.2.0.2.0 - 64bit Production
ORACLE_HOME = /u01/app/10.2/product/10.2.0
System name:    Linux
Node name:      test-srv-001
Release:        2.6.5-7.244-smp
Version:        #1 SMP Mon Dec 12 18:32:25 UTC 2005
Machine:        x86_64
Instance name: +ASM
Redo thread mounted by this instance: 0 <none>
Oracle process number: 12
Unix process pid: 28053, image: oracle@test-srv-001 (ARB0)
*** SERVICE NAME:() 2011-04-13 03:41:36.358
*** SESSION ID:(39.11) 2011-04-13 03:41:36.358
ARB0 relocating file +DATADG1.1.1 (2 entries)
ORA-15041: diskgroup space exhausted

SQL> alter diskgroup datadg1 resize disks in 
failgroup datadg1_failure_group2;

Diskgroup altered.

SQL> select name, path, header_status, state, failgroup, total_mb, 
free_mb from v$asm_disk
/

NAME           HEADER_STATUS  STATE    FAILGROUP   TOTAL_MB    FREE_MB
-------------- ------------  -------- ----------- ---------- --------
DATADG1_0000 CANDIDATE     HUNG                        0 0
DATADG1_0001 MEMBER        NORMAL   DATADG1_FG2    68608   18359

ASM alert log, when tried resizing the failgroup

Wed Apr 13 03:41:39 2011
SUCCESS: rebalance completed for group 1/0xadb30ab8 (DATADG1)
NOTE: PST update: grp = 1
Wed Apr 13 03:41:39 2011
NOTE: group DATADG1: relocated PST to: disk 0001 (PST copy 0)
Wed Apr 13 03:41:39 2011
WARNING: disk DATADG1_0001 has contents beyond AU 13239
NOTE: PST update: grp = 1
Wed Apr 13 03:41:39 2011
NOTE: group DATADG1: relocated PST to: disk 0001 (PST copy 0)
Wed Apr 13 03:42:16 2011
Advertisements
This entry was posted in Oracle Automatic Storage Management and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s