OraFAQ Forum: Server Administration » Unable to start second node in RAC

Home » RDBMS Server » Server Administration » Unable to start second node in RAC (RHEL 5.1; Oracle 10G R2)

Show: Today's Messages :: Polls :: Message Navigator
E-mail to friend

Unable to start second node in RAC [message #499941]

Wed, 16 March 2011 16:23

akshay_johari
Messages: 4
Registered: March 2011
Location: INDIA

Junior Member

Hi,

I have setup a RAC machine of two nodes on Oracle 10GR2(with 10.2.0.4 Patchset). All setup went well. I am able to connect one node from sqlplus but the other node is connected to idle instance. I tried to startup the database with mount and nomount option but it fails. Following is the error :

SQL> startup nomount;
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/LCLCLKS/spfileLCLCLKS.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA/LCLCLKS/spfileLCLCLKS.ora
ORA-15077: could not locate ASM instance serving a required diskgroup
SQL>

I am using ASM and it is running on both nodes even I am able to connect the SQL prompt from it. I am able to view the diskgroups I created from the second node but through "asmcmd" option I can't go inside the diskgroup. It gives :

ASMCMD> ls
DATA/
INDEX/
ASMCMD> cd DATA
asmcmd: diskgroup 'data' does not exist or is not mounted

Please suggest how I resolve this issue. On first node, everything is fine and I am able to import the database on it, as well.

Thanks, Aky

Report message to a moderator

Re: Unable to start second node in RAC [message #499964 is a reply to message #499941]

Thu, 17 March 2011 00:22

Michel Cadot
Messages: 68648
Registered: March 2007
Location: Nanterre, France, http://...

Senior Member
Account Moderator

Connect to ASM instances (on both nodes) and execute:

select a.name dg_name, state, total_mb, offline_disks
from v$asm_diskgroup a
order by 1
/
select d.name disk_name, 
       case 
         when mount_status = 'CLOSED' and header_status in ('CANDIDATE', 'FORMER') 
           then 'Free'
         else d.state
       end state,
       d.mount_status, d.header_status, d.total_mb, g.name group_name
from v$asm_disk d, v$asm_diskgroup g
where g.group_number = d.group_number
order by 1
/

Regards
Michel

[Updated on: Thu, 17 March 2011 00:25]

Report message to a moderator

Re: Unable to start second node in RAC [message #500083 is a reply to message #499964]

Thu, 17 March 2011 13:01

akshay_johari
Messages: 4
Registered: March 2011
Location: INDIA

Junior Member

Thanks for reply. Following is the output of the given command

SQL> set head off;
SQL> set lines 1000;
SQL> set pages 0;
SQL> select a.name dg_name, state, total_mb, offline_disks
2 from v$asm_diskgroup a
3 order by 1
4 /
select d.name disk_name,
case
when mount_status = 'CLOSED' and header_status in ('CANDIDATE', 'FORMER')
then 'Free'
else d.state
end state,
d.mount_status, d.header_status, d.total_mb, g.name group_name
from v$asm_disk d, v$asm_diskgroup g
where g.group_number = d.group_number
order by 1
/DATA MOUNTED 297555 0
INDEX MOUNTED 195321 0

SQL> 2 3 4 5 6 7 8 9 10 11
VOL1 NORMAL CACHED MEMBER 297555 DATA
VOL2 NORMAL CACHED MEMBER 195321 INDEX

The output is same on both nodes.

Thanks, Aky

Report message to a moderator

Re: Unable to start second node in RAC [message #500089 is a reply to message #500083]

Thu, 17 March 2011 13:22

Michel Cadot
Messages: 68648
Registered: March 2007
Location: Nanterre, France, http://...

Senior Member
Account Moderator

This is unreadable.
Put the queries inside a script and execute it, do not copy and paste inside your unix windows we can't read the result, can you?

In addition, Please read OraFAQ Forum Guide, especially "How to format your post?" section.
Make sure that lines of code do not exceed 80 characters when you format.
Use the "Preview Message" button to verify.
See how I posted the queries and see how they come in your post; don't you see the difference?

In the end, add "select * from v$version;" in both your ASM instances and DB instance you can start.

Regards
Michel

Report message to a moderator

Re: Unable to start second node in RAC [message #500101 is a reply to message #500089]

Thu, 17 March 2011 14:45

akshay_johari
Messages: 4
Registered: March 2011
Location: INDIA

Junior Member

On First node, result is(Same on ASM and DB instance) :

DG_NAME                        STATE         TOTAL_MB OFFLINE_DISKS
------------------------------ ----------- ---------- -------------
DATA                           CONNECTED       297555             0
INDEX                          CONNECTED       195321             0


DISK_NAME                      STATE    MOUNT_S HEADER_STATU   TOTAL_MB
------------------------------ -------- ------- ------------ ----------
GROUP_NAME
------------------------------
VOL1                           NORMAL   OPENED  UNKNOWN          297555
DATA

VOL2                           NORMAL   OPENED  UNKNOWN          195321
INDEX

select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi
PL/SQL Release 10.2.0.4.0 - Production
CORE    10.2.0.4.0      Production
TNS for Linux: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production

But on Second node, it is:

ASM result:

DG_NAME                        STATE         TOTAL_MB OFFLINE_DISKS
------------------------------ ----------- ---------- -------------
DATA                           MOUNTED         297555             0
INDEX                          MOUNTED         195321             0


DISK_NAME                      STATE    MOUNT_S HEADER_STATU   TOTAL_MB
------------------------------ -------- ------- ------------ ----------
GROUP_NAME
------------------------------
VOL1                           NORMAL   CACHED  MEMBER           297555
DATA

VOL2                           NORMAL   CACHED  MEMBER           195321
INDEX

DB Instance result:

DG_NAME                        STATE         TOTAL_MB OFFLINE_DISKS
------------------------------ ----------- ---------- -------------
DATA                           MOUNTED         297555             0
INDEX                          MOUNTED         195321             0


no rows selected

select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi
PL/SQL Release 10.2.0.4.0 - Production
CORE    10.2.0.4.0      Production
TNS for Linux: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production

Report message to a moderator

Re: Unable to start second node in RAC [message #500104 is a reply to message #500101]

Thu, 17 March 2011 14:56

Michel Cadot
Messages: 68648
Registered: March 2007
Location: Nanterre, France, http://...

Senior Member
Account Moderator

You say that you have been able to query both DB databases but in your first post you said you cannot even nomount it!
So it seems that now everything is fine.

Regards
Michel

Report message to a moderator

Re: Unable to start second node in RAC [message #500110 is a reply to message #500104]

Thu, 17 March 2011 15:09

akshay_johari
Messages: 4
Registered: March 2011
Location: INDIA

Junior Member

I restarted the 2nd machine and after that I am able to connect to start database with nomount option. But when I try to mount the database, it gives the following:

SQL> startup;
ORACLE instance started.

Total System Global Area 1610612736 bytes
Fixed Size                  2084296 bytes
Variable Size             385876536 bytes
Database Buffers         1207959552 bytes
Redo Buffers               14692352 bytes
ORA-00600: internal error code, arguments: [kccchb_3], [4555], [4535], [0], [],
[], [], []


SQL>

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01507: database not mounted


SQL> alter database mount;
`alter database mount
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [kccchb_3], [4555], [4535], [0], [],
[], [], []


SQL>

Also, I found the following in listener status :

Instance "+ASM2", status BLOCKED, has 1 handler(s) for this service...

Instance "LCLCLKS2", status BLOCKED, has 2 handler(s) for this service...

Thanks, Aky

Report message to a moderator

Re: Unable to start second node in RAC [message #500111 is a reply to message #500110]

Thu, 17 March 2011 15:12

BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal

Senior Member

ORA-00600/ORA-07445/ORA-03113 = Oracle bug => search on Metalink and/or call Oracle support

Report message to a moderator

Re: Unable to start second node in RAC [message #500697 is a reply to message #500111]

Tue, 22 March 2011 22:05

Arju
Messages: 1554
Registered: June 2007
Location: Dhaka,Bangladesh. Mobile:...

Senior Member

Send me the alert log and trace file whenever you perform alter database mount.

Report message to a moderator

Re: Unable to start second node in RAC [message #500705 is a reply to message #500697]

Wed, 23 March 2011 00:25

Michel Cadot
Messages: 68648
Registered: March 2007
Location: Nanterre, France, http://...

Senior Member
Account Moderator

Do you mean if he sends it you will break your Oracle license and investigate for him the problem?

Regards
Michel

[Updated on: Wed, 23 March 2011 00:25]

Report message to a moderator

Previous Topic:	Create User Email
Next Topic:	SYS objects and Catproc Invalid

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Thu May 09 07:33:25 CDT 2024