Oracle 11gR2 RAC startup sequence

The below article applies to Oracle RAC 11GR2 and further releases.

Below are the steps in brief:

1.     Startup of ohasd.bin process. it further startup the ohasd.bin process.

2.     ohasd access OLR file and get GPNP Profile location and spawns cssd process.

3.     CSSD daemon access GPNP Profile, get voting disk location, access it and join the cluster.

4.     Ohasd process access the ASM spfile and startup the ASM instance.

5.     Ohasd’s orarootagent process starts up the crsd process. The crsd process reads OCR file and completes the cluster initialization.

6.     The crsd process spawns the oraagent and orarootagent process and these two further spawns the subsequent crs resources processes.

The below are the steps with detailed information.

 1.     Startup of ohasd.bin process.

The init.ohasd process entries are present in /etc/inittab file, so it get started automatically by the OS after server boots up. This entry is added by root.sh script after grid installation. The entry is like below:

 

h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null

 

35 means run level i.e. 

3 is for Multiuser mode with Networking 

5 is for Multiuser mode with networking and GUI

 

It means the OS must have the run level equals to either 3 or 5 to get this process started. We can check the current run level of the OS using “who –r” command.

 

[oracle@edwdev02 ~]$ who -r

         run-level 3  2021-06-24 12:35

[oracle@edwdev02 ~]$

 

respawn means if the process was not started, then start it.

 

The process init has pid=1,

It’s absolute location is /sbin/init on Linux, Solaris and hp-ux OS platforms

Whereas its location is /usr/sbin/init on AIX OS Platforms.

 

[root@node1 ~]# ps -ef|grep init

root         1     0  0 Jun24 ?        00:00:44 /sbin/init

oracle   31134 31026  0 19:29 pts/1    00:00:00 grep init

[root@node1 ~]#

 

This init process starts “init.ohasd”process. This init.ohasd process will further startup ohasd.bin process which is also called ohasd service.

 

2.     ohasd access OLR file and get GPNP Profile location and spawns cssd process.

The OLR file location is present in /etc/oracle/olr.loc. PFB the reference output:

[root@node1 ~]# cat /etc/oracle/olr.loc

olrconfig_loc=/u01/app/11.2.0/grid/cdata/node1.olr

crs_home=/u01/app/11.2.0/grid

 

[root@node1 ~]# ls -ltr /u01/app/11.2.0/grid/cdata/node1.olr

-rw-------. 1 root oinstall 272756736 Jan  8 21:40 /u01/app/11.2.0/grid/cdata/node1.olr

[root@node1 ~]#

 

The OLR file is present in $GRID_HOME/cdata/ directory. It is named as ‘<hostname>.olr’

The ohasd process get the location of OLR file from /etc/oracle/olr.loc and access the main OLR file and gets the below info which is stored in it:

          I.        GPNP file details. It is present in $GRID_HOME/gpnp/<hostname>/profiles/peer/

         II.        OCR latest backup time and location.

        III.        CRS_HOME

       IV.        Active crs version.

        V.        Local host name and crs version

       VI.        Status of the resources to know which are to be started and which are to be kept OFFLINE.

      VII.        Start and stop dependencies of the resouces i.e.

Hard Dependency means it must full-fill

Weak Dependency means it should full-fill.

GPNP Profile stands for Grid Plug and Play Profile. The file is located in CRS_HOME/gpnp/<node_name>/profile/peer/profile.xml. And this profile consists of cluster name, hostname, network profiles with IP addresses, OCR. If we do any modifications for voting disk, profile will be updated.

ohasd.bin process then further spawns:
   1. its orarootagent process.
   2. its oraagent process.
   3. cssdagent process.
   4. cssdmonitor process.

The ohasd’s orarootagent process further spawns the gpnpd process and cssdagent process further spawns the cssd daemon process.

 

3.     CSSD daemon access GPNP Profile, get voting disk location, access it and join the cluster.

 

The cssd daemon then access the local GPNP profile of the node at /u01/app/12.2.0.2/grid/gpnp/node2/profiles/peer/profile.xml. It contains the below info:

 

   i. Name of the ASM Diskgroup containing the Voting Files
   ii. ASM SPFILE location (Diskgroup name)
   iii. ASM Diskgroup Discovery String

 

Fyi below info:
Oracle ASM reserves several physical blocks at a fixed location for every Oracle ASM disk to store the the voting disk file i.e. the physical location of voting disk file is fixed on each asm disk. As a result , Oracle Clusterware can access the voting disks present in ASM even if the ASM instance is down.

The cssd process mainly access the voting disk file’s diskgroup location from it. It then physically scan the headers of each disk present in that diskgroup and find out the disks which contains the voting disk file. Now, it has got the voting disk files, so it will now join the cluster.

 

4.     Ohasd process access the ASM spfile and startup the ASM instance.

The same way ASM spfile is read to startup the asm instance.

5.     Ohasd’s orarootagent process starts up the crsd process. the crsd process reads OCR file and completes the cluster initialization.

6.     The crsd process spawns the oraagent and orarootagent process.

 

The CRSD orarootagent then further spawns the below resources:
   1. Network resource : for the public network
   2. SCAN VIPs
   3. Node VIPs : VIPs for each node
   4. ACFS Registry
   5. GNS VIP : VIP for GNS if you use the GNS option

and the CRSD oraagent spawns the below resources:

   1. ASM Resources
   2. Diskgroups
   3. DB Resources
   4. SCAN Listener : Listener for SCAN listening on SCAN VIP
   5. Listener : Node Listener listening on the Node VIP
   6. Services : Database Services
   7. ONS
   8. GSD
   9. GNS