Oracle Restart: srvctl fails to start database with error CRS-5010

I’ve installed Oracle Grid Infrastructure on one of my standalone test servers. This software includes Oracle ASM as volume manager and Oracle Restart for managing Oracle components. As recommended, Grid Infrastructure and RDBMS software are installed under different users.

Grid Infrastructure: 11.2.0.3.0
RDBMS: 11.2.0.3.0
OS: OEL 5.8

After rebooting server I’ve noticed that database is not up so I’ve tried to run startup.

# Display status of the database
[grid@l01test1 ~]$ srvctl status database -d testdb
Database is not running.

# Startup
[grid@l01test1 ~]$ srvctl start database -d testdb
PRCR-1079 : Failed to start resource ora.testdb.db
CRS-5010: Update of configuration file "/u02/app/orcl/product/11.2.0/dbhome_1/dbs/inittestdb.ora" failed: details at "(:CLSN00014:)" in "/u02/app/11.2.0/grid/log/l01test1/agent/ohasd/oraagent_grid/oraagent_grid.log"
CRS-5017: The resource action "ora.testdb.db start" encountered the following error:
CRS-5010: Update of configuration file "/u02/app/orcl/product/11.2.0/dbhome_1/dbs/inittestdb.ora" failed: details at "(:CLSN00014:)" in "/u02/app/11.2.0/grid/log/l01test1/agent/ohasd/oraagent_grid/oraagent_grid.log"
. For details refer to "(:CLSN00107:)" in "/u02/app/11.2.0/grid/log/l01test1/agent/ohasd/oraagent_grid/oraagent_grid.log".

CRS-2674: Start of 'ora.testdb.db' on 'l01test1' failed

Hm, startup procedure failed because inittestdb.ora was missing.

Let’s display configuration of the database and check spfile location.

# Display database configuration
[grid@l01test1 ~]$ srvctl config database -d testdb -a

Database unique name: testdb
Database name: testdb
Oracle home: /u02/app/orcl/product/11.2.0/dbhome_1
Oracle user: orcl
Spfile: +DATA/testdb/spfiletestdb.ora
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Database instance: testdb
Disk Groups: DATA,FRA,DISK2
Services:
Database is enabled

[grid@l01test1 ~]$ asmcmd
ASMCMD> cd +DATA/testdb/
ASMCMD> ls
CONTROLFILE/
DATAFILE/
ONLINELOG/
PARAMETERFILE/
TEMPFILE/
mydir/
spfiletestdb.ora

Everything seemed fine to me.

Later I’ve found out why startup sequence failed.
Oracle tried to create "inittestdb.ora" file as grid user in $ORACLE_HOME/dbs folder of another user. It failed due to insufficient privileges.

To overcome this error I’ve changed permissions on $ORACLE_HOME/dbs folder.

[orcl@l01test1 dbhome_1]$ chmod 775 $ORACLE_HOME/dbs

Now when we start database everything works fine.

# Start
[grid@l01test1 tmp]$ srvctl start database -d testdb

# Check status
[grid@l01test1 tmp]$ srvctl status database -d testdb
Database is running.

With changed privileges grid user managed to create “inittestdb.ora” file in $ORACLE_HOME/dbs folder. This “inittestdb.ora” file has only one row - pointer to server parameter file.

[orcl@l01test1 dbs]$ ls -l inittestdb.ora
-rw-r--r-- 1 grid oinstall 62 Apr 14 10:37 inittestdb.ora

[orcl@l01test1 dbs]$ cat inittestdb.ora
SPFILE='+DATA/testdb/spfiletestdb.ora'          # line added by Agent

REFERENCES
Oracle Restart: srvctl fails to start database with error CRS-5010 if RDBMS and Grid under different users [ID 1335607.1]
http://docs.oracle.com/cd/E11882_01/server.112/e25494/restart005.htm#i1009665