Wednesday 17 September 2014

ERROR: NMO not setuid-root (Unix-only)

Thus error came up while I was specifying a host credentials on EM 12c.

 After some research I found that I had not run root.sh on the agent home.

Solution:

# cd $AGENT_HOME
# ./root.sh
Finished product-specific root actions.
/etc exist
Finished product-specific root actions.
#

This solved my problem!


Wednesday 10 September 2014

Setup of RMAN Catalog

The benefits of running RMAN backup and recovery with a catalog cannot not be overstated. A simple internet search will reveal several reasons why a serious DBA handling several databases should consider setting up a recovery catalog.

The steps involved are pretty straight forward.

1. Creation of the catalog database using DBCA or scripts.



2. Creation of catalog owner and granting necessary roles and privileges.

SQL> create user mycat identified by mycat;

- Grant necessary roles, priviledges to user

SQL> grant recovery_catalog_owner to mycat;
SQL> grant connect, resource to mycat;

3. Creation of the recovery catalog

$ rman catalog mycat/mycat
RMAN> create catalog;

4. Registering the databases which will use this database as recovery catalog



run these commands from each of the concerned databases.


$ rman target / catalog mycat/mycat@CATDB
RMAN> register database;

- Verify that the registration was successful by running REPORT SCHEMA

RMAN> report schema;

To unregister a database TESTDB from RMAN catalog, there are two ways.



1. Unregistering from the target database.


Connect to the target and catalog

rman target / catalog=mycat/mycat@catdb

perform clean -up by deleting existing backups

RMAN> LIST BACKUP SUMMARY;
RMAN> DELETE BACKUP DEVICE TYPE SBT;
RMAN> DELETE BACKUP DEVICE TYPE DISK;

Unregister database

RMAN> UNREGISTER DATABASE;

OR

RMAN> UNREGISTER DATABASE NOPROMPT;

2. Unregistering when there is no access to the target database.

Start RMAN, connect only to the catalog.

$ rman catalog=mycat/mycat@catdb

Unregister the database by name.

RMAN> UNREGISTER DATABASE TESTDB NOPROMPT;

In a case where there is more than one database in the catalog with the same name, you will need to use the DBID to identify the database to unregister. You can find this using the LIST INCARNATION command.

RMAN> LIST INCARNATION OF DATABASE TESTDB;

Once you have the DBID, you can unregister the database using the following script.

    RUN
    { 
      SET DBID 1111111111;
      UNREGISTER DATABASE TESTDB NOPROMPT;
    }









ORA-12514: TNS listener cannot resolve service name error

This came up after I created a test RMAN catalog database on one of my test serves that had three other databases running on it. These databases are actually ones that I restored while testing my full database backups, therefore I never really bothered about the networking aspect of them as I never connect to them from outside the server.

I had to setup the tnsnames.ora and listener configuartion to enable connections to and from the RMAN catalog database. This was a pretty straight forward thing but I unfortunately skipped a key configuration which caused the error ORA-12514 every time I tried to connect to the Catalog DB from any of the others.

Note that #tnsping CATDB was successful, which got me more confused and at the same time convinced that all was well with the SERVICE_NAME definition.

I also ran #lsnrctl service and I could see the CATDB service running.

However, the problem was the SERVICE_NAME I defined for the Catalog DB. 

Original configuration in tnsnames.ora

CATDB=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL = TCP)(HOST =
1.1.1.1)(PORT = 1521))
                (CONNECT_DATA = (SERVICE_NAME = CATDB))


Correct configuration

CATDB=
        (DESCRIPTION=
                (ADDRESS=(PROTOCOL = TCP)(HOST = 1.1.1.1)(PORT = 1521))
                (CONNECT_DATA = (SERVICE_NAME = CATDB.xxxxxx.local))


I did not fully specify the service name!

This resolved my problem and I was able to connect to the Catalog DB.



Tuesday 9 September 2014

ORA-01180: can not create datafile 1 ORA-01110: data file 1: '/dir/system01.dbf' during RMAN restore.

I got this error while trying do a full database restore to another host. I have always done same without issues.

Changes:
The source database is now the primary in a ADG set up.
Source database now has FRA confirgured.
Source database opened with reset logs during a migration to a new host as the original host had to be maintained.

Road to resolution:

At first I checked the directory on the new host to confirm that oracle user has appropriate permission to write to the directory.

I also checked to ensure that the backup sets are fully cataloged and that the restored control file is the most recent.

I tried the process one more time and still got the same error.

I went on MOS and found two notes ID 1265151.1 & ID 392237.1.
 
RMAN restore fails with ORA-01180: can not create datafile 1 (Doc ID 1265151.1)


RMAN restore of database fails with ORA-01180: Cannot create datafile 1 (Doc ID 392237.1)

"The current incarnation only contains incremental level 1 backups of the database.
There are NO level 0 backups in this incarnation.
As the physical datafiles do not exist RMAN attempts to create the physical files but this is NOT allowed if the file belongs to the SYSTEM tablespace."

solution: Reset the database to the previous incarnation and run the restore and recovery again:
 select count(*) from v$database_incarnation;

  COUNT(*)
----------
         4
SQL> select INCARNATION#, RESETLOGS_TIME from  v$database_incarnation order by RESETLOGS_TIME desc;

INCARNATION# RESETLOGS_TIME
------------ --------------------
           4 02:28:32 19:aug:2014
           2 04:23:11 08:aug:2014
           3 10:03:41 25:jul:2014
           1 07:01:36 09:dec:2011

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

>rman target /

Recovery Manager: Release 11.2.0.2.0 - Production on Tue Sep 9 13:04:29 2014

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

connected to target database: XXXXXX (DBID=111111111, not open)

RMAN> reset database to incarnation 2;

RMAN>restore database;


Walla!!! it was successful.









Wednesday 27 August 2014

ORA-12516: TNS:listener could not find available handler with matching protocol

I got this error while trying to connect to a database from one of our new application servers.

oerr returned the following:

$ oerr ora 12516
12516, 00000, "TNS:listener could not find available handler with matching protocol stack"
// *Cause: None of the known and available service handlers for the given
// SERVICE_NAME support the client's protocol stack: transport, session,
// and presentation protocols.
// *Action: Check to make sure that the service handlers (e.g. dispatchers)
// for the given SERVICE_NAME are registered with the listener, are accepting
// connections, and that they are properly configured to support the desired
// protocols.


This message did not help as I had ensured that the tnames.ora is properly configured and I was sure of the SERVICE name.

SOLUTION:

 I increased processes parameter to 3000.

alter system set processes=3000 scope=spfile;

The database was restarted. Then the error below showed up!

ORA-00371: not enough shared pool memory, should be atleast 5207647846 bytes

This was fixed by modifying the shared_pool_memory parameter to a little over the suggested value above.

Issued solved!

renaming an Oracle 11.2.0.2 database



To change an Oracle 11.2.0.2 DB name from DB1 to DB2

    STEP 1: Backup the database.
   
    STEP 2: Do a cleab shutdown of the database and mount the database after:

        SQL> SHUTDOWN IMMEDIATE
        SQL> STARTUP MOUNT

    STEP 3: Invoke the DBNEWID utility (nid) specifying the new DBNAME from the command line using a user with SYSDBA privilege:

        nid TARGET=sys/password@DB1 DBNAME=DB2

        # nid TARGET=sys/password@TSH1 DBNAME=DB2
        DBNEWID: Release 11.2.0.2.0 - Production on Wed Aug 27 12:12:52 2014

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

########################
########################
########################
########################
########################
########################
########################

        Database name changed to DB2.
        Modify parameter file and generate a new password file before restarting.
        Database ID for database DB2 changed to 3437107090.
        All previous backups and archived redo logs for this database are unusable.
        Shut down database and open with RESETLOGS option.
        Succesfully changed database name and ID.
        DBNEWID - Completed succesfully.

    STEP 4: Create new pfile from spfile.

    Shutdown the database:

        SHUTDOWN IMMEDIATE

    STEP 5: Modify the DB_NAME parameter in the initialization parameter file to reflect the new DB name .

      
    STEP 6: Create a new password file:

      
    orapwd file=$ORACLE_HOME/dbs/orapwDB2 password=choicepassword


    STEP 7: Rename the SPFILE to match the new DBNAME.
   

    STEP 8: On UNIX/Linux, reset the ORACLE_SID environment variable:
   

        ORACLE_SID=DB2; export ORACLE_SID

    STEP 9: Alter the listener.ora and tnsnames.ora setting to match the new database name and restart the listener:
   

        lsnrctl reload

    STEP 10: Open the database with RESETLOGS:
   

        STARTUP MOUNT
        ALTER DATABASE OPEN RESETLOGS;

    STEP 11: Backup the database.



 ##########################################################################################################################

Issues encountered.



execution of DBNEWID failed initially with the error shown below:

NID-00111: Oracle error reported from target database while executing
    begin       dbms_backup_restore.nidprocessdf(:fno, :istemp, :skipped, :idchged,                                                                          :nmchged);    end;
ORA-01000: maximum open cursors exceeded



Change of database name and database ID failed.
Must finish change or REVERT changes before attempting any database operation.
DBNEWID - Completed with errors.



I tried to reverse my action using

 nid target=/ REVERT=Y

This also failed with the error below:

NID-00111: Oracle error reported from target database while executing
    begin       dbms_backup_restore.nidprocessdf(:fno, :istemp, :skipped, :idchged,                                                                          :nmchged);    end;
ORA-01000: maximum open cursors exceeded



Reversion of database name and database ID failed.
Must finish REVERT changes before attempting any database operation.
DBNEWID - Completed with errors.



Resolution:

I had to correct the error ORA-01000: maximum open cursors exceeded

SYS> show parameter open_cursors

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
open_cursors                         integer     100

SYS> alter system set open_cursors=1024 SCOPE=BOTH;

System altered.

SYS> show parameter open_cursors

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
open_cursors                         integer     1024


The rename ran successfully after correcting this error.

###########################################################################################################

BEST PRACTICES:

Take controlfile backups – binary and trace backups before the bid operation.

If there are tempfiles in the dictionary, make sure they are accessible. Check both v$tempfile and dba_temp_files, not just dba_temp_files. The nid operation tries to update the dbname in tempfiles too and if it is not found, the operation fails. If there are tempfiles that doesn’t exist, it is better to drop them from the dictionary before the nid operation.

nid requires the database to be mounted in exclusive mode, so set the cluster_database=false if it is RAC(mandatory, else nid will say that the DB needs to be mounted in exclusive mode.

Check the open_cursors parameter and it is safe to be set to a very high value during the operation, this needs to be reverted back to the original value after nid operation.

SOURCES:

http://www.dbapundits.com/blog/oracle-utilities/nid-failure-and-open_cursors/
http://www.dba-oracle.com/t_rename_database_oracle_sid.htm


Tuesday 26 August 2014

ORA-12549: TNS: Operating System Resource Quota Exceeded

I faced this error on a new standby database server running on AIX 6.1. The error is however pretty self-explanatory.

I ensured that ulimit values for the oracle user are appropriate.
# ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        unlimited
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) unlimited
threads(per process) unlimited
processes(per user)  unlimited

Second I checked the current value of maxuproc for all users. It was at default.

# lsattr -El sys0 |grep maxuproc
maxuproc        128              Maximum number of PROCESSES allowed per user      True

128 is the default value, this will be increased.

# chdev -l sys0  -a maxuproc=10240

# lsattr -El sys0 |grep maxuproc
maxuproc        10240             Maximum number of PROCESSES allowed per user      True

I rebooted the LPAR easily as this was not a production server.

Problem solved!