ORA-12516: TNS:listener could not find available handler with matching protocol

I got this error while trying to connect to a database from one of our new application servers.

oerr returned the following:

$ oerr ora 12516
12516, 00000, "TNS:listener could not find available handler with matching protocol stack"
// *Cause: None of the known and available service handlers for the given
// SERVICE_NAME support the client's protocol stack: transport, session,
// and presentation protocols.
// *Action: Check to make sure that the service handlers (e.g. dispatchers)
// for the given SERVICE_NAME are registered with the listener, are accepting
// connections, and that they are properly configured to support the desired
// protocols.


This message did not help as I had ensured that the tnames.ora is properly configured and I was sure of the SERVICE name.

SOLUTION:

 I increased processes parameter to 3000.

alter system set processes=3000 scope=spfile;

The database was restarted. Then the error below showed up!

ORA-00371: not enough shared pool memory, should be atleast 5207647846 bytes

This was fixed by modifying the shared_pool_memory parameter to a little over the suggested value above.

Issued solved!

renaming an Oracle 11.2.0.2 database

To change an Oracle 11.2.0.2 DB name from DB1 to DB2

    STEP 1: Backup the database.
   
    STEP 2: Do a cleab shutdown of the database and mount the database after:

        SQL> SHUTDOWN IMMEDIATE
        SQL> STARTUP MOUNT

    STEP 3: Invoke the DBNEWID utility (nid) specifying the new DBNAME from the command line using a user with SYSDBA privilege:

        nid TARGET=sys/password@DB1 DBNAME=DB2

        # nid TARGET=sys/password@TSH1 DBNAME=DB2
        DBNEWID: Release 11.2.0.2.0 - Production on Wed Aug 27 12:12:52 2014

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

########################
########################
########################
########################
########################
########################
########################

        Database name changed to DB2.
        Modify parameter file and generate a new password file before restarting.
        Database ID for database DB2 changed to 3437107090.
        All previous backups and archived redo logs for this database are unusable.
        Shut down database and open with RESETLOGS option.
        Succesfully changed database name and ID.
        DBNEWID - Completed succesfully.

    STEP 4: Create new pfile from spfile.

    Shutdown the database:

        SHUTDOWN IMMEDIATE

    STEP 5: Modify the DB_NAME parameter in the initialization parameter file to reflect the new DB name .

      
    STEP 6: Create a new password file:

      
    orapwd file=$ORACLE_HOME/dbs/orapwDB2 password=choicepassword


    STEP 7: Rename the SPFILE to match the new DBNAME.
   

    STEP 8: On UNIX/Linux, reset the ORACLE_SID environment variable:
   

        ORACLE_SID=DB2; export ORACLE_SID

    STEP 9: Alter the listener.ora and tnsnames.ora setting to match the new database name and restart the listener:
   

        lsnrctl reload

    STEP 10: Open the database with RESETLOGS:
   

        STARTUP MOUNT
        ALTER DATABASE OPEN RESETLOGS;

    STEP 11: Backup the database.



 ##########################################################################################################################

Issues encountered.



execution of DBNEWID failed initially with the error shown below:

NID-00111: Oracle error reported from target database while executing
    begin       dbms_backup_restore.nidprocessdf(:fno, :istemp, :skipped, :idchged,                                                                          :nmchged);    end;
ORA-01000: maximum open cursors exceeded



Change of database name and database ID failed.
Must finish change or REVERT changes before attempting any database operation.
DBNEWID - Completed with errors.



I tried to reverse my action using

 nid target=/ REVERT=Y

This also failed with the error below:

NID-00111: Oracle error reported from target database while executing
    begin       dbms_backup_restore.nidprocessdf(:fno, :istemp, :skipped, :idchged,                                                                          :nmchged);    end;
ORA-01000: maximum open cursors exceeded



Reversion of database name and database ID failed.
Must finish REVERT changes before attempting any database operation.
DBNEWID - Completed with errors.



Resolution:

I had to correct the error ORA-01000: maximum open cursors exceeded

SYS> show parameter open_cursors

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
open_cursors                         integer     100

SYS> alter system set open_cursors=1024 SCOPE=BOTH;

System altered.

SYS> show parameter open_cursors

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
open_cursors                         integer     1024


The rename ran successfully after correcting this error.

###########################################################################################################

BEST PRACTICES:

Take controlfile backups – binary and trace backups before the bid operation.

If there are tempfiles in the dictionary, make sure they are accessible. Check both v$tempfile and dba_temp_files, not just dba_temp_files. The nid operation tries to update the dbname in tempfiles too and if it is not found, the operation fails. If there are tempfiles that doesn’t exist, it is better to drop them from the dictionary before the nid operation.

nid requires the database to be mounted in exclusive mode, so set the cluster_database=false if it is RAC(mandatory, else nid will say that the DB needs to be mounted in exclusive mode.

Check the open_cursors parameter and it is safe to be set to a very high value during the operation, this needs to be reverted back to the original value after nid operation.

SOURCES:

http://www.dbapundits.com/blog/oracle-utilities/nid-failure-and-open_cursors/
http://www.dba-oracle.com/t_rename_database_oracle_sid.htm

ORA-12549: TNS: Operating System Resource Quota Exceeded

I faced this error on a new standby database server running on AIX 6.1. The error is however pretty self-explanatory.

I ensured that ulimit values for the oracle user are appropriate.
# ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        unlimited
memory(kbytes)       unlimited
coredump(blocks)     unlimited
nofiles(descriptors) unlimited
threads(per process) unlimited
processes(per user)  unlimited

Second I checked the current value of maxuproc for all users. It was at default.

# lsattr -El sys0 |grep maxuproc
maxuproc        128              Maximum number of PROCESSES allowed per user      True

128 is the default value, this will be increased.

# chdev -l sys0  -a maxuproc=10240

# lsattr -El sys0 |grep maxuproc
maxuproc        10240             Maximum number of PROCESSES allowed per user      True

I rebooted the LPAR easily as this was not a production server.

Problem solved!

ORA-1652: unable to extend temp segment by 128 in tablespace TEMP

Hi all,

Been noticing this error in my alert log for a while. Then I got a call from one user complaining of a failing query..this is a scheduled query that runs early in the mornings daily.

Solution to my case:

1. I looked through my alert log to see if I will find any special incidents around the period which these errors were encountered. I didn't find any!

2. I looked at AWR and ADDM reports for the periods around this occurrence. No serious observation except that I found "Undersized PGA". This I did not take seriously.

3. I immediately proceeded to increase the size of my TEMP tablespace by resizing the one of the datafiles. This had no effect whatsoever because the error continued to show up in the alert logs and the query kept failing.

4. I had them schedule the query to a period when there's less transaction on the database (weekend). It still failed. This morning, I came in a looked at my alert logs only to find several incidents of ORA-1652 all through the weekend! This was a period when activity on the DB is not anywhere near optimal.

6. All the while, "Undersized PGA" kept showing up in my ADDM reports.So I decided to increase my pga_aggregate_target. We ran the query again and it worked!

Lesson:

Pay attention to seemingly unimportant alerts especially from your ADDM reports. I must add that all the while that I had "Undersized PGA", it stated that this had less than 2% impact on my peformance.

OUI - 10022: Fixing a corrupt OraInventory

I ran into this error while deploying OEM 12c agent to a remote AIX 6.1 server running Oracle DB 11.2.0.2.0.

The following error came up:

OUI-10022:The target area $ORACLE_BASE/oraInventory cannot be used because it is in an invalid state.. This implies my global oracle inventory is corrupt. First I checked to ensure that I have the appropriate permission (755) on the oraInventory direction. This was in place so permissions could not be the issue.

For AIX/UNIX, the OraInventory can be located by checking /etc/oraInst.loc. Having located the file, the contents looked like this:

inventory_loc=<location_for_oraInventory>
inst_group=<name_of_group_for_installtion> 

I navigated to the inventory location and renamed the existing oraInventory directory.

mv oraInventory oraInventory_orig

Next, I navigated to $ORACLE_HOME/oui/bin where I ran OUI as follows:

 ./runInstaller -silent -ignoreSysPrereqs -attachHome ORACLE_HOME="oracle_home_directory" ORACLE_HOME_NAME="OraDB11g_home1"


Verify the newly created oraInventory using:

$ORACLE_HOME/OPatch/opatch lsinventory -detail

ORA-01516: nonexistent log file, data file, or temporary file .....................

I got this error while trying to resize (increase) one of my datafiles. I was baffled as the message is tell me the file is nonexistent and here I was staring at the file in the directory!

I ensured the said file was online and available, checked my syntax to see if I missed anything or if I had mixed up and upper case for a lower case and vice versa but I found nothing.

Big thanks to Vijay Kumar of http://vijaykumar-dbahub.blogspot.com. His post bailed me out!

I resized describing the datafile by its file#.

SQL> select FILE#,CREATION_TIME,name from v$datafile where name like 'full/path/to/datafile.dbf';

 SQL> alter database datafile (data file number) resize 20000M;

Database altered.

Root Cause Analysis still according to Vijay Kumar.

I created a backup controlfile to trace and then opened it with VI and alas! there was a space right in betweem the .dbf in the datafile name and the ' symbol. i.e.

'/oradata4/ORA/datafiles/ORA_data/XXX_XXX_X_XXXX.dbf  '

instead of

'/oradata4/ORA/datafiles/ORA_data/XXX_XXX_X_XXXX.dbf '


This was rectified by renaming the datafile.

Running SQL Access Advisor from SQL Plus

Following one of the recommendations from running ADDM on one of my production databases, I decided to run SQL Access Advisor rather than just creating a couple of Indexes suggested by Oracle.

My problem, EM is not available on that database server! I needed to find a way to run SQL Access Advisor without going through EM.

Thankfully, I have some other databases with EM running and knowing that Oracle put this really nice feature in EM where you get the chance to see the exact SQL statements that are being executed against your database. I quickly connected to one of my test databases and I proceeded to Advisor Central > SQL Advisors > SQL Access Advisor.


PS: I decided to run an comprehensive analysis with SQL AA.

I followed the steps until I got the chance to click the SHOW SQL button. The output is below:

DECLARE

taskname varchar2(30) := 'desired_task_name';
task_desc varchar2(256) := 'SQL Access Advisor';
task_or_template varchar2(30) := 'SQLACCESS_EMTASK';
task_id number := 0;
num_found number;
sts_name varchar2(256) := 'desired_task_name_sts';
sts_cursor dbms_sqltune.sqlset_cursor;

BEGIN
/* Create Task */
dbms_advisor.create_task(DBMS_ADVISOR.SQLACCESS_ADVISOR,task_id,taskname,task_desc,task_or_template);
/* Reset Task */
dbms_advisor.reset_task(taskname);
/* Delete Previous STS Workload Task Link */
select count(*) into num_found from user_advisor_sqla_wk_map where task_name = taskname and workload_name = sts_name;
IF num_found > 0 THEN
dbms_advisor.delete_sqlwkld_ref(taskname,sts_name,1);
END IF;
/* Delete Previous STS */
select count(*) into num_found from user_advisor_sqlw_sum where workload_name = sts_name;
IF num_found > 0 THEN
dbms_sqltune.delete_sqlset(sts_name);
END IF;
/* Create STS */
dbms_sqltune.create_sqlset(sts_name, 'Obtain workload from cursor cache');
/* Select all statements in the cursor cache. */
OPEN sts_cursor FOR
SELECT VALUE(P)
FROM TABLE(dbms_sqltune.select_cursor_cache) P;
/* Load the statements into STS. */
dbms_sqltune.load_sqlset(sts_name, sts_cursor);
CLOSE sts_cursor;
/* Link STS Workload to Task */
dbms_advisor.add_sqlwkld_ref(taskname,sts_name,1);
/* Set STS Workload Parameters */
dbms_advisor.set_task_parameter(taskname,'VALID_ACTION_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'VALID_MODULE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'SQL_LIMIT',DBMS_ADVISOR.ADVISOR_UNLIMITED);
dbms_advisor.set_task_parameter(taskname,'VALID_USERNAME_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'VALID_TABLE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'INVALID_TABLE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'INVALID_ACTION_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'INVALID_USERNAME_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'INVALID_MODULE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'VALID_SQLSTRING_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'INVALID_SQLSTRING_LIST','"@!"');
/* Set Task Parameters */
dbms_advisor.set_task_parameter(taskname,'ANALYSIS_SCOPE','ALL');
dbms_advisor.set_task_parameter(taskname,'RANKING_MEASURE','PRIORITY,OPTIMIZER_COST');
dbms_advisor.set_task_parameter(taskname,'DEF_PARTITION_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'TIME_LIMIT',10000);
dbms_advisor.set_task_parameter(taskname,'MODE','COMPREHENSIVE');
dbms_advisor.set_task_parameter(taskname,'STORAGE_CHANGE',DBMS_ADVISOR.ADVISOR_UNLIMITED);
dbms_advisor.set_task_parameter(taskname,'DML_VOLATILITY','TRUE');
dbms_advisor.set_task_parameter(taskname,'WORKLOAD_SCOPE','PARTIAL');
dbms_advisor.set_task_parameter(taskname,'DEF_INDEX_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'DEF_INDEX_OWNER',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'DEF_MVIEW_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'DEF_MVIEW_OWNER',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'DEF_MVLOG_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);
dbms_advisor.set_task_parameter(taskname,'CREATION_COST','TRUE');
dbms_advisor.set_task_parameter(taskname,'JOURNALING','4');
dbms_advisor.set_task_parameter(taskname,'DAYS_TO_EXPIRE','30');
/* Execute Task */
dbms_advisor.execute_task(taskname);
END;
/




To read the output:
#################################################################
SET LONG 100000
SET PAGESIZE 50000
SELECT DBMS_ADVISOR.get_task_script('desired_task_name') AS script
FROM   dual;
SET PAGESIZE 24