Wednesday, August 6, 2014

Unable to create Patch Object Exception occured : PatchObject constructor: Input file does not exist.

Version: 

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

Issue:

Error found on invoking patch conflict check using opatch for a PSU patch on database Home.

Error:

Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail"not executed 
Unable to create Patch Object.
Exception occured : PatchObject constructor: Input file "/u01/stage/etc/config/actions" or "/u01/stage/etc/config/inventory" does not exist.

OPatch succeeded.

Reason:

Error is due to the location (directory) given for the Patch folder given using -ph.

I used the command, "opatch prereq CheckConflictAgainstOHWithDetail -ph ./". Since I gave -ph . the command should have been executed from the directory where the patch was extracted.

In this case the patch folder after extraction was /u01/stage/18522509. 

But I invoked the patch conflict check from /u01/stage and specified -ph ./ where it will look for patch files in the current directory and resulted in the error.

Fix:

Move to the folder created after extracting the patch zip file and execute the command again from there.

[oracle@DBServer1 stage]$ cd  /u01/stage/18522509
[oracle@DBServer1 18522509]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./

or specify the full directory to -ph command from current directory

opatch prereq CheckConflictAgainstOHWithDetail -ph /u01/stage/18522509

Tuesday, August 5, 2014

ORA-20001: Latest xml inventory is not loaded into table

Version:

Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production

Issue:

April 2014 PSU - Patch 18522516 - 12.1.0.1.4 Patch Set Update is applied successfully to the DB Home. On the post patching step, to loadi modified SQL files into the database, datapatch is invoked and it results in the error.

Error:

bash-4.1$ ./datapatch -verbose
SQL Patching tool version 12.1.0.1.0 on Tue Aug  5 01:08:49 2014
Copyright (c) 2014, Oracle.  All rights reserved.

Connecting to database...OK
Determining current state...
Currently installed SQL Patches: 
DBD::Oracle::st execute failed: ORA-20001: Latest xml inventory is not loaded into table
ORA-06512: at "SYS.DBMS_QOPATCH", line 1011
ORA-06512: at line 4 (DBD ERROR: OCIStmtExecute) [for Statement "DECLARE
       x XMLType;
     BEGIN
       x := dbms_qopatch.get_pending_activity;
       ? := x.getStringVal();
     END;" with ParamValues: :p1=undef] at /u01/app/product/12.1.0/dbhome_1/sqlpatch/sqlpatch.pm line 1227.

Description:

An error lookup just shows that latest inventory file is not loaded. It could be due to some kind of permission issues with the QOpatch directory and files under it. Database uses two directories to pint to $ORACLE_HOME/QOPatch location, OPATCH_SCRIPT_DIR and OPATCH_LOG_DIR.

Solution:

Followed the Oracle support Doc (Doc ID 1602089.1) and tried workarounds mentioned there, like recreating OPATCH_SCRIPT_DIR and OPATCH_LOG_DIR from sqlplus. Also edited opatch script file as mentioned in the support doc. But both steps did not work.

1. Check the _disable_directory_link_check parameter value in the DB 
2. If its value is FALSE, set it to TRUE
3. Bounce the DB
4. Rinvoke datapatch command.

1. SQL> SELECT a.ksppinm "Parameter",b.ksppstvl "Session Value",c.ksppstvl "Instance Value" FROM x$ksppi a, x$ksppcv b, x$ksppsv c  WHERE a.indx = b.indx AND a.indx = c.indx AND a.ksppinm LIKE '/_disable_direc%' escape '/';   

Parameter                      Session Value Instance Value
------------------------------ ------------- ---------------
_disable_directory_link_check   FALSE         FALSE

2. SQL> alter system set "_disable_directory_link_check"=TRUE scope=spfile;

System altered.

3. SQL> Shutdown immediate
    SQL> startup;

4. ./datapatch -verbose
.....................
.....................
     Installing patches...
     Patch installation complete.  Total patches installed: 1
     Validating logfiles...
     Patch 18522516 apply: SUCCESS

Monday, August 4, 2014

The OPatch version is not applicable for current OUI version

Version:

Oracle WLS 12.1.2.0

Issue:

Opatch lsinventory returns error when opatch command is issued from 12.1.2.0 version of Weblogic. 

Error:

opatch lsinventory -jdk $JAVA_HOME

OPatch Version : 13.1.0.0.0
OUI  Version : 12.1.0.1.0
The OPatch version is not applicable for current OUI version.

Since OUI Version starts with 12.1, Please go to 'My Oracle Support' and get right OPatch 12.1.*.*.* based on patch 6880880, release version 12.1.0.0.0 and \appropriate platform.

OPatch failed with error code 19

Reason:

The Error was due to the mismatch in opatch version and the inventory version. A DB software of 12C version is also available in same server. In my case  MW_HOME environment variable is set and PATH is set to $MW_HOME/OPatch directory. But the ORACLE_HOME environment variable was still set to the DB Home. Because of that the opatch utility checked the inventory file which it got from the Oracle DB Home oraInst.loc and error out

Solution:

Set ORACLE_HOME to MW_HOME_PATH location and re invoke opatch. 
Still better to close the previous Unix session and open a new one to avoid reference to any variables set previously.

After the correct ORACLE_HOME setting the versions read as follows

OPatch version    : 13.1.0.0.0

OUI version       : 13.1.0.0.0

Note: The error can come also if the opatch version is old compared to the inventory version

Friday, August 1, 2014

ORA-39142: incompatible version number 4.1 in dump file with IMPDP

Scenario:

Import to a database using IMPDP is erroring out

Versions:

Source DB 12.1.0.1.0
Target DB 11.2.0.4.0

Error:

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORA-39001: invalid argument value
ORA-39000: bad dump file specification
ORA-39142: incompatible version number 4.1 in dump file "/u01/stage/schema_dump.dmp"

Reason:

As the error message rightly points out it is due to the incompatibility of EXPDP/IMPDP versions used.
Here the export was done using 12.1.0.1.0 EXPDP utility and dump created was tried to be imported using 11.2.0.4.0 IMPDP utility, resulting in the error.

Solution:

Use the "VERSION" option available with datapump.

The export data pump has the "VERSION" option which allows to create the dump file which is compatible with the previous releases.

Here is the syntax and description of VERSION from Oracle’s document.

VERSION={COMPATIBLE | LATEST | version_string}

The legal values for the VERSION parameter are as follows:

COMPATIBLE – This is the default value. The version of the metadata corresponds to the database compatibility level. 
Database compatibility must be set to 9.2 or higher.
LATEST – The version of the metadata corresponds to the database release.
version_string – A specific database release (for example, 11.2.0). In Oracle Database 11g, this value cannot be lower than 9.2.

Eg:

expdp system directory=EXPDP_DIR dumpfile=schema.dmp log=schema_expdp.log SCHEMAS=scott version=11.2.0.0.0

Monday, July 21, 2014

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr] in Oracle 12C

Version:

DB: 12.1.0.1.0
OS: OEL 6, 64 bit

Issue:

The file system on which DB files were available crashed, resulting in a DB crash. The next DB startup is failing with Internal errors. DB is coming up to the mount stage only.

Error:

Database mounted.
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1],
[1895], [12597], [12603], [], [], [], [], [], [], []

Reason:

Because of the file system crash when DB was running,the data files became in accessible. This resulted in a "lost Write" into the Online RedoLogs. So the normal Instance Recovery procedure became impossible to happen causing the ORA-600 error.

Solution:

We can manually recover the DB by specifying the online redo thread at the time of crash if the full set of redo logs are available in the storage. 

Note: It is better to have a full backup of all the files before trying the steps.

1. Backup all DB files if possible

2. Identify the current log sequence group and file.

SQL> startup mount;

SQL> select GROUP#,STATUS from v$log;

    GROUP# STATUS
---------- ----------------
         1 INACTIVE
         3 INACTIVE
         2 CURRENT

SQL> select GROUP#,MEMBER from v$logfile;

    GROUP# MEMBER
---------  -----------------------
1   /u01/oradata/orcl/redo01.log  
2   /u01/oradata/orcl/redo02.log
3   /u01/oradata/orcl/redo03.log


Here Group# 2 is current. So I am going to pass it manually

3. Manually Recover the DB

SQL> recover database until cancel using backup controlfile;
ORA-00279: change 23977161 generated at 07/18/2014 07:00:59 needed for thread 1
ORA-00289: suggestion :
/u01/app/oracle/product/121010/dbs/arch1_1895_847740493.dbf
ORA-00280: change 23977161 for thread 1 is in sequence #1895


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
/u01/oradata/orcl/redo02.log
Log applied.
Media recovery complete.
SQL> alter database open resetlogs;

Database altered.

DB is now open.

Note: If the above step fails then the only option left is to restore/recover from latest backup.

Thursday, July 17, 2014

ORA-04036: PGA memory used by the instance exceeds PGA_AGGREGATE_LIMIT

Version:

DB: 12.1.0.1.0 EE

Issue:

Errors are obtained while running migration script in a 12C DB and the sessions are getting killed.

ORA-00028: your session has been killed
ORA-04036: PGA memory used by the instance exceeds PGA_AGGREGATE_LIMIT
PGA memory used by the instance exceeds PGA_AGGREGATE_LIMIT of 2048 MB

Reason:


From Oracle 12C, a new parameter called PGA_AGGREGATE_LIMIT sets a hard limit on the amount of PGA taken up by an instance.

The parameter pga_aggregate_limit, if not explicitly defined, defaults to the greater of these three numbers:
a) 2 GB
b) 3 MB times the value of the processes parameter
c) 2 times the value of the parameter pga_aggregate_target

SQL> show parameter pga_ag

NAME                                            TYPE                        VALUE
------------------------------------ -----------       ------------------------------
pga_aggregate_limit                       big integer                       2G
pga_aggregate_target                     big integer                       764M

When the overall PGA occupied by all the sessions in that instance exceed the limit, Oracle kills the session holding the most untunable PGA memory, releasing all PGA memory held by that session.

The error can be noticed in the alert log file and the incident would be reported in a trace file, where the current offending SQL can be found.

ORA-04036: PGA memory used by the instance exceeds PGA_AGGREGATE_LIMIT
Incident details in: /u01/app/diag/rdbms/orcl/orcl/incident/incdir_24367/orcl_ora_13239_i24367.trc

In trace file

----- Current SQL Statement for this session (sql_id=2fsstt9a4tqz7) -----


Solution:

One way is to go back to the previous (pre-12c) behavior. To do that set the value of pga_aggregate_limit to 0 and the parameter will not have any impact.

alter system set pga_aggregate_limit=0 scope=both;

Obviously another option is to set this value to a much higher value if you have enough physical memory in the server

alter system set pga_aggregate_limit=3072M scope=both;

Tuesday, July 1, 2014

Could not connect to asm due to following error ora-01031: insufficient privilege on RAC DB creation

Version: 

11.2.0.3.0 in RHEL 6

Scenario: 

A RAC DB creation on two nodes using DBCA.
Chose ASM as storage type in the Storage Type details screen of DBCA. But the disk groups those were created already were not visible.

Tried giving +DATA manually and clicked Next, and ended up getting the following error.


"Could not connect to asm due to following error ora-01031: insufficient privilege"

Verified from ASMCMD > lsdg command that all the disk groups were mounted.

Reason:

This was clearly a case of missing privilege as indicated by the error

Grid software owner was "grid" and DB S/W owner was "oracle". I missed to add "asmdba" as a secondary group to oracle user. 

[root@node1 app]# id oracle
uid=1901(oracle) gid=54321(oinstall) groups=8500(dba),8501(oper),54321(oinstall)


Solution:

Added asmdba as a secondary group to oracle. 

/usr/sbin/usermod -g oinstall -G dba,oper,asmdba oracle

[root@node1 app]# id oracle
uid=1901(oracle) gid=54321(oinstall) groups=8500(dba),8501(oper),9601(asmdba),54321(oinstall)

Took a new terminal session as "oracle" and re initiated DBCA.

This time the disk groups were displayed and chosen by default.