From oracle 12c onward Bundle patches/PSU patch can be applied to oracle GRID or database home using opatchauto utility which simplifies the patching process.
NOTE – > WHAT IS PROACTIVE BUNDLE PATCH
However recently i faced an issue where due to opatch utility issue, we are unable to use optachauto utility . So we went ahead with the manual patching method. In the below article we will explain all the steps in detail.
ENVIROMENT:
Note – > The patching will be done in rolling fashion. I.e first patch will be applied on node1 , during which node2 will be running fine. and once node1 patching is over and its services are up, we can proceed with patching on node 2.
In this below demo, We have shown the steps for first node ( dbhost-n1).
1. Download the latest bundle patch from oracle support.
This patchset comes with below patches:
2. Copy the patch to both the nodes and unzip there.
cd /dumparea/APR2020BP unzip p30805478_121020_SOLARIS64.zip
3. Check the recommended opatch version for both db_home and grid_home
For the recommended opatch version, always check the readme file.
A per readme file 12.2.0.19 or later is recommended. Lets check in our environment.
cd $GRID_HOME /gridapp/app/oracle/product/grid12c/OPatch# ./opatch version OPatch Version: 12.2.0.1.21 OPatch succeeded.
In our case the the opatch version(12.2.0.21) is latest. So no need to upgrade the opatch utility.
If your opatch version is lower than the recommended, then follow the below link to upgrade your opatch version:
LINK – > How to install/upgrade OPatch in RAC
4. Check for patch conflicts with GRID_HOME:
cd $GRID_HOME/OPatch
./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30691015
./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30703508
./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30703511
./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/26983807
cd /gridapp/app/oracle/product/grid12c/OPatch oracle@dbhost-n1:.$ ./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30691015 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-16_15-42-45PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded. oracle@dbhost-n1:$ ./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30703508 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-16_15-42-56PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded. oracle@dbhost-n1:...$ ./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30703511 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-16_15-43-04PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded. oracle@dbhost-n1:.$ ./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/26983807 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-16_15-43-12PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded.
If any conflict is reported, then those need to be taken care.
4. Check for patch conflicts with DB_HOME:
./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30691015 ./opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /dumparea/APR2020BP/30805478/30703508
If any conflict is reported, then those need to be taken care.
5. Check whether sufficient space available for patching in GRID_HOME
% cat /tmp/patch_list_gihome.txt /dumparea/APR2020BP/30805478/26983807 /dumparea/APR2020BP/30805478/30703511 /dumparea/APR2020BP/30805478/30703508 /dumparea/APR2020BP/30805478/30691015 -- space checker: -------------------------- oracle@dbhost-n1:.../oracle/product/grid12c/OPatch$ ./opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-16_15-44-55PM_1.log Invoking prereq "checksystemspace" Prereq "checkSystemSpace" passed. OPatch succeeded.
5. Check whether sufficient space available for patching in DB_HOME:
$cat /tmp/patch_list_dbhome.txt /dumparea/APR2020BP/30805478/30703508 /dumparea/APR2020BP/30805478/30691015 $ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /oracle/app/oracle/product/12.1.0.2/dbhome_1 Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /oracle/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2020-06-16_15-44-55PM_1.log Invoking prereq "checksystemspace" Prereq "checkSystemSpace" passed. OPatch succeeded.
6. shutdown the services running on db_home:
Shutdown database instance and listener services running on this node1
export ORACLE_HOME=/oracle/app/oracle/product/12.1.0.2/dbhome_1 export PATH=$ORACLE_HOME/bin:$PATH srvctl stop home -o /oracle/app/oracle/product/12.1.0.2/dbhome_1 -s /dumparea/db_prepatch_stat.txt -n dbhost-n1
7. Prepare the CRS for patching:
This script need to be executed from root user.
root#/gridapp/app/oracle/product/grid12c/crs/install/rootcrs.sh -prepatch
root@dbhost-n1:~# /gridapp/app/oracle/product/grid12c/crs/install/rootcrs.sh -prepatch Using configuration parameter file: /gridapp/app/oracle/product/grid12c/crs/install/crsconfig_params Oracle Clusterware active version on the cluster is [12.1.0.2.0]. The cluster upgrade state is [ROLLING PATCH]. The cluster active patch level is [4267482287]. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.crsd' on 'dbhost-n1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.LISTENER_TEST.lsnr' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.REDOA.dg' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.OCRVD.dg' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.MGMT.dg' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.cvu' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.REDOB.dg' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.oc4j' on 'dbhost-n1' CRS-2677: Stop of 'ora.cvu' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.cvu' on 'dbhost-n2' CRS-2677: Stop of 'ora.OCRVD.dg' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.MGMT.dg' on 'dbhost-n1' succeeded CRS-2676: Start of 'ora.cvu' on 'dbhost-n2' succeeded CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.scan2.vip' on 'dbhost-n1' CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.scan3.vip' on 'dbhost-n1' CRS-2677: Stop of 'ora.LISTENER_TEST.lsnr' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.LISTENER_GSM.lsnr' on 'dbhost-n1' CRS-2677: Stop of 'ora.DATA.dg' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.LISTENER_GSM.lsnr' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.dbhost-n1.vip' on 'dbhost-n1' CRS-2677: Stop of 'ora.scan2.vip' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.scan2.vip' on 'dbhost-n2' CRS-2677: Stop of 'ora.REDOB.dg' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.REDOA.dg' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.ARCH.dg' on 'dbhost-n1' CRS-2677: Stop of 'ora.scan3.vip' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.scan3.vip' on 'dbhost-n2' CRS-2677: Stop of 'ora.ARCH.dg' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'dbhost-n1' CRS-2677: Stop of 'ora.asm' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.dbhost-n1.vip' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.dbhost-n1.vip' on 'dbhost-n2' CRS-2677: Stop of 'ora.oc4j' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.oc4j' on 'dbhost-n2' CRS-2676: Start of 'ora.dbhost-n1.vip' on 'dbhost-n2' succeeded CRS-2676: Start of 'ora.scan2.vip' on 'dbhost-n2' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'dbhost-n2' CRS-2676: Start of 'ora.scan3.vip' on 'dbhost-n2' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'dbhost-n2' CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'dbhost-n2' succeeded CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'dbhost-n2' succeeded CRS-2676: Start of 'ora.oc4j' on 'dbhost-n2' succeeded CRS-2673: Attempting to stop 'ora.ons' on 'dbhost-n1' CRS-2677: Stop of 'ora.ons' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'dbhost-n1' CRS-2677: Stop of 'ora.net1.network' on 'dbhost-n1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'dbhost-n1' has completed CRS-2677: Stop of 'ora.crsd' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.storage' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.evmd' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dbhost-n1' CRS-2677: Stop of 'ora.storage' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'dbhost-n1' CRS-2677: Stop of 'ora.drivers.acfs' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.evmd' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.asm' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'dbhost-n1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.ctssd' on 'dbhost-n1' CRS-2677: Stop of 'ora.crf' on 'dbhost-n1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'dbhost-n1' CRS-2677: Stop of 'ora.cssd' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'dbhost-n1' CRS-2677: Stop of 'ora.gipcd' on 'dbhost-n1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'dbhost-n1' has completed CRS-4133: Oracle High Availability Services has been stopped. 2020/06/17 14:10:54 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector. 2020/06/17 14:11:44 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector. 2020/06/17 14:11:46 CLSRSC-347: Successfully unlock /gridapp/app/oracle/product/grid12c
8. Apply the patches to grid home:
oracle@dbhost-n1:.../oracle/product/grid12c/OPatch$ ./opatch apply -oh /gridapp/app/oracle/product/grid12c -local /dumparea/APR2020BP/30805478/26983807 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-12-53PM_1.log Verifying environment and performing prerequisite checks... The following patch(es) are duplicate patches with patches installed in the Oracle Home. [ 26983807] You have already installed same patch(es) with same UPI(s) or same version(s). These patch(es) will be skipped. OPatch Session completed with warnings. Log file location: /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-12-53PM_1.log OPatch completed with warnings.
oracle@dbhost-n1:.../oracle/product/grid12c/OPatch$ ./opatch apply -oh /gridapp/app/oracle/product/grid12c -local /dumparea/APR2020BP/30805478/30703511 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-13-14PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 30703511 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/gridapp/app/oracle/product/grid12c') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '30703511' to OH '/gridapp/app/oracle/product/grid12c' Patching component oracle.usm, 12.1.0.2.0... Patch 30703511 successfully applied. Sub-set patch [29423125] has become inactive due to the application of a super-set patch [30703511]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-13-14PM_1.log OPatch succeeded.
oracle@dbhost-n1:.../oracle/product/grid12c/OPatch$ ./opatch apply -oh /gridapp/app/oracle/product/grid12c -local /dumparea/APR2020BP/30805478/30703508 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-13-44PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 30703508 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/gridapp/app/oracle/product/grid12c') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '30703508' to OH '/gridapp/app/oracle/product/grid12c' Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.has.crs, 12.1.0.2.0... Patching component oracle.has.db, 12.1.0.2.0... Patching component oracle.has.common, 12.1.0.2.0... Patching component oracle.has.deconfig, 12.1.0.2.0... Patching component oracle.has.common.cvu, 12.1.0.2.0... Patching component oracle.has.cvu, 12.1.0.2.0... Patching component oracle.has.rsf, 12.1.0.2.0... Patch 30703508 successfully applied. Sub-set patch [29509318] has become inactive due to the application of a super-set patch [30703508]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-13-44PM_1.log OPatch succeeded.
oracle@dbhost-n1:.../oracle/product/grid12c/OPatch$ ./opatch apply -oh /gridapp/app/oracle/product/grid12c -local /dumparea/APR2020BP/30805478/30691015 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /gridapp/app/oracle/product/grid12c Central Inventory : /gridapp/app/oraInventory from : /gridapp/app/oracle/product/grid12c/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-15-56PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 29972716 30364137 30691015 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/gridapp/app/oracle/product/grid12c') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying sub-patch '29972716' to OH '/gridapp/app/oracle/product/grid12c' ApplySession: Optional component(s) [ oracle.ctx, 12.1.0.2.0 ] , [ oracle.rdbms.lbac, 12.1.0.2.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.ldap.rsf, 12.1.0.2.0... Patching component oracle.network.rsf, 12.1.0.2.0... Patching component oracle.rdbms.rsf, 12.1.0.2.0... Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.rdbms.deconfig, 12.1.0.2.0... Patching component oracle.has.crs, 12.1.0.2.0... Patching component oracle.rdbms.rman, 12.1.0.2.0... Patching component oracle.rdbms.dbscripts, 12.1.0.2.0... Patching component oracle.rdbms.util, 12.1.0.2.0... Patching component oracle.rdbms.crs, 12.1.0.2.0... Patching component oracle.ldap.rsf.ic, 12.1.0.2.0... Applying sub-patch '30364137' to OH '/gridapp/app/oracle/product/grid12c' ApplySession: Optional component(s) [ oracle.rdbms.oci, 12.1.0.2.0 ] , [ oracle.rdbms.drdaas, 12.1.0.2.0 ] , [ oracle.precomp.lang, 12.1.0.2.0 ] , [ oracle.precomp.common, 12.1.0.2.0 ] , [ oracle.rdbms.tg4tera, 12.1.0.2.0 ] , [ oracle.rdbms.tg4sybs, 12.1.0.2.0 ] , [ oracle.rdbms.tg4ifmx, 12.1.0.2.0 ] , [ oracle.rdbms.tg4db2, 12.1.0.2.0 ] , [ oracle.rdbms.tg4msql, 12.1.0.2.0 ] , [ oracle.rdbms.hsodbc, 12.1.0.2.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.ldap.rsf, 12.1.0.2.0... Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.rdbms.rsf, 12.1.0.2.0... Patching component oracle.rdbms.util, 12.1.0.2.0... Patching component oracle.ldap.rsf.ic, 12.1.0.2.0... Patching component oracle.rdbms.dbscripts, 12.1.0.2.0... Patching component oracle.rdbms.rman, 12.1.0.2.0... Patching component oracle.rdbms.crs, 12.1.0.2.0... Patching component oracle.has.crs, 12.1.0.2.0... Applying sub-patch '30691015' to OH '/gridapp/app/oracle/product/grid12c' ApplySession: Optional component(s) [ oracle.rdbms.dv, 12.1.0.2.0 ] , [ oracle.xdk, 12.1.0.2.0 ] , [ oracle.oraolap, 12.1.0.2.0 ] , [ oracle.precomp.lang, 12.1.0.2.0 ] , [ oracle.precomp.common, 12.1.0.2.0 ] , [ oracle.ctx, 12.1.0.2.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.network.rsf, 12.1.0.2.0... Patching component oracle.rdbms.rsf, 12.1.0.2.0... Patching component oracle.ldap.rsf, 12.1.0.2.0... Patching component oracle.tfa, 12.1.0.2.0... Patching component oracle.rdbms.rman, 12.1.0.2.0... Patching component oracle.nlsrtl.rsf, 12.1.0.2.0... Patching component oracle.ldap.rsf.ic, 12.1.0.2.0... Patching component oracle.ctx.rsf, 12.1.0.2.0... Patching component oracle.assistants.server, 12.1.0.2.0... Patching component oracle.xdk.parser.java, 12.1.0.2.0... Patching component oracle.xdk.rsf, 12.1.0.2.0... Patching component oracle.rdbms.dbscripts, 12.1.0.2.0... Patching component oracle.swd.oui, 12.1.0.2.0... Patching component oracle.ordim.server, 12.1.0.2.0... Composite patch 30691015 successfully applied. Log file location: /gridapp/app/oracle/product/grid12c/cfgtoollogs/opatch/opatch2020-06-17_14-15-56PM_1.log OPatch succeeded.
9. Run prepatch.sh for DB_HOME:
This script is available under OCW patch directory.(30703508 patch)
oracle@dbhost-n1h$ /dumparea/APR2020BP/30805478/30703508/custom/scripts/prepatch.sh -dbhome /oracle/app/oracle/product/12.1.0.2/dbhome_1 /dumparea/APR2020BP/30805478/30703508/custom/scripts/prepatch.sh completed successfully.
10. Apply the patches to db_home:
oracle@dbhost-n1:...oduct/12.1.0.2/dbhome_1/OPatch$ ./opatch apply -oh /oracle/app/oracle/product/12.1.0.2/dbhome_1 -local /dumparea/APR2020BP/30805478/30691015 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /oracle/app/oracle/product/12.1.0.2/dbhome_1 Central Inventory : /gridapp/app/oraInventory from : /oracle/app/oracle/product/12.1.0.2/dbhome_1/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /oracle/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2020-06-17_14-35-53PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 29972716 30364137 30691015 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/oracle/app/oracle/product/12.1.0.2/dbhome_1') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying sub-patch '29972716' to OH '/oracle/app/oracle/product/12.1.0.2/dbhome_1' ApplySession: Optional component(s) [ oracle.has.crs, 12.1.0.2.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.ldap.rsf, 12.1.0.2.0... Patching component oracle.network.rsf, 12.1.0.2.0... Patching component oracle.rdbms.rsf, 12.1.0.2.0... Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.ctx, 12.1.0.2.0... Patching component oracle.rdbms.deconfig, 12.1.0.2.0... Patching component oracle.rdbms.rman, 12.1.0.2.0... Patching component oracle.rdbms.dbscripts, 12.1.0.2.0... Patching component oracle.rdbms.util, 12.1.0.2.0... Patching component oracle.rdbms.lbac, 12.1.0.2.0... Patching component oracle.rdbms.crs, 12.1.0.2.0... Patching component oracle.ldap.rsf.ic, 12.1.0.2.0... Applying sub-patch '30364137' to OH '/oracle/app/oracle/product/12.1.0.2/dbhome_1' ApplySession: Optional component(s) [ oracle.rdbms.drdaas, 12.1.0.2.0 ] , [ oracle.has.crs, 12.1.0.2.0 ] , [ oracle.rdbms.tg4tera, 12.1.0.2.0 ] , [ oracle.rdbms.tg4sybs, 12.1.0.2.0 ] , [ oracle.rdbms.tg4ifmx, 12.1.0.2.0 ] , [ oracle.rdbms.tg4db2, 12.1.0.2.0 ] , [ oracle.rdbms.tg4msql, 12.1.0.2.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.ldap.rsf, 12.1.0.2.0... Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.rdbms.rsf, 12.1.0.2.0... Patching component oracle.rdbms.util, 12.1.0.2.0... Patching component oracle.ldap.rsf.ic, 12.1.0.2.0... Patching component oracle.rdbms.dbscripts, 12.1.0.2.0... Patching component oracle.rdbms.rman, 12.1.0.2.0... Patching component oracle.rdbms.oci, 12.1.0.2.0... Patching component oracle.rdbms.crs, 12.1.0.2.0... Patching component oracle.precomp.lang, 12.1.0.2.0... Patching component oracle.precomp.common, 12.1.0.2.0... Patching component oracle.rdbms.hsodbc, 12.1.0.2.0... Applying sub-patch '30691015' to OH '/oracle/app/oracle/product/12.1.0.2/dbhome_1' Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.network.rsf, 12.1.0.2.0... Patching component oracle.rdbms.rsf, 12.1.0.2.0... Patching component oracle.ldap.rsf, 12.1.0.2.0... Patching component oracle.tfa, 12.1.0.2.0... Patching component oracle.rdbms.rman, 12.1.0.2.0... Patching component oracle.nlsrtl.rsf, 12.1.0.2.0... Patching component oracle.ldap.rsf.ic, 12.1.0.2.0... Patching component oracle.ctx.rsf, 12.1.0.2.0... Patching component oracle.rdbms.dv, 12.1.0.2.0... Patching component oracle.xdk, 12.1.0.2.0... Patching component oracle.assistants.server, 12.1.0.2.0... Patching component oracle.xdk.parser.java, 12.1.0.2.0... Patching component oracle.oraolap, 12.1.0.2.0... Patching component oracle.xdk.rsf, 12.1.0.2.0... Patching component oracle.rdbms.dbscripts, 12.1.0.2.0... Patching component oracle.precomp.lang, 12.1.0.2.0... Patching component oracle.precomp.common, 12.1.0.2.0... Patching component oracle.swd.oui, 12.1.0.2.0... Patching component oracle.ordim.server, 12.1.0.2.0... Patching component oracle.ctx, 12.1.0.2.0... Composite patch 30691015 successfully applied. Sub-set patch [21463894] has become inactive due to the application of a super-set patch [30691015]. Sub-set patch [18747342] has become inactive due to the application of a super-set patch [30691015]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /oracle/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2020-06-17_14-35-53PM_1.log OPatch succeeded.
oracle@dbhost-n1:...oduct/12.1.0.2/dbhome_1/OPatch$ ./opatch apply -oh /oracle/app/oracle/product/12.1.0.2/dbhome_1 -local /dumparea/APR2020BP/30805478/30703508 Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /oracle/app/oracle/product/12.1.0.2/dbhome_1 Central Inventory : /gridapp/app/oraInventory from : /oracle/app/oracle/product/12.1.0.2/dbhome_1/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.1.0.2.0 Log file location : /oracle/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2020-06-17_14-39-09PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 30703508 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/oracle/app/oracle/product/12.1.0.2/dbhome_1') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '30703508' to OH '/oracle/app/oracle/product/12.1.0.2/dbhome_1' ApplySession: Optional component(s) [ oracle.has.crs, 12.1.0.2.0 ] , [ oracle.has.cvu, 12.1.0.2.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.rdbms, 12.1.0.2.0... Patching component oracle.has.db, 12.1.0.2.0... Patching component oracle.has.common, 12.1.0.2.0... Patching component oracle.has.deconfig, 12.1.0.2.0... Patching component oracle.has.common.cvu, 12.1.0.2.0... Patching component oracle.has.rsf, 12.1.0.2.0... Patch 30703508 successfully applied. Sub-set patch [29509318] has become inactive due to the application of a super-set patch [30703508]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /oracle/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2020-06-17_14-39-09PM_1.log OPatch succeeded.
11. Run the postpatch.sh script
oracle@dbhost-n1:...oduct/12.1.0.2/dbhome_1/OPatch$ /dumparea/APR2020BP/30805478/30703508/custom/scripts/postpatch.sh -dbhome /oracle/app/oracle/product/12.1.0.2/dbhome_1 Reading /oracle/app/oracle/product/12.1.0.2/dbhome_1/install/params.ora.. Reading /oracle/app/oracle/product/12.1.0.2/dbhome_1/install/params.ora.. Found uninstantiated /oracle/app/oracle/product/12.1.0.2/dbhome_1/crs/sbs/srvctl.sbs file, copying to /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/srvctl Parsing file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/srvctl Parsing file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/srvconfig Parsing file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/cluvfy Parsing file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/kfod Verifying file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/racgwrap Skipping the missing file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/racgwrap Verifying file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/srvctl Verifying file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/srvconfig Verifying file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/cluvfy Verifying file /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/kfod Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/srvctl Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/srvconfig Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/cluvfy Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/kfod Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/diskmon.bin Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/lsnodes Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/osdbagrp Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/rawutl Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/srvm/admin/ractrans Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/srvm/admin/getcrshome Reapplying file permissions on /oracle/app/oracle/product/12.1.0.2/dbhome_1/bin/crsdiag.pl Postpatch completed successfully
12. Run the rootadd_rdbms.sh script:( as root user)
root@dbhost-n1:~# /gridapp/app/oracle/product/grid12c/rdbms/install/rootadd_rdbms.sh
13. Run rootcrs.sh script post patch:
root@dbhost-n1:~# /gridapp/app/oracle/product/grid12c/crs/install/rootcrs.sh -postpatch Using configuration parameter file: /gridapp/app/oracle/product/grid12c/crs/install/crsconfig_params 2020/06/17 14:41:17 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector. 2020/06/17 14:44:10 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector. 2020/06/17 14:44:14 CLSRSC-329: Replacing Clusterware entries in file '/etc/inittab' CRS-4123: Oracle High Availability Services has been started. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dbhost-n1' CRS-2677: Stop of 'ora.drivers.acfs' on 'dbhost-n1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'dbhost-n1' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.evmd' on 'dbhost-n1' CRS-2672: Attempting to start 'ora.mdnsd' on 'dbhost-n1' CRS-2676: Start of 'ora.mdnsd' on 'dbhost-n1' succeeded CRS-2676: Start of 'ora.evmd' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'dbhost-n1' CRS-2676: Start of 'ora.gpnpd' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'dbhost-n1' CRS-2676: Start of 'ora.gipcd' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'dbhost-n1' CRS-2676: Start of 'ora.cssdmonitor' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'dbhost-n1' CRS-2672: Attempting to start 'ora.diskmon' on 'dbhost-n1' CRS-2676: Start of 'ora.diskmon' on 'dbhost-n1' succeeded CRS-2676: Start of 'ora.cssd' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'dbhost-n1' CRS-2672: Attempting to start 'ora.ctssd' on 'dbhost-n1' CRS-2676: Start of 'ora.ctssd' on 'dbhost-n1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'dbhost-n1' CRS-2676: Start of 'ora.asm' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.storage' on 'dbhost-n1' CRS-2676: Start of 'ora.storage' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.crf' on 'dbhost-n1' CRS-2676: Start of 'ora.crf' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'dbhost-n1' CRS-2676: Start of 'ora.crsd' on 'dbhost-n1' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-6017: Processing resource auto-start for servers: dbhost-n1 CRS-2672: Attempting to start 'ora.net1.network' on 'dbhost-n1' CRS-2676: Start of 'ora.net1.network' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.ons' on 'dbhost-n1' CRS-2673: Attempting to stop 'ora.dbhost-n1.vip' on 'dbhost-n2' CRS-2677: Stop of 'ora.dbhost-n1.vip' on 'dbhost-n2' succeeded CRS-2672: Attempting to start 'ora.dbhost-n1.vip' on 'dbhost-n1' CRS-2676: Start of 'ora.ons' on 'dbhost-n1' succeeded CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'dbhost-n2' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'dbhost-n2' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'dbhost-n2' CRS-2677: Stop of 'ora.scan1.vip' on 'dbhost-n2' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'dbhost-n1' CRS-2676: Start of 'ora.dbhost-n1.vip' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.LISTENER.lsnr' on 'dbhost-n1' CRS-2672: Attempting to start 'ora.LISTENER_GSM.lsnr' on 'dbhost-n1' CRS-2672: Attempting to start 'ora.LISTENER_TEST.lsnr' on 'dbhost-n1' CRS-2676: Start of 'ora.LISTENER.lsnr' on 'dbhost-n1' succeeded CRS-2676: Start of 'ora.LISTENER_TEST.lsnr' on 'dbhost-n1' succeeded CRS-2676: Start of 'ora.LISTENER_GSM.lsnr' on 'dbhost-n1' succeeded CRS-2676: Start of 'ora.scan1.vip' on 'dbhost-n1' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'dbhost-n1' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'dbhost-n1' succeeded CRS-6016: Resource auto-start has completed for server dbhost-n1 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. Oracle Clusterware active version on the cluster is [12.1.0.2.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [1679636340]. dbhost-n1 2020/06/17 14:46:54 /gridapp/app/oracle/product/grid12c/cdata/dbhost-n1/backup_20200617_144654.olr 1679636340 dbhost-n1 2017/05/03 17:15:15 /gridapp/app/oracle/product/grid12c/cdata/dbhost-n1/backup_20170503_171515.olr 0
12. Start db services:
export ORACLE_HOME=/oracle/app/oracle/product/12.1.0.2/dbhome_1 export PATH=$ORACLE_HOME/bin:$PATH srvctl stop home -o /oracle/app/oracle/product/12.1.0.2/dbhome_1 -s /dumparea/db_prepatch_stat.txt -n dbhost-n1
Now patching activities on node1 ( dbhost-n1) has been completed. So proceed with same patching on node2 ( dbhost-n2).
NOTE – > PLEASE MAKE SURE YOU DO ALL THE ABOVE STEPS ON OTHER NODES AS WELL.
11. Run the datapatch utility.
Once all nodes GRID_HOME and DB_HOME has been patched, Run the datapatch script from DB_HOME.
from node 1: export ORACLE_HOME=/oracle/app/oracle/product/12.1.0.2/dbhome_1 cd $ORACLE_HOME/OPatch/ ./datapatch -verbose
With this patching activity has been completed with manual method.
You can check the inventory details as below.
$ORACLE_HOME/OPatch/opatch lsinventory
$GRID_HOME/OPatch/opatch lsinvenotry
from db :
select * from dba_registry_sqlpatch;
SQL> select patch_id,version,action,ACTION_TIME from dba_registry_sqlpatch order by ACTION_TIME desc; PATCH_ID VERSION ACTION ACTION_TIME ---------- -------------------- --------------- --------------------------------------------------------------------------- 29496791 12.1.0.2 APPLY 16-JUN-20 05.29.28.229750 PM 22674709 12.1.0.2 APPLY 16-JUN-20 05.29.27.650900 PM 28487821 12.1.0.2 ROLLBACK 16-JUN-20 05.29.27.635975 PM 28731800 12.1.0.2 APPLY 09-APR-19 10.05.40.703442 AM 28487821 12.1.0.2 APPLY 13-SEP-18 10.54.26.749312 AM 27547374 12.1.0.2 APPLY 29-AUG-18 09.53.53.069411 AM 26925263 12.1.0.2 APPLY 11-APR-18 11.37.14.482368 AM 26717470 12.1.0.2 APPLY 08-NOV-17 01.16.15.961195 PM 25397136 12.1.0.2 APPLY 04-MAY-17 09.29.08.515045 AM
