This tutorial explains how to delete a node from a multi node grid infrastructure setup.
Current GRID NODES -> DBHOST3( node 1) , DBHOST4 ( node 2)
Node to be deleted -> DBHOST4
Grid version -> 19c
STEPS OVERVIEW:
- Precheck
- Delete the instance from DBHOST4 using dbca
- Deinstall the oracle database home from DBHOST4
- Deinstall grid infrastructure home from DBHOST4
- Delete clusterware config
- Postcheck
1. PRECHECK
check whether both the nodes were unpinned or not.
[oracle@DBHOST3 ~]$ olsnodes -s -t
DBHOST3 Active Unpinned
DBHOST4 Active Unpinned
Check the cluster resources:
[oracle@DBHOST3 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE DBHOST3 STABLE
ONLINE ONLINE DBHOST4 STABLE
ora.MGMT.GHCHKPT.advm
OFFLINE OFFLINE DBHOST3 STABLE
OFFLINE OFFLINE DBHOST4 STABLE
ora.chad
ONLINE ONLINE DBHOST3 STABLE
ONLINE ONLINE DBHOST4 STABLE
ora.helper
OFFLINE OFFLINE DBHOST3 STABLE
OFFLINE OFFLINE DBHOST4 IDLE,STABLE
ora.mgmt.ghchkpt.acfs
OFFLINE OFFLINE DBHOST3 STABLE
OFFLINE OFFLINE DBHOST4 STABLE
ora.net1.network
ONLINE ONLINE DBHOST3 STABLE
ONLINE ONLINE DBHOST4 STABLE
ora.ons
ONLINE ONLINE DBHOST3 STABLE
ONLINE ONLINE DBHOST4 STABLE
ora.proxy_advm
OFFLINE OFFLINE DBHOST3 STABLE
OFFLINE OFFLINE DBHOST4 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ARCH.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 OFFLINE OFFLINE STABLE
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE DBHOST4 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE DBHOST3 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE DBHOST3 STABLE
ora.MGMT.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 OFFLINE OFFLINE STABLE
ora.OCRVD.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 OFFLINE OFFLINE STABLE
ora.REDOA.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 OFFLINE OFFLINE STABLE
ora.REDOB.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 Started,STABLE
2 ONLINE ONLINE DBHOST4 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE ONLINE DBHOST4 STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE DBHOST3 STABLE
ora.DBHOST3.vip
1 ONLINE ONLINE DBHOST3 STABLE
ora.DBHOST4.vip
1 ONLINE ONLINE DBHOST4 STABLE
ora.mgmtdb
1 ONLINE ONLINE DBHOST4 Open,STABLE
ora.lrmprod.db
1 ONLINE ONLINE DBHOST3 Open,HOME=/oracle/ap
p/oracle/product/19.
3.0/dbhome_1,STABLE
2 ONLINE ONLINE DBHOST4 Open,HOME=/oracle/ap
p/oracle/product/19.
3.0/dbhome_1,STABLE
ora.qosmserver
1 ONLINE ONLINE DBHOST4 STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE DBHOST4 STABLE
ora.scan2.vip
1 ONLINE ONLINE DBHOST3 STABLE
ora.scan3.vip
1 ONLINE ONLINE DBHOST3 STABLE
--------------------------------------------------------------------------------
2. Delete oracle instance using dbca
If any database instance is running on the node, then this need to be removed before deinstalling grid.
[oracle@DBHOST3 ~]$ crsctl status resource -w "TYPE = ora.database.type" -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.lrmprod.db
1 ONLINE ONLINE DBHOST3 Open,HOME=/oracle/ap
p/oracle/product/19.
3.0/dbhome_1,STABLE
2 ONLINE ONLINE DBHOST4 Open,HOME=/oracle/ap
p/oracle/product/19.
3.0/dbhome_1,STABLE
--------------------------------------------------------------------------------
Run dbca from node1:
[oracle@DBHOST03~] cd $ORACLE_HOME/bin
[oracle@DBHOST03~]./dbca
Instance successfully deleted. Now proceed with deinstalling the oracle db home.
3. DEINSTALL ORACLE DATABASE HOME:( RUN ON NODE2)
[oracle@DBHOST4]$ cd $ORACLE_HOME/deinstall
[oracle@DBHOST4 deinstall]$ ./deinstall -local
Checking for required files and bootstrapping ...
Please wait ...
Location of logs /tmp/deinstall2021-06-22_12-10-54PM/logs/
############ ORACLE DECONFIG TOOL START ############
######################### DECONFIG CHECK OPERATION START #########################
## [START] Install check configuration ##
Checking for existence of the Oracle home location /oracle/app/oracle/product/19.3.0/dbhome_1
Oracle Home type selected for deinstall is: Oracle Real Application Cluster Database
Oracle Base selected for deinstall is: /oracle/app/oracle
Checking for existence of central inventory location /crsapp/app/oraInventory
Checking for existence of the Oracle Grid Infrastructure home
The following nodes are part of this cluster: DBHOST4
Checking for sufficient temp space availability on node(s) : 'DBHOST4'
## [END] Install check configuration ##
Network Configuration check config START
Network de-configuration trace file location: /tmp/deinstall2021-06-22_12-10-54PM/logs/netdc_check2021-06-22_12-11-02PM.log
Network Configuration check config END
Database Check Configuration START
Database de-configuration trace file location: /tmp/deinstall2021-06-22_12-10-54PM/logs/databasedc_check2021-06-22_12-11-02PM.log
Use comma as separator when specifying list of values as input
Specify the list of database names that are configured locally on this node for this Oracle home. Local configurations of the discovered databases will be removed []:
Database Check Configuration END
######################### DECONFIG CHECK OPERATION END #########################
####################### DECONFIG CHECK OPERATION SUMMARY #######################
Oracle Grid Infrastructure Home is:
The following nodes are part of this cluster: DBHOST4
The cluster node(s) on which the Oracle home deinstallation will be performed are:DBHOST4
Oracle Home selected for deinstall is: /oracle/app/oracle/product/19.3.0/dbhome_1
Inventory Location where the Oracle home registered is: /crsapp/app/oraInventory
Do you want to continue (y - yes, n - no)? [n]: y
A log of this session will be written to: '/tmp/deinstall2021-06-22_12-10-54PM/logs/deinstall_deconfig2021-06-22_12-11-01-PM.out'
Any error messages from this session will be written to: '/tmp/deinstall2021-06-22_12-10-54PM/logs/deinstall_deconfig2021-06-22_12-11-01-PM.err'
######################## DECONFIG CLEAN OPERATION START ########################
Database de-configuration trace file location: /tmp/deinstall2021-06-22_12-10-54PM/logs/databasedc_clean2021-06-22_12-11-02PM.log
Network Configuration clean config START
Network de-configuration trace file location: /tmp/deinstall2021-06-22_12-10-54PM/logs/netdc_clean2021-06-22_12-11-02PM.log
Network Configuration clean config END
######################### DECONFIG CLEAN OPERATION END #########################
####################### DECONFIG CLEAN OPERATION SUMMARY #######################
#######################################################################
############# ORACLE DECONFIG TOOL END #############
Using properties file /tmp/deinstall2021-06-22_12-10-54PM/response/deinstall_2021-06-22_12-11-01-PM.rsp
Location of logs /tmp/deinstall2021-06-22_12-10-54PM/logs/
############ ORACLE DEINSTALL TOOL START ############
####################### DEINSTALL CHECK OPERATION SUMMARY #######################
A log of this session will be written to: '/tmp/deinstall2021-06-22_12-10-54PM/logs/deinstall_deconfig2021-06-22_12-11-01-PM.out'
Any error messages from this session will be written to: '/tmp/deinstall2021-06-22_12-10-54PM/logs/deinstall_deconfig2021-06-22_12-11-01-PM.err'
######################## DEINSTALL CLEAN OPERATION START ########################
## [START] Preparing for Deinstall ##
Setting LOCAL_NODE to DBHOST4
Setting CLUSTER_NODES to DBHOST4
Setting CRS_HOME to false
Setting oracle.installer.invPtrLoc to /tmp/deinstall2021-06-22_12-10-54PM/oraInst.loc
Setting oracle.installer.local to true
## [END] Preparing for Deinstall ##
Setting the force flag to false
Setting the force flag to cleanup the Oracle Base
Oracle Universal Installer clean START
Detach Oracle home '/oracle/app/oracle/product/19.3.0/dbhome_1' from the central inventory on the local node : Done
Failed to delete the directory '/oracle/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchautodb/driver/2021-06-21-14-26-25/machine-readable'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/oracle/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchautodb/driver/2021-06-21-14-26-25/.tmp'. Either user has no permission to delete or it is in use.
Delete directory '/oracle/app/oracle/product/19.3.0/dbhome_1' on the local node : Failed <<<<
Delete directory '/crsapp/app/oraInventory' on the local node : Done
Failed to delete the directory '/oracle/app/oracle'. The directory is not empty.
Delete directory '/oracle/app/oracle' on the local node : Failed <<<<
Oracle Universal Installer cleanup completed with errors.
Oracle Universal Installer clean END
## [START] Oracle install clean ##
## [END] Oracle install clean ##
######################### DEINSTALL CLEAN OPERATION END #########################
####################### DEINSTALL CLEAN OPERATION SUMMARY #######################
Successfully detached Oracle home '/oracle/app/oracle/product/19.3.0/dbhome_1' from the central inventory on the local node.
Failed to delete directory '/oracle/app/oracle/product/19.3.0/dbhome_1' on the local node due to error : Either user has no permission to delete or file is in use.
Review the permissions and manually delete '/oracle/app/oracle/product/19.3.0/dbhome_1' on local node.
Failed to delete directory '/oracle/app/oracle' on the local node due to error : Either user has no permission to delete or file is in use.
Review the permissions and manually delete '/oracle/app/oracle' on local node.
Oracle Universal Installer cleanup completed with errors.
Run 'rm -r /etc/oraInst.loc' as root on node(s) 'DBHOST4' at the end of the session.
Run 'rm -r /opt/ORCLfmap' as root on node(s) 'DBHOST4' at the end of the session.
Review the permissions and contents of '/oracle/app/oracle' on nodes(s) 'DBHOST4'.
If there are no Oracle home(s) associated with '/oracle/app/oracle', manually delete '/oracle/app/oracle' and its contents.
Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################
############# ORACLE DEINSTALL TOOL END #############
[oracle@DBHOST4 deinstall]$
Few error like failed to delete directory got reported. We can delete those directories manually.
[oracle@DBHOST4]$ rm -rf /oracle/app/oracle
4. DEINSTALL GRID HOME USING GRIDSETUP.SH ( Run from node 1)
[oracle@DBHOST3]cd $GRID_HOME
[oracle@DBHOST3]./gridSetup.sh
RUN ON NODE2:
[root@DBHOST4 ~]# /crsapp/app/oracle/product/19c/crs/install/rootdeinstall.sh
Using configuration parameter file: /tmp/deinstall_OraGI19Home1.rsp
The log of current session can be found at:
/crsapp/app/oraInventory/logs/crsdeconfig_DBHOST4_2021-06-22_12-02-56AM.log
Redirecting to /bin/systemctl restart rsyslog.service
2021/06/22 12:04:18 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.
2021/06/22 12:04:18 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.
2021/06/22 12:04:19 CLSRSC-336: Successfully deconfigured Oracle Clusterware stack on this node
Checking for required files and bootstrapping ...
Please wait ...
Using properties file /tmp/deinstall_OraGI19Home1.rsp
Location of logs /crsapp/app/oraInventory/logs/
############ ORACLE DEINSTALL TOOL START ############
A log of this session will be written to: '/crsapp/app/oraInventory/logs/deinstall_deconfig2021-06-22_12-04-29-PM.out'
Any error messages from this session will be written to: '/crsapp/app/oraInventory/logs/deinstall_deconfig2021-06-22_12-04-29-PM.err'
######################## DEINSTALL CLEAN OPERATION START ########################
## [START] Preparing for Deinstall ##
Setting LOCAL_NODE to DBHOST4
Setting CLUSTER_NODES to DBHOST4
Setting CRS_HOME to true
Setting oracle.installer.invPtrLoc to /tmp/deinstall2021-06-22_12-04-22PM/oraInst.loc
Setting oracle.installer.local to true
## [END] Preparing for Deinstall ##
Setting the force flag to false
Setting the force flag to cleanup the Oracle Base
Oracle Universal Installer clean START
Detach Oracle home '/crsapp/app/oracle/product/19c' from the central inventory on the local node : Done
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/jet/v6.0.0'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/jet'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/css/fonts'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/css/images'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/css/rat'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/css/site'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/css/codeMirror'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa/css'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps/tfa'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat/webapps'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home/tomcat'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4/tfa_home'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa/DBHOST4'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c/tfa'. The directory is not empty.
Failed to delete the directory '/crsapp/app/oracle/product/19c'. The directory is not empty.
Delete directory '/crsapp/app/oracle/product/19c' on the local node : Failed <<<<
Failed to delete the directory '/crsapp/app/oracle_base/tfa'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle_base/oracle.ahf'. Either user has no permission to delete or it is in use.
Failed to delete the directory '/crsapp/app/oracle_base'. The directory is not empty.
Delete directory '/crsapp/app/oracle_base' on the local node : Failed <<<<
Oracle Universal Installer cleanup completed with errors.
Oracle Universal Installer clean END
## [START] Oracle install clean ##
## [END] Oracle install clean ##
######################### DEINSTALL CLEAN OPERATION END #########################
####################### DEINSTALL CLEAN OPERATION SUMMARY #######################
Successfully detached Oracle home '/crsapp/app/oracle/product/19c' from the central inventory on the local node.
Failed to delete directory '/crsapp/app/oracle/product/19c' on the local node due to error : Either user has no permission to delete or file is in use.
Review the permissions and manually delete '/crsapp/app/oracle/product/19c' on local node.
Failed to delete directory '/crsapp/app/oracle_base' on the local node due to error : Either user has no permission to delete or file is in use.
Review the permissions and manually delete '/crsapp/app/oracle_base' on local node.
Oracle Universal Installer cleanup completed with errors.
Review the permissions and contents of '/crsapp/app/oracle_base' on nodes(s) 'DBHOST4'.
If there are no Oracle home(s) associated with '/crsapp/app/oracle_base', manually delete '/crsapp/app/oracle_base' and its contents.
Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################
############# ORACLE DEINSTALL TOOL END #############
5. Delete clusterware config [ run From node 1]
-- as root user run below command from node 1
[root@DBHOST3 ~]# /crsapp/app/oracle/product/19c/bin/crsctl delete node -n DBHOST4
CRS-4661: Node DBHOST4 successfully deleted.
Delete remaining directories:
Deinstall script might fail to delete few file/directories. So we can delete them manually.
rm -rf /crsapp/app/oracle/product/19c
rm -rf /crsapp/app/oracle_base
6. POSTCHECK:
Run cluvfy postcheck script
[oracle@DBHOST3 ~]$ cluvfy stage -post nodedel -n DBHOST4 -verbose
Verifying Node Removal ...
Verifying CRS Integrity ...PASSED
Verifying Clusterware Version Consistency ...PASSED
Verifying Node Removal ...PASSED
Post-check for node removal was successful.
CVU operation performed: stage -post nodedel
Date: Jun 22, 2021 12:08:38 PM
CVU home: /crsapp/app/oracle/product/19c/
User: oracle
[oracle@DBHOST3 ~]$
Check updated cluster resource:
[oracle@DBHOST3 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE DBHOST3 STABLE
ora.MGMT.GHCHKPT.advm
OFFLINE OFFLINE DBHOST3 STABLE
ora.chad
ONLINE ONLINE DBHOST3 STABLE
ora.helper
OFFLINE OFFLINE DBHOST3 STABLE
ora.mgmt.ghchkpt.acfs
OFFLINE OFFLINE DBHOST3 STABLE
ora.net1.network
ONLINE ONLINE DBHOST3 STABLE
ora.ons
ONLINE ONLINE DBHOST3 STABLE
ora.proxy_advm
OFFLINE OFFLINE DBHOST3 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ARCH.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE DBHOST3 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE DBHOST3 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE DBHOST3 STABLE
ora.MGMT.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.OCRVD.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.REDOA.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.REDOB.dg(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 Started,STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE DBHOST3 STABLE
2 ONLINE OFFLINE STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE DBHOST3 STABLE
ora.DBHOST3.vip
1 ONLINE ONLINE DBHOST3 STABLE
ora.mgmtdb
1 ONLINE ONLINE DBHOST3 Open,STABLE
ora.lrmprod.db
1 ONLINE ONLINE DBHOST3 Open,HOME=/oracle/ap
p/oracle/product/19.
3.0/dbhome_1,STABLE
ora.qosmserver
1 ONLINE ONLINE DBHOST3 STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE DBHOST3 STABLE
ora.scan2.vip
1 ONLINE ONLINE DBHOST3 STABLE
ora.scan3.vip
1 ONLINE ONLINE DBHOST3 STABLE
--------------------------------------------------------------------------------
We have successfully removed a node from the multinode grid.
Thank you for sharing this simple yet informational article . Very objectively explained indeed . So kind of you ..
Thanks once again ..