RAC 10gr2 Guide

eTEAM Oracle 10g RAC Installation Guide
Document Information
Document Owner: eTeam DBA
Prepared by: Karthik
Document Version No.: V.01.00.00
Preparation Date: August 11th, 2006
Reviewers
Name Position
Version History
Version No. Version Date Revised by Description Filename
V.01.00.01 08/11/06 Initial Document
Creation
ADP Confidential Page 1 of 139 Last modified by kappukuttan

©2006 Automatic Data Processing, Inc. All Rights Reserved.
Document Information................................................................................................... 1
Reviewers ..................................................................................................................... 1
Version History ............................................................................................................. 1
RAC Architecture Overview .......................................................................................... 4
System Requirements................................................................................................... 5
Storage Foundation........................................................................................................................................... 5
EMC Shared Storage and I/O Fencing ........................................................................................................... 5
Private and Virtual IP’s ................................................................................................................................... 5
ifconfig -a on d1sol5................................................................................................................................... 6
ifconfig -a on d1sol6................................................................................................................................... 6
UNIX Preinstallation Steps .............................................................................................................................. 7
Create Oracle User and Group .................................................................................................................. 7
Configure Secure Shell -SSH..................................................................................................................... 7
Configure rsh ............................................................................................................................................. 8
Set Environment variables ......................................................................................................................... 8
Configure Kernel Resources ...................................................................................................................... 9

Before the Installation .................................................................................................................................... 11
Pre-Install checks for 10gR2 RDBMS (10.2.x) - SUN Solaris Platforms ................................................... 11
Copying SFRAC Libraries to /opt/ORCLcluster/lib ................................................................................... 12
Creating $CRS_HOME on each node...................................................................................................... 12
Create volumes for CRS and VOTE disk on shared disk ......................................................................... 12
Modify the CVM service group (main.cf) to add CRS resources............................................................... 13
Final Checkup before installing CRS and Oracle Binaries....................................................................... 14

Installing CRS for Oracle 10g........................................................................................................................ 14
Installing the Oracle 10g Binaries ................................................................................................................. 35
After Oracle Binary Installation.................................................................................................................... 52

Copying and Verifying IPC and VCSMM libraries..................................................................................... 52
Verifying ODM library............................................................................................................................... 52

Adding Oracle 10g Patches 10.2.0.2 .............................................................................................................. 53
Adding patch 10.2.0.2 to CRS.................................................................................................................. 53
Unzip the file ............................................................................................................................................ 53
Set the Environment Variables................................................................................................................. 53
Stop all Oracle processes ........................................................................................................................ 53
Adding patch 10.2.0.2 to Oracle Binaries ................................................................................................. 64
Post Installation ....................................................................................................................................... 76
Mandatory Patch for 10.2.0.2................................................................................................................... 78

main.cf with CRS resources ......................................................................................................................... 118

RAC Architecture Overview
Oracle Real Application Clusters (RAC) allows Oracle Database to run any packaged or custom application,
unchanged across a set of clustered servers. This provides the highest levels of availability and the most flexible
scalability. If a clustered server fails, Oracle continues running on the remaining servers. In addition, when you
need more processing power, simply add another server without taking users offline. To keep costs low, even
the highest-end systems can be built out of standardized, commodity parts.
Oracle Real Application Clusters provides a foundation for Oracle’s Enterprise Grid Computing Architecture.
Oracle RAC technology enables a low-cost hardware platform to deliver the highest quality of service that
rivals and exceeds the levels of availability achieved by the most expensive, mainframe SMP computers. By
dramatically reducing administration costs and providing new levels of administration flexibility, Oracle is
enabling the enterprise Grid environment.
Real Application Clusters run on top of a hardware cluster. A cluster is a group of independent servers (nodes)
that cooperate as a single system. The primary cluster components are processor nodes, a cluster interconnect,
and a shared storage subsystem. The nodes share access to the storage subsystem and resources that manage
data, but they do not physically share main memory in their respective nodes. Oracle cluster database combines
the memory in the individual nodes to provide a single view of the distributed cache memory for the entire
database system. A node can be made up of multiple processors. Each node has its own dedicated system
memory as well as its own operating system and database instance.

System Requirements
Please consult with the infrastructure team on the requirements for Sun servers, EMC Storage, Cluster
Interconnects, Public networks, Switch options, Memory, Swap and CPU. Moreover, check the RAC/Veritas on
Sun certification matrix for RAC updates on currently supported hardware/software in Metalink.
Before the installation of Oracle 10g RAC on Solaris 10, make sure following components has been installed
and configured appropriately.
Storage Foundation
Installed Veritas Storage foundation for Oracle Real Application Cluster (SFRAC 4.1 packages with MP1) and
Clustering configured on all Nodes that are part of the Cluster.
EMC Shared Storage and I/O Fencing
Verify that the shared disk arrays support SCSI-3 persistent reservations and I/O fencing.
Three types of shared storage required
1. Data disks to store shared data, created based on the file system layout,
2. CRS disks to store OCR and VOTE volumes and Directories
3. Coordinator disks for I/O fencing, at least three-coordinator disk possibly smallest LUN’s on
different disk arrays to resolve potential split-brain conditions and prevent data corruption.
Private and Virtual IP’s
Apart from host IP, make sure a public virtual IP exist on public NIC and a private IP address on the Private
NIC. Refer to page 44 and 47 on Release Notes: VERITAS Storage Foundation 4.1 for Oracle RAC
Maintenance Pack 1 Document id 280145 regarding creation of Private and Public IP address.
Hostname Host IP Virtual IP Private IP

d1sol5 192.168.201.203 d1sol5-virt-192.168.201.193 d1sol5-priv-10.10.10.1
d1sol6 192.168.201.204 d1sol6-virt-192.168.201.194 d1sol6-priv10.10.10.2

ifconfig -a on d1sol5
ip for Veritas
Cluster
Public Virtual ip on
Public NIC
Private ip on Private
NIC
ifconfig -a on d1sol6
Public Virtual ip on
Public NIC
Private ip on Private
NIC

UNIX Preinstallation Steps
Create Oracle User and Group
Logon as root and Create a local group (dba and oinstall) and local user (oracle)
# groupadd -g 201 dba
# groupadd -g 202 oinstall
# useradd -c "Oracle Software Owner" -g dba -G oinstall -u 201 -m -d /export/home/oracle -s /bin/ksh
oracle
Create a password for user oracle

# passwd oracle
To verify the oracle account,

# id oracle
Verifying that the User nobody Exists

# id nobody
Configure Secure Shell -SSH
From each node, logon as oracle

$ mkdir ~/.ssh
$ chmod 755 ~/.ssh
$ /usr/bin/ssh-keygen -t rsa
Cut and paste the following line separately

$ /usr/bin/ssh-keygen -t dsa
From the first node (d1sol5) ONLY, logged in as oracle (copy the local account's keys so that ssh to the local
node will work):
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
Now copy the keys to the other node (d1sol6) so that we can ssh to the remote node without being prompted for
a password.
$ ssh oracle@d1sol6 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
(Run each of them separately. SSH will prompt for the oracle password each time and if the commands are
pasted at the same time, the other commands will be lost when the first one flushes the input buffer prior to
prompting for the password.)
$ ssh oracle@d1sol6 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

$ chmod 644 ~/.ssh/authorized_keys

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$ ssh oracle@d1sol5 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ ssh oracle@d1sol5 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$ chmod 644 ~/.ssh/authorized_keys
As oracle on the node where the Oracle 10g Release 2 software will be installed (d1sol5)
$ exec /usr/bin/ssh-agent $SHELL
$ /usr/bin/ssh-add
To Test
$ ssh d1sol6 date
Should connect to the remote host without password prompt, If not refer to Metalink and Oracle Document for
further information on setting up the ssh.
Configure rsh
Make sure that the user Oracle can rsh between the nodes without having to use or confirm a password. Veritas
Vxfentsthdw command requires “+” in /.rhosts file. If not, in the user oracle account directory
/export/home/oracle, create a .rhosts file that includes a plus character(“+”) in the first line. On Node1 (d1sol5)
create a sample test.txt file and copy the file to node2 (d1sol6). Logon to node2 (d1sol6) and check if the file
exists. This tests both rsh and rcp services.
$ touch test.txt
$ rcp test.txt d1sol6:/tmp
$ rsh d1sol6
$ ls -al /tmp/test.txt
$ rm /tmp/test.txt
$ rm test.txt
Set Environment variables
Edit .profile file in /export/home/oracle directory and add following lines, save it
ORACLE_BASE=/opt/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/10g/db_1; export ORACLE_HOME
CRS_HOME=$ORACLE_BASE/product/10g/crs; export CRS_HOME
LD_LIBRARY_PATH_64=$ORACLE_HOME/lib:$CRS_HOME/lib; export LD_LIBRARY_PATH_64
LD_LIBRARY_PATH=$ORACLE_HOME/lib32; export LD_LIBRARY_PATH
CLASSPATH=$CLASSPATH:$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/j
lib:$ORACLE_HOME/network/jlib; export CLASSPATH
ORACLE_TERM=xterm; export ORACLE_TERM
PATH=$ORACLE_HOME/bin:/usr/ccs/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/lo
cal/sbin:/sbin:/bin::/usr/lib/vxvm/bin:/opt/VRTSvxfs/sbin:/opt/VRTSvcs/bin:/
opt/VRTS/bin:/opt/VRTSvcs/rac/bin:/opt/VRTSob/bin:/etc:/opt/EMCpower/bin/spa
rcv9:/etc/emc/bin:/usr/dt/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/ucb:/et
c:/usr/openwin/bin:.; export PATH
EDITOR=vi; export EDITOR

set -o vi
stty istrip
stty erase ^H
umask 022
export umask
ulimit -n 8192 >/dev/null 2>&1
ulimit -s 32768 >/dev/null 2>&1
PS1=`hostname`':$PWD[$ORACLE_SID]$ '
ULIMIT=999999999999999
export ULIMIT
HISTSIZE=1000
export HISTSIZE
HISTFILE=/var/userlog/`who am i|cut -f1 -d' '`.log
export HISTFILE
alias h=history
ORAENV_ASK=NO
MAIL=/usr/mail/${LOGNAME:?}
Configure Kernel Resources
Prior to Solaris 10, the System V IPC resources, consisting primarily of shared memory, message queues, and
semaphores, were set in the /etc/system file and reconfiguration needs a reboot. In Solaris 10 it is no longer
requires changes in the /etc/system file. Instead, it uses the resource control facility. For more information, refer
to “8.2 Appendix-2 Setting System V IPC kernel parameters” in Metalink Doc ID: 317257.1
Parameter Value Solaris 10 Resource Default Value

Control
set shmsys:shminfo_shmmax 18446744073709551615 project.max-shm-memory 1/4 of physical memory
set shmsys:shminfo_shmmin 1 N/A
set shmsys:shminfo_shmmni 256 project.max-shm-ids 128
set semsys:seminfo_semmni 0x1002 project.max-sem-ids 128
set semsys:seminfo_semmns 0x7631 N/A
set semsys:seminfo_semmnu 0x7631 Obsolete
set semsys:seminfo_semopm 4096 project.max-sem-ops 512
set semsys:seminfo_semvmx 32767 Obsolete
set shmsys:shminfo_shmseg 256 N/A
set semsys:seminfo_semmap 0x7631 Obsolete

set semsys:seminfo_semmsl 4096 project.max-sem-nsems 512
set semsys:seminfo_semume 512 Obsolete

Before moving further, please refer to Bug No 5237047 regarding INCORRECT SYSTEM
REQUIREMENTS FOR SOLARIS 10 in Metalink
On each node, logon as root
# projadd -U oracle -G dba -c "OracleProject" -p 201 user.oracle

# projmod -s -K "project.max-shm-memory=(priv,10g,deny)" user.oracle
# projmod -s -K "project.max-shm-ids=(priv,256,deny)" user.oracle
# projmod -s -K "project.max-sem-ids=(priv,4096,deny)" user.oracle
# projmod -s -K "project.max-sem-ops=(priv,4096,deny)" user.oracle
# projmod -s -K "project.max-sem-nsems=(priv,4096,deny)" user.oracle
To set UDP parameters to 65536 bytes, write a startup script in /etc/init.d
# cd /etc/init.d
# touch udp_rac
# chmod 775 udp_rac
# vi udp_rac
---------------------------Script Start-------------------------------------
#!/sbin/sh
case "$1" in
'start')
ndd -set /dev/udp udp_xmit_hiwat 65536
ndd -set /dev/udp udp_recv_hiwat 65536
;;
.
'state')
ndd /dev/udp udp_xmit_hiwat
ndd /dev/udp udp_recv_hiwat
;;
.
*)
echo "Usage: $0 { start | state }"
exit 1
;;
esac
---------------------------Script End---------------------------------------
# cd /etc/rc3.d
# ln -s /etc/init.d/udp_rac /etc/rc3.d/S86udp_rac
to view/verify the settings logon as oracle and run

$ prctl $$

As the srvctl utility not unable to get the right SHM related settings using prctl/projmod, Add following entries
to /etc/system file and reboot the servers. For more information, Please refer to Metalink Doc id 367442.1
forceload: sys/semsys
forceload: sys/shmsys
set shmsys:shminfo_shmmax=18446744073709551615
set shmsys:shminfo_shmmin=1
set shmsys:shminfo_shmmni=256
set semsys:seminfo_semmni=0x1002
set semsys:seminfo_semmns=0x7631
set semsys:seminfo_semmnu=0x7631
set semsys:seminfo_semopm=4096
set semsys:seminfo_semvmx=32767
set shmsys:shminfo_shmseg=256
set semsys:seminfo_semmap=0x7631
set semsys:seminfo_semmsl=4096
set semsys:seminfo_semume=512
set tune_t_fsflushr=30
set autoup=900
set rlim_fd_max=65536
set rlim_fd_cur=32768
set sq_max_size=0
set tcp:tcp_conn_hash_size=8192
Reboot the servers
ulimit settings, following command will display the current ulimit settings:
$ ulimit -a
If needed add following entries to the .profile for oracle to reset stack and nofiles values
ulimit -n 4096 >/dev/null 2>&1
ulimit -s 32768 >/dev/null 2>&1
Before the Installation
Pre-Install checks for 10gR2 RDBMS (10.2.x) - SUN Solaris Platforms Refer to Doc ID: 334567.1
Logon as Oracle
$ cd /ora/scr/preinstall
$ tar xvf sun_10gR2_hcve_rdbms_091705.tar
Set the oracle environment variables, ORACLE_HOME
$ perl validate.pl 10gR2_rdbms_solaris_hcve_091705.txt
Or validate 10gR2_rdbms_solaris_hcve_091705.txt
Please review the log files and rectify the components that failed the test.
Copying SFRAC Libraries to /opt/ORCLcluster/lib
Logon as root and Copy SFRAC libraries to enable RAC recognize the cluster environment
# cp /opt/VRTSvcs/rac/lib/libskgxn2_64.so /opt/ORCLcluster/lib/libskgxn2.so
Creating $CRS_HOME on each node
CRS and Oracle Binaries will be installed locally on each server. Make sure $CRS_HOME is under
$ORACLE_BASE.
Make directories for

ORACLE_BASE is /opt/oracle
CRS_HOME is /opt/oracle/product/10g/crs
ORACLE_HOME is /opt/oracle/product/10g/db_1
Create volumes for CRS and VOTE disk on shared disk
Create OCR and VOTE-disk volumes and directories on the cluster file system. Assume that emcpower13s2 is
the allocated LUN, execute following commands as root on one of the node (d1sol5)
# vxdg -s init crs_oradg emcpower13s2
# vxassist -g crs_oradg make crs_vol 200M
# vxvol -g crs_oradg startall
# mkfs -F vxfs /dev/vx/rdsk/crs_oradg/crs_vol
On both nodes (d1sol5 and d1sol6)

# mkdir /ora_crs
# mount -F vxfs -o cluster /dev/vx/dsk/crs_oradg/crs_vol /ora_crs
# chown oracle:dba /ora_crs
# chmod 755 /ora_crs
On one node (d1sol5), logon as oracle

$ cd /ora_crs
$ mkdir CRS_OCR
$ mkdir VOTE-disk

Modify the CVM service group (main.cf) to add CRS resources.
Refer to the Modified main.cf in the appendix
Log into one of the nodes in the cluster (d1sol5) as root

# haconf -dump makero
# hastop -all -force
# cd /etc/VRTSvcs/conf/config
# cp main.cf main.orig
# vi main.cf
Include the following entries,
At the beginning of the file

include “PrivNIC.cf”
And in CVM service group

CFSMount crs_ora_mnt (
MountPoint = "/ora_crs"
BlockDevice = "/dev/vx/dsk/crs_oradg/crs_vol"
)
CVMVolDg crs_voldg (
CVMDiskGroup = crs_oradg
CVMVolume = { crs_vol }
CVMActivation = sw
)
PrivNIC ora_priv (
Device = { ipge2 = 0, ipge3 = 1 }
Address @d1sol5 = "10.10.10.1"
Address @d1sol6 = "10.10.10.2"
NetMask = "255.255.255.0"
)
And in dependencies
crs_ora_mnt requires crs_voldg
crs_ora_mnt requires vxfsckd
crs_voldg requires cvm_clus
Save and Close main.cf

Verify the syntax
# hacf –verify /etc/VRTSvcs/conf/config
Copy main.cf to the second node (d1sol6)
now start VCS in all nodes
#hastart
And check the VCS status
#hastatus –summ
All added resources should be online on both the nodes
Final Checkup before installing CRS and Oracle Binaries
Logon as oracle
1. Check ssh config for local node and remote node
2. Use prctl $$ to check the semaphores values
3. Check the Oracle user profile settings
4. Get host ip, public virtual ip and it’s interface, private ip and it’s interface name.
5. Bring down the ip on public virtual interface (as root) # ifconfig ipge0:2 down
6. Create volumes and directories for CRS and VOTE disk
7. CRS resources added to the veritas cluster main.cf and restart SFRAC
8. Copied SFRAC libskgxn2_64.so
9. Check the DISPLAY environmental variable, run xclock and verify
10.
Installing CRS for Oracle 10g
Make sure all required software is under /ora/scr directory.
Logon to node1 (d1sol5) as Oracle
$ cd /ora/scr/crs
$ ./runInstaller
Use ./runInstaller –ignoreSysPrereqs to ignore the prerequisites



Change the Name to CRS_HOME and path to /opt/oracle/product/10g/crs


Click Edit and modify Virtual hostnames (d1sol5_vip to d1sol5_virt and d1sol6_vip t d1sol6_virt) to
match the names in /etc/hosts

ipge0 is the public nic, so click Edit and change ipge0 interface type to Public





Don’t click the OK button now. Run the scripts one by one on each node, configure Virtual ip’s using vipca and
then comeback and click OK

Logon as root on node1 (d1sol5) and run

# /opt/oracle/oraInventory/orainstRoot.sh
and
# /opt/oracle/product/10g/crs/root.sh

Logon as root on 2nd node (d1sol6) and run

# /opt/oracle/oraInventory/orainstRoot.sh
and
# /opt/oracle/product/10g/crs/root.sh

Make sure CSS is active on both the nodes
At the end of the script, we might receive following error messages saying “ipge0 is not public”
Running vipca(silent) for configuring nodeapps

The given interface(s), "ipge0" is not public. Public interfaces should be used
to configure virtual IPs.
This is a known issues with vipca, Refer to page 60 of Release Notes: VERITAS Storage Foundation 4.1 for
Oracle RAC Maintenance Pack 1 Document id 280145. please invoke vipca manually
root@d1sol6 # cd /oracle/product/10g/crs/bin
root@d1sol6 # ./vipca



Enter Virtual IP alias name, d1sol5-virt.campd.local, press tab remaining column values will be filled
automatically


Click OK after running the scripts and vipca


Installing the Oracle 10g Binaries
Log on as oracle
$ cd /ora/scr/db
$ ./runInstaller


Select Custom

Change Name to ORACLE_HOME and Path to /opt/oracle/product/10g/db_1




Ignore the OUI finding in the area of “noexec_user_stack for the Solaris 10 OS (Refer Metalink 377679.1)
Checking for noexec_user_stack=1; found no entry.Failed <<<<

Check complete. The overall result of this check is: Failed <<<<





Don’t click OK now


# /opt/oracle/product/10g/db_1/root.sh


#/opt/oracle/product/10g/db_1/root.sh



After Oracle Binary Installation
Copying and Verifying IPC and VCSMM libraries
Logon as oracle and copy

$ cp /opt/VRTSvcs/rac/lib/libskgxp102_64.so $ORACLE_HOME/lib/libskgxp10.so
Verifying ODM library
Login as Oracle user

$ cd /opt/oracle/product/10g/db_1/lib
$ mv /opt/oracle/product/10g/db_1/lib/libodm10.so
/opt/oracle/product/10g/db_1/lib/libodm10.so.old
$ ln -s /usr/lib/sparcv9/libodm.so libodm10.so
$ ls -al /opt/oracle/product/10g/db_1/lib/libodm10.so
(makesure owner is oracle)
On d1sol5
On d1sol6

Adding Oracle 10g Patches 10.2.0.2

(Metalink Patch # 4547817)
Adding patch 10.2.0.2 to CRS
Should always upgrade Oracle Clusterware before you upgrade Oracle RAC Binaries
If we are installing this patch set on an existing Oracle Real Applications Cluster (RAC) 10g release 10.2.0.1
installation, we must run Oracle Universal Installer from the same node(d1sol5) from which the Oracle
Database software was installed.
Unzip the file

$ unzip p4547817_10202_SOLARIS64.zip
Set the Environment Variables
Set ORACLE_HOME, CRS_HOME in the .profile of Oracle user.
Stop all Oracle processes
Login as oracle user
On d1sol5
$ srvctl stop nodeapps -n d1sol5
On d1sol6
Logon as root
Shut down the Oracle Clusterware process by entering the following command on all nodes as the root user:

On d1sol5
# /etc/init.d/init.crs stop
On d1sol6
Make sure there is no Oracle processes running

$ ps -ef|grep oracle|grep -v grep
Log in as the Oracle user on node1 (d1sol5)
Start Oracle Universal Installer

$ cd /ora/scr/patch/10.2.0.2/Disk1
$ ./runInstaller
On the Welcome screen, click Next.


Change the Name to CRS_HOME and Path to /opt/oracle/product/10g/crs



Don’t click on Exit

Don’t click on Exit

Login as root on d1sol5 and run

# /opt/oracle/product/10g/crs/bin/crsctl stop crs
# /opt/oracle/product/10g/crs/install/root102.sh
To Check is crs is stopped

# ps -ef|grep d.bin
Refer to Metalink Doc id 358258.1 regarding crsctl stop crs, Failure 15 At Css Context Initialization When
Shutting Down Crs

Login as root on node 2 (d1sol6) and run

# /opt/oracle/product/10g/crs/bin/crsctl stop crs
# /opt/oracle/product/10g/crs/install/root102.sh

Click Exit now, after running the scripts as root user

Adding patch 10.2.0.2 to Oracle Binaries
Login as Oracle User
Stop all Oracle Processes
On d1sol5
On d1sol6
Login as root User
Shut down the Oracle Clusterware process by entering the following command on all nodes as the root user:
On d1sol5
On d1sol6
Make sure there is no Oracle processes running

# ps -ef|grep oracle|grep -v grep

Log in as the oracle user on node1(d1sol5)
Start Oracle Universal Installer

$ cd /ora/scr/patch/10.2.0.2/Disk1
$ ./runInstaller


Enter Name as ORACLE_HOME and Path as /opt/oracle/product/10g/db_1




Don’t click OK now







Post Installation
During patchset installation, all new files and directories are created with restricted access, by default. Users or
third party applications with a different group identifier from that of the database, which try to access client-side
utilities or libraries in the database home, will see permission errors when trying to access these files or
directories. Perform the following steps to change the permissions:
Change your directory to:
$ cd $ORACLE_HOME/install
Run changePerm.sh and specify the patched server Oracle home location, before accessing client-side utilities
or libraries in the database home. For RAC home, we need to run this script on all the nodes.
Login as Oracle
On d1sol5
$ cd /opt/oracle/product/10g/db_1/install
$ ./changePerm.sh
Look for errors in /tmp/changePerm_err.log and resolve it.

# chown oracle:dba externaljob.ora
# chmod 644 externaljob.ora

On d1sol6
$ cd /opt/oracle/product/10g/db_1/install
$ ./changePerm.sh

Mandatory Patch for 10.2.0.2
After installing 10.2.0.2 apply the mandatory patch for (bug 5117016) for incorrectly located in
$ORACLE_HOME/RDBMS/LIB/
For more information, Refer Metalink Doc id 5117016.8
This patch is applicable only on the server side. Ensure all the processes started out of the ORACLE_HOME
are being shutdown prior to applying this patch using srvctl , on every node
To apply the patch, unzip the PSE container file:

$ unzip p5117016_10202_SOLARIS64.zip
Set your current directory to the directory where the patch is located
$ cd /ora/scr/patch/5117016
Ensure that the directory containing the opatch script appears in the $PATH, then run the following command
on every node
$ $ORACLE_HOME/OPatch/opatch apply -local

On d1sol5

On d1sol6

Check the inventory

$ $ORACLE_HOME/OPatch/opatch lsinventory

Oracle® Database Critical Patch Update Release 10.2.0.2

Patch 5225799
Ensure that the Oracle Database installation is the same release for which we are applying this patch. In other
words, only apply the Release 10.2.0.2 CPUJul2006 patch to an Oracle Database Release 10.2.0.2.
Review Section 4, "Known Issues" before proceeding with this installation.
1. Shut down the instance on one node (d1sol5)
2. Shut down all nodeapps services on the node (d1sol5) in step 1:

Apply patch on the node (d1sol5) in step 1
3. Set current directory to the directory where the patch is located and then run OPatch by entering the
following command:
$ cd /ora/scr/patch/cpu/5225799
$ORACLE_HOME/OPatch/opatch apply OPatch.SKIP_VERIFY=true
(We need to specify OPatch.SKIP_VERIFY=true. For more information please refer to 353150.1 - OPatch
Failing Validation Phase When Archiving Really Did Not Fail)
4. If there are errors, refer to Section 4, "Known Issues".
Start all nodeapps services on the node in step 1:

$ srvctl start nodeapps -n d1sol5
Shut down the instance on the next node.

Repeat steps 1-6 on all the nodes in the cluster.
Check the opatch inventory
Logon as root and start crs on both nodes

# /etc/init.d/init.crs start

Check the inventory

$ $ORACLE_HOME/OPatch/opatch lsinventory

Creating the Oracle Database
CRS should be running
To check crs daemon running or not

$CRS_HOME/bin/crs_stat
Create the mount points and directories for the Oracle Database
Login as root and
# mkdir /hapdb
# cd /hapdb
# mkdir archive flashdb data1 index1 redo1 redo2 redocopy system temp
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_archive_vol /hapdb/archive
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_flashdb_vol /hapdb/flashdb
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_data1_vol /hapdb/data1
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_index1_vol /hapdb/index1
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_redo1_vol /hapdb/redo1
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_redo2_vol /hapdb/redo2
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_redocopy_vol /hapdb/redocopy
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_system_vol /hapdb/system
# mount -F vxfs -o cluster /dev/vx/dsk/ora_dg1/ora_temp_vol /hapdb/temp
# chown -R oracle:dba archive flashdb data1 index1 redo1 redo2 redocopy system temp
Verify all file system exist and owner is Oracle
Default template is already created for this database. So Copy hapdb.dbt to

/opt/oracle/product/10g/db_1/assistants/dbca/templates. If not select Custom or General Purpose database
template
Before creating the database using dbca, configure LISTENER using netca.
$ netca






Creating 10g database using dbca

Logon as Oracle
$ dbca




If hapdb template is not available select Custom Database option












Click on All Initialization Parameters and Change the location for all dump (a/b/c/udump) init. parameters









After creating the database, check spfile is in /hapdb/system/admin/pfile/ directory if not

$ cd $ORACLE_HOME/dbs
Change the location of spfile in inithapdb1.ora to SPFILE='/hapdb/system/admin/pfile/spfilehapdb.ora'
And move the file from /hapdb/data1/hapdb to /hapdb/system/admin/pfile/

(No issue when used a different template, not hapdb_new )
To check the status
root@d1sol6 # /opt/oracle/product/10g/crs/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....L5.lsnr application ONLINE ONLINE d1sol5
ora.d1sol5.gsd application ONLINE ONLINE d1sol5
ora.d1sol5.ons application ONLINE ONLINE d1sol5
ora.d1sol5.vip application ONLINE ONLINE d1sol5
ora....L6.lsnr application ONLINE ONLINE d1sol6
ora.d1sol6.gsd application ONLINE ONLINE d1sol6
ora.d1sol6.ons application ONLINE ONLINE d1sol6
ora.d1sol6.vip application ONLINE ONLINE d1sol6
ora.hapdb.db application ONLINE ONLINE d1sol5

ora....b1.inst application ONLINE ONLINE d1sol5
ora....b2.inst application ONLINE ONLINE d1sol6
To check the service status

$ srvctl status service –d hapdb
Configure the CVM and Oracle Service Groups for Oracle 10g
Log into one of the nodes in the cluster (d1sol5) as root

# haconf -dump makero
# hastop -all -force
# cd /etc/VRTSvcs/conf/config
# cp main.cf main.before10gdb
# vi main.cf
Refer to the Modified and final main.cf in the appendix
Save and Close main.cf
Verify the syntax
# hacf –verify /etc/VRTSvcs/conf/config
Copy main.cf to the second node (d1sol6)
now start VCS in all nodes
#hastart
And check the VCS status
#hastatus –summ
All added resources should be online on both the nodes
If voting is disk unavailable, server will be rebooted every few minutes, to avoid the reboot disable the crs
# /etc/init.d/init.crs disable
Correct the issue and reenable it and reboot the server

# /etc/init.d/init.crs enable
# reboot
If Veritas Agent is used for Detail Monitoring (Secondary monitoring) then
create user vcsmon

identified by vcsmon
default tablespace TOOLS
temporary tablespace TEMP
quota 500K on TOOLS
/
grant create session to vcsmon
/
create table vcsmon. vcstable_d1sol5 ( tstamp date )
/
insert into vcsmon. vcstable_d1sol5 ( tstamp ) values ( SYSDATE )
/
create table vcsmon. vcstable_d1sol6 ( tstamp date )
/
insert into vcsmon. vcstable_d1sol6 ( tstamp ) values ( SYSDATE )
/
To scp and unzip the files

root@D1SOL5 # scp 10gr2_cluster_sol.cpio.gz root@d1sol6:/opt/oracle/software/crs/
10gr2_cluster_sol.cp 100%
|**********************************************************************************
*********| 368 MB 00:51
root@D1SOL5 # gunzip 10gr2_cluster_sol.cpio.gz
root@D1SOL5 # cpio -idmv <10gr2_cluster_sol.cpio
Stopping CRS Before Stopping VCS in Oracle 10g Clusters
In SFRAC clusters with Oracle 10g, stop the CRS daemons before using the hastop -all
command to stop VCS. The required sequence of commands is:
1. Stop Oracle on each node:
# hagrp -offline oracle_group -sys -system_name
2. Stop the CRS daemons on each node:
3. Stop VCS. From one node, type:
# hastop -all
After 10.2.0.2 patchset is installed, RDBMS crashes with internal errors on remote nodes
Doc id 365851.1
1. Ensure all instances are shut down

2. Manually copy $ORACLE_HOME/rdbms/lib/libknlopt.a from the local node to all remote nodes .
3. Relink Oracle on all remote nodes
eg: cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk ioracle

main.cf with CRS resources
include "types.cf"
include "CFSTypes.cf"
include "CVMTypes.cf"
include "OracleTypes.cf"
include "PrivNIC.cf"
cluster d1sol-c (
UserNames = { admin = IhiAhcHeiDiiGqiChf }
ClusterAddress = "192.168.201.192"
Administrators = { admin }
CredRenewFrequency = 0
UseFence = SCSI3
HacliUserLevel = COMMANDROOT
CounterInterval = 5
)
system d1sol5 (
)

system d1sol6 (
)
group ClusterService (
SystemList = { d1sol5 = 0, d1sol6 = 1 }
AutoStartList = { d1sol5, d1sol6 }
OnlineRetryLimit = 3
OnlineRetryInterval = 120
)
IP webip (
Device = ipge0
Address = "192.168.201.192"
NetMask = "255.255.255.0"
)
NIC csgnic (
Device = ipge0
)
VRTSWebApp VCSweb (
Critical = 0
AppName = vcs
InstallDir = "/opt/VRTSweb/VERITAS"
TimeForOnline = 5
RestartLimit = 3
)
VCSweb requires webip

webip requires csgnic
// resource dependency tree

//
// group ClusterService
// {
// VRTSWebApp VCSweb
// {
// IP webip
// {
// NIC csgnic
// }
// }
// }
group cvm (
AutoFailOver = 0
Parallel = 1
)
)
CVMActivation = sw
)
PrivNIC ora_priv (
Address @d1sol5 = "10.10.10.1"
Address @d1sol6 = "10.10.10.2"
NetMask = "255.255.255.0"
)
CFSQlogckd qlogckd (
Critical = 0
)
CFSfsckd vxfsckd (
)
CVMCluster cvm_clus (
CVMClustName = d1sol-c
CVMNodeId = { d1sol5 = 0, d1sol6 = 1 }
CVMTransport = gab
CVMTimeout = 200
)
CVMVxconfigd cvm_vxconfigd (
Critical = 0
CVMVxconfigdArgs = { syslog }
)

cvm_clus requires cvm_vxconfigd
qlogckd requires cvm_clus
vxfsckd requires qlogckd

//
// group cvm
// {
// CFSMount crs_ora_mnt
// {
// CVMVolDg crs_voldg
// {
// CVMCluster cvm_clus
// {
// CVMVxconfigd cvm_vxconfigd
// }
// }
// CFSfsckd vxfsckd
// {
// CFSQlogckd qlogckd
// {
// {
// }
// }
// }
// }
// PrivNIC ora_priv
// }

main.cf with CVM and Oracle Service Groups and without the Oracle Agent
include "types.cf"
cluster d1sol-c (
UserNames = { admin = IhiAhcHeiDiiGqiChf, vcsop = GjkIjgJgjRkd }
Operators = { vcsop }
UseFence = SCSI3
CounterInterval = 5
)
system d1sol5 (
)
system d1sol6 (
)
)
IP webip (
Device = ipge0
Address = "192.168.201.192"
NetMask = "255.255.255.0"
)
NIC csgnic (
Device = ipge0
)
NotifierMngr ntfr (
SmtpServer = "160.0.10.50"
SmtpRecipients = { "kappuk@lexus.elabor.com" = Warning }
)
VRTSWebApp VCSweb (

Critical = 0
AppName = vcs
TimeForOnline = 5
RestartLimit = 3
)


//
// {
// {
// IP webip
// {
// NIC csgnic
// }
// }
// NotifierMngr ntfr
// }
group cvm (
AutoFailOver = 0
Parallel = 1
)
Application cssd (
Critical = 0
StartProgram = "/opt/VRTSvcs/rac/bin/cssd-online"
StopProgram = "/opt/VRTSvcs/rac/bin/cssd-offline stopcrs"
CleanProgram = "/opt/VRTSvcs/rac/bin/cssd-clean"
MonitorProgram = "/opt/VRTSvcs/rac/bin/cssd-monitor"
)
)
CFSMount hapdb_archive_mnt (
Critical = 0
MountPoint = "/hapdb/archive"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_archive_vol"
)
CFSMount hapdb_data1_mnt (
Critical = 0
MountPoint = "/hapdb/data1"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_data1_vol"
)
CFSMount hapdb_flashdb_mnt (
Critical = 0
MountPoint = "/hapdb/flashdb"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_flashdb_vol"
)
CFSMount hapdb_index1_mnt (
Critical = 0
MountPoint = "/hapdb/index1"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_index1_vol"
)
CFSMount hapdb_redo1_mnt (
Critical = 0
MountPoint = "/hapdb/redo1"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redo1_vol"
)
Critical = 0
)
CFSMount hapdb_redocopy_mnt (
Critical = 0
MountPoint = "/hapdb/redocopy"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redocopy_vol"
)
CFSMount hapdb_system_mnt (
Critical = 0
MountPoint = "/hapdb/system"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_system_vol"
)
CFSMount hapdb_temp_mnt (
Critical = 0
MountPoint = "/hapdb/temp"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_temp_vol"
)
Critical = 0
)
CFSfsckd vxfsckd (
)
CVMTransport = gab
CVMTimeout = 200
)
CVMActivation = sw
)
CVMVolDg hapdb_archive_voldg (
CVMDiskGroup = ora_dg1
CVMVolume = { ora_archive_vol }
CVMActivation = sw
)
CVMVolDg hapdb_data1_voldg (
CVMVolume = { ora_data1_vol }
CVMActivation = sw
)
CVMVolDg hapdb_flashdb_voldg (
CVMVolume = { ora_flashdb_vol }
CVMActivation = sw
)
CVMVolDg hapdb_index1_voldg (
CVMVolume = { ora_index1_vol }
CVMActivation = sw
)
CVMVolDg hapdb_redo1_voldg (
CVMVolume = { ora_redo1_vol }
CVMActivation = sw
)
CVMActivation = sw
)
CVMVolDg hapdb_redocopy_voldg (
CVMVolume = { ora_redocopy_vol }
CVMActivation = sw
)
CVMVolDg hapdb_system_voldg (
CVMVolume = { ora_system_vol }
CVMActivation = sw
)
CVMVolDg hapdb_temp_voldg (
CVMVolume = { ora_temp_vol }
CVMActivation = sw
)
Critical = 0
)
PrivNIC ora_priv (
Address @d1sol5 = "10.10.10.1"
Address @d1sol6 = "10.10.10.2"
NetMask = "255.255.255.0"
)

cssd requires crs_ora_mnt
cssd requires hapdb_archive_mnt
cssd requires hapdb_data1_mnt
cssd requires hapdb_flashdb_mnt
cssd requires hapdb_index1_mnt
cssd requires hapdb_redo1_mnt
cssd requires hapdb_redo2_mnt
cssd requires hapdb_redocopy_mnt
cssd requires hapdb_system_mnt
cssd requires hapdb_temp_mnt
cssd requires ora_priv

hapdb_archive_mnt requires hapdb_archive_voldg
hapdb_archive_mnt requires vxfsckd
hapdb_archive_voldg requires cvm_clus
hapdb_data1_mnt requires hapdb_data1_voldg
hapdb_data1_mnt requires vxfsckd
hapdb_data1_voldg requires cvm_clus
hapdb_flashdb_mnt requires hapdb_flashdb_voldg
hapdb_flashdb_mnt requires vxfsckd
hapdb_flashdb_voldg requires cvm_clus
hapdb_index1_mnt requires hapdb_index1_voldg
hapdb_index1_mnt requires vxfsckd
hapdb_index1_voldg requires cvm_clus
hapdb_redo1_mnt requires hapdb_redo1_voldg
hapdb_redo1_mnt requires vxfsckd
hapdb_redo1_voldg requires cvm_clus
hapdb_redo2_mnt requires vxfsckd
hapdb_redo2_voldg requires cvm_clus
hapdb_redocopy_mnt requires hapdb_redocopy_voldg
hapdb_redocopy_mnt requires vxfsckd
hapdb_redocopy_voldg requires cvm_clus
hapdb_system_mnt requires hapdb_system_voldg
hapdb_system_mnt requires vxfsckd
hapdb_system_voldg requires cvm_clus
hapdb_temp_mnt requires hapdb_temp_voldg
hapdb_temp_mnt requires vxfsckd
hapdb_temp_voldg requires cvm_clus

//
// group cvm
// {
// Application cssd
// {
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// {
// {
// }
// }
// }
// CFSMount hapdb_archive_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_archive_voldg
// {
// {
// }
// }
// }
// CFSMount hapdb_data1_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_data1_voldg
// {
// {
// }
// }
// }
// CFSMount hapdb_index1_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_index1_voldg
// {
// {
// }
// }
// }
// CFSMount hapdb_redo1_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_redo1_voldg
// {
// {
// }
// }
// }
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// {
// {
// }
// }
// }
// CFSMount hapdb_redocopy_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_redocopy_voldg
// {
// {
// }
// }
// }
// CFSMount hapdb_system_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_system_voldg
// {
// {
// }
// }
// }
// CFSMount hapdb_temp_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_temp_voldg
// {
// {
// }
// }
// }
// CFSMount hapdb_flashdb_mnt
// {
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// CVMVolDg hapdb_flashdb_voldg
// {
// {
// }
// }
// }
// PrivNIC ora_priv
// }
// }

main.cf with CVM and Oracle Service Groups and with the Oracle Agent
(make sure table required for detail monitoring is already created)
include "types.cf"
cluster d1sol-c (
UserNames = { admin = IhiAhcHeiDiiGqiChf }
UseFence = SCSI3
CounterInterval = 5
)
system d1sol5 (
)
system d1sol6 (
)
)
IP webip (
Device = ipge0
Address = "192.168.201.192"
NetMask = "255.255.255.0"
)
NIC csgnic (
Device = ipge0
)

NotifierMngr ntfr (
SmtpServer = "160.0.10.50"
SmtpRecipients = { "kappuk@lexus.elabor.com" = Warning }
)
VRTSWebApp VCSweb (
Critical = 0
AppName = vcs
TimeForOnline = 5
RestartLimit = 3
)


//
// {
// {
// IP webip
// {
// NIC csgnic
// }
// }
// NotifierMngr ntfr
// }
group cvm (
AutoFailOver = 0
Parallel = 1
)
Application cssd (
Critical = 0
StartProgram = "/opt/VRTSvcs/rac/bin/cssd-online"
StopProgram = "/opt/VRTSvcs/rac/bin/cssd-offline stopcrs"
CleanProgram = "/opt/VRTSvcs/rac/bin/cssd-clean"
MonitorProgram = "/opt/VRTSvcs/rac/bin/cssd-monitor"
)
)
Critical = 0
)
CFSfsckd vxfsckd (
)
CVMTransport = gab
CVMTimeout = 200
)
CVMActivation = sw
)
Critical = 0
)
PrivNIC ora_priv (
Address @d1sol5 = "10.10.10.1"
Address @d1sol6 = "10.10.10.2"
NetMask = "255.255.255.0"
)

cssd requires crs_ora_mnt
cssd requires ora_priv

//
// group cvm
// {
// Application cssd
// {
// {
// {
// {
// }
// }
// CFSfsckd vxfsckd
// {
// {
// {
// }
// }
// }
// }
// PrivNIC ora_priv
// }
// }
group hapdb_rac (
AutoFailOver = 0
Parallel = 1
)
CFSMount hapdb_archive_mnt (
Critical = 0
MountPoint = "/hapdb/archive"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_archive_vol"
)
CFSMount hapdb_data1_mnt (
Critical = 0
MountPoint = "/hapdb/data1"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_data1_vol"
)
CFSMount hapdb_flashdb_mnt (
Critical = 0
MountPoint = "/hapdb/flashdb"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_flashdb_vol"
)
CFSMount hapdb_index1_mnt (
Critical = 0
MountPoint = "/hapdb/index1"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_index1_vol"
)
Critical = 0
)
Critical = 0
)
CFSMount hapdb_redocopy_mnt (
Critical = 0
MountPoint = "/hapdb/redocopy"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_redocopy_vol"
)
CFSMount hapdb_system_mnt (
Critical = 0
MountPoint = "/hapdb/system"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_system_vol"
)
CFSMount hapdb_temp_mnt (
Critical = 0
MountPoint = "/hapdb/temp"
BlockDevice = "/dev/vx/dsk/ora_dg1/ora_temp_vol"
)
CVMVolDg hapdb_archive_voldg (
CVMVolume = { ora_archive_vol }
CVMActivation = sw
)
CVMVolDg hapdb_data1_voldg (
CVMVolume = { ora_data1_vol }
CVMActivation = sw
)
CVMVolDg hapdb_flashdb_voldg (
CVMVolume = { ora_flashdb_vol }
CVMActivation = sw
)
CVMVolDg hapdb_index1_voldg (
CVMVolume = { ora_index1_vol }
CVMActivation = sw
)
CVMActivation = sw
)
CVMActivation = sw
)
CVMVolDg hapdb_redocopy_voldg (
CVMVolume = { ora_redocopy_vol }
CVMActivation = sw
)
CVMVolDg hapdb_system_voldg (
CVMVolume = { ora_system_vol }
CVMActivation = sw
)
CVMVolDg hapdb_temp_voldg (
CVMVolume = { ora_temp_vol }
CVMActivation = sw
)
Oracle Ora-hapdb (
Sid @d1sol5 = hapdb1
Sid @d1sol6 = hapdb2
Owner = oracle
Home = "/oracle/product/10g/db_1"
Pfile @d1sol5 = "/opt/oracle/product/10g/db_1/dbs/inithapdb1.ora"
Pfile @d1sol6 = "/opt/oracle/product/10g/db_1/dbs/inithapdb2.ora"
EnvFile = "/opt/VRTSvcs/bin/Oracle/envfile"
MonScript = "/opt/VRTSvcs/bin/Oracle/SqlTest.pl"
DetailMonitor = 1
User = vcsmon
Pword = vcsmon
Table @d1sol5 = vcstable_d1sol5
Table @d1sol6 = vcstable_d1sol6
)
requires group cvm online local firm

Ora-hapdb requires hapdb_archive_mnt
Ora-hapdb requires hapdb_data1_mnt
Ora-hapdb requires hapdb_index1_mnt
Ora-hapdb requires hapdb_redo1_mnt
Ora-hapdb requires hapdb_redo2_mnt
Ora-hapdb requires hapdb_redocopy_mnt
Ora-hapdb requires hapdb_system_mnt
Ora-hapdb requires hapdb_temp_mnt
Ora-hapdb requires hapdb_flashdb_mnt
hapdb_archive_mnt requires hapdb_archive_voldg
hapdb_data1_mnt requires hapdb_data1_voldg
hapdb_index1_mnt requires hapdb_index1_voldg
hapdb_redocopy_mnt requires hapdb_redocopy_voldg
hapdb_system_mnt requires hapdb_system_voldg
hapdb_temp_mnt requires hapdb_temp_voldg
hapdb_flashdb_mnt requires hapdb_flashdb_voldg

//
// group hapdb_rac
// {
// Oracle Ora-hapdb
// {
// CFSMount hapdb_archive_mnt
// {
// CVMVolDg hapdb_archive_voldg
// CVMVolDg hapdb_data1_voldg
// CVMVolDg hapdb_index1_voldg
// CVMVolDg hapdb_redocopy_voldg
// CVMVolDg hapdb_system_voldg
// CVMVolDg hapdb_temp_voldg
// }
// CFSMount hapdb_data1_mnt
// CFSMount hapdb_index1_mnt
// CFSMount hapdb_redocopy_mnt
// CFSMount hapdb_system_mnt
// CFSMount hapdb_temp_mnt
// }
// CFSMount hapdb_flashdb_mnt
// CVMVolDg hapdb_flashdb_voldg
// }


RAC 10gr2 Guide

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

RAC 10gr2 Guide

Diunggah oleh

Hak Cipta:

Format Tersedia

eTEAM Oracle 10g RAC Installation Guide

ADP Confidential Page 1 of 139 Last modified by kappukuttan

Version History ............................................................................................................. 1

RAC Architecture Overview .......................................................................................... 4

Create Oracle User and Group .................................................................................................................. 7

Configure Secure Shell -SSH..................................................................................................................... 7

Configure rsh ............................................................................................................................................. 8

Set Environment variables ......................................................................................................................... 8

Configure Kernel Resources ...................................................................................................................... 9

Copying SFRAC Libraries to /opt/ORCLcluster/lib ................................................................................... 12

Creating $CRS_HOME on each node...................................................................................................... 12

Modify the CVM service group (main.cf) to add CRS resources............................................................... 13

Final Checkup before installing CRS and Oracle Binaries....................................................................... 14

ADP Confidential Page 2 of 139 Last modified by kappukuttan

Copying and Verifying IPC and VCSMM libraries..................................................................................... 52

Verifying ODM library............................................................................................................................... 52

Adding patch 10.2.0.2 to CRS.................................................................................................................. 53

Unzip the file ............................................................................................................................................ 53

Set the Environment Variables................................................................................................................. 53

Stop all Oracle processes ........................................................................................................................ 53

Adding patch 10.2.0.2 to Oracle Binaries ................................................................................................. 64

Post Installation ....................................................................................................................................... 76

Mandatory Patch for 10.2.0.2................................................................................................................... 78

ADP Confidential Page 3 of 139 Last modified by kappukuttan

RAC Architecture Overview

ADP Confidential Page 4 of 139 Last modified by kappukuttan

EMC Shared Storage and I/O Fencing

Private and Virtual IP’s

Hostname Host IP Virtual IP Private IP

ADP Confidential Page 5 of 139 Last modified by kappukuttan

ADP Confidential Page 6 of 139 Last modified by kappukuttan

UNIX Preinstallation Steps

Create Oracle User and Group

Create a password for user oracle

To verify the oracle account,

Verifying that the User nobody Exists

Configure Secure Shell -SSH

From each node, logon as oracle

Cut and paste the following line separately

ADP Confidential Page 7 of 139 Last modified by kappukuttan

$ chmod 644 ~/.ssh/authorized_keys

Set Environment variables

EDITOR=vi; export EDITOR

Configure Kernel Resources

Parameter Value Solaris 10 Resource Default Value

set semsys:seminfo_semmap 0x7631 Obsolete

ADP Confidential Page 9 of 139 Last modified by kappukuttan

On each node, logon as root

# projadd -U oracle -G dba -c "OracleProject" -p 201 user.oracle

To set UDP parameters to 65536 bytes, write a startup script in /etc/init.d

to view/verify the settings logon as oracle and run

ADP Confidential Page 10 of 139 Last modified by kappukuttan

Reboot the servers

Before the Installation

Copying SFRAC Libraries to /opt/ORCLcluster/lib

Creating $CRS_HOME on each node

Make directories for

Create volumes for CRS and VOTE disk on shared disk

On both nodes (d1sol5 and d1sol6)

On one node (d1sol5), logon as oracle

ADP Confidential Page 12 of 139 Last modified by kappukuttan

Modify the CVM service group (main.cf) to add CRS resources.

Refer to the Modified main.cf in the appendix