Rac Installation by Nadir SB

Oracle 11gR2 2-Node RAC on VMware Workstation 8
This demo describes how to install Oracle 11gR2 (11.2.0.1) 2 Node (Linux) RAC on VMware Workstation-8. The
hardware/software resources used in this demo are given below.
Dell XPS-15 (Core i7) with Windows-7 (Home Premium) 64bit
RAM-16GB 8GB RAM is enough for 2 Node RAC in Virtual machine
50 GB free space for each RAC Node Just to install 2 Node RAC, 20 GB free space on each
node is enough
OEL-5.8 (64bit) https://edelivery.oracle.com/linux
VMware Workstation-8 VMware
Oracle 11.2.0.1 Grid Grid
Oracle 11.2.0.1 Database (Part-1 & 2) Database
Oracle ASMLib ASMLib
Note: OEL-5.5 or advanced Linux version can be used for 11gR2 RAC installation. One advantage of using OEL-5.8 is ,
all required RPMs for ASMLib are already installed in OS.Prior to start RAC installation it is assumed that VMware-8
and OEL-5.8 are installed in Laptop. Installation of LINUX and VMware are not covered in this document.
Create Linux Virtual Machine (Node-1)
Start VMware workstation. Go to File --> New Virtual Machine

TusarRout'sOracleBlog
http://gohappydba.wordpress.com/
Select "Custom" and proceed.
Click on Next.
Select option as appropriate and click Next.

select Linux with Version and click Next

Put Virtual Machine Name (RAC1) and Location to store VM files
Default No of Processor is 1, click on Next

Put Memory for Node-1 and click Next. I have 16GB RAM in my Laptop, so I put 3GB for Node-1. If you have 8GB RAM
in Laptop, you can use 2GB (2.5 GB is preferable) for each nodes.
select "Use Bridge Networking" and click on Next
Select the default "LSI Logic" as I am using SCSI disk in my Laptop.

5

Create new virtual disk
Select SCSI and proceed.

Put Disk size and select the below option. I used 50GB for each RAC node (for future use, to stage Grid/ RDBMS
software), but 20GB (10GB for Linux and 10GB for Grid+Database is enough for each RAC node just for installation
purpose.
Put Disk name (file name which will be used a disk). By default it takes Node_Name.vmdk (RAC1.vmdk)

Verify Installation details and click on Finish to start Linux OS installation.
After successful installation of Linux OS on VM Node-1, it displays as given below. Start RAC1 by clicking on Power on
button:
8

Check required RPMs and additional Setup for Oracle 11gR2 on OEL-5
Most of the RPMs are installed with OEL OS installation.
binutils-2.17.50.0.6
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.125
elfutils-libelf-devel-0.125
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.5-24
glibc-2.5-24 (32 bit)
glibc-common-2.5
glibc-devel-2.5
glibc-devel-2.5 (32 bit)
glibc-headers-2.5
ksh-20060214
libaio-0.3.106
libaio-0.3.106 (32 bit)
libaio-devel-0.3.106
libaio-devel-0.3.106 (32 bit)
libgcc-4.1.2
libgcc-4.1.2 (32 bit)
libstdc++-4.1.2
libstdc++-4.1.2 (32 bit)
libstdc++-devel 4.1.2
make-3.81
sysstat-7.0.2
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-devel-0.125
glibc-devel-2.5
libaio-devel-0.3.106
libaio-devel-0.3.106 (32 bit)
sysstat-7.0.2
unixODBC-2.2.11
unixODBC-2.2.11 (32 bit)
unixODBC-devel-2.2.11
unixODBC-devel-2.2.11 (32 bit)

Check RPM is installed or not with below command.
[root@rac1 ~]# rpm -qa | grep inutils-2.*

binutils-2.17.50.0.6-20.el5
[root@rac1 ~]#
To install RPM ,use the below command with root user. All the RPMs are available in Server directory under Linux
media-CD/DVD. Also many of the RPMs can be downloaded from http://rpm.pbone.net/
[root@rac1 ~]# rpm -ivh unixODBC-2.2.11-10.el5.x86_64.rpm
Check kernel parameters for Oracle as given below. Please note that any parameter value which is higher than below
mentioned value shouldn't be modified. If the value is less than as mentioned below, then needs to be modified. Edit
/etc/sysctl.conf as root user. Use /sbin/sysctl -p command to apply the new settings.
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 1054504960
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=1048586
Edit the /etc/pam.d/login file and add following line:
[root@rac1 ~]# cat /etc/pam.d/login |grep pam_limits.so

session required /lib/security/pam_limits.so
Edit the /etc/security/limits.conf file and add following lines:
oracle soft nproc 2047

oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
Change the setting of SELinux to permissive/disabled by editing the "/etc/selinux/config" file
[root@rac1 ~]# cat /etc/sysconfig/selinux |grep SELINUX=

SELINUX=disabled
[root@rac1 ~]# /usr/sbin/getenforce

Disabled
Disable the Linux firewall. First stop the services. To permanently disable the firewall, use the second command.
10

[root@rac1 ~]# service iptables stop
Flushing firewall rules: [ OK ]

Setting chains to policy ACCEPT: filter [ OK ]
Unloading iptables modules: [ OK ]
[root@rac1 ~]# chkconfig iptables off
Create oracle user, OS groups and directories where Oracle software will be installed.
[root@rac1 ~]# groupadd dba
[root@rac1 ~]# groupadd oinstall
[root@rac1 ~]# groupadd asmdba
[root@rac1 ~]# groupadd asmadmin
[root@rac1 ~]# useradd -g oinstall -G dba,asmdba,asmadmin oracle
Set password for user: oracle
[root@rac1 ~]# passwd oracle
Create directory where oracle software will be installed and set the ownership.
[root@rac1 ~]# mkdir -p /oracle/11.2.0/grid

[root@rac1 ~]# mkdir -p /oracle/11.2.0/product
[root@rac1 ~]# chown oracle:dba /oracle/11.2.0/
[root@rac1 ~]# chmod -R 775 /oracle/11.2.0
Login as the "oracle" user and add the following lines at the end of the "/home/oracle/.bash_profile" file.
# Oracle Settings
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
if [ $USER = "oracle" ]; then

if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
ASMLib Setup
The below RPMs are required for ASMLib. As we are using OEL-5.8 with kernel version: 2.6.32-300.10.1.el5uek, there
is no separate oracleasmlib* RPM available. The ASMLib kernel module is build with EUK kernel, so no need to install
that RPM to make ASM working without any issue. More information is available @ http://sethmiller.org/it/oracleasmlib-
not-necessary/
oracleasm-support-2.1.7-1.el5.x86_64.rpm
oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm
oracleasmlib-2.0.4-1.el5.x86_64.rpm --> Not required in OEL-5.8 and higher OS version
11

[root@rac1 ~]# rpm -qa |grep oracleasm*
oracleasm-support-2.1.8-1.el5
oracleasm-2.6.18-348.12.1.el5-2.0.5-1.el5
[root@rac1 ~]# rpm -qa |grep oracleasmlib*

[root@rac1 ~]#
Configure the ASMLib driver (to be owned by oracle and the dba group). It needs to be loaded on every reboot.
[root@rac1 ~]# /etc/init.d/oracleasm configure

Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface [oracle]:

Default group to own the driver interface [dba]:
Start Oracle ASM library driver on boot (y/n) [y]:
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]
Start the ASMLib driver.
[root@rac1 ~]# /etc/init.d/oracleasm start

Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]
Create shared disk to be used by ASM ( Disk groups)
Shutdown RAC1. Right click on RAC1 --> Settings --> Click on "Add"
12

Select Hard Disk and click on Next
Select "Create a new virtual disk"
13

Select SCSI, Independent and Persistent options.
Put the size of Disk. I put 10GB for doing some testing on Disk group. Otherwise 5GB is enough for demo purpose.
14

Put a Disk file name and place the file in a separate location (C:\TUSAR\VMWARE\RACSHARE) because this disk will
be shared with RAC2 node. Click Finish to create the disk. It takes few minutes to create the disk.
New Virtual Disk will be displayed as given below. Click on "Advanced"
15

Select device node as SCSI 1:0 from drop down menu (which means SCSI controller 1, device 0) and press OK.
Now we have to modify VMware hardware profile file (vmx) for RAC1 to make this disk as shared (in cluster).These
changes force VM not try to buffer reads and writes to the disk directly.Take a backup of the below file before modifying.
16

We need to put the below lines in rac1.vmx file and save it. If any of the parameter is already present , don't put it twice
(it will throw error during VM startup)
disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
scsi1.sharedBus = "virtual"
The new disk (ASM_DISK1) has the following entries. No need to change anything with below entries.
scsi1:0.present = "TRUE"
scsi1:0.fileName = "C:\TUSAR\VMWARE\RACSHARE\ASM_DISK1"
scsi1:0.writeThrough = "TRUE"
scsi1:0.mode = "independent-persistent"
scsi1:0.deviceType = "disk"
scsi1:0.redo = ""
Start RAC1 VM to format new disk. Use "fdisk" command to partition new disk (/dev/sdb).
[root@rac1 ~]# fdisk /dev/sdb
Command (m for help): u

Changing display/entry units to sectors
Command (m for help): n

Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First sector (63-10485759, default 63): 2048
17

Last sector or +size or +sizeM or +sizeK (2048-10485759, default 10485759):

Using default value 10485759
Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.

Syncing disks.
After partitioning new disk (/dev/sdb1), you will find an entry like below with command "fdisk -l"
Disk /dev/sdb: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System

/dev/sdb1 1 1306 10484736 83 Linux
Now we can use this disk for ASM. Use the below command to create ASM disk.
[root@rac1 ~]# oracleasm createdisk DATA /dev/sdb1

Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# oracleasm scandisks

Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Verify the ASM disks.
[root@rac1 ~]# oracleasm listdisks

DATA
Shutdown RAC1 node to configure Network interfaces. Taking backup of RAC1 is a good idea here as we are done with
all setups except Network Interface.
Configure Network Interfaces
The Oracle RAC requires at least two network connections between cluster nodes. One network will be the public IP and
the second will be a private IP reserved for inter-cluster traffic.
In VMware Workstation, Go to Edit->Virtual Network Editor.
18

This allows us to create new virtual networks that allow our guest VMs to talk amongst themselves, to the host computer
and/or to the outside world. We are going to create two virtual networks (VMnet2 and VMnet3) by clicking on Add
Network. For both VMnet2 and VMnet3 DHCP is enabled.
VMnet2 will be a host-only network allowing communication to other VMs and the host, but not to the outside world. We
will assign a IP of 10.10.1.0 to VMnet2, and a Subnet mask of 255.255.255.0
VMnet3 will also be a host-only network allowing communication to other VMs and the host, but not to the outside world.
We will assign a Subnet IP of 10.10.2.0 to VMnet3, and a Subnet mask of 255.255.255.0
Now we have our two networks created in VMware, we will add two new NICs to our VM.
Right click on the RAC1 in the VMware Workstation menu, and select Settings. This will bring up the Virtual Machine
Settings panel. Now click Add. The Add Hardware Wizard allows us to select Network Adapter
19

On the Network Adapter Type menu, we will select the Custom: Specific virtual network radio button, and use the drop
down to select VMNet2(Host-only). Click Finish to create the NIC.
Now repeat the above steps to add another Network Adapter, this time using VMNet3.
20

Now repeat the above steps to add another Network Adapter, this time using VMNet3.The final VM hardware
configuration should look like this:
21

Now we can clone our VM (RAC1) which will be RAC2. From this point forward we will have two machines i.e. RAC1
and RAC2.As RAC1 is now shutdown, go to VM --> Manage --> Clone.
Select Create a full clone, put VM name: RAC2 and Location to store VM files. Click Finish to start clone.
22

After successful clone, we have two VM i.e. RAC1 and RAC2. Now start both VM and login as root.Now we need to
configure Network IP add, hostname.
Log in to Node-1 (RAC1) as root and select System->Administration->Network. There will be three network interfaces
(devices) as given below.
eth0 : The bridged network adapter that connects us to the outside world (no configuration required)
eth1 : It is using VMnet2, the network we intend to be our public RAC network
eth2 : It is using VMnet3 which is what we plan to use for private cluster traffic
Select the eth1 adapter and click the edit button to bring up the Ethernet Device control panel. We will select Statically
set IP addresses to assign a static IP address to this NIC. We will assign IP address 10.10.1.10. Make sure that
Activate device when computer starts remains enabled.
Now repeat the above steps for eth2, this time statically assigning the IP address 10.10.2.10
23

Make sure you save your chances before closing the Network Configuration editor window.
24

[root@rac1 ~]# /etc/init.d/network restart

Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0... done.
[ OK ]
Bringing up interface eth1: [ OK ]
Bringing up interface eth2: [ OK ]
Check with "ifconfig -a" command in RAC1
[root@rac1 ~]# ifconfig -a

eth0 Link encap:Ethernet HWaddr 00:0C:29:74:C4:4D
inet addr:192.168.0.103 Bcast:192.168.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:120 errors:0 dropped:0 overruns:0 frame:0
TX packets:137 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:30398 (29.6 KiB) TX bytes:29031 (28.3 KiB)
eth1 Link encap:Ethernet HWaddr 00:0C:29:74:C4:57

eth2 Link encap:Ethernet HWaddr 00:0C:29:74:C4:61

RX bytes:5275335 (5.0 MiB) TX bytes:2998636 (2.8 MiB)
lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
Now log in to RAC2 and follow the above steps for IP Add configuration. Once this is done, we have the below n/w
interfaces ready with following IP Add.
N/W Interfaces RAC1 RAC2

eth1 10.10.1.10 10.10.1.20
eth2 10.10.2.10 10.10.2.20
25

Check hostname in the file /etc/sysconfig/network with root account, edit this file and change the host name on each
machine. If we change hostname in this file, we need to reboot VM. My domain name is "miracle.com" (Oracle's
Miracle)
RAC1:
[root@rac1 ~]# cat /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=rac1.miracle.com
RAC2:
[root@rac2 ~]# cat /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=rac2.miracle.com
Check with "ifconfig -a" command in RAC2
[root@rac2 ~]# ifconfig -a

eth0 Link encap:Ethernet HWaddr 00:0C:29:04:BC:50
eth1 Link encap:Ethernet HWaddr 00:0C:29:04:BC:5A

eth2 Link encap:Ethernet HWaddr 00:0C:29:04:BC:64

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
It's a good idea to bounce both VM (RAC1 and RAC2) and check if everything back with above as expected. Please
note, now onwards whatever changes we do , it mean for both the nodes (RAC1 and RAC2) accordingly.
26

Configure DNS for using SCAN
Grid instiller demands a VIP to be used as SCAN VIP (for SCAN listener). It doesn't allow to proceed with installation
without providing the VIP (SCAN). As a work around, we can place VIP (which must be free and is not used in the
network) in /etc/hosts files in both the nodes to make the installer continue with Grid installation. Later you can use local
listener instead of SCAN (disable) and remove these entries from /etc/hosts.
In our demo, we are going to use SCAN listener (which is an added benefit to handle user connections irrespective of
number of nodes present in a cluster. For using SCAN, we need to configure DNS to resolve VIP addresses to be used
by SCAN. Oracle recommends three VIP should be enough for SCAN to handle multiple number of database
connection. Here we are using two VIP for SCAN (just for demo purpose, later we can add one more VIP to SCAN).
We are using SCAN VIP, still I suggest to place SCAN VIP entries in /etc/hosts during installation, later we can remove
these SCAN entries.
[root@rac1 ~]# cat /etc/hosts

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
10.10.1.10 rac1.miracle.com rac1

10.10.1.11 rac1-vip.miracle.com rac1-vip

10.10.2.10 rac1-priv.miracle.com rac1-priv

10.10.1.12 rac-scan.miracle.com rac-scan

We will be using these VIPs for SCAN and these need to be configured in DNS.
[root@rac2 ~]# cat /etc/hosts

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6




N/W Interfaces RAC1 RAC2
27

Public IP 10.10.1.10 10.10.1.20

Private IP 10.10.2.10 10.10.2.20
VIP IP 10.10.1.11 10.10.1.21
SCAN IP 10.10.1.12 10.10.1.22
The below RPM needs to be installed to start DNS in Linux Server. It is already available in OEL-5.8.
[root@rac1 ~]# rpm -qa |grep bind-*

bind-9.3.6-20.P1.el5
For DNS setup, we need to create file /etc/named.conf with below entries. If the file is already present, then before
editing take a backup.
[root@rac1 ~]# cat /etc/named.conf

options {
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
forwarders { 192.168.0.1; };
};
include "/etc/rndc.key";
zone "miracle.com" IN {
type master;
file "miracle.zone";
allow-update { none; };
};
In real scenario, DNS server holds this file. In our case you can place this file in both RAC1 and RAC2 (with same
entries) as given below. It helps to operate both the nodes as DSN server.
[root@rac2 ~]# cat /etc/named.conf

options {
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
forwarders { 192.168.0.1; };
};
include "/etc/rndc.key";
zone "miracle.com" IN {
type master;
file "miracle.zone";
allow-update { none; };
};
Change forwarders IP as per your Tertiary DNS (In my case it is 192.168.0.1). Go to the below location to check what is
the Tertiary DNS. Generally it is 192.168.1.1 in most of the home networks..
28

System->Administration->Network
Create file /var/named/miracle.zone with following entries in both nodes.
RAC1:
[root@rac1 ~]# cat /var/named/miracle.zone

$TTL 86400
@ IN SOA miracle.com. miracle.com.(
42 ; serial (d. adams)
3H ; refresh
15M ; retry
1W ; expiry
1D ) ; minimum
miracle.com. IN NS 10.10.1.20
localhost IN A 127.0.0.1
rac1.miracle.com. IN A 10.10.1.10
rac1-vip.miracle.com. IN A 10.10.1.11
rac-scan.miracle.com. IN A 10.10.1.12
RAC2:
29

[root@rac2 ~]# cat /var/named/miracle.zone

$TTL 86400
@ IN SOA miracle.com. miracle.com.(
42 ; serial (d. adams)
3H ; refresh
15M ; retry
1W ; expiry
1D ) ; minimum
miracle.com. IN NS 10.10.1.20
localhost IN A 127.0.0.1
RAC1: Put the below entries in /etc/resolve.conf file
[root@rac1 ~]# cat /etc/resolv.conf

; generated by /sbin/dhclient-script
nameserver 10.10.1.10 # rac1 DNS server
nameserver 192.168.0.1 # Primary DNS in the domain
search miracle.com # Local Domain
RAC2:
[root@rac2 ~]# cat /etc/resolv.conf

; generated by /sbin/dhclient-script
nameserver 192.168.0.1 # Primary DNS in the domain
search miracle.com # Local Domain
Now we have to modify Ethernet adapter files to avoid any overwrite of /etc/resolve.conf file.
[root@rac1 network-scripts]# pwd

/etc/sysconfig/network-scripts
[root@rac1 network-scripts]# ls -lrt ifcfg-eth*
-rw-r--r-- 3 root root 216 Jul 27 10:45 ifcfg-eth2
Place "PEERDNS=no" in both ifcfg-eth1 and ifcfg-eth2 files (both nodes) to avoid any overwrite of /etc/resolve.conf
[root@rac1 network-scripts]# pwd

/etc/sysconfig/network-scripts
[root@rac1 network-scripts]# cat ifcfg-eth1
# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
30

DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
HWADDR=00:0c:29:74:c4:57
NETMASK=255.255.255.0
IPADDR=10.10.1.10
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=no
[root@rac1 network-scripts]# cat ifcfg-eth2

# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth2
BOOTPROTO=none
ONBOOT=yes
HWADDR=00:0c:29:74:c4:61
NETMASK=255.255.255.0
IPADDR=10.10.2.10
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=no
Now we can start DNS service on both nodes.
[root@rac1 ~]# service named start

Starting named: [ OK ]
[root@rac2 ~]# service named start

Starting named: [ OK ]
We also need to set the DNS service to auto-start on reboot (both nodes)
[root@rac1~]# chkconfig named on
[root@rac2~]# chkconfig named on
Now we can do look up the SCAN IP addresses. Now check whether rac-scan can resolve the IP add from DNS with
round-robin format (on both nodes).
[root@rac1 network-scripts]# nslookup rac-scan

Server: 10.10.1.10
Address: 10.10.1.10#53
Name: rac-scan.miracle.com
Address: 10.10.1.22 --> 1st SCAN VIP
Address: 10.10.1.12 --> 2nd SCAN VIP
31


Server: 10.10.1.10
Address: 10.10.1.10#53

Server: 10.10.1.10
Address: 10.10.1.10#53
Also check nslookup for other RAC IP add (on both nodes)
[root@rac1 network-scripts]# nslookup rac1

Server: 10.10.1.10
Address: 10.10.1.10#53
Name: rac1.miracle.com
Address: 10.10.1.10
[root@rac1 network-scripts]# nslookup rac1-vip

Server: 10.10.1.10
Address: 10.10.1.10#53
Name: rac1-vip.miracle.com
Address: 10.10.1.11
[root@rac1 network-scripts]# nslookup rac2

Server: 10.10.1.10
Address: 10.10.1.10#53
Name: rac2.miracle.com
Address: 10.10.1.20
[root@rac1 network-scripts]# nslookup rac2-vip

Server: 10.10.1.10
Address: 10.10.1.10#53
Name: rac2-vip.miracle.com
Address: 10.10.1.21
It confirms that DNS is working fine, so SCAN can used as expected.
Setup NTP service to synchronize time between our cluster nodes
Check if NTP service is running.
32

[root@rac1 ~]# pgrep ntpd

4489
If it is running we need to stop for few adjustments (both nodes)
[root@rac1 ~]# service ntpd stop
Backup /etc/ntp.conf and modify as given below(both nodes)
RAC1:
[root@rac1 ~]# cat /etc/ntp.conf

# --- GENERAL CONFIGURATION ---
server rac1
server rac2
fudge rac1 stratum 10
# Drift file.
driftfile /etc/ntp/drift
RAC2:
[root@rac2 ~]# cat /etc/ntp.conf

# --- GENERAL CONFIGURATION ---
server rac1
server rac2
fudge rac2 stratum 10
# Drift file.
driftfile /etc/ntp/drift
change the ownership of the /etc/ntp directory:
[root@rac1 etc]# chown ntp:ntp /etc/ntp
[root@rac1 etc]# ls -lrt |grep drw |grep ntp

drwxr-xr-x 2 ntp ntp 4096 Jul 9 20:24 ntp
Add the slewing option (it prevents the NTP daemon from resetting the clock in the event of a gap occurring).Modify
/etc/sysconfig/ntpd file and add the -x option.
[root@rac1 etc]# cat /etc/sysconfig/ntpd

# Drop root to id 'ntp:ntp' by default.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"
# Set to 'yes' to sync hw clock after successful ntpdate

SYNC_HWCLOCK=no
# Additional options for ntpdate
33

NTPDATE_OPTIONS=""
Restart NTP service.
[root@rac1 etc]# service ntpd start

Starting ntpd: [ OK ]
Check NTP services on both nodes.
[root@rac1 ~]# ntpq -p

remote refid st t when poll reach delay offset jitter
==============================================================================
rac1.miracle.co .INIT. 16 u - 1024 0 0.000 0.000 0.000
rac2.miracle.co .INIT. 16 u 62 1024 0 0.000 0.000 0.000
[root@rac2 ~]# ntpq -p

remote refid st t when poll reach delay offset jitter
==============================================================================
rac1.miracle.co .INIT. 16 u 78 1024 0 0.000 0.000 0.000
rac2.miracle.co .INIT. 16 u - 1024 0 0.000 0.000 0.000
Make NTP service auto-start on reboot (both nodes)
[root@rac1 ~]# chkconfig ntpd on
[root@rac2 ~]# chkconfig ntpd on
Shared disk setup in Node-2 (RAC2)
We have already created the shared disk and configured that disk to be used by ASM in RAC1. After cloning of RAC2
(from RAC1), this shared disk must be visible to RAC2. In case the partition table is not updated in RAC2, then we need
to do partition the same disk in RAC2 to update the partition table. Check partition of new disk (/dev/sdb1) is visible in
RAC2 with command "fdisk -l".
Disk /dev/sdb: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System

/dev/sdb1 1 1306 10484736 83 Linux
If it is not visible then we have to do partition of /dev/sdb once again in RAC2.
[root@rac2 ~]# fdisk /dev/sdb
34

Command (m for help): u

Changing display/entry units to sectors
Command (m for help): n

Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First sector (63-10485759, default 63): 2048
Last sector or +size or +sizeM or +sizeK (2048-10485759, default 10485759):
Using default value 10485759
Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.

Syncing disks.
Please note if and only if the partition is not reflected on RAC2, then do the partition as given above. In general it
reflects on RAC2 as it is being copied from RAC1 (clone).
Verify the ASM disks in RAC2.
[root@rac2 ~]# oracleasm scandisks

Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@rac2 ~]# oracleasm listdisks

DATA
Now the shared disk is visible to both RAC1 and RAC2.
Install the Oracle Grid Infrastructure software
Log in with oracle user, copy downloaded software (Grid and Database) in Node-1 (RAC1) and unzip with user "oracle"
[oracle@rac1 oracle]$ ls -lrt *.zip

-rwxr-xr-x 1 oracle oinstall 1052897657 Jul 10 19:24 linux.x64_11gR2_grid.zip
-rwxr-xr-x 1 oracle oinstall 1239269270 Jul 12 11:27 linux.x64_11gR2_database_1of2.zip
[oracle@rac1 oracle]$ ls -lrt grid
total 40
-rwxrwxr-x 1 oracle oinstall 3795 Jan 29 2009 runcluvfy.sh
-rwxr-xr-x 1 oracle oinstall 3227 Aug 15 2009 runInstaller
drwxrwxr-x 2 oracle oinstall 4096 Aug 15 2009 rpm
drwxr-xr-x 4 oracle oinstall 4096 Aug 15 2009 install
drwxr-xr-x 14 oracle oinstall 4096 Aug 15 2009 stage
drwxrwxr-x 2 oracle oinstall 4096 Aug 15 2009 response
drwxrwxr-x 2 oracle oinstall 4096 Aug 15 2009 sshsetup
drwxr-xr-x 9 oracle oinstall 4096 Aug 16 2009 doc
35

-rw-r--r-- 1 oracle oinstall 4228 Aug 18 2009 welcome.html
Before starting with Grid installation, we need to setup SSH on both the nodes. In 10g CRS installation, we use to do
SSH setup manually in all the nodes, but in 11gR2 Grid, the following script will setup SSH on both nodes with running
from one node. Log in to RAC1 as oracle user and open a command terminal, run the below script.
RAC1:
[oracle@rac1 ~]$ cd /oracle/grid/sshsetup
[oracle@rac1 sshsetup]$ ./sshUserSetup.sh \

-user oracle -hosts "rac1 rac2" \
-noPromptPassphrase -confirm -advanced
It will prompt twice for oracle password for each node (RAC1, RAC2) and will setup SSH on both nodes. After successful
run of this script, check ssh on both nodes.
RAC1:
[oracle@rac1 ~]$ ssh -x -l oracle rac1 date

Sun Jul 27 15:50:39 IST 2014
Sun Jul 27 15:50:43 IST 2014
[oracle@rac1 ~]$ ssh -x -l oracle rac1-priv date
Sun Jul 27 15:50:45 IST 2014
Sun Jul 27 15:50:48 IST 2014
RAC2:

Sun Jul 27 15:53:16 IST 2014
Sun Jul 27 15:53:21 IST 2014
Sun Jul 27 15:53:26 IST 2014
Sun Jul 27 15:53:30 IST 2014
Now run runcluvfy.sh in RAC1 and confirm all the settings are fine before installation of Grid.
[oracle@rac1 oracle]$ cd /oracle/grid

[oracle@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2
The output of above script is attached here.
36

Before Grid installation check the Display setting in terminal. VNC, Hummingbird or any X-Windows system can be used
to access the server terminal to start the runinstaller. Run "xclock" in terminal and check the Display setting is correct or
not. If you find a clock as given below, that it confirms that Display setting is fine.
It looks all the setup is fine, so we can proceed with Grid installation.
[oracle@rac1 grid]$ ./runInstaller &
It will start the graphical installer as given below.
37

Cluster Name: rac-cluster
Scan Name: rac-scan
SCAN Port: 1561
38

Click Add to include second node hostname, vip.
39

40

Change Discovery Path: /dev/oracleasm/disks
41

Put account passwords (SYS, SYSASM).
42

43

44

45

Run orainstRoot.sh in RAC1, after finish up then run orainstRoot.sh in RAC2. Then run root.sh in RAC1 and after finish
up, run root.sh in RAC2.Please note, run all the scripts one by one and after successful run of all scripts press OK.
46

The output of these scripts are attached here. Verify the output and confirm that there is no error.
The Grid install is now complete.
Install Oracle Database 11gR2 software

47

Unzip Database software with oracle user in RAC1.
[oracle@rac1 oracle]$ pwd

/oracle
[oracle@rac1 oracle]$ ls -lrt *database*zip
[oracle@rac1 oracle]$ cd database/

[oracle@rac1 database]$ ls -lrt
total 36
-rwxr-xr-x 1 oracle oinstall 3226 Aug 15 2009 runInstaller
drwxr-xr-x 2 oracle oinstall 4096 Aug 15 2009 rpm
drwxr-xr-x 4 oracle oinstall 4096 Aug 15 2009 install
drwxr-xr-x 14 oracle oinstall 4096 Aug 15 2009 stage
drwxrwxr-x 2 oracle oinstall 4096 Aug 15 2009 sshsetup
drwxrwxr-x 2 oracle oinstall 4096 Aug 15 2009 response
drwxr-xr-x 12 oracle oinstall 4096 Aug 17 2009 doc
-rw-r--r-- 1 oracle oinstall 5402 Aug 18 2009 welcome.html
Run runinstaller to start Database installation.

[oracle@rac1 database]$ ./runInstaller &
48

49

50

51

52

53

54

55

56

57

58

59

run root.sh in RAC1 and RAC2 one by one.
60

Oracle RAC 11gR2 Database is now installed in two nodes.
Check status of Grid services in both nodes.
[root@rac1 ~]# /oracle/11.2.0/grid/bin/crs_stat -t -v

Name Type R/RA F/FT Target State Host
----------------------------------------------------------------------
ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1
ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1
ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE rac1
ora.gsd ora.gsd.type 0/5 0/ ONLINE ONLINE rac1
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type 0/5 0/0 ONLINE ONLINE rac1
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1
ora.prod.db ora....se.type 0/2 0/1 ONLINE ONLINE rac1
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1
ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE rac1
ora.rac1.gsd application 0/5 0/0 ONLINE ONLINE rac1
ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1
[root@rac2 ~]# /oracle/11.2.0/grid/bin/crs_stat -t -v

61

Name Type R/RA F/FT Target State Host

----------------------------------------------------------------------
ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1
ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1
ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE rac1
ora.gsd ora.gsd.type 0/5 0/ ONLINE ONLINE rac1
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type 0/5 0/0 ONLINE ONLINE rac1
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1
ora.prod.db ora....se.type 0/2 0/1 ONLINE ONLINE rac1
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1
ISSUE: After installation of Oracle RAC 11gR2 in two nodes, I found there is only one SCAN Listener is running in one
node. I have configured two SCAN VIP with DNS during Grid installation. Even after successful Grid installation, only
one SCAN listener was running. The below configuration enables two SCAN Listener with respect to two SCAN VIP
(DNA). Solution for this issue is described in the following document along with SCAN Listener and Database connection
details.
RAC11gR2 SCAN Listener Issue and Solution
Hope this helps.....
Thanks,
Tusar
Reference:
http://gruffdba.wordpress.com/2012/10/26/oracle-11gr2-2-node-rac-on-vmware-workstation-8-
introduction/
http://www.oracle-base.com/articles/11g/oracle-db-11gr2-rac-installation-on-ol5-using-vmware-
server-2.php
62

Rac Installation by Nadir SB

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Rac Installation by Nadir SB

Diunggah oleh

Hak Cipta:

Format Tersedia

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Dell XPS-15 (Core i7) with Windows-7 (Home Premium) 64bit

RAM-16GB 8GB RAM is enough for 2 Node RAC in Virtual machine

OEL-5.8 (64bit) https://edelivery.oracle.com/linux

VMware Workstation-8 VMware

Oracle 11.2.0.1 Grid Grid

Oracle 11.2.0.1 Database (Part-1 & 2) Database

Oracle ASMLib ASMLib

Create Linux Virtual Machine (Node-1)

Start VMware workstation. Go to File --> New Virtual Machine

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Select "Custom" and proceed.

Select option as appropriate and click Next.

Oracle 11gR2 2-Node RAC on VMware Workstation 8

select Linux with Version and click Next

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Put Virtual Machine Name (RAC1) and Location to store VM files

Default No of Processor is 1, click on Next

Oracle 11gR2 2-Node RAC on VMware Workstation 8

select "Use Bridge Networking" and click on Next

Select the default "LSI Logic" as I am using SCSI disk in my Laptop.

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Create new virtual disk

Select SCSI and proceed.

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Verify Installation details and click on Finish to start Linux OS installation.

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Most of the RPMs are installed with OEL OS installation.

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Check RPM is installed or not with below command.

[root@rac1 ~]# rpm -qa | grep inutils-2.*

[root@rac1 ~]# rpm -ivh unixODBC-2.2.11-10.el5.x86_64.rpm

Edit the /etc/pam.d/login file and add following line:

[root@rac1 ~]# cat /etc/pam.d/login |grep pam_limits.so

Edit the /etc/security/limits.conf file and add following lines:

oracle soft nproc 2047

Change the setting of SELinux to permissive/disabled by editing the "/etc/selinux/config" file

[root@rac1 ~]# cat /etc/sysconfig/selinux |grep SELINUX=

[root@rac1 ~]# /usr/sbin/getenforce

Oracle 11gR2 2-Node RAC on VMware Workstation 8

[root@rac1 ~]# service iptables stop

Flushing firewall rules: [ OK ]

[root@rac1 ~]# chkconfig iptables off

Set password for user: oracle

[root@rac1 ~]# passwd oracle

[root@rac1 ~]# mkdir -p /oracle/11.2.0/grid

if [ $USER = "oracle" ]; then

Oracle 11gR2 2-Node RAC on VMware Workstation 8

[root@rac1 ~]# rpm -qa |grep oracleasm*

[root@rac1 ~]# rpm -qa |grep oracleasmlib*

[root@rac1 ~]# /etc/init.d/oracleasm configure

Default user to own the driver interface [oracle]:

Start the ASMLib driver.

[root@rac1 ~]# /etc/init.d/oracleasm start

Create shared disk to be used by ASM ( Disk groups)

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Select Hard Disk and click on Next

Select "Create a new virtual disk"

Oracle 11gR2 2-Node RAC on VMware Workstation 8

Select SCSI, Independent and Persistent options.

Oracle 11gR2 2-Node RAC on VMware Workstation 8