May 2008
ASM ON LINUX
Configuring Automatic Storage
Management On Linux.
There are 3 storage options to setup ASM on Linux:
1) Raw devices (10.1 and Onwards)
2) ASMLIB (10.1 and Onwards).
3) Block devices (10.2 and Onwards).
ASM ON LINUX
Creating Disks Partitions for ASM
ASM ON LINUX
Creating Disks Partitions for ASM
Identify the device name for the disks that you want to use.
# /sbin/fdisk -l
Depending on the type of disk, the device name can vary.
Oracle recommends that you create a single whole-disk partition on each
disk that you want to use.
Use either fdisk or parted to create a single whole-disk partition on the disk
devices that you want to use.
ASM ON LINUX
Creating Disks Partitions for ASM
Disk Type
Device Name
Format
Description
IDE disk
/dev/hdxn
/dev/hda1
SCSI disk
/dev/sdxn
/dev/sdb1
RAID disk
/dev/rd/cxdypz
/dev/ida/cxdypz
/dev/ida/c0d1
Multipath disk
(e.g. emcpower)
/dev/emcpowerxn
/dev/dm-xx
/dev/emcpowera1
ASM ON LINUX
1. Raw Devices.
1) Identify the existing partitions:
ASM ON LINUX
1. Raw Devices.
2) Open the /etc/sysconfig/rawdevices and add a raw device for
each partition that you created and you want to bound:
ASM ON LINUX
1. Raw Devices.
3) To bind the partitions to the raw devices, enter the following
command:
ASM ON LINUX
1. Raw Devices.
4) For the raw device that you created for ASM, set the owner,
group, and permissions on the device file:
ASM ON LINUX
1. Raw Devices.
5) Verify the raw disk through ASM:
10
ASM ON LINUX
2. Block Devices
1) Identify the existing partitions:
11
ASM ON LINUX
2. Block Devices
2) Use /sbin/partprobe diskpath on each node to update the
kernel partition table :
12
ASM ON LINUX
2. Block Devices
3) For the block device that you created for ASM, set the owner,
group, and permissions on the device file:
13
ASM ON LINUX
2. Block Devices
4) Update asm_diskstring parameter with the block devices
directory:
14
ASM ON LINUX
2. Block Devices
5) Create a diskgroup using block devices:
15
3. ASMLIB
INTRODUCTION
ASMLIB is a supported library for the ASM feature. The objective of
ASMLIB is to provide a more streamlined and efficient mechanism for
identifying and accessing block devices used by ASM diskgroups. This
API serves as alternative to the standard operating system interface. An
ASMLIB is a storage-management interface between Oracle Kernel and
disk storage.
The ASMLIB kernel driver is released under GNU General Public
License (GPL), and Oracle Corporation freely delivers an ASMLIB for
Linux Platforms. This library is provided to enable ASM I/O to Linux
disks without the limitations of the standard UNIX I/O API.
http://oss.oracle.com/
16
ASMLIB
BENEFITS
The main ASMLIB functions are grouped into three collections of
functions:
17
ASMLIB
Oracle Linux ASMLib Installation
You can download the Oracle ASMLIB software from OTN.
There are three packages for each platform. The two essential
packages are the oracleasmlib package, which provides the
actual ASM library, and the oracleasm support package, which
provides the utilities to configure and enable the ASM driver. The
remaining package provides the kernel driver for the ASMLib
(loaded into the OS kernel at system boot).
Example:
oracleasm-support-2.0.3-1.i386.rpm
oracleasmlib-2.0.2-1.i386.rpm
oracleasm-2.6.9-42.0.3.EL-2.0.3-1.i686.rpm
18
ASMLIB
Oracle Linux ASMLib Installation
After the ASMLIB software is installed, you need to make the
ASM driver available by executing the /etc/init.d/oracleasm
configure
command.
This
operation
creates
the
/dev/oracleasm mount point used by the ASMLIB to
communicate with ASM driver. When using RAC, instalaltion and
configuration must be completed on all nodes of the cluster.
# /etc/init.d/oracleasm configure
19
ASMLIB
Oracle Linux ASMLib Installation
In order to place a disk under ASM management, it must first marked to
prevent inadvertent use of incorrect disk by ASM. This is accomplished
by using the /etc/init.d/oracleasm createdisk command:
# /etc/init.d/oracleasm createdisk
With RAC, this operation needs to performed only on one node
because it is a shared-disk architecture. However, the other nodes in
the cluster need to ensure the disk is seen and valid. Therefore, the
other nodes in cluster need to execute the /etc/init.d/oracleasm
scandisks command.
# /etc/init.d/oracleasm scandisks
20
ASMLIB
Oracle Linux ASMLib Installation
After the disk are marked, the ASM initialization parameter can
be set to appropriate values.
ASM_DISKSTRING='ORCL:*'
On 10gR2 and onwards if the disks are not discovered using the
string
ORCL:*,
the
alternative
option
is
using
/dev/oracleasm/disks.
This could be set in parameter
asm_diskstring or using this path in the DDL statement when
creating a diskgroup or adding a disk. This is only a temporary
workaround until you fix the discovery issue and it does not
pretend to be final solution since with this action you are
bypassing the ASMLIB API.
Author: Esteban Bernal (Storage Team)
21
ASMLIB
Oracle Linux ASMLib Installation
Checklist.
I. Install the ASMLIB packages (on each node for RAC).
http://otn.oracle.com.com/tech/linux/asmlib
Install oracleasm-support, oracleasmlib and kernel-related packages.
22
ASMLIB
Oracle Linux ASMLib Installation
Oracle Linux ASMLIB Installation:
1.
2.
oracleasm-support-version.arch.rpm
oracleasm-kernel-version.arch.rpm
oracleasmlib-version.arch.rpm
Author: Esteban Bernal (Storage Team)
23
ASMLIB
ASMLIB Configuration
24
ASMLIB
Oracle Linux ASMLib Installation
3.
Install the proper packages for your platform. For example, if you
are using Red Hat Enterprise Linux AS 3.0 enterprise kernel,
enter a command similar to the following:
# rpm -i oracleasm-support-1.0.0-1.i386.rpm \
oracleasm-2.4.9-e-enterprise-1.0.0-1.i386.rpm \
oracleasmlib-1.0.0-1.i386.rpm
4.
25
ASMLIB
Oracle Linux ASMLib Installation
You will be prompted for the following:
The UID of the driver owner. This will be the UID for the
Oracle user.
The GID of the driver group. This will be the GID for the dba
group.
Whether the ASMLIB driver should be loaded at startup. The
corrects answer is YES.
26
ASMLIB
Oracle Linux ASMLib Installation
The script then completes the following tasks:
5.
27
ASMLIB
ASMLib Installation for Enterprise
Linux via ULN
The script then completes the following tasks:
1) Installing ASMLIB via ULN
# up2date -i oracleasm-support oracleasmlib oracleasm-`uname -r`
This will install the support tools, the library, and the kernel driver for
the currently running kernel.
2) Upgrading ASMLib via ULN
# up2date -i kernel-smp-2.6.9-100.EL oracleasm-2.6.9-100.ELsmp
28
ASMLIB
ASM Library Disk Creation
29
ASMLIB
ASM Library Disk Creation
30
ASMLIB
ASM Library Disk Creation
3. Using fdisk, create a single whole-disk partition on the device that you
want to use.
4. Mark a disk as an ASM disk:
# /etc/init.d/oracleasm createdisk DISK1 /dev/hda7
In this example, DISK1 is the tag or name that you want to assign to the
disk.
5. On RAC, to make the disk available on other cluster nodes, enter the
following command as root on each node:
# /etc/init.d/oracleasm scandisks
This command identifies all the shared disks attached to the node that
are marked as ASM disks.
31
ASMLIB
ASM Library Disk Configuration
Important oracleasm options:
enable / disable: Use the disable and enable options to charge the
behavior of the ASM library driver when the system starts. The enable
option causes the ASM library driver load when the system starts.
start / stop / restart: Use the start, stop and restart options to load
or unload the ASM library driver without restarting the system.
createdisk: Use this option to mark a disk for use with the ASM library
and name it.
deletedisk: Use the deletedisk option to unmark a named disk device.
Do not use this command to unmark disks that are being used by an
ASM diskgroup. You must drop the disk from the ASM diskgroup before
you unmark it.
32
ASMLIB
ASM Library Disk Configuration***
33
ASMLIB
ASMLIB Configuration
(Example Installation)
34
ASMLIB
ASMLIB Configuration
35
ASMLIB
ASMLIB Configuration
36
ASMLIB
ASMLIB Configuration
37
ASMLIB
ASMLIB Configuration
38
ASMLIB
ASMLIB Configuration
39
ASMLIB
ASMLIB Upgrade
i.
ii.
iii.
iv.
v.
40
ASMLIB
ASMLib Additional Information.
41
ASMLIB
Migrating Raw Devices to ASMLib
1) Disk should not be mapped thru raw devices, ASM cannot see the
disk twice.
# raw /dev/raw/raw12 0 0
# raw /dev/raw/raw13 0 0
2) Remove it from /etc/sysconfig/rawdevices
3) Set: asm_diskstring='ORCL:*
instead of: asm_diskstring=/dev/raw/*
4) Shutdown the ASM instance (Very important!).
5) Label the disk thru the oracleasm command:
# /etc/init.d/oracleasm force-renamedisk /dev/hda18 DISK1
# /etc/init.d/oracleasm force-renamedisk /dev/hda22 DISK2
Author: Esteban Bernal (Storage Team)
42
ASMLIB
Configuring ASMLib on Multipath Disks
Edit the /etc/sysconfig/oracleasm file to define the scanorder:
# ORACLEASM_ENABELED: 'true' means to load the driver on boot.
ORACLEASM_ENABLED=true
# ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
ORACLEASM_UID=oracle
# ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
ORACLEASM_GID=dba
# ORACLEASM_SCANBOOT: 'true' means fix disk perms on boot
ORACLEASM_SCANBOOT=true
# ORACLEASM_CLEARBOOT: 'true' means clean old disk perms on boot
ORACLEASM_CLEARBOOT=true
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER="emcpower sd" #emcpowera emcpowerb , scsi
#Or
ORACLEASM_SCANORDER="emcpower hda" #emcpowera emcpowerb, ATA
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
ORACLEASM_SCANEXCLUDE="sd raw"
43
ASMLIB
Device Persistence guaranteed thru
ASMLIB
1) Devices name has changed during reboot (e.g. /dev/sdg to
/dev/sdh) ASMLIB will discover the disk with no problems.
2) Oracle (ASM) does not need OS permissions to access the
real disks (e.g. /dev/sda1).
3) The disks must belong to root (Security reasons).
4) Using ASMLIB guarantees device persistence for ASM and
the device still belongs to Oracle.
44
ASMLIB
Source Code (Oracle ASMLib 2.0 Kernel
Driver Source)
Access the source code repository for this project in one of the
following ways:
http://oss.oracle.com/projects/oracleasm/source.html
45
ASMLIB
Troubleshooting
# df -ha
# lsmod
# cat /proc/filesystems
# /etc/init.d/oracleasm configure
$> cat /etc/*release
$> uname -a
# rpm -qa |grep oracleasm
$> ls -l /dev/oracleasm/disks
SQL> alter system set asm_diskstring = ORCL:* scope=both;
$> /usr/sbin/oracleasm-discover
$> kfod asm_diskstring='ORCL:*' disks=all
$> kfed read /dev/oracleasm/disks/VOl1
46
ASMLIB
Troubleshooting
47
ASMLIB
Troubleshooting
48
ASMLIB
Troubleshooting
49
ASMLIB
Troubleshooting
50
ASMLIB
Troubleshooting
51
ASMLIB
Troubleshooting
52
ASMLIB
Troubleshooting
53
ASMLIB
Troubleshooting
54
ASMLIB
Troubleshooting
55
ASMLIB
References
=)> http://www.oracle.com/technology/tech/linux/asmlib/index.html
=)> http://asm.us.oracle.com/
=)> Note.275315.1 Installing & Configuring Oracle ASMLib
=)> Note.309815.1 Configuring Oracle ASMLib on Multipath Disks
=)> Note.394959.1 Device Persistence and Oracle Linux ASMLib
=)> Note.394955.1 Migrating Raw Devices to ASMLib on Linux
=)> Note.269194.1 Troubleshooting ASMLIB on Linux
=)> Note.313387.1 How to know Which Disks Are Handled by ASMLib Kernel
Driver?
Author: Esteban Bernal (Storage Team)
56
ASMLIB
References
=)> Note.461718.1 How To Install ASMLIB API via the Unbreakable Linux
Network
=)> http://oss.oracle.com/projects/oracleasm-support/
=)> http://oss.oracle.com/projects/oracleasm/
57