Configuring and Managing Fence Devices for the High Availability Add-On
Configuring and Managing Fence Devices for the High Availability Add-On
.
Legal Notice
Co pyright 20 15 Red Hat, Inc. and o thers.
This do cument is licensed by Red Hat under the Creative Co mmo ns Attributio n-ShareAlike 3.0
Unpo rted License. If yo u distribute this do cument, o r a mo dified versio n o f it, yo u must pro vide
attributio n to Red Hat, Inc. and pro vide a link to the o riginal. If the do cument is mo dified, all Red
Hat trademarks must be remo ved.
Red Hat, as the licenso r o f this do cument, waives the right to enfo rce, and agrees no t to assert,
Sectio n 4 d o f CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shado wman lo go , JBo ss, MetaMatrix, Fedo ra, the Infinity
Lo go , and RHCE are trademarks o f Red Hat, Inc., registered in the United States and o ther
co untries.
Linux is the registered trademark o f Linus To rvalds in the United States and o ther co untries.
Java is a registered trademark o f Oracle and/o r its affiliates.
XFS is a trademark o f Silico n Graphics Internatio nal Co rp. o r its subsidiaries in the United
States and/o r o ther co untries.
MySQL is a registered trademark o f MySQL AB in the United States, the Euro pean Unio n and
o ther co untries.
No de.js is an o fficial trademark o f Jo yent. Red Hat So ftware Co llectio ns is no t fo rmally
related to o r endo rsed by the o fficial Jo yent No de.js o pen so urce o r co mmercial pro ject.
The OpenStack Wo rd Mark and OpenStack Lo go are either registered trademarks/service
marks o r trademarks/service marks o f the OpenStack Fo undatio n, in the United States and o ther
co untries and are used with the OpenStack Fo undatio n's permissio n. We are no t affiliated with,
endo rsed o r spo nso red by the OpenStack Fo undatio n, o r the OpenStack co mmunity.
All o ther trademarks are the pro perty o f their respective o wners.
Abstract
Fencing is the disco nnectio n o f a no de fro m the cluster's shared sto rage. Fencing cuts o ff I/O
fro m shared sto rage, thus ensuring data integrity. This manual do cuments the co nfiguratio n o f
fencing o n clustered systems using High Availability Add-On and details the co nfiguratio n o f
suppo rted fence devices.
T able of Contents
. .reface
P
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3. . . . . . . . . .
1. Do c ument Co nventio ns
3
1.1. Typ o g rap hic Co nventio ns
3
1.2. Pull-q uo te Co nventio ns
4
1.3. No tes and Warning s
5
2 . G etting Help and G iving Feed b ac k
5
2 .1. Do Yo u Need Help ?
5
2 .2. We Need Feed b ac k
6
. .hapt
C
. . . .er
. .1. .. Fencing
. . . . . . . .Pre. . . Configurat
. . . . . . . . . .ion
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .
1.1. Co nfig uring ACPI Fo r Us e with Integ rated Fenc e Devic es
7
1.1.1. Dis ab ling ACPI So ft-O ff with c hkc o nfig Manag ement
8
1.1.2. Dis ab ling ACPI So ft-O ff with the BIO S
8
1.1.3. Dis ab ling ACPI Co mp letely in the g rub .c o nf File
1.2. SELinux
10
11
. .hapt
C
. . . .er
. .2. .. Configuring
. . . . . . . . . . . Fencing
. . . . . . . .wit
. .h
. .t.he
. . ccs
. . . .Command
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 2. . . . . . . . . .
2 .1. Co nfig uring Fenc e Devic es
12
2 .2. Lis ting Fenc e Devic es and Fenc e Devic e O p tio ns
14
2 .3. Co nfig uring Fenc ing fo r Clus ter Memb ers
17
2 .3.1. Co nfig uring a Sing le Po wer-Bas ed Fenc e Devic e fo r a No d e
17
2 .3.2. Co nfig uring a Sing le Sto rag e-Bas ed Fenc e Devic e fo r a No d e
19
2 .3.3. Co nfig uring a Bac kup Fenc e Devic e
21
2 .3.4. Co nfig uring a No d e with Red und ant Po wer
24
2 .3.5. Tes ting the Fenc e Co nfig uratio n
27
2 .3.6 . Remo ving Fenc e Metho d s and Fenc e Ins tanc es
27
. .hapt
C
. . . .er
. .3.
. .Configuring
. . . . . . . . . . .Fencing
. . . . . . . wit
. . . h. .Conga
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. 8. . . . . . . . . .
3 .1. Co nfig uring Fenc e Daemo n Pro p erties
28
3 .2. Co nfig uring Fenc e Devic es
28
3 .2.1. Creating a Fenc e Devic e
29
3 .2.2. Mo d ifying a Fenc e Devic e
30
3 .2.3. Deleting a Fenc e Devic e
30
3 .3. Co nfig uring Fenc ing fo r Clus ter Memb ers
30
3 .3.1. Co nfig uring a Sing le Fenc e Devic e fo r a No d e
31
3 .3.2. Co nfig uring a Bac kup Fenc e Devic e
3 .3.3. Co nfig uring a No d e with Red und ant Po wer
3 .3.4. Tes ting the Fenc e Co nfig uratio n
31
32
33
. .hapt
C
. . . .er
. .4. .. Fence
. . . . . .Devices
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
...........
4 .1. APC Po wer Switc h o ver Telnet and SSH
36
4 .2. APC Po wer Switc h o ver SNMP
38
4 .3. Bro c ad e Fab ric Switc h
40
4 .4. Cis c o MDS
42
4 .5. Cis c o UCS
45
4 .6 . Dell Drac 5
46
4 .7. Eato n Netwo rk Po wer Switc h
49
4 .8 . Eg enera Blad eFrame
51
4 .9 . Emers o n Netwo rk Po wer Switc h (SNMP interfac e)
52
4 .10 . ePo werSwitc h
53
4 .11. Fenc e Virt (Serial/VMChannel Mo d e_
55
4 .12. Fenc e Virt (Multic as t Mo d e)
56
4 .13. Fujits u-Siemens Remo teView Servic e Bo ard (RSB)
56
56
58
59
61
63
65
68
70
73
75
77
77
79
80
82
. .ppendix
A
. . . . . . . A.
. . Revision
. . . . . . . . .Hist
. . . ory
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8. 5. . . . . . . . . .
I.ndex
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8. 5. . . . . . . . . .
P reface
Preface
1. Document Convent ions
This manual uses several conventions to highlight certain words and phrases and draw attention to
specific pieces of information.
Desktop
Desktop1
photos
scripts
stuff
svgs
svn
Source-code listings are also set in mo no -spaced ro man but add syntax highlighting as follows:
static int kvm_vm_ioctl_deassign_device(struct kvm *kvm,
struct kvm_assigned_pci_dev *assigned_dev)
{
int r = 0;
struct kvm_assigned_dev_kernel *match;
mutex_lock(& kvm->lock);
match = kvm_find_assigned_dev(& kvm->arch.assigned_dev_head,
assigned_dev->assigned_dev_id);
if (!match) {
printk(KERN_INFO "%s: device hasn't been assigned
P reface
before, "
"so cannot be deassigned\n", __func__);
r = -EINVAL;
goto out;
}
kvm_deassign_device(kvm, match);
kvm_free_assigned_device(kvm, match);
out:
mutex_unlock(& kvm->lock);
return r;
}
Note
Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should
have no negative consequences, but you might miss out on a trick that makes your life easier.
Important
Important boxes detail things that are easily missed: configuration changes that only apply to
the current session, or services that need restarting before an update will apply. Ignoring a
box labeled Important will not cause data loss but may cause irritation and frustration.
Warning
Warnings should not be ignored. Ignoring warnings will most likely cause data loss.
Red Hat also hosts a large number of electronic mailing lists for discussion of Red Hat software and
technology. You can find a list of publicly available mailing lists at
https://www.redhat.com/mailman/listinfo. Click the name of any mailing list to subscribe to that list or
to access the list archives.
1.1. Configuring ACPI For Use wit h Int egrat ed Fence Devices
If your cluster uses integrated fence devices, you must configure ACPI (Advanced Configuration and
Power Interface) to ensure immediate and complete fencing.
Note
For the most current information about integrated fence devices supported by Red Hat High
Availability Add-On, refer to http://www.redhat.com/cluster_suite/hardware/.
If a cluster node is configured to be fenced by an integrated fence device, disable ACPI Soft-Off for
that node. D isabling ACPI Soft-Off allows an integrated fence device to turn off a node immediately
and completely rather than attempting a clean shutdown (for example, shutd o wn -h no w).
Otherwise, if ACPI Soft-Off is enabled, an integrated fence device can take four or more seconds to
turn off a node (refer to note that follows). In addition, if ACPI Soft-Off is enabled and a node panics
or freezes during shutdown, an integrated fence device may not be able to turn off the node. Under
those circumstances, fencing is delayed or unsuccessful. Consequently, when a node is fenced with
an integrated fence device and ACPI Soft-Off is enabled, a cluster recovers slowly or requires
administrative intervention to recover.
Note
The amount of time required to fence a node depends on the integrated fence device used.
Some integrated fence devices perform the equivalent of pressing and holding the power
button; therefore, the fence device turns off the node in four to five seconds. Other integrated
fence devices perform the equivalent of pressing the power button momentarily, relying on the
operating system to turn off the node; therefore, the fence device turns off the node in a time
span much longer than four to five seconds.
To disable ACPI Soft-Off, use chkco nfi g management and verify that the node turns off immediately
when fenced. The preferred way to disable ACPI Soft-Off is with chkco nfi g management: however, if
that method is not satisfactory for your cluster, you can disable ACPI Soft-Off with one of the
following alternate methods:
Changing the BIOS setting to " instant-off" or an equivalent setting that turns off the node without
delay
Note
D isabling ACPI Soft-Off with the BIOS may not be possible with some computers.
Appending acpi = o ff to the kernel boot command line of the /bo o t/g rub/g rub. co nf file
Important
This method completely disables ACPI; some computers do not boot correctly if ACPI is
completely disabled. Use this method only if the other methods are not effective for your
cluster.
The following sections provide procedures for the preferred method and alternate methods of
disabling ACPI Soft-Off:
Section 1.1.1, D isabling ACPI Soft-Off with chkco nfi g Management Preferred method
Section 1.1.2, D isabling ACPI Soft-Off with the BIOS First alternate method
Section 1.1.3, D isabling ACPI Completely in the g rub. co nf File Second alternate method
Note
This is the preferred method of disabling ACPI Soft-Off.
D isable ACPI Soft-Off with chkco nfi g management at each cluster node as follows:
1. Run either of the following commands:
chkco nfi g --d el acpi d This command removes acpi d from chkco nfi g
management.
OR
chkco nfi g --l evel 234 5 acpi d o ff This command turns off acpi d .
2. Reboot the node.
3. When the cluster is configured and running, verify that the node turns off immediately when
fenced.
Note
You can fence the node with the fence_no d e command or C o n g a.
Note
D isabling ACPI Soft-Off with the BIOS may not be possible with some computers.
You can disable ACPI Soft-Off by configuring the BIOS of each cluster node as follows:
1. Reboot the node and start the BIO S C MO S Setup Uti l i ty program.
2. Navigate to the Po wer menu (or equivalent power management menu).
3. At the Po wer menu, set the So f t - O f f b y PWR - B T T N function (or equivalent) to In st an t O f f (or the equivalent setting that turns off the node via the power button without delay).
Example 1.1, BIO S C MO S Setup Uti l i ty: Soft-Off by PWR-BTTN set to Instant-Off
shows a Po wer menu with AC PI Fu n ct io n set to En ab led and So f t - O f f b y PWR - B T T N
set to In st an t - O f f .
Note
The equivalents to AC PI Fu n ct io n , So f t - O f f b y PWR - B T T N , and In st an t - O f f may
vary among computers. However, the objective of this procedure is to configure the
BIOS so that the computer is turned off via the power button without delay.
4. Exit the BIO S C MO S Setup Uti l i ty program, saving the BIOS configuration.
5. When the cluster is configured and running, verify that the node turns off immediately when
fenced.
Note
You can fence the node with the fence_no d e command or C o n g a.
|
POWER ON Function
[BUTTON ONLY]|
|
| x KB Power ON Password
Enter
|
|
| x Hot Key Power ON
Ctrl-F1
|
|
+------------------------------------------|-----------------+
This example shows AC PI Fu n ct io n set to En ab led , and So f t - O f f b y PWR - B T T N set to
In st an t - O f f .
Important
This method completely disables ACPI; some computers do not boot correctly if ACPI is
completely disabled. Use this method only if the other methods are not effective for your cluster.
You can disable ACPI completely by editing the g rub. co nf file of each cluster node as follows:
1. Open /bo o t/g rub/g rub. co nf with a text editor.
2. Append acpi = o ff to the kernel boot command line in /bo o t/g rub/g rub. co nf (refer to
Example 1.2, Kernel Boot Command Line with acpi = o ff Appended to It ).
3. Reboot the node.
4. When the cluster is configured and running, verify that the node turns off immediately when
fenced.
Note
You can fence the node with the fence_no d e command or C o n g a.
10
#boot=/dev/hda
default=0
timeout=5
serial --unit=0 --speed=115200
terminal --timeout=5 serial console
title Red Hat Enterprise Linux Server (2.6.32-193.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-193.el6.x86_64 ro
root=/dev/mapper/vg_doc01-lv_root console=ttyS0,115200n8 acpi=off
initrd /initramrs-2.6.32-131.0.15.el6.x86_64.img
In this example, acpi = o ff has been appended to the kernel boot command line the line
starting with " kernel /vmlinuz-2.6.32-193.el6.x86_64.img" .
1.2. SELinux
The High Availability Add-On for Red Hat Enterprise Linux 6 supports SELinux in the enfo rci ng
state with the SELinux policy type set to targ eted .
Note
When using SELinux with the High Availability Add-On in a VM environment, you should
ensure that the SELinux boolean fenced _can_netwo rk_co nnect is persistently set to o n.
This allows the fence_xvm fencing agent to work properly, enabling the system to fence
virtual machines.
For more information about SELinux, refer to Deployment Guide for Red Hat Enterprise Linux 6.
11
Important
Make sure that your deployment of High Availability Add-On meets your needs and can be
supported. Consult with an authorized Red Hat representative to verify your configuration
prior to deployment. In addition, allow time for a configuration burn-in period to test failure
modes.
Important
This chapter references commonly used cl uster. co nf elements and attributes. For a
comprehensive list and description of cl uster. co nf elements and attributes, refer to the
cluster schema at /usr/share/cl uster/cl uster. rng , and the annotated schema at
/usr/share/d o c/cman-X. Y . ZZ/cl uster_co nf. html (for example
/usr/share/d o c/cman-3. 0 . 12/cl uster_co nf. html ).
12
The po st-jo i n_d el ay attribute is the number of seconds the fence daemon (fenced ) waits
before fencing a node after the node joins the fence domain. The po st_jo i n_d el ay default
value is 6 . A typical setting for po st_jo i n_d el ay is between 20 and 30 seconds, but can vary
according to cluster and network performance.
You reset the values of the po st_fai l _d el ay and po st_jo i n_d el ay attributes with the -setfenced aemo n option of the ccs command. Note, however, that executing the ccs -setfenced aemo n command overwrites all existing fence daemon properties.
For example, to configure a value for the po st_fai l _d el ay attribute, execute the following
command. This command will overwrite the values of all other exisiting fence daemon properties that
you can set with this command.
ccs -h host --setfencedaemon post_fail_delay=value
To configure a value for the po st_jo i n_d el ay attribute, execute the following command. This
command will overwrite the values of all other exisiting fence daemon properties that you can set with
this command.
ccs -h host --setfencedaemon post_join_delay=value
To configure a value for both the the po st_jo i n_d el ay attribute and the po st_fai l _d el ay
attribute, execute the following command:
ccs -h host --setfencedaemon post_fail_delay=value post_join_delay=value
Note
For more information about the po st_jo i n_d el ay and po st_fai l _d el ay attributes as
well as the additional fence daemon properties you can modify, refer to the fenced(8) man
page and refer to the cluster schema at /usr/share/cl uster/cl uster. rng , and the
annotated schema at /usr/share/d o c/cman-X. Y . ZZ/cl uster_co nf. html .
To configure a fence device for a cluster, execute the following command:
ccs -h host --addfencedev
devicename
[fencedeviceoptions]
For example, to configure an APC fence device in the configuration file on the cluster node no d e-0 1
named myfence with an IP address of apc_i p_exampl e, a login of l o g i n_exampl e, and a
password of passwo rd _exampl e, execute the following command:
ccs -h node-01 --addfencedev myfence agent=fence_apc
ipaddr=apc_ip_example login=login_example passwd=password_example
The following example shows the fenced evi ces section of the cl uster. co nf configuration file
after you have added this APC fence device:
<fencedevices>
<fencedevice agent="fence_apc" ipaddr="apc_ip_example"
13
2.2. List ing Fence Devices and Fence Device Opt ions
You can use the ccs command to print a list of available fence devices and to print a list of options
for each available fence type. You can also use the ccs command to print a list of fence devices
currently configured for your cluster.
To print a list of fence devices currently available for your cluster, execute the following command:
ccs -h host --lsfenceopts
For example, the following command lists the fence devices available on the cluster node no d e-0 1,
showing sample output.
[root@ ask-03 ~]# ccs -h no d e-0 1 --l sfenceo pts
fence_apc - Fence agent for APC over telnet/ssh
fence_apc_snmp - Fence agent for APC, Tripplite PDU over SNMP
fence_bladecenter - Fence agent for IBM BladeCenter
fence_bladecenter_snmp - Fence agent for IBM BladeCenter over SNMP
fence_brocade - Fence agent for HP Brocade over telnet/ssh
fence_cisco_mds - Fence agent for Cisco MDS
fence_cisco_ucs - Fence agent for Cisco UCS
fence_drac - fencing agent for Dell Remote Access Card
fence_drac5 - Fence agent for Dell DRAC CMC/5
fence_eaton_snmp - Fence agent for Eaton over SNMP
fence_egenera - I/O Fencing agent for the Egenera BladeFrame
fence_emerson - Fence agent for Emerson over SNMP
fence_eps - Fence agent for ePowerSwitch
fence_hpblade - Fence agent for HP BladeSystem
fence_ibmblade - Fence agent for IBM BladeCenter over SNMP
14
15
16
17
node01.example.com
node01.example.com
node01.example.com
node01.example.com
Example 2.1, cl uster. co nf After Adding Power-Based Fence Methods shows a cl uster. co nf
configuration file after you have added these fencing methods and instances to each node in the
cluster.
18
</fencedevices>
<rm>
</rm>
</cluster>
Note that when you have finished configuring all of the components of your cluster, you will need to
sync the cluster configuration file to all of the nodes.
node01.example.com
2. Add a fence instance for the method. You must specify the fence device to use for the node,
the node this instance applies to, the name of the method, and any options for this method
that are specific to this node:
ccs -h host --addfenceinst fencedevicename node method [options]
For example, to configure a fence instance in the configuration file on the cluster node no d e0 1. exampl e. co m that uses the SAN switch power port 11 on the fence device named
sanswi tch1 to fence cluster node no d e-0 1. exampl e. co m using the method named SAN,
execute the following command:
ccs -h node01.example.com --addfenceinst sanswitch1
node01.example.com SAN port=11
3. To configure unfencing for the storage based fence device on this node, execute the following
command:
ccs -h host --addunfence fencedevicename node action=on|off
You will need to add a fence method for each node in the cluster. The following commands configure
a fence method for each node with the method name SAN. The device for the fence method specifies
sanswi tch as the device name, which is a device previously configured with the --addfencedev
19
option, as described in Section 2.1, Configuring Fence D evices . Each node is configured with a
unique SAN physical port number: The port number for no d e-0 1. exampl e. co m is 11, the port
number for no d e-0 2. exampl e. co m is 12, and the port number for no d e-0 3. exampl e. co m is
13.
ccs -h node01.example.com
ccs -h node01.example.com
ccs -h node01.example.com
ccs -h node01.example.com
SAN port=11
ccs -h node01.example.com
SAN port=12
ccs -h node01.example.com
SAN port=13
ccs -h node01.example.com
port=11 action=on
ccs -h node01.example.com
port=12 action=on
ccs -h node01.example.com
port=13 action=on
20
</fence>
<unfence>
<device name="sanswitch1" port="13" action="on"/>
</unfence>
</clusternode>
</clusternodes>
<fencedevices>
<fencedevice agent="fence_sanbox2" ipaddr="san_ip_example"
login="login_example" name="sanswitch1" passwd="password_example"/>
</fencedevices>
<rm>
</rm>
</cluster>
Note that when you have finished configuring all of the components of your cluster, you will need to
sync the cluster configuration file to all of the nodes.
Note
The order in which the system will use the fencing methods you have configured follows their
order in the cluster configuration file. The first method you configure with the ccs command is
the primary fencing method, and the second method you configure is the backup fencing
method. To change the order, you can remove the primary fencing method from the
configuration file, then add that method back.
Note that at any time you can print a list of fence methods and instances currently configured for a
node by executing the following command. If you do not specify a node, this command will list the
fence methods and instances currently configured for all nodes.
ccs -h host --lsfenceinst [node]
Use the following procedure to configure a node with a primary fencing method that uses a fence
device named apc, which uses the fence_apc fencing agent, and a backup fencing device that
uses a fence device named sanswi tch1, which uses the fence_sanbo x2 fencing agent. Since the
sanswi tch1 device is a storage-based fencing agent, you will need to configure unfencing for that
device as well.
1. Add a primary fence method for the node, providing a name for the fence method.
ccs -h host --addmethod method node
For example, to configure a fence method named AP C as the primary method for the node
no d e-0 1. exampl e. co m in the configuration file on the cluster node no d e0 1. exampl e. co m, execute the following command:
21
node01.example.com
4. Add a fence instance for the backup method. You must specify the fence device to use for the
node, the node this instance applies to, the name of the method, and any options for this
method that are specific to this node:
ccs -h host --addfenceinst fencedevicename node method [options]
For example, to configure a fence instance in the configuration file on the cluster node no d e0 1. exampl e. co m that uses the SAN switch power port 11 on the fence device named
sanswi tch1 to fence cluster node no d e-0 1. exampl e. co m using the method named SAN,
execute the following command:
ccs -h node01.example.com --addfenceinst sanswitch1
node01.example.com SAN port=11
5. Since the sanswi tch1 device is a storage-based device, you must configure unfencing for
this device.
ccs -h node01.example.com --addunfence sanswitch1
node01.example.com port=11 action=on
You can continue to add fencing methods as needed.
This procedure configures a fence device and a backup fence device for one node in the cluster. You
will need to configure fencing for the other nodes in the cluster as well.
22
Example 2.3, cl uster. co nf After Adding Backup Fence Methods shows a cl uster. co nf
configuration file after you have added a power-based primary fencing method and a storage-based
backup fencing method to each node in the cluster.
23
<rm>
</rm>
</cluster>
Note that when you have finished configuring all of the components of your cluster, you will need to
sync the cluster configuration file to all of the nodes.
Note
The order in which the system will use the fencing methods you have configured follows their
order in the cluster configuration file. The first method you configure is the primary fencing
method, and the second method you configure is the backup fencing method. To change the
order, you can remove the primary fencing method from the configuration file, then add that
method back.
24
3. Add a fence instance for the first power supply to the fence method. You must specify the
fence device to use for the node, the node this instance applies to, the name of the method,
and any options for this method that are specific to this node. At this point you configure the
acti o n attribute as o ff.
ccs -h host --addfenceinst fencedevicename node method [options]
action=off
For example, to configure a fence instance in the configuration file on the cluster node no d e0 1. exampl e. co m that uses the APC switch power port 1 on the fence device named apc1 to
fence cluster node no d e-0 1. exampl e. co m using the method named AP C -d ual , and
setting the acti o n attribute to o ff, execute the following command:
ccs -h node01.example.com --addfenceinst apc1 node01.example.com
APC-dual port=1 action=off
4. Add a fence instance for the second power supply to the fence method. You must specify the
fence device to use for the node, the node this instance applies to, the name of the method,
and any options for this method that are specific to this node. At this point you configure the
acti o n attribute as o ff for this instance as well:
ccs -h host --addfenceinst fencedevicename node method [options]
action=off
For example, to configure a second fence instance in the configuration file on the cluster
node no d e-0 1. exampl e. co m that uses the APC switch power port 1 on the fence device
named apc2 to fence cluster node no d e-0 1. exampl e. co m using the same method as you
specified for the first instance named AP C -d ual , and setting the acti o n attribute to o ff,
execute the following command:
ccs -h node01.example.com --addfenceinst apc2 node01.example.com
APC-dual port=1 action=off
5. At this point, add another fence instance for the first power supply to the fence method,
configuring the acti o n attribute as o n. You must specify the fence device to use for the
node, the node this instance applies to, the name of the method, and any options for this
method that are specific to this node, and specifying the acti o n attribute as o n:
ccs -h host --addfenceinst fencedevicename node method [options]
action=on
For example, to configure a fence instance in the configuration file on the cluster node no d e0 1. exampl e. co m that uses the APC switch power port 1 on the fence device named apc1 to
fence cluster node no d e-0 1. exampl e. co m using the method named AP C -d ual , and
setting the acti o n attribute to o n, execute the following command:
ccs -h node01.example.com --addfenceinst apc1 node01.example.com
APC-dual port=1 action=on
6. Add another fence instance for second power supply to the fence method, specifying the
acti o n attribute as o n for this instance. You must specify the fence device to use for the
node, the node this instance applies to, the name of the method, and any options for this
method that are specific to this node as well as the acti o n attribute of o n.
25
26
Note that when you have finished configuring all of the components of your cluster, you will need to
sync the cluster configuration file to all of the nodes.
To remove all fence instances of a fence device from a fence method, execute the following command:
ccs -h host --rmfenceinst fencedevicename node method
For example, to remove all instances of the fence device named apc1 from the method named AP C d ual configured for no d e0 1. exampl e. co m from the cluster configuration file on cluster node
no d e0 1. exampl e. co m, execute the following command:
ccs -h node01.example.com --rmfenceinst apc1 node01.example.com APC-dual
27
Note
Conga is a graphical user interface that you can use to administer the Red Hat High
Availability Add-On. Note, however, that in order to use this interface effectively you need to
have a good and clear understanding of the underlying concepts. Learning about cluster
configuration by exploring the available features in the user interface is not recommended, as
it may result in a system that is not robust enough to keep all services running when
components fail.
Section 3.2, Configuring Fence D evices
Note
For more information about Po st Jo in D elay and Po st Fail D elay, refer to the fenced(8)
man page.
28
Note
If this is an initial cluster configuration, no fence devices have been created, and therefore
none are displayed.
Figure 3.1, luci fence devices configuration page shows the fence devices configuration screen
before any fence devices have been created.
29
2. Specify the information in the Ad d Fence D evi ce (Instance) dialog box according to
the type of fence device. In some cases you will need to specify additional node-specific
parameters for the fence device when you configure fencing for the individual nodes.
3. Click Submi t.
After the fence device has been added, it appears on the Fen ce D evices configuration page.
Note
Fence devices that are in use cannot be deleted. To delete a fence device that a node is
currently using, first update the node fence configuration for any node using the device and
then delete the device.
To delete a fence device, follow these steps:
1. From the Fen ce D evices configuration page, check the box to the left of the fence device or
devices to select the devices to delete.
2. Click D el ete and wait for the configuration to be updated. A message appears indicating
which devices are being deleted.
When the configuration has been updated, the deleted fence device no longer appears in the display.
30
Note
For non-power fence methods (that is, SAN/storage fencing), U n f en cin g is selected
by default on the node-specific parameters display. This ensures that a fenced node's
access to storage is not re-enabled until the node has been rebooted. For information
on unfencing a node, refer to the fence_no d e(8) man page.
8. Click Submi t. This returns you to the node-specific screen with the fence method and fence
instance displayed.
31
2. Beneath the display of the primary method you defined, click Ad d Fence Metho d .
3. Enter a name for the backup fencing method that you are configuring for this node and click
Submi t. This displays the node-specific screen that now displays the method you have just
added, below the primary fence method.
4. Configure a fence instance for this method by clicking Ad d Fence Instance. This displays
a drop-down menu from which you can select a fence device you have previously configured,
as described in Section 3.2.1, Creating a Fence D evice .
5. Select a fence device for this method. If this fence device requires that you configure nodespecific parameters, the display shows the parameters to configure.
6. Click Submi t. This returns you to the node-specific screen with the fence method and fence
instance displayed.
You can continue to add fencing methods as needed. You can rearrange the order of fencing
methods that will be used for this node by clicking on Mo ve U p and Mo ve D o wn .
32
9. Click Submi t. This returns you to the node-specific screen with the fence method and fence
instance displayed.
10. Under the same fence method for which you have configured the first power fencing device,
click Ad d Fence Instance. This displays a drop-down menu from which you can select
the second power fencing devices you have previously configured, as described in
Section 3.2.1, Creating a Fence D evice .
11. Select the second of the power fence devices for this method and enter the appropriate
parameters for this device.
12. Click Submi t. This returns you to the node-specific screen with the fence methods and fence
instances displayed, showing that each device will power the system off in sequence and
power the system on in sequence. This is shown in Figure 3.2, D ual-Power Fencing
Configuration .
33
34
Fen ce Ag en t
R ef eren ce t o Paramet er
D escrip t io n
APC Power
Switch
(telnet/SSH)
APC Power
Switch over
SNMP
Brocade
Fabric Switch
Cisco MD S
Cisco UCS
D ell D RAC 5
D ell iD RAC
fence_apc
fence_apc_snmp
fence_brocade
Eaton Network
Power Switch
(SNMP
Interface)
Egenera
BladeFrame
Emerson
Network Power
Switch (SNMP
Interface)
ePowerSwitch
Fence kdump
Fence virt
fence_eaton_snmp
Fujitsu
Siemens
Remoteview
Service Board
(RSB)
HP
BladeSystem
fence_rsb
fence_cisco_mds
fence_cisco_ucs
fence_drac5
fence_idrac
fence_egenera
fence_emerson
fence_eps
fence_kdump
fence_virt
fence_hpblade
35
Fen ce D evice
Fen ce Ag en t
R ef eren ce t o Paramet er
D escrip t io n
HP iLO D evice
(Integrated
Lights Out),
HP iLO2
fence_ilo
fence_ilo2
HPiLO3
fence_ilo3
HPiLO4
fence_ilo4
HP iLO
(Integrated
Lights Out) MP
IBM
BladeCenter
IBM
BladeCenter
SNMP
IBM Integrated
Management
Module
fence_ilo_mp
IBM iPD U
fence_ipdu
IF MIB
Intel Modular
IPMI
(Intelligent
Platform
Management
Interface) Lan
RHEV-M REST
API
fence_ifmib
fence_intelmodular
fence_ipmilan
SCSI Fencing
fence_scsi
VMware
Fencing
(SOAP
Interface)
WTI Power
Switch
fence_vmware_soap
fence_bladecenter
fence_ibmblade
fence_imm
fence_rhevm
fence_wti
Table 4.2, APC Power Switch (telnet/SSH) lists the fence device parameters used by fence_apc,
the fence agent for APC over telnet/SSH.
T ab le 4 .2. APC Po wer Swit ch ( t eln et /SSH )
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
IP Port
(optional)
Login
Password
Password
Script
(optional)
Power Wait
(seconds)
Power Timeout
(seconds)
i pad d r
A name for the APC device connected to the cluster into which
the fence daemon logs via telnet/ssh.
The IP address or hostname assigned to the device.
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Port
Switch
(optional)
D elay
(optional)
Use SSH
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
SSH Options
Path to SSH
Identity File
i ppo rt
login
passwd
passwd _scri
pt
po wer_wai t
po wer_ti meo
ut
The TCP port to use to connect to the device. The default port is
23, or 22 if Use SSH is selected.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
Number of seconds to wait after issuing a power off or power on
command.
Number of seconds to wait before testing for a status change
after issuing a power on or power on command. The default
value is 20.
Number of seconds to wait for a command prompt after issuing a
command. The default value is 3.
Number of seconds to wait for a command prompt after login.
The default value is 5.
Number of attempts to retry a power on operation. The default
value is 1.
The port.
The switch number for the APC switch that connects to the node
when you have multiple daisy-chained switches.
d el ay
The number of seconds to wait before fencing is started. The
default value is 0.
secure
Indicates that system will use SSH to access the device. When
using SSH, you must specify either a password, a password
script, or an identity file.
ssh_o pti o ns SSH options to use. The default value is -1 -c bl o wfi sh.
i d enti ty_fi The identity file for SSH.
le
po rt
swi tch
Figure 4.1, APC Power Switch shows the configuration screen for adding an APC Power Switch
fence device.
37
<fencedevices>
<fencedevice agent="fence_apc" name="apc" ipaddr="apctelnet.example.com" login="root" passwd="password123"/>
</fencedevices>
38
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
UD P/TCP port
i pad d r
A name for the APC device connected to the cluster into which
the fence daemon logs via the SNMP protocol.
The IP address or hostname assigned to the device.
Login
Password
Password
Script
(optional)
SNMP Version
login
passwd
passwd _scri
pt
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP Privacy
Protocol
Password
SNMP Privacy
Protocol Script
ud ppo rt
snmp_versi o
n
co mmuni ty
The SNMP version to use (1, 2c, 3); the default value is 1.
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
snmp_pri v_p
asswd _scri p
t
po wer_wai t
Power Wait
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
The UD P/TCP port to use for connection with the device; the
default value is 161.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
po rt
The port.
d el ay
Figure 4.2, APC Power Switch over SNMP shows the configuration screen for adding an APC
Power Switch fence device.
39
<fencedevice>
<fencedevice agent="fence_apc_snmp" community="private"
ipaddr="192.168.0.1" login="root" \
name="apcpwsnmptst1" passwd="password123" power_wait="60"
snmp_priv_passwd="password123"/>
</fencedevices>
Table 4.4, Brocade Fabric Switch lists the fence device parameters used by fence_bro cad e, the
fence agent for Brocade FC switches.
T ab le 4 .4 . B ro cad e Fab ric Swit ch
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
Login
Password
Password
Script
(optional)
Force IP
Family
name
i pad d r
login
passwd
passwd _scri
pt
i net4 _o nl y
,
i net6 _o nl y
cmd _pro mpt
Force
Command
Prompt
Power Wait
po wer_wai t
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Port
D elay
(optional)
Use SSH
SSH Options
Path to SSH
Identity File
Unfencing
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
Figure 4.3, Brocade Fabric Switch shows the configuration screen for adding an Brocade Fabric
Switch fence device.
41
<fencedevices>
<fencedevice agent="fence_brocade" ipaddr="brocadetest.example.com"
login="brocadetest" \
name="brocadetest" passwd="brocadetest"/>
</fencedevices>
4 .4 . Cisco MDS
Table 4.5, Cisco MD S lists the fence device parameters used by fence_ci sco _md s, the fence
agent for Cisco MD S.
T ab le 4 .5. C isco MD S
lu ci Field
42
cl uster. co n
f At t rib u t e
D escrip t io n
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
UD P/TCP port
(optional)
Login
Password
Password
Script
(optional)
SNMP Version
i pad d r
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP Privacy
Protocol
Password
SNMP Privacy
Protocol Script
ud ppo rt
login
passwd
passwd _scri
pt
snmp_versi o
n
co mmuni ty
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
snmp_pri v_p
asswd _scri p
t
po wer_wai t
Power Wait
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
The UD P/TCP port to use for connection with the device; the
default value is 161.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
po rt
The port.
d el ay
Figure 4.4, Cisco MD S shows the configuration screen for adding an Cisco MD S fence device.
43
Fig u re 4 .4 . C isco MD S
The following command creates a fence device instance for a Cisco MD S device:
ccs -f cluster.conf --addfencedev mds agent=fence_cisco_mds
ipaddr=192.168.0.1 name=ciscomdstest1 login=root \
passwd=password123 power_wait=60 snmp_priv_passwd=password123 udpport=161
The following is the cl uster. co nf entry for the fence_ci sco _md s device:
44
<fencedevices>
<fencedevice agent="fence_cisco_mds" community="private"
ipaddr="192.168.0.1" login="root" \
name="ciscomdstest1" passwd="password123" power_wait="60"
snmp_priv_passwd="password123" \
udpport="161"/>
</fencedevices>
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
IP port
(optional)
Login
Password
Password
Script
(optional)
Use SSL
SubOrganization
Power Wait
(seconds)
Power Timeout
(seconds)
name
i pad d r
i ppo rt
login
passwd
passwd _scri
pt
ssl
subo rg
po wer_wai t
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
po wer_ti meo
ut
po rt
d el ay
Figure 4.5, Cisco UCS shows the configuration screen for adding a Cisco UCS fence device.
45
<fencedevices>
<fencedevice agent="fence_cisco_ucs" ipaddr="192.168.0.1" login="root"
name="ciscoucstest1" \
passwd="password123" power_wait="60" ssl="on" suborg="/org-RHEL/orgFence/"/>
</fencedevices>
Table 4.7, D ell D RAC 5 lists the fence device parameters used by fence_d rac5, the fence agent for
D ell D RAC 5.
T ab le 4 .7. D ell D R AC 5
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
IP Port
(optional)
Login
Password
Password
Script
(optional)
Use SSH
name
i pad d r
i ppo rt
login
passwd
passwd _scri
pt
SSH Options
Path to SSH
Identity File
Module Name
Force
Command
Prompt
Power Wait
(seconds)
D elay
(seconds)
Power Timeout
(seconds)
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Indicates that the system will use SSH to access the device.
When using SSH, you must specify either a password, a
password script, or an identity file.
ssh_o pti o ns SSH options to use. The default value is -1 -c bl o wfi sh.
i d enti ty_fi The identity file for SSH.
le
mo d ul e_nam (optional) The module name for the D RAC when you have
e
multiple D RAC modules.
cmd _pro mpt
The command prompt to use. The default value is '\$'.
secure
po wer_wai t
d el ay
po wer_ti meo
ut
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
Figure 4.6, D ell D rac 5 shows the configuration screen for adding a D ell D rac 5 device
47
<fencedevices>
<fencedevice agent="fence_drac5" cmd_prompt="\$" ipaddr="192.168.0.1"
login="root" module_name="drac1" \
name="delldrac5test1" passwd="password123" power_wait="60"/>
</fencedevices>
48
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
UD P/TCP Port
(optional)
Login
Password
Password
Script
(optional)
SNMP Version
i pad d r
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP Privacy
Protocol
Password
SNMP Privacy
Protocol Script
ud ppo rt
login
passwd
passwd _scri
pt
snmp_versi o
n
co mmuni ty
The SNMP version to use (1, 2c, 3); the default value is 1.
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
snmp_pri v_p
asswd _scri p
t
po wer_wai t
Power wait
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
The UD P/TCP port to use for connection with the device; the
default value is 161.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
49
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Port (Outlet)
Number
D elay
(optional)
po rt
d el ay
Figure 4.7, Eaton Network Power Switch shows the configuration screen for adding an Eaton
Network Power Switch fence device.
50
The following command creates a fence device instance for an Eaton Network Power Switch device:
ccs -f cluster.conf --addfencedev eatontest agent=fence_eaton_snmp
ipaddr=192.168.0.1 login=root \
passwd=password123 power_wait=60 snmp_priv_passwd=eatonpassword123
udpport=161
The following is the cl uster. co nf entry for the fence_eato n_snmp device:
<fencedevices>
<fencedevice agent="fence_eaton_snmp" community="private"
ipaddr="eatonhost" login="eatonlogin" \
name="eatontest" passwd="password123" passwd_script="eatonpwscr"
power_wait="3333" \
snmp_priv_passwd="eatonprivprotpass"
snmp_priv_passwd_script="eatonprivprotpwscr" udpport="161"/>
</fencedevices>
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
CServer
cserver
ESH Path
(optional)
Username
lpan
pserver
D elay
(optional)
Unfencing
esh
user
l pan
pserver
d el ay
unfence
section of the
cluster
configuration
file
Figure 4.8, Egenera BladeFrame shows the configuration screen for adding an Egenera
BladeFrame fence device.
51
<fencedevices>
<fencedevice agent="fence_egenera" cserver="cservertest"
name="egeneratest" user="root"/>
</fencedevices>
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
UD P/TCP Port
(optional)
i pad d r
52
ud ppo rt
UD P/TCP port to use for connections with the device; the default
value is 161.
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Login
Password
Password
Script
(optional)
SNMP Version
login
passwd
passwd _scri
pt
snmp_versi o
n
co mmuni ty
The SNMP version to use (1, 2c, 3); the default value is 1.
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
snmp_pri v_p
asswd _scri p
t
po wer_wai t
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP privacy
protocol
password
SNMP Privacy
Protocol Script
Power Wait
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
po rt
d el ay
4 .10. ePowerSwit ch
Table 4.11, ePowerSwitch lists the fence device parameters used by fence_eps, the fence agent for
ePowerSwitch.
T ab le 4 .11. ePo werSwit ch
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
53
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
IP Address or
Hostname
Login
Password
Password
Script
(optional)
Name of
Hidden Page
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
i pad d r
login
passwd
passwd _scri
pt
hi d d en_pag
e
retry_o n
po rt
d el ay
Figure 4.9, ePowerSwitch shows the configuration screen for adding an ePowerSwitch fence
device.
54
<fencedevices>
<fencedevice agent="fence_eps" hidden_page="hidden.htm"
ipaddr="192.168.0.1" login="root" name="epstest1" \
passwd="password123"/>
</fencedevices>
cl uster. co n
f At t rib u t e
D escrip t io n
Name
Serial D evice
name
seri al _d ev
i ce
Serial
Parameters
VM Channel IP
Address
Timeout
(optional)
D omain
seri al _para
ms
channel _ad
d ress
ti meo ut
D elay
(optional)
po rt (formerly
d o mai n)
i ppo rt
d el ay
The following command creates a fence device instance for virtual machines using serial mode.
ccs -f cluster.conf --addfencedev fencevirt1 agent=fence_virt
serial_device=/dev/ttyS1 serial_params=19200, 8N1
The following is the cl uster. co nf entry for the fence_vi rt device:
<fencedevices>
55
cl uster. co n
f At t rib u t e
D escrip t io n
Name
Timeout
D omain
name
ti meo ut
po rt (formerly
d o mai n)
d el ay
D elay
(optional)
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
Login
Password
Password
Script
(optional)
TCP Port
name
i pad d r
login
passwd
passwd _scri
pt
i ppo rt
Force
Command
Prompt
Power Wait
(seconds)
D elay
(seconds)
Power Timeout
(seconds)
The port number on which the telnet service listens. The default
value is 3172.
The command prompt to use. The default value is '\$'.
56
po wer_wai t
d el ay
po wer_ti meo
ut
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
Figure 4.10, Fujitsu-Siemens RSB shows the configuration screen for adding an Fujitsu-Siemens
RSB fence device.
<fencedevices>
<fencedevice agent="fence_rsb" ipaddr="192.168.0.1" login="root"
57
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
IP Port
(optional)
Login
i pad d r
Password
passwd
Password
Script
(optional)
Force
Command
Prompt
Missing port
returns OFF
instead of
failure
Power Wait
(seconds)
Power Timeout
(seconds)
passwd _scri
pt
mi ssi ng _as
_o ff
po wer_wai t
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Use SSH
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
SSH Options
Path to SSH
Identity File
58
i ppo rt
login
po wer_ti meo
ut
Indicates that the system will use SSH to access the device.
When using SSH, you must specify either a password, a
password script, or an identity file.
ssh_o pti o ns SSH options to use. The default value is -1 -c bl o wfi sh.
i d enti ty_fi The identity file for SSH.
le
secure
Figure 4.11, HP BladeSystem shows the configuration screen for adding an HP BladeSystem fence
device.
<fencedevices>
<fencedevice agent="fence_hpblade" cmd_prompt="c7000oa>"
ipaddr="hpbladeaddr" ipport="13456" \
login="root" missing_as_off="on" name="hpbladetest1"
passwd="password123" passwd_script="hpbladepwscr" \
power_wait="60"/>
</fencedevices>
The fence agents for HP iLO devices fence_i l o and HP iLO2 devices fence_i l o 2. share the
same implementation. Table 4.16, HP iLO (Integrated Lights Out) and HP iLO2 lists the fence device
parameters used by these agents.
T ab le 4 .16 . H P iLO ( In t eg rat ed Lig h t s O u t ) an d H P iLO 2
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
IP Port
(optional)
Login
Password
Password
Script
(optional)
Power Wait
(seconds)
D elay
(seconds)
Power Timeout
(seconds)
name
i pad d r
i ppo rt
TCP port to use for connection with the device. The default value
is 443.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
login
passwd
passwd _scri
pt
po wer_wai t
d el ay
po wer_ti meo
ut
Figure 4.12, HP iLO shows the configuration screen for adding an HP iLO fence device.
60
<fencedevices>
<fencedevice agent="fence_ilo" ipaddr="192.168.0.1" login="root"
name="hpilotest1" passwd="password123" \
power_wait="60"/>
</fencedevices>
61
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
IP Port
(optional)
Login
Password
Password
Script
(optional)
Use SSH
name
i pad d r
i ppo rt
login
passwd
passwd _scri
pt
SSH Options
Path to SSH
Identity File
Force
Command
Prompt
Power Wait
(seconds)
D elay
(seconds)
Power Timeout
(seconds)
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Indicates that the system will use SSH to access the device.
When using SSH, you must specify either a password, a
password script, or an identity file.
ssh_o pti o ns SSH options to use. The default value is -1 -c bl o wfi sh.
i d enti ty_fi The Identity file for SSH.
le
cmd _pro mpt
The command prompt to use. The default value is 'MP>', 'hpiLO>'.
secure
po wer_wai t
d el ay
po wer_ti meo
ut
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
Figure 4.13, HP iLO MP shows the configuration screen for adding an HP iLO MPfence device.
62
<fencedevices>
<fencedevice agent="fence_ilo_mp" cmd_prompt="hpiLO-& gt;"
ipaddr="192.168.0.1" login="root" name="hpilomptest1"
passwd="password123" power_wait="60"/>
</fencedevices>
63
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
IP port
(optional)
Login
Password
Password
Script
(optional)
Power Wait
(seconds)
Power Timeout
(seconds)
name
i pad d r
i ppo rt
login
passwd
passwd _scri
pt
po wer_wai t
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Use SSH
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
SSH Options
Path to SSH
Identity File
po wer_ti meo
ut
Indicates that system will use SSH to access the device. When
using SSH, you must specify either a password, a password
script, or an identity file.
ssh_o pti o ns SSH options to use. The default value is -1 -c bl o wfi sh.
i d enti ty_fi The identity file for SSH.
le
secure
Figure 4.14, IBM BladeCenter shows the configuration screen for adding an IBM BladeCenter fence
device.
64
<fencedevices>
<fencedevice agent="fence_bladecenter" ipaddr="192.168.0.1"
login="root" name="bladecentertest1" passwd="password123" \
power_wait="60"/>
</fencedevices>
65
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
UD P/TCP Port
(optional)
Login
Password
Password
Script
(optional)
SNMP Version
i pad d r
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP privacy
protocol
password
SNMP Privacy
Protocol Script
ud ppo rt
login
passwd
passwd _scri
pt
snmp_versi o
n
co mmuni ty
The SNMP version to use (1, 2c, 3); the default value is 1.
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
snmp_pri v_p
asswd _scri p
t
po wer_wai t
Power Wait
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
UD P/TCP port to use for connections with the device; the default
value is 161.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
po rt
d el ay
Figure 4.15, IBM BladeCenter SNMP shows the configuration screen for adding an IBM
BladeCenter SNMP fence device.
66
<fencedevices>
<fencedevice agent="fence_ibmblade" community="private"
67
cl uster. co n
f At t rib u t e
D escrip t io n
Name
name
IP Address or
Hostname
UD P/TCP Port
i pad d r
A name for the IBM iPD U device connected to the cluster into
which the fence daemon logs via the SNMP protocol.
The IP address or hostname assigned to the device.
Login
Password
Password
Script
(optional)
SNMP Version
login
passwd
passwd _scri
pt
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP Privacy
Protocol
Password
SNMP Privacy
Protocol Script
ud ppo rt
snmp_versi o
n
co mmuni ty
The SNMP version to use (1, 2c, 3); the default value is 1.
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
snmp_pri v_p
asswd _scri p
t
po wer_wai t
Power Wait
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
68
The UD P/TCP port to use for connection with the device; the
default value is 161.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
shel l _ti me
o ut
l o g i n_ti me
o ut
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
retry_o n
po rt
d el ay
Figure 4.16, IBM iPD U shows the configuration screen for adding an IBM iPD U fence device.
69
The following command creates a fence device instance for an IBM iPD U device:
ccs -f cluster.conf --addfencedev ipdutest1 agent=fence_ipdu
community=ipdusnmpcom ipaddr=192.168.0.1 login=root \
passwd=password123 snmp_priv_passwd=snmpasswd123 power_wait=60
snmp_priv_prot=AES udpport=111
The following is the cl uster. co nf entry for the fence_i pd u device:
<fencedevices>
<fencedevice agent="fence_ipdu" community="ipdusnmpcom"
ipaddr="ipduhost" login="root" name="ipdutest1" \
passwd="password123" power_wait="60"
snmp_priv_passwd="ipduprivprotpasswd" snmp_priv_prot="AES" \
udpport="111"/>
</fencedevices>
4 .20. IF-MIB
Table 4.21, IF MIB lists the fence device parameters used by fence_i fmi b, the fence agent for IFMIB devices.
T ab le 4 .21. IF MIB
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
UD P/TCP Port
(optional)
Login
Password
Password
Script
(optional)
SNMP Version
name
i pad d r
ud ppo rt
The UD P/TCP port to use for connection with the device; the
default value is 161.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP Privacy
Protocol
Password
70
login
passwd
passwd _scri
pt
snmp_versi o
n
co mmuni ty
The SNMP version to use (1, 2c, 3); the default value is 1.
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
lu ci Field
cl uster. co n
f At t rib u t e
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
D escrip t io n
The script that supplies a password for SNMP privacy protocol.
Using this supersedes the SN MP p rivacy p ro t o co l p asswo rd
parameter.
Number of seconds to wait after issuing a power off or power on
command.
Number of seconds to wait before testing for a status change
after issuing a power on or power on command. The default
value is 20.
Number of seconds to wait for a command prompt after issuing a
command. The default value is 3.
Number of seconds to wait for a command prompt after login.
The default value is 5.
Number of attempts to retry a power on operation. The default
value is 1.
po rt
d el ay
Figure 4.17, IF-MIB shows the configuration screen for adding an IF-MIB fence device.
71
<fencedevices>
72
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
UD P/TCP Port
(optional)
Login
Password
Password
Script
(optional)
SNMP Version
name
i pad d r
ud ppo rt
The UD P/TCP port to use for connection with the device; the
default value is 161.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
SNMP
Community
SNMP Security
Level
SNMP
Authentication
Protocol
SNMP Privacy
Protocol
SNMP Privacy
Protocol
Password
SNMP Privacy
Protocol Script
login
passwd
passwd _scri
pt
snmp_versi o
n
co mmuni ty
The SNMP version to use (1, 2c, 3); the default value is 1.
snmp_sec_l e
vel
snmp_auth_p
ro t
snmp_pri v_p
ro t
snmp_pri v_p
asswd
snmp_pri v_p
asswd _scri p
t
po wer_wai t
Power Wait
(seconds)
Power Timeout po wer_ti meo
(seconds)
ut
Shell Timeout
(seconds)
Login Timeout
(seconds)
shel l _ti me
o ut
l o g i n_ti me
o ut
73
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Times to Retry
Power On
Operation
Port (Outlet)
Number
D elay
(optional)
retry_o n
po rt
d el ay
Figure 4.18, Intel Modular shows the configuration screen for adding an Intel Modular fence device.
74
The following command creates a fence device instance for an Intel Modular device:
ccs -f cluster.conf --addfencedev intelmodular1 agent=fence_intelmodular
community=private ipaddr=192.168.0.1 login=root \
passwd=password123 snmp_priv_passwd=snmpasswd123 power_wait=60
udpport=161
The following is the cl uster. co nf entry for the fence_i ntel mo d ul ar device:
<fencedevices>
<fencedevice agent="fence_intelmodular" community="private"
ipaddr="192.168.0.1" login="root" name="intelmodular1" \
passwd="password123" power_wait="60" snmp_priv_passwd="snmpasswd123"
udpport="161"/>
</fencedevices>
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
Login
name
i pad d r
login
Password
Password
Script
(optional)
Authentication
Type
Use Lanplus
passwd
passwd _scri
pt
auth
l anpl us
Ciphersuite to
ci pher
use
Privilege level
pri vl vl
IPMI Operation ti meo ut
Timeout
75
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Power Wait
(seconds)
po wer_wai t
D elay
(optional)
d el ay
Figure 4.19, IPMI over LAN shows the configuration screen for adding an IPMI over LAN device
76
<fencedevices>
<fencedevice agent="fence_ipmilan" auth="password" cipher="3"
ipaddr="192.168.0.1" lanplus="on" login="root" \
name="ipmitest1" passwd="password123"/>
</fencedevices>
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Family
IP Port
(optional)
Operation
Timeout
(seconds)
(optional)
Node name
name
fami l y
i ppo rt
ti meo ut
no d ename
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
IP Port
(optional)
Login
Password
Password
Script
(optional)
Use SSL
Power Wait
(seconds)
name
i pad d r
i ppo rt
login
passwd
passwd _scri
pt
ssl
po wer_wai t
77
lu ci Field
cl uster. co n
f At t rib u t e
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
D escrip t io n
Number of seconds to wait before testing for a status change
after issuing a power on or power on command. The default
value is 20.
Number of seconds to wait for a command prompt after issuing a
command. The default value is 3.
Number of seconds to wait for a command prompt after login.
The default value is 5.
Number of attempts to retry a power on operation. The default
value is 1.
po rt
d el ay
Figure 4.20, RHEV-M REST API shows the configuration screen for adding an RHEV-M REST API
device
78
<fencedevices>
<fencedevice agent="fence_rhevm" ipaddr="192.168.0.1" login="root"
name="rhevmtest1" passwd="password123" \
power_wait="60" ssl="on"/>
</fencedevices>
Note
Use of SCSI persistent reservations as a fence method is supported with the following
limitations:
When using SCSI fencing, all nodes in the cluster must register with the same devices so
that each node can remove another node's registration key from all the devices it is
registered with.
D evices used for the cluster volumes should be a complete LUN, not partitions. SCSI
persistent reservations work on an entire LUN, meaning that access is controlled to each
LUN, not individual partitions.
It is recommended that devices used for the cluster volumes be specified in the format
/d ev/d i sk/by-i d /xxx where possible. D evices specified in this format are consistent
among all nodes and will point to the same disk, unlike devices specified in a format such as
/d ev/sd a which can point to different disks from machine to machine and across reboots.
cl uster. co n
f At t rib u t e
D escrip t io n
Name
Unfencing
name
unfence
section of the
cluster
configuration
file
Node name
no d ename
79
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
key
D elay
(optional)
d el ay
(overrides node name) Key to use for the current operation. This
key should be unique to a node. For the " on" action, the key
specifies the key use to register the local node. For the " off"
action,this key specifies the key to be removed from the device(s).
The number of seconds to wait before fencing is started. The
default value is 0.
Figure 4.21, SCSI Fencing shows the configuration screen for adding an SCSI fence device
<fencedevices>
<<fencedevice agent="fence_scsi" name="scsifencetest1"/>
</fencedevices>
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
name
i pad d r
80
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
IP Port
(optional)
Login
Password
Password
Script
(optional)
Power Wait
(seconds)
Power Timeout
(seconds)
i ppo rt
The TCP port to use for connection with the device. The default
port is 80, or 443 if Use SSL is selected.
The login name used to access the device.
The password used to authenticate the connection to the device.
The script that supplies a password for access to the fence
device. Using this supersedes the Passwo rd parameter.
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
VM name
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
VM UUID
D elay
(optional)
Use SSL
uui d
d el ay
login
passwd
passwd _scri
pt
po wer_wai t
po wer_ti meo
ut
po rt
ssl
Figure 4.22, VMWare over SOAP Fencing shows the configuration screen for adding a VMWare
over SOAP fence device
81
<fencedevices>
<fencedevice agent="fence_vmware_soap" ipaddr="192.168.0.1"
login="root" name="vmwaresoaptest1" passwd="password123" \
power_wait="60" separator="."/>
</fencedevices>
82
lu ci Field
cl uster. co n
f At t rib u t e
D escrip t io n
Name
IP Address or
Hostname
IP Port
(optional)
Login
Password
Password
Script
(optional)
Force
command
prompt
Power Wait
(seconds)
Power Timeout
(seconds)
name
i pad d r
i ppo rt
login
passwd
passwd _scri
pt
po wer_wai t
Shell Timeout
(seconds)
Login Timeout
(seconds)
Times to Retry
Power On
Operation
Use SSH
shel l _ti me
o ut
l o g i n_ti me
o ut
retry_o n
SSH Options
Path to SSH
Identity File
Port
po wer_ti meo
ut
Indicates that system will use SSH to access the device. When
using SSH, you must specify either a password, a password
script, or an identity file.
ssh_o pti o ns SSH options to use. The default value is -1 -c bl o wfi sh.
i d enti ty_fi The identity file for SSH.
le
po rt
Physical plug number or name of virtual machine.
secure
Figure 4.23, WTI Fencing shows the configuration screen for adding a WTI fence device
83
<fencedevices>
<fencedevice agent="fence_wti" cmd_prompt="VMR& gt;"
ipaddr="192.168.0.1" login="root" name="wtipwrsw1" \
passwd="password123" power_wait="60"/>
</fencedevices>
84
Wed Ju l 8 2015
St even Levin e
R evisio n 2- 1
T h u Ap r 16 2015
Release for Beta of Red Hat Enterprise Linux 6.7
St even Levin e
R evisio n 1- 16
Resolves # 1023808
St even Levin e
Mo n Ap r 13 2015
Wed O ct 8 2014
St even Levin e
R evisio n 1- 11
T h u Au g 7 2014
Release for Beta of Red Hat Enterprise Linux 6.6
St even Levin e
R evisio n 1- 10
T h u Ju l 31 2014
Resolves: #856311
D ocuments fence_check man page.
St even Levin e
Resolves: #1104910
Updates fence parameter tables with new fence device parameters.
R evisio n 1- 9
Wed N o v 20 2013
Release for GA of Red Hat Enterprise Linux 6.5
Jo h n H a
R evisio n 1- 4
Mo n N o v 28 2012
Release for Beta of Red Hat Enterprise Linux 6.5
Jo h n H a
R evisio n 1- 2
Mo n N o v 28 2012
Release for Beta of Red Hat Enterprise Linux 6.4
Jo h n H a
Index
A
AC PI
- configuring, Configuring ACPI For Use with Integrated Fence D evices
APC p o wer swit ch o ver SN MP f en ce d evice , APC Po wer Swit ch o ver SN MP
APC p o wer swit ch o ver t eln et /SSH f en ce d evice , APC Po wer Swit ch o ver T eln et an d
SSH
B
B ro cad e f ab ric swit ch f en ce d evice , B ro cad e Fab ric Swit ch
85
D
D ell D R AC 5 f en ce d evice , D ell D rac 5
D ell iD R AC f en ce d evice , IPMI o ver LAN
E
Eat o n n et wo rk p o wer swit ch , Eat o n N et wo rk Po wer Swit ch
Eg en era B lad eFrame f en ce d evice , Eg en era B lad eFrame
Emerso n n et wo rk p o wer swit ch f en ce d evice , Emerso n N et wo rk Po wer Swit ch
( SN MP in t erf ace)
ePo werSwit ch f en ce d evice , ePo werSwit ch
F
f eed b ack
- contact information for this manual, We Need Feedback
f en ce
- configuration, Fencing Pre-Configuration
- devices, Fence D evices
f en ce ag en t
- fence_apc, APC Power Switch over Telnet and SSH
- fence_apc_snmp, APC Power Switch over SNMP
- fence_bladecenter, IBM BladeCenter
- fence_brocade, Brocade Fabric Switch
- fence_cisco_mds, Cisco MD S
- fence_cisco_ucs, Cisco UCS
- fence_drac5, D ell D rac 5
- fence_eaton_snmp, Eaton Network Power Switch
- fence_egenera, Egenera BladeFrame
- fence_emerson, Emerson Network Power Switch (SNMP interface)
- fence_eps, ePowerSwitch
- fence_hpblade, Hewlett-Packard BladeSystem
- fence_ibmblade, IBM BladeCenter over SNMP
- fence_idrac, IPMI over LAN
- fence_ifmib, IF-MIB
- fence_ilo, Hewlett-Packard iLO
- fence_ilo2, Hewlett-Packard iLO
- fence_ilo3, IPMI over LAN
- fence_ilo4, IPMI over LAN
- fence_ilo_mp, Hewlett-Packard iLO MP
- fence_imm, IPMI over LAN
- fence_intelmodular, Intel Modular
- fence_ipdu, IBM iPD U
- fence_ipmilan, IPMI over LAN
- fence_kdump, Fence kdump
- fence_rhevm, RHEV-M REST API
- fence_rsb, Fujitsu-Siemens RemoteView Service Board (RSB)
- fence_scsi, SCSI Persistent Reservations
86
87
H
h elp
- getting help, D o You Need Help?
H P B lad esyst em f en ce d evice , H ewlet t - Packard B lad eSyst em
H P iLO f en ce d evice, H ewlet t - Packard iLO
H P iLO MP f en ce d evice , H ewlet t - Packard iLO MP
H P iLO 2 f en ce d evice, H ewlet t - Packard iLO
H P iLO 3 f en ce d evice, IPMI o ver LAN
H P iLO 4 f en ce d evice, IPMI o ver LAN
88
R
R H EV- M R EST API f en ce d evice , R H EV- M R EST API
S
SC SI f en cin g , SC SI Persist en t R eservat io n s
SELin u x
- configuring, SELinux
T
t ab les
- fence devices, parameters, Fence D evices
V
VMware ( SO AP in t erf ace) f en ce d evice , VMWare o ver SO AP API
W
WT I p o wer swit ch f en ce d evice , WT I Po wer Swit ch
89