LVM Troubleshooting 3

LVM Troubleshooting 3
--> How to Grow an LVM Physical Volume after resizing the disk?
Note : This procedure does have the potential to lose data on the disk if done i
mproperly, we strongly recommend a backup be performed before proceeding.
For example, we have resized a disk from 50Gb to 120Gb
# pvs
PV VG Fmt Attr PSize PFree
/dev/sdb1 VolGroup02 lvm2 a 50.00g 2.00g
Whilst the underlying storage (eg: /dev/sdb) may have been resized, the partitio
n we are using as a physical volume (eg: /dev/sdb1) remains at the smaller size.
Well need to resize the partition, then resize the Physical Volume before you can
proceed.
First we would confirm the actual storage size with fdisk -ul /dev/sdb and observe
the increased disk size. Depending on how the storage is presented, we may need
to reboot for this to appear.
We will then need to resize the partition on the disk. We can achieve this by ob
serving the starting sector in fdisk -ul /dev/sdb, then removing the partition w
ith fdisk and re-creating it with the same starting sector but the (default) las
t sector of the drive as the ending sector. Then write the partition table and c
onfirm the change (and the correct starting sector) with fdisk -ul /dev/sdb.
Now We are ready to pvresize /dev/sdb1 to grow the PV onto the rest of the expan
ded partition.This will create free extents within the Volume Group which we can
then grow a Logical Volume into.
If we run your LV resize with lvresize -r, it will grow the filesystem we have w
ithin the Logical Volume as well.
--> Delete a LVM partition
Delete LVM partition from the /etc/fstab For example:
/dev/sda2 / ext3 defaults 1 1
/dev/sda1 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/sd3 swap swap defaults 0 0
/dev/volumegroup/lvm /var ext3 defaults 0 0
umount LVM partition
# umount /dev/volumegroup/lvm
Disable lvm
# lvchange -an /dev/volumegroup/lvm
Delete lvm volume
# lvremove /dev/volumegroup/lvm
Disable volume group
# vgchange -an volumegroup
Delete volume group
# vgremove volumegroup
Delete phisical Volume
# pvremove /dev/sdc1 /dev/sdc2
--> Restore a volume group in Red Hat Enterprise Linux if one of the physical vo
lumes that constitutes the volume group has failed.
NOTE : These commands have the potential to corrupt data, should be executed at
ones own discretion
This procedure requires a recent backup of the LVM configuration. This can be ge
nerated with the command vgcfgbackup and is stored in the file /etc/lvm/backup/<
volume group name>.
The /etc/lvm/archive directory also contains recent configurations that are crea
ted when modifications to the volume group metadata are made. It is recommended
that these files be regularly backed up to a safe location so that they will be
available if required for recovery purposes.
Assuming a physical volume has been lost that was a part of a volume group the f
ollowing procedure may be followed. The procedure will replace the failed physic
al volume with a new disk rendering any remaining logical volumes accessible for
recovery purposes.
The procedure for recovery is as follows:
1. Execute the following command to display information about the volume group i
n question:
# vgdisplay partial verbose
The output will be similar to the following (note that the partial flag is requir
ed to activate or manipulate a volume group having one or more physical volumes
missing and that use of this flag with LVM2 activation commands (vgchange -a) wi
ll force volumes to be activated in a read-only state):
Partial mode. Incomplete volume groups will be activated read-only.
Finding all volume groups
Finding volume group volGroup00?
Couldnt find device with uuid ?eWicl-1HSB-Fkcz-wrMf-DzMd-Dgx2-Kyc11j.
Couldnt find device with uuid ?eWicl-1HSB-Fkcz-wrMf-DzMd-Dgx2-Kyc11j.
Couldnt find device with uuid ?eWicl-1HSB-Fkcz-wrMf-DzMd-Dgx2-Kyc11j.
Couldnt find device with uuid ?eWicl-1HSB-Fkcz-wrMf-DzMd-Dgx2-Kyc11j.
Volume group
VG Name volGroup00
System ID
Format lvm2
Metadata Areas 4
Metadata Sequence No 33
VG Access read
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 5
Act PV 5
VG Size 776.00 MB
PE Size 4.00 MB
Total PE 194
Alloc PE / Size 194 / 776.00 MB
Free PE / Size 0 / 0
VG UUID PjnqwZ-AYXR-BUyo-9VMN-uSRZ-AFlj-WOaA6z
Logical volume
LV Name /dev/volGroup00/myLVM
VG Name volGroup00
LV UUID az6REi-mkt5-sDpS-4TyH-GBj2-cisD-olf6SW
LV Write Access read/write
LV Status available
# open 0
LV Size 776.00 MB
Current LE 194
Segments 5
Allocation inherit
Read ahead sectors 0
Block device 253:0
Physical volumes
PV Name /dev/hda8
PV UUID azYDV8-e2DT-oxGi-5S9Q-yVsM-dxoB-DgC4qN
PV Status allocatable
Total PE / Free PE 48 / 0
PV Name /dev/hda10
PV UUID SWICqb-YIbb-g1MW-CY60-AkNQ-gNBu-GCMWOi
PV Status allocatable
Total PE / Free PE 48 / 0
PV Name /dev/hda11
PV UUID pts536-Ycd5-kNHR-VMZY-jZRv-nTx1-XZFrYy
PV Status allocatable
Total PE / Free PE 48 / 0
PV Name /dev/hda14
PV UUID OtIMPe-SZK4-arxr-jGlp-eiHY-2OA6-kyntME
PV Status allocatable
Total PE / Free PE 25 / 0
PV Name unknown device
PV UUID 9eWicl-1HSB-Fkcz-wrMf-DzMd-Dgx2-Kyc11j
PV Status allocatable
Total PE / Free PE 25 / 0
Note the PV UUID line:
PV UUID 9eWicl-1HSB-Fkcz-wrMf-DzMd-Dgx2-Kyc11j
This line contains the universally unique identifier (UUID) of the physical volu
me that failed and will be needed in the next step.
2. If the physical volume failed, it must be replaced with a disk or partition t
hat is equal in size or larger than the failed volume. If the disk did not fail
but was overwritten or corrupted, the same volume can be re-used. Run the follow
ing command to re-initialize the physical volume:
# pvcreate restorefile /etc/lvm/backup/<volume group name> uuid <UUID> <device>
In the above command the UUID is the value taken from the output in step 1. In t
his example the full command would be:
# pvcreate restorefile /etc/lvm/backup/volGroup00 uuid 9eWicl-1HSB-Fkcz-wrMf-DzMd-
Dgx2-Kyc11j /dev/hda15
Couldnt find device with uuid 9eWicl-1HSB-Fkcz-wrMf-DzMd-Dgx2-Kyc11j.
Physical volume /dev/hda15? successfully created
Note that when overwriting a previously-used LVM2 physical volume (for example w
hen recovering from a situation where the volume had been inadvertently overwrit
ten) the -ff option must be given to the pvcreate command.
3. Now the new physical volume has been initialized with the UUID of the old phy
sical volume. The volume group metadata may be restored with the following comma
nd:
# vgcfgrestore file /etc/lvm/backup/<volume group name > <volume group name>
Continuing the earlier example the exact command would be:
# vgcfgrestore file /etc/lvm/backup/volGroup00 volGroup00
Restored volume group volGroup00
4. To check that the new physical volume is intact and the volume group is funct
ioning correctly execute vgdisplay -v.
Note: This procedure will not restore any data lost from a physical volume that
has failed and been replaced. If a physical volume has been partially overwritte
n (for example, the label or metadata regions have been damaged or destroyed) th
en user data may still exist in the data area of the volume and this may be reco
vered using standard tools after restoring access to the volume group using thes
e steps.
--> Logical Volume Manager (LVM) snapshot and how do we use it?
Logical Volume Manager (LVM) provides the ability to take a snapshot of any logi
cal volume for the purpose of obtaining a backup of a partition in a consistent
state. Traditionally the solution has been to mount the partition read-only, app
ly table-level write locks to databases or shut down the database engine etc.; a
ll measures which adversely impact availability (but not as much as data loss wi
thout a backup will). With LVM snapshots it is possible to obtain a consistent b
ackup without compromising availability.
The LVM snapshot works by logging the changes to the filesystem to the snapshot
partition, rather than mirroring the partition. Thus when you create a snapshot
partition you do not need to use space equal to the size of the partition that y
ou are taking a snapshot of, but rather the amount of changes that it will under
go during the lifetime of the snapshot. This is a function of both how much data
is being written to the partition and also how long you intend keeping the LVM
snapshot.
Below example shows about LVM snapshot creation. Here we create a logical volume
of 500MB to use to take a snapshot. This will allow 500MB of changes on the vol
ume we are taking a snapshot of during the lifetime of the snapshot.
The following command will create /dev/ops/dbbackup as a snapshot of /dev/ops/da
tabases.
# lvcreate -L500M -s -n dbbackup /dev/ops/databases
lvcreate WARNING: the snapshot must be disabled if it gets full
lvcreate INFO: using default snapshot chunk size of 64 KB for /dev/ops/dbbackup
lvcreate doing automatic backup of ops
lvcreate logical volume /dev/ops/dbbackup successfully created
Now we create the mount point and mount the snapshot.
# mkdir /mnt/ops/dbbackup
# mount /dev/ops/dbbackup /mnt/ops/dbbackup
mount: block device /dev/ops/dbbackup is write-protected, mounting read-only
After performing the backup of the snapshot partition we release the snapshot. T
he snapshot will be automatically released when it fills up, but maintaining inc
urs a system overhead in the meantime.
# umount /mnt/ops/dbbackup
# lvremove /dev/ops/dbbackup
lvremove do you really want to remove /dev/ops/dbbackup? [y/n]: y
lvremove doing automatic backup of volume group ops
lvremove logical volume /dev/ops/dbbackup successfully removed
--> How do we Create New LVM volume from a LVM snapshot?
create a sparse file, providing room for the volumegroup
# dd if=/dev/zero of=file bs=1 count=1 seek=3G
setup the sparse file as block device
# losetup -f file
create a volumegroup and volume
# pvcreate /dev/loop0
# vgcreate vgtest /dev/loop0
# lvcreate -l 10 -n lvoriginal vg00
create a filesystem, create content on it
# mkdir /mnt/tmp /mnt/tmp2
# mkfs.ext4 /dev/vg00/lvoriginal
# mount /dev/vg00/lvoriginal /mnt/tmp
# echo state1 >>/mnt/tmp/contents
create the mirror the volumegroup has to have enough free PEs
# lvconvert -m 1 /dev/vg00/lvoriginal
# lvconvert splitmirrors 1 -n lvclone /dev/vg00/lvoriginal
change the contents on the original volume
# echo state2 >>/mnt/tmp/contents
now access the clone volume and verify it represents the originals old state
# mount /dev/vgtest/lvolnew /mnt/tmp2
# cat /mnt/tmp2/contents
# cat /mnt/tmp/contents
--> How can I boot from an LVM snapshot on Red Hat Enterprise Linux?
The snapshot has to be in the same volume group as the original root logical vol
ume.Often, other file systems should be snapshotted at the same time (eg. /var,
/usr) if they are separate file systems to root.
Procedure:
Step 1 : Create a snapshot of any local filesystems (for RHEL6, it is recommende
d that you do not put a - in the name as it makes addressing the volume more compl
icated):
# lvcreate -s -n varsnapshot -L 1G /dev/VolGroup00/var
# lvcreate -s -n rootsnapshot -L 2G /dev/VolGroup00/root
Step 2 : Mount the root snapshot so we can change the /etc/fstab of the snapshot
version:
# mkdir /mnt/snapshot
# mount /dev/VolGroup00/rootsnapshot /mnt/snapshot
# vi /mnt/snapshot/etc/fstab
Step 3 : Change the entries in /mnt/etc/fstab to point to the snapshot volumes r
ather than the original devices:
/dev/VolGroup00/rootsnapshot / ext3 defaults 1 1
/dev/VolGroup00/varsnapshot /var ext3 defaults 1 2
Step 4 : Now unmount the snapshot:
# cd /tmp
# umount /mnt/snapshot
Step 5 : Add an entry in grub to boot into the snapshot:
Step 5a For Red Hat Enterprise Linux 5, copy the current default grub.conf entry
, and make a new entry pointing to the snapshot version:
/boot/grub/grub.conf entry before:

default=0

title Red Hat Enterprise Linux 5 (2.6.18-194.el5)

root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/root
initrd /initrd-2.6.18-194.el5.img
After:

default=0

title Snapshot (2.6.18-194.el5)

root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/rootsnapshot
initrd /initrd-2.6.18-194.el5.img
title Red Hat Enterprise Linux 5 (2.6.18-194.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/VolGroup00/root
initrd /initrd-2.6.18-194.el5.img
Step 5b For Red Hat Enterprise Linux 6, copy the default grub.conf entry, and ma
ake a new entry pointing to the snapshot version:
/boot/grub/grub.conf before:

default=0

title Red Hat Enterprise Linux Server (2.6.32-279.9.1.el6.x86_64)

root (hd0,0)
kernel /vmlinuz-2.6.32-279.9.1.el6.x86_64 ro root=/dev/mapper/VolGroup00-rootvol
rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crash
kernel=auto rd_LVM_LV=VolGroup00/rootvol KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM
initrd /initramfs-2.6.32-279.9.1.el6.x86_64.img
/boot/grub/grub.conf after:

default=0

title Snapshot (2.6.32-279.9.1.el6.x86_64)

root (hd0,0)
kernel /vmlinuz-2.6.32-279.9.1.el6.x86_64 ro root=/dev/mapper/VolGroup00-rootsna
pshot rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb
crashkernel=auto rd_LVM_LV=VolGroup00/rootvol KEYBOARDTYPE=pc KEYTABLE=us rd_NO_
DM
initrd /initramfs-2.6.32-279.9.1.el6.x86_64.img
title Red Hat Enterprise Linux Server (2.6.32-279.9.1.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-279.9.1.el6.x86_64 ro root=/dev/mapper/VolGroup00-rootvol
rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crash
kernel=auto rd_LVM_LV=VolGroup00/rootvol KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM
initrd /initramfs-2.6.32-279.9.1.el6.x86_64.img
NOTE: On the grub menu entry on RHEL6, change root= to point to the snapshot but D
O NOT change rd_LVM_LV to point to the snapshot, because this will prevent both
the real and snapshot devices from activating on boot. Snapshots cannot be activ
ated without the real volume being activated as well.
Step 6 : Now you can boot into the snapshot by choosing the correct grub menu en
try. To boot back onto the real LVM device, just select the original grub menu e
ntry.
Step 7 : You can verify that you are booted into the snapshot version by checkin
g which LVM device is mounted:
# mount | grep VolGroup00
/dev/mapper/VolGroup00-rootsnapshot on / type ext4 (rw)
/dev/mapper/VolGroup00-varsnapshot on /var type ext4 (rw)
You can remove the snapshot with the following procedure:
Step 1) Remove the grub entry from /boot/grub/grub.conf for your snapshot volume
.
Step 2) Boot into (or ensure you are already booted into) the real LVM volume:
# mount | grep VolGroup00
/dev/mapper/VolGroup00-root on / type ext4 (rw)
/dev/mapper/VolGroup00-var on /var type ext4 (rw)
Step 3) Remove the snapshot volumes:
# lvremove /dev/VolGroup00/rootsnapshot
# lvremove /dev/VolGroup00/varsnapshot
Summary
To boot into an LVM snapshot of the root filesystem, you must change only the fo
llowing locations:
/etc/fstab on the LVM snapshot volume (do not change fstab on the real volume)
/boot/grub/grub.conf to add an entry that points to the snapshot device as the r
oot disk.
There is no need to rebuild initrd to boot into the snapshot.