Wednesday, April 30, 2008

Ubuntu LVM on Software RAID

I've been playing with LVM and software RAID on Ubuntu. Install the packages:
apt-get install lvm2 dmsetup mdadm 
After doing the above you'll want to reboot with your new initrd so that you have the kernel modules to do an lvcreate later. I've got a cheap USB device which I'll set up with software RAID1 on two partitions for fun:
$ fdisk /dev/sdb
...
Command (m for help): p

Disk /dev/sdb: 257 MB, 257949696 bytes
8 heads, 62 sectors/track, 1015 cylinders
Units = cylinders of 496 * 512 = 253952 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         505      125209   fd  Linux raid autodetect
/dev/sdb2             506        1010      125240   fd  Linux raid autodetect  
Create the RAID1 device from the partitions:
$ mdadm --build /dev/md0 --level=1 --raid-devices=2 /dev/sdb{1,2}
mdadm: array /dev/md0 built and started.
$ 
Make a file system on the new device while I watch the RAID sync:
$ mkfs -j /dev/md0 &
$ while [ 1 ]; do cat /proc/mdstat; sleep 4; done
Personalities : [raid1] 
md0 : active raid1 sdb2[1] sdb1[0]
      125209 blocks super non-persistent [2/2] [UU]
      [==================>..]  resync = 91.0% (114944/125209) finish=0.3min speed=506K/sec
      
unused devices: 
Personalities : [raid1] 
md0 : active raid1 sdb2[1] sdb1[0]
      125209 blocks super non-persistent [2/2] [UU]
      [==================>..]  resync = 92.6% (116928/125209) finish=0.2min speed=504K/sec
      
unused devices: 
....
Personalities : [raid1] 
md0 : active raid1 sdb2[1] sdb1[0]
      125209 blocks super non-persistent [2/2] [UU]
      [===================>.]  resync = 99.1% (125120/125209) finish=0.0min speed=519K/sec
      
unused devices: 
Personalities : [raid1] 
md0 : active raid1 sdb2[1] sdb1[0]
      125209 blocks super non-persistent [2/2] [UU]
Mount the new filesystem to test it. Then umount it.
$ mount /dev/md0 /mnt/data/
$ df -h /mnt/data/
Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              119M  5.6M  107M   5% /mnt/data
$ umount !$
Add the RAID to an LVM by creating a physical volume, a volume group then a logical volume:
pvcreate /dev/md0
vgcreate volgroup00 /dev/md0
lvcreate -n logvol00 -l 30 volgroup00
Remember physical volumes get added to volume groups and then we add logical volumes so I'm going through the above steps quickly to make it clear how the above are linked together. You can {pv,vg,lv}display at each step of the above as appropriate (that's how I got 30 extents). Make a filesystem and then mount your volume:
mkfs -j /dev/volgroup00/logvol00
mount /dev/volgroup00/logvol00 /mnt/data/
Now we want to extend it in a very inconvenient way (for sadism). We'll undo everything and build it again but with two smaller RAID1 devices... it's funny if you pretend you're in a Monty Python script. Note the LVM stack going back down in reverse:
lvremove /dev/volgroup00/logvol00
vgremove volgroup00
pvremove /dev/md0
Disassemble your RAID device and verify that it's gone:
mdadm -S /dev/md0
cat /proc/mdstat
Finally go back to fdisk and delete your partitions and rebuild them in quaters:
$ fdisk -l /dev/sdb

Disk /dev/sdb: 257 MB, 257949696 bytes
8 heads, 62 sectors/track, 1015 cylinders
Units = cylinders of 496 * 512 = 253952 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         253       62713   fd  Linux raid autodetect
/dev/sdb2             254         506       62744   fd  Linux raid autodetect
/dev/sdb3             507         759       62744   fd  Linux raid autodetect
/dev/sdb4             760        1012       62744   fd  Linux raid autodetect
$
Now create two RAID1 devices:
$ mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb{1,2}
mdadm: array /dev/md0 started.
$ mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb{3,4}
mdadm: array /dev/md1 started.
$ 
It will be syncing md0 first and then md1, but we can go on:
$ cat /proc/mdstat 
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sdb4[1] sdb3[0]
      62656 blocks [2/2] [UU]
        resync=DELAYED
      
md0 : active raid1 sdb2[1] sdb1[0]
      62592 blocks [2/2] [UU]
      [=========>...........]  resync = 48.3% (30720/62592) finish=1.0min speed=498K/sec
Add the first RAID device to a physical volume:
$ pvcreate /dev/md0
  Physical volume "/dev/md0" successfully created
but don't add the second one yet, we'll do that later to extend. Next create a volume group and then a logical volume. Then we make a file system and mount it:
vgcreate volgroup /dev/md0
lvcreate -n data -l 15 volgroup
mkfs -j  /dev/volgroup/data
mount /dev/volgroup/data /mnt/data/
Copy some awesome music to your new LVM (What's LVM practice without early 90's metal?):
cp ~/tunes/disincarnate/Dreams\ of\ the\ Carrion\ Kind/* /mnt/data/
You might even want to play one of the songs on your device as you extend the volume. Add the other RAID device as an available physical volume for LVM and then extend volgroup to use it:
pvcreate /dev/md1
vgextend volgroup /dev/md1
vgdisplay will show that you have more space. Note also the current size of /mnt/data:
$ vgdisplay | grep 15
  Alloc PE / Size       15 / 60.00 MB
  Free  PE / Size       15 / 60.00 MB
$ df -h /mnt/data | grep M
Filesystem            Size  Used Avail Use% Mounted on
                       59M   52M  4.0M  93% /mnt/data
$ 
Extend volgroup into the extra 15 extents:
$ lvextend -l +15 /dev/volgroup/data 
  Extending logical volume data to 120.00 MB
  Logical volume data successfully resized
$
Then extend the filesystem (while listening the the music on the same filesystem):
$ resize2fs /dev/mapper/volgroup-data
resize2fs 1.40.2 (12-Jul-2007)
Filesystem at /dev/mapper/volgroup-data is mounted on /mnt/data; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/mapper/volgroup-data to 122880 (1k) blocks.
The filesystem on /dev/mapper/volgroup-data is now 122880 blocks long.

$
Note the new size:
$ df -h /mnt/data | grep M
Filesystem            Size  Used Avail Use% Mounted on
                      117M   52M   60M  47% /mnt/data
$
Now I've got a USB device with four partitions and each pair is in a RAID1 as well as the member of an LVM full of obscure metal. What else should one have in his pocket?

No comments: