Xen Backups

Earlier I pointed out how rapleaf.com migrated xen VMs with minimal downtime by using LVM snapshots. I wrote a script to do this very thing to backup my VMs and it works well.

I have three Xen servers and each copies its VMs to the other [backup_target = server_N + 1 % count(xen_servers)]. I shut down each VM, make a snapshot and bring each back up. This results in a 60 second outage for all of the VMs plus boot time with doing a virsh shutdown and then xm create for each VM. I am copying only the image files from /var/lib/xen/images/ and each VM's config file from /etc/xen/.

The majority of time is then spent doing an rsync, however it is not time without service because the VMs would already be back up and the snapshot serves a stable shutdown VM for the backup. My rsync user has the scpsftprsynconly shell, which works nicely and is cleaner than setting up chroot jail. I then only need to boot the VMs on their other host to restore service if there is a hardware problem.