[GH-ISSUE #306] Live Daily Backup Of VM's

kerem commented

2026-02-27 16:38:32 +03:00

Owner

Originally created by @jtsoftex on GitHub (May 13, 2014).
Original GitHub issue: https://github.com/retspen/webvirtmgr/issues/306

I guess we someone else raised this issue here and I saw there are few lines here on it but I can't see a proper solution.

Can we add in UI something like take a daily backup of VM's in backend and let them there for say 7 days and then do like rollover.

Also for now is there any command/script for it so that we can take live backup for full VM without stopping it..

Apperaciate your early response on it.

Thanks

Originally created by @jtsoftex on GitHub (May 13, 2014). Original GitHub issue: https://github.com/retspen/webvirtmgr/issues/306 I guess we someone else raised this issue here and I saw there are few lines here on it but I can't see a proper solution. Can we add in UI something like take a daily backup of VM's in backend and let them there for say 7 days and then do like rollover. Also for now is there any command/script for it so that we can take live backup for full VM without stopping it.. Apperaciate your early response on it. Thanks

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@retspen commented on GitHub (May 25, 2014):

Live backup have some problem need suspend VM and then copy your hdd image.

@retspen commented on GitHub (May 25, 2014): Live backup have some problem need suspend VM and then copy your hdd image.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@AlexH-HankIT commented on GitHub (May 25, 2014):

What about LVM Snapshots for Live Backup?

@AlexH-HankIT commented on GitHub (May 25, 2014): What about LVM Snapshots for Live Backup?

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@retspen commented on GitHub (May 25, 2014):

Libvirt doesn't support LVM Snapshots

@retspen commented on GitHub (May 25, 2014): Libvirt doesn't support LVM Snapshots

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@AlexH-HankIT commented on GitHub (May 25, 2014):

Libvirt doesn't have to support it. You can always create an LVM snapshot, if you installed lvm correctly. Can webvirtmgr execute a bash command?

@AlexH-HankIT commented on GitHub (May 25, 2014): Libvirt doesn't have to support it. You can always create an LVM snapshot, if you installed lvm correctly. Can webvirtmgr execute a bash command?

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@retspen commented on GitHub (May 27, 2014):

WebVirtMgr can manage VM's trough libvirt API and can't execute a bash command.

@retspen commented on GitHub (May 27, 2014): WebVirtMgr can manage VM's trough libvirt API and can't execute a bash command.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@retspen commented on GitHub (May 30, 2014):

#!/bin/bash

DAY=$(date +"%w")
# Get a list of the virtual machines that are running on the server.
vmlist=`/usr/bin/virsh list --all | grep running | awk {'print $2'}`

# For each virtual machine on the virtual host server, get the list of disk images that are in use.
for vm in $vmlist
        do
        # Suspend the virtual server
        /usr/bin/virsh suspend $vm
        #echo "$vm suspended at" `date`

        # For each disk, copy it to the temporary rsnapshot directory
        #echo "Backing up $vm.img from $vm"
        hda=`virsh domblklist $vm | grep vda | awk {' print $2'}`
        /usr/bin/rsync --inplace $hda /mnt/backup/images/$DAY-$vm.img

        # Resume the virtual server
        /usr/bin/virsh resume $vm
        #echo "$vm resumed at" `date`
done

@retspen commented on GitHub (May 30, 2014): ``` #!/bin/bash DAY=$(date +"%w") # Get a list of the virtual machines that are running on the server. vmlist=`/usr/bin/virsh list --all | grep running | awk {'print $2'}` # For each virtual machine on the virtual host server, get the list of disk images that are in use. for vm in $vmlist do # Suspend the virtual server /usr/bin/virsh suspend $vm #echo "$vm suspended at" `date` # For each disk, copy it to the temporary rsnapshot directory #echo "Backing up $vm.img from $vm" hda=`virsh domblklist $vm | grep vda | awk {' print $2'}` /usr/bin/rsync --inplace $hda /mnt/backup/images/$DAY-$vm.img # Resume the virtual server /usr/bin/virsh resume $vm #echo "$vm resumed at" `date` done ```

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@MACscr commented on GitHub (Jun 19, 2014):

Why do guests have to be suspended in order to be backed up? I mean, daily backups of kvm instances is a pretty common feature for vm panels. Servers typically run 24/7 and having to suspend or shutdown to backup doesnt sound to great to me.

@MACscr commented on GitHub (Jun 19, 2014): Why do guests have to be suspended in order to be backed up? I mean, daily backups of kvm instances is a pretty common feature for vm panels. Servers typically run 24/7 and having to suspend or shutdown to backup doesnt sound to great to me.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@retspen commented on GitHub (Jun 19, 2014):

No problem you can copy a image without suspend and shutdown but when you coping the image when I write data to your database maybe something can be lost.

@retspen commented on GitHub (Jun 19, 2014): No problem you can copy a image without suspend and shutdown but when you coping the image when I write data to your database maybe something can be lost.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@jtsoftex commented on GitHub (Jun 23, 2014):

restpen, thanks for sharing script!!

Also, if we want to take like 3 days rollover backup then we can add this line the end, may be it will help some.

"find /path/to/backup/*.img -mtime +3 -exec rm {} ;"
where +3 means it will delete files those are 3 days old. You can change that value according to you.

Also, restpen, can we add this in UI with variables ?

@jtsoftex commented on GitHub (Jun 23, 2014): restpen, thanks for sharing script!! Also, if we want to take like 3 days rollover backup then we can add this line the end, may be it will help some. "find /path/to/backup/*.img -mtime +3 -exec rm {} \;" where +3 means it will delete files those are 3 days old. You can change that value according to you. Also, restpen, can we add this in UI with variables ?

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@retspen commented on GitHub (Jun 25, 2014):

No, we can't. Libvirt can't have access to console.

@retspen commented on GitHub (Jun 25, 2014): No, we can't. Libvirt can't have access to console.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@lub commented on GitHub (Jul 4, 2014):

Maybe there is a solution with external Snapshots (at least for qcow2)?

Create an external Snapshot -> copy the original image -> delete the Snapshot

@lub commented on GitHub (Jul 4, 2014): Maybe there is a solution with external Snapshots (at least for qcow2)? Create an external Snapshot -> copy the original image -> delete the Snapshot

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@AlexH-HankIT commented on GitHub (Jul 4, 2014):

I tried that a while ago. But it seems like libvirt cant remove an external snapshot while the vm is running.
http://wiki.libvirt.org/page/I_created_an_external_snapshot,_but_libvirt_won%27t_let_me_delete_or_revert_to_it

LVM Snapshots are perfect for backup, but since libvirt doesnt support them you have to do it without gui. I wrote a script to backup vms on logical volumes with filesystems (ext4 for example). It is executed via cronjob every friday. I can provide the script if anybody is interested.

@AlexH-HankIT commented on GitHub (Jul 4, 2014): I tried that a while ago. But it seems like libvirt cant remove an external snapshot while the vm is running. http://wiki.libvirt.org/page/I_created_an_external_snapshot,_but_libvirt_won%27t_let_me_delete_or_revert_to_it LVM Snapshots are perfect for backup, but since libvirt doesnt support them you have to do it without gui. I wrote a script to backup vms on logical volumes with filesystems (ext4 for example). It is executed via cronjob every friday. I can provide the script if anybody is interested.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@brylie commented on GitHub (Oct 22, 2014):

Thanks @retspen :-) How can this script be made to write to a log file? I.e. it will be running as a cron job, and it would be nice to see a log of success/failure messages.

@brylie commented on GitHub (Oct 22, 2014): Thanks @retspen :-) How can this script be made to write to a log file? I.e. it will be running as a cron job, and it would be nice to see a log of success/failure messages.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@brylie commented on GitHub (Oct 22, 2014):

I am having difficulty getting the script to work. It stops at "Backing up ..." and never displays the "... resumed at" statement.

Here is my bash file:

#!/bin/bash

backup_directory='/var/backups/images/'

DAY=$(date +"%Y-%m-%d")
# Get a list of the virtual machines that are running on the server.
vmlist=`/usr/bin/virsh list --all | grep running | awk {'print $2'}`

# For each virtual machine on the virtual host server, get the list of disk images that are in use.
for vm in $vmlist
        do
        # Suspend the virtual server
        /usr/bin/virsh suspend $vm
        echo "$vm suspended at" `date`

        # For each disk, copy it to the temporary rsnapshot directory
        echo "Backing up $vm.img from $vm"
        hda=`virsh domblklist $vm | grep vda | awk {' print $2'}`
        /usr/bin/rsync --inplace $hda $backup_directory$DAY-$vm.img

        # Resume the virtual server
        /usr/bin/virsh resume $vm
        echo "$vm resumed at" `date`
done

@brylie commented on GitHub (Oct 22, 2014): I am having difficulty getting the script to work. It stops at "Backing up ..." and never displays the "... resumed at" statement. Here is my bash file: ``` bash #!/bin/bash backup_directory='/var/backups/images/' DAY=$(date +"%Y-%m-%d") # Get a list of the virtual machines that are running on the server. vmlist=`/usr/bin/virsh list --all | grep running | awk {'print $2'}` # For each virtual machine on the virtual host server, get the list of disk images that are in use. for vm in $vmlist do # Suspend the virtual server /usr/bin/virsh suspend $vm echo "$vm suspended at" `date` # For each disk, copy it to the temporary rsnapshot directory echo "Backing up $vm.img from $vm" hda=`virsh domblklist $vm | grep vda | awk {' print $2'}` /usr/bin/rsync --inplace $hda $backup_directory$DAY-$vm.img # Resume the virtual server /usr/bin/virsh resume $vm echo "$vm resumed at" `date` done ```

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@brylie commented on GitHub (Oct 22, 2014):

I added the --progress flag to rsync, and see that the backup just takes a while. Some of our images are >100Gb.

Is there a way to do this with less downtime?

@brylie commented on GitHub (Oct 22, 2014): I added the --progress flag to rsync, and see that the backup just takes a while. Some of our images are >100Gb. Is there a way to do this with less downtime?

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@AlexH-HankIT commented on GitHub (Oct 22, 2014):

I wouldn't suspend the vm at all. You should always use LVM for your virtual machines. It's snapshot capabilities also allows a live backup of your vms.

If you run the script via cron the output will automatically send via mail to root@domain. You can also redirect it into a log file. Just put "> /var/log/logfile" behind the line in your /etc/crontab.

@AlexH-HankIT commented on GitHub (Oct 22, 2014): I wouldn't suspend the vm at all. You should always use LVM for your virtual machines. It's snapshot capabilities also allows a live backup of your vms. If you run the script via cron the output will automatically send via mail to root@domain. You can also redirect it into a log file. Just put "> /var/log/logfile" behind the line in your /etc/crontab.

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@brylie commented on GitHub (Oct 24, 2014):

I just want to make sure that the image is not corrupted. Would the following script be advisable @MrCrankHank?

#!/bin/bash

backup_directory='/var/backups/images/'

DAY=$(date +"%Y-%m-%d")

# Get a list of the virtual machines that are running on the server.
vmlist=`/usr/bin/virsh list --all | grep running | awk {'print $2'}`

# For each virtual machine on the virtual host server, get the list of disk images that are in use.
for vm in $vmlist
        do
        # For each disk, copy it to the temporary rsnapshot directory
        hda=`virsh domblklist $vm | grep vda | awk {' print $2'}`
        /usr/bin/rsync --inplace $hda $backup_directory$DAY-$vm.img

        # Resume the virtual server
        /usr/bin/virsh resume $vm
done

@brylie commented on GitHub (Oct 24, 2014): I just want to make sure that the image is not corrupted. Would the following script be advisable @MrCrankHank? ``` bash #!/bin/bash backup_directory='/var/backups/images/' DAY=$(date +"%Y-%m-%d") # Get a list of the virtual machines that are running on the server. vmlist=`/usr/bin/virsh list --all | grep running | awk {'print $2'}` # For each virtual machine on the virtual host server, get the list of disk images that are in use. for vm in $vmlist do # For each disk, copy it to the temporary rsnapshot directory hda=`virsh domblklist $vm | grep vda | awk {' print $2'}` /usr/bin/rsync --inplace $hda $backup_directory$DAY-$vm.img # Resume the virtual server /usr/bin/virsh resume $vm done ```

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@brylie commented on GitHub (Oct 24, 2014):

You should always use LVM for your virtual machines. It's snapshot capabilities also allows a live backup of your vms.

@MrCrankHank does this mean that we need to select LVM when installing the guest operating system? Does this also mean that the host OS should have LVM enabled?

@brylie commented on GitHub (Oct 24, 2014): <blockquote> You should always use LVM for your virtual machines. It's snapshot capabilities also allows a live backup of your vms. </blockquote> @MrCrankHank does this mean that we need to select LVM when installing the guest operating system? Does this also mean that the host OS should have LVM enabled?

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@AlexH-HankIT commented on GitHub (Oct 25, 2014):

The script should work. but i prefer a live backup of my vms, because some vms (Windows) have problems with suspending. If your vms are fine with suspending and you don't care about the downtime, then go ahead and use the script. But if there problems i strongly recommend to configure lvm on the storage where your vms are. What storage do you use currently?

@AlexH-HankIT commented on GitHub (Oct 25, 2014): The script should work. but i prefer a live backup of my vms, because some vms (Windows) have problems with suspending. If your vms are fine with suspending and you don't care about the downtime, then go ahead and use the script. But if there problems i strongly recommend to configure lvm on the storage where your vms are. What storage do you use currently?

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@brylie commented on GitHub (Oct 25, 2014):

I generally choose the LVM option when installing guest OSs, but cannot be certain of VMs installed by other admins.

Can LVM be enabled after the initial installation? How are you performing the live backups?

@brylie commented on GitHub (Oct 25, 2014): I generally choose the LVM option when installing guest OSs, but cannot be certain of VMs installed by other admins. Can LVM be enabled after the initial installation? How are you performing the live backups?

kerem commented

2026-02-27 16:38:33 +03:00

Author

Owner

@AlexH-HankIT commented on GitHub (Oct 25, 2014):

You don't need lvm inside the guest. It's important that the guests filesystem lives on a lvm volume. On my host i have a Volume Group called VG_data01. In this volume group i have a logical volume for every guest harddrive. If i want to create a backup of one vm, i snapshot the logical volume of that guest an copy the file system with dd (then gzip it of course...). Of course this can be easily scripted.

As far as i know you can add a Volume Group under "Storages" in Webvirtmgr. If you do so, libvirt should take care of creating the logical volumes for your vms (don't know for sure so, i'm using another product).

One downside of this setup is, that you can't use thin provisioning.

@AlexH-HankIT commented on GitHub (Oct 25, 2014): You don't need lvm inside the guest. It's important that the guests filesystem lives on a lvm volume. On my host i have a Volume Group called VG_data01. In this volume group i have a logical volume for every guest harddrive. If i want to create a backup of one vm, i snapshot the logical volume of that guest an copy the file system with dd (then gzip it of course...). Of course this can be easily scripted. As far as i know you can add a Volume Group under "Storages" in Webvirtmgr. If you do so, libvirt should take care of creating the logical volumes for your vms (don't know for sure so, i'm using another product). One downside of this setup is, that you can't use thin provisioning.

Rows
Columns

[GH-ISSUE #306] Live Daily Backup Of VM's #239