How does the Snapshot Work for Image-Based Protection?

At InterOp this week, the point really came home to me again that most people are simply confused about how the snapshot capability works with image-based protection. It's kind of hard to appreciate the value and benefits of an image-based data protection solution like vRanger Pro unless you understand how well it is designed to work with the VM image. So, let me take a few minutes to describe how this operates and hopefully clear up some of the confusion.

I find that pictures really help, so here is my offering:

This picture illustrates the entire process of how vRanger Pro initiates, executes, and transmits a backup of a VM image on VMware systems. I'll describe the entire process and how it works in a future blog entry. For now, focus on the snapshot process which happens on the VMware system.

Here's how it works:

1) VMware uses a snapshot process to release the VMDK file for the VM so that it can be read by the vRanger Pro backup process. This happens immediately and quickly, because the snapshot is not a mirror. There is no need to wait for a mirror to be written. There is no need to find duplicate disk space for a mirror.

2) The VMDK file is then available for backup. vRanger Pro reads out the blocks required for the backup, depending on whether the backup is a full, incremental or differential. The blocks read also depend on other capabilities of vRanger Pro which may be enabled, including Active Block Mapping and Change Block Tracking.

3) During backup, end-users and applications can still operate. VMware enables this by holding any new writes in a temporary location until the backup is complete, and the VMDK file is once again available for use.

4) When the backup is complete, VMware releases the snapshot - and resolves any changes held in the cache location by updating the affected blocks in the VMDK.

A common misconception is that an image-based backup is a snapshot. There is another misconception, which is that the backup is made from a snapshot copy of the VMDK image. Neither is correct. The image backup copy is a block-by-block read from the VM file. However, the backup gains these advantages from the fact that VMware releases its VMDK file through a snapshot mechanism:

  • End-users and applications can continue to work during the time of the backup
  • The entire system environment is frozen at a single point in time for the backup - which makes it more highly reliable upon recovery, since not only is data consistent to a point in time within an application but across the entire system's operating environment
  • The snapshot is very quick to make because it uses Copy on Write (COW) capabilities rather than a mirror
  • All of the blocks for the entire system are managed into a single file, which is faster and easier to manage with better compression and de-duplication, and faster transmission, than can be done by managing the hundreds or thousands of files found within the image individually

In short, the snapshot mechanism is a key advantage that helps to make Backup 2.0 image-based data collection less disruptive, more reliable, faster, and more resource-efficient than traditional methods.

Taking it a Bit Deeper for Those Who Care

On a VMware system, when a snapshot is created a number of files are created in the directory for that virtual machine:

  • <machine name>-SnapshotX.vmsn (Where X is the number of the snapshot taken) -- This file stores the state of the virtual machine when the snapshot was taken.
  • <machine name>-SnapshotX.vmem (Where X is the number of the snapshot taken) -- This file stores the state of the virtual machine memory when the snapshot was taken.
  • <machine name>-nnnnnn.vmdk (where nnnnnn is the number of the disk image, not corresponding to the snapshot number) -- These are log files which store changes to the virtual machine, since snapshot was taken.

When a backup process starts it begins by requesting the VMware system to take a snapshot through the VMware API. The virtual machine is put into read-only state and a snapshot (or delta file since storage has a different definition for snapshots) is created. This delta file collects changes to the virtual machine which are written from the end-users and applications during the backup process. Once the backup is complete, the VMware system merges the delta file to update the virtual machine to a current processing state. The detal file is then discarded.

Having snapshots open and growing is a bad thing for resources on production systems, which is why the Vizioncore data protection products including vRanger Pro and vReplicator are all optimized to reduce the time that snapshots are open.