Replicating a Veeam Backup Repository with Windows Deduplication

I have already discussed the advantages of using Windows Deduplication for Veeam Backup Repositories, and introduced the idea of using Replacador to replicate a windows deduplication volume.

Now we will set up Replacador on a windows deduplication volume that holds a Veeam Backup Repository and then run the replication process.

First browse to C:\LaserVault\Replacador and execute the ReplacadorConfig application.

We will use this to define a replication task for our source and destination volumes.

In this first example, we will replicate the volume to another volume on an external drive on the same server.  The same process also runs across a network or the Internet.  We will give an example of that later.

In the Replication Configuration Screen, press the Add Definition button and define a replication task.

In this case, we make the task name vmbackup, the machine name is ‘.’  ( a period means the current machine).

The volume Path is D:\  Normally a deduplication volume will be a drive letter on the current machine.

replacador4

Don’t click OK yet.

Each replication needs at least one Destination. You can actually replicate to multiple destinations at once.

Click the Add Destination button and a new form opens to define the destination.

In this case we are replicating to another volume on the same server, so the Machine Name is ‘.’ for the local server again.

In the case of a different network location, this could be the VNC name of the target machine, or its IP address.

The username is the local username on the target machine plus a password.  In this case we are using administrator, but you could use the system account or whatever is appropriate.  The user needs to have sufficient authority to run Windows Deduplication garbage collection on the target machine.

The volume path is the local pathname on the target machine to the deduplication volume that will be a clone of the source deduplication volume.

The UNC path is the UNC version of this target deduplication volume, consisting of \\machinename\\volume

In the case where the target is on this local machine, just use the volume path again.

replacador5

Now click OK, then OK, then OK, and your replication is configured.

The Replacador Manual explains how to run the Replacador Transfer program from the command line or thru the task scheduler, but since we are just testing, we will just click to execute the ReplacadorTransfer application.  Since there is only one replication task defined and the default action of the application is to replicate, it will do exactly what we want.

When you first start the Replacador Transfer, it looks like nothing is happening.  Actually Replacador transfer starts a deduplication job on the source volume to make sure that everything is ready for replication.  If you have already deduplicated the volume, this part of the task will just take a minute or two.

Once the source volume is deduplicated, a command window will open and display the replication progress.

replacador7

You can get a better idea what is going one by looking at the Task Manager performance screen.

replacador8

Replication is really just a specialized copy task that takes very few CPU or memory resources.  The limiting factor is the speed of reading, transmitting, and writing the data on the target volume.

The whole point to replication is to reduce the data traffic to the minimum needed to move the changes from the source volume to the destination volume.  The first replication will be a large one, which is why it is sometimes a good idea to replicate to an external drive to seed the actual target server volume.  After that, the replication process should be a small fraction of the original deduplication volume content, even for a new full backup.

When Replacador is done, you will have an exact copy of the deduplicated volume on the target server.  Each time you replicate in the future, only the changed chunks and reparse points on the deduplicated volume will be sent to the target volume.  The original files will not be reflated at any time in the process.

 

Replication for Windows 2012 R2 Deduplication with Replacador

Windows Deduplication is a free feature in Windows Server 2012 and Server 2012 R2.  It works great and I recommend it.

I’ve  also worked with other deduplication systems, including Data Domain, Avamar, ExeGrid, GreenBytes, Opensolaris and Nexenta.

Deduplication works especially well for backup files. With a deduplication system, you can store many backups on one deduplication appliance, because deduplication only stores each unique chunk of data once.

Besides the obvious advantage of taking up much less disk space for each new backup, deduplication reduces the amount of data transmission you need in order to replicate the deduplicated backup across the network or Internet to an offsite location.

The reduction in data seems magical when you first encounter it.  It actually makes it possible to replicate a backup in a reasonable amount of time with the Internet connection you already have, for most people.

The one problem with using Windows Deduplication instead of another backup appliance is that it does not have replication built into it.

We decided to do something about this, so we wrote Replacador as a replication system for Windows Deduplication.

Replacador looks at two Windows Deduplication volumes, and keeps them synchronized.  First, the source volume is optimized to turn all in policy files into deduplication chunks and reparse points. All the new or changed chunks on the source volume are copied to the destination volume, along with the reparse points. Anything that has been deleted from the source volume is deleted from the destination volume.

Finally, garbage collection is run on both source and destination volumes to keep them synchronized.

The source volume has to be a locally attached volume on the source system.  The destination volume can be a second volume on the source system, such as an external drive.  Usually it is a volume on another server that is on the network or the internet.  Both servers must be running the same version of Windows 2012.  We suggest that they should be running release R2 because the deduplication process is much faster on R2.

The volumes do not have to be the same size, but you will probably want them to be.  The second volume has to have enough room for all the chunks, the reparse points, and some extra room for garbage collection to run.

The reason we support replicating to an external drive is to make it easy to ‘seed’ a new remote deduplication volume when you first start deduplicating.  You can replicate to an external drive on the source system, carry it to the remote system, and replicate from the external drive to the destination drive one time.  This can save days or even weeks of data transmission, in some cases.

This also make it possible to replicate back from the destination volume to an external drive to quickly restore to a source system in case of catastrophic loss of the first volume, due to tornado, fire, flood, etc.

An external disk drive can also be used as a source volume for deduplication.  Some external drives support RAID data protection.  A good example of this is the Western Digital Duo series.  The Duo 8TB costs less than $350 and provides 4 TB of protected storage.  There is also a 12 TB Duo with 6Tb of available storage.

There is a beta test version of Replicator available.  You will also need an authorization code to use Replacador.

Here is the PDF of the documentation.

Replacador Configuration and Use

To get an activation code, browse to c:\LaserVault\Replacador and click the Replacador Configuration application.

Click the Authorization button on the lower left.

replacador1

Replacador generates a unique serial number for your server.  Copy the contents of the Serial Number and paste it into an email and send it to ReplacadorCode@laservault.com.

We will send you a code good for 30 days.

Replacador2

 

When you receive the code, paste it into the Authorization Key and click OK.

Next, setup and run Replacador 

Backing up with Veeam to your Windows 2012 R2 Deduplication Appliance

If you are following along with our idea of making your own deduplication appliance, you might be interested in using it as one or more Veeam Backup Repositories.

Veeam has some postings about using Windows Deduplication with Veeam backups, and they seem to think it is a great idea.

I do too, and I think our Replacador replication for Windows Deduplication makes things even better.

We have been backing up our own Windows discrete servers for years with plain old Windows Backup.  As we gradually migrated our servers to our first big Hyper-V server (called Borg1 for some reason) we kept using Windows Backup doing full backups every night, deduplicating, and replicating.

When I first heard of Veeam it was in reference to deduplication.  When I read about Veeams positive attitude towards Windows Deduplication, I became even more interested.  So we decided to install Veeam Enterprise Plus trial edition.

I set it up on our Borg1 server and defined the backup job for most of our VMs.  I skipped our document management VM for now because I’m impatient to run the tests faster and that data doesn’t change much.  I will add that in later for production.

I set up a Veeam Backup Repository on one of our UBD servers running Windows 2012 R2 with a deduplication volume. Actually I just twiddled my thumbs while Veeam did all the work.  I did get to make some important decisions about the settings for the Backup Repository, which I will share with you in a future post.

I set the system up for forward incremental backups with a full backup once a week.

The first Veeam backup took about an hour and a half and moved 800 GB of data across the network.  I ran Windows Deduplication on the volume and it compressed and deduped about 60%.  The deduplication job ran in a couple of hours, and of course was mostly compression for the first day.

I couldn’t wait a whole day to do another Veeam backup, so I did the same backup again after a couple of hours.  This was over the same 800 GB set of VMs.

The Veeam backup ran in 6 minutes and 47 seconds. It sent about 26 billion bytes of data to the Backup Repository.

Windows Deduplication ran in 15 minutes or so. The 26 GB of data on disk became 1 GB of deduplicated data. Over our 30 Mbit per second Internet upload we will be replicating in about six minutes.

According to what I have read about Veeam, it is deduplicating within a single backup, across VMs. What Windows Deduplication is adding is the deduplication across multiple backups. This means even a periodic full backup will take up very little space on the deduplicated volume and very little replication bandwidth.

The only replication job that should be somewhat large is the very first one, and our Replacador software supports replication to an external drive, which means you can seed the replication to a drive then send it to your DR site for immediate protection.

Every time I think of Veeam, I am saying WOW.  What an incredible product. If you haven’t tried it yet, spend an hour or two and set it up. And smile.

I am going to publish the settings and statistics for my first Veeam backup jobs in another post.