Extra backup protection for Veeam with Synology snapshots 2


Background

In some of my previous blog posts, I have been exploring the functionality and capabilities of a Synology NAS which so far has been storage configuration and testing as well as Synology native Office 365 backup. I also wrote about data protection strategies to pose the question, what is enough in terms of data protection? Recently I have been reading about the enhanced 3-2-1 backup rule which was 3 copies of your data, 2 different media types, 1 offsite. This has been expanded to the 3-2-1-1-0 rule to further enhance your data protection posture. The extra 1 and 0 are 1 copy that is air-gapped, immutable or offline and the 0 is for zero recovery errors. A backup is only as good as its ability to be recovered from, right?

With that in mind, I thought I would explore the Synology storage snapshot feature to try and satisfy the air-gapped, offline criteria. Now arguably this is not an offline copy of the data, however, it is not accessible from the Veeam management console which is where a compromised account could delete backup sets from. It could just be the get out of jail free card you need to recover backup data quickly in the event backup sets have been deleted.

Setup

For this to work with a Synology device, the storage needs to be carved out as a thin iSCSI volume. Thick provisioned storage does not support storage-based snapshots as far as I could tell. I will run through the process of configuring both so you can see the differences with each.

Step 1 – Create a thick iSCSI LUN

First off we need to log onto the NAS and create an iSCSI LUN and Target. This is the storage volume and destination the Veeam repository server will connect to. The thick LUN will be used for Veeam primary backups.

Open iSCSI manager from within DSM.

SynologyP5_001

Synology offers a tool to manage storage for multiple NAS devices from either Windows or a VMware environment. I chose not to download this as I am only managing one device.

SynologyP5_002

We want to create a new LUN.

SynologyP5_003

Define the LUN name, choose where to take the space from. Define the size and choose the space allocation method. Notice that snapshot is not available to tick with the thick provisioned volume.

SynologyP5_004

As an iSCSI target does not exist, the setup wizard asks us to create one.

SynologyP5_005

Give the target a name and modify the IQN if you would like it to be something more easily identifiable from the device that will connect to it. For extra security, I enabled CHAP user name and password option which stops anonymous devices connecting to the LUN.

SynologyP5_006

Confirm the settings.

SynologyP5_007

And now we have a new 10TB LUN ready to go.

SynologyP5_008

Step 2 – Present storage to Windows and Veeam

A point to note with my setup, I am only able to make use of a single 1Gbps network interface on the NAS, so there is no multipathing enabled to the device.

Within Windows, open the iSCSI initiator, browse to the Discovery tab and click on Discover Portal.

SynologyP5_009

Type in the IP address of the NAS device. The dedicated 1Gbps network interface I mentioned before is connected to a dedicated storage subnet. After typing in the IP address, click the Advanced button,

SynologyP5_010

On the Advanced Settings page, choose which NIC on the server to connect from. You do not need to specify CHAP credentials to make the initial discovery connection to the NAS.

SynologyP5_011

Now click back over to the Targets tab, you should see a new LUN waiting to be connected to with the IQN that was specified earlier. Select it and click connect.

SynologyP5_012

Same deal as before with the advanced settings, except this time the CHAP credentials need to be specified before the connection will be allowed to the LUN.

SynologyP5_013

Now open Disk Manager.

SynologyP5_014

Find the new disk in the list of disks, right-click on it and chose properties and make sure it is the Synology device. You do not want to accidentally format the wrong volume!

SynologyP5_015

As this is going to be used for Veeam, I formatted the volume with ReFS to make use of Veeams Fast Clone functionality.

SynologyP5_016

Now into Veeam to add the new backup repository.

SynologyP5_017

This is for all intents and purposes, direct attached storage.

SynologyP5_018

Mounted to a Windows server.

SynologyP5_019

Give the backup repository a name.

SynologyP5_020

Browse to the server where the storage is mounted and chose the drive letter.

SynologyP5_021

Click on advanced to enabled per-vm backup chains.

SynologyP5_022

Like this.

SynologyP5_023

Then pretty much click through until the wizard is complete.

SynologyP5_024

SynologyP5_025

SynologyP5_026

et voilà

A quick bit of testing to evaluate data throughput. Seems about right for a 1Gbps network.

SynologyP5_027

Interestingly though, I was seeing pretty much 100% cache hit on the Synology device when writing the backup set. I am going to put that down to a lack of network bandwidth to saturate the cache.

Update 17th May 2021

One of my fellow UK Veeam user Group leaders, James Kilby, pointed out the following for why the cache hit rate is likely 100%.

SynologyP5_028

Step 3 – Create a thin iSCSI LUN

For the thin-provisioned LUN, I will use this as a backup copy target. In reality, you wouldn’t have both the primary backup and backup copy data on the same device, but I have this set up for demonstration purposes. I will skip some of the setup steps I covered above in this section to avoid duplication.

Back to Synology DSM and open the iSCSI manager again.

This time I will create a thin-provisioned LUN. Notice how the bottom three options are now available, with Snapshot being one of them.

SynologyP5_029

We now have a new thin provisioned LUN.

SynologyP5_030

Notice how there is a note to show there is No Scheduled Protection. Let’s fix that.

SynologyP5_031

I have enabled a snapshot schedule to run every day at midnight.

SynologyP5_032

And I want to keep 7 snapshots. This means I should have an additional 7 days of backup copy job retention above what I configure in Veeam.

SynologyP5_033

There is also an option for Synology to coordinate creating an application-consistent snapshot of the data. This is not something I need to use for this setup but could be useful for other applications that work with quisence.

SynologyP5_034

Again a little bit more testing from Veeam. Throughput is a little better with the backup copy job. I am not sure if this is because Synology is doing something clever in the background with hardware-assisted data transfer or just the numbers are skewed, but the figure is faster than a 1Gbps network can provide.

SynologyP5_036

and in this test, we see a lot of cache misses, which is to be expected. The backup data that is being copied should not still reside in the cache tier once it has been written to the device, so it will be pulling it from the spinning disks behind it.

SynologyP5_035

Recovery from backup deletion

So all the pieces are in place, what to do if the worst happens and somehow the backup sets are deleted from the disk?

Let’s simulate this exact scenario and see what happens.

So you have a nice backup set on disk, when suddenly….

SynologyP5_037

Someone deletes them all.

At this point, disable the associated backup jobs.

SynologyP5_040

Log onto the Synology NAS and navigate to Snapshot in the iSCSI settings.

SynologyP5_039

Select the LUN and then choose Snapshot List from the Snapshot drop-down menu.

SynologyP5_041

Choose the snapshot to roll back to and click restore.

SynologyP5_042

If the Veeam repository server is still connected you will receive the following warning. It is very important to disconnect the server from the storage otherwise this doesn’t work. Ask me how I know that 😀

SynologyP5_043

Disconnect the target from the iSCSI Manager in Windows.

SynologyP5_044

Hit the restore button from DSM and then reconnect the storage. With any luck, everything will be back to the way it was previously.

SynologyP5_037

Enable the job and it should be like nothing happened.

SynologyP5_045

What is really cool is how quickly the LUN was rolled back to the snapshot. 1 second!

SynologyP5_046

Conclusion

We have proven that enabling LUN level snapshots can prove to be another effective step to preventing a total data loss scenario. This is not an air gap solution but just another layer of defence against the bad guys. The more layers we deploy, the harder it will be to defeat them all.

If you would like some further reading, check out this blog from Synology about how to protect yourself from ransomware.


Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

2 thoughts on “Extra backup protection for Veeam with Synology snapshots

  • Albert

    Hi, I use to connect Veeam to Synology repository through NFS instead of iSCSI LUN, I have heard that is less susceptible to be attacked by ransomware due to not to be presented as “native” Windows file system. Is it correct?

    • Ian Post author

      Hi Albert,

      I guess it depends on what you are trying to achieve here. Yes, it is less susceptible in the way that a ransomware attack would not have access to a native Windows volume to perform some kind of crypto locker attack. In much the same way if you utilised an SMB share on a Synology device, it would be less likely to be attacked.

      The concern that none of those storage presentation types addresses though, is if a compromised account has access to the Veeam server. There would be nothing stopping some one opening the console or running some PowerShell code to delete the backup files. At least with something like a snapshot on the Synology device you have an option to restore the data assuming that both the Synology and Veeam account are not compromised. Building up those layers of defence.

      Cheers,

      Ian