I have noticed over the last 12 months or so that ransomware attacks are becoming even more sophisticated in what their attack vector is. The nasty ones will delete common backups files, like Veeam backups, before proceeding to encrypt all other files. This begs the questions, is it enough to have one data protection vendor in play these days or should you be looking at data protection in-depth strategy? You may have heard about Defense in Depth strategy to protect against threats from the perimeter of your network, all the way to the core where the data lives. But all it takes is one user, with too many privileges to click on that link in a bogus e-mail and BAM, there is ransomware in the network.
I must caveat that this article is my thoughts only, do not treat is as a best practice guide. Every environment and situation is different. The principles outlined below though can go along way to ensuring you have a data recovery plan should the worst happen.
Data Protection in Depth
When I was thinking about this, it doesn’t really differ from a defense in depth strategy. It relies on multiple layers to help protect the data at the core. I think in this day in age to rely solely on a local backup, especially one that is always online and readily available, will sooner or later lead to complete loss of data. Believe me, I have seen it happen.
Let’s take a look at some methods you can employ to help better protect data with backups or another method of duplicating that data.
Working from the inside out this would be the path of data recovery you would typically take, each layer should have its own Security and Hardening applied.
Let’s start with the data its self, the lifeblood of many an organisation and ultimately the target of a ransomware attack. There should be appropriate permission in place with only the users who need access to the data, able to access the data. This will limit the attack surface if a ransomware attack did take place. Avoid making all users global admins (yes it does happen, typically put in place by #YOLOSysAdmin) and follow principles like creating read-only groups, read and modify and full control over the files. Netwrix is one product I use that can help identify configuration issues in the environment with NTFS permissions. They wrote a great article here to help define a robust NTFS access control plan.
For the rest of these sections, I am going to primarily focus on Veeam, as that is what I know and love 🙂
When I talk about a nearline backup, I mean something that is quick to recover from and always online. Typically this is a device with some kind of hard drives in there. It could be a server, a NAS, or a purpose-built platform that offers additional benefits like hardware compressions and deduplication. The point being it is quick to recover data from. A nearline backup is typically used if someone has accidentally deleted a file or an e-mail and they would like to recover the data quickly.
Being disk-based, these kinds of repositories tend to be accessed via SMB or they are an NTFS or ReFS based partition if you are using a Windows operating system. Physically securing these devices makes sense but to help protect against loss of backups, we should follow similar principles as we do to protecting the core data. Least privilege access to any backup repository should always be configured. There is an excellent article written by Edwin Weijdema on this very topic. Check it out here. In a nutshell, don’t domain join a repository, don’t use the default admin account, lockdown firewalls, and NTFS permissions to only allow access to the repository from the Veeam management servers.
If you are using Veeam, there are a number of backup targets that could be classed as an offsite backup. This could be cloud-based object storage, a Veeam cloud connect partner, you may have another building somewhere in a campus hosting a backup target, it could be rotated hard drives or anything else in between. The point here is though, it is your insurance policy should something happen to the data and to the nearline backups.
Now depending on where you store the data, this can offer additional protection from ransomware and malicious users, some of which I will discuss here.
If you find yourself in a situation where someone has compromised the Veeam management server and deletes all the backups, then what?
This is where making use of those third-party backup targets comes into play, they bring their own special sauce to help protect data, even from a ransomware attack or an insider threat.
Veeam Cloud Connect. Many Veeam partners offer BaaS (Backup as a Service) with Veeam Cloud Connect, it is a backup target in the cloud. What makes this less susceptible to attack is, it’s off your network so a credential theft attack to spread ransomware should not wipe out those backups but more importantly Insider Threat Protection can be enabled on those backups. Which for want of a better explanation is a recycle bin on the service provider side. If a malicious user deletes the backups in a cloud connect repository, they are moved to a hidden recycle bin that only the service provider has access to.
Object storage can offer immutability for backups. Take a look at the integration Veeam has with Amazon S3 Object Lock. This means that the data blocks are locked and can not be removed. Thus maintaining data even if someone tries to delete it or overwrite it.
Exagrid is offering something called a retention timelock which delays the deletion of any blocks of data by a specified number of days. So even if a backup is deleted from within Veeam, it will not be removed from the Exagrid appliance until the specified retention time is up. What makes this even nicer is that to change the time lock duration, two-factor authentication is required.
Other solutions like an offsite NAS, you can make use of its built-in replication technology to create another copy of the data not visible to Veeam.
Air Gap Backups
An air-gapped backup is one that is completely off the network, there is no way at all anyone could log onto a device and click delete on it as it is not physically connected to anything, hence the term air-gapped. Tape is the easiest example to use here. If the s**t has really hit the fan and every previous precaution for data protection has failed an air gap backup would be one of the last lines of recovery. Ransomware can’t attack it if it’s offline. A malicious attacker cant delete it if it’s offline.
Now, that’s not to say you shouldn’t think about security and hardening for those air-gapped backups. It’s no good taking a backup to tape and then tossing into the glove box in the car over the weekend. Anything could happen to it.
As a minimum, they should be stored in a fireproof safe, preferably stored off-site.
Many companies use this kind of storage for long term archive as well. Just remember that tape does not last forever and each LTO generation is only compatible with at the most two prior versions.
Repeat after me, SAN snapshots are not backups. They do however deserve a mention here. Many modern-day SANs offer the ability to create a snapshot of their volumes for a quick rollback of a volume. Let’s propose that the worst has happened, ransomware has taken down all files and deleted backups. Air gap backups are broken and offsite backups are toast. The volumes on a SAN where many servers run from are typically not exposed to a production environment in a way where an attacker could manipulate them and delete them. Having the ability to roll back a volume to a known good state could just be the get out of jail free card you need.
Securing access to the SAN should still follow all the same precautions as everything else, don’t set the access control list to any, lock it down to only those devices that need access, and apply a password to authenticate connectivity.
This list is not exhaustive but I hope I have provided some food for thought to help you assess your data protection requirements and evaluate what is important.
Remember that a backup is only as good as the last time it was tested. This process can be automated in Veeam with SureBackup. Other backup vendors offer similar functionality.