The basic tenets of that article still hold, but technology has moved considerably since those days, and we get questions from time to time about good backup practices for advisors, so I thought I'd put together a bit of a refresher on the topic.
As a refresher, the biggest thing to understand about backup requirements for financial advisory firms is that no backup application is going to be "compliant" in and of itself. Compliance isn't about the specific software you use - it's about whether you achieve the following goals:
- Meet compliance requirements by showing how a given documents has (or hasn't) changed over time. This is especially important for scanned documents.
- Meet disaster recovery requirements. You really need to be able to restore your entire system from a single piece of media - without having to wait 2 days for a disk to be mailed to you (this is a primary issue that I have with online backup systems - trying to recover a huge server worth of files over the Internet isn't prctical - especially because in a disaster scenario, it is unlikely that you will even have Internet access)
- Ensure that any data written to media that can be easily taken off site is encrypted
The big change in technology from what was described in the original paper is the move away from tape media to external hard disks.
The same grandfather/father/son media rotation strategy could technically be achieved using external hard disks - however, it would be very expensive to have to have 20 or more hard disks. Fortunately, disk capacity has increased dramatically over the years, so it now makes sense to use 2 or 3 external disks in a rotation, storing many snapshots on each disk. The reason for using multiple disks in rotation is to protect against the case where one of those disks might be damaged, in which case you can purchase a new disk and initialize it using the data from one of the undamaged disks.
One key technology that you need to understand when storing many snapshots to a single piece of media is the concept of data de-duplication. From one day to the next (and indeed, from one year the next), the vast majority of the data on your server’s hard drives does not change. This opens up the possibility of only storing data that changes, which can dramatically increase the number of backup snapshots that can be stored in a given amount of space, at the expense of a bit more accounting overhead in the software to keep track of things.
How Many Snapshots
For your working documents (your scanned files, Word and Excel documents, etc…) you would ideally hold on to every version of them ever created. This may sounds like a lot of data, but it’s actually quite modest.
For large monolithic files like database files (i.e. the files that backup up the various SQL Server databases in your firm), storing every single version would be quite difficult – these files tend to change constantly – plus the files are very, very large. For disaster recovery purposes, you probably want to have versions of these files going back 2 to 4 weeks (in case there is a database corruption that takes a couple of days to notice), but it is doubtful that any backup older than that would have any practical value.
The operating system and application executable files on your server don’t change very often, so holding on to multiple versions would be ok – but practically speaking, you would never restore files older than about 2 weeks from backup – so holding on to lots of old snapshots of these files is not beneficial.
Back up the entire disk, or the files?
There are two strategies when it comes to backing up your server – some backup software focuses on one or the other – some backup software can do both:
File level backup – These backup systems read files from the source disk and store them on the backup media. These backups tend to be better at allowing for purging of older versions of specific files. Traditionally, however, they have not done a good job on the disaster recovery front – restoring from backup generally involves re-installing the server operating system, then recovering data files.
Block level backup – These backup systems read the low level bits that are actually stored on the source disk, and write them to the backup media. These systems tend to be much better in disaster recover scenarios (recovering involves writing the data directly to the disk – there is usually no operating system install required). Data de-duplication tends to work better in block level backup systems, so they tend to be more efficient in how they consume space on the backup media. Block level backup has it’s downsides, though – in that they have been traditionally not very good at selective purging of versions for certain files.
No Silver Bullet - Yet
The ideal backup solution would be one that is capable of:
- holding on to all versions of some files (like your documents) while still being able to automatically purge older versions of other files (like your database and operating system files).
- backing up to multiple external disks in a 2 or 3 disk rotation
- encrypting all data written to the external disks
- data de-duplication to ensure that the backup disks are used efficiently
- use block level backup to optimize the disaster recovery process, and make the best use of the backup media
At this time, unfortunately, I am not aware of a single backup solution that meets all of these requirements. It is possible to achieve something somewhat similar by using multiple backup strategies, or possibly segmenting the disks on the server.
In the absence of a concrete recommendation for backup, I thought that I’d describe how we do backup at Trumpet. Please bear in mind that we are not a financial advisory firm, so we do not have the same compliance requirements that you do. Don’t just blindly copy our system without evaluating it against the requirements described at the top of this article.
At Trumpet, we use two different systems to achieve the requirements (we don't have compliance requirements, but I still like to have past versions of files). To do this, we do a file level differential backup of our documents to an *internal* hard disk on our server. And we do block-level backup of our entire server's disks to external hard disks (encrypted!) that are taken off site every day in a rotation. We also use Microsoft’s VSS technology to allow users to easily recover past snapshots of documents, and the VSS snapshots are part of the block level backup that we perform.
Finally, note that selection and configuration of your backup system is an activity that your system administrator is uniquely situated to help you with, and while we do understand the technology, we really aren't set up to provide anything more than general suggestions and commentary about direction your system administrator may want to take things.
If anyone has a backup system/strategy that they are really happy with, please post a comment so others can benefit from your find – I look forward to reading your comments.