[Elecraft] <Long & OT> Data protection and recovery techniques

[kevinr]

Good Afternoon,

Recently there was discussion on backup strategies.  Here are my current 
methods.

I use three data protection strategies. The first is possible because of 
the way I design my computers. I use a small solid state drive for the 
OS and for all of the applications I use. Once everything is installed 
and all the settings massaged I make an ISO image of the drive and save 
it on my server. Then I clone a drive from that ISO and put it into a 
drawer. If I ever have a major OS crash I can pull that drive out of 
storage, install it into the box, and be running again in a few minutes.

The second method I use for data protection requires a versioning 
application. I use Subversion. When a set of folders with their 
attendant files is ready I create a Subversion repository to hold them. 
This repository is a database of files. Files can be of any type I have 
tried: C code, text files, CAD files, SPICE files, circuit board 
layouts, bills of material, etc. While I am working I commit my changes 
many times a day. If I need to recover a file from a week ago I can type 
svn update XXX and roll back my file system to when the XXX snapshot was 
current. I can also check out the repository to another computer and 
work on it. The first thing I do, when I start working on any of my 
computers, is type svn update. This pulls any changes from the 
repository and brings my files up to date. Then I write code, prose, or 
CAD committing the changes as I work. By typing svn update I bring each 
of my laptops and work computers up to the same revision number.

The third method of data protection is more common. I make copies of my 
files. I have found backup schemes which use compression or encryption 
get in my way. When I want to recover something it is not normally the 
entire system; I only need a small subset of the whole. So my backups 
consist of exact copies of files in the folder structure I built. If I 
need to, I can move entire chunks of the file hierarchy to recover from 
a major crash. For the most sensitive data I make a copy on the same 
computer but on a different drive and again on a separate machine.

The tool I use for this third method of data protection is a command 
built into Linux but available to Windows and Mac users too. Rsync lets 
me ‘push’ data from where it is to a remote location or ‘pull’ from a 
remote location to a local file. Rsync works across a network or across 
the Internet so you can have your files anywhere you have access. Linux 
provides me with another valuable command called cron. It is used to 
perform a task periodically, hours, days, weeks, months, etc. Once I set 
up a cron job I don’t need to worry about backing up my files. It all 
happens behind the scenes at whatever time of day I choose.

The first time you run rsync on a set of folders it copies the entire 
set. The next time it runs only the modified files are copied. Normal 
backup commands make compressed copies of all of the files in the chosen 
set of folders. Over time they build up a large set of files which are 
not readily accessible by humans taking up an inordinate amount of 
space. Rsync creates a mirror image of what your file system looks like 
right now. If you need one data file from the middle of that pile of 
files you can find it using your normal file system search mechanism. 
You do not need to decompress or decrypt anything. Everything is in 
front of you, easy to read.

I know this may seem Linux specific but it is not. That is just where I 
learned how to do this. Subversion, rsync, and a cron equivalent are 
available for both Linux and Windows systems and I suspect there are Mac 
versions too. The modern Macintosh OS is a flavor of Linux if you 
scratch beneath the surface. My first method of data protection is 
specific to how I build computers. Keep the OS and applications on one 
small drive while keeping all of the user created and collected data on 
a second, larger drive (or in my case across the network on the file 
server). The OS and application set do not change very rapidly so you 
can make a copy for later. The data changes on a regular basis so you 
need another method to preserve it.

Hopefully, these suggestions will help you keep your most precious data 
intact.

73 & GL with any hardware crashes,

    Kevin.  KD5ONS

-