[wplug-board] WPLUG's information assets (backups - part 3 of 3)

Vance Kochenderfer vkochend at nyx.net
Sun Jan 9 23:14:06 EST 2011


There are any number of possible backup solutions out there.  The
design goals of my proposed approach are:

A. Use standard tools, so bare-metal restoration does not require
some exotic and possibly difficult-to-find package.

B. Operate automatically in as hands-off a fashion as possible,
while still informing the admin if errors occur.

C. Permit backups to be stored in multiple locations (targets).

D. Retain historical backups so a restore can be done from some
past checkpoint.

E. Do not require the backup target (that is, the machine hosting
the backup) to have special software installed.

F. Prevent viewing the contents of the backup by anyone not
authorized, including the owner of the backup target.

Attached is a backup script I developed for use by WPLUG.  It more
or less meets the goals listed above:

A. The script uses duplicity, which while not a standard POSIX
tool, is commonly available on Linux.  It provides a nice front
end to the rdiff tool which identifies differences between files.
It is possible to restore a duplicity archive using only tar, cp,
rdiff, and rm.

B. This is the major point of the script.  A configuration file
must be set up once for each backup target, then the script
handles the rest automagically.

C. You can specify as many targets as you like.  The script will
use one in round-robin fashion each time it is run.

D. Duplicity produces incremental backups.

E. The target only needs an SSH daemon accepting SCP connections.

F. Backups are GPG-encrypted with a 48-byte symmetric key.

The readme file in the attached archive describes how things work
in technical detail.  Please refer to it or the scripts themselves
for the full story.  The code should be considered alpha stage at
this point - it has not been extensively tested.

My projected use case was to recruit individuals who would be
willing to donate bandwidth and disk space to host backups.

Copying our files to random locations obviously presents the
possibility that they could be read by the owners of these target
machines or anyone who may exploit one of these machines.  Strong
encryption is used to protect all the files written.  Reading the
files requires either breaking GPG or obtaining a copy of the
encryption key.  The key is stored on the server, and the script
enforces that it must only be readable by root.

For disaster recovery purposes, obviously copies of the encryption
key must be held by trusted parties separate from the server.

The amount of data that I would expect to be backed up is:

/etc 35M - main location of server config data

/home 2.4G
     /board - member rolls (spreadsheets), maintenance log,
              backups of old (penguin) web site, some config data
     /monkeybot - all the files for monkeybot (including perl)

/root 23M - some config data and random stuff

/usr/lib/mailman 38M - mailman binaries and some config data

/var/lib 727M
        /mysql - databases
        /mailman - mailing list archives, subscriber lists, and
                   main config data

/var/spool 3.3M
          /postfix - e-mail temporarily held while in transit

/var/www 112M - wiki files, including config data and uploaded
                files

For a total of 3.4G.  It may be possible to trim this down, but I
took an inclusive approach rather than risk leaving out something
important.

While I was expecting to use donated space, it may be possible to
use Dropbox, Amazon S3, or some similar provider as a target.  I
presume that this would have a price tag attached.

Please reply with your comments on the validity of the approach
and any alternative proposals you may have.  Also, comments on the
specific code itself are welcome, if we decide this is the correct
path.

Vance Kochenderfer        |  "Get me out of these ropes and into a
vkochend at nyx.net          |   good belt of Scotch"    -Nick Danger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wplug-backup-0.06.tar.gz
Type: application/octet-stream
Size: 12969 bytes
Desc: gzip compressed data - deflate method
Url : http://www.wplug.org/pipermail/wplug-board/attachments/20110109/f82c609d/attachment.obj 


More information about the wplug-board mailing list