[wplug-internet] Backups

[Bill Moran]

In response to Vance Kochenderfer <vkochend at nyx.net>:

> I have been thinking over what's been proposed.  Not having dealt
> with encrypted filesystems before, this is the setup as I've
> pictured it.  Please correct me if this is wrong anywhere.
> 
> Assume that Bill's machine is bill.example.com.  A user named
> "wplug" has been created on this box, and a ~5GB file created as
> /home/wplug/linode.img.  A public login key has also been dropped
> into /home/wplug/.ssh/authorized_keys.
> 
> 1. From the Linode server, we establish a connection to Bill's
> machine via sshfs:
>   wplug at bill.example.com: <--sshfs--> /mnt/wmoran
> 
> 2. Then associate the file on Bill's machine with a loop device:
>   /mnt/wmoran/linode.img <--losetup--> /dev/loop0
> 
> 3. Set up the dm-crypt encrypted device:
>   /dev/loop0 <--cryptsetup--> wmoranfs
> 
> 4. The encrypted device can then be mounted as a plain old
> filesystem (will need to format it first):
>   /dev/mapper/wmoranfs <--mount -t ext2--> /mnt/backup1
> 
> 5. Then we can perform the actual backup via rsync:
>   /mnt/backup1 <--rsync--> /var/lib/mailman, /var/lib/mysql, ...
> 
> I was afraid that step #1 would be a problem until I learned that
> the SFTP protocol used by sshfs is capable of random file access.
> Assuming that OpenSSH supports that on both ends, this should
> mean that only changes to the linode.img file will be transferred
> over the network, instead of the whole 5GB.
> 
> For step #3, we will need to keep the encryption key on the Linode
> server.  Because dm-crypt uses symmetric encryption, this means
> that anyone who gets their hands on this key will be able to
> decrypt the linode.img file.  Then again, anyone who's able to
> read the key probably has access to the server itself, so this may
> be acceptable.
> 
> After step #5, we should be able to take the linode.img file,
> mount it (either over a network or locally) on any Linux system
> supporting dm-crypt using the symmetric key, and get to the
> backed-up files inside.
> 
> This wiki page looks like a decent reference for everything but
> the sshfs part <http://wiki.centos.org/HowTos/EncryptedFilesystem>.
> 
> It seems to me that the path forward is then as follows.
> 
> A. Write up a script that does steps #1-5.
> 
> B. Create the symmetric encryption key - place it on the server
> and distribute it to trusted WPLUGers for safekeeping.  Also
> create a restore script for recovering files from the image.
> 
> C. Setup an account and ssh public key on Bill's machine.
> 
> D. Create a smallish test image file on his machine and verify
> that the numbered steps above indeed work by backing up a portion
> of the server's filesystem.  Check that rsyncing results in only
> the changes being transferred over the network.
> 
> E. Create the full-sized linode.img on Bill's machine and do a
> backup using the script from A.  Verify that performance is
> acceptable (i.e., that it doesn't take 24h or something similarly
> ridiculous to complete).
> 
> F. Verify that the files can be restored from the saved image with
> the key and script from B.  I can buy a separate Linode 360 and
> try to restore onto that.  Since they'll give a prorated refund
> for the portion of the month I don't use, it should only cost a
> few bucks to try out and would be a good simulation of disaster
> recovery.
> 
> G. Set up a cron job (how often?) to periodically connect to
> Bill's machine and run the script from A.
> 
> H. Recruit others to serve as alternate backup hosts.  All they
> should need is a broadband connection, sshd, and a filesystem
> capable of holding a ~5GB file.
> 
> Am I missing anything?  Who can start on step A?

This looks like a good plan to me.  I'll get port forwarding and an
account set up on my home system this week and email the login details.
Feel free to generate an ssh key and send me the public whenever.

-- 
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/