[wplug-internet] Backups

[Michael Semcheski]

The id_rsa.pub for root at wplug.org can be found here:

http://www.wplug.org/nojump/id_rsa.pub

I'm assuming the script on the linode machine will have to run as root
-- at least until we create another user with access to alll the
relevant files.

I'm working on setting things up on my end now.  I'll give everyone a
status update if I get anything to work.

On Mon, Jun 22, 2009 at 8:51 AM, Bill Moran<wmoran at potentialtech.com> wrote:
> In response to Vance Kochenderfer <vkochend at nyx.net>:
>
>> I have been thinking over what's been proposed.  Not having dealt
>> with encrypted filesystems before, this is the setup as I've
>> pictured it.  Please correct me if this is wrong anywhere.
>>
>> Assume that Bill's machine is bill.example.com.  A user named
>> "wplug" has been created on this box, and a ~5GB file created as
>> /home/wplug/linode.img.  A public login key has also been dropped
>> into /home/wplug/.ssh/authorized_keys.
>>
>> 1. From the Linode server, we establish a connection to Bill's
>> machine via sshfs:
>>   wplug at bill.example.com: <--sshfs--> /mnt/wmoran
>>
>> 2. Then associate the file on Bill's machine with a loop device:
>>   /mnt/wmoran/linode.img <--losetup--> /dev/loop0
>>
>> 3. Set up the dm-crypt encrypted device:
>>   /dev/loop0 <--cryptsetup--> wmoranfs
>>
>> 4. The encrypted device can then be mounted as a plain old
>> filesystem (will need to format it first):
>>   /dev/mapper/wmoranfs <--mount -t ext2--> /mnt/backup1
>>
>> 5. Then we can perform the actual backup via rsync:
>>   /mnt/backup1 <--rsync--> /var/lib/mailman, /var/lib/mysql, ...
>>
>> I was afraid that step #1 would be a problem until I learned that
>> the SFTP protocol used by sshfs is capable of random file access.
>> Assuming that OpenSSH supports that on both ends, this should
>> mean that only changes to the linode.img file will be transferred
>> over the network, instead of the whole 5GB.
>>
>> For step #3, we will need to keep the encryption key on the Linode
>> server.  Because dm-crypt uses symmetric encryption, this means
>> that anyone who gets their hands on this key will be able to
>> decrypt the linode.img file.  Then again, anyone who's able to
>> read the key probably has access to the server itself, so this may
>> be acceptable.
>>
>> After step #5, we should be able to take the linode.img file,
>> mount it (either over a network or locally) on any Linux system
>> supporting dm-crypt using the symmetric key, and get to the
>> backed-up files inside.
>>
>> This wiki page looks like a decent reference for everything but
>> the sshfs part <http://wiki.centos.org/HowTos/EncryptedFilesystem>.
>>
>> It seems to me that the path forward is then as follows.
>>
>> A. Write up a script that does steps #1-5.
>>
>> B. Create the symmetric encryption key - place it on the server
>> and distribute it to trusted WPLUGers for safekeeping.  Also
>> create a restore script for recovering files from the image.
>>
>> C. Setup an account and ssh public key on Bill's machine.
>>
>> D. Create a smallish test image file on his machine and verify
>> that the numbered steps above indeed work by backing up a portion
>> of the server's filesystem.  Check that rsyncing results in only
>> the changes being transferred over the network.
>>
>> E. Create the full-sized linode.img on Bill's machine and do a
>> backup using the script from A.  Verify that performance is
>> acceptable (i.e., that it doesn't take 24h or something similarly
>> ridiculous to complete).
>>
>> F. Verify that the files can be restored from the saved image with
>> the key and script from B.  I can buy a separate Linode 360 and
>> try to restore onto that.  Since they'll give a prorated refund
>> for the portion of the month I don't use, it should only cost a
>> few bucks to try out and would be a good simulation of disaster
>> recovery.
>>
>> G. Set up a cron job (how often?) to periodically connect to
>> Bill's machine and run the script from A.
>>
>> H. Recruit others to serve as alternate backup hosts.  All they
>> should need is a broadband connection, sshd, and a filesystem
>> capable of holding a ~5GB file.
>>
>> Am I missing anything?  Who can start on step A?
>
> This looks like a good plan to me.  I'll get port forwarding and an
> account set up on my home system this week and email the login details.
> Feel free to generate an ssh key and send me the public whenever.
>
> --
> Bill Moran
> http://www.potentialtech.com
> http://people.collaborativefusion.com/~wmoran/
> _______________________________________________
> wplug-internet mailing list
> wplug-internet at wplug.org
> http://www.wplug.org/mailman/listinfo/wplug-internet
>