Wednesday, February 22, 2012

Using rsync to back up multiple platforms

Introduction
For a small business or office, proprietary backup software may be too expensive or overkill. Some of the open source solutions, such as Amanda, may be overkill as well. Rsync provides a simple solution that can be run on the three platforms most likely to be encountered in a SOHO environment.

Installation and configuration for GNU/Linux
Download and install the package appropriate to your distribution. If you use Debian, apt-get install rsync. You’ll need ssh as well, so if you don’t already have it now is the time to grab it. You’ll need both packages on your backup server, too.

Decide if you want a generic backup account on the server or create individual accounts for your users. My set up is going to serve as a backup for a webserver, mysql, and a couple of other things in addition to user data, so I’m going to do both. Create the necessary accounts on the server. You might want to add a backup directory to /etc/skel, the examples below will use ~/backup.

The following steps may get a little tedious, so skip around if you know your configuration already works. Otherwise, this will help to keep your troubleshooting to a minimum.

On the client, we need to create a public/private key pair so run the following:

$ cd ~/.ssh
$ ssh-keygen –t dsa –f batchkey –N “” #Note: there is a space between the N and the double quotes
$ scp batchkey.pub username@backupserver:~/

This series of commands generated a plaintext key by using an empty password option for the –N flag and then copied the public key to the user’s home directory on the backup machine. Now to install the public key on the backup server:

Backupserver$ cat batchkey.pub >> ~/.ssh/authorized_keys
Backupserver$ chmod 600 ~/.ssh/authorized_keys
Backupserver$ mv batchkey.pub ~/.ssh

This series of commands appended the contents of batchkey.pub to the authorized_keys file, changes the permissions to read and write for the owner only and then moves the batchkey.pub into the directory just to keep things tidy.

Open up the authorized_keys file with your favorite editor and add the following to the front of the line (it is one big line and has to stay that way, so don’t press Enter):

no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,from=”myhost” ssh-dss

The ‘ssh-dss’ is already in the file, it is in the example to illustrate where your change will go. You can add a command option after the ‘from’ option, but I don’t have this working yet and won’t put it in here until I do. The additions to the key restrict what can be done when using the key for login.

While you are logged in to the server, open up the sshd_config file and change PermitRootLogin to no and verify that PubkeyAuthentication is set to yes.

Log out of the backup server and try to log back in using your key:

$ ssh –i ~/.ssh/batchkey user@host

If it works, you’re golden. If not, check your authorized_keys file for whitespace around the commas, newlines, misspelled options, etc.

Once you have passwordless ssh working, try out the following rsync command:

$ rsync –avz –e “ssh –i /home/user/.ssh/batchkey” /local/directory/to/backup user@backupserver:~/backup/

Here’s the breakdown on the options:

-a archive mode (preserves file permissions, time, etc)
-z compress during transfer
-e use ssh tunnel

If you run in to problems, check your command option if you put one in, check permissions on the private key file on the local box and authorized_keys on the remote box. If it works, wrap it up in a shell script (don’t forget to make it executable):

#!/bin/sh
/usr/bin/rsync –az –e “ssh –i /home/user/.ssh/batchkey” /local/directory/to/backup user@backupserver:~/backup/

Another option is to use a rotating incremental backup. I revised an example at rsync.samba.org, which is here. The big difference is that you end up with a week’s worth of files that you can retrieve, but have a mirror of the system you are backing up. The value of this didn’t truly sink in until I had to restore a box that was using the above script and every_single_file_ever was restored. Not nice if you have just reorganized how your files are stored.

Add a line to cron to make it automatic and you’re done:

$ crontab –e
0 3 * * * /path/to/backup/script

If you’re doing this for something other than a user directory or have multiple accounts on one machine to take care of, you’ll want to use a bit of shell scripting trickery to loop through the user accounts. Then save the script in /etc/cron.daily/, change the ownership to root, and go ride bikes.


Installation and Configuration for OS X
As of 10.3, rsync and ssh are installed by default. Make sure it is installed on your version by typing ‘which rsync’ and ‘which ssh’. Everything above applies here except that the daily system cron is not set up in nice little directories where you can just drop your scripts (as root, of course) and the changes to the authorized_keys file are slightly different.
There are scripts called ‘daily’, ‘weekly’, and ‘monthly’ that you could edit, I suppose, but I don’t think I would. There is also a directory called /etc/periodic that has daily, weekly, and monthly directories; this would be a better place to put your scripts. Apple does dumb stuff like replace your edited httpd.conf file with a default one, so I wouldn’t trust them not to touch something like this. Anyway, you can write the script mentioned above, place it somewhere out of the way like /Library/Scripts, and then add an entry to /etc/crontab. The other way to do it would be to have a crontab for the individual user (best if you only need to worry about one account, don’t run as root unless you have to), or have a root crontab. If you use radmind, be aware that root’s home directory is not copied unless you do some modifying of the transcripts.
The other way to do it is to use a Login or Logout hook. Check out this Apple document to see how to do it in the command line or go to Mike Bombich’s site for a nice graphical tool. This may be a better way to go, but the script will run as root, so keep that in mind when you put together your shell script. Ah heck, here’s the quick and dirty way to do it:
$ sudo -s #careful, you’ve just entered the Root Zone
$ mkdir -p /Library/Scripts/Admin
$ cd /Library/Scripts/Admin
$ vi backup.sh
#!/bin/sh
# This script will run rsync only when the specified user “user” logs out.
if [ "$1" = "user" ]
then
/usr/bin/rsync –az –e “ssh –i /Users/user/.ssh/batchkey” /local/directory/to/backup user@backupserver:~/backup/

fi
exit 0
$ chmod ug+x backup.sh
#Test it!
$ defaults write com.apple.loginwindow LogoutHook /Library/Scripts/Admin/backup.sh
$ exit #get out of the root shell before you forget
Read the Apple doc linked above for more information on Login/Logout Hooks.
The authorized_keys file will cause ssh to hang when trying to login with the -i option if the ‘no-pty’ option is included. It should look like this:
no-port-forwarding,no-X11-forwarding,no-agent-forwarding,from=”myhost” ssh-dss


Installation and Configuration for Windows 2000/XP
Welcome to hell. Download Cygwin and pick some packages:

An editor (nano, vi, whatever) from +Editors;
Rsync from +Net;
Cygrunsrv from +Admin if you plan to use Rsync as a Windows service;
Openssh from +Net (this will also install the ssl packages).

Check the links below for an article on setting up Cygwin for rsync. There are some good tips on how to install it for use only by an Admin account so your users can’t mess up the settings. It is worth noting that it is probably easiest to do the following from the account you intend to back up, just to be sure that permissions are correct and everything is where you thought it would be.

The tests above still need to be done to ensure that you are able to connect through ssh, so run the following from the cygwin shell:

$ ssh user@remotehost

So far, so good. Generate the key pair:

$ cd ~/.ssh
$ ssh-keygen –t dsa –f batchkey –N “”

Install the public key:
$ scp batchkey.pub username@backupserver:~/

Log in to your backup server and do the following:
Backupserver$ cat batchkey.pub >> ~/.ssh/authorized_keys
Backupserver$ chmod 600 ~/.ssh/authorized_keys
Backupserver$ mv batchkey.pub ~/.ssh

If you read the section on GNU/Linux, it is almost exactly the same. If you haven’t read it, go back to it and check out the section referencing changing the key entry to limit what can be done with the public key login. NOTE: do not add ‘no-pty’ to the key entry. I haven’t had a chance to check out why, but that entry will cause your log in to hang and you’ll have to kill it. Note that your .ssh directory is NOT in your Windows home directory (c:\Documents and Settings\User), it is in your Cygwin home directory. To see what this is, enter ‘pwd’ in the Cygwin shell.

Now you need to make sure that you can use ssh with your key. Log out of the backup server and try to log back in using your key:

$ ssh –i ~/.ssh/batchkey user@host

If it fails, go back and check out the authorized_keys file.

Once you have that sorted, try the rsync command from the cygwin shell:

rsync –avz –e “ssh –i ~/.ssh/batchkey” /local/directory/to/backup/ user@backupserver:~/backup/

The rsync command you put in your batch file will not look like this. If this works, try a batch file like this:

@cls
@echo off

rem rsync backup script

C:\cygwin\bin\rsync -az -e “C:\cygwin\bin\ssh -i /home/Administrator/.ssh/batchkey” /cygdrive/c/”Documents and Settings”/Administrator/Desktop/backup/ user@backupserver:~/backup/

The line starting with C is all one line. I had so much trouble getting that to work that I couldn’t be bothered trying to make it look pretty. If you have trouble, add ‘PAUSE’ to the next line, so it looks like this:

@cls
@echo off

rem rsync backup script

C:\cygwin\bin\rsync -az -e “C:\cygwin\bin\ssh -i /home/Administrator/.ssh/batchkey” /cygdrive/c/”Documents and Settings”/Administrator/Desktop/backup/ user@backupserver:~/backup/

PAUSE

This will prompt you to strike a key before continuing, giving you a chance to read any error messages. Save it as something.bat, fix any problems, then remove the PAUSE.

Copy the script to C:\WINDOWS\system32\GroupPolicy\User\Scripts\Logoff, then launch gpedit.msc from the Run prompt. Click on User configuration -> Windows Settings -> Scripts (Logon/Logoff), then click on Logoff. Click Add, browse for your batch file, and select it.

Do some stuff to your backup, like deleting a file or two, then logoff from the client machine, log back in and check your backup. Whatever you removed should have been replaced by your backup batch file.

Resources

Read through at least the first three, they contain a lot of good information and I relied upon them heavily to get my set up working, the fourth is a good quick reference, and the last you should have on your shelf.

No comments:

Post a Comment