Backups

Once your initial server configuration has been completed, the next step should be to configure the backups for the system.  Ideally backups would be run on a daily basis and performed ad-hoc prior to configuration changes.  Backups should not be kept on the same physical media as the rest of the system.  Instead, tapes or external drives should be used.

Tape drives of any appreciable size are very expensive and as a result, the most common form of backup seen for home servers is typically an external drive.  These are relatively inexpensive and portable, making them ideal for home use.   Since they are portable, it's even feasible for a small organization to use them for backups by purchasing a number of similar units and rotating the drives off-site daily in the same manner as tapes.

Using external disk drives are easy.  USB attached drives should be automatically detected by the system and available for partitioning.  In the event that you are using a direct attached SCSI or SAS device, you can easily rescan the SCSI bus using a convenient script that SUSE has included with the distribution:

rescan-scsi-bus.sh

Once the device has been detected, you can partition it by using the command line...

Command Line:

(Our disk will be /dev/sde)

In a terminal, type:

Fdisk /dev/sde

To create a new partition, use the following options:

N

(for new partition)

P

(primary partition)

1

(Partition number)

After this accept the defaults.  Then, back at the Command Menu, use the following options:

T

(toggle partition type)

L

(list partition codes)

The list presented here shows all of the different types of partitions that can be used for the disk  You can select the code for the partition type that you would like.  (See 'Which partition should I use?' below.)  For this example, we will be using Linux (83).  Once you have selected the code and are back at the Command menu, type:

W

(write the partition information and exit)

Next, format the filesystem by typing:

mke2fs /dev/sde1

Once done, create a mount point location and mount the volume:

mkdir /media/disk

mount /dev/sde1 /media/disk

If you'd like, you can add it to automatically mount by editing /etc/fstab.  However, since this is a backup disk, it's safer to leave it un-mounted until we need to use it.  This is a personal choice.

What partition type should I use?

For the example, we are using the native Linux EXT3.   If you are only planning to mount this volume from this server, this is a simple and safe way to go.  However, there are other options:

• FAT32 - This partition type is easily mountable from all windows devices, so in the event of a server loss, you could easily mount the external disk and access the files from any Windows device or a Linux Server.  There are two draw backs to this.  First, should the drive go missing, anyone else can also easily mount the external device and access the files.   Second, the file system has a 4 GB file limitation, which could cause your backups to fail.
• NTFS - This partition type is easily mountable from most windows devices, so in the event of a server loss, you could easily mount the external disk and access the files from most Windows devices or a Linux Server.  There are two draw backs to this.  First, as with FAT32, should the drive go missing, anyone else can also easily mount the external device and access the files.   Second, the file system write functionality is relatively new in Linux and may not be reliable.
• XFS - This partition type is not mountable from Windows devices, but should easily be mountable in Linux.  The benefit that this has over plain vanilla EXT3 is that it is designed to handle many large files.  If you're planning on storing large files on your server, this may be a good option.

Alternate Disks

If you are planning to use a rotation of removable drives, be sure to follow the partitioning steps (above) for each of the drives that you will be using.  Once this has been completed, move on to the backups.

A note on restores:   Should you use multiple disks for backups, remember to test each of restores!

Backup Options

The Open Source world provides a number of backup options, including TAR, CPIO, AMANDA, etc. We cannot cover every possible option here, so we are going to focus on the most ready examples of a backup in Red Hat, TAR.

Backup Script

So, is one backup method better than the other?  Not necessarily.   They just work in different manners.  TAR is simply ubiquitous and is considered tried and true.

So how do the scripts work?  There are two scripts:

Daily.sh - This script performs daily incremental backups on the list of folders that you specify.  The backups cover all files changed in the last three days, so in the event of a corrupt file or failed backup, you're still covered.

Monthly.sh - This script performs monthly full backups on the list of folders that you specify.  This script also cleans up daily backup files that are older than a month old.

There is no need to edit the scripts unless you'd like to change something.  Instead, there are three files that are read to tell the scripts what you'd like them to do:

Backups.lst - This file contains a list of the folders that you would like to perform daily backups on.

Monthly.lst - This file contains a list of the folders that you would like to perform monthly backups on.  This may be the  same as the daily list, but if you have large data archives, you may not.  For example, you may not want monthly full backups of your MP3 Collection.

Services.lst - Some backups, such as backups of databases, require the services to be deactivated for the duration of the backup.  If this is the case, simply add any services involved into this list and the backup will handle taking them down are starting them.

To use the scripts:

Create a folder for the scripts to live in, the backups to go to and for the logs to go to:

mkdir /root/scripts

mkdir /media/disk/backups

mkdir /var/log/backup

Copy the scripts (below) into their respective file names in /root/scripts.  Don't forget to make them executable:

chmod +x daily.sh

chmox +x monthly.sh

Create the list of folders for the backups (backup.lst) and the monthly backups (monthly.lst), as well as any services that you feel need to be shutdown (services.lst).  

Note that it is possible to specify the root directory (/) as the directory to backup, however if you do that, you'll backup a lot of 'garbage' directories (/dev,/proc), which are not needed.  For the purposes of this article, our backup.lst file will look like:

/root

/etc

/home

/usr/local

The monthly file will be the same folders and the service file will be empty.

We complete the installation by scheduling our backups.  Type:

crontab -e

Add the following lines:

@daily /root/scripts/daily.sh | mail root -s"Daily Backup"

@monthly /root/scripts/monthly.sh | mail root -s"Monthly Backup"

Once we're done, we run the initial fill by typing:

./root/scripts/monthly.sh

If you wish to run daily backups manually (such as before or after changes, this can be done by typing:

./root/scripts/daily.sh

Your backups will be located in /media/disk/backups:

The files are named by:

{date}-{folder}-{backup type}-{random}.tgz

You can restore the files by click on the file from the folder and date from which you'd like to restore.  Then, simply drag the folder back to its original location.   You can also restore everything by going to the root directory and untarring the specific files, starting with monthly.  For example:

tar -zxf 20090913-home-M-FULL-27379.tgz

tar -zxf 20090913-home-D-INCR-15064.tgz

Be sure to test your backups by restoring to a temporary location, such as /tmp!

Daily.Sh:

#!/bin/bash

cd /root/scripts

echo " "

echo "Starting Daily Backup Process:"

echo " "

date

#Creating Variables:

DAY=`date +%Y%m%d`

volume="/media/disk"

BUDIR="/media/disk/backups"

LOGDIR="/var/log/backup"

#Check for mounted volume:

echo "Checking for Mounted Volume:"

echo " "

if mount|grep $volume; then

#echo "Volume is Mounted"

echo " "

else

echo "Volume is Not Mounted - Attempting to Mount...."

mount /dev/sde1 /media/disk

fi

if mount|grep $volume; then

echo "Volume is Mounted - Starting Backup Process:"

else

echo "Volume is Not Mounted - Backups will NOT continue."

exit

fi

echo " "

#Stopping Services:

echo "Stopping Services:"

for s in `cat services.lst`

do

service $s stop

done

#Backups:

for i in `cat backup.lst`

do

BUNAME=`echo $i | awk -F/ '{print $NF}'`

   echo "Creating $BUDIR/$DAY-$BUNAME-D-INCR-$RANDOM.tgz:"

echo " "  

find $i -mtime -2 -type f -print | tar zcvf $BUDIR/$DAY-$BUNAME-D-INCR-$RANDOM.tgz -T -

echo " " >> $LOGDIR/$DAY-D-INCR.log

echo $i: >> $LOGDIR/$DAY-D-INCR.log

find $i -mtime -2 >> $LOGDIR/$DAY-D-INCR.log

echo " "

done

#Starting Services:

echo "Starting Services:"

for s in `cat services.lst`

do

service $s start

done

date

Monthly.sh:

#!/bin/bash

cd /root/scripts

echo " "

echo "Starting Monthly Backup Process:"

echo " "

date

#Creating Variables:

DAY=`date +%Y%m%d`

volume="/media/disk"

BUDIR="/media/disk/backups"

LOGDIR="/var/log/backup"

#Check for mounted volume:

echo "Checking for Mounted Volume:"

echo " "

if mount|grep $volume; then

#echo "Volume is Mounted"

echo " "

else

echo "Volume is Not Mounted - Attempting to Mount...."

mount /dev/sde1 /media/disk

fi

if mount|grep $volume; then

echo "Volume is Mounted - Starting Backup Process:"

else

echo "Volume is Not Mounted - Backups will NOT continue."

exit

fi

echo " "

#Stopping Services:

echo "Stopping Services:"

for s in `cat services.lst`

do

service $s stop

done

#Backups:

echo "Starting Backups"

for i in `cat monthly.lst`

do

BUNAME=`echo $i | awk -F/ '{print $NF}'`

   echo "Creating $BUDIR/$DAY-$BUNAME-M-FULL-$RANDOM.tgz:"

echo " "  

tar zcvf $BUDIR/$DAY-$BUNAME-M-FULL-$RANDOM.tgz $i

echo " " >> $LOGDIR/$DAY-M-FULL.log

echo $i: >> $LOGDIR/$DAY-M-FULL.log

find $i -mtime +0 >> $LOGDIR/$DAY-M-FULL.log

echo " "

done

#Starting Services:

echo "Starting Services:"

for s in `cat services.lst`

do

service $s start

done

#Cleaning up:

echo "Removing Daily backups that are older than two weeks:"

find /media/disk/backups/*-D-* -ctime +14 -delete

echo "Removing Monthly backups that are older than a month:"

find /media/disk/backups/*-M-* -ctime +35 -delete

date