Rdiff-backup

Installation

Sur Debian 5 --> v1.2.5 (la différence de version mineure (.5 pour .8) ne provoque qu'un avertissement)

apt-get install rdiff-backup

Si on veut une version plus récente, il faut compiler :

apt-get install librsync-dev python-dev
wget http://savannah.nongnu.org/download/rdiff-backup/rdiff-backup-1.2.8.tar.gz
tar -xvzf rdiff-backup-1.2.8.tar.gz
cd rdiff-backup-1.2.8
python setup.py install

Sur CentOS, installation repos RPMForge (http://wiki.centos.org/AdditionalResources/Repositories/RPMForge).
Sur CentOS4 :

wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.1-1.el4.rf.i386.rpm

Sur CentOS5 :

wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.1-1.el5.rf.i386.rpm

Puis

rpm -i rpmforge-release-0.5.1-1.el*.rf.i386.rpm
vi /etc/yum.repos.d/rpmforge.repo # --> mettre enabled à 0
yum --enablerepo=rpmforge install rdiff-backup

---

Test d'accès à un serveur :

rdiff-backup --test-server root@192.168.1.21::/home

---

Sauvegarde à partir d'un backup existant (cp ou rsync) : le répertoire destination existe mais n'est pas au format rdiff (manque les infos rdiff-data), il faut le forcer la première fois :

rdiff-backup --force -v5 root@192.168.1.2::/etc /backup/etc > rdiff_server1.log 2>&1

Ensuite on peut enlever le "--force".

---

Nettoyage Des incréments supérieurs à une fenêtre de rétention :

rdiff-backup -v5 --remove-older-than 4W /backup

Le "4W" signifie "4 weeks", soit 4 semaines à conserver. On peut donner un nombre de jours ("D") ou d'heures ("H"). Tous les incréments supérieurs à ce délai seront supprimés.

---

Liste des incréments :
Sur le serveur de sauvegarde :

rdiff-backup  -l /home/backup/server1/home/

A partir du serveur sauvegardé (remote-schema n'est utile que si le port ssh n'est pas standard) :

rdiff-backup -l --remote-schema='ssh -p 225 -C %s rdiff-backup --server' root@192.168.1.250::/home/backup

---

Restore - Exemples

# /etc/hosts d'il y a 1 heure (--force si le fichier existe et qu'il faut l'écraser)
rdiff-backup -v5 -r "1h" --force ./etc/hosts root@192.168.1.2::/etc/hosts

# /etc/ d'il y a 3 jours
rdiff-backup -v5 -r "3D" --force ./etc root@192.168.1.2::/etc

---

DEPANNAGE Si après un problème de sauvegarde (disque plein, coupure de session ssh, etc...), les backups suivants se terminent en erreur (erreurs checksum ou autres), il faut supprimer le répertoire rdiff-backup-data et relancer un backup avec --force pour repartir à zéro (on perd l'historique des incréments !).

---

Exemple de script rdbackup.sh

Lancé du serveur de backup

#!/bin/bash
# Backup Script (RDIFF-BACKUP on SSH)
# FSo 06/2010
# V 2.1
# from sbackup.sh (RSYNC)
# Changelog :
# 2.1 - Added handle for a pre-backup script (if pre-backup script ended with error <> 0, main script stops)
# 2.0 - Moving variables in a .conf file
#   Variables :
#SERVERNAME=$1
#SERVERIP=$2
#SERVERPORT=$3
#SERVERUSR=$4
#RETENTION="--remove-older-than $5"
#LIST=$6
#EXCLUDE=$7
#PRESCRIPT=$8
#BCK_DEV=/home
#BCK_DIR=$BCK_DEV/backup/$SERVERNAME
#RDIFF_OPTS="-v5 --force"

# Fix variables
LOG_FILE=rdiff_ephoto.log
STAT_FILE=rdiff_ephoto.stats
PID_FILE=rdiff_ephoto.pid

if [ $# -gt 0 ]
then
# let use the given full path .conf file
   . $1
else
# default .conf file must be placed in the same directory than .sh
   . $(dirname $0)/rdbackup.conf
fi

while read line
do
   REMOTE_DIR="$REMOTE_DIR $line"
   shift
done < $LIST

if [ ! -e $BCK_DIR/$PID_FILE ]
then
 # Script pre-backup
   if [ -f $PRESCRIPT ]
   then
      $PRESCRIPT
      if [ $? -ne 0 ]
      then
         exit 1
      fi
   fi

   echo $$ > $BCK_DIR/$PID_FILE
   /usr/bin/logger "Sauvegarde rdiff $SERVERNAME"
   if [ -e  $BCK_DIR/$LOG_FILE ]
   then
        mv $BCK_DIR/$LOG_FILE $BCK_DIR/$LOG_FILE.0
   fi
   if [ -e  $BCK_DIR/$STAT_FILE ]
   then
        mv $BCK_DIR/$STAT_FILE $BCK_DIR/$STAT_FILE.0
   fi
   echo "Sauvegarde rdiff $SERVERNAME" > $BCK_DIR/$LOG_FILE
   #Boucle de backup
   ERROR_FLAG=0
   ERROR=0
   ERRORS="ERREURS: "
   for rdir in $(echo $REMOTE_DIR)
   do
      if [ ! -d ${BCK_DIR}${rdir} ]
      then
         mkdir -p ${BCK_DIR}${rdir}
      fi

      if [ ! "x$EXCLUDE" == "x" ]
      then
         EXCLUDE_OPT=""
         while read line
         do
            if  [[ $line =~ ^$rdir ]]
            then
               EXCLUDE_OPT="$EXCLUDE_OPT --exclude $line"
            fi
            shift
         done < $EXCLUDE
      fi
      # rdiff
      date >> $BCK_DIR/$LOG_FILE
      echo >> $BCK_DIR/$LOG_FILE
      echo RDIFF-BACKUP OF $rdir >> $BCK_DIR/$LOG_FILE
      echo rdiff-backup $RDIFF_OPTS --remote-schema "ssh -p $SERVERPORT %s rdiff-backup --server" $EXCLUDE_OPT $SERVERUSR@$SERVERIP::$rdir ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE
      rdiff-backup $RDIFF_OPTS --remote-schema "ssh -p $SERVERPORT %s rdiff-backup --server" $EXCLUDE_OPT $SERVERUSR@$SERVERIP::$rdir ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE 2>&1
      ERROR=$?
      if [ $ERROR -ne 0 ]
      then
         ERRORS="$ERRORS $BASE=$ERROR:"
         ERROR_FLAG=1
      else
         echo CLEANING BACKUPS OF $rdir >> $BCK_DIR/$LOG_FILE
         echo rdiff-backup $RDIFF_OPTS $RETENTION ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE 2>&1
         rdiff-backup $RDIFF_OPTS $RETENTION ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE 2>&1
         ERROR=$?
         if [ $ERROR -ne 0 ]
         then
            ERRORS="$ERRORS $BASE=$ERROR:"
            ERROR_FLAG=1
         fi
      fi
      echo Backup statistics of $rdir >> $BCK_DIR/$STAT_FILE
      cat ${BCK_DIR}${rdir}/rdiff-backup-data/session_statistics.$(date "+%Y-%m-%d")* >> $BCK_DIR/$STAT_FILE
      echo >> $BCK_DIR/$STAT_FILE
   done
   if [ $ERROR_FLAG -eq 1 ]
   then
      (echo $ERRORS
       date
       echo Voir $BCK_DIR/$LOG_FILE
       grep -i warning $BCK_DIR/$LOG_FILE
       grep -i error $BCK_DIR/$LOG_FILE
       cat $BCK_DIR/$STAT_FILE
      )| mail -s "ERREUR RDIFF SUR $SERVERNAME $(date +%d/%m)" $MAIL_ADMIN
   else
      (date
       ls -l $BCK_DIR/$LOG_FILE
       df -k $BCK_DIR
       cat $BCK_DIR/$STAT_FILE
      )| mail -s "Sauvegarde $SERVERNAME $(date +%d/%m) terminee" $MAIL_ADMIN
   fi
   date >> $BCK_DIR/$LOG_FILE
   /usr/bin/logger "FIN rdiff $SERVERNAME"
   rm -f $BCK_DIR/$PID_FILE
else
    (echo "$HOSTNAME:$BCK_DIR/$PID_FILE existe : abandon de $0") | mail -s "Erreur sur backup $SERVERNAME" $MAIL_ADMIN
fi

---

Fichier rdbackup.conf

# Configuration rdbackup
SERVERNAME=server.example.net
SERVERIP=192.168.1.10
SERVERPORT=22
SERVERUSR=root
MAIL_ADMIN=<Mail Admin>
RETENTION="--remove-older-than 1W"
LIST=/opt/systools/backup/server.list
EXCLUDE=/opt/systools/backup/server.excl
PRESCRIPT=
# ou : PRESCRIPT=./server_pre_backup.sh
BCK_DIR=/home/backup/$SERVERNAME
RDIFF_OPTS="-v5 --force"

rdbackup2.2.sh : lancé du serveur A SAUVEGARDER vers le serveur de backup

#!/bin/bash
# Backup Script (RDIFF-BACKUP over SSH and RSYNC)
# FSo 06/2010
# V 2.2
# from sbackup.sh (RSYNC)
# Changelog :
# 2.2 - Reverse backup from client to backup server
# 2.1 - Added handle for a pre-backup script (if pre-backup script ended with error <> 0, main script stops)
# 2.0 - Moving variables in a .conf file 

# Fix variables
EPHOTO_DIR=/home/.sites/28/site1
LOG_FILE=rdiff_ephoto.log
STAT_FILE=rdiff_ephoto.stats
PID_FILE=rdiff_ephoto.pid

if [ $# -gt 0 ]
then
# let use the given full path .conf file
   . $1
else
# ePhoto specific
   . $EPHOTO_DIR/rdbackup.conf
fi

while read line
do
   REMOTE_DIR="$REMOTE_DIR $line"
   shift
done < $LIST

if [ ! -e $EPHOTO_DIR/$PID_FILE ]
then
 # Script pre-backup
   if [ -f $PRESCRIPT ]
   then
      $PRESCRIPT
      if [ $? -ne 0 ]
      then
         exit 1
      fi
   fi
   mysqldump ephoto $MYSQLDUMP_OPTS --password=$MYSQL_PASSWORD > $EPHOTO_DIR/mysqldump.sql
   echo $$ > $EPHOTO_DIR/$PID_FILE
   /usr/bin/logger "Sauvegarde rdiff $SERVERNAME"
   if [ -e  $EPHOTO_DIR/$LOG_FILE ]
   then
        mv $EPHOTO_DIR/$LOG_FILE $EPHOTO_DIR/$LOG_FILE.0
   fi
   if [ -e  $EPHOTO_DIR/$STAT_FILE ]
   then
        mv $EPHOTO_DIR/$STAT_FILE $EPHOTO_DIR/$STAT_FILE.0
   fi
   echo "Sauvegarde rdiff $SERVERNAME" > $EPHOTO_DIR/$LOG_FILE
   #Boucle de backup
   ERROR_FLAG=0
   ERROR=0
   ERRORS="ERREURS: "
   for rdir in $(echo $REMOTE_DIR)
   do
      if [ ! "x$EXCLUDE" == "x" ]
      then
         EXCLUDE_OPT=""
         while read line
         do
            if  $line =~ ^$rdir 
            then
               EXCLUDE_OPT="$EXCLUDE_OPT --exclude $line"
            fi
            shift
         done < $EXCLUDE
      fi
      # rdiff
      date >> $EPHOTO_DIR/$LOG_FILE
      echo >> $EPHOTO_DIR/$LOG_FILE
      echo RDIFF-BACKUP OF $rdir >> $EPHOTO_DIR/$LOG_FILE
      echo rdiff-backup $RDIFF_OPTS --remote-schema "ssh -p $SERVERPORT %s rdiff-backup --server" $EXCLUDE_OPT ${rdir} $SERVERUSR@$SERVERIP::${BCK_DIR}$rdir >> $EPHOTO_DIR/$LOG_FILE
      rdiff-backup $RDIFF_OPTS --remote-schema "ssh -p $SERVERPORT %s rdiff-backup --server" $EXCLUDE_OPT ${rdir} $SERVERUSR@$SERVERIP::${BCK_DIR}$rdir >>  $EPHOTO_DIR/$LOG_FILE 2>&1
      ERROR=$?
      if [ $ERROR -ne 0 ]
      then
         ERRORS="$ERRORS $BASE=$ERROR:"
         ERROR_FLAG=1
      else
         echo CLEANING BACKUPS OF $rdir >> $EPHOTO_DIR/$LOG_FILE
         echo rdiff-backup $RDIFF_OPTS $SERVERUSR@$SERVERIP::$RETENTION ${BCK_DIR}${rdir} >> $EPHOTO_DIR/$LOG_FILE 2>&1
         rdiff-backup $RDIFF_OPTS $RETENTION $SERVERUSR@$SERVERIP::${BCK_DIR}${rdir} >> $EPHOTO_DIR/$LOG_FILE 2>&1
         ERROR=$?
         if [ $ERROR -ne 0 ]
         then
            ERRORS="$ERRORS $BASE=$ERROR:"
            ERROR_FLAG=1
         fi
      fi
   done
   if [ $ERROR_FLAG -eq 1 ]
   then
      (echo $ERRORS
       date
       echo Voir $EPHOTO_DIR/$LOG_FILE
       grep -i warning $EPHOTO_DIR/$LOG_FILE
       grep -i error $EPHOTO_DIR/$LOG_FILE
      )| mail -s "ERREUR RDIFF SUR $SERVERNAME $(date +%d/%m)" $MAIL_ADMIN
   else
      (date
       ls -l $EPHOTO_DIR/$LOG_FILE
       df -k $EPHOTO_DIR
      )| mail -s "Sauvegarde $SERVERNAME $(date +%d/%m) terminee" $MAIL_ADMIN
   fi
   date >> $EPHOTO_DIR/$LOG_FILE
   /usr/bin/logger "FIN rdiff $SERVERNAME"
   rm -f $EPHOTO_DIR/$PID_FILE
else
    (echo "$HOSTNAME:$EPHOTO_DIR/$PID_FILE existe : abandon de $0") | mail -s "Erreur sur backup $SERVERNAME" $MAIL_ADMIN
fi

---

Pour mémoire, script v1 sans rdbackup.conf

#!/bin/bash
# Script de sauvegarde des serveurs (RDIFF sur SSH)
# FSo 06/2010
# v1.0 from sbackup.sh (RSYNC) v1.2
# Changelog :
#
# svg_server.sh <hostname> <ip> <port> <user valide pour ssh> <duree retention> <fichier liste backup> <fichier exclusions>
# exemple cron.d :
#00 02 * * *      root /backup/rdbackup.sh server.example.fr 123.123.12.123 22 root 1W /backup/server.list /backup/server.excl > /backup/log/svg_server.cron.err 2>&1
#
SERVERNAME=$1
SERVERIP=$2
SERVERPORT=$3
SERVERUSR=$4
RETENTION="--remove-older-than $5"
LIST=$6
EXCLUDE=$7
BCK_DEV=/home
BCK_DIR=$BCK_DEV/backup/$SERVERNAME
LOG_FILE=rdiff_$SERVERNAME.log
STAT_FILE=rdiff_$SERVERNAME.stats
PID_FILE=/var/run/rdiff_$SERVERNAME.pid
RDIFF_OPTS="-v5 --force"

if [ $# -lt 6 ]
then
   echo Usage : $0 serverFQDN serverIP serverSSHPORT SSHUSER RETENTION BACKUPLISTfile [EXCLUDEfile]
   exit 1
fi

shift; shift; shift
while read line
do
   REMOTE_DIR="$REMOTE_DIR $line"
   shift
done < $LIST

if [ ! -e $PID_FILE ]
then
   echo $$ > $PID_FILE
   /usr/bin/logger "Sauvegarde rdiff $SERVERNAME"
   if [ -e  $BCK_DIR/$LOG_FILE ]
   then
        mv $BCK_DIR/$LOG_FILE $BCK_DIR/$LOG_FILE.0
   fi
   if [ -e  $BCK_DIR/$STAT_FILE ]
   then
        mv $BCK_DIR/$STAT_FILE $BCK_DIR/$STAT_FILE.0
   fi
   echo "Sauvegarde rdiff $SERVERNAME" > $BCK_DIR/$LOG_FILE 
   #Boucle de backup
   ERROR_FLAG=0
   ERROR=0
   ERRORS="ERREURS: "
   for rdir in $(echo $REMOTE_DIR)
   do
      if [ ! -d ${BCK_DIR}${rdir} ]
      then
         mkdir -p ${BCK_DIR}${rdir}
      fi 

      if [ ! "x$EXCLUDE" == "x" ]
      then
         EXCLUDE_OPT=""
         while read line
         do
            if [[ $line =~ ^$rdir ]]
            then
               EXCLUDE_OPT="$EXCLUDE_OPT --exclude $line"
            fi
            shift
            done < $EXCLUDE
      fi
      # rdiff
      date >> $BCK_DIR/$LOG_FILE
      echo >> $BCK_DIR/$LOG_FILE
      echo RDIFF-BACKUP OF $rdir >> $BCK_DIR/$LOG_FILE
      echo rdiff-backup $RDIFF_OPTS --remote-schema "ssh -p $SERVERPORT %s rdiff-backup --server" $EXCLUDE_OPT $SERVERUSR@$SERVERIP::$rdir ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE
      rdiff-backup $RDIFF_OPTS --remote-schema "ssh -p $SERVERPORT %s rdiff-backup --server" $EXCLUDE_OPT $SERVERUSR@$SERVERIP::$rdir ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE 2>&1
      ERROR=$?
      if [ $ERROR -ne 0 ]
      then 
         ERRORS="$ERRORS $BASE=$ERROR:"
         ERROR_FLAG=1
      fi
      echo CLEANING BACKUPS OF $rdir >> $BCK_DIR/$LOG_FILE
      echo rdiff-backup $RDIFF_OPTS $RETENTION ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE 2>&1
      rdiff-backup $RDIFF_OPTS $RETENTION ${BCK_DIR}${rdir} >> $BCK_DIR/$LOG_FILE 2>&1
      ERROR=$?
      if [ $ERROR -ne 0 ]
      then 
         ERRORS="$ERRORS $BASE=$ERROR:"
         ERROR_FLAG=1
      fi
      echo Backup statistics of $rdir >> $BCK_DIR/$STAT_FILE
      cat ${BCK_DIR}${rdir}/rdiff-backup-data/session_statistics.$(date "+%Y-%m-%d")* >> $BCK_DIR/$STAT_FILE 
      echo >> $BCK_DIR/$STAT_FILE
   done
   if [ $ERROR_FLAG -eq 1 ]
   then
      (echo $ERRORS
       date
       echo Voir $BCK_DIR/$LOG_FILE
       grep -i warning $BCK_DIR/$LOG_FILE
       grep -i error $BCK_DIR/$LOG_FILE
       cat $BCK_DIR/$STAT_FILE
      )| mail -s "ERREUR RDIFF SUR $SERVERNAME $(date +%d/%m)" mail_admin@example.net
   else
      (date
       ls -l $BCK_DIR/$LOG_FILE
       df -k $BCK_DIR
       cat $BCK_DIR/$STAT_FILE
      )| mail -s "Sauvegarde $SERVERNAME $(date +%d/%m) terminee" mail_admin@example.net
   fi
   date >> $BCK_DIR/$LOG_FILE
   /usr/bin/logger "FIN rdiff $SERVERNAME"
   rm -f $PID_FILE
else
    (echo "$SERVERNAME:$PID_FILE existe : abandon de $0") | mail -s "Erreur sur backup $SERVERNAME" mail_admin@example.net
fi