backups

This is the nice solution for those who use Cpanel server ( version 11 and above ) to setup rsync directly from whm .

Latest edit: rsyncable option is now included in cpbackup by cPanel, so all the instructions below are moot for version 11 and up.

Edits: The simplified version after much testing and trial is provided below. Thanks to Chirpy for finding the exact method that works so simply! Also added a simple rotation script.

Modify the root crontab changing the line shown as follows:

Quote:
0 1 * * * /scripts/cpbackup

to:

0 1 * * * export GZIP=”–rsyncable” ; /scripts/cpbackup 

That’s it!

Here’s a simple script to rotate the backups:
Code:
#!/bin/sh

# This script rotates the backup files daily, keeping 7 days worth.

DATEFORMAT=$ (date +%a)
BACKUPSOURCE=/backup
BACKUPDEST=/Data/backup/rotation/

rm -Rf $ BACKUPDEST/$ DATEFORMAT
mkdir $ BACKUPDEST/$ DATEFORMAT
cp -R $ BACKUPSOURCE/* $ BACKUPDEST/$ DATEFORMAT
The original post is below.

******** Original Post ************
Note: This post may become obsolete if it turns out that setting a permanent environment variable fixes the issue. If so, this will get updated to reflect that.

Using the Full Backup feature built into WHM is nice, because it grabs pretty much everything and creates stand-alone backups. The problem with this backup, however, is that because it uses gzip compression, it isn’t rsync-friendly.

Rsync is a nifty file transfer tool that only transfers the changes in a file. So, once you have downloaded a backup once, rather than download the whole backup again to update it, it only sends the changes to the file and incorporates them into the existing file. In the case of files that have been compressed using the default gzip settings, almost the entire file is transferred again because of the adaptive compression algorithm.
Ok, seriously, the good news is that later versions of gzip have the capability of playing nice with rysnc by way of the ‘–rsyncable’ switch. With an rsyncable gzip file, rsync can process it much more efficiently. (You can check your version of gzip to see if it will handle rsync by looking at the help: ‘gzip –help’)

So how do we make that happen? The good news is it’s a simple, one-line modification to the ‘cpbackup’ script located in the /scripts directory. The bad news is that cpanel will overwrite this file every time it runs the ‘upcp’ script. This means we have to protect it somehow. Another nifty script will take care of that.

So, first, let’s modify the ‘cpbackup’ script:

1) make a copy of the script and call it ‘cpbackupunedited’

Quote:
cp /scripts/cpbackup /scripts/cpbackupunedited 

2) Open the ‘cpbackup’ script and add the following line just after the remarks at the top, but before the ‘BEGIN’ statement
Quote:
$ ENV{‘GZIP’} .= “–rsyncable”; 

3) Save the script. Now, make another copy of the file, this time calling it ‘cpbackupedited’
Quote:
cp /scripts/cpbackup /scripts/cpbackupedited 

Your backups are now going to work with rsync rather nicely! For instance, on the first test I made using a 1.5gig set of gzip files, 35 megs were transferred, in contrast to the 1.2gigs transferred without the –rsyncable switch. That speeds up transfer times quite a bit!

Now, we need to protect our cpbackup file. While we could ‘chattr’ so that it can’t be overwritten, there is another, more refined method that will let us know if there are changes in the script that need to be looked at. This is a modification of the ‘watchwwwacct’ script you can read more about elsewhere on the forum. I’ll post just the modified script for sake of brevity:

 

Quote:
#!/usr/bin/perl
# You may need to change this path to /usr/local/bin/perl

$ mailprog = “/usr/sbin/sendmail”;

#************************************************* *************
#
# Script to monitor cpbackup script customizations: watchcpbackup V1.1
#
#
# Set a few variables below and upload this script to anywhere on your server,
# then set a cron job to run the script every hour. I put mine in a subfolder
# of the servers scripts folder. (/scripts/custom)
#
# To set the cronjob, in shell, type crontab -e, then enter what’s between the quotes
# on the following line as a new line in your cron listings:
# “0 * * * * /scripts/custom/watchcpbackup.cgi”
#
# Ownership and permissions of the script should be root:root, and 0700.
#
# This script only needs to run after a cpanel upgrade, so you could set
# the cron job to run it 1 hour after upcp runs, but then if you change
# when upcp runs, you will need to change this also. This script is fast,
# so it’s easier to just run it every hour.
#
# **** Using this script ****
# ————————————————————————————–
# To use this script, you need to make a copy of the real cpbackup script and call it
# cpbackupunedited, then after your customizations, make a copy of your custom one and
# call it cpbackupedited. These copies need to be in the same directory as cpbackup.
# What this script does is compare your custom cpbackup script to a copy of it.
# If a cpanel update changes the cpbackup script, this script will notice the change.
# Now the fun part. Many upgrades only change the cpbackup script back to what it was
# originally. If it’s changed, this script compares the new cpbackup to a copy of the
# original one. If the update merely wrote the original one back, it would match the
copy. This script would then take the copy of your custom one and reuse it. Now, if
# the update altered the script, you would be emailed and told that your customizations
# are lost and that you will need to redo them. The script does tell you where changes
# were made to help with reapplying your customizations.
# customized my cpbackup script over a year ago and have only had to redo the changes
# maybe half a dozen times.
# ————————————————————————————–
#
#
# Registered users of this script will be notified of any future updates.
# If you registered this copy with me, put your email here for future reference.
# This copy is registered to:
#
#************************************************* *************
# This is where the email will be sent when a change is detected.
# If you use spamassassin, you should include a name, like this:
# $ sendto_email = ‘YourName <you@youremail.com>’;
$ sendto_email = ‘YourName <you@youremail.com>’;

# This is the sender for the email message.
# Change it if you wish.
$ sender_email = ‘YourName <you@youremail.com>’;

# This is where your cpbackup file is located.
# It shouldn’t need to be changed.
$ path = “/scripts”;

 

$ diff1 = system(“cmp $ path/cpbackup $ path/cpbackupedited”);

if ($ diff1 eq “0″) {
exit;
}
else {
$ diff2 = system(“cmp $ path/cpbackup $ path/cpbackupunedited”);
}

if ($ diff2 eq “0″) {
system(“cp -f $ path/cpbackupedited $ path/cpbackup”);

# Open The Mail Program
open(MAIL,”|$ mailprog -t”);
print MAIL “Content-Type: text/html; charset=iso-8859-1\n”;

print MAIL “To: $ sendto_email\n”;
print MAIL “From: $ sender_email\n”;

print MAIL “Subject: cpbackup file changed and restored\n”;

print MAIL “<b>The cpbackup file was changed back to the original and has been automatically replaced with the edited version.</b><br><br>\n\n”;

close (MAIL);

}

if ($ diff2 ne “0″) {

# Open The Mail Program
open(MAIL,”|$ mailprog -t”);
print MAIL “Content-Type: text/html; charset=iso-8859-1\n”;

print MAIL “To: $ sendto_email\n”;
print MAIL “From: $ sender_email\n”;

print MAIL “Subject: cpbackup file changed\n”;

print MAIL “<b>The cpbackup file has been changed and no longer matches the original file. You will need to redo your custom work.</b><br><br>\n\n”;

close (MAIL);

} 

Lastly, we need to setup our crontab to run this watcher script *after* ‘upcp’ runs, but *before* the cpbackup script runs. Edit the crontab (as root, obviously):

 

Quote:
crontab -e 

Look for the line with ‘/scripts/upcp’ and change it:
Quote:
40 0 * * * /scripts/upcp

becomes

40 0 * * * /scripts/upcp; /scripts/watchcpbackup 

Save it, and you’re done!
If you want to automate the rsyncing of your files, take a peek at the following HOW-TO, which is clear and works great:

ShareThis

Web Hosting Blog