Incremental Backups How to set up a simple but effective backup system

[rvalkass 5]

This is a sort of follow up to my topic about my new backup hardware: My New Backup Solution

 

We all know we should take regular backups of the contents of our PCs, but how many of us actually do? My previous method for taking backups was burning a pile of DVDs whenever I felt like it. A nightmare to do, even worse to store, and absolute hell if I ever decided to restore anything from one of them! So, I went in search of a new backup method, and here I present the software part of my solution.

 

What are Incremental Backups?

 

Most times when some software makes a backup, it backs up everything. Every file, every folder, gets copied and neatly stored away, ready for you to retrieve next time something goes wrong. Nice, but it takes an awfully long time, and takes up the same space as whatever is already on your hard drive. Perhaps it gets compressed, but it doesn't make a massive difference. The backups are still huge.

 

Incremental backups, however, only backup changed files. Each backup is stored as a compressed archive file (of whatever format you choose). Each archive contains a copy of the files changed since the last backup was made. After so many of these a 'checkpoint' is made, holding the latest version of all those files. This system can be nested many levels deep. This means that backup sizes can be kept down, and yet they are also quick to restore.

 

For example, the system I have set up has 3 levels, each of 5 backups. So, 5 level-3 backups are made, then a level-2 backup containing all those changes. After 5 level-2 backups (5x5 level-3 backups), a level-1 backup is made. After 5 level-1 backups are made (5x5x5 level-3 backups) then a level-0 backup is taken. A level-0 backup is a full backup, containing all files and folders in their current state. At this rate, I only need to take a full backup every 125 days.

 

What Software is Available?

 

I run Linux, so I opted for backup2l. It uses a simple text configuration file, is freely available, and performs all the backup and restore tasks completely maintenance free.

 

How Do You Set it Up?

Install the package for backup2l, either using Adept, Synaptic, or running apt-get install backup2l.

Open the configuration file: /etc/backup2l.conf

Edit the relevant sections. Everything is fairly well explained, but here's a summary of my changes:

Line 52: Change the path to your backup device (e.g. external hard drive)

Line 113: Uncomment (remove the # at the start) to confirm that you have configured it

Set up a cron job to run backup2l however often you want it to run:

Open a command line and type: sudo crontab -e

Add a line to run backup2l each night at 23:15 : 15 23 * * * backup2l --backup

Save the file

You're done!

Does it Work?

 

Yes! At the moment, my full backups are 33.6GB in size. The level-3 backups are currently around 100MB in size. The advantage is that I now have nightly backups of my system, without the hassle of taking a 33.6GB backup every day!

 

But I Don't Run Linux!

 

Shame on you

 

There are other pieces of software that do the same job for Windows and Mac. However, I haven't tried any of these, so try them out first to make sure they work. If you find one you like, pop a little review below so everyone can find it.

 

Freebyte Backup for Windows

SilverKeeper for Mac

[OpaQue 15]

ON Linux, We achieve this using the command rsync.rsync -auv-a, --archive (this will preserve permissions)-u, --update (this will skip files that are newer on the reciever)-v, -- verbose (so you can see what is going on)rsync -auv /path/to/source/directory /path/to/destination/directory

[acricket 0]

yeah its very good.. i will give a try admin

[Mr Inviso 0]

For windows XP. I use a program call livecopy. When I first started using it a few years ago it was free ware. I just went and Goggled it to get the site link and found out it is now Shareware. With a $30.00 price tag. I'm not surprised. It is a great program for keeping synchronized copies of folders.

[miladinoski 1]

This is a sort of follow up to my topic about my new backup hardware: My New Backup Solution

 

...

Great post rvalkass. I knew of programs that can do an incremental backup, but only for WindowsŽ and Apple's Mac OS XŽ which I despise btw and I will definitely try and use it because I rarely do manual backups Anyways, you should write more things like this, I'm happy to be in a forum with many Linux users and geeks

 

ON Linux, We achieve this using the command rsync.

 

rsync -auv

 

-a, --archive (this will preserve permissions)

-u, --update (this will skip files that are newer on the reciever)

-v, -- verbose (so you can see what is going on)

 

rsync -auv /path/to/source/directory /path/to/destination/directory

Nice addition to rvalkass' post.

For windows XP. I use a program call livecopy. When I first started using it a few years ago it was free ware. I just went and Goggled it to get the site link and found out it is now Shareware. With a $30.00 price tag. I'm not surprised. It is a great program for keeping synchronized copies of folders.

That's the price you pay for running WindowsŽ

[Tks 0]

Thanks for the piece of work you share but there is one more addition I would like to know if you could help me with.My friend basicall works on 3D stuff so they need backup copies of everything in place so as to lose minimal amount of data. I have implemented RAID-1 (Mirroring) there.To me that is fine with their backup requirements but there is thing my friend told me that caught my attention. If some file is corrupted, that corrupted file get mirrored on the other drive too. My friend, when he worked in his previous office had a Linux box where they implemented some kind of script or a piece of software. What it did was it made backup of the data 3 times over. like 1 folder with all the backup there and then another folder with a full backup after 10 minutes and another one after 10 minutes. These three folder were moved over and over like after another 10 minutes, the first backup moved to the second and the second moved in thirds position and the third backup, before 20 minutes was scraped off and inplace of the first one a whole new backup was created so if any file was corrupted, they would take the file off of the backup created 10 or 20 minuted before.I hope I have cleared what I am trying to say here. I have been searching on google for some time here for something that could help me out on this, if anyone of you could help me out on this that it would be a big favour.

[sakmac 0]

For windows, I would recommend Norton Ghost or something along those lines, because it can update an image of your computer, so if your whole computer crashes you can literally reinstall your whole computer, and all your settings there. It is a paid software so you shouldn't buy it if your computer doesn't meet the min. system requirements, because you know how bad norton is on lower end computers lol.

[ragav.bpl 0]

This is a sort of follow up to my topic about my new backup hardware: My New Backup Solution

 

We all know we should take regular backups of the contents of our PCs, but how many of us actually do? My previous method for taking backups was burning a pile of DVDs whenever I felt like it. A nightmare to do, even worse to store, and absolute hell if I ever decided to restore anything from one of them! So, I went in search of a new backup method, and here I present the software part of my solution.

 

What are Incremental Backups?

 

Most times when some software makes a backup, it backs up everything. Every file, every folder, gets copied and neatly stored away, ready for you to retrieve next time something goes wrong. Nice, but it takes an awfully long time, and takes up the same space as whatever is already on your hard drive. Perhaps it gets compressed, but it doesn't make a massive difference. The backups are still huge.

 

Incremental backups, however, only backup changed files. Each backup is stored as a compressed archive file (of whatever format you choose). Each archive contains a copy of the files changed since the last backup was made. After so many of these a 'checkpoint' is made, holding the latest version of all those files. This system can be nested many levels deep. This means that backup sizes can be kept down, and yet they are also quick to restore.

 

For example, the system I have set up has 3 levels, each of 5 backups. So, 5 level-3 backups are made, then a level-2 backup containing all those changes. After 5 level-2 backups (5x5 level-3 backups), a level-1 backup is made. After 5 level-1 backups are made (5x5x5 level-3 backups) then a level-0 backup is taken. A level-0 backup is a full backup, containing all files and folders in their current state. At this rate, I only need to take a full backup every 125 days.

 

What Software is Available?

 

I run Linux, so I opted for backup2l. It uses a simple text configuration file, is freely available, and performs all the backup and restore tasks completely maintenance free.

 

How Do You Set it Up?

Install the package for backup2l, either using Adept, Synaptic, or running apt-get install backup2l.

Open the configuration file: /etc/backup2l.conf

Edit the relevant sections. Everything is fairly well explained, but here's a summary of my changes:

Line 52: Change the path to your backup device (e.g. external hard drive)

Line 113: Uncomment (remove the # at the start) to confirm that you have configured it

Set up a cron job to run backup2l however often you want it to run:

Open a command line and type: sudo crontab -e

Add a line to run backup2l each night at 23:15 : 15 23 * * * backup2l --backup

Save the file

You're done!

Does it Work?

 

Yes! At the moment, my full backups are 33.6GB in size. The level-3 backups are currently around 100MB in size. The advantage is that I now have nightly backups of my system, without the hassle of taking a 33.6GB backup every day!

 

But I Don't Run Linux!

 

Shame on you

 

There are other pieces of software that do the same job for Windows and Mac. However, I haven't tried any of these, so try them out first to make sure they work. If you find one you like, pop a little review below so everyone can find it.

 

Freebyte Backup for Windows

SilverKeeper for Mac

Hi rvalkass,

 

Glad to read your post as it is really very very informative and i am sure it will help a lot of people.

 

I have a question to you as I use a program call livecopy for windows xp, is it a good program to use for this purpose hope to listen from you soon....

[rvalkass 5]

I have a question to you as I use a program call livecopy for windows xp, is it a good program to use for this purpose hope to listen from you soon....

I've never used LiveCopy, but it appears to be more of a synchronising software, something similar to Microsoft's Briefcases. Basically, it would keep two folders synchronised to make sure they both always have the latest versions of the files inside them, no matter which copy was edited. So no, unfortunately that's not really a suitable piece of software for doing incremental backups. Take a look at the link I provided in my original post for Windows software, or search for some: search:incremental backup windows

[Tks 0]

Thanks for replying to my post but I was basically refering to the linux operating system for this. I have used Norton Ghost and believe me, it IS a system hog for all the resources available. It's not limited to just lower end systems but I have a QuadCore 2.4 GHz and 2 Gigabytes of Ram. It's still seems to slow down the computer.Isn't there anything like a script or something backs up the whole Hard Drive with a different of 10 minutes and creating 3 backups like I mentioned above. I am still trying to look for scripts and softwares on the internet that could do this for me but alas! I still didn't have any luck with it.

[rvalkass 5]

Thanks for replying to my post but I was basically refering to the linux operating system for this. I have used Norton Ghost and believe me, it IS a system hog for all the resources available. It's not limited to just lower end systems but I have a QuadCore 2.4 GHz and 2 Gigabytes of Ram. It's still seems to slow down the computer.Isn't there anything like a script or something backs up the whole Hard Drive with a different of 10 minutes and creating 3 backups like I mentioned above. I am still trying to look for scripts and softwares on the internet that could do this for me but alas! I still didn't have any luck with it.

It would be possible with Linux and backup2l. Have a cron job run backup2l every 10 minutes, and configure backup2l to keep however many backups you wanted. Just beware that a full hard disk backup will take longer than 10 minutes, so you would want to use it incrementally, so that only changed files are stored in the majority of backups (full backups are made much less often).