Make a time machine with rsync

... ok so we might have to be careful about violating physics and destroying the universe here, so lets go carefully... I had half an idea that rsync could do something more than just make a simple mirror of files, what is more useful than just a dumb mirror is having a historical versioning of files, sure you could do this with successive mirrors but it could take a long time and certainly is very wasteful... After some time to fettle the command line parameters I think I have what I want... Lets do a little test, first I did an initial backup of a remote server. I created a new file with a single line of text, then using exactly the same command did another backup I then modified the file, and did another backup Finally I deleted the file and you guessed it did another backup In the root of my backup lets look at the backup structure you're only seeing just the file we're interested in...

$ ls backup_2018-05-28_17-42-44/
backup_2018-05-28_17-42-26             mod-del-test.txt

so in our backup directory we have a file and another backup folder

ls backup_2018-05-28_17-42-44/backup_2018-05-28_17-42-26/
mod-del-test.txt

in that directory we have just a file... looking at the content of the files from oldest to newest...

$ cat backup_2018-05-28_17-42-44/backup_2018-05-28_17-42-26/mod-del-test.txt 
this file will be modified and then deleted between backups

$ cat backup_2018-05-28_17-42-44/mod-del-test.txt 
this file will be modified and then deleted between backups
modified!

So this looks good, right at the top we have a backup that's right up to date and even if a file is deleted, you can go back in time and see multiple versions of the file, the deeper you go into the directory hierarchy the further back in time you go. Because it's rather a long winded command to run it makes sense to put it into a simple script

sudo rsync -avz --delete --backup -e ssh \

looking at the first part of the rsync command lets go through the parameters -a is for archive which is a short cut for a number of other options which taken together amount to recursion and I want everything ! -v gives us some extra information, -z uses compression while transmitting the files, --delete --backup and also --backup-dir work in concert

--backup-dir="backup_$(date '+%Y-%m-%d_%H-%M-%S')" \

while the delete parameter might fill you with dread... (will my server be deleted!?) it's actually needed to delete files from the backup that have been deleted, also without it you'd end up with multiple versions of the backup-date folders, rather than one hierarchy. I'm running rsync with sudo here so I can be sure that all the ownership and permissions are preserved. so lets look at one of my backup scripts (yes I have multiple already!)

#!/bin/bash

# change this to your servers web root or whatever want to backup trailing / important
SOURCE=someone@someserver.com:/path/to/some/server/data/

# where the backup will be stored
DESTINATION=/home/auser/remotebackups/someserver/

sudo rsync -avz --delete --backup -e ssh \
            --backup-dir="backup_$(date '+%Y-%m-%d_%H-%M-%S')" \
            $SOURCE $DESTINATION

Even doing an initial backup is fast and sucessive backups are usually lightning quick, should have done this years ago...