[How to] Make periodic backups of whole OSMC system


#41

The errors you are getting are because nfs doesn’t support extended attributes. See here.

The files should still be backed up - they just won’t have their extended attributes attached. Whether this is a problem I wouldn’t like to say.

In testing this, I find that the backup takes forever over a network because rsync on the OSMC device has to examine every file on the backup device. That could be speeded up by running an rsync daemon on the backup device but I didn’t have that in mind when I wrote the script. I just use a local (USB) disc so no problems with xattrs and it takes about 40mins. Note that I keep TV shows on my SD card and delete them when viewed so there is usually a few GB of changes to back up and it takes longer than it would if all my media was on a NAS.

HTH


#42

My tests showed that backups are slower over NFS but not that bad.

The Vero 4K test directly to a SanDisk Ultra SD card backed up 2.1GB in 4 minutes 53 seconds.That’s about 7MB/sec.

The Pi 3 over a 100 Mbit/sec LAN to an NFS4 server backed up 4.33GB in just over 35 minutes.That’s about 2MB/sec, including network overheads.


#43

Not too bad. My test yesterday backed up a 16GB (full) linux partition to an nfs attached to a Pi on USB, through wifi (n). Extra space taken: 2.8GB, time taken 8hrs 39mins :unamused:

I suspect a straight copy of 16GB would have been quicker.


#44

That’s quite impressive, though WiFi introduces a wildcard.

Of course, my tests were first-time copies. I’ll try again with a day-2 backup.


#45

I repeated Pi 3 over NFS , this time for day-2 backup.

It took 34 minutes, as against 35 minutes for the initial run.

osmc@osmc:~$ sudo ./osmc-backup-0.1.5
Mon Jun 12 19:03:51 BST 2017 Backup version 0.1.5
Going to back up to /mnt/archive/2017-06-12 (unless I can't find it).
looking for mounted media
media found at /mnt/archive
10801280 KiB available on drive

Counting and deleting backups older than 2017-06-05 or if more than 7 of them
Backup from 2017-06-11 kept
Now 10801280 KiB available on drive

system is on /dev/mmcblk0p2
Backing up system partition to /mnt/archive/2017-06-12/system
There are 1 previous backups
Found latest backup at /mnt/archive/2017-06-11/system
Incremental backup: copying new and changed files to /mnt/archive/2017-06-12/system
rsync: rsync_xal_set: lsetxattr(""/mnt/archive/2017-06-12/system/usr/bin/systemd-detect-virt"","security.capability") failed: Operation not supported (95)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]
Making a copy of the new and changed files in /mnt/archive/2017-06-12/system-delta
Cloning unchanged files from previous backup
Deleting files no longer on source
Time to back up system: 00h:34m:14s

boot is on /dev/mmcblk0p1
Backing up boot partition to /mnt/archive/2017-06-12/boot
There are 1 previous backups
Found latest backup at /mnt/archive/2017-06-11/boot
Incremental backup: copying new and changed files to /mnt/archive/2017-06-12/boot
Making a copy of the new and changed files in /mnt/archive/2017-06-12/boot-delta
Cloning unchanged files from previous backup
Deleting files no longer on source
Time to back up boot: 00h:00m:25s

126688 KiB used in this backup
10674592 KiB available on drive
All done!

#46

:slight_smile:


#47

Sorry if this is a stupid question, but given the differential nature of the backups, how do you actually restore one specific one? Is there a command to incrementally integrate and restore in order all the previous versions until you get the latest?


#48

Each backup is a complete snapshot at the time it was made. To restore, you just copy sudo cp -a all the files in boot/ to the boot partition and all the files in system/ to the main partition. You can’t copy them back to an SD card which is running OSMC. You have to plug both the target SD card and the backup device into a machine running linux (OSMC, Raspbian, Ubuntu or whatever).


#49

@dillthedog @grahamh
If you want to have a backup solution that also works well over slow network connections take a look at http://borgbackup.readthedocs.io


#50

Indeed. But this is not just about backup for me. I want to have a copy of my files, not a load of jigsaw pieces that can only be put together with a particular piece of software. And sometimes compare files across different times or different distros. Not being able to do it across a network doesn’t bother me, either, but I do want to understand what the implications of using nfs or smb are if ppl want to do that.


#51

While it requires a piece of software the interisting part is that you can mount each version of the incremental backup as an individual Filesystem which easily would allow you to compare different versions.


#52

Have you managed to get it running on OSMC / armhf? The installation instructions seem to be a bit non-intuitive.

I also get the impression that it needs quite a lot of memory for the dedupe hash table.


#53

Looks like borg does do xattrs: http://borgbackup.readthedocs.io/en/stable/installation.html#features-platforms
(read the table wrong before)


#54

Which part are you referring to? The table says that xattr is supported on Linux ARM.


#55

No, haven’t tried it on OSMC itself as I use a nfs root install so the OSMC backup comes automatically with my server backup.

But a quick search indicate that it must be technically possible.


#56

I ran into a few issues yesterday with pip. Will have another look if I get time today.


#57

Well reading that github ticket, memory will definitely be an issue when running borg-client on the Pi.


#58

So presumably any xattrs are not needed by OSMC and we don’t have to worry about preserving them in a backup?


#59

Not sure if you referring to your rsync method or the borg-backup approach.
Generally I am not aware that OSMC uses xattrs for it’s operation but for sure we can not exclude that anyone might use it.


#60

Neither - or both! What I’m saying is if your system files are an nfs share, then the OSMC device will not be able to read xattrs even if it wanted to.