Skip to contentSkip to conversation

btrfs-send-receive vs rsync for (incremental/full) backups

DISCUSSION Martin 3 participants 4 messages Dec 13, 2013 Btrfs file system

lore
Share & tools
lore

Martin

author
#0 lore raw

OK... So for backing up across a local network to a second physical host...

Is btrfs-send-receive stable enough now to be used?

How does send-receive compare to using rsync for backups?

Any comments please from those using such things?

Thanks,
Martin

#1 lore raw
Quoted 3 lines · Martin

OK... So for backing up across a local network to a second physical host...

Is btrfs-send-receive stable enough now to be used?

It seems to be OK for me at the moment -- I'm not running it across
a network (yet), but I am using it between different filesystems on the same host, using pipes.

Quoted 1 line

How does send-receive compare to using rsync for backups?

It's more fiddly to get right, and needs a bit more thought about
how to set it up sanely. I have a python library that I use to write my backup scripts. I'll have to write it up one of these days...

Quoted 1 line

Any comments please from those using such things?

Works for me, so far, as far as I know. Although we've got recent
reports of *something* still going wrong with systemd journal files. No investigation of that yet, though.

Hugo.

Signature · 4 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- In one respect at least, the Martians are a happy people: ---    
                          they have no lawyers.                          
#2 lore raw
Quoted 8 lines · Hugo Mills
Quoted 3 lines · Martin

OK... So for backing up across a local network to a second physical host...

Is btrfs-send-receive stable enough now to be used?

It seems to be OK for me at the moment -- I'm not running it across
a network (yet), but I am using it between different filesystems on the same host, using pipes.

Question. Are the source file system's checksums (and metadata) preserved in the send file? Or does the send file just contain data, and checksums are recomputed on receive? I'm curious if the send file can be stored on a non-checksumming file system, and yet have a means to verify the integrity of the files once received. Either the checksums in the send file being check during receive, or subsequent to receive completion by using scrub?

And yes I can confirm that send outputting to a file on a 2nd drive, and receiving it to yet another drive does work, and the source and destination subvolumes are the same in seemingly every way. I didn't separately rsync checksum compare the two subvolumes, however. That seems like possibly a good test. I also haven't tried doing it over a network.

Chris Murphy

#3 lore raw
Quoted 6 lines · Chris Murphy
Quoted 3 lines · Hugo Mills
Quoted 2 lines · Martin

OK... So for backing up across a local network to a second physical host...

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Is btrfs-send-receive stable enough now to be used?

It seems to be OK for me at the moment -- I'm not running it across
a network (yet), but I am using it between different filesystems on the same host, using pipes.

Question. Are the source file system's checksums (and metadata) preserved in the send file?

I don't think checksums are transported (from what I remember of
the FAR format).

Quoted 6 lines

Or does the send file just contain data, and checksums are recomputed on receive? I'm curious if the send file can be stored on a non-checksumming file system, and yet have a means to verify the integrity of the files once received. Either the checksums in the send file being check during receive, or subsequent to receive completion by using scrub?

The FAR format contains a sequence of what amount to FS commands:
there are equivalents of cp, rm, chmod, chown, setfattr, cat (of data inline in the FAR file). There's also a data type that clones an extent from a file in another subvolume with a given UUID. Assuming you have some way of keeping track of subvolume UUIDs on an FS that doesn't support them, there's no reason you can't generate or unpack FAR files on a non-checksumming (and non-CoW) FS.

Quoted 6 lines

And yes I can confirm that send outputting to a file on a 2nd drive, and receiving it to yet another drive does work, and the source and destination subvolumes are the same in seemingly every way. I didn't separately rsync checksum compare the two subvolumes, however. That seems like possibly a good test. I also haven't tried doing it over a network.

Hugo.

Signature · 4 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Once is happenstance; twice is coincidence; three times ---     
                            is enemy action.