Copying related snapshots to another server with btrfs send/receive?
Another question I just came up with.
If I have historical snapshots like so:
backup
backup.sav1
backup.sav2
backup.sav3
If I want to copy them up to another server, can btrfs send/receive let me copy all of the to another btrfs pool while keeping the duplicated block relationship between all of them?
Note that the backup.sav dirs will never change, so I won't need incremental backups on those, just a one time send.
I believe this is supposed to work, correct?
The only part I'm not clear about is am I supposed to copy them all at once in the same send command, or one by one?
If they had to be copied together and if I create a new snapshot of
backup: backup.sav4
If I use btrfs send to that same destination, is btrfs send/receive indeed able to keep the shared block relationship?
Thanks,
Marc
Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901Quoted 18 lines · Marc MERLIN
Another question I just came up with.
If I have historical snapshots like so:
backup
backup.sav1
backup.sav2
backup.sav3If I want to copy them up to another server, can btrfs send/receive let me copy all of the to another btrfs pool while keeping the duplicated block relationship between all of them?
Note that the backup.sav dirs will never change, so I won't need incremental backups on those, just a one time send.
I believe this is supposed to work, correct?The only part I'm not clear about is am I supposed to copy them all at once in the same send command, or one by one?
Quoted 8 lines
If they had to be copied together and if I create a new snapshot of
backup: backup.sav4If I use btrfs send to that same destination, is btrfs send/receive indeed able to keep the shared block relationship?
Thanks,
Marc
I'm not sure if they can be sent in one go. :-/
Sending one-at-a-time, the shared-data relationship will be kept by using the -p (parent) parameter. Send will only send the differences and receive will create a new snapshot, adjusting for those differences, even when the receive is run on a remote server.
$ btrfs send backup | btrfs receive $path/ $ btrfs send -p backup backup.sav1 | btrfs receive $path/ $ btrfs send -p backup.sav1 backup.sav2 | btrfs receive $path/ $ btrfs send -p backup.sav2 backup.sav3 | btrfs receive $path/ $ btrfs send -p backup.sav3 backup.sav4 | btrfs receive $path/
Signature · 4 lines
__________ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97
Quoted 10 lines · Brendan Hide
Sending one-at-a-time, the shared-data relationship will be kept by using the -p (parent) parameter. Send will only send the differences and receive will create a new snapshot, adjusting for those differences, even when the receive is run on a remote server.
$ btrfs send backup | btrfs receive $path/ $ btrfs send -p backup backup.sav1 | btrfs receive $path/ $ btrfs send -p backup.sav1 backup.sav2 | btrfs receive $path/ $ btrfs send -p backup.sav2 backup.sav3 | btrfs receive $path/ $ btrfs send -p backup.sav3 backup.sav4 | btrfs receive $path/
So this is exactly the same than what I do incremental backups with brrfs send, but -p only works if the snapshot is read only, does it not? I do use that for my incremental syncs and don't mind read only snapshots there, but if I have read/write snapshots that are there for other reasons than btrfs send incrementals, can I still send them that way with -p?
(I thought that wouldn't work)
Thanks,
Marc
Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901Quoted 19 lines · Marc MERLIN
Quoted 10 lines · Brendan Hide
Sending one-at-a-time, the shared-data relationship will be kept by using the -p (parent) parameter. Send will only send the differences and receive will create a new snapshot, adjusting for those differences, even when the receive is run on a remote server.
$ btrfs send backup | btrfs receive $path/ $ btrfs send -p backup backup.sav1 | btrfs receive $path/ $ btrfs send -p backup.sav1 backup.sav2 | btrfs receive $path/ $ btrfs send -p backup.sav2 backup.sav3 | btrfs receive $path/ $ btrfs send -p backup.sav3 backup.sav4 | btrfs receive $path/So this is exactly the same than what I do incremental backups with brrfs send, but -p only works if the snapshot is read only, does it not? I do use that for my incremental syncs and don't mind read only snapshots there, but if I have read/write snapshots that are there for other reasons than btrfs send incrementals, can I still send them that way with -p?
(I thought that wouldn't work)
Quoted 2 lines
Thanks,
Marc
Yes, -p (parent) and -c (clone source) are the only ways I'm aware of to push subvolumes across while ensuring data-sharing relationship remains intact. This will end up being much the same as doing incremental backups:
From the man page section on -c:
"You must not specify clone sources unless you guarantee that these
snapshots are exactly in the same state on both sides, the sender and the receiver. It is allowed to omit the '-p <parent>' option when '-c <clone-src>' options are given, in which case 'btrfs send' will determine a suitable parent among the clone sources itself."
-p does require that the sources be read-only. I suspect -c does as well. This means that it won't be so simple as you want your sources to be read-write. Probably the only way then would be to make read-only snapshots whenever you want to sync these over while also ensuring that you keep at least one read-only snapshot intact - again, much like incremental backups.
Signature · 4 lines
__________ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97
Quoted 11 lines · Brendan Hide
Yes, -p (parent) and -c (clone source) are the only ways I'm aware of to push subvolumes across while ensuring data-sharing relationship remains intact. This will end up being much the same as doing incremental backups:
From the man page section on -c:
"You must not specify clone sources unless you guarantee that
these snapshots are exactly in the same state on both sides, the sender and the receiver. It is allowed to omit the '-p <parent>' option when '-c <clone-src>' options are given, in which case 'btrfs send' will determine a suitable parent among the clone sources itself."
Right. I had read that, but it was not super clear to me how it can be useful, especially if it's supposed to find the source clone by itself.
Quoted 1 line
From what you said and what I read, I think the source might be allowed
to be read write, otherwise it would be simpler for btrfs send to know that the source has not changed.
I think I'll have to do more testing with this when I get some time.
Thanks,
Marc
Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901Brendan Hide posted on Sun, 04 May 2014 09:54:38 +0200 as excerpted:
Quoted 7 lines
From the man page section on -c:
"You must not specify clone sources unless you guarantee that these
snapshots are exactly in the same state on both sides, the sender and the receiver. It is allowed to omit the '-p <parent>' option when '-c <clone-src>' options are given, in which case 'btrfs send' will determine a suitable parent among the clone sources itself."
Quoted 6 lines
-p does require that the sources be read-only. I suspect -c does as well. This means that it won't be so simple as you want your sources to be read-write. Probably the only way then would be to make read-only snapshots whenever you want to sync these over while also ensuring that you keep at least one read-only snapshot intact - again, much like incremental backups.
I don't claim in any way to be a send/receive expert as I don't use it for my use-case at all. However...
It's worth noting in the context of that manpage quote, that really the only practical way to guarantee that the snapshots are exactly the same on both sides is to have them read-only the entire time. Because the moment you make them writable on either side all bets are off as to whether something has been written, thereby killing the exact-same-state guarantee. =:^(
*However*: snapshotting a read-only snapshot and making the new one writable is easy enough[1]. Just keep the originals read-only so they can be used as parents/clones, and make a second, writable snapshot of the first, to do your writable stuff in.
---
[1] Snapshotting a snapshot: I'm getting a metaphorical flashing light saying I need to go check the wiki FAQ that deals with this again before I post, but unfortunately I can't check out why ATM as I just upgraded firefox and cairo and am currently getting a blank window where the firefox content should be, that will hopefully be gone and the content displayed after I reboot and get rid of the still loaded old libs, so unfortunately I can't check that flashing light ATM and am writing blind. Hopefully that flashing light warning isn't for something /too/ major that I'm overlooking! =:^(
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 7 lines · Duncan
*However*: snapshotting a read-only snapshot and making the new one writable is easy enough[1]. Just keep the originals read-only so they can be used as parents/clones, and make a second, writable snapshot of the first, to do your writable stuff in.
---
[1] Snapshotting a snapshot: I'm getting a metaphorical flashing light
I already snapshot ro snapshots as rw snapshots and that works fine. It's actually rely on this in my script:
http://marc.merlins.org/perso/btrfs/post_2014-03-22_Btrfs-Tips_-Doing-Fast-Incremental-Backups-With-Btrfs-Send-and-Receive.html
(skip to the bottom)
# We make a read-write snapshot in case you want to use it for a chroot
# and some testing with a writeable filesystem or want to boot from a # last good known snapshot. btrfs subvolume snapshot "$src_newsnap" "$src_newsnaprw" $ssh btrfs subvolume snapshot "$dest_pool/$src_newsnap" "$dest_pool/$src_newsnaprw"
Marc
Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901