Planning for subvolumes of subvolumes and btrfs send/receive
Can you help me design this right?
Long story short, I'm wondering if I can use btrfs send to copy sub subvolumes (by snapshotting a parent subvolume, and hopefully getting all the children underneath). My reading so far, says no.
So, Ideally I would have:
/mnt/btrfs_pool/backup: backup is a subvolume of btrfs_pool
/mnt/btrfs_pool/backup/debian/machine1:/mnt/btrfs_pool/backup/debian/machine2:/mnt/btrfs_pool/backup/debian/machine3:
machinex would be a subvolume ideally speaking for quota counting purposes
/mnt/btrfs_pool/backup/debian/machine1/current/mnt/btrfs_pool/backup/debian/machine1/201401/mnt/btrfs_pool/backup/debian/machine1/201308
...
current would be a subvolume, that would be snapshotted to 201401 with an rsync of diffs on top
Same thing for 201308
In other words, they'd also be subvolumes, and I'm using snapshots so that I can collapse the identical data (I'm currently using hardlinks for that).
This would all work fine from what I can see, but from here, if I try to
do
btrfs subvolume snapshot -r /mnt/btrfs_pool/backup/debian/machine1 /mnt/btrfs_pool/backup/debian/machine1_ro_send do to a btrfs send next, my understanding is that /mnt/btrfs_pool/backup/debian/machine1_ro_send will effectively have no data because all my data is in the subvolumes
under that.
As a result, I'd then need a complicated script that goes in each subvolume and makes independent snapshots for btrfs send/receive.
(In this case, I'm copying my backup data to another backup backup server)
Is there an easier way to do this, or you can really only btrfs send the very last level of subvolume if I want the data in there?
I suppose I could use cp -a --reflink instead of snapshots to replicate the identical data, but the reason I don't love this is that there is no way I know to replicate that reflink relationship to another filesystem unless I use btrfs send/receive. Is that correct?
Thanks,
Marc
Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901Marc MERLIN posted on Sun, 20 Apr 2014 12:46:27 -0700 as excerpted:
Quoted 3 lines
Long story short, I'm wondering if I can use btrfs send to copy sub subvolumes (by snapshotting a parent subvolume, and hopefully getting all the children underneath). My reading so far, says no.
I don't do much with subvolumes and absolutely nothing with send/receive myself, but from what I've read, no, for the simple case (but see below) that cannot work.
The reason is that snapshots (including the read-only snapshot done for a send) stop at subvolume borders.
In the large-internal-write-NOCOW-file dedicated subvolume case, that's a positive, since the dedicated subvolume can be used to avoid snapshotting the nocow stuff on the subvolume when snapshotting a parent, but in this case the same subvolume property is a negative.
Now somebody *DID* mention multi-subvolume sends, and indeed, checking the manpage, there appears to be allowance for that by naming multiple subvolumes (and multiple clone-sources when doing the incremental), *BUT*, there appears to be no corresponding allowance for multiple subvolume snapshotting, so your script would need a "for subvol in $subvols" type construct at least for the snapshotting, and once you do that, you might as well do multiple sends as well, as that's likely to be more robust than a the larger all-in-one send, if one of the subvolume sends/receives fails for some reason.
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 5 lines · Marc MERLIN
Can you help me design this right?
Long story short, I'm wondering if I can use btrfs send to copy sub subvolumes (by snapshotting a parent subvolume, and hopefully getting all the children underneath). My reading so far, says no.
That's my understanding thus far also. Seems like first a recursive read-only snapshot creation is a pre-requisite, since a read-only snapshot is needed first in order to send it.
I think the seed device method is a better way to do this, though I'm not sure what state its in.
Chris Murphy
Quoted 5 lines · Chris Murphy
Quoted 2 lines · Marc MERLIN
Can you help me design this right?
Quoted 7 lines
Quoted 3 lines
Long story short, I'm wondering if I can use btrfs send to copy sub subvolumes (by snapshotting a parent subvolume, and hopefully getting all the children underneath). My reading so far, says no.
That's my understanding thus far also. Seems like first a recursive read-only snapshot creation is a pre-requisite, since a read-only snapshot is needed first in order to send it.
I think the seed device method is a better way to do this, though I'm not sure what state its in.
Thank you both for the confirmation.
This indeed makes it too much of a pain for me, so I think I will go back to making hardlinks like I did with ext4
That way I won't need subvolumes of subvolumes and will be able to do snapshots and replication more easily.
Thanks,
Marc
Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901Quoted 12 lines · Marc MERLIN
Quoted 11 lines · Chris Murphy
Quoted 5 lines · Marc MERLIN
Can you help me design this right?
Long story short, I'm wondering if I can use btrfs send to copy sub subvolumes (by snapshotting a parent subvolume, and hopefully getting all the children underneath). My reading so far, says no.
That's my understanding thus far also. Seems like first a recursive read-only snapshot creation is a pre-requisite, since a read-only snapshot is needed first in order to send it.
Quoted 8 lines
Quoted 1 line
I think the seed device method is a better way to do this, though I'm not sure what state its in.
Thank you both for the confirmation.
This indeed makes it too much of a pain for me, so I think I will go back to making hardlinks like I did with ext4
That way I won't need subvolumes of subvolumes and will be able to do snapshots and replication more easily.
I see hard links as completely different to either subvolume/snapshot/reflink. Three hardlinks for a file all point to one file, they're aren't four unique files. But with the latter, three reflinks or snapshots are independent from the parent, including permissions/ACLs, and xattr including selinux labels.
Chris Murphy
Quoted 1 line · Chris Murphy
I see hard links as completely different to either subvolume/snapshot/reflink. Three hardlinks for a file all point to one file, they're aren't four unique files. But with the latter, three reflinks or snapshots are independent from the parent, including permissions/ACLs, and xattr including selinux labels.
You are correct, but if I'm deduping files for backups of the same data, hardlinks work fine, but more importantly, they do not get lost if I copy the filesystem to another machine.
reflink relationship pretty much gets lost no matter what tool you use, except hopefully btrfs send/receive.
Marc
Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/