Are there plans for full btrfs filesystem backups?

I use send/receive to backup some subvolumes, but there is no good way to back up all of my subvolumes in one swoop.
Backing up with tar/cp/rsync is of course no good since it loses all the btrfs attributes including COW deduped blocks.

For one, my recent btrfs filesystem crash reminded me that docker creates a boatload of subvolumes which my backups didn't cacch, so now it's all gone.

Are there any plans for a full filesystem send/receive, not just a subvolume send/receive?

Note that this is the same problem when you are migtrating to new drives or bcache, and you can't play tricks with adding a virtual drive of 8TB just to mirror to it and remove the original mirror later.

Thanks,
Marc

"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901

Marc MERLIN posted on Sun, 10 May 2015 09:49:44 -0700 as excerpted:

Recent discussion suggests yes, tho it's a relatively new plan.

More specifically, the recent discussion seemed to settle around the need for a send/receive --recursive option, which would recurse into nested subvolumes. IIRC, the systemd folks requested that for their container management, among other things. And if I remember the thread correctly, I /believe/ one of the devs said they agreed and were taking on the project.

That would of course allow recursing from ID5/the-root-subvolume, thus filling your request, but it would be more flexible than full filesystem send/receive, as it could handle arbitrary subvolume subtrees, too.

Previous to that, I don't believe it was on the roadmap. It reads like a bunch of people, now including you, independently becoming aware that it was a problem at roughly the same time.

Meanwhile, there's another potential alternative as well, for the whole filesystem non-incremental case. If you're backing up everything, you can of course dd/netcat the individual component devices making literal byte for byte copies, tho you'd want to do it to a remote machine, to avoid the kernel detecting a local copy as part of the same filesystem, due to UUID duplication. But incrementals, parallel to send/receive with parent, would seem to be impractical.

Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

That was for snapshot creation and destruction, not send receive.

To me, an exact Btrfs clone suggests only a Btrfs destination at the moment, because there's no superset functionality elsewhere for everything in Btrfs to exist. One way to get to an "exact clone" would be Btrfs seed device to effectively freeze the original, and then "send" the entire seed somehow rather than just a subvolume.

A btrfs send file on non-Btrfs destinations, is a maybe, the problem there is no checksums exist in the send file and maybe some other things (?) but presumably that could be extended so that the file could be a complete representation of the file system warts and all - which itself may not be desirable so it's like, how exactly do we really want these things to be? Hmm?

Chris Murphy

Chris Murphy posted on Sun, 10 May 2015 22:57:24 -0600 as excerpted:

I believe that was it, yes.

But of course recursive snapshot creation/deletion would be a practical prerequisite to recursive send/receive, since in that case read-only snapshots would need to be created in ordered to be sent, and similarly created on the receive side to recreate the nested snapshot tree on the send side.

Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman