btrfs send/receive for backup of subvolumes

Hi

My query speicific to incremental backup/recovery with btrfs send/receive option available for btrfs subvolume backups.

say i have /b1/s1 subvolume and snapshot of s1 is /b1/snap1_s1

I run btrfs send /b1/snap_s1 | btrfs receive /backup where /backup is my backup btrfs file system.

Now I will take snapshot of s1 again ( after modifiying data in s1) which is /b1/snap2_s1

Using btrfs send -p snap1_s1 snap2_s2 and destination as /backup I can send incremental to backup store.

My question is is there a way I can have option recover files from /backup from incremental backup or full backup (i.e. from copy of snap2_s1 or snap1_s1)

And I do not want to make 2 copies of 1 full and 1 incremetal backup since if full backup is 1Gb and if there is 0.6Gb change for incremental, I would like to backup only difference for incremental backup but still would able to get option to recover files from full or incremental from backedup subvolume.

is there a way I can achieve this using current btrfs send/receive and other snapshot and tracking options?

thank you in advance.

sri posted on Tue, 05 May 2015 11:30:27 +0000 as excerpted:

own. But in case you didn't and for others who may google this thread...

Disclaimer: While I'm a btrfs user and list regular, I'm not a btrfs dev, and my own use-case doesn't involve send/receive, so while the following is as I understand the functionality based on the wiki documentation and previous list discussion, I've never used send/receive myself, so if actual functionality appears to conflict with the below, consider me wrong and go with actual functionality...

AFAIK, when you btrfs receive a send, btrfs creates a subvolume duplicating the one being sent from the send side.

So in the scenario above, you should end up with two different snapshots under /backup, one for s1 and one for s2. Due to the parent relationship and the fact that btrfs is copy-on-write, where extents in s1 and s2 differ, the difference is sent (as a copy of the new extent along with metadata describing its place in reference to the parent) and the snapshots occupy the space used by both. Where they do NOT differ, there is no difference to send, and the snapshots share the space, so while either one can be accessed, they're actually referencing the same shared extent and only that shared copy of that extent exists.

So as long as you keep both snapshots, you can access both, but given 1 GiB in snap_s1 and 0.6 GiB worth of changes between it and snap_s2, the total space used by the two should still be only 1.6 GiB, even tho you can access the full 1 GiB of either snapshot.

Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Disclaimer: I'm just a btrfs user, not an expert.

When dealing with a snapshot/subvolume, it should not matter how it was created - once instantiated, the file contents should always be the same. The resulting filesystem on the destination you've targeted with "btrfs receive" should be usable in exactly in the same way as the original regardless of whether it was created using an incremental send or otherwise.

If you'd like some guarantee other than just blind faith that this is the case, I've written a script [1] to help report sha512sums and PGP signatures of directories. I wrote it to detect if the content of my snapshots remain exactly (or at least mostly - see [2]) the same regardless of what the eventual filesystem is (btrfs or not) and how many send/receive transports had occurred.

You may find the full set of my scripts at [3] useful, the main script can automatically snapshot all of your btrfs filesystems and each of their subvolumes (or not, if you like). It does so by creating snapshots under a '.snapshotz' directory in the root of each subvolume, in the form of:
/path/to/subvol/.snapshotz/
YYYY-MM-DDTHHMMSS+hhmm (i.e. a valid
isodate). It also takes care of snapshot pruning, accumulating measurement reports, and I've included some helper scripts to automate transport of snapshots over ssh or between local filesystems.

Feel free to share any feedback you may have, but check the list of bugs at [4] before you start using this code seriously.

[1] https://github.com/csirac2/snazzer/blob/master/doc/snazzer-measure.md
[2] https://bugzilla.kernel.org/show_bug.cgi?id=95201
[3] https://github.com/csirac2/snazzer
[4] https://github.com/csirac2/snazzer/issues