backing up a collection of snapshot subvolumes
I have a remote machine with a filesystem for which I periodically take incremental snapshots for historical reasons. These snapshots are stored in an archival filesystem tree on a file server. Older snapshots are removed and newer ones added on a rotational basis. I need to be able to backup this archive by syncing it with a set of backup drives. Due to the size, I need to back it up incrementally rather than sending the entire content each time. Due to the snapshot rotation, I need to be able to update the state of the archive backup filesystem as a whole, in much the same manner that rsync handles file trees.
It seems that I cannot use "btrfs send", as the archive directory contains the snapshots as subvolumes.
I cannot use rsync as it treats subvolumes as simple directories, and does not preserve subvolume attributes. Rsync also does not support reflinks, so the snapshot directory content will no longer be reflinked to other snapshots on the archive backup. I cannot use hard links in the incrementals as hard links do not cross subvolume boundaries.
Thoughts anyone ?
Quoted 9 lines · J. Hart
I have a remote machine with a filesystem for which I periodically take incremental snapshots for historical reasons. These snapshots are stored in an archival filesystem tree on a file server. Older snapshots are removed and newer ones added on a rotational basis. I need to be able to backup this archive by syncing it with a set of backup drives. Due to the size, I need to back it up incrementally rather than sending the entire content each time. Due to the snapshot rotation, I need to be able to update the state of the archive backup filesystem as a whole, in much the same manner that rsync handles file trees.
If I have understood your requirement correctly, this seems to be exactly matched to the capabilities of btrbk. I use btrbk to maintain a similar backup disk which contains a full copy of my main data disk along with various snapshots.
Quoted 2 lines
It seems that I cannot use "btrfs send", as the archive directory contains the snapshots as subvolumes.
I'm not sure that you mean. If your problem is that btrfs send does not cross subvolume boundaries then that is true: you would need to configure btrbk to back up each subvolume. I have a cron job that checks that all subvolumes (except the snapshots btrbk creates) are listed in my btrbk configuration file.
Graham
Am Tue, 25 Apr 2017 00:02:13 -0400
schrieb "J. Hart" <[email protected]>:
Quoted 14 lines
I have a remote machine with a filesystem for which I periodically take incremental snapshots for historical reasons. These snapshots are stored in an archival filesystem tree on a file server. Older snapshots are removed and newer ones added on a rotational basis. I need to be able to backup this archive by syncing it with a set of backup drives. Due to the size, I need to back it up incrementally rather than sending the entire content each time. Due to the snapshot rotation, I need to be able to update the state of the archive backup filesystem as a whole, in much the same manner that rsync handles file trees.
It seems that I cannot use "btrfs send", as the archive directory contains the snapshots as subvolumes.
Quoted 8 lines
I cannot use rsync as it treats subvolumes as simple directories, and does not preserve subvolume attributes. Rsync also does not support reflinks, so the snapshot directory content will no longer be reflinked to other snapshots on the archive backup. I cannot use hard links in the incrementals as hard links do not cross subvolume boundaries.
Thoughts anyone ?
If this is for archival purpose only and storage efficiency and speed is your primary concern, try borgbackup.
Borgbackup deploys its own deduplication and doesn't rely on btrfs snapshot capabilities. You can store your archives wherever you want (even on non-btrfs), and it won't store any data blocks twice.
Upon restore, you could simply recreate the snapshot/subvolume from a similar tree and then rsync a tree restored from borgbackup back to this snapshot to get back a state with the benefits of btrfs snapshots.
Borg also has an adapter for fuse to get a mounted view into the archives, tho, it is slow and probably takes a lot of RAM. But it is good enough for single file lookups and navigation.
Signature · 4 lines
Regards, Kai Replies to list-only preferred.