using snapshot for backup: best practise?
I am using fsfreeze when running a backup to ensure a consistent filesystem.
While the backup is running writes to the filesystem are suspended and the whole system is unresponsive, e.g. logins are not possible.
On certain errors the unfreeze will not happen and the system is locked forever.
Using snapshots seems a better idea for backups :-)
But snapshots do not include subvolumes.
For example the / filesystem has the subvolumes:
/home/home/tux/test/var/spool
When I run the command:
btrfs subvolume snapshot / /.snapshot/_
the snapshot will contain only the root subvolume.
I have to manually add:
rmdir /.snapshot/_/home
btrfs subvolume snapshot /home /.snapshot/_/home
rmdir /.snapshot/_/home/tux/test
btrfs subvolume snapshot /home/tux/test /.snapshot/_/home/tux/test rmdir /.snapshot/_/var/spool
btrfs subvolume snapshot /var/spool /.snapshot/_/var/spool
Then run the backup on /.snapshot/_ und afterwards:
btrfs subvolume del /.snapshot/_/var/spool
btrfs subvolume del /.snapshot/_/home/tux/test
btrfs subvolume del /.snapshot/_/home
btrfs subvolume del /.snapshot/_
But this will work only for this special example!
And I have hundreds of systems to backup with different filesystem layout!
Is there a best practise "Using snapshots for making backup"? I need automatic detecting, creating and removing of nested snapshots.
Signature · 6 lines
Ullrich Horlacher Server und Virtualisierung Rechenzentrum TIK Universitaet Stuttgart E-Mail: [email protected] Allmandring 30a Tel: ++49-711-68565868 70569 Stuttgart (Germany) WWW: https://www.tik.uni-stuttgart.de/ REF:<[email protected]>
I do various sorts of backups but the main one built around snapshots uses btrbk. It doesn't solve the problem you mention below but it does reduce the problem to listing each subvolume once in the config file (which can easily be automated if you want). For example, here is an extract from the btrbk.conf on one of my systems:
#
# Data pool
#
volume /mnt/data
snapshot_dir btrbk_snapshots
snapshot_create onchange
preserve_day_of_week monday
# On the disk itself only keep recent snapshots
snapshot_preserve_min 5d
snapshot_preserve 10d 4w
timestamp_format long-iso
# On the backup disk keep historic monthlies
target_preserve_min no
target_preserve 30d 8w *m
target send-receive /snapshots/black_snapshots
#Do not snapshot the deb-proxy cache
#subvolume home/cache
# Do not snapshot emergency
#subvolume emergency
subvolume home/boinc-client
# For boinc just keep the latest plus one daily
snapshot_preserve_min latest
snapshot_preserve 1d
target_preserve 4d 1w 1m
subvolume home/cobb
subvolume home/default-user
subvolume home/dmarc-staging
subvolume home/imap-archive
subvolume projects
subvolume home/mywiki
.
.
.
And, of course, btrbk handles all the management of the snapshots themselves (see the first half of the example file above, which keeps 10 daily snapshots and 4 weekly snapshots on the disk itself and a different selection on another disk).
Quoted 45 lines · Ulli Horlacher
I am using fsfreeze when running a backup to ensure a consistent filesystem.
While the backup is running writes to the filesystem are suspended and the whole system is unresponsive, e.g. logins are not possible.
On certain errors the unfreeze will not happen and the system is locked forever.Using snapshots seems a better idea for backups :-)
But snapshots do not include subvolumes.
For example the / filesystem has the subvolumes:
/home/home/tux/test/var/spoolWhen I run the command:
btrfs subvolume snapshot / /.snapshot/_
the snapshot will contain only the root subvolume.
I have to manually add:
rmdir /.snapshot/_/home
btrfs subvolume snapshot /home /.snapshot/_/home
rmdir /.snapshot/_/home/tux/test
btrfs subvolume snapshot/home/tux/test/.snapshot/_/home/tux/test rmdir /.snapshot/_/var/spool
btrfs subvolume snapshot/var/spool/.snapshot/_/var/spoolThen run the backup on /.snapshot/_ und afterwards:
btrfs subvolume del /.snapshot/_/var/spool
btrfs subvolume del /.snapshot/_/home/tux/test
btrfs subvolume del /.snapshot/_/home
btrfs subvolume del /.snapshot/_But this will work only for this special example!
And I have hundreds of systems to backup with different filesystem layout!Is there a best practise "Using snapshots for making backup"? I need automatic detecting, creating and removing of nested snapshots.
Quoted 3 lines · Graham Cobb
I do various sorts of backups but the main one built around snapshots uses btrbk. It doesn't solve the problem you mention below but it does reduce the problem to listing each subvolume once in the config file
btrbk does some sort of snapshot rotation?
For this I have already a perfect solution:
https://fex.belwue.de/linuxtools/snaprotate.html
But snapshots are not a backup.
Signature · 6 lines
Ullrich Horlacher Server und Virtualisierung Rechenzentrum TIK Universitaet Stuttgart E-Mail: [email protected] Allmandring 30a Tel: ++49-711-68565868 70569 Stuttgart (Germany) WWW: https://www.tik.uni-stuttgart.de/ REF:<[email protected]>
Quoted 9 lines · Ulli Horlacher
Using snapshots seems a better idea for backups :-)
But snapshots do not include subvolumes.
When I run the command:
btrfs subvolume snapshot / /.snapshot/_
the snapshot will contain only the root subvolume.
Mewanwhile I have written some (more) tools to handle nested subvolumes and snapshots.
Example:
root@fex:~# btrfs_list /local/test ACCESS-MODE SUBVOLUME <- SNAPSHOT-PARENT rw /local/test rw /local/test/sv1 rw /local/test/sv1/sv1_1 rw /local/test/sv1/sv1_1/ss <- /local/test/sv1/sv1_1 rw /local/test/sv1/sv1_2 rw /local/test/sv2
root@fex:~# btrfs_rsnapshot /local/test Create a snapshot of '/local/test' in '/local/test/.snapshot/test' Create a snapshot of '/local/test/sv1' in '/local/test/.snapshot/test/sv1' Create a snapshot of '/local/test/sv1/sv1_1' in '/local/test/.snapshot/test/sv1/sv1_1' Create a snapshot of '/local/test/sv1/sv1_1/ss' in '/local/test/.snapshot/test/sv1/sv1_1/ss' Create a snapshot of '/local/test/sv1/sv1_2' in '/local/test/.snapshot/test/sv1/sv1_2' Create a snapshot of '/local/test/sv2' in '/local/test/.snapshot/test/sv2'
root@fex:~# btrfs_list /local/test ACCESS-MODE SUBVOLUME <- SNAPSHOT-PARENT rw /local/test ro /local/test/.snapshot/test <- /local/test ro /local/test/.snapshot/test/sv1 <- /local/test/sv1 ro /local/test/.snapshot/test/sv1/sv1_1 <- /local/test/sv1/sv1_1 ro /local/test/.snapshot/test/sv1/sv1_1/ss <- /local/test/sv1/sv1_1/ss ro /local/test/.snapshot/test/sv1/sv1_2 <- /local/test/sv1/sv1_2 ro /local/test/.snapshot/test/sv2 <- /local/test/sv2 rw /local/test/sv1 rw /local/test/sv1/sv1_1 rw /local/test/sv1/sv1_1/ss <- /local/test/sv1/sv1_1 rw /local/test/sv1/sv1_2 rw /local/test/sv2
root@fex:~# btrfs_rsnapshot -d /local/test Delete subvolume (no-commit): '/local/test/.snapshot/test/sv2' Delete subvolume (no-commit): '/local/test/.snapshot/test/sv1/sv1_2' Delete subvolume (no-commit): '/local/test/.snapshot/test/sv1/sv1_1/ss' Delete subvolume (no-commit): '/local/test/.snapshot/test/sv1/sv1_1' Delete subvolume (no-commit): '/local/test/.snapshot/test/sv1' Delete subvolume (no-commit): '/local/test/.snapshot/test'
root@fex:~# btrfs_list /local/test ACCESS-MODE SUBVOLUME <- SNAPSHOT-PARENT rw /local/test rw /local/test/sv1 rw /local/test/sv1/sv1_1 rw /local/test/sv1/sv1_1/ss <- /local/test/sv1/sv1_1 rw /local/test/sv1/sv1_2 rw /local/test/sv2
Signature · 6 lines
Ullrich Horlacher Server und Virtualisierung Rechenzentrum TIK Universitaet Stuttgart E-Mail: [email protected] Allmandring 30a Tel: ++49-711-68565868 70569 Stuttgart (Germany) WWW: https://www.tik.uni-stuttgart.de/ REF:<[email protected]>
Quoted 11 lines · Ulli Horlacher
I am using fsfreeze when running a backup to ensure a consistent filesystem.
While the backup is running writes to the filesystem are suspended and the whole system is unresponsive, e.g. logins are not possible.
On certain errors the unfreeze will not happen and the system is locked forever.Using snapshots seems a better idea for backups :-)
But snapshots do not include subvolumes.
Omar Sandoval had patches to add recursive snapshotting to btrfs-progs, but they never made it in, for reasons I can't remember.
Quoted 33 lines
For example the / filesystem has the subvolumes:
/home/home/tux/test/var/spoolWhen I run the command:
btrfs subvolume snapshot / /.snapshot/_
the snapshot will contain only the root subvolume.
I have to manually add:
rmdir /.snapshot/_/home
btrfs subvolume snapshot /home /.snapshot/_/home
rmdir /.snapshot/_/home/tux/test
btrfs subvolume snapshot/home/tux/test/.snapshot/_/home/tux/test rmdir /.snapshot/_/var/spool
btrfs subvolume snapshot/var/spool/.snapshot/_/var/spoolThen run the backup on /.snapshot/_ und afterwards:
btrfs subvolume del /.snapshot/_/var/spool
btrfs subvolume del /.snapshot/_/home/tux/test
btrfs subvolume del /.snapshot/_/home
btrfs subvolume del /.snapshot/_But this will work only for this special example!
And I have hundreds of systems to backup with different filesystem layout!Is there a best practise "Using snapshots for making backup"? I need automatic detecting, creating and removing of nested snapshots.