rsync -ax and subvolumes
I run into the problem that "rsync -ax" sees btrfs subvolumes as "other filesystems" and ignores them.
framstag@tux:~: rsync -h | grep one-file-system
-x, --one-file-system don't cross filesystem boundaries
How could I tell rsync to include btrfs subvolumes, but exclude other mounted filesystems?
Do I have to write a rsync wrapper script and find all subvolumes to have them included? Nasty task...
Signature · 6 lines
Ullrich Horlacher Server und Virtualisierung Rechenzentrum TIK Universitaet Stuttgart E-Mail: [email protected] Allmandring 30a Tel: ++49-711-68565868 70569 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ REF:<[email protected]>
Hi!
Am Do., 10. Okt. 2019 um 19:32 Uhr schrieb Ulli Horlacher <[email protected]>:
Quoted 3 lines
I run into the problem that "rsync -ax" sees btrfs subvolumes as "other filesystems" and ignores them.
I worked around it by mounting the btrfs-pool at a special directory:
mount -o subvolid=0 /dev/disk/by-label/rootfs /mnt/btrfs-pool
Then run rsync from /mnt/btrfs-pool without the "-x" flag. It won't see mount points with foreign filesystems anyway below that mount point.
Quoted 2 lines
How could I tell rsync to include btrfs subvolumes, but exclude other mounted filesystems?
You can't, btrfs subvolumes are distinctive filesystems when viewed from the API: they have their own set of inodes and thus their own dev-id (and when that changes between two directories, you're crossing a filesystem boundary).
Quoted 2 lines
Do I have to write a rsync wrapper script and find all subvolumes to have them included? Nasty task...
The trick above should do, as long as you don't mind seeing another view of your btrfs-pool opposed to your root mount view. It has the benefit of seeing all your stuff. And it has the benefit of no longer seeing API mounts there (like proc, sys, dev) and you can should rsync without any more excludes needed.
Actually, you could also just bind-mount into /mnt/btrfs, bind-mounts won't inherit other mounts but will still see pure subvolumes.
HTH
Kai
Quoted 6 lines · Kai Krakow
Quoted 2 lines
I run into the problem that "rsync -ax" sees btrfs subvolumes as "other filesystems" and ignores them.
I worked around it by mounting the btrfs-pool at a special directory:
mount -o subvolid=0
/dev/disk/by-label/rootfs/mnt/btrfs-pool
This is only possible by root, but not by regular users. Yes, there are true multi-user systems still out there :-)
Quoted 2 lines
Actually, you could also just bind-mount into
/mnt/btrfs, bind-mounts won't inherit other mounts but will still see pure subvolumes.
Again, only possible by root.
Signature · 6 lines
Ullrich Horlacher Server und Virtualisierung Rechenzentrum TIK Universitaet Stuttgart E-Mail: [email protected] Allmandring 30a Tel: ++49-711-68565868 70569 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ REF:<CAMthOuMV7MgB4b87RsijYr9e0UsjMUDNk+QRXeauFdb3cZcTjw@mail.gmail.com>
Quoted 2 lines · Kai Krakow
Actually, you could also just bind-mount into
/mnt/btrfs, bind-mounts won't inherit other mounts but will still see pure subvolumes.
Next problem:
root@ptm1:~# sysinfo System: Linux ptm1 4.4.180-94.100-default x86_64 Distribution: SUSE Linux Enterprise Server 12 SP3
root@ptm1:~# mount | grep sda /dev/sda2 on / type btrfs (rw,relatime,space_cache,subvolid=453,subvol=/@/.snapshots/128/snapshot) /dev/sda2 on /.snapshots type btrfs (rw,relatime,space_cache,subvolid=270,subvol=/@/.snapshots) /dev/sda2 on /opt type btrfs (rw,relatime,space_cache,subvolid=259,subvol=/@/opt) /dev/sda2 on /var/log type btrfs (rw,relatime,space_cache,subvolid=264,subvol=/@/var/log) /dev/sda2 on /srv type btrfs (rw,relatime,space_cache,subvolid=260,subvol=/@/srv) /dev/sda2 on /var/tmp type btrfs (rw,relatime,space_cache,subvolid=267,subvol=/@/var/tmp) /dev/sda2 on /var/spool type btrfs (rw,relatime,space_cache,subvolid=266,subvol=/@/var/spool) /dev/sda2 on /usr/local type btrfs (rw,relatime,space_cache,subvolid=262,subvol=/@/usr/local) /dev/sda2 on /var/opt type btrfs (rw,relatime,space_cache,subvolid=265,subvol=/@/var/opt) /dev/sda2 on /home type btrfs (rw,relatime,space_cache,subvolid=258,subvol=/@/home) /dev/sda2 on /var/lib/machines type btrfs (rw,relatime,space_cache,subvolid=1235,subvol=/@/var/lib/machines) /dev/sda2 on /tmp type btrfs (rw,relatime,space_cache,subvolid=261,subvol=/@/tmp) /dev/sda2 on /var/crash type btrfs (rw,relatime,space_cache,subvolid=263,subvol=/@/var/crash)
root@ptm1:~# mount -o bind / /mnt/tmp
root@ptm1:~# find /opt | wc -l 564
root@ptm1:~# find /mnt/tmp/opt | wc -l 1
I want to copy everything from /dev/sda2 with rsync, but not from other devices.
rsynv -avH / /mnt/usb # copies also /proc /dev etc rsynv -avHx / /mnt/usb # copies ONLY / but not /opt /var/log etc
And yes, it must be rsync, because I run this command often, for syncing.
Signature · 6 lines
Ullrich Horlacher Server und Virtualisierung Rechenzentrum TIK Universitaet Stuttgart E-Mail: [email protected] Allmandring 30a Tel: ++49-711-68565868 70569 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/ REF:<CAMthOuMV7MgB4b87RsijYr9e0UsjMUDNk+QRXeauFdb3cZcTjw@mail.gmail.com>
Quoted 11 lines · Ulli Horlacher
Quoted 6 lines · Kai Krakow
Quoted 2 lines
I run into the problem that "rsync -ax" sees btrfs subvolumes as "other filesystems" and ignores them.
I worked around it by mounting the btrfs-pool at a special directory:
mount -o subvolid=0
/dev/disk/by-label/rootfs/mnt/btrfs-poolThis is only possible by root, but not by regular users. Yes, there are true multi-user systems still out there :-)
And that is what sudo or capabilities are for.
sudo will even let you get as specific as command line arguments, so you can specify an exact mount command that can be run (including ensuring that it's read-only) and an exact unmount command that can be run.
If you want to go the capabilities route, you'll need CAP_MOUNT. This is a lot riskier than using sudo though.
That said, if you really want _everything_, you're going to need to either be root anyway, or have the CAP_DAC_READ_SEARCH (or CAP_DAC_OVERRIDE) capability, because there are some files you just won't be able to read otherwise (at minimum the contents of /root and any properly secured authentication logs in /var/log, as well as possibly other things under /var and possibly some things under /etc).
Quoted 7 lines
Quoted 4 lines
Actually, you could also just bind-mount into
/mnt/btrfs, bind-mounts won't inherit other mounts but will still see pure subvolumes.Again, only possible by root.
11.10.2019 9:15, Ulli Horlacher пишет:
Quoted 26 lines · Kai Krakow
Quoted 3 lines
Actually, you could also just bind-mount into
/mnt/btrfs, bind-mounts won't inherit other mounts but will still see pure subvolumes.Next problem:
root@ptm1:~# sysinfo System: Linux ptm1 4.4.180-94.100-default x86_64 Distribution: SUSE Linux Enterprise Server 12 SP3root@ptm1:~# mount | grep sda /dev/sda2 on / type btrfs (rw,relatime,space_cache,subvolid=453,subvol=/@/.snapshots/128/snapshot) /dev/sda2 on /.snapshots type btrfs (rw,relatime,space_cache,subvolid=270,subvol=/@/.snapshots) /dev/sda2 on /opt type btrfs (rw,relatime,space_cache,subvolid=259,subvol=/@/opt) /dev/sda2 on /var/log type btrfs (rw,relatime,space_cache,subvolid=264,subvol=/@/var/log) /dev/sda2 on /srv type btrfs (rw,relatime,space_cache,subvolid=260,subvol=/@/srv) /dev/sda2 on /var/tmp type btrfs (rw,relatime,space_cache,subvolid=267,subvol=/@/var/tmp) /dev/sda2 on /var/spool type btrfs (rw,relatime,space_cache,subvolid=266,subvol=/@/var/spool) /dev/sda2 on /usr/local type btrfs (rw,relatime,space_cache,subvolid=262,subvol=/@/usr/local) /dev/sda2 on /var/opt type btrfs (rw,relatime,space_cache,subvolid=265,subvol=/@/var/opt) /dev/sda2 on /home type btrfs (rw,relatime,space_cache,subvolid=258,subvol=/@/home) /dev/sda2 on /var/lib/machines type btrfs (rw,relatime,space_cache,subvolid=1235,subvol=/@/var/lib/machines) /dev/sda2 on /tmp type btrfs (rw,relatime,space_cache,subvolid=261,subvol=/@/tmp) /dev/sda2 on /var/crash type btrfs (rw,relatime,space_cache,subvolid=263,subvol=/@/var/crash)
Quoted 8 lines
root@ptm1:~# mount -o bind / /mnt/tmproot@ptm1:~# find /opt | wc -l 564root@ptm1:~# find /mnt/tmp/opt | wc -l 1
Of course. /opt is mount point. You will need to replicate the same mount hierarchy under /mnt/tmp.
Am Fr., 11. Okt. 2019 um 08:17 Uhr schrieb Ulli Horlacher <[email protected]>:
Quoted 7 lines · Kai Krakow
Quoted 3 lines
Actually, you could also just bind-mount into
/mnt/btrfs, bind-mounts won't inherit other mounts but will still see pure subvolumes.Next problem:
The other problem first: As already pointed out, there's "sudo", and for the job you're trying to do, you probably need root permissions anyway. OTOH, I don't see the problem why you don't just statically mount subvolid=0 into /mnt/btrfs-pool permanently. Either that way, or you could use automount. It's easy with systemd:
$ grep btrfs-pool /etc/fstab LABEL=system /mnt/btrfs-pool btrfs noauto,compress=zstd,subvolid=0,x-systemd.automount
You could create an intermediate directory in /mnt to allow one specific user only:
/mnt/for-user/btrfs-pool
Then adjust chown/chmod on /mnt/for-user to limit it to just one user (or group if you want), that is: remove rwx from the others.
Apparently, with btrfs, the "ro" flag afaik is not per subvolume mount. But I'm using systemd to limit access to read-only for the service doing the backup:
$ egrep '^(Read|Privat|Protect)' /etc/systemd/system/system-backup.service ProtectSystem=strict PrivateTmp=yes ReadWriteDirectories=/root ReadWriteDirectories=/srv/laniakea/backups/jupiter.sol.local.borg ReadOnlyDirectories=/mnt/btrfs-pool
I'm using borg instead of rsync (because it is much faster and storage is used much more efficient), so it needs write access to /root. If you're using rsync, you don't need that. I used rsync a few years back for the same job, it was just too slow and resource intensive.
Now let's get to your job:
Quoted 6 lines
root@ptm1:~# mount | grep sda /dev/sda2 on / type btrfs (rw,relatime,space_cache,subvolid=453,subvol=/@/.snapshots/128/snapshot) /dev/sda2 on /.snapshots type btrfs (rw,relatime,space_cache,subvolid=270,subvol=/@/.snapshots) /dev/sda2 on /opt type btrfs (rw,relatime,space_cache,subvolid=259,subvol=/@/opt) /dev/sda2 on /var/log type btrfs (rw,relatime,space_cache,subvolid=264,subvol=/@/var/log) /dev/sda2 on /srv type btrfs (rw,relatime,space_cache,subvolid=260,subvol=/@/srv)
Quoted 8 lines
/dev/sda2 on /var/tmp type btrfs (rw,relatime,space_cache,subvolid=267,subvol=/@/var/tmp) /dev/sda2 on /var/spool type btrfs (rw,relatime,space_cache,subvolid=266,subvol=/@/var/spool) /dev/sda2 on /usr/local type btrfs (rw,relatime,space_cache,subvolid=262,subvol=/@/usr/local) /dev/sda2 on /var/opt type btrfs (rw,relatime,space_cache,subvolid=265,subvol=/@/var/opt) /dev/sda2 on /home type btrfs (rw,relatime,space_cache,subvolid=258,subvol=/@/home) /dev/sda2 on /var/lib/machines type btrfs (rw,relatime,space_cache,subvolid=1235,subvol=/@/var/lib/machines) /dev/sda2 on /tmp type btrfs (rw,relatime,space_cache,subvolid=261,subvol=/@/tmp) /dev/sda2 on /var/crash type btrfs (rw,relatime,space_cache,subvolid=263,subvol=/@/var/crash)
Ah, that funny OpenSuSE structure...
Quoted 1 line
root@ptm1:~# mount -o bind / /mnt/tmp
Bind mounting the "/" mount namespace will exclude all sub mounts, and OpenSuSE uses a lot. It would work if subvols would be embedded into the normal FS hierarchy.
Quoted 5 lines
root@ptm1:~# find /opt | wc -l 564root@ptm1:~# find /mnt/tmp/opt | wc -l 1
Expected given your mtab.
Quoted 4 lines
I want to copy everything from
/dev/sda2with rsync, but not from other devices.rsynv -avH /
/mnt/usb# copies also /proc /dev etc rsynv -avHx //mnt/usb# copies ONLY / but not /opt/var/logetc
Try this, pretending you mounted subvolid=0 to /mnt/btrfs-pool:
# NOTE: do not use "-x" here:
rsync -av $(findmnt -nr /dev/sda2 | perl -ne'/subvol=([^,]+)/ && \
print "/mnt/btrfs-pool$1"') /srv/backups/test-backup/
Or:
# NOTE: you should use "-x" here:
rsync -avx $(findmnt -nr /dev/sda2 | awk '{ print $1 }' \
/srv/backups/test-backup/
The first example may explode if your mountpoints contain funny characters (like "," or spaces). I didn't really test, I did just a quick rsync run.
Both examples may have surprising consequences if source and destination is the same device. You have been warned. But that's not really different from how you used it.
Quoted 1 line
And yes, it must be rsync, because I run this command often, for syncing.
That's not exactly explaining your use-case: I don't see the correlation between "often" and "rsync", but I guess you mean the process should be fast because you want to run the command often. But then again, there's faster software than rsync.
HTH
Kai