Announcing the btrfs-list tool
Hi!
I've been using btrfs as my main NAS filesystem for a bunch of years now, and I always found that managing dozens of subvolumes having dozens of snapshots each was not really straightforward, especially when trying to get an overview of "what's taking space on my filesystem". I was missing the niceness of the "zfs list" command, which does that quite well.
So I've started to write a wrapper around btrfs qgroup show and btrfs fi df (then btrfs fi usage when it got out), to implement this broad overview of my filesystem I was missing so much. Adding some features along the way, I thought that after all it might also be useful to others than me!
As stdout is better than a thousand words, here's for example how it can show you a complex hierarchy of intricated snapshots:
root@nas:~# btrfs-list /mnt/a
NAME TYPE REFER USED MOUNTPOINT
4582dacf fs - 0.00 (1002.88M free)
[main] mainvol 16.00k 16.00k
/mnt/a
sub1 subvol 16.00k 16.00k
sub1/.snap1 snap 16.00k 16.00k
sub1/.snap2 snap 16.00k 16.00k
sub1/subsub1 snap 16.00k 16.00k
sub1/subsub1/.snap3 snap 16.00k 16.00k
sub1/subsub1/subsubsub1 snap 16.00k 16.00k
sub1/subsub1/subsubsubsub1 snap 16.00k 16.00kOr how it can help you find big snapshots that you might want to get rid of if you're running out of space:
root@nas:~# btrfs-list --snap-min-used 4G --snap-only /tank
NAME TYPE REFER USED MOUNTPOINT
backups/.snaps/skyline/20130213_231649_lastskyline rosnap
22.52G 19.58G
backups/.snaps/box/20171231_221207_monthly.12 rosnap
88.73G 4.96G
backups/.snaps/box/20180130_221209_monthly.11 rosnap
91.25G 4.90G
backups/.snaps/box/20180307_154215_monthly.10 rosnap
96.28G 10.72G
backups/.snaps/box/20190120_193004_weekly.3 rosnap
56.45G 4.25G
backups/.snaps/nasroot/20180122_091325_monthly.12 rosnap
34.65G 10.79G
backups/.snaps/nasroot/20180221_092311_monthly.11 rosnap
31.96G 4.98G
backups/.snaps/nasroot/20180323_092734_monthly.10 rosnap
33.69G 7.05G
backups/.snaps/nasroot/20180820_205559_monthly.5 rosnap
31.74G 5.37G
.syncthing-bkp rosnap
40.48G 8.15GWant to give it a try? Here you are:
https://github.com/speed47/btrfs-list
There are a couple snapshots there too, to give you a rough idea of how it looks like (yes, it also has nice colors)
Happy btrfs weekend!
Signature · 1 line
Stéphane.
02.03.2019 22:06, Stéphane Lesimple пишет:
Quoted 17 lines
Hi!
I've been using btrfs as my main NAS filesystem for a bunch of years now, and I always found that managing dozens of subvolumes having dozens of snapshots each was not really straightforward, especially when trying to get an overview of "what's taking space on my filesystem". I was missing the niceness of the "zfs list" command, which does that quite well.
So I've started to write a wrapper around btrfs qgroup show and btrfs fi df (then btrfs fi usage when it got out), to implement this broad overview of my filesystem I was missing so much.
Adding some features along the way, I thought that after all it might also be useful to others than me!As stdout is better than a thousand words, here's for example how it can show you a complex hierarchy of intricated snapshots:
Quoted 15 lines
root@nas:~# btrfs-list /mnt/a NAME TYPE REFER USED MOUNTPOINT 4582dacf fs - 0.00 (1002.88M free) [main] mainvol 16.00k 16.00k /mnt/a sub1 subvol 16.00k 16.00k sub1/.snap1 snap 16.00k 16.00k sub1/.snap2 snap 16.00k 16.00k sub1/subsub1 snap 16.00k 16.00k sub1/subsub1/.snap3 snap 16.00k 16.00k sub1/subsub1/subsubsub1 snap 16.00k 16.00k sub1/subsub1/subsubsubsub1 snap 16.00k 16.00k
btrfs does not have anything resembling USED in zfs, so I'm afraid this will be confusing to anyone coming from zfs. In zfs datasets are hierarchical (including snapshots) so USED on every level is exactly the amount of space that will become available in parent dataset if this one is destroyed. This does not matter much without quota (as all datasets share one common pool of free space) but becomes quite visible as soon as you restrict dataset space consumption. In btrfs every subvolume is independent; what you show as USED is exclusive space; deleting subvolume does not mean its exclusively consumed space will be now accounted as available in another subvolume.
Showing hierarchical list with USED column creates false expectations; I'd rather call it EXCL to make its meaning clear.
That said, tool that lists qgroups with volume names is surely welcome.
Quoted 35 lines
Or how it can help you find big snapshots that you might want to get rid of if you're running out of space:
root@nas:~# btrfs-list --snap-min-used 4G --snap-only /tank NAME TYPE REFER USED MOUNTPOINT backups/.snaps/skyline/20130213_231649_lastskyline rosnap 22.52G 19.58G backups/.snaps/box/20171231_221207_monthly.12 rosnap 88.73G 4.96G backups/.snaps/box/20180130_221209_monthly.11 rosnap 91.25G 4.90G backups/.snaps/box/20180307_154215_monthly.10 rosnap 96.28G 10.72G backups/.snaps/box/20190120_193004_weekly.3 rosnap 56.45G 4.25G backups/.snaps/nasroot/20180122_091325_monthly.12 rosnap 34.65G 10.79G backups/.snaps/nasroot/20180221_092311_monthly.11 rosnap 31.96G 4.98G backups/.snaps/nasroot/20180323_092734_monthly.10 rosnap 33.69G 7.05G backups/.snaps/nasroot/20180820_205559_monthly.5 rosnap 31.74G 5.37G .syncthing-bkp rosnap 40.48G 8.15GWant to give it a try? Here you are:
https://github.com/speed47/btrfs-listThere are a couple snapshots there too, to give you a rough idea of how it looks like (yes, it also has nice colors)
Happy btrfs weekend!
Quoted 43 lines · Le 2 mars 2019 21:28:29 Andrei Borzenkov <[email protected]> a écrit :
02.03.2019 22:06, Stéphane Lesimple пишет:
Quoted 17 lines
Hi!
I've been using btrfs as my main NAS filesystem for a bunch of years now, and I always found that managing dozens of subvolumes having dozens of snapshots each was not really straightforward, especially when trying to get an overview of "what's taking space on my filesystem". I was missing the niceness of the "zfs list" command, which does that quite well.
So I've started to write a wrapper around btrfs qgroup show and btrfs fi df (then btrfs fi usage when it got out), to implement this broad overview of my filesystem I was missing so much.
Adding some features along the way, I thought that after all it might also be useful to others than me!As stdout is better than a thousand words, here's for example how it can show you a complex hierarchy of intricated snapshots:
Quoted 13 lines
root@nas:~# btrfs-list /mnt/a NAME TYPE REFER USED MOUNTPOINT 4582dacf fs - 0.00 (1002.88M free) [main] mainvol 16.00k 16.00k /mnt/a sub1 subvol 16.00k 16.00k sub1/.snap1 snap 16.00k 16.00k sub1/.snap2 snap 16.00k 16.00k sub1/subsub1 snap 16.00k 16.00k sub1/subsub1/.snap3 snap 16.00k 16.00k sub1/subsub1/subsubsub1 snap 16.00k 16.00k sub1/subsub1/subsubsubsub1 snap 16.00k 16.00kbtrfs does not have anything resembling USED in zfs, so I'm afraid this will be confusing to anyone coming from zfs. In zfs datasets are hierarchical (including snapshots) so USED on every level is exactly the amount of space that will become available in parent dataset if this one is destroyed. This does not matter much without quota (as all datasets share one common pool of free space) but becomes quite visible as soon as you restrict dataset space consumption. In btrfs every subvolume is independent; what you show as USED is exclusive space; deleting subvolume does not mean its exclusively consumed space will be now accounted as available in another subvolume.
Quoted 2 lines
Showing hierarchical list with USED column creates false expectations; I'd rather call it EXCL to make its meaning clear.
You're right. I personally just use the quota feature to track the readonly snapshots that I use as backups, hence I have only the auto-generated 0/ level qgroups on my filesystem.
As USED could be confusing in more complex setups I've replaced this terminology by EXCL as you mentioned, and adjusted the help text accordingly. Thanks!
Quoted 38 lines
That said, tool that lists qgroups with volume names is surely welcome.
Quoted 34 lines
Or how it can help you find big snapshots that you might want to get rid of if you're running out of space:
root@nas:~# btrfs-list --snap-min-used 4G --snap-only /tank NAME TYPE REFER USED MOUNTPOINT backups/.snaps/skyline/20130213_231649_lastskyline rosnap 22.52G 19.58G backups/.snaps/box/20171231_221207_monthly.12 rosnap 88.73G 4.96G backups/.snaps/box/20180130_221209_monthly.11 rosnap 91.25G 4.90G backups/.snaps/box/20180307_154215_monthly.10 rosnap 96.28G 10.72G backups/.snaps/box/20190120_193004_weekly.3 rosnap 56.45G 4.25G backups/.snaps/nasroot/20180122_091325_monthly.12 rosnap 34.65G 10.79G backups/.snaps/nasroot/20180221_092311_monthly.11 rosnap 31.96G 4.98G backups/.snaps/nasroot/20180323_092734_monthly.10 rosnap 33.69G 7.05G backups/.snaps/nasroot/20180820_205559_monthly.5 rosnap 31.74G 5.37G .syncthing-bkp rosnap 40.48G 8.15GWant to give it a try? Here you are:
https://github.com/speed47/btrfs-listThere are a couple snapshots there too, to give you a rough idea of how it looks like (yes, it also has nice colors)
Happy btrfs weekend!