Skip to content

RFC btrfs: simple quotas

Boris Burkov linux-btrfs 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest9 patches 13 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9btrfs: simple quotas modefs/btrfs/fs.h fs/btrfs/ioctl.c fs/btrfs/qgroup.c +3 more+69 -211 reply
2/9btrfs: new function for recording simple quota usagefs/btrfs/qgroup.c fs/btrfs/qgroup.h+94 -10 replies
3/9btrfs: track original extent subvol in a new inline reffs/btrfs/accessors.h fs/btrfs/backref.c fs/btrfs/extent-tree.c +4 more+70 -117 replies
4/9btrfs: track metadata owning root in delayed refsfs/btrfs/delayed-ref.c fs/btrfs/delayed-ref.h fs/btrfs/extent-tree.c +1 more+29 -130 replies
5/9btrfs: record simple quota deltasfs/btrfs/delayed-ref.c fs/btrfs/delayed-ref.h fs/btrfs/extent-tree.c +3 more+106 -120 replies
6/9btrfs: auto hierarchy for simple qgroups of nested subvolsfs/btrfs/ioctl.c fs/btrfs/qgroup.c fs/btrfs/qgroup.h +1 more+48 -83 replies
7/9btrfs: check generation when recording simple quota deltafs/btrfs/accessors.h fs/btrfs/extent-tree.c fs/btrfs/fs.h +3 more+31 -40 replies
8/9btrfs: expose the qgroup mode via sysfsfs/btrfs/sysfs.c+26 -00 replies
9/9btrfs: free qgroup rsv on io failurefs/btrfs/ordered-data.c+5 -10 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH RFC 0/9] btrfs: simple quotas

btrfs quota groups (qgroups) are a compelling feature of btrfs that allow flexible control for limiting subvolume data and metadata usage. However, due to btrfs's high level decision to tradeoff snapshot performance against ref-counting performance, qgroups suffer from non-trivial performance issues that make them unattractive in certain workloads. Particularly, frequent backref walking during writes and during commits can make operations increasingly expensive as the number of snapshots scales up. For that reason, we have never been able to commit to using qgroups in production at Meta, despite significant interest from people running container workloads, where we would benefit from protecting the rest of the host from a buggy application in a container running away with disk usage.

This patch series introduces a simplified version of qgroups called simple quotas (squotas) which never computes global reference counts for extents, and thus has similar performance characteristics to normal, quotas disabled, btrfs. The "trick" is that in simple quotas mode, we account all extents permanently to the subvolume in which they were originally created. That allows us to make all accounting 1:1 with extent item lifetime, removing the need to walk backrefs. However, this sacrifices the ability to compute shared vs. exclusive usage. It also results in counter-intuitive, though still predictable and simple, accounting in the cases where an original extent is removed while a shared copy still exists. Qgroups is able to detect that case and count the remaining copy as an exclusive owner, while squotas is not. As a result, squotas works best when the original extent is immutable and outlives any clones.

In order to track the original creating subvolume of a data extent in the face of reflinks, it is necessary to add additional accounting to the extent item. To save space, this is done with a new inline ref item. However, the downside of this approach is that it makes enabling squota an incompat change, denoted by the new incompat bit SIMPLE_QUOTA. When this bit is set and quotas are enabled, new extent items get the extra accounting, and freed extent items check for the accounting to find their creating subvolume.

Squotas reuses the api of qgroups. The only difference is that when you enable quotas via btrfs quota enable, you pass the --simple flag. Squotas will always report exclusive == shared for each qgroup.

This is still a preliminary RFC patch series, so not all the ducks are fully in a row. In particular, some userspace parts are missing, like meaningful integration with fsck, which will drive further testing.

My current branches for btrfs-progs and fstests do contain some (sloppy) minimal support needed to run and test the feature:
btrfs-progs: https://github.com/boryas/btrfs-progs/tree/squota-progs
fstests: https://github.com/boryas/fstests/tree/squota-test

Current testing methodology:
- New fstest (btrfs/400 in the squota-test branch)
- Run all fstests with squota enabled at mkfs time. Not all tests are passing in this regime, though this is actually true of qgroups as well. Most of the issues have to do with leaking reserved space in less commonly tested cases like I/O failures. My intent is to get this test suite fully passing.
- Run all fstests without squota enabled at mkfs time

Basic performance test:
In this test, I ran a workload which generated K files in a subvolume, then took L snapshots of that subvolume, then unshared each file in each subvolume. The measurement is just total walltime. K is the row index and L the column index, so in these tables, we vary between 1 and 100 files and 1 and 10000 snapshots. The "n" table is no quotas, the "q" table is qgroups and the "s" table is squotas. As you can see, "n" and "s" are quite similar, while "q" falls of a cliff as the number of snapshots increases. More sophisticated and realistic performance testing that doesn't abuse such an insane number of snapshots is still to come.

n

        1       10      100     1000    10000
1       0.18    0.24    1.58    16.49   211.34
10      0.28    0.43    2.80    29.74   324.70
100     0.55    0.99    6.57    65.13   717.51

q

        1       10      100     1000    10000
1       2.19    0.35    2.32    25.78   756.62
10      0.34    0.48    3.24    68.72   3731.73
100     0.64    0.80    7.63    215.54  14170.73

s

        1       10      100     1000    10000
1       2.19    0.32    1.83    19.19   231.75
10      0.31    0.43    2.86    28.86   351.42
100     0.70    0.90    6.75    67.89   742.93

Boris Burkov (9):
btrfs: simple quotas mode
btrfs: new function for recording simple quota usage
btrfs: track original extent subvol in a new inline ref
btrfs: track metadata owning root in delayed refs
btrfs: record simple quota deltas
btrfs: auto hierarchy for simple qgroups of nested subvols
btrfs: check generation when recording simple quota delta
btrfs: expose the qgroup mode via sysfs
btrfs: free qgroup rsv on io failure

 fs/btrfs/accessors.h            |   6 +
 fs/btrfs/backref.c              |   3 +
 fs/btrfs/delayed-ref.c          |  13 +-
 fs/btrfs/delayed-ref.h          |  28 ++++-
 fs/btrfs/extent-tree.c          | 143 +++++++++++++++++----
 fs/btrfs/fs.h                   |   7 +-
 fs/btrfs/ioctl.c                |   4 +-
 fs/btrfs/ordered-data.c         |   6 +-
 fs/btrfs/print-tree.c           |  12 ++
 fs/btrfs/qgroup.c               | 216 +++++++++++++++++++++++++++++---
 fs/btrfs/qgroup.h               |  29 ++++-
 fs/btrfs/ref-verify.c           |   3 +
 fs/btrfs/relocation.c           |  11 +-
 fs/btrfs/sysfs.c                |  26 ++++
 fs/btrfs/transaction.c          |  11 +-
 fs/btrfs/tree-checker.c         |   3 +
 include/uapi/linux/btrfs.h      |   1 +
 include/uapi/linux/btrfs_tree.h |  13 ++
 18 files changed, 471 insertions(+), 64 deletions(-)
Signature · 1 line
2.40.0