Skip to content

btrfs: qgroup: address the performance penalty for subvolume dropping

Qu Wenruo linux-btrfs 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest5 patches 2 repliesIn review
  2. v35 patches 4 repliesSuperseded
  3. v25 patches 0 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v15 patches5 repliesSuperseded

Changes in v4 from the cover letter

- Fix a kmalloc() usage, which can lead to kobject warnings Since kobject_init_and_add() relies on certain values of a member, kmalloc() can leave kobj->state_initialized to be true, and cause crash at qgroups_kobj initialization time.

- Add a script in the cover letter to verify the behavior Will later be submitted as a test case.

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5btrfs: sysfs: introduce qgroup global attribute groupsfs/btrfs/sysfs.c+69 -40 replies
2/5btrfs: introduce BTRFS_QGROUP_STATUS_FLAGS_MASK for later expansionfs/btrfs/qgroup.c include/uapi/linux/btrfs_tree.h+8 -20 replies
3/5btrfs: introduce BTRFS_QGROUP_RUNTIME_FLAG_CANCEL_RESCANfs/btrfs/qgroup.c fs/btrfs/qgroup.h+31 -171 reply
4/5btrfs: introduce BTRFS_QGROUP_RUNTIME_FLAG_NO_ACCOUNTING to skip qgroup accountingfs/btrfs/qgroup.c fs/btrfs/qgroup.h+15 -40 replies
5/5btrfs: skip subtree scan if it's too high to avoid low stall in btrfs_commit_transaction()fs/btrfs/ctree.h fs/btrfs/disk-io.c fs/btrfs/qgroup.c +1 more+59 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.

Read the v4 thread

Cover letter [PATCH v4 0/5] btrfs: qgroup: address the performance penalty for subvolume dropping

[CHANGELOG]
v4:
- Fix a kmalloc() usage, which can lead to kobject warnings Since kobject_init_and_add() relies on certain values of a member, kmalloc() can leave kobj->state_initialized to be true, and cause crash at qgroups_kobj initialization time.

- Add a script in the cover letter to verify the behavior Will later be submitted as a test case.

v3:
- Rebased to latest misc-next

v2:
- Split /sys/fs/btrfs/<uuid>/qgroups/qgroup_flags into two entries

- Update the cover letter to explain the drop_subtree_threshold better

Btrfs qgroup has a long history of bringing huge performance penalty, from subvolume dropping to balance.

Although we solved the problem for balance, but the subvolume dropping problem is still unresolved, as we really need to do all the costly backref for all the involved subtrees, or qgroup numbers will be inconsistent.

But the performance penalty is sometimes too big, so big that it's better just to disable qgroup, do the drop, then do the rescan.

This patchset will address the problem by introducing a user configurable sysfs interface, to allow certain high subtree dropping to mark qgroup inconsistent, and skip the whole accounting.

The following things are needed for this objective:

- New qgroups attributes

Instead of plain qgroup kobjects, we need extra attributes like drop_subtree_threshold.

This patchset will introduce two new attributes to the existing qgroups kobject:
* qgroups_flags
To indicate the qgroup status flags like ON, RESCAN, INCONSISTENT.

* drop_subtree_threshold
To show the subtree dropping level threshold.
The default value is BTRFS_MAX_LEVEL (8), which means all subtree dropping will go through the qgroup accounting, while costly it will try to keep qgroup numbers as consistent as possible.

Users can specify values like 3, meaning any subtree which is at level 3 or higher will mark qgroup inconsistent and skip all the costly accounting.

NOTE: if a snapshot is create with tree root level 3, dropping the snapshot with drop_subtree_threshold 3 will not mark the qgroup inconsistent.
Since the level threshold is for shared subtree node, not the snapshot root node.

In the case of newly created snapshot, only its (root level - 1) tree blocks are shared subtrees.

This only affects subvolume dropping.

- Skip qgroup accounting when the numbers are already inconsistent

But still keeps the qgroup relationship correct, thus users can keep its qgroup organization while do the rescan later.

All the behavior can be verified using the following simple script:

btrfs dev scan -u
mkfs.btrfs -U $fsid -f -n 4k $dev
mount $dev $mnt
btrfs subv create $mnt/subv

# Bump the tree level to 2
for (( i = 0; i < 8192; i++)); do
xfs_io -f -c "pwrite 0 2k" $mnt/subv/inline1_$i > /dev/null xfs_io -f -c "pwrite 0 2k" $mnt/subv/inline2_$i > /dev/null xfs_io -f -c "pwrite 0 4k" $mnt/subv/regular_$i > /dev/null
done
sync
btrfs subv snapshot $mnt/subv $mnt/snap
btrfs quota enable $mnt
btrfs quota rescan -w $mnt
echo 2 > /sys/fs/btrfs/<fsid>/qgroups/drop_subtree_threshold btrfs subv delete $mnt/snap
btrfs subv sync $mnt

After above workload, btrfs qgroup show should give the following warning:

WARNING: qgroup data inconsistent, rescan recommended

This sysfs interface needs user space tools to monitor and set the values for each btrfs.
And it's also user space daemon's responsibility to save the drop_subtree_threshold values.

As introducing a new on-disk format just for qgroup is a little overkilled to an optional feature to me.

Currently the target user space tool is snapper, which by default utilizes qgroups for its space-aware snapshots reclaim mechanism.

Qu Wenruo (5):
btrfs: sysfs: introduce qgroup global attribute groups
btrfs: introduce BTRFS_QGROUP_STATUS_FLAGS_MASK for later expansion
btrfs: introduce BTRFS_QGROUP_RUNTIME_FLAG_CANCEL_RESCAN
btrfs: introduce BTRFS_QGROUP_RUNTIME_FLAG_NO_ACCOUNTING to skip
qgroup accounting
btrfs: skip subtree scan if it's too high to avoid low stall in
btrfs_commit_transaction()

 fs/btrfs/ctree.h                |   1 +
 fs/btrfs/disk-io.c              |   1 +
 fs/btrfs/qgroup.c               |  84 ++++++++++++++++++------
 fs/btrfs/qgroup.h               |   3 +
 fs/btrfs/sysfs.c                | 112 ++++++++++++++++++++++++++++++--
 include/uapi/linux/btrfs_tree.h |   4 ++
 6 files changed, 180 insertions(+), 25 deletions(-)
Signature · 1 line
2.37.2