btrfs: qgroup: mark qgroup inconsistent for more
cover btrfs: qgroup: mark qgroup inconsistent for more
Commit 011b46c30476 ("btrfs: skip subtree scan if it's too high to avoid low stall in btrfs_commit_transaction()") covers the most common case that can cause a long qgroup stall, where a large subtree is rescanned when dropping a snapshot.
But there are more corner cases for the same long qgroup stall:
- A qgroup rescan is queued immediately after marking inconsistent The new rescan will immediately revert the NO_ACCOUNTING flag, thus still results long qgroup stalls.
- A subvolume with no shared subtree
This can still queue a lot of workload into the current transaction.
This series covers both less-common cases.
Qu Wenruo (2):
btrfs: reject new qgroup rescan during subvolume dropping
btrfs: avoid long stall when dropping a non-shared large subvolume
fs/btrfs/disk-io.c | 4 ++++ fs/btrfs/extent-tree.c | 11 +++++++++++ fs/btrfs/qgroup.c | 29 ++++++++++++++++++++++++++++- fs/btrfs/qgroup.h | 15 +++++++++++++++ 4 files changed, 58 insertions(+), 1 deletion(-)
Signature · 1 line
2.55.0
PATCH 1/2 btrfs: reject new qgroup rescan during subvolume dropping
Commit 011b46c30476 ("btrfs: skip subtree scan if it's too high to avoid low stall in btrfs_commit_transaction()") introduced a threshold to skip huge subtree scan during subvolume dropping.
But that's not covering all cases, e.g. rescan can still be started immediately after that huge subtree skipping.
This will cause rescan to do the same accounting for that subtree anyway, still causing a long stall during transaction commit.
Introduce a new runtime qgroup flag, BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN, so that during cleanup of a subvolume, no new qgroup rescan can be initiated.
The rejection uses the same -EINPROGRESS, as if there is already a running qgroup rescan.
And since we have the extra bit, we can no longer allow plain assignment in btrfs_quota_enable(), as the plain assignment will override the REJECT_RESCAN bit.
To co-operate this new flag:
- Make btrfs_quota_enable() to only set BTRFS_QGROUP_STATUS_BIT_ON So it won't override the existing BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN bit.
- Make btrfs_quota_disable() to clear every non-rescan bit This includes:
* BTRFS_QGROUP_STATUS_BIT_ON
* BTRFS_QGROUP_STATUS_BIT_INCONSISTENT
* BTRFS_QGROUP_RUNTIME_BIT_NO_ACCOUNTING
For rescan related bits, they are either cleared by the rescan thread, or by the caller who rejects rescan.
--- fs/btrfs/disk-io.c | 4 ++++ fs/btrfs/qgroup.c | 10 +++++++++- fs/btrfs/qgroup.h | 13 +++++++++++++ 3 files changed, 26 insertions(+), 1 deletion(-)
fs/btrfs/disk-io.c +4 -0
@@ -1496,7 +1496,11 @@ static int cleaner_kthread(void *arg)btrfs_run_delayed_iputs (fs_info); +set_bit (BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN , + &fs_info->qgroup_flags); again =btrfs_clean_one_deleted_snapshot (fs_info); +clear_bit (BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN , + &fs_info->qgroup_flags);mutex_unlock (&fs_info->cleaner_mutex);/*
fs/btrfs/qgroup.c +9 -1
@@ -1105,7 +1105,7 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info, struct btrfs_qgroup_status_item);btrfs_set_qgroup_status_generation (leaf, ptr, trans->transid);btrfs_set_qgroup_status_version (leaf, ptr,BTRFS_QGROUP_STATUS_VERSION ); - fs_info->qgroup_flags = (1UL <<BTRFS_QGROUP_STATUS_BIT_ON ); +set_bit (BTRFS_QGROUP_STATUS_BIT_ON , &fs_info->qgroup_flags);if (simple) {set_bit (BTRFS_QGROUP_STATUS_BIT_SIMPLE_MODE , &fs_info->qgroup_flags);btrfs_set_fs_incompat (fs_info,SIMPLE_QUOTA );
@@ -1407,8 +1407,14 @@ int btrfs_quota_disable(struct btrfs_fs_info *fs_info)spin_lock (&fs_info->qgroup_lock); quota_root = fs_info->quota_root; fs_info->quota_root =NULL ; +/* +* Clear all on-disk and runtime bits, except RESCAN related ones, that +* are either handled by rescan thread, or the caller who rejects rescan. +*/ clear_bit (BTRFS_QGROUP_STATUS_BIT_ON , &fs_info->qgroup_flags);clear_bit (BTRFS_QGROUP_STATUS_BIT_SIMPLE_MODE , &fs_info->qgroup_flags); +clear_bit (BTRFS_QGROUP_STATUS_BIT_INCONSISTENT , &fs_info->qgroup_flags); +clear_bit (BTRFS_QGROUP_RUNTIME_BIT_NO_ACCOUNTING , &fs_info->qgroup_flags); fs_info->qgroup_drop_subtree_thres =BTRFS_QGROUP_DROP_SUBTREE_THRES_DEFAULT ;spin_unlock (&fs_info->qgroup_lock);
@@ -3997,6 +4003,8 @@ qgroup_rescan_init(struct btrfs_fs_info *fs_info, u64 progress_objectid, if (init_flags) { if (test_bit(BTRFS_QGROUP_STATUS_BIT_RESCAN, + &fs_info->qgroup_flags) || + test_bit(BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN, &fs_info->qgroup_flags)) { ret = -EINPROGRESS; } else if (!test_bit(BTRFS_QGROUP_STATUS_BIT_ON,
fs/btrfs/qgroup.h +13 -0
@@ -124,6 +124,19 @@ struct btrfs_qgroup_swapped_blocks;#define BTRFS_QGROUP_RUNTIME_BIT_CANCEL_RESCAN (BITS_PER_LONG - 1)#define BTRFS_QGROUP_RUNTIME_BIT_NO_ACCOUNTING (BITS_PER_LONG - 2) +/* +* No new rescan allowed when set. +* +* During huge subtree dropping, qgroup will be marked inconsistent, +* and skip all future accounting to avoid long stall. +* But an immediate rescan will re-enable qgroup and still stall +* the system. +* +* This bit is to avoid such rescan during the duration of a subvolume +* dropping. +*/ +#define BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN (BITS_PER_LONG - 3) +#define BTRFS_QGROUP_DROP_SUBTREE_THRES_DEFAULT (3)/*
Signature · 1 line
2.55.0
PATCH 2/2 btrfs: avoid long stall when dropping a non-shared large subvolume
In reply to Qu WenruoCommit 011b46c30476 ("btrfs: skip subtree scan if it's too high to avoid low stall in btrfs_commit_transaction()") introduced a mechanism to skip large subtree during snapshot dropping.
But even for a subvolume without any shared tree blocks, we can still queue quite a lot of qgroup records into one transaction, and cause a long qgroup related stall.
So also add a check against the subvolume root level, to determine if we need to mark qgroup inconsistent.
--- fs/btrfs/extent-tree.c | 11 +++++++++++ fs/btrfs/qgroup.c | 19 +++++++++++++++++++ fs/btrfs/qgroup.h | 2 ++ 3 files changed, 32 insertions(+)
fs/btrfs/extent-tree.c +11 -0
@@ -6315,6 +6315,17 @@ int btrfs_drop_snapshot(struct btrfs_root *root, bool update_ref, bool for_relocset_bit (BTRFS_ROOT_DELETING , &root->state); unfinished_drop =test_bit (BTRFS_ROOT_UNFINISHED_DROP , &root->state); +/* +* For subvolume dropping, check if the subvolume is large enough +* so that we need to mark qgroup inconsistent to avoid long qgroup +* stall. +* +* Even for a subvolume without any snapshot, there can still be +* a lot of qgroup records queued into one transaction. +*/ +if (!for_reloc) +btrfs_qgroup_check_tree_drop (fs_info, rootid, +btrfs_header_level (root->node));if (btrfs_disk_key_objectid (&root_item->drop_progress) ==0 ) { level =btrfs_header_level (root->node); path->nodes[level] =btrfs_lock_root_node (root);
fs/btrfs/qgroup.c +19 -0
@@ -2751,6 +2751,25 @@ int btrfs_qgroup_trace_subtree(struct btrfs_trans_handle *trans,return 0 ; } +void btrfs_qgroup_check_tree_drop (struct btrfs_fs_info *fs_info, +u64 rootid,u8 level) +{ +u8 drop_subtree_thres; + +if (btrfs_qgroup_mode (fs_info) !=BTRFS_QGROUP_MODE_FULL ) +return ; + +if (!btrfs_is_fstree (rootid)) +return ; + +spin_lock (&fs_info->qgroup_lock); + drop_subtree_thres = fs_info->qgroup_drop_subtree_thres; +spin_unlock (&fs_info->qgroup_lock); + +if (level >= drop_subtree_thres) +qgroup_mark_inconsistent (fs_info,"subtree level reached threshold" ); +} +static void qgroup_iterator_nested_add (struct list_head *head,struct btrfs_qgroup *qgroup) {if (!list_empty (&qgroup->nested_iterator))
fs/btrfs/qgroup.h +2 -0
@@ -378,6 +378,8 @@ int btrfs_qgroup_trace_leaf_items(struct btrfs_trans_handle *trans,int btrfs_qgroup_trace_subtree (struct btrfs_trans_handle *trans,struct extent_buffer *root_eb,u64 root_gen,int root_level); +void btrfs_qgroup_check_tree_drop (struct btrfs_fs_info *fs_info, +u64 rootid,u8 level);int btrfs_qgroup_account_extent (struct btrfs_trans_handle *trans,u64 bytenr,u64 num_bytes,struct ulist *old_roots,struct ulist *new_roots);
Signature · 1 line
2.55.0
Quoted 7 lines · Qu Wenruo
Commit
011b46c30476("btrfs: skip subtree scan if it's too high to avoid low stall inbtrfs_commit_transaction()") covers the most common case that can cause a long qgroup stall, where a large subtree is rescanned when dropping a snapshot.But there are more corner cases for the same long qgroup stall:
Quoted 8 lines
- A qgroup rescan is queued immediately after marking inconsistent The new rescan will immediately revert the
NO_ACCOUNTINGflag, thus still results long qgroup stalls.- A subvolume with no shared subtree
This can still queue a lot of workload into the current transaction.This series covers both less-common cases.
Thanks,
Boris
Quoted 14 lines
Qu Wenruo (2):
btrfs: reject new qgroup rescan during subvolume dropping
btrfs: avoid long stall when dropping a non-shared large subvolumefs/btrfs/disk-io.c | 4 ++++ fs/btrfs/extent-tree.c | 11 +++++++++++ fs/btrfs/qgroup.c | 29 ++++++++++++++++++++++++++++- fs/btrfs/qgroup.h | 15 +++++++++++++++ 4 files changed, 58 insertions(+), 1 deletion(-)Signature · 1 line
2.55.0