Qgroup metadata reservation rework
Subsystems (2)
cover Qgroup metadata reservation rework
[Overall]
The previous rework on qgroup reservation system put a lot of effort on data, which works quite fine.
But it takes less focus on metadata reservation, causing some problem like metadata reservation underflow and noisy kernel warning.
This patchset will try to address the remaining problem of metadata reservation.
The idea of new qgroup metadata reservation is to use 2 types of metadata reservation:
1) Per-transaction reservation
Life span will be inside a transaction. Will be freed at transaction commit time.
2) Preallocated reservation
For case where we reserve space before starting a transaction. Operation like dealloc and delayed-inode/item belongs to this type.
This works similar to block_rsv, its reservation can be reserved/released at any timing caller like.
The only point to notice is, if preallocated reservation is used and finished without problem, it should be converted to per-transaction type instead of just freeing.
This is to co-operate with qgroup update at commit time.
For preallocated type, this patch will integrate them into inode_rsv mechanism reworked by Josef, and delayed-inode/item reservation.
[Problem: Over-reserve]
Currently the patchset addresses metadata underflow quite well, but due to the over-reserve nature of btrfs and highly bounded to inode_rsv, qgroup metadata reservation also tends to be over-reserved.
This is especially obvious for small limit.
For 128M limit, it's will only be able to write about 70M before hitting quota limit.
Although for larger limit, like 5G limit, it can reach 4.5G or more before hitting limit.
Such over-reserved behavior can lead to some problem with existing test cases (where limit is normally less than 20M).
While it's also possible to be addressed by use more accurate space other than max estimations.
For example, to calculate metadata needed for delalloc, we use btrfs_calc_trans_metadata_size(), which always try to reserve space for CoW a full-height tree, and will also include csum size. Both calculate is way over-killed for qgroup metadata reservation.
[Patch structure]
The patch is consist of 2 main parts:
1) Type based qgroup reservation
The original patchset is sent several months ago. Nothing is modified at all, just rebased. And not conflict at all.
It's from patch 1 to patch 6.
2) Split meta qgroup reservation into per-trans and prealloc sub types The real work to address metadata underflow.
Due to the over-reserve problem, this part is still in RFC state. But the framework should mostly be fine, only needs extra fine-tuning to get more accurate qgroup rsv to avoid too early limit.
It's from patch 7 to 14.
Qu Wenruo (14):
btrfs: qgroup: Skeleton to support separate qgroup reservation type
btrfs: qgroup: Introduce helpers to update and access new qgroup rsv
btrfs: qgroup: Make qgroup_reserve and its callers to use separate
reservation type
btrfs: qgroup: Fix wrong qgroup reservation update for relationship
modification
btrfs: qgroup: Update trace events to use new separate rsv types
btrfs: qgroup: Cleanup the remaining old reservation counters
btrfs: qgroup: Split meta rsv type into meta_prealloc and
meta_pertrans
btrfs: qgroup: Don't use root->qgroup_meta_rsv for qgroup
btrfs: qgroup: Introduce function to convert META_PREALLOC into
META_PERTRANS
btrfs: qgroup: Use separate meta reservation type for delalloc
btrfs: delayed-inode: Use new qgroup meta rsv for delayed inode and
item
btrfs: qgroup: Use root->qgroup_meta_rsv_* to record qgroup meta
reserved space
btrfs: qgroup: Update trace events for metadata reservation Revert "btrfs: qgroups: Retry after commit on getting EDQUOT"
fs/btrfs/ctree.h | 15 +- fs/btrfs/delayed-inode.c | 50 +++++-- fs/btrfs/disk-io.c | 2 +- fs/btrfs/extent-tree.c | 49 +++--- fs/btrfs/file.c | 15 +- fs/btrfs/free-space-cache.c | 2 +- fs/btrfs/inode-map.c | 4 +- fs/btrfs/inode.c | 27 ++-- fs/btrfs/ioctl.c | 10 +- fs/btrfs/ordered-data.c | 2 +- fs/btrfs/qgroup.c | 350 ++++++++++++++++++++++++++++++++----------- fs/btrfs/qgroup.h | 102 ++++++++++++- fs/btrfs/relocation.c | 9 +- fs/btrfs/transaction.c | 8 +- include/trace/events/btrfs.h | 73 ++++++++- 15 files changed, 537 insertions(+), 181 deletions(-)
Signature · 1 line
2.15.1
PATCH 1/14 btrfs: qgroup: Skeleton to support separate qgroup reservation type
Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv to restore different types of reservation.
This patch only updates the header and needed modification to pass compile.
--- fs/btrfs/qgroup.c | 16 ++++++++++------ fs/btrfs/qgroup.h | 27 +++++++++++++++++++++++++-- 2 files changed, 35 insertions(+), 8 deletions(-)
fs/btrfs/qgroup.c +10 -6
@@ -2444,7 +2444,8 @@ static int qgroup_reserve(struct btrfs_root *root, u64 num_bytes, bool enforce) }void btrfs_qgroup_free_refroot (struct btrfs_fs_info *fs_info, -u64 ref_root,u64 num_bytes) +u64 ref_root,u64 num_bytes, +enum btrfs_qgroup_rsv_type type) {struct btrfs_root *quota_root;struct btrfs_qgroup *qgroup;
@@ -2936,7 +2937,8 @@ static int qgroup_free_reserved_data(struct inode *inode, goto out; freed += changeset.bytes_changed; } -btrfs_qgroup_free_refroot (root->fs_info, root->objectid, freed); +btrfs_qgroup_free_refroot (root->fs_info, root->objectid, freed, +BTRFS_QGROUP_RSV_DATA ); ret = freed; out:extent_changeset_release (&changeset);
@@ -2968,7 +2970,7 @@ static int __btrfs_qgroup_release_data(struct inode *inode,if (free)btrfs_qgroup_free_refroot (BTRFS_I (inode)->root->fs_info,BTRFS_I (inode)->root->objectid, - changeset.bytes_changed); + changeset.bytes_changed,BTRFS_QGROUP_RSV_DATA ); ret = changeset.bytes_changed; out:extent_changeset_release (&changeset);
@@ -3045,7 +3047,8 @@ void btrfs_qgroup_free_meta_all(struct btrfs_root *root)if (reserved ==0 )return ;trace_qgroup_meta_reserve (root, -(s64 )reserved); -btrfs_qgroup_free_refroot (fs_info, root->objectid, reserved); +btrfs_qgroup_free_refroot (fs_info, root->objectid, reserved, +BTRFS_QGROUP_RSV_META ); }void btrfs_qgroup_free_meta (struct btrfs_root *root,int num_bytes)
@@ -3060,7 +3063,8 @@ void btrfs_qgroup_free_meta(struct btrfs_root *root, int num_bytes)WARN_ON (atomic64_read (&root->qgroup_meta_rsv) < num_bytes);atomic64_sub (num_bytes, &root->qgroup_meta_rsv);trace_qgroup_meta_reserve (root, -(s64 )num_bytes); -btrfs_qgroup_free_refroot (fs_info, root->objectid, num_bytes); +btrfs_qgroup_free_refroot (fs_info, root->objectid, num_bytes, +BTRFS_QGROUP_RSV_META ); }/*
@@ -3088,7 +3092,7 @@ void btrfs_qgroup_check_reserved_leak(struct inode *inode) }btrfs_qgroup_free_refroot (BTRFS_I (inode)->root->fs_info,BTRFS_I (inode)->root->objectid, - changeset.bytes_changed); + changeset.bytes_changed,BTRFS_QGROUP_RSV_DATA ); }extent_changeset_release (&changeset);
fs/btrfs/qgroup.h +25 -2
@@ -61,6 +61,26 @@ struct btrfs_qgroup_extent_record {struct ulist *old_roots; }; +enum btrfs_qgroup_rsv_type { +BTRFS_QGROUP_RSV_DATA =0 , +BTRFS_QGROUP_RSV_META , +BTRFS_QGROUP_RSV_LAST , +}; + +/* +* Represents how many bytes we have reserved for this qgroup. +* +* Each type should have different reservation behavior. +* E.g, data follows its io_tree flag modification, while +* *currently* meta is just reserve-and-clear during transcation. +* +* TODO: Add new type for reservation which can survive transaction commit. +* Currect metadata reservation behavior is not suitable for such case. +*/ +struct btrfs_qgroup_rsv { +u64 values[BTRFS_QGROUP_RSV_LAST ]; +}; +/* * one struct for each qgroup, organized in fs_info->qgroup_tree. */
@@ -88,6 +108,7 @@ struct btrfs_qgroup {* reservation tracking */ u64 reserved; +struct btrfs_qgroup_rsv rsv;/* * lists
@@ -228,12 +249,14 @@ int btrfs_qgroup_inherit(struct btrfs_trans_handle *trans, struct btrfs_fs_info *fs_info, u64 srcid, u64 objectid, struct btrfs_qgroup_inherit *inherit);void btrfs_qgroup_free_refroot (struct btrfs_fs_info *fs_info, -u64 ref_root,u64 num_bytes); +u64 ref_root,u64 num_bytes, +enum btrfs_qgroup_rsv_type type);static inline void btrfs_qgroup_free_delayed_ref (struct btrfs_fs_info *fs_info,u64 ref_root,u64 num_bytes) {trace_btrfs_qgroup_free_delayed_ref (fs_info, ref_root, num_bytes); -btrfs_qgroup_free_refroot (fs_info, ref_root, num_bytes); +btrfs_qgroup_free_refroot (fs_info, ref_root, num_bytes, +BTRFS_QGROUP_RSV_DATA ); }#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
Signature · 1 line
2.15.1
PATCH 2/14 btrfs: qgroup: Introduce helpers to update and access new qgroup rsv
In reply to Qu WenruoIntroduce helpers to:
1) Get total reserved space
For limit calculation
2) Add/release reserved space for given type With underflow detection and warning
3) Add/release reserved space according to child qgroup
--- fs/btrfs/qgroup.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+)
fs/btrfs/qgroup.c +68 -0
@@ -47,6 +47,74 @@* - check all ioctl parameters */ +/* +* Helpers to access qgroup reservation +* +* Callers should ensure the lock context and type are valid +*/ + +static u64 qgroup_rsv_total (const struct btrfs_qgroup *qgroup) +{ +u64 ret =0 ; +int i; + +for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) + ret += qgroup->rsv.values[i]; + +return ret; +} + +#ifdef CONFIG_BTRFS_DEBUG +static const char *qgroup_rsv_type_str (enum btrfs_qgroup_rsv_type type) +{ +if (type ==BTRFS_QGROUP_RSV_DATA ) +return "data" ; +if (type ==BTRFS_QGROUP_RSV_META ) +return "meta" ; +return NULL ; +} +#endif + +static void qgroup_rsv_add (struct btrfs_qgroup *qgroup,u64 num_bytes, +enum btrfs_qgroup_rsv_type type) +{ + qgroup->rsv.values[type] += num_bytes; +} + +static void qgroup_rsv_release (struct btrfs_qgroup *qgroup,u64 num_bytes, +enum btrfs_qgroup_rsv_type type) +{ +if (qgroup->rsv.values[type] >= num_bytes) { + qgroup->rsv.values[type] -= num_bytes; +return ; + } +#ifdef CONFIG_BTRFS_DEBUG +WARN_RATELIMIT (1 , +"qgroup %llu %s reserved space underflow, have %llu to free %llu" , + qgroup->qgroupid,qgroup_rsv_type_str (type), + qgroup->rsv.values[type], num_bytes); +#endif + qgroup->rsv.values[type] =0 ; +} + +static void qgroup_rsv_add_by_qgroup (struct btrfs_qgroup *dest, +struct btrfs_qgroup *src) +{ +int i; + +for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) +qgroup_rsv_add (dest, src->rsv.values[i], i); +} + +static void qgroup_rsv_release_by_qgroup (struct btrfs_qgroup *dest, +struct btrfs_qgroup *src) +{ +int i; + +for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) +qgroup_rsv_release (dest, src->rsv.values[i], i); +} +static void btrfs_qgroup_update_old_refcnt (struct btrfs_qgroup *qg,u64 seq,int mod) {
Signature · 1 line
2.15.1
Re: [PATCH 02/14] btrfs: qgroup: Introduce helpers to update and access new qgroup rsv
Quoted 52 lines · Qu Wenruo
Introduce helpers to:
1) Get total reserved space
For limit calculation
2) Add/release reserved space for given type With underflow detection and warning
3) Add/release reserved space according to child qgroupSigned-off-by Qu Wenruo <[email protected]>--- fs/btrfs/qgroup.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+)fs/btrfs/qgroup.c +31
-0@@ -47,6 +47,74 @@* - check all ioctl parameters */ +/* +* Helpers to access qgroup reservation +* +* Callers should ensure the lock context and type are valid +*/ + +static u64 qgroup_rsv_total (const struct btrfs_qgroup *qgroup) +{ +u64 ret =0 ; +int i; + +for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) + ret += qgroup->rsv.values[i]; + +return ret; +} + +#ifdef CONFIG_BTRFS_DEBUG +static const char *qgroup_rsv_type_str (enum btrfs_qgroup_rsv_type type) +{ +if (type ==BTRFS_QGROUP_RSV_DATA ) +return "data" ; +if (type ==BTRFS_QGROUP_RSV_META ) +return "meta" ; +return NULL ; +} +#endif + +static void qgroup_rsv_add (struct btrfs_qgroup *qgroup,u64 num_bytes, +enum btrfs_qgroup_rsv_type type) +{
fs/btrfs/qgroup.c line 90
Quoted 10 lines
fs/btrfs/qgroup.c +10
-0+ qgroup->rsv.values[type] += num_bytes; +} + +static void qgroup_rsv_release (struct btrfs_qgroup *qgroup,u64 num_bytes, +enum btrfs_qgroup_rsv_type type) +{ +if (qgroup->rsv.values[type] >= num_bytes) { + qgroup->rsv.values[type] -= num_bytes; +return ; + }
nit: Generally I'd prefer that the if tests for the exceptional condition and handle it inside i.e. we expect most of the time for underflow to not occur. Not a big deal so doesn't warrant a resend but something to think about in the future.
Quoted 31 lines
fs/btrfs/qgroup.c +27
-0+#ifdef CONFIG_BTRFS_DEBUG +WARN_RATELIMIT (1 , +"qgroup %llu %s reserved space underflow, have %llu to free %llu" , + qgroup->qgroupid,qgroup_rsv_type_str (type), + qgroup->rsv.values[type], num_bytes); +#endif + qgroup->rsv.values[type] =0 ; +} + +static void qgroup_rsv_add_by_qgroup (struct btrfs_qgroup *dest, +struct btrfs_qgroup *src) +{ +int i; + +for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) +qgroup_rsv_add (dest, src->rsv.values[i], i); +} + +static void qgroup_rsv_release_by_qgroup (struct btrfs_qgroup *dest, +struct btrfs_qgroup *src) +{ +int i; + +for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) +qgroup_rsv_release (dest, src->rsv.values[i], i); +} +static void btrfs_qgroup_update_old_refcnt (struct btrfs_qgroup *qg,u64 seq,int mod) {
PATCH 3/14 btrfs: qgroup: Make qgroup_reserve and its callers to use separate reservation type
In reply to Qu WenruoSince most caller of qgroup_reserve() is already defined by type, converting qgroup_reserve() is quite an easy work.
--- fs/btrfs/qgroup.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
fs/btrfs/qgroup.c +9 -11
@@ -2401,17 +2401,18 @@ int btrfs_qgroup_inherit(struct btrfs_trans_handle *trans,static bool qgroup_check_limits (const struct btrfs_qgroup *qg,u64 num_bytes) {if ((qg->lim_flags &BTRFS_QGROUP_LIMIT_MAX_RFER ) && - qg->reserved + (s64 )qg->rfer + num_bytes > qg->max_rfer) +qgroup_rsv_total (qg) + (s64 )qg->rfer + num_bytes > qg->max_rfer)return false;if ((qg->lim_flags &BTRFS_QGROUP_LIMIT_MAX_EXCL ) && - qg->reserved + (s64 )qg->excl + num_bytes > qg->max_excl) +qgroup_rsv_total (qg) + (s64 )qg->excl + num_bytes > qg->max_excl)return false;return true; } -static int qgroup_reserve (struct btrfs_root *root,u64 num_bytes,bool enforce) +static int qgroup_reserve (struct btrfs_root *root,u64 num_bytes,bool enforce, +enum btrfs_qgroup_rsv_type type) {struct btrfs_root *quota_root;struct btrfs_qgroup *qgroup;
@@ -2463,7 +2464,7 @@ static int qgroup_reserve(struct btrfs_root *root, u64 num_bytes, bool enforce)* Commit the tree and retry, since we may have * deletions which would free up space. */ -if (!retried && qg->reserved >0 ) { +if (!retried &&qgroup_rsv_total (qg) >0 ) {struct btrfs_trans_handle *trans;spin_unlock (&fs_info->qgroup_lock);
@@ -2503,7 +2504,7 @@ static int qgroup_reserve(struct btrfs_root *root, u64 num_bytes, bool enforce) qg =unode_aux_to_qgroup (unode);trace_qgroup_update_reserve (fs_info, qg, num_bytes); - qg->reserved += num_bytes; +qgroup_rsv_add (qg, num_bytes, type); } out:
@@ -2550,10 +2551,7 @@ void btrfs_qgroup_free_refroot(struct btrfs_fs_info *fs_info, qg =unode_aux_to_qgroup (unode);trace_qgroup_update_reserve (fs_info, qg, -(s64 )num_bytes); -if (qg->reserved < num_bytes) -report_reserved_underflow (fs_info, qg, num_bytes); -else - qg->reserved -= num_bytes; +qgroup_rsv_release (qg, num_bytes, type);list_for_each_entry (glist, &qg->groups, next_group) { ret = ulist_add(fs_info->qgroup_ulist,
@@ -2941,7 +2939,7 @@ int btrfs_qgroup_reserve_data(struct inode *inode, to_reserve, QGROUP_RESERVE);if (ret <0 ) goto cleanup; - ret =qgroup_reserve (root, to_reserve, true); + ret =qgroup_reserve (root, to_reserve, true,BTRFS_QGROUP_RSV_DATA );if (ret <0 ) goto cleanup;
@@ -3095,7 +3093,7 @@ int btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes,BUG_ON (num_bytes !=round_down (num_bytes, fs_info->nodesize));trace_qgroup_meta_reserve (root, (s64 )num_bytes); - ret =qgroup_reserve (root, num_bytes, enforce); + ret =qgroup_reserve (root, num_bytes, enforce,BTRFS_QGROUP_RSV_META );if (ret <0 )return ret;atomic64_add (num_bytes, &root->qgroup_meta_rsv);
Signature · 1 line
2.15.1
PATCH 4/14 btrfs: qgroup: Fix wrong qgroup reservation update for relationship modification
In reply to Qu WenruoWhen modifying qgroup relationship, for qgroup which only owns exclusive extents, we will go through quick update path.
In quick update path, we will adding/substracting exclusive and reference number for parent qgroup, since the source (child) qgroup only have exclusive extents, destination (parent) qgroup will also own or lose these extents exclusively.
So should be the same for reservation, since later reservation adding/releasing will also affect parent qgroup, without the servation carried from child, parent will underflow reservation or have dead reservation which will never be freed.
However original code doesn't do the same thing for reservation. It handles qgroup reservation quite differently:
It removes qgroup reservation, as it's allocating space from the reserved qgroup for relationship adding.
But does nothing for qgroup reservation if we're removing a qgroup relationship.
According to the original code, it looks just like because we're adding qgroup->rfer, the code assumes we're writing new data, so it's follows the normal write routine, by reducing qgroup->reserved and adding qgroup->rfer/excl.
This old behavior is wrong, and should be fixed to follow the same excl/rfer behavior.
Just fix it by using the correct behavior described above.
--- fs/btrfs/qgroup.c | 44 +++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-)
fs/btrfs/qgroup.c +23 -21
@@ -1071,21 +1071,30 @@ static void report_reserved_underflow(struct btrfs_fs_info *fs_info,#endif qgroup->reserved =0 ; } +/* -* The easy accounting, if we are adding/removing the only ref for an extent -* then this qgroup and all of the parent qgroups get their reference and -* exclusive counts adjusted. +* The easy accounting, we're updating qgroup relationship whose child qgroup +* only has exclusive extents. +* +* In this case, all exclsuive extents will also be exlusive for parent, so +* excl/rfer just get added/removed. +* +* So is qgroup reservation space, which should also be added/removed to +* parent. +* Or when child tries to release reservation space, parent will underflow its +* reservation (for relationship adding case). * * Caller should hold fs_info->qgroup_lock. */ static int __qgroup_excl_accounting (struct btrfs_fs_info *fs_info,struct ulist *tmp,u64 ref_root, -u64 num_bytes,int sign) +struct btrfs_qgroup *src,int sign) {struct btrfs_qgroup *qgroup;struct btrfs_qgroup_list *glist;struct ulist_node *unode;struct ulist_iterator uiter; +u64 num_bytes = src->excl;int ret =0 ; qgroup =find_qgroup_rb (fs_info, ref_root);
@@ -1098,13 +1107,11 @@ static int __qgroup_excl_accounting(struct btrfs_fs_info *fs_info,WARN_ON (sign <0 && qgroup->excl < num_bytes); qgroup->excl += sign * num_bytes; qgroup->excl_cmpr += sign * num_bytes; -if (sign >0 ) { -trace_qgroup_update_reserve (fs_info, qgroup, -(s64 )num_bytes); -if (qgroup->reserved < num_bytes) -report_reserved_underflow (fs_info, qgroup, num_bytes); -else - qgroup->reserved -= num_bytes; - } + +if (sign >0 ) +qgroup_rsv_add_by_qgroup (qgroup, src); +else +qgroup_rsv_release_by_qgroup (qgroup, src);qgroup_dirty (fs_info, qgroup);
@@ -1124,15 +1131,10 @@ static int __qgroup_excl_accounting(struct btrfs_fs_info *fs_info, qgroup->rfer_cmpr += sign * num_bytes;WARN_ON (sign <0 && qgroup->excl < num_bytes); qgroup->excl += sign * num_bytes; -if (sign >0 ) { -trace_qgroup_update_reserve (fs_info, qgroup, - -(s64 )num_bytes); -if (qgroup->reserved < num_bytes) -report_reserved_underflow (fs_info, qgroup, - num_bytes); -else - qgroup->reserved -= num_bytes; - } +if (sign >0 ) +qgroup_rsv_add_by_qgroup (qgroup, src); +else +qgroup_rsv_release_by_qgroup (qgroup, src); qgroup->excl_cmpr += sign * num_bytes;qgroup_dirty (fs_info, qgroup);
@@ -1175,7 +1177,7 @@ static int quick_update_accounting(struct btrfs_fs_info *fs_info,if (qgroup->excl == qgroup->rfer) { ret =0 ; err =__qgroup_excl_accounting (fs_info, tmp, dst, - qgroup->excl, sign); + qgroup, sign);if (err <0 ) { ret = err; goto out;
Signature · 1 line
2.15.1
PATCH 5/14 btrfs: qgroup: Update trace events to use new separate rsv types
In reply to Qu Wenruo--- fs/btrfs/qgroup.c | 36 +++++++++++++++++++++--------------- include/trace/events/btrfs.h | 17 ++++++++++++----- 2 files changed, 33 insertions(+), 20 deletions(-)
fs/btrfs/qgroup.c +21 -15
@@ -75,15 +75,19 @@ static const char *qgroup_rsv_type_str(enum btrfs_qgroup_rsv_type type) }#endif -static void qgroup_rsv_add (struct btrfs_qgroup *qgroup,u64 num_bytes, +static void qgroup_rsv_add (struct btrfs_fs_info *fs_info, +struct btrfs_qgroup *qgroup,u64 num_bytes,enum btrfs_qgroup_rsv_type type) { +trace_qgroup_update_reserve (fs_info, qgroup, num_bytes, type); qgroup->rsv.values[type] += num_bytes; } -static void qgroup_rsv_release (struct btrfs_qgroup *qgroup,u64 num_bytes, +static void qgroup_rsv_release (struct btrfs_fs_info *fs_info, +struct btrfs_qgroup *qgroup,u64 num_bytes,enum btrfs_qgroup_rsv_type type) { +trace_qgroup_update_reserve (fs_info, qgroup, -(s64 )num_bytes, type);if (qgroup->rsv.values[type] >= num_bytes) { qgroup->rsv.values[type] -= num_bytes;return ;
@@ -97,22 +101,24 @@ static void qgroup_rsv_release(struct btrfs_qgroup *qgroup, u64 num_bytes, qgroup->rsv.values[type] =0 ; } -static void qgroup_rsv_add_by_qgroup (struct btrfs_qgroup *dest, -struct btrfs_qgroup *src) +static void qgroup_rsv_add_by_qgroup (struct btrfs_fs_info *fs_info, +struct btrfs_qgroup *dest, +struct btrfs_qgroup *src) {int i;for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) -qgroup_rsv_add (dest, src->rsv.values[i], i); +qgroup_rsv_add (fs_info, dest, src->rsv.values[i], i); } -static void qgroup_rsv_release_by_qgroup (struct btrfs_qgroup *dest, +static void qgroup_rsv_release_by_qgroup (struct btrfs_fs_info *fs_info, +struct btrfs_qgroup *dest,struct btrfs_qgroup *src) {int i;for (i =0 ; i <BTRFS_QGROUP_RSV_LAST ; i++) -qgroup_rsv_release (dest, src->rsv.values[i], i); +qgroup_rsv_release (fs_info, dest, src->rsv.values[i], i); } static void btrfs_qgroup_update_old_refcnt(struct btrfs_qgroup *qg, u64 seq,
@@ -1109,9 +1115,9 @@ static int __qgroup_excl_accounting(struct btrfs_fs_info *fs_info, qgroup->excl_cmpr += sign * num_bytes;if (sign >0 ) -qgroup_rsv_add_by_qgroup (qgroup, src); +qgroup_rsv_add_by_qgroup (fs_info, qgroup, src);else -qgroup_rsv_release_by_qgroup (qgroup, src); +qgroup_rsv_release_by_qgroup (fs_info, qgroup, src);qgroup_dirty (fs_info, qgroup);
@@ -1132,9 +1138,9 @@ static int __qgroup_excl_accounting(struct btrfs_fs_info *fs_info,WARN_ON (sign <0 && qgroup->excl < num_bytes); qgroup->excl += sign * num_bytes;if (sign >0 ) -qgroup_rsv_add_by_qgroup (qgroup, src); +qgroup_rsv_add_by_qgroup (fs_info, qgroup, src);else -qgroup_rsv_release_by_qgroup (qgroup, src); +qgroup_rsv_release_by_qgroup (fs_info, qgroup, src); qgroup->excl_cmpr += sign * num_bytes;qgroup_dirty (fs_info, qgroup);
@@ -2505,8 +2511,8 @@ static int qgroup_reserve(struct btrfs_root *root, u64 num_bytes, bool enforce, qg =unode_aux_to_qgroup (unode); -trace_qgroup_update_reserve (fs_info, qg, num_bytes); -qgroup_rsv_add (qg, num_bytes, type); +trace_qgroup_update_reserve (fs_info, qg, num_bytes, type); +qgroup_rsv_add (fs_info, qg, num_bytes, type); } out:
@@ -2552,8 +2558,8 @@ void btrfs_qgroup_free_refroot(struct btrfs_fs_info *fs_info, qg =unode_aux_to_qgroup (unode); -trace_qgroup_update_reserve (fs_info, qg, -(s64 )num_bytes); -qgroup_rsv_release (qg, num_bytes, type); +trace_qgroup_update_reserve (fs_info, qg, -(s64 )num_bytes, type); +qgroup_rsv_release (fs_info, qg, num_bytes, type);list_for_each_entry (glist, &qg->groups, next_group) { ret = ulist_add(fs_info->qgroup_ulist,
include/trace/events/btrfs.h +12 -5
@@ -70,6 +70,11 @@ TRACE_DEFINE_ENUM(COMMIT_TRANS); { BTRFS_FILE_EXTENT_REG, "REG" }, \ { BTRFS_FILE_EXTENT_PREALLOC, "PREALLOC"}) +#define show_qgroup_rsv_type (type) \ + __print_symbolic(type, \ + { BTRFS_QGROUP_RSV_DATA, "DATA" }, \ + { BTRFS_QGROUP_RSV_META, "META" }) +#define BTRFS_GROUP_FLAGS \ { BTRFS_BLOCK_GROUP_DATA, "DATA"}, \ { BTRFS_BLOCK_GROUP_SYSTEM, "SYSTEM"}, \
@@ -1598,24 +1603,26 @@ TRACE_EVENT(qgroup_update_counters,TRACE_EVENT (qgroup_update_reserve, TP_PROTO(struct btrfs_fs_info *fs_info, struct btrfs_qgroup *qgroup, - s64 diff), + s64 diff, int type), -TP_ARGS (fs_info, qgroup, diff), +TP_ARGS (fs_info, qgroup, diff, type),TP_STRUCT__entry_btrfs ( __field( u64, qgid ) __field( u64, cur_reserved ) __field( s64, diff ) +__field ( int, type ) ),TP_fast_assign_btrfs (fs_info, __entry->qgid = qgroup->qgroupid; - __entry->cur_reserved = qgroup->reserved; + __entry->cur_reserved = qgroup->rsv.values[type]; __entry->diff = diff; ), -TP_printk_btrfs ("qgid=%llu cur_reserved=%llu diff=%lld" , - __entry->qgid, __entry->cur_reserved, __entry->diff) +TP_printk_btrfs ("qgid=%llu type=%s cur_reserved=%llu diff=%lld" , + __entry->qgid,show_qgroup_rsv_type (__entry->type), + __entry->cur_reserved, __entry->diff) ); TRACE_EVENT(qgroup_meta_reserve,
Signature · 1 line
2.15.1
PATCH 6/14 btrfs: qgroup: Cleanup the remaining old reservation counters
In reply to Qu WenruoSo qgroup is switched to new separate types reservation system.
--- fs/btrfs/qgroup.c | 13 ------------- fs/btrfs/qgroup.h | 1 - 2 files changed, 14 deletions(-)
fs/btrfs/qgroup.c +0 -13
@@ -1065,19 +1065,6 @@ static void qgroup_dirty(struct btrfs_fs_info *fs_info,list_add (&qgroup->dirty, &fs_info->dirty_qgroups); } -static void report_reserved_underflow (struct btrfs_fs_info *fs_info, -struct btrfs_qgroup *qgroup, -u64 num_bytes) -{ -#ifdef CONFIG_BTRFS_DEBUG -WARN_ON (qgroup->reserved < num_bytes); -btrfs_debug (fs_info, -"qgroup %llu reserved space underflow, have: %llu, to free: %llu" , - qgroup->qgroupid, qgroup->reserved, num_bytes); -#endif - qgroup->reserved =0 ; -} -/* * The easy accounting, we're updating qgroup relationship whose child qgroup * only has exclusive extents.
fs/btrfs/qgroup.h +0 -1
@@ -107,7 +107,6 @@ struct btrfs_qgroup {/* * reservation tracking */ -u64 reserved;struct btrfs_qgroup_rsv rsv;/*
Signature · 1 line
2.15.1
PATCH 7/14 btrfs: qgroup: Split meta rsv type into meta_prealloc and meta_pertrans
In reply to Qu WenruoBtrfs uses 2 different method to resever metadata qgroup space.
1) Reserve at btrfs_start_transaction() time
This is quite straightforward, caller will use the trans handler allocated to modify b-trees.
In this case, reserved metadata should be kept until qgroup numbers are updated.
2) Reserve by using block_rsv first, and later btrfs_join_transaction() This is more complicated, caller will reserve space using block_rsv first, and then later call btrfs_join_transaction() to get a trans handler.
In this case, before we modify trees, the reserved space can be modified on demand, and after btrfs_join_transaction(), such reserved space should also be kept until qgroup numbers are updated.
Since this two types behaves quite different to each other, split the original "META" reservation type into 2 sub-types:
META_PERTRANS:
For above case 1)
META_PREALLOC:
For reservation happened before btrfs_join_transaction() of case 2)
NOTE: This patch will only convert existing qgroup meta reservation callers according to its situation, not ensuring all callers are at correct timing.
Such fix will be address in later patches.
--- fs/btrfs/extent-tree.c | 8 +++--- fs/btrfs/qgroup.c | 22 +++++++------- fs/btrfs/qgroup.h | 68 ++++++++++++++++++++++++++++++++++++++++---- fs/btrfs/transaction.c | 8 +++--- include/trace/events/btrfs.h | 5 ++-- 5 files changed, 86 insertions(+), 25 deletions(-)
fs/btrfs/extent-tree.c +4 -4
@@ -5996,7 +5996,7 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root,if (test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags)) {/* One for parent inode, two for dir entries */ num_bytes =3 * fs_info->nodesize; - ret =btrfs_qgroup_reserve_meta (root, num_bytes, true); + ret =btrfs_qgroup_reserve_meta_prealloc (root, num_bytes, true);if (ret)return ret; }else {
@@ -6015,7 +6015,7 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root, ret =btrfs_block_rsv_migrate (global_rsv, rsv, num_bytes,1 );if (ret && *qgroup_reserved) -btrfs_qgroup_free_meta (root, *qgroup_reserved); +btrfs_qgroup_free_meta_prealloc (root, *qgroup_reserved);return ret; }
@@ -6091,7 +6091,7 @@ int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes)spin_unlock (&inode->lock);if (test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags)) { - ret =btrfs_qgroup_reserve_meta (root, + ret =btrfs_qgroup_reserve_meta_prealloc (root, nr_extents * fs_info->nodesize, true);if (ret) goto out_fail;
@@ -6099,7 +6099,7 @@ int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes) ret =btrfs_inode_rsv_refill (inode, flush);if (unlikely (ret)) { -btrfs_qgroup_free_meta (root, +btrfs_qgroup_free_meta_prealloc (root, nr_extents * fs_info->nodesize); goto out_fail; }
fs/btrfs/qgroup.c +12 -10
@@ -69,8 +69,10 @@ static const char *qgroup_rsv_type_str(enum btrfs_qgroup_rsv_type type) {if (type ==BTRFS_QGROUP_RSV_DATA )return "data" ; -if (type ==BTRFS_QGROUP_RSV_META ) -return "meta" ; +if (type ==BTRFS_QGROUP_RSV_META_PERTRANS ) +return "meta_pertrans" ; +if (type ==BTRFS_QGROUP_RSV_META_PREALLOC ) +return "meta_prealloc" ;return NULL ; }#endif
@@ -3076,8 +3078,8 @@ int btrfs_qgroup_release_data(struct inode *inode, u64 start, u64 len)return __btrfs_qgroup_release_data (inode,NULL , start, len,0 ); } -int btrfs_qgroup_reserve_meta (struct btrfs_root *root,int num_bytes, -bool enforce) +int __btrfs_qgroup_reserve_meta (struct btrfs_root *root,int num_bytes, +enum btrfs_qgroup_rsv_type type,bool enforce) {struct btrfs_fs_info *fs_info = root->fs_info;int ret;
@@ -3088,14 +3090,14 @@ int btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes,BUG_ON (num_bytes !=round_down (num_bytes, fs_info->nodesize));trace_qgroup_meta_reserve (root, (s64 )num_bytes); - ret =qgroup_reserve (root, num_bytes, enforce,BTRFS_QGROUP_RSV_META ); + ret =qgroup_reserve (root, num_bytes, enforce, type);if (ret <0 )return ret;atomic64_add (num_bytes, &root->qgroup_meta_rsv);return ret; } -void btrfs_qgroup_free_meta_all (struct btrfs_root *root) +void btrfs_qgroup_free_meta_all_pertrans (struct btrfs_root *root) {struct btrfs_fs_info *fs_info = root->fs_info;u64 reserved;
@@ -3109,10 +3111,11 @@ void btrfs_qgroup_free_meta_all(struct btrfs_root *root)return ;trace_qgroup_meta_reserve (root, -(s64 )reserved);btrfs_qgroup_free_refroot (fs_info, root->objectid, reserved, -BTRFS_QGROUP_RSV_META ); +BTRFS_QGROUP_RSV_META_PERTRANS ); } -void btrfs_qgroup_free_meta (struct btrfs_root *root,int num_bytes) +void __btrfs_qgroup_free_meta (struct btrfs_root *root,int num_bytes, +enum btrfs_qgroup_rsv_type type) {struct btrfs_fs_info *fs_info = root->fs_info;
@@ -3124,8 +3127,7 @@ void btrfs_qgroup_free_meta(struct btrfs_root *root, int num_bytes)WARN_ON (atomic64_read (&root->qgroup_meta_rsv) < num_bytes);atomic64_sub (num_bytes, &root->qgroup_meta_rsv);trace_qgroup_meta_reserve (root, -(s64 )num_bytes); -btrfs_qgroup_free_refroot (fs_info, root->objectid, num_bytes, -BTRFS_QGROUP_RSV_META ); +btrfs_qgroup_free_refroot (fs_info, root->objectid, num_bytes, type); }/*
fs/btrfs/qgroup.h +63 -5
81 lines not on this page · raw message
fs/btrfs/transaction.c +4 -4
@@ -508,8 +508,8 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,*/ if (num_items && root != fs_info->chunk_root) { qgroup_reserved = num_items * fs_info->nodesize; - ret =btrfs_qgroup_reserve_meta (root, qgroup_reserved, - enforce_qgroups); + ret =btrfs_qgroup_reserve_meta_pertrans (root, qgroup_reserved, + enforce_qgroups);if (ret)return ERR_PTR (ret);
@@ -606,7 +606,7 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,btrfs_block_rsv_release (fs_info, &fs_info->trans_block_rsv, num_bytes); reserve_fail: -btrfs_qgroup_free_meta (root, qgroup_reserved); +btrfs_qgroup_free_meta_pertrans (root, qgroup_reserved);return ERR_PTR (ret); }
@@ -1298,7 +1298,7 @@ static noinline int commit_fs_roots(struct btrfs_trans_handle *trans,spin_lock (&fs_info->fs_roots_radix_lock);if (err)break ; -btrfs_qgroup_free_meta_all (root); +btrfs_qgroup_free_meta_all_pertrans (root); } }spin_unlock (&fs_info->fs_roots_radix_lock);
include/trace/events/btrfs.h +3 -2
@@ -72,8 +72,9 @@ TRACE_DEFINE_ENUM(COMMIT_TRANS);#define show_qgroup_rsv_type (type) \ __print_symbolic(type, \ - { BTRFS_QGROUP_RSV_DATA, "DATA" }, \ - { BTRFS_QGROUP_RSV_META, "META" }) + { BTRFS_QGROUP_RSV_DATA, "DATA" }, \ + { BTRFS_QGROUP_RSV_META_PERTRANS, "META_PERTRANS" }, \ + { BTRFS_QGROUP_RSV_META_PREALLOC, "META_PREALLOC" })#define BTRFS_GROUP_FLAGS \ { BTRFS_BLOCK_GROUP_DATA, "DATA"}, \
Signature · 1 line
2.15.1
PATCH 8/14 btrfs: qgroup: Don't use root->qgroup_meta_rsv for qgroup
In reply to Qu WenruoSince qgroup has seperate metadata reservation types now, we can completely get rid of the old root->qgroup_meta_rsv, which mostly acts as current META_PERTRANS reservation type.
--- fs/btrfs/ctree.h | 3 --- fs/btrfs/disk-io.c | 1 - fs/btrfs/qgroup.c | 34 +++++++++++++++++++++++++--------- 3 files changed, 25 insertions(+), 13 deletions(-)
fs/btrfs/ctree.h +0 -3
@@ -1260,9 +1260,6 @@ struct btrfs_root {int send_in_progress;struct btrfs_subvolume_writers *subv_writers;atomic_t will_be_snapshotted; - -/* For qgroup metadata space reserve */ -atomic64_t qgroup_meta_rsv; };struct btrfs_file_private {
fs/btrfs/disk-io.c +0 -1
@@ -1184,7 +1184,6 @@ static void __setup_root(struct btrfs_root *root, struct btrfs_fs_info *fs_info,atomic_set (&root->orphan_inodes,0 );refcount_set (&root->refs,1 );atomic_set (&root->will_be_snapshotted,0 ); -atomic64_set (&root->qgroup_meta_rsv,0 ); root->log_transid =0 ; root->log_transid_committed =-1 ; root->last_log_commit =0 ;
fs/btrfs/qgroup.c +25 -9
@@ -2509,6 +2509,16 @@ static int qgroup_reserve(struct btrfs_root *root, u64 num_bytes, bool enforce,return ret; } +/* +* Free @num_bytes of reserved space with @type for qgroup. +* (normally level 0 qgroup). +* +* Will handle all higher level qgroup either. +* +* NOTE: If @num_bytes is (u64)-1, this means to free all bytes of +* this qgroup. +* This special case is only used for META_PERTRANS type. +*/ void btrfs_qgroup_free_refroot (struct btrfs_fs_info *fs_info,u64 ref_root,u64 num_bytes,enum btrfs_qgroup_rsv_type type)
@@ -2525,6 +2535,10 @@ void btrfs_qgroup_free_refroot(struct btrfs_fs_info *fs_info,if (num_bytes ==0 )return ; +if (num_bytes == (u64)-1 && type !=BTRFS_QGROUP_RSV_META_PERTRANS ) { +WARN (1 ,"%s: Invalid type to free" , __func__); +return ; + }spin_lock (&fs_info->qgroup_lock); quota_root = fs_info->quota_root;
@@ -2535,6 +2549,13 @@ void btrfs_qgroup_free_refroot(struct btrfs_fs_info *fs_info,if (!qgroup) goto out; +/* +* We're freeing all pertrans rsv, get current value from level 0 +* qgroup as real num_bytes to free. +*/ +if (num_bytes == (u64)-1 ) + num_bytes = qgroup->rsv.values[type]; +ulist_reinit (fs_info->qgroup_ulist); ret =ulist_add (fs_info->qgroup_ulist, qgroup->qgroupid, (uintptr_t )qgroup,GFP_ATOMIC );
@@ -3093,24 +3114,21 @@ int __btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes, ret =qgroup_reserve (root, num_bytes, enforce, type);if (ret <0 )return ret; -atomic64_add (num_bytes, &root->qgroup_meta_rsv);return ret; }void btrfs_qgroup_free_meta_all_pertrans (struct btrfs_root *root) {struct btrfs_fs_info *fs_info = root->fs_info; -u64 reserved;if (!test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags) || !is_fstree (root->objectid))return ; - reserved =atomic64_xchg (&root->qgroup_meta_rsv,0 ); -if (reserved ==0 ) -return ; -trace_qgroup_meta_reserve (root, -(s64 )reserved); -btrfs_qgroup_free_refroot (fs_info, root->objectid, reserved, +/* TODO: Update trace point to handle such free */ +trace_qgroup_meta_reserve (root,0 ); +/* Special value -1 means to free all reserved space */ +btrfs_qgroup_free_refroot (fs_info, root->objectid, (u64)-1 ,BTRFS_QGROUP_RSV_META_PERTRANS ); }
@@ -3124,8 +3142,6 @@ void __btrfs_qgroup_free_meta(struct btrfs_root *root, int num_bytes,return ;BUG_ON (num_bytes !=round_down (num_bytes, fs_info->nodesize)); -WARN_ON (atomic64_read (&root->qgroup_meta_rsv) < num_bytes); -atomic64_sub (num_bytes, &root->qgroup_meta_rsv);trace_qgroup_meta_reserve (root, -(s64 )num_bytes);btrfs_qgroup_free_refroot (fs_info, root->objectid, num_bytes, type); }
Signature · 1 line
2.15.1
PATCH 9/14 btrfs: qgroup: Introduce function to convert META_PREALLOC into META_PERTRANS
In reply to Qu WenruoFor meta_prealloc reservation user, after btrfs_join_transaction() caller will modify tree so part (or even all) meta_prealloc reservation should be converted to meta_pertrans until transaction commit time.
This patch introduce a new function, btrfs_qgroup_convert_reserved_meta() to do this for META_PREALLOC reservation user.
--- fs/btrfs/qgroup.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/qgroup.h | 8 ++++++++ 2 files changed, 64 insertions(+)
fs/btrfs/qgroup.c +56 -0
@@ -3146,6 +3146,62 @@ void __btrfs_qgroup_free_meta(struct btrfs_root *root, int num_bytes,btrfs_qgroup_free_refroot (fs_info, root->objectid, num_bytes, type); } +static void qgroup_convert_meta (struct btrfs_fs_info *fs_info,u64 ref_root, +int num_bytes) +{ +struct btrfs_root *quota_root = fs_info->quota_root; +struct btrfs_qgroup *qgroup; +struct ulist_node *unode; +struct ulist_iterator uiter; +int ret =0 ; + +if (num_bytes ==0 ) +return ; +if (!quota_root) +return ; + +spin_lock (&fs_info->qgroup_lock); + qgroup =find_qgroup_rb (fs_info, ref_root); +if (!qgroup) + goto out; +ulist_reinit (fs_info->qgroup_ulist); + ret =ulist_add (fs_info->qgroup_ulist, qgroup->qgroupid, + (uintptr_t )qgroup,GFP_ATOMIC ); +if (ret <0 ) + goto out; +ULIST_ITER_INIT (&uiter); +while ((unode =ulist_next (fs_info->qgroup_ulist, &uiter))) { +struct btrfs_qgroup *qg; +struct btrfs_qgroup_list *glist; + + qg =unode_aux_to_qgroup (unode); + +qgroup_rsv_release (fs_info, qg, num_bytes, +BTRFS_QGROUP_RSV_META_PREALLOC ); +qgroup_rsv_add (fs_info, qg, num_bytes, +BTRFS_QGROUP_RSV_META_PERTRANS ); +list_for_each_entry (glist, &qg->groups, next_group) { + ret =ulist_add (fs_info->qgroup_ulist, + glist->group->qgroupid, + (uintptr_t )glist->group,GFP_ATOMIC ); +if (ret <0 ) + goto out; + } + } +out: +spin_unlock (&fs_info->qgroup_lock); +} + +void btrfs_qgroup_convert_reserved_meta (struct btrfs_root *root,int num_bytes) +{ +struct btrfs_fs_info *fs_info = root->fs_info; + +if (!test_bit (BTRFS_FS_QUOTA_ENABLED , &fs_info->flags) || + !is_fstree (root->objectid)) +return ; +qgroup_convert_meta (fs_info, root->objectid, num_bytes); +} +/* * Check qgroup reserved space leaking, normally at destroy inode * time
fs/btrfs/qgroup.h +8 -0
@@ -332,5 +332,13 @@ static inline void btrfs_qgroup_free_meta_prealloc(struct btrfs_root *root,*/ void btrfs_qgroup_free_meta_all_pertrans (struct btrfs_root *root); +/* +* Convert @num_bytes of META_PREALLOCATED reservation to META_PERTRANS. +* +* This is called when preallocated meta reservation needs to be used. +* Normally after btrfs_join_transaction() call. +*/ +void btrfs_qgroup_convert_reserved_meta (struct btrfs_root *root,int num_bytes); +void btrfs_qgroup_check_reserved_leak (struct inode *inode);#endif /* __BTRFS_QGROUP__ */
Signature · 1 line
2.15.1
PATCH 10/14 btrfs: qgroup: Use separate meta reservation type for delalloc
In reply to Qu WenruoBefore this patch, btrfs qgroup is mixing per-transcation meta rsv with preallocated meta rsv, making it quite easy to underflow qgroup meta reservation.
Since we have the new qgroup meta rsv types, apply it to delalloc reservation.
Now for delalloc, most of its reserved space will use META_PREALLOC qgroup rsv type.
And for callers reducing outstanding extent like btrfs_finish_ordered_io(), they will convert corresponding META_PREALLOC reservation to META_PERTRANS.
This is mainly due to the fact that current qgroup numbers will only be updated in btrfs_commit_transaction(), that's to say if we don't keep such placeholder reservation, we can exceed qgroup limitation.
And for callers freeing outstanding extent in error handler, we will just free META_PREALLOC bytes.
This behavior makes callers of btrfs_qgroup_release_meta() or btrfs_qgroup_convert_meta() to be aware of which type they are. So in this patch, btrfs_delalloc_release_metadata() and its callers get an extra parameter to info qgroup to do correct meta convert/release.
The good news is, even we use the wrong type (convert or free), it won't cause obvious bug, as prealloc type is always in good shape, and the type only affects how per-trans meta is increased or not.
So the worst case will be at most metadata limitation can be sometimes exceeded (no convert at all) or metadata limitation is reached too soon (no free at all).
--- fs/btrfs/ctree.h | 9 ++++++--- fs/btrfs/extent-tree.c | 45 +++++++++++++++++++++++++-------------------- fs/btrfs/file.c | 15 ++++++++------- fs/btrfs/free-space-cache.c | 2 +- fs/btrfs/inode-map.c | 4 ++-- fs/btrfs/inode.c | 27 ++++++++++++++------------- fs/btrfs/ioctl.c | 10 ++++++---- fs/btrfs/ordered-data.c | 2 +- fs/btrfs/relocation.c | 9 +++++---- 9 files changed, 68 insertions(+), 55 deletions(-)
fs/btrfs/ctree.h +6 -3
@@ -2728,7 +2728,8 @@ int btrfs_check_data_free_space(struct inode *inode,void btrfs_free_reserved_data_space (struct inode *inode,struct extent_changeset *reserved,u64 start,u64 len);void btrfs_delalloc_release_space (struct inode *inode, -struct extent_changeset *reserved,u64 start,u64 len); +struct extent_changeset *reserved, +u64 start,u64 len,bool qgroup_free);void btrfs_free_reserved_data_space_noquota (struct inode *inode,u64 start,u64 len); void btrfs_trans_release_metadata(struct btrfs_trans_handle *trans,
@@ -2743,10 +2744,12 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root, u64 *qgroup_reserved, bool use_global_rsv);void btrfs_subvolume_release_metadata (struct btrfs_fs_info *fs_info,struct btrfs_block_rsv *rsv); -void btrfs_delalloc_release_extents (struct btrfs_inode *inode,u64 num_bytes); +void btrfs_delalloc_release_extents (struct btrfs_inode *inode,u64 num_bytes, +bool qgroup_free);int btrfs_delalloc_reserve_metadata (struct btrfs_inode *inode,u64 num_bytes); -void btrfs_delalloc_release_metadata (struct btrfs_inode *inode,u64 num_bytes); +void btrfs_delalloc_release_metadata (struct btrfs_inode *inode,u64 num_bytes, +bool qgroup_free);int btrfs_delalloc_reserve_space (struct inode *inode,struct extent_changeset **reserved,u64 start,u64 len);void btrfs_init_block_rsv (struct btrfs_block_rsv *rsv,unsigned short type);
fs/btrfs/extent-tree.c +25 -20
129 lines not on this page · raw message
fs/btrfs/file.c +8 -7
@@ -1690,7 +1690,7 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file, force_page_uptodate);if (ret) {btrfs_delalloc_release_extents (BTRFS_I (inode), - reserve_bytes); + reserve_bytes, true);break ; }
@@ -1702,7 +1702,7 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file,if (extents_locked == -EAGAIN ) goto again;btrfs_delalloc_release_extents (BTRFS_I (inode), - reserve_bytes); + reserve_bytes, true); ret = extents_locked;break ; }
@@ -1737,7 +1737,7 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file, fs_info->sb->s_blocksize_bits;if (only_release_metadata) {btrfs_delalloc_release_metadata (BTRFS_I (inode), - release_bytes); + release_bytes, true); }else {u64 __pos;
@@ -1746,7 +1746,7 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file, (dirty_pages <<PAGE_SHIFT );btrfs_delalloc_release_space (inode, data_reserved, __pos, - release_bytes); + release_bytes, true); } }
@@ -1760,7 +1760,8 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file,unlock_extent_cached (&BTRFS_I (inode)->io_tree, lockstart, lockend, &cached_state,GFP_NOFS ); -btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes); +btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes, + (ret !=0 ));if (ret) {btrfs_drop_pages (pages, num_pages);break ;
@@ -1800,11 +1801,11 @@ static noinline ssize_t __btrfs_buffered_write(struct file *file,if (only_release_metadata) {btrfs_end_write_no_snapshotting (root);btrfs_delalloc_release_metadata (BTRFS_I (inode), - release_bytes); + release_bytes, true); }else {btrfs_delalloc_release_space (inode, data_reserved,round_down (pos, fs_info->sectorsize), - release_bytes); + release_bytes, true); } }
fs/btrfs/free-space-cache.c +1 -1
@@ -3548,7 +3548,7 @@ int btrfs_write_out_ino_cache(struct btrfs_root *root,if (ret) {if (release_metadata)btrfs_delalloc_release_metadata (BTRFS_I (inode), - inode->i_size); + inode->i_size, true);#ifdef DEBUG btrfs_err(fs_info, "failed to write free ino cache for root %llu",
fs/btrfs/inode-map.c +2 -2
@@ -500,12 +500,12 @@ int btrfs_save_ino_cache(struct btrfs_root *root, ret =btrfs_prealloc_file_range_trans (inode, trans,0 ,0 , prealloc, prealloc, prealloc, &alloc_hint);if (ret) { -btrfs_delalloc_release_extents (BTRFS_I (inode), prealloc); +btrfs_delalloc_release_extents (BTRFS_I (inode), prealloc, true); goto out_put; } ret =btrfs_write_out_ino_cache (root, trans, path, inode); -btrfs_delalloc_release_extents (BTRFS_I (inode), prealloc); +btrfs_delalloc_release_extents (BTRFS_I (inode), prealloc, false); out_put:iput (inode); out_release:
fs/btrfs/inode.c +14 -13
@@ -1839,7 +1839,7 @@ static void btrfs_clear_bit_hook(void *private_data,*/ if (*bits &EXTENT_CLEAR_META_RESV && root != fs_info->tree_root) -btrfs_delalloc_release_metadata (inode, len); +btrfs_delalloc_release_metadata (inode, len, false);/* For sanity tests. */ if (btrfs_is_testing (fs_info))
@@ -2102,7 +2102,7 @@ static void btrfs_writepage_fixup_worker(struct btrfs_work *work) 0);ClearPageChecked (page);set_page_dirty (page); -btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE ); +btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE , false); out:unlock_extent_cached (&BTRFS_I (inode)->io_tree, page_start, page_end, &cached_state,GFP_NOFS );
@@ -4758,8 +4758,8 @@ int btrfs_truncate_block(struct inode *inode, loff_t from, loff_t len, page =find_or_create_page (mapping, index, mask);if (!page) {btrfs_delalloc_release_space (inode, data_reserved, - block_start, blocksize); -btrfs_delalloc_release_extents (BTRFS_I (inode), blocksize); + block_start, blocksize, true); +btrfs_delalloc_release_extents (BTRFS_I (inode), blocksize, true); ret = -ENOMEM ; goto out; }
@@ -4827,8 +4827,8 @@ int btrfs_truncate_block(struct inode *inode, loff_t from, loff_t len, out_unlock:if (ret)btrfs_delalloc_release_space (inode, data_reserved, block_start, - blocksize); -btrfs_delalloc_release_extents (BTRFS_I (inode), blocksize); + blocksize, true); +btrfs_delalloc_release_extents (BTRFS_I (inode), blocksize, (ret !=0 ));unlock_page (page);put_page (page); out:
@@ -8812,7 +8812,7 @@ static ssize_t btrfs_direct_IO(struct kiocb *iocb, struct iov_iter *iter)if (ret <0 && ret != -EIOCBQUEUED ) {if (dio_data.reserve)btrfs_delalloc_release_space (inode, data_reserved, - offset, dio_data.reserve); + offset, dio_data.reserve, true);/* * On error we might have left some ordered extents * without submitting corresponding bios for them, so
@@ -8828,8 +8828,8 @@ static ssize_t btrfs_direct_IO(struct kiocb *iocb, struct iov_iter *iter) false); }else if (ret >=0 && (size_t )ret < count)btrfs_delalloc_release_space (inode, data_reserved, - offset, count - (size_t )ret); -btrfs_delalloc_release_extents (BTRFS_I (inode), count); + offset, count - (size_t )ret, true); +btrfs_delalloc_release_extents (BTRFS_I (inode), count, false); } out:if (wakeup)
@@ -9148,7 +9148,8 @@ int btrfs_page_mkwrite(struct vm_fault *vmf)if (reserved_space <PAGE_SIZE ) { end = page_start + reserved_space -1 ;btrfs_delalloc_release_space (inode, data_reserved, - page_start,PAGE_SIZE - reserved_space); + page_start,PAGE_SIZE - reserved_space, + true); } }
@@ -9198,16 +9199,16 @@ int btrfs_page_mkwrite(struct vm_fault *vmf) out_unlock:if (!ret) { -btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE ); +btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE , true);sb_end_pagefault (inode->i_sb);extent_changeset_free (data_reserved);return VM_FAULT_LOCKED ; }unlock_page (page); out: -btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE ); +btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE , (ret !=0 ));btrfs_delalloc_release_space (inode, data_reserved, page_start, - reserved_space); + reserved_space, (ret !=0 )); out_noreserve:sb_end_pagefault (inode->i_sb);extent_changeset_free (data_reserved);
fs/btrfs/ioctl.c +6 -4
@@ -1198,7 +1198,7 @@ static int cluster_pages_for_defrag(struct inode *inode,spin_unlock (&BTRFS_I (inode)->lock);btrfs_delalloc_release_space (inode, data_reserved, start_index <<PAGE_SHIFT , - (page_cnt - i_done) <<PAGE_SHIFT ); + (page_cnt - i_done) <<PAGE_SHIFT , true); }
@@ -1217,7 +1217,8 @@ static int cluster_pages_for_defrag(struct inode *inode,unlock_page (pages[i]);put_page (pages[i]); } -btrfs_delalloc_release_extents (BTRFS_I (inode), page_cnt <<PAGE_SHIFT ); +btrfs_delalloc_release_extents (BTRFS_I (inode), page_cnt <<PAGE_SHIFT , + false);extent_changeset_free (data_reserved);return i_done; out:
@@ -1227,8 +1228,9 @@ static int cluster_pages_for_defrag(struct inode *inode, }btrfs_delalloc_release_space (inode, data_reserved, start_index <<PAGE_SHIFT , - page_cnt <<PAGE_SHIFT ); -btrfs_delalloc_release_extents (BTRFS_I (inode), page_cnt <<PAGE_SHIFT ); + page_cnt <<PAGE_SHIFT , true); +btrfs_delalloc_release_extents (BTRFS_I (inode), page_cnt <<PAGE_SHIFT , + true);extent_changeset_free (data_reserved);return ret;
fs/btrfs/ordered-data.c +1 -1
@@ -610,7 +610,7 @@ void btrfs_remove_ordered_extent(struct inode *inode,btrfs_mod_outstanding_extents (btrfs_inode,-1 );spin_unlock (&btrfs_inode->lock);if (root != fs_info->tree_root) -btrfs_delalloc_release_metadata (btrfs_inode, entry->len); +btrfs_delalloc_release_metadata (btrfs_inode, entry->len, false); tree = &btrfs_inode->ordered_tree;spin_lock_irq (&tree->lock);
fs/btrfs/relocation.c +5 -4
@@ -3226,7 +3226,7 @@ static int relocate_file_extent_cluster(struct inode *inode, mask);if (!page) {btrfs_delalloc_release_metadata (BTRFS_I (inode), -PAGE_SIZE ); +PAGE_SIZE , true); ret = -ENOMEM ; goto out; }
@@ -3245,9 +3245,9 @@ static int relocate_file_extent_cluster(struct inode *inode,unlock_page (page);put_page (page);btrfs_delalloc_release_metadata (BTRFS_I (inode), -PAGE_SIZE ); +PAGE_SIZE , true);btrfs_delalloc_release_extents (BTRFS_I (inode), -PAGE_SIZE ); +PAGE_SIZE , true); ret = -EIO ; goto out; }
@@ -3278,7 +3278,8 @@ static int relocate_file_extent_cluster(struct inode *inode,put_page (page); index++; -btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE ); +btrfs_delalloc_release_extents (BTRFS_I (inode),PAGE_SIZE , + false);balance_dirty_pages_ratelimited (inode->i_mapping);btrfs_throttle (fs_info); }
Signature · 1 line
2.15.1
PATCH 11/14 btrfs: delayed-inode: Use new qgroup meta rsv for delayed inode and item
In reply to Qu WenruoQuite similar for delalloc, some modification to delayed-inode and delayed-item reservation.
Also needs extra parameter for release case to distinguish normal release and error release.
--- fs/btrfs/delayed-inode.c | 50 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 16 deletions(-)
fs/btrfs/delayed-inode.c +34 -16
@@ -22,6 +22,7 @@#include "disk-io.h" #include "transaction.h" #include "ctree.h" +#include "qgroup.h" #define BTRFS_DELAYED_WRITEBACK 512#define BTRFS_DELAYED_BACKGROUND 128
@@ -528,11 +529,12 @@ static struct btrfs_delayed_item *__btrfs_next_delayed_item( }static int btrfs_delayed_item_reserve_metadata (struct btrfs_trans_handle *trans, -struct btrfs_fs_info *fs_info, +struct btrfs_root *root,struct btrfs_delayed_item *item) {struct btrfs_block_rsv *src_rsv;struct btrfs_block_rsv *dst_rsv; +struct btrfs_fs_info *fs_info = root->fs_info;u64 num_bytes;int ret;
@@ -543,6 +545,10 @@ static int btrfs_delayed_item_reserve_metadata(struct btrfs_trans_handle *trans, dst_rsv = &fs_info->delayed_block_rsv; num_bytes =btrfs_calc_trans_metadata_size (fs_info,1 ); + ret =btrfs_qgroup_reserve_meta_prealloc (root, num_bytes, true); +if (ret <0 ) +return ret; + ret =btrfs_block_rsv_migrate (src_rsv, dst_rsv, num_bytes,1 );if (!ret) { trace_btrfs_space_reservation(fs_info, "delayed_item",
@@ -554,15 +560,17 @@ static int btrfs_delayed_item_reserve_metadata(struct btrfs_trans_handle *trans,return ret; } -static void btrfs_delayed_item_release_metadata (struct btrfs_fs_info *fs_info, +static void btrfs_delayed_item_release_metadata (struct btrfs_root *root,struct btrfs_delayed_item *item) {struct btrfs_block_rsv *rsv; +struct btrfs_fs_info *fs_info = root->fs_info;if (!item->bytes_reserved)return ; rsv = &fs_info->delayed_block_rsv; +btrfs_qgroup_convert_reserved_meta (root, item->bytes_reserved);trace_btrfs_space_reservation (fs_info,"delayed_item" , item->key.objectid, item->bytes_reserved,0 );
@@ -587,6 +595,9 @@ static int btrfs_delayed_inode_reserve_metadata( num_bytes =btrfs_calc_trans_metadata_size (fs_info,1 ); + ret =btrfs_qgroup_reserve_meta_prealloc (root, num_bytes, true); +if (ret <0 ) +return ret;/* * btrfs_dirty_inode will update the inode under btrfs_join_transaction * which doesn't reserve space for speed. This is a problem since we
@@ -606,8 +617,10 @@ static int btrfs_delayed_inode_reserve_metadata(* EAGAIN to make us stop the transaction we have, so return * ENOSPC instead so that btrfs_dirty_inode knows what to do. */ -if (ret == -EAGAIN ) +if (ret == -EAGAIN ) { ret = -ENOSPC ; +btrfs_qgroup_free_meta_prealloc (root, num_bytes); + }if (!ret) { node->bytes_reserved = num_bytes; trace_btrfs_space_reservation(fs_info,
@@ -629,7 +642,8 @@ static int btrfs_delayed_inode_reserve_metadata( }static void btrfs_delayed_inode_release_metadata (struct btrfs_fs_info *fs_info, -struct btrfs_delayed_node *node) +struct btrfs_delayed_node *node, +bool qgroup_free) {struct btrfs_block_rsv *rsv;
@@ -641,6 +655,12 @@ static void btrfs_delayed_inode_release_metadata(struct btrfs_fs_info *fs_info, node->inode_id, node->bytes_reserved, 0);btrfs_block_rsv_release (fs_info, rsv, node->bytes_reserved); +if (qgroup_free) +btrfs_qgroup_free_meta_prealloc (node->root, + node->bytes_reserved); +else +btrfs_qgroup_convert_reserved_meta (node->root, + node->bytes_reserved); node->bytes_reserved =0 ; }
@@ -742,7 +762,7 @@ static int btrfs_batch_insert_items(struct btrfs_root *root, curr->data_len); slot++; -btrfs_delayed_item_release_metadata (fs_info, curr); +btrfs_delayed_item_release_metadata (root, curr);list_del (&curr->tree_list);btrfs_release_delayed_item (curr);
@@ -764,7 +784,6 @@ static int btrfs_insert_delayed_item(struct btrfs_trans_handle *trans,struct btrfs_path *path, struct btrfs_delayed_item *delayed_item) { - struct btrfs_fs_info *fs_info = root->fs_info; struct extent_buffer *leaf;char *ptr;int ret;
@@ -782,7 +801,7 @@ static int btrfs_insert_delayed_item(struct btrfs_trans_handle *trans, delayed_item->data_len);btrfs_mark_buffer_dirty (leaf); -btrfs_delayed_item_release_metadata (fs_info, delayed_item); +btrfs_delayed_item_release_metadata (root, delayed_item);return 0 ; }
@@ -834,7 +853,6 @@ static int btrfs_batch_delete_items(struct btrfs_trans_handle *trans,struct btrfs_path *path, struct btrfs_delayed_item *item) { - struct btrfs_fs_info *fs_info = root->fs_info; struct btrfs_delayed_item *curr, *next;struct extent_buffer *leaf;struct btrfs_key key;
@@ -884,7 +902,7 @@ static int btrfs_batch_delete_items(struct btrfs_trans_handle *trans, goto out;list_for_each_entry_safe (curr, next, &head, tree_list) { -btrfs_delayed_item_release_metadata (fs_info, curr); +btrfs_delayed_item_release_metadata (root, curr);list_del (&curr->tree_list);btrfs_release_delayed_item (curr); }
@@ -1027,7 +1045,7 @@ static int __btrfs_update_delayed_inode(struct btrfs_trans_handle *trans, no_iref:btrfs_release_path (path); err_out: -btrfs_delayed_inode_release_metadata (fs_info, node); +btrfs_delayed_inode_release_metadata (fs_info, node, (ret <0 ));btrfs_release_delayed_inode (node);return ret;
@@ -1420,7 +1438,7 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,btrfs_set_stack_dir_type (dir_item, type);memcpy ((char *)(dir_item +1 ), name, name_len); - ret =btrfs_delayed_item_reserve_metadata (trans, fs_info, delayed_item); + ret =btrfs_delayed_item_reserve_metadata (trans, dir->root, delayed_item);/* * we have reserved enough space when we start a new transaction, * so reserving metadata failure is impossible
@@ -1457,7 +1475,7 @@ static int btrfs_delete_delayed_insertion_item(struct btrfs_fs_info *fs_info,return 1 ; } -btrfs_delayed_item_release_metadata (fs_info, item); +btrfs_delayed_item_release_metadata (node->root, item);btrfs_release_delayed_item (item);mutex_unlock (&node->mutex);return 0 ;
@@ -1492,7 +1510,7 @@ int btrfs_delete_delayed_dir_index(struct btrfs_trans_handle *trans, item->key = item_key; - ret =btrfs_delayed_item_reserve_metadata (trans, fs_info, item); + ret =btrfs_delayed_item_reserve_metadata (trans, dir->root, item);/* * we have reserved enough space when we start a new transaction, * so reserving metadata failure is impossible.
@@ -1865,7 +1883,7 @@ static void __btrfs_kill_delayed_node(struct btrfs_delayed_node *delayed_node)mutex_lock (&delayed_node->mutex); curr_item =__btrfs_first_delayed_insertion_item (delayed_node);while (curr_item) { -btrfs_delayed_item_release_metadata (fs_info, curr_item); +btrfs_delayed_item_release_metadata (root, curr_item); prev_item = curr_item; curr_item =__btrfs_next_delayed_item (prev_item);btrfs_release_delayed_item (prev_item);
@@ -1873,7 +1891,7 @@ static void __btrfs_kill_delayed_node(struct btrfs_delayed_node *delayed_node) curr_item =__btrfs_first_delayed_deletion_item (delayed_node);while (curr_item) { -btrfs_delayed_item_release_metadata (fs_info, curr_item); +btrfs_delayed_item_release_metadata (root, curr_item); prev_item = curr_item; curr_item =__btrfs_next_delayed_item (prev_item);btrfs_release_delayed_item (prev_item);
@@ -1883,7 +1901,7 @@ static void __btrfs_kill_delayed_node(struct btrfs_delayed_node *delayed_node)btrfs_release_delayed_iref (delayed_node);if (test_bit (BTRFS_DELAYED_NODE_INODE_DIRTY , &delayed_node->flags)) { -btrfs_delayed_inode_release_metadata (fs_info, delayed_node); +btrfs_delayed_inode_release_metadata (fs_info, delayed_node, false);btrfs_release_delayed_inode (delayed_node); }mutex_unlock (&delayed_node->mutex);
Signature · 1 line
2.15.1
PATCH 12/14 btrfs: qgroup: Use root->qgroup_meta_rsv_* to record qgroup meta reserved space
In reply to Qu WenruoFor quota disabled->enable case, it's possible that at reservation time quota was not enabled so no byte was really reserved, while at release time, quota is enabled so we will try to release some bytes we didn't really own.
Such situation can cause metadata reserveation underflow, for both types, also less possible for per-trans type since quota enable will commit transaction.
To address this, record qgroup meta reserved bytes into root->qgroup_meta_rsv_pertrans/prealloc.
So at releasing time we won't free any bytes we didn't reserve.
For DATA, it's already handled by io_tree, so nothing needs to be worried about.
--- fs/btrfs/ctree.h | 5 +++++ fs/btrfs/disk-io.c | 1 + fs/btrfs/qgroup.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 68 insertions(+), 4 deletions(-)
fs/btrfs/ctree.h +5 -0
@@ -1260,6 +1260,11 @@ struct btrfs_root {int send_in_progress;struct btrfs_subvolume_writers *subv_writers;atomic_t will_be_snapshotted; + +/* For qgroup metadata reserved space */ +spinlock_t qgroup_meta_rsv_lock; +u64 qgroup_meta_rsv_pertrans; +u64 qgroup_meta_rsv_prealloc; };struct btrfs_file_private {
fs/btrfs/disk-io.c +1 -0
@@ -1168,6 +1168,7 @@ static void __setup_root(struct btrfs_root *root, struct btrfs_fs_info *fs_info,spin_lock_init (&root->accounting_lock);spin_lock_init (&root->log_extents_lock[0 ]);spin_lock_init (&root->log_extents_lock[1 ]); +spin_lock_init (&root->qgroup_meta_rsv_lock);mutex_init (&root->objectid_mutex);mutex_init (&root->log_mutex);mutex_init (&root->ordered_extent_mutex);
fs/btrfs/qgroup.c +62 -4
97 lines not on this page · raw message
Signature · 1 line
2.15.1
PATCH 13/14 btrfs: qgroup: Update trace events for metadata reservation
In reply to Qu WenruoNow trace_qgroup_meta_reserve() will have extra type parameter.
And introduce two new trace events:
1) trace_qgroup_meta_free_all_pertrans()
For btrfs_qgroup_free_meta_all_pertrans()
2) trace_qgroup_meta_convert()
For btrfs_qgroup_convert_reserved_meta()
--- fs/btrfs/qgroup.c | 7 +++--- include/trace/events/btrfs.h | 55 ++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 57 insertions(+), 5 deletions(-)
fs/btrfs/qgroup.c +4 -3
@@ -3150,7 +3150,7 @@ int __btrfs_qgroup_reserve_meta(struct btrfs_root *root, int num_bytes,return 0 ;BUG_ON (num_bytes !=round_down (num_bytes, fs_info->nodesize)); -trace_qgroup_meta_reserve (root, (s64 )num_bytes); +trace_qgroup_meta_reserve (root, type, (s64 )num_bytes); ret =qgroup_reserve (root, num_bytes, enforce, type);if (ret <0 )return ret;
@@ -3175,7 +3175,7 @@ void btrfs_qgroup_free_meta_all_pertrans(struct btrfs_root *root)return ;/* TODO: Update trace point to handle such free */ -trace_qgroup_meta_reserve (root,0 ); +trace_qgroup_meta_free_all_pertrans (root);/* Special value -1 means to free all reserved space */ btrfs_qgroup_free_refroot (fs_info, root->objectid, (u64)-1 ,BTRFS_QGROUP_RSV_META_PERTRANS );
@@ -3197,7 +3197,7 @@ void __btrfs_qgroup_free_meta(struct btrfs_root *root, int num_bytes,*/ num_bytes =sub_root_meta_rsv (root, num_bytes, type);BUG_ON (num_bytes !=round_down (num_bytes, fs_info->nodesize)); -trace_qgroup_meta_reserve (root, -(s64 )num_bytes); +trace_qgroup_meta_reserve (root, type, -(s64 )num_bytes);btrfs_qgroup_free_refroot (fs_info, root->objectid, num_bytes, type); }
@@ -3257,6 +3257,7 @@ void btrfs_qgroup_convert_reserved_meta(struct btrfs_root *root, int num_bytes)/* Same as btrfs_qgroup_free_meta_prealloc() */ num_bytes =sub_root_meta_rsv (root, num_bytes,BTRFS_QGROUP_RSV_META_PREALLOC ); +trace_qgroup_meta_convert (root, num_bytes);qgroup_convert_meta (fs_info, root->objectid, num_bytes); }
include/trace/events/btrfs.h +53 -2
73 lines not on this page · raw message
Signature · 1 line
2.15.1
PATCH 14/14 Revert "btrfs: qgroups: Retry after commit on getting EDQUOT"
In reply to Qu WenruoThis reverts commit 48a89bc4f2ceab87bc858a8eb189636b09c846a7.
The idea to commit transaction and free some space after hitting qgroup limit is good, although the problem is it will easily cause deadlocks.
One deadlock example is caused by trying to flush data while still holding it:
Call Trace: __schedule+0x49d/0x10f0 schedule+0xc6/0x290 schedule_timeout+0x187/0x1c0 wait_for_completion+0x204/0x3a0 btrfs_wait_ordered_extents+0xa40/0xaf0 [btrfs] qgroup_reserve+0x913/0xa10 [btrfs] btrfs_qgroup_reserve_data+0x3ef/0x580 [btrfs] btrfs_check_data_free_space+0x96/0xd0 [btrfs] __btrfs_buffered_write+0x3ac/0xd40 [btrfs] btrfs_file_write_iter+0x62a/0xba0 [btrfs] __vfs_write+0x320/0x430 vfs_write+0x107/0x270 SyS_write+0xbf/0x150 do_syscall_64+0x1b0/0x3d0 entry_SYSCALL64_slow_path+0x25/0x25
Another case can be caused by trying to commit one transaction while nesting with trans handler hold by ourselves:btrfs_start_transaction()
|- btrfs_qgroup_reserve_meta_pertrans()
|- qgroup_reserve()
|- btrfs_join_transaction()
|- btrfs_commit_transaction()
The retry is causing more problem than expectation when limit is enabled.
At least graceful EDQUOT is way better than kernel deadlock.
--- fs/btrfs/qgroup.c | 22 ---------------------- 1 file changed, 22 deletions(-)
fs/btrfs/qgroup.c +0 -22
@@ -2416,7 +2416,6 @@ static int qgroup_reserve(struct btrfs_root *root, u64 num_bytes, bool enforce,struct btrfs_fs_info *fs_info = root->fs_info;u64 ref_root = root->root_key.objectid;int ret =0 ; -int retried =0 ;struct ulist_node *unode;struct ulist_iterator uiter;
@@ -2457,27 +2456,6 @@ static int qgroup_reserve(struct btrfs_root *root, u64 num_bytes, bool enforce, qg =unode_aux_to_qgroup (unode);if (enforce && !qgroup_check_limits (qg, num_bytes)) { -/* -* Commit the tree and retry, since we may have -* deletions which would free up space. -*/ -if (!retried &&qgroup_rsv_total (qg) >0 ) { -struct btrfs_trans_handle *trans; - -spin_unlock (&fs_info->qgroup_lock); - ret =btrfs_start_delalloc_inodes (root,0 ); -if (ret) -return ret; -btrfs_wait_ordered_extents (root,U64_MAX ,0 , (u64)-1 ); - trans =btrfs_join_transaction (root); -if (IS_ERR (trans)) -return PTR_ERR (trans); - ret =btrfs_commit_transaction (trans); -if (ret) -return ret; - retried++; - goto retry; - } ret = -EDQUOT ; goto out; }
Signature · 1 line
2.15.1
Quoted 46 lines · Qu Wenruo
[Overall]
The previous rework on qgroup reservation system put a lot of effort on data, which works quite fine.But it takes less focus on metadata reservation, causing some problem like metadata reservation underflow and noisy kernel warning.
This patchset will try to address the remaining problem of metadata reservation.
The idea of new qgroup metadata reservation is to use 2 types of metadata reservation:
1) Per-transaction reservation
Life span will be inside a transaction. Will be freed at transaction commit time.2) Preallocated reservation
For case where we reserve space before starting a transaction. Operation like dealloc and delayed-inode/item belongs to this type.This works similar to
block_rsv, its reservation can be reserved/released at any timing caller like.The only point to notice is, if preallocated reservation is used and finished without problem, it should be converted to per-transaction type instead of just freeing.
This is to co-operate with qgroup update at commit time.For preallocated type, this patch will integrate them into
inode_rsvmechanism reworked by Josef, and delayed-inode/item reservation.[Problem: Over-reserve]
Currently the patchset addresses metadata underflow quite well, but due to the over-reserve nature of btrfs and highly bounded toinode_rsv, qgroup metadata reservation also tends to be over-reserved.This is especially obvious for small limit.
For 128M limit, it's will only be able to write about 70M before hitting quota limit.
Although for larger limit, like 5G limit, it can reach 4.5G or more before hitting limit.Such over-reserved behavior can lead to some problem with existing test cases (where limit is normally less than 20M).
Quoted 7 lines
While it's also possible to be addressed by use more accurate space other than max estimations.
For example, to calculate metadata needed for delalloc, we use
btrfs_calc_trans_metadata_size(), which always try to reserve space for CoW a full-height tree, and will also include csum size. Both calculate is way over-killed for qgroup metadata reservation.
In private chat with Chris couple of months ago we discussed making the reservation a lot less pessimistic. One assumption which we could exploit is the fact that upon a tree split it's unlikely we will create more than 1 additional level in the tree. So we could potentially modify btrfs_calc_trans_metadata_size to take a root parameter and instead of
BTRFS_MAX_LEVEL * 2 we could change this to root_level * 2. How does that sound?
Quoted 56 lines
[Patch structure]
The patch is consist of 2 main parts:
1) Type based qgroup reservation
The original patchset is sent several months ago. Nothing is modified at all, just rebased. And not conflict at all.It's from patch 1 to patch 6.
2) Split meta qgroup reservation into per-trans and prealloc sub types The real work to address metadata underflow.
Due to the over-reserve problem, this part is still in RFC state. But the framework should mostly be fine, only needs extra fine-tuning to get more accurate qgroup rsv to avoid too early limit.It's from patch 7 to 14.
Qu Wenruo (14):
btrfs: qgroup: Skeleton to support separate qgroup reservation type
btrfs: qgroup: Introduce helpers to update and access new qgroup rsv
btrfs: qgroup: Makeqgroup_reserveand its callers to use separate
reservation type
btrfs: qgroup: Fix wrong qgroup reservation update for relationship
modification
btrfs: qgroup: Update trace events to use new separate rsv types
btrfs: qgroup: Cleanup the remaining old reservation counters
btrfs: qgroup: Split meta rsv type intometa_preallocand
meta_pertrans
btrfs: qgroup: Don't useroot->qgroup_meta_rsvfor qgroup
btrfs: qgroup: Introduce function to convertMETA_PREALLOCinto
META_PERTRANS
btrfs: qgroup: Use separate meta reservation type for delalloc
btrfs: delayed-inode: Use new qgroup meta rsv for delayed inode and
item
btrfs: qgroup: Use root->qgroup_meta_rsv_* to record qgroup meta
reserved space
btrfs: qgroup: Update trace events for metadata reservation Revert "btrfs: qgroups: Retry after commit on getting EDQUOT"fs/btrfs/ctree.h | 15 +- fs/btrfs/delayed-inode.c | 50 +++++-- fs/btrfs/disk-io.c | 2 +- fs/btrfs/extent-tree.c | 49 +++--- fs/btrfs/file.c | 15 +- fs/btrfs/free-space-cache.c | 2 +- fs/btrfs/inode-map.c | 4 +- fs/btrfs/inode.c | 27 ++-- fs/btrfs/ioctl.c | 10 +- fs/btrfs/ordered-data.c | 2 +- fs/btrfs/qgroup.c | 350 ++++++++++++++++++++++++++++++++----------- fs/btrfs/qgroup.h | 102 ++++++++++++- fs/btrfs/relocation.c | 9 +- fs/btrfs/transaction.c | 8 +- include/trace/events/btrfs.h | 73 ++++++++- 15 files changed, 537 insertions(+), 181 deletions(-)
Quoted 56 lines · Nikolay Borisov
Quoted 46 lines · Qu Wenruo
[Overall]
The previous rework on qgroup reservation system put a lot of effort on data, which works quite fine.But it takes less focus on metadata reservation, causing some problem like metadata reservation underflow and noisy kernel warning.
This patchset will try to address the remaining problem of metadata reservation.
The idea of new qgroup metadata reservation is to use 2 types of metadata reservation:
1) Per-transaction reservation
Life span will be inside a transaction. Will be freed at transaction commit time.2) Preallocated reservation
For case where we reserve space before starting a transaction. Operation like dealloc and delayed-inode/item belongs to this type.This works similar to
block_rsv, its reservation can be reserved/released at any timing caller like.The only point to notice is, if preallocated reservation is used and finished without problem, it should be converted to per-transaction type instead of just freeing.
This is to co-operate with qgroup update at commit time.For preallocated type, this patch will integrate them into
inode_rsvmechanism reworked by Josef, and delayed-inode/item reservation.[Problem: Over-reserve]
Currently the patchset addresses metadata underflow quite well, but due to the over-reserve nature of btrfs and highly bounded toinode_rsv, qgroup metadata reservation also tends to be over-reserved.This is especially obvious for small limit.
For 128M limit, it's will only be able to write about 70M before hitting quota limit.
Although for larger limit, like 5G limit, it can reach 4.5G or more before hitting limit.Such over-reserved behavior can lead to some problem with existing test cases (where limit is normally less than 20M).
Quoted 7 lines
While it's also possible to be addressed by use more accurate space other than max estimations.
For example, to calculate metadata needed for delalloc, we use
btrfs_calc_trans_metadata_size(), which always try to reserve space for CoW a full-height tree, and will also include csum size. Both calculate is way over-killed for qgroup metadata reservation.
Quoted 8 lines
In private chat with Chris couple of months ago we discussed making the reservation a lot less pessimistic. One assumption which we could exploit is the fact that upon a tree split it's unlikely we will create more than 1 additional level in the tree. So we could potentially modify
btrfs_calc_trans_metadata_sizeto take a root parameter and instead of
BTRFS_MAX_LEVEL* 2 we could change this toroot_level* 2. How does that sound?
That sounds unsafe for low levels, because the reservations can be made with level N and the tree height can increase at commit time to > N, then we'd go ENOSPC. But BTRFS_MAX_LEVEL can be reduced to min(root_level + 1, 4) and even the +1 can be dropped for higher levels if the root node is filled up to some percent (say < 50%). The 4 is a guess and could be possibly 3, with some more precise calculations.
Quoted 57 lines · David Sterba
Quoted 56 lines · Nikolay Borisov
Quoted 53 lines · Qu Wenruo
[Overall]
The previous rework on qgroup reservation system put a lot of effort on data, which works quite fine.But it takes less focus on metadata reservation, causing some problem like metadata reservation underflow and noisy kernel warning.
This patchset will try to address the remaining problem of metadata reservation.
The idea of new qgroup metadata reservation is to use 2 types of metadata reservation:
1) Per-transaction reservation
Life span will be inside a transaction. Will be freed at transaction commit time.2) Preallocated reservation
For case where we reserve space before starting a transaction. Operation like dealloc and delayed-inode/item belongs to this type.This works similar to
block_rsv, its reservation can be reserved/released at any timing caller like.The only point to notice is, if preallocated reservation is used and finished without problem, it should be converted to per-transaction type instead of just freeing.
This is to co-operate with qgroup update at commit time.For preallocated type, this patch will integrate them into
inode_rsvmechanism reworked by Josef, and delayed-inode/item reservation.[Problem: Over-reserve]
Currently the patchset addresses metadata underflow quite well, but due to the over-reserve nature of btrfs and highly bounded toinode_rsv, qgroup metadata reservation also tends to be over-reserved.This is especially obvious for small limit.
For 128M limit, it's will only be able to write about 70M before hitting quota limit.
Although for larger limit, like 5G limit, it can reach 4.5G or more before hitting limit.Such over-reserved behavior can lead to some problem with existing test cases (where limit is normally less than 20M).
While it's also possible to be addressed by use more accurate space other than max estimations.
For example, to calculate metadata needed for delalloc, we use
btrfs_calc_trans_metadata_size(), which always try to reserve space for CoW a full-height tree, and will also include csum size. Both calculate is way over-killed for qgroup metadata reservation.
Quoted 8 lines
Quoted 8 lines
In private chat with Chris couple of months ago we discussed making the reservation a lot less pessimistic. One assumption which we could exploit is the fact that upon a tree split it's unlikely we will create more than 1 additional level in the tree. So we could potentially modify
btrfs_calc_trans_metadata_sizeto take a root parameter and instead of
BTRFS_MAX_LEVEL* 2 we could change this toroot_level* 2. How does that sound?
My plan is more aggressive.
Since qgroup doesn't really need to care about low level space reservation for extents, here we only needs to care about "net" modification.
For item insert, it will takes one new tree block for split at most (if I don't miss something).
So for qgroup, one ordered extent should only reserve one nodesize.
In fact, for per-trans reservation, we have been using this for a long time.
Quoted 7 lines
That sounds unsafe for low levels, because the reservations can be made with level N and the tree height can increase at commit time to > N, then we'd go ENOSPC. But
BTRFS_MAX_LEVELcan be reduced tomin(root_level + 1, 4)and even the +1 can be dropped for higher levels if the root node is filled up to some percent (say < 50%). The 4 is a guess and could be possibly 3, with some more precise calculations.
So the idea only applies to qgroup, nothing is changed for low level space reservation.
Although this needs extra work in qgroup side.
Current patchset completely rely on current outstanding extent mechanism to ensure its reservation is reserved and released correctly.
If qgroup is going to do its own reservation calculation, it needs extra work to handle.
But compared to too early EDQUOT, it's still worthy anyway.
Thanks,
Qu
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines · Qu Wenruo
The patch is consist of 2 main parts:
1) Type based qgroup reservation
The original patchset is sent several months ago. Nothing is modified at all, just rebased. And not conflict at all.It's from patch 1 to patch 6.
Quoted 7 lines
2) Split meta qgroup reservation into per-trans and prealloc sub types The real work to address metadata underflow.
Due to the over-reserve problem, this part is still in RFC state. But the framework should mostly be fine, only needs extra fine-tuning to get more accurate qgroup rsv to avoid too early limit.It's from patch 7 to 14.
I'm going to add the whole patchset to next, the first part has been there for some time and no test failures were reported. I optimistically expect that the second part will also be fine. However I'm not sure how good is the test coverage of qgroups in fstests, so lack of reports may not mean anything.
Quoted 14 lines · David Sterba
Quoted 13 lines · Qu Wenruo
The patch is consist of 2 main parts:
1) Type based qgroup reservation
The original patchset is sent several months ago. Nothing is modified at all, just rebased. And not conflict at all.It's from patch 1 to patch 6.
2) Split meta qgroup reservation into per-trans and prealloc sub types The real work to address metadata underflow.
Due to the over-reserve problem, this part is still in RFC state. But the framework should mostly be fine, only needs extra fine-tuning to get more accurate qgroup rsv to avoid too early limit.
Quoted 5 lines
Quoted 1 line
It's from patch 7 to 14.
I'm going to add the whole patchset to next, the first part has been there for some time and no test failures were reported. I optimistically expect that the second part will also be fine.
The type based reservation is completely fine, since it doesn't introduce anything new, just a preparation for the incoming meta rework.
However I prefer not to push the whole patchset to upstream until over-reserve behavior is solved.
Since it breaks quite some test cases with small limit.
Quoted 3 lines
However I'm not sure how
good is the test coverage of qgroups in fstests, so lack of reports may not mean anything.
That's why I submitted mkfs.btrfs quota support some times ago. :^)
Thanks,
Qu
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 21 lines · Qu Wenruo
Quoted 14 lines · David Sterba
Quoted 13 lines · Qu Wenruo
The patch is consist of 2 main parts:
1) Type based qgroup reservation
The original patchset is sent several months ago. Nothing is modified at all, just rebased. And not conflict at all.It's from patch 1 to patch 6.
2) Split meta qgroup reservation into per-trans and prealloc sub types The real work to address metadata underflow.
Due to the over-reserve problem, this part is still in RFC state. But the framework should mostly be fine, only needs extra fine-tuning to get more accurate qgroup rsv to avoid too early limit.Quoted 5 lines
Quoted 1 line
It's from patch 7 to 14.
I'm going to add the whole patchset to next, the first part has been there for some time and no test failures were reported. I optimistically expect that the second part will also be fine.
Quoted 6 lines
The type based reservation is completely fine, since it doesn't introduce anything new, just a preparation for the incoming meta rework.
However I prefer not to push the whole patchset to upstream until over-reserve behavior is solved.
Since it breaks quite some test cases with small limit.
Merging plan for this patchset from last week was to postpone until 4.18 due to lack of final testing here. I've tried to run this with quotas enabled an fstests that led to warnings in the power failure simulation tests.
As there's going to be one more rc, this gives us one more week to decide if its' ok-ish to merge this patch and fix the fallouts during the normal cycle.
Given that this patchset has been in for-next for a long time, I'd do the merge now and focus on testing for that patchset as the rest of devel patches looks good.
Please let me know if you have objections.
Quoted 25 lines · David Sterba
Quoted 24 lines · Qu Wenruo
Quoted 14 lines · David Sterba
Quoted 13 lines · Qu Wenruo
The patch is consist of 2 main parts:
1) Type based qgroup reservation
The original patchset is sent several months ago. Nothing is modified at all, just rebased. And not conflict at all.It's from patch 1 to patch 6.
2) Split meta qgroup reservation into per-trans and prealloc sub types The real work to address metadata underflow.
Due to the over-reserve problem, this part is still in RFC state. But the framework should mostly be fine, only needs extra fine-tuning to get more accurate qgroup rsv to avoid too early limit.Quoted 5 lines
Quoted 1 line
It's from patch 7 to 14.
I'm going to add the whole patchset to next, the first part has been there for some time and no test failures were reported. I optimistically expect that the second part will also be fine.
The type based reservation is completely fine, since it doesn't introduce anything new, just a preparation for the incoming meta rework.
Quoted 8 lines
Quoted 3 lines
However I prefer not to push the whole patchset to upstream until over-reserve behavior is solved.
Since it breaks quite some test cases with small limit.Merging plan for this patchset from last week was to postpone until 4.18 due to lack of final testing here. I've tried to run this with quotas enabled an fstests that led to warnings in the power failure simulation tests.
Which test case?
Quoted 10 lines
As there's going to be one more rc, this gives us one more week to decide if its' ok-ish to merge this patch and fix the fallouts during the normal cycle.
Given that this patchset has been in for-next for a long time, I'd do the merge now and focus on testing for that patchset as the rest of devel patches looks good.
Please let me know if you have objections.
I'm completely fine with the decision.
Thanks,
Qu
Quoted 1 line
Quoted 6 lines · Qu Wenruo
Quoted 4 lines
Merging plan for this patchset from last week was to postpone until 4.18 due to lack of final testing here. I've tried to run this with quotas enabled an fstests that led to warnings in the power failure simulation tests.
Which test case?
eg shared/298, the log is then flooded by this message
[153085.406444] ------------[ cut here ]------------ [153085.411426] qgroup 5 data reserved space underflow, have 32661504 to free 268435456 [153085.411584] WARNING: CPU: 7 PID: 22475 at fs/btrfs/qgroup.c:102 qgroup_rsv_release+0x1fa/0x210 [btrfs] [153085.499574] CPU: 7 PID: 22475 Comm: fio Tainted: G W 4.16.0-rc6-1.ge195904-vanilla+ #192 [153085.509186] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008 [153085.516008] RIP: 0010:qgroup_rsv_release+0x1fa/0x210 [btrfs] [153085.521899] RSP: 0018:ffffb1de01bf7c18 EFLAGS: 00010296 [153085.527367] RAX: 0000000000000047 RBX: ffff928ce4950c00 RCX: 0000000000000002 [153085.534799] RDX: 0000000000000007 RSI: 0000000000000001 RDI: 0000000000000202 [153085.542236] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [153085.549670] R10: 0000000000000000 R11: ffffb1de01bf7aa0 R12: ffff928ce4950c00 [153085.557106] R13: 0000000000000000 R14: 0000000010000000 R15: ffff928c3bb54000 [153085.564545] FS: 00007f90afdc5740(0000) GS:ffff928ce7400000(0000) knlGS:0000000000000000 [153085.572945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [153085.578925] CR2: 000000000076daa8 CR3: 00000001f86e0000 CR4: 00000000000006e0 [153085.586367] Call Trace: [153085.589112] btrfs_qgroup_free_refroot+0x174/0x2c0 [btrfs] [153085.594873] __btrfs_run_delayed_refs+0xb9e/0x17e0 [btrfs] [153085.600671] btrfs_run_delayed_refs+0x10d/0x1c0 [btrfs] [153085.606179] btrfs_commit_transaction+0x30/0xad0 [btrfs] [153085.611782] btrfs_sync_file+0x393/0x4f0 [btrfs] [153085.616652] do_fsync+0x38/0x60 [153085.620021] SyS_fsync+0xc/0x10 [153085.623402] do_syscall_64+0x71/0x1a0 [153085.627296] entry_SYSCALL_64_after_hwframe+0x42/0xb7 [153085.632590] RIP: 0033:0x7f90af0942cd [153085.636379] RSP: 002b:00007ffcb59a4c20 EFLAGS: 00000293 ORIG_RAX: 000000000000004a [153085.644249] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f90af0942cd [153085.651675] RDX: 0000000000001000 RSI: 00000000013e5960 RDI: 0000000000000004 [153085.659111] RBP: 0000000000000000 R08: 0000000000000008 R09: 9e37fffffffc0001 [153085.666549] R10: 000000000f75e468 R11: 0000000000000293 R12: 0000000000000000 [153085.673985] R13: 0000000006400000 R14: 00007f90968acfb8 R15: 00007f90adad4db0 [153085.700848] ---[ end trace 48a650f3d217fbac ]---
or generic/475
[69595.290236] WARNING: CPU: 1 PID: 3659 at fs/btrfs/extent-tree.c:124 btrfs_put_block_group+0x52/0x60 [btrfs] [69595.300499] CPU: 1 PID: 3659 Comm: umount Tainted: G W 4.16.0-rc6-1.ge195904-vanilla+ #192 [69595.300502] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008 [69595.300555] RIP: 0010:btrfs_put_block_group+0x52/0x60 [btrfs] [69595.300569] RSP: 0018:ffffb1de02d67dd0 EFLAGS: 00010286 [69595.393015] RAX: 0000000000000001 RBX: ffff928ca4b7dc00 RCX: 0000000000000002 [69595.393018] RDX: ffff928ca4b7ddd8 RSI: 0000000000000001 RDI: ffff928ca4b7dc00 [69595.393024] RBP: ffff928ca4b7dc00 R08: ffff928cd6c70e10 R09: 0000000000000000 [69595.420315] R10: 0000000000000000 R11: ffff928cd6c70e10 R12: ffff928c9c7640d0 [69595.420318] R13: ffff928c9c764000 R14: ffff928c9c764120 R15: 0000000000000000 [69595.420322] FS: 00007ff5e66a2840(0000) GS:ffff928ce6800000(0000) knlGS:0000000000000000 [69595.420328] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [69595.449136] CR2: 00007efdd6e6b594 CR3: 0000000211943000 CR4: 00000000000006e0 [69595.449140] Call Trace: [69595.449207] btrfs_free_block_groups+0x164/0x420 [btrfs] [69595.464610] close_ctree+0x127/0x2f0 [btrfs] [69595.469046] generic_shutdown_super+0x69/0x110 [69595.469054] kill_anon_super+0xe/0x20 [69595.469102] btrfs_kill_super+0x13/0x100 [btrfs] [69595.469114] deactivate_locked_super+0x39/0x70 [69595.469122] cleanup_mnt+0x3b/0x70 [69595.469128] task_work_run+0x80/0xb0 [69595.469139] exit_to_usermode_loop+0x71/0xa0 [69595.469148] do_syscall_64+0x157/0x1a0 [69595.469158] entry_SYSCALL_64_after_hwframe+0x42/0xb7 [69595.469164] RIP: 0033:0x7ff5e5f99d37 [69595.469167] RSP: 002b:00007ffeb054adf8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6 [69595.469172] RAX: 0000000000000000 RBX: 0000563360af5060 RCX: 00007ff5e5f99d37 [69595.469175] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000563360af52c0 [69595.469178] RBP: 0000563360af52c0 R08: 00007ff5e628b97d R09: 0000000000000005 [69595.469181] R10: 0000563360af5280 R11: 0000000000000246 R12: 00007ff5e649a1a4 [69595.469183] R13: 0000000000000000 R14: 0000000000000000 R15: 00000000ffffffff [69595.469339] ---[ end trace 48a650f3d217fba6 ]---
[69595.530603] WARNING: CPU: 1 PID: 3659 at fs/btrfs/extent-tree.c:9945 btrfs_free_block_groups+0x2cb/0x420 [btrfs] [69595.590550] CPU: 1 PID: 3659 Comm: umount Tainted: G W 4.16.0-rc6-1.ge195904-vanilla+ #192 [69595.590552] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008 [69595.590594] RIP: 0010:btrfs_free_block_groups+0x2cb/0x420 [btrfs] [69595.590614] RSP: 0018:ffffb1de02d67de0 EFLAGS: 00010286 [69595.683331] RAX: ffff928cb96e2908 RBX: 0000000000000000 RCX: 0000000000000002 [69595.683334] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff928cb96e0800 [69595.683339] RBP: ffff928cb96e7c00 R08: 0000000000000000 R09: 0000000000000000 [69595.710642] R10: 0000000000000000 R11: 0000000000000002 R12: ffff928cb96e2800 [69595.710645] R13: ffff928c9c764000 R14: ffff928cb96e2908 R15: ffff928c9c766b50 [69595.710649] FS: 00007ff5e66a2840(0000) GS:ffff928ce6800000(0000) knlGS:0000000000000000 [69595.710654] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [69595.739454] CR2: 00007efdd6e6b594 CR3: 0000000211943000 CR4: 00000000000006e0 [69595.739457] Call Trace: [69595.739524] close_ctree+0x127/0x2f0 [btrfs] [69595.753856] generic_shutdown_super+0x69/0x110 [69595.753864] kill_anon_super+0xe/0x20 [69595.753902] btrfs_kill_super+0x13/0x100 [btrfs] [69595.753912] deactivate_locked_super+0x39/0x70 [69595.771747] cleanup_mnt+0x3b/0x70 [69595.771755] task_work_run+0x80/0xb0 [69595.779135] exit_to_usermode_loop+0x71/0xa0 [69595.779144] do_syscall_64+0x157/0x1a0 [69595.787566] entry_SYSCALL_64_after_hwframe+0x42/0xb7 [69595.787572] RIP: 0033:0x7ff5e5f99d37 [69595.796590] RSP: 002b:00007ffeb054adf8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6 [69595.796596] RAX: 0000000000000000 RBX: 0000563360af5060 RCX: 00007ff5e5f99d37 [69595.796599] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000563360af52c0 [69595.796602] RBP: 0000563360af52c0 R08: 00007ff5e628b97d R09: 0000000000000005 [69595.796605] R10: 0000563360af5280 R11: 0000000000000246 R12: 00007ff5e649a1a4 [69595.796608] R13: 0000000000000000 R14: 0000000000000000 R15: 00000000ffffffff [69595.840989] ---[ end trace 48a650f3d217fba7 ]--- [69595.841079] BTRFS info (device dm-0): space_info 1 has 725651456 free, is not full [69595.874673] BTRFS info (device dm-0): space_info total=2155872256, used=1430224896, pinned=0, reserved=18446744073709547520, may_use=0, readonly=0
Quoted 45 lines · David Sterba
Quoted 6 lines · Qu Wenruo
Quoted 4 lines
Merging plan for this patchset from last week was to postpone until 4.18 due to lack of final testing here. I've tried to run this with quotas enabled an fstests that led to warnings in the power failure simulation tests.
Which test case?
eg shared/298, the log is then flooded by this message
[153085.406444] ------------[ cut here ]------------ [153085.411426] qgroup 5 data reserved space underflow, have 32661504 to free 268435456 [153085.411584] WARNING: CPU: 7 PID: 22475 at fs/btrfs/qgroup.c:102 qgroup_rsv_release+0x1fa/0x210 [btrfs] [153085.499574] CPU: 7 PID: 22475 Comm: fio Tainted: G W 4.16.0-rc6-1.ge195904-vanilla+ #192 [153085.509186] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008 [153085.516008] RIP: 0010:qgroup_rsv_release+0x1fa/0x210 [btrfs] [153085.521899] RSP: 0018:ffffb1de01bf7c18 EFLAGS: 00010296 [153085.527367] RAX: 0000000000000047 RBX: ffff928ce4950c00 RCX: 0000000000000002 [153085.534799] RDX: 0000000000000007 RSI: 0000000000000001 RDI: 0000000000000202 [153085.542236] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [153085.549670] R10: 0000000000000000 R11: ffffb1de01bf7aa0 R12: ffff928ce4950c00 [153085.557106] R13: 0000000000000000 R14: 0000000010000000 R15: ffff928c3bb54000 [153085.564545] FS: 00007f90afdc5740(0000) GS:ffff928ce7400000(0000) knlGS:0000000000000000 [153085.572945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [153085.578925] CR2: 000000000076daa8 CR3: 00000001f86e0000 CR4: 00000000000006e0 [153085.586367] Call Trace: [153085.589112] btrfs_qgroup_free_refroot+0x174/0x2c0 [btrfs] [153085.594873] __btrfs_run_delayed_refs+0xb9e/0x17e0 [btrfs] [153085.600671] btrfs_run_delayed_refs+0x10d/0x1c0 [btrfs] [153085.606179] btrfs_commit_transaction+0x30/0xad0 [btrfs] [153085.611782] btrfs_sync_file+0x393/0x4f0 [btrfs] [153085.616652] do_fsync+0x38/0x60 [153085.620021] SyS_fsync+0xc/0x10 [153085.623402] do_syscall_64+0x71/0x1a0 [153085.627296] entry_SYSCALL_64_after_hwframe+0x42/0xb7 [153085.632590] RIP: 0033:0x7f90af0942cd [153085.636379] RSP: 002b:00007ffcb59a4c20 EFLAGS: 00000293 ORIG_RAX: 000000000000004a [153085.644249] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f90af0942cd [153085.651675] RDX: 0000000000001000 RSI: 00000000013e5960 RDI: 0000000000000004 [153085.659111] RBP: 0000000000000000 R08: 0000000000000008 R09: 9e37fffffffc0001 [153085.666549] R10: 000000000f75e468 R11: 0000000000000293 R12: 0000000000000000 [153085.673985] R13: 0000000006400000 R14: 00007f90968acfb8 R15: 00007f90adad4db0 [153085.700848] ---[ end trace 48a650f3d217fbac ]---
Quoted 1 line
or generic/475
This one is totally unrelated with this patchset, as it happens without it and without qgroups enabled.
It happens very often actually, it's just some proper cleanup missing after some fatal error that lead to a transaction abort.
Quoted 73 lines
[69595.290236] WARNING: CPU: 1 PID: 3659 at fs/btrfs/extent-tree.c:124 btrfs_put_block_group+0x52/0x60 [btrfs] [69595.300499] CPU: 1 PID: 3659 Comm: umount Tainted: G W 4.16.0-rc6-1.ge195904-vanilla+ #192 [69595.300502] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008 [69595.300555] RIP: 0010:btrfs_put_block_group+0x52/0x60 [btrfs] [69595.300569] RSP: 0018:ffffb1de02d67dd0 EFLAGS: 00010286 [69595.393015] RAX: 0000000000000001 RBX: ffff928ca4b7dc00 RCX: 0000000000000002 [69595.393018] RDX: ffff928ca4b7ddd8 RSI: 0000000000000001 RDI: ffff928ca4b7dc00 [69595.393024] RBP: ffff928ca4b7dc00 R08: ffff928cd6c70e10 R09: 0000000000000000 [69595.420315] R10: 0000000000000000 R11: ffff928cd6c70e10 R12: ffff928c9c7640d0 [69595.420318] R13: ffff928c9c764000 R14: ffff928c9c764120 R15: 0000000000000000 [69595.420322] FS: 00007ff5e66a2840(0000) GS:ffff928ce6800000(0000) knlGS:0000000000000000 [69595.420328] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [69595.449136] CR2: 00007efdd6e6b594 CR3: 0000000211943000 CR4: 00000000000006e0 [69595.449140] Call Trace: [69595.449207] btrfs_free_block_groups+0x164/0x420 [btrfs] [69595.464610] close_ctree+0x127/0x2f0 [btrfs] [69595.469046] generic_shutdown_super+0x69/0x110 [69595.469054] kill_anon_super+0xe/0x20 [69595.469102] btrfs_kill_super+0x13/0x100 [btrfs] [69595.469114] deactivate_locked_super+0x39/0x70 [69595.469122] cleanup_mnt+0x3b/0x70 [69595.469128] task_work_run+0x80/0xb0 [69595.469139] exit_to_usermode_loop+0x71/0xa0 [69595.469148] do_syscall_64+0x157/0x1a0 [69595.469158] entry_SYSCALL_64_after_hwframe+0x42/0xb7 [69595.469164] RIP: 0033:0x7ff5e5f99d37 [69595.469167] RSP: 002b:00007ffeb054adf8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6 [69595.469172] RAX: 0000000000000000 RBX: 0000563360af5060 RCX: 00007ff5e5f99d37 [69595.469175] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000563360af52c0 [69595.469178] RBP: 0000563360af52c0 R08: 00007ff5e628b97d R09: 0000000000000005 [69595.469181] R10: 0000563360af5280 R11: 0000000000000246 R12: 00007ff5e649a1a4 [69595.469183] R13: 0000000000000000 R14: 0000000000000000 R15: 00000000ffffffff [69595.469339] ---[ end trace 48a650f3d217fba6 ]---[69595.530603] WARNING: CPU: 1 PID: 3659 at fs/btrfs/extent-tree.c:9945 btrfs_free_block_groups+0x2cb/0x420 [btrfs] [69595.590550] CPU: 1 PID: 3659 Comm: umount Tainted: G W 4.16.0-rc6-1.ge195904-vanilla+ #192 [69595.590552] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008 [69595.590594] RIP: 0010:btrfs_free_block_groups+0x2cb/0x420 [btrfs] [69595.590614] RSP: 0018:ffffb1de02d67de0 EFLAGS: 00010286 [69595.683331] RAX: ffff928cb96e2908 RBX: 0000000000000000 RCX: 0000000000000002 [69595.683334] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff928cb96e0800 [69595.683339] RBP: ffff928cb96e7c00 R08: 0000000000000000 R09: 0000000000000000 [69595.710642] R10: 0000000000000000 R11: 0000000000000002 R12: ffff928cb96e2800 [69595.710645] R13: ffff928c9c764000 R14: ffff928cb96e2908 R15: ffff928c9c766b50 [69595.710649] FS: 00007ff5e66a2840(0000) GS:ffff928ce6800000(0000) knlGS:0000000000000000 [69595.710654] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [69595.739454] CR2: 00007efdd6e6b594 CR3: 0000000211943000 CR4: 00000000000006e0 [69595.739457] Call Trace: [69595.739524] close_ctree+0x127/0x2f0 [btrfs] [69595.753856] generic_shutdown_super+0x69/0x110 [69595.753864] kill_anon_super+0xe/0x20 [69595.753902] btrfs_kill_super+0x13/0x100 [btrfs] [69595.753912] deactivate_locked_super+0x39/0x70 [69595.771747] cleanup_mnt+0x3b/0x70 [69595.771755] task_work_run+0x80/0xb0 [69595.779135] exit_to_usermode_loop+0x71/0xa0 [69595.779144] do_syscall_64+0x157/0x1a0 [69595.787566] entry_SYSCALL_64_after_hwframe+0x42/0xb7 [69595.787572] RIP: 0033:0x7ff5e5f99d37 [69595.796590] RSP: 002b:00007ffeb054adf8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6 [69595.796596] RAX: 0000000000000000 RBX: 0000563360af5060 RCX: 00007ff5e5f99d37 [69595.796599] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000563360af52c0 [69595.796602] RBP: 0000563360af52c0 R08: 00007ff5e628b97d R09: 0000000000000005 [69595.796605] R10: 0000563360af5280 R11: 0000000000000246 R12: 00007ff5e649a1a4 [69595.796608] R13: 0000000000000000 R14: 0000000000000000 R15: 00000000ffffffff [69595.840989] ---[ end trace 48a650f3d217fba7 ]--- [69595.841079] BTRFS info (device dm-0): space_info 1 has 725651456 free, is not full [69595.874673] BTRFS info (device dm-0): space_info total=2155872256, used=1430224896, pinned=0, reserved=18446744073709547520, may_use=0, readonly=0--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
Quoted 11 lines · Qu Wenruo
Quoted 9 lines
As there's going to be one more rc, this gives us one more week to decide if its' ok-ish to merge this patch and fix the fallouts during the normal cycle.
Given that this patchset has been in for-next for a long time, I'd do the merge now and focus on testing for that patchset as the rest of devel patches looks good.
Please let me know if you have objections.
I'm completely fine with the decision.
So, I've merged the patchset, updated a few comments or changelogs.
There was one fixup for btrfs_delalloc_release_space, one more call was added meanwhile (765f3cebff0023d05 "btrfs: Handle btrfs_set_extent_delalloc failure in relocate_file_extent_cluster"), the new parameter was set to true as it's an error handling kind of use.
The folloing types of patches will be accepted during the 4.17 development cycle that are related to this patchset: improved error handling, better reporting, tracpoint updates or fixes of bugs triggered by the reservation type split. Ie. no unnecessary cleanups or new functionality.