xfs: fix a couple sparse chunk alloc problems
v3 cover xfs: fix a couple sparse chunk alloc problems
Hi all,
Here's v3 of the series to fix the sparse chunk alloc shutdown. The original report is available here[1].
This is mostly the same fundamental idea as v2, but the implementation has been slightly reworked. Instead of creating and using a _minleft() freelist calculation variant, we rework the min freelist helper to calculate a min and a max for multi-alloc cases. The max is used appropriately based on allocations that set args->minleft.
Patches 1 and 2 are unchanged from v2. Patch 3 is new and refactors the semantics of xfs_alloc_min_freelist() as described above. Patch 4 is the same general fix as v2, but rather than create its own helper the max value is passed along with the min and they are used appropriately for length checks and available space calculations.
I've also since realized that XFS_DEBUG seems to be what defeats the custom reproducer, for whatever reason, so I can confirm this still survives that test on !DEBUG. Otherwise the series survives fstests without regression.
Thoughts, reviews, flames appreciated.
Brian
v3:
- Added patch 3 to refactor xfs_alloc_min_freelist() and calculate min/max.
- Updated patch 4 to use the max value for space available and longest extent checks.
v2: https://lore.kernel.org/linux-xfs/[email protected]/
- Reworked fix logic into allocator instead of sparse inode alloc specific.
- Dropped Fixes: tag since this is no longer directly correlated to sparse inodes.
v1: https://lore.kernel.org/linux-xfs/[email protected]/
[1] https://lore.kernel.org/linux-xfs/[email protected]/
Brian Foster (4):
xfs: set minleft correctly for sparse chunk errortag allocation
xfs: support additional levels in the agfl minimum calculation
xfs: calculate AGFL max to support multiple-alloc transactions
xfs: incorporate increased AGFL min requirement for minleft allocs
fs/xfs/libxfs/xfs_alloc.c | 106 +++++++++++++++++++++++++++---------- fs/xfs/libxfs/xfs_alloc.h | 7 +-- fs/xfs/libxfs/xfs_bmap.c | 7 ++- fs/xfs/libxfs/xfs_ialloc.c | 14 ++--- 4 files changed, 93 insertions(+), 41 deletions(-)
Signature · 1 line
2.55.0
PATCH v3 1/4 xfs: set minleft correctly for sparse chunk errortag allocation
The errortag instrumentation for forced sparse chunk allocation jumps straight to the allocation path without setting args.minleft. minleft is unconditionally set to ->inobt_maxlevels for the normal allocation path. Lift the assignment to the initial args setup so it covers all possible paths.
--- fs/xfs/libxfs/xfs_ialloc.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_ialloc.c +4 -6
@@ -733,6 +733,10 @@ xfs_ialloc_ag_alloc( igeo->maxicount)return -ENOSPC ; args.minlen = args.maxlen = igeo->ialloc_blks; + +/* Allow space for the inode btree to split. */ + args.minleft = igeo->inobt_maxlevels; +/* * First try to allocate inodes contiguous with the last-allocated * chunk of inodes. If the filesystem is striped, this will fill
@@ -764,8 +768,6 @@ xfs_ialloc_ag_alloc( args.alignment =1 ; args.minalignslop = igeo->cluster_align -1 ; -/* Allow space for the inode btree to split. */ - args.minleft = igeo->inobt_maxlevels; error =xfs_alloc_vextent_exact_bno (&args,xfs_agbno_to_fsb (pag, args.agbno));if (error)
@@ -804,10 +806,6 @@ xfs_ialloc_ag_alloc(* Allocate a fixed-size extent of inodes. */ args.prod =1 ; -/* -* Allow space for the inode btree to split. -*/ - args.minleft = igeo->inobt_maxlevels; error =xfs_alloc_vextent_near_bno (&args,xfs_agbno_to_fsb (pag,be32_to_cpu (agi->agi_root)));
Signature · 1 line
2.55.0
PATCH v3 2/4 xfs: support additional levels in the agfl minimum calculation
In reply to Brian Fosterxfs_alloc_min_freelist() calculates the worst case AGFL block requirement for a full split plus partial refill for each alloc btree. An upcoming patch needs to calculate the requirement for multiple level increases, so add an optional extra levels parameter and factor out a wrapper function for the common case of a single split. No functional changes.
--- fs/xfs/libxfs/xfs_alloc.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +23 -8
@@ -2439,17 +2439,22 @@ xfs_alloc_longest_free_extent(/* * Compute the minimum length of the AGFL in the given AG. If @pag is NULL, -* return the largest possible minimum length. +* return the largest possible minimum length. The base calculation accounts +* for a single full split per btree. @extra_levels adds additional split +* levels to compute the prospective AGFL requirement increase for +* multi-allocation transactions. */ -unsigned int -xfs_alloc_min_freelist ( +static unsigned int +__xfs_alloc_min_freelist (struct xfs_mount *mp, -struct xfs_perag *pag) +struct xfs_perag *pag, +unsigned int extra_levels) {/* AG btrees have at least 1 level. */ const unsigned int bno_level = pag ? pag->pagf_bno_level :1 ;const unsigned int cnt_level = pag ? pag->pagf_cnt_level :1 ;const unsigned int rmap_level = pag ? pag->pagf_rmap_level :1 ; +const unsigned int levels =1 + extra_levels;unsigned int min_free;ASSERT (mp->m_alloc_maxlevels >0 );
@@ -2476,15 +2481,25 @@ xfs_alloc_min_freelist(*/ /* space needed by-bno freespace btree */ - min_free =min (bno_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free =min (bno_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed by-size freespace btree */ - min_free +=min (cnt_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free +=min (cnt_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed reverse mapping used space btree */ -if (xfs_has_rmapbt (mp)) - min_free +=min (rmap_level +1 , mp->m_rmap_maxlevels) *2 -2 ; +if (xfs_has_rmapbt (mp)) { + min_free +=min (rmap_level + levels, + mp->m_rmap_maxlevels) *2 -2 ; + }return min_free; } +unsigned int +xfs_alloc_min_freelist ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,0 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation
Signature · 1 line
2.55.0
Re: [External] : [PATCH v3 2/4] xfs: support additional levels in the agfl minimum calculation
Quoted 10 lines · Brian Foster
xfs_alloc_min_freelist()calculates the worst case AGFL block requirement for a full split plus partial refill for each alloc btree. An upcoming patch needs to calculate the requirement for multiple level increases, so add an optional extra levels parameter and factor out a wrapper function for the common case of a single split. No functional changes.Assisted-by LLMSigned-off-by Brian Foster <[email protected]>Reviewed-by "Darrick J. Wong" <[email protected]>
you can also add
Quoted 65 lines
--- fs/xfs/libxfs/xfs_alloc.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-)fs/xfs/libxfs/xfs_alloc.c +23
-8@@ -2439,17 +2439,22 @@ xfs_alloc_longest_free_extent(/* * Compute the minimum length of the AGFL in the given AG. If @pag is NULL, -* return the largest possible minimum length. +* return the largest possible minimum length. The base calculation accounts +* for a single full split per btree. @extra_levels adds additional split +* levels to compute the prospective AGFL requirement increase for +* multi-allocation transactions. */ -unsigned int -xfs_alloc_min_freelist ( +static unsigned int +__xfs_alloc_min_freelist (struct xfs_mount *mp, -struct xfs_perag *pag) +struct xfs_perag *pag, +unsigned int extra_levels) {/* AG btrees have at least 1 level. */ const unsigned int bno_level = pag ? pag->pagf_bno_level :1 ;const unsigned int cnt_level = pag ? pag->pagf_cnt_level :1 ;const unsigned int rmap_level = pag ? pag->pagf_rmap_level :1 ; +const unsigned int levels =1 + extra_levels;unsigned int min_free;ASSERT (mp->m_alloc_maxlevels >0 );@@ -2476,15 +2481,25 @@ xfs_alloc_min_freelist(*/ /* space needed by-bno freespace btree */ - min_free =min (bno_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free =min (bno_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed by-size freespace btree */ - min_free +=min (cnt_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free +=min (cnt_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed reverse mapping used space btree */ -if (xfs_has_rmapbt (mp)) - min_free +=min (rmap_level +1 , mp->m_rmap_maxlevels) *2 -2 ; +if (xfs_has_rmapbt (mp)) { + min_free +=min (rmap_level + levels, + mp->m_rmap_maxlevels) *2 -2 ; + }return min_free; } +unsigned int +xfs_alloc_min_freelist ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,0 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation
PATCH v3 3/4 xfs: calculate AGFL max to support multiple-alloc transactions
In reply to Brian FosterRework xfs_alloc_min_freelist() to return an (optional) max value along with the historical min freelist value. The max value includes an extra level in the btree based calculation to account for btree splits in the first allocation of a multi-alloc sequence.
This value reflects the worst case AGFL requirement that can be expected across multiple allocations within a single AG and single transaction. For example, consider the case of allocating an inode chunk and then an inobt block on inode record insertion.
While here, rename the need variable in xfs_alloc_fix_freelist() for clarity with upcoming changes.
Note that no callers calculate the max AGFL value as of yet. No functional changes in this patch.
--- fs/xfs/libxfs/xfs_alloc.c | 35 ++++++++++++++++++++++------------- fs/xfs/libxfs/xfs_alloc.h | 4 ++-- fs/xfs/libxfs/xfs_bmap.c | 5 +++-- fs/xfs/libxfs/xfs_ialloc.c | 4 +++- 4 files changed, 30 insertions(+), 18 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +22 -13
@@ -2445,7 +2445,7 @@ xfs_alloc_longest_free_extent(* multi-allocation transactions. */ static unsigned int -__xfs_alloc_min_freelist ( +__xfs_alloc_freelist (struct xfs_mount *mp,struct xfs_perag *pag,unsigned int extra_levels)
@@ -2492,12 +2492,21 @@ __xfs_alloc_min_freelist(return min_free; } -unsigned int -xfs_alloc_min_freelist ( +/* +* Compute the minimum and maximum length of the AGFL in the given AG. The max +* value in this context refers to the max requirement the AG might see in a +* multi-alloc transaction. If @pag is NULL, return the largest possible values. +*/ +void +xfs_alloc_freelist (struct xfs_mount *mp, -struct xfs_perag *pag) +struct xfs_perag *pag, +unsigned int *min_free, +unsigned int *max_free) { -return __xfs_alloc_min_freelist (mp, pag,0 ); + *min_free =__xfs_alloc_freelist (mp, pag,0 ); +if (max_free) + *max_free =__xfs_alloc_freelist (mp, pag,1 ); }/*
@@ -2849,7 +2858,7 @@ xfs_alloc_fix_freelist(struct xfs_buf *agflbp =NULL ;struct xfs_alloc_arg targs;/* local allocation arguments */ xfs_agblock_t bno;/* freelist block */ -xfs_extlen_t need;/* total blocks needed in freelist */ +xfs_extlen_t min_free;/* total blocks needed in freelist */ int error =0 ;/* deferred ops (AGFL block frees) require permanent transactions */
@@ -2877,8 +2886,8 @@ xfs_alloc_fix_freelist( goto out_agbp_relse; } - need =xfs_alloc_min_freelist (mp, pag); -if (!xfs_alloc_space_available (args, need, alloc_flags | +xfs_alloc_freelist (mp, pag, &min_free,NULL ); +if (!xfs_alloc_space_available (args, min_free, alloc_flags |XFS_ALLOC_FLAG_CHECK )) goto out_agbp_relse;
@@ -2901,8 +2910,8 @@ xfs_alloc_fix_freelist(xfs_agfl_reset (tp, agbp, pag);/* If there isn't enough total space or single-extent, reject it. */ - need =xfs_alloc_min_freelist (mp, pag); -if (!xfs_alloc_space_available (args, need, alloc_flags)) +xfs_alloc_freelist (mp, pag, &min_free,NULL ); +if (!xfs_alloc_space_available (args, min_free, alloc_flags)) goto out_agbp_relse;if (IS_ENABLED (CONFIG_XFS_DEBUG ) && args->alloc_minlen_only) {
@@ -2944,7 +2953,7 @@ xfs_alloc_fix_freelist(else targs.oinfo =XFS_RMAP_OINFO_AG ;while (!(alloc_flags &XFS_ALLOC_FLAG_NOSHRINK ) && - pag->pagf_flcount > need) { + pag->pagf_flcount > min_free) { error =xfs_alloc_get_freelist (pag, tp, agbp, &bno,0 );if (error) goto out_agbp_relse;
@@ -2978,9 +2987,9 @@ xfs_alloc_fix_freelist( goto out_agbp_relse;/* Make the freelist longer if it's too short. */ -while (pag->pagf_flcount < need) { +while (pag->pagf_flcount < min_free) { targs.agbno =0 ; - targs.maxlen = need - pag->pagf_flcount; + targs.maxlen = min_free - pag->pagf_flcount; targs.resv =XFS_AG_RESV_AGFL ;/* Allocate as many blocks as possible at once. */
fs/xfs/libxfs/xfs_alloc.h +2 -2
@@ -71,8 +71,8 @@ unsigned int xfs_alloc_ag_max_usable(struct xfs_mount *mp);xfs_extlen_t xfs_alloc_longest_free_extent (struct xfs_perag *pag,xfs_extlen_t need,xfs_extlen_t reserved); -unsigned int xfs_alloc_min_freelist (struct xfs_mount *mp, -struct xfs_perag *pag); +void xfs_alloc_freelist (struct xfs_mount *mp,struct xfs_perag *pag, +unsigned int *min_free,unsigned int *max_free);int xfs_alloc_get_freelist (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agfbp,xfs_agblock_t *bnop,int btreeblk); int xfs_alloc_put_freelist(struct xfs_perag *pag, struct xfs_trans *tp,
fs/xfs/libxfs/xfs_bmap.c +3 -2
@@ -3150,6 +3150,7 @@ xfs_bmap_longest_free_extent( xfs_extlen_t *blen) {xfs_extlen_t longest; +unsigned int min_free;int error =0 ;if (!xfs_perag_initialised_agf (pag)) {
@@ -3159,8 +3160,8 @@ xfs_bmap_longest_free_extent(return error; } - longest =xfs_alloc_longest_free_extent (pag, -xfs_alloc_min_freelist (pag_mount (pag), pag), +xfs_alloc_freelist (pag_mount (pag), pag, &min_free,NULL ); + longest =xfs_alloc_longest_free_extent (pag, min_free,xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;
fs/xfs/libxfs/xfs_ialloc.c +3 -1
@@ -3074,6 +3074,7 @@ xfs_ialloc_calc_rootino( {struct xfs_ino_geometry *igeo =M_IGEO (mp);xfs_agblock_t first_bno; +unsigned int min_free;/* * Pre-calculate the geometry of AG 0. We know what it looks like
@@ -3092,7 +3093,8 @@ xfs_ialloc_calc_rootino( first_bno +=1 ;/* ...the initial AGFL... */ - first_bno +=xfs_alloc_min_freelist (mp,NULL ); +xfs_alloc_freelist (mp,NULL , &min_free,NULL ); + first_bno += min_free;/* ...the free inode btree root... */ if (xfs_has_finobt (mp))
Signature · 1 line
2.55.0
Re: [External] : [PATCH v3 3/4] xfs: calculate AGFL max to support multiple-alloc transactions
Quoted 16 lines · Brian Foster
Rework
xfs_alloc_min_freelist()to return an (optional) max value along with the historical min freelist value. The max value includes an extra level in the btree based calculation to account for btree splits in the first allocation of a multi-alloc sequence.This value reflects the worst case AGFL requirement that can be expected across multiple allocations within a single AG and single transaction. For example, consider the case of allocating an inode chunk and then an inobt block on inode record insertion.
While here, rename the need variable in
xfs_alloc_fix_freelist()for clarity with upcoming changes.Note that no callers calculate the max AGFL value as of yet. No functional changes in this patch.
Quoted 9 lines
Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 35 ++++++++++++++++++++++------------- fs/xfs/libxfs/xfs_alloc.h | 4 ++-- fs/xfs/libxfs/xfs_bmap.c | 5 +++-- fs/xfs/libxfs/xfs_ialloc.c | 4 +++- 4 files changed, 30 insertions(+), 18 deletions(-)
looks good
PATCH v3 4/4 xfs: incorporate increased AGFL min requirement for minleft allocs
In reply to Brian FosterMatt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by xfs_alloc_fix_freelist() is:
free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7
... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3
This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs. The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement.
To address this, first have xfs_alloc_freelist() return both min and max freelist values. The min value is the current AGFL requirement and remains used for actual AGFL sizing. The max value calculates the worst case AGFL requirement after potential allocbt splits during the current allocation. Incorporate the max value into space availability checks for AG selection and the longest free extent calculation. The latter is necessary because callers like the bmap layer can size allocation requests based on the longest free extent. Without this, aligned allocs can end up oversized, prematurely fail, and fall back to non-aligned to make up the difference.
This ensures the selected AG has enough blocks for both the caller's minleft value and the worst case AGFL increase. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the inode allocation fails gracefully with -ENOSPC.
--- fs/xfs/libxfs/xfs_alloc.c | 58 +++++++++++++++++++++++++++------------ fs/xfs/libxfs/xfs_alloc.h | 3 +- fs/xfs/libxfs/xfs_bmap.c | 6 ++-- 3 files changed, 47 insertions(+), 20 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +41 -17
@@ -2397,31 +2397,36 @@ xfs_alloc_compute_maxlevels( }/* -* Find the length of the longest extent in an AG. The 'need' parameter -* specifies how much space we're going to need for the AGFL and the -* 'reserved' parameter tells us how many blocks in this AG are reserved for +* Find the length of the longest extent in an AG. The @min_free and @max_free +* parameters specify how much space we're going to need for the AGFL and the +* @reserved parameter tells us how many blocks in this AG are reserved for * other callers. */ xfs_extlen_t xfs_alloc_longest_free_extent (struct xfs_perag *pag, -xfs_extlen_t need, +xfs_extlen_t min_free, +xfs_extlen_t max_free,xfs_extlen_t reserved) {xfs_extlen_t delta =0 ;/* -* If the AGFL needs a recharge, we'll have to subtract that from the -* longest extent. +* If the AGFL needs a recharge, subtract that from the longest extent +* because AGFL refill happens before the alloc. */ -if (need > pag->pagf_flcount) - delta = need - pag->pagf_flcount; +if (min_free > pag->pagf_flcount) + delta = min_free - pag->pagf_flcount;/* -* If we cannot maintain others' reservations with space from the -* not-longest freesp extents, we'll have to subtract /that/ from -* the longest extent too. +* Extra AGFL blocks beyond the min are reserved by ->minleft during +* allocation. Similar to reserved, these blocks are not available to +* this allocation. Check if we can preserve the combined total without +* the longest extent. If not, deduct the necessary blocks from the +* longest extent. */ +if (max_free > min_free) + reserved += max_free - min_free;if (pag->pagf_freeblks - pag->pagf_longest < reserved) delta += reserved - (pag->pagf_freeblks - pag->pagf_longest);
@@ -2519,6 +2524,7 @@ static boolxfs_alloc_space_available ( struct xfs_alloc_arg *args, xfs_extlen_t min_free, + xfs_extlen_t max_free, int flags) { struct xfs_perag *pag = args->pag;
@@ -2526,15 +2532,32 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true; reservation =xfs_ag_resv_needed (pag, args->resv); +/* +* minleft implies a multi-alloc transaction. If set, the first alloc +* might cause btree splits that increase the AGFL requirement for the +* next. This worst case requirement is calculated in max_free. +* +* We don't prepopulate the AGFL because we don't know in advance if +* splits will occur. Instead, add the delta to minleft so it is +* accounted for in AG selection. This ensures the AG has enough space +* for the caller's minleft plus that needed to repopulate the AGFL on +* the next alloc if splits do occur. +*/ + minleft = args->minleft; +if (minleft) + minleft += max_free - min_free; +/* do we have enough contiguous free space for the allocation? */ alloc_len = args->minlen + (args->alignment -1 ) + args->minalignslop; - longest =xfs_alloc_longest_free_extent (pag, min_free, reservation); + longest =xfs_alloc_longest_free_extent (pag, min_free, + minleft ? max_free : min_free, reservation);if (longest < alloc_len)return false;
@@ -2545,7 +2568,7 @@ xfs_alloc_space_available(*/ agflcount =min_t (xfs_extlen_t, pag->pagf_flcount, min_free); available = (int )(pag->pagf_freeblks + agflcount - - reservation - min_free - args->minleft); + reservation - min_free - minleft);if (available < (int )max (args->total, alloc_len))return false;
@@ -2859,6 +2882,7 @@ xfs_alloc_fix_freelist(struct xfs_alloc_arg targs;/* local allocation arguments */ xfs_agblock_t bno;/* freelist block */ xfs_extlen_t min_free;/* total blocks needed in freelist */ +xfs_extlen_t max_free;/* max freelist requirement */ int error =0 ;/* deferred ops (AGFL block frees) require permanent transactions */
@@ -2886,8 +2910,8 @@ xfs_alloc_fix_freelist( goto out_agbp_relse; } -xfs_alloc_freelist (mp, pag, &min_free,NULL ); -if (!xfs_alloc_space_available (args, min_free, alloc_flags | +xfs_alloc_freelist (mp, pag, &min_free, &max_free); +if (!xfs_alloc_space_available (args, min_free, max_free, alloc_flags |XFS_ALLOC_FLAG_CHECK )) goto out_agbp_relse;
@@ -2910,8 +2934,8 @@ xfs_alloc_fix_freelist(xfs_agfl_reset (tp, agbp, pag);/* If there isn't enough total space or single-extent, reject it. */ -xfs_alloc_freelist (mp, pag, &min_free,NULL ); -if (!xfs_alloc_space_available (args, min_free, alloc_flags)) +xfs_alloc_freelist (mp, pag, &min_free, &max_free); +if (!xfs_alloc_space_available (args, min_free, max_free, alloc_flags)) goto out_agbp_relse;if (IS_ENABLED (CONFIG_XFS_DEBUG ) && args->alloc_minlen_only) {
fs/xfs/libxfs/xfs_alloc.h +2 -1
@@ -70,7 +70,8 @@ unsigned int xfs_alloc_set_aside(struct xfs_mount *mp);unsigned int xfs_alloc_ag_max_usable (struct xfs_mount *mp);xfs_extlen_t xfs_alloc_longest_free_extent (struct xfs_perag *pag, -xfs_extlen_t need,xfs_extlen_t reserved); +xfs_extlen_t min_free,xfs_extlen_t max_free, +xfs_extlen_t reserved);void xfs_alloc_freelist (struct xfs_mount *mp,struct xfs_perag *pag,unsigned int *min_free,unsigned int *max_free); int xfs_alloc_get_freelist(struct xfs_perag *pag, struct xfs_trans *tp,
fs/xfs/libxfs/xfs_bmap.c +4 -2
@@ -3151,6 +3151,7 @@ xfs_bmap_longest_free_extent( {xfs_extlen_t longest;unsigned int min_free; +unsigned int max_free;int error =0 ;if (!xfs_perag_initialised_agf (pag)) {
@@ -3160,8 +3161,9 @@ xfs_bmap_longest_free_extent(return error; } -xfs_alloc_freelist (pag_mount (pag), pag, &min_free,NULL ); - longest =xfs_alloc_longest_free_extent (pag, min_free, +/* bmap allocs always have minleft set, so account for max_free */ +xfs_alloc_freelist (pag_mount (pag), pag, &min_free, &max_free); + longest =xfs_alloc_longest_free_extent (pag, min_free, max_free,xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;
Signature · 1 line
2.55.0
Re: [External] : [PATCH v3 4/4] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 105 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by
xfs_alloc_fix_freelist()is:free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs. The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement.
To address this, first have
xfs_alloc_freelist()return both min and max freelist values. The min value is the current AGFL requirement and remains used for actual AGFL sizing. The max value calculates the worst case AGFL requirement after potential allocbt splits during the current allocation. Incorporate the max value into space availability checks for AG selection and the longest free extent calculation. The latter is necessary because callers like the bmap layer can size allocation requests based on the longest free extent. Without this, aligned allocs can end up oversized, prematurely fail, and fall back to non-aligned to make up the difference.This ensures the selected AG has enough blocks for both the caller's minleft value and the worst case AGFL increase. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the inode allocation fails gracefully with
-ENOSPC.Assisted-by LLMReported-by Matt Fleming <[email protected]>Signed-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 58 +++++++++++++++++++++++++++------------ fs/xfs/libxfs/xfs_alloc.h | 3 +- fs/xfs/libxfs/xfs_bmap.c | 6 ++-- 3 files changed, 47 insertions(+), 20 deletions(-)fs/xfs/libxfs/xfs_alloc.c +9
-11@@ -2397,31 +2397,36 @@ xfs_alloc_compute_maxlevels( }/* -* Find the length of the longest extent in an AG. The 'need' parameter -* specifies how much space we're going to need for the AGFL and the -* 'reserved' parameter tells us how many blocks in this AG are reserved for +* Find the length of the longest extent in an AG. The @min_free and @max_free +* parameters specify how much space we're going to need for the AGFL and the +* @reserved parameter tells us how many blocks in this AG are reserved for * other callers. */ xfs_extlen_t xfs_alloc_longest_free_extent (struct xfs_perag *pag, -xfs_extlen_t need, +xfs_extlen_t min_free, +xfs_extlen_t max_free,xfs_extlen_t reserved) {xfs_extlen_t delta =0 ;/* -* If the AGFL needs a recharge, we'll have to subtract that from the -* longest extent. +* If the AGFL needs a recharge, subtract that from the longest extent +* because AGFL refill happens before the alloc. */ -if (need > pag->pagf_flcount) - delta = need - pag->pagf_flcount; +if (min_free > pag->pagf_flcount) + delta = min_free - pag->pagf_flcount;/* -* If we cannot maintain others' reservations with space from the -* not-longest freesp extents, we'll have to subtract /that/ from -* the longest extent too.
fs/xfs/libxfs/xfs_alloc.c xfs_alloc_compute_maxlevels( · line 2431
Quoted 10 lines
fs/xfs/libxfs/xfs_alloc.c +7
-0+* Extra AGFL blocks beyond the min are reserved by ->minleft during +* allocation. Similar to reserved, these blocks are not available to +* this allocation. Check if we can preserve the combined total without +* the longest extent. If not, deduct the necessary blocks from the +* longest extent. */ +if (max_free > min_free) + reserved += max_free - min_free;if (pag->pagf_freeblks - pag->pagf_longest < reserved) delta += reserved - (pag->pagf_freeblks - pag->pagf_longest);
The only concern that I have is we have to assume the caller calculated the max_free. But ....