xfs: fix a couple sparse chunk alloc problems
v2 cover xfs: fix a couple sparse chunk alloc problems
Hi all,
Here's v2 of the series to fix the sparse chunk alloc shutdown. The original report is available here[1]. After some discussion on v1, this takes a slightly different approach to treat this as an allocator flaw. The solution is essentially to add the additional min_free block requirement to minleft for allocs where minleft != 0. See the commit log for patch 3 for further detail.
Patch 1 is the same DEBUG mode bug fix from v1. Patch 2 refactors the agfl min free helper for use in patch 3. Patch 3 makes the actual logic fix.
This survives some fstests runs without regression. For some reason the sparse inode reproducer is failing to work properly on the current baseline (can't create the requisite fs state) so I can't reconfirm that, but I don't expect anything would have changed there.
Thoughts, reviews, flames appreciated.
Brian
v2:
- Reworked fix logic into allocator instead of sparse inode alloc specific.
- Dropped Fixes: tag since this is no longer directly correlated to sparse inodes.
v1: https://lore.kernel.org/linux-xfs/[email protected]/
[1] https://lore.kernel.org/linux-xfs/[email protected]/
Brian Foster (3):
xfs: set minleft correctly for sparse chunk errortag allocation
xfs: support additional levels in the agfl minimum calculation
xfs: incorporate increased AGFL min requirement for minleft allocs
fs/xfs/libxfs/xfs_alloc.c | 60 ++++++++++++++++++++++++++++++++------ fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- fs/xfs/libxfs/xfs_ialloc.c | 10 +++---- 4 files changed, 58 insertions(+), 16 deletions(-)
Signature · 1 line
2.55.0
PATCH v2 1/3 xfs: set minleft correctly for sparse chunk errortag allocation
The errortag instrumentation for forced sparse chunk allocation jumps straight to the allocation path without setting args.minleft. minleft is unconditionally set to ->inobt_maxlevels for the normal allocation path. Lift the assignment to the initial args setup so it covers all possible paths.
--- fs/xfs/libxfs/xfs_ialloc.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
fs/xfs/libxfs/xfs_ialloc.c +4 -6
@@ -733,6 +733,10 @@ xfs_ialloc_ag_alloc( igeo->maxicount)return -ENOSPC ; args.minlen = args.maxlen = igeo->ialloc_blks; + +/* Allow space for the inode btree to split. */ + args.minleft = igeo->inobt_maxlevels; +/* * First try to allocate inodes contiguous with the last-allocated * chunk of inodes. If the filesystem is striped, this will fill
@@ -764,8 +768,6 @@ xfs_ialloc_ag_alloc( args.alignment =1 ; args.minalignslop = igeo->cluster_align -1 ; -/* Allow space for the inode btree to split. */ - args.minleft = igeo->inobt_maxlevels; error =xfs_alloc_vextent_exact_bno (&args,xfs_agbno_to_fsb (pag, args.agbno));if (error)
@@ -804,10 +806,6 @@ xfs_ialloc_ag_alloc(* Allocate a fixed-size extent of inodes. */ args.prod =1 ; -/* -* Allow space for the inode btree to split. -*/ - args.minleft = igeo->inobt_maxlevels; error =xfs_alloc_vextent_near_bno (&args,xfs_agbno_to_fsb (pag,be32_to_cpu (agi->agi_root)));
Signature · 1 line
2.55.0
Re: [PATCH v2 1/3] xfs: set minleft correctly for sparse chunk errortag allocation
Quoted 10 lines · Brian Foster
The errortag instrumentation for forced sparse chunk allocation jumps straight to the allocation path without setting args.minleft. minleft is unconditionally set to ->inobt_maxlevels for the normal allocation path. Lift the assignment to the initial args setup so it covers all possible paths.
Assisted-by LLMFixes 1cdadee11f8d ("xfs: randomly do sparse inode allocations in DEBUG mode")Signed-off-by Brian Foster <[email protected]>Reviewed-by Mark Tinguely <[email protected]>
This makes sense to me;
--D
Quoted 43 lines
--- fs/xfs/libxfs/xfs_ialloc.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)fs/xfs/libxfs/xfs_ialloc.c +4
-6@@ -733,6 +733,10 @@ xfs_ialloc_ag_alloc( igeo->maxicount)return -ENOSPC ; args.minlen = args.maxlen = igeo->ialloc_blks; + +/* Allow space for the inode btree to split. */ + args.minleft = igeo->inobt_maxlevels; +/* * First try to allocate inodes contiguous with the last-allocated * chunk of inodes. If the filesystem is striped, this will fill @@ -764,8 +768,6 @@ xfs_ialloc_ag_alloc( args.alignment =1 ; args.minalignslop = igeo->cluster_align -1 ; -/* Allow space for the inode btree to split. */ - args.minleft = igeo->inobt_maxlevels; error =xfs_alloc_vextent_exact_bno (&args,xfs_agbno_to_fsb (pag, args.agbno));if (error)@@ -804,10 +806,6 @@ xfs_ialloc_ag_alloc(* Allocate a fixed-size extent of inodes. */ args.prod =1 ; -/* -* Allow space for the inode btree to split. -*/ - args.minleft = igeo->inobt_maxlevels; error =xfs_alloc_vextent_near_bno (&args,xfs_agbno_to_fsb (pag,be32_to_cpu (agi->agi_root)));Signature · 1 line
2.55.0
Re: [PATCH v2 1/3] xfs: set minleft correctly for sparse chunk errortag allocation
In reply to Brian FosterQuoted 19 lines · Brian Foster
The errortag instrumentation for forced sparse chunk allocation jumps straight to the allocation path without setting args.minleft. minleft is unconditionally set to ->inobt_maxlevels for the normal allocation path. Lift the assignment to the initial args setup so it covers all possible paths.
Assisted-by LLMFixes 1cdadee11f8d ("xfs: randomly do sparse inode allocations in DEBUG mode")Signed-off-by Brian Foster <[email protected]>Reviewed-by Mark Tinguely <[email protected]>--- fs/xfs/libxfs/xfs_ialloc.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)fs/xfs/libxfs/xfs_ialloc.c +0
-0
fs/xfs/libxfs/xfs_ialloc.c xfs_ialloc_ag_alloc( · line 738
Quoted 6 lines
fs/xfs/libxfs/xfs_ialloc.c +3
-0@@ -733,6 +733,10 @@ xfs_ialloc_ag_alloc( igeo->maxicount)return -ENOSPC ; args.minlen = args.maxlen = igeo->ialloc_blks; + +/* Allow space for the inode btree to split. */ + args.minleft = igeo->inobt_maxlevels;
As an extra question: is that reservation even correct? We can split both the inobt and the finobt on insert, and this only reserves space for a inobt split....
Secondly, why always reserve space for a max level split? The amount we need is depedent on the current level of the btree, and for all other btree types our reservations are based on the current level. i.e. we know the math needed to make this dependent on current btree levels, and we know that we are inserting into multiple btrees of different heights here, so maybe we should fix this reservation whilst we are here, too?
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 1/3] xfs: set minleft correctly for sparse chunk errortag allocation
Quoted 15 lines · Dave Chinner
Quoted 14 lines · Brian Foster
The errortag instrumentation for forced sparse chunk allocation jumps straight to the allocation path without setting args.minleft. minleft is unconditionally set to ->inobt_maxlevels for the normal allocation path. Lift the assignment to the initial args setup so it covers all possible paths.
Assisted-by LLMFixes 1cdadee11f8d ("xfs: randomly do sparse inode allocations in DEBUG mode")Signed-off-by Brian Foster <[email protected]>Reviewed-by Mark Tinguely <[email protected]>--- fs/xfs/libxfs/xfs_ialloc.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
Quoted 16 lines
fs/xfs/libxfs/xfs_ialloc.c xfs_ialloc_ag_alloc( · line 738
Quoted 11 lines
fs/xfs/libxfs/xfs_ialloc.c +3
-0@@ -733,6 +733,10 @@ xfs_ialloc_ag_alloc( igeo->maxicount)return -ENOSPC ; args.minlen = args.maxlen = igeo->ialloc_blks; + +/* Allow space for the inode btree to split. */ + args.minleft = igeo->inobt_maxlevels;As an extra question: is that reservation even correct? We can split both the inobt and the finobt on insert, and this only reserves space for a inobt split....
I think I mentioned in one of the earlier postings that finobt is covered by the metadata perag res, so minleft isn't necessary. One of the longer term ideas worth considering is to perhaps revisit why inobt blocks aren't reserved and see if it's worth doing that.
Quoted 8 lines
Secondly, why always reserve space for a max level split? The amount we need is depedent on the current level of the btree, and for all other btree types our reservations are based on the current level. i.e. we know the math needed to make this dependent on current btree levels, and we know that we are inserting into multiple btrees of different heights here, so maybe we should fix this reservation whilst we are here, too?
I thought the reason we do this historically is because we haven't selected an AG at this point, but that could be wrong. The current code looks like it has the AGI buffer at this point at least, so technically I think we could tailor minleft to the current AG if we wanted to.
That said, I'm skeptical of the value for one.. how many blocks would we expect to make available in the best case? Enough for an extra chunk or two? Second, I really don't want to turn this into a squeeze more blocks out of -ENOSPC exercise so if we do want to dig into something like this, I'd prefer to defer it to a follow on series.
Brian
Quoted 5 lines
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v2 2/3 xfs: support additional levels in the agfl minimum calculation
In reply to Brian Fosterxfs_alloc_min_freelist() calculates the worst case AGFL block requirement for a full split plus partial refill for each alloc btree. An upcoming patch needs to calculate the requirement for multiple level increases, so add an optional extra levels parameter and factor out a wrapper function for the common case of a single split. No functional changes.
--- fs/xfs/libxfs/xfs_alloc.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +23 -8
@@ -2439,17 +2439,22 @@ xfs_alloc_longest_free_extent(/* * Compute the minimum length of the AGFL in the given AG. If @pag is NULL, -* return the largest possible minimum length. +* return the largest possible minimum length. The base calculation accounts +* for a single full split per btree. @extra_levels adds additional split +* levels to compute the prospective AGFL requirement increase for +* multi-allocation transactions. */ -unsigned int -xfs_alloc_min_freelist ( +static unsigned int +__xfs_alloc_min_freelist (struct xfs_mount *mp, -struct xfs_perag *pag) +struct xfs_perag *pag, +unsigned int extra_levels) {/* AG btrees have at least 1 level. */ const unsigned int bno_level = pag ? pag->pagf_bno_level :1 ;const unsigned int cnt_level = pag ? pag->pagf_cnt_level :1 ;const unsigned int rmap_level = pag ? pag->pagf_rmap_level :1 ; +const unsigned int levels =1 + extra_levels;unsigned int min_free;ASSERT (mp->m_alloc_maxlevels >0 );
@@ -2476,15 +2481,25 @@ xfs_alloc_min_freelist(*/ /* space needed by-bno freespace btree */ - min_free =min (bno_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free =min (bno_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed by-size freespace btree */ - min_free +=min (cnt_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free +=min (cnt_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed reverse mapping used space btree */ -if (xfs_has_rmapbt (mp)) - min_free +=min (rmap_level +1 , mp->m_rmap_maxlevels) *2 -2 ; +if (xfs_has_rmapbt (mp)) { + min_free +=min (rmap_level + levels, + mp->m_rmap_maxlevels) *2 -2 ; + }return min_free; } +unsigned int +xfs_alloc_min_freelist ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,0 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation
Signature · 1 line
2.55.0
Re: [PATCH v2 2/3] xfs: support additional levels in the agfl minimum calculation
Quoted 9 lines · Brian Foster
xfs_alloc_min_freelist()calculates the worst case AGFL block requirement for a full split plus partial refill for each alloc btree. An upcoming patch needs to calculate the requirement for multiple level increases, so add an optional extra levels parameter and factor out a wrapper function for the common case of a single split. No functional changes.Assisted-by LLMSigned-off-by Brian Foster <[email protected]>
This looks like a simple enough hoist, so I'll move on to the next patch to see how it's used. In the meantime,
--D
Quoted 69 lines
--- fs/xfs/libxfs/xfs_alloc.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-)fs/xfs/libxfs/xfs_alloc.c +23
-8@@ -2439,17 +2439,22 @@ xfs_alloc_longest_free_extent(/* * Compute the minimum length of the AGFL in the given AG. If @pag is NULL, -* return the largest possible minimum length. +* return the largest possible minimum length. The base calculation accounts +* for a single full split per btree. @extra_levels adds additional split +* levels to compute the prospective AGFL requirement increase for +* multi-allocation transactions. */ -unsigned int -xfs_alloc_min_freelist ( +static unsigned int +__xfs_alloc_min_freelist (struct xfs_mount *mp, -struct xfs_perag *pag) +struct xfs_perag *pag, +unsigned int extra_levels) {/* AG btrees have at least 1 level. */ const unsigned int bno_level = pag ? pag->pagf_bno_level :1 ;const unsigned int cnt_level = pag ? pag->pagf_cnt_level :1 ;const unsigned int rmap_level = pag ? pag->pagf_rmap_level :1 ; +const unsigned int levels =1 + extra_levels;unsigned int min_free;ASSERT (mp->m_alloc_maxlevels >0 );@@ -2476,15 +2481,25 @@ xfs_alloc_min_freelist(*/ /* space needed by-bno freespace btree */ - min_free =min (bno_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free =min (bno_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed by-size freespace btree */ - min_free +=min (cnt_level +1 , mp->m_alloc_maxlevels) *2 -2 ; + min_free +=min (cnt_level + levels, mp->m_alloc_maxlevels) *2 -2 ;/* space needed reverse mapping used space btree */ -if (xfs_has_rmapbt (mp)) - min_free +=min (rmap_level +1 , mp->m_rmap_maxlevels) *2 -2 ; +if (xfs_has_rmapbt (mp)) { + min_free +=min (rmap_level + levels, + mp->m_rmap_maxlevels) *2 -2 ; + }return min_free; } +unsigned int +xfs_alloc_min_freelist ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,0 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation Signature · 1 line
2.55.0
PATCH v2 3/3 xfs: incorporate increased AGFL min requirement for minleft allocs
In reply to Brian FosterMatt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by xfs_alloc_fix_freelist() is:
free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7
... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3
This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs.
The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement. To address this, create a variant of the AGFL min free calculation for minleft allocations that incorporates an additional allocbt level increase.
We do not add the additional blocks directly to min_free because this would lead to spurious AGFL block allocations and frees in the common case (i.e. no btree splits). Instead, add the surplus block requirement to the minleft value used to select the AG. This ensures the AG has enough blocks for the caller's minleft value plus the worst case increase in the AGFL. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the sparse inode allocation fails gracefully with -ENOSPC.
--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)
fs/xfs/libxfs/xfs_alloc.c +28 -1
@@ -2500,6 +2500,20 @@ xfs_alloc_min_freelist(return __xfs_alloc_min_freelist (mp, pag,0 ); } +/* +* Return the minimum freelist requirement considering a potential allocbt split +* from the current allocation. Use this when computing longest free extent for +* allocations with minleft set to ensure that the available extent length +* accounts for the subsequent allocation's increased AGFL requirement. +*/ +unsigned int +xfs_alloc_min_freelist_minleft ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,1 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation
@@ -2517,6 +2531,7 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true;
@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that +* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + } + agflcount =min_t (xfs_extlen_t, pag->pagf_flcount, min_free); available = (int )(pag->pagf_freeblks + agflcount - - reservation - min_free - args->minleft); + reservation - min_free - minleft);if (available < (int )max (args->total, alloc_len))return false;
fs/xfs/libxfs/xfs_alloc.h +2 -0
@@ -73,6 +73,8 @@ xfs_extlen_t xfs_alloc_longest_free_extent(struct xfs_perag *pag, xfs_extlen_t need, xfs_extlen_t reserved);unsigned int xfs_alloc_min_freelist (struct xfs_mount *mp,struct xfs_perag *pag); +unsigned int xfs_alloc_min_freelist_minleft (struct xfs_mount *mp, +struct xfs_perag *pag);int xfs_alloc_get_freelist (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agfbp,xfs_agblock_t *bnop,int btreeblk); int xfs_alloc_put_freelist(struct xfs_perag *pag, struct xfs_trans *tp,
fs/xfs/libxfs/xfs_bmap.c +1 -1
@@ -3160,7 +3160,7 @@ xfs_bmap_longest_free_extent( } longest =xfs_alloc_longest_free_extent (pag, -xfs_alloc_min_freelist (pag_mount (pag), pag), +xfs_alloc_min_freelist_minleft (pag_mount (pag), pag),xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;
Signature · 1 line
2.55.0
Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 136 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by
xfs_alloc_fix_freelist()is:free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs.
The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement. To address this, create a variant of the AGFL min free calculation for minleft allocations that incorporates an additional allocbt level increase.
We do not add the additional blocks directly to
min_freebecause this would lead to spurious AGFL block allocations and frees in the common case (i.e. no btree splits). Instead, add the surplus block requirement to the minleft value used to select the AG. This ensures the AG has enough blocks for the caller's minleft value plus the worst case increase in the AGFL. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the sparse inode allocation fails gracefully with-ENOSPC.Reported-by Matt Fleming <[email protected]>Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_alloc.c +28
-1@@ -2500,6 +2500,20 @@ xfs_alloc_min_freelist(return __xfs_alloc_min_freelist (mp, pag,0 ); } +/* +* Return the minimum freelist requirement considering a potential allocbt split +* from the current allocation. Use this when computing longest free extent for +* allocations with minleft set to ensure that the available extent length +* accounts for the subsequent allocation's increased AGFL requirement. +*/ +unsigned int +xfs_alloc_min_freelist_minleft ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,1 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation @@ -2517,6 +2531,7 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true;@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that +* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + } + agflcount =min_t (xfs_extlen_t, pag->pagf_flcount, min_free); available = (int )(pag->pagf_freeblks + agflcount - - reservation - min_free - args->minleft); + reservation - min_free - minleft);if (available < (int )max (args->total, alloc_len))return false;fs/xfs/libxfs/xfs_alloc.h +2
-0@@ -73,6 +73,8 @@ xfs_extlen_t xfs_alloc_longest_free_extent(struct xfs_perag *pag, xfs_extlen_t need, xfs_extlen_t reserved);unsigned int xfs_alloc_min_freelist (struct xfs_mount *mp,struct xfs_perag *pag); +unsigned int xfs_alloc_min_freelist_minleft (struct xfs_mount *mp, +struct xfs_perag *pag);int xfs_alloc_get_freelist (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agfbp,xfs_agblock_t *bnop,int btreeblk); int xfs_alloc_put_freelist(struct xfs_perag *pag, struct xfs_trans *tp,fs/xfs/libxfs/xfs_bmap.c +0
-0
fs/xfs/libxfs/xfs_bmap.c xfs_bmap_longest_free_extent( · line 3161
Quoted 2 lines
fs/xfs/libxfs/xfs_bmap.c +0
-0@@ -3160,7 +3160,7 @@ xfs_bmap_longest_free_extent( }
I intended to add a comment here but lost track. I.e., something like:
/*
* Use the minleft freelist helper because minleft can be set for bmbt
* allocs. If we don't factor it in here, the alloc can be sized
* incorrectly and fail.
*/
I'll wait for any further feedback before reposting.
Brian
Quoted 10 lines
fs/xfs/libxfs/xfs_bmap.c +1
-1longest =xfs_alloc_longest_free_extent (pag, -xfs_alloc_min_freelist (pag_mount (pag), pag), +xfs_alloc_min_freelist_minleft (pag_mount (pag), pag),xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;Signature · 1 line
2.55.0
Re: [External] : Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 152 lines · Brian Foster
Quoted 136 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by
xfs_alloc_fix_freelist()is:free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs.
The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement. To address this, create a variant of the AGFL min free calculation for minleft allocations that incorporates an additional allocbt level increase.
We do not add the additional blocks directly to
min_freebecause this would lead to spurious AGFL block allocations and frees in the common case (i.e. no btree splits). Instead, add the surplus block requirement to the minleft value used to select the AG. This ensures the AG has enough blocks for the caller's minleft value plus the worst case increase in the AGFL. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the sparse inode allocation fails gracefully with-ENOSPC.Reported-by Matt Fleming <[email protected]>Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_alloc.c +28
-1@@ -2500,6 +2500,20 @@ xfs_alloc_min_freelist(return __xfs_alloc_min_freelist (mp, pag,0 ); } +/* +* Return the minimum freelist requirement considering a potential allocbt split +* from the current allocation. Use this when computing longest free extent for +* allocations with minleft set to ensure that the available extent length +* accounts for the subsequent allocation's increased AGFL requirement. +*/ +unsigned int +xfs_alloc_min_freelist_minleft ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,1 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation @@ -2517,6 +2531,7 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true;@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that +* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + } + agflcount =min_t (xfs_extlen_t, pag->pagf_flcount, min_free); available = (int )(pag->pagf_freeblks + agflcount - - reservation - min_free - args->minleft); + reservation - min_free - minleft);if (available < (int )max (args->total, alloc_len))return false;fs/xfs/libxfs/xfs_alloc.h +2
-0@@ -73,6 +73,8 @@ xfs_extlen_t xfs_alloc_longest_free_extent(struct xfs_perag *pag, xfs_extlen_t need, xfs_extlen_t reserved);unsigned int xfs_alloc_min_freelist (struct xfs_mount *mp,struct xfs_perag *pag); +unsigned int xfs_alloc_min_freelist_minleft (struct xfs_mount *mp, +struct xfs_perag *pag);int xfs_alloc_get_freelist (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agfbp,xfs_agblock_t *bnop,int btreeblk); int xfs_alloc_put_freelist(struct xfs_perag *pag, struct xfs_trans *tp,fs/xfs/libxfs/xfs_bmap.c +0
-0fs/xfs/libxfs/xfs_bmap.c xfs_bmap_longest_free_extent( · line 3161
Quoted 2 lines
fs/xfs/libxfs/xfs_bmap.c +0
-0@@ -3160,7 +3160,7 @@ xfs_bmap_longest_free_extent( }I intended to add a comment here but lost track. I.e., something like:
/*
* Use the minleft freelist helper because minleft can be set for bmbt
* allocs. If we don't factor it in here, the alloc can be sized
* incorrectly and fail.
*/I'll wait for any further feedback before reposting.
Brian
Quoted 10 lines
Quoted 10 lines
fs/xfs/libxfs/xfs_bmap.c +1
-1longest =xfs_alloc_longest_free_extent (pag, -xfs_alloc_min_freelist (pag_mount (pag), pag), +xfs_alloc_min_freelist_minleft (pag_mount (pag), pag),xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;Signature · 1 line
2.55.0
thank you for the comment clarification.
I was confused by that change.
Could you CC stable. We saw a version of this problem in Linux 5.15
Re: [External] : Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 164 lines · Mark Tinguely
Quoted 148 lines · Brian Foster
Quoted 133 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by
xfs_alloc_fix_freelist()is:free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs.
The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement. To address this, create a variant of the AGFL min free calculation for minleft allocations that incorporates an additional allocbt level increase.
We do not add the additional blocks directly to
min_freebecause this would lead to spurious AGFL block allocations and frees in the common case (i.e. no btree splits). Instead, add the surplus block requirement to the minleft value used to select the AG. This ensures the AG has enough blocks for the caller's minleft value plus the worst case increase in the AGFL. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the sparse inode allocation fails gracefully with-ENOSPC.Reported-by Matt Fleming <[email protected]>Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_alloc.c +28
-1@@ -2500,6 +2500,20 @@ xfs_alloc_min_freelist(return __xfs_alloc_min_freelist (mp, pag,0 ); } +/* +* Return the minimum freelist requirement considering a potential allocbt split +* from the current allocation. Use this when computing longest free extent for +* allocations with minleft set to ensure that the available extent length +* accounts for the subsequent allocation's increased AGFL requirement. +*/ +unsigned int +xfs_alloc_min_freelist_minleft ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,1 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation @@ -2517,6 +2531,7 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true;@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that +* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + } + agflcount =min_t (xfs_extlen_t, pag->pagf_flcount, min_free); available = (int )(pag->pagf_freeblks + agflcount - - reservation - min_free - args->minleft); + reservation - min_free - minleft);if (available < (int )max (args->total, alloc_len))return false;fs/xfs/libxfs/xfs_alloc.h +2
-0@@ -73,6 +73,8 @@ xfs_extlen_t xfs_alloc_longest_free_extent(struct xfs_perag *pag, xfs_extlen_t need, xfs_extlen_t reserved);unsigned int xfs_alloc_min_freelist (struct xfs_mount *mp,struct xfs_perag *pag); +unsigned int xfs_alloc_min_freelist_minleft (struct xfs_mount *mp, +struct xfs_perag *pag);int xfs_alloc_get_freelist (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agfbp,xfs_agblock_t *bnop,int btreeblk); int xfs_alloc_put_freelist(struct xfs_perag *pag, struct xfs_trans *tp,fs/xfs/libxfs/xfs_bmap.c +0
-0fs/xfs/libxfs/xfs_bmap.c xfs_bmap_longest_free_extent( · line 3160
Quoted 1 line
fs/xfs/libxfs/xfs_bmap.c +0
-0@@ -3160,7 +3160,7 @@ xfs_bmap_longest_free_extent( }I intended to add a comment here but lost track. I.e., something like:
/*
* Use the minleft freelist helper because minleft can be set for bmbt
* allocs. If we don't factor it in here, the alloc can be sized
* incorrectly and fail.
*/I'll wait for any further feedback before reposting.
Brian
Quoted 10 lines
Quoted 10 lines
fs/xfs/libxfs/xfs_bmap.c +1
-1longest =xfs_alloc_longest_free_extent (pag, -xfs_alloc_min_freelist (pag_mount (pag), pag), +xfs_alloc_min_freelist_minleft (pag_mount (pag), pag),xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;Signature · 1 line
2.55.0thank you for the comment clarification.
I was confused by that change.
Quoted 2 lines
Could you CC stable. We saw a version of this problem in Linux 5.15
Sure, though I'm not sure how far back this will apply cleanly.
Curious.. was the instance you saw also an inobt alloc failure leading to dirty trans cancel and shutdown, or something different but similar?
Brian
Re: [External] : Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 8 lines · Brian Foster
On Fri, Aug 14, 2026 at 09: 15: 15AM -0500, Mark Tinguely wrote: > On 8/14/26 8: 57 AM, Brian Foster wrote: > > On Fri, Aug 14, 2026 at 09: 22: 39AM -0400, Brian Foster wrote: > > > Matt Fleming reports a filesystem shutdown due
Quoted 4 lines · Mark Tinguely
Quoted 3 lines · Brian Foster
Quoted 2 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation
(deleted text)
Quoted 8 lines
Quoted 7 lines
thank you for the comment clarification.
I was confused by that change.Could you CC stable. We saw a version of this problem in Linux 5.15
Quoted 7 lines
Sure, though I'm not sure how far back this will apply cleanly.
Curious.. was the instance you saw also an inobt alloc failure leading to dirty trans cancel and shutdown, or something different but similar?
Brian
yes the v5.15 problem was a similar sparse inode chunk / inode btree insert failure. I admit that I did not have the numbers to realize why the inode btree insert failed.
although this is a fundamental allocation flaw, the sparse inode chunk allocation seems to be the just right/wrong conditions to trigger it.
thanks for the fix
Mark Tinguely
Re: [External] : Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 23 lines · Mark Tinguely
Quoted 8 lines · Brian Foster
On Fri, Aug 14, 2026 at 09: 15: 15AM -0500, Mark Tinguely wrote: > On 8/14/26 8: 57 AM, Brian Foster wrote: > > On Fri, Aug 14, 2026 at 09: 22: 39AM -0400, Brian Foster wrote: > > > Matt Fleming reports a filesystem shutdown due
Quoted 4 lines · Mark Tinguely
Quoted 3 lines · Brian Foster
Quoted 2 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation
(deleted text)
Quoted 10 lines
Quoted 7 lines
thank you for the comment clarification.
I was confused by that change.Could you CC stable. We saw a version of this problem in Linux 5.15
Sure, though I'm not sure how far back this will apply cleanly.
Quoted 9 lines
Quoted 5 lines
Curious.. was the instance you saw also an inobt alloc failure leading to dirty trans cancel and shutdown, or something different but similar?
Brian
yes the v5.15 problem was a similar sparse inode chunk / inode btree insert failure. I admit that I did not have the numbers to realize why the inode btree insert failed.
Ok.
Quoted 3 lines
although this is a fundamental allocation flaw, the sparse inode chunk allocation seems to be the just right/wrong conditions to trigger it.
I think that's probably right, but I had the Fixes: tag on v1 and dropped it here just because I think it's kind of a tenuous connection now. Re: the discussion on v1, presumably we could have the same thing happen with bmap allocs and just not see the shutdown because those allocs are usually more flexible and can fall back to different AGs.
Regardless, wrt stable I think it would be reasonable to target back to the release that enabled sparse inodes (v4.2) since that's how this is reproduced in practice, so it probably doesn't make much difference either way.
Brian
Quoted 8 lines
Fixes 5419040fc0f3a xfs: introduce inode record hole mask for sparse inode chunks ??thanks for the fix
Mark Tinguely
Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
In reply to Brian FosterQuoted 43 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by
xfs_alloc_fix_freelist()is:free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs.
The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement. To address this, create a variant of the AGFL min free calculation for minleft allocations that incorporates an additional allocbt level increase.
Quoted 8 lines
We do not add the additional blocks directly to
min_freebecause this would lead to spurious AGFL block allocations and frees in the common case (i.e. no btree splits). Instead, add the surplus block requirement to the minleft value used to select the AG. This ensures the AG has enough blocks for the caller's minleft value plus the worst case increase in the AGFL. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the sparse inode allocation fails gracefully with-ENOSPC.
Hrmm. So if I'm understanding this correctly, you're adding to minleft (if one has been set) enough space to handle an AGFL expansion resulting from the free space / rmap btrees expanding in height? And you're effectively saying that we'd rather fail creation and writes with ENOSPC a little earlier to handle that, than risk a shutdown?
--D
Quoted 96 lines
Reported-by Matt Fleming <[email protected]>Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_alloc.c +28
-1@@ -2500,6 +2500,20 @@ xfs_alloc_min_freelist(return __xfs_alloc_min_freelist (mp, pag,0 ); } +/* +* Return the minimum freelist requirement considering a potential allocbt split +* from the current allocation. Use this when computing longest free extent for +* allocations with minleft set to ensure that the available extent length +* accounts for the subsequent allocation's increased AGFL requirement. +*/ +unsigned int +xfs_alloc_min_freelist_minleft ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,1 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation @@ -2517,6 +2531,7 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true;@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that +* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + } + agflcount =min_t (xfs_extlen_t, pag->pagf_flcount, min_free); available = (int )(pag->pagf_freeblks + agflcount - - reservation - min_free - args->minleft); + reservation - min_free - minleft);if (available < (int )max (args->total, alloc_len))return false;fs/xfs/libxfs/xfs_alloc.h +2
-0@@ -73,6 +73,8 @@ xfs_extlen_t xfs_alloc_longest_free_extent(struct xfs_perag *pag, xfs_extlen_t need, xfs_extlen_t reserved);unsigned int xfs_alloc_min_freelist (struct xfs_mount *mp,struct xfs_perag *pag); +unsigned int xfs_alloc_min_freelist_minleft (struct xfs_mount *mp, +struct xfs_perag *pag);int xfs_alloc_get_freelist (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agfbp,xfs_agblock_t *bnop,int btreeblk); int xfs_alloc_put_freelist(struct xfs_perag *pag, struct xfs_trans *tp,fs/xfs/libxfs/xfs_bmap.c +1
-1@@ -3160,7 +3160,7 @@ xfs_bmap_longest_free_extent( } longest =xfs_alloc_longest_free_extent (pag, -xfs_alloc_min_freelist (pag_mount (pag), pag), +xfs_alloc_min_freelist_minleft (pag_mount (pag), pag),xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;Signature · 1 line
2.55.0
Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 53 lines · Darrick J. Wong
Quoted 43 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by
xfs_alloc_fix_freelist()is:free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs.
The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement. To address this, create a variant of the AGFL min free calculation for minleft allocations that incorporates an additional allocbt level increase.
Quoted 8 lines
We do not add the additional blocks directly to
min_freebecause this would lead to spurious AGFL block allocations and frees in the common case (i.e. no btree splits). Instead, add the surplus block requirement to the minleft value used to select the AG. This ensures the AG has enough blocks for the caller's minleft value plus the worst case increase in the AGFL. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the sparse inode allocation fails gracefully with-ENOSPC.
Quoted 6 lines
Hrmm. So if I'm understanding this correctly, you're adding to minleft (if one has been set) enough space to handle an AGFL expansion resulting from the free space / rmap btrees expanding in height? And you're effectively saying that we'd rather fail creation and writes with ENOSPC a little earlier to handle that, than risk a shutdown?
Yep, pretty much. minleft != 0 is a signal for multiple allocs. The caller obviously sets the value based on its own needs. The first alloc fixes up the AGFL and evaluates minleft and whatnot based on that, but if that alloc splits the trees, the caller's minleft alone doesn't allow the AG to meet minimum available space requirements for the second alloc because the split(s) cause the space requirements to increase. Hence, the allocator has to make sure the caller's multiple-allocation sequence is safe from the start.
FWIW I feel like we should eventually be able to evolve this into something a little more elegant, but I haven't quite thought of what that is and it's kind of a corner case that so far only seems to affect the sparse inode use case, so a targeted/backportable fix seems reasonable (IMO).
Brian
Quoted 99 lines
--D
Quoted 96 lines
Reported-by Matt Fleming <[email protected]>Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_alloc.c +28
-1@@ -2500,6 +2500,20 @@ xfs_alloc_min_freelist(return __xfs_alloc_min_freelist (mp, pag,0 ); } +/* +* Return the minimum freelist requirement considering a potential allocbt split +* from the current allocation. Use this when computing longest free extent for +* allocations with minleft set to ensure that the available extent length +* accounts for the subsequent allocation's increased AGFL requirement. +*/ +unsigned int +xfs_alloc_min_freelist_minleft ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,1 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation @@ -2517,6 +2531,7 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true;@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that +* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + } + agflcount =min_t (xfs_extlen_t, pag->pagf_flcount, min_free); available = (int )(pag->pagf_freeblks + agflcount - - reservation - min_free - args->minleft); + reservation - min_free - minleft);if (available < (int )max (args->total, alloc_len))return false;fs/xfs/libxfs/xfs_alloc.h +2
-0@@ -73,6 +73,8 @@ xfs_extlen_t xfs_alloc_longest_free_extent(struct xfs_perag *pag, xfs_extlen_t need, xfs_extlen_t reserved);unsigned int xfs_alloc_min_freelist (struct xfs_mount *mp,struct xfs_perag *pag); +unsigned int xfs_alloc_min_freelist_minleft (struct xfs_mount *mp, +struct xfs_perag *pag);int xfs_alloc_get_freelist (struct xfs_perag *pag,struct xfs_trans *tp,struct xfs_buf *agfbp,xfs_agblock_t *bnop,int btreeblk); int xfs_alloc_put_freelist(struct xfs_perag *pag, struct xfs_trans *tp,fs/xfs/libxfs/xfs_bmap.c +1
-1@@ -3160,7 +3160,7 @@ xfs_bmap_longest_free_extent( } longest =xfs_alloc_longest_free_extent (pag, -xfs_alloc_min_freelist (pag_mount (pag), pag), +xfs_alloc_min_freelist_minleft (pag_mount (pag), pag),xfs_ag_resv_needed (pag,XFS_AG_RESV_NONE ));if (*blen < longest) *blen = longest;Signature · 1 line
2.55.0
Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
In reply to Brian FosterQuoted 100 lines · Brian Foster
Matt Fleming reports a filesystem shutdown due to inobt block allocation failure during sparse chunk allocation. Inode creation can involve multiple allocations in a transaction: the initial chunk allocation and inode btree blocks via inobt record insertion. This is expected to be safe by using the minleft parameter on the chunk allocation to guarantee the selected AG has blocks available for a followup inobt insertion.
The sequence that leads to this failure is that the alloc and inode btrees are all full (require a full split on next insertion) and the AG has just enough free space to satisfy a sparse chunk allocation with minleft set (i.e. 7 blocks in this example). The chunk allocation splits a free extent, triggers full allocbt splits, and consumes 4 free blocks for the chunk and 4 AGFL blocks for the btrees.
Next, the inobt record insertion triggers an inobt split. The AG has enough free blocks, but the allocbt splits caused by the chunk allocation have increased the min AGFL requirement for the AG due to btree level increases. The AGFL requirement as calculated by
xfs_alloc_fix_freelist()is:free + AGFL - res - minfree - minleft = avail
This evaluates to the following on initial chunk allocation:
2514 + 8 - 2505 - 8 - 2 = 7... and then after the chunk allocation but before the inobt block allocation:
2510 + 4 - 2505 - 12 - 0 = -3This causes the inobt alloc to fail despite minleft being set in the first allocation. The error path cancels the dirty transaction and shuts down the fs.
The problem here is that while minleft ensures free blocks are available for the inobt insert, it is not sufficient to cover the increase of the AGFL min free requirement. To address this, create a variant of the AGFL min free calculation for minleft allocations that incorporates an additional allocbt level increase.
We do not add the additional blocks directly to
min_freebecause this would lead to spurious AGFL block allocations and frees in the common case (i.e. no btree splits). Instead, add the surplus block requirement to the minleft value used to select the AG. This ensures the AG has enough blocks for the caller's minleft value plus the worst case increase in the AGFL. In the example above, the initial calculation now evaluates to 3 blocks available instead of 7 and the sparse inode allocation fails gracefully with-ENOSPC.Reported-by Matt Fleming <[email protected]>Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_alloc.c +17
-0@@ -2500,6 +2500,20 @@ xfs_alloc_min_freelist(return __xfs_alloc_min_freelist (mp, pag,0 ); } +/* +* Return the minimum freelist requirement considering a potential allocbt split +* from the current allocation. Use this when computing longest free extent for +* allocations with minleft set to ensure that the available extent length +* accounts for the subsequent allocation's increased AGFL requirement. +*/ +unsigned int +xfs_alloc_min_freelist_minleft ( +struct xfs_mount *mp, +struct xfs_perag *pag) +{ +return __xfs_alloc_min_freelist (mp, pag,1 ); +} +/* * Check if the operation we are fixing up the freelist for should go ahead or * not. If we are freeing blocks, we always allow it, otherwise the allocation @@ -2517,6 +2531,7 @@ xfs_alloc_space_available(xfs_extlen_t reservation;/* blocks that are still reserved */ int available;xfs_extlen_t agflcount; +xfs_extlen_t minleft;if (flags &XFS_ALLOC_FLAG_FREEING )return true;@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that
fs/xfs/libxfs/xfs_alloc.c xfs_alloc_space_available( · line 2562
Quoted 10 lines
fs/xfs/libxfs/xfs_alloc.c +9
-0+* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + }
This seems fragile to me. It is based on the assumption that min_free is calculated from xfs_alloc_min_freelist() by the caller, and then this calculates the difference between what the caller should have calculated and what is actually needed.
Where this mod is placed also results in the longest available extent check not taking into account this modified min_free requirement, whereas the check in xfs_bmap_longest_free_extent() is modified to take this modified minleft value into account. i.e. the checks w.r.t. minleft and longest extents are no longer consistent across the layers.
I'm also concerned that this results in the xfs_alloc_space_available() caller using different values of "need" and "minleft" to what the actual space availablity calculation is using; that feels like a future landmine to me.
i.e. the xfs_alloc_space_available() caller already knows is minleft is set, so if it were to use xfs_alloc_min_freelist_minleft(), then there would not need to be this "correction" in this code and all the values would be consistent.
Unless I'm missing something subtle, I think that the callers should not need to know it should call xfs_alloc_min_freelist_minleft() or xfs_alloc_min_freelist() as it feels like exposing internal AGFL space/btree accounting requirements into an external API. All the caller needs to signal is whether this is the first of a chain of allocations or not (i.e. args->minleft != 0), and the internal alloc code should handle it from there.
Hence I suspect it would be much cleaner just to add a 'bool multialloc' parameter to xfs_alloc_min_freelist() and have all callers set it appropriately. That would avoid the need for the wrapper functions and keep this AGFL reservation wart completely internal to the AGFL reservation calculation....
Thoughts?
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 3/3] xfs: incorporate increased AGFL min requirement for minleft allocs
Quoted 1 line · Dave Chinner
On Fri, Aug 14, 2026 at 09:22:39AM -0400, Brian Foster wrote:
...
Quoted 14 lines
Quoted 14 lines
Reported-by Matt Fleming <[email protected]>Assisted-by LLMSigned-off-by Brian Foster <[email protected]>--- fs/xfs/libxfs/xfs_alloc.c | 29 ++++++++++++++++++++++++++++- fs/xfs/libxfs/xfs_alloc.h | 2 ++ fs/xfs/libxfs/xfs_bmap.c | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-)fs/xfs/libxfs/xfs_alloc.c +0
-0
...
Quoted 29 lines
Quoted 6 lines
fs/xfs/libxfs/xfs_alloc.c +2
-0@@ -2533,10 +2548,22 @@ xfs_alloc_space_available(* Do we have enough free space remaining for the allocation? Don't * account extra agfl blocks because we are about to defer free them, * making them unavailable until the current transaction commits. +* +* If minleft is set, this allocation might cause an allocbt split that fs/xfs/libxfs/xfs_alloc.c xfs_alloc_space_available( · line 2562
Quoted 10 lines
fs/xfs/libxfs/xfs_alloc.c +9
-0+* increases the AGFL minimum for the next allocation in the +* transaction. Reserve that space from the available block count +* (without prematurely growing the AGFL) to prevent the subsequent +* allocation from failing due to an increased min_free requirement. */ + minleft = args->minleft; +if (minleft) { + minleft +=xfs_alloc_min_freelist_minleft (args->mp, pag) - + min_free; + }This seems fragile to me. It is based on the assumption that
min_freeis calculated fromxfs_alloc_min_freelist()by the caller, and then this calculates the difference between what the caller should have calculated and what is actually needed.Where this mod is placed also results in the longest available extent check not taking into account this modified
min_freerequirement, whereas the check inxfs_bmap_longest_free_extent()is modified to take this modified minleft value into account. i.e. the checks w.r.t. minleft and longest extents are no longer consistent across the layers.
Quoted 5 lines
I'm also concerned that this results in the
xfs_alloc_space_available()caller using different values of "need" and "minleft" to what the actual space availablity calculation is using; that feels like a future landmine to me.
I agree in principle wrt the fragility. This is kind of what I meant by expressing preference for something more elegant in the other subthread conversation w/ Darrick; I just haven't quite figured out what that looks like yet.
Quoted 5 lines
i.e. the
xfs_alloc_space_available()caller already knows is minleft is set, so if it were to usexfs_alloc_min_freelist_minleft(), then there would not need to be this "correction" in this code and all the values would be consistent.
Quoted 8 lines
Unless I'm missing something subtle, I think that the callers should not need to know it should call
xfs_alloc_min_freelist_minleft()orxfs_alloc_min_freelist()as it feels like exposing internal AGFL space/btree accounting requirements into an external API. All the caller needs to signal is whether this is the first of a chain of allocations or not (i.e.args->minleft!= 0), and the internal alloc code should handle it from there.
I also agree wrt to the layering confusion, though I find that the existing API also suffers a bit from this.
Quoted 8 lines
Hence I suspect it would be much cleaner just to add a 'bool multialloc' parameter to
xfs_alloc_min_freelist()and have all callers set it appropriately. That would avoid the need for the wrapper functions and keep this AGFL reservation wart completely internal to the AGFL reservation calculation....Thoughts?
If I follow correctly, this centralizes the extra level calculation to _min_freelist(). I agree the code is cleaner, but the reason I didn't do that is because this would artificially alloc and free blocks through the AGFL for every such multi-allocation.
I.e., a multi-alloc comes in and we calculate the multi-level worst case bump for each allocbt, set that to min_free, bump the AGFL to the new min_free and carry on with the alloc. The common case is going to be no new splits (and certainly not "everything splits"), so either the next allocation in the multi-alloc (or the next alloc thereafter) will remove every block that was added for splits that never occurred. So IMO the tradeoff for cleaner code here is worse behavior.
Hmmm.. I suppose we could still implement this same sort of behavior using a multi-alloc parameter. For example, suppose we replaced xfs_alloc_min_freelist() with an xfs_alloc_freelist() that took your multi-alloc param and returned a min and a max. The min is the current return value, the max is the prospective worst case (i.e. min == max if multi-alloc == false, else +1 level).
The internal alloc path passes both min/max to _space_available() so it can do consistent max length checks and calculate/add the delta to minleft for the available calculation. The AGFL min_free value is still based on the min. The bmap case passes multi-alloc = true and uses the max for its own length checks. Hm?
Brian
Quoted 5 lines
-Dave.
Signature · 2 lines
Dave Chinner [email protected]