xfs_repair: estimate per-AG btree slack better
PATCH 1/1 xfs_repair: estimate per-AG btree slack better
From: Darrick J. Wong <[email protected]>
The slack calculation for per-AG btrees is a bit inaccurate because it only disables slack space in the new btrees when the amount of free space in the AG (not counting the btrees) is less than 3/32ths of the AG. In other words, it assumes that the btrees will fit in less than 9 percent of the space.
However, there's one scenario where this goes wrong -- if the rmapbt consumes a significant portion of the AG space. Say a filesystem is hosting a VM image farm that starts with perfectly shared images. As time goes by, random writes to those images will slowly cause the rmapbt to increase in size as blocks within those images get COWed.
Suppose that the rmapbt now consumes 20% of the space in the AG, that the AG is nearly full, and that the blocks in the old rmapbt are mostly full. At the start of phase5_func, mk_incore_fstree will return that num_freeblocks is ~20% of the AG size. Hence the slack calculation will conclude that there's plenty of space in the AG and new btrees will be built with 25% slack in the blocks. If the size of these new expanded btrees is larger than the free space in the AG, repair will fail to allocate btree blocks and fail, causing severe filesystem damage.
To combat this, estimate the worst case size of the AG btrees given the number of records we intend to put in them, subtract that worst case figure from num_freeblocks, and feed that to bulkload_estimate_ag_slack. This results in tighter packing of new btree blocks when space is dear, and hopefully fewer problems. This /can/ be reproduced with generic/333 if you hack it to keep COWing blocks until the filesystem is totally out of space, even if reflink has long since refused to share more blocks.
--- libxfs/libxfs_api_defs.h | 4 ++ repair/agbtree.c | 90 ++++++++++++++++++++++++++++++++++++++++------ repair/agbtree.h | 3 ++ repair/phase5.c | 18 +++++++-- repair/rmap.c | 44 ++++++++++++++++++++++ repair/rmap.h | 3 ++ 6 files changed, 146 insertions(+), 16 deletions(-)
libxfs/libxfs_api_defs.h +4 -0
@@ -25,6 +25,7 @@#define xfs_ag_resv_free libxfs_ag_resv_free#define xfs_alloc_ag_max_usable libxfs_alloc_ag_max_usable +#define xfs_allocbt_calc_size libxfs_allocbt_calc_size#define xfs_allocbt_maxlevels_ondisk libxfs_allocbt_maxlevels_ondisk#define xfs_allocbt_maxrecs libxfs_allocbt_maxrecs#define xfs_allocbt_stage_cursor libxfs_allocbt_stage_cursor
@@ -115,6 +116,7 @@#define xfs_highbit32 libxfs_highbit32#define xfs_highbit64 libxfs_highbit64#define xfs_ialloc_calc_rootino libxfs_ialloc_calc_rootino +#define xfs_iallocbt_calc_size libxfs_iallocbt_calc_size#define xfs_iallocbt_maxlevels_ondisk libxfs_iallocbt_maxlevels_ondisk#define xfs_ialloc_read_agi libxfs_ialloc_read_agi#define xfs_idata_realloc libxfs_idata_realloc
@@ -146,6 +148,7 @@#define xfs_read_agf libxfs_read_agf#define xfs_refc_block libxfs_refc_block#define xfs_refcountbt_calc_reserves libxfs_refcountbt_calc_reserves +#define xfs_refcountbt_calc_size libxfs_refcountbt_calc_size#define xfs_refcountbt_init_cursor libxfs_refcountbt_init_cursor#define xfs_refcountbt_maxlevels_ondisk libxfs_refcountbt_maxlevels_ondisk#define xfs_refcountbt_maxrecs libxfs_refcountbt_maxrecs
@@ -155,6 +158,7 @@#define xfs_rmap_alloc libxfs_rmap_alloc#define xfs_rmapbt_calc_reserves libxfs_rmapbt_calc_reserves +#define xfs_rmapbt_calc_size libxfs_rmapbt_calc_size#define xfs_rmapbt_init_cursor libxfs_rmapbt_init_cursor#define xfs_rmapbt_maxlevels_ondisk libxfs_rmapbt_maxlevels_ondisk#define xfs_rmapbt_maxrecs libxfs_rmapbt_maxrecs
repair/agbtree.c +78 -12
158 lines not on this page · raw message
repair/agbtree.h +3 -0
@@ -59,4 +59,7 @@ void init_refc_cursor(struct repair_ctx *sc, struct xfs_perag *pag,void build_refcount_tree (struct repair_ctx *sc,xfs_agnumber_t agno,struct bt_rebuild *btr); +xfs_extlen_t estimate_agbtree_blocks (struct xfs_perag *pag, +unsigned int free_extents); +#endif /* __XFS_REPAIR_AG_BTREE_H__ */
repair/phase5.c +14 -4
@@ -447,6 +447,8 @@ phase5_func(int extra_blocks =0 ;uint num_freeblocks;xfs_agblock_t num_extents; +unsigned int est_agfreeblocks =0 ; +unsigned int total_btblocks;if (verbose)do_log (_ (" - agno = %d\n" ), agno);
@@ -474,12 +476,20 @@ _("unable to rebuild AG %u. Not enough free space in on-disk AG.\n"), agno); } -init_ino_cursors (&sc, pag, num_freeblocks, &sb_icount_ag[agno], +/* +* Estimate the number of free blocks in this AG after rebuilding +* all btrees. +*/ + total_btblocks =estimate_agbtree_blocks (pag, num_extents); +if (num_freeblocks > total_btblocks) + est_agfreeblocks = num_freeblocks - total_btblocks; + +init_ino_cursors (&sc, pag, est_agfreeblocks, &sb_icount_ag[agno], &sb_ifree_ag[agno], &btr_ino, &btr_fino); -init_rmapbt_cursor (&sc, pag, num_freeblocks, &btr_rmap); +init_rmapbt_cursor (&sc, pag, est_agfreeblocks, &btr_rmap); -init_refc_cursor (&sc, pag, num_freeblocks, &btr_refc); +init_refc_cursor (&sc, pag, est_agfreeblocks, &btr_refc); num_extents =count_bno_extents_blocks (agno, &num_freeblocks);/*
@@ -507,7 +517,7 @@ _("unable to rebuild AG %u. Not enough free space in on-disk AG.\n"),/* * track blocks that we might really lose */ -init_freespace_cursors (&sc, pag, num_freeblocks, &num_extents, +init_freespace_cursors (&sc, pag, est_agfreeblocks, &num_extents, &extra_blocks, &btr_bno, &btr_cnt);/*
repair/rmap.c +44 -0
47 lines not on this page · raw message
repair/rmap.h +3 -0
@@ -48,4 +48,7 @@ extern int fix_inode_reflink_flags(struct xfs_mount *, xfs_agnumber_t);extern void fix_freelist (struct xfs_mount *,xfs_agnumber_t ,bool );extern void rmap_store_agflcount (struct xfs_mount *,xfs_agnumber_t ,int ); +xfs_extlen_t estimate_rmapbt_blocks (struct xfs_perag *pag); +xfs_extlen_t estimate_refcountbt_blocks (struct xfs_perag *pag); +#endif /* RMAP_H_ */
Hi.
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The slack calculation for per-AG btrees is a bit inaccurate because it only disables slack space in the new btrees when the amount of free space in the AG (not counting the btrees) is less than 3/32ths of the AG. In other words, it assumes that the btrees will fit in less than 9 percent of the space.
.
.
.
Quoted 2 lines
Signed-off-by Darrick J. Wong <[email protected]>
This looks fine, with a small caveat below...
.
.
.
repair/agbtree.c _("Error %d while creating refcount btree for AG %u.\n"), error, agno); · line 708
Quoted 8 lines
repair/agbtree.c +8
-0+ +static xfs_extlen_t +estimate_allocbt_blocks ( +struct xfs_perag *pag, +unsigned int nr_extents) +{ +return libxfs_allocbt_calc_size (pag->pag_mount, nr_extents) *2 ; +}
Forgive my ignorance here, but what's the reason of the magic number? It seems to me by multiplying by 2 here, you are considering a split of every single leaf for the calculated btree size, but I'm not sure if that's the intention, could you please confirm or correct me? :)
Other than that, the patch looks good
Signature · 1 line
Carlos Maiolino
Quoted 22 lines · Carlos Maiolino
Hi.
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The slack calculation for per-AG btrees is a bit inaccurate because it only disables slack space in the new btrees when the amount of free space in the AG (not counting the btrees) is less than 3/32ths of the AG. In other words, it assumes that the btrees will fit in less than 9 percent of the space.
.
.
.Quoted 2 lines
Signed-off-by Darrick J. Wong <[email protected]>This looks fine, with a small caveat below...
.
.
.
Quoted 13 lines
repair/agbtree.c _("Error %d while creating refcount btree for AG %u.\n"), error, agno); · line 708
Quoted 8 lines
repair/agbtree.c +8
-0+ +static xfs_extlen_t +estimate_allocbt_blocks ( +struct xfs_perag *pag, +unsigned int nr_extents) +{ +return libxfs_allocbt_calc_size (pag->pag_mount, nr_extents) *2 ; +}Forgive my ignorance here, but what's the reason of the magic number? It seems to me by multiplying by 2 here, you are considering a split of every single leaf for the calculated btree size, but I'm not sure if that's the intention, could you please confirm or correct me? :)
Ah, I should document that better...
/* Account for space consumed by both free space btrees */ return libxfs_allocbt_calc_size(...) * 2;
--D
Quoted 6 lines
Other than that, the patch looks good
Reviewed-by Carlos Maiolino <[email protected]>Signature · 1 line
Carlos Maiolino
Quoted 37 lines · Darrick J. Wong
Quoted 22 lines · Carlos Maiolino
Hi.
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The slack calculation for per-AG btrees is a bit inaccurate because it only disables slack space in the new btrees when the amount of free space in the AG (not counting the btrees) is less than 3/32ths of the AG. In other words, it assumes that the btrees will fit in less than 9 percent of the space.
.
.
.Quoted 2 lines
Signed-off-by Darrick J. Wong <[email protected]>This looks fine, with a small caveat below...
.
.
.Quoted 13 lines
repair/agbtree.c _("Error %d while creating refcount btree for AG %u.\n"), error, agno); · line 708
Quoted 8 lines
repair/agbtree.c +8
-0+ +static xfs_extlen_t +estimate_allocbt_blocks ( +struct xfs_perag *pag, +unsigned int nr_extents) +{ +return libxfs_allocbt_calc_size (pag->pag_mount, nr_extents) *2 ; +}Forgive my ignorance here, but what's the reason of the magic number? It seems to me by multiplying by 2 here, you are considering a split of every single leaf for the calculated btree size, but I'm not sure if that's the intention, could you please confirm or correct me? :)
Quoted 4 lines
Ah, I should document that better...
/* Account for space consumed by both free space btrees */ return libxfs_allocbt_calc_size(...) * 2;
Thanks, can I update your patch with the above comment, or do you want to send it again?
Quoted 9 lines
--D
Quoted 6 lines
Other than that, the patch looks good
Reviewed-by Carlos Maiolino <[email protected]>--
Carlos Maiolino
Signature · 1 line
Carlos Maiolino
Quoted 38 lines · Carlos Maiolino
Quoted 37 lines · Darrick J. Wong
Quoted 35 lines · Carlos Maiolino
Hi.
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The slack calculation for per-AG btrees is a bit inaccurate because it only disables slack space in the new btrees when the amount of free space in the AG (not counting the btrees) is less than 3/32ths of the AG. In other words, it assumes that the btrees will fit in less than 9 percent of the space.
.
.
.Quoted 2 lines
Signed-off-by Darrick J. Wong <[email protected]>This looks fine, with a small caveat below...
.
.
.repair/agbtree.c _("Error %d while creating refcount btree for AG %u.\n"), error, agno); · line 708
Quoted 8 lines
repair/agbtree.c +8
-0+ +static xfs_extlen_t +estimate_allocbt_blocks ( +struct xfs_perag *pag, +unsigned int nr_extents) +{ +return libxfs_allocbt_calc_size (pag->pag_mount, nr_extents) *2 ; +}Forgive my ignorance here, but what's the reason of the magic number? It seems to me by multiplying by 2 here, you are considering a split of every single leaf for the calculated btree size, but I'm not sure if that's the intention, could you please confirm or correct me? :)
Quoted 7 lines
Quoted 4 lines
Ah, I should document that better...
/* Account for space consumed by both free space btrees */ return libxfs_allocbt_calc_size(...) * 2;Thanks, can I update your patch with the above comment, or do you want to send it again?
You can add it, if that'll save time. I don't have any other changes pending for that patch.
--D
Quoted 12 lines
Quoted 9 lines
--D
Quoted 6 lines
Other than that, the patch looks good
Reviewed-by Carlos Maiolino <[email protected]>--
Carlos MaiolinoSignature · 1 line
Carlos Maiolino
Quoted 41 lines · Darrick J. Wong
Quoted 40 lines · Carlos Maiolino
Quoted 39 lines · Darrick J. Wong
Quoted 22 lines · Carlos Maiolino
Hi.
Quoted 7 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
The slack calculation for per-AG btrees is a bit inaccurate because it only disables slack space in the new btrees when the amount of free space in the AG (not counting the btrees) is less than 3/32ths of the AG. In other words, it assumes that the btrees will fit in less than 9 percent of the space.
.
.
.Quoted 2 lines
Signed-off-by Darrick J. Wong <[email protected]>This looks fine, with a small caveat below...
.
.
.Quoted 13 lines
repair/agbtree.c _("Error %d while creating refcount btree for AG %u.\n"), error, agno); · line 708
Quoted 8 lines
repair/agbtree.c +8
-0+ +static xfs_extlen_t +estimate_allocbt_blocks ( +struct xfs_perag *pag, +unsigned int nr_extents) +{ +return libxfs_allocbt_calc_size (pag->pag_mount, nr_extents) *2 ; +}Forgive my ignorance here, but what's the reason of the magic number? It seems to me by multiplying by 2 here, you are considering a split of every single leaf for the calculated btree size, but I'm not sure if that's the intention, could you please confirm or correct me? :)
Ah, I should document that better...
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
/* Account for space consumed by both free space btrees */ return libxfs_allocbt_calc_size(...) * 2;Thanks, can I update your patch with the above comment, or do you want to send it again?
You can add it, if that'll save time. I don't have any other changes pending for that patch.
Ok, won't take much of my time and will prevent you to need to send it again.
Quoted 15 lines
--D
Quoted 12 lines
Quoted 9 lines
--D
Quoted 6 lines
Other than that, the patch looks good
Reviewed-by Carlos Maiolino <[email protected]>--
Carlos Maiolino--
Carlos Maiolino
Signature · 1 line
Carlos Maiolino