repair: fix process_rt_rec_dups
PATCH 1/1 repair: fix process_rt_rec_dups
search_rt_dup_extent takes a xfs_rtblock_t, not an RT extent number.
---
What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.
repair/dinode.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
repair/dinode.c +1 -3
@@ -194,13 +194,11 @@ process_rt_rec_dups( struct xfs_bmbt_irec *irec) {xfs_fsblock_t b; -xfs_rtblock_t ext;for (b =rounddown (irec->br_startblock, mp->m_sb.sb_rextsize); b < irec->br_startblock + irec->br_blockcount; b += mp->m_sb.sb_rextsize) { - ext = (xfs_rtblock_t ) b / mp->m_sb.sb_rextsize; -if (search_rt_dup_extent (mp, ext)) { +if (search_rt_dup_extent (mp, b)) { do_warn( _("data fork in rt ino %" PRIu64 " claims dup rt extent," "off - %" PRIu64 ", start - %" PRIu64 ", count %" PRIu64 "\n"),
Signature · 1 line
2.39.2
Quoted 8 lines · Christoph Hellwig
search_rt_dup_extenttakes axfs_rtblock_t, not an RT extent number.Signed-off-by Christoph Hellwig <[email protected]>---
What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.
I think you've misread the code -- phase 4 builds the rt_dup tree by walks all the rtextents, and adding the duplicates:
for (rtx = 0; rtx < mp->m_sb.sb_rextents; rtx++) {
bstate = get_rtbmap(rtx);
switch (bstate) {
...
case XR_E_FS_MAP:
if (rt_start == 0)
continue;
else {
/*
* add extent and reset extent state
*/
add_rt_dup_extent(rt_start, rt_len);
rt_start = 0;
rt_len = 0;
}
break;
case XR_E_MULT:
if (rt_start == 0) {
rt_start = rtx;
rt_len = 1;
} else if (rt_len == XFS_MAX_BMBT_EXTLEN) {
/*
* large extent case
*/
add_rt_dup_extent(rt_start, rt_len);
rt_start = rtx;
rt_len = 1;
} else
rt_len++;
break;So I think the reason why you've never seen false duplicates is that the rt_dup tree intervals measure rt extents, not rt blocks. The units conversion in process_rt_rec_dups is correct.
However, none of that is at all obvious because of the dual uses of xfs_rtblock_t for rt blocks and rt extents.
I guess I ought to post the xfsprogs version of those rt units cleanups. :)
--D
Quoted 25 lines
repair/dinode.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)repair/dinode.c +1
-3@@ -194,13 +194,11 @@ process_rt_rec_dups( struct xfs_bmbt_irec *irec) {xfs_fsblock_t b; -xfs_rtblock_t ext;for (b =rounddown (irec->br_startblock, mp->m_sb.sb_rextsize); b < irec->br_startblock + irec->br_blockcount; b += mp->m_sb.sb_rextsize) { - ext = (xfs_rtblock_t ) b / mp->m_sb.sb_rextsize; -if (search_rt_dup_extent (mp, ext)) { +if (search_rt_dup_extent (mp, b)) { do_warn( _("data fork in rt ino %" PRIu64 " claims dup rt extent," "off - %" PRIu64 ", start - %" PRIu64 ", count %" PRIu64 "\n"),Signature · 1 line
2.39.2
Quoted 6 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig
search_rt_dup_extenttakes axfs_rtblock_t, not an RT extent number.Signed-off-by Christoph Hellwig <[email protected]>---
Quoted 6 lines
Quoted 3 lines
What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.
I think you've misread the code -- phase 4 builds the
rt_duptree by walks all the rtextents, and adding the duplicates:
Hmm.
So yes, add_rt_dup_extent seems to be called on an actual rtext, but scan_bmapbt calls search_rt_dup_extent with what is clearly
a fsbno_t. So something is fishy here for sure..
Quoted 3 lines
So I think the reason why you've never seen false duplicates is that the
rt_duptree intervals measure rt extents, not rt blocks. The units conversion inprocess_rt_rec_dupsis correct.
Note that I don't see error with the patch either, so either way the coverage isn't good enough..
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 8 lines · Christoph Hellwig
search_rt_dup_extenttakes axfs_rtblock_t, not an RT extent number.Signed-off-by Christoph Hellwig <[email protected]>---
What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.
Quoted 8 lines
Quoted 2 lines
I think you've misread the code -- phase 4 builds the
rt_duptree by walks all the rtextents, and adding the duplicates:Hmm.
So yes,
add_rt_dup_extentseems to be called on an actual rtext, butscan_bmapbtcallssearch_rt_dup_extentwith what is clearly
afsbno_t. So something is fishy here for sure..
Yes, I just noticed that scam_bmapbt thing. Yikes.
Quoted 6 lines
Quoted 3 lines
So I think the reason why you've never seen false duplicates is that the
rt_duptree intervals measure rt extents, not rt blocks. The units conversion inprocess_rt_rec_dupsis correct.Note that I don't see error with the patch either, so either way the coverage isn't good enough..
Correct. The scan_bmapbt query is in units of rtblocks. For rextsize==1 there's no error here; for rextsize > 2, the search_rt_dup_extent queries probe well past the end of the rt_ext_tree structure, so they never find the duplicate extents.
This /does/ explain why the one time I tried crosslinking rt extents I was surprised that xfs_repair didn't seem to detect them consistently.
Hmm, let me go clean all that up in to a TOTALLY UNTESTED patch.
--D
From: Darrick J. Wong <[email protected]>
Subject: [PATCH] xfs_repair: fix confusing rt space units in the duplicate detection code
Christoph Hellwig stumbled over the crosslinked file data detection code in xfs_repair. While trying to make sense of his fixpatch, I realized that the variable names and unit types are very misleading.
The rt dup tree builder inserts records in units of realtime extents. One query of the rt dup tree passes in a realtime extent number, but one of them does not. Confusingly, all the variable names have "block" even though they really mean "extent". This makes a real difference for rextsize > 1 filesystems, though given the lack of complaints I'm guessing there aren't many users.
Clean up this whole mess by fixing the variable names of the duplicates tree and the state array to reflect the units that are stored in the data structure, and fix the buggy query code. Later on in this patchset we'll fix the variable types too.
This seems to have been broken since before the start of the git repo.
--- repair/incore.c | 16 ++++++----- repair/incore.h | 15 ++++------ repair/incore_ext.c | 74 +++++++++++++++++++++++++++------------------------ repair/phase4.c | 12 ++++---- repair/scan.c | 6 +++- 5 files changed, 63 insertions(+), 60 deletions(-)
repair/incore.c +8 -8
@@ -178,21 +178,21 @@ static size_t rt_bmap_size;*/ int get_rtbmap ( -xfs_rtblock_t bno) +xfs_rtblock_t rtx) { -return (*(rt_bmap + bno /XR_BB_NUM ) >> - ((bno %XR_BB_NUM ) *XR_BB )) &XR_BB_MASK ; +return (*(rt_bmap + rtx /XR_BB_NUM ) >> + ((rtx %XR_BB_NUM ) *XR_BB )) &XR_BB_MASK ; }void set_rtbmap ( -xfs_rtblock_t bno, +xfs_rtblock_t rtx,int state) { - *(rt_bmap + bno /XR_BB_NUM ) = - ((*(rt_bmap + bno /XR_BB_NUM ) & - (~((uint64_t )XR_BB_MASK << ((bno %XR_BB_NUM ) *XR_BB )))) | - (((uint64_t ) state) << ((bno %XR_BB_NUM ) *XR_BB ))); + *(rt_bmap + rtx /XR_BB_NUM ) = + ((*(rt_bmap + rtx /XR_BB_NUM ) & + (~((uint64_t )XR_BB_MASK << ((rtx %XR_BB_NUM ) *XR_BB )))) | + (((uint64_t ) state) << ((rtx %XR_BB_NUM ) *XR_BB ))); } static void
repair/incore.h +6 -9
@@ -28,8 +28,8 @@ void set_bmap_ext(xfs_agnumber_t agno, xfs_agblock_t agbno,int get_bmap_ext (xfs_agnumber_t agno,xfs_agblock_t agbno,xfs_agblock_t maxbno,xfs_extlen_t *blen); -void set_rtbmap (xfs_rtblock_t bno,int state); -int get_rtbmap (xfs_rtblock_t bno); +void set_rtbmap (xfs_rtblock_t rtx,int state); +int get_rtbmap (xfs_rtblock_t rtx);static inline void set_bmap (xfs_agnumber_t agno,xfs_agblock_t agbno,int state)
@@ -70,8 +70,8 @@ typedef struct extent_tree_node { typedef struct rt_extent_tree_node { avlnode_t avl_node; - xfs_rtblock_t rt_startblock; /* starting realtime block */ - xfs_extlen_t rt_blockcount; /* number of blocks in extent */ + xfs_rtblock_t rt_startrtx; /* starting rt extent number */ + xfs_extlen_t rt_rtxlen; /* number of rt extents */ extent_state_t rt_state; /* see state flags below */ #if 0
@@ -164,11 +164,8 @@ int add_dup_extent(xfs_agnumber_t agno, xfs_agblock_t startblock, xfs_extlen_t blockcount);int search_dup_extent (xfs_agnumber_t agno,xfs_agblock_t start_agbno,xfs_agblock_t end_agbno); -void add_rt_dup_extent (xfs_rtblock_t startblock, -xfs_extlen_t blockcount); - -int search_rt_dup_extent (xfs_mount_t *mp, -xfs_rtblock_t bno); +void add_rt_dup_extent (xfs_rtblock_t startrtx,xfs_extlen_t rtxlen); +int search_rt_dup_extent (struct xfs_mount *mp,xfs_rtblock_t rtx);/* * extent/tree recyling and deletion routines
repair/incore_ext.c +39 -35
148 lines not on this page · raw message
repair/phase4.c +6 -6
@@ -229,7 +229,7 @@ void phase4(xfs_mount_t *mp) {ino_tree_node_t *irec; -xfs_rtblock_t bno; +xfs_rtblock_t rtx;xfs_rtblock_t rt_start;xfs_extlen_t rt_len;xfs_agnumber_t i;
@@ -330,14 +330,14 @@ phase4(xfs_mount_t *mp) rt_start =0 ; rt_len =0 ; -for (bno =0 ; bno < mp->m_sb.sb_rextents; bno++) { - bstate =get_rtbmap (bno); +for (rtx =0 ; rtx < mp->m_sb.sb_rextents; rtx++) { + bstate =get_rtbmap (rtx);switch (bstate) {case XR_E_BAD_STATE :default :do_warn (_ ("unknown rt extent state, extent %" PRIu64"\n" ), - bno); + rtx); fallthrough;case XR_E_METADATA :case XR_E_UNKNOWN :
@@ -360,14 +360,14 @@ phase4(xfs_mount_t *mp)break ;case XR_E_MULT :if (rt_start ==0 ) { - rt_start = bno; + rt_start = rtx; rt_len =1 ; }else if (rt_len ==XFS_MAX_BMBT_EXTLEN ) {/* * large extent case */ add_rt_dup_extent (rt_start, rt_len); - rt_start = bno; + rt_start = rtx; rt_len =1 ; }else rt_len++;
repair/scan.c +4 -2
@@ -418,8 +418,10 @@ _("bad state %d, inode %" PRIu64 " bmap block 0x%" PRIx64 "\n"), XFS_FSB_TO_AGBNO(mp, bno), XFS_FSB_TO_AGBNO(mp, bno) + 1))return (1 ); - }else { -if (search_rt_dup_extent (mp, bno)) + }else { +xfs_rtblock_t rtx = bno / mp->m_sb.sb_rextsize; + +if (search_rt_dup_extent (mp, rtx))return (1 ); } }
Quoted 9 lines · Darrick J. Wong
Correct. The
scan_bmapbtquery is in units of rtblocks. For rextsize==1 there's no error here; for rextsize > 2, thesearch_rt_dup_extentqueries probe well past the end of thert_ext_treestructure, so they never find the duplicate extents.This /does/ explain why the one time I tried crosslinking rt extents I was surprised that
xfs_repairdidn't seem to detect them consistently.Hmm, let me go clean all that up in to a TOTALLY UNTESTED patch.
This does looks sensible, but I suspect it makes more sense to do this after merging the series to introduce xfs_rtxnum_t and friends in the RT code. I'll cook up a patch to just fix scan_bmapbt for now.