Skip to contentSkip to conversation

repair: fix process_rt_rec_dups

PATCH In review Christoph Hellwig 2 participants 5 messages Nov 8, 2023 – Nov 9, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 repair: fix process_rt_rec_dups

search_rt_dup_extent takes a xfs_rtblock_t, not an RT extent number.

---

What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.

 repair/dinode.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
repair/dinode.c +1 -3
index c10dd1fa3..9aa367138 100644
@@ -194,13 +194,11 @@ process_rt_rec_dups(
 	struct xfs_bmbt_irec	*irec)
 {
 	xfs_fsblock_t		b;
-	xfs_rtblock_t		ext;
 
 	for (b = rounddown(irec->br_startblock, mp->m_sb.sb_rextsize);
 	     b < irec->br_startblock + irec->br_blockcount;
 	     b += mp->m_sb.sb_rextsize) {
-		ext = (xfs_rtblock_t) b / mp->m_sb.sb_rextsize;
-		if (search_rt_dup_extent(mp, ext))  {
+		if (search_rt_dup_extent(mp, b))  {
 			do_warn(
 _("data fork in rt ino %" PRIu64 " claims dup rt extent,"
 "off - %" PRIu64 ", start - %" PRIu64 ", count %" PRIu64 "\n"),
Signature · 1 line
2.39.2
#1 lore raw
Quoted 8 lines · Christoph Hellwig

search_rt_dup_extent takes a xfs_rtblock_t, not an RT extent number.

---

What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.

I think you've misread the code -- phase 4 builds the rt_dup tree by walks all the rtextents, and adding the duplicates:

	for (rtx = 0; rtx < mp->m_sb.sb_rextents; rtx++)  {
		bstate = get_rtbmap(rtx);
		switch (bstate)  {
			...
		case XR_E_FS_MAP:
			if (rt_start == 0)
				continue;
			else  {
				/*
				 * add extent and reset extent state
				 */
				add_rt_dup_extent(rt_start, rt_len);
				rt_start = 0;
				rt_len = 0;
			}
			break;
		case XR_E_MULT:
			if (rt_start == 0)  {
				rt_start = rtx;
				rt_len = 1;
			} else if (rt_len == XFS_MAX_BMBT_EXTLEN)  {
				/*
				 * large extent case
				 */
				add_rt_dup_extent(rt_start, rt_len);
				rt_start = rtx;
				rt_len = 1;
			} else
				rt_len++;
			break;

So I think the reason why you've never seen false duplicates is that the rt_dup tree intervals measure rt extents, not rt blocks. The units conversion in process_rt_rec_dups is correct.

However, none of that is at all obvious because of the dual uses of xfs_rtblock_t for rt blocks and rt extents.

I guess I ought to post the xfsprogs version of those rt units cleanups. :)

--D

Quoted 25 lines
 repair/dinode.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
repair/dinode.c +1 -3
index c10dd1fa3..9aa367138 100644
@@ -194,13 +194,11 @@ process_rt_rec_dups(
 	struct xfs_bmbt_irec	*irec)
 {
 	xfs_fsblock_t		b;
-	xfs_rtblock_t		ext;
 
 	for (b = rounddown(irec->br_startblock, mp->m_sb.sb_rextsize);
 	     b < irec->br_startblock + irec->br_blockcount;
 	     b += mp->m_sb.sb_rextsize) {
-		ext = (xfs_rtblock_t) b / mp->m_sb.sb_rextsize;
-		if (search_rt_dup_extent(mp, ext))  {
+		if (search_rt_dup_extent(mp, b))  {
 			do_warn(
 _("data fork in rt ino %" PRIu64 " claims dup rt extent,"
 "off - %" PRIu64 ", start - %" PRIu64 ", count %" PRIu64 "\n"),
Signature · 1 line
2.39.2
#2 lore raw
Quoted 6 lines · Darrick J. Wong
Quoted 5 lines · Christoph Hellwig

search_rt_dup_extent takes a xfs_rtblock_t, not an RT extent number.

---

Quoted 6 lines
Quoted 3 lines

What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.

I think you've misread the code -- phase 4 builds the rt_dup tree by walks all the rtextents, and adding the duplicates:

Hmm.

So yes, add_rt_dup_extent seems to be called on an actual rtext, but scan_bmapbt calls search_rt_dup_extent with what is clearly
a fsbno_t. So something is fishy here for sure..

Quoted 3 lines

So I think the reason why you've never seen false duplicates is that the rt_dup tree intervals measure rt extents, not rt blocks. The units conversion in process_rt_rec_dups is correct.

Note that I don't see error with the patch either, so either way the coverage isn't good enough..

#3 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 8 lines · Christoph Hellwig

search_rt_dup_extent takes a xfs_rtblock_t, not an RT extent number.

---

What scares me about this is that no test seems to hit this and report false duplicates. I'll need to see if I can come up with an artifical reproducers of some kind.

Quoted 8 lines
Quoted 2 lines

I think you've misread the code -- phase 4 builds the rt_dup tree by walks all the rtextents, and adding the duplicates:

Hmm.

So yes, add_rt_dup_extent seems to be called on an actual rtext, but scan_bmapbt calls search_rt_dup_extent with what is clearly
a fsbno_t. So something is fishy here for sure..

Yes, I just noticed that scam_bmapbt thing. Yikes.

Quoted 6 lines
Quoted 3 lines

So I think the reason why you've never seen false duplicates is that the rt_dup tree intervals measure rt extents, not rt blocks. The units conversion in process_rt_rec_dups is correct.

Note that I don't see error with the patch either, so either way the coverage isn't good enough..

Correct. The scan_bmapbt query is in units of rtblocks. For rextsize==1 there's no error here; for rextsize > 2, the search_rt_dup_extent queries probe well past the end of the rt_ext_tree structure, so they never find the duplicate extents.

This /does/ explain why the one time I tried crosslinking rt extents I was surprised that xfs_repair didn't seem to detect them consistently.

Hmm, let me go clean all that up in to a TOTALLY UNTESTED patch.

--D

From: Darrick J. Wong <[email protected]>
Subject: [PATCH] xfs_repair: fix confusing rt space units in the duplicate detection code

Christoph Hellwig stumbled over the crosslinked file data detection code in xfs_repair. While trying to make sense of his fixpatch, I realized that the variable names and unit types are very misleading.

The rt dup tree builder inserts records in units of realtime extents. One query of the rt dup tree passes in a realtime extent number, but one of them does not. Confusingly, all the variable names have "block" even though they really mean "extent". This makes a real difference for rextsize > 1 filesystems, though given the lack of complaints I'm guessing there aren't many users.

Clean up this whole mess by fixing the variable names of the duplicates tree and the state array to reflect the units that are stored in the data structure, and fix the buggy query code. Later on in this patchset we'll fix the variable types too.

This seems to have been broken since before the start of the git repo.

---
 repair/incore.c     |   16 ++++++-----
 repair/incore.h     |   15 ++++------
 repair/incore_ext.c |   74 +++++++++++++++++++++++++++------------------------
 repair/phase4.c     |   12 ++++----
 repair/scan.c       |    6 +++-
 5 files changed, 63 insertions(+), 60 deletions(-)
repair/incore.c +8 -8
index f7a89e70d91..7f4a52bf7de 100644
@@ -178,21 +178,21 @@ static size_t		rt_bmap_size;
  */
 int
 get_rtbmap(
-	xfs_rtblock_t	bno)
+	xfs_rtblock_t	rtx)
 {
-	return (*(rt_bmap + bno /  XR_BB_NUM) >>
-		((bno % XR_BB_NUM) * XR_BB)) & XR_BB_MASK;
+	return (*(rt_bmap + rtx /  XR_BB_NUM) >>
+		((rtx % XR_BB_NUM) * XR_BB)) & XR_BB_MASK;
 }
 
 void
 set_rtbmap(
-	xfs_rtblock_t	bno,
+	xfs_rtblock_t	rtx,
 	int		state)
 {
-	*(rt_bmap + bno / XR_BB_NUM) =
-	 ((*(rt_bmap + bno / XR_BB_NUM) &
-	  (~((uint64_t) XR_BB_MASK << ((bno % XR_BB_NUM) * XR_BB)))) |
-	 (((uint64_t) state) << ((bno % XR_BB_NUM) * XR_BB)));
+	*(rt_bmap + rtx / XR_BB_NUM) =
+	 ((*(rt_bmap + rtx / XR_BB_NUM) &
+	  (~((uint64_t) XR_BB_MASK << ((rtx % XR_BB_NUM) * XR_BB)))) |
+	 (((uint64_t) state) << ((rtx % XR_BB_NUM) * XR_BB)));
 }
 
 static void
repair/incore.h +6 -9
index 53609f683af..7e9bc127f45 100644
@@ -28,8 +28,8 @@ void		set_bmap_ext(xfs_agnumber_t agno, xfs_agblock_t agbno,
 int		get_bmap_ext(xfs_agnumber_t agno, xfs_agblock_t agbno,
 			     xfs_agblock_t maxbno, xfs_extlen_t *blen);
 
-void		set_rtbmap(xfs_rtblock_t bno, int state);
-int		get_rtbmap(xfs_rtblock_t bno);
+void		set_rtbmap(xfs_rtblock_t rtx, int state);
+int		get_rtbmap(xfs_rtblock_t rtx);
 
 static inline void
 set_bmap(xfs_agnumber_t agno, xfs_agblock_t agbno, int state)
@@ -70,8 +70,8 @@ typedef struct extent_tree_node  {
 
 typedef struct rt_extent_tree_node  {
 	avlnode_t		avl_node;
-	xfs_rtblock_t		rt_startblock;	/* starting realtime block */
-	xfs_extlen_t		rt_blockcount;	/* number of blocks in extent */
+	xfs_rtblock_t		rt_startrtx;	/* starting rt extent number */
+	xfs_extlen_t		rt_rtxlen;	/* number of rt extents */
 	extent_state_t		rt_state;	/* see state flags below */
 
 #if 0
@@ -164,11 +164,8 @@ int		add_dup_extent(xfs_agnumber_t agno, xfs_agblock_t startblock,
 			xfs_extlen_t blockcount);
 int		search_dup_extent(xfs_agnumber_t agno,
 			xfs_agblock_t start_agbno, xfs_agblock_t end_agbno);
-void		add_rt_dup_extent(xfs_rtblock_t	startblock,
-				xfs_extlen_t	blockcount);
-
-int		search_rt_dup_extent(xfs_mount_t	*mp,
-					xfs_rtblock_t	bno);
+void		add_rt_dup_extent(xfs_rtblock_t startrtx, xfs_extlen_t rtxlen);
+int		search_rt_dup_extent(struct xfs_mount *mp, xfs_rtblock_t rtx);
 
 /*
  * extent/tree recyling and deletion routines
repair/incore_ext.c +39 -35

148 lines not on this page · raw message

repair/phase4.c +6 -6
index 28ecf56f45b..93adff1786f 100644
@@ -229,7 +229,7 @@ void
 phase4(xfs_mount_t *mp)
 {
 	ino_tree_node_t		*irec;
-	xfs_rtblock_t		bno;
+	xfs_rtblock_t		rtx;
 	xfs_rtblock_t		rt_start;
 	xfs_extlen_t		rt_len;
 	xfs_agnumber_t		i;
@@ -330,14 +330,14 @@ phase4(xfs_mount_t *mp)
 	rt_start = 0;
 	rt_len = 0;
 
-	for (bno = 0; bno < mp->m_sb.sb_rextents; bno++)  {
-		bstate = get_rtbmap(bno);
+	for (rtx = 0; rtx < mp->m_sb.sb_rextents; rtx++)  {
+		bstate = get_rtbmap(rtx);
 		switch (bstate)  {
 		case XR_E_BAD_STATE:
 		default:
 			do_warn(
 	_("unknown rt extent state, extent %" PRIu64 "\n"),
-				bno);
+				rtx);
 			fallthrough;
 		case XR_E_METADATA:
 		case XR_E_UNKNOWN:
@@ -360,14 +360,14 @@ phase4(xfs_mount_t *mp)
 			break;
 		case XR_E_MULT:
 			if (rt_start == 0)  {
-				rt_start = bno;
+				rt_start = rtx;
 				rt_len = 1;
 			} else if (rt_len == XFS_MAX_BMBT_EXTLEN)  {
 				/*
 				 * large extent case
 				 */
 				add_rt_dup_extent(rt_start, rt_len);
-				rt_start = bno;
+				rt_start = rtx;
 				rt_len = 1;
 			} else
 				rt_len++;
repair/scan.c +4 -2
index 9a5edb40c25..c91f4c3fe71 100644
@@ -418,8 +418,10 @@ _("bad state %d, inode %" PRIu64 " bmap block 0x%" PRIx64 "\n"),
 					XFS_FSB_TO_AGBNO(mp, bno),
 					XFS_FSB_TO_AGBNO(mp, bno) + 1))
 				return(1);
-		} else  {
-			if (search_rt_dup_extent(mp, bno))
+		} else {
+			xfs_rtblock_t	rtx = bno / mp->m_sb.sb_rextsize;
+
+			if (search_rt_dup_extent(mp, rtx))
 				return(1);
 		}
 	}
#4 lore raw
Quoted 9 lines · Darrick J. Wong

Correct. The scan_bmapbt query is in units of rtblocks. For rextsize==1 there's no error here; for rextsize > 2, the search_rt_dup_extent queries probe well past the end of the rt_ext_tree structure, so they never find the duplicate extents.

This /does/ explain why the one time I tried crosslinking rt extents I was surprised that xfs_repair didn't seem to detect them consistently.

Hmm, let me go clean all that up in to a TOTALLY UNTESTED patch.

This does looks sensible, but I suspect it makes more sense to do this after merging the series to introduce xfs_rtxnum_t and friends in the RT code. I'll cook up a patch to just fix scan_bmapbt for now.