Skip to contentSkip to conversation

xfs: fix media verification ioctl for internal rt volumes

PATCH Applied Darrick J. Wong 3 participants 5 messages Aug 21 – Sep 3

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: fix media verification ioctl for internal rt volumes

From: Darrick J. Wong <[email protected]>

A media scan of a filesystem containing an internal rt volume produced an error in xfs_scrub phase 6 complaining about a truncated realtime device. The rt device wasn't truncated, but the media scan code thought we were trying to start a scan past the end of m_rtdev_targp. That in turn is an alias for m_ddev_targp, but in xfs_configure_buftarg we set nr_sectors to the size of the data section. We don't account for an internal realtime section, so the kernel doesn't scan any part of it. Oops.

Reproducer:

 # mkfs.xfs -f /dev/sda -r zoned=1 -d rtinherit=1
 # mount /dev/sda /mnt
 # dd if=/dev/zero of=/mnt/a bs=1024k count=100
 # sync
 # xfs_info /mnt
 meta-data=/dev/sda               isize=512    agcount=4, agsize=32768 blks
          =                       sectsz=512   attr=2, projid32bit=1
          =                       crc=1        finobt=1, sparse=1, rmapbt=1
          =                       reflink=0    bigtime=1 inobtcount=1 nrext64=1
          =                       exchange=1   metadir=1
 data     =                       bsize=4096   blocks=131072, imaxpct=25
          =                       sunit=0      swidth=0 blks
 naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=1
 log      =internal log           bsize=4096   blocks=16384, version=2
          =                       sectsz=512   sunit=0 blks, lazy-count=1
 realtime =internal               extsz=4096   blocks=1114112, rtextents=1114112
          =                       rgcount=17   rgsize=65536 extents
          =                       zoned=1      start=131072 reserved=53248

IOWS: 512M data volume, 3.1G internal rt section. Now let's try some media verification:

 # xfs_io -c 'verifymedia -d' -c 'verifymedia -r' /mnt
 verified 536870912/536870912 bytes at offset 0
 512 MiB, 1 ops; 0.0496 sec (10.067 GiB/sec and 20.1345 ops/sec)
 verified 536870912/536870912 bytes at offset 0
 512 MiB, 1 ops; 0.0409 sec (12.222 GiB/sec and 24.4439 ops/sec)

Notice how xfs_io says we only verified 512M of the rt volume? If you run btrace in the background you'll see that we read the first 512M of the volume (aka the data section) twice and never read anything from the rt section.

An earlier fix tried messing with the buftarg geometry, but I've decided on a more targetted fix for the media verification code. All we have to do is calculate the starting and ending daddr for the device that we're verifying, and clamp the user's input values to that range. This leads to some bogosity in the output reporting:

 # xfs_io -c 'verifymedia -d' -c 'verifymedia -r' /mnt/t
 verified 536870912/536870912 bytes at offset 0
 512 MiB, 1 ops; 0.0606 sec (8.248 GiB/sec and 16.4968 ops/sec)
 verified 5100273664/5100273664 bytes at offset 0
 4.750 GiB, 1 ops; 0.3329 sec (14.267 GiB/sec and 3.0035 ops/sec)

Because we don't have a way to report that we didn't really do anything at all for that first 512M of address space of the rt "device". But at least we're no longer ignoring real media.

(Note that the fsmap/bmap/fiemap calls all report physical addresses for the internal rt volume as offsets from the start of the data device, and the media verifier call consumes the same. We baked that into the user-visible behavior in 6.15, so we're stuck with that sparse hole at the beginning.)

Cc <[email protected]> # v6.15
Fixes bdc03eb5f98f6f ("xfs: allow internal RT devices for zoned mode")
---
 fs/xfs/xfs_verify_media.c |   24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)
fs/xfs/xfs_verify_media.c +17 -7
index 5ead3976d51151..b75c81f8fcc037 100644
@@ -268,6 +268,8 @@ xfs_verify_media(
 	struct xfs_buftarg	*btp = NULL;
 	struct bio		*bio;
 	struct folio		*folio;
+	xfs_daddr_t		dev_start = 0;
+	xfs_daddr_t		dev_end = 0;
 	xfs_daddr_t		daddr;
 	uint64_t		bbcount;
 	int			error = 0;
@@ -277,24 +279,33 @@ xfs_verify_media(
 	switch (me->me_dev) {
 	case XFS_DEV_DATA:
 		btp = mp->m_ddev_targp;
+		dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_dblocks);
 		break;
 	case XFS_DEV_LOG:
-		if (mp->m_logdev_targp != mp->m_ddev_targp)
+		if (mp->m_logdev_targp != mp->m_ddev_targp) {
 			btp = mp->m_logdev_targp;
+			dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_logblocks);
+		}
 		break;
 	case XFS_DEV_RT:
 		btp = mp->m_rtdev_targp;
+		dev_start = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rtstart);
+		dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rtstart +
+					    mp->m_sb.sb_rblocks);
 		break;
 	}
 	if (!btp)
 		return -ENODEV;
 
 	/*
-	 * If the caller told us to verify beyond the end of the disk, tell the
-	 * user exactly where that was.
+	 * If the caller told us to verify before the start or beyond the end
+	 * of the disk volume, tell the user exactly where the volume starts
+	 * and ends.
 	 */
-	if (me->me_end_daddr > btp->bt_nr_sectors)
-		me->me_end_daddr = btp->bt_nr_sectors;
+	if (me->me_end_daddr > dev_end)
+		me->me_end_daddr = dev_end;
+	if (me->me_start_daddr < dev_start)
+		me->me_start_daddr = dev_start;
 
 	/* start and end have to be aligned to the lba size */
 	if (!IS_ALIGNED(BBTOB(me->me_start_daddr | me->me_end_daddr),
@@ -323,8 +334,7 @@ xfs_verify_media(
 	 *    verifying.
 	 */
 	daddr = me->me_start_daddr;
-	bbcount = min_t(sector_t, me->me_end_daddr, btp->bt_nr_sectors) -
-			  me->me_start_daddr;
+	bbcount = me->me_end_daddr - me->me_start_daddr;
 
 	folio = xfs_verify_alloc_folio(xfs_verify_iosize(me, btp, bbcount));
 	if (!folio)
Reviewed-by on v1 1/1
#1 lore raw

Looks good:

I'm starting to wonder if operating on FSBs instead of BBs in the media verification ioctl would clean up much of this. But I guess we care about being able to pinpoint media errors to sub-FSB ranges?

#2 lore raw
Quoted 7 lines · Christoph Hellwig

Looks good:

I'm starting to wonder if operating on FSBs instead of BBs in the media verification ioctl would clean up much of this. But I guess we care about being able to pinpoint media errors to sub-FSB ranges?

It would probably clean things up ... but OTOH all the other userspace APIs use BBs and don't expose the segmented FSB/RTBs to userland.

Anyway thanks for reviewing :)

--D

maintainer Reviewed-by on v1 1/1
#3 lore raw
In reply to Darrick J. Wong
Quoted 68 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

A media scan of a filesystem containing an internal rt volume produced an error in xfs_scrub phase 6 complaining about a truncated realtime device. The rt device wasn't truncated, but the media scan code thought we were trying to start a scan past the end of m_rtdev_targp. That in turn is an alias for m_ddev_targp, but in xfs_configure_buftarg we set nr_sectors to the size of the data section. We don't account for an internal realtime section, so the kernel doesn't scan any part of it. Oops.

Reproducer:

 # mkfs.xfs -f /dev/sda -r zoned=1 -d rtinherit=1
 # mount /dev/sda /mnt
 # dd if=/dev/zero of=/mnt/a bs=1024k count=100
 # sync
 # xfs_info /mnt
 meta-data=/dev/sda               isize=512    agcount=4, agsize=32768 blks
          =                       sectsz=512   attr=2, projid32bit=1
          =                       crc=1        finobt=1, sparse=1, rmapbt=1
          =                       reflink=0    bigtime=1 inobtcount=1 nrext64=1
          =                       exchange=1   metadir=1
 data     =                       bsize=4096   blocks=131072, imaxpct=25
          =                       sunit=0      swidth=0 blks
 naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=1
 log      =internal log           bsize=4096   blocks=16384, version=2
          =                       sectsz=512   sunit=0 blks, lazy-count=1
 realtime =internal               extsz=4096   blocks=1114112, rtextents=1114112
          =                       rgcount=17   rgsize=65536 extents
          =                       zoned=1      start=131072 reserved=53248

IOWS: 512M data volume, 3.1G internal rt section. Now let's try some media verification:

 # xfs_io -c 'verifymedia -d' -c 'verifymedia -r' /mnt
 verified 536870912/536870912 bytes at offset 0
 512 MiB, 1 ops; 0.0496 sec (10.067 GiB/sec and 20.1345 ops/sec)
 verified 536870912/536870912 bytes at offset 0
 512 MiB, 1 ops; 0.0409 sec (12.222 GiB/sec and 24.4439 ops/sec)

Notice how xfs_io says we only verified 512M of the rt volume? If you run btrace in the background you'll see that we read the first 512M of the volume (aka the data section) twice and never read anything from the rt section.

An earlier fix tried messing with the buftarg geometry, but I've decided on a more targetted fix for the media verification code. All we have to do is calculate the starting and ending daddr for the device that we're verifying, and clamp the user's input values to that range. This leads to some bogosity in the output reporting:

 # xfs_io -c 'verifymedia -d' -c 'verifymedia -r' /mnt/t
 verified 536870912/536870912 bytes at offset 0
 512 MiB, 1 ops; 0.0606 sec (8.248 GiB/sec and 16.4968 ops/sec)
 verified 5100273664/5100273664 bytes at offset 0
 4.750 GiB, 1 ops; 0.3329 sec (14.267 GiB/sec and 3.0035 ops/sec)

Because we don't have a way to report that we didn't really do anything at all for that first 512M of address space of the rt "device". But at least we're no longer ignoring real media.

(Note that the fsmap/bmap/fiemap calls all report physical addresses for the internal rt volume as offsets from the start of the data device, and the media verifier call consumes the same. We baked that into the user-visible behavior in 6.15, so we're stuck with that sparse hole at the beginning.)

Quoted 3 lines
Cc <[email protected]> # v6.15
Fixes bdc03eb5f98f6f ("xfs: allow internal RT devices for zoned mode")

Looks good:

Quoted 67 lines
---
 fs/xfs/xfs_verify_media.c |   24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)
fs/xfs/xfs_verify_media.c +17 -7
index 5ead3976d51151..b75c81f8fcc037 100644
@@ -268,6 +268,8 @@ xfs_verify_media(
 	struct xfs_buftarg	*btp = NULL;
 	struct bio		*bio;
 	struct folio		*folio;
+	xfs_daddr_t		dev_start = 0;
+	xfs_daddr_t		dev_end = 0;
 	xfs_daddr_t		daddr;
 	uint64_t		bbcount;
 	int			error = 0;
@@ -277,24 +279,33 @@ xfs_verify_media(
 	switch (me->me_dev) {
 	case XFS_DEV_DATA:
 		btp = mp->m_ddev_targp;
+		dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_dblocks);
 		break;
 	case XFS_DEV_LOG:
-		if (mp->m_logdev_targp != mp->m_ddev_targp)
+		if (mp->m_logdev_targp != mp->m_ddev_targp) {
 			btp = mp->m_logdev_targp;
+			dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_logblocks);
+		}
 		break;
 	case XFS_DEV_RT:
 		btp = mp->m_rtdev_targp;
+		dev_start = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rtstart);
+		dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rtstart +
+					    mp->m_sb.sb_rblocks);
 		break;
 	}
 	if (!btp)
 		return -ENODEV;
 
 	/*
-	 * If the caller told us to verify beyond the end of the disk, tell the
-	 * user exactly where that was.
+	 * If the caller told us to verify before the start or beyond the end
+	 * of the disk volume, tell the user exactly where the volume starts
+	 * and ends.
 	 */
-	if (me->me_end_daddr > btp->bt_nr_sectors)
-		me->me_end_daddr = btp->bt_nr_sectors;
+	if (me->me_end_daddr > dev_end)
+		me->me_end_daddr = dev_end;
+	if (me->me_start_daddr < dev_start)
+		me->me_start_daddr = dev_start;
 
 	/* start and end have to be aligned to the lba size */
 	if (!IS_ALIGNED(BBTOB(me->me_start_daddr | me->me_end_daddr),
@@ -323,8 +334,7 @@ xfs_verify_media(
 	 *    verifying.
 	 */
 	daddr = me->me_start_daddr;
-	bbcount = min_t(sector_t, me->me_end_daddr, btp->bt_nr_sectors) -
-			  me->me_start_daddr;
+	bbcount = me->me_end_daddr - me->me_start_daddr;
 
 	folio = xfs_verify_alloc_folio(xfs_verify_iosize(me, btp, bbcount));
 	if (!folio)
maintainer Applied
#4 lore raw
In reply to Darrick J. Wong
Quoted 10 lines · Darrick J. Wong

A media scan of a filesystem containing an internal rt volume produced an error in xfs_scrub phase 6 complaining about a truncated realtime device. The rt device wasn't truncated, but the media scan code thought we were trying to start a scan past the end of m_rtdev_targp. That in turn is an alias for m_ddev_targp, but in xfs_configure_buftarg we set nr_sectors to the size of the data section. We don't account for an internal realtime section, so the kernel doesn't scan any part of it. Oops.

[...]

Applied to for-next, thanks!

[1/1] xfs: fix media verification ioctl for internal rt volumes

      commit: 153c75d93cd59268f893f3ed66b62cc81428eef4

Best regards,

Signature · 1 line
Carlos Maiolino <[email protected]>