Skip to contentSkip to conversation

xfs/841: create a block device that must exist

PATCH In review Christoph Hellwig 4 participants 6 messages Feb 2 – Mar 4

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs/841: create a block device that must exist

This test currently creates a block device node for /dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:

mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address

Instead, create a node for the backing device for $TEST_DIR, which must exist.

---
 tests/xfs/841 | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tests/xfs/841 +4 -1
index ee2368d4a746..ddb1b3bea104 100755
@@ -85,9 +85,12 @@ _create_proto_dir()
 	$here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true
 
 	# Block device (requires root)
-	mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true
+	# Uses the device for $TEST_DIR to ensure it always exists.
+	mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR) \
+		2> /dev/null || true
 
 	# Character device (requires root)
+	# Uses /dev/null, which should always exist
 	mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true
 }
 
Signature · 1 line
2.47.3
Reviewed-by on v1 1/1
#1 lore raw

Thanks Christoph

Quoted 35 lines · Christoph Hellwig

This test currently creates a block device node for /dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:

mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address

Instead, create a node for the backing device for $TEST_DIR, which must
exist.

---
 tests/xfs/841 | 5 ++++-

 1 file changed, 4 insertions(+), 1 deletion(-)
tests/xfs/841 +0 -0
index ee2368d4a746..ddb1b3bea104 100755
@@ -85,9 +85,12 @@ _create_proto_dir()

  $here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true
  # Block device (requires root)

tests/xfs/841 +2 -1
-	mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true
+	# Uses the device for $TEST_DIR to ensure it always exists.
+	mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR)

\

tests/xfs/841 +1 -0
+		2> /dev/null || true


  # Character device (requires root)

tests/xfs/841 +1 -0
+	# Uses /dev/null, which should always exist

  mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true }

Reviewed-by on v1 1/1
#2 lore raw
In reply to Christoph Hellwig
Quoted 11 lines · Christoph Hellwig

This test currently creates a block device node for /dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:

mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address

Instead, create a node for the backing device for $TEST_DIR, which must exist.

---

This patch makes sense to me, thanks.

Quoted 25 lines
 tests/xfs/841 | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tests/xfs/841 +4 -1
index ee2368d4a746..ddb1b3bea104 100755
@@ -85,9 +85,12 @@ _create_proto_dir()
 	$here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true
 
 	# Block device (requires root)
-	mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true
+	# Uses the device for $TEST_DIR to ensure it always exists.
+	mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR) \
+		2> /dev/null || true
 
 	# Character device (requires root)
+	# Uses /dev/null, which should always exist
 	mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true
 }
 
Signature · 1 line
2.47.3
#3 lore raw
In reply to Christoph Hellwig
Quoted 8 lines · Christoph Hellwig

This test currently creates a block device node for /dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:

mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address

Instead, create a node for the backing device for $TEST_DIR, which must exist.

Hrm. I'm still noticing regressions with this test, particularly when the blocksize of the test filesystem is different from the block size of the $IMG_FILE filesystem.

So I started looking for fsblock discrepancies between xfs_reproducible_test.img.[1-3] and noticed that EOF block contents are different if the file being copied in has sparse holes in it that are not aligned to the fsblock size of the new filesystem.

IOWs, if $TEST_DIR is a 1k fsblock filesystem and the $IMG_FILE filesystems have 4k fsblocks, you can write 1k at odd offsets (e.g. 1k, 3k, 5k, 7k, etc) and the created file does not match the file in the proto dir.

I traced this to design thinko in libxfs_file_write -- it assumes that the caller always wants to write the full block, even if the buffer passed in does not cover the entire range of the block. Therefore it'll zero the head and the tail of the xfs_buf before writing it out.

Thus, the region at 1k causes mkfs to allocate fileoff 0 (which is a 4k block). Then libxfs_file_write writes 1024 bytes of zeroes, 1024 bytes of copied-in data, and 2048 bytes of more zeroes.

Next, the region at 3k causes mkfs to re-call libxfs_file_write, but this time it writes 3072 bytes of zeroes and 1024 bytes of copied-in data, thus obliterating the first write.

That bug's on me, and I'll fix it in writefile by rounding data_pos and hole_pos outward as needed to be aligned to the block size of the copied in filesystem. And I'll update xfs/841 to compare $PROTO_DIR against what's in the new filesystem.

That fixes the data corruption problem, but then the test still fails because now the space map isn't the same between mkfs invocations.

--D

Quoted 28 lines
---
 tests/xfs/841 | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tests/xfs/841 +4 -1
index ee2368d4a746..ddb1b3bea104 100755
@@ -85,9 +85,12 @@ _create_proto_dir()
 	$here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true
 
 	# Block device (requires root)
-	mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true
+	# Uses the device for $TEST_DIR to ensure it always exists.
+	mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR) \
+		2> /dev/null || true
 
 	# Character device (requires root)
+	# Uses /dev/null, which should always exist
 	mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true
 }
 
Signature · 1 line
2.47.3
#4 lore raw
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Christoph Hellwig

This test currently creates a block device node for /dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:

Quoted 8 lines
Quoted 4 lines

mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address

Instead, create a node for the backing device for $TEST_DIR, which must exist.

Hrm. I'm still noticing regressions with this test, particularly when the blocksize of the test filesystem is different from the block size of the $IMG_FILE filesystem.

That is with the test in general, and not because of the block device fix, right? Your description seems to indicate that, I'm just a bit confused as it is replying to my incremental patch.

Quoted 4 lines

So I started looking for fsblock discrepancies between xfs_reproducible_test.img.[1-3] and noticed that EOF block contents are different if the file being copied in has sparse holes in it that are not aligned to the fsblock size of the new filesystem.

Oooh.

Quoted 5 lines

Next, the region at 3k causes mkfs to re-call libxfs_file_write, but this time it writes 3072 bytes of zeroes and 1024 bytes of copied-in data, thus obliterating the first write.

That bug's on me,

Yeah.

Quoted 7 lines

and I'll fix it in writefile by rounding data_pos and hole_pos outward as needed to be aligned to the block size of the copied in filesystem. And I'll update xfs/841 to compare $PROTO_DIR against what's in the new filesystem.

That fixes the data corruption problem, but then the test still fails because now the space map isn't the same between mkfs invocations.

Aarg. But I'm glad we got a test for this feature that's uncovering old buggy/sloppy libxfs code..

#5 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 8 lines · Christoph Hellwig

This test currently creates a block device node for /dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:

mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address

Instead, create a node for the backing device for $TEST_DIR, which must exist.

Quoted 7 lines
Quoted 3 lines

Hrm. I'm still noticing regressions with this test, particularly when the blocksize of the test filesystem is different from the block size of the $IMG_FILE filesystem.

That is with the test in general, and not because of the block device fix, right? Your description seems to indicate that, I'm just a bit confused as it is replying to my incremental patch.

Correct, I'm just complaining about x841 in general. Your bdev fix eliminated one of the sources of test failures.

Quoted 21 lines
Quoted 4 lines

So I started looking for fsblock discrepancies between xfs_reproducible_test.img.[1-3] and noticed that EOF block contents are different if the file being copied in has sparse holes in it that are not aligned to the fsblock size of the new filesystem.

Oooh.

Quoted 5 lines

Next, the region at 3k causes mkfs to re-call libxfs_file_write, but this time it writes 3072 bytes of zeroes and 1024 bytes of copied-in data, thus obliterating the first write.

That bug's on me,

Yeah.

Quoted 5 lines

and I'll fix it in writefile by rounding data_pos and hole_pos outward as needed to be aligned to the block size of the copied in filesystem. And I'll update xfs/841 to compare $PROTO_DIR against what's in the new filesystem.

Quoted 5 lines
Quoted 2 lines

That fixes the data corruption problem, but then the test still fails because now the space map isn't the same between mkfs invocations.

Aarg. But I'm glad we got a test for this feature that's uncovering old buggy/sloppy libxfs code..

Yeah. I missed the detail that none of the existing protofile tests actually tried to import sparse files, let alone checked the results.

I'll try to send the fix patches in a bit but QA blew up last night so I should probably go figure out why there's suddenly log corruption, or if some cloud thing got really FUBARd at 17:08 last night.

--D