xfs/841: create a block device that must exist
PATCH 1/1 xfs/841: create a block device that must exist
This test currently creates a block device node for /dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:
mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address
Instead, create a node for the backing device for $TEST_DIR, which must exist.
--- tests/xfs/841 | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tests/xfs/841 +4 -1
@@ -85,9 +85,12 @@ _create_proto_dir() $here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true # Block device (requires root) - mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true + # Uses the device for $TEST_DIR to ensure it always exists. + mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR) \ + 2> /dev/null || true # Character device (requires root) + # Uses /dev/null, which should always exist mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true }
Signature · 1 line
2.47.3
Thanks Christoph
Quoted 35 lines · Christoph Hellwig
This test currently creates a block device node for
/dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address
Instead, create a node for the backing device for $TEST_DIR, which must
exist.Signed-off-by Christoph Hellwig <[email protected]>---
tests/xfs/841 | 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)tests/xfs/841 +0
-0@@ -85,9 +85,12 @@ _create_proto_dir()$here/src/af_unix "$PROTO_DIR/socket" 2>
/dev/null|| true
# Block device (requires root)tests/xfs/841 +2
-1- mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true + # Uses the device for $TEST_DIR to ensure it always exists. + mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR)\
tests/xfs/841 +1
-0+ 2> /dev/null || true
# Character device (requires root)tests/xfs/841 +1
-0+ # Uses /dev/null, which should always existmknod "$PROTO_DIR/chardev" c 1 3 2>
/dev/null|| true }
Quoted 11 lines · Christoph Hellwig
This test currently creates a block device node for
/dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address
Instead, create a node for the backing device for $TEST_DIR, which must exist.
Signed-off-by Christoph Hellwig <[email protected]>---
This patch makes sense to me, thanks.
Quoted 25 lines
tests/xfs/841 | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tests/xfs/841 +4
-1@@ -85,9 +85,12 @@ _create_proto_dir() $here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true # Block device (requires root) - mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true + # Uses the device for $TEST_DIR to ensure it always exists. + mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR) \ + 2> /dev/null || true # Character device (requires root) + # Uses /dev/null, which should always exist mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true }Signature · 1 line
2.47.3
Quoted 8 lines · Christoph Hellwig
This test currently creates a block device node for
/dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address
Instead, create a node for the backing device for $TEST_DIR, which must exist.
Hrm. I'm still noticing regressions with this test, particularly when the blocksize of the test filesystem is different from the block size of the $IMG_FILE filesystem.
So I started looking for fsblock discrepancies between xfs_reproducible_test.img.[1-3] and noticed that EOF block contents are different if the file being copied in has sparse holes in it that are not aligned to the fsblock size of the new filesystem.
IOWs, if $TEST_DIR is a 1k fsblock filesystem and the $IMG_FILE filesystems have 4k fsblocks, you can write 1k at odd offsets (e.g. 1k, 3k, 5k, 7k, etc) and the created file does not match the file in the proto dir.
I traced this to design thinko in libxfs_file_write -- it assumes that the caller always wants to write the full block, even if the buffer passed in does not cover the entire range of the block. Therefore it'll zero the head and the tail of the xfs_buf before writing it out.
Thus, the region at 1k causes mkfs to allocate fileoff 0 (which is a 4k block). Then libxfs_file_write writes 1024 bytes of zeroes, 1024 bytes of copied-in data, and 2048 bytes of more zeroes.
Next, the region at 3k causes mkfs to re-call libxfs_file_write, but this time it writes 3072 bytes of zeroes and 1024 bytes of copied-in data, thus obliterating the first write.
That bug's on me, and I'll fix it in writefile by rounding data_pos and hole_pos outward as needed to be aligned to the block size of the copied in filesystem. And I'll update xfs/841 to compare $PROTO_DIR against what's in the new filesystem.
That fixes the data corruption problem, but then the test still fails because now the space map isn't the same between mkfs invocations.
--D
Quoted 28 lines
Signed-off-by Christoph Hellwig <[email protected]>--- tests/xfs/841 | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)tests/xfs/841 +4
-1@@ -85,9 +85,12 @@ _create_proto_dir() $here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true # Block device (requires root) - mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true + # Uses the device for $TEST_DIR to ensure it always exists. + mknod "$PROTO_DIR/blockdev" b $(stat -c '%Hd %Ld' $TEST_DIR) \ + 2> /dev/null || true # Character device (requires root) + # Uses /dev/null, which should always exist mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true }Signature · 1 line
2.47.3
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Christoph Hellwig
This test currently creates a block device node for
/dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:
Quoted 8 lines
Quoted 4 lines
mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address
Instead, create a node for the backing device for $TEST_DIR, which must exist.
Hrm. I'm still noticing regressions with this test, particularly when the blocksize of the test filesystem is different from the block size of the $IMG_FILE filesystem.
That is with the test in general, and not because of the block device fix, right? Your description seems to indicate that, I'm just a bit confused as it is replying to my incremental patch.
Quoted 4 lines
So I started looking for fsblock discrepancies between xfs_reproducible_test.img.[1-3] and noticed that EOF block contents are different if the file being copied in has sparse holes in it that are not aligned to the fsblock size of the new filesystem.
Oooh.
Quoted 5 lines
Next, the region at 3k causes mkfs to re-call
libxfs_file_write, but this time it writes 3072 bytes of zeroes and 1024 bytes of copied-in data, thus obliterating the first write.That bug's on me,
Yeah.
Quoted 7 lines
and I'll fix it in writefile by rounding
data_posandhole_posoutward as needed to be aligned to the block size of the copied in filesystem. And I'll update xfs/841 to compare $PROTO_DIR against what's in the new filesystem.That fixes the data corruption problem, but then the test still fails because now the space map isn't the same between mkfs invocations.
Aarg. But I'm glad we got a test for this feature that's uncovering old buggy/sloppy libxfs code..
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · Darrick J. Wong
Quoted 8 lines · Christoph Hellwig
This test currently creates a block device node for
/dev/ram0, which isn't guaranteed to exist, and can thus cause the test to fail with:mkfs.xfs: cannot open $TEST_DIR/proto/blockdev: No such device or address
Instead, create a node for the backing device for $TEST_DIR, which must exist.
Quoted 7 lines
Quoted 3 lines
Hrm. I'm still noticing regressions with this test, particularly when the blocksize of the test filesystem is different from the block size of the $IMG_FILE filesystem.
That is with the test in general, and not because of the block device fix, right? Your description seems to indicate that, I'm just a bit confused as it is replying to my incremental patch.
Correct, I'm just complaining about x841 in general. Your bdev fix eliminated one of the sources of test failures.
Quoted 21 lines
Quoted 4 lines
So I started looking for fsblock discrepancies between xfs_reproducible_test.img.[1-3] and noticed that EOF block contents are different if the file being copied in has sparse holes in it that are not aligned to the fsblock size of the new filesystem.
Oooh.
Quoted 5 lines
Next, the region at 3k causes mkfs to re-call
libxfs_file_write, but this time it writes 3072 bytes of zeroes and 1024 bytes of copied-in data, thus obliterating the first write.That bug's on me,
Yeah.
Quoted 5 lines
and I'll fix it in writefile by rounding
data_posandhole_posoutward as needed to be aligned to the block size of the copied in filesystem. And I'll update xfs/841 to compare $PROTO_DIR against what's in the new filesystem.
Quoted 5 lines
Quoted 2 lines
That fixes the data corruption problem, but then the test still fails because now the space map isn't the same between mkfs invocations.
Aarg. But I'm glad we got a test for this feature that's uncovering old buggy/sloppy libxfs code..
Yeah. I missed the detail that none of the existing protofile tests actually tried to import sparse files, let alone checked the results.
I'll try to send the fix patches in a bit but QA blew up last night so I should probably go figure out why there's suddenly log corruption, or if some cloud thing got really FUBARd at 17:08 last night.
--D