generic: shutdown might leave NULL files with nonzero di_size
PATCH 1/1 generic: shutdown might leave NULL files with nonzero di_size
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their di_size > 0.
---
Hi,
There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to fstests. I've tried generic/044~049, no one can reproduce this bug, so I have to write this new one. It fails on rhel-6.5 [1], and test passed on later kernel.
I hard to say which patch fix this issue exactly, it's fixed by a patchset which does code improvement/cleanup.
Thanks,
Zorro
[1]
# ./check generic/999 FSTYP -- xfs (non-debug) PLATFORM -- Linux/x86_64 MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1 MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratch
generic/999 2s ... - output mismatch (see /root/xfstests-dev/results//generic/999.out.bad)
--- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400
+++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400
@@ -1 +1,3 @@
QA output created by 999
+ - /mnt/scratch/1 get no extents, but its di_size > 0
+/mnt/scratch/1:
...
(Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)Ran: generic/999
Failures: generic/999
Failed 1 of 1 tests
tests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 51 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.out
tests/generic/999 new file +46 -0
@@ -0,0 +1,46 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_xfs_io_command fiemap +_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount" +_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap + # Check if the file has any extent + grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap + if [ $? -ne 0 ];then + echo " - $f get no extents, but its di_size > 0" + cat $tmp.fiemap + break + fi +done + +# success, all done +status=0 +exit
tests/generic/999.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extents
Signature · 1 line
2.31.1
Quoted 58 lines · Zorro Lang
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their
di_size> 0.Signed-off-by Zorro Lang <[email protected]>---
Hi,
There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to fstests. I've tried generic/044~049, no one can reproduce this bug, so I have to write this new one. It fails on rhel-6.5 [1], and test passed on later kernel.
I hard to say which patch fix this issue exactly, it's fixed by a patchset which does code improvement/cleanup.
Thanks,
Zorro[1]
# ./check generic/999 FSTYP -- xfs (non-debug) PLATFORM -- Linux/x86_64 MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1 MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratchgeneric/999 2s ... - output mismatch (see
/root/xfstests-dev/results//generic/999.out.bad)--- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400 +++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400 @@ -1 +1,3 @@ QA output created by 999 + - /mnt/scratch/1 get no extents, but its di_size > 0 +/mnt/scratch/1: ... (Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)Ran: generic/999
Failures: generic/999
Failed 1 of 1 teststests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 51 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.outtests/generic/999 new file +6
-0@@ -0,0 +1,46 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +#
tests/generic/999 line 16
Quoted 10 lines
tests/generic/999 new file +10
-0+# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_xfs_io_command fiemap
/me would've thought you'd use the xfs_io stat/bmap commands to detect either nextents > 0 (stat) or actual mappings returned (bmap), but I guess if RHEL 6.5 xfsprogs has a fiemap command then this is fine with me.
If the answer to the above is "um, RHEL 6.5 xfsprogs *does* have FIEMAP", then there's little point in rewriting a stable regression test, so:
--D
Quoted 44 lines
tests/generic/999 new file +30
-0+_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount" +_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap + # Check if the file has any extent + grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap + if [ $? -ne 0 ];then + echo " - $f get no extents, but its di_size > 0" + cat $tmp.fiemap + break + fi +done + +# success, all done +status=0 +exittests/generic/999.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extentsSignature · 1 line
2.31.1
Quoted 66 lines · Darrick J. Wong
Quoted 65 lines · Zorro Lang
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their
di_size> 0.Signed-off-by Zorro Lang <[email protected]>---
Hi,
There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to fstests. I've tried generic/044~049, no one can reproduce this bug, so I have to write this new one. It fails on rhel-6.5 [1], and test passed on later kernel.
I hard to say which patch fix this issue exactly, it's fixed by a patchset which does code improvement/cleanup.
Thanks,
Zorro[1]
# ./check generic/999 FSTYP -- xfs (non-debug) PLATFORM -- Linux/x86_64 MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1 MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratchgeneric/999 2s ... - output mismatch (see
/root/xfstests-dev/results//generic/999.out.bad)--- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400 +++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400 @@ -1 +1,3 @@ QA output created by 999 + - /mnt/scratch/1 get no extents, but its di_size > 0 +/mnt/scratch/1: ... (Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)Ran: generic/999
Failures: generic/999
Failed 1 of 1 teststests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 51 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.outtests/generic/999 new file +13
-0@@ -0,0 +1,46 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here
Quoted 8 lines
tests/generic/999 line 16
Quoted 3 lines
tests/generic/999 new file +3
-0+_supported_fs xfs +_require_scratch +_require_xfs_io_command fiemap/me would've thought you'd use the
xfs_iostat/bmap commands to detect either nextents > 0 (stat) or actual mappings returned (bmap), but I guess if RHEL 6.5 xfsprogs has a fiemap command then this is fine with me.
Ah, you're right, rhel-6 xfs_io doesn't support fiemap :)
And yes, I wrote this case as a xfs specific case at first, by using xfs_bmap. The original case (of us) uses xfs_bmap too. But I thought fiemap can help it to be a generic case to cover more fs, so I turn to use fiemap.
Quoted 3 lines
If the answer to the above is "um, RHEL 6.5 xfsprogs *does* have FIEMAP", then there's little point in rewriting a stable regression test, so:
I don't worry about rhel6 testing, due to I keep our internal case (for rhel6 only). I'd like to see more generic cases, except fiemap can't help this test :)
Thanks,
Zorro
Quoted 50 lines
Reviewed-by Darrick J. Wong <[email protected]>--D
Quoted 44 lines
tests/generic/999 new file +30
-0+_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount" +_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap + # Check if the file has any extent + grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap + if [ $? -ne 0 ];then + echo " - $f get no extents, but its di_size > 0" + cat $tmp.fiemap + break + fi +done + +# success, all done +status=0 +exittests/generic/999.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extentsSignature · 1 line
2.31.1
Quoted 65 lines · Zorro Lang
Quoted 64 lines · Darrick J. Wong
Quoted 63 lines · Zorro Lang
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their
di_size> 0.Signed-off-by Zorro Lang <[email protected]>---
Hi,
There was an very old xfs bug on rhel-6.5, I'd like to share its reproducer to fstests. I've tried generic/044~049, no one can reproduce this bug, so I have to write this new one. It fails on rhel-6.5 [1], and test passed on later kernel.
I hard to say which patch fix this issue exactly, it's fixed by a patchset which does code improvement/cleanup.
Thanks,
Zorro[1]
# ./check generic/999 FSTYP -- xfs (non-debug) PLATFORM -- Linux/x86_64 MKFS_OPTIONS -- -f -bsize=4096 /dev/loop1 MOUNT_OPTIONS -- -o context=system_u:object_r:nfs_t:s0 /dev/loop1 /mnt/scratchgeneric/999 2s ... - output mismatch (see
/root/xfstests-dev/results//generic/999.out.bad)--- tests/generic/999.out 2022-11-04 00:54:11.123353054 -0400 +++ /root/xfstests-dev/results//generic/999.out.bad 2022-11-04 04:24:57.861673433 -0400 @@ -1 +1,3 @@ QA output created by 999 + - /mnt/scratch/1 get no extents, but its di_size > 0 +/mnt/scratch/1: ... (Run 'diff -u tests/generic/045.out /root/xfstests-dev/results//generic/999.out.bad' to see the entire diff)Ran: generic/999
Failures: generic/999
Failed 1 of 1 teststests/generic/999 | 46 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 51 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.outtests/generic/999 new file +11
-0@@ -0,0 +1,46 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown
Quoted 8 lines
Quoted 8 lines
tests/generic/999 line 16
Quoted 5 lines
tests/generic/999 new file +5
-0+ +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_xfs_io_command fiemap/me would've thought you'd use the
xfs_iostat/bmap commands to detect either nextents > 0 (stat) or actual mappings returned (bmap), but I
Wait a minute... the "stat" command can help this case too?
I didn't use bmap command due to it only for xfs, but stat is a common command, if it helps, that's better than fiemap (due to it supports old system). I'll give it a test, and will resend the patch if it works. Thanks your suggestion!
Quoted 69 lines
Quoted 2 lines
guess if RHEL 6.5 xfsprogs has a fiemap command then this is fine with me.
Ah, you're right, rhel-6
xfs_iodoesn't support fiemap :)And yes, I wrote this case as a xfs specific case at first, by using
xfs_bmap. The original case (of us) usesxfs_bmaptoo. But I thought fiemap can help it to be a generic case to cover more fs, so I turn to use fiemap.Quoted 3 lines
If the answer to the above is "um, RHEL 6.5 xfsprogs *does* have FIEMAP", then there's little point in rewriting a stable regression test, so:
I don't worry about rhel6 testing, due to I keep our internal case (for rhel6 only). I'd like to see more generic cases, except fiemap can't help this test :)
Thanks,
ZorroQuoted 50 lines
Reviewed-by Darrick J. Wong <[email protected]>--D
Quoted 44 lines
tests/generic/999 new file +30
-0+_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount" +_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + $XFS_IO_PROG -c "fiemap" $f > $tmp.fiemap + # Check if the file has any extent + grep -Eq '^[[:space:]]+[0-9]+:' $tmp.fiemap + if [ $? -ne 0 ];then + echo " - $f get no extents, but its di_size > 0" + cat $tmp.fiemap + break + fi +done + +# success, all done +status=0 +exittests/generic/999.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extentsSignature · 1 line
2.31.1