generic: shutdown might leave NULL files with nonzero di_size
PATCH v2 1/1 generic: shutdown might leave NULL files with nonzero di_size
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their di_size > 0.
---
V2 replace "fiemap" with "stat" command, to check if a file has extents. That helps this case more common.
Thanks,
Zorro
tests/generic/999 | 42 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 47 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.out
tests/generic/999 new file +42 -0
@@ -0,0 +1,42 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount" +_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + # Check if the file has any extent + if [ "$(stat -c "%b" $f)" = "0" ];then + echo " - $f get no extents, but its di_size > 0" + break + fi +done + +# success, all done +status=0 +exit
tests/generic/999.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extents
Signature · 1 line
2.31.1
tests/generic/999
Quoted 24 lines · Zorro Lang
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their
di_size> 0.Signed-off-by Zorro Lang <[email protected]>---
V2 replace "fiemap" with "stat" command, to check if a file has extents. That helps this case more common.
Thanks,
Zorrotests/generic/999 | 42 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 47 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.outtests/generic/999 new file +0
-0
Ugh sorry ^^^^^^^ I didn't notice this part and wrote my previous
response thinking this was an xfs-only test...
Quoted 29 lines
tests/generic/999 new file +28
-0@@ -0,0 +1,42 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount"
tests/generic/999 line 38
Quoted 10 lines
tests/generic/999 new file +10
-0+_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + # Check if the file has any extent + if [ "$(stat -c "%b" $f)" = "0" ];then + echo " - $f get no extents, but its di_size > 0" + break + fi +done
...so whereas I was trying to suggest that you could use the GETFSXATTR ioctl to return the extent count:
$XFS_IO_PROG -c stat $f | grep fsxattr.nextents | awk '{print $3}'
But that won't work outside of XFS. To make this generic, I think you have to do something like:
$FILEFRAG_PROG -v $f | wc -l
to see if there are any extents.
--D
Quoted 18 lines
tests/generic/999 new file +4
-0+ +# success, all done +status=0 +exittests/generic/999.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extentsSignature · 1 line
2.31.1
Quoted 20 lines · Darrick J. Wong
Quoted 19 lines · Zorro Lang
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their
di_size> 0.Signed-off-by Zorro Lang <[email protected]>---
V2 replace "fiemap" with "stat" command, to check if a file has extents. That helps this case more common.
Thanks,
Zorrotests/generic/999 | 42 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 47 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.out
Quoted 8 lines
tests/generic/999
Quoted 5 lines
tests/generic/999 new file +0
-0Ugh sorry ^^^^^^^ I didn't notice this part and wrote my previous
response thinking this was an xfs-only test...
Oh, my bad, I forgot to change the "_supported_fs xfs" to "generic".
Quoted 44 lines
Quoted 30 lines
tests/generic/999 new file +28
-0@@ -0,0 +1,42 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount"tests/generic/999 line 38
Quoted 10 lines
tests/generic/999 new file +10
-0+_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + # Check if the file has any extent + if [ "$(stat -c "%b" $f)" = "0" ];then + echo " - $f get no extents, but its di_size > 0" + break + fi +done...so whereas I was trying to suggest that you could use the GETFSXATTR ioctl to return the extent count:
Quoted 6 lines
$XFS_IO_PROG -c stat $f | grep fsxattr.nextents | awk '{print $3}'
But that won't work outside of XFS. To make this generic, I think you have to do something like:
$FILEFRAG_PROG -v $f | wc -l
I'm wondering if we must check extent is 0, how about check allocated block = 0? I tried [ "$(stat -c "%b" $f)" = "0" ], it fails on old kernel without this bug fix, and test passed on new kernel. Does that make sense for you?
Thanks,
Zorro
Quoted 24 lines
to see if there are any extents.
--D
Quoted 18 lines
tests/generic/999 new file +4
-0+ +# success, all done +status=0 +exittests/generic/999.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extentsSignature · 1 line
2.31.1
Quoted 78 lines · Zorro Lang
Quoted 20 lines · Darrick J. Wong
Quoted 19 lines · Zorro Lang
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their
di_size> 0.Signed-off-by Zorro Lang <[email protected]>---
V2 replace "fiemap" with "stat" command, to check if a file has extents. That helps this case more common.
Thanks,
Zorrotests/generic/999 | 42 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 47 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.outQuoted 8 lines
tests/generic/999
Quoted 5 lines
tests/generic/999 new file +0
-0Ugh sorry ^^^^^^^ I didn't notice this part and wrote my previous
response thinking this was an xfs-only test...Oh, my bad, I forgot to change the "
_supported_fsxfs" to "generic".Quoted 46 lines
Quoted 30 lines
tests/generic/999 new file +28
-0@@ -0,0 +1,42 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount"tests/generic/999 line 38
Quoted 10 lines
tests/generic/999 new file +10
-0+_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + # Check if the file has any extent + if [ "$(stat -c "%b" $f)" = "0" ];then + echo " - $f get no extents, but its di_size > 0" + break + fi +done...so whereas I was trying to suggest that you could use the GETFSXATTR ioctl to return the extent count:
$XFS_IO_PROG -c stat $f | grep fsxattr.nextents | awk '{print $3}'
Quoted 8 lines
Quoted 4 lines
But that won't work outside of XFS. To make this generic, I think you have to do something like:
$FILEFRAG_PROG -v $f | wc -l
I'm wondering if we must check extent is 0, how about check allocated block = 0? I tried [ "$(stat -c "%b" $f)" = "0" ], it fails on old kernel without this bug fix, and test passed on new kernel. Does that make sense for you?
What if an LSM (e.g. selinux) attach enough security attrs to push the xattr data into an external block? I don't think that happens often for XFS, but I think it will for ext3 with 128byte inodes.
--D
Quoted 28 lines
Thanks,
ZorroQuoted 24 lines
to see if there are any extents.
--D
Quoted 18 lines
tests/generic/999 new file +4
-0+ +# success, all done +status=0 +exittests/generic/999.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extentsSignature · 1 line
2.31.1
Quoted 82 lines · Darrick J. Wong
Quoted 81 lines · Zorro Lang
Quoted 20 lines · Darrick J. Wong
Quoted 19 lines · Zorro Lang
An old issue might cause on-disk inode sizes are logged prematurely via the free eofblocks path on file close. Then fs shutdown might leave NULL files but their
di_size> 0.Signed-off-by Zorro Lang <[email protected]>---
V2 replace "fiemap" with "stat" command, to check if a file has extents. That helps this case more common.
Thanks,
Zorrotests/generic/999 | 42 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/999.out | 5 +++++ 2 files changed, 47 insertions(+) create mode 100755 tests/generic/999 create mode 100644 tests/generic/999.outQuoted 8 lines
tests/generic/999
Quoted 5 lines
tests/generic/999 new file +0
-0Ugh sorry ^^^^^^^ I didn't notice this part and wrote my previous
response thinking this was an xfs-only test...Oh, my bad, I forgot to change the "
_supported_fsxfs" to "generic".Quoted 49 lines
Quoted 30 lines
tests/generic/999 new file +28
-0@@ -0,0 +1,42 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test No. 999 +# +# Test an issue in the truncate codepath where on-disk inode sizes are logged +# prematurely via the free eofblocks path on file close. +# +. ./common/preamble +_begin_fstest auto quick shutdown + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_scratch_shutdown +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount + +echo "Create many small files with one extent at least" +for ((i=0; i<10000; i++));do + $XFS_IO_PROG -f -c "pwrite 0 4k" $SCRATCH_MNT/file.$i >/dev/null 2>&1 +done + +echo "Shutdown the fs suddently" +_scratch_shutdown + +echo "Cycle mount"tests/generic/999 line 38
Quoted 10 lines
tests/generic/999 new file +10
-0+_scratch_cycle_mount + +echo "Check file's (di_size > 0) extents" +for f in $(find $SCRATCH_MNT -type f -size +0);do + # Check if the file has any extent + if [ "$(stat -c "%b" $f)" = "0" ];then + echo " - $f get no extents, but its di_size > 0" + break + fi +done...so whereas I was trying to suggest that you could use the GETFSXATTR ioctl to return the extent count:
$XFS_IO_PROG -c stat $f | grep fsxattr.nextents | awk '{print $3}'
But that won't work outside of XFS. To make this generic, I think you have to do something like:
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
$FILEFRAG_PROG -v $f | wc -l
I'm wondering if we must check extent is 0, how about check allocated block = 0? I tried [ "$(stat -c "%b" $f)" = "0" ], it fails on old kernel without this bug fix, and test passed on new kernel. Does that make sense for you?
What if an LSM (e.g. selinux) attach enough security attrs to push the xattr data into an external block? I don't think that happens often for
Oh! You're right, I forgot the ablocks. Due to fstests always use
"-o context=system_u:object_r:root_t:s0" to mount, I thought there's not selinux labels to take xattr space by default, so ablocks generally is 0 (I think:).
But I agree with you, "$(stat -c "%b" $f)" can be affected by ablocks, so filefrag might be a better solution.
Thanks,
Zorro
Quoted 33 lines
XFS, but I think it will for ext3 with 128byte inodes.
--D
Quoted 28 lines
Thanks,
ZorroQuoted 24 lines
to see if there are any extents.
--D
Quoted 18 lines
tests/generic/999 new file +4
-0+ +# success, all done +status=0 +exittests/generic/999.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 999 +Create many small files with one extent at least +Shutdown the fs suddently +Cycle mount +Check file's (di_size > 0) extentsSignature · 1 line
2.31.1