test i_blocks for truncated largefiles
v5 cover test i_blocks for truncated largefiles
Hello,
Thank you for your comments. Please see next revision.
V3
* introduced create_sizedfile() helper
* amended g/694 and g/698 to utilize this helper
V4
* copy-pasted the helper as suggested by Zorro (just proposed a different comment)
* updated the tests to use it and check its return value
* added _fixed_by_kernel_commit directive into g/694
V5
* updated helper comment as Darrick suggested
* changed the error message when failure to create test file occurs
Pavel Reichl (2):
common: new helper to alloacate fixed size files
generic: test i_blocks for truncated large files
common/rc | 23 +++++++++++++++++++ tests/generic/694 | 8 ++++++- tests/generic/698 | 51 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/698.out | 2 ++ 4 files changed, 83 insertions(+), 1 deletion(-) create mode 100755 tests/generic/698 create mode 100644 tests/generic/698.out
Signature · 1 line
2.37.3
PATCH v5 1/2 common: new helper to alloacate fixed size files
Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.
--- common/rc | 23 +++++++++++++++++++++++ tests/generic/694 | 8 +++++++- 2 files changed, 30 insertions(+), 1 deletion(-)
common/rc +23 -0
@@ -4925,6 +4925,29 @@ hexdump() _fail "Use _hexdump(), please!" } +# Try to create a file with inode->i_blocks >= (length / blocksize). +# There may be some small overhead, e.g. ext2 filesystem allocates a +# substantial number of blocks to store block mappings. Those are accounted +# to i_blocks. +_create_file_sized() +{ + local length=$1 + local file=$2 + local tmp=`mktemp -u` + local ret=0 + + $XFS_IO_PROG -ft -c "falloc 0 $length" $file >$tmp.out 2>&1 + ret=$? + if (grep -Eq "Operation not supported|command .* not found" $tmp.out);then + # fallocate isn't supported, fallback to general buffer write + $XFS_IO_PROG -ft -c "pwrite 0 $length" $file >$tmp.out 2>&1 + ret=$? + fi + [ $ret -ne 0 ] && cat $tmp.out + rm -f $tmp.out + return $ret +} + init_rc ################################################################################
tests/generic/694 +7 -1
@@ -21,6 +21,9 @@ _cleanup() } _supported_fs generic +_fixed_by_kernel_commit 0c336d6e33f4 \ + "exfat: fix incorrect loading of i_blocks for large file" + _require_test _require_fs_space $TEST_DIR $((4 * 1024 * 1024)) #kB
@@ -30,7 +33,10 @@ junk_dir=$TEST_DIR/$seq junk_file=$junk_dir/junk mkdir -p $junk_dir -$XFS_IO_PROG -f -c "pwrite -W 0 4G" $junk_file > /dev/null +_create_file_sized 4G $junk_file +if [ $? -ne 0 ]; then + echo "Could not create 4G test file" +fi iblocks=`stat -c '%b' $junk_file`
Signature · 1 line
2.37.3
Re: [PATCH v5 1/2] common: new helper to alloacate fixed size files
Quoted 49 lines · Pavel Reichl
Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.
Signed-off-by Pavel Reichl <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>--- common/rc | 23 +++++++++++++++++++++++ tests/generic/694 | 8 +++++++- 2 files changed, 30 insertions(+), 1 deletion(-)common/rc +23
-0@@ -4925,6 +4925,29 @@ hexdump() _fail "Use _hexdump(), please!" } +# Try to create a file with inode->i_blocks >= (length / blocksize). +# There may be some small overhead, e.g. ext2 filesystem allocates a +# substantial number of blocks to store block mappings. Those are accounted +# to i_blocks. +_create_file_sized() +{ + local length=$1 + local file=$2 + local tmp=`mktemp -u` + local ret=0 + + $XFS_IO_PROG -ft -c "falloc 0 $length" $file >$tmp.out 2>&1 + ret=$? + if (grep -Eq "Operation not supported|command .* not found" $tmp.out);then + # fallocate isn't supported, fallback to general buffer write + $XFS_IO_PROG -ft -c "pwrite 0 $length" $file >$tmp.out 2>&1 + ret=$? + fi + [ $ret -ne 0 ] && cat $tmp.out + rm -f $tmp.out + return $ret +} + init_rc ################################################################################tests/generic/694 +0
-0
tests/generic/694 _cleanup() · line 26
Quoted 6 lines
tests/generic/694 +3
-0@@ -21,6 +21,9 @@ _cleanup() } _supported_fs generic +_fixed_by_kernel_commit 0c336d6e33f4 \ + "exfat: fix incorrect loading of i_blocks for large file" +
Thanks for adding this too, this version looks good to me now.
Quoted 18 lines
tests/generic/694 +4
-1_require_test _require_fs_space $TEST_DIR $((4 * 1024 * 1024)) #kB@@ -30,7 +33,10 @@ junk_dir=$TEST_DIR/$seq junk_file=$junk_dir/junk mkdir -p $junk_dir -$XFS_IO_PROG -f -c "pwrite -W 0 4G" $junk_file > /dev/null +_create_file_sized 4G $junk_file +if [ $? -ne 0 ]; then + echo "Could not create 4G test file" +fi iblocks=`stat -c '%b' $junk_file`Signature · 1 line
2.37.3
Re: [PATCH v5 1/2] common: new helper to alloacate fixed size files
In reply to Pavel ReichlQuoted 5 lines · Pavel Reichl
Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.
Signed-off-by Pavel Reichl <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>
Technically speaking, I only RVB-tagged the second patch, not this first one...
Quoted 54 lines
--- common/rc | 23 +++++++++++++++++++++++ tests/generic/694 | 8 +++++++- 2 files changed, 30 insertions(+), 1 deletion(-)common/rc +23
-0@@ -4925,6 +4925,29 @@ hexdump() _fail "Use _hexdump(), please!" } +# Try to create a file with inode->i_blocks >= (length / blocksize). +# There may be some small overhead, e.g. ext2 filesystem allocates a +# substantial number of blocks to store block mappings. Those are accounted +# to i_blocks. +_create_file_sized() +{ + local length=$1 + local file=$2 + local tmp=`mktemp -u` + local ret=0 + + $XFS_IO_PROG -ft -c "falloc 0 $length" $file >$tmp.out 2>&1 + ret=$? + if (grep -Eq "Operation not supported|command .* not found" $tmp.out);then + # fallocate isn't supported, fallback to general buffer write + $XFS_IO_PROG -ft -c "pwrite 0 $length" $file >$tmp.out 2>&1 + ret=$? + fi + [ $ret -ne 0 ] && cat $tmp.out + rm -f $tmp.out + return $ret +} + init_rc ################################################################################tests/generic/694 +3
-0@@ -21,6 +21,9 @@ _cleanup() } _supported_fs generic +_fixed_by_kernel_commit 0c336d6e33f4 \ + "exfat: fix incorrect loading of i_blocks for large file" + _require_test _require_fs_space $TEST_DIR $((4 * 1024 * 1024)) #kB
tests/generic/694 junk_dir=$TEST_DIR/$seq · line 38
Quoted 7 lines
tests/generic/694 +3
-1@@ -30,7 +33,10 @@ junk_dir=$TEST_DIR/$seq junk_file=$junk_dir/junk mkdir -p $junk_dir -$XFS_IO_PROG -f -c "pwrite -W 0 4G" $junk_file > /dev/null +_create_file_sized 4G $junk_file +if [ $? -ne 0 ]; then + echo "Could not create 4G test file"
...but this does look good now.
--D
Quoted 7 lines
tests/generic/694 +1
-0+fi iblocks=`stat -c '%b' $junk_file`Signature · 1 line
2.37.3
Re: [PATCH v5 1/2] common: new helper to alloacate fixed size files
Quoted 8 lines · Darrick J. Wong
Quoted 5 lines · Pavel Reichl
Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.
Signed-off-by Pavel Reichl <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>Technically speaking, I only RVB-tagged the second patch, not this first one...
Oh, sorry about that - my bad.
PATCH v5 2/2 generic: test i_blocks for truncated large files
In reply to Pavel ReichlThis is a regression test for an incorrect computation of i_blocks for truncated files larger than 4 GiB. Bug was filed for exFAT.
Test is based on reproducer provied by Christophe Vu-Brugier as part of kernel patch-fix submission.
--- tests/generic/698 | 51 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/698.out | 2 ++ 2 files changed, 53 insertions(+) create mode 100755 tests/generic/698 create mode 100644 tests/generic/698.out
tests/generic/698 new file +51 -0
@@ -0,0 +1,51 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat Inc. All Rights Reserved. +# +# FS QA Test 698 +# +# Verify that i_blocks for truncated files larger than 4 GiB have correct +# values. +# +# This test verifies the problem fixed in kernel with commit +# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB +# +. ./common/preamble +. ./common/filter + +_begin_fstest auto + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -r -f $tmp.* $junk_dir +} + +_supported_fs generic +_fixed_by_kernel_commit 92fba084b79e \ + "exfat: fix i_blocks for files truncated over 4 GiB" + +_require_test +_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB + +junk_dir=$TEST_DIR/$seq +junk_file=$junk_dir/junk +mkdir -p $junk_dir + +_create_file_sized 5G $junk_file +if [ $? -ne 0 ]; then + echo "Could not create 5G test file" +fi + +truncate -s 4G $junk_file + +block_size=`stat -c '%B' $junk_file` +iblocks_after_truncate=`stat -c '%b' $junk_file` +iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size)) + +_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v + +status=0 + +exit
tests/generic/698.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 698 +Number of allocated blocks after truncate is in range
Signature · 1 line
2.37.3
Re: [PATCH v5 2/2] generic: test i_blocks for truncated large files
Quoted 50 lines · Pavel Reichl
This is a regression test for an incorrect computation of
i_blocksfor truncated files larger than 4 GiB. Bug was filed for exFAT.Test is based on reproducer provied by Christophe Vu-Brugier as part of kernel patch-fix submission.
Signed-off-by Pavel Reichl <[email protected]>Reviewed-by Darrick J. Wong <[email protected]>--- tests/generic/698 | 51 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/698.out | 2 ++ 2 files changed, 53 insertions(+) create mode 100755 tests/generic/698 create mode 100644 tests/generic/698.outtests/generic/698 new file +29
-0@@ -0,0 +1,51 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat Inc. All Rights Reserved. +# +# FS QA Test 698 +# +# Verify that i_blocks for truncated files larger than 4 GiB have correct +# values. +# +# This test verifies the problem fixed in kernel with commit +# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB +# +. ./common/preamble +. ./common/filter + +_begin_fstest auto + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -r -f $tmp.* $junk_dir +} + +_supported_fs generic +_fixed_by_kernel_commit 92fba084b79e \ + "exfat: fix i_blocks for files truncated over 4 GiB" + +_require_test
tests/generic/698 line 39
Quoted 10 lines
tests/generic/698 new file +10
-0+_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB + +junk_dir=$TEST_DIR/$seq +junk_file=$junk_dir/junk +mkdir -p $junk_dir + +_create_file_sized 5G $junk_file +if [ $? -ne 0 ]; then + echo "Could not create 5G test file" +fi
Good to me,
Quoted 23 lines
tests/generic/698 new file +12
-0+ +truncate -s 4G $junk_file + +block_size=`stat -c '%B' $junk_file` +iblocks_after_truncate=`stat -c '%b' $junk_file` +iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size)) + +_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v + +status=0 + +exittests/generic/698.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 698 +Number of allocated blocks after truncate is in rangeSignature · 1 line
2.37.3