Skip to contentSkip to conversation

test i_blocks for truncated largefiles

PATCH In review Pavel Reichl 3 participants 7 messages Sep 22, 2022

lore
Share & tools
lore
#0 lore raw

v5 cover test i_blocks for truncated largefiles

Hello,

Thank you for your comments. Please see next revision.

V3
* introduced create_sizedfile() helper
* amended g/694 and g/698 to utilize this helper

V4
* copy-pasted the helper as suggested by Zorro (just proposed a different comment)
* updated the tests to use it and check its return value
* added _fixed_by_kernel_commit directive into g/694

V5
* updated helper comment as Darrick suggested
* changed the error message when failure to create test file occurs

Pavel Reichl (2):
common: new helper to alloacate fixed size files
generic: test i_blocks for truncated large files

 common/rc             | 23 +++++++++++++++++++
 tests/generic/694     |  8 ++++++-
 tests/generic/698     | 51 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/698.out |  2 ++
 4 files changed, 83 insertions(+), 1 deletion(-)
 create mode 100755 tests/generic/698
 create mode 100644 tests/generic/698.out
Signature · 1 line
2.37.3
#1 lore raw

PATCH v5 1/2 common: new helper to alloacate fixed size files

Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.

---
 common/rc         | 23 +++++++++++++++++++++++
 tests/generic/694 |  8 +++++++-
 2 files changed, 30 insertions(+), 1 deletion(-)
common/rc +23 -0
index a25cbcd0..228fcb37 100644
@@ -4925,6 +4925,29 @@ hexdump()
 	_fail "Use _hexdump(), please!"
 }
 
+# Try to create a file with inode->i_blocks >= (length / blocksize).
+# There may be some small overhead, e.g. ext2 filesystem allocates a
+# substantial number of blocks to store block mappings. Those are accounted
+# to i_blocks.
+_create_file_sized()
+{
+	local length=$1
+	local file=$2
+	local tmp=`mktemp -u`
+	local ret=0
+
+	$XFS_IO_PROG -ft -c "falloc 0 $length" $file >$tmp.out 2>&1
+	ret=$?
+	if (grep -Eq "Operation not supported|command .* not found" $tmp.out);then
+		# fallocate isn't supported, fallback to general buffer write
+		$XFS_IO_PROG -ft -c "pwrite 0 $length" $file >$tmp.out 2>&1
+		ret=$?
+	fi
+	[ $ret -ne 0 ] && cat $tmp.out
+	rm -f $tmp.out
+	return $ret
+}
+
 init_rc
 
 ################################################################################
tests/generic/694 +7 -1
index dfd988df..c96c2154 100755
@@ -21,6 +21,9 @@ _cleanup()
 }
 
 _supported_fs generic
+_fixed_by_kernel_commit 0c336d6e33f4 \
+	"exfat: fix incorrect loading of i_blocks for large file"
+
 _require_test
 _require_fs_space $TEST_DIR $((4 * 1024 * 1024)) #kB
 
@@ -30,7 +33,10 @@ junk_dir=$TEST_DIR/$seq
 junk_file=$junk_dir/junk
 mkdir -p $junk_dir
 
-$XFS_IO_PROG -f -c "pwrite -W 0 4G" $junk_file > /dev/null
+_create_file_sized 4G $junk_file
+if [ $? -ne 0 ]; then
+	echo "Could not create 4G test file"
+fi
 
 iblocks=`stat -c '%b' $junk_file`
 
Signature · 1 line
2.37.3
Reviewed-by on v5 1/2
#2 lore raw

Re: [PATCH v5 1/2] common: new helper to alloacate fixed size files

Quoted 49 lines · Pavel Reichl

Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.

---
 common/rc         | 23 +++++++++++++++++++++++
 tests/generic/694 |  8 +++++++-
 2 files changed, 30 insertions(+), 1 deletion(-)
common/rc +23 -0
index a25cbcd0..228fcb37 100644
@@ -4925,6 +4925,29 @@ hexdump()
 	_fail "Use _hexdump(), please!"
 }
 
+# Try to create a file with inode->i_blocks >= (length / blocksize).
+# There may be some small overhead, e.g. ext2 filesystem allocates a
+# substantial number of blocks to store block mappings. Those are accounted
+# to i_blocks.
+_create_file_sized()
+{
+	local length=$1
+	local file=$2
+	local tmp=`mktemp -u`
+	local ret=0
+
+	$XFS_IO_PROG -ft -c "falloc 0 $length" $file >$tmp.out 2>&1
+	ret=$?
+	if (grep -Eq "Operation not supported|command .* not found" $tmp.out);then
+		# fallocate isn't supported, fallback to general buffer write
+		$XFS_IO_PROG -ft -c "pwrite 0 $length" $file >$tmp.out 2>&1
+		ret=$?
+	fi
+	[ $ret -ne 0 ] && cat $tmp.out
+	rm -f $tmp.out
+	return $ret
+}
+
 init_rc
 
 ################################################################################
tests/generic/694 +0 -0
index dfd988df..c96c2154 100755

tests/generic/694 _cleanup() · line 26

Quoted 6 lines
tests/generic/694 +3 -0
@@ -21,6 +21,9 @@ _cleanup()
 }
 
 _supported_fs generic
+_fixed_by_kernel_commit 0c336d6e33f4 \
+	"exfat: fix incorrect loading of i_blocks for large file"
+

Thanks for adding this too, this version looks good to me now.

Quoted 18 lines
tests/generic/694 +4 -1
 _require_test
 _require_fs_space $TEST_DIR $((4 * 1024 * 1024)) #kB
 
@@ -30,7 +33,10 @@ junk_dir=$TEST_DIR/$seq
 junk_file=$junk_dir/junk
 mkdir -p $junk_dir
 
-$XFS_IO_PROG -f -c "pwrite -W 0 4G" $junk_file > /dev/null
+_create_file_sized 4G $junk_file
+if [ $? -ne 0 ]; then
+	echo "Could not create 4G test file"
+fi
 
 iblocks=`stat -c '%b' $junk_file`
 
Signature · 1 line
2.37.3
Reviewed-by on v5 1/2
#3 lore raw

Re: [PATCH v5 1/2] common: new helper to alloacate fixed size files

In reply to Pavel Reichl
Quoted 5 lines · Pavel Reichl

Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.

Technically speaking, I only RVB-tagged the second patch, not this first one...

Quoted 54 lines
---
 common/rc         | 23 +++++++++++++++++++++++
 tests/generic/694 |  8 +++++++-
 2 files changed, 30 insertions(+), 1 deletion(-)
common/rc +23 -0
index a25cbcd0..228fcb37 100644
@@ -4925,6 +4925,29 @@ hexdump()
 	_fail "Use _hexdump(), please!"
 }
 
+# Try to create a file with inode->i_blocks >= (length / blocksize).
+# There may be some small overhead, e.g. ext2 filesystem allocates a
+# substantial number of blocks to store block mappings. Those are accounted
+# to i_blocks.
+_create_file_sized()
+{
+	local length=$1
+	local file=$2
+	local tmp=`mktemp -u`
+	local ret=0
+
+	$XFS_IO_PROG -ft -c "falloc 0 $length" $file >$tmp.out 2>&1
+	ret=$?
+	if (grep -Eq "Operation not supported|command .* not found" $tmp.out);then
+		# fallocate isn't supported, fallback to general buffer write
+		$XFS_IO_PROG -ft -c "pwrite 0 $length" $file >$tmp.out 2>&1
+		ret=$?
+	fi
+	[ $ret -ne 0 ] && cat $tmp.out
+	rm -f $tmp.out
+	return $ret
+}
+
 init_rc
 
 ################################################################################
tests/generic/694 +3 -0
index dfd988df..c96c2154 100755
@@ -21,6 +21,9 @@ _cleanup()
 }
 
 _supported_fs generic
+_fixed_by_kernel_commit 0c336d6e33f4 \
+	"exfat: fix incorrect loading of i_blocks for large file"
+
 _require_test
 _require_fs_space $TEST_DIR $((4 * 1024 * 1024)) #kB
 

tests/generic/694 junk_dir=$TEST_DIR/$seq · line 38

Quoted 7 lines
tests/generic/694 +3 -1
@@ -30,7 +33,10 @@ junk_dir=$TEST_DIR/$seq
 junk_file=$junk_dir/junk
 mkdir -p $junk_dir
 
-$XFS_IO_PROG -f -c "pwrite -W 0 4G" $junk_file > /dev/null
+_create_file_sized 4G $junk_file
+if [ $? -ne 0 ]; then
+	echo "Could not create 4G test file"

...but this does look good now.

--D

Quoted 7 lines
tests/generic/694 +1 -0
+fi
 
 iblocks=`stat -c '%b' $junk_file`
 
Signature · 1 line
2.37.3
#4 lore raw

Re: [PATCH v5 1/2] common: new helper to alloacate fixed size files

Quoted 8 lines · Darrick J. Wong
Quoted 5 lines · Pavel Reichl

Helper that creates files of specified size using falloc if supported, otherwise pwrite is used.

Technically speaking, I only RVB-tagged the second patch, not this first one...

Oh, sorry about that - my bad.

#5 lore raw

PATCH v5 2/2 generic: test i_blocks for truncated large files

In reply to Pavel Reichl

This is a regression test for an incorrect computation of i_blocks for truncated files larger than 4 GiB. Bug was filed for exFAT.

Test is based on reproducer provied by Christophe Vu-Brugier as part of kernel patch-fix submission.

---
 tests/generic/698     | 51 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/698.out |  2 ++
 2 files changed, 53 insertions(+)
 create mode 100755 tests/generic/698
 create mode 100644 tests/generic/698.out
tests/generic/698 new file +51 -0
new file mode 100755
index 00000000..6aeea64e
@@ -0,0 +1,51 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022  Red Hat Inc. All Rights Reserved.
+#
+# FS QA Test 698
+#
+# Verify that i_blocks for truncated files larger than 4 GiB have correct
+# values.
+#
+# This test verifies the problem fixed in kernel with commit
+# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB
+#
+. ./common/preamble
+. ./common/filter
+
+_begin_fstest auto
+
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.* $junk_dir
+}
+
+_supported_fs generic
+_fixed_by_kernel_commit 92fba084b79e \
+	"exfat: fix i_blocks for files truncated over 4 GiB"
+
+_require_test
+_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB
+
+junk_dir=$TEST_DIR/$seq
+junk_file=$junk_dir/junk
+mkdir -p $junk_dir
+
+_create_file_sized 5G $junk_file
+if [ $? -ne 0 ]; then
+	echo "Could not create 5G test file"
+fi
+
+truncate -s 4G $junk_file
+
+block_size=`stat -c '%B' $junk_file`
+iblocks_after_truncate=`stat -c '%b' $junk_file`
+iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size))
+
+_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v
+
+status=0
+
+exit
tests/generic/698.out new file +2 -0
new file mode 100644
index 00000000..cbb02d37
@@ -0,0 +1,2 @@
+QA output created by 698
+Number of allocated blocks after truncate is in range
Signature · 1 line
2.37.3
Reviewed-by on v5 2/2
#6 lore raw

Re: [PATCH v5 2/2] generic: test i_blocks for truncated large files

Quoted 50 lines · Pavel Reichl

This is a regression test for an incorrect computation of i_blocks for truncated files larger than 4 GiB. Bug was filed for exFAT.

Test is based on reproducer provied by Christophe Vu-Brugier as part of kernel patch-fix submission.

---
 tests/generic/698     | 51 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/698.out |  2 ++
 2 files changed, 53 insertions(+)
 create mode 100755 tests/generic/698
 create mode 100644 tests/generic/698.out
tests/generic/698 new file +29 -0
new file mode 100755
index 00000000..6aeea64e
@@ -0,0 +1,51 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022  Red Hat Inc. All Rights Reserved.
+#
+# FS QA Test 698
+#
+# Verify that i_blocks for truncated files larger than 4 GiB have correct
+# values.
+#
+# This test verifies the problem fixed in kernel with commit
+# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB
+#
+. ./common/preamble
+. ./common/filter
+
+_begin_fstest auto
+
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.* $junk_dir
+}
+
+_supported_fs generic
+_fixed_by_kernel_commit 92fba084b79e \
+	"exfat: fix i_blocks for files truncated over 4 GiB"
+
+_require_test

tests/generic/698 line 39

Quoted 10 lines
tests/generic/698 new file +10 -0
+_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB
+
+junk_dir=$TEST_DIR/$seq
+junk_file=$junk_dir/junk
+mkdir -p $junk_dir
+
+_create_file_sized 5G $junk_file
+if [ $? -ne 0 ]; then
+	echo "Could not create 5G test file"
+fi

Good to me,

Quoted 23 lines
tests/generic/698 new file +12 -0
+
+truncate -s 4G $junk_file
+
+block_size=`stat -c '%B' $junk_file`
+iblocks_after_truncate=`stat -c '%b' $junk_file`
+iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size))
+
+_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v
+
+status=0
+
+exit
tests/generic/698.out new file +2 -0
new file mode 100644
index 00000000..cbb02d37
@@ -0,0 +1,2 @@
+QA output created by 698
+Number of allocated blocks after truncate is in range
Signature · 1 line
2.37.3