Skip to contentSkip to conversation

common/config: remove default 4k blocksize from XFS_MKFS_OPTIONS

PATCH In review Zorro Lang 4 participants 7 messages May 25, 2021 – May 30, 2021

lore
Share & tools
lore

Zorro Lang

author
#0 lore raw

PATCH 1/2 common/config: remove default 4k blocksize from XFS_MKFS_OPTIONS

xfstests set "-bsize=4k" to XFS_MKFS_OPTIONS by default, then give it to MKFS_OPTIOPNS. So MKFS_OPTIOPNS always contains "-bsize=4k" except we set XFS_MKFS_OPTIONS manually.

It's useless to set XFS_MKFS_OPTIONS to "-bsize=4096" by default, especially that will cause all cases with _scratch_mkfs_blocksized() always fail as "-b size option respecified", when test on XFS. For
exmaple: generic/222

---
 common/config | 1 -
 1 file changed, 1 deletion(-)
common/config +0 -1
index a47e462c..ecd59fae 100644
@@ -60,7 +60,6 @@ export EMAIL=root@localhost    # where auto-qa will send its status messages
 export HOST_OPTIONS=${HOST_OPTIONS:=local.config}
 export CHECK_OPTIONS=${CHECK_OPTIONS:="-g auto"}
 export BENCH_PASSES=${BENCH_PASSES:=5}
-export XFS_MKFS_OPTIONS=${XFS_MKFS_OPTIONS:=-bsize=4096}
 export TIME_FACTOR=${TIME_FACTOR:=1}
 export LOAD_FACTOR=${LOAD_FACTOR:=1}
 export DEBUGFS_MNT=${DEBUGFS_MNT:="/sys/kernel/debug"}
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH 2/2 generic: test small swapfile without page-aligned contiguous blocks

If a swapfile doesn't contain even a single page-aligned contiguous range of blocks, it's an invalid swapfile, and might cause kernel issue. This case covered commit 5808fecc5723 ("iomap: Fix negative assignment to unsigned sis->pages in iomap_swapfile_activate").

---
 tests/generic/638     | 79 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/638.out |  2 ++
 tests/generic/group   |  1 +
 3 files changed, 82 insertions(+)
 create mode 100755 tests/generic/638
 create mode 100644 tests/generic/638.out
tests/generic/638 new file +79 -0
new file mode 100755
index 00000000..74bc39cb
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Red Hat Inc.  All Rights Reserved.
+#
+# FS QA Test 638
+#
+# Test small swapfile which doesn't contain even a single page-aligned contiguous
+# range of blocks. This case covered commit 5808fecc5723 ("iomap: Fix negative
+# assignment to unsigned sis->pages in iomap_swapfile_activate").
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+_require_scratch_swapfile
+_require_test_program mkswap
+_require_test_program swapon
+
+make_unaligned_swapfile()
+{
+	local fname=$1
+	local n=$((psize / bsize - 1))
+
+	# Make sure the swapfile doesn't contain even a single page-aligned
+	# contiguous range of blocks. This's necessary to cover the bug
+	$XFS_IO_PROG -f -t -c "pwrite 0 $(((psize + bsize) * n))" $fname >> $seqres.full 2>&1
+	for((i=1; i<=n; i++));do
+		$XFS_IO_PROG -c "fcollapse $(((psize - bsize) * i)) $bsize" $fname
+	done
+	chmod 0600 $fname
+	$CHATTR_PROG +C $fname > /dev/null 2>&1
+	$here/src/mkswap $fname
+}
+
+_scratch_mkfs >> $seqres.full 2>&1
+_scratch_mount
+psize=`get_page_size`
+bsize=`_get_block_size $SCRATCH_MNT`
+# Due to we need page-unaligned blocks, so blocksize < pagesize is necessary.
+# If not, try to make a smaller enough block size
+if [ $bsize -ge $psize ];then
+	_scratch_unmount
+	_scratch_mkfs_blocksized 1024 >> $seqres.full 2>&1
+	if [ $? -ne 0 ];then
+		_notrun "Can't make filesystem block size < page size"
+	fi
+	_scratch_mount
+	bsize=1024
+fi
+swapfile=$SCRATCH_MNT/$seq.swapfile
+make_unaligned_swapfile $swapfile
+$here/src/swapon $swapfile
+swapoff $swapfile
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
tests/generic/638.out new file +2 -0
new file mode 100644
index 00000000..3113b1e3
@@ -0,0 +1,2 @@
+QA output created by 638
+Silence is golden
tests/generic/group +1 -0
index b9614c69..5b9b3d1b 100644
@@ -640,3 +640,4 @@
 635 auto quick atime bigtime shutdown
 636 auto quick swap
 637 auto quick rw
+638 auto quick swap
Signature · 1 line
2.31.1
#2 lore raw

Re: [PATCH 2/2] generic: test small swapfile without page-aligned contiguous blocks

Quoted 69 lines · Zorro Lang

If a swapfile doesn't contain even a single page-aligned contiguous range of blocks, it's an invalid swapfile, and might cause kernel issue. This case covered commit 5808fecc5723 ("iomap: Fix negative assignment to unsigned sis->pages in iomap_swapfile_activate").

---
 tests/generic/638     | 79 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/638.out |  2 ++
 tests/generic/group   |  1 +
 3 files changed, 82 insertions(+)
 create mode 100755 tests/generic/638
 create mode 100644 tests/generic/638.out
tests/generic/638 new file +49 -0
new file mode 100755
index 00000000..74bc39cb
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Red Hat Inc.  All Rights Reserved.
+#
+# FS QA Test 638
+#
+# Test small swapfile which doesn't contain even a single page-aligned contiguous
+# range of blocks. This case covered commit 5808fecc5723 ("iomap: Fix negative
+# assignment to unsigned sis->pages in iomap_swapfile_activate").
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+_require_scratch_swapfile
+_require_test_program mkswap
+_require_test_program swapon
+
+make_unaligned_swapfile()
+{
+	local fname=$1
+	local n=$((psize / bsize - 1))
+
+	# Make sure the swapfile doesn't contain even a single page-aligned
+	# contiguous range of blocks. This's necessary to cover the bug
+	$XFS_IO_PROG -f -t -c "pwrite 0 $(((psize + bsize) * n))" $fname >> $seqres.full 2>&1
+	for((i=1; i<=n; i++));do
+		$XFS_IO_PROG -c "fcollapse $(((psize - bsize) * i)) $bsize" $fname

tests/generic/638 line 59

Quoted 10 lines
tests/generic/638 new file +10 -0
+	done
+	chmod 0600 $fname
+	$CHATTR_PROG +C $fname > /dev/null 2>&1
+	$here/src/mkswap $fname
+}
+
+_scratch_mkfs >> $seqres.full 2>&1
+_scratch_mount
+psize=`get_page_size`
+bsize=`_get_block_size $SCRATCH_MNT`

_get_file_block_size, since fcollapse requires arguments aligned to the file's allocation unit, which isn't necessarily the same as the fs block size (bigalloc, rt, etc.)...

tests/generic/638 line 64

Quoted 5 lines
tests/generic/638 new file +5 -0
+# Due to we need page-unaligned blocks, so blocksize < pagesize is necessary.
+# If not, try to make a smaller enough block size
+if [ $bsize -ge $psize ];then
+	_scratch_unmount
+	_scratch_mkfs_blocksized 1024 >> $seqres.full 2>&1

...which leads me to the next question, which is: Should the above helper override mkfs options as necessary to enforce that the allocation unit is 1024 bytes? Or is it time for a new helper?

tests/generic/638 line 69

Quoted 5 lines
tests/generic/638 new file +5 -0
+	if [ $? -ne 0 ];then
+		_notrun "Can't make filesystem block size < page size"
+	fi
+	_scratch_mount
+	bsize=1024

You probably want to re-check _get_file_block_size to make sure that you actually got blocksize < pagesize in the format attempt.

--D

Quoted 30 lines
tests/generic/638 new file +10 -0
+fi
+swapfile=$SCRATCH_MNT/$seq.swapfile
+make_unaligned_swapfile $swapfile
+$here/src/swapon $swapfile
+swapoff $swapfile
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
tests/generic/638.out new file +2 -0
new file mode 100644
index 00000000..3113b1e3
@@ -0,0 +1,2 @@
+QA output created by 638
+Silence is golden
tests/generic/group +1 -0
index b9614c69..5b9b3d1b 100644
@@ -640,3 +640,4 @@
 635 auto quick atime bigtime shutdown
 636 auto quick swap
 637 auto quick rw
+638 auto quick swap
Signature · 1 line
2.31.1
author
#3 lore raw

Re: [PATCH 2/2] generic: test small swapfile without page-aligned contiguous blocks

Quoted 76 lines · Darrick J. Wong
Quoted 75 lines · Zorro Lang

If a swapfile doesn't contain even a single page-aligned contiguous range of blocks, it's an invalid swapfile, and might cause kernel issue. This case covered commit 5808fecc5723 ("iomap: Fix negative assignment to unsigned sis->pages in iomap_swapfile_activate").

---
 tests/generic/638     | 79 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/638.out |  2 ++
 tests/generic/group   |  1 +
 3 files changed, 82 insertions(+)
 create mode 100755 tests/generic/638
 create mode 100644 tests/generic/638.out
tests/generic/638 new file +55 -0
new file mode 100755
index 00000000..74bc39cb
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Red Hat Inc.  All Rights Reserved.
+#
+# FS QA Test 638
+#
+# Test small swapfile which doesn't contain even a single page-aligned contiguous
+# range of blocks. This case covered commit 5808fecc5723 ("iomap: Fix negative
+# assignment to unsigned sis->pages in iomap_swapfile_activate").
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+_require_scratch_swapfile
+_require_test_program mkswap
+_require_test_program swapon
+
+make_unaligned_swapfile()
+{
+	local fname=$1
+	local n=$((psize / bsize - 1))
+
+	# Make sure the swapfile doesn't contain even a single page-aligned
+	# contiguous range of blocks. This's necessary to cover the bug
+	$XFS_IO_PROG -f -t -c "pwrite 0 $(((psize + bsize) * n))" $fname >> $seqres.full 2>&1
+	for((i=1; i<=n; i++));do
+		$XFS_IO_PROG -c "fcollapse $(((psize - bsize) * i)) $bsize" $fname
+	done
+	chmod 0600 $fname
+	$CHATTR_PROG +C $fname > /dev/null 2>&1
+	$here/src/mkswap $fname
+}
+
Quoted 8 lines

tests/generic/638 line 59

Quoted 4 lines
tests/generic/638 new file +4 -0
+_scratch_mkfs >> $seqres.full 2>&1
+_scratch_mount
+psize=`get_page_size`
+bsize=`_get_block_size $SCRATCH_MNT`

_get_file_block_size, since fcollapse requires arguments aligned to the file's allocation unit, which isn't necessarily the same as the fs block size (bigalloc, rt, etc.)...

You're right.

Quoted 10 lines

tests/generic/638 line 64

Quoted 6 lines
tests/generic/638 new file +5 -0
+# Due to we need page-unaligned blocks, so blocksize < pagesize is necessary.
+# If not, try to make a smaller enough block size
+if [ $bsize -ge $psize ];then
+	_scratch_unmount
+	_scratch_mkfs_blocksized 1024 >> $seqres.full 2>&1

...which leads me to the next question, which is: Should the above helper override mkfs options as necessary to enforce that the allocation unit is 1024 bytes? Or is it time for a new helper?

Hmm... the _scratch_mkfs_blocksized() does:

xfs)
_scratch_mkfs_xfs $MKFS_OPTIONS -b size=$blocksize

_scratch_mkfs_xfs() already use $MKFS_OPTIONS internally, don't need to give $MKFS_OPTIONS to it again. Although _scratch_mkfs_xfs deals with $MKFS_OPTIONS, it just drops whole $MKFS_OPTIONS if there're conflicts.

The _scratch_mkfs_geom() does overwrite, but _scratch_mkfs_sized doesn't (sized filesystem is rarely seen in local.config). We might let _scratch_mkfs_blocksized learn _scratch_mkfs_geom.

Quoted 9 lines

tests/generic/638 line 69

Quoted 6 lines
tests/generic/638 new file +5 -0
+	if [ $? -ne 0 ];then
+		_notrun "Can't make filesystem block size < page size"
+	fi
+	_scratch_mount
+	bsize=1024

You probably want to re-check _get_file_block_size to make sure that you actually got blocksize < pagesize in the format attempt.

Yes, you're right. _scratch_mkfs_blocksized can't make sure to get that blocksize.

Thanks,
Zorro

Quoted 34 lines

--D

Quoted 30 lines
tests/generic/638 new file +10 -0
+fi
+swapfile=$SCRATCH_MNT/$seq.swapfile
+make_unaligned_swapfile $swapfile
+$here/src/swapon $swapfile
+swapoff $swapfile
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
tests/generic/638.out new file +2 -0
new file mode 100644
index 00000000..3113b1e3
@@ -0,0 +1,2 @@
+QA output created by 638
+Silence is golden
tests/generic/group +1 -0
index b9614c69..5b9b3d1b 100644
@@ -640,3 +640,4 @@
 635 auto quick atime bigtime shutdown
 636 auto quick swap
 637 auto quick rw
+638 auto quick swap
Signature · 1 line
2.31.1
#4 lore raw

Re: [PATCH 2/2] generic: test small swapfile without page-aligned contiguous blocks

Quoted 122 lines · Zorro Lang
Quoted 76 lines · Darrick J. Wong
Quoted 75 lines · Zorro Lang

If a swapfile doesn't contain even a single page-aligned contiguous range of blocks, it's an invalid swapfile, and might cause kernel issue. This case covered commit 5808fecc5723 ("iomap: Fix negative assignment to unsigned sis->pages in iomap_swapfile_activate").

---
 tests/generic/638     | 79 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/638.out |  2 ++
 tests/generic/group   |  1 +
 3 files changed, 82 insertions(+)
 create mode 100755 tests/generic/638
 create mode 100644 tests/generic/638.out
tests/generic/638 new file +55 -0
new file mode 100755
index 00000000..74bc39cb
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Red Hat Inc.  All Rights Reserved.
+#
+# FS QA Test 638
+#
+# Test small swapfile which doesn't contain even a single page-aligned contiguous
+# range of blocks. This case covered commit 5808fecc5723 ("iomap: Fix negative
+# assignment to unsigned sis->pages in iomap_swapfile_activate").
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+_require_scratch_swapfile
+_require_test_program mkswap
+_require_test_program swapon
+
+make_unaligned_swapfile()
+{
+	local fname=$1
+	local n=$((psize / bsize - 1))
+
+	# Make sure the swapfile doesn't contain even a single page-aligned
+	# contiguous range of blocks. This's necessary to cover the bug
+	$XFS_IO_PROG -f -t -c "pwrite 0 $(((psize + bsize) * n))" $fname >> $seqres.full 2>&1
+	for((i=1; i<=n; i++));do
+		$XFS_IO_PROG -c "fcollapse $(((psize - bsize) * i)) $bsize" $fname
+	done
+	chmod 0600 $fname
+	$CHATTR_PROG +C $fname > /dev/null 2>&1
+	$here/src/mkswap $fname
+}
+
Quoted 8 lines

tests/generic/638 line 59

Quoted 4 lines
tests/generic/638 new file +4 -0
+_scratch_mkfs >> $seqres.full 2>&1
+_scratch_mount
+psize=`get_page_size`
+bsize=`_get_block_size $SCRATCH_MNT`

_get_file_block_size, since fcollapse requires arguments aligned to the file's allocation unit, which isn't necessarily the same as the fs block size (bigalloc, rt, etc.)...

You're right.

Quoted 10 lines

tests/generic/638 line 64

Quoted 6 lines
tests/generic/638 new file +5 -0
+# Due to we need page-unaligned blocks, so blocksize < pagesize is necessary.
+# If not, try to make a smaller enough block size
+if [ $bsize -ge $psize ];then
+	_scratch_unmount
+	_scratch_mkfs_blocksized 1024 >> $seqres.full 2>&1

...which leads me to the next question, which is: Should the above helper override mkfs options as necessary to enforce that the allocation unit is 1024 bytes? Or is it time for a new helper?

Hmm... the _scratch_mkfs_blocksized() does:

xfs)
_scratch_mkfs_xfs $MKFS_OPTIONS -b size=$blocksize

_scratch_mkfs_xfs() already use $MKFS_OPTIONS internally, don't need to give $MKFS_OPTIONS to it again. Although _scratch_mkfs_xfs deals with $MKFS_OPTIONS, it just drops whole $MKFS_OPTIONS if there're conflicts.

The _scratch_mkfs_geom() does overwrite, but _scratch_mkfs_sized doesn't (sized filesystem is rarely seen in local.config). We might let _scratch_mkfs_blocksized learn _scratch_mkfs_geom.

Quoted 9 lines

tests/generic/638 line 69

Quoted 6 lines
tests/generic/638 new file +5 -0
+	if [ $? -ne 0 ];then
+		_notrun "Can't make filesystem block size < page size"
+	fi
+	_scratch_mount
+	bsize=1024

You probably want to re-check _get_file_block_size to make sure that you actually got blocksize < pagesize in the format attempt.

Quoted 2 lines

Yes, you're right. _scratch_mkfs_blocksized can't make sure to get that blocksize.

Sorry, about delay in getting a test ready for this kernel bug. Also I only now noticed this patch of yours, after I had sent a patch generic/639 to cover this bug. (sheer coincidence).

I have tested [1] on both x86 and Power platform for all filesystems config. (ext4-1k, ext4-4k. ext4-64k, ext2, xfs-1k, xfs-4k, xfs-64k, btrfs) (64k if for Power) But I am fine if you would like to push your patch upstream.

[1]: https://patchwork.kernel.org/project/fstests/patch/e1f9798462ef60648db24b6291e1b149b114f2f2.1622105066.git.riteshh@linux.ibm.com/

-ritesh

Acked-by on v1 1/2
#5 lore raw
In reply to Zorro Lang
Quoted 10 lines · Zorro Lang

xfstests set "-bsize=4k" to XFS_MKFS_OPTIONS by default, then give it to MKFS_OPTIOPNS. So MKFS_OPTIOPNS always contains "-bsize=4k" except we set XFS_MKFS_OPTIONS manually.

It's useless to set XFS_MKFS_OPTIONS to "-bsize=4096" by default, especially that will cause all cases with _scratch_mkfs_blocksized() always fail as "-b size option respecified", when test on XFS. For
exmaple: generic/222

/me wonders what the historical context is (a) for having an $FSTYP-specific MKFS_OPTIONS variable, and (b) for setting bsize=4096 here. All I can say is that I've tripped over this numerous times and wouldn't really mind if it went away. :P

(acked in the sense that I'm not /opposed/ but really really wants to know the historical reasons)

--D

Quoted 19 lines
---
 common/config | 1 -
 1 file changed, 1 deletion(-)
common/config +0 -1
index a47e462c..ecd59fae 100644
@@ -60,7 +60,6 @@ export EMAIL=root@localhost    # where auto-qa will send its status messages
 export HOST_OPTIONS=${HOST_OPTIONS:=local.config}
 export CHECK_OPTIONS=${CHECK_OPTIONS:="-g auto"}
 export BENCH_PASSES=${BENCH_PASSES:=5}
-export XFS_MKFS_OPTIONS=${XFS_MKFS_OPTIONS:=-bsize=4096}
 export TIME_FACTOR=${TIME_FACTOR:=1}
 export LOAD_FACTOR=${LOAD_FACTOR:=1}
 export DEBUGFS_MNT=${DEBUGFS_MNT:="/sys/kernel/debug"}
Signature · 1 line
2.31.1
#6 lore raw
Quoted 17 lines · Darrick J. Wong
Quoted 10 lines · Zorro Lang

xfstests set "-bsize=4k" to XFS_MKFS_OPTIONS by default, then give it to MKFS_OPTIOPNS. So MKFS_OPTIOPNS always contains "-bsize=4k" except we set XFS_MKFS_OPTIONS manually.

It's useless to set XFS_MKFS_OPTIONS to "-bsize=4096" by default, especially that will cause all cases with _scratch_mkfs_blocksized() always fail as "-b size option respecified", when test on XFS. For
exmaple: generic/222

/me wonders what the historical context is (a) for having an $FSTYP-specific MKFS_OPTIONS variable, and (b) for setting bsize=4096 here. All I can say is that I've tripped over this numerous times and wouldn't really mind if it went away. :P

Quoted 4 lines

(acked in the sense that I'm not /opposed/ but really really wants to know the historical reasons)

Looked at the git log, and it was added in 2004 without any explaination. I'll let the patch pending for another week, to see if anyone knows the history behind this.

commit cb6beb975950121f0569c5dede6136fb8ae0b8b8

Author: ptools <ptools>
Date:   Tue Jun 15 07:32:36 2004 +0000

Fixed merge problems

Thanks,
Eryu