btrfs-progs: test/common: Introduce run_mustfail_stdout and enhance prepare_test_dev

PATCH In review Qu Wenruo 3 participants 6 messages Oct 27, 2017 – Oct 30, 2017 Btrfs file system

Share & tools

Qu Wenruo

author

2017-10-27 00:53 UTC #0 lore raw

PATCH v3 1/2 btrfs-progs: test/common: Introduce run_mustfail_stdout and enhance prepare_test_dev

Introduce a new function, run_mustfail_stdout(), which is similar to run_mustfail(), with extra output redirection.
Also this new run_mustfail_stdout() doesn't set pipefail and use temporary file to catch the output.
So it doesn't support command with pipeline.

Also enhance prepare_test_dev() to truncates the file even it already exists.
So test device size will be reset each time prepare_test_dev() get called.

Signed-off-by Qu Wenruo <[email protected]>

---
changelog:
v2:
Nothing
v3:
Split the test infrastructure modification into its own patch.

---
 tests/common | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 57 insertions(+), 1 deletion(-)

tests/common +57 -1

index eb525a4d02c5..80392c3a6d7d 100644
@@ -236,6 +236,58 @@ run_mustfail()
 	fi
 }
 
+# The first parameter is error message to print if it fails, just like
+# run_must_fail().
+# Also we don't use pipefail to avoid disturbing other script, so here we
+# use temporary output file.
+# So it doesn't support pipeline in the @cmd
+run_mustfail_stdout()
+{
+	local spec
+	local ins
+	local cmd
+	local msg
+	local ret
+	local tmp_output
+
+	tmp_output=$(mktemp --tmpdir btrfs-progs-test--mustfail-stdtout.XXXXXX)
+
+	msg="$1"
+	shift
+
+	if _is_file_or_command "$msg"; then
+		echo "ASSERTION FAIL: 1st argument of run_mustfail_stdout must be a message"
+		exit 1
+	fi
+
+	ins=$(_get_spec_ins "$@")
+	spec=$(($ins-1))
+	cmd=$(eval echo "\${$spec}")
+	spec=$(_cmd_spec "${@:$spec}")
+	set -- "${@:1:$(($ins-1))}" $spec "${@: $ins}"
+	echo "############### $@" >> "$RESULTS" 2>&1
+	if [[ $TEST_LOG =~ tty ]]; then echo "CMD(mustfail): $@" > /dev/tty; fi
+	if [ "$1" = 'root_helper' ]; then
+		"$@" 2>&1 > "$tmp_output"
+	else
+		$INSTRUMENT "$@" 2>&1 > "$tmp_output"
+	fi
+	ret=$?
+
+	cat "$tmp_output" >> "$RESULTS"
+	cat "$tmp_output"
+	rm "$tmp_output"
+
+	if [ "$ret" != 0 ]; then
+		echo "failed (expected): $@" >> "$RESULTS"
+		return 0
+	else
+		echo "succeeded (unexpected!): $@" >> "$RESULTS"
+		_fail "unexpected success: $msg"
+		return 1
+	fi
+}
+
 check_prereq()
 {
 	if ! [ -f "$TOP/$1" ]; then

@@ -389,8 +441,12 @@ prepare_test_dev()
 	# num[K/M/G/T...]
 	local size="$1"
 
-	[[ "$TEST_DEV" ]] && return
 	[[ "$size" ]] || size='2G'
+	# Still truncate it to new size
+	if [[ "$TEST_DEV" ]]; then
+		truncate -s "$size" "$TEST_DEV"
+		return;
+	fi
 
 	echo "\$TEST_DEV not given, use $TOP/test/test.img as fallback" >> \
 		"$RESULTS"

Signature · 1 line

2.14.3

Qu Wenruo

author

2017-10-27 00:53 UTC #1 lore raw

PATCH v3 2/2 btrfs-progs: test/mkfs: Test if the minimal device size is valid

New test case to test if the minimal device size given by "mkfs.btrfs" failure case is valid.

Signed-off-by Qu Wenruo <[email protected]>

---
 tests/mkfs-tests/010-small-image/test.sh | 49 ++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100755 tests/mkfs-tests/010-small-image/test.sh

tests/mkfs-tests/010-small-image/test.sh new file +49 -0

new file mode 100755
index 000000000000..f55760df061f
@@ -0,0 +1,49 @@
+#!/bin/bash
+# test if the reported minimal size of mkfs.btrfs is valid
+
+source "$TOP"/tests/common
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper
+
+do_test()
+{
+	# Well, 1M small enough to fail, we just use the output
+	# to get the minimal device size
+	prepare_test_dev 1M
+	output=$(run_mustfail_stdout "mkfs.btrfs for small image" \
+		 "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV")
+	good_size=$(echo "$output" | grep -oP "(?<=is )\d+")
+
+	prepare_test_dev "$good_size"
+	run_check "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV"
+	run_check $SUDO_HELPER mount "$TEST_DEV" "$TEST_MNT"
+	run_check $SUDO_HELPER umount "$TEST_MNT"
+}
+
+do_test -n 4k	-m single	-d single
+do_test -n 4k	-m single	-d dup
+do_test -n 4k	-m dup		-d single
+do_test -n 4k	-m dup		-d dup
+
+do_test -n 8k	-m single	-d single
+do_test -n 8k	-m single	-d dup
+do_test -n 8k	-m dup		-d single
+do_test -n 8k	-m dup		-d dup
+
+do_test -n 16k	-m single	-d single
+do_test -n 16k	-m single	-d dup
+do_test -n 16k	-m dup		-d single
+do_test -n 16k	-m dup		-d dup
+
+do_test -n 32k	-m single	-d single
+do_test -n 32k	-m single	-d dup
+do_test -n 32k	-m dup		-d single
+do_test -n 32k	-m dup		-d dup
+
+do_test -n 64k	-m single	-d single
+do_test -n 64k	-m single	-d dup
+do_test -n 64k	-m dup		-d single
+do_test -n 64k	-m dup		-d dup

Signature · 1 line

2.14.3

David Sterba

2017-10-27 17:01 UTC #2 lore raw

Re: [PATCH v3 2/2] btrfs-progs: test/mkfs: Test if the minimal device size is valid

Quoted 28 lines · Qu Wenruo

New test case to test if the minimal device size given by "mkfs.btrfs" failure case is valid.

Signed-off-by Qu Wenruo <[email protected]>

---
 tests/mkfs-tests/010-small-image/test.sh | 49 ++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100755 tests/mkfs-tests/010-small-image/test.sh

tests/mkfs-tests/010-small-image/test.sh new file +13 -0

new file mode 100755
index 000000000000..f55760df061f
@@ -0,0 +1,49 @@
+#!/bin/bash
+# test if the reported minimal size of mkfs.btrfs is valid
+
+source "$TOP"/tests/common
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper
+
+do_test()
+{
+	# Well, 1M small enough to fail, we just use the output

tests/mkfs-tests/010-small-image/test.sh line 23

Quoted 10 lines

tests/mkfs-tests/010-small-image/test.sh new file +10 -0

+	# to get the minimal device size
+	prepare_test_dev 1M
+	output=$(run_mustfail_stdout "mkfs.btrfs for small image" \
+		 "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV")
+	good_size=$(echo "$output" | grep -oP "(?<=is )\d+")
+
+	prepare_test_dev "$good_size"
+	run_check "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV"
+	run_check $SUDO_HELPER mount "$TEST_DEV" "$TEST_MNT"
+	run_check $SUDO_HELPER umount "$TEST_MNT"

There are helpers for the default mounts, you can use them to save some
typing: run_check_mount_test_dev and run_check_umount_test_dev.

tests/mkfs-tests/010-small-image/test.sh line 26

Quoted 3 lines

tests/mkfs-tests/010-small-image/test.sh new file +3 -0
+}
+
+do_test -n 4k	-m single	-d single

I don't seem to get this test to work, it fails with this in the log:

=== Entering btrfs-progs/tests/mkfs-tests/010-small-image
$TEST_DEV not given, use btrfs-progs/test/test.img as fallback
############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.

failed (expected): btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img ############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               b9b55b1b-4096-46ee-acd0-15151a8050d6
Node size:          4096
Sector size:        4096
Filesystem size:    16.00MiB
Block group profiles:
  Data:             single            1.56MiB
  Metadata:         single            1.56MiB
  System:           single            4.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1

Devices:
ID SIZE PATH
1 16.00MiB btrfs-progs/tests/test.img

############### root_helper mount btrfs-progs/tests/test.img btrfs-progs/tests/mnt ############### root_helper umount btrfs-progs/tests/mnt
############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.

failed (expected): btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img ############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img
ERROR: not enough free space to allocate chunk
btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.

failed: btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img test failed for case 010-small-image
---

The output is unexpected and does not contain the minimal size. The tested branch is current devel (7141063af3c9a423dda), your mkfs --rootdir rework patches are there. Can you please have a look?

Qu Wenruo

2017-10-27 23:59 UTC #3 lore raw

Re: [PATCH v3 2/2] btrfs-progs: test/mkfs: Test if the minimal device size is valid

Quoted 43 lines · David Sterba

Quoted 28 lines · Qu Wenruo
New test case to test if the minimal device size given by "mkfs.btrfs" failure case is valid.
Signed-off-by Qu Wenruo <[email protected]>
---
 tests/mkfs-tests/010-small-image/test.sh | 49 ++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100755 tests/mkfs-tests/010-small-image/test.sh
tests/mkfs-tests/010-small-image/test.sh new file +13 -0
new file mode 100755
index 000000000000..f55760df061f
@@ -0,0 +1,49 @@
+#!/bin/bash
+# test if the reported minimal size of mkfs.btrfs is valid
+
+source "$TOP"/tests/common
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper
+
+do_test()
+{
+	# Well, 1M small enough to fail, we just use the output
tests/mkfs-tests/010-small-image/test.sh line 23
Quoted 10 lines
tests/mkfs-tests/010-small-image/test.sh new file +10 -0
+	# to get the minimal device size
+	prepare_test_dev 1M
+	output=$(run_mustfail_stdout "mkfs.btrfs for small image" \
+		 "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV")
+	good_size=$(echo "$output" | grep -oP "(?<=is )\d+")
+
+	prepare_test_dev "$good_size"
+	run_check "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV"
+	run_check $SUDO_HELPER mount "$TEST_DEV" "$TEST_MNT"
+	run_check $SUDO_HELPER umount "$TEST_MNT"
There are helpers for the default mounts, you can use them to save some
typing: run_check_mount_test_dev and run_check_umount_test_dev.

Quoted 5 lines

tests/mkfs-tests/010-small-image/test.sh line 26
Quoted 3 lines
tests/mkfs-tests/010-small-image/test.sh new file +3 -0
+}
+
+do_test -n 4k	-m single	-d single
I don't seem to get this test to work, it fails with this in the log:

Strange, it passed without problem on my box.

=== Entering /home/adam/btrfs/btrfs-progs/tests/mkfs-tests/010-small-image $TEST_DEV not given, use /home/adam/btrfs/btrfs-progs/test/test.img as fallback
############### /home/adam/btrfs/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single /home/adam/btrfs/btrfs-progs/tests/test.img btrfs-progs v4.13.2
See http://btrfs.wiki.kernel.org for more information.

failed (expected): /home/adam/btrfs/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single /home/adam/btrfs/btrfs-progs/tests/test.img ############### /home/adam/btrfs/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single /home/adam/btrfs/btrfs-progs/tests/test.img btrfs-progs v4.13.2
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               04213076-9678-420b-b748-5e5b6394e025
Node size:          4096
Sector size:        4096
Filesystem size:    44.00MiB
Block group profiles:
  Data:             single            4.38MiB
  Metadata:         single            4.38MiB
  System:           single            4.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1

Devices:
ID SIZE PATH
1 44.00MiB /home/adam/btrfs/btrfs-progs/tests/test.img

############### root_helper mount /home/adam/btrfs/btrfs-progs/tests/test.img /home/adam/btrfs/btrfs-progs/tests/mnt
############### root_helper umount /home/adam/btrfs/btrfs-progs/tests/mnt ############### /home/adam/btrfs/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup /home/adam/btrfs/btrfs-progs/tests/test.img btrfs-progs v4.13.2
See http://btrfs.wiki.kernel.org for more information.

failed (expected): /home/adam/btrfs/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup /home/adam/btrfs/btrfs-progs/tests/test.img ############### /home/adam/btrfs/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup /home/adam/btrfs/btrfs-progs/tests/test.img btrfs-progs v4.13.2
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               f7c3a0db-abf1-4061-99f0-1028ac713ed6
Node size:          4096
Sector size:        4096
Filesystem size:    164.00MiB
Block group profiles:
  Data:             DUP              64.00MiB
  Metadata:         single            8.00MiB
  System:           single            4.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1   164.00MiB  /home/adam/btrfs/btrfs-progs/tests/test.img

Maybe it's related to the preparation patch for that? Like the prepare_test_dev() function modification?

Thanks,
Qu

Quoted 52 lines

=== Entering btrfs-progs/tests/mkfs-tests/010-small-image
$TEST_DEV not given, use btrfs-progs/test/test.img as fallback
############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
failed (expected): btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img ############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
Label:              (null)
UUID:               b9b55b1b-4096-46ee-acd0-15151a8050d6
Node size:          4096
Sector size:        4096
Filesystem size:    16.00MiB
Block group profiles:
  Data:             single            1.56MiB
  Metadata:         single            1.56MiB
  System:           single            4.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
ID SIZE PATH
1 16.00MiB btrfs-progs/tests/test.img
############### root_helper mount btrfs-progs/tests/test.img btrfs-progs/tests/mnt ############### root_helper umount btrfs-progs/tests/mnt
############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
failed (expected): btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img ############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img
ERROR: not enough free space to allocate chunk
btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
failed: btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img test failed for case 010-small-image
---
The output is unexpected and does not contain the minimal size. The tested branch is current devel (7141063af3c9a423dda), your mkfs --rootdir rework patches are there. Can you please have a look?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Qu Wenruo

2017-10-30 00:53 UTC #4 lore raw

Re: [PATCH v3 2/2] btrfs-progs: test/mkfs: Test if the minimal device size is valid

In reply to David Sterba

Quoted 43 lines · David Sterba

Quoted 28 lines · Qu Wenruo
New test case to test if the minimal device size given by "mkfs.btrfs" failure case is valid.
Signed-off-by Qu Wenruo <[email protected]>
---
 tests/mkfs-tests/010-small-image/test.sh | 49 ++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100755 tests/mkfs-tests/010-small-image/test.sh
tests/mkfs-tests/010-small-image/test.sh new file +13 -0
new file mode 100755
index 000000000000..f55760df061f
@@ -0,0 +1,49 @@
+#!/bin/bash
+# test if the reported minimal size of mkfs.btrfs is valid
+
+source "$TOP"/tests/common
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper
+
+do_test()
+{
+	# Well, 1M small enough to fail, we just use the output
tests/mkfs-tests/010-small-image/test.sh line 23
Quoted 10 lines
tests/mkfs-tests/010-small-image/test.sh new file +10 -0
+	# to get the minimal device size
+	prepare_test_dev 1M
+	output=$(run_mustfail_stdout "mkfs.btrfs for small image" \
+		 "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV")
+	good_size=$(echo "$output" | grep -oP "(?<=is )\d+")
+
+	prepare_test_dev "$good_size"
+	run_check "$TOP"/mkfs.btrfs -f $@ "$TEST_DEV"
+	run_check $SUDO_HELPER mount "$TEST_DEV" "$TEST_MNT"
+	run_check $SUDO_HELPER umount "$TEST_MNT"
There are helpers for the default mounts, you can use them to save some
typing: run_check_mount_test_dev and run_check_umount_test_dev.

Quoted 5 lines

tests/mkfs-tests/010-small-image/test.sh line 26
Quoted 3 lines
tests/mkfs-tests/010-small-image/test.sh new file +3 -0
+}
+
+do_test -n 4k	-m single	-d single
I don't seem to get this test to work, it fails with this in the log:

Forgot to mention, the test case is for minimal-device-size fix patchset:
Enhance minimal device size calculation to fix mkfs failure on small file

Considering I sent too many patchset recently and most of them have dependency, should I resend a big patchset to address the dependency?

IIRC the dependency should be:
rootdir rework -> rootdir refactor -> minimal device size fix (including test cases)

And later mkfs qgroup support also depends on rootdir rework (just conflicts, not code dependency)

Thanks,
Qu

Quoted 52 lines

=== Entering btrfs-progs/tests/mkfs-tests/010-small-image
$TEST_DEV not given, use btrfs-progs/test/test.img as fallback
############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
failed (expected): btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img ############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
Label:              (null)
UUID:               b9b55b1b-4096-46ee-acd0-15151a8050d6
Node size:          4096
Sector size:        4096
Filesystem size:    16.00MiB
Block group profiles:
  Data:             single            1.56MiB
  Metadata:         single            1.56MiB
  System:           single            4.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
ID SIZE PATH
1 16.00MiB btrfs-progs/tests/test.img
############### root_helper mount btrfs-progs/tests/test.img btrfs-progs/tests/mnt ############### root_helper umount btrfs-progs/tests/mnt
############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
failed (expected): btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img ############### btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img
ERROR: not enough free space to allocate chunk
btrfs-progs v4.13.3-61-g0615ffa351c2
See http://btrfs.wiki.kernel.org for more information.
failed: btrfs-progs/mkfs.btrfs -f -n 4k -m single -d dup btrfs-progs/tests/test.img test failed for case 010-small-image
---
The output is unexpected and does not contain the minimal size. The tested branch is current devel (7141063af3c9a423dda), your mkfs --rootdir rework patches are there. Can you please have a look?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

David Sterba

2017-10-27 16:55 UTC #5 lore raw

In reply to Qu Wenruo

Quoted 10 lines · Qu Wenruo

Introduce a new function, run_mustfail_stdout(), which is similar to run_mustfail(), with extra output redirection.
Also this new run_mustfail_stdout() doesn't set pipefail and use temporary file to catch the output.
So it doesn't support command with pipeline.
Also enhance prepare_test_dev() to truncates the file even it already exists.
So test device size will be reset each time prepare_test_dev() get called.

That's a completely unrelated change to the above and should be also split, please.