btrfs-progs: tests: Make 64K page size system happier

[BUG]
On aarch64 with 64k page size, mkfs.btrfs -s option doesn't work:

  $ mkfs.btrfs  -s 4096 ~/10G.img  -f
  btrfs-progs v5.1.1
  See http://btrfs.wiki.kernel.org for more information.

  Label:              (null)
  UUID:               c2a09334-aaca-4980-aefa-4b3e27390658
  Node size:          65536
  Sector size:        65536		<< Still 64K, not 4K
  Filesystem size:    10.00GiB
  Block group profiles:
    Data:             single            8.00MiB
    Metadata:         DUP             256.00MiB
    System:           DUP               8.00MiB
  SSD detected:       no
  Incompat features:  extref, skinny-metadata
  Number of devices:  1

Devices:
ID SIZE PATH
1 10.00GiB /home/adam/10G.img

[CAUSE]
This is because we automatically detect sectorsize based on current system page size, then get the maxium number between user specified -s parameter and system page size.

It's fine for x86 as it has fixed page size 4K, also the minimium valid sector size.

But for system like aarch64 or ppc64le, where we can have 64K page size, and it makes us unable to create a 4k sector sized btrfs.

[FIX]
Only do auto detect when no -s|--sectorsize option is specified.

Signed-off-by Qu Wenruo <[email protected]>

---
 mkfs/main.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

mkfs/main.c +2 -0

index 8dbec0717b89..26d84e9dafc3 100644
@@ -817,6 +817,7 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 	char *source_dir = NULL;
 	bool source_dir_set = false;
 	bool shrink_rootdir = false;
+	bool sectorsize_set = false;
 	u64 source_dir_size = 0;
 	u64 min_dev_size;
 	u64 shrink_size;

@@ -906,6 +907,7 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 				}
 			case 's':
 				sectorsize = parse_size(optarg);
+				sectorsize_set = true;
 				break;
 			case 'b':
 				block_count = parse_size(optarg);

mkfs/main.c +5 -0

+		error(
+		"invalid sectorsize: %u, expect either 4k, 8k, 16k, 32k or 64k",
+			sectorsize);
+		goto error;
+	}
 	stripesize = sectorsize;
 	saved_optind = optind;
 	dev_cnt = argc - optind;

Quoted 65 lines · Qu Wenruo

[BUG]
On aarch64 with 64k page size, mkfs.btrfs -s option doesn't work:

  $ mkfs.btrfs  -s 4096 ~/10G.img  -f
  btrfs-progs v5.1.1
  See http://btrfs.wiki.kernel.org for more information.

  Label:              (null)
  UUID:               c2a09334-aaca-4980-aefa-4b3e27390658
  Node size:          65536
  Sector size:        65536		<< Still 64K, not 4K
  Filesystem size:    10.00GiB
  Block group profiles:
    Data:             single            8.00MiB
    Metadata:         DUP             256.00MiB
    System:           DUP               8.00MiB
  SSD detected:       no
  Incompat features:  extref, skinny-metadata
  Number of devices:  1

Devices:
ID SIZE PATH
1 10.00GiB /home/adam/10G.img

[CAUSE]
This is because we automatically detect sectorsize based on current system page size, then get the maxium number between user specified -s parameter and system page size.

It's fine for x86 as it has fixed page size 4K, also the minimium valid sector size.

But for system like aarch64 or ppc64le, where we can have 64K page size, and it makes us unable to create a 4k sector sized btrfs.

[FIX]
Only do auto detect when no -s|--sectorsize option is specified.

Signed-off-by Qu Wenruo <[email protected]>

---
 mkfs/main.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

mkfs/main.c +2 -0

index 8dbec0717b89..26d84e9dafc3 100644
@@ -817,6 +817,7 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 	char *source_dir = NULL;
 	bool source_dir_set = false;
 	bool shrink_rootdir = false;
+	bool sectorsize_set = false;
 	u64 source_dir_size = 0;
 	u64 min_dev_size;
 	u64 shrink_size;

@@ -906,6 +907,7 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 				}
 			case 's':
 				sectorsize = parse_size(optarg);
+				sectorsize_set = true;
 				break;
 			case 'b':
 				block_count = parse_size(optarg);

@@ -943,7 +945,15 @@ int BOX_MAIN(mkfs)(int argc, char **argv)
 		printf("See %s for more information.\n\n", PACKAGE_URL);
 	}
 

mkfs/main.c int BOX_MAIN(mkfs)(int argc, char **argv) · line 949

Quoted 3 lines

mkfs/main.c +2 -1

-	sectorsize = max(sectorsize, (u32)sysconf(_SC_PAGESIZE));
+	if (!sectorsize_set)
+		sectorsize = max(sectorsize, (u32)sysconf(_SC_PAGESIZE));

This means it's possible for the user to create a filesystem that is not mountable on his current machine, due to the presence of the following check in validate_super:

if (sectorsize != PAGE_SIZE) {
btrfs_err(..)
}

Perhaps the risk is not that big since if someone creates such a filesystem they will almost instantly realize it won't work and re-create it properly.

mkfs/main.c int BOX_MAIN(mkfs)(int argc, char **argv) · line 951

Quoted 5 lines

mkfs/main.c +2 -0

+	if (!is_power_of_2(sectorsize) || sectorsize < 4096 ||
+	    sectorsize > SZ_64K) {

nit: Perhaps this check should be modified so that it follows the kernel style :
if (!is_power_of_2(sectorsize) || sectorsize < 4096 ||

              sectorsize > BTRFS_MAX_METADATA_BLOCKSIZE) {

MAX_METADATA_BLOCKSIZE is defined as 64k but using the same defines seems more clear to me.

Quoted 11 lines

mkfs/main.c +5 -0

+		error(
+		"invalid sectorsize: %u, expect either 4k, 8k, 16k, 32k or 64k",
+			sectorsize);
+		goto error;
+	}
 	stripesize = sectorsize;
 	saved_optind = optind;
 	dev_cnt = argc - optind;

[BUG]
When doing test on platforms with page size other than 4K (e.g aarch64
can use 64K page size, or ppc64le), certain test wills fail like:
[TEST/fsck] 012-leaf-corruption
mount: /home/adam/btrfs-progs/tests/mnt: wrong fs type, bad option, bad superblock on /dev/loop4, missing codepage or helper program, or other error.
find: '/home/adam/btrfs-progs/tests//mnt/lost+found': No such file or directory
inode 1862 not recovered correctly
test failed for case 012-leaf-corruption

[TEST/fsck] 028-unaligned-super-dev-sizes
failed: mount -t btrfs -o loop ./dev_and_super_mismatch_unaligned.raw.restored /home/adam/btrfs-progs/tests//mnt test failed for case 028-unaligned-super-dev-sizes

[TEST/fsck] 037-freespacetree-repair
failed: /home/adam/btrfs-progs/mkfs.btrfs -f -n 4k /home/adam/btrfs-progs/tests//test.img test failed for case 037-freespacetree-repair

[CAUSE]

For fsck/012 and fsck/028, it's caused by the lack of subpage size sector size support, thus we require kernel page size to match on-disk sector size:
BTRFS error (device loop4): sectorsize 4096 not supported yet, only support 65536 BTRFS error (device loop4): superblock contains fatal errors
BTRFS error (device loop4): open_ctree failed

For fsck/037, it's mkfs causing the problem as we're using 4k nodesize, but on 64K page sized system, we will use 64K sectorsize and cause conflicts.

[FIX]
Considering it's easier and easier to get aarch64 boards with enough performance (e.g rpi4, rk3399, S922) to compile kernel and run tests, let's skip such tests before widespread complain comes.

This patch will introduce a new check, check_prereq_mount_with_sectorsize(), which will test if kernel can mount btrfs with specified sectorsize. So that even one day we support subpage sized sectorsize, we won't need to update test case again.

For fsck/037, also specify sector size manually. And since in that case we still need to mount the fs, also add check_prereq_mount_with_sectorsize() call.

Signed-off-by Qu Wenruo <[email protected]>

---
 tests/common                                  | 29 +++++++++++++++++++
 tests/fsck-tests/012-leaf-corruption/test.sh  |  1 +
 .../028-unaligned-super-dev-sizes/test.sh     |  1 +
 .../037-freespacetree-repair/test.sh          |  3 +-
 4 files changed, 33 insertions(+), 1 deletion(-)

tests/common +18 -0

index 79a16f1e187d..5ad16b69b61d 100644
@@ -306,6 +306,35 @@ check_prereq()
 	fi
 }
 
+# Require to mount images with speicified sectorsize
+# This is to make sure we can run this test on different arch
+# (e.g aarch64 with 64K pagesize)
+check_prereq_mount_with_sectorsize()
+{
+	prepare_test_dev 128M
+	check_prereq mkfs.btrfs
+	setup_root_helper
+
+	local sectorsize=$1
+	local loop_opt
+
+	if [[ -b "$TEST_DEV" ]]; then
+		loop_opt=""
+	elif [[ -f "$TEST_DEV" ]]; then
+		loop_opt="-o loop"
+	else
+		_fail "Invalid \$TEST_DEV: $TEST_DEV"

tests/common +1 -0

+
 check_global_prereq()
 {
 	which "$1" &> /dev/null

tests/fsck-tests/012-leaf-corruption/test.sh +0 -0

index 68d9f695d4de..d5da1d210f28 100755

tests/fsck-tests/012-leaf-corruption/test.sh +0 -0

 check_image test.img
 check_leaf_corrupt_no_data_ext test.img

tests/fsck-tests/028-unaligned-super-dev-sizes/test.sh +1 -0

index 4015df2d8570..49fa35241d04 100755
@@ -6,6 +6,7 @@
 source "$TEST_TOP/common"
 
 check_prereq btrfs
+check_prereq_mount_with_sectorsize 4096
 setup_root_helper
 
 check_all_images

tests/fsck-tests/037-freespacetree-repair/test.sh +1 -0

index 7f547a33512d..32e6651ac705 100755
@@ -10,6 +10,7 @@ prepare_test_dev 256M
 
 check_prereq btrfs
 check_prereq mkfs.btrfs
+check_prereq_mount_with_sectorsize 4096
 check_global_prereq grep
 check_global_prereq tail
 check_global_prereq head

tests/fsck-tests/037-freespacetree-repair/test.sh +0 -0

 run_check_mount_test_dev -oclear_cache,space_cache=v2
 
 # create files which will populate the FST

Signature · 1 line

2.22.0

Quoted 82 lines · Qu Wenruo

[BUG]
When doing test on platforms with page size other than 4K (e.g aarch64
can use 64K page size, or ppc64le), certain test wills fail like:
[TEST/fsck] 012-leaf-corruption
mount: /home/adam/btrfs-progs/tests/mnt: wrong fs type, bad option, bad superblock on /dev/loop4, missing codepage or helper program, or other error.
find: '/home/adam/btrfs-progs/tests//mnt/lost+found': No such file or directory
inode 1862 not recovered correctly
test failed for case 012-leaf-corruption

[TEST/fsck] 028-unaligned-super-dev-sizes
failed: mount -t btrfs -o loop ./dev_and_super_mismatch_unaligned.raw.restored /home/adam/btrfs-progs/tests//mnt test failed for case 028-unaligned-super-dev-sizes

[TEST/fsck] 037-freespacetree-repair
failed: /home/adam/btrfs-progs/mkfs.btrfs -f -n 4k /home/adam/btrfs-progs/tests//test.img test failed for case 037-freespacetree-repair

[CAUSE]

For fsck/012 and fsck/028, it's caused by the lack of subpage size sector size support, thus we require kernel page size to match on-disk sector size:
BTRFS error (device loop4): sectorsize 4096 not supported yet, only support 65536 BTRFS error (device loop4): superblock contains fatal errors
BTRFS error (device loop4): open_ctree failed

For fsck/037, it's mkfs causing the problem as we're using 4k nodesize, but on 64K page sized system, we will use 64K sectorsize and cause conflicts.

[FIX]
Considering it's easier and easier to get aarch64 boards with enough performance (e.g rpi4, rk3399, S922) to compile kernel and run tests, let's skip such tests before widespread complain comes.

This patch will introduce a new check, check_prereq_mount_with_sectorsize(), which will test if kernel can mount btrfs with specified sectorsize. So that even one day we support subpage sized sectorsize, we won't need to update test case again.

For fsck/037, also specify sector size manually. And since in that case we still need to mount the fs, also add check_prereq_mount_with_sectorsize() call.

Signed-off-by Qu Wenruo <[email protected]>

---
 tests/common                                  | 29 +++++++++++++++++++
 tests/fsck-tests/012-leaf-corruption/test.sh  |  1 +
 .../028-unaligned-super-dev-sizes/test.sh     |  1 +
 .../037-freespacetree-repair/test.sh          |  3 +-
 4 files changed, 33 insertions(+), 1 deletion(-)

tests/common +23 -0

index 79a16f1e187d..5ad16b69b61d 100644
@@ -306,6 +306,35 @@ check_prereq()
 	fi
 }
 
+# Require to mount images with speicified sectorsize
+# This is to make sure we can run this test on different arch
+# (e.g aarch64 with 64K pagesize)
+check_prereq_mount_with_sectorsize()
+{
+	prepare_test_dev 128M
+	check_prereq mkfs.btrfs
+	setup_root_helper
+
+	local sectorsize=$1
+	local loop_opt
+
+	if [[ -b "$TEST_DEV" ]]; then
+		loop_opt=""
+	elif [[ -f "$TEST_DEV" ]]; then
+		loop_opt="-o loop"
+	else
+		_fail "Invalid \$TEST_DEV: $TEST_DEV"
+	fi
+
+	run_check_mkfs_test_dev -f -s $sectorsize
+	$SUDO_HELPER mount -t btrfs $loop_opt "$TEST_DEV" "$TEST_MNT" \
+		&> /dev/null

tests/common check_prereq() · line 336

Quoted 5 lines

tests/common +5 -0

+	if [ $? -ne 0 ]; then
+		_not_run "kernel doesn't support sectorsize $sectorsize"
+	fi
+	run_check_umount_test_dev
+}

I think the reason why each requires the specific 4k size should be documented there.

This helper hides several things, like forcing 128M on
the default test device, mkfs and mount. Those are probably inevitable to make sure the combination is supported but the helper should be independent and not possibly interfere with anything the test could use.

The helper could be added as a separate patch, where the updated tests should document the reasons for using it.

Quoted 11 lines

tests/common +1 -0

+
 check_global_prereq()
 {
 	which "$1" &> /dev/null

tests/fsck-tests/012-leaf-corruption/test.sh +0 -0

index 68d9f695d4de..d5da1d210f28 100755
@@ -107,6 +107,7 @@ check_leaf_corrupt_no_data_ext()
 

tests/fsck-tests/012-leaf-corruption/test.sh check_leaf_corrupt_no_data_ext() · line 111

Quoted 4 lines

tests/fsck-tests/012-leaf-corruption/test.sh +1 -0

 setup_root_helper
 
+check_prereq_mount_with_sectorsize 4096
 generate_leaf_corrupt_no_data_ext test.img

So the tests with pre-generated images can't succeed so the test must be skipped, unless we have images for all valid page size values.

Quoted 31 lines

tests/fsck-tests/012-leaf-corruption/test.sh +0 -0

 check_image test.img
 check_leaf_corrupt_no_data_ext test.img

tests/fsck-tests/028-unaligned-super-dev-sizes/test.sh +1 -0

index 4015df2d8570..49fa35241d04 100755
@@ -6,6 +6,7 @@
 source "$TEST_TOP/common"
 
 check_prereq btrfs
+check_prereq_mount_with_sectorsize 4096
 setup_root_helper
 
 check_all_images

tests/fsck-tests/037-freespacetree-repair/test.sh +1 -0

index 7f547a33512d..32e6651ac705 100755
@@ -10,6 +10,7 @@ prepare_test_dev 256M
 
 check_prereq btrfs
 check_prereq mkfs.btrfs
+check_prereq_mount_with_sectorsize 4096
 check_global_prereq grep
 check_global_prereq tail
 check_global_prereq head

@@ -55,7 +56,7 @@ if ! [ -f "/sys/fs/btrfs/features/free_space_tree" ]; then
 	exit
 fi
 

tests/fsck-tests/037-freespacetree-repair/test.sh if ! [ -f "/sys/fs/btrfs/features/free_space_tree" ]; then · line 59

Quoted 2 lines

tests/fsck-tests/037-freespacetree-repair/test.sh +1 -1

-run_check_mkfs_test_dev -n 4k
+run_check_mkfs_test_dev -s 4k -n 4k

Can the test be updated so it always succeeds, ie. giving it a valid sectorsize/nodesize based on getconf? Besides the hardcoded size values (for the device and files), I don't see anything that would mandate 4k sectorsize.

Quoted 6 lines

tests/fsck-tests/037-freespacetree-repair/test.sh +0 -0

 run_check_mount_test_dev -oclear_cache,space_cache=v2
 
 # create files which will populate the FST

Signature · 1 line

2.22.0

[BUG]
Test case 010-minimal-size fails on aarch64 with 64K page size:

      [TEST/mkfs]   010-minimal-size

failed: /home/adam/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single /home/adam/btrfs-progs/tests//test.img test failed for case 010-minimal-size
make: *** [Makefile:361: test-mkfs] Error 1

[CAUSE]
Mkfs.btrfs defaults to page size as sector size. However this test uses 4k, 16k, 32K and 64K as node size. 4K node size will conflict with 64K sector size.

[FIX]
- Specify sector size 4K manually
So at least no conflict at mkfs time.

- Skip the test case if kernel can't mount with 4k sector size So once we add such support, the test can be automatically re-enabled.

Signed-off-by Qu Wenruo <[email protected]>

---
 tests/mkfs-tests/010-minimal-size/test.sh | 41 ++++++++++++-----------
 1 file changed, 21 insertions(+), 20 deletions(-)

tests/mkfs-tests/010-minimal-size/test.sh +0 -0

index 8480e4c5ae23..b49fad63e519 100755

Quoted 29 lines · Qu Wenruo

[BUG]
Test case 010-minimal-size fails on aarch64 with 64K page size:

      [TEST/mkfs]   010-minimal-size

failed: /home/adam/btrfs-progs/mkfs.btrfs -f -n 4k -m single -d single /home/adam/btrfs-progs/tests//test.img test failed for case 010-minimal-size
make: *** [Makefile:361: test-mkfs] Error 1

[CAUSE]
Mkfs.btrfs defaults to page size as sector size. However this test uses 4k, 16k, 32K and 64K as node size. 4K node size will conflict with 64K sector size.

[FIX]
- Specify sector size 4K manually
So at least no conflict at mkfs time.

- Skip the test case if kernel can't mount with 4k sector size So once we add such support, the test can be automatically re-enabled.

Signed-off-by Qu Wenruo <[email protected]>

---
 tests/mkfs-tests/010-minimal-size/test.sh | 41 ++++++++++++-----------
 1 file changed, 21 insertions(+), 20 deletions(-)

tests/mkfs-tests/010-minimal-size/test.sh +0 -0

index 8480e4c5ae23..b49fad63e519 100755

tests/mkfs-tests/010-minimal-size/test.sh source "$TEST_TOP/common" · line 8

Quoted 4 lines

tests/mkfs-tests/010-minimal-size/test.sh +1 -0

@@ -5,6 +5,7 @@ source "$TEST_TOP/common"
 
 check_prereq mkfs.btrfs
 check_prereq btrfs
+check_prereq_mount_with_sectorsize 4096

How about

	if `getconf PAGE_SIZE` != 4096; then
		_not_run "requires 4k pages"
	fi

and be done?

I don't like hardcoding the sectorsize, the effect should be the same when the test is skipped at the beginning.

When we have the subpage-size functionality, there will be an incompat bit exported through sysfs that we can check.

Until then the cude
pagesize check should do. It is going to be a big change so we'll have to revisit all tests anyway.

Most convert tests needs to mount the converted image, and both reiserfs and ext* uses 4k block size, on 32K page size system we can't mount them and will cause test failure.

Skip most of convert tests, except 007-unsupported-block-sizes, which should fail on all systems.

tests/convert-tests/006-large-hole-extent/test.sh +0 -0

 check_global_prereq mke2fs
 
 default_mke2fs="mke2fs -t ext4 -b 4096"

Since I got another rockpro64, finally I could do some tests with aarch64 64K page size mode. (The first board is working as a NAS for a while)

Unsurprisingly there are several false test alerts in btrfs-progs selftests.

Although there is no existing CI service based on 64K page sized system, we'd better support for 64K page size as it's easier and easier to get SBC with good enough aarch64 SoC to compile kernel/btrfs-progs and run various tests on them.

The first patch fix a bug which mkfs can't accept any sector size on 64K page size system.

The remaining patches enhance test cases to make them work on 64K page size system (skip those tests unless kernel support subpage sized sector size)

Qu Wenruo (5):
btrfs-progs: mkfs: Apply the sectorsize user specified on 64k page
size system
btrfs-progs: fsck-tests: Check if current kernel can mount fs with
specified sector size
btrfs-progs: mkfs-tests: Skip 010-minimal-size if we can't mount with
4k sector size
btrfs-progs: misc-tests: Make test cases work or skipped on 64K page
size system
btrfs-progs: convert-tests: Skip tests if kernel doesn't support
subpage sized sector size