btrfs-progs: move RST feature back to experimental
cover btrfs-progs: move RST feature back to experimental
Although we have exported raid-stripe-tree feature to end users, the feature is still experimental as only kernels with CONFIG_BTRFS_DEBUG can even mount it.
This results a feature mismatch in btrfs-progs and kernels, and can lead to complains/confusion from end users.
This patchset would hide RST feature back behind experimental builds for mkfs, and completely disable rst for convert, so end users won't and can't enable RST by accident.
Now RST related test cases (mkfs/029, mkfs/030) are skipped for regular builds, and a new test case (mkfs/033) is skipped for experimental builds to make sure there is really no way to enable RST support for non-experimental builds.
The new experimental build detection also provides the basis for other experimental features like csum conversion in the future (recent days).
Both regular and experimental builds have passed the full test suite (except random misc/055 failure, which is caused by a clear-stale behavior change, and will be addressed separately)
Qu Wenruo (4):
btrfs-progs: convert: remove raid-stripe-tree support
btrfs-progs: hide rst related mkfs tests behind experimental builds
btrfs-progs: fsfeatures: move RST back to experimental
btrfs-progs: mkfs-tests: ensure regular builds won't enable rst
feature
common/fsfeatures.c | 2 ++ common/fsfeatures.h | 3 +- mkfs/main.c | 2 ++ tests/common | 29 ++++++++++++++++ tests/convert-tests/001-ext2-basic/test.sh | 2 +- tests/convert-tests/003-ext4-basic/test.sh | 2 +- .../005-delete-all-rollback/test.sh | 2 +- .../convert-tests/010-reiserfs-basic/test.sh | 2 +- .../011-reiserfs-delete-all-rollback/test.sh | 2 +- tests/convert-tests/024-ntfs-basic/test.sh | 2 +- tests/mkfs-tests/029-raid-stripe-tree/test.sh | 1 + tests/mkfs-tests/030-zoned-rst/test.sh | 1 + tests/mkfs-tests/033-zoned-reject-rst/test.sh | 34 +++++++++++++++++++ 13 files changed, 76 insertions(+), 8 deletions(-) create mode 100755 tests/mkfs-tests/033-zoned-reject-rst/test.sh
--
2.45.2
PATCH 1/4 btrfs-progs: convert: remove raid-stripe-tree support
The raid-stripe-tree (RST) feature is for zoned devices to support extra data profiles, and is not yet a stable feature (still requires CONFIG_BTRFS_DEBUG enabled kernel to support it).
Furthermore the supported filesystems (ext*, reiserfs and ntfs) doesn't even support zoned devices, and even we force RST support for btrfs-convert, we would only create an empty tree for RST, as btrfs convert would only result SINGLE data profile with SINGLE/DUP metadata profile, neither needs RST at all.
Meanwhile enabling RST for btrfs-convert would only cause problems for false test failures as we incorrectly allow RST feature for btrfs-convert.
This patch fixes the problem by remove raid-stripe-tree support from btrfs-convert and remove the test cases support for RST.
This patch is mostly reverting commit 346a3819237b ("btrfs-progs:
convert: add raid-stripe-tree to allowed features"), but keeps the test infrastructure to support bgt features for convert.
--- common/fsfeatures.h | 3 +-- tests/convert-tests/001-ext2-basic/test.sh | 2 +- tests/convert-tests/003-ext4-basic/test.sh | 2 +- tests/convert-tests/005-delete-all-rollback/test.sh | 2 +- tests/convert-tests/010-reiserfs-basic/test.sh | 2 +- tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh | 2 +- tests/convert-tests/024-ntfs-basic/test.sh | 2 +- 7 files changed, 7 insertions(+), 8 deletions(-)
common/fsfeatures.h +1 -2
@@ -68,8 +68,7 @@ static const struct btrfs_mkfs_features btrfs_convert_allowed_features = {BTRFS_FEATURE_INCOMPAT_BIG_METADATA |BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF |BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA | -BTRFS_FEATURE_INCOMPAT_NO_HOLES | - BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE, + BTRFS_FEATURE_INCOMPAT_NO_HOLES, .runtime_flags = BTRFS_FEATURE_RUNTIME_QUOTA, };
tests/convert-tests/001-ext2-basic/test.sh +1 -1
@@ -12,7 +12,7 @@ check_kernel_support_acl # Iterate over defaults and options that are not tied to hardware capabilities # or number of devices -for feature in '' 'block-group-tree' 'raid-stripe-tree'; do +for feature in '' 'block-group-tree'; do convert_test ext2 "$feature" "ext2 4k nodesize" 4096 mke2fs -b 4096 convert_test ext2 "$feature" "ext2 16k nodesize" 16384 mke2fs -b 4096 convert_test ext2 "$feature" "ext2 64k nodesize" 65536 mke2fs -b 4096
tests/convert-tests/003-ext4-basic/test.sh +1 -1
@@ -12,7 +12,7 @@ check_kernel_support_acl # Iterate over defaults and options that are not tied to hardware capabilities # or number of devices -for feature in '' 'block-group-tree' 'raid-stripe-tree' ; do +for feature in '' 'block-group-tree'; do convert_test ext4 "$feature" "ext4 4k nodesize" 4096 mke2fs -t ext4 -b 4096 convert_test ext4 "$feature" "ext4 16k nodesize" 16384 mke2fs -t ext4 -b 4096 convert_test ext4 "$feature" "ext4 64k nodesize" 65536 mke2fs -t ext4 -b 4096
tests/convert-tests/005-delete-all-rollback/test.sh +1 -1
@@ -68,7 +68,7 @@ do_test() { # Iterate over defaults and options that are not tied to hardware capabilities # or number of devices -for feature in '' 'block-group-tree' 'raid-stripe-tree' ; do +for feature in '' 'block-group-tree'; do do_test "$feature" "ext4 4k nodesize" 4096 mke2fs -t ext4 -b 4096 do_test "$feature" "ext4 16k nodesize" 16384 mke2fs -t ext4 -b 4096 do_test "$feature" "ext4 64k nodesize" 65536 mke2fs -t ext4 -b 4096
tests/convert-tests/010-reiserfs-basic/test.sh +1 -1
@@ -15,7 +15,7 @@ prepare_test_dev # Iterate over defaults and options that are not tied to hardware capabilities # or number of devices -for feature in '' 'block-group-tree' 'raid-stripe-tree'; do +for feature in '' 'block-group-tree'; do convert_test reiserfs "$feature" "reiserfs 4k nodesize" 4096 mkreiserfs -b 4096 convert_test reiserfs "$feature" "reiserfs 16k nodesize" 16384 mkreiserfs -b 4096 convert_test reiserfs "$feature" "reiserfs 64k nodesize" 65536 mkreiserfs -b 4096
tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh +1 -1
@@ -70,7 +70,7 @@ do_test() { # Iterate over defaults and options that are not tied to hardware capabilities # or number of devices -for feature in '' 'block-group-tree' 'raid-stripe-tree'; do +for feature in '' 'block-group-tree'; do do_test "$feature" "reiserfs 4k nodesize" 4096 mkreiserfs -b 4096 do_test "$feature" "reiserfs 16k nodesize" 16384 mkreiserfs -b 4096 do_test "$feature" "reiserfs 64k nodesize" 65536 mkreiserfs -b 4096
tests/convert-tests/024-ntfs-basic/test.sh +1 -1
@@ -17,6 +17,6 @@ prepare_test_dev # Iterate over defaults and options that are not tied to hardware capabilities # or number of devices. Test only 4K block size as minimum. -for feature in '' 'block-group-tree' 'raid-stripe-tree'; do +for feature in '' 'block-group-tree'; do convert_test ntfs "$feature" "ntfs 4k nodesize" 4096 mkfs.ntfs -s 4096 done
Signature · 1 line
2.45.2
PATCH 2/4 btrfs-progs: hide rst related mkfs tests behind experimental builds
In reply to Qu WenruoCurrently we unconditionally run mkfs/029 and mkfs/030 as we export RST feature through mkfs.btrfs as supported features.
But considering the mkfs.btrfs features are mismatch with kernel support (only a BTRFS_DEBUG kernel can support that), we're going to hide RST behind experimental builds.
In that case RST related tests cases also need to be behind experimental builds as regular builds of mkfs.btrfs will no longer support RST feature.
This patch would introduce two helpers:
- _test_config()
To verify if certain config is set to 1
- check_experimental_build()
A wrapper of "_test_config EXPERIMENTAL", and skip the test if btrfs-progs has no experimental features.
So that test cases can be skipped for non-experimental builds.
--- tests/common | 22 +++++++++++++++++++ tests/mkfs-tests/029-raid-stripe-tree/test.sh | 1 + tests/mkfs-tests/030-zoned-rst/test.sh | 1 + 3 files changed, 24 insertions(+)
tests/common +22 -0
@@ -6,6 +6,21 @@ # Temporary array for building the final command, injecting arguments as needed declare -a cmd_array +# Check if certain config is set to 1 +_test_config() +{ + local feature="$1" + + if [ ! -f "${TOP}/include/config.h" ]; then + echo "include/config.h not exists" + exit 1 + fi + if grep -q "${feature}.*1" "${TOP}/include/config.h"; then + return 0 + fi + return 1 +} + # assert that argument is not empty and is an existing path (file or directory) _assert_path() {
@@ -370,6 +385,13 @@ run_mustfail_stdout() fi } +check_experimental_build() +{ + if ! _test_config "EXPERIMENTAL"; then + _not_run "This test requires experimental build" + fi +} + check_prereq() { # Internal tools for testing, not shipped with the package
tests/mkfs-tests/029-raid-stripe-tree/test.sh +1 -0
@@ -3,6 +3,7 @@ source "$TEST_TOP/common" || exit +check_experimental_build check_prereq mkfs.btrfs check_prereq btrfs
tests/mkfs-tests/030-zoned-rst/test.sh +1 -0
@@ -3,6 +3,7 @@ source "$TEST_TOP/common" || exit +check_experimental_build setup_root_helper setup_nullbdevs 4 128 4 prepare_nullbdevs
Signature · 1 line
2.45.2
PATCH 3/4 btrfs-progs: fsfeatures: move RST back to experimental
In reply to Qu WenruoCurrently raid-stripe-tree feature is still experimental as it requires a BTRFS_DEBUG kernel to recognize it.
To avoid confusion move it back to experimental so regular users won't incorrectly set it.
And since RST is no longer supported by default, also change the RST profile detection so that for non-experimental build we won't enable RST according to the data profiles.
--- common/fsfeatures.c | 2 ++ mkfs/main.c | 2 ++ 2 files changed, 4 insertions(+)
common/fsfeatures.c +2 -0
@@ -222,6 +222,7 @@ static const struct btrfs_feature mkfs_features[] = { VERSION_NULL(default), .desc = "block group tree, more efficient block group tracking to reduce mount time" }, +#if EXPERIMENTAL { .name = "rst", .incompat_flag = BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE,
@@ -238,6 +239,7 @@ static const struct btrfs_feature mkfs_features[] = { VERSION_NULL(default), .desc = "raid stripe tree, enhanced file extent tracking" }, +#endif { .name = "squota", .incompat_flag = BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA,
mkfs/main.c +2 -0
@@ -1693,7 +1693,9 @@ int BOX_MAIN(mkfs)(int argc, char **argv)case BTRFS_BLOCK_GROUP_RAID1C4 :case BTRFS_BLOCK_GROUP_RAID0 :case BTRFS_BLOCK_GROUP_RAID10 : +#if EXPERIMENTAL features.incompat_flags |=BTRFS_FEATURE_INCOMPAT_RAID_STRIPE_TREE ; +#endif break ;default :break ;
Signature · 1 line
2.45.2
PATCH 4/4 btrfs-progs: mkfs-tests: ensure regular builds won't enable rst feature
In reply to Qu WenruoThis test case will check the following behaivors:
- Regular zoned supported data/metadata profiles
Should success
- RST with zoned feature
Should fail for non-experimental builds
- zoned with data profiles that only RST supports
Should fail for non-experimental builds.
Meanwhile for experimental builds it should success and enable RST feature automatically (verified in mkfs/030)
--- tests/common | 7 ++++ tests/mkfs-tests/033-zoned-reject-rst/test.sh | 34 +++++++++++++++++++ 2 files changed, 41 insertions(+) create mode 100755 tests/mkfs-tests/033-zoned-reject-rst/test.sh
tests/common +7 -0
@@ -392,6 +392,13 @@ check_experimental_build() fi } +check_regular_build() +{ + if _test_config "EXPERIMENTAL"; then + _not_run "This test requires non-experimental build" + fi +} + check_prereq() { # Internal tools for testing, not shipped with the package
tests/mkfs-tests/033-zoned-reject-rst/test.sh new file +34 -0
@@ -0,0 +1,34 @@ +#!/bin/bash +# Verify mkfs for all currently supported profiles of zoned + raid-stripe-tree + +source "$TEST_TOP/common" || exit + +check_regular_build +setup_root_helper +setup_nullbdevs 4 128 4 +prepare_nullbdevs +TEST_DEV=${nullb_devs[1]} + +profiles="dup raid1 raid1c3 raid1c4 raid10" + +# The existing supported profiles. +run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d single -m single "${nullb_devs[@]}" +run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d single -m DUP "${nullb_devs[@]}" + +# RST feature is rejected +run_mustfail "RST feature created" \ + $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned,raid-stripe-tree -d single -m DUP "${nullb_devs[@]}" + +for dprofile in $profiles; do + # make sure extra data profiles won't enable RST for non-experimental build + run_mustfail "unsupported profile created" \ + $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d "$dprofile" -m DUP "${nullb_devs[@]}" +done + +# The old unsupported profiles should fail no matter experimental build or not. +run_mustfail "unsupported profile raid56 created" \ + $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d raid5 -m raid5 "${nullb_devs[@]}" +run_mustfail "unsupported profile raid56 created" \ + $SUDO_HELPER "$TOP/mkfs.btrfs" -f -O zoned -d raid6 -m raid6 "${nullb_devs[@]}" + +cleanup_nullbdevs
Signature · 1 line
2.45.2
Quoted 3 lines
This patchset would hide RST feature back behind experimental builds for mkfs, and completely disable rst for convert, so end users won't and can't enable RST by accident.
How about introduce a option named like "--experimental" to enable these feature?
It can work perfectly and don't need me to re-compiling another btrfs-progs.
HAN Yuwei
Quoted 4 lines · HAN Yuwei
Quoted 3 lines
This patchset would hide RST feature back behind experimental builds for mkfs, and completely disable rst for convert, so end users won't and can't enable RST by accident.
How about introduce a option named like "--experimental" to enable these feature?
Well, that's exactly what we (at least me) want, to make it harder for end uers to reach it.
For you, with your advanced subpage and zoned setup, compiling btrfs-progs should only be a small hassle, and you only need to do it once.
Thanks,
Qu
Quoted 5 lines
It can work perfectly and don't need me to re-compiling another btrfs-progs.
HAN Yuwei
Quoted 11 lines · Qu Wenruo
Quoted 5 lines · HAN Yuwei
Quoted 3 lines
This patchset would hide RST feature back behind experimental builds for mkfs, and completely disable rst for convert, so end users won't and can't enable RST by accident.
How about introduce a option named like "--experimental" to enable these feature?
Well, that's exactly what we (at least me) want, to make it harder for end uers to reach it.
I completely agree with Qu here. As much as I love you being an early adopter for my RST stuff, it is not in a shape where I want it to be used by people.
Also there will be a change in the on-disk format as well, so please do not (yet) use it.
Quoted 13 lines
For you, with your advanced subpage and zoned setup, compiling btrfs-progs should only be a small hassle, and you only need to do it once.
Thanks,
QuQuoted 6 lines
It can work perfectly and don't need me to re-compiling another btrfs-progs.
HAN Yuwei