btrfs-progs: fsck: fix false warning on sprouted filesystems
cover btrfs-progs: fsck: fix false warning on sprouted filesystems
During my attempt to utilize seed device at mkfs time, I found there there is a small bug that btrfs check always reports devices size related warning on the sprouted device.
It turns out we didn't iterate seed devices at all during btrfs check.
The first patch will fix the problem and enhance the warning output. The second one will add a regression test for it.
Qu Wenruo (2):
btrfs-progs: check: fix wrong total bytes check for seed device
btrfs-progs: fsck-tests: check warning for seed and sprouted
filesystems
check/main.c | 18 ++++--- .../fsck-tests/057-seed-false-alerts/test.sh | 51 +++++++++++++++++++ 2 files changed, 63 insertions(+), 6 deletions(-) create mode 100755 tests/fsck-tests/057-seed-false-alerts/test.sh
Signature · 1 line
2.35.1
PATCH 1/2 btrfs-progs: check: fix wrong total bytes check for seed device
[BUG]
The following script can lead to false positive from btrfs check:
mkfs.btrfs -f $dev1
mount $dev1 $mnt
btrfstune -S1 $dev1
mount $dev1 $mnt
btrfs dev add -f $dev2 $mnt
umount $mnt
# Now dev1 is seed, and dev2 is the rw fs.
btrfs check $dev2
...
[2/7] checking extents
WARNING: minor unaligned/mismatch device size detected
WARNING: recommended to use 'btrfs rescue fix-device-size' to fix it
...
This false positive only happens on $dev2, $dev1 is completely fine.
[CAUSE]
The warning is from is_super_size_valid(), in that function we verify the super block total bytes (@super_bytes) is correct against the total device bytes (@total_bytes).
However the when calculating @total_bytes, we only use devices in current fs_devices, which only contains RW devices.
Thus all bytes from seed device are not taken into consideration, and trigger the false positive.
[FIX]
Fix it by also iterating seed devices.
Since we're here, also output @total_bytes and @super_bytes when outputting the warning message, to allow end users have a better idea on what's going wrong.
--- check/main.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
check/main.c +12 -6
@@ -8550,13 +8550,17 @@ static int check_device_used(struct device_record *dev_rec,*/ static bool is_super_size_valid (void ) { -struct btrfs_device *dev; -struct list_head *dev_list = &gfs_info->fs_devices->devices; +struct btrfs_fs_devices *fs_devices = gfs_info->fs_devices; +const u64 super_bytes =btrfs_super_total_bytes (gfs_info->super_copy);u64 total_bytes =0 ; -u64 super_bytes =btrfs_super_total_bytes (gfs_info->super_copy); -list_for_each_entry (dev, dev_list, dev_list) - total_bytes += dev->total_bytes; +while (fs_devices) { +struct btrfs_device *dev; + +list_for_each_entry (dev, &fs_devices->devices, dev_list) + total_bytes += dev->total_bytes; + fs_devices = fs_devices->seed; + }/* Important check, which can cause unmountable fs */ if (super_bytes < total_bytes) {
@@ -8579,7 +8583,9 @@ static bool is_super_size_valid(void)if (!IS_ALIGNED (super_bytes, gfs_info->sectorsize) || !IS_ALIGNED (total_bytes, gfs_info->sectorsize) || super_bytes != total_bytes) { -warning ("minor unaligned/mismatch device size detected" ); +warning ("minor unaligned/mismatch device size detected:" ); +warning (" super block total bytes=%llu found total bytes=%llu" , + super_bytes, total_bytes);warning ("recommended to use 'btrfs rescue fix-device-size' to fix it" ); }
Signature · 1 line
2.35.1
Re: [PATCH 1/2] btrfs-progs: check: fix wrong total bytes check for seed device
Quoted 31 lines · Qu Wenruo
[BUG]
The following script can lead to false positive from btrfs check:mkfs.btrfs -f $dev1
mount $dev1 $mnt
btrfstune -S1 $dev1
mount $dev1 $mnt
btrfs dev add -f $dev2 $mnt
umount $mnt# Now dev1 is seed, and dev2 is the rw fs.
btrfs check $dev2
...
[2/7] checking extents
WARNING: minor unaligned/mismatch device size detected
WARNING: recommended to use 'btrfs rescue fix-device-size' to fix it
...This false positive only happens on $dev2, $dev1 is completely fine.
[CAUSE]
The warning is fromis_super_size_valid(), in that function we verify the super block total bytes (@super_bytes) is correct against the total device bytes (@total_bytes).However the when calculating @total_bytes, we only use devices in current
fs_devices, which only contains RW devices.Thus all bytes from seed device are not taken into consideration, and trigger the false positive.
Quoted 8 lines
[FIX]
Fix it by also iterating seed devices.Since we're here, also output @total_bytes and @super_bytes when outputting the warning message, to allow end users have a better idea on what's going wrong.
Signed-off-by Qu Wenruo <[email protected]>
Quoted 42 lines
--- check/main.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)check/main.c +12
-6@@ -8550,13 +8550,17 @@ static int check_device_used(struct device_record *dev_rec,*/ static bool is_super_size_valid (void ) { -struct btrfs_device *dev; -struct list_head *dev_list = &gfs_info->fs_devices->devices; +struct btrfs_fs_devices *fs_devices = gfs_info->fs_devices; +const u64 super_bytes =btrfs_super_total_bytes (gfs_info->super_copy);u64 total_bytes =0 ; -u64 super_bytes =btrfs_super_total_bytes (gfs_info->super_copy); -list_for_each_entry (dev, dev_list, dev_list) - total_bytes += dev->total_bytes; +while (fs_devices) { +struct btrfs_device *dev; + +list_for_each_entry (dev, &fs_devices->devices, dev_list) + total_bytes += dev->total_bytes; + fs_devices = fs_devices->seed; + }/* Important check, which can cause unmountable fs */ if (super_bytes < total_bytes) {@@ -8579,7 +8583,9 @@ static bool is_super_size_valid(void)if (!IS_ALIGNED (super_bytes, gfs_info->sectorsize) || !IS_ALIGNED (total_bytes, gfs_info->sectorsize) || super_bytes != total_bytes) { -warning ("minor unaligned/mismatch device size detected" ); +warning ("minor unaligned/mismatch device size detected:" ); +warning (" super block total bytes=%llu found total bytes=%llu" , + super_bytes, total_bytes);warning ("recommended to use 'btrfs rescue fix-device-size' to fix it" ); }
Re: [PATCH 1/2] btrfs-progs: check: fix wrong total bytes check for seed device
In reply to Qu Wenruocheck/main.c static bool is_super_size_valid(void) · line 8588
Quoted 8 lines · Qu Wenruo
check/main.c +3
-1@@ -8579,7 +8583,9 @@ static bool is_super_size_valid(void)if (!IS_ALIGNED (super_bytes, gfs_info->sectorsize) || !IS_ALIGNED (total_bytes, gfs_info->sectorsize) || super_bytes != total_bytes) { -warning ("minor unaligned/mismatch device size detected" ); +warning ("minor unaligned/mismatch device size detected:" ); +warning (" super block total bytes=%llu found total bytes=%llu" , + super_bytes, total_bytes);
Minor thing, the message in warning() should be split the string and indent, not add another call, that's how it it is done now. If we decide to do some kind of auto split or reformat then we can do it inside the helper.
Quoted 5 lines
check/main.c +0
-0warning ("recommended to use 'btrfs rescue fix-device-size' to fix it" ); }Signature · 1 line
2.35.1
PATCH 2/2 btrfs-progs: fsck-tests: check warning for seed and sprouted filesystems
In reply to Qu WenruoPreviously we had a bug that btrfs check would report false warning for a sprouted filesystem.
So this patch will add a new test case to make sure neither seed nor and sprouted filesystem will cause such false warning.
--- .../fsck-tests/057-seed-false-alerts/test.sh | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100755 tests/fsck-tests/057-seed-false-alerts/test.sh
tests/fsck-tests/057-seed-false-alerts/test.sh new file +51 -0
@@ -0,0 +1,51 @@ +#!/bin/bash +# +# Make sure "btrfs check" won't report false alerts on sprouted filesystems +# + +source "$TEST_TOP/common" + +check_prereq btrfs +check_prereq mkfs.btrfs +check_prereq btrfstune +check_global_prereq losetup + +setup_loopdevs 2 +prepare_loopdevs +dev1=${loopdevs[1]} +dev2=${loopdevs[2]} +TEST_DEV=$dev1 + +setup_root_helper + +run_check $SUDO_HELPERS "$TOP/mkfs.btrfs" -f $dev1 +run_check $SUDO_HELPERS "$TOP/btrfstune" -S1 $dev1 +run_check_mount_test_dev +run_check $SUDO_HELPERS "$TOP/btrfs" device add -f $dev2 $TEST_MNT + +# Here we can not use umount helper, as it uses the seed device to do the +# umount. We need to manually unmout using the mount point +run_check $SUDO_HELPERS umount $TEST_MNT + +seed_output=$(_mktemp --tmpdir btrfs-progs-seed-check-stdout.XXXXXX) +sprouted_output=$(_mktemp --tmpdir btrfs-progs-sprouted-check-stdout.XXXXXX) + +# The false alerts are just warnings, so we need to save and filter +# the output +run_check_stdout "$TOP/btrfs" check "$dev1" >> "$seed_output" +run_check_stdout "$TOP/btrfs" check "$dev2" >> "$sprouted_output" + +# There should be no warning for both seed and sprouted fs +if grep -q "WARNING" "$seed_output"; then + cleanup_loopdevs + rm -f -- "$seed_output" "$sprouted_output" + _fail "false alerts detected for seed fs" +fi +if grep -q "WARNING" "$sprouted_output"; then + cleanup_loopdevs + rm -f -- "$seed_output" "$sprouted_output" + _fail "false alerts detected for sprouted fs" +fi + +cleanup_loopdevs +rm -f -- "$seed_output" "$sprouted_output"
Signature · 1 line
2.35.1
Re: [PATCH 2/2] btrfs-progs: fsck-tests: check warning for seed and sprouted filesystems
Quoted 29 lines · Qu Wenruo
Previously we had a bug that btrfs check would report false warning for a sprouted filesystem.
So this patch will add a new test case to make sure neither seed nor and sprouted filesystem will cause such false warning.
Signed-off-by Qu Wenruo <[email protected]>--- .../fsck-tests/057-seed-false-alerts/test.sh | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100755 tests/fsck-tests/057-seed-false-alerts/test.shtests/fsck-tests/057-seed-false-alerts/test.sh new file +11
-0@@ -0,0 +1,51 @@ +#!/bin/bash +# +# Make sure "btrfs check" won't report false alerts on sprouted filesystems +# + +source "$TEST_TOP/common" + +check_prereq btrfs +check_prereq mkfs.btrfs +check_prereq btrfstune +check_global_prereq losetup
tests/fsck-tests/057-seed-false-alerts/test.sh line 21
Quoted 10 lines
tests/fsck-tests/057-seed-false-alerts/test.sh new file +10
-0+ +setup_loopdevs 2 +prepare_loopdevs +dev1=${loopdevs[1]} +dev2=${loopdevs[2]} +TEST_DEV=$dev1 + +setup_root_helper + +run_check $SUDO_HELPERS "$TOP/mkfs.btrfs" -f $dev1
^^^^^^^^^^^^^
It's $SUDO_HELPER, otherwise it does not work, also please quote all shell variables, everywhere.
tests/fsck-tests/057-seed-false-alerts/test.sh line 30
Quoted 9 lines
tests/fsck-tests/057-seed-false-alerts/test.sh new file +9
-0+run_check $SUDO_HELPERS "$TOP/btrfstune" -S1 $dev1 +run_check_mount_test_dev +run_check $SUDO_HELPERS "$TOP/btrfs" device add -f $dev2 $TEST_MNT + +# Here we can not use umount helper, as it uses the seed device to do the +# umount. We need to manually unmout using the mount point +run_check $SUDO_HELPERS umount $TEST_MNT + +seed_output=$(_mktemp --tmpdir btrfs-progs-seed-check-stdout.XXXXXX)
This won't work as intended, there's a helper _mktemp_dir that creates a temporary directory, otherwise --tmpdir is interpreted as the direcotry name.
Re: [PATCH 2/2] btrfs-progs: fsck-tests: check warning for seed and sprouted filesystems
Quoted 45 lines · David Sterba
Quoted 29 lines · Qu Wenruo
Previously we had a bug that btrfs check would report false warning for a sprouted filesystem.
So this patch will add a new test case to make sure neither seed nor and sprouted filesystem will cause such false warning.
Signed-off-by Qu Wenruo <[email protected]>--- .../fsck-tests/057-seed-false-alerts/test.sh | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+)create mode 100755
tests/fsck-tests/057-seed-false-alerts/test.shtests/fsck-tests/057-seed-false-alerts/test.sh new file +11
-0@@ -0,0 +1,51 @@ +#!/bin/bash +# +# Make sure "btrfs check" won't report false alerts on sprouted filesystems +# + +source "$TEST_TOP/common" + +check_prereq btrfs +check_prereq mkfs.btrfs +check_prereq btrfstune +check_global_prereq losetuptests/fsck-tests/057-seed-false-alerts/test.sh line 21
Quoted 10 lines
tests/fsck-tests/057-seed-false-alerts/test.sh new file +10
-0+ +setup_loopdevs 2 +prepare_loopdevs +dev1=${loopdevs[1]} +dev2=${loopdevs[2]} +TEST_DEV=$dev1 + +setup_root_helper + +run_check $SUDO_HELPERS "$TOP/mkfs.btrfs" -f $dev1^^^^^^^^^^^^^It's $SUDO_HELPER, otherwise it does not work, also please quote all shell variables, everywhere.
Quoted 13 lines
tests/fsck-tests/057-seed-false-alerts/test.sh line 30
Quoted 9 lines
tests/fsck-tests/057-seed-false-alerts/test.sh new file +9
-0+run_check $SUDO_HELPERS "$TOP/btrfstune" -S1 $dev1 +run_check_mount_test_dev +run_check $SUDO_HELPERS "$TOP/btrfs" device add -f $dev2 $TEST_MNT + +# Here we can not use umount helper, as it uses the seed device to do the +# umount. We need to manually unmout using the mount point +run_check $SUDO_HELPERS umount $TEST_MNT + +seed_output=$(_mktemp --tmpdir btrfs-progs-seed-check-stdout.XXXXXX)This won't work as intended, there's a helper
_mktemp_dirthat creates a temporary directory, otherwise --tmpdir is interpreted as the direcotry name.
My bad, it should be mktemp without the "_" prefix, just like what we did in all the run* helpers.
And I'm not expected to create a directory, thus I don't need to _mktemp_dir.
Thanks,
Qu
Re: [PATCH 2/2] btrfs-progs: fsck-tests: check warning for seed and sprouted filesystems
Quoted 9 lines · Qu Wenruo
Quoted 3 lines
This won't work as intended, there's a helper
_mktemp_dirthat creates a temporary directory, otherwise --tmpdir is interpreted as the direcotry name.My bad, it should be mktemp without the "_" prefix, just like what we did in all the run* helpers.
And I'm not expected to create a directory, thus I don't need to
_mktemp_dir.
Right, I figured it later and dropped the _dir name and verified it works.
Re: [PATCH 2/2] btrfs-progs: fsck-tests: check warning for seed and sprouted filesystems
In reply to Qu WenruoQuoted 47 lines · Qu Wenruo
Quoted 44 lines · David Sterba
Quoted 31 lines · Qu Wenruo
Previously we had a bug that btrfs check would report false warning for a sprouted filesystem.
So this patch will add a new test case to make sure neither seed nor and sprouted filesystem will cause such false warning.
Signed-off-by Qu Wenruo <[email protected]>---
.../fsck-tests/057-seed-false-alerts/test.sh | 51 +++++++++++++++++++1 file changed, 51 insertions(+)create mode 100755
tests/fsck-tests/057-seed-false-alerts/test.shtests/fsck-tests/057-seed-false-alerts/test.sh new file +3
-0@@ -0,0 +1,51 @@ +#!/bin/bash +# +# Make sure "btrfs check" won't report false alerts on sproutedfilesystems
tests/fsck-tests/057-seed-false-alerts/test.sh new file +8
-0+# + +source "$TEST_TOP/common" + +check_prereq btrfs +check_prereq mkfs.btrfs +check_prereq btrfstune +check_global_prereq losetuptests/fsck-tests/057-seed-false-alerts/test.sh line 21
Quoted 10 lines
tests/fsck-tests/057-seed-false-alerts/test.sh new file +10
-0+ +setup_loopdevs 2 +prepare_loopdevs +dev1=${loopdevs[1]} +dev2=${loopdevs[2]} +TEST_DEV=$dev1 + +setup_root_helper + +run_check $SUDO_HELPERS "$TOP/mkfs.btrfs" -f $dev1^^^^^^^^^^^^^
Quoted 2 lines
Quoted 2 lines
It's $SUDO_HELPER, otherwise it does not work, also please quote all shell variables, everywhere.
Surprise surprise, it's from my vim's context based completion.
It turns out that fsck/056 has the same problem...
Thanks,
Qu
Quoted 24 lines
Quoted 15 lines
tests/fsck-tests/057-seed-false-alerts/test.sh line 30
Quoted 11 lines
tests/fsck-tests/057-seed-false-alerts/test.sh new file +5
-0+run_check $SUDO_HELPERS "$TOP/btrfstune" -S1 $dev1 +run_check_mount_test_dev +run_check $SUDO_HELPERS "$TOP/btrfs" device add -f $dev2 $TEST_MNT + +# Here we can not use umount helper, as it uses the seed device todo the
tests/fsck-tests/057-seed-false-alerts/test.sh new file +4
-0+# umount. We need to manually unmout using the mount point +run_check $SUDO_HELPERS umount $TEST_MNT + +seed_output=$(_mktemp --tmpdir btrfs-progs-seed-check-stdout.XXXXXX)This won't work as intended, there's a helper
_mktemp_dirthat creates a temporary directory, otherwise --tmpdir is interpreted as the direcotry name.My bad, it should be mktemp without the "_" prefix, just like what we did in all the run* helpers.
And I'm not expected to create a directory, thus I don't need to
_mktemp_dir.Thanks,
Qu
Re: [PATCH 2/2] btrfs-progs: fsck-tests: check warning for seed and sprouted filesystems
Quoted 10 lines · Qu Wenruo
Quoted 10 lines
Quoted 10 lines
tests/fsck-tests/057-seed-false-alerts/test.sh line 21
Quoted 8 lines
tests/fsck-tests/057-seed-false-alerts/test.sh new file +8
-0+prepare_loopdevs +dev1=${loopdevs[1]} +dev2=${loopdevs[2]} +TEST_DEV=$dev1 + +setup_root_helper + +run_check $SUDO_HELPERS "$TOP/mkfs.btrfs" -f $dev1^^^^^^^^^^^^^
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
It's $SUDO_HELPER, otherwise it does not work, also please quote all shell variables, everywhere.
Surprise surprise, it's from my vim's context based completion.
It turns out that fsck/056 has the same problem...
I see, fixed as well.
Quoted 6 lines · Qu Wenruo
During my attempt to utilize seed device at mkfs time, I found there there is a small bug that btrfs check always reports devices size related warning on the sprouted device.
It turns out we didn't iterate seed devices at all during btrfs check.
Quoted 7 lines
The first patch will fix the problem and enhance the warning output. The second one will add a regression test for it.
Qu Wenruo (2):
btrfs-progs: check: fix wrong total bytes check for seed device
btrfs-progs: fsck-tests: check warning for seed and sprouted
filesystems
With the test fixups added to devel, thanks.