Skip to contentSkip to conversation

fstests: btrfs validate sysfs fsid and usage

PATCH Superseded Anand Jain 2 participants 8 messages Oct 20, 2021 Btrfs file system

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Anand Jain

author
#0 lore raw

cover fstests: btrfs validate sysfs fsid and usage

Patch 1 adds a helper to check if the kernel is latest enough to provide the fsid from the sysfs.
Patch 2 validates the sysfs fsid against the fsid from the sb Patch 3 is a test case to make sure if the btrfs filesystem usage is successful on a sprout filesystem with a seed device missing.

Anand Jain (3):
common/btrfs: add _require_btrfs_sysfs_fsid helper btrfs/248: validate sysfs fsid
btrfs/249: test btrfs filesystem usage command on missing seed device

 common/btrfs        | 14 ++++++++++
 tests/btrfs/248     | 66 ++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/248.out |  2 ++
 tests/btrfs/249     | 67 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/249.out |  2 ++
 5 files changed, 151 insertions(+)
 create mode 100755 tests/btrfs/248
 create mode 100644 tests/btrfs/248.out
 create mode 100755 tests/btrfs/249
 create mode 100644 tests/btrfs/249.out
Signature · 1 line
2.27.0
author
#1 lore raw

PATCH 1/3 common/btrfs: add _require_btrfs_sysfs_fsid helper

It checks if the kernel has the following patch
btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
that added sysfs interface to get fsid.

---
 common/btrfs | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
common/btrfs +14 -0
index ac880bddf524..5d938c19b56a 100644
@@ -445,3 +445,17 @@ _scratch_btrfs_is_zoned()
 	[ `_zone_type ${SCRATCH_DEV}` != "none" ] && return 0
 	return 1
 }
+
+_require_btrfs_sysfs_fsid()
+{
+	local fsid
+
+	fsid=$($BTRFS_UTIL_PROG filesystem show $TEST_DIR |grep uuid: |\
+	       awk '{print $NF}')
+
+	# Check if the kernel has sysfs fsid support.
+	# Following kernel patch adds it:
+	#   btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
+	test -f /sys/fs/btrfs/$fsid/devinfo/1/fsid ||\
+		_notrun "Need btrfs sysfs fsid support"
+}
Signature · 1 line
2.27.0
author
#2 lore raw

PATCH 2/3 btrfs/248: validate sysfs fsid

In reply to Anand Jain

Validate if the sysfs fsid is the same as fsid as obtained from the superblock.

---
 tests/btrfs/248     | 66 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/248.out |  2 ++
 2 files changed, 68 insertions(+)
 create mode 100755 tests/btrfs/248
 create mode 100644 tests/btrfs/248.out
tests/btrfs/248 new file +66 -0
new file mode 100755
index 000000000000..856cd489d4e4
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Anand Jain.  All Rights Reserved.
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test 248
+#
+# Validate if the sysfs devinfo/<devid>/fsid behaves properly
+# Steps:
+#  Create a sprout filesystem (an rw device on top of a seed device)
+#  Read the seed and sprout devices fsid from the superblock
+#  Validate with the sysfs fsid
+
+. ./common/preamble
+_begin_fstest auto quick seed volume
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_test
+_require_scratch_dev_pool 2
+_require_btrfs_forget_or_module_loadable
+_require_command "$BTRFS_TUNE_PROG" btrfstune
+_require_btrfs_command inspect-internal dump-super
+_require_btrfs_sysfs_fsid
+
+_scratch_dev_pool_get 1
+# use the scratch device as a seed device
+seed_dev=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
+
+# use the spare device as a sprout device
+_spare_dev_get
+
+# create seed device
+_scratch_pool_mkfs >> $seqres.full 2>&1
+$BTRFS_TUNE_PROG -S 1 $seed_dev
+_scratch_mount >> $seqres.full 2>&1
+
+# create a sprout device
+$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
+_scratch_unmount
+
+# record the fsid of both seed and sprout devices
+seedfsid=$($BTRFS_UTIL_PROG inspect-internal dump-super $seed_dev |grep ^fsid |\
+	   $AWK_PROG '{print $2}')
+sproutfsid=$($BTRFS_UTIL_PROG inspect-internal dump-super $SPARE_DEV |\
+	     grep ^fsid |$AWK_PROG '{print $2}')
+
+# validate it with the fsid as shown in the sysfs
+_mount -o device=$seed_dev $SPARE_DEV $SCRATCH_MNT
+
+cat /sys/fs/btrfs/$sproutfsid/devinfo/2/fsid | grep -v $sproutfsid
+cat /sys/fs/btrfs/$sproutfsid/devinfo/1/fsid | grep -v $seedfsid
+
+_scratch_unmount
+_spare_dev_put
+_scratch_dev_pool_put
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
tests/btrfs/248.out new file +2 -0
new file mode 100644
index 000000000000..58af9173bea3
@@ -0,0 +1,2 @@
+QA output created by 248
+Silence is golden
Signature · 1 line
2.27.0
author
#3 lore raw

PATCH 3/3 btrfs/249: test btrfs filesystem usage command on missing seed device

In reply to Anand Jain

If there is a missing seed device in a sprout, the btrfs filesystem usage command fails, which is fixed by the following patches:

btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
btrfs-progs: read fsid from the sysfs in device_is_seed

Test if it works now after these patches in the kernel and in the btrfs-progs respectively.

---
 tests/btrfs/249     | 67 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/249.out |  2 ++
 2 files changed, 69 insertions(+)
 create mode 100755 tests/btrfs/249
 create mode 100644 tests/btrfs/249.out
tests/btrfs/249 new file +67 -0
new file mode 100755
index 000000000000..f8f2f07052c6
@@ -0,0 +1,67 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Anand Jain.  All Rights Reserved.
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test 249
+#
+# Validate if the command 'btrfs filesystem usage' works with missing seed
+# device
+# Steps:
+#  Create a degraded raid1 seed device
+#  Create a sprout filesystem (an rw device on top of a seed device)
+#  Dump 'btrfs filesystem usage', check it didn't fail
+#
+# Tests btrfs-progs bug fixed by the kernel patch and a btrfs-prog patch
+#   btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
+#   btrfs-progs: read fsid from the sysfs in device_is_seed
+
+. ./common/preamble
+_begin_fstest auto quick seed volume
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch_dev_pool 3
+_require_command "$WIPEFS_PROG" wipefs
+_require_btrfs_forget_or_module_loadable
+
+_scratch_dev_pool_get 2
+# use the scratch devices as seed devices
+seed_dev1=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
+seed_dev2=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $2 }')
+
+# use the spare device as a sprout device
+_spare_dev_get
+sprout_dev=$SPARE_DEV
+
+# create raid1 seed filesystem
+_scratch_pool_mkfs "-draid1 -mraid1" >> $seqres.full 2>&1
+$BTRFS_TUNE_PROG -S 1 $seed_dev1
+$WIPEFS_PROG -a $seed_dev1 >> $seqres.full 2>&1
+_btrfs_forget_or_module_reload
+_mount -o degraded $seed_dev2 $SCRATCH_MNT >> $seqres.full 2>&1
+
+# create a sprout device
+$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
+
+# dump filesystem usage if it fails error goes to the bad.out file
+$BTRFS_UTIL_PROG filesystem usage $SCRATCH_MNT >> $seqres.full
+# also check for the error code
+ret=$?
+if [ $ret != 0 ]; then
+	_fail "FAILED: btrfs filesystem usage, ret $ret"
+fi
+
+_scratch_unmount
+_spare_dev_put
+_scratch_dev_pool_put
+
+echo "Silence is golden"
+# success, all done
+status=0
+exit
tests/btrfs/249.out new file +2 -0
new file mode 100644
index 000000000000..b79a5dca8820
@@ -0,0 +1,2 @@
+QA output created by 249
+Silence is golden
Signature · 1 line
2.27.0
#4 lore raw

Re: [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device

Quoted 11 lines · Anand Jain

If there is a missing seed device in a sprout, the btrfs filesystem usage command fails, which is fixed by the following patches:

btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
btrfs-progs: read fsid from the sysfs in device_is_seed

Test if it works now after these patches in the kernel and in the btrfs-progs respectively.

Shouldn't this use

as well? I wish there was a way to detect that btrfs-progs had support for reading it but I suppose this is a good enough gate. Maybe add a

Quoted 34 lines
---
 tests/btrfs/249     | 67 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/249.out |  2 ++
 2 files changed, 69 insertions(+)
 create mode 100755 tests/btrfs/249
 create mode 100644 tests/btrfs/249.out
tests/btrfs/249 new file +21 -0
new file mode 100755
index 000000000000..f8f2f07052c6
@@ -0,0 +1,67 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Anand Jain.  All Rights Reserved.
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test 249
+#
+# Validate if the command 'btrfs filesystem usage' works with missing seed
+# device
+# Steps:
+#  Create a degraded raid1 seed device
+#  Create a sprout filesystem (an rw device on top of a seed device)
+#  Dump 'btrfs filesystem usage', check it didn't fail
+#
+# Tests btrfs-progs bug fixed by the kernel patch and a btrfs-prog patch
+#   btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
+#   btrfs-progs: read fsid from the sysfs in device_is_seed
+
+. ./common/preamble
+_begin_fstest auto quick seed volume
+

tests/btrfs/249 line 31

Quoted 10 lines
tests/btrfs/249 new file +10 -0
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch_dev_pool 3
+_require_command "$WIPEFS_PROG" wipefs
+_require_btrfs_forget_or_module_loadable

Need

_require_btrfs_sysfs_fsid

here I think.

Quoted 16 lines
tests/btrfs/249 new file +16 -0
+
+_scratch_dev_pool_get 2
+# use the scratch devices as seed devices
+seed_dev1=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
+seed_dev2=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $2 }')
+
+# use the spare device as a sprout device
+_spare_dev_get
+sprout_dev=$SPARE_DEV
+
+# create raid1 seed filesystem
+_scratch_pool_mkfs "-draid1 -mraid1" >> $seqres.full 2>&1
+$BTRFS_TUNE_PROG -S 1 $seed_dev1
+$WIPEFS_PROG -a $seed_dev1 >> $seqres.full 2>&1
+_btrfs_forget_or_module_reload
+_mount -o degraded $seed_dev2 $SCRATCH_MNT >> $seqres.full 2>&1

tests/btrfs/249 line 57

Quoted 10 lines
tests/btrfs/249 new file +10 -0
+
+# create a sprout device
+$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
+
+# dump filesystem usage if it fails error goes to the bad.out file
+$BTRFS_UTIL_PROG filesystem usage $SCRATCH_MNT >> $seqres.full
+# also check for the error code
+ret=$?
+if [ $ret != 0 ]; then
+	_fail "FAILED: btrfs filesystem usage, ret $ret"

Can you add "check your btrfs-progs version" here or something? In case I'm an idiot and forget to update btrfs-progs on the overnight xfstests boxes? Thanks,

Josef

author
#5 lore raw

Re: [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device

Quoted 10 lines · Josef Bacik
Quoted 9 lines · Anand Jain

If there is a missing seed device in a sprout, the btrfs filesystem usage command fails, which is fixed by the following patches:

btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
btrfs-progs: read fsid from the sysfs in device_is_seed

Test if it works now after these patches in the kernel and in the btrfs-progs respectively.

Quoted 8 lines
Quoted 2 lines

Shouldn't this use

as well? I wish there was a way to detect that btrfs-progs had support for reading it but I suppose this is a good enough gate. Maybe add a

_require_btrfs_sysfs_fsid ?

The problem is about the nonexistence of the sysfs interface to read fsid. Adding it will fail to reproduce the problem.

So if there is no sysfs interface, then btrfs-progs will fail back to the read-sb method, which shall expose it to fail.

Quoted 49 lines
Quoted 36 lines

---
  tests/btrfs/249     | 67 +++++++++++++++++++++++++++++++++++++++++++++
  tests/btrfs/249.out |  2 ++
  2 files changed, 69 insertions(+)

create mode 100755 tests/btrfs/249
create mode 100644 tests/btrfs/249.out

tests/btrfs/249 new file +21 -0
new file mode 100755
index 000000000000..f8f2f07052c6
@@ -0,0 +1,67 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Anand Jain.  All Rights Reserved.
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test 249
+#
+# Validate if the command 'btrfs filesystem usage' works with missing seed
+# device
+# Steps:
+#  Create a degraded raid1 seed device
+#  Create a sprout filesystem (an rw device on top of a seed device)
+#  Dump 'btrfs filesystem usage', check it didn't fail
+#
+# Tests btrfs-progs bug fixed by the kernel patch and a btrfs-prog patch
+#   btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
+#   btrfs-progs: read fsid from the sysfs in device_is_seed
+
+. ./common/preamble
+_begin_fstest auto quick seed volume
+

tests/btrfs/249 line 31

Quoted 10 lines
tests/btrfs/249 new file +10 -0
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_require_scratch_dev_pool 3
+_require_command "$WIPEFS_PROG" wipefs
+_require_btrfs_forget_or_module_loadable

Need

Quoted 3 lines

_require_btrfs_sysfs_fsid

here I think.

As above.

Quoted 22 lines
Quoted 22 lines
tests/btrfs/249 new file +21 -0
+
+_scratch_dev_pool_get 2
+# use the scratch devices as seed devices
+seed_dev1=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $1 }')
+seed_dev2=$(echo $SCRATCH_DEV_POOL | $AWK_PROG '{ print $2 }')
+
+# use the spare device as a sprout device
+_spare_dev_get
+sprout_dev=$SPARE_DEV
+
+# create raid1 seed filesystem
+_scratch_pool_mkfs "-draid1 -mraid1" >> $seqres.full 2>&1
+$BTRFS_TUNE_PROG -S 1 $seed_dev1
+$WIPEFS_PROG -a $seed_dev1 >> $seqres.full 2>&1
+_btrfs_forget_or_module_reload
+_mount -o degraded $seed_dev2 $SCRATCH_MNT >> $seqres.full 2>&1
+
+# create a sprout device
+$BTRFS_UTIL_PROG device add -f $SPARE_DEV $SCRATCH_MNT >> $seqres.full 2>&1
+
+# dump filesystem usage if it fails error goes to the bad.out file
Quoted 9 lines

tests/btrfs/249 line 57

Quoted 5 lines
tests/btrfs/249 new file +5 -0
+$BTRFS_UTIL_PROG filesystem usage $SCRATCH_MNT >> $seqres.full
+# also check for the error code
+ret=$?
+if [ $ret != 0 ]; then
+	_fail "FAILED: btrfs filesystem usage, ret $ret"

Can you add "check your btrfs-progs version" here or something? In case I'm an idiot and forget to update btrfs-progs on the overnight xfstests boxes? Thanks,

Sure.

Thanks, Anand

Quoted 2 lines

Josef

#6 lore raw

Re: [PATCH 3/3] btrfs/249: test btrfs filesystem usage command on missing seed device

Quoted 22 lines · Anand Jain
Quoted 10 lines · Josef Bacik
Quoted 9 lines · Anand Jain

If there is a missing seed device in a sprout, the btrfs filesystem usage command fails, which is fixed by the following patches:

btrfs: sysfs add devinfo/fsid to retrieve fsid from the device
btrfs-progs: read fsid from the sysfs in device_is_seed

Test if it works now after these patches in the kernel and in the btrfs-progs respectively.

Quoted 8 lines
Quoted 2 lines

Shouldn't this use

as well? I wish there was a way to detect that btrfs-progs had support for reading it but I suppose this is a good enough gate. Maybe add a

Quoted 8 lines

_require_btrfs_sysfs_fsid ?

The problem is about the nonexistence of the sysfs interface to read fsid. Adding it will fail to reproduce the problem.

So if there is no sysfs interface, then btrfs-progs will fail back to the read-sb method, which shall expose it to fail.

Sigh sorry, I typed this out and then decided I wanted to comment below, and forgot to delete this bit.

Yeah sorry I'm thinking about it in terms of the CI testing for us, we'll start failing this test once its merged without the fix merged, but I guess that's kinda what we want. Let's just leave this as it is. Thanks,

Josef

Reviewed-by on the v1 series
#7 lore raw
In reply to Anand Jain
Quoted 11 lines · Anand Jain

Patch 1 adds a helper to check if the kernel is latest enough to provide the fsid from the sysfs.
Patch 2 validates the sysfs fsid against the fsid from the sb Patch 3 is a test case to make sure if the btrfs filesystem usage is successful on a sprout filesystem with a seed device missing.

Anand Jain (3):
common/btrfs: add _require_btrfs_sysfs_fsid helper btrfs/248: validate sysfs fsid
btrfs/249: test btrfs filesystem usage command on missing seed device

You can add

Thanks,

Josef