xfstests: test btrfs replace on RAID 5/6

PATCH In review Omar Sandoval 5 participants 15 messages Jul 23, 2015 – Jul 28, 2015 Btrfs file system

Share & tools

Omar Sandoval

author

2015-07-23 20:52 UTC #0 lore raw

cover xfstests: test btrfs replace on RAID 5/6

Btrfs has supported replace on RAID 5 and 6 since 3.19, but xfstests hasn't been updated to reflect that. Patches 1 and 2 in this series fix that.

Additionally, I'm including Wang Yanfeng's test for my patch series "Btrfs: RAID 5/6 missing device scrub+replace", updated to use the infrastructure added by patch 2.

Thanks!

Omar Sandoval (2):
btrfs/011: test replace on RAID 5/6 now that it's supported
btrfs: add replace missing and replace RAID 5/6 to profile configs

Wang Yanfeng (1):
btrfs: add a test of replace missing dev in diff raid

 common/rc           | 96 +++++++++++++++++++++++++++--------------------------
 tests/btrfs/011     |  4 +--
 tests/btrfs/095     | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/095.out |  2 ++
 tests/btrfs/group   |  1 +
 5 files changed, 147 insertions(+), 49 deletions(-)
 create mode 100755 tests/btrfs/095
 create mode 100644 tests/btrfs/095.out

Signature · 1 line

2.4.6

Omar Sandoval

author

2015-07-23 20:52 UTC #1 lore raw

PATCH 1/3 btrfs/011: test replace on RAID 5/6 now that it's supported

btrfs replace has been supported on RAID 5/6 since Linux 3.19.

Signed-off-by Omar Sandoval <[email protected]>

---
 tests/btrfs/011 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

tests/btrfs/011 +2 -2

index f4f2fbed68d8..c7d35fa46062 100755
@@ -282,8 +282,8 @@ workout "-m dup -d single" 1 cancel quick
 workout "-m dup -d dup -M" 1 no quick
 workout "-m raid0 -d raid0" 2 no quick
 workout "-m raid1 -d raid1" 2 no thorough
-#workout "-m raid5 -d raid5" 2 no quick # not yet supported for btrfs replace
-#workout "-m raid6 -d raid6" 3 no quick # not yet supported for btrfs replace
+workout "-m raid5 -d raid5" 2 no quick
+workout "-m raid6 -d raid6" 3 no quick
 workout "-m raid10 -d raid10" 4 no quick
 
 echo "*** done"

Signature · 1 line

2.4.6

Eryu Guan

Reviewed-by on v1 1/3

2015-07-24 03:01 UTC #2 lore raw

Re: [PATCH 1/3] btrfs/011: test replace on RAID 5/6 now that it's supported

Quoted 3 lines · Omar Sandoval

btrfs replace has been supported on RAID 5/6 since Linux 3.19.
Signed-off-by Omar Sandoval <[email protected]>

Reviewed-by Eryu Guan <[email protected]>

Quoted 26 lines

---
 tests/btrfs/011 | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

tests/btrfs/011 +2 -2

index f4f2fbed68d8..c7d35fa46062 100755
@@ -282,8 +282,8 @@ workout "-m dup -d single" 1 cancel quick
 workout "-m dup -d dup -M" 1 no quick
 workout "-m raid0 -d raid0" 2 no quick
 workout "-m raid1 -d raid1" 2 no thorough
-#workout "-m raid5 -d raid5" 2 no quick # not yet supported for btrfs replace
-#workout "-m raid6 -d raid6" 3 no quick # not yet supported for btrfs replace
+workout "-m raid5 -d raid5" 2 no quick
+workout "-m raid6 -d raid6" 3 no quick
 workout "-m raid10 -d raid10" 4 no quick
 
 echo "*** done"

Signature · 6 lines

2.4.6

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Sterba

maintainer Reviewed-by on v1 1/3

2015-07-24 12:10 UTC #3 lore raw

Re: [PATCH 1/3] btrfs/011: test replace on RAID 5/6 now that it's supported

In reply to Omar Sandoval

Quoted 3 lines · Omar Sandoval

btrfs replace has been supported on RAID 5/6 since Linux 3.19.
Signed-off-by Omar Sandoval <[email protected]>

Reviewed-by David Sterba <[email protected]>

Omar Sandoval

author

2015-07-23 20:52 UTC #4 lore raw

PATCH 2/3 btrfs: add replace missing and replace RAID 5/6 to profile configs

In reply to Omar Sandoval

Replacing and scrubbing RAID 5/6 is now supported on Btrfs. Enable it in _btrfs_get_profile_configs while making it more generic to also support replace missing.

Signed-off-by Omar Sandoval <[email protected]>

---
 common/rc | 96 ++++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 49 insertions(+), 47 deletions(-)

common/rc +49 ~~-47~~

index 610045eab304..3e6fdb6ebcfa 100644
@@ -2748,60 +2748,62 @@ _btrfs_get_profile_configs()
 		return
 	fi
 
-	# no user specified btrfs profile configs, export the default configs
 	if [ -z "$BTRFS_PROFILE_CONFIGS" ]; then
-		# default configs
-		_btrfs_profile_configs=(
-			"-m single -d single"
-			"-m dup -d single"
-			"-m raid0 -d raid0"
-			"-m raid1 -d raid0"
-			"-m raid1 -d raid1"
-			"-m raid10 -d raid10"
-			"-m raid5 -d raid5"
-			"-m raid6 -d raid6"
+		# Default configurations to test.
+		local configs=(
+			"single:single"
+			"dup:single"
+			"raid0:raid0"
+			"raid1:raid0"
+			"raid1:raid1"
+			"raid10:raid10"
+			"raid5:raid5"
+			"raid6:raid6"
 		)
+	else
+		# User-provided configurations.
+		local configs=(${BTRFS_PROFILE_CONFIGS[@]})
+	fi
 
-		# remove dup/raid5/raid6 profiles if we're doing device replace
-		# dup profile indicates only one device being used (SCRATCH_DEV),
-		# but we don't want to replace SCRATCH_DEV, which will be used in
-		# _scratch_mount/_check_scratch_fs etc.
-		# and raid5/raid6 doesn't support replace yet
+	_btrfs_profile_configs=()
+	for cfg in "${configs[@]}"; do
+		local supported=true
+		local profiles=(${cfg/:/ })
 		if [ "$1" == "replace" ]; then
-			_btrfs_profile_configs=(
-				"-m single -d single"
-				"-m raid0 -d raid0"
-				"-m raid1 -d raid0"
-				"-m raid1 -d raid1"
-				"-m raid10 -d raid10"
-				# add these back when raid5/6 is working with replace
-				#"-m raid5 -d raid5"
-				#"-m raid6 -d raid6"
+			# We can't do replace with these profiles because they
+			# imply only one device ($SCRATCH_DEV), and we need to
+			# keep $SCRATCH_DEV around for _scratch_mount
+			# and _check_scratch_fs.
+			local unsupported=(
+				"single"
+				"dup"
 			)
+		elif [ "$1" == "replace-missing" ]; then
+			# We can't replace missing devices with these profiles
+			# because there isn't enough redundancy.
+			local unsupported=(
+				"single"
+				"dup"
+				"raid0"
+			)
+		else
+			local unsupported=()
 		fi
-		export _btrfs_profile_configs
-		return
-	fi
-
-	# parse user specified btrfs profile configs
-	local i=0
-	local cfg=""
-	for cfg in $BTRFS_PROFILE_CONFIGS; do
-		# turn "metadata:data" format to "-m metadata -d data"
-		# and assign it to _btrfs_profile_configs array
-		cfg=`echo "$cfg" | sed -e 's/^/-m /' -e 's/:/ -d /'`
-		_btrfs_profile_configs[$i]="$cfg"
-		let i=i+1
-	done
-
-	if [ "$1" == "replace" ]; then
-		if echo ${_btrfs_profile_configs[*]} | grep -q raid[56]; then
-			_notrun "RAID5/6 doesn't support btrfs device replace yet"
-		fi
-		if echo ${_btrfs_profile_configs[*]} | grep -q dup; then
-			_notrun "Do not set dup profile in btrfs device replace test"
+		for unsupp in "${unsupported[@]}"; do
+			if [ "${profiles[0]}" == "$unsupp" -o "${profiles[1]}" == "$unsupp" ]; then
+			     if [ -z "$BTRFS_PROFILE_CONFIGS" ]; then
+				     # For the default config, just omit it.
+				     supported=false
+			     else
+				     # For user-provided config, don't run the test.
+				     _notrun "Profile $unsupp not supported for $1"
+			     fi
+			fi
+		done
+		if "$supported"; then
+			_btrfs_profile_configs+=("-m ${profiles[0]} -d ${profiles[1]}")
 		fi
-	fi
+	done
 	export _btrfs_profile_configs
 }

Signature · 1 line

2.4.6

Eryu Guan

Reviewed-by on v1 2/3

2015-07-24 03:03 UTC #5 lore raw

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

Quoted 5 lines · Omar Sandoval

Replacing and scrubbing RAID 5/6 is now supported on Btrfs. Enable it in _btrfs_get_profile_configs while making it more generic to also support replace missing.
Signed-off-by Omar Sandoval <[email protected]>

Looks great! Tested with default configs and user-defined configs.

Reviewed-by Eryu Guan <[email protected]>

Quoted 125 lines

---
 common/rc | 96 ++++++++++++++++++++++++++++++++-------------------------------
 1 file changed, 49 insertions(+), 47 deletions(-)

common/rc +49 ~~-47~~

index 610045eab304..3e6fdb6ebcfa 100644
@@ -2748,60 +2748,62 @@ _btrfs_get_profile_configs()
 		return
 	fi
 
-	# no user specified btrfs profile configs, export the default configs
 	if [ -z "$BTRFS_PROFILE_CONFIGS" ]; then
-		# default configs
-		_btrfs_profile_configs=(
-			"-m single -d single"
-			"-m dup -d single"
-			"-m raid0 -d raid0"
-			"-m raid1 -d raid0"
-			"-m raid1 -d raid1"
-			"-m raid10 -d raid10"
-			"-m raid5 -d raid5"
-			"-m raid6 -d raid6"
+		# Default configurations to test.
+		local configs=(
+			"single:single"
+			"dup:single"
+			"raid0:raid0"
+			"raid1:raid0"
+			"raid1:raid1"
+			"raid10:raid10"
+			"raid5:raid5"
+			"raid6:raid6"
 		)
+	else
+		# User-provided configurations.
+		local configs=(${BTRFS_PROFILE_CONFIGS[@]})
+	fi
 
-		# remove dup/raid5/raid6 profiles if we're doing device replace
-		# dup profile indicates only one device being used (SCRATCH_DEV),
-		# but we don't want to replace SCRATCH_DEV, which will be used in
-		# _scratch_mount/_check_scratch_fs etc.
-		# and raid5/raid6 doesn't support replace yet
+	_btrfs_profile_configs=()
+	for cfg in "${configs[@]}"; do
+		local supported=true
+		local profiles=(${cfg/:/ })
 		if [ "$1" == "replace" ]; then
-			_btrfs_profile_configs=(
-				"-m single -d single"
-				"-m raid0 -d raid0"
-				"-m raid1 -d raid0"
-				"-m raid1 -d raid1"
-				"-m raid10 -d raid10"
-				# add these back when raid5/6 is working with replace
-				#"-m raid5 -d raid5"
-				#"-m raid6 -d raid6"
+			# We can't do replace with these profiles because they
+			# imply only one device ($SCRATCH_DEV), and we need to
+			# keep $SCRATCH_DEV around for _scratch_mount
+			# and _check_scratch_fs.
+			local unsupported=(
+				"single"
+				"dup"
 			)
+		elif [ "$1" == "replace-missing" ]; then
+			# We can't replace missing devices with these profiles
+			# because there isn't enough redundancy.
+			local unsupported=(
+				"single"
+				"dup"
+				"raid0"
+			)
+		else
+			local unsupported=()
 		fi
-		export _btrfs_profile_configs
-		return
-	fi
-
-	# parse user specified btrfs profile configs
-	local i=0
-	local cfg=""
-	for cfg in $BTRFS_PROFILE_CONFIGS; do
-		# turn "metadata:data" format to "-m metadata -d data"
-		# and assign it to _btrfs_profile_configs array
-		cfg=`echo "$cfg" | sed -e 's/^/-m /' -e 's/:/ -d /'`
-		_btrfs_profile_configs[$i]="$cfg"
-		let i=i+1
-	done
-
-	if [ "$1" == "replace" ]; then
-		if echo ${_btrfs_profile_configs[*]} | grep -q raid[56]; then
-			_notrun "RAID5/6 doesn't support btrfs device replace yet"
-		fi
-		if echo ${_btrfs_profile_configs[*]} | grep -q dup; then
-			_notrun "Do not set dup profile in btrfs device replace test"
+		for unsupp in "${unsupported[@]}"; do
+			if [ "${profiles[0]}" == "$unsupp" -o "${profiles[1]}" == "$unsupp" ]; then
+			     if [ -z "$BTRFS_PROFILE_CONFIGS" ]; then
+				     # For the default config, just omit it.
+				     supported=false
+			     else
+				     # For user-provided config, don't run the test.
+				     _notrun "Profile $unsupp not supported for $1"
+			     fi
+			fi
+		done
+		if "$supported"; then
+			_btrfs_profile_configs+=("-m ${profiles[0]} -d ${profiles[1]}")
 		fi
-	fi
+	done
 	export _btrfs_profile_configs
 }

Signature · 6 lines

2.4.6

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Sterba

maintainer

2015-07-24 12:09 UTC #6 lore raw

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

In reply to Omar Sandoval

common/rc _btrfs_get_profile_configs() · line 2779

Quoted 7 lines · Omar Sandoval

common/rc +7 -0

+			# We can't do replace with these profiles because they
+			# imply only one device ($SCRATCH_DEV), and we need to
+			# keep $SCRATCH_DEV around for _scratch_mount
+			# and _check_scratch_fs.
+			local unsupported=(
+				"single"
+				"dup"

DUP does imply single device, but why does 'single' ?

Omar Sandoval

author

2015-07-24 17:51 UTC #7 lore raw

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

Quoted 10 lines · David Sterba

common/rc _btrfs_get_profile_configs() · line 2779
Quoted 7 lines · Omar Sandoval
common/rc +7 -0
+			# We can't do replace with these profiles because they
+			# imply only one device ($SCRATCH_DEV), and we need to
+			# keep $SCRATCH_DEV around for _scratch_mount
+			# and _check_scratch_fs.
+			local unsupported=(
+				"single"
+				"dup"
DUP does imply single device, but why does 'single' ?

It does not, I apparently forgot that you could use single to concatenate multiple devices. I'll fix that in v2.

Thanks for reviewing!

Signature · 1 line

Omar

Brendan Hide

2015-07-27 22:39 UTC #8 lore raw

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 10 lines · Omar Sandoval

Quoted 8 lines · David Sterba
Quoted 5 lines · Omar Sandoval
(quoted diff) +1 -0
+			# We can't do replace with these profiles because they +
# imply only one device ($SCRATCH_DEV), and we need to +			#
keep $SCRATCH_DEV around for _scratch_mount +			# and
_check_scratch_fs. +			local unsupported=( +				"single" +
"dup"
DUP does imply single device, but why does 'single' ?

Quoted 5 lines

It does not, I apparently forgot that you could use single to concatenate multiple devices. I'll fix that in v2.
Thanks for reviewing!

Late to the party. DUP *implies* single device but there are cases where dup is used on a multi-device fs. Even if the use-cases aren't good or intended to be long-term, they are still valid, right?

- --
__________
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBAgAGBQJVtq9AAAoJEE+uni74c4qN5eYIAJAGznsi3RD1tchbSLwhMXJk bJJ4ORB9taLXHykSfYTsHIaUoVpcVR6tT/I1jz5070DY3mKkQ16a8nwtSxPba4Lv QiS8YRegFiHMYzZbH1T7Tnm6R9g/RZsaU7GS3JhP9HUYG7hIWGRRuoiOjYn/hoLw uMXuIFOkPKGYDgyAhDIp3KDYlBjMHT6Oun7CcpvTjXiOnzJFFp3MSt3b6mmmdMVV YKWpWyKVh7qlENEoqKb4exqr1WGYKU+kBLXRs4wdm3xb66EcWYs0Er1u6v+K1trx nryFrfUxYtMJsSuR9ZJm88DOsXKAuX1LEdRKVOlq7krsIK8HlTizccMXAl10gKk= =ndkL
-----END PGP SIGNATURE-----

Hugo Mills

2015-07-27 22:44 UTC #9 lore raw

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

Quoted 14 lines · Brendan Hide

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 10 lines · Omar Sandoval
Quoted 8 lines · David Sterba
Quoted 5 lines · Omar Sandoval
(quoted diff) +1 -0
+			# We can't do replace with these profiles because they +
# imply only one device ($SCRATCH_DEV), and we need to +			#
keep $SCRATCH_DEV around for _scratch_mount +			# and
_check_scratch_fs. +			local unsupported=( +				"single" +
"dup"
DUP does imply single device, but why does 'single' ?

Quoted 8 lines

Quoted 5 lines
It does not, I apparently forgot that you could use single to concatenate multiple devices. I'll fix that in v2.
Thanks for reviewing!
Late to the party. DUP *implies* single device but there are cases where dup is used on a multi-device fs. Even if the use-cases aren't good or intended to be long-term, they are still valid, right?

Yes, but bear in mind that any new (presumably metadata) chunks
allocated on the multi-device FS will end up as RAID-1, because of the automatic upgrade from DUP to RAID-1. Any balance will also do the upgrade.

Hugo.

PS. Can we get rid of that upgrade? It's a pain the arse in so many ways.

Quoted 16 lines

- --
__________
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBAgAGBQJVtq9AAAoJEE+uni74c4qN5eYIAJAGznsi3RD1tchbSLwhMXJk bJJ4ORB9taLXHykSfYTsHIaUoVpcVR6tT/I1jz5070DY3mKkQ16a8nwtSxPba4Lv QiS8YRegFiHMYzZbH1T7Tnm6R9g/RZsaU7GS3JhP9HUYG7hIWGRRuoiOjYn/hoLw uMXuIFOkPKGYDgyAhDIp3KDYlBjMHT6Oun7CcpvTjXiOnzJFFp3MSt3b6mmmdMVV YKWpWyKVh7qlENEoqKb4exqr1WGYKU+kBLXRs4wdm3xb66EcWYs0Er1u6v+K1trx nryFrfUxYtMJsSuR9ZJm88DOsXKAuX1LEdRKVOlq7krsIK8HlTizccMXAl10gKk= =ndkL
-----END PGP SIGNATURE-----

Signature · 4 lines

Hugo Mills             | I'm on a 30-day diet. So far I've lost 18 days.
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

David Sterba

maintainer

2015-07-28 16:52 UTC #10 lore raw

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

In reply to Brendan Hide

Quoted 8 lines · Brendan Hide

Quoted 5 lines
It does not, I apparently forgot that you could use single to concatenate multiple devices. I'll fix that in v2.
Thanks for reviewing!
Late to the party. DUP *implies* single device but there are cases where dup is used on a multi-device fs. Even if the use-cases aren't good or intended to be long-term, they are still valid, right?

You're right, DUP is reported by 'fi df' after 2nd device is added and this state (even if it's temporary) has to be taken into account.

Omar Sandoval

author

2015-07-28 17:42 UTC #11 lore raw

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

Quoted 4 lines · David Sterba

Quoted 3 lines · Brendan Hide
Quoted 3 lines
It does not, I apparently forgot that you could use single to concatenate multiple devices. I'll fix that in v2.

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
Thanks for reviewing!
Late to the party. DUP *implies* single device but there are cases where dup is used on a multi-device fs. Even if the use-cases aren't good or intended to be long-term, they are still valid, right?
You're right, DUP is reported by 'fi df' after 2nd device is added and this state (even if it's temporary) has to be taken into account.

I wouldn't read too much into this, _btrfs_get_profile_configs is just for tests that mkfs a bunch of new filesystems for testing, and in that case, DUP is invalid for multiple devices. It's not meant to be an authoritative source of truth regarding what metadata/data profiles could be legal.

Thanks,

Signature · 1 line

Omar

Omar Sandoval

author

2015-07-23 20:52 UTC #12 lore raw

PATCH 3/3 btrfs: add a test of replace missing dev in diff raid

In reply to Omar Sandoval

From: Wang Yanfeng <[email protected]>

Test of missing device replace in different raid modes. This test requires SCRATCH_DEV_POOL contain 5 same size devices.

This issue has been fixed by Omar's patch:
Btrfs: RAID 5/6 missing device scrub+replace

Signed-off-by Wang Yanfeng <[email protected]>

Signed-off-by Omar Sandoval <[email protected]>

---
 tests/btrfs/095     | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/095.out |  2 ++
 tests/btrfs/group   |  1 +
 3 files changed, 96 insertions(+)
 create mode 100755 tests/btrfs/095
 create mode 100644 tests/btrfs/095.out

tests/btrfs/095 new file +93 -0

new file mode 100755
index 000000000000..2dfb919fe4f6
@@ -0,0 +1,93 @@
+#! /bin/bash
+# FS QA Test No. btrfs/095
+#
+# Test of missing device replace in different raid mode
+#
+# Be sure $SCRATCH_DEV_POOL including 5 devices. And all
+# devices in pool must be in the same size.
+#
+# To check the fs after replacing a dev, a scrub run is performed.
+#
+# This issue has been fixed by Omar Sandoval's patch:
+# 	Btrfs: RAID 5/6 missing device scrub+replace
+#-----------------------------------------------------------------------
+# Copyright (c) 2015 Fujitsu.  All Rights Reserved.
+# Author: Wang Yanfeng <[email protected]>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+status=1
+trap "exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool 5
+_btrfs_get_profile_configs replace-missing
+
+rm -f $seqres.full
+
+REPLACE_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $5}'`
+export SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | awk '{print $1,$2,$3,$4}'`
+
+run_test () {
+	local mkfs_opts=$1
+	_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 || _fail "mkfs failed"
+	_scratch_mount
+
+	PRUNE_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+	PRUNE_DEV_ID=`$BTRFS_UTIL_PROG fi show $SCRATCH_MNT | grep $PRUNE_DEV | awk '{print $2}'`
+
+	# dd some data
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file1 bs=1M count=1 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file2 bs=1M count=2 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file3 bs=1M count=4 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
+
+	# prune the device PRUNE_DEV && remount by degraded mode
+	umount $SCRATCH_MNT
+	dd if=/dev/zero of=$PRUNE_DEV bs=1M count=1 >>$seqres.full 2>&1 \
+		|| _fail "dd failed"
+	mount -o degraded $SCRATCH_DEV $SCRATCH_MNT
+
+	# replace the missing dev $PRUNE_DEV with $REPLACE_DEV and scrub it
+	$BTRFS_UTIL_PROG replace start -B -r $PRUNE_DEV_ID $REPLACE_DEV \
+		$SCRATCH_MNT -f >>$seqres.full 2>&1 || _fail "replace failed"
+	$BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >>$seqres.full 2>&1 \
+		|| _fail "scrub failed"
+
+	umount $SCRATCH_MNT
+}
+
+echo "Silence is golden"
+for t in "${_btrfs_profile_configs[@]}"; do
+	run_test "$t"
+done
+
+status=0
+exit

tests/btrfs/095.out new file +2 -0

new file mode 100644
index 000000000000..80ad3b9c2a7b
@@ -0,0 +1,2 @@
+QA output created by 095
+Silence is golden

tests/btrfs/group +1 -0

index ffe18bff0d21..f20a191c24be 100644
@@ -96,3 +96,4 @@
 092 auto quick send
 093 auto quick clone
 094 auto quick send
+095 auto

Signature · 1 line

2.4.6

Eryu Guan

2015-07-24 03:28 UTC #13 lore raw

Re: [PATCH 3/3] btrfs: add a test of replace missing dev in diff raid

Quoted 62 lines · Omar Sandoval

From: Wang Yanfeng <[email protected]>

Test of missing device replace in different raid modes. This test requires SCRATCH_DEV_POOL contain 5 same size devices.

This issue has been fixed by Omar's patch:
Btrfs: RAID 5/6 missing device scrub+replace

Signed-off-by Wang Yanfeng <[email protected]>

Signed-off-by Omar Sandoval <[email protected]>

---
 tests/btrfs/095     | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/095.out |  2 ++
 tests/btrfs/group   |  1 +
 3 files changed, 96 insertions(+)
 create mode 100755 tests/btrfs/095
 create mode 100644 tests/btrfs/095.out

tests/btrfs/095 new file +38 -0

new file mode 100755
index 000000000000..2dfb919fe4f6
@@ -0,0 +1,93 @@
+#! /bin/bash
+# FS QA Test No. btrfs/095
+#
+# Test of missing device replace in different raid mode
+#
+# Be sure $SCRATCH_DEV_POOL including 5 devices. And all
+# devices in pool must be in the same size.
+#
+# To check the fs after replacing a dev, a scrub run is performed.
+#
+# This issue has been fixed by Omar Sandoval's patch:
+# 	Btrfs: RAID 5/6 missing device scrub+replace
+#-----------------------------------------------------------------------
+# Copyright (c) 2015 Fujitsu.  All Rights Reserved.
+# Author: Wang Yanfeng <[email protected]>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+status=1
+trap "exit \$status" 0 1 2 3 15
+

tests/btrfs/095 line 48

Quoted 10 lines

tests/btrfs/095 new file +10 -0

+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool 5

Need a call of _require_scratch_dev_pool_equal_size here

tests/btrfs/095 line 57

Quoted 9 lines

tests/btrfs/095 new file +9 -0

+_btrfs_get_profile_configs replace-missing
+
+rm -f $seqres.full
+
+REPLACE_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $5}'`
+export SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | awk '{print $1,$2,$3,$4}'`
+
+run_test () {
+	local mkfs_opts=$1

echo the $mkfs_opts to $seqres.full is useful for debug purpose, so if some profile config fails we know what config is failing.

echo "Test $mkfs_opts" >>$seqres.full

tests/btrfs/095 line 58

Quoted 1 line

tests/btrfs/095 new file +1 -0

+	_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 || _fail "mkfs failed"

Continue to test next profile config if mkfs failed, _fail breaks out the whole test.

tests/btrfs/095 line 70

Quoted 12 lines

tests/btrfs/095 new file +12 -0

+	_scratch_mount
+
+	PRUNE_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+	PRUNE_DEV_ID=`$BTRFS_UTIL_PROG fi show $SCRATCH_MNT | grep $PRUNE_DEV | awk '{print $2}'`
+
+	# dd some data
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file1 bs=1M count=1 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file2 bs=1M count=2 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file3 bs=1M count=4 \
+		>>$seqres.full 2>&1 || _fail "dd failed"

I don't think the _fail here is necessary either.

tests/btrfs/095 line 73

Quoted 3 lines

tests/btrfs/095 new file +3 -0

+
+	# prune the device PRUNE_DEV && remount by degraded mode
+	umount $SCRATCH_MNT

_scratch_unmount

tests/btrfs/095 line 75

Quoted 2 lines

tests/btrfs/095 new file +2 -0

+	dd if=/dev/zero of=$PRUNE_DEV bs=1M count=1 >>$seqres.full 2>&1 \
+		|| _fail "dd failed"

$WIPEFS_PROG -a $PRUNE_DEV ?

tests/btrfs/095 line 76

Quoted 1 line

tests/btrfs/095 new file +1 -0
+	mount -o degraded $SCRATCH_DEV $SCRATCH_MNT

_scratch_mount "-o degraded"

tests/btrfs/095 line 80

Quoted 4 lines

tests/btrfs/095 new file +4 -0

+
+	# replace the missing dev $PRUNE_DEV with $REPLACE_DEV and scrub it
+	$BTRFS_UTIL_PROG replace start -B -r $PRUNE_DEV_ID $REPLACE_DEV \
+		$SCRATCH_MNT -f >>$seqres.full 2>&1 || _fail "replace failed"

Don't _fail here, just echo error messages to break golden image. Better to append the error message to $seqres.full too, for debug purpose.

tests/btrfs/095 line 82

Quoted 2 lines

tests/btrfs/095 new file +2 -0

+	$BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >>$seqres.full 2>&1 \
+		|| _fail "scrub failed"

Here too.

tests/btrfs/095 line 84

Quoted 2 lines

tests/btrfs/095 new file +2 -0
+
+	umount $SCRATCH_MNT

_scratch_unmount

Quoted 22 lines

tests/btrfs/095 new file +9 -0

+}
+
+echo "Silence is golden"
+for t in "${_btrfs_profile_configs[@]}"; do
+	run_test "$t"
+done
+
+status=0
+exit

tests/btrfs/095.out new file +2 -0

new file mode 100644
index 000000000000..80ad3b9c2a7b
@@ -0,0 +1,2 @@
+QA output created by 095
+Silence is golden

tests/btrfs/group +0 -0

index ffe18bff0d21..f20a191c24be 100644

tests/btrfs/group line 99

Quoted 4 lines

tests/btrfs/group +1 -0

@@ -96,3 +96,4 @@
 092 auto quick send
 093 auto quick clone
 094 auto quick send
+095 auto

Should also be in "replace" group.

Thanks,
Eryu

Quoted 7 lines

Signature · 6 lines

2.4.6

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Omar Sandoval

author

2015-07-24 17:52 UTC #14 lore raw

Re: [PATCH 3/3] btrfs: add a test of replace missing dev in diff raid

Quoted 173 lines · Eryu Guan

Quoted 62 lines · Omar Sandoval
From: Wang Yanfeng <[email protected]>
Test of missing device replace in different raid modes. This test requires SCRATCH_DEV_POOL contain 5 same size devices.
This issue has been fixed by Omar's patch:
Btrfs: RAID 5/6 missing device scrub+replace
Signed-off-by Wang Yanfeng <[email protected]>
Signed-off-by Omar Sandoval <[email protected]>
---
 tests/btrfs/095     | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/095.out |  2 ++
 tests/btrfs/group   |  1 +
 3 files changed, 96 insertions(+)
 create mode 100755 tests/btrfs/095
 create mode 100644 tests/btrfs/095.out
tests/btrfs/095 new file +38 -0
new file mode 100755
index 000000000000..2dfb919fe4f6
@@ -0,0 +1,93 @@
+#! /bin/bash
+# FS QA Test No. btrfs/095
+#
+# Test of missing device replace in different raid mode
+#
+# Be sure $SCRATCH_DEV_POOL including 5 devices. And all
+# devices in pool must be in the same size.
+#
+# To check the fs after replacing a dev, a scrub run is performed.
+#
+# This issue has been fixed by Omar Sandoval's patch:
+# 	Btrfs: RAID 5/6 missing device scrub+replace
+#-----------------------------------------------------------------------
+# Copyright (c) 2015 Fujitsu.  All Rights Reserved.
+# Author: Wang Yanfeng <[email protected]>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+status=1
+trap "exit \$status" 0 1 2 3 15
+
tests/btrfs/095 line 48
Quoted 10 lines
tests/btrfs/095 new file +10 -0
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_need_to_be_root
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_scratch_dev_pool 5
Need a call of _require_scratch_dev_pool_equal_size here
tests/btrfs/095 line 57
Quoted 9 lines
tests/btrfs/095 new file +9 -0
+_btrfs_get_profile_configs replace-missing
+
+rm -f $seqres.full
+
+REPLACE_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $5}'`
+export SCRATCH_DEV_POOL=`echo $SCRATCH_DEV_POOL | awk '{print $1,$2,$3,$4}'`
+
+run_test () {
+	local mkfs_opts=$1
echo the $mkfs_opts to $seqres.full is useful for debug purpose, so if some profile config fails we know what config is failing.
echo "Test $mkfs_opts" >>$seqres.full
tests/btrfs/095 line 58
Quoted 1 line
tests/btrfs/095 new file +1 -0
+	_scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1 || _fail "mkfs failed"
Continue to test next profile config if mkfs failed, _fail breaks out the whole test.
tests/btrfs/095 line 70
Quoted 12 lines
tests/btrfs/095 new file +12 -0
+	_scratch_mount
+
+	PRUNE_DEV=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+	PRUNE_DEV_ID=`$BTRFS_UTIL_PROG fi show $SCRATCH_MNT | grep $PRUNE_DEV | awk '{print $2}'`
+
+	# dd some data
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file1 bs=1M count=1 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file2 bs=1M count=2 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
+	dd if=/dev/urandom of="$SCRATCH_MNT"/file3 bs=1M count=4 \
+		>>$seqres.full 2>&1 || _fail "dd failed"
I don't think the _fail here is necessary either.
tests/btrfs/095 line 73
Quoted 3 lines
tests/btrfs/095 new file +3 -0
+
+	# prune the device PRUNE_DEV && remount by degraded mode
+	umount $SCRATCH_MNT
_scratch_unmount
tests/btrfs/095 line 75
Quoted 2 lines
tests/btrfs/095 new file +2 -0
+	dd if=/dev/zero of=$PRUNE_DEV bs=1M count=1 >>$seqres.full 2>&1 \
+		|| _fail "dd failed"
$WIPEFS_PROG -a $PRUNE_DEV ?
tests/btrfs/095 line 76
Quoted 1 line
tests/btrfs/095 new file +1 -0
+	mount -o degraded $SCRATCH_DEV $SCRATCH_MNT
_scratch_mount "-o degraded"
tests/btrfs/095 line 80
Quoted 4 lines
tests/btrfs/095 new file +4 -0
+
+	# replace the missing dev $PRUNE_DEV with $REPLACE_DEV and scrub it
+	$BTRFS_UTIL_PROG replace start -B -r $PRUNE_DEV_ID $REPLACE_DEV \
+		$SCRATCH_MNT -f >>$seqres.full 2>&1 || _fail "replace failed"
Don't _fail here, just echo error messages to break golden image. Better to append the error message to $seqres.full too, for debug purpose.
tests/btrfs/095 line 82
Quoted 2 lines
tests/btrfs/095 new file +2 -0
+	$BTRFS_UTIL_PROG scrub start -B $SCRATCH_MNT >>$seqres.full 2>&1 \
+		|| _fail "scrub failed"
Here too.
tests/btrfs/095 line 84
Quoted 2 lines
tests/btrfs/095 new file +2 -0
+
+	umount $SCRATCH_MNT
_scratch_unmount
Quoted 22 lines
tests/btrfs/095 new file +9 -0
+}
+
+echo "Silence is golden"
+for t in "${_btrfs_profile_configs[@]}"; do
+	run_test "$t"
+done
+
+status=0
+exit
tests/btrfs/095.out new file +2 -0
new file mode 100644
index 000000000000..80ad3b9c2a7b
@@ -0,0 +1,2 @@
+QA output created by 095
+Silence is golden
tests/btrfs/group +0 -0
index ffe18bff0d21..f20a191c24be 100644
tests/btrfs/group line 99
Quoted 4 lines
tests/btrfs/group +1 -0
@@ -96,3 +96,4 @@
 092 auto quick send
 093 auto quick clone
 094 auto quick send
+095 auto
Should also be in "replace" group.

Quoted 2 lines

Thanks,
Eryu

Thanks for reviewing, I'll fix this all up in a v2.

Signature · 1 line

Omar