Skip to contentSkip to conversation

fix issues during suspended replace operation

PATCH In review Anand Jain 3 participants 7 messages Aug 12, 2022 – Sep 22, 2022 Btrfs file system

lore
Share & tools
lore

Anand Jain

author
#0 lore raw

cover fix issues during suspended replace operation

While trying to reproduce the issue reported in the ML. Found few issues as in the individual independent patches below.

Anand Jain (2):
btrfs: fix assert during replace-cancel of suspended replace-operation
btrfs: add info when mount fails due to stale replace target

 fs/btrfs/dev-replace.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Signature · 1 line
2.33.1
author
#1 lore raw

PATCH 1/2 btrfs: fix assert during replace-cancel of suspended replace-operation

If the filesystem mounts with the replace-operation in a suspended state and try to cancel the suspended replace-operation, we hit the assert. The assert came from the commit fe97e2e173af ("btrfs: dev-replace: replace's scrub must not be running in suspended state") that was actually not required. So just remove it.

 $ mount /dev/sda5 /btrfs

BTRFS info (device sda5): cannot continue dev_replace, tgtdev is missing BTRFS info (device sda5): you may cancel the operation after 'mount -o degraded'

 $ mount -o degraded /dev/sda5 /btrfs <-- success.
 $ btrfs replace cancel /btrfs
    kernel: assertion failed: ret != -ENOTCONN, in fs/btrfs/dev-replace.c:1131
    kernel: ------------[ cut here ]------------
    kernel: kernel BUG at fs/btrfs/ctree.h:3750!

After the patch:

 $ btrfs replace cancel /btrfs

BTRFS info (device sda5): suspended dev_replace from /dev/sda5 (devid 1) to <missing disk> canceled

---
 fs/btrfs/dev-replace.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/dev-replace.c +1 -2
index 488f2105c5d0..9d46a702bc11 100644
@@ -1124,8 +1124,7 @@ int btrfs_dev_replace_cancel(struct btrfs_fs_info *fs_info)
 		up_write(&dev_replace->rwsem);
 
 		/* Scrub for replace must not be running in suspended state */
-		ret = btrfs_scrub_cancel(fs_info);
-		ASSERT(ret != -ENOTCONN);
+		btrfs_scrub_cancel(fs_info);
 
 		trans = btrfs_start_transaction(root, 0);
 		if (IS_ERR(trans)) {
Signature · 1 line
2.33.1
#2 lore raw

Re: [PATCH 1/2] btrfs: fix assert during replace-cancel of suspended replace-operation

Quoted 20 lines · Anand Jain

If the filesystem mounts with the replace-operation in a suspended state and try to cancel the suspended replace-operation, we hit the assert. The assert came from the commit fe97e2e173af ("btrfs: dev-replace: replace's scrub must not be running in suspended state") that was actually not required. So just remove it.

 $ mount /dev/sda5 /btrfs

BTRFS info (device sda5): cannot continue dev_replace, tgtdev is missing BTRFS info (device sda5): you may cancel the operation after 'mount -o degraded'

 $ mount -o degraded /dev/sda5 /btrfs <-- success.
 $ btrfs replace cancel /btrfs
    kernel: assertion failed: ret != -ENOTCONN, in fs/btrfs/dev-replace.c:1131
    kernel: ------------[ cut here ]------------
    kernel: kernel BUG at fs/btrfs/ctree.h:3750!
Quoted 5 lines

After the patch:

 $ btrfs replace cancel /btrfs

BTRFS info (device sda5): suspended dev_replace from /dev/sda5 (devid 1) to <missing disk> canceled

Anand, can you please add a test case to fstests?
This is a scenario with no coverage at all in fstests, therefore specially useful to have it there.

Thanks.

Quoted 23 lines
---
 fs/btrfs/dev-replace.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/dev-replace.c +1 -3
index 488f2105c5d0..9d46a702bc11 100644
@@ -1124,8 +1124,7 @@ int btrfs_dev_replace_cancel(struct btrfs_fs_info *fs_info)
                up_write(&dev_replace->rwsem);
 
                /* Scrub for replace must not be running in suspended state */
-               ret = btrfs_scrub_cancel(fs_info);
-               ASSERT(ret != -ENOTCONN);
+               btrfs_scrub_cancel(fs_info);
 
                trans = btrfs_start_transaction(root, 0);
                if (IS_ERR(trans)) {
--

2.33.1

Signature · 3 lines
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”
author
#3 lore raw

Re: [PATCH 1/2] btrfs: fix assert during replace-cancel of suspended replace-operation

Quoted 23 lines · Filipe Manana
Quoted 22 lines · Anand Jain

If the filesystem mounts with the replace-operation in a suspended state and try to cancel the suspended replace-operation, we hit the assert. The assert came from the commit fe97e2e173af ("btrfs: dev-replace: replace's scrub must not be running in suspended state") that was actually not required. So just remove it.

  $ mount /dev/sda5 /btrfs

BTRFS info (device sda5): cannot continue dev_replace, tgtdev is missing BTRFS info (device sda5): you may cancel the operation after 'mount -o degraded'

  $ mount -o degraded /dev/sda5 /btrfs <-- success.
  $ btrfs replace cancel /btrfs
     kernel: assertion failed: ret != -ENOTCONN, in fs/btrfs/dev-replace.c:1131
     kernel: ------------[ cut here ]------------
     kernel: kernel BUG at fs/btrfs/ctree.h:3750!

After the patch:

Quoted 8 lines
Quoted 3 lines
  $ btrfs replace cancel /btrfs

BTRFS info (device sda5): suspended dev_replace from /dev/sda5 (devid 1) to <missing disk> canceled

Anand, can you please add a test case to fstests?
This is a scenario with no coverage at all in fstests, therefore specially useful to have it there.

I thought about it before and found that unless we implement the replace-pause sub-command, we can't get a pending replace item in an unmounted btrfs using a script.

However, to test it manually, I did an abrupt reboot (or power-off, I think).

Thanks.

Quoted 27 lines

Thanks.

Quoted 23 lines
---
  fs/btrfs/dev-replace.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/dev-replace.c +1 -3
index 488f2105c5d0..9d46a702bc11 100644
@@ -1124,8 +1124,7 @@ int btrfs_dev_replace_cancel(struct btrfs_fs_info *fs_info)
                 up_write(&dev_replace->rwsem);
 
                 /* Scrub for replace must not be running in suspended state */
-               ret = btrfs_scrub_cancel(fs_info);
-               ASSERT(ret != -ENOTCONN);
+               btrfs_scrub_cancel(fs_info);
 
                 trans = btrfs_start_transaction(root, 0);
                 if (IS_ERR(trans)) {
--

2.33.1

author
#4 lore raw

PATCH 2/2 btrfs: add info when mount fails due to stale replace target

In reply to Anand Jain

If the replace-target device re-appears after the suspended replace is cancelled, it blocks the mount operation as it can't find the matching replace-item in the metadata. As shown below,

BTRFS error (device sda5): replace devid present without an active replace item

To overcome this situation, the user can run the command

   btrfs device scan --forget <device-path-to-devid=0>

and try the mount command again. And also, to avoid repeating the issue, superblock on the devid=0 must be wiped.

wipefs -a device-path-to-devid=0.

This patch adds some info when this situation occurs.

---
 fs/btrfs/dev-replace.c | 2 ++
 1 file changed, 2 insertions(+)
fs/btrfs/dev-replace.c +2 -0
index 9d46a702bc11..7202b76ce59f 100644
@@ -166,6 +166,8 @@ int btrfs_init_dev_replace(struct btrfs_fs_info *fs_info)
 		if (btrfs_find_device(fs_info->fs_devices, &args)) {
 			btrfs_err(fs_info,
 			"replace devid present without an active replace item");
+			btrfs_info(fs_info,
+	"mount after the command 'btrfs deivce scan --forget <devpath-of-id-0>'");
 			ret = -EUCLEAN;
 		} else {
 			dev_replace->srcdev = NULL;
Signature · 1 line
2.33.1
#5 lore raw

Re: [PATCH 2/2] btrfs: add info when mount fails due to stale replace target

Quoted 29 lines · Anand Jain

If the replace-target device re-appears after the suspended replace is cancelled, it blocks the mount operation as it can't find the matching replace-item in the metadata. As shown below,

BTRFS error (device sda5): replace devid present without an active replace item

To overcome this situation, the user can run the command

   btrfs device scan --forget <device-path-to-devid=0>

and try the mount command again. And also, to avoid repeating the issue, superblock on the devid=0 must be wiped.

wipefs -a device-path-to-devid=0.

This patch adds some info when this situation occurs.

---
 fs/btrfs/dev-replace.c | 2 ++
 1 file changed, 2 insertions(+)
fs/btrfs/dev-replace.c +0 -0
index 9d46a702bc11..7202b76ce59f 100644

fs/btrfs/dev-replace.c int btrfs_init_dev_replace(struct btrfs_fs_info *fs_info) · line 170

Quoted 5 lines
fs/btrfs/dev-replace.c +2 -0
@@ -166,6 +166,8 @@ int btrfs_init_dev_replace(struct btrfs_fs_info *fs_info)
 		if (btrfs_find_device(fs_info->fs_devices, &args)) {
 			btrfs_err(fs_info,
 			"replace devid present without an active replace item");
+			btrfs_info(fs_info,
+	"mount after the command 'btrfs deivce scan --forget <devpath-of-id-0>'");

The messages should be on the same level and in one message, I've reprhrased it a bit.

#6 lore raw
In reply to Anand Jain
Quoted 6 lines · Anand Jain

While trying to reproduce the issue reported in the ML. Found few issues as in the individual independent patches below.

Anand Jain (2):
btrfs: fix assert during replace-cancel of suspended replace-operation
btrfs: add info when mount fails due to stale replace target

Added to misc-next, thanks.