Skip to contentSkip to conversation

btrfs: Introduce new rescue= mount options

PATCH In review Qu Wenruo 3 participants 7 messages Apr 15, 2019 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs: Introduce new rescue= mount options

This patchset can be fetched from github:
https://github.com/adam900710/linux/tree/rescue_options

The basis HEAD is v5.1-rc4 tag.

There are quite a lot btrfs extent tree corruption report in the mail list.
Since btrfs will do mount time block group item search, one corrupted leaf containing block group item will prevent the whole fs to be mounted.

This patchset will try to address the problem by introducing a new mount option, "rescue=skip_bg", as a last-resort rescue.
Of course this option will have a lot of restriction to prevent further screwing up the fs, including:
- Permanent RO
No remount rw is allowed

- No dirty log
Either clean the log or use rescue=no_log_replay mount option

This "rescue=skip_bg" has some advantage compared to user space tool like "btrfs-restore":
- Unified recovery tool
User can use any tool they're familiar with, as long as the kernel doesn't panic.

- More info for subvolume.
"btrfs subv list" can work now!

Also move the following mount options to "rescue=" group:
- nologreplay
to rescue=no_log_replay

- usebackuproot
to rescue=use_backup_root

Old options are still available for compatibility purpose, but they are deprecated in favor of new 'rescue=' super option.

Different rescue sub options can be separated by ':', like:
"rescue=no_log_replay:skip_bg:use_backup_root".
Or the traditional but longer way like:
"rescue=no_log_replay,rescue=skip_bg"

The separation character is chosen by:
- No conflicts with existing character
Especially no conflict with ','.

- No extra escaping/quota
Original plan is ';', but since it'll be interpreted by bash, it's changed to current ':'.

Changelog:
v2:
- Introduce 'rescue=' super option.
- Rename original 'usebackuproot' and 'nologreplay'.
It at least makes my vim spell check happier.
- Remove 'recovery' mount option.
As its successor is now deprecated, not need to keep the predecessor.

v2.1:
- Rebase to v5.1-rc4.
- Fix the typos in the cover letter.

Qu Wenruo (3):
btrfs: Remove "recovery" mount option
btrfs: Introduce "rescue=" mount option
btrfs: Introduce new mount option to skip block group items scan

 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     | 29 +++++++++++--
 fs/btrfs/extent-tree.c | 59 ++++++++++++++++++++++++++
 fs/btrfs/super.c       | 96 ++++++++++++++++++++++++++++++++++++------
 fs/btrfs/volumes.c     |  7 +++
 5 files changed, 176 insertions(+), 16 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH 1/3 btrfs: Remove "recovery" mount option

Commit 8dcddfa048de ("btrfs: Introduce new mount option usebackuproot to replace recovery") deprecates "recovery" mount option in 2016, and it has been 3 years, it should be OK to remove "recovery" mount option.

As we're even going to deprecate the successor, "usebackuproot" mount option, there isn't really much need to keep the original option.

---
 fs/btrfs/super.c | 9 ---------
 1 file changed, 9 deletions(-)
fs/btrfs/super.c +0 -9
index 120e4340792a..4625e7fbeba5 100644
@@ -311,7 +311,6 @@ enum {
 	Opt_defrag, Opt_nodefrag,
 	Opt_discard, Opt_nodiscard,
 	Opt_nologreplay,
-	Opt_norecovery,
 	Opt_ratio,
 	Opt_rescan_uuid_tree,
 	Opt_skip_balance,
@@ -329,7 +328,6 @@ enum {
 
 	/* Deprecated options */
 	Opt_alloc_start,
-	Opt_recovery,
 	Opt_subvolrootid,
 
 	/* Debugging options */
@@ -374,7 +372,6 @@ static const match_table_t tokens = {
 	{Opt_discard, "discard"},
 	{Opt_nodiscard, "nodiscard"},
 	{Opt_nologreplay, "nologreplay"},
-	{Opt_norecovery, "norecovery"},
 	{Opt_ratio, "metadata_ratio=%u"},
 	{Opt_rescan_uuid_tree, "rescan_uuid_tree"},
 	{Opt_skip_balance, "skip_balance"},
@@ -396,7 +393,6 @@ static const match_table_t tokens = {
 
 	/* Deprecated options */
 	{Opt_alloc_start, "alloc_start=%s"},
-	{Opt_recovery, "recovery"},
 	{Opt_subvolrootid, "subvolrootid=%d"},
 
 	/* Debugging options */
@@ -670,7 +666,6 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			btrfs_clear_and_info(info, NOTREELOG,
 					     "enabling tree log");
 			break;
-		case Opt_norecovery:
 		case Opt_nologreplay:
 			btrfs_set_and_info(info, NOLOGREPLAY,
 					   "disabling log replay at mount time");
@@ -759,10 +754,6 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			btrfs_clear_and_info(info, AUTO_DEFRAG,
 					     "disabling auto defrag");
 			break;
-		case Opt_recovery:
-			btrfs_warn(info,
-				   "'recovery' is deprecated, use 'usebackuproot' instead");
-			/* fall through */
 		case Opt_usebackuproot:
 			btrfs_info(info,
 				   "trying to use backup root at mount time");
Signature · 1 line
2.21.0
#2 lore raw

Re: [PATCH v2.1 1/3] btrfs: Remove "recovery" mount option

Quoted 6 lines · Qu Wenruo

Commit 8dcddfa048de ("btrfs: Introduce new mount option usebackuproot to replace recovery") deprecates "recovery" mount option in 2016, and it has been 3 years, it should be OK to remove "recovery" mount option.

As we're even going to deprecate the successor, "usebackuproot" mount option, there isn't really much need to keep the original option.

Has it been documented somewhere, when the mount option is due to be finally removed?

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#3 lore raw

Re: [PATCH v2.1 1/3] btrfs: Remove "recovery" mount option

Quoted 10 lines · Johannes Thumshirn
Quoted 6 lines · Qu Wenruo

Commit 8dcddfa048de ("btrfs: Introduce new mount option usebackuproot to replace recovery") deprecates "recovery" mount option in 2016, and it has been 3 years, it should be OK to remove "recovery" mount option.

As we're even going to deprecate the successor, "usebackuproot" mount option, there isn't really much need to keep the original option.

Has it been documented somewhere, when the mount option is due to be finally removed?

It is mentioned as deprecated in btrfs(5):
recovery
(since: 3.2, default: off, deprecated since: 4.5)

Note
this option has been replaced by usebackuproot and should not be used but will work on 4.5+ kernels.

The removal date is not mentioned though.

Thanks,
Qu

Reviewed-by
#4 lore raw

Re: [PATCH v2.1 1/3] btrfs: Remove "recovery" mount option

Quoted 10 lines · Qu Wenruo

It is mentioned as deprecated in btrfs(5):
recovery
(since: 3.2, default: off, deprecated since: 4.5)

Note
this option has been replaced by usebackuproot and should not be used but will work on 4.5+ kernels.

The removal date is not mentioned though.

Hmm ok, just because xfs for example give an schedule for removal of mount options.

Either way, code wise it looks ok to me:

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
author
#5 lore raw

PATCH 2/3 btrfs: Introduce "rescue=" mount option

In reply to Qu Wenruo

This patch introduces a new "rescue=" mount option for all those mount options for data recovery.

Different rescue sub options are seperated by ':'.
(The original plan is to use ';', but ';' needs to be escaped/quoted, or it will be interpreted by bash)

The following mount options are converted to "rescue=", old mount options are deprecated but still available for compatibility purpose:

- usebackuproot
Now it's "rescue=use_backup_root"

- nologreplay
Now it's "rescue=no_log_replay"

The new underscore is here to make the option more readable and make spell check happier.

---
 fs/btrfs/super.c | 65 +++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 62 insertions(+), 3 deletions(-)
fs/btrfs/super.c +62 -3
index 4625e7fbeba5..87968b9d5499 100644
@@ -310,7 +310,6 @@ enum {
 	Opt_datasum, Opt_nodatasum,
 	Opt_defrag, Opt_nodefrag,
 	Opt_discard, Opt_nodiscard,
-	Opt_nologreplay,
 	Opt_ratio,
 	Opt_rescan_uuid_tree,
 	Opt_skip_balance,
@@ -323,7 +322,6 @@ enum {
 	Opt_subvolid,
 	Opt_thread_pool,
 	Opt_treelog, Opt_notreelog,
-	Opt_usebackuproot,
 	Opt_user_subvol_rm_allowed,
 
 	/* Deprecated options */
@@ -341,6 +339,8 @@ enum {
 #ifdef CONFIG_BTRFS_FS_REF_VERIFY
 	Opt_ref_verify,
 #endif
+	/* Rescue options */
+	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,
 	Opt_err,
 };
 
@@ -401,6 +401,7 @@ static const match_table_t tokens = {
 	{Opt_check_integrity_print_mask, "check_int_print_mask=%u"},
 	{Opt_enospc_debug, "enospc_debug"},
 	{Opt_noenospc_debug, "noenospc_debug"},
+	{Opt_rescue, "rescue=%s"},
 #ifdef CONFIG_BTRFS_DEBUG
 	{Opt_fragment_data, "fragment=data"},
 	{Opt_fragment_metadata, "fragment=metadata"},
@@ -412,6 +413,55 @@ static const match_table_t tokens = {
 	{Opt_err, NULL},
 };
 
+static const match_table_t rescue_tokens = {
+	{Opt_usebackuproot, "use_backup_root"},
+	{Opt_nologreplay, "no_log_replay"},
+	{Opt_err, NULL},
+};
+
+static int parse_rescue_options(struct btrfs_fs_info *info, const char *options)
+{
+	char *opts;
+	char *orig;
+	char *p;
+	substring_t args[MAX_OPT_ARGS];
+	int ret = 0;
+
+	opts = kstrdup(options, GFP_KERNEL);
+	if (!opts)
+		return -ENOMEM;
+	orig = opts;
+
+	while ((p = strsep(&opts, ":")) != NULL) {
+		int token;
+
+		if (!*p)
+			continue;
+		token = match_token(p, rescue_tokens, args);
+		switch (token){
+		case Opt_usebackuproot:
+			btrfs_info(info,
+				   "trying to use backup root at mount time");
+			btrfs_set_opt(info->mount_opt, USEBACKUPROOT);
+			break;
+		case Opt_nologreplay:
+			btrfs_set_and_info(info, NOLOGREPLAY,
+					   "disabling log replay at mount time");
+			break;
+		case Opt_err:
+			btrfs_info(info, "unrecognized rescue option '%s'", p);
+			ret = -EINVAL;
+			goto out;
+		default:
+			break;
+		}
+
+	}
+out:
+	kfree(orig);
+	return ret;
+}
+
 /*
  * Regular mount options parser.  Everything that is needed only when
  * reading in a new superblock is parsed here.
@@ -667,6 +717,8 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 					     "enabling tree log");
 			break;
 		case Opt_nologreplay:
+			btrfs_warn(info,
+	"'nologreplay' is deprecated, use 'rescue=no_log_replay' instead");
 			btrfs_set_and_info(info, NOLOGREPLAY,
 					   "disabling log replay at mount time");
 			break;
@@ -755,6 +807,8 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 					     "disabling auto defrag");
 			break;
 		case Opt_usebackuproot:
+			btrfs_warn(info,
+	"'usebackuproot' is deprecated, use 'rescue=use_backup_root' instead");
 			btrfs_info(info,
 				   "trying to use backup root at mount time");
 			btrfs_set_opt(info->mount_opt, USEBACKUPROOT);
@@ -841,6 +895,11 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			btrfs_set_opt(info->mount_opt, REF_VERIFY);
 			break;
 #endif
+		case Opt_rescue:
+			ret = parse_rescue_options(info, args[0].from);
+			if (ret < 0)
+				goto out;
+			break;
 		case Opt_err:
 			btrfs_info(info, "unrecognized mount option '%s'", p);
 			ret = -EINVAL;
@@ -1307,7 +1366,7 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
 	if (btrfs_test_opt(info, NOTREELOG))
 		seq_puts(seq, ",notreelog");
 	if (btrfs_test_opt(info, NOLOGREPLAY))
-		seq_puts(seq, ",nologreplay");
+		seq_puts(seq, ",rescue=no_log_replay");
 	if (btrfs_test_opt(info, FLUSHONCOMMIT))
 		seq_puts(seq, ",flushoncommit");
 	if (btrfs_test_opt(info, DISCARD))
Signature · 1 line
2.21.0
author
#6 lore raw

PATCH 3/3 btrfs: Introduce new mount option to skip block group items scan

In reply to Qu Wenruo

[PROBLEM]
There are some reports of corrupted fs which can't be mounted due to corrupted extent tree.

However under such situation, it's more likely the fs/subvolume trees are still fine.

For such case we normally go btrfs-restore and salvage as much as we can. However btrfs-restore can't list subvolumes as "btrfs subv list", making it harder to restore a fs.

[ENHANCEMENT]
This patch will introduce a new mount option "rescue=skip_bg" to skip the mount time block group scan, and use chunk info purely to populate fake block group cache.

The mount option has the following dependency:
- RO mount
Obviously.

- No log tree or use nologreplay mount option

- No way to remoutn RW
Similar to nologreplay option.

This should allow kernel to accept most extent tree corruption, and salvage data and subvolume info.

---
 fs/btrfs/ctree.h       |  1 +
 fs/btrfs/disk-io.c     | 29 ++++++++++++++++++---
 fs/btrfs/extent-tree.c | 59 ++++++++++++++++++++++++++++++++++++++++++
 fs/btrfs/super.c       | 24 ++++++++++++++++-
 fs/btrfs/volumes.c     |  7 +++++
 5 files changed, 115 insertions(+), 5 deletions(-)
fs/btrfs/ctree.h +1 -0
index b3642367a595..4636c60e2601 100644
@@ -1442,6 +1442,7 @@ static inline u32 BTRFS_MAX_XATTR_SIZE(const struct btrfs_fs_info *info)
 #define BTRFS_MOUNT_FREE_SPACE_TREE	(1 << 26)
 #define BTRFS_MOUNT_NOLOGREPLAY		(1 << 27)
 #define BTRFS_MOUNT_REF_VERIFY		(1 << 28)
+#define BTRFS_MOUNT_SKIP_BG		(1 << 29)
 
 #define BTRFS_DEFAULT_COMMIT_INTERVAL	(30)
 #define BTRFS_DEFAULT_MAX_INLINE	(2048)
fs/btrfs/disk-io.c +25 -4
index 6fe9197f6ee4..711d01f9a614 100644
@@ -2327,11 +2327,15 @@ static int btrfs_read_roots(struct btrfs_fs_info *fs_info)
 
 	root = btrfs_read_tree_root(tree_root, &location);
 	if (IS_ERR(root)) {
-		ret = PTR_ERR(root);
-		goto out;
+		if (!btrfs_test_opt(fs_info, SKIP_BG)) {
+			ret = PTR_ERR(root);
+			goto out;
+		}
+		fs_info->extent_root = NULL;
+	} else {
+		set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
+		fs_info->extent_root = root;
 	}
-	set_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state);
-	fs_info->extent_root = root;
 
 	location.objectid = BTRFS_DEV_TREE_OBJECTID;
 	root = btrfs_read_tree_root(tree_root, &location);
@@ -2949,6 +2953,23 @@ int open_ctree(struct super_block *sb,
 		goto fail_alloc;
 	}
 
+	/* Skip bg needs RO and no log tree replay */
+	if (btrfs_test_opt(fs_info, SKIP_BG)) {
+		if (!sb_rdonly(sb)) {
+			btrfs_err(fs_info,
+		"skip_bg mount option can only be used with read-only mount");
+			err = -EINVAL;
+			goto fail_alloc;
+		}
+		if (btrfs_super_log_root(disk_super) &&
+		    !btrfs_test_opt(fs_info, NOTREELOG)) {
+			btrfs_err(fs_info,
+	"skip_bg must be used with notreelog mount option for dirty log");
+			err = -EINVAL;
+			goto fail_alloc;
+		}
+	}
+
 	ret = btrfs_init_workqueues(fs_info, fs_devices);
 	if (ret) {
 		err = ret;
fs/btrfs/extent-tree.c +59 -0

71 lines not on this page · raw message

fs/btrfs/super.c +23 -1
index 87968b9d5499..f27a7b1917f7 100644
@@ -340,7 +340,7 @@ enum {
 	Opt_ref_verify,
 #endif
 	/* Rescue options */
-	Opt_rescue, Opt_usebackuproot, Opt_nologreplay,
+	Opt_rescue, Opt_usebackuproot, Opt_nologreplay, Opt_rescue_skip_bg,
 	Opt_err,
 };
 
@@ -416,6 +416,7 @@ static const match_table_t tokens = {
 static const match_table_t rescue_tokens = {
 	{Opt_usebackuproot, "use_backup_root"},
 	{Opt_nologreplay, "no_log_replay"},
+	{Opt_rescue_skip_bg, "skip_bg"},
 	{Opt_err, NULL},
 };
 
@@ -448,6 +449,10 @@ static int parse_rescue_options(struct btrfs_fs_info *info, const char *options)
 			btrfs_set_and_info(info, NOLOGREPLAY,
 					   "disabling log replay at mount time");
 			break;
+		case Opt_rescue_skip_bg:
+			btrfs_set_and_info(info, SKIP_BG,
+				"skip mount time block group searching");
+			break;
 		case Opt_err:
 			btrfs_info(info, "unrecognized rescue option '%s'", p);
 			ret = -EINVAL;
@@ -1367,6 +1372,8 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
 		seq_puts(seq, ",notreelog");
 	if (btrfs_test_opt(info, NOLOGREPLAY))
 		seq_puts(seq, ",rescue=no_log_replay");
+	if (btrfs_test_opt(info, SKIP_BG))
+		seq_puts(seq, ",rescue=skip_bg");
 	if (btrfs_test_opt(info, FLUSHONCOMMIT))
 		seq_puts(seq, ",flushoncommit");
 	if (btrfs_test_opt(info, DISCARD))
@@ -1796,6 +1803,14 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data)
 	if (ret)
 		goto restore;
 
+	if (btrfs_test_opt(fs_info, SKIP_BG) !=
+	    (old_opts & BTRFS_MOUNT_SKIP_BG)) {
+		btrfs_err(fs_info,
+		"rescue=skip_bg mount option can't be changed during remount");
+		ret = -EINVAL;
+		goto restore;
+	}
+
 	btrfs_remount_begin(fs_info, old_opts, *flags);
 	btrfs_resize_thread_pool(fs_info,
 		fs_info->thread_pool_size, old_thread_pool_size);
@@ -1857,6 +1872,13 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data)
 			goto restore;
 		}
 
+		if (btrfs_test_opt(fs_info, SKIP_BG)) {
+			btrfs_err(fs_info,
+		"remounting read-write with rescue=skip_bg is not allowed");
+			ret = -EINVAL;
+			goto restore;
+		}
+
 		ret = btrfs_cleanup_fs_roots(fs_info);
 		if (ret)
 			goto restore;
fs/btrfs/volumes.c +7 -0
index db934ceae9c1..443f52c7630d 100644
@@ -7903,6 +7903,13 @@ int btrfs_verify_dev_extents(struct btrfs_fs_info *fs_info)
 	u64 prev_dev_ext_end = 0;
 	int ret = 0;
 
+	/*
+	 * For rescue=skip_bg mount option, we're already RO and are salvaging
+	 * data, no need for such strict check.
+	 */
+	if (btrfs_test_opt(fs_info, SKIP_BG))
+		return 0;
+
 	key.objectid = 1;
 	key.type = BTRFS_DEV_EXTENT_KEY;
 	key.offset = 0;
Signature · 1 line
2.21.0