btrfs: rescue= mount options enhancement to support interrupted csum conversion
v2 cover btrfs: rescue= mount options enhancement to support interrupted csum conversion
[CHANGELOG]
v2:
- Add the new rescue options to supported rescue options sysfs interface
- Add the state char 'S' for skipping metadata csum
- Add an info line of unsupported super flags if rescue=ignoresuperflags is specified
- Add const prefix for one-time btrfs_test_opt() result
[REPO]
https://github.com/adam900710/linux/tree/rescue_changes
[BACKGROUND]
There is an adventurous user using btrfstune to convert a 32T btrfs from crc32c to xxhash.
However for such huge fs, it takes too long time and the reporter canceled the conversion.
This makes the reporter unable to mount the fs at all.
[CAUSE]
First of all, for a half converted fs, we will never allow RW mount, so everything must be done in rescue mode.
There are several different stages of csum conversion, and at different stage it requires different handling from kernel:
- Generationg new data csums
At this stage only the super flags (CHANGING_DATA_CSUM flag) is preventing the kernel from mounting.
Intrdoce "rescue=ignoresuperflags" to address this.
- Deleting old data cums
The same super flags problem, with possible missing data csums.
Despite the new "rescue=ignoresuperflags", end users will also need the existing "rescue=ignoredatacsums" mount option.
- Renaming the objectid of new data cums
The new csums' objectid will be changed to the regular one.
During this we can hit data csum mismatch.
So the same "rescue=ignoresuperflags:ignoredatacsums" can handle it already.
- Rewriting metadata csums
This part is done in-place (no COW), with a new super flags (CHANGING_META_CSUM).
So here introduce a new "rescue=ignoremetacsums" to ignore the metadata checksum verification (and rely on the remaining sanity checks like tree-checkers).
The first 2 patches are just small cleanups, meanwhile the last two are the new "rescue=" mount options to handle interrupted csum change.
Qu Wenruo (4):
btrfs: remove unused Opt enums
btrfs: output the unrecognized super flags as hex
btrfs: introduce new "rescue=ignoremetacsums" mount option
btrfs: introduce new "rescue=ignoresuperflags" mount option
fs/btrfs/bio.c | 2 +- fs/btrfs/disk-io.c | 35 +++++++++++++++++++++++++---------- fs/btrfs/file-item.c | 2 +- fs/btrfs/fs.h | 5 ++++- fs/btrfs/messages.c | 3 ++- fs/btrfs/super.c | 27 +++++++++++++++++++++++---- fs/btrfs/sysfs.c | 2 ++ fs/btrfs/zoned.c | 2 +- 8 files changed, 59 insertions(+), 19 deletions(-)
Signature · 1 line
2.45.2
PATCH v2 1/4 btrfs: remove unused Opt enums
The following three Opt_* enums are not utilized at all:
- Opt_ignorebadroots
- Opt_ignoredatacsums
- Opt_rescue_all
All those enums are from the old day where we have dedicated mount options, meanwhile nowadays they are all moved to "rescue=" mount option groups, and no more global token for them.
So we can safely remove them now.
--- fs/btrfs/super.c | 3 --- 1 file changed, 3 deletions(-)
fs/btrfs/super.c +0 -3
@@ -125,9 +125,6 @@ enum { Opt_rescue, Opt_usebackuproot, Opt_nologreplay, - Opt_ignorebadroots, - Opt_ignoredatacsums, - Opt_rescue_all,/* Debugging options */ Opt_enospc_debug,
Signature · 1 line
2.45.2
PATCH v2 2/4 btrfs: output the unrecognized super flags as hex
In reply to Qu WenruoMost of the extra suepr flags are beyond 32bits (from CHANGING_FSID_V2 to CHANGING_*_CSUMS), thus using %llu is not only too long and pretty hard to read.
--- fs/btrfs/disk-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/btrfs/disk-io.c +1 -1
@@ -2347,7 +2347,7 @@ int btrfs_validate_super(const struct btrfs_fs_info *fs_info, ret = -EINVAL ; }if (btrfs_super_flags (sb) & ~BTRFS_SUPER_FLAG_SUPP ) { -btrfs_err (fs_info,"unrecognized or unsupported super flag: %llu" , +btrfs_err (fs_info,"unrecognized or unsupported super flag: 0x%llx" ,btrfs_super_flags (sb) & ~BTRFS_SUPER_FLAG_SUPP ); ret = -EINVAL ; }
Signature · 1 line
2.45.2
PATCH v2 3/4 btrfs: introduce new "rescue=ignoremetacsums" mount option
In reply to Qu WenruoThis patch introduces "rescue=ignoremetacsums" to ignore metadata csums, meanwhile all the other metadata sanity checks are still kept as is.
This new mount option is mostly to allow the kernel to mount an interrupted checksum conversion (at the metadata csum overwrite stage).
And since the main part of metadata sanity checks is inside tree-checker, we shouldn't lose much safety, and the new mount option is rescue mount option it requires full read-only mount.
--- fs/btrfs/bio.c | 2 +- fs/btrfs/disk-io.c | 19 +++++++++++++------ fs/btrfs/file-item.c | 2 +- fs/btrfs/fs.h | 4 +++- fs/btrfs/messages.c | 3 ++- fs/btrfs/super.c | 13 ++++++++++++- fs/btrfs/sysfs.c | 1 + fs/btrfs/zoned.c | 2 +- 8 files changed, 34 insertions(+), 12 deletions(-)
fs/btrfs/bio.c +1 -1
@@ -732,7 +732,7 @@ static bool btrfs_submit_chunk(struct btrfs_bio *bbio, int mirror_num)* point, so they are handled as part of the no-checksum case. */ if (inode && !(inode->flags &BTRFS_INODE_NODATASUM ) && - !test_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state) && + !test_bit (BTRFS_FS_STATE_NO_DATA_CSUMS , &fs_info->fs_state) && !btrfs_is_data_reloc_root (inode->root)) {if (should_async_write (bbio) &&btrfs_wq_submit_bio (bbio, bioc, &smap, mirror_num))
fs/btrfs/disk-io.c +13 -6
@@ -367,6 +367,7 @@ int btrfs_validate_extent_buffer(struct extent_buffer *eb,u8 result[BTRFS_CSUM_SIZE ];const u8 *header_csum;int ret =0 ; +const bool ignore_csum =btrfs_test_opt (fs_info,IGNOREMETACSUMS );ASSERT (check);
@@ -399,13 +400,17 @@ int btrfs_validate_extent_buffer(struct extent_buffer *eb,if (memcmp (result, header_csum, csum_size) !=0 ) {btrfs_warn_rl (fs_info, -"checksum verify failed on logical %llu mirror %u wanted " CSUM_FMT " found " CSUM_FMT " level %d" , +"checksum verify failed on logical %llu mirror %u wanted " +CSUM_FMT " found " CSUM_FMT " level %d%s" , eb->start, eb->read_mirror,CSUM_FMT_VALUE (csum_size, header_csum),CSUM_FMT_VALUE (csum_size, result), -btrfs_header_level (eb)); - ret = -EUCLEAN ; - goto out; +btrfs_header_level (eb), + ignore_csum ?", ignoring" :"" ); +if (!ignore_csum) { + ret = -EUCLEAN ; + goto out; + } }if (found_level != check->level) {
@@ -2131,7 +2136,7 @@ static int load_global_roots_objectid(struct btrfs_root *tree_root,/* If we have IGNOREDATACSUMS skip loading these roots. */ if (objectid ==BTRFS_CSUM_TREE_OBJECTID &&btrfs_test_opt (fs_info,IGNOREDATACSUMS )) { -set_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state); +set_bit (BTRFS_FS_STATE_NO_DATA_CSUMS , &fs_info->fs_state);return 0 ; }
@@ -2184,7 +2189,7 @@ static int load_global_roots_objectid(struct btrfs_root *tree_root,if (!found || ret) {if (objectid ==BTRFS_CSUM_TREE_OBJECTID ) -set_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state); +set_bit (BTRFS_FS_STATE_NO_DATA_CSUMS , &fs_info->fs_state);if (!btrfs_test_opt (fs_info,IGNOREBADROOTS )) ret = ret ? ret : -ENOENT ;
@@ -2865,6 +2870,8 @@ static int init_mount_fs_info(struct btrfs_fs_info *fs_info, struct super_blockif (sb_rdonly (sb))set_bit (BTRFS_FS_STATE_RO , &fs_info->fs_state); +if (btrfs_test_opt (fs_info,IGNOREMETACSUMS )) +set_bit (BTRFS_FS_STATE_SKIP_META_CSUMS , &fs_info->fs_state);return btrfs_alloc_stripe_hash_table (fs_info); }
fs/btrfs/file-item.c +1 -1
@@ -353,7 +353,7 @@ blk_status_t btrfs_lookup_bio_sums(struct btrfs_bio *bbio)u32 bio_offset =0 ;if ((inode->flags &BTRFS_INODE_NODATASUM ) || -test_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state)) +test_bit (BTRFS_FS_STATE_NO_DATA_CSUMS , &fs_info->fs_state))return BLK_STS_OK ;/*
fs/btrfs/fs.h +3 -1
@@ -98,7 +98,8 @@ enum {/* The btrfs_fs_info created for self-tests */ BTRFS_FS_STATE_DUMMY_FS_INFO , -BTRFS_FS_STATE_NO_CSUMS , +BTRFS_FS_STATE_NO_DATA_CSUMS , +BTRFS_FS_STATE_SKIP_META_CSUMS ,/* Indicates there was an error cleaning up a log tree. */ BTRFS_FS_STATE_LOG_CLEANUP_ERROR,
@@ -224,6 +225,7 @@ enum {BTRFS_MOUNT_IGNOREDATACSUMS = (1UL <<28 ),BTRFS_MOUNT_NODISCARD = (1UL <<29 ),BTRFS_MOUNT_NOSPACECACHE = (1UL <<30 ), + BTRFS_MOUNT_IGNOREMETACSUMS = (1UL << 31), };/*
fs/btrfs/messages.c +2 -1
@@ -20,7 +20,8 @@ static const char fs_state_chars[] = { [BTRFS_FS_STATE_TRANS_ABORTED] = 'A', [BTRFS_FS_STATE_DEV_REPLACING] = 'R', [BTRFS_FS_STATE_DUMMY_FS_INFO] = 0, - [BTRFS_FS_STATE_NO_CSUMS] = 'C', + [BTRFS_FS_STATE_NO_DATA_CSUMS] = 'C', + [BTRFS_FS_STATE_SKIP_META_CSUMS] = 'S', [BTRFS_FS_STATE_LOG_CLEANUP_ERROR] = 'L', };
fs/btrfs/super.c +12 -1
@@ -175,6 +175,7 @@ enum { Opt_rescue_nologreplay, Opt_rescue_ignorebadroots, Opt_rescue_ignoredatacsums, + Opt_rescue_ignoremetacsums, Opt_rescue_parameter_all, };
@@ -184,7 +185,9 @@ static const struct constant_table btrfs_parameter_rescue[] = { { "ignorebadroots", Opt_rescue_ignorebadroots }, { "ibadroots", Opt_rescue_ignorebadroots }, { "ignoredatacsums", Opt_rescue_ignoredatacsums }, + { "ignoremetacsums", Opt_rescue_ignoremetacsums}, { "idatacsums", Opt_rescue_ignoredatacsums }, + { "imetacsums", Opt_rescue_ignoremetacsums}, { "all", Opt_rescue_parameter_all }, {} };
@@ -570,8 +573,12 @@ static int btrfs_parse_param(struct fs_context *fc, struct fs_parameter *param)case Opt_rescue_ignoredatacsums:btrfs_set_opt (ctx->mount_opt,IGNOREDATACSUMS );break ; +case Opt_rescue_ignoremetacsums: +btrfs_set_opt (ctx->mount_opt,IGNOREMETACSUMS ); +break ;case Opt_rescue_parameter_all:btrfs_set_opt (ctx->mount_opt,IGNOREDATACSUMS ); +btrfs_set_opt (ctx->mount_opt,IGNOREMETACSUMS );btrfs_set_opt (ctx->mount_opt,IGNOREBADROOTS );btrfs_set_opt (ctx->mount_opt,NOLOGREPLAY );break ;
@@ -646,7 +653,8 @@ bool btrfs_check_options(const struct btrfs_fs_info *info, unsigned long *mount_if (!(flags &SB_RDONLY ) && (check_ro_option (info, *mount_opt,BTRFS_MOUNT_NOLOGREPLAY ,"nologreplay" ) ||check_ro_option (info, *mount_opt,BTRFS_MOUNT_IGNOREBADROOTS ,"ignorebadroots" ) || -check_ro_option (info, *mount_opt,BTRFS_MOUNT_IGNOREDATACSUMS ,"ignoredatacsums" ))) +check_ro_option (info, *mount_opt,BTRFS_MOUNT_IGNOREDATACSUMS ,"ignoredatacsums" ) || +check_ro_option (info, *mount_opt,BTRFS_MOUNT_IGNOREMETACSUMS ,"ignoremetacsums" ))) ret = false; if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE) &&
@@ -1062,6 +1070,8 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)print_rescue_option (seq,"ignorebadroots" , &printed);if (btrfs_test_opt (info,IGNOREDATACSUMS ))print_rescue_option (seq,"ignoredatacsums" , &printed); +if (btrfs_test_opt (info,IGNOREMETACSUMS )) +print_rescue_option (seq,"ignoremetacsums" , &printed);if (btrfs_test_opt (info,FLUSHONCOMMIT ))seq_puts (seq,",flushoncommit" );if (btrfs_test_opt (info,DISCARD_SYNC ))
@@ -1419,6 +1429,7 @@ static void btrfs_emit_options(struct btrfs_fs_info *info,btrfs_info_if_set (info, old,USEBACKUPROOT ,"trying to use backup root at mount time" );btrfs_info_if_set (info, old,IGNOREBADROOTS ,"ignoring bad roots" );btrfs_info_if_set (info, old,IGNOREDATACSUMS ,"ignoring data csums" ); +btrfs_info_if_set (info, old,IGNOREMETACSUMS ,"ignoring meta csums" );btrfs_info_if_unset (info, old,NODATACOW ,"setting datacow" );btrfs_info_if_unset (info, old,SSD ,"not using ssd optimizations" );
fs/btrfs/sysfs.c +1 -0
@@ -385,6 +385,7 @@ static const char *rescue_opts[] = {"nologreplay" ,"ignorebadroots" ,"ignoredatacsums" , +"ignoremetacsums" , "all", };
fs/btrfs/zoned.c +1 -1
@@ -1844,7 +1844,7 @@ void btrfs_finish_ordered_zoned(struct btrfs_ordered_extent *ordered)* here so that we don't attempt to log the csums later. */ if ((inode->flags &BTRFS_INODE_NODATASUM ) || -test_bit (BTRFS_FS_STATE_NO_CSUMS , &fs_info->fs_state)) { +test_bit (BTRFS_FS_STATE_NO_DATA_CSUMS , &fs_info->fs_state)) {while ((sum =list_first_entry_or_null (&ordered->list,typeof (*sum), list))) {list_del (&sum->list);
Signature · 1 line
2.45.2
PATCH v2 4/4 btrfs: introduce new "rescue=ignoresuperflags" mount option
In reply to Qu WenruoThis new mount option would allow the kernel to skip the super flags check, it's mostly to allow the kernel to do a rescue mount of an interrupted checksum conversion.
--- fs/btrfs/disk-io.c | 16 ++++++++++++---- fs/btrfs/fs.h | 1 + fs/btrfs/super.c | 13 ++++++++++++- fs/btrfs/sysfs.c | 1 + 4 files changed, 26 insertions(+), 5 deletions(-)
fs/btrfs/disk-io.c +12 -4
@@ -2346,15 +2346,23 @@ int btrfs_validate_super(const struct btrfs_fs_info *fs_info,u64 nodesize =btrfs_super_nodesize (sb);u64 sectorsize =btrfs_super_sectorsize (sb);int ret =0 ; +const bool ignore_flags =btrfs_test_opt (fs_info,IGNORESUPERFLAGS );if (btrfs_super_magic (sb) !=BTRFS_MAGIC ) {btrfs_err (fs_info,"no valid FS found" ); ret = -EINVAL ; } -if (btrfs_super_flags (sb) & ~BTRFS_SUPER_FLAG_SUPP ) { -btrfs_err (fs_info,"unrecognized or unsupported super flag: 0x%llx" , -btrfs_super_flags (sb) & ~BTRFS_SUPER_FLAG_SUPP ); - ret = -EINVAL ; +if ((btrfs_super_flags (sb) & ~BTRFS_SUPER_FLAG_SUPP )) { +if (!ignore_flags) { +btrfs_err (fs_info, +"unrecognized or unsupported super flag: 0x%llx" , +btrfs_super_flags (sb) & ~BTRFS_SUPER_FLAG_SUPP ); + ret = -EINVAL ; + }else { +btrfs_info (fs_info, +"unrecognized or unsupported super flags: 0x%llx, ignoring" , +btrfs_super_flags (sb) & ~BTRFS_SUPER_FLAG_SUPP ); + } }if (btrfs_super_root_level (sb) >=BTRFS_MAX_LEVEL ) { btrfs_err(fs_info, "tree_root level too big: %d >= %d",
fs/btrfs/fs.h +1 -0
@@ -226,6 +226,7 @@ enum {BTRFS_MOUNT_NODISCARD = (1UL <<29 ),BTRFS_MOUNT_NOSPACECACHE = (1UL <<30 ),BTRFS_MOUNT_IGNOREMETACSUMS = (1UL <<31 ), + BTRFS_MOUNT_IGNORESUPERFLAGS = (1UL << 32), };/*
fs/btrfs/super.c +12 -1
@@ -176,6 +176,7 @@ enum { Opt_rescue_ignorebadroots, Opt_rescue_ignoredatacsums, Opt_rescue_ignoremetacsums, + Opt_rescue_ignoresuperflags, Opt_rescue_parameter_all, };
@@ -186,8 +187,10 @@ static const struct constant_table btrfs_parameter_rescue[] = { { "ibadroots", Opt_rescue_ignorebadroots }, { "ignoredatacsums", Opt_rescue_ignoredatacsums }, { "ignoremetacsums", Opt_rescue_ignoremetacsums}, + { "ignoresuperflags", Opt_rescue_ignoresuperflags}, { "idatacsums", Opt_rescue_ignoredatacsums }, { "imetacsums", Opt_rescue_ignoremetacsums}, + { "isuperflags", Opt_rescue_ignoresuperflags}, { "all", Opt_rescue_parameter_all }, {} };
@@ -576,9 +579,13 @@ static int btrfs_parse_param(struct fs_context *fc, struct fs_parameter *param)case Opt_rescue_ignoremetacsums:btrfs_set_opt (ctx->mount_opt,IGNOREMETACSUMS );break ; +case Opt_rescue_ignoresuperflags: +btrfs_set_opt (ctx->mount_opt,IGNORESUPERFLAGS ); +break ;case Opt_rescue_parameter_all:btrfs_set_opt (ctx->mount_opt,IGNOREDATACSUMS );btrfs_set_opt (ctx->mount_opt,IGNOREMETACSUMS ); +btrfs_set_opt (ctx->mount_opt,IGNORESUPERFLAGS );btrfs_set_opt (ctx->mount_opt,IGNOREBADROOTS );btrfs_set_opt (ctx->mount_opt,NOLOGREPLAY );break ;
@@ -654,7 +661,8 @@ bool btrfs_check_options(const struct btrfs_fs_info *info, unsigned long *mount_ (check_ro_option (info, *mount_opt,BTRFS_MOUNT_NOLOGREPLAY ,"nologreplay" ) ||check_ro_option (info, *mount_opt,BTRFS_MOUNT_IGNOREBADROOTS ,"ignorebadroots" ) ||check_ro_option (info, *mount_opt,BTRFS_MOUNT_IGNOREDATACSUMS ,"ignoredatacsums" ) || - check_ro_option(info, *mount_opt, BTRFS_MOUNT_IGNOREMETACSUMS, "ignoremetacsums"))) +check_ro_option (info, *mount_opt,BTRFS_MOUNT_IGNOREMETACSUMS ,"ignoremetacsums" ) || + check_ro_option(info, *mount_opt, BTRFS_MOUNT_IGNORESUPERFLAGS, "ignoresuperflags"))) ret = false; if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE) &&
@@ -1072,6 +1080,8 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)print_rescue_option (seq,"ignoredatacsums" , &printed);if (btrfs_test_opt (info,IGNOREMETACSUMS ))print_rescue_option (seq,"ignoremetacsums" , &printed); +if (btrfs_test_opt (info,IGNORESUPERFLAGS )) +print_rescue_option (seq,"ignoresuperflags" , &printed);if (btrfs_test_opt (info,FLUSHONCOMMIT ))seq_puts (seq,",flushoncommit" );if (btrfs_test_opt (info,DISCARD_SYNC ))
@@ -1430,6 +1440,7 @@ static void btrfs_emit_options(struct btrfs_fs_info *info,btrfs_info_if_set (info, old,IGNOREBADROOTS ,"ignoring bad roots" );btrfs_info_if_set (info, old,IGNOREDATACSUMS ,"ignoring data csums" );btrfs_info_if_set (info, old,IGNOREMETACSUMS ,"ignoring meta csums" ); +btrfs_info_if_set (info, old,IGNORESUPERFLAGS ,"ignoring unknown super flags" );btrfs_info_if_unset (info, old,NODATACOW ,"setting datacow" );btrfs_info_if_unset (info, old,SSD ,"not using ssd optimizations" );
fs/btrfs/sysfs.c +1 -0
@@ -386,6 +386,7 @@ static const char *rescue_opts[] = {"ignorebadroots" ,"ignoredatacsums" ,"ignoremetacsums" , +"ignoresuperflags" , "all", };
Signature · 1 line
2.45.2
Re: [PATCH v2 4/4] btrfs: introduce new "rescue=ignoresuperflags" mount option
Hi Qu,
Quoted 6 lines · Qu Wenruo
This new mount option would allow the kernel to skip the super flags check, it's mostly to allow the kernel to do a rescue mount of an interrupted checksum conversion.
Reviewed-by Josef Bacik <[email protected]>Signed-off-by Qu Wenruo <[email protected]>
Thanks for your patch, which is now commit cbc2bc70b8c22974 ("btrfs:
introduce new "rescue=ignoresuperflags" mount option") in next-20240709.
fs/btrfs/fs.h enum { · line 229
Quoted 7 lines
fs/btrfs/fs.h +1
-0@@ -226,6 +226,7 @@ enum {BTRFS_MOUNT_NODISCARD = (1UL <<29 ),BTRFS_MOUNT_NOSPACECACHE = (1UL <<30 ),BTRFS_MOUNT_IGNOREMETACSUMS = (1UL <<31 ), + BTRFS_MOUNT_IGNORESUPERFLAGS = (1UL << 32),
"1UL" was changed to "1ULL" while applying, but that is not sufficient, as all other mount flags handling still operates on "unsigned long", which is 32-bit ont 32-bit platforms.
Hence [email protected] reported several build failures on 32-bit platforms (e.g. m68k[1]):
fs/btrfs/super.c:666:48: error: conversion from 'enum <anonymous>' to 'long unsigned int' changes value from '4294967296' to '0' [-Werror=overflow]
[1] http://kisskb.ellerman.id.au/kisskb/buildresult/15197832/
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds