btrfs-progs: recover from failed metadata_uuid
cover btrfs-progs: recover from failed metadata_uuid
The kernel reunites split-brained devices after a failed btrfstune -m|M operation. We can achieve the same in btrfs-progs. So port it here. Ref the discussion here:
https://lore.kernel.org/all/[email protected]/
Patch 1/16 wasn't integrated as part of the set
[PATCH 00/10 v2] fixes and preparatory related to metadata_uuid
it's now merged with this patchset.
Patches [2-6,11,12] are cleanup patches.
Patches [7,8,10] are preparatory.
Patch [9] addresses a bug.
Patches [13, 14, 15] provide recovery from previously failed btrfstune -m|M operations.
Patch [16] enhances the misc-test 034-metadata-uuid to also validate this new recovery feature.
This set has been successfully tested with the btrfs-progs testsuite.
This patchset is on top the latest devel last commit:
8aba9b0052b6 btrfs-progs: btrfstune: consolidate error handling in main()
Anand Jain (16):
btrfs-progs: track num_devices per fs_devices
btrfs-progs: tune can use local fs_info variable
btrfs-progs: rename set_metadata_uuid arg to new_fsid_str
btrfs-progs: rename set_metadata_uuid new_fsid to fsid
btrfs-progs: rename set_metadata_uuid new_uuid to new_fsid
btrfs-progs: rename set_metadata_uuid uuid_changed to fsid_changed
btrfs-progs: pass fsid in check_unfinished_fsid_change arg2
btrfs-progs: pass metadata_uuid in check_unfinished_fsid_change arg3
btrfs-progs: fix return without flag reset commit in tune
btrfs-progs: preparing the latest device's superblock for commit
btrfs-progs: rename fs_devices::list to match the kernel
btrfs-progs: rename fs_devices::latest_trans to match the kernel
btrfs-progs: tune use the latest bdev in fs_devices for super_copy
btrfs-progs: add support to fix superblock with CHANGING_FSID_V2 flag
btrfs-progs: recover from the failed btrfstune -m|M
btrfs-progs: test btrfstune -m|M ability to fix previous failures
cmds/filesystem.c | 14 +- common/device-scan.c | 2 +- kernel-shared/disk-io.c | 3 + kernel-shared/disk-io.h | 5 + kernel-shared/volumes.c | 203 +++++++++++++++++++-- kernel-shared/volumes.h | 5 +- tests/misc-tests/034-metadata-uuid/test.sh | 70 +++++-- tune/change-metadata-uuid.c | 56 +++--- tune/main.c | 35 ++-- 9 files changed, 312 insertions(+), 81 deletions(-)
Signature · 1 line
2.39.3
PATCH 1/16 btrfs-progs: track num_devices per fs_devices
Similar to the kernel we need to track the number of devices scanned per fs_devices. A preparation patch to fix incomplete fsid changing.
--- kernel-shared/volumes.c | 1 + kernel-shared/volumes.h | 1 + 2 files changed, 2 insertions(+)
kernel-shared/volumes.c +1 -0
@@ -406,6 +406,7 @@ static int device_list_add(const char *path,btrfs_stack_device_bytes_used (&disk_super->dev_item);list_add (&device->dev_list, &fs_devices->devices); device->fs_devices = fs_devices; + fs_devices->num_devices++; }else if (!device->name ||strcmp (device->name, path)) {char *name;
kernel-shared/volumes.h +1 -0
@@ -88,6 +88,7 @@ struct btrfs_fs_devices {u64 latest_trans;u64 lowest_devid; +u64 num_devices;u64 missing_devices;u64 total_rw_bytes;
Signature · 1 line
2.39.3
PATCH 2/16 btrfs-progs: tune can use local fs_info variable
In reply to Anand JainSince the root pointer dereferences for the fs_info several times, it is rational to save the fs_info.
--- tune/main.c | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-)
tune/main.c +17 -15
@@ -133,6 +133,7 @@ static const struct cmd_struct tune_cmd = {int BOX_MAIN (btrfstune )(int argc,char *argv[]) {struct btrfs_root *root; +struct btrfs_fs_info *fs_info;unsigned ctree_flags =OPEN_CTREE_WRITES ;int success =0 ;int total =0 ;
@@ -296,6 +297,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) ret =1 ; goto free_out; } + fs_info = root->fs_info;/* * As we increment the generation number here, it is unlikely that the
@@ -309,9 +311,9 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])* all the partner devices. */ if ((change_metadata_uuid || random_fsid || new_fsid_str) && - root->fs_info->fs_devices->missing_devices >0 ) { + fs_info->fs_devices->missing_devices >0 ) {error ("missing %lld device(s), failing the command" , - root->fs_info->fs_devices->missing_devices); + fs_info->fs_devices->missing_devices); ret =1 ; goto out; }
@@ -322,17 +324,17 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) ret =1 ; goto out; } -if (btrfs_fs_compat_ro (root->fs_info,BLOCK_GROUP_TREE )) { +if (btrfs_fs_compat_ro (fs_info,BLOCK_GROUP_TREE )) {error ("the filesystem already has block group tree feature" ); ret =1 ; goto out; } -if (!btrfs_fs_compat_ro (root->fs_info,FREE_SPACE_TREE_VALID )) { +if (!btrfs_fs_compat_ro (fs_info,FREE_SPACE_TREE_VALID )) {error ("the filesystem doesn't have space cache v2, needs to be mounted with \"-o space_cache=v2\" first" ); ret =1 ; goto out; } - ret =convert_to_bg_tree (root->fs_info); + ret =convert_to_bg_tree (fs_info);if (ret <0 ) {error ("failed to convert the filesystem to block group tree feature" ); goto out;
@@ -340,12 +342,12 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) goto out; }if (to_fst) { -if (btrfs_fs_compat_ro (root->fs_info,FREE_SPACE_TREE_VALID )) { +if (btrfs_fs_compat_ro (fs_info,FREE_SPACE_TREE_VALID )) {error ("filesystem already has free-space-tree feature" ); ret =1 ; goto out; } - ret =convert_to_fst (root->fs_info); + ret =convert_to_fst (fs_info);if (ret <0 )error ("failed to convert the filesystem to free-space-tree feature" ); goto out;
@@ -356,12 +358,12 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) ret =1 ; goto out; } -if (!btrfs_fs_compat_ro (root->fs_info,BLOCK_GROUP_TREE )) { +if (!btrfs_fs_compat_ro (fs_info,BLOCK_GROUP_TREE )) {error ("filesystem doesn't have block-group-tree feature" ); ret =1 ; goto out; } - ret =convert_to_extent_tree (root->fs_info); + ret =convert_to_extent_tree (fs_info);if (ret <0 ) {error ("failed to convert the filesystem from block group tree feature" ); goto out;
@@ -369,7 +371,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) goto out; }if (seeding_flag) { -if (btrfs_fs_incompat (root->fs_info,METADATA_UUID )) { +if (btrfs_fs_incompat (fs_info,METADATA_UUID )) {error ("SEED flag cannot be changed on a metadata-uuid changed fs" ); ret =1 ; goto out;
@@ -402,7 +404,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])if (csum_type !=-1 ) {/* TODO: check conflicting flags */ pr_verbose (LOG_DEFAULT ,"Proceed to switch checksums\n" ); - ret =btrfs_change_csum_type (root->fs_info, csum_type); + ret =btrfs_change_csum_type (fs_info, csum_type); }if (change_metadata_uuid) {
@@ -424,8 +426,8 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) }if (random_fsid || (new_fsid_str && !change_metadata_uuid)) { -if (btrfs_fs_incompat (root->fs_info,METADATA_UUID ) || - root->fs_info->fs_devices->active_metadata_uuid) { +if (btrfs_fs_incompat (fs_info,METADATA_UUID ) || + fs_info->fs_devices->active_metadata_uuid) {error ("Cannot rewrite fsid while METADATA_UUID flag is active. \n" "Ensure fsid and metadata_uuid match before retrying." );
@@ -445,7 +447,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) goto out; } } - ret =change_uuid (root->fs_info, new_fsid_str); + ret =change_uuid (fs_info, new_fsid_str);if (!ret) success++; total++;
@@ -454,7 +456,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])if (success == total) { ret =0 ; }else { - root->fs_info->readonly =1 ; + fs_info->readonly =1 ; ret =1 ;error ("btrfstune failed" ); }
Signature · 1 line
2.39.3
PATCH 3/16 btrfs-progs: rename set_metadata_uuid arg to new_fsid_str
In reply to Anand JainIn preparation to use check_unfinished_fsid_change() to support the ability to reunite devices after a failed 'btrfstune -m|M' command, %uuid_string arg is actually carries new fsid to be used. So just name it to %new_fsid_str.
--- tune/change-metadata-uuid.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tune/change-metadata-uuid.c +3 -3
@@ -24,7 +24,7 @@#include "common/messages.h" #include "tune/tune.h" -int set_metadata_uuid (struct btrfs_root *root,const char *uuid_string) +int set_metadata_uuid (struct btrfs_root *root,const char *new_fsid_string) {struct btrfs_super_block *disk_super;uuid_t new_fsid, unused1, unused2;
@@ -50,8 +50,8 @@ int set_metadata_uuid(struct btrfs_root *root, const char *uuid_string)return 1 ; } -if (uuid_string) -uuid_parse (uuid_string, new_fsid); +if (new_fsid_string) +uuid_parse (new_fsid_string, new_fsid);else uuid_generate (new_fsid);
Signature · 1 line
2.39.3
PATCH 4/16 btrfs-progs: rename set_metadata_uuid new_fsid to fsid
In reply to Anand JainThe %new_fsid is not only new it can be the fsid from the passed disk so just rename it to %fsid. Also, in the next patch the %new_fsid will be a bool variable to indicate if the %fsid is new from the fsid in the disk.
--- tune/change-metadata-uuid.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
tune/change-metadata-uuid.c +9 -9
@@ -27,7 +27,7 @@int set_metadata_uuid (struct btrfs_root *root,const char *new_fsid_string) {struct btrfs_super_block *disk_super; -uuid_t new_fsid, unused1, unused2; +uuid_t fsid, unused1, unused2;struct btrfs_trans_handle *trans;bool new_uuid = true;u64 incompat_flags;
@@ -51,11 +51,11 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string) }if (new_fsid_string) -uuid_parse (new_fsid_string, new_fsid); +uuid_parse (new_fsid_string, fsid);else -uuid_generate (new_fsid); +uuid_generate (fsid); - new_uuid = (memcmp (new_fsid, disk_super->fsid,BTRFS_FSID_SIZE ) !=0 ); + new_uuid = (memcmp (fsid, disk_super->fsid,BTRFS_FSID_SIZE ) !=0 );/* Step 1 sets the in progress flag */ trans =btrfs_start_transaction (root,1 );
@@ -66,23 +66,23 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)return ret;if (new_uuid && uuid_changed &&memcmp (disk_super->metadata_uuid, - new_fsid,BTRFS_FSID_SIZE ) ==0 ) { + fsid,BTRFS_FSID_SIZE ) ==0 ) {/* * Changing fsid to be the same as metadata uuid, so just * disable the flag */ -memcpy (disk_super->fsid, &new_fsid,BTRFS_FSID_SIZE ); +memcpy (disk_super->fsid, &fsid,BTRFS_FSID_SIZE ); incompat_flags &= ~BTRFS_FEATURE_INCOMPAT_METADATA_UUID ;btrfs_set_super_incompat_flags (disk_super, incompat_flags);memset (disk_super->metadata_uuid,0 ,BTRFS_FSID_SIZE ); }else if (new_uuid && uuid_changed &&memcmp (disk_super->metadata_uuid, - new_fsid,BTRFS_FSID_SIZE )) { + fsid,BTRFS_FSID_SIZE )) {/* * Changing fsid on an already changed FS, in this case we * only change the fsid and don't touch metadata uuid as it * has already the correct value */ -memcpy (disk_super->fsid, &new_fsid,BTRFS_FSID_SIZE ); +memcpy (disk_super->fsid, &fsid,BTRFS_FSID_SIZE ); }else if (new_uuid && !uuid_changed) {/* * First time changing the fsid, copy the fsid to metadata_uuid
@@ -91,7 +91,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)btrfs_set_super_incompat_flags (disk_super, incompat_flags);memcpy (disk_super->metadata_uuid, disk_super->fsid,BTRFS_FSID_SIZE ); -memcpy (disk_super->fsid, &new_fsid,BTRFS_FSID_SIZE ); +memcpy (disk_super->fsid, &fsid,BTRFS_FSID_SIZE ); }else {/* Setting the same fsid as current, do nothing */ return 0 ;
Signature · 1 line
2.39.3
PATCH 5/16 btrfs-progs: rename set_metadata_uuid new_uuid to new_fsid
In reply to Anand Jain%new_uuid is being used to say there is a new fsid. So why not just call it %new_fsid.
--- tune/change-metadata-uuid.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tune/change-metadata-uuid.c +5 -5
@@ -29,7 +29,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)struct btrfs_super_block *disk_super;uuid_t fsid, unused1, unused2;struct btrfs_trans_handle *trans; -bool new_uuid = true; +bool new_fsid = true;u64 incompat_flags;bool uuid_changed;u64 super_flags;
@@ -55,7 +55,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)else uuid_generate (fsid); - new_uuid = (memcmp (fsid, disk_super->fsid,BTRFS_FSID_SIZE ) !=0 ); + new_fsid = (memcmp (fsid, disk_super->fsid,BTRFS_FSID_SIZE ) !=0 );/* Step 1 sets the in progress flag */ trans =btrfs_start_transaction (root,1 );
@@ -65,7 +65,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)if (ret <0 )return ret; -if (new_uuid && uuid_changed &&memcmp (disk_super->metadata_uuid, +if (new_fsid && uuid_changed &&memcmp (disk_super->metadata_uuid, fsid,BTRFS_FSID_SIZE ) ==0 ) {/* * Changing fsid to be the same as metadata uuid, so just
@@ -75,7 +75,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string) incompat_flags &= ~BTRFS_FEATURE_INCOMPAT_METADATA_UUID ;btrfs_set_super_incompat_flags (disk_super, incompat_flags);memset (disk_super->metadata_uuid,0 ,BTRFS_FSID_SIZE ); - }else if (new_uuid && uuid_changed &&memcmp (disk_super->metadata_uuid, + }else if (new_fsid && uuid_changed &&memcmp (disk_super->metadata_uuid, fsid,BTRFS_FSID_SIZE )) {/* * Changing fsid on an already changed FS, in this case we
@@ -83,7 +83,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)* has already the correct value */ memcpy (disk_super->fsid, &fsid,BTRFS_FSID_SIZE ); - }else if (new_uuid && !uuid_changed) { + }else if (new_fsid && !uuid_changed) {/* * First time changing the fsid, copy the fsid to metadata_uuid */
Signature · 1 line
2.39.3
PATCH 6/16 btrfs-progs: rename set_metadata_uuid uuid_changed to fsid_changed
In reply to Anand JainWe never change the metadata_uuid; we only change the fsid. So '%fsid_changed' flows more appropriately than '%uuid_changed'.
--- tune/change-metadata-uuid.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tune/change-metadata-uuid.c +5 -5
@@ -31,14 +31,14 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)struct btrfs_trans_handle *trans;bool new_fsid = true;u64 incompat_flags; -bool uuid_changed; +bool fsid_changed;u64 super_flags;int ret; disk_super = root->fs_info->super_copy; super_flags =btrfs_super_flags (disk_super); incompat_flags =btrfs_super_incompat_flags (disk_super); - uuid_changed = incompat_flags &BTRFS_FEATURE_INCOMPAT_METADATA_UUID ; + fsid_changed = incompat_flags &BTRFS_FEATURE_INCOMPAT_METADATA_UUID ;if (super_flags &BTRFS_SUPER_FLAG_SEEDING ) {error ("cannot set metadata UUID on a seed device" );
@@ -65,7 +65,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)if (ret <0 )return ret; -if (new_fsid && uuid_changed &&memcmp (disk_super->metadata_uuid, +if (new_fsid && fsid_changed &&memcmp (disk_super->metadata_uuid, fsid,BTRFS_FSID_SIZE ) ==0 ) {/* * Changing fsid to be the same as metadata uuid, so just
@@ -75,7 +75,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string) incompat_flags &= ~BTRFS_FEATURE_INCOMPAT_METADATA_UUID ;btrfs_set_super_incompat_flags (disk_super, incompat_flags);memset (disk_super->metadata_uuid,0 ,BTRFS_FSID_SIZE ); - }else if (new_fsid && uuid_changed &&memcmp (disk_super->metadata_uuid, + }else if (new_fsid && fsid_changed &&memcmp (disk_super->metadata_uuid, fsid,BTRFS_FSID_SIZE )) {/* * Changing fsid on an already changed FS, in this case we
@@ -83,7 +83,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)* has already the correct value */ memcpy (disk_super->fsid, &fsid,BTRFS_FSID_SIZE ); - }else if (new_fsid && !uuid_changed) { + }else if (new_fsid && !fsid_changed) {/* * First time changing the fsid, copy the fsid to metadata_uuid */
Signature · 1 line
2.39.3
PATCH 7/16 btrfs-progs: pass fsid in check_unfinished_fsid_change arg2
In reply to Anand JainIn preparation to use check_unfinished_fsid_change() to support the ability to reunite devices after a failed 'btrfstune -m|M' command, delete unused1 argument instead reuse %fsid as the function check_unfinished_fsid_change() returns the fsid.
--- tune/change-metadata-uuid.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tune/change-metadata-uuid.c +2 -2
@@ -27,7 +27,7 @@int set_metadata_uuid (struct btrfs_root *root,const char *new_fsid_string) {struct btrfs_super_block *disk_super; -uuid_t fsid, unused1, unused2; +uuid_t fsid, unused2;struct btrfs_trans_handle *trans;bool new_fsid = true;u64 incompat_flags;
@@ -45,7 +45,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)return 1 ; } -if (check_unfinished_fsid_change (root->fs_info, unused1, unused2)) { +if (check_unfinished_fsid_change (root->fs_info, fsid, unused2)) {error ("UUID rewrite in progress, cannot change metadata_uuid" );return 1 ; }
Signature · 1 line
2.39.3
PATCH 8/16 btrfs-progs: pass metadata_uuid in check_unfinished_fsid_change arg3
In reply to Anand JainIn preparation to use check_unfinished_fsid_change() to support the ability to reunite devices after a failed 'btrfstune -m|M' command, rename %unused2 to %metadata_uuid as the function check_unfinished_fsid_change() write the metadata_uuid from the ctree to it.
--- tune/change-metadata-uuid.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tune/change-metadata-uuid.c +2 -2
@@ -27,7 +27,7 @@int set_metadata_uuid (struct btrfs_root *root,const char *new_fsid_string) {struct btrfs_super_block *disk_super; -uuid_t fsid, unused2; +uuid_t fsid, metadata_uuid;struct btrfs_trans_handle *trans;bool new_fsid = true;u64 incompat_flags;
@@ -45,7 +45,7 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)return 1 ; } -if (check_unfinished_fsid_change (root->fs_info, fsid, unused2)) { +if (check_unfinished_fsid_change (root->fs_info, fsid, metadata_uuid)) {error ("UUID rewrite in progress, cannot change metadata_uuid" );return 1 ; }
Signature · 1 line
2.39.3
PATCH 9/16 btrfs-progs: fix return without flag reset commit in tune
In reply to Anand JainIn the function set_metadata_uuid(), we set the flag BTRFS_SUPER_FLAG_CHANGING_FSID_V2 in step 1 at line 71 as shown below:
71 super_flags |= BTRFS_SUPER_FLAG_CHANGING_FSID_V2; 72 btrfs_set_super_flags(disk_super, super_flags); 73 ret = btrfs_commit_transaction(trans, root);
However, we fail to reset this flag if there is no change in the fsid on the incoming disks, as we return too early.
105 } else {
106 /* Setting the same fsid as current, do nothing */
107 return 0;
Fix this by allowing the thread to pass through the step 2, where we reset the flag.
--- tune/change-metadata-uuid.c | 3 --- 1 file changed, 3 deletions(-)
tune/change-metadata-uuid.c +0 -3
@@ -92,9 +92,6 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string)memcpy (disk_super->metadata_uuid, disk_super->fsid,BTRFS_FSID_SIZE );memcpy (disk_super->fsid, &fsid,BTRFS_FSID_SIZE ); - }else { -/* Setting the same fsid as current, do nothing */ -return 0 ; } trans =btrfs_start_transaction (root,1 );
Signature · 1 line
2.39.3
PATCH 10/16 btrfs-progs: preparing the latest device's superblock for commit
In reply to Anand JainThis patch provides a flag to copy the superblock of the latest device to the fs_info::super_copy for the commit process, rather than using the superblock from the device specified in the argument.
This serves as groundwork to enable recovery from an incomplete btrfstune -M|m|u|U operation.
--- kernel-shared/disk-io.c | 3 +++ kernel-shared/disk-io.h | 5 +++++ 2 files changed, 8 insertions(+)
kernel-shared/disk-io.c +3 -0
@@ -1513,6 +1513,9 @@ static struct btrfs_fs_info *__open_ctree_fd(int fp, struct open_ctree_args *ocaif (flags &OPEN_CTREE_RECOVER_SUPER ) ret =btrfs_read_dev_super (fs_devices->latest_bdev, disk_super, sb_bytenr,SBREAD_RECOVER ); +else if (flags &OPEN_CTREE_USE_LATEST_BDEV ) + ret =btrfs_read_dev_super (fs_devices->latest_bdev, disk_super, + sb_bytenr, sbflags);else ret =btrfs_read_dev_super (fp, disk_super, sb_bytenr, sbflags);
kernel-shared/disk-io.h +5 -0
@@ -104,6 +104,11 @@ enum btrfs_open_ctree_flags {* specific checks and only do the superficial checks. */ OPEN_CTREE_SKIP_LEAF_ITEM_CHECKS = (1U <<17 ), + +/* +* Use the superblock of the latest device for the transaction commit. +*/ + OPEN_CTREE_USE_LATEST_BDEV = (1U << 18), };/*
Signature · 1 line
2.39.3
PATCH 11/16 btrfs-progs: rename fs_devices::list to match the kernel
In reply to Anand JainAligning with the kernel's struct btrfs_fs_devices:fs_list, rename btrfs_fs_devices::list to btrfs_fs_devices::fs_list.
--- cmds/filesystem.c | 14 +++++++------- common/device-scan.c | 2 +- kernel-shared/volumes.c | 12 ++++++------ kernel-shared/volumes.h | 2 +- 4 files changed, 15 insertions(+), 15 deletions(-)
cmds/filesystem.c +7 -7
@@ -485,7 +485,7 @@ static void free_fs_devices(struct btrfs_fs_devices *fs_devices) cur_seed = next_seed; } -list_del (&fs_devices->list); +list_del (&fs_devices->fs_list);free (fs_devices); }
@@ -555,7 +555,7 @@ static int find_and_copy_seed(struct btrfs_fs_devices *seed, struct list_head *fs_uuids) {struct btrfs_fs_devices *cur_fs; -list_for_each_entry (cur_fs, fs_uuids, list) +list_for_each_entry (cur_fs, fs_uuids, fs_list)if (!memcmp (seed->fsid, cur_fs->fsid,BTRFS_FSID_SIZE ))return copy_fs_devices (copy, cur_fs);
@@ -591,7 +591,7 @@ static int search_umounted_fs_uuids(struct list_head *all_uuids,* The fs_uuids list is global, and open_ctree_* will * modify it, make a private copy here */ -list_for_each_entry (cur_fs, fs_uuids, list) { +list_for_each_entry (cur_fs, fs_uuids, fs_list) {/* don't bother handle all fs, if search target specified */ if (search) {if (uuid_search (cur_fs, search) ==0 )
@@ -616,7 +616,7 @@ static int search_umounted_fs_uuids(struct list_head *all_uuids, goto out; } -list_add (&fs_copy->list, all_uuids); +list_add (&fs_copy->fs_list, all_uuids); } out:
@@ -635,7 +635,7 @@ static int map_seed_devices(struct list_head *all_uuids) fs_uuids =btrfs_scanned_uuids (); -list_for_each_entry (cur_fs, all_uuids, list) { +list_for_each_entry (cur_fs, all_uuids, fs_list) {struct open_ctree_args oca = {0 }; device = list_first_entry(&cur_fs->devices,
@@ -837,7 +837,7 @@ devs_only: goto out; } -list_for_each_entry (fs_devices, &all_uuids, list) +list_for_each_entry (fs_devices, &all_uuids, fs_list)print_one_uuid (fs_devices, unit_mode);if (search && !found) {
@@ -846,7 +846,7 @@ devs_only: }while (!list_empty (&all_uuids)) { fs_devices =list_entry (all_uuids.next, - struct btrfs_fs_devices, list); + struct btrfs_fs_devices, fs_list);free_fs_devices (fs_devices); } out:
common/device-scan.c +1 -1
@@ -257,7 +257,7 @@ int btrfs_register_all_devices(void) all_uuids =btrfs_scanned_uuids (); -list_for_each_entry (fs_devices, all_uuids, list) { +list_for_each_entry (fs_devices, all_uuids, fs_list) {list_for_each_entry (device, &fs_devices->devices, dev_list) {if (*device->name) err =btrfs_register_one_device (device->name);
kernel-shared/volumes.c +6 -6
@@ -319,7 +319,7 @@ static struct btrfs_fs_devices *find_fsid(u8 *fsid, u8 *metadata_uuid) {struct btrfs_fs_devices *fs_devices; -list_for_each_entry (fs_devices, &fs_uuids, list) { +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { if (metadata_uuid && (memcmp(fsid, fs_devices->fsid, BTRFS_FSID_SIZE) == 0) && (memcmp(metadata_uuid, fs_devices->metadata_uuid,
@@ -357,7 +357,7 @@ static int device_list_add(const char *path,if (!fs_devices)return -ENOMEM ;INIT_LIST_HEAD (&fs_devices->devices); -list_add (&fs_devices->list, &fs_uuids); +list_add (&fs_devices->fs_list, &fs_uuids);memcpy (fs_devices->fsid, disk_super->fsid,BTRFS_FSID_SIZE );if (metadata_uuid) memcpy(fs_devices->metadata_uuid,
@@ -489,11 +489,11 @@ again: orig = fs_devices; fs_devices = seed_devices; -list_del (&orig->list); +list_del (&orig->fs_list);free (orig); goto again; }else { -list_del (&fs_devices->list); +list_del (&fs_devices->fs_list);free (fs_devices); }
@@ -506,7 +506,7 @@ void btrfs_close_all_devices(void)while (!list_empty (&fs_uuids)) { fs_devices = list_entry(fs_uuids.next, struct btrfs_fs_devices, - list); + fs_list);btrfs_close_devices (fs_devices); } }
@@ -2227,7 +2227,7 @@ static int open_seed_devices(struct btrfs_fs_info *fs_info, u8 *fsid) goto out; }INIT_LIST_HEAD (&fs_devices->devices); -list_add (&fs_devices->list, &fs_uuids); +list_add (&fs_devices->fs_list, &fs_uuids);memcpy (fs_devices->fsid, fsid,BTRFS_FSID_SIZE ); }
kernel-shared/volumes.h +1 -1
@@ -96,7 +96,7 @@ struct btrfs_fs_devices {int latest_bdev;int lowest_bdev;struct list_head devices; -struct list_head list; +struct list_head fs_list;struct btrfs_fs_devices *seed;
Signature · 1 line
2.39.3
PATCH 12/16 btrfs-progs: rename fs_devices::latest_trans to match the kernel
In reply to Anand JainAligning progs's struct btrfs_fs_devices with the kernel rename btrfs_fs_devices::latest_trans to btrfs_fs_devices::latest_generation.
--- kernel-shared/volumes.c | 6 +++--- kernel-shared/volumes.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)
kernel-shared/volumes.c +3 -3
@@ -368,7 +368,7 @@ static int device_list_add(const char *path, fs_devices->latest_devid = devid;/* Below we would set this to found_transid */ - fs_devices->latest_trans =0 ; + fs_devices->latest_generation =0 ; fs_devices->lowest_devid = (u64)-1 ; fs_devices->chunk_alloc_policy =BTRFS_CHUNK_ALLOC_REGULAR ; device =NULL ;
@@ -438,9 +438,9 @@ static int device_list_add(const char *path,if (metadata_uuid) fs_devices->active_metadata_uuid = true; -if (found_transid > fs_devices->latest_trans) { +if (found_transid > fs_devices->latest_generation) { fs_devices->latest_devid = devid; - fs_devices->latest_trans = found_transid; + fs_devices->latest_generation = found_transid; fs_devices->total_devices = device->total_devs; }if (fs_devices->lowest_devid > devid) {
kernel-shared/volumes.h +1 -1
@@ -85,7 +85,7 @@ struct btrfs_fs_devices {/* the device with this id has the most recent copy of the super */ u64 latest_devid; -u64 latest_trans; +u64 latest_generation;u64 lowest_devid;u64 num_devices;
Signature · 1 line
2.39.3
PATCH 13/16 btrfs-progs: tune use the latest bdev in fs_devices for super_copy
In reply to Anand Jainbtrfstune relies on the superblock of the device specified in the btrfstune argument for fs_info::super_copy. Instead, should use fs_devices::latest_bdev.
To support for reuniting devices following past failures of btrfstune -m|M|u|U as in the following patch, use fs_devices::latest_bdev.
btrfs-progs: add support to fix superblock with CHANGING_FSID_V2 flag
btrfs-progs: recover from the failed btrfstune -m|M
--- tune/main.c | 3 +++ 1 file changed, 3 insertions(+)
tune/main.c +3 -0
@@ -290,6 +290,9 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[]) goto free_out; } +if (change_metadata_uuid || random_fsid || new_fsid_str) + ctree_flags |=OPEN_CTREE_USE_LATEST_BDEV ; + root =open_ctree_fd (fd, device,0 , ctree_flags);if (!root) {
Signature · 1 line
2.39.3
PATCH 14/16 btrfs-progs: add support to fix superblock with CHANGING_FSID_V2 flag
In reply to Anand JainMost of the code and functions in this patch is copied from the kernel. Now, with this patch applied, there is no need to mount the device to complete the incomplete 'btrfstune -m|M' command (CHANING_FSID_V2 flag). Instead, the same command could be run, which will successfully complete the operation.
Currently, the 'tests/misc-tests/034-metadata-uuid' tests the kernel using four sets of disk images with CHANING_FSID_V2. Now, this test case has been updated (as in the next patch) to test the the progs part.
--- kernel-shared/volumes.c | 184 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 179 insertions(+), 5 deletions(-)
kernel-shared/volumes.c +179 -5
@@ -332,6 +332,159 @@ static struct btrfs_fs_devices *find_fsid(u8 *fsid, u8 *metadata_uuid)return NULL ; } +static u8 *btrfs_sb_fsid_ptr (struct btrfs_super_block *sb) +{ +bool has_metadata_uuid = (btrfs_super_incompat_flags (sb) & +BTRFS_FEATURE_INCOMPAT_METADATA_UUID ); + +return has_metadata_uuid ? sb->metadata_uuid : sb->fsid; +} + +static bool match_fsid_fs_devices (const struct btrfs_fs_devices *fs_devices, +const u8 *fsid,const u8 *metadata_fsid) +{ +if (memcmp (fsid, fs_devices->fsid,BTRFS_FSID_SIZE ) !=0 ) +return false; + +if (!metadata_fsid) +return true; + +if (memcmp (metadata_fsid, fs_devices->metadata_uuid,BTRFS_FSID_SIZE ) !=0 ) +return false; + +return true; +} + +/* +* First check if the metadata_uuid is different from the fsid in the given +* fs_devices. Then check if the given fsid is the same as the metadata_uuid +* in the fs_devices. If it is, return true; otherwise, return false. +*/ +static inline bool check_fsid_changed (const struct btrfs_fs_devices *fs_devices, +const u8 *fsid) +{ +return memcmp (fs_devices->fsid, fs_devices->metadata_uuid, +BTRFS_FSID_SIZE ) !=0 && +memcmp (fs_devices->metadata_uuid, fsid,BTRFS_FSID_SIZE ) ==0 ; +} + +static struct btrfs_fs_devices *find_fsid_with_metadata_uuid ( +struct btrfs_super_block *disk_super) +{ + +struct btrfs_fs_devices *fs_devices; + +/* +* Handle scanned device having completed its fsid change but +* belonging to a fs_devices that was created by first scanning +* a device which didn't have its fsid/metadata_uuid changed +* at all and the CHANGING_FSID_V2 flag set. +*/ +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +if (!fs_devices->changing_fsid) +continue ; + +if (match_fsid_fs_devices (fs_devices, disk_super->metadata_uuid, + fs_devices->fsid)) +return fs_devices; + } + +/* +* Handle scanned device having completed its fsid change but +* belonging to a fs_devices that was created by a device that +* has an outdated pair of fsid/metadata_uuid and +* CHANGING_FSID_V2 flag set. +*/ +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +if (!fs_devices->changing_fsid) +continue ; + +if (check_fsid_changed (fs_devices, disk_super->metadata_uuid)) +return fs_devices; + } + +return find_fsid (disk_super->fsid, disk_super->metadata_uuid); +} + +/* +* Handle scanned device having its CHANGING_FSID_V2 flag set and the fs_devices +* being created with a disk that has already completed its fsid change. Such +* disk can belong to an fs which has its FSID changed or to one which doesn't. +* Handle both cases here. +*/ +static struct btrfs_fs_devices *find_fsid_inprogress ( +struct btrfs_super_block *disk_super) +{ +struct btrfs_fs_devices *fs_devices; + +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +if (fs_devices->changing_fsid) +continue ; + +if (check_fsid_changed (fs_devices, disk_super->fsid)) +return fs_devices; + } + +return find_fsid (disk_super->fsid,NULL ); +} + +static struct btrfs_fs_devices *find_fsid_changed ( +struct btrfs_super_block *disk_super) +{ +struct btrfs_fs_devices *fs_devices; + +/* +* Handles the case where scanned device is part of an fs that had +* multiple successful changes of FSID but currently device didn't +* observe it. Meaning our fsid will be different than theirs. We need +* to handle two subcases : +* 1 - The fs still continues to have different METADATA/FSID uuids. +* 2 - The fs is switched back to its original FSID (METADATA/FSID +* are equal). +*/ +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +/* Changed UUIDs */ +if (check_fsid_changed (fs_devices, disk_super->metadata_uuid) && +memcmp (fs_devices->fsid, disk_super->fsid, +BTRFS_FSID_SIZE ) !=0 ) +return fs_devices; + +/* Unchanged UUIDs */ +if (memcmp (fs_devices->metadata_uuid, fs_devices->fsid, +BTRFS_FSID_SIZE ) ==0 && +memcmp (fs_devices->fsid, disk_super->metadata_uuid, +BTRFS_FSID_SIZE ) ==0 ) +return fs_devices; + } + +return NULL ; +} + +static struct btrfs_fs_devices *find_fsid_reverted_metadata ( +struct btrfs_super_block *disk_super) +{ +struct btrfs_fs_devices *fs_devices; + +/* +* Handle the case where the scanned device is part of an fs whose last +* metadata UUID change reverted it to the original FSID. At the same +* time fs_devices was first created by another constituent device +* which didn't fully observe the operation. This results in an +* btrfs_fs_devices created with metadata/fsid different AND +* btrfs_fs_devices::fsid_change set AND the metadata_uuid of the +* fs_devices equal to the FSID of the disk. +*/ +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +if (!fs_devices->changing_fsid) +continue ; + +if (check_fsid_changed (fs_devices, disk_super->fsid)) +return fs_devices; + } + +return NULL ; +} +static int device_list_add (const char *path,struct btrfs_super_block *disk_super,struct btrfs_fs_devices **fs_devices_ret)
@@ -346,11 +499,18 @@ static int device_list_add(const char *path, (BTRFS_SUPER_FLAG_CHANGING_FSID | BTRFS_SUPER_FLAG_CHANGING_FSID_V2)); -if (metadata_uuid) - fs_devices =find_fsid (disk_super->fsid, - disk_super->metadata_uuid); -else - fs_devices =find_fsid (disk_super->fsid,NULL ); +if (changing_fsid) { +if (!metadata_uuid) + fs_devices =find_fsid_inprogress (disk_super); +else + fs_devices =find_fsid_changed (disk_super); + }else if (metadata_uuid) { + fs_devices =find_fsid_with_metadata_uuid (disk_super); + }else { + fs_devices =find_fsid_reverted_metadata (disk_super); +if (!fs_devices) + fs_devices =find_fsid (disk_super->fsid,NULL ); + }if (!fs_devices) { fs_devices =kzalloc (sizeof (*fs_devices),GFP_NOFS );
@@ -375,7 +535,21 @@ static int device_list_add(const char *path, }else { device =find_device (fs_devices, devid, disk_super->dev_item.uuid); +/* +* If this disk has been pulled into an fs devices created by +* a device which had the CHANGING_FSID_V2 flag then replace the +* metadata_uuid/fsid values of the fs_devices. +*/ +if (fs_devices->changing_fsid && + found_transid > fs_devices->latest_generation) { +memcpy (fs_devices->fsid, disk_super->fsid, +BTRFS_FSID_SIZE ); +memcpy (fs_devices->metadata_uuid, +btrfs_sb_fsid_ptr (disk_super),BTRFS_FSID_SIZE ); + fs_devices->changing_fsid = false; + } } +if (!device) { device =kzalloc (sizeof (*device),GFP_NOFS );if (!device) {
Signature · 1 line
2.39.3
PATCH 15/16 btrfs-progs: recover from the failed btrfstune -m|M
In reply to Anand JainCurrently, to fix device following the write failure of one or more devices during btrfstune -m|M, we rely on the kernel's ability to reassemble devices, even when they possess distinct fsids.
Kernel hinges combinations of metadata_uuid and generation number, with additional cues taken from the fsid and the BTRFS_SUPER_FLAG_CHANGING_FSID_V2 flag. This patch adds this capability to btrfs-progs.
In complex scenarios (such as multiple fsids with the same metadata_uuid and matching generation), user intervention becomes necessary to resolve the situations which btrfs-prog can do better.
--- tune/change-metadata-uuid.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
tune/change-metadata-uuid.c +16 -7
@@ -46,14 +46,23 @@ int set_metadata_uuid(struct btrfs_root *root, const char *new_fsid_string) }if (check_unfinished_fsid_change (root->fs_info, fsid, metadata_uuid)) { -error ("UUID rewrite in progress, cannot change metadata_uuid" ); -return 1 ; - } +if (new_fsid_string) { +uuid_t tmp; -if (new_fsid_string) -uuid_parse (new_fsid_string, fsid); -else -uuid_generate (fsid); +uuid_parse (new_fsid_string, tmp); +if (memcmp (tmp, fsid,BTRFS_FSID_SIZE )) { +error ( +"new fsid %s is not the same with unfinished fsid change" , + new_fsid_string); +return -EINVAL ; + } + } + }else { +if (new_fsid_string) +uuid_parse (new_fsid_string, fsid); +else +uuid_generate (fsid); + } new_fsid = (memcmp (fsid, disk_super->fsid,BTRFS_FSID_SIZE ) !=0 );
Signature · 1 line
2.39.3
PATCH 16/16 btrfs-progs: test btrfstune -m|M ability to fix previous failures
In reply to Anand JainThe misc-test/034-metadata_uuid test case, has four sets of disk images to simulate failed writes during btrfstune -m|M operations. As of now, this tests kernel only. Update the test case to verify btrfstune -m|M's capacity to recover from the same scenarios.
--- tests/misc-tests/034-metadata-uuid/test.sh | 70 ++++++++++++++++------ 1 file changed, 53 insertions(+), 17 deletions(-)
tests/misc-tests/034-metadata-uuid/test.sh +53 -17
@@ -195,13 +195,42 @@ check_multi_fsid_unchanged() { check_flag_cleared "$1" "$2" } -failure_recovery() { +failure_recovery_progs() { + local image1 + local image2 + local loop1 + local loop2 + local devcount + + image1=$(extract_image "$1") + image2=$(extract_image "$2") + loop1=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image1") + loop2=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image2") + + run_check $SUDO_HELPER udevadm settle + + # Scan to make sure btrfs detects both devices before trying to mount + #run_check "$TOP/btrfstune" -m --noscan --device="$loop1" "$loop2" + run_check "$TOP/btrfstune" -m "$loop2" + + # perform any specific check + "$3" "$loop1" "$loop2" + + # cleanup + run_check $SUDO_HELPER losetup -d "$loop1" + run_check $SUDO_HELPER losetup -d "$loop2" + rm -f -- "$image1" "$image2" +} + +failure_recovery_kernel() { local image1 local image2 local loop1 local loop2 local devcount + reload_btrfs + image1=$(extract_image "$1") image2=$(extract_image "$2") loop1=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image1")
@@ -226,47 +255,55 @@ failure_recovery() { rm -f -- "$image1" "$image2" } +failure_recovery() { + failure_recovery_progs $@ + failure_recovery_kernel $@ +} + reload_btrfs() { run_check $SUDO_HELPER rmmod btrfs run_check $SUDO_HELPER modprobe btrfs } -# for full coverage we need btrfs to actually be a module -modinfo btrfs > /dev/null 2>&1 || _not_run "btrfs must be a module" -run_mayfail $SUDO_HELPER modprobe -r btrfs || _not_run "btrfs must be unloadable" -run_mayfail $SUDO_HELPER modprobe btrfs || _not_run "loading btrfs module failed" +test_progs() { + run_check_mkfs_test_dev + check_btrfstune + + run_check_mkfs_test_dev + check_dump_super_output -run_check_mkfs_test_dev -check_btrfstune + run_check_mkfs_test_dev + check_image_restore +} + +check_kernel_reloadable() { + # for full coverage we need btrfs to actually be a module + modinfo btrfs > /dev/null 2>&1 || _not_run "btrfs must be a module" + run_mayfail $SUDO_HELPER modprobe -r btrfs || _not_run "btrfs must be unloadable" + run_mayfail $SUDO_HELPER modprobe btrfs || _not_run "loading btrfs module failed" +} -run_check_mkfs_test_dev -check_dump_super_output +check_kernel_reloadable -run_check_mkfs_test_dev -check_image_restore +test_progs # disk1 is an image which has no metadata uuid flags set and disk2 is part of # the same fs but has the in-progress flag set. Test that whicever is scanned # first will result in consistent filesystem. failure_recovery "./disk1.raw.xz" "./disk2.raw.xz" check_inprogress_flag -reload_btrfs failure_recovery "./disk2.raw.xz" "./disk1.raw.xz" check_inprogress_flag # disk4 contains an image in with the in-progress flag set and disk 3 is part # of the same filesystem but has both METADATA_UUID incompat and a new # metadata uuid set. So disk 3 must always take precedence -reload_btrfs failure_recovery "./disk3.raw.xz" "./disk4.raw.xz" check_completed -reload_btrfs failure_recovery "./disk4.raw.xz" "./disk3.raw.xz" check_completed # disk5 contains an image which has undergone a successful fsid change more # than once, disk6 on the other hand is member of the same filesystem but # hasn't completed its last change. Thus it has both the FSID_CHANGING flag set # and METADATA_UUID flag set. -reload_btrfs failure_recovery "./disk5.raw.xz" "./disk6.raw.xz" check_multi_fsid_change -reload_btrfs failure_recovery "./disk6.raw.xz" "./disk5.raw.xz" check_multi_fsid_change # disk7 contains an image which has undergone a successful fsid change once to
@@ -275,5 +312,4 @@ failure_recovery "./disk6.raw.xz" "./disk5.raw.xz" check_multi_fsid_change # during the process change. So disk 7 looks as if it never underwent fsid change # and disk 8 has FSID_CHANGING_FLAG and METADATA_UUID but is stale. failure_recovery "./disk7.raw.xz" "./disk8.raw.xz" check_multi_fsid_unchanged -reload_btrfs failure_recovery "./disk8.raw.xz" "./disk7.raw.xz" check_multi_fsid_unchanged
Signature · 1 line
2.39.3
Re: [PATCH 16/16] btrfs-progs: test btrfstune -m|M ability to fix previous failures
Quoted 25 lines · Anand Jain
The misc-test/034-metadata_uuid test case, has four sets of disk images to simulate failed writes during btrfstune -m|M operations. As of now, this tests kernel only. Update the test case to verify btrfstune -m|M's capacity to recover from the same scenarios.
Signed-off-by Anand Jain <[email protected]>--- tests/misc-tests/034-metadata-uuid/test.sh | 70 ++++++++++++++++------ 1 file changed, 53 insertions(+), 17 deletions(-)tests/misc-tests/034-metadata-uuid/test.sh +6
-1@@ -195,13 +195,42 @@ check_multi_fsid_unchanged() { check_flag_cleared "$1" "$2" } -failure_recovery() { +failure_recovery_progs() { + local image1 + local image2 + local loop1 + local loop2 + local devcount
tests/misc-tests/034-metadata-uuid/test.sh check_multi_fsid_unchanged() { · line 213
Quoted 10 lines
tests/misc-tests/034-metadata-uuid/test.sh +10
-0+ + image1=$(extract_image "$1") + image2=$(extract_image "$2") + loop1=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image1") + loop2=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image2") + + run_check $SUDO_HELPER udevadm settle + + # Scan to make sure btrfs detects both devices before trying to mount + #run_check "$TOP/btrfstune" -m --noscan --device="$loop1" "$loop2"
Why is this line here? It looks like a valid command line and then it's confusing. If there's some condition like that you want to require scanning then put it to comment and explain it, e.g. "we can't use --noscan here because ...".
Quoted 10 lines
tests/misc-tests/034-metadata-uuid/test.sh +10
-0+ run_check "$TOP/btrfstune" -m "$loop2" + + # perform any specific check + "$3" "$loop1" "$loop2" + + # cleanup + run_check $SUDO_HELPER losetup -d "$loop1" + run_check $SUDO_HELPER losetup -d "$loop2" + rm -f -- "$image1" "$image2" +}
Re: [PATCH 16/16] btrfs-progs: test btrfstune -m|M ability to fix previous failures
Quoted 35 lines · David Sterba
Quoted 33 lines · Anand Jain
The misc-test/034-metadata_uuid test case, has four sets of disk images to simulate failed writes during btrfstune -m|M operations. As of now, this tests kernel only. Update the test case to verify btrfstune -m|M's capacity to recover from the same scenarios.
Signed-off-by Anand Jain <[email protected]>--- tests/misc-tests/034-metadata-uuid/test.sh | 70 ++++++++++++++++------ 1 file changed, 53 insertions(+), 17 deletions(-)tests/misc-tests/034-metadata-uuid/test.sh +14
-1@@ -195,13 +195,42 @@ check_multi_fsid_unchanged() { check_flag_cleared "$1" "$2" } -failure_recovery() { +failure_recovery_progs() { + local image1 + local image2 + local loop1 + local loop2 + local devcount + + image1=$(extract_image "$1") + image2=$(extract_image "$2") + loop1=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image1") + loop2=$(run_check_stdout $SUDO_HELPER losetup --find --show "$image2") + + run_check $SUDO_HELPER udevadm settle +
Quoted 6 lines
tests/misc-tests/034-metadata-uuid/test.sh check_multi_fsid_unchanged() { · line 212
Quoted 1 line
tests/misc-tests/034-metadata-uuid/test.sh +1
-0+ # Scan to make sure btrfs detects both devices before trying to mount >> + #run_check "$TOP/btrfstune" -m --noscan --device="$loop1" "$loop2"Why is this line here? It looks like a valid command line and then it's confusing. If there's some condition like that you want to require scanning then put it to comment and explain it, e.g. "we can't use --noscan here because ...".
My bad. Those two comment lines shouldn't be here, and it's okay to remove them.
It was taken from the patchset that added the --device and --noscan options. I'll clean it up.
Thanks, Anand
Quoted 10 lines
Quoted 10 lines
tests/misc-tests/034-metadata-uuid/test.sh +10
-0+ run_check "$TOP/btrfstune" -m "$loop2" + + # perform any specific check + "$3" "$loop1" "$loop2" + + # cleanup + run_check $SUDO_HELPER losetup -d "$loop1" + run_check $SUDO_HELPER losetup -d "$loop2" + rm -f -- "$image1" "$image2" +}
Quoted 37 lines · Anand Jain
The kernel reunites split-brained devices after a failed
btrfstune -m|Moperation. We can achieve the same in btrfs-progs. So port it here. Ref the discussion here:https://lore.kernel.org/all/[email protected]/Patch 1/16 wasn't integrated as part of the set
[PATCH 00/10 v2] fixes and preparatory related tometadata_uuid
it's now merged with this patchset.Patches [2-6,11,12] are cleanup patches.
Patches [7,8,10] are preparatory.
Patch [9] addresses a bug.
Patches [13, 14, 15] provide recovery from previously failed
btrfstune -m|Moperations.Patch [16] enhances the misc-test
034-metadata-uuidto also validate this new recovery feature.This set has been successfully tested with the btrfs-progs testsuite.
This patchset is on top the latest devel last commit:
8aba9b0052b6btrfs-progs: btrfstune: consolidate error handling inmain()Anand Jain (16):
btrfs-progs: tracknum_devicesperfs_devices
btrfs-progs: tune can use localfs_infovariable
btrfs-progs: renameset_metadata_uuidarg tonew_fsid_str
btrfs-progs: renameset_metadata_uuidnew_fsidto fsid
btrfs-progs: renameset_metadata_uuidnew_uuidtonew_fsid
btrfs-progs: renameset_metadata_uuiduuid_changedtofsid_changed
btrfs-progs: pass fsid incheck_unfinished_fsid_changearg2
btrfs-progs: passmetadata_uuidincheck_unfinished_fsid_changearg3
Quoted 8 lines
btrfs-progs: fix return without flag reset commit in tune
btrfs-progs: preparing the latest device's superblock for commit
btrfs-progs: renamefs_devices::listto match the kernel
btrfs-progs: renamefs_devices::latest_transto match the kernel
btrfs-progs: tune use the latest bdev infs_devicesforsuper_copy
btrfs-progs: add support to fix superblock withCHANGING_FSID_V2flag
btrfs-progs: recover from the failed btrfstune -m|M
btrfs-progs: test btrfstune -m|M ability to fix previous failures
Patches added to devel, thanks.
Quoted 47 lines · David Sterba
Quoted 37 lines · Anand Jain
The kernel reunites split-brained devices after a failed
btrfstune -m|Moperation. We can achieve the same in btrfs-progs. So port it here. Ref the discussion here:https://lore.kernel.org/all/[email protected]/Patch 1/16 wasn't integrated as part of the set
[PATCH 00/10 v2] fixes and preparatory related tometadata_uuid
it's now merged with this patchset.Patches [2-6,11,12] are cleanup patches.
Patches [7,8,10] are preparatory.
Patch [9] addresses a bug.
Patches [13, 14, 15] provide recovery from previously failed
btrfstune -m|Moperations.Patch [16] enhances the misc-test
034-metadata-uuidto also validate this new recovery feature.This set has been successfully tested with the btrfs-progs testsuite.
This patchset is on top the latest devel last commit:
8aba9b0052b6btrfs-progs: btrfstune: consolidate error handling inmain()Anand Jain (16):
btrfs-progs: tracknum_devicesperfs_devices
btrfs-progs: tune can use localfs_infovariable
btrfs-progs: renameset_metadata_uuidarg tonew_fsid_str
btrfs-progs: renameset_metadata_uuidnew_fsidto fsid
btrfs-progs: renameset_metadata_uuidnew_uuidtonew_fsid
btrfs-progs: renameset_metadata_uuiduuid_changedtofsid_changed
btrfs-progs: pass fsid incheck_unfinished_fsid_changearg2
btrfs-progs: passmetadata_uuidincheck_unfinished_fsid_changearg3Quoted 8 lines
btrfs-progs: fix return without flag reset commit in tune
btrfs-progs: preparing the latest device's superblock for commit
btrfs-progs: renamefs_devices::listto match the kernel
btrfs-progs: renamefs_devices::latest_transto match the kernel
btrfs-progs: tune use the latest bdev infs_devicesforsuper_copy
btrfs-progs: add support to fix superblock withCHANGING_FSID_V2flag
btrfs-progs: recover from the failed btrfstune -m|M
btrfs-progs: test btrfstune -m|M ability to fix previous failures
Quoted 1 line
Patches added to devel, thanks.
On my machine the metadata uuid test does not run because the module is not loadable, but the GH actions report a failure:
https://github.com/kdave/btrfs-progs/actions/runs/5956097489/job/16156138260
Quoted 48 lines · David Sterba
Quoted 47 lines · David Sterba
Quoted 45 lines · Anand Jain
The kernel reunites split-brained devices after a failed
btrfstune -m|Moperation. We can achieve the same in btrfs-progs. So port it here. Ref the discussion here:https://lore.kernel.org/all/[email protected]/Patch 1/16 wasn't integrated as part of the set
[PATCH 00/10 v2] fixes and preparatory related tometadata_uuid
it's now merged with this patchset.Patches [2-6,11,12] are cleanup patches.
Patches [7,8,10] are preparatory.
Patch [9] addresses a bug.
Patches [13, 14, 15] provide recovery from previously failed
btrfstune -m|Moperations.Patch [16] enhances the misc-test
034-metadata-uuidto also validate this new recovery feature.This set has been successfully tested with the btrfs-progs testsuite.
This patchset is on top the latest devel last commit:
8aba9b0052b6btrfs-progs: btrfstune: consolidate error handling inmain()Anand Jain (16):
btrfs-progs: tracknum_devicesperfs_devices
btrfs-progs: tune can use localfs_infovariable
btrfs-progs: renameset_metadata_uuidarg tonew_fsid_str
btrfs-progs: renameset_metadata_uuidnew_fsidto fsid
btrfs-progs: renameset_metadata_uuidnew_uuidtonew_fsid
btrfs-progs: renameset_metadata_uuiduuid_changedtofsid_changed
btrfs-progs: pass fsid incheck_unfinished_fsid_changearg2
btrfs-progs: passmetadata_uuidincheck_unfinished_fsid_changearg3
btrfs-progs: fix return without flag reset commit in tune
btrfs-progs: preparing the latest device's superblock for commit
btrfs-progs: renamefs_devices::listto match the kernel
btrfs-progs: renamefs_devices::latest_transto match the kernel
btrfs-progs: tune use the latest bdev infs_devicesforsuper_copy
btrfs-progs: add support to fix superblock withCHANGING_FSID_V2flag
btrfs-progs: recover from the failed btrfstune -m|M
btrfs-progs: test btrfstune -m|M ability to fix previous failures
Quoted 5 lines
Quoted 1 line
Patches added to devel, thanks.
On my machine the metadata uuid test does not run because the module is not loadable, but the GH actions report a failure:
https://github.com/kdave/btrfs-progs/actions/runs/5956097489/job/1615613826
Local VM successfully runs misc-tests/034*. However, on OCI, the same error as GH. Error reports missing device. It appears, inconsistent results due to the varying device scan order from system to system. I am looking more.
Quoted 10 lines · Anand Jain
Quoted 5 lines
Quoted 1 line
Patches added to devel, thanks.
On my machine the metadata uuid test does not run because the module is not loadable, but the GH actions report a failure:
https://github.com/kdave/btrfs-progs/actions/runs/5956097489/job/1615613826Local VM successfully runs misc-tests/034*. However, on OCI, the same error as GH. Error reports missing device. It appears, inconsistent results due to the varying device scan order from system to system. I am looking more.
Patches 13-16 have been removed from devel until the issue is resolved. I've enabled build tests for pull requests you can use the github CI for testing too (open a PR against devel or master branch).
Quoted 7 lines · David Sterba
Quoted 6 lines · Anand Jain
Quoted 5 lines
Quoted 1 line
Patches added to devel, thanks.
On my machine the metadata uuid test does not run because the module is not loadable, but the GH actions report a failure:
https://github.com/kdave/btrfs-progs/actions/runs/5956097489/job/1615613826
Quoted 8 lines
Quoted 4 lines
Local VM successfully runs misc-tests/034*. However, on OCI, the same error as GH. Error reports missing device. It appears, inconsistent results due to the varying device scan order from system to system. I am looking more.
Patches 13-16 have been removed from devel until the issue is resolved. I've enabled build tests for pull requests you can use the github CI for testing too (open a PR against devel or master branch).
Yes, I noticed that patches 1 to 12 have been merged. I am able to reproduce the bug. V3 is tested working fine.