More split-brain fixes for metadata uuid feature
cover More split-brain fixes for metadata uuid feature
Here are 4 patches which fix a newly found split-brain scenario in the METADATA_UUID_INCOMPAT code. They are mostly the identical with Su's original submission but I have reworked the changelogs and some function names. Hence, I retained his authorship of the patches.
First 2 patches factor out some code with the hopes of making find_fisd a bit more readable and simplifying the myriad of nested 'if' in device_list_add.
Patch 3 extends find_fsid_changed to handle the case where a disk with METADATA_UUID_INCOMPAT and FSID_CHANGING_IN_PROGRESS is scanned after a disk which has successfully been switched to FSID == METADATA_UUID state and has created btrfs_fs_devices.
Patch 4 handles the counterpart situation - a fully switched disk is scanned after one which has had METADATA_UUID_INCOMPAT and FSID_CHANGING_IN_PROGRESS set.
This series should be applied to stable branches following 5.0 when the metadata_uuid feature got introduced.
This patchset was tested with btrfs-progs' misc 034-metadata-uuid test and a full xfstest run with no regressions. I will also be sending an improvement to the test case which exercises the newly added code.
btrfs: Call find_fsid from find_fsid_inprogress
btrfs: Factor out metadata_uuid code from find_fsid.
btrfs: Handle another split brain scenario with metadata uuid feature
btrfs: Fix split-brain handling when changing FSID to metadata uuid
fs/btrfs/volumes.c | 158 ++++++++++++++++++++++++++++----------------- 1 file changed, 100 insertions(+), 58 deletions(-)
--
2.17.1
PATCH 1/4 btrfs: Call find_fsid from find_fsid_inprogress
From: Su Yue <[email protected]>
Since find_fsid_inprogress should also handle the case in which an fs didn't change its FSID make it call find_fsid directly. This makes the code in device_list_add simpler by eliminating a conditional call of find_fsid. No functional changes.
--- fs/btrfs/volumes.c | 19 ++++++------------- 1 file changed, 6 insertions(+), 13 deletions(-)
fs/btrfs/volumes.c +6 -13
@@ -671,7 +671,9 @@ static int btrfs_open_one_device(struct btrfs_fs_devices *fs_devices,/* * Handle scanned device having its CHANGING_FSID_V2 flag set and the fs_devices -* being created with a disk that has already completed its fsid change. +* being created with a disk that has already completed its fsid change. Such +* disk can belong to an fs which has its FSID changed or to one which doesn't. +* Handle both cases here. */ static struct btrfs_fs_devices *find_fsid_inprogress (struct btrfs_super_block *disk_super)
@@ -687,7 +689,7 @@ static struct btrfs_fs_devices *find_fsid_inprogress( } } -return NULL ; +return find_fsid (disk_super->fsid,NULL ); }
@@ -736,19 +738,10 @@ static noinline struct btrfs_device *device_list_add(const char *path, BTRFS_SUPER_FLAG_CHANGING_FSID_V2);if (fsid_change_in_progress) { -if (!has_metadata_uuid) { -/* -* When we have an image which has CHANGING_FSID_V2 set -* it might belong to either a filesystem which has -* disks with completed fsid change or it might belong -* to fs with no UUID changes in effect, handle both. -*/ +if (!has_metadata_uuid) fs_devices =find_fsid_inprogress (disk_super); -if (!fs_devices) - fs_devices =find_fsid (disk_super->fsid,NULL ); - }else { +else fs_devices =find_fsid_changed (disk_super); - } }else if (has_metadata_uuid) { fs_devices =find_fsid (disk_super->fsid, disk_super->metadata_uuid);
--
2.17.1
Re: [PATCH 1/4] btrfs: Call find_fsid from find_fsid_inprogress
Quoted 9 lines · Nikolay Borisov
From: Su Yue <[email protected]>
Since
find_fsid_inprogressshould also handle the case in which an fs didn't change its FSID make it callfind_fsiddirectly. This makes the code indevice_list_addsimpler by eliminating a conditional call offind_fsid. No functional changes.Signed-off-by Su Yue <[email protected]>Signed-off-by Nikolay Borisov <[email protected]>
Thanks,
Josef
PATCH 2/4 btrfs: Factor out metadata_uuid code from find_fsid.
In reply to Nikolay BorisovFrom: Su Yue <[email protected]>
find_fsid became rather hairy with the introduction of metadata uuid changing feature. Alleviate this by factoring out the metadata uuid specific code in a dedicated function which deals with finding correct fsid for a device with changed uuid.
--- fs/btrfs/volumes.c | 77 +++++++++++++++++++++++++--------------------- 1 file changed, 42 insertions(+), 35 deletions(-)
fs/btrfs/volumes.c +42 -35
@@ -440,39 +440,6 @@ static noinline struct btrfs_fs_devices *find_fsid(ASSERT (fsid); -if (metadata_fsid) { -/* -* Handle scanned device having completed its fsid change but -* belonging to a fs_devices that was created by first scanning -* a device which didn't have its fsid/metadata_uuid changed -* at all and the CHANGING_FSID_V2 flag set. -*/ -list_for_each_entry (fs_devices, &fs_uuids, fs_list) { -if (fs_devices->fsid_change && -memcmp (metadata_fsid, fs_devices->fsid, -BTRFS_FSID_SIZE ) ==0 && -memcmp (fs_devices->fsid, fs_devices->metadata_uuid, -BTRFS_FSID_SIZE ) ==0 ) { -return fs_devices; - } - } -/* -* Handle scanned device having completed its fsid change but -* belonging to a fs_devices that was created by a device that -* has an outdated pair of fsid/metadata_uuid and -* CHANGING_FSID_V2 flag set. -*/ -list_for_each_entry (fs_devices, &fs_uuids, fs_list) { -if (fs_devices->fsid_change && -memcmp (fs_devices->metadata_uuid, - fs_devices->fsid,BTRFS_FSID_SIZE ) !=0 && -memcmp (metadata_fsid, fs_devices->metadata_uuid, -BTRFS_FSID_SIZE ) ==0 ) { -return fs_devices; - } - } - } -/* Handle non-split brain cases */ list_for_each_entry (fs_devices, &fs_uuids, fs_list) {if (metadata_fsid) {
@@ -488,6 +455,47 @@ static noinline struct btrfs_fs_devices *find_fsid(return NULL ; } +static struct btrfs_fs_devices *find_fsid_with_metadata_uuid ( +struct btrfs_super_block *disk_super) +{ + +struct btrfs_fs_devices *fs_devices; + +/* +* Handle scanned device having completed its fsid change but +* belonging to a fs_devices that was created by first scanning +* a device which didn't have its fsid/metadata_uuid changed +* at all and the CHANGING_FSID_V2 flag set. +*/ +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +if (fs_devices->fsid_change && +memcmp (disk_super->metadata_uuid, fs_devices->fsid, +BTRFS_FSID_SIZE ) ==0 && +memcmp (fs_devices->fsid, fs_devices->metadata_uuid, +BTRFS_FSID_SIZE ) ==0 ) { +return fs_devices; + } + } +/* +* Handle scanned device having completed its fsid change but +* belonging to a fs_devices that was created by a device that +* has an outdated pair of fsid/metadata_uuid and +* CHANGING_FSID_V2 flag set. +*/ +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +if (fs_devices->fsid_change && +memcmp (fs_devices->metadata_uuid, + fs_devices->fsid,BTRFS_FSID_SIZE ) !=0 && +memcmp (disk_super->metadata_uuid, fs_devices->metadata_uuid, +BTRFS_FSID_SIZE ) ==0 ) { +return fs_devices; + } + } + +return find_fsid (disk_super->fsid, disk_super->metadata_uuid); +} + + static int btrfs_get_bdev_and_sb(const char *device_path, fmode_t flags, void *holder, int flush, struct block_device **bdev,
@@ -743,8 +751,7 @@ static noinline struct btrfs_device *device_list_add(const char *path,else fs_devices =find_fsid_changed (disk_super); }else if (has_metadata_uuid) { - fs_devices =find_fsid (disk_super->fsid, - disk_super->metadata_uuid); + fs_devices =find_fsid_with_metadata_uuid (disk_super); }else { fs_devices =find_fsid (disk_super->fsid,NULL ); }
--
2.17.1
Re: [PATCH 2/4] btrfs: Factor out metadata_uuid code from find_fsid.
Quoted 9 lines · Nikolay Borisov
From: Su Yue <[email protected]>
find_fsidbecame rather hairy with the introduction of metadata uuid changing feature. Alleviate this by factoring out the metadata uuid specific code in a dedicated function which deals with finding correct fsid for a device with changed uuid.Signed-off-by Su Yue <[email protected]>Signed-off-by Nikolay Borisov <[email protected]>
Thanks,
Josef
PATCH 3/4 btrfs: Handle another split brain scenario with metadata uuid feature
In reply to Nikolay BorisovThere is one more cases which isn't handled by the original metadata uuid work. Namely, when a filesystem has METADATA_UUID incompat bit and the user decides to change the FSID to the original one e.g. have metadata_uuid and fsid match. In case of power failure while this operation is in progress we could end up in a situation where some of the disks have the incompat bit removed and the other half have both METADATA_UUID_INCOMPAT and FSID_CHANGING_IN_PROGRESS flags.
This patch handles the case where a disk that has successfully changed its FSID such that it equals METADATA_UUID is scanned first. Subsequently when a disk with both METADATA_UUID_INCOMPAT/FSID_CHANGING_IN_PROGRESS flags is scanned find_fsid_changed won't be able to find an appropriate btrfs_fs_devices. This is done by extending find_fsid_changed to correctly find btrfs_fs_devices whose metadata_uuid/fsid are the same and they match the metadata_uuid of the currently scanned device.
--- fs/btrfs/volumes.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-)
fs/btrfs/volumes.c +17 -8
@@ -709,17 +709,26 @@ static struct btrfs_fs_devices *find_fsid_changed(/* * Handles the case where scanned device is part of an fs that had * multiple successful changes of FSID but curently device didn't -* observe it. Meaning our fsid will be different than theirs. +* observe it. Meaning our fsid will be different than theirs. We need +* to handle two subcases : +* 1 - The fs still continues to have different METADATA/FSID uuids. +* 2 - The fs is switched back to its original FSID (METADATA/FSID +* are equal). */ list_for_each_entry (fs_devices, &fs_uuids, fs_list) { -if (memcmp (fs_devices->metadata_uuid, fs_devices->fsid, -BTRFS_FSID_SIZE ) !=0 && -memcmp (fs_devices->metadata_uuid, disk_super->metadata_uuid, -BTRFS_FSID_SIZE ) ==0 && -memcmp (fs_devices->fsid, disk_super->fsid, -BTRFS_FSID_SIZE ) !=0 ) { +bool changed_fsdevices = +memcmp (fs_devices->metadata_uuid, fs_devices->fsid, +BTRFS_FSID_SIZE ) !=0 && +memcmp (fs_devices->metadata_uuid, + disk_super->metadata_uuid,BTRFS_FSID_SIZE ) ==0 && +memcmp (fs_devices->fsid, disk_super->fsid,BTRFS_FSID_SIZE ) !=0 ; + +bool unchanged_fsdevices = +memcmp (fs_devices->metadata_uuid, fs_devices->fsid, +BTRFS_FSID_SIZE ) ==0 && +memcmp (fs_devices->fsid, disk_super->metadata_uuid,BTRFS_FSID_SIZE ) ==0 ; +if (changed_fsdevices || unchanged_fsdevices)return fs_devices; - } }return NULL ;
--
2.17.1find_fsid_changed
Re: [PATCH 3/4] btrfs: Handle another split brain scenario with metadata uuid feature
Quoted 17 lines · Nikolay Borisov
There is one more cases which isn't handled by the original metadata uuid work. Namely, when a filesystem has
METADATA_UUIDincompat bit and the user decides to change the FSID to the original one e.g. havemetadata_uuidand fsid match. In case of power failure while this operation is in progress we could end up in a situation where some of the disks have the incompat bit removed and the other half have bothMETADATA_UUID_INCOMPATandFSID_CHANGING_IN_PROGRESSflags.This patch handles the case where a disk that has successfully changed its FSID such that it equals
METADATA_UUIDis scanned first. Subsequently when a disk with both METADATA_UUID_INCOMPAT/FSID_CHANGING_IN_PROGRESS flags is scannedfind_fsid_changedwon't be able to find an appropriatebtrfs_fs_devices. This is done by extendingfind_fsid_changedto correctly findbtrfs_fs_deviceswhose metadata_uuid/fsid are the same and they match themetadata_uuidof the currently scanned device.
Quoted 3 lines
Fixes cc5de4e70256 ("btrfs: Handle final split-brain possibility during fsid change")Reported-by Su Yue <[email protected]>Signed-off-by Nikolay Borisov <[email protected]>
Thanks,
Josef
Re: [PATCH 3/4] btrfs: Handle another split brain scenario with metadata uuid feature
In reply to Nikolay BorisovQuoted 10 lines · Nikolay Borisov
fs/btrfs/volumes.c +3
-6list_for_each_entry (fs_devices, &fs_uuids, fs_list) { -if (memcmp (fs_devices->metadata_uuid, fs_devices->fsid, -BTRFS_FSID_SIZE ) !=0 && -memcmp (fs_devices->metadata_uuid, disk_super->metadata_uuid, -BTRFS_FSID_SIZE ) ==0 && -memcmp (fs_devices->fsid, disk_super->fsid, -BTRFS_FSID_SIZE ) !=0 ) { +bool changed_fsdevices = +memcmp (fs_devices->metadata_uuid, fs_devices->fsid, + BTRFS_FSID_SIZE) != 0 &&
fs/btrfs/volumes.c static struct btrfs_fs_devices *find_fsid_changed( · line 731
Quoted 10 lines
fs/btrfs/volumes.c +9
-0+memcmp (fs_devices->metadata_uuid, + disk_super->metadata_uuid,BTRFS_FSID_SIZE ) ==0 && +memcmp (fs_devices->fsid, disk_super->fsid,BTRFS_FSID_SIZE ) !=0 ; + +bool unchanged_fsdevices = +memcmp (fs_devices->metadata_uuid, fs_devices->fsid, +BTRFS_FSID_SIZE ) ==0 && +memcmp (fs_devices->fsid, disk_super->metadata_uuid,BTRFS_FSID_SIZE ) ==0 ; +if (changed_fsdevices || unchanged_fsdevices)return fs_devices;
This is ugly, I've converted it to if (memcmp) and dropped the variables.
PATCH 4/4 btrfs: Fix split-brain handling when changing FSID to metadata uuid
In reply to Nikolay BorisovCurrent code doesn't correctly handle the situation which arises when a file system that has METADATA_UUID_INCOMPAT flag set has its FSID changed to the one in metadata uuid. This causes the incompat flag to disappear. In case of a power failure we could end up in a situation where part of the disks in a multi-disk filesystem are correctly reverted to METADATA_UUID_INCOMPAT flag unset state, while others have METADATA_UUID_INCOMPAT set and CHANGING_FSID_V2_IN_PROGRESS.
This patch corrects the behavior required to handle the case where a disk of the second type is scanned first, creating the necessary btrfs_fs_devices. Subsequently, when a disk which has already completed the transition is scanned it should overwrite the data in btrfs_fs_devices.
--- fs/btrfs/volumes.c | 41 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 4 deletions(-)
fs/btrfs/volumes.c +37 -4
@@ -733,6 +733,32 @@ static struct btrfs_fs_devices *find_fsid_changed(return NULL ; } + +static struct btrfs_fs_devices *find_fsid_reverted_metadata ( +struct btrfs_super_block *disk_super) +{ +struct btrfs_fs_devices *fs_devices; + +/* +* Handles the case where the scanned device is part of an fs whose last +* metadata uuid change reverted it to the original FSID. At the same time +* fs_devices was first created by another constitutent device which didn't +* fully observer the operation. This results in an btrfs_fs_devices +* created with metadata/fsid different AND btrfs_fs_devices::fsid_change +* set AND the metadata_uuid of the fs_devices equal to the FSID of the +* disk. +*/ +list_for_each_entry (fs_devices, &fs_uuids, fs_list) { +if (memcmp (fs_devices->fsid, fs_devices->metadata_uuid, +BTRFS_FSID_SIZE ) !=0 && +memcmp (fs_devices->metadata_uuid, disk_super->fsid, +BTRFS_FSID_SIZE ) ==0 && + fs_devices->fsid_change) +return fs_devices; + } + +return NULL ; +}/* * Add new device to list of registered devices *
@@ -762,7 +788,9 @@ static noinline struct btrfs_device *device_list_add(const char *path, }else if (has_metadata_uuid) { fs_devices =find_fsid_with_metadata_uuid (disk_super); }else { - fs_devices =find_fsid (disk_super->fsid,NULL ); + fs_devices =find_fsid_reverted_metadata (disk_super); +if (!fs_devices) + fs_devices =find_fsid (disk_super->fsid,NULL ); }
@@ -792,12 +820,17 @@ static noinline struct btrfs_device *device_list_add(const char *path,* a device which had the CHANGING_FSID_V2 flag then replace the * metadata_uuid/fsid values of the fs_devices. */ -if (has_metadata_uuid && fs_devices->fsid_change && +if (fs_devices->fsid_change && found_transid > fs_devices->latest_generation) {memcpy (fs_devices->fsid, disk_super->fsid,BTRFS_FSID_SIZE ); -memcpy (fs_devices->metadata_uuid, - disk_super->metadata_uuid,BTRFS_FSID_SIZE ); + +if (has_metadata_uuid) +memcpy (fs_devices->metadata_uuid, + disk_super->metadata_uuid,BTRFS_FSID_SIZE ); +else +memcpy (fs_devices->metadata_uuid, + disk_super->fsid,BTRFS_FSID_SIZE ); fs_devices->fsid_change = false; }
--
2.17.1
Re: [PATCH 4/4] btrfs: Fix split-brain handling when changing FSID to metadata uuid
Quoted 8 lines · Nikolay Borisov
Current code doesn't correctly handle the situation which arises when a file system that has
METADATA_UUID_INCOMPATflag set has its FSID changed to the one in metadata uuid. This causes the incompat flag to disappear. In case of a power failure we could end up in a situation where part of the disks in a multi-disk filesystem are correctly reverted toMETADATA_UUID_INCOMPATflag unset state, while others haveMETADATA_UUID_INCOMPATset andCHANGING_FSID_V2_IN_PROGRESS.
Quoted 8 lines
This patch corrects the behavior required to handle the case where a disk of the second type is scanned first, creating the necessary
btrfs_fs_devices. Subsequently, when a disk which has already completed the transition is scanned it should overwrite the data inbtrfs_fs_devices.Reported-by Su Yue <[email protected]>Signed-off-by Nikolay Borisov <[email protected]>
Thanks,
Josef
Quoted 20 lines · Nikolay Borisov
Here are 4 patches which fix a newly found split-brain scenario in the
METADATA_UUID_INCOMPATcode. They are mostly the identical with Su's original submission but I have reworked the changelogs and some function names. Hence, I retained his authorship of the patches.First 2 patches factor out some code with the hopes of making
find_fisda bit more readable and simplifying the myriad of nested 'if' indevice_list_add.Patch 3 extends
find_fsid_changedto handle the case where a disk withMETADATA_UUID_INCOMPATandFSID_CHANGING_IN_PROGRESSis scanned after a disk which has successfully been switched to FSID ==METADATA_UUIDstate and has createdbtrfs_fs_devices.Patch 4 handles the counterpart situation - a fully switched disk is scanned after one which has had
METADATA_UUID_INCOMPATandFSID_CHANGING_IN_PROGRESSset.This series should be applied to stable branches following 5.0 when the
metadata_uuidfeature got introduced.
Quoted 8 lines
This patchset was tested with btrfs-progs' misc 034-metadata-uuid test and a full xfstest run with no regressions. I will also be sending an improvement to the test case which exercises the newly added code.
btrfs: Call
find_fsidfromfind_fsid_inprogress
btrfs: Factor outmetadata_uuidcode fromfind_fsid.
btrfs: Handle another split brain scenario with metadata uuid feature
btrfs: Fix split-brain handling when changing FSID to metadata uuid
I'm adding it to for-next, thanks.
Quoted 25 lines · David Sterba
Quoted 24 lines · Nikolay Borisov
Here are 4 patches which fix a newly found split-brain scenario in the
METADATA_UUID_INCOMPATcode. They are mostly the identical with Su's original submission but I have reworked the changelogs and some function names. Hence, I retained his authorship of the patches.First 2 patches factor out some code with the hopes of making
find_fisda bit more readable and simplifying the myriad of nested 'if' indevice_list_add.Patch 3 extends
find_fsid_changedto handle the case where a disk withMETADATA_UUID_INCOMPATandFSID_CHANGING_IN_PROGRESSis scanned after a disk which has successfully been switched to FSID ==METADATA_UUIDstate and has createdbtrfs_fs_devices.Patch 4 handles the counterpart situation - a fully switched disk is scanned after one which has had
METADATA_UUID_INCOMPATandFSID_CHANGING_IN_PROGRESSset.This series should be applied to stable branches following 5.0 when the
metadata_uuidfeature got introduced.This patchset was tested with btrfs-progs' misc 034-metadata-uuid test and a full xfstest run with no regressions. I will also be sending an improvement to the test case which exercises the newly added code.
Quoted 6 lines
Quoted 4 lines
btrfs: Call
find_fsidfromfind_fsid_inprogress
btrfs: Factor outmetadata_uuidcode fromfind_fsid.
btrfs: Handle another split brain scenario with metadata uuid feature
btrfs: Fix split-brain handling when changing FSID to metadata uuidI'm adding it to for-next, thanks.
Moving to misc-next.