btrfs: Allow forced conversion of metadata to dup profile on multiple devices
PATCH 1/1 btrfs: Allow forced conversion of metadata to dup profile on multiple devices
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete / /dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.
After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete / /dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.
--- fs/btrfs/volumes.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)
fs/btrfs/volumes.c +0 -0
@@ -3152,10 +3152,8 @@ int btrfs_balance(struct btrfs_balance_control
*bctl,
num_devices--;
}
btrfs_dev_replace_unlock(&fs_info->dev_replace);
- allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE;
- if (num_devices == 1)
- allowed |= BTRFS_BLOCK_GROUP_DUP;
- else if (num_devices > 1)
+ allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE | BTRFS_BLOCK_GROUP_DUP;
+ if (num_devices > 1)
allowed |= (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1);
if (num_devices > 2)
allowed |= BTRFS_BLOCK_GROUP_RAID5;
@@ -3221,6 +3219,21 @@ int btrfs_balance(struct btrfs_balance_control *bctl,
goto out;
}
}
+ if (((bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) &&
+ (bctl->sys.target & ~BTRFS_BLOCK_GROUP_DUP) ||
+ (bctl->meta.flags & BTRFS_BALANCE_ARGS_CONVERT) &&
+ (bctl->meta.target & ~BTRFS_BLOCK_GROUP_DUP)) &&
+ (num_devs > 1)) {+ if (bctl->flags & BTRFS_BALANCE_FORCE) {
+ btrfs_info(fs_info, "force conversion of metadata "
+ "to dup profile on multiple devices");
+ } else {
+ btrfs_err(fs_info, "balance will reduce metadata "
+ "integrity, use force if you want this");
+ ret = -EINVAL;
+ goto out;
+ }
+ }
} while (read_seqretry(&fs_info->profiles_lock, seq)); if (bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) {
Signature · 1 line
1.8.5.4
Quoted 21 lines · Austin S Hemmelgarn
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete //dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete //dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.
Quoted 1 line
Signed-off-by Austin S. Hemmelgarn <[email protected]>
Sounds useful. The muliple devices + DUP is allowed setup when the device is added, this patch only adds the 'delete' counterpart. The imroved data loss protection during the process is a good thing.
Quoted 22 lines · David Sterba
Quoted 21 lines · Austin S Hemmelgarn
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete //dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete //dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.
Quoted 6 lines
Quoted 1 line
Signed-off-by Austin S. Hemmelgarn <[email protected]>Reviewed-by David Sterba <[email protected]>Sounds useful. The muliple devices + DUP is allowed setup when the device is added, this patch only adds the 'delete' counterpart. The imroved data loss protection during the process is a good thing.
Hi,
Have you actually tried to queue it? Unless I'm missing something, it won't compile, and on top of that, it seems to be corrupted too..
IIRC muliple devices + DUP is "allowed" only until the first balance, has that changed?
Thanks,
Ilya
Quoted 26 lines · Ilya Dryomov
Quoted 25 lines · David Sterba
Quoted 21 lines · Austin S Hemmelgarn
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete //dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete //dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.Quoted 1 line
Signed-off-by Austin S. Hemmelgarn <[email protected]>Reviewed-by David Sterba <[email protected]>
Quoted 8 lines
Quoted 3 lines
Sounds useful. The muliple devices + DUP is allowed setup when the device is added, this patch only adds the 'delete' counterpart. The imroved data loss protection during the process is a good thing.
Hi,
Have you actually tried to queue it? Unless I'm missing something, it won't compile, and on top of that, it seems to be corrupted too..
The patch itself was made using git, AFAICT it should be fine. I've personally built and tested it using UML.
Quoted 3 lines
IIRC muliple devices + DUP is "allowed" only until the first balance, has that changed?
This is just a limitation of how the kernel handles balances, DUP profiles with multiple devices work, it's just terribly inefficient. The primary use case is converting a multi-device FS with RAID for metadata to a single device FS without having to reduce integrity.
Quoted 4 lines
Thanks,
Ilya
Quoted 31 lines · Austin S Hemmelgarn
Quoted 30 lines · Ilya Dryomov
Quoted 28 lines · David Sterba
Quoted 21 lines · Austin S Hemmelgarn
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete //dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete //dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.Quoted 1 line
Signed-off-by Austin S. Hemmelgarn <[email protected]>Reviewed-by David Sterba <[email protected]>Sounds useful. The muliple devices + DUP is allowed setup when the device is added, this patch only adds the 'delete' counterpart. The imroved data loss protection during the process is a good thing.
Quoted 6 lines
Quoted 4 lines
Hi,
Have you actually tried to queue it? Unless I'm missing something, it won't compile, and on top of that, it seems to be corrupted too..
The patch itself was made using git, AFAICT it should be fine. I've personally built and tested it using UML.
It doesn't look fine. It was generated with git, but it got corrupted on the way: either how you pasted it or the email client you use is the problem.
Quoted 31 lines · Austin S Hemmelgarn
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete //dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete //dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.Signed-off-by Austin S. Hemmelgarn <[email protected]>--- fs/btrfs/volumes.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)fs/btrfs/volumes.c +0
-0
fs/btrfs/volumes.c int btrfs_balance(struct btrfs_balance_control · line 3152
Quoted 1 line
fs/btrfs/volumes.c +0
-0@@ -3152,10 +3152,8 @@ int btrfs_balance(struct btrfs_balance_control *bctl,
^^^, that should be a single line
Quoted 13 lines
fs/btrfs/volumes.c +2
-4num_devices--; }btrfs_dev_replace_unlock (&fs_info->dev_replace); - allowed =BTRFS_AVAIL_ALLOC_BIT_SINGLE ; -if (num_devices ==1 ) - allowed |=BTRFS_BLOCK_GROUP_DUP ; -else if (num_devices >1 ) + allowed =BTRFS_AVAIL_ALLOC_BIT_SINGLE |BTRFS_BLOCK_GROUP_DUP ; +if (num_devices >1 ) allowed |= (BTRFS_BLOCK_GROUP_RAID0 |BTRFS_BLOCK_GROUP_RAID1 );if (num_devices >2 )allowed |= BTRFS_BLOCK_GROUP_RAID5;
fs/btrfs/volumes.c int btrfs_balance(struct btrfs_balance_control · line 3219
Quoted 1 line
fs/btrfs/volumes.c +0
-0@@ -3221,6 +3219,21 @@ int btrfs_balance(struct btrfs_balance_control *bctl,
^^^, ditto
Quoted 10 lines
fs/btrfs/volumes.c +7
-0goto out; } } +if (((bctl->sys.flags &BTRFS_BALANCE_ARGS_CONVERT ) && + (bctl->sys.target & ~BTRFS_BLOCK_GROUP_DUP ) || + (bctl->meta.flags &BTRFS_BALANCE_ARGS_CONVERT ) && + (bctl->meta.target & ~BTRFS_BLOCK_GROUP_DUP )) && + (num_devs >1 )) { +if (bctl->flags &BTRFS_BALANCE_FORCE ) { + btrfs_info(fs_info, "force conversion of metadata "
fs/btrfs/volumes.c int btrfs_balance(struct btrfs_balance_control · line 3239
Quoted 10 lines
fs/btrfs/volumes.c +8
-0+ "to dup profile on multiple devices"); + }else { +btrfs_err (fs_info,"balance will reduce metadata " +"integrity, use force if you want this" ); + ret = -EINVAL ; + goto out; + } + } }while (read_seqretry (&fs_info->profiles_lock, seq));if (bctl->sys.flags &BTRFS_BALANCE_ARGS_CONVERT ) {
^^^, there should be 3 lines of context here. It looks like an empty line between "} while ..." and "if (bctl..." is missing.
There is probably more, those just stand out.
Thanks,
Ilya
Quoted 119 lines · Ilya Dryomov
Quoted 31 lines · Austin S Hemmelgarn
Quoted 30 lines · Ilya Dryomov
Quoted 28 lines · David Sterba
Quoted 21 lines · Austin S Hemmelgarn
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete //dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete //dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.Quoted 1 line
Signed-off-by Austin S. Hemmelgarn <[email protected]>Reviewed-by David Sterba <[email protected]>Sounds useful. The muliple devices + DUP is allowed setup when the device is added, this patch only adds the 'delete' counterpart. The imroved data loss protection during the process is a good thing.
Quoted 6 lines
Quoted 4 lines
Hi,
Have you actually tried to queue it? Unless I'm missing something, it won't compile, and on top of that, it seems to be corrupted too..
The patch itself was made using git, AFAICT it should be fine. I've personally built and tested it using UML.
It doesn't look fine. It was generated with git, but it got corrupted on the way: either how you pasted it or the email client you use is the problem.
Quoted 31 lines · Austin S Hemmelgarn
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete //dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete //dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.Signed-off-by Austin S. Hemmelgarn <[email protected]>--- fs/btrfs/volumes.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)fs/btrfs/volumes.c +0
-0fs/btrfs/volumes.c int btrfs_balance(struct btrfs_balance_control · line 3152
Quoted 1 line
fs/btrfs/volumes.c +0
-0@@ -3152,10 +3152,8 @@ int btrfs_balance(struct btrfs_balance_control *bctl,^^^, that should be a single line
Quoted 13 lines
fs/btrfs/volumes.c +2
-4num_devices--; }btrfs_dev_replace_unlock (&fs_info->dev_replace); - allowed =BTRFS_AVAIL_ALLOC_BIT_SINGLE ; -if (num_devices ==1 ) - allowed |=BTRFS_BLOCK_GROUP_DUP ; -else if (num_devices >1 ) + allowed =BTRFS_AVAIL_ALLOC_BIT_SINGLE |BTRFS_BLOCK_GROUP_DUP ; +if (num_devices >1 ) allowed |= (BTRFS_BLOCK_GROUP_RAID0 |BTRFS_BLOCK_GROUP_RAID1 );if (num_devices >2 )allowed |= BTRFS_BLOCK_GROUP_RAID5;fs/btrfs/volumes.c int btrfs_balance(struct btrfs_balance_control · line 3219
Quoted 1 line
fs/btrfs/volumes.c +0
-0@@ -3221,6 +3219,21 @@ int btrfs_balance(struct btrfs_balance_control *bctl,^^^, ditto
Quoted 10 lines
fs/btrfs/volumes.c +7
-0goto out; } } +if (((bctl->sys.flags &BTRFS_BALANCE_ARGS_CONVERT ) && + (bctl->sys.target & ~BTRFS_BLOCK_GROUP_DUP ) || + (bctl->meta.flags &BTRFS_BALANCE_ARGS_CONVERT ) && + (bctl->meta.target & ~BTRFS_BLOCK_GROUP_DUP )) && + (num_devs >1 )) { +if (bctl->flags &BTRFS_BALANCE_FORCE ) { + btrfs_info(fs_info, "force conversion of metadata "fs/btrfs/volumes.c int btrfs_balance(struct btrfs_balance_control · line 3239
Quoted 10 lines
fs/btrfs/volumes.c +8
-0+ "to dup profile on multiple devices"); + }else { +btrfs_err (fs_info,"balance will reduce metadata " +"integrity, use force if you want this" ); + ret = -EINVAL ; + goto out; + } + } }while (read_seqretry (&fs_info->profiles_lock, seq));if (bctl->sys.flags &BTRFS_BALANCE_ARGS_CONVERT ) {
Quoted 9 lines
^^^, there should be 3 lines of context here. It looks like an empty line between "} while ..." and "if (bctl..." is missing.
There is probably more, those just stand out.
Thanks,
Ilya
I'll rebase and regenerate the patch tonight when I get off of work, I'm thinking my mail client might have mangled it somehow.