Issue with btrfs balance
I just recently discovered something about btrfs filesystem balance that (as far as I can see) isn't documented anywhere, and doesn't necessarily have an obvious (to the average user) explanation.
Apparently, trying to use -mconvert=dup or -sconvert=dup on a multi-device filesystem using one of the RAID profiles for metadata fails with a statement to look at the kernel log, which doesn't show anything at all about the failure.
Based on what I've been able to understand from the source, it appears that the kernel stops you from converting to a dup profile for metadata in this case because it thinks that such a profile doesn't work on multiple devices, despite the fact that you can take a single device filesystem, and a device, and it will still work fine even without converting the metadata/system profiles.
I feel at the very least, this should be documented, and the kernel should give at least some indication of what went wrong. Ideally, this should be changed to allow converting to dup so that when converting a multi-device filesystem to single-device, you never have to have metadata or system chunks use a single profile.
I've experienced the following with balance:
Setup:
- Kernel 3.12.9
- 11 DVD sized (4.3GB) loopback devices.
(9 Read-Only Seed devices + 2 Read/Write devices)
- 9 device seed created with "-m single -d single" and made Read-only with "btrfstune -S 1 ..."
- 2 devices was added at different dates. NO balance performed until now.
- NOW add 1 more device to the array and perform a balance.
Result:
Balance did run for a while and exited displaying "Process Killed". Any attempt to unmount the array failed, preventing any shutdown. Hence I had no option other than hard reset.
After reboot, issuing balance command gave the message "Balance in progress".
I cancelled the balance and tried to remove the last device which ended with a kernel crash. So I dumped 2 + 1 normal devices.
Former 9 multi device seed was OK and was mountable.
Regards,
Imran
Quoted 5 lines · Austin S Hemmelgarn
<snip>
Apparently, trying to use -mconvert=dup or -sconvert=dup on a multi-device filesystem using one of the RAID profiles for metadata fails with a statement to look at the kernel log, which doesn't show anything at all about the failure.
^ If this is the case then it is definitely a bug. Can you provide some version info? Specifically kernel, btrfs-tools, and Distro.
Quoted 6 lines
<snip> it appears
that the kernel stops you from converting to a dup profile for metadata in this case because it thinks that such a profile doesn't work on multiple devices, despite the fact that you can take a single device filesystem, and a device, and it will still work fine even without converting the metadata/system profiles.
I believe dup used to work on multiple devices but the facility was removed. In the standard case it doesn't make sense to use dup with multiple devices: It uses the same amount of diskspace but is more vulnerable than the RAID1 alternative.
Quoted 4 lines
<snip> Ideally, this
should be changed to allow converting to dup so that when converting a multi-device filesystem to single-device, you never have to have metadata or system chunks use a single profile.
This is a good use-case for having the facility. I'm thinking that, if it is brought back in, the only caveat is that appropriate warnings should be put in place to indicate that it is inappropriate.
My guess on how you'd like to migrate from raid1/raid1 to single/dup, assuming sda and sdb:
btrfs balance start -dconvert=single -mconvert=dup /
btrfs device delete /dev/sdb /
Signature · 4 lines
__________ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97
Quoted 8 lines · Brendan Hide
Quoted 5 lines · Austin S Hemmelgarn
<snip>
Apparently, trying to use -mconvert=dup or -sconvert=dup on a multi-device filesystem using one of the RAID profiles for metadata fails with a statement to look at the kernel log, which doesn't show anything at all about the failure.^ If this is the case then it is definitely a bug. Can you provide some version info? Specifically kernel, btrfs-tools, and Distro.
In this case, btrfs-progs 3.12, kernel 3.13.2, and Gentoo.
Quoted 14 lines
Quoted 6 lines
<snip> it appears
that the kernel stops you from converting to a dup profile for metadata in this case because it thinks that such a profile doesn't work on multiple devices, despite the fact that you can take a single device filesystem, and a device, and it will still work fine even without converting the metadata/system profiles.I believe dup used to work on multiple devices but the facility was removed. In the standard case it doesn't make sense to use dup with multiple devices: It uses the same amount of diskspace but is more vulnerable than the RAID1 alternative.
Quoted 4 lines
<snip> Ideally, this
should be changed to allow converting to dup so that when converting a multi-device filesystem to single-device, you never have to have metadata or system chunks use a single profile.
Quoted 9 lines
This is a good use-case for having the facility. I'm thinking that, if it is brought back in, the only caveat is that appropriate warnings should be put in place to indicate that it is inappropriate.
My guess on how you'd like to migrate from raid1/raid1 to single/dup, assuming sda and sdb:
btrfs balance start -dconvert=single -mconvert=dup /
btrfs device delete/dev/sdb/
Ideally, yes. The exact command I tried to use was:
btrfs balance start -dconvert=single -mconvert=dup -sconvert=dup -f -v / Trying again without the system chunk conversion also failed.
Quoted 22 lines · Brendan Hide
Quoted 5 lines · Austin S Hemmelgarn
<snip>
Apparently, trying to use -mconvert=dup or -sconvert=dup on a multi-device filesystem using one of the RAID profiles for metadata fails with a statement to look at the kernel log, which doesn't show anything at all about the failure.^ If this is the case then it is definitely a bug. Can you provide some version info? Specifically kernel, btrfs-tools, and Distro.
Quoted 6 lines
<snip> it appears
that the kernel stops you from converting to a dup profile for metadata in this case because it thinks that such a profile doesn't work on multiple devices, despite the fact that you can take a single device filesystem, and a device, and it will still work fine even without converting the metadata/system profiles.I believe dup used to work on multiple devices but the facility was removed. In the standard case it doesn't make sense to use dup with multiple devices: It uses the same amount of diskspace but is more vulnerable than the RAID1 alternative.
Quoted 4 lines
<snip> Ideally, this
should be changed to allow converting to dup so that when converting a multi-device filesystem to single-device, you never have to have metadata or system chunks use a single profile.
Quoted 9 lines
This is a good use-case for having the facility. I'm thinking that, if it is brought back in, the only caveat is that appropriate warnings should be put in place to indicate that it is inappropriate.
My guess on how you'd like to migrate from raid1/raid1 to single/dup, assuming sda and sdb:
btrfs balance start -dconvert=single -mconvert=dup /
btrfs device delete/dev/sdb/
Do you happen to know which git repository and branch is preferred to base patches on? I'm getting ready to write one to fix this, and would like to make it as easy as possible for the developers to merge.
Quoted 4 lines · Austin S. Hemmelgarn
Quoted 1 line · Austin S Hemmelgarn
Do you happen to know which git repository and branch is preferred to base patches on? I'm getting ready to write one to fix this, and would like to make it as easy as possible for the developers to merge.
A list of the "main" repositories is maintained at https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories
I'd suggest David Sterba's branch as he maintains it for userspace-tools integration.
Signature · 4 lines
__________ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97
Quoted 6 lines · Brendan Hide
Quoted 5 lines · Austin S. Hemmelgarn
Quoted 1 line · Austin S Hemmelgarn
Do you happen to know which git repository and branch is preferred to base patches on? I'm getting ready to write one to fix this, and would like to make it as easy as possible for the developers to merge.
Quoted 6 lines
A list of the "main" repositories is maintained at
https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositoriesI'd suggest David Sterba's branch as he maintains it for userspace-tools integration.
In this case, it will need to be patched both in the userspace tools, and in the kernel, it's the kernel itself that prevents the balance, cause it thinks that you can't do dup profiles with multiple devices.
PATCH 1/1 Allow forced conversion of metadata to dup profile on multiple devices
Currently, btrfs balance start fails when trying to convert metadata or system chunks to dup profile on filesystems with multiple devices. This requires that a conversion from a multi-device filesystem to a single device filesystem use the following methodology:
1. btrfs balance start -dconvert=single -mconvert=single \
-sconvert=single -f /
2. btrfs device delete / /dev/sdx
3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are big) where you don't have the protection guarantees of multiple copies of metadata chunks.
After applying this patch, one can instead use the following methodology for conversion from a multi-device filesystem to a single device filesystem:
1. btrfs balance start -dconvert=single -mconvert=dup \
-sconvert=dup -f /
2. btrfs device delete / /dev/sdx
This greatly reduces the chances of the operation causing data loss due to a read error during the device delete.
--- fs/btrfs/volumes.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)
fs/btrfs/volumes.c +17 -4
@@ -3152,10 +3152,8 @@ int btrfs_balance(struct btrfs_balance_control *bctl, num_devices--; }btrfs_dev_replace_unlock (&fs_info->dev_replace); - allowed =BTRFS_AVAIL_ALLOC_BIT_SINGLE ; -if (num_devices ==1 ) - allowed |=BTRFS_BLOCK_GROUP_DUP ; -else if (num_devices >1 ) + allowed =BTRFS_AVAIL_ALLOC_BIT_SINGLE |BTRFS_BLOCK_GROUP_DUP ; +if (num_devices >1 ) allowed |= (BTRFS_BLOCK_GROUP_RAID0 |BTRFS_BLOCK_GROUP_RAID1 );if (num_devices >2 ) allowed |=BTRFS_BLOCK_GROUP_RAID5 ;
@@ -3221,6 +3219,21 @@ int btrfs_balance(struct btrfs_balance_control *bctl, goto out; } } +if (((bctl->sys.flags &BTRFS_BALANCE_ARGS_CONVERT ) && + (bctl->sys.target & ~BTRFS_BLOCK_GROUP_DUP ) || + (bctl->meta.flags &BTRFS_BALANCE_ARGS_CONVERT ) && + (bctl->meta.target & ~BTRFS_BLOCK_GROUP_DUP )) && + (num_devs >1 )) { +if (bctl->flags &BTRFS_BALANCE_FORCE ) { +btrfs_info (fs_info,"force conversion of metadata " +"to dup profile on multiple devices" ); + }else { +btrfs_err (fs_info,"balance will reduce metadata " +"integrity, use force if you want this" ); + ret = -EINVAL ; + goto out; + } + } }while (read_seqretry (&fs_info->profiles_lock, seq));if (bctl->sys.flags &BTRFS_BALANCE_ARGS_CONVERT ) {
Signature · 1 line
1.8.5.4