balance conversion from RAID-1 to RAID-10 leaves some metadata in RAID-1?
Just finished a conversion from RAID-1 to RAID-10, on Linux 5.5.6:
# time btrfs balance start -dconvert=raid10 -mconvert=raid10 /data/lxd Done, had to relocate 2064 out of 2064 chunks
real 1811m51.427s user 0m0.000s sys 1259m34.448s
# echo $? 0
# dmesg (...) [211862.205521] BTRFS info (device sda2): found 40805 extents [211910.773888] BTRFS info (device sda2): found 40805 extents [211965.321150] BTRFS info (device sda2): relocating block group 3492979671040 flags data|raid1 [211972.743476] BTRFS info (device sda2): found 37221 extents [212020.193342] BTRFS info (device sda2): found 37221 extents [212084.580739] BTRFS info (device sda2): balance: ended with status: 0
To my surprise, some metadata is still RAID-1 - is it expected?
# btrfs fi usage /data/lxd
Overall:
Device size: 6.91TiB
Device allocated: 6.70TiB
Device unallocated: 210.40GiB
Device missing: 0.00B
Used: 3.96TiB
Free (estimated): 1.46TiB (min: 1.46TiB)
Data ratio: 2.00
Metadata ratio: 2.00
Global reserve: 512.00MiB (used: 0.00B)Data,RAID10: Size:3.32TiB, Used:1.97TiB (59.20%)
/dev/sda2 1.66TiB /dev/sdd2 1.66TiB /dev/sdc2 1.66TiB /dev/sdb2 1.66TiB
Metadata,RAID1: Size:2.00GiB, Used:85.02MiB (4.15%)
/dev/sdc2 2.00GiB
/dev/sdb2 2.00GiB
Metadata,RAID10: Size:29.00GiB, Used:12.55GiB (43.27%)
/dev/sda2 14.50GiB /dev/sdd2 14.50GiB /dev/sdc2 14.50GiB /dev/sdb2 14.50GiB
System,RAID10: Size:64.00MiB, Used:400.00KiB (0.61%)
/dev/sda2 32.00MiB /dev/sdd2 32.00MiB /dev/sdc2 32.00MiB /dev/sdb2 32.00MiB
Unallocated:
/dev/sda2 53.60GiB /dev/sdd2 53.60GiB /dev/sdc2 51.60GiB /dev/sdb2 51.60GiB
# btrfs fi df /data/lxd Data, RAID10: total=3.32TiB, used=1.97TiB System, RAID10: total=64.00MiB, used=400.00KiB Metadata, RAID10: total=29.00GiB, used=12.55GiB Metadata, RAID1: total=2.00GiB, used=85.02MiB GlobalReserve, single: total=512.00MiB, used=0.00B
# btrfs fi show /data/lxd
Label: 'lxd5' uuid: 2b77b498-a644-430b-9dd9-2ad3d381448a
Total devices 4 FS bytes used 1.98TiB
devid 1 size 1.73TiB used 1.67TiB path /dev/sda2
devid 3 size 1.73TiB used 1.67TiB path /dev/sdd2
devid 4 size 1.73TiB used 1.68TiB path /dev/sdc2
devid 5 size 1.73TiB used 1.68TiB path /dev/sdb2Tomasz Chmielewski
https://lxadm.com
Quoted 2 lines · Tomasz Chmielewski
To my surprise, some metadata is still RAID-1 - is it expected?
I'd say it's not expected but also balance is pretty complicated. Try
'btrfs balance start -dconvert=raid10,soft /mountpoint/'
What does dmesg report for that command? And are those raid1 bg's converted to raid10? I don't think it should matter in this case, but what's the btrfs-progs version?
Signature · 1 line
Chris Murphy
Quoted 11 lines · Chris Murphy
Quoted 2 lines · Tomasz Chmielewski
To my surprise, some metadata is still RAID-1 - is it expected?
I'd say it's not expected but also balance is pretty complicated. Try
'btrfs balance start -dconvert=raid10,soft /mountpoint/'
What does dmesg report for that command? And are those raid1 bg's converted to raid10? I don't think it should matter in this case, but what's the btrfs-progs version?
I did one more balance, for metadata only:
btrfs balance start -dconvert=raid10 /mountpoint/
And it fully converted the remaining RAID-1 metadata to RAID-10.
Still, a bit "weird".
btrfs-progs is 5.4.1.
Tomasz Chmielewski
https://lxadm.com
Quoted 11 lines · Tomasz Chmielewski
Quoted 9 lines · Chris Murphy
Quoted 2 lines · Tomasz Chmielewski
To my surprise, some metadata is still RAID-1 - is it expected?
I'd say it's not expected but also balance is pretty complicated. Try
'btrfs balance start -dconvert=raid10,soft /mountpoint/'
Quoted 7 lines
Quoted 3 lines
What does dmesg report for that command? And are those raid1 bg's converted to raid10? I don't think it should matter in this case, but what's the btrfs-progs version?
I did one more balance, for metadata only:
btrfs balance start -dconvert=raid10 /mountpoint/
Sorry mine had a typo too; should be '-mconvert=raid10,soft /mountpoint/'
Quoted 4 lines
And it fully converted the remaining RAID-1 metadata to RAID-10.
Still, a bit "weird".
Yep, I think Hugo's guess is a likely explanation.
Signature · 1 line
Chris Murphy
Quoted 24 lines · Tomasz Chmielewski
Just finished a conversion from RAID-1 to RAID-10, on Linux 5.5.6:
# time btrfs balance start -dconvert=raid10 -mconvert=raid10 /data/lxd Done, had to relocate 2064 out of 2064 chunksreal 1811m51.427s user 0m0.000s sys 1259m34.448s# echo $? 0# dmesg (...) [211862.205521] BTRFS info (device sda2): found 40805 extents [211910.773888] BTRFS info (device sda2): found 40805 extents [211965.321150] BTRFS info (device sda2): relocating block group 3492979671040 flags data|raid1 [211972.743476] BTRFS info (device sda2): found 37221 extents [212020.193342] BTRFS info (device sda2): found 37221 extents [212084.580739] BTRFS info (device sda2): balance: ended with status: 0
Quoted 1 line
To my surprise, some metadata is still RAID-1 - is it expected?
It's definitely something that can happen. I think what happens is
that new chunks are created while the balance process is going, so they never get picked up by the converter. I'd guess it's most likely to happen on an FS with active writes on it while the balance is running.
If it happens again, you can finish off just the unconverted chunks
by adding the "soft" option to the conversion filter:
# btrfs balance start -dconvert=raid10,sort -mconvert=raid10,soft /data/lxd
which will convert only those chunks that aren't already in the target RAID level.
Hugo.
Quoted 62 lines
# btrfs fi usage /data/lxd Overall: Device size: 6.91TiB Device allocated: 6.70TiB Device unallocated: 210.40GiB Device missing: 0.00B Used: 3.96TiB Free (estimated): 1.46TiB (min: 1.46TiB) Data ratio: 2.00 Metadata ratio: 2.00 Global reserve: 512.00MiB (used: 0.00B)Data,RAID10: Size:3.32TiB, Used:1.97TiB (59.20%)
/dev/sda2 1.66TiB /dev/sdd2 1.66TiB /dev/sdc2 1.66TiB /dev/sdb2 1.66TiBMetadata,RAID1: Size:2.00GiB, Used:85.02MiB (4.15%)
/dev/sdc22.00GiB
/dev/sdb22.00GiBMetadata,RAID10: Size:29.00GiB, Used:12.55GiB (43.27%)
/dev/sda2 14.50GiB /dev/sdd2 14.50GiB /dev/sdc2 14.50GiB /dev/sdb2 14.50GiBSystem,RAID10: Size:64.00MiB, Used:400.00KiB (0.61%)
/dev/sda2 32.00MiB /dev/sdd2 32.00MiB /dev/sdc2 32.00MiB /dev/sdb2 32.00MiBUnallocated:
/dev/sda2 53.60GiB /dev/sdd2 53.60GiB /dev/sdc2 51.60GiB /dev/sdb2 51.60GiB# btrfs fi df /data/lxd Data, RAID10: total=3.32TiB, used=1.97TiB System, RAID10: total=64.00MiB, used=400.00KiB Metadata, RAID10: total=29.00GiB, used=12.55GiB Metadata, RAID1: total=2.00GiB, used=85.02MiB GlobalReserve, single: total=512.00MiB, used=0.00B# btrfs fi show /data/lxd Label: 'lxd5' uuid: 2b77b498-a644-430b-9dd9-2ad3d381448a Total devices 4 FS bytes used 1.98TiB devid 1 size 1.73TiB used 1.67TiB path /dev/sda2 devid 3 size 1.73TiB used 1.67TiB path /dev/sdd2 devid 4 size 1.73TiB used 1.68TiB path /dev/sdc2 devid 5 size 1.73TiB used 1.68TiB path /dev/sdb2Tomasz Chmielewski
https://lxadm.com
Signature · 4 lines
Hugo Mills | The problem with programmers and the law is not that hugo@... carfax.org.uk | they treat laws as code, but that they don't http://carfax.org.uk/ | understand the VM it runs on. PGP: E2AB1DE4 |