What's the status of BTRFS RST instead of linux md raid5?
So, I'm aware that the built in btrfs raid5 support that's been there for 10-ish years is not recommended due to hard to fix or unfixable flaws.
But with raid5, once I have a checksum mismatch, btrfs could easily run all combinations of the blocks to find out which one can be removed to yield a correct checksum for that data block, but obviously those layers don't talk, so nothing happens outside of btrfs telling me the checksum is bad, and there is no good way to rebuild anything, especially when it's bit rot in a read only snapshot where I cannot reasonably delete or rewrite the file without some surgery.
Gemini tells me that RST with the future that fixes this so I can use raid5 or raid6 directly in btrfs where everything talks to one another, but I can't find good docs or status on RST.
Even this list has very little about RST outside of some old patches.
So, is RST kind of ready and documented, or is it still very experimental and "read the source, luke" ?
Thanks,
Marc
Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R. Home page: http://marc.merlins.org/
Quoted 12 lines · Marc MERLIN
So, I'm aware that the built in btrfs raid5 support that's been there for 10-ish years is not recommended due to hard to fix or unfixable flaws.
But with raid5, once I have a checksum mismatch, btrfs could easily run all combinations of the blocks to find out which one can be removed to yield a correct checksum for that data block, but obviously those layers don't talk, so nothing happens outside of btrfs telling me the checksum is bad, and there is no good way to rebuild anything, especially when it's bit rot in a read only snapshot where I cannot reasonably delete or rewrite the file without some surgery.
Quoted 7 lines
Gemini tells me that RST with the future that fixes this so I can use raid5 or raid6 directly in btrfs where everything talks to one another, but I can't find good docs or status on RST.
Even this list has very little about RST outside of some old patches.So, is RST kind of ready and documented, or is it still very experimental and "read the source, luke" ?
RST is still considered experimental and the development on the RAID5/6 support is unfortunately delayed due to different more urgent fires that have to be extinguished.
Note while RST can handle non zoned devices, it's primary focus always was always to enable RAID on zoned block devices. Which it does for RAID0, RAID1 and RAID10.
As for RAID5 I have a prototype doing writes and reads for the trivial cases (i.e. full stripe write/read).
My apologies to everyone waiting for this. I haven't abandoned the project, I just need more time to get it done.
If you're looking for documentation please have a look here: https://github.com/kdave/btrfs-progs/blob/devel/Documentation/dev/design-raid-stripe-tree.txt
Quoted 9 lines · Johannes Thumshirn
RST is still considered experimental and the development on the RAID5/6 support is unfortunately delayed due to different more urgent fires that have to be extinguished.
Note while RST can handle non zoned devices, it's primary focus always was always to enable RAID on zoned block devices. Which it does for RAID0, RAID1 and RAID10.
Quoted 8 lines
As for RAID5 I have a prototype doing writes and reads for the trivial cases (i.e. full stripe write/read).
My apologies to everyone waiting for this. I haven't abandoned the project, I just need more time to get it done.
If you're looking for documentation please have a look here: https://github.com/kdave/btrfs-progs/blob/devel/Documentation/dev/design-raid-stripe-tree.txt
Thank you so much for the status update, *much* appreciated. As for when, this is all unpaid work I assume, so it gets done when it gets done, no worries.
For what it's worth, I've ben doing mdadm raid5 with btrfs since 2014, and unfortuantely it's failed in too many ways, almost each time causing a btrfs filesystem that was impossible to recover (usually ro mount was ok, but impossible to get back in read/write state, which is very inconvenient for a 20TB+ filesystem dueto how long it takes to restore backups)
As luck would have it, this just happened to me again
https://yhbt.net/lore/linux-btrfs/[email protected]/
I know I don't know the deep insides of the btrfs code, but it's unfortunately failed enough times on top of mdadm raid5 that I can't wait to switch to RST when it's ready. It's been 12 years already, so I can wait a bit longer still :)
Marc
Signature · 3 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R. Home page: http://marc.merlins.org/ | PGP 7F55D5F27AAF9D08