XRAID - declustered parity RAID for (zoned) XFS
We're happy to announce the initial public release of XRAID - declustered parity RAID integrated into XFS.
The two primary aims of XRAID are to support parity RAID on zoned devices like SMR hard drives, and to provide a fast RAID implementation without inherent data integrity problems. XRAID, just like zoned XFS also works on conventional hard drives and SSDs. Optimizing for SSD performance has not been a focus, but even without that, writes are up to 20 times faster than mdraid.
XRAID builds on the zoned allocator, which uses the RT subvolume to store file data separately from metadata, so that only file data is placed on sequential write required zones while metadata is stored on conventional devices or zones.
XRAID takes advantage of this organization to only provide parity RAID for file data, where parity RAID works efficiently. Metadata redundancy is delegated to existing stacked Linux drivers like MD RAID mirroring, which is more suitable than parity RAID for small objects that are updated in place.
The RAID implementation sits below the Zoned XFS I/O path, similar to a stacked block driver, and aggregates the physical zones of the underlying devices into logical zones that are used by the Zoned XFS as realtime groups.
Declustered RAID describes a variant of RAID that distributes the D + P data and parity blocks in each stripe across a storage pool that contains more than D + P disks so that all disks participate in read, write and reconstruction operations and spare capacity is distributed over all disks.
Declustered RAID provides much faster rebuild times than conventional RAID, and eases using of a large number of disks in a single storage pool. XRAID can rebuild to all devices that have spare capacity, and the rebuild order is dynamic, allowing to rebuild zones with multiple lost redundancies first, and rebuilding data zones before parity zones to reduce the overhead of degraded reads.
Work on XRAID has been ongoing for a year and a half, and while we aren't anywhere near ready for an upstream submission, we are interested in feedback and testing from a wider audience now. The current implementation supports zero to 2 parities. We did experiments with the SnapRAID parity library to support 3 and more parities, and while that is not included in the current version, support for more parities is planned and the code is prepared for it.
The xfsprogs branch includes mkfs support and a new "xraid" utility for managing devices and inspecting an XRAID file-systems. We have also extended Zonar (The zoned file-system visualisation tool) to show how each logical zone maps onto physical zones on devices.
A presentation at FOSSY this year has more information including benchmark numbers on zoned and conventional devices:
https://verein.lst.de/~hch/raid-sucks.pdf
The code repositories are available here:
git://git.infradead.org/users/hch/linux-xraid.git
git://git.infradead.org/users/hch/xfsprogs-xraid.git
git://git.infradead.org/users/hch/xfstests-xraid.git
Gitweb:
https://git.infradead.org/users/hch/linux-xraid.git/shortlog/refs/heads/xfs-raid
https://git.infradead.org/users/hch/xfsprogs-xraid.git/shortlog/refs/heads/xfs-raid
https://git.infradead.org/users/hch/xfstests-xraid.git/shortlog/refs/heads/xfs-raid
Feel feel to join our [email protected] mailing list here: