Skip to contentSkip to conversation

Linux RAID metadata format docs?

DISCUSSION Jeff Garzik 4 participants 6 messages Nov 14, 2003 – Nov 22, 2003 MD RAID

lore
Share & tools
lore
#0 lore raw

Do any docs exist, that describe the on-disk md format?

I'm talking with a vendor that would like to support booting off of "Linux RAID" (as they call it), from their card's BIOS, which implies the vendor would need to add support for the md format to their BIOS.

Jeff

#1 lore raw
Quoted 1 line · Jeff Garzik

Do any docs exist, that describe the on-disk md format?

Not as such. An H-file isn't a document in itself...

Quoted 5 lines

I'm talking with a vendor that would like to support booting off of "Linux RAID" (as they call it), from their card's BIOS, which implies the vendor would need to add support for the md format to their BIOS.

Jeff

Problem there is the multitude of partition schemes, and that RAID superblocks are inside the partitions...

Nevertheless I would love to be able to boot from _any_ of disks in a bootable RAID1 diskset. I have just experimented with Highpoint's RocketRaid 1520, which almost can do that. It fails if the primary disk is offline, but succeeds if secondary is out.
I also liked their superblock location - very early in disk. Negative thing in the HPT BIOS is, that if it detects a fault, it wants user input instead of just trying starting disks a bit more persistently, and continuing _soon_ the boot in degraded RAID1 mode without any user input.

After all, Sun has been able to do that for a very long time, it should be possible in PC hardware, too.
(In Suns we have to define a set of boot devices into prom level,
and there all devices are different, and retain their identity forever independent of plugging new interface cards into system, but that is another story...)

/Matti Aarnio

#2 lore raw
Quoted 5 lines · Matti Aarnio
Quoted 1 line · Jeff Garzik

Do any docs exist, that describe the on-disk md format?

Not as such. An H-file isn't a document in itself...

Quoted 8 lines
Quoted 5 lines

I'm talking with a vendor that would like to support booting off of "Linux RAID" (as they call it), from their card's BIOS, which implies the vendor would need to add support for the md format to their BIOS.

Jeff

Problem there is the multitude of partition schemes, and that RAID superblocks are inside the partitions...

Yep. Thought I consider it fairly pointless to support anything but ms-dos partitions and EFI partitions. Given limited development resources, I don't see a huge need for (for example) BSD disklabel support.

Quoted 9 lines

Nevertheless I would love to be able to boot from _any_ of disks in a bootable RAID1 diskset. I have just experimented with Highpoint's RocketRaid 1520, which almost can do that. It fails if the primary disk is offline, but succeeds if secondary is out.
I also liked their superblock location - very early in disk. Negative thing in the HPT BIOS is, that if it detects a fault, it wants user input instead of just trying starting disks a bit more persistently, and continuing _soon_ the boot in degraded RAID1 mode without any user input.

Yeah, the main point is to be able to root in degraded mode, as well as regular mode.

Jeff

#3 lore raw
In reply to Matti Aarnio
Quoted 3 lines · Matti Aarnio

Nevertheless I would love to be able to boot from _any_ of disks in a bootable RAID1 diskset. I have just experimented with Highpoint's RocketRaid 1520, which almost can do that. It fails if the primary

<snip>

Quoted 2 lines

After all, Sun has been able to do that for a very long time, it should be possible in PC hardware, too.

Nope, Sun's booting is achieved in the OBP (Open Boot Prom) which is arguably more advanced than the current PC BIOSes.

Quoted 1 line

(In Suns we have to define a set of boot devices into prom level,

Which is specified based on the PCI/SBUS slot the controller is attached to. One Day(TM) we'll be able to see the PeeCee world drop it's ISA slot independence but IRQ/IOport uniqueness, not to mention it's real mode DOS compatible 0x80, 0x81 harddisk ordering... :(

Quoted 3 lines

and there all devices are different, and retain their identity forever independent of plugging new interface cards into system, but that is another story...)

That's only a function of the order of entries in /etc/path_to_inst, which is achievable (to an extend) in linux with modules.conf trickery.

Hendrik

#4 lore raw
In reply to Jeff Garzik
Quoted 1 line · On Friday November 14, [email protected] wrote:

Do any docs exist, that describe the on-disk md format?

Not to my knowledge, beyond source code.

Quoted 4 lines

I'm talking with a vendor that would like to support booting off of "Linux RAID" (as they call it), from their card's BIOS, which implies the vendor would need to add support for the md format to their BIOS.

I don't see that knowledge of the RAID format is needed by the bios.

It might be needed by the boot sector, and certainly by the next stage (whether that be the kernel itself, or grub or whatever).

Based on my limited understanding, the bios should only need:
1/ to be able to try subsequent drives if the first fails, 2/ be able to tell the boot sector which drive was used 3/ to allow the boot sector to indicate failure
and maybe
4/ be able to have a dynamically determined boot order
(so that when one drive fails, it gets put to the bottom of the

    list).

Clearly 1,2,3 don't need the bios to know anything about RAID layout. 4 could use raid layout information, and an easy interface for the kernel to tell it when things have changed would be a lot easier and more general.

I think the simplest approach would be:

Bios attempts each potential boot device in some order, loading the boot sector and telling the boot sector that the device is "0x80". If the boot sector fails (e.g. bad code, such as all zeros), it tries the next.

The boot sector tries to load a kernel or second-stage off "0x80" as
normal. The kernel or second-stage understands raid and makes sure
the correct root is assembled.

Some daemon monitors drives that fail, and attempts to over-write sector 0 of any such drive that is a boot-candidate.

This wouldn't quite do the right thing if:
The "first" drive goes off line for some reason.
A new kernel/second-stage is installed on remaining drive(s). machine is rebooted and "first" drive comes back online during the boot.

The "first" drive will look OK, but will not have a current kernel on it.

Does any of this make sense?

Were you thinking of some RAID level other than RAID1?

NeilBrown

#5 lore raw
Quoted 6 lines · Neil Brown
Quoted 1 line · On Friday November 14, [email protected] wrote:

Do any docs exist, that describe the on-disk md format?

Not to my knowledge, beyond source code.

Thanks.

Quoted 1 line

I don't see that knowledge of the RAID format is needed by the bios.

[...]

Quoted 13 lines

The boot sector tries to load a kernel or second-stage off "0x80" as
normal. The kernel or second-stage understands raid and makes sure
the correct root is assembled.

Some daemon monitors drives that fail, and attempts to over-write sector 0 of any such drive that is a boot-candidate.

This wouldn't quite do the right thing if:
The "first" drive goes off line for some reason.
A new kernel/second-stage is installed on remaining drive(s). machine is rebooted and "first" drive comes back online during the boot.

Quoted 6 lines

The "first" drive will look OK, but will not have a current kernel on it.

Does any of this make sense?

Were you thinking of some RAID level other than RAID1?

All RAID levels, 1 and 5 particularly. To me, it boils down to a choice: support in the bootloader, and/or support in the BIOS. If the BIOS vendor is willing to support the _Linux_ md format, it seems to me that's a more reliable place to handle booting and failover in RAID1 or 5. OTOH, while coordinating multiple boot loaders (since one must predict behavior of lilo/grub in all raid1/5 failure scenarios) seems much more difficult, but ultimate not specific to one vendor.

By tradition the BIOS/firmware is the software that loads the bootloader. It feels like the right thing to me, for that to continue even in the face of RAID arrays. Users of "fake RAID" (i.e. the vendors market it as RAID, even though it's really software RAID) are used to their RAID arrays being accessible as a BIOS device, as weird as that may be :)

Jeff