getting a linux boot loader (preferably grub) installed on an intel imsm raid

Hi all,

/// The problem ///

There are binary packages out there that do the trick based on device-mapper (dmraid) (e.g. Ubuntu). But I would like to do it

a) manually (because thats why I use Linux, because you can do things
manually and you can learn a lot (even if you don't succeed in the end)

b) based on mdadm-imsm. Just there seems really /no/ documentation out
there on how to do it (or I successfully managed to google around it since days now ;).

I will be very glad for any bit of information that might help me to eliminate some of the below parameters.
And Again: I would be more than glad to contribute (back to the community) by assembling a wiki page about all this, once I figured out all the bits and pieces.

//// The situation ///

I am still trying to figure out how to install grub / grub2 on my intel imsm raid. And it's a striping RAID-0. Because the documentation out there is overall for RAID-1 (or 5) and the few bits I found for RAID-0, just do not work for me.

/// The questions ///

Maybe someone could shed some light on these open questions:

a) Do I need to install it to the Volumne (/dev/md126) or to the
Container (/dev/md127)? - My guess: The volume because it represents the RAID, hence should be what gets assembled for the int13 BIOS handler by the Option-ROM of the intel-sw-raid-controller.

b) What device do I need to put in the /grub/boot/device.map file as (hd0):
(/dev/md126, /dev/md127 or ...)?
My guess: The same like a).

c) Do you suggest grub1 (grub legacy) or grub2?

*** For grub2: The default way to install it: grub-install does not
(seem to) work as grub2 then tries to find the /boot device itself and grub built from bzr does not (yet) support mdadm arrays with metadata v1.2 that are partitioned :(
As I am quiet new to grub2 it would be very helpful to get an idea of
what of the (many) tools to use and if possible with what parameters.
Plus I wonder if with grub2 there will be problems because grub2 will
try to assemble the RAID itsself.

*** For grub: I did quiet some installations but all my trials did
always end in an Error 22 on "setup (hd0)" in the grub shell. <<< If grub would be your recommendation I can post the details!

Thanks a lot for any hint in advance!

K. Posern

P.S.: I would be still interested to hear how the two solutions that support an intel BIOS-supported (mdadm-imsm >=v3 and dmraid >=v1) raid compare to each other in terms of: speed (throughput), load/overhead (cpu/ram), stability, features.

There is a section in regards to external metadata
https://raid.wiki.kernel.org/index.php/RAID_setup#External_Metadata

But I guess I need to update with additional information on how to install to it.

Do you have Intel MSM Option ROM on your system? If so, then you create a RAID volume in the RAID optionROM and then when you do a install say for example Fedora 12, you specifiy you have "advanced storage device" (I think?) and you will see the RAID volume available for installation. I had issues with Fedora 13. There is a bug that causes crash when you attempt to install to an IMSM volume. And yes you install to the volume. The container can have 2 different RAID volumes.

Just FYI Intel is moving forward with MDRAID.

Hi Dave,

Thanks for your answer!

So it seems I might have found somebody to fight through this RAID/grub jungle? :)

/// about my hardware ///

What I have (and can offer as a test-platform): Sony VAIO notebook where CTRL+I during startup brings me to
Intel(R) Rapid Storage Technology - Option ROM - 9.5.0.1037
At the moment I have 1 RAID-0 on all 4 SATA drives.
I think this version (Sony limited?) only supports RAID-0 and maybe 1.

mdadm --examine on the container /dev/md127 gives me:

	Magic : Intel Raid ISM Cfg Sig
	Version : 1.2.01
	Faimily : ba0a95b0
	Generation : 0000025c

lspci shows me:
00:1f.2 RAID bus controller: Intel Corporation Mobile 82801 SATA RAID
Controller (rev. 05)

/// about Fedora (13) ////

My core goal was more to explore and document the MANUAL setup of grub/grub2 (for people compiling from source or using Gentoo).
Basically to document the knowledge needed to do-it-yourself in order to save people like me some precious time in collecting all the bits and pieces themselves. For the several distributions out there: I would have probably just linked to their own HowTos once we get information that it works (like I got this information for Ubuntu for example).

I personally use Gentoo --> no fancy GUI/install-scripts, but everything manual :)

As I said: Ubuntu has a device-mapper based binary package of grub2 that seems to install itsself in a way that allows to boot from an Intel OROM RAID (fakeraid). But that doesn't help me with Gentoo.

But I am really stuck alone, so I am willing to throw a Fedora on my machine if in exchange I would get the grub commands to make grub boot my intel RAID-0 :)

Nice :)

Do you happen to know if mdadm just integrated (parts of) the dmraid code? Because that's one of the many /rumors/ I picked up ;)
(and that I would like to put to an end ;)

Let me know.

Thanks!

Knuth

---

Well theoretically the OROM would present the RAID volume to the bootloader as a disk that it can read from. So if you mark your RAID volume for booting in BIOS, it should be able to pick up the bootloader from there right? Are you not able to install to /dev/md127 or whatever device node your RAID volume is on?

Also technically the imsm RAID is not fakeraid because it uses MDRAID instead of special driver to "fake" RAID in Linux. It's just Linux software RAID with OROM support.

I have seen some patches posted on this mailing list in regards to putting in DMRAID support so MDRAID can handle DMRAID arrays. I believe it's still work in progress. Neil can probably tell you more about that.

Hi Dave,

Thanks for your answer!

No. That is exactly the problem here. As mentioned in my original posting:

*** For grub2: The default way to install it: grub-install
/dev/md126 (= imsm volume) just returns 3 error messages.
Even though the bzr version of grub2 does support mdadm.imsm raids
with metada version >=1.2, it does /not/ support mdadm raids which are partitioned.
So I would need to put /dev/md126 into device.map (as the OROM of the intel fakeraid will assemble the raid-0 for grub to access it as a normal drive) and I need to install grub2 on the RAID-0 Volume /dev/md126! ... But how?
As I am quiet new to grub2 it would be very helpful to get an idea
of what of the (many) tools to use and if possible with what parameters.
Plus I wonder if with grub2 there will be problems because grub2
will try to assemble the RAID itsself.

*** For grub: I did quite some installations but all my trials
to install grub to /dev/md126 (= imsm Volume) did always end in an Error 22 on "setup (hd0)" in the grub shell. <<< If grub would be your recommendation I can post the details!

I am not sure about the /official/ definition of the term "fakeraid". I would have said that because you can create/delete the RAID within the OROM (so without any Operating System (e.g. Software-RAID Program involved) --> "Fakeraid".

Actually: Is there any hardware support within the:
00:1f.2 RAID bus controller: Intel Corporation Mobile 82801 SATA RAID
Controller (rev. 05)
Like an xor chip that is used to outsource some of the work under windows, but not under linux (because as you stated: as under linux there is no special driver, but linux uses mdadm to access to access the container and the volume)? If yes, are there any plans to develop a driver to outsource the xors to the RAID chip for Linux?

Knuth

I will have to create a setup and see how that works with grub. I guess the better question would also be how do distros like SUSE or Fedora/RHEL perform this as they use grub as well....

Yes. But you will need the Xeon class CPUs (C55xx and C35xx I think?) that has the IOAT DMA (CBDMA) engines to do this. If you have the IOAT DMA engines that has XOR caps, Linux can use the DMA engine to do XOR or PQ offloading. See drivers/dma/ioat/

So I created a RAID0 like your setup. I have md0 as the container and md127 as the RAID0 volume. Fedora 12 installed grub on /dev/md127. The grub version from F12 is 0.97. I was able to rerun grub-install on /dev/md127 successfully. My device map contains:
(hd0) /dev/md127

Do you have to use grub2?

Thanks again!

No.

Attached my results with grub-0.97 (-r10).
The error 22 I mentioned... maybe I am just not seeing it... like something /very/ obvious... ?

/dev/md/126 is my volume with the partitions, /dev/md127 is my container

I can give you ssh access to the machine, just send me your pub key.

XMPP chat: [email protected]

Knuth

/vola # grub-install /dev/md126 $1 #367 root@sysresccd pts/1

expr: non-integer argument

GNU GRUB version 0.97 (640K lower / 9216K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ]
grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
grub> setup --stage2=/boot/grub/stage2 --prefix=/grub (hd0,)
Checking if "/grub/stage1" exists... yes
Checking if "/grub/stage2" exists... yes
Checking if "/grub/e2fs_stage1_5" exists... yes
Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
Running "install --stage2=/boot/grub/stage2 /grub/stage1 (hd0,0) /grub/stage2 p /grub/menu.lst "... failed

Error 22: No such partition
grub> quit

/vola # grub --device-map=/dev/null $1 #367 root@sysresccd pts/1

GNU GRUB version 0.97 (640K lower / 9216K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename. ]

grub> device(hd0) /dev/md126

Error 27: Unrecognized command

grub> device (hd0) /dev/md126

grub> root (hd0,
Possible partitions are:

   Partition num: 0,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 2,  Filesystem type unknown, partition type 0x7
   Partition num: 4,  Filesystem type unknown, partition type 0x82
   Partition num: 5,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 6,  Filesystem type unknown, partition type 0x83
   Partition num: 7,  Filesystem type unknown, partition type 0x83
   Partition num: 8,  Filesystem type unknown, partition type 0x83
   Partition num: 9,  Filesystem type unknown, partition type 0x7

grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0x83

grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 18 sectors are embedded.
succeeded
Running "install /boot/grub/stage1 (hd0) (hd0)1+18 p (hd0,0)/boot/grub/stage2
/boot/grub/menu.lst"... failed

Error 22: No such partition

grub>