scsi_wait_scan not working (2.6.30.5)

What could be the reason for scsi_wait_scan not waiting untill all disks are found?

I'm testing 2.6.30.5 on hardware with LSI Logic / Symbios Logic MegaRAID 530 SCSI 320-0X RAID controller and modprobe scsi_wait_scan finishes earlier than disks are found.

My initrd (romfs) does:
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mm.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko insmod /lib/modules/2.6.30.5-0.3/kernel/lib/crc-t10dif.ko
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/sd_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_wait_scan.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/exportfs/exportfs.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/xfs/xfs.ko
if [ "${ROOT##/dev/}" != "${ROOT}" ]; then
rootnr="$(busybox awk -v rootnode="${ROOT##/dev/}" '$4 == rootnode { print 256
* $1 + $2 }' /proc/partitions)"
if [ -n "$rootnr" ]; then
echo "$rootnr" > /proc/sys/kernel/real-root-dev
fi

Now if I add sleep few seconds or /bin/sh at the end of this initrd, then boot and then disks are detected properly and rootfs is mounted properly (after I exit from sh in case when /bin/sh is used).

The question remains - why scsi_wait_scan doesn't wait?

Quoted 26 lines · Arkadiusz Miskiewicz

What could be the reason for scsi_wait_scan not waiting untill all disks are found?

I'm testing 2.6.30.5 on hardware with LSI Logic / Symbios Logic MegaRAID 530 SCSI 320-0X RAID controller and modprobe scsi_wait_scan finishes earlier than disks are found.

My initrd (romfs) does:
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mm.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko insmod /lib/modules/2.6.30.5-0.3/kernel/lib/crc-t10dif.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/sd_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_wait_scan.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/exportfs/exportfs.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/xfs/xfs.ko
if [ "${ROOT##/dev/}" != "${ROOT}" ]; then
rootnr="$(busybox awk -v rootnode="${ROOT##/dev/}" '$4 == rootnode { print 256 * $1 + $2 }' /proc/partitions)"
if [ -n "$rootnr" ]; then
echo "$rootnr" > /proc/sys/kernel/real-root-dev
fi

Quoted 5 lines

Now if I add sleep few seconds or /bin/sh at the end of this initrd, then boot and then disks are detected properly and rootfs is mounted properly (after I exit from sh in case when /bin/sh is used).

The question remains - why scsi_wait_scan doesn't wait?

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver.

Hopefully when
all initrds are configured to wait until root appears, this problem will go away.

James

Quoted 37 lines · James Bottomley

Quoted 26 lines · Arkadiusz Miskiewicz

What could be the reason for scsi_wait_scan not waiting untill all disks are found?

I'm testing 2.6.30.5 on hardware with LSI Logic / Symbios Logic MegaRAID 530 SCSI 320-0X RAID controller and modprobe scsi_wait_scan finishes earlier than disks are found.

My initrd (romfs) does:
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mm.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko insmod /lib/modules/2.6.30.5-0.3/kernel/lib/crc-t10dif.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/sd_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_wait_scan.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/exportfs/exportfs.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/xfs/xfs.ko
if [ "${ROOT##/dev/}" != "${ROOT}" ]; then
rootnr="$(busybox awk -v rootnode="${ROOT##/dev/}" '$4 == rootnode { print 256 * $1 + $2 }' /proc/partitions)"
if [ -n "$rootnr" ]; then
echo "$rootnr" > /proc/sys/kernel/real-root-dev
fi

Quoted 5 lines

Now if I add sleep few seconds or /bin/sh at the end of this initrd, then boot and then disks are detected properly and rootfs is mounted properly (after I exit from sh in case when /bin/sh is used).

The question remains - why scsi_wait_scan doesn't wait?

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver.

Quoted 3 lines

Hopefully when
all initrds are configured to wait until root appears, this problem will go away.

Uh, ugly. I'll disable SCSI_SCAN_ASYNC here then.

Anyway what's the point of scsi_wait_scan module if initrd is still required to wait until root appears? (unless current behaviour is broken behaviour meant to be fixed?)

Quoted 40 lines · Arkadiusz Miskiewicz

Quoted 39 lines · James Bottomley

Quoted 27 lines · Arkadiusz Miskiewicz

What could be the reason for scsi_wait_scan not waiting untill all disks are found?

I'm testing 2.6.30.5 on hardware with LSI Logic / Symbios Logic MegaRAID 530 SCSI 320-0X RAID controller and modprobe scsi_wait_scan finishes earlier than disks are found.

My initrd (romfs) does:
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mm.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mbox. ko insmod /lib/modules/2.6.30.5-0.3/kernel/lib/crc-t10dif.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/sd_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_wait_scan.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/exportfs/exportfs.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/xfs/xfs.ko
if [ "${ROOT##/dev/}" != "${ROOT}" ]; then rootnr="$(busybox awk -v rootnode="${ROOT##/dev/}" '$4 == rootnode { print 256 * $1 + $2 }' /proc/partitions)"
if [ -n "$rootnr" ]; then
echo "$rootnr" > /proc/sys/kernel/real-root-dev
fi

Quoted 5 lines

Now if I add sleep few seconds or /bin/sh at the end of this initrd, then boot and then disks are detected properly and rootfs is mounted properly (after I exit from sh in case when /bin/sh is used).

The question remains - why scsi_wait_scan doesn't wait?

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver.

Quoted 5 lines

Quoted 3 lines

Hopefully when
all initrds are configured to wait until root appears, this problem will go away.

Uh, ugly. I'll disable SCSI_SCAN_ASYNC here then.

That won't actually help: the sd async scan is a separate mechanism not tied to that variable

Quoted 4 lines

Anyway what's the point of scsi_wait_scan module if initrd is still required to wait until root appears? (unless current behaviour is broken behaviour meant to be fixed?)

It was supposed to be an interim measure until the ditributions got their act together for initrd booting by waiting for the root disk to appear. It's just been "interim" for a bit longer than expected.

James

Quoted 46 lines · you

Quoted 45 lines · Arkadiusz Miskiewicz

Quoted 43 lines · James Bottomley

Quoted 27 lines · Arkadiusz Miskiewicz

What could be the reason for scsi_wait_scan not waiting untill all disks are found?

I'm testing 2.6.30.5 on hardware with LSI Logic / Symbios Logic MegaRAID 530 SCSI 320-0X RAID controller and modprobe scsi_wait_scan finishes earlier than disks are found.

My initrd (romfs) does:
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mm. ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mbo x. ko insmod /lib/modules/2.6.30.5-0.3/kernel/lib/crc-t10dif.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/sd_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_wait_scan.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/exportfs/exportfs.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/xfs/xfs.ko
if [ "${ROOT##/dev/}" != "${ROOT}" ]; then rootnr="$(busybox awk -v rootnode="${ROOT##/dev/}" '$4 == rootnode { print 256 * $1 + $2 }' /proc/partitions)"
if [ -n "$rootnr" ]; then
echo "$rootnr" > /proc/sys/kernel/real-root-dev
fi

Quoted 6 lines

Now if I add sleep few seconds or /bin/sh at the end of this initrd, then boot and then disks are detected properly and rootfs is mounted properly (after I exit from sh in case when /bin/sh is used).

The question remains - why scsi_wait_scan doesn't wait?

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver.

Hopefully when
all initrds are configured to wait until root appears, this problem will go away.

Quoted 4 lines

Quoted 1 line

Uh, ugly. I'll disable SCSI_SCAN_ASYNC here then.

That won't actually help: the sd async scan is a separate mechanism not tied to that variable

I assume there is no configurable way to disable this?

Quoted 52 lines · Arkadiusz Miskiewicz

Quoted 50 lines · you

Quoted 41 lines · Arkadiusz Miskiewicz

Quoted 40 lines · James Bottomley

Quoted 27 lines · Arkadiusz Miskiewicz

What could be the reason for scsi_wait_scan not waiting untill all disks are found?

I'm testing 2.6.30.5 on hardware with LSI Logic / Symbios Logic MegaRAID 530 SCSI 320-0X RAID controller and modprobe scsi_wait_scan finishes earlier than disks are found.

My initrd (romfs) does:
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mm. ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid_mbo x. ko insmod /lib/modules/2.6.30.5-0.3/kernel/lib/crc-t10dif.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/sd_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_wait_scan.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/exportfs/exportfs.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/xfs/xfs.ko
if [ "${ROOT##/dev/}" != "${ROOT}" ]; then rootnr="$(busybox awk -v rootnode="${ROOT##/dev/}" '$4 == rootnode { print 256 * $1 + $2 }' /proc/partitions)"
if [ -n "$rootnr" ]; then
echo "$rootnr" > /proc/sys/kernel/real-root-dev
fi

Quoted 6 lines

Now if I add sleep few seconds or /bin/sh at the end of this initrd, then boot and then disks are detected properly and rootfs is mounted properly (after I exit from sh in case when /bin/sh is used).

The question remains - why scsi_wait_scan doesn't wait?

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver.

Quoted 5 lines

Quoted 3 lines

Hopefully when
all initrds are configured to wait until root appears, this problem will go away.

Uh, ugly. I'll disable SCSI_SCAN_ASYNC here then.

That won't actually help: the sd async scan is a separate mechanism not tied to that variable

Quoted 1 line

I assume there is no configurable way to disable this?

I found this patch

http://marc.info/?l=linux-scsi&m=124388639906873&w=2

but it doesn't work on 30.5 (it's still not waiting untill everything is discovered)

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver. Hopefully when all initrds are configured to wait until root appears, this problem will go away.

Quoted 6 lines · James Bottomley

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver. Hopefully when all initrds are configured to wait until root appears, this problem will go away.

This change has been causing me some puzzlement working out the right way to do things in our infrastructure management code. Imagine I want a shell function which takes a host and target name and returns a device node name, ready to use. (In practice the login and logout functions would do reference counting to avoid multiple logins to the same target for different consumers, and set up device mapper targets and so on, but we can ignore that here.)

Previously, because iscsiadm --login does the equivalent of

echo - - - > /sys/class/scsi_host/hostX/scan

it was sufficient for me to do udevadm settle and then something like

  shopt -s nullglob
  local SESSION TARGET SESSION BLOCK KERNEL
  for SESSION in /sys/class/scsi_host/host*/device/session*; do
    for TARGET in $SESSION/iscsi_session/*/targetname; do
      [ "`< "$TARGET"`" = "$1" ] || continue
      for BLOCK in "$SESSION"/target*/*/block/*; do
        KERNEL=${BLOCK##*/}
        echo /dev/$KERNEL
        return $SUCCESS
      done
    done
  done
  return $EMISSING

to find my block device. Now I can't do that, and it's not obvious what the official way is to block until the outstanding async sd attachment has completed. Maybe I'm going about this completely the wrong way? Polling in a loop with a sleep isn't very nice, and clearly can't be how people are meant to do this, but everything else I've come up with is worse or more intrusive to the system.

The way it's supposed to work is that you listen for the uevent that signals the sd binding is complete, so you actually get notice asynchronously. The event carries device information with it. udev uses the node information to send VPD inquiries, which is how it does the by-id and by-uuid mappings.

Quoted 59 lines · Arkadiusz Miskiewicz

Quoted 49 lines · Arkadiusz Miskiewicz

Quoted 48 lines · you

Quoted 47 lines · Arkadiusz Miskiewicz

Quoted 45 lines · James Bottomley

Quoted 28 lines · Arkadiusz Miskiewicz

What could be the reason for scsi_wait_scan not waiting untill all disks are found?

I'm testing 2.6.30.5 on hardware with LSI Logic / Symbios Logic MegaRAID 530 SCSI 320-0X RAID controller and modprobe scsi_wait_scan finishes earlier than disks are found.

My initrd (romfs) does:
insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid _mm. ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/megaraid/megaraid _mbo x. ko insmod /lib/modules/2.6.30.5-0.3/kernel/lib/crc-t10dif.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/sd_mod.ko insmod /lib/modules/2.6.30.5-0.3/kernel/drivers/scsi/scsi_wait_scan.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/exportfs/exportfs.ko insmod /lib/modules/2.6.30.5-0.3/kernel/fs/xfs/xfs.ko if [ "${ROOT##/dev/}" != "${ROOT}" ]; then rootnr="$(busybox awk -v rootnode="${ROOT##/dev/}" '$4 == rootnode { print 256 * $1 + $2 }' /proc/partitions)" if [ -n "$rootnr" ]; then
echo "$rootnr" > /proc/sys/kernel/real-root-dev
fi

Quoted 6 lines

Now if I add sleep few seconds or /bin/sh at the end of this initrd, then boot and then disks are detected properly and rootfs is mounted properly (after I exit from sh in case when /bin/sh is used).

The question remains - why scsi_wait_scan doesn't wait?

It's caused by the sd async patches. What's happening is wait_scan is waiting until all the scans are complete, but now sd attachment may not be completed by the time that happens. So, although you have a scanned disk, you can't mount it without and attached sd driver.

Hopefully when
all initrds are configured to wait until root appears, this problem will go away.

Quoted 6 lines

Quoted 4 lines

Quoted 1 line

Uh, ugly. I'll disable SCSI_SCAN_ASYNC here then.

That won't actually help: the sd async scan is a separate mechanism not tied to that variable

I assume there is no configurable way to disable this?

I found this patch

Quoted 4 lines

http://marc.info/?l=linux-scsi&m=124388639906873&w=2

but it doesn't work on 30.5 (it's still not waiting untill everything is discovered)

Did you remember to rebuild your initrd after applying this patch and rebuilding the module?

Can you provide the dmesg log showing what happens during bootup with the patch applied?

Alan Stern

Quoted 2 lines

Can you provide the dmesg log showing what happens during bootup with the patch applied?

Hm, can console= point to netconsole somehow, so userspace messages could be seen via netconsole? Anyway from netconsole:

Aug 30 01:07:16 rhea   13.144139] netconsole: network logging started
Aug 30 01:07:16 rhea   13.173297] SCSI subsystem initialized
Aug 30 01:07:16 rhea   13.189630] megaraid cmm: 2.20.2.7 (Release Date: Sun Jul 16 00:01:03 EST 2006)
Aug 30 01:07:16 rhea   13.216688] megaraid: 2.20.5.1 (Release Date: Thu Nov 16 15:32:35 EST 2006)
Aug 30 01:07:16 rhea   13.238942] megaraid: probe new device 0x1000:0x0407:0x1000:0x0530:
Aug 30 01:07:16 rhea bus: 2:slot 3:func 0
Aug 30 01:07:16 rhea   13.271180] megaraid 0000:02:03.0: PCI INT A -> GSI 27 (level, low) -> IRQ 27
Aug 30 01:07:16 rhea   13.316707] megaraid: fw version:[414G] bios version:[A100]
Aug 30 01:07:16 rhea   13.334528] scsi0 : LSI Logic MegaRAID driver
Aug 30 01:07:16 rhea   13.352156] scsi[0]: scanning scsi channel 0 [Phy 0]
Aug 30 01:07:16 rhea for: non-raid devices
Aug 30 01:07:16 rhea   13.381065] Driver 'sd' needs updating - please use bus_type methods
Aug 30 01:07:16 rhea   13.420430] SGI XFS with ACLs, security attributes, large block/inode numbers, no debug enabled
Aug 30 01:07:16 rhea   13.453259] SGI XFS Quota Management subsystem
Aug 30 01:07:16 rhea   13.583779] VFS: Cannot open root device "802" or unknown-block(8,2)
Aug 30 01:07:16 rhea   13.602814] Please append a correct "root=" boot option; here are the available partitions:
Aug 30 01:07:16 rhea   13.627824] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,2)
Aug 30 01:07:16 rhea   13.652637] Pid: 1, comm: swapper Not tainted 2.6.30.5-vs2.3.0.36.14-pre7-0.3 #4
Aug 30 01:07:16 rhea   13.674797] Call Trace:
Aug 30 01:07:16 rhea   13.682138]  [<ffffffff8056e75d>] ? panic+0x7a/0x12e
Aug 30 01:07:16 rhea   13.697003]  [<ffffffff8056e851>] ? printk+0x40/0x47
Aug 30 01:07:16 rhea   13.711890]  [<ffffffff8072e090>] ? mount_block_root+0x1d6/0x271
Aug 30 01:07:16 rhea   13.729892]  [<ffffffff8072ecdb>] ? initrd_load+0x228/0x324
Aug 30 01:07:16 rhea   13.746585]  [<ffffffff8072e24f>] ? prepare_namespace+0xd5/0x18d
Aug 30 01:07:16 rhea   13.764578]  [<ffffffff8072d6cf>] ? kernel_init+0x178/0x188
Aug 30 01:07:16 rhea   13.781282]  [<ffffffff8020ceba>] ? child_rip+0xa/0x20
Aug 30 01:07:16 rhea   13.796686]  [<ffffffff8072d557>] ? kernel_init+0x0/0x188
Aug 30 01:07:16 rhea   13.812858]  [<ffffffff8020ceb0>] ? child_rip+0x0/0x20
Aug 30 01:07:16 rhea   13.828253] Rebooting in 60 seconds

...

and of course adding sleep 10 after scsi_wait_scan fixes boot

Quoted 45 lines · Arkadiusz Miskiewicz

Quoted 2 lines

Can you provide the dmesg log showing what happens during bootup with the patch applied?

Hm, can console= point to netconsole somehow, so userspace messages could be seen via netconsole? Anyway from netconsole:

Aug 30 01:07:16 rhea   13.144139] netconsole: network logging started
Aug 30 01:07:16 rhea   13.173297] SCSI subsystem initialized
Aug 30 01:07:16 rhea   13.189630] megaraid cmm: 2.20.2.7 (Release Date:
Sun Jul 16 00:01:03 EST 2006) Aug 30 01:07:16 rhea   13.216688] megaraid:
2.20.5.1 (Release Date: Thu Nov 16 15:32:35 EST 2006) Aug 30 01:07:16
rhea   13.238942] megaraid: probe new device 0x1000:0x0407:0x1000:0x0530:
Aug 30 01:07:16 rhea bus: 2:slot 3:func 0
Aug 30 01:07:16 rhea   13.271180] megaraid 0000:02:03.0: PCI INT A -> GSI

27 (level, low) -> IRQ 27 Aug 30 01:07:16 rhea 13.316707] megaraid: fw
version:[414G] bios version:[A100] Aug 30 01:07:16 rhea 13.334528]
scsi0 : LSI Logic MegaRAID driver Aug 30 01:07:16 rhea 13.352156]
scsi[0]: scanning scsi channel 0 [Phy 0] Aug 30 01:07:16 rhea for:
non-raid devices

Aug 30 01:07:16 rhea   13.381065] Driver 'sd' needs updating - please use

bus_type methods Aug 30 01:07:16 rhea 13.420430] SGI XFS with ACLs,
security attributes, large block/inode numbers, no debug enabled Aug 30
01:07:16 rhea 13.453259] SGI XFS Quota Management subsystem Aug 30
01:07:16 rhea 13.583779] VFS: Cannot open root device "802" or
unknown-block(8,2) Aug 30 01:07:16 rhea 13.602814] Please append a
correct "root=" boot option; here are the available partitions: Aug 30
01:07:16 rhea 13.627824] Kernel panic - not syncing: VFS: Unable to
mount root fs on unknown-block(8,2) Aug 30 01:07:16 rhea 13.652637]
Pid: 1, comm: swapper Not tainted 2.6.30.5-vs2.3.0.36.14-pre7-0.3 #4 Aug
30 01:07:16 rhea 13.674797] Call Trace:

Aug 30 01:07:16 rhea   13.682138]  [<ffffffff8056e75d>] ?
panic+0x7a/0x12e Aug 30 01:07:16 rhea   13.697003]  [<ffffffff8056e851>]
? printk+0x40/0x47 Aug 30 01:07:16 rhea   13.711890] 
[<ffffffff8072e090>] ? mount_block_root+0x1d6/0x271 Aug 30 01:07:16 rhea 
 13.729892]  [<ffffffff8072ecdb>] ? initrd_load+0x228/0x324 Aug 30
01:07:16 rhea   13.746585]  [<ffffffff8072e24f>] ?
prepare_namespace+0xd5/0x18d Aug 30 01:07:16 rhea   13.764578] 
[<ffffffff8072d6cf>] ? kernel_init+0x178/0x188 Aug 30 01:07:16 rhea  
13.781282]  [<ffffffff8020ceba>] ? child_rip+0xa/0x20 Aug 30 01:07:16
rhea   13.796686]  [<ffffffff8072d557>] ? kernel_init+0x0/0x188 Aug 30
01:07:16 rhea   13.812858]  [<ffffffff8020ceb0>] ? child_rip+0x0/0x20 Aug

30 01:07:16 rhea 13.828253] Rebooting in 60 seconds

...

Quoted 1 line

and of course adding sleep 10 after scsi_wait_scan fixes boot

Not especially helpful, I'm afraid. If you could add some extra printk statements, it would be better. For example, you might add some KERN_INFO messages to sd.c at the start of sd_probe(), just before the async_schedule() call near the end, and at the start of sd_probe_async(). In addition, some messages demarking the various steps of wait_scan_init() in scsi_wait_scan.c would be good.

Let's see what these messages produce both with and without the "sleep 10" in your initrd.

Alan Stern

with sleep

Aug 30 11:11:52 rhea kernel: [    6.950127] console [netcon0] enabled                                                                                        
Aug 30 11:11:52 rhea kernel: [   13.116253] netconsole: network logging started                                                                              
Aug 30 11:11:52 rhea kernel: [   13.145344] SCSI subsystem initialized                                                                                       
Aug 30 11:11:52 rhea kernel: [   13.160103] megaraid cmm: 2.20.2.7 (Release Date: Sun Jul 16 00:01:03 EST 2006)                                              
Aug 30 11:11:52 rhea kernel: [   13.187529] megaraid: 2.20.5.1 (Release Date: Thu Nov 16 15:32:35 EST 2006)                                                  
Aug 30 11:11:52 rhea kernel: [   13.209685] megaraid: probe new device 0x1000:0x0407:0x1000:0x0530: bus 2:slot 3:func 0                                      
Aug 30 11:11:52 rhea kernel: [   13.241872]   alloc irq_desc for 27 on cpu 0 node 0                                                                          
Aug 30 11:11:52 rhea kernel: [   13.241876]   alloc kstat_irqs on cpu 0 node 0                                                                               
Aug 30 11:11:52 rhea kernel: [   13.241885] megaraid 0000:02:03.0: PCI INT A -> GSI 27 (level, low) -> IRQ 27                                                
Aug 30 11:11:52 rhea kernel: [   13.286708] megaraid: fw version:[414G] bios version:[A100]                                                                  
Aug 30 11:11:52 rhea kernel: [   13.305284] scsi0 : LSI Logic MegaRAID driver                                                                                

Aug 30 11:11:52 rhea kernel: [   13.322942] scsi[0]: scanning scsi channel 0 [Phy 0] for non-raid devices                                                    
Aug 30 11:11:52 rhea kernel: [   13.355882] Driver 'sd' needs updating - please use bus_type methods                                                         
Aug 30 11:11:52 rhea kernel: [   13.379162] wait_scan_init: BEFORE wait_for_device_probe                                                                     
Aug 30 11:11:52 rhea kernel: [   13.397928] wait_scan_init: AFTER wait_for_device_probe                                                                      
Aug 30 11:11:52 rhea kernel: [   13.417739] wait_scan_init: BEFORE scsi_complete_async_scans                                                                 
Aug 30 11:11:52 rhea kernel: [   13.438822] wait_scan_init: AFTER scsi_complete_async_scans                                                                  
Aug 30 11:11:52 rhea kernel: [   13.459688] wait_scan_init: BEFORE async_synchronize_full                                                                    
Aug 30 11:11:52 rhea kernel: [   13.479998] wait_scan_init: AFTER wait_for_device_probe                                                                      

Aug 30 11:11:52 rhea kernel: [   13.902193] scsi 0:0:6:0: Processor         ESG-SHV  SCA HSBP M29     1.08 PQ: 0 ANSI: 2                                     
Aug 30 11:11:52 rhea kernel: [   15.930664] scsi[0]: scanning scsi channel 1 [Phy 1] for non-raid devices                                                    
Aug 30 11:11:52 rhea kernel: [   19.706530] scsi[0]: scanning scsi channel 2 [virtual] for logical drives                                                    
Aug 30 11:11:52 rhea kernel: [   19.727104] scsi 0:2:0:0: Direct-Access     MegaRAID LD 0 RAID5  280G 414G PQ: 0 ANSI: 2                                     
Aug 30 11:11:52 rhea kernel: [   19.757564] sd_probe: BEFORE async_schedule                                                                                  
Aug 30 11:11:52 rhea kernel: [   19.770091] sd_probe: AFTER async_schedule                                                                                   
Aug 30 11:11:52 rhea kernel: [   19.782507] sd_probe_async: ENTERED                                                                                          
Aug 30 11:11:52 rhea kernel: [   19.793223] sd 0:2:0:0: [sda] 574218240 512-byte hardware sectors: (293 GB/273 GiB)                                          
Aug 30 11:11:52 rhea kernel: [   19.816173] sd 0:2:0:0: [sda] Write Protect is off                                                                           
Aug 30 11:11:52 rhea kernel: [   19.830525] sd 0:2:0:0: [sda] Mode Sense: 00 00 00 00                                                                        
Aug 30 11:11:52 rhea kernel: [   19.830549] sd 0:2:0:0: [sda] Asking for cache data failed                                                                   
Aug 30 11:11:52 rhea kernel: [   19.846978] sd 0:2:0:0: [sda] Assuming drive cache: write through                                                            
Aug 30 11:11:52 rhea kernel: [   19.865496] sd 0:2:0:0: [sda] Asking for cache data failed                                                                   
Aug 30 11:11:52 rhea kernel: [   19.881920] sd 0:2:0:0: [sda] Assuming drive cache: write through                                                            

Aug 30 11:11:52 rhea kernel: [   19.900198]  sda: sda1 sda2 sda3                                                                                             
Aug 30 11:11:52 rhea kernel: [   19.924019] sd 0:2:0:0: [sda] Attached SCSI disk                                                                             
Aug 30 11:11:52 rhea kernel: [   23.113336] eth0: no IPv6 routers present                                                                                    
Aug 30 11:11:52 rhea kernel: [   23.754010] SGI XFS with ACLs, security attributes, large block/inode numbers, no debug enabled                              
Aug 30 11:11:52 rhea kernel: [   23.786463] SGI XFS Quota Management subsystem                                                                               
Aug 30 11:11:52 rhea kernel: [   23.915852] XFS mounting filesystem sda2                                                                                     
Aug 30 11:11:52 rhea kernel: [   24.050956] Ending clean XFS mount for filesystem: sda2                                                                      
Aug 30 11:11:52 rhea kernel: [   24.051050] VFS: Mounted root (xfs filesystem) readonly on device 8:2.