Skip to contentSkip to conversation

raid1 boot regression in 2.6.37 [bisected]

DISCUSSION Thomas Jarosch 1 participant 1 message Mar 25, 2011 MD RAID

lore
Share & tools
lore
#0 lore raw

Hello,

I've just updated from kernel 2.6.34.7 to kernel 2.6.37.5 and one HP Proliant DL320 G3 box with a raid1 software RAID stopped booting. (also two other non-HP boxes).

We run this script at boot time via dracut:
----------------------------------
#!/bin/sh
. /lib/dracut-lib.sh

info "Telling kernel to auto-detect RAID arrays" /sbin/initqueue --settled --name kerneldetectraid /sbin/mdadm --auto-detect
----------------------------------

With the "bad" commit in place, the kernel doesn't output any md message at all. I've bisected it down to this commit:

e804ac780e2f01cb3b914daca2fd4780d1743db1 is the first bad commit commit e804ac780e2f01cb3b914daca2fd4780d1743db1

Author: Tejun Heo <[email protected]>
Date:   Fri Oct 15 15:36:08 2010 +0200

md: fix and update workqueue usage

Workqueue usage in md has two problems.

* Flush can be used during or depended upon by memory reclaim, but md uses the system workqueue for flush_work which may lead to deadlock.

* md depends on flush_scheduled_work() to achieve exclusion against completion of removal of previous instances. flush_scheduled_work() may incur unexpected amount of delay and is scheduled to be removed.

This patch adds two workqueues to md - md_wq and md_misc_wq. The former is guaranteed to make forward progress under memory pressure and serves flush_work. The latter serves as the flush domain for other works.

:040000 040000 f6b6a34a71864263ed253866c5f8abe7f766ac6b
dc2eff4a91825142b7c88cf54751fc7acdf1a6d2 M drivers

I manually verified that the commit before it (57dab0bdf689d42972975ec646d862b0900a4bf3) works and the "bad" commit prevents the box from booting.

Some more info:

# mdadm --version
mdadm - v2.6.9 - 10th March 2009
# mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Wed May 27 17:52:40 2009
     Raid Level : raid1
     Array Size : 2562240 (2.44 GiB 2.62 GB)
  Used Dev Size : 2562240 (2.44 GiB 2.62 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

Update Time : Fri Mar 25 17:11:33 2011
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : 0ee8da2c:5803478b:e399b924:6520c535
Events : 0.160

Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1

Any idea what might go wrong? May be building a kernel with lock debugging on Monday might help. I think I also tried kernel 2.6.38 though I'll very on Monday, too.

Have a nice weekend,
Thomas

PS: Sorry Tejun for the HTML crap in my first mail.