raid1 boot regression in 2.6.37 [bisected]
Hello,
I've just updated from kernel 2.6.34.7 to kernel 2.6.37.5 and one HP Proliant DL320 G3 box with a raid1 software RAID stopped booting. (also two other non-HP boxes).
We run this script at boot time via dracut:
----------------------------------
#!/bin/sh
. /lib/dracut-lib.sh
info "Telling kernel to auto-detect RAID arrays" /sbin/initqueue --settled --name kerneldetectraid /sbin/mdadm --auto-detect
----------------------------------
With the "bad" commit in place, the kernel doesn't output any md message at all. I've bisected it down to this commit:
e804ac780e2f01cb3b914daca2fd4780d1743db1 is the first bad commit commit e804ac780e2f01cb3b914daca2fd4780d1743db1
Author: Tejun Heo <[email protected]> Date: Fri Oct 15 15:36:08 2010 +0200
md: fix and update workqueue usage
Workqueue usage in md has two problems.
* Flush can be used during or depended upon by memory reclaim, but md uses the system workqueue for flush_work which may lead to deadlock.
* md depends on flush_scheduled_work() to achieve exclusion against completion of removal of previous instances. flush_scheduled_work() may incur unexpected amount of delay and is scheduled to be removed.
This patch adds two workqueues to md - md_wq and md_misc_wq. The former is guaranteed to make forward progress under memory pressure and serves flush_work. The latter serves as the flush domain for other works.
:040000 040000 f6b6a34a71864263ed253866c5f8abe7f766ac6bdc2eff4a91825142b7c88cf54751fc7acdf1a6d2 M drivers
I manually verified that the commit before it (57dab0bdf689d42972975ec646d862b0900a4bf3) works and the "bad" commit prevents the box from booting.
Some more info:
# mdadm --version mdadm - v2.6.9 - 10th March 2009
# mdadm --detail /dev/md0
/dev/md0:
Version : 0.90
Creation Time : Wed May 27 17:52:40 2009
Raid Level : raid1
Array Size : 2562240 (2.44 GiB 2.62 GB)
Used Dev Size : 2562240 (2.44 GiB 2.62 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent Update Time : Fri Mar 25 17:11:33 2011
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 0ee8da2c:5803478b:e399b924:6520c535
Events : 0.160
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
Any idea what might go wrong? May be building a kernel with lock debugging on Monday might help. I think I also tried kernel 2.6.38 though I'll very on Monday, too.
Have a nice weekend,
Thomas
PS: Sorry Tejun for the HTML crap in my first mail.