lpfc driver loading is rebooting the box: 2.6.33 and above.

I moved forwarded and a server started rebooting while loading the kernel current kernels.

I isolated it to the lpfc driver.

Upsteam kernels tried so far:

3.6.34-rc1
3.4.33-git15

With these kernels I see the following when the driver loads:

"
[ 276.643991] Emulex LightPulse Fibre Channel SCSI driver 8.3.9

[ 276.643996] Copyright(c) 2004-2009 Emulex. All rights reserved.
[ 276.644058] lpfc 0000:1a:00.0: PCI INT A -> GSI 30 (level, low) -> IRQ 30
[ 276.644067] lpfc 0000:1a:00.0: setting latency timer to 64
[ 276.645003] scsi5 : on PCI bus 1a device 00 irq 30
[ 277.480150] lpfc 0000:1a:00.0: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 278.167137] lpfc 0000:1a:00.0: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 278.167171] alloc irq_desc for 72 on node -1
[ 278.167174] alloc kstat_irqs on node -1
[ 278.167181] alloc irq_2_iommu on node -1
[ 278.167195] lpfc 0000:1a:00.0: irq 72 for MSI/MSI-X
[ 278.857352] lpfc 0000:1a:00.0: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 279.547573] lpfc 0000:1a:00.0: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 280.237779] lpfc 0000:1a:00.0: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 280.927994] lpfc 0000:1a:00.0: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 280.927999] lpfc 0000:1a:00.0: 0:1477 Failed to set up hba
[ 281.028380] lpfc 0000:1a:00.0: PCI INT A disabled
[ 281.028411] alloc irq_desc for 37 on node -1
[ 281.028412] alloc kstat_irqs on node -1
[ 281.028416] alloc irq_2_iommu on node -1
[ 281.028422] lpfc 0000:1a:00.1: PCI INT B -> GSI 37 (level, low) -> IRQ 37
[ 281.028427] lpfc 0000:1a:00.1: setting latency timer to 64
[ 281.028939] scsi6 : on PCI bus 1a device 01 irq 37
[ 281.717949] lpfc 0000:1a:00.1: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 282.408169] lpfc 0000:1a:00.1: 0:0440 Adapter failed to init, READ_REV has missing revision information.
[ 282.408210] lpfc 0000:1a:00.1: irq 72 for MSI/MSI-X
[ 283.098375] lpfc 0000:1a:00.1: 0:0440 Adapter failed to inelm
 "

A little while (less then 30 seconds) after loading the driver (rmmod sometimes extends the life of the box). I get a hardware level pci fault and my box reboots.

My FW has been constant during this at 282a4 (current I believe) and lspci reports the card as

1a:00.0 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02) 1a:00.1 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02)

The system is a 2 socket Intel based server IBM x3550m2.

I forwarded ported some of the older versions of the driver 8.3.7 and 8.3.6 and that didn't not fix the issue, I am not convinced it is a direct driver change that is causing this. I also tried "lpfc_use_msi" and "lpfc_sli_mode" modules options without any affect on behavior.

Then I started testing whole kernels, .config file remained as close as possible. After some boots it seem 2.6.33-rc8 and before works. It is a relatively small window of change and I am still digging but I thought I would post and see if there were any ideas or suggestions out there.

Anyone seen anything like this?

Thanks,
Keith Mannthey
LTC FS-Dev

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Update on the issue. It gets weird.

2.6.33 stock worked ok.

A different .config file was working just fine on an identical setup so I started peeling away at .config changes.

I really have no idea about this option change and it will have to be punted to those more in the know.

the diff between a good and a bad kernel is....

@@ -130,7 +130,7 @@
CONFIG_RD_BZIP2=y
# CONFIG_RD_LZMA is not set
# CONFIG_RD_LZO is not set
-# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
+CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_EMBEDDED=y

This is some gcc optimization flag change. In the working config I have CONFIG_CC_OPTIMIZE_FOR_SIZE=y. If I don't optimize I get the badness.

gcc -v
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.3 --enable-linux-futex --without-system-libunwind --with-cpu=generic --build=x86_64-suse-linux
Thread model: posix
gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)

Thanks,
Keith Mannthey
LTC LTC-FS