4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9
A message that is not in the archive (sent off-list or not yet received): [email protected]
Chad, Himanshu
Before I bisect or go chase changes, wanted to reach out because the driver seems to be the same version. Perhaps this is a PCIE change in the kernel for 4.10 affecting the load.
Its the same targetLIO server I have been using for a long time with 4.9
27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
With 4.9 I have no issues loading the driver for my targetLIO server. (DL380G8)
# modinfo qla2xxx | more filename: /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 94A8431A85BFF854B97B02D
[ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffadce989a1000. [ 10.455108] scsi host4: qla2xxx [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffadce989a9000. [ 10.662120] scsi host5: qla2xxx [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
Rebooting on the same server with 4.10 fails to load
Linux 4.10.0+
# modinfo qla2xxx | more filename: /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 939E0595E8A3C2E1BE94392
[ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b1000. [ 10.407865] scsi host5: qla2xxx [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b9000. [ 10.645932] scsi host5: qla2xxx [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error -22
Thanks
Laurence
----- Original Message -----
Quoted 71 lines
From: "Laurence Oberman" <[email protected]>
To: "Chad Dupuis" <[email protected]>, "Himanshu Madhani" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Sunday, March 12, 2017 7:39:23 AM
Subject: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Chad, Himanshu
Before I bisect or go chase changes, wanted to reach out because the driver seems to be the same version.
Perhaps this is a PCIE change in the kernel for 4.10 affecting the load. Its the same targetLIO server I have been using for a long time with 4.927:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
With 4.9 I have no issues loading the driver for my targetLIO server. (DL380G8)
# modinfo qla2xxx | more filename: /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 94A8431A85BFF854B97B02D[ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffadce989a1000. [ 10.455108] scsi host4: qla2xxx [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffadce989a9000. [ 10.662120] scsi host5: qla2xxx [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
Rebooting on the same server with 4.10 fails to load
Linux 4.10.0+
# modinfo qla2xxx | more filename: /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 939E0595E8A3C2E1BE94392[ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b1000. [ 10.407865] scsi host5: qla2xxx [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b9000. [ 10.645932] scsi host5: qla2xxx [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error -22
Quoted 3 lines
Thanks
Laurence
I started bisecting this, cannot believe others have not bumped into this on 4.10.
This is a generic QLE2562 and firmware is loaded by the driver so wondering why I am seeing this and other are not. There is nothing special with the PCIE bus on this DL380G8.
Anyway during the bisect I got to a point where in the 4.10 commits I still saw the "BAR 3" message but the probe worked.
[ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
see this above but probe did not fail
[ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 0xffffc02f98989000. [ 7.414738] scsi host4: qla2xxx
[ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 0xffffc02f989b1000. [ 7.625691] scsi host6: qla2xxx [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.1 hdma+ host#=6
I marked that still as bad and am continuing, I have 9 builds to go.
Thanks
Laurence
Hi Laurence,
Quoted 120 lines
-----Original Message-----
From: Laurence Oberman [mailto:[email protected]]
Sent: Sunday, March 12, 2017 11:31 AM
To: Dupuis, Chad <[email protected]>; Madhani, Himanshu <[email protected]>
Cc: Linux SCSI List <[email protected]>
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 74 lines
From: "Laurence Oberman" <[email protected]>
To: "Chad Dupuis" <[email protected]>, "Himanshu Madhani" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Sunday, March 12, 2017 7:39:23 AM
Subject: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Chad, Himanshu
Before I bisect or go chase changes, wanted to reach out because the driver seems to be the same version.
Perhaps this is a PCIE change in the kernel for 4.10 affecting the load. Its the same targetLIO server I have been using for a long time with 4.927:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
With 4.9 I have no issues loading the driver for my targetLIO server. (DL380G8)
# modinfo qla2xxx | more filename: /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 94A8431A85BFF854B97B02D[ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffadce989a1000. [ 10.455108] scsi host4: qla2xxx [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffadce989a9000. [ 10.662120] scsi host5: qla2xxx [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
Rebooting on the same server with 4.10 fails to load
Linux 4.10.0+
# modinfo qla2xxx | more filename: /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 939E0595E8A3C2E1BE94392[ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b1000. [ 10.407865] scsi host5: qla2xxx [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b9000. [ 10.645932] scsi host5: qla2xxx [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error -22Quoted 3 lines
Thanks
LaurenceI started bisecting this, cannot believe others have not bumped into this on 4.10.
This is a generic QLE2562 and firmware is loaded by the driver so wondering why I am seeing this and other are not.
There is nothing special with the PCIE bus on this DL380G8.Anyway during the bisect I got to a point where in the 4.10 commits I still saw the "BAR 3" message but the probe worked.
[ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.see this above but probe did not fail
[ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 0xffffc02f98989000. [ 7.414738] scsi host4: qla2xxx[ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 0xffffc02f989b1000. [ 7.625691] scsi host6: qla2xxx [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @
Quoted 6 lines
0000:27:00.1 hdma+ host#=6
I marked that still as bad and am continuing, I have 9 builds to go.
Thanks
Laurence
We had send patches to address issues discovered when the PCI Affinity were merged into 4.10 kernel.
Can you please check if your tree has following patches (Currently they are included in 4.10.2 stable)
Also can you apply following patch which addresses issue in PCI layer (This should be included in 4.10 stable branch soon )
https://www.spinics.net/lists/linux-pci/msg58742.html
Thanks,
Himanshu
----- Original Message -----
Quoted 163 lines
From: "Himanshu Madhani" <[email protected]>
To: "Laurence Oberman" <[email protected]>, "Chad Dupuis" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Monday, March 13, 2017 12:39:03 PM
Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Hi Laurence,
Quoted 136 lines
-----Original Message-----
From: Laurence Oberman [mailto:[email protected]]
Sent: Sunday, March 12, 2017 11:31 AM
To: Dupuis, Chad <[email protected]>; Madhani, Himanshu <[email protected]>
Cc: Linux SCSI List <[email protected]>
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 83 lines
From: "Laurence Oberman" <[email protected]>
To: "Chad Dupuis" <[email protected]>, "Himanshu Madhani" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Sunday, March 12, 2017 7:39:23 AM
Subject: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Chad, Himanshu
Before I bisect or go chase changes, wanted to reach out because the driver seems to be the same version.
Perhaps this is a PCIE change in the kernel for 4.10 affecting the load. Its the same targetLIO server I have been using for a long time with 4.927:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
With 4.9 I have no issues loading the driver for my targetLIO server. (DL380G8)
# modinfo qla2xxx | more filename: /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 94A8431A85BFF854B97B02D[ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffadce989a1000. [ 10.455108] scsi host4: qla2xxx [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffadce989a9000. [ 10.662120] scsi host5: qla2xxx [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8)@
0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).Rebooting on the same server with 4.10 fails to load
Linux 4.10.0+
# modinfo qla2xxx | more filename: /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 939E0595E8A3C2E1BE94392[ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b1000. [ 10.407865] scsi host5: qla2xxx [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b9000. [ 10.645932] scsi host5: qla2xxx [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error -22Quoted 3 lines
Thanks
LaurenceI started bisecting this, cannot believe others have not bumped into this on
4.10.
This is a generic QLE2562 and firmware is loaded by the driver so wondering why I am seeing this and other are not.
There is nothing special with the PCIE bus on this DL380G8.Anyway during the bisect I got to a point where in the 4.10 commits I still saw
the "BAR 3" message but the probe worked.[ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.see this above but probe did not fail
[ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 0xffffc02f98989000. [ 7.414738] scsi host4: qla2xxx[ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 0xffffc02f989b1000. [ 7.625691] scsi host6: qla2xxx [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @Quoted 6 lines
0000:27:00.1 hdma+ host#=6
I marked that still as bad and am continuing, I have 9 builds to go.
Thanks
LaurenceWe had send patches to address issues discovered when the PCI Affinity were merged into 4.10 kernel.
Can you please check if your tree has following patches (Currently they are included in 4.10.2 stable)
Quoted 8 lines
Also can you apply following patch which addresses issue in PCI layer (This should be included in 4.10 stable branch soon )
https://www.spinics.net/lists/linux-pci/msg58742.html
Thanks,
Himanshu
Hello Himanshu
Many thanks for coming back to me.
OK, I will test all of this and let you know.
I dont think I have these and this would explain all this.
Regards
Laurence
----- Original Message -----
Quoted 194 lines
From: "Laurence Oberman" <[email protected]>
To: "Himanshu Madhani" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Monday, March 13, 2017 12:54:12 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 176 lines
From: "Himanshu Madhani" <[email protected]>
To: "Laurence Oberman" <[email protected]>, "Chad Dupuis" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Monday, March 13, 2017 12:39:03 PM
Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Hi Laurence,
Quoted 145 lines
-----Original Message-----
From: Laurence Oberman [mailto:[email protected]]
Sent: Sunday, March 12, 2017 11:31 AM
To: Dupuis, Chad <[email protected]>; Madhani, Himanshu <[email protected]>
Cc: Linux SCSI List <[email protected]>
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb)
with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 86 lines
From: "Laurence Oberman" <[email protected]>
To: "Chad Dupuis" <[email protected]>, "Himanshu Madhani" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Sunday, March 12, 2017 7:39:23 AM
Subject: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Chad, Himanshu
Before I bisect or go chase changes, wanted to reach out because the driver seems to be the same version.
Perhaps this is a PCIE change in the kernel for 4.10 affecting the load.
Its the same targetLIO server I have been using for a long time with 4.927:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
With 4.9 I have no issues loading the driver for my targetLIO server. (DL380G8)
# modinfo qla2xxx | more filename: /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 94A8431A85BFF854B97B02D[ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffadce989a1000. [ 10.455108] scsi host4: qla2xxx [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffadce989a9000. [ 10.662120] scsi host5: qla2xxx [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/sx8)
@
0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).Rebooting on the same server with 4.10 fails to load
Linux 4.10.0+
# modinfo qla2xxx | more filename: /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 939E0595E8A3C2E1BE94392[ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b1000. [ 10.407865] scsi host5: qla2xxx [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b9000. [ 10.645932] scsi host5: qla2xxx [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error -22Quoted 3 lines
Thanks
LaurenceI started bisecting this, cannot believe others have not bumped into this on
4.10.
This is a generic QLE2562 and firmware is loaded by the driver so wondering
why I am seeing this and other are not.
There is nothing special with the PCIE bus on this DL380G8.Anyway during the bisect I got to a point where in the 4.10 commits I still
saw
the "BAR 3" message but the probe worked.[ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.see this above but probe did not fail
[ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 0xffffc02f98989000. [ 7.414738] scsi host4: qla2xxx[ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 0xffffc02f989b1000. [ 7.625691] scsi host6: qla2xxx [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s x8)Quoted 7 lines
@
0000:27:00.1 hdma+ host#=6I marked that still as bad and am continuing, I have 9 builds to go.
Thanks
LaurenceWe had send patches to address issues discovered when the PCI Affinity were
merged into 4.10 kernel.Can you please check if your tree has following patches (Currently they are included in 4.10.2 stable)
Quoted 8 lines
Also can you apply following patch which addresses issue in PCI layer (This should be included in 4.10 stable branch soon )
https://www.spinics.net/lists/linux-pci/msg58742.html
Thanks,
Himanshu
Quoted 9 lines
Hello Himanshu
Many thanks for coming back to me.
OK, I will test all of this and let you know.
I dont think I have these and this would explain all this.Regards
Laurence
Hello Himanshu
This is still failing for me on the HP DL380G8
My Stable tree
root@dhcp40-131 ~]# uname -a
Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+
$ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c Author: Michael Hernandez <[email protected]> Date: Wed Feb 15 15:37:21 2017 -0800
scsi: qla2xxx: Fix Regression introduced by pci_alloc_irq_vectors_affinity call.
commit 67f2db8792f96d8f7521461635d25f9c80245d80 upstream.
For target mode, we need to increase minimum vectors value by one to account for ATIO queue.
Following stack trace will be seen
$ git show 699abe249b0e2ca590284424d95490e92ccd6aeb commit 699abe249b0e2ca590284424d95490e92ccd6aeb Author: Michael Hernandez <[email protected]> Date: Wed Feb 15 15:37:20 2017 -0800
scsi: qla2xxx: Fix response queue count for Target mode.
commit d0d2c68b759bbf678e078fd0c71b5fde65a9392c upstream.
Target mode initialization was not calculating response queue values correctly resulting into one less MSI-X vector.
$ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5 commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5 Author: Michael Hernandez <[email protected]> Date: Wed Feb 15 15:37:19 2017 -0800
scsi: qla2xxx: Cleaned up queue configuration code.
commit f54f2cb540b53d55a81d620e816810d59be5cb1b upstream.
This patch cleaned up queue configuration code, such that once initialized, we should not touch msix_count value. This will prevent incorrect numbers of MSI-X vectors requested while performing target mode configuration.
Tested, and fails
root@dhcp40-131 ~]# dmesg | grep qla
[ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffb3e4589a1000. [ 9.788259] scsi host4: qla2xxx [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 106 iobase 0xffffb3e4589a9000. [ 10.030273] scsi host4: qla2xxx [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with error -22
Now
Patched
https://www.spinics.net/lists/linux-pci/msg58742.html
[loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci patching file drivers/pci/msi.c
Hunk #1 succeeded at 1072 (offset -19 lines).
Hunk #2 succeeded at 1120 (offset -27 lines).
Hunk #3 succeeded at 1200 (offset -27 lines).
Rebuild kernel
Reboot and test
Still fails
[ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 iobase 0xffffa3ff58991000. [ 8.670118] scsi host4: qla2xxx [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 iobase 0xffffa3ff589a1000. [ 8.912089] scsi host4: qla2xxx [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with error -22
Went to test on another system , HP Microserver G8
Same HBA
Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
Works here but complains about the vector count support
[ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector count: 32. [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an ISP2532 irq 16 iobase 0xffffbd9100c99000. [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed to enable support with 32 vectors, using 4 vectors. [ 8.759660] scsi host6: qla2xxx [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A - HP 8Gb Dual Channel PCI-e 2.0 FC HBA. [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @ 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5). [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector count: 32. [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an ISP2532 irq 17 iobase 0xffffbd9100cb9000. [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed to enable support with 32 vectors, using 4 vectors. [ 8.999658] scsi host7: qla2xxx [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A - HP 8Gb Dual Channel PCI-e 2.0 FC HBA. [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe (5.0GT/s x8) @ 0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5).
So something about the DL380 here needs to be figured out as well as the MSI-X messages above.
Earlier kernels up to 4.9.13 are all fine on both servers
Thanks
Laurence
----- Original Message -----
Quoted 352 lines
From: "Laurence Oberman" <[email protected]>
To: "Himanshu Madhani" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Monday, March 13, 2017 9:06:38 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 208 lines
From: "Laurence Oberman" <[email protected]>
To: "Himanshu Madhani" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Monday, March 13, 2017 12:54:12 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 187 lines
From: "Himanshu Madhani" <[email protected]>
To: "Laurence Oberman" <[email protected]>, "Chad Dupuis" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Monday, March 13, 2017 12:39:03 PM
Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Hi Laurence,
Quoted 154 lines
-----Original Message-----
From: Laurence Oberman [mailto:[email protected]]
Sent: Sunday, March 12, 2017 11:31 AM
To: Dupuis, Chad <[email protected]>; Madhani, Himanshu <[email protected]>
Cc: Linux SCSI List <[email protected]>
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb)
with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 93 lines
From: "Laurence Oberman" <[email protected]>
To: "Chad Dupuis" <[email protected]>, "Himanshu Madhani" <[email protected]>
Cc: "Linux SCSI List" <[email protected]>
Sent: Sunday, March 12, 2017 7:39:23 AM
Subject: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9Chad, Himanshu
Before I bisect or go chase changes, wanted to reach out because the driver seems to be the same version.
Perhaps this is a PCIE change in the kernel for 4.10 affecting the load.
Its the same targetLIO server I have been using for a long time with 4.927:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to
PCI Express HBA (rev 02)With 4.9 I have no issues loading the driver for my targetLIO server. (DL380G8)
# modinfo qla2xxx | more filename: /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 94A8431A85BFF854B97B02D[ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffadce989a1000. [ 10.455108] scsi host4: qla2xxx [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffadce989a9000. [ 10.662120] scsi host5: qla2xxx [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/sx8)
@
0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).Rebooting on the same server with 4.10 fails to load
Linux 4.10.0+
# modinfo qla2xxx | more filename: /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko firmware: ql2500_fw.bin version: 8.07.00.38-k license: GPL description: QLogic Fibre Channel HBA Driver author: QLogic Corporation srcversion: 939E0595E8A3C2E1BE94392[ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b1000. [ 10.407865] scsi host5: qla2xxx [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 0xffffacbf189b9000. [ 10.645932] scsi host5: qla2xxx [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error -22Quoted 3 lines
Thanks
LaurenceI started bisecting this, cannot believe others have not bumped into this
on
4.10.
This is a generic QLE2562 and firmware is loaded by the driver so wondering
why I am seeing this and other are not.
There is nothing special with the PCIE bus on this DL380G8.Anyway during the bisect I got to a point where in the 4.10 commits I still
saw
the "BAR 3" message but the probe worked.[ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.see this above but probe did not fail
[ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 0xffffc02f98989000. [ 7.414738] scsi host4: qla2xxx[ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 0xffffc02f989b1000. [ 7.625691] scsi host6: qla2xxx [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/sQuoted 8 lines
x8)
@
0000:27:00.1 hdma+ host#=6I marked that still as bad and am continuing, I have 9 builds to go.
Thanks
LaurenceWe had send patches to address issues discovered when the PCI Affinity were
merged into 4.10 kernel.Can you please check if your tree has following patches (Currently they are
included in 4.10.2 stable)Quoted 9 lines
Also can you apply following patch which addresses issue in PCI layer (This
should be included in 4.10 stable branch soon )https://www.spinics.net/lists/linux-pci/msg58742.html
Thanks,
HimanshuQuoted 9 lines
Hello Himanshu
Many thanks for coming back to me.
OK, I will test all of this and let you know.
I dont think I have these and this would explain all this.Regards
LaurenceHello Himanshu
This is still failing for me on the HP DL380G8My Stable tree
root@dhcp40-131 ~]# uname -a
Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+$ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c Author: Michael Hernandez <[email protected]> Date: Wed Feb 15 15:37:21 2017 -0800scsi: qla2xxx: Fix Regression introduced by
pci_alloc_irq_vectors_affinitycall.commit
67f2db8792f96d8f7521461635d25f9c80245d80upstream.For target mode, we need to increase minimum vectors value by one to account for ATIO queue.
Following stack trace will be seen
$ git show 699abe249b0e2ca590284424d95490e92ccd6aeb commit 699abe249b0e2ca590284424d95490e92ccd6aeb Author: Michael Hernandez <[email protected]> Date: Wed Feb 15 15:37:20 2017 -0800scsi: qla2xxx: Fix response queue count for Target mode.
commit
d0d2c68b759bbf678e078fd0c71b5fde65a9392cupstream.Target mode initialization was not calculating response queue values correctly resulting into one less MSI-X vector.
$ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5 commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5 Author: Michael Hernandez <[email protected]> Date: Wed Feb 15 15:37:19 2017 -0800scsi: qla2xxx: Cleaned up queue configuration code.
commit
f54f2cb540b53d55a81d620e816810d59be5cb1bupstream.This patch cleaned up queue configuration code, such that once initialized, we should not touch
msix_countvalue. This will prevent incorrect numbers of MSI-X vectors requested while performing target mode configuration.Tested, and fails
root@dhcp40-131 ~]# dmesg | grep qla
[ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffb3e4589a1000. [ 9.788259] scsi host4: qla2xxx [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 106 iobase 0xffffb3e4589a9000. [ 10.030273] scsi host4: qla2xxx [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with error -22Now
Patched
https://www.spinics.net/lists/linux-pci/msg58742.html[loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci patching file
drivers/pci/msi.c
Hunk #1 succeeded at 1072 (offset -19 lines).
Hunk #2 succeeded at 1120 (offset -27 lines).
Hunk #3 succeeded at 1200 (offset -27 lines).Rebuild kernel
Reboot and testStill fails
[ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 iobase 0xffffa3ff58991000. [ 8.670118] scsi host4: qla2xxx [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 iobase 0xffffa3ff589a1000. [ 8.912089] scsi host4: qla2xxx [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with error -22Went to test on another system , HP Microserver G8
Same HBA
Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)Works here but complains about the vector count support
[ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector count: 32. [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an ISP2532 irq 16 iobase 0xffffbd9100c99000. [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed to enable support with 32 vectors, using 4 vectors. [ 8.759660] scsi host6: qla2xxx [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A - HP 8Gb Dual Channel PCI-e 2.0 FC HBA. [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @ 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5). [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector count: 32. [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an ISP2532 irq 17 iobase 0xffffbd9100cb9000. [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed to enable support with 32 vectors, using 4 vectors. [ 8.999658] scsi host7: qla2xxx [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A - HP 8Gb Dual Channel PCI-e 2.0 FC HBA. [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe (5.0GT/s x8) @0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5).
Quoted 8 lines
So something about the DL380 here needs to be figured out as well as the MSI-X messages above.
Earlier kernels up to 4.9.13 are all fine on both servers
Thanks
Laurence
I meant to add
With the G8 Microserver links seem to come up with only the 4 MSI-X vectors but I get this so it not fully healthy
[ 373.823667] Rounding down aligned max_sectors from 65535 to 65528 [ 399.994495] qla2xxx [0000:07:00.0]-00af:6: Performing ISP error recovery - ha=ffffa0e934210000. [ 400.728968] qla2xxx [0000:07:00.0]-500a:6: LOOP UP detected (8 Gbps). [ 401.473405] qla2xxx [0000:07:00.0]-5030:6: Error entry - invalid handle/queue (1c01). ***** Note [ 426.966519] qla2xxx [0000:07:00.1]-00af:7: Performing ISP error recovery - ha=ffffa0e934e30000. [ 427.697601] qla2xxx [0000:07:00.1]-500a:7: LOOP UP detected (8 Gbps). [ 428.473948] qla2xxx [0000:07:00.1]-5030:7: Error entry - invalid handle/queue (1c01). ***** Note
----- Original Message -----
> From: "Laurence Oberman" <[email protected]>
> To: "Himanshu Madhani" <[email protected]>
> Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
> Sent: Monday, March 13, 2017 9:06:38 PM
> Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9
>
>
>
> ----- Original Message -----
> > From: "Laurence Oberman" <[email protected]>
> > To: "Himanshu Madhani" <[email protected]>
> > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List"
> > <[email protected]>
> > Sent: Monday, March 13, 2017 12:54:12 PM
> > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based
> > 8Gb) with BAR 3 error, work fine on 4.9
> >
> >
> >
> > ----- Original Message -----
> > > From: "Himanshu Madhani" <[email protected]>
> > > To: "Laurence Oberman" <[email protected]>, "Chad Dupuis"
> > > <[email protected]>
> > > Cc: "Linux SCSI List" <[email protected]>
> > > Sent: Monday, March 13, 2017 12:39:03 PM
> > > Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > >
> > > Hi Laurence,
> > >
> > > > -----Original Message-----
> > > > From: Laurence Oberman [mailto:[email protected]]
> > > > Sent: Sunday, March 12, 2017 11:31 AM
> > > > To: Dupuis, Chad <[email protected]>; Madhani, Himanshu
> > > > <[email protected]>
> > > > Cc: Linux SCSI List <[email protected]>
> > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based
> > > > 8Gb)
> > > > with BAR 3 error, work fine on 4.9
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Laurence Oberman" <[email protected]>
> > > > > To: "Chad Dupuis" <[email protected]>, "Himanshu Madhani"
> > > > > <[email protected]>
> > > > > Cc: "Linux SCSI List" <[email protected]>
> > > > > Sent: Sunday, March 12, 2017 7:39:23 AM
> > > > > Subject: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based
> > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > >
> > > > > Chad, Himanshu
> > > > >
> > > > > Before I bisect or go chase changes, wanted to reach out because the
> > > > > driver seems to be the same version.
> > > > > Perhaps this is a PCIE change in the kernel for 4.10 affecting the
> > > > > load.
> > > > > Its the same targetLIO server I have been using for a long time with
> > > > > 4.9
> > > > >
> > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel
> > > > > to
> > > > > PCI Express HBA (rev 02)
> > > > >
> > > > > With 4.9 I have no issues loading the driver for my targetLIO server.
> > > > > (DL380G8)
> > > > >
> > > > > # modinfo qla2xxx | more
> > > > > filename:
> > > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > > > firmware: ql2500_fw.bin
> > > > > version: 8.07.00.38-k
> > > > > license: GPL
> > > > > description: QLogic Fibre Channel HBA Driver
> > > > > author: QLogic Corporation
> > > > > srcversion: 94A8431A85BFF854B97B02D
> > > > >
> > > > > [ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel
> > > > > HBA
> > > > > Driver: 8.07.00.38-k.
> > > > > [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq
> > > > > 106
> > > > > iobase
> > > > > 0xffffadce989a1000.
> > > > > [ 10.455108] scsi host4: qla2xxx
> > > > > [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 -
> > > > > PCI-Express
> > > > > Dual Channel 8Gb Fibre Channel HBA.
> > > > > [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s
> > > > > x8)
> > > > > @
> > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
> > > > > [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq
> > > > > 110
> > > > > iobase
> > > > > 0xffffadce989a9000.
> > > > > [ 10.662120] scsi host5: qla2xxx
> > > > > [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 -
> > > > > PCI-Express
> > > > > Dual Channel 8Gb Fibre Channel HBA.
> > > > > [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s
> > > > > x8)
> > > > > @
> > > > > 0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
> > > > >
> > > > > Rebooting on the same server with 4.10 fails to load
> > > > >
> > > > > Linux 4.10.0+
> > > > > # modinfo qla2xxx | more
> > > > > filename:
> > > > > /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > > > firmware: ql2500_fw.bin
> > > > > version: 8.07.00.38-k
> > > > > license: GPL
> > > > > description: QLogic Fibre Channel HBA Driver
> > > > > author: QLogic Corporation
> > > > > srcversion: 939E0595E8A3C2E1BE94392
> > > > >
> > > > > [ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel
> > > > > HBA
> > > > > Driver: 8.07.00.38-k.
> > > > > [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
> > > > > [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq
> > > > > 110
> > > > > iobase
> > > > > 0xffffacbf189b1000.
> > > > > [ 10.407865] scsi host5: qla2xxx
> > > > > [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error -22
> > > > > [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled.
> > > > > [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq
> > > > > 110
> > > > > iobase
> > > > > 0xffffacbf189b9000.
> > > > > [ 10.645932] scsi host5: qla2xxx
> > > > > [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error -22
> > > > >
> > > > > Thanks
> > > > > Laurence
> > > > >
> > > >
> > > > I started bisecting this, cannot believe others have not bumped into
> > > > this
> > > > on
> > > > 4.10.
> > > > This is a generic QLE2562 and firmware is loaded by the driver so
> > > > wondering
> > > > why I am seeing this and other are not.
> > > > There is nothing special with the PCIE bus on this DL380G8.
> > > >
> > > > Anyway during the bisect I got to a point where in the 4.10 commits I
> > > > still
> > > > saw
> > > > the "BAR 3" message but the probe worked.
> > > >
> > > > [ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA
> > > > Driver:
> > > > 8.07.00.38-k.
> > > > [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
> > > >
> > > > see this above but probe did not fail
> > > >
> > > > [ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97
> > > > iobase
> > > > 0xffffc02f98989000.
> > > > [ 7.414738] scsi host4: qla2xxx
> > > >
> > > > [ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 -
> > > > PCI-Express
> > > > Dual
> > > > Channel 8Gb Fibre Channel HBA.
> > > > [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s
> > > > x8)
> > > > @
> > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
> > > > [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled.
> > > > [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100
> > > > iobase
> > > > 0xffffc02f989b1000.
> > > > [ 7.625691] scsi host6: qla2xxx
> > > > [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 -
> > > > PCI-Express
> > > > Dual
> > > > Channel 8Gb Fibre Channel HBA.
> > > > [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s
> > > > x8)
> > > > @
> > > > 0000:27:00.1 hdma+ host#=6
> > > >
> > > > I marked that still as bad and am continuing, I have 9 builds to go.
> > > >
> > > > Thanks
> > > > Laurence
> > >
> > > We had send patches to address issues discovered when the PCI Affinity
> > > were
> > > merged into 4.10 kernel.
> > >
> > > Can you please check if your tree has following patches (Currently they
> > > are
> > > included in 4.10.2 stable)
> > >
> > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c
> > >
> > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=699abe249b0e2ca590284424d95490e92ccd6aeb
> > >
> > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=cf9d9110806493e2ea2db7d00a1a991401dfb9d5
> > >
> > > Also can you apply following patch which addresses issue in PCI layer
> > > (This
> > > should be included in 4.10 stable branch soon )
> > >
> > > https://www.spinics.net/lists/linux-pci/msg58742.html
> > >
> > > Thanks,
> > > Himanshu
> > >
> >
> > Hello Himanshu
> >
> > Many thanks for coming back to me.
> > OK, I will test all of this and let you know.
> > I dont think I have these and this would explain all this.
> >
> > Regards
> > Laurence
> >
>
> Hello Himanshu
> This is still failing for me on the HP DL380G8
>
> My Stable tree
>
> root@dhcp40-131 ~]# uname -a
> Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+
>
> $ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c
> commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c
> Author: Michael Hernandez <[email protected]>
> Date: Wed Feb 15 15:37:21 2017 -0800
>
> scsi: qla2xxx: Fix Regression introduced by
> pci_alloc_irq_vectors_affinity call.
>
> commit 67f2db8792f96d8f7521461635d25f9c80245d80 upstream.
>
> For target mode, we need to increase minimum vectors value by one to
> account for ATIO queue.
>
> Following stack trace will be seen
>
> $ git show 699abe249b0e2ca590284424d95490e92ccd6aeb
> commit 699abe249b0e2ca590284424d95490e92ccd6aeb
> Author: Michael Hernandez <[email protected]>
> Date: Wed Feb 15 15:37:20 2017 -0800
>
> scsi: qla2xxx: Fix response queue count for Target mode.
>
> commit d0d2c68b759bbf678e078fd0c71b5fde65a9392c upstream.
>
> Target mode initialization was not calculating response queue values
> correctly resulting into one less MSI-X vector.
>
> $ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5
> commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5
> Author: Michael Hernandez <[email protected]>
> Date: Wed Feb 15 15:37:19 2017 -0800
>
> scsi: qla2xxx: Cleaned up queue configuration code.
>
> commit f54f2cb540b53d55a81d620e816810d59be5cb1b upstream.
>
> This patch cleaned up queue configuration code, such that once
> initialized, we should not touch msix_count value. This will prevent
> incorrect numbers of MSI-X vectors requested while performing target
> mode configuration.
>
>
> Tested, and fails
>
> root@dhcp40-131 ~]# dmesg | grep qla
> [ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA
> Driver: 8.07.00.38-k.
> [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
> [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase
> 0xffffb3e4589a1000.
> [ 9.788259] scsi host4: qla2xxx
> [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with error -22
> [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled.
> [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 106 iobase
> 0xffffb3e4589a9000.
> [ 10.030273] scsi host4: qla2xxx
> [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with error -22
>
>
> Now
> Patched
> https://www.spinics.net/lists/linux-pci/msg58742.html
>
> [loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci
> patching file drivers/pci/msi.c
> Hunk #1 succeeded at 1072 (offset -19 lines).
> Hunk #2 succeeded at 1120 (offset -27 lines).
> Hunk #3 succeeded at 1200 (offset -27 lines).
>
> Rebuild kernel
> Reboot and test
>
> Still fails
>
> [ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA
> Driver: 8.07.00.38-k.
> [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
> [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 iobase
> 0xffffa3ff58991000.
> [ 8.670118] scsi host4: qla2xxx
> [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with error -22
> [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled.
> [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 iobase
> 0xffffa3ff589a1000.
> [ 8.912089] scsi host4: qla2xxx
> [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with error -22
>
> Went to test on another system , HP Microserver G8
> Same HBA
> Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express
> HBA (rev 02)
>
> Works here but complains about the vector count support
>
> [ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA
> Driver: 8.07.00.38-k.
> [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector count: 32.
> [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an ISP2532 irq 16 iobase
> 0xffffbd9100c99000.
> [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed to enable support
> with 32 vectors, using 4 vectors.
> [ 8.759660] scsi host6: qla2xxx
> [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A - HP 8Gb Dual
> Channel PCI-e 2.0 FC HBA.
> [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @
> 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5).
> [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector count: 32.
> [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an ISP2532 irq 17 iobase
> 0xffffbd9100cb9000.
> [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed to enable support
> with 32 vectors, using 4 vectors.
> [ 8.999658] scsi host7: qla2xxx
> [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A - HP 8Gb Dual
> Channel PCI-e 2.0 FC HBA.
> [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe (5.0GT/s x8) @
> 0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5).
>
> So something about the DL380 here needs to be figured out as well as the
> MSI-X messages above.
>
> Earlier kernels up to 4.9.13 are all fine on both servers
>
> Thanks
> Laurence
>
I meant to add
With the G8 Microserver links seem to come up with only the 4 MSI-X vectors but I get this
so it not fully healthy
[ 373.823667] Rounding down aligned max_sectors from 65535 to 65528
[ 399.994495] qla2xxx [0000:07:00.0]-00af:6: Performing ISP error recovery - ha=ffffa0e934210000.
[ 400.728968] qla2xxx [0000:07:00.0]-500a:6: LOOP UP detected (8 Gbps).
[ 401.473405] qla2xxx [0000:07:00.0]-5030:6: Error entry - invalid handle/queue (1c01). ***** Note
[ 426.966519] qla2xxx [0000:07:00.1]-00af:7: Performing ISP error recovery - ha=ffffa0e934e30000.
[ 427.697601] qla2xxx [0000:07:00.1]-500a:7: LOOP UP detected (8 Gbps).
[ 428.473948] qla2xxx [0000:07:00.1]-5030:7: Error entry - invalid handle/queue (1c01). ***** NoteI have tested the ISP2532 adapter on my DL380p G8 server and did not run into issue described. Can you share complete log with me.
[ 615.007032] qla2xxx [0000:27:00.0]-4800:7: DPC handler sleeping. [ 615.124217] qla2xxx [0000:27:00.0]-00fb:7: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 615.124231] qla2xxx [0000:27:00.0]-00fc:7: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=7 fw=8.06.02 (90d5). [ 615.196043] qla2xxx [0000:27:00.0]-580e:7: Asynchronous P2P MODE received. [ 615.235284] qla2xxx [0000:27:00.0]-480b:7: Reset marker scheduled. [ 615.235288] qla2xxx [0000:27:00.0]-480c:7: Reset marker end. [ 615.235291] qla2xxx [0000:27:00.0]-480f:7: Loop resync scheduled. [ 615.237903] qla2xxx [0000:27:00.0]-500a:7: LOOP UP detected (8 Gbps). [ 615.237945] qla2xxx [0000:27:00.0]-5812:7: Port database changed ffff 0006 0000. [ 615.237948] qla2xxx [0000:27:00.0]-107ff:7: Mark all dev lost [ 615.760115] qla2xxx [0000:27:00.0]-8837:7: F/W Ready - OK. [ 615.760177] qla2xxx [0000:27:00.0]-883a:7: fw_state=3 (3, 0, 1e, 800 0) curr time=100013490. [ 615.760285] qla2xxx [0000:27:00.0]-280e:7: HBA in F P2P topology. [ 615.760304] qla2xxx [0000:27:00.0]-2814:7: Configure loop -- dpc flags = 0x152e0. [ 615.763509] qla2xxx [0000:27:00.0]-287d:7: NodeName = 20000024ff460a5c. [ 615.763514] qla2xxx [0000:27:00.0]-28a5:7: Manufacturer = QLogic Corporation. [ 615.763519] qla2xxx [0000:27:00.0]-28a6:7: Serial no. = BFD1230D31975. [ 615.763521] qla2xxx [0000:27:00.0]-28a7:7: Model Name = QLE2562. [ 615.763537] qla2xxx [0000:27:00.0]-28a8:7: Model Desc = PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 615.763540] qla2xxx [0000:27:00.0]-28a9:7: Hardware ver = PX2810403-01 F. [ 615.763543] qla2xxx [0000:27:00.0]-28aa:7: Driver ver = 8.07.00.38-k-debug. [ 615.763547] qla2xxx [0000:27:00.0]-28ab:7: Optrom version = 46.51. [ 615.763551] qla2xxx [0000:27:00.0]-28ac:7: Firmware vers = 8.06.02 (90d5). [ 615.763555] qla2xxx [0000:27:00.0]-28ae:7: OS Name and Version = Linux 4.11.0-rc2+ #13 SMP Mon Mar 13 21:51:13 PDT 2017. [ 615.763558] qla2xxx [0000:27:00.0]-28af:7: CT Payload Length = 0x80000. [ 615.763562] qla2xxx [0000:27:00.0]-28b0:7: Symbolic Name = QLE2562 FW:v8.06.02 DVR:v8.07.00.38-k-debug. [ 615.763564] qla2xxx [0000:27:00.0]-28b1:7: Vendor Id = 77100000. [ 615.763567] qla2xxx [0000:27:00.0]-28b2:7: Port Num = 1000000. [ 615.763569] qla2xxx [0000:27:00.0]-28b3:7: Fabric Name = 1000000533aa0b87. [ 615.763572] qla2xxx [0000:27:00.0]-28b4:7: BIOS Name = BIOS 3.00 [ 615.763575] qla2xxx [0000:27:00.0]-28b1:7: Vendor Identifier = QLGC. [ 615.763577] qla2xxx [0000:27:00.0]-28b5:7: RHBA identifier = 21000024ff460a5c. [ 615.775398] qla2xxx [0000:27:00.0]-28b9:7: RHBA FDMI V2 exiting normally. [ 615.775405] qla2xxx [0000:27:00.0]-28ba:7: FC4_TYPES=01 00. [ 615.775409] qla2xxx [0000:27:00.0]-28bb:7: Supported Port Speed = 1b000000. [ 615.775411] qla2xxx [0000:27:00.0]-28bc:7: Current_Speed = 10000000. [ 615.775414] qla2xxx [0000:27:00.0]-28bc:7: Max_Frame_Size = 80000. [ 615.775417] qla2xxx [0000:27:00.0]-28be:7: OS_Device_Name = qla2xxx:host7. [ 615.775420] qla2xxx [0000:27:00.0]-283d:7: HostName=dut5108. [ 615.775423] qla2xxx [0000:27:00.0]-28c0:7: Node Name = 20000024ff460a5c. [ 615.775425] qla2xxx [0000:27:00.0]-28c1:7: Port Name = 21000024ff460a5c. [ 615.775429] qla2xxx [0000:27:00.0]-28c2:7: port symbolic name = QLE2562 FW:v8.06.02 DVR:v8.07.00.38-k-debug
Manufacturer: HP
Product Name: ProLiant DL380p Gen8
Version: Not Specified
Serial Number: USE334CSJL
UUID: 32333536-3030-5355-4533-333443534A4C Wake-up Type: Power Switch
SKU Number: 653200-B21
Family: ProLiant
Thanks,
Himanshu
----- Original Message -----
Quoted 508 lines
From: "Himanshu Madhani" <[email protected]>
To: "Laurence Oberman" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Tuesday, March 14, 2017 5:11:13 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message ----- > From: "Laurence Oberman" <[email protected]> > To: "Himanshu Madhani" <[email protected]> > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > <[email protected]> > Sent: Monday, March 13, 2017 9:06:38 PM > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based > 8Gb) with BAR 3 error, work fine on 4.9 > > > > ----- Original Message ----- > > From: "Laurence Oberman" <[email protected]> > > To: "Himanshu Madhani" <[email protected]> > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > > <[email protected]> > > Sent: Monday, March 13, 2017 12:54:12 PM > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > (ISP2532-based > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > ----- Original Message ----- > > > From: "Himanshu Madhani" <[email protected]> > > > To: "Laurence Oberman" <[email protected]>, "Chad Dupuis" > > > <[email protected]> > > > Cc: "Linux SCSI List" <[email protected]> > > > Sent: Monday, March 13, 2017 12:39:03 PM > > > Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx > > > (ISP2532-based > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > Hi Laurence, > > > > > > > -----Original Message----- > > > > From: Laurence Oberman [mailto:[email protected]] > > > > Sent: Sunday, March 12, 2017 11:31 AM > > > > To: Dupuis, Chad <[email protected]>; Madhani, Himanshu > > > > <[email protected]> > > > > Cc: Linux SCSI List <[email protected]> > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > > (ISP2532-based > > > > 8Gb) > > > > with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > From: "Laurence Oberman" <[email protected]> > > > > > To: "Chad Dupuis" <[email protected]>, "Himanshu Madhani" > > > > > <[email protected]> > > > > > Cc: "Linux SCSI List" <[email protected]> > > > > > Sent: Sunday, March 12, 2017 7:39:23 AM > > > > > Subject: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > (ISP2532-based > > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > Chad, Himanshu > > > > > > > > > > Before I bisect or go chase changes, wanted to reach out > > > > > because the > > > > > driver seems to be the same version. > > > > > Perhaps this is a PCIE change in the kernel for 4.10 affecting > > > > > the > > > > > load. > > > > > Its the same targetLIO server I have been using for a long time > > > > > with > > > > > 4.9 > > > > > > > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre > > > > > Channel > > > > > to > > > > > PCI Express HBA (rev 02) > > > > > > > > > > With 4.9 I have no issues loading the driver for my targetLIO > > > > > server. > > > > > (DL380G8) > > > > > > > > > > # modinfo qla2xxx | more > > > > > filename: > > > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > firmware: ql2500_fw.bin > > > > > version: 8.07.00.38-k > > > > > license: GPL > > > > > description: QLogic Fibre Channel HBA Driver > > > > > author: QLogic Corporation > > > > > srcversion: 94A8431A85BFF854B97B02D > > > > > > > > > > [ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > > > Channel > > > > > HBA > > > > > Driver: 8.07.00.38-k. > > > > > [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 > > > > > irq > > > > > 106 > > > > > iobase > > > > > 0xffffadce989a1000. > > > > > [ 10.455108] scsi host4: qla2xxx > > > > > [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - > > > > > PCI-Express > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe > > > > > (5.0GT/s > > > > > x8) > > > > > @ > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > > [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 > > > > > irq > > > > > 110 > > > > > iobase > > > > > 0xffffadce989a9000. > > > > > [ 10.662120] scsi host5: qla2xxx > > > > > [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 - > > > > > PCI-Express > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe > > > > > (5.0GT/s > > > > > x8) > > > > > @ > > > > > 0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5). > > > > > > > > > > Rebooting on the same server with 4.10 fails to load > > > > > > > > > > Linux 4.10.0+ > > > > > # modinfo qla2xxx | more > > > > > filename: > > > > > /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > firmware: ql2500_fw.bin > > > > > version: 8.07.00.38-k > > > > > license: GPL > > > > > description: QLogic Fibre Channel HBA Driver > > > > > author: QLogic Corporation > > > > > srcversion: 939E0595E8A3C2E1BE94392 > > > > > > > > > > [ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > > > Channel > > > > > HBA > > > > > Driver: 8.07.00.38-k. > > > > > [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > > > enabled. > > > > > [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 > > > > > irq > > > > > 110 > > > > > iobase > > > > > 0xffffacbf189b1000. > > > > > [ 10.407865] scsi host5: qla2xxx > > > > > [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with error > > > > > -22 > > > > > [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > > > enabled. > > > > > [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 > > > > > irq > > > > > 110 > > > > > iobase > > > > > 0xffffacbf189b9000. > > > > > [ 10.645932] scsi host5: qla2xxx > > > > > [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with error > > > > > -22 > > > > > > > > > > Thanks > > > > > Laurence > > > > > > > > > > > > > I started bisecting this, cannot believe others have not bumped > > > > into > > > > this > > > > on > > > > 4.10. > > > > This is a generic QLE2562 and firmware is loaded by the driver so > > > > wondering > > > > why I am seeing this and other are not. > > > > There is nothing special with the PCIE bus on this DL380G8. > > > > > > > > Anyway during the bisect I got to a point where in the 4.10 > > > > commits I > > > > still > > > > saw > > > > the "BAR 3" message but the probe worked. > > > > > > > > [ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > > Channel HBA > > > > Driver: > > > > 8.07.00.38-k. > > > > [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. > > > > > > > > see this above but probe did not fail > > > > > > > > [ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 > > > > irq 97 > > > > iobase > > > > 0xffffc02f98989000. > > > > [ 7.414738] scsi host4: qla2xxx > > > > > > > > [ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - > > > > PCI-Express > > > > Dual > > > > Channel 8Gb Fibre Channel HBA. > > > > [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe > > > > (5.0GT/s > > > > x8) > > > > @ > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. > > > > [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 > > > > irq 100 > > > > iobase > > > > 0xffffc02f989b1000. > > > > [ 7.625691] scsi host6: qla2xxx > > > > [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - > > > > PCI-Express > > > > Dual > > > > Channel 8Gb Fibre Channel HBA. > > > > [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe > > > > (5.0GT/s > > > > x8) > > > > @ > > > > 0000:27:00.1 hdma+ host#=6 > > > > > > > > I marked that still as bad and am continuing, I have 9 builds to > > > > go. > > > > > > > > Thanks > > > > Laurence > > > > > > We had send patches to address issues discovered when the PCI > > > Affinity > > > were > > > merged into 4.10 kernel. > > > > > > Can you please check if your tree has following patches (Currently > > > they > > > are > > > included in 4.10.2 stable) > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=699abe249b0e2ca590284424d95490e92ccd6aeb > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > > > > Also can you apply following patch which addresses issue in PCI > > > layer > > > (This > > > should be included in 4.10 stable branch soon ) > > > > > > https://www.spinics.net/lists/linux-pci/msg58742.html > > > > > > Thanks, > > > Himanshu > > > > > > > Hello Himanshu > > > > Many thanks for coming back to me. > > OK, I will test all of this and let you know. > > I dont think I have these and this would explain all this. > > > > Regards > > Laurence > > > > Hello Himanshu > This is still failing for me on the HP DL380G8 > > My Stable tree > > root@dhcp40-131 ~]# uname -a > Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+ > > $ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > Author: Michael Hernandez <[email protected]> > Date: Wed Feb 15 15:37:21 2017 -0800 > > scsi: qla2xxx: Fix Regression introduced by > pci_alloc_irq_vectors_affinity call. > > commit 67f2db8792f96d8f7521461635d25f9c80245d80 upstream. > > For target mode, we need to increase minimum vectors value by one > to > account for ATIO queue. > > Following stack trace will be seen > > $ git show 699abe249b0e2ca590284424d95490e92ccd6aeb > commit 699abe249b0e2ca590284424d95490e92ccd6aeb > Author: Michael Hernandez <[email protected]> > Date: Wed Feb 15 15:37:20 2017 -0800 > > scsi: qla2xxx: Fix response queue count for Target mode. > > commit d0d2c68b759bbf678e078fd0c71b5fde65a9392c upstream. > > Target mode initialization was not calculating response queue > values > correctly resulting into one less MSI-X vector. > > $ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > Author: Michael Hernandez <[email protected]> > Date: Wed Feb 15 15:37:19 2017 -0800 > > scsi: qla2xxx: Cleaned up queue configuration code. > > commit f54f2cb540b53d55a81d620e816810d59be5cb1b upstream. > > This patch cleaned up queue configuration code, such that once > initialized, we should not touch msix_count value. This will > prevent > incorrect numbers of MSI-X vectors requested while performing > target > mode configuration. > > > Tested, and fails > > root@dhcp40-131 ~]# dmesg | grep qla > [ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA > Driver: 8.07.00.38-k. > [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. > [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 > iobase > 0xffffb3e4589a1000. > [ 9.788259] scsi host4: qla2xxx > [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with error -22 > [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. > [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 106 > iobase > 0xffffb3e4589a9000. > [ 10.030273] scsi host4: qla2xxx > [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with error -22 > > > Now > Patched > https://www.spinics.net/lists/linux-pci/msg58742.html > > [loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci > patching file drivers/pci/msi.c > Hunk #1 succeeded at 1072 (offset -19 lines). > Hunk #2 succeeded at 1120 (offset -27 lines). > Hunk #3 succeeded at 1200 (offset -27 lines). > > Rebuild kernel > Reboot and test > > Still fails > > [ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA > Driver: 8.07.00.38-k. > [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. > [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 > iobase > 0xffffa3ff58991000. > [ 8.670118] scsi host4: qla2xxx > [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with error -22 > [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. > [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 > iobase > 0xffffa3ff589a1000. > [ 8.912089] scsi host4: qla2xxx > [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with error -22 > > Went to test on another system , HP Microserver G8 > Same HBA > Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI > Express > HBA (rev 02) > > Works here but complains about the vector count support > > [ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA > Driver: 8.07.00.38-k. > [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector count: 32. > [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an ISP2532 irq 16 > iobase > 0xffffbd9100c99000. > [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed to enable > support > with 32 vectors, using 4 vectors. > [ 8.759660] scsi host6: qla2xxx > [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A - HP 8Gb > Dual > Channel PCI-e 2.0 FC HBA. > [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe (5.0GT/s > x8) @ > 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5). > [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector count: 32. > [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an ISP2532 irq 17 > iobase > 0xffffbd9100cb9000. > [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed to enable > support > with 32 vectors, using 4 vectors. > [ 8.999658] scsi host7: qla2xxx > [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A - HP 8Gb > Dual > Channel PCI-e 2.0 FC HBA. > [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe (5.0GT/s > x8) @ > 0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5). > > So something about the DL380 here needs to be figured out as well as > the > MSI-X messages above. > > Earlier kernels up to 4.9.13 are all fine on both servers > > Thanks > Laurence > I meant to add With the G8 Microserver links seem to come up with only the 4 MSI-X vectors but I get this so it not fully healthy [ 373.823667] Rounding down aligned max_sectors from 65535 to 65528 [ 399.994495] qla2xxx [0000:07:00.0]-00af:6: Performing ISP error recovery - ha=ffffa0e934210000. [ 400.728968] qla2xxx [0000:07:00.0]-500a:6: LOOP UP detected (8 Gbps). [ 401.473405] qla2xxx [0000:07:00.0]-5030:6: Error entry - invalid handle/queue (1c01). ***** Note [ 426.966519] qla2xxx [0000:07:00.1]-00af:7: Performing ISP error recovery - ha=ffffa0e934e30000. [ 427.697601] qla2xxx [0000:07:00.1]-500a:7: LOOP UP detected (8 Gbps). [ 428.473948] qla2xxx [0000:07:00.1]-5030:7: Error entry - invalid handle/queue (1c01). ***** NoteI have tested the ISP2532 adapter on my DL380p G8 server and did not run into issue described. Can you share complete log with me.
[ 615.007032] qla2xxx [0000:27:00.0]-4800:7: DPC handler sleeping. [ 615.124217] qla2xxx [0000:27:00.0]-00fb:7: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 615.124231] qla2xxx [0000:27:00.0]-00fc:7: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=7 fw=8.06.02 (90d5). [ 615.196043] qla2xxx [0000:27:00.0]-580e:7: Asynchronous P2P MODE received. [ 615.235284] qla2xxx [0000:27:00.0]-480b:7: Reset marker scheduled. [ 615.235288] qla2xxx [0000:27:00.0]-480c:7: Reset marker end. [ 615.235291] qla2xxx [0000:27:00.0]-480f:7: Loop resync scheduled. [ 615.237903] qla2xxx [0000:27:00.0]-500a:7: LOOP UP detected (8 Gbps). [ 615.237945] qla2xxx [0000:27:00.0]-5812:7: Port database changed ffff 0006 0000. [ 615.237948] qla2xxx [0000:27:00.0]-107ff:7: Mark all dev lost [ 615.760115] qla2xxx [0000:27:00.0]-8837:7: F/W Ready - OK. [ 615.760177] qla2xxx [0000:27:00.0]-883a:7: fw_state=3 (3, 0, 1e, 800 0) curr time=100013490. [ 615.760285] qla2xxx [0000:27:00.0]-280e:7: HBA in F P2P topology. [ 615.760304] qla2xxx [0000:27:00.0]-2814:7: Configure loop -- dpc flags = 0x152e0. [ 615.763509] qla2xxx [0000:27:00.0]-287d:7: NodeName = 20000024ff460a5c. [ 615.763514] qla2xxx [0000:27:00.0]-28a5:7: Manufacturer = QLogic Corporation. [ 615.763519] qla2xxx [0000:27:00.0]-28a6:7: Serial no. = BFD1230D31975. [ 615.763521] qla2xxx [0000:27:00.0]-28a7:7: Model Name = QLE2562. [ 615.763537] qla2xxx [0000:27:00.0]-28a8:7: Model Desc = PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 615.763540] qla2xxx [0000:27:00.0]-28a9:7: Hardware ver = PX2810403-01 F. [ 615.763543] qla2xxx [0000:27:00.0]-28aa:7: Driver ver = 8.07.00.38-k-debug. [ 615.763547] qla2xxx [0000:27:00.0]-28ab:7: Optrom version = 46.51. [ 615.763551] qla2xxx [0000:27:00.0]-28ac:7: Firmware vers = 8.06.02 (90d5). [ 615.763555] qla2xxx [0000:27:00.0]-28ae:7: OS Name and Version = Linux 4.11.0-rc2+ #13 SMP Mon Mar 13 21:51:13 PDT 2017. [ 615.763558] qla2xxx [0000:27:00.0]-28af:7: CT Payload Length = 0x80000. [ 615.763562] qla2xxx [0000:27:00.0]-28b0:7: Symbolic Name = QLE2562 FW:v8.06.02 DVR:v8.07.00.38-k-debug. [ 615.763564] qla2xxx [0000:27:00.0]-28b1:7: Vendor Id = 77100000. [ 615.763567] qla2xxx [0000:27:00.0]-28b2:7: Port Num = 1000000. [ 615.763569] qla2xxx [0000:27:00.0]-28b3:7: Fabric Name = 1000000533aa0b87. [ 615.763572] qla2xxx [0000:27:00.0]-28b4:7: BIOS Name = BIOS 3.00 [ 615.763575] qla2xxx [0000:27:00.0]-28b1:7: Vendor Identifier = QLGC. [ 615.763577] qla2xxx [0000:27:00.0]-28b5:7: RHBA identifier = 21000024ff460a5c. [ 615.775398] qla2xxx [0000:27:00.0]-28b9:7: RHBA FDMI V2 exiting normally. [ 615.775405] qla2xxx [0000:27:00.0]-28ba:7: FC4_TYPES=01 00. [ 615.775409] qla2xxx [0000:27:00.0]-28bb:7: Supported Port Speed = 1b000000. [ 615.775411] qla2xxx [0000:27:00.0]-28bc:7: Current_Speed = 10000000. [ 615.775414] qla2xxx [0000:27:00.0]-28bc:7: Max_Frame_Size = 80000. [ 615.775417] qla2xxx [0000:27:00.0]-28be:7: OS_Device_Name = qla2xxx:host7. [ 615.775420] qla2xxx [0000:27:00.0]-283d:7: HostName=dut5108. [ 615.775423] qla2xxx [0000:27:00.0]-28c0:7: Node Name = 20000024ff460a5c. [ 615.775425] qla2xxx [0000:27:00.0]-28c1:7: Port Name = 21000024ff460a5c. [ 615.775429] qla2xxx [0000:27:00.0]-28c2:7: port symbolic name = QLE2562FW:v8.06.02 DVR:v8.07.00.38-k-debug
Manufacturer: HP
Product Name: ProLiant DL380p Gen8
Version: Not Specified
Serial Number: USE334CSJL
UUID: 32333536-3030-5355-4533-333443534A4C Wake-up Type: Power Switch
SKU Number: 653200-B21
Family: ProLiant
Quoted 4 lines
Thanks,
Himanshu
Hello Himanshu
I just now cloned from Linus's tree and patched with the PCI fixes as you suggested.
This gets me this kernel and its failing.
4.11.0-rc2+
[ 9.099570] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.099704] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 9.099705] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 iobase 0xfffface3d8999000. [ 9.248817] Console: switching to colour frame buffer device 128x48 [ 9.305051] scsi host4: qla2xxx [ 9.341336] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 9.341475] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 9.341478] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 iobase 0xfffface3d89a9000. [ 9.547011] scsi host4: qla2xxx [ 9.583160] qla2xxx: probe of 0000:27:00.1 failed with error -22
I dont know why yours is not failing here.
Did you manage to test with the kernel I provided you yet.
My server:
System Information
Manufacturer: HP
Product Name: ProLiant DL380p Gen8
Version: Not Specified
Serial Number: USE246MH8W
UUID: 32333536-3030-5355-4532-34364D483857 Wake-up Type: Power Switch
SKU Number: 653200-B21
Family: ProLiant
My ROM
Vendor: HP Version: P70 Release Date: 07/01/2015 Address: 0xF0000
All prior kernels are working fine so I know the hardware is good.
This is what I have installed
21:00.0 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev 58) 21:00.1 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev 58) 24:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03) 24:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03) 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02) 27:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
I will attach my boot log in a separate email just to you and Chad
Thanks
Laurence
----- Original Message -----
Quoted 620 lines
From: "Laurence Oberman" <[email protected]>
To: "Himanshu Madhani" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Tuesday, March 14, 2017 8:02:32 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 546 lines
From: "Himanshu Madhani" <[email protected]>
To: "Laurence Oberman" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Tuesday, March 14, 2017 5:11:13 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message ----- > From: "Laurence Oberman" <[email protected]> > To: "Himanshu Madhani" <[email protected]> > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > <[email protected]> > Sent: Monday, March 13, 2017 9:06:38 PM > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > (ISP2532-based > 8Gb) with BAR 3 error, work fine on 4.9 > > > > ----- Original Message ----- > > From: "Laurence Oberman" <[email protected]> > > To: "Himanshu Madhani" <[email protected]> > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > > <[email protected]> > > Sent: Monday, March 13, 2017 12:54:12 PM > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > (ISP2532-based > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > ----- Original Message ----- > > > From: "Himanshu Madhani" <[email protected]> > > > To: "Laurence Oberman" <[email protected]>, "Chad Dupuis" > > > <[email protected]> > > > Cc: "Linux SCSI List" <[email protected]> > > > Sent: Monday, March 13, 2017 12:39:03 PM > > > Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx > > > (ISP2532-based > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > Hi Laurence, > > > > > > > -----Original Message----- > > > > From: Laurence Oberman [mailto:[email protected]] > > > > Sent: Sunday, March 12, 2017 11:31 AM > > > > To: Dupuis, Chad <[email protected]>; Madhani, Himanshu > > > > <[email protected]> > > > > Cc: Linux SCSI List <[email protected]> > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > > (ISP2532-based > > > > 8Gb) > > > > with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > From: "Laurence Oberman" <[email protected]> > > > > > To: "Chad Dupuis" <[email protected]>, "Himanshu > > > > > Madhani" > > > > > <[email protected]> > > > > > Cc: "Linux SCSI List" <[email protected]> > > > > > Sent: Sunday, March 12, 2017 7:39:23 AM > > > > > Subject: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > (ISP2532-based > > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > Chad, Himanshu > > > > > > > > > > Before I bisect or go chase changes, wanted to reach out > > > > > because the > > > > > driver seems to be the same version. > > > > > Perhaps this is a PCIE change in the kernel for 4.10 > > > > > affecting > > > > > the > > > > > load. > > > > > Its the same targetLIO server I have been using for a long > > > > > time > > > > > with > > > > > 4.9 > > > > > > > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre > > > > > Channel > > > > > to > > > > > PCI Express HBA (rev 02) > > > > > > > > > > With 4.9 I have no issues loading the driver for my targetLIO > > > > > server. > > > > > (DL380G8) > > > > > > > > > > # modinfo qla2xxx | more > > > > > filename: > > > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > firmware: ql2500_fw.bin > > > > > version: 8.07.00.38-k > > > > > license: GPL > > > > > description: QLogic Fibre Channel HBA Driver > > > > > author: QLogic Corporation > > > > > srcversion: 94A8431A85BFF854B97B02D > > > > > > > > > > [ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > > > Channel > > > > > HBA > > > > > Driver: 8.07.00.38-k. > > > > > [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an > > > > > ISP2532 > > > > > irq > > > > > 106 > > > > > iobase > > > > > 0xffffadce989a1000. > > > > > [ 10.455108] scsi host4: qla2xxx > > > > > [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 > > > > > - > > > > > PCI-Express > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe > > > > > (5.0GT/s > > > > > x8) > > > > > @ > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > > [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an > > > > > ISP2532 > > > > > irq > > > > > 110 > > > > > iobase > > > > > 0xffffadce989a9000. > > > > > [ 10.662120] scsi host5: qla2xxx > > > > > [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562 > > > > > - > > > > > PCI-Express > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe > > > > > (5.0GT/s > > > > > x8) > > > > > @ > > > > > 0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5). > > > > > > > > > > Rebooting on the same server with 4.10 fails to load > > > > > > > > > > Linux 4.10.0+ > > > > > # modinfo qla2xxx | more > > > > > filename: > > > > > /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > firmware: ql2500_fw.bin > > > > > version: 8.07.00.38-k > > > > > license: GPL > > > > > description: QLogic Fibre Channel HBA Driver > > > > > author: QLogic Corporation > > > > > srcversion: 939E0595E8A3C2E1BE94392 > > > > > > > > > > [ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > > > Channel > > > > > HBA > > > > > Driver: 8.07.00.38-k. > > > > > [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > > > enabled. > > > > > [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an > > > > > ISP2532 > > > > > irq > > > > > 110 > > > > > iobase > > > > > 0xffffacbf189b1000. > > > > > [ 10.407865] scsi host5: qla2xxx > > > > > [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with > > > > > error > > > > > -22 > > > > > [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > > > enabled. > > > > > [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an > > > > > ISP2532 > > > > > irq > > > > > 110 > > > > > iobase > > > > > 0xffffacbf189b9000. > > > > > [ 10.645932] scsi host5: qla2xxx > > > > > [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with > > > > > error > > > > > -22 > > > > > > > > > > Thanks > > > > > Laurence > > > > > > > > > > > > > I started bisecting this, cannot believe others have not bumped > > > > into > > > > this > > > > on > > > > 4.10. > > > > This is a generic QLE2562 and firmware is loaded by the driver > > > > so > > > > wondering > > > > why I am seeing this and other are not. > > > > There is nothing special with the PCIE bus on this DL380G8. > > > > > > > > Anyway during the bisect I got to a point where in the 4.10 > > > > commits I > > > > still > > > > saw > > > > the "BAR 3" message but the probe worked. > > > > > > > > [ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > > Channel HBA > > > > Driver: > > > > 8.07.00.38-k. > > > > [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > > enabled. > > > > > > > > see this above but probe did not fail > > > > > > > > [ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 > > > > irq 97 > > > > iobase > > > > 0xffffc02f98989000. > > > > [ 7.414738] scsi host4: qla2xxx > > > > > > > > [ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 - > > > > PCI-Express > > > > Dual > > > > Channel 8Gb Fibre Channel HBA. > > > > [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe > > > > (5.0GT/s > > > > x8) > > > > @ > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > > enabled. > > > > [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 > > > > irq 100 > > > > iobase > > > > 0xffffc02f989b1000. > > > > [ 7.625691] scsi host6: qla2xxx > > > > [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - > > > > PCI-Express > > > > Dual > > > > Channel 8Gb Fibre Channel HBA. > > > > [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe > > > > (5.0GT/s > > > > x8) > > > > @ > > > > 0000:27:00.1 hdma+ host#=6 > > > > > > > > I marked that still as bad and am continuing, I have 9 builds > > > > to > > > > go. > > > > > > > > Thanks > > > > Laurence > > > > > > We had send patches to address issues discovered when the PCI > > > Affinity > > > were > > > merged into 4.10 kernel. > > > > > > Can you please check if your tree has following patches > > > (Currently > > > they > > > are > > > included in 4.10.2 stable) > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=699abe249b0e2ca590284424d95490e92ccd6aeb > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > > > > Also can you apply following patch which addresses issue in PCI > > > layer > > > (This > > > should be included in 4.10 stable branch soon ) > > > > > > https://www.spinics.net/lists/linux-pci/msg58742.html > > > > > > Thanks, > > > Himanshu > > > > > > > Hello Himanshu > > > > Many thanks for coming back to me. > > OK, I will test all of this and let you know. > > I dont think I have these and this would explain all this. > > > > Regards > > Laurence > > > > Hello Himanshu > This is still failing for me on the HP DL380G8 > > My Stable tree > > root@dhcp40-131 ~]# uname -a > Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+ > > $ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > Author: Michael Hernandez <[email protected]> > Date: Wed Feb 15 15:37:21 2017 -0800 > > scsi: qla2xxx: Fix Regression introduced by > pci_alloc_irq_vectors_affinity call. > > commit 67f2db8792f96d8f7521461635d25f9c80245d80 upstream. > > For target mode, we need to increase minimum vectors value by one > to > account for ATIO queue. > > Following stack trace will be seen > > $ git show 699abe249b0e2ca590284424d95490e92ccd6aeb > commit 699abe249b0e2ca590284424d95490e92ccd6aeb > Author: Michael Hernandez <[email protected]> > Date: Wed Feb 15 15:37:20 2017 -0800 > > scsi: qla2xxx: Fix response queue count for Target mode. > > commit d0d2c68b759bbf678e078fd0c71b5fde65a9392c upstream. > > Target mode initialization was not calculating response queue > values > correctly resulting into one less MSI-X vector. > > $ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > Author: Michael Hernandez <[email protected]> > Date: Wed Feb 15 15:37:19 2017 -0800 > > scsi: qla2xxx: Cleaned up queue configuration code. > > commit f54f2cb540b53d55a81d620e816810d59be5cb1b upstream. > > This patch cleaned up queue configuration code, such that once > initialized, we should not touch msix_count value. This will > prevent > incorrect numbers of MSI-X vectors requested while performing > target > mode configuration. > > > Tested, and fails > > root@dhcp40-131 ~]# dmesg | grep qla > [ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel > HBA > Driver: 8.07.00.38-k. > [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. > [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq > 106 > iobase > 0xffffb3e4589a1000. > [ 9.788259] scsi host4: qla2xxx > [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with error -22 > [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. > [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq > 106 > iobase > 0xffffb3e4589a9000. > [ 10.030273] scsi host4: qla2xxx > [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with error -22 > > > Now > Patched > https://www.spinics.net/lists/linux-pci/msg58742.html > > [loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci > patching file drivers/pci/msi.c > Hunk #1 succeeded at 1072 (offset -19 lines). > Hunk #2 succeeded at 1120 (offset -27 lines). > Hunk #3 succeeded at 1200 (offset -27 lines). > > Rebuild kernel > Reboot and test > > Still fails > > [ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel > HBA > Driver: 8.07.00.38-k. > [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. > [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq > 101 > iobase > 0xffffa3ff58991000. > [ 8.670118] scsi host4: qla2xxx > [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with error -22 > [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. > [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq > 101 > iobase > 0xffffa3ff589a1000. > [ 8.912089] scsi host4: qla2xxx > [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with error -22 > > Went to test on another system , HP Microserver G8 > Same HBA > Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI > Express > HBA (rev 02) > > Works here but complains about the vector count support > > [ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel > HBA > Driver: 8.07.00.38-k. > [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector count: 32. > [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an ISP2532 irq 16 > iobase > 0xffffbd9100c99000. > [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed to enable > support > with 32 vectors, using 4 vectors. > [ 8.759660] scsi host6: qla2xxx > [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A - HP > 8Gb > Dual > Channel PCI-e 2.0 FC HBA. > [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe (5.0GT/s > x8) @ > 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5). > [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector count: 32. > [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an ISP2532 irq 17 > iobase > 0xffffbd9100cb9000. > [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed to enable > support > with 32 vectors, using 4 vectors. > [ 8.999658] scsi host7: qla2xxx > [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A - HP > 8Gb > Dual > Channel PCI-e 2.0 FC HBA. > [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe (5.0GT/s > x8) @ > 0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5). > > So something about the DL380 here needs to be figured out as well as > the > MSI-X messages above. > > Earlier kernels up to 4.9.13 are all fine on both servers > > Thanks > Laurence > I meant to add With the G8 Microserver links seem to come up with only the 4 MSI-X vectors but I get this so it not fully healthy [ 373.823667] Rounding down aligned max_sectors from 65535 to 65528 [ 399.994495] qla2xxx [0000:07:00.0]-00af:6: Performing ISP error recovery - ha=ffffa0e934210000. [ 400.728968] qla2xxx [0000:07:00.0]-500a:6: LOOP UP detected (8 Gbps). [ 401.473405] qla2xxx [0000:07:00.0]-5030:6: Error entry - invalid handle/queue (1c01). ***** Note [ 426.966519] qla2xxx [0000:07:00.1]-00af:7: Performing ISP error recovery - ha=ffffa0e934e30000. [ 427.697601] qla2xxx [0000:07:00.1]-500a:7: LOOP UP detected (8 Gbps). [ 428.473948] qla2xxx [0000:07:00.1]-5030:7: Error entry - invalid handle/queue (1c01). ***** NoteI have tested the ISP2532 adapter on my DL380p G8 server and did not run into
issue described. Can you share complete log with me.[ 615.007032] qla2xxx [0000:27:00.0]-4800:7: DPC handler sleeping. [ 615.124217] qla2xxx [0000:27:00.0]-00fb:7: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 615.124231] qla2xxx [0000:27:00.0]-00fc:7: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=7 fw=8.06.02 (90d5). [ 615.196043] qla2xxx [0000:27:00.0]-580e:7: Asynchronous P2P MODE received. [ 615.235284] qla2xxx [0000:27:00.0]-480b:7: Reset marker scheduled. [ 615.235288] qla2xxx [0000:27:00.0]-480c:7: Reset marker end. [ 615.235291] qla2xxx [0000:27:00.0]-480f:7: Loop resync scheduled. [ 615.237903] qla2xxx [0000:27:00.0]-500a:7: LOOP UP detected (8 Gbps). [ 615.237945] qla2xxx [0000:27:00.0]-5812:7: Port database changed ffff 0006 0000. [ 615.237948] qla2xxx [0000:27:00.0]-107ff:7: Mark all dev lost [ 615.760115] qla2xxx [0000:27:00.0]-8837:7: F/W Ready - OK. [ 615.760177] qla2xxx [0000:27:00.0]-883a:7: fw_state=3 (3, 0, 1e, 800 0) curr time=100013490. [ 615.760285] qla2xxx [0000:27:00.0]-280e:7: HBA in F P2P topology. [ 615.760304] qla2xxx [0000:27:00.0]-2814:7: Configure loop -- dpc flags = 0x152e0. [ 615.763509] qla2xxx [0000:27:00.0]-287d:7: NodeName = 20000024ff460a5c. [ 615.763514] qla2xxx [0000:27:00.0]-28a5:7: Manufacturer = QLogic Corporation. [ 615.763519] qla2xxx [0000:27:00.0]-28a6:7: Serial no. = BFD1230D31975. [ 615.763521] qla2xxx [0000:27:00.0]-28a7:7: Model Name = QLE2562. [ 615.763537] qla2xxx [0000:27:00.0]-28a8:7: Model Desc = PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 615.763540] qla2xxx [0000:27:00.0]-28a9:7: Hardware ver = PX2810403-01 F. [ 615.763543] qla2xxx [0000:27:00.0]-28aa:7: Driver ver = 8.07.00.38-k-debug. [ 615.763547] qla2xxx [0000:27:00.0]-28ab:7: Optrom version = 46.51. [ 615.763551] qla2xxx [0000:27:00.0]-28ac:7: Firmware vers = 8.06.02 (90d5). [ 615.763555] qla2xxx [0000:27:00.0]-28ae:7: OS Name and Version = Linux 4.11.0-rc2+ #13 SMP Mon Mar 13 21:51:13 PDT 2017. [ 615.763558] qla2xxx [0000:27:00.0]-28af:7: CT Payload Length = 0x80000. [ 615.763562] qla2xxx [0000:27:00.0]-28b0:7: Symbolic Name = QLE2562 FW:v8.06.02 DVR:v8.07.00.38-k-debug. [ 615.763564] qla2xxx [0000:27:00.0]-28b1:7: Vendor Id = 77100000. [ 615.763567] qla2xxx [0000:27:00.0]-28b2:7: Port Num = 1000000. [ 615.763569] qla2xxx [0000:27:00.0]-28b3:7: Fabric Name = 1000000533aa0b87. [ 615.763572] qla2xxx [0000:27:00.0]-28b4:7: BIOS Name = BIOS 3.00 [ 615.763575] qla2xxx [0000:27:00.0]-28b1:7: Vendor Identifier = QLGC. [ 615.763577] qla2xxx [0000:27:00.0]-28b5:7: RHBA identifier = 21000024ff460a5c. [ 615.775398] qla2xxx [0000:27:00.0]-28b9:7: RHBA FDMI V2 exiting normally. [ 615.775405] qla2xxx [0000:27:00.0]-28ba:7: FC4_TYPES=01 00. [ 615.775409] qla2xxx [0000:27:00.0]-28bb:7: Supported Port Speed = 1b000000. [ 615.775411] qla2xxx [0000:27:00.0]-28bc:7: Current_Speed = 10000000. [ 615.775414] qla2xxx [0000:27:00.0]-28bc:7: Max_Frame_Size = 80000. [ 615.775417] qla2xxx [0000:27:00.0]-28be:7: OS_Device_Name = qla2xxx:host7. [ 615.775420] qla2xxx [0000:27:00.0]-283d:7: HostName=dut5108. [ 615.775423] qla2xxx [0000:27:00.0]-28c0:7: Node Name = 20000024ff460a5c. [ 615.775425] qla2xxx [0000:27:00.0]-28c1:7: Port Name = 21000024ff460a5c. [ 615.775429] qla2xxx [0000:27:00.0]-28c2:7: port symbolic name = QLE2562FW:v8.06.02 DVR:v8.07.00.38-k-debug
Manufacturer: HP
Product Name: ProLiant DL380p Gen8
Version: Not Specified
Serial Number: USE334CSJL
UUID: 32333536-3030-5355-4533-333443534A4C Wake-up Type: Power Switch
SKU Number: 653200-B21
Family: ProLiantQuoted 4 lines
Thanks,
HimanshuHello Himanshu
I just now cloned from Linus's tree and patched with the PCI fixes as you suggested.
This gets me this kernel and its failing.
4.11.0-rc2+
[ 9.099570] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 9.099704] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 9.099705] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 iobase 0xfffface3d8999000. [ 9.248817] Console: switching to colour frame buffer device 128x48 [ 9.305051] scsi host4: qla2xxx [ 9.341336] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 9.341475] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 9.341478] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 iobase 0xfffface3d89a9000. [ 9.547011] scsi host4: qla2xxx [ 9.583160] qla2xxx: probe of 0000:27:00.1 failed with error -22I dont know why yours is not failing here.
Did you manage to test with the kernel I provided you yet.My server:
System Information
Manufacturer: HP
Product Name: ProLiant DL380p Gen8
Version: Not Specified
Serial Number: USE246MH8W
UUID: 32333536-3030-5355-4532-34364D483857 Wake-up Type: Power Switch
SKU Number: 653200-B21
Family: ProLiantMy ROM
Vendor: HP Version: P70 Release Date: 07/01/2015 Address: 0xF0000All prior kernels are working fine so I know the hardware is good.
This is what I have installed
21:00.0 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev
58)
21:00.1 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev
58)
24:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
24:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
27:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
Quoted 5 lines
I will attach my boot log in a separate email just to you and Chad
Thanks
Laurence
Hello
Himanshu
Please let me have your .config you are using for your kernel.
Thanks
Laurence
----- Original Message -----
> From: "Laurence Oberman" <[email protected]>
> To: "Himanshu Madhani" <[email protected]>
> Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
> Sent: Tuesday, March 14, 2017 8:02:32 PM
> Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9
>
>
>
> ----- Original Message -----
> > From: "Himanshu Madhani" <[email protected]>
> > To: "Laurence Oberman" <[email protected]>
> > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List"
> > <[email protected]>
> > Sent: Tuesday, March 14, 2017 5:11:13 PM
> > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based
> > 8Gb) with BAR 3 error, work fine on 4.9
> >
> >
> > ----- Original Message -----
> > > From: "Laurence Oberman" <[email protected]>
> > > To: "Himanshu Madhani" <[email protected]>
> > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List"
> > > <[email protected]>
> > > Sent: Monday, March 13, 2017 9:06:38 PM
> > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx
> > > (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Laurence Oberman" <[email protected]>
> > > > To: "Himanshu Madhani" <[email protected]>
> > > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List"
> > > > <[email protected]>
> > > > Sent: Monday, March 13, 2017 12:54:12 PM
> > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx
> > > > (ISP2532-based
> > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Himanshu Madhani" <[email protected]>
> > > > > To: "Laurence Oberman" <[email protected]>, "Chad Dupuis"
> > > > > <[email protected]>
> > > > > Cc: "Linux SCSI List" <[email protected]>
> > > > > Sent: Monday, March 13, 2017 12:39:03 PM
> > > > > Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx
> > > > > (ISP2532-based
> > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > >
> > > > > Hi Laurence,
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Laurence Oberman [mailto:[email protected]]
> > > > > > Sent: Sunday, March 12, 2017 11:31 AM
> > > > > > To: Dupuis, Chad <[email protected]>; Madhani, Himanshu
> > > > > > <[email protected]>
> > > > > > Cc: Linux SCSI List <[email protected]>
> > > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx
> > > > > > (ISP2532-based
> > > > > > 8Gb)
> > > > > > with BAR 3 error, work fine on 4.9
> > > > > >
> > > > > >
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > > From: "Laurence Oberman" <[email protected]>
> > > > > > > To: "Chad Dupuis" <[email protected]>, "Himanshu
> > > > > > > Madhani"
> > > > > > > <[email protected]>
> > > > > > > Cc: "Linux SCSI List" <[email protected]>
> > > > > > > Sent: Sunday, March 12, 2017 7:39:23 AM
> > > > > > > Subject: 4.10+ qla2xxx driver wont load for qla2xxx
> > > > > > > (ISP2532-based
> > > > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > > >
> > > > > > > Chad, Himanshu
> > > > > > >
> > > > > > > Before I bisect or go chase changes, wanted to reach out
> > > > > > > because the
> > > > > > > driver seems to be the same version.
> > > > > > > Perhaps this is a PCIE change in the kernel for 4.10
> > > > > > > affecting
> > > > > > > the
> > > > > > > load.
> > > > > > > Its the same targetLIO server I have been using for a long
> > > > > > > time
> > > > > > > with
> > > > > > > 4.9
> > > > > > >
> > > > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre
> > > > > > > Channel
> > > > > > > to
> > > > > > > PCI Express HBA (rev 02)
> > > > > > >
> > > > > > > With 4.9 I have no issues loading the driver for my targetLIO
> > > > > > > server.
> > > > > > > (DL380G8)
> > > > > > >
> > > > > > > # modinfo qla2xxx | more
> > > > > > > filename:
> > > > > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > > > > > firmware: ql2500_fw.bin
> > > > > > > version: 8.07.00.38-k
> > > > > > > license: GPL
> > > > > > > description: QLogic Fibre Channel HBA Driver
> > > > > > > author: QLogic Corporation
> > > > > > > srcversion: 94A8431A85BFF854B97B02D
> > > > > > >
> > > > > > > [ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre
> > > > > > > Channel
> > > > > > > HBA
> > > > > > > Driver: 8.07.00.38-k.
> > > > > > > [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an
> > > > > > > ISP2532
> > > > > > > irq
> > > > > > > 106
> > > > > > > iobase
> > > > > > > 0xffffadce989a1000.
> > > > > > > [ 10.455108] scsi host4: qla2xxx
> > > > > > > [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562
> > > > > > > -
> > > > > > > PCI-Express
> > > > > > > Dual Channel 8Gb Fibre Channel HBA.
> > > > > > > [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe
> > > > > > > (5.0GT/s
> > > > > > > x8)
> > > > > > > @
> > > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
> > > > > > > [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an
> > > > > > > ISP2532
> > > > > > > irq
> > > > > > > 110
> > > > > > > iobase
> > > > > > > 0xffffadce989a9000.
> > > > > > > [ 10.662120] scsi host5: qla2xxx
> > > > > > > [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic QLE2562
> > > > > > > -
> > > > > > > PCI-Express
> > > > > > > Dual Channel 8Gb Fibre Channel HBA.
> > > > > > > [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: PCIe
> > > > > > > (5.0GT/s
> > > > > > > x8)
> > > > > > > @
> > > > > > > 0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
> > > > > > >
> > > > > > > Rebooting on the same server with 4.10 fails to load
> > > > > > >
> > > > > > > Linux 4.10.0+
> > > > > > > # modinfo qla2xxx | more
> > > > > > > filename:
> > > > > > > /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > > > > > firmware: ql2500_fw.bin
> > > > > > > version: 8.07.00.38-k
> > > > > > > license: GPL
> > > > > > > description: QLogic Fibre Channel HBA Driver
> > > > > > > author: QLogic Corporation
> > > > > > > srcversion: 939E0595E8A3C2E1BE94392
> > > > > > >
> > > > > > > [ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre
> > > > > > > Channel
> > > > > > > HBA
> > > > > > > Driver: 8.07.00.38-k.
> > > > > > > [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not
> > > > > > > enabled.
> > > > > > > [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an
> > > > > > > ISP2532
> > > > > > > irq
> > > > > > > 110
> > > > > > > iobase
> > > > > > > 0xffffacbf189b1000.
> > > > > > > [ 10.407865] scsi host5: qla2xxx
> > > > > > > [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed with
> > > > > > > error
> > > > > > > -22
> > > > > > > [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not
> > > > > > > enabled.
> > > > > > > [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an
> > > > > > > ISP2532
> > > > > > > irq
> > > > > > > 110
> > > > > > > iobase
> > > > > > > 0xffffacbf189b9000.
> > > > > > > [ 10.645932] scsi host5: qla2xxx
> > > > > > > [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed with
> > > > > > > error
> > > > > > > -22
> > > > > > >
> > > > > > > Thanks
> > > > > > > Laurence
> > > > > > >
> > > > > >
> > > > > > I started bisecting this, cannot believe others have not bumped
> > > > > > into
> > > > > > this
> > > > > > on
> > > > > > 4.10.
> > > > > > This is a generic QLE2562 and firmware is loaded by the driver
> > > > > > so
> > > > > > wondering
> > > > > > why I am seeing this and other are not.
> > > > > > There is nothing special with the PCIE bus on this DL380G8.
> > > > > >
> > > > > > Anyway during the bisect I got to a point where in the 4.10
> > > > > > commits I
> > > > > > still
> > > > > > saw
> > > > > > the "BAR 3" message but the probe worked.
> > > > > >
> > > > > > [ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre
> > > > > > Channel HBA
> > > > > > Driver:
> > > > > > 8.07.00.38-k.
> > > > > > [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not
> > > > > > enabled.
> > > > > >
> > > > > > see this above but probe did not fail
> > > > > >
> > > > > > [ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532
> > > > > > irq 97
> > > > > > iobase
> > > > > > 0xffffc02f98989000.
> > > > > > [ 7.414738] scsi host4: qla2xxx
> > > > > >
> > > > > > [ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic QLE2562 -
> > > > > > PCI-Express
> > > > > > Dual
> > > > > > Channel 8Gb Fibre Channel HBA.
> > > > > > [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: PCIe
> > > > > > (5.0GT/s
> > > > > > x8)
> > > > > > @
> > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
> > > > > > [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not
> > > > > > enabled.
> > > > > > [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532
> > > > > > irq 100
> > > > > > iobase
> > > > > > 0xffffc02f989b1000.
> > > > > > [ 7.625691] scsi host6: qla2xxx
> > > > > > [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 -
> > > > > > PCI-Express
> > > > > > Dual
> > > > > > Channel 8Gb Fibre Channel HBA.
> > > > > > [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe
> > > > > > (5.0GT/s
> > > > > > x8)
> > > > > > @
> > > > > > 0000:27:00.1 hdma+ host#=6
> > > > > >
> > > > > > I marked that still as bad and am continuing, I have 9 builds
> > > > > > to
> > > > > > go.
> > > > > >
> > > > > > Thanks
> > > > > > Laurence
> > > > >
> > > > > We had send patches to address issues discovered when the PCI
> > > > > Affinity
> > > > > were
> > > > > merged into 4.10 kernel.
> > > > >
> > > > > Can you please check if your tree has following patches
> > > > > (Currently
> > > > > they
> > > > > are
> > > > > included in 4.10.2 stable)
> > > > >
> > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c
> > > > >
> > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=699abe249b0e2ca590284424d95490e92ccd6aeb
> > > > >
> > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=cf9d9110806493e2ea2db7d00a1a991401dfb9d5
> > > > >
> > > > > Also can you apply following patch which addresses issue in PCI
> > > > > layer
> > > > > (This
> > > > > should be included in 4.10 stable branch soon )
> > > > >
> > > > > https://www.spinics.net/lists/linux-pci/msg58742.html
> > > > >
> > > > > Thanks,
> > > > > Himanshu
> > > > >
> > > >
> > > > Hello Himanshu
> > > >
> > > > Many thanks for coming back to me.
> > > > OK, I will test all of this and let you know.
> > > > I dont think I have these and this would explain all this.
> > > >
> > > > Regards
> > > > Laurence
> > > >
> > >
> > > Hello Himanshu
> > > This is still failing for me on the HP DL380G8
> > >
> > > My Stable tree
> > >
> > > root@dhcp40-131 ~]# uname -a
> > > Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+
> > >
> > > $ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c
> > > commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c
> > > Author: Michael Hernandez <[email protected]>
> > > Date: Wed Feb 15 15:37:21 2017 -0800
> > >
> > > scsi: qla2xxx: Fix Regression introduced by
> > > pci_alloc_irq_vectors_affinity call.
> > >
> > > commit 67f2db8792f96d8f7521461635d25f9c80245d80 upstream.
> > >
> > > For target mode, we need to increase minimum vectors value by one
> > > to
> > > account for ATIO queue.
> > >
> > > Following stack trace will be seen
> > >
> > > $ git show 699abe249b0e2ca590284424d95490e92ccd6aeb
> > > commit 699abe249b0e2ca590284424d95490e92ccd6aeb
> > > Author: Michael Hernandez <[email protected]>
> > > Date: Wed Feb 15 15:37:20 2017 -0800
> > >
> > > scsi: qla2xxx: Fix response queue count for Target mode.
> > >
> > > commit d0d2c68b759bbf678e078fd0c71b5fde65a9392c upstream.
> > >
> > > Target mode initialization was not calculating response queue
> > > values
> > > correctly resulting into one less MSI-X vector.
> > >
> > > $ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5
> > > commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5
> > > Author: Michael Hernandez <[email protected]>
> > > Date: Wed Feb 15 15:37:19 2017 -0800
> > >
> > > scsi: qla2xxx: Cleaned up queue configuration code.
> > >
> > > commit f54f2cb540b53d55a81d620e816810d59be5cb1b upstream.
> > >
> > > This patch cleaned up queue configuration code, such that once
> > > initialized, we should not touch msix_count value. This will
> > > prevent
> > > incorrect numbers of MSI-X vectors requested while performing
> > > target
> > > mode configuration.
> > >
> > >
> > > Tested, and fails
> > >
> > > root@dhcp40-131 ~]# dmesg | grep qla
> > > [ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel
> > > HBA
> > > Driver: 8.07.00.38-k.
> > > [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
> > > [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq
> > > 106
> > > iobase
> > > 0xffffb3e4589a1000.
> > > [ 9.788259] scsi host4: qla2xxx
> > > [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with error -22
> > > [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled.
> > > [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq
> > > 106
> > > iobase
> > > 0xffffb3e4589a9000.
> > > [ 10.030273] scsi host4: qla2xxx
> > > [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with error -22
> > >
> > >
> > > Now
> > > Patched
> > > https://www.spinics.net/lists/linux-pci/msg58742.html
> > >
> > > [loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci
> > > patching file drivers/pci/msi.c
> > > Hunk #1 succeeded at 1072 (offset -19 lines).
> > > Hunk #2 succeeded at 1120 (offset -27 lines).
> > > Hunk #3 succeeded at 1200 (offset -27 lines).
> > >
> > > Rebuild kernel
> > > Reboot and test
> > >
> > > Still fails
> > >
> > > [ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel
> > > HBA
> > > Driver: 8.07.00.38-k.
> > > [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
> > > [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq
> > > 101
> > > iobase
> > > 0xffffa3ff58991000.
> > > [ 8.670118] scsi host4: qla2xxx
> > > [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with error -22
> > > [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled.
> > > [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq
> > > 101
> > > iobase
> > > 0xffffa3ff589a1000.
> > > [ 8.912089] scsi host4: qla2xxx
> > > [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with error -22
> > >
> > > Went to test on another system , HP Microserver G8
> > > Same HBA
> > > Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI
> > > Express
> > > HBA (rev 02)
> > >
> > > Works here but complains about the vector count support
> > >
> > > [ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel
> > > HBA
> > > Driver: 8.07.00.38-k.
> > > [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector count: 32.
> > > [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an ISP2532 irq 16
> > > iobase
> > > 0xffffbd9100c99000.
> > > [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed to enable
> > > support
> > > with 32 vectors, using 4 vectors.
> > > [ 8.759660] scsi host6: qla2xxx
> > > [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A - HP
> > > 8Gb
> > > Dual
> > > Channel PCI-e 2.0 FC HBA.
> > > [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe (5.0GT/s
> > > x8) @
> > > 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5).
> > > [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector count: 32.
> > > [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an ISP2532 irq 17
> > > iobase
> > > 0xffffbd9100cb9000.
> > > [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed to enable
> > > support
> > > with 32 vectors, using 4 vectors.
> > > [ 8.999658] scsi host7: qla2xxx
> > > [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A - HP
> > > 8Gb
> > > Dual
> > > Channel PCI-e 2.0 FC HBA.
> > > [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe (5.0GT/s
> > > x8) @
> > > 0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5).
> > >
> > > So something about the DL380 here needs to be figured out as well as
> > > the
> > > MSI-X messages above.
> > >
> > > Earlier kernels up to 4.9.13 are all fine on both servers
> > >
> > > Thanks
> > > Laurence
> > >
> >
> > I meant to add
> >
> > With the G8 Microserver links seem to come up with only the 4 MSI-X
> > vectors but I get this
> > so it not fully healthy
> >
> > [ 373.823667] Rounding down aligned max_sectors from 65535 to 65528
> > [ 399.994495] qla2xxx [0000:07:00.0]-00af:6: Performing ISP error
> > recovery - ha=ffffa0e934210000.
> > [ 400.728968] qla2xxx [0000:07:00.0]-500a:6: LOOP UP detected (8
> > Gbps).
> > [ 401.473405] qla2xxx [0000:07:00.0]-5030:6: Error entry - invalid
> > handle/queue (1c01). ***** Note
> > [ 426.966519] qla2xxx [0000:07:00.1]-00af:7: Performing ISP error
> > recovery - ha=ffffa0e934e30000.
> > [ 427.697601] qla2xxx [0000:07:00.1]-500a:7: LOOP UP detected (8
> > Gbps).
> > [ 428.473948] qla2xxx [0000:07:00.1]-5030:7: Error entry - invalid
> > handle/queue (1c01). ***** Note
> >
> > I have tested the ISP2532 adapter on my DL380p G8 server and did not run
> > into
> > issue described. Can you share complete log with me.
> >
> > [ 615.007032] qla2xxx [0000:27:00.0]-4800:7: DPC handler sleeping.
> > [ 615.124217] qla2xxx [0000:27:00.0]-00fb:7: QLogic QLE2562 - PCI-Express
> > Dual Channel 8Gb Fibre Channel HBA.
> > [ 615.124231] qla2xxx [0000:27:00.0]-00fc:7: ISP2532: PCIe (5.0GT/s x8) @
> > 0000:27:00.0 hdma+ host#=7 fw=8.06.02 (90d5).
> > [ 615.196043] qla2xxx [0000:27:00.0]-580e:7: Asynchronous P2P MODE
> > received.
> > [ 615.235284] qla2xxx [0000:27:00.0]-480b:7: Reset marker scheduled.
> > [ 615.235288] qla2xxx [0000:27:00.0]-480c:7: Reset marker end.
> > [ 615.235291] qla2xxx [0000:27:00.0]-480f:7: Loop resync scheduled.
> > [ 615.237903] qla2xxx [0000:27:00.0]-500a:7: LOOP UP detected (8 Gbps).
> > [ 615.237945] qla2xxx [0000:27:00.0]-5812:7: Port database changed ffff
> > 0006
> > 0000.
> > [ 615.237948] qla2xxx [0000:27:00.0]-107ff:7: Mark all dev lost
> > [ 615.760115] qla2xxx [0000:27:00.0]-8837:7: F/W Ready - OK.
> > [ 615.760177] qla2xxx [0000:27:00.0]-883a:7: fw_state=3 (3, 0, 1e, 800 0)
> > curr time=100013490.
> > [ 615.760285] qla2xxx [0000:27:00.0]-280e:7: HBA in F P2P topology.
> > [ 615.760304] qla2xxx [0000:27:00.0]-2814:7: Configure loop -- dpc flags =
> > 0x152e0.
> > [ 615.763509] qla2xxx [0000:27:00.0]-287d:7: NodeName = 20000024ff460a5c.
> > [ 615.763514] qla2xxx [0000:27:00.0]-28a5:7: Manufacturer = QLogic
> > Corporation.
> > [ 615.763519] qla2xxx [0000:27:00.0]-28a6:7: Serial no. = BFD1230D31975.
> > [ 615.763521] qla2xxx [0000:27:00.0]-28a7:7: Model Name = QLE2562.
> > [ 615.763537] qla2xxx [0000:27:00.0]-28a8:7: Model Desc = PCI-Express Dual
> > Channel 8Gb Fibre Channel HBA.
> > [ 615.763540] qla2xxx [0000:27:00.0]-28a9:7: Hardware ver = PX2810403-01
> > F.
> > [ 615.763543] qla2xxx [0000:27:00.0]-28aa:7: Driver ver =
> > 8.07.00.38-k-debug.
> > [ 615.763547] qla2xxx [0000:27:00.0]-28ab:7: Optrom version = 46.51.
> > [ 615.763551] qla2xxx [0000:27:00.0]-28ac:7: Firmware vers = 8.06.02
> > (90d5).
> > [ 615.763555] qla2xxx [0000:27:00.0]-28ae:7: OS Name and Version = Linux
> > 4.11.0-rc2+ #13 SMP Mon Mar 13 21:51:13 PDT 2017.
> > [ 615.763558] qla2xxx [0000:27:00.0]-28af:7: CT Payload Length = 0x80000.
> > [ 615.763562] qla2xxx [0000:27:00.0]-28b0:7: Symbolic Name = QLE2562
> > FW:v8.06.02 DVR:v8.07.00.38-k-debug.
> > [ 615.763564] qla2xxx [0000:27:00.0]-28b1:7: Vendor Id = 77100000.
> > [ 615.763567] qla2xxx [0000:27:00.0]-28b2:7: Port Num = 1000000.
> > [ 615.763569] qla2xxx [0000:27:00.0]-28b3:7: Fabric Name =
> > 1000000533aa0b87.
> > [ 615.763572] qla2xxx [0000:27:00.0]-28b4:7: BIOS Name = BIOS 3.00
> > [ 615.763575] qla2xxx [0000:27:00.0]-28b1:7: Vendor Identifier = QLGC.
> > [ 615.763577] qla2xxx [0000:27:00.0]-28b5:7: RHBA identifier =
> > 21000024ff460a5c.
> > [ 615.775398] qla2xxx [0000:27:00.0]-28b9:7: RHBA FDMI V2 exiting
> > normally.
> > [ 615.775405] qla2xxx [0000:27:00.0]-28ba:7: FC4_TYPES=01 00.
> > [ 615.775409] qla2xxx [0000:27:00.0]-28bb:7: Supported Port Speed =
> > 1b000000.
> > [ 615.775411] qla2xxx [0000:27:00.0]-28bc:7: Current_Speed = 10000000.
> > [ 615.775414] qla2xxx [0000:27:00.0]-28bc:7: Max_Frame_Size = 80000.
> > [ 615.775417] qla2xxx [0000:27:00.0]-28be:7: OS_Device_Name =
> > qla2xxx:host7.
> > [ 615.775420] qla2xxx [0000:27:00.0]-283d:7: HostName=dut5108.
> > [ 615.775423] qla2xxx [0000:27:00.0]-28c0:7: Node Name = 20000024ff460a5c.
> > [ 615.775425] qla2xxx [0000:27:00.0]-28c1:7: Port Name = 21000024ff460a5c.
> > [ 615.775429] qla2xxx [0000:27:00.0]-28c2:7: port symbolic name = QLE2562
> > FW:v8.06.02 DVR:v8.07.00.38-k-debug
> >
> >
> > Manufacturer: HP
> > Product Name: ProLiant DL380p Gen8
> > Version: Not Specified
> > Serial Number: USE334CSJL
> > UUID: 32333536-3030-5355-4533-333443534A4C
> > Wake-up Type: Power Switch
> > SKU Number: 653200-B21
> > Family: ProLiant
> >
> > Thanks,
> > Himanshu
> >
> >
>
> Hello Himanshu
>
> I just now cloned from Linus's tree and patched with the PCI fixes as you
> suggested.
>
> This gets me this kernel and its failing.
>
> 4.11.0-rc2+
>
> [ 9.099570] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA
> Driver: 8.07.00.38-k.
> [ 9.099704] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled.
> [ 9.099705] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 iobase
> 0xfffface3d8999000.
> [ 9.248817] Console: switching to colour frame buffer device 128x48
> [ 9.305051] scsi host4: qla2xxx
> [ 9.341336] qla2xxx: probe of 0000:27:00.0 failed with error -22
> [ 9.341475] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled.
> [ 9.341478] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 iobase
> 0xfffface3d89a9000.
> [ 9.547011] scsi host4: qla2xxx
> [ 9.583160] qla2xxx: probe of 0000:27:00.1 failed with error -22
>
> I dont know why yours is not failing here.
> Did you manage to test with the kernel I provided you yet.
>
> My server:
> System Information
> Manufacturer: HP
> Product Name: ProLiant DL380p Gen8
> Version: Not Specified
> Serial Number: USE246MH8W
> UUID: 32333536-3030-5355-4532-34364D483857
> Wake-up Type: Power Switch
> SKU Number: 653200-B21
> Family: ProLiant
>
> My ROM
> Vendor: HP
> Version: P70
> Release Date: 07/01/2015
> Address: 0xF0000
>
> All prior kernels are working fine so I know the hardware is good.
>
> This is what I have installed
>
> 21:00.0 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev
> 58)
> 21:00.1 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev
> 58)
> 24:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel
> Host Adapter (rev 03)
> 24:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel
> Host Adapter (rev 03)
> 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI
> Express HBA (rev 02)
> 27:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI
> Express HBA (rev 02)
>
> I will attach my boot log in a separate email just to you and Chad
>
> Thanks
> Laurence
>
Hello
Himanshu
Please let me have your .config you are using for your kernel.
Thanks
LaurenceSee attached config file.
----- Original Message -----
Quoted 736 lines
From: "Himanshu Madhani" <[email protected]>
To: "Laurence Oberman" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Tuesday, March 14, 2017 8:32:25 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message ----- > From: "Laurence Oberman" <[email protected]> > To: "Himanshu Madhani" <[email protected]> > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > <[email protected]> > Sent: Tuesday, March 14, 2017 8:02:32 PM > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based > 8Gb) with BAR 3 error, work fine on 4.9 > > > > ----- Original Message ----- > > From: "Himanshu Madhani" <[email protected]> > > To: "Laurence Oberman" <[email protected]> > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > > <[email protected]> > > Sent: Tuesday, March 14, 2017 5:11:13 PM > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > (ISP2532-based > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > ----- Original Message ----- > > > From: "Laurence Oberman" <[email protected]> > > > To: "Himanshu Madhani" <[email protected]> > > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > > > <[email protected]> > > > Sent: Monday, March 13, 2017 9:06:38 PM > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > (ISP2532-based > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Laurence Oberman" <[email protected]> > > > > To: "Himanshu Madhani" <[email protected]> > > > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > > > > <[email protected]> > > > > Sent: Monday, March 13, 2017 12:54:12 PM > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > > (ISP2532-based > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > From: "Himanshu Madhani" <[email protected]> > > > > > To: "Laurence Oberman" <[email protected]>, "Chad Dupuis" > > > > > <[email protected]> > > > > > Cc: "Linux SCSI List" <[email protected]> > > > > > Sent: Monday, March 13, 2017 12:39:03 PM > > > > > Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > (ISP2532-based > > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > Hi Laurence, > > > > > > > > > > > -----Original Message----- > > > > > > From: Laurence Oberman [mailto:[email protected]] > > > > > > Sent: Sunday, March 12, 2017 11:31 AM > > > > > > To: Dupuis, Chad <[email protected]>; Madhani, > > > > > > Himanshu > > > > > > <[email protected]> > > > > > > Cc: Linux SCSI List <[email protected]> > > > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > > (ISP2532-based > > > > > > 8Gb) > > > > > > with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > From: "Laurence Oberman" <[email protected]> > > > > > > > To: "Chad Dupuis" <[email protected]>, "Himanshu > > > > > > > Madhani" > > > > > > > <[email protected]> > > > > > > > Cc: "Linux SCSI List" <[email protected]> > > > > > > > Sent: Sunday, March 12, 2017 7:39:23 AM > > > > > > > Subject: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > > > (ISP2532-based > > > > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > Chad, Himanshu > > > > > > > > > > > > > > Before I bisect or go chase changes, wanted to reach > > > > > > > out > > > > > > > because the > > > > > > > driver seems to be the same version. > > > > > > > Perhaps this is a PCIE change in the kernel for 4.10 > > > > > > > affecting > > > > > > > the > > > > > > > load. > > > > > > > Its the same targetLIO server I have been using for a > > > > > > > long > > > > > > > time > > > > > > > with > > > > > > > 4.9 > > > > > > > > > > > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb > > > > > > > Fibre > > > > > > > Channel > > > > > > > to > > > > > > > PCI Express HBA (rev 02) > > > > > > > > > > > > > > With 4.9 I have no issues loading the driver for my > > > > > > > targetLIO > > > > > > > server. > > > > > > > (DL380G8) > > > > > > > > > > > > > > # modinfo qla2xxx | more > > > > > > > filename: > > > > > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > > > firmware: ql2500_fw.bin > > > > > > > version: 8.07.00.38-k > > > > > > > license: GPL > > > > > > > description: QLogic Fibre Channel HBA Driver > > > > > > > author: QLogic Corporation > > > > > > > srcversion: 94A8431A85BFF854B97B02D > > > > > > > > > > > > > > [ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic > > > > > > > Fibre > > > > > > > Channel > > > > > > > HBA > > > > > > > Driver: 8.07.00.38-k. > > > > > > > [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 106 > > > > > > > iobase > > > > > > > 0xffffadce989a1000. > > > > > > > [ 10.455108] scsi host4: qla2xxx > > > > > > > [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic > > > > > > > QLE2562 > > > > > > > - > > > > > > > PCI-Express > > > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > > > [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: > > > > > > > PCIe > > > > > > > (5.0GT/s > > > > > > > x8) > > > > > > > @ > > > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > > > > [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 110 > > > > > > > iobase > > > > > > > 0xffffadce989a9000. > > > > > > > [ 10.662120] scsi host5: qla2xxx > > > > > > > [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic > > > > > > > QLE2562 > > > > > > > - > > > > > > > PCI-Express > > > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > > > [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: ISP2532: > > > > > > > PCIe > > > > > > > (5.0GT/s > > > > > > > x8) > > > > > > > @ > > > > > > > 0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5). > > > > > > > > > > > > > > Rebooting on the same server with 4.10 fails to load > > > > > > > > > > > > > > Linux 4.10.0+ > > > > > > > # modinfo qla2xxx | more > > > > > > > filename: > > > > > > > /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > > > firmware: ql2500_fw.bin > > > > > > > version: 8.07.00.38-k > > > > > > > license: GPL > > > > > > > description: QLogic Fibre Channel HBA Driver > > > > > > > author: QLogic Corporation > > > > > > > srcversion: 939E0595E8A3C2E1BE94392 > > > > > > > > > > > > > > [ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic > > > > > > > Fibre > > > > > > > Channel > > > > > > > HBA > > > > > > > Driver: 8.07.00.38-k. > > > > > > > [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > > > > > enabled. > > > > > > > [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 110 > > > > > > > iobase > > > > > > > 0xffffacbf189b1000. > > > > > > > [ 10.407865] scsi host5: qla2xxx > > > > > > > [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed > > > > > > > with > > > > > > > error > > > > > > > -22 > > > > > > > [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > > > > > enabled. > > > > > > > [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 110 > > > > > > > iobase > > > > > > > 0xffffacbf189b9000. > > > > > > > [ 10.645932] scsi host5: qla2xxx > > > > > > > [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed > > > > > > > with > > > > > > > error > > > > > > > -22 > > > > > > > > > > > > > > Thanks > > > > > > > Laurence > > > > > > > > > > > > > > > > > > > I started bisecting this, cannot believe others have not > > > > > > bumped > > > > > > into > > > > > > this > > > > > > on > > > > > > 4.10. > > > > > > This is a generic QLE2562 and firmware is loaded by the > > > > > > driver > > > > > > so > > > > > > wondering > > > > > > why I am seeing this and other are not. > > > > > > There is nothing special with the PCIE bus on this > > > > > > DL380G8. > > > > > > > > > > > > Anyway during the bisect I got to a point where in the > > > > > > 4.10 > > > > > > commits I > > > > > > still > > > > > > saw > > > > > > the "BAR 3" message but the probe worked. > > > > > > > > > > > > [ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic > > > > > > Fibre > > > > > > Channel HBA > > > > > > Driver: > > > > > > 8.07.00.38-k. > > > > > > [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > > > > enabled. > > > > > > > > > > > > see this above but probe did not fail > > > > > > > > > > > > [ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an > > > > > > ISP2532 > > > > > > irq 97 > > > > > > iobase > > > > > > 0xffffc02f98989000. > > > > > > [ 7.414738] scsi host4: qla2xxx > > > > > > > > > > > > [ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic > > > > > > QLE2562 - > > > > > > PCI-Express > > > > > > Dual > > > > > > Channel 8Gb Fibre Channel HBA. > > > > > > [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: > > > > > > PCIe > > > > > > (5.0GT/s > > > > > > x8) > > > > > > @ > > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > > > [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > > > > enabled. > > > > > > [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an > > > > > > ISP2532 > > > > > > irq 100 > > > > > > iobase > > > > > > 0xffffc02f989b1000. > > > > > > [ 7.625691] scsi host6: qla2xxx > > > > > > [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic > > > > > > QLE2562 - > > > > > > PCI-Express > > > > > > Dual > > > > > > Channel 8Gb Fibre Channel HBA. > > > > > > [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: > > > > > > PCIe > > > > > > (5.0GT/s > > > > > > x8) > > > > > > @ > > > > > > 0000:27:00.1 hdma+ host#=6 > > > > > > > > > > > > I marked that still as bad and am continuing, I have 9 > > > > > > builds > > > > > > to > > > > > > go. > > > > > > > > > > > > Thanks > > > > > > Laurence > > > > > > > > > > We had send patches to address issues discovered when the > > > > > PCI > > > > > Affinity > > > > > were > > > > > merged into 4.10 kernel. > > > > > > > > > > Can you please check if your tree has following patches > > > > > (Currently > > > > > they > > > > > are > > > > > included in 4.10.2 stable) > > > > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=699abe249b0e2ca590284424d95490e92ccd6aeb > > > > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > > > > > > > > Also can you apply following patch which addresses issue in > > > > > PCI > > > > > layer > > > > > (This > > > > > should be included in 4.10 stable branch soon ) > > > > > > > > > > https://www.spinics.net/lists/linux-pci/msg58742.html > > > > > > > > > > Thanks, > > > > > Himanshu > > > > > > > > > > > > > Hello Himanshu > > > > > > > > Many thanks for coming back to me. > > > > OK, I will test all of this and let you know. > > > > I dont think I have these and this would explain all this. > > > > > > > > Regards > > > > Laurence > > > > > > > > > > Hello Himanshu > > > This is still failing for me on the HP DL380G8 > > > > > > My Stable tree > > > > > > root@dhcp40-131 ~]# uname -a > > > Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+ > > > > > > $ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > Author: Michael Hernandez <[email protected]> > > > Date: Wed Feb 15 15:37:21 2017 -0800 > > > > > > scsi: qla2xxx: Fix Regression introduced by > > > pci_alloc_irq_vectors_affinity call. > > > > > > commit 67f2db8792f96d8f7521461635d25f9c80245d80 upstream. > > > > > > For target mode, we need to increase minimum vectors value > > > by one > > > to > > > account for ATIO queue. > > > > > > Following stack trace will be seen > > > > > > $ git show 699abe249b0e2ca590284424d95490e92ccd6aeb > > > commit 699abe249b0e2ca590284424d95490e92ccd6aeb > > > Author: Michael Hernandez <[email protected]> > > > Date: Wed Feb 15 15:37:20 2017 -0800 > > > > > > scsi: qla2xxx: Fix response queue count for Target mode. > > > > > > commit d0d2c68b759bbf678e078fd0c71b5fde65a9392c upstream. > > > > > > Target mode initialization was not calculating response > > > queue > > > values > > > correctly resulting into one less MSI-X vector. > > > > > > $ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > Author: Michael Hernandez <[email protected]> > > > Date: Wed Feb 15 15:37:19 2017 -0800 > > > > > > scsi: qla2xxx: Cleaned up queue configuration code. > > > > > > commit f54f2cb540b53d55a81d620e816810d59be5cb1b upstream. > > > > > > This patch cleaned up queue configuration code, such that > > > once > > > initialized, we should not touch msix_count value. This > > > will > > > prevent > > > incorrect numbers of MSI-X vectors requested while > > > performing > > > target > > > mode configuration. > > > > > > > > > Tested, and fails > > > > > > root@dhcp40-131 ~]# dmesg | grep qla > > > [ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > Channel > > > HBA > > > Driver: 8.07.00.38-k. > > > [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > enabled. > > > [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 > > > irq > > > 106 > > > iobase > > > 0xffffb3e4589a1000. > > > [ 9.788259] scsi host4: qla2xxx > > > [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with error > > > -22 > > > [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > enabled. > > > [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 > > > irq > > > 106 > > > iobase > > > 0xffffb3e4589a9000. > > > [ 10.030273] scsi host4: qla2xxx > > > [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with error > > > -22 > > > > > > > > > Now > > > Patched > > > https://www.spinics.net/lists/linux-pci/msg58742.html > > > > > > [loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci > > > patching file drivers/pci/msi.c > > > Hunk #1 succeeded at 1072 (offset -19 lines). > > > Hunk #2 succeeded at 1120 (offset -27 lines). > > > Hunk #3 succeeded at 1200 (offset -27 lines). > > > > > > Rebuild kernel > > > Reboot and test > > > > > > Still fails > > > > > > [ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > Channel > > > HBA > > > Driver: 8.07.00.38-k. > > > [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > enabled. > > > [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 > > > irq > > > 101 > > > iobase > > > 0xffffa3ff58991000. > > > [ 8.670118] scsi host4: qla2xxx > > > [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with error > > > -22 > > > [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > enabled. > > > [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 > > > irq > > > 101 > > > iobase > > > 0xffffa3ff589a1000. > > > [ 8.912089] scsi host4: qla2xxx > > > [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with error > > > -22 > > > > > > Went to test on another system , HP Microserver G8 > > > Same HBA > > > Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to > > > PCI > > > Express > > > HBA (rev 02) > > > > > > Works here but complains about the vector count support > > > > > > [ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > Channel > > > HBA > > > Driver: 8.07.00.38-k. > > > [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector > > > count: 32. > > > [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an ISP2532 > > > irq 16 > > > iobase > > > 0xffffbd9100c99000. > > > [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed to > > > enable > > > support > > > with 32 vectors, using 4 vectors. > > > [ 8.759660] scsi host6: qla2xxx > > > [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A - > > > HP > > > 8Gb > > > Dual > > > Channel PCI-e 2.0 FC HBA. > > > [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe > > > (5.0GT/s > > > x8) @ > > > 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5). > > > [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector > > > count: 32. > > > [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an ISP2532 > > > irq 17 > > > iobase > > > 0xffffbd9100cb9000. > > > [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed to > > > enable > > > support > > > with 32 vectors, using 4 vectors. > > > [ 8.999658] scsi host7: qla2xxx > > > [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A - > > > HP > > > 8Gb > > > Dual > > > Channel PCI-e 2.0 FC HBA. > > > [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe > > > (5.0GT/s > > > x8) @ > > > 0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5). > > > > > > So something about the DL380 here needs to be figured out as > > > well as > > > the > > > MSI-X messages above. > > > > > > Earlier kernels up to 4.9.13 are all fine on both servers > > > > > > Thanks > > > Laurence > > > > > > > I meant to add > > > > With the G8 Microserver links seem to come up with only the 4 > > MSI-X > > vectors but I get this > > so it not fully healthy > > > > [ 373.823667] Rounding down aligned max_sectors from 65535 to > > 65528 > > [ 399.994495] qla2xxx [0000:07:00.0]-00af:6: Performing ISP > > error > > recovery - ha=ffffa0e934210000. > > [ 400.728968] qla2xxx [0000:07:00.0]-500a:6: LOOP UP detected (8 > > Gbps). > > [ 401.473405] qla2xxx [0000:07:00.0]-5030:6: Error entry - > > invalid > > handle/queue (1c01). ***** Note > > [ 426.966519] qla2xxx [0000:07:00.1]-00af:7: Performing ISP > > error > > recovery - ha=ffffa0e934e30000. > > [ 427.697601] qla2xxx [0000:07:00.1]-500a:7: LOOP UP detected (8 > > Gbps). > > [ 428.473948] qla2xxx [0000:07:00.1]-5030:7: Error entry - > > invalid > > handle/queue (1c01). ***** Note > > > > I have tested the ISP2532 adapter on my DL380p G8 server and did not > > run > > into > > issue described. Can you share complete log with me. > > > > [ 615.007032] qla2xxx [0000:27:00.0]-4800:7: DPC handler sleeping. > > [ 615.124217] qla2xxx [0000:27:00.0]-00fb:7: QLogic QLE2562 - > > PCI-Express > > Dual Channel 8Gb Fibre Channel HBA. > > [ 615.124231] qla2xxx [0000:27:00.0]-00fc:7: ISP2532: PCIe (5.0GT/s > > x8) @ > > 0000:27:00.0 hdma+ host#=7 fw=8.06.02 (90d5). > > [ 615.196043] qla2xxx [0000:27:00.0]-580e:7: Asynchronous P2P MODE > > received. > > [ 615.235284] qla2xxx [0000:27:00.0]-480b:7: Reset marker scheduled. > > [ 615.235288] qla2xxx [0000:27:00.0]-480c:7: Reset marker end. > > [ 615.235291] qla2xxx [0000:27:00.0]-480f:7: Loop resync scheduled. > > [ 615.237903] qla2xxx [0000:27:00.0]-500a:7: LOOP UP detected (8 > > Gbps). > > [ 615.237945] qla2xxx [0000:27:00.0]-5812:7: Port database changed > > ffff > > 0006 > > 0000. > > [ 615.237948] qla2xxx [0000:27:00.0]-107ff:7: Mark all dev lost > > [ 615.760115] qla2xxx [0000:27:00.0]-8837:7: F/W Ready - OK. > > [ 615.760177] qla2xxx [0000:27:00.0]-883a:7: fw_state=3 (3, 0, 1e, > > 800 0) > > curr time=100013490. > > [ 615.760285] qla2xxx [0000:27:00.0]-280e:7: HBA in F P2P topology. > > [ 615.760304] qla2xxx [0000:27:00.0]-2814:7: Configure loop -- dpc > > flags = > > 0x152e0. > > [ 615.763509] qla2xxx [0000:27:00.0]-287d:7: NodeName = > > 20000024ff460a5c. > > [ 615.763514] qla2xxx [0000:27:00.0]-28a5:7: Manufacturer = QLogic > > Corporation. > > [ 615.763519] qla2xxx [0000:27:00.0]-28a6:7: Serial no. = > > BFD1230D31975. > > [ 615.763521] qla2xxx [0000:27:00.0]-28a7:7: Model Name = QLE2562. > > [ 615.763537] qla2xxx [0000:27:00.0]-28a8:7: Model Desc = > > PCI-Express Dual > > Channel 8Gb Fibre Channel HBA. > > [ 615.763540] qla2xxx [0000:27:00.0]-28a9:7: Hardware ver = > > PX2810403-01 > > F. > > [ 615.763543] qla2xxx [0000:27:00.0]-28aa:7: Driver ver = > > 8.07.00.38-k-debug. > > [ 615.763547] qla2xxx [0000:27:00.0]-28ab:7: Optrom version = 46.51. > > [ 615.763551] qla2xxx [0000:27:00.0]-28ac:7: Firmware vers = 8.06.02 > > (90d5). > > [ 615.763555] qla2xxx [0000:27:00.0]-28ae:7: OS Name and Version = > > Linux > > 4.11.0-rc2+ #13 SMP Mon Mar 13 21:51:13 PDT 2017. > > [ 615.763558] qla2xxx [0000:27:00.0]-28af:7: CT Payload Length = > > 0x80000. > > [ 615.763562] qla2xxx [0000:27:00.0]-28b0:7: Symbolic Name = QLE2562 > > FW:v8.06.02 DVR:v8.07.00.38-k-debug. > > [ 615.763564] qla2xxx [0000:27:00.0]-28b1:7: Vendor Id = 77100000. > > [ 615.763567] qla2xxx [0000:27:00.0]-28b2:7: Port Num = 1000000. > > [ 615.763569] qla2xxx [0000:27:00.0]-28b3:7: Fabric Name = > > 1000000533aa0b87. > > [ 615.763572] qla2xxx [0000:27:00.0]-28b4:7: BIOS Name = BIOS 3.00 > > [ 615.763575] qla2xxx [0000:27:00.0]-28b1:7: Vendor Identifier = > > QLGC. > > [ 615.763577] qla2xxx [0000:27:00.0]-28b5:7: RHBA identifier = > > 21000024ff460a5c. > > [ 615.775398] qla2xxx [0000:27:00.0]-28b9:7: RHBA FDMI V2 exiting > > normally. > > [ 615.775405] qla2xxx [0000:27:00.0]-28ba:7: FC4_TYPES=01 00. > > [ 615.775409] qla2xxx [0000:27:00.0]-28bb:7: Supported Port Speed = > > 1b000000. > > [ 615.775411] qla2xxx [0000:27:00.0]-28bc:7: Current_Speed = > > 10000000. > > [ 615.775414] qla2xxx [0000:27:00.0]-28bc:7: Max_Frame_Size = 80000. > > [ 615.775417] qla2xxx [0000:27:00.0]-28be:7: OS_Device_Name = > > qla2xxx:host7. > > [ 615.775420] qla2xxx [0000:27:00.0]-283d:7: HostName=dut5108. > > [ 615.775423] qla2xxx [0000:27:00.0]-28c0:7: Node Name = > > 20000024ff460a5c. > > [ 615.775425] qla2xxx [0000:27:00.0]-28c1:7: Port Name = > > 21000024ff460a5c. > > [ 615.775429] qla2xxx [0000:27:00.0]-28c2:7: port symbolic name = > > QLE2562 > > FW:v8.06.02 DVR:v8.07.00.38-k-debug > > > > > > Manufacturer: HP > > Product Name: ProLiant DL380p Gen8 > > Version: Not Specified > > Serial Number: USE334CSJL > > UUID: 32333536-3030-5355-4533-333443534A4C > > Wake-up Type: Power Switch > > SKU Number: 653200-B21 > > Family: ProLiant > > > > Thanks, > > Himanshu > > > > > > Hello Himanshu > > I just now cloned from Linus's tree and patched with the PCI fixes as > you > suggested. > > This gets me this kernel and its failing. > > 4.11.0-rc2+ > > [ 9.099570] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA > Driver: 8.07.00.38-k. > [ 9.099704] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. > [ 9.099705] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 101 > iobase > 0xfffface3d8999000. > [ 9.248817] Console: switching to colour frame buffer device 128x48 > [ 9.305051] scsi host4: qla2xxx > [ 9.341336] qla2xxx: probe of 0000:27:00.0 failed with error -22 > [ 9.341475] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. > [ 9.341478] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 101 > iobase > 0xfffface3d89a9000. > [ 9.547011] scsi host4: qla2xxx > [ 9.583160] qla2xxx: probe of 0000:27:00.1 failed with error -22 > > I dont know why yours is not failing here. > Did you manage to test with the kernel I provided you yet. > > My server: > System Information > Manufacturer: HP > Product Name: ProLiant DL380p Gen8 > Version: Not Specified > Serial Number: USE246MH8W > UUID: 32333536-3030-5355-4532-34364D483857 > Wake-up Type: Power Switch > SKU Number: 653200-B21 > Family: ProLiant > > My ROM > Vendor: HP > Version: P70 > Release Date: 07/01/2015 > Address: 0xF0000 > > All prior kernels are working fine so I know the hardware is good. > > This is what I have installed > > 21:00.0 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller > (rev > 58) > 21:00.1 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller > (rev > 58) > 24:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre > Channel > Host Adapter (rev 03) > 24:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre > Channel > Host Adapter (rev 03) > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to > PCI > Express HBA (rev 02) > 27:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to > PCI > Express HBA (rev 02) > > I will attach my boot log in a separate email just to you and Chad > > Thanks > Laurence > Hello Himanshu Please let me have your .config you are using for your kernel. Thanks Laurence
Quoted 3 lines
See attached config file.
Hello Himanshu
So for me
On the same system
Up to 4.9.13 is fine
4.10.0-rc1 sees the BAR 3 not enabled message but still probes
[ 7.410348] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 7.410513] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 7.410516] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 0xffffbcd818991000. [ 7.616333] scsi host5: qla2xxx [ 7.619442] qla2xxx [0000:27:00.0]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.619448] qla2xxx [0000:27:00.0]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=5 fw=8.03.00 (90d5). [ 7.619591] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 7.619593] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 0xffffbcd8189c1000. [ 7.825251] scsi host6: qla2xxx [ 7.828326] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.828331] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.1 hdma+ host#=6 fw=8.03.
4.10.0-rc8 sees the BAR 3 not enabled messages and fails to probe
[ 9.817953] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.038158] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffb464989a1000. [ 10.280796] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.281032] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.281036] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 106 iobase 0xffffb464989a9000. [ 10.482427] scsi host4: qla2xxx [ 10.518773] qla2xxx: probe of 0000:27:00.1 failed with error -22
Same for 4.11, failing to probe
I will keep looking into it and maybe its something on my side as you seem to have the same hardware and clearly yours is working.
I will see if some strange PCI BIOS setting is enabled and have other servers I will try as well.
in qla2x00_iospace_config has to be because !ha->mqiobase ..
..
if (ha->mqiobase) {
ql_dbg_pci(ql_dbg_init, ha->pdev, 0x0018,
"MQIO Base=%p.\n", ha->mqiobase);
/* Read MSIX vector size of the board */
pci_read_config_word(ha->pdev, QLA_PCI_MSIX_CONTROL, &msix);
ha->msix_count = msix + 1;
/* Max queues are bounded by available msix vectors */
/* MB interrupt uses 1 vector */
ha->max_req_queues = ha->msix_count - 1;
ha->max_rsp_queues = ha->max_req_queues;
/* Queue pairs is the max value minus the base queue pair */
ha->max_qpairs = ha->max_rsp_queues - 1;
ql_dbg_pci(ql_dbg_init, ha->pdev, 0x0188,
"Max no of queues pairs: %d.\n", ha->max_qpairs);
ql_log_pci(ql_log_info, ha->pdev, 0x001a,
"MSI-X vector count: %d.\n", ha->msix_count);
} else
ql_log_pci(ql_log_info, ha->pdev, 0x001b,
"BAR 3 not enabled.\n");Thanks
Laurence
----- Original Message -----
Quoted 874 lines
From: "Laurence Oberman" <[email protected]>
To: "Himanshu Madhani" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Tuesday, March 14, 2017 11:49:51 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message -----
Quoted 786 lines
From: "Himanshu Madhani" <[email protected]>
To: "Laurence Oberman" <[email protected]>
Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" <[email protected]>
Sent: Tuesday, March 14, 2017 8:32:25 PM
Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9----- Original Message ----- > From: "Laurence Oberman" <[email protected]> > To: "Himanshu Madhani" <[email protected]> > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > <[email protected]> > Sent: Tuesday, March 14, 2017 8:02:32 PM > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > (ISP2532-based > 8Gb) with BAR 3 error, work fine on 4.9 > > > > ----- Original Message ----- > > From: "Himanshu Madhani" <[email protected]> > > To: "Laurence Oberman" <[email protected]> > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > > <[email protected]> > > Sent: Tuesday, March 14, 2017 5:11:13 PM > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > (ISP2532-based > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > ----- Original Message ----- > > > From: "Laurence Oberman" <[email protected]> > > > To: "Himanshu Madhani" <[email protected]> > > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI List" > > > <[email protected]> > > > Sent: Monday, March 13, 2017 9:06:38 PM > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > (ISP2532-based > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Laurence Oberman" <[email protected]> > > > > To: "Himanshu Madhani" <[email protected]> > > > > Cc: "Chad Dupuis" <[email protected]>, "Linux SCSI > > > > List" > > > > <[email protected]> > > > > Sent: Monday, March 13, 2017 12:54:12 PM > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > > (ISP2532-based > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > From: "Himanshu Madhani" <[email protected]> > > > > > To: "Laurence Oberman" <[email protected]>, "Chad > > > > > Dupuis" > > > > > <[email protected]> > > > > > Cc: "Linux SCSI List" <[email protected]> > > > > > Sent: Monday, March 13, 2017 12:39:03 PM > > > > > Subject: RE: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > (ISP2532-based > > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > Hi Laurence, > > > > > > > > > > > -----Original Message----- > > > > > > From: Laurence Oberman [mailto:[email protected]] > > > > > > Sent: Sunday, March 12, 2017 11:31 AM > > > > > > To: Dupuis, Chad <[email protected]>; Madhani, > > > > > > Himanshu > > > > > > <[email protected]> > > > > > > Cc: Linux SCSI List <[email protected]> > > > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > > (ISP2532-based > > > > > > 8Gb) > > > > > > with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > From: "Laurence Oberman" <[email protected]> > > > > > > > To: "Chad Dupuis" <[email protected]>, "Himanshu > > > > > > > Madhani" > > > > > > > <[email protected]> > > > > > > > Cc: "Linux SCSI List" <[email protected]> > > > > > > > Sent: Sunday, March 12, 2017 7:39:23 AM > > > > > > > Subject: 4.10+ qla2xxx driver wont load for qla2xxx > > > > > > > (ISP2532-based > > > > > > > 8Gb) with BAR 3 error, work fine on 4.9 > > > > > > > > > > > > > > Chad, Himanshu > > > > > > > > > > > > > > Before I bisect or go chase changes, wanted to reach > > > > > > > out > > > > > > > because the > > > > > > > driver seems to be the same version. > > > > > > > Perhaps this is a PCIE change in the kernel for 4.10 > > > > > > > affecting > > > > > > > the > > > > > > > load. > > > > > > > Its the same targetLIO server I have been using for a > > > > > > > long > > > > > > > time > > > > > > > with > > > > > > > 4.9 > > > > > > > > > > > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb > > > > > > > Fibre > > > > > > > Channel > > > > > > > to > > > > > > > PCI Express HBA (rev 02) > > > > > > > > > > > > > > With 4.9 I have no issues loading the driver for my > > > > > > > targetLIO > > > > > > > server. > > > > > > > (DL380G8) > > > > > > > > > > > > > > # modinfo qla2xxx | more > > > > > > > filename: > > > > > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > > > firmware: ql2500_fw.bin > > > > > > > version: 8.07.00.38-k > > > > > > > license: GPL > > > > > > > description: QLogic Fibre Channel HBA Driver > > > > > > > author: QLogic Corporation > > > > > > > srcversion: 94A8431A85BFF854B97B02D > > > > > > > > > > > > > > [ 8.906351] qla2xxx [0000:00:00.0]-0005: : QLogic > > > > > > > Fibre > > > > > > > Channel > > > > > > > HBA > > > > > > > Driver: 8.07.00.38-k. > > > > > > > [ 10.014052] qla2xxx [0000:27:00.0]-001d: : Found > > > > > > > an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 106 > > > > > > > iobase > > > > > > > 0xffffadce989a1000. > > > > > > > [ 10.455108] scsi host4: qla2xxx > > > > > > > [ 10.460206] qla2xxx [0000:27:00.0]-00fb:4: QLogic > > > > > > > QLE2562 > > > > > > > - > > > > > > > PCI-Express > > > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > > > [ 10.460215] qla2xxx [0000:27:00.0]-00fc:4: > > > > > > > ISP2532: > > > > > > > PCIe > > > > > > > (5.0GT/s > > > > > > > x8) > > > > > > > @ > > > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > > > > [ 10.460545] qla2xxx [0000:27:00.1]-001d: : Found > > > > > > > an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 110 > > > > > > > iobase > > > > > > > 0xffffadce989a9000. > > > > > > > [ 10.662120] scsi host5: qla2xxx > > > > > > > [ 11.007841] qla2xxx [0000:27:00.1]-00fb:5: QLogic > > > > > > > QLE2562 > > > > > > > - > > > > > > > PCI-Express > > > > > > > Dual Channel 8Gb Fibre Channel HBA. > > > > > > > [ 11.007849] qla2xxx [0000:27:00.1]-00fc:5: > > > > > > > ISP2532: > > > > > > > PCIe > > > > > > > (5.0GT/s > > > > > > > x8) > > > > > > > @ > > > > > > > 0000:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5). > > > > > > > > > > > > > > Rebooting on the same server with 4.10 fails to load > > > > > > > > > > > > > > Linux 4.10.0+ > > > > > > > # modinfo qla2xxx | more > > > > > > > filename: > > > > > > > /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko > > > > > > > firmware: ql2500_fw.bin > > > > > > > version: 8.07.00.38-k > > > > > > > license: GPL > > > > > > > description: QLogic Fibre Channel HBA Driver > > > > > > > author: QLogic Corporation > > > > > > > srcversion: 939E0595E8A3C2E1BE94392 > > > > > > > > > > > > > > [ 8.754040] qla2xxx [0000:00:00.0]-0005: : QLogic > > > > > > > Fibre > > > > > > > Channel > > > > > > > HBA > > > > > > > Driver: 8.07.00.38-k. > > > > > > > [ 9.979523] qla2xxx [0000:27:00.0]-001b: : BAR 3 > > > > > > > not > > > > > > > enabled. > > > > > > > [ 10.201268] qla2xxx [0000:27:00.0]-001d: : Found > > > > > > > an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 110 > > > > > > > iobase > > > > > > > 0xffffacbf189b1000. > > > > > > > [ 10.407865] scsi host5: qla2xxx > > > > > > > [ 10.444281] qla2xxx: probe of 0000:27:00.0 failed > > > > > > > with > > > > > > > error > > > > > > > -22 > > > > > > > [ 10.444519] qla2xxx [0000:27:00.1]-001b: : BAR 3 > > > > > > > not > > > > > > > enabled. > > > > > > > [ 10.444522] qla2xxx [0000:27:00.1]-001d: : Found > > > > > > > an > > > > > > > ISP2532 > > > > > > > irq > > > > > > > 110 > > > > > > > iobase > > > > > > > 0xffffacbf189b9000. > > > > > > > [ 10.645932] scsi host5: qla2xxx > > > > > > > [ 10.682233] qla2xxx: probe of 0000:27:00.1 failed > > > > > > > with > > > > > > > error > > > > > > > -22 > > > > > > > > > > > > > > Thanks > > > > > > > Laurence > > > > > > > > > > > > > > > > > > > I started bisecting this, cannot believe others have > > > > > > not > > > > > > bumped > > > > > > into > > > > > > this > > > > > > on > > > > > > 4.10. > > > > > > This is a generic QLE2562 and firmware is loaded by the > > > > > > driver > > > > > > so > > > > > > wondering > > > > > > why I am seeing this and other are not. > > > > > > There is nothing special with the PCIE bus on this > > > > > > DL380G8. > > > > > > > > > > > > Anyway during the bisect I got to a point where in the > > > > > > 4.10 > > > > > > commits I > > > > > > still > > > > > > saw > > > > > > the "BAR 3" message but the probe worked. > > > > > > > > > > > > [ 7.208237] qla2xxx [0000:00:00.0]-0005: : QLogic > > > > > > Fibre > > > > > > Channel HBA > > > > > > Driver: > > > > > > 8.07.00.38-k. > > > > > > [ 7.208492] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > > > > enabled. > > > > > > > > > > > > see this above but probe did not fail > > > > > > > > > > > > [ 7.208494] qla2xxx [0000:27:00.0]-001d: : Found an > > > > > > ISP2532 > > > > > > irq 97 > > > > > > iobase > > > > > > 0xffffc02f98989000. > > > > > > [ 7.414738] scsi host4: qla2xxx > > > > > > > > > > > > [ 7.419267] qla2xxx [0000:27:00.0]-00fb:4: QLogic > > > > > > QLE2562 - > > > > > > PCI-Express > > > > > > Dual > > > > > > Channel 8Gb Fibre Channel HBA. > > > > > > [ 7.419278] qla2xxx [0000:27:00.0]-00fc:4: ISP2532: > > > > > > PCIe > > > > > > (5.0GT/s > > > > > > x8) > > > > > > @ > > > > > > 0000:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5). > > > > > > [ 7.419698] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > > > > enabled. > > > > > > [ 7.419701] qla2xxx [0000:27:00.1]-001d: : Found an > > > > > > ISP2532 > > > > > > irq 100 > > > > > > iobase > > > > > > 0xffffc02f989b1000. > > > > > > [ 7.625691] scsi host6: qla2xxx > > > > > > [ 7.629218] qla2xxx [0000:27:00.1]-00fb:6: QLogic > > > > > > QLE2562 - > > > > > > PCI-Express > > > > > > Dual > > > > > > Channel 8Gb Fibre Channel HBA. > > > > > > [ 7.629222] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: > > > > > > PCIe > > > > > > (5.0GT/s > > > > > > x8) > > > > > > @ > > > > > > 0000:27:00.1 hdma+ host#=6 > > > > > > > > > > > > I marked that still as bad and am continuing, I have 9 > > > > > > builds > > > > > > to > > > > > > go. > > > > > > > > > > > > Thanks > > > > > > Laurence > > > > > > > > > > We had send patches to address issues discovered when > > > > > the > > > > > PCI > > > > > Affinity > > > > > were > > > > > merged into 4.10 kernel. > > > > > > > > > > Can you please check if your tree has following patches > > > > > (Currently > > > > > they > > > > > are > > > > > included in 4.10.2 stable) > > > > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=699abe249b0e2ca590284424d95490e92ccd6aeb > > > > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?id=cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > > > > > > > > Also can you apply following patch which addresses issue > > > > > in > > > > > PCI > > > > > layer > > > > > (This > > > > > should be included in 4.10 stable branch soon ) > > > > > > > > > > https://www.spinics.net/lists/linux-pci/msg58742.html > > > > > > > > > > Thanks, > > > > > Himanshu > > > > > > > > > > > > > Hello Himanshu > > > > > > > > Many thanks for coming back to me. > > > > OK, I will test all of this and let you know. > > > > I dont think I have these and this would explain all this. > > > > > > > > Regards > > > > Laurence > > > > > > > > > > Hello Himanshu > > > This is still failing for me on the HP DL380G8 > > > > > > My Stable tree > > > > > > root@dhcp40-131 ~]# uname -a > > > Linux xxxxxx.desklab.eng.bos.redhat.com 4.11.0-rc2+ > > > > > > $ git show 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > commit 154f7a1523b717cad67e0b3d2ed4c660fd1b2e7c > > > Author: Michael Hernandez <[email protected]> > > > Date: Wed Feb 15 15:37:21 2017 -0800 > > > > > > scsi: qla2xxx: Fix Regression introduced by > > > pci_alloc_irq_vectors_affinity call. > > > > > > commit 67f2db8792f96d8f7521461635d25f9c80245d80 upstream. > > > > > > For target mode, we need to increase minimum vectors > > > value > > > by one > > > to > > > account for ATIO queue. > > > > > > Following stack trace will be seen > > > > > > $ git show 699abe249b0e2ca590284424d95490e92ccd6aeb > > > commit 699abe249b0e2ca590284424d95490e92ccd6aeb > > > Author: Michael Hernandez <[email protected]> > > > Date: Wed Feb 15 15:37:20 2017 -0800 > > > > > > scsi: qla2xxx: Fix response queue count for Target mode. > > > > > > commit d0d2c68b759bbf678e078fd0c71b5fde65a9392c upstream. > > > > > > Target mode initialization was not calculating response > > > queue > > > values > > > correctly resulting into one less MSI-X vector. > > > > > > $ git show cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > commit cf9d9110806493e2ea2db7d00a1a991401dfb9d5 > > > Author: Michael Hernandez <[email protected]> > > > Date: Wed Feb 15 15:37:19 2017 -0800 > > > > > > scsi: qla2xxx: Cleaned up queue configuration code. > > > > > > commit f54f2cb540b53d55a81d620e816810d59be5cb1b upstream. > > > > > > This patch cleaned up queue configuration code, such that > > > once > > > initialized, we should not touch msix_count value. This > > > will > > > prevent > > > incorrect numbers of MSI-X vectors requested while > > > performing > > > target > > > mode configuration. > > > > > > > > > Tested, and fails > > > > > > root@dhcp40-131 ~]# dmesg | grep qla > > > [ 8.007786] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > Channel > > > HBA > > > Driver: 8.07.00.38-k. > > > [ 9.583949] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > enabled. > > > [ 9.583951] qla2xxx [0000:27:00.0]-001d: : Found an > > > ISP2532 > > > irq > > > 106 > > > iobase > > > 0xffffb3e4589a1000. > > > [ 9.788259] scsi host4: qla2xxx > > > [ 9.824578] qla2xxx: probe of 0000:27:00.0 failed with > > > error > > > -22 > > > [ 9.824824] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > enabled. > > > [ 9.824828] qla2xxx [0000:27:00.1]-001d: : Found an > > > ISP2532 > > > irq > > > 106 > > > iobase > > > 0xffffb3e4589a9000. > > > [ 10.030273] scsi host4: qla2xxx > > > [ 10.066602] qla2xxx: probe of 0000:27:00.1 failed with > > > error > > > -22 > > > > > > > > > Now > > > Patched > > > https://www.spinics.net/lists/linux-pci/msg58742.html > > > > > > [loberman@dhcp40-131 linux-stable]$ patch -p1 < ../patch_pci > > > patching file drivers/pci/msi.c > > > Hunk #1 succeeded at 1072 (offset -19 lines). > > > Hunk #2 succeeded at 1120 (offset -27 lines). > > > Hunk #3 succeeded at 1200 (offset -27 lines). > > > > > > Rebuild kernel > > > Reboot and test > > > > > > Still fails > > > > > > [ 8.464491] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > Channel > > > HBA > > > Driver: 8.07.00.38-k. > > > [ 8.464672] qla2xxx [0000:27:00.0]-001b: : BAR 3 not > > > enabled. > > > [ 8.464675] qla2xxx [0000:27:00.0]-001d: : Found an > > > ISP2532 > > > irq > > > 101 > > > iobase > > > 0xffffa3ff58991000. > > > [ 8.670118] scsi host4: qla2xxx > > > [ 8.706325] qla2xxx: probe of 0000:27:00.0 failed with > > > error > > > -22 > > > [ 8.706451] qla2xxx [0000:27:00.1]-001b: : BAR 3 not > > > enabled. > > > [ 8.706453] qla2xxx [0000:27:00.1]-001d: : Found an > > > ISP2532 > > > irq > > > 101 > > > iobase > > > 0xffffa3ff589a1000. > > > [ 8.912089] scsi host4: qla2xxx > > > [ 8.948324] qla2xxx: probe of 0000:27:00.1 failed with > > > error > > > -22 > > > > > > Went to test on another system , HP Microserver G8 > > > Same HBA > > > Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel > > > to > > > PCI > > > Express > > > HBA (rev 02) > > > > > > Works here but complains about the vector count support > > > > > > [ 8.056438] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre > > > Channel > > > HBA > > > Driver: 8.07.00.38-k. > > > [ 8.097427] qla2xxx [0000:07:00.0]-001a: : MSI-X vector > > > count: 32. > > > [ 8.127130] qla2xxx [0000:07:00.0]-001d: : Found an > > > ISP2532 > > > irq 16 > > > iobase > > > 0xffffbd9100c99000. > > > [ 8.221834] qla2xxx [0000:07:00.0]-00c6:6: MSI-X: Failed > > > to > > > enable > > > support > > > with 32 vectors, using 4 vectors. > > > [ 8.759660] scsi host6: qla2xxx > > > [ 8.760085] qla2xxx [0000:07:00.0]-00fb:6: QLogic HPAJ764A > > > - > > > HP > > > 8Gb > > > Dual > > > Channel PCI-e 2.0 FC HBA. > > > [ 8.760092] qla2xxx [0000:07:00.0]-00fc:6: ISP2532: PCIe > > > (5.0GT/s > > > x8) @ > > > 0000:07:00.0 hdma+ host#=6 fw=8.03.00 (90d5). > > > [ 8.760197] qla2xxx [0000:07:00.1]-001a: : MSI-X vector > > > count: 32. > > > [ 8.760199] qla2xxx [0000:07:00.1]-001d: : Found an > > > ISP2532 > > > irq 17 > > > iobase > > > 0xffffbd9100cb9000. > > > [ 8.760428] qla2xxx [0000:07:00.1]-00c6:7: MSI-X: Failed > > > to > > > enable > > > support > > > with 32 vectors, using 4 vectors. > > > [ 8.999658] scsi host7: qla2xxx > > > [ 9.000124] qla2xxx [0000:07:00.1]-00fb:7: QLogic HPAJ764A > > > - > > > HP > > > 8Gb > > > Dual > > > Channel PCI-e 2.0 FC HBA. > > > [ 9.000130] qla2xxx [0000:07:00.1]-00fc:7: ISP2532: PCIe > > > (5.0GT/s > > > x8) @ > > > 0000:07:00.1 hdma+ host#=7 fw=8.03.00 (90d5). > > > > > > So something about the DL380 here needs to be figured out as > > > well as > > > the > > > MSI-X messages above. > > > > > > Earlier kernels up to 4.9.13 are all fine on both servers > > > > > > Thanks > > > Laurence > > > > > > > I meant to add > > > > With the G8 Microserver links seem to come up with only the 4 > > MSI-X > > vectors but I get this > > so it not fully healthy > > > > [ 373.823667] Rounding down aligned max_sectors from 65535 to > > 65528 > > [ 399.994495] qla2xxx [0000:07:00.0]-00af:6: Performing ISP > > error > > recovery - ha=ffffa0e934210000. > > [ 400.728968] qla2xxx [0000:07:00.0]-500a:6: LOOP UP detected > > (8 > > Gbps). > > [ 401.473405] qla2xxx [0000:07:00.0]-5030:6: Error entry - > > invalid > > handle/queue (1c01). ***** Note > > [ 426.966519] qla2xxx [0000:07:00.1]-00af:7: Performing ISP > > error > > recovery - ha=ffffa0e934e30000. > > [ 427.697601] qla2xxx [0000:07:00.1]-500a:7: LOOP UP detected > > (8 > > Gbps). > > [ 428.473948] qla2xxx [0000:07:00.1]-5030:7: Error entry - > > invalid > > handle/queue (1c01). ***** Note > > > > I have tested the ISP2532 adapter on my DL380p G8 server and did > > not > > run > > into > > issue described. Can you share complete log with me. > > > > [ 615.007032] qla2xxx [0000:27:00.0]-4800:7: DPC handler sleeping. > > [ 615.124217] qla2xxx [0000:27:00.0]-00fb:7: QLogic QLE2562 - > > PCI-Express > > Dual Channel 8Gb Fibre Channel HBA. > > [ 615.124231] qla2xxx [0000:27:00.0]-00fc:7: ISP2532: PCIe > > (5.0GT/s > > x8) @ > > 0000:27:00.0 hdma+ host#=7 fw=8.06.02 (90d5). > > [ 615.196043] qla2xxx [0000:27:00.0]-580e:7: Asynchronous P2P MODE > > received. > > [ 615.235284] qla2xxx [0000:27:00.0]-480b:7: Reset marker > > scheduled. > > [ 615.235288] qla2xxx [0000:27:00.0]-480c:7: Reset marker end. > > [ 615.235291] qla2xxx [0000:27:00.0]-480f:7: Loop resync > > scheduled. > > [ 615.237903] qla2xxx [0000:27:00.0]-500a:7: LOOP UP detected (8 > > Gbps). > > [ 615.237945] qla2xxx [0000:27:00.0]-5812:7: Port database changed > > ffff > > 0006 > > 0000. > > [ 615.237948] qla2xxx [0000:27:00.0]-107ff:7: Mark all dev lost > > [ 615.760115] qla2xxx [0000:27:00.0]-8837:7: F/W Ready - OK. > > [ 615.760177] qla2xxx [0000:27:00.0]-883a:7: fw_state=3 (3, 0, 1e, > > 800 0) > > curr time=100013490. > > [ 615.760285] qla2xxx [0000:27:00.0]-280e:7: HBA in F P2P > > topology. > > [ 615.760304] qla2xxx [0000:27:00.0]-2814:7: Configure loop -- dpc > > flags = > > 0x152e0. > > [ 615.763509] qla2xxx [0000:27:00.0]-287d:7: NodeName = > > 20000024ff460a5c. > > [ 615.763514] qla2xxx [0000:27:00.0]-28a5:7: Manufacturer = QLogic > > Corporation. > > [ 615.763519] qla2xxx [0000:27:00.0]-28a6:7: Serial no. = > > BFD1230D31975. > > [ 615.763521] qla2xxx [0000:27:00.0]-28a7:7: Model Name = QLE2562. > > [ 615.763537] qla2xxx [0000:27:00.0]-28a8:7: Model Desc = > > PCI-Express Dual > > Channel 8Gb Fibre Channel HBA. > > [ 615.763540] qla2xxx [0000:27:00.0]-28a9:7: Hardware ver = > > PX2810403-01 > > F. > > [ 615.763543] qla2xxx [0000:27:00.0]-28aa:7: Driver ver = > > 8.07.00.38-k-debug. > > [ 615.763547] qla2xxx [0000:27:00.0]-28ab:7: Optrom version = > > 46.51. > > [ 615.763551] qla2xxx [0000:27:00.0]-28ac:7: Firmware vers = > > 8.06.02 > > (90d5). > > [ 615.763555] qla2xxx [0000:27:00.0]-28ae:7: OS Name and Version = > > Linux > > 4.11.0-rc2+ #13 SMP Mon Mar 13 21:51:13 PDT 2017. > > [ 615.763558] qla2xxx [0000:27:00.0]-28af:7: CT Payload Length = > > 0x80000. > > [ 615.763562] qla2xxx [0000:27:00.0]-28b0:7: Symbolic Name = > > QLE2562 > > FW:v8.06.02 DVR:v8.07.00.38-k-debug. > > [ 615.763564] qla2xxx [0000:27:00.0]-28b1:7: Vendor Id = 77100000. > > [ 615.763567] qla2xxx [0000:27:00.0]-28b2:7: Port Num = 1000000. > > [ 615.763569] qla2xxx [0000:27:00.0]-28b3:7: Fabric Name = > > 1000000533aa0b87. > > [ 615.763572] qla2xxx [0000:27:00.0]-28b4:7: BIOS Name = BIOS 3.00 > > [ 615.763575] qla2xxx [0000:27:00.0]-28b1:7: Vendor Identifier = > > QLGC. > > [ 615.763577] qla2xxx [0000:27:00.0]-28b5:7: RHBA identifier = > > 21000024ff460a5c. > > [ 615.775398] qla2xxx [0000:27:00.0]-28b9:7: RHBA FDMI V2 exiting > > normally. > > [ 615.775405] qla2xxx [0000:27:00.0]-28ba:7: FC4_TYPES=01 00. > > [ 615.775409] qla2xxx [0000:27:00.0]-28bb:7: Supported Port Speed > > = > > 1b000000. > > [ 615.775411] qla2xxx [0000:27:00.0]-28bc:7: Current_Speed = > > 10000000. > > [ 615.775414] qla2xxx [0000:27:00.0]-28bc:7: Max_Frame_Size = > > 80000. > > [ 615.775417] qla2xxx [0000:27:00.0]-28be:7: OS_Device_Name = > > qla2xxx:host7. > > [ 615.775420] qla2xxx [0000:27:00.0]-283d:7: HostName=dut5108. > > [ 615.775423] qla2xxx [0000:27:00.0]-28c0:7: Node Name = > > 20000024ff460a5c. > > [ 615.775425] qla2xxx [0000:27:00.0]-28c1:7: Port Name = > > 21000024ff460a5c. > > [ 615.775429] qla2xxx [0000:27:00.0]-28c2:7: port symbolic name = > > QLE2562 > > FW:v8.06.02 DVR:v8.07.00.38-k-debug > > > > > > Manufacturer: HP > > Product Name: ProLiant DL380p Gen8 > > Version: Not Specified > > Serial Number: USE334CSJL > > UUID: 32333536-3030-5355-4533-333443534A4C > > Wake-up Type: Power Switch > > SKU Number: 653200-B21 > > Family: ProLiant > > > > Thanks, > > Himanshu > > > > > > Hello Himanshu > > I just now cloned from Linus's tree and patched with the PCI fixes as > you > suggested. > > This gets me this kernel and its failing. > > 4.11.0-rc2+ > > [ 9.099570] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel > HBA > Driver: 8.07.00.38-k. > [ 9.099704] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. > [ 9.099705] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq > 101 > iobase > 0xfffface3d8999000. > [ 9.248817] Console: switching to colour frame buffer device > 128x48 > [ 9.305051] scsi host4: qla2xxx > [ 9.341336] qla2xxx: probe of 0000:27:00.0 failed with error -22 > [ 9.341475] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. > [ 9.341478] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq > 101 > iobase > 0xfffface3d89a9000. > [ 9.547011] scsi host4: qla2xxx > [ 9.583160] qla2xxx: probe of 0000:27:00.1 failed with error -22 > > I dont know why yours is not failing here. > Did you manage to test with the kernel I provided you yet. > > My server: > System Information > Manufacturer: HP > Product Name: ProLiant DL380p Gen8 > Version: Not Specified > Serial Number: USE246MH8W > UUID: 32333536-3030-5355-4532-34364D483857 > Wake-up Type: Power Switch > SKU Number: 653200-B21 > Family: ProLiant > > My ROM > Vendor: HP > Version: P70 > Release Date: 07/01/2015 > Address: 0xF0000 > > All prior kernels are working fine so I know the hardware is good. > > This is what I have installed > > 21:00.0 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller > (rev > 58) > 21:00.1 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller > (rev > 58) > 24:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre > Channel > Host Adapter (rev 03) > 24:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre > Channel > Host Adapter (rev 03) > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel > to > PCI > Express HBA (rev 02) > 27:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel > to > PCI > Express HBA (rev 02) > > I will attach my boot log in a separate email just to you and Chad > > Thanks > Laurence > Hello Himanshu Please let me have your .config you are using for your kernel. Thanks LaurenceQuoted 3 lines
See attached config file.
Hello Himanshu
So for me
On the same system
Up to 4.9.13 is fine
4.10.0-rc1 sees the BAR 3 not enabled message but still probes
[ 7.410348] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 7.410513] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 7.410516] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 0xffffbcd818991000. [ 7.616333] scsi host5: qla2xxx [ 7.619442] qla2xxx [0000:27:00.0]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.619448] qla2xxx [0000:27:00.0]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @ 0000:27:00.0 hdma+ host#=5 fw=8.03.00 (90d5). [ 7.619591] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 7.619593] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 0xffffbcd8189c1000. [ 7.825251] scsi host6: qla2xxx [ 7.828326] qla2xxx [0000:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 7.828331] qla2xxx [0000:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @0000:27:00.1 hdma+ host#=6 fw=8.03.
4.10.0-rc8 sees the BAR 3 not enabled messages and fails to probe
[ 9.817953] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 10.038158] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 0xffffb464989a1000. [ 10.280796] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 10.281032] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 10.281036] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 106 iobase 0xffffb464989a9000. [ 10.482427] scsi host4: qla2xxx [ 10.518773] qla2xxx: probe of 0000:27:00.1 failed with error -22Same for 4.11, failing to probe
I will keep looking into it and maybe its something on my side as you seem to have the same hardware
and clearly yours is working.I will see if some strange PCI BIOS setting is enabled and have other servers I will try as well.
in
qla2x00_iospace_confighas to be because !ha->mqiobase ..
..if (ha->mqiobase) { ql_dbg_pci(ql_dbg_init, ha->pdev, 0x0018, "MQIO Base=%p.\n", ha->mqiobase); /* Read MSIX vector size of the board */ pci_read_config_word(ha->pdev, QLA_PCI_MSIX_CONTROL, &msix); ha->msix_count = msix + 1; /* Max queues are bounded by available msix vectors */ /* MB interrupt uses 1 vector */ ha->max_req_queues = ha->msix_count - 1; ha->max_rsp_queues = ha->max_req_queues; /* Queue pairs is the max value minus the base queue pair */ ha->max_qpairs = ha->max_rsp_queues - 1; ql_dbg_pci(ql_dbg_init, ha->pdev, 0x0188, "Max no of queues pairs: %d.\n", ha->max_qpairs); ql_log_pci(ql_log_info, ha->pdev, 0x001a, "MSI-X vector count: %d.\n", ha->msix_count); } else ql_log_pci(ql_log_info, ha->pdev, 0x001b, "BAR 3 not enabled.\n");
Quoted 4 lines
Thanks
Laurence
Hello Himanshu
This issue seems to be related to the actual card in my case.
I moved another QLogic QLE2562 into the same server and put it in another PCIE slot
This new one works :)
0a:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02) 0a:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
[ 27.302805] qla2xxx [0000:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 8.07.00.38-k. [ 27.342988] qla2xxx [0000:0a:00.0]-001a: : MSI-X vector count: 32. [ 27.342990] qla2xxx [0000:0a:00.0]-001d: : Found an ISP2532 irq 16 iobase 0xffffa46ed8a39000. [ 28.106369] scsi host4: qla2xxx [ 28.202267] qla2xxx [0000:0a:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 28.249343] qla2xxx [0000:0a:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 0000:0a:00.0 hdma+ host#=4 fw=8.06.02 (90d5). [ 28.300604] qla2xxx [0000:0a:00.1]-001a: : MSI-X vector count: 32. [ 28.300607] qla2xxx [0000:0a:00.1]-001d: : Found an ISP2532 irq 17 iobase 0xffffa46ed8c41000. [ 28.513178] scsi host5: qla2xxx
The other one still fails
27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02) 27:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
[ 28.603987] qla2xxx [0000:0a:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 28.651661] qla2xxx [0000:0a:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @ 0000:0a:00.1 hdma+ host#=5 fw=8.06.02 (90d5). [ 28.704216] qla2xxx [0000:27:00.0]-001b: : BAR 3 not enabled. [ 28.731964] qla2xxx [0000:27:00.0]-001d: : Found an ISP2532 irq 191 iobase 0xffffa46ed8a89000. [ 28.979223] scsi host6: qla2xxx [ 29.030409] qla2xxx: probe of 0000:27:00.0 failed with error -22 [ 29.059306] qla2xxx [0000:27:00.1]-001b: : BAR 3 not enabled. [ 29.087118] qla2xxx [0000:27:00.1]-001d: : Found an ISP2532 irq 191 iobase 0xffffa46ed8e59000. [ 29.361249] scsi host6: qla2xxx [ 29.412445] qla2xxx: probe of 0000:27:00.1 failed with error -22
I must be honest this had me going for days because 4.9.13 works fine :)
Anyway, next week I will see if I can get hold of you to chat about it in Boston because its a stange one and this could affect others ifd we dont figure out what is different between these cards.
Thanks for your patience and assistance
Laurence