PCI: Limit VPD length for Broadcom 5708S

PATCH In review Eric Dumazet 4 participants 11 messages Jul 31, 2008 – Nov 3, 2008 2 messages not in the archive

Subsystems (3)

Networking Network drivers PCI

lore

Share & tools

lore

Eric Dumazet

author

2008-07-31 18:27 UTC #0 lore raw

PATCH 1/1 PCI: Limit VPD length for Broadcom 5708S

While trying to upgrade kernel from 2.6.25 to 2.6.26 on a ProLiant BL460c G1, I faced this problem on integrated Broadcom 5708S NIC when trying to UP them.

bnx2: fw sync timeout, reset code = 1030003

I noticed that commit 99cb233d60cbe644203f19938c729ea2bb004d70 (PCI: Limit VPD read/write lengths for Broadcom 5706, 5708, 5709 rev.A) was probably a good candidate to solve the problem.

Apparently, it added PCI quirks for a list of Broadcom NICs, but quirk_brcm_570x_limit_vpd() ignores part of them.

I found the following patch useful to restore correct operation on my machine.

Some information about this nic :

03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12)
Subsystem: Hewlett-Packard Company NC373i Integrated Multifunction Gigabit Server Adapter
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 64 (16000ns min), Cache Line Size 10
Interrupt: pin A routed to IRQ 222
Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M]
[virtual] Expansion ROM at d1200000 [disabled] [size=16K]
Capabilities: [40] PCI-X non-bridge device.
Command: DPERE- ERO- RBC=0 OST=4
Status: Bus=3 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, DC=simple, DMMRBC=0, DMOST=4, DMCRS=2, RSCEM-
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable+
Address: 00000000feeff00c Data: 4179

after applying patch :

# hexdump -C "/sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0/0000:03:00.0/vpd"
00000000  82 2f 00 48 50 20 4e 43  33 37 33 69 20 4d 75 6c  |./.HP NC373i Mul|
00000010  74 69 66 75 6e 63 74 69  6f 6e 20 47 69 67 61 62  |tifunction Gigab|
00000020  69 74 20 53 65 72 76 65  72 20 41 64 61 70 74 65  |it Server Adapte|
00000030  72 00 90 4a 00 50 4e 03  4e 2f 41 45 43 03 4e 2f  |r..J.PN.N/AEC.N/|
00000040  41 53 4e 0a 30 31 32 33  34 35 36 37 38 39 4d 4e  |ASN.0123456789MN|
00000050  04 31 30 33 43 52 56 26  a2 00 00 00 00 00 00 00  |.103CRV&........|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 78  |...............x|
00000080

Patch against current Linus tree, should also be sent for linux-2.6.26.2 inclusion.

I have no idea if 5706S and 5709S also needs this truncation of VPD length

Thank you

[PATCH] PCI: Limit VPD length for Broadcom 5708S

BCM5708S wont work correctly unless VPD length truncated to 128

Signed-off-by Eric Dumazet <[email protected]>

drivers/pci/quirks.c +0 -0

index 0fb3650..59667e5 100644
@@ -1756,9 +1756,12 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_VIA, 0x324e, quirk_via_cx700_pci_parking_c
 */

static void __devinit quirk_brcm_570x_limit_vpd(struct pci_dev *dev) {
- /* Only disable the VPD capability for 5706, 5708, and 5709 rev. A */
+ /*
+ * Only disable the VPD capability for 5706, 5708, 5708S and 5709 rev. A
+ */

	if ((dev->device == PCI_DEVICE_ID_NX2_5706) ||
	    (dev->device == PCI_DEVICE_ID_NX2_5708) ||
+	    (dev->device == PCI_DEVICE_ID_NX2_5708S) ||
	    ((dev->device == PCI_DEVICE_ID_NX2_5709) &&
	     (dev->revision & 0xf0) == 0x0)) {
		if (dev->vpd)

Jesse Barnes

2008-07-31 19:39 UTC #1 lore raw

Quoted 3 lines · Eric Dumazet

While trying to upgrade kernel from 2.6.25 to 2.6.26 on a ProLiant BL460c G1, I faced this problem on integrated Broadcom 5708S NIC when trying to UP them.

Thanks Eric, I applied this to my for-linus branch. I'll bounce the original over to stable@.

Jesse

A message that is not in the archive (sent off-list or not yet received): [email protected]

Jesse Barnes

2008-07-31 19:41 UTC #4 lore raw

Re: [PATCH] PCI: Add limits to VPD read/write lengths for Broadcom 5706S, 5708S

Quoted 5 lines · Benjamin Li

Reading beyond the valid VPD limit will hang the Broadcom 5706S, 5708S. During the check we need to add the additionial 5706S and 5708S PCI ID's when limiting the read/write VPD length.
Signed-off-by Benjamin Li <[email protected]>

I already applied Eric's 5708S patch, can you re-diff against my for-linus branch (in a few minutes after it mirrors out) and resend? And please cc [email protected], since they'll want these crash fixes too.

Thanks,
Jesse

Eric Dumazet

author

2008-10-27 22:27 UTC #5 lore raw

In reply to Eric Dumazet

Quoted 78 lines · Eric Dumazet

While trying to upgrade kernel from 2.6.25 to 2.6.26 on a ProLiant BL460c G1, I faced this problem on integrated Broadcom 5708S NIC when trying to UP them.
bnx2: fw sync timeout, reset code = 1030003
I noticed that commit 99cb233d60cbe644203f19938c729ea2bb004d70 (PCI: Limit VPD read/write lengths for Broadcom 5706, 5708, 5709 rev.A) was probably a good candidate to solve the problem.
Apparently, it added PCI quirks for a list of Broadcom NICs, but quirk_brcm_570x_limit_vpd() ignores part of them.
I found the following patch useful to restore correct operation on my machine.
Some information about this nic :
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12)
Subsystem: Hewlett-Packard Company NC373i Integrated
Multifunction Gigabit Server Adapter
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
       Latency: 64 (16000ns min), Cache Line Size 10
       Interrupt: pin A routed to IRQ 222
       Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M]
       [virtual] Expansion ROM at d1200000 [disabled] [size=16K]
       Capabilities: [40] PCI-X non-bridge device.
               Command: DPERE- ERO- RBC=0 OST=4
               Status: Bus=3 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, 
DC=simple, DMMRBC=0, DMOST=4, DMCRS=2, RSCEM-
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/0
Enable+
               Address: 00000000feeff00c  Data: 4179
after applying patch :
# hexdump -C "/sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0/0000:03:00.0/vpd"
00000000  82 2f 00 48 50 20 4e 43  33 37 33 69 20 4d 75 6c  |./.HP NC373i Mul|
00000010  74 69 66 75 6e 63 74 69  6f 6e 20 47 69 67 61 62  |tifunction Gigab|
00000020  69 74 20 53 65 72 76 65  72 20 41 64 61 70 74 65  |it Server Adapte|
00000030  72 00 90 4a 00 50 4e 03  4e 2f 41 45 43 03 4e 2f  |r..J.PN.N/AEC.N/|
00000040  41 53 4e 0a 30 31 32 33  34 35 36 37 38 39 4d 4e  |ASN.0123456789MN|
00000050  04 31 30 33 43 52 56 26  a2 00 00 00 00 00 00 00  |.103CRV&........|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 78  |...............x|
00000080
Patch against current Linus tree, should also be sent for linux-2.6.26.2 inclusion.
I have no idea if 5706S and 5709S also needs this truncation of VPD length
Thank you
[PATCH] PCI: Limit VPD length for Broadcom 5708S
BCM5708S wont work correctly unless VPD length truncated to 128
Signed-off-by Eric Dumazet <[email protected]>
drivers/pci/quirks.c +0 -0
index 0fb3650..59667e5 100644
@@ -1756,9 +1756,12 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_VIA, 
0x324e, quirk_via_cx700_pci_parking_c
*/
static void __devinit quirk_brcm_570x_limit_vpd(struct pci_dev *dev) {
drivers/pci/quirks.c +2 -1
-    /*  Only disable the VPD capability for 5706, 5708, and 5709 rev. A */
+    /*
+     * Only disable the VPD capability for 5706, 5708, 5708S and 5709 
rev. A

drivers/pci/quirks.c

Quoted 7 lines

drivers/pci/quirks.c +2 -0

+     */
    if ((dev->device == PCI_DEVICE_ID_NX2_5706) ||
        (dev->device == PCI_DEVICE_ID_NX2_5708) ||
+        (dev->device == PCI_DEVICE_ID_NX2_5708S) ||
        ((dev->device == PCI_DEVICE_ID_NX2_5709) &&
         (dev->revision & 0xf0) == 0x0)) {
        if (dev->vpd)

Hi all

Just tried linux-2.6.28-rc2 on same platform.

Same problem again...

Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.8.1 (Oct 7, 2008)
bnx2 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth0: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem f6000000, IRQ 16, node addr 00:1e:0b:ec:d3:dc bnx2 0000:07:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth1: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 16, node addr 00:1e:0b:ec:d3:d2 bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:07:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030006

2.6.27.4 is running fine on this machine

Any ideas ?

Thank you

Eric

Benjamin Li

2008-10-28 03:26 UTC #6 lore raw

Hi Eric,

It looks like the same problem again: we are reading past a valid VPD address.

After some initial debugging, it looks like dev->vpd is NULL when the Broadcom 5706C/5706S/5708C/570S/5709 quirk is called. Because of this dev->vpd->len is never set to the proper length to limit the VPD reads. I will debug further to determine what has changed.

Thanks again for pointing out the problem.

-Ben

Quoted 132 lines · Eric Dumazet

Quoted 93 lines · Eric Dumazet
While trying to upgrade kernel from 2.6.25 to 2.6.26 on a ProLiant BL460c G1,
I faced this problem on integrated Broadcom 5708S NIC when trying to UP them.
bnx2: fw sync timeout, reset code = 1030003
I noticed that commit 99cb233d60cbe644203f19938c729ea2bb004d70 (PCI:
Limit VPD read/write lengths for Broadcom 5706, 5708, 5709 rev.A) was probably a good candidate to solve the problem.
Apparently, it added PCI quirks for a list of Broadcom NICs, but quirk_brcm_570x_limit_vpd() ignores part of them.
I found the following patch useful to restore correct operation on my machine.
Some information about this nic :
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12)
Subsystem: Hewlett-Packard Company NC373i Integrated
Multifunction Gigabit Server Adapter
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
       Latency: 64 (16000ns min), Cache Line Size 10
       Interrupt: pin A routed to IRQ 222
       Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M]
       [virtual] Expansion ROM at d1200000 [disabled] [size=16K]
       Capabilities: [40] PCI-X non-bridge device.
               Command: DPERE- ERO- RBC=0 OST=4
               Status: Bus=3 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, 
DC=simple, DMMRBC=0, DMOST=4, DMCRS=2, RSCEM-
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/0
Enable+
               Address: 00000000feeff00c  Data: 4179
after applying patch :
# hexdump -C 
"/sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0/0000:03:00.0/vpd"
00000000  82 2f 00 48 50 20 4e 43  33 37 33 69 20 4d 75 6c  |./.HP 
NC373i Mul|
00000010  74 69 66 75 6e 63 74 69  6f 6e 20 47 69 67 61 62  |tifunction 
Gigab|
00000020  69 74 20 53 65 72 76 65  72 20 41 64 61 70 74 65  |it Server 
Adapte|
00000030  72 00 90 4a 00 50 4e 03  4e 2f 41 45 43 03 4e 2f  
|r..J.PN.N/AEC.N/|
00000040  41 53 4e 0a 30 31 32 33  34 35 36 37 38 39 4d 4e  
|ASN.0123456789MN|
00000050  04 31 30 33 43 52 56 26  a2 00 00 00 00 00 00 00  
|.103CRV&........|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  
|................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 78  
|...............x|
00000080
Patch against current Linus tree, should also be sent for linux-2.6.26.2 inclusion.
I have no idea if 5706S and 5709S also needs this truncation of VPD length
Thank you
[PATCH] PCI: Limit VPD length for Broadcom 5708S
BCM5708S wont work correctly unless VPD length truncated to 128
Signed-off-by Eric Dumazet <[email protected]>
drivers/pci/quirks.c +0 -0
index 0fb3650..59667e5 100644
@@ -1756,9 +1756,12 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_VIA, 
0x324e, quirk_via_cx700_pci_parking_c
*/
static void __devinit quirk_brcm_570x_limit_vpd(struct pci_dev *dev) {
drivers/pci/quirks.c +2 -1
-    /*  Only disable the VPD capability for 5706, 5708, and 5709 rev. A */
+    /*
+     * Only disable the VPD capability for 5706, 5708, 5708S and 5709 
rev. A
drivers/pci/quirks.c
Quoted 7 lines
drivers/pci/quirks.c +2 -0
+     */
    if ((dev->device == PCI_DEVICE_ID_NX2_5706) ||
        (dev->device == PCI_DEVICE_ID_NX2_5708) ||
+        (dev->device == PCI_DEVICE_ID_NX2_5708S) ||
        ((dev->device == PCI_DEVICE_ID_NX2_5709) &&
         (dev->revision & 0xf0) == 0x0)) {
        if (dev->vpd)
Hi all
Just tried linux-2.6.28-rc2 on same platform.
Same problem again...
Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.8.1 (Oct 7, 2008)
bnx2 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth0: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem f6000000, IRQ 16, node addr 00:1e:0b:ec:d3:dc bnx2 0000:07:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth1: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 16, node addr 00:1e:0b:ec:d3:d2 bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:07:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030006
2.6.27.4 is running fine on this machine
Any ideas ?
Thank you
Eric

Eric Dumazet

author

2008-10-28 06:34 UTC #7 lore raw

Quoted 5 lines · Benjamin Li

Hi Eric,
It looks like the same problem again: we are reading past a valid VPD address.

Quoted 7 lines

After some initial debugging, it looks like dev->vpd is NULL when the Broadcom 5706C/5706S/5708C/570S/5709 quirk is called. Because of this dev->vpd->len is never set to the proper length to limit the VPD reads. I will debug further to determine what has changed.
Thanks again for pointing out the problem.

I did a git bisect this morning :

$ git bisect good
201de56eb22f1ff3f36804bc70cbff220b50f067 is first bad commit
commit 201de56eb22f1ff3f36804bc70cbff220b50f067
Author: Zhao, Yu <[email protected]>
Date:   Mon Oct 13 19:49:55 2008 +0800

PCI: centralize the capabilities code in probe.c

This patch centralizes the initialization and release functions of various PCI capabilities in probe.c, which makes the introduction of new capability support functions cleaner in the future.

Signed-off-by Yu Zhao <[email protected]>

Signed-off-by Jesse Barnes <[email protected]>

:040000 040000 2491e9db72bd403b118fd05030af759e5299cae9 8ffd2fc80eff495e4fd4b46ed97d10cb7bf92f3b M drivers

$ git bisect log
git bisect start
# good: [3fa8749e584b55f1180411ab1b51117190bac1e5] Linux 2.6.27
git bisect good 3fa8749e584b55f1180411ab1b51117190bac1e5
# good: [3fa8749e584b55f1180411ab1b51117190bac1e5] Linux 2.6.27
git bisect good 3fa8749e584b55f1180411ab1b51117190bac1e5
# good: [3fa8749e584b55f1180411ab1b51117190bac1e5] Linux 2.6.27
git bisect good 3fa8749e584b55f1180411ab1b51117190bac1e5
# bad: [8b5f12d04b2e93842f3dda01f029842047bf3f81] syncookies: fix inclusion of tcp options in syn-ack
git bisect bad 8b5f12d04b2e93842f3dda01f029842047bf3f81
# bad: [8b5f12d04b2e93842f3dda01f029842047bf3f81] syncookies: fix inclusion of tcp options in syn-ack
git bisect bad 8b5f12d04b2e93842f3dda01f029842047bf3f81
# good: [cf2fa66055d718ae13e62451bb546505f63906a2] Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-2.6
git bisect good cf2fa66055d718ae13e62451bb546505f63906a2
# good: [01e8ef11bc1a74e65678ed55795f59266d4add01] x86: sysfs: kill owner field from attribute
git bisect good 01e8ef11bc1a74e65678ed55795f59266d4add01
# bad: [9779a8325a9bbf4ccd3853e0e4064984cf9da9c9] Merge branch 'for-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/dvrabel/uwb
git bisect bad 9779a8325a9bbf4ccd3853e0e4064984cf9da9c9
# good: [7110879cf2afbfb7af79675f5ff109e63d631c25] Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
git bisect good 7110879cf2afbfb7af79675f5ff109e63d631c25
# good: [92b29b86fe2e183d44eb467e5e74a5f718ef2e43] Merge branch 'tracing-v28-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
git bisect good 92b29b86fe2e183d44eb467e5e74a5f718ef2e43
# bad: [72441bdc76f7f71d7b75cdaa48f26dbb1f3d932e] Merge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6
git bisect bad 72441bdc76f7f71d7b75cdaa48f26dbb1f3d932e
# bad: [e3d2f927f788adcdabc42f8a1616f6cc56c53bbe] Merge git://git.kernel.org/pub/scm/linux/kernel/git/kyle/parisc-2.6
git bisect bad e3d2f927f788adcdabc42f8a1616f6cc56c53bbe
# bad: [201de56eb22f1ff3f36804bc70cbff220b50f067] PCI: centralize the capabilities code in probe.c
git bisect bad 201de56eb22f1ff3f36804bc70cbff220b50f067
# good: [7f2feec140f1f1e4f701e013a2bf8284a9ec2a3c] PCI: pciehp: replace printk with dev_printk
git bisect good 7f2feec140f1f1e4f701e013a2bf8284a9ec2a3c
# good: [c8761fe80ed052634153438405c9048611ae7ae1] PCI: fix hotplug get_##name return value problem
git bisect good c8761fe80ed052634153438405c9048611ae7ae1
# good: [022edd86d7c864bc8fadc3c8ac4e6a464472ab05] PCI: use resource_size() everywhere.
git bisect good 022edd86d7c864bc8fadc3c8ac4e6a464472ab05
# good: [e354597cce8d219d135d65e585dc4f30323486b9] PCI: fix 64-vbit prefetchable memory resource BARs
git bisect good e354597cce8d219d135d65e585dc4f30323486b9
# good: [280c73d3691fb182fa55b0160737c2c0feb79471] PCI: centralize the capabilities code in pci-sysfs.c
git bisect good 280c73d3691fb182fa55b0160737c2c0feb79471

Quoted 127 lines

-Ben
Quoted 105 lines · Eric Dumazet
Quoted 78 lines · Eric Dumazet
While trying to upgrade kernel from 2.6.25 to 2.6.26 on a ProLiant BL460c G1, I faced this problem on integrated Broadcom 5708S NIC when trying to UP them.
bnx2: fw sync timeout, reset code = 1030003
I noticed that commit 99cb233d60cbe644203f19938c729ea2bb004d70 (PCI: Limit VPD read/write lengths for Broadcom 5706, 5708, 5709 rev.A) was probably a good candidate to solve the problem.
Apparently, it added PCI quirks for a list of Broadcom NICs, but quirk_brcm_570x_limit_vpd() ignores part of them.
I found the following patch useful to restore correct operation on my machine.
Some information about this nic :
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12)
Subsystem: Hewlett-Packard Company NC373i Integrated
Multifunction Gigabit Server Adapter
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
       Latency: 64 (16000ns min), Cache Line Size 10
       Interrupt: pin A routed to IRQ 222
       Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M]
       [virtual] Expansion ROM at d1200000 [disabled] [size=16K]
       Capabilities: [40] PCI-X non-bridge device.
               Command: DPERE- ERO- RBC=0 OST=4
               Status: Bus=3 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, 
DC=simple, DMMRBC=0, DMOST=4, DMCRS=2, RSCEM-
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/0
Enable+
               Address: 00000000feeff00c  Data: 4179
after applying patch :
# hexdump -C "/sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0/0000:03:00.0/vpd"
00000000  82 2f 00 48 50 20 4e 43  33 37 33 69 20 4d 75 6c  |./.HP NC373i Mul|
00000010  74 69 66 75 6e 63 74 69  6f 6e 20 47 69 67 61 62  |tifunction Gigab|
00000020  69 74 20 53 65 72 76 65  72 20 41 64 61 70 74 65  |it Server Adapte|
00000030  72 00 90 4a 00 50 4e 03  4e 2f 41 45 43 03 4e 2f  |r..J.PN.N/AEC.N/|
00000040  41 53 4e 0a 30 31 32 33  34 35 36 37 38 39 4d 4e  |ASN.0123456789MN|
00000050  04 31 30 33 43 52 56 26  a2 00 00 00 00 00 00 00  |.103CRV&........|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 78  |...............x|
00000080
Patch against current Linus tree, should also be sent for linux-2.6.26.2 inclusion.
I have no idea if 5706S and 5709S also needs this truncation of VPD length
Thank you
[PATCH] PCI: Limit VPD length for Broadcom 5708S
BCM5708S wont work correctly unless VPD length truncated to 128
Signed-off-by Eric Dumazet <[email protected]>
drivers/pci/quirks.c +0 -0
index 0fb3650..59667e5 100644
@@ -1756,9 +1756,12 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_VIA, 
0x324e, quirk_via_cx700_pci_parking_c
*/
static void __devinit quirk_brcm_570x_limit_vpd(struct pci_dev *dev) {
drivers/pci/quirks.c +2 -1
-    /*  Only disable the VPD capability for 5706, 5708, and 5709 rev. A */
+    /*
+     * Only disable the VPD capability for 5706, 5708, 5708S and 5709 
rev. A
drivers/pci/quirks.c
Quoted 7 lines
drivers/pci/quirks.c +2 -0
+     */
    if ((dev->device == PCI_DEVICE_ID_NX2_5706) ||
        (dev->device == PCI_DEVICE_ID_NX2_5708) ||
+        (dev->device == PCI_DEVICE_ID_NX2_5708S) ||
        ((dev->device == PCI_DEVICE_ID_NX2_5709) &&
         (dev->revision & 0xf0) == 0x0)) {
        if (dev->vpd)
Hi all
Just tried linux-2.6.28-rc2 on same platform.
Same problem again...
Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.8.1 (Oct 7, 2008)
bnx2 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth0: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem f6000000, IRQ 16, node addr 00:1e:0b:ec:d3:dc bnx2 0000:07:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth1: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 16, node addr 00:1e:0b:ec:d3:d2 bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:07:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030006
Quoted 11 lines
2.6.27.4 is running fine on this machine
Any ideas ?
Thank you
Eric
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Yu Zhao

2008-10-28 07:39 UTC #8 lore raw

Quoted 22 lines · Eric Dumazet

Quoted 5 lines · Benjamin Li
Hi Eric,
It looks like the same problem again: we are reading past a valid VPD address.
Quoted 7 lines
After some initial debugging, it looks like dev->vpd is NULL when the Broadcom 5706C/5706S/5708C/570S/5709 quirk is called. Because of this dev->vpd->len is never set to the proper length to limit the VPD reads. I will debug further to determine what has changed.
Thanks again for pointing out the problem.
I did a git bisect this morning :
$ git bisect good
201de56eb22f1ff3f36804bc70cbff220b50f067 is first bad commit
commit 201de56eb22f1ff3f36804bc70cbff220b50f067
Author: Zhao, Yu <[email protected]>
Date:   Mon Oct 13 19:49:55 2008 +0800

Quoted 9 lines

PCI: centralize the capabilities code in probe.c
This patch centralizes the initialization and release functions of various PCI capabilities in probe.c, which makes the introduction of new capability support functions cleaner in the future.
Signed-off-by Yu Zhao <[email protected]>
Signed-off-by Jesse Barnes <[email protected]>

Yes, the VPD initialization code has been moved to a later stage that is after the pci_fixup_header. Can you please try following fix?

Thanks,
Yu

VPD quirks needs to be called after the capability is initialized. Since VPD initialization is move to after pci_fixup_header, VPD quirks should be done at pci_fixup_final stage correspondingly.

Signed-off-by Yu Zhao <[email protected]>

drivers/pci/quirks.c +18 ~~-18~~

index bbf66ea..5049a47 100644
@@ -1692,24 +1692,24 @@ static void __devinit quirk_brcm_570x_limit_vpd(struct pci_dev *dev)
 	}
 }
 
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5706,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5706S,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5708,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5708S,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5709,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5709S,
-			 quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5706,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5706S,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5708,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5708S,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5709,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5709S,
+			quirk_brcm_570x_limit_vpd);
 
 #ifdef CONFIG_PCI_MSI
 /* Some chipsets do not support MSI. We cannot easily rely on setting

Eric Dumazet

author Tested-by

2008-10-28 08:38 UTC #9 lore raw

Quoted 28 lines · Yu Zhao

Quoted 27 lines · Eric Dumazet
Quoted 5 lines · Benjamin Li
Hi Eric,
It looks like the same problem again: we are reading past a valid VPD address.
Quoted 7 lines
After some initial debugging, it looks like dev->vpd is NULL when the Broadcom 5706C/5706S/5708C/570S/5709 quirk is called. Because of this dev->vpd->len is never set to the proper length to limit the VPD reads. I will debug further to determine what has changed.
Thanks again for pointing out the problem.
I did a git bisect this morning :
$ git bisect good
201de56eb22f1ff3f36804bc70cbff220b50f067 is first bad commit
commit 201de56eb22f1ff3f36804bc70cbff220b50f067
Author: Zhao, Yu <[email protected]>
Date:   Mon Oct 13 19:49:55 2008 +0800
PCI: centralize the capabilities code in probe.c
This patch centralizes the initialization and release functions of various PCI capabilities in probe.c, which makes the introduction of new capability support functions cleaner in the future.

Quoted 7 lines

Quoted 3 lines
Signed-off-by Yu Zhao <[email protected]>
Signed-off-by Jesse Barnes <[email protected]>
Yes, the VPD initialization code has been moved to a later stage that is after the pci_fixup_header. Can you please try following fix?

Yes, this patch corrects the problem, thanks Yu.

Tested-by Eric Dumazet <[email protected]>

Quoted 58 lines

Thanks,
Yu

VPD quirks needs to be called after the capability is initialized. Since VPD initialization is move to after pci_fixup_header, VPD quirks should be done at pci_fixup_final stage correspondingly.

Signed-off-by Yu Zhao <[email protected]>

drivers/pci/quirks.c +18 ~~-18~~

index bbf66ea..5049a47 100644
@@ -1692,24 +1692,24 @@ static void __devinit quirk_brcm_570x_limit_vpd(struct pci_dev *dev)
 	}
 }
 
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5706,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5706S,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5708,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5708S,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5709,
-			 quirk_brcm_570x_limit_vpd);
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM,
-			 PCI_DEVICE_ID_NX2_5709S,
-			 quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5706,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5706S,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5708,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5708S,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5709,
+			quirk_brcm_570x_limit_vpd);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
+			PCI_DEVICE_ID_NX2_5709S,
+			quirk_brcm_570x_limit_vpd);
 
 #ifdef CONFIG_PCI_MSI
 /* Some chipsets do not support MSI. We cannot easily rely on setting

Jesse Barnes

2008-10-30 16:14 UTC #10 lore raw

In reply to Yu Zhao

Quoted 6 lines · Yu Zhao

VPD quirks needs to be called after the capability is initialized. Since VPD initialization is move to after pci_fixup_header, VPD quirks should be done at pci_fixup_final stage correspondingly.
Signed-off-by Yu Zhao <[email protected]>

Thanks, I'll queue this up for the next pull.

Jesse

Jesse Barnes

2008-11-03 22:37 UTC #11 lore raw

In reply to Yu Zhao

Quoted 5 lines · Yu Zhao

VPD quirks needs to be called after the capability is initialized. Since VPD initialization is move to after pci_fixup_header, VPD quirks should be done at pci_fixup_final stage correspondingly.
Signed-off-by Yu Zhao <[email protected]>

Thanks Yu. Applied to my for-linus branch. Looks like that was the only affected quirk.

Jesse

Benjamin Li

2008-10-28 16:25 UTC #12 lore raw

In reply to Benjamin Li

Hi Eric,

I think this is the commit that broke the Broadcom NIC VPD quirk fix.

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=201de56eb22f1ff3f36804bc70cbff220b50f067

This is because the quirk relied on pci_vpd_pci22_init() being called before pci_fixup_device() is called.

Now pci_fixup_device() is being called before pci_vpd_pci22_init(). Because of this reordering, dev->vpd is NULL rendering the quirk ineffective.

I will work on a solution and keep you posted.

Thanks again.

-Ben

Quoted 147 lines · Benjamin Li

Hi Eric,
It looks like the same problem again: we are reading past a valid VPD address.
After some initial debugging, it looks like dev->vpd is NULL when the Broadcom 5706C/5706S/5708C/570S/5709 quirk is called. Because of this dev->vpd->len is never set to the proper length to limit the VPD reads. I will debug further to determine what has changed.
Thanks again for pointing out the problem.
-Ben
Quoted 132 lines · Eric Dumazet
Quoted 93 lines · Eric Dumazet
While trying to upgrade kernel from 2.6.25 to 2.6.26 on a ProLiant BL460c G1,
I faced this problem on integrated Broadcom 5708S NIC when trying to UP them.
bnx2: fw sync timeout, reset code = 1030003
I noticed that commit 99cb233d60cbe644203f19938c729ea2bb004d70 (PCI:
Limit VPD read/write lengths for Broadcom 5706, 5708, 5709 rev.A) was probably a good candidate to solve the problem.
Apparently, it added PCI quirks for a list of Broadcom NICs, but quirk_brcm_570x_limit_vpd() ignores part of them.
I found the following patch useful to restore correct operation on my machine.
Some information about this nic :
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12)
Subsystem: Hewlett-Packard Company NC373i Integrated
Multifunction Gigabit Server Adapter
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
       Latency: 64 (16000ns min), Cache Line Size 10
       Interrupt: pin A routed to IRQ 222
       Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M]
       [virtual] Expansion ROM at d1200000 [disabled] [size=16K]
       Capabilities: [40] PCI-X non-bridge device.
               Command: DPERE- ERO- RBC=0 OST=4
               Status: Bus=3 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, 
DC=simple, DMMRBC=0, DMOST=4, DMCRS=2, RSCEM-
Capabilities: [48] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/0
Enable+
               Address: 00000000feeff00c  Data: 4179
after applying patch :
# hexdump -C 
"/sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0/0000:03:00.0/vpd"
00000000  82 2f 00 48 50 20 4e 43  33 37 33 69 20 4d 75 6c  |./.HP 
NC373i Mul|
00000010  74 69 66 75 6e 63 74 69  6f 6e 20 47 69 67 61 62  |tifunction 
Gigab|
00000020  69 74 20 53 65 72 76 65  72 20 41 64 61 70 74 65  |it Server 
Adapte|
00000030  72 00 90 4a 00 50 4e 03  4e 2f 41 45 43 03 4e 2f  
|r..J.PN.N/AEC.N/|
00000040  41 53 4e 0a 30 31 32 33  34 35 36 37 38 39 4d 4e  
|ASN.0123456789MN|
00000050  04 31 30 33 43 52 56 26  a2 00 00 00 00 00 00 00  
|.103CRV&........|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  
|................|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 78  
|...............x|
00000080
Patch against current Linus tree, should also be sent for linux-2.6.26.2 inclusion.
I have no idea if 5706S and 5709S also needs this truncation of VPD length
Thank you
[PATCH] PCI: Limit VPD length for Broadcom 5708S
BCM5708S wont work correctly unless VPD length truncated to 128
Signed-off-by Eric Dumazet <[email protected]>
drivers/pci/quirks.c +0 -0
index 0fb3650..59667e5 100644
@@ -1756,9 +1756,12 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_VIA, 
0x324e, quirk_via_cx700_pci_parking_c
*/
static void __devinit quirk_brcm_570x_limit_vpd(struct pci_dev *dev) {
drivers/pci/quirks.c +2 -1
-    /*  Only disable the VPD capability for 5706, 5708, and 5709 rev. A */
+    /*
+     * Only disable the VPD capability for 5706, 5708, 5708S and 5709 
rev. A
drivers/pci/quirks.c
Quoted 7 lines
drivers/pci/quirks.c +2 -0
+     */
    if ((dev->device == PCI_DEVICE_ID_NX2_5706) ||
        (dev->device == PCI_DEVICE_ID_NX2_5708) ||
+        (dev->device == PCI_DEVICE_ID_NX2_5708S) ||
        ((dev->device == PCI_DEVICE_ID_NX2_5709) &&
         (dev->revision & 0xf0) == 0x0)) {
        if (dev->vpd)
Hi all
Just tried linux-2.6.28-rc2 on same platform.
Same problem again...
Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.8.1 (Oct 7, 2008)
bnx2 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth0: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem f6000000, IRQ 16, node addr 00:1e:0b:ec:d3:dc bnx2 0000:07:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
eth1: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 16, node addr 00:1e:0b:ec:d3:d2 bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:07:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030003
bnx2 0000:03:00.0: irq 45 for MSI/MSI-X
bnx2: fw sync timeout, reset code = 1030006
2.6.27.4 is running fine on this machine
Any ideas ?
Thank you
Eric