net-tg3: Initialize REG_BASE_ADDR at PCI config offset 120 to 0
Subsystems (3)
PATCH 1/1 net-tg3: Initialize REG_BASE_ADDR at PCI config offset 120 to 0
The new tg3 driver leaves REG_BASE_ADDR (PCI config offset 120) uninitialized. From power on reset this register may have garbage in it. The Register Base Address register defines the device local address of a register. The data pointed to by this location is read or written using the Register Data register (PCI config offset 128). When REG_BASE_ADDR has garbage any read or write of Register Data Register (PCI 128) will cause the PCI bus to lock up. The TCO watchdog will fire and bring down the system.
--- drivers/net/ethernet/broadcom/tg3.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/net/ethernet/broadcom/tg3.c +3 -0
@@ -16499,6 +16499,9 @@ static int tg3_get_invariants(struct tg3 *tp, const struct pci_device_id *ent)/* Clear this out for sanity. */ tw32 (TG3PCI_MEM_WIN_BASE_ADDR ,0 ); +/* Clear TG3PCI_REG_BASE_ADDR to prevent hangs. */ +tw32 (TG3PCI_REG_BASE_ADDR ,0 ); +pci_read_config_dword (tp->pdev,TG3PCI_PCISTATE , &pci_state_reg); if ((pci_state_reg & PCISTATE_CONV_PCI_MODE) == 0 &&
Signature · 1 line
1.8.5.1
Quoted 9 lines · Nat Gurumoorthy
The new tg3 driver leaves
REG_BASE_ADDR(PCI config offset 120) uninitialized. From power on reset this register may have garbage in it. The Register Base Address register defines the device local address of a register. The data pointed to by this location is read or written using the Register Data register (PCI config offset 128). WhenREG_BASE_ADDRhas garbage any read or write of Register Data Register (PCI 128) will cause the PCI bus to lock up. The TCO watchdog will fire and bring down the system.
Is this to prevent problem from other software that may be scanning the PCI config space? It won't help if this happens before the tg3 driver is loaded, right?
Thanks.
Michael,
We had crashes when the PCI config space got scanned via/sys/devices/pcixxxx/....../config.
I agree that this fix will not help if the scan happens before the tg3 driver gets loaded.
Regards
Nat
Quoted 16 lines · Michael Chan
Quoted 9 lines · Nat Gurumoorthy
The new tg3 driver leaves
REG_BASE_ADDR(PCI config offset 120) uninitialized. From power on reset this register may have garbage in it. The Register Base Address register defines the device local address of a register. The data pointed to by this location is read or written using the Register Data register (PCI config offset 128). WhenREG_BASE_ADDRhas garbage any read or write of Register Data Register (PCI 128) will cause the PCI bus to lock up. The TCO watchdog will fire and bring down the system.Is this to prevent problem from other software that may be scanning the PCI config space? It won't help if this happens before the tg3 driver is loaded, right?
Thanks.
Signature · 2 lines
Regards Nat Gurumoorthy AB6SJ
Hello.
Quoted 6 lines · Natarajan Gurumoorthy
Michael,
We had crashes when the PCI config space got scanned via/sys/devices/pcixxxx/....../config.I agree that this fix will not help if the scan happens before the tg3 driver gets loaded.
Then perhaps a better place for such fixup would be a PCI quirk?
Quoted 19 lines
Regards
NatQuoted 15 lines · Michael Chan
Quoted 7 lines · Nat Gurumoorthy
The new tg3 driver leaves
REG_BASE_ADDR(PCI config offset 120) uninitialized. From power on reset this register may have garbage in it. The Register Base Address register defines the device local address of a register. The data pointed to by this location is read or written using the Register Data register (PCI config offset 128). WhenREG_BASE_ADDRhas garbage any read or write of Register Data Register (PCI 128) will cause the PCI bus to lock up. The TCO watchdog will fire and bring down the system.Is this to prevent problem from other software that may be scanning the PCI config space? It won't help if this happens before the tg3 driver is loaded, right?
Thanks.
WBR, Sergei
Quoted 8 lines · Sergei Shtylyov
Quoted 5 lines
We had crashes when the PCI config space got scanned via
/sys/devices/pcixxxx/....../config.I agree that this fix will not help if the scan happens before the tg3 driver gets loaded.
Then perhaps a better place for such fixup would be a PCI quirk?
Yes, I agree. Thanks.
Quoted 11 lines · Michael Chan
Quoted 8 lines · Sergei Shtylyov
Quoted 5 lines
We had crashes when the PCI config space got scanned via
/sys/devices/pcixxxx/....../config.I agree that this fix will not help if the scan happens before the tg3 driver gets loaded.
Then perhaps a better place for such fixup would be a PCI quirk?
Yes, I agree. Thanks.
On second thought, I think your original patch should be sufficient and we don't need to add the PCI quirk to cover so many devices. The reason is that indirect access needs to be explicitly enabled in the MISC_HOST_CTRL (0x68) register. The default value for register 0x68 should have indirect access disabled.
Nat, does this match what you're seeing? Did you ever see any system crash before tg3 loads?
Michael,
The only time I see crashes is after the tg3 driver has been
loaded into the system. I our use case we are poking around /sys/devices/pcixxxx/......../config.
I guess you will incorporate the original patch into the driver
and we can abandon this patch.
Regards
Nat
Quoted 22 lines · Michael Chan
Quoted 11 lines · Michael Chan
Quoted 8 lines · Sergei Shtylyov
Quoted 5 lines
We had crashes when the PCI config space got scanned via
/sys/devices/pcixxxx/....../config.I agree that this fix will not help if the scan happens before the tg3 driver gets loaded.
Then perhaps a better place for such fixup would be a PCI quirk?
Yes, I agree. Thanks.
On second thought, I think your original patch should be sufficient and we don't need to add the PCI quirk to cover so many devices. The reason is that indirect access needs to be explicitly enabled in the
MISC_HOST_CTRL(0x68) register. The default value for register0x68should have indirect access disabled.Nat, does this match what you're seeing? Did you ever see any system crash before tg3 loads?
Signature · 2 lines
Regards Nat Gurumoorthy AB6SJ
Michael Chan
Acked-byQuoted 3 lines · Natarajan Gurumoorthy
The only time I see crashes is after the tg3 driver has been
loaded into the system. I our use case we are poking around/sys/devices/pcixxxx/......../config.
David, please apply this first patch.
David Miller
maintainerFrom: Michael Chan <[email protected]>
Date: Mon, 9 Dec 2013 22:58:44 -0800
Quoted 13 lines · Michael Chan
Quoted 11 lines · Sergei Shtylyov
Quoted 8 lines
Quoted 5 lines
We had crashes when the PCI config space got scanned via
/sys/devices/pcixxxx/....../config.I agree that this fix will not help if the scan happens before the tg3 driver gets loaded.
Then perhaps a better place for such fixup would be a PCI quirk?
Yes, I agree. Thanks.
Quoted 8 lines
On second thought, I think your original patch should be sufficient and we don't need to add the PCI quirk to cover so many devices. The reason is that indirect access needs to be explicitly enabled in the
MISC_HOST_CTRL(0x68) register. The default value for register0x68should have indirect access disabled.Nat, does this match what you're seeing? Did you ever see any system crash before tg3 loads?
What if the kernel is booted via kexec, and the driver in the kernel we are kexec'ing from left indirect access enabled in MISC_HOST_CTRL?
Quoted 2 lines · David Miller
What if the kernel is booted via kexec, and the driver in the kernel we are kexec'ing from left indirect access enabled in
MISC_HOST_CTRL?
That should be ok. The driver will only use valid register offsets in indirect mode during run time, so register 0x78 should point to a valid register. If indirect mode is never used by the driver, it will point to zero with this patch. So register 0x78 will be valid (or zero) in the kexec'ed kernel.
David Miller
maintainerFrom: Michael Chan <[email protected]>
Date: Tue, 10 Dec 2013 10:49:39 -0800
Quoted 10 lines · David Miller
Quoted 3 lines
What if the kernel is booted via kexec, and the driver in the kernel we are kexec'ing from left indirect access enabled in
MISC_HOST_CTRL?That should be ok. The driver will only use valid register offsets in indirect mode during run time, so register
0x78should point to a valid register. If indirect mode is never used by the driver, it will point to zero with this patch. So register0x78will be valid (or zero) in the kexec'ed kernel.
Ok, that may be true, but I'd like to consider the much larger issue at hand.
If the indirect mechanism is enabled, some of the offsets that may be in there might be value, but would be entirely undesirable to be read because the read has side effects.
What if the interrupt status register is what gets read if the user scans config space at just the wrong moment, and therefore an interrupt gets lost?
I understand that the patch we are discussing is a serious improvement from the current situation, so I will apply it and queue it up for -stable.
However I think we need to do something reasonable to prevent the kinds of situations I described above.
Thanks.
Quoted 11 lines · David Miller
Ok, that may be true, but I'd like to consider the much larger issue at hand.
If the indirect mechanism is enabled, some of the offsets that may be in there might be value, but would be entirely undesirable to be read because the read has side effects.
What if the interrupt status register is what gets read if the user scans config space at just the wrong moment, and therefore an interrupt gets lost?
I did a quick audit of the driver to see if we are seriously exposed to this problem. Only very old chips (5700 to 5703) use indirect register access as workarounds for various issues. I looked at the IRQ path which uses status block to determine if there is work to do. If status block contains no new data, we check the PCI_STATE register and the bits don't get cleared on read. MAC status register which is used to determine link status does not have bits that are cleared on read. So we should be ok.
We use the statistics block for counters on these older chips so there is no conflict. On newer chips, we read counters from registers which get cleared on read, but we never use indirect methods to read these counters.
On all new chips, indirect method is not used except to download firmware (on 57766 only). Userspace read will not interfere with firmware download.
So it looks to me that we are in good shape. Thanks.
David Miller
maintainerFrom: Michael Chan <[email protected]>
Date: Wed, 11 Dec 2013 15:21:18 -0800
Quoted 1 line
So it looks to me that we are in good shape. Thanks.
Thanks for checking this out, and providing such detailed analysis.