pci_bus warnings about adjacent RW1C bits w/ iproc-pci
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using pcie_iproc; is this something to be worried about?
[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00
[ 2.632358] pci_bus 0000:00: root bus resource [mem
0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,
will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4 may corrupt adjacent RW1C bits
[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4 may corrupt adjacent RW1C bits
[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4c may corrupt adjacent RW1C bits
[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits
[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x4 may corrupt adjacent RW1C bits
[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0
offset 0xc may corrupt adjacent RW1C bits
[ 2.705531] PCI: bus0: Fast back to back transfers disabled
[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus
00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits
[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0xc8 may corrupt adjacent RW1C bits
[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits
[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0
offset 0x3e may corrupt adjacent RW1C bits
[ 2.758262] pci 0000:01:00.0: enabling Extended Tags
[ 2.763577] PCI: bus1: Fast back to back transfers disabled
[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem
0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem
0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01]
[ 2.788293] pci 0000:00:00.0: bridge window [mem
0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP
[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00
[ 2.922349] pci_bus 0001:00: root bus resource [mem
0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,
will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled
[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus
00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled
[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem
0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem
0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01]
[ 2.976434] pci 0001:00:00.0: bridge window [mem
0x40000000-0x400fffff]
Signature · 1 line
Florian
Hi Florian,
This is from the following commit:
commit fb26592301200dbbe4a9943fe188b57a46716900
Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500
PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
Regards,
Ray
Quoted 133 lines · Florian Fainelli
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
Quoted 25 lines · Ray Jui
Hi Florian,
This is from the following commit:
commit
fb26592301200dbbe4a9943fe188b57a46716900Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
Signed-off-by Bjorn Helgaas <[email protected]>Enthusiastically-Acked-by Russell King <[email protected]>Acked-by Thierry Reding <[email protected]>
Quoted 8 lines
This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
This warning is ratelimited, but maybe even that is overkill. It looks like we currently warn on every config write of less than 32 bits. How many of those do you see? Once per boot might be enough, or maybe once per device (so we get a hint when loading a new driver).
Quoted 134 lines · Florian Fainelli
Quoted 133 lines
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
Hi, we get about 9 of those messages on the Broadcom Settop box boot. I think there's more but it is rate limited. We could do individual 8-bit/16-bit/32-bit writes for the RC itself (we don't currently) but not for the downstream devices without changes to the HW. --Jim
Quoted 174 lines · Bjorn Helgaas
Quoted 25 lines · Ray Jui
Hi Florian,
This is from the following commit:
commit
fb26592301200dbbe4a9943fe188b57a46716900Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
Signed-off-by Bjorn Helgaas <[email protected]>Enthusiastically-Acked-by Russell King <[email protected]>Acked-by Thierry Reding <[email protected]>Quoted 8 lines
This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
This warning is ratelimited, but maybe even that is overkill. It looks like we currently warn on every config write of less than 32 bits. How many of those do you see? Once per boot might be enough, or maybe once per device (so we get a hint when loading a new driver).
Quoted 134 lines · Florian Fainelli
Quoted 133 lines
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
Quoted 35 lines · Bjorn Helgaas
Quoted 25 lines · Ray Jui
Hi Florian,
This is from the following commit:
commit
fb26592301200dbbe4a9943fe188b57a46716900Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
Signed-off-by Bjorn Helgaas <[email protected]>Enthusiastically-Acked-by Russell King <[email protected]>Acked-by Thierry Reding <[email protected]>Quoted 8 lines
This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
Quoted 4 lines
This warning is ratelimited, but maybe even that is overkill. It looks like we currently warn on every config write of less than 32 bits. How many of those do you see? Once per boot might be enough, or maybe once per device (so we get a hint when loading a new driver).
It's fairly trivial to change the warning in question to dev_warn_once(). If this is acceptable, I can crank it out.
Thanks,
Jon
Quoted 134 lines
Quoted 134 lines · Florian Fainelli
Quoted 133 lines
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
BTW, can someone point out to me what precise provision in the PCI spec says that 8/16/32 bit config space writes must be provided?
Quoted 181 lines · Jon Mason
Quoted 35 lines · Bjorn Helgaas
Quoted 25 lines · Ray Jui
Hi Florian,
This is from the following commit:
commit
fb26592301200dbbe4a9943fe188b57a46716900Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
Signed-off-by Bjorn Helgaas <[email protected]>Enthusiastically-Acked-by Russell King <[email protected]>Acked-by Thierry Reding <[email protected]>Quoted 8 lines
This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
Quoted 4 lines
This warning is ratelimited, but maybe even that is overkill. It looks like we currently warn on every config write of less than 32 bits. How many of those do you see? Once per boot might be enough, or maybe once per device (so we get a hint when loading a new driver).
It's fairly trivial to change the warning in question to
dev_warn_once(). If this is acceptable, I can crank it out.Thanks,
JonQuoted 134 lines
Quoted 134 lines · Florian Fainelli
Quoted 133 lines
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
PiBCVFcsIGNhbiBzb21lb25lIHBvaW50IG91dCB0byBtZSB3aGF0IHByZWNpc2UgcHJvdmlzaW9u IGluIHRoZSBQQ0kNCj4gc3BlYyBzYXlzIHRoYXQgOC8xNi8zMiBiaXQgY29uZmlnIHNwYWNlIHdy aXRlcyBtdXN0IGJlIHByb3ZpZGVkPw0KDQpGb3IgUENJZToNCg0KMi4yLjUuIEZpcnN0L0xhc3Qg RFcgQnl0ZSBFbmFibGVzIFJ1bGVzDQpCeXRlIEVuYWJsZXMgYXJlIGluY2x1ZGVkIHdpdGggTWVt b3J5LCBJL08sIGFuZCBDb25maWd1cmF0aW9uIFJlcXVlc3RzLiBUaGlzIHNlY3Rpb24gZGVmaW5l cyB0aGUNCmNvcnJlc3BvbmRpbmcgcnVsZXMuIEJ5dGUgRW5hYmxlcywgd2hlbiBwcmVzZW50IGlu IHRoZSBSZXF1ZXN0IGhlYWRlciwgYXJlIGxvY2F0ZWQgaW4gYnl0ZSA3IG9mIHRoZQ0KaGVhZGVy IChzZWUgRmlndXJlIDItOSkuDQoNCglEYXZpZA0KDQo=
Quoted 2 lines · Jim Quinlan
BTW, can someone point out to me what precise provision in the PCI spec says that 8/16/32 bit config space writes must be provided?
PCI r3.0, sec 3.2.2.3.2 ("Software Generation of Configuration Transactions") says
In both Type 0 and Type 1 translations, byte enables for the data transfers must be directly copied from the processor bus.
Maybe I should have mentioned that in the changelog; I guess I thought it was obvious from the example in the changelog that we can't handle RW1C bits in 8- and 16-bit registers correctly unless the hardware can perform 8- and 16-bit config writes.
Quoted 182 lines · Jon Mason
Quoted 181 lines · Bjorn Helgaas
Quoted 35 lines · Ray Jui
Quoted 25 lines
Hi Florian,
This is from the following commit:
commit
fb26592301200dbbe4a9943fe188b57a46716900Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
Signed-off-by Bjorn Helgaas <[email protected]>Enthusiastically-Acked-by Russell King <[email protected]>Acked-by Thierry Reding <[email protected]>Quoted 8 lines
This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
Quoted 4 lines
This warning is ratelimited, but maybe even that is overkill. It looks like we currently warn on every config write of less than 32 bits. How many of those do you see? Once per boot might be enough, or maybe once per device (so we get a hint when loading a new driver).
It's fairly trivial to change the warning in question to
dev_warn_once(). If this is acceptable, I can crank it out.Thanks,
JonQuoted 134 lines
Quoted 134 lines · Florian Fainelli
Quoted 133 lines
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
Quoted 36 lines · Jon Mason
Quoted 35 lines · Bjorn Helgaas
Quoted 33 lines · Ray Jui
Hi Florian,
This is from the following commit:
commit
fb26592301200dbbe4a9943fe188b57a46716900Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
Signed-off-by Bjorn Helgaas <[email protected]>Enthusiastically-Acked-by Russell King <[email protected]>Acked-by Thierry Reding <[email protected]>This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
Quoted 7 lines
Quoted 4 lines
This warning is ratelimited, but maybe even that is overkill. It looks like we currently warn on every config write of less than 32 bits. How many of those do you see? Once per boot might be enough, or maybe once per device (so we get a hint when loading a new driver).
It's fairly trivial to change the warning in question to
dev_warn_once(). If this is acceptable, I can crank it out.
I think my rationale was to make dmesg greppable for the device, e.g., if we had a problem with device 0000:00:00.0, "dmesg | grep 0000:00:00.0" would show this warning.
If we do a dev_warn_once(), we'll only see the warning once per system, so we may see the warning for "pci_bus 0000:00", but we could have an RW1C problem with a device on a different bus. So I'm a little hesitant to do that.
I'd really like it if we had the pci_dev; then we could add a bit in the pci_dev and print the warning once for each device that's vulnerable. But we don't.
We used to have the host bridge driver emit one warning, but that was unreliable because several drivers with this hardware limitation didn't emit a warning. fb2659230120 only removed the warning from hisi and rockchip. At least mach-ks8695, mach-sa1100, tegra, xgene, hisi, iproc, and qcom also have the limitation but didn't emit a warning.
Ironically, rockchip still has the hardware limitation but doesn't emit a warning because it doesn't use pci_generic_config_write32().
Quoted 134 lines
Quoted 134 lines
Quoted 134 lines · Florian Fainelli
Quoted 133 lines
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
Quoted 35 lines · Bjorn Helgaas
Quoted 25 lines · Ray Jui
Hi Florian,
This is from the following commit:
commit
fb26592301200dbbe4a9943fe188b57a46716900Author: Bjorn Helgaas <[email protected]> Date: Mon Oct 31 16:00:01 2016 -0500PCI: Warn on possible RW1C corruption for sub-32 bit config writes
Hardware that supports only 32-bit config writes is not spec-compliant. For example, if software performs a 16-bit write, we must do a 32-bit read, merge in the 16 bits we intend to write, followed by a 32-bit write. If the 16 bits we *don't* intend to write happen to have any RW1C (write-one- to-clear) bits set, we just inadvertently cleared something we shouldn't have.
Add a rate-limited warning when we do sub-32 bit config writes. Remove similar probe-time warnings from some of the affected host bridge drivers.
Signed-off-by Bjorn Helgaas <[email protected]>Enthusiastically-Acked-by Russell King <[email protected]>Acked-by Thierry Reding <[email protected]>Quoted 8 lines
This warning message will show up on every root complex that can only support 32-bit config register write (instead of 8-bit, 16-bit writes) and iProc based host controller happens to be one of them.
This limitation violates the PCIe spec as the 32-bit write may clear RW1C bits of a config register. But I believe those registers are usually used in AER which we do not currently support in the iProc based hos controllers.
Quoted 4 lines
This warning is ratelimited, but maybe even that is overkill. It looks like we currently warn on every config write of less than 32 bits. How many of those do you see? Once per boot might be enough, or maybe once per device (so we get a hint when loading a new driver).
These were the exact number that I saw but I did not bind the Wi-Fi driver to the WLAN card, so there could have been more once the Wi-Fi driver attempted to initialize the device.
Would it be helpful to differentiate e.g: new drivers being submitted that may support 8/16/32 bit accesses from drivers like iProc PCIe which works with silicon that does not support that?
Either way is fine with me really.
Quoted 135 lines
Quoted 135 lines · Florian Fainelli
Quoted 133 lines
Hi,
With 4.14-rc3 on a BCM5301x device I am seeing the following PCI core warnings using
pcie_iproc; is this something to be worried about?[ 2.626059] pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00[ 2.632358] pci_bus 0000:00: root bus resource [mem0x08000000-0x0fffffff]
[ 2.639207] pci_bus 0000:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.647188] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.656838] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.666565] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4cmay corrupt adjacent RW1C bits[ 2.676524] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.686270] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x4may corrupt adjacent RW1C bits[ 2.695903] pci_bus 0000:00: 1-byte config write to 0000:00:00.0offset
0xcmay corrupt adjacent RW1C bits[ 2.705531] PCI: bus0: Fast back to back transfers disabled[ 2.711086] pci 0000:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.719076] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.728796] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0xc8may corrupt adjacent RW1C bits[ 2.738513] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.748225] pci_bus 0000:00: 2-byte config write to 0000:00:00.0offset
0x3emay corrupt adjacent RW1C bits[ 2.758262] pci 0000:01:00.0: enabling Extended Tags[ 2.763577] PCI: bus1: Fast back to back transfers disabled[ 2.769205] pci 0000:00:00.0: BAR 8: assigned [mem0x08000000-0x080fffff]
[ 2.776017] pci 0000:01:00.0: BAR 0: assigned [mem0x08000000-0x08003fff 64bit]
[ 2.783338] pci 0000:00:00.0: PCI bridge to [bus 01][ 2.788293] pci 0000:00:00.0: bridge window [mem0x08000000-0x080fffff]
[ 2.911536] pcie_iproc_bcma bcma0:8: link: UP[ 2.916052] pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00[ 2.922349] pci_bus 0001:00: root bus resource [mem0x40000000-0x47ffffff]
[ 2.929200] pci_bus 0001:00: No busn resource found for root bus,will use [bus 00-ff]
[ 2.937482] PCI: bus0: Fast back to back transfers disabled[ 2.943075] pci 0001:00:00.0: bridge configuration invalid ([bus00-00]), reconfiguring
[ 2.951713] PCI: bus1: Fast back to back transfers disabled[ 2.957354] pci 0001:00:00.0: BAR 8: assigned [mem0x40000000-0x400fffff]
[ 2.964166] pci 0001:01:00.0: BAR 0: assigned [mem0x40000000-0x40007fff 64bit]
[ 2.971461] pci 0001:00:00.0: PCI bridge to [bus 01][ 2.976434] pci 0001:00:00.0: bridge window [mem0x40000000-0x400fffff]
--
Florian
Signature · 1 line
Florian