PCI: Add pci=nobwctrl to disable the PCIe bandwidth controller
Subsystems (3)
cover PCI: Add pci=nobwctrl to disable the PCIe bandwidth controller
Enabling the PCIe bandwidth controller sets the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs; commit 46a9f70e93ef ("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") works around one such platform with a per-device quirk.
A per-device quirk only helps once the offending device is identified and a quirk has shipped, which is awkward for a boot hang since the machine cannot be booted to report the culprit. The only existing escape hatch, pcie_ports=compat, is a blunt instrument that also disables AER, PME, hotplug and DPC.
This series adds a pci=nobwctrl boot option to disable just the bandwidth controller system-wide, giving users an immediate, rebuild-free workaround while a targeted quirk is worked out.
Patch 1 adds the pci_bwctrl_available()/pci_no_bwctrl() infrastructure (mirroring pci_aer_available()/pci_no_aer()) with no functional change. Patch 2 wires up the pci=nobwctrl parameter and documents it.
Tested on a QEMU q35 topology with PCIe ports and switches: without the parameter the bwctrl service devices are present as before; with pci=nobwctrl they all disappear while PME/AER/hotplug services stay intact and boot is clean.
Guixin Liu (2):
PCI/bwctrl: Add pci_bwctrl_available()
PCI: Add pci=nobwctrl parameter to disable the PCIe bandwidth
controller
Documentation/admin-guide/kernel-parameters.txt | 6 ++++++ drivers/pci/pci.c | 2 ++ drivers/pci/pci.h | 4 ++++ drivers/pci/pcie/bwctrl.c | 12 ++++++++++++ drivers/pci/pcie/portdrv.c | 3 ++- 5 files changed, 26 insertions(+), 1 deletion(-)
Signature · 1 line
2.43.7
PATCH 1/2 PCI/bwctrl: Add pci_bwctrl_available()
Enabling the PCIe bandwidth controller sets the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs, e.g. commit 46a9f70e93ef ("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") works around one such case with a per-device quirk.
In preparation for a system-wide switch to turn the bandwidth controller off, add pci_bwctrl_available() and pci_no_bwctrl(), mirroring pci_aer_available()/pci_no_aer(), and consult the former when deciding whether to enable PCIE_PORT_SERVICE_BWCTRL.
No functional change intended: the controller stays available by default.
--- drivers/pci/pci.h | 4 ++++ drivers/pci/pcie/bwctrl.c | 12 ++++++++++++ drivers/pci/pcie/portdrv.c | 3 ++- 3 files changed, 18 insertions(+), 1 deletion(-)
drivers/pci/pci.h +4 -0
@@ -1121,8 +1121,12 @@ static inline void pcie_ecrc_get_policy(char *str) { }#ifdef CONFIG_PCIEPORTBUS void pcie_reset_lbms (struct pci_dev *port); +void pci_no_bwctrl (void ); +bool pci_bwctrl_available (void );#else static inline void pcie_reset_lbms (struct pci_dev *port) {} +static inline void pci_no_bwctrl (void ) {} +static inline bool pci_bwctrl_available (void ) {return false; }#endif struct pci_dev_reset_methods {
drivers/pci/pcie/bwctrl.c +12 -0
@@ -48,6 +48,18 @@ struct pcie_bwctrl_data {/* Prevent port removal during Link Speed changes. */ static DECLARE_RWSEM (pcie_bwctrl_setspeed_rwsem); +static bool pcie_bwctrl_disable; + +void pci_no_bwctrl (void ) +{ + pcie_bwctrl_disable = true; +} + +bool pci_bwctrl_available (void ) +{ +return !pcie_bwctrl_disable; +} +static bool pcie_valid_speed (enum pci_bus_speed speed) {return (speed >=PCIE_SPEED_2_5GT ) && (speed <=PCIE_SPEED_64_0GT );
drivers/pci/pcie/portdrv.c +2 -1
@@ -274,7 +274,8 @@ static int get_port_device_capability(struct pci_dev *dev)pcie_capability_read_dword (dev,PCI_EXP_LNKCAP , &linkcap);if (linkcap &PCI_EXP_LNKCAP_LBNC && -hweight8 (dev->supported_speeds) >1 ) +hweight8 (dev->supported_speeds) >1 && +pci_bwctrl_available ()) services |=PCIE_PORT_SERVICE_BWCTRL ; }
Signature · 1 line
2.43.7
[email protected]
AI reviewRe: [PATCH v1 1/2] PCI/bwctrl: Add pci_bwctrl_available()
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 7 lines
Enabling the PCIe bandwidth controller sets the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs, e.g. commit
46a9f70e93ef("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") works around one such case with a per-device quirk.
Quoted 8 lines
In preparation for a system-wide switch to turn the bandwidth controller off, add
pci_bwctrl_available()andpci_no_bwctrl(), mirroring pci_aer_available()/pci_no_aer(), and consult the former when deciding whether to enablePCIE_PORT_SERVICE_BWCTRL.No functional change intended: the controller stays available by default.
Signed-off-by Guixin Liu <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH 2/2 PCI: Add pci=nobwctrl parameter to disable the PCIe bandwidth controller
In reply to Guixin LiuThe PCIe bandwidth controller enables the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs; commit 46a9f70e93ef ("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") addresses one such platform with a per-device quirk.
Per-device quirks only help once the offending device has been identified and a quirk has shipped, which is awkward for a boot hang since the machine cannot be booted to report it. The only existing escape hatch, pcie_ports=compat, is a blunt instrument that also disables AER, PME, hotplug and DPC.
Add a pci=nobwctrl boot option to disable just the bandwidth controller system-wide, giving users an immediate, rebuild-free workaround while a targeted quirk is worked out.
--- Documentation/admin-guide/kernel-parameters.txt | 6 ++++++ drivers/pci/pci.c | 2 ++ 2 files changed, 8 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +6 -0
@@ -5053,6 +5053,12 @@ Kernel parameters noaer [PCIE] If the PCIEAER kernel config parameter is enabled, this kernel boot option can be used to disable the use of PCIE advanced error reporting. + nobwctrl [PCIE] Disable the PCIe bandwidth controller, + which manages Link Bandwidth Management + Interrupts. Enabling these interrupts has been + observed to cause boot hangs on some (typically + old) platforms; use this option to work around + such issues. nodomains [PCI] Disable support for multiple PCI root domains (aka PCI segments, in ACPI-speak). nommconf [X86] Disable use of MMCONFIG for PCI
drivers/pci/pci.c +2 -0
@@ -6756,6 +6756,8 @@ static int __init pci_setup(char *str) pcie_ats_disabled = true; }else if (!strcmp (str,"noaer" )) {pci_no_aer (); + }else if (!strcmp (str,"nobwctrl" )) { +pci_no_bwctrl (); }else if (!strcmp (str,"earlydump" )) { pci_early_dump = true; }else if (!strncmp (str,"realloc=" ,8 )) {
Signature · 1 line
2.43.7
[email protected]
AI reviewRe: [PATCH v1 2/2] PCI: Add pci=nobwctrl parameter to disable the PCIe bandwidth controller
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
The PCIe bandwidth controller enables the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs; commit
46a9f70e93ef("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") addresses one such platform with a per-device quirk.Per-device quirks only help once the offending device has been identified and a quirk has shipped, which is awkward for a boot hang since the machine cannot be booted to report it. The only existing escape hatch, pcie_ports=compat, is a blunt instrument that also disables AER, PME, hotplug and DPC.
Quoted 5 lines
Add a pci=nobwctrl boot option to disable just the bandwidth controller system-wide, giving users an immediate, rebuild-free workaround while a targeted quirk is worked out.
Signed-off-by Guixin Liu <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Quoted 7 lines · Guixin Liu
Enabling the PCIe bandwidth controller sets the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs; commit
46a9f70e93ef("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") works around one such platform with a per-device quirk.
Quoted 5 lines
A per-device quirk only helps once the offending device is identified and a quirk has shipped, which is awkward for a boot hang since the machine cannot be booted to report the culprit. The only existing escape hatch, pcie_ports=compat, is a blunt instrument that also disables AER, PME, hotplug and DPC.
Ideally we want to identify all devices that have problems if the bandwidth controller is enabled and add quirks for every one of them.
In a sense, a command line option disincentivizes reporting and fixing affected devices because users can just work around the problem.
Additionally, there is a general dislike in the community for adding new command line options.
Are you seeing issues on different devices than the one that's already been quirked? If so, a patch to add quirks for them would be welcome.
Thanks,
Lukas
Quoted 22 lines · Lukas Wunner
Quoted 7 lines · Guixin Liu
Enabling the PCIe bandwidth controller sets the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs; commit
46a9f70e93ef("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") works around one such platform with a per-device quirk.Quoted 5 lines
A per-device quirk only helps once the offending device is identified and a quirk has shipped, which is awkward for a boot hang since the machine cannot be booted to report the culprit. The only existing escape hatch, pcie_ports=compat, is a blunt instrument that also disables AER, PME, hotplug and DPC.
Ideally we want to identify all devices that have problems if the bandwidth controller is enabled and add quirks for every one of them.
In a sense, a command line option disincentivizes reporting and fixing affected devices because users can just work around the problem.
Additionally, there is a general dislike in the community for adding new command line options.
Quoted 7 lines
Are you seeing issues on different devices than the one that's already been quirked? If so, a patch to add quirks for them would be welcome.
Thanks,
Lukas
Thanks Lukas, that's fair - I fully agree that per-device quirks are the right long-term fix, and I'll gladly send quirks for any device I can reproduce this on.
I'd argue this option isn't really an alternative to quirks, though. It's what lets you *get to* a quirk in the first place, and it closes a gap that is specific to bwctrl:
- To write a bwctrl quirk you first have to prove bwctrl is the culprit. pcie_ports=compat can get a hung machine booted, but it tears down *all* port services at once, so it can't tell you whether the offender was bwctrl, AER, PME or hotplug. pci=nobwctrl isolates exactly one variable, which is precisely the information a quirk needs. In that sense it feeds the quirk pipeline rather than discouraging reports.
- compat is also too blunt as a workaround on modern systems: losing native AER/DPC/hotplug just to silence bwctrl is a large regression in error handling. A surgical switch keeps everything else native.
- bwctrl is currently the only PCIe port service with neither a native/_OSC gate nor a disable of its own. hotplug, AER and PME are all gated on pcie_ports_native || host->native_*, and pci=noaer already exists as precedent. This mostly brings bwctrl up to parity rather than inventing a new class of knob.
If the command-line parameter itself is the sticking point, I'm happy to implement this in whatever form you prefer - e.g. gating bwctrl registration the same way the other services are gated, or a per-port sysfs runtime control. My goal is to close the gap; the mechanism is negotiable.
Best Regards,
Guixin Liu
Quoted 58 lines · Guixin Liu
Quoted 22 lines · Lukas Wunner
Quoted 7 lines · Guixin Liu
Enabling the PCIe bandwidth controller sets the Link Bandwidth Management Interrupt Enable (LBMIE) and Link Autonomous Bandwidth Interrupt Enable (LABIE) bits. On some (typically old) platforms this has been observed to cause boot hangs; commit
46a9f70e93ef("PCI/bwctrl: Disable BW controller on Intel P45 using a quirk") works around one such platform with a per-device quirk.Quoted 5 lines
A per-device quirk only helps once the offending device is identified and a quirk has shipped, which is awkward for a boot hang since the machine cannot be booted to report the culprit. The only existing escape hatch, pcie_ports=compat, is a blunt instrument that also disables AER, PME, hotplug and DPC.
Ideally we want to identify all devices that have problems if the bandwidth controller is enabled and add quirks for every one of them.
In a sense, a command line option disincentivizes reporting and fixing affected devices because users can just work around the problem.
Additionally, there is a general dislike in the community for adding new command line options.
Quoted 7 lines
Are you seeing issues on different devices than the one that's already been quirked? If so, a patch to add quirks for them would be welcome.
Thanks,
Lukas
Thanks Lukas, that's fair - I fully agree that per-device quirks are the right long-term fix, and I'll gladly send quirks for any device I can reproduce this on.
I'd argue this option isn't really an alternative to quirks, though. It's what lets you *get to* a quirk in the first place, and it closes a gap that is specific to bwctrl:
- To write a bwctrl quirk you first have to prove bwctrl is the culprit. pcie_ports=compat can get a hung machine booted, but it tears down *all* port services at once, so it can't tell you whether the offender was bwctrl, AER, PME or hotplug. pci=nobwctrl isolates exactly one variable, which is precisely the information a quirk needs. In that sense it feeds the quirk pipeline rather than discouraging reports.
- compat is also too blunt as a workaround on modern systems: losing native AER/DPC/hotplug just to silence bwctrl is a large regression in error handling. A surgical switch keeps everything else native.
- bwctrl is currently the only PCIe port service with neither a native/_OSC gate nor a disable of its own. hotplug, AER and PME are all gated on
pcie_ports_native|| host->native_*, and pci=noaer already exists as precedent. This mostly brings bwctrl up to parity rather than inventing a new class of knob.
Quoted 6 lines
If the command-line parameter itself is the sticking point, I'm happy to implement this in whatever form you prefer - e.g. gating bwctrl registration the same way the other services are gated, or a per-port sysfs runtime control. My goal is to close the gap; the mechanism is negotiable.
I'd argue the other way that developers can still add quirks for testing whether the crash is due to bwctrl or not.
I kind of agree with Lukas that once we add this knob, it may demotivate developers from sending the quirks.
- Mani
Signature · 1 line
மணிவண்ணன் சதாசிவம்