PCI: Block power management of certain ports with slot implemented bit set
PATCH v2 1/1 PCI: Block power management of certain ports with slot implemented bit set
Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is connected to Alpine Ridge Thunderbolt controller. This port has slot implemented bit set in the config space but other than that it is not hotplug capable in the sense we are expecting in Linux (it has dev->is_hotplug_bridge set to 0):
00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5
Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 Memory behind bridge: 78000000-8fffffff [size=384M]
Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M]
...
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
...
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #8, PowerLimit 25.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState+
This system is using ACPI based hotplug to notify the OS that it needs to rescan the PCI bus (ACPI hotplug).
If there is nothing connected to any of the Thunderbolt ports the root port will not have any runtime PM active children and is thus automatically runtime suspended pretty soon after boot by PCI PM core. Now, when a device is connected the BIOS SMI handler responsible for enumerating newly added devices is not able to find anything because the port is in D3.
For this reason we block power management of PCIe root and downstream ports that have slot implemented set and have node in ACPI namespace.
---
Changes from v1:
- Block PM for all root and downstream ports with slot implemented bit set and has an ACPI companion.
The previous version can be found here:
https://patchwork.kernel.org/patch/10711553/
drivers/pci/pci.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
drivers/pci/pci.c +17 -1
@@ -2510,10 +2510,14 @@ void pci_config_pm_runtime_put(struct pci_dev *pdev)*/ bool pci_bridge_d3_possible (struct pci_dev *bridge) { +int type; +if (!pci_is_pcie (bridge))return false; -switch (pci_pcie_type (bridge)) { + type =pci_pcie_type (bridge); + +switch (type) {case PCI_EXP_TYPE_ROOT_PORT :case PCI_EXP_TYPE_UPSTREAM :case PCI_EXP_TYPE_DOWNSTREAM :
@@ -2546,6 +2550,18 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge)if (bridge->is_hotplug_bridge)return false; +/* +* Some systems such as Gigabyte X299 the root port is +* not marked hotplug capable but ACPI based hotplug is +* still used to bring in the Thunderbolt controller. To +* make sure those ports do not enter D3 and possibly +* confuse the BIOS SMI handler, block D3 for them. +*/ +if (has_acpi_companion (&bridge->dev) && + type !=PCI_EXP_TYPE_UPSTREAM && +pcie_caps_reg (bridge) &PCI_EXP_FLAGS_SLOT ) +return false; +/* * It should be safe to put PCIe ports from 2015 or newer * to D3.
Signature · 1 line
2.19.2
Quoted 31 lines · Mika Westerberg
Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is connected to Alpine Ridge Thunderbolt controller. This port has slot implemented bit set in the config space but other than that it is not hotplug capable in the sense we are expecting in Linux (it has
dev->is_hotplug_bridgeset to 0):00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5
Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 Memory behind bridge: 78000000-8fffffff [size=384M]
Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M]
...
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
...
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #8, PowerLimit 25.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState+This system is using ACPI based hotplug to notify the OS that it needs to rescan the PCI bus (ACPI hotplug).
If there is nothing connected to any of the Thunderbolt ports the root port will not have any runtime PM active children and is thus automatically runtime suspended pretty soon after boot by PCI PM core. Now, when a device is connected the BIOS SMI handler responsible for enumerating newly added devices is not able to find anything because the port is in D3.
Quoted 6 lines
For this reason we block power management of PCIe root and downstream ports that have slot implemented set and have node in ACPI namespace.
Reported-by Kedar A Dongre <[email protected]>Signed-off-by Mika Westerberg <[email protected]>
Quoted 55 lines
---
Changes from v1:- Block PM for all root and downstream ports with slot implemented bit set and has an ACPI companion.
The previous version can be found here:
https://patchwork.kernel.org/patch/10711553/
drivers/pci/pci.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)drivers/pci/pci.c +17
-2@@ -2510,10 +2510,14 @@ void pci_config_pm_runtime_put(struct pci_dev *pdev)*/ bool pci_bridge_d3_possible (struct pci_dev *bridge) { +int type; +if (!pci_is_pcie (bridge))return false; -switch (pci_pcie_type (bridge)) { + type =pci_pcie_type (bridge); + +switch (type) {case PCI_EXP_TYPE_ROOT_PORT :case PCI_EXP_TYPE_UPSTREAM :case PCI_EXP_TYPE_DOWNSTREAM :@@ -2546,6 +2550,18 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge)if (bridge->is_hotplug_bridge)return false; +/* +* Some systems such as Gigabyte X299 the root port is +* not marked hotplug capable but ACPI based hotplug is +* still used to bring in the Thunderbolt controller. To +* make sure those ports do not enter D3 and possibly +* confuse the BIOS SMI handler, block D3 for them. +*/ +if (has_acpi_companion (&bridge->dev) && + type !=PCI_EXP_TYPE_UPSTREAM && +pcie_caps_reg (bridge) &PCI_EXP_FLAGS_SLOT ) +return false; +/* * It should be safe to put PCIe ports from 2015 or newer * to D3. -- 2.19.2
Quoted 33 lines · Rafael J. Wysocki
Quoted 31 lines · Mika Westerberg
Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is connected to Alpine Ridge Thunderbolt controller. This port has slot implemented bit set in the config space but other than that it is not hotplug capable in the sense we are expecting in Linux (it has
dev->is_hotplug_bridgeset to 0):00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5
Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 Memory behind bridge: 78000000-8fffffff [size=384M]
Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M]
...
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
...
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #8, PowerLimit 25.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState+This system is using ACPI based hotplug to notify the OS that it needs to rescan the PCI bus (ACPI hotplug).
If there is nothing connected to any of the Thunderbolt ports the root port will not have any runtime PM active children and is thus automatically runtime suspended pretty soon after boot by PCI PM core. Now, when a device is connected the BIOS SMI handler responsible for enumerating newly added devices is not able to find anything because the port is in D3.
Quoted 8 lines
Quoted 6 lines
For this reason we block power management of PCIe root and downstream ports that have slot implemented set and have node in ACPI namespace.
Reported-by Kedar A Dongre <[email protected]>Signed-off-by Mika Westerberg <[email protected]>Reviewed-by Rafael J. Wysocki <[email protected]>
Thanks!
However, I'm having second toughts about this because I remembered that people put a lot of effort getting discrete graphics with power resource attached to the root port powering off properly. If the root port matches the criteria in this patch it will not be able to go into D3 anymore. It might affect others such as M.2 connected NVMe or WiFi chip as well. For that reason I would still prefer blacklist, at least for now.
Quoted 44 lines · Mika Westerberg
Quoted 33 lines · Rafael J. Wysocki
Quoted 31 lines · Mika Westerberg
Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is connected to Alpine Ridge Thunderbolt controller. This port has slot implemented bit set in the config space but other than that it is not hotplug capable in the sense we are expecting in Linux (it has
dev->is_hotplug_bridgeset to 0):00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5
Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 Memory behind bridge: 78000000-8fffffff [size=384M]
Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M]
...
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
...
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #8, PowerLimit 25.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState+This system is using ACPI based hotplug to notify the OS that it needs to rescan the PCI bus (ACPI hotplug).
If there is nothing connected to any of the Thunderbolt ports the root port will not have any runtime PM active children and is thus automatically runtime suspended pretty soon after boot by PCI PM core. Now, when a device is connected the BIOS SMI handler responsible for enumerating newly added devices is not able to find anything because the port is in D3.
Quoted 8 lines
Quoted 6 lines
For this reason we block power management of PCIe root and downstream ports that have slot implemented set and have node in ACPI namespace.
Reported-by Kedar A Dongre <[email protected]>Signed-off-by Mika Westerberg <[email protected]>Reviewed-by Rafael J. Wysocki <[email protected]>
Quoted 8 lines
Thanks!
However, I'm having second toughts about this because I remembered that people put a lot of effort getting discrete graphics with power resource attached to the root port powering off properly. If the root port matches the criteria in this patch it will not be able to go into D3 anymore. It might affect others such as M.2 connected NVMe or WiFi chip as well. For that reason I would still prefer blacklist, at least for now.
Blacklist would be fine by me too.
Hi Peter,
Quoted 43 lines · Mika Westerberg
Quoted 33 lines · Rafael J. Wysocki
Quoted 31 lines · Mika Westerberg
Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is connected to Alpine Ridge Thunderbolt controller. This port has slot implemented bit set in the config space but other than that it is not hotplug capable in the sense we are expecting in Linux (it has
dev->is_hotplug_bridgeset to 0):00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5
Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 Memory behind bridge: 78000000-8fffffff [size=384M]
Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M]
...
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
...
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #8, PowerLimit 25.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState+This system is using ACPI based hotplug to notify the OS that it needs to rescan the PCI bus (ACPI hotplug).
If there is nothing connected to any of the Thunderbolt ports the root port will not have any runtime PM active children and is thus automatically runtime suspended pretty soon after boot by PCI PM core. Now, when a device is connected the BIOS SMI handler responsible for enumerating newly added devices is not able to find anything because the port is in D3.
Quoted 8 lines
Quoted 6 lines
For this reason we block power management of PCIe root and downstream ports that have slot implemented set and have node in ACPI namespace.
Reported-by Kedar A Dongre <[email protected]>Signed-off-by Mika Westerberg <[email protected]>Reviewed-by Rafael J. Wysocki <[email protected]>
Quoted 8 lines
Thanks!
However, I'm having second toughts about this because I remembered that people put a lot of effort getting discrete graphics with power resource attached to the root port powering off properly. If the root port matches the criteria in this patch it will not be able to go into D3 anymore. It might affect others such as M.2 connected NVMe or WiFi chip as well. For that reason I would still prefer blacklist, at least for now.
Would this patch:
https://patchwork.ozlabs.org/patch/1021317/
break runtime D3cold for the discrete GPU on Optimus laptops such as your Clevo P651RA? Specifically, is the Root Port above the GPU marked "(Slot+)" in lspci -vv? (There doesn't seem to be raw lspci output in https://github.com/Lekensteyn/acpi-stuff)
Thanks,
Lukas
Hi Lukas,
Quoted 55 lines · Lukas Wunner
Hi Peter,
Quoted 43 lines · Mika Westerberg
Quoted 33 lines · Rafael J. Wysocki
Quoted 31 lines · Mika Westerberg
Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is connected to Alpine Ridge Thunderbolt controller. This port has slot implemented bit set in the config space but other than that it is not hotplug capable in the sense we are expecting in Linux (it has
dev->is_hotplug_bridgeset to 0):00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5
Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 Memory behind bridge: 78000000-8fffffff [size=384M]
Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M]
...
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
...
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #8, PowerLimit 25.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
Changed: MRL- PresDet+ LinkState+This system is using ACPI based hotplug to notify the OS that it needs to rescan the PCI bus (ACPI hotplug).
If there is nothing connected to any of the Thunderbolt ports the root port will not have any runtime PM active children and is thus automatically runtime suspended pretty soon after boot by PCI PM core. Now, when a device is connected the BIOS SMI handler responsible for enumerating newly added devices is not able to find anything because the port is in D3.
Quoted 8 lines
Quoted 6 lines
For this reason we block power management of PCIe root and downstream ports that have slot implemented set and have node in ACPI namespace.
Reported-by Kedar A Dongre <[email protected]>Signed-off-by Mika Westerberg <[email protected]>Reviewed-by Rafael J. Wysocki <[email protected]>Quoted 8 lines
Thanks!
However, I'm having second toughts about this because I remembered that people put a lot of effort getting discrete graphics with power resource attached to the root port powering off properly. If the root port matches the criteria in this patch it will not be able to go into D3 anymore. It might affect others such as M.2 connected NVMe or WiFi chip as well. For that reason I would still prefer blacklist, at least for now.
Quoted 8 lines
Would this patch:
https://patchwork.ozlabs.org/patch/1021317/break runtime D3cold for the discrete GPU on Optimus laptops such as your Clevo P651RA? Specifically, is the Root Port above the GPU marked "(Slot+)" in lspci -vv? (There doesn't seem to be raw lspci output in https://github.com/Lekensteyn/acpi-stuff)
Thanks for bringing this into my attention. There are a couple of full lspci dumps, for example for the Dell XPS 9560.
https://github.com/Lekensteyn/acpi-stuff/blob/master/d3test/XPS9560/lspci-bare-metal.txt
This has upstream port 00:01.0 attached to GPU 01:00.0
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 05) (prog-if 00 [Normal decode])
...
Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00and would indeed be negatively affected by this patch. I can observe the same for the Clevo P651RA (for which I can also send the full lspci dump if you need).
Signature · 3 lines
Kind regards, Peter Wu https://lekensteyn.nl
Quoted 14 lines · Peter Wu
Quoted 8 lines
Would this patch:
https://patchwork.ozlabs.org/patch/1021317/break runtime D3cold for the discrete GPU on Optimus laptops such as your Clevo P651RA? Specifically, is the Root Port above the GPU marked "(Slot+)" in lspci -vv? (There doesn't seem to be raw lspci output in https://github.com/Lekensteyn/acpi-stuff)
Thanks for bringing this into my attention. There are a couple of full lspci dumps, for example for the Dell XPS 9560.
https://github.com/Lekensteyn/acpi-stuff/blob/master/d3test/XPS9560/lspci-bare-metal.txt
This has upstream port 00:01.0 attached to GPU 01:00.0
Quoted 7 lines
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 05) (prog-if 00 [Normal decode]) ... Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00and would indeed be negatively affected by this patch. I can observe the same for the Clevo P651RA (for which I can also send the full lspci dump if you need).
Thanks both.
I guess that leaves us the blacklist. I can re-send it rebased on top of v5.0-rc1 if there are no objections.
Quoted 2 lines · Mika Westerberg
I guess that leaves us the blacklist. I can re-send it rebased on top of v5.0-rc1 if there are no objections.
Please constrain either the blacklist entry for the Gigabyte mainboard or the blacklist check itself to x86 using IS_ENABLED() or #ifdef to avoid code bloat on other arches.
Thanks,
Lukas