PCI: Disable PTM on Upstream Ports during suspend
PATCH 1/1 PCI: Disable PTM on Upstream Ports during suspend
From: Bjorn Helgaas <[email protected]>
During suspend, we want to disable PTM on Root Ports because that allows some chips, e.g., Intel mobile chips since Coffee Lake, to enter a lower-power PM state.
Previously we only disabled PTM for Root Ports, but PCIe r6.0, sec 2.2.8, strongly recommends that functions support generation of Messages in non-D0 states, so we must assume that Switch Upstream Ports or Endpoints may send PTM Request Messages while in D1, D2, and D3hot.
A PTM Message received by a Downstream Port, e.g., a Root Port, with PTM disabled must be treated as an Unsupported Request (sec 6.21.3).
With this topology:
0000:00:1d.0 Root Port to [bus 08-71]
0000:08:00.0 Switch Upstream Port to [bus 09-71]
Kai-Heng reported errors like this:
pcieport 0000:00:1d.0: AER: Uncorrected (Non-Fatal) error received: 0000:00:1d.0
pcieport 0000:00:1d.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
pcieport 0000:00:1d.0: device [8086:7ab0] error status/mask=00100000/00004000
pcieport 0000:00:1d.0: [20] UnsupReq (First)
pcieport 0000:00:1d.0: AER: TLP Header: 34000000 08000052 00000000 00000000
Decoding this (from PCIe r6.0, sec 2.2.1.1, 2.2.8.10) shows that 00:1d.0 logged an Unsupported Request error when it received a PTM Request with Requester ID 08:00.0.
To prevent this error, disable PTM when suspending Upstream Ports (including those on Endpoints), not just Root Ports.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215453
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216210
Based-on: https://lore.kernel.org/r/[email protected]
--- drivers/pci/pci.c | 30 ++++++++++++++---------------- drivers/pci/pcie/ptm.c | 8 +++++++- 2 files changed, 21 insertions(+), 17 deletions(-)
drivers/pci/pci.c +14 -16
@@ -2707,14 +2707,19 @@ int pci_prepare_to_sleep(struct pci_dev *dev)return -EIO ;/* -* There are systems (for example, Intel mobile chips since Coffee -* Lake) where the power drawn while suspended can be significantly -* reduced by disabling PTM on PCIe root ports as this allows the -* port to enter a lower-power PM state and the SoC to reach a -* lower-power idle state as a whole. +* We want to disable PTM on Root Ports because that allows some +* chips, e.g., Intel mobile chips since Coffee Lake, to enter a +* lower-power PM state. +* +* PCIe r6.0, sec 2.2.8, strongly recommends that functions support +* generation of messages in non-D0 states, so we assume Switch +* Upstream Ports or Endpoints may send PTM Requests while in D1, +* D2, and D3hot. A PTM message received by a Downstream Port +* (including a Root Port) with PTM disabled must be treated as an +* Unsupported Request (sec 6.21.3). To prevent this error, +* disable PTM in *all* devices, not just Root Ports. */ -if (pci_pcie_type (dev) ==PCI_EXP_TYPE_ROOT_PORT ) -pci_disable_ptm (dev); +pci_disable_ptm (dev);pci_enable_wake (dev, target_state, wakeup);
@@ -2764,15 +2769,8 @@ int pci_finish_runtime_suspend(struct pci_dev *dev)if (target_state ==PCI_POWER_ERROR )return -EIO ; -/* -* There are systems (for example, Intel mobile chips since Coffee -* Lake) where the power drawn while suspended can be significantly -* reduced by disabling PTM on PCIe root ports as this allows the -* port to enter a lower-power PM state and the SoC to reach a -* lower-power idle state as a whole. -*/ -if (pci_pcie_type (dev) ==PCI_EXP_TYPE_ROOT_PORT ) -pci_disable_ptm (dev); +/* See rationale above for disabling PTM */ +pci_disable_ptm (dev);__pci_enable_wake (dev, target_state,pci_dev_run_wake (dev));
drivers/pci/pcie/ptm.c +7 -1
@@ -31,12 +31,18 @@ static void pci_ptm_info(struct pci_dev *dev)void pci_disable_ptm (struct pci_dev *dev) { -int ptm; +int type, ptm;u16 ctrl;if (!pci_is_pcie (dev))return ; + type =pci_pcie_type (dev); +if (!(type ==PCI_EXP_TYPE_ROOT_PORT || + type ==PCI_EXP_TYPE_UPSTREAM || + type ==PCI_EXP_TYPE_ENDPOINT )) +return ; + ptm =pci_find_ext_capability (dev,PCI_EXT_CAP_ID_PTM );if (!ptm)return ;
Signature · 1 line
2.25.1
Quoted 37 lines · Bjorn Helgaas
From: Bjorn Helgaas <[email protected]>
During suspend, we want to disable PTM on Root Ports because that allows some chips, e.g., Intel mobile chips since Coffee Lake, to enter a lower-power PM state.
Previously we only disabled PTM for Root Ports, but PCIe r6.0, sec 2.2.8, strongly recommends that functions support generation of Messages in non-D0 states, so we must assume that Switch Upstream Ports or Endpoints may send PTM Request Messages while in D1, D2, and D3hot.
A PTM Message received by a Downstream Port, e.g., a Root Port, with PTM disabled must be treated as an Unsupported Request (sec 6.21.3).
With this topology:
0000:00:1d.0 Root Port to [bus 08-71]
0000:08:00.0 Switch Upstream Port to [bus 09-71]Kai-Heng reported errors like this:
pcieport 0000:00:1d.0: AER: Uncorrected (Non-Fatal) error received: 0000:00:1d.0
pcieport 0000:00:1d.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
pcieport 0000:00:1d.0: device [8086:7ab0] error status/mask=00100000/00004000
pcieport 0000:00:1d.0: [20] UnsupReq (First)
pcieport 0000:00:1d.0: AER: TLP Header: 34000000 08000052 00000000 00000000Decoding this (from PCIe r6.0, sec 2.2.1.1, 2.2.8.10) shows that 00:1d.0 logged an Unsupported Request error when it received a PTM Request with Requester ID 08:00.0.
To prevent this error, disable PTM when suspending Upstream Ports (including those on Endpoints), not just Root Ports.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215453
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=216210
Based-on: https://lore.kernel.org/r/[email protected]
Quoted 7 lines
Based-on-patch-by Kai-Heng Feng <[email protected]>Reported-by Kai-Heng Feng <[email protected]>Tested-by Kai-Heng Feng <[email protected]>Signed-off-by Bjorn Helgaas <[email protected]>Cc David E. Box <[email protected]>Cc Sathyanarayanan Kuppuswamy <[email protected]>---
Looks good to me
Quoted 77 lines
drivers/pci/pci.c | 30 ++++++++++++++---------------- drivers/pci/pcie/ptm.c | 8 +++++++- 2 files changed, 21 insertions(+), 17 deletions(-)drivers/pci/pci.c +14
-16@@ -2707,14 +2707,19 @@ int pci_prepare_to_sleep(struct pci_dev *dev)return -EIO ;/* -* There are systems (for example, Intel mobile chips since Coffee -* Lake) where the power drawn while suspended can be significantly -* reduced by disabling PTM on PCIe root ports as this allows the -* port to enter a lower-power PM state and the SoC to reach a -* lower-power idle state as a whole. +* We want to disable PTM on Root Ports because that allows some +* chips, e.g., Intel mobile chips since Coffee Lake, to enter a +* lower-power PM state. +* +* PCIe r6.0, sec 2.2.8, strongly recommends that functions support +* generation of messages in non-D0 states, so we assume Switch +* Upstream Ports or Endpoints may send PTM Requests while in D1, +* D2, and D3hot. A PTM message received by a Downstream Port +* (including a Root Port) with PTM disabled must be treated as an +* Unsupported Request (sec 6.21.3). To prevent this error, +* disable PTM in *all* devices, not just Root Ports. */ -if (pci_pcie_type (dev) ==PCI_EXP_TYPE_ROOT_PORT ) -pci_disable_ptm (dev); +pci_disable_ptm (dev);pci_enable_wake (dev, target_state, wakeup);@@ -2764,15 +2769,8 @@ int pci_finish_runtime_suspend(struct pci_dev *dev)if (target_state ==PCI_POWER_ERROR )return -EIO ; -/* -* There are systems (for example, Intel mobile chips since Coffee -* Lake) where the power drawn while suspended can be significantly -* reduced by disabling PTM on PCIe root ports as this allows the -* port to enter a lower-power PM state and the SoC to reach a -* lower-power idle state as a whole. -*/ -if (pci_pcie_type (dev) ==PCI_EXP_TYPE_ROOT_PORT ) -pci_disable_ptm (dev); +/* See rationale above for disabling PTM */ +pci_disable_ptm (dev);__pci_enable_wake (dev, target_state,pci_dev_run_wake (dev));drivers/pci/pcie/ptm.c +7
-1@@ -31,12 +31,18 @@ static void pci_ptm_info(struct pci_dev *dev)void pci_disable_ptm (struct pci_dev *dev) { -int ptm; +int type, ptm;u16 ctrl;if (!pci_is_pcie (dev))return ; + type =pci_pcie_type (dev); +if (!(type ==PCI_EXP_TYPE_ROOT_PORT || + type ==PCI_EXP_TYPE_UPSTREAM || + type ==PCI_EXP_TYPE_ENDPOINT )) +return ; + ptm =pci_find_ext_capability (dev,PCI_EXT_CAP_ID_PTM );if (!ptm)return ;
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Hi Bjorn,
drivers/pci/pcie/ptm.c static void pci_ptm_info(struct pci_dev *dev) · line 44
Quoted 5 lines · Bjorn Helgaas
drivers/pci/pcie/ptm.c +5
-0+ type =pci_pcie_type (dev); +if (!(type ==PCI_EXP_TYPE_ROOT_PORT || + type ==PCI_EXP_TYPE_UPSTREAM || + type ==PCI_EXP_TYPE_ENDPOINT )) +return ;
Perhaps switch () instead?
switch (pci_pcie_type(dev)) {
case PCI_EXP_TYPE_ROOT_PORT:
case PCI_EXP_TYPE_UPSTREAM:
case PCI_EXP_TYPE_ENDPOINT:
break; default: return; }
Either way,
[+cc Rafael, Rajvi, linux-pm]
Quoted 13 lines · Mika Westerberg
drivers/pci/pcie/ptm.c static void pci_ptm_info(struct pci_dev *dev) · line 44
Quoted 5 lines · Bjorn Helgaas
drivers/pci/pcie/ptm.c +5
-0+ type =pci_pcie_type (dev); +if (!(type ==PCI_EXP_TYPE_ROOT_PORT || + type ==PCI_EXP_TYPE_UPSTREAM || + type ==PCI_EXP_TYPE_ENDPOINT )) +return ;Perhaps switch () instead?
switch (
pci_pcie_type(dev)) {
casePCI_EXP_TYPE_ROOT_PORT:
casePCI_EXP_TYPE_UPSTREAM:
casePCI_EXP_TYPE_ENDPOINT:
Quoted 8 lines
break; default: return; }Either way,
Reviewed-by Mika Westerberg <[email protected]>
Thanks! I dropped this patch for now. I think Rajvi's similar patch [1] is much better because we only need this kind of check in one place -- where we enable PTM.
Also, we need to tighten this up so we *always* disable PTM [2]. The patch I posted still calls pci_disable_ptm() from pci_prepare_to_sleep(), which means we only do it when !pdev->state_saved, so we leave PTM enabled if the driver saves its own state.
Bjorn
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/CAJZ5v0iHckqia4OywKzSNWFCaq7eOkJcm5yXJdT2_sNdd36gDw@mail.gmail.com