Skip to contentSkip to conversation

PCI/ACPI: PCI/ACPI: Validate devices with power resources support D3

PATCH In review Mario Limonciello 4 participants 35 messages Oct 31, 2022 – Jan 13, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 PCI/ACPI: PCI/ACPI: Validate devices with power resources support D3

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

When the thunderbolt driver is loaded two device links are created:
* USB4 router <- PCIe root port for tunneling
* USB4 router <- XHCI PCIe device

These device links are created because the ACPI devices declare the usb4-host-interface _DSD [3]. For both links the USB4 router is the supplier and these other devices are the consumers.
Here is a demonstration of this topology that occurs:

|

├─ 00:03.1
|       | "PCIe root port used for tunneling"
|       | ACPI Path: \_SB_.PCI0.GP11
|       | ACPI Power Resources: Yes
|       | ACPI _S0W return value: 0
|       | Device Links: supplier:pci:0000:c4:00.5
|       └─ PCIe Power state: D0
└─ 00:08.3
        | ACPI Path: \_SB_.PCI0.GP19
        ├─ PCIe Power state: D0
        ├─ c4:00.3
        |       | "XHCI PCIe device used for tunneling"
        |       | ACPI Path: \_SB_.PCI0.GP19.XHC3
        |       | ACPI Power Resources: Yes
        |       | ACPI _S0W return value: 4
        |       | Device Links: supplier:pci:0000:c4:00.5
        |       └─ PCIe Power state: D3cold
        └─ c4:00.5
                | "USB4 Router"
                | ACPI Path: \_SB_.PCI0.GP19.NHI0
                | ACPI Power Resources: Yes
                | ACPI _S0W return value: 4
                | Device Links: consumer:pci:0000:00:03.1 consumer:pci:0000:c4:00.3
                └─ PCIe Power state: D3cold

Currently runtime PM is allowed for all of these devices. This means that when all consumers are idle long enough, they will runtime suspend to enter their deepest allowed sleep state. Once all consumers are in their deepest allowed sleep state the suppliers will enter the deepest sleep state as well.

* The PCIe root port for tunneling doesn't support waking from D3hot or D3cold so although runtime suspended it stays in D0.
* The XHCI PCIe device supports wakeup from D3cold so it runtime suspends to D3cold.
* Both consumers are in their deepest state and the USB4 router supports wakeup from D3cold, so it runtime suspends into this state.

At least for AMD's case the hardware designer's expectation is the USB4 router should have also remained in D0 since the PCIe root port for tunneling remained in D0 and a device link exists between the two devices. The current Linux behavior is undefined.

Instead of making the assertion that the device can support D3 (and thus runtime PM) solely from the presence of ACPI power resources, move the check to later on in the function, which will have validated that the ACPI device supports wake from D3hot or D3cold.

This fix prevents the USB4 router being put into D3 when the firmware says that ACPI device representing the PCIe root port for tunneling can't handle it while still allowing ACPI devices that don't have the HotplugSupportInD3 _DSD to also enter D3 if they have power resources that can wake from D3.

---
v4->v5:
* Update github->git.kernel.org
* Correct arrow direction
* Clarify some commit message comments from Lukas' feedback.
v3->v4:
* Pick up tags
* Add more details to the commit message
v2->v3:
* Reword commit message
v1->v2:
* Just return value of acpi_pci_power_manageable (Rafael)
* Remove extra word in commit message

---
 drivers/pci/pci-acpi.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/pci/pci-acpi.c +2 -5
index a46fec776ad77..8c6aec50dd471 100644
@@ -984,10 +984,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
-
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
@@ -1023,7 +1019,8 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	    obj->integer.value == 1)
 		return true;
 
-	return false;
+	/* Assume D3 support if the bridge is power-manageable by ACPI. */
+	return acpi_pci_power_manageable(dev);
 }
 
 int acpi_pci_set_power_state(struct pci_dev *dev, pci_power_t state)
Signature · 1 line
2.34.1
#1 lore raw
Quoted 26 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

When the thunderbolt driver is loaded two device links are created:
* USB4 router <- PCIe root port for tunneling
* USB4 router <- XHCI PCIe device

These device links are created because the ACPI devices declare the usb4-host-interface _DSD [3]. For both links the USB4 router is the supplier and these other devices are the consumers.
Here is a demonstration of this topology that occurs:

Quoted 5 lines

|

├─ 00:03.1
|       | "PCIe root port used for tunneling"
|       | ACPI Path: \_SB_.PCI0.GP11
|       | ACPI Power Resources: Yes

I guess this means it has _PR0 and/or _PS0? (same for devices below)

Quoted 3 lines
|       | ACPI _S0W return value: 0
|       | Device Links: supplier:pci:0000:c4:00.5
|       └─ PCIe Power state: D0

What bus does 00:03.1 lead to? I guess it doesn't lead to any of the devices below?

Quoted 3 lines
└─ 00:08.3
        | ACPI Path: \_SB_.PCI0.GP19
        ├─ PCIe Power state: D0

I guess 00:08.3 is a Root Port leading to bus c4?

Quoted 6 lines
        ├─ c4:00.3
        |       | "XHCI PCIe device used for tunneling"
        |       | ACPI Path: \_SB_.PCI0.GP19.XHC3
        |       | ACPI Power Resources: Yes
        |       | ACPI _S0W return value: 4
        |       | Device Links: supplier:pci:0000:c4:00.5
Quoted 8 lines
        |       └─ PCIe Power state: D3cold
        └─ c4:00.5
                | "USB4 Router"
                | ACPI Path: \_SB_.PCI0.GP19.NHI0
                | ACPI Power Resources: Yes
                | ACPI _S0W return value: 4
                | Device Links: consumer:pci:0000:00:03.1 consumer:pci:0000:c4:00.3
                └─ PCIe Power state: D3cold

I'm reading the excellent Documentation/driver-api/device_link.rst and trying to match up with this topology. This is a case where 00:03.1 is a consumer of c4:00.5. The driver core automatically enforces that children are suspended before parents, but since 00:03.1 is an aunt (not a child) of c4:00.5, the device link enforces the requirement that 00:03.1 be suspended before c4:00.5. Right?

Is c4:00.5 an NHI?

The PCI power states shown above are the failure case? c4:00.5 *should* remain in D0 as long as 00:03.1 is in D0, but was instead put in D3cold?

Quoted 1 line

Currently runtime PM is allowed for all of these devices.

This is because they all have _PR0 and/or _PS0? (Diagram doesn't show that for 00:08.3, but I assume it must?)

And I guess they all must have dev->is_hotplug_bridge set?

Quoted 13 lines

This means that
when all consumers are idle long enough, they will runtime suspend to enter their deepest allowed sleep state. Once all consumers are in their deepest allowed sleep state the suppliers will enter the deepest sleep state as well.

* The PCIe root port for tunneling doesn't support waking from D3hot or D3cold so although runtime suspended it stays in D0.
* The XHCI PCIe device supports wakeup from D3cold so it runtime suspends to D3cold.
* Both consumers are in their deepest state and the USB4 router supports wakeup from D3cold, so it runtime suspends into this state.

Quoted 4 lines

At least for AMD's case the hardware designer's expectation is the USB4 router should have also remained in D0 since the PCIe root port for tunneling remained in D0 and a device link exists between the two devices. The current Linux behavior is undefined.

Is the requirement that the supplier can never be in a lower-power state than the consumer?

I guess that's *not* actually a requirement even though that's the effect of this patch in this situation. If it *were* that simple, we would just check the supplier and consumer power states instead of looking at all these ACPI properties.

Quoted 10 lines

Instead of making the assertion that the device can support D3 (and thus runtime PM) solely from the presence of ACPI power resources, move the check to later on in the function, which will have validated that the ACPI device supports wake from D3hot or D3cold.

This fix prevents the USB4 router being put into D3 when the firmware says that ACPI device representing the PCIe root port for tunneling can't handle it while still allowing ACPI devices that don't have the HotplugSupportInD3 _DSD to also enter D3 if they have power resources that can wake from D3.

So I guess this changes what acpi_pci_bridge_d3() returns for 00:03.1? Previously it returned "true" because it has _PR0/_PS0 (so acpi_pci_power_manageable() is true), but now it will return "false" because _S0W says it can only wake from D0?

Quoted 52 lines

---
v4->v5:
* Update github->git.kernel.org
* Correct arrow direction
* Clarify some commit message comments from Lukas' feedback.
v3->v4:
* Pick up tags
* Add more details to the commit message
v2->v3:
* Reword commit message
v1->v2:
* Just return value of acpi_pci_power_manageable (Rafael)
* Remove extra word in commit message

---
 drivers/pci/pci-acpi.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/pci/pci-acpi.c +2 -5
index a46fec776ad77..8c6aec50dd471 100644
@@ -984,10 +984,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
-
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
@@ -1023,7 +1019,8 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	    obj->integer.value == 1)
 		return true;
 
-	return false;
+	/* Assume D3 support if the bridge is power-manageable by ACPI. */
+	return acpi_pci_power_manageable(dev);
 }
 
 int acpi_pci_set_power_state(struct pci_dev *dev, pci_power_t state)
Signature · 1 line
2.34.1
#2 lore raw
Quoted 27 lines · Bjorn Helgaas
Quoted 26 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

When the thunderbolt driver is loaded two device links are created:
* USB4 router <- PCIe root port for tunneling
* USB4 router <- XHCI PCIe device

These device links are created because the ACPI devices declare the usb4-host-interface _DSD [3]. For both links the USB4 router is the supplier and these other devices are the consumers.
Here is a demonstration of this topology that occurs:

Quoted 7 lines
Quoted 5 lines

|

├─ 00:03.1
|       | "PCIe root port used for tunneling"
|       | ACPI Path: \_SB_.PCI0.GP11
|       | ACPI Power Resources: Yes

I guess this means it has _PR0 and/or _PS0? (same for devices below)

Yeah.

Quoted 7 lines
Quoted 4 lines
|       | ACPI _S0W return value: 0
|       | Device Links: supplier:pci:0000:c4:00.5
|       └─ PCIe Power state: D0

What bus does 00:03.1 lead to? I guess it doesn't lead to any of the devices below?

By default - nothing. The topology was drawn correctly.

When you plug in a USB4 device with PCIe the newly enumerated devices hang off this bus.

Quoted 7 lines
Quoted 4 lines
└─ 00:08.3
         | ACPI Path: \_SB_.PCI0.GP19
         ├─ PCIe Power state: D0

I guess 00:08.3 is a Root Port leading to bus c4?

Right.

Quoted 15 lines
Quoted 6 lines
         ├─ c4:00.3
         |       | "XHCI PCIe device used for tunneling"
         |       | ACPI Path: \_SB_.PCI0.GP19.XHC3
         |       | ACPI Power Resources: Yes
         |       | ACPI _S0W return value: 4
         |       | Device Links: supplier:pci:0000:c4:00.5
Quoted 8 lines
         |       └─ PCIe Power state: D3cold
         └─ c4:00.5
                 | "USB4 Router"
                 | ACPI Path: \_SB_.PCI0.GP19.NHI0
                 | ACPI Power Resources: Yes
                 | ACPI _S0W return value: 4
                 | Device Links: consumer:pci:0000:00:03.1 consumer:pci:0000:c4:00.3
                 └─ PCIe Power state: D3cold
Quoted 6 lines

I'm reading the excellent Documentation/driver-api/device_link.rst and trying to match up with this topology. This is a case where 00:03.1 is a consumer of c4:00.5. The driver core automatically enforces that children are suspended before parents, but since 00:03.1 is an aunt (not a child) of c4:00.5, the device link enforces the requirement that 00:03.1 be suspended before c4:00.5. Right?

Yup!

Quoted 2 lines

Is c4:00.5 an NHI?

Yes.

Quoted 4 lines

The PCI power states shown above are the failure case? c4:00.5 *should* remain in D0 as long as 00:03.1 is in D0, but was instead put in D3cold?

Yes.

Quoted 5 lines
Quoted 2 lines

Currently runtime PM is allowed for all of these devices.

This is because they all have _PR0 and/or _PS0? (Diagram doesn't show that for 00:08.3, but I assume it must?)

For the root port used for tunneling it's because pci_bridge_d3_possible() returns TRUE. This returns true because platform_pci_bridge(d3) return TRUE.

For the NHI it's because thunderbolt.ko sets this for all NHIs (see drivers/thunderbolt/nhi.c).

Quoted 2 lines

And I guess they all must have dev->is_hotplug_bridge set?

The PCIe root port for tunneling does; yes.

Quoted 19 lines
Quoted 14 lines

This means that
when all consumers are idle long enough, they will runtime suspend to enter their deepest allowed sleep state. Once all consumers are in their deepest allowed sleep state the suppliers will enter the deepest sleep state as well.

* The PCIe root port for tunneling doesn't support waking from D3hot or D3cold so although runtime suspended it stays in D0.
* The XHCI PCIe device supports wakeup from D3cold so it runtime suspends to D3cold.
* Both consumers are in their deepest state and the USB4 router supports wakeup from D3cold, so it runtime suspends into this state.

Quoted 4 lines

At least for AMD's case the hardware designer's expectation is the USB4 router should have also remained in D0 since the PCIe root port for tunneling remained in D0 and a device link exists between the two devices. The current Linux behavior is undefined.

Quoted 7 lines

Is the requirement that the supplier can never be in a lower-power state than the consumer?

I guess that's *not* actually a requirement even though that's the effect of this patch in this situation. If it *were* that simple, we would just check the supplier and consumer power states instead of looking at all these ACPI properties.

Yeah I think that's too broad of a generalization.

I don't know how realistic this is but for example what if the supplier supported D3 but the consumer supported D2?

Quoted 12 lines
Quoted 11 lines

Instead of making the assertion that the device can support D3 (and thus runtime PM) solely from the presence of ACPI power resources, move the check to later on in the function, which will have validated that the ACPI device supports wake from D3hot or D3cold.

This fix prevents the USB4 router being put into D3 when the firmware says that ACPI device representing the PCIe root port for tunneling can't handle it while still allowing ACPI devices that don't have the HotplugSupportInD3 _DSD to also enter D3 if they have power resources that can wake from D3.

Quoted 5 lines

So I guess this changes what acpi_pci_bridge_d3() returns for 00:03.1? Previously it returned "true" because it has _PR0/_PS0 (so acpi_pci_power_manageable() is true), but now it will return "false" because _S0W says it can only wake from D0?

Exactly.

#3 lore raw
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

If that's enough, maybe we don't need to complicate this with all the Thunderbolt and device link stuff. Which would be great, because the code change itself has nothing to do with those things.

Bjorn

#4 lore raw
Quoted 21 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Quoted 5 lines

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

It is.

Quoted 3 lines

If that's enough, maybe we don't need to complicate this with all the Thunderbolt and device link stuff. Which would be great, because the code change itself has nothing to do with those things.

Indeed.

#5 lore raw
Quoted 28 lines · Rafael J. Wysocki
Quoted 21 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Quoted 5 lines

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines

It is.

Quoted 3 lines

If that's enough, maybe we don't need to complicate this with all the Thunderbolt and device link stuff. Which would be great, because the code change itself has nothing to do with those things.

Indeed.

I'd think it's still useful to leave "something" in the commit message about how we got to that conclusion though.

Bjorn, do you want me to to attempt to rewrite the commit message and send a v6, or would you like to?

Thanks,

#6 lore raw
Quoted 34 lines · Mario
Quoted 33 lines · Rafael J. Wysocki
Quoted 20 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Quoted 5 lines

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

It is.

Quoted 3 lines

If that's enough, maybe we don't need to complicate this with all the Thunderbolt and device link stuff. Which would be great, because the code change itself has nothing to do with those things.

Quoted 7 lines
Quoted 1 line

Indeed.

I'd think it's still useful to leave "something" in the commit message about how we got to that conclusion though.

Bjorn, do you want me to to attempt to rewrite the commit message and send a v6, or would you like to?

Let me give it a try and post it for your reaction.

Bjorn

#7 lore raw
In reply to Rafael J. Wysocki
Quoted 22 lines · Rafael J. Wysocki
Quoted 21 lines · Bjorn Helgaas
Quoted 18 lines · Mario
Quoted 17 lines · Bjorn Helgaas
Quoted 16 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Quoted 7 lines
Quoted 5 lines

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-pcie-root-ports#identifying-pcie-root-ports-supporting-hot-plug-in-d3

#8 lore raw
Quoted 29 lines · Bjorn Helgaas
Quoted 28 lines · Rafael J. Wysocki
Quoted 26 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

It's impossible to know for sure the logic in Windows, but from all the discussion and patches that have flowed related to it my inference is the logic for Windows must only examine and use the "HotPlugSupportInD3" property if the device also has _S0W.

Quoted 6 lines

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

A lot of this confusion and this patch of mine stem from c6e331312ebfb being too broad to start out with IMO. Wouldn't it have made more sense to only match and allowlist that specific topology combination (dGPU connected to hotplug port and missing those properties)?

Quoted 1 line

[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flearn.microsoft.com%2Fen-us%2Fwindows-hardware%2Fdrivers%2Fpci%2Fdsd-for-pcie-root-ports%23identifying-pcie-root-ports-supporting-hot-plug-in-d3&amp;data=05%7C01%7Cmario.limonciello%40amd.com%7Cc883ba6351534df445f408dac76ac5e5%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041558659543898%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=5Qv3wUYB%2FXJhbeu%2Fh3A0swvgRaB8afjyEYzu9SpHK%2Bo%3D&amp;reserved=0

#9 lore raw
In reply to Bjorn Helgaas
Quoted 30 lines · Bjorn Helgaas
Quoted 28 lines · Rafael J. Wysocki
Quoted 26 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

Quoted 6 lines

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-pcie-root-ports#identifying-pcie-root-ports-supporting-hot-plug-in-d3

#10 lore raw
Quoted 48 lines · Rafael J. Wysocki
Quoted 29 lines · Bjorn Helgaas
Quoted 28 lines · Rafael J. Wysocki
Quoted 26 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Quoted 6 lines
Quoted 6 lines

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-pcie-root-ports#identifying-pcie-root-ports-supporting-hot-plug-in-d3

#11 lore raw
Quoted 72 lines · Bjorn Helgaas
Quoted 48 lines · Rafael J. Wysocki
Quoted 29 lines · Bjorn Helgaas
Quoted 28 lines · Rafael J. Wysocki
Quoted 26 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

Now, for compatibility with systems where ports have neither _S0W nor the HotPlugSupportInD3 property, the acpi_pci_power_manageable() return value should determine the outcome regardless of the adev->wakeup.flags.valid value, so the latter should only determine whether or not the HotPlugSupportInD3 property will be inspected (which may cause true to be returned before the "power manageable" check).

IOW, something like this (after checking _S0W):

if (adev->wakeup.flags.valid &&
!acpi_dev_get_property(adev, "HotPlugSupportInD3",
ACPI_TYPE_INTEGER, &obj) &&

    obj->integer.value == 1)
        return true;

return acpi_pci_power_manageable(dev);

Where the if () condition basically means that wakeup signaling is supported (and there is no indication that it cannot be done from D3 as per the previous _S0W check) and hotplug signaling from D3 is supported.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-pcie-root-ports#identifying-pcie-root-ports-supporting-hot-plug-in-d3

#12 lore raw
Quoted 90 lines · Rafael J. Wysocki
Quoted 71 lines · Bjorn Helgaas
Quoted 48 lines · Rafael J. Wysocki
Quoted 29 lines · Bjorn Helgaas
Quoted 28 lines · Rafael J. Wysocki
Quoted 26 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3? I guess in this case we assume wakeup would be via native PME/hotplug signaling?

Quoted 16 lines

Now, for compatibility with systems where ports have neither _S0W nor the HotPlugSupportInD3 property, the acpi_pci_power_manageable() return value should determine the outcome regardless of the adev->wakeup.flags.valid value, so the latter should only determine whether or not the HotPlugSupportInD3 property will be inspected (which may cause true to be returned before the "power manageable" check).

IOW, something like this (after checking _S0W):

if (adev->wakeup.flags.valid &&
!acpi_dev_get_property(adev, "HotPlugSupportInD3",
ACPI_TYPE_INTEGER, &obj) &&

    obj->integer.value == 1)
        return true;
Quoted 13 lines

return acpi_pci_power_manageable(dev);

Where the if () condition basically means that wakeup signaling is supported (and there is no indication that it cannot be done from D3 as per the previous _S0W check) and hotplug signaling from D3 is supported.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-pcie-root-ports#identifying-pcie-root-ports-supporting-hot-plug-in-d3

I think you're suggesting the patch below, which will make acpi_pci_bridge_d3(dev) return "true" if:

- Root Port can wake from D3hot or D3cold, has "HotPlugSupportInD3", and has wakeup.flags.valid, OR

- Root Port can wake from D3hot or D3cold, and "dev" has _PR0 or

    _PS0

Previously, all bridges with _PR0 or _PS0 could use D3; now we also require that the Root Port's _S0W says it can wake from at least D3hot.

drivers/pci/pci-acpi.c +10 -17
index a46fec776ad7..66c9ae1dc5da 100644
@@ -984,10 +984,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
-
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
@@ -996,14 +992,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	if (!adev)
 		return false;
 
-	/*
-	 * If the Root Port cannot signal wakeup signals at all, i.e., it
-	 * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
-	 * events from low-power states including D3hot and D3cold.
-	 */
-	if (!adev->wakeup.flags.valid)
-		return false;
-
 	/*
 	 * If the Root Port cannot wake itself from D3hot or D3cold, we
 	 * can't use D3.
@@ -1014,16 +1002,21 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 
 	/*
 	 * The "HotPlugSupportInD3" property in a Root Port _DSD indicates
-	 * the Port can signal hotplug events while in D3.  We assume any
-	 * bridges *below* that Root Port can also signal hotplug events
-	 * while in D3.
+	 * the Port can signal hotplug events while in D3.  This differs
+	 * from _S0W because _S0W may rely on GPEs, which can only be used
+	 * directly for wake signaling, not hotplug events.
+	 *
+	 * We assume any bridges *below* that Root Port can also signal
+	 * hotplug events while in D3.
 	 */
-	if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+	if (adev->wakeup.flags.valid &&
+	    !acpi_dev_get_property(adev, "HotPlugSupportInD3",
 				   ACPI_TYPE_INTEGER, &obj) &&
 	    obj->integer.value == 1)
 		return true;
 
-	return false;
+	/* Assume D3 support if the bridge is power-manageable by ACPI. */
+	return acpi_pci_power_manageable(dev);
 }
 
 int acpi_pci_set_power_state(struct pci_dev *dev, pci_power_t state)
#13 lore raw
Quoted 92 lines · Bjorn Helgaas
Quoted 90 lines · Rafael J. Wysocki
Quoted 71 lines · Bjorn Helgaas
Quoted 48 lines · Rafael J. Wysocki
Quoted 29 lines · Bjorn Helgaas
Quoted 28 lines · Rafael J. Wysocki
Quoted 26 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 7 lines
Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

_S0W may only cause acpi_pci_bridge_d3() to return false, it is not sufficient for true to be returned.

Quoted 2 lines

I guess in this case we assume wakeup would
be via native PME/hotplug signaling?

This may be taken into consideration at one point, if need be, but not in this particular patch IMO.

Quoted 30 lines
Quoted 16 lines

Now, for compatibility with systems where ports have neither _S0W nor the HotPlugSupportInD3 property, the acpi_pci_power_manageable() return value should determine the outcome regardless of the adev->wakeup.flags.valid value, so the latter should only determine whether or not the HotPlugSupportInD3 property will be inspected (which may cause true to be returned before the "power manageable" check).

IOW, something like this (after checking _S0W):

if (adev->wakeup.flags.valid &&
!acpi_dev_get_property(adev, "HotPlugSupportInD3",
ACPI_TYPE_INTEGER, &obj) &&

    obj->integer.value == 1)
        return true;
Quoted 13 lines

return acpi_pci_power_manageable(dev);

Where the if () condition basically means that wakeup signaling is supported (and there is no indication that it cannot be done from D3 as per the previous _S0W check) and hotplug signaling from D3 is supported.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-pcie-root-ports#identifying-pcie-root-ports-supporting-hot-plug-in-d3

Quoted 8 lines

I think you're suggesting the patch below, which will make acpi_pci_bridge_d3(dev) return "true" if:

- Root Port can wake from D3hot or D3cold, has "HotPlugSupportInD3", and has wakeup.flags.valid, OR

- Root Port can wake from D3hot or D3cold, and "dev" has _PR0 or

    _PS0

Well, not exactly. The second point should be

- Root Port's ACPI companion ('dev') has _PR0 or _PS0.

Quoted 3 lines

Previously, all bridges with _PR0 or _PS0 could use D3; now we also require that the Root Port's _S0W says it can wake from at least D3hot.

Yes, if _S0W is present and it evaluates successfully, then it is required to confirm that wakeup signaling from at least D3hot is supported for 'true' to be returned (but it is not sufficient for that by itself).

That's the only functional change made by that patch and yes, the patch below is what I mean.

Quoted 47 lines
drivers/pci/pci-acpi.c +8 -16
index a46fec776ad7..66c9ae1dc5da 100644
@@ -984,10 +984,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
        if (acpi_pci_disabled || !dev->is_hotplug_bridge)
                return false;
 
-       /* Assume D3 support if the bridge is power-manageable by ACPI. */
-       if (acpi_pci_power_manageable(dev))
-               return true;
-
        rpdev = pcie_find_root_port(dev);
        if (!rpdev)
                return false;
@@ -996,14 +992,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
        if (!adev)
                return false;
 
-       /*
-        * If the Root Port cannot signal wakeup signals at all, i.e., it
-        * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
-        * events from low-power states including D3hot and D3cold.
-        */
-       if (!adev->wakeup.flags.valid)
-               return false;
-
        /*
         * If the Root Port cannot wake itself from D3hot or D3cold, we
         * can't use D3.
@@ -1014,16 +1002,21 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 
        /*
         * The "HotPlugSupportInD3" property in a Root Port _DSD indicates
-        * the Port can signal hotplug events while in D3.  We assume any
-        * bridges *below* that Root Port can also signal hotplug events
-        * while in D3.
+        * the Port can signal hotplug events while in D3.  This differs
+        * from _S0W because _S0W may rely on GPEs, which can only be used
+        * directly for wake signaling, not hotplug events.
+        *
+        * We assume any bridges *below* that Root Port can also signal
+        * hotplug events while in D3.
         */
-       if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+       if (adev->wakeup.flags.valid &&
+           !acpi_dev_get_property(adev, "HotPlugSupportInD3",

drivers/pci/pci-acpi.c bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1022

Quoted 10 lines
drivers/pci/pci-acpi.c +2 -1
                                   ACPI_TYPE_INTEGER, &obj) &&
            obj->integer.value == 1)
                return true;
 
-       return false;
+       /* Assume D3 support if the bridge is power-manageable by ACPI. */
+       return acpi_pci_power_manageable(dev);
 }
 
 int acpi_pci_set_power_state(struct pci_dev *dev, pci_power_t state)

Note that the adev->wakeup.flags.valid is still necessary in the cases when _S0W is not present, because in those cases wakeup support implies that it is supported in all D-states. It is sort of redundant when _S0W is present, but the current code has it and this particular patch doesn't (or even shouldn't) change that.

#14 lore raw

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

Quoted 97 lines · Rafael J. Wysocki
Quoted 96 lines · Bjorn Helgaas
Quoted 94 lines · Rafael J. Wysocki
Quoted 71 lines · Bjorn Helgaas
Quoted 48 lines · Rafael J. Wysocki
Quoted 29 lines · Bjorn Helgaas
Quoted 28 lines · Rafael J. Wysocki
Quoted 26 lines · Bjorn Helgaas
Quoted 12 lines · Mario
Quoted 11 lines · Bjorn Helgaas
Quoted 10 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

Quoted 5 lines
Quoted 2 lines

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Quoted 45 lines

_S0W may only cause acpi_pci_bridge_d3() to return false, it is not sufficient for true to be returned.

Quoted 2 lines

I guess in this case we assume wakeup would
be via native PME/hotplug signaling?

This may be taken into consideration at one point, if need be, but not in this particular patch IMO.

Quoted 36 lines
Quoted 16 lines

Now, for compatibility with systems where ports have neither _S0W nor the HotPlugSupportInD3 property, the acpi_pci_power_manageable() return value should determine the outcome regardless of the adev->wakeup.flags.valid value, so the latter should only determine whether or not the HotPlugSupportInD3 property will be inspected (which may cause true to be returned before the "power manageable" check).

IOW, something like this (after checking _S0W):

if (adev->wakeup.flags.valid &&
!acpi_dev_get_property(adev, "HotPlugSupportInD3",
ACPI_TYPE_INTEGER, &obj) &&

    obj->integer.value == 1)
        return true;
Quoted 13 lines

return acpi_pci_power_manageable(dev);

Where the if () condition basically means that wakeup signaling is supported (and there is no indication that it cannot be done from D3 as per the previous _S0W check) and hotplug signaling from D3 is supported.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

This patch says that if dev's Root Port has "HotPlugSupportInD3", we don't need _PS0 or _PR0 for dev. I guess that must be true, because previously the fact that we checked for "HotPlugSupportInD3" meant the device did NOT have _PS0 or _PR0.

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-pcie-root-ports#identifying-pcie-root-ports-supporting-hot-plug-in-d3

I think you're suggesting the patch below, which will make acpi_pci_bridge_d3(dev) return "true" if:

- Root Port can wake from D3hot or D3cold, has "HotPlugSupportInD3", and has wakeup.flags.valid, OR

Quoted 6 lines
Quoted 2 lines

- Root Port can wake from D3hot or D3cold, and "dev" has _PR0 or

    _PS0

Well, not exactly. The second point should be

- Root Port's ACPI companion ('dev') has _PR0 or _PS0.

With the patch below, the RP _S0W must either fail or return D3hot or D3cold (this is what I meant by "RP can wake from D3hot or D3cold") before we even look for _PR0/_PS0.

Maybe we're talking about two different things -- you suggest we should check whether the *Root Port* has _PR0 or _PS0, but the current code checks the bridge "dev", which might be *below* the Root Port IIUC.

Quoted 8 lines
Quoted 3 lines

Previously, all bridges with _PR0 or _PS0 could use D3; now we also require that the Root Port's _S0W says it can wake from at least D3hot.

Yes, if _S0W is present and it evaluates successfully, then it is required to confirm that wakeup signaling from at least D3hot is supported for 'true' to be returned (but it is not sufficient for that by itself).

Hmm. In the case where _S0W is present and says at least D3hot but wakeup.flags is not valid, the patch below returns 'true' if "dev" has _PR0 or _PS0.

Quoted 59 lines

That's the only functional change made by that patch and yes, the patch below is what I mean.

Quoted 56 lines
drivers/pci/pci-acpi.c +10 -17
index a46fec776ad7..66c9ae1dc5da 100644
@@ -984,10 +984,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
        if (acpi_pci_disabled || !dev->is_hotplug_bridge)
                return false;
 
-       /* Assume D3 support if the bridge is power-manageable by ACPI. */
-       if (acpi_pci_power_manageable(dev))
-               return true;
-
        rpdev = pcie_find_root_port(dev);
        if (!rpdev)
                return false;
@@ -996,14 +992,6 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
        if (!adev)
                return false;
 
-       /*
-        * If the Root Port cannot signal wakeup signals at all, i.e., it
-        * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
-        * events from low-power states including D3hot and D3cold.
-        */
-       if (!adev->wakeup.flags.valid)
-               return false;
-
        /*
         * If the Root Port cannot wake itself from D3hot or D3cold, we
         * can't use D3.
@@ -1014,16 +1002,21 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 
        /*
         * The "HotPlugSupportInD3" property in a Root Port _DSD indicates
-        * the Port can signal hotplug events while in D3.  We assume any
-        * bridges *below* that Root Port can also signal hotplug events
-        * while in D3.
+        * the Port can signal hotplug events while in D3.  This differs
+        * from _S0W because _S0W may rely on GPEs, which can only be used
+        * directly for wake signaling, not hotplug events.
+        *
+        * We assume any bridges *below* that Root Port can also signal
+        * hotplug events while in D3.
         */
-       if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+       if (adev->wakeup.flags.valid &&
+           !acpi_dev_get_property(adev, "HotPlugSupportInD3",
                                   ACPI_TYPE_INTEGER, &obj) &&
            obj->integer.value == 1)
                return true;
 
-       return false;
+       /* Assume D3 support if the bridge is power-manageable by ACPI. */
+       return acpi_pci_power_manageable(dev);
 }
 
Quoted 7 lines

drivers/pci/pci-acpi.c bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1022

Quoted 1 line
drivers/pci/pci-acpi.c +0 -0
 int acpi_pci_set_power_state(struct pci_dev *dev, pci_power_t state)

Note that the adev->wakeup.flags.valid is still necessary in the cases when _S0W is not present, because in those cases wakeup support implies that it is supported in all D-states. It is sort of redundant when _S0W is present, but the current code has it and this particular patch doesn't (or even shouldn't) change that.

In the current patch, wakeup.flags is only relevant if the RP has "HotPlugSupportInD3". If "dev" has _PR0 or _PS0, we'll return 'true' even if wakeup.flags is not valid. Maybe that's wrong?

Bjorn

#15 lore raw
Quoted 4 lines · Bjorn Helgaas

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

No worries, doing my best to address that.

Quoted 101 lines · Rafael J. Wysocki
Quoted 100 lines · Bjorn Helgaas
Quoted 98 lines · Rafael J. Wysocki
Quoted 90 lines · Bjorn Helgaas
Quoted 71 lines · Rafael J. Wysocki
Quoted 48 lines · Bjorn Helgaas
Quoted 29 lines · Rafael J. Wysocki
Quoted 28 lines · Bjorn Helgaas
Quoted 26 lines · Mario
Quoted 12 lines · Bjorn Helgaas
Quoted 11 lines · Mario Limonciello
Quoted 10 lines

Firmware typically advertises that ACPI devices that represent PCIe devices can support D3 by a combination of the value returned by _S0W as well as the HotPlugSupportInD3 _DSD [1].

acpi_pci_bridge_d3 looks for this combination but also contains an assumption that if an ACPI device contains power resources the PCIe device it's associated with can support D3. This was introduced from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support wake from D3 in _S0W for the ACPI device representing the PCIe root port used for tunneling. The PCIe device will still be opted into runtime PM in the kernel [2] because of the logic within acpi_pci_bridge_d3. This currently happens because the ACPI device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

Quoted 6 lines
Quoted 2 lines

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Yes, the current code will return 'true' if _PR0 or _PS0 is present for dev regardless of anything else.

The proposed change is to make that conditional on whether or not _S0W for the root port says that wakeup from D3 is supported (or it is not present or unusable).

I see that I've missed one point now which is when the root port doesn't have an ACPI companion, in which case we should go straight for the "dev is power manageable" check. Let me redo the patch to address this.

#16 lore raw
Quoted 119 lines · CET Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

No worries, doing my best to address that.

Quoted 101 lines · Rafael J. Wysocki
Quoted 100 lines · Bjorn Helgaas
Quoted 98 lines · Rafael J. Wysocki
Quoted 90 lines · Bjorn Helgaas
Quoted 71 lines · Rafael J. Wysocki
Quoted 48 lines · Bjorn Helgaas
Quoted 29 lines · Rafael J. Wysocki
Quoted 28 lines · Bjorn Helgaas
Quoted 26 lines · Mario
Quoted 12 lines · Bjorn Helgaas
Quoted 11 lines · Mario Limonciello
Quoted 10 lines

Firmware typically advertises that ACPI devices that represent PCIe
devices can support D3 by a combination of the value returned by
_S0W as well as the HotPlugSupportInD3 _DSD [1].

`acpi_pci_bridge_d3` looks for this combination but also contains
an assumption that if an ACPI device contains power resources the PCIe
device it's associated with can support D3. This was introduced
from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for
D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support
wake from D3 in _S0W for the ACPI device representing the PCIe root
port used for tunneling. The PCIe device will still be opted into
runtime PM in the kernel [2] because of the logic within
`acpi_pci_bridge_d3`. This currently happens because the ACPI
device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

Quoted 6 lines
Quoted 2 lines

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Yes, the current code will return 'true' if _PR0 or _PS0 is present for dev regardless of anything else.

Quoted 7 lines

The proposed change is to make that conditional on whether or not _S0W for the root port says that wakeup from D3 is supported (or it is not present or unusable).

I see that I've missed one point now which is when the root port doesn't have an ACPI companion, in which case we should go straight for the "dev is power manageable" check.

Moreover, it is possible that the bridge passed to acpi_pci_bridge_d3() has its own _S0W or a wakeup GPE if it is power-manageable via ACPI. In those cases it is not necessary to ask the Root Port's _S0W about wakeup from D3, so overall I would go for the patch like the below (not really tested).

This works in the same way as the current code (unless I have missed anything) except for the case when the "target" bridge is power-manageable via ACPI, but it cannot signal wakeup via ACPI and has no _S0W. In that case it will consult the upstream Root Port's _S0W to check whether or not wakeup from D3 is supported.

[Note that if dev_has_acpi_pm is 'true', it is kind of pointless to look for the "HotPlugSupportInD3" property of the Root Port, because the function is going to return 'true' regardless, but I'm not sure if adding an extra if () for handling this particular case is worth the hassle.]

---
 drivers/pci/pci-acpi.c |   32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)
drivers/pci/pci-acpi.c +25 -7
Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================
@@ -975,6 +975,7 @@ bool acpi_pci_power_manageable(struct pc
 
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
+	bool dev_has_acpi_pm = false;
 	struct pci_dev *rpdev;
 	struct acpi_device *adev;
 	acpi_status status;
@@ -984,17 +985,34 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev && acpi_device_power_manageable(adev)) {
+		/*
+		 * Let the bridge go into D3 if it can signal wakeup from these
+		 * states (i.e. it has _S0W which says so or it can signal
+		 * wakeup via ACPI).
+		 */
+		status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+		if (ACPI_SUCCESS(status)) {
+			if (state >= ACPI_STATE_D3_HOT)
+				return true;
+		} else if (adev->wakeup.flags.valid) {
+			return true;
+		}
+		/*
+		 * If the bridge is power-manageable by ACPI, let it go into D3
+		 * by default.
+		 */
+		dev_has_acpi_pm = true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
-		return false;
+		return dev_has_acpi_pm;
 
 	adev = ACPI_COMPANION(&rpdev->dev);
 	if (!adev)
-		return false;
+		return dev_has_acpi_pm;
 
 	/*
 	 * If the Root Port cannot signal wakeup signals at all, i.e., it
@@ -1002,7 +1020,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * events from low-power states including D3hot and D3cold.
 	 */
 	if (!adev->wakeup.flags.valid)
-		return false;
+		return dev_has_acpi_pm;
 
 	/*
 	 * If the Root Port cannot wake itself from D3hot or D3cold, we
@@ -1023,7 +1041,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	    obj->integer.value == 1)
 		return true;
 
-	return false;
+	return dev_has_acpi_pm;
 }
 
 int acpi_pci_set_power_state(struct pci_dev *dev, pci_power_t state)
#17 lore raw
Quoted 139 lines · Rafael J. Wysocki
Quoted 119 lines · CET Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

No worries, doing my best to address that.

Quoted 101 lines · Rafael J. Wysocki
Quoted 100 lines · Bjorn Helgaas
Quoted 98 lines · Rafael J. Wysocki
Quoted 90 lines · Bjorn Helgaas
Quoted 71 lines · Rafael J. Wysocki
Quoted 48 lines · Bjorn Helgaas
Quoted 29 lines · Rafael J. Wysocki
Quoted 28 lines · Bjorn Helgaas
Quoted 26 lines · Mario
Quoted 12 lines · Bjorn Helgaas
Quoted 11 lines · Mario Limonciello

Firmware typically advertises that ACPI devices that represent PCIe
devices can support D3 by a combination of the value returned by
_S0W as well as the HotPlugSupportInD3 _DSD [1].

`acpi_pci_bridge_d3` looks for this combination but also contains
an assumption that if an ACPI device contains power resources the PCIe
device it's associated with can support D3. This was introduced
from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for
D3 if power managed by ACPI").

Quoted 6 lines
Quoted 6 lines

Some firmware configurations for "AMD Pink Sardine" do not support
wake from D3 in _S0W for the ACPI device representing the PCIe root
port used for tunneling. The PCIe device will still be opted into
runtime PM in the kernel [2] because of the logic within
`acpi_pci_bridge_d3`. This currently happens because the ACPI
device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

Quoted 6 lines
Quoted 2 lines

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Yes, the current code will return 'true' if _PR0 or _PS0 is present for dev regardless of anything else.

Quoted 7 lines

The proposed change is to make that conditional on whether or not _S0W for the root port says that wakeup from D3 is supported (or it is not present or unusable).

I see that I've missed one point now which is when the root port doesn't have an ACPI companion, in which case we should go straight for the "dev is power manageable" check.

Moreover, it is possible that the bridge passed to acpi_pci_bridge_d3() has its own _S0W or a wakeup GPE if it is power-manageable via ACPI. In those cases it is not necessary to ask the Root Port's _S0W about wakeup from D3, so overall I would go for the patch like the below (not really tested).

This works in the same way as the current code (unless I have missed anything) except for the case when the "target" bridge is power-manageable via ACPI, but it cannot signal wakeup via ACPI and has no _S0W. In that case it will consult the upstream Root Port's _S0W to check whether or not wakeup from D3 is supported.

Quoted 4 lines

[Note that if dev_has_acpi_pm is 'true', it is kind of pointless to look for the "HotPlugSupportInD3" property of the Root Port, because the function is going to return 'true' regardless, but I'm not sure if adding an extra if () for handling this particular case is worth the hassle.]

I think this has a lot of potential. I haven't tried it, but I wonder if splitting out the Root Port-specific parts to a separate function would be helpful, if only to make it more obvious that there may be two different devices involved.

If there are two devices ("dev" is a bridge below a Root Port), I guess support in the Root Port is not necessarily required? E.g., could "dev" assert a wakeup GPE that's not routed through the Root Port? If Root Port support *is* required, maybe it would read more clearly to test that first, before looking at the downstream device.

Quoted 77 lines
---
 drivers/pci/pci-acpi.c |   32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)

Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================

drivers/pci/pci-acpi.c +25 -7
@@ -975,6 +975,7 @@ bool acpi_pci_power_manageable(struct pc
 
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
+	bool dev_has_acpi_pm = false;
 	struct pci_dev *rpdev;
 	struct acpi_device *adev;
 	acpi_status status;
@@ -984,17 +985,34 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev && acpi_device_power_manageable(adev)) {
+		/*
+		 * Let the bridge go into D3 if it can signal wakeup from these
+		 * states (i.e. it has _S0W which says so or it can signal
+		 * wakeup via ACPI).
+		 */
+		status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+		if (ACPI_SUCCESS(status)) {
+			if (state >= ACPI_STATE_D3_HOT)
+				return true;
+		} else if (adev->wakeup.flags.valid) {
+			return true;
+		}
+		/*
+		 * If the bridge is power-manageable by ACPI, let it go into D3
+		 * by default.
+		 */
+		dev_has_acpi_pm = true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
-		return false;
+		return dev_has_acpi_pm;
 
 	adev = ACPI_COMPANION(&rpdev->dev);
 	if (!adev)
-		return false;
+		return dev_has_acpi_pm;
 
 	/*
 	 * If the Root Port cannot signal wakeup signals at all, i.e., it
@@ -1002,7 +1020,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * events from low-power states including D3hot and D3cold.
 	 */
 	if (!adev->wakeup.flags.valid)
-		return false;
+		return dev_has_acpi_pm;
 
 	/*
 	 * If the Root Port cannot wake itself from D3hot or D3cold, we
@@ -1023,7 +1041,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	    obj->integer.value == 1)
 		return true;
 
-	return false;
+	return dev_has_acpi_pm;
 }
 
 int acpi_pci_set_power_state(struct pci_dev *dev, pci_power_t state)
#18 lore raw
Quoted 150 lines · CET Bjorn Helgaas
Quoted 139 lines · Rafael J. Wysocki
Quoted 119 lines · CET Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

No worries, doing my best to address that.

Quoted 101 lines · Rafael J. Wysocki
Quoted 100 lines · Bjorn Helgaas
Quoted 98 lines · Rafael J. Wysocki
Quoted 90 lines · Bjorn Helgaas
Quoted 71 lines · Rafael J. Wysocki
Quoted 48 lines · Bjorn Helgaas
Quoted 29 lines · Rafael J. Wysocki
Quoted 28 lines · Bjorn Helgaas
Quoted 26 lines · Mario
Quoted 18 lines · Bjorn Helgaas

On Mon, Oct 31, 2022 at 05:33:55PM -0500, Mario Limonciello wrote:
Firmware typically advertises that ACPI devices that represent PCIe
devices can support D3 by a combination of the value returned by
_S0W as well as the HotPlugSupportInD3 _DSD [1].

`acpi_pci_bridge_d3` looks for this combination but also contains
an assumption that if an ACPI device contains power resources the PCIe
device it's associated with can support D3. This was introduced
from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for
D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support
wake from D3 in _S0W for the ACPI device representing the PCIe root
port used for tunneling. The PCIe device will still be opted into
runtime PM in the kernel [2] because of the logic within
`acpi_pci_bridge_d3`. This currently happens because the ACPI
device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't mean the device can wake itself from a low-power state. The OS has to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

Quoted 6 lines
Quoted 2 lines

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Yes, the current code will return 'true' if _PR0 or _PS0 is present for dev regardless of anything else.

Quoted 7 lines

The proposed change is to make that conditional on whether or not _S0W for the root port says that wakeup from D3 is supported (or it is not present or unusable).

I see that I've missed one point now which is when the root port doesn't have an ACPI companion, in which case we should go straight for the "dev is power manageable" check.

Moreover, it is possible that the bridge passed to acpi_pci_bridge_d3() has its own _S0W or a wakeup GPE if it is power-manageable via ACPI. In those cases it is not necessary to ask the Root Port's _S0W about wakeup from D3, so overall I would go for the patch like the below (not really tested).

This works in the same way as the current code (unless I have missed anything) except for the case when the "target" bridge is power-manageable via ACPI, but it cannot signal wakeup via ACPI and has no _S0W. In that case it will consult the upstream Root Port's _S0W to check whether or not wakeup from D3 is supported.

Quoted 4 lines

[Note that if dev_has_acpi_pm is 'true', it is kind of pointless to look for the "HotPlugSupportInD3" property of the Root Port, because the function is going to return 'true' regardless, but I'm not sure if adding an extra if () for handling this particular case is worth the hassle.]

I think this has a lot of potential. I haven't tried it, but I wonder if splitting out the Root Port-specific parts to a separate function would be helpful, if only to make it more obvious that there may be two different devices involved.

Quoted 5 lines

If there are two devices ("dev" is a bridge below a Root Port), I guess support in the Root Port is not necessarily required? E.g., could "dev" assert a wakeup GPE that's not routed through the Root Port? If Root Port support *is* required, maybe it would read more clearly to test that first, before looking at the downstream device.

Sorry for the delay.

I don't really think that Root Port support is required for a bridge below a Root Port if that bridge itself is power-manageable via ACPI. Moreover, I don't think that the _S0W of a Root Port has any bearing on devices below it that have their own _S0W.

So what we really want appears to be to evaluate _S0W for the target bridge, regardless of whether or not it is a Root Port, and return 'false' if that produces D2 or a shallower power state. Otherwise, we can do what we've done so far.

The patch below implements, this - please let me know what you think.

---
From: Rafael J. Wysocki <[email protected]>
Subject: [PATCH] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

---
 drivers/pci/pci-acpi.c |   39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)
drivers/pci/pci-acpi.c +29 -10
Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================
@@ -984,15 +984,33 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+		if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
+	if (rpdev != dev)
+		adev = ACPI_COMPANION(&rpdev->dev);
+
 	if (!adev)
 		return false;
 
@@ -1005,13 +1023,14 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
-		return false;
-
+	if (rpdev != dev) {
+		status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+		if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+			return false;
+	}
 	/*
 	 * The "HotPlugSupportInD3" property in a Root Port _DSD indicates
 	 * the Port can signal hotplug events while in D3.  We assume any
#19 lore raw
Quoted 164 lines · CET Rafael J. Wysocki
Quoted 150 lines · CET Bjorn Helgaas
Quoted 139 lines · Rafael J. Wysocki
Quoted 119 lines · CET Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

No worries, doing my best to address that.

Quoted 101 lines · Rafael J. Wysocki
Quoted 100 lines · Bjorn Helgaas
Quoted 98 lines · Rafael J. Wysocki
Quoted 90 lines · Bjorn Helgaas
Quoted 71 lines · Rafael J. Wysocki
Quoted 48 lines · Bjorn Helgaas
Quoted 29 lines · Rafael J. Wysocki
Quoted 28 lines · Bjorn Helgaas
Quoted 26 lines · Mario

On 11/11/2022 11:41, Bjorn Helgaas wrote:
On Mon, Oct 31, 2022 at 05:33:55PM -0500, Mario Limonciello wrote:
Firmware typically advertises that ACPI devices that represent PCIe
devices can support D3 by a combination of the value returned by
_S0W as well as the HotPlugSupportInD3 _DSD [1].

`acpi_pci_bridge_d3` looks for this combination but also contains
an assumption that if an ACPI device contains power resources the PCIe
device it's associated with can support D3. This was introduced
from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for
D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support
wake from D3 in _S0W for the ACPI device representing the PCIe root
port used for tunneling. The PCIe device will still be opted into
runtime PM in the kernel [2] because of the logic within
`acpi_pci_bridge_d3`. This currently happens because the ACPI
device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't
mean the device can wake itself from a low-power state. The OS has
to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

Quoted 6 lines
Quoted 2 lines

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Yes, the current code will return 'true' if _PR0 or _PS0 is present for dev regardless of anything else.

Quoted 7 lines

The proposed change is to make that conditional on whether or not _S0W for the root port says that wakeup from D3 is supported (or it is not present or unusable).

I see that I've missed one point now which is when the root port doesn't have an ACPI companion, in which case we should go straight for the "dev is power manageable" check.

Moreover, it is possible that the bridge passed to acpi_pci_bridge_d3() has its own _S0W or a wakeup GPE if it is power-manageable via ACPI. In those cases it is not necessary to ask the Root Port's _S0W about wakeup from D3, so overall I would go for the patch like the below (not really tested).

This works in the same way as the current code (unless I have missed anything) except for the case when the "target" bridge is power-manageable via ACPI, but it cannot signal wakeup via ACPI and has no _S0W. In that case it will consult the upstream Root Port's _S0W to check whether or not wakeup from D3 is supported.

Quoted 4 lines

[Note that if dev_has_acpi_pm is 'true', it is kind of pointless to look for the "HotPlugSupportInD3" property of the Root Port, because the function is going to return 'true' regardless, but I'm not sure if adding an extra if () for handling this particular case is worth the hassle.]

I think this has a lot of potential. I haven't tried it, but I wonder if splitting out the Root Port-specific parts to a separate function would be helpful, if only to make it more obvious that there may be two different devices involved.

Quoted 5 lines

If there are two devices ("dev" is a bridge below a Root Port), I guess support in the Root Port is not necessarily required? E.g., could "dev" assert a wakeup GPE that's not routed through the Root Port? If Root Port support *is* required, maybe it would read more clearly to test that first, before looking at the downstream device.

Sorry for the delay.

I don't really think that Root Port support is required for a bridge below a Root Port if that bridge itself is power-manageable via ACPI. Moreover, I don't think that the _S0W of a Root Port has any bearing on devices below it that have their own _S0W.

Quoted 7 lines

So what we really want appears to be to evaluate _S0W for the target bridge, regardless of whether or not it is a Root Port, and return 'false' if that produces D2 or a shallower power state. Otherwise, we can do what we've done so far.

The patch below implements, this - please let me know what you think.

And here's a v2 with somewhat less code duplication.

---
From: Rafael J. Wysocki <[email protected]>
Subject: [PATCH] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

---
 drivers/acpi/device_pm.c |   16 ++++++++++++++++
 drivers/pci/pci-acpi.c   |   34 ++++++++++++++++++++++++----------
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 41 insertions(+), 10 deletions(-)
drivers/pci/pci-acpi.c +24 -10
Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================
@@ -977,22 +977,37 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 {
 	struct pci_dev *rpdev;
 	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_no_wakeup_from_d3(adev))
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
+	if (rpdev != dev)
+		adev = ACPI_COMPANION(&rpdev->dev);
+
 	if (!adev)
 		return false;
 
@@ -1005,11 +1020,10 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpdev != dev && acpi_dev_no_wakeup_from_d3(adev))
 		return false;
 
 	/*
drivers/acpi/device_pm.c +16 -0
Index: linux-pm/drivers/acpi/device_pm.c
===================================================================
@@ -484,6 +484,22 @@ void acpi_dev_power_up_children_with_adr
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
 }
 
+/**
+ * acpi_dev_no_wakeup_from_d3 - Check if wakeup signaling from D3 is supported
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return 'true' if it is successful and the power
+ * state returned by it is D2 or shallower.
+ */
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	return ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);
include/acpi/acpi_bus.h +1 -0
Index: linux-pm/include/acpi/acpi_bus.h
===================================================================
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
author Reviewed-by
#20 lore raw
Quoted 165 lines · Rafael J. Wysocki
Quoted 164 lines · CET Rafael J. Wysocki
Quoted 150 lines · CET Bjorn Helgaas
Quoted 139 lines · Rafael J. Wysocki
Quoted 119 lines · CET Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

No worries, doing my best to address that.

Quoted 101 lines · Rafael J. Wysocki
Quoted 100 lines · Bjorn Helgaas
Quoted 98 lines · Rafael J. Wysocki
Quoted 90 lines · Bjorn Helgaas
Quoted 71 lines · Rafael J. Wysocki
Quoted 48 lines · Bjorn Helgaas
Quoted 29 lines · Rafael J. Wysocki
Quoted 28 lines · Bjorn Helgaas

On Fri, Nov 11, 2022 at 12:58:28PM -0600, Limonciello, Mario wrote:
On 11/11/2022 11:41, Bjorn Helgaas wrote:
On Mon, Oct 31, 2022 at 05:33:55PM -0500, Mario Limonciello wrote:
Firmware typically advertises that ACPI devices that represent PCIe
devices can support D3 by a combination of the value returned by
_S0W as well as the HotPlugSupportInD3 _DSD [1].

`acpi_pci_bridge_d3` looks for this combination but also contains
an assumption that if an ACPI device contains power resources the PCIe
device it's associated with can support D3. This was introduced
from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for
D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support
wake from D3 in _S0W for the ACPI device representing the PCIe root
port used for tunneling. The PCIe device will still be opted into
runtime PM in the kernel [2] because of the logic within
`acpi_pci_bridge_d3`. This currently happens because the ACPI
device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't
mean the device can wake itself from a low-power state. The OS has
to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

Quoted 6 lines
Quoted 2 lines

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Yes, the current code will return 'true' if _PR0 or _PS0 is present for dev regardless of anything else.

Quoted 7 lines

The proposed change is to make that conditional on whether or not _S0W for the root port says that wakeup from D3 is supported (or it is not present or unusable).

I see that I've missed one point now which is when the root port doesn't have an ACPI companion, in which case we should go straight for the "dev is power manageable" check.

Moreover, it is possible that the bridge passed to acpi_pci_bridge_d3() has its own _S0W or a wakeup GPE if it is power-manageable via ACPI. In those cases it is not necessary to ask the Root Port's _S0W about wakeup from D3, so overall I would go for the patch like the below (not really tested).

This works in the same way as the current code (unless I have missed anything) except for the case when the "target" bridge is power-manageable via ACPI, but it cannot signal wakeup via ACPI and has no _S0W. In that case it will consult the upstream Root Port's _S0W to check whether or not wakeup from D3 is supported.

Quoted 4 lines

[Note that if dev_has_acpi_pm is 'true', it is kind of pointless to look for the "HotPlugSupportInD3" property of the Root Port, because the function is going to return 'true' regardless, but I'm not sure if adding an extra if () for handling this particular case is worth the hassle.]

I think this has a lot of potential. I haven't tried it, but I wonder if splitting out the Root Port-specific parts to a separate function would be helpful, if only to make it more obvious that there may be two different devices involved.

Quoted 5 lines

If there are two devices ("dev" is a bridge below a Root Port), I guess support in the Root Port is not necessarily required? E.g., could "dev" assert a wakeup GPE that's not routed through the Root Port? If Root Port support *is* required, maybe it would read more clearly to test that first, before looking at the downstream device.

Sorry for the delay.

I don't really think that Root Port support is required for a bridge below a Root Port if that bridge itself is power-manageable via ACPI. Moreover, I don't think that the _S0W of a Root Port has any bearing on devices below it that have their own _S0W.

Quoted 6 lines
Quoted 6 lines

So what we really want appears to be to evaluate _S0W for the target bridge, regardless of whether or not it is a Root Port, and return 'false' if that produces D2 or a shallower power state. Otherwise, we can do what we've done so far.

The patch below implements, this - please let me know what you think.

This looks good to me, thanks.

Quoted 17 lines
Quoted 1 line

And here's a v2 with somewhat less code duplication.

---
From: Rafael J. Wysocki <[email protected]>
Subject: [PATCH] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

Quoted 5 lines

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

Quoted 111 lines
---
  drivers/acpi/device_pm.c |   16 ++++++++++++++++
  drivers/pci/pci-acpi.c   |   34 ++++++++++++++++++++++++----------
  include/acpi/acpi_bus.h  |    1 +
  3 files changed, 41 insertions(+), 10 deletions(-)

Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================

drivers/pci/pci-acpi.c +24 -10
@@ -977,22 +977,37 @@ bool acpi_pci_bridge_d3(struct pci_dev *
  {
  	struct pci_dev *rpdev;
  	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
  	const union acpi_object *obj;
 
  	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
  		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_no_wakeup_from_d3(adev))
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
  	rpdev = pcie_find_root_port(dev);
  	if (!rpdev)
  		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
+	if (rpdev != dev)
+		adev = ACPI_COMPANION(&rpdev->dev);
+
  	if (!adev)
  		return false;
 
@@ -1005,11 +1020,10 @@ bool acpi_pci_bridge_d3(struct pci_dev *
  		return false;
 
  	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
  	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpdev != dev && acpi_dev_no_wakeup_from_d3(adev))
  		return false;
 
  	/*

Index: linux-pm/drivers/acpi/device_pm.c
===================================================================

drivers/acpi/device_pm.c +16 -0
@@ -484,6 +484,22 @@ void acpi_dev_power_up_children_with_adr
  	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
  }
 
+/**
+ * acpi_dev_no_wakeup_from_d3 - Check if wakeup signaling from D3 is supported
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return 'true' if it is successful and the power
+ * state returned by it is D2 or shallower.
+ */
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	return ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT;
+}
+
  #ifdef CONFIG_PM
  static DEFINE_MUTEX(acpi_pm_notifier_lock);
  static DEFINE_MUTEX(acpi_pm_notifier_install_lock);

Index: linux-pm/include/acpi/acpi_bus.h
===================================================================

include/acpi/acpi_bus.h +1 -0
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
  int acpi_device_update_power(struct acpi_device *device, int *state_p);
  bool acpi_bus_power_manageable(acpi_handle handle);
  void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev);
  int acpi_device_power_add_dependent(struct acpi_device *adev,
  				    struct device *dev);
  void acpi_device_power_remove_dependent(struct acpi_device *adev,
#21 lore raw
In reply to Rafael J. Wysocki
Quoted 170 lines · CET Rafael J. Wysocki
Quoted 169 lines · CET Rafael J. Wysocki
Quoted 150 lines · CET Bjorn Helgaas
Quoted 139 lines · Rafael J. Wysocki
Quoted 119 lines · CET Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas

Hi Rafael,

Sorry, I'm still confused (my perpetual state :)).

No worries, doing my best to address that.

Quoted 101 lines · Rafael J. Wysocki
Quoted 100 lines · Bjorn Helgaas
Quoted 98 lines · Rafael J. Wysocki
Quoted 90 lines · Bjorn Helgaas
Quoted 71 lines · Rafael J. Wysocki
Quoted 48 lines · Bjorn Helgaas
Quoted 29 lines · Rafael J. Wysocki
Quoted 28 lines · Bjorn Helgaas

On Fri, Nov 11, 2022 at 12:58:28PM -0600, Limonciello, Mario wrote:
On 11/11/2022 11:41, Bjorn Helgaas wrote:
On Mon, Oct 31, 2022 at 05:33:55PM -0500, Mario Limonciello wrote:
Firmware typically advertises that ACPI devices that represent PCIe
devices can support D3 by a combination of the value returned by
_S0W as well as the HotPlugSupportInD3 _DSD [1].

`acpi_pci_bridge_d3` looks for this combination but also contains
an assumption that if an ACPI device contains power resources the PCIe
device it's associated with can support D3. This was introduced
from commit c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for
D3 if power managed by ACPI").

Some firmware configurations for "AMD Pink Sardine" do not support
wake from D3 in _S0W for the ACPI device representing the PCIe root
port used for tunneling. The PCIe device will still be opted into
runtime PM in the kernel [2] because of the logic within
`acpi_pci_bridge_d3`. This currently happens because the ACPI
device contains power resources.

Wait. Is this as simple as just recognizing that:

_PS0 means the OS has a knob to put the device in D0, but it doesn't
mean the device can wake itself from a low-power state. The OS has
to use _S0W to learn the device's ability to wake itself.

Quoted 7 lines
Quoted 1 line

It is.

Now I'm confused again about what "HotPlugSupportInD3" means. The MS web page [1] says it identifies Root Ports capable of handling hot plug events while in D3. That sounds kind of related to _S0W: If _S0W says "I can wake myself from D3hot and D3cold", how is that different from "I can handle hotplug events in D3"?

For native PME/hot-plug signaling there is no difference. This is the same interrupt by the spec after all IIRC.

For GPE-based signaling, though, there is a difference, because GPEs can only be used directly for wake signaling (this is related to _PRW). In particular, the only provision in the ACPI spec for device hot-add are the Bus Check and Device Check notification values (0 and
1) which require AML to run and evaluate Notify() on specific AML objects.

Quoted 3 lines

Hence, there is no spec-defined way to tell the OS that "something can be hot-added under this device while in D3 and you will get notified about that".

So I guess acpi_pci_bridge_d3() looks for:

- "wake signaling while in D3" (_S0W) and
- "notification of hotplug while in D3" ("HotPlugSupportInD3")

For Root Ports with both those abilities (or bridges below such Root Ports), we allow D3, and this patch doesn't change that.

What this patch *does* change is that all bridges with _PS0 or _PR0 previously could use D3, but now will only be able to use D3 if they are also (or are below) a Root Port that can signal wakeup (wakeup.flags.valid) and can wakeup from D3hot or D3cold (_S0W).

And this fixes the Pink Sardine because it has Root Ports that do Thunderbolt tunneling, and they have _PS0 or _PR0 but their _S0W says they cannot wake from D3. Previously we put those in D3, but they couldn't wake up. Now we won't put them in D3.

Quoted 4 lines

I guess there's a possibility that this could break or cause higher power consumption on systems that were fixed by c6e331312ebf ("PCI/ACPI: Whitelist hotplug ports for D3 if power managed by ACPI"). I don't know enough about that scenario. Maybe Lukas will chime in.

Well, it is possible that some of these systems will be affected.

One of such cases is when the port in question has _S0W which says that wakeup from D3 is not supported. In that case I think the kernel should honor the _S0W input, because there may be a good reason known to the platform integrator for it.

The other case is when wakeup.flags.valid is unset for the port's ACPI companion which means that the port cannot signal wakeup through ACPI-related means at all and this may be problematic, especially in the system-wide suspend case in which the wakeup capability is not too relevant unless there is a system wakeup device under the port.

Quoted 4 lines

I don't think that the adev->wakeup.flags.valid check has any bearing on the _S0W check - if there is _S0W and it says "no wakeup from D3", it should still be taken into account - so that check can be moved past the _S0W check.

So if _S0W says it can wake from D3, but wakeup.flags is not valid, it's still OK to use D3?

Quoted 6 lines
Quoted 2 lines

No, it isn't, as per the code today and I don't think that this particular part should be changed now.

But the current upstream code checks acpi_pci_power_manageable(dev) first, so if "dev" has _PR0 or _PS0, we'll use D3 even if _S0W says it can wake from D3 and wakeup.flags is not valid.

Yes, the current code will return 'true' if _PR0 or _PS0 is present for dev regardless of anything else.

Quoted 7 lines

The proposed change is to make that conditional on whether or not _S0W for the root port says that wakeup from D3 is supported (or it is not present or unusable).

I see that I've missed one point now which is when the root port doesn't have an ACPI companion, in which case we should go straight for the "dev is power manageable" check.

Moreover, it is possible that the bridge passed to acpi_pci_bridge_d3() has its own _S0W or a wakeup GPE if it is power-manageable via ACPI. In those cases it is not necessary to ask the Root Port's _S0W about wakeup from D3, so overall I would go for the patch like the below (not really tested).

This works in the same way as the current code (unless I have missed anything) except for the case when the "target" bridge is power-manageable via ACPI, but it cannot signal wakeup via ACPI and has no _S0W. In that case it will consult the upstream Root Port's _S0W to check whether or not wakeup from D3 is supported.

Quoted 4 lines

[Note that if dev_has_acpi_pm is 'true', it is kind of pointless to look for the "HotPlugSupportInD3" property of the Root Port, because the function is going to return 'true' regardless, but I'm not sure if adding an extra if () for handling this particular case is worth the hassle.]

I think this has a lot of potential. I haven't tried it, but I wonder if splitting out the Root Port-specific parts to a separate function would be helpful, if only to make it more obvious that there may be two different devices involved.

Quoted 5 lines

If there are two devices ("dev" is a bridge below a Root Port), I guess support in the Root Port is not necessarily required? E.g., could "dev" assert a wakeup GPE that's not routed through the Root Port? If Root Port support *is* required, maybe it would read more clearly to test that first, before looking at the downstream device.

Sorry for the delay.

I don't really think that Root Port support is required for a bridge below a Root Port if that bridge itself is power-manageable via ACPI. Moreover, I don't think that the _S0W of a Root Port has any bearing on devices below it that have their own _S0W.

So what we really want appears to be to evaluate _S0W for the target bridge, regardless of whether or not it is a Root Port, and return 'false' if that produces D2 or a shallower power state. Otherwise, we can do what we've done so far.

Quoted 4 lines
Quoted 2 lines

The patch below implements, this - please let me know what you think.

And here's a v2 with somewhat less code duplication.

I'm wondering if you have any comments on this one?

Quoted 130 lines

---
From: Rafael J. Wysocki <[email protected]>
Subject: [PATCH] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

---
 drivers/acpi/device_pm.c |   16 ++++++++++++++++
 drivers/pci/pci-acpi.c   |   34 ++++++++++++++++++++++++----------
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 41 insertions(+), 10 deletions(-)

Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================

drivers/pci/pci-acpi.c +24 -10
@@ -977,22 +977,37 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 {
 	struct pci_dev *rpdev;
 	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_no_wakeup_from_d3(adev))
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
+	if (rpdev != dev)
+		adev = ACPI_COMPANION(&rpdev->dev);
+
 	if (!adev)
 		return false;
 
@@ -1005,11 +1020,10 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpdev != dev && acpi_dev_no_wakeup_from_d3(adev))
 		return false;
 
 	/*

Index: linux-pm/drivers/acpi/device_pm.c
===================================================================

drivers/acpi/device_pm.c +16 -0
@@ -484,6 +484,22 @@ void acpi_dev_power_up_children_with_adr
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
 }
 
+/**
+ * acpi_dev_no_wakeup_from_d3 - Check if wakeup signaling from D3 is supported
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return 'true' if it is successful and the power
+ * state returned by it is D2 or shallower.
+ */
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	return ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);

Index: linux-pm/include/acpi/acpi_bus.h
===================================================================

include/acpi/acpi_bus.h +1 -0
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
#22 lore raw
In reply to Rafael J. Wysocki
Quoted 13 lines · Rafael J. Wysocki

On Monday, January 2, 2023 5:34:19 PM CET Rafael J. Wysocki wrote:
...

Quoted 9 lines

I don't really think that Root Port support is required for a bridge below a Root Port if that bridge itself is power-manageable via ACPI. Moreover, I don't think that the _S0W of a Root Port has any bearing on devices below it that have their own _S0W.

So what we really want appears to be to evaluate _S0W for the target bridge, regardless of whether or not it is a Root Port, and return 'false' if that produces D2 or a shallower power state. Otherwise, we can do what we've done so far.

Review

Quoted 7 lines
(quoted diff) +7 -0
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	return ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT;

This returns "false" (i.e., "yes, device can signal wakeup from D3") if _S0W doesn't exist. Is that right?

I think this might be less confusing as:

  bool acpi_dev_can_wake_from_d3(struct acpi_device *adev)
  {
    status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
    return ACPI_SUCCESS(status) && state >= ACPI_STATE_D3_HOT;
  }

That would look like this (including minor change to add "rp_adev" to make it more obviously that it's different from "adev"):

drivers/acpi/device_pm.c +16 -0
index 97450f4003cc..789a717d4819 100644
@@ -484,6 +484,22 @@ void acpi_dev_power_up_children_with_adr(struct acpi_device *adev)
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
 }
 
+/**
+ * acpi_dev_can_wake_from_d3 - Check if wakeup signaling from D3 is supported
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return 'true' if it is successful and the power
+ * state returned by it is D3hot or deeper.
+ */
+bool acpi_dev_can_wake_from_d3(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	return ACPI_SUCCESS(status) && state >= ACPI_STATE_D3_HOT;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);
drivers/pci/pci-acpi.c +26 -14
index 068d6745bf98..03dbb57047be 100644
@@ -976,24 +976,37 @@ bool acpi_pci_power_manageable(struct pci_dev *dev)
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	struct pci_dev *rpdev;
-	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
+	struct acpi_device *adev, *rp_adev;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (!acpi_dev_can_wake_from_d3(adev))
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
-	if (!adev)
+	rp_adev = (rpdev == dev) ? adev : ACPI_COMPANION(&rpdev->dev);
+	if (!rp_adev)
 		return false;
 
 	/*
@@ -1001,15 +1014,14 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	 * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
 	 * events from low-power states including D3hot and D3cold.
 	 */
-	if (!adev->wakeup.flags.valid)
+	if (!rp_adev->wakeup.flags.valid)
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rp_adev != adev && !acpi_dev_can_wake_from_d3(rp_adev))
 		return false;
 
 	/*
@@ -1018,7 +1030,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	 * bridges *below* that Root Port can also signal hotplug events
 	 * while in D3.
 	 */
-	if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+	if (!acpi_dev_get_property(rp_adev, "HotPlugSupportInD3",
 				   ACPI_TYPE_INTEGER, &obj) &&
 	    obj->integer.value == 1)
 		return true;
include/acpi/acpi_bus.h +1 -0
index cd3b75e08ec3..0d0c6aa367e0 100644
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle handle, int *state_p);
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+bool acpi_dev_can_wake_from_d3(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
#23 lore raw
Quoted 15 lines · Bjorn Helgaas
Quoted 12 lines · Rafael J. Wysocki

On Monday, January 2, 2023 5:34:19 PM CET Rafael J. Wysocki wrote:
...

Quoted 9 lines

I don't really think that Root Port support is required for a bridge below a Root Port if that bridge itself is power-manageable via ACPI. Moreover, I don't think that the _S0W of a Root Port has any bearing on devices below it that have their own _S0W.

So what we really want appears to be to evaluate _S0W for the target bridge, regardless of whether or not it is a Root Port, and return 'false' if that produces D2 or a shallower power state. Otherwise, we can do what we've done so far.

Quoted 10 lines

drivers/acpi/device_pm.c void acpi_dev_power_up_children_with_adr(struct acpi_device *adev)

Quoted 7 lines
drivers/acpi/device_pm.c +7 -0
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev)
+{
+     unsigned long long state;
+     acpi_status status;
+
+     status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+     return ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT;

This returns "false" (i.e., "yes, device can signal wakeup from D3") if _S0W doesn't exist. Is that right?

Yes, it is.

The reason why I did it that way was because the bridge cannot signal wakeup from D3 if both the following conditions take place:

1. There is _S0W and it can be evaluated successfully.
2. _S0W produces D2 or a shallower power state.

In particular, if 1 is not the case, it is still not clear whether or not the bridge can signal wakeup from D3 and additional checks are needed.

Quoted 7 lines

I think this might be less confusing as:

  bool acpi_dev_can_wake_from_d3(struct acpi_device *adev)
  {
    status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
    return ACPI_SUCCESS(status) && state >= ACPI_STATE_D3_HOT;
  }

So I don't think the above will work, because !acpi_dev_can_wake_from_d3(adev) is also true if _S0W is not present, for example, in which case acpi_pci_bridge_d3() should not return 'false' right away.

However, the additional function can simply return the value produced by _S0W or ACPI_STATE_UNKNOWN on all errors and its caller can do the checks as needed which is done here:

https://patchwork.kernel.org/project/linux-acpi/patch/5659681.DvuYhMxLoT@kreacher/

(posted as a proper patch, because I wanted patchwork to pick it up).

I've also picked up the idea of using rpadev for representing the ACPI companion of the Root Port in acpi_pci_bridge_d3().

Cheers!

#24 lore raw
Quoted 46 lines · Rafael J. Wysocki
Quoted 14 lines · Bjorn Helgaas
Quoted 12 lines · Rafael J. Wysocki

On Monday, January 2, 2023 5:34:19 PM CET Rafael J. Wysocki wrote:
...

Quoted 9 lines

I don't really think that Root Port support is required for a bridge below a Root Port if that bridge itself is power-manageable via ACPI. Moreover, I don't think that the _S0W of a Root Port has any bearing on devices below it that have their own _S0W.

So what we really want appears to be to evaluate _S0W for the target bridge, regardless of whether or not it is a Root Port, and return 'false' if that produces D2 or a shallower power state. Otherwise, we can do what we've done so far.

Quoted 10 lines

drivers/acpi/device_pm.c void acpi_dev_power_up_children_with_adr(struct acpi_device *adev)

Quoted 7 lines
drivers/acpi/device_pm.c +7 -0
+bool acpi_dev_no_wakeup_from_d3(struct acpi_device *adev)
+{
+     unsigned long long state;
+     acpi_status status;
+
+     status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+     return ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT;

This returns "false" (i.e., "yes, device can signal wakeup from D3") if _S0W doesn't exist. Is that right?

Yes, it is.

The reason why I did it that way was because the bridge cannot signal wakeup from D3 if both the following conditions take place:

1. There is _S0W and it can be evaluated successfully.
2. _S0W produces D2 or a shallower power state.

In particular, if 1 is not the case, it is still not clear whether or not the bridge can signal wakeup from D3 and additional checks are needed.

Quoted 7 lines

I think this might be less confusing as:

  bool acpi_dev_can_wake_from_d3(struct acpi_device *adev)
  {
    status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
    return ACPI_SUCCESS(status) && state >= ACPI_STATE_D3_HOT;
  }
Quoted 4 lines

So I don't think the above will work, because !acpi_dev_can_wake_from_d3(adev) is also true if _S0W is not present, for example, in which case acpi_pci_bridge_d3() should not return 'false' right away.

OK, that makes sense, thanks!

Quoted 12 lines

However, the additional function can simply return the value produced by _S0W or ACPI_STATE_UNKNOWN on all errors and its caller can do the checks as needed which is done here:

https://patchwork.kernel.org/project/linux-acpi/patch/5659681.DvuYhMxLoT@kreacher/

(posted as a proper patch, because I wanted patchwork to pick it up).

I've also picked up the idea of using rpadev for representing the ACPI companion of the Root Port in acpi_pci_bridge_d3().

Cheers!

#25 lore raw

PATCH v3 1/1 PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

In reply to Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

---

v2 -> v3:
* Use rpadev for the ACPI companion of the Root Port in acpi_pci_bridge_d3(() to avoid confusion.
* Make the function evaluating _S0W return the value produced by it or "unknown state" on errors and let its caller deal with that value.

---
 drivers/acpi/device_pm.c |   19 +++++++++++++++++++
 drivers/pci/pci-acpi.c   |   45 +++++++++++++++++++++++++++++++--------------
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 51 insertions(+), 14 deletions(-)
drivers/pci/pci-acpi.c +31 -14
Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================
@@ -976,24 +976,41 @@ bool acpi_pci_power_manageable(struct pc
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	struct pci_dev *rpdev;
-	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
+	struct acpi_device *adev, *rpadev;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_power_state_for_wake(adev) <= ACPI_STATE_D3_HOT)
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
-	if (!adev)
+	if (rpdev == dev)
+		rpadev = adev;
+	else
+		rpadev = ACPI_COMPANION(&rpdev->dev);
+
+	if (!rpadev)
 		return false;
 
 	/*
@@ -1001,15 +1018,15 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
 	 * events from low-power states including D3hot and D3cold.
 	 */
-	if (!adev->wakeup.flags.valid)
+	if (!rpadev->wakeup.flags.valid)
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpadev != adev &&
+	    acpi_dev_power_state_for_wake(rpadev) <= ACPI_STATE_D3_HOT)
 		return false;
 
 	/*
@@ -1018,7 +1035,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * bridges *below* that Root Port can also signal hotplug events
 	 * while in D3.
 	 */
-	if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+	if (!acpi_dev_get_property(rpadev, "HotPlugSupportInD3",
 				   ACPI_TYPE_INTEGER, &obj) &&
 	    obj->integer.value == 1)
 		return true;
drivers/acpi/device_pm.c +19 -0
Index: linux-pm/drivers/acpi/device_pm.c
===================================================================
@@ -484,6 +484,25 @@ void acpi_dev_power_up_children_with_adr
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
 }
 
+/**
+ * acpi_dev_power_state_for_wake - Deepest power state for wakeup signaling
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return the value produced by it or return
+ * ACPI_STATE_UNKNOWN on errors (including _S0W not present).
+ */
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	if (ACPI_FAILURE(status))
+		return ACPI_STATE_UNKNOWN;
+
+	return state;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);
include/acpi/acpi_bus.h +1 -0
Index: linux-pm/include/acpi/acpi_bus.h
===================================================================
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
#26 lore raw

Re: [PATCH v3] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 50 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

---

v2 -> v3:
* Use rpadev for the ACPI companion of the Root Port in acpi_pci_bridge_d3(() to avoid confusion.
* Make the function evaluating _S0W return the value produced by it or "unknown state" on errors and let its caller deal with that value.

---
 drivers/acpi/device_pm.c |   19 +++++++++++++++++++
 drivers/pci/pci-acpi.c   |   45 +++++++++++++++++++++++++++++++--------------
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 51 insertions(+), 14 deletions(-)

Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================

drivers/pci/pci-acpi.c +1 -5
@@ -976,24 +976,41 @@ bool acpi_pci_power_manageable(struct pc
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	struct pci_dev *rpdev;
-	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
+	struct acpi_device *adev, *rpadev;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))

drivers/pci/pci-acpi.c bool acpi_pci_power_manageable(struct pc · line 993

Quoted 10 lines
drivers/pci/pci-acpi.c +9 -1
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_power_state_for_wake(adev) <= ACPI_STATE_D3_HOT)

The comment suggests that this should check for "<= ACPI_STATE_D2" (not ACPI_STATE_D3_HOT). Or is there some subtlety here that I'm missing?

Quoted 33 lines
drivers/pci/pci-acpi.c +16 -3
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
-	if (!adev)
+	if (rpdev == dev)
+		rpadev = adev;
+	else
+		rpadev = ACPI_COMPANION(&rpdev->dev);
+
+	if (!rpadev)
 		return false;
 
 	/*
@@ -1001,15 +1018,15 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
 	 * events from low-power states including D3hot and D3cold.
 	 */
-	if (!adev->wakeup.flags.valid)
+	if (!rpadev->wakeup.flags.valid)
 		return false;
 

drivers/pci/pci-acpi.c bool acpi_pci_bridge_d3(struct pci_dev * · line 1029

Quoted 10 lines
drivers/pci/pci-acpi.c +4 -4
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpadev != adev &&
+	    acpi_dev_power_state_for_wake(rpadev) <= ACPI_STATE_D3_HOT)

Same question here.

Quoted 57 lines
drivers/pci/pci-acpi.c +1 -1
 		return false;
 
 	/*
@@ -1018,7 +1035,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * bridges *below* that Root Port can also signal hotplug events
 	 * while in D3.
 	 */
-	if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+	if (!acpi_dev_get_property(rpadev, "HotPlugSupportInD3",
 				   ACPI_TYPE_INTEGER, &obj) &&
 	    obj->integer.value == 1)
 		return true;

Index: linux-pm/drivers/acpi/device_pm.c
===================================================================

drivers/acpi/device_pm.c +19 -0
@@ -484,6 +484,25 @@ void acpi_dev_power_up_children_with_adr
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
 }
 
+/**
+ * acpi_dev_power_state_for_wake - Deepest power state for wakeup signaling
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return the value produced by it or return
+ * ACPI_STATE_UNKNOWN on errors (including _S0W not present).
+ */
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	if (ACPI_FAILURE(status))
+		return ACPI_STATE_UNKNOWN;
+
+	return state;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);

Index: linux-pm/include/acpi/acpi_bus.h
===================================================================

include/acpi/acpi_bus.h +1 -0
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
#27 lore raw

Re: [PATCH v3] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 50 lines · Bjorn Helgaas
Quoted 48 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

---

v2 -> v3:
* Use rpadev for the ACPI companion of the Root Port in acpi_pci_bridge_d3(() to avoid confusion.
* Make the function evaluating _S0W return the value produced by it or "unknown state" on errors and let its caller deal with that value.

---
 drivers/acpi/device_pm.c |   19 +++++++++++++++++++
 drivers/pci/pci-acpi.c   |   45 +++++++++++++++++++++++++++++++--------------
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 51 insertions(+), 14 deletions(-)

Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================

drivers/pci/pci-acpi.c +1 -3
@@ -976,24 +976,41 @@ bool acpi_pci_power_manageable(struct pc
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
      struct pci_dev *rpdev;
-     struct acpi_device *adev;
-     acpi_status status;
-     unsigned long long state;
+     struct acpi_device *adev, *rpadev;
      const union acpi_object *obj;
 
      if (acpi_pci_disabled || !dev->is_hotplug_bridge)
              return false;
 
Quoted 16 lines

drivers/pci/pci-acpi.c bool acpi_pci_power_manageable(struct pc · line 993

Quoted 12 lines
drivers/pci/pci-acpi.c +9 -3
-     /* Assume D3 support if the bridge is power-manageable by ACPI. */
-     if (acpi_pci_power_manageable(dev))
-             return true;
+     adev = ACPI_COMPANION(&dev->dev);
+     if (adev) {
+             /*
+              * If the bridge has _S0W, whether or not it can go into D3
+              * depends on what is returned by that object.  In particular,
+              * if the power state returned by _S0W is D2 or shallower,
+              * entering D3 should not be allowed.
+              */
+             if (acpi_dev_power_state_for_wake(adev) <= ACPI_STATE_D3_HOT)

The comment suggests that this should check for "<= ACPI_STATE_D2" (not ACPI_STATE_D3_HOT). Or is there some subtlety here that I'm missing?

No, this is a mistake. I'll send a v4.

#28 lore raw

PATCH v4 1/1 PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

In reply to Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

---

v3 -> v4:
* Use ACPI_STATE_D2 in the checks in acpi_pci_bridge_d3().

v2 -> v3:
* Use rpadev for the ACPI companion of the Root Port in acpi_pci_bridge_d3(() to avoid confusion.
* Make the function evaluating _S0W return the value produced by it or "unknown state" on errors and let its caller deal with that value.

---
 drivers/acpi/device_pm.c |   19 +++++++++++++++++++
 drivers/pci/pci-acpi.c   |   45 +++++++++++++++++++++++++++++++--------------
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 51 insertions(+), 14 deletions(-)
drivers/pci/pci-acpi.c +31 -14
Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================
@@ -976,24 +976,41 @@ bool acpi_pci_power_manageable(struct pc
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	struct pci_dev *rpdev;
-	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
+	struct acpi_device *adev, *rpadev;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_power_state_for_wake(adev) <= ACPI_STATE_D2)
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
-	if (!adev)
+	if (rpdev == dev)
+		rpadev = adev;
+	else
+		rpadev = ACPI_COMPANION(&rpdev->dev);
+
+	if (!rpadev)
 		return false;
 
 	/*
@@ -1001,15 +1018,15 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
 	 * events from low-power states including D3hot and D3cold.
 	 */
-	if (!adev->wakeup.flags.valid)
+	if (!rpadev->wakeup.flags.valid)
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpadev != adev &&
+	    acpi_dev_power_state_for_wake(rpadev) <= ACPI_STATE_D2)
 		return false;
 
 	/*
@@ -1018,7 +1035,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * bridges *below* that Root Port can also signal hotplug events
 	 * while in D3.
 	 */
-	if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+	if (!acpi_dev_get_property(rpadev, "HotPlugSupportInD3",
 				   ACPI_TYPE_INTEGER, &obj) &&
 	    obj->integer.value == 1)
 		return true;
drivers/acpi/device_pm.c +19 -0
Index: linux-pm/drivers/acpi/device_pm.c
===================================================================
@@ -484,6 +484,25 @@ void acpi_dev_power_up_children_with_adr
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
 }
 
+/**
+ * acpi_dev_power_state_for_wake - Deepest power state for wakeup signaling
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return the value produced by it or return
+ * ACPI_STATE_UNKNOWN on errors (including _S0W not present).
+ */
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	if (ACPI_FAILURE(status))
+		return ACPI_STATE_UNKNOWN;
+
+	return state;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);
include/acpi/acpi_bus.h +1 -0
Index: linux-pm/include/acpi/acpi_bus.h
===================================================================
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
#29 lore raw

Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 11 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

Quoted 6 lines

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

Applied to pci/pm for v6.3, thanks!

It'd be great if we could include a short description of the problems users might see. I think the original problem was that on some AMD systems we put a USB4 router in D3 when it should remain in D0. And I assume this means something doesn't wake up when it should? Or maybe we miss a hotplug event?

If somebody has an example or some text, I'll add it to the commit log.

Quoted 146 lines

---

v3 -> v4:
* Use ACPI_STATE_D2 in the checks in acpi_pci_bridge_d3().

v2 -> v3:
* Use rpadev for the ACPI companion of the Root Port in acpi_pci_bridge_d3(() to avoid confusion.
* Make the function evaluating _S0W return the value produced by it or "unknown state" on errors and let its caller deal with that value.

---
 drivers/acpi/device_pm.c |   19 +++++++++++++++++++
 drivers/pci/pci-acpi.c   |   45 +++++++++++++++++++++++++++++++--------------
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 51 insertions(+), 14 deletions(-)

Index: linux-pm/drivers/pci/pci-acpi.c
===================================================================

drivers/pci/pci-acpi.c +31 -14
@@ -976,24 +976,41 @@ bool acpi_pci_power_manageable(struct pc
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	struct pci_dev *rpdev;
-	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
+	struct acpi_device *adev, *rpadev;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_power_state_for_wake(adev) <= ACPI_STATE_D2)
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as it
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
-	if (!adev)
+	if (rpdev == dev)
+		rpadev = adev;
+	else
+		rpadev = ACPI_COMPANION(&rpdev->dev);
+
+	if (!rpadev)
 		return false;
 
 	/*
@@ -1001,15 +1018,15 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug
 	 * events from low-power states including D3hot and D3cold.
 	 */
-	if (!adev->wakeup.flags.valid)
+	if (!rpadev->wakeup.flags.valid)
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root Port
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpadev != adev &&
+	    acpi_dev_power_state_for_wake(rpadev) <= ACPI_STATE_D2)
 		return false;
 
 	/*
@@ -1018,7 +1035,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * bridges *below* that Root Port can also signal hotplug events
 	 * while in D3.
 	 */
-	if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+	if (!acpi_dev_get_property(rpadev, "HotPlugSupportInD3",
 				   ACPI_TYPE_INTEGER, &obj) &&
 	    obj->integer.value == 1)
 		return true;

Index: linux-pm/drivers/acpi/device_pm.c
===================================================================

drivers/acpi/device_pm.c +19 -0
@@ -484,6 +484,25 @@ void acpi_dev_power_up_children_with_adr
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present, NULL);
 }
 
+/**
+ * acpi_dev_power_state_for_wake - Deepest power state for wakeup signaling
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return the value produced by it or return
+ * ACPI_STATE_UNKNOWN on errors (including _S0W not present).
+ */
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL, &state);
+	if (ACPI_FAILURE(status))
+		return ACPI_STATE_UNKNOWN;
+
+	return state;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);

Index: linux-pm/include/acpi/acpi_bus.h
===================================================================

include/acpi/acpi_bus.h +1 -0
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
#30 lore raw

RE: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

[Public]

Quoted 36 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:02
To: Rafael J. Wysocki <[email protected]>
Cc: [email protected]; Limonciello, Mario <[email protected]>; Rafael J. Wysocki <[email protected]>; Len Brown <[email protected]>; Bjorn Helgaas <[email protected]>; Mika Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 11 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

Quoted 4 lines

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

[email protected]/

Quoted 2 lines

Applied to pci/pm for v6.3, thanks!

Quoted 8 lines

It'd be great if we could include a short description of the problems users might see. I think the original problem was that on some AMD systems we put a USB4 router in D3 when it should remain in D0. And I assume this means something doesn't wake up when it should? Or maybe we miss a hotplug event?

If somebody has an example or some text, I'll add it to the commit log.

Here's a blurb for what happens on AMD side:

When the platform is configured to not allow the PCIe port used for tunneling to wakeup from D3 it will runtime suspend into D0 and the USB4 controller which is a consumer will runtime suspend into D3.

This inconsistency leads to failures to initialize PCIe tunnels for USB4 devices.

Quoted 163 lines
Quoted 8 lines

---

v3 -> v4:
* Use ACPI_STATE_D2 in the checks in acpi_pci_bridge_d3().

v2 -> v3:
* Use rpadev for the ACPI companion of the Root Port in

acpi_pci_bridge_d3(()

Quoted 2 lines

to avoid confusion.
* Make the function evaluating _S0W return the value produced by it or

"unknown

Quoted 5 lines

state" on errors and let its caller deal with that value.

---
 drivers/acpi/device_pm.c |   19 +++++++++++++++++++
 drivers/pci/pci-acpi.c   |   45 +++++++++++++++++++++++++++++++-------

-------

Quoted 5 lines
 include/acpi/acpi_bus.h  |    1 +
 3 files changed, 51 insertions(+), 14 deletions(-)

Index: linux-pm/drivers/pci/pci-acpi.c

==========================================================
=========

Quoted 17 lines
drivers/pci/pci-acpi.c +1 -5
@@ -976,24 +976,41 @@ bool acpi_pci_power_manageable(struct pc
 bool acpi_pci_bridge_d3(struct pci_dev *dev)
 {
 	struct pci_dev *rpdev;
-	struct acpi_device *adev;
-	acpi_status status;
-	unsigned long long state;
+	struct acpi_device *adev, *rpadev;
 	const union acpi_object *obj;
 
 	if (acpi_pci_disabled || !dev->is_hotplug_bridge)
 		return false;
 
-	/* Assume D3 support if the bridge is power-manageable by ACPI. */
-	if (acpi_pci_power_manageable(dev))

drivers/pci/pci-acpi.c bool acpi_pci_power_manageable(struct pc · line 993

Quoted 10 lines
drivers/pci/pci-acpi.c +9 -1
-		return true;
+	adev = ACPI_COMPANION(&dev->dev);
+	if (adev) {
+		/*
+		 * If the bridge has _S0W, whether or not it can go into D3
+		 * depends on what is returned by that object.  In particular,
+		 * if the power state returned by _S0W is D2 or shallower,
+		 * entering D3 should not be allowed.
+		 */
+		if (acpi_dev_power_state_for_wake(adev) <=

ACPI_STATE_D2)

drivers/pci/pci-acpi.c bool acpi_pci_power_manageable(struct pc

Quoted 4 lines
drivers/pci/pci-acpi.c +4 -0
+			return false;
+
+		/*
+		 * Otherwise, assume that the bridge can enter D3 so long as

it

Quoted 23 lines
drivers/pci/pci-acpi.c +11 -2
+		 * is power-manageable via ACPI.
+		 */
+		if (acpi_device_power_manageable(adev))
+			return true;
+	}
 
 	rpdev = pcie_find_root_port(dev);
 	if (!rpdev)
 		return false;
 
-	adev = ACPI_COMPANION(&rpdev->dev);
-	if (!adev)
+	if (rpdev == dev)
+		rpadev = adev;
+	else
+		rpadev = ACPI_COMPANION(&rpdev->dev);
+
+	if (!rpadev)
 		return false;
 
 	/*
@@ -1001,15 +1018,15 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * doesn't supply a wakeup GPE via _PRW, it cannot signal hotplug

drivers/pci/pci-acpi.c bool acpi_pci_bridge_d3(struct pci_dev * · line 1025

Quoted 10 lines
drivers/pci/pci-acpi.c +2 -3
 	 * events from low-power states including D3hot and D3cold.
 	 */
-	if (!adev->wakeup.flags.valid)
+	if (!rpadev->wakeup.flags.valid)
 		return false;
 
 	/*
-	 * If the Root Port cannot wake itself from D3hot or D3cold, we
-	 * can't use D3.
+	 * In the bridge-below-a-Root-Port case, evaluate _S0W for the Root

Port

drivers/pci/pci-acpi.c bool acpi_pci_bridge_d3(struct pci_dev *

Quoted 3 lines
drivers/pci/pci-acpi.c +1 -1
+	 * to verify whether or not it can signal wakeup from D3.
 	 */
-	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL,

&state);

Quoted 15 lines
drivers/pci/pci-acpi.c +3 -2
-	if (ACPI_SUCCESS(status) && state < ACPI_STATE_D3_HOT)
+	if (rpadev != adev &&
+	    acpi_dev_power_state_for_wake(rpadev) <= ACPI_STATE_D2)
 		return false;
 
 	/*
@@ -1018,7 +1035,7 @@ bool acpi_pci_bridge_d3(struct pci_dev *
 	 * bridges *below* that Root Port can also signal hotplug events
 	 * while in D3.
 	 */
-	if (!acpi_dev_get_property(adev, "HotPlugSupportInD3",
+	if (!acpi_dev_get_property(rpadev, "HotPlugSupportInD3",
 				   ACPI_TYPE_INTEGER, &obj) &&
 	    obj->integer.value == 1)
 		return true;
Quoted 2 lines

Index: linux-pm/drivers/acpi/device_pm.c

==========================================================
=========

drivers/acpi/device_pm.c void acpi_dev_power_up_children_with_adr · line 484

Quoted 4 lines
drivers/acpi/device_pm.c +0 -0
@@ -484,6 +484,25 @@ void acpi_dev_power_up_children_with_adr
 	acpi_dev_for_each_child(adev, acpi_power_up_if_adr_present,

NULL);

drivers/acpi/device_pm.c void acpi_dev_power_up_children_with_adr

Quoted 4 lines
drivers/acpi/device_pm.c +2 -0
 }
 
+/**
+ * acpi_dev_power_state_for_wake - Deepest power state for wakeup

signaling

drivers/acpi/device_pm.c void acpi_dev_power_up_children_with_adr

Quoted 11 lines
drivers/acpi/device_pm.c +11 -0
+ * @adev: ACPI companion of the target device.
+ *
+ * Evaluate _S0W for @adev and return the value produced by it or return
+ * ACPI_STATE_UNKNOWN on errors (including _S0W not present).
+ */
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev)
+{
+	unsigned long long state;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(adev->handle, "_S0W", NULL,

&state);

Quoted 11 lines
drivers/acpi/device_pm.c +6 -0
+	if (ACPI_FAILURE(status))
+		return ACPI_STATE_UNKNOWN;
+
+	return state;
+}
+
 #ifdef CONFIG_PM
 static DEFINE_MUTEX(acpi_pm_notifier_lock);
 static DEFINE_MUTEX(acpi_pm_notifier_install_lock);

Index: linux-pm/include/acpi/acpi_bus.h

==========================================================
=========

Quoted 13 lines
include/acpi/acpi_bus.h +1 -0
@@ -533,6 +533,7 @@ int acpi_bus_update_power(acpi_handle ha
 int acpi_device_update_power(struct acpi_device *device, int *state_p);
 bool acpi_bus_power_manageable(acpi_handle handle);
 void acpi_dev_power_up_children_with_adr(struct acpi_device *adev);
+u8 acpi_dev_power_state_for_wake(struct acpi_device *adev);
 int acpi_device_power_add_dependent(struct acpi_device *adev,
 				    struct device *dev);
 void acpi_device_power_remove_dependent(struct acpi_device *adev,
#31 lore raw

Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 45 lines · Mario
Quoted 36 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:02
To: Rafael J. Wysocki <[email protected]>
Cc: [email protected]; Limonciello, Mario <[email protected]>; Rafael J. Wysocki <[email protected]>; Len Brown <[email protected]>; Bjorn Helgaas <[email protected]>; Mika Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 11 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

Quoted 4 lines

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

[email protected]/

Quoted 2 lines

Applied to pci/pm for v6.3, thanks!

Quoted 8 lines

It'd be great if we could include a short description of the problems users might see. I think the original problem was that on some AMD systems we put a USB4 router in D3 when it should remain in D0. And I assume this means something doesn't wake up when it should? Or maybe we miss a hotplug event?

If somebody has an example or some text, I'll add it to the commit log.

Quoted 8 lines

Here's a blurb for what happens on AMD side:

When the platform is configured to not allow the PCIe port used for tunneling to wakeup from D3 it will runtime suspend into D0 and the USB4 controller which is a consumer will runtime suspend into D3.

This inconsistency leads to failures to initialize PCIe tunnels for USB4 devices.

And what is J. Random User going to see? DisplayPort not working ever? It works to begin with, but not after a suspend? Devices in a dock not being able to wake the system?

I don't really know what "PCIe tunnels for USB4 devices not being initialized" means for me. I want to know what a problem report from a non-expert user might look like.

Bjorn

#32 lore raw

RE: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

[AMD Official Use Only - General]

Quoted 70 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:41
To: Limonciello, Mario <[email protected]>
Cc: Rafael J. Wysocki <[email protected]>; [email protected]; Rafael J. Wysocki <[email protected]>; Len Brown <[email protected]>; Bjorn Helgaas <[email protected]>; Mika Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 6 lines · Mario
Quoted 6 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:02
To: Rafael J. Wysocki <[email protected]>
Cc: [email protected]; Limonciello, Mario <[email protected]>; Rafael J. Wysocki <[email protected]>;

Len

Quoted 39 lines
Quoted 30 lines

Brown <[email protected]>; Bjorn Helgaas <[email protected]>; Mika Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 11 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

Quoted 4 lines

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

[email protected]/

Quoted 2 lines

Applied to pci/pm for v6.3, thanks!

Quoted 8 lines

It'd be great if we could include a short description of the problems users might see. I think the original problem was that on some AMD systems we put a USB4 router in D3 when it should remain in D0. And I assume this means something doesn't wake up when it should? Or maybe we miss a hotplug event?

If somebody has an example or some text, I'll add it to the commit log.

Quoted 8 lines

Here's a blurb for what happens on AMD side:

When the platform is configured to not allow the PCIe port used for tunneling to wakeup from D3 it will runtime suspend into D0 and the USB4 controller which is a consumer will runtime suspend into D3.

This inconsistency leads to failures to initialize PCIe tunnels for USB4 devices.

Quoted 8 lines

And what is J. Random User going to see? DisplayPort not working ever? It works to begin with, but not after a suspend? Devices in a dock not being able to wake the system?

I don't really know what "PCIe tunnels for USB4 devices not being initialized" means for me. I want to know what a problem report from a non-expert user might look like.

DP tunnels aren't affected, so monitors should still work.

In terms of what doesn't work it depends on the architecture of the the connected device. Here's some concrete up-leveled examples:

USB4 docks contain that a PCIe network adapter and that adapter won't work when the dock is plugged in after the system boot.

USB4 docks that contain a USB network adapter should work properly, but downstream USB4 or TBT3 devices connected to that USB4 dock will not work when the device or dock is plugged in after the system boots.

TBT3 storage devices connected after the system has booted will not work.

Thanks,

#33 lore raw

Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 87 lines · Mario

[AMD Official Use Only - General]

Quoted 70 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:41
To: Limonciello, Mario <[email protected]>
Cc: Rafael J. Wysocki <[email protected]>; [email protected]; Rafael J. Wysocki <[email protected]>; Len Brown <[email protected]>; Bjorn Helgaas <[email protected]>; Mika Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 6 lines · Mario
Quoted 6 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:02
To: Rafael J. Wysocki <[email protected]>
Cc: [email protected]; Limonciello, Mario <[email protected]>; Rafael J. Wysocki <[email protected]>;

Len

Quoted 39 lines
Quoted 30 lines

Brown <[email protected]>; Bjorn Helgaas <[email protected]>; Mika Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 11 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for the target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge regardless of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

Quoted 4 lines

However, still allow bridges without _S0W that are power-manageable via ACPI to enter D3 to retain the current code behavior in that case.

[email protected]/

Quoted 2 lines

Applied to pci/pm for v6.3, thanks!

Quoted 8 lines

It'd be great if we could include a short description of the problems users might see. I think the original problem was that on some AMD systems we put a USB4 router in D3 when it should remain in D0. And I assume this means something doesn't wake up when it should? Or maybe we miss a hotplug event?

If somebody has an example or some text, I'll add it to the commit log.

Quoted 8 lines

Here's a blurb for what happens on AMD side:

When the platform is configured to not allow the PCIe port used for tunneling to wakeup from D3 it will runtime suspend into D0 and the USB4 controller which is a consumer will runtime suspend into D3.

This inconsistency leads to failures to initialize PCIe tunnels for USB4 devices.

Quoted 7 lines

And what is J. Random User going to see? DisplayPort not working ever? It works to begin with, but not after a suspend? Devices in a dock not being able to wake the system?

I don't really know what "PCIe tunnels for USB4 devices not being initialized" means for me. I want to know what a problem report from a non-expert user might look like.

DP tunnels aren't affected, so monitors should still work.

In terms of what doesn't work it depends on the architecture of the the connected device. Here's some concrete up-leveled examples:

Quoted 8 lines

USB4 docks contain that a PCIe network adapter and that adapter won't work when the dock is plugged in after the system boot.

USB4 docks that contain a USB network adapter should work properly, but downstream USB4 or TBT3 devices connected to that USB4 dock will not work when the device or dock is plugged in after the system boots.

TBT3 storage devices connected after the system has booted will not work.

Thanks, this is exactly the sort of thing I was looking for. Since they all mention "connected after boot," I assume the issue with the current code is that a hotplug notification is being missed because a bridge is in D3.

If the devices are present and enumerated at boot, there's no issue with suspend/resume, right?

What do you think of the following possible text? I don't want to be overly specific because I don't think it's practical to list every scenario. We just need a hook to make people think "Hmm, I'm seeing a dock issue; maybe this is the fix."

This fixes problems where a hotplug notification is missed because a bridge is in D3. That means hot-added devices such as USB4 docks (and the devices they contain) and Thunderbolt 3 devices may not work.

#34 lore raw

RE: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

[Public]

Quoted 123 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Friday, January 13, 2023 11:52
To: Limonciello, Mario <[email protected]>
Cc: Rafael J. Wysocki <[email protected]>; [email protected]; Rafael J. Wysocki <[email protected]>; Len Brown <[email protected]>; Bjorn Helgaas <[email protected]>; Mika Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge into account in acpi_pci_bridge_d3(()

Quoted 9 lines · Mario

[AMD Official Use Only - General]

Quoted 5 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:41
To: Limonciello, Mario <[email protected]>
Cc: Rafael J. Wysocki <[email protected]>; [email protected];

Rafael J.

Quoted 1 line
Quoted 1 line

Wysocki <[email protected]>; Len Brown <[email protected]>; Bjorn

Helgaas

Quoted 2 lines
Quoted 2 lines

<[email protected]>; Mika Westerberg
<[email protected]>; Mehta, Sanju

<[email protected]>;

Quoted 4 lines
Quoted 4 lines

Lukas Wunner <[email protected]>; Rafael J . Wysocki <[email protected]>; [email protected]; linux- [email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target bridge

into

Quoted 9 lines
Quoted 9 lines

account in acpi_pci_bridge_d3(()

Quoted 6 lines · Mario
Quoted 6 lines

-----Original Message-----
From: Bjorn Helgaas <[email protected]>
Sent: Thursday, January 12, 2023 16:02
To: Rafael J. Wysocki <[email protected]>
Cc: [email protected]; Limonciello, Mario
<[email protected]>; Rafael J. Wysocki

<[email protected]>;

Quoted 2 lines
Quoted 2 lines

Len

Quoted 1 line
Quoted 1 line

Brown <[email protected]>; Bjorn Helgaas <[email protected]>;

Mika

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Westerberg <[email protected]>; Mehta, Sanju <[email protected]>; Lukas Wunner <[email protected]>;

Rafael J .

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Wysocki <[email protected]>; [email protected];

linux-

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

[email protected]; Linux PM <[email protected]>
Subject: Re: [PATCH v4] PCI / ACPI: PM: Take _S0W of the target

bridge into

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

account in acpi_pci_bridge_d3(()

Quoted 5 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

It is generally questionable to allow a PCI bridge to go into D3 if it has _S0W returning D2 or a shallower power state, so modify acpi_pci_bridge_d3(() to always take the return value of _S0W for

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

target bridge into accout. That is, make it return 'false' if _S0W returns D2 or a shallower power state for the target bridge

regardless

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

of its ancestor PCIe Root Port properties. Of course, this also causes 'false' to be returned if the PCIe Root Port itself is the target and its _S0W returns D2 or a shallower power state.

However, still allow bridges without _S0W that are power-

manageable via

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

ACPI to enter D3 to retain the current code behavior in that case.

[email protected]/

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

Applied to pci/pm for v6.3, thanks!

It'd be great if we could include a short description of the problems users might see. I think the original problem was that on some AMD systems we put a USB4 router in D3 when it should remain in D0. And

I

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

assume this means something doesn't wake up when it should? Or

maybe

Quoted 34 lines
Quoted 14 lines
Quoted 5 lines
Quoted 4 lines

we miss a hotplug event?

If somebody has an example or some text, I'll add it to the commit log.

Quoted 8 lines

Here's a blurb for what happens on AMD side:

When the platform is configured to not allow the PCIe port used for tunneling to wakeup from D3 it will runtime suspend into D0 and the USB4 controller which is a consumer will runtime suspend into D3.

This inconsistency leads to failures to initialize PCIe tunnels for USB4 devices.

Quoted 7 lines

And what is J. Random User going to see? DisplayPort not working ever? It works to begin with, but not after a suspend? Devices in a dock not being able to wake the system?

I don't really know what "PCIe tunnels for USB4 devices not being initialized" means for me. I want to know what a problem report from a non-expert user might look like.

DP tunnels aren't affected, so monitors should still work.

In terms of what doesn't work it depends on the architecture of the the connected device. Here's some concrete up-leveled examples:

USB4 docks contain that a PCIe network adapter and that adapter won't work when the dock is plugged in after the system boot.

USB4 docks that contain a USB network adapter should work properly, but downstream USB4 or TBT3 devices connected to that USB4 dock will not work when the device or dock is plugged in after the system boots.

Quoted 6 lines
Quoted 1 line

TBT3 storage devices connected after the system has booted will not work.

Thanks, this is exactly the sort of thing I was looking for. Since they all mention "connected after boot," I assume the issue with the current code is that a hotplug notification is being missed because a bridge is in D3.

Yeah.

Quoted 4 lines

If the devices are present and enumerated at boot, there's no issue with suspend/resume, right?

I intentionally left that out because it's going to be conditional based on a lot of other things.

Quoted 9 lines

What do you think of the following possible text? I don't want to be overly specific because I don't think it's practical to list every scenario. We just need a hook to make people think "Hmm, I'm seeing a dock issue; maybe this is the fix."

This fixes problems where a hotplug notification is missed because a bridge is in D3. That means hot-added devices such as USB4 docks (and the devices they contain) and Thunderbolt 3 devices may not work.

Perfect.