PCI/PM: Put devices to low power state on shutdown
PATCH v3 1/1 PCI/PM: Put devices to low power state on shutdown
From: Kai-Heng Feng <[email protected]>
Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is connected.
The following error message can be found during shutdown:
pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0 pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
pcieport 0000:09:04.0: device [8086:0b26] error status/mask=00000080/00002000
pcieport 0000:09:04.0: [ 7] BadDLLP
Calling aer_remove() during shutdown can quiesce the error message, however the spurious wakeup still happens.
The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown to solve the issue.
ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4."
The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5.
---
v3:
* Pick up tags
* V2 was waiting for Rafael to review, rebase on pci/next and resend.
--- drivers/pci/pci-driver.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/pci/pci-driver.c +8 -0
@@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)if (drv && drv->shutdown) drv->shutdown (pci_dev); +/* +* If driver already changed device's power state, it can mean the +* wakeup setting is in place, or a workaround is used. Hence keep it +* as is. +*/ +if (!kexec_in_progress && pci_dev->current_state ==PCI_D0 ) +pci_prepare_to_sleep (pci_dev); +/* * If this is a kexec reboot, turn off Bus Master bit on the * device to tell it to not continue to do DMA. Don't touch
Signature · 1 line
2.43.0
Rafael J. Wysocki
maintainerQuoted 18 lines · Mario Limonciello
From: Kai-Heng Feng <[email protected]>
Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is connected.
The following error message can be found during shutdown:
pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0 pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
pcieport 0000:09:04.0: device [8086:0b26] error status/mask=00000080/00002000
pcieport 0000:09:04.0: [ 7] BadDLLPCalling
aer_remove()during shutdown can quiesce the error message, however the spurious wakeup still happens.The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown to solve the issue.
Quoted 7 lines
ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4."
The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5.
That's fine as long as you assume that ->shutdown() is only used for implementing ACPI S5, but it is not.
Quoted 10 lines
Cc AceLan Kao <[email protected]>Reviewed-by Mario Limonciello <[email protected]>Tested-by Mario Limonciello <[email protected]>Signed-off-by Kai-Heng Feng <[email protected]>Tested-by Mark Pearson <[email protected]>Tested-by Denis Benato <[email protected]>Tested-by Merthan Karakaş <[email protected]>Signed-off-by Mario Limonciello <[email protected]>
Quoted 4 lines
---
v3:
* Pick up tags
* V2 was waiting for Rafael to review, rebase on pci/next and resend.
Is this change going to break kexec?
Quoted 24 lines
--- drivers/pci/pci-driver.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/pci/pci-driver.c +8
-1@@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)if (drv && drv->shutdown) drv->shutdown (pci_dev); +/* +* If driver already changed device's power state, it can mean the +* wakeup setting is in place, or a workaround is used. Hence keep it +* as is. +*/ +if (!kexec_in_progress && pci_dev->current_state ==PCI_D0 ) +pci_prepare_to_sleep (pci_dev); +/* * If this is a kexec reboot, turn off Bus Master bit on the * device to tell it to not continue to do DMA. Don't touch --
Quoted 23 lines · Rafael J. Wysocki
Quoted 22 lines · Mario Limonciello
From: Kai-Heng Feng <[email protected]>
Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is connected.
The following error message can be found during shutdown:
pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0 pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
pcieport 0000:09:04.0: device [8086:0b26] error status/mask=00000080/00002000
pcieport 0000:09:04.0: [ 7] BadDLLPCalling
aer_remove()during shutdown can quiesce the error message, however the spurious wakeup still happens.The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown to solve the issue.
ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4."
Quoted 6 lines
Quoted 3 lines
The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5.
That's fine as long as you assume that ->shutdown() is only used for implementing ACPI S5, but it is not.
I suppose you're meaning things like:
kernel_restart_prepare()
->device_shutdown()
->->each device's shutdown() CB
->->each driver's shutdown() CB
Is there somewhere "better" to do this so it's truly only tied to S5?
Quoted 16 lines
Quoted 11 lines
Cc AceLan Kao <[email protected]>Reviewed-by Mario Limonciello <[email protected]>Tested-by Mario Limonciello <[email protected]>Signed-off-by Kai-Heng Feng <[email protected]>Tested-by Mark Pearson <[email protected]>Tested-by Denis Benato <[email protected]>Tested-by Merthan Karakaş <[email protected]>Signed-off-by Mario Limonciello <[email protected]>Quoted 4 lines
---
v3:
* Pick up tags
* V2 was waiting for Rafael to review, rebase on pci/next and resend.
Quoted 1 line
Is this change going to break kexec?
There is an explicit check in the below code for kexec_in_progress, so my expectation was that kexec kept working. I didn't explicitly test this myself when I tested KH's change before sending it again.
KH, Did you double check that on your side?
Quoted 25 lines
Quoted 25 lines
--- drivers/pci/pci-driver.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/pci/pci-driver.c +8
-1@@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)if (drv && drv->shutdown) drv->shutdown (pci_dev); +/* +* If driver already changed device's power state, it can mean the +* wakeup setting is in place, or a workaround is used. Hence keep it +* as is. +*/ +if (!kexec_in_progress && pci_dev->current_state ==PCI_D0 ) +pci_prepare_to_sleep (pci_dev); +/* * If this is a kexec reboot, turn off Bus Master bit on the * device to tell it to not continue to do DMA. Don't touch --
Rafael J. Wysocki
maintainerQuoted 32 lines · Mario Limonciello
Quoted 23 lines · Rafael J. Wysocki
Quoted 22 lines · Mario Limonciello
From: Kai-Heng Feng <[email protected]>
Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is connected.
The following error message can be found during shutdown:
pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0 pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
pcieport 0000:09:04.0: device [8086:0b26] error status/mask=00000080/00002000
pcieport 0000:09:04.0: [ 7] BadDLLPCalling
aer_remove()during shutdown can quiesce the error message, however the spurious wakeup still happens.The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown to solve the issue.
ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4."
Quoted 6 lines
Quoted 3 lines
The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5.
That's fine as long as you assume that ->shutdown() is only used for implementing ACPI S5, but it is not.
Quoted 8 lines
I suppose you're meaning things like:
kernel_restart_prepare()
->device_shutdown()
->->each device'sshutdown()CB
->->each driver'sshutdown()CBIs there somewhere "better" to do this so it's truly only tied to S5?
On systems that use ACPI reboot is also a flavor of S5, but what about systems that have PCI, but don't use ACPI?
The change you are proposing is going to affect more systems than just the ones having the problem described in the patch changelog, so it is kind of risky and breaking reboot is a big deal.
Quoted 50 lines
Quoted 16 lines
Quoted 11 lines
Cc AceLan Kao <[email protected]>Reviewed-by Mario Limonciello <[email protected]>Tested-by Mario Limonciello <[email protected]>Signed-off-by Kai-Heng Feng <[email protected]>Tested-by Mark Pearson <[email protected]>Tested-by Denis Benato <[email protected]>Tested-by Merthan Karakaş <[email protected]>Signed-off-by Mario Limonciello <[email protected]>Quoted 4 lines
---
v3:
* Pick up tags
* V2 was waiting for Rafael to review, rebase on pci/next and resend.Quoted 1 line
Is this change going to break kexec?
There is an explicit check in the below code for
kexec_in_progress, so my expectation was that kexec kept working. I didn't explicitly test this myself when I tested KH's change before sending it again.KH, Did you double check that on your side?
Quoted 25 lines
Quoted 25 lines
--- drivers/pci/pci-driver.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/pci/pci-driver.c +8
-1@@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)if (drv && drv->shutdown) drv->shutdown (pci_dev); +/* +* If driver already changed device's power state, it can mean the +* wakeup setting is in place, or a workaround is used. Hence keep it +* as is. +*/ +if (!kexec_in_progress && pci_dev->current_state ==PCI_D0 ) +pci_prepare_to_sleep (pci_dev); +/* * If this is a kexec reboot, turn off Bus Master bit on the * device to tell it to not continue to do DMA. Don't touch --
Quoted 40 lines · Rafael J. Wysocki
Quoted 39 lines · Mario Limonciello
Quoted 23 lines · Rafael J. Wysocki
Quoted 22 lines · Mario Limonciello
From: Kai-Heng Feng <[email protected]>
Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is connected.
The following error message can be found during shutdown:
pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0 pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
pcieport 0000:09:04.0: device [8086:0b26] error status/mask=00000080/00002000
pcieport 0000:09:04.0: [ 7] BadDLLPCalling
aer_remove()during shutdown can quiesce the error message, however the spurious wakeup still happens.The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown to solve the issue.
ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4."
Quoted 6 lines
Quoted 3 lines
The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5.
That's fine as long as you assume that ->shutdown() is only used for implementing ACPI S5, but it is not.
I suppose you're meaning things like:
kernel_restart_prepare()
->device_shutdown()
->->each device'sshutdown()CB
->->each driver'sshutdown()CB
Quoted 8 lines
Quoted 1 line
Is there somewhere "better" to do this so it's truly only tied to S5?
On systems that use ACPI reboot is also a flavor of S5, but what about systems that have PCI, but don't use ACPI?
The change you are proposing is going to affect more systems than just the ones having the problem described in the patch changelog, so it is kind of risky and breaking reboot is a big deal.
How would you feel about if instead this was using one of the power off notifiers? That could narrow this down to only being applied on ACPI systems.
Something like this (not yet tested):
drivers/pci/pci-acpi.c +17 -0
@@ -10,6 +10,7 @@#include <linux/delay.h> #include <linux/init.h> #include <linux/irqdomain.h> +#include <linux/kexec.h> #include <linux/pci.h> #include <linux/msi.h> #include <linux/pci_hotplug.h>
@@ -18,6 +19,7 @@#include <linux/pci-ecam.h> #include <linux/pm_runtime.h> #include <linux/pm_qos.h> +#include <linux/reboot.h> #include <linux/rwsem.h> #include "pci.h"
@@ -1438,6 +1440,16 @@ static void pci_acpi_set_external_facing(struct pci_dev *dev) dev->external_facing =1 ; } +static int pci_acpi_power_off (struct sys_off_data *data) +{ +struct pci_dev *pci_dev =to_pci_dev (data->dev); + +if (!kexec_in_progress && pci_dev->current_state ==PCI_D0 ) +pci_prepare_to_sleep (pci_dev); + +return NOTIFY_DONE ; +} +void pci_acpi_setup (struct device *dev,struct acpi_device *adev) {struct pci_dev *pci_dev =to_pci_dev (dev);
@@ -1465,6 +1477,11 @@ void pci_acpi_setup(struct device *dev, struct acpi_device *adev)if (pci_is_bridge (pci_dev))acpi_dev_power_up_children_with_adr (adev); + +devm_register_sys_off_handler (dev,SYS_OFF_MODE_POWER_OFF_PREPARE , +SYS_OFF_PRIO_DEFAULT , + &pci_acpi_power_off,NULL ); + }void pci_acpi_cleanup (struct device *dev,struct acpi_device *adev)
Quoted 51 lines
Quoted 51 lines
Quoted 17 lines
Quoted 12 lines
Cc AceLan Kao <[email protected]>Reviewed-by Mario Limonciello <[email protected]>Tested-by Mario Limonciello <[email protected]>Signed-off-by Kai-Heng Feng <[email protected]>Tested-by Mark Pearson <[email protected]>Tested-by Denis Benato <[email protected]>Tested-by Merthan Karakaş <[email protected]>Signed-off-by Mario Limonciello <[email protected]>Quoted 4 lines
---
v3:
* Pick up tags
* V2 was waiting for Rafael to review, rebase on pci/next and resend.Quoted 1 line
Is this change going to break kexec?
There is an explicit check in the below code for
kexec_in_progress, so my expectation was that kexec kept working. I didn't explicitly test this myself when I tested KH's change before sending it again.KH, Did you double check that on your side?
Quoted 25 lines
Quoted 25 lines
--- drivers/pci/pci-driver.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/pci/pci-driver.c +8
-1@@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)if (drv && drv->shutdown) drv->shutdown (pci_dev); +/* +* If driver already changed device's power state, it can mean the +* wakeup setting is in place, or a workaround is used. Hence keep it +* as is. +*/ +if (!kexec_in_progress && pci_dev->current_state ==PCI_D0 ) +pci_prepare_to_sleep (pci_dev); +/* * If this is a kexec reboot, turn off Bus Master bit on the * device to tell it to not continue to do DMA. Don't touch --
Quoted 34 lines · Mario Limonciello
External email: Use caution opening links or attachments
Quoted 23 lines · Rafael J. Wysocki
Quoted 22 lines · Mario Limonciello
From: Kai-Heng Feng <[email protected]>
Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is connected.
The following error message can be found during shutdown:
pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0 pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID) pcieport 0000:09:04.0: device [8086:0b26] error status/mask=00000080/00002000 pcieport 0000:09:04.0: [ 7] BadDLLPCalling
aer_remove()during shutdown can quiesce the error message, however the spurious wakeup still happens.The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown to solve the issue.
ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4."
Quoted 6 lines
Quoted 3 lines
The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5.
That's fine as long as you assume that ->shutdown() is only used for implementing ACPI S5, but it is not.
Quoted 8 lines
I suppose you're meaning things like:
kernel_restart_prepare()
->device_shutdown()
->->each device'sshutdown()CB
->->each driver'sshutdown()CBIs there somewhere "better" to do this so it's truly only tied to S5?
Alternatively, is it possible to put the change under '#ifdef CONFIG_ACPI' or 'IS_ENABLED(CONFIG_ACPI)', so it won't affect anything other than ACPI based system?
Kai-Heng
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Cc AceLan Kao <[email protected]>Reviewed-by Mario Limonciello <[email protected]>Tested-by Mario Limonciello <[email protected]>Signed-off-by Kai-Heng Feng <[email protected]>Tested-by Mark Pearson <[email protected]>Tested-by Denis Benato <[email protected]>Tested-by Merthan Karakaş <[email protected]>Signed-off-by Mario Limonciello <[email protected]>---
v3:
* Pick up tags
* V2 was waiting for Rafael to review, rebase on pci/next and resend.
Quoted 7 lines
Quoted 1 line
Is this change going to break kexec?
There is an explicit check in the below code for
kexec_in_progress, so my expectation was that kexec kept working. I didn't explicitly test this myself when I tested KH's change before sending it again.KH, Did you double check that on your side?
Kexec remains to work on my setup.
Kai-Heng
Quoted 27 lines
Quoted 26 lines
Quoted 26 lines
---
drivers/pci/pci-driver.c| 8 ++++++++1 file changed, 8 insertions(+)drivers/pci/pci-driver.c +8
-0@@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev) if (drv && drv->shutdown) drv->shutdown(pci_dev); + /* + * If driver already changed device's power state, it can mean the + * wakeup setting is in place, or a workaround is used. Hence keep it + * as is. + */ + if (!kexec_in_progress && pci_dev->current_state == PCI_D0) + pci_prepare_to_sleep(pci_dev); + /** If this is a kexec reboot, turn off Bus Master bit on the * device to tell it to not continue to do DMA. Don't touch