Improvements to S5 power consumption
Subsystems (10)
v3 cover Improvements to S5 power consumption
From: Mario Limonciello <[email protected]>
A variety of issues both in function and in power consumption have been raised as a result of devices not being put into a low power state when the system is powered off.
There have been some localized changes[1] to PCI core to help these issues, but they have had various downsides.
This series instead tries to use the S4 flow when the system is being powered off. This lines up the behavior with what other operating systems do as well. If for some reason that fails or is not supported, unwind and do the previous S5 flow that will wake all devices and run their shutdown() callbacks.
Previous submissions [1]:
Mario Limonciello (5):
PM: Use hibernate flows for system power off
PCI: Put PCIe ports with downstream devices into D3 at hibernate drm/amd: Avoid evicting resources at S5
scsi: Add PM_EVENT_POWEROFF into suspend callbacks
usb: sl811-hcd: Add PM_EVENT_POWEROFF into suspend callbacks
drivers/base/power/main.c | 7 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 + drivers/pci/pci-driver.c | 92 ++++++++++++++-------- drivers/scsi/mesh.c | 1 + drivers/scsi/stex.c | 1 + drivers/usb/host/sl811-hcd.c | 1 + include/linux/pm.h | 3 + include/trace/events/power.h | 3 +- kernel/reboot.c | 6 ++ 9 files changed, 84 insertions(+), 34 deletions(-)
Signature · 1 line
2.43.0
PATCH v3 1/5 PM: Use hibernate flows for system power off
From: Mario Limonciello <[email protected]>
When the system is powered off the kernel will call device_shutdown() which will issue callbacks into PCI core to wake up a device and call it's shutdown() callback. This will leave devices in ACPI D0 which can cause some devices to misbehave with spurious wakeups and also leave some devices on which will consume power needlessly.
The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown solves the issue.
ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4."
The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5.
To accomplish this, introduce a new PMSG_POWEROFF event that the PM core will use. During shutdown the use the new event to call all the device hibernate callbacks when the kernel is compiled with hibernate support. If compiled without hibernate support or hibernate fails fall back into the previous shutdown flow.
---
v3:
* Add new PMSG_POWEROFF and PM_EVENT_POWEROFF which alias to poweroff callbacks
* Don't try to cleanup on dpm_suspend_start() or dpm_suspend_end() failures Jump right into normal shutdown flow instead.
--- drivers/base/power/main.c | 7 +++++++ include/linux/pm.h | 3 +++ include/trace/events/power.h | 3 ++- kernel/reboot.c | 6 ++++++ 4 files changed, 18 insertions(+), 1 deletion(-)
drivers/base/power/main.c +7 -0
@@ -85,6 +85,8 @@ static const char *pm_verb(int event)return "restore" ;case PM_EVENT_RECOVER :return "recover" ; +case PM_EVENT_POWEROFF : +return "poweroff" ;default :return "(unknown PM event)" ; }
@@ -355,6 +357,7 @@ static pm_callback_t pm_op(const struct dev_pm_ops *ops, pm_message_t state)case PM_EVENT_FREEZE :case PM_EVENT_QUIESCE :return ops->freeze; +case PM_EVENT_POWEROFF :case PM_EVENT_HIBERNATE :return ops->poweroff;case PM_EVENT_THAW :
@@ -389,6 +392,7 @@ static pm_callback_t pm_late_early_op(const struct dev_pm_ops *ops,case PM_EVENT_FREEZE :case PM_EVENT_QUIESCE :return ops->freeze_late; +case PM_EVENT_POWEROFF :case PM_EVENT_HIBERNATE :return ops->poweroff_late;case PM_EVENT_THAW :
@@ -423,6 +427,7 @@ static pm_callback_t pm_noirq_op(const struct dev_pm_ops *ops, pm_message_t statcase PM_EVENT_FREEZE :case PM_EVENT_QUIESCE :return ops->freeze_noirq; +case PM_EVENT_POWEROFF :case PM_EVENT_HIBERNATE :return ops->poweroff_noirq;case PM_EVENT_THAW :
@@ -1296,6 +1301,8 @@ static pm_message_t resume_event(pm_message_t sleep_state)return PMSG_RECOVER ;case PM_EVENT_HIBERNATE :return PMSG_RESTORE ; +case PM_EVENT_POWEROFF : +return PMSG_ON ; }return PMSG_ON ; }
include/linux/pm.h +3 -0
@@ -506,6 +506,7 @@ const struct dev_pm_ops name = { \* RECOVER Creation of a hibernation image or restoration of the main * memory contents from a hibernation image has failed, call * ->thaw() and ->complete() for all devices. +* POWEROFF System will poweroff, call ->poweroff() for all devices. * * The following PM_EVENT_ messages are defined for internal use by * kernel subsystems. They are never issued by the PM core.
@@ -536,6 +537,7 @@ const struct dev_pm_ops name = { \#define PM_EVENT_USER 0x0100#define PM_EVENT_REMOTE 0x0200#define PM_EVENT_AUTO 0x0400 +#define PM_EVENT_POWEROFF 0x0800#define PM_EVENT_SLEEP (PM_EVENT_SUSPEND | PM_EVENT_HIBERNATE)#define PM_EVENT_USER_SUSPEND (PM_EVENT_USER | PM_EVENT_SUSPEND)
@@ -550,6 +552,7 @@ const struct dev_pm_ops name = { \#define PMSG_QUIESCE ((struct pm_message){ .event = PM_EVENT_QUIESCE, })#define PMSG_SUSPEND ((struct pm_message){ .event = PM_EVENT_SUSPEND, })#define PMSG_HIBERNATE ((struct pm_message){ .event = PM_EVENT_HIBERNATE, }) +#define PMSG_POWEROFF ((struct pm_message){ .event = PM_EVENT_POWEROFF, })#define PMSG_RESUME ((struct pm_message){ .event = PM_EVENT_RESUME, })#define PMSG_THAW ((struct pm_message){ .event = PM_EVENT_THAW, })#define PMSG_RESTORE ((struct pm_message){ .event = PM_EVENT_RESTORE, })
include/trace/events/power.h +2 -1
@@ -199,7 +199,8 @@ TRACE_EVENT(pstate_sample, { PM_EVENT_HIBERNATE, "hibernate" }, \ { PM_EVENT_THAW, "thaw" }, \ { PM_EVENT_RESTORE, "restore" }, \ - { PM_EVENT_RECOVER, "recover" }) + { PM_EVENT_RECOVER, "recover" }, \ + { PM_EVENT_POWEROFF, "poweroff" }) DEFINE_EVENT(cpu, cpu_frequency,
kernel/reboot.c +6 -0
@@ -13,6 +13,7 @@#include <linux/kexec.h> #include <linux/kmod.h> #include <linux/kmsg_dump.h> +#include <linux/pm.h> #include <linux/reboot.h> #include <linux/suspend.h> #include <linux/syscalls.h>
@@ -305,6 +306,11 @@ static void kernel_shutdown_prepare(enum system_states state) (state == SYSTEM_HALT) ? SYS_HALT : SYS_POWER_OFF, NULL); system_state = state;usermodehelper_disable (); +#ifdef CONFIG_HIBERNATE_CALLBACKS +if (!dpm_suspend_start (PMSG_POWEROFF ) && !dpm_suspend_end (PMSG_POWEROFF )) +return ; +pr_emerg ("Failed to power off devices, using shutdown instead.\n" ); +#endif device_shutdown (); }/**
Signature · 1 line
2.43.0
PATCH v3 2/5 PCI: Put PCIe ports with downstream devices into D3 at hibernate
In reply to Mario LimoncielloFrom: Mario Limonciello <[email protected]>
For the suspend flow PCIe ports that have downstream devices are put into the appropriate D3 state when children are not in D0. For the hibernate flow, PCIe ports with downstream devices stay in D0 however. This can lead to PCIe ports that are remained powered on needlessly during hibernate.
Adjust the pci_pm_poweroff_noirq() to follow the same flow as pci_pm_suspend_noirq() in that PCIe ports that are power manageable should without downstream devices in D0 should be put into their appropriate sleep state.
---
v3:
* Split out common code between suspend_noirq() and poweroff_noirq() to a helper function
--- drivers/pci/pci-driver.c | 92 ++++++++++++++++++++++++++-------------- 1 file changed, 59 insertions(+), 33 deletions(-)
drivers/pci/pci-driver.c +59 -33
@@ -844,6 +844,54 @@ static int pci_pm_suspend_late(struct device *dev)return pm_generic_suspend_late (dev); } +/** +* pci_pm_set_prepare_bus_pm +* @pci_dev: pci device +* +* Prepare the device to go into a low power state by saving state +* and configure bus PM policy. +* +* Return: TRUE for bus PM will be used +* FALSE for bus PM will be skipped +*/ +static bool pci_pm_set_prepare_bus_pm (struct pci_dev *pci_dev) +{ +if (!pci_dev->state_saved) { +pci_save_state (pci_dev); + +/* +* If the device is a bridge with a child in D0 below it, +* it needs to stay in D0, so check skip_bus_pm to avoid +* putting it into a low-power state in that case. +*/ +if (!pci_dev->skip_bus_pm &&pci_power_manageable (pci_dev)) +pci_prepare_to_sleep (pci_dev); + } + +pci_dbg (pci_dev,"PCI PM: Sleep power state: %s\n" , +pci_power_name (pci_dev->current_state)); + +if (pci_dev->current_state ==PCI_D0 ) { + pci_dev->skip_bus_pm = true; +/* +* Per PCI PM r1.2, table 6-1, a bridge must be in D0 if any +* downstream device is in D0, so avoid changing the power state +* of the parent bridge by setting the skip_bus_pm flag for it. +*/ +if (pci_dev->bus->self) + pci_dev->bus->self->skip_bus_pm = true; + } + +if (pci_dev->skip_bus_pm &&pm_suspend_no_platform ()) { +pci_dbg (pci_dev,"PCI PM: Skipped\n" ); +return FALSE; + } + +pci_pm_set_unknown_state (pci_dev); + +return TRUE; +} +static int pci_pm_suspend_noirq (struct device *dev) {struct pci_dev *pci_dev =to_pci_dev (dev);
@@ -878,38 +926,8 @@ static int pci_pm_suspend_noirq(struct device *dev) } } -if (!pci_dev->state_saved) { -pci_save_state (pci_dev); - -/* -* If the device is a bridge with a child in D0 below it, -* it needs to stay in D0, so check skip_bus_pm to avoid -* putting it into a low-power state in that case. -*/ -if (!pci_dev->skip_bus_pm &&pci_power_manageable (pci_dev)) -pci_prepare_to_sleep (pci_dev); - } - -pci_dbg (pci_dev,"PCI PM: Suspend power state: %s\n" , -pci_power_name (pci_dev->current_state)); - -if (pci_dev->current_state ==PCI_D0 ) { - pci_dev->skip_bus_pm = true; -/* -* Per PCI PM r1.2, table 6-1, a bridge must be in D0 if any -* downstream device is in D0, so avoid changing the power state -* of the parent bridge by setting the skip_bus_pm flag for it. -*/ -if (pci_dev->bus->self) - pci_dev->bus->self->skip_bus_pm = true; - } - -if (pci_dev->skip_bus_pm &&pm_suspend_no_platform ()) { -pci_dbg (pci_dev,"PCI PM: Skipped\n" ); +if (!pci_pm_set_prepare_bus_pm (pci_dev)) goto Fixup; - } - -pci_pm_set_unknown_state (pci_dev);/* * Some BIOSes from ASUS have a bug: If a USB EHCI host controller's
@@ -1136,6 +1154,8 @@ static int pci_pm_poweroff(struct device *dev)struct pci_dev *pci_dev =to_pci_dev (dev);const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm :NULL ; + pci_dev->skip_bus_pm = false; +if (pci_has_legacy_pm_support (pci_dev))return pci_legacy_suspend (dev,PMSG_HIBERNATE );
@@ -1199,8 +1219,8 @@ static int pci_pm_poweroff_noirq(struct device *dev)return error; } -if (!pci_dev->state_saved && !pci_has_subordinate (pci_dev)) -pci_prepare_to_sleep (pci_dev); +if (!pci_pm_set_prepare_bus_pm (pci_dev)) + goto Fixup;/* * The reason for doing this here is the same as for the analogous code
@@ -1209,6 +1229,7 @@ static int pci_pm_poweroff_noirq(struct device *dev)if (pci_dev->class ==PCI_CLASS_SERIAL_USB_EHCI )pci_write_config_word (pci_dev,PCI_COMMAND ,0 ); +Fixup:pci_fixup_device (pci_fixup_suspend_late, pci_dev);return 0 ;
@@ -1218,10 +1239,15 @@ static int pci_pm_restore_noirq(struct device *dev) {struct pci_dev *pci_dev =to_pci_dev (dev);const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm :NULL ; +pci_power_t prev_state = pci_dev->current_state; +bool skip_bus_pm = pci_dev->skip_bus_pm;pci_pm_default_resume_early (pci_dev);pci_fixup_device (pci_fixup_resume_early, pci_dev); +if (!skip_bus_pm && prev_state == PCI_D3cold) +pci_pm_bridge_power_up_actions (pci_dev); +if (pci_has_legacy_pm_support (pci_dev))return 0 ;
Signature · 1 line
2.43.0
kernel test robot
build botRe: [PATCH v3 2/5] PCI: Put PCIe ports with downstream devices into D3 at hibernate
kernel test robot noticed the following build errors:
Hi Mario,
kernel test robot noticed the following build errors:
[auto build test ERROR on rafael-pm/linux-next]
[also build test ERROR on rafael-pm/bleeding-edge mkp-scsi/for-next jejb-scsi/for-next linus/master v6.16-rc1 next-20250606] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Mario-Limonciello/PM-Use-hibernate-flows-for-system-power-off/20250609-105658 base: https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git linux-next patch link: https://lore.kernel.org/r/20250609024619.407257-3-superm1%40kernel.org patch subject: [PATCH v3 2/5] PCI: Put PCIe ports with downstream devices into D3 at hibernate config: i386-buildonly-randconfig-003-20250609 (https://download.01.org/0day-ci/archive/20250609/[email protected]/config) compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250609/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> drivers/pci/pci-driver.c:1221:7: error: call to undeclared function 'pci_pm_set_prepare_bus_pm'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
1221 | if (!pci_pm_set_prepare_bus_pm(pci_dev))
| ^1 error generated.
vim +/pci_pm_set_prepare_bus_pm +1221 drivers/pci/pci-driver.c
1195
1196 static int pci_pm_poweroff_noirq(struct device *dev)
1197 {
1198 struct pci_dev *pci_dev = to_pci_dev(dev);
1199 const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
1200
1201 if (dev_pm_skip_suspend(dev))
1202 return 0;
1203
1204 if (pci_has_legacy_pm_support(pci_dev))
1205 return pci_legacy_suspend_late(dev);
1206
1207 if (!pm) {
1208 pci_fixup_device(pci_fixup_suspend_late, pci_dev);
1209 return 0;
1210 }
1211
1212 if (pm->poweroff_noirq) {
1213 int error;
1214
1215 error = pm->poweroff_noirq(dev);
1216 suspend_report_result(dev, pm->poweroff_noirq, error);
1217 if (error)
1218 return error;
1219 }
1220
> 1221 if (!pci_pm_set_prepare_bus_pm(pci_dev))
1222 goto Fixup;
1223
1224 /*
1225 * The reason for doing this here is the same as for the analogous code
1226 * in pci_pm_suspend_noirq().
1227 */
1228 if (pci_dev->class == PCI_CLASS_SERIAL_USB_EHCI)
1229 pci_write_config_word(pci_dev, PCI_COMMAND, 0);
1230
1231 Fixup:
1232 pci_fixup_device(pci_fixup_suspend_late, pci_dev);
1233
1234 return 0;
1235 }
1236
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH v3 3/5 drm/amd: Avoid evicting resources at S5
In reply to Mario LimoncielloFrom: Mario Limonciello <[email protected]>
Normally resources are evicted on dGPUs at suspend or hibernate and on APUs at hibernate. These steps are unnecessary when using the S4 callbacks to put the system into S5.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
@@ -4966,6 +4966,10 @@ static int amdgpu_device_evict_resources(struct amdgpu_device *adev)if (!adev->in_s4 && (adev->flags &AMD_IS_APU ))return 0 ; +/* No need to evict when going to S5 through S4 callbacks */ +if (system_state ==SYSTEM_HALT || system_state ==SYSTEM_POWER_OFF ) +return 0 ; + ret =amdgpu_ttm_evict_resources (adev,TTM_PL_VRAM );if (ret)DRM_WARN ("evicting device resources failed\n" );
Signature · 1 line
2.43.0
PATCH v3 4/5 scsi: Add PM_EVENT_POWEROFF into suspend callbacks
In reply to Mario LimoncielloFrom: Mario Limonciello <[email protected]>
When the ACPI core uses hibernation callbacks for shutdown drivers will receive PM_EVENT_POWEROFF and should handle it the same as PM_EVENT_HIBERNATE would have been used.
---
v3:
* New patch
--- drivers/scsi/mesh.c | 1 + drivers/scsi/stex.c | 1 + 2 files changed, 2 insertions(+)
drivers/scsi/mesh.c +1 -0
@@ -1762,6 +1762,7 @@ static int mesh_suspend(struct macio_dev *mdev, pm_message_t mesg)case PM_EVENT_SUSPEND :case PM_EVENT_HIBERNATE :case PM_EVENT_FREEZE : +case PM_EVENT_POWEROFF :break ;default :return 0 ;
drivers/scsi/stex.c +1 -0
@@ -1965,6 +1965,7 @@ static int stex_choice_sleep_mic(struct st_hba *hba, pm_message_t state)case PM_EVENT_SUSPEND :return ST_S3 ;case PM_EVENT_HIBERNATE : +case PM_EVENT_POWEROFF : hba->msi_lock =0 ;return ST_S4 ;default :
Signature · 1 line
2.43.0
PATCH v3 5/5 usb: sl811-hcd: Add PM_EVENT_POWEROFF into suspend callbacks
In reply to Mario LimoncielloFrom: Mario Limonciello <[email protected]>
When the ACPI core uses hibernation callbacks for shutdown drivers will receive PM_EVENT_POWEROFF and should handle it the same as PM_EVENT_HIBERNATE would have been used.
--- drivers/usb/host/sl811-hcd.c | 1 + 1 file changed, 1 insertion(+)
drivers/usb/host/sl811-hcd.c +1 -0
@@ -1748,6 +1748,7 @@ sl811h_suspend(struct platform_device *dev, pm_message_t state)break ;case PM_EVENT_SUSPEND :case PM_EVENT_HIBERNATE : +case PM_EVENT_POWEROFF :case PM_EVENT_PRETHAW :/* explicitly discard hw state */ port_power (sl811,0 );break ;
Signature · 1 line
2.43.0