Skip to contentSkip to conversation

PM: runtime/cpuidle: Improve suspend-to-ram support for cpuidle-psci

PATCH In review Ulf Hansson 3 participants 6 messages Apr 1, 2022 – Apr 19, 2022

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

cover PM: runtime/cpuidle: Improve suspend-to-ram support for cpuidle-psci

Maulik Shah reported a problem with suspend-to-ram for one of the Qualcomm platforms that is using PSCI OSI mode. Briefly described, the genpd on/off notifiers doesn't get sent when the boot CPU is turned on/off, which prevents the needed platform specific operations to be executed.

This series intends to solve these problems.

Note that, I have tested this on Dragonboard 410c, with some local hacks to the PSCI firmware driver. This was needed because the PSCI firmware didn't support SYSTEM_SUSPEND, so I basically made psci_system_suspend_enter() to call cpu_do_idle() and forced a suspend ops to be registered.

Maulik, can you please test this at your setup?

Kind regards
Ulf Hansson

Ulf Hansson (2):
PM: runtime: Allow to call __pm_runtime_set_status() from atomic
context
cpuidle: psci: Improve support for suspend-to-ram for PSCI OSI mode

 drivers/base/power/runtime.c   |  9 ++++---
 drivers/cpuidle/cpuidle-psci.c | 46 ++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 4 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/2 PM: runtime: Allow to call __pm_runtime_set_status() from atomic context

The only two users of __pm_runtime_set_status() are pm_runtime_set_active() and pm_runtime_set_suspended(). These are widely used and should be called from non-atomic context to work as expected. However, it would be convenient to allow them be called from atomic context too, as shown from a subsequent change, so let's add support for this.

---
 drivers/base/power/runtime.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/base/power/runtime.c +5 -4
index d4059e6ffeae..91d757914686 100644
@@ -1210,12 +1210,13 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
 {
 	struct device *parent = dev->parent;
 	bool notify_parent = false;
+	unsigned long flags;
 	int error = 0;
 
 	if (status != RPM_ACTIVE && status != RPM_SUSPENDED)
 		return -EINVAL;
 
-	spin_lock_irq(&dev->power.lock);
+	spin_lock_irqsave(&dev->power.lock, flags);
 
 	/*
 	 * Prevent PM-runtime from being enabled for the device or return an
@@ -1226,7 +1227,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
 	else
 		error = -EAGAIN;
 
-	spin_unlock_irq(&dev->power.lock);
+	spin_unlock_irqrestore(&dev->power.lock, flags);
 
 	if (error)
 		return error;
@@ -1247,7 +1248,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
 		device_links_read_unlock(idx);
 	}
 
-	spin_lock_irq(&dev->power.lock);
+	spin_lock_irqsave(&dev->power.lock, flags);
 
 	if (dev->power.runtime_status == status || !parent)
 		goto out_set;
@@ -1288,7 +1289,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
 		dev->power.runtime_error = 0;
 
  out:
-	spin_unlock_irq(&dev->power.lock);
+	spin_unlock_irqrestore(&dev->power.lock, flags);
 
 	if (notify_parent)
 		pm_request_idle(parent);
Signature · 1 line
2.25.1
Tested-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] PM: runtime: Allow to call __pm_runtime_set_status() from atomic context

Hi,

Quoted 7 lines · Ulf Hansson

The only two users of __pm_runtime_set_status() are pm_runtime_set_active() and pm_runtime_set_suspended(). These are widely used and should be called from non-atomic context to work as expected. However, it would be convenient to allow them be called from atomic context too, as shown from a subsequent change, so let's add support for this.

Thanks,
Maulik

Quoted 50 lines
---
  drivers/base/power/runtime.c | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)
drivers/base/power/runtime.c +5 -4
index d4059e6ffeae..91d757914686 100644
@@ -1210,12 +1210,13 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
  {
  	struct device *parent = dev->parent;
  	bool notify_parent = false;
+	unsigned long flags;
  	int error = 0;
 
  	if (status != RPM_ACTIVE && status != RPM_SUSPENDED)
  		return -EINVAL;
 
-	spin_lock_irq(&dev->power.lock);
+	spin_lock_irqsave(&dev->power.lock, flags);
 
  	/*
  	 * Prevent PM-runtime from being enabled for the device or return an
@@ -1226,7 +1227,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
  	else
  		error = -EAGAIN;
 
-	spin_unlock_irq(&dev->power.lock);
+	spin_unlock_irqrestore(&dev->power.lock, flags);
 
  	if (error)
  		return error;
@@ -1247,7 +1248,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
  		device_links_read_unlock(idx);
  	}
 
-	spin_lock_irq(&dev->power.lock);
+	spin_lock_irqsave(&dev->power.lock, flags);
 
  	if (dev->power.runtime_status == status || !parent)
  		goto out_set;
@@ -1288,7 +1289,7 @@ int __pm_runtime_set_status(struct device *dev, unsigned int status)
  		dev->power.runtime_error = 0;
 
   out:
-	spin_unlock_irq(&dev->power.lock);
+	spin_unlock_irqrestore(&dev->power.lock, flags);
 
  	if (notify_parent)
  		pm_request_idle(parent);
#3 lore raw

PATCH 2/2 cpuidle: psci: Improve support for suspend-to-ram for PSCI OSI mode

In reply to Ulf Hansson

When PSCI OSI mode is supported the syscore flag is set for the CPU devices that becomes attached to their PM domains (genpds). In the suspend-to-idle case, we call dev_pm_genpd_suspend|resume() to allow genpd to properly manage the power-off/on operations (pick an idlestate and manage the on/off notifications).

For suspend-to-ram, dev_pm_genpd_suspend|resume() is currently not being called, which causes a problem that the genpd on/off notifiers do not get sent as expected. This prevents the platform-specific operations from being executed, typically needed just before/after the boot CPU is being turned off/on.

To deal with this problem, let's register a syscore ops for cpuidle-psci when PSCI OSI mode is being used and call dev_pm_genpd_suspend|resume() from them. In this way, genpd regains control of the PM domain topology and then sends the on/off notifications when it's appropriate.

---
 drivers/cpuidle/cpuidle-psci.c | 46 ++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)
drivers/cpuidle/cpuidle-psci.c +46 -0
index b51b5df08450..540105ca0781 100644
@@ -23,6 +23,7 @@
 #include <linux/pm_runtime.h>
 #include <linux/slab.h>
 #include <linux/string.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cpuidle.h>
 
@@ -131,6 +132,49 @@ static int psci_idle_cpuhp_down(unsigned int cpu)
 	return 0;
 }
 
+static void psci_idle_syscore_switch(bool suspend)
+{
+	bool cleared = false;
+	struct device *dev;
+	int cpu;
+
+	for_each_possible_cpu(cpu) {
+		dev = per_cpu_ptr(&psci_cpuidle_data, cpu)->dev;
+
+		if (dev && suspend) {
+			dev_pm_genpd_suspend(dev);
+		} else if (dev) {
+			dev_pm_genpd_resume(dev);
+
+			/* Account for userspace having offlined a CPU. */
+			if (pm_runtime_status_suspended(dev))
+				pm_runtime_set_active(dev);
+
+			/* Clear domain state to re-start fresh. */
+			if (!cleared) {
+				psci_set_domain_state(0);
+				cleared = true;
+			}
+		}
+	}
+}
+
+static int psci_idle_syscore_suspend(void)
+{
+	psci_idle_syscore_switch(true);
+	return 0;
+}
+
+static void psci_idle_syscore_resume(void)
+{
+	psci_idle_syscore_switch(false);
+}
+
+static struct syscore_ops psci_idle_syscore_ops = {
+	.suspend = psci_idle_syscore_suspend,
+	.resume = psci_idle_syscore_resume,
+};
+
 static void psci_idle_init_cpuhp(void)
 {
 	int err;
@@ -138,6 +182,8 @@ static void psci_idle_init_cpuhp(void)
 	if (!psci_cpuidle_use_cpuhp)
 		return;
 
+	register_syscore_ops(&psci_idle_syscore_ops);
+
 	err = cpuhp_setup_state_nocalls(CPUHP_AP_CPU_PM_STARTING,
 					"cpuidle/psci:online",
 					psci_idle_cpuhp_up,
Signature · 1 line
2.25.1
Tested-by on v1 2/2
#4 lore raw

Re: [PATCH 2/2] cpuidle: psci: Improve support for suspend-to-ram for PSCI OSI mode

Hi,

Quoted 12 lines · Ulf Hansson

When PSCI OSI mode is supported the syscore flag is set for the CPU devices that becomes attached to their PM domains (genpds). In the suspend-to-idle case, we call dev_pm_genpd_suspend|resume() to allow genpd to properly manage the power-off/on operations (pick an idlestate and manage the on/off notifications).

For suspend-to-ram, dev_pm_genpd_suspend|resume() is currently not being called, which causes a problem that the genpd on/off notifiers do not get sent as expected. This prevents the platform-specific operations from being executed, typically needed just before/after the boot CPU is being turned off/on.

Quoted 8 lines

To deal with this problem, let's register a syscore ops for cpuidle-psci when PSCI OSI mode is being used and call dev_pm_genpd_suspend|resume() from them. In this way, genpd regains control of the PM domain topology and then sends the on/off notifications when it's appropriate.

Thanks,
Maulik

#5 lore raw
In reply to Ulf Hansson
Quoted 19 lines · Ulf Hansson

Maulik Shah reported a problem with suspend-to-ram for one of the Qualcomm platforms that is using PSCI OSI mode. Briefly described, the genpd on/off notifiers doesn't get sent when the boot CPU is turned on/off, which prevents the needed platform specific operations to be executed.

This series intends to solve these problems.

Note that, I have tested this on Dragonboard 410c, with some local hacks to the PSCI firmware driver. This was needed because the PSCI firmware didn't support SYSTEM_SUSPEND, so I basically made psci_system_suspend_enter() to call cpu_do_idle() and forced a suspend ops to be registered.

Maulik, can you please test this at your setup?

Kind regards
Ulf Hansson

Quoted 8 lines

Ulf Hansson (2):
PM: runtime: Allow to call __pm_runtime_set_status() from atomic
context
cpuidle: psci: Improve support for suspend-to-ram for PSCI OSI mode

 drivers/base/power/runtime.c   |  9 ++++---
 drivers/cpuidle/cpuidle-psci.c | 46 ++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 4 deletions(-)

Both patches applied as 5.19 material (in the PM-sleep category, because they really are system-wide suspend support material).

Thanks!