Skip to contentSkip to conversation

PM / Domains: Add GENPD_FLAG_SUSPEND_ON flag

PATCH In review Sibi Sankar 5 participants 16 messages Aug 11, 2020 – Aug 18, 2020

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

PATCH 1/2 PM / Domains: Add GENPD_FLAG_SUSPEND_ON flag

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

---
 drivers/base/power/domain.c | 3 ++-
 include/linux/pm_domain.h   | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/base/power/domain.c +2 -1
index 2cb5e04cf86cd..ba78ac4a450d4 100644
@@ -129,6 +129,7 @@ static const struct genpd_lock_ops genpd_spin_ops = {
 #define genpd_is_active_wakeup(genpd)	(genpd->flags & GENPD_FLAG_ACTIVE_WAKEUP)
 #define genpd_is_cpu_domain(genpd)	(genpd->flags & GENPD_FLAG_CPU_DOMAIN)
 #define genpd_is_rpm_always_on(genpd)	(genpd->flags & GENPD_FLAG_RPM_ALWAYS_ON)
+#define genpd_is_suspend_on(genpd)	(genpd->flags & GENPD_FLAG_SUSPEND_ON)
 
 static inline bool irq_safe_dev_in_no_sleep_domain(struct device *dev,
 		const struct generic_pm_domain *genpd)
@@ -949,7 +950,7 @@ static void genpd_sync_power_off(struct generic_pm_domain *genpd, bool use_lock,
 {
 	struct gpd_link *link;
 
-	if (!genpd_status_on(genpd) || genpd_is_always_on(genpd))
+	if (!genpd_status_on(genpd) || genpd_is_always_on(genpd) || genpd_is_suspend_on(genpd))
 		return;
 
 	if (genpd->suspended_count != genpd->device_count
include/linux/pm_domain.h +5 -0
index ee11502a575b0..3002a2d68936a 100644
@@ -55,6 +55,10 @@
  *
  * GENPD_FLAG_RPM_ALWAYS_ON:	Instructs genpd to always keep the PM domain
  *				powered on except for system suspend.
+ *
+ * GENPD_FLAG_SUSPEND_ON:	Instructs genpd to keep the PM domain powered
+ *				on during suspend and runtime PM controlled
+ *				otherwise.
  */
 #define GENPD_FLAG_PM_CLK	 (1U << 0)
 #define GENPD_FLAG_IRQ_SAFE	 (1U << 1)
@@ -62,6 +66,7 @@
 #define GENPD_FLAG_ACTIVE_WAKEUP (1U << 3)
 #define GENPD_FLAG_CPU_DOMAIN	 (1U << 4)
 #define GENPD_FLAG_RPM_ALWAYS_ON (1U << 5)
+#define GENPD_FLAG_SUSPEND_ON	 (1U << 6)
 
 enum gpd_status {
 	GPD_STATE_ACTIVE = 0,	/* PM domain is active */
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#1 lore raw

PATCH 2/2 soc: qcom: aoss: Use GENPD_FLAG_SUSPEND_ON flag

All the power domains exposed as part of AOSS QMP driver require to stay powered on for suspend. They are powered on when the remote processors boots up and powered off on remote processor crash/shutdown. Mark the power domains with GENPD_FLAG_SUSPEND_ON to model this behavior.

---
 drivers/soc/qcom/qcom_aoss.c | 1 +
 1 file changed, 1 insertion(+)
drivers/soc/qcom/qcom_aoss.c +1 -0
index ed2c687c16b31..5a5b4bf928147 100644
@@ -366,6 +366,7 @@ static int qmp_pd_add(struct qmp *qmp)
 		res[i].pd.name = sdm845_resources[i];
 		res[i].pd.power_on = qmp_pd_power_on;
 		res[i].pd.power_off = qmp_pd_power_off;
+		res[i].pd.flags = GENPD_FLAG_SUSPEND_ON;
 
 		ret = pm_genpd_init(&res[i].pd, NULL, true);
 		if (ret < 0) {
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
Reviewed-by Tested-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] soc: qcom: aoss: Use GENPD_FLAG_SUSPEND_ON flag

Hi,

Quoted 16 lines · Sibi Sankar

All the power domains exposed as part of AOSS QMP driver require to stay powered on for suspend. They are powered on when the remote processors boots up and powered off on remote processor crash/shutdown. Mark the power domains with GENPD_FLAG_SUSPEND_ON to model this behavior.

---
 drivers/soc/qcom/qcom_aoss.c | 1 +
 1 file changed, 1 insertion(+)
drivers/soc/qcom/qcom_aoss.c +0 -0
index ed2c687c16b31..5a5b4bf928147 100644

drivers/soc/qcom/qcom_aoss.c static int qmp_pd_add(struct qmp *qmp) · line 369

Quoted 4 lines
drivers/soc/qcom/qcom_aoss.c +1 -0
@@ -366,6 +366,7 @@ static int qmp_pd_add(struct qmp *qmp)
                res[i].pd.name = sdm845_resources[i];
                res[i].pd.power_on = qmp_pd_power_on;
                res[i].pd.power_off = qmp_pd_power_off;
+               res[i].pd.flags = GENPD_FLAG_SUSPEND_ON;

This makes my patch [1] unnecessary.

[1] https://lore.kernel.org/r/20200805091141.1.I86b3faaecb0d82997b599b1300f879606c71e116@changeid

-Doug

Reviewed-by Tested-by on v1 1/2
#3 lore raw
In reply to Sibi Sankar

Hi,

Quoted 7 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines
---
 drivers/base/power/domain.c | 3 ++-
 include/linux/pm_domain.h   | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)

Seems sane to me.

#4 lore raw
In reply to Sibi Sankar

Quoting Sibi Sankar (2020-08-11 12:02:51)

include/linux/pm_domain.h line 60

Quoted 11 lines
include/linux/pm_domain.h +3 -0
index ee11502a575b0..3002a2d68936a 100644
@@ -55,6 +55,10 @@
  *
  * GENPD_FLAG_RPM_ALWAYS_ON:   Instructs genpd to always keep the PM domain
  *                             powered on except for system suspend.
+ *
+ * GENPD_FLAG_SUSPEND_ON:      Instructs genpd to keep the PM domain powered
+ *                             on during suspend and runtime PM controlled

Maybe, "powered on across system suspend (if it is already powered on)" to match the hunk above that talks about system suspend for GENPD_FLAG_RPM_ALWAYS_ON. Otherwise someone may think that this powers on the genpd during suspend or powers it on during runtime suspend.

Quoted 4 lines
include/linux/pm_domain.h +1 -0
+ *                             otherwise.
  */
 #define GENPD_FLAG_PM_CLK       (1U << 0)
 #define GENPD_FLAG_IRQ_SAFE     (1U << 1)
#5 lore raw

Hey Stephen,
Thanks for taking time to review the
series!

Quoted 12 lines · Stephen Boyd

Quoting Sibi Sankar (2020-08-11 12:02:51)

Quoted 11 lines
include/linux/pm_domain.h +0 -0
index ee11502a575b0..3002a2d68936a 100644
@@ -55,6 +55,10 @@
  *
  * GENPD_FLAG_RPM_ALWAYS_ON:   Instructs genpd to always keep the PM 

domain
* powered on except for system suspend.
+ *
+ * GENPD_FLAG_SUSPEND_ON: Instructs genpd to keep the PM domain

Quoted 8 lines
Quoted 3 lines

powered
+ * on during suspend and runtime PM
controlled

Maybe, "powered on across system suspend (if it is already powered on)" to match the hunk above that talks about system suspend for GENPD_FLAG_RPM_ALWAYS_ON. Otherwise someone may think that this powers on the genpd during suspend or powers it on during runtime suspend.

Sure, I'll add ^^ in the next re-spin.

Quoted 5 lines
Quoted 5 lines

+ * otherwise.
*/
#define GENPD_FLAG_PM_CLK (1U << 0)
#define GENPD_FLAG_IRQ_SAFE (1U << 1)

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
#6 lore raw
In reply to Sibi Sankar
Quoted 7 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Seems like a useful use-case, but i think there should be a bit more description/documentation about what is the expected/desired behavior during system suspsend when a power-domain with this flag is already runtime-PM suspended. Similarily, on system resume, what is the expected/desired behavior?

Kevin

#7 lore raw

Kevin,
Thanks for taking time to review the
series!

Quoted 8 lines · Kevin Hilman
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 7 lines
Quoted 1 line

Seems like a useful use-case, but i think there should be a bit more description/documentation about what is the expected/desired behavior during system suspsend when a power-domain with this flag is already runtime-PM suspended. Similarily, on system resume, what is the expected/desired behavior?

SUSPEND_ON flag is only aimed at
keeping power domains powered on
across suspend (only if its already
powered on). Also if the power domain
is runtime-PM suspended we wouldn't
want to power it on during resume.

drivers/base/power/domain.c +1 -1
index 0a5afca250d03..547c091618008 100644
@@ -1003,7 +1003,7 @@ static void genpd_sync_power_on(struct generic_pm_domain *genpd, bool use_lock,
 {
        struct gpd_link *link;
 
-       if (genpd_status_on(genpd))
+       if (genpd_status_on(genpd) || genpd_is_suspend_on(genpd))
                return;
 

I'll add the ^^ diff in the next
re-spin to prevent power on of
a runtime-PM suspended power
domain.

Quoted 2 lines

Kevin

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
#8 lore raw
In reply to Sibi Sankar
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Kind regards
Uffe

Quoted 55 lines
---
 drivers/base/power/domain.c | 3 ++-
 include/linux/pm_domain.h   | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/base/power/domain.c +2 -1
index 2cb5e04cf86cd..ba78ac4a450d4 100644
@@ -129,6 +129,7 @@ static const struct genpd_lock_ops genpd_spin_ops = {
 #define genpd_is_active_wakeup(genpd)  (genpd->flags & GENPD_FLAG_ACTIVE_WAKEUP)
 #define genpd_is_cpu_domain(genpd)     (genpd->flags & GENPD_FLAG_CPU_DOMAIN)
 #define genpd_is_rpm_always_on(genpd)  (genpd->flags & GENPD_FLAG_RPM_ALWAYS_ON)
+#define genpd_is_suspend_on(genpd)     (genpd->flags & GENPD_FLAG_SUSPEND_ON)
 
 static inline bool irq_safe_dev_in_no_sleep_domain(struct device *dev,
                const struct generic_pm_domain *genpd)
@@ -949,7 +950,7 @@ static void genpd_sync_power_off(struct generic_pm_domain *genpd, bool use_lock,
 {
        struct gpd_link *link;
 
-       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd))
+       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd) || genpd_is_suspend_on(genpd))
                return;
 
        if (genpd->suspended_count != genpd->device_count
include/linux/pm_domain.h +5 -1
index ee11502a575b0..3002a2d68936a 100644
@@ -55,6 +55,10 @@
  *
  * GENPD_FLAG_RPM_ALWAYS_ON:   Instructs genpd to always keep the PM domain
  *                             powered on except for system suspend.
+ *
+ * GENPD_FLAG_SUSPEND_ON:      Instructs genpd to keep the PM domain powered
+ *                             on during suspend and runtime PM controlled
+ *                             otherwise.
  */
 #define GENPD_FLAG_PM_CLK       (1U << 0)
 #define GENPD_FLAG_IRQ_SAFE     (1U << 1)
@@ -62,6 +66,7 @@
 #define GENPD_FLAG_ACTIVE_WAKEUP (1U << 3)
 #define GENPD_FLAG_CPU_DOMAIN   (1U << 4)
 #define GENPD_FLAG_RPM_ALWAYS_ON (1U << 5)
+#define GENPD_FLAG_SUSPEND_ON   (1U << 6)
 
 enum gpd_status {
        GPD_STATE_ACTIVE = 0,   /* PM domain is active */
--

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project

#9 lore raw

Uffe,
Thanks for taking time to review the series!

Quoted 8 lines · Ulf Hansson
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Yes this is a co-processor behavior that needs to be obeyed. Specifically application processor notifies the Always on Subsystem (AOSS) that a particular co-processor is up using the power domains exposed by AOSS QMP driver. AOSS uses this information to wait for the co-processors to suspend before starting its sleep sequence. The application processor powers off these power domains only if the co-processor has crashed or powered off.

Quoted 70 lines

Kind regards
Uffe

Quoted 66 lines
---
 drivers/base/power/domain.c | 3 ++-
 include/linux/pm_domain.h   | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/base/power/domain.c +0 -0
index 2cb5e04cf86cd..ba78ac4a450d4 100644
@@ -129,6 +129,7 @@ static const struct genpd_lock_ops genpd_spin_ops 
 = {
 #define genpd_is_active_wakeup(genpd)  (genpd->flags & 
 GENPD_FLAG_ACTIVE_WAKEUP)
 #define genpd_is_cpu_domain(genpd)     (genpd->flags & 
 GENPD_FLAG_CPU_DOMAIN)
 #define genpd_is_rpm_always_on(genpd)  (genpd->flags & 

GENPD_FLAG_RPM_ALWAYS_ON)
+#define genpd_is_suspend_on(genpd) (genpd->flags &
GENPD_FLAG_SUSPEND_ON)

static inline bool irq_safe_dev_in_no_sleep_domain(struct device
*dev,
const struct generic_pm_domain *genpd)

drivers/base/power/domain.c +1 -1
@@ -949,7 +950,7 @@ static void genpd_sync_power_off(struct 
 generic_pm_domain *genpd, bool use_lock,
 {
        struct gpd_link *link;
 
-       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd))
+       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd) || 
 genpd_is_suspend_on(genpd))
                return;
        if (genpd->suspended_count != genpd->device_count
include/linux/pm_domain.h +0 -0
index ee11502a575b0..3002a2d68936a 100644
@@ -55,6 +55,10 @@
  *
  * GENPD_FLAG_RPM_ALWAYS_ON:   Instructs genpd to always keep the PM 

domain
* powered on except for system suspend.
+ *
+ * GENPD_FLAG_SUSPEND_ON: Instructs genpd to keep the PM domain
powered
+ * on during suspend and runtime PM
controlled
+ * otherwise.
*/
#define GENPD_FLAG_PM_CLK (1U << 0)
#define GENPD_FLAG_IRQ_SAFE (1U << 1)

include/linux/pm_domain.h +1 -1
@@ -62,6 +66,7 @@
 #define GENPD_FLAG_ACTIVE_WAKEUP (1U << 3)
 #define GENPD_FLAG_CPU_DOMAIN   (1U << 4)
 #define GENPD_FLAG_RPM_ALWAYS_ON (1U << 5)
+#define GENPD_FLAG_SUSPEND_ON   (1U << 6)
 
 enum gpd_status {
        GPD_STATE_ACTIVE = 0,   /* PM domain is active */
--

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
#10 lore raw
Quoted 20 lines · Sibi Sankar

Uffe,
Thanks for taking time to review the series!

Quoted 8 lines · Ulf Hansson
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Quoted 11 lines

Yes this is a co-processor behavior that needs to be obeyed. Specifically application processor notifies the Always on Subsystem (AOSS) that a particular co-processor is up using the power domains exposed by AOSS QMP driver. AOSS uses this information to wait for the co-processors to suspend before starting its sleep sequence. The application processor powers off these power domains only if the co-processor has crashed or powered off.

Thanks for clarifying!

Although, can you please elaborate a bit more on the actual use case? What are the typical co-processor and what drivers are involved in managing it?

As you may know, runtime PM becomes disabled during system suspend of a device. Which means, if the driver tries to power off the coprocessor (via calling pm_runtime_put() for example), somewhere in the system suspend phase of the corresponding device, its attached PM domain stays powered on when managed by genpd.

Then in the suspend_noirq phase, genpd tries to power off the PM domain, unless there are wakeups to consider.

Taking the above into account, wouldn't that mean that you potentially may end up keeping the PM domain powered on, even if it actually can be powered off in the suspend_noirq phase by genpd?

Kind regards
Uffe

Quoted 72 lines
Quoted 61 lines
Quoted 61 lines
---
 drivers/base/power/domain.c | 3 ++-
 include/linux/pm_domain.h   | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/base/power/domain.c +0 -0
index 2cb5e04cf86cd..ba78ac4a450d4 100644
@@ -129,6 +129,7 @@ static const struct genpd_lock_ops genpd_spin_ops
 = {
 #define genpd_is_active_wakeup(genpd)  (genpd->flags &
 GENPD_FLAG_ACTIVE_WAKEUP)
 #define genpd_is_cpu_domain(genpd)     (genpd->flags &
 GENPD_FLAG_CPU_DOMAIN)
 #define genpd_is_rpm_always_on(genpd)  (genpd->flags &

GENPD_FLAG_RPM_ALWAYS_ON)
+#define genpd_is_suspend_on(genpd) (genpd->flags &
GENPD_FLAG_SUSPEND_ON)

static inline bool irq_safe_dev_in_no_sleep_domain(struct device
*dev,
const struct generic_pm_domain *genpd)

drivers/base/power/domain.c +1 -1
@@ -949,7 +950,7 @@ static void genpd_sync_power_off(struct
 generic_pm_domain *genpd, bool use_lock,
 {
        struct gpd_link *link;
 
-       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd))
+       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd) ||
 genpd_is_suspend_on(genpd))
                return;
        if (genpd->suspended_count != genpd->device_count
include/linux/pm_domain.h +0 -0
index ee11502a575b0..3002a2d68936a 100644
@@ -55,6 +55,10 @@
  *
  * GENPD_FLAG_RPM_ALWAYS_ON:   Instructs genpd to always keep the PM

domain
* powered on except for system suspend.
+ *
+ * GENPD_FLAG_SUSPEND_ON: Instructs genpd to keep the PM domain
powered
+ * on during suspend and runtime PM
controlled
+ * otherwise.
*/
#define GENPD_FLAG_PM_CLK (1U << 0)
#define GENPD_FLAG_IRQ_SAFE (1U << 1)

include/linux/pm_domain.h +1 -0
@@ -62,6 +66,7 @@
 #define GENPD_FLAG_ACTIVE_WAKEUP (1U << 3)
 #define GENPD_FLAG_CPU_DOMAIN   (1U << 4)
 #define GENPD_FLAG_RPM_ALWAYS_ON (1U << 5)
+#define GENPD_FLAG_SUSPEND_ON   (1U << 6)
 
Quoted 7 lines
Quoted 7 lines
include/linux/pm_domain.h +0 -1
 enum gpd_status {
        GPD_STATE_ACTIVE = 0,   /* PM domain is active */
--

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.

#11 lore raw
Quoted 33 lines · Ulf Hansson
Quoted 20 lines · Sibi Sankar

Uffe,
Thanks for taking time to review the series!

Quoted 8 lines · Ulf Hansson
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Quoted 11 lines

Yes this is a co-processor behavior that needs to be obeyed. Specifically application processor notifies the Always on Subsystem (AOSS) that a particular co-processor is up using the power domains exposed by AOSS QMP driver. AOSS uses this information to wait for the co-processors to suspend before starting its sleep sequence. The application processor powers off these power domains only if the co-processor has crashed or powered off.

Quoted 5 lines

Thanks for clarifying!

Although, can you please elaborate a bit more on the actual use case? What are the typical co-processor and what drivers are involved in managing it?

The co-processors using the power domains exposed by qcom_aoss driver are modem, audio dsp, compute dsp managed using qcom_q6v5_mss and qcom_q6v5_pas driver.

Quoted 6 lines

As you may know, runtime PM becomes disabled during system suspend of a device. Which means, if the driver tries to power off the coprocessor (via calling pm_runtime_put() for example), somewhere in the system suspend phase of the corresponding device, its attached PM domain stays powered on when managed by genpd.

The drivers aren't really expected do anything during suspend/resume pretty much because the co-processors enter low-power modes independent to that of the application processor. On co-processor crash the remoteproc core does a pm_stay_awake followed by a pm_relax after crash recovery.

Quoted 83 lines

Then in the suspend_noirq phase, genpd tries to power off the PM domain, unless there are wakeups to consider.

Taking the above into account, wouldn't that mean that you potentially may end up keeping the PM domain powered on, even if it actually can be powered off in the suspend_noirq phase by genpd?

Kind regards
Uffe

Quoted 72 lines
Quoted 61 lines
Quoted 61 lines
---
 drivers/base/power/domain.c | 3 ++-
 include/linux/pm_domain.h   | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/base/power/domain.c +0 -0
index 2cb5e04cf86cd..ba78ac4a450d4 100644
@@ -129,6 +129,7 @@ static const struct genpd_lock_ops genpd_spin_ops
 = {
 #define genpd_is_active_wakeup(genpd)  (genpd->flags &
 GENPD_FLAG_ACTIVE_WAKEUP)
 #define genpd_is_cpu_domain(genpd)     (genpd->flags &
 GENPD_FLAG_CPU_DOMAIN)
 #define genpd_is_rpm_always_on(genpd)  (genpd->flags &

GENPD_FLAG_RPM_ALWAYS_ON)
+#define genpd_is_suspend_on(genpd) (genpd->flags &
GENPD_FLAG_SUSPEND_ON)

static inline bool irq_safe_dev_in_no_sleep_domain(struct device
*dev,
const struct generic_pm_domain *genpd)

drivers/base/power/domain.c +1 -1
@@ -949,7 +950,7 @@ static void genpd_sync_power_off(struct
 generic_pm_domain *genpd, bool use_lock,
 {
        struct gpd_link *link;
 
-       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd))
+       if (!genpd_status_on(genpd) || genpd_is_always_on(genpd) ||
 genpd_is_suspend_on(genpd))
                return;
        if (genpd->suspended_count != genpd->device_count
include/linux/pm_domain.h +0 -0
index ee11502a575b0..3002a2d68936a 100644
@@ -55,6 +55,10 @@
  *
  * GENPD_FLAG_RPM_ALWAYS_ON:   Instructs genpd to always keep the PM

domain
* powered on except for system suspend.
+ *
+ * GENPD_FLAG_SUSPEND_ON: Instructs genpd to keep the PM domain
powered
+ * on during suspend and runtime PM
controlled
+ * otherwise.
*/
#define GENPD_FLAG_PM_CLK (1U << 0)
#define GENPD_FLAG_IRQ_SAFE (1U << 1)

include/linux/pm_domain.h +1 -0
@@ -62,6 +66,7 @@
 #define GENPD_FLAG_ACTIVE_WAKEUP (1U << 3)
 #define GENPD_FLAG_CPU_DOMAIN   (1U << 4)
 #define GENPD_FLAG_RPM_ALWAYS_ON (1U << 5)
+#define GENPD_FLAG_SUSPEND_ON   (1U << 6)
 
Quoted 7 lines
Quoted 7 lines
include/linux/pm_domain.h +0 -1
 enum gpd_status {
        GPD_STATE_ACTIVE = 0,   /* PM domain is active */
--

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
#12 lore raw
Quoted 53 lines · Sibi Sankar
Quoted 33 lines · Ulf Hansson
Quoted 20 lines · Sibi Sankar

Uffe,
Thanks for taking time to review the series!

Quoted 8 lines · Ulf Hansson
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Quoted 11 lines

Yes this is a co-processor behavior that needs to be obeyed. Specifically application processor notifies the Always on Subsystem (AOSS) that a particular co-processor is up using the power domains exposed by AOSS QMP driver. AOSS uses this information to wait for the co-processors to suspend before starting its sleep sequence. The application processor powers off these power domains only if the co-processor has crashed or powered off.

Quoted 5 lines

Thanks for clarifying!

Although, can you please elaborate a bit more on the actual use case? What are the typical co-processor and what drivers are involved in managing it?

The co-processors using the power domains exposed by qcom_aoss driver are modem, audio dsp, compute dsp managed using qcom_q6v5_mss and qcom_q6v5_pas driver.

Quoted 6 lines

As you may know, runtime PM becomes disabled during system suspend of a device. Which means, if the driver tries to power off the coprocessor (via calling pm_runtime_put() for example), somewhere in the system suspend phase of the corresponding device, its attached PM domain stays powered on when managed by genpd.

Quoted 8 lines

The drivers aren't really expected do anything during suspend/resume pretty much because the co-processors enter low-power modes independent to that of the application processor. On co-processor crash the remoteproc core does a pm_stay_awake followed by a pm_relax after crash recovery.

Okay, thanks again for clarifying. You have convinced me about the need for a new flag to cope with these use cases.

Would you mind updating the commit message with some of the information you just provided?

Additionally, to make it clear that the flag should be used to keep the PM domain powered on during system suspend, but only if it's already powered on - please rename the flag to GENPD_FLAG_NO_SUSPEND, and update the corresponding description of it in the header file.

[...]

Kind regards
Uffe

#13 lore raw
Quoted 66 lines · Ulf Hansson
Quoted 53 lines · Sibi Sankar
Quoted 33 lines · Ulf Hansson
Quoted 20 lines · Sibi Sankar

Uffe,
Thanks for taking time to review the series!

Quoted 8 lines · Ulf Hansson
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Quoted 11 lines

Yes this is a co-processor behavior that needs to be obeyed. Specifically application processor notifies the Always on Subsystem (AOSS) that a particular co-processor is up using the power domains exposed by AOSS QMP driver. AOSS uses this information to wait for the co-processors to suspend before starting its sleep sequence. The application processor powers off these power domains only if the co-processor has crashed or powered off.

Quoted 5 lines

Thanks for clarifying!

Although, can you please elaborate a bit more on the actual use case? What are the typical co-processor and what drivers are involved in managing it?

The co-processors using the power domains exposed by qcom_aoss driver are modem, audio dsp, compute dsp managed using qcom_q6v5_mss and qcom_q6v5_pas driver.

Quoted 6 lines

As you may know, runtime PM becomes disabled during system suspend of a device. Which means, if the driver tries to power off the coprocessor (via calling pm_runtime_put() for example), somewhere in the system suspend phase of the corresponding device, its attached PM domain stays powered on when managed by genpd.

Quoted 8 lines

The drivers aren't really expected do anything during suspend/resume pretty much because the co-processors enter low-power modes independent to that of the application processor. On co-processor crash the remoteproc core does a pm_stay_awake followed by a pm_relax after crash recovery.

Okay, thanks again for clarifying. You have convinced me about the need for a new flag to cope with these use cases.

Quoted 7 lines

Would you mind updating the commit message with some of the information you just provided?

Additionally, to make it clear that the flag should be used to keep the PM domain powered on during system suspend, but only if it's already powered on - please rename the flag to GENPD_FLAG_NO_SUSPEND, and update the corresponding description of it in the header file.

Thanks, naming it ^^ makes more sense :)

https://lore.kernel.org/lkml/[email protected]/

Also we wouldn't want to power on
runtime suspended power domains with
the NO_SUSPEND flag set, on resume as
explained ^^. Do you agree with that
as well?

Quoted 5 lines

[...]

Kind regards
Uffe

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.
#14 lore raw
Quoted 78 lines · Sibi Sankar
Quoted 66 lines · Ulf Hansson
Quoted 53 lines · Sibi Sankar
Quoted 33 lines · Ulf Hansson
Quoted 20 lines · Sibi Sankar

Uffe,
Thanks for taking time to review the series!

Quoted 8 lines · Ulf Hansson
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Quoted 11 lines

Yes this is a co-processor behavior that needs to be obeyed. Specifically application processor notifies the Always on Subsystem (AOSS) that a particular co-processor is up using the power domains exposed by AOSS QMP driver. AOSS uses this information to wait for the co-processors to suspend before starting its sleep sequence. The application processor powers off these power domains only if the co-processor has crashed or powered off.

Quoted 5 lines

Thanks for clarifying!

Although, can you please elaborate a bit more on the actual use case? What are the typical co-processor and what drivers are involved in managing it?

The co-processors using the power domains exposed by qcom_aoss driver are modem, audio dsp, compute dsp managed using qcom_q6v5_mss and qcom_q6v5_pas driver.

Quoted 6 lines

As you may know, runtime PM becomes disabled during system suspend of a device. Which means, if the driver tries to power off the coprocessor (via calling pm_runtime_put() for example), somewhere in the system suspend phase of the corresponding device, its attached PM domain stays powered on when managed by genpd.

Quoted 8 lines

The drivers aren't really expected do anything during suspend/resume pretty much because the co-processors enter low-power modes independent to that of the application processor. On co-processor crash the remoteproc core does a pm_stay_awake followed by a pm_relax after crash recovery.

Okay, thanks again for clarifying. You have convinced me about the need for a new flag to cope with these use cases.

Quoted 7 lines

Would you mind updating the commit message with some of the information you just provided?

Additionally, to make it clear that the flag should be used to keep the PM domain powered on during system suspend, but only if it's already powered on - please rename the flag to GENPD_FLAG_NO_SUSPEND, and update the corresponding description of it in the header file.

Thanks, naming it ^^ makes more sense :)

Quoted 7 lines

https://lore.kernel.org/lkml/[email protected]/

Also we wouldn't want to power on runtime suspended power domains with the NO_SUSPEND flag set, on resume as explained ^^. Do you agree with that as well?

Actually no.

Instead, I think that deserves a separate flag, as it may very well turn out that resuming can be skipped for other cases than "NO_SUSPEND".

Therefore, please add a GENPD_FLAG_NO_RESUME for this.

Kind regards
Uffe

#15 lore raw
Quoted 87 lines · Ulf Hansson
Quoted 78 lines · Sibi Sankar
Quoted 66 lines · Ulf Hansson
Quoted 53 lines · Sibi Sankar
Quoted 33 lines · Ulf Hansson
Quoted 20 lines · Sibi Sankar

Uffe,
Thanks for taking time to review the series!

Quoted 8 lines · Ulf Hansson
Quoted 6 lines · Sibi Sankar

This is for power domains which needs to stay powered on for suspend but can be powered on/off as part of runtime PM. This flag is aimed at power domains coupled to remote processors which enter suspend states independent to that of the application processor. Such power domains are turned off only on remote processor crash/shutdown.

Quoted 5 lines

As Kevin also requested, please elaborate more on the use case.

Why exactly must the PM domain stay powered on during system suspend? Is there a wakeup configured that needs to be managed - or is there a co-processor/FW behaviour that needs to be obeyed to?

Quoted 11 lines

Yes this is a co-processor behavior that needs to be obeyed. Specifically application processor notifies the Always on Subsystem (AOSS) that a particular co-processor is up using the power domains exposed by AOSS QMP driver. AOSS uses this information to wait for the co-processors to suspend before starting its sleep sequence. The application processor powers off these power domains only if the co-processor has crashed or powered off.

Quoted 5 lines

Thanks for clarifying!

Although, can you please elaborate a bit more on the actual use case? What are the typical co-processor and what drivers are involved in managing it?

The co-processors using the power domains exposed by qcom_aoss driver are modem, audio dsp, compute dsp managed using qcom_q6v5_mss and qcom_q6v5_pas driver.

Quoted 6 lines

As you may know, runtime PM becomes disabled during system suspend of a device. Which means, if the driver tries to power off the coprocessor (via calling pm_runtime_put() for example), somewhere in the system suspend phase of the corresponding device, its attached PM domain stays powered on when managed by genpd.

Quoted 8 lines

The drivers aren't really expected do anything during suspend/resume pretty much because the co-processors enter low-power modes independent to that of the application processor. On co-processor crash the remoteproc core does a pm_stay_awake followed by a pm_relax after crash recovery.

Okay, thanks again for clarifying. You have convinced me about the need for a new flag to cope with these use cases.

Quoted 7 lines

Would you mind updating the commit message with some of the information you just provided?

Additionally, to make it clear that the flag should be used to keep the PM domain powered on during system suspend, but only if it's already powered on - please rename the flag to GENPD_FLAG_NO_SUSPEND, and update the corresponding description of it in the header file.

Thanks, naming it ^^ makes more sense :)

Quoted 7 lines

https://lore.kernel.org/lkml/[email protected]/

Also we wouldn't want to power on runtime suspended power domains with the NO_SUSPEND flag set, on resume as explained ^^. Do you agree with that as well?

Quoted 7 lines

Actually no.

Instead, I think that deserves a separate flag, as it may very well turn out that resuming can be skipped for other cases than "NO_SUSPEND".

Therefore, please add a GENPD_FLAG_NO_RESUME for this.

Thanks I'll do that in v2

Quoted 3 lines

Kind regards
Uffe

Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project.