Skip to contentSkip to conversation

arm64: Disallow disabling boot CPU based on config

PATCH In review Sneh Mankad 6 participants 14 messages Jul 3 – Jul 28

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 arm64: Disallow disabling boot CPU based on config

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

If CPU0 is already offline before starting suspend to ram the freeze_secondary_cpus() picks alternate CPU as primary / last CPU and proceed further to invoke PSCI SYSTEM_SUSPEND.
This leads to a system crash.

In order to prevent such an issue introduce PM_SLEEP_SMP_CPU_ZERO_STRICT config and when enabled prohibit the CPU0 from getting disabled.

---
Changes in v2:
- Moved the check to arm64 specific code.
- Link to v1: https://lore.kernel.org/r/20260605-disable_boot_cpu_offline-v1-1-4c68fe1a6cf8@oss.qualcomm.com

---
 arch/arm64/Kconfig       | 9 +++++++++
 arch/arm64/kernel/psci.c | 6 ++++++
 2 files changed, 15 insertions(+)
arch/arm64/Kconfig +9 -0
index fe60738e5943ba279e5571862423df4fed3db661..21697a535a25d286a2f8afe4921a41b13cc32c0a 100644
@@ -263,6 +263,15 @@ config ARM64
 	help
 	  ARM 64-bit (AArch64) Linux support.
 
+config PM_SLEEP_SMP_CPU_ZERO_STRICT
+       bool "Disallow boot CPU (CPU0) offline"
+       depends on ARCH_QCOM
+       depends on HOTPLUG_CPU
+       depends on SUSPEND
+       help
+         Disallow boot CPU (CPU0) offline when the suspend_ops->enter()
+         has to be executed by boot CPU.
+
 config RUSTC_SUPPORTS_ARM64
 	def_bool y
 	depends on CPU_LITTLE_ENDIAN
arch/arm64/kernel/psci.c +6 -0
index fabd732d0a2dfee37074ef4ebb6ce5894871c8bd..4ad90ae6f8bacf0cbd3203d66580107d467ea232 100644
@@ -49,6 +49,12 @@ static int cpu_psci_cpu_boot(unsigned int cpu)
 #ifdef CONFIG_HOTPLUG_CPU
 static bool cpu_psci_cpu_can_disable(unsigned int cpu)
 {
+#ifdef CONFIG_PM_SLEEP_SMP_CPU_ZERO_STRICT
+	if (cpu == get_boot_cpu_id()) {
+		pr_info("Disabling boot CPU is not supported\n");
+		return false;
+	}
+#endif
 	return !psci_tos_resident_on(cpu);
 }
 

---
base-commit: ba3e43a9e601636f5edb54e259a74f96ca3b8fd8
change-id: 20260603-disable_boot_cpu_offline-eb4f55ac96f2

Best regards,

Signature · 1 line
Sneh Mankad <[email protected]>
reviewer
#1 lore raw
Quoted 1 line · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco

Are those released products?

Are those mobile phone parts, or somthing else?

Quoted 9 lines

support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

If CPU0 is already offline before starting suspend to ram the freeze_secondary_cpus() picks alternate CPU as primary / last CPU and proceed further to invoke PSCI SYSTEM_SUSPEND.
This leads to a system crash.

Ok, so that's a firmware bug.

Why does the FW permit CPU0 to be offlined in the first place if it can't handle this?

What does PSCI_MIGRATE_INFO_TYPE report?

Ideally it'd report Uniprocessor (UP) not migrate capable (1), which would prevent CPU_OFF on that CPU, and would force suspend to happen there...

Quoted 2 lines

In order to prevent such an issue introduce PM_SLEEP_SMP_CPU_ZERO_STRICT config and when enabled prohibit the CPU0 from getting disabled.

I don't think it makes sense for this to be a config option.

This is a platform-specific property, and it's possible to build a kernel that boots on this platform and/or other platforms.

Quoted 15 lines

---
Changes in v2:
- Moved the check to arm64 specific code.
- Link to v1: https://lore.kernel.org/r/20260605-disable_boot_cpu_offline-v1-1-4c68fe1a6cf8@oss.qualcomm.com

---
 arch/arm64/Kconfig       | 9 +++++++++
 arch/arm64/kernel/psci.c | 6 ++++++
 2 files changed, 15 insertions(+)
arch/arm64/Kconfig +0 -0
index fe60738e5943ba279e5571862423df4fed3db661..21697a535a25d286a2f8afe4921a41b13cc32c0a 100644

arch/arm64/Kconfig config ARM64 · line 268

Quoted 6 lines
arch/arm64/Kconfig +3 -0
@@ -263,6 +263,15 @@ config ARM64
 	help
 	  ARM 64-bit (AArch64) Linux support.
 
+config PM_SLEEP_SMP_CPU_ZERO_STRICT
+       bool "Disallow boot CPU (CPU0) offline"
+       depends on ARCH_QCOM

Why can't others select this?

arch/arm64/Kconfig config ARM64 · line 273

Quoted 5 lines
arch/arm64/Kconfig +5 -0
+       depends on HOTPLUG_CPU
+       depends on SUSPEND
+       help
+         Disallow boot CPU (CPU0) offline when the suspend_ops->enter()
+         has to be executed by boot CPU.

As above, I don't think this makse sense as a config option.

Either we handle the FW bug, or we do not.

Mark.

Quoted 31 lines
arch/arm64/Kconfig +1 -0
+
 config RUSTC_SUPPORTS_ARM64
 	def_bool y
 	depends on CPU_LITTLE_ENDIAN
arch/arm64/kernel/psci.c +6 -1
index fabd732d0a2dfee37074ef4ebb6ce5894871c8bd..4ad90ae6f8bacf0cbd3203d66580107d467ea232 100644
@@ -49,6 +49,12 @@ static int cpu_psci_cpu_boot(unsigned int cpu)
 #ifdef CONFIG_HOTPLUG_CPU
 static bool cpu_psci_cpu_can_disable(unsigned int cpu)
 {
+#ifdef CONFIG_PM_SLEEP_SMP_CPU_ZERO_STRICT
+	if (cpu == get_boot_cpu_id()) {
+		pr_info("Disabling boot CPU is not supported\n");
+		return false;
+	}
+#endif
 	return !psci_tos_resident_on(cpu);
 }
 
 
---

base-commit: ba3e43a9e601636f5edb54e259a74f96ca3b8fd8
change-id: 20260603-disable_boot_cpu_offline-eb4f55ac96f2

Best regards,

Signature · 1 line
Sneh Mankad <[email protected]>
#2 lore raw
Quoted 8 lines · Mark Rutland

arch/arm64/Kconfig config ARM64 · line 273

Quoted 5 lines · Sneh Mankad
arch/arm64/Kconfig +5 -0
+       depends on HOTPLUG_CPU
+       depends on SUSPEND
+       help
+         Disallow boot CPU (CPU0) offline when the suspend_ops->enter()
+         has to be executed by boot CPU.

As above, I don't think this makse sense as a config option.

Correct. That's pointless as it can be determined at boot time either through firmware (ACPI/DT) or if that's not possible for whatever reasons then a runtime detection quirk which depends on that particular platform.

#3 lore raw
In reply to Mark Rutland
Quoted 4 lines · Mark Rutland
Quoted 1 line · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco

Are those released products?

Yes, the SoCs are present in upstream:

arch/arm64/boot/dts/qcom/lemans.dtsi
arch/arm64/boot/dts/qcom/monaco.dtsi

Quoted 2 lines

Are those mobile phone parts, or somthing else?

These are ride (automotive) chipsets and IoT boards.

Quoted 11 lines
Quoted 10 lines

support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

If CPU0 is already offline before starting suspend to ram the freeze_secondary_cpus() picks alternate CPU as primary / last CPU and proceed further to invoke PSCI SYSTEM_SUSPEND.
This leads to a system crash.

Quoted 4 lines

Ok, so that's a firmware bug.

Why does the FW permit CPU0 to be offlined in the first place if it can't handle this?

FW permits CPU0 disablement because it functions properly in regular run time. But there is an additional hardware constraint in above mentioned platforms, that suspend to ram has to be performed via boot CPU / CPU0. Due to this constraint, CPU0 is required to be online when suspend to ram is triggered.

Quoted 1 line

What does PSCI_MIGRATE_INFO_TYPE report?

It is a optional function call, which is not implemented in firmware.

Quoted 3 lines

Ideally it'd report Uniprocessor (UP) not migrate capable (1), which would prevent CPU_OFF on that CPU, and would force suspend to happen there...

Also, above platforms are multicore (8 CPUs each).

Quoted 7 lines
Quoted 2 lines

In order to prevent such an issue introduce PM_SLEEP_SMP_CPU_ZERO_STRICT config and when enabled prohibit the CPU0 from getting disabled.

I don't think it makes sense for this to be a config option.

This is a platform-specific property, and it's possible to build a kernel that boots on this platform and/or other platforms.

Yes, but for that reason the config will only be enabled for SoCs that have this constraint, it will remain disabled for other SoCs.

Quoted 18 lines
Quoted 18 lines

---
Changes in v2:
- Moved the check to arm64 specific code.
- Link to v1: https://lore.kernel.org/r/20260605-disable_boot_cpu_offline-v1-1-4c68fe1a6cf8@oss.qualcomm.com

---
 arch/arm64/Kconfig       | 9 +++++++++
 arch/arm64/kernel/psci.c | 6 ++++++
 2 files changed, 15 insertions(+)
arch/arm64/Kconfig +0 -0
index fe60738e5943ba279e5571862423df4fed3db661..21697a535a25d286a2f8afe4921a41b13cc32c0a 100644
@@ -263,6 +263,15 @@ config ARM64
 	help
 	  ARM 64-bit (AArch64) Linux support.
 
Quoted 5 lines

arch/arm64/Kconfig config ARM64 · line 268

Quoted 3 lines
arch/arm64/Kconfig +3 -0
+config PM_SLEEP_SMP_CPU_ZERO_STRICT
+       bool "Disallow boot CPU (CPU0) offline"
+       depends on ARCH_QCOM

Why can't others select this?

I can remove ARCH_QCOM dependency if any other want to use this.

Quoted 7 lines

arch/arm64/Kconfig config ARM64 · line 273

Quoted 6 lines
arch/arm64/Kconfig +5 -0
+       depends on HOTPLUG_CPU
+       depends on SUSPEND
+       help
+         Disallow boot CPU (CPU0) offline when the suspend_ops->enter()
+         has to be executed by boot CPU.
Quoted 6 lines

As above, I don't think this makse sense as a config option.

Either we handle the FW bug, or we do not.

Mark.

As mentioned above, its not a FW bug. CPU0 is allowed to be disabled from FW and HW constraint is for suspend to ram to be performed on boot CPU.

Thanks,
Sneh

reviewer
#4 lore raw
Quoted 7 lines · Sneh Mankad
Quoted 1 line · Mark Rutland

On Fri, Jul 03, 2026 at 04:50:02PM +0530, Sneh Mankad wrote:

FW permits CPU0 disablement because it functions properly in regular run time. But there is an additional hardware constraint in above mentioned platforms, that suspend to ram has to be performed via boot CPU / CPU0. Due to this constraint, CPU0 is required to be online when suspend to ram is triggered.

The PSCI spec doesn't permit that.

Even if HW has that constraint, it's up to FW to hide that from the OS.

[...]

Quoted 5 lines
Quoted 2 lines

This is a platform-specific property, and it's possible to build a kernel that boots on this platform and/or other platforms.

Yes, but for that reason the config will only be enabled for SoCs that have this constraint, it will remain disabled for other SoCs.

Please re-read what I said. A config option can't help if you have a single kernel that supports multiple platforms.

Quoted 3 lines

As mentioned above, its not a FW bug. CPU0 is allowed to be disabled from FW and HW constraint is for suspend to ram to be performed on boot CPU.

Please see my other reply at:

https://lore.kernel.org/linux-arm-kernel/[email protected]/T/#m872da3cc4ded3386dce92d4bd4bdf9c33d3e2d0b

This is *definitely* a firmware bug.

To be clear, I'm not saying we can't do something to address that, but let's not pretend that this is compliant with the PSCI spec.

Mark.

#5 lore raw
Quoted 15 lines · Mark Rutland
Quoted 7 lines · Sneh Mankad
Quoted 1 line · Mark Rutland

On Fri, Jul 03, 2026 at 04:50:02PM +0530, Sneh Mankad wrote:

FW permits CPU0 disablement because it functions properly in regular run time. But there is an additional hardware constraint in above mentioned platforms, that suspend to ram has to be performed via boot CPU / CPU0. Due to this constraint, CPU0 is required to be online when suspend to ram is triggered.

The PSCI spec doesn't permit that.

Even if HW has that constraint, it's up to FW to hide that from the OS.

[...]

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

This is a platform-specific property, and it's possible to build a kernel that boots on this platform and/or other platforms.

Yes, but for that reason the config will only be enabled for SoCs that have this constraint, it will remain disabled for other SoCs.

Please re-read what I said. A config option can't help if you have a single kernel that supports multiple platforms.

+1

Quoted 4 lines
Quoted 3 lines

As mentioned above, its not a FW bug. CPU0 is allowed to be disabled from FW and HW constraint is for suspend to ram to be performed on boot CPU.

Quoted 9 lines

Please see my other reply at:

https://lore.kernel.org/linux-arm-kernel/[email protected]/T/#m872da3cc4ded3386dce92d4bd4bdf9c33d3e2d0b

This is *definitely* a firmware bug.

To be clear, I'm not saying we can't do something to address that, but let's not pretend that this is compliant with the PSCI spec.

+1

Just curious if CPU_OFF on this CPU0 returns DENIED or just proceeds and ends up with cpu and/or system hang ?

Signature · 2 lines
Regards,
Sudeep
#6 lore raw
Quoted 39 lines · Sudeep Holla
Quoted 15 lines · Mark Rutland
Quoted 7 lines · Sneh Mankad
Quoted 1 line · Mark Rutland

On Fri, Jul 03, 2026 at 04:50:02PM +0530, Sneh Mankad wrote:

FW permits CPU0 disablement because it functions properly in regular run time. But there is an additional hardware constraint in above mentioned platforms, that suspend to ram has to be performed via boot CPU / CPU0. Due to this constraint, CPU0 is required to be online when suspend to ram is triggered.

The PSCI spec doesn't permit that.

Even if HW has that constraint, it's up to FW to hide that from the OS.

[...]

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

This is a platform-specific property, and it's possible to build a kernel that boots on this platform and/or other platforms.

Yes, but for that reason the config will only be enabled for SoCs that have this constraint, it will remain disabled for other SoCs.

Please re-read what I said. A config option can't help if you have a single kernel that supports multiple platforms.

+1

Quoted 10 lines
Quoted 3 lines

As mentioned above, its not a FW bug. CPU0 is allowed to be disabled from FW and HW constraint is for suspend to ram to be performed on boot CPU.

Please see my other reply at:

https://lore.kernel.org/linux-arm-kernel/[email protected]/T/#m872da3cc4ded3386dce92d4bd4bdf9c33d3e2d0b

This is *definitely* a firmware bug.

Quoted 8 lines
Quoted 3 lines

To be clear, I'm not saying we can't do something to address that, but let's not pretend that this is compliant with the PSCI spec.

+1

Just curious if CPU_OFF on this CPU0 returns DENIED or just proceeds and ends up with cpu and/or system hang ?

My apologies for side-stepping the discussion, I just wanted to share that I have explored the exact same problem with other vendor's PSCI implementations.

In fact, I think it would be interesting to see how many PSCI implementations that really got this correct. :-)

That said, I think a DT property should be a generic PSCI DT property, but then set for those platforms that need it.

Kind regards
Uffe

#7 lore raw
In reply to Sneh Mankad

(It is always good to cc all PSCI maintainer for any ARM64 CPU hotpug/suspend related changes)

Quoted 5 lines · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

And you fail to explain why they have that constraint.

Is it because some secure context is not allowed to migrate ?

We already have a mechanism for that in place and this hack is not at all required.

Signature · 2 lines
Regards,
Sudeep
#8 lore raw

Hi Sudeep,

Le 03/07/2026 à 17:51, Sudeep Holla a écrit :

Quoted 9 lines

(It is always good to cc all PSCI maintainer for any ARM64 CPU hotpug/suspend related changes)

Quoted 5 lines · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

Quoted 6 lines

And you fail to explain why they have that constraint.

Is it because some secure context is not allowed to migrate ?

We already have a mechanism for that in place and this hack is not at all required.

Do you mean a mechanism for the secure context or for preventing CPU0 ?

Can you clarify ?

#9 lore raw
Quoted 15 lines · Daniel Lezcano

Hi Sudeep,

Le 03/07/2026 à 17:51, Sudeep Holla a écrit :

Quoted 11 lines

(It is always good to cc all PSCI maintainer for any ARM64 CPU hotpug/suspend related changes)

Quoted 5 lines · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

And you fail to explain why they have that constraint.

I still need the above to understand the issue/constraint better.

Quoted 6 lines
Quoted 4 lines

Is it because some secure context is not allowed to migrate ?

We already have a mechanism for that in place and this hack is not at all required.

Do you mean a mechanism for the secure context or for preventing CPU0 ?

I meant constraint based on secure context.

Signature · 2 lines
Regards,
Sudeep
#10 lore raw

[ Cc'ed Maulik ]

Quoted 14 lines · Sudeep Holla
Quoted 13 lines · Daniel Lezcano

Hi Sudeep,

Le 03/07/2026 à 17:51, Sudeep Holla a écrit :

Quoted 9 lines

(It is always good to cc all PSCI maintainer for any ARM64 CPU hotpug/suspend related changes)

Quoted 5 lines · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

And you fail to explain why they have that constraint.

I still need the above to understand the issue/constraint better.

Ok, I think Maulik can give some details here.

Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

Is it because some secure context is not allowed to migrate ?

We already have a mechanism for that in place and this hack is not at all required.

Do you mean a mechanism for the secure context or for preventing CPU0 ?

I meant constraint based on secure context.

#11 lore raw
In reply to Sudeep Holla
Quoted 14 lines · Sudeep Holla
Quoted 13 lines · Daniel Lezcano

Hi Sudeep,

Le 03/07/2026 à 17:51, Sudeep Holla a écrit :

Quoted 9 lines

(It is always good to cc all PSCI maintainer for any ARM64 CPU hotpug/suspend related changes)

Quoted 5 lines · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

And you fail to explain why they have that constraint.

I still need the above to understand the issue/constraint better.

Above mentioned SoCs have boot CPU fixed to CPU0 in HW, whenever SoC boots up/cold boots it starts with CPU0.

These SoCs support suspend to ram which leads to ACPI S3 similar state (where SoC is turned off and DDR is retained) PSCI SYSTEM_SUSPEND typically will be executed on boot core itself unless it is already offlined and non boot CPUs gets offlined using PSCI CPU_OFF.

As HW constraint always makes the SoC to boot with boot CPU, consider a scenario, where

Boot CPU is already disabled / offline => suspend to ram is triggered => SoC enters ACPI S3 similar state (only DDR is retained and rest of the SoC is off) <So far good>

External wake up arrives (say power key press) => SoC starts booting with CPU0 => CPU0 becomes first one to "land" in kernel now.

Kernel may later bring up other non-boot CPUs via PSCI CPU_ON calls.t
However Kernel had already marked CPU0 as disabled/ offline but same ended up in kernel without PSCI CPU_ON call.

To prevent this inconsistent state, before starting suspend to ram, need to make sure CPU0 is always online from kernel/ disable offlining of the boot CPU.

Although the HW constraint needs boot CPU to be online only when suspend to ram is triggered, current patch disallows disabling it for simplicity.

Quoted 10 lines
Quoted 7 lines
Quoted 5 lines

Is it because some secure context is not allowed to migrate ?

We already have a mechanism for that in place and this hack is not at all required.

Do you mean a mechanism for the secure context or for preventing CPU0 ?

I meant constraint based on secure context.

This is not because secure context not allowed to migrate but above mentioned HW constraints.

Thanks,
Sneh

reviewer
#12 lore raw
Quoted 27 lines · Sneh Mankad
Quoted 9 lines · Sudeep Holla
Quoted 8 lines · Daniel Lezcano

Hi Sudeep,

Le 03/07/2026 à 17:51, Sudeep Holla a écrit :

Quoted 4 lines

(It is always good to cc all PSCI maintainer for any ARM64 CPU hotpug/suspend related changes)

On Fri, Jul 03, 2026 at 04:50:02PM +0530, Sneh Mankad wrote:

Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

And you fail to explain why they have that constraint.

I still need the above to understand the issue/constraint better.

Above mentioned SoCs have boot CPU fixed to CPU0 in HW, whenever SoC boots up/cold boots it starts with CPU0.

These SoCs support suspend to ram which leads to ACPI S3 similar state (where SoC is turned off and DDR is retained) PSCI SYSTEM_SUSPEND typically will be executed on boot core itself unless it is already offlined and non boot CPUs gets offlined using PSCI CPU_OFF.

Quoted 6 lines

As HW constraint always makes the SoC to boot with boot CPU, consider a scenario, where

Boot CPU is already disabled / offline => suspend to ram is triggered => SoC enters ACPI S3 similar state (only DDR is retained and rest of the SoC is off) <So far good>

External wake up arrives (say power key press) => SoC starts booting with CPU0 => CPU0 becomes first one to "land" in kernel now.

IIUC you're saying that Linux calls SYSTEM_SUSPEND on a CPU other than CPU0, and the FW returns to Linux on CPU0. The HW constraints are irrelevant; FW should be handling that transparently from the PoV of the OS, and even if it always physically boots on CPU0, it should go wake the other CPU, then offline CPU0.

Look at the PSCI spec:

https://developer.arm.com/documentation/den0022/fb/

See section 5.20 ("SYSTEM_SUSPEND"), and in particular, the description in 5.20.1 ("Intended use"):

To use this API, a calling OS must power down all but one core through calls to CPU_OFF. From this point on, the remaining core can call SYSTEM_SUSPEND, passing the necessary entry_point_address and context_id parameters to enable resumption on wakeup. A calling OS can use the AFFINITY_INFO function to ensure that all cores are OFF prior to calling SYSTEM_SUSPEND.

Note that there's no requirement that the OS calls this on a specific CPU; just that all others are offline.

Further, note the sentence at the end of that section:

The core which calls SYSTEM_SUSPEND is the one that resumes execution at the specified entry_point_address on wakeup.

... which means that waking up on another CPU is a violation of the spec.

If we have to deal with broken FW, we can devise something, but this is *certainly* a firmware bug.

Mark.

#13 lore raw
In reply to Sneh Mankad
Quoted 30 lines · Sneh Mankad
Quoted 14 lines · Sudeep Holla
Quoted 13 lines · Daniel Lezcano

Hi Sudeep,

Le 03/07/2026 à 17:51, Sudeep Holla a écrit :

Quoted 9 lines

(It is always good to cc all PSCI maintainer for any ARM64 CPU hotpug/suspend related changes)

Quoted 5 lines · Sneh Mankad

The Qualcomm SoCs like LeMans, Monaco support suspend to ram which leads the SoC to ACPI S3 similar state where SoC is turned off and DDR is retained. The hardware design on these SoCs forces a constraint to suspend and resume the system on boot CPU / CPU0.

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

And you fail to explain why they have that constraint.

I still need the above to understand the issue/constraint better.

Above mentioned SoCs have boot CPU fixed to CPU0 in HW, whenever SoC boots up/cold boots it starts with CPU0.

These SoCs support suspend to ram which leads to ACPI S3 similar state (where SoC is turned off and DDR is retained).
PSCI SYSTEM_SUSPEND typically will be executed on boot core itself unless it is already offlined and non boot CPUs gets offlined using PSCI CPU_OFF.

Quoted 7 lines

As HW constraint always makes the SoC to boot with boot CPU, consider a scenario, where

Boot CPU is already disabled / offline => suspend to ram is triggered => SoC enters ACPI S3 similar state (only DDR is retained and rest of the SoC is off).
<So far good>

Not really, see below ...

Quoted 4 lines

External wake up arrives (say power key press) => SoC starts booting with CPU0 => CPU0 becomes first one to "land" in kernel now.

Ideally the firmware could have handled it by booting/waking the suspended CPU and turning itself off even if it is some h/w limitation to ensure the firmware is PSCI spec compliant.

Quoted 4 lines

Kernel may later bring up other non-boot CPUs via PSCI CPU_ON calls. However Kernel had already marked CPU0 as disabled/ offline but same ended up in kernel without PSCI CPU_ON call.

... You resumed back on a wrong CPU.

Quoted 3 lines

To prevent this inconsistent state, before starting suspend to ram, need to make sure CPU0 is always online from kernel/ disable offlining of the boot CPU.

At least not in the way this patch does. Kconfig is not an option. Why is the firmware not handling it properly ? Just resuming random CPU into the kernel is firmware bug which either needs to be handled as f/w errata or good if the firmware can be fixed.

Quoted 3 lines

Although the HW constraint needs boot CPU to be online only when suspend to ram is triggered, current patch disallows disabling it for simplicity.

That's very strange, so it sounds like not a real h/w constrain to make CPU0 as non-hotpluggable. PSCI CPU_SUSPEND resume path must handle wake up on wrong CPU correctly and land/resume on the correct CPU in the kernel.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Is it because some secure context is not allowed to migrate ?

We already have a mechanism for that in place and this hack is not at all required.

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

Do you mean a mechanism for the secure context or for preventing CPU0 ?

I meant constraint based on secure context.

This is not because secure context not allowed to migrate but above mentioned HW constraints.

Sure not secure context related but a HW constraint that secure/PSCI firmware ignored to handle correctly.

Fix the firmware or explore ways to handle it as FW errata.

Signature · 2 lines
Regards,
Sudeep