cpufreq: Fix race between suspend/resume and CPU hotplug
PATCH 1/1 cpufreq: Fix race between suspend/resume and CPU hotplug
CPU hotplug operations can race with cpufreq_suspend() and cpufreq_resume(), leading to null pointer dereferences when accessing governor data. This occurs because there is no synchronization between suspend/resume operations and CPU hotplug, allowing concurrent access to policy->governor_data while it is being freed or initialized.
Detailed race condition scenario:
1. Thread A (cpufreq_suspend) starts execution:
- Iterates through active policies
- Calls cpufreq_stop_governor(policy) for each policy
- Sets cpufreq_suspended = true
2. Thread B (CPU hotplug) executes concurrently:
- Calls cpu_down(cpu)
- Calls cpuhp_cpufreq_offline(cpu)
- Calls cpufreq_offline(cpu)
- Inside cpufreq_offline():
* Stops governor: policy->governor->stop(policy)
* Exits governor: policy->governor->exit(policy)
* Frees governor_data: kfree(policy->governor_data)
* Sets policy->governor_data = NULL
3. Race window between step 1 and step 2:
- Thread A is iterating policies and stopping governors
- Thread B is concurrently executing CPU offline
- Both threads may access the same policy->governor_data
- Thread B frees governor_data while Thread A is still using it
- Thread A accesses freed governor_data → null pointer dereference
Similarly, cpufreq_resume() can race with CPU hotplug where governor_data is being initialized while hotplug is trying to access it, leading to accessing uninitialized data.
--- drivers/cpufreq/cpufreq.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/cpufreq/cpufreq.c +7 -1
@@ -1979,6 +1979,7 @@ void cpufreq_suspend(void)if (!cpufreq_driver)return ; +cpus_read_lock ();if (!has_target () && !cpufreq_driver->suspend) goto suspend;
@@ -1998,6 +1999,7 @@ void cpufreq_suspend(void) suspend: cpufreq_suspended = true; +cpus_read_unlock (); }/**
@@ -2017,10 +2019,11 @@ void cpufreq_resume(void)if (unlikely (!cpufreq_suspended))return ; +cpus_read_lock (); cpufreq_suspended = false;if (!has_target () && !cpufreq_driver->resume) -return ; + goto out;pr_debug ("%s: Resuming Governors\n" , __func__);
@@ -2038,6 +2041,9 @@ void cpufreq_resume(void) __func__, policy->cpu); } } + +out: +cpus_read_unlock (); }/**
--
2.34.1
#/******本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#
Rafael J. Wysocki
maintainerQuoted 4 lines · Tianxiang Chen
CPU hotplug operations can race with
cpufreq_suspend()andcpufreq_resume(), leading to null pointer dereferences when accessing governor data.
So how exactly would CPU hotplug be started during a system suspend or resume?
Quoted 86 lines
This occurs because there is
no synchronization between suspend/resume operations and CPU hotplug, allowing concurrent access topolicy->governor_datawhile it is being freed or initialized.Detailed race condition scenario:
1. Thread A (
cpufreq_suspend) starts execution:
- Iterates through active policies
- Callscpufreq_stop_governor(policy)for each policy
- Setscpufreq_suspended= true2. Thread B (CPU hotplug) executes concurrently:
- Callscpu_down(cpu)
- Callscpuhp_cpufreq_offline(cpu)
- Callscpufreq_offline(cpu)
- Insidecpufreq_offline():
* Stops governor:policy->governor->stop(policy)
* Exits governor:policy->governor->exit(policy)
* Freesgovernor_data:kfree(policy->governor_data)
* Setspolicy->governor_data= NULL3. Race window between step 1 and step 2:
- Thread A is iterating policies and stopping governors
- Thread B is concurrently executing CPU offline
- Both threads may access the samepolicy->governor_data
- Thread B freesgovernor_datawhile Thread A is still using it
- Thread A accesses freedgovernor_data→ null pointer dereferenceSimilarly,
cpufreq_resume()can race with CPU hotplug wheregovernor_datais being initialized while hotplug is trying to access it, leading to accessing uninitialized data.Signed-off-by Tianxiang Chen <[email protected]>--- drivers/cpufreq/cpufreq.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/cpufreq/cpufreq.c +7
-2@@ -1979,6 +1979,7 @@ void cpufreq_suspend(void)if (!cpufreq_driver)return ; +cpus_read_lock ();if (!has_target () && !cpufreq_driver->suspend) goto suspend;@@ -1998,6 +1999,7 @@ void cpufreq_suspend(void) suspend: cpufreq_suspended = true; +cpus_read_unlock (); }/** @@ -2017,10 +2019,11 @@ void cpufreq_resume(void)if (unlikely (!cpufreq_suspended))return ; +cpus_read_lock (); cpufreq_suspended = false;if (!has_target () && !cpufreq_driver->resume) -return ; + goto out;pr_debug ("%s: Resuming Governors\n" , __func__);@@ -2038,6 +2041,9 @@ void cpufreq_resume(void) __func__, policy->cpu); } } + +out: +cpus_read_unlock (); }/** -- 2.34.1
#/******本邮件及其附件含有小米公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#
Re: [PATCH] cpufreq: fix race between hotplug and suspend
Quoted 1 line · Rafael J. Wysocki
So how exactly would CPU hotplug be started during a system suspend or resume?
Hi Rafael,
Thank you for your question. Let me explain the two scenarios:
1. cpufreq_suspend() During Reboot (Confirmed Issue)
The real and reproducible race I encountered occurs during system reboot.
Call chain:
kernel_restart() -> kernel_restart_prepare()
-> device_shutdown() -> cpufreq_suspend()
Different from the regular suspend path, the reboot path does NOT call freeze_processes() at all.
All userspace processes, drivers and kernel threads are still running when cpufreq_suspend() executes. This allows CPU hotplug (offline/online) operations to run concurrently with cpufreq_suspend().
2. System suspend/resume (Less Likely but Possible)
CPU hotplug is less likely during system suspend/resume. However, non-freezable kernel threads may keep running throughout the entire process, which may still trigger CPU hotplug in theory.
So I added cpus_read_lock()/cpus_read_unlock() to block CPU hotplug while resume is in progress.
Signature · 2 lines
Thx and BRs, Tianxiang Chen
Rafael J. Wysocki
maintainerRe: [PATCH] cpufreq: fix race between hotplug and suspend
Quoted 9 lines · Tianxiang Chen
Quoted 1 line · Rafael J. Wysocki
So how exactly would CPU hotplug be started during a system suspend or resume?
Hi Rafael,
Thank you for your question. Let me explain the two scenarios:
1.
cpufreq_suspend()During Reboot (Confirmed Issue)
Which needs to be mentioned in the patch changelog.
Quoted 8 lines
The real and reproducible race I encountered occurs during system reboot.
Call chain:
kernel_restart()->kernel_restart_prepare()-> device_shutdown() -> cpufreq_suspend()Different from the regular suspend path, the reboot path does NOT call
freeze_processes()at all.
That's correct.
Quoted 9 lines
All userspace processes, drivers and kernel threads are still running when
cpufreq_suspend()executes. This allows CPU hotplug (offline/online) operations to run concurrently withcpufreq_suspend().2. System suspend/resume (Less Likely but Possible)
CPU hotplug is less likely during system suspend/resume. However, non-freezable kernel threads may keep running throughout the entire process, which may still trigger CPU hotplug in theory.
Which would be a bug in the kernel thread in question. So not really.
Quoted 2 lines
So I added cpus_read_lock()/cpus_read_unlock() to block CPU hotplug while resume is in progress.
Please resend the patch with a changelog actually mentioning the failure that you have observed.
Thanks!
PATCH v2 1/1 cpufreq: Fix hotplug-suspend race during reboot
During system reboot, cpufreq_suspend() is called via the kernel_restart() -> device_shutdown() -> pm_notifier_call_chain() path. Unlike the normal system suspend path, the reboot path does not call freeze_processes(), so userspace processes and kernel threads remain active.
This allows CPU hotplug operations to run concurrently with cpufreq_suspend(). The original code has no synchronization with CPU hotplug, leading to a race condition where governor_data can be freed by the hotplug path while cpufreq_suspend() is still accessing it, resulting in a null pointer dereference:
Unable to handle kernel NULL pointer dereference Call Trace: do_kernel_fault+0x28/0x3c cpufreq_suspend+0xdc/0x160 device_shutdown+0x18/0x200 kernel_restart+0x40/0x80 arm64_sys_reboot+0x1b0/0x200
Fix this by adding cpus_read_lock()/cpus_read_unlock() to cpufreq_suspend() to block CPU hotplug operations while suspend is in progress.
---
v2:
- Update changelog to explicitly mention reboot scenario
- Add observed crash trace
--- drivers/cpufreq/cpufreq.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/cpufreq/cpufreq.c +2 -0
@@ -1979,6 +1979,7 @@ void cpufreq_suspend(void)if (!cpufreq_driver)return ; +cpus_read_lock ();if (!has_target () && !cpufreq_driver->suspend) goto suspend;
@@ -1998,6 +1999,7 @@ void cpufreq_suspend(void) suspend: cpufreq_suspended = true; +cpus_read_unlock (); }/**
Signature · 1 line
2.34.1
Re: [PATCH v2] cpufreq: Fix hotplug-suspend race during reboot
Quoted 46 lines · Tianxiang Chen
During system reboot,
cpufreq_suspend()is called via thekernel_restart()->device_shutdown()->pm_notifier_call_chain()path. Unlike the normal system suspend path, the reboot path does not callfreeze_processes(), so userspace processes and kernel threads remain active.This allows CPU hotplug operations to run concurrently with
cpufreq_suspend(). The original code has no synchronization with CPU hotplug, leading to a race condition wheregovernor_datacan be freed by the hotplug path whilecpufreq_suspend()is still accessing it, resulting in a null pointer dereference:Unable to handle kernel NULL pointer dereference Call Trace: do_kernel_fault+0x28/0x3c cpufreq_suspend+0xdc/0x160 device_shutdown+0x18/0x200 kernel_restart+0x40/0x80 arm64_sys_reboot+0x1b0/0x200Fix this by adding cpus_read_lock()/cpus_read_unlock() to
cpufreq_suspend()to block CPU hotplug operations while suspend is in progress.Signed-off-by Tianxiang Chen <[email protected]>---
v2:
- Update changelog to explicitly mention reboot scenario
- Add observed crash trace--- drivers/cpufreq/cpufreq.c | 2 ++ 1 file changed, 2 insertions(+)drivers/cpufreq/cpufreq.c +1
-0@@ -1979,6 +1979,7 @@ void cpufreq_suspend(void)if (!cpufreq_driver)return ; +cpus_read_lock ();if (!has_target () && !cpufreq_driver->suspend) goto suspend;
drivers/cpufreq/cpufreq.c void cpufreq_suspend(void) · line 2005
Quoted 7 lines
drivers/cpufreq/cpufreq.c +1
-0@@ -1998,6 +1999,7 @@ void cpufreq_suspend(void) suspend: cpufreq_suspended = true; +cpus_read_unlock (); }/**
Hi Tianxiang,
May I know did you test this with lockdep enabled? Specifically, does the new cpus_read_lock() → policy->rwsem ordering in cpufreq_suspend() trigger any lockdep warnings? Thanks
Signature · 2 lines
Thx and BRs, Zhongqiu Han
Re: [PATCH v2] cpufreq: Fix hotplug-suspend race during reboot
Quoted 3 lines · Zhongqiu Han
May I know did you test this with lockdep enabled? Specifically, does the new
cpus_read_lock()->policy->rwsemordering incpufreq_suspend()trigger any lockdep warnings? Thanks
Hi Zhongqiu,
Thanks for the review.
I did test v2 with lockdep enabled and was NOT able to reproduce any warning.
No circular-locking splat or "possible deadlock" report was observed in dmesg across multiple runs.
My reasoning for why the new order should be safe:
* The patch establishes cpus_read_lock() -> policy->rwsem.
* The hotplug path already holds cpu_hotplug_lock (write side, via the hotplug core) before taking policy->rwsem inside cpufreq_offline()/cpufreq_online(), i.e. the same direction.
* I grep'd cpufreq and did not find an existing path that takes policy->rwsem first and then acquires cpus_read_lock() underneath. If I missed one, please point me at it.
* cpus_read_lock() is a percpu-rwsem read side and is re-entrant, so even if an outer caller already holds it (e.g. via a pm notifier running inside a hotplug callback) this is safe.
May I ask whether you have actually observed a lockdep splat on this change on any downstream tree, or is this a precautionary question? If you have a specific call chain in mind, I would like to add targeted coverage before v3 so we can nail it down definitively.
Thanks,
Tianxiang
Zhongqiu Han
Reviewed-byRe: [PATCH v2] cpufreq: Fix hotplug-suspend race during reboot
Quoted 7 lines · Tianxiang Chen
Quoted 3 lines · Zhongqiu Han
May I know did you test this with lockdep enabled? Specifically, does the new
cpus_read_lock()->policy->rwsemordering incpufreq_suspend()trigger any lockdep warnings? ThanksHi Zhongqiu,
Quoted 7 lines
Thanks for the review.
I did test v2 with lockdep enabled and was NOT able to reproduce any warning.
No circular-locking splat or "possible deadlock" report was observed in dmesg across multiple runs.
Thanks for the confirmation.
Quoted 14 lines
My reasoning for why the new order should be safe:
* The patch establishes
cpus_read_lock()->policy->rwsem.
* The hotplug path already holdscpu_hotplug_lock(write side, via the hotplug core) before takingpolicy->rwseminside cpufreq_offline()/cpufreq_online(), i.e. the same direction.
* I grep'd cpufreq and did not find an existing path that takespolicy->rwsemfirst and then acquirescpus_read_lock()underneath. If I missed one, please point me at it.
*cpus_read_lock()is a percpu-rwsem read side and is re-entrant, so even if an outer caller already holds it (e.g. via a pm notifier running inside a hotplug callback) this is safe.
Quoted 4 lines
May I ask whether you have actually observed a lockdep splat on this change on any downstream tree, or is this a precautionary question? If you have a specific call chain in mind, I would like to add targeted coverage before v3 so we can nail it down definitively.
This was mainly a precautionary question on my side, to make sure there aren't any hidden locking concerns. In my experience, running new locking changes under lockdep can be a useful sanity check, so I wanted to double‑check.
For context, none of the existing cpufreq driver .suspend callbacks currently invoke cpus_read_lock() so this does not affect any existing suspend paths.
Looks okay to me.
Quoted 3 lines
Thanks,
Tianxiang
Signature · 2 lines
Thx and BRs, Zhongqiu Han
Re: [PATCH v2] cpufreq: Fix hotplug-suspend race during reboot
Quoted 46 lines · Zhongqiu Han
Quoted 7 lines · Tianxiang Chen
Quoted 3 lines · Zhongqiu Han
May I know did you test this with lockdep enabled? Specifically, does the new
cpus_read_lock()->policy->rwsemordering incpufreq_suspend()trigger any lockdep warnings? ThanksHi Zhongqiu,
Quoted 7 lines
Thanks for the review.
I did test v2 with lockdep enabled and was NOT able to reproduce any warning.
No circular-locking splat or "possible deadlock" report was observed in dmesg across multiple runs.
Thanks for the confirmation.
Quoted 14 lines
My reasoning for why the new order should be safe:
* The patch establishes
cpus_read_lock()->policy->rwsem.
* The hotplug path already holdscpu_hotplug_lock(write side, via the hotplug core) before takingpolicy->rwseminside cpufreq_offline()/cpufreq_online(), i.e. the same direction.
* I grep'd cpufreq and did not find an existing path that takespolicy->rwsemfirst and then acquirescpus_read_lock()underneath. If I missed one, please point me at it.
*cpus_read_lock()is a percpu-rwsem read side and is re-entrant, so even if an outer caller already holds it (e.g. via a pm notifier running inside a hotplug callback) this is safe.Quoted 4 lines
May I ask whether you have actually observed a lockdep splat on this change on any downstream tree, or is this a precautionary question? If you have a specific call chain in mind, I would like to add targeted coverage before v3 so we can nail it down definitively.
This was mainly a precautionary question on my side, to make sure there aren't any hidden locking concerns. In my experience, running new locking changes under lockdep can be a useful sanity check, so I wanted to double‑check.
Quoted 8 lines
For context, none of the existing cpufreq driver .suspend callbacks currently invoke
cpus_read_lock()so this does not affect any existing suspend paths.Looks okay to me.
Reviewed-by Zhongqiu Han <[email protected]>
Applied as 7.1-rc material, thanks!