powerpc: fix preempt_count imbalances in perf and kexec paths
v3 cover powerpc: fix preempt_count imbalances in perf and kexec paths
Hi all,
This patch series fixes some minor preempt_count bookkeeping issues in arch/powerpc/ found during a preemption leak audit prompted by the lazy/full preemption model changes. These are get_cpu/put_cpu and get_cpu_var/put_cpu_var pairing errors that leave preempt_count incorrectly elevated or underflowed.
v2 -> v3:
- Patches 2 and 3: use raw_smp_processor_id() instead of smp_processor_id(). Both paths run with interrupts disabled, so the plain accessor is sufficient.
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/
Testing:
- Patches 2 and 3: kexec boot-tested on PowerNV (powernv_defconfig) and pseries (pseries_defconfig).
- Patch 1: compile-tested only (ppc64e_defconfig + CONFIG_FSL_EMB_PERFMON); no e500/fsl_emb hardware to runtime-test.
Please let me know your comments.
Thanks,
Aboorva
Aboorva Devarajan (3):
powerpc/perf: fix preempt count underflow in fsl_emb_pmu_del powerpc/powernv: fix preempt count leak in
pnv_kexec_wait_secondaries_down
powerpc/kexec: fix double get_cpu() imbalance in kexec_prepare_cpus
arch/powerpc/kexec/core_64.c | 4 +--- arch/powerpc/perf/core-fsl-emb.c | 3 ++- arch/powerpc/platforms/powernv/setup.c | 3 ++- 3 files changed, 5 insertions(+), 5 deletions(-)
base-commit: 03da3bc22934393e1dcc63ed3c50457b6d7e345e
Signature · 1 line
2.54.0
PATCH v3 1/3 powerpc/perf: fix preempt count underflow in fsl_emb_pmu_del
fsl_emb_pmu_del() unconditionally calls put_cpu_var(cpu_hw_events) at the 'out:' label, but only calls the matching get_cpu_var() after the 'i < 0' early-return check. When event->hw.idx is negative the function jumps to 'out:' without having taken get_cpu_var(), and the trailing put_cpu_var() then issues an unmatched preempt_enable(), underflowing preempt_count.
On a CONFIG_PREEMPT=y kernel preempt_count would underflow and eventually present as a 'scheduling while atomic' BUG.
Move put_cpu_var() to pair with get_cpu_var() so the percpu access is correctly bracketed and the 'out:' label only handles perf_pmu_enable.
--- arch/powerpc/perf/core-fsl-emb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/perf/core-fsl-emb.c +2 -1
@@ -366,9 +366,10 @@ static void fsl_emb_pmu_del(struct perf_event *event, int flags) cpuhw->n_events--; +put_cpu_var (cpu_hw_events); + out:perf_pmu_enable (event->pmu); -put_cpu_var (cpu_hw_events); }static void fsl_emb_pmu_start (struct perf_event *event,int ef_flags)
Signature · 1 line
2.54.0
PATCH v3 2/3 powerpc/powernv: fix preempt count leak in pnv_kexec_wait_secondaries_down
In reply to Aboorva Devarajanpnv_kexec_wait_secondaries_down() calls get_cpu() to obtain the current CPU id but never calls the matching put_cpu(), leaking one preempt_disable() nesting level on every invocation.
In practice the imbalance does not trigger a visible splat because the kexec teardown path is a one-way trip: IRQs are already disabled, no schedule() occurs after the leak, and default_machine_kexec() overwrites preempt_count with HARDIRQ_OFFSET before jumping into kexec_sequence() which never returns. However the bookkeeping is still wrong.
The function only needs the current CPU id, and this path runs with interrupts disabled and the CPU pinned, so the preempt_disable() side-effect of get_cpu() is unnecessary. Replace it with raw_smp_processor_id().
--- arch/powerpc/platforms/powernv/setup.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/platforms/powernv/setup.c +2 -1
@@ -396,7 +396,8 @@ static void pnv_kexec_wait_secondaries_down(void) {int my_cpu, i, notified =-1 ; - my_cpu =get_cpu (); +/* Called with interrupts disabled, so the CPU is pinned. */ + my_cpu =raw_smp_processor_id ();for_each_online_cpu (i) {uint8_t status;
Signature · 1 line
2.54.0
PATCH v3 3/3 powerpc/kexec: fix double get_cpu() imbalance in kexec_prepare_cpus
In reply to Aboorva Devarajankexec_prepare_cpus_wait() calls get_cpu() internally to obtain the current CPU id. kexec_prepare_cpus() calls kexec_prepare_cpus_wait() twice -- once for KEXEC_STATE_IRQS_OFF and once for KEXEC_STATE_REAL_MODE -- but only issues a single put_cpu() at the end, leaving preempt_count elevated by one extra nesting level.
In practice the imbalance does not trigger a 'scheduling while atomic' splat because the kexec path is a one-way trip: IRQs are already disabled, no schedule() occurs after the leak, and default_machine_kexec() overwrites preempt_count with HARDIRQ_OFFSET before jumping into kexec_sequence() which never returns. However the bookkeeping is still wrong.
kexec_prepare_cpus() calls local_irq_disable()/hard_irq_disable() before invoking kexec_prepare_cpus_wait(), so the CPU is already pinned and the get_cpu()/put_cpu() preempt_disable() bracketing is unnecessary. Only the current CPU id is needed, so replace get_cpu() with raw_smp_processor_id() and drop the now-unneeded put_cpu().
--- arch/powerpc/kexec/core_64.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
arch/powerpc/kexec/core_64.c +1 -3
@@ -169,7 +169,7 @@ static void kexec_prepare_cpus_wait(int wait_state)int my_cpu, i, notified=-1 ;hw_breakpoint_disable (); - my_cpu =get_cpu (); + my_cpu =raw_smp_processor_id ();/* Make sure each CPU has at least made it to the state we need. * * FIXME: There is a (slim) chance of a problem if not all of the CPUs
@@ -267,8 +267,6 @@ static void kexec_prepare_cpus(void)/* after we tell the others to go down */ if (ppc_md.kexec_cpu_down) ppc_md.kexec_cpu_down(0, 0); - - put_cpu(); }#else /* ! SMP */
Signature · 1 line
2.54.0
Re: [PATCH v3 3/3] powerpc/kexec: fix double get_cpu() imbalance in kexec_prepare_cpus
Quoted 39 lines · Aboorva Devarajan
kexec_prepare_cpus_wait()callsget_cpu()internally to obtain the current CPU id.kexec_prepare_cpus()callskexec_prepare_cpus_wait()twice -- once forKEXEC_STATE_IRQS_OFFand once forKEXEC_STATE_REAL_MODE-- but only issues a singleput_cpu()at the end, leavingpreempt_countelevated by one extra nesting level.In practice the imbalance does not trigger a 'scheduling while atomic' splat because the kexec path is a one-way trip: IRQs are already disabled, no
schedule()occurs after the leak, anddefault_machine_kexec()overwritespreempt_countwithHARDIRQ_OFFSETbefore jumping intokexec_sequence()which never returns. However the bookkeeping is still wrong.
kexec_prepare_cpus()calls local_irq_disable()/hard_irq_disable() before invokingkexec_prepare_cpus_wait(), so the CPU is already pinned and the get_cpu()/put_cpu()preempt_disable()bracketing is unnecessary. Only the current CPU id is needed, so replaceget_cpu()withraw_smp_processor_id()and drop the now-unneededput_cpu().Fixes 1fc711f7ffb0 ("powerpc/kexec: Fix race in kexec shutdown")Signed-off-by Aboorva Devarajan <[email protected]>--- arch/powerpc/kexec/core_64.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)arch/powerpc/kexec/core_64.c +1
-1@@ -169,7 +169,7 @@ static void kexec_prepare_cpus_wait(int wait_state)int my_cpu, i, notified=-1 ;hw_breakpoint_disable (); - my_cpu =get_cpu (); + my_cpu =raw_smp_processor_id ();/* Make sure each CPU has at least made it to the state we need. * * FIXME: There is a (slim) chance of a problem if not all of the CPUs
arch/powerpc/kexec/core_64.c static void kexec_prepare_cpus(void) · line 272
Quoted 8 lines
arch/powerpc/kexec/core_64.c +0
-2@@ -267,8 +267,6 @@ static void kexec_prepare_cpus(void)/* after we tell the others to go down */ if (ppc_md.kexec_cpu_down) ppc_md.kexec_cpu_down(0, 0); - - put_cpu(); }#else /* ! SMP */
Quoted 8 lines · Aboorva Devarajan
Hi all,
This patch series fixes some minor
preempt_countbookkeeping issues inarch/powerpc/found during a preemption leak audit prompted by the lazy/full preemption model changes. These are get_cpu/put_cpu and get_cpu_var/put_cpu_var pairing errors that leavepreempt_countincorrectly elevated or underflowed.
All 3 patches looks good. Aren't these candidates for stable tree? Then why not add Cc: tag too?
For all 3 -
Quoted 6 lines · Aboorva Devarajan
This patch series fixes some minor
preempt_countbookkeeping issues inarch/powerpc/found during a preemption leak audit prompted by the lazy/full preemption model changes. These are get_cpu/put_cpu and get_cpu_var/put_cpu_var pairing errors that leavepreempt_countincorrectly elevated or underflowed.
Quoted 6 lines
v2 -> v3:
- Patches 2 and 3: useraw_smp_processor_id()instead ofsmp_processor_id(). Both paths run with interrupts disabled, so the plain accessor is sufficient.[...]
Applied to powerpc/next.
[1/3] powerpc/perf: fix preempt count underflow in fsl_emb_pmu_del
https://git.kernel.org/powerpc/c/81e3a86030462824a67d697739cf3f387f4ba350 [2/3] powerpc/powernv: fix preempt count leak in pnv_kexec_wait_secondaries_down https://git.kernel.org/powerpc/c/0ecd26e93e698c8327521910fc6296f5b84a4b92 [3/3] powerpc/kexec: fix double get_cpu() imbalance in kexec_prepare_cpus https://git.kernel.org/powerpc/c/5c86f1c1f972761a04bf22f4c0618d1aa714185b
cheers