s390/pai: Handle multiple PMU stop callback invocations
PATCH 1/1 s390/pai: Handle multiple PMU stop callback invocations
Handle the following scenario:
The kernel protects itself against a very high sampling load and throttles the sampling using:
perf_event_throttle() --> PMU->stop()
Shortly later the scheduler may terminate the task and removes it from the CPU. It again calls
PMU->stop()
which results in two invocations of PMU->stop() called back to back. Protect against this and check the PERF_HES_STOPPED bit on function entry. If it is already set return.
Clear bit PERF_HES_STOPPED in PMU->start().
# Cc: [email protected] # v6.19+
--- arch/s390/kernel/perf_pai.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/s390/kernel/perf_pai.c +8 -0
@@ -484,6 +484,7 @@ static void pai_start(struct perf_event *event, int flags, cpump->event = event; } } + event->hw.state &= ~PERF_HES_STOPPED ; }static void paicrypt_start (struct perf_event *event,int flags)
@@ -530,6 +531,13 @@ static void pai_stop(struct perf_event *event, int flags)struct pai_mapptr *mp =this_cpu_ptr (pai_root[idx].mapptr);struct pai_map *cpump = mp->mapptr; +/* Cope with multiple invocations: +* 1. perf_event_throttle() --> PMU->stop() +* 2. task schedules out --> PMU->stop() +* Check for event already stopped. +*/ +if (event->hw.state &PERF_HES_STOPPED ) +return ;if (!event->attr.sample_period) {/* Counting */ pai_pmu[idx].pmu->read (event); }else {/* Sampling */
Signature · 1 line
2.55.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Unsafe iteration over cpump->syswide_list in pai_have_samples() leads to a kernel crash (LIST_POISON dereference) if an event overflows and is synchronously throttled during the loop. --
--- Patch [1]: [PATCH] s390/pai: Handle multiple PMU stop callback invocations --- commit 8c88016c23029bda1d214155e79ad2671b8e3eb5
Author: Thomas Richter <[email protected]>
s390/pai: Handle multiple PMU stop callback invocations
The kernel protects itself against high sampling load by throttling it via perf_event_throttle(), calling PMU->stop(). Shortly after, the task may be scheduled out, resulting in a second PMU->stop() invocation. This patch adds a check for the PERF_HES_STOPPED bit to prevent duplicate handling.
Quoted 4 lines
arch/s390/kernel/perf_pai.c +0
-0
[ ... ]
arch/s390/kernel/perf_pai.c static void pai_stop(struct perf_event *event, int flags) · line 520
Quoted 11 lines
arch/s390/kernel/perf_pai.c +7
-0@@ -510,6 +511,13 @@ static void pai_stop(struct perf_event *event, int flags)struct pai_mapptr *mp =this_cpu_ptr (pai_root[idx].mapptr);struct pai_map *cpump = mp->mapptr; +/* Cope with multiple invocations: +* 1. perf_event_throttle() --> PMU->stop() +* 2. task schedules out --> PMU->stop() +* Check for event already stopped. +*/ +if (event->hw.state &PERF_HES_STOPPED ) +return ;
[Severity: High]
This is a pre-existing issue, but can this synchronous throttle event corrupt the list iteration during sample processing?
Looking at the iteration in pai_have_samples():
arch/s390/kernel/perf_pai.c:pai_have_samples() {
...
list_for_each_entry(event, &cpump->syswide_list, hw.tp_list)
pai_have_sample(event, cpump);
...
}If an event reaches its sample limit, perf_event_overflow() triggers synchronously, invoking perf_event_throttle() which then calls pai_stop().
When pai_stop() is called for a system-wide sampling event, it executes:
list_del(PAI_SWLIST(event));
Will deleting the event currently being iterated leave the list pointer at a poison value, causing a panic when the loop in pai_have_samples() continues? Does pai_have_samples() need to use list_for_each_entry_safe() instead?
Quoted 3 lines
arch/s390/kernel/perf_pai.c +0
-0if (!event->attr.sample_period) {/* Counting */ pai_pmu[idx].pmu->read (event); }else {/* Sampling */
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 4 lines · On 8/13/26 09:21, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Unsafe iteration overcpump->syswide_listinpai_have_samples()leads to a kernel crash (LIST_POISONdereference) if an event overflows and is synchronously throttled during the loop.
This can not happen. The syswide_list is anchored by per-CPU pai_map::syswide_list and tracks all events which are bound to that particular CPU. The event was installed only on that particular CPU and does not move with the process/task. Each CPU has its own list anchored by pai_map::syswide_list.
The call chains are either
paicrypt_sched_task() or paiext_sched_task
+--> pai_have_samples() iterates over syswide_list and calls
+--> pai_have_sampleor
pai_stop()
+--> list_del() to remove the event from that CPU.
Since a CPU can only execute one given code path at any one time:
- either schedule out
- or stop the event
both can not happen at the same time. PAI PMU does not install an interrupt handler, there is no supoprt for interrupts.
I think this is safe.
Quoted 64 lines
--
--- Patch [1]: [PATCH] s390/pai: Handle multiple PMU stop callback invocations --- commit
8c88016c23029bda1d214155e79ad2671b8e3eb5
Author: Thomas Richter <[email protected]>s390/pai: Handle multiple PMU stop callback invocations
The kernel protects itself against high sampling load by throttling it via
perf_event_throttle(), callingPMU->stop(). Shortly after, the task may be scheduled out, resulting in a secondPMU->stop()invocation. This patch adds a check for thePERF_HES_STOPPEDbit to prevent duplicate handling.Cc [email protected] # v6.19+Fixes ac03223f0723 ("s390/pai_crypto: Add common pai_stop() function")Quoted 4 lines
arch/s390/kernel/perf_pai.c +0
-0[ ... ]
arch/s390/kernel/perf_pai.c static void pai_stop(struct perf_event *event, int flags) · line 520
Quoted 11 lines
arch/s390/kernel/perf_pai.c +7
-0@@ -510,6 +511,13 @@ static void pai_stop(struct perf_event *event, int flags)struct pai_mapptr *mp =this_cpu_ptr (pai_root[idx].mapptr);struct pai_map *cpump = mp->mapptr; +/* Cope with multiple invocations: +* 1. perf_event_throttle() --> PMU->stop() +* 2. task schedules out --> PMU->stop() +* Check for event already stopped. +*/ +if (event->hw.state &PERF_HES_STOPPED ) +return ;[Severity: High]
This is a pre-existing issue, but can this synchronous throttle event corrupt the list iteration during sample processing?Looking at the iteration in
pai_have_samples():arch/s390/kernel/perf_pai.c:pai_have_samples() { ... list_for_each_entry(event, &cpump->syswide_list, hw.tp_list) pai_have_sample(event, cpump); ... }If an event reaches its sample limit,
perf_event_overflow()triggers synchronously, invokingperf_event_throttle()which then callspai_stop().When
pai_stop()is called for a system-wide sampling event, it executes:list_del(PAI_SWLIST(event));Will deleting the event currently being iterated leave the list pointer at a poison value, causing a panic when the loop in
pai_have_samples()continues? Doespai_have_samples()need to uselist_for_each_entry_safe()instead?Quoted 3 lines
arch/s390/kernel/perf_pai.c +0
-0if (!event->attr.sample_period) {/* Counting */ pai_pmu[idx].pmu->read (event); }else {/* Sampling */
Signature · 9 lines
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany -- IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Wolfgang Wendt Geschäftsführung: David Faller Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
Quoted 5 lines · On 8/13/26 09:21, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Unsafe iteration overcpump->syswide_listinpai_have_samples()leads to a kernel crash (LIST_POISONdereference) if an event overflows and is synchronously throttled during the loop. --
This can not happen. The syswide_list is anchored by per-CPU pai_map::syswide_list and tracks all events which are bound to that particular CPU. The event was installed only on that particular CPU and does not move with the process/task. Each CPU has its own list anchored by
pai_map::syswide_list.
The call chains are either
paicrypt_sched_task() or paiext_sched_task
+--> pai_have_samples() iterates over syswide_list and calls
+--> pai_have_sampleor
pai_stop()
+--> list_del() to remove the event from that CPU.
Since a CPU can only execute one given code path at any one time:
- either schedule out
- or stop the event
both can not happen at the same time. PAI PMU does not install an interrupt handler, there is no supoprt for interrupts.
I think this is safe or am I mistaken?
...
Signature · 9 lines
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany -- IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Wolfgang Wendt Geschäftsführung: David Faller Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
Heiko Carstens
maintainerQuoted 17 lines · Thomas Richter
Quoted 5 lines · On 8/13/26 09:21, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Unsafe iteration overcpump->syswide_listinpai_have_samples()leads to a kernel crash (LIST_POISONdereference) if an event overflows and is synchronously throttled during the loop. --This can not happen. The
syswide_listis anchored by per-CPUpai_map::syswide_listand tracks all events which are bound to that particular CPU. The event was installed only on that particular CPU and does not move with the process/task. Each CPU has its own list anchored by
pai_map::syswide_list.The call chains are either
paicrypt_sched_task()orpaiext_sched_task+--> pai_have_samples() iterates over syswide_list and calls +--> pai_have_sample
...
Quoted 1 line
I think this is safe or am I mistaken?
Looks like this is not safe. The following can happen:
-> pai_have_sample()
-> pai_push_sample()
-> perf_event_overflow()
-> event->pmu->stop()
-> pai_stop()
-> list_del() modifies the list being traversed
-> potential crash when list traversal continuesAs sashiko proposed: using list_for_each_entry_safe() instead of list_for_each_entry() would avoid that potential scenario.