perf/core: Fix software event sample data, then stop zeroing it
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest3 patches 2 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | perf/core: Fix perf_sample_data reuse across software eventskernel/ | +7 | 0 replies | |
| 2/3 | perf/core: Don't let BPF programs read an uninitialized sample addresskernel/ | +3 | 0 replies | |
| 3/3 | perf/core: Don't zero perf_sample_data in do_perf_sw_event()kernel/ | +1 | 2 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it
do_perf_sw_event() builds samples for software events (context switches, page faults, ...) in an on-stack struct perf_sample_data. This series fixes two bugs around that struct and then stops zero-initializing it, a 256-byte memset that every context switch pays once a context-switch counter is open anywhere.
Patch 1 initializes the struct for each matching event. Today a sampling event inherits the sample ID, timestamp, pid/tid and callchain prepared for the event before it, and sometimes a record size smaller than what is then written, overrunning the records that follow in its ring buffer. Two concurrent "perf record -e page-faults,context-switches -c 1 -a" sessions make the older one fail. An unprivileged user's page-fault event can do this to root's CPU-wide session. Broken since v6.3.
Patch 2 stops BPF programs from reading an unset data->addr through bpf_perf_event_data::addr. On INIT_STACK_NONE kernels this returned leftover stack, mostly kernel pointers, on almost all context-switch and cpu-clock samples in my tests. INIT_STACK_ALL_ZERO mostly hides it. It affects all event types, needs CAP_BPF and CAP_PERFMON, and has been there since v6.1.
Patch 3 marks the struct __uninitialized. On every path from do_perf_sw_event(), each field is written before it is read, counting events never read the struct, and it is never copied as a whole, so its padding never leaves the kernel. A privileged BPF program using bpf_cast_to_kern_ctx() can still read unprepared fields, and would see stack contents instead of 0, as it already does on INIT_STACK_NONE kernels.
Patches 1-2 are for perf/urgent and stable, patch 3 for perf/core.
Performance: with sched_yield() ping-pong on one CPU, software counters add on average 54 ns to a ~627 ns context switch, and patch 3 brings that to 44 ns (10 rounds, spec_rstack_overflow=off).
Usama Arif (3):
perf/core: Fix perf_sample_data reuse across software events perf/core: Don't let BPF programs read an uninitialized sample address perf/core: Don't zero perf_sample_data in do_perf_sw_event()
kernel/events/core.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
Signature · 1 line
2.53.1-Meta