Skip to content

perf python: Follow-up fixes and shell test deflaking

Ian Rogers linux-perf-users lkml 3 versions Applied

Status from this message: Thanks, applied to perf-tools-next, for v7.4.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest9 patches 1 replyApplied
  2. v29 patches 0 repliesSuperseded
  3. v19 patches 0 repliesSuperseded

Changes in v3 from the cover letter

- Wrap _open_live_evlist() inside the try/except KeyboardInterrupt block in stat-cpi.py so signals arriving during event initialization are caught and cleaned up gracefully.

Patches in v3 Applied

#Subject and filesSizeRepliesApplied
1/9perf evsel: Clamp sample_id size for ksymbol, bpf, and text_poke eventstools/perf/util/evlist.c tools/perf/util/evsel.c tools/perf/util/evsel.h+63 -30 replies
2/9perf python sctop: Fix offline interval printing and test flakinesstools/perf/python/sctop.py tools/perf/tests/shell/test_sctop_python.sh+34 -310 replies
3/9perf python stat-cpi: Fix live mode signal races and test flakinesstools/perf/python/stat-cpi.py tools/perf/tests/shell/test_stat_cpi_python.sh+31 -130 replies
4/9perf test: Deflake Intel PT Python shell tests under loadtools/perf/tests/shell/test_export_to_postgresql_python.sh tools/perf/tests/shell/test_export_to_sqlite_python.sh tools/perf/tests/shell/test_intel_pt_events_python.sh+79 -630 replies
5/9perf test: Deflake failed-syscalls Python shell tests under loadtools/perf/tests/shell/test_failed_syscalls_by_pid_python.sh tools/perf/tests/shell/test_failed_syscalls_python.sh+71 -510 replies
6/9perf test: Reduce overhead and contention in Python shell teststools/perf/tests/shell/test_check_perf_trace_python.sh tools/perf/tests/shell/test_compaction_times_python.sh tools/perf/tests/shell/test_event_analyzing_sample_python.sh +16 more+58 -430 replies
7/9perf python event_analyzing_sample: Default to in-memory SQLite databasetools/perf/python/event_analyzing_sample.py tools/perf/tests/shell/test_event_analyzing_sample_python.sh+17 -260 replies
8/9perf python: Initialize debug output on module loadtools/perf/util/debug.c tools/perf/util/python.c+2 -10 replies
9/9perf pmu: Fix race with concurrent tracepoint creation and removal in perf listtools/perf/util/pmus.c tools/perf/util/tp_pmu.c+23 -60 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCH v3 0/9] perf python: Follow-up fixes and shell test deflaking

This series contains follow-up fixes for the standalone Python scripts and their shell tests, particularly when running 'perf test' as root under heavy parallel load:

- Clamp sample_id size calculation for PERF_RECORD_KSYMBOL, PERF_RECORD_BPF_EVENT, and PERF_RECORD_TEXT_POKE records to work around kernels without "perf/core: Restore header fields in sideband output callbacks" [1].
- Fix offline interval printing in sctop.py and live signal races in stat-cpi.py.
- Deflake Intel PT, failed-syscalls, and other Python shell tests under parallel 'perf test' load by scoping tracepoint checks to 'perf list tracepoint', passing '-B -N --no-bpf-event' to 'perf record', and avoiding unnecessary system-wide ('-a') recordings.
- Default event_analyzing_sample.py to an in-memory SQLite database (':memory:') when '--db' is not specified.
- Initialize perf debug output ('perf_debug_setup()') when loading the 'perf' Python extension module.
- Fix races in 'perf list' ('perf_pmus__print_pmu_events()' and 'tp_pmu.c') when concurrent tests dynamically create and remove kprobe/uprobes tracepoints.

v3:
- Wrap _open_live_evlist() inside the try/except KeyboardInterrupt block in stat-cpi.py so signals arriving during event initialization are caught and cleaned up gracefully.

v2:
- Fix 8-byte alignment calculation for PERF_RECORD_TEXT_POKE payload in evsel__event_size() by aligning fixed + old_len + new_len.
- Keep offline interval clock advancing on all samples in sctop.py while flushing any remaining syscalls at EOF.
- Use 'mktemp -d' private directories in test_sctop_python.sh and test_failed_syscalls_python.sh so temporary files are not unlinked and recreated directly in /tmp.
- Explicitly include <stdlib.h> in tools/perf/util/pmus.c.

[1] https://lore.kernel.org/r/[email protected]

Ian Rogers (9):
perf evsel: Clamp sample_id size for ksymbol, bpf, and text_poke
events
perf python sctop: Fix offline interval printing and test flakiness perf python stat-cpi: Fix live mode signal races and test flakiness perf test: Deflake Intel PT Python shell tests under load perf test: Deflake failed-syscalls Python shell tests under load perf test: Reduce overhead and contention in Python shell tests perf python event_analyzing_sample: Default to in-memory SQLite
database
perf python: Initialize debug output on module load
perf pmu: Fix race with concurrent tracepoint creation and removal in
perf list

 tools/perf/python/event_analyzing_sample.py   | 35 +++-------
 tools/perf/python/sctop.py                    |  7 +-
 tools/perf/python/stat-cpi.py                 | 32 ++++++---
 .../shell/test_check_perf_trace_python.sh     |  8 ++-
 .../shell/test_compaction_times_python.sh     |  8 ++-
 .../test_event_analyzing_sample_python.sh     | 11 ++-
 .../shell/test_export_to_postgresql_python.sh | 49 +++++++------
 .../shell/test_export_to_sqlite_python.sh     | 50 ++++++++------
 .../test_failed_syscalls_by_pid_python.sh     | 68 +++++++++++--------
 .../shell/test_failed_syscalls_python.sh      | 54 +++++++++------
 .../tests/shell/test_flamegraph_python.sh     |  5 +-
 .../shell/test_futex_contention_python.sh     |  7 +-
 tools/perf/tests/shell/test_gecko_python.sh   |  3 +-
 .../shell/test_intel_pt_events_python.sh      | 43 ++++++------
 .../tests/shell/test_mem_phys_addr_python.sh  |  8 ++-
 .../shell/test_net_dropmonitor_python.sh      |  9 +--
 .../tests/shell/test_netdev_times_python.sh   |  6 +-
 .../tests/shell/test_powerpc_hcalls_python.sh |  4 +-
 .../tests/shell/test_rw_by_file_python.sh     |  5 +-
 .../perf/tests/shell/test_rw_by_pid_python.sh |  4 +-
 tools/perf/tests/shell/test_rwtop_python.sh   |  4 +-
 .../shell/test_sched_migration_python.sh      |  4 +-
 tools/perf/tests/shell/test_sctop_python.sh   | 58 ++++++++--------
 .../tests/shell/test_stackcollapse_python.sh  |  2 +-
 .../perf/tests/shell/test_stat_cpi_python.sh  | 12 +++-
 .../test_syscall_counts_by_pid_python.sh      |  6 +-
 .../tests/shell/test_syscall_counts_python.sh |  5 +-
 tools/perf/tests/shell/test_task_analyzer.sh  |  3 +-
 .../tests/shell/test_wakeup_latency_python.sh |  7 +-
 tools/perf/util/debug.c                       |  2 +-
 tools/perf/util/evlist.c                      |  6 +-
 tools/perf/util/evsel.c                       | 59 +++++++++++++++-
 tools/perf/util/evsel.h                       |  1 +
 tools/perf/util/pmus.c                        | 21 ++++--
 tools/perf/util/python.c                      |  1 +
 tools/perf/util/tp_pmu.c                      |  8 ++-
 36 files changed, 378 insertions(+), 237 deletions(-)
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog