perf tests: Robustness and performance improvements
Status from this message: Applied to perf-tools-next, thanks!
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v4 Latest13 patches 1 replyApplied
- v313 patches 6 repliesSuperseded
- v212 patches 3 repliesSuperseded
Earlier versions · 1
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v1 | 12 patches | 0 replies | Superseded |
Changes in v4 from the cover letter
- Rebased on top of latest ptn/perf-tools-next branch.
- Resolved merge conflicts in util/metricgroup.c, util/parse-events.c, and util/python.c from upstream evlist reference counting changes.
- Fixed a ShellCheck error in trace_record_replay.sh where local was used outside of a function.
- Cleaned up line length warnings from checkpatch.pl in test_brstack.sh.
- Simplified the perf kvm record test loop in kvm.sh to run sleep directly as a workload (instead of running in the background and killing), using -e cpu-clock explicitly to bypass the perf kvm option handling bug on x86/PowerPC platforms while remaining portable to architectures lacking a hardware cycles event (e.g. s390x).
Patches in v4 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/13 | perf parse-events: Restrict core PMU bypass to --cputype optiontools/ tools/ tools/ +8 more | +87 | 0 replies | |
| 2/13 | perf test: Truncate test description to fit terminal widthtools/ | +89 | 0 replies | |
| 3/13 | perf tests workloads: Support sub-second durations in noploop and thlooptools/ tools/ | +25 | 0 replies | |
| 4/13 | perf tests: Add robust record retry helper and use subsecond workloadstools/ tools/ tools/ +2 more | +214 | 0 replies | |
| 5/13 | perf tests: Skip metrics validation if system-wide recording lacks permissiontools/ tools/ tools/ | +60 | 0 replies | |
| 6/13 | perf tests: Fix Python JIT dump profiling test failuretools/ | +53 | 0 replies | |
| 7/13 | perf tests: Fix flakiness in trace record and replay testtools/ tools/ | +40 | 0 replies | |
| 8/13 | perf tests: Fix flakiness in BPF counters test on hybrid systemstools/ | +18 | 0 replies | |
| 9/13 | perf tests: Fix flakiness in branch stack sampling teststools/ | +66 | 0 replies | |
| 10/13 | perf tests: Speed up off-cpu profiling teststools/ | +7 | 0 replies | |
| 11/13 | perf tests: Speed up lock contention analysis shell testtools/ | +19 | 0 replies | |
| 12/13 | perf tests: Speed up metrics checking shell teststools/ tools/ | +2 | 0 replies | |
| 13/13 | perf tests: Include error output for skipped tests in JUnit XMLtools/ | +8 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.
Cover letter [PATCH v4 00/13] perf tests: Robustness and performance improvements
This patch series introduces several robustness and performance improvements to the perf test suite, alongside fixes for long-standing flakiness.
Key changes across the series:
- Introduces robust retry logic (perf_record_with_retry) to prevent spurious test failures due to transient EPERM issues during system-wide background profiling, safely extracting error messages from the PERF_RECORD_LOGS array.
- Significantly speeds up test execution across multiple scripts (such as kvm, record, trace, off-cpu, and lock contention) by supporting sub-second durations in the noploop and thloop workloads, reducing unnecessary wait times.
- Fixes flakiness in the BPF counters hybrid test by parsing taskset to determine valid CPUs rather than relying on shuf, which can fail under cgroups or missing binaries.
- Fixes JIT dump file leaks and TOCTOU vulnerabilities by securely staging with mktemp -d to guarantee cleanup.
- Decouples format alignments in builtin-test.c to gracefully truncate test descriptions without losing alignment or failing under non-TTY environments.
- Restricts uncore PMU bypass in event parsing strictly to --cputype to prevent unintended metric evaluation impacts.
- Injects trace output into JUnit XML <skipped> blocks to aid debugging.
Changes in v4:
- Rebased on top of latest ptn/perf-tools-next branch.
- Resolved merge conflicts in util/metricgroup.c, util/parse-events.c, and util/python.c from upstream evlist reference counting changes.
- Fixed a ShellCheck error in trace_record_replay.sh where local was used outside of a function.
- Cleaned up line length warnings from checkpatch.pl in test_brstack.sh.
- Simplified the perf kvm record test loop in kvm.sh to run sleep directly as a workload (instead of running in the background and killing), using -e cpu-clock explicitly to bypass the perf kvm option handling bug on x86/PowerPC platforms while remaining portable to architectures lacking a hardware cycles event (e.g. s390x).
Changes in v3:
- Fix line length warnings in checkpatch.pl.
- Fix wrap commit descriptions in checkpatch.pl.
- Re-aligned parse_events_state struct initializations.
- Added explanatory comment for the 10000 magic number in get_term_width().
- Added inline comments (e.g. /*cputype_filter=*/false) to boolean literal arguments.
- Fixed patch 4 to securely read from PERF_RECORD_LOGS array instead of unsafe wildcard cleanup.
- Fixed patch 6 TOCTOU vulnerability via mktemp -d staging directory.
- Implemented patch 13 to inject trace output into JUnit XML <skipped> blocks for debugging.
- Add Assisted-by tags to all commits.
Changes in v2:
- Drop the sleep 0 patch from v1.
- Introduce 'perf_record_with_retry' helper to encapsulate the retry logic and cleanly output error messages (patch 4).
- Fix issue with perf record lacking sufficient permissions on some setups (patch 5).
- Improve subsecond duration support in noploop/thloop (patch 3).
- Reduce test durations to avoid excessive overall runtimes.
Ian Rogers (13):
perf parse-events: Restrict core PMU bypass to --cputype option perf test: Truncate test description to fit terminal width perf tests workloads: Support sub-second durations in noploop and
thloop
perf tests: Add robust record retry helper and use subsecond workloads perf tests: Skip metrics validation if system-wide recording lacks
permission
perf tests: Fix Python JIT dump profiling test failure perf tests: Fix flakiness in trace record and replay test perf tests: Fix flakiness in BPF counters test on hybrid systems perf tests: Fix flakiness in branch stack sampling tests perf tests: Speed up off-cpu profiling tests
perf tests: Speed up lock contention analysis shell test perf tests: Speed up metrics checking shell tests
perf tests: Include error output for skipped tests in JUnit XML
tools/perf/builtin-script.c | 1 + tools/perf/builtin-stat.c | 20 +- tools/perf/tests/builtin-test.c | 173 ++++++++++-------- tools/perf/tests/expand-cgroup.c | 3 +- tools/perf/tests/parse-events.c | 11 +- tools/perf/tests/parse-metric.c | 3 +- tools/perf/tests/pmu-events.c | 10 +- tools/perf/tests/shell/jitdump-python.sh | 79 +++++--- tools/perf/tests/shell/kvm.sh | 61 +++--- .../tests/shell/lib/perf_metric_validation.py | 11 +- tools/perf/tests/shell/lib/perf_record.sh | 58 ++++++ tools/perf/tests/shell/lock_contention.sh | 32 ++-- tools/perf/tests/shell/pipe_test.sh | 4 +- tools/perf/tests/shell/record.sh | 173 +++++++++--------- tools/perf/tests/shell/record_lbr.sh | 50 +++-- tools/perf/tests/shell/record_offcpu.sh | 14 +- tools/perf/tests/shell/stat_all_metrics.sh | 77 +++++--- tools/perf/tests/shell/stat_bpf_counters.sh | 28 ++- tools/perf/tests/shell/stat_metrics_values.sh | 9 +- tools/perf/tests/shell/test_brstack.sh | 107 ++++++----- tools/perf/tests/shell/trace_record_replay.sh | 38 +++- tools/perf/tests/workloads/noploop.c | 17 +- tools/perf/tests/workloads/thloop.c | 16 +- tools/perf/util/metricgroup.c | 26 ++- tools/perf/util/metricgroup.h | 4 +- tools/perf/util/parse-events.c | 32 ++-- tools/perf/util/parse-events.h | 17 +- tools/perf/util/python.c | 3 +- 28 files changed, 687 insertions(+), 390 deletions(-) create mode 100644 tools/perf/tests/shell/lib/perf_record.sh
Signature · 1 line
2.55.0.rc0.786.g65d90a0328-goog