Skip to content

perf test: Per-test timeouts and JUnit XML schema compliance

Ian Rogers linux-perf-users lkml 6 versions – Applied

Status from this message: Applied to perf-tools-next, thanks!

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest3 patches 2 repliesApplied
  2. v53 patches 0 repliesSuperseded
  3. v43 patches 3 repliesApplied
Earlier versions · 3
VersionPostedPatchesRepliesStatus
v32 patches2 repliesSuperseded
v22 patches0 repliesSuperseded
v12 patches0 repliesSuperseded

Changes in v6 from the cover letter

- Reap finished child tests (invoking kill_timed_out_group()) before draining process.err in finish_test() and finish_tests_parallel(), and skip draining in handle_child_pipe_activity() once a timed out test has finished, so a noisy surviving subprocess cannot block the pipe drain loop before the process group is killed.

Patches in v6 Applied

#Subject and filesSizeRepliesApplied
1/3perf test: Add per-test timeout supporttools/perf/Documentation/perf-test.txt tools/perf/tests/builtin-test.c tools/perf/tests/tests.h+271 -370 replies
2/3perf test: Improve JUnit XML schema compliance and completenesstools/perf/Documentation/perf-test.txt tools/perf/tests/builtin-test.c+212 -1020 replies
3/3perf test: Don't count mlock budget wait time towards test timeouttools/perf/tests/builtin-test.c+63 -180 replies

Reviews trailers per patch

2 of 3 patches have a Reviewed-by or Acked-by.

PatchNamhyung Kim
1/3A-b
2/3A-b
3/3

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v6 thread

Cover letter [PATCH v6 0/3] perf test: Per-test timeouts and JUnit XML schema compliance

This series adds per-test timeout support to 'perf test', improves JUnit XML output to validate against strict JUnit schemas, and ensures time spent waiting for the shared mlock budget does not count towards a test's timeout or reported duration:

1. Add a per-test timeout (default 120s, configurable via '-t'/'--timeout', 0 to disable, disabled with '-F'/'--dont-fork'). Forked tests run in their own process groups and the existing polling loops in finish_test() and finish_tests_parallel() check elapsed runtimes, sending SIGTERM on timeout and escalating to SIGKILL after 1 second. Timed out tests are reported as 'TIMEOUT!', listed in the summary, and recorded as '<error message="TIMEOUT" type="timeout">' in JUnit XML output.

2. Update 'perf test -j' output to validate cleanly against both the Apache Ant/Windy Road JUnit schema (JUnit.xsd) and the Jenkins xUnit/Maven Surefire schema (junit-10.xsd), including required '<testsuite>' attributes, '<properties>' from uname, '<system-out/>' and '<system-err/>', self-closing tags, apostrophe escaping, and UTF-8/XML 1.0 codepoint validation.

3. Record each mlock-serialized test's start time in the shared mlock_budget mapping when it acquires mlock_budget->lock in run_test_case_mlocked(), so tests waiting for the mlock budget do not time out while waiting in line and their reported elapsed time reflects actual execution time.

v6:
- Reap finished child tests (invoking kill_timed_out_group()) before draining process.err in finish_test() and finish_tests_parallel(), and skip draining in handle_child_pipe_activity() once a timed out test has finished, so a noisy surviving subprocess cannot block the pipe drain loop before the process group is killed.

v5:
- Rebase onto tmp.perf-tools-next (patch 3 of v4 was applied upstream) and add Namhyung's Acked-by to patches 1 and 2.
- Add patch 3 so tests waiting on mlock_budget->lock don't time out while waiting.
- Increase the default per-test timeout from 60s to 120s. When running the full suite as root (where CAP_IPC_LOCK lifts the mlock limit and mlock_budget is unused), all ~180 non-exclusive tests start simultaneously at t=0 and contend heavily on CPUs, /proc scanning, and vmlinux/DWARF symbol resolution. On a 28-CPU hybrid machine, a 60s timeout resulted in 33 timeouts across the parallel burst and large sequential suites; increasing the default to 120s allowed longer tests ('kernel lock contention analysis test' at 115.3s, 'task-analyzer python test' at 94.7s, 'perf data type profiling tests' at 92.6s, 'perf metrics value validation' at 91.2s, 'perf record tests' at 79.0s, and 'python profiling with jitdump' at 62.7s) to complete, reducing timeouts from 33 to 2 ('perf record LBR tests' and 'perftool-testsuite_report') and overall suite wall-clock time from 503.5s to 363.9s.

v4:
- Drop the ITIMER_REAL timer and SIGALRM handler in patch 1, relying on the existing 10ms/100ms polling loops in finish_test() and finish_tests_parallel() to detect timeouts.
- Only fall back to kill(pid, sig) when kill(-pid, sig) fails, and ignore SIGTERM/SIGINT while waiting for subprocesses in run_test_child().
- Add patch 3 to avoid stack trace noise in verbose=1 failure snippets.

v3:
- Block SIGINT/SIGTERM while a test process may be reaped and until its pid is cleared, and skip already reaped tests in the signal path.

v2:
- Reset SIGALRM to SIG_DFL in the forked test child.
- Replace the blocking waitpid(-1) loop with a bounded, non-blocking wait that only runs when interrupted by SIGTERM/SIGINT.
- Ensure the SIGKILL escalation reaches the whole process group before the zombie is reaped.
- Factor out timespec_diff_secs() and TEST_KILL_GRACE_SECS.

Assisted-by Antigravity:gemini-3.1-pro

Ian Rogers (3):
perf test: Add per-test timeout support
perf test: Improve JUnit XML schema compliance and completeness perf test: Don't count mlock budget wait time towards test timeout

 tools/perf/Documentation/perf-test.txt |  12 +-
 tools/perf/tests/builtin-test.c        | 662 +++++++++++++++++++------
 tools/perf/tests/tests.h               |   7 +-
 3 files changed, 535 insertions(+), 146 deletions(-)
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog