perf sched latency: Refine outputs, unit scaling, and histogram support
Status from this message: Applied to perf-tools-next, thanks!
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v9 Latest4 patches 2 repliesApplied
- v84 patches 3 repliesSuperseded
- v74 patches 6 repliesSuperseded
Earlier versions · 6
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v6 | 4 patches | 1 reply | Superseded | |
| v5 | 3 patches | 4 replies | Superseded | |
| v4 | 3 patches | 1 reply | Superseded | |
| v3 | 3 patches | 3 replies | Superseded | |
| v2 | 3 patches | 2 replies | Superseded | |
| v1 | 1 patch | 1 reply | Superseded |
Changes in v9 from the cover letter
- Added Reviewed-by: tag (Ian Rogers)
- Rebased against perf-tools-next/perf-tools-next (Namhyung Kim)
- Link to v8: https://lore.kernel.org/lkml/[email protected]/
Patches in v9 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | perf sched: Suppress latency table output when trace samples are missingtools/ | +12 | 0 replies | |
| 2/4 | perf sched: Handle missing trace samples in pipe modetools/ | +51 | 0 replies | |
| 3/4 | perf sched latency: Auto-scale latency and runtime display unitstools/ | +32 | 0 replies | |
| 4/4 | perf sched latency: Add histogram and time interval optionstools/ tools/ tools/ | +239 | 0 replies |
Reviews trailers per patch
All 4 patches have a Reviewed-by or Acked-by.
| Patch | Ian Rogers |
|---|---|
| 1/4 | R-b |
| 2/4 | R-b |
| 3/4 | R-b |
| 4/4 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v9 0/4] perf sched latency: Refine outputs, unit scaling, and histogram support
Hi Namhyung, Ian, Arnaldo,
This patch series improves 'perf sched latency' by suppressing misleading empty table output, extending pipe mode stream processing, introducing dynamic unit auto-scaling for latency and runtime statistics, and adding latency histogram visualisation alongside time-span filtering.
Patch 1 addresses an issue where 'perf sched latency' fell through and returned success (0) when perf_session__has_traces() failed due to missing tracepoint events in a perf.data file. This caused empty header tables and zeroed summary statistics to be rendered. In addition, thread__get_runtime() in map_switch_event() is guarded against potential NULL pointer dereferences under memory allocation failures.
Patch 2 extends pipe mode stream support. Because event attributes in pipe mode are received dynamically during event processing, session->evlist is not populated prior to event processing.
To handle pipe input correctly:
- Register the missing .attr, .tracing_data, .build_id, and .feature callbacks in cmd_sched()
- Promote the handlers array to file-scope (latency_handlers[]) and dynamically assign matching tracepoint handlers (or process_sched_ignore) inside perf_sched__process_tracepoint_sample() when evsel->handler is NULL; replace process_sched_wakeup_ignore() with process_sched_ignore()
- Perform the trace check post-processing when handling pipe data
Patch 3 introduces dynamic auto-scaling for latency and runtime display columns (Runtime, Avg delay, Max delay). Previously, all values were unconditionally formatted in milliseconds (ms), making microsecond- or second-scale latencies difficult to read. Columns are now dynamically scaled to the most appropriate unit (ns, us, ms, s), column headers are updated, and format specifiers are aligned character-for-character with table headers.
Patch 4 adds three new command-line options to 'perf sched latency':
--histogram (-H):
Displays an ASCII bar chart of CPU wait latencies between snapshots
--hist-mode:
Configures the bucketing scheme to either logarithmic (log) or 100 us equal-width linear (linear) mode
--time:
Filters trace event processing to a specified [start,stop] time span
Changes since v8:
- Added Reviewed-by: tag (Ian Rogers)
- Rebased against perf-tools-next/perf-tools-next (Namhyung Kim)
- Link to v8: https://lore.kernel.org/lkml/[email protected]/
Changes since v7:
- Added a guard clause in perf_sched__process_tracepoint_sample() to reduce loop indentation (Namhyung Kim)
- Converted dummy process_sched_wakeup_ignore() references to process_sched_ignore and eliminated redundant code (Namhyung Kim)
- Replaced string comparisons with faster integer checks (i.e., thread__tid(...) != 0) for swapper (idle thread) handling (Namhyung Kim)
- Added automated test cases for --histogram, --hist-mode, and --time into tools/perf/tests/shell/sched.sh (Namhyung Kim)
- Link to v7: https://lore.kernel.org/lkml/[email protected]/
Changes since v6:
- Fixed O(N) hot-path traversal overhead in perf_sched__process_tracepoint_sample() for unhandled tracepoints by assigning a dummy ignore handler (process_sched_ignore)
- Replaced evlist__set_tracepoints_handlers() in sample processing with a per-evsel lookup to prevent -EEXIST early aborts and avoid dropping late-arriving pipe events
- Updated commit message for the pipe mode patch to reflect the single-pass handler assignment logic
- Link to v6: https://lore.kernel.org/lkml/[email protected]/
Changes since v5:
- Split pipe mode trace sample handling from Patch 1 into a standalone patch (Namhyung Kim)
- Added a Fixes: tag to the pipe mode patch referencing commit 27295592c22e ("perf session: Share the common trace sample_check routine as perf_session__has_traces")
- Updated commit log with before and after illustrations of table header formatting
- Link to v5: https://lore.kernel.org/lkml/[email protected]/
Changes since v4:
- Added the missing .feature callback to sched.tool in cmd_sched()
- Fixed memory leak by reusing uncompleted work atoms when sched_in events are skipped or lost
- Promoted the handlers array to file-scope (i.e., latency_handlers[]) and added a dynamic lookup via evlist__set_tracepoints_handlers() inside perf_sched__process_tracepoint_sample() whenever a sample arrives with an unattached handler (i.e., evsel->handler == NULL)
- Prevented double-counting wakeups (i.e., ignore sched:sched_wakeup) in pipe mode and non-pipe modes
- Added missing trailing pipe in output_lat_thread's new auto-scaling format string
- Stripped redundant prefix strings from each row's format string to produce a clean, tabular output
- Link to v4: https://lore.kernel.org/lkml/[email protected]/
Changes since v3:
- Registered Missing Callbacks. In perf_tool__init configuration inside cmd_sched(), added the .attr, .tracing_data, and .build_id callbacks. Without these callbacks, pipe mode drops header attributes entirely, preventing tracepoints from being populated in session->evlist
- Introduced an explicit post-processing pipe check. This ensures that pipe mode aborts correctly and does not produce superfluous empty latency tables when no trace samples are available
- Prevented potential NULL pointer dereference. Added a NULL check for thread__get_runtime() in map_switch_event() under memory allocation failures
- Fixed column alignment. Modified format specifiers to align character-for-character with header column widths across all rows
- Adjusted header label padding. Updated header label padding and column width delimiters in perf_sched__lat() to match the exact field format specifiers printed in output_lat_thread()
- Fixed swapper histogram inclusion. Excluded "swapper" (i.e., CPU-specific idle thread) from global_hist buckets
- Preserved task state machine across --time bounds. Moved time-window filtering inside add_sched_in_event() to gate latency statistics recording without breaking wakeup state tracking across time interval boundaries
- Link to v3: https://lore.kernel.org/lkml/[email protected]/
Changes since v2:
- Ensured vertical pipe separators align across all latency table columns
- Excluded "swapper" (idle thread) latency samples from global_hist
- Preserved task state machine transitions across --time boundaries
- Suppressed empty table headers, total lines, and histogram graphs when no matching trace samples exist (e.g., when --time, --CPU, or --pids exclude all samples), outputting "No matching trace samples found." instead
- Linked to v2: https://lore.kernel.org/lkml/[email protected]/
Changes since v1:
- Fixed integer overflow in latency_bucket() (Ian Rogers)
- Linked to v1: https://lore.kernel.org/lkml/[email protected]/
Aaron Tomlin (4):
perf sched: Suppress latency table output when trace samples are
missing
perf sched: Handle missing trace samples in pipe mode perf sched latency: Auto-scale latency and runtime display units perf sched latency: Add histogram and time interval options
tools/perf/Documentation/perf-sched.txt | 6 + tools/perf/builtin-sched.c | 347 ++++++++++++++++++++---- tools/perf/tests/shell/sched.sh | 29 ++ 3 files changed, 332 insertions(+), 50 deletions(-)
Signature · 1 line
2.55.0