v5 of the perf weight annotation patchkit
Fixes the latest sashiko issues
PATCH v5 1/4 perf tools record: Modernize -W man page
Modernize the -W / --weight description in the manpage to cover more cases that are supported now.
--- tools/perf/Documentation/perf-record.txt | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
tools/perf/Documentation/perf-record.txt +5 -3
@@ -480,9 +480,11 @@ Note that this feature may not be available on all processors. -W:: --weight:: -Enable weightened sampling. An additional weight is recorded per sample and can be -displayed with the weight and local_weight sort keys. This currently works for TSX -abort events and some memory events in precise mode on modern Intel CPUs. +Enable weighted sampling. One or more additional weights can be recorded per sample +and can be displayed with the weight, weight2, weight3 and local_weight sort keys. +Possible uses for weights are memory or instruction retirement latencies. +Support depends on the PMU and the event, as well as its configuration. Typically +it requires use of 'p'. --namespaces:: Record events of type PERF_RECORD_NAMESPACES. This enables 'cgroup_id' sort key.
Signature · 1 line
2.55.0
PATCH v5 2/4 perf tools top: Add --weight option
In reply to Andi KleenAdd a -W/--weight option to perf top to collect weights too. Useful with follow on patches.
--- tools/perf/Documentation/perf-top.txt | 6 ++++++ tools/perf/builtin-top.c | 2 ++ 2 files changed, 8 insertions(+)
tools/perf/Documentation/perf-top.txt +6 -0
@@ -25,6 +25,12 @@ OPTIONS --count=<count>:: Event period to sample. +-W:: +--weight:: +Enable weighted sampling. One or more additional weights can be recorded per sample +and can be displayed with the weight, weight2, weight3 and local_weight sort keys. +Possible uses for weights are memory or instruction retirement latencies. + -C <cpu-list>:: --cpu=<cpu>:: Monitor only on the list of CPUs provided. Multiple CPUs can be provided as a
tools/perf/builtin-top.c +2 -0
@@ -1493,6 +1493,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter),OPT_U64 ('c' ,"count" , &opts->user_interval,"event period to sample" ), +OPT_BOOLEAN ('W' ,"weight" , &opts->sample_weight, +"sample by weight (on special events only)" ),OPT_STRING ('p' ,"pid" , &target->pid,"pid" ,"profile events on existing process id" ), OPT_STRING('t', "tid", &target->tid, "tid",
Signature · 1 line
2.55.0
PATCH v5 3/4 perf tools: Add support for displaying weights in annotate
In reply to Andi KleenAdd support for showing all the three possible per IP weights in annotate. The weights are shown by defaults if any are non zero. This is useful, especially with the new insn lat statistics, but also for all the existing weights.
Add a hotkey to the interactive browser to turn them off (w), as well as a perf annotate command line option.
The weights are stored unconditionally in the sym_hist_entry, which will increase memory consumption somewhat.
--- tools/perf/Documentation/perf-annotate.txt | 3 + tools/perf/Documentation/perf-report.txt | 6 + tools/perf/builtin-annotate.c | 3 + tools/perf/builtin-report.c | 3 + tools/perf/ui/browsers/annotate.c | 20 +++- tools/perf/util/annotate.c | 121 +++++++++++++++++++-- tools/perf/util/annotate.h | 44 +++++++- tools/perf/util/symbol_conf.h | 13 ++- 8 files changed, 194 insertions(+), 19 deletions(-)
tools/perf/Documentation/perf-annotate.txt +3 -0
@@ -95,6 +95,9 @@ include::itrace.txt[] --gtk:: Use the GTK interface. +--weights:: Show or hide (with `--no-weights`) weight columns in annotation output. + By default, weight columns are shown when samples contain non-zero weights. + -C:: --cpu=<cpu>:: Only report samples for the list of CPUs provided. Multiple CPUs can be provided as a comma-separated list with no space: 0,1. Ranges of
tools/perf/Documentation/perf-report.txt +6 -0
@@ -64,6 +64,9 @@ OPTIONS --symbol-filter=:: Only show symbols that match (partially) with this filter. +--weights:: Show or hide (with `--no-weights`) weight columns in annotation output. + By default, weight columns are shown when samples contain non-zero weights. + -U:: --hide-unresolved:: Only display entries resolved to a symbol.
@@ -353,6 +356,9 @@ OPTIONS --gtk:: Use the GTK2 interface. +--weights:: Show or hide (with `--no-weights`) weight columns in annotation output. + By default, weight columns are shown when samples contain non-zero weights. + -k:: --vmlinux=<file>:: vmlinux pathname
tools/perf/builtin-annotate.c +3 -0
@@ -719,6 +719,8 @@ int cmd_annotate(int argc, const char **argv)OPT_BOOLEAN (0 ,"tui" , &annotate.use_tui,"Use the TUI interface" ), #endifOPT_BOOLEAN (0 ,"stdio" , &annotate.use_stdio,"Use the stdio interface" ), +OPT_BOOLEAN (0 ,"weights" , &symbol_conf.annotate_weight, +"Show or hide weight columns in annotation. Default show if non zero." ),OPT_BOOLEAN (0 ,"stdio2" , &annotate.use_stdio2,"Use the stdio interface" ),OPT_BOOLEAN (0 ,"ignore-vmlinux" , &symbol_conf.ignore_vmlinux, "don't load vmlinux even if found"),
@@ -788,6 +790,7 @@ int cmd_annotate(int argc, const char **argv)set_option_flag (options,0 ,"show-total-period" ,PARSE_OPT_EXCLUSIVE );set_option_flag (options,0 ,"show-nr-samples" ,PARSE_OPT_EXCLUSIVE ); + symbol_conf.annotate_weight = true;annotation_options__init ();
tools/perf/builtin-report.c +3 -0
@@ -1364,6 +1364,8 @@ int cmd_report(int argc, const char **argv)#endif OPT_BOOLEAN (0 ,"stdio" , &report.use_stdio,"Use the stdio interface" ), +OPT_BOOLEAN (0 ,"weights" , &symbol_conf.annotate_weight, +"Show or hide weight columns in annotation. Default show if non-zero." ),OPT_BOOLEAN (0 ,"header" , &report.header,"Show data header." ),OPT_BOOLEAN (0 ,"header-only" , &report.header_only, "Show only data header."),
@@ -1525,6 +1527,7 @@ int cmd_report(int argc, const char **argv)* reference exited threads. */ symbol_conf.keep_exited_threads = true; + symbol_conf.annotate_weight = true;annotation_options__init ();
tools/perf/ui/browsers/annotate.c +16 -4
@@ -189,7 +189,7 @@ static void annotate_browser__draw_current_jump(struct ui_browser *browser)struct map_symbol *ms = ab->b.priv;struct symbol *sym = ms->sym;struct annotation *notes =symbol__annotation (sym); -u8 pcnt_width =annotation__pcnt_width (notes); +u8 pcnt_width =annotation__pcnt_width (notes, ab->evsel);u8 cntr_width =annotation__br_cntr_width ();int width;int diff =0 ;
@@ -255,7 +255,8 @@ static unsigned int annotate_browser__refresh(struct ui_browser *browser) {struct annotation *notes =browser__annotation (browser);int ret =ui_browser__list_head_refresh (browser); -int pcnt_width =annotation__pcnt_width (notes); +int pcnt_width =annotation__pcnt_width (notes, + container_of(browser, struct annotate_browser, b)->evsel);if (annotate_opts.jump_arrows)annotate_browser__draw_current_jump (browser);
@@ -972,6 +973,7 @@ static int annotate_browser__run(struct annotate_browser *browser,"O Bump offset level (jump targets -> +call -> all -> cycle thru)\n" "s Toggle source code view\n" "t Circulate percent, total period, samples view\n" +"w Toggle weight columns\n" "c Show min/max cycle\n" "/ Search string\n" "k Toggle line numbers\n"
@@ -1091,6 +1093,14 @@ static int annotate_browser__run(struct annotate_browser *browser, symbol_conf.show_total_period = true;annotation__update_column_widths (notes);continue ; +case 'w' : + symbol_conf.annotate_weight = !symbol_conf.annotate_weight; + browser->b.width = notes->src->widths.max_line_len + +annotation__pcnt_width (notes, evsel) + +annotation__cycles_width (notes) + +annotation__br_cntr_width (); +ui_browser__refresh_dimensions (&browser->b); +continue ;case 'c' :if (annotate_opts.show_minmax_cycle) annotate_opts.show_minmax_cycle = false;
@@ -1225,10 +1235,12 @@ int __hist_entry__tui_annotate(struct hist_entry *he, struct map_symbol *ms, browser.type_hash =hashmap__new (type_hash, type_equal,/*ctx=*/ NULL ); } - browser.b.width = notes->src->widths.max_line_len; + browser.b.width = notes->src->widths.max_line_len + +annotation__pcnt_width (notes, evsel) + +annotation__cycles_width (notes) + +annotation__br_cntr_width (); browser.b.nr_entries = notes->src->nr_entries; browser.b.entries = ¬es->src->source; - browser.b.width +=18 ;/* Percentage */ if (annotate_opts.hide_src_code)ui_browser__init_asm_mode (&browser.b);
tools/perf/util/annotate.c +109 -12
221 lines not on this page · raw message
tools/perf/util/annotate.h +42 -2
71 lines not on this page · raw message
tools/perf/util/symbol_conf.h +12 -1
@@ -27,6 +27,16 @@ enum a2l_style { };#define MAX_A2L_STYLE (A2L_STYLE_CMD + 1) +enum symbol__weight_mode { +WEIGHT_NONE =0 , +WEIGHT_WEIGHT , +WEIGHT_INSNLAT , +WEIGHT_WEIGHT3 , +}; + +#define for_each_weight (_weight) \ + for ((_weight) = WEIGHT_WEIGHT; (_weight) <= WEIGHT_WEIGHT3; (_weight)++) +struct symbol_conf {bool nanosecs;unsigned short priv_size;
@@ -69,7 +79,8 @@ struct symbol_conf { annotate_data_sample, skip_empty, enable_latency, - prefer_latency; + prefer_latency, + annotate_weight;const char *vmlinux_name, *kallsyms_name, *source_prefix,
Signature · 1 line
2.55.0
PATCH v5 4/4 perf tools: Add test for weight annotation
In reply to Andi KleenAdd a test case to check if --stdio weight annotation works
--- tools/perf/tests/shell/annotate_weight.sh | 63 +++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100755 tools/perf/tests/shell/annotate_weight.sh
tools/perf/tests/shell/annotate_weight.sh new file +63 -0
@@ -0,0 +1,63 @@ +#!/bin/bash +# perf annotate weight regression test +# SPDX-License-Identifier: GPL-2.0 + +set -e + +shelldir=$(dirname "$0") +# shellcheck source=tools/perf/tests/shell/lib/perf_has_symbol.sh +. "${shelldir}"/lib/perf_has_symbol.sh + +testsym="test_loop" +skip_test_missing_symbol "${testsym}" + +perfdata=$(mktemp /tmp/__perf_test.annotate_weight.XXXXX) +record_log=$(mktemp /tmp/__perf_test.annotate_weight.XXXXX.log) +report_out=$(mktemp /tmp/__perf_test.annotate_weight.XXXXX.report) +annotate_out=$(mktemp /tmp/__perf_test.annotate_weight.XXXXX.annotate) + +cleanup() { + rm -f "${perfdata}" "${record_log}" "${report_out}" "${annotate_out}" + trap - EXIT TERM INT +} + +trap 'cleanup; exit 1' TERM INT +trap cleanup EXIT + +# mem-loads:pu requests a precise user PEBS event whose sample weight should +# be populated by -W. Unsupported PEBS/weight PMUs are skipped below. +if ! perf record -W -e mem-loads:pu -o "${perfdata}" -- perf test -w thloop \ + > /dev/null 2> "${record_log}"; then + echo "[SKIP] precise PEBS weight sampling is unavailable" + exit 2 +fi + +# Confirm the PMU actually produced nonzero weights. A successful record alone +# is insufficient: some PMUs accept the event but provide no weight payload. +if ! perf report --stdio -i "${perfdata}" --fields=weight1,weight2,weight3,symbol --percent-limit 0 \ + > "${report_out}" 2> "${record_log}"; then + echo "[SKIP] weighted samples cannot be decoded" + exit 2 +fi + +if ! awk '$1 ~ /^[0-9]/ && ($1 + 0) > 0 { found = 1 } END { exit !found }' \ + "${report_out}"; then + echo "[SKIP] PEBS weight sampling produced no nonzero weights" + exit 2 +fi + +perf annotate --stdio -i "${perfdata}" --symbol "${testsym}" \ + > "${annotate_out}" 2> "${record_log}" + +grep -q 'Percent Weight' "${annotate_out}" + +# The second numeric column is the rendered average weight. This assertion +# fails on the original regression because calc_percent() left it at zero. +if ! awk '$1 ~ /^[0-9]/ && $2 ~ /^[0-9]/ && ($2 + 0) > 0 { found = 1 } END { exit !found }' \ + "${annotate_out}"; then + echo "Annotation output contained no nonzero weight" + cat "${annotate_out}" + exit 1 +fi + +echo "PEBS annotation weights: PASS"
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH v5 1/4] perf tools record: Modernize -W man page
In reply to Andi KleenSashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
Modernize the -W / --weight description in the manpage to cover more cases that are supported now.
Signed-off-by Andi Kleen <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[email protected]
AI reviewRe: [PATCH v5 2/4] perf tools top: Add --weight option
In reply to Andi KleenThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Buffer overflow or NULL pointer dereference in for_each_drm_fdinfo_in_dir() due to unchecked reallocarray() failure.
- [High] Use-After-Free and memory leak in add_event() when strndup() fails.
--
commit f1674c1fafe728985095507f0cfd95caa7a7179c
Author: Andi Kleen <[email protected]>
perf tools top: Add --weight option
Add a -W/--weight option to perf top to collect weights too. Useful with follow on patches.
tools/perf/Documentation/perf-top.txt OPTIONS · line 30
Quoted 11 lines
tools/perf/Documentation/perf-top.txt +3
-0@@ -25,6 +25,12 @@ OPTIONS --count=<count>:: Event period to sample. +-W:: +--weight:: +Enable weighted sampling. One or more additional weights can be recorded per sample
[ ... ]
While reviewing this, I noticed some pre-existing issues in tools/perf/util/drm_pmu.c:
tools/perf/util/drm_pmu.c:for_each_drm_fdinfo_in_dir() {
...
if (minors->minors_num == minors->minors_len) {
unsigned int *tmp = reallocarray(minors->minors, minors->minors_len + 4,
sizeof(unsigned int));
if (tmp) {
minors->minors = tmp;
minors->minors_len += 4;
}
}
minors->minors[minors->minors_num++] = minor;
...
}[Severity: High]
This is a pre-existing issue, but does this code write out of bounds or dereference a NULL pointer?
If reallocarray() fails, tmp is NULL, so minors->minors and minors->minors_len are not updated.
However, the function proceeds to unconditionally assign minor to minors->minors[minors->minors_num++].
If the array was initially NULL, this dereferences a NULL pointer. If it was already allocated, this writes past the end of the un-resized buffer.
tools/perf/util/drm_pmu.c:add_event() {
...
tmp = reallocarray(*events, *num_events + 1, sizeof(struct drm_pmu_event));
if (!tmp)
return -ENOMEM;
tmp[*num_events].unit = unit;
tmp[*num_events].desc = desc;
tmp[*num_events].name = strndup(line, colon - line);
if (!tmp[*num_events].name)
return -ENOMEM;
(*num_events)++;
*events = tmp;
return 0;
}[Severity: High]
This is a pre-existing issue, but does this code leave a dangling pointer and leak memory when strndup() fails?
If reallocarray() succeeds and moves the memory block, the old memory pointed to by *events is freed.
If strndup() then fails, the function returns -ENOMEM without updating *events to point to tmp.
This leaks the newly allocated tmp block and leaves *events as a dangling pointer to the old, freed memory block.
Since the caller read_drm_pmus_cb() ignores the return value of add_event() and continues parsing, a subsequent match could pass this dangling *events pointer back into reallocarray(), triggering a use-after-free.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
[email protected]
AI reviewRe: [PATCH v5 3/4] perf tools: Add support for displaying weights in annotate
In reply to Andi KleenSashiko has reviewed this patch and found no issues. It looks great!
Quoted 5 lines
Add support for showing all the three possible per IP weights in annotate. The weights are shown by defaults if any are non zero. This is useful, especially with the new insn lat statistics, but also for all the existing weights.
Quoted 8 lines
Add a hotkey to the interactive browser to turn them off (w), as well as a perf annotate command line option.
The weights are stored unconditionally in the
sym_hist_entry, which will increase memory consumption somewhat.Assisted-by omp:GPT-5.6-LunaSigned-off-by Andi Kleen <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH v5 4/4] perf tools: Add test for weight annotation
In reply to Andi KleenThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] The script conflates PEBS support checking with workload execution, masking workload failures as…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] The script conflates PEBS support checking with workload execution, masking workload failures as skipped tests.
- [Medium] Silent test failure due to unhandled grep -q execution under set -e.
--
commit 8db1611c3b92cee56576e6157914227faf90fc5a
Author: Andi Kleen <[email protected]>
perf tools: Add test for weight annotation
This commit adds a shell script test case to check if the --stdio weight annotation functions correctly in perf annotate.
Quoted 5 lines
tools/perf/tests/shell/annotate_weight.sh new file +0
-0
[ ... ]
tools/perf/tests/shell/annotate_weight.sh line 33
Quoted 10 lines
tools/perf/tests/shell/annotate_weight.sh new file +10
-0+trap 'cleanup; exit 1' TERM INT +trap cleanup EXIT + +# mem-loads:pu requests a precise user PEBS event whose sample weight should +# be populated by -W. Unsupported PEBS/weight PMUs are skipped below. +if ! perf record -W -e mem-loads:pu -o "${perfdata}" -- perf test -w thloop \ + > /dev/null 2> "${record_log}"; then + echo "[SKIP] precise PEBS weight sampling is unavailable" + exit 2 +fi
[Severity: Medium]
If the inner workload (perf test -w thloop) crashes or otherwise fails to execute, does this conditional block improperly catch the failure and report it as an unsupported hardware feature?
It looks like any non-zero exit status from the child process will result in a skipped test, which might mask underlying environment or test suite bugs.
[ ... ]
tools/perf/tests/shell/annotate_weight.sh line 52
Quoted 8 lines
tools/perf/tests/shell/annotate_weight.sh new file +8
-0+ echo "[SKIP] PEBS weight sampling produced no nonzero weights" + exit 2 +fi + +perf annotate --stdio -i "${perfdata}" --symbol "${testsym}" \ + > "${annotate_out}" 2> "${record_log}" + +grep -q 'Percent Weight' "${annotate_out}"
[Severity: Medium]
Since set -e is enabled at the top of the script, will this isolated grep command cause an abrupt and silent exit if the 'Percent Weight' header is missing from the annotation output?
If the string is missing, grep will exit with code 1, which might instantly abort the script without printing diagnostic output, making regressions harder to debug.
Quoted 2 lines
tools/perf/tests/shell/annotate_weight.sh new file +2
-0+ +# The second numeric column is the rendered average weight. This assertion
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH v5 2/4] perf tools top: Add --weight option
In reply to [email protected]Quoted 7 lines
[Severity: High]
This is a pre-existing issue, but does this code write out of bounds or dereference a NULL pointer?If
reallocarray()fails, tmp is NULL, sominors->minorsandminors->minors_lenare not updated.
OOM checking in user space is generally a folly anyways because if it ever happens you're already in russian roulette with the OOM killer killing you.
The right fix would be to use a xreallocarray that exits on error.
But it has nothing to do with these patches.
-Andi
Re: [PATCH v5 4/4] perf tools: Add test for weight annotation
In reply to [email protected]Quoted 13 lines
tools/perf/tests/shell/annotate_weight.sh line 33
Quoted 7 lines
tools/perf/tests/shell/annotate_weight.sh new file +7
-0+# mem-loads:pu requests a precise user PEBS event whose sample weight should +# be populated by -W. Unsupported PEBS/weight PMUs are skipped below. +if ! perf record -W -e mem-loads:pu -o "${perfdata}" -- perf test -w thloop \ + > /dev/null 2> "${record_log}"; then + echo "[SKIP] precise PEBS weight sampling is unavailable" + exit 2 +fi[Severity: Medium]
If the inner workload (perf test -w thloop) crashes or otherwise fails to execute, does this conditional block improperly catch the failure and report it as an unsupported hardware feature?
Quoted 2 lines
It looks like any non-zero exit status from the child process will result in a skipped test, which might mask underlying environment or test suite bugs.
There are other tests for this, so this is acceptable.
Quoted 16 lines
[ ... ]
tools/perf/tests/shell/annotate_weight.sh line 52
Quoted 8 lines
tools/perf/tests/shell/annotate_weight.sh new file +8
-0+ echo "[SKIP] PEBS weight sampling produced no nonzero weights" + exit 2 +fi + +perf annotate --stdio -i "${perfdata}" --symbol "${testsym}" \ + > "${annotate_out}" 2> "${record_log}" + +grep -q 'Percent Weight' "${annotate_out}"[Severity: Medium]
Since set -e is enabled at the top of the script, will this isolated grep command cause an abrupt and silent exit if the 'Percent Weight' header is missing from the annotation output?
Quoted 3 lines
If the string is missing, grep will exit with code 1, which might instantly abort the script without printing diagnostic output, making regressions harder to debug.
it will exit with 1 which makes perf test fail I believe.
-Andi
Quoted 2 lines · Andi Kleen
Fixes the latest sashiko issues
Thanks,
Namhyung