perf test all metrics: Fully ignore Default metric failures
Ian Rogers
reviewerPATCH 1/1 perf test all metrics: Fully ignore Default metric failures
Determine if a metric is default from perf list --raw-dump $m eg:
``` $ perf list --raw-dump l1_prefetch_miss_rate Default4 l1_prefetch_miss_rate ```
If a metric has "not supported" or "no supported events" then ignore these failures for default metrics. Tidy up the skip/fail messages in the output to make them easier to spot/read.
--- tools/perf/tests/shell/stat_all_metrics.sh | 27 ++++++++++++++-------- 1 file changed, 17 insertions(+), 10 deletions(-)
tools/perf/tests/shell/stat_all_metrics.sh +17 -10
@@ -27,19 +27,20 @@ for m in $(perf list --raw-dump metrics); do fi if [[ "$result" =~ "Cannot resolve IDs for" || "$result" =~ "No supported events found" ]] then - if [[ "$m" == @(l1_prefetch_miss_rate|stalled_cycles_per_instruction) ]] + if [[ $(perf list --raw-dump $m) == "Default"* ]] then - # Default metrics that may use unsupported events. + echo "[Ignored $m] failed but as a Default metric this can be expected" + echo $result continue fi - echo "Metric contains missing events" + echo "[Failed $m] Metric contains missing events" echo $result err=1 # Fail continue elif [[ "$result" =~ \ "Access to performance monitoring and observability operations is limited" ]] then - echo "Permission failure" + echo "[Skipped $m] Permission failure" echo $result if [[ $err -eq 0 ]] then
@@ -48,7 +49,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "in per-thread mode, enable system wide" ]] then - echo "Permissions - need system wide mode" + echo "[Skipped $m] Permissions - need system wide mode" echo $result if [[ $err -eq 0 ]] then
@@ -57,7 +58,13 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "<not supported>" ]] then - echo "Not supported events" + if [[ $(perf list --raw-dump $m) == "Default"* ]] + then + echo "[Ignored $m] failed but as a Default metric this can be expected" + echo $result + continue + fi + echo "[Skipped $m] Not supported events" echo $result if [[ $err -eq 0 ]] then
@@ -66,7 +73,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "<not counted>" ]] then - echo "Not counted events" + echo "[Skipped $m] Not counted events" echo $result if [[ $err -eq 0 ]] then
@@ -75,7 +82,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "FP_ARITH" || "$result" =~ "AMX" ]] then - echo "FP issues" + echo "[Skipped $m] FP issues" echo $result if [[ $err -eq 0 ]] then
@@ -84,7 +91,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "PMM" ]] then - echo "Optane memory issues" + echo "[Skipped $m] Optane memory issues" echo $result if [[ $err -eq 0 ]] then
@@ -101,7 +108,7 @@ for m in $(perf list --raw-dump metrics); do # No error result and metric shown. continue fi - echo "Metric '$m' has non-zero error '$result_err' or not printed in:" + echo "[Failed $m] has non-zero error '$result_err' or not printed in:" echo "$result" err=1 done
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Quoted 1 line · Ian Rogers
Reported-by Thomas Richter <[email protected]>
Thanks for fixing this.
Signature · 9 lines
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany -- IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Wolfgang Wendt Geschäftsführung: David Faller Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
Quoted 9 lines · Ian Rogers
Determine if a metric is default from
perf list --raw-dump $meg:``` $ perf list --raw-dump l1_prefetch_miss_rate Default4 l1_prefetch_miss_rate ```If a metric has "not supported" or "no supported events" then ignore these failures for default metrics. Tidy up the skip/fail messages in the output to make them easier to spot/read.
Quoted 6 lines
Reported-by Thomas Richter <[email protected]>Reported-by Namhyung Kim <[email protected]>Reported-by James Clark <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Quoted 93 lines
--- tools/perf/tests/shell/stat_all_metrics.sh | 27 ++++++++++++++-------- 1 file changed, 17 insertions(+), 10 deletions(-)tools/perf/tests/shell/stat_all_metrics.sh +17
-10@@ -27,19 +27,20 @@ for m in $(perf list --raw-dump metrics); do fi if [[ "$result" =~ "Cannot resolve IDs for" || "$result" =~ "No supported events found" ]] then - if [[ "$m" == @(l1_prefetch_miss_rate|stalled_cycles_per_instruction) ]] + if [[ $(perf list --raw-dump $m) == "Default"* ]] then - # Default metrics that may use unsupported events. + echo "[Ignored $m] failed but as a Default metric this can be expected" + echo $result continue fi - echo "Metric contains missing events" + echo "[Failed $m] Metric contains missing events" echo $result err=1 # Fail continue elif [[ "$result" =~ \ "Access to performance monitoring and observability operations is limited" ]] then - echo "Permission failure" + echo "[Skipped $m] Permission failure" echo $result if [[ $err -eq 0 ]] then@@ -48,7 +49,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "in per-thread mode, enable system wide" ]] then - echo "Permissions - need system wide mode" + echo "[Skipped $m] Permissions - need system wide mode" echo $result if [[ $err -eq 0 ]] then@@ -57,7 +58,13 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "<not supported>" ]] then - echo "Not supported events" + if [[ $(perf list --raw-dump $m) == "Default"* ]] + then + echo "[Ignored $m] failed but as a Default metric this can be expected" + echo $result + continue + fi + echo "[Skipped $m] Not supported events" echo $result if [[ $err -eq 0 ]] then@@ -66,7 +73,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "<not counted>" ]] then - echo "Not counted events" + echo "[Skipped $m] Not counted events" echo $result if [[ $err -eq 0 ]] then@@ -75,7 +82,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "FP_ARITH" || "$result" =~ "AMX" ]] then - echo "FP issues" + echo "[Skipped $m] FP issues" echo $result if [[ $err -eq 0 ]] then@@ -84,7 +91,7 @@ for m in $(perf list --raw-dump metrics); do continue elif [[ "$result" =~ "PMM" ]] then - echo "Optane memory issues" + echo "[Skipped $m] Optane memory issues" echo $result if [[ $err -eq 0 ]] then@@ -101,7 +108,7 @@ for m in $(perf list --raw-dump metrics); do # No error result and metric shown. continue fi - echo "Metric '$m' has non-zero error '$result_err' or not printed in:" + echo "[Failed $m] has non-zero error '$result_err' or not printed in:" echo "$result" err=1 done
Namhyung Kim
maintainerQuoted 9 lines · Ian Rogers
Determine if a metric is default from
perf list --raw-dump $meg:``` $ perf list --raw-dump l1_prefetch_miss_rate Default4 l1_prefetch_miss_rate ```If a metric has "not supported" or "no supported events" then ignore these failures for default metrics. Tidy up the skip/fail messages in the output to make them easier to spot/read.
Quoted 6 lines
Reported-by Thomas Richter <[email protected]>Reported-by Namhyung Kim <[email protected]>Reported-by James Clark <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Thanks for the fix!
$ sudo ./perf test -vv "all metrics" 103: perf all metrics test: --- start --- test child forked, pid 3329979 Testing CPUs_utilized Testing backend_cycles_idle [Ignored backend_cycles_idle] failed but as a Default metric this can be expected Performance counter stats for 'system wide': <not counted> cpu-cycles <not supported> stalled-cycles-backend 0.013059060 seconds time elapsed Testing branch_frequency Testing branch_miss_rate Testing cs_per_second Testing cycles_frequency Testing frontend_cycles_idle Testing insn_per_cycle Testing migrations_per_second Testing page_faults_per_second Testing stalled_cycles_per_instruction [Ignored stalled_cycles_per_instruction] failed but as a Default metric this can be expected Error: No supported events found. The stalled-cycles-backend event is not supported. Testing l1d_miss_rate Testing llc_miss_rate [Ignored llc_miss_rate] failed but as a Default metric this can be expected Error: No supported events found. The LLC-loads event is not supported. Testing dtlb_miss_rate Testing itlb_miss_rate Testing l1i_miss_rate Testing l1_prefetch_miss_rate [Ignored l1_prefetch_miss_rate] failed but as a Default metric this can be expected Performance counter stats for 'system wide': <not counted> L1-dcache-prefetches <not supported> L1-dcache-prefetch-misses 0.012868128 seconds time elapsed Testing branch_misprediction_ratio Testing all_remote_links_outbound Testing nps1_die_to_dram Testing all_l2_cache_accesses Testing all_l2_cache_hits Testing all_l2_cache_misses Testing ic_fetch_miss_ratio Testing l2_cache_accesses_from_l2_hwpf Testing l2_cache_misses_from_l2_hwpf Testing l3_read_miss_latency Testing l1_itlb_misses ---- end(0) ---- 103: perf all metrics test : Ok
Quoted 10 lines · Ian Rogers
Determine if a metric is default from
perf list --raw-dump $meg:``` $ perf list --raw-dump l1_prefetch_miss_rate Default4 l1_prefetch_miss_rate ```If a metric has "not supported" or "no supported events" then ignore these failures for default metrics. Tidy up the skip/fail messages in the output to make them easier to spot/read.
[...]
Applied to perf-tools-next, thanks!
Best regards,
Namhyung