perf stat: Fix false NMI watchdog warning in aggregation modes
PATCH v2 1/1 perf stat: Fix false NMI watchdog warning in aggregation modes
In aggregation modes (e.g. --per-socket, --per-die, etc.), a counter might not be scheduled or counted on specific aggregate groups if it was not assigned to the CPUs belonging to those groups. However, the printout() check triggers the "print_free_counters_hint" logic unconditionally for any supported counter with a missing count. This results in a false "Some events weren't counted. Try disabling the NMI watchdog" warning.
Furthermore, the NMI watchdog only reserves performance counters on core PMUs. Uncore PMU events (e.g. CHA, IMC) are not affected by the NMI watchdog, but their failures also falsely triggered this warning.
This warning was originally introduced in commit 02d492e5dcb7 ("perf stat: Issue a HW watchdog disable hint").
To fix this, restrict setting of print_free_counters_hint to only trigger for core PMU events by checking counter->pmu and counter->pmu->is_core.
Example before/after:
$ perf stat -M lpm_miss_lat --metric-only --per-socket -a -- sleep 1
Before:
Performance counter stats for 'system wide':
ns lpm_miss_lat_rem ns lpm_miss_lat_loc
S0 126 202.3 207.9 S1 126 231.9 259.3
1.006029831 seconds time elapsed
Some events weren't counted. Try disabling the NMI watchdog:
echo 0 > /proc/sys/kernel/nmi_watchdog perf stat ...
echo 1 > /proc/sys/kernel/nmi_watchdog
After:
Performance counter stats for 'system wide':
ns lpm_miss_lat_rem ns lpm_miss_lat_loc
S0 126 202.3 207.9 S1 126 231.9 259.3
1.006029831 seconds time elapsed
--- tools/perf/util/stat-display.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/stat-display.c +2 -2
@@ -821,9 +821,9 @@ static void printout(struct perf_stat_config *config, struct outstate *os, ok = false;if (counter->supported) { -if (!evlist__has_hybrid_pmus (counter->evlist)) { +if (!evlist__has_hybrid_pmus (counter->evlist) && + counter->pmu && counter->pmu->is_core) config->print_free_counters_hint =1 ; - } } }
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Directly accessing `counter->pmu` without lazy initialization suppresses the NMI watchdog hint…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Directly accessing counter->pmu without lazy initialization suppresses the NMI watchdog hint during perf stat report. --
commit 8ecd592e1137cf0841158570884e6514049ebf6f
Author: Chun-Tse Shao <[email protected]>
perf stat: Fix false NMI watchdog warning in aggregation modes
This commit fixes false NMI watchdog warnings in aggregation modes by restricting the hint to core PMU events. It prevents the warning from triggering on uncore PMU events by checking the event's PMU and its core flag.
Quoted 4 lines
tools/perf/util/stat-display.c +0
-0
[ ... ]
tools/perf/util/stat-display.c static void printout(struct perf_stat_config *config, struct outstate *os, · line 827
Quoted 7 lines
tools/perf/util/stat-display.c +2
-1@@ -823,9 +823,9 @@ static void printout(struct perf_stat_config *config, struct outstate *os, ok = false;if (counter->supported) { -if (!evlist__has_hybrid_pmus (counter->evlist)) { +if (!evlist__has_hybrid_pmus (counter->evlist) && + counter->pmu && counter->pmu->is_core)
[Severity: Medium]
During a perf stat report on non-hybrid machines, will counter->pmu be null here because it is lazily initialized when constructed from perf.data?
Since evlist__has_hybrid_pmus() immediately returns false on non-hybrid machines without iterating over the events, counter->pmu would remain uninitialized.
Does this code safely handle lazy initialization, or should it use evsel__find_pmu(counter) to guarantee the PMU is resolved before checking is_core?
Quoted 4 lines
tools/perf/util/stat-display.c +0
-1config->print_free_counters_hint =1 ; - } } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 43 lines · Chun-Tse Shao
In aggregation modes (e.g. --per-socket, --per-die, etc.), a counter might not be scheduled or counted on specific aggregate groups if it was not assigned to the CPUs belonging to those groups. However, the
printout()check triggers the "print_free_counters_hint" logic unconditionally for any supported counter with a missing count. This results in a false "Some events weren't counted. Try disabling the NMI watchdog" warning.Furthermore, the NMI watchdog only reserves performance counters on core PMUs. Uncore PMU events (e.g. CHA, IMC) are not affected by the NMI watchdog, but their failures also falsely triggered this warning.
This warning was originally introduced in commit
02d492e5dcb7("perf stat: Issue a HW watchdog disable hint").To fix this, restrict setting of
print_free_counters_hintto only trigger for core PMU events by checkingcounter->pmuandcounter->pmu->is_core.Example before/after:
$ perf stat -M lpm_miss_lat --metric-only --per-socket -a -- sleep 1Before:
Performance counter stats for 'system wide':ns
lpm_miss_lat_remnslpm_miss_lat_locS0 126 202.3 207.9 S1 126 231.9 259.31.006029831 seconds time elapsedSome events weren't counted. Try disabling the NMI watchdog:
echo 0 >/proc/sys/kernel/nmi_watchdogperf stat ...
echo 1 >/proc/sys/kernel/nmi_watchdogAfter:
Performance counter stats for 'system wide':
Quoted 8 lines
ns
lpm_miss_lat_remnslpm_miss_lat_locS0 126 202.3 207.9 S1 126 231.9 259.31.006029831 seconds time elapsedAssisted-by Gemini:gemini-nextSigned-off-by Chun-Tse Shao <[email protected]>
The suggestion for something like:
``` pmu = evsel__find_pmu(counter); if (pmu && pmu->is_core) ```
isn't really necessary because we nearly always set the PMU. Also, a case where we lack a pmu has historically been for a core PMU, making the whole thing contradictory. The patch as-is is clear.
Thanks,
Ian
Quoted 23 lines
--- tools/perf/util/stat-display.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/perf/util/stat-display.c +2
-3@@ -821,9 +821,9 @@ static void printout(struct perf_stat_config *config, struct outstate *os, ok = false;if (counter->supported) { -if (!evlist__has_hybrid_pmus (counter->evlist)) { +if (!evlist__has_hybrid_pmus (counter->evlist) && + counter->pmu && counter->pmu->is_core) config->print_free_counters_hint =1 ; - } } } --2.54.0.1136.gdb2ca164c4-goog
Quoted 10 lines · Ian Rogers
Quoted 6 lines · Chun-Tse Shao
To fix this, restrict setting of
print_free_counters_hintto only trigger for core PMU events by checkingcounter->pmuandcounter->pmu->is_core.Assisted-by Gemini:gemini-nextSigned-off-by Chun-Tse Shao <[email protected]>Reviewed-by Ian Rogers <[email protected]>
Quoted 8 lines
The suggestion for something like:
``` pmu = evsel__find_pmu(counter); if (pmu && pmu->is_core) ```isn't really necessary because we nearly always set the PMU. Also, a case where we lack a pmu has historically been for a core PMU, making the whole thing contradictory. The patch as-is is clear.
Thanks, applied to perf-tools-next, for v7.2.
- Arnaldo