perf/x86/intel: Fix spurious NMI on fixed counter
Subsystems (3)
PATCH 1/1 perf/x86/intel: Fix spurious NMI on fixed counter
From: Kan Liang <[email protected]>
If a user first sample a PEBS event on a fixed counter, then sample a non-PEBS event on the same fixed counter on Icelake, it will trigger spurious NMI. For example,
perf record -e 'cycles:p' -a
perf record -e 'cycles' -a
The error message for spurious NMI.
[June 21 15:38] Uhhuh. NMI received for unknown reason 30 on CPU 2. [ +0.000000] Do you have a strange power saving mode enabled? [ +0.000000] Dazed and confused, but trying to continue
The issue was introduced by the following commit:
commit 6f55967ad9d9 ("perf/x86/intel: Fix race in intel_pmu_disable_event()")
The commit moves the intel_pmu_pebs_disable() after intel_pmu_disable_fixed(), which returns immediately.
The related bit of PEBS_ENABLE MSR will never be cleared for the fixed counter. Then a non-PEBS event runs on the fixed counter, but the bit on PEBS_ENABLE is still set, which trigger spurious NMI.
Check and disable PEBS for fixed counter after intel_pmu_disable_fixed().
--- arch/x86/events/intel/core.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
arch/x86/events/intel/core.c +3 -5
@@ -2160,12 +2160,10 @@ static void intel_pmu_disable_event(struct perf_event *event) cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx); cpuc->intel_cp_status &= ~(1ull << hwc->idx); -if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL )) { +if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL ))intel_pmu_disable_fixed (hwc); -return ; - } - -x86_pmu_disable_event (event); +else +x86_pmu_disable_event (event);/* * Needs to be called after x86_pmu_disable_event,
Signature · 1 line
2.14.5
Quoted 42 lines · On Tue, Jun 25, 2019 at 07:21:35AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
If a user first sample a PEBS event on a fixed counter, then sample a non-PEBS event on the same fixed counter on Icelake, it will trigger spurious NMI. For example,
perf record -e 'cycles:p' -a
perf record -e 'cycles' -aThe error message for spurious NMI.
[June 21 15:38] Uhhuh. NMI received for unknown reason 30 on CPU 2. [ +0.000000] Do you have a strange power saving mode enabled? [ +0.000000] Dazed and confused, but trying to continue
The issue was introduced by the following commit:
commit
6f55967ad9d9("perf/x86/intel: Fix race inintel_pmu_disable_event()")The commit moves the
intel_pmu_pebs_disable()afterintel_pmu_disable_fixed(), which returns immediately.
The related bit ofPEBS_ENABLEMSR will never be cleared for the fixed counter. Then a non-PEBS event runs on the fixed counter, but the bit onPEBS_ENABLEis still set, which trigger spurious NMI.Check and disable PEBS for fixed counter after
intel_pmu_disable_fixed().Reported-by Yi, Ammy <[email protected]>Signed-off-by Kan Liang <[email protected]>Fixes 6f55967ad9d9 ("perf/x86/intel: Fix race in intel_pmu_disable_event()")--- arch/x86/events/intel/core.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)arch/x86/events/intel/core.c +0
-0@@ -2160,12 +2160,10 @@ static void intel_pmu_disable_event(struct perf_event *event) cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx); cpuc->intel_cp_status &= ~(1ull << hwc->idx);
arch/x86/events/intel/core.c static void intel_pmu_disable_event(struct perf_event *event) · line 2167
Quoted 10 lines
arch/x86/events/intel/core.c +3
-5-if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL )) { +if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL ))intel_pmu_disable_fixed (hwc); -return ; - } - -x86_pmu_disable_event (event); +else +x86_pmu_disable_event (event);
oops, I overlooed this, looks good
thanks,
jirka
Quoted 5 lines
arch/x86/events/intel/core.c +0
-0/* * Needs to be called after x86_pmu_disable_event, Signature · 1 line
2.14.5
Quoted 56 lines · Jiri Olsa
Quoted 42 lines · On Tue, Jun 25, 2019 at 07:21:35AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
If a user first sample a PEBS event on a fixed counter, then sample a non-PEBS event on the same fixed counter on Icelake, it will trigger spurious NMI. For example,
perf record -e 'cycles:p' -a
perf record -e 'cycles' -aThe error message for spurious NMI.
[June 21 15:38] Uhhuh. NMI received for unknown reason 30 on CPU 2. [ +0.000000] Do you have a strange power saving mode enabled? [ +0.000000] Dazed and confused, but trying to continue
The issue was introduced by the following commit:
commit 6f55967ad9d9 ("perf/x86/intel: Fix race in intel_pmu_disable_event()")The commit moves the
intel_pmu_pebs_disable()afterintel_pmu_disable_fixed(), which returns immediately.
The related bit ofPEBS_ENABLEMSR will never be cleared for the fixed counter. Then a non-PEBS event runs on the fixed counter, but the bit onPEBS_ENABLEis still set, which trigger spurious NMI.Check and disable PEBS for fixed counter after
intel_pmu_disable_fixed().Reported-by Yi, Ammy <[email protected]>Signed-off-by Kan Liang <[email protected]>Fixes 6f55967ad9d9 ("perf/x86/intel: Fix race in intel_pmu_disable_event()")--- arch/x86/events/intel/core.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)arch/x86/events/intel/core.c +0
-0@@ -2160,12 +2160,10 @@ static void intel_pmu_disable_event(struct perf_event *event) cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx); cpuc->intel_cp_status &= ~(1ull << hwc->idx);arch/x86/events/intel/core.c static void intel_pmu_disable_event(struct perf_event *event) · line 2167
Quoted 10 lines
arch/x86/events/intel/core.c +3
-5-if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL )) { +if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL ))intel_pmu_disable_fixed (hwc); -return ; - } - -x86_pmu_disable_event (event); +else +x86_pmu_disable_event (event);oops, I overlooed this, looks good
Quoted 5 lines
Acked-by Jiri Olsa <[email protected]>thanks,
jirka
Hi,
Could this fix be accepted?
Thanks
Jin Yao
Quoted 5 lines
Quoted 5 lines
arch/x86/events/intel/core.c +0
-0/* * Needs to be called after x86_pmu_disable_event, Signature · 1 line
2.14.5
Peter Zijlstra
maintainerQuoted 31 lines · Yao
Quoted 28 lines · Jiri Olsa
Quoted 27 lines · On Tue, Jun 25, 2019 at 07:21:35AM -0700, [email protected] wrote:
From: Kan Liang <[email protected]>
If a user first sample a PEBS event on a fixed counter, then sample a non-PEBS event on the same fixed counter on Icelake, it will trigger spurious NMI. For example,
perf record -e 'cycles:p' -a
perf record -e 'cycles' -aThe error message for spurious NMI.
[June 21 15:38] Uhhuh. NMI received for unknown reason 30 on CPU 2. [ +0.000000] Do you have a strange power saving mode enabled? [ +0.000000] Dazed and confused, but trying to continue
The issue was introduced by the following commit:
commit 6f55967ad9d9 ("perf/x86/intel: Fix race in intel_pmu_disable_event()")The commit moves the
intel_pmu_pebs_disable()afterintel_pmu_disable_fixed(), which returns immediately.
The related bit ofPEBS_ENABLEMSR will never be cleared for the fixed counter. Then a non-PEBS event runs on the fixed counter, but the bit onPEBS_ENABLEis still set, which trigger spurious NMI.Check and disable PEBS for fixed counter after
intel_pmu_disable_fixed().
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Reported-by Yi, Ammy <[email protected]>Signed-off-by Kan Liang <[email protected]>Fixes 6f55967ad9d9 ("perf/x86/intel: Fix race in intel_pmu_disable_event()")oops, I overlooed this, looks good
Acked-by Jiri Olsa <[email protected]>
Have it now, thanks!
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
oops, I overlooed this, looks good
Acked-by Jiri Olsa <[email protected]>Have it now, thanks!
Can Kan's patch please be merged asap and also put into stable for 5.2?
The regression causes crashes on Icelake when fixed counters are used in PEBS groups, and presumably also on Goldmont Plus.
I just had to debug this again until I realized it was the same problem.
-Andi
tip-bot for Kan Liang
merge botperf/x86/intel: Fix spurious NMI on fixed counter
Commit-ID: e4557c1a46b0d32746bd309e1941914b5a6912b4 Gitweb: https://git.kernel.org/tip/e4557c1a46b0d32746bd309e1941914b5a6912b4 Author: Kan Liang <[email protected]> AuthorDate: Tue, 25 Jun 2019 07:21:35 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Sat, 13 Jul 2019 11:21:29 +0200
perf/x86/intel: Fix spurious NMI on fixed counter
If a user first sample a PEBS event on a fixed counter, then sample a non-PEBS event on the same fixed counter on Icelake, it will trigger spurious NMI. For example:
perf record -e 'cycles:p' -a
perf record -e 'cycles' -a
The error message for spurious NMI:
[June 21 15:38] Uhhuh. NMI received for unknown reason 30 on CPU 2.
[ +0.000000] Do you have a strange power saving mode enabled?
[ +0.000000] Dazed and confused, but trying to continue
The bug was introduced by the following commit:
commit 6f55967ad9d9 ("perf/x86/intel: Fix race in intel_pmu_disable_event()")
The commit moves the intel_pmu_pebs_disable() after intel_pmu_disable_fixed(), which returns immediately. The related bit of PEBS_ENABLE MSR will never be cleared for the fixed counter. Then a non-PEBS event runs on the fixed counter, but the bit on PEBS_ENABLE is still set, which triggers spurious NMIs.
Check and disable PEBS for fixed counters after intel_pmu_disable_fixed().
--- arch/x86/events/intel/core.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
arch/x86/events/intel/core.c +3 -5
@@ -2161,12 +2161,10 @@ static void intel_pmu_disable_event(struct perf_event *event) cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx); cpuc->intel_cp_status &= ~(1ull << hwc->idx); -if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL )) { +if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL ))intel_pmu_disable_fixed (hwc); -return ; - } - -x86_pmu_disable_event (event); +else +x86_pmu_disable_event (event);/* * Needs to be called after x86_pmu_disable_event,