Skip to contentSkip to conversation

perf/x86/intel: Disable check_msr for real hw

RFC In review Jiri Olsa 7 participants 14 messages Jun 14, 2019 – Jun 24, 2019

Subsystems (2)
lore
Share & tools
lore

Jiri Olsa

author
#0 lore raw

RFC 1/1 perf/x86/intel: Disable check_msr for real hw

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

We could also check if LBR tracing is enabled and make appropriate checks, but this change is simpler ;-)

ideas? thanks,
jirka

---
 arch/x86/events/intel/core.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +8 -0
index 71001f005bfe..1194ae7e1992 100644
@@ -20,6 +20,7 @@
 #include <asm/intel-family.h>
 #include <asm/apic.h>
 #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
 #include "../perf_event.h"
 
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
 {
 	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess up with potentionaly enabled regs.
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;
+
 	/*
 	 * Read the current value, change it and read it back to see if it
 	 * matches, this is needed to detect certain hardware emulators
Signature · 1 line
2.21.0
maintainer
#1 lore raw
Quoted 32 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

We could also check if LBR tracing is enabled and make appropriate checks, but this change is simpler ;-)

ideas? thanks,
jirka

---
 arch/x86/events/intel/core.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +1 -0
index 71001f005bfe..1194ae7e1992 100644
@@ -20,6 +20,7 @@
 #include <asm/intel-family.h>
 #include <asm/apic.h>
 #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
 #include "../perf_event.h"
 

arch/x86/events/intel/core.c static bool check_msr(unsigned long msr, u64 mask) · line 4059

Quoted 9 lines
arch/x86/events/intel/core.c +6 -0
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
 {
 	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess up with potentionaly enabled regs.
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;

Yeah, I think that works, or !boot_cpu_has(X86_FEATURE_HYPERVISOR).

Quoted 6 lines
arch/x86/events/intel/core.c +0 -0
 	/*
 	 * Read the current value, change it and read it back to see if it
 	 * matches, this is needed to detect certain hardware emulators
Signature · 1 line
2.21.0
#2 lore raw
In reply to Jiri Olsa
Quoted 7 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

Yes, the check_msr patch was to fix a bug report in guest. I didn't get similar bug report for real hardware. I think it should be OK to disable it for real hardware.

Thanks,
Kan

Quoted 39 lines

We could also check if LBR tracing is enabled and make appropriate checks, but this change is simpler ;-)

ideas? thanks,
jirka

---
  arch/x86/events/intel/core.c | 8 ++++++++
  1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +8 -0
index 71001f005bfe..1194ae7e1992 100644
@@ -20,6 +20,7 @@
  #include <asm/intel-family.h>
  #include <asm/apic.h>
  #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
  #include "../perf_event.h"
 
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
  {
  	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess up with potentionaly enabled regs.
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;
+
  	/*
  	 * Read the current value, change it and read it back to see if it
  	 * matches, this is needed to detect certain hardware emulators
author
#3 lore raw

PATCH 1/1 perf/x86/intel: Disable check_msr for real hw

Quoted 7 lines · Kan
Quoted 4 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

Quoted 8 lines
Quoted 3 lines

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

Yes, the check_msr patch was to fix a bug report in guest. I didn't get similar bug report for real hardware. I think it should be OK to disable it for real hardware.

thanks for confirmation, attaching the full patch

thanks,
jirka

---
Tom Vaden reported false failure of check_msr function, because some servers can do POST tracing and enable LBR tracing during the boot.

Kan confirmed that check_msr patch was to fix a bug report in guest, so it's ok to disable it for real HW.

---
 arch/x86/events/intel/core.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +8 -0
index 71001f005bfe..1194ae7e1992 100644
@@ -20,6 +20,7 @@
 #include <asm/intel-family.h>
 #include <asm/apic.h>
 #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
 #include "../perf_event.h"
 
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
 {
 	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess up with potentionaly enabled regs.
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;
+
 	/*
 	 * Read the current value, change it and read it back to see if it
 	 * matches, this is needed to detect certain hardware emulators
Signature · 1 line
2.21.0
#4 lore raw
Quoted 28 lines · Jiri Olsa
Quoted 7 lines · Kan
Quoted 4 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

Quoted 8 lines
Quoted 3 lines

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

Yes, the check_msr patch was to fix a bug report in guest. I didn't get similar bug report for real hardware. I think it should be OK to disable it for real hardware.

thanks for confirmation, attaching the full patch

thanks,
jirka

---
Tom Vaden reported false failure of check_msr function, because some servers can do POST tracing and enable LBR tracing during the boot.

Quoted 6 lines

Kan confirmed that check_msr patch was to fix a bug report in guest, so it's ok to disable it for real HW.

Thanks for handling the patch.

Quoted 31 lines
---
  arch/x86/events/intel/core.c | 8 ++++++++
  1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +8 -0
index 71001f005bfe..1194ae7e1992 100644
@@ -20,6 +20,7 @@
  #include <asm/intel-family.h>
  #include <asm/apic.h>
  #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
  #include "../perf_event.h"
 
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
  {
  	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess up with potentionaly enabled regs.
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;
+
  	/*
  	 * Read the current value, change it and read it back to see if it
  	 * matches, this is needed to detect certain hardware emulators
#5 lore raw

[tip:perf/core] perf/x86/intel: Disable check_msr for real HW

In reply to Jiri Olsa
perf/x86/intel: Disable check_msr for real HW
Commit-ID:  d0e1a507bdc761a14906f03399d933ea639a1756
Gitweb:     https://git.kernel.org/tip/d0e1a507bdc761a14906f03399d933ea639a1756
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 16 Jun 2019 16:13:13 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Mon, 17 Jun 2019 12:36:24 +0200

perf/x86/intel: Disable check_msr for real HW

Tom Vaden reported false failure of the check_msr() function, because some servers can do POST tracing and enable LBR tracing during bootup.

Kan confirmed that check_msr patch was to fix a bug report in guest, so it's ok to disable it for real HW.

Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Liang Kan <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Thomas Gleixner <[email protected]>
[ Readability edits. ]
---
 arch/x86/events/intel/core.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +8 -0
index 5e6ae481dee7..bda450ff51ee 100644
@@ -20,6 +20,7 @@
 #include <asm/intel-family.h>
 #include <asm/apic.h>
 #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
 #include "../perf_event.h"
 
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
 {
 	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess with potentionaly enabled registers:
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;
+
 	/*
 	 * Read the current value, change it and read it back to see if it
 	 * matches, this is needed to detect certain hardware emulators
#6 lore raw
In reply to Jiri Olsa
Quoted 8 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

Quoted 5 lines

We could also check if LBR tracing is enabled and make appropriate checks, but this change is simpler ;-)

ideas? thanks,
jirka

Sorry for the late comment. I see this patch has been merged now.

Unfortunately I don't think it's a good idea. The problem is that the hypervisor flags are only set for a few hypervisors that Linux knows about. But in practice there are many more Hypervisors around that will not cause these flags to be set. But these are still likely to miss MSRs.

The other hypervisors are relatively obscure, but eventually someone will hit problems.

-Andi

#8 lore raw
In reply to Jiri Olsa
Quoted 27 lines · Jiri Olsa
Quoted 17 lines · Andi Kleen
Quoted 8 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

Quoted 5 lines

We could also check if LBR tracing is enabled and make appropriate checks, but this change is simpler ;-)

ideas? thanks,
jirka

Sorry for the late comment. I see this patch has been merged now.

Quoted 8 lines

Unfortunately I don't think it's a good idea. The problem is that the hypervisor flags are only set for a few hypervisors that Linux knows about. But in practice there are many more Hypervisors around that will not cause these flags to be set. But these are still likely to miss MSRs.

The other hypervisors are relatively obscure, but eventually someone will hit problems.

Quoted 8 lines

any idea if there's any other flag/way we could use to detect those?

adding few virtualization folks to the loop
and attaching the original patch

thanks,
jirka

Would it be reasonable to change the sense of the original patch in commit 4550931 to only enable the check for the set of "certain hardware emulators" and leave the check otherwise disabled by default? I'm assuming that set is known (and small)?

Quoted 43 lines

---
Tom Vaden reported false failure of check_msr function, because some servers can do POST tracing and enable LBR tracing during the boot.

Kan confirmed that check_msr patch was to fix a bug report in guest, so it's ok to disable it for real HW.

---
  arch/x86/events/intel/core.c | 8 ++++++++
  1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +8 -0
index 71001f005bfe..1194ae7e1992 100644
@@ -20,6 +20,7 @@
  #include <asm/intel-family.h>
  #include <asm/apic.h>
  #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
  #include "../perf_event.h"
 
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
  {
  	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess up with potentionaly enabled regs.
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;
+
  	/*
  	 * Read the current value, change it and read it back to see if it
  	 * matches, this is needed to detect certain hardware emulators
author
#7 lore raw
In reply to Andi Kleen
Quoted 17 lines · Andi Kleen
Quoted 8 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

Quoted 5 lines

We could also check if LBR tracing is enabled and make appropriate checks, but this change is simpler ;-)

ideas? thanks,
jirka

Sorry for the late comment. I see this patch has been merged now.

Quoted 8 lines

Unfortunately I don't think it's a good idea. The problem is that the hypervisor flags are only set for a few hypervisors that Linux knows about. But in practice there are many more Hypervisors around that will not cause these flags to be set. But these are still likely to miss MSRs.

The other hypervisors are relatively obscure, but eventually someone will hit problems.

any idea if there's any other flag/way we could use to detect those?

adding few virtualization folks to the loop
and attaching the original patch

thanks,
jirka

---
Tom Vaden reported false failure of check_msr function, because some servers can do POST tracing and enable LBR tracing during the boot.

Kan confirmed that check_msr patch was to fix a bug report in guest, so it's ok to disable it for real HW.

---
 arch/x86/events/intel/core.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/events/intel/core.c +8 -0
index 71001f005bfe..1194ae7e1992 100644
@@ -20,6 +20,7 @@
 #include <asm/intel-family.h>
 #include <asm/apic.h>
 #include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
 
 #include "../perf_event.h"
 
@@ -4050,6 +4051,13 @@ static bool check_msr(unsigned long msr, u64 mask)
 {
 	u64 val_old, val_new, val_tmp;
 
+	/*
+	 * Disable the check for real HW, so we don't
+	 * mess up with potentionaly enabled regs.
+	 */
+	if (hypervisor_is_type(X86_HYPER_NATIVE))
+		return true;
+
 	/*
 	 * Read the current value, change it and read it back to see if it
 	 * matches, this is needed to detect certain hardware emulators
Signature · 1 line
2.21.0
#9 lore raw
Quoted 24 lines · Jiri Olsa
Quoted 23 lines · Andi Kleen
Quoted 8 lines · Jiri Olsa

hi,
the HPE server can do POST tracing and have enabled LBR tracing during the boot, which makes check_msr fail falsly.

It looks like check_msr code was added only to check on guests MSR access, would it be then ok to disable check_msr for real hardware? (as in patch below)

Quoted 5 lines

We could also check if LBR tracing is enabled and make appropriate checks, but this change is simpler ;-)

ideas? thanks,
jirka

Sorry for the late comment. I see this patch has been merged now.

Unfortunately I don't think it's a good idea. The problem is that the hypervisor flags are only set for a few hypervisors that Linux knows about. But in practice there are many more Hypervisors around that will not cause these flags to be set. But these are still likely to miss MSRs.

Quoted 4 lines
Quoted 2 lines

The other hypervisors are relatively obscure, but eventually someone will hit problems.

any idea if there's any other flag/way we could use to detect those?

There's no silver bullet, the best is boot_cpu_has(X86_FEATURE_HYPERVISOR).

Paolo

#10 lore raw
In reply to Jiri Olsa
Quoted 4 lines
Quoted 2 lines

The other hypervisors are relatively obscure, but eventually someone will hit problems.

any idea if there's any other flag/way we could use to detect those?

I'm not aware of a generic way to detect any hypervisor unfortunately.

There are hypervisor reserved cpuid ranges, in theory you could probe the existence of those. But there might be always some which don't have extra CPUIDs.

Quoted 4 lines

adding few virtualization folks to the loop
and attaching the original patch

Quoted 8 lines

thanks,
jirka

---
Tom Vaden reported false failure of check_msr function, because some servers can do POST tracing and enable LBR tracing during the boot.

Just to understand the original problem, the LBR registers get locked somehow? It would be reasonable to not use LBRs in this case. We just need to make sure everything else is still probed.

-Andi

author
#11 lore raw
Quoted 24 lines · Andi Kleen
Quoted 4 lines
Quoted 2 lines

The other hypervisors are relatively obscure, but eventually someone will hit problems.

any idea if there's any other flag/way we could use to detect those?

I'm not aware of a generic way to detect any hypervisor unfortunately.

There are hypervisor reserved cpuid ranges, in theory you could probe the existence of those. But there might be always some which don't have extra CPUIDs.

Quoted 4 lines

adding few virtualization folks to the loop
and attaching the original patch

Quoted 8 lines

thanks,
jirka

---
Tom Vaden reported false failure of check_msr function, because some servers can do POST tracing and enable LBR tracing during the boot.

Quoted 4 lines

Just to understand the original problem, the LBR registers get locked somehow? It would be reasonable to not use LBRs in this case. We just need to make sure everything else is still probed.

Tom, plz correctme if I'm wrongm but AFAIK because the LBR tracing is enabled during the boot the lbr_from/lbr_to registers will fail the check_msr 'val_new != val_tmp' check

if there's no good way to detect this, maybe we add boot option to disable the check_msr check

jirka

#12 lore raw
Quoted 3 lines

Tom, plz correctme if I'm wrongm but AFAIK because the LBR tracing is enabled during the boot the lbr_from/lbr_to registers will fail the check_msr 'val_new != val_tmp' check

Ok this should be handleable. It should be enough to check the ctrl register, if that working likely we don't need to check the data registers which might be changing.

-Andi

author
#13 lore raw
Quoted 7 lines · Andi Kleen
Quoted 3 lines

Tom, plz correctme if I'm wrongm but AFAIK because the LBR tracing is enabled during the boot the lbr_from/lbr_to registers will fail the check_msr 'val_new != val_tmp' check

Ok this should be handleable. It should be enough to check the ctrl register, if that working likely we don't need to check the data registers which might be changing.

if the control register check is enough, we should be ok then, I'll send new version

thanks,
jirka