Skip to contentSkip to conversation

perf/x86/intel: Save/restore cpuc->active_pebs_data_cfg when using guest PEBS

PATCH Applied Like Xu 4 participants 6 messages May 17, 2023 – Jun 12, 2023

Subsystems (3)
lore
Share & tools
lore

Like Xu

author
#0 lore raw

PATCH 1/1 perf/x86/intel: Save/restore cpuc->active_pebs_data_cfg when using guest PEBS

From: Like Xu <[email protected]>

After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest. Fix it by using the active host value from cpuc->active_pebs_data_cfg.

Cc Kan Liang <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/x86/events/intel/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
index 070cc4ef2672..89b9c1cebb61 100644
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	if (x86_pmu.intel_cap.pebs_baseline) {
 		arr[(*nr)++] = (struct perf_guest_switch_msr){
 			.msr = MSR_PEBS_DATA_CFG,
-			.host = cpuc->pebs_data_cfg,
+			.host = cpuc->active_pebs_data_cfg,
 			.guest = kvm_pmu->pebs_data_cfg,
 		};
 	}
Signature · 1 line
2.40.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 7 lines · Like Xu

From: Like Xu <[email protected]>

After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest.

I believe we clear the SW bit when it takes effect.

+ if (cpuc->pebs_data_cfg & PEBS_UPDATE_DS_SW) {
+ cpuc->pebs_data_cfg = pebs_data_cfg;
+ pebs_update_threshold(cpuc);
+ }

I think the SW bit can only be seen in a shot period between add() and enable(). Is it caused by a VM enter which just happens on the period?

Quoted 1 line

Fix it by using the active host value from cpuc->active_pebs_data_cfg.

I don't see a problem of using active_pebs_data_cfg, since it reflects the current MSR setting. Just curious about how it's triggered.

Quoted 5 lines
Cc Kan Liang <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

Thanks,
Kan

Quoted 16 lines
 arch/x86/events/intel/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
index 070cc4ef2672..89b9c1cebb61 100644
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	if (x86_pmu.intel_cap.pebs_baseline) {
 		arr[(*nr)++] = (struct perf_guest_switch_msr){
 			.msr = MSR_PEBS_DATA_CFG,
-			.host = cpuc->pebs_data_cfg,
+			.host = cpuc->active_pebs_data_cfg,
 			.guest = kvm_pmu->pebs_data_cfg,
 		};
 	}
author
#2 lore raw
Quoted 18 lines · Kan
Quoted 7 lines · Like Xu

From: Like Xu <[email protected]>

After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest.

I believe we clear the SW bit when it takes effect.

(quoted diff) +4 -0
+	if (cpuc->pebs_data_cfg & PEBS_UPDATE_DS_SW) {
+		cpuc->pebs_data_cfg = pebs_data_cfg;
+		pebs_update_threshold(cpuc);
+	}
Quoted 2 lines

I think the SW bit can only be seen in a shot period between add() and enable(). Is it caused by a VM enter which just happens on the period?

What happens here is that when *intel_pmu_pebs_del()* is called, the pebs_update_state() also triggers:

	cpuc->pebs_data_cfg |= PEBS_UPDATE_DS_SW;

and the new value will then be used for the next kvm_entry.

The KVM created pebs perf_event is not added/enabled at this point and the cpuc->pebs_data_cfg strangely holds a non-zero value.

Perhaps there is more room for perf fixes here, but for guest pebs usages, using active_pebs_data_cfg in intel_guest_get_msrs() is part of what is needed.

Quoted 33 lines
Quoted 2 lines

Fix it by using the active host value from cpuc->active_pebs_data_cfg.

I don't see a problem of using active_pebs_data_cfg, since it reflects the current MSR setting. Just curious about how it's triggered.

Quoted 5 lines
Cc Kan Liang <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

Thanks,
Kan

Quoted 16 lines
  arch/x86/events/intel/core.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
index 070cc4ef2672..89b9c1cebb61 100644
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
  	if (x86_pmu.intel_cap.pebs_baseline) {
  		arr[(*nr)++] = (struct perf_guest_switch_msr){
  			.msr = MSR_PEBS_DATA_CFG,
-			.host = cpuc->pebs_data_cfg,
+			.host = cpuc->active_pebs_data_cfg,
  			.guest = kvm_pmu->pebs_data_cfg,
  		};
  	}
#3 lore raw
Quoted 21 lines · Like Xu
Quoted 20 lines · Kan
Quoted 9 lines · Like Xu

From: Like Xu <[email protected]>

After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing
PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest.

I believe we clear the SW bit when it takes effect.

+    if (cpuc->pebs_data_cfg & PEBS_UPDATE_DS_SW) { +        cpuc->pebs_data_cfg = pebs_data_cfg; +        pebs_update_threshold(cpuc);
+    }

Quoted 4 lines
Quoted 2 lines

I think the SW bit can only be seen in a shot period between add() and enable(). Is it caused by a VM enter which just happens on the period?

What happens here is that when *intel_pmu_pebs_del()* is called,

Ah, yes. I think it's useless to set the bit for the removal. We may need a cleanup patch to improve it.

Thanks,
Kan

Quoted 45 lines

the pebs_update_state() also triggers:
    cpuc->pebs_data_cfg |= PEBS_UPDATE_DS_SW;
and the new value will then be used for the next kvm_entry.

The KVM created pebs perf_event is not added/enabled at this point and the cpuc->pebs_data_cfg strangely holds a non-zero value.

Perhaps there is more room for perf fixes here, but for guest pebs usages, using active_pebs_data_cfg in intel_guest_get_msrs() is part of what is needed.

Quoted 34 lines
Quoted 2 lines

Fix it by using the active host value from cpuc->active_pebs_data_cfg.

I don't see a problem of using active_pebs_data_cfg, since it reflects the current MSR setting. Just curious about how it's triggered.

Quoted 5 lines
Cc Kan Liang <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

Thanks,
Kan

Quoted 17 lines

  arch/x86/events/intel/core.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
index 070cc4ef2672..89b9c1cebb61 100644
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr, void *data)
       if (x86_pmu.intel_cap.pebs_baseline) {
           arr[(*nr)++] = (struct perf_guest_switch_msr){
               .msr = MSR_PEBS_DATA_CFG,
-            .host = cpuc->pebs_data_cfg,
+            .host = cpuc->active_pebs_data_cfg,
               .guest = kvm_pmu->pebs_data_cfg,
           };

      }

merge bot Applied
#4 lore raw
In reply to Like Xu
The following commit has been merged into the perf/urgent branch of tip:

The following commit has been merged into the perf/urgent branch of tip:

Commit-ID:     3c845304d2d723f20d5b91fef5d133ff94825d76
Gitweb:        https://git.kernel.org/tip/3c845304d2d723f20d5b91fef5d133ff94825d76
Author:        Like Xu <[email protected]>
AuthorDate:    Wed, 17 May 2023 21:38:08 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 23 May 2023 10:01:13 +02:00

perf/x86/intel: Save/restore cpuc->active_pebs_data_cfg when using guest PEBS

After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest. Fix it by using the active host value from cpuc->active_pebs_data_cfg.

Fixes b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG")
---
 arch/x86/events/intel/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
index 070cc4e..89b9c1c 100644
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	if (x86_pmu.intel_cap.pebs_baseline) {
 		arr[(*nr)++] = (struct perf_guest_switch_msr){
 			.msr = MSR_PEBS_DATA_CFG,
-			.host = cpuc->pebs_data_cfg,
+			.host = cpuc->active_pebs_data_cfg,
 			.guest = kvm_pmu->pebs_data_cfg,
 		};
 	}
#5 lore raw
In reply to Like Xu

+KVM

Quoted 8 lines · Like Xu

From: Like Xu <[email protected]>

After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest. Fix it by using the active host value from cpuc->active_pebs_data_cfg.

In the future, please Cc: [email protected] when posting fixes that obviously affect KVM. I wasted several hours bisecting these crashes. In hindsight, I should have searched all of lore sooner, but it really shouldn't have been that hard for me to find this fix.

Quoted 23 lines
Cc Kan Liang <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/x86/events/intel/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
index 070cc4ef2672..89b9c1cebb61 100644
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	if (x86_pmu.intel_cap.pebs_baseline) {
 		arr[(*nr)++] = (struct perf_guest_switch_msr){
 			.msr = MSR_PEBS_DATA_CFG,
-			.host = cpuc->pebs_data_cfg,
+			.host = cpuc->active_pebs_data_cfg,
 			.guest = kvm_pmu->pebs_data_cfg,
 		};
 	}
Signature · 1 line
2.40.1