perf/x86/intel: Save/restore cpuc->active_pebs_data_cfg when using guest PEBS
Subsystems (3)
PATCH 1/1 perf/x86/intel: Save/restore cpuc->active_pebs_data_cfg when using guest PEBS
From: Like Xu <[email protected]>
After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest. Fix it by using the active host value from cpuc->active_pebs_data_cfg.
--- arch/x86/events/intel/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (x86_pmu.intel_cap.pebs_baseline) { arr[(*nr)++] = (struct perf_guest_switch_msr ){ .msr =MSR_PEBS_DATA_CFG , - .host = cpuc->pebs_data_cfg, + .host = cpuc->active_pebs_data_cfg, .guest = kvm_pmu->pebs_data_cfg, }; }
Signature · 1 line
2.40.1
Quoted 7 lines · Like Xu
From: Like Xu <[email protected]>
After commit
b752ea0c28e3("perf/x86/intel/ds: Flush PEBS DS when changingPEBS_DATA_CFG"), thecpuc->pebs_data_cfgmay save some bits that are not supported by real hardware, such asPEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values forPEBS_DATA_CFGMSR, thus crashing the host when PEBS is used for guest.
I believe we clear the SW bit when it takes effect.
+ if (cpuc->pebs_data_cfg & PEBS_UPDATE_DS_SW) {
+ cpuc->pebs_data_cfg = pebs_data_cfg;
+ pebs_update_threshold(cpuc);
+ }
I think the SW bit can only be seen in a shot period between add() and enable(). Is it caused by a VM enter which just happens on the period?
Quoted 1 line
Fix it by using the active host value from
cpuc->active_pebs_data_cfg.
I don't see a problem of using active_pebs_data_cfg, since it reflects the current MSR setting. Just curious about how it's triggered.
Quoted 5 lines
Cc Kan Liang <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by Like Xu <[email protected]>---
Thanks,
Kan
Quoted 16 lines
arch/x86/events/intel/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/events/intel/core.c +1
-1@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (x86_pmu.intel_cap.pebs_baseline) { arr[(*nr)++] = (struct perf_guest_switch_msr ){ .msr =MSR_PEBS_DATA_CFG , - .host = cpuc->pebs_data_cfg, + .host = cpuc->active_pebs_data_cfg, .guest = kvm_pmu->pebs_data_cfg, }; }
Quoted 18 lines · Kan
Quoted 7 lines · Like Xu
From: Like Xu <[email protected]>
After commit
b752ea0c28e3("perf/x86/intel/ds: Flush PEBS DS when changingPEBS_DATA_CFG"), thecpuc->pebs_data_cfgmay save some bits that are not supported by real hardware, such asPEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values forPEBS_DATA_CFGMSR, thus crashing the host when PEBS is used for guest.I believe we clear the SW bit when it takes effect.
(quoted diff) +4
-0+ if (cpuc->pebs_data_cfg & PEBS_UPDATE_DS_SW) { + cpuc->pebs_data_cfg = pebs_data_cfg; + pebs_update_threshold(cpuc); + }
Quoted 2 lines
I think the SW bit can only be seen in a shot period between
add()andenable(). Is it caused by a VM enter which just happens on the period?
What happens here is that when *intel_pmu_pebs_del()* is called, the pebs_update_state() also triggers:
cpuc->pebs_data_cfg |= PEBS_UPDATE_DS_SW;
and the new value will then be used for the next kvm_entry.
The KVM created pebs perf_event is not added/enabled at this point and the cpuc->pebs_data_cfg strangely holds a non-zero value.
Perhaps there is more room for perf fixes here, but for guest pebs usages, using active_pebs_data_cfg in intel_guest_get_msrs() is part of what is needed.
Quoted 33 lines
Quoted 2 lines
Fix it by using the active host value from
cpuc->active_pebs_data_cfg.I don't see a problem of using
active_pebs_data_cfg, since it reflects the current MSR setting. Just curious about how it's triggered.Quoted 5 lines
Cc Kan Liang <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by Like Xu <[email protected]>---
Reviewed-by Kan Liang <[email protected]>Thanks,
KanQuoted 16 lines
arch/x86/events/intel/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/events/intel/core.c +1
-1@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (x86_pmu.intel_cap.pebs_baseline) { arr[(*nr)++] = (struct perf_guest_switch_msr ){ .msr =MSR_PEBS_DATA_CFG , - .host = cpuc->pebs_data_cfg, + .host = cpuc->active_pebs_data_cfg, .guest = kvm_pmu->pebs_data_cfg, }; }
Quoted 21 lines · Like Xu
Quoted 20 lines · Kan
Quoted 9 lines · Like Xu
From: Like Xu <[email protected]>
After commit
b752ea0c28e3("perf/x86/intel/ds: Flush PEBS DS when changingPEBS_DATA_CFG"), thecpuc->pebs_data_cfgmay save some bits that are not supported by real hardware, such asPEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values forPEBS_DATA_CFGMSR, thus crashing the host when PEBS is used for guest.I believe we clear the SW bit when it takes effect.
+ if (
cpuc->pebs_data_cfg&PEBS_UPDATE_DS_SW) { +cpuc->pebs_data_cfg=pebs_data_cfg; +pebs_update_threshold(cpuc);
+ }
Quoted 4 lines
Quoted 2 lines
I think the SW bit can only be seen in a shot period between
add()andenable(). Is it caused by a VM enter which just happens on the period?What happens here is that when *intel_pmu_pebs_del()* is called,
Ah, yes. I think it's useless to set the bit for the removal. We may need a cleanup patch to improve it.
Thanks,
Kan
Quoted 45 lines
the
pebs_update_state()also triggers:
cpuc->pebs_data_cfg |=PEBS_UPDATE_DS_SW;
and the new value will then be used for the nextkvm_entry.The KVM created pebs
perf_eventis not added/enabled at this point and thecpuc->pebs_data_cfgstrangely holds a non-zero value.Perhaps there is more room for perf fixes here, but for guest pebs usages, using
active_pebs_data_cfginintel_guest_get_msrs()is part of what is needed.Quoted 34 lines
Quoted 2 lines
Fix it by using the active host value from
cpuc->active_pebs_data_cfg.I don't see a problem of using
active_pebs_data_cfg, since it reflects the current MSR setting. Just curious about how it's triggered.Quoted 5 lines
Cc Kan Liang <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by Like Xu <[email protected]>---
Reviewed-by Kan Liang <[email protected]>Thanks,
KanQuoted 17 lines
arch/x86/events/intel/core.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/events/intel/core.c +1
-1@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) if (x86_pmu.intel_cap.pebs_baseline) { arr[(*nr)++] = (struct perf_guest_switch_msr){ .msr = MSR_PEBS_DATA_CFG, - .host = cpuc->pebs_data_cfg, + .host = cpuc->active_pebs_data_cfg, .guest = kvm_pmu->pebs_data_cfg, };}
The following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 3c845304d2d723f20d5b91fef5d133ff94825d76 Gitweb: https://git.kernel.org/tip/3c845304d2d723f20d5b91fef5d133ff94825d76 Author: Like Xu <[email protected]> AuthorDate: Wed, 17 May 2023 21:38:08 +08:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Tue, 23 May 2023 10:01:13 +02:00
perf/x86/intel: Save/restore cpuc->active_pebs_data_cfg when using guest PEBS
After commit b752ea0c28e3 ("perf/x86/intel/ds: Flush PEBS DS when changing PEBS_DATA_CFG"), the cpuc->pebs_data_cfg may save some bits that are not supported by real hardware, such as PEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values for PEBS_DATA_CFG MSR, thus crashing the host when PEBS is used for guest. Fix it by using the active host value from cpuc->active_pebs_data_cfg.
--- arch/x86/events/intel/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (x86_pmu.intel_cap.pebs_baseline) { arr[(*nr)++] = (struct perf_guest_switch_msr ){ .msr =MSR_PEBS_DATA_CFG , - .host = cpuc->pebs_data_cfg, + .host = cpuc->active_pebs_data_cfg, .guest = kvm_pmu->pebs_data_cfg, }; }
Sean Christopherson
reviewer+KVM
Quoted 8 lines · Like Xu
From: Like Xu <[email protected]>
After commit
b752ea0c28e3("perf/x86/intel/ds: Flush PEBS DS when changingPEBS_DATA_CFG"), thecpuc->pebs_data_cfgmay save some bits that are not supported by real hardware, such asPEBS_UPDATE_DS_SW. This would cause the VMX hardware MSR switching mechanism to save/restore invalid values forPEBS_DATA_CFGMSR, thus crashing the host when PEBS is used for guest. Fix it by using the active host value fromcpuc->active_pebs_data_cfg.
In the future, please Cc: [email protected] when posting fixes that obviously affect KVM. I wasted several hours bisecting these crashes. In hindsight, I should have searched all of lore sooner, but it really shouldn't have been that hard for me to find this fix.
Quoted 23 lines
Cc Kan Liang <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by Like Xu <[email protected]>--- arch/x86/events/intel/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/events/intel/core.c +1
-1@@ -4074,7 +4074,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (x86_pmu.intel_cap.pebs_baseline) { arr[(*nr)++] = (struct perf_guest_switch_msr ){ .msr =MSR_PEBS_DATA_CFG , - .host = cpuc->pebs_data_cfg, + .host = cpuc->active_pebs_data_cfg, .guest = kvm_pmu->pebs_data_cfg, }; }Signature · 1 line
2.40.1