KVM: emulate: Don't inject #GP when emulating RDMPC if CR0.PE=0
PATCH v2 1/3 KVM: emulate: Don't inject #GP when emulating RDMPC if CR0.PE=0
From: Wanpeng Li <[email protected]>
DM mentioned that, RDPMC:
IF (((CR4.PCE = 1) or (CPL = 0) or (CR0.PE = 0)) and (ECX indicates a supported counter))
THEN
EAX := counter[31:0];
EDX := ZeroExtend(counter[MSCB:32]);
ELSE (* ECX is not valid or CR4.PCE is 0 and CPL is 1, 2, or 3 and CR0.PE is 1 *)
#GP(0);
FI;Let's add the CR0.PE is 1 checking to rdpmc emulate, though this isn't strictly necessary since it's impossible for CPL to be >0 if CR0.PE=0.
---
v1 -> v2:
* update patch description
arch/x86/kvm/emulate.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/emulate.c +2 -1
@@ -4213,6 +4213,7 @@ static int check_rdtsc(struct x86_emulate_ctxt *ctxt)static int check_rdpmc (struct x86_emulate_ctxt *ctxt) {u64 cr4 = ctxt->ops->get_cr (ctxt,4 ); +u64 cr0 = ctxt->ops->get_cr (ctxt,0 );u64 rcx =reg_read (ctxt,VCPU_REGS_RCX );/*
@@ -4222,7 +4223,7 @@ static int check_rdpmc(struct x86_emulate_ctxt *ctxt)if (enable_vmware_backdoor &&is_vmware_backdoor_pmc (rcx))return X86EMUL_CONTINUE ; -if ((!(cr4 &X86_CR4_PCE ) && ctxt->ops->cpl (ctxt)) || +if ((!(cr4 &X86_CR4_PCE ) && ctxt->ops->cpl (ctxt) && (cr0 &X86_CR0_PE )) || ctxt->ops->check_pmc (ctxt, rcx))return emulate_gp (ctxt,0 );
Signature · 1 line
2.25.1
PATCH v2 2/3 KVM: vPMU: Fill get_msr MSR_CORE_PERF_GLOBAL_OVF_CTRL w/ 0
From: Wanpeng Li <[email protected]>
SDM section 18.2.3 mentioned that:
"IA32_PERF_GLOBAL_OVF_CTL MSR allows software to clear overflow indicator(s) of
any general-purpose or fixed-function counters via a single WRMSR."
It is R/W mentioned by SDM, we read this msr on bare-metal during perf testing, the value is always 0 for ICX/SKX boxes on hands. Let's fill get_msr MSR_CORE_PERF_GLOBAL_OVF_CTRL w/ 0 as hardware behavior and drop global_ovf_ctrl variable.
---
v1 -> v2:
* drop 'u64 global_ovf_ctrl' directly
arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/vmx/pmu_intel.c | 6 ++---- 2 files changed, 2 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -1
@@ -499,7 +499,6 @@ struct kvm_pmu {u64 fixed_ctr_ctrl;u64 global_ctrl;u64 global_status; -u64 global_ovf_ctrl;u64 counter_bitmask[2 ];u64 global_ctrl_mask;u64 global_ovf_ctrl_mask;
arch/x86/kvm/vmx/pmu_intel.c +2 -4
@@ -365,7 +365,7 @@ static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data = pmu->global_ctrl;return 0 ;case MSR_CORE_PERF_GLOBAL_OVF_CTRL : - msr_info->data = pmu->global_ovf_ctrl; + msr_info->data =0 ;return 0 ;default : if ((pmc = get_gp_pmc(pmu, msr, MSR_IA32_PERFCTR0)) ||
@@ -423,7 +423,6 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)if (!(data & pmu->global_ovf_ctrl_mask)) {if (!msr_info->host_initiated) pmu->global_status &= ~data; - pmu->global_ovf_ctrl = data;return 0 ; }break ;
@@ -588,8 +587,7 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu) pmc->counter =0 ; } - pmu->fixed_ctr_ctrl = pmu->global_ctrl = pmu->global_status = - pmu->global_ovf_ctrl =0 ; + pmu->fixed_ctr_ctrl = pmu->global_ctrl = pmu->global_status =0 ;intel_pmu_release_guest_lbr_event (vcpu); }
Signature · 1 line
2.25.1
PATCH v2 3/3 KVM: vCPU kick tax cut for running vCPU
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
Sometimes a vCPU kick is following a pending request, even if @vcpu is the running vCPU. It suffers from both rcuwait_wake_up() which has rcu/memory barrier operations and cmpxchg(). Let's check vcpu->wait before rcu_wait_wake_up() and whether @vcpu is the running vCPU before cmpxchg() to tax cut this overhead.
We evaluate the kvm-unit-test/vmexit.flat on an Intel ICX box, most of the scores can improve ~600 cpu cycles especially when APICv is disabled.
tscdeadline_immed
tscdeadlineself_ipi_sti_nop
..............x2apic_self_ipi_tpr_sti_hlt
---
v1 -> v2:
* move checking running vCPU logic to kvm_vcpu_kick
* check rcuwait_active(&vcpu->wait) etc
virt/kvm/kvm_main.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
virt/kvm/kvm_main.c +10 -3
@@ -3314,8 +3314,15 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu) {int me, cpu; -if (kvm_vcpu_wake_up (vcpu)) -return ; + me =get_cpu (); + +if (rcuwait_active (&vcpu->wait) &&kvm_vcpu_wake_up (vcpu)) + goto out; + +if (vcpu ==__this_cpu_read (kvm_running_vcpu)) { +WARN_ON_ONCE (vcpu->mode ==IN_GUEST_MODE ); + goto out; + }/* * Note, the vCPU could get migrated to a different pCPU at any point
@@ -3324,12 +3331,12 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)* IPI is to force the vCPU to leave IN_GUEST_MODE, and migrating the * vCPU also requires it to leave IN_GUEST_MODE. */ - me =get_cpu ();if (kvm_arch_vcpu_should_kick (vcpu)) { cpu =READ_ONCE (vcpu->cpu);if (cpu != me && (unsigned )cpu < nr_cpu_ids &&cpu_online (cpu))smp_send_reschedule (cpu); } +out:put_cpu (); }EXPORT_SYMBOL_GPL (kvm_vcpu_kick);
Signature · 1 line
2.25.1
Sean Christopherson
maintainerRe: [PATCH v2 3/3] KVM: vCPU kick tax cut for running vCPU
Quoted 32 lines · Wanpeng Li
From: Wanpeng Li <[email protected]>
Sometimes a vCPU kick is following a pending request, even if @vcpu is the running vCPU. It suffers from both
rcuwait_wake_up()which has rcu/memory barrier operations andcmpxchg(). Let's checkvcpu->waitbeforercu_wait_wake_up()and whether @vcpu is the running vCPU beforecmpxchg()to tax cut this overhead.We evaluate the kvm-unit-test/vmexit.flat on an Intel ICX box, most of the scores can improve ~600 cpu cycles especially when APICv is disabled.
tscdeadline_immed
tscdeadlineself_ipi_sti_nop
..............x2apic_self_ipi_tpr_sti_hltSuggested-by Sean Christopherson <[email protected]>Signed-off-by Wanpeng Li <[email protected]>---
v1 -> v2:
* move checking running vCPU logic tokvm_vcpu_kick
* checkrcuwait_active(&vcpu->wait)etcvirt/kvm/kvm_main.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)virt/kvm/kvm_main.c +0
-0
virt/kvm/kvm_main.c void kvm_vcpu_kick(struct kvm_vcpu *vcpu) · line 3319
Quoted 8 lines
virt/kvm/kvm_main.c +3
-2@@ -3314,8 +3314,15 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu) {int me, cpu; -if (kvm_vcpu_wake_up (vcpu)) -return ; + me =get_cpu (); + +if (rcuwait_active (&vcpu->wait) &&kvm_vcpu_wake_up (vcpu))
This needs to use kvm_arch_vcpu_get_wait(), not vcpu->wait, because PPC has some funky wait stuff.
One potential issue I didn't think of before. rcuwait_active() comes with the below warning, which means we might be at risk of a false negative that could result in a missed wakeup. I'm not postive on that though.
/*
* Note: this provides no serialization and, just as with waitqueues,
* requires care to estimate as to whether or not the wait is active. */
Quoted 26 lines
virt/kvm/kvm_main.c +7
-1+ goto out; + +if (vcpu ==__this_cpu_read (kvm_running_vcpu)) { +WARN_ON_ONCE (vcpu->mode ==IN_GUEST_MODE ); + goto out; + }/* * Note, the vCPU could get migrated to a different pCPU at any point @@ -3324,12 +3331,12 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)* IPI is to force the vCPU to leave IN_GUEST_MODE, and migrating the * vCPU also requires it to leave IN_GUEST_MODE. */ - me =get_cpu ();if (kvm_arch_vcpu_should_kick (vcpu)) { cpu =READ_ONCE (vcpu->cpu);if (cpu != me && (unsigned )cpu < nr_cpu_ids &&cpu_online (cpu))smp_send_reschedule (cpu); } +out:put_cpu (); }EXPORT_SYMBOL_GPL (kvm_vcpu_kick);Signature · 1 line
2.25.1
Re: [PATCH v2 3/3] KVM: vCPU kick tax cut for running vCPU
Quoted 46 lines · Sean Christopherson
Quoted 32 lines · Wanpeng Li
From: Wanpeng Li <[email protected]>
Sometimes a vCPU kick is following a pending request, even if @vcpu is the running vCPU. It suffers from both
rcuwait_wake_up()which has rcu/memory barrier operations andcmpxchg(). Let's checkvcpu->waitbeforercu_wait_wake_up()and whether @vcpu is the running vCPU beforecmpxchg()to tax cut this overhead.We evaluate the kvm-unit-test/vmexit.flat on an Intel ICX box, most of the scores can improve ~600 cpu cycles especially when APICv is disabled.
tscdeadline_immed
tscdeadlineself_ipi_sti_nop
..............x2apic_self_ipi_tpr_sti_hltSuggested-by Sean Christopherson <[email protected]>Signed-off-by Wanpeng Li <[email protected]>---
v1 -> v2:
* move checking running vCPU logic tokvm_vcpu_kick
* checkrcuwait_active(&vcpu->wait)etcvirt/kvm/kvm_main.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)virt/kvm/kvm_main.c +0
-0virt/kvm/kvm_main.c void kvm_vcpu_kick(struct kvm_vcpu *vcpu) · line 3319
Quoted 8 lines
virt/kvm/kvm_main.c +3
-2@@ -3314,8 +3314,15 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu) {int me, cpu; -if (kvm_vcpu_wake_up (vcpu)) -return ; + me =get_cpu (); + +if (rcuwait_active (&vcpu->wait) &&kvm_vcpu_wake_up (vcpu))This needs to use
kvm_arch_vcpu_get_wait(), notvcpu->wait, because PPC has some funky wait stuff.
Quoted 3 lines
One potential issue I didn't think of before.
rcuwait_active()comes with the below warning, which means we might be at risk of a false negative that could result in a missed wakeup. I'm not postive on that though.
There is only ever a single waiting vCPU, an event will be requested before kick the sleeping vCPU and it will be checked after setting vcpu->wait to task. I can't find scenario could result in a missed wakeup.
Wanpeng
Quoted 5 lines
/*
* Note: this provides no serialization and, just as with waitqueues,
* requires care to estimate as to whether or not the wait is active. */
Quoted 3 lines · Wanpeng Li
From: Wanpeng Li <[email protected]>
DM mentioned that, RDPMC:
Heh, missing 'S' in "SDM".
Quoted 9 lines
IF (((CR4.PCE = 1) or (CPL = 0) or (CR0.PE = 0)) and (ECX indicates a supported counter))
THEN
EAX := counter[31:0];
EDX := ZeroExtend(counter[MSCB:32]);
ELSE (* ECX is not valid or CR4.PCE is 0 and CPL is 1, 2, or 3 and CR0.PE is 1 *)
#GP(0);
FI;Quoted 5 lines
Let's add the CR0.PE is 1 checking to rdpmc emulate, though this isn't strictly necessary since it's impossible for CPL to be >0 if CR0.PE=0.
Signed-off-by Wanpeng Li <[email protected]>---