IPI virtualization support for VM
v3 cover IPI virtualization support for VM
Current IPI process in guest VM will virtualize the writing to interrupt command register(ICR) of the local APIC which will cause VM-exit anyway on source vCPU. Frequent VM-exit could induce much overhead accumulated if running IPI intensive task.
IPI virtualization as a new VT-x feature targets to eliminate VM-exits when issuing IPI on source vCPU. It introduces a new VM-execution control - "IPI virtualization"(bit4) in the tertiary processor-based VM-exection controls and a new data structure - "PID-pointer table address" and "Last PID-pointer index" referenced by the VMCS. When "IPI virtualization" is enabled, processor emulateds following kind of writes to APIC registers that would send IPIs, moreover without causing VM-exits.
- Memory-mapped ICR writes
- MSR-mapped ICR writes
- SENDUIPI execution
This patch series implement IPI virtualization support in KVM.
Patches 1-4 add tertiary processor-based VM-execution support framework.
Patch 5 implement interrupt dispatch support in x2APIC mode with APIC-write VM exit. In previous platform, no CPU would produce APIC-write VM exit with exit qulification 300H when the "virtual x2APIC mode" VM-execution control was 1.
Patch 6 implement IPI virtualization related function including feature enabling through tertiary processor-based VM-execution in various scenario of VMCS configuration, PID table setup in vCPU creation and vCPU block consideration.
Document for IPI virtualization is now available at the latest "Intel Architecture Instruction Set Extensions Programming Reference".
We did experiment to measure average time sending IPI from source vCPU to the target vCPU completing the IPI handling by kvm unittest w/ and w/o IPI virtualization. When IPI virtualizatin enabled, it will reduce 22.21% and 15.98% cycles consuming in xAPIC mode and x2APIC mode respectly.
KMV unittest:vmexit/ipi, 2 vCPU, AP was modified to run in idle loop instead of halt to ensure no VM exit impact on target vCPU.
Cycles of IPI
xAPIC mode x2APIC mode
test w/o IPIv w/ IPIv w/o IPIv w/ IPIv
1 6106 4816 4265 3768
2 6244 4656 4404 3546
3 6165 4658 4233 3474
4 5992 4710 4363 3430
5 6083 4741 4215 3551
6 6238 4904 4304 3547
7 6164 4617 4263 3709
8 5984 4763 4518 3779
9 5931 4712 4645 3667
10 5955 4530 4332 3724
11 5897 4673 4283 3569
12 6140 4794 4178 3598
13 6183 4728 4363 3628
14 5991 4994 4509 3842
15 5866 4665 4520 3739
16 6032 4654 4229 3701
17 6050 4653 4185 3726
18 6004 4792 4319 3746
19 5961 4626 4196 3392
20 6194 4576 4433 3760Average cycles 6059 4713.1 4337.85 3644.8 %Reduction -22.21% -15.98%
--------------------------------------
IPI microbenchmark:
(https://lore.kernel.org/kvm/[email protected])
2 vCPUs, 1:1 pin vCPU to pCPU, guest VM runs with idle=poll, x2APIC mode
Result with IPIv enabled:
Dry-run: 0, 272798 ns Self-IPI: 5094123, 11114037 ns Normal IPI: 131697087, 173321200 ns Broadcast IPI: 0, 155649075 ns Broadcast lock: 0, 161518031 ns
Result with IPIv disabled:
Dry-run: 0, 272766 ns Self-IPI: 5091788, 11123699 ns Normal IPI: 145215772, 174558920 ns Broadcast IPI: 0, 175785384 ns Broadcast lock: 0, 149076195 ns
As IPIv can benefit unicast IPI to other CPU, Noraml IPI test case gain about 9.73% time saving on average out of 15 test runs when IPIv is enabled.
w/o IPIv w/ IPIv Normal IPI: 145944306.6 ns 131742993.1 ns %Reduction -9.73%
--------------------------------------
hackbench:
8 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 100000
w/o IPIv w/ IPIv Time: 91.887 74.605 %Reduction: -18.808%
96 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 1000000
w/o IPIv w/ IPIv Time: 287.504 235.185 %Reduction: -18.198%
--------------------------------------
v2 -> v3:
1. Misc change on tertiary execution control
definition and capability setup
2. Alternative to get tertiary execution
control configuration
v1 -> v2:
1. Refine the IPIv enabling logic for VM.
Remove ipiv_active definition per vCPU.
Gao Chao (1):
KVM: VMX: enable IPI virtualization
Robert Hoo (4):
x86/feat_ctl: Add new VMX feature, Tertiary VM-Execution control
KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit
variation
KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
KVM: VMX: dump_vmcs() reports tertiary_exec_control field as well
Zeng Guang (1):
KVM: x86: Support interrupt dispatch in x2APIC mode with APIC-write VM
exit
arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/vmx.h | 11 +++ arch/x86/include/asm/vmxfeatures.h | 5 +- arch/x86/kernel/cpu/feat_ctl.c | 11 ++- arch/x86/kvm/lapic.c | 9 ++- arch/x86/kvm/vmx/capabilities.h | 14 ++++ arch/x86/kvm/vmx/evmcs.c | 2 + arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/posted_intr.c | 22 ++++-- arch/x86/kvm/vmx/vmcs.h | 1 + arch/x86/kvm/vmx/vmx.c | 114 +++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmx.h | 27 ++++--- 12 files changed, 193 insertions(+), 25 deletions(-)
Signature · 1 line
2.25.1
PATCH v3 1/6 x86/feat_ctl: Add new VMX feature, Tertiary VM-Execution control
From: Robert Hoo <[email protected]>
New VMX capability MSR IA32_VMX_PROCBASED_CTLS3 conresponse to this new VM-Execution control field. And it is 64bit allow-1 semantics, not like previous capability MSRs 32bit allow-0 and 32bit allow-1. So with Tertiary VM-Execution control field introduced, 2 vmx_feature leaves are introduced, TERTIARY_CTLS_LOW and TERTIARY_CTLS_HIGH.
--- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/vmxfeatures.h | 3 ++- arch/x86/kernel/cpu/feat_ctl.c | 11 ++++++++++- 3 files changed, 13 insertions(+), 2 deletions(-)
arch/x86/include/asm/msr-index.h +1 -0
@@ -919,6 +919,7 @@#define MSR_IA32_VMX_TRUE_EXIT_CTLS 0x0000048f#define MSR_IA32_VMX_TRUE_ENTRY_CTLS 0x00000490#define MSR_IA32_VMX_VMFUNC 0x00000491 +#define MSR_IA32_VMX_PROCBASED_CTLS3 0x00000492/* VMX_BASIC bits and bitmasks */ #define VMX_BASIC_VMCS_SIZE_SHIFT 32
arch/x86/include/asm/vmxfeatures.h +2 -1
@@ -5,7 +5,7 @@/* * Defines VMX CPU feature bits */ -#define NVMXINTS 3/* N 32-bit words worth of info */ +#define NVMXINTS 5/* N 32-bit words worth of info */ /* * Note: If the comment begins with a quoted string, that string is used
@@ -43,6 +43,7 @@#define VMX_FEATURE_RDTSC_EXITING ( 1*32+ 12)/* "" VM-Exit on RDTSC */ #define VMX_FEATURE_CR3_LOAD_EXITING ( 1*32+ 15)/* "" VM-Exit on writes to CR3 */ #define VMX_FEATURE_CR3_STORE_EXITING ( 1*32+ 16)/* "" VM-Exit on reads from CR3 */ +#define VMX_FEATURE_TERTIARY_CONTROLS (1*32 + 17)/* "" Enable Tertiary VM-Execution Controls */ #define VMX_FEATURE_CR8_LOAD_EXITING ( 1*32+ 19)/* "" VM-Exit on writes to CR8 */ #define VMX_FEATURE_CR8_STORE_EXITING ( 1*32+ 20)/* "" VM-Exit on reads from CR8 */ #define VMX_FEATURE_VIRTUAL_TPR ( 1*32+ 21)/* "vtpr" TPR virtualization, a.k.a. TPR shadow */
arch/x86/kernel/cpu/feat_ctl.c +10 -1
@@ -15,6 +15,8 @@ enum vmx_feature_leafs {MISC_FEATURES =0 ,PRIMARY_CTLS ,SECONDARY_CTLS , +TERTIARY_CTLS_LOW , +TERTIARY_CTLS_HIGH , NR_VMX_FEATURE_WORDS, };
@@ -22,7 +24,7 @@ enum vmx_feature_leafs {static void init_vmx_capabilities (struct cpuinfo_x86 *c) { -u32 supported, funcs, ept, vpid, ign; +u32 supported, funcs, ept, vpid, ign, low, high;BUILD_BUG_ON (NVMXINTS !=NR_VMX_FEATURE_WORDS );
@@ -42,6 +44,13 @@ static void init_vmx_capabilities(struct cpuinfo_x86 *c)rdmsr_safe (MSR_IA32_VMX_PROCBASED_CTLS2 , &ign, &supported); c->vmx_capability[SECONDARY_CTLS ] = supported; +/* +* For tertiary execution controls MSR, it's actually a 64bit allowed-1. +*/ +rdmsr_safe (MSR_IA32_VMX_PROCBASED_CTLS3 , &low, &high); + c->vmx_capability[TERTIARY_CTLS_LOW ] = low; + c->vmx_capability[TERTIARY_CTLS_HIGH ] = high; +rdmsr (MSR_IA32_VMX_PINBASED_CTLS , ign, supported);rdmsr_safe (MSR_IA32_VMX_VMFUNC , &ign, &funcs);
Signature · 1 line
2.25.1
PATCH v3 2/6 KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit variation
In reply to Zeng GuangFrom: Robert Hoo <[email protected]>
The Tertiary VM-Exec Control, different from previous control fields, is 64 bit. So extend BUILD_CONTROLS_SHADOW() by adding a 'bit' parameter, to support both 32 bit and 64 bit fields' auxiliary functions building. Also, define the auxiliary functions for Tertiary control field here, using the new BUILD_CONTROLS_SHADOW().
--- arch/x86/kvm/vmx/vmx.h | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx/vmx.h +12 -11
@@ -413,31 +413,32 @@ static inline u8 vmx_get_rvi(void)return vmcs_read16 (GUEST_INTR_STATUS ) &0xff ; } -#define BUILD_CONTROLS_SHADOW (lname, uname) \ -static inline void lname##_controls_set(struct vcpu_vmx *vmx, u32 val) \ +#define BUILD_CONTROLS_SHADOW (lname, uname, bits) \ +static inline void lname##_controls_set(struct vcpu_vmx *vmx, u##bits val) \ { \ if (vmx->loaded_vmcs->controls_shadow.lname != val) { \ - vmcs_write32(uname, val); \ + vmcs_write##bits(uname, val); \ vmx->loaded_vmcs->controls_shadow.lname = val; \ } \ } \ -static inline u32 lname##_controls_get(struct vcpu_vmx *vmx) \ +static inline u##bits lname##_controls_get(struct vcpu_vmx *vmx) \ { \ return vmx->loaded_vmcs->controls_shadow.lname; \ } \ -static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u32 val) \ +static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) | val); \ } \ -static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u32 val) \ +static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) & ~val); \ } -BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS) -BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS) -BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL) +BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(tertiary_exec, TERTIARY_VM_EXEC_CONTROL, 64)static inline void vmx_register_cache_reset (struct kvm_vcpu *vcpu) {
Signature · 1 line
2.25.1
Sean Christopherson
maintainerRe: [PATCH v3 2/6] KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit variation
Quoted 19 lines · Zeng Guang
From: Robert Hoo <[email protected]>
The Tertiary VM-Exec Control, different from previous control fields, is 64 bit. So extend
BUILD_CONTROLS_SHADOW()by adding a 'bit' parameter, to support both 32 bit and 64 bit fields' auxiliary functions building. Also, define the auxiliary functions for Tertiary control field here, using the newBUILD_CONTROLS_SHADOW().Suggested-by Sean Christopherson <[email protected]>Signed-off-by Robert Hoo <[email protected]>--- arch/x86/kvm/vmx/vmx.h | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)arch/x86/kvm/vmx/vmx.h +0
-0
arch/x86/kvm/vmx/vmx.h static inline u8 vmx_get_rvi(void) · line 417
Quoted 7 lines
arch/x86/kvm/vmx/vmx.h +2
-2@@ -413,31 +413,32 @@ static inline u8 vmx_get_rvi(void)return vmcs_read16 (GUEST_INTR_STATUS ) &0xff ; } -#define BUILD_CONTROLS_SHADOW (lname, uname) \ -static inline void lname##_controls_set(struct vcpu_vmx *vmx, u32 val) \ +#define BUILD_CONTROLS_SHADOW (lname, uname, bits) \ +static inline void lname##_controls_set(struct vcpu_vmx *vmx, u##bits val) \
Align the trailing backslashes (with tabs when possible). It's a lot of unfortunate churn, but it really does make the code easier to read. An alternative is to split "static inline" to a separate line.
Quoted 23 lines
arch/x86/kvm/vmx/vmx.h +4
-5{ \ if (vmx->loaded_vmcs->controls_shadow.lname != val) { \ -vmcs_write32 (uname, val); \ + vmcs_write##bits(uname, val); \ vmx->loaded_vmcs->controls_shadow.lname = val; \ } \ } \ -static inline u32 lname##_controls_get(struct vcpu_vmx *vmx) \ +static inline u##bits lname##_controls_get(struct vcpu_vmx *vmx) \ { \ return vmx->loaded_vmcs->controls_shadow.lname; \ } \ -static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u32 val) \ +static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) | val); \ } \ -static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u32 val) \ +static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) & ~val); \ } -BUILD_CONTROLS_SHADOW (vm_entry,VM_ENTRY_CONTROLS )
arch/x86/kvm/vmx/vmx.h static inline u8 vmx_get_rvi(void) · line 441
Quoted 10 lines
arch/x86/kvm/vmx/vmx.h +6
-4-BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS) -BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW (secondary_exec,SECONDARY_VM_EXEC_CONTROL ) +BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW (tertiary_exec,TERTIARY_VM_EXEC_CONTROL ,64 )
This fails to compile because all the TERTIARY collateral is in a later patch.
I think I'd also prefer hiding the 32/64 param via more macros, e.g.
#define __BUILD_CONTROLS_SHADOW(lname, uname, bits) \
static inline void lname##_controls_set(struct vcpu_vmx *vmx, u##bits val) \
{ \
if (vmx->loaded_vmcs->controls_shadow.lname != val) { \
vmcs_write##bits(uname, val); \
vmx->loaded_vmcs->controls_shadow.lname = val; \
} \
} \
static inline u##bits lname##_controls_get(struct vcpu_vmx *vmx) \
{ \
return vmx->loaded_vmcs->controls_shadow.lname; \
} \
static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u##bits val) \
{ \
lname##_controls_set(vmx, lname##_controls_get(vmx) | val); \
} \
static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u##bits val) \
{ \
lname##_controls_set(vmx, lname##_controls_get(vmx) & ~val); \
}#define BUILD_CONTROLS_SHADOW(lname, uname) __BUILD_CONTROLS_SHADOW(lname, uname, 32)
#define BUILD_CONTROLS_SHADOW64(lname, uname) __BUILD_CONTROLS_SHADOW(lname, uname, 64)
Re: [PATCH v3 2/6] KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit variation
Quoted 57 lines · Sean Christopherson
Quoted 19 lines · Zeng Guang
From: Robert Hoo <[email protected]>
The Tertiary VM-Exec Control, different from previous control fields, is 64 bit. So extend
BUILD_CONTROLS_SHADOW()by adding a 'bit' parameter, to support both 32 bit and 64 bit fields' auxiliary functions building. Also, define the auxiliary functions for Tertiary control field here, using the newBUILD_CONTROLS_SHADOW().Suggested-by Sean Christopherson <[email protected]>Signed-off-by Robert Hoo <[email protected]>--- arch/x86/kvm/vmx/vmx.h | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)arch/x86/kvm/vmx/vmx.h +0
-0arch/x86/kvm/vmx/vmx.h static inline u8 vmx_get_rvi(void) · line 417
Quoted 7 lines
arch/x86/kvm/vmx/vmx.h +2
-2@@ -413,31 +413,32 @@ static inline u8 vmx_get_rvi(void)return vmcs_read16 (GUEST_INTR_STATUS ) &0xff ; } -#define BUILD_CONTROLS_SHADOW (lname, uname) \ -static inline void lname##_controls_set(struct vcpu_vmx *vmx, u32 val) \ +#define BUILD_CONTROLS_SHADOW (lname, uname, bits) \ +static inline void lname##_controls_set(struct vcpu_vmx *vmx, u##bits val) \Align the trailing backslashes (with tabs when possible). It's a lot of unfortunate churn, but it really does make the code easier to read. An alternative is to split "static inline" to a separate line.
Quoted 26 lines
arch/x86/kvm/vmx/vmx.h +4
-8{ \ if (vmx->loaded_vmcs->controls_shadow.lname != val) { \ -vmcs_write32 (uname, val); \ + vmcs_write##bits(uname, val); \ vmx->loaded_vmcs->controls_shadow.lname = val; \ } \ } \ -static inline u32 lname##_controls_get(struct vcpu_vmx *vmx) \ +static inline u##bits lname##_controls_get(struct vcpu_vmx *vmx) \ { \ return vmx->loaded_vmcs->controls_shadow.lname; \ } \ -static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u32 val) \ +static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) | val); \ } \ -static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u32 val) \ +static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) & ~val); \ } -BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS) -BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS) -BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW (exec,CPU_BASED_VM_EXEC_CONTROL )
Quoted 8 lines
arch/x86/kvm/vmx/vmx.h static inline u8 vmx_get_rvi(void) · line 441
Quoted 7 lines
arch/x86/kvm/vmx/vmx.h +6
-1-BUILD_CONTROLS_SHADOW (secondary_exec,SECONDARY_VM_EXEC_CONTROL ) +BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW (tertiary_exec,TERTIARY_VM_EXEC_CONTROL ,64 )This fails to compile because all the TERTIARY collateral is in a later patch.
Alternative to derive relative TERTIARY collateral and prepare them in this patch. Ok for that ?
Quoted 25 lines
I think I'd also prefer hiding the 32/64 param via more macros, e.g.
#define
__BUILD_CONTROLS_SHADOW(lname, uname, bits)\
static inline void lname##_controls_set(structvcpu_vmx*vmx, u##bits val) \{ \ if (vmx->loaded_vmcs->controls_shadow.lname != val) { \ vmcs_write##bits(uname, val); \ vmx->loaded_vmcs->controls_shadow.lname = val; \ } \ } \ static inline u##bits lname##_controls_get(struct vcpu_vmx *vmx) \ { \ return vmx->loaded_vmcs->controls_shadow.lname; \ } \ static inline void lname##_controls_setbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) | val); \ } \ static inline void lname##_controls_clearbit(struct vcpu_vmx *vmx, u##bits val) \ { \ lname##_controls_set(vmx, lname##_controls_get(vmx) & ~val); \ }#define
BUILD_CONTROLS_SHADOW(lname, uname)__BUILD_CONTROLS_SHADOW(lname, uname, 32)
#defineBUILD_CONTROLS_SHADOW64(lname, uname)__BUILD_CONTROLS_SHADOW(lname, uname, 64)
Sean Christopherson
maintainerRe: [PATCH v3 2/6] KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit variation
Quoted 8 lines · Zeng Guang
Quoted 3 lines · Sean Christopherson
Quoted 1 line · Zeng Guang
+BUILD_CONTROLS_SHADOW(tertiary_exec,
TERTIARY_VM_EXEC_CONTROL, 64)This fails to compile because all the TERTIARY collateral is in a later patch.
Alternative to derive relative TERTIARY collateral and prepare them in this patch. Ok for that ?
It'd be better to move this line to the next patch, that way all the TERTIARY stuff is in a single patch and isolated from controls shadow enhancement.
Paolo Bonzini
maintainerRe: [PATCH v3 2/6] KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit variation
In reply to Sean ChristophersonQuoted 14 lines · Sean Christopherson
arch/x86/kvm/vmx/vmx.h static inline u8 vmx_get_rvi(void) · line 441
Quoted 11 lines
arch/x86/kvm/vmx/vmx.h +6
-5-BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS) -BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS) -BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW (secondary_exec,SECONDARY_VM_EXEC_CONTROL ) +BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW (tertiary_exec,TERTIARY_VM_EXEC_CONTROL ,64 )This fails to compile because all the TERTIARY collateral is in a later patch.
Quoted 3 lines
I think I'd also prefer hiding the 32/64 param via more macros, e.g.
#define
__BUILD_CONTROLS_SHADOW(lname, uname, bits)\
No, please don't. :) Also because the 64 bit version is used only once.
Agreed on keeping everything here except for TERTIARY_VM_EXEC_CONTROL, and moving that line to patch 3.
Paolo
Sean Christopherson
maintainerRe: [PATCH v3 2/6] KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit variation
Quoted 15 lines · Paolo Bonzini
Quoted 14 lines · Sean Christopherson
arch/x86/kvm/vmx/vmx.h static inline u8 vmx_get_rvi(void) · line 441
Quoted 11 lines
arch/x86/kvm/vmx/vmx.h +6
-5-BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS) -BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS) -BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL) -BUILD_CONTROLS_SHADOW (secondary_exec,SECONDARY_VM_EXEC_CONTROL ) +BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32) +BUILD_CONTROLS_SHADOW (tertiary_exec,TERTIARY_VM_EXEC_CONTROL ,64 )This fails to compile because all the TERTIARY collateral is in a later patch.
Quoted 5 lines
Quoted 3 lines
I think I'd also prefer hiding the 32/64 param via more macros, e.g.
#define
__BUILD_CONTROLS_SHADOW(lname, uname, bits)\No, please don't. :) Also because the 64 bit version is used only once.
LOL, fine. I'm beginning to think you have a filter for "macros" that sounds off alarms and flashing lights ;-)
PATCH v3 3/6 KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
In reply to Zeng GuangFrom: Robert Hoo <[email protected]>
Check VMX feature on tertiary execution control in VMCS config setup. Currently it's not supported for hyper-v and disabled for now.
--- arch/x86/include/asm/vmx.h | 3 +++ arch/x86/kvm/vmx/capabilities.h | 7 ++++++ arch/x86/kvm/vmx/evmcs.c | 2 ++ arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/vmcs.h | 1 + arch/x86/kvm/vmx/vmx.c | 40 ++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 1 + 7 files changed, 54 insertions(+), 1 deletion(-)
arch/x86/include/asm/vmx.h +3 -0
@@ -31,6 +31,7 @@#define CPU_BASED_RDTSC_EXITING VMCS_CONTROL_BIT(RDTSC_EXITING)#define CPU_BASED_CR3_LOAD_EXITING VMCS_CONTROL_BIT(CR3_LOAD_EXITING)#define CPU_BASED_CR3_STORE_EXITING VMCS_CONTROL_BIT(CR3_STORE_EXITING) +#define CPU_BASED_ACTIVATE_TERTIARY_CONTROLS VMCS_CONTROL_BIT(TERTIARY_CONTROLS)#define CPU_BASED_CR8_LOAD_EXITING VMCS_CONTROL_BIT(CR8_LOAD_EXITING)#define CPU_BASED_CR8_STORE_EXITING VMCS_CONTROL_BIT(CR8_STORE_EXITING)#define CPU_BASED_TPR_SHADOW VMCS_CONTROL_BIT(VIRTUAL_TPR)
@@ -221,6 +222,8 @@ enum vmcs_field {ENCLS_EXITING_BITMAP_HIGH =0x0000202F ,TSC_MULTIPLIER =0x00002032 ,TSC_MULTIPLIER_HIGH =0x00002033 , +TERTIARY_VM_EXEC_CONTROL =0x00002034 , +TERTIARY_VM_EXEC_CONTROL_HIGH =0x00002035 ,GUEST_PHYSICAL_ADDRESS =0x00002400 ,GUEST_PHYSICAL_ADDRESS_HIGH =0x00002401 , VMCS_LINK_POINTER = 0x00002800,
arch/x86/kvm/vmx/capabilities.h +7 -0
@@ -59,6 +59,7 @@ struct vmcs_config {u32 pin_based_exec_ctrl;u32 cpu_based_exec_ctrl;u32 cpu_based_2nd_exec_ctrl; +u64 cpu_based_3rd_exec_ctrl;u32 vmexit_ctrl;u32 vmentry_ctrl;struct nested_vmx_msrs nested;
@@ -131,6 +132,12 @@ static inline bool cpu_has_secondary_exec_ctrls(void)CPU_BASED_ACTIVATE_SECONDARY_CONTROLS ; } +static inline bool cpu_has_tertiary_exec_ctrls (void ) +{ +return vmcs_config.cpu_based_exec_ctrl & +CPU_BASED_ACTIVATE_TERTIARY_CONTROLS ; +} +static inline bool cpu_has_vmx_virtualize_apic_accesses (void ) { return vmcs_config.cpu_based_2nd_exec_ctrl &
arch/x86/kvm/vmx/evmcs.c +2 -0
@@ -299,8 +299,10 @@ const unsigned int nr_evmcs_1_fields = ARRAY_SIZE(vmcs_field_to_evmcs_1);__init void evmcs_sanitize_exec_ctrls (struct vmcs_config *vmcs_conf) { + vmcs_conf->cpu_based_exec_ctrl &= ~EVMCS1_UNSUPPORTED_EXEC_CTRL ; vmcs_conf->pin_based_exec_ctrl &= ~EVMCS1_UNSUPPORTED_PINCTRL ; vmcs_conf->cpu_based_2nd_exec_ctrl &= ~EVMCS1_UNSUPPORTED_2NDEXEC ; + vmcs_conf->cpu_based_3rd_exec_ctrl =0 ; vmcs_conf->vmexit_ctrl &= ~EVMCS1_UNSUPPORTED_VMEXIT_CTRL ; vmcs_conf->vmentry_ctrl &= ~EVMCS1_UNSUPPORTED_VMENTRY_CTRL ;
arch/x86/kvm/vmx/evmcs.h +1 -0
@@ -50,6 +50,7 @@ DECLARE_STATIC_KEY_FALSE(enable_evmcs);*/ #define EVMCS1_UNSUPPORTED_PINCTRL (PIN_BASED_POSTED_INTR | \ PIN_BASED_VMX_PREEMPTION_TIMER) +#define EVMCS1_UNSUPPORTED_EXEC_CTRL (CPU_BASED_ACTIVATE_TERTIARY_CONTROLS)#define EVMCS1_UNSUPPORTED_2NDEXEC \ (SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY | \ SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES | \
arch/x86/kvm/vmx/vmcs.h +1 -0
@@ -48,6 +48,7 @@ struct vmcs_controls_shadow {u32 pin;u32 exec;u32 secondary_exec; +u64 tertiary_exec; };/*
arch/x86/kvm/vmx/vmx.c +39 -1
@@ -2391,6 +2391,23 @@ static __init int adjust_vmx_controls(u32 ctl_min, u32 ctl_opt,return 0 ; } +static __init int adjust_vmx_controls_64 (u64 ctl_min,u64 ctl_opt, +u32 msr,u64 *result) +{ +u64 vmx_msr; +u64 ctl = ctl_min | ctl_opt; + +rdmsrl (msr, vmx_msr); + ctl &= vmx_msr;/* bit == 1 means it can be set */ + +/* Ensure minimum (required) set of control bits are supported. */ +if (ctl_min & ~ctl) +return -EIO ; + + *result = ctl; +return 0 ; +} +static __init int setup_vmcs_config (struct vmcs_config *vmcs_conf,struct vmx_capability *vmx_cap) {
@@ -2399,6 +2416,7 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf,u32 _pin_based_exec_control =0 ;u32 _cpu_based_exec_control =0 ;u32 _cpu_based_2nd_exec_control =0 ; +u64 _cpu_based_3rd_exec_control =0 ;u32 _vmexit_control =0 ;u32 _vmentry_control =0 ;
@@ -2420,7 +2438,8 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, opt =CPU_BASED_TPR_SHADOW |CPU_BASED_USE_MSR_BITMAPS | -CPU_BASED_ACTIVATE_SECONDARY_CONTROLS ; +CPU_BASED_ACTIVATE_SECONDARY_CONTROLS | +CPU_BASED_ACTIVATE_TERTIARY_CONTROLS ;if (adjust_vmx_controls (min, opt,MSR_IA32_VMX_PROCBASED_CTLS , &_cpu_based_exec_control) <0 )return -EIO ;
@@ -2494,6 +2513,16 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, "1-setting enable VPID VM-execution control\n"); } +if (_cpu_based_exec_control &CPU_BASED_ACTIVATE_TERTIARY_CONTROLS ) { +u64 opt3 =0 ; +u64 min3 =0 ; + +if (adjust_vmx_controls_64 (min3, opt3, +MSR_IA32_VMX_PROCBASED_CTLS3 , + &_cpu_based_3rd_exec_control)) +return -EIO ; + } + min =VM_EXIT_SAVE_DEBUG_CONTROLS |VM_EXIT_ACK_INTR_ON_EXIT ;#ifdef CONFIG_X86_64 min |=VM_EXIT_HOST_ADDR_SPACE_SIZE ;
@@ -2581,6 +2610,7 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf, vmcs_conf->pin_based_exec_ctrl = _pin_based_exec_control; vmcs_conf->cpu_based_exec_ctrl = _cpu_based_exec_control; vmcs_conf->cpu_based_2nd_exec_ctrl = _cpu_based_2nd_exec_control; + vmcs_conf->cpu_based_3rd_exec_ctrl = _cpu_based_3rd_exec_control; vmcs_conf->vmexit_ctrl = _vmexit_control; vmcs_conf->vmentry_ctrl = _vmentry_control;
@@ -4148,6 +4178,11 @@ u32 vmx_exec_control(struct vcpu_vmx *vmx)return exec_control; } +u64 vmx_tertiary_exec_control (struct vcpu_vmx *vmx) +{ +return vmcs_config.cpu_based_3rd_exec_ctrl; +} +/* * Adjust a single secondary execution control bit to intercept/allow an * instruction in the guest. This is usually done based on whether or not a
@@ -4319,6 +4354,9 @@ static void init_vmcs(struct vcpu_vmx *vmx)secondary_exec_controls_set (vmx, vmx->secondary_exec_control); } +if (cpu_has_tertiary_exec_ctrls ()) +tertiary_exec_controls_set (vmx,vmx_tertiary_exec_control (vmx)); +if (kvm_vcpu_apicv_active (&vmx->vcpu)) {vmcs_write64 (EOI_EXIT_BITMAP0 ,0 );vmcs_write64 (EOI_EXIT_BITMAP1 ,0 );
arch/x86/kvm/vmx/vmx.h +1 -0
@@ -478,6 +478,7 @@ static inline u32 vmx_vmexit_ctrl(void)u32 vmx_exec_control (struct vcpu_vmx *vmx);u32 vmx_pin_based_exec_ctrl (struct vcpu_vmx *vmx); +u64 vmx_tertiary_exec_control (struct vcpu_vmx *vmx);static inline struct kvm_vmx *to_kvm_vmx (struct kvm *kvm) {
Signature · 1 line
2.25.1
Sean Christopherson
maintainerRe: [PATCH v3 3/6] KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
Quoted 1 line · Zeng Guang
+u64
vmx_tertiary_exec_control(struct vcpu_vmx *vmx)
Make this static and drop the declaration from vmx.h, there's no nested user (yet), and I'm also working on a patch to rework how prepare_vmcs02_early() gets KVMs desires without having to call these heleprs, i.e. I want to bury all of these in vmx.c.
Quoted 32 lines
arch/x86/kvm/vmx/vmx.c +7
-0+{ +return vmcs_config.cpu_based_3rd_exec_ctrl; +} +/* * Adjust a single secondary execution control bit to intercept/allow an * instruction in the guest. This is usually done based on whether or not a @@ -4319,6 +4354,9 @@ static void init_vmcs(struct vcpu_vmx *vmx)secondary_exec_controls_set (vmx, vmx->secondary_exec_control); } +if (cpu_has_tertiary_exec_ctrls ()) +tertiary_exec_controls_set (vmx,vmx_tertiary_exec_control (vmx)); +if (kvm_vcpu_apicv_active (&vmx->vcpu)) {vmcs_write64 (EOI_EXIT_BITMAP0 ,0 );vmcs_write64 (EOI_EXIT_BITMAP1 ,0 );arch/x86/kvm/vmx/vmx.h +1
-0@@ -478,6 +478,7 @@ static inline u32 vmx_vmexit_ctrl(void)u32 vmx_exec_control (struct vcpu_vmx *vmx);u32 vmx_pin_based_exec_ctrl (struct vcpu_vmx *vmx); +u64 vmx_tertiary_exec_control (struct vcpu_vmx *vmx);static inline struct kvm_vmx *to_kvm_vmx (struct kvm *kvm) {Signature · 1 line
2.25.1
Re: [PATCH v3 3/6] KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
Quoted 5 lines · Sean Christopherson
Quoted 1 line · Zeng Guang
+u64
vmx_tertiary_exec_control(struct vcpu_vmx *vmx)Make this static and drop the declaration from
vmx.h, there's no nested user (yet),
I assume that this feature isn't actually virtualizable, since it's based on posted interrupts, which are not virtualizable.
Re: [PATCH v3 3/6] KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
In reply to Sean ChristophersonQuoted 6 lines · Sean Christopherson
Quoted 1 line · Zeng Guang
+u64
vmx_tertiary_exec_control(struct vcpu_vmx *vmx)Make this static and drop the declaration from
vmx.h, there's no nested user (yet), and I'm also working on a patch to rework howprepare_vmcs02_early()gets KVMs desires without having to call these heleprs, i.e. I want to bury all of these invmx.c.
OK. Thus it could be static.
Quoted 32 lines
Quoted 32 lines
arch/x86/kvm/vmx/vmx.c +7
-0+{ +return vmcs_config.cpu_based_3rd_exec_ctrl; +} +/* * Adjust a single secondary execution control bit to intercept/allow an * instruction in the guest. This is usually done based on whether or not a @@ -4319,6 +4354,9 @@ static void init_vmcs(struct vcpu_vmx *vmx)secondary_exec_controls_set (vmx, vmx->secondary_exec_control); } +if (cpu_has_tertiary_exec_ctrls ()) +tertiary_exec_controls_set (vmx,vmx_tertiary_exec_control (vmx)); +if (kvm_vcpu_apicv_active (&vmx->vcpu)) {vmcs_write64 (EOI_EXIT_BITMAP0 ,0 );vmcs_write64 (EOI_EXIT_BITMAP1 ,0 );arch/x86/kvm/vmx/vmx.h +1
-0@@ -478,6 +478,7 @@ static inline u32 vmx_vmexit_ctrl(void)u32 vmx_exec_control (struct vcpu_vmx *vmx);u32 vmx_pin_based_exec_ctrl (struct vcpu_vmx *vmx); +u64 vmx_tertiary_exec_control (struct vcpu_vmx *vmx);static inline struct kvm_vmx *to_kvm_vmx (struct kvm *kvm) {Signature · 1 line
2.25.1
PATCH v3 4/6 KVM: VMX: dump_vmcs() reports tertiary_exec_control field as well
In reply to Zeng GuangFrom: Robert Hoo <[email protected]>
Add tertiary_exec_control field report in dump_vmcs()
--- arch/x86/kvm/vmx/vmx.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +7 -2
@@ -5745,6 +5745,7 @@ void dump_vmcs(struct kvm_vcpu *vcpu)struct vcpu_vmx *vmx =to_vmx (vcpu);u32 vmentry_ctl, vmexit_ctl;u32 cpu_based_exec_ctrl, pin_based_exec_ctrl, secondary_exec_control; +u64 tertiary_exec_control =0 ;unsigned long cr4;int efer_slot;
@@ -5762,6 +5763,9 @@ void dump_vmcs(struct kvm_vcpu *vcpu)if (cpu_has_secondary_exec_ctrls ()) secondary_exec_control =vmcs_read32 (SECONDARY_VM_EXEC_CONTROL ); +if (cpu_has_tertiary_exec_ctrls ()) + tertiary_exec_control =vmcs_read64 (TERTIARY_VM_EXEC_CONTROL ); +pr_err ("VMCS %p, last attempted VM-entry on CPU %d\n" , vmx->loaded_vmcs->vmcs, vcpu->arch.last_vmentry_cpu);pr_err ("*** Guest State ***\n" );
@@ -5860,8 +5864,9 @@ void dump_vmcs(struct kvm_vcpu *vcpu)vmx_dump_msrs ("host autoload" , &vmx->msr_autoload.host);pr_err ("*** Control State ***\n" ); -pr_err ("PinBased=%08x CPUBased=%08x SecondaryExec=%08x\n" , - pin_based_exec_ctrl, cpu_based_exec_ctrl, secondary_exec_control); +pr_err ("PinBased=0x%08x CPUBased=0x%08x SecondaryExec=0x%08x TertiaryExec=0x%016llx\n" , + pin_based_exec_ctrl, cpu_based_exec_ctrl, secondary_exec_control, + tertiary_exec_control);pr_err ("EntryControls=%08x ExitControls=%08x\n" , vmentry_ctl, vmexit_ctl); pr_err("ExceptionBitmap=%08x PFECmask=%08x PFECmatch=%08x\n", vmcs_read32(EXCEPTION_BITMAP),
Signature · 1 line
2.25.1
PATCH v3 5/6 KVM: x86: Support interrupt dispatch in x2APIC mode with APIC-write VM exit
In reply to Zeng GuangSince IA x86 platform introduce features of IPI virtualization and User Interrupts, new behavior applies to the execution of WRMSR ICR register that causes APIC-write VM exit instead of MSR-write VM exit in x2APIC mode.
This requires KVM to emulate writing 64-bit value to offset 300H on the virtual-APIC page(VICR) for guest running in x2APIC mode when APIC-wrtie VM exit occurs. Prevoisely KVM doesn't consider this situation as CPU never produce APIC-write VM exit in x2APIC mode before.
--- arch/x86/kvm/lapic.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
arch/x86/kvm/lapic.c +8 -1
@@ -2188,7 +2188,14 @@ void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset)/* hw has done the conditional check and inst decode */ offset &=0xff0 ; -kvm_lapic_reg_read (vcpu->arch.apic, offset,4 , &val); +if (apic_x2apic_mode (vcpu->arch.apic) && (offset ==APIC_ICR )) { +u64 icr_val = *((u64 *)(vcpu->arch.apic->regs + offset)); + +kvm_lapic_reg_write (vcpu->arch.apic,APIC_ICR2 , (u32)(icr_val>>32 )); + val = (u32 )icr_val; + }else { +kvm_lapic_reg_read (vcpu->arch.apic, offset,4 , &val); + }/* TODO: optimize to just emulate side effect w/o one more write */ kvm_lapic_reg_write (vcpu->arch.apic, offset, val);
Signature · 1 line
2.25.1
PATCH v3 6/6 KVM: VMX: enable IPI virtualization
In reply to Zeng GuangFrom: Gao Chao <[email protected]>
With IPI virtualization enabled, the processor emulates writes to APIC registers that would send IPIs. The processor sets the bit corresponding to the vector in target vCPU's PIR and may send a notification (IPI) specified by NDST and NV fields in target vCPU's PID. It is similar to what IOMMU engine does when dealing with posted interrupt from devices.
A PID-pointer table is used by the processor to locate the PID of a vCPU with the vCPU's APIC ID.
Like VT-d PI, if a vCPU goes to blocked state, VMM needs to switch its notification vector to wakeup vector. This can ensure that when an IPI for blocked vCPUs arrives, VMM can get control and wake up blocked vCPUs. And if a VCPU is preempted, its posted interrupt notification is suppressed.
Note that IPI virtualization can only virualize physical-addressing, flat mode, unicast IPIs. Sending other IPIs would still cause a VM exit and need to be handled by VMM.
--- arch/x86/include/asm/vmx.h | 8 ++++ arch/x86/include/asm/vmxfeatures.h | 2 + arch/x86/kvm/vmx/capabilities.h | 7 +++ arch/x86/kvm/vmx/posted_intr.c | 22 +++++++--- arch/x86/kvm/vmx/vmx.c | 69 ++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmx.h | 3 ++ 6 files changed, 101 insertions(+), 10 deletions(-)
arch/x86/include/asm/vmx.h +8 -0
@@ -76,6 +76,11 @@#define SECONDARY_EXEC_ENABLE_USR_WAIT_PAUSE VMCS_CONTROL_BIT(USR_WAIT_PAUSE)#define SECONDARY_EXEC_BUS_LOCK_DETECTION VMCS_CONTROL_BIT(BUS_LOCK_DETECTION) +/* +* Definitions of Tertiary Processor-Based VM-Execution Controls. +*/ +#define TERTIARY_EXEC_IPI_VIRT VMCS_CONTROL_BIT(IPI_VIRT) +#define PIN_BASED_EXT_INTR_MASK VMCS_CONTROL_BIT(INTR_EXITING)#define PIN_BASED_NMI_EXITING VMCS_CONTROL_BIT(NMI_EXITING)#define PIN_BASED_VIRTUAL_NMIS VMCS_CONTROL_BIT(VIRTUAL_NMIS)
@@ -159,6 +164,7 @@ static inline int vmx_misc_mseg_revid(u64 vmx_misc)enum vmcs_field {VIRTUAL_PROCESSOR_ID =0x00000000 ,POSTED_INTR_NV =0x00000002 , +LAST_PID_POINTER_INDEX =0x00000008 ,GUEST_ES_SELECTOR =0x00000800 ,GUEST_CS_SELECTOR =0x00000802 ,GUEST_SS_SELECTOR =0x00000804 ,
@@ -224,6 +230,8 @@ enum vmcs_field {TSC_MULTIPLIER_HIGH =0x00002033 ,TERTIARY_VM_EXEC_CONTROL =0x00002034 ,TERTIARY_VM_EXEC_CONTROL_HIGH =0x00002035 , +PID_POINTER_TABLE =0x00002042 , +PID_POINTER_TABLE_HIGH =0x00002043 ,GUEST_PHYSICAL_ADDRESS =0x00002400 ,GUEST_PHYSICAL_ADDRESS_HIGH =0x00002401 , VMCS_LINK_POINTER = 0x00002800,
arch/x86/include/asm/vmxfeatures.h +2 -0
@@ -86,4 +86,6 @@#define VMX_FEATURE_ENCLV_EXITING ( 2*32+ 28)/* "" VM-Exit on ENCLV (leaf dependent) */ #define VMX_FEATURE_BUS_LOCK_DETECTION ( 2*32+ 30)/* "" VM-Exit when bus lock caused */ +/* Tertiary Processor-Based VM-Execution Controls, word 3 */ +#define VMX_FEATURE_IPI_VIRT ( 3*32 + 4)/* "" Enable IPI virtualization */ #endif /* _ASM_X86_VMXFEATURES_H */
arch/x86/kvm/vmx/capabilities.h +7 -0
@@ -12,6 +12,7 @@ extern bool __read_mostly enable_ept;extern bool __read_mostly enable_unrestricted_guest;extern bool __read_mostly enable_ept_ad_bits;extern bool __read_mostly enable_pml; +extern bool __read_mostly enable_ipiv;extern int __read_mostly pt_mode;#define PT_MODE_SYSTEM 0
@@ -283,6 +284,12 @@ static inline bool cpu_has_vmx_apicv(void)cpu_has_vmx_posted_intr (); } +static inline bool cpu_has_vmx_ipiv (void ) +{ +return vmcs_config.cpu_based_3rd_exec_ctrl & +TERTIARY_EXEC_IPI_VIRT ; +} +static inline bool cpu_has_vmx_flexpriority (void ) { return cpu_has_vmx_tpr_shadow() &&
arch/x86/kvm/vmx/posted_intr.c +16 -6
@@ -81,9 +81,12 @@ void vmx_vcpu_pi_put(struct kvm_vcpu *vcpu) {struct pi_desc *pi_desc =vcpu_to_pi_desc (vcpu); -if (!kvm_arch_has_assigned_device (vcpu->kvm) || - !irq_remapping_cap (IRQ_POSTING_CAP ) || - !kvm_vcpu_apicv_active (vcpu)) +if (!kvm_vcpu_apicv_active (vcpu)) +return ; + +if ((!kvm_arch_has_assigned_device (vcpu->kvm) || + !irq_remapping_cap (IRQ_POSTING_CAP )) && + !enable_ipiv)return ;/* Set SN when the vCPU is preempted */
@@ -141,9 +144,16 @@ int pi_pre_block(struct kvm_vcpu *vcpu)struct pi_desc old, new;struct pi_desc *pi_desc =vcpu_to_pi_desc (vcpu); -if (!kvm_arch_has_assigned_device (vcpu->kvm) || - !irq_remapping_cap (IRQ_POSTING_CAP ) || - !kvm_vcpu_apicv_active (vcpu)) +if (!kvm_vcpu_apicv_active (vcpu)) +return 0 ; + +/* Put vCPU into a list and set NV to wakeup vector if it is +* one of the following cases: +* 1. any assigned device is in use. +* 2. IPI virtualization is enabled. +*/ +if ((!kvm_arch_has_assigned_device (vcpu->kvm) || + !irq_remapping_cap (IRQ_POSTING_CAP )) && !enable_ipiv)return 0 ;WARN_ON (irqs_disabled ());
arch/x86/kvm/vmx/vmx.c +65 -4
147 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.h +3 -0
@@ -351,6 +351,9 @@ struct kvm_vmx {unsigned int tss_addr;bool ept_identity_pagetable_done;gpa_t ept_identity_map_addr; +/* PID table for IPI virtualization */ +u64 *pid_table; +u16 pid_last_index; };bool nested_vmx_allowed (struct kvm_vcpu *vcpu);
Signature · 1 line
2.25.1
Quoted 10 lines · Zeng Guang
Current IPI process in guest VM will virtualize the writing to interrupt command register(ICR) of the local APIC which will cause VM-exit anyway on source vCPU. Frequent VM-exit could induce much overhead accumulated if running IPI intensive task.
IPI virtualization as a new VT-x feature targets to eliminate VM-exits when issuing IPI on source vCPU. It introduces a new VM-execution control - "IPI virtualization"(bit4) in the tertiary processor-based VM-exection controls and a new data structure - "PID-pointer table
VM-execution
Quoted 2 lines
address" and "Last PID-pointer index" referenced by the VMCS. When "IPI virtualization" is enabled, processor emulateds following kind of writes
emulates
Quoted 6 lines
to APIC registers that would send IPIs, moreover without causing VM-exits.
- Memory-mapped ICR writes
- MSR-mapped ICR writes
- SENDUIPI executionThis patch series implement IPI virtualization support in KVM.
implements
Quoted 5 lines
Patches 1-4 add tertiary processor-based VM-execution support framework.
Patch 5 implement interrupt dispatch support in x2APIC mode with
implements
Quoted 2 lines
APIC-write VM exit. In previous platform, no CPU would produce APIC-write VM exit with exit qulification 300H when the "virtual x2APIC
qualification
Quoted 5 lines
mode" VM-execution control was 1.
Patch 6 implement IPI virtualization related function including feature enabling through tertiary processor-based VM-execution in various scenario of VMCS configuration, PID table setup in vCPU creation
scenarios
Quoted 11 lines
and vCPU block consideration.
Document for IPI virtualization is now available at the latest "Intel Architecture Instruction Set Extensions Programming Reference".
We did experiment to measure average time sending IPI from source vCPU to the target vCPU completing the IPI handling by kvm unittest w/ and w/o IPI virtualization. When IPI virtualizatin enabled, it will reduce
virtualization
Quoted 2 lines
22.21% and 15.98% cycles consuming in xAPIC mode and x2APIC mode respectly.
respectively
Quoted 2 lines
KMV unittest:vmexit/ipi, 2 vCPU, AP was modified to run in idle loop
KVM
Quoted 1 line
instead of halt to ensure no VM exit impact on target vCPU.
Are you going to post the kvm-init-test changes?
Quoted 25 lines
Cycles of IPI
xAPIC mode x2APIC mode
test w/o IPIv w/ IPIv w/o IPIv w/ IPIv
1 6106 4816 4265 3768
2 6244 4656 4404 3546
3 6165 4658 4233 3474
4 5992 4710 4363 3430
5 6083 4741 4215 3551
6 6238 4904 4304 3547
7 6164 4617 4263 3709
8 5984 4763 4518 3779
9 5931 4712 4645 3667
10 5955 4530 4332 3724
11 5897 4673 4283 3569
12 6140 4794 4178 3598
13 6183 4728 4363 3628
14 5991 4994 4509 3842
15 5866 4665 4520 3739
16 6032 4654 4229 3701
17 6050 4653 4185 3726
18 6004 4792 4319 3746
19 5961 4626 4196 3392
20 6194 4576 4433 3760Quoted 8 lines
Average cycles 6059 4713.1 4337.85 3644.8 %Reduction -22.21% -15.98%--------------------------------------
IPI microbenchmark:
(https://lore.kernel.org/kvm/[email protected])2 vCPUs, 1:1 pin vCPU to pCPU, guest VM runs with idle=poll, x2APIC mode
For dedicated CPUs, we would just disable the MONITOR/MWAIT intercepts and expose MONITOR/MWAIT to the guest. We would not recommend that anyone use idle=poll.
Quoted 10 lines
Result with IPIv enabled:
Dry-run: 0, 272798 ns Self-IPI: 5094123, 11114037 ns Normal IPI: 131697087, 173321200 ns Broadcast IPI: 0, 155649075 ns Broadcast lock: 0, 161518031 nsResult with IPIv disabled:
Quoted 8 lines
Dry-run: 0, 272766 ns Self-IPI: 5091788, 11123699 ns Normal IPI: 145215772, 174558920 ns Broadcast IPI: 0, 175785384 ns Broadcast lock: 0, 149076195 nsAs IPIv can benefit unicast IPI to other CPU, Noraml IPI test case gain
Normal
Quoted 2 lines
about 9.73% time saving on average out of 15 test runs when IPIv is enabled.
Can you share the CDFs?
Quoted 65 lines
w/o IPIv w/ IPIv Normal IPI: 145944306.6 ns 131742993.1 ns %Reduction -9.73%--------------------------------------
hackbench:8 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 100000w/o IPIv w/ IPIv Time: 91.887 74.605 %Reduction: -18.808%96 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 1000000w/o IPIv w/ IPIv Time: 287.504 235.185 %Reduction: -18.198%--------------------------------------
v2 -> v3:
1. Misc change on tertiary execution control
definition and capability setup
2. Alternative to get tertiary execution
control configurationv1 -> v2:
1. Refine the IPIv enabling logic for VM.
Removeipiv_activedefinition per vCPU.Gao Chao (1):
KVM: VMX: enable IPI virtualizationRobert Hoo (4):
x86/feat_ctl: Add new VMX feature, Tertiary VM-Execution control
KVM: VMX: ExtendBUILD_CONTROLS_SHADOWmacro to support 64-bit
variation
KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
KVM: VMX:dump_vmcs()reportstertiary_exec_controlfield as wellZeng Guang (1):
KVM: x86: Support interrupt dispatch in x2APIC mode with APIC-write VM
exitarch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/vmx.h | 11 +++ arch/x86/include/asm/vmxfeatures.h | 5 +- arch/x86/kernel/cpu/feat_ctl.c | 11 ++- arch/x86/kvm/lapic.c | 9 ++- arch/x86/kvm/vmx/capabilities.h | 14 ++++ arch/x86/kvm/vmx/evmcs.c | 2 + arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/posted_intr.c | 22 ++++-- arch/x86/kvm/vmx/vmcs.h | 1 + arch/x86/kvm/vmx/vmx.c | 114 +++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmx.h | 27 ++++--- 12 files changed, 193 insertions(+), 25 deletions(-)--
2.25.1
Quoted 47 lines · Jim Mattson
On Thu, Aug 5, 2021 at 8:38 AM Zeng Guang <[email protected]> wrote:
Current IPI process in guest VM will virtualize the writing to interrupt command register(ICR) of the local APIC which will cause VM-exit anyway on source vCPU. Frequent VM-exit could induce much overhead accumulated if running IPI intensive task.
IPI virtualization as a new VT-x feature targets to eliminate VM-exits when issuing IPI on source vCPU. It introduces a new VM-execution control - "IPI virtualization"(bit4) in the tertiary processor-based VM-exection controls and a new data structure - "PID-pointer table VM-execution
Quoted 2 lines
address" and "Last PID-pointer index" referenced by the VMCS. When "IPI virtualization" is enabled, processor emulateds following kind of writes
emulates
to APIC registers that would send IPIs, moreover without causing VM-exits.
- Memory-mapped ICR writes
- MSR-mapped ICR writes
- SENDUIPI executionThis patch series implement IPI virtualization support in KVM. implements
Quoted 4 lines
Patches 1-4 add tertiary processor-based VM-execution support framework.
Patch 5 implement interrupt dispatch support in x2APIC mode with
implements
Quoted 2 lines
APIC-write VM exit. In previous platform, no CPU would produce APIC-write VM exit with exit qulification 300H when the "virtual x2APIC
qualification
Quoted 5 lines
mode" VM-execution control was 1.
Patch 6 implement IPI virtualization related function including feature enabling through tertiary processor-based VM-execution in various scenario of VMCS configuration, PID table setup in vCPU creation
scenarios
Quoted 11 lines
and vCPU block consideration.
Document for IPI virtualization is now available at the latest "Intel Architecture Instruction Set Extensions Programming Reference".
We did experiment to measure average time sending IPI from source vCPU to the target vCPU completing the IPI handling by kvm unittest w/ and w/o IPI virtualization. When IPI virtualizatin enabled, it will reduce
Quoted 6 lines
virtualization
Quoted 2 lines
22.21% and 15.98% cycles consuming in xAPIC mode and x2APIC mode respectly.
respectively
Quoted 1 line
KMV unittest:vmexit/ipi, 2 vCPU, AP was modified to run in idle loop
KVM
Thanks for your all corrections.
Quoted 2 lines
Quoted 1 line
instead of halt to ensure no VM exit impact on target vCPU.
Are you going to post the kvm-init-test changes?
We modified unit test common code specific for the IPIv test purpose, currently no plan to post those changes to kvm-uint-tests.
Quoted 27 lines
Quoted 27 lines
Cycles of IPI xAPIC mode x2APIC mode test w/o IPIv w/ IPIv w/o IPIv w/ IPIv 1 6106 4816 4265 3768 2 6244 4656 4404 3546 3 6165 4658 4233 3474 4 5992 4710 4363 3430 5 6083 4741 4215 3551 6 6238 4904 4304 3547 7 6164 4617 4263 3709 8 5984 4763 4518 3779 9 5931 4712 4645 3667 10 5955 4530 4332 3724 11 5897 4673 4283 3569 12 6140 4794 4178 3598 13 6183 4728 4363 3628 14 5991 4994 4509 3842 15 5866 4665 4520 3739 16 6032 4654 4229 3701 17 6050 4653 4185 3726 18 6004 4792 4319 3746 19 5961 4626 4196 3392 20 6194 4576 4433 3760Average cycles 6059 4713.1 4337.85 3644.8 %Reduction -22.21% -15.98%
Quoted 8 lines
Quoted 5 lines
--------------------------------------
IPI microbenchmark:
(https://lore.kernel.org/kvm/[email protected])2 vCPUs, 1:1 pin vCPU to pCPU, guest VM runs with idle=poll, x2APIC mode
For dedicated CPUs, we would just disable the MONITOR/MWAIT intercepts and expose MONITOR/MWAIT to the guest. We would not recommend that anyone use idle=poll.
OK. We thought that straightforward way to make guest keep running in idle without side effect of VM exit and other factors, so used idle=poll for test.
Quoted 17 lines
Quoted 17 lines
Result with IPIv enabled:
Dry-run: 0, 272798 ns Self-IPI: 5094123, 11114037 ns Normal IPI: 131697087, 173321200 ns Broadcast IPI: 0, 155649075 ns Broadcast lock: 0, 161518031 nsResult with IPIv disabled:
Dry-run: 0, 272766 ns Self-IPI: 5091788, 11123699 ns Normal IPI: 145215772, 174558920 ns Broadcast IPI: 0, 175785384 ns Broadcast lock: 0, 149076195 ns
Quoted 5 lines
Quoted 1 line
As IPIv can benefit unicast IPI to other CPU, Noraml IPI test case gain
Normal
Quoted 2 lines
about 9.73% time saving on average out of 15 test runs when IPIv is enabled.
Can you share the CDFs?
Normal IPI statistics:
test w/o IPIv (unit:ns) w/ IPIv (unit:ns) 1 153346049 140907046
2 147218648 141660618
3 145215772 117890672
4 146621682 136430470
5 144821472 136199421
6 144704378 131676928
7 141403224 131697087
8 144775766 125476250
9 140658192 137263330
10 144768626 138593127
11 145166679 131946752
12 145020451 116852889
13 148161353 131406280
14 148378655 130174353
15 148903652 127969674
Quoted 64 lines
Quoted 64 lines
w/o IPIv w/ IPIv
Normal IPI: 145944306.6 ns 131742993.1 ns
%Reduction -9.73%--------------------------------------
hackbench:8 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 100000w/o IPIv w/ IPIv
Time: 91.887 74.605
%Reduction: -18.808%96 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 1000000w/o IPIv w/ IPIv
Time: 287.504 235.185
%Reduction: -18.198%--------------------------------------
v2 -> v3:
1. Misc change on tertiary execution control
definition and capability setup
2. Alternative to get tertiary execution
control configurationv1 -> v2:
1. Refine the IPIv enabling logic for VM.
Removeipiv_activedefinition per vCPU.Gao Chao (1):
KVM: VMX: enable IPI virtualizationRobert Hoo (4):
x86/feat_ctl: Add new VMX feature, Tertiary VM-Execution control
KVM: VMX: ExtendBUILD_CONTROLS_SHADOWmacro to support 64-bit
variation
KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
KVM: VMX:dump_vmcs()reportstertiary_exec_controlfield as wellZeng Guang (1):
KVM: x86: Support interrupt dispatch in x2APIC mode with APIC-write VM
exitarch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/vmx.h | 11 +++ arch/x86/include/asm/vmxfeatures.h | 5 +- arch/x86/kernel/cpu/feat_ctl.c | 11 ++- arch/x86/kvm/lapic.c | 9 ++- arch/x86/kvm/vmx/capabilities.h | 14 ++++ arch/x86/kvm/vmx/evmcs.c | 2 + arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/posted_intr.c | 22 ++++-- arch/x86/kvm/vmx/vmcs.h | 1 + arch/x86/kvm/vmx/vmx.c | 114 +++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmx.h | 27 ++++--- 12 files changed, 193 insertions(+), 25 deletions(-)--
2.25.1