IPI virtualization support for VM
v9 cover IPI virtualization support for VM
Currently, issuing an IPI except self-ipi in guest on Intel CPU always causes a VM-exit. It can lead to non-negligible overhead to some workloads involving frequent IPIs when running in VMs.
IPI virtualization is a new VT-x feature, targeting to eliminate VM-exits on source vCPUs when issuing unicast, physical-addressing IPIs. Once it is enabled, the processor virtualizes following kinds of operations that send IPIs without causing VM-exits:
- Memory-mapped ICR writes
- MSR-mapped ICR writes
- SENDUIPI execution
This patch series implements IPI virtualization support in KVM.
Patches 1-4 add tertiary processor-based VM-execution support framework, which is used to enumerate IPI virtualization.
Patch 5 handles APIC-write VM exit due to writes to ICR MSR when guest works in x2APIC mode. This is a new case introduced by Intel VT-x.
Patch 6 cleanup code in vmx_refresh_apicv_exec_ctrl(). Prepare for IPIv status dynamical update along with APICv status change.
Patch 7 move kvm_arch_vcpu_precreate() under kvm->lock protection. This patch is prepared for IPIv PID-table allocation prior to the creation of vCPUs.
Patch 8 provide userspace capability to set maximum possible VCPU ID for current VM. IPIv can refer to this value to allocate memory for PID-pointer table.
Patch 9 implements IPI virtualization related function including feature enabling through tertiary processor-based VM-execution in various scenarios of VMCS configuration, PID table setup in vCPU creation and vCPU block consideration.
Document for IPI virtualization is now available at the latest "Intel Architecture Instruction Set Extensions Programming Reference".
We did experiment to measure average time sending IPI from source vCPU to the target vCPU completing the IPI handling by kvm unittest w/ and w/o IPI virtualization. When IPI virtualization enabled, it will reduce 22.21% and 15.98% cycles consuming in xAPIC mode and x2APIC mode respectively.
--------------------------------------
KVM unittest:vmexit/ipi
2 vCPU, AP was modified to run in idle loop instead of halt to ensure no VM exit impact on target vCPU.
Cycles of IPI
xAPIC mode x2APIC mode
test w/o IPIv w/ IPIv w/o IPIv w/ IPIv
1 6106 4816 4265 3768
2 6244 4656 4404 3546
3 6165 4658 4233 3474
4 5992 4710 4363 3430
5 6083 4741 4215 3551
6 6238 4904 4304 3547
7 6164 4617 4263 3709
8 5984 4763 4518 3779
9 5931 4712 4645 3667
10 5955 4530 4332 3724
11 5897 4673 4283 3569
12 6140 4794 4178 3598
13 6183 4728 4363 3628
14 5991 4994 4509 3842
15 5866 4665 4520 3739
16 6032 4654 4229 3701
17 6050 4653 4185 3726
18 6004 4792 4319 3746
19 5961 4626 4196 3392
20 6194 4576 4433 3760Average cycles 6059 4713.1 4337.85 3644.8 %Reduction -22.21% -15.98%
--------------------------------------
IPI microbenchmark:
(https://lore.kernel.org/kvm/[email protected])
2 vCPUs, 1:1 pin vCPU to pCPU, guest VM runs with idle=poll, x2APIC mode
Result with IPIv enabled:
Dry-run: 0, 272798 ns Self-IPI: 5094123, 11114037 ns Normal IPI: 131697087, 173321200 ns Broadcast IPI: 0, 155649075 ns Broadcast lock: 0, 161518031 ns
Result with IPIv disabled:
Dry-run: 0, 272766 ns Self-IPI: 5091788, 11123699 ns Normal IPI: 145215772, 174558920 ns Broadcast IPI: 0, 175785384 ns Broadcast lock: 0, 149076195 ns
As IPIv can benefit unicast IPI to other CPU, Normal IPI test case gain about 9.73% time saving on average out of 15 test runs when IPIv is enabled.
Normal IPI statistics (unit:ns):
test w/o IPIv w/ IPIv
1 153346049 140907046
2 147218648 141660618
3 145215772 117890672
4 146621682 136430470
5 144821472 136199421
6 144704378 131676928
7 141403224 131697087
8 144775766 125476250
9 140658192 137263330
10 144768626 138593127
11 145166679 131946752
12 145020451 116852889
13 148161353 131406280
14 148378655 130174353
15 148903652 127969674Average time 145944306.6 131742993.1 ns
%Reduction -9.73%
--------------------------------------
hackbench:
8 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 100000
w/o IPIv w/ IPIv Time 91.887 74.605 %Reduction -18.808%
96 vCPUs, guest VM fre run, x2APIC mode
./hackbench -p -l 1000000
w/o IPIv w/ IPIv Time 287.504 235.185 %Reduction -18.198%
--------------------------------------v8->v9:
1. Drop patch to forbid change of APIC ID.
2. Change max_vcpu_ids only set once
3. Refactor vCPU pre-creation code
v7->v8:
1. Add trace in kvm_apic_write_nodecode() to track vICR Write in APIC Write VM-exit handling
2. Move IPIv PID table allocation done in the vCPU pre-creation (kvm_arch_vcpu_precreate()) protected by kvm->lock.
3. Misc code refine
v6->v7:
1. Revise kvm_apic_write_nodecode() on dealing with vICR busy bit in x2apic mode
2. Merge PID-table memory allocation with max_vcpu_id into IPIv enabling patch
3. Change to allocate PID-table, setup vCPU's PID-table entry and IPIv related VMCS fields once IPIv can be enabled, which support runtime enabling IPIv.
v5->v6:
1. Adapt kvm_apic_write_nodecode() implementation based on Sean's fix of x2apic's ICR register process.
2. Drop the patch handling IPIv table entry setting in case APIC ID changed, instead applying Levitsky's patch to disallow setting APIC ID in any case.
3. Drop the patch resizing the PID-pointer table on demand. Allow userspace to set maximum vcpu id at runtime that IPIv can refer to the practical value to allocate memory for PID-pointer table.
v4 -> v5:
1. Deal with enable_ipiv parameter following current vmcs configuration rule.
2. Allocate memory for PID-pointer table dynamically
3. Support guest runtime modify APIC ID in xAPIC mode
4. Helper to judge possibility to take PI block in IPIv case
v3 -> v4:
1. Refine code style of patch 2
2. Move tertiary control shadow build into patch 3
3. Make vmx_tertiary_exec_control to be static function
v2 -> v3:
1. Misc change on tertiary execution control
definition and capability setup
2. Alternative to get tertiary execution
control configuration
v1 -> v2:
1. Refine the IPIv enabling logic for VM.
Remove ipiv_active definition per vCPU.
--------------------------------------
Chao Gao (1):
KVM: VMX: enable IPI virtualization
Robert Hoo (4):
x86/cpu: Add new VMX feature, Tertiary VM-Execution control
KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit
variation
KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
KVM: VMX: Report tertiary_exec_control field in dump_vmcs()
Zeng Guang (4):
KVM: x86: Add support for vICR APIC-write VM-Exits in x2APIC mode
KVM: VMX: Clean up vmx_refresh_apicv_exec_ctrl()
KVM: Move kvm_arch_vcpu_precreate() under kvm->lock
KVM: x86: Allow userspace set maximum VCPU id for VM
Documentation/virt/kvm/api.rst | 18 ++++ arch/s390/kvm/kvm-s390.c | 2 - arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 7 ++ arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/vmx.h | 11 +++ arch/x86/include/asm/vmxfeatures.h | 5 +- arch/x86/kernel/cpu/feat_ctl.c | 9 +- arch/x86/kvm/lapic.c | 24 ++++- arch/x86/kvm/vmx/capabilities.h | 13 +++ arch/x86/kvm/vmx/evmcs.c | 2 + arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/posted_intr.c | 15 +++- arch/x86/kvm/vmx/posted_intr.h | 2 + arch/x86/kvm/vmx/vmcs.h | 1 + arch/x86/kvm/vmx/vmx.c | 137 +++++++++++++++++++++++++---- arch/x86/kvm/vmx/vmx.h | 64 ++++++++------ arch/x86/kvm/x86.c | 29 +++++- virt/kvm/kvm_main.c | 10 ++- 19 files changed, 294 insertions(+), 58 deletions(-)
Signature · 1 line
2.27.0
Kindly PING!
Thanks for your time.
BR,
Zeng Guang
Quoted 241 lines · Guang
Currently, issuing an IPI except self-ipi in guest on Intel CPU always causes a VM-exit. It can lead to non-negligible overhead to some workloads involving frequent IPIs when running in VMs.
IPI virtualization is a new VT-x feature, targeting to eliminate VM-exits on source vCPUs when issuing unicast, physical-addressing IPIs. Once it is enabled, the processor virtualizes following kinds of operations that send IPIs without causing VM-exits:
- Memory-mapped ICR writes
- MSR-mapped ICR writes
- SENDUIPI executionThis patch series implements IPI virtualization support in KVM.
Patches 1-4 add tertiary processor-based VM-execution support framework, which is used to enumerate IPI virtualization.
Patch 5 handles APIC-write VM exit due to writes to ICR MSR when guest works in x2APIC mode. This is a new case introduced by Intel VT-x.
Patch 6 cleanup code in
vmx_refresh_apicv_exec_ctrl(). Prepare for IPIv status dynamical update along with APICv status change.Patch 7 move
kvm_arch_vcpu_precreate()underkvm->lockprotection. This patch is prepared for IPIv PID-table allocation prior to the creation of vCPUs.Patch 8 provide userspace capability to set maximum possible VCPU ID for current VM. IPIv can refer to this value to allocate memory for PID-pointer table.
Patch 9 implements IPI virtualization related function including feature enabling through tertiary processor-based VM-execution in various scenarios of VMCS configuration, PID table setup in vCPU creation and vCPU block consideration.
Document for IPI virtualization is now available at the latest "Intel Architecture Instruction Set Extensions Programming Reference".
We did experiment to measure average time sending IPI from source vCPU to the target vCPU completing the IPI handling by kvm unittest w/ and w/o IPI virtualization. When IPI virtualization enabled, it will reduce 22.21% and 15.98% cycles consuming in xAPIC mode and x2APIC mode respectively.
--------------------------------------
KVM unittest:vmexit/ipi2 vCPU, AP was modified to run in idle loop instead of halt to ensure no VM exit impact on target vCPU.
Cycles of IPI xAPIC mode x2APIC mode test w/o IPIv w/ IPIv w/o IPIv w/ IPIv 1 6106 4816 4265 3768 2 6244 4656 4404 3546 3 6165 4658 4233 3474 4 5992 4710 4363 3430 5 6083 4741 4215 3551 6 6238 4904 4304 3547 7 6164 4617 4263 3709 8 5984 4763 4518 3779 9 5931 4712 4645 3667 10 5955 4530 4332 3724 11 5897 4673 4283 3569 12 6140 4794 4178 3598 13 6183 4728 4363 3628 14 5991 4994 4509 3842 15 5866 4665 4520 3739 16 6032 4654 4229 3701 17 6050 4653 4185 3726 18 6004 4792 4319 3746 19 5961 4626 4196 3392 20 6194 4576 4433 3760Average cycles 6059 4713.1 4337.85 3644.8 %Reduction -22.21% -15.98%--------------------------------------
IPI microbenchmark:
(https://lore.kernel.org/kvm/[email protected])2 vCPUs, 1:1 pin vCPU to pCPU, guest VM runs with idle=poll, x2APIC mode
Result with IPIv enabled:
Dry-run: 0, 272798 ns Self-IPI: 5094123, 11114037 ns Normal IPI: 131697087, 173321200 ns Broadcast IPI: 0, 155649075 ns Broadcast lock: 0, 161518031 nsResult with IPIv disabled:
Dry-run: 0, 272766 ns Self-IPI: 5091788, 11123699 ns Normal IPI: 145215772, 174558920 ns Broadcast IPI: 0, 175785384 ns Broadcast lock: 0, 149076195 nsAs IPIv can benefit unicast IPI to other CPU, Normal IPI test case gain about 9.73% time saving on average out of 15 test runs when IPIv is enabled.
Normal IPI statistics (unit:ns):
test w/o IPIv w/ IPIv 1 153346049 140907046 2 147218648 141660618 3 145215772 117890672 4 146621682 136430470 5 144821472 136199421 6 144704378 131676928 7 141403224 131697087 8 144775766 125476250 9 140658192 137263330 10 144768626 138593127 11 145166679 131946752 12 145020451 116852889 13 148161353 131406280 14 148378655 130174353 15 148903652 127969674Average time 145944306.6 131742993.1 ns
%Reduction -9.73%--------------------------------------
hackbench:8 vCPUs, guest VM free run, x2APIC mode
./hackbench -p -l 100000w/o IPIv w/ IPIv
Time 91.887 74.605
%Reduction -18.808%96 vCPUs, guest VM fre run, x2APIC mode
./hackbench -p -l 1000000w/o IPIv w/ IPIv
Time 287.504 235.185
%Reduction -18.198%--------------------------------------
v8->v9:
1. Drop patch to forbid change of APIC ID.
2. Changemax_vcpu_idsonly set once
3. Refactor vCPU pre-creation code
v7->v8:
1. Add trace inkvm_apic_write_nodecode()to track vICR Write in APIC Write VM-exit handling
2. Move IPIv PID table allocation done in the vCPU pre-creation (kvm_arch_vcpu_precreate()) protected bykvm->lock.
3. Misc code refine
v6->v7:
1. Revisekvm_apic_write_nodecode()on dealing with vICR busy bit in x2apic mode
2. Merge PID-table memory allocation withmax_vcpu_idinto IPIv enabling patch
3. Change to allocate PID-table, setup vCPU's PID-table entry and IPIv related VMCS fields once IPIv can be enabled, which support runtime enabling IPIv.
v5->v6:
1. Adaptkvm_apic_write_nodecode()implementation based on Sean's fix of x2apic's ICR register process.
2. Drop the patch handling IPIv table entry setting in case APIC ID changed, instead applying Levitsky's patch to disallow setting APIC ID in any case.
3. Drop the patch resizing the PID-pointer table on demand. Allow userspace to set maximum vcpu id at runtime that IPIv can refer to the practical value to allocate memory for PID-pointer table.v4 -> v5:
1. Deal withenable_ipivparameter following current vmcs configuration rule.
2. Allocate memory for PID-pointer table dynamically
3. Support guest runtime modify APIC ID in xAPIC mode
4. Helper to judge possibility to take PI block in IPIv casev3 -> v4:
1. Refine code style of patch 2
2. Move tertiary control shadow build into patch 3
3. Makevmx_tertiary_exec_controlto be static functionv2 -> v3:
1. Misc change on tertiary execution control
definition and capability setup
2. Alternative to get tertiary execution
control configurationv1 -> v2:
1. Refine the IPIv enabling logic for VM.
Removeipiv_activedefinition per vCPU.--------------------------------------
Chao Gao (1):
KVM: VMX: enable IPI virtualizationRobert Hoo (4):
x86/cpu: Add new VMX feature, Tertiary VM-Execution control
KVM: VMX: ExtendBUILD_CONTROLS_SHADOWmacro to support 64-bit
variation
KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config
KVM: VMX: Reporttertiary_exec_controlfield indump_vmcs()Zeng Guang (4):
KVM: x86: Add support for vICR APIC-write VM-Exits in x2APIC mode
KVM: VMX: Clean upvmx_refresh_apicv_exec_ctrl()
KVM: Movekvm_arch_vcpu_precreate()underkvm->lock
KVM: x86: Allow userspace set maximum VCPU id for VMDocumentation/virt/kvm/api.rst | 18 ++++ arch/s390/kvm/kvm-s390.c | 2 - arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 7 ++ arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/vmx.h | 11 +++ arch/x86/include/asm/vmxfeatures.h | 5 +- arch/x86/kernel/cpu/feat_ctl.c | 9 +- arch/x86/kvm/lapic.c | 24 ++++- arch/x86/kvm/vmx/capabilities.h | 13 +++ arch/x86/kvm/vmx/evmcs.c | 2 + arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/posted_intr.c | 15 +++- arch/x86/kvm/vmx/posted_intr.h | 2 + arch/x86/kvm/vmx/vmcs.h | 1 + arch/x86/kvm/vmx/vmx.c | 137 +++++++++++++++++++++++++---- arch/x86/kvm/vmx/vmx.h | 64 ++++++++------ arch/x86/kvm/x86.c | 29 +++++- virt/kvm/kvm_main.c | 10 ++- 19 files changed, 294 insertions(+), 58 deletions(-)
Paolo Bonzini
maintainerQuoted 28 lines · Zeng Guang
Currently, issuing an IPI except self-ipi in guest on Intel CPU always causes a VM-exit. It can lead to non-negligible overhead to some workloads involving frequent IPIs when running in VMs.
IPI virtualization is a new VT-x feature, targeting to eliminate VM-exits on source vCPUs when issuing unicast, physical-addressing IPIs. Once it is enabled, the processor virtualizes following kinds of operations that send IPIs without causing VM-exits:
- Memory-mapped ICR writes
- MSR-mapped ICR writes
- SENDUIPI executionThis patch series implements IPI virtualization support in KVM.
Patches 1-4 add tertiary processor-based VM-execution support framework, which is used to enumerate IPI virtualization.
Patch 5 handles APIC-write VM exit due to writes to ICR MSR when guest works in x2APIC mode. This is a new case introduced by Intel VT-x.
Patch 6 cleanup code in
vmx_refresh_apicv_exec_ctrl(). Prepare for IPIv status dynamical update along with APICv status change.Patch 7 move
kvm_arch_vcpu_precreate()underkvm->lockprotection. This patch is prepared for IPIv PID-table allocation prior to the creation of vCPUs.
Quoted 8 lines
Patch 8 provide userspace capability to set maximum possible VCPU ID for current VM. IPIv can refer to this value to allocate memory for PID-pointer table.
Patch 9 implements IPI virtualization related function including feature enabling through tertiary processor-based VM-execution in various scenarios of VMCS configuration, PID table setup in vCPU creation and vCPU block consideration.
I queued it, but I am not going to send it to Linus until I get selftests for KVM_CAP_MAX_VCPU_ID. Selftests are generally _not_ optional for new userspace API.
Please send a patch on top of kvm/queue.
Paolo
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · Zeng Guang
I don't see "[PATCH v9 4/9] KVM: VMX: Report
tertiary_exec_controlfield indump_vmcs()" in kvm/queue. Does it not need ?
Added now (somehow the patches were not threaded, so I had to catch them one by one from lore).
Quoted 3 lines
Selftests for
KVM_CAP_MAX_VCPU_IDis posted in V2 which is revised on top of
kvm/queue.
([PATCH v2] kvm: selftests: AddKVM_CAP_MAX_VCPU_IDcap test - Zeng Guang (kernel.org) <https://lore.kernel.org/lkml/[email protected]/>)
Queued, thanks.
Paolo
Sean Christopherson
reviewerQuoted 8 lines · Paolo Bonzini
Quoted 3 lines · Zeng Guang
I don't see "[PATCH v9 4/9] KVM: VMX: Report
tertiary_exec_controlfield indump_vmcs()" in kvm/queue. Does it not need ?Added now (somehow the patches were not threaded, so I had to catch them one by one from lore).
Quoted 6 lines
Quoted 4 lines
Selftests for
KVM_CAP_MAX_VCPU_IDis posted in V2 which is revised on top of kvm/queue.
([PATCH v2] kvm: selftests: AddKVM_CAP_MAX_VCPU_IDcap test - Zeng
Guang (kernel.org) <https://lore.kernel.org/lkml/[email protected]/>)Queued, thanks.
Shouldn't we have a solution for the read-only APIC_ID mess before this is merged?
Quoted 9 lines · Sean Christopherson
Quoted 8 lines · Paolo Bonzini
Quoted 3 lines · Zeng Guang
I don't see "[PATCH v9 4/9] KVM: VMX: Report
tertiary_exec_controlfield indump_vmcs()" in kvm/queue. Does it not need ?Added now (somehow the patches were not threaded, so I had to catch them one by one from lore).
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Selftests for
KVM_CAP_MAX_VCPU_IDis posted in V2 which is revised on top of kvm/queue.
([PATCH v2] kvm: selftests: AddKVM_CAP_MAX_VCPU_IDcap test - Zeng
Guang (kernel.org) <https://lore.kernel.org/lkml/[email protected]/>)Queued, thanks.
Shouldn't we have a solution for the read-only
APIC_IDmess before this is merged?
We can add a new inhibit to disable APICv if guest attempts to change APIC ID when IPIv (or AVIC) is enabled. Maxim also thinks using a new inhibit is the right direction [1].
If no objection to this approach and Maxim doesn't have the patch, we can post one. But we will rely on Maxim to fix APIC ID mess for nested AVIC.
+ Maxim
Quoted 27 lines · Chao Gao
Quoted 9 lines · Sean Christopherson
Quoted 8 lines · Paolo Bonzini
Quoted 3 lines · Zeng Guang
I don't see "[PATCH v9 4/9] KVM: VMX: Report
tertiary_exec_controlfield indump_vmcs()" in kvm/queue. Does it not need ?Added now (somehow the patches were not threaded, so I had to catch them one by one from lore).
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Selftests for
KVM_CAP_MAX_VCPU_IDis posted in V2 which is revised on top of kvm/queue.
([PATCH v2] kvm: selftests: AddKVM_CAP_MAX_VCPU_IDcap test - Zeng
Guang (kernel.org) <https://lore.kernel.org/lkml/[email protected]/>)Queued, thanks.
Shouldn't we have a solution for the read-only
APIC_IDmess before this is merged?We can add a new inhibit to disable APICv if guest attempts to change APIC ID when IPIv (or AVIC) is enabled. Maxim also thinks using a new inhibit is the right direction [1].
If no objection to this approach and Maxim doesn't have the patch, we can post one. But we will rely on Maxim to fix APIC ID mess for nested AVIC.
Quoted 21 lines · Chao Gao
+ Maxim
Quoted 18 lines · Chao Gao
Quoted 17 lines · Sean Christopherson
Quoted 8 lines · Paolo Bonzini
Quoted 3 lines · Zeng Guang
I don't see "[PATCH v9 4/9] KVM: VMX: Report
tertiary_exec_controlfield indump_vmcs()" in kvm/queue. Does it not need ?Added now (somehow the patches were not threaded, so I had to catch them one by one from lore).
Quoted 6 lines
Quoted 4 lines
Selftests for
KVM_CAP_MAX_VCPU_IDis posted in V2 which is revised on top of kvm/queue.
([PATCH v2] kvm: selftests: AddKVM_CAP_MAX_VCPU_IDcap test - Zeng
Guang (kernel.org) <https://lore.kernel.org/lkml/[email protected]/>)Queued, thanks.
Shouldn't we have a solution for the read-only
APIC_IDmess before this is merged?
Paolo & Sean,
If a solution for read-only APIC ID mess is needed before merging IPIv series, do you think the Maxim's patch [1] after some improvement will suffice? Let us know if there is any gap.
Sean Christopherson
reviewerQuoted 4 lines · Chao Gao
Quoted 3 lines · Chao Gao
+ Maxim
On Tue, May 17, 2022 at 09:53:26PM +0800, Chao Gao wrote:
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines · Sean Christopherson
Quoted 1 line
Shouldn't we have a solution for the read-only
APIC_IDmess before this is merged?Paolo & Sean,
If a solution for read-only APIC ID mess is needed before merging IPIv series, do you think the Maxim's patch [1] after some improvement will suffice? Let us know if there is any gap.
Yep, inhibiting APICv if APIC ID is changed should do the trick, and it's nice and simple. I can't think of any gaps.