KVM: VMX: Handle the immediate form of MSR instructions
Sean Christopherson
maintainerv3 cover KVM: VMX: Handle the immediate form of MSR instructions
On behalf of Xin, to avoid having to resolve conflicts when applying. This applies on the fastpath cleanup series:
https://lore.kernel.org/all/[email protected]
This patch set handles two newly introduced VM exit reasons associated with the immediate form of MSR instructions to ensure proper virtualization of these instructions.
The immediate form of MSR access instructions are primarily motivated by performance, not code size: by having the MSR number in an immediate, it is available *much* earlier in the pipeline, which allows the hardware much more leeway about how a particular MSR is handled.
For proper virtualization of the immediate form of MSR instructions, Intel VMX architecture adds the following changes:
1) The immediate form of RDMSR uses VM exit reason 84.
2) The immediate form of WRMSRNS uses VM exit reason 85.
3) For both VM exit reasons 84 and 85, the exit qualification is set to the MSR address causing the VM exit.
4) Bits 3 ~ 6 of the VM exit instruction information field represent the operand register used in the immediate form of MSR instruction.
5) The VM-exit instruction length field records the size of the immediate form of the MSR instruction.
Note: The VMX specification for the immediate form of MSR instructions was inadvertently omitted from the last published ISE, but it will be included in the upcoming edition.
Linux bare metal support of the immediate form of MSR instructions is still under development; however, the KVM support effort is proceeding independently of the bare metal implementation.
v3:
- Rebase on the fastpath cleanups.
- Split patches to better isolate the functional changes.
- Massage and expand on a changelogs.
- Make a handful of (mostly) stylistic changes (shouldn't affect functionality, key word "should").
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/lkml/[email protected]
Sean Christopherson (1):
KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as
appropriate
Xin Li (5):
x86/cpufeatures: Add a CPU feature bit for MSR immediate form
instructions
KVM: x86: Rename handle_fastpath_set_msr_irqoff() to
handle_fastpath_wrmsr()
KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
KVM: VMX: Support the immediate form of WRMSRNS in the VM-Exit
fastpath
KVM: x86: Advertise support for the immediate form of MSR instructions
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/kvm_host.h | 4 ++ arch/x86/include/uapi/asm/vmx.h | 6 +- arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kvm/cpuid.c | 6 +- arch/x86/kvm/reverse_cpuid.h | 5 ++ arch/x86/kvm/svm/svm.c | 8 ++- arch/x86/kvm/vmx/nested.c | 13 ++++- arch/x86/kvm/vmx/vmx.c | 26 ++++++++- arch/x86/kvm/vmx/vmx.h | 5 ++ arch/x86/kvm/x86.c | 94 ++++++++++++++++++++++-------- arch/x86/kvm/x86.h | 3 +- 12 files changed, 139 insertions(+), 33 deletions(-)
base-commit: 53d61a43a7973f812caa08fa922b607574befef4
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Sean Christopherson
maintainerPATCH v3 1/6 x86/cpufeatures: Add a CPU feature bit for MSR immediate form instructions
From: Xin Li <[email protected]>
The immediate form of MSR access instructions are primarily motivated by performance, not code size: by having the MSR number in an immediate, it is available *much* earlier in the pipeline, which allows the hardware much more leeway about how a particular MSR is handled.
Use a scattered CPU feature bit for MSR immediate form instructions.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -491,6 +491,7 @@#define X86_FEATURE_TSA_SQ_NO (21*32+11)/* AMD CPU not vulnerable to TSA-SQ */ #define X86_FEATURE_TSA_L1_NO (21*32+12)/* AMD CPU not vulnerable to TSA-L1 */ #define X86_FEATURE_CLEAR_CPU_BUF_VM (21*32+13)/* Clear CPU buffers using VERW before VMRUN */ +#define X86_FEATURE_MSR_IMM (21*32+14)/* MSR immediate form instructions */ /* * BUG word(s)
arch/x86/kernel/cpu/scattered.c +1 -0
@@ -27,6 +27,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_APERFMPERF, CPUID_ECX, 0, 0x00000006, 0 }, { X86_FEATURE_EPB, CPUID_ECX, 3, 0x00000006, 0 }, { X86_FEATURE_INTEL_PPIN, CPUID_EBX, 0, 0x00000007, 1 }, + { X86_FEATURE_MSR_IMM, CPUID_ECX, 5, 0x00000007, 1 }, { X86_FEATURE_APX, CPUID_EDX, 21, 0x00000007, 1 }, { X86_FEATURE_RRSBA_CTRL, CPUID_EDX, 2, 0x00000007, 2 }, { X86_FEATURE_BHI_CTRL, CPUID_EDX, 4, 0x00000007, 2 },
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Sean Christopherson
maintainerPATCH v3 2/6 KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as appropriate
In reply to Sean ChristophersonRename "ecx" variables in {RD,WR}MSR and RDPMC helpers to "msr" and "pmc" respectively, in anticipation of adding support for the immediate variants of RDMSR and WRMSRNS, and to better document what the variables hold (versus where the data originated).
No functional change intended.
--- arch/x86/kvm/x86.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
arch/x86/kvm/x86.c +12 -12
@@ -1572,10 +1572,10 @@ EXPORT_SYMBOL_GPL(kvm_get_dr);int kvm_emulate_rdpmc (struct kvm_vcpu *vcpu) { -u32 ecx =kvm_rcx_read (vcpu); +u32 pmc =kvm_rcx_read (vcpu);u64 data; -if (kvm_pmu_rdpmc (vcpu, ecx, &data)) { +if (kvm_pmu_rdpmc (vcpu, pmc, &data)) {kvm_inject_gp (vcpu,0 );return 1 ; }
@@ -2026,23 +2026,23 @@ static int kvm_msr_user_space(struct kvm_vcpu *vcpu, u32 index,int kvm_emulate_rdmsr (struct kvm_vcpu *vcpu) { -u32 ecx =kvm_rcx_read (vcpu); +u32 msr =kvm_rcx_read (vcpu);u64 data;int r; - r =kvm_get_msr_with_filter (vcpu, ecx, &data); + r =kvm_get_msr_with_filter (vcpu, msr, &data);if (!r) { -trace_kvm_msr_read (ecx, data); +trace_kvm_msr_read (msr, data);kvm_rax_write (vcpu, data &-1u );kvm_rdx_write (vcpu, (data >>32 ) &-1u ); }else {/* MSR read failed? See if we should ask user space */ -if (kvm_msr_user_space (vcpu, ecx,KVM_EXIT_X86_RDMSR ,0 , +if (kvm_msr_user_space (vcpu, msr,KVM_EXIT_X86_RDMSR ,0 , complete_fast_rdmsr, r))return 0 ; -trace_kvm_msr_read_ex (ecx); +trace_kvm_msr_read_ex (msr); }return kvm_x86_call (complete_emulated_msr)(vcpu, r);
@@ -2051,23 +2051,23 @@ EXPORT_SYMBOL_GPL(kvm_emulate_rdmsr);int kvm_emulate_wrmsr (struct kvm_vcpu *vcpu) { -u32 ecx =kvm_rcx_read (vcpu); +u32 msr =kvm_rcx_read (vcpu);u64 data =kvm_read_edx_eax (vcpu);int r; - r =kvm_set_msr_with_filter (vcpu, ecx, data); + r =kvm_set_msr_with_filter (vcpu, msr, data);if (!r) { -trace_kvm_msr_write (ecx, data); +trace_kvm_msr_write (msr, data); }else {/* MSR write failed? See if we should ask user space */ -if (kvm_msr_user_space (vcpu, ecx,KVM_EXIT_X86_WRMSR , data, +if (kvm_msr_user_space (vcpu, msr,KVM_EXIT_X86_WRMSR , data, complete_fast_msr_access, r))return 0 ;/* Signal all other negative errors to userspace */ if (r <0 )return r; -trace_kvm_msr_write_ex (ecx, data); +trace_kvm_msr_write_ex (msr, data); }return kvm_x86_call (complete_emulated_msr)(vcpu, r);
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Re: [PATCH v3 2/6] KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as appropriate
Quoted 8 lines · Sean Christopherson
Rename "ecx" variables in {RD,WR}MSR and RDPMC helpers to "msr" and "pmc" respectively, in anticipation of adding support for the immediate variants of RDMSR and WRMSRNS, and to better document what the variables hold (versus where the data originated).
No functional change intended.
Signed-off-by Sean Christopherson<[email protected]>
Yeah, this makes the immediate MSR handling changes in the following patches prominent.
Sean Christopherson
maintainerPATCH v3 3/6 KVM: x86: Rename handle_fastpath_set_msr_irqoff() to handle_fastpath_wrmsr()
In reply to Sean ChristophersonFrom: Xin Li <[email protected]>
Rename the WRMSR fastpath API to drop "irqoff", as that information is redundant (the fastpath always runs with IRQs disabled), and to prepare for adding a fastpath for the immediate variant of WRMSRNS.
No functional change intended.
--- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/x86.c | 4 ++-- arch/x86/kvm/x86.h | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/svm/svm.c +1 -1
@@ -4197,7 +4197,7 @@ static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu)case SVM_EXIT_MSR :if (!control->exit_info_1)break ; -return handle_fastpath_set_msr_irqoff (vcpu); +return handle_fastpath_wrmsr (vcpu);case SVM_EXIT_HLT :return handle_fastpath_hlt (vcpu);case SVM_EXIT_INVD :
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -7170,7 +7170,7 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu,switch (vmx_get_exit_reason (vcpu).basic) {case EXIT_REASON_MSR_WRITE : -return handle_fastpath_set_msr_irqoff (vcpu); +return handle_fastpath_wrmsr (vcpu);case EXIT_REASON_PREEMPTION_TIMER :return handle_fastpath_preemption_timer (vcpu, force_immediate_exit);case EXIT_REASON_HLT :
arch/x86/kvm/x86.c +2 -2
@@ -2142,7 +2142,7 @@ static inline bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu)kvm_request_pending (vcpu) ||xfer_to_guest_mode_work_pending (); } -fastpath_t handle_fastpath_set_msr_irqoff (struct kvm_vcpu *vcpu) +fastpath_t handle_fastpath_wrmsr (struct kvm_vcpu *vcpu) {u64 data =kvm_read_edx_eax (vcpu);u32 msr =kvm_rcx_read (vcpu);
@@ -2167,7 +2167,7 @@ fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu)return EXIT_FASTPATH_REENTER_GUEST ; } -EXPORT_SYMBOL_GPL (handle_fastpath_set_msr_irqoff); +EXPORT_SYMBOL_GPL (handle_fastpath_wrmsr);/* * Adapt set_msr() to msr_io()'s calling convention
arch/x86/kvm/x86.h +1 -1
@@ -437,7 +437,7 @@ int x86_decode_emulated_instruction(struct kvm_vcpu *vcpu, int emulation_type, void *insn, int insn_len);int x86_emulate_instruction (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa,int emulation_type,void *insn,int insn_len); -fastpath_t handle_fastpath_set_msr_irqoff (struct kvm_vcpu *vcpu); +fastpath_t handle_fastpath_wrmsr (struct kvm_vcpu *vcpu);fastpath_t handle_fastpath_hlt (struct kvm_vcpu *vcpu);fastpath_t handle_fastpath_invd (struct kvm_vcpu *vcpu);
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Re: [PATCH v3 3/6] KVM: x86: Rename handle_fastpath_set_msr_irqoff() to handle_fastpath_wrmsr()
Quoted 8 lines · Sean Christopherson
From: Xin Li <[email protected]>
Rename the WRMSR fastpath API to drop "irqoff", as that information is redundant (the fastpath always runs with IRQs disabled), and to prepare for adding a fastpath for the immediate variant of WRMSRNS.
No functional change intended.
I think it makes sense to add:
I didn't add it in v2 because the changes are part of a patch that handles the immediate form MSR instructions.
Quoted 3 lines
Signed-off-by Xin Li (Intel) <[email protected]>[sean: split to separate patch, write changelog]Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v3 4/6 KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
In reply to Sean ChristophersonFrom: Xin Li <[email protected]>
Add support for the immediate forms of RDMSR and WRMSRNS (currently Intel-only). The immediate variants are only valid in 64-bit mode, and use a single general purpose register for the data (the register is also encoded in the instruction, i.e. not implicit like regular RDMSR/WRMSR).
The immediate variants are primarily motivated by performance, not code
size: by having the MSR index in an immediate, it is available *much* earlier in the CPU pipeline, which allows hardware much more leeway about how a particular MSR is handled.
Intel VMX support for the immediate forms of MSR accesses communicates exit information to the host as follows:
1) The immediate form of RDMSR uses VM-Exit Reason 84.
2) The immediate form of WRMSRNS uses VM-Exit Reason 85.
3) For both VM-Exit reasons 84 and 85, the Exit Qualification field is set to the MSR index that triggered the VM-Exit.
4) Bits 3 ~ 6 of the VM-Exit Instruction Information field are set to the register encoding used by the immediate form of the instruction, i.e. the destination register for RDMSR, and the source for WRMSRNS.
5) The VM-Exit Instruction Length field records the size of the immediate form of the MSR instruction.
To deal with userspace RDMSR exits, stash the destination register in a new kvm_vcpu_arch field, similar to cui_linear_rip, pio, etc. Alternatively, the register could be saved in kvm_run.msr or re-retrieved from the VMCS, but the former would require sanitizing the value to ensure userspace doesn't clobber the value to an out-of-bounds index, and the latter would require a new one-off kvm_x86_ops hook.
Don't bother adding support for the instructions in KVM's emulator, as the only way for RDMSR/WRMSR to be encountered is if KVM is emulating large swaths of code due to invalid guest state, and a vCPU cannot have invalid guest state while in 64-bit mode.
--- arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/include/uapi/asm/vmx.h | 6 +++- arch/x86/kvm/vmx/nested.c | 13 ++++++-- arch/x86/kvm/vmx/vmx.c | 21 +++++++++++++ arch/x86/kvm/vmx/vmx.h | 5 +++ arch/x86/kvm/x86.c | 55 +++++++++++++++++++++++++++------ 6 files changed, 90 insertions(+), 13 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -0
@@ -929,6 +929,7 @@ struct kvm_vcpu_arch {bool emulate_regs_need_sync_from_vcpu;int (*complete_userspace_io)(struct kvm_vcpu *vcpu);unsigned long cui_linear_rip; +int cui_rdmsr_imm_reg;gpa_t time;s8 pvclock_tsc_shift;
@@ -2158,7 +2159,9 @@ int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data, bool host_initiatint kvm_get_msr (struct kvm_vcpu *vcpu,u32 index,u64 *data);int kvm_set_msr (struct kvm_vcpu *vcpu,u32 index,u64 data);int kvm_emulate_rdmsr (struct kvm_vcpu *vcpu); +int kvm_emulate_rdmsr_imm (struct kvm_vcpu *vcpu,u32 msr,int reg);int kvm_emulate_wrmsr (struct kvm_vcpu *vcpu); +int kvm_emulate_wrmsr_imm (struct kvm_vcpu *vcpu,u32 msr,int reg);int kvm_emulate_as_nop (struct kvm_vcpu *vcpu);int kvm_emulate_invd (struct kvm_vcpu *vcpu);int kvm_emulate_mwait (struct kvm_vcpu *vcpu);
arch/x86/include/uapi/asm/vmx.h +5 -1
@@ -94,6 +94,8 @@#define EXIT_REASON_BUS_LOCK 74#define EXIT_REASON_NOTIFY 75#define EXIT_REASON_TDCALL 77 +#define EXIT_REASON_MSR_READ_IMM 84 +#define EXIT_REASON_MSR_WRITE_IMM 85#define VMX_EXIT_REASONS \ { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \
@@ -158,7 +160,9 @@ { EXIT_REASON_TPAUSE, "TPAUSE" }, \ { EXIT_REASON_BUS_LOCK, "BUS_LOCK" }, \ { EXIT_REASON_NOTIFY, "NOTIFY" }, \ - { EXIT_REASON_TDCALL, "TDCALL" } + { EXIT_REASON_TDCALL, "TDCALL" }, \ + { EXIT_REASON_MSR_READ_IMM, "MSR_READ_IMM" }, \ + { EXIT_REASON_MSR_WRITE_IMM, "MSR_WRITE_IMM" } #define VMX_EXIT_REASON_FLAGS \ { VMX_EXIT_REASONS_FAILED_VMENTRY, "FAILED_VMENTRY" }
arch/x86/kvm/vmx/nested.c +11 -2
@@ -6216,19 +6216,26 @@ static bool nested_vmx_exit_handled_msr(struct kvm_vcpu *vcpu,struct vmcs12 *vmcs12, union vmx_exit_reason exit_reason) { - u32 msr_index = kvm_rcx_read(vcpu); + u32 msr_index;gpa_t bitmap;if (!nested_cpu_has (vmcs12,CPU_BASED_USE_MSR_BITMAPS ))return true; +if (exit_reason.basic ==EXIT_REASON_MSR_READ_IMM || + exit_reason.basic ==EXIT_REASON_MSR_WRITE_IMM ) + msr_index =vmx_get_exit_qual (vcpu); +else + msr_index =kvm_rcx_read (vcpu); +/* * The MSR_BITMAP page is divided into four 1024-byte bitmaps, * for the four combinations of read/write and low/high MSR numbers. * First we need to figure out which of the four to use: */ bitmap = vmcs12->msr_bitmap; -if (exit_reason.basic ==EXIT_REASON_MSR_WRITE ) +if (exit_reason.basic ==EXIT_REASON_MSR_WRITE || + exit_reason.basic ==EXIT_REASON_MSR_WRITE_IMM ) bitmap +=2048 ;if (msr_index >=0xc0000000 ) { msr_index -=0xc0000000 ;
@@ -6527,6 +6534,8 @@ static bool nested_vmx_l1_wants_exit(struct kvm_vcpu *vcpu,return nested_cpu_has2 (vmcs12,SECONDARY_EXEC_DESC );case EXIT_REASON_MSR_READ :case EXIT_REASON_MSR_WRITE : +case EXIT_REASON_MSR_READ_IMM : +case EXIT_REASON_MSR_WRITE_IMM :return nested_vmx_exit_handled_msr (vcpu, vmcs12, exit_reason);case EXIT_REASON_INVALID_STATE :return true;
arch/x86/kvm/vmx/vmx.c +21 -0
@@ -6003,6 +6003,23 @@ static int handle_notify(struct kvm_vcpu *vcpu)return 1 ; } +static int vmx_get_msr_imm_reg (struct kvm_vcpu *vcpu) +{ +return vmx_get_instr_info_reg (vmcs_read32 (VMX_INSTRUCTION_INFO )); +} + +static int handle_rdmsr_imm (struct kvm_vcpu *vcpu) +{ +return kvm_emulate_rdmsr_imm (vcpu,vmx_get_exit_qual (vcpu), +vmx_get_msr_imm_reg (vcpu)); +} + +static int handle_wrmsr_imm (struct kvm_vcpu *vcpu) +{ +return kvm_emulate_wrmsr_imm (vcpu,vmx_get_exit_qual (vcpu), +vmx_get_msr_imm_reg (vcpu)); +} +/* * The exit handlers return 1 if the exit was handled fully and guest execution * may resume. Otherwise they set the kvm_run parameter to indicate what needs
@@ -6061,6 +6078,8 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_ENCLS] = handle_encls, [EXIT_REASON_BUS_LOCK] = handle_bus_lock_vmexit, [EXIT_REASON_NOTIFY] = handle_notify, + [EXIT_REASON_MSR_READ_IMM] = handle_rdmsr_imm, + [EXIT_REASON_MSR_WRITE_IMM] = handle_wrmsr_imm, }; static const int kvm_vmx_max_exit_handlers =
@@ -6495,6 +6514,8 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)#ifdef CONFIG_MITIGATION_RETPOLINE if (exit_reason.basic ==EXIT_REASON_MSR_WRITE )return kvm_emulate_wrmsr (vcpu); +else if (exit_reason.basic ==EXIT_REASON_MSR_WRITE_IMM ) +return handle_wrmsr_imm (vcpu);else if (exit_reason.basic ==EXIT_REASON_PREEMPTION_TIMER )return handle_preemption_timer (vcpu);else if (exit_reason.basic ==EXIT_REASON_INTERRUPT_WINDOW )
arch/x86/kvm/vmx/vmx.h +5 -0
@@ -706,6 +706,11 @@ static inline bool vmx_guest_state_valid(struct kvm_vcpu *vcpu)void dump_vmcs (struct kvm_vcpu *vcpu); +static inline int vmx_get_instr_info_reg (u32 vmx_instr_info) +{ +return (vmx_instr_info >>3 ) &0xf ; +} +static inline int vmx_get_instr_info_reg2 (u32 vmx_instr_info) {return (vmx_instr_info >>28 ) &0xf ;
arch/x86/kvm/x86.c +45 -10
98 lines not on this page · raw message
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
arch/x86/kvm/vmx/vmx.c static int handle_notify(struct kvm_vcpu *vcpu) · line 6021
Quoted 11 lines · Sean Christopherson
arch/x86/kvm/vmx/vmx.c +11
-0+static int handle_rdmsr_imm (struct kvm_vcpu *vcpu) +{ +return kvm_emulate_rdmsr_imm (vcpu,vmx_get_exit_qual (vcpu), +vmx_get_msr_imm_reg (vcpu)); +} + +static int handle_wrmsr_imm (struct kvm_vcpu *vcpu) +{ +return kvm_emulate_wrmsr_imm (vcpu,vmx_get_exit_qual (vcpu), +vmx_get_msr_imm_reg (vcpu)); +}
We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
Quoted 15 lines · Xiaoyao Li
arch/x86/kvm/vmx/vmx.c static int handle_notify(struct kvm_vcpu *vcpu) · line 6021
Quoted 11 lines · Sean Christopherson
arch/x86/kvm/vmx/vmx.c +11
-0+static int handle_rdmsr_imm (struct kvm_vcpu *vcpu) +{ + return kvm_emulate_rdmsr_imm(vcpu, vmx_get_exit_qual(vcpu), + vmx_get_msr_imm_reg(vcpu)); +} + +static int handle_wrmsr_imm (struct kvm_vcpu *vcpu) +{ + return kvm_emulate_wrmsr_imm(vcpu, vmx_get_exit_qual(vcpu), + vmx_get_msr_imm_reg(vcpu)); +}We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Indeed.
There is a virtualization hole of this feature for the accesses to the MSRs not intercepted. IIUIC, there is no other control in VMX for this feature. If the feature is supported in hardware, the guest will succeed when it accesses to the MSRs not intercepted even when the feature is not exposed to the guest, but the guest will get #UD when access to the MSRs intercepted if KVM injects #UD.
But I guess this is the guest's fault by not following the CPUID, KVM should still follow the spec?
Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
Quoted 4 lines · Binbin Wu
Quoted 2 lines
We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Indeed.
Good catch!
Quoted 6 lines
There is a virtualization hole of this feature for the accesses to the MSRs not intercepted. IIUIC, there is no other control in VMX for this feature. If the feature is supported in hardware, the guest will succeed when it accesses to the MSRs not intercepted even when the feature is not exposed to the guest, but the guest will get #UD when access to the MSRs intercepted if KVM injects #UD.
hpa mentioned this when I just started the work. But I managed to forget it later... Sigh!
Quoted 3 lines
But I guess this is the guest's fault by not following the CPUID, KVM should still follow the spec?
I think we should still inject #UD when a MSR is intercepted by KVM.
Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
Quoted 15 lines · Xin Li
Quoted 4 lines · Binbin Wu
Quoted 2 lines
We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Indeed.
Good catch!
Quoted 6 lines
There is a virtualization hole of this feature for the accesses to the MSRs not intercepted. IIUIC, there is no other control in VMX for this feature. If the feature is supported in hardware, the guest will succeed when it accesses to the MSRs not intercepted even when the feature is not exposed to the guest, but the guest will get #UD when access to the MSRs intercepted if KVM injects #UD.
Quoted 9 lines
hpa mentioned this when I just started the work. But I managed to forget it later... Sigh!
Quoted 3 lines
But I guess this is the guest's fault by not following the CPUID, KVM should still follow the spec?
I think we should still inject #UD when a MSR is intercepted by KVM.
For handle_wrmsr_imm(), it seems we need to check guest_cpu_cap_has(X86_FEATURE_WRMSRNS) as well, since immediate form of MSR write is only supported on WRMSRNS instruction.
It leads to another topic, do we need to bother checking the opcode of the instruction on EXIT_REASON_MSR_WRITE and inject #UD when it is WRMSRNS instuction and !guest_cpu_cap_has(X86_FEATURE_WRMSRNS)?
WRMSRNS has virtualization hole as well, but KVM at least can emulate the architectural behavior when the write on MSRs are not pass through.
Sean Christopherson
maintainerRe: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Xin Li
Quoted 4 lines · Binbin Wu
Quoted 2 lines
We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Indeed.
Good catch!
Quoted 7 lines
There is a virtualization hole of this feature for the accesses to the MSRs not intercepted. IIUIC, there is no other control in VMX for this feature. If the feature is supported in hardware, the guest will succeed when it accesses to the MSRs not intercepted even when the feature is not exposed to the guest, but the guest will get #UD when access to the MSRs intercepted if KVM injects #UD.
hpa mentioned this when I just started the work. But I managed to forget it later... Sigh!
Quoted 1 line
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
But I guess this is the guest's fault by not following the CPUID, KVM should
still follow the spec?I think we should still inject #UD when a MSR is intercepted by KVM.
Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse than eating with the virtualization hole. Practically speaking, the only guest that's going to be surprised by the hole is a guest that's fuzzing opcodes, and a guest that's fuzzing opcodes at CPL0 isn't is going to create an inherently unstable environment no matter what.
Though that raises the question of whether or not KVM should emulate WRMSRNS and whatever the official name for the "RDMSR with immediate" instruction is (I can't find it in the SDM). I'm leaning "no", because outside of forced emulation, KVM should only "need" to emulate the instructions if Unrestricted Guest is disabled, the instructions should only be supported on CPUs with unrestricted guest, there's no sane reason (other than testing) to run a guest without Unrestricted Guest, and using the instructions in Big RM would be quite bizarre. On the other hand, adding emulation support should be quite easy...
Side topic, does RDMSRLIST have any VMX controls?
Quoted 10 lines
For
handle_wrmsr_imm(), it seems we need to checkguest_cpu_cap_has(X86_FEATURE_WRMSRNS)as well, since immediate form of MSR write is only supported on WRMSRNS instruction.It leads to another topic, do we need to bother checking the opcode of the instruction on
EXIT_REASON_MSR_WRITEand inject #UD when it is WRMSRNS instuction and !guest_cpu_cap_has(X86_FEATURE_WRMSRNS)?WRMSRNS has virtualization hole as well, but KVM at least can emulate the architectural behavior when the write on MSRs are not pass through.
Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
Quoted 26 lines · Sean Christopherson
Quoted 25 lines · Xiaoyao Li
Quoted 24 lines · Xin Li
Quoted 4 lines · Binbin Wu
Quoted 2 lines
We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Indeed.
Good catch!
Quoted 7 lines
There is a virtualization hole of this feature for the accesses to the MSRs not intercepted. IIUIC, there is no other control in VMX for this feature. If the feature is supported in hardware, the guest will succeed when it accesses to the MSRs not intercepted even when the feature is not exposed to the guest, but the guest will get #UD when access to the MSRs intercepted if KVM injects #UD.
hpa mentioned this when I just started the work. But I managed to forget it later... Sigh!
Quoted 4 lines
But I guess this is the guest's fault by not following the CPUID, KVM should
still follow the spec?
Quoted 7 lines
Quoted 1 line
Quoted 1 line
I think we should still inject #UD when a MSR is intercepted by KVM.
Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse than eating with the virtualization hole. Practically speaking, the only guest that's going to be surprised by the hole is a guest that's fuzzing opcodes, and a guest that's fuzzing opcodes at CPL0 isn't is going to create an inherently unstable environment no matter what.
Hmm, a malicious guest could *smartly* avoid causing such vmexits. So more or less that is wasteful and no objection here.
Quoted 11 lines
Though that raises the question of whether or not KVM should emulate WRMSRNS and whatever the official name for the "RDMSR with immediate" instruction is (I can't find it in the SDM). I'm leaning "no", because outside of forced emulation, KVM should only "need" to emulate the instructions if Unrestricted Guest is disabled, the instructions should only be supported on CPUs with unrestricted guest, there's no sane reason (other than testing) to run a guest without Unrestricted Guest, and using the instructions in Big RM would be quite bizarre. On the other hand, adding emulation support should be quite easy...
Side topic, does RDMSRLIST have any VMX controls?
Yes, bit 6 of Tertiary Processor-Based VM-Execution Controls, below is C&P from Intel SDM:
Enable MSR-list instructions control: if this control is 0, any execution of RDMSRLIST or WRMSRLIST causes a #UD.
If the control is 1, the instruction commences normally, writing one MSR at a time. Writes to certain MSRs are treated specially as described above for WRMSR and WRMSRNS. In addition, attempts to access specific MSRs may cause VM exits.
For RDMSRLIST and WRMSRLIST, the exit qualification depends on the setting of the “use MSR bitmaps” VM-execution control. If the control is 0, the exit qualification is zero. If the control is 1, the exit qualification is the index of the MSR whose access caused the VM exit.
Re: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
In reply to Sean ChristophersonQuoted 22 lines · Sean Christopherson
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Xin Li
Quoted 4 lines · Binbin Wu
Quoted 2 lines
We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Indeed.
Good catch!
Quoted 7 lines
There is a virtualization hole of this feature for the accesses to the MSRs not intercepted. IIUIC, there is no other control in VMX for this feature. If the feature is supported in hardware, the guest will succeed when it accesses to the MSRs not intercepted even when the feature is not exposed to the guest, but the guest will get #UD when access to the MSRs intercepted if KVM injects #UD.
hpa mentioned this when I just started the work. But I managed to forget it later... Sigh!
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
But I guess this is the guest's fault by not following the CPUID, KVM should
still follow the spec?I think we should still inject #UD when a MSR is intercepted by KVM.
Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse than eating with the virtualization hole.
Then could we document this design decision somewhere?
I believe people won't stop wondering why not inject #UD when no guest CPUID, when reading the code.
Quoted 8 lines
Practically speaking, the only guest
that's going to be surprised by the hole is a guest that's fuzzing opcodes, and a guest that's fuzzing opcodes at CPL0 isn't is going to create an inherently unstable environment no matter what.Though that raises the question of whether or not KVM should emulate WRMSRNS and whatever the official name for the "RDMSR with immediate" instruction is (I can't find it in the SDM).
do you mean because guest might be able to use immediate form of MSR access even if the CPUID doesn't advertise it, should KVM emulate it on platform doesn't support it, to make sure immediate form of MSR access is always supported?
Quoted 19 lines
I'm leaning "no", because outside of forced emulation, KVM
should only "need" to emulate the instructions if Unrestricted Guest is disabled, the instructions should only be supported on CPUs with unrestricted guest, there's no sane reason (other than testing) to run a guest without Unrestricted Guest, and using the instructions in Big RM would be quite bizarre. On the other hand, adding emulation support should be quite easy...Side topic, does RDMSRLIST have any VMX controls?
Quoted 10 lines
For
handle_wrmsr_imm(), it seems we need to checkguest_cpu_cap_has(X86_FEATURE_WRMSRNS)as well, since immediate form of MSR write is only supported on WRMSRNS instruction.It leads to another topic, do we need to bother checking the opcode of the instruction on
EXIT_REASON_MSR_WRITEand inject #UD when it is WRMSRNS instuction and !guest_cpu_cap_has(X86_FEATURE_WRMSRNS)?WRMSRNS has virtualization hole as well, but KVM at least can emulate the architectural behavior when the write on MSRs are not pass through.
Sean Christopherson
maintainerRe: [PATCH v3 4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel
Quoted 27 lines · Xiaoyao Li
Quoted 26 lines · Sean Christopherson
Quoted 25 lines · Xiaoyao Li
Quoted 24 lines · Xin Li
Quoted 4 lines · Binbin Wu
Quoted 2 lines
We need to inject #UD for !guest_cpu_has(X86_FEATURE_MSR_IMM)
Indeed.
Good catch!
Quoted 7 lines
There is a virtualization hole of this feature for the accesses to the MSRs not intercepted. IIUIC, there is no other control in VMX for this feature. If the feature is supported in hardware, the guest will succeed when it accesses to the MSRs not intercepted even when the feature is not exposed to the guest, but the guest will get #UD when access to the MSRs intercepted if KVM injects #UD.
hpa mentioned this when I just started the work. But I managed to forget it later... Sigh!
Quoted 4 lines
But I guess this is the guest's fault by not following the CPUID, KVM should
still follow the spec?
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
I think we should still inject #UD when a MSR is intercepted by KVM.
Hmm, no, inconsistent behavior (from the guest's perspective) is likely worse than eating with the virtualization hole.
Then could we document this design decision somewhere?
Yeah, Documentation/virt/kvm/x86/errata.rst would be the place for that.
Quoted 8 lines
I believe people won't stop wondering why not inject #UD when no guest CPUID, when reading the code.
Quoted 5 lines
Practically speaking, the only guest that's going to be surprised by the hole is a guest that's fuzzing opcodes, and a guest that's fuzzing opcodes at CPL0 isn't is going to create an inherently unstable environment no matter what.
Quoted 8 lines
Quoted 3 lines
Though that raises the question of whether or not KVM should emulate WRMSRNS and whatever the official name for the "RDMSR with immediate" instruction is (I can't find it in the SDM).
do you mean because guest might be able to use immediate form of MSR access even if the CPUID doesn't advertise it, should KVM emulate it on platform doesn't support it, to make sure immediate form of MSR access is always supported?
No, I'm calling out that as implemented, KVM doesn't support emulating WRMSRNS at all. Most of me just doesn't care. The only way to encounter WRMSRNS would be to disable unrestricted guest and execute WRMSRNS in an odd context, or to force emulation. Adding emulation support just for forced emulation is absurd, and adding WRMSRNS emulation for !URG is almost as ridiculous.
Sean Christopherson
maintainerPATCH v3 5/6 KVM: VMX: Support the immediate form of WRMSRNS in the VM-Exit fastpath
In reply to Sean ChristophersonFrom: Xin Li <[email protected]>
Add support for handling "WRMSRNS with an immediate" VM-Exits in KVM's fastpath. On Intel, all writes to the x2APIC ICR and to the TSC Deadline MSR are non-serializing, i.e. it's highly likely guest kernels will switch to using WRMSRNS when possible. And in general, any MSR written via WRMSRNS is probably worth handling in the fastpath, as the entire point of WRMSRNS is to shave cycles in hot paths.
--- arch/x86/kvm/vmx/vmx.c | 3 +++ arch/x86/kvm/x86.c | 17 +++++++++++++---- arch/x86/kvm/x86.h | 1 + 3 files changed, 17 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/vmx.c +3 -0
@@ -7192,6 +7192,9 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu,switch (vmx_get_exit_reason (vcpu).basic) {case EXIT_REASON_MSR_WRITE :return handle_fastpath_wrmsr (vcpu); +case EXIT_REASON_MSR_WRITE_IMM : +return handle_fastpath_wrmsr_imm (vcpu,vmx_get_exit_qual (vcpu), +vmx_get_msr_imm_reg (vcpu));case EXIT_REASON_PREEMPTION_TIMER :return handle_fastpath_preemption_timer (vcpu, force_immediate_exit);case EXIT_REASON_HLT :
arch/x86/kvm/x86.c +13 -4
@@ -2177,11 +2177,8 @@ static inline bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu)kvm_request_pending (vcpu) ||xfer_to_guest_mode_work_pending (); } -fastpath_t handle_fastpath_wrmsr (struct kvm_vcpu *vcpu) +static fastpath_t __handle_fastpath_wrmsr (struct kvm_vcpu *vcpu,u32 msr,u64 data) { -u64 data =kvm_read_edx_eax (vcpu); -u32 msr =kvm_rcx_read (vcpu); -switch (msr) {case APIC_BASE_MSR + (APIC_ICR >>4 ): if (!lapic_in_kernel(vcpu) || !apic_x2apic_mode(vcpu->arch.apic) ||
@@ -2202,8 +2199,20 @@ fastpath_t handle_fastpath_wrmsr(struct kvm_vcpu *vcpu)return EXIT_FASTPATH_REENTER_GUEST ; } + +fastpath_t handle_fastpath_wrmsr (struct kvm_vcpu *vcpu) +{ +return __handle_fastpath_wrmsr (vcpu,kvm_rcx_read (vcpu), +kvm_read_edx_eax (vcpu)); +}EXPORT_SYMBOL_GPL (handle_fastpath_wrmsr); +fastpath_t handle_fastpath_wrmsr_imm (struct kvm_vcpu *vcpu,u32 msr,int reg) +{ +return __handle_fastpath_wrmsr (vcpu, msr,kvm_register_read (vcpu, reg)); +} +EXPORT_SYMBOL_GPL (handle_fastpath_wrmsr_imm); +/* * Adapt set_msr() to msr_io()'s calling convention */
arch/x86/kvm/x86.h +1 -0
@@ -438,6 +438,7 @@ int x86_decode_emulated_instruction(struct kvm_vcpu *vcpu, int emulation_type,int x86_emulate_instruction (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa,int emulation_type,void *insn,int insn_len);fastpath_t handle_fastpath_wrmsr (struct kvm_vcpu *vcpu); +fastpath_t handle_fastpath_wrmsr_imm (struct kvm_vcpu *vcpu,u32 msr,int reg);fastpath_t handle_fastpath_hlt (struct kvm_vcpu *vcpu);fastpath_t handle_fastpath_invd (struct kvm_vcpu *vcpu);
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Sean Christopherson
maintainerPATCH v3 6/6 KVM: x86: Advertise support for the immediate form of MSR instructions
In reply to Sean ChristophersonFrom: Xin Li <[email protected]>
Advertise support for the immediate form of MSR instructions to userspace if the instructions are supported by the underlying CPU, and KVM is using VMX, i.e. is running on an Intel-compatible CPU.
For SVM, explicitly clear X86_FEATURE_MSR_IMM to ensure KVM doesn't over- report support if AMD-compatible CPUs ever implement the immediate forms, as SVM will likely require explicit enablement in KVM.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/cpuid.c | 6 +++++- arch/x86/kvm/reverse_cpuid.h | 5 +++++ arch/x86/kvm/svm/svm.c | 6 +++++- 4 files changed, 16 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -774,6 +774,7 @@ enum kvm_only_cpuid_leafs {CPUID_7_2_EDX ,CPUID_24_0_EBX ,CPUID_8000_0021_ECX , +CPUID_7_1_ECX ,NR_KVM_CPU_CAPS , NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS,
arch/x86/kvm/cpuid.c +5 -1
@@ -985,6 +985,10 @@ void kvm_set_cpu_caps(void)F (LAM ), ); +kvm_cpu_cap_init (CPUID_7_1_ECX , + SCATTERED_F(MSR_IMM), + ); + kvm_cpu_cap_init(CPUID_7_1_EDX, F(AVX_VNNI_INT8), F(AVX_NE_CONVERT),
@@ -1411,9 +1415,9 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) goto out;cpuid_entry_override (entry,CPUID_7_1_EAX ); +cpuid_entry_override (entry,CPUID_7_1_ECX );cpuid_entry_override (entry,CPUID_7_1_EDX ); entry->ebx =0 ; - entry->ecx =0 ; }if (max_idx >=2 ) { entry =do_host_cpuid (array, function,2 );
arch/x86/kvm/reverse_cpuid.h +5 -0
@@ -25,6 +25,9 @@#define KVM_X86_FEATURE_SGX2 KVM_X86_FEATURE(CPUID_12_EAX, 1)#define KVM_X86_FEATURE_SGX_EDECCSSA KVM_X86_FEATURE(CPUID_12_EAX, 11) +/* Intel-defined sub-features, CPUID level 0x00000007:1 (ECX) */ +#define KVM_X86_FEATURE_MSR_IMM KVM_X86_FEATURE(CPUID_7_1_ECX, 5) +/* Intel-defined sub-features, CPUID level 0x00000007:1 (EDX) */ #define X86_FEATURE_AVX_VNNI_INT8 KVM_X86_FEATURE(CPUID_7_1_EDX, 4)#define X86_FEATURE_AVX_NE_CONVERT KVM_X86_FEATURE(CPUID_7_1_EDX, 5)
@@ -87,6 +90,7 @@ static const struct cpuid_reg reverse_cpuid[] = { [CPUID_7_2_EDX] = { 7, 2, CPUID_EDX}, [CPUID_24_0_EBX] = { 0x24, 0, CPUID_EBX}, [CPUID_8000_0021_ECX] = {0x80000021, 0, CPUID_ECX}, + [CPUID_7_1_ECX] = { 7, 1, CPUID_ECX}, };/*
@@ -128,6 +132,7 @@ static __always_inline u32 __feature_translate(int x86_feature)KVM_X86_TRANSLATE_FEATURE (BHI_CTRL );KVM_X86_TRANSLATE_FEATURE (TSA_SQ_NO );KVM_X86_TRANSLATE_FEATURE (TSA_L1_NO ); +KVM_X86_TRANSLATE_FEATURE (MSR_IMM );default :return x86_feature; }
arch/x86/kvm/svm/svm.c +5 -1
@@ -5311,8 +5311,12 @@ static __init void svm_set_cpu_caps(void)/* CPUID 0x8000001F (SME/SEV features) */ sev_set_cpu_caps (); -/* Don't advertise Bus Lock Detect to guest if SVM support is absent */ +/* +* Clear capabilities that are automatically configured by common code, +* but that require explicit SVM support (that isn't yet implemented). +*/ kvm_cpu_cap_clear (X86_FEATURE_BUS_LOCK_DETECT ); +kvm_cpu_cap_clear (X86_FEATURE_MSR_IMM ); }static __init int svm_hardware_setup (void )
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Quoted 4 lines · Sean Christopherson
On behalf of Xin, to avoid having to resolve conflicts when applying. This applies on the fastpath cleanup series:
https://lore.kernel.org/all/[email protected]
Thanks for being considerate, and this is a much better use of time :)
Quoted 6 lines
v3:
- Rebase on the fastpath cleanups.
- Split patches to better isolate the functional changes.
- Massage and expand on a changelogs.
- Make a handful of (mostly) stylistic changes (shouldn't affect functionality, key word "should").
Tested after applying both patch sets, it works well.
Quoted 9 lines · Sean Christopherson
On behalf of Xin, to avoid having to resolve conflicts when applying. This applies on the fastpath cleanup series:
https://lore.kernel.org/all/[email protected]This patch set handles two newly introduced VM exit reasons associated with the immediate form of MSR instructions to ensure proper virtualization of these instructions.
[...]
Applied to kvm-x86 misc, thanks!
[1/6] x86/cpufeatures: Add a CPU feature bit for MSR immediate form instructions
https://github.com/kvm-x86/linux/commit/3c7cb8414533 [2/6] KVM: x86: Rename local "ecx" variables to "msr" and "pmc" as appropriate https://github.com/kvm-x86/linux/commit/ec400f6c2f27 [3/6] KVM: x86: Rename handle_fastpath_set_msr_irqoff() to handle_fastpath_wrmsr() https://github.com/kvm-x86/linux/commit/87a877de367d [4/6] KVM: x86: Add support for RDMSR/WRMSRNS w/ immediate on Intel https://github.com/kvm-x86/linux/commit/885df2d2109a [5/6] KVM: VMX: Support the immediate form of WRMSRNS in the VM-Exit fastpath https://github.com/kvm-x86/linux/commit/ec93675a3251 [6/6] KVM: x86: Advertise support for the immediate form of MSR instructions https://github.com/kvm-x86/linux/commit/d90ebf5a06ec