Intel SPEC CTRL virtualization support
Subsystems (4)
cover Intel SPEC CTRL virtualization support
This series introduces "virtual IA32_SPEC_CTRL" support. Here are introduction and use cases of this new feature.
### Virtual IA32_SPEC_CTRL
Virtual IA32_SPEC_CTRL [1] is a new VMX feature on Intel CPUs. This feature allows VMM to fix some bits of IA32_SPEC_CTRL MSR even when the MSR is pass-thru'd to a guest.
### Use cases of virtual IA32_SPEC_CTRL
Software mitigations like Retpoline and software BHB-clearing sequence depend on CPU microarchitectures. And guest cannot know exactly the underlying microarchitecture. When a guest is migrated between processors of different microarchitectures, software mitigations which work perfectly on previous microachitecture may be not effective on the new one. To fix the problem, some hardware mitigations should be used in conjunction with software mitigations. Using virtual IA32_SPEC_CTRL, VMM can enforce hardware mitigations transparently to guests and avoid those hardware mitigations being accidentally disabled when guest changes IA32_SPEC_CTRL MSR.
### Intention of this series
This series adds the capability of enforcing hardware mitigations for guests transparently and efficiently (i.e., without intecepting IA32_SPEC_CTRL MSR accesses) to kvm. The capability can be used to solve the VM migration issue in a pool consisting of processors of different microarchitectures.
Specifically, below are two target scenarios of this series:
Scenario 1: If retpoline is used by a VM to mitigate IMBTI in CPL0, VMM
can set RRSBA_DIS_S on parts enumerates RRSBA. Note that the VM is presented with a microarchitecture doesn't enumerate RRSBA.
Scenario 2: If a VM uses software BHB-clearing sequence on transitions
into CPL0 to mitigate BHI, VMM can use "virtual IA32_SPEC_CTRL" to set BHI_DIS_S on newer hardware which doesn't enumerate BHI_NO.
Intel defines some virtual MSRs for guests to notify VMM of software mitigations in use. Such information allows VMM to enable hardware mitigations only when necessary (i.e., VMs care about the vulnerability and are using software mitigiation) to minimize the performance impact to other VMs. The support of the virtual MSRs also added in this series.
### Organization of this series
This series is developed based on v6.1-rc7 (commit ef4d3ea40565a781c25847e9cb96c1bd9f462bc6)
1. Patch 1 introduce Intel SPEC_CTRL BHI related definition.
2. Patch 2/3 advertises {RRSBA/BHI}_CTRL for retpoline/BHB-clearing sequence mitigations for KVM.
3. Patch 4 introduce Intel virtual IA32_SPEC_CTRL
4. Patch 5-7 virtualize a set of virtual MSRs for guests to communicate software mitigations in use.
5. Patch 8-9 enable "virtual IA32_SPEC_CTRL" VMX feature with virtual MSRs support in KVM.
### Documentation
Refer to below link for more information:
https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/branch-history-injection.html
Pawan Gupta (1):
x86/bugs: Use Virtual MSRs to request hardware mitigations
Zhang Chen (8):
x86/speculation: Introduce Intel SPEC_CTRL BHI related definition
KVM: x86: Add a kvm-only leaf for RRSBA_CTRL
KVM: x86: Add a kvm-only leaf for BHI_CTRL x86/kvm/vmx: Virtualize Intel IA32_SPEC_CTRL
kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest
MSR_IA32_ARCH_CAPABILITIES
kvm/x86: Add MSR_VIRTUAL_MITIGATION_ENUM/CTRL emulation x86/kvm/vmx: Initialize SPEC_CTRL MASK for RRSBA x86/kvm/vmx: Initialize SPEC_CTRL MASK for BHI
arch/x86/include/asm/msr-index.h | 29 +++++++ arch/x86/include/asm/vmx.h | 5 ++ arch/x86/include/asm/vmxfeatures.h | 2 + arch/x86/kernel/cpu/bugs.c | 24 ++++++ arch/x86/kvm/cpuid.c | 4 + arch/x86/kvm/reverse_cpuid.h | 9 ++ arch/x86/kvm/vmx/capabilities.h | 5 ++ arch/x86/kvm/vmx/vmx.c | 109 ++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 27 +++++- arch/x86/kvm/x86.c | 23 +++++- tools/arch/x86/include/asm/msr-index.h | 6 ++ 11 files changed, 238 insertions(+), 5 deletions(-)
Signature · 1 line
2.25.1
RFC PATCH 1/9 x86/speculation: Introduce Intel SPEC_CTRL BHI related definition
Define BHI_NO bit and new control of BHI hardware mitigation in IA32_SPEC_CTRL. These definitions are used by following KVM patches to determine whether to enforce BHI hardware mitigiations for guests transparently.
BHI_NO means the processor isn't vulnernable to BHI attacks. BHI_DIS_S is a new indirect predictor control. Once enabled, BHI_DIS_S prevents predicted targets of indirect branches executed in CPL0/1/2 from being selected based on branch history from branches executed in CPL3. While set in the VMX root, it also prevents predicted targets executed in CPL0 from being selected based on branch history from branches executed in a VMX non-root.
Branch History Injection (BHI) describes a specific form of intra-mode BTI, where an attacker may manipulate branch history before transitioning from user to supervisor mode (or from VMX non-root/guest to root mode) in an effort to cause an indirect branch predictor to select a specific predictor entry for an indirect branch, and a disclosure gadget at the predicted target will transiently execute. This may be possible since the relevant branch history may contain branches taken in previous security contexts, and in particular, in other predictor modes.
Refer to below link for more information:
https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/branch-history-injection.html
--- arch/x86/include/asm/msr-index.h | 6 ++++++ tools/arch/x86/include/asm/msr-index.h | 6 ++++++ 2 files changed, 12 insertions(+)
arch/x86/include/asm/msr-index.h +6 -0
@@ -53,6 +53,8 @@#define SPEC_CTRL_SSBD BIT(SPEC_CTRL_SSBD_SHIFT)/* Speculative Store Bypass Disable */ #define SPEC_CTRL_RRSBA_DIS_S_SHIFT 6/* Disable RRSBA behavior */ #define SPEC_CTRL_RRSBA_DIS_S BIT(SPEC_CTRL_RRSBA_DIS_S_SHIFT) +#define SPEC_CTRL_BHI_DIS_S_SHIFT 10/* Enable BHI_DIS_S behavior */ +#define SPEC_CTRL_BHI_DIS_S BIT(SPEC_CTRL_BHI_DIS_S_SHIFT)#define MSR_IA32_PRED_CMD 0x00000049/* Prediction Command */ #define PRED_CMD_IBPB BIT(0)/* Indirect Branch Prediction Barrier */
@@ -150,6 +152,10 @@* are restricted to targets in * kernel. */ +#define ARCH_CAP_BHI_NO BIT(20)/* +* Not susceptible to Branch History +* Injection. +*/ #define ARCH_CAP_PBRSB_NO BIT(24)/* * Not susceptible to Post-Barrier * Return Stack Buffer Predictions.
tools/arch/x86/include/asm/msr-index.h +6 -0
@@ -53,6 +53,8 @@#define SPEC_CTRL_SSBD BIT(SPEC_CTRL_SSBD_SHIFT)/* Speculative Store Bypass Disable */ #define SPEC_CTRL_RRSBA_DIS_S_SHIFT 6/* Disable RRSBA behavior */ #define SPEC_CTRL_RRSBA_DIS_S BIT(SPEC_CTRL_RRSBA_DIS_S_SHIFT) +#define SPEC_CTRL_BHI_DIS_S_SHIFT 10/* Enable BHI_DIS_S behavior */ +#define SPEC_CTRL_BHI_DIS_S BIT(SPEC_CTRL_BHI_DIS_S_SHIFT)#define MSR_IA32_PRED_CMD 0x00000049/* Prediction Command */ #define PRED_CMD_IBPB BIT(0)/* Indirect Branch Prediction Barrier */
@@ -150,6 +152,10 @@* are restricted to targets in * kernel. */ +#define ARCH_CAP_BHI_NO BIT(20)/* +* Not susceptible to Branch History +* Injection. +*/ #define ARCH_CAP_PBRSB_NO BIT(24)/* * Not susceptible to Post-Barrier * Return Stack Buffer Predictions.
Signature · 1 line
2.25.1
RFC PATCH 2/9 KVM: x86: Add a kvm-only leaf for RRSBA_CTRL
In reply to Zhang ChenKVM needs to check if guests can see RRSBA_CTRL. If a guest is using retpoline and cannot see RRSBA_CTRL and the host enumerates RRSBA, KVM is responsible for setting RRSBA_DIS_S for the guest.
This allows VM migration from parts doesn't enumerates RRSBA to those that enumerate RRSBA.
--- arch/x86/kvm/cpuid.c | 4 ++++ arch/x86/kvm/reverse_cpuid.h | 7 +++++++ 2 files changed, 11 insertions(+)
arch/x86/kvm/cpuid.c +4 -0
@@ -668,6 +668,10 @@ void kvm_set_cpu_caps(void)SF (SGX1 ) |SF (SGX2 ) ); +kvm_cpu_cap_init_scattered (CPUID_7_2_EDX , +SF (RRSBA_CTRL ) + ); + kvm_cpu_cap_mask(CPUID_8000_0001_ECX, F(LAHF_LM) | F(CMP_LEGACY) | 0 /*SVM*/ | 0 /* ExtApicSpace */ | F(CR8_LEGACY) | F(ABM) | F(SSE4A) | F(MISALIGNSSE) |
arch/x86/kvm/reverse_cpuid.h +7 -0
@@ -13,6 +13,7 @@*/ enum kvm_only_cpuid_leafs {CPUID_12_EAX =NCAPINTS , +CPUID_7_2_EDX ,NR_KVM_CPU_CAPS ,NKVMCAPINTS =NR_KVM_CPU_CAPS -NCAPINTS ,
@@ -24,6 +25,9 @@ enum kvm_only_cpuid_leafs {#define KVM_X86_FEATURE_SGX1 KVM_X86_FEATURE(CPUID_12_EAX, 0)#define KVM_X86_FEATURE_SGX2 KVM_X86_FEATURE(CPUID_12_EAX, 1) +/* Intel-defined sub-features, CPUID level 0x00000007:2 (EDX)*/ +#define KVM_X86_FEATURE_RRSBA_CTRL KVM_X86_FEATURE(CPUID_7_2_EDX, 2) +struct cpuid_reg {u32 function;u32 index;
@@ -46,6 +50,7 @@ static const struct cpuid_reg reverse_cpuid[] = { [CPUID_8000_0007_EBX] = {0x80000007, 0, CPUID_EBX}, [CPUID_7_EDX] = { 7, 0, CPUID_EDX}, [CPUID_7_1_EAX] = { 7, 1, CPUID_EAX}, + [CPUID_7_2_EDX] = { 7, 2, CPUID_EDX}, [CPUID_12_EAX] = {0x00000012, 0, CPUID_EAX}, [CPUID_8000_001F_EAX] = {0x8000001f, 0, CPUID_EAX}, };
@@ -78,6 +83,8 @@ static __always_inline u32 __feature_translate(int x86_feature)return KVM_X86_FEATURE_SGX1 ; else if (x86_feature == X86_FEATURE_SGX2) return KVM_X86_FEATURE_SGX2; + else if (x86_feature == X86_FEATURE_RRSBA_CTRL) + return KVM_X86_FEATURE_RRSBA_CTRL;return x86_feature; }
Signature · 1 line
2.25.1
Re: [RFC PATCH 2/9] KVM: x86: Add a kvm-only leaf for RRSBA_CTRL
Quoted 5 lines · Zhang Chen
KVM needs to check if guests can see
RRSBA_CTRL. If a guest is using retpoline and cannot seeRRSBA_CTRLand the host enumerates RRSBA, KVM is responsible for settingRRSBA_DIS_Sfor the guest.This allows VM migration from parts doesn't enumerates RRSBA to those
perhaps s/parts doesn't enumerates/parts that do not enumerate/ reads better?
Quoted 1 line
that enumerate RRSBA.
and here maybe s/that enumerate RRSBA/do/ ?
Thanks and BR,
Ricardo
RE: [RFC PATCH 2/9] KVM: x86: Add a kvm-only leaf for RRSBA_CTRL
Quoted 16 lines
-----Original Message-----
From: Ricardo Neri <[email protected]>
Sent: Thursday, December 15, 2022 5:34 AM
To: Zhang, Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Gao, Chao <[email protected]>; Pawan Gupta <[email protected]>; Paolo Bonzini <[email protected]>; Christopherson,, Sean <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>
Subject: Re: [RFC PATCH 2/9] KVM: x86: Add a kvm-only leaf forRRSBA_CTRLOn Sun, Dec 11, 2022 at 12:00:39AM +0800, Zhang Chen wrote:
Quoted 8 lines
Quoted 5 lines
KVM needs to check if guests can see
RRSBA_CTRL. If a guest is using retpoline and cannot seeRRSBA_CTRLand the host enumerates RRSBA, KVM is responsible for settingRRSBA_DIS_Sfor the guest.This allows VM migration from parts doesn't enumerates RRSBA to those
perhaps s/parts doesn't enumerates/parts that do not enumerate/ reads better?
OK.
Quoted 4 lines
Quoted 2 lines
that enumerate RRSBA.
and here maybe s/that enumerate RRSBA/do/ ?
Looks good to me. Thanks for your comments.
I will fix patch2/3 in the next version.
Thanks
Chen
Quoted 3 lines
Thanks and BR,
Ricardo
RFC PATCH 3/9 KVM: x86: Add a kvm-only leaf for BHI_CTRL
In reply to Zhang ChenKVM needs to check if guests can see BHI_CTRL. If a guest is using BHB-clearing sequence and cannot see BHI_CTRL and the host enumerates BHI, KVM is responsible for setting BHI_DIS_S for the guest.
This allows VM migration from parts doesn't enumerate BHI to those that enumerate BHI.
--- arch/x86/kvm/cpuid.c | 2 +- arch/x86/kvm/reverse_cpuid.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +1 -1
@@ -669,7 +669,7 @@ void kvm_set_cpu_caps(void) );kvm_cpu_cap_init_scattered (CPUID_7_2_EDX , -SF (RRSBA_CTRL ) +SF (RRSBA_CTRL ) |F (BHI_CTRL ) ); kvm_cpu_cap_mask(CPUID_8000_0001_ECX,
arch/x86/kvm/reverse_cpuid.h +2 -0
@@ -27,6 +27,8 @@ enum kvm_only_cpuid_leafs {/* Intel-defined sub-features, CPUID level 0x00000007:2 (EDX)*/ #define KVM_X86_FEATURE_RRSBA_CTRL KVM_X86_FEATURE(CPUID_7_2_EDX, 2) +/* X86_FEATURE_BHI_CTRL only used by KVM */ +#define X86_FEATURE_BHI_CTRL KVM_X86_FEATURE(CPUID_7_2_EDX, 4)struct cpuid_reg {u32 function;
Signature · 1 line
2.25.1
Re: [RFC PATCH 3/9] KVM: x86: Add a kvm-only leaf for BHI_CTRL
Quoted 6 lines · Zhang Chen
KVM needs to check if guests can see
BHI_CTRL. If a guest is using BHB-clearing sequence and cannot seeBHI_CTRLand the host enumerates BHI, KVM is responsible for settingBHI_DIS_Sfor the guest.This allows VM migration from parts doesn't enumerate BHI to those that enumerate BHI.
Similar comment as in patch 2.
RFC PATCH 4/9 x86/kvm/vmx: Virtualize Intel IA32_SPEC_CTRL
In reply to Zhang ChenCurrently KVM disables interception of IA32_SPEC_CTRL after a non-0 is written to IA32_SPEC_CTRL by guest. Then, guest is allowed to write any value to hardware.
"virtualize IA32_SPEC_CTRL" is a new tertiary vm-exec control. It provides VMM the capability to restrict the value of IA32_SPEC_CTRL in hardware even if the MSR isn't intercepted (e.g., prevent VMs from changing some bits in IA32_SPEC_CTRL) in an efficient way.
Two new fields are added to VMCS,IA32_SPEC_CTRL_MASK: Setting a bit in this field prevents guest software from modifying the corresponding bit in the IA32_SPEC_CTRL MSRIA32_SPEC_CTRL_SHADOW: This field contains the value that guest software expects to be in the IA32_SPEC_CTRL MSR
In VMX non-root mode, when IA32_SPEC_CTRL isn't intercepted by VMM, guest's accesses to IA32_SPEC_CTRL is virtualized by the processor according to the two new fields:
RDMSR IA32_SPEC_CTRL returns the shadow value.
WRMSR IA32_SPEC_CTRL writes EDX:EAX to the shadow field and calculate a new value according to the guest value (EDX:EAX), current value of IA32_SPEC_CTRL MSR and the IA32_SPEC_CTRL_MASK field (specifically, (cur_val & mask) | (guest_val & ~mask)) and write it to IA32_SPEC_CTRL MSR.
Enable "virtual IA32_SPEC_CTRL" if it is supported. With "virtual IA32_SPEC_CTRL" enabled, IA32_SPEC_CTRL MSR value seen from guest p.o.v is different from the value in hardware when guest is running. We refer to the two values as below:
1. effective value of IA32_SPEC_CTRL. This value is the one programmed in hardware when the vCPU is running.
2. shadow value of IA32_SPEC_CTRL. This value is returned when rdmsr is used inside a guest to read IA32_SPEC_CTRL. This value doesn't affect CPU's enablement of indirect predictor controls.
In KVM, vmx->spec_ctrl always stores the effective value of IA32_SPEC_CTRL when guest is running (even when "virtual IA32_SPEC_CTRL" is disabled. In this case, the shadow value is equal to the effective one). When "virtual IA32_SPEC_CTRL" is enabled, the shadow value of IA32_SPEC_CTRL is stored in IA32_SPEC_CTRL_SHADOW field in VMCS.
IA32_SPEC_CTRL_MASK is always 0 for now. It means all supported bits in hardware is allowed to be toggled by guest's wrmsr. The mask will be changed by following patches.
Note "virtual IA32_SPEC_CTRL" is now used by VMM to enforce some bits of IA32_SPEC_CTRL to 1 (i.e., enabled some HW mitigations transparently for guests). In theory, VMM can disable some HW mitigations behind guests. But to keep this series simple, we leave that for future work.
--- arch/x86/include/asm/vmx.h | 5 +++++ arch/x86/include/asm/vmxfeatures.h | 2 ++ arch/x86/kvm/vmx/capabilities.h | 5 +++++ arch/x86/kvm/vmx/vmx.c | 30 ++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/vmx.h | 10 +++++++++- 5 files changed, 49 insertions(+), 3 deletions(-)
arch/x86/include/asm/vmx.h +5 -0
@@ -81,6 +81,7 @@* Definitions of Tertiary Processor-Based VM-Execution Controls. */ #define TERTIARY_EXEC_IPI_VIRT VMCS_CONTROL_BIT(IPI_VIRT) +#define TERTIARY_EXEC_VIRT_SPEC_CTRL VMCS_CONTROL_BIT(VIRT_SPEC_CTRL)#define PIN_BASED_EXT_INTR_MASK VMCS_CONTROL_BIT(INTR_EXITING)#define PIN_BASED_NMI_EXITING VMCS_CONTROL_BIT(NMI_EXITING)
@@ -233,6 +234,10 @@ enum vmcs_field {TERTIARY_VM_EXEC_CONTROL_HIGH =0x00002035 ,PID_POINTER_TABLE =0x00002042 ,PID_POINTER_TABLE_HIGH =0x00002043 , +IA32_SPEC_CTRL_MASK =0x0000204A , +IA32_SPEC_CTRL_MASK_HIGH =0x0000204B , +IA32_SPEC_CTRL_SHADOW =0x0000204C , +IA32_SPEC_CTRL_SHADOW_HIGH =0x0000204D ,GUEST_PHYSICAL_ADDRESS =0x00002400 ,GUEST_PHYSICAL_ADDRESS_HIGH =0x00002401 , VMCS_LINK_POINTER = 0x00002800,
arch/x86/include/asm/vmxfeatures.h +2 -0
@@ -89,4 +89,6 @@/* Tertiary Processor-Based VM-Execution Controls, word 3 */ #define VMX_FEATURE_IPI_VIRT ( 3*32+ 4)/* Enable IPI virtualization */ +#define VMX_FEATURE_VIRT_SPEC_CTRL ( 3*32+ 7)/* Virtualize IA32_SPEC_CTRL */ +#endif /* _ASM_X86_VMXFEATURES_H */
arch/x86/kvm/vmx/capabilities.h +5 -0
@@ -138,6 +138,11 @@ static inline bool cpu_has_tertiary_exec_ctrls(void)CPU_BASED_ACTIVATE_TERTIARY_CONTROLS ; } +static inline bool cpu_has_virt_spec_ctrl (void ) +{ +return vmcs_config.cpu_based_3rd_exec_ctrl &TERTIARY_EXEC_VIRT_SPEC_CTRL ; +} +static inline bool cpu_has_vmx_virtualize_apic_accesses (void ) { return vmcs_config.cpu_based_2nd_exec_ctrl &
arch/x86/kvm/vmx/vmx.c +28 -2
@@ -1898,7 +1898,10 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) !guest_has_spec_ctrl_msr(vcpu))return 1 ; - msr_info->data =to_vmx (vcpu)->spec_ctrl; +if (cpu_has_virt_spec_ctrl ()) + msr_info->data =vmcs_read64 (IA32_SPEC_CTRL_SHADOW ); +else + msr_info->data =to_vmx (vcpu)->spec_ctrl;break ;case MSR_IA32_SYSENTER_CS : msr_info->data =vmcs_read32 (GUEST_SYSENTER_CS );
@@ -2160,10 +2163,22 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) !guest_has_spec_ctrl_msr(vcpu))return 1 ; -if (kvm_spec_ctrl_test_value (data)) +if (kvm_spec_ctrl_test_value (data | vmx->spec_ctrl_mask))return 1 ; vmx->spec_ctrl = data; + +if (cpu_has_virt_spec_ctrl ()) { +vmcs_write64 (IA32_SPEC_CTRL_SHADOW , data); +/* +* Some bits are allowed to be toggled by guest. +* Update the effective value of IA32_SPEC_CTRL +* MSR according to the value written by guest +* but keep bits in the mask set. +*/ + vmx->spec_ctrl = data | vmx->spec_ctrl_mask; + } +if (!data)break ;
@@ -4673,6 +4688,11 @@ static void init_vmcs(struct vcpu_vmx *vmx)if (cpu_has_vmx_xsaves ())vmcs_write64 (XSS_EXIT_BITMAP ,VMX_XSS_EXIT_BITMAP ); +if (cpu_has_virt_spec_ctrl ()) { +vmcs_write64 (IA32_SPEC_CTRL_SHADOW ,0 ); +vmcs_write64 (IA32_SPEC_CTRL_MASK , vmx->spec_ctrl_mask); + } +if (enable_pml) {vmcs_write64 (PML_ADDRESS ,page_to_phys (vmx->pml_pg));vmcs_write16 (GUEST_PML_INDEX ,PML_ENTITY_NUM -1 );
@@ -4738,6 +4758,12 @@ static void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)__vmx_vcpu_reset (vcpu); vmx->rmode.vm86_active =0 ; + +if (cpu_has_virt_spec_ctrl ()) { + vmx->spec_ctrl_mask =0 ; +vmcs_write64 (IA32_SPEC_CTRL_MASK , vmx->spec_ctrl_mask); +vmcs_write64 (IA32_SPEC_CTRL_SHADOW ,0 ); + } vmx->spec_ctrl =0 ; vmx->msr_ia32_umwait_control =0 ;
arch/x86/kvm/vmx/vmx.h +9 -1
@@ -290,7 +290,14 @@ struct vcpu_vmx {u64 msr_guest_kernel_gs_base;#endif +/* The value of hardware IA32_SPEC_CTRL MSR when guest is running */ u64 spec_ctrl; +/* +* The bits KVM doesn't allow guest to toggle. +* A bit set in the mask should always be set in guest +* IA32_SPEC_CTRL_MSR. +*/ +u64 spec_ctrl_mask;u32 msr_ia32_umwait_control;/*
@@ -589,7 +596,8 @@ static inline u8 vmx_get_rvi(void)#define KVM_REQUIRED_VMX_TERTIARY_VM_EXEC_CONTROL 0#define KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL \ - (TERTIARY_EXEC_IPI_VIRT) + (TERTIARY_EXEC_IPI_VIRT | \ + TERTIARY_EXEC_VIRT_SPEC_CTRL)#define BUILD_CONTROLS_SHADOW (lname, uname, bits) \ static inline void lname##_controls_set(struct vcpu_vmx *vmx, u##bits val) \
Signature · 1 line
2.25.1
RFC PATCH 5/9 x86/bugs: Use Virtual MSRs to request hardware mitigations
In reply to Zhang ChenFrom: Pawan Gupta <[email protected]>
Guests that have different family/model than the host may not be aware of hardware mitigations(such as RRSBA_DIS_S) available on host. This is particularly true when guests migrate. To solve this problem Intel processors have added a virtual MSR interface through which guests can report their mitigation status and request VMM to deploy relevant hardware mitigations.
Use this virtualized MSR interface to request relevant hardware controls for retpoline mitigation.
--- arch/x86/include/asm/msr-index.h | 23 +++++++++++++++++++++++ arch/x86/kernel/cpu/bugs.c | 24 ++++++++++++++++++++++++ 2 files changed, 47 insertions(+)
arch/x86/include/asm/msr-index.h +23 -0
@@ -165,6 +165,7 @@* IA32_XAPIC_DISABLE_STATUS MSR * supported */ +#define ARCH_CAP_VIRTUAL_ENUM BIT(63)/* MSR_VIRTUAL_ENUMERATION supported */ #define MSR_IA32_FLUSH_CMD 0x0000010b#define L1D_FLUSH BIT(0)/*
@@ -1062,6 +1063,28 @@#define MSR_IA32_VMX_MISC_INTEL_PT (1ULL << 14)#define MSR_IA32_VMX_MISC_VMWRITE_SHADOW_RO_FIELDS (1ULL << 29)#define MSR_IA32_VMX_MISC_PREEMPTION_TIMER_SCALE 0x1F + +/* Intel virtual MSRs */ +#define MSR_VIRTUAL_ENUMERATION 0x50000000 +#define VIRT_ENUM_MITIGATION_CTRL_SUPPORT BIT(0)/* +* Mitigation ctrl via virtual +* MSRs supported +*/ + +#define MSR_VIRTUAL_MITIGATION_ENUM 0x50000001 +#define MITI_ENUM_BHB_CLEAR_SEQ_S_SUPPORT BIT(0)/* VMM supports BHI_DIS_S */ +#define MITI_ENUM_RETPOLINE_S_SUPPORT BIT(1)/* VMM supports RRSBA_DIS_S */ + +#define MSR_VIRTUAL_MITIGATION_CTRL 0x50000002 +#define MITI_CTRL_BHB_CLEAR_SEQ_S_USED BIT(0)/* +* Request VMM to deploy +* BHI_DIS_S mitigation +*/ +#define MITI_CTRL_RETPOLINE_S_USED BIT(1)/* +* Request VMM to deploy +* RRSBA_DIS_S mitigation +*/ +/* AMD-V MSRs */ #define MSR_VM_CR 0xc0010114
arch/x86/kernel/cpu/bugs.c +24 -0
@@ -1379,6 +1379,28 @@ static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_dump_stack (); } +/* Speculation control using virtualized MSRs */ +static void __init spec_ctrl_setup_virtualized_msr (void ) +{ +u64 msr_virt_enum, msr_mitigation_enum, msr_mitigation_ctrl; + +if (!(x86_read_arch_cap_msr () &ARCH_CAP_VIRTUAL_ENUM )) +return ; + +rdmsrl (MSR_VIRTUAL_ENUMERATION , msr_virt_enum); +if (!(msr_virt_enum &VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return ; + +rdmsrl (MSR_VIRTUAL_MITIGATION_ENUM , msr_mitigation_enum); +/* When retpoline is being used, request relevant hardware controls */ +if (boot_cpu_has (X86_FEATURE_RETPOLINE ) && + msr_mitigation_enum &MITI_ENUM_RETPOLINE_S_SUPPORT ) { +rdmsrl (MSR_VIRTUAL_MITIGATION_CTRL , msr_mitigation_ctrl); + msr_mitigation_ctrl |=MITI_CTRL_RETPOLINE_S_USED ; +wrmsrl (MSR_VIRTUAL_MITIGATION_CTRL , msr_mitigation_ctrl); + } +} +static void __init spectre_v2_select_mitigation (void ) {enum spectre_v2_mitigation_cmd cmd =spectre_v2_parse_cmdline ();
@@ -1485,6 +1507,8 @@ static void __init spectre_v2_select_mitigation(void) mode == SPECTRE_V2_RETPOLINE)spec_ctrl_disable_kernel_rrsba (); +spec_ctrl_setup_virtualized_msr (); + spectre_v2_enabled = mode;pr_info ("%s\n" , spectre_v2_strings[mode]);
Signature · 1 line
2.25.1
Pawan Gupta
reviewerRe: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
Quoted 93 lines · Zhang Chen
From: Pawan Gupta <[email protected]>
Guests that have different family/model than the host may not be aware of hardware
mitigations(such as RRSBA_DIS_S)available on host. This is particularly true when guests migrate. To solve this problem Intel processors have added a virtual MSR interface through which guests can report their mitigation status and request VMM to deploy relevant hardware mitigations.Use this virtualized MSR interface to request relevant hardware controls for retpoline mitigation.
Signed-off-by Pawan Gupta <[email protected]>--- arch/x86/include/asm/msr-index.h | 23 +++++++++++++++++++++++ arch/x86/kernel/cpu/bugs.c | 24 ++++++++++++++++++++++++ 2 files changed, 47 insertions(+)arch/x86/include/asm/msr-index.h +23
-0@@ -165,6 +165,7 @@* IA32_XAPIC_DISABLE_STATUS MSR * supported */ +#define ARCH_CAP_VIRTUAL_ENUM BIT(63)/* MSR_VIRTUAL_ENUMERATION supported */ #define MSR_IA32_FLUSH_CMD 0x0000010b#define L1D_FLUSH BIT(0)/* @@ -1062,6 +1063,28 @@#define MSR_IA32_VMX_MISC_INTEL_PT (1ULL << 14)#define MSR_IA32_VMX_MISC_VMWRITE_SHADOW_RO_FIELDS (1ULL << 29)#define MSR_IA32_VMX_MISC_PREEMPTION_TIMER_SCALE 0x1F + +/* Intel virtual MSRs */ +#define MSR_VIRTUAL_ENUMERATION 0x50000000 +#define VIRT_ENUM_MITIGATION_CTRL_SUPPORT BIT(0)/* +* Mitigation ctrl via virtual +* MSRs supported +*/ + +#define MSR_VIRTUAL_MITIGATION_ENUM 0x50000001 +#define MITI_ENUM_BHB_CLEAR_SEQ_S_SUPPORT BIT(0)/* VMM supports BHI_DIS_S */ +#define MITI_ENUM_RETPOLINE_S_SUPPORT BIT(1)/* VMM supports RRSBA_DIS_S */ + +#define MSR_VIRTUAL_MITIGATION_CTRL 0x50000002 +#define MITI_CTRL_BHB_CLEAR_SEQ_S_USED BIT(0)/* +* Request VMM to deploy +* BHI_DIS_S mitigation +*/ +#define MITI_CTRL_RETPOLINE_S_USED BIT(1)/* +* Request VMM to deploy +* RRSBA_DIS_S mitigation +*/ +/* AMD-V MSRs */ #define MSR_VM_CR 0xc0010114arch/x86/kernel/cpu/bugs.c +22
-0@@ -1379,6 +1379,28 @@ static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_dump_stack (); } +/* Speculation control using virtualized MSRs */ +static void __init spec_ctrl_setup_virtualized_msr (void ) +{ +u64 msr_virt_enum, msr_mitigation_enum, msr_mitigation_ctrl; + +if (!(x86_read_arch_cap_msr () &ARCH_CAP_VIRTUAL_ENUM )) +return ; + +rdmsrl (MSR_VIRTUAL_ENUMERATION , msr_virt_enum); +if (!(msr_virt_enum &VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return ; + +rdmsrl (MSR_VIRTUAL_MITIGATION_ENUM , msr_mitigation_enum); +/* When retpoline is being used, request relevant hardware controls */ +if (boot_cpu_has (X86_FEATURE_RETPOLINE ) && + msr_mitigation_enum &MITI_ENUM_RETPOLINE_S_SUPPORT ) { +rdmsrl (MSR_VIRTUAL_MITIGATION_CTRL , msr_mitigation_ctrl); + msr_mitigation_ctrl |=MITI_CTRL_RETPOLINE_S_USED ; +wrmsrl (MSR_VIRTUAL_MITIGATION_CTRL , msr_mitigation_ctrl); + } +} +static void __init spectre_v2_select_mitigation (void ) {enum spectre_v2_mitigation_cmd cmd =spectre_v2_parse_cmdline ();
arch/x86/kernel/cpu/bugs.c static void __init spectre_v2_select_mitigation(void) · line 1510
Quoted 4 lines
arch/x86/kernel/cpu/bugs.c +1
-0@@ -1485,6 +1507,8 @@ static void __init spectre_v2_select_mitigation(void) mode == SPECTRE_V2_RETPOLINE)spec_ctrl_disable_kernel_rrsba (); +spec_ctrl_setup_virtualized_msr ();
I think this also needs to be called during secondary CPU initialization.
RE: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
Quoted 124 lines
-----Original Message-----
From: Pawan Gupta <[email protected]>
Sent: Tuesday, December 13, 2022 4:24 AM
To: Zhang, Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Gao, Chao <[email protected]>; Paolo Bonzini <[email protected]>; Christopherson,, Sean <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>
Subject: Re: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigationsQuoted 24 lines · Zhang Chen
From: Pawan Gupta <[email protected]>
Guests that have different family/model than the host may not be aware of hardware
mitigations(such as RRSBA_DIS_S)available on host. This is particularly true when guests migrate. To solve this problem Intel processors have added a virtual MSR interface through which guests can report their mitigation status and request VMM to deploy relevant hardware mitigations.Use this virtualized MSR interface to request relevant hardware controls for retpoline mitigation.
Signed-off-by Pawan Gupta <[email protected]>--- arch/x86/include/asm/msr-index.h | 23 +++++++++++++++++++++++ arch/x86/kernel/cpu/bugs.c | 24 ++++++++++++++++++++++++ 2 files changed, 47 insertions(+)arch/x86/include/asm/msr-index.h +0
-0arch/x86/include/asm/msr-index.h line 165
Quoted 1 line
arch/x86/include/asm/msr-index.h +0
-0@@ -165,6 +165,7 @@*
IA32_XAPIC_DISABLE_STATUSMSRarch/x86/include/asm/msr-index.h
Quoted 3 lines
arch/x86/include/asm/msr-index.h +1
-0* supported */ +#define ARCH_CAP_VIRTUAL_ENUM BIT(63)/*
MSR_VIRTUAL_ENUMERATIONsupported */arch/x86/include/asm/msr-index.h line 1064
Quoted 6 lines
arch/x86/include/asm/msr-index.h +0
-0#define MSR_IA32_FLUSH_CMD 0x0000010b#define L1D_FLUSH BIT(0)/* @@ -1062,6 +1063,28 @@#define MSR_IA32_VMX_MISC_INTEL_PT (1ULL << 14)#define MSR_IA32_VMX_MISC_VMWRITE_SHADOW_RO_FIELDS (1ULL <<29)
arch/x86/include/asm/msr-index.h
Quoted 6 lines
arch/x86/include/asm/msr-index.h +5
-0#define MSR_IA32_VMX_MISC_PREEMPTION_TIMER_SCALE 0x1F + +/* Intel virtual MSRs */ +#define MSR_VIRTUAL_ENUMERATION 0x50000000 +#define VIRT_ENUM_MITIGATION_CTRL_SUPPORT BIT(0)/* +* Mitigation ctrl via virtual
arch/x86/include/asm/msr-index.h
Quoted 5 lines
arch/x86/include/asm/msr-index.h +5
-0+* MSRs supported +*/ + +#define MSR_VIRTUAL_MITIGATION_ENUM 0x50000001 +#define MITI_ENUM_BHB_CLEAR_SEQ_S_SUPPORT BIT(0)/* VMM supports
BHI_DIS_S*/arch/x86/include/asm/msr-index.h line 1076
Quoted 1 line
arch/x86/include/asm/msr-index.h +1
-0+#define MITI_ENUM_RETPOLINE_S_SUPPORT BIT(1)/* VMM supports
RRSBA_DIS_S*/arch/x86/include/asm/msr-index.h
Quoted 4 lines
arch/x86/include/asm/msr-index.h +4
-0+ +#define MSR_VIRTUAL_MITIGATION_CTRL 0x50000002 +#define MITI_CTRL_BHB_CLEAR_SEQ_S_USED BIT(0)/* +* Request VMM to deploy
arch/x86/include/asm/msr-index.h line 1081
Quoted 1 line
arch/x86/include/asm/msr-index.h +1
-0+* BHI_DIS_S mitigation
arch/x86/include/asm/msr-index.h
Quoted 3 lines
arch/x86/include/asm/msr-index.h +3
-0+*/ +#define MITI_CTRL_RETPOLINE_S_USED BIT(1)/* +* Request VMM to deploy
arch/x86/include/asm/msr-index.h line 1085
Quoted 1 line
arch/x86/include/asm/msr-index.h +1
-0+* RRSBA_DIS_S mitigation
arch/x86/kernel/cpu/bugs.c static void __init
Quoted 10 lines
arch/x86/include/asm/msr-index.h +2
-0+*/ +/* AMD-V MSRs */ #define MSR_VM_CR 0xc0010114arch/x86/kernel/cpu/bugs.c +0
-0@@ -1379,6 +1379,28 @@ static void __initspectre_v2_determine_rsb_fill_type_at_vmexit(enum
spectre_v2_Quoted 7 lines
arch/x86/kernel/cpu/bugs.c +4
-0dump_stack (); } +/* Speculation control using virtualized MSRs */ static void __init +spec_ctrl_setup_virtualized_msr (void ) +{ +u64 msr_virt_enum, msr_mitigation_enum, msr_mitigation_ctrl;arch/x86/kernel/cpu/bugs.c static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_
Quoted 10 lines
arch/x86/kernel/cpu/bugs.c +10
-0+ +if (!(x86_read_arch_cap_msr () &ARCH_CAP_VIRTUAL_ENUM )) +return ; + +rdmsrl (MSR_VIRTUAL_ENUMERATION , msr_virt_enum); +if (!(msr_virt_enum &VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return ; + +rdmsrl (MSR_VIRTUAL_MITIGATION_ENUM , msr_mitigation_enum); +/* When retpoline is being used, request relevant hardware controls */
arch/x86/kernel/cpu/bugs.c static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_
Quoted 3 lines
arch/x86/kernel/cpu/bugs.c +3
-0+if (boot_cpu_has (X86_FEATURE_RETPOLINE ) && + msr_mitigation_enum &MITI_ENUM_RETPOLINE_S_SUPPORT ) { + rdmsrl(MSR_VIRTUAL_MITIGATION_CTRL,
msr_mitigation_ctrl);arch/x86/kernel/cpu/bugs.c static void __init · line 1400
Quoted 2 lines
arch/x86/kernel/cpu/bugs.c +2
-0+ msr_mitigation_ctrl |=MITI_CTRL_RETPOLINE_S_USED ; + wrmsrl(MSR_VIRTUAL_MITIGATION_CTRL,
msr_mitigation_ctrl);arch/x86/kernel/cpu/bugs.c static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_ · line 1406
Quoted 6 lines
arch/x86/kernel/cpu/bugs.c +3
-0+ } +} +static void __init spectre_v2_select_mitigation (void ) {enum spectre_v2_mitigation_cmd cmd =spectre_v2_parse_cmdline ();@@
Quoted 7 lines
arch/x86/kernel/cpu/bugs.c static void __init spectre_v2_select_mitigation(void) · line 1510
Quoted 5 lines
-1485,6 +1507,8 @@ static void
__initspectre_v2_select_mitigation(void)arch/x86/kernel/cpu/bugs.c +1
-0mode == SPECTRE_V2_RETPOLINE)spec_ctrl_disable_kernel_rrsba (); +spec_ctrl_setup_virtualized_msr ();I think this also needs to be called during secondary CPU initialization.
Yes, we need setup virtual MSRs for each CPU.
I will add the " spec_ctrl_setup_virtualized_msr()" to the "start_secondary()" in the arch/x86/kernel/smpboot.c next version.
Thanks
Chen
Sean Christopherson
maintainerRe: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
In reply to Zhang ChenQuoted 6 lines · Zhang Chen
From: Pawan Gupta <[email protected]>
Guests that have different family/model than the host may not be aware of hardware
mitigations(such as RRSBA_DIS_S)available on host. This is particularly true when guests migrate. To solve this problem Intel processors have added a virtual MSR interface
Is there any actual "processor" support here? To me, this looks like Intel is foisting a paravirt interface on KVM and other hypervisors without collaborating with said hypervisors' developers and maintainers.
I get that some of the mitigations are vendor specific, but things like RETPOLINE aren't vendor specific. I haven't followed all of the mitigation stuff very closely, but I wouldn't be surprised if there are mitigations now or in the future that are common across architectures, e.g. arm64 and x86-64. Intel doing its own thing means AMD and arm64 will likely follow suit, and suddenly KVM is supporting multiple paravirt interfaces for very similar things, without having any control over the APIs. That's all kinds of backwards.
And having to wait for Intel to roll out new documentation when software inevitably comes up with some clever new mitigation doesn't exactly fill my heart with joy.
Re: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
Quoted 9 lines · Sean Christopherson
Quoted 6 lines · Zhang Chen
From: Pawan Gupta <[email protected]>
Guests that have different family/model than the host may not be aware of hardware
mitigations(such as RRSBA_DIS_S)available on host. This is particularly true when guests migrate. To solve this problem Intel processors have added a virtual MSR interfaceIs there any actual "processor" support here?
No.
Quoted 11 lines
To me, this looks like Intel is
foisting a paravirt interface on KVM and other hypervisors without collaborating with said hypervisors' developers and maintainers.I get that some of the mitigations are vendor specific, but things like RETPOLINE aren't vendor specific. I haven't followed all of the mitigation stuff very closely, but I wouldn't be surprised if there are mitigations now or in the future that are common across architectures, e.g. arm64 and x86-64. Intel doing its own thing means AMD and arm64 will likely follow suit, and suddenly KVM is supporting multiple paravirt interfaces for very similar things, without having any control over the APIs. That's all kinds of backwards.
But if the interface is defined by KVM rather than Intel, it will likely end up with different interfaces for different VMMs, then Linux guest needs to support all of them. And KVM needs to implement Hyper-V's and Xen's interface to support Hyper-V enlightened and Xen enlightened guest. This is a _real_ problem and complicates KVM/Linux in a similar way as multiple paravirt interfaces.
The use case of this paravirt interface is specific to Intel CPU microarchitecture. Supporting multiple paravirt interfaces may not happen in the near future if there is no use case for AMD and arm64.
Quoted 3 lines
And having to wait for Intel to roll out new documentation when software inevitably comes up with some clever new mitigation doesn't exactly fill my heart with joy.
There should be some misunderstanding.
A bit for a software mitigation will be added if
1. the software mitigation works well on existing processors. And
2. there will be a new processor on which the software mitigation won't be fully effective (due to some microarchiecture change).
IOW, the new documentation comes along with a new microarchitecture (new processor) rather than software mitigations.
Sean Christopherson
maintainerRe: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
Quoted 12 lines · Chao Gao
Quoted 10 lines · Sean Christopherson
To me, this looks like Intel is foisting a paravirt interface on KVM and other hypervisors without collaborating with said hypervisors' developers and maintainers.
I get that some of the mitigations are vendor specific, but things like RETPOLINE aren't vendor specific. I haven't followed all of the mitigation stuff very closely, but I wouldn't be surprised if there are mitigations now or in the future that are common across architectures, e.g. arm64 and x86-64. Intel doing its own thing means AMD and arm64 will likely follow suit, and suddenly KVM is supporting multiple paravirt interfaces for very similar things, without having any control over the APIs. That's all kinds of backwards.
Quoted 5 lines
But if the interface is defined by KVM rather than Intel, it will likely end up with different interfaces for different VMMs, then Linux guest needs to support all of them. And KVM needs to implement Hyper-V's and Xen's interface to support Hyper-V enlightened and Xen enlightened guest. This is a _real_ problem and complicates KVM/Linux in a similar way as multiple paravirt interfaces.
I never said the PV interfaces should be defined by KVM. I 100% agree that any one hypervisor defining its own interface will suffer the same problem.
I think having a PV interface for coordinating mitigations between host and guest is a great idea. What I don't like is tying the interface to "hardware" and defining the interface without even trying to collaborate with others.
Quoted 1 line
The use case of this paravirt interface is specific to Intel CPU microarchitecture.
Well yeah, because the interface was designed only to work for Intel CPUs.
Quoted 2 lines
Supporting multiple paravirt interfaces may not happen in the near future if there is no use case for AMD and arm64.
I'll take that bet. The vast majority of problems that are solved by PV interfaces are common to all architectures and vendors, e.g. steal time, PV spinlocks, async page faults, directed yield, confidential VMs (GHCB vs. GHCI), etc. I highly doubt Intel is the only hardware vendor that will ever benefit from paravirtualizing mitigations.
Re: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
Quoted 13 lines · Sean Christopherson
Quoted 12 lines · Chao Gao
Quoted 10 lines · Sean Christopherson
To me, this looks like Intel is foisting a paravirt interface on KVM and other hypervisors without collaborating with said hypervisors' developers and maintainers.
I get that some of the mitigations are vendor specific, but things like RETPOLINE aren't vendor specific. I haven't followed all of the mitigation stuff very closely, but I wouldn't be surprised if there are mitigations now or in the future that are common across architectures, e.g. arm64 and x86-64. Intel doing its own thing means AMD and arm64 will likely follow suit, and suddenly KVM is supporting multiple paravirt interfaces for very similar things, without having any control over the APIs. That's all kinds of backwards.
Quoted 8 lines
Quoted 5 lines
But if the interface is defined by KVM rather than Intel, it will likely end up with different interfaces for different VMMs, then Linux guest needs to support all of them. And KVM needs to implement Hyper-V's and Xen's interface to support Hyper-V enlightened and Xen enlightened guest. This is a _real_ problem and complicates KVM/Linux in a similar way as multiple paravirt interfaces.
I never said the PV interfaces should be defined by KVM. I 100% agree that any one hypervisor defining its own interface will suffer the same problem.
I am thinking there are only two options:
1. Intel defines the interface.
2. Every VMM defines its own interface.
Any middle ground between the two options?
Quoted 3 lines
I think having a PV interface for coordinating mitigations between host and guest is a great idea. What I don't like is tying the interface to "hardware" and defining
Do you think something below is better than the intel-defined interface? add a new KVM_CAP_* and a KVM_FEATURE_* in hypervisor CPUID leaf to enumerate the interface. And add a new virtual MSR 0x4b564dxx for guest to report in-use software mitigations and assign one bit for each software mitigation. On MSR write emulation, call into vmx code to enable some hardware mitigations if the corresponding software mitigations are not effective on host.
I am afraid it is late to switch to above approach because e.g., if Hyper-V decides to support the intel-defined interface, KVM probably needs to support it for Hyper-V guests.
Can we emulate IA32_ARCH_CAPABILITIES for AMD guests so that AMD can re-use intel-defined interface?
Quoted 1 line
the interface without even trying to collaborate with others.
I will pass on this to related Intel folks and ask them to collaborate with KVM community on future PV interfaces.
Sean Christopherson
maintainerRe: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
Quoted 23 lines · Chao Gao
Quoted 13 lines · Sean Christopherson
Quoted 12 lines · Chao Gao
Quoted 10 lines · Sean Christopherson
To me, this looks like Intel is foisting a paravirt interface on KVM and other hypervisors without collaborating with said hypervisors' developers and maintainers.
I get that some of the mitigations are vendor specific, but things like RETPOLINE aren't vendor specific. I haven't followed all of the mitigation stuff very closely, but I wouldn't be surprised if there are mitigations now or in the future that are common across architectures, e.g. arm64 and x86-64. Intel doing its own thing means AMD and arm64 will likely follow suit, and suddenly KVM is supporting multiple paravirt interfaces for very similar things, without having any control over the APIs. That's all kinds of backwards.
Quoted 8 lines
Quoted 5 lines
But if the interface is defined by KVM rather than Intel, it will likely end up with different interfaces for different VMMs, then Linux guest needs to support all of them. And KVM needs to implement Hyper-V's and Xen's interface to support Hyper-V enlightened and Xen enlightened guest. This is a _real_ problem and complicates KVM/Linux in a similar way as multiple paravirt interfaces.
I never said the PV interfaces should be defined by KVM. I 100% agree that any one hypervisor defining its own interface will suffer the same problem.
Quoted 6 lines
I am thinking there are only two options:
1. Intel defines the interface.
2. Every VMM defines its own interface.Any middle ground between the two options?
Work with other x86 hardware vendors to define a common interface? Ask hypervisor developers to define a common, extensible interface?
Maybe it turns out that it's impossible to abstract anything away and everything ends up being vendor-specific anyways, but not even trying to provide a common interace is extremely frustrating, especially since all this mitigation stuff has been going on for ~5 years. There's been plenty of time to establish relationships and points of contact.
Quoted 4 lines
Quoted 2 lines
I think having a PV interface for coordinating mitigations between host and guest is a great idea. What I don't like is tying the interface to "hardware" and defining
Do you think something below is better than the intel-defined interface?
No, KVM doing its own thing would only exacerbate the issue.
Quoted 9 lines
add a new KVM_CAP_* and a KVM_FEATURE_* in hypervisor CPUID leaf to enumerate the interface. And add a new virtual MSR 0x4b564dxx for guest to report in-use software mitigations and assign one bit for each software mitigation. On MSR write emulation, call into vmx code to enable some hardware mitigations if the corresponding software mitigations are not effective on host.
I am afraid it is late to switch to above approach because e.g., if Hyper-V decides to support the intel-defined interface, KVM probably needs to support it for Hyper-V guests.
Hence my frustration.
RE: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigations
Quoted 59 lines
-----Original Message-----
From: Sean Christopherson <[email protected]>
Sent: Friday, December 23, 2022 2:32 AM
To: Gao, Chao <[email protected]>
Cc: Zhang, Chen <[email protected]>; [email protected]; linux- [email protected]; [email protected]; Pawan Gupta <[email protected]>; Paolo Bonzini <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>
Subject: Re: [RFC PATCH 5/9] x86/bugs: Use Virtual MSRs to request hardware mitigationsQuoted 5 lines · Chao Gao
Quoted 4 lines · Sean Christopherson
Quoted 3 lines · Chao Gao
Quoted 2 lines · Sean Christopherson
To me, this looks like Intel is foisting a paravirt interface on KVM and other hypervisors without collaborating with said hypervisors'
developers and maintainers.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
I get that some of the mitigations are vendor specific, but things like RETPOLINE aren't vendor specific. I haven't followed all of the mitigation stuff very closely, but I wouldn't be surprised if there are mitigations now or in the future that are common across architectures, e.g. arm64 and x86-64. Intel doing its own thing means AMD and arm64 will likely follow suit, and suddenly KVM is supporting multiple paravirt interfaces for very similar things, without
having any control over the APIs. That's all kinds of backwards.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
But if the interface is defined by KVM rather than Intel, it will likely end up with different interfaces for different VMMs, then Linux guest needs to support all of them. And KVM needs to implement Hyper-V's and Xen's interface to support Hyper-V enlightened and Xen enlightened guest. This is a _real_ problem and
complicates KVM/Linux in a similar way as multiple paravirt interfaces.
Quoted 3 lines
Quoted 3 lines
I never said the PV interfaces should be defined by KVM. I 100% agree that any one hypervisor defining its own interface will suffer the same
problem.
Quoted 7 lines
I am thinking there are only two options:
1. Intel defines the interface.
2. Every VMM defines its own interface.Any middle ground between the two options?
Work with other x86 hardware vendors to define a common interface? Ask hypervisor developers to define a common, extensible interface?
Maybe it turns out that it's impossible to abstract anything away and everything ends up being vendor-specific anyways, but not even trying to provide a common interace is extremely frustrating, especially since all this mitigation stuff has been going on for ~5 years. There's been plenty of time to establish relationships and points of contact.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
I think having a PV interface for coordinating mitigations between host and guest is a great idea. What I don't like is tying the interface to "hardware" and defining
Do you think something below is better than the intel-defined interface?
No, KVM doing its own thing would only exacerbate the issue.
Hi Sean,
Rethink about it and synced with Chao, we didn't think of a better solution than Intel-defined interface. If you have no more suggestions/comments here, please review the rest of patches from this series when you have time. I will try to address comments and push this series to RFC V2.
Thanks
Chen
Quoted 13 lines
Quoted 6 lines
add a new KVM_CAP_* and a KVM_FEATURE_* in hypervisor CPUID leaf to enumerate the interface. And add a new virtual MSR 0x4b564dxx for guest to report in-use software mitigations and assign one bit for each software mitigation. On MSR write emulation, call into vmx code to enable some hardware mitigations if the corresponding software
mitigations are not effective on host.
Quoted 4 lines
I am afraid it is late to switch to above approach because e.g., if Hyper-V decides to support the intel-defined interface, KVM probably needs to support it for Hyper-V guests.
Hence my frustration.
RFC PATCH 6/9 kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest MSR_IA32_ARCH_CAPABILITIES
In reply to Zhang ChenAdd the 63 bit in MSR_IA32_ARCH_CAPABILITIES for enable the virtual MSRs. Virtual MSRs can allow guests to notify VMM whether or not they are using specific software mitigation, allowing a VMM to enable there hardware control only where necessary.
As Intel spec defination, expose virtual MSR for guest.
Make guest have ability to check virtual MSR 0x50000000.
--- arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ arch/x86/kvm/vmx/vmx.h | 1 + arch/x86/kvm/x86.c | 16 +++++++++++++++- 3 files changed, 31 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +15 -0
@@ -2001,6 +2001,12 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_IA32_DEBUGCTLMSR : msr_info->data =vmcs_read64 (GUEST_IA32_DEBUGCTL );break ; +case MSR_VIRTUAL_ENUMERATION : +if (!msr_info->host_initiated && + !(vcpu->arch.arch_capabilities &ARCH_CAP_VIRTUAL_ENUM )) +return 1 ; + msr_info->data = vmx->msr_virtual_enumeration; +break ;default : find_uret_msr: msr =vmx_find_uret_msr (vmx, msr_info->index);
@@ -2375,6 +2381,15 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) } ret =kvm_set_msr_common (vcpu, msr_info);break ; +case MSR_VIRTUAL_ENUMERATION : +if (msr_info->host_initiated && + !(vcpu->arch.arch_capabilities &ARCH_CAP_VIRTUAL_ENUM )) +return 1 ; +if (data & ~VIRT_ENUM_MITIGATION_CTRL_SUPPORT ) +return 1 ; + vmx->msr_virtual_enumeration = data & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT ; +break ;default : find_uret_msr:
arch/x86/kvm/vmx/vmx.h +1 -0
@@ -298,6 +298,7 @@ struct vcpu_vmx {* IA32_SPEC_CTRL_MSR. */ u64 spec_ctrl_mask; +u64 msr_virtual_enumeration;u32 msr_ia32_umwait_control;/*
arch/x86/kvm/x86.c +15 -1
@@ -1532,6 +1532,8 @@ static const u32 emulated_msrs_all[] = {MSR_IA32_VMX_EPT_VPID_CAP ,MSR_IA32_VMX_VMFUNC , +MSR_VIRTUAL_ENUMERATION , +MSR_K7_HWCR , MSR_KVM_POLL_CONTROL, };
@@ -1567,6 +1569,7 @@ static const u32 msr_based_features_all[] = {MSR_IA32_UCODE_REV ,MSR_IA32_ARCH_CAPABILITIES ,MSR_IA32_PERF_CAPABILITIES , + MSR_VIRTUAL_ENUMERATION, };static u32 msr_based_features[ARRAY_SIZE (msr_based_features_all)];
@@ -1588,7 +1591,8 @@ static unsigned int num_msr_based_features; ARCH_CAP_SKIP_VMENTRY_L1DFLUSH | ARCH_CAP_SSB_NO | ARCH_CAP_MDS_NO | \ ARCH_CAP_PSCHANGE_MC_NO | ARCH_CAP_TSX_CTRL_MSR | ARCH_CAP_TAA_NO | \ ARCH_CAP_SBDR_SSDP_NO | ARCH_CAP_FBSDP_NO | ARCH_CAP_PSDP_NO | \ - ARCH_CAP_FB_CLEAR | ARCH_CAP_RRSBA | ARCH_CAP_PBRSB_NO) + ARCH_CAP_FB_CLEAR | ARCH_CAP_RRSBA | ARCH_CAP_PBRSB_NO | \ + ARCH_CAP_VIRTUAL_ENUM)static u64 kvm_get_arch_capabilities (void ) {
@@ -1607,6 +1611,13 @@ static u64 kvm_get_arch_capabilities(void)*/ data |=ARCH_CAP_PSCHANGE_MC_NO ; +/* +* Virtual MSRs can allow guests to notify VMM whether or not +* they are using specific software mitigation, allowing a VMM +* to enable there hardware control only where necessary. +*/ + data |=ARCH_CAP_VIRTUAL_ENUM ; +/* * If we're doing cache flushes (either "always" or "cond") * we will do one whenever the guest does a vmlaunch/vmresume.
@@ -1657,6 +1668,9 @@ static int kvm_get_msr_feature(struct kvm_msr_entry *msr)case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ; +case MSR_VIRTUAL_ENUMERATION : + msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ; +break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
Signature · 1 line
2.25.1
Re: [RFC PATCH 6/9] kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest MSR_IA32_ARCH_CAPABILITIES
Quoted 91 lines · Zhang Chen
Add the 63 bit in
MSR_IA32_ARCH_CAPABILITIESfor enable the virtual MSRs. Virtual MSRs can allow guests to notify VMM whether or not they are using specific software mitigation, allowing a VMM to enable there hardware control only where necessary.
As Intel spec defination, expose virtual MSR for guest.
Make guest have ability to check virtual MSR0x50000000.Signed-off-by Zhang Chen <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ arch/x86/kvm/vmx/vmx.h | 1 + arch/x86/kvm/x86.c | 16 +++++++++++++++- 3 files changed, 31 insertions(+), 1 deletion(-)arch/x86/kvm/vmx/vmx.c +15
-0@@ -2001,6 +2001,12 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_IA32_DEBUGCTLMSR : msr_info->data =vmcs_read64 (GUEST_IA32_DEBUGCTL );break ; +case MSR_VIRTUAL_ENUMERATION : +if (!msr_info->host_initiated && + !(vcpu->arch.arch_capabilities &ARCH_CAP_VIRTUAL_ENUM )) +return 1 ; + msr_info->data = vmx->msr_virtual_enumeration; +break ;default : find_uret_msr: msr =vmx_find_uret_msr (vmx, msr_info->index);@@ -2375,6 +2381,15 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) } ret =kvm_set_msr_common (vcpu, msr_info);break ; +case MSR_VIRTUAL_ENUMERATION : +if (msr_info->host_initiated && + !(vcpu->arch.arch_capabilities &ARCH_CAP_VIRTUAL_ENUM )) +return 1 ; +if (data & ~VIRT_ENUM_MITIGATION_CTRL_SUPPORT ) +return 1 ; + vmx->msr_virtual_enumeration = data & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT ; +break ;default : find_uret_msr:arch/x86/kvm/vmx/vmx.h +1
-0@@ -298,6 +298,7 @@ struct vcpu_vmx {* IA32_SPEC_CTRL_MSR. */ u64 spec_ctrl_mask; +u64 msr_virtual_enumeration;u32 msr_ia32_umwait_control;/* arch/x86/kvm/x86.c +5
-1@@ -1532,6 +1532,8 @@ static const u32 emulated_msrs_all[] = {MSR_IA32_VMX_EPT_VPID_CAP ,MSR_IA32_VMX_VMFUNC , +MSR_VIRTUAL_ENUMERATION , +MSR_K7_HWCR , MSR_KVM_POLL_CONTROL, };@@ -1567,6 +1569,7 @@ static const u32 msr_based_features_all[] = {MSR_IA32_UCODE_REV ,MSR_IA32_ARCH_CAPABILITIES ,MSR_IA32_PERF_CAPABILITIES , + MSR_VIRTUAL_ENUMERATION, };static u32 msr_based_features[ARRAY_SIZE (msr_based_features_all)];@@ -1588,7 +1591,8 @@ static unsigned int num_msr_based_features; ARCH_CAP_SKIP_VMENTRY_L1DFLUSH | ARCH_CAP_SSB_NO | ARCH_CAP_MDS_NO | \ ARCH_CAP_PSCHANGE_MC_NO | ARCH_CAP_TSX_CTRL_MSR | ARCH_CAP_TAA_NO | \ ARCH_CAP_SBDR_SSDP_NO | ARCH_CAP_FBSDP_NO | ARCH_CAP_PSDP_NO | \ - ARCH_CAP_FB_CLEAR | ARCH_CAP_RRSBA | ARCH_CAP_PBRSB_NO) + ARCH_CAP_FB_CLEAR | ARCH_CAP_RRSBA | ARCH_CAP_PBRSB_NO | \ + ARCH_CAP_VIRTUAL_ENUM)static u64 kvm_get_arch_capabilities (void ) {
arch/x86/kvm/x86.c static u64 kvm_get_arch_capabilities(void) · line 1619
Quoted 9 lines
arch/x86/kvm/x86.c +6
-0@@ -1607,6 +1611,13 @@ static u64 kvm_get_arch_capabilities(void)*/ data |=ARCH_CAP_PSCHANGE_MC_NO ; +/* +* Virtual MSRs can allow guests to notify VMM whether or not +* they are using specific software mitigation, allowing a VMM +* to enable there hardware control only where necessary. +*/ + data |=ARCH_CAP_VIRTUAL_ENUM ;
IMO, this is: data &= ARCH_CAP_VIRTUAL_ENUM; because it requires platform support.
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct kvm_msr_entry *msr) · line 1672
Quoted 10 lines
arch/x86/kvm/x86.c +3
-0+/* * If we're doing cache flushes (either "always" or "cond") * we will do one whenever the guest does a vmlaunch/vmresume. @@ -1657,6 +1668,9 @@ static int kvm_get_msr_feature(struct kvm_msr_entry *msr)case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ; +case MSR_VIRTUAL_ENUMERATION : + msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;
Need to check bit 63 of host MSR_ARCH_CAPABILITIES before expose the feature.
Quoted 4 lines
arch/x86/kvm/x86.c +1
-0+break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
RE: [RFC PATCH 6/9] kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest MSR_IA32_ARCH_CAPABILITIES
Quoted 122 lines
-----Original Message-----
From: Yang, Weijiang <[email protected]>
Sent: Wednesday, December 21, 2022 12:04 PM
To: Zhang, Chen <[email protected]>
Cc: Gao, Chao <[email protected]>; Pawan Gupta <[email protected]>; Paolo Bonzini <[email protected]>; Christopherson,, Sean <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 6/9] kvm/x86: AddARCH_CAP_VIRTUAL_ENUMfor guestMSR_IA32_ARCH_CAPABILITIESarch/x86/kvm/vmx/vmx.c static int vmx_get_msr(struct kvm_vcpu *vcpu,
Quoted 19 lines · Zhang Chen
Add the 63 bit in
MSR_IA32_ARCH_CAPABILITIESfor enable the virtual MSRs. Virtual MSRs can allow guests to notify VMM whether or not they are using specific software mitigation, allowing a VMM to enable there hardware control only where necessary.
As Intel spec defination, expose virtual MSR for guest.
Make guest have ability to check virtual MSR0x50000000.Signed-off-by Zhang Chen <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ arch/x86/kvm/vmx/vmx.h | 1 + arch/x86/kvm/x86.c | 16 +++++++++++++++- 3 files changed, 31 insertions(+), 1 deletion(-)arch/x86/kvm/vmx/vmx.c index renamed from arch/x86/kvm/vmx/vmx.c +0
-0
407061b369b4..6ed6b743be0e100644arch/x86/kvm/vmx/vmx.c +0
-0@@ -2001,6 +2001,12 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu,struct
msr_data*msr_info)arch/x86/kvm/vmx/vmx.c static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +3
-0case MSR_IA32_DEBUGCTLMSR : msr_info->data =vmcs_read64 (GUEST_IA32_DEBUGCTL );break ; +case MSR_VIRTUAL_ENUMERATION : + if (!msr_info->host_initiated && + !(vcpu->arch.arch_capabilities &
ARCH_CAP_VIRTUAL_ENUM))arch/x86/kvm/vmx/vmx.c static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +3
-0+return 1 ; + msr_info->data = vmx->msr_virtual_enumeration; +break ;default : find_uret_msr: msr = vmx_find_uret_msr(vmx, msr_info->index); @@ -2375,6+2381,15
Quoted 1 line
@@ static int vmx_set_msr(struct
kvm_vcpu*vcpu, structmsr_data
*msr_info)arch/x86/kvm/vmx/vmx.c static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
Quoted 6 lines
} ret = kvm_set_msr_common(vcpu, msr_info); break;arch/x86/kvm/vmx/vmx.c +3
-0+case MSR_VIRTUAL_ENUMERATION : + if (msr_info->host_initiated && + !(vcpu->arch.arch_capabilities &
ARCH_CAP_VIRTUAL_ENUM))arch/x86/kvm/vmx/vmx.c static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +5
-0+return 1 ; +if (data & ~VIRT_ENUM_MITIGATION_CTRL_SUPPORT ) +return 1 ; + vmx->msr_virtual_enumeration = data & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT;Quoted 38 lines
arch/x86/kvm/vmx/vmx.c +1
-0+break ;default : find_uret_msr:arch/x86/kvm/vmx/vmx.h index renamed from arch/x86/kvm/vmx/vmx.h +0
-0
c5a41ae14237..fc873cf45f70100644arch/x86/kvm/vmx/vmx.h +1
-0@@ -298,6 +298,7 @@ struct vcpu_vmx {* IA32_SPEC_CTRL_MSR. */ u64 spec_ctrl_mask; +u64 msr_virtual_enumeration;u32 msr_ia32_umwait_control;/* arch/x86/kvm/x86.c index renamed from arch/x86/kvm/x86.c +0
-0
2835bd796639..6be0a3f1281f100644arch/x86/kvm/x86.c +3
-0@@ -1532,6 +1532,8 @@ static const u32 emulated_msrs_all[] = {MSR_IA32_VMX_EPT_VPID_CAP ,MSR_IA32_VMX_VMFUNC , +MSR_VIRTUAL_ENUMERATION , +MSR_K7_HWCR , MSR_KVM_POLL_CONTROL, };@@ -1567,6 +1569,7 @@ static const u32 msr_based_features_all[] = {MSR_IA32_UCODE_REV ,MSR_IA32_ARCH_CAPABILITIES ,MSR_IA32_PERF_CAPABILITIES , + MSR_VIRTUAL_ENUMERATION, };static u32 msr_based_features[ARRAY_SIZE (msr_based_features_all)];arch/x86/kvm/x86.c static unsigned int num_msr_based_features; · line 1591
Quoted 1 line
arch/x86/kvm/x86.c +0
-0@@ -1588,7 +1591,8 @@ static unsigned int num_msr_based_features; ARCH_CAP_SKIP_VMENTRY_L1DFLUSH | ARCH_CAP_SSB_NO |
ARCH_CAP_MDS_NO| \arch/x86/kvm/x86.c static unsigned int num_msr_based_features; · line 1592
Quoted 1 line
arch/x86/kvm/x86.c +0
-0ARCH_CAP_PSCHANGE_MC_NO | ARCH_CAP_TSX_CTRL_MSR |
ARCH_CAP_TAA_NO| \arch/x86/kvm/x86.c static unsigned int num_msr_based_features; · line 1593
Quoted 1 line
arch/x86/kvm/x86.c +0
-0ARCH_CAP_SBDR_SSDP_NO | ARCH_CAP_FBSDP_NO |
ARCH_CAP_PSDP_NO| \arch/x86/kvm/x86.c static unsigned int num_msr_based_features; · line 1594
Quoted 2 lines
arch/x86/kvm/x86.c +1
-1- ARCH_CAP_FB_CLEAR | ARCH_CAP_RRSBA | ARCH_CAP_PBRSB_NO) + ARCH_CAP_FB_CLEAR | ARCH_CAP_RRSBA | ARCH_CAP_PBRSB_NO |\
Quoted 8 lines
arch/x86/kvm/x86.c +1
-0+ ARCH_CAP_VIRTUAL_ENUM) static u64 kvm_get_arch_capabilities(void) {@@ -1607,6 +1611,13 @@ static u64 kvm_get_arch_capabilities(void)*/ data |=ARCH_CAP_PSCHANGE_MC_NO ;
Quoted 10 lines
arch/x86/kvm/x86.c static u64 kvm_get_arch_capabilities(void) · line 1619
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+/* +* Virtual MSRs can allow guests to notify VMM whether or not +* they are using specific software mitigation, allowing a VMM +* to enable there hardware control only where necessary. +*/ + data |=ARCH_CAP_VIRTUAL_ENUM ;IMO, this is: data &=
ARCH_CAP_VIRTUAL_ENUM; because it requires platform support.
Intel defined the virtual MSRs for software mitigations for all platforms. KVM should be unconditionally opened it for the software mitigation in migration pools. For example migration from the old platform to the new platform.
Please check the Software Mitigations in Migration Pools section in documents:
https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/branch-history-injection.html
Quoted 8 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct
Quoted 7 lines
arch/x86/kvm/x86.c +1
-0+/* * If we're doing cache flushes (either "always" or "cond") * we will do one whenever the guest does a vmlaunch/vmresume. @@ -1657,6 +1668,9 @@ static int kvm_get_msr_feature(struct
kvm_msr_entry*msr)
Quoted 9 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct kvm_msr_entry *msr) · line 1672
Quoted 5 lines
arch/x86/kvm/x86.c +2
-0case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ; +case MSR_VIRTUAL_ENUMERATION : + msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;Need to check bit 63 of host
MSR_ARCH_CAPABILITIESbefore expose the feature.
Refer to the above comments.
Thanks
Chen
Quoted 6 lines
Quoted 6 lines
arch/x86/kvm/x86.c +1
-0+break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
Re: [RFC PATCH 6/9] kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest MSR_IA32_ARCH_CAPABILITIES
On 12/29/2022 10:58 AM, Zhang, Chen wrote:
[...]
Quoted 18 lines
Quoted 9 lines
\
Quoted 8 lines
arch/x86/kvm/x86.c +1
-0+ ARCH_CAP_VIRTUAL_ENUM) static u64 kvm_get_arch_capabilities(void) {@@ -1607,6 +1611,13 @@ static u64 kvm_get_arch_capabilities(void)*/ data |=ARCH_CAP_PSCHANGE_MC_NO ;Quoted 9 lines
arch/x86/kvm/x86.c static u64 kvm_get_arch_capabilities(void) · line 1619
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+/* +* Virtual MSRs can allow guests to notify VMM whether or not +* they are using specific software mitigation, allowing a VMM +* to enable there hardware control only where necessary. +*/ + data |=ARCH_CAP_VIRTUAL_ENUM ;IMO, this is: data &=
ARCH_CAP_VIRTUAL_ENUM; because it requires platform support.
Quoted 5 lines
Intel defined the virtual MSRs for software mitigations for all platforms. KVM should be unconditionally opened it for the software mitigation in migration pools. For example migration from the old platform to the new platform.
Please check the Software Mitigations in Migration Pools section in documents:
https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/branch-history-injection.html
If this series running on old platforms, how VMM can set specific vmcs fields,
e.g., “virtualize IA32_SPEC_CTRL” VM-execution control, to mitigate guest issues?
Quoted 26 lines
Quoted 8 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct
Quoted 7 lines
arch/x86/kvm/x86.c +1
-0+/* * If we're doing cache flushes (either "always" or "cond") * we will do one whenever the guest does a vmlaunch/vmresume. @@ -1657,6 +1668,9 @@ static int kvm_get_msr_feature(struct
kvm_msr_entry*msr)Quoted 8 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct kvm_msr_entry *msr) · line 1672
Quoted 5 lines
arch/x86/kvm/x86.c +2
-0case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ; +case MSR_VIRTUAL_ENUMERATION : + msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;Need to check bit 63 of host
MSR_ARCH_CAPABILITIESbefore expose the feature.Refer to the above comments.
Thanks
ChenQuoted 5 lines
Quoted 5 lines
arch/x86/kvm/x86.c +1
-0+break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
RE: [RFC PATCH 6/9] kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest MSR_IA32_ARCH_CAPABILITIES
Quoted 45 lines
-----Original Message-----
From: Yang, Weijiang <[email protected]>
Sent: Thursday, December 29, 2022 3:03 PM
To: Zhang, Chen <[email protected]>
Cc: Gao, Chao <[email protected]>; Pawan Gupta <[email protected]>; Paolo Bonzini <[email protected]>; Christopherson,, Sean <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 6/9] kvm/x86: AddARCH_CAP_VIRTUAL_ENUMfor guestMSR_IA32_ARCH_CAPABILITIESOn 12/29/2022 10:58 AM, Zhang, Chen wrote:
[...]
Quoted 18 lines
Quoted 9 lines
\
Quoted 8 lines
arch/x86/kvm/x86.c +1
-0+ ARCH_CAP_VIRTUAL_ENUM) static u64 kvm_get_arch_capabilities(void) {@@ -1607,6 +1611,13 @@ static u64 kvm_get_arch_capabilities(void)*/ data |=ARCH_CAP_PSCHANGE_MC_NO ;Quoted 9 lines
arch/x86/kvm/x86.c static u64 kvm_get_arch_capabilities(void) · line 1619
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+/* +* Virtual MSRs can allow guests to notify VMM whether or not +* they are using specific software mitigation, allowing a VMM +* to enable there hardware control only where necessary. +*/ + data |=ARCH_CAP_VIRTUAL_ENUM ;IMO, this is: data &=
ARCH_CAP_VIRTUAL_ENUM; because it requires platform support.Quoted 2 lines
Intel defined the virtual MSRs for software mitigations for all platforms. KVM should be unconditionally opened it for the software mitigation in
migration pools.
Quoted 2 lines
For example migration from the old platform to the new platform. Please check the Software Mitigations in Migration Pools section in
documents:
Quoted 8 lines
Quoted 3 lines
https://www.intel.com/content/www/us/en/developer/articles/technical/s
oftware-security-guidance/technical-documentation/branch-history-injec tion.htmlIf this series running on old platforms, how VMM can set specific vmcs fields,
e.g., “virtualize IA32_SPEC_CTRL” VM-execution control, to mitigate guest issues?
Enable the virtual MSRs does not means to enable the “virtualize IA32_SPEC_CTRL”. KVM will check "cpu_has_virt_spec_ctrl()" before set specific VMCS.
Thanks
Chen
Quoted 28 lines
Quoted 28 lines
Quoted 10 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct
Quoted 9 lines
arch/x86/kvm/x86.c +1
-0+/* * If we're doing cache flushes (either "always" or "cond") * we will do one whenever the guest does a vmlaunch/vmresume. @@ -1657,6 +1668,9 @@ static int kvm_get_msr_feature(struct
kvm_msr_entry*msr)Quoted 8 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct kvm_msr_entry *msr) · line 1672
Quoted 5 lines
arch/x86/kvm/x86.c +2
-0case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ; +case MSR_VIRTUAL_ENUMERATION : + msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;Need to check bit 63 of host
MSR_ARCH_CAPABILITIESbefore expose the feature.Refer to the above comments.
Thanks
ChenQuoted 5 lines
Quoted 5 lines
arch/x86/kvm/x86.c +1
-0+break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
Re: [RFC PATCH 6/9] kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest MSR_IA32_ARCH_CAPABILITIES
Quoted 46 lines · Chen
Quoted 46 lines
-----Original Message-----
From: Yang, Weijiang <[email protected]>
Sent: Thursday, December 29, 2022 3:03 PM
To: Zhang, Chen <[email protected]>
Cc: Gao, Chao <[email protected]>; Pawan Gupta <[email protected]>; Paolo Bonzini <[email protected]>; Christopherson,, Sean <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 6/9] kvm/x86: AddARCH_CAP_VIRTUAL_ENUMfor guestMSR_IA32_ARCH_CAPABILITIESOn 12/29/2022 10:58 AM, Zhang, Chen wrote:
[...]
Quoted 17 lines
Quoted 9 lines
\
Quoted 8 lines
arch/x86/kvm/x86.c +1
-0+ ARCH_CAP_VIRTUAL_ENUM) static u64 kvm_get_arch_capabilities(void) {@@ -1607,6 +1611,13 @@ static u64 kvm_get_arch_capabilities(void)*/ data |=ARCH_CAP_PSCHANGE_MC_NO ;Quoted 8 lines
arch/x86/kvm/x86.c static u64 kvm_get_arch_capabilities(void) · line 1619
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+/* +* Virtual MSRs can allow guests to notify VMM whether or not +* they are using specific software mitigation, allowing a VMM +* to enable there hardware control only where necessary. +*/ + data |=ARCH_CAP_VIRTUAL_ENUM ;IMO, this is: data &=
ARCH_CAP_VIRTUAL_ENUM; because it requires platform support.Quoted 2 lines
Intel defined the virtual MSRs for software mitigations for all platforms. KVM should be unconditionally opened it for the software mitigation in
migration pools.
Quoted 2 lines
For example migration from the old platform to the new platform. Please check the Software Mitigations in Migration Pools section in
documents:
Quoted 3 lines
https://www.intel.com/content/www/us/en/developer/articles/technical/s
oftware-security-guidance/technical-documentation/branch-history-injec tion.html
Quoted 6 lines
Quoted 4 lines
If this series running on old platforms, how VMM can set specific vmcs fields,
e.g., “virtualize IA32_SPEC_CTRL” VM-execution control, to mitigate guest issues?
Enable the virtual MSRs does not means to enable the “virtualize IA32_SPEC_CTRL”. KVM will check "
cpu_has_virt_spec_ctrl()" before set specific VMCS.
What I'm concerned is, if the feature is not supported on some platforms, unconditionally
exposing to guest would make it wrongly "think" the feature is supported on HW, but actually
WMM can do nothing to mitigate issues. Read through the patch set, looks like guest cannot
even get any errors during feature configuration in this case. So I suggest add more
sanity checks. Did I miss something?
Quoted 27 lines
Thanks
ChenQuoted 23 lines
Quoted 23 lines
Quoted 7 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct
Quoted 6 lines
arch/x86/kvm/x86.c +1
-0+/* * If we're doing cache flushes (either "always" or "cond") * we will do one whenever the guest does a vmlaunch/vmresume. @@ -1657,6 +1668,9 @@ static int kvm_get_msr_feature(struct
kvm_msr_entry*msr)Quoted 7 lines
arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct kvm_msr_entry *msr) · line 1672
Quoted 5 lines
arch/x86/kvm/x86.c +2
-0case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ; +case MSR_VIRTUAL_ENUMERATION : + msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;Need to check bit 63 of host
MSR_ARCH_CAPABILITIESbefore expose the feature.Refer to the above comments.
Thanks
ChenQuoted 4 lines
Quoted 4 lines
arch/x86/kvm/x86.c +1
-0+break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
RE: [RFC PATCH 6/9] kvm/x86: Add ARCH_CAP_VIRTUAL_ENUM for guest MSR_IA32_ARCH_CAPABILITIES
Quoted 83 lines
-----Original Message-----
From: Yang, Weijiang <[email protected]>
Sent: Thursday, December 29, 2022 4:39 PM
To: Zhang, Chen <[email protected]>
Cc: Gao, Chao <[email protected]>; Pawan Gupta <[email protected]>; Paolo Bonzini <[email protected]>; Christopherson,, Sean <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 6/9] kvm/x86: AddARCH_CAP_VIRTUAL_ENUMfor guestMSR_IA32_ARCH_CAPABILITIESQuoted 49 lines · Chen
Quoted 49 lines
-----Original Message-----
From: Yang, Weijiang <[email protected]>
Sent: Thursday, December 29, 2022 3:03 PM
To: Zhang, Chen <[email protected]>
Cc: Gao, Chao <[email protected]>; Pawan Gupta <[email protected]>; Paolo Bonzini <[email protected]>; Christopherson,, Sean <[email protected]>; H. Peter Anvin <[email protected]>; Dave Hansen <[email protected]>; Borislav Petkov <[email protected]>; Ingo Molnar <[email protected]>; Thomas Gleixner <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 6/9] kvm/x86: AddARCH_CAP_VIRTUAL_ENUMfor guestMSR_IA32_ARCH_CAPABILITIESOn 12/29/2022 10:58 AM, Zhang, Chen wrote:
[...]
Quoted 17 lines
Quoted 9 lines
\
Quoted 8 lines
arch/x86/kvm/x86.c +1
-0+ ARCH_CAP_VIRTUAL_ENUM) static u64 kvm_get_arch_capabilities(void) {@@ -1607,6 +1611,13 @@ static u64 kvm_get_arch_capabilities(void)*/ data |=ARCH_CAP_PSCHANGE_MC_NO ;Quoted 8 lines
arch/x86/kvm/x86.c static u64 kvm_get_arch_capabilities(void) · line 1619
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+/* +* Virtual MSRs can allow guests to notify VMM whether or not +* they are using specific software mitigation, allowing a VMM +* to enable there hardware control only where necessary. +*/ + data |=ARCH_CAP_VIRTUAL_ENUM ;IMO, this is: data &=
ARCH_CAP_VIRTUAL_ENUM; because it requires platform support.Quoted 3 lines
Intel defined the virtual MSRs for software mitigations for all platforms. KVM should be unconditionally opened it for the software mitigation in
migration pools.
Quoted 2 lines
For example migration from the old platform to the new platform. Please check the Software Mitigations in Migration Pools section in
documents:
Quoted 5 lines
https://www.intel.com/content/www/us/en/developer/articles/technical
/s oftware-security-guidance/technical-documentation/branch-history-inj ec
tion.htmlQuoted 6 lines
Quoted 5 lines
If this series running on old platforms, how VMM can set specific vmcs fields,
e.g., “virtualize IA32_SPEC_CTRL” VM-execution control, to mitigate guest issues?
Enable the virtual MSRs does not means to enable the “virtualize
IA32_SPEC_CTRL”.
Quoted 1 line
KVM will check "
cpu_has_virt_spec_ctrl()" before set specific VMCS.What I'm concerned is, if the feature is not supported on some platforms, unconditionally
exposing to guest would make it wrongly "think" the feature is supported on HW, but actually
Quoted 7 lines
WMM can do nothing to mitigate issues. Read through the patch set, looks like guest cannot
even get any errors during feature configuration in this case. So I suggest add more
sanity checks. Did I miss something?
Understand your concern, but I think no need to concern about it. Although we have unconditionally turned on virtual MSRs in all platforms, The detailed virtual MSR bits still can tell guest what feature is supported on HW. QEMU will setup the MSR_VIRTUAL_MITIGATION_ENUM(0x50000001) bits
Like the BHB_CLEAR_SEQ_S_SUPPORT/RETPOLINE_S_SUPPORT before boot VMs.
Thanks
Chen
Quoted 30 lines
Quoted 11 lines
Thanks
ChenQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
arch/x86/kvm/x86.c +1
-0+/* * If we're doing cache flushes (either "always" or "cond") * we will do one whenever the guest does a vmlaunch/vmresume.
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 9 lines
Review static int kvm_get_msr_feature(struct
Quoted 1 line
(quoted diff) +0
-0@@ -1657,6 +1668,9 @@ static int kvm_get_msr_feature(struct
kvm_msr_entry*msr)arch/x86/kvm/x86.c static int kvm_get_msr_feature(struct kvm_msr_entry *msr) · line 1672
Quoted 5 lines
arch/x86/kvm/x86.c +2
-0case MSR_IA32_UCODE_REV :rdmsrl_safe (msr->index, &msr->data);break ; +case MSR_VIRTUAL_ENUMERATION : + msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;Need to check bit 63 of host
MSR_ARCH_CAPABILITIESbefore expose the feature.Refer to the above comments.
Thanks
ChenQuoted 4 lines
Quoted 4 lines
arch/x86/kvm/x86.c +1
-0+break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
RFC PATCH 7/9 kvm/x86: Add MSR_VIRTUAL_MITIGATION_ENUM/CTRL emulation
In reply to Zhang ChenIntroduce Intel virtual MSR_VIRTUAL_MITIGATION_ENUM(0x50000001) and MSR_VIRTUAL_MITIGATION_CTRL(0x50000002).
The MSR_VIRTUAL_MITIGATION_ENUM to tell guest about supported mitigations and enable the MSR_VIRTUAL_MITIGATION_CTRL virtual MSRs for guest, VMM will help to setup virtual spec ctrl mask for SPEC_CTRL_RRSBA_DIS_S, SPEC_CTRL_BHI_DIS_S as guest's needs.
--- arch/x86/kvm/vmx/vmx.h | 16 ++++++++++++++++ arch/x86/kvm/x86.c | 7 +++++++ 2 files changed, 23 insertions(+)
arch/x86/kvm/vmx/vmx.h +16 -0
@@ -29,6 +29,10 @@#endif #define MAX_NR_LOADSTORE_MSRS 8 +#define MITI_ENUM_SUPPORTED (MITI_ENUM_BHB_CLEAR_SEQ_S_SUPPORT | \ + MITI_ENUM_RETPOLINE_S_SUPPORT) +#define MITI_CTRL_USED (MITI_CTRL_BHB_CLEAR_SEQ_S_USED | \ + MITI_CTRL_RETPOLINE_S_USED)struct vmx_msrs {unsigned int nr;
@@ -301,6 +305,18 @@ struct vcpu_vmx {u64 msr_virtual_enumeration;u32 msr_ia32_umwait_control; +/* +* Guest read only, Only available if MITIGATION_CTRL_SUPPORT +* is enumerated. +*/ +u64 msr_virtual_mitigation_enum; + +/* +* Read/Write, Only available if MITIGATION_CTRL_SUPPORT +* is enumerated. +*/ +u64 msr_virtual_mitigation_ctrl; +/* * loaded_vmcs points to the VMCS currently used in this vcpu. For a * non-nested (L1) guest, it always points to vmcs01. For a nested
arch/x86/kvm/x86.c +7 -0
@@ -1533,6 +1533,8 @@ static const u32 emulated_msrs_all[] = {MSR_IA32_VMX_VMFUNC ,MSR_VIRTUAL_ENUMERATION , +MSR_VIRTUAL_MITIGATION_ENUM , +MSR_VIRTUAL_MITIGATION_CTRL ,MSR_K7_HWCR , MSR_KVM_POLL_CONTROL,
@@ -1570,6 +1572,7 @@ static const u32 msr_based_features_all[] = {MSR_IA32_ARCH_CAPABILITIES ,MSR_IA32_PERF_CAPABILITIES ,MSR_VIRTUAL_ENUMERATION , + MSR_VIRTUAL_MITIGATION_ENUM, };static u32 msr_based_features[ARRAY_SIZE (msr_based_features_all)];
@@ -1671,6 +1674,10 @@ static int kvm_get_msr_feature(struct kvm_msr_entry *msr)case MSR_VIRTUAL_ENUMERATION : msr->data =VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;break ; +case MSR_VIRTUAL_MITIGATION_ENUM : + msr->data =MITI_ENUM_BHB_CLEAR_SEQ_S_SUPPORT | +MITI_ENUM_RETPOLINE_S_SUPPORT ; +break ;default :return static_call (kvm_x86_get_msr_feature)(msr); }
Signature · 1 line
2.25.1
RFC PATCH 8/9 x86/kvm/vmx: Initialize SPEC_CTRL MASK for RRSBA
In reply to Zhang ChenVMMs can address mitigations issues in migration pool by applying the needed controls whenever the guest is operating on a newer processor. If a guest is using retpoline to mitigate intra-mode BTI in CPL0, the VMM can set RRSBA_DIS_S when the guest runs on hardware which enumerates RRSBA.
--- arch/x86/kvm/vmx/vmx.c | 57 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 56 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +56 -1
@@ -2007,6 +2007,20 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return 1 ; msr_info->data = vmx->msr_virtual_enumeration;break ; +case MSR_VIRTUAL_MITIGATION_ENUM : +if (!msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; + msr_info->data = vmx->msr_virtual_mitigation_enum; +break ; +case MSR_VIRTUAL_MITIGATION_CTRL : +if (!msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; + msr_info->data = vmx->msr_virtual_mitigation_ctrl; +break ;default : find_uret_msr: msr =vmx_find_uret_msr (vmx, msr_info->index);
@@ -2056,7 +2070,7 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)struct vmx_uret_msr *msr;int ret =0 ;u32 msr_index = msr_info->index; -u64 data = msr_info->data; +u64 data = msr_info->data, arch_msr;u32 index;switch (msr_index) {
@@ -2390,6 +2404,46 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) vmx->msr_virtual_enumeration = data &VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;break ; +case MSR_VIRTUAL_MITIGATION_ENUM : +if (msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; +if (data & ~MITI_ENUM_SUPPORTED ) +return 1 ; + vmx->msr_virtual_mitigation_enum = data; +break ; +case MSR_VIRTUAL_MITIGATION_CTRL : +if (!msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; +if (data & ~MITI_CTRL_USED ) +return 1 ; + +if (boot_cpu_has (X86_FEATURE_ARCH_CAPABILITIES )) +rdmsrl (MSR_IA32_ARCH_CAPABILITIES , arch_msr); + +if (data &MITI_CTRL_RETPOLINE_S_USED && +boot_cpu_has (X86_FEATURE_RRSBA_CTRL ) && + arch_msr &ARCH_CAP_RRSBA ) + vmx->spec_ctrl_mask |=SPEC_CTRL_RRSBA_DIS_S ; +else + vmx->spec_ctrl_mask &= ~SPEC_CTRL_RRSBA_DIS_S ; + +if (cpu_has_virt_spec_ctrl ()) { +vmcs_write64 (IA32_SPEC_CTRL_MASK , vmx->spec_ctrl_mask); + }else if (vmx->spec_ctrl_mask) { +pr_err_once ("Virtual spec ctrl is missing. Cannot keep " +"bits in %llx always set\n" , + vmx->spec_ctrl_mask); + vmx->spec_ctrl_mask =0 ; + } + + vmx->spec_ctrl = vmx->spec_ctrl | vmx->spec_ctrl_mask; + + vmx->msr_virtual_mitigation_ctrl = data; +break ;default : find_uret_msr:
@@ -4774,6 +4828,7 @@ static void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vmx->rmode.vm86_active =0 ; + vmx->msr_virtual_mitigation_ctrl =0 ;if (cpu_has_virt_spec_ctrl ()) { vmx->spec_ctrl_mask =0 ;vmcs_write64 (IA32_SPEC_CTRL_MASK , vmx->spec_ctrl_mask);
Signature · 1 line
2.25.1
Re: [RFC PATCH 8/9] x86/kvm/vmx: Initialize SPEC_CTRL MASK for RRSBA
Quoted 67 lines · Zhang Chen
VMMs can address mitigations issues in migration pool by applying the needed controls whenever the guest is operating on a newer processor. If a guest is using retpoline to mitigate intra-mode BTI in CPL0, the VMM can set
RRSBA_DIS_Swhen the guest runs on hardware which enumerates RRSBA.Signed-off-by Zhang Chen <[email protected]>---
arch/x86/kvm/vmx/vmx.c| 57 +++++++++++++++++++++++++++++++++++++++++-1 file changed, 56 insertions(+), 1 deletion(-)arch/x86/kvm/vmx/vmx.c +32
-1@@ -2007,6 +2007,20 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return 1 ; msr_info->data = vmx->msr_virtual_enumeration;break ; +case MSR_VIRTUAL_MITIGATION_ENUM : +if (!msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; + msr_info->data = vmx->msr_virtual_mitigation_enum; +break ; +case MSR_VIRTUAL_MITIGATION_CTRL : +if (!msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; + msr_info->data = vmx->msr_virtual_mitigation_ctrl; +break ;default : find_uret_msr: msr =vmx_find_uret_msr (vmx, msr_info->index);@@ -2056,7 +2070,7 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)struct vmx_uret_msr *msr;int ret =0 ;u32 msr_index = msr_info->index; -u64 data = msr_info->data; +u64 data = msr_info->data, arch_msr;u32 index;switch (msr_index) {@@ -2390,6 +2404,46 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) vmx->msr_virtual_enumeration = data &VIRT_ENUM_MITIGATION_CTRL_SUPPORT ;break ; +case MSR_VIRTUAL_MITIGATION_ENUM : +if (msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; +if (data & ~MITI_ENUM_SUPPORTED ) +return 1 ; + vmx->msr_virtual_mitigation_enum = data; +break ; +case MSR_VIRTUAL_MITIGATION_CTRL : +if (!msr_info->host_initiated && + !(vmx->msr_virtual_enumeration & +VIRT_ENUM_MITIGATION_CTRL_SUPPORT )) +return 1 ; +if (data & ~MITI_CTRL_USED ) +return 1 ; +
arch/x86/kvm/vmx/vmx.c static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 2434
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +9
-0+if (boot_cpu_has (X86_FEATURE_ARCH_CAPABILITIES )) +rdmsrl (MSR_IA32_ARCH_CAPABILITIES , arch_msr); + +if (data &MITI_CTRL_RETPOLINE_S_USED && +boot_cpu_has (X86_FEATURE_RRSBA_CTRL ) && + arch_msr &ARCH_CAP_RRSBA ) + vmx->spec_ctrl_mask |=SPEC_CTRL_RRSBA_DIS_S ; +else + vmx->spec_ctrl_mask &= ~SPEC_CTRL_RRSBA_DIS_S ;
IIUC, only above chunk belongs to this patch. Other changes should be moved to patch 7, which deals with MSRs emulation.
RFC PATCH 9/9 x86/kvm/vmx: Initialize SPEC_CTRL MASK for BHI
In reply to Zhang ChenVMMs can address mitigations issues in migration pool by applying the needed controls whenever the guest is operating on a newer processor. If a guest is using the BHB-clearing sequence on transitions into CPL0 to mitigate BHI, the VMM can use the “virtual IA32_SPEC_CTRL” VM-execution control to set BHI_DIS_S on newer hardware which does not enumerate BHI_NO.
--- arch/x86/kvm/vmx/vmx.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kvm/vmx/vmx.c +7 -0
@@ -2431,6 +2431,13 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)else vmx->spec_ctrl_mask &= ~SPEC_CTRL_RRSBA_DIS_S ; +if (data &MITI_CTRL_BHB_CLEAR_SEQ_S_USED && +kvm_cpu_cap_has (X86_FEATURE_BHI_CTRL ) && + !(arch_msr &ARCH_CAP_BHI_NO )) + vmx->spec_ctrl_mask |=SPEC_CTRL_BHI_DIS_S ; +else + vmx->spec_ctrl_mask &= ~SPEC_CTRL_BHI_DIS_S ; +if (cpu_has_virt_spec_ctrl ()) {vmcs_write64 (IA32_SPEC_CTRL_MASK , vmx->spec_ctrl_mask); }else if (vmx->spec_ctrl_mask) {
Signature · 1 line
2.25.1