SEV-ES hypervisor support
Subsystems (4)
v5 cover SEV-ES hypervisor support
From: Tom Lendacky <[email protected]>
This patch series provides support for running SEV-ES guests under KVM.
Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the SEV support to protect the guest register state from the hypervisor. See "AMD64 Architecture Programmer's Manual Volume 2: System Programming", section "15.35 Encrypted State (SEV-ES)" [1].
In order to allow a hypervisor to perform functions on behalf of a guest, there is architectural support for notifying a guest's operating system when certain types of VMEXITs are about to occur. This allows the guest to selectively share information with the hypervisor to satisfy the requested function. The notification is performed using a new exception, the VMM Communication exception (#VC). The information is shared through the Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction. The GHCB format and the protocol for using it is documented in "SEV-ES Guest-Hypervisor Communication Block Standardization" [2].
Under SEV-ES, a vCPU save area (VMSA) must be encrypted. SVM is updated to build the initial VMSA and then encrypt it before running the guest. Once encrypted, it must not be modified by the hypervisor. Modification of the VMSA will result in the VMRUN instruction failing with a SHUTDOWN exit code. KVM must support the VMGEXIT exit code in order to perform the necessary functions required of the guest. The GHCB is used to exchange the information needed by both the hypervisor and the guest.
Register data from the GHCB is copied into the KVM register variables and accessed as usual during handling of the exit. Upon return to the guest, updated registers are copied back to the GHCB for the guest to act upon.
There are changes to some of the intercepts that are needed under SEV-ES. For example, CR0 writes cannot be intercepted, so the code needs to ensure that the intercept is not enabled during execution or that the hypervisor does not try to read the register as part of exit processing. Another example is shutdown processing, where the vCPU cannot be directly reset.
Support is added to handle VMGEXIT events and implement the GHCB protocol. This includes supporting standard exit events, like a CPUID instruction intercept, to new support, for things like AP processor booting. Much of the existing SVM intercept support can be re-used by setting the exit code information from the VMGEXIT and calling the appropriate intercept handlers.
Finally, to launch and run an SEV-ES guest requires changes to the vCPU initialization, loading and execution.
[1] https://www.amd.com/system/files/TechDocs/24593.pdf
[2] https://developer.amd.com/wp-content/resources/56421.pdf
---
These patches are based on the KVM queue branch:
https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
dc924b062488 ("KVM: SVM: check CR4 changes against vcpu->arch")
A version of the tree can also be found at:
https://github.com/AMDESE/linux/tree/sev-es-v5
This tree has one addition patch that is not yet part of the queue tree that is required to run any SEV guest:
[PATCH] KVM: x86: adjust SEV for commit 7e8e6eed75e
https://lore.kernel.org/kvm/[email protected]/
Changes from v4:
- Updated the tracking support for CR0/CR4
Changes from v3:
- Some krobot fixes.
- Some checkpatch cleanups.
Changes from v2:
- Update the freeing of the VMSA page to account for the encrypted memory cache coherency feature as well as the VM page flush feature.
- Update the GHCB dump function with a bit more detail.
- Don't check for RAX being present as part of a string IO operation.
- Include RSI when syncing from GHCB to support KVM hypercall arguments.
- Add GHCB usage field validation check.
Changes from v1:
- Removed the VMSA indirection support:
- On LAUNCH_UPDATE_VMSA, sync traditional VMSA over to the new SEV-ES VMSA area to be encrypted.
- On VMGEXIT VMEXIT, directly copy valid registers into vCPU arch register array from GHCB. On VMRUN (following a VMGEXIT), directly copy dirty vCPU arch registers to GHCB.
- Removed reg_read_override()/reg_write_override() KVM ops.
- Added VMGEXIT exit-reason validation.
- Changed kvm_vcpu_arch variable vmsa_encrypted to guest_state_protected
- Updated the tracking support for EFER/CR0/CR4/CR8 to minimize changes to the x86.c code
- Updated __set_sregs to not set any register values (previously supported setting the tracked values of EFER/CR0/CR4/CR8)
- Added support for reporting SMM capability at the VM-level. This allows an SEV-ES guest to indicate SMM is not supported
- Updated FPU support to check for a guest FPU save area before using it. Updated SVM to free guest FPU for an SEV-ES guest during KVM create_vcpu op.
- Removed changes to the kvm_skip_emulated_instruction()
- Added VMSA validity checks before invoking LAUNCH_UPDATE_VMSA
- Minor code restructuring in areas for better readability
Tom Lendacky (34):
x86/cpu: Add VM page flush MSR availablility as a CPUID feature
KVM: SVM: Remove the call to sev_platform_status() during setup
KVM: SVM: Add support for SEV-ES capability in KVM
KVM: SVM: Add GHCB accessor functions for retrieving fields
KVM: SVM: Add support for the SEV-ES VMSA
KVM: x86: Mark GPRs dirty when written
KVM: SVM: Add required changes to support intercepts under SEV-ES
KVM: SVM: Prevent debugging under SEV-ES
KVM: SVM: Do not allow instruction emulation under SEV-ES
KVM: SVM: Cannot re-initialize the VMCB after shutdown with SEV-ES
KVM: SVM: Prepare for SEV-ES exit handling in the sev.c file
KVM: SVM: Add initial support for a VMGEXIT VMEXIT
KVM: SVM: Create trace events for VMGEXIT processing
KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x002
KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x004
KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x100
KVM: SVM: Create trace events for VMGEXIT MSR protocol processing
KVM: SVM: Support MMIO for an SEV-ES guest
KVM: SVM: Support string IO operations for an SEV-ES guest
KVM: SVM: Add support for EFER write traps for an SEV-ES guest
KVM: SVM: Add support for CR0 write traps for an SEV-ES guest
KVM: SVM: Add support for CR4 write traps for an SEV-ES guest
KVM: SVM: Add support for CR8 write traps for an SEV-ES guest
KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
KVM: SVM: Do not report support for SMM for an SEV-ES guest
KVM: SVM: Guest FPU state save/restore not needed for SEV-ES guest
KVM: SVM: Add support for booting APs for an SEV-ES guest
KVM: SVM: Add NMI support for an SEV-ES guest
KVM: SVM: Set the encryption mask for the SVM host save area
KVM: SVM: Update ASID allocation to support SEV-ES guests
KVM: SVM: Provide support for SEV-ES vCPU creation/loading
KVM: SVM: Provide support for SEV-ES vCPU loading
KVM: SVM: Provide an updated VMRUN invocation for SEV-ES guests
KVM: SVM: Provide support to launch and run an SEV-ES guest
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/kvm_host.h | 12 +- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/svm.h | 40 +- arch/x86/include/uapi/asm/svm.h | 28 + arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kernel/cpu/vmware.c | 12 +- arch/x86/kvm/Kconfig | 3 +- arch/x86/kvm/kvm_cache_regs.h | 51 +- arch/x86/kvm/svm/sev.c | 933 +++++++++++++++++++++++++++-- arch/x86/kvm/svm/svm.c | 446 +++++++++++--- arch/x86/kvm/svm/svm.h | 166 ++++- arch/x86/kvm/svm/vmenter.S | 50 ++ arch/x86/kvm/trace.h | 97 +++ arch/x86/kvm/vmx/vmx.c | 6 +- arch/x86/kvm/x86.c | 344 +++++++++-- arch/x86/kvm/x86.h | 9 + 17 files changed, 1962 insertions(+), 238 deletions(-)
base-commit: dc924b062488a0376aae41d3e0a27dc99f852a5e
Signature · 1 line
2.28.0
PATCH v5 1/34 x86/cpu: Add VM page flush MSR availablility as a CPUID feature
From: Tom Lendacky <[email protected]>
On systems that do not have hardware enforced cache coherency between encrypted and unencrypted mappings of the same physical page, the hypervisor can use the VM page flush MSR (0xc001011e) to flush the cache contents of an SEV guest page. When a small number of pages are being flushed, this can be used in place of issuing a WBINVD across all CPUs.
CPUID 0x8000001f_eax[2] is used to determine if the VM page flush MSR is available. Add a CPUID feature to indicate it is supported and define the MSR.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/scattered.c | 1 + 3 files changed, 3 insertions(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -237,6 +237,7 @@#define X86_FEATURE_VMCALL ( 8*32+18)/* "" Hypervisor supports the VMCALL instruction */ #define X86_FEATURE_VMW_VMMCALL ( 8*32+19)/* "" VMware prefers VMMCALL hypercall instruction */ #define X86_FEATURE_SEV_ES ( 8*32+20)/* AMD Secure Encrypted Virtualization - Encrypted State */ +#define X86_FEATURE_VM_PAGE_FLUSH ( 8*32+21)/* "" VM Page Flush MSR is supported */ /* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */ #define X86_FEATURE_FSGSBASE ( 9*32+ 0)/* RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/
arch/x86/include/asm/msr-index.h +1 -0
@@ -470,6 +470,7 @@#define MSR_AMD64_ICIBSEXTDCTL 0xc001103c#define MSR_AMD64_IBSOPDATA4 0xc001103d#define MSR_AMD64_IBS_REG_COUNT_MAX 8/* includes MSR_AMD64_IBSBRTARGET */ +#define MSR_AMD64_VM_PAGE_FLUSH 0xc001011e#define MSR_AMD64_SEV_ES_GHCB 0xc0010130#define MSR_AMD64_SEV 0xc0010131#define MSR_AMD64_SEV_ENABLED_BIT 0
arch/x86/kernel/cpu/scattered.c +1 -0
@@ -44,6 +44,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_SEV, CPUID_EAX, 1, 0x8000001f, 0 }, { X86_FEATURE_SEV_ES, CPUID_EAX, 3, 0x8000001f, 0 }, { X86_FEATURE_SME_COHERENT, CPUID_EAX, 10, 0x8000001f, 0 }, + { X86_FEATURE_VM_PAGE_FLUSH, CPUID_EAX, 2, 0x8000001f, 0 }, { 0, 0, 0, 0, 0 } };
Signature · 1 line
2.28.0
PATCH v5 2/34 KVM: SVM: Remove the call to sev_platform_status() during setup
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
When both KVM support and the CCP driver are built into the kernel instead of as modules, KVM initialization can happen before CCP initialization. As a result, sev_platform_status() will return a failure when it is called from sev_hardware_setup(), when this isn't really an error condition.
Since sev_platform_status() doesn't need to be called at this time anyway, remove the invocation from sev_hardware_setup().
--- arch/x86/kvm/svm/sev.c | 22 +--------------------- 1 file changed, 1 insertion(+), 21 deletions(-)
arch/x86/kvm/svm/sev.c +1 -21
@@ -1127,9 +1127,6 @@ void sev_vm_destroy(struct kvm *kvm)int __init sev_hardware_setup (void ) { -struct sev_user_data_status *status; -int rc; -/* Maximum number of encrypted guests supported simultaneously */ max_sev_asid =cpuid_ecx (0x8000001F );
@@ -1148,26 +1145,9 @@ int __init sev_hardware_setup(void)if (!sev_reclaim_asid_bitmap)return 1 ; - status =kmalloc (sizeof (*status),GFP_KERNEL ); -if (!status) -return 1 ; - -/* -* Check SEV platform status. -* -* PLATFORM_STATUS can be called in any state, if we failed to query -* the PLATFORM status then either PSP firmware does not support SEV -* feature or SEV firmware is dead. -*/ - rc =sev_platform_status (status,NULL ); -if (rc) - goto err; -pr_info ("SEV supported\n" ); -err: -kfree (status); -return rc; +return 0 ; }void sev_hardware_teardown (void )
Signature · 1 line
2.28.0
PATCH v5 3/34 KVM: SVM: Add support for SEV-ES capability in KVM
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Add support to KVM for determining if a system is capable of supporting SEV-ES as well as determining if a guest is an SEV-ES guest.
--- arch/x86/kvm/Kconfig | 3 ++- arch/x86/kvm/svm/sev.c | 47 ++++++++++++++++++++++++++++++++++-------- arch/x86/kvm/svm/svm.c | 20 +++++++++--------- arch/x86/kvm/svm/svm.h | 17 ++++++++++++++- 4 files changed, 66 insertions(+), 21 deletions(-)
arch/x86/kvm/Kconfig +2 -1
@@ -100,7 +100,8 @@ config KVM_AMD_SEV depends on KVM_AMD && X86_64 depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) help - Provides support for launching Encrypted VMs on AMD processors. + Provides support for launching Encrypted VMs (SEV) and Encrypted VMs + with Encrypted State (SEV-ES) on AMD processors. config KVM_MMU_AUDIT bool "Audit KVM MMU"
arch/x86/kvm/svm/sev.c +38 -9
74 lines not on this page · raw message
arch/x86/kvm/svm/svm.c +10 -10
@@ -187,9 +187,13 @@ static int vgif = true;module_param (vgif, int,0444 );/* enable/disable SEV support */ -static int sev =IS_ENABLED (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT ); +int sev =IS_ENABLED (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT );module_param (sev, int,0444 ); +/* enable/disable SEV-ES support */ +int sev_es =IS_ENABLED (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT ); +module_param (sev_es, int,0444 ); +static bool __read_mostly dump_invalid_vmcb =0 ;module_param (dump_invalid_vmcb, bool,0644 );
@@ -959,15 +963,11 @@ static __init int svm_hardware_setup(void)kvm_enable_efer_bits (EFER_SVME |EFER_LMSLE ); } -if (sev) { -if (boot_cpu_has (X86_FEATURE_SEV ) && -IS_ENABLED (CONFIG_KVM_AMD_SEV )) { - r =sev_hardware_setup (); -if (r) - sev = false; - }else { - sev = false; - } +if (IS_ENABLED (CONFIG_KVM_AMD_SEV ) && sev) { +sev_hardware_setup (); + }else { + sev = false; + sev_es = false; }svm_adjust_mmio_mask ();
arch/x86/kvm/svm/svm.h +16 -1
@@ -61,6 +61,7 @@ enum {struct kvm_sev_info {bool active;/* SEV enabled guest */ +bool es_active;/* SEV-ES enabled guest */ unsigned int asid;/* ASID used for this guest */ unsigned int handle;/* SEV firmware handle */ int fd;/* SEV device fd */
@@ -352,6 +353,9 @@ static inline bool gif_set(struct vcpu_svm *svm)#define MSR_CR3_LONG_MBZ_MASK 0xfff0000000000000U#define MSR_INVALID 0xffffffffU +extern int sev; +extern int sev_es; +u32 svm_msrpm_offset (u32 msr);u32 *svm_vcpu_alloc_msrpm (void );void svm_vcpu_init_msrpm (struct kvm_vcpu *vcpu,u32 *msrpm);
@@ -484,6 +488,17 @@ static inline bool sev_guest(struct kvm *kvm)#endif } +static inline bool sev_es_guest (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_AMD_SEV +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; + +return sev_guest (kvm) && sev->es_active; +#else +return false; +#endif +} +static inline bool svm_sev_enabled (void ) {return IS_ENABLED (CONFIG_KVM_AMD_SEV ) ? max_sev_asid :0 ;
@@ -496,7 +511,7 @@ int svm_register_enc_region(struct kvm *kvm,int svm_unregister_enc_region (struct kvm *kvm,struct kvm_enc_region *range);void pre_sev_run (struct vcpu_svm *svm,int cpu); -int __init sev_hardware_setup (void ); +void __init sev_hardware_setup (void );void sev_hardware_teardown (void );#endif
Signature · 1 line
2.28.0
PATCH v5 4/34 KVM: SVM: Add GHCB accessor functions for retrieving fields
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Update the GHCB accessor functions to add functions for retrieve GHCB fields by name. Update existing code to use the new accessor functions.
--- arch/x86/include/asm/svm.h | 10 ++++++++++ arch/x86/kernel/cpu/vmware.c | 12 ++++++------ 2 files changed, 16 insertions(+), 6 deletions(-)
arch/x86/include/asm/svm.h +10 -0
@@ -379,6 +379,16 @@ struct vmcb { (unsigned long *)&ghcb->save.valid_bitmap); \ } \ \ + static inline u64 ghcb_get_##field(struct ghcb *ghcb) \ + { \ + return ghcb->save.field; \ + } \ + \ + static inline u64 ghcb_get_##field##_if_valid(struct ghcb *ghcb) \ + { \ + return ghcb_##field##_is_valid(ghcb) ? ghcb->save.field : 0; \ + } \ + \ static inline void ghcb_set_##field(struct ghcb *ghcb, u64 value) \ { \ __set_bit(GHCB_BITMAP_IDX(field), \
arch/x86/kernel/cpu/vmware.c +6 -6
@@ -501,12 +501,12 @@ static bool vmware_sev_es_hcall_finish(struct ghcb *ghcb, struct pt_regs *regs) ghcb_rbp_is_valid(ghcb)))return false; - regs->bx = ghcb->save.rbx; - regs->cx = ghcb->save.rcx; - regs->dx = ghcb->save.rdx; - regs->si = ghcb->save.rsi; - regs->di = ghcb->save.rdi; - regs->bp = ghcb->save.rbp; + regs->bx =ghcb_get_rbx (ghcb); + regs->cx =ghcb_get_rcx (ghcb); + regs->dx =ghcb_get_rdx (ghcb); + regs->si =ghcb_get_rsi (ghcb); + regs->di =ghcb_get_rdi (ghcb); + regs->bp =ghcb_get_rbp (ghcb);return true; }
Signature · 1 line
2.28.0
PATCH v5 5/34 KVM: SVM: Add support for the SEV-ES VMSA
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Allocate a page during vCPU creation to be used as the encrypted VM save area (VMSA) for the SEV-ES guest. Provide a flag in the kvm_vcpu_arch structure that indicates whether the guest state is protected.
When freeing a VMSA page that has been encrypted, the cache contents must be flushed using the MSR_AMD64_VM_PAGE_FLUSH before freeing the page.
--- arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/kvm/svm/sev.c | 67 +++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 24 +++++++++++- arch/x86/kvm/svm/svm.h | 5 +++ 4 files changed, 97 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -0
@@ -805,6 +805,9 @@ struct kvm_vcpu_arch {*/ bool enforce; } pv_cpuid; + +/* Protected Guests */ +bool guest_state_protected; };struct kvm_lpage_info {
arch/x86/kvm/svm/sev.c +67 -0
79 lines not on this page · raw message
arch/x86/kvm/svm/svm.c +22 -2
@@ -1289,6 +1289,7 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm;struct page *vmcb_page; +struct page *vmsa_page =NULL ;int err;BUILD_BUG_ON (offsetof(struct vcpu_svm , vcpu) !=0 );
@@ -1299,9 +1300,19 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu)if (!vmcb_page) goto out; +if (sev_es_guest (svm->vcpu.kvm)) { +/* +* SEV-ES guests require a separate VMSA page used to contain +* the encrypted register state of the guest. +*/ + vmsa_page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); +if (!vmsa_page) + goto error_free_vmcb_page; + } + err =avic_init_vcpu (svm);if (err) - goto error_free_vmcb_page; + goto error_free_vmsa_page;/* We initialize this flag to true to make sure that the is_running * bit would be set the first time the vcpu is loaded.
@@ -1311,12 +1322,16 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu) svm->msrpm =svm_vcpu_alloc_msrpm ();if (!svm->msrpm) - goto error_free_vmcb_page; + goto error_free_vmsa_page;svm_vcpu_init_msrpm (vcpu, svm->msrpm); svm->vmcb =page_address (vmcb_page); svm->vmcb_pa =__sme_set (page_to_pfn (vmcb_page) <<PAGE_SHIFT ); + +if (vmsa_page) + svm->vmsa =page_address (vmsa_page); + svm->asid_generation =0 ;init_vmcb (svm);
@@ -1325,6 +1340,9 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu)return 0 ; +error_free_vmsa_page: +if (vmsa_page) +__free_page (vmsa_page); error_free_vmcb_page:__free_page (vmcb_page); out:
@@ -1352,6 +1370,8 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu)svm_free_nested (svm); +sev_free_vcpu (vcpu); +__free_page (pfn_to_page (__sme_clr (svm->vmcb_pa) >>PAGE_SHIFT ));__free_pages (virt_to_page (svm->msrpm),MSRPM_ALLOC_ORDER ); }
arch/x86/kvm/svm/svm.h +5 -0
@@ -168,6 +168,10 @@ struct vcpu_svm {DECLARE_BITMAP (read,MAX_DIRECT_ACCESS_MSRS );DECLARE_BITMAP (write,MAX_DIRECT_ACCESS_MSRS ); } shadow_msr_intercept; + +/* SEV-ES support */ +struct vmcb_save_area *vmsa; +struct ghcb *ghcb; };struct svm_cpu_data {
@@ -513,5 +517,6 @@ int svm_unregister_enc_region(struct kvm *kvm,void pre_sev_run (struct vcpu_svm *svm,int cpu);void __init sev_hardware_setup (void );void sev_hardware_teardown (void ); +void sev_free_vcpu (struct kvm_vcpu *vcpu);#endif
Signature · 1 line
2.28.0
PATCH v5 6/34 KVM: x86: Mark GPRs dirty when written
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
When performing VMGEXIT processing for an SEV-ES guest, register values will be synced between KVM and the GHCB. Prepare for detecting when a GPR has been updated (marked dirty) in order to determine whether to sync the register to the GHCB.
--- arch/x86/kvm/kvm_cache_regs.h | 51 ++++++++++++++++++----------------- 1 file changed, 26 insertions(+), 25 deletions(-)
arch/x86/kvm/kvm_cache_regs.h +26 -25
@@ -9,6 +9,31 @@ (X86_CR4_PVI |X86_CR4_DE |X86_CR4_PCE |X86_CR4_OSFXSR \ | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD | X86_CR4_FSGSBASE) +static inline bool kvm_register_is_available (struct kvm_vcpu *vcpu, +enum kvm_reg reg) +{ +return test_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); +} + +static inline bool kvm_register_is_dirty (struct kvm_vcpu *vcpu, +enum kvm_reg reg) +{ +return test_bit (reg, (unsigned long *)&vcpu->arch.regs_dirty); +} + +static inline void kvm_register_mark_available (struct kvm_vcpu *vcpu, +enum kvm_reg reg) +{ +__set_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); +} + +static inline void kvm_register_mark_dirty (struct kvm_vcpu *vcpu, +enum kvm_reg reg) +{ +__set_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); +__set_bit (reg, (unsigned long *)&vcpu->arch.regs_dirty); +} +#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\ { \
@@ -18,6 +43,7 @@ static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ unsigned long val) \ { \ vcpu->arch.regs[VCPU_REGS_##uname] = val; \ + kvm_register_mark_dirty(vcpu, VCPU_REGS_##uname); \ } BUILD_KVM_GPR_ACCESSORS(rax, RAX)BUILD_KVM_GPR_ACCESSORS (rbx,RBX )
@@ -37,31 +63,6 @@ BUILD_KVM_GPR_ACCESSORS(r14, R14) BUILD_KVM_GPR_ACCESSORS(r15, R15)#endif -static inline bool kvm_register_is_available (struct kvm_vcpu *vcpu, -enum kvm_reg reg) -{ -return test_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); -} - -static inline bool kvm_register_is_dirty (struct kvm_vcpu *vcpu, -enum kvm_reg reg) -{ -return test_bit (reg, (unsigned long *)&vcpu->arch.regs_dirty); -} - -static inline void kvm_register_mark_available (struct kvm_vcpu *vcpu, -enum kvm_reg reg) -{ -__set_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); -} - -static inline void kvm_register_mark_dirty (struct kvm_vcpu *vcpu, -enum kvm_reg reg) -{ -__set_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); -__set_bit (reg, (unsigned long *)&vcpu->arch.regs_dirty); -} -static inline unsigned long kvm_register_read (struct kvm_vcpu *vcpu,int reg) {if (WARN_ON_ONCE ((unsigned int )reg >=NR_VCPU_REGS ))
Signature · 1 line
2.28.0
PATCH v5 7/34 KVM: SVM: Add required changes to support intercepts under SEV-ES
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
When a guest is running under SEV-ES, the hypervisor cannot access the guest register state. There are numerous places in the KVM code where certain registers are accessed that are not allowed to be accessed (e.g. RIP, CR0, etc). Add checks to prevent register accesses and add intercept update support at various points within the KVM code.
Also, when handling a VMGEXIT, exceptions are passed back through the GHCB. Since the RDMSR/WRMSR intercepts (may) inject a #GP on error, update the SVM intercepts to handle this for SEV-ES guests.
--- arch/x86/include/asm/svm.h | 3 +- arch/x86/kvm/svm/svm.c | 111 +++++++++++++++++++++++++++++++++---- arch/x86/kvm/x86.c | 6 +- 3 files changed, 107 insertions(+), 13 deletions(-)
arch/x86/include/asm/svm.h +2 -1
@@ -178,7 +178,8 @@ struct __attribute__ ((__packed__)) vmcb_control_area {#define LBR_CTL_ENABLE_MASK BIT_ULL(0)#define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1) -#define SVM_INTERRUPT_SHADOW_MASK 1 +#define SVM_INTERRUPT_SHADOW_MASK BIT_ULL(0) +#define SVM_GUEST_INTERRUPT_MASK BIT_ULL(1)#define SVM_IOIO_STR_SHIFT 2#define SVM_IOIO_REP_SHIFT 3
arch/x86/kvm/svm/svm.c +101 -10
184 lines not on this page · raw message
arch/x86/kvm/x86.c +4 -2
@@ -4018,7 +4018,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) {int idx; -if (vcpu->preempted) +if (vcpu->preempted && !vcpu->arch.guest_state_protected) vcpu->arch.preempted_in_kernel = !kvm_x86_ops.get_cpl (vcpu);/*
@@ -8161,7 +8161,9 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu) {struct kvm_run *kvm_run = vcpu->run; - kvm_run->if_flag = (kvm_get_rflags (vcpu) &X86_EFLAGS_IF ) !=0 ; + kvm_run->if_flag = (vcpu->arch.guest_state_protected) + ?kvm_arch_interrupt_allowed (vcpu) + : (kvm_get_rflags (vcpu) &X86_EFLAGS_IF ) !=0 ; kvm_run->flags =is_smm (vcpu) ?KVM_RUN_X86_SMM :0 ; kvm_run->cr8 =kvm_get_cr8 (vcpu); kvm_run->apic_base =kvm_get_apic_base (vcpu);
Signature · 1 line
2.28.0
Quoted 3 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
This patch series provides support for running SEV-ES guests under KVM.
I cut the first send of this series short and resent it with a corrected email address for Sean (since he is copied on all the patches), so please look at the subsequent submission.
Sorry about that.
Tom