SEV-ES hypervisor support
Subsystems (4)
cover SEV-ES hypervisor support
From: Tom Lendacky <[email protected]>
This patch series provides support for running SEV-ES guests under KVM.
Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the SEV support to protect the guest register state from the hypervisor. See "AMD64 Architecture Programmer's Manual Volume 2: System Programming", section "15.35 Encrypted State (SEV-ES)" [1].
In order to allow a hypervisor to perform functions on behalf of a guest, there is architectural support for notifying a guest's operating system when certain types of VMEXITs are about to occur. This allows the guest to selectively share information with the hypervisor to satisfy the requested function. The notification is performed using a new exception, the VMM Communication exception (#VC). The information is shared through the Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction. The GHCB format and the protocol for using it is documented in "SEV-ES Guest-Hypervisor Communication Block Standardization" [2].
Under SEV-ES, a vCPU save area (VMSA) must be encrypted. SVM is updated to build the initial VMSA and then encrypt it before running the guest. Once encrypted, it must not be modified by the hypervisor. Modification of the VMSA will result in the VMRUN instruction failing with a SHUTDOWN exit code. KVM must support the VMGEXIT exit code in order to perform the necessary functions required of the guest. The GHCB is used to exchange the information needed by both the hypervisor and the guest.
To simplify access to the VMSA and the GHCB, SVM uses an accessor function to obtain the address of the either the VMSA or the GHCB, depending on the stage of execution of the guest.
There are changes to some of the intercepts that are needed under SEV-ES. For example, CR0 writes cannot be intercepted, so the code needs to ensure that the intercept is not enabled during execution or that the hypervisor does not try to read the register as part of exit processing. Another example is shutdown processing, where the vCPU cannot be directly reset.
Support is added to handle VMGEXIT events and implement the GHCB protocol. This includes supporting standard exit events, like a CPUID instruction intercept, to new support, for things like AP processor booting. Much of the existing SVM intercept support can be re-used by setting the exit code information from the VMGEXIT and calling the appropriate intercept handlers.
Finally, to launch and run an SEV-ES guest requires changes to the vCPU initialization, loading and execution.
[1] https://www.amd.com/system/files/TechDocs/24593.pdf
[2] https://developer.amd.com/wp-content/resources/56421.pdf
---
These patches are based on a commit of the KVM next branch. However, I had to backport recent SEV-ES guest patches (a previous series to the actual patches that are now in the tip tree) into my development branch, since there are prereq patches needed by this series. As a result, this patch series will not successfully build or apply to the KVM next branch as is.
A version of the tree can be found at:
https://github.com/AMDESE/linux/tree/sev-es-5.8-v3
Tom Lendacky (35):
KVM: SVM: Remove the call to sev_platform_status() during setup
KVM: SVM: Add support for SEV-ES capability in KVM
KVM: SVM: Add indirect access to the VM save area
KVM: SVM: Make GHCB accessor functions available to the hypervisor
KVM: SVM: Add initial support for SEV-ES GHCB access to KVM
KVM: SVM: Add required changes to support intercepts under SEV-ES
KVM: SVM: Modify DRx register intercepts for an SEV-ES guest
KVM: SVM: Prevent debugging under SEV-ES
KVM: SVM: Do not emulate MMIO under SEV-ES
KVM: SVM: Cannot re-initialize the VMCB after shutdown with SEV-ES
KVM: SVM: Prepare for SEV-ES exit handling in the sev.c file
KVM: SVM: Add initial support for a VMGEXIT VMEXIT
KVM: SVM: Create trace events for VMGEXIT processing
KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x002
KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x004
KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x100
KVM: SVM: Create trace events for VMGEXIT MSR protocol processing
KVM: SVM: Support MMIO for an SEV-ES guest
KVM: SVM: Support port IO operations for an SEV-ES guest
KVM: SVM: Add SEV/SEV-ES support for intercepting INVD
KVM: SVM: Add support for EFER write traps for an SEV-ES guest
KVM: SVM: Add support for CR0 write traps for an SEV-ES guest
KVM: SVM: Add support for CR4 write traps for an SEV-ES guest
KVM: SVM: Add support for CR8 write traps for an SEV-ES guest
KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
KVM: SVM: Guest FPU state save/restore not needed for SEV-ES guest
KVM: SVM: Add support for booting APs for an SEV-ES guest
KVM: X86: Update kvm_skip_emulated_instruction() for an SEV-ES guest
KVM: SVM: Add NMI support for an SEV-ES guest
KVM: SVM: Set the encryption mask for the SVM host save area
KVM: SVM: Update ASID allocation to support SEV-ES guests
KVM: SVM: Provide support for SEV-ES vCPU creation/loading
KVM: SVM: Provide support for SEV-ES vCPU loading
KVM: SVM: Provide an updated VMRUN invocation for SEV-ES guests
KVM: SVM: Provide support to launch and run an SEV-ES guest
arch/x86/include/asm/kvm_host.h | 16 + arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/svm.h | 35 +- arch/x86/include/uapi/asm/svm.h | 28 ++ arch/x86/kernel/cpu/vmware.c | 12 +- arch/x86/kvm/Kconfig | 3 +- arch/x86/kvm/cpuid.c | 1 + arch/x86/kvm/kvm_cache_regs.h | 30 +- arch/x86/kvm/mmu/mmu.c | 7 + arch/x86/kvm/svm/nested.c | 125 +++--- arch/x86/kvm/svm/sev.c | 590 ++++++++++++++++++++++++-- arch/x86/kvm/svm/svm.c | 704 ++++++++++++++++++++++++------- arch/x86/kvm/svm/svm.h | 357 ++++++++++++++-- arch/x86/kvm/svm/vmenter.S | 50 +++ arch/x86/kvm/trace.h | 99 +++++ arch/x86/kvm/vmx/vmx.c | 7 + arch/x86/kvm/x86.c | 357 ++++++++++++++-- arch/x86/kvm/x86.h | 8 + 18 files changed, 2070 insertions(+), 360 deletions(-)
Signature · 1 line
2.28.0
RFC PATCH 1/35 KVM: SVM: Remove the call to sev_platform_status() during setup
From: Tom Lendacky <[email protected]>
When both KVM support and the CCP driver are built into the kernel instead of as modules, KVM initialization happens before CCP initialization. As a result, sev_platform_status() will return a failure when it is called from sev_hardware_setup(), when this isn't really an error condition.
Since sev_platform_status() doesn't need to be called at this time anyway, remove the invocation from sev_hardware_setup().
--- arch/x86/kvm/svm/sev.c | 22 +--------------------- 1 file changed, 1 insertion(+), 21 deletions(-)
arch/x86/kvm/svm/sev.c +1 -21
@@ -1117,9 +1117,6 @@ void sev_vm_destroy(struct kvm *kvm)int __init sev_hardware_setup (void ) { -struct sev_user_data_status *status; -int rc; -/* Maximum number of encrypted guests supported simultaneously */ max_sev_asid =cpuid_ecx (0x8000001F );
@@ -1138,26 +1135,9 @@ int __init sev_hardware_setup(void)if (!sev_reclaim_asid_bitmap)return 1 ; - status =kmalloc (sizeof (*status),GFP_KERNEL ); -if (!status) -return 1 ; - -/* -* Check SEV platform status. -* -* PLATFORM_STATUS can be called in any state, if we failed to query -* the PLATFORM status then either PSP firmware does not support SEV -* feature or SEV firmware is dead. -*/ - rc =sev_platform_status (status,NULL ); -if (rc) - goto err; -pr_info ("SEV supported\n" ); -err: -kfree (status); -return rc; +return 0 ; }void sev_hardware_teardown (void )
Signature · 1 line
2.28.0
RFC PATCH 2/35 KVM: SVM: Add support for SEV-ES capability in KVM
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Add support to KVM for determining if a system is capable of supporting SEV-ES as well as determining if a guest is an SEV-ES guest.
--- arch/x86/kvm/Kconfig | 3 ++- arch/x86/kvm/svm/sev.c | 47 ++++++++++++++++++++++++++++++++++-------- arch/x86/kvm/svm/svm.c | 20 +++++++++--------- arch/x86/kvm/svm/svm.h | 17 ++++++++++++++- 4 files changed, 66 insertions(+), 21 deletions(-)
arch/x86/kvm/Kconfig +2 -1
@@ -99,7 +99,8 @@ config KVM_AMD_SEV depends on KVM_AMD && X86_64 depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) help - Provides support for launching Encrypted VMs on AMD processors. + Provides support for launching Encrypted VMs (SEV) and Encrypted VMs + with Encrypted State (SEV-ES) on AMD processors. config KVM_MMU_AUDIT bool "Audit KVM MMU"
arch/x86/kvm/svm/sev.c +38 -9
74 lines not on this page · raw message
arch/x86/kvm/svm/svm.c +10 -10
@@ -187,9 +187,13 @@ static int vgif = true;module_param (vgif, int,0444 );/* enable/disable SEV support */ -static int sev =IS_ENABLED (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT ); +int sev =IS_ENABLED (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT );module_param (sev, int,0444 ); +/* enable/disable SEV-ES support */ +int sev_es =IS_ENABLED (CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT ); +module_param (sev_es, int,0444 ); +static bool __read_mostly dump_invalid_vmcb =0 ;module_param (dump_invalid_vmcb, bool,0644 );
@@ -860,15 +864,11 @@ static __init int svm_hardware_setup(void)kvm_enable_efer_bits (EFER_SVME |EFER_LMSLE ); } -if (sev) { -if (boot_cpu_has (X86_FEATURE_SEV ) && -IS_ENABLED (CONFIG_KVM_AMD_SEV )) { - r =sev_hardware_setup (); -if (r) - sev = false; - }else { - sev = false; - } +if (IS_ENABLED (CONFIG_KVM_AMD_SEV ) && sev) { +sev_hardware_setup (); + }else { + sev = false; + sev_es = false; }svm_adjust_mmio_mask ();
arch/x86/kvm/svm/svm.h +16 -1
@@ -60,6 +60,7 @@ enum {struct kvm_sev_info {bool active;/* SEV enabled guest */ +bool es_active;/* SEV-ES enabled guest */ unsigned int asid;/* ASID used for this guest */ unsigned int handle;/* SEV firmware handle */ int fd;/* SEV device fd */
@@ -348,6 +349,9 @@ static inline bool gif_set(struct vcpu_svm *svm)#define MSR_CR3_LONG_RESERVED_MASK 0xfff0000000000fe7U#define MSR_INVALID 0xffffffffU +extern int sev; +extern int sev_es; +u32 svm_msrpm_offset (u32 msr);void svm_set_efer (struct kvm_vcpu *vcpu,u64 efer);void svm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0);
@@ -474,6 +478,17 @@ static inline bool sev_guest(struct kvm *kvm)#endif } +static inline bool sev_es_guest (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_AMD_SEV +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; + +return sev_guest (kvm) && sev->es_active; +#else +return false; +#endif +} +static inline bool svm_sev_enabled (void ) {return IS_ENABLED (CONFIG_KVM_AMD_SEV ) ? max_sev_asid :0 ;
@@ -486,7 +501,7 @@ int svm_register_enc_region(struct kvm *kvm,int svm_unregister_enc_region (struct kvm *kvm,struct kvm_enc_region *range);void pre_sev_run (struct vcpu_svm *svm,int cpu); -int __init sev_hardware_setup (void ); +void __init sev_hardware_setup (void );void sev_hardware_teardown (void );#endif
Signature · 1 line
2.28.0
RFC PATCH 3/35 KVM: SVM: Add indirect access to the VM save area
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
In order to later support accessing the GHCB structure in a similar way as the VM save area (VMSA) structure, change all accesses to the VMSA into function calls. Later on, this will allow the hypervisor support to decide between accessing the VMSA or GHCB in a central location. Accesses to a nested VMCB structure save area remain as direct save area accesses.
The functions are created using VMSA accessor macros.
--- arch/x86/kvm/svm/nested.c | 125 +++++++++++++++-------------- arch/x86/kvm/svm/svm.c | 165 +++++++++++++++++++------------------- arch/x86/kvm/svm/svm.h | 129 ++++++++++++++++++++++++++++- 3 files changed, 273 insertions(+), 146 deletions(-)
arch/x86/kvm/svm/nested.c +63 -62
@@ -367,28 +367,29 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,static void nested_prepare_vmcb_save (struct vcpu_svm *svm,struct vmcb *nested_vmcb) {/* Load the nested guest state */ - svm->vmcb->save.es = nested_vmcb->save.es; - svm->vmcb->save.cs = nested_vmcb->save.cs; - svm->vmcb->save.ss = nested_vmcb->save.ss; - svm->vmcb->save.ds = nested_vmcb->save.ds; - svm->vmcb->save.gdtr = nested_vmcb->save.gdtr; - svm->vmcb->save.idtr = nested_vmcb->save.idtr; +svm_es_write (svm, &nested_vmcb->save.es); +svm_cs_write (svm, &nested_vmcb->save.cs); +svm_ss_write (svm, &nested_vmcb->save.ss); +svm_ds_write (svm, &nested_vmcb->save.ds); +svm_gdtr_write (svm, &nested_vmcb->save.gdtr); +svm_idtr_write (svm, &nested_vmcb->save.idtr);kvm_set_rflags (&svm->vcpu, nested_vmcb->save.rflags);svm_set_efer (&svm->vcpu, nested_vmcb->save.efer);svm_set_cr0 (&svm->vcpu, nested_vmcb->save.cr0);svm_set_cr4 (&svm->vcpu, nested_vmcb->save.cr4); - svm->vmcb->save.cr2 = svm->vcpu.arch.cr2 = nested_vmcb->save.cr2; +svm_cr2_write (svm, nested_vmcb->save.cr2); + svm->vcpu.arch.cr2 = nested_vmcb->save.cr2;kvm_rax_write (&svm->vcpu, nested_vmcb->save.rax);kvm_rsp_write (&svm->vcpu, nested_vmcb->save.rsp);kvm_rip_write (&svm->vcpu, nested_vmcb->save.rip);/* In case we don't even reach vcpu_run, the fields are not updated */ - svm->vmcb->save.rax = nested_vmcb->save.rax; - svm->vmcb->save.rsp = nested_vmcb->save.rsp; - svm->vmcb->save.rip = nested_vmcb->save.rip; - svm->vmcb->save.dr7 = nested_vmcb->save.dr7; +svm_rax_write (svm, nested_vmcb->save.rax); +svm_rsp_write (svm, nested_vmcb->save.rsp); +svm_rip_write (svm, nested_vmcb->save.rip); +svm_dr7_write (svm, nested_vmcb->save.dr7); svm->vcpu.arch.dr6 = nested_vmcb->save.dr6; - svm->vmcb->save.cpl = nested_vmcb->save.cpl; +svm_cpl_write (svm, nested_vmcb->save.cpl); }static void nested_prepare_vmcb_control (struct vcpu_svm *svm)
@@ -451,7 +452,6 @@ int nested_svm_vmrun(struct vcpu_svm *svm)int ret;struct vmcb *nested_vmcb;struct vmcb *hsave = svm->nested.hsave; -struct vmcb *vmcb = svm->vmcb;struct kvm_host_map map;u64 vmcb_gpa;
@@ -460,7 +460,7 @@ int nested_svm_vmrun(struct vcpu_svm *svm)return 1 ; } - vmcb_gpa = svm->vmcb->save.rax; + vmcb_gpa =svm_rax_read (svm); ret =kvm_vcpu_map (&svm->vcpu,gpa_to_gfn (vmcb_gpa), &map);if (ret == -EINVAL ) {kvm_inject_gp (&svm->vcpu,0 );
@@ -481,7 +481,7 @@ int nested_svm_vmrun(struct vcpu_svm *svm) goto out; } - trace_kvm_nested_vmrun(svm->vmcb->save.rip, vmcb_gpa, + trace_kvm_nested_vmrun(svm_rip_read(svm), vmcb_gpa, nested_vmcb->save.rip, nested_vmcb->control.int_ctl, nested_vmcb->control.event_inj,
@@ -500,25 +500,25 @@ int nested_svm_vmrun(struct vcpu_svm *svm)* Save the old vmcb, so we don't need to pick what we save, but can * restore everything when a VMEXIT occurs */ - hsave->save.es = vmcb->save.es; - hsave->save.cs = vmcb->save.cs; - hsave->save.ss = vmcb->save.ss; - hsave->save.ds = vmcb->save.ds; - hsave->save.gdtr = vmcb->save.gdtr; - hsave->save.idtr = vmcb->save.idtr; + hsave->save.es = *svm_es_read (svm); + hsave->save.cs = *svm_cs_read (svm); + hsave->save.ss = *svm_ss_read (svm); + hsave->save.ds = *svm_ds_read (svm); + hsave->save.gdtr = *svm_gdtr_read (svm); + hsave->save.idtr = *svm_idtr_read (svm); hsave->save.efer = svm->vcpu.arch.efer; hsave->save.cr0 =kvm_read_cr0 (&svm->vcpu); hsave->save.cr4 = svm->vcpu.arch.cr4; hsave->save.rflags =kvm_get_rflags (&svm->vcpu); hsave->save.rip =kvm_rip_read (&svm->vcpu); - hsave->save.rsp = vmcb->save.rsp; - hsave->save.rax = vmcb->save.rax; + hsave->save.rsp =svm_rsp_read (svm); + hsave->save.rax =svm_rax_read (svm);if (npt_enabled) - hsave->save.cr3 = vmcb->save.cr3; + hsave->save.cr3 =svm_cr3_read (svm);else hsave->save.cr3 =kvm_read_cr3 (&svm->vcpu); -copy_vmcb_control_area (&hsave->control, &vmcb->control); +copy_vmcb_control_area (&hsave->control, &svm->vmcb->control); svm->nested.nested_run_pending =1 ;
@@ -544,20 +544,21 @@ int nested_svm_vmrun(struct vcpu_svm *svm)return ret; } -void nested_svm_vmloadsave (struct vmcb *from_vmcb,struct vmcb *to_vmcb) +void nested_svm_vmloadsave (struct vmcb_save_area *from_vmsa, +struct vmcb_save_area *to_vmsa) { - to_vmcb->save.fs = from_vmcb->save.fs; - to_vmcb->save.gs = from_vmcb->save.gs; - to_vmcb->save.tr = from_vmcb->save.tr; - to_vmcb->save.ldtr = from_vmcb->save.ldtr; - to_vmcb->save.kernel_gs_base = from_vmcb->save.kernel_gs_base; - to_vmcb->save.star = from_vmcb->save.star; - to_vmcb->save.lstar = from_vmcb->save.lstar; - to_vmcb->save.cstar = from_vmcb->save.cstar; - to_vmcb->save.sfmask = from_vmcb->save.sfmask; - to_vmcb->save.sysenter_cs = from_vmcb->save.sysenter_cs; - to_vmcb->save.sysenter_esp = from_vmcb->save.sysenter_esp; - to_vmcb->save.sysenter_eip = from_vmcb->save.sysenter_eip; + to_vmsa->fs = from_vmsa->fs; + to_vmsa->gs = from_vmsa->gs; + to_vmsa->tr = from_vmsa->tr; + to_vmsa->ldtr = from_vmsa->ldtr; + to_vmsa->kernel_gs_base = from_vmsa->kernel_gs_base; + to_vmsa->star = from_vmsa->star; + to_vmsa->lstar = from_vmsa->lstar; + to_vmsa->cstar = from_vmsa->cstar; + to_vmsa->sfmask = from_vmsa->sfmask; + to_vmsa->sysenter_cs = from_vmsa->sysenter_cs; + to_vmsa->sysenter_esp = from_vmsa->sysenter_esp; + to_vmsa->sysenter_eip = from_vmsa->sysenter_eip; }int nested_svm_vmexit (struct vcpu_svm *svm)
@@ -588,24 +589,24 @@ int nested_svm_vmexit(struct vcpu_svm *svm)/* Give the current vmcb to the guest */ svm_set_gif (svm, false); - nested_vmcb->save.es = vmcb->save.es; - nested_vmcb->save.cs = vmcb->save.cs; - nested_vmcb->save.ss = vmcb->save.ss; - nested_vmcb->save.ds = vmcb->save.ds; - nested_vmcb->save.gdtr = vmcb->save.gdtr; - nested_vmcb->save.idtr = vmcb->save.idtr; + nested_vmcb->save.es = *svm_es_read (svm); + nested_vmcb->save.cs = *svm_cs_read (svm); + nested_vmcb->save.ss = *svm_ss_read (svm); + nested_vmcb->save.ds = *svm_ds_read (svm); + nested_vmcb->save.gdtr = *svm_gdtr_read (svm); + nested_vmcb->save.idtr = *svm_idtr_read (svm); nested_vmcb->save.efer = svm->vcpu.arch.efer; nested_vmcb->save.cr0 =kvm_read_cr0 (&svm->vcpu); nested_vmcb->save.cr3 =kvm_read_cr3 (&svm->vcpu); - nested_vmcb->save.cr2 = vmcb->save.cr2; + nested_vmcb->save.cr2 =svm_cr2_read (svm); nested_vmcb->save.cr4 = svm->vcpu.arch.cr4; nested_vmcb->save.rflags =kvm_get_rflags (&svm->vcpu); nested_vmcb->save.rip =kvm_rip_read (&svm->vcpu); nested_vmcb->save.rsp =kvm_rsp_read (&svm->vcpu); nested_vmcb->save.rax =kvm_rax_read (&svm->vcpu); - nested_vmcb->save.dr7 = vmcb->save.dr7; + nested_vmcb->save.dr7 =svm_dr7_read (svm); nested_vmcb->save.dr6 = svm->vcpu.arch.dr6; - nested_vmcb->save.cpl = vmcb->save.cpl; + nested_vmcb->save.cpl =svm_cpl_read (svm); nested_vmcb->control.int_state = vmcb->control.int_state; nested_vmcb->control.exit_code = vmcb->control.exit_code;
@@ -625,9 +626,9 @@ int nested_svm_vmexit(struct vcpu_svm *svm) nested_vmcb->control.event_inj_err = svm->nested.ctl.event_inj_err; nested_vmcb->control.pause_filter_count = - svm->vmcb->control.pause_filter_count; + vmcb->control.pause_filter_count; nested_vmcb->control.pause_filter_thresh = - svm->vmcb->control.pause_filter_thresh; + vmcb->control.pause_filter_thresh;/* Restore the original control entries */ copy_vmcb_control_area (&vmcb->control, &hsave->control);
@@ -638,12 +639,12 @@ int nested_svm_vmexit(struct vcpu_svm *svm) svm->nested.ctl.nested_cr3 =0 ;/* Restore selected save entries */ - svm->vmcb->save.es = hsave->save.es; - svm->vmcb->save.cs = hsave->save.cs; - svm->vmcb->save.ss = hsave->save.ss; - svm->vmcb->save.ds = hsave->save.ds; - svm->vmcb->save.gdtr = hsave->save.gdtr; - svm->vmcb->save.idtr = hsave->save.idtr; +svm_es_write (svm, &hsave->save.es); +svm_cs_write (svm, &hsave->save.cs); +svm_ss_write (svm, &hsave->save.ss); +svm_ds_write (svm, &hsave->save.ds); +svm_gdtr_write (svm, &hsave->save.gdtr); +svm_idtr_write (svm, &hsave->save.idtr);kvm_set_rflags (&svm->vcpu, hsave->save.rflags);svm_set_efer (&svm->vcpu, hsave->save.efer);svm_set_cr0 (&svm->vcpu, hsave->save.cr0 |X86_CR0_PE );
@@ -651,11 +652,11 @@ int nested_svm_vmexit(struct vcpu_svm *svm)kvm_rax_write (&svm->vcpu, hsave->save.rax);kvm_rsp_write (&svm->vcpu, hsave->save.rsp);kvm_rip_write (&svm->vcpu, hsave->save.rip); - svm->vmcb->save.dr7 =0 ; - svm->vmcb->save.cpl =0 ; - svm->vmcb->control.exit_int_info =0 ; +svm_dr7_write (svm,0 ); +svm_cpl_write (svm,0 ); + vmcb->control.exit_int_info =0 ; -vmcb_mark_all_dirty (svm->vmcb); +vmcb_mark_all_dirty (vmcb); trace_kvm_nested_vmexit_inject(nested_vmcb->control.exit_code, nested_vmcb->control.exit_info_1,
@@ -673,7 +674,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm)return 1 ;if (npt_enabled) - svm->vmcb->save.cr3 = hsave->save.cr3; +svm_cr3_write (svm, hsave->save.cr3);/* * Drop what we picked up for L2 via svm_complete_interrupts() so it
@@ -819,7 +820,7 @@ int nested_svm_check_permissions(struct vcpu_svm *svm)return 1 ; } -if (svm->vmcb->save.cpl) { +if (svm_cpl_read (svm)) {kvm_inject_gp (&svm->vcpu,0 );return 1 ; }
@@ -888,7 +889,7 @@ static void nested_svm_nmi(struct vcpu_svm *svm)static void nested_svm_intr (struct vcpu_svm *svm) { -trace_kvm_nested_intr_vmexit (svm->vmcb->save.rip); +trace_kvm_nested_intr_vmexit (svm_rip_read (svm)); svm->vmcb->control.exit_code =SVM_EXIT_INTR ; svm->vmcb->control.exit_info_1 =0 ;
arch/x86/kvm/svm/svm.c +82 -83
485 lines not on this page · raw message
arch/x86/kvm/svm/svm.h +128 -1
139 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 4/35 KVM: SVM: Make GHCB accessor functions available to the hypervisor
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Update the GHCB accessor functions so that some of the macros can be used by KVM when accessing the GHCB via the VMSA accessors. This will avoid duplicating code and make access to the GHCB somewhat transparent.
--- arch/x86/include/asm/svm.h | 15 +++++++++++++-- arch/x86/kernel/cpu/vmware.c | 12 ++++++------ 2 files changed, 19 insertions(+), 8 deletions(-)
arch/x86/include/asm/svm.h +13 -2
@@ -349,15 +349,26 @@ struct vmcb {#define DEFINE_GHCB_ACCESSORS (field) \ static inline bool ghcb_##field##_is_valid(const struct ghcb *ghcb) \ { \ + const struct vmcb_save_area *vmsa = &ghcb->save; \ + \ return test_bit(GHCB_BITMAP_IDX(field), \ - (unsigned long *)&ghcb->save.valid_bitmap); \ + (unsigned long *)vmsa->valid_bitmap); \ + } \ + \ + static inline u64 ghcb_get_##field(struct ghcb *ghcb) \ + { \ + const struct vmcb_save_area *vmsa = &ghcb->save; \ + \ + return vmsa->field; \ } \ \ static inline void ghcb_set_##field(struct ghcb *ghcb, u64 value) \ { \ + struct vmcb_save_area *vmsa = &ghcb->save; \ + \ __set_bit(GHCB_BITMAP_IDX(field), \ (unsigned long *)&ghcb->save.valid_bitmap); \ - ghcb->save.field = value; \ + vmsa->field = value; \ }DEFINE_GHCB_ACCESSORS (cpl)
arch/x86/kernel/cpu/vmware.c +6 -6
@@ -501,12 +501,12 @@ static bool vmware_sev_es_hcall_finish(struct ghcb *ghcb, struct pt_regs *regs) ghcb_rbp_is_valid(ghcb)))return false; - regs->bx = ghcb->save.rbx; - regs->cx = ghcb->save.rcx; - regs->dx = ghcb->save.rdx; - regs->si = ghcb->save.rsi; - regs->di = ghcb->save.rdi; - regs->bp = ghcb->save.rbp; + regs->bx =ghcb_get_rbx (ghcb); + regs->cx =ghcb_get_rcx (ghcb); + regs->dx =ghcb_get_rdx (ghcb); + regs->si =ghcb_get_rsi (ghcb); + regs->di =ghcb_get_rdi (ghcb); + regs->bp =ghcb_get_rbp (ghcb);return true; }
Signature · 1 line
2.28.0
RFC PATCH 5/35 KVM: SVM: Add initial support for SEV-ES GHCB access to KVM
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Provide initial support for accessing the GHCB when needing to access registers for an SEV-ES guest. The support consists of:
- Accessing the GHCB instead of the VMSA when reading and writing guest registers (after the VMSA has been encrypted).
- Creating register access override functions for reading and writing guest registers from the common KVM support.
- Allocating pages for the VMSA and GHCB when creating each vCPU
- The VMSA page holds the encrypted VMSA for the vCPU
- The GHCB page is used to hold a copy of the guest GHCB during VMGEXIT processing.
--- arch/x86/include/asm/kvm_host.h | 7 ++ arch/x86/include/asm/msr-index.h | 1 + arch/x86/kvm/kvm_cache_regs.h | 30 +++++-- arch/x86/kvm/svm/svm.c | 138 ++++++++++++++++++++++++++++++- arch/x86/kvm/svm/svm.h | 65 ++++++++++++++- 5 files changed, 230 insertions(+), 11 deletions(-)
arch/x86/include/asm/kvm_host.h +7 -0
@@ -788,6 +788,9 @@ struct kvm_vcpu_arch {/* AMD MSRC001_0015 Hardware Configuration */ u64 msr_hwcr; + +/* SEV-ES support */ +bool vmsa_encrypted; };struct kvm_lpage_info {
@@ -1227,6 +1230,10 @@ struct kvm_x86_ops {int (*enable_direct_tlbflush)(struct kvm_vcpu *vcpu);void (*migrate_timers)(struct kvm_vcpu *vcpu); + +void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg); +void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg, +unsigned long val); };struct kvm_x86_nested_ops {
arch/x86/include/asm/msr-index.h +1 -0
@@ -466,6 +466,7 @@#define MSR_AMD64_IBSBRTARGET 0xc001103b#define MSR_AMD64_IBSOPDATA4 0xc001103d#define MSR_AMD64_IBS_REG_COUNT_MAX 8/* includes MSR_AMD64_IBSBRTARGET */ +#define MSR_AMD64_VM_PAGE_FLUSH 0xc001011e#define MSR_AMD64_SEV_ES_GHCB 0xc0010130#define MSR_AMD64_SEV 0xc0010131#define MSR_AMD64_SEV_ENABLED_BIT 0
arch/x86/kvm/kvm_cache_regs.h +21 -9
@@ -9,15 +9,21 @@ (X86_CR4_PVI |X86_CR4_DE |X86_CR4_PCE |X86_CR4_OSFXSR \ | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_PGE | X86_CR4_TSD) -#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ -static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\ -{ \ - return vcpu->arch.regs[VCPU_REGS_##uname]; \ -} \ -static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ - unsigned long val) \ -{ \ - vcpu->arch.regs[VCPU_REGS_##uname] = val; \ +#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ +static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu) \ +{ \ + if (kvm_x86_ops.reg_read_override) \ + kvm_x86_ops.reg_read_override(vcpu, VCPU_REGS_##uname); \ + \ + return vcpu->arch.regs[VCPU_REGS_##uname]; \ +} \ +static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ + unsigned long val) \ +{ \ + if (kvm_x86_ops.reg_write_override) \ + kvm_x86_ops.reg_write_override(vcpu, VCPU_REGS_##uname, val); \ + \ + vcpu->arch.regs[VCPU_REGS_##uname] = val; \ } BUILD_KVM_GPR_ACCESSORS(rax, RAX)BUILD_KVM_GPR_ACCESSORS (rbx,RBX )
@@ -67,6 +73,9 @@ static inline unsigned long kvm_register_read(struct kvm_vcpu *vcpu, int reg)if (WARN_ON_ONCE ((unsigned int )reg >=NR_VCPU_REGS ))return 0 ; +if (kvm_x86_ops.reg_read_override) + kvm_x86_ops.reg_read_override (vcpu, reg); +if (!kvm_register_is_available (vcpu, reg)) kvm_x86_ops.cache_reg (vcpu, reg);
@@ -79,6 +88,9 @@ static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg,if (WARN_ON_ONCE ((unsigned int )reg >=NR_VCPU_REGS ))return ; +if (kvm_x86_ops.reg_write_override) + kvm_x86_ops.reg_write_override (vcpu, reg, val); + vcpu->arch.regs[reg] = val;kvm_register_mark_dirty (vcpu, reg); }
arch/x86/kvm/svm/svm.c +137 -1
182 lines not on this page · raw message
arch/x86/kvm/svm/svm.h +64 -1
115 lines not on this page · raw message
Signature · 1 line
2.28.0
A message that is not in the archive (sent off-list or not yet received): 20200914205801.GA7084@sjchrist-ice
Re: [RFC PATCH 05/35] KVM: SVM: Add initial support for SEV-ES GHCB access to KVM
Quoted 30 lines · Sean Christopherson
Quoted 29 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Provide initial support for accessing the GHCB when needing to access registers for an SEV-ES guest. The support consists of:
- Accessing the GHCB instead of the VMSA when reading and writing guest registers (after the VMSA has been encrypted).
- Creating register access override functions for reading and writing guest registers from the common KVM support.
- Allocating pages for the VMSA and GHCB when creating each vCPU
- The VMSA page holds the encrypted VMSA for the vCPU
- The GHCB page is used to hold a copy of the guest GHCB during VMGEXIT processing.Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/include/asm/kvm_host.h | 7 ++ arch/x86/include/asm/msr-index.h | 1 + arch/x86/kvm/kvm_cache_regs.h | 30 +++++-- arch/x86/kvm/svm/svm.c | 138 ++++++++++++++++++++++++++++++- arch/x86/kvm/svm/svm.h | 65 ++++++++++++++- 5 files changed, 230 insertions(+), 11 deletions(-)arch/x86/include/asm/kvm_host.h +0
-0@@ -788,6 +788,9 @@ struct kvm_vcpu_arch {
Quoted 11 lines
arch/x86/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 793
Quoted 5 lines
arch/x86/include/asm/kvm_host.h +3
-0/* AMD MSRC001_0015 Hardware Configuration */ u64 msr_hwcr; + +/* SEV-ES support */ +bool vmsa_encrypted;Peeking a little into the future, Intel needs a very similar flag for TDX[*]. At a glance throughout the series,, I don't see anything that is super SEV-ES specific, so I think we could do s/vmsa_encrypted/guest_state_protected (or something along those lines).
Yup, I can do that.
Quoted 78 lines
Quoted 75 lines
arch/x86/include/asm/kvm_host.h +4
-0};struct kvm_lpage_info {@@ -1227,6 +1230,10 @@ struct kvm_x86_ops {int (*enable_direct_tlbflush)(struct kvm_vcpu *vcpu);void (*migrate_timers)(struct kvm_vcpu *vcpu); + +void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg); +void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg, +unsigned long val); };struct kvm_x86_nested_ops {arch/x86/include/asm/msr-index.h +1
-0@@ -466,6 +466,7 @@#define MSR_AMD64_IBSBRTARGET 0xc001103b#define MSR_AMD64_IBSOPDATA4 0xc001103d#define MSR_AMD64_IBS_REG_COUNT_MAX 8/* includes MSR_AMD64_IBSBRTARGET */ +#define MSR_AMD64_VM_PAGE_FLUSH 0xc001011e#define MSR_AMD64_SEV_ES_GHCB 0xc0010130#define MSR_AMD64_SEV 0xc0010131#define MSR_AMD64_SEV_ENABLED_BIT 0arch/x86/kvm/kvm_cache_regs.h +18
-9@@ -9,15 +9,21 @@ (X86_CR4_PVI |X86_CR4_DE |X86_CR4_PCE |X86_CR4_OSFXSR \ | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_PGE | X86_CR4_TSD) -#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ -static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\ -{ \ - return vcpu->arch.regs[VCPU_REGS_##uname]; \ -} \ -static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ - unsigned long val) \ -{ \ - vcpu->arch.regs[VCPU_REGS_##uname] = val; \ +#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ +static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu) \ +{ \ + if (kvm_x86_ops.reg_read_override) \ + kvm_x86_ops.reg_read_override(vcpu, VCPU_REGS_##uname); \ + \ + return vcpu->arch.regs[VCPU_REGS_##uname]; \ +} \ +static __always_inline void kvm_##lname##_write(struct kvm_vcpu *vcpu, \ + unsigned long val) \ +{ \ + if (kvm_x86_ops.reg_write_override) \ + kvm_x86_ops.reg_write_override(vcpu, VCPU_REGS_##uname, val); \ + \ + vcpu->arch.regs[VCPU_REGS_##uname] = val; \ } BUILD_KVM_GPR_ACCESSORS(rax, RAX)BUILD_KVM_GPR_ACCESSORS (rbx,RBX )@@ -67,6 +73,9 @@ static inline unsigned long kvm_register_read(struct kvm_vcpu *vcpu, int reg)if (WARN_ON_ONCE ((unsigned int )reg >=NR_VCPU_REGS ))return 0 ; +if (kvm_x86_ops.reg_read_override) + kvm_x86_ops.reg_read_override (vcpu, reg); +if (!kvm_register_is_available (vcpu, reg)) kvm_x86_ops.cache_reg (vcpu, reg);@@ -79,6 +88,9 @@ static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg,if (WARN_ON_ONCE ((unsigned int )reg >=NR_VCPU_REGS ))return ;
Quoted 9 lines
arch/x86/kvm/kvm_cache_regs.h static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg, · line 92
Quoted 2 lines
arch/x86/kvm/kvm_cache_regs.h +2
-0+if (kvm_x86_ops.reg_write_override) + kvm_x86_ops.reg_write_override (vcpu, reg, val);There has to be a more optimal approach for propagating registers between
vcpu->arch.regsand the VMSA than adding a per-GPR hook. Why not simply copy the entire set of registers to/from the VMSA on every exit and entry? AFAICT,valid_bitsis only used in the read path, and KVM doesn't do anything sophistated when it hits a !valid_bits reads.
That would probably be ok. And actually, the code might be able to just check the GHCB valid bitmap for valid regs on exit, copy them and then clear the bitmap. The write code could check if vmsa_encrypted is set and then set a "valid" bit for the reg that could be used to set regs on entry.
I'm not sure if turning kvm_vcpu_arch.regs into a struct and adding a valid bit would be overkill or not.
Quoted 8 lines
If unconditional copying has a noticeable impact on e.g. IRQ handling latency, the save/restore could be limited to exits that may access guest state, which is presumably a well-defined, limited list. Such exits are basically a slow path anyways, especially if the guest kernel is taking a #VC on the front eend. Adding hooks to KVM_{GET,SET}_REGS to ensure userspace accesses are handled correcty would be trivial.
Quoted 5 lines
Adding per-GPR hooks will bloat common KVM for both VMX and SVM, and will likely have a noticeable performance impact on SVM due to adding 60-70 cycles to every GPR access for the retpoline. Static calls will take the sting out of that, but it's still a lot of code bytes, that IMO, are completely unecessary.
Yes, definitely something I need to look into more.
Quoted 148 lines
Quoted 148 lines
arch/x86/kvm/kvm_cache_regs.h +1
-0+ vcpu->arch.regs[reg] = val;kvm_register_mark_dirty (vcpu, reg); }arch/x86/kvm/svm/svm.c +97
-1@@ -1175,6 +1175,7 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu)struct page *msrpm_pages;struct page *hsave_page;struct page *nested_msrpm_pages; +struct page *vmsa_page =NULL ;int err;BUILD_BUG_ON (offsetof(struct vcpu_svm , vcpu) !=0 );@@ -1197,9 +1198,19 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu)if (!hsave_page) goto free_page3; +if (sev_es_guest (svm->vcpu.kvm)) { +/* +* SEV-ES guests require a separate VMSA page used to contain +* the encrypted register state of the guest. +*/ + vmsa_page =alloc_page (GFP_KERNEL ); +if (!vmsa_page) + goto free_page4; + } + err =avic_init_vcpu (svm);if (err) - goto free_page4; + goto free_page5;/* We initialize this flag to true to make sure that the is_running * bit would be set the first time the vcpu is loaded. @@ -1219,6 +1230,12 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu) svm->vmcb =page_address (page);clear_page (svm->vmcb); svm->vmcb_pa =__sme_set (page_to_pfn (page) <<PAGE_SHIFT ); + +if (vmsa_page) { + svm->vmsa =page_address (vmsa_page); +clear_page (svm->vmsa); + } + svm->asid_generation =0 ;init_vmcb (svm);@@ -1227,6 +1244,9 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu)return 0 ; +free_page5: +if (vmsa_page) +__free_page (vmsa_page); free_page4:__free_page (hsave_page); free_page3:@@ -1258,6 +1278,26 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu)*/ svm_clear_current_vmcb (svm->vmcb); +if (sev_es_guest (vcpu->kvm)) { +struct kvm_sev_info *sev = &to_kvm_svm (vcpu->kvm)->sev_info; + +if (vcpu->arch.vmsa_encrypted) { +u64 page_to_flush; + +/* +* The VMSA page was used by hardware to hold guest +* encrypted state, be sure to flush it before returning +* it to the system. This is done using the VM Page +* Flush MSR (which takes the page virtual address and +* guest ASID). +*/ + page_to_flush = (u64 )svm->vmsa | sev->asid; +wrmsrl (MSR_AMD64_VM_PAGE_FLUSH , page_to_flush); + } + +__free_page (virt_to_page (svm->vmsa)); + } +__free_page (pfn_to_page (__sme_clr (svm->vmcb_pa) >>PAGE_SHIFT ));__free_pages (virt_to_page (svm->msrpm),MSRPM_ALLOC_ORDER );__free_page (virt_to_page (svm->nested.hsave));@@ -4012,6 +4052,99 @@ static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) (svm->vmcb->control.intercept & (1ULL <<INTERCEPT_INIT )); } +/* +* These return values represent the offset in quad words within the VM save +* area. This allows them to be accessed by casting the save area to a u64 +* array. +*/ +#define VMSA_REG_ENTRY (_field) (offsetof(struct vmcb_save_area, _field) / sizeof(u64)) +#define VMSA_REG_UNDEF VMSA_REG_ENTRY(valid_bitmap) +static inline unsigned int vcpu_to_vmsa_entry (enum kvm_reg reg) +{ +switch (reg) { +case VCPU_REGS_RAX :return VMSA_REG_ENTRY (rax); +case VCPU_REGS_RBX :return VMSA_REG_ENTRY (rbx); +case VCPU_REGS_RCX :return VMSA_REG_ENTRY (rcx); +case VCPU_REGS_RDX :return VMSA_REG_ENTRY (rdx); +case VCPU_REGS_RSP :return VMSA_REG_ENTRY (rsp); +case VCPU_REGS_RBP :return VMSA_REG_ENTRY (rbp); +case VCPU_REGS_RSI :return VMSA_REG_ENTRY (rsi); +case VCPU_REGS_RDI :return VMSA_REG_ENTRY (rdi); +#ifdef CONFIG_X86_64 +case VCPU_REGS_R8 :return VMSA_REG_ENTRY (r8); +case VCPU_REGS_R9 :return VMSA_REG_ENTRY (r9); +case VCPU_REGS_R10 :return VMSA_REG_ENTRY (r10); +case VCPU_REGS_R11 :return VMSA_REG_ENTRY (r11); +case VCPU_REGS_R12 :return VMSA_REG_ENTRY (r12); +case VCPU_REGS_R13 :return VMSA_REG_ENTRY (r13); +case VCPU_REGS_R14 :return VMSA_REG_ENTRY (r14); +case VCPU_REGS_R15 :return VMSA_REG_ENTRY (r15); +#endif +case VCPU_REGS_RIP :return VMSA_REG_ENTRY (rip); +default : +WARN_ONCE (1 ,"unsupported VCPU to VMSA register conversion\n" ); +return VMSA_REG_UNDEF ; + } +} + +/* For SEV-ES guests, populate the vCPU register from the appropriate VMSA/GHCB */ +static void svm_reg_read_override (struct kvm_vcpu *vcpu,enum kvm_reg reg) +{ +struct vmcb_save_area *vmsa; +struct vcpu_svm *svm; +unsigned int entry; +unsigned long val; +u64 *vmsa_reg; + +if (!sev_es_guest (vcpu->kvm)) +return ; + + entry =vcpu_to_vmsa_entry (reg); +if (entry ==VMSA_REG_UNDEF ) +return ; + + svm =to_svm (vcpu); + vmsa =get_vmsa (svm); + vmsa_reg = (u64 *)vmsa; + val = (unsigned long )vmsa_reg[entry]; +
Quoted 8 lines
arch/x86/kvm/svm/svm.c static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) · line 4114
Quoted 4 lines
arch/x86/kvm/svm/svm.c +4
-0+/* If a GHCB is mapped, check the bitmap of valid entries */ +if (svm->ghcb) { +if (!test_bit (entry, (unsigned long *)vmsa->valid_bitmap)) + val =0 ;Is KVM relying on this being 0? Would it make sense to stuff something like
0xaaaa... or0xdeadbeefdeadbeefso that consumption of bogus data is more noticeable?
No, KVM isn't relying on this being 0. I thought about using something other than 0 here, but settled on just using 0. I'm open to changing that, though. I'm not sure if there's an easy way to short-circuit the intercept and respond back with an error at this point, that would be optimal.
Quoted 87 lines
Quoted 74 lines
arch/x86/kvm/svm/svm.c +36
-0+ } + + vcpu->arch.regs[reg] = val; +} + +/* For SEV-ES guests, set the vCPU register in the appropriate VMSA */ +static void svm_reg_write_override (struct kvm_vcpu *vcpu,enum kvm_reg reg, +unsigned long val) +{ +struct vmcb_save_area *vmsa; +struct vcpu_svm *svm; +unsigned int entry; +u64 *vmsa_reg; + + entry =vcpu_to_vmsa_entry (reg); +if (entry ==VMSA_REG_UNDEF ) +return ; + + svm =to_svm (vcpu); + vmsa =get_vmsa (svm); + vmsa_reg = (u64 *)vmsa; + +/* If a GHCB is mapped, set the bit to indicate a valid entry */ +if (svm->ghcb) { +unsigned int index = entry /8 ; +unsigned int shift = entry %8 ; + + vmsa->valid_bitmap[index] |=BIT (shift); + } + + vmsa_reg[entry] = val; +} +static void svm_vm_destroy (struct kvm *kvm) {avic_vm_destroy (kvm);@@ -4150,6 +4283,9 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .need_emulation_on_page_fault = svm_need_emulation_on_page_fault, .apic_init_signal_blocked = svm_apic_init_signal_blocked, + + .reg_read_override = svm_reg_read_override, + .reg_write_override = svm_reg_write_override, };static struct kvm_x86_init_ops svm_init_ops__initdata = {arch/x86/kvm/svm/svm.h +11
-1@@ -159,6 +159,10 @@ struct vcpu_svm {*/ struct list_head ir_list;spinlock_t ir_list_lock; + +/* SEV-ES support */ +struct vmcb_save_area *vmsa; +struct ghcb *ghcb; };struct svm_cpu_data {@@ -509,9 +513,34 @@ void sev_hardware_teardown(void);static inline struct vmcb_save_area *get_vmsa (struct vcpu_svm *svm) { -return &svm->vmcb->save; +struct vmcb_save_area *vmsa; + +if (sev_es_guest (svm->vcpu.kvm)) { +/* +* Before LAUNCH_UPDATE_VMSA, use the actual SEV-ES save area +* to construct the initial state. Afterwards, use the mapped +* GHCB in a VMGEXIT or the traditional save area as a scratch arch/x86/kvm/svm/svm.h void sev_hardware_teardown(void); · line 532
Quoted 10 lines
arch/x86/kvm/svm/svm.h +10
-0+* area when outside of a VMGEXIT. +*/ +if (svm->vcpu.arch.vmsa_encrypted) { +if (svm->ghcb) + vmsa = &svm->ghcb->save; +else + vmsa = &svm->vmcb->save; + }else { + vmsa = svm->vmsa; + }Not sure if it's actually better, but this whole thing could be:
Quoted 8 lines
if (!sev_es_guest(svm->vcpu.kvm)) return &svm->vmcb->save; if (!svm->vcpu.arch.vmsa_encrypted) return svm->vmsa; return svm->ghcb ? &svm->ghcb->save : &svm->vmcb->save;
It does look cleaner.
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/svm/svm.h +5
-0+ }else { + vmsa = &svm->vmcb->save; + } + +return vmsa; }
Quoted 14 lines
arch/x86/kvm/svm/svm.h static inline struct vmcb_save_area *get_vmsa(struct vcpu_svm *svm) · line 560
Quoted 12 lines
arch/x86/kvm/svm/svm.h +5
-0+#define SEV_ES_SET_VALID (_vmsa, _field) \ + __set_bit(GHCB_BITMAP_IDX(_field), \ + (unsigned long *)(_vmsa)->valid_bitmap) +#define DEFINE_VMSA_SEGMENT_ENTRY (_field, _entry, _size) \ static inline _size \ svm_##_field##_read_##_entry(struct vcpu_svm *svm) \@@ -528,6 +557,9 @@ static inline struct vmcb_save_area *get_vmsa(struct vcpu_svm *svm)struct vmcb_save_area *vmsa =get_vmsa (svm); \ \ vmsa->_field._entry = value; \ +if (svm->vcpu.arch.vmsa_encrypted) { \Pretty sure braces are unnecessary on all these.
Yup, I can get rid of them.
Thanks,
Tom
Quoted 43 lines
Quoted 43 lines
arch/x86/kvm/svm/svm.h +14
-0+SEV_ES_SET_VALID (vmsa, _field); \ + } \ } \#define DEFINE_VMSA_SEGMENT_ACCESSOR (_field) \@@ -551,6 +583,9 @@ static inline struct vmcb_save_area *get_vmsa(struct vcpu_svm *svm)struct vmcb_save_area *vmsa =get_vmsa (svm); \ \ vmsa->_field = *seg; \ +if (svm->vcpu.arch.vmsa_encrypted) { \ +SEV_ES_SET_VALID (vmsa, _field); \ + } \ }DEFINE_VMSA_SEGMENT_ACCESSOR (cs)@@ -579,6 +614,9 @@ DEFINE_VMSA_SEGMENT_ACCESSOR(tr)struct vmcb_save_area *vmsa =get_vmsa (svm); \ \ vmsa->_field = value; \ +if (svm->vcpu.arch.vmsa_encrypted) { \ +SEV_ES_SET_VALID (vmsa, _field); \ + } \ } \ \ static inline void \@@ -587,6 +625,9 @@ DEFINE_VMSA_SEGMENT_ACCESSOR(tr)struct vmcb_save_area *vmsa =get_vmsa (svm); \ \ vmsa->_field &= value; \ +if (svm->vcpu.arch.vmsa_encrypted) { \ +SEV_ES_SET_VALID (vmsa, _field); \ + } \ } \ \ static inline void \@@ -595,6 +636,9 @@ DEFINE_VMSA_SEGMENT_ACCESSOR(tr)struct vmcb_save_area *vmsa =get_vmsa (svm); \ \ vmsa->_field |= value; \ +if (svm->vcpu.arch.vmsa_encrypted) { \ +SEV_ES_SET_VALID (vmsa, _field); \ + } \
Re: [RFC PATCH 05/35] KVM: SVM: Add initial support for SEV-ES GHCB access to KVM
Quoted 15 lines · Tom Lendacky
Quoted 4 lines · Sean Christopherson
Quoted 4 lines
arch/x86/kvm/kvm_cache_regs.h +0
-0@@ -79,6 +88,9 @@ static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg,if (WARN_ON_ONCE ((unsigned int )reg >=NR_VCPU_REGS ))return ;Quoted 9 lines
arch/x86/kvm/kvm_cache_regs.h static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg, · line 92
Quoted 2 lines
arch/x86/kvm/kvm_cache_regs.h +2
-0+if (kvm_x86_ops.reg_write_override) + kvm_x86_ops.reg_write_override (vcpu, reg, val);There has to be a more optimal approach for propagating registers between
vcpu->arch.regsand the VMSA than adding a per-GPR hook. Why not simply copy the entire set of registers to/from the VMSA on every exit and entry? AFAICT,valid_bitsis only used in the read path, and KVM doesn't do anything sophistated when it hits a !valid_bits reads.
Quoted 7 lines
That would probably be ok. And actually, the code might be able to just check the GHCB valid bitmap for valid regs on exit, copy them and then clear the bitmap. The write code could check if
vmsa_encryptedis set and then set a "valid" bit for the reg that could be used to set regs on entry.I'm not sure if turning kvm_vcpu_arch.regs into a struct and adding a valid bit would be overkill or not.
KVM already has space in regs_avail and regs_dirty for GPRs, they're just not used by the get/set helpers because they're always loaded/stored for both SVM and VMX.
I assume nothing will break if KVM "writes" random GPRs in the VMSA? I can't see how the guest would achieve any level of security if it wantonly consumes GPRs, i.e. it's the guest's responsibility to consume only the relevant GPRs.
If that holds true, than avoiding the copying isn't functionally necessary, and is really just a performance optimization. One potentially crazy idea would be to change vcpu->arch.regs to be a pointer (defaults a __regs array), and then have SEV-ES switch it to point directly at the VMSA array (I think the layout is identical for x86-64?).
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
arch/x86/kvm/svm/svm.c +11
-0@@ -4012,6 +4052,99 @@ static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) (svm->vmcb->control.intercept & (1ULL <<INTERCEPT_INIT )); } +/* +* These return values represent the offset in quad words within the VM save +* area. This allows them to be accessed by casting the save area to a u64 +* array. +*/ +#define VMSA_REG_ENTRY (_field) (offsetof(struct vmcb_save_area, _field) / sizeof(u64)) +#define VMSA_REG_UNDEF VMSA_REG_ENTRY(valid_bitmap) +static inline unsigned int vcpu_to_vmsa_entry (enum kvm_reg reg) +{ +switch (reg) { +case VCPU_REGS_RAX :return VMSA_REG_ENTRY (rax);
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/svm/svm.c static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) · line 4073
Quoted 8 lines
arch/x86/kvm/svm/svm.c +8
-0+case VCPU_REGS_RBX :return VMSA_REG_ENTRY (rbx); +case VCPU_REGS_RCX :return VMSA_REG_ENTRY (rcx); +case VCPU_REGS_RDX :return VMSA_REG_ENTRY (rdx); +case VCPU_REGS_RSP :return VMSA_REG_ENTRY (rsp); +case VCPU_REGS_RBP :return VMSA_REG_ENTRY (rbp); +case VCPU_REGS_RSI :return VMSA_REG_ENTRY (rsi); +case VCPU_REGS_RDI :return VMSA_REG_ENTRY (rdi); +#ifdef CONFIG_X86_64
Is KVM SEV-ES going to support 32-bit builds?
Quoted 46 lines
Quoted 37 lines
Quoted 37 lines
arch/x86/kvm/svm/svm.c +37
-0+case VCPU_REGS_R8 :return VMSA_REG_ENTRY (r8); +case VCPU_REGS_R9 :return VMSA_REG_ENTRY (r9); +case VCPU_REGS_R10 :return VMSA_REG_ENTRY (r10); +case VCPU_REGS_R11 :return VMSA_REG_ENTRY (r11); +case VCPU_REGS_R12 :return VMSA_REG_ENTRY (r12); +case VCPU_REGS_R13 :return VMSA_REG_ENTRY (r13); +case VCPU_REGS_R14 :return VMSA_REG_ENTRY (r14); +case VCPU_REGS_R15 :return VMSA_REG_ENTRY (r15); +#endif +case VCPU_REGS_RIP :return VMSA_REG_ENTRY (rip); +default : +WARN_ONCE (1 ,"unsupported VCPU to VMSA register conversion\n" ); +return VMSA_REG_UNDEF ; + } +} + +/* For SEV-ES guests, populate the vCPU register from the appropriate VMSA/GHCB */ +static void svm_reg_read_override (struct kvm_vcpu *vcpu,enum kvm_reg reg) +{ +struct vmcb_save_area *vmsa; +struct vcpu_svm *svm; +unsigned int entry; +unsigned long val; +u64 *vmsa_reg; + +if (!sev_es_guest (vcpu->kvm)) +return ; + + entry =vcpu_to_vmsa_entry (reg); +if (entry ==VMSA_REG_UNDEF ) +return ; + + svm =to_svm (vcpu); + vmsa =get_vmsa (svm); + vmsa_reg = (u64 *)vmsa; + val = (unsigned long )vmsa_reg[entry]; +Quoted 8 lines
arch/x86/kvm/svm/svm.c static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) · line 4114
Quoted 4 lines
arch/x86/kvm/svm/svm.c +4
-0+/* If a GHCB is mapped, check the bitmap of valid entries */ +if (svm->ghcb) { +if (!test_bit (entry, (unsigned long *)vmsa->valid_bitmap)) + val =0 ;Is KVM relying on this being 0? Would it make sense to stuff something like
0xaaaa... or0xdeadbeefdeadbeefso that consumption of bogus data is more noticeable?
Quoted 4 lines
No, KVM isn't relying on this being 0. I thought about using something other than 0 here, but settled on just using 0. I'm open to changing that, though. I'm not sure if there's an easy way to short-circuit the intercept and respond back with an error at this point, that would be optimal.
Ya, responding with an error would be ideal. At this point, we're taking the same lazy approach for TDX and effectively consuming garbage if the guest requests emulation but doesn't expose the necessary GPRs. That being said, TDX's guest/host ABI is quite rigid, so all the "is this register valid" checks could be hardcoded into the higher level "emulation" flows.
Would that also be an option for SEV-ES?
Re: [RFC PATCH 05/35] KVM: SVM: Add initial support for SEV-ES GHCB access to KVM
Quoted 24 lines · Sean Christopherson
Quoted 15 lines · Tom Lendacky
Quoted 4 lines · Sean Christopherson
Quoted 4 lines
arch/x86/kvm/kvm_cache_regs.h +0
-0@@ -79,6 +88,9 @@ static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg,if (WARN_ON_ONCE ((unsigned int )reg >=NR_VCPU_REGS ))return ;Quoted 9 lines
arch/x86/kvm/kvm_cache_regs.h static inline void kvm_register_write(struct kvm_vcpu *vcpu, int reg, · line 92
Quoted 2 lines
arch/x86/kvm/kvm_cache_regs.h +2
-0+if (kvm_x86_ops.reg_write_override) + kvm_x86_ops.reg_write_override (vcpu, reg, val);There has to be a more optimal approach for propagating registers between
vcpu->arch.regsand the VMSA than adding a per-GPR hook. Why not simply copy the entire set of registers to/from the VMSA on every exit and entry? AFAICT,valid_bitsis only used in the read path, and KVM doesn't do anything sophistated when it hits a !valid_bits reads.Quoted 7 lines
That would probably be ok. And actually, the code might be able to just check the GHCB valid bitmap for valid regs on exit, copy them and then clear the bitmap. The write code could check if
vmsa_encryptedis set and then set a "valid" bit for the reg that could be used to set regs on entry.I'm not sure if turning kvm_vcpu_arch.regs into a struct and adding a valid bit would be overkill or not.
Quoted 7 lines
KVM already has space in
regs_availandregs_dirtyfor GPRs, they're just not used by the get/set helpers because they're always loaded/stored for both SVM and VMX.I assume nothing will break if KVM "writes" random GPRs in the VMSA? I can't see how the guest would achieve any level of security if it wantonly consumes GPRs, i.e. it's the guest's responsibility to consume only the relevant GPRs.
Right, the guest should only read the registers that it is expecting to be provided by the hypervisor as set forth in the GHCB spec. It shouldn't load any other registers that the hypervisor provides. The Linux SEV-ES guest support follows this model and will only load the registers that are specified via the GHCB spec for a particular NAE event, ignoring anything else provided.
Quoted 6 lines
If that holds true, than avoiding the copying isn't functionally necessary, and is really just a performance optimization. One potentially crazy idea would be to change
vcpu->arch.regsto be a pointer (defaults a__regsarray), and then have SEV-ES switch it to point directly at the VMSA array (I think the layout is identical for x86-64?).
That would be nice, but it isn't quite laid out like that. Before SEV-ES support, RAX and RSP were the only GPRs saved. With the arrival of SEV-ES, the remaining registers were added to the VMSA, but a number of bytes after RAX and RSP. So right now, there are reserved areas where RAX and RSP would have been at the new register block in the VMSA (see offset 0x300 in the VMSA layout of the APM volume 2,
https://www.amd.com/system/files/TechDocs/24593.pdf).
I might be able to move the RAX and RSP values before the VMSA is encrypted (or the GHCB returned), assuming those fields would stay reserved, but I don't think that can be guaranteed.
Let me see if I can put something together using regs_avail and regs_dirty.
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
arch/x86/kvm/svm/svm.c +13
-0@@ -4012,6 +4052,99 @@ static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) (svm->vmcb->control.intercept & (1ULL <<INTERCEPT_INIT )); } +/* +* These return values represent the offset in quad words within the VM save +* area. This allows them to be accessed by casting the save area to a u64 +* array. +*/ +#define VMSA_REG_ENTRY (_field) (offsetof(struct vmcb_save_area, _field) / sizeof(u64)) +#define VMSA_REG_UNDEF VMSA_REG_ENTRY(valid_bitmap) +static inline unsigned int vcpu_to_vmsa_entry (enum kvm_reg reg) +{ +switch (reg) { +case VCPU_REGS_RAX :return VMSA_REG_ENTRY (rax); +case VCPU_REGS_RBX :return VMSA_REG_ENTRY (rbx); +case VCPU_REGS_RCX :return VMSA_REG_ENTRY (rcx);
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/kvm/svm/svm.c static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) · line 4073
Quoted 6 lines
arch/x86/kvm/svm/svm.c +6
-0+case VCPU_REGS_RDX :return VMSA_REG_ENTRY (rdx); +case VCPU_REGS_RSP :return VMSA_REG_ENTRY (rsp); +case VCPU_REGS_RBP :return VMSA_REG_ENTRY (rbp); +case VCPU_REGS_RSI :return VMSA_REG_ENTRY (rsi); +case VCPU_REGS_RDI :return VMSA_REG_ENTRY (rdi); +#ifdef CONFIG_X86_64 Is KVM SEV-ES going to support 32-bit builds?
No, SEV-ES won't support 32-bit builds and since those fields are always defined, I can just remove this #ifdef.
Quoted 52 lines
Quoted 47 lines
Quoted 38 lines
Quoted 38 lines
arch/x86/kvm/svm/svm.c +37
-0+case VCPU_REGS_R8 :return VMSA_REG_ENTRY (r8); +case VCPU_REGS_R9 :return VMSA_REG_ENTRY (r9); +case VCPU_REGS_R10 :return VMSA_REG_ENTRY (r10); +case VCPU_REGS_R11 :return VMSA_REG_ENTRY (r11); +case VCPU_REGS_R12 :return VMSA_REG_ENTRY (r12); +case VCPU_REGS_R13 :return VMSA_REG_ENTRY (r13); +case VCPU_REGS_R14 :return VMSA_REG_ENTRY (r14); +case VCPU_REGS_R15 :return VMSA_REG_ENTRY (r15); +#endif +case VCPU_REGS_RIP :return VMSA_REG_ENTRY (rip); +default : +WARN_ONCE (1 ,"unsupported VCPU to VMSA register conversion\n" ); +return VMSA_REG_UNDEF ; + } +} + +/* For SEV-ES guests, populate the vCPU register from the appropriate VMSA/GHCB */ +static void svm_reg_read_override (struct kvm_vcpu *vcpu,enum kvm_reg reg) +{ +struct vmcb_save_area *vmsa; +struct vcpu_svm *svm; +unsigned int entry; +unsigned long val; +u64 *vmsa_reg; + +if (!sev_es_guest (vcpu->kvm)) +return ; + + entry =vcpu_to_vmsa_entry (reg); +if (entry ==VMSA_REG_UNDEF ) +return ; + + svm =to_svm (vcpu); + vmsa =get_vmsa (svm); + vmsa_reg = (u64 *)vmsa; + val = (unsigned long )vmsa_reg[entry]; +Quoted 8 lines
arch/x86/kvm/svm/svm.c static bool svm_apic_init_signal_blocked(struct kvm_vcpu *vcpu) · line 4114
Quoted 4 lines
arch/x86/kvm/svm/svm.c +4
-0+/* If a GHCB is mapped, check the bitmap of valid entries */ +if (svm->ghcb) { +if (!test_bit (entry, (unsigned long *)vmsa->valid_bitmap)) + val =0 ;Is KVM relying on this being 0? Would it make sense to stuff something like
0xaaaa... or0xdeadbeefdeadbeefso that consumption of bogus data is more noticeable?Quoted 4 lines
No, KVM isn't relying on this being 0. I thought about using something other than 0 here, but settled on just using 0. I'm open to changing that, though. I'm not sure if there's an easy way to short-circuit the intercept and respond back with an error at this point, that would be optimal.
Quoted 7 lines
Ya, responding with an error would be ideal. At this point, we're taking the same lazy approach for TDX and effectively consuming garbage if the guest requests emulation but doesn't expose the necessary GPRs. That being said, TDX's guest/host ABI is quite rigid, so all the "is this register valid" checks could be hardcoded into the higher level "emulation" flows.
Would that also be an option for SEV-ES?
Meaning adding the expected input checks at VMEXIT time in the VMGEXIT handler, so that accesses later are guaranteed to be good? That is an option and might also address one of the other points you brought up about
about receiving exits that are not supported/expected.
Thanks,
Tom
Quoted 1 line
RFC PATCH 6/35 KVM: SVM: Add required changes to support intercepts under SEV-ES
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
When a guest is running under SEV-ES, the hypervisor cannot access the guest register state. There are numerous places in the KVM code where certain registers are accessed that are not allowed to be accessed (e.g. RIP, CR0, etc). Add checks to prevent register accesses and intercept updates at various points within the KVM code.
--- arch/x86/include/asm/svm.h | 3 +- arch/x86/kvm/cpuid.c | 1 + arch/x86/kvm/svm/svm.c | 114 ++++++++++++++++++++++++++++++++++--- arch/x86/kvm/x86.c | 6 +- 4 files changed, 113 insertions(+), 11 deletions(-)
arch/x86/include/asm/svm.h +2 -1
@@ -130,7 +130,8 @@ struct __attribute__ ((__packed__)) vmcb_control_area {#define LBR_CTL_ENABLE_MASK BIT_ULL(0)#define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1) -#define SVM_INTERRUPT_SHADOW_MASK 1 +#define SVM_INTERRUPT_SHADOW_MASK BIT_ULL(0) +#define SVM_GUEST_INTERRUPT_MASK BIT_ULL(1)#define SVM_IOIO_STR_SHIFT 2#define SVM_IOIO_REP_SHIFT 3
arch/x86/kvm/cpuid.c +1 -0
@@ -115,6 +115,7 @@ void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu) MSR_IA32_MISC_ENABLE_MWAIT); } } +EXPORT_SYMBOL_GPL (kvm_update_cpuid_runtime);static void kvm_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu) {
arch/x86/kvm/svm/svm.c +106 -8
188 lines not on this page · raw message
arch/x86/kvm/x86.c +4 -2
@@ -3771,7 +3771,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) {int idx; -if (vcpu->preempted) +if (vcpu->preempted && !vcpu->arch.vmsa_encrypted) vcpu->arch.preempted_in_kernel = !kvm_x86_ops.get_cpl (vcpu);/*
@@ -7774,7 +7774,9 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu) {struct kvm_run *kvm_run = vcpu->run; - kvm_run->if_flag = (kvm_get_rflags (vcpu) &X86_EFLAGS_IF ) !=0 ; + kvm_run->if_flag = (vcpu->arch.vmsa_encrypted) + ?kvm_arch_interrupt_allowed (vcpu) + : (kvm_get_rflags (vcpu) &X86_EFLAGS_IF ) !=0 ; kvm_run->flags =is_smm (vcpu) ?KVM_RUN_X86_SMM :0 ; kvm_run->cr8 =kvm_get_cr8 (vcpu); kvm_run->apic_base =kvm_get_apic_base (vcpu);
Signature · 1 line
2.28.0
RFC PATCH 7/35 KVM: SVM: Modify DRx register intercepts for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
An SEV-ES guest must only and always intercept DR7 reads and writes. Update set_dr_intercepts() and clr_dr_intercepts() to account for this.
--- arch/x86/kvm/svm/svm.h | 89 ++++++++++++++++++++++++------------------ 1 file changed, 50 insertions(+), 39 deletions(-)
arch/x86/kvm/svm/svm.h +50 -39
@@ -190,6 +190,28 @@ static inline struct kvm_svm *to_kvm_svm(struct kvm *kvm) return container_of(kvm, struct kvm_svm, kvm); } +static inline bool sev_guest (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_AMD_SEV +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; + +return sev->active; +#else +return false; +#endif +} + +static inline bool sev_es_guest (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_AMD_SEV +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; + +return sev_guest (kvm) && sev->es_active; +#else +return false; +#endif +} +static inline void vmcb_mark_all_dirty (struct vmcb *vmcb) { vmcb->control.clean =0 ;
@@ -244,26 +266,35 @@ static inline bool is_cr_intercept(struct vcpu_svm *svm, int bit)return vmcb->control.intercept_cr & (1U << bit); } +#define SVM_DR_INTERCEPTS \ + ((1 << INTERCEPT_DR0_READ) \ + | (1 << INTERCEPT_DR1_READ) \ + | (1 << INTERCEPT_DR2_READ) \ + | (1 << INTERCEPT_DR3_READ) \ + | (1 << INTERCEPT_DR4_READ) \ + | (1 << INTERCEPT_DR5_READ) \ + | (1 << INTERCEPT_DR6_READ) \ + | (1 << INTERCEPT_DR7_READ) \ + | (1 << INTERCEPT_DR0_WRITE) \ + | (1 << INTERCEPT_DR1_WRITE) \ + | (1 << INTERCEPT_DR2_WRITE) \ + | (1 << INTERCEPT_DR3_WRITE) \ + | (1 << INTERCEPT_DR4_WRITE) \ + | (1 << INTERCEPT_DR5_WRITE) \ + | (1 << INTERCEPT_DR6_WRITE) \ + | (1 << INTERCEPT_DR7_WRITE)) + +#define SVM_SEV_ES_DR_INTERCEPTS \ + ((1 << INTERCEPT_DR7_READ) \ + | (1 << INTERCEPT_DR7_WRITE)) +static inline void set_dr_intercepts (struct vcpu_svm *svm) {struct vmcb *vmcb =get_host_vmcb (svm); - vmcb->control.intercept_dr = (1 <<INTERCEPT_DR0_READ ) - | (1 <<INTERCEPT_DR1_READ ) - | (1 <<INTERCEPT_DR2_READ ) - | (1 <<INTERCEPT_DR3_READ ) - | (1 <<INTERCEPT_DR4_READ ) - | (1 <<INTERCEPT_DR5_READ ) - | (1 <<INTERCEPT_DR6_READ ) - | (1 <<INTERCEPT_DR7_READ ) - | (1 <<INTERCEPT_DR0_WRITE ) - | (1 <<INTERCEPT_DR1_WRITE ) - | (1 <<INTERCEPT_DR2_WRITE ) - | (1 <<INTERCEPT_DR3_WRITE ) - | (1 <<INTERCEPT_DR4_WRITE ) - | (1 <<INTERCEPT_DR5_WRITE ) - | (1 <<INTERCEPT_DR6_WRITE ) - | (1 <<INTERCEPT_DR7_WRITE ); + vmcb->control.intercept_dr = + (sev_es_guest (svm->vcpu.kvm)) ?SVM_SEV_ES_DR_INTERCEPTS + :SVM_DR_INTERCEPTS ;recalc_intercepts (svm); }
@@ -272,7 +303,9 @@ static inline void clr_dr_intercepts(struct vcpu_svm *svm) {struct vmcb *vmcb =get_host_vmcb (svm); - vmcb->control.intercept_dr =0 ; + vmcb->control.intercept_dr = + (sev_es_guest (svm->vcpu.kvm)) ?SVM_SEV_ES_DR_INTERCEPTS + :0 ;recalc_intercepts (svm); }
@@ -472,28 +505,6 @@ void svm_vcpu_unblocking(struct kvm_vcpu *vcpu);extern unsigned int max_sev_asid; -static inline bool sev_guest (struct kvm *kvm) -{ -#ifdef CONFIG_KVM_AMD_SEV -struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; - -return sev->active; -#else -return false; -#endif -} - -static inline bool sev_es_guest (struct kvm *kvm) -{ -#ifdef CONFIG_KVM_AMD_SEV -struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; - -return sev_guest (kvm) && sev->es_active; -#else -return false; -#endif -} -static inline bool svm_sev_enabled (void ) {return IS_ENABLED (CONFIG_KVM_AMD_SEV ) ? max_sev_asid :0 ;
Signature · 1 line
2.28.0
RFC PATCH 8/35 KVM: SVM: Prevent debugging under SEV-ES
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Since the guest register state of an SEV-ES guest is encrypted, debugging is not supported. Update the code to prevent guest debugging when the guest is an SEV-ES guest. This includes adding a callable function that is used to determine if the guest supports being debugged.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 3 +++ 4 files changed, 28 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1234,6 +1234,8 @@ struct kvm_x86_ops {void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg);void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg,unsigned long val); + +bool (*allow_debug)(struct kvm *kvm); };struct kvm_x86_nested_ops {
arch/x86/kvm/svm/svm.c +16 -0
@@ -1729,6 +1729,9 @@ static void svm_set_dr6(struct vcpu_svm *svm, unsigned long value) {struct vmcb *vmcb = svm->vmcb; +if (svm->vcpu.arch.vmsa_encrypted) +return ; +if (unlikely (value !=svm_dr6_read (svm))) {svm_dr6_write (svm, value);vmcb_mark_dirty (vmcb,VMCB_DR );
@@ -1739,6 +1742,9 @@ static void svm_sync_dirty_debug_regs(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu); +if (vcpu->arch.vmsa_encrypted) +return ; +get_debugreg (vcpu->arch.db[0 ],0 );get_debugreg (vcpu->arch.db[1 ],1 );get_debugreg (vcpu->arch.db[2 ],2 );
@@ -1757,6 +1763,9 @@ static void svm_set_dr7(struct kvm_vcpu *vcpu, unsigned long value) {struct vcpu_svm *svm =to_svm (vcpu); +if (vcpu->arch.vmsa_encrypted) +return ; +svm_dr7_write (svm, value);vmcb_mark_dirty (svm->vmcb,VMCB_DR ); }
@@ -4243,6 +4252,11 @@ static void svm_reg_write_override(struct kvm_vcpu *vcpu, enum kvm_reg reg, vmsa_reg[entry] = val; } +static bool svm_allow_debug (struct kvm *kvm) +{ +return !sev_es_guest (kvm); +} +static void svm_vm_destroy (struct kvm *kvm) {avic_vm_destroy (kvm);
@@ -4384,6 +4398,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .reg_read_override = svm_reg_read_override, .reg_write_override = svm_reg_write_override, + + .allow_debug = svm_allow_debug, };static struct kvm_x86_init_ops svm_init_ops__initdata = {
arch/x86/kvm/vmx/vmx.c +7 -0
@@ -7879,6 +7879,11 @@ static bool vmx_check_apicv_inhibit_reasons(ulong bit)return supported &BIT (bit); } +static bool vmx_allow_debug (struct kvm *kvm) +{ +return true; +} +static struct kvm_x86_ops vmx_x86_ops__initdata = { .hardware_unsetup = hardware_unsetup,
@@ -8005,6 +8010,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .need_emulation_on_page_fault = vmx_need_emulation_on_page_fault, .apic_init_signal_blocked = vmx_apic_init_signal_blocked, .migrate_timers = vmx_migrate_timers, + + .allow_debug = vmx_allow_debug, };static __init int hardware_setup (void )
arch/x86/kvm/x86.c +3 -0
@@ -9279,6 +9279,9 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,unsigned long rflags;int i, r; +if (!kvm_x86_ops.allow_debug (vcpu->kvm)) +return -EINVAL ; +vcpu_load (vcpu);if (dbg->control & (KVM_GUESTDBG_INJECT_DB |KVM_GUESTDBG_INJECT_BP )) {
Signature · 1 line
2.28.0
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 20 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since the guest register state of an SEV-ES guest is encrypted, debugging is not supported. Update the code to prevent guest debugging when the guest is an SEV-ES guest. This includes adding a callable function that is used to determine if the guest supports being debugged.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 3 +++ 4 files changed, 28 insertions(+)arch/x86/include/asm/kvm_host.h +0
-0
arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1238
Quoted 5 lines
arch/x86/include/asm/kvm_host.h +2
-0@@ -1234,6 +1234,8 @@ struct kvm_x86_ops {void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg);void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg,unsigned long val); + +bool (*allow_debug)(struct kvm *kvm);
Why add both allow_debug() and vmsa_encrypted? I assume there are scenarios where allow_debug() != vmsa_encrypted? E.g. is there a debug mode for SEV-ES where the VMSA is not encrypted, but KVM (ironically) can't intercept #DBs or something?
Alternatively, have you explored using a new VM_TYPE for SEV-ES guests? With a genericized vmsa_encrypted, that would allow something like the following for scenarios where the VMSA is not (yet?) encrypted for an SEV-ES guest. I don't love bleeding the VM type into x86.c, but for one-off quirks like this I think it'd be preferable to adding a kvm_x86_ops hook.
int kvm_arch_vcpu_ioctl_set_guest_debug(...)
{
if (vcpu->arch.guest_state_protected ||
kvm->arch.vm_type == KVM_X86_SEV_ES_VM)
return -EINVAL;
}Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 16 lines · Sean Christopherson
Quoted 15 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since the guest register state of an SEV-ES guest is encrypted, debugging is not supported. Update the code to prevent guest debugging when the guest is an SEV-ES guest. This includes adding a callable function that is used to determine if the guest supports being debugged.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 3 +++ 4 files changed, 28 insertions(+)
Quoted 15 lines
arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1238
Quoted 10 lines
arch/x86/include/asm/kvm_host.h +2
-0@@ -1234,6 +1234,8 @@ struct kvm_x86_ops {void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg);void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg,unsigned long val); + +bool (*allow_debug)(struct kvm *kvm);Why add both
allow_debug()andvmsa_encrypted? I assume there are scenarios whereallow_debug()!=vmsa_encrypted? E.g. is there a debug mode for SEV-ES where the VMSA is not encrypted, but KVM (ironically) can't intercept #DBs or something?
No, once the guest has had LAUNCH_UPDATE_VMSA run against the vCPUs, then the vCPU states are all encrypted. But that doesn't mean that debugging can't be done in the future.
Quoted 7 lines
Alternatively, have you explored using a new
VM_TYPEfor SEV-ES guests? With a genericizedvmsa_encrypted, that would allow something like the following for scenarios where the VMSA is not (yet?) encrypted for an SEV-ES guest. I don't love bleeding the VM type intox86.c, but for one-off quirks like this I think it'd be preferable to adding akvm_x86_opshook.
Quoted 7 lines
int
kvm_arch_vcpu_ioctl_set_guest_debug(...){ if (vcpu->arch.guest_state_protected || kvm->arch.vm_type == KVM_X86_SEV_ES_VM) return -EINVAL; }
I haven't explored that, I'll look into it.
Thanks,
Tom
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 33 lines · Tom Lendacky
Quoted 16 lines · Sean Christopherson
Quoted 15 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since the guest register state of an SEV-ES guest is encrypted, debugging is not supported. Update the code to prevent guest debugging when the guest is an SEV-ES guest. This includes adding a callable function that is used to determine if the guest supports being debugged.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 3 +++ 4 files changed, 28 insertions(+)Quoted 15 lines
arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1238
Quoted 10 lines
arch/x86/include/asm/kvm_host.h +2
-0@@ -1234,6 +1234,8 @@ struct kvm_x86_ops {void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg);void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg,unsigned long val); + +bool (*allow_debug)(struct kvm *kvm);Why add both
allow_debug()andvmsa_encrypted? I assume there are scenarios whereallow_debug()!=vmsa_encrypted? E.g. is there a debug mode for SEV-ES where the VMSA is not encrypted, but KVM (ironically) can't intercept #DBs or something?
Quoted 3 lines
No, once the guest has had
LAUNCH_UPDATE_VMSArun against the vCPUs, then the vCPU states are all encrypted. But that doesn't mean that debugging can't be done in the future.
I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 34 lines · Sean Christopherson
Quoted 33 lines · Tom Lendacky
Quoted 31 lines · Sean Christopherson
Quoted 20 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since the guest register state of an SEV-ES guest is encrypted, debugging is not supported. Update the code to prevent guest debugging when the guest is an SEV-ES guest. This includes adding a callable function that is used to determine if the guest supports being debugged.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 3 +++ 4 files changed, 28 insertions(+)arch/x86/include/asm/kvm_host.h +0
-0arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1238
Quoted 5 lines
arch/x86/include/asm/kvm_host.h +2
-0@@ -1234,6 +1234,8 @@ struct kvm_x86_ops {void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg);void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg,unsigned long val); + +bool (*allow_debug)(struct kvm *kvm);Why add both
allow_debug()andvmsa_encrypted? I assume there are scenarios whereallow_debug()!=vmsa_encrypted? E.g. is there a debug mode for SEV-ES where the VMSA is not encrypted, but KVM (ironically) can't intercept #DBs or something?
Quoted 7 lines
Quoted 3 lines
No, once the guest has had
LAUNCH_UPDATE_VMSArun against the vCPUs, then the vCPU states are all encrypted. But that doesn't mean that debugging can't be done in the future.I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
Thanks,
Tom
Quoted 1 line
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 43 lines · Tom Lendacky
Quoted 34 lines · Sean Christopherson
Quoted 33 lines · Tom Lendacky
Quoted 31 lines · Sean Christopherson
Quoted 20 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since the guest register state of an SEV-ES guest is encrypted, debugging is not supported. Update the code to prevent guest debugging when the guest is an SEV-ES guest. This includes adding a callable function that is used to determine if the guest supports being debugged.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 3 +++ 4 files changed, 28 insertions(+)arch/x86/include/asm/kvm_host.h +0
-0arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1238
Quoted 5 lines
arch/x86/include/asm/kvm_host.h +2
-0@@ -1234,6 +1234,8 @@ struct kvm_x86_ops {void (*reg_read_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg);void (*reg_write_override)(struct kvm_vcpu *vcpu,enum kvm_reg reg,unsigned long val); + +bool (*allow_debug)(struct kvm *kvm);Why add both
allow_debug()andvmsa_encrypted? I assume there are scenarios whereallow_debug()!=vmsa_encrypted? E.g. is there a debug mode for SEV-ES where the VMSA is not encrypted, but KVM (ironically) can't intercept #DBs or something?Quoted 7 lines
Quoted 3 lines
No, once the guest has had
LAUNCH_UPDATE_VMSArun against the vCPUs, then the vCPU states are all encrypted. But that doesn't mean that debugging can't be done in the future.I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Quoted 5 lines
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
So I don't think there's any guarantee that the KVM_SET_GUEST_DEBUG ioctl couldn't be called before the VMSA is encrypted, meaning I can't check the arch.guest_state_protected bit for that call. So if we really want to get rid of the allow_debug() op, I'd need some other way to indicate that this is an SEV-ES / protected state guest.
How are you planning on blocking this ioctl for TDX? Would the arch.guest_state_protected bit be sit earlier than is done for SEV-ES?
Thanks,
Tom
Quoted 5 lines
Thanks,
TomQuoted 1 line
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 12 lines · Tom Lendacky
Quoted 10 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
Quoted 5 lines
So I don't think there's any guarantee that the
KVM_SET_GUEST_DEBUGioctl couldn't be called before the VMSA is encrypted, meaning I can't check the arch.guest_state_protected bit for that call. So if we really want to get rid of theallow_debug()op, I'd need some other way to indicate that this is an SEV-ES / protected state guest.
Would anything break if KVM "speculatively" set guest_state_protected before LAUNCH_UPDATE_VMSA? E.g. does KVM need to emulate before LAUNCH_UPDATE_VMSA?
Quoted 2 lines
How are you planning on blocking this ioctl for TDX? Would the arch.guest_state_protected bit be sit earlier than is done for SEV-ES?
Yep, guest_state_protected is set from time zero (kvm_x86_ops.vm_init) as guest state is encrypted/inaccessible from the get go. The flag actually gets turned off for debuggable TDX guests, but that's also forced to happen before the KVM_RUN can be invoked (TDX architecture) and is a one-time configuration, i.e. userspace can flip the switch exactly once, and only at a very specific point in time.
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 13 lines · Sean Christopherson
Quoted 12 lines · Tom Lendacky
Quoted 10 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
Quoted 8 lines
Quoted 5 lines
So I don't think there's any guarantee that the
KVM_SET_GUEST_DEBUGioctl couldn't be called before the VMSA is encrypted, meaning I can't check the arch.guest_state_protected bit for that call. So if we really want to get rid of theallow_debug()op, I'd need some other way to indicate that this is an SEV-ES / protected state guest.Would anything break if KVM "speculatively" set
guest_state_protectedbeforeLAUNCH_UPDATE_VMSA? E.g. does KVM need to emulate beforeLAUNCH_UPDATE_VMSA?
Yes, the way the code is set up, the guest state (VMSA) is initialized in the same way it is today (mostly) and that state is encrypted by the LAUNCH_UPDATE_VMSA call. I check the guest_state_protected bit to decide on whether to direct the updates to the real VMSA (before it's encrypted) or the GHCB (that's the get_vmsa() function from patch #5).
Thanks,
Tom
Quoted 11 lines
Quoted 3 lines
How are you planning on blocking this ioctl for TDX? Would the arch.guest_state_protected bit be sit earlier than is done for SEV-ES?
Yep,
guest_state_protectedis set from time zero (kvm_x86_ops.vm_init) as guest state is encrypted/inaccessible from the get go. The flag actually gets turned off for debuggable TDX guests, but that's also forced to happen before theKVM_RUNcan be invoked (TDX architecture) and is a one-time configuration, i.e. userspace can flip the switch exactly once, and only at a very specific point in time.
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 22 lines · Tom Lendacky
Quoted 19 lines · Sean Christopherson
Quoted 17 lines · Tom Lendacky
Quoted 10 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
So I don't think there's any guarantee that the
KVM_SET_GUEST_DEBUGioctl couldn't be called before the VMSA is encrypted, meaning I can't check the arch.guest_state_protected bit for that call. So if we really want to get rid of theallow_debug()op, I'd need some other way to indicate that this is an SEV-ES / protected state guest.
Quoted 8 lines
Quoted 2 lines
Would anything break if KVM "speculatively" set
guest_state_protectedbeforeLAUNCH_UPDATE_VMSA? E.g. does KVM need to emulate beforeLAUNCH_UPDATE_VMSA?Yes, the way the code is set up, the guest state (VMSA) is initialized in the same way it is today (mostly) and that state is encrypted by the
LAUNCH_UPDATE_VMSAcall. I check theguest_state_protectedbit to decide on whether to direct the updates to the real VMSA (before it's encrypted) or the GHCB (that's theget_vmsa()function from patch #5).
Ah, gotcha. Would it work to set guest_state_protected[*] from time zero, and move vmsa_encrypted to struct vcpu_svm? I.e. keep vmsa_encrypted, but use it only for guiding get_vmsa() and related behavior.
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 32 lines · Sean Christopherson
Quoted 22 lines · Tom Lendacky
Quoted 19 lines · Sean Christopherson
Quoted 17 lines · Tom Lendacky
Quoted 10 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
So I don't think there's any guarantee that the
KVM_SET_GUEST_DEBUGioctl couldn't be called before the VMSA is encrypted, meaning I can't check the arch.guest_state_protected bit for that call. So if we really want to get rid of theallow_debug()op, I'd need some other way to indicate that this is an SEV-ES / protected state guest.Quoted 8 lines
Quoted 2 lines
Would anything break if KVM "speculatively" set
guest_state_protectedbeforeLAUNCH_UPDATE_VMSA? E.g. does KVM need to emulate beforeLAUNCH_UPDATE_VMSA?Yes, the way the code is set up, the guest state (VMSA) is initialized in the same way it is today (mostly) and that state is encrypted by the
LAUNCH_UPDATE_VMSAcall. I check theguest_state_protectedbit to decide on whether to direct the updates to the real VMSA (before it's encrypted) or the GHCB (that's theget_vmsa()function from patch #5).
Quoted 3 lines
Ah, gotcha. Would it work to set guest_state_protected[*] from time zero, and move
vmsa_encryptedto structvcpu_svm? I.e. keepvmsa_encrypted, but use it only for guidingget_vmsa()and related behavior.
It is mainly __set_sregs() that needs to know when to allow the register writes and when not to. During guest initialization, __set_sregs is how some of the VMSA is initialized by Qemu.
Thanks,
Tom
Quoted 1 line
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 33 lines · Tom Lendacky
Quoted 32 lines · Sean Christopherson
Quoted 30 lines · Tom Lendacky
Quoted 13 lines · Sean Christopherson
Quoted 12 lines · Tom Lendacky
Quoted 10 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
Quoted 8 lines
Quoted 5 lines
So I don't think there's any guarantee that the
KVM_SET_GUEST_DEBUGioctl couldn't be called before the VMSA is encrypted, meaning I can't check the arch.guest_state_protected bit for that call. So if we really want to get rid of theallow_debug()op, I'd need some other way to indicate that this is an SEV-ES / protected state guest.Would anything break if KVM "speculatively" set
guest_state_protectedbeforeLAUNCH_UPDATE_VMSA? E.g. does KVM need to emulate beforeLAUNCH_UPDATE_VMSA?Yes, the way the code is set up, the guest state (VMSA) is initialized in the same way it is today (mostly) and that state is encrypted by the
LAUNCH_UPDATE_VMSAcall. I check theguest_state_protectedbit to decide on whether to direct the updates to the real VMSA (before it's encrypted) or the GHCB (that's theget_vmsa()function from patch #5).
Quoted 7 lines
Quoted 3 lines
Ah, gotcha. Would it work to set guest_state_protected[*] from time zero, and move
vmsa_encryptedto structvcpu_svm? I.e. keepvmsa_encrypted, but use it only for guidingget_vmsa()and related behavior.It is mainly
__set_sregs()that needs to know when to allow the register writes and when not to. During guest initialization,__set_sregsis how some of the VMSA is initialized by Qemu.
Hmm. I assume that also means KVM_SET_REGS and KVM_GET_XCRS are also legal before the VMSA is encrypted? If so, then the current behavior of setting vmsa_encrypted "late" make sense. KVM_SET_FPU/XSAVE can be handled by not allocating guest_fpu, i.e. they can be disallowed from time zero without adding an SEV-ES specific check.
Which brings us back to KVM_SET_GUEST_DEBUG. What would happen if that were allowed prior to VMSA encryption? If LAUNCH_UPDATE_VMSA acts as a sort of reset, one thought would be to allow KVM_SET_GUEST_DEBUG and then sanitize KVM's state during LAUNCH_UPDATE_VMSA. Or perhaps even better, disallow LAUNCH_UPDATE_VMSA if vcpu->guest_debug!=0. That would allow using debug capabilities up until LAUNCH_UPDATE_VMSA without adding much burden to KVM.
Re: [RFC PATCH 08/35] KVM: SVM: Prevent debugging under SEV-ES
Quoted 48 lines · Sean Christopherson
Quoted 33 lines · Tom Lendacky
Quoted 32 lines · Sean Christopherson
Quoted 30 lines · Tom Lendacky
Quoted 13 lines · Sean Christopherson
Quoted 12 lines · Tom Lendacky
Quoted 10 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
I don't quite follow the "doesn't mean debugging can't be done in the future". Does that imply that debugging could be supported for SEV-ES guests, even if they have an encrypted VMSA?
Almost anything can be done with software. It would require a lot of hypervisor and guest code and changes to the GHCB spec, etc. So given that, probably just the check for arch.guest_state_protected is enough for now. I'll just need to be sure none of the debugging paths can be taken before the VMSA is encrypted.
Quoted 8 lines
Quoted 5 lines
So I don't think there's any guarantee that the
KVM_SET_GUEST_DEBUGioctl couldn't be called before the VMSA is encrypted, meaning I can't check the arch.guest_state_protected bit for that call. So if we really want to get rid of theallow_debug()op, I'd need some other way to indicate that this is an SEV-ES / protected state guest.Would anything break if KVM "speculatively" set
guest_state_protectedbeforeLAUNCH_UPDATE_VMSA? E.g. does KVM need to emulate beforeLAUNCH_UPDATE_VMSA?Yes, the way the code is set up, the guest state (VMSA) is initialized in the same way it is today (mostly) and that state is encrypted by the
LAUNCH_UPDATE_VMSAcall. I check theguest_state_protectedbit to decide on whether to direct the updates to the real VMSA (before it's encrypted) or the GHCB (that's theget_vmsa()function from patch #5).Quoted 7 lines
Quoted 3 lines
Ah, gotcha. Would it work to set guest_state_protected[*] from time zero, and move
vmsa_encryptedto structvcpu_svm? I.e. keepvmsa_encrypted, but use it only for guidingget_vmsa()and related behavior.It is mainly
__set_sregs()that needs to know when to allow the register writes and when not to. During guest initialization,__set_sregsis how some of the VMSA is initialized by Qemu.Hmm. I assume that also means
KVM_SET_REGSandKVM_GET_XCRSare also legal before the VMSA is encrypted? If so, then the current behavior of settingvmsa_encrypted"late" make sense. KVM_SET_FPU/XSAVE can be handled by not allocatingguest_fpu, i.e. they can be disallowed from time zero without adding an SEV-ES specific check.
Quoted 6 lines
Which brings us back to
KVM_SET_GUEST_DEBUG. What would happen if that were allowed prior to VMSA encryption? IfLAUNCH_UPDATE_VMSAacts as a sort of reset, one thought would be to allowKVM_SET_GUEST_DEBUGand then sanitize KVM's state duringLAUNCH_UPDATE_VMSA. Or perhaps even better, disallowLAUNCH_UPDATE_VMSAif vcpu->guest_debug!=0. That would allow using debug capabilities up untilLAUNCH_UPDATE_VMSAwithout adding much burden to KVM.
I think the vcpu->guest_debug check before the LAUNCH_UPDATE_VMSA would be good. I'll remove the allow_debug() op and replace it with the guest_state_protected check in its place.
Thanks,
Tom
Quoted 1 line
RFC PATCH 9/35 KVM: SVM: Do not emulate MMIO under SEV-ES
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
When a guest is running as an SEV-ES guest, it is not possible to emulate MMIO. Add support to prevent trying to perform MMIO emulation.
--- arch/x86/kvm/mmu/mmu.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kvm/mmu/mmu.c +7 -0
@@ -5485,6 +5485,13 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,if (!mmio_info_in_cache (vcpu, cr2_or_gpa, direct) && !is_guest_mode (vcpu)) emulation_type |=EMULTYPE_ALLOW_RETRY_PF ; emulate: +/* +* When the guest is an SEV-ES guest, emulation is not possible. Allow +* the guest to handle the MMIO emulation. +*/ +if (vcpu->arch.vmsa_encrypted) +return 1 ; +/* * On AMD platforms, under certain conditions insn_len may be zero on #NPF. * This can happen if a guest gets a page-fault on data access but the HW
Signature · 1 line
2.28.0
Re: [RFC PATCH 09/35] KVM: SVM: Do not emulate MMIO under SEV-ES
Quoted 15 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
When a guest is running as an SEV-ES guest, it is not possible to emulate MMIO. Add support to prevent trying to perform MMIO emulation.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/x86/kvm/mmu/mmu.c +0
-0
arch/x86/kvm/mmu/mmu.c int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code, · line 5493
Quoted 9 lines
arch/x86/kvm/mmu/mmu.c +6
-0@@ -5485,6 +5485,13 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,if (!mmio_info_in_cache (vcpu, cr2_or_gpa, direct) && !is_guest_mode (vcpu)) emulation_type |=EMULTYPE_ALLOW_RETRY_PF ; emulate: +/* +* When the guest is an SEV-ES guest, emulation is not possible. Allow +* the guest to handle the MMIO emulation. +*/ +if (vcpu->arch.vmsa_encrypted) +return 1 ;
A better approach is to refactor need_emulation_on_page_fault() (the hook that's just out of sight in this patch) into a more generic kvm_x86_ops.is_emulatable() so that the latter can be used to kill emulation everywhere, and for other reasons. E.g. TDX obviously shares very similar logic, but SGX also adds a case where KVM can theoretically end up in an emulator path without the ability to access the necessary guest state.
I have exactly such a prep patch (because SGX and TDX...), I'll get it posted in the next day or two.
Quoted 7 lines
arch/x86/kvm/mmu/mmu.c +1
-0+/* * On AMD platforms, under certain conditions insn_len may be zero on #NPF. * This can happen if a guest gets a page-fault on data access but the HW Signature · 1 line
2.28.0
Re: [RFC PATCH 09/35] KVM: SVM: Do not emulate MMIO under SEV-ES
Quoted 33 lines · Sean Christopherson
Quoted 15 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
When a guest is running as an SEV-ES guest, it is not possible to emulate MMIO. Add support to prevent trying to perform MMIO emulation.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/x86/kvm/mmu/mmu.c +0
-0arch/x86/kvm/mmu/mmu.c int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code, · line 5493
Quoted 9 lines
arch/x86/kvm/mmu/mmu.c +6
-0@@ -5485,6 +5485,13 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,if (!mmio_info_in_cache (vcpu, cr2_or_gpa, direct) && !is_guest_mode (vcpu)) emulation_type |=EMULTYPE_ALLOW_RETRY_PF ; emulate: +/* +* When the guest is an SEV-ES guest, emulation is not possible. Allow +* the guest to handle the MMIO emulation. +*/ +if (vcpu->arch.vmsa_encrypted) +return 1 ;A better approach is to refactor
need_emulation_on_page_fault()(the hook that's just out of sight in this patch) into a more generickvm_x86_ops.is_emulatable()so that the latter can be used to kill emulation everywhere, and for other reasons. E.g. TDX obviously shares very similar logic, but SGX also adds a case where KVM can theoretically end up in an emulator path without the ability to access the necessary guest state.
Quoted 2 lines
I have exactly such a prep patch (because SGX and TDX...), I'll get it posted in the next day or two.
Sounds good. I'll check it out when it's posted.
Thanks,
Tom
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +1
-0+/* * On AMD platforms, under certain conditions insn_len may be zero on #NPF. * This can happen if a guest gets a page-fault on data access but the HW Signature · 1 line
2.28.0
RFC PATCH 10/35 KVM: SVM: Cannot re-initialize the VMCB after shutdown with SEV-ES
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
When a SHUTDOWN VMEXIT is encountered, normally the VMCB is re-initialized so that the guest can be re-launched. But when a guest is running as an SEV-ES guest, the VMSA cannot be re-initialized because it has been encrypted. For now, just return -EINVAL to prevent a possible attempt at a guest reset.
--- arch/x86/kvm/svm/svm.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kvm/svm/svm.c +7 -0
@@ -1953,6 +1953,13 @@ static int shutdown_interception(struct vcpu_svm *svm) {struct kvm_run *kvm_run = svm->vcpu.run; +/* +* The VM save area has already been encrypted so it +* cannot be reinitialized - just terminate. +*/ +if (sev_es_guest (svm->vcpu.kvm)) +return -EINVAL ; +/* * VMCB is undefined after a SHUTDOWN intercept * so reinitialize it.
Signature · 1 line
2.28.0
RFC PATCH 11/35 KVM: SVM: Prepare for SEV-ES exit handling in the sev.c file
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
This is a pre-patch to consolidate some exit handling code into callable functions. Follow-on patches for SEV-ES exit handling will then be able to use them from the sev.c file.
--- arch/x86/kvm/svm/svm.c | 64 +++++++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 26 deletions(-)
arch/x86/kvm/svm/svm.c +38 -26
@@ -3047,6 +3047,43 @@ static void dump_vmcb(struct kvm_vcpu *vcpu) "excp_to:", save->last_excp_to); } +static bool svm_is_supported_exit (struct kvm_vcpu *vcpu,u64 exit_code) +{ +if (exit_code <ARRAY_SIZE (svm_exit_handlers) && + svm_exit_handlers[exit_code]) +return true; + +vcpu_unimpl (vcpu,"svm: unexpected exit reason 0x%llx\n" , exit_code); +dump_vmcb (vcpu); + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON ; + vcpu->run->internal.ndata =2 ; + vcpu->run->internal.data[0 ] = exit_code; + vcpu->run->internal.data[1 ] = vcpu->arch.last_vmentry_cpu; + +return false; +} + +static int svm_invoke_exit_handler (struct vcpu_svm *svm,u64 exit_code) +{ +if (!svm_is_supported_exit (&svm->vcpu, exit_code)) +return 0 ; + +#ifdef CONFIG_RETPOLINE +if (exit_code ==SVM_EXIT_MSR ) +return msr_interception (svm); +else if (exit_code ==SVM_EXIT_VINTR ) +return interrupt_window_interception (svm); +else if (exit_code ==SVM_EXIT_INTR ) +return intr_interception (svm); +else if (exit_code ==SVM_EXIT_HLT ) +return halt_interception (svm); +else if (exit_code ==SVM_EXIT_NPF ) +return npf_interception (svm); +#endif +return svm_exit_handlers[exit_code](svm); +} +static void svm_get_exit_info (struct kvm_vcpu *vcpu,u64 *info1,u64 *info2) {struct vmcb_control_area *control = &to_svm (vcpu)->vmcb->control;
@@ -3113,32 +3150,7 @@ static int handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)if (exit_fastpath !=EXIT_FASTPATH_NONE )return 1 ; -if (exit_code >=ARRAY_SIZE (svm_exit_handlers) - || !svm_exit_handlers[exit_code]) { -vcpu_unimpl (vcpu,"svm: unexpected exit reason 0x%x\n" , exit_code); -dump_vmcb (vcpu); - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror = -KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON ; - vcpu->run->internal.ndata =2 ; - vcpu->run->internal.data[0 ] = exit_code; - vcpu->run->internal.data[1 ] = vcpu->arch.last_vmentry_cpu; -return 0 ; - } - -#ifdef CONFIG_RETPOLINE -if (exit_code ==SVM_EXIT_MSR ) -return msr_interception (svm); -else if (exit_code ==SVM_EXIT_VINTR ) -return interrupt_window_interception (svm); -else if (exit_code ==SVM_EXIT_INTR ) -return intr_interception (svm); -else if (exit_code ==SVM_EXIT_HLT ) -return halt_interception (svm); -else if (exit_code ==SVM_EXIT_NPF ) -return npf_interception (svm); -#endif -return svm_exit_handlers[exit_code](svm); +return svm_invoke_exit_handler (svm, exit_code); }static void reload_tss (struct kvm_vcpu *vcpu)
Signature · 1 line
2.28.0
A message that is not in the archive (sent off-list or not yet received): 20200915172148.GE8420@sjchrist-ice
Re: [RFC PATCH 11/35] KVM: SVM: Prepare for SEV-ES exit handling in the sev.c file
Quoted 39 lines · Sean Christopherson
Quoted 22 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
This is a pre-patch to consolidate some exit handling code into callable functions. Follow-on patches for SEV-ES exit handling will then be able to use them from the
sev.cfile.Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/svm/svm.c | 64 +++++++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 26 deletions(-)arch/x86/kvm/svm/svm.c +3
-0@@ -3047,6 +3047,43 @@ static void dump_vmcb(struct kvm_vcpu *vcpu) "excp_to:", save->last_excp_to); } +static bool svm_is_supported_exit (struct kvm_vcpu *vcpu,u64 exit_code) +{ + if (exit_code < ARRAY_SIZE(svm_exit_handlers) &&arch/x86/kvm/svm/svm.c static void dump_vmcb(struct kvm_vcpu *vcpu) · line 3062
Quoted 10 lines
arch/x86/kvm/svm/svm.c +10
-0+ svm_exit_handlers[exit_code]) +return true; + +vcpu_unimpl (vcpu,"svm: unexpected exit reason 0x%llx\n" , exit_code); +dump_vmcb (vcpu); + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON ; + vcpu->run->internal.ndata =2 ; + vcpu->run->internal.data[0 ] = exit_code; + vcpu->run->internal.data[1 ] = vcpu->arch.last_vmentry_cpu;Based on the name "
is_supported_exit", I would prefer thatvcpu->runbe filled in by the caller. Looking at the below code wheresvm_is_supported_exit()is checked, without diving into the implementation of the helper it's not at all clear thatvcpu->runis filled.
Quoted 3 lines
Assuming
svm_invoke_exit_handler()is the only user, it probably makes sense to fillvcpu->runin the caller. If there will be multiple callers, then it'd be nice to renamesvm_is_supported_exit()to e.g.svm_handle_invalid_exit()or so.
Will change.
Quoted 31 lines
Quoted 13 lines
arch/x86/kvm/svm/svm.c +12
-0+ +return false; +} + +static int svm_invoke_exit_handler (struct vcpu_svm *svm,u64 exit_code) +{ +if (!svm_is_supported_exit (&svm->vcpu, exit_code)) +return 0 ; + +#ifdef CONFIG_RETPOLINE +if (exit_code ==SVM_EXIT_MSR ) +return msr_interception (svm);arch/x86/kvm/svm/svm.c static void dump_vmcb(struct kvm_vcpu *vcpu) · line 3084
Quoted 10 lines
arch/x86/kvm/svm/svm.c +10
-0+ else if (exit_code == SVM_EXIT_VINTR) + return interrupt_window_interception(svm); + else if (exit_code == SVM_EXIT_INTR) + return intr_interception(svm); + else if (exit_code == SVM_EXIT_HLT) + return halt_interception(svm); + else if (exit_code == SVM_EXIT_NPF) + return npf_interception(svm); +#endif +return svm_exit_handlers[exit_code](svm);Now I see why
kvm_skip_emulated_instruction()is bailing on SEV-ES guests, #VMGEXIT simply routes through the legacy exit handlers. Which totally makes sense from a code reuse perspective, but the lack of sanity checking with that approach is undesirable, e.g. I assume there are a big pile of exit codes that are flat out unsupported for SEV-ES, and ideally KVM would yell loudly if it tries to doskip_emulated_instruction()for a protected guest.
Quoted 3 lines
Rather than route through the legacy handlers, I suspect it will be more desirable in the long run to have a separate path for #VMGEXIT, i.e. a path that does the back half of emulation (the front half being the "fetch" phase).
Except there are some automatic exits (AE events) that don't go through VMGEXIT and would need to be sure the RIP isn't updated. I can audit the AE events and see what's possible.
Additionally, maybe just ensuring that kvm_x86_ops.get_rflags() doesn't return something with the TF flag set eliminates the need for the change to kvm_skip_emulated_instruction().
Quoted 6 lines
The biggest downsides would be code duplication and ongoing maintenance. Our current approach for TDX is to eat that overhead, because it's not _that_ much code. But, maybe there's a middle ground, e.g. using the existing flows but having them skip (heh)
kvm_skip_emulated_instruction()for protected guests.
Quoted 7 lines
There are a few flows, e.g. MMIO emulation, that will need dedicated implementations, but I'm 99% certain we can put those in
x86.cand share them between SEV-ES and TDX.One question that will impact KVM's options: can KVM inject exceptions to SEV-ES guests? E.g. if the guest request emulation of a bogus WRMSR, is the #GP delivered as an actual #GP, or is the error "returned" via the GHCB?
Yes, for SEV-ES guest, you can inject exceptions. But, when using VMGEXIT for, e.g. WRMSR, you would pass an exception error code back to the #VC handler that will propagate that exception in the guest with the registers associated with the #VC.
Thanks,
Tom
Quoted 8 lines
The most annoying hiccup is that TDX doesn't use the "standard" GPRs, e.g. MSR index isn't passed via ECX. I'll play around with a common
x86.cimplementation to see how painful it will be to use for TDX. Given that SEV-ES is more closely aligned with legacy behavior (in terms of registers usage), getting SEV-ES working on a common base should be relatively easy, at least in theory :-).
RFC PATCH 12/35 KVM: SVM: Add initial support for a VMGEXIT VMEXIT
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
SEV-ES adds a new VMEXIT reason code, VMGEXIT. Initial support for a VMGEXIT includes reading the guest GHCB and performing the requested action. Since many of the VMGEXIT exit reasons correspond to existing VMEXIT reasons, the information from the GHCB is copied into the VMCB control exit code areas and then the standard exit handlers are invoked, similar to standard VMEXIT processing. Before restarting the vCPU, the now updated SVM GHCB is copied back to the guest GHCB.
--- arch/x86/include/asm/svm.h | 2 +- arch/x86/include/uapi/asm/svm.h | 7 ++++ arch/x86/kvm/svm/sev.c | 65 +++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 6 ++- arch/x86/kvm/svm/svm.h | 7 ++++ 5 files changed, 85 insertions(+), 2 deletions(-)
arch/x86/include/asm/svm.h +1 -1
@@ -82,7 +82,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {u32 exit_int_info_err;u64 nested_ctl;u64 avic_vapic_bar; -u8 reserved_4[8 ]; +u64 ghcb_gpa;u32 event_inj;u32 event_inj_err;u64 nested_cr3;
arch/x86/include/uapi/asm/svm.h +7 -0
@@ -80,6 +80,7 @@#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402 +#define SVM_EXIT_VMGEXIT 0x403/* SEV-ES software-defined VMGEXIT events */ #define SVM_VMGEXIT_MMIO_READ 0x80000001
@@ -185,6 +186,12 @@ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \ + { SVM_EXIT_VMGEXIT, "vmgexit" }, \ + { SVM_VMGEXIT_MMIO_READ, "vmgexit_mmio_read" }, \ + { SVM_VMGEXIT_MMIO_WRITE, "vmgexit_mmio_write" }, \ + { SVM_VMGEXIT_NMI_COMPLETE, "vmgexit_nmi_complete" }, \ + { SVM_VMGEXIT_AP_HLT_LOOP, "vmgexit_ap_hlt_loop" }, \ + { SVM_VMGEXIT_AP_JUMP_TABLE, "vmgexit_ap_jump_table" }, \ { SVM_EXIT_ERR, "invalid_guest_state" }
arch/x86/kvm/svm/sev.c +65 -0
79 lines not on this page · raw message
arch/x86/kvm/svm/svm.c +5 -1
@@ -2929,6 +2929,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception, [SVM_EXIT_AVIC_UNACCELERATED_ACCESS] = avic_unaccelerated_access_interception, + [SVM_EXIT_VMGEXIT] = sev_handle_vmgexit, };static void dump_vmcb (struct kvm_vcpu *vcpu)
@@ -2968,6 +2969,7 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)pr_err ("%-20s%lld\n" ,"nested_ctl:" , control->nested_ctl);pr_err ("%-20s%016llx\n" ,"nested_cr3:" , control->nested_cr3);pr_err ("%-20s%016llx\n" ,"avic_vapic_bar:" , control->avic_vapic_bar); +pr_err ("%-20s%016llx\n" ,"ghcb:" , control->ghcb_gpa);pr_err ("%-20s%08x\n" ,"event_inj:" , control->event_inj);pr_err ("%-20s%08x\n" ,"event_inj_err:" , control->event_inj_err);pr_err ("%-20s%lld\n" ,"virt_ext:" , control->virt_ext);
@@ -3064,7 +3066,7 @@ static bool svm_is_supported_exit(struct kvm_vcpu *vcpu, u64 exit_code)return false; } -static int svm_invoke_exit_handler (struct vcpu_svm *svm,u64 exit_code) +int svm_invoke_exit_handler (struct vcpu_svm *svm,u64 exit_code) {if (!svm_is_supported_exit (&svm->vcpu, exit_code))return 0 ;
@@ -3080,6 +3082,8 @@ static int svm_invoke_exit_handler(struct vcpu_svm *svm, u64 exit_code)return halt_interception (svm); else if (exit_code == SVM_EXIT_NPF) return npf_interception(svm); + else if (exit_code == SVM_EXIT_VMGEXIT) + return sev_handle_vmgexit(svm);#endif return svm_exit_handlers[exit_code](svm); }
arch/x86/kvm/svm/svm.h +7 -0
@@ -17,6 +17,7 @@#include <linux/kvm_types.h> #include <linux/kvm_host.h> +#include <linux/bits.h> #include <asm/svm.h>
@@ -163,6 +164,7 @@ struct vcpu_svm {/* SEV-ES support */ struct vmcb_save_area *vmsa;struct ghcb *ghcb; +struct kvm_host_map ghcb_map; };struct svm_cpu_data {
@@ -399,6 +401,7 @@ bool svm_smi_blocked(struct kvm_vcpu *vcpu);bool svm_nmi_blocked (struct kvm_vcpu *vcpu);bool svm_interrupt_blocked (struct kvm_vcpu *vcpu);void svm_set_gif (struct vcpu_svm *svm,bool value); +int svm_invoke_exit_handler (struct vcpu_svm *svm,u64 exit_code);/* nested.c */
@@ -503,6 +506,9 @@ void svm_vcpu_unblocking(struct kvm_vcpu *vcpu);/* sev.c */ +#define GHCB_MSR_INFO_POS 0 +#define GHCB_MSR_INFO_MASK (BIT_ULL(12) - 1) +extern unsigned int max_sev_asid;static inline bool svm_sev_enabled (void )
@@ -519,6 +525,7 @@ int svm_unregister_enc_region(struct kvm *kvm,void pre_sev_run (struct vcpu_svm *svm,int cpu);void __init sev_hardware_setup (void );void sev_hardware_teardown (void ); +int sev_handle_vmgexit (struct vcpu_svm *svm);/* VMSA Accessor functions */
Signature · 1 line
2.28.0
RFC PATCH 13/35 KVM: SVM: Create trace events for VMGEXIT processing
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Add trace events for entry to and exit from VMGEXIT processing. The vCPU id and the exit reason will be common for the trace events. The exit info fields and valid bitmap fields will represent the input and output values for the entry and exit events, respectively.
--- arch/x86/kvm/svm/sev.c | 6 +++++ arch/x86/kvm/trace.h | 55 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 2 ++ 3 files changed, 63 insertions(+)
arch/x86/kvm/svm/sev.c +6 -0
@@ -14,9 +14,11 @@#include <linux/psp-sev.h> #include <linux/pagemap.h> #include <linux/swap.h> +#include <linux/trace_events.h> #include "x86.h" #include "svm.h" +#include "trace.h" static int sev_flush_asids (void );static DECLARE_RWSEM (sev_deactivate_lock);
@@ -1185,6 +1187,8 @@ static void pre_sev_es_run(struct vcpu_svm *svm)if (!svm->ghcb)return ; +trace_kvm_vmgexit_exit (svm->vcpu.vcpu_id, svm->ghcb); +kvm_vcpu_unmap (&svm->vcpu, &svm->ghcb_map, true); svm->ghcb =NULL ; }
@@ -1246,6 +1250,8 @@ int sev_handle_vmgexit(struct vcpu_svm *svm) svm->ghcb = svm->ghcb_map.hva; ghcb = svm->ghcb_map.hva; +trace_kvm_vmgexit_enter (svm->vcpu.vcpu_id, ghcb); + control->exit_code =lower_32_bits (ghcb_get_sw_exit_code (ghcb)); control->exit_code_hi =upper_32_bits (ghcb_get_sw_exit_code (ghcb)); control->exit_info_1 =ghcb_get_sw_exit_info_1 (ghcb);
arch/x86/kvm/trace.h +55 -0
61 lines not on this page · raw message
arch/x86/kvm/x86.c +2 -0
@@ -10790,3 +10790,5 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_avic_unaccelerated_access);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_incomplete_ipi);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_ga_log);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_apicv_update_request); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_enter); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_exit);
Signature · 1 line
2.28.0
RFC PATCH 14/35 KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x002
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The GHCB defines a GHCB MSR protocol using the lower 12-bits of the GHCB MSR (in the hypervisor this corresponds to the GHCB GPA field in the VMCB).
Function 0x002 is a request to set the GHCB MSR value to the SEV INFO as per the specification via the VMCB GHCB GPA field.
--- arch/x86/kvm/svm/sev.c | 26 +++++++++++++++++++++++++- arch/x86/kvm/svm/svm.h | 17 +++++++++++++++++ 2 files changed, 42 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/sev.c +25 -1
@@ -20,6 +20,7 @@#include "svm.h" #include "trace.h" +static u8 sev_enc_bit;static int sev_flush_asids (void );static DECLARE_RWSEM (sev_deactivate_lock);static DEFINE_MUTEX (sev_bitmap_lock);
@@ -1130,6 +1131,9 @@ void __init sev_hardware_setup(void)/* Retrieve SEV CPUID information */ cpuid (0x8000001f , &eax, &ebx, &ecx, &edx); +/* Set encryption bit location for SEV-ES guests */ + sev_enc_bit = ebx &0x3f ; +/* Maximum number of encrypted guests supported simultaneously */ max_sev_asid = ecx;
@@ -1219,9 +1223,29 @@ void pre_sev_run(struct vcpu_svm *svm, int cpu)vmcb_mark_dirty (svm->vmcb,VMCB_ASID ); } +static void set_ghcb_msr (struct vcpu_svm *svm,u64 value) +{ + svm->vmcb->control.ghcb_gpa = value; +} +static int sev_handle_vmgexit_msr_protocol (struct vcpu_svm *svm) { -return -EINVAL ; +struct vmcb_control_area *control = &svm->vmcb->control; +u64 ghcb_info; + + ghcb_info = control->ghcb_gpa &GHCB_MSR_INFO_MASK ; + +switch (ghcb_info) { +case GHCB_MSR_SEV_INFO_REQ : +set_ghcb_msr (svm,GHCB_MSR_SEV_INFO (GHCB_VERSION_MAX , +GHCB_VERSION_MIN , + sev_enc_bit)); +break ; +default : +return -EINVAL ; + } + +return 1 ; }int sev_handle_vmgexit (struct vcpu_svm *svm)
arch/x86/kvm/svm/svm.h +17 -0
@@ -506,9 +506,26 @@ void svm_vcpu_unblocking(struct kvm_vcpu *vcpu);/* sev.c */ +#define GHCB_VERSION_MAX 1ULL +#define GHCB_VERSION_MIN 1ULL +#define GHCB_MSR_INFO_POS 0#define GHCB_MSR_INFO_MASK (BIT_ULL(12) - 1) +#define GHCB_MSR_SEV_INFO_RESP 0x001 +#define GHCB_MSR_SEV_INFO_REQ 0x002 +#define GHCB_MSR_VER_MAX_POS 48 +#define GHCB_MSR_VER_MAX_MASK 0xffff +#define GHCB_MSR_VER_MIN_POS 32 +#define GHCB_MSR_VER_MIN_MASK 0xffff +#define GHCB_MSR_CBIT_POS 24 +#define GHCB_MSR_CBIT_MASK 0xff +#define GHCB_MSR_SEV_INFO (_max, _min, _cbit) \ + ((((_max) & GHCB_MSR_VER_MAX_MASK) << GHCB_MSR_VER_MAX_POS) | \ + (((_min) & GHCB_MSR_VER_MIN_MASK) << GHCB_MSR_VER_MIN_POS) | \ + (((_cbit) & GHCB_MSR_CBIT_MASK) << GHCB_MSR_CBIT_POS) | \ + GHCB_MSR_SEV_INFO_RESP) +extern unsigned int max_sev_asid;static inline bool svm_sev_enabled (void )
Signature · 1 line
2.28.0
RFC PATCH 15/35 KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x004
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The GHCB defines a GHCB MSR protocol using the lower 12-bits of the GHCB MSR (in the hypervisor this corresponds to the GHCB GPA field in the VMCB).
Function 0x004 is a request for CPUID information. Only a single CPUID result register can be sent per invocation, so the protocol defines the register that is requested. The GHCB MSR value is set to the CPUID register value as per the specification via the VMCB GHCB GPA field.
--- arch/x86/kvm/svm/sev.c | 55 ++++++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/svm/svm.h | 9 +++++++ 2 files changed, 62 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/sev.c +53 -2
@@ -1223,6 +1223,18 @@ void pre_sev_run(struct vcpu_svm *svm, int cpu)vmcb_mark_dirty (svm->vmcb,VMCB_ASID ); } +static void set_ghcb_msr_bits (struct vcpu_svm *svm,u64 value,u64 mask, +unsigned int pos) +{ + svm->vmcb->control.ghcb_gpa &= ~(mask << pos); + svm->vmcb->control.ghcb_gpa |= (value & mask) << pos; +} + +static u64 get_ghcb_msr_bits (struct vcpu_svm *svm,u64 mask,unsigned int pos) +{ +return (svm->vmcb->control.ghcb_gpa >> pos) & mask; +} +static void set_ghcb_msr (struct vcpu_svm *svm,u64 value) { svm->vmcb->control.ghcb_gpa = value;
@@ -1232,6 +1244,7 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) {struct vmcb_control_area *control = &svm->vmcb->control;u64 ghcb_info; +int ret =1 ; ghcb_info = control->ghcb_gpa &GHCB_MSR_INFO_MASK ;
@@ -1241,11 +1254,49 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)GHCB_VERSION_MIN , sev_enc_bit));break ; +case GHCB_MSR_CPUID_REQ : { +u64 cpuid_fn, cpuid_reg, cpuid_value; + + cpuid_fn =get_ghcb_msr_bits (svm, +GHCB_MSR_CPUID_FUNC_MASK , +GHCB_MSR_CPUID_FUNC_POS ); + +/* Initialize the registers needed by the CPUID intercept */ +svm_rax_write (svm, cpuid_fn); +svm_rcx_write (svm,0 ); + + ret =svm_invoke_exit_handler (svm,SVM_EXIT_CPUID ); +if (!ret) { + ret = -EINVAL ; +break ; + } + + cpuid_reg =get_ghcb_msr_bits (svm, +GHCB_MSR_CPUID_REG_MASK , +GHCB_MSR_CPUID_REG_POS ); +if (cpuid_reg ==0 ) + cpuid_value =svm_rax_read (svm); +else if (cpuid_reg ==1 ) + cpuid_value =svm_rbx_read (svm); +else if (cpuid_reg ==2 ) + cpuid_value =svm_rcx_read (svm); +else + cpuid_value =svm_rdx_read (svm); + +set_ghcb_msr_bits (svm, cpuid_value, +GHCB_MSR_CPUID_VALUE_MASK , +GHCB_MSR_CPUID_VALUE_POS ); + +set_ghcb_msr_bits (svm,GHCB_MSR_CPUID_RESP , +GHCB_MSR_INFO_MASK , +GHCB_MSR_INFO_POS ); +break ; + }default : -return -EINVAL ; + ret = -EINVAL ; } -return 1 ; +return ret; }int sev_handle_vmgexit (struct vcpu_svm *svm)
arch/x86/kvm/svm/svm.h +9 -0
@@ -526,6 +526,15 @@ void svm_vcpu_unblocking(struct kvm_vcpu *vcpu); (((_cbit) & GHCB_MSR_CBIT_MASK) << GHCB_MSR_CBIT_POS) | \ GHCB_MSR_SEV_INFO_RESP) +#define GHCB_MSR_CPUID_REQ 0x004 +#define GHCB_MSR_CPUID_RESP 0x005 +#define GHCB_MSR_CPUID_FUNC_POS 32 +#define GHCB_MSR_CPUID_FUNC_MASK 0xffffffff +#define GHCB_MSR_CPUID_VALUE_POS 32 +#define GHCB_MSR_CPUID_VALUE_MASK 0xffffffff +#define GHCB_MSR_CPUID_REG_POS 30 +#define GHCB_MSR_CPUID_REG_MASK 0x3 +extern unsigned int max_sev_asid;static inline bool svm_sev_enabled (void )
Signature · 1 line
2.28.0
RFC PATCH 16/35 KVM: SVM: Add support for SEV-ES GHCB MSR protocol function 0x100
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The GHCB defines a GHCB MSR protocol using the lower 12-bits of the GHCB MSR (in the hypervisor this corresponds to the GHCB GPA field in the VMCB).
Function 0x100 is a request for termination of the guest. The guest has encountered some situation for which it has requested to be terminated. The GHCB MSR value contains the reason for the request.
--- arch/x86/kvm/svm/sev.c | 13 +++++++++++++ arch/x86/kvm/svm/svm.h | 6 ++++++ 2 files changed, 19 insertions(+)
arch/x86/kvm/svm/sev.c +13 -0
@@ -1292,6 +1292,19 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) GHCB_MSR_INFO_POS);break ; } +case GHCB_MSR_TERM_REQ : { +u64 reason_set, reason_code; + + reason_set =get_ghcb_msr_bits (svm, +GHCB_MSR_TERM_REASON_SET_MASK , +GHCB_MSR_TERM_REASON_SET_POS ); + reason_code =get_ghcb_msr_bits (svm, +GHCB_MSR_TERM_REASON_MASK , +GHCB_MSR_TERM_REASON_POS ); +pr_info ("SEV-ES guest requested termination: %#llx:%#llx\n" , + reason_set, reason_code); + fallthrough; + }default : ret = -EINVAL ; }
arch/x86/kvm/svm/svm.h +6 -0
@@ -535,6 +535,12 @@ void svm_vcpu_unblocking(struct kvm_vcpu *vcpu);#define GHCB_MSR_CPUID_REG_POS 30#define GHCB_MSR_CPUID_REG_MASK 0x3 +#define GHCB_MSR_TERM_REQ 0x100 +#define GHCB_MSR_TERM_REASON_SET_POS 12 +#define GHCB_MSR_TERM_REASON_SET_MASK 0xf +#define GHCB_MSR_TERM_REASON_POS 16 +#define GHCB_MSR_TERM_REASON_MASK 0xff +extern unsigned int max_sev_asid;static inline bool svm_sev_enabled (void )
Signature · 1 line
2.28.0
RFC PATCH 17/35 KVM: SVM: Create trace events for VMGEXIT MSR protocol processing
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Add trace events for entry to and exit from VMGEXIT MSR protocol processing. The vCPU will be common for the trace events. The MSR protocol processing is guided by the GHCB GPA in the VMCB, so the GHCB GPA will represent the input and output values for the entry and exit events, respectively. Additionally, the exit event will contain the return code for the event.
--- arch/x86/kvm/svm/sev.c | 6 ++++++ arch/x86/kvm/trace.h | 44 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 2 ++ 3 files changed, 52 insertions(+)
arch/x86/kvm/svm/sev.c +6 -0
@@ -1248,6 +1248,9 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) ghcb_info = control->ghcb_gpa &GHCB_MSR_INFO_MASK ; +trace_kvm_vmgexit_msr_protocol_enter (svm->vcpu.vcpu_id, + control->ghcb_gpa); +switch (ghcb_info) {case GHCB_MSR_SEV_INFO_REQ : set_ghcb_msr(svm, GHCB_MSR_SEV_INFO(GHCB_VERSION_MAX,
@@ -1309,6 +1312,9 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm) ret = -EINVAL ; } +trace_kvm_vmgexit_msr_protocol_exit (svm->vcpu.vcpu_id, + control->ghcb_gpa, ret); +return ret; }
arch/x86/kvm/trace.h +44 -0
50 lines not on this page · raw message
arch/x86/kvm/x86.c +2 -0
@@ -10792,3 +10792,5 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_avic_ga_log);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_apicv_update_request);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_enter);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_exit); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_msr_protocol_enter); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_msr_protocol_exit);
Signature · 1 line
2.28.0
RFC PATCH 18/35 KVM: SVM: Support MMIO for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
For an SEV-ES guest, MMIO is performed to a shared (un-encrypted) page so that both the hypervisor and guest can read or write to it and each see the contents.
The GHCB specification provides software-defined VMGEXIT exit codes to indicate a request for an MMIO read or an MMIO write. Add support to recognize the MMIO requests and invoke SEV-ES specific routines that can complete the MMIO operation. These routines use common KVM support to complete the MMIO operation.
--- arch/x86/kvm/svm/sev.c | 116 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 3 + arch/x86/kvm/svm/svm.h | 6 ++ arch/x86/kvm/x86.c | 123 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.h | 5 ++ 5 files changed, 253 insertions(+)
arch/x86/kvm/svm/sev.c +116 -0
@@ -1191,6 +1191,24 @@ static void pre_sev_es_run(struct vcpu_svm *svm)if (!svm->ghcb)return ; +if (svm->ghcb_sa_free) { +/* +* The scratch area lives outside the GHCB, so there is a +* buffer that, depending on the operation performed, may +* need to be synced, then freed. +*/ +if (svm->ghcb_sa_sync) { +kvm_write_guest (svm->vcpu.kvm, +ghcb_get_sw_scratch (svm->ghcb), + svm->ghcb_sa, svm->ghcb_sa_len); + svm->ghcb_sa_sync = false; + } + +kfree (svm->ghcb_sa); + svm->ghcb_sa =NULL ; + svm->ghcb_sa_free = false; + } +trace_kvm_vmgexit_exit (svm->vcpu.vcpu_id, svm->ghcb);kvm_vcpu_unmap (&svm->vcpu, &svm->ghcb_map, true);
@@ -1223,6 +1241,86 @@ void pre_sev_run(struct vcpu_svm *svm, int cpu)vmcb_mark_dirty (svm->vmcb,VMCB_ASID ); } +#define GHCB_SCRATCH_AREA_LIMIT (16ULL * PAGE_SIZE) +static bool setup_vmgexit_scratch (struct vcpu_svm *svm,bool sync,u64 len) +{ +struct vmcb_control_area *control = &svm->vmcb->control; +struct ghcb *ghcb = svm->ghcb; +u64 ghcb_scratch_beg, ghcb_scratch_end; +u64 scratch_gpa_beg, scratch_gpa_end; +void *scratch_va; + + scratch_gpa_beg =ghcb_get_sw_scratch (ghcb); +if (!scratch_gpa_beg) { +pr_err ("vmgexit: scratch gpa not provided\n" ); +return false; + } + + scratch_gpa_end = scratch_gpa_beg + len; +if (scratch_gpa_end < scratch_gpa_beg) { +pr_err ("vmgexit: scratch length (%#llx) not valid for scratch address (%#llx)\n" , + len, scratch_gpa_beg); +return false; + } + +if ((scratch_gpa_beg &PAGE_MASK ) == control->ghcb_gpa) { +/* Scratch area begins within GHCB */ + ghcb_scratch_beg = control->ghcb_gpa + + offsetof(struct ghcb , shared_buffer); + ghcb_scratch_end = control->ghcb_gpa + + offsetof(struct ghcb , reserved_1); + +/* +* If the scratch area begins within the GHCB, it must be +* completely contained in the GHCB shared buffer area. +*/ +if (scratch_gpa_beg < ghcb_scratch_beg || + scratch_gpa_end > ghcb_scratch_end) { +pr_err ("vmgexit: scratch area is outside of GHCB shared buffer area (%#llx - %#llx)\n" , + scratch_gpa_beg, scratch_gpa_end); +return false; + } + + scratch_va = (void *)svm->ghcb; + scratch_va += (scratch_gpa_beg - control->ghcb_gpa); + }else { +/* +* The guest memory must be read into a kernel buffer, so +* limit the size +*/ +if (len >GHCB_SCRATCH_AREA_LIMIT ) { +pr_err ("vmgexit: scratch area exceeds KVM limits (%#llx requested, %#llx limit)\n" , + len,GHCB_SCRATCH_AREA_LIMIT ); +return false; + } + scratch_va =kzalloc (len,GFP_KERNEL ); +if (!scratch_va) +return false; + +if (kvm_read_guest (svm->vcpu.kvm, scratch_gpa_beg, scratch_va, len)) { +/* Unable to copy scratch area from guest */ +pr_err ("vmgexit: kvm_read_guest for scratch area failed\n" ); + +kfree (scratch_va); +return false; + } + +/* +* The scratch area is outside the GHCB. The operation will +* dictate whether the buffer needs to be synced before running +* the vCPU next time (i.e. a read was requested so the data +* must be written back to the guest memory). +*/ + svm->ghcb_sa_sync = sync; + svm->ghcb_sa_free = true; + } + + svm->ghcb_sa = scratch_va; + svm->ghcb_sa_len = len; + +return true; +} +static void set_ghcb_msr_bits (struct vcpu_svm *svm,u64 value,u64 mask,unsigned int pos) {
@@ -1356,6 +1454,24 @@ int sev_handle_vmgexit(struct vcpu_svm *svm) ret = -EINVAL ;switch (ghcb_get_sw_exit_code (ghcb)) { +case SVM_VMGEXIT_MMIO_READ : +if (!setup_vmgexit_scratch (svm, true, control->exit_info_2)) +break ; + + ret =kvm_sev_es_mmio_read (&svm->vcpu, + control->exit_info_1, + control->exit_info_2, + svm->ghcb_sa); +break ; +case SVM_VMGEXIT_MMIO_WRITE : +if (!setup_vmgexit_scratch (svm, false, control->exit_info_2)) +break ; + + ret =kvm_sev_es_mmio_write (&svm->vcpu, + control->exit_info_1, + control->exit_info_2, + svm->ghcb_sa); +break ;case SVM_VMGEXIT_UNSUPPORTED_EVENT : pr_err("vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n", control->exit_info_1,
arch/x86/kvm/svm/svm.c +3 -0
@@ -1306,6 +1306,9 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu) }__free_page (virt_to_page (svm->vmsa)); + +if (svm->ghcb_sa_free) +kfree (svm->ghcb_sa); }__free_page (pfn_to_page (__sme_clr (svm->vmcb_pa) >>PAGE_SHIFT ));
arch/x86/kvm/svm/svm.h +6 -0
@@ -165,6 +165,12 @@ struct vcpu_svm {struct vmcb_save_area *vmsa;struct ghcb *ghcb;struct kvm_host_map ghcb_map; + +/* SEV-ES scratch area support */ +void *ghcb_sa; +u64 ghcb_sa_len; +bool ghcb_sa_sync; +bool ghcb_sa_free; };struct svm_cpu_data {
arch/x86/kvm/x86.c +123 -0
129 lines not on this page · raw message
arch/x86/kvm/x86.h +5 -0
@@ -398,4 +398,9 @@ bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu); __reserved_bits; \ }) +int kvm_sev_es_mmio_write (struct kvm_vcpu *vcpu,gpa_t src,unsigned int bytes, +void *dst); +int kvm_sev_es_mmio_read (struct kvm_vcpu *vcpu,gpa_t src,unsigned int bytes, +void *dst); +#endif
Signature · 1 line
2.28.0
RFC PATCH 19/35 KVM: SVM: Support port IO operations for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
For an SEV-ES guest, port IO is performed to a shared (un-encrypted) page so that both the hypervisor and guest can read or write to it and each see the contents.
For port IO operations, invoke SEV-ES specific routines that can complete the operation using common KVM port IO support.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/sev.c | 9 ++++++ arch/x86/kvm/svm/svm.c | 11 +++++-- arch/x86/kvm/svm/svm.h | 1 + arch/x86/kvm/x86.c | 51 +++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.h | 3 ++ 6 files changed, 73 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -613,6 +613,7 @@ struct kvm_vcpu_arch {struct kvm_pio_request pio;void *pio_data; +void *guest_ins_data;u8 event_exit_inst_len;
arch/x86/kvm/svm/sev.c +9 -0
@@ -1483,3 +1483,12 @@ int sev_handle_vmgexit(struct vcpu_svm *svm)return ret; } + +int sev_es_string_io (struct vcpu_svm *svm,int size,unsigned int port,int in) +{ +if (!setup_vmgexit_scratch (svm, in, svm->vmcb->control.exit_info_2)) +return -EINVAL ; + +return kvm_sev_es_string_io (&svm->vcpu, size, port, + svm->ghcb_sa, svm->ghcb_sa_len, in); +}
arch/x86/kvm/svm/svm.c +8 -3
@@ -1984,11 +1984,16 @@ static int io_interception(struct vcpu_svm *svm) ++svm->vcpu.stat.io_exits; string = (io_info &SVM_IOIO_STR_MASK ) !=0 ; in = (io_info &SVM_IOIO_TYPE_MASK ) !=0 ; -if (string) -return kvm_emulate_instruction (vcpu,0 ); - port = io_info >>16 ; size = (io_info &SVM_IOIO_SIZE_MASK ) >>SVM_IOIO_SIZE_SHIFT ; + +if (string) { +if (sev_es_guest (vcpu->kvm)) +return sev_es_string_io (svm, size, port, in); +else +return kvm_emulate_instruction (vcpu,0 ); + } + svm->next_rip = svm->vmcb->control.exit_info_2;return kvm_fast_pio (&svm->vcpu, size, port, in);
arch/x86/kvm/svm/svm.h +1 -0
@@ -564,6 +564,7 @@ void pre_sev_run(struct vcpu_svm *svm, int cpu);void __init sev_hardware_setup (void );void sev_hardware_teardown (void );int sev_handle_vmgexit (struct vcpu_svm *svm); +int sev_es_string_io (struct vcpu_svm *svm,int size,unsigned int port,int in);/* VMSA Accessor functions */
arch/x86/kvm/x86.c +51 -0
63 lines not on this page · raw message
arch/x86/kvm/x86.h +3 -0
@@ -402,5 +402,8 @@ int kvm_sev_es_mmio_write(struct kvm_vcpu *vcpu, gpa_t src, unsigned int bytes, void *dst);int kvm_sev_es_mmio_read (struct kvm_vcpu *vcpu,gpa_t src,unsigned int bytes,void *dst); +int kvm_sev_es_string_io (struct kvm_vcpu *vcpu,unsigned int size, +unsigned int port,void *data,unsigned int count, +int in);#endif
Signature · 1 line
2.28.0
RFC PATCH 20/35 KVM: SVM: Add SEV/SEV-ES support for intercepting INVD
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The INVD instruction intercept performs emulation. Emulation can't be done on an SEV or SEV-ES guest because the guest memory is encrypted.
Provide a specific intercept routine for the INVD intercept. Within this intercept routine, skip the instruction for an SEV or SEV-ES guest since it is emulated as a NOP anyway.
--- arch/x86/kvm/svm/svm.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +12 -1
@@ -2275,6 +2275,17 @@ static int iret_interception(struct vcpu_svm *svm)return 1 ; } +static int invd_interception (struct vcpu_svm *svm) +{ +/* +* Can't do emulation on any type of SEV guest and INVD is emulated +* as a NOP, so just skip it. +*/ +return (sev_guest (svm->vcpu.kvm)) + ?kvm_skip_emulated_instruction (&svm->vcpu) + :kvm_emulate_instruction (&svm->vcpu,0 ); +} +static int invlpg_interception (struct vcpu_svm *svm) {if (!static_cpu_has (X86_FEATURE_DECODEASSISTS ))
@@ -2912,7 +2923,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_RDPMC] = rdpmc_interception, [SVM_EXIT_CPUID] = cpuid_interception, [SVM_EXIT_IRET] = iret_interception, - [SVM_EXIT_INVD] = emulate_on_interception, + [SVM_EXIT_INVD] = invd_interception, [SVM_EXIT_PAUSE] = pause_interception, [SVM_EXIT_HLT] = halt_interception, [SVM_EXIT_INVLPG] = invlpg_interception,
Signature · 1 line
2.28.0
Re: [RFC PATCH 20/35] KVM: SVM: Add SEV/SEV-ES support for intercepting INVD
Quoted 19 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
The INVD instruction intercept performs emulation. Emulation can't be done on an SEV or SEV-ES guest because the guest memory is encrypted.
Provide a specific intercept routine for the INVD intercept. Within this intercept routine, skip the instruction for an SEV or SEV-ES guest since it is emulated as a NOP anyway.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/svm/svm.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)arch/x86/kvm/svm/svm.c +0
-0
arch/x86/kvm/svm/svm.c static int iret_interception(struct vcpu_svm *svm) · line 2284
Quoted 10 lines
arch/x86/kvm/svm/svm.c +7
-0@@ -2275,6 +2275,17 @@ static int iret_interception(struct vcpu_svm *svm)return 1 ; } +static int invd_interception (struct vcpu_svm *svm) +{ +/* +* Can't do emulation on any type of SEV guest and INVD is emulated +* as a NOP, so just skip it. +*/ +return (sev_guest (svm->vcpu.kvm))
Should this be a standalone/backported fix for SEV?
Quoted 19 lines
arch/x86/kvm/svm/svm.c +5
-1+ ?kvm_skip_emulated_instruction (&svm->vcpu) + :kvm_emulate_instruction (&svm->vcpu,0 ); +} +static int invlpg_interception (struct vcpu_svm *svm) {if (!static_cpu_has (X86_FEATURE_DECODEASSISTS ))@@ -2912,7 +2923,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_RDPMC] = rdpmc_interception, [SVM_EXIT_CPUID] = cpuid_interception, [SVM_EXIT_IRET] = iret_interception, - [SVM_EXIT_INVD] = emulate_on_interception, + [SVM_EXIT_INVD] = invd_interception, [SVM_EXIT_PAUSE] = pause_interception, [SVM_EXIT_HLT] = halt_interception, [SVM_EXIT_INVLPG] = invlpg_interception,Signature · 1 line
2.28.0
Re: [RFC PATCH 20/35] KVM: SVM: Add SEV/SEV-ES support for intercepting INVD
Quoted 23 lines · Sean Christopherson
Quoted 22 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
The INVD instruction intercept performs emulation. Emulation can't be done on an SEV or SEV-ES guest because the guest memory is encrypted.
Provide a specific intercept routine for the INVD intercept. Within this intercept routine, skip the instruction for an SEV or SEV-ES guest since it is emulated as a NOP anyway.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/svm/svm.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)arch/x86/kvm/svm/svm.c +0
-0@@ -2275,6 +2275,17 @@ static int iret_interception(struct vcpu_svm *svm)return 1 ; }
Quoted 9 lines
arch/x86/kvm/svm/svm.c static int iret_interception(struct vcpu_svm *svm) · line 2284
Quoted 7 lines
arch/x86/kvm/svm/svm.c +7
-0+static int invd_interception (struct vcpu_svm *svm) +{ +/* +* Can't do emulation on any type of SEV guest and INVD is emulated +* as a NOP, so just skip it. +*/ +return (sev_guest (svm->vcpu.kvm))Should this be a standalone/backported fix for SEV?
Yes. Let me split it out and send it separately.
Thanks,
Tom
Quoted 20 lines
Quoted 20 lines
arch/x86/kvm/svm/svm.c +5
-1+ ?kvm_skip_emulated_instruction (&svm->vcpu) + :kvm_emulate_instruction (&svm->vcpu,0 ); +} +static int invlpg_interception (struct vcpu_svm *svm) {if (!static_cpu_has (X86_FEATURE_DECODEASSISTS ))@@ -2912,7 +2923,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_RDPMC] = rdpmc_interception, [SVM_EXIT_CPUID] = cpuid_interception, [SVM_EXIT_IRET] = iret_interception, - [SVM_EXIT_INVD] = emulate_on_interception, + [SVM_EXIT_INVD] = invd_interception, [SVM_EXIT_PAUSE] = pause_interception, [SVM_EXIT_HLT] = halt_interception, [SVM_EXIT_INVLPG] = invlpg_interception,Signature · 1 line
2.28.0
RFC PATCH 21/35 KVM: SVM: Add support for EFER write traps for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
For SEV-ES guests, the interception of EFER write access is not recommended. EFER interception occurs prior to EFER being modified and the hypervisor is unable to modify EFER itself because the register is located in the encrypted register state.
SEV-ES guests introduce a new EFER write trap. This trap provides intercept support of an EFER write after it has been modified. The new EFER value is provided in the VMCB EXITINFO1 field, allowing the hypervisor to track the setting of the guest EFER.
Add support to track the value of the guest EFER value using the EFER write trap so that the hypervisor understands the guest operating mode.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 ++ arch/x86/kvm/svm/svm.c | 12 ++++++++++++ arch/x86/kvm/x86.c | 12 ++++++++++++ 4 files changed, 27 insertions(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1427,6 +1427,7 @@ void kvm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);int kvm_task_switch (struct kvm_vcpu *vcpu,u16 tss_selector,int idt_index,int reason,bool has_error_code,u32 error_code); +int kvm_track_efer (struct kvm_vcpu *vcpu,u64 efer);int kvm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0);int kvm_set_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3);int kvm_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4);
arch/x86/include/uapi/asm/svm.h +2 -0
@@ -77,6 +77,7 @@#define SVM_EXIT_MWAIT_COND 0x08c#define SVM_EXIT_XSETBV 0x08d#define SVM_EXIT_RDPRU 0x08e +#define SVM_EXIT_EFER_WRITE_TRAP 0x08f#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402
@@ -183,6 +184,7 @@ { SVM_EXIT_MONITOR, "monitor" }, \ { SVM_EXIT_MWAIT, "mwait" }, \ { SVM_EXIT_XSETBV, "xsetbv" }, \ + { SVM_EXIT_EFER_WRITE_TRAP, "write_efer_trap" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \
arch/x86/kvm/svm/svm.c +12 -0
@@ -2466,6 +2466,17 @@ static int cr8_write_interception(struct vcpu_svm *svm)return 0 ; } +static int efer_trap (struct vcpu_svm *svm) +{ +int ret; + + ret =kvm_track_efer (&svm->vcpu, svm->vmcb->control.exit_info_1); +if (ret) +return ret; + +return kvm_complete_insn_gp (&svm->vcpu,0 ); +} +static int svm_get_msr_feature (struct kvm_msr_entry *msr) { msr->data =0 ;
@@ -2944,6 +2955,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_MWAIT] = mwait_interception, [SVM_EXIT_XSETBV] = xsetbv_interception, [SVM_EXIT_RDPRU] = rdpru_interception, + [SVM_EXIT_EFER_WRITE_TRAP] = efer_trap, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,
arch/x86/kvm/x86.c +12 -0
@@ -1480,6 +1480,18 @@ static int set_efer(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return 0 ; } +int kvm_track_efer (struct kvm_vcpu *vcpu,u64 efer) +{ +struct msr_data msr_info; + + msr_info.host_initiated = false; + msr_info.index =MSR_EFER ; + msr_info.data = efer; + +return set_efer (vcpu, &msr_info); +} +EXPORT_SYMBOL_GPL (kvm_track_efer); +void kvm_enable_efer_bits (u64 mask) { efer_reserved_bits &= ~mask;
Signature · 1 line
2.28.0
A message that is not in the archive (sent off-list or not yet received): 20200914220800.GI7192@sjchrist-ice
Re: [RFC PATCH 21/35] KVM: SVM: Add support for EFER write traps for an SEV-ES guest
Quoted 64 lines · Sean Christopherson
Quoted 63 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
For SEV-ES guests, the interception of EFER write access is not recommended. EFER interception occurs prior to EFER being modified and the hypervisor is unable to modify EFER itself because the register is located in the encrypted register state.
SEV-ES guests introduce a new EFER write trap. This trap provides intercept support of an EFER write after it has been modified. The new EFER value is provided in the VMCB EXITINFO1 field, allowing the hypervisor to track the setting of the guest EFER.
Add support to track the value of the guest EFER value using the EFER write trap so that the hypervisor understands the guest operating mode.
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 ++ arch/x86/kvm/svm/svm.c | 12 ++++++++++++ arch/x86/kvm/x86.c | 12 ++++++++++++ 4 files changed, 27 insertions(+)arch/x86/include/asm/kvm_host.h +1
-0@@ -1427,6 +1427,7 @@ void kvm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);int kvm_task_switch (struct kvm_vcpu *vcpu,u16 tss_selector,int idt_index,int reason,bool has_error_code,u32 error_code); +int kvm_track_efer (struct kvm_vcpu *vcpu,u64 efer);int kvm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0);int kvm_set_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3);int kvm_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4);arch/x86/include/uapi/asm/svm.h +2
-0@@ -77,6 +77,7 @@#define SVM_EXIT_MWAIT_COND 0x08c#define SVM_EXIT_XSETBV 0x08d#define SVM_EXIT_RDPRU 0x08e +#define SVM_EXIT_EFER_WRITE_TRAP 0x08f#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402@@ -183,6 +184,7 @@ { SVM_EXIT_MONITOR, "monitor" }, \ { SVM_EXIT_MWAIT, "mwait" }, \ { SVM_EXIT_XSETBV, "xsetbv" }, \ + { SVM_EXIT_EFER_WRITE_TRAP, "write_efer_trap" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \arch/x86/kvm/svm/svm.c +0
-0@@ -2466,6 +2466,17 @@ static int cr8_write_interception(struct vcpu_svm *svm)return 0 ; }
Quoted 9 lines
arch/x86/kvm/svm/svm.c static int cr8_write_interception(struct vcpu_svm *svm) · line 2474
Quoted 6 lines
arch/x86/kvm/svm/svm.c +6
-0+static int efer_trap (struct vcpu_svm *svm) +{ +int ret; + + ret =kvm_track_efer (&svm->vcpu, svm->vmcb->control.exit_info_1); +if (ret)Shouldn't this be a WARN or something? E.g. KVM thinks the WRMSR has faulted, while it obviously hasn't, which means KVM's internal model is now out of sync.
Makes sense, I can add something here.
Quoted 25 lines
Quoted 25 lines
arch/x86/kvm/svm/svm.c +6
-0+return ret; + +return kvm_complete_insn_gp (&svm->vcpu,0 ); +} +static int svm_get_msr_feature (struct kvm_msr_entry *msr) { msr->data =0 ;@@ -2944,6 +2955,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_MWAIT] = mwait_interception, [SVM_EXIT_XSETBV] = xsetbv_interception, [SVM_EXIT_RDPRU] = rdpru_interception, + [SVM_EXIT_EFER_WRITE_TRAP] = efer_trap, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,arch/x86/kvm/x86.c +0
-0@@ -1480,6 +1480,18 @@ static int set_efer(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return 0 ; }
Quoted 14 lines
arch/x86/kvm/x86.c static int set_efer(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 1493
Quoted 11 lines
arch/x86/kvm/x86.c +11
-0+int kvm_track_efer (struct kvm_vcpu *vcpu,u64 efer) +{ +struct msr_data msr_info; + + msr_info.host_initiated = false; + msr_info.index =MSR_EFER ; + msr_info.data = efer; + +return set_efer (vcpu, &msr_info); +} +EXPORT_SYMBOL_GPL (kvm_track_efer);I don't see any reason to put this in
x86.c, just copy-paste the guts intoefer_trap()and s/set_efer/kvm_set_msr_common.
Ok, I can do that. I'll add a comment to indicate that the result of doing that is that set_efer() is ultimately invoked through that path.
Thanks,
Tom
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/x86.c +1
-0+void kvm_enable_efer_bits (u64 mask) { efer_reserved_bits &= ~mask;Signature · 1 line
2.28.0
RFC PATCH 22/35 KVM: SVM: Add support for CR0 write traps for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
For SEV-ES guests, the interception of control register write access is not recommended. Control register interception occurs prior to the control register being modified and the hypervisor is unable to modify the control register itself because the register is located in the encrypted register state.
SEV-ES guests introduce new control register write traps. These traps provide intercept support of a control register write after the control register has been modified. The new control register value is provided in the VMCB EXITINFO1 field, allowing the hypervisor to track the setting of the guest control registers.
Add support to track the value of the guest CR0 register using the control register write trap so that the hypervisor understands the guest operating mode.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/svm.h | 17 +++++++++++++ arch/x86/kvm/svm/svm.c | 20 +++++++++++++++ arch/x86/kvm/x86.c | 43 ++++++++++++++++++++++++--------- 4 files changed, 69 insertions(+), 12 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1432,6 +1432,7 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0);int kvm_set_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3);int kvm_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4);int kvm_set_cr8 (struct kvm_vcpu *vcpu,unsigned long cr8); +int kvm_track_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0);int kvm_set_dr (struct kvm_vcpu *vcpu,int dr,unsigned long val);int kvm_get_dr (struct kvm_vcpu *vcpu,int dr,unsigned long *val);unsigned long kvm_get_cr8 (struct kvm_vcpu *vcpu);
arch/x86/include/uapi/asm/svm.h +17 -0
@@ -78,6 +78,22 @@#define SVM_EXIT_XSETBV 0x08d#define SVM_EXIT_RDPRU 0x08e#define SVM_EXIT_EFER_WRITE_TRAP 0x08f +#define SVM_EXIT_CR0_WRITE_TRAP 0x090 +#define SVM_EXIT_CR1_WRITE_TRAP 0x091 +#define SVM_EXIT_CR2_WRITE_TRAP 0x092 +#define SVM_EXIT_CR3_WRITE_TRAP 0x093 +#define SVM_EXIT_CR4_WRITE_TRAP 0x094 +#define SVM_EXIT_CR5_WRITE_TRAP 0x095 +#define SVM_EXIT_CR6_WRITE_TRAP 0x096 +#define SVM_EXIT_CR7_WRITE_TRAP 0x097 +#define SVM_EXIT_CR8_WRITE_TRAP 0x098 +#define SVM_EXIT_CR9_WRITE_TRAP 0x099 +#define SVM_EXIT_CR10_WRITE_TRAP 0x09a +#define SVM_EXIT_CR11_WRITE_TRAP 0x09b +#define SVM_EXIT_CR12_WRITE_TRAP 0x09c +#define SVM_EXIT_CR13_WRITE_TRAP 0x09d +#define SVM_EXIT_CR14_WRITE_TRAP 0x09e +#define SVM_EXIT_CR15_WRITE_TRAP 0x09f#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402
@@ -185,6 +201,7 @@ { SVM_EXIT_MWAIT, "mwait" }, \ { SVM_EXIT_XSETBV, "xsetbv" }, \ { SVM_EXIT_EFER_WRITE_TRAP, "write_efer_trap" }, \ + { SVM_EXIT_CR0_WRITE_TRAP, "write_cr0_trap" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \
arch/x86/kvm/svm/svm.c +20 -0
@@ -2413,6 +2413,25 @@ static int cr_interception(struct vcpu_svm *svm)return kvm_complete_insn_gp (&svm->vcpu, err); } +static int cr_trap (struct vcpu_svm *svm) +{ +unsigned int cr; + + cr = svm->vmcb->control.exit_code -SVM_EXIT_CR0_WRITE_TRAP ; + +switch (cr) { +case 0 : +kvm_track_cr0 (&svm->vcpu, svm->vmcb->control.exit_info_1); +break ; +default : +WARN (1 ,"unhandled CR%d write trap" , cr); +kvm_queue_exception (&svm->vcpu,UD_VECTOR ); +return 1 ; + } + +return kvm_complete_insn_gp (&svm->vcpu,0 ); +} +static int dr_interception (struct vcpu_svm *svm) {int reg, dr;
@@ -2956,6 +2975,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_XSETBV] = xsetbv_interception, [SVM_EXIT_RDPRU] = rdpru_interception, [SVM_EXIT_EFER_WRITE_TRAP] = efer_trap, + [SVM_EXIT_CR0_WRITE_TRAP] = cr_trap, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,
arch/x86/kvm/x86.c +31 -12
64 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [RFC PATCH 22/35] KVM: SVM: Add support for CR0 write traps for an SEV-ES guest
arch/x86/kvm/x86.c bool pdptrs_changed(struct kvm_vcpu *vcpu) · line 803
Quoted 11 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
arch/x86/kvm/x86.c +2
-0@@ -799,11 +799,29 @@ bool pdptrs_changed(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (pdptrs_changed); +static void kvm_post_set_cr0 (struct kvm_vcpu *vcpu,unsigned long old_cr0, +unsigned long cr0)
What about using __kvm_set_cr*() instead of kvm_post_set_cr*()? That would show that __kvm_set_cr*() is a subordinate of kvm_set_cr*(), and from the SVM side would provide the hint that the code is skipping the front end of kvm_set_cr*().
Quoted 36 lines
arch/x86/kvm/x86.c +18
-6+{ +unsigned long update_bits =X86_CR0_PG |X86_CR0_WP ; + +if ((cr0 ^ old_cr0) &X86_CR0_PG ) { +kvm_clear_async_pf_completion_queue (vcpu); +kvm_async_pf_hash_reset (vcpu); + } + +if ((cr0 ^ old_cr0) & update_bits) +kvm_mmu_reset_context (vcpu); + +if (((cr0 ^ old_cr0) &X86_CR0_CD ) && +kvm_arch_has_noncoherent_dma (vcpu->kvm) && + !kvm_check_has_quirk (vcpu->kvm,KVM_X86_QUIRK_CD_NW_CLEARED )) +kvm_zap_gfn_range (vcpu->kvm,0 , ~0ULL ); +} +int kvm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0) {unsigned long old_cr0 =kvm_read_cr0 (vcpu);unsigned long pdptr_bits =X86_CR0_CD |X86_CR0_NW |X86_CR0_PG ; -unsigned long update_bits =X86_CR0_PG |X86_CR0_WP ; cr0 |=X86_CR0_ET ;@@ -842,22 +860,23 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) kvm_x86_ops.set_cr0 (vcpu, cr0); -if ((cr0 ^ old_cr0) &X86_CR0_PG ) { -kvm_clear_async_pf_completion_queue (vcpu); -kvm_async_pf_hash_reset (vcpu); - } +kvm_post_set_cr0 (vcpu, old_cr0, cr0); -if ((cr0 ^ old_cr0) & update_bits)
arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 869
Quoted 10 lines
arch/x86/kvm/x86.c +4
-5-kvm_mmu_reset_context (vcpu); +return 0 ; +} +EXPORT_SYMBOL_GPL (kvm_set_cr0); -if (((cr0 ^ old_cr0) &X86_CR0_CD ) && -kvm_arch_has_noncoherent_dma (vcpu->kvm) && - !kvm_check_has_quirk (vcpu->kvm,KVM_X86_QUIRK_CD_NW_CLEARED )) -kvm_zap_gfn_range (vcpu->kvm,0 , ~0ULL ); +int kvm_track_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0)
I really dislike the "track" terminology. For me, using "track" as the verb in a function implies the function activates tracking. But it's probably a moot point, because similar to EFER, I don't see any reason to put the front end of the emulation into x86.c. Both getting old_cr0 and setting vcpu->arch.cr0 can be done in svm.c
Quoted 17 lines
arch/x86/kvm/x86.c +7
-1+{ +unsigned long old_cr0 =kvm_read_cr0 (vcpu); + + vcpu->arch.cr0 = cr0; + +kvm_post_set_cr0 (vcpu, old_cr0, cr0);return 0 ; } -EXPORT_SYMBOL_GPL (kvm_set_cr0); +EXPORT_SYMBOL_GPL (kvm_track_cr0);void kvm_lmsw (struct kvm_vcpu *vcpu,unsigned long msw) {Signature · 1 line
2.28.0
Re: [RFC PATCH 22/35] KVM: SVM: Add support for CR0 write traps for an SEV-ES guest
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
arch/x86/kvm/x86.c +0
-0@@ -799,11 +799,29 @@ bool pdptrs_changed(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (pdptrs_changed);
Quoted 7 lines
arch/x86/kvm/x86.c bool pdptrs_changed(struct kvm_vcpu *vcpu) · line 803
Quoted 2 lines
arch/x86/kvm/x86.c +2
-0+static void kvm_post_set_cr0 (struct kvm_vcpu *vcpu,unsigned long old_cr0, +unsigned long cr0)What about using __kvm_set_cr*() instead of kvm_post_set_cr*()? That would show that __kvm_set_cr*() is a subordinate of kvm_set_cr*(), and from the SVM side would provide the hint that the code is skipping the front end of kvm_set_cr*().
Ok, I'll change this (and the others) to __kvm_set_cr* and export them.
Quoted 42 lines
Quoted 42 lines
arch/x86/kvm/x86.c +21
-7+{ +unsigned long update_bits =X86_CR0_PG |X86_CR0_WP ; + +if ((cr0 ^ old_cr0) &X86_CR0_PG ) { +kvm_clear_async_pf_completion_queue (vcpu); +kvm_async_pf_hash_reset (vcpu); + } + +if ((cr0 ^ old_cr0) & update_bits) +kvm_mmu_reset_context (vcpu); + +if (((cr0 ^ old_cr0) &X86_CR0_CD ) && +kvm_arch_has_noncoherent_dma (vcpu->kvm) && + !kvm_check_has_quirk (vcpu->kvm,KVM_X86_QUIRK_CD_NW_CLEARED )) +kvm_zap_gfn_range (vcpu->kvm,0 , ~0ULL ); +} +int kvm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0) {unsigned long old_cr0 =kvm_read_cr0 (vcpu);unsigned long pdptr_bits =X86_CR0_CD |X86_CR0_NW |X86_CR0_PG ; -unsigned long update_bits =X86_CR0_PG |X86_CR0_WP ; cr0 |=X86_CR0_ET ;@@ -842,22 +860,23 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) kvm_x86_ops.set_cr0 (vcpu, cr0); -if ((cr0 ^ old_cr0) &X86_CR0_PG ) { -kvm_clear_async_pf_completion_queue (vcpu); -kvm_async_pf_hash_reset (vcpu); - } +kvm_post_set_cr0 (vcpu, old_cr0, cr0); -if ((cr0 ^ old_cr0) & update_bits) -kvm_mmu_reset_context (vcpu); +return 0 ; +} +EXPORT_SYMBOL_GPL (kvm_set_cr0);
Quoted 11 lines
arch/x86/kvm/x86.c int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 869
Quoted 5 lines
arch/x86/kvm/x86.c +1
-4-if (((cr0 ^ old_cr0) &X86_CR0_CD ) && -kvm_arch_has_noncoherent_dma (vcpu->kvm) && - !kvm_check_has_quirk (vcpu->kvm,KVM_X86_QUIRK_CD_NW_CLEARED )) -kvm_zap_gfn_range (vcpu->kvm,0 , ~0ULL ); +int kvm_track_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0)I really dislike the "track" terminology. For me, using "track" as the verb in a function implies the function activates tracking. But it's probably a moot point, because similar to EFER, I don't see any reason to put the front end of the emulation into
x86.c. Both gettingold_cr0and settingvcpu->arch.cr0can be done insvm.c
Yup, I can move that to svm.c.
Thanks,
Tom
Quoted 18 lines
Quoted 18 lines
arch/x86/kvm/x86.c +7
-1+{ +unsigned long old_cr0 =kvm_read_cr0 (vcpu); + + vcpu->arch.cr0 = cr0; + +kvm_post_set_cr0 (vcpu, old_cr0, cr0);return 0 ; } -EXPORT_SYMBOL_GPL (kvm_set_cr0); +EXPORT_SYMBOL_GPL (kvm_track_cr0);void kvm_lmsw (struct kvm_vcpu *vcpu,unsigned long msw) {Signature · 1 line
2.28.0
Paolo Bonzini
maintainerRe: [RFC PATCH 22/35] KVM: SVM: Add support for CR0 write traps for an SEV-ES guest
In reply to Sean ChristophersonQuoted 6 lines · Sean Christopherson
arch/x86/kvm/x86.c bool pdptrs_changed(struct kvm_vcpu *vcpu) · line 803
Quoted 2 lines
arch/x86/kvm/x86.c +2
-0+static void kvm_post_set_cr0 (struct kvm_vcpu *vcpu,unsigned long old_cr0, +unsigned long cr0)What about using __kvm_set_cr*() instead of kvm_post_set_cr*()? That would show that __kvm_set_cr*() is a subordinate of kvm_set_cr*(), and from the SVM side would provide the hint that the code is skipping the front end of kvm_set_cr*().
No, kvm_post_set_cr0 is exactly the right name because it doesn't set any state. __kvm_set_cr0 tells me that it is a (rarely used) way to set CR0, which this function isn't.
Sorry Tom for not catching this earlier.
Paolo
Quoted 34 lines
Quoted 34 lines
arch/x86/kvm/x86.c +18
-5+{ +unsigned long update_bits =X86_CR0_PG |X86_CR0_WP ; + +if ((cr0 ^ old_cr0) &X86_CR0_PG ) { +kvm_clear_async_pf_completion_queue (vcpu); +kvm_async_pf_hash_reset (vcpu); + } + +if ((cr0 ^ old_cr0) & update_bits) +kvm_mmu_reset_context (vcpu); + +if (((cr0 ^ old_cr0) &X86_CR0_CD ) && +kvm_arch_has_noncoherent_dma (vcpu->kvm) && + !kvm_check_has_quirk (vcpu->kvm,KVM_X86_QUIRK_CD_NW_CLEARED )) +kvm_zap_gfn_range (vcpu->kvm,0 , ~0ULL ); +} +int kvm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0) {unsigned long old_cr0 =kvm_read_cr0 (vcpu);unsigned long pdptr_bits =X86_CR0_CD |X86_CR0_NW |X86_CR0_PG ; -unsigned long update_bits =X86_CR0_PG |X86_CR0_WP ; cr0 |=X86_CR0_ET ;@@ -842,22 +860,23 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) kvm_x86_ops.set_cr0 (vcpu, cr0); -if ((cr0 ^ old_cr0) &X86_CR0_PG ) { -kvm_clear_async_pf_completion_queue (vcpu); -kvm_async_pf_hash_reset (vcpu); - } +kvm_post_set_cr0 (vcpu, old_cr0, cr0);
RFC PATCH 23/35 KVM: SVM: Add support for CR4 write traps for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
For SEV-ES guests, the interception of control register write access is not recommended. Control register interception occurs prior to the control register being modified and the hypervisor is unable to modify the control register itself because the register is located in the encrypted register state.
SEV-ES guests introduce new control register write traps. These traps provide intercept support of a control register write after the control register has been modified. The new control register value is provided in the VMCB EXITINFO1 field, allowing the hypervisor to track the setting of the guest control registers.
Add support to track the value of the guest CR4 register using the control register write trap so that the hypervisor understands the guest operating mode.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/svm.h | 1 + arch/x86/kvm/svm/svm.c | 4 ++++ arch/x86/kvm/x86.c | 20 ++++++++++++++++++++ 4 files changed, 26 insertions(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1433,6 +1433,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3);int kvm_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4);int kvm_set_cr8 (struct kvm_vcpu *vcpu,unsigned long cr8);int kvm_track_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0); +int kvm_track_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4);int kvm_set_dr (struct kvm_vcpu *vcpu,int dr,unsigned long val);int kvm_get_dr (struct kvm_vcpu *vcpu,int dr,unsigned long *val);unsigned long kvm_get_cr8 (struct kvm_vcpu *vcpu);
arch/x86/include/uapi/asm/svm.h +1 -0
@@ -202,6 +202,7 @@ { SVM_EXIT_XSETBV, "xsetbv" }, \ { SVM_EXIT_EFER_WRITE_TRAP, "write_efer_trap" }, \ { SVM_EXIT_CR0_WRITE_TRAP, "write_cr0_trap" }, \ + { SVM_EXIT_CR4_WRITE_TRAP, "write_cr4_trap" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \
arch/x86/kvm/svm/svm.c +4 -0
@@ -2423,6 +2423,9 @@ static int cr_trap(struct vcpu_svm *svm)case 0 :kvm_track_cr0 (&svm->vcpu, svm->vmcb->control.exit_info_1);break ; +case 4 : +kvm_track_cr4 (&svm->vcpu, svm->vmcb->control.exit_info_1); +break ;default :WARN (1 ,"unhandled CR%d write trap" , cr);kvm_queue_exception (&svm->vcpu,UD_VECTOR );
@@ -2976,6 +2979,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_RDPRU] = rdpru_interception, [SVM_EXIT_EFER_WRITE_TRAP] = efer_trap, [SVM_EXIT_CR0_WRITE_TRAP] = cr_trap, + [SVM_EXIT_CR4_WRITE_TRAP] = cr_trap, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,
arch/x86/kvm/x86.c +20 -0
@@ -1033,6 +1033,26 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) }EXPORT_SYMBOL_GPL (kvm_set_cr4); +int kvm_track_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) +{ +unsigned long old_cr4 =kvm_read_cr4 (vcpu); +unsigned long pdptr_bits =X86_CR4_PGE |X86_CR4_PSE |X86_CR4_PAE | +X86_CR4_SMEP |X86_CR4_SMAP |X86_CR4_PKE ; + +if (kvm_x86_ops.set_cr4 (vcpu, cr4)) +return 1 ; + +if (((cr4 ^ old_cr4) & pdptr_bits) || + (!(cr4 &X86_CR4_PCIDE ) && (old_cr4 &X86_CR4_PCIDE ))) +kvm_mmu_reset_context (vcpu); + +if ((cr4 ^ old_cr4) & (X86_CR4_OSXSAVE |X86_CR4_PKE )) +kvm_update_cpuid_runtime (vcpu); + +return 0 ; +} +EXPORT_SYMBOL_GPL (kvm_track_cr4); +int kvm_set_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3) {bool skip_tlb_flush = false;
Signature · 1 line
2.28.0
Re: [RFC PATCH 23/35] KVM: SVM: Add support for CR4 write traps for an SEV-ES guest
Quoted 6 lines · Tom Lendacky
arch/x86/kvm/x86.c +0
-0@@ -1033,6 +1033,26 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) }
arch/x86/kvm/x86.c int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) · line 1043
Quoted 10 lines
arch/x86/kvm/x86.c +8
-0EXPORT_SYMBOL_GPL (kvm_set_cr4); +int kvm_track_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) +{ +unsigned long old_cr4 =kvm_read_cr4 (vcpu); +unsigned long pdptr_bits =X86_CR4_PGE |X86_CR4_PSE |X86_CR4_PAE | +X86_CR4_SMEP |X86_CR4_SMAP |X86_CR4_PKE ; + +if (kvm_x86_ops.set_cr4 (vcpu, cr4)) +return 1 ;
Pretty much all the same comments as EFER and CR0, e.g. call svm_set_cr4() directly instead of bouncing through kvm_x86_ops. And with that, this can be called __kvm_set_cr4() to be consistent with __kvm_set_cr0().
Quoted 18 lines
arch/x86/kvm/x86.c +12
-0+ +if (((cr4 ^ old_cr4) & pdptr_bits) || + (!(cr4 &X86_CR4_PCIDE ) && (old_cr4 &X86_CR4_PCIDE ))) +kvm_mmu_reset_context (vcpu); + +if ((cr4 ^ old_cr4) & (X86_CR4_OSXSAVE |X86_CR4_PKE )) +kvm_update_cpuid_runtime (vcpu); + +return 0 ; +} +EXPORT_SYMBOL_GPL (kvm_track_cr4); +int kvm_set_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3) {bool skip_tlb_flush = false;Signature · 1 line
2.28.0
Paolo Bonzini
maintainerRe: [RFC PATCH 23/35] KVM: SVM: Add support for CR4 write traps for an SEV-ES guest
Quoted 11 lines · Sean Christopherson
arch/x86/kvm/x86.c int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) · line 1043
Quoted 8 lines
arch/x86/kvm/x86.c +8
-0+int kvm_track_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) +{ +unsigned long old_cr4 =kvm_read_cr4 (vcpu); +unsigned long pdptr_bits =X86_CR4_PGE |X86_CR4_PSE |X86_CR4_PAE | +X86_CR4_SMEP |X86_CR4_SMAP |X86_CR4_PKE ; + +if (kvm_x86_ops.set_cr4 (vcpu, cr4)) +return 1 ;Pretty much all the same comments as EFER and CR0, e.g. call
svm_set_cr4()directly instead of bouncing throughkvm_x86_ops. And with that, this can be called__kvm_set_cr4()to be consistent with__kvm_set_cr0().
I agree with calling svm_set_cr4 directly, but then this should be kvm_post_set_cr4.
Paolo
Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+ +if (((cr4 ^ old_cr4) & pdptr_bits) || + (!(cr4 &X86_CR4_PCIDE ) && (old_cr4 &X86_CR4_PCIDE ))) +kvm_mmu_reset_context (vcpu); + +if ((cr4 ^ old_cr4) & (X86_CR4_OSXSAVE |X86_CR4_PKE )) +kvm_update_cpuid_runtime (vcpu); + +return 0 ; +}
RFC PATCH 24/35 KVM: SVM: Add support for CR8 write traps for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
For SEV-ES guests, the interception of control register write access is not recommended. Control register interception occurs prior to the control register being modified and the hypervisor is unable to modify the control register itself because the register is located in the encrypted register state.
SEV-ES guests introduce new control register write traps. These traps provide intercept support of a control register write after the control register has been modified. The new control register value is provided in the VMCB EXITINFO1 field, allowing the hypervisor to track the setting of the guest control registers.
Add support to track the value of the guest CR8 register using the control register write trap so that the hypervisor understands the guest operating mode.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/svm.h | 1 + arch/x86/kvm/svm/svm.c | 4 ++++ arch/x86/kvm/x86.c | 6 ++++++ 4 files changed, 12 insertions(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1434,6 +1434,7 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4);int kvm_set_cr8 (struct kvm_vcpu *vcpu,unsigned long cr8);int kvm_track_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0);int kvm_track_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4); +int kvm_track_cr8 (struct kvm_vcpu *vcpu,unsigned long cr8);int kvm_set_dr (struct kvm_vcpu *vcpu,int dr,unsigned long val);int kvm_get_dr (struct kvm_vcpu *vcpu,int dr,unsigned long *val);unsigned long kvm_get_cr8 (struct kvm_vcpu *vcpu);
arch/x86/include/uapi/asm/svm.h +1 -0
@@ -203,6 +203,7 @@ { SVM_EXIT_EFER_WRITE_TRAP, "write_efer_trap" }, \ { SVM_EXIT_CR0_WRITE_TRAP, "write_cr0_trap" }, \ { SVM_EXIT_CR4_WRITE_TRAP, "write_cr4_trap" }, \ + { SVM_EXIT_CR8_WRITE_TRAP, "write_cr8_trap" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \
arch/x86/kvm/svm/svm.c +4 -0
@@ -2426,6 +2426,9 @@ static int cr_trap(struct vcpu_svm *svm)case 4 :kvm_track_cr4 (&svm->vcpu, svm->vmcb->control.exit_info_1);break ; +case 8 : +kvm_track_cr8 (&svm->vcpu, svm->vmcb->control.exit_info_1); +break ;default :WARN (1 ,"unhandled CR%d write trap" , cr);kvm_queue_exception (&svm->vcpu,UD_VECTOR );
@@ -2980,6 +2983,7 @@ static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = { [SVM_EXIT_EFER_WRITE_TRAP] = efer_trap, [SVM_EXIT_CR0_WRITE_TRAP] = cr_trap, [SVM_EXIT_CR4_WRITE_TRAP] = cr_trap, + [SVM_EXIT_CR8_WRITE_TRAP] = cr_trap, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,
arch/x86/kvm/x86.c +6 -0
@@ -1109,6 +1109,12 @@ unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (kvm_get_cr8); +int kvm_track_cr8 (struct kvm_vcpu *vcpu,unsigned long cr8) +{ +return kvm_set_cr8 (vcpu, cr8); +} +EXPORT_SYMBOL_GPL (kvm_track_cr8); +static void kvm_update_dr0123 (struct kvm_vcpu *vcpu) {int i;
Signature · 1 line
2.28.0
Re: [RFC PATCH 24/35] KVM: SVM: Add support for CR8 write traps for an SEV-ES guest
arch/x86/kvm/x86.c unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) · line 1114
Quoted 11 lines · Tom Lendacky
arch/x86/kvm/x86.c +3
-0@@ -1109,6 +1109,12 @@ unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (kvm_get_cr8); +int kvm_track_cr8 (struct kvm_vcpu *vcpu,unsigned long cr8) +{ +return kvm_set_cr8 (vcpu, cr8);
I'm guessing this was added to achieve consistency at the SVM call sites. With the previously suggested changes, kvm_track_cr8() can simply be dropped.
Quoted 9 lines
arch/x86/kvm/x86.c +3
-0+} +EXPORT_SYMBOL_GPL (kvm_track_cr8); +static void kvm_update_dr0123 (struct kvm_vcpu *vcpu) {int i;Signature · 1 line
2.28.0
Re: [RFC PATCH 24/35] KVM: SVM: Add support for CR8 write traps for an SEV-ES guest
Quoted 9 lines · Sean Christopherson
Quoted 8 lines · Tom Lendacky
arch/x86/kvm/x86.c +0
-0@@ -1109,6 +1109,12 @@ unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (kvm_get_cr8);
Quoted 7 lines
arch/x86/kvm/x86.c unsigned long kvm_get_cr8(struct kvm_vcpu *vcpu) · line 1114
Quoted 3 lines
arch/x86/kvm/x86.c +3
-0+int kvm_track_cr8 (struct kvm_vcpu *vcpu,unsigned long cr8) +{ +return kvm_set_cr8 (vcpu, cr8);I'm guessing this was added to achieve consistency at the SVM call sites. With the previously suggested changes,
kvm_track_cr8()can simply be dropped.
Yup.
Thanks,
Tom
Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/x86.c +3
-0+} +EXPORT_SYMBOL_GPL (kvm_track_cr8); +static void kvm_update_dr0123 (struct kvm_vcpu *vcpu) {int i;Signature · 1 line
2.28.0
RFC PATCH 25/35 KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Since many of the registers used by the SEV-ES are encrypted and cannot be read or written, adjust the __get_sregs() / __set_sregs() to only get or set the registers being tracked (efer, cr0, cr4 and cr8) once the VMSA is encrypted.
For __get_sregs(), return the actual value that is in use by the guest as determined by the write trap support of the registers.
For __set_sregs(), set the arch specific value that KVM believes the guest to be using. Note, this will not set the guest's actual value so it might only be useful for such things as live migration.
--- arch/x86/kvm/x86.c | 56 +++++++++++++++++++++++++++------------------- 1 file changed, 33 insertions(+), 23 deletions(-)
arch/x86/kvm/x86.c +33 -23
@@ -9090,6 +9090,9 @@ static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) {struct desc_ptr dt; +if (vcpu->arch.vmsa_encrypted) + goto tracking_regs; +kvm_get_segment (vcpu, &sregs->cs,VCPU_SREG_CS );kvm_get_segment (vcpu, &sregs->ds,VCPU_SREG_DS );kvm_get_segment (vcpu, &sregs->es,VCPU_SREG_ES );
@@ -9107,12 +9110,15 @@ static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) sregs->gdt.limit = dt.size; sregs->gdt.base = dt.address; - sregs->cr0 =kvm_read_cr0 (vcpu); sregs->cr2 = vcpu->arch.cr2; sregs->cr3 =kvm_read_cr3 (vcpu); + +tracking_regs: + sregs->cr0 =kvm_read_cr0 (vcpu); sregs->cr4 =kvm_read_cr4 (vcpu); sregs->cr8 =kvm_get_cr8 (vcpu); sregs->efer = vcpu->arch.efer; + sregs->apic_base =kvm_get_apic_base (vcpu);memset (sregs->interrupt_bitmap,0 ,sizeof (sregs->interrupt_bitmap));
@@ -9248,18 +9254,6 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs)if (kvm_set_apic_base (vcpu, &apic_base_msr)) goto out; - dt.size = sregs->idt.limit; - dt.address = sregs->idt.base; - kvm_x86_ops.set_idt (vcpu, &dt); - dt.size = sregs->gdt.limit; - dt.address = sregs->gdt.base; - kvm_x86_ops.set_gdt (vcpu, &dt); - - vcpu->arch.cr2 = sregs->cr2; - mmu_reset_needed |=kvm_read_cr3 (vcpu) != sregs->cr3; - vcpu->arch.cr3 = sregs->cr3; -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); -kvm_set_cr8 (vcpu, sregs->cr8); mmu_reset_needed |= vcpu->arch.efer != sregs->efer;
@@ -9276,6 +9270,14 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs)if (cpuid_update_needed)kvm_update_cpuid_runtime (vcpu); +if (vcpu->arch.vmsa_encrypted) + goto tracking_regs; + + vcpu->arch.cr2 = sregs->cr2; + mmu_reset_needed |=kvm_read_cr3 (vcpu) != sregs->cr3; + vcpu->arch.cr3 = sregs->cr3; +kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); + idx =srcu_read_lock (&vcpu->kvm->srcu);if (is_pae_paging (vcpu)) {load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu));
@@ -9283,16 +9285,12 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) }srcu_read_unlock (&vcpu->kvm->srcu, idx); -if (mmu_reset_needed) -kvm_mmu_reset_context (vcpu); - - max_bits =KVM_NR_INTERRUPTS ; - pending_vec =find_first_bit ( - (const unsigned long *)sregs->interrupt_bitmap, max_bits); -if (pending_vec < max_bits) { -kvm_queue_interrupt (vcpu, pending_vec, false); -pr_debug ("Set back pending irq %d\n" , pending_vec); - } + dt.size = sregs->idt.limit; + dt.address = sregs->idt.base; + kvm_x86_ops.set_idt (vcpu, &dt); + dt.size = sregs->gdt.limit; + dt.address = sregs->gdt.base; + kvm_x86_ops.set_gdt (vcpu, &dt);kvm_set_segment (vcpu, &sregs->cs,VCPU_SREG_CS );kvm_set_segment (vcpu, &sregs->ds,VCPU_SREG_DS );
@@ -9312,6 +9310,18 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) !is_protmode(vcpu)) vcpu->arch.mp_state =KVM_MP_STATE_RUNNABLE ; +tracking_regs: +if (mmu_reset_needed) +kvm_mmu_reset_context (vcpu); + + max_bits =KVM_NR_INTERRUPTS ; + pending_vec =find_first_bit ( + (const unsigned long *)sregs->interrupt_bitmap, max_bits); +if (pending_vec < max_bits) { +kvm_queue_interrupt (vcpu, pending_vec, false); +pr_debug ("Set back pending irq %d\n" , pending_vec); + } +kvm_make_request (KVM_REQ_EVENT , vcpu); ret =0 ;
Signature · 1 line
2.28.0
Re: [RFC PATCH 25/35] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
Quoted 6 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since many of the registers used by the SEV-ES are encrypted and cannot be read or written, adjust the
__get_sregs()/__set_sregs()to only get or set the registers being tracked (efer, cr0, cr4 and cr8) once the VMSA is encrypted.
Is there an actual use case for writing said registers after the VMSA is encrypted? Assuming there's a separate "debug mode" and live migration has special logic, can KVM simply reject the ioctl() if guest state is protected?
Re: [RFC PATCH 25/35] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
Quoted 11 lines · Sean Christopherson
Quoted 6 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since many of the registers used by the SEV-ES are encrypted and cannot be read or written, adjust the
__get_sregs()/__set_sregs()to only get or set the registers being tracked (efer, cr0, cr4 and cr8) once the VMSA is encrypted.Is there an actual use case for writing said registers after the VMSA is encrypted? Assuming there's a separate "debug mode" and live migration has special logic, can KVM simply reject the
ioctl()if guest state is protected?
Yeah, I originally had it that way but one of the folks looking at live migration for SEV-ES thought it would be easier given the way Qemu does things. But I think it's easy enough to batch the tracking registers into the VMSA state that is being transferred during live migration. Let me check that out and likely the SET ioctl() could just skip all the regs.
Thanks,
Tom
Quoted 1 line
Re: [RFC PATCH 25/35] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
Quoted 13 lines · Tom Lendacky
Quoted 11 lines · Sean Christopherson
Quoted 6 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since many of the registers used by the SEV-ES are encrypted and cannot be read or written, adjust the
__get_sregs()/__set_sregs()to only get or set the registers being tracked (efer, cr0, cr4 and cr8) once the VMSA is encrypted.Is there an actual use case for writing said registers after the VMSA is encrypted? Assuming there's a separate "debug mode" and live migration has special logic, can KVM simply reject the
ioctl()if guest state is protected?
Quoted 5 lines
Yeah, I originally had it that way but one of the folks looking at live migration for SEV-ES thought it would be easier given the way Qemu does things. But I think it's easy enough to batch the tracking registers into the VMSA state that is being transferred during live migration. Let me check that out and likely the SET
ioctl()could just skip all the regs.
Hmm, that would be ideal. How are the tracked registers validated when they're loaded at the destination? It seems odd/dangerous that KVM would have full control over efer/cr0/cr4/cr8. I.e. why is KVM even responsibile for migrating that information, e.g. as opposed to migrating an opaque blob that contains encrypted versions of those registers?
Re: [RFC PATCH 25/35] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
Quoted 20 lines · Sean Christopherson
Quoted 13 lines · Tom Lendacky
Quoted 11 lines · Sean Christopherson
Quoted 6 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since many of the registers used by the SEV-ES are encrypted and cannot be read or written, adjust the
__get_sregs()/__set_sregs()to only get or set the registers being tracked (efer, cr0, cr4 and cr8) once the VMSA is encrypted.Is there an actual use case for writing said registers after the VMSA is encrypted? Assuming there's a separate "debug mode" and live migration has special logic, can KVM simply reject the
ioctl()if guest state is protected?Quoted 5 lines
Yeah, I originally had it that way but one of the folks looking at live migration for SEV-ES thought it would be easier given the way Qemu does things. But I think it's easy enough to batch the tracking registers into the VMSA state that is being transferred during live migration. Let me check that out and likely the SET
ioctl()could just skip all the regs.
Quoted 6 lines
Hmm, that would be ideal. How are the tracked registers validated when they're loaded at the destination? It seems odd/dangerous that KVM would have full control over efer/cr0/cr4/cr8. I.e. why is KVM even responsibile for migrating that information, e.g. as opposed to migrating an opaque blob that contains encrypted versions of those registers?
KVM doesn't have control of them. They are part of the guest's encrypted state and that is what the guest uses. KVM can't alter the value that the guest is using for them once the VMSA is encrypted. However, KVM makes some decisions based on the values it thinks it knows. For example, early on I remember the async PF support failing because the CR0 that KVM thought the guest had didn't have the PE bit set, even though the guest was in protected mode. So KVM didn't include the error code in the exception it injected (is_protmode() was false) and things failed. Without syncing these values after live migration, things also fail (probably for the same reason). So the idea is to just keep KVM apprised of the values that the guest has.
Thanks,
Tom
Re: [RFC PATCH 25/35] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
Quoted 28 lines · Tom Lendacky
Quoted 20 lines · Sean Christopherson
Quoted 13 lines · Tom Lendacky
Quoted 11 lines · Sean Christopherson
Quoted 6 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Since many of the registers used by the SEV-ES are encrypted and cannot be read or written, adjust the
__get_sregs()/__set_sregs()to only get or set the registers being tracked (efer, cr0, cr4 and cr8) once the VMSA is encrypted.Is there an actual use case for writing said registers after the VMSA is encrypted? Assuming there's a separate "debug mode" and live migration has special logic, can KVM simply reject the
ioctl()if guest state is protected?Quoted 5 lines
Yeah, I originally had it that way but one of the folks looking at live migration for SEV-ES thought it would be easier given the way Qemu does things. But I think it's easy enough to batch the tracking registers into the VMSA state that is being transferred during live migration. Let me check that out and likely the SET
ioctl()could just skip all the regs.Quoted 6 lines
Hmm, that would be ideal. How are the tracked registers validated when they're loaded at the destination? It seems odd/dangerous that KVM would have full control over efer/cr0/cr4/cr8. I.e. why is KVM even responsibile for migrating that information, e.g. as opposed to migrating an opaque blob that contains encrypted versions of those registers?
Quoted 11 lines
KVM doesn't have control of them. They are part of the guest's encrypted state and that is what the guest uses. KVM can't alter the value that the guest is using for them once the VMSA is encrypted. However, KVM makes some decisions based on the values it thinks it knows. For example, early on I remember the async PF support failing because the CR0 that KVM thought the guest had didn't have the PE bit set, even though the guest was in protected mode. So KVM didn't include the error code in the exception it injected (
is_protmode()was false) and things failed. Without syncing these values after live migration, things also fail (probably for the same reason). So the idea is to just keep KVM apprised of the values that the guest has.
Ah, gotcha. Migrating tracked state through the VMSA would probably be ideal. The semantics of __set_sregs() kinda setting state but not reaaaally setting state would be weird.
Paolo Bonzini
maintainerRe: [RFC PATCH 25/35] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
Quoted 12 lines · Sean Christopherson
Quoted 11 lines
KVM doesn't have control of them. They are part of the guest's encrypted state and that is what the guest uses. KVM can't alter the value that the guest is using for them once the VMSA is encrypted. However, KVM makes some decisions based on the values it thinks it knows. For example, early on I remember the async PF support failing because the CR0 that KVM thought the guest had didn't have the PE bit set, even though the guest was in protected mode. So KVM didn't include the error code in the exception it injected (
is_protmode()was false) and things failed. Without syncing these values after live migration, things also fail (probably for the same reason). So the idea is to just keep KVM apprised of the values that the guest has.
Quoted 3 lines
Ah, gotcha. Migrating tracked state through the VMSA would probably be ideal. The semantics of
__set_sregs()kinda setting state but not reaaaally setting state would be weird.
How would that work with TDX?
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH 25/35] KVM: x86: Update __get_sregs() / __set_sregs() to support SEV-ES
Quoted 13 lines · Paolo Bonzini
Quoted 12 lines · Sean Christopherson
Quoted 11 lines
KVM doesn't have control of them. They are part of the guest's encrypted state and that is what the guest uses. KVM can't alter the value that the guest is using for them once the VMSA is encrypted. However, KVM makes some decisions based on the values it thinks it knows. For example, early on I remember the async PF support failing because the CR0 that KVM thought the guest had didn't have the PE bit set, even though the guest was in protected mode. So KVM didn't include the error code in the exception it injected (
is_protmode()was false) and things failed. Without syncing these values after live migration, things also fail (probably for the same reason). So the idea is to just keep KVM apprised of the values that the guest has.
Quoted 5 lines
Quoted 3 lines
Ah, gotcha. Migrating tracked state through the VMSA would probably be ideal. The semantics of
__set_sregs()kinda setting state but not reaaaally setting state would be weird.How would that work with TDX?
Can you elaborate? I.e. how would what work with TDX?
RFC PATCH 26/35 KVM: SVM: Guest FPU state save/restore not needed for SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The guest FPU is automatically restored on VMRUN and saved on VMEXIT by the hardware, so there is no reason to do this in KVM.
--- arch/x86/kvm/svm/svm.c | 8 ++++++-- arch/x86/kvm/x86.c | 18 ++++++++++++++---- 2 files changed, 20 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/svm.c +6 -2
@@ -3682,7 +3682,9 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)svm_set_dr6 (svm,DR6_FIXED_1 |DR6_RTM );clgi (); -kvm_load_guest_xsave_state (vcpu); + +if (!sev_es_guest (svm->vcpu.kvm)) +kvm_load_guest_xsave_state (vcpu);if (lapic_in_kernel (vcpu) && vcpu->arch.apic->lapic_timer.timer_advance_ns)
@@ -3728,7 +3730,9 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)if (unlikely (svm->vmcb->control.exit_code ==SVM_EXIT_NMI ))kvm_before_interrupt (&svm->vcpu); -kvm_load_host_xsave_state (vcpu); +if (!sev_es_guest (svm->vcpu.kvm)) +kvm_load_host_xsave_state (vcpu); +stgi ();/* Any pending NMI will happen here */
arch/x86/kvm/x86.c +14 -4
@@ -8896,9 +8896,14 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu)kvm_save_current_fpu (vcpu->arch.user_fpu); -/* PKRU is separately restored in kvm_x86_ops.run. */ -__copy_kernel_to_fpregs (&vcpu->arch.guest_fpu->state, - ~XFEATURE_MASK_PKRU ); +/* +* An encrypted save area means that the guest state can't be +* set by the hypervisor, so skip trying to set it. +*/ +if (!vcpu->arch.vmsa_encrypted) +/* PKRU is separately restored in kvm_x86_ops.run. */ +__copy_kernel_to_fpregs (&vcpu->arch.guest_fpu->state, + ~XFEATURE_MASK_PKRU );fpregs_mark_activate ();fpregs_unlock ();
@@ -8911,7 +8916,12 @@ static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu) {fpregs_lock (); -kvm_save_current_fpu (vcpu->arch.guest_fpu); +/* +* An encrypted save area means that the guest state can't be +* read/saved by the hypervisor, so skip trying to save it. +*/ +if (!vcpu->arch.vmsa_encrypted) +kvm_save_current_fpu (vcpu->arch.guest_fpu);copy_kernel_to_fpregs (&vcpu->arch.user_fpu->state);
Signature · 1 line
2.28.0
A message that is not in the archive (sent off-list or not yet received): 20200914213917.GD7192@sjchrist-ice
Re: [RFC PATCH 26/35] KVM: SVM: Guest FPU state save/restore not needed for SEV-ES guest
Quoted 9 lines · Sean Christopherson
Quoted 4 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
The guest FPU is automatically restored on VMRUN and saved on VMEXIT by the hardware, so there is no reason to do this in KVM.
I assume hardware has its own buffer? If so, a better approach would be to not allocate arch.guest_fpu in the first place, and then rework KVM to key off !guest_fpu.
Yup, let me look into that.
Thanks,
Tom
Quoted 72 lines
Quoted 72 lines
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/svm/svm.c | 8 ++++++-- arch/x86/kvm/x86.c | 18 ++++++++++++++---- 2 files changed, 20 insertions(+), 6 deletions(-)arch/x86/kvm/svm/svm.c +6
-2@@ -3682,7 +3682,9 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)svm_set_dr6 (svm,DR6_FIXED_1 |DR6_RTM );clgi (); -kvm_load_guest_xsave_state (vcpu); + +if (!sev_es_guest (svm->vcpu.kvm)) +kvm_load_guest_xsave_state (vcpu);if (lapic_in_kernel (vcpu) && vcpu->arch.apic->lapic_timer.timer_advance_ns)@@ -3728,7 +3730,9 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)if (unlikely (svm->vmcb->control.exit_code ==SVM_EXIT_NMI ))kvm_before_interrupt (&svm->vcpu); -kvm_load_host_xsave_state (vcpu); +if (!sev_es_guest (svm->vcpu.kvm)) +kvm_load_host_xsave_state (vcpu); +stgi ();/* Any pending NMI will happen here */ arch/x86/kvm/x86.c +14
-4@@ -8896,9 +8896,14 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu)kvm_save_current_fpu (vcpu->arch.user_fpu); -/* PKRU is separately restored in kvm_x86_ops.run. */ -__copy_kernel_to_fpregs (&vcpu->arch.guest_fpu->state, - ~XFEATURE_MASK_PKRU ); +/* +* An encrypted save area means that the guest state can't be +* set by the hypervisor, so skip trying to set it. +*/ +if (!vcpu->arch.vmsa_encrypted) +/* PKRU is separately restored in kvm_x86_ops.run. */ +__copy_kernel_to_fpregs (&vcpu->arch.guest_fpu->state, + ~XFEATURE_MASK_PKRU );fpregs_mark_activate ();fpregs_unlock ();@@ -8911,7 +8916,12 @@ static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu) {fpregs_lock (); -kvm_save_current_fpu (vcpu->arch.guest_fpu); +/* +* An encrypted save area means that the guest state can't be +* read/saved by the hypervisor, so skip trying to save it. +*/ +if (!vcpu->arch.vmsa_encrypted) +kvm_save_current_fpu (vcpu->arch.guest_fpu);copy_kernel_to_fpregs (&vcpu->arch.user_fpu->state);Signature · 1 line
2.28.0
Re: [RFC PATCH 26/35] KVM: SVM: Guest FPU state save/restore not needed for SEV-ES guest
Quoted 4 lines · Tom Lendacky
Quoted 3 lines · Sean Christopherson
Quoted 2 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
The guest FPU is automatically restored on VMRUN and saved on VMEXIT by the hardware, so there is no reason to do this in KVM.
I assume hardware has its own buffer? If so, a better approach would be to not allocate arch.guest_fpu in the first place, and then rework KVM to key off !guest_fpu.
Yup, let me look into that.
Heh, it's on our todo list as well :-)
RFC PATCH 27/35 KVM: SVM: Add support for booting APs for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
Typically under KVM, an AP is booted using the INIT-SIPI-SIPI sequence, where the guest vCPU register state is updated and then the vCPU is VMRUN to begin execution of the AP. For an SEV-ES guest, this won't work because the guest register state is encrypted.
Following the GHCB specification, the hypervisor must not alter the guest register state, so KVM must track an AP/vCPU boot. Should the guest want to park the AP, it must use the AP Reset Hold exit event in place of, for example, a HLT loop.
First AP boot (first INIT-SIPI-SIPI sequence):
Execute the AP (vCPU) as it was initialized and measured by the SEV-ES support. It is up to the guest to transfer control of the AP to the proper location.
Subsequent AP boot:
KVM will expect to receive an AP Reset Hold exit event indicating that the vCPU is being parked and will require an INIT-SIPI-SIPI sequence to awaken it. When the AP Reset Hold exit event is received, KVM will place the vCPU into a simulated HLT mode. Upon receiving the INIT-SIPI-SIPI sequence, KVM will make the vCPU runnable. It is again up to the guest to then transfer control of the AP to the proper location.
The GHCB specification also requires the hypervisor to save the address of an AP Jump Table so that, for example, vCPUs that have been parked by UEFI can be started by the OS. Provide support for the AP Jump Table set/get exit code.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/sev.c | 48 +++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 7 +++++ arch/x86/kvm/svm/svm.h | 3 +++ arch/x86/kvm/x86.c | 9 +++++++ 5 files changed, 69 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1237,6 +1237,8 @@ struct kvm_x86_ops { unsigned long val);bool (*allow_debug)(struct kvm *kvm); + +void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu,u8 vector); };struct kvm_x86_nested_ops {
arch/x86/kvm/svm/sev.c +48 -0
63 lines not on this page · raw message
arch/x86/kvm/svm/svm.c +7 -0
@@ -4343,6 +4343,11 @@ static bool svm_allow_debug(struct kvm *kvm)return !sev_es_guest (kvm); } +static void svm_vcpu_deliver_sipi_vector (struct kvm_vcpu *vcpu,u8 vector) +{ +sev_vcpu_deliver_sipi_vector (vcpu, vector); +} +static void svm_vm_destroy (struct kvm *kvm) {avic_vm_destroy (kvm);
@@ -4486,6 +4491,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .reg_write_override = svm_reg_write_override, .allow_debug = svm_allow_debug, + + .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector, };static struct kvm_x86_init_ops svm_init_ops__initdata = {
arch/x86/kvm/svm/svm.h +3 -0
@@ -67,6 +67,7 @@ struct kvm_sev_info {int fd;/* SEV device fd */ unsigned long pages_locked;/* Number of pages locked */ struct list_head regions_list;/* List of registered regions */ +u64 ap_jump_table;/* SEV-ES AP Jump Table address */ };struct kvm_svm {
@@ -165,6 +166,7 @@ struct vcpu_svm {struct vmcb_save_area *vmsa;struct ghcb *ghcb;struct kvm_host_map ghcb_map; +bool ap_hlt_loop;/* SEV-ES scratch area support */ void *ghcb_sa;
@@ -565,6 +567,7 @@ void __init sev_hardware_setup(void);void sev_hardware_teardown (void );int sev_handle_vmgexit (struct vcpu_svm *svm);int sev_es_string_io (struct vcpu_svm *svm,int size,unsigned int port,int in); +void sev_vcpu_deliver_sipi_vector (struct kvm_vcpu *vcpu,u8 vector);/* VMSA Accessor functions */
arch/x86/kvm/x86.c +9 -0
@@ -9772,6 +9772,15 @@ void kvm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) {struct kvm_segment cs; +/* +* For SEV-ES, the register state can't be altered by KVM. If the VMSA +* is encrypted, call the vcpu_deliver_sipi_vector() x86 op. +*/ +if (vcpu->arch.vmsa_encrypted) { + kvm_x86_ops.vcpu_deliver_sipi_vector (vcpu, vector); +return ; + } +kvm_get_segment (vcpu, &cs,VCPU_SREG_CS ); cs.selector = vector <<8 ; cs.base = vector <<12 ;
Signature · 1 line
2.28.0
RFC PATCH 28/35 KVM: X86: Update kvm_skip_emulated_instruction() for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The register state for an SEV-ES guest is encrypted so the value of the RIP cannot be updated. For an automatic exit, the RIP will be advanced as necessary. For a non-automatic exit, it is up to the #VC handler in the guest to advance the RIP.
Add support to skip any RIP updates in kvm_skip_emulated_instruction() for an SEV-ES guest.
--- arch/x86/kvm/x86.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +5 -1
@@ -6874,13 +6874,17 @@ static int kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu)int kvm_skip_emulated_instruction (struct kvm_vcpu *vcpu) { -unsigned long rflags = kvm_x86_ops.get_rflags (vcpu); +unsigned long rflags;int r; r = kvm_x86_ops.skip_emulated_instruction (vcpu);if (unlikely (!r))return 0 ; +if (vcpu->arch.vmsa_encrypted) +return 1 ; + + rflags = kvm_x86_ops.get_rflags (vcpu);/* * rflags is the old, "raw" value of the flags. The new value has * not been saved yet.
Signature · 1 line
2.28.0
Re: [RFC PATCH 28/35] KVM: X86: Update kvm_skip_emulated_instruction() for an SEV-ES guest
Quoted 9 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
The register state for an SEV-ES guest is encrypted so the value of the RIP cannot be updated. For an automatic exit, the RIP will be advanced as necessary. For a non-automatic exit, it is up to the #VC handler in the guest to advance the RIP.
Add support to skip any RIP updates in
kvm_skip_emulated_instruction()for an SEV-ES guest.
Is there a reason this can't be handled in svm? E.g. can KVM be reworked to effectively split the emulation logic so that it's a bug for KVM to end up trying to modify RIP?
Also, patch 06 modifies SVM's skip_emulated_instruction() to skip the RIP update, but keeps the "svm_set_interrupt_shadow(vcpu, 0)" logic. Seems like either that change or this one is wrong.
Quoted 31 lines
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/x86.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)arch/x86/kvm/x86.c +5
-1@@ -6874,13 +6874,17 @@ static int kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu)int kvm_skip_emulated_instruction (struct kvm_vcpu *vcpu) { -unsigned long rflags = kvm_x86_ops.get_rflags (vcpu); +unsigned long rflags;int r; r = kvm_x86_ops.skip_emulated_instruction (vcpu);if (unlikely (!r))return 0 ; +if (vcpu->arch.vmsa_encrypted) +return 1 ; + + rflags = kvm_x86_ops.get_rflags (vcpu);/* * rflags is the old, "raw" value of the flags. The new value has * not been saved yet. Signature · 1 line
2.28.0
Re: [RFC PATCH 28/35] KVM: X86: Update kvm_skip_emulated_instruction() for an SEV-ES guest
Quoted 11 lines · Sean Christopherson
Quoted 9 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
The register state for an SEV-ES guest is encrypted so the value of the RIP cannot be updated. For an automatic exit, the RIP will be advanced as necessary. For a non-automatic exit, it is up to the #VC handler in the guest to advance the RIP.
Add support to skip any RIP updates in
kvm_skip_emulated_instruction()for an SEV-ES guest.
Quoted 7 lines
Is there a reason this can't be handled in svm? E.g. can KVM be reworked to effectively split the emulation logic so that it's a bug for KVM to end up trying to modify RIP?
Also, patch 06 modifies SVM's
skip_emulated_instruction()to skip the RIP update, but keeps the "svm_set_interrupt_shadow(vcpu, 0)" logic. Seems like either that change or this one is wrong.
I added this because of the get_rflags() call. But let me look into changing get_rflags() in svm.c and see if this patch can go away.
Thanks,
Tom
Quoted 32 lines
Quoted 32 lines
Signed-off-by Tom Lendacky <[email protected]>--- arch/x86/kvm/x86.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)arch/x86/kvm/x86.c +5
-1@@ -6874,13 +6874,17 @@ static int kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu)int kvm_skip_emulated_instruction (struct kvm_vcpu *vcpu) { -unsigned long rflags = kvm_x86_ops.get_rflags (vcpu); +unsigned long rflags;int r; r = kvm_x86_ops.skip_emulated_instruction (vcpu);if (unlikely (!r))return 0 ; +if (vcpu->arch.vmsa_encrypted) +return 1 ; + + rflags = kvm_x86_ops.get_rflags (vcpu);/* * rflags is the old, "raw" value of the flags. The new value has * not been saved yet. Signature · 1 line
2.28.0
RFC PATCH 29/35 KVM: SVM: Add NMI support for an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The GHCB specification defines how NMIs are to be handled for an SEV-ES guest. To detect the completion of an NMI the hypervisor must not intercept the IRET instruction (because a #VC while running the NMI will issue an IRET) and, instead, must receive an NMI Complete exit event from the guest.
Update the KVM support for detecting the completion of NMIs in the guest to follow the GHCB specification. When an SEV-ES guest is active, the IRET instruction will no longer be intercepted. Now, when the NMI Complete exit event is received, the iret_interception() function will be called to simulate the completion of the NMI.
--- arch/x86/kvm/svm/sev.c | 3 +++ arch/x86/kvm/svm/svm.c | 20 +++++++++++++------- 2 files changed, 16 insertions(+), 7 deletions(-)
arch/x86/kvm/svm/sev.c +3 -0
@@ -1474,6 +1474,9 @@ int sev_handle_vmgexit(struct vcpu_svm *svm) control->exit_info_2, svm->ghcb_sa);break ; +case SVM_VMGEXIT_NMI_COMPLETE : + ret =svm_invoke_exit_handler (svm,SVM_EXIT_IRET ); +break ;case SVM_VMGEXIT_AP_HLT_LOOP : svm->ap_hlt_loop = true; ret =kvm_emulate_halt (&svm->vcpu);
arch/x86/kvm/svm/svm.c +13 -7
@@ -2268,9 +2268,11 @@ static int cpuid_interception(struct vcpu_svm *svm)static int iret_interception (struct vcpu_svm *svm) { ++svm->vcpu.stat.nmi_window_exits; -svm_clr_intercept (svm,INTERCEPT_IRET ); svm->vcpu.arch.hflags |=HF_IRET_MASK ; - svm->nmi_iret_rip =kvm_rip_read (&svm->vcpu); +if (!sev_es_guest (svm->vcpu.kvm)) { +svm_clr_intercept (svm,INTERCEPT_IRET ); + svm->nmi_iret_rip =kvm_rip_read (&svm->vcpu); + }kvm_make_request (KVM_REQ_EVENT , &svm->vcpu);return 1 ; }
@@ -3242,7 +3244,8 @@ static void svm_inject_nmi(struct kvm_vcpu *vcpu) svm->vmcb->control.event_inj =SVM_EVTINJ_VALID |SVM_EVTINJ_TYPE_NMI ; vcpu->arch.hflags |=HF_NMI_MASK ; -svm_set_intercept (svm,INTERCEPT_IRET ); +if (!sev_es_guest (svm->vcpu.kvm)) +svm_set_intercept (svm,INTERCEPT_IRET ); ++vcpu->stat.nmi_injections; }
@@ -3326,10 +3329,12 @@ static void svm_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked)if (masked) { svm->vcpu.arch.hflags |=HF_NMI_MASK ; -svm_set_intercept (svm,INTERCEPT_IRET ); +if (!sev_es_guest (svm->vcpu.kvm)) +svm_set_intercept (svm,INTERCEPT_IRET ); }else { svm->vcpu.arch.hflags &= ~HF_NMI_MASK ; -svm_clr_intercept (svm,INTERCEPT_IRET ); +if (!sev_es_guest (svm->vcpu.kvm)) +svm_clr_intercept (svm,INTERCEPT_IRET ); } }
@@ -3507,8 +3512,9 @@ static void svm_complete_interrupts(struct vcpu_svm *svm)* If we've made progress since setting HF_IRET_MASK, we've * executed an IRET and can allow NMI injection. */ -if ((svm->vcpu.arch.hflags &HF_IRET_MASK ) - &&kvm_rip_read (&svm->vcpu) != svm->nmi_iret_rip) { +if ((svm->vcpu.arch.hflags &HF_IRET_MASK ) && + (sev_es_guest (svm->vcpu.kvm) || +kvm_rip_read (&svm->vcpu) != svm->nmi_iret_rip)) { svm->vcpu.arch.hflags &= ~(HF_NMI_MASK |HF_IRET_MASK );kvm_make_request (KVM_REQ_EVENT , &svm->vcpu); }
Signature · 1 line
2.28.0
RFC PATCH 30/35 KVM: SVM: Set the encryption mask for the SVM host save area
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The SVM host save area is used to restore some host state on VMEXIT of an SEV-ES guest. After allocating the save area, clear it and add the encryption mask to the SVM host save area physical address that is programmed into the VM_HSAVE_PA MSR.
--- arch/x86/kvm/svm/sev.c | 1 - arch/x86/kvm/svm/svm.c | 3 ++- arch/x86/kvm/svm/svm.h | 2 ++ 3 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/sev.c +0 -1
@@ -30,7 +30,6 @@ unsigned int max_sev_asid;static unsigned int min_sev_asid;static unsigned long *sev_asid_bitmap;static unsigned long *sev_reclaim_asid_bitmap; -#define __sme_page_pa (x) __sme_set(page_to_pfn(x) << PAGE_SHIFT)struct enc_region {struct list_head list;
arch/x86/kvm/svm/svm.c +2 -1
@@ -478,7 +478,7 @@ static int svm_hardware_enable(void)wrmsrl (MSR_EFER , efer |EFER_SVME ); -wrmsrl (MSR_VM_HSAVE_PA ,page_to_pfn (sd->save_area) <<PAGE_SHIFT ); +wrmsrl (MSR_VM_HSAVE_PA ,__sme_page_pa (sd->save_area));if (static_cpu_has (X86_FEATURE_TSCRATEMSR )) {wrmsrl (MSR_AMD64_TSC_RATIO ,TSC_RATIO_DEFAULT );
@@ -546,6 +546,7 @@ static int svm_cpu_init(int cpu) sd->save_area =alloc_page (GFP_KERNEL );if (!sd->save_area) goto free_cpu_data; +clear_page (page_address (sd->save_area));if (svm_sev_enabled ()) { sd->sev_vmcbs = kmalloc_array(max_sev_asid + 1,
arch/x86/kvm/svm/svm.h +2 -0
@@ -21,6 +21,8 @@#include <asm/svm.h> +#define __sme_page_pa (x) __sme_set(page_to_pfn(x) << PAGE_SHIFT) +static const u32 host_save_user_msrs[] = { #ifdef CONFIG_X86_64 MSR_STAR, MSR_LSTAR, MSR_CSTAR, MSR_SYSCALL_MASK, MSR_KERNEL_GS_BASE,
Signature · 1 line
2.28.0
RFC PATCH 31/35 KVM: SVM: Update ASID allocation to support SEV-ES guests
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
SEV and SEV-ES guests each have dedicated ASID ranges. Update the ASID allocation routine to return an ASID in the respective range.
--- arch/x86/kvm/svm/sev.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
arch/x86/kvm/svm/sev.c +14 -11
@@ -61,19 +61,19 @@ static int sev_flush_asids(void) }/* Must be called with the sev_bitmap_lock held */ -static bool __sev_recycle_asids (void ) +static bool __sev_recycle_asids (int min_asid,int max_asid) {int pos;/* Check if there are any ASIDs to reclaim before performing a flush */ - pos =find_next_bit (sev_reclaim_asid_bitmap, - max_sev_asid, min_sev_asid -1 ); -if (pos >= max_sev_asid) + pos =find_next_bit (sev_reclaim_asid_bitmap, max_sev_asid, min_asid); +if (pos >= max_asid)return false;if (sev_flush_asids ())return false; +/* The flush process will flush all reclaimable SEV and SEV-ES ASIDs */ bitmap_xor (sev_asid_bitmap, sev_asid_bitmap, sev_reclaim_asid_bitmap, max_sev_asid);bitmap_zero (sev_reclaim_asid_bitmap, max_sev_asid);
@@ -81,20 +81,23 @@ static bool __sev_recycle_asids(void)return true; } -static int sev_asid_new (void ) +static int sev_asid_new (struct kvm_sev_info *sev) { +int pos, min_asid, max_asid;bool retry = true; -int pos;mutex_lock (&sev_bitmap_lock);/* -* SEV-enabled guest must use asid from min_sev_asid to max_sev_asid. +* SEV-enabled guests must use asid from min_sev_asid to max_sev_asid. +* SEV-ES-enabled guest can use from 1 to min_sev_asid - 1. */ + min_asid = sev->es_active ?0 : min_sev_asid -1 ; + max_asid = sev->es_active ? min_sev_asid -1 : max_sev_asid; again: - pos =find_next_zero_bit (sev_asid_bitmap, max_sev_asid, min_sev_asid -1 ); -if (pos >= max_sev_asid) { -if (retry &&__sev_recycle_asids ()) { + pos =find_next_zero_bit (sev_asid_bitmap, max_sev_asid, min_asid); +if (pos >= max_asid) { +if (retry &&__sev_recycle_asids (min_asid, max_asid)) { retry = false; goto again; }
@@ -176,7 +179,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)if (unlikely (sev->active))return ret; - asid =sev_asid_new (); + asid =sev_asid_new (sev);if (asid <0 )return ret;
Signature · 1 line
2.28.0
RFC PATCH 32/35 KVM: SVM: Provide support for SEV-ES vCPU creation/loading
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
An SEV-ES vCPU requires additional VMCB initialization requirements for vCPU creation and vCPU load/put requirements. This includes:
General VMCB initialization changes:
- Set a VMCB control bit to enable SEV-ES support on the vCPU.
- Set the VMCB encrypted VM save area address.
- CRx registers are part of the encrypted register state and cannot be updated. Remove CRx the register read and write intercepts and replace them with CRx register write traps to track the CRx register values.
- Certain MSR values are part of the encrypted register state and cannot be updated. Remove certain MSR intercepts (EFER, CR_PAT, etc.).
- Remove the #GP intercept (no support for "enable_vmware_backdoor").
- Remove the XSETBV intercept since the hypervisor cannot modify XCR0.
General vCPU creation changes:
- Set the initial GHCB gpa value as per the GHCB specification.
General vCPU load changes:
- SEV-ES hardware will restore certain registers on VMEXIT, but not save them on VMRUN (see Table B-3 and Table B-4 of the AMD64 APM Volume 2). During vCPU loading, perform a VMSAVE to the per-CPU SVM save area and save the current value of XCR0 to the per-CPU SVM save area.
--- arch/x86/include/asm/svm.h | 15 ++++++++++- arch/x86/kvm/svm/sev.c | 54 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 19 +++++++++++--- arch/x86/kvm/svm/svm.h | 3 +++ 4 files changed, 87 insertions(+), 4 deletions(-)
arch/x86/include/asm/svm.h +14 -1
@@ -53,6 +53,16 @@ enum {INTERCEPT_MWAIT_COND ,INTERCEPT_XSETBV ,INTERCEPT_RDPRU , +TRAP_EFER_WRITE , +TRAP_CR0_WRITE , +TRAP_CR1_WRITE , +TRAP_CR2_WRITE , +TRAP_CR3_WRITE , +TRAP_CR4_WRITE , +TRAP_CR5_WRITE , +TRAP_CR6_WRITE , +TRAP_CR7_WRITE , + TRAP_CR8_WRITE, };
@@ -96,6 +106,8 @@ struct __attribute__ ((__packed__)) vmcb_control_area {u8 reserved_6[8 ];/* Offset 0xe8 */ u64 avic_logical_id;/* Offset 0xf0 */ u64 avic_physical_id;/* Offset 0xf8 */ +u8 reserved_7[8 ]; +u64 vmsa_pa;/* Used for an SEV-ES guest */ };
@@ -150,6 +162,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {#define SVM_NESTED_CTL_NP_ENABLE BIT(0)#define SVM_NESTED_CTL_SEV_ENABLE BIT(1) +#define SVM_NESTED_CTL_SEV_ES_ENABLE BIT(2)struct vmcb_seg {u16 selector;
@@ -249,7 +262,7 @@ struct ghcb {static inline void __unused_size_checks (void ) {BUILD_BUG_ON (sizeof (struct vmcb_save_area ) !=1032 ); -BUILD_BUG_ON (sizeof (struct vmcb_control_area ) !=256 ); +BUILD_BUG_ON (sizeof (struct vmcb_control_area ) !=272 );BUILD_BUG_ON (sizeof (struct ghcb ) !=4096 ); }
arch/x86/kvm/svm/sev.c +54 -0
57 lines not on this page · raw message
arch/x86/kvm/svm/svm.c +16 -3
@@ -91,7 +91,7 @@ static DEFINE_PER_CPU(u64, current_tsc_ratio);static const struct svm_direct_access_msrs {u32 index;/* Index of the MSR */ -bool always;/* True if intercept is always on */ +bool always;/* True if intercept is initially cleared */ } direct_access_msrs[] = { { .index =MSR_STAR , .always = true }, { .index =MSR_IA32_SYSENTER_CS , .always = true },
@@ -109,6 +109,9 @@ static const struct svm_direct_access_msrs { { .index = MSR_IA32_LASTBRANCHTOIP, .always = false }, { .index = MSR_IA32_LASTINTFROMIP, .always = false }, { .index = MSR_IA32_LASTINTTOIP, .always = false }, + { .index = MSR_EFER, .always = false }, + { .index = MSR_IA32_CR_PAT, .always = false }, + { .index = MSR_AMD64_SEV_ES_GHCB, .always = true }, { .index = MSR_INVALID, .always = false }, };
@@ -598,8 +601,7 @@ static bool msr_write_intercepted(struct kvm_vcpu *vcpu, unsigned msr)return !!test_bit (bit_write, &tmp); } -static void set_msr_interception (u32 *msrpm,unsigned msr, -int read,int write) +void set_msr_interception (u32 *msrpm,unsigned int msr,int read,int write) {u8 bit_read, bit_write;unsigned long tmp;
@@ -1147,6 +1149,11 @@ static void init_vmcb(struct vcpu_svm *svm)if (sev_guest (svm->vcpu.kvm)) { svm->vmcb->control.nested_ctl |=SVM_NESTED_CTL_SEV_ENABLE ;clr_exception_intercept (svm,UD_VECTOR ); + +if (sev_es_guest (svm->vcpu.kvm)) { +/* Perform SEV-ES specific VMCB updates */ +sev_es_init_vmcb (svm); + } }vmcb_mark_all_dirty (svm->vmcb);
@@ -1253,6 +1260,10 @@ static int svm_create_vcpu(struct kvm_vcpu *vcpu)svm_init_osvw (vcpu); vcpu->arch.microcode_version =0x01000065 ; +if (sev_es_guest (svm->vcpu.kvm)) +/* Perform SEV-ES specific VMCB creation updates */ +sev_es_create_vcpu (svm); +return 0 ; free_page5:
@@ -1375,6 +1386,7 @@ static void svm_vcpu_put(struct kvm_vcpu *vcpu)loadsegment (gs, svm->host.gs);#endif #endif +for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++)wrmsrl (host_save_user_msrs[i], svm->host_user_msrs[i]); }
@@ -3039,6 +3051,7 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)pr_err ("%-20s%016llx\n" ,"avic_backing_page:" , control->avic_backing_page);pr_err ("%-20s%016llx\n" ,"avic_logical_id:" , control->avic_logical_id);pr_err ("%-20s%016llx\n" ,"avic_physical_id:" , control->avic_physical_id); +pr_err ("%-20s%016llx\n" ,"vmsa_pa:" , control->vmsa_pa);pr_err ("VMCB State Save Area:\n" ); pr_err("%-5s s: %04x a: %04x l: %08x b: %016llx\n", "es:",
arch/x86/kvm/svm/svm.h +3 -0
@@ -412,6 +412,7 @@ bool svm_nmi_blocked(struct kvm_vcpu *vcpu);bool svm_interrupt_blocked (struct kvm_vcpu *vcpu);void svm_set_gif (struct vcpu_svm *svm,bool value);int svm_invoke_exit_handler (struct vcpu_svm *svm,u64 exit_code); +void set_msr_interception (u32 *msrpm,unsigned int msr,int read,int write);/* nested.c */
@@ -570,6 +571,8 @@ void sev_hardware_teardown(void);int sev_handle_vmgexit (struct vcpu_svm *svm);int sev_es_string_io (struct vcpu_svm *svm,int size,unsigned int port,int in);void sev_vcpu_deliver_sipi_vector (struct kvm_vcpu *vcpu,u8 vector); +void sev_es_init_vmcb (struct vcpu_svm *svm); +void sev_es_create_vcpu (struct vcpu_svm *svm);/* VMSA Accessor functions */
Signature · 1 line
2.28.0
RFC PATCH 33/35 KVM: SVM: Provide support for SEV-ES vCPU loading
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
An SEV-ES vCPU requires additional VMCB vCPU load/put requirements. SEV-ES hardware will restore certain registers on VMEXIT, but not save them on VMRUM (see Table B-3 and Table B-4 of the AMD64 APM Volume 2), so make the following changes:
General vCPU load changes:
- During vCPU loading, perform a VMSAVE to the per-CPU SVM save area and save the current value of XCR0 to the per-CPU SVM save area as these registers will be restored on VMEXIT.
General vCPU put changes:
- Do not attempt to restore registers that SEV-ES hardware has already restored on VMEXIT.
--- arch/x86/kvm/svm/sev.c | 48 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 36 +++++++++++++++++++------------ arch/x86/kvm/svm/svm.h | 22 +++++++++++++------ 3 files changed, 87 insertions(+), 19 deletions(-)
arch/x86/kvm/svm/sev.c +48 -0
@@ -17,11 +17,14 @@#include <linux/trace_events.h> #include <asm/trapnr.h> +#include <asm/fpu/internal.h> #include "x86.h" #include "svm.h" #include "trace.h" +#define __ex (x) __kvm_handle_fault_on_reboot(x) +static u8 sev_enc_bit;static int sev_flush_asids (void );static DECLARE_RWSEM (sev_deactivate_lock);
@@ -1599,3 +1602,48 @@ void sev_es_create_vcpu(struct vcpu_svm *svm)GHCB_VERSION_MIN , sev_enc_bit)); } + +void sev_es_vcpu_load (struct vcpu_svm *svm,int cpu) +{ +struct svm_cpu_data *sd =per_cpu (svm_data, cpu); +struct vmcb_save_area *hostsa; +unsigned int i; + +/* +* As an SEV-ES guest, hardware will restore the host state on VMEXIT, +* of which one step is to perform a VMLOAD. Since hardware does not +* perform a VMSAVE on VMRUN, the host savearea must be updated. +*/ + asm volatile(__ex("vmsave") : : "a" (__sme_page_pa(sd->save_area)) : "memory"); + +/* +* Certain MSRs are restored on VMEXIT, only save ones that aren't +* saved via the vmsave above. + */ + for (i = 0; i < NR_HOST_SAVE_USER_MSRS; i++) { +if (host_save_user_msrs[i].sev_es_restored) +continue ; + +rdmsrl (host_save_user_msrs[i].index, svm->host_user_msrs[i]); + } + +/* XCR0 is restored on VMEXIT, save the current host value */ + hostsa = (struct vmcb_save_area *)(page_address (sd->save_area) +0x400 ); + hostsa->xcr0 =xgetbv (XCR_XFEATURE_ENABLED_MASK ); +} + +void sev_es_vcpu_put (struct vcpu_svm *svm) +{ +unsigned int i; + +/* +* Certain MSRs are restored on VMEXIT and were saved with vmsave in +* sev_es_vcpu_load() above. Only restore ones that weren't. +*/ +for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++) { +if (host_save_user_msrs[i].sev_es_restored) +continue ; + +wrmsrl (host_save_user_msrs[i].index, svm->host_user_msrs[i]); + } +}
arch/x86/kvm/svm/svm.c +23 -13
@@ -1340,15 +1340,20 @@ static void svm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)vmcb_mark_all_dirty (svm->vmcb); } +if (sev_es_guest (svm->vcpu.kvm)) { +sev_es_vcpu_load (svm, cpu); + }else {#ifdef CONFIG_X86_64 -rdmsrl (MSR_GS_BASE ,to_svm (vcpu)->host.gs_base); +rdmsrl (MSR_GS_BASE ,to_svm (vcpu)->host.gs_base);#endif -savesegment (fs, svm->host.fs); -savesegment (gs, svm->host.gs); - svm->host.ldt =kvm_read_ldt (); +savesegment (fs, svm->host.fs); +savesegment (gs, svm->host.gs); + svm->host.ldt =kvm_read_ldt (); -for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++) -rdmsrl (host_save_user_msrs[i], svm->host_user_msrs[i]); +for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++) +rdmsrl (host_save_user_msrs[i].index, + svm->host_user_msrs[i]); + }if (static_cpu_has (X86_FEATURE_TSCRATEMSR )) {u64 tsc_ratio = vcpu->arch.tsc_scaling_ratio;
@@ -1376,19 +1381,24 @@ static void svm_vcpu_put(struct kvm_vcpu *vcpu)avic_vcpu_put (vcpu); ++vcpu->stat.host_state_reload; -kvm_load_ldt (svm->host.ldt); +if (sev_es_guest (svm->vcpu.kvm)) { +sev_es_vcpu_put (svm); + }else { +kvm_load_ldt (svm->host.ldt);#ifdef CONFIG_X86_64 -loadsegment (fs, svm->host.fs); -wrmsrl (MSR_KERNEL_GS_BASE , current->thread.gsbase); -load_gs_index (svm->host.gs); +loadsegment (fs, svm->host.fs); +wrmsrl (MSR_KERNEL_GS_BASE , current->thread.gsbase); +load_gs_index (svm->host.gs);#else #ifdef CONFIG_X86_32_LAZY_GS -loadsegment (gs, svm->host.gs); +loadsegment (gs, svm->host.gs);#endif #endif -for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++) -wrmsrl (host_save_user_msrs[i], svm->host_user_msrs[i]); +for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++) +wrmsrl (host_save_user_msrs[i].index, + svm->host_user_msrs[i]); + } }static unsigned long svm_get_rflags (struct kvm_vcpu *vcpu)
arch/x86/kvm/svm/svm.h +16 -6
@@ -23,15 +23,23 @@#define __sme_page_pa (x) __sme_set(page_to_pfn(x) << PAGE_SHIFT) -static const u32 host_save_user_msrs[] = { +static const struct svm_host_save_msrs { +u32 index;/* Index of the MSR */ +bool sev_es_restored;/* True if MSR is restored on SEV-ES VMEXIT */ +} host_save_user_msrs[] = { #ifdef CONFIG_X86_64 -MSR_STAR ,MSR_LSTAR ,MSR_CSTAR ,MSR_SYSCALL_MASK ,MSR_KERNEL_GS_BASE , -MSR_FS_BASE , + { .index =MSR_STAR , .sev_es_restored = true }, + { .index =MSR_LSTAR , .sev_es_restored = true }, + { .index =MSR_CSTAR , .sev_es_restored = true }, + { .index =MSR_SYSCALL_MASK , .sev_es_restored = true }, + { .index =MSR_KERNEL_GS_BASE , .sev_es_restored = true }, + { .index =MSR_FS_BASE , .sev_es_restored = true }, #endif -MSR_IA32_SYSENTER_CS ,MSR_IA32_SYSENTER_ESP ,MSR_IA32_SYSENTER_EIP , -MSR_TSC_AUX , + { .index =MSR_IA32_SYSENTER_CS , .sev_es_restored = true }, + { .index =MSR_IA32_SYSENTER_ESP , .sev_es_restored = true }, + { .index =MSR_IA32_SYSENTER_EIP , .sev_es_restored = true }, + { .index =MSR_TSC_AUX , .sev_es_restored = false }, }; -#define NR_HOST_SAVE_USER_MSRS ARRAY_SIZE(host_save_user_msrs)#define MSRPM_OFFSETS 16
@@ -573,6 +581,8 @@ int sev_es_string_io(struct vcpu_svm *svm, int size, unsigned int port, int in);void sev_vcpu_deliver_sipi_vector (struct kvm_vcpu *vcpu,u8 vector);void sev_es_init_vmcb (struct vcpu_svm *svm);void sev_es_create_vcpu (struct vcpu_svm *svm); +void sev_es_vcpu_load (struct vcpu_svm *svm,int cpu); +void sev_es_vcpu_put (struct vcpu_svm *svm);/* VMSA Accessor functions */
Signature · 1 line
2.28.0
RFC PATCH 34/35 KVM: SVM: Provide an updated VMRUN invocation for SEV-ES guests
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
The guest vCPU register state of an SEV-ES guest will be restored on VMRUN and save saved on VMEXIT. Therefore, there is no need to restore the guest registers directly and through VMLOAD before VMRUN and no need to save the guest registers directly and through VMSAVE on VMEXIT.
Update the svm_vcpu_run() function to skip register state saving and restoring and provide an alternative function for running an SEV-ES guest in vmenter.S
--- arch/x86/kvm/svm/svm.c | 36 +++++++++++++++++---------- arch/x86/kvm/svm/svm.h | 5 ++++ arch/x86/kvm/svm/vmenter.S | 50 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 78 insertions(+), 13 deletions(-)
arch/x86/kvm/svm/svm.c +23 -13
@@ -3640,16 +3640,20 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu,guest_enter_irqoff ();lockdep_hardirqs_on (CALLER_ADDR0 ); -__svm_vcpu_run (svm->vmcb_pa, (unsigned long *)&svm->vcpu.arch.regs); +if (sev_es_guest (svm->vcpu.kvm)) { +__svm_sev_es_vcpu_run (svm->vmcb_pa); + }else { +__svm_vcpu_run (svm->vmcb_pa, (unsigned long *)&svm->vcpu.arch.regs);#ifdef CONFIG_X86_64 -native_wrmsrl (MSR_GS_BASE , svm->host.gs_base); +native_wrmsrl (MSR_GS_BASE , svm->host.gs_base);#else -loadsegment (fs, svm->host.fs); +loadsegment (fs, svm->host.fs);#ifndef CONFIG_X86_32_LAZY_GS -loadsegment (gs, svm->host.gs); +loadsegment (gs, svm->host.gs);#endif #endif + }/* * VMEXIT disables interrupts (host state), but tracing and lockdep
@@ -3676,9 +3680,11 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)fastpath_t exit_fastpath;struct vcpu_svm *svm =to_svm (vcpu); -svm_rax_write (svm, vcpu->arch.regs[VCPU_REGS_RAX ]); -svm_rsp_write (svm, vcpu->arch.regs[VCPU_REGS_RSP ]); -svm_rip_write (svm, vcpu->arch.regs[VCPU_REGS_RIP ]); +if (!sev_es_guest (svm->vcpu.kvm)) { +svm_rax_write (svm, vcpu->arch.regs[VCPU_REGS_RAX ]); +svm_rsp_write (svm, vcpu->arch.regs[VCPU_REGS_RSP ]); +svm_rip_write (svm, vcpu->arch.regs[VCPU_REGS_RIP ]); + }/* * Disable singlestep if we're injecting an interrupt/exception.
@@ -3700,7 +3706,8 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)sync_lapic_to_cr8 (vcpu); -svm_cr2_write (svm, vcpu->arch.cr2); +if (!sev_es_guest (svm->vcpu.kvm)) +svm_cr2_write (svm, vcpu->arch.cr2);/* * Run with all-zero DR6 unless needed, so that we can get the exact cause
@@ -3748,14 +3755,17 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)if (unlikely (!msr_write_intercepted (vcpu,MSR_IA32_SPEC_CTRL ))) svm->spec_ctrl =native_read_msr (MSR_IA32_SPEC_CTRL ); -reload_tss (vcpu); +if (!sev_es_guest (svm->vcpu.kvm)) +reload_tss (vcpu);x86_spec_ctrl_restore_host (svm->spec_ctrl, svm->virt_spec_ctrl); - vcpu->arch.cr2 =svm_cr2_read (svm); - vcpu->arch.regs[VCPU_REGS_RAX ] =svm_rax_read (svm); - vcpu->arch.regs[VCPU_REGS_RSP ] =svm_rsp_read (svm); - vcpu->arch.regs[VCPU_REGS_RIP ] =svm_rip_read (svm); +if (!sev_es_guest (svm->vcpu.kvm)) { + vcpu->arch.cr2 =svm_cr2_read (svm); + vcpu->arch.regs[VCPU_REGS_RAX ] =svm_rax_read (svm); + vcpu->arch.regs[VCPU_REGS_RSP ] =svm_rsp_read (svm); + vcpu->arch.regs[VCPU_REGS_RIP ] =svm_rip_read (svm); + }if (unlikely (svm->vmcb->control.exit_code ==SVM_EXIT_NMI ))kvm_before_interrupt (&svm->vcpu);
arch/x86/kvm/svm/svm.h +5 -0
@@ -584,6 +584,11 @@ void sev_es_create_vcpu(struct vcpu_svm *svm);void sev_es_vcpu_load (struct vcpu_svm *svm,int cpu);void sev_es_vcpu_put (struct vcpu_svm *svm); +/* vmenter.S */ + +void __svm_sev_es_vcpu_run (unsigned long vmcb_pa); +void __svm_vcpu_run (unsigned long vmcb_pa,unsigned long *regs); +/* VMSA Accessor functions */ static inline struct vmcb_save_area *get_vmsa (struct vcpu_svm *svm)
arch/x86/kvm/svm/vmenter.S +50 -0
53 lines not on this page · raw message
Signature · 1 line
2.28.0
RFC PATCH 35/35 KVM: SVM: Provide support to launch and run an SEV-ES guest
In reply to Tom LendackyFrom: Tom Lendacky <[email protected]>
An SEV-ES guest requires some additional steps to be launched as compared to an SEV guest:
- Implement additional VMCB initialization requirements for SEV-ES.
- Update MSR_VM_HSAVE_PA to include the encryption bit if SME is active.
- Add additional MSRs to the list of direct access MSRs so that the intercepts can be disabled.
- Measure all vCPUs using the LAUNCH_UPDATE_VMSA SEV command after all calls to LAUNCH_UPDATE_DATA have been performed but before the call to LAUNCH_MEASURE has been performed.
- Use VMSAVE to save host information that is not saved on VMRUN but is restored on VMEXIT.
- Modify the VMRUN path to eliminate guest register state restoring and saving.
At this point the guest can be run. However, the run sequence is different for an SEV-ES guest compared to a normal or even an SEV guest. Because the guest register state is encrypted, it is all saved as part of VMRUN/VMEXIT and does not require restoring before or saving after a VMRUN instruction. As a result, all that is required to perform a VMRUN is to save the RBP and RAX registers, issue the VMRUN and then restore RAX and RBP.
Additionally, certain state is automatically saved and restored with an SEV-ES VMRUN. As a result certain register states are not required to be restored upon VMEXIT (e.g. FS, GS, etc.), so only do that if the guest is not an SEV-ES guest.
--- arch/x86/kvm/svm/sev.c | 60 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+)
arch/x86/kvm/svm/sev.c +60 -0
@@ -201,6 +201,16 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_es_guest_init (struct kvm *kvm,struct kvm_sev_cmd *argp) +{ +if (!sev_es) +return -ENOTTY ; + +to_kvm_svm (kvm)->sev_info.es_active = true; + +return sev_guest_init (kvm, argp); +} +static int sev_bind_asid (struct kvm *kvm,unsigned int handle,int *error) {struct sev_data_activate *data;
@@ -501,6 +511,50 @@ static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_launch_update_vmsa (struct kvm *kvm,struct kvm_sev_cmd *argp) +{ +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; +struct sev_data_launch_update_vmsa *vmsa; +int i, ret; + +if (!sev_es_guest (kvm)) +return -ENOTTY ; + + vmsa =kzalloc (sizeof (*vmsa),GFP_KERNEL ); +if (!vmsa) +return -ENOMEM ; + +for (i =0 ; i < kvm->created_vcpus; i++) { +struct vcpu_svm *svm =to_svm (kvm->vcpus[i]); +struct vmcb_save_area *save =get_vmsa (svm); + +/* Set XCR0 before encrypting */ + save->xcr0 = svm->vcpu.arch.xcr0; + +/* +* The LAUNCH_UPDATE_VMSA command will perform in-place +* encryption of the VMSA memory content (i.e it will write +* the same memory region with the guest's key), so invalidate +* it first. +*/ +clflush_cache_range (svm->vmsa,PAGE_SIZE ); + + vmsa->handle = sev->handle; + vmsa->address =__sme_pa (svm->vmsa); + vmsa->len =PAGE_SIZE ; + ret =sev_issue_cmd (kvm,SEV_CMD_LAUNCH_UPDATE_VMSA , vmsa, + &argp->error); +if (ret) + goto e_free; + + svm->vcpu.arch.vmsa_encrypted = true; + } + +e_free: +kfree (vmsa); +return ret; +} +static int sev_launch_measure (struct kvm *kvm,struct kvm_sev_cmd *argp) {void __user *measure = (void __user *)(uintptr_t )argp->data;
@@ -948,12 +1002,18 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)case KVM_SEV_INIT : r =sev_guest_init (kvm, &sev_cmd);break ; +case KVM_SEV_ES_INIT : + r =sev_es_guest_init (kvm, &sev_cmd); +break ;case KVM_SEV_LAUNCH_START : r =sev_launch_start (kvm, &sev_cmd);break ;case KVM_SEV_LAUNCH_UPDATE_DATA : r =sev_launch_update_data (kvm, &sev_cmd);break ; +case KVM_SEV_LAUNCH_UPDATE_VMSA : + r =sev_launch_update_vmsa (kvm, &sev_cmd); +break ;case KVM_SEV_LAUNCH_MEASURE : r =sev_launch_measure (kvm, &sev_cmd);break ;
Signature · 1 line
2.28.0
Quoted 3 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
This patch series provides support for running SEV-ES guests under KVM.
From the x86/VMX side of things, the GPR hooks are the only changes that I strongly dislike.
For the vmsa_encrypted flag and related things like allow_debug(), I'd really like to aim for a common implementation between SEV-ES and TDX[*] from the get go, within reason obviously. From a code perspective, I don't think it will be too onerous as the basic tenets are quite similar, e.g. guest state is off limits, FPU state is autoswitched, etc..., but I suspect (or maybe worry?) that there are enough minor differences that we'll want a more generic way of marking ioctls() as disallowed to avoid having one-off checks all over the place.
That being said, it may also be that there are some ioctls() that should be disallowed under SEV-ES, but aren't in this series. E.g. I assume kvm_vcpu_ioctl_smi() should be rejected as KVM can't do the necessary emulation (I assume this applies to vanilla SEV as well?).
One thought to try and reconcile the differences between SEV-ES and TDX would be expicitly list which ioctls() are and aren't supported and go from there? E.g. if there is 95% overlap than we probably don't need to get fancy with generic allow/deny.
Given that we don't yet have publicly available KVM code for TDX, what if I generate and post a list of ioctls() that are denied by either SEV-ES or TDX, organized by the denier(s)? Then for the ioctls() that are denied by one and not the other, we add a brief explanation of why it's denied?
If that sounds ok, I'll get the list and the TDX side of things posted tomorrow.
Thanks!
[*] https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
Quoted 17 lines · Sean Christopherson
Quoted 3 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
This patch series provides support for running SEV-ES guests under KVM.
From the x86/VMX side of things, the GPR hooks are the only changes that I strongly dislike.
For the
vmsa_encryptedflag and related things likeallow_debug(), I'd really like to aim for a common implementation between SEV-ES and TDX[*] from the get go, within reason obviously. From a code perspective, I don't think it will be too onerous as the basic tenets are quite similar, e.g. guest state is off limits, FPU state is autoswitched, etc..., but I suspect (or maybe worry?) that there are enough minor differences that we'll want a more generic way of markingioctls()as disallowed to avoid having one-off checks all over the place.
Quoted 4 lines
That being said, it may also be that there are some
ioctls()that should be disallowed under SEV-ES, but aren't in this series. E.g. I assumekvm_vcpu_ioctl_smi()should be rejected as KVM can't do the necessary emulation (I assume this applies to vanilla SEV as well?).
Right, SMM isn't currently supported under SEV-ES. SEV does support SMM, though, since the register state can be altered to change over to the SMM register state. So the SMI ioctl() is ok for SEV.
Quoted 6 lines
One thought to try and reconcile the differences between SEV-ES and TDX would be expicitly list which
ioctls()are and aren't supported and go from there? E.g. if there is 95% overlap than we probably don't need to get fancy with generic allow/deny.
Quoted 7 lines
Given that we don't yet have publicly available KVM code for TDX, what if I generate and post a list of
ioctls()that are denied by either SEV-ES or TDX, organized by the denier(s)? Then for theioctls()that are denied by one and not the other, we add a brief explanation of why it's denied?If that sounds ok, I'll get the list and the TDX side of things posted tomorrow.
That sounds good.
Thanks,
Tom
Quoted 18 lines · Tom Lendacky
Quoted 17 lines · Sean Christopherson
Quoted 3 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
This patch series provides support for running SEV-ES guests under KVM.
From the x86/VMX side of things, the GPR hooks are the only changes that I strongly dislike.
For the
vmsa_encryptedflag and related things likeallow_debug(), I'd really like to aim for a common implementation between SEV-ES and TDX[*] from the get go, within reason obviously. From a code perspective, I don't think it will be too onerous as the basic tenets are quite similar, e.g. guest state is off limits, FPU state is autoswitched, etc..., but I suspect (or maybe worry?) that there are enough minor differences that we'll want a more generic way of markingioctls()as disallowed to avoid having one-off checks all over the place.
Quoted 8 lines
Quoted 4 lines
That being said, it may also be that there are some
ioctls()that should be disallowed under SEV-ES, but aren't in this series. E.g. I assumekvm_vcpu_ioctl_smi()should be rejected as KVM can't do the necessary emulation (I assume this applies to vanilla SEV as well?).Right, SMM isn't currently supported under SEV-ES. SEV does support SMM, though, since the register state can be altered to change over to the SMM register state. So the SMI
ioctl()is ok for SEV.
But isn't guest memory inaccessible for SEV? E.g. how does KVM emulate the save/restore to/from SMRAM?
Quoted 22 lines · Sean Christopherson
Quoted 21 lines · Tom Lendacky
Quoted 20 lines · Sean Christopherson
Quoted 3 lines · Tom Lendacky
From: Tom Lendacky <[email protected]>
This patch series provides support for running SEV-ES guests under KVM.
From the x86/VMX side of things, the GPR hooks are the only changes that I strongly dislike.
For the
vmsa_encryptedflag and related things likeallow_debug(), I'd really like to aim for a common implementation between SEV-ES and TDX[*] from the get go, within reason obviously. From a code perspective, I don't think it will be too onerous as the basic tenets are quite similar, e.g. guest state is off limits, FPU state is autoswitched, etc..., but I suspect (or maybe worry?) that there are enough minor differences that we'll want a more generic way of markingioctls()as disallowed to avoid having one-off checks all over the place.That being said, it may also be that there are some
ioctls()that should be disallowed under SEV-ES, but aren't in this series. E.g. I assumekvm_vcpu_ioctl_smi()should be rejected as KVM can't do the necessary emulation (I assume this applies to vanilla SEV as well?).
Quoted 5 lines
Quoted 3 lines
Right, SMM isn't currently supported under SEV-ES. SEV does support SMM, though, since the register state can be altered to change over to the SMM register state. So the SMI
ioctl()is ok for SEV.But isn't guest memory inaccessible for SEV? E.g. how does KVM emulate the save/restore to/from SMRAM?
In SEV, to support the SMM, the guest BIOS (Ovmf) maps the SMM state save area as unencrypted. This allows the KVM to access the SMM state saved area as unencrypted. SVM also provides intercepts for the RSM, so KVM does not need to fetch and decode the instruction bytes to know whether the VMEXIT was due to exiting from the SMM mode.
Quoted 10 lines · Tom Lendacky
Quoted 7 lines · Sean Christopherson
Given that we don't yet have publicly available KVM code for TDX, what if I generate and post a list of
ioctls()that are denied by either SEV-ES or TDX, organized by the denier(s)? Then for theioctls()that are denied by one and not the other, we add a brief explanation of why it's denied?If that sounds ok, I'll get the list and the TDX side of things posted tomorrow.
That sounds good.
TDX completely blocks the following ioctl()s:
kvm_vcpu_ioctl_interrupt
kvm_vcpu_ioctl_smi
kvm_vcpu_ioctl_x86_setup_mce
kvm_vcpu_ioctl_x86_set_mce
kvm_vcpu_ioctl_x86_get_debugregs
kvm_vcpu_ioctl_x86_set_debugregs
kvm_vcpu_ioctl_x86_get_xsave
kvm_vcpu_ioctl_x86_set_xsave
kvm_vcpu_ioctl_x86_get_xcrs
kvm_vcpu_ioctl_x86_set_xcrs
kvm_arch_vcpu_ioctl_get_regs
kvm_arch_vcpu_ioctl_set_regs
kvm_arch_vcpu_ioctl_get_sregs
kvm_arch_vcpu_ioctl_set_sregs
kvm_arch_vcpu_ioctl_set_guest_debug
kvm_arch_vcpu_ioctl_get_fpu
kvm_arch_vcpu_ioctl_set_fpu
Looking through the code, I think kvm_arch_vcpu_ioctl_get_mpstate() and kvm_arch_vcpu_ioctl_set_mpstate() should also be disallowed, we just haven't actually done so.
There are also two helper functions that are "blocked". dm_request_for_irq_injection() returns false if guest_state_protected, and post_kvm_run_save() shoves dummy state.
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
kvm_vcpu_ioctl_x86_get_vcpu_events:
if (!vcpu->kvm->arch.guest_state_protected)
events->interrupt.shadow = kvm_x86_ops.get_interrupt_shadow(vcpu); kvm_arch_vcpu_put:
if (vcpu->preempted && !vcpu->kvm->arch.guest_state_protected)
vcpu->arch.preempted_in_kernel = !kvm_x86_ops.get_cpl(vcpu); kvm_vcpu_ioctl_x86_set_vcpu_events:
u32 allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING | KVM_VCPUEVENT_VALID_SIPI_VECTOR | KVM_VCPUEVENT_VALID_SHADOW | KVM_VCPUEVENT_VALID_SMM | KVM_VCPUEVENT_VALID_PAYLOAD; if (vcpu->kvm->arch.guest_state_protected) allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING;
kvm_arch_vcpu_ioctl_run:
if (vcpu->kvm->arch.guest_state_protected) kvm_sync_valid_fields = KVM_SYNC_X86_EVENTS; else kvm_sync_valid_fields = KVM_SYNC_X86_VALID_FIELDS;
In addition to the more generic guest_state_protected, we also (obviously tentatively) have a few other flags to deal with aspects of TDX that I'm fairly certain don't apply to SEV-ES:
tsc_immutable - KVM doesn't have write access to the TSC offset of the
guest.
eoi_intercept_unsupported - KVM can't intercept EOIs (doesn't have access
to EOI bitmaps) and so can't support level triggered interrupts, at least not without extra pain.
readonly_mem_unsupported - Secure EPT (analagous to SNP) requires RWX
permissions for all private/encrypted memory. S-EPT isn't optional, so we get the joy of adding this right off the bat...
Apologies, Sean.
I thought I had replied to this but found it instead in my drafts folder...
I've taken much of your feedback and incorporated that into the next version of the patches that I submitted and updated this response based on that, too.
Quoted 7 lines · Sean Christopherson
Quoted 6 lines · Tom Lendacky
Quoted 5 lines · Sean Christopherson
Given that we don't yet have publicly available KVM code for TDX, what if I generate and post a list of
ioctls()that are denied by either SEV-ES or TDX, organized by the denier(s)? Then for theioctls()that are denied by one and not the other, we add a brief explanation of why it's denied?
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
If that sounds ok, I'll get the list and the TDX side of things posted tomorrow.
That sounds good.
TDX completely blocks the following ioctl()s:
SEV-ES doesn't need to completely block these ioctls. SEV-SNP is likely to do more of that. SEV-ES will still allow interrupts to be injected, or registers to be retrieved (which will only contain what was provided in the GHCB exchange), etc.
Quoted 10 lines
kvm_vcpu_ioctl_interrupt
kvm_vcpu_ioctl_smi
kvm_vcpu_ioctl_x86_setup_mce
kvm_vcpu_ioctl_x86_set_mce
kvm_vcpu_ioctl_x86_get_debugregs
kvm_vcpu_ioctl_x86_set_debugregs
kvm_vcpu_ioctl_x86_get_xsave
kvm_vcpu_ioctl_x86_set_xsave
kvm_vcpu_ioctl_x86_get_xcrs
Quoted 8 lines
kvm_vcpu_ioctl_x86_set_xcrs
kvm_arch_vcpu_ioctl_get_regs
kvm_arch_vcpu_ioctl_set_regs
kvm_arch_vcpu_ioctl_get_sregs
kvm_arch_vcpu_ioctl_set_sregs
kvm_arch_vcpu_ioctl_set_guest_debug
kvm_arch_vcpu_ioctl_get_fpu
kvm_arch_vcpu_ioctl_set_fpu
Of the listed ioctls, really the only ones I've updated are:
kvm_vcpu_ioctl_x86_get_xsave
kvm_vcpu_ioctl_x86_set_xsave
kvm_arch_vcpu_ioctl_get_sregs
This allows reading of the tracking value registers
kvm_arch_vcpu_ioctl_set_sregs
This prevents setting of register values
kvm_arch_vcpu_ioctl_set_guest_debug
kvm_arch_vcpu_ioctl_get_fpu
kvm_arch_vcpu_ioctl_set_fpu
Quoted 4 lines
Looking through the code, I think
kvm_arch_vcpu_ioctl_get_mpstate()andkvm_arch_vcpu_ioctl_set_mpstate()should also be disallowed, we just haven't actually done so.
I haven't done anything with these either.
Quoted 4 lines
There are also two helper functions that are "blocked".
dm_request_for_irq_injection()returns false ifguest_state_protected, andpost_kvm_run_save()shoves dummy state.
... and these.
Quoted 6 lines
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
For SEV-ES, we don't have those restrictions.
Quoted 39 lines
kvm_vcpu_ioctl_x86_get_vcpu_events:if (!vcpu->kvm->arch.guest_state_protected) events->interrupt.shadow = kvm_x86_ops.get_interrupt_shadow(vcpu);kvm_arch_vcpu_put: if (vcpu->preempted && !vcpu->kvm->arch.guest_state_protected) vcpu->arch.preempted_in_kernel = !kvm_x86_ops.get_cpl(vcpu);
kvm_vcpu_ioctl_x86_set_vcpu_events:u32 allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING | KVM_VCPUEVENT_VALID_SIPI_VECTOR | KVM_VCPUEVENT_VALID_SHADOW | KVM_VCPUEVENT_VALID_SMM | KVM_VCPUEVENT_VALID_PAYLOAD; if (vcpu->kvm->arch.guest_state_protected) allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING;
kvm_arch_vcpu_ioctl_run:if (vcpu->kvm->arch.guest_state_protected) kvm_sync_valid_fields = KVM_SYNC_X86_EVENTS; else kvm_sync_valid_fields = KVM_SYNC_X86_VALID_FIELDS;In addition to the more generic
guest_state_protected, we also (obviously tentatively) have a few other flags to deal with aspects of TDX that I'm fairly certain don't apply to SEV-ES:
tsc_immutable- KVM doesn't have write access to the TSC offset of the
guest.
eoi_intercept_unsupported- KVM can't intercept EOIs (doesn't have access
to EOI bitmaps) and so can't support level triggered interrupts, at least not without extra pain.
Quoted 4 lines
readonly_mem_unsupported- Secure EPT (analagous to SNP) requires RWX
permissions for all private/encrypted memory. S-EPT isn't optional, so we get the joy of adding this right off the bat...
Yes, most of the above stuff doesn't apply to SEV-ES.
Thanks,
Tom
Quoted 1 line
Paolo Bonzini
maintainerQuoted 10 lines · Sean Christopherson
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
kvm_vcpu_ioctl_x86_get_vcpu_events: if (!vcpu->kvm->arch.guest_state_protected) events->interrupt.shadow = kvm_x86_ops.get_interrupt_shadow(vcpu);
Perhaps an alternative implementation can enter the vCPU with immediate exit until no events are pending, and then return all zeroes?
Paolo
Quoted 9 lines · Paolo Bonzini
Quoted 8 lines · Sean Christopherson
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except
for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
Quoted 7 lines
Quoted 4 lines
kvm_vcpu_ioctl_x86_get_vcpu_events:
if (!vcpu->kvm->arch.guest_state_protected)
events->interrupt.shadow=kvm_x86_ops.get_interrupt_shadow(vcpu);Perhaps an alternative implementation can enter the vCPU with immediate exit until no events are pending, and then return all zeroes?
SEV-SNP has support for restricting injections, but SEV-ES does not. Perhaps a new boolean, guest_restricted_injection, can be used instead of basing it on guest_state_protected.
Thanks,
Tom
Quoted 3 lines
Paolo
Sean Christopherson
maintainerQuoted 15 lines · Tom Lendacky
Quoted 14 lines · Paolo Bonzini
Quoted 12 lines · Sean Christopherson
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except
for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
kvm_vcpu_ioctl_x86_get_vcpu_events:
if (!vcpu->kvm->arch.guest_state_protected)
events->interrupt.shadow=kvm_x86_ops.get_interrupt_shadow(vcpu);
Quoted 6 lines
Quoted 2 lines
Perhaps an alternative implementation can enter the vCPU with immediate exit until no events are pending, and then return all zeroes?
SEV-SNP has support for restricting injections, but SEV-ES does not. Perhaps a new boolean,
guest_restricted_injection, can be used instead of basing it onguest_state_protected.
Ya, that probably makes sense. I suspect the easiest way to resolve these conflicts will be to land the SEV-ES series and then tweak things as needed for TDX. Easiest in the sense that it should be fairly obvious what can be covered by guest_state_protected and what needs a dedicated flag.
Sean Christopherson
maintainerQuoted 8 lines · Paolo Bonzini
Quoted 7 lines · Sean Christopherson
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
Quoted 6 lines
Quoted 3 lines
kvm_vcpu_ioctl_x86_get_vcpu_events: if (!vcpu->kvm->arch.guest_state_protected) events->interrupt.shadow = kvm_x86_ops.get_interrupt_shadow(vcpu);Perhaps an alternative implementation can enter the vCPU with immediate exit until no events are pending, and then return all zeroes?
This can't work. If the guest has STI blocking, e.g. it did STI->TDVMCALL with a valid vIRQ in GUEST_RVI, then events->interrupt.shadow should technically be non-zero to reflect the STI blocking. But, the immediate exit (a hardware IRQ for TDX guests) will cause VM-Exit before the guest can execute any instructions and thus the guest will never clear STI blocking and never consume the pending event. Or there could be a valid vIRQ, but GUEST_RFLAGS.IF=0, in which case KVM would need to run the guest for an indeterminate amount of time to wait for the vIRQ to be consumed.
Tangentially related, I haven't looked through the official external TDX docs, but I suspect that vmcs.GUEST_RVI is listed as inaccessible for production TDs. This will be changed as the VMM needs access to GUEST_RVI to handle STI->TDVMCALL(HLT), otherwise the VMM may incorrectly put the vCPU into a blocked (not runnable) state even though it has a pending wake event.
Paolo Bonzini
maintainerQuoted 12 lines · Sean Christopherson
Quoted 11 lines
Quoted 9 lines
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
kvm_vcpu_ioctl_x86_get_vcpu_events: if (!vcpu->kvm->arch.guest_state_protected) events->interrupt.shadow = kvm_x86_ops.get_interrupt_shadow(vcpu);Perhaps an alternative implementation can enter the vCPU with immediate exit until no events are pending, and then return all zeroes?
Quoted 8 lines
This can't work. If the guest has STI blocking, e.g. it did
STI->TDVMCALLwith a valid vIRQ inGUEST_RVI, thenevents->interrupt.shadowshould technically be non-zero to reflect the STI blocking. But, the immediate exit (a hardware IRQ for TDX guests) will cause VM-Exit before the guest can execute any instructions and thus the guest will never clear STI blocking and never consume the pending event. Or there could be a valid vIRQ, but GUEST_RFLAGS.IF=0, in which case KVM would need to run the guest for an indeterminate amount of time to wait for the vIRQ to be consumed.
Delayed interrupts are fine, since they are injected according to RVI and the posted interrupt descriptor. I'm thinking more of events (exceptions and interrupts) that caused an EPT violation exit and were recorded in the IDT-vectored info field.
Paolo
Quoted 6 lines
Tangentially related, I haven't looked through the official external TDX docs, but I suspect that vmcs.GUEST_RVI is listed as inaccessible for production TDs. This will be changed as the VMM needs access to
GUEST_RVIto handleSTI->TDVMCALL(HLT), otherwise the VMM may incorrectly put the vCPU into a blocked (not runnable) state even though it has a pending wake event.
Sean Christopherson
maintainer+Isaku and Xiaoyao
Quoted 22 lines · Paolo Bonzini
Quoted 12 lines · Sean Christopherson
Quoted 11 lines
Quoted 9 lines
TDX also selectively blocks/skips portions of other ioctl()s so that the TDX code itself can yell loudly if e.g. .get_cpl() is invoked. The event injection restrictions are due to direct injection not being allowed (except for NMIs); all IRQs have to be routed through APICv (posted interrupts) and exception injection is completely disallowed.
kvm_vcpu_ioctl_x86_get_vcpu_events: if (!vcpu->kvm->arch.guest_state_protected) events->interrupt.shadow = kvm_x86_ops.get_interrupt_shadow(vcpu);Perhaps an alternative implementation can enter the vCPU with immediate exit until no events are pending, and then return all zeroes?
Quoted 8 lines
This can't work. If the guest has STI blocking, e.g. it did
STI->TDVMCALLwith a valid vIRQ inGUEST_RVI, thenevents->interrupt.shadowshould technically be non-zero to reflect the STI blocking. But, the immediate exit (a hardware IRQ for TDX guests) will cause VM-Exit before the guest can execute any instructions and thus the guest will never clear STI blocking and never consume the pending event. Or there could be a valid vIRQ, but GUEST_RFLAGS.IF=0, in which case KVM would need to run the guest for an indeterminate amount of time to wait for the vIRQ to be consumed.
Quoted 4 lines
Delayed interrupts are fine, since they are injected according to RVI and the posted interrupt descriptor. I'm thinking more of events (exceptions and interrupts) that caused an EPT violation exit and were recorded in the IDT-vectored info field.
Ah. As is, I don't believe KVM has access to this information. TDX-Module handles the actual EPT violation, as well as event reinjection. The EPT violation reported by SEAMRET is synthesized, and IIRC the IDT-vectoring field is not readable.
Regardless, is there an actual a problem with having a "pending" exception that isn't reported to userspace? Obviously the info needs to be migrated, but that will be taken care of by virtue of migrating the VMCS.
Paolo Bonzini
maintainerQuoted 4 lines · Sean Christopherson
Quoted 4 lines
Delayed interrupts are fine, since they are injected according to RVI and the posted interrupt descriptor. I'm thinking more of events (exceptions and interrupts) that caused an EPT violation exit and were recorded in the IDT-vectored info field.
Quoted 8 lines
Ah. As is, I don't believe KVM has access to this information. TDX-Module handles the actual EPT violation, as well as event reinjection. The EPT violation reported by SEAMRET is synthesized, and IIRC the IDT-vectoring field is not readable.
Regardless, is there an actual a problem with having a "pending" exception that isn't reported to userspace? Obviously the info needs to be migrated, but that will be taken care of by virtue of migrating the VMCS.
No problem, I suppose we would just have to get used to not being able to look into the state of migrated VMs.
Paolo