KVM: SVM: Add Page Modification Logging (PML) support
v6 cover KVM: SVM: Add Page Modification Logging (PML) support
This series implements Page Modification Logging (PML) for guests on AMD processors, bringing hardware-assisted dirty logging support. PML is designed to track guest modified memory pages. PML enables the hypervisor to identify which pages in a guest's memory have been modified since the last checkpoint or during live migration.
The PML feature uses two new VMCB fields (PML_ADDR and PML_INDEX) and generates a VMEXIT when the 4KB log buffer becomes full.
The feature is enabled by default when hardware support is detected and can be disabled via the 'pml' module parameter.
Changelog:
v6:
* Collect RoBs
* Rebase on latest kvm/master
* Dropped SEV PML selftest (Kai Huang)
* Added guard(svm_vmcb01) helpers to temporarily switch to vmcb01
* Use guard(svm_vmcb01) to ensure vmcb01 is targeted during PML updates when L2 is active, similar to VMX. Dropped patch moving nested CPU dirty logging to common code; upstream now updates vmcs01 on-demand via temporary VMCS switch instead of deferring updates.
v5: https://lore.kernel.org/kvm/[email protected]/
* Rebased on latest kvm/next
* Use EXPORT_SYMBOL_FOR_KVM_INTERNAL (Kai Huang)
* Use cpu_dirty_log_size instead of enable_pml for PML checks, with this patch moving enable_pml to common code is not needed (Kai Huang)
* Added SEV PML self test
Nikunj A Dadhania (7):
KVM: x86: Carve out PML flush routine
KVM: x86: Move PML page to common vcpu arch structure
KVM: VMX: Use cpu_dirty_log_size instead of enable_pml for PML checks x86/cpufeatures: Add Page modification logging
KVM: SVM: Use BIT_ULL for 64-bit nested_ctl bit definitions
KVM: nSVM: Add helpers to temporarily switch to vmcb01
KVM: SVM: Add Page modification logging support
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/kvm_host.h | 2 + arch/x86/include/asm/svm.h | 12 ++- arch/x86/include/uapi/asm/svm.h | 2 + arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kvm/svm/nested.c | 13 ++- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 125 ++++++++++++++++++++++++++++- arch/x86/kvm/svm/svm.h | 3 + arch/x86/kvm/vmx/vmx.c | 50 ++++-------- arch/x86/kvm/vmx/vmx.h | 7 -- arch/x86/kvm/x86.c | 31 +++++++ arch/x86/kvm/x86.h | 8 ++ 13 files changed, 206 insertions(+), 51 deletions(-)
base-commit: df83746075778958954aa0460cca55f4b3fc9c02
Signature · 1 line
2.48.1
PATCH v6 1/7 KVM: x86: Carve out PML flush routine
Move the PML (Page Modification Logging) buffer flushing logic from VMX-specific code to common x86 KVM code to enable reuse by SVM and avoid code duplication.
The AMD SVM PML implementations share the same behavior as VMX PML:
1) The PML buffer is a 4K page with 512 entries
2) Hardware records dirty GPAs in reverse order (from index 511 to 0)
3) Hardware clears bits 11:0 when recording GPAs
No functional change intended for VMX, except tone down the WARN_ON() to WARN_ON_ONCE() for the page alignment check. If hardware exhibits this behavior once, it's likely to occur repeatedly, so use WARN_ON_ONCE() to avoid log flooding while still capturing the unexpected condition.
The refactoring prepares for SVM to leverage the same PML flushing implementation.
--- arch/x86/kvm/vmx/vmx.c | 26 ++------------------------ arch/x86/kvm/vmx/vmx.h | 5 ----- arch/x86/kvm/x86.c | 31 +++++++++++++++++++++++++++++++ arch/x86/kvm/x86.h | 8 ++++++++ 4 files changed, 41 insertions(+), 29 deletions(-)
arch/x86/kvm/vmx/vmx.c +2 -24
@@ -6502,37 +6502,15 @@ static void vmx_destroy_pml_buffer(struct vcpu_vmx *vmx)static void vmx_flush_pml_buffer (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -u16 pml_idx, pml_tail_index; -u64 *pml_buf; -int i; +u16 pml_idx; pml_idx =vmcs_read16 (GUEST_PML_INDEX );/* Do nothing if PML buffer is empty */ if (pml_idx ==PML_HEAD_INDEX )return ; -/* -* PML index always points to the next available PML buffer entity -* unless PML log has just overflowed. -*/ - pml_tail_index = (pml_idx >=PML_LOG_NR_ENTRIES ) ?0 : pml_idx +1 ; -/* -* PML log is written backwards: the CPU first writes the entry 511 -* then the entry 510, and so on. -* -* Read the entries in the same order they were written, to ensure that -* the dirty ring is filled in the same order the CPU wrote them. -*/ - pml_buf =page_address (vmx->pml_pg); - -for (i =PML_HEAD_INDEX ; i >= pml_tail_index; i--) { -u64 gpa; - - gpa = pml_buf[i]; -WARN_ON (gpa & (PAGE_SIZE -1 )); -kvm_vcpu_mark_page_dirty (vcpu, gpa >>PAGE_SHIFT ); - } +kvm_flush_pml_buffer (vcpu, vmx->pml_pg, pml_idx);/* reset PML index */ vmcs_write16 (GUEST_PML_INDEX ,PML_HEAD_INDEX );
arch/x86/kvm/vmx/vmx.h +0 -5
@@ -262,11 +262,6 @@ struct vcpu_vmx {unsigned int ple_window;bool ple_window_dirty; -/* Support for PML */ -#define PML_LOG_NR_ENTRIES 512 -/* PML is written backwards: this is the first entry written by the CPU */ -#define PML_HEAD_INDEX (PML_LOG_NR_ENTRIES-1) -struct page *pml_pg;/* apic deadline value in host tsc */
arch/x86/kvm/x86.c +31 -0
@@ -6720,6 +6720,37 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)kvm_vcpu_kick (vcpu); } +void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,struct page *pml_page,u16 pml_idx) +{ +u16 pml_tail_index; +u64 *pml_buf; +int i; + +/* +* PML index always points to the next available PML buffer entity +* unless PML log has just overflowed. +*/ + pml_tail_index = (pml_idx >=PML_LOG_NR_ENTRIES ) ?0 : pml_idx +1 ; + +/* +* PML log is written backwards: the CPU first writes the entry 511 +* then the entry 510, and so on. +* +* Read the entries in the same order they were written, to ensure that +* the dirty ring is filled in the same order the CPU wrote them. +*/ + pml_buf =page_address (pml_page); + +for (i =PML_HEAD_INDEX ; i >= pml_tail_index; i--) { +u64 gpa; + + gpa = pml_buf[i]; +WARN_ON_ONCE (gpa & (PAGE_SIZE -1 )); +kvm_vcpu_mark_page_dirty (vcpu, gpa >>PAGE_SHIFT ); + } +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_flush_pml_buffer); +int kvm_vm_ioctl_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) {
arch/x86/kvm/x86.h +8 -0
@@ -763,4 +763,12 @@ static inline bool kvm_is_valid_u_s_cet(struct kvm_vcpu *vcpu, u64 data) return true; } + +/* Support for PML */ +#define PML_LOG_NR_ENTRIES 512 +/* PML is written backwards: this is the first entry written by the CPU */ +#define PML_HEAD_INDEX (PML_LOG_NR_ENTRIES-1) + +void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,struct page *pml_pg,u16 pml_idx); +#endif
Signature · 1 line
2.48.1
Sean Christopherson
maintainerRe: [PATCH v6 1/7] KVM: x86: Carve out PML flush routine
arch/x86/kvm/x86.c void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) · line 6723
Quoted 9 lines · Nikunj A Dadhania
arch/x86/kvm/x86.c +1
-0@@ -6720,6 +6720,37 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)kvm_vcpu_kick (vcpu); } +void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,struct page *pml_page,u16 pml_idx)
Swap the order of patches 1 and 2, then you don't need to add @pml_page, only to remove it one patch later.
Re: [PATCH v6 1/7] KVM: x86: Carve out PML flush routine
Quoted 9 lines · Sean Christopherson
Quoted 8 lines · Nikunj A Dadhania
arch/x86/kvm/x86.c +0
-0@@ -6720,6 +6720,37 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)kvm_vcpu_kick (vcpu); }
Quoted 4 lines
arch/x86/kvm/x86.c void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) · line 6723
Quoted 1 line
arch/x86/kvm/x86.c +1
-0+void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,struct page *pml_page,u16 pml_idx)Swap the order of patches 1 and 2, then you don't need to add @pml_page, only to remove it one patch later.
That makes sense—I'll swap the order of patches 1 and 2 and address this in my next revision.
Regards,
Nikunj
PATCH v6 2/7 KVM: x86: Move PML page to common vcpu arch structure
In reply to Nikunj A DadhaniaMove the PML page pointer from VMX-specific vcpu_vmx structure to the common kvm_vcpu_arch structure to enable sharing between VMX and SVM implementations. Only the page pointer is moved to x86 common code while keeping allocation logic vendor-specific, since AMD requires snp_safe_alloc_page() for PML buffer allocation.
Update all VMX references accordingly, and simplify the kvm_flush_pml_buffer() interface by removing the page parameter since it can now access the page directly from the vcpu structure.
No functional change, restructuring to prepare for SVM PML support.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 24 ++++++++++++------------ arch/x86/kvm/vmx/vmx.h | 2 -- arch/x86/kvm/x86.c | 4 ++-- arch/x86/kvm/x86.h | 2 +- 5 files changed, 17 insertions(+), 17 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -873,6 +873,8 @@ struct kvm_vcpu_arch {*/ struct kvm_mmu_memory_cache mmu_external_spt_cache; +struct page *pml_page; +/* * QEMU userspace and the guest each have their own FPU state. * In vcpu_run, we switch between the user and guest FPU contexts.
arch/x86/kvm/vmx/vmx.c +12 -12
@@ -4934,7 +4934,8 @@ int vmx_vcpu_precreate(struct kvm *kvm)static void init_vmcs (struct vcpu_vmx *vmx) { -struct kvm *kvm = vmx->vcpu.kvm; +struct kvm_vcpu *vcpu = &vmx->vcpu; +struct kvm *kvm = vcpu->kvm;struct kvm_vmx *kvm_vmx =to_kvm_vmx (kvm);if (nested)
@@ -5026,7 +5027,7 @@ static void init_vmcs(struct vcpu_vmx *vmx)vmcs_write64 (XSS_EXIT_BITMAP ,VMX_XSS_EXIT_BITMAP );if (enable_pml) { -vmcs_write64 (PML_ADDRESS ,page_to_phys (vmx->pml_pg)); +vmcs_write64 (PML_ADDRESS ,page_to_phys (vcpu->arch.pml_page));vmcs_write16 (GUEST_PML_INDEX ,PML_HEAD_INDEX ); }
@@ -6491,17 +6492,16 @@ void vmx_get_entry_info(struct kvm_vcpu *vcpu, u32 *intr_info, u32 *error_code) *error_code =0 ; } -static void vmx_destroy_pml_buffer (struct vcpu_vmx *vmx) +static void vmx_destroy_pml_buffer (struct kvm_vcpu *vcpu) { -if (vmx->pml_pg) { -__free_page (vmx->pml_pg); - vmx->pml_pg =NULL ; +if (vcpu->arch.pml_page) { +__free_page (vcpu->arch.pml_page); + vcpu->arch.pml_page =NULL ; } }static void vmx_flush_pml_buffer (struct kvm_vcpu *vcpu) { -struct vcpu_vmx *vmx =to_vmx (vcpu);u16 pml_idx; pml_idx =vmcs_read16 (GUEST_PML_INDEX );
@@ -6510,7 +6510,7 @@ static void vmx_flush_pml_buffer(struct kvm_vcpu *vcpu)if (pml_idx ==PML_HEAD_INDEX )return ; -kvm_flush_pml_buffer (vcpu, vmx->pml_pg, pml_idx); +kvm_flush_pml_buffer (vcpu, pml_idx);/* reset PML index */ vmcs_write16 (GUEST_PML_INDEX ,PML_HEAD_INDEX );
@@ -7738,7 +7738,7 @@ void vmx_vcpu_free(struct kvm_vcpu *vcpu)struct vcpu_vmx *vmx =to_vmx (vcpu);if (enable_pml) -vmx_destroy_pml_buffer (vmx); +vmx_destroy_pml_buffer (vcpu);free_vpid (vmx->vpid);nested_vmx_free_vcpu (vcpu);free_loaded_vmcs (vmx->loaded_vmcs);
@@ -7767,8 +7767,8 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)* for the guest), etc. */ if (enable_pml) { - vmx->pml_pg =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); -if (!vmx->pml_pg) + vcpu->arch.pml_page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); +if (!vcpu->arch.pml_page) goto free_vpid; }
@@ -7839,7 +7839,7 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu) free_vmcs:free_loaded_vmcs (vmx->loaded_vmcs); free_pml: -vmx_destroy_pml_buffer (vmx); +vmx_destroy_pml_buffer (vcpu); free_vpid:free_vpid (vmx->vpid);return err;
arch/x86/kvm/vmx/vmx.h +0 -2
@@ -262,8 +262,6 @@ struct vcpu_vmx {unsigned int ple_window;bool ple_window_dirty; -struct page *pml_pg; -/* apic deadline value in host tsc */ u64 hv_deadline_tsc;
arch/x86/kvm/x86.c +2 -2
@@ -6720,7 +6720,7 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)kvm_vcpu_kick (vcpu); } -void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,struct page *pml_page,u16 pml_idx) +void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,u16 pml_idx) {u16 pml_tail_index;u64 *pml_buf;
@@ -6739,7 +6739,7 @@ void kvm_flush_pml_buffer(struct kvm_vcpu *vcpu, struct page *pml_page, u16 pml_* Read the entries in the same order they were written, to ensure that * the dirty ring is filled in the same order the CPU wrote them. */ - pml_buf =page_address (pml_page); + pml_buf =page_address (vcpu->arch.pml_page);for (i =PML_HEAD_INDEX ; i >= pml_tail_index; i--) {u64 gpa;
arch/x86/kvm/x86.h +1 -1
@@ -769,6 +769,6 @@ static inline bool kvm_is_valid_u_s_cet(struct kvm_vcpu *vcpu, u64 data)/* PML is written backwards: this is the first entry written by the CPU */ #define PML_HEAD_INDEX (PML_LOG_NR_ENTRIES-1) -void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,struct page *pml_pg,u16 pml_idx); +void kvm_flush_pml_buffer (struct kvm_vcpu *vcpu,u16 pml_idx);#endif
Signature · 1 line
2.48.1
Sean Christopherson
maintainerRe: [PATCH v6 2/7] KVM: x86: Move PML page to common vcpu arch structure
arch/x86/kvm/vmx/vmx.c int vmx_vcpu_precreate(struct kvm *kvm) · line 4938
Quoted 11 lines · Nikunj A Dadhania
arch/x86/kvm/vmx/vmx.c +2
-1@@ -4934,7 +4934,8 @@ int vmx_vcpu_precreate(struct kvm *kvm)static void init_vmcs (struct vcpu_vmx *vmx) { -struct kvm *kvm = vmx->vcpu.kvm; +struct kvm_vcpu *vcpu = &vmx->vcpu; +struct kvm *kvm = vcpu->kvm;
There are 9+ existing uses of "vmx->vcpu" in init_vmcs(). I'd rather insert a prep patch to pass @vcpu instead of @vmx, do to_vmx() at the top, and switch all existing vmx->vcpu usage.
From: Sean Christopherson <[email protected]>
Date: Fri, 8 May 2026 15:55:55 -0700
Subject: [PATCH] KVM: VMX: Pass @vcpu, not @vmx to init_vmcs()
Pass @vcpu instead of @vmx to init_vmcs(), and switch all of the vmx->vcpu usage to a simple vcpu.
No functional change intended.
--- arch/x86/kvm/vmx/vmx.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-)
arch/x86/kvm/vmx/vmx.c +13 -12
@@ -4823,10 +4823,11 @@ int vmx_vcpu_precreate(struct kvm *kvm)#define VMX_XSS_EXIT_BITMAP 0 -static void init_vmcs (struct vcpu_vmx *vmx) +static void init_vmcs (struct kvm_vcpu *vcpu) { -struct kvm *kvm = vmx->vcpu.kvm; +struct kvm *kvm = vcpu->kvm;struct kvm_vmx *kvm_vmx =to_kvm_vmx (kvm); +struct vcpu_vmx *vmx =to_vmx (vcpu);if (nested)nested_vmx_set_vmcs_shadowing_bitmap ();
@@ -4851,7 +4852,7 @@ static void init_vmcs(struct vcpu_vmx *vmx)if (cpu_has_tertiary_exec_ctrls ())tertiary_exec_controls_set (vmx,vmx_tertiary_exec_control (vmx)); -if (enable_apicv &&lapic_in_kernel (&vmx->vcpu)) { +if (enable_apicv &&lapic_in_kernel (vcpu)) {vmcs_write64 (EOI_EXIT_BITMAP0 ,0 );vmcs_write64 (EOI_EXIT_BITMAP1 ,0 );vmcs_write64 (EOI_EXIT_BITMAP2 ,0 );
@@ -4863,7 +4864,7 @@ static void init_vmcs(struct vcpu_vmx *vmx)vmcs_write64 (POSTED_INTR_DESC_ADDR ,__pa ((&vmx->vt.pi_desc))); } -if (vmx_can_use_ipiv (&vmx->vcpu)) { +if (vmx_can_use_ipiv (vcpu)) {vmcs_write64 (PID_POINTER_TABLE ,__pa (kvm_vmx->pid_table));vmcs_write16 (LAST_PID_POINTER_INDEX , kvm->arch.max_vcpu_ids -1 ); }
@@ -4898,15 +4899,15 @@ static void init_vmcs(struct vcpu_vmx *vmx)vmcs_write64 (VM_ENTRY_MSR_LOAD_ADDR ,__pa (vmx->msr_autoload.guest.val));if (vmcs_config.vmentry_ctrl &VM_ENTRY_LOAD_IA32_PAT ) -vmcs_write64 (GUEST_IA32_PAT , vmx->vcpu.arch.pat); +vmcs_write64 (GUEST_IA32_PAT , vcpu->arch.pat);vm_exit_controls_set (vmx,vmx_get_initial_vmexit_ctrl ());/* 22.2.1, 20.8.1 */ vm_entry_controls_set (vmx,vmx_get_initial_vmentry_ctrl ()); - vmx->vcpu.arch.cr0_guest_owned_bits =vmx_l1_guest_owned_cr0_bits (); -vmcs_writel (CR0_GUEST_HOST_MASK , ~vmx->vcpu.arch.cr0_guest_owned_bits); + vcpu->arch.cr0_guest_owned_bits =vmx_l1_guest_owned_cr0_bits (); +vmcs_writel (CR0_GUEST_HOST_MASK , ~vcpu->arch.cr0_guest_owned_bits);set_cr4_guest_host_mask (vmx);
@@ -4921,7 +4922,7 @@ static void init_vmcs(struct vcpu_vmx *vmx)vmcs_write16 (GUEST_PML_INDEX ,PML_HEAD_INDEX ); } -vmx_write_encls_bitmap (&vmx->vcpu,NULL ); +vmx_write_encls_bitmap (vcpu,NULL );if (vmx_pt_mode_is_host_guest ()) {memset (&vmx->pt_desc,0 ,sizeof (vmx->pt_desc));
@@ -4934,13 +4935,13 @@ static void init_vmcs(struct vcpu_vmx *vmx)vmcs_writel (GUEST_SYSENTER_ESP ,0 );vmcs_writel (GUEST_SYSENTER_EIP ,0 ); -vmx_guest_debugctl_write (&vmx->vcpu,0 ); +vmx_guest_debugctl_write (vcpu,0 );if (cpu_has_vmx_tpr_shadow ()) {vmcs_write64 (VIRTUAL_APIC_PAGE_ADDR ,0 ); -if (cpu_need_tpr_shadow (&vmx->vcpu)) +if (cpu_need_tpr_shadow (vcpu))vmcs_write64 (VIRTUAL_APIC_PAGE_ADDR , -__pa (vmx->vcpu.arch.apic->regs)); +__pa (vcpu->arch.apic->regs));vmcs_write32 (TPR_THRESHOLD ,0 ); }
@@ -4951,7 +4952,7 @@ static void __vmx_vcpu_reset(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -init_vmcs (vmx); +init_vmcs (vcpu);if (nested &&kvm_check_has_quirk (vcpu->kvm,KVM_X86_QUIRK_STUFF_FEATURE_MSRS ))
base-commit: 4c88d35f51436169884ce87bcf26f31e10106e71
Re: [PATCH v6 2/7] KVM: x86: Move PML page to common vcpu arch structure
Quoted 21 lines · Sean Christopherson
arch/x86/kvm/vmx/vmx.c int vmx_vcpu_precreate(struct kvm *kvm) · line 4938
Quoted 11 lines · Nikunj A Dadhania
arch/x86/kvm/vmx/vmx.c +2
-1@@ -4934,7 +4934,8 @@ int vmx_vcpu_precreate(struct kvm *kvm)static void init_vmcs (struct vcpu_vmx *vmx) { -struct kvm *kvm = vmx->vcpu.kvm; +struct kvm_vcpu *vcpu = &vmx->vcpu; +struct kvm *kvm = vcpu->kvm;There are 9+ existing uses of "
vmx->vcpu" ininit_vmcs(). I'd rather insert a prep patch to pass @vcpu instead of @vmx, doto_vmx()at the top, and switch all existingvmx->vcpuusage.From: Sean Christopherson <[email protected]>
Date: Fri, 8 May 2026 15:55:55 -0700
Subject: [PATCH] KVM: VMX: Pass @vcpu, not @vmx toinit_vmcs()
Quoted 6 lines
Pass @vcpu instead of @vmx to
init_vmcs(), and switch all of thevmx->vcpuusage to a simple vcpu.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Ack. I’ll include this as a preparatory patch in my series unless it lands in your tree before I post.
Regards,
Nikunj
PATCH v6 3/7 KVM: VMX: Use cpu_dirty_log_size instead of enable_pml for PML checks
In reply to Nikunj A DadhaniaReplace the enable_pml check with cpu_dirty_log_size in VMX PML code to determine whether PML is enabled on a per-VM basis. The enable_pml module parameter is a global setting that doesn't reflect per-VM capabilities, whereas cpu_dirty_log_size accurately indicates whether a specific VM has PML enabled.
For example, TDX VMs don't yet support PML. Using cpu_dirty_log_size ensures the check correctly reflects this, while enable_pml would incorrectly indicate PML is available.
This also improves consistency with kvm_mmu_update_cpu_dirty_logging(), which already uses cpu_dirty_log_size to determine PML enablement.
--- arch/x86/kvm/vmx/vmx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -8439,7 +8439,7 @@ void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -if (WARN_ON_ONCE (!enable_pml)) +if (WARN_ON_ONCE (!vcpu->kvm->arch.cpu_dirty_log_size))return ;guard (vmx_vmcs01)(vcpu);
Signature · 1 line
2.48.1
PATCH v6 4/7 x86/cpufeatures: Add Page modification logging
In reply to Nikunj A DadhaniaPage modification logging(PML) is a hardware feature designed to track guest modified memory pages. PML enables the hypervisor to identify which pages in a guest's memory have been changed since the last checkpoint or during live migration.
The PML feature is advertised via CPUID leaf 0x8000000A ECX[4] bit.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -228,6 +228,7 @@#define X86_FEATURE_PVUNLOCK ( 8*32+20)/* PV unlock function */ #define X86_FEATURE_VCPUPREEMPT ( 8*32+21)/* PV vcpu_is_preempted function */ #define X86_FEATURE_TDX_GUEST ( 8*32+22)/* "tdx_guest" Intel Trust Domain Extensions Guest */ +#define X86_FEATURE_PML ( 8*32+23)/* AMD Page Modification logging */ /* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */ #define X86_FEATURE_FSGSBASE ( 9*32+ 0)/* "fsgsbase" RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/
arch/x86/kernel/cpu/scattered.c +1 -0
@@ -53,6 +53,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_AMD_FAST_CPPC, CPUID_EDX, 15, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, + { X86_FEATURE_PML, CPUID_ECX, 4, 0x8000000a, 0 }, { X86_FEATURE_X2AVIC_EXT, CPUID_ECX, 6, 0x8000000a, 0 }, { X86_FEATURE_COHERENCY_SFW_NO, CPUID_EBX, 31, 0x8000001f, 0 }, { X86_FEATURE_SMBA, CPUID_EBX, 2, 0x80000020, 0 },
Signature · 1 line
2.48.1
PATCH v6 5/7 KVM: SVM: Use BIT_ULL for 64-bit nested_ctl bit definitions
In reply to Nikunj A DadhaniaReplace BIT() with BIT_ULL() for SVM nested control bit definitions since nested_ctl is a 64-bit field in the VMCB control area structure.
No functional change intended.
--- arch/x86/include/asm/svm.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/include/asm/svm.h +3 -3
@@ -239,9 +239,9 @@ struct __attribute__ ((__packed__)) vmcb_control_area {#define SVM_IOIO_SIZE_MASK (7 << SVM_IOIO_SIZE_SHIFT)#define SVM_IOIO_ASIZE_MASK (7 << SVM_IOIO_ASIZE_SHIFT) -#define SVM_NESTED_CTL_NP_ENABLE BIT(0) -#define SVM_NESTED_CTL_SEV_ENABLE BIT(1) -#define SVM_NESTED_CTL_SEV_ES_ENABLE BIT(2) +#define SVM_NESTED_CTL_NP_ENABLE BIT_ULL(0) +#define SVM_NESTED_CTL_SEV_ENABLE BIT_ULL(1) +#define SVM_NESTED_CTL_SEV_ES_ENABLE BIT_ULL(2)#define SVM_TSC_RATIO_RSVD 0xffffff0000000000ULL
Signature · 1 line
2.48.1
PATCH v6 6/7 KVM: nSVM: Add helpers to temporarily switch to vmcb01
In reply to Nikunj A DadhaniaIf KVM needs to update controls that belong in vmcb01 while L2 is active, it must switch to vmcb01 and update immediately instead of deferring until the next nested VM-Exit. Deferring updates creates ordering and state consistency problems, e.g. KVM thinks a control is enabled while vmcb01 still has stale state.
Add helpers to temporarily switch to vmcb01 to ensure updates happen on-demand.
--- arch/x86/kvm/svm/svm.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
arch/x86/kvm/svm/svm.c +24 -0
@@ -1251,6 +1251,30 @@ void svm_switch_vmcb(struct vcpu_svm *svm, struct kvm_vmcb_info *target_vmcb) svm->vmcb = target_vmcb->ptr; } +static void svm_load_vmcb01 (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +if (!is_guest_mode (vcpu)) { +WARN_ON_ONCE (svm->vmcb != svm->vmcb01.ptr); +return ; + } + +WARN_ON_ONCE (svm->vmcb != svm->nested.vmcb02.ptr); +svm_switch_vmcb (svm, &svm->vmcb01); +} + +static void svm_put_vmcb01 (struct kvm_vcpu *vcpu) +{ +if (!is_guest_mode (vcpu)) +return ; + +svm_switch_vmcb (to_svm (vcpu), &to_svm (vcpu)->nested.vmcb02); +} + +DEFINE_GUARD(svm_vmcb01, struct kvm_vcpu *, + svm_load_vmcb01(_T), svm_put_vmcb01(_T)) +static int svm_vcpu_precreate (struct kvm *kvm) {return avic_alloc_physical_id_table (kvm);
Signature · 1 line
2.48.1
PATCH v6 7/7 KVM: SVM: Add Page modification logging support
In reply to Nikunj A DadhaniaCurrently, dirty logging relies on write protecting guest memory and marking dirty GFNs during subsequent write faults. This method works but incurs overhead due to additional write faults for each dirty GFN.
Implement support for the Page Modification Logging (PML) feature, a hardware-assisted method for efficient dirty logging. PML automatically logs dirty GPA[51:12] to a 4K buffer when the CPU sets NPT D-bits. Two new VMCB fields are utilized: PML_ADDR and PML_INDEX. The PML_INDEX is initialized to 511 (8 bytes per GPA entry), and the CPU decreases the PML_INDEX after logging each GPA. When the PML buffer is full, a VMEXIT(PML_FULL) with exit code 0x407 is generated.
PML operates on guest physical addresses at the NPT level, tracking D-bit updates in page tables rather than memory content. This allows it to work identically for normal and confidential computing guests (SEV/SEV-ES/SEV-SNP):, enabling cpu_dirty_log_size to be set uniformly for all AMD VMs without special-casing encrypted guests
When L2 is active, svm->vmcb points to vmcb02, so updates to PML controls must explicitly target vmcb01 to ensure the L1's state remains correct. Use the svm_vmcb01 guard to ensure vmcb01 is active during the update.
Disable PML for nested guests.
Add a new module parameter to enable/disable PML, and enable it by default when supported
--- arch/x86/include/asm/svm.h | 6 +- arch/x86/include/uapi/asm/svm.h | 2 + arch/x86/kvm/svm/nested.c | 13 +++- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 101 +++++++++++++++++++++++++++++++- arch/x86/kvm/svm/svm.h | 3 + 6 files changed, 122 insertions(+), 5 deletions(-)
arch/x86/include/asm/svm.h +5 -1
@@ -165,7 +165,10 @@ struct __attribute__ ((__packed__)) vmcb_control_area {u8 reserved_9[22 ];u64 allowed_sev_features;/* Offset 0x138 */ u64 guest_sev_features;/* Offset 0x140 */ -u8 reserved_10[664 ]; +u8 reserved_10[128 ]; +u64 pml_addr;/* Offset 0x1c8 */ +u16 pml_index;/* Offset 0x1d0 */ +u8 reserved_11[526 ];/* * Offset 0x3e0, 32 bytes reserved * for use by hypervisor/software.
@@ -242,6 +245,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {#define SVM_NESTED_CTL_NP_ENABLE BIT_ULL(0)#define SVM_NESTED_CTL_SEV_ENABLE BIT_ULL(1)#define SVM_NESTED_CTL_SEV_ES_ENABLE BIT_ULL(2) +#define SVM_NESTED_CTL_PML_ENABLE BIT_ULL(11)#define SVM_TSC_RATIO_RSVD 0xffffff0000000000ULL
arch/x86/include/uapi/asm/svm.h +2 -0
@@ -101,6 +101,7 @@#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402#define SVM_EXIT_VMGEXIT 0x403 +#define SVM_EXIT_PML_FULL 0x407/* SEV-ES software-defined VMGEXIT events */ #define SVM_VMGEXIT_MMIO_READ 0x80000001ull
@@ -236,6 +237,7 @@ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \ { SVM_EXIT_VMGEXIT, "vmgexit" }, \ + { SVM_EXIT_PML_FULL, "pml_full" }, \ { SVM_VMGEXIT_MMIO_READ, "vmgexit_mmio_read" }, \ { SVM_VMGEXIT_MMIO_WRITE, "vmgexit_mmio_write" }, \ { SVM_VMGEXIT_NMI_COMPLETE, "vmgexit_nmi_complete" }, \
arch/x86/kvm/svm/nested.c +12 -1
@@ -790,12 +790,23 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, V_NMI_BLOCKING_MASK); } -/* Copied from vmcb01. msrpm_base can be overwritten later. */ +/* Copied from vmcb01. msrpm_base/nested_ctl can be overwritten later. */ vmcb02->control.nested_ctl = vmcb01->control.nested_ctl; vmcb02->control.iopm_base_pa = vmcb01->control.iopm_base_pa; vmcb02->control.msrpm_base_pa = vmcb01->control.msrpm_base_pa;vmcb_mark_dirty (vmcb02,VMCB_PERM_MAP ); +/* +* Disable PML for nested guests. When L2 runs with PML enabled, the +* CPU logs L2 GPAs rather than L1 GPAs, breaking dirty page tracking +* for the L0 hypervisor. +*/ +if (pml) { + vmcb02->control.nested_ctl &= ~SVM_NESTED_CTL_PML_ENABLE ; + vmcb02->control.pml_addr =0 ; + vmcb02->control.pml_index =-1 ; + } +/* * Stash vmcb02's counter if the guest hasn't moved past the guilty * instruction; otherwise, reset the counter to '0'.
arch/x86/kvm/svm/sev.c +1 -1
@@ -4820,7 +4820,7 @@ struct page *snp_safe_alloc_page_node(int node, gfp_t gfp)* Allocate an SNP-safe page to workaround the SNP erratum where * the CPU will incorrectly signal an RMP violation #PF if a * hugepage (2MB or 1GB) collides with the RMP entry of a -* 2MB-aligned VMCB, VMSA, or AVIC backing page. +* 2MB-aligned VMCB, VMSA, PML or AVIC backing page. * * Allocate one extra page, choose a page which is not * 2MB-aligned, and free the other.
arch/x86/kvm/svm/svm.c +99 -2
176 lines not on this page · raw message
arch/x86/kvm/svm/svm.h +3 -0
@@ -50,6 +50,7 @@ extern int vgif;extern bool intercept_smi;extern bool vnmi;extern int lbrv; +extern bool pml;extern int tsc_aux_uret_slot__ro_after_init ;
@@ -746,6 +747,8 @@ static inline void svm_enable_intercept_for_msr(struct kvm_vcpu *vcpu,svm_set_intercept_for_msr (vcpu, msr, type, true); } +void svm_update_cpu_dirty_logging (struct kvm_vcpu *vcpu); +/* nested.c */ #define NESTED_EXIT_HOST 0/* Exit handled on host level */
Signature · 1 line
2.48.1
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Sashiko reported a couple of issues [1]. Let me address them here:
Quoted 15 lines · Nikunj A Dadhania
arch/x86/kvm/svm/svm.c +10
-0@@ -1206,6 +1209,16 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)if (vcpu->kvm->arch.bus_lock_detection_enabled)svm_set_intercept (svm,INTERCEPT_BUSLOCK ); +if (pml) { +/* +* Populate the page address and index here, PML is enabled +* when dirty logging is enabled on the memslot through +* svm_update_cpu_dirty_logging() +*/ + control->pml_addr = (u64 )__sme_set (page_to_phys (vcpu->arch.pml_page)); + control->pml_index =PML_HEAD_INDEX ; + } +
Quoted 3 lines
If the guest receives an INIT IPI and
init_vmcb()is called to reset the vCPU, does unconditionally settingpml_indextoPML_HEAD_INDEXdiscard any un-flushed dirty GPAs logged by the hardware?
There are two scenarios where init_vmcb() is called:
1) During vCPU creation time, where we need to set pml_index to PML_HEAD_INDEX
2) During vCPU reset, when init_event=true
Before vCPU reset:
vcpu_enter_guest()
└─> kvm_x86_call(vcpu_run) [VMRUN]
└─> [guest executes, PML accumulates dirty pages] └─> VMEXIT
└─> svm_handle_exit() --> PML buffer flushed here └─> return to vcpu_run()
vCPU Reset:
vcpu_enter_guest()
├─> kvm_check_request(KVM_REQ_EVENT)
├─> kvm_apic_accept_events()
│ └─> kvm_vcpu_reset(..., true)
│ └─> init_vmcb(..., true)
│ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed
└─> kvm_x86_call(): Next VMRUN
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?
AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Quoted 20 lines
arch/x86/kvm/svm/svm.c +15
-0+static void svm_flush_pml_buffer (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); +struct vmcb_control_area *control = &svm->vmcb->control; + +/* Do nothing if PML buffer is empty */ +if (control->pml_index ==PML_HEAD_INDEX ) +return ; + +kvm_flush_pml_buffer (vcpu, control->pml_index); + +/* Reset the PML index */ + control->pml_index =PML_HEAD_INDEX ; +} +Since the AMD SVM architecture aggressively caches VMCB control fields across VMRUN instructions using VMCB clean bits, will the hardware recognize this updated
pml_index?
Quoted 7 lines
If the VMCB isn't explicitly marked as dirty here, for example by calling
vmcb_mark_dirty(svm->vmcb, VMCB_NPT), could the CPU continue using its cached index?That cached index would be less than 0 after a
PML_FULLexit, so would the CPU immediately trigger anotherPML_FULLexit on the next logged write and cause an infinite VM-Exit loop?
I got clarification from the design team: No clean bits were defined for the PML_ADDR and PML_INDEX fields, so on a VMRUN, microcode always loads them from the VMCB if the feature is enabled.
Regards,
Nikunj
1. https://sashiko.dev/#/patchset/20260407063245.2755579-1-nikunj%40amd.com
Sean Christopherson
maintainerRe: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 18 lines · Nikunj A. Dadhania
Sashiko reported a couple of issues [1]. Let me address them here:
Quoted 14 lines · Nikunj A Dadhania
(quoted diff) +10
-0@@ -1206,6 +1209,16 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) if (vcpu->kvm->arch.bus_lock_detection_enabled) svm_set_intercept(svm, INTERCEPT_BUSLOCK); + if (pml) { + /* + * Populate the page address and index here, PML is enabled + * when dirty logging is enabled on the memslot through + * svm_update_cpu_dirty_logging() + */ + control->pml_addr = (u64)__sme_set(page_to_phys(vcpu->arch.pml_page)); + control->pml_index = PML_HEAD_INDEX; + } +
Quoted 8 lines
Quoted 3 lines
If the guest receives an INIT IPI and
init_vmcb()is called to reset the vCPU, does unconditionally settingpml_indextoPML_HEAD_INDEXdiscard any un-flushed dirty GPAs logged by the hardware?There are two scenarios where
init_vmcb()is called:1) During vCPU creation time, where we need to set
pml_indextoPML_HEAD_INDEX
2) During vCPU reset, when init_event=true
Nit, please just call this INIT, even though KVM calls the function "reset". #1 is KVM's one and only true RESET flow.
Quoted 17 lines
Before vCPU reset:
vcpu_enter_guest()
└─>kvm_x86_call(vcpu_run)[VMRUN]
└─> [guest executes, PML accumulates dirty pages] └─> VMEXIT
└─>svm_handle_exit()--> PML buffer flushed here └─> return tovcpu_run()vCPU Reset:
vcpu_enter_guest()
├─>kvm_check_request(KVM_REQ_EVENT)
├─>kvm_apic_accept_events()
│ └─>kvm_vcpu_reset(..., true)
│ └─>init_vmcb(..., true)
│ └─>control->pml_index=PML_HEAD_INDEX-- PML buffer was already flushed
└─>kvm_x86_call(): Next VMRUN
Quoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit f7f39c50edb9 ("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skips kvm_x86_ops.handle_exit(), and specifically can give userspace control without going through vmx_flush_pml_buffer():
if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated kvm_x86_ops hook to flush the buffer on a fastpath userspace exit. But, I dislike one-off kvm_x86_ops hooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident with bus_lock_detected.
Yikes! And I think userspace could see a stale CR0 and/or CR3 on AMD. Hmm, but waiting until the full exit path to grab CR0 and CR3 is flawed on its own, e.g. it's a bug waiting to happen if KVM consumes CR3 in the fastpath.
So over two patches, I think the fix for those issues is the below. I'll test and send a mini-series. I don't think there's anything you need to do; I can resolve the resulting conflict easy enough.
x86/kvm/svm/svm.c +8 -7
@@ -3644,13 +3644,8 @@ static int svm_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)struct vcpu_svm *svm =to_svm (vcpu);struct kvm_run *kvm_run = vcpu->run; -/* SEV-ES guests must use the CR write traps to track CR registers. */ -if (!is_sev_es_guest (vcpu)) { -if (!svm_is_intercept (svm,INTERCEPT_CR0_WRITE )) - vcpu->arch.cr0 = svm->vmcb->save.cr0; -if (npt_enabled) - vcpu->arch.cr3 = svm->vmcb->save.cr3; - } +if (unlikely (exit_fastpath ==EXIT_FASTPATH_EXIT_USERSPACE )) +return 0 ;if (is_guest_mode (vcpu)) {int vmexit;
@@ -4502,11 +4497,17 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)if (!static_cpu_has (X86_FEATURE_V_SPEC_CTRL ))x86_spec_ctrl_restore_host (svm->virt_spec_ctrl); +/* SEV-ES guests must use the CR write traps to track CR registers. */ if (!is_sev_es_guest (vcpu)) { vcpu->arch.cr2 = svm->vmcb->save.cr2; vcpu->arch.regs[VCPU_REGS_RAX ] = svm->vmcb->save.rax; vcpu->arch.regs[VCPU_REGS_RSP ] = svm->vmcb->save.rsp; vcpu->arch.regs[VCPU_REGS_RIP ] = svm->vmcb->save.rip; + +if (!svm_is_intercept (svm,INTERCEPT_CR0_WRITE )) + vcpu->arch.cr0 = svm->vmcb->save.cr0; +if (npt_enabled) + vcpu->arch.cr3 = svm->vmcb->save.cr3; } vcpu->arch.regs_dirty =0 ;
x86/kvm/vmx/vmx.c +3 -0
@@ -6687,6 +6687,9 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)if (enable_pml && !is_guest_mode (vcpu))vmx_flush_pml_buffer (vcpu); +if (unlikely (exit_fastpath ==EXIT_FASTPATH_EXIT_USERSPACE )) +return 0 ; +/* * KVM should never reach this point with a pending nested VM-Enter. * More specifically, short-circuiting VM-Entry to emulate L2 due to
x86/kvm/x86.c +0 -3
@@ -11602,9 +11602,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (vcpu->arch.apic_attention)kvm_lapic_sync_from_vapic (vcpu); -if (unlikely (exit_fastpath ==EXIT_FASTPATH_EXIT_USERSPACE )) -return 0 ; - r =kvm_x86_call (handle_exit)(vcpu, exit_fastpath);return r;
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 25 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);
Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.
Seems vmx_vcpu_reset() doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?
How about doing the same for SVM?
Alternatively, we can always manually flush PML buffer upon INIT before resetting PML index. This still allows to do the EXIT_FASTPATH_EXIT_USERSPACE check in the common code.
Not sure which way is better, though.
Sean Christopherson
maintainerRe: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 32 lines · Kai Huang
Quoted 25 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.
Quoted 5 lines
Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
Hmm, actually, how is that going to work? Dropping PML entries just because a vCPU hit SHUTDOWN isn't going to fly. E.g. if a VM crashes while it's being migrated by the host, then it could end up with corrupted, incoherent data on the target due to leaving dirtied pages behind.
Quoted 5 lines
Alternatively, we can always manually flush PML buffer upon INIT before resetting PML index. This still allows to do the
EXIT_FASTPATH_EXIT_USERSPACEcheck in the common code.Not sure which way is better, though.
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 33 lines · Sean Christopherson
Quoted 32 lines · Kai Huang
Quoted 30 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.
Quoted 8 lines
Quoted 5 lines
Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
You mean KVM needs to reset VMCB to reflect the architecturally defined INIT state for a vCPU? Or the hardware itself may reset VMCB thus may reset PML index?
Quoted 4 lines
Hmm, actually, how is that going to work? Dropping PML entries just because a vCPU hit SHUTDOWN isn't going to fly.
Not dropping, but just leave PML index unchanged. The PML buffer itself is still there unchanged, and PML is still working in hardware thus the buffer will eventually get flushed.
This is the case for VMX AFAICT, thus I wonder whether this also works for SVM.
Quoted 3 lines
E.g. if a VM crashes while it's being
migrated by the host, then it could end up with corrupted, incoherent data on the target due to leaving dirtied pages behind.
If we leave PML index unchanged upon INIT as mentioned above, I don't think we will lose any dirty GPA tracking. But maybe I am missing something.
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 23 lines · Kai
Quoted 22 lines · Sean Christopherson
Quoted 21 lines · Kai Huang
Quoted 20 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);
Ah right.
Quoted 24 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.Quoted 8 lines
Quoted 5 lines
Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
You mean KVM needs to reset VMCB to reflect the architecturally defined INIT state for a vCPU? Or the hardware itself may reset VMCB thus may reset PML index?
Quoted 4 lines
Hmm, actually, how is that going to work? Dropping PML entries just because a vCPU hit SHUTDOWN isn't going to fly.
Quoted 6 lines
Not dropping, but just leave PML index unchanged. The PML buffer itself is still there unchanged, and PML is still working in hardware thus the buffer will eventually get flushed.
This is the case for VMX AFAICT, thus I wonder whether this also works for SVM.
Theoretically, while the migration is active and we did a fast path user space exit and never returned, the entries remain in PML buffer. If we flush every time on VM exit, as per Sean's suggestion, PML buffer is always flushed before returning to user space.
Quoted 7 lines
Quoted 4 lines
E.g. if a VM crashes while it's being
migrated by the host, then it could end up with corrupted, incoherent data on the target due to leaving dirtied pages behind.If we leave PML index unchanged upon INIT as mentioned above, I don't think we will lose any dirty GPA tracking. But maybe I am missing something.
Regards
Nikunj
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 60 lines · Nikunj A. Dadhania
Quoted 23 lines · Kai
Quoted 22 lines · Sean Christopherson
Quoted 21 lines · Kai Huang
Quoted 20 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);Ah right.
Quoted 28 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.Quoted 8 lines
Quoted 5 lines
Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
You mean KVM needs to reset VMCB to reflect the architecturally defined INIT state for a vCPU? Or the hardware itself may reset VMCB thus may reset PML index?
Quoted 4 lines
Hmm, actually, how is that going to work? Dropping PML entries just because a vCPU hit SHUTDOWN isn't going to fly.
Not dropping, but just leave PML index unchanged. The PML buffer itself is still there unchanged, and PML is still working in hardware thus the buffer will eventually get flushed.
Quoted 7 lines
Quoted 2 lines
This is the case for VMX AFAICT, thus I wonder whether this also works for SVM.
Theoretically, while the migration is active and we did a fast path user space exit and never returned, the entries remain in PML buffer. If we flush every time on VM exit, as per Sean's suggestion, PML buffer is always flushed before returning to user space.
Oh indeed I missed something here. I assumed the last GET_DIRTY_LOG ioctl can always flush the PML buffer. But I forgot KVM flushes the buffer by kicking all vcpus.
So yeah we have a loophole on the EXIT_FASTPATH_EXIT_USERSPACE, which seems can be controlled by userspace (e.g., single step debugging?).
So what Sean posted makes sense. Sorry for the noise.
Or maybe we could simply flush PML buffer in {vmx|svm}_vcpu_enter_exit()?
Sean Christopherson
maintainerRe: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
In reply to Huang, KaiQuoted 38 lines · Kai Huang
Quoted 37 lines · Sean Christopherson
Quoted 36 lines · Kai Huang
Quoted 25 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
You mean KVM needs to reset VMCB to reflect the architecturally defined INIT state for a vCPU? Or the hardware itself may reset VMCB thus may reset PML index?
Neither. The APM states that the VMCB is undefined after SHUTDOWN. PML index
could be anything:
15.14.3 Shutdown Intercept
When this intercept occurs, any condition that normally causes a shutdown causes a #VMEXIT to the VMM instead. After an intercepted shutdown, the state saved in the VMCB is undefined.
KVM synthesizes an INIT because it's the least awful option.
static int shutdown_interception(struct kvm_vcpu *vcpu) {
struct kvm_run *kvm_run = vcpu->run; struct vcpu_svm *svm = to_svm(vcpu);
/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
* KVM_MP_STATE_SHUTDOWN and can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*
* The VM save area for SEV-ES guests has already been encrypted so it
* cannot be reinitialized, i.e. synthesizing INIT is futile. */
if (!is_sev_es_guest(vcpu)) {
clear_page(svm->vmcb);
#ifdef CONFIG_KVM_SMM if (is_smm(vcpu)) kvm_smm_changed(vcpu, false); #endif kvm_vcpu_reset(vcpu, true); } kvm_run->exit_reason = KVM_EXIT_SHUTDOWN; return 0;
}
Now, maybe the APM is trying to say only the save area is undefined, in which case PML Index is fine and can and should be left alone. But if that's the case, the APM needs to be updated to make explicitly clear what fields in the VMCS are and are not valid after SHUTDOWN.
Quoted 7 lines
Quoted 3 lines
Hmm, actually, how is that going to work? Dropping PML entries just because a vCPU hit SHUTDOWN isn't going to fly.
Not dropping, but just leave PML index unchanged. The PML buffer itself is still there unchanged, and PML is still working in hardware thus the buffer will eventually get flushed.
As above, the PML index could be garbage.
Quoted 2 lines
This is the case for VMX AFAICT, thus I wonder whether this also works for SVM.
No, VMX is fine.
Quoted 6 lines
Quoted 3 lines
E.g. if a VM crashes while it's being
migrated by the host, then it could end up with corrupted, incoherent data on the target due to leaving dirtied pages behind.If we leave PML index unchanged upon INIT as mentioned above, I don't think we will lose any dirty GPA tracking. But maybe I am missing something.
Correct. The question is whether or not AMD's architecture guarantees PML Index will be valid/accurate after SHUTDOWN.
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 85 lines · Sean Christopherson
Quoted 38 lines · Kai Huang
Quoted 37 lines · Sean Christopherson
Quoted 36 lines · Kai Huang
Quoted 25 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?Quoted 8 lines
Quoted 4 lines
Quoted 1 line
How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
You mean KVM needs to reset VMCB to reflect the architecturally defined INIT state for a vCPU? Or the hardware itself may reset VMCB thus may reset PML index?
Neither. The APM states that the VMCB is undefined after SHUTDOWN. PML index
could be anything:15.14.3 Shutdown Intercept
When this intercept occurs, any condition that normally causes a shutdown causes a #VMEXIT to the VMM instead. After an intercepted shutdown, the state saved in the VMCB is undefined.KVM synthesizes an INIT because it's the least awful option.
static int
shutdown_interception(struct kvm_vcpu *vcpu){
structkvm_run*kvm_run=vcpu->run; structvcpu_svm*svm =to_svm(vcpu);/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
*KVM_MP_STATE_SHUTDOWNand can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*
* The VM save area for SEV-ES guests has already been encrypted so it
* cannot be reinitialized, i.e. synthesizing INIT is futile. */
if (!is_sev_es_guest(vcpu)) {
clear_page(svm->vmcb);#ifdef CONFIG_KVM_SMM if (is_smm(vcpu)) kvm_smm_changed(vcpu, false); #endif kvm_vcpu_reset(vcpu, true); } kvm_run->exit_reason = KVM_EXIT_SHUTDOWN; return 0;
Quoted 7 lines
}
Now, maybe the APM is trying to say only the save area is undefined, in which case PML Index is fine and can and should be left alone. But if that's the case, the APM needs to be updated to make explicitly clear what fields in the VMCS are and are not valid after SHUTDOWN.
Quoted 1 line
Quoted 1 line
Thanks for all the info! :-)
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
In reply to Sean ChristophersonQuoted 85 lines · Sean Christopherson
Quoted 38 lines · Kai Huang
Quoted 37 lines · Sean Christopherson
Quoted 36 lines · Kai Huang
Quoted 25 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?Quoted 8 lines
Quoted 4 lines
Quoted 1 line
How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
You mean KVM needs to reset VMCB to reflect the architecturally defined INIT state for a vCPU? Or the hardware itself may reset VMCB thus may reset PML index?
Neither. The APM states that the VMCB is undefined after SHUTDOWN. PML index
could be anything:15.14.3 Shutdown Intercept
When this intercept occurs, any condition that normally causes a shutdown causes a #VMEXIT to the VMM instead. After an intercepted shutdown, the state saved in the VMCB is undefined.KVM synthesizes an INIT because it's the least awful option.
static int
shutdown_interception(struct kvm_vcpu *vcpu){
structkvm_run*kvm_run=vcpu->run; structvcpu_svm*svm =to_svm(vcpu);/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
*KVM_MP_STATE_SHUTDOWNand can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*
* The VM save area for SEV-ES guests has already been encrypted so it
* cannot be reinitialized, i.e. synthesizing INIT is futile. */
if (!is_sev_es_guest(vcpu)) {
clear_page(svm->vmcb);#ifdef CONFIG_KVM_SMM if (is_smm(vcpu)) kvm_smm_changed(vcpu, false); #endif kvm_vcpu_reset(vcpu, true); } kvm_run->exit_reason = KVM_EXIT_SHUTDOWN; return 0;
Quoted 6 lines
}
Now, maybe the APM is trying to say only the save area is undefined, in which case PML Index is fine and can and should be left alone. But if that's the case, the APM needs to be updated to make explicitly clear what fields in the VMCS are and are not valid after SHUTDOWN.
I'm checking to see if I can get clarification on this.
Thanks,
Tom
Quoted 26 lines
Quoted 8 lines
Quoted 4 lines
Hmm, actually, how is that going to work? Dropping PML entries just because a vCPU hit SHUTDOWN isn't going to fly.
Not dropping, but just leave PML index unchanged. The PML buffer itself is still there unchanged, and PML is still working in hardware thus the buffer will eventually get flushed.
As above, the PML index could be garbage.
Quoted 2 lines
This is the case for VMX AFAICT, thus I wonder whether this also works for SVM.
No, VMX is fine.
Quoted 6 lines
Quoted 3 lines
E.g. if a VM crashes while it's being
migrated by the host, then it could end up with corrupted, incoherent data on the target due to leaving dirtied pages behind.If we leave PML index unchanged upon INIT as mentioned above, I don't think we will lose any dirty GPA tracking. But maybe I am missing something.
Correct. The question is whether or not AMD's architecture guarantees PML Index will be valid/accurate after SHUTDOWN.
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 88 lines · Tom Lendacky
Quoted 87 lines · Sean Christopherson
Quoted 38 lines · Kai Huang
Quoted 37 lines · Sean Christopherson
Quoted 36 lines · Kai Huang
Quoted 25 lines · Sean Christopherson
Quoted 9 lines
vCPU Reset:
vcpu_enter_guest()├─> kvm_check_request(KVM_REQ_EVENT) ├─> kvm_apic_accept_events() │ └─> kvm_vcpu_reset(..., true) │ └─> init_vmcb(..., true) │ └─> control->pml_index = PML_HEAD_INDEX -- PML buffer was already flushed └─> kvm_x86_call(): Next VMRUNQuoted 5 lines
Quoted 3 lines
Could this result in the hypervisor losing track of dirty memory during live migration, leading to memory corruption on the destination host, since
svm_flush_pml_buffer()isn't called before resetting the index?AFAIU, no. The PML buffer is always flushed opportunistically at every VM exit.
Huh. There's a pre-existing bug here. Commit
f7f39c50edb9("KVM: x86: Exit to userspace if fastpath triggers one on instruction skip") added a path that skipskvm_x86_ops.handle_exit(), and specifically can give userspace control without going throughvmx_flush_pml_buffer():if (unlikely(exit_fastpath == EXIT_FASTPATH_EXIT_USERSPACE)) return 0; r = kvm_x86_call(handle_exit)(vcpu, exit_fastpath);Quoted 5 lines
Given that SVM support for PML is (obviously) on its way, it's mildly tempting to add a dedicated
kvm_x86_opshook to flush the buffer on a fastpath userspace exit. But, I dislike one-offkvm_x86_opshooks, and that only works if there's no other vendor action required. E.g. very theoretically, a fastpath userspace exit could also be coincident withbus_lock_detected.Seems
vmx_vcpu_reset()doesn't reset PML index upon INIT event, which seems to be fine since we are not losing any dirty GPA tracking AFAICT (otherwise we already have a bug for VMX here)?Quoted 8 lines
Quoted 4 lines
Quoted 1 line
How about doing the same for SVM?
We don't really have that luxury. On SHUTDOWN (even intercepted SHUTDOWN), the state of the VMCB is technically undefined. I.e. KVM needs to write _something_.
You mean KVM needs to reset VMCB to reflect the architecturally defined INIT state for a vCPU? Or the hardware itself may reset VMCB thus may reset PML index?
Neither. The APM states that the VMCB is undefined after SHUTDOWN. PML index
could be anything:15.14.3 Shutdown Intercept
When this intercept occurs, any condition that normally causes a shutdown causes a #VMEXIT to the VMM instead. After an intercepted shutdown, the state saved in the VMCB is undefined.KVM synthesizes an INIT because it's the least awful option.
static int
shutdown_interception(struct kvm_vcpu *vcpu){
structkvm_run*kvm_run=vcpu->run; structvcpu_svm*svm =to_svm(vcpu);/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
*KVM_MP_STATE_SHUTDOWNand can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*
* The VM save area for SEV-ES guests has already been encrypted so it
* cannot be reinitialized, i.e. synthesizing INIT is futile. */
if (!is_sev_es_guest(vcpu)) {
clear_page(svm->vmcb);#ifdef CONFIG_KVM_SMM if (is_smm(vcpu)) kvm_smm_changed(vcpu, false); #endif kvm_vcpu_reset(vcpu, true); } kvm_run->exit_reason = KVM_EXIT_SHUTDOWN; return 0;}
Quoted 6 lines
Quoted 4 lines
Now, maybe the APM is trying to say only the save area is undefined, in which case PML Index is fine and can and should be left alone. But if that's the case, the APM needs to be updated to make explicitly clear what fields in the VMCS are and are not valid after SHUTDOWN.
I'm checking to see if I can get clarification on this.
PML_INDEX will be valid even when a SHUTDOWN is intercepted. I'm working to see if we can update the APM to identify what would be valid and what wouldn't be in this situation.
For a triple-fault SHUTDOWN, the saved x86 state could be inconsistent because the CPU stopped in the middle of a sequence and redirected to shutdown. Everything will be saved, but it might not be valid x86 state. The question then becomes are there fields within the VMCB control area that are based on x86 state that might not be valid (e.g. V_IRQ or nRIP, etc.).
Thanks,
Tom
Quoted 32 lines
Thanks,
TomQuoted 26 lines
Quoted 8 lines
Quoted 4 lines
Hmm, actually, how is that going to work? Dropping PML entries just because a vCPU hit SHUTDOWN isn't going to fly.
Not dropping, but just leave PML index unchanged. The PML buffer itself is still there unchanged, and PML is still working in hardware thus the buffer will eventually get flushed.
As above, the PML index could be garbage.
Quoted 2 lines
This is the case for VMX AFAICT, thus I wonder whether this also works for SVM.
No, VMX is fine.
Quoted 6 lines
Quoted 3 lines
E.g. if a VM crashes while it's being
migrated by the host, then it could end up with corrupted, incoherent data on the target due to leaving dirtied pages behind.If we leave PML index unchanged upon INIT as mentioned above, I don't think we will lose any dirty GPA tracking. But maybe I am missing something.
Correct. The question is whether or not AMD's architecture guarantees PML Index will be valid/accurate after SHUTDOWN.
Sean Christopherson
maintainerRe: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 8 lines · Tom Lendacky
Quoted 6 lines · Tom Lendacky
Quoted 4 lines
Now, maybe the APM is trying to say only the save area is undefined, in which case PML Index is fine and can and should be left alone. But if that's the case, the APM needs to be updated to make explicitly clear what fields in the VMCS are and are not valid after SHUTDOWN.
I'm checking to see if I can get clarification on this.
Quoted 7 lines
PML_INDEXwill be valid even when a SHUTDOWN is intercepted. I'm working to see if we can update the APM to identify what would be valid and what wouldn't be in this situation.For a triple-fault SHUTDOWN, the saved x86 state could be inconsistent because the CPU stopped in the middle of a sequence and redirected to shutdown. Everything will be saved, but it might not be valid x86 state.
Yeah, I assumed that the APM wording was an overly-broad description of why guest state would be garbage.
Quoted 3 lines
The question then becomes are there fields within the VMCB control area that are based on x86 state that might not be valid (e.g.
V_IRQor nRIP, etc.).
FWIW, even a conservative iset of "these fields/areas are guaranteed to be sane and consistent" would be super helpful. The big problem right now is that the wording reads like the entire GHCB is suspect.
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 20 lines · Sean Christopherson
Quoted 8 lines · Tom Lendacky
Quoted 6 lines · Tom Lendacky
Quoted 4 lines
Now, maybe the APM is trying to say only the save area is undefined, in which case PML Index is fine and can and should be left alone. But if that's the case, the APM needs to be updated to make explicitly clear what fields in the VMCS are and are not valid after SHUTDOWN.
I'm checking to see if I can get clarification on this.
Quoted 7 lines
PML_INDEXwill be valid even when a SHUTDOWN is intercepted. I'm working to see if we can update the APM to identify what would be valid and what wouldn't be in this situation.For a triple-fault SHUTDOWN, the saved x86 state could be inconsistent because the CPU stopped in the middle of a sequence and redirected to shutdown. Everything will be saved, but it might not be valid x86 state.
Yeah, I assumed that the APM wording was an overly-broad description of why guest state would be garbage.
Quoted 7 lines
Quoted 3 lines
The question then becomes are there fields within the VMCB control area that are based on x86 state that might not be valid (e.g.
V_IRQor nRIP, etc.).FWIW, even a conservative iset of "these fields/areas are guaranteed to be sane and consistent" would be super helpful. The big problem right now is that the wording reads like the entire GHCB is suspect.
I believe you meant VMCB above, not GHCB. :-)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
The APM states that the VMCB is undefined after SHUTDOWN. PML index could be anything:
15.14.3 Shutdown Intercept
When this intercept occurs, any condition that normally causes a shutdown causes a #VMEXIT to the VMM instead. After an intercepted shutdown, the state saved in the VMCB is undefined.
The anticipated APM update will clarify the wording as follows:
"After an intercepted shutdown, the VMCB control area is valid (with the
exception of offsets 60h, 61h, and 68h) and the VMCB state save area is undefined."
Based on this clarification, I will initialize the PML index only during reset and leave it unchanged in the case of an INIT event.
arch/x86/kvm/svm/svm.c +3 -1
@@ -1216,7 +1216,9 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)* svm_update_cpu_dirty_logging() */ control->pml_addr = (u64 )__sme_set (page_to_phys (vcpu->arch.pml_page)); - control->pml_index =PML_HEAD_INDEX ; + +if (!init_event) + control->pml_index =PML_HEAD_INDEX ; }if (sev_guest (vcpu->kvm))
Regards,
Nikunj
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
In reply to Nikunj A Dadhaniaarch/x86/kvm/svm/svm.c static int bus_lock_exit(struct kvm_vcpu *vcpu) · line 3288
Quoted 12 lines
arch/x86/kvm/svm/svm.c +8
-0@@ -3253,6 +3278,54 @@ static int bus_lock_exit(struct kvm_vcpu *vcpu)return 0 ; } +void svm_update_cpu_dirty_logging (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +if (WARN_ON_ONCE (!vcpu->kvm->arch.cpu_dirty_log_size)) +return ; + +guard (svm_vmcb01)(vcpu);
Why do we need this? Why not update vmcb01 directly below?
IIUC VMX needs this because it updates the loaded VMCS, but the VMCB is just memory.
Quoted 14 lines
arch/x86/kvm/svm/svm.c +14
-0+ +/* +* Note, nr_memslots_dirty_logging can be changed concurrent with this +* code, but in that case another update request will be made and so +* the guest will never run with a stale PML value. +*/ +if (atomic_read (&vcpu->kvm->nr_memslots_dirty_logging)) + svm->vmcb->control.nested_ctl |=SVM_NESTED_CTL_PML_ENABLE ; +else + svm->vmcb->control.nested_ctl &= ~SVM_NESTED_CTL_PML_ENABLE ; + +vmcb_mark_dirty (svm->vmcb,VMCB_NPT ); +} +
Sean Christopherson
maintainerRe: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 4 lines · Yosry Ahmed
Quoted 4 lines
arch/x86/kvm/svm/svm.c +0
-0@@ -3253,6 +3278,54 @@ static int bus_lock_exit(struct kvm_vcpu *vcpu)return 0 ; }
Quoted 10 lines
arch/x86/kvm/svm/svm.c static int bus_lock_exit(struct kvm_vcpu *vcpu) · line 3288
Quoted 8 lines
arch/x86/kvm/svm/svm.c +8
-0+void svm_update_cpu_dirty_logging (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +if (WARN_ON_ONCE (!vcpu->kvm->arch.cpu_dirty_log_size)) +return ; + +guard (svm_vmcb01)(vcpu);Why do we need this? Why not update vmcb01 directly below?
Yeah, I don't see the point of patch 6. guard(svm_vmcb01) doesn't exist in the current code base because it's not needed.
Quoted 2 lines
IIUC VMX needs this because it updates the loaded VMCS, but the VMCB is just memory.
Yep. If VMWRITE could target an arbitrary VMCS pointer, VMX wouldn't need the helper either.
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 5 lines · Sean Christopherson
Quoted 4 lines · Yosry Ahmed
Quoted 4 lines
arch/x86/kvm/svm/svm.c +0
-0@@ -3253,6 +3278,54 @@ static int bus_lock_exit(struct kvm_vcpu *vcpu)return 0 ; }
Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/svm/svm.c static int bus_lock_exit(struct kvm_vcpu *vcpu) · line 3288
Quoted 8 lines
arch/x86/kvm/svm/svm.c +8
-0+void svm_update_cpu_dirty_logging (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +if (WARN_ON_ONCE (!vcpu->kvm->arch.cpu_dirty_log_size)) +return ; + +guard (svm_vmcb01)(vcpu);Why do we need this? Why not update vmcb01 directly below?
Are you suggesting something like below:
void svm_update_cpu_dirty_logging(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
struct vmcb *vmcb01 = svm->vmcb01.ptr;
if (WARN_ON_ONCE(!vcpu->kvm->arch.cpu_dirty_log_size))
return;
/*
* Note, nr_memslots_dirty_logging can be changed concurrent with this
* code, but in that case another update request will be made and so
* the guest will never run with a stale PML value.
*/
if (atomic_read(&vcpu->kvm->nr_memslots_dirty_logging))
vmcb01->control.nested_ctl |= SVM_NESTED_CTL_PML_ENABLE;
else
vmcb01->control.nested_ctl &= ~SVM_NESTED_CTL_PML_ENABLE;
vmcb_mark_dirty(vmcb01, VMCB_NPT);
}Quoted 3 lines
Yeah, I don't see the point of patch 6.
guard(svm_vmcb01)doesn't exist in the current code base because it's not needed.
Sure, I can drop patch6. I wanted to keep the code similar.
Quoted 6 lines
Quoted 3 lines
IIUC VMX needs this because it updates the loaded VMCS, but the VMCB is just memory.
Yep. If VMWRITE could target an arbitrary VMCS pointer, VMX wouldn't need the helper either.
Regards
Nikunj
Re: [PATCH v6 7/7] KVM: SVM: Add Page modification logging support
Quoted 19 lines · Nikunj A. Dadhania
Quoted 5 lines · Sean Christopherson
Quoted 4 lines · Yosry Ahmed
Quoted 4 lines
arch/x86/kvm/svm/svm.c +0
-0@@ -3253,6 +3278,54 @@ static int bus_lock_exit(struct kvm_vcpu *vcpu)return 0 ; }Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/svm/svm.c static int bus_lock_exit(struct kvm_vcpu *vcpu) · line 3288
Quoted 8 lines
arch/x86/kvm/svm/svm.c +8
-0+void svm_update_cpu_dirty_logging (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +if (WARN_ON_ONCE (!vcpu->kvm->arch.cpu_dirty_log_size)) +return ; + +guard (svm_vmcb01)(vcpu);Why do we need this? Why not update vmcb01 directly below?
Quoted 1 line
Are you suggesting something like below:
Yes.
Quoted 21 lines
void
svm_update_cpu_dirty_logging(struct kvm_vcpu *vcpu){ struct vcpu_svm *svm = to_svm(vcpu); struct vmcb *vmcb01 = svm->vmcb01.ptr; if (WARN_ON_ONCE(!vcpu->kvm->arch.cpu_dirty_log_size)) return; /* * Note, nr_memslots_dirty_logging can be changed concurrent with this * code, but in that case another update request will be made and so * the guest will never run with a stale PML value. */ if (atomic_read(&vcpu->kvm->nr_memslots_dirty_logging)) vmcb01->control.nested_ctl |= SVM_NESTED_CTL_PML_ENABLE; else vmcb01->control.nested_ctl &= ~SVM_NESTED_CTL_PML_ENABLE; vmcb_mark_dirty(vmcb01, VMCB_NPT); }