KVM: nSVM: Emulate Page Modification Logging
cover KVM: nSVM: Emulate Page Modification Logging
Emulate Page Modification Logging for nested SVM guests, allowing an L1 hypervisor to use PML to track dirty pages in its L2 guests, e.g. for nested live migration. Hardware PML is never enabled for L2; KVM instead logs L2 GPAs into L1's buffer during the nested NPT walk and synthesizes PML Full exits when it fills, following the existing nVMX emulation.
Based on kvm-x86/next (dea4f6380f4e).
Tested on AMD hardware with PML:
- A 16G 2-vCPU L2 guest running stress-ng was ping-pong migrated 100 times within L1, with liveness and memory writes checked after each iteration.
- nested_dirty_log_test and dirty_log_test pass with npt=1 pml=1, npt=1 pml=0 and npt=0, with no kernel warnings.
Nikunj A Dadhania (5):
KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
KVM: nSVM: Add nested PML emulation
KVM: nSVM: Advertise PML to L1
KVM: selftests: nVMX: Add nested PML coverage to nested_dirty_log_test
KVM: selftests: nSVM: Add nested PML coverage to nested_dirty_log_test
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/cpuid.c | 6 +- arch/x86/kvm/mmu/paging_tmpl.h | 6 +- arch/x86/kvm/reverse_cpuid.h | 5 + arch/x86/kvm/svm/nested.c | 78 ++++++- arch/x86/kvm/svm/svm.c | 3 + arch/x86/kvm/svm/svm.h | 8 + .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/include/x86/svm.h | 6 +- .../selftests/kvm/include/x86/svm_util.h | 4 + tools/testing/selftests/kvm/include/x86/vmx.h | 5 + tools/testing/selftests/kvm/lib/x86/svm.c | 3 + tools/testing/selftests/kvm/lib/x86/vmx.c | 18 ++ .../selftests/kvm/x86/nested_dirty_log_test.c | 191 +++++++++++++++++- 14 files changed, 329 insertions(+), 6 deletions(-)
base-commit: dea4f6380f4ec96e8f3b0aaf2343764bd0d25ef4
Signature · 1 line
2.48.1
RFC PATCH 1/5 KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
Gate PML logging on the walk being the nested GPA walk, as only that walk yields an L2 GPA.
No functional change intended, as the EPT template only backs the nested GPA walk. The upcoming nested SVM PML support runs this code for PTTYPE 64, which other walks reach too.
--- arch/x86/kvm/mmu/paging_tmpl.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/paging_tmpl.h +3 -1
@@ -235,7 +235,9 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, !(pte & PT_GUEST_DIRTY_MASK)) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte));#if PTTYPE ==PTTYPE_EPT -if (kvm_nested_call (write_log_dirty)(vcpu, addr)) +/* Only the nested GPA walk yields an L2 GPA to log. */ +if (w == &vcpu->arch.ngpa_walk && +kvm_nested_call (write_log_dirty)(vcpu, addr))return -EINVAL ;#endif pte |=PT_GUEST_DIRTY_MASK ;
Signature · 1 line
2.48.1
[email protected]
AI reviewRe: [RFC PATCH 1/5] KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] kvm: x86/mmu: L2 GFN type confusion and missed PML logs for L2 page tables --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kvm: x86/mmu: L2 GFN type confusion and missed PML logs for L2 page tables --
commit 39018b0fb43ef6b337e5d8b61b880163828654c8
Author: Nikunj A Dadhania <[email protected]>
KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
This patch gates PML logging on the walk being the nested GPA walk, assuming only that walk yields an L2 GPA. No functional change is intended as the EPT template only backs the nested GPA walk.
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, · line 240
Quoted 12 lines
arch/x86/kvm/mmu/paging_tmpl.h +3
-1@@ -235,7 +235,9 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, !(pte & PT_GUEST_DIRTY_MASK)) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte));#if PTTYPE ==PTTYPE_EPT -if (kvm_nested_call (write_log_dirty)(vcpu, addr)) +/* Only the nested GPA walk yields an L2 GPA to log. */ +if (w == &vcpu->arch.ngpa_walk && +kvm_nested_call (write_log_dirty)(vcpu, addr))
[Severity: High]
Does this change miss PML logs for L2 page table modifications that occur during an L2 gva_walk?
When KVM emulates setting A/D bits in an L2 page table (during an L2 gva_walk), it is writing to an L2 GPA. Hardware logs page table updates to the PML buffer, so it seems KVM must do the same.
By ignoring gva_walk, does this fail to log L2 page table modifications to L1's PML buffer, breaking L1's dirty page tracking?
Additionally, looking further down in update_accessed_dirty_bits():
arch/x86/kvm/mmu/paging_tmpl.h:update_accessed_dirty_bits() {
...
ret = __try_cmpxchg_user(ptep_user, &orig_pte, pte, fault);
if (ret)
return ret;
kvm_vcpu_mark_page_dirty(vcpu, table_gfn);
...
}During a nested guest page table walk (for example, when
w == &vcpu->arch.gva_walk for L2), walk_addr_generic() populates walker->table_gfn with the untranslated L2 GFN of the guest page table.
When update_accessed_dirty_bits() sets A/D bits in the L2 page table, it passes this untranslated L2 GFN to kvm_vcpu_mark_page_dirty().
Since KVM's memslots use L1 physical addresses, does looking up an L2 GFN here cause type confusion that could corrupt L1's dirty bitmap and fail to dirty-track the actual L1 backing page (real_gpa)?
Quoted 3 lines
arch/x86/kvm/mmu/paging_tmpl.h +0
-0return -EINVAL ;#endif pte |=PT_GUEST_DIRTY_MASK ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [RFC PATCH 1/5] KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
Quoted 15 lines · On 10/15/2026 4:25 PM, [email protected] wrote:
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kvm: x86/mmu: L2 GFN type confusion and missed PML logs for L2 page tables --commit
39018b0fb43ef6b337e5d8b61b880163828654c8
Author: Nikunj A Dadhania <[email protected]>KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
This patch gates PML logging on the walk being the nested GPA walk, assuming only that walk yields an L2 GPA. No functional change is intended as the EPT template only backs the nested GPA walk.
Quoted 16 lines
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, · line 240
Quoted 12 lines
arch/x86/kvm/mmu/paging_tmpl.h +3
-1@@ -235,7 +235,9 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, !(pte & PT_GUEST_DIRTY_MASK)) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte));#if PTTYPE ==PTTYPE_EPT -if (kvm_nested_call (write_log_dirty)(vcpu, addr)) +/* Only the nested GPA walk yields an L2 GPA to log. */ +if (w == &vcpu->arch.ngpa_walk && +kvm_nested_call (write_log_dirty)(vcpu, addr))[Severity: High]
Does this change miss PML logs for L2 page table modifications that occur during an L2gva_walk?
No, those are logged, just not from the gva_walk. Walking L2's page tables translates each page-table page through the nested walk, as a write:
walk_addr_generic(w = gva_walk)
nested_access = (have_ad ? PFERR_WRITE_MASK : 0) | PFERR_USER_MASK; kvm_translate_gpa(gva_walk, gfn_to_gpa(table_gfn), nested_access | ...)
svm_translate_nested_gpa() /* switches w to ngpa_walk */
walk_addr_generic(w = ngpa_walk) /* write_fault is set */
update_accessed_dirty_bits()
write_log_dirty(vcpu, addr) /* L2 GPA of the page table */
Logging from the gva_walk would be wrong rather than extra: 'addr' there is an L2 GVA, not a GPA.
Quoted 7 lines
When KVM emulates setting A/D bits in an L2 page table (during an L2
gva_walk), it is writing to an L2 GPA. Hardware logs page table updates to the PML buffer, so it seems KVM must do the same.By ignoring
gva_walk, does this fail to log L2 page table modifications to L1's PML buffer, breaking L1's dirty page tracking?
Regards,
Nikunj
Re: [RFC PATCH 1/5] KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
In reply to Nikunj A DadhaniaQuoted 19 lines · Nikunj A Dadhania
Gate PML logging on the walk being the nested GPA walk, as only that walk yields an L2 GPA.
No functional change intended, as the EPT template only backs the nested GPA walk. The upcoming nested SVM PML support runs this code for PTTYPE 64, which other walks reach too.
Assisted-by LLMSigned-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/kvm/mmu/paging_tmpl.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)arch/x86/kvm/mmu/paging_tmpl.h +0
-0
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, · line 237
Quoted 3 lines
arch/x86/kvm/mmu/paging_tmpl.h +0
-0@@ -235,7 +235,9 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, !(pte & PT_GUEST_DIRTY_MASK)) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte));#if PTTYPE ==PTTYPE_EPT
Do we still need this?
Quoted 11 lines
arch/x86/kvm/mmu/paging_tmpl.h +3
-2-if (kvm_nested_call (write_log_dirty)(vcpu, addr)) +/* Only the nested GPA walk yields an L2 GPA to log. */ +if (w == &vcpu->arch.ngpa_walk && +kvm_nested_call (write_log_dirty)(vcpu, addr))return -EINVAL ;#endif pte |=PT_GUEST_DIRTY_MASK ; --2.48.1
Re: [RFC PATCH 1/5] KVM: x86/mmu: Log dirty GPAs only for the nested GPA walk
Quoted 15 lines · Yosry Ahmed
Quoted 14 lines · Nikunj A Dadhania
Gate PML logging on the walk being the nested GPA walk, as only that walk yields an L2 GPA.
No functional change intended, as the EPT template only backs the nested GPA walk. The upcoming nested SVM PML support runs this code for PTTYPE 64, which other walks reach too.
Assisted-by LLMSigned-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/kvm/mmu/paging_tmpl.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 10 lines
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, · line 237
Quoted 8 lines
arch/x86/kvm/mmu/paging_tmpl.h +0
-0@@ -235,7 +235,9 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu, !(pte & PT_GUEST_DIRTY_MASK)) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte));#if PTTYPE ==PTTYPE_EPT Do we still need this?
For correctness, no, the ngpa_walk check is sufficient on its own.
But dropping the #if also widens this to PTTYPE 32, i.e. a non-PAE L1 that uses NPT. With KVM as L1 this cannot happen: a non-PAE build disables NPT, so L1 never sets NP, which PML requires.
if (!IS_ENABLED(CONFIG_X86_64) && !IS_ENABLED(CONFIG_X86_PAE))
npt_enabled = false;That leaves a path I have no way to exercise, so that is why the #if is still there.
No strong preference though, so I'll go with whatever you and Sean prefer for v2.
Regards,
Nikunj
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/mmu/paging_tmpl.h +3
-1-if (kvm_nested_call (write_log_dirty)(vcpu, addr)) +/* Only the nested GPA walk yields an L2 GPA to log. */ +if (w == &vcpu->arch.ngpa_walk && +kvm_nested_call (write_log_dirty)(vcpu, addr))return -EINVAL ;#endif pte |=PT_GUEST_DIRTY_MASK ;
RFC PATCH 2/5 KVM: nSVM: Add nested PML emulation
In reply to Nikunj A DadhaniaEmulate Page Modification Logging (PML) for nested SVM guests, allowing L1 hypervisors to use PML to track dirty pages in their L2 guests. KVM emulates PML in software by write-protecting L2 SPTEs and logging dirty GPAs to L1's PML buffer via the write_log_dirty hook.
PML_FULL exits are prioritized over NPF exits to ensure L1 processes the PML buffer before continuing L2 execution.
Clear any pending PML Full on nested VMRUN. The flag is derived from L1's log index, and a fault that KVM discards instead of injecting can leave it set after L1 has drained its buffer. No event is lost: if the buffer is genuinely full, the next dirty write raises PML Full again.
--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/svm/nested.c | 75 +++++++++++++++++++++++++++++++++- arch/x86/kvm/svm/svm.h | 8 ++++ 3 files changed, 83 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
@@ -234,7 +234,7 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu,if (level == walker->level && write_fault && !(pte &PT_GUEST_DIRTY_MASK )) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte)); -#if PTTYPE ==PTTYPE_EPT +#if (PTTYPE ==PTTYPE_EPT ) || (PTTYPE ==64 )/* Only the nested GPA walk yields an L2 GPA to log. */ if (w == &vcpu->arch.ngpa_walk &&kvm_nested_call (write_log_dirty)(vcpu, addr))
arch/x86/kvm/svm/nested.c +74 -1
127 lines not on this page · raw message
arch/x86/kvm/svm/svm.h +8 -0
@@ -196,6 +196,8 @@ struct vmcb_ctrl_area_cached {u64 exit_info_2;u32 exit_int_info;u32 exit_int_info_err; +u64 pml_addr; +u16 pml_index;u64 misc_ctl;u32 event_inj;u32 event_inj_err;
@@ -235,6 +237,7 @@ struct svm_nested_state {struct vmcb_save_area_cached save;bool initialized; +bool pml_full;/* * Indicates whether MSR bitmap for L2 needs to be rebuilt due to
@@ -657,6 +660,11 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm)return svm->nested.ctl.misc_ctl &SVM_MISC_ENABLE_NP ; } +static inline bool nested_pml_enabled (struct vcpu_svm *svm) +{ +return svm->nested.ctl.misc_ctl &SVM_MISC_ENABLE_PML ; +} +static inline bool l2_has_separate_pat (struct kvm_vcpu *vcpu) {/*
Signature · 1 line
2.48.1
[email protected]
AI reviewRe: [RFC PATCH 2/5] KVM: nSVM: Add nested PML emulation
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] kvm: nSVM: spurious PML_FULL VM-Exit injection on discarded faults --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kvm: nSVM: spurious PML_FULL VM-Exit injection on discarded faults --
commit ac3a6ec9423b67a67c9dfb9dd5ebd698f451f130
Author: Nikunj A Dadhania <[email protected]>
KVM: nSVM: Add nested PML emulation
This commit emulates Page Modification Logging (PML) for nested SVM guests, allowing L1 hypervisors to track dirty pages in their L2 guests. KVM emulates PML in software by write-protecting L2 SPTEs and logging dirty GPAs to L1's PML buffer.
Quoted 4 lines
arch/x86/kvm/svm/nested.c +0
-0
[ ... ]
Quoted 6 lines
arch/x86/kvm/svm/nested.c +1
-1@@ -63,7 +64,14 @@ static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu,if (WARN_ON_ONCE (hweight64 (fault_stage) !=1 )) fault_stage =PFERR_GUEST_FINAL_MASK ; - vmcb->control.exit_code =SVM_EXIT_NPF ; +/*
arch/x86/kvm/svm/nested.c static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu, · line 77
Quoted 10 lines
arch/x86/kvm/svm/nested.c +7
-0+* Only the exit code differs for a PML Full; hardware leaves the +* EXITINFO fields as the nested page fault that filled the buffer +* would set them. +*/ + vmcb->control.exit_code = svm->nested.pml_full ?SVM_EXIT_PML_FULL :SVM_EXIT_NPF ; + svm->nested.pml_full = false; + vmcb->control.exit_info_1 = fault_stage | (fault->error_code & ~PFERR_GUEST_FAULT_STAGE_MASK ); vmcb->control.exit_info_2 = fault->address;
[Severity: High]
If L2 triggers a legitimate but unrelated nested page fault (such as a read fault) after a previous discarded fault left pml_full set, does this blindly consume the lingering pml_full flag?
It looks like this would inject an SVM_EXIT_PML_FULL exit into L1 carrying the error code of a read fault. Since hardware PML only logs writes, could this architecturally impossible state crash or confuse the L1 hypervisor?
[ ... ]
Quoted 12 lines
arch/x86/kvm/svm/nested.c +8
-0@@ -2169,6 +2200,47 @@ static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa,return w->gva_to_gpa (vcpu, w, gpa, access, exception); } +static int nested_svm_write_pml_buffer (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +struct vcpu_svm *svm =to_svm (vcpu); +u16 index; +gpa_t dst; + +if (WARN_ON_ONCE (!is_guest_mode (vcpu)) || !nested_pml_enabled (svm)) +return 0 ;
arch/x86/kvm/svm/nested.c static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, · line 2220
Quoted 10 lines
arch/x86/kvm/svm/nested.c +10
-0+ +/* The flag can be set here if the fault that raised it was discarded. */ +if (svm->nested.pml_full) +return 1 ; + + index = svm->nested.ctl.pml_index; +if (index >=PML_LOG_NR_ENTRIES ) { + svm->nested.pml_full = true; +return 1 ; + }
[Severity: High]
If KVM resolves or discards the page fault internally without injecting it into L1, this leaves the pml_full flag set while L2 execution resumes.
The comment acknowledges that the flag can be left set by discarded faults. However, is it safe to leave it set if the next fault isn't guaranteed to be a write?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [RFC PATCH 2/5] KVM: nSVM: Add nested PML emulation
Quoted 36 lines · On 10/15/2026 4:33 PM, [email protected] wrote:
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kvm: nSVM: spuriousPML_FULLVM-Exit injection on discarded faults --commit
ac3a6ec9423b67a67c9dfb9dd5ebd698f451f130
Author: Nikunj A Dadhania <[email protected]>KVM: nSVM: Add nested PML emulation
This commit emulates Page Modification Logging (PML) for nested SVM guests, allowing L1 hypervisors to track dirty pages in their L2 guests. KVM emulates PML in software by write-protecting L2 SPTEs and logging dirty GPAs to L1's PML buffer.
Quoted 4 lines
arch/x86/kvm/svm/nested.c +0
-0[ ... ]
Quoted 13 lines
arch/x86/kvm/svm/nested.c +8
-1@@ -63,7 +64,14 @@ static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu,if (WARN_ON_ONCE (hweight64 (fault_stage) !=1 )) fault_stage =PFERR_GUEST_FINAL_MASK ; - vmcb->control.exit_code =SVM_EXIT_NPF ; +/* +* Only the exit code differs for a PML Full; hardware leaves the +* EXITINFO fields as the nested page fault that filled the buffer +* would set them. +*/ + vmcb->control.exit_code = svm->nested.pml_full ?SVM_EXIT_PML_FULL :SVM_EXIT_NPF ; + svm->nested.pml_full = false; +
Quoted 8 lines
arch/x86/kvm/svm/nested.c static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu, · line 77
Quoted 3 lines
arch/x86/kvm/svm/nested.c +0
-0vmcb->control.exit_info_1 = fault_stage | (fault->error_code & ~PFERR_GUEST_FAULT_STAGE_MASK ); vmcb->control.exit_info_2 = fault->address;[Severity: High]
If L2 triggers a legitimate but unrelated nested page fault (such as a read fault) after a previous discarded fault leftpml_fullset, does this blindly consume the lingeringpml_fullflag?
Yes. Both findings are about the same flag - one at the point it is consumed, one at the point it is left set - so this answers both.
nVMX does the same: vmx->nested.pml_full is set in nested_vmx_write_pml_buffer() and consumed unconditionally in nested_ept_inject_page_fault(), with the same discard paths in shared MMU code. I kept the flag here to match that rather than introduce a different mechanism in a first posting.
The effect is bounded: L1 takes one spurious PML Full, and as pml_full_interception() ignores the EXITINFO fields, L1 drains, re-enters L2 and gets the real fault on the retry.
The clean fix could be to carry the indication on the fault rather than on the vCPU - a synthetic PFERR bit, set by the walk that raised it and tested in nested_svm_inject_npf_exit(). Something like the below, roughly:
#define PFERR_PML_FULL BIT_ULL(50)
#define PFERR_SYNTHETIC_MASK (PFERR_IMPLICIT_ACCESS | \
PFERR_PRIVATE_ACCESS | PFERR_PML_FULL)
/* update_accessed_dirty_bits() */
if (w == &vcpu->arch.ngpa_walk &&
kvm_nested_call(write_log_dirty)(vcpu, addr))
return -EBUSY;
/* walk_addr_generic(), tagging the fault it is about to raise */
if (unlikely(ret == -EBUSY))
errcode |= PFERR_PML_FULL;
/* nested_svm_inject_npf_exit() */
pml_full = fault->error_code & PFERR_PML_FULL;
vmcb->control.exit_code = pml_full ? SVM_EXIT_PML_FULL : SVM_EXIT_NPF;It would also let nVMX drop its flag and the WARN_ON_ONCE in nested_ept_inject_page_fault().
Separately, Yosry asked on this thread about factoring nested_svm_write_pml_buffer() and its nVMX counterpart into common code. I will wait for Sean's input on both before reworking either.
Quoted 34 lines
It looks like this would inject an
SVM_EXIT_PML_FULLexit into L1 carrying the error code of a read fault. Since hardware PML only logs writes, could this architecturally impossible state crash or confuse the L1 hypervisor?[ ... ]
Quoted 12 lines
arch/x86/kvm/svm/nested.c +8
-0@@ -2169,6 +2200,47 @@ static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa,return w->gva_to_gpa (vcpu, w, gpa, access, exception); } +static int nested_svm_write_pml_buffer (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +struct vcpu_svm *svm =to_svm (vcpu); +u16 index; +gpa_t dst; + +if (WARN_ON_ONCE (!is_guest_mode (vcpu)) || !nested_pml_enabled (svm)) +return 0 ;arch/x86/kvm/svm/nested.c static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, · line 2220
Quoted 10 lines
arch/x86/kvm/svm/nested.c +10
-0+ +/* The flag can be set here if the fault that raised it was discarded. */ +if (svm->nested.pml_full) +return 1 ; + + index = svm->nested.ctl.pml_index; +if (index >=PML_LOG_NR_ENTRIES ) { + svm->nested.pml_full = true; +return 1 ; + }[Severity: High]
If KVM resolves or discards the page fault internally without injecting it into L1, this leaves thepml_fullflag set while L2 execution resumes.
Quoted 3 lines
The comment acknowledges that the flag can be left set by discarded faults. However, is it safe to leave it set if the next fault isn't guaranteed to be a write?
Regards
Nikunj
Re: [RFC PATCH 2/5] KVM: nSVM: Add nested PML emulation
In reply to Nikunj A DadhaniaQuoted 155 lines · Nikunj A Dadhania
Emulate Page Modification Logging (PML) for nested SVM guests, allowing L1 hypervisors to use PML to track dirty pages in their L2 guests. KVM emulates PML in software by write-protecting L2 SPTEs and logging dirty GPAs to L1's PML buffer via the
write_log_dirtyhook.
PML_FULLexits are prioritized over NPF exits to ensure L1 processes the PML buffer before continuing L2 execution.Clear any pending PML Full on nested VMRUN. The flag is derived from L1's log index, and a fault that KVM discards instead of injecting can leave it set after L1 has drained its buffer. No event is lost: if the buffer is genuinely full, the next dirty write raises PML Full again.
Assisted-by LLMSigned-off-by Nikunj A Dadhania <[email protected]>--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/svm/nested.c | 75 +++++++++++++++++++++++++++++++++- arch/x86/kvm/svm/svm.h | 8 ++++ 3 files changed, 83 insertions(+), 2 deletions(-)arch/x86/kvm/mmu/paging_tmpl.h +1
-1@@ -234,7 +234,7 @@ static int FNAME(update_accessed_dirty_bits)(struct kvm_vcpu *vcpu,if (level == walker->level && write_fault && !(pte &PT_GUEST_DIRTY_MASK )) {trace_kvm_mmu_set_dirty_bit (table_gfn, index,sizeof (pte)); -#if PTTYPE ==PTTYPE_EPT +#if (PTTYPE ==PTTYPE_EPT ) || (PTTYPE ==64 )/* Only the nested GPA walk yields an L2 GPA to log. */ if (w == &vcpu->arch.ngpa_walk &&kvm_nested_call (write_log_dirty)(vcpu, addr))arch/x86/kvm/svm/nested.c +62
-1@@ -17,6 +17,7 @@#include <linux/kvm_types.h> #include <linux/kvm_host.h> #include <linux/kernel.h> +#include <linux/mm.h> #include <asm/msr-index.h> #include <asm/debugreg.h> @@ -63,7 +64,14 @@ static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu,if (WARN_ON_ONCE (hweight64 (fault_stage) !=1 )) fault_stage =PFERR_GUEST_FINAL_MASK ; - vmcb->control.exit_code =SVM_EXIT_NPF ; +/* +* Only the exit code differs for a PML Full; hardware leaves the +* EXITINFO fields as the nested page fault that filled the buffer +* would set them. +*/ + vmcb->control.exit_code = svm->nested.pml_full ?SVM_EXIT_PML_FULL :SVM_EXIT_NPF ; + svm->nested.pml_full = false; + vmcb->control.exit_info_1 = fault_stage | (fault->error_code & ~PFERR_GUEST_FAULT_STAGE_MASK ); vmcb->control.exit_info_2 = fault->address;@@ -416,6 +424,16 @@ static bool nested_vmcb_check_controls(struct kvm_vcpu *vcpu, !kvm_vcpu_is_legal_gpa(vcpu, control->nested_cr3)))return false; +if (control->misc_ctl &SVM_MISC_ENABLE_PML ) { +if (CC (!(control->misc_ctl &SVM_MISC_ENABLE_NP ))) +return false; +if (CC (!IS_ALIGNED (control->pml_addr,PAGE_SIZE ))) +return false; +if (CC (!nested_svm_check_bitmap_pa (vcpu, control->pml_addr, +PAGE_SIZE ))) +return false; + } +if (CC (!nested_svm_check_bitmap_pa (vcpu, control->msrpm_base_pa,MSRPM_SIZE )))return false;@@ -556,6 +574,8 @@ void __nested_copy_vmcb_control_to_cache(struct kvm_vcpu *vcpu, to->nested_cr3 = from->nested_cr3; to->pause_filter_count = from->pause_filter_count; to->pause_filter_thresh = from->pause_filter_thresh; + to->pml_addr = from->pml_addr; + to->pml_index = from->pml_index;/* Copy asid here because nested_vmcb_check_controls() will check it */ to->asid = from->asid;@@ -909,6 +929,13 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm)if (pml) vmcb02->control.pml_index =-1 ; +/* +* Clear any pending PML Full, as L1's log index has just been +* reloaded from vmcb12. A fault that KVM discards instead of +* injecting can leave the flag set after L1 drained its buffer. +*/ + svm->nested.pml_full = false; +/* * Stash vmcb02's counter if the guest hasn't moved past the guilty * instruction; otherwise, reset the counter to '0'. @@ -1311,6 +1338,8 @@ static int nested_svm_vmexit_update_vmcb12(struct kvm_vcpu *vcpu) vmcb12->control.event_inj =0 ; vmcb12->control.event_inj_err =0 ; vmcb12->control.int_ctl = svm->nested.ctl.int_ctl; + vmcb12->control.pml_addr = svm->nested.ctl.pml_addr; + vmcb12->control.pml_index = svm->nested.ctl.pml_index; trace_kvm_nested_vmexit_inject(vmcb12->control.exit_code, vmcb12->control.exit_info_1,@@ -1886,6 +1915,8 @@ static void nested_copy_vmcb_cache_to_control(struct vmcb_control_area *dst, dst->misc_ctl2 = from->misc_ctl2; dst->pause_filter_count = from->pause_filter_count; dst->pause_filter_thresh = from->pause_filter_thresh; + dst->pml_addr = from->pml_addr; + dst->pml_index = from->pml_index;/* 'clean' and 'hv_enlightenments' are not changed by KVM */ }@@ -2169,6 +2200,47 @@ static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa,return w->gva_to_gpa (vcpu, w, gpa, access, exception); } +static int nested_svm_write_pml_buffer (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +struct vcpu_svm *svm =to_svm (vcpu); +u16 index; +gpa_t dst; + +if (WARN_ON_ONCE (!is_guest_mode (vcpu)) || !nested_pml_enabled (svm)) +return 0 ; + +/* The flag can be set here if the fault that raised it was discarded. */ +if (svm->nested.pml_full) +return 1 ; + + index = svm->nested.ctl.pml_index; +if (index >=PML_LOG_NR_ENTRIES ) { + svm->nested.pml_full = true; +return 1 ; + } + + gpa &= ~0xFFFULL ; + dst = svm->nested.ctl.pml_addr +sizeof (u64) * index; + +/* +* Drop the entry if the write fails. VMRUN only verifies that L1's +* buffer address is a legal GPA, not that it is backed, so a failure +* here means L1 has pointed PML at memory it does not have, and only +* L1's own dirty tracking suffers. +*/ +if (kvm_write_guest_page (vcpu->kvm,gpa_to_gfn (dst), &gpa, +offset_in_page (dst),sizeof (gpa)))
arch/x86/kvm/svm/nested.c static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, · line 2242
Quoted 10 lines
arch/x86/kvm/svm/nested.c +10
-0+return 0 ; + +/* +* Decrement PML_INDEX after logging the GPA (buffer fills backwards +* from index 511 to 0) +*/ + svm->nested.ctl.pml_index = index -1 ; + +return 0 ; +}
This implementation is essentially the same as Intel, with the main difference being where the data (pml_full, pml_address, pml_index) live. I wonder if there's a way to share the implementation. I don't have good ideas to be honest. One of them is getting pointers to these vendor-specific variables at the top of a vendor-neutral function and use them (ew?), or maybe have copies in vcpu->arch and sync them back to the vmcb12/vmcs12 at some point (less ew?).
I always fail at predicting what Sean will think, but I think I am 90% sure he won't like either of these ideas, so hopefully he will come up with something better!
Re: [RFC PATCH 2/5] KVM: nSVM: Add nested PML emulation
Quoted 42 lines · Yosry Ahmed
Quoted 40 lines · Nikunj A Dadhania
arch/x86/kvm/svm/nested.c +36
-0@@ -2169,6 +2200,47 @@ static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa,return w->gva_to_gpa (vcpu, w, gpa, access, exception); } +static int nested_svm_write_pml_buffer (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +struct vcpu_svm *svm =to_svm (vcpu); +u16 index; +gpa_t dst; + +if (WARN_ON_ONCE (!is_guest_mode (vcpu)) || !nested_pml_enabled (svm)) +return 0 ; + +/* The flag can be set here if the fault that raised it was discarded. */ +if (svm->nested.pml_full) +return 1 ; + + index = svm->nested.ctl.pml_index; +if (index >=PML_LOG_NR_ENTRIES ) { + svm->nested.pml_full = true; +return 1 ; + } + + gpa &= ~0xFFFULL ; + dst = svm->nested.ctl.pml_addr +sizeof (u64) * index; + +/* +* Drop the entry if the write fails. VMRUN only verifies that L1's +* buffer address is a legal GPA, not that it is backed, so a failure +* here means L1 has pointed PML at memory it does not have, and only +* L1's own dirty tracking suffers. +*/ +if (kvm_write_guest_page (vcpu->kvm,gpa_to_gfn (dst), &gpa, +offset_in_page (dst),sizeof (gpa))) +return 0 ; + +/* +* Decrement PML_INDEX after logging the GPA (buffer fills backwards +* from index 511 to 0) +*/
Quoted 8 lines
arch/x86/kvm/svm/nested.c static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, · line 2242
Quoted 4 lines
arch/x86/kvm/svm/nested.c +4
-0+ svm->nested.ctl.pml_index = index -1 ; + +return 0 ; +}This implementation is essentially the same as Intel, with the main difference being where the data (
pml_full,pml_address,pml_index) live. I wonder if there's a way to share the implementation.
I did look at this. The genuinely common part is just the range check, the write and the decrement, ~10 lines; the enabled check, the full flag and the exit synthesis all differ.
Quoted 4 lines
I don't> have good ideas to be honest. One of them is getting pointers to these vendor-specific variables at the top of a vendor-neutral function and use them (ew?), or maybe have copies in
vcpu->archand sync them back to the vmcb12/vmcs12 at some point (less ew?).
I would steer away from copies in vcpu->arch though: svm->nested.ctl is already a copy of vmcb12 that gets synced back, while vmcs12 is written in place, so the copies would sit at different depths per vendor.
Quoted 3 lines
I always fail at predicting what Sean will think, but I think I am 90% sure he won't like either of these ideas, so hopefully he will come up with something better!
It feels like more churn than it is worth, but I am open to it. Let us see what Sean thinks.
Regards
Nikunj
RFC PATCH 3/5 KVM: nSVM: Advertise PML to L1
In reply to Nikunj A DadhaniaEnumerate PML to L1 via CPUID 0x8000000A ECX[4] so that an L1 hypervisor can use PML to track dirty pages in its L2 guests, and clear the capability when the host lacks PML in hardware.
Honor the capability on VMRUN by stripping PML from L1's VMCB when the feature is not exposed to the guest.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/cpuid.c | 6 +++++- arch/x86/kvm/reverse_cpuid.h | 5 +++++ arch/x86/kvm/svm/nested.c | 3 +++ arch/x86/kvm/svm/svm.c | 3 +++ 5 files changed, 17 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -711,6 +711,7 @@ enum kvm_only_cpuid_leafs {CPUID_7_1_ECX ,CPUID_1E_1_EAX ,CPUID_24_1_ECX , +CPUID_8000_000A_ECX ,NR_KVM_CPU_CAPS , NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS,
arch/x86/kvm/cpuid.c +5 -1
@@ -1195,6 +1195,10 @@ void kvm_initialize_cpu_caps(void)kvm_cpu_cap_set (X86_FEATURE_VIRT_SSBD );/* All SVM features required additional vendor module enabling. */ +kvm_cpu_cap_init (CPUID_8000_000A_ECX , + SCATTERED_F(PML), + ); + kvm_cpu_cap_init(CPUID_8000_000A_EDX, VENDOR_F(NPT), VENDOR_F(VMCBCLEAN),
@@ -1848,7 +1852,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) entry->eax =1 ;/* SVM revision 1 */ entry->ebx =8 ;/* Lets support 8 ASIDs in case we add proper ASID emulation to nested SVM */ - entry->ecx =0 ;/* Reserved */ +cpuid_entry_override (entry,CPUID_8000_000A_ECX );cpuid_entry_override (entry,CPUID_8000_000A_EDX );break ;case 0x80000019 :
arch/x86/kvm/reverse_cpuid.h +5 -0
@@ -71,6 +71,9 @@/* CPUID level 0x80000007 (EDX). */ #define KVM_X86_FEATURE_CONSTANT_TSC KVM_X86_FEATURE(CPUID_8000_0007_EDX, 8) +/* CPUID level 0x8000000A (ECX) */ +#define KVM_X86_FEATURE_PML KVM_X86_FEATURE(CPUID_8000_000A_ECX, 4) +/* CPUID level 0x80000022 (EAX) */ #define KVM_X86_FEATURE_PERFMON_V2 KVM_X86_FEATURE(CPUID_8000_0022_EAX, 0)
@@ -111,6 +114,7 @@ static const struct cpuid_reg reverse_cpuid[] = { [CPUID_7_1_ECX] = { 7, 1, CPUID_ECX}, [CPUID_1E_1_EAX] = { 0x1e, 1, CPUID_EAX}, [CPUID_24_1_ECX] = { 0x24, 1, CPUID_ECX}, + [CPUID_8000_000A_ECX] = {0x8000000a, 0, CPUID_ECX}, };/*
@@ -153,6 +157,7 @@ static __always_inline u32 __feature_translate(int x86_feature)KVM_X86_TRANSLATE_FEATURE (TSA_SQ_NO );KVM_X86_TRANSLATE_FEATURE (TSA_L1_NO );KVM_X86_TRANSLATE_FEATURE (MSR_IMM ); +KVM_X86_TRANSLATE_FEATURE (PML );default :return x86_feature; }
arch/x86/kvm/svm/nested.c +3 -0
@@ -552,6 +552,9 @@ void __nested_copy_vmcb_control_to_cache(struct kvm_vcpu *vcpu,if (!gmet_enabled || !guest_cpu_cap_has (vcpu,X86_FEATURE_GMET )) to->misc_ctl &= ~SVM_MISC_ENABLE_GMET ; +if (!pml || !guest_cpu_cap_has (vcpu,X86_FEATURE_PML )) + to->misc_ctl &= ~SVM_MISC_ENABLE_PML ; +if (!guest_cpu_cap_has (vcpu,X86_FEATURE_LBRV )) to->misc_ctl2 &= ~SVM_MISC2_ENABLE_V_LBR ;
arch/x86/kvm/svm/svm.c +3 -0
@@ -5751,6 +5751,9 @@ static __init void svm_set_cpu_caps(void)if (!lbrv)kvm_cpu_cap_clear (X86_FEATURE_BUS_LOCK_DETECT ); +if (!pml) +kvm_cpu_cap_clear (X86_FEATURE_PML ); +kvm_setup_xss_caps ();kvm_finalize_cpu_caps (); }
Signature · 1 line
2.48.1
RFC PATCH 4/5 KVM: selftests: nVMX: Add nested PML coverage to nested_dirty_log_test
In reply to Nikunj A DadhaniaAdd coverage for nested PML, i.e. for KVM logging L2's dirty GPAs into L1's PML buffer on L1's behalf. Verify each page is logged exactly once, no matter how many times L2 writes to it, and that L1 takes a PML_FULL exit when the buffer fills, can drain it, and can resume L2 without losing entries.
Seed PML_INDEX past the end of the buffer so that L2's first write takes a PML_FULL exit before anything is logged. This keeps the number of exits deterministic, and leaves the buffer contents identical to a run that never fills, so the logging checks apply unchanged.
--- tools/testing/selftests/kvm/include/x86/vmx.h | 5 + tools/testing/selftests/kvm/lib/x86/vmx.c | 18 +++ .../selftests/kvm/x86/nested_dirty_log_test.c | 149 +++++++++++++++++- 3 files changed, 171 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/include/x86/vmx.h +5 -0
@@ -471,6 +471,10 @@ struct vmx_pages {void *stack;u64 eptp_gpa; + +/* PML buffer */ +void *pml; +u64 pml_gpa; };union vmx_basic {
@@ -515,6 +519,7 @@ static inline bool kvm_cpu_has_vmx_virtualize_apic_accesses(void)return kvm_cpu_has_secondary_exec_control (SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ); } +bool kvm_cpu_has_vmx_pml (void );void vm_enable_ept (struct kvm_vm *vm);void prepare_virtualize_apic_accesses (struct vmx_pages *vmx,struct kvm_vm *vm);
tools/testing/selftests/kvm/lib/x86/vmx.c +18 -0
@@ -118,6 +118,9 @@ vcpu_alloc_vmx(struct kvm_vm *vm, gva_t *p_vmx_gva) vmx->stack = (void *)vm_alloc_stack (vm,1 ); + vmx->pml = (void *)vm_alloc_page (vm); + vmx->pml_gpa =addr_gva2gpa (vm, (uintptr_t )vmx->pml); +if (vm->stage2_mmu.pgd_created) vmx->eptp_gpa = vm->stage2_mmu.pgd;
@@ -437,6 +440,21 @@ bool kvm_cpu_has_secondary_exec_control(u32 ctrl)return ctrl_msr & ctrl; } +bool kvm_cpu_has_vmx_pml (void ) +{ +u64 ctrl; + +if (!kvm_cpu_has (X86_FEATURE_VMX )) +return false; + + ctrl =kvm_get_feature_msr (MSR_IA32_VMX_TRUE_PROCBASED_CTLS ) >>32 ; +if (!(ctrl &CPU_BASED_ACTIVATE_SECONDARY_CONTROLS )) +return false; + + ctrl =kvm_get_feature_msr (MSR_IA32_VMX_PROCBASED_CTLS2 ) >>32 ; +return ctrl &SECONDARY_EXEC_ENABLE_PML ; +} +void prepare_virtualize_apic_accesses (struct vmx_pages *vmx,struct kvm_vm *vm) { vmx->apic_access = (void *)vm_alloc_page (vm);
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +148 -1
162 lines not on this page · raw message
Signature · 1 line
2.48.1
[email protected]
AI reviewRe: [RFC PATCH 4/5] KVM: selftests: nVMX: Add nested PML coverage to nested_dirty_log_test
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] selftests: kvm: nVMX test asserts on invalid guest PML index --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] selftests: kvm: nVMX test asserts on invalid guest PML index
--
commit 88c8168ec493a327cfe182178b11f281733406e7
Author: Nikunj A Dadhania <[email protected]>
KVM: selftests: nVMX: Add nested PML coverage to nested_dirty_log_test
This patch adds coverage for nested PML to nested_dirty_log_test. It verifies that L1 takes a PML_FULL exit when the buffer fills, can drain it, and can resume L2 without losing entries.
Quoted 4 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +0
-0
[ ... ]
Quoted 8 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +4
-0@@ -192,6 +192,149 @@ static void test_handle_ucall_sync(struct kvm_vm *vm, u64 arg, } } +#define PML_LOG_NR_ENTRIES 512 +#define PML_HEAD_INDEX (PML_LOG_NR_ENTRIES - 1) + +#define PML_FULL_MAX_EXITS 64
[ ... ]
Quoted 17 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +17
-0+static void l1_pml_vmx_code (struct vmx_pages *vmx) +{ +int nr_pml_full =0 ; + +GUEST_ASSERT (vmx->vmcs_gpa); +prepare_for_vmx_operation (vmx); +load_vmcs (vmx); + +/* PML requires EPT. */ +GUEST_ASSERT (vmx->eptp_gpa); + +prepare_vmcs (vmx, l2_pml_guest_code); + +vmwrite (PML_ADDRESS , vmx->pml_gpa); +vmwrite (GUEST_PML_INDEX ,PML_LOG_NR_ENTRIES ); +vmwrite (SECONDARY_VM_EXEC_CONTROL , +vmread (SECONDARY_VM_EXEC_CONTROL ) |SECONDARY_EXEC_ENABLE_PML );
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c static void test_handle_ucall_sync(struct kvm_vm *vm, u64 arg, · line 242
Quoted 10 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +10
-0+ +vmlaunch (); + +for (;;) { +u64 reason =vmread (VM_EXIT_REASON ); + +if (reason ==EXIT_REASON_VMCALL ) +break ; + +GUEST_ASSERT_EQ (reason,EXIT_REASON_PML_FULL );
[Severity: Medium]
Since the test intentionally sets GUEST_PML_INDEX to 512, shouldn't this result in a VM-entry failure instead of a successful entry followed by a PML_FULL exit?
According to the Intel SDM (Volume 3C, Section 26.3.1.5 "Checks on Guest Non-Register State"), the guest PML index must be less than 512 when PML is enabled.
Does this assertion rely on KVM improperly omitting the guest state check? If KVM is updated in the future to correctly emulate the architecturally mandated check and fail the VM-entry, will this assertion break the test suite?
Quoted 9 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +9
-0+GUEST_ASSERT (++nr_pml_full <PML_FULL_MAX_EXITS ); + +vmwrite (GUEST_PML_INDEX ,PML_HEAD_INDEX ); +vmresume (); + } + +GUEST_SYNC_ARGS (vmread (GUEST_PML_INDEX ), nr_pml_full,0 ,0 ,0 ); +GUEST_DONE (); +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [RFC PATCH 4/5] KVM: selftests: nVMX: Add nested PML coverage to nested_dirty_log_test
Quoted 57 lines · On 10/15/2026 4:34 PM, [email protected] wrote:
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] selftests: kvm: nVMX test asserts on invalid guest PML index
--commit
88c8168ec493a327cfe182178b11f281733406e7
Author: Nikunj A Dadhania <[email protected]>KVM: selftests: nVMX: Add nested PML coverage to
nested_dirty_log_testThis patch adds coverage for nested PML to
nested_dirty_log_test. It verifies that L1 takes aPML_FULLexit when the buffer fills, can drain it, and can resume L2 without losing entries.Quoted 4 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +0
-0[ ... ]
Quoted 8 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +4
-0@@ -192,6 +192,149 @@ static void test_handle_ucall_sync(struct kvm_vm *vm, u64 arg, } } +#define PML_LOG_NR_ENTRIES 512 +#define PML_HEAD_INDEX (PML_LOG_NR_ENTRIES - 1) + +#define PML_FULL_MAX_EXITS 64[ ... ]
Quoted 24 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +24
-0+static void l1_pml_vmx_code (struct vmx_pages *vmx) +{ +int nr_pml_full =0 ; + +GUEST_ASSERT (vmx->vmcs_gpa); +prepare_for_vmx_operation (vmx); +load_vmcs (vmx); + +/* PML requires EPT. */ +GUEST_ASSERT (vmx->eptp_gpa); + +prepare_vmcs (vmx, l2_pml_guest_code); + +vmwrite (PML_ADDRESS , vmx->pml_gpa); +vmwrite (GUEST_PML_INDEX ,PML_LOG_NR_ENTRIES ); +vmwrite (SECONDARY_VM_EXEC_CONTROL , +vmread (SECONDARY_VM_EXEC_CONTROL ) |SECONDARY_EXEC_ENABLE_PML ); + +vmlaunch (); + +for (;;) { +u64 reason =vmread (VM_EXIT_REASON ); + +if (reason ==EXIT_REASON_VMCALL )
Quoted 8 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c static void test_handle_ucall_sync(struct kvm_vm *vm, u64 arg, · line 242
Quoted 3 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +3
-0+break ; + +GUEST_ASSERT_EQ (reason,EXIT_REASON_PML_FULL );[Severity: Medium]
Since the test intentionally setsGUEST_PML_INDEXto 512, shouldn't this result in a VM-entry failure instead of a successful entry followed by aPML_FULLexit?
It does not, and the test works as intended on both vendors. KVM does not check guest_pml_index at VM-entry; it is only consumed in nested_vmx_write_pml_buffer().
The seed is deliberate: on AMD the APM defines an index outside the valid range as the trigger for a PML Full exit, and the test uses the same setup for both vendors to get one deterministic PML_FULL.
Regards,
Nikunj
RFC PATCH 5/5 KVM: selftests: nSVM: Add nested PML coverage to nested_dirty_log_test
In reply to Nikunj A DadhaniaExtend the nested PML test to cover nSVM.
--- .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/include/x86/svm.h | 6 ++- .../selftests/kvm/include/x86/svm_util.h | 4 ++ tools/testing/selftests/kvm/lib/x86/svm.c | 3 ++ .../selftests/kvm/x86/nested_dirty_log_test.c | 48 +++++++++++++++++-- 5 files changed, 58 insertions(+), 4 deletions(-)
tools/testing/selftests/kvm/include/x86/processor.h +1 -0
@@ -219,6 +219,7 @@ struct kvm_x86_cpu_feature {#define X86_FEATURE_RDPRU KVM_X86_CPU_FEATURE(0x80000008, 0, EBX, 4)#define X86_FEATURE_AMD_IBPB KVM_X86_CPU_FEATURE(0x80000008, 0, EBX, 12)#define X86_FEATURE_EFER_LMSLE_MBZ KVM_X86_CPU_FEATURE(0x80000008, 0, EBX, 20) +#define X86_FEATURE_PML KVM_X86_CPU_FEATURE(0x8000000A, 0, ECX, 4)#define X86_FEATURE_NPT KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 0)#define X86_FEATURE_LBRV KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 1)#define X86_FEATURE_NRIPS KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 3)
tools/testing/selftests/kvm/include/x86/svm.h +5 -1
@@ -115,7 +115,10 @@ struct __attribute__ ((__packed__)) vmcb_control_area {u64 avic_physical_id;/* Offset 0xf8 */ u8 reserved_7[8 ];u64 vmsa_pa;/* Used for an SEV-ES guest */ -u8 reserved_8[720 ]; +u8 reserved_8[184 ];/* Offset 0x110 */ +u64 pml_addr;/* Offset 0x1c8 */ +u16 pml_index;/* Offset 0x1d0 */ +u8 reserved_9[526 ];/* Offset 0x1d2 */ /* * Offset 0x3e0, 32 bytes reserved * for use by hypervisor/software.
@@ -174,6 +177,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {#define SVM_MISC_ENABLE_NP BIT(0)#define SVM_MISC_ENABLE_SEV BIT(1) +#define SVM_MISC_ENABLE_PML BIT(11)#define SVM_MISC2_ENABLE_V_LBR BIT_ULL(0)#define SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE BIT_ULL(1)
tools/testing/selftests/kvm/include/x86/svm_util.h +4 -0
@@ -33,6 +33,10 @@ struct svm_test_data {/* NPT */ u64 ncr3_gpa; + +/* PML buffer */ +void *pml;/* gva */ +u64 pml_gpa; };static inline void vmmcall (void )
tools/testing/selftests/kvm/lib/x86/svm.c +3 -0
@@ -45,6 +45,9 @@ vcpu_alloc_svm(struct kvm_vm *vm, gva_t *p_svm_gva) svm->stack = (void *)vm_alloc_stack (vm,1 ); + svm->pml = (void *)vm_alloc_page (vm); + svm->pml_gpa =addr_gva2gpa (vm, (uintptr_t )svm->pml); +if (vm->stage2_mmu.pgd_created) svm->ncr3_gpa = vm->stage2_mmu.pgd;
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +45 -3
68 lines not on this page · raw message
Signature · 1 line
2.48.1
[email protected]
AI reviewRe: [RFC PATCH 5/5] KVM: selftests: nSVM: Add nested PML coverage to nested_dirty_log_test
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] selftests: kvm: l2_pml_guest_code executes VMX-specific vmcall() on SVM --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] selftests: kvm: l2_pml_guest_code executes VMX-specific vmcall() on SVM --
commit cb64ef5e62e8854d8adff372ccbc0426babe6f16
Author: Nikunj A Dadhania <[email protected]>
KVM: selftests: nSVM: Add nested PML coverage to nested_dirty_log_test
This commit extends the existing nested PML selftest to also cover AMD's nested Page Modification Logging (nSVM PML) by introducing an SVM-specific L1 guest code path.
Quoted 7 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +0
-0@@ -250,6 +250,45 @@ static void l1_pml_vmx_code(struct vmx_pages *vmx)GUEST_DONE (); }
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c static void l1_pml_vmx_code(struct vmx_pages *vmx) · line 261
Quoted 10 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +9
-0+static void l1_pml_svm_code (struct svm_test_data *svm) +{ +struct vmcb_control_area *ctrl = &svm->vmcb->control; +int nr_pml_full =0 ; + +/* PML requires NPT. */ +GUEST_ASSERT (svm->ncr3_gpa); + +generic_svm_setup (svm, l2_pml_guest_code);
[Severity: High]
Could this setup lead to an Invalid Opcode (#UD) exception on AMD systems?
Looking at l2_pml_guest_code(), it appears to unconditionally execute the vmcall() instruction:
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c:l2_pml_guest_code() {
...
vmcall();
...
}If l2_pml_guest_code() emits the Intel-specific VMCALL instruction, won't executing VMCALL on AMD systems raise an Invalid Opcode (#UD) exception instead of causing a VM-exit to L1?
Since generic_svm_setup() does not intercept #UD, does this exception get delivered to the L2 guest, triggering the host's selftest abort handler and crashing the nested AMD guest?
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c static void l1_pml_vmx_code(struct vmx_pages *vmx) · line 271
Quoted 10 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +10
-0+ + ctrl->pml_addr = svm->pml_gpa; + ctrl->pml_index =PML_LOG_NR_ENTRIES ; + ctrl->misc_ctl |=SVM_MISC_ENABLE_PML ; + +run_guest (svm->vmcb, svm->vmcb_gpa); + +for (;;) { +if (ctrl->exit_code ==SVM_EXIT_VMMCALL ) +break ;
[Severity: High]
Will this loop exit condition ever be met?
If l2_pml_guest_code() crashes with a #UD exception before completing the hypercall, L1 won't receive a clean SVM_EXIT_VMMCALL, causing the test to fail on AMD.
Would it be better to use a generic wrapper (like vmmcall()) in L2 that handles both VMX and SVM properly?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Re: [RFC PATCH 5/5] KVM: selftests: nSVM: Add nested PML coverage to nested_dirty_log_test
Quoted 39 lines · On 10/15/2026 4:34 PM, [email protected] wrote:
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] selftests: kvm:l2_pml_guest_codeexecutes VMX-specificvmcall()on SVM --commit
cb64ef5e62e8854d8adff372ccbc0426babe6f16
Author: Nikunj A Dadhania <[email protected]>KVM: selftests: nSVM: Add nested PML coverage to
nested_dirty_log_testThis commit extends the existing nested PML selftest to also cover AMD's nested Page Modification Logging (nSVM PML) by introducing an SVM-specific L1 guest code path.
Quoted 7 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +0
-0@@ -250,6 +250,45 @@ static void l1_pml_vmx_code(struct vmx_pages *vmx)GUEST_DONE (); }tools/testing/selftests/kvm/x86/nested_dirty_log_test.c static void l1_pml_vmx_code(struct vmx_pages *vmx) · line 261
Quoted 10 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +9
-0+static void l1_pml_svm_code (struct svm_test_data *svm) +{ +struct vmcb_control_area *ctrl = &svm->vmcb->control; +int nr_pml_full =0 ; + +/* PML requires NPT. */ +GUEST_ASSERT (svm->ncr3_gpa); + +generic_svm_setup (svm, l2_pml_guest_code);[Severity: High]
Could this setup lead to an Invalid Opcode (#UD) exception on AMD systems?Looking at
l2_pml_guest_code(), it appears to unconditionally execute thevmcall()instruction:
Quoted 8 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c:l2_pml_guest_code() { ... vmcall(); ... }Quoted 1 line
If
l2_pml_guest_code()emits the Intel-specific VMCALL instruction, won'texecuting VMCALL on AMD systems raise an Invalid Opcode (#UD) exception instead of causing a VM-exit to L1?
VMCALL does #UD on AMD, but KVM intercepts #UD and the emulator rewrites the instruction to VMMCALL and re-executes it, see emulator_fix_hypercall() and svm_patch_hypercall(). L1 intercepts VMMCALL, so L2 exits to L1 with SVM_EXIT_VMMCALL. l2_guest_code() in this file already uses vmcall().
Quoted 5 lines
Since
generic_svm_setup()does not intercept #UD, does this exception get delivered to the L2 guest, triggering the host's selftest abort handler and crashing the nested AMD guest?
Quoted 13 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c static void l1_pml_vmx_code(struct vmx_pages *vmx) · line 271
Quoted 10 lines
tools/testing/selftests/kvm/x86/nested_dirty_log_test.c +10
-0+ + ctrl->pml_addr = svm->pml_gpa; + ctrl->pml_index =PML_LOG_NR_ENTRIES ; + ctrl->misc_ctl |=SVM_MISC_ENABLE_PML ; + +run_guest (svm->vmcb, svm->vmcb_gpa); + +for (;;) { +if (ctrl->exit_code ==SVM_EXIT_VMMCALL ) +break ;[Severity: High]
Will this loop exit condition ever be met?
Yes, for the reason above. l1_svm_code() predates this series and already breaks on SVM_EXIT_VMMCALL.
Regards,
Nikunj