KVM: x86: Introduce memory protection attributes
Subsystems (10)
Paolo Bonzini
maintainerv4 cover KVM: x86: Introduce memory protection attributes
This series introduces a mechanism to let userspace block read, write or execute access to individual GFNs via KVM's memory attribute mechanism, and have them reported via KVM_EXIT_MEMORY_FAULT.
KVM needs to check the attributes anytime KVM takes GPAs as input for any action initiated by the guest; if the memory attributes are incompatible with such action, it should be stopped. This means that there are quite a lot of cases to handle, and while in most cases they occur before committing to the execution of an instructions (for example, page walks), there are other cases where a full implementation of attributes would basically require an implementation of transactional memory accesses:
notably (from inevitable to where I finally drew the line) nested virtualization, emulation, SMM switch, task switch. In the worst case, the processor might have executed the instruction before issuing a trap-like vmexit - and if the instruction length is not provided and the instruction is not accessible, there's no practical alternative to returning -EIO and marking the VM as dead.
For this reason, documentation now clarifies that invalid accesses may not be recoverable and may be reported in the same way as MMIO.
Along the way, the patches fix some issues in the memory attributes code, that surfaced due to having more than one attribute. Still, no matter how careful I tried to be, I'm ready for the absolute massacre that Sashiko will make of this stuff.
Paolo
v3->v4:
- New patch "KVM: nVMX/nSVM: check unconditionally whether CR3 is valid"
- New patch "KVM: x86/emulate: Check TSS busy-bit writes during task switches"
- Rebase on top of new VM memory attributes code
- Use kvm_check_gen in Longarch
- New patches to surface memory faults from the emulator and from nested GPA translation (15, 29-36, 38-39)
- Clarify in the docs that memory faults in some cases may not be recoverable
Anish Moorthy (1):
KVM: Define and communicate KVM_EXIT_MEMORY_FAULT RWX flags to
userspace
Nicolas Saenz Julienne (11):
KVM: selftests: Take into account mixed memory fault flags
KVM: x86/mmu: Init memslot hugepage information for non-private_mem
VMs too
KVM: Introduce NR/NW/NX memory attributes
KVM: Include memory protections in result of gfn->hva conversion
KVM: Take memory protections into account for memory read/write/fetch
KVM: Encapsulate memattrs array into anonymous struct
KVM: Introduce a generation number for memory attributes
KVM: Take memory protections into account for accesses with cached
gfn->hva
KVM: pfncache: Fail to refresh if it contains memory protections
KVM: x86/mmu: Do not prefetch sptes on gfns backed by memory
attributes
KVM: x86/mmu: Take memory protection attributes into account during
faults
Paolo Bonzini (33):
KVM: nVMX/nSVM: check unconditionally whether CR3 is valid
KVM: x86/hyperv: do not overwrite hc->ingpa for SIGNAL_EVENT hypercall
KVM: x86: apply nGPA->GPA translation to KVM_HC_CLOCK_PAIRING
KVM: x86/emulate: Check TSS busy-bit writes during task switches
KVM: loongarch: do full validity check on the gfn-to-hva cache
KVM: selftests: Test address translation for Hyper-V direct L2
hypercalls
selftests: kvm: drop TEST_ASSERT_SUPPORTED_ATTRIBUTES
KVM: x86: Introduce memory fault on invalid hypercalls reads/writes
KVM: selftests: test hypercall memory fault exits
KVM: x86/mmu: intersect writability from __kvm_faultin_pfn with
fault->map_writable
KVM: x86/mmu: Extend map_writable to a full ACC_* mask
KVM: VMX: return error from kvm_set_cr*
KVM: adjust for presence of more than one attribute
KVM: Introduce kvm_fetch_guest_page() and use it for x86
KVM: Take memory protections into account for __kvm_vcpu_map
KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen()
KVM: x86/emulate: change return value of emulator_io_permitted to int
KVM: x86/emulate: Propagate memory faults to userspace
KVM: x86: "Handle" failed decode when skipping instructions
KVM: vmx/sgx: make return value of sgx_gva_to_gpa consistent with its
callers
KVM: x86: Allow page fault fixup to return errors
KVM: x86/mmu: Adjust gva_to_gpa to allow returning an error
KVM: x86/mmu: Propagate errors from nested GPA translation
KVM: x86: Propagate memory faults through load_pdptrs()
KVM: x86/mmu: Issue memory fault exit if walk failed due to memory
attribute
KVM: x86: mark VM as dead after KVM_SET_SREGS fails to load PDPTRs
KVM: VMX: mark VM dead on shadow VMCS or eVMCS access failures
KVM: let kvm_arch_post_set_memory_attributes drop mmu_lock
KVM: x86/mmu: Obsolete all roots if memattr contains gPTEs
KVM: x86: selftests: Introduce memory protection attributes test
KVM: x86: selftests: Introduce memory attributes PTE test
KVM: x86: selftests: Introduce memory attributes side-channel tests
KVM: x86: split out parts of load_pdptrs
Documentation/virt/kvm/api.rst | 57 ++- arch/loongarch/kvm/vcpu.c | 12 +- arch/x86/include/asm/kvm_host.h | 18 +- arch/x86/kvm/Kconfig | 2 +- arch/x86/kvm/emulate.c | 82 +++- arch/x86/kvm/hyperv.c | 174 +++++-- arch/x86/kvm/kvm_emulate.h | 3 + arch/x86/kvm/mmu.h | 28 +- arch/x86/kvm/mmu/mmu.c | 200 +++++++-- arch/x86/kvm/mmu/mmu_internal.h | 21 +- arch/x86/kvm/mmu/mmutrace.h | 36 ++ arch/x86/kvm/mmu/paging_tmpl.h | 63 ++- arch/x86/kvm/mmu/spte.c | 12 +- arch/x86/kvm/mmu/spte.h | 13 +- arch/x86/kvm/mmu/tdp_mmu.c | 2 +- arch/x86/kvm/regs.c | 103 +++-- arch/x86/kvm/regs.h | 9 + arch/x86/kvm/svm/nested.c | 51 +-- arch/x86/kvm/svm/svm.c | 19 +- arch/x86/kvm/vmx/hyperv.c | 7 +- arch/x86/kvm/vmx/nested.c | 53 ++- arch/x86/kvm/vmx/sgx.c | 35 +- arch/x86/kvm/vmx/vmx.c | 59 ++- arch/x86/kvm/x86.c | 204 ++++++--- arch/x86/kvm/x86.h | 2 +- include/linux/kvm_host.h | 133 +++++- include/linux/kvm_types.h | 6 +- include/trace/events/kvm.h | 14 +- include/uapi/linux/kvm.h | 7 + tools/include/uapi/linux/kvm.h | 3 + tools/testing/selftests/kvm/Makefile.kvm | 2 + .../testing/selftests/kvm/include/kvm_util.h | 37 +- .../selftests/kvm/include/x86/processor.h | 1 + .../testing/selftests/kvm/lib/x86/processor.c | 5 + .../testing/selftests/kvm/memory_attributes.c | 424 ++++++++++++++++++ tools/testing/selftests/kvm/x86/hcall_fault.c | 246 ++++++++++ .../testing/selftests/kvm/x86/hyperv_evmcs.c | 16 +- .../selftests/kvm/x86/hyperv_svm_test.c | 15 +- .../selftests/kvm/x86/memory_attributes.c | 415 +++++++++++++++++ .../kvm/x86/private_mem_kvm_exits_test.c | 5 +- virt/kvm/kvm_main.c | 315 +++++++++++-- virt/kvm/pfncache.c | 30 +- 42 files changed, 2498 insertions(+), 441 deletions(-) create mode 100644 tools/testing/selftests/kvm/memory_attributes.c create mode 100644 tools/testing/selftests/kvm/x86/hcall_fault.c create mode 100644 tools/testing/selftests/kvm/x86/memory_attributes.c
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 1/45 KVM: nVMX/nSVM: check unconditionally whether CR3 is valid
nested_vmcb_check_save() checks CR3 legality only if long mode is active, but it really needs to be checked in all cases - the check uses must be zero bits of the host, and indeed nested_svm_load_cr3() later will reject illegal CR3 in every mode. Move the check earlier, and thus out of nested_svm_load_cr3().
Likewise for nested_vmx_load_cr3() and nested_vmx_check_guest_state().
--- arch/x86/kvm/svm/nested.c | 9 +++------ arch/x86/kvm/vmx/nested.c | 8 ++------ 2 files changed, 5 insertions(+), 12 deletions(-)
arch/x86/kvm/svm/nested.c +3 -6
@@ -443,7 +443,8 @@ static bool nested_vmcb_check_save(struct kvm_vcpu *vcpu,return false;if (CC ((save->cr0 &X86_CR0_CD ) ==0 && (save->cr0 &X86_CR0_NW )) || -CC (save->cr0 & ~0xffffffffULL )) +CC (save->cr0 & ~0xffffffffULL ) || +CC (!kvm_vcpu_is_legal_cr3 (vcpu, save->cr3)))return false;if (CC (!kvm_dr6_valid (save->dr6)) ||CC (!kvm_dr7_valid (save->dr7)))
@@ -456,8 +457,7 @@ static bool nested_vmcb_check_save(struct kvm_vcpu *vcpu,*/ if ((save->efer &EFER_LME ) && (save->cr0 &X86_CR0_PG )) {if (CC (!(save->cr4 &X86_CR4_PAE )) || -CC (!(save->cr0 &X86_CR0_PE )) || -CC (!kvm_vcpu_is_legal_cr3 (vcpu, save->cr3))) +CC (!(save->cr0 &X86_CR0_PE )))return false; if (CC((save->cs.attrib & SVM_SELECTOR_L_MASK) &&
@@ -717,9 +717,6 @@ static void nested_svm_transition_tlb_flush(struct kvm_vcpu *vcpu)static int nested_svm_load_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3,bool nested_npt,bool reload_pdptrs) { -if (CC (!kvm_vcpu_is_legal_cr3 (vcpu, cr3))) -return -EINVAL ; -if (reload_pdptrs &&is_pae_paging (vcpu)) {if (nested_npt)kvm_register_mark_for_reload (vcpu,VCPU_REG_PDPTR );
arch/x86/kvm/vmx/nested.c +2 -6
@@ -1235,11 +1235,6 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool nested_ept, bool reload_pdptrs, enum vm_entry_failure_code *entry_failure_code) { - if (CC(!kvm_vcpu_is_legal_cr3(vcpu, cr3))) { - *entry_failure_code =ENTRY_FAIL_DEFAULT ; -return -EINVAL ; - } -/* * If PAE paging and EPT are both on, CR3 is not used by the CPU and * must not be dereferenced.
@@ -3323,7 +3318,8 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu, *entry_failure_code =ENTRY_FAIL_DEFAULT ;if (CC (!nested_guest_cr0_valid (vcpu, vmcs12->guest_cr0)) || -CC (!nested_guest_cr4_valid (vcpu, vmcs12->guest_cr4))) +CC (!nested_guest_cr4_valid (vcpu, vmcs12->guest_cr4)) || +CC (!kvm_vcpu_is_legal_cr3 (vcpu, vmcs12->guest_cr3)))return -EINVAL ;if (CC (vmcs12->guest_cr4 &X86_CR4_CET && !(vmcs12->guest_cr0 &X86_CR0_WP )))
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 01/45] KVM: nVMX/nSVM: check unconditionally whether CR3 is valid
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM: nSVM: missing L1 host CR3 check in nested_svm_vmexit() after RSM - [Medium] KVM: nSVM…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM: nSVM: missing L1 host CR3 check in nested_svm_vmexit() after RSM
- [Medium] KVM: nSVM: unvalidated L2 CR3 and dead check in svm_set_nested_state() --
commit 111aed734bf3c907c07b30f4fd256e2dd365f8e8
Author: Paolo Bonzini <[email protected]>
KVM: nVMX/nSVM: check unconditionally whether CR3 is valid
This commit moves the CR3 validity check out of nested_svm_load_cr3() and nested_vmx_load_cr3() into nested_vmcb_check_save() and nested_vmx_check_guest_state() so that CR3 is checked unconditionally in all processor modes during guest state validation.
Quoted 4 lines
arch/x86/kvm/svm/nested.c +0
-0
[ ... ]
arch/x86/kvm/svm/nested.c static void nested_svm_transition_tlb_flush(struct kvm_vcpu *vcpu) · line 722
Quoted 7 lines
arch/x86/kvm/svm/nested.c +0
-3@@ -717,9 +717,6 @@ static void nested_svm_transition_tlb_flush(struct kvm_vcpu *vcpu)static int nested_svm_load_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3,bool nested_npt,bool reload_pdptrs) { -if (CC (!kvm_vcpu_is_legal_cr3 (vcpu, cr3))) -return -EINVAL ; -
[Severity: High]
Does removing kvm_vcpu_is_legal_cr3() from nested_svm_load_cr3() allow an unvalidated L1 host CR3 to be loaded on nested #VMEXIT after returning from SMM?
When an SMI arrives while L2 is running, svm_enter_smm() saves L1's host state from vmcb01->save into guest memory at MSR_VM_HSAVE_PA + 0x400, where the SMM handler can modify the saved CR3. On RSM, svm_leave_smm() in arch/x86/kvm/svm/svm.c copies that HSAVE state back into vmcb01->save, but only validates vmcb12:
arch/x86/kvm/svm/svm.c:svm_leave_smm() {
...
svm_copy_vmrun_state(vcpu, &svm->vmcb01.ptr->save, m_save.map.hva + 0x400);
/*
* Enter the nested guest now
*/
vmcb_mark_all_dirty(svm->vmcb01.ptr);
vmcb12 = m.map.hva;
nested_copy_vmcb_control_to_cache(svm, &vmcb12->control);
nested_copy_vmcb_save_to_cache(svm, &vmcb12->save);
if (nested_svm_check_cached_vmcb12(vcpu) < 0)
return 1;
...
}On the subsequent nested #VMEXIT from L2 to L1, nested_svm_vmexit() in arch/x86/kvm/svm/nested.c reloads vmcb01->save.cr3 via nested_svm_load_cr3() and expects an invalid CR3 to trigger a triple fault:
arch/x86/kvm/svm/nested.c:nested_svm_vmexit() {
...
if (nested_svm_load_cr3(vcpu, vmcb01->save.cr3, false, true))
kvm_make_request(KVM_REQ_TRIPLE_FAULT, vcpu);
...
}Without kvm_vcpu_is_legal_cr3() in nested_svm_load_cr3(), if L1 is in 64-bit mode (or 32-bit non-PAE mode) so is_pae_paging(vcpu) is false, nested_svm_load_cr3() returns 0 and installs the illegal CR3 into vcpu->arch.cr3. When NPT is enabled, svm_load_mmu_pgd() then writes it into vmcb01->save.cr3, causing hardware VMRUN of L1 to fail with SVM_EXIT_ERR (KVM_EXIT_FAIL_ENTRY) instead of injecting KVM_REQ_TRIPLE_FAULT.
Should svm_leave_smm() validate the restored host state in vmcb01->save via nested_vmcb_check_save() (similar to how vmx_leave_smm() calls nested_vmx_check_restored_vmcs12()), or should nested_svm_vmexit() still verify kvm_vcpu_is_legal_cr3()?
[Severity: Medium]
Similarly, in svm_set_nested_state() in arch/x86/kvm/svm/nested.c, nested_vmcb_check_save() only validates L1's saved host state in user_vmcb->save (copied into save_cached), while L2's current state is held in vcpu->arch and checked separately at the top of svm_set_nested_state():
arch/x86/kvm/svm/nested.c:svm_set_nested_state() {
...
cr0 = kvm_read_cr0(vcpu);
if (((cr0 & X86_CR0_CD) == 0) && (cr0 & X86_CR0_NW))
goto out_free;
/*
* Validate host state saved from before VMRUN (see
* nested_svm_check_permissions). Note that the g_pat field is not
* validated, because (a) it may have been clobbered by SMM before
* KVM_GET_NESTED_STATE, and (b) it is not loaded at emulated
* #VMEXIT.
*/
__nested_copy_vmcb_save_to_cache(&save_cached, save);
if (!(save->cr0 & X86_CR0_PG) ||
!(save->cr0 & X86_CR0_PE) ||
(save->rflags & X86_EFLAGS_VM) ||
((save->efer & EFER_LMA) && !(save->efer & EFER_LME)) ||
!nested_vmcb_check_save(vcpu, &save_cached, false))
goto out_free;
...
ret = nested_svm_load_cr3(&svm->vcpu, vcpu->arch.cr3,
nested_npt_enabled(svm), false);
if (ret)
goto out_free;
...
}Because kvm_is_valid_sregs() in arch/x86/kvm/regs.c only checks kvm_vcpu_is_legal_cr3() when long mode is active, and because userspace may also update CPUID MAXPHYADDR between KVM_SET_SREGS and KVM_SET_NESTED_STATE, removing kvm_vcpu_is_legal_cr3() from nested_svm_load_cr3() leaves L2's vcpu->arch.cr3 unvalidated here. In addition, because svm_set_nested_state() passes reload_pdptrs = false to nested_svm_load_cr3(), nested_svm_load_cr3() can no longer return a non-zero value on this path, leaving the if (ret) check dead.
Should svm_set_nested_state() also check kvm_vcpu_is_legal_cr3(vcpu, vcpu->arch.cr3) alongside the L2 CR0 check?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Paolo Bonzini
maintainerPATCH v4 2/45 KVM: x86/hyperv: do not overwrite hc->ingpa for SIGNAL_EVENT hypercall
In reply to Paolo BonziniWhen a guest makes a HVCALL_SIGNAL_EVENT hypercall that is not fast (i.e. has arguments in memory), kvm_hvcall_signal_event() reads the connection ID and overwrites hc->ingpa.
If the connection ID that is valid in userspace but not registered in the kernel conn_to_evt, KVM exits to userspace; however, hc->param still signals that the hypercall was not fast, and userspace will then treat the connection ID as an address.
--- arch/x86/kvm/hyperv.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
arch/x86/kvm/hyperv.c +9 -6
@@ -2437,19 +2437,22 @@ static u16 kvm_hvcall_signal_event(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *h {struct kvm_hv *hv =to_kvm_hv (vcpu->kvm);struct eventfd_ctx *eventfd; +u64 conn_id;if (unlikely (!hc->fast)) {int ret;gpa_t gpa = hc->ingpa; - if ((gpa & (__alignof__(hc->ingpa) - 1)) || - offset_in_page(gpa) + sizeof(hc->ingpa) > PAGE_SIZE) +if ((gpa & (__alignof__(conn_id ) -1 )) || +offset_in_page (gpa) +sizeof (conn_id) >PAGE_SIZE )return HV_STATUS_INVALID_ALIGNMENT ; ret =kvm_vcpu_read_guest (vcpu, gpa, - &hc->ingpa,sizeof (hc->ingpa)); + &conn_id,sizeof (conn_id));if (ret <0 )return HV_STATUS_INVALID_ALIGNMENT ; + }else { + conn_id = hc->ingpa; }/*
@@ -2457,15 +2460,15 @@ static u16 kvm_hvcall_signal_event(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *h* have no use for it, and in all known usecases it is zero, so just * report lookup failure if it isn't. */ -if (hc->ingpa &0xffff00000000ULL ) +if (conn_id &0xffff00000000ULL )return HV_STATUS_INVALID_PORT_ID ;/* remaining bits are reserved-zero */ -if (hc->ingpa & ~KVM_HYPERV_CONN_ID_MASK ) +if (conn_id & ~KVM_HYPERV_CONN_ID_MASK )return HV_STATUS_INVALID_HYPERCALL_INPUT ;/* the eventfd is protected by vcpu->kvm->srcu, but conn_to_evt isn't */ rcu_read_lock (); - eventfd =idr_find (&hv->conn_to_evt, hc->ingpa); + eventfd =idr_find (&hv->conn_to_evt, conn_id);rcu_read_unlock ();if (!eventfd)return HV_STATUS_INVALID_PORT_ID ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 3/45 KVM: x86: apply nGPA->GPA translation to KVM_HC_CLOCK_PAIRING
In reply to Paolo BonziniKVM_HC_CLOCK_PAIRING writes to a guest physical addresses. Treat it as an L2 address when running nested, which is consistent with the Hyper-V TLB flush hypercalls for example.
This is technically backwards incompatible, but it should be extremely unlikely to happen in the wild and thus we can treat it as a bugfix, I think.
--- arch/x86/kvm/x86.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
arch/x86/kvm/x86.c +14 -6
@@ -7256,8 +7256,8 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, {struct kvm_clock_pairing clock_pairing;struct timespec64 ts; +size_t offset =0 ;u64 cycle; -int ret;if (clock_type !=KVM_CLOCK_PAIRING_WALLCLOCK )return -KVM_EOPNOTSUPP ;
@@ -7278,12 +7278,20 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, clock_pairing.flags =0 ;memset (&clock_pairing.pad,0 ,sizeof (clock_pairing.pad)); - ret =0 ; -if (kvm_write_guest (vcpu->kvm, paddr, &clock_pairing, -sizeof (struct kvm_clock_pairing ))) - ret = -KVM_EFAULT ; +while (offset <sizeof (clock_pairing)) { +gpa_t gpa =kvm_translate_gpa (vcpu, &vcpu->arch.gva_walk, paddr + offset, +PFERR_WRITE_MASK |PFERR_GUEST_FINAL_MASK ,NULL ,0 ); +size_t len =min_t (size_t,sizeof (clock_pairing) - offset, +PAGE_SIZE -offset_in_page (gpa)); - return ret; +if (gpa ==INVALID_GPA ) +return -KVM_EFAULT ; +if (kvm_write_guest (vcpu->kvm, gpa, (u8 *)&clock_pairing + offset, len)) +return -KVM_EFAULT ; + + offset += len; + } + return 0; }#endif
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 4/45 KVM: x86/emulate: Check TSS busy-bit writes during task switches
In reply to Paolo BonziniTask switching ignores failures when clearing the old TSS descriptor's busy bit and setting the new descriptor's busy bit. A memory attribute restriction can therefore drop either write while the task switch continues with inconsistent descriptor state.
Check both writes and stop the task switch on failure. Memory attribute faults reach the existing emulation-failure exit in kvm_task_switch().
--- arch/x86/kvm/emulate.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/emulate.c +6 -2
@@ -2925,7 +2925,9 @@ static int emulator_do_task_switch(struct x86_emulate_ctxt *ctxt,if (reason ==TASK_SWITCH_IRET || reason ==TASK_SWITCH_JMP ) { curr_tss_desc.type &= ~(1 <<1 );/* clear busy flag */ -write_segment_descriptor (ctxt, old_tss_sel, &curr_tss_desc); + ret =write_segment_descriptor (ctxt, old_tss_sel, &curr_tss_desc); +if (ret !=X86EMUL_CONTINUE ) +return ret; }if (reason ==TASK_SWITCH_IRET )
@@ -2949,7 +2951,9 @@ static int emulator_do_task_switch(struct x86_emulate_ctxt *ctxt,if (reason !=TASK_SWITCH_IRET ) { next_tss_desc.type |= (1 <<1 );/* set busy flag */ -write_segment_descriptor (ctxt, tss_selector, &next_tss_desc); + ret =write_segment_descriptor (ctxt, tss_selector, &next_tss_desc); +if (ret !=X86EMUL_CONTINUE ) +return ret; } ops->set_cr (ctxt,0 , ops->get_cr (ctxt,0 ) |X86_CR0_TS );
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 04/45] KVM: x86/emulate: Check TSS busy-bit writes during task switches
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM: x86/emulate: unhandled write_segment_descriptor() error in task switch - [High] KVM…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM: x86/emulate: unhandled write_segment_descriptor() error in task switch
- [High] KVM: x86/emulate: task switch drops faults and IO_NEEDED on descriptor writes --
commit 31629465ad13b50735702c0107b5866c7a5a5806
Author: Paolo Bonzini <[email protected]>
KVM: x86/emulate: Check TSS busy-bit writes during task switches
This commit checks the return values of the write_segment_descriptor() calls that clear the old TSS busy bit and set the new TSS busy bit in emulator_do_task_switch(), returning early on failure so that failed writes stop the task switch.
arch/x86/kvm/emulate.c static int emulator_do_task_switch(struct x86_emulate_ctxt *ctxt, · line 2930
Quoted 12 lines
arch/x86/kvm/emulate.c +3
-1@@ -2925,7 +2925,9 @@ static int emulator_do_task_switch(struct x86_emulate_ctxt *ctxt,if (reason ==TASK_SWITCH_IRET || reason ==TASK_SWITCH_JMP ) { curr_tss_desc.type &= ~(1 <<1 );/* clear busy flag */ -write_segment_descriptor (ctxt, old_tss_sel, &curr_tss_desc); + ret =write_segment_descriptor (ctxt, old_tss_sel, &curr_tss_desc); +if (ret !=X86EMUL_CONTINUE ) +return ret;
[Severity: High]
When write_segment_descriptor() fails in arch/x86/kvm/emulate.c:emulator_do_task_switch(), will returning the non-X86EMUL_CONTINUE status actually reach the emulation-failure exit in kvm_task_switch(), or does it cause emulator_task_switch() to return EMULATION_OK while skipping register writeback?
Tracing the call chain from write_segment_descriptor() -> linear_write_system() -> emulator_write_std() -> kvm_write_guest_virt_helper():
arch/x86/kvm/x86.c:kvm_write_guest_virt_helper() {
...
if (gpa == INVALID_GPA)
return X86EMUL_PROPAGATE_FAULT;
ret = kvm_vcpu_write_guest(vcpu, gpa, data, towrite);
if (ret < 0) {
r = X86EMUL_IO_NEEDED;
goto out;
}
...
}At this commit (prior to the later commit "KVM: x86/emulate: Propagate memory faults to userspace" in the series), any write failure returns X86EMUL_PROPAGATE_FAULT or X86EMUL_IO_NEEDED (without setting vcpu->mmio_needed), and never returns X86EMUL_UNHANDLEABLE. Even after commit "KVM: x86/emulate: Propagate memory faults to userspace" wires up X86EMUL_MEMORY_FAULT to EMULATION_MEMORY_FAULT, non-memory-attribute failures still return X86EMUL_PROPAGATE_FAULT (for example, when the GDT page is mapped read-only in the guest page tables with CR0.WP=1) or X86EMUL_IO_NEEDED (for example, when the GDT is in a read-only memslot).
In arch/x86/kvm/emulate.c:emulator_task_switch():
if (rc == X86EMUL_CONTINUE) {
ctxt->eip = ctxt->_eip;
writeback_registers(ctxt);
}
return (rc == X86EMUL_UNHANDLEABLE) ? EMULATION_FAILED : EMULATION_OK;And in arch/x86/kvm/x86.c:kvm_task_switch():
if (ret || vcpu->mmio_needed) goto unhandled_task_switch; kvm_rip_write(vcpu, ctxt->eip); kvm_set_rflags(vcpu, ctxt->eflags); return 1;
When this first write_segment_descriptor() call fails on an IRET or JMP task switch, skip_emulated_instruction() has already advanced RIP, and emulator_task_switch() returns EMULATION_OK (0) with vcpu->mmio_needed false. Doesn't kvm_task_switch() then silently turn the task switch into a no-op and resume the guest without injecting an exception or exiting to userspace?
arch/x86/kvm/emulate.c static int emulator_do_task_switch(struct x86_emulate_ctxt *ctxt, · line 2956
Quoted 11 lines
arch/x86/kvm/emulate.c +3
-1}if (reason ==TASK_SWITCH_IRET )@@ -2949,7 +2951,9 @@ static int emulator_do_task_switch(struct x86_emulate_ctxt *ctxt,if (reason !=TASK_SWITCH_IRET ) { next_tss_desc.type |= (1 <<1 );/* set busy flag */ -write_segment_descriptor (ctxt, tss_selector, &next_tss_desc); + ret =write_segment_descriptor (ctxt, tss_selector, &next_tss_desc); +if (ret !=X86EMUL_CONTINUE ) +return ret;
[Severity: High]
Similarly, if this second write_segment_descriptor() call in arch/x86/kvm/emulate.c:emulator_do_task_switch() fails with X86EMUL_PROPAGATE_FAULT or X86EMUL_IO_NEEDED (for instance, if the GDT page containing next_tss_desc is read-only or unmapped under the new CR3 just loaded by load_state_from_tss32() when segment descriptors are VM86/NULL or already have the Accessed bit set, or if the GDT resides in a read-only memslot), can returning early here leave the vCPU in a half-switched state?
At this point in emulator_do_task_switch(), task_switch_16() or task_switch_32() has already committed the new task's CR3, ctxt->eflags, and segment registers (LDTR, ES, CS, SS, DS, FS, GS). Returning early skips setting X86_CR0_TS, loading VCPU_SREG_TR, pushing error_code, and updating dr7 in emulator_do_task_switch(), and skips updating ctxt->eip = ctxt->_eip and calling writeback_registers(ctxt) in emulator_task_switch().
Because emulator_task_switch() still translates X86EMUL_PROPAGATE_FAULT and X86EMUL_IO_NEEDED into EMULATION_OK without setting ctxt->have_exception, and arch/x86/kvm/x86.c:kvm_task_switch() neither checks ctxt->have_exception nor takes the unhandled_task_switch path when ret is 0 and vcpu->mmio_needed is false:
ret = emulator_task_switch(ctxt, tss_selector, idt_index, reason, has_error_code, error_code); /* * Report an error userspace if MMIO is needed, as KVM doesn't support * MMIO during a task switch (or any other complex operation). */ if (ret || vcpu->mmio_needed) goto unhandled_task_switch; kvm_rip_write(vcpu, ctxt->eip); kvm_set_rflags(vcpu, ctxt->eflags); return 1;
Won't kvm_task_switch() drop the architectural fault or write failure and resume the guest with the new task's CR3, EFLAGS, and segments but the old task's TR, RIP, and general-purpose registers?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Paolo Bonzini
maintainerPATCH v4 5/45 KVM: loongarch: do full validity check on the gfn-to-hva cache
In reply to Paolo BonziniIf the HVA or the memslots are invalid, reinitialize the cache.
--- arch/loongarch/kvm/vcpu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/loongarch/kvm/vcpu.c +4 -2
@@ -165,7 +165,8 @@ static void kvm_update_stolen_time(struct kvm_vcpu *vcpu) gpa &=KVM_STEAL_PHYS_MASK ; slots =kvm_memslots (vcpu->kvm); -if (slots->generation != ghc->generation || gpa != ghc->gpa) { +if (unlikely (slots->generation != ghc->generation || gpa != ghc->gpa || +kvm_is_error_hva (ghc->hva) || !ghc->memslot)) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;return ;
@@ -1820,7 +1821,8 @@ static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) gpa &=KVM_STEAL_PHYS_MASK ; ghc = &vcpu->arch.st.cache; slots =kvm_memslots (vcpu->kvm); -if (slots->generation != ghc->generation || gpa != ghc->gpa) { +if (unlikely (slots->generation != ghc->generation || gpa != ghc->gpa || +kvm_is_error_hva (ghc->hva) || !ghc->memslot)) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;return ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 6/45 KVM: selftests: Test address translation for Hyper-V direct L2 hypercalls
In reply to Paolo BonziniTest that the TLB flush hypercalls, when executed by L2, translate their input addresses through the guest's EPT or NPT.
--- tools/testing/selftests/kvm/x86/hyperv_evmcs.c | 12 +++++++++++- tools/testing/selftests/kvm/x86/hyperv_svm_test.c | 13 ++++++++++++- 2 files changed, 23 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/x86/hyperv_evmcs.c +11 -1
@@ -18,6 +18,8 @@#include "hyperv.h" #include "vmx.h" +#define L2_HCALL_INPUT_GPA 0x100000000ull +static int ud_count;static void guest_ud_handler (struct ex_regs *regs)
@@ -64,6 +66,8 @@ void l2_guest_code(void)rdmsr_from_l2 (MSR_GS_BASE );/* intercepted */ /* L2 TLB flush tests */ +hyperv_hypercall (HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE , +L2_HCALL_INPUT_GPA ,0 );hyperv_hypercall (HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE |HV_HYPERCALL_FAST_BIT ,0x0 ,HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES |HV_FLUSH_ALL_PROCESSORS );rdmsr_from_l2 (MSR_FS_BASE );
@@ -239,7 +243,7 @@ static struct kvm_vcpu *save_restore_vm(struct kvm_vm *vm,int main (int argc,char *argv[]) {gva_t vmx_pages_gva =0 , hv_pages_gva =0 ; -gva_t hcall_page; +gva_t hcall_input_page, hcall_page;struct kvm_vcpu *vcpu;struct kvm_vm *vm;
@@ -250,17 +254,23 @@ int main(int argc, char *argv[])TEST_REQUIRE (kvm_has_cap (KVM_CAP_NESTED_STATE ));TEST_REQUIRE (kvm_has_cap (KVM_CAP_HYPERV_ENLIGHTENED_VMCS ));TEST_REQUIRE (kvm_hv_cpu_has (HV_X64_NESTED_DIRECT_FLUSH )); +TEST_REQUIRE (kvm_cpu_has_ept ()); vm =vm_create_with_one_vcpu (&vcpu, guest_code); +vm_enable_ept (vm); hcall_page =vm_alloc_pages (vm,1 );memset (addr_gva2hva (vm, hcall_page),0x0 ,getpagesize ()); + hcall_input_page =vm_alloc_page (vm); +memset (addr_gva2hva (vm, hcall_input_page),0 ,getpagesize ());vcpu_set_hv_cpuid (vcpu);vcpu_enable_evmcs (vcpu);vcpu_alloc_vmx (vm, &vmx_pages_gva);vcpu_alloc_hyperv_test_pages (vm, &hv_pages_gva); +tdp_identity_map_default_memslots (vm); +tdp_map (vm,L2_HCALL_INPUT_GPA ,addr_gva2gpa (vm, hcall_input_page),PAGE_SIZE );vcpu_args_set (vcpu,3 , vmx_pages_gva, hv_pages_gva,addr_gva2gpa (vm, hcall_page));vcpu_set_msr (vcpu,HV_X64_MSR_VP_INDEX , vcpu->id);
tools/testing/selftests/kvm/x86/hyperv_svm_test.c +12 -1
@@ -18,6 +18,8 @@#include "svm_util.h" #include "hyperv.h" +#define L2_HCALL_INPUT_GPA 0x100000000ull +/* Exit to L1 from L2 with RDMSR instruction */ static inline void rdmsr_from_l2 (u32 msr) {
@@ -45,6 +47,8 @@ void l2_guest_code(void)GUEST_SYNC (5 );/* L2 TLB flush tests */ +hyperv_hypercall (HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE , +L2_HCALL_INPUT_GPA ,0 ); hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES |
@@ -146,7 +150,7 @@ static void __attribute__((__flatten__)) guest_code(struct svm_test_data *svm,int main (int argc,char *argv[]) {gva_t nested_gva =0 , hv_pages_gva =0 ; -gva_t hcall_page; +gva_t hcall_input_page, hcall_page;struct kvm_vcpu *vcpu;struct kvm_vm *vm;struct ucall uc;
@@ -154,15 +158,22 @@ int main(int argc, char *argv[])TEST_REQUIRE (kvm_cpu_has (X86_FEATURE_SVM ));TEST_REQUIRE (kvm_hv_cpu_has (HV_X64_NESTED_DIRECT_FLUSH )); +TEST_REQUIRE (kvm_cpu_has_npt ());/* Create VM */ vm =vm_create_with_one_vcpu (&vcpu, guest_code); +vm_enable_npt (vm);vcpu_set_hv_cpuid (vcpu);vcpu_alloc_svm (vm, &nested_gva);vcpu_alloc_hyperv_test_pages (vm, &hv_pages_gva); hcall_page =vm_alloc_pages (vm,1 );memset (addr_gva2hva (vm, hcall_page),0x0 ,getpagesize ()); + hcall_input_page =vm_alloc_page (vm); +memset (addr_gva2hva (vm, hcall_input_page),0 ,getpagesize ()); + +tdp_identity_map_default_memslots (vm); +tdp_map (vm,L2_HCALL_INPUT_GPA ,addr_gva2gpa (vm, hcall_input_page),PAGE_SIZE );vcpu_args_set (vcpu,3 , nested_gva, hv_pages_gva,addr_gva2gpa (vm, hcall_page));vcpu_set_msr (vcpu,HV_X64_MSR_VP_INDEX , vcpu->id);
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 7/45 KVM: selftests: Take into account mixed memory fault flags
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Update x86_64/private_mem_kvm_exits_test to take into account memory fault flags might contain multiple bits set while remaining valid---for example KVM might return KVM_MEMORY_EXIT_FLAG_READ in addition to KVM_MEMORY_EXIT_FLAG_PRIVATE.
--- tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c +4 -1
@@ -91,6 +91,8 @@ static void test_private_access_memslot_deleted(void) }else {TEST_ASSERT_EQ (exit_reason,KVM_EXIT_MEMORY_FAULT );TEST_ASSERT_EQ (vcpu->run->memory_fault.flags,KVM_MEMORY_EXIT_FLAG_PRIVATE ); +TEST_ASSERT (vcpu->run->memory_fault.flags &KVM_MEMORY_EXIT_FLAG_PRIVATE , +"Memory fault didn't occur on a private memory access" );TEST_ASSERT_EQ (vcpu->run->memory_fault.gpa,EXITS_TEST_GPA );TEST_ASSERT_EQ (vcpu->run->memory_fault.size,EXITS_TEST_SIZE ); }
@@ -127,7 +129,8 @@ static void test_private_access_memslot_not_private(void) exit_reason =run_vcpu_get_exit_reason (vcpu);TEST_ASSERT_EQ (exit_reason,KVM_EXIT_MEMORY_FAULT ); -TEST_ASSERT_EQ (vcpu->run->memory_fault.flags,KVM_MEMORY_EXIT_FLAG_PRIVATE ); +TEST_ASSERT (vcpu->run->memory_fault.flags &KVM_MEMORY_EXIT_FLAG_PRIVATE , +"Memory fault didn't occur on a private memory access" );TEST_ASSERT_EQ (vcpu->run->memory_fault.gpa,EXITS_TEST_GPA );TEST_ASSERT_EQ (vcpu->run->memory_fault.size,EXITS_TEST_SIZE );
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 07/45] KVM: selftests: Take into account mixed memory fault flags
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] KVM: selftests: leftover flags check in test_private_access_memslot_deleted() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] KVM: selftests: leftover flags check in test_private_access_memslot_deleted() --
commit 6287705468987b3051cb61deae85c0b625189112
Author: Nicolas Saenz Julienne <[email protected]>
KVM: selftests: Take into account mixed memory fault flags
This commit updates private_mem_kvm_exits_test to check that KVM_MEMORY_EXIT_FLAG_PRIVATE is set in memory_fault.flags using a bitwise test rather than strict equality, allowing other valid flags such as KVM_MEMORY_EXIT_FLAG_READ to be set alongside it.
tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c static void test_private_access_memslot_deleted(void) · line 95
Quoted 10 lines
tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c +2
-0@@ -91,6 +91,8 @@ static void test_private_access_memslot_deleted(void) }else {TEST_ASSERT_EQ (exit_reason,KVM_EXIT_MEMORY_FAULT );TEST_ASSERT_EQ (vcpu->run->memory_fault.flags,KVM_MEMORY_EXIT_FLAG_PRIVATE ); +TEST_ASSERT (vcpu->run->memory_fault.flags &KVM_MEMORY_EXIT_FLAG_PRIVATE , +"Memory fault didn't occur on a private memory access" );
[Severity: Medium]
Should the existing TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, KVM_MEMORY_EXIT_FLAG_PRIVATE) in test_private_access_memslot_deleted() be removed here as well, just like in test_private_access_memslot_not_private()?
With the old equality check left in place, test_private_access_memslot_deleted() will still fail if KVM sets additional flags such as KVM_MEMORY_EXIT_FLAG_READ in vcpu->run->memory_fault.flags.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Paolo Bonzini
maintainerPATCH v4 8/45 selftests: kvm: drop TEST_ASSERT_SUPPORTED_ATTRIBUTES
In reply to Paolo BonziniTEST_ASSERT_SUPPORTED_ATTRIBUTES() is supposedly reminding tha __vm_set_memory_attributes() does not support multiple attributes, but that is not really a problem; the tests need to know that multiple attributes can be overwritten, but that is not a problem for a simple wrapper around KVM_SET_MEMORY_ATTRIBUTES.
--- tools/testing/selftests/kvm/include/kvm_util.h | 12 ------------ 1 file changed, 12 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +0 -12
@@ -473,14 +473,6 @@ static inline void vm_enable_cap(struct kvm_vm *vm, u32 cap, u64 arg0)int kvm_gpa_to_guest_memfd (struct kvm_vm *vm,gpa_t gpa,off_t *fd_offset,size_t *nr_bytes); -/* -* KVM_SET_MEMORY_ATTRIBUTES{,2} overwrites _all_ attributes. These -* flows need significant enhancements to support multiple attributes. -*/ -#define TEST_ASSERT_SUPPORTED_ATTRIBUTES (attributes) \ - TEST_ASSERT(!(attributes) || (attributes) == KVM_MEMORY_ATTRIBUTE_PRIVATE, \ - "Update me to support multiple attributes!") -static inline void vm_set_memory_attributes (struct kvm_vm *vm,gpa_t gpa,size_t size,u64 attributes) {
@@ -491,8 +483,6 @@ static inline void vm_set_memory_attributes(struct kvm_vm *vm, gpa_t gpa, .flags = 0, }; -TEST_ASSERT_SUPPORTED_ATTRIBUTES (attributes); -vm_ioctl (vm,KVM_SET_MEMORY_ATTRIBUTES , &attr); }
@@ -543,8 +533,6 @@ static inline void gmem_set_memory_attributes(int fd, u64 offset, .flags = 0, }; -TEST_ASSERT_SUPPORTED_ATTRIBUTES (attributes); -__TEST_REQUIRE (kvm_check_cap (KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES ) >0 ,"No valid attributes for guest_memfd ioctl!" );
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 9/45 KVM: Define and communicate KVM_EXIT_MEMORY_FAULT RWX flags to userspace
In reply to Paolo BonziniFrom: Anish Moorthy <[email protected]>
kvm_prepare_memory_fault_exit() already takes parameters describing the RWX-ness of the relevant access but doesn't actually do anything with them. Define and use the flags necessary to pass this information on to userspace.
--- Documentation/virt/kvm/api.rst | 5 +++++ include/linux/kvm_host.h | 9 ++++++++- include/uapi/linux/kvm.h | 3 +++ 3 files changed, 16 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +5 -0
@@ -7481,6 +7481,9 @@ spec refer, https://github.com/riscv/riscv-sbi-doc. /* KVM_EXIT_MEMORY_FAULT */ struct { + #define KVM_MEMORY_EXIT_FLAG_READ (1ULL << 0) + #define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1) + #define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2) #define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3) __u64 flags; __u64 gpa;
@@ -7492,6 +7495,8 @@ could not be resolved by KVM. The 'gpa' and 'size' (in bytes) describe the guest physical address range [gpa, gpa + size) of the fault. The 'flags' field describes properties of the faulting access that are likely pertinent: + - KVM_MEMORY_EXIT_FLAG_READ/WRITE/EXEC - When set, indicates that the memory + fault occurred on a read/write/exec access respectively. - KVM_MEMORY_EXIT_FLAG_PRIVATE - When set, indicates the memory fault occurred on a private memory access. When clear, indicates the fault occurred on a shared access.
include/linux/kvm_host.h +8 -1
@@ -2556,8 +2556,15 @@ static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, vcpu->run->memory_fault.gpa = gpa; vcpu->run->memory_fault.size = size; -/* RWX flags are not (yet) defined or communicated to userspace. */ vcpu->run->memory_fault.flags =0 ; + +if (is_write) + vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_WRITE ; +else if (is_exec) + vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_EXEC ; +else + vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_READ ; +if (is_private) vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_PRIVATE ; }
include/uapi/linux/kvm.h +3 -0
@@ -457,6 +457,9 @@ struct kvm_run { } notify;/* KVM_EXIT_MEMORY_FAULT */ struct { +#define KVM_MEMORY_EXIT_FLAG_READ (1ULL << 0) +#define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1) +#define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2)#define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3)__u64 flags;__u64 gpa;
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 09/45] KVM: Define and communicate KVM_EXIT_MEMORY_FAULT RWX flags to userspace
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] KVM: flag change in kvm_prepare_memory_fault_exit() breaks selftest --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] KVM: flag change in kvm_prepare_memory_fault_exit() breaks selftest --
commit 97c15077daea43ee7bc4ecb13d9cec2a636fe69e
Author: Anish Moorthy <[email protected]>
KVM: Define and communicate KVM_EXIT_MEMORY_FAULT RWX flags to userspace
This commit defines KVM_MEMORY_EXIT_FLAG_READ, KVM_MEMORY_EXIT_FLAG_WRITE, and KVM_MEMORY_EXIT_FLAG_EXEC in the UAPI header and updates kvm_prepare_memory_fault_exit() to populate vcpu->run->memory_fault.flags with the corresponding access type flag.
Quoted 7 lines
include/linux/kvm_host.h +0
-0@@ -2556,8 +2556,15 @@ static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, vcpu->run->memory_fault.gpa = gpa; vcpu->run->memory_fault.size = size;
include/linux/kvm_host.h static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, · line 2566
Quoted 10 lines
include/linux/kvm_host.h +7
-1-/* RWX flags are not (yet) defined or communicated to userspace. */ vcpu->run->memory_fault.flags =0 ; + +if (is_write) + vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_WRITE ; +else if (is_exec) + vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_EXEC ; +else + vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_READ ;
[Severity: Medium]
Does setting KVM_MEMORY_EXIT_FLAG_READ here break test_private_access_memslot_deleted() in tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c when !kvm_has_gmem_attributes?
In that test, the vCPU runs guest_repeatedly_read() on private guest memory while the backing memslot is deleted, causing kvm_handle_noslot_fault() -> kvm_mmu_prepare_memory_fault_exit() -> kvm_prepare_memory_fault_exit() to set vcpu->run->memory_fault.flags to KVM_MEMORY_EXIT_FLAG_PRIVATE | KVM_MEMORY_EXIT_FLAG_READ.
Earlier commit 628770546898 ("KVM: selftests: Take into account mixed memory fault flags") added a bitwise check in test_private_access_memslot_deleted(), but left the exact equality assertion in place right above it:
tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c:test_private_access_memslot_deleted() {
...
} else {
TEST_ASSERT_EQ(exit_reason, KVM_EXIT_MEMORY_FAULT);
TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, KVM_MEMORY_EXIT_FLAG_PRIVATE);
TEST_ASSERT(vcpu->run->memory_fault.flags & KVM_MEMORY_EXIT_FLAG_PRIVATE,
"Memory fault didn't occur on a private memory access");
TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, EXITS_TEST_GPA);
TEST_ASSERT_EQ(vcpu->run->memory_fault.size, EXITS_TEST_SIZE);
}
...
}Should that leftover TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, KVM_MEMORY_EXIT_FLAG_PRIVATE) be removed so the selftest does not fail once KVM_MEMORY_EXIT_FLAG_READ is set?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Paolo Bonzini
maintainerPATCH v4 10/45 KVM: x86: Introduce memory fault on invalid hypercalls reads/writes
In reply to Paolo BonziniAllow userspace to perform arbitrary actions when a hypercall refers to an invalid address, by exiting with KVM_EXIT_MEMORY_FAULT. This will for example allow userspace to perform a VTL call.
--- Documentation/virt/kvm/api.rst | 19 ++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/hyperv.c | 157 +++++++++++++++++++++++++------- arch/x86/kvm/x86.c | 34 +++++-- include/uapi/linux/kvm.h | 1 + 5 files changed, 173 insertions(+), 39 deletions(-)
Documentation/virt/kvm/api.rst +19 -0
@@ -9186,6 +9186,25 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key interpretation are disabled. If cmma has already been enabled or the hpage_2g module parameter is not set to 1, -EINVAL is returned. +7.48 KVM_CAP_HCALL_FAULT_EXIT +------------------------------------ + +:Architectures: x86 +:Parameters: args[0] is 0 to disable, 1 to enable + +When enabled, KVM checks the memory that is read or written by +hypercalls (including slow Hyper-V hypercalls and KVM_HC_CLOCK_PAIRING). +An inaccessible input page causes a KVM_EXIT_MEMORY_FAULT with +KVM_MEMORY_EXIT_FLAG_READ. An inaccessible or read-only output page causes +a KVM_EXIT_MEMORY_FAULT with KVM_MEMORY_EXIT_FLAG_WRITE. The reported +range identifies the page containing the parameter GPA. + +Hypercall parameters that are unused by the selected hypercall are not checked. + +For Hyper-V, unknown hypercalls are passed to userspace without checking their +parameter pages. The fault GPA is in the physical address space of the +VM managed by userspace, after nested GPA translation. + 8. Other capabilities. ======================
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1178,6 +1178,7 @@ struct kvm_arch {bool has_protected_eoi;bool has_protected_pmu;bool pre_fault_allowed; +bool hcall_fault_exit;struct hlist_head *mmu_page_hash;struct list_head active_mmu_pages;struct kvm_possible_nx_huge_pages possible_nx_huge_pages[KVM_NR_MMU_TYPES ];
arch/x86/kvm/hyperv.c +125 -32
256 lines not on this page · raw message
arch/x86/kvm/x86.c +27 -7
@@ -2253,6 +2253,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_HYPERV_ENFORCE_CPUID :case KVM_CAP_SYS_HYPERV_CPUID :#endif +case KVM_CAP_HCALL_FAULT_EXIT :case KVM_CAP_PCI_SEGMENT :case KVM_CAP_DEBUGREGS :case KVM_CAP_X86_ROBUST_SINGLESTEP :
@@ -4252,6 +4253,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,mutex_unlock (&kvm->lock);break ; } +case KVM_CAP_HCALL_FAULT_EXIT : + kvm->arch.hcall_fault_exit = cap->args[0 ]; + r =0 ; +break ;default : r = -EINVAL ;break ;
@@ -7252,25 +7257,28 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_x86_vendor_exit);#ifdef CONFIG_X86_64 static int kvm_pv_clock_pairing (struct kvm_vcpu *vcpu,gpa_t paddr, -unsigned long clock_type) +unsigned long clock_type,unsigned long *ret) {struct kvm_clock_pairing clock_pairing;struct timespec64 ts;size_t offset =0 ; +bool writable; +unsigned long hva;u64 cycle; + *ret = -KVM_EOPNOTSUPP ;if (clock_type !=KVM_CLOCK_PAIRING_WALLCLOCK ) -return -KVM_EOPNOTSUPP ; +return 0 ;/* * When tsc is in permanent catchup mode guests won't be able to use * pvclock_read_retry loop to get consistent view of pvclock */ if (vcpu->arch.tsc_always_catchup) -return -KVM_EOPNOTSUPP ; +return 0 ;if (!kvm_get_walltime_and_clockread (&ts, &cycle)) -return -KVM_EOPNOTSUPP ; +return 0 ; clock_pairing.sec = ts.tv_sec; clock_pairing.nsec = ts.tv_nsec;
@@ -7278,6 +7286,7 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, clock_pairing.flags =0 ;memset (&clock_pairing.pad,0 ,sizeof (clock_pairing.pad)); + *ret = -KVM_EFAULT ;while (offset <sizeof (clock_pairing)) {gpa_t gpa =kvm_translate_gpa (vcpu, &vcpu->arch.gva_walk, paddr + offset,PFERR_WRITE_MASK |PFERR_GUEST_FINAL_MASK ,NULL ,0 );
@@ -7285,12 +7294,22 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, PAGE_SIZE - offset_in_page(gpa));if (gpa ==INVALID_GPA ) -return -KVM_EFAULT ; +return 0 ; +if (vcpu->kvm->arch.hcall_fault_exit) { + hva =kvm_vcpu_gfn_to_hva_prot (vcpu,gpa_to_gfn (gpa), &writable); +if (kvm_is_error_hva (hva) || !writable) { +kvm_prepare_memory_fault_exit (vcpu, gpa &PAGE_MASK ,PAGE_SIZE , + true, false, false); +return -EFAULT ; + } + }if (kvm_write_guest (vcpu->kvm, gpa, (u8 *)&clock_pairing + offset, len)) -return -KVM_EFAULT ; +return 0 ; offset += len; } + + *ret =0 ;return 0 ; }#endif
@@ -7449,7 +7468,8 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl,break ;#ifdef CONFIG_X86_64 case KVM_HC_CLOCK_PAIRING : - ret =kvm_pv_clock_pairing (vcpu, a0, a1); +if (kvm_pv_clock_pairing (vcpu, a0, a1, &ret)) +return -EFAULT ;break ;#endif case KVM_HC_SEND_IPI :
include/uapi/linux/kvm.h +1 -0
@@ -1003,6 +1003,7 @@ struct kvm_enable_cap {#define KVM_CAP_PPC_COMPAT_CAPS 250#define KVM_CAP_ARM_PMU_V3_STRICT 251#define KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES 252 +#define KVM_CAP_HCALL_FAULT_EXIT 253struct kvm_irq_routing_irqchip {__u32 irqchip;
Signature · 1 line
2.52.0
Re: [PATCH 10/45] KVM: x86: Introduce memory fault on invalid hypercalls reads/writes
Quoted 8 lines · Paolo Bonzini
+7.48
KVM_CAP_HCALL_FAULT_EXIT
+------------------------------------Documentation/virt/kvm/api.rst +5
-0+ +:Architectures: x86 +:Parameters: args[0] is 0 to disable, 1 to enable + +When enabled, KVM checks the memory that is read or written by
Documentation/virt/kvm/api.rst enabled, cmma can't be enabled anymore and pfmfi and the storage key
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+hypercalls (including slow Hyper-V hypercalls and KVM_HC_CLOCK_PAIRING). +An inaccessible input page causes a KVM_EXIT_MEMORY_FAULT with +KVM_MEMORY_EXIT_FLAG_READ. An inaccessible or read-only output page causes +a KVM_EXIT_MEMORY_FAULT with KVM_MEMORY_EXIT_FLAG_WRITE. The reported +range identifies the page containing the parameter GPA. + +Hypercall parameters that are unused by the selected hypercall are not checked. + +For Hyper-V, unknown hypercalls are passed to userspace without checking their +parameter pages.
Is that true? I have a patch in my VSM tree to pass unknown hypercalls to userspace for the default: case, but nothing *yet* does so?
Also, Xen hypercalls (the ones that are handled in-kernel) aren't covered here, and probably should be.
Can we have the selftests check for corner cases like the hypercall argument pointing at the last word of a page, extending onto the next page (which is absent). Not sure that one is working right as you have it either.
I wonder if we want a generic kvm_hcall_copy_from_guest() which returns -EFAULT as appropriate, and its callers can then handle it consistently without having to worry about TOCTOU issues? Or is that just kvm_read_guest()?
Paolo Bonzini
maintainerPATCH v4 11/45 KVM: selftests: test hypercall memory fault exits
In reply to Paolo BonziniCover various scenarios where hypercalls are invoked with invalid or read-only GPAs, and check that they exit to userspace.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + tools/testing/selftests/kvm/x86/hcall_fault.c | 246 ++++++++++++++++++ .../testing/selftests/kvm/x86/hyperv_evmcs.c | 4 + .../selftests/kvm/x86/hyperv_svm_test.c | 2 + 4 files changed, 253 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/hcall_fault.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -83,6 +83,7 @@ TEST_GEN_PROGS_x86 += x86/evmcs_smm_controls_test TEST_GEN_PROGS_x86 += x86/exit_on_emulation_failure_test TEST_GEN_PROGS_x86 += x86/fastops_test TEST_GEN_PROGS_x86 += x86/fix_hypercall_test +TEST_GEN_PROGS_x86 += x86/hcall_fault TEST_GEN_PROGS_x86 += x86/hwcr_msr_test TEST_GEN_PROGS_x86 += x86/hyperv_clock TEST_GEN_PROGS_x86 += x86/hyperv_cpuid
tools/testing/selftests/kvm/x86/hcall_fault.c new file +246 -0
246 lines not on this page · raw message
tools/testing/selftests/kvm/x86/hyperv_evmcs.c +4 -0
@@ -227,6 +227,8 @@ static struct kvm_vcpu *save_restore_vm(struct kvm_vm *vm,/* Restore state in a new VM. */ vcpu =vm_recreate_with_one_vcpu (vm); +if (kvm_has_cap (KVM_CAP_HCALL_FAULT_EXIT )) +vm_enable_cap (vm,KVM_CAP_HCALL_FAULT_EXIT ,1 );vcpu_set_hv_cpuid (vcpu);vcpu_enable_evmcs (vcpu);vcpu_load_state (vcpu, state);
@@ -257,6 +259,8 @@ int main(int argc, char *argv[])TEST_REQUIRE (kvm_cpu_has_ept ()); vm =vm_create_with_one_vcpu (&vcpu, guest_code); +if (kvm_has_cap (KVM_CAP_HCALL_FAULT_EXIT )) +vm_enable_cap (vm,KVM_CAP_HCALL_FAULT_EXIT ,1 );vm_enable_ept (vm); hcall_page =vm_alloc_pages (vm,1 );
tools/testing/selftests/kvm/x86/hyperv_svm_test.c +2 -0
@@ -162,6 +162,8 @@ int main(int argc, char *argv[])/* Create VM */ vm =vm_create_with_one_vcpu (&vcpu, guest_code); +if (kvm_has_cap (KVM_CAP_HCALL_FAULT_EXIT )) +vm_enable_cap (vm,KVM_CAP_HCALL_FAULT_EXIT ,1 );vm_enable_npt (vm);vcpu_set_hv_cpuid (vcpu);vcpu_alloc_svm (vm, &nested_gva);
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 11/45] KVM: selftests: test hypercall memory fault exits
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] KVM: selftests: missing TSC clocksource check in test_clock_pairing_fault() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] KVM: selftests: missing TSC clocksource check in test_clock_pairing_fault() --
commit ec6214ca720546d1e406955e679ce191a860ab11
Author: Paolo Bonzini <[email protected]>
KVM: selftests: test hypercall memory fault exits
This commit adds a new x86 KVM selftest to verify that hypercalls invoked with invalid or read-only GPAs exit to userspace with KVM_EXIT_MEMORY_FAULT when KVM_CAP_HCALL_FAULT_EXIT is enabled, and updates existing Hyper-V tests to enable the capability.
Quoted 6 lines
tools/testing/selftests/kvm/x86/hcall_fault.c new file +0
-0@@ -0,0 +1,246 @@
[ ... ]
Quoted 5 lines
tools/testing/selftests/kvm/x86/hcall_fault.c new file +5
-0+static void clock_pairing_guest (gpa_t gpa) +{ +kvm_hypercall (KVM_HC_CLOCK_PAIRING , gpa,KVM_CLOCK_PAIRING_WALLCLOCK ,0 ,0 ); +GUEST_DONE (); +}
[ ... ]
Quoted 22 lines
tools/testing/selftests/kvm/x86/hcall_fault.c new file +22
-0+static void test_clock_pairing_fault (bool readonly) +{ +struct kvm_vcpu *vcpu; +struct kvm_vm *vm; +gpa_t gpa; + +/* Unlike create_vm do not enable Hyper-V hypercalls. */ + vm =vm_create_with_one_vcpu (&vcpu, clock_pairing_guest); +create_test_regions (vm); +vm_enable_cap (vm,KVM_CAP_HCALL_FAULT_EXIT ,1 ); + gpa = readonly ?READONLY_GPA :UNMAPPED_OUT_GPA ; +vcpu_args_set (vcpu,1 , gpa); + +assert_memory_fault (vcpu, gpa,KVM_MEMORY_EXIT_FLAG_WRITE ); +kvm_vm_free (vm); +} + +int main (void ) +{ +TEST_REQUIRE (kvm_has_cap (KVM_CAP_HYPERV_CPUID )); +TEST_REQUIRE (kvm_has_cap (KVM_CAP_HCALL_FAULT_EXIT )); + TEST_REQUIRE(kvm_cpuid_has(kvm_get_supported_hv_cpuid(),
tools/testing/selftests/kvm/x86/hcall_fault.c line 244
Quoted 10 lines
tools/testing/selftests/kvm/x86/hcall_fault.c new file +10
-0+ HV_ENABLE_EXTENDED_HYPERCALLS)); + +test_unused_gpas (); +test_readonly_input (); +test_input_fault (); +test_output_fault (false, false); +test_output_fault (true, false); +test_output_fault (false, true); +test_clock_pairing_fault (false); +test_clock_pairing_fault (true);
[Severity: Medium]
In arch/x86/kvm/x86.c, kvm_pv_clock_pairing() checks whether the host clocksource is TSC-based via kvm_get_walltime_and_clockread() before translating the GPA or checking vcpu->kvm->arch.hcall_fault_exit:
arch/x86/kvm/x86.c:kvm_pv_clock_pairing() {
...
*ret = -KVM_EOPNOTSUPP;
if (clock_type != KVM_CLOCK_PAIRING_WALLCLOCK)
return 0;
...
if (!kvm_get_walltime_and_clockread(&ts, &cycle))
return 0;
...
while (offset < sizeof(clock_pairing)) {
...
if (vcpu->kvm->arch.hcall_fault_exit) {
hva = kvm_vcpu_gfn_to_hva_prot(vcpu, gpa_to_gfn(gpa), &writable);
if (kvm_is_error_hva(hva) || !writable) {
kvm_prepare_memory_fault_exit(vcpu, gpa & PAGE_MASK, PAGE_SIZE,
true, false, false);
return -EFAULT;
}
}
}If the host clocksource is not TSC-based (e.g., kvm-clock, hpet, or acpi_pm), kvm_pv_clock_pairing() returns 0 with *ret = -KVM_EOPNOTSUPP without preparing a memory fault exit, and clock_pairing_guest() proceeds to GUEST_DONE(), causing assert_memory_fault() to fail on KVM_EXIT_IO.
Should main() guard the test_clock_pairing_fault() calls with sys_clocksource_is_based_on_tsc()?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Paolo Bonzini
maintainerPATCH v4 12/45 KVM: x86/mmu: intersect writability from __kvm_faultin_pfn with fault->map_writable
In reply to Paolo Bonzinifault->map_writable is currently a pure output of __kvm_faultin_pfn(), which is the only thing that restricts it. This will no longer hold once memory protections derived from memory attributes are applied: those compute their own access permissions that combine with those from __kvm_faultin_pfn().
Applying them *before* faulting in the pfn lets a fault that violates the attributes exit to userspace without the cost of gup and/or an async #PF; but it means that permissions will then be restricted in two independent steps, first by memory attributes and then by __kvm_faultin_pfn(). Switch fault->map_writable to that model by letting kvm_mmu_faultin_pfn() only clear bits rather than assign them.
No functional change intended: nothing writes fault->map_writable between the initializer and __kvm_mmu_faultin_pfn() yet, so the AND is equivalent to the assignment it replaces.
--- arch/x86/kvm/mmu/mmu.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
arch/x86/kvm/mmu/mmu.c +8 -4
@@ -4688,7 +4688,7 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,return r; } - fault->map_writable = !(fault->slot->flags &KVM_MEM_READONLY ); + fault->map_writable &= !(fault->slot->flags &KVM_MEM_READONLY ); fault->max_level =kvm_max_level_for_order (max_order);return RET_PF_CONTINUE ;
@@ -4698,13 +4698,14 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) {unsigned int foll = fault->write ?FOLL_WRITE :0 ; +bool writable;if (fault->is_private ||kvm_memslot_is_gmem_only (fault->slot))return kvm_mmu_faultin_pfn_gmem (vcpu, fault); foll |=FOLL_NOWAIT ; fault->pfn =__kvm_faultin_pfn (fault->slot, fault->gfn, foll, - &fault->map_writable, &fault->refcounted_page); + &writable, &fault->refcounted_page);/* * If resolving the page failed because I/O is needed to fault-in the
@@ -4713,7 +4714,7 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu,* other failures are terminal, i.e. retrying won't help. */ if (fault->pfn !=KVM_PFN_ERR_NEEDS_IO ) -return RET_PF_CONTINUE ; + goto out_pf_continue;if (!fault->prefetch &&kvm_can_do_async_pf (vcpu)) {trace_kvm_try_async_get_page (fault->addr, fault->gfn);
@@ -4734,8 +4735,10 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, foll |=FOLL_INTERRUPTIBLE ; foll &= ~FOLL_NOWAIT ; fault->pfn =__kvm_faultin_pfn (fault->slot, fault->gfn, foll, - &fault->map_writable, &fault->refcounted_page); + &writable, &fault->refcounted_page); +out_pf_continue: + fault->map_writable &= writable;return RET_PF_CONTINUE ; }
@@ -5054,6 +5057,7 @@ static int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .is_private = err & PFERR_PRIVATE_ACCESS, .pfn = KVM_PFN_ERR_FAULT, + .map_writable = true, };int r;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 13/45 KVM: x86/mmu: Extend map_writable to a full ACC_* mask
In reply to Paolo BonziniSupport for memory protection attributes opens the door to installing non-executable mappings. Instead of introducing yet another member in struct kvm_page_fault and another argument to make_spte(), make the existing member map_writable a mask of ACC_* bits. This also avoids the need for make_spte() to map a single bool to either the NX bit or the XS/XU bits together.
Unlike for mappings that are not writable because the fault did not request write premission, it is not not necessary to track executability for these SPTEs; the gfn is always available and it will be possible to access the attributes directly in FNAME(sync_spte).
--- arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++--------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 8 +++++--- arch/x86/kvm/mmu/spte.c | 12 ++++++------ arch/x86/kvm/mmu/spte.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 2 +- 6 files changed, 26 insertions(+), 21 deletions(-)
arch/x86/kvm/mmu/mmu.c +12 -9
@@ -3158,7 +3158,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot,u64 spte;/* Prefetching always gets a writable pfn. */ -bool host_writable = !fault || fault->map_writable; +unsigned host_access = fault ? fault->host_access :ACC_ALL ;bool prefetch = !fault || fault->prefetch;bool write_fault = fault && fault->write;
@@ -3195,7 +3195,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, } wrprot =make_spte (vcpu, sp, slot, pte_access, gfn, pfn, *sptep, prefetch, - false, host_writable, &spte); + false, host_access, &spte);if (*sptep == spte) { ret =RET_PF_SPURIOUS ;
@@ -3643,7 +3643,7 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, fault->slot =NULL ; fault->pfn =KVM_PFN_NOSLOT ; - fault->map_writable = false; + fault->host_access =0 ;/* * If MMIO caching is disabled, emulate immediately without
@@ -4668,7 +4668,8 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) {kvm_release_faultin_page (vcpu->kvm, fault->refcounted_page, - r ==RET_PF_RETRY , fault->map_writable); + r ==RET_PF_RETRY , + !!(fault->host_access &ACC_WRITE_MASK )); } static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,
@@ -4688,9 +4689,10 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,return r; } - fault->map_writable &= !(fault->slot->flags &KVM_MEM_READONLY ); - fault->max_level =kvm_max_level_for_order (max_order); +if (fault->slot->flags &KVM_MEM_READONLY ) + fault->host_access &= ~ACC_WRITE_MASK ; + fault->max_level =kvm_max_level_for_order (max_order);return RET_PF_CONTINUE ; }
@@ -4738,7 +4740,8 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, &writable, &fault->refcounted_page); out_pf_continue: - fault->map_writable &= writable; +if (!writable) + fault->host_access &= ~ACC_WRITE_MASK ;return RET_PF_CONTINUE ; }
@@ -5057,7 +5060,7 @@ static int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .is_private = err & PFERR_PRIVATE_ACCESS, .pfn = KVM_PFN_ERR_FAULT, - .map_writable = true, + .host_access = ACC_ALL, };int r;
@@ -5252,7 +5255,7 @@ int kvm_tdp_mmu_map_private_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, kvm_pfn_t pfn) .gfn = gfn, .slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn), .pfn = pfn, - .map_writable = true, + .host_access = ACC_ALL, };struct kvm *kvm = vcpu->kvm;int r;
arch/x86/kvm/mmu/mmu_internal.h +1 -1
@@ -280,7 +280,7 @@ struct kvm_page_fault {unsigned long mmu_seq;kvm_pfn_t pfn;struct page *refcounted_page; -bool map_writable; +u8 host_access;/* * Indicates the guest is trying to write a gfn that contains one or
arch/x86/kvm/mmu/paging_tmpl.h +5 -3
@@ -938,7 +938,7 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,*/ static int FNAME (sync_spte )(struct kvm_vcpu *vcpu,struct kvm_mmu_page *sp,int i) { -bool host_writable; +u8 host_access;gpa_t first_pte_gpa;u64 *sptep, spte;struct kvm_memory_slot *slot;
@@ -995,11 +995,13 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int sptep = &sp->spt[i]; spte = *sptep; - host_writable = spte & shadow_host_writable_mask; + host_access =ACC_ALL ; +if (!(spte & shadow_host_writable_mask)) + host_access &= ~ACC_WRITE_MASK ; slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn);make_spte (vcpu, sp, slot, pte_access, gfn,spte_to_pfn (spte), spte, true, true, - host_writable, &spte); + host_access, &spte);/* * There is no need to mark the pfn dirty, as the new protections must
arch/x86/kvm/mmu/spte.c +6 -6
@@ -189,7 +189,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, u64 old_spte, bool prefetch, bool synchronizing, - bool host_writable, u64 *new_spte) + unsigned int host_access, u64 *new_spte) { int level = sp->role.level;u64 spte =SPTE_MMU_PRESENT_MASK ;
@@ -207,6 +207,11 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,if (!prefetch || synchronizing) spte |= shadow_accessed_mask; +if (host_access &ACC_WRITE_MASK ) + spte |= shadow_host_writable_mask; + + pte_access &= host_access; +/* * For simplicity, enforce the NX huge page mitigation even if not * strictly necessary. KVM could ignore the mitigation if paging is
@@ -246,11 +251,6 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,if (kvm_x86_ops.get_mt_mask) spte |=kvm_x86_call (get_mt_mask)(vcpu, gfn,kvm_is_mmio_pfn (pfn, &is_host_mmio)); -if (host_writable) - spte |= shadow_host_writable_mask; -else - pte_access &= ~ACC_WRITE_MASK ; -if (shadow_me_value && !kvm_is_mmio_pfn (pfn, &is_host_mmio)) spte |= shadow_me_value;
arch/x86/kvm/mmu/spte.h +1 -1
@@ -563,7 +563,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, u64 old_spte, bool prefetch, bool synchronizing, - bool host_writable, u64 *new_spte); + unsigned int host_access, u64 *new_spte);u64 make_small_spte (struct kvm *kvm,u64 huge_spte,union kvm_mmu_page_role role,int index);u64 make_huge_spte (struct kvm *kvm,u64 small_spte,int level);
arch/x86/kvm/mmu/tdp_mmu.c +1 -1
@@ -1143,7 +1143,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu,else wrprot =make_spte (vcpu, sp, fault->slot, sp->role.access, iter->gfn, fault->pfn, iter->old_spte, fault->prefetch, - false, fault->map_writable, &new_spte); + false, fault->host_access, &new_spte);if (new_spte == iter->old_spte) ret =RET_PF_SPURIOUS ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 14/45 KVM: x86/mmu: Init memslot hugepage information for non-private_mem VMs too
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
The list of supported memory attributes is about to grow, and they will be available on any VM type (as opposed to only ones targeted at confidential computing). As such, update the check in kvm_mmu_init_memslot_memory_attributes() to initialize huge page information if any kind of memory attributes is available for the VM.
--- arch/x86/kvm/mmu/mmu.c | 3 ++- include/linux/kvm_host.h | 6 ++++++ virt/kvm/kvm_main.c | 2 +- 3 files changed, 9 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu/mmu.c +2 -1
@@ -8361,7 +8361,8 @@ void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, {int level; -if (!kvm_arch_has_private_mem (kvm)) +if (!kvm_arch_has_private_mem (kvm) && + !kvm_supported_vm_mem_attributes (kvm))return ;for (level =PG_LEVEL_2M ; level <=KVM_MAX_HUGEPAGE_LEVEL ; level++) {
include/linux/kvm_host.h +6 -0
@@ -2589,6 +2589,12 @@ bool kvm_arch_pre_set_vm_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range);bool kvm_arch_post_set_vm_memory_attributes (struct kvm *kvm,struct kvm_gfn_range *range); +u64 kvm_supported_vm_mem_attributes (struct kvm *kvm); +#else +static inline u64 kvm_supported_vm_mem_attributes (struct kvm *kvm) +{ +return 0 ; +}#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ #ifdef CONFIG_KVM_PRIVATE_MEMATTR_VM
virt/kvm/kvm_main.c +1 -1
@@ -2474,7 +2474,7 @@ static u64 kvm_supports_private_mem(struct kvm *kvm)#endif #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES -static u64 kvm_supported_vm_mem_attributes (struct kvm *kvm) +u64 kvm_supported_vm_mem_attributes (struct kvm *kvm) {if (gmem_in_place_conversion || !kvm_supports_private_mem (kvm))return 0 ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 15/45 KVM: VMX: return error from kvm_set_cr*
In reply to Paolo BonziniThis is already done by svm.c, copy it over to vmx.c so that kvm_complete_insn_gp() can return -EFAULT on PDPTR loads.
--- arch/x86/kvm/vmx/vmx.c | 12 ++++++++---- arch/x86/kvm/x86.c | 2 ++ 2 files changed, 10 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/vmx.c +8 -4
@@ -5630,6 +5630,7 @@ static int handle_set_cr0(struct kvm_vcpu *vcpu, unsigned long val)if (is_guest_mode (vcpu)) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);unsigned long orig_val = val; +int r;/* * We get here when L2 changed cr0 in a way that did not change
@@ -5642,8 +5643,9 @@ static int handle_set_cr0(struct kvm_vcpu *vcpu, unsigned long val) val = (val & ~vmcs12->cr0_guest_host_mask) | (vmcs12->guest_cr0 & vmcs12->cr0_guest_host_mask); -if (kvm_set_cr0 (vcpu, val)) -return 1 ; + r =kvm_set_cr0 (vcpu, val); +if (r) +return r;vmcs_writel (CR0_READ_SHADOW , orig_val);return 0 ; }else {
@@ -5656,12 +5658,14 @@ static int handle_set_cr4(struct kvm_vcpu *vcpu, unsigned long val)if (is_guest_mode (vcpu)) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);unsigned long orig_val = val; +int r;/* analogously to handle_set_cr0 */ val = (val & ~vmcs12->cr4_guest_host_mask) | (vmcs12->guest_cr4 & vmcs12->cr4_guest_host_mask); -if (kvm_set_cr4 (vcpu, val)) -return 1 ; + r =kvm_set_cr4 (vcpu, val); +if (r) +return r;vmcs_writel (CR4_READ_SHADOW , orig_val);return 0 ; }else
arch/x86/kvm/x86.c +2 -0
@@ -580,6 +580,8 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_requeue_exception);int kvm_complete_insn_gp (struct kvm_vcpu *vcpu,int err) { +if (err <0 ) +return err;if (err)kvm_inject_gp (vcpu,0 );else
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 16/45 KVM: adjust for presence of more than one attribute
In reply to Paolo BonziniRemove or adjust code that assumed that the only valid attribute was KVM_MEMORY_ATTRIBUTE_PRIVATE, in preparation for the introduction of RWX memory attributes.
In particular, the optimization in kvm_range_has_memory_attributes simply is not valid if attrs == 0 but some of the attributes are being ignored. The WARNs in arch/x86/kvm/mmu/mmu.c are just a duplicate of checking kvm_supported_mem_attributes() during the ioctl, so drop it.
--- arch/x86/kvm/mmu/mmu.c | 9 +-------- virt/kvm/kvm_main.c | 10 +++++++--- 2 files changed, 8 insertions(+), 11 deletions(-)
arch/x86/kvm/mmu/mmu.c +1 -8
@@ -8226,9 +8226,6 @@ bool kvm_arch_pre_set_vm_memory_attributes(struct kvm *kvm,* Zapping SPTEs in this case ensures KVM will reassess whether or not * a hugepage can be used for affected ranges. */ -if (WARN_ON_ONCE (!kvm_arch_has_private_mem (kvm))) -return false; -if (WARN_ON_ONCE (range->end <= range->start))return false;
@@ -8304,11 +8301,7 @@ bool kvm_arch_post_set_vm_memory_attributes(struct kvm *kvm,* can't map memory PRIVATE. KVM mustn't create a SHARED hugepage over * a range that has PRIVATE GFNs, and conversely converting a range to * SHARED may now allow hugepages. -*/ -if (WARN_ON_ONCE (!kvm_arch_has_private_mem (kvm))) -return false; - -/* +* * The sequence matters here: upper levels consume the result of lower * level's scanning. */
virt/kvm/kvm_main.c +7 -3
@@ -2490,10 +2490,12 @@ bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end, unsigned long mask, unsigned long attrs) { XA_STATE(xas, &kvm->mem_attr_array, start); +unsigned long supported_attrs;unsigned long index;void *entry; - mask &=kvm_supported_vm_mem_attributes (kvm); + supported_attrs =kvm_supported_vm_mem_attributes (kvm); + mask &= supported_attrs;if (attrs & ~mask)return false;
@@ -2522,8 +2524,10 @@ bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,*/ for (index = start; index < end; index++) {do { - entry = attrs ?xas_next (&xas) : -xas_next_entry (&xas, end -1 ); +if (!attrs && mask == supported_attrs) + entry =xas_next_entry (&xas, end -1 ); +else + entry =xas_next (&xas); }while (xas_retry (&xas, entry));if (!entry)
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 16/45] KVM: adjust for presence of more than one attribute
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] KVM: premature true return in kvm_range_has_vm_memory_attributes() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] KVM: premature true return in kvm_range_has_vm_memory_attributes() --
commit bc4dd6682729ee9a313a549f79ade166886ce9b6
Author: Paolo Bonzini <[email protected]>
KVM: adjust for presence of more than one attribute
This commit removes private-memory WARN checks in x86 MMU code and adjusts kvm_range_has_vm_memory_attributes() so it no longer assumes KVM_MEMORY_ATTRIBUTE_PRIVATE is the only valid memory attribute.
Quoted 21 lines
virt/kvm/kvm_main.c +3
-1@@ -2490,10 +2490,12 @@ bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end, unsigned long mask, unsigned long attrs) { XA_STATE(xas, &kvm->mem_attr_array, start); +unsigned long supported_attrs;unsigned long index;void *entry; - mask &=kvm_supported_vm_mem_attributes (kvm); + supported_attrs =kvm_supported_vm_mem_attributes (kvm); + mask &= supported_attrs;if (attrs & ~mask)return false;@@ -2522,8 +2524,10 @@ bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,*/ for (index = start; index < end; index++) {
virt/kvm/kvm_main.c bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end, · line 2533
Quoted 10 lines
virt/kvm/kvm_main.c +4
-2do { - entry = attrs ?xas_next (&xas) : -xas_next_entry (&xas, end -1 ); +if (!attrs && mask == supported_attrs) + entry =xas_next_entry (&xas, end -1 ); +else + entry =xas_next (&xas); }while (xas_retry (&xas, entry));if (!entry)
[Severity: Medium]
When attrs == 0 and mask != supported_attrs, doesn't switching to xas_next(&xas) cause kvm_range_has_vm_memory_attributes() to return true prematurely on the first GFN that has no entry in kvm->mem_attr_array?
Unlike xas_next_entry(&xas, end - 1), which scans forward up to end - 1 for the next present entry, xas_next(&xas) only inspects the single slot at the current index and returns NULL as soon as that single GFN has no attributes.
Immediately after the loop in kvm_range_has_vm_memory_attributes():
if (!entry) return !attrs; WARN_ON_ONCE(!xa_to_value(entry)); if (xas.xa_index != index || (xa_to_value(entry) & mask) != attrs) return false;
Because !entry returns !attrs (which is true when attrs == 0), encountering any empty slot in [start, end) immediately returns true without checking the remaining GFNs in [index + 1, end).
Additionally, if xas_next(&xas) encounters a reserved XA_ZERO_ENTRY while attrs == 0, xas_retry(&xas, entry) returns true without resetting xas, so the next xas_next(&xas) call in the do-while loop advances xas.xa_index to index + 1 and can fail the xas.xa_index != index check.
Since empty and XA_ZERO_ENTRY slots have effective attributes of 0 and always satisfy (0 & mask) == 0 for any mask, should this still use xas_next_entry(&xas, end - 1) whenever !attrs, and instead only check xas.xa_index != index when attrs != 0 (or advance index to xas.xa_index)?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
Paolo Bonzini
maintainerPATCH v4 17/45 KVM: Introduce NR/NW/NX memory attributes
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Introduce memory attributes to map guest physical memory regions as non-readable, non-writable, and/or non-executable. Only a subset of flag combinations is supported. Notably write and exec permissions require read permission, and memory protection attributes are incompatible with private memory.
As mentioned in 5a475554db1e ("KVM: Introduce per-page memory attributes", 2023-11-13), bits 0-2 of the memory attributes were reserved for RWX protection; they are negated to support current memory attribute users which use 0 to indicate no special treatment. Since 0 is not available, a non-negated version of the flags would need an extra bit to express no-access (R=0/W=0/X=0) mappings.
Unfortunately this precaution did not age too well; KVM now supports MBEC/GMET and adding mode-based memory protections will require a non-contiguous bit. But that's something left for later.
Different architectures may have different limitations on the set of valid protections, for example execution-only and XU=0 mappings are supported by Intel but not AMD processors[1]. So, add an architecture-specific callback and add a basic implementation for x86.
[1] When adding support for MBEC/GMET, since NX would remain to mean no execution at all, it is possible to use either an NXS bit or two separate NXS/NXU bits in addition to NX. The former would only support permissions that are available with either MBEC or GMET.
--- Documentation/virt/kvm/api.rst | 29 +++++++++++++-- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/Kconfig | 2 +- arch/x86/kvm/mmu/mmu.c | 63 ++++++++++++++++++++++++--------- arch/x86/kvm/x86.c | 2 -- include/linux/kvm_host.h | 28 +++++++++++++++ include/uapi/linux/kvm.h | 3 ++ virt/kvm/kvm_main.c | 46 ++++++++++++++++++++---- 8 files changed, 145 insertions(+), 29 deletions(-)
Documentation/virt/kvm/api.rst +26 -3
@@ -6445,15 +6445,38 @@ of guest physical memory. __u64 flags; }; + #define KVM_MEMORY_ATTRIBUTE_NR (1ULL << 0) + #define KVM_MEMORY_ATTRIBUTE_NW (1ULL << 1) + #define KVM_MEMORY_ATTRIBUTE_NX (1ULL << 2) #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) The address and size must be page aligned. The supported attributes can be retrieved via ioctl(KVM_CHECK_EXTENSION) on KVM_CAP_MEMORY_ATTRIBUTES. If executed on a VM, KVM_CAP_MEMORY_ATTRIBUTES precisely returns the attributes supported by that VM. If executed at system scope, KVM_CAP_MEMORY_ATTRIBUTES -returns all attributes supported by KVM. The only attribute defined at this -time is KVM_MEMORY_ATTRIBUTE_PRIVATE, which marks the associated gfn as being -guest private memory. +returns all attributes supported by KVM. The attribute defined at this +time are: + + - KVM_MEMORY_ATTRIBUTE_NR/NW/NX - Respectively marks the memory region as + non-read, non-write and/or non-exec. Note that write-only, exec-only + and write-exec mappings are not supported. When accesses to page are + denied by memory attributes, KVM **can** inform userspace through a + KVM_EXIT_MEMORY_FAULT exit, and userspace is allowed to restart the + VM after fixing the issue. + + Note however that while KVM tries to report failures as + KVM_EXIT_MEMORY_FAULT in as many cases as possible, in some cases such + accesses are not recoverable. KVM can report these accesses in the same + way as accesses to unmapped regions, for example as KVM_EXIT_SHUTDOWN, + KVM_EXIT_INTERNAL_ERROR, inject exceptions, or even by returning -EIO. + For exit reasons other than KVM_EXIT_MEMORY_FAULT, resetting the + attribute does not guarantee that execution can safely resume. + + These attributes must therefore not be used as a replacement for + userfaultfd. + + - KVM_MEMORY_ATTRIBUTE_PRIVATE - Which marks the associated gfn as being guest + private memory. Note, there is no "get" API. Userspace is responsible for explicitly tracking the state of a gfn/page as needed.
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1882,6 +1882,7 @@ enum kvm_intr_type {#define kvm_arch_has_gmem_convert () (!!kvm_x86_ops.gmem_make_private)#endif +#define kvm_arch_has_memory_protection_attr (kvm) (!(kvm)->arch.has_private_mem)#define kvm_arch_has_readonly_mem (kvm) (!(kvm)->arch.has_protected_state)#ifdef CONFIG_X86_64
arch/x86/kvm/Kconfig +1 -1
@@ -44,6 +44,7 @@ config KVM_X86 select KVM_GENERIC_PRE_FAULT_MEMORY select KVM_WERROR if WERROR select KVM_GUEST_MEMFD if X86_64 + select KVM_VM_MEMORY_ATTRIBUTES config KVM tristate "Kernel-based Virtual Machine (KVM) support"
@@ -79,7 +80,6 @@ config KVM_WERROR config KVM_PRIVATE_MEMATTR_VM bool "Enable per-VM PRIVATE vs. SHARED attributes (for CoCo VMs)" depends on KVM_SW_PROTECTED_VM || KVM_INTEL_TDX || KVM_AMD_SEV - select KVM_VM_MEMORY_ATTRIBUTES default KVM_INTEL_TDX || KVM_AMD_SEV help Enable support for tracking PRIVATE vs. SHARED memory using per-VM
arch/x86/kvm/mmu/mmu.c +47 -16
89 lines not on this page · raw message
arch/x86/kvm/x86.c +0 -2
@@ -10263,9 +10263,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm, } } -#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES kvm_mmu_init_memslot_memory_attributes (kvm, slot); -#endif if (kvm_page_track_create_memslot (kvm, slot, npages)) goto out_free;
include/linux/kvm_host.h +28 -0
@@ -726,6 +726,15 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm_vcpu *vcpu) }#endif +#ifdef kvm_arch_has_memory_protection_attr +bool kvm_arch_mem_attributes_supported_prot (struct kvm *kvm,unsigned long attrs); +#else +static inline bool kvm_arch_has_memory_protection_attr (struct kvm *kvm) +{ +return false; +} +#endif +#ifndef kvm_arch_has_readonly_memstatic inline bool kvm_arch_has_readonly_mem (struct kvm *kvm) {
@@ -2577,12 +2586,31 @@ static inline bool kvm_memslot_is_gmem_only(const struct kvm_memory_slot *slot)return slot->flags &KVM_MEMSLOT_GMEM_ONLY ; } +#define KVM_MEMORY_ATTRIBUTE_PROT \ + (KVM_MEMORY_ATTRIBUTE_NR | KVM_MEMORY_ATTRIBUTE_NW | KVM_MEMORY_ATTRIBUTE_NX) + +static inline bool kvm_mem_attributes_may_read (u64 attrs) +{ +return !(attrs &KVM_MEMORY_ATTRIBUTE_NR ); +} + +static inline bool kvm_mem_attributes_may_write (u64 attrs) +{ +return !(attrs &KVM_MEMORY_ATTRIBUTE_NW ); +} + +static inline bool kvm_mem_attributes_may_exec (u64 attrs) +{ +return !(attrs &KVM_MEMORY_ATTRIBUTE_NX ); +} +#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static inline unsigned long kvm_get_vm_memory_attributes (struct kvm *kvm,gfn_t gfn) {return xa_to_value (xa_load (&kvm->mem_attr_array, gfn)); } +bool kvm_vm_mem_attributes_valid (struct kvm *kvm,unsigned long attrs);bool kvm_range_has_vm_memory_attributes (struct kvm *kvm,gfn_t start,gfn_t end,unsigned long mask,unsigned long attrs); bool kvm_arch_pre_set_vm_memory_attributes(struct kvm *kvm,
include/uapi/linux/kvm.h +3 -0
@@ -1673,6 +1673,9 @@ struct kvm_memory_attributes2 {__u64 reserved[11 ]; }; +#define KVM_MEMORY_ATTRIBUTE_NR (1ULL << 0) +#define KVM_MEMORY_ATTRIBUTE_NW (1ULL << 1) +#define KVM_MEMORY_ATTRIBUTE_NX (1ULL << 2)#define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)#define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest_memfd)
virt/kvm/kvm_main.c +39 -7
74 lines not on this page · raw message
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 17/45] KVM: Introduce NR/NW/NX memory attributes
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] KVM: wrong macro in #else of kvm_supports_memory_protection_attr() - [Medium] KVM: x86…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] KVM: wrong macro in #else of kvm_supports_memory_protection_attr()
- [Medium] KVM: x86: selecting KVM_VM_MEMORY_ATTRIBUTES on 32-bit breaks build --
commit 7813791b6014f78cd1b4dde881abb82a354594d5
Author: Nicolas Saenz Julienne <[email protected]>
KVM: Introduce NR/NW/NX memory attributes
This commit introduces the KVM_MEMORY_ATTRIBUTE_NR, KVM_MEMORY_ATTRIBUTE_NW, and KVM_MEMORY_ATTRIBUTE_NX memory attributes to map guest physical memory regions as non-readable, non-writable, and/or non-executable, and adds basic support on x86.
arch/x86/kvm/Kconfig config KVM_X86 · line 47
Quoted 9 lines
arch/x86/kvm/Kconfig +1
-0@@ -44,6 +44,7 @@ config KVM_X86 select KVM_GENERIC_PRE_FAULT_MEMORY select KVM_WERROR if WERROR select KVM_GUEST_MEMFD if X86_64 + select KVM_VM_MEMORY_ATTRIBUTES
[Severity: Medium]
Does selecting KVM_VM_MEMORY_ATTRIBUTES unconditionally in config KVM_X86 break 32-bit x86 builds?
Previously, KVM_VM_MEMORY_ATTRIBUTES was selected by KVM_PRIVATE_MEMATTR_VM, which only depended on 64-bit options (KVM_SW_PROTECTED_VM, KVM_INTEL_TDX, and KVM_AMD_SEV). Selecting it unconditionally under KVM_X86 enables CONFIG_KVM_VM_MEMORY_ATTRIBUTES on 32-bit x86 builds, which triggers the compile-time assertion in virt/kvm/kvm_main.c:
kvm_vm_ioctl_set_mem_attributes() {
...
BUILD_BUG_ON(sizeof(attrs->attributes) != sizeof(unsigned long));
}Also, if KVM_VM_MEMORY_ATTRIBUTES is gated on X86_64, removing the CONFIG_KVM_VM_MEMORY_ATTRIBUTES guards in arch/x86/kvm/mmu/mmu.c and arch/x86/kvm/x86.c (around kvm_mmu_init_memslot_memory_attributes() in kvm_alloc_memslot_metadata(), and the calls to kvm_get_vm_memory_attributes() and kvm_range_has_vm_memory_attributes() in mmu.c) will leave those callers referencing symbols that have no fallback definitions when CONFIG_KVM_VM_MEMORY_ATTRIBUTES is disabled.
[ ... ]
Quoted 16 lines
include/linux/kvm_host.h +8
-0@@ -726,6 +726,15 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm_vcpu *vcpu) }#endif +#ifdef kvm_arch_has_memory_protection_attr +bool kvm_arch_mem_attributes_supported_prot (struct kvm *kvm,unsigned long attrs); +#else +static inline bool kvm_arch_has_memory_protection_attr (struct kvm *kvm) +{ +return false; +} +#endif
[ ... ]
Quoted 17 lines
virt/kvm/kvm_main.c +9
-0@@ -2473,13 +2473,28 @@ static u64 kvm_supports_private_mem(struct kvm *kvm)#define kvm_supports_private_mem (kvm) false#endif + +#ifdef kvm_arch_has_memory_protection_attr +static u64 kvm_supports_memory_protection_attr (struct kvm *kvm) +{ +return !kvm ||kvm_arch_has_memory_protection_attr (kvm); +} +#else +#define kvm_supports_private_mem (kvm) false +#endif
virt/kvm/kvm_main.c static u64 kvm_supports_private_mem(struct kvm *kvm) · line 2492
Quoted 10 lines
virt/kvm/kvm_main.c +5
-2+#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES u64 kvm_supported_vm_mem_attributes (struct kvm *kvm) { -if (gmem_in_place_conversion || !kvm_supports_private_mem (kvm)) -return 0 ; +u64 supported_attrs =0 ; + +if (kvm_supports_memory_protection_attr (kvm)) + supported_attrs |=KVM_MEMORY_ATTRIBUTE_PROT ;
[Severity: Medium]
Should the #else branch here define kvm_supports_memory_protection_attr(kvm) instead of kvm_supports_private_mem(kvm)?
On architectures that do not define kvm_arch_has_memory_protection_attr as a preprocessor macro, the #else branch defines kvm_supports_private_mem(kvm) as false. This leaves kvm_supports_memory_protection_attr() undeclared when called by kvm_supported_vm_mem_attributes(), while also overriding the preceding definition of kvm_supports_private_mem(kvm) on architectures that define kvm_arch_has_private_mem.
In addition, when kvm_arch_has_memory_protection_attr is not defined as a macro, include/linux/kvm_host.h omits the prototype for kvm_arch_mem_attributes_supported_prot(), which is still compiled unconditionally as a non-static __weak function in virt/kvm/kvm_main.c when CONFIG_KVM_VM_MEMORY_ATTRIBUTES is enabled, causing a missing-prototype warning.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=17
kernel test robot
build botRe: [PATCH 17/45] KVM: Introduce NR/NW/NX memory attributes
In reply to Paolo Bonzinikernel test robot noticed the following build errors:
Hi Paolo,
kernel test robot noticed the following build errors:
[auto build test ERROR on kvm/queue]
[also build test ERROR on kvm/next linus/master next-20261030] [cannot apply to mst-vhost/linux-next kvm/linux-next v7.3] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Paolo-Bonzini/KVM-nVMX-nSVM-check-unconditionally-whether-CR3-is-valid/20261102-124045 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20261102174512.8903-1-pbonzini%40redhat.com patch subject: [PATCH 17/45] KVM: Introduce NR/NW/NX memory attributes config: i386-randconfig-1010-20261103 (https://download.01.org/0day-ci/archive/20261103/[email protected]/config) compiler: gcc-14 (Debian 14.2.0-19) 14.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261103/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from <command-line>:
In function 'kvm_vm_ioctl_set_mem_attributes',
inlined from 'kvm_vm_ioctl' at arch/x86/kvm/../../../virt/kvm/kvm_main.c:5483:7:
>> include/linux/compiler_types.h:702:45: error: call to '__compiletime_assert_799' declared with attribute error: BUILD_BUG_ON failed: sizeof(attrs->attributes) != sizeof(unsigned long)
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:683:25: note: in definition of macro '__compiletime_assert'
683 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:702:9: note: in expansion of macro '_compiletime_assert'
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:40:37: note: in expansion of macro 'compiletime_assert'
40 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
| ^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:51:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
51 | BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
| ^~~~~~~~~~~~~~~~
arch/x86/kvm/../../../virt/kvm/kvm_main.c:2736:9: note: in expansion of macro 'BUILD_BUG_ON'
2736 | BUILD_BUG_ON(sizeof(attrs->attributes) != sizeof(unsigned long));
| ^~~~~~~~~~~~vim +/__compiletime_assert_799 +702 include/linux/compiler_types.h
eb5c2d4b45e3d2d Will Deacon 2020-07-21 688 eb5c2d4b45e3d2d Will Deacon 2020-07-21 689 #define _compiletime_assert(condition, msg, prefix, suffix) \ eb5c2d4b45e3d2d Will Deacon 2020-07-21 690 __compiletime_assert(condition, msg, prefix, suffix) eb5c2d4b45e3d2d Will Deacon 2020-07-21 691 eb5c2d4b45e3d2d Will Deacon 2020-07-21 692 /** eb5c2d4b45e3d2d Will Deacon 2020-07-21 693 * compiletime_assert - break build and emit msg if condition is false eb5c2d4b45e3d2d Will Deacon 2020-07-21 694 * @condition: a compile-time constant condition to check eb5c2d4b45e3d2d Will Deacon 2020-07-21 695 * @msg: a message to emit if condition is false eb5c2d4b45e3d2d Will Deacon 2020-07-21 696 * eb5c2d4b45e3d2d Will Deacon 2020-07-21 697 * In tradition of POSIX assert, this macro will break the build if the eb5c2d4b45e3d2d Will Deacon 2020-07-21 698 * supplied condition is *false*, emitting the supplied error message if the eb5c2d4b45e3d2d Will Deacon 2020-07-21 699 * compiler has support to do so. eb5c2d4b45e3d2d Will Deacon 2020-07-21 700 */ eb5c2d4b45e3d2d Will Deacon 2020-07-21 701 #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2d Will Deacon 2020-07-21 @702 _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)eb5c2d4b45e3d2d Will Deacon 2020-07-21 703
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Paolo Bonzini
maintainerPATCH v4 18/45 KVM: Include memory protections in result of gfn->hva conversion
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
All paths that lead to guest memory accesses now need to check whether memory attributes allow that access. Users of gfn_to_hva, kvm_vcpu_gfn_to_hva and their *_prot variant can get it more or less for free via erroneous return values, so do this first.
Note however that this is not true of the variants that take a cached kvm_memslots pointer. These include caches (gfn-to-hva and gfn-to-pfn) and page faults, both of which will need specific changes; but the more optimized functions in virt/kvm/kvm_main.c such as kvm_read_guest() and kvm_write_guest() also retrieve the memslot high in the call chain, and therefore they will need changes in __kvm_read/write_guest_page().
--- include/linux/kvm_host.h | 19 ++++++++++++++ virt/kvm/kvm_main.c | 55 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 70 insertions(+), 4 deletions(-)
include/linux/kvm_host.h +19 -0
@@ -2623,6 +2623,11 @@ static inline u64 kvm_supported_vm_mem_attributes(struct kvm *kvm) {return 0 ; } + +static inline unsigned long kvm_get_vm_memory_attributes (struct kvm *kvm,gfn_t gfn) +{ +return 0 ; +}#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ #ifdef CONFIG_KVM_PRIVATE_MEMATTR_VM
@@ -2653,6 +2658,20 @@ static inline bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn) }#endif /* kvm_arch_has_private_mem */ +static inline int kvm_mem_attributes_may_read_gfn (struct kvm *kvm,gfn_t gfn) +{ +unsigned long attrs =kvm_get_vm_memory_attributes (kvm, gfn); + +return kvm_mem_attributes_may_read (attrs); +} + +static inline int kvm_mem_attributes_may_write_gfn (struct kvm *kvm,gfn_t gfn) +{ +unsigned long attrs =kvm_get_vm_memory_attributes (kvm, gfn); + +return kvm_mem_attributes_may_write (attrs); +} +#ifdef CONFIG_KVM_GUEST_MEMFD bool kvm_gmem_is_private_gfn (struct kvm *kvm,gfn_t gfn);bool kvm_arch_supports_gmem_init_shared (struct kvm *kvm);
virt/kvm/kvm_main.c +51 -4
79 lines not on this page · raw message
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 19/45 KVM: Introduce kvm_fetch_guest_page() and use it for x86
In reply to Paolo BonziniIntroduce a separate function so that the emulator will be able to take into account execution memory attributes rather than read.
--- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 9 +++++++++ 3 files changed, 13 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
@@ -4916,8 +4916,8 @@ static int kvm_fetch_guest_virt(struct x86_emulate_ctxt *ctxt, offset = addr & (PAGE_SIZE -1 );if (WARN_ON (offset + bytes >PAGE_SIZE )) bytes = (unsigned )PAGE_SIZE - offset; - ret =kvm_vcpu_read_guest_page (vcpu, gpa >>PAGE_SHIFT , val, - offset, bytes); + ret =kvm_vcpu_fetch_guest_page (vcpu, gpa >>PAGE_SHIFT , val, + offset, bytes);if (unlikely (ret <0 ))return X86EMUL_IO_NEEDED ;
include/linux/kvm_host.h +2 -0
@@ -1456,6 +1456,8 @@ DEFINE_VCPU_MAP_CLASS(_readonly);unsigned long kvm_vcpu_gfn_to_hva (struct kvm_vcpu *vcpu,gfn_t gfn);unsigned long kvm_vcpu_gfn_to_hva_prot (struct kvm_vcpu *vcpu,gfn_t gfn,bool *writable); +int kvm_vcpu_fetch_guest_page (struct kvm_vcpu *vcpu,gfn_t gfn,void *data, +int offset,int len);int kvm_vcpu_read_guest_page (struct kvm_vcpu *vcpu,gfn_t gfn,void *data,int offset,int len); int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, void *data,
virt/kvm/kvm_main.c +9 -0
@@ -3398,6 +3398,15 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_read_guest_page); +int kvm_vcpu_fetch_guest_page (struct kvm_vcpu *vcpu,gfn_t gfn,void *data, +int offset,int len) +{ +struct kvm_memory_slot *slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn); + +return __kvm_read_guest_page (slot, gfn, data, offset, len); +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_fetch_guest_page); +int kvm_read_guest (struct kvm *kvm,gpa_t gpa,void *data,unsigned long len) {gfn_t gfn = gpa >>PAGE_SHIFT ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 20/45 KVM: Take memory protections into account for memory read/write/fetch
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Take into account memory attributes when accessing guest memory through the kvm_{read,write,fetch}*() family of functions.
All of these pass a struct kvm_memory_slot pointer to the actual workhorse functions, in order to share code between the VM-wide and vCPU-specific version of the functions (the latter of which handles the multi-address-space case). For this reason they need specific changes and do not work even though the gfn_to_hva() path has been taught already about memory protection attributes.
--- virt/kvm/kvm_main.c | 34 ++++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-)
virt/kvm/kvm_main.c +26 -8
@@ -3362,11 +3362,13 @@ static int next_segment(unsigned long len, int offset) }/* Copy @len bytes from guest memory at '(@gfn * PAGE_SIZE) + @offset' to @data */ -static int __kvm_read_guest_page (struct kvm_memory_slot *slot,gfn_t gfn, -void *data,int offset,int len) +static int __kvm_read_guest_page (struct kvm *kvm,struct kvm_memory_slot *slot, +gfn_t gfn,void *data,int offset,int len, +unsigned long attr) {int r;unsigned long addr; +unsigned long attrs;if (WARN_ON_ONCE (offset + len >PAGE_SIZE ))return -EFAULT ;
@@ -3374,6 +3376,11 @@ static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn, addr =gfn_to_hva_memslot_prot (slot, gfn,NULL );if (kvm_is_error_hva (addr))return -EFAULT ; + + attrs =kvm_get_vm_memory_attributes (kvm, gfn); +if (attrs & attr) +return -EFAULT ; + r =__copy_from_user (data, (void __user *)addr + offset, len);if (r)return -EFAULT ;
@@ -3385,7 +3392,8 @@ int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, {struct kvm_memory_slot *slot =gfn_to_memslot (kvm, gfn); -return __kvm_read_guest_page (slot, gfn, data, offset, len); +return __kvm_read_guest_page (kvm, slot, gfn, data, offset, len, +KVM_MEMORY_ATTRIBUTE_NR ); }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_read_guest_page);
@@ -3394,7 +3402,8 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, {struct kvm_memory_slot *slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn); -return __kvm_read_guest_page (slot, gfn, data, offset, len); +return __kvm_read_guest_page (vcpu->kvm, slot, gfn, data, offset, len, +KVM_MEMORY_ATTRIBUTE_NR ); }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_read_guest_page);
@@ -3403,7 +3412,8 @@ int kvm_vcpu_fetch_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, {struct kvm_memory_slot *slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn); -return __kvm_read_guest_page (slot, gfn, data, offset, len); +return __kvm_read_guest_page (vcpu->kvm, slot, gfn, data, offset, len, +KVM_MEMORY_ATTRIBUTE_NX ); }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_fetch_guest_page);
@@ -3447,8 +3457,9 @@ int kvm_vcpu_read_guest(struct kvm_vcpu *vcpu, gpa_t gpa, void *data, unsigned l }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_read_guest); -static int __kvm_read_guest_atomic (struct kvm_memory_slot *slot,gfn_t gfn, -void *data,int offset,unsigned long len) +static int __kvm_read_guest_atomic (struct kvm *kvm, +struct kvm_memory_slot *slot,gfn_t gfn, +void *data,int offset,unsigned long len) {int r;unsigned long addr;
@@ -3456,6 +3467,9 @@ static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn,if (WARN_ON_ONCE (offset + len >PAGE_SIZE ))return -EFAULT ; +if (!kvm_mem_attributes_may_read_gfn (kvm, gfn)) +return -EFAULT ; + addr =gfn_to_hva_memslot_prot (slot, gfn,NULL );if (kvm_is_error_hva (addr))return -EFAULT ;
@@ -3474,7 +3488,7 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa,struct kvm_memory_slot *slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn);int offset =offset_in_page (gpa); -return __kvm_read_guest_atomic (slot, gfn, data, offset, len); +return __kvm_read_guest_atomic (vcpu->kvm, slot, gfn, data, offset, len); }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_read_guest_atomic);
@@ -3492,6 +3506,10 @@ static int __kvm_write_guest_page(struct kvm *kvm, addr =gfn_to_hva_memslot (memslot, gfn);if (kvm_is_error_hva (addr))return -EFAULT ; + +if (!kvm_mem_attributes_may_write_gfn (kvm, gfn)) +return -EFAULT ; + r =__copy_to_user ((void __user *)addr + offset, data, len);if (r)return -EFAULT ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 21/45 KVM: Take memory protections into account for __kvm_vcpu_map
In reply to Paolo BonziniTake into account memory attributes when establishing a long term map through kvm_vcpu_map()/kvm_vcpu_map_readonly().
Unlike the gfn-to-pfn cache, changes to the attributes will not be propagated, so that it will be possible to place a nested VMCS, VMCB, or APIC access page on a non-readable/writable page. This will be fixed separately by using a gfn-to-pfn cache for nested virtualization data structures, in the meanwhile apply a stopgap fix.
--- virt/kvm/kvm_main.c | 5 +++++ 1 file changed, 5 insertions(+)
virt/kvm/kvm_main.c +5 -0
@@ -3302,6 +3302,11 @@ int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map,if (WARN_ON_ONCE (map->hva))kvm_vcpu_unmap (vcpu, map); +if (!kvm_mem_attributes_may_read_gfn (vcpu->kvm, gfn)) +return -EFAULT ; +if (writable && !kvm_mem_attributes_may_write_gfn (vcpu->kvm, gfn)) +return -EFAULT ; + map->pinned_page =NULL ; map->page =NULL ; map->hva =NULL ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 22/45 KVM: Encapsulate memattrs array into anonymous struct
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
The metadata surrounding memory attributes is about to grow, so encapsulate the memory attributes array within an anonymous struct to provide namespacing.
--- include/linux/kvm_host.h | 8 +++++--- virt/kvm/kvm_main.c | 10 +++++----- 2 files changed, 10 insertions(+), 8 deletions(-)
include/linux/kvm_host.h +5 -3
@@ -876,8 +876,10 @@ struct kvm {struct notifier_block pm_notifier;#endif #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES -/* Protected by slots_lock (for writes) and RCU (for reads) */ -struct xarray mem_attr_array; +struct { +/* Protected by slots_lock (for writes) and RCU (for reads) */ +struct xarray array; + } mem_attrs;#endif char stats_id[KVM_STATS_NAME_SIZE ]; };
@@ -2609,7 +2611,7 @@ static inline bool kvm_mem_attributes_may_exec(u64 attrs)#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES static inline unsigned long kvm_get_vm_memory_attributes (struct kvm *kvm,gfn_t gfn) { -return xa_to_value (xa_load (&kvm->mem_attr_array, gfn)); +return xa_to_value (xa_load (&kvm->mem_attrs.array, gfn)); }bool kvm_vm_mem_attributes_valid (struct kvm *kvm,unsigned long attrs);
virt/kvm/kvm_main.c +5 -5
@@ -1135,7 +1135,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)rcuwait_init (&kvm->mn_memslots_update_rcuwait);xa_init (&kvm->vcpu_array);#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES -xa_init_flags (&kvm->mem_attr_array,XA_FLAGS_ACCOUNT ); +xa_init_flags (&kvm->mem_attrs.array,XA_FLAGS_ACCOUNT );#endif INIT_LIST_HEAD (&kvm->gpc_list);
@@ -1320,7 +1320,7 @@ static void kvm_destroy_vm(struct kvm *kvm)srcu_barrier (&kvm->srcu);cleanup_srcu_struct (&kvm->srcu);#ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES -xa_destroy (&kvm->mem_attr_array); +xa_destroy (&kvm->mem_attrs.array);#endif kvm_arch_free_vm (kvm);preempt_notifier_dec ();
@@ -2504,7 +2504,7 @@ u64 kvm_supported_vm_mem_attributes(struct kvm *kvm)bool kvm_range_has_vm_memory_attributes (struct kvm *kvm,gfn_t start,gfn_t end,unsigned long mask,unsigned long attrs) { -XA_STATE (xas, &kvm->mem_attr_array, start); +XA_STATE (xas, &kvm->mem_attrs.array, start);unsigned long supported_attrs;unsigned long index;void *entry;
@@ -2668,7 +2668,7 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,* allocates, so no reservations are needed when clearing. */ for (i = start; entry && i < end; i++) { - r =xa_reserve (&kvm->mem_attr_array, i,GFP_KERNEL_ACCOUNT ); + r =xa_reserve (&kvm->mem_attrs.array, i,GFP_KERNEL_ACCOUNT );if (r) goto out_unlock;
@@ -2678,7 +2678,7 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,kvm_handle_gfn_range (kvm, &pre_set_range);for (i = start; i < end; i++) { - r =xa_err (xa_store (&kvm->mem_attr_array, i, entry, + r =xa_err (xa_store (&kvm->mem_attrs.array, i, entry,GFP_KERNEL_ACCOUNT ));KVM_BUG_ON (r, kvm);cond_resched ();
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 23/45 KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen()
In reply to Paolo BonziniIn many cases, retrieving kvm_memslots is followed by a check on the generation of the slots. Introduce a helper function that either does the check alone, or compounds it with returning the struct kvm_memslots* to the caller.
--- arch/loongarch/kvm/vcpu.c | 12 ++++-------- arch/x86/kvm/x86.c | 10 ++-------- include/linux/kvm_host.h | 13 +++++++++++++ virt/kvm/kvm_main.c | 8 ++++---- virt/kvm/pfncache.c | 10 ++++------ 5 files changed, 27 insertions(+), 26 deletions(-)
arch/loongarch/kvm/vcpu.c +4 -8
@@ -154,7 +154,6 @@ static void kvm_update_stolen_time(struct kvm_vcpu *vcpu)u32 version;u64 steal;gpa_t gpa; -struct kvm_memslots *slots;struct kvm_steal_time __user *st;struct gfn_to_hva_cache *ghc;
@@ -164,9 +163,8 @@ static void kvm_update_stolen_time(struct kvm_vcpu *vcpu)return ; gpa &=KVM_STEAL_PHYS_MASK ; - slots =kvm_memslots (vcpu->kvm); -if (unlikely (slots->generation != ghc->generation || gpa != ghc->gpa || -kvm_is_error_hva (ghc->hva) || !ghc->memslot)) { +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || + kvm_is_error_hva(ghc->hva) || !ghc->memslot) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;return ;
@@ -1803,7 +1801,6 @@ static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) {gpa_t gpa;struct gfn_to_hva_cache *ghc; -struct kvm_memslots *slots;struct kvm_steal_time __user *st; gpa = vcpu->arch.st.guest_addr;
@@ -1820,9 +1817,8 @@ static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) gpa &=KVM_STEAL_PHYS_MASK ; ghc = &vcpu->arch.st.cache; - slots =kvm_memslots (vcpu->kvm); -if (unlikely (slots->generation != ghc->generation || gpa != ghc->gpa || -kvm_is_error_hva (ghc->hva) || !ghc->memslot)) { +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || + kvm_is_error_hva(ghc->hva) || !ghc->memslot) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;return ;
arch/x86/kvm/x86.c +2 -8
@@ -2062,7 +2062,6 @@ static void record_steal_time(struct kvm_vcpu *vcpu) {struct gfn_to_hva_cache *ghc = &vcpu->arch.st.cache;struct kvm_steal_time __user *st; -struct kvm_memslots *slots;gpa_t gpa = vcpu->arch.st.msr_val &KVM_STEAL_VALID_BITS ;u64 steal;u32 version;
@@ -2078,9 +2077,7 @@ static void record_steal_time(struct kvm_vcpu *vcpu)if (WARN_ON_ONCE (current->mm != vcpu->kvm->mm))return ; - slots =kvm_memslots (vcpu->kvm); - -if (unlikely (slots->generation != ghc->generation || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa ||kvm_is_error_hva (ghc->hva) || !ghc->memslot)) {/* We rely on the fact that it fits in a single page. */
@@ -2630,7 +2627,6 @@ static void kvm_steal_time_set_preempted(struct kvm_vcpu *vcpu) {struct gfn_to_hva_cache *ghc = &vcpu->arch.st.cache;struct kvm_steal_time __user *st; -struct kvm_memslots *slots;static const u8 preempted =KVM_VCPU_PREEMPTED ;gpa_t gpa = vcpu->arch.st.msr_val &KVM_STEAL_VALID_BITS ;
@@ -2657,9 +2653,7 @@ static void kvm_steal_time_set_preempted(struct kvm_vcpu *vcpu)if (unlikely (current->mm != vcpu->kvm->mm))return ; - slots =kvm_memslots (vcpu->kvm); - -if (unlikely (slots->generation != ghc->generation || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa ||kvm_is_error_hva (ghc->hva) || !ghc->memslot))return ;
include/linux/kvm_host.h +13 -0
@@ -2676,6 +2676,19 @@ static inline int kvm_mem_attributes_may_write_gfn(struct kvm *kvm, gfn_t gfn)return kvm_mem_attributes_may_write (attrs); } +static inline bool kvm_memslots_check_gen (struct kvm *kvm,u64 slots_generation,struct kvm_memslots **p_slots) +{ +struct kvm_memslots *slots = *p_slots =kvm_memslots (kvm); +return slots->generation == slots_generation; +} + +static inline bool kvm_check_gen (struct kvm *kvm,u64 slots_generation) +{ +struct kvm_memslots *slots; +return kvm_memslots_check_gen (kvm, slots_generation, &slots); +} + +#ifdef CONFIG_KVM_GUEST_MEMFD bool kvm_gmem_is_private_gfn (struct kvm *kvm,gfn_t gfn);bool kvm_arch_supports_gmem_init_shared (struct kvm *kvm);
virt/kvm/kvm_main.c +4 -4
@@ -3635,14 +3635,14 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned int offset, unsigned long len) { - struct kvm_memslots *slots = kvm_memslots(kvm); + struct kvm_memslots *slots;int r;gpa_t gpa = ghc->gpa + offset;if (WARN_ON_ONCE (len + offset > ghc->len))return -EINVAL ; -if (slots->generation != ghc->generation) { +if (unlikely (!kvm_memslots_check_gen (kvm, ghc->generation, &slots))) {if (__kvm_gfn_to_hva_cache_init (slots, ghc, ghc->gpa, ghc->len))return -EFAULT ; }
@@ -3673,14 +3673,14 @@ int kvm_read_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc, void *data, unsigned int offset, unsigned long len) { - struct kvm_memslots *slots = kvm_memslots(kvm); + struct kvm_memslots *slots;int r;gpa_t gpa = ghc->gpa + offset;if (WARN_ON_ONCE (len + offset > ghc->len))return -EINVAL ; -if (slots->generation != ghc->generation) { +if (unlikely (!kvm_memslots_check_gen (kvm, ghc->generation, &slots))) {if (__kvm_gfn_to_hva_cache_init (slots, ghc, ghc->gpa, ghc->len))return -EFAULT ; }
virt/kvm/pfncache.c +4 -6
@@ -72,8 +72,6 @@ static bool kvm_gpc_is_valid_len(gpa_t gpa, unsigned long uhva,bool kvm_gpc_check (struct gfn_to_pfn_cache *gpc,unsigned long len) { -struct kvm_memslots *slots =kvm_memslots (gpc->kvm); -if (!gpc->active)return false;
@@ -81,7 +79,7 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len)* If the page was cached from a memslot, make sure the memslots have * not been re-configured. */ -if (!kvm_is_error_gpa (gpc->gpa) && gpc->generation != slots->generation) +if (!kvm_is_error_gpa (gpc->gpa) && !kvm_check_gen (gpc->kvm, gpc->generation))return false;if (kvm_is_error_hva (gpc->uhva))
@@ -290,12 +288,12 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned lif (gpc->uhva != old_uhva) hva_change = true; }else { -struct kvm_memslots *slots =kvm_memslots (gpc->kvm); +struct kvm_memslots *slots; page_offset =offset_in_page (gpa); -if (gpc->gpa != gpa || gpc->generation != slots->generation || -kvm_is_error_hva (gpc->uhva)) { +if (!kvm_memslots_check_gen (gpc->kvm, gpc->generation, &slots) || + gpc->gpa != gpa ||kvm_is_error_hva (gpc->uhva)) {gfn_t gfn =gpa_to_gfn (gpa); gpc->gpa = gpa;
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 23/45] KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] KVM: loongarch: missing closing parenthesis in steal time if conditions --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] KVM: loongarch: missing closing parenthesis in steal time if conditions --
commit 12eefe95ee8b4f21b2235f3a2ed98ccc73873acc
Author: Paolo Bonzini <[email protected]>
KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen()
This commit introduces the kvm_check_gen() and kvm_memslots_check_gen() helper functions to check the memslot generation and optionally return the kvm_memslots pointer, converting several callers across KVM to use them.
Quoted 34 lines
arch/loongarch/kvm/vcpu.c +2
-5@@ -154,7 +154,6 @@ static void kvm_update_stolen_time(struct kvm_vcpu *vcpu)u32 version;u64 steal;gpa_t gpa; -struct kvm_memslots *slots;struct kvm_steal_time __user *st;struct gfn_to_hva_cache *ghc;@@ -164,9 +163,8 @@ static void kvm_update_stolen_time(struct kvm_vcpu *vcpu)return ; gpa &=KVM_STEAL_PHYS_MASK ; - slots =kvm_memslots (vcpu->kvm); -if (unlikely (slots->generation != ghc->generation || gpa != ghc->gpa || -kvm_is_error_hva (ghc->hva) || !ghc->memslot)) { +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || + kvm_is_error_hva(ghc->hva) || !ghc->memslot) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;return ;@@ -1803,7 +1801,6 @@ static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) {gpa_t gpa;struct gfn_to_hva_cache *ghc; -struct kvm_memslots *slots;struct kvm_steal_time __user *st; gpa = vcpu->arch.st.guest_addr;@@ -1820,9 +1817,8 @@ static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu)
arch/loongarch/kvm/vcpu.c static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) · line 1824
Quoted 10 lines
arch/loongarch/kvm/vcpu.c +2
-3gpa &=KVM_STEAL_PHYS_MASK ; ghc = &vcpu->arch.st.cache; - slots =kvm_memslots (vcpu->kvm); -if (unlikely (slots->generation != ghc->generation || gpa != ghc->gpa || -kvm_is_error_hva (ghc->hva) || !ghc->memslot)) { +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || + kvm_is_error_hva(ghc->hva) || !ghc->memslot) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;return ;
[Severity: Low]
In both kvm_update_stolen_time() and kvm_vcpu_set_pv_preempted() in arch/loongarch/kvm/vcpu.c, replacing the generation check dropped one of the closing parentheses at the end of !ghc->memslot)) {, leaving the outer if condition unclosed and breaking the LoongArch KVM build.
Note that a later commit in the series (c500cbe0be12) updates the arguments to kvm_check_gen() at both sites without restoring the missing closing parenthesis.
Should both conditions end with !ghc->memslot)) { instead?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=23
kernel test robot
build botRe: [PATCH 23/45] KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen()
In reply to Paolo Bonzinikernel test robot noticed the following build errors:
Hi Paolo,
kernel test robot noticed the following build errors:
[auto build test ERROR on kvm/queue]
[also build test ERROR on kvm/next linus/master next-20261030] [cannot apply to mst-vhost/linux-next kvm/linux-next v7.3] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Paolo-Bonzini/KVM-nVMX-nSVM-check-unconditionally-whether-CR3-is-valid/20261102-124045 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20261102174512.8903-7-pbonzini%40redhat.com patch subject: [PATCH 23/45] KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen() config: loongarch-defconfig (https://download.01.org/0day-ci/archive/20261104/[email protected]/config) compiler: clang version 24.0.0git (https://github.com/llvm/llvm-project 65303e81f481bbfe386b8596423f94c43198ba94) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261104/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> arch/loongarch/kvm/vcpu.c:167:53: error: expected ')'
167 | kvm_is_error_hva(ghc->hva) || !ghc->memslot) {
| ^
arch/loongarch/kvm/vcpu.c:166:5: note: to match this '('
166 | if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->generation) || gpa != ghc->gpa ||
| ^
arch/loongarch/kvm/vcpu.c:1821:53: error: expected ')'
1821 | kvm_is_error_hva(ghc->hva) || !ghc->memslot) {
| ^
arch/loongarch/kvm/vcpu.c:1820:5: note: to match this '('
1820 | if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->generation) || gpa != ghc->gpa ||
| ^2 errors generated.
vim +167 arch/loongarch/kvm/vcpu.c
151
152 static void kvm_update_stolen_time(struct kvm_vcpu *vcpu)
153 {
154 u32 version;
155 u64 steal;
156 gpa_t gpa;
157 struct kvm_steal_time __user *st;
158 struct gfn_to_hva_cache *ghc;
159
160 ghc = &vcpu->arch.st.cache;
161 gpa = vcpu->arch.st.guest_addr;
162 if (!(gpa & KVM_STEAL_PHYS_VALID))
163 return;
164
165 gpa &= KVM_STEAL_PHYS_MASK;
166 if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->generation) || gpa != ghc->gpa ||
> 167 kvm_is_error_hva(ghc->hva) || !ghc->memslot) { 168 if (kvm_gfn_to_hva_cache_init(vcpu->kvm, ghc, gpa, sizeof(*st))) {
169 ghc->gpa = INVALID_GPA;
170 return;
171 }
172 }
173
174 st = (struct kvm_steal_time __user *)ghc->hva;
175 if (kvm_guest_has_pv_feature(vcpu, KVM_FEATURE_PREEMPT)) {
176 unsafe_put_user(0, &st->preempted, out);
177 vcpu->arch.st.preempted = 0;
178 }
179
180 unsafe_get_user(version, &st->version, out);
181 if (version & 1)
182 version += 1; /* first time write, random junk */
183
184 version += 1;
185 unsafe_put_user(version, &st->version, out);
186 smp_wmb();
187
188 unsafe_get_user(steal, &st->steal, out);
189 steal += current->sched_info.run_delay - vcpu->arch.st.last_steal;
190 vcpu->arch.st.last_steal = current->sched_info.run_delay;
191 unsafe_put_user(steal, &st->steal, out);
192
193 smp_wmb();
194 version += 1;
195 unsafe_put_user(version, &st->version, out);
196 out:
197 mark_page_dirty_in_slot(vcpu->kvm, ghc->memslot, gpa_to_gfn(ghc->gpa));
198 }
199
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Paolo Bonzini
maintainerPATCH v4 24/45 KVM: Introduce a generation number for memory attributes
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Introduce a generation number to track memory attribute modifications. This will allow KVM components to invalidate any assumptions they might have about guest's physical addresses and their permissions when memory attributes change.
Like with memory slot updates, it's mandatory for components that access guest memory based on cached information to do so within a KVM SRCU read-side critical section, and that they validate the generation number before accessing memory. This, in combination with the synchronize_srcu() call within the memory attributes ioctl handler, ensures the following:
- A memory attribute modification operation only returns after all users of outdated GPA data are done running.
- Any component accessing cached data after the memory attribute modification returned will see the updated generation number.
Additionally, loads/stores of the generation number have acquire/release semantics; which ensures all attribute writes are visible before updating the generation, and loads from attributes happen after having read the current generation number.
Ultimately, since synchronize_srcu_expedited() is an expensive operation, only perform it when absolutely necessary. Do so if the introduced memory attribute is known to require synchronization or if the attribute being cleared contained a memory attribute that required synchronization.
There shouldn't be any performance loss for memory attributes that don't require synchronization (and in general for any VMM that does not apply memory protections), because the attributes generation will always remain unchanged.
--- arch/loongarch/kvm/vcpu.c | 6 ++- arch/x86/kvm/x86.c | 4 +- include/linux/kvm_host.h | 33 +++++++++++++-- include/linux/kvm_types.h | 6 ++- include/trace/events/kvm.h | 14 +++++-- virt/kvm/kvm_main.c | 82 +++++++++++++++++++++++++++++++++----- virt/kvm/pfncache.c | 13 +++--- 7 files changed, 129 insertions(+), 29 deletions(-)
arch/loongarch/kvm/vcpu.c +4 -2
@@ -163,7 +163,8 @@ static void kvm_update_stolen_time(struct kvm_vcpu *vcpu)return ; gpa &=KVM_STEAL_PHYS_MASK ; -if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->slots_generation, ghc->attrs_generation) || + gpa != ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;
@@ -1817,7 +1818,8 @@ static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) gpa &=KVM_STEAL_PHYS_MASK ; ghc = &vcpu->arch.st.cache; -if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->slots_generation, ghc->attrs_generation) || + gpa != ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;
arch/x86/kvm/x86.c +2 -2
@@ -2077,7 +2077,7 @@ static void record_steal_time(struct kvm_vcpu *vcpu)if (WARN_ON_ONCE (current->mm != vcpu->kvm->mm))return ; -if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->slots_generation, ghc->attrs_generation) || gpa != ghc->gpa ||kvm_is_error_hva (ghc->hva) || !ghc->memslot)) {/* We rely on the fact that it fits in a single page. */
@@ -2653,7 +2653,7 @@ static void kvm_steal_time_set_preempted(struct kvm_vcpu *vcpu)if (unlikely (current->mm != vcpu->kvm->mm))return ; -if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->slots_generation, ghc->attrs_generation) || gpa != ghc->gpa ||kvm_is_error_hva (ghc->hva) || !ghc->memslot))return ;
include/linux/kvm_host.h +29 -4
@@ -879,6 +879,7 @@ struct kvm {struct {/* Protected by slots_lock (for writes) and RCU (for reads) */ struct xarray array; +u64 generation; } mem_attrs;#endif char stats_id[KVM_STATS_NAME_SIZE ];
@@ -2621,7 +2622,23 @@ bool kvm_arch_pre_set_vm_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range);bool kvm_arch_post_set_vm_memory_attributes (struct kvm *kvm,struct kvm_gfn_range *range); +bool kvm_range_has_any_vm_memory_attributes (struct kvm *kvm,gfn_t start,gfn_t end, +unsigned long mask);u64 kvm_supported_vm_mem_attributes (struct kvm *kvm); + +static inline u64 kvm_mem_attributes_generation (struct kvm *kvm) +{ +RCU_LOCKDEP_WARN (!lockdep_is_held (&kvm->slots_lock) && + !srcu_read_lock_held (&kvm->srcu), +"Suspicious memory attribute generation usage\n" ); + +/* +* The acquire pairs with the release in kvm_vm_set_mem_attributes(). +* Memory attributes should only be queried _after_ storing the +* generation number. +*/ +return smp_load_acquire (&kvm->mem_attrs.generation); +}#else static inline u64 kvm_supported_vm_mem_attributes (struct kvm *kvm) {
@@ -2632,6 +2649,11 @@ static inline unsigned long kvm_get_vm_memory_attributes(struct kvm *kvm, gfn_t {return 0 ; } + +static inline u64 kvm_mem_attributes_generation (struct kvm *kvm) +{ +return 0 ; +}#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */ #ifdef CONFIG_KVM_PRIVATE_MEMATTR_VM
@@ -2676,16 +2698,19 @@ static inline int kvm_mem_attributes_may_write_gfn(struct kvm *kvm, gfn_t gfn)return kvm_mem_attributes_may_write (attrs); } -static inline bool kvm_memslots_check_gen (struct kvm *kvm,u64 slots_generation,struct kvm_memslots **p_slots) +static inline bool kvm_memslots_check_gen (struct kvm *kvm,u64 slots_generation, +u64 attrs_generation,struct kvm_memslots **p_slots) {struct kvm_memslots *slots = *p_slots =kvm_memslots (kvm); -return slots->generation == slots_generation; +return slots->generation == slots_generation && +kvm_mem_attributes_generation (kvm) == attrs_generation; } -static inline bool kvm_check_gen (struct kvm *kvm,u64 slots_generation) +static inline bool kvm_check_gen (struct kvm *kvm,u64 slots_generation, +u64 attrs_generation) {struct kvm_memslots *slots; -return kvm_memslots_check_gen (kvm, slots_generation, &slots); +return kvm_memslots_check_gen (kvm, slots_generation, attrs_generation, &slots); }
include/linux/kvm_types.h +4 -2
@@ -74,7 +74,8 @@ typedef u64 hfn_t;typedef hfn_t kvm_pfn_t ;struct gfn_to_hva_cache { -u64 generation; +u64 slots_generation; +u64 attrs_generation;gpa_t gpa;unsigned long hva;unsigned long len;
@@ -82,7 +83,8 @@ struct gfn_to_hva_cache { };struct gfn_to_pfn_cache { -u64 generation; +u64 slots_generation; +u64 attrs_generation;gpa_t gpa;unsigned long uhva;struct kvm_memory_slot *memslot;
include/trace/events/kvm.h +10 -4
@@ -371,23 +371,29 @@ TRACE_EVENT(kvm_dirty_ring_exit,* @attr: The value of the attribute being set. */ TRACE_EVENT (kvm_set_vm_mem_attributes, - TP_PROTO(gfn_t start, gfn_t end, unsigned long attr), -TP_ARGS (start, end, attr), + TP_PROTO(gfn_t start, gfn_t end, unsigned long attr, bool sync, u64 generation), +TP_ARGS (start, end, attr, sync, generation),TP_STRUCT__entry ( __field(gfn_t, start) __field(gfn_t, end) __field(unsigned long, attr) + __field(bool, sync) +__field (u64, generation) ),TP_fast_assign ( __entry->start = start; __entry->end = end; __entry->attr = attr; + __entry->sync = sync; + __entry->generation = generation; ), -TP_printk ("%#016llx -- %#016llx [0x%lx]" , - __entry->start, __entry->end, __entry->attr) +TP_printk ("%#016llx -- %#016llx [0x%lx], sync %d gen %llu" , + __entry->start, __entry->end, __entry->attr, + __entry->sync, __entry->generation) + );#endif /* CONFIG_KVM_VM_MEMORY_ATTRIBUTES */
virt/kvm/kvm_main.c +72 -10
158 lines not on this page · raw message
virt/kvm/pfncache.c +8 -5
@@ -76,10 +76,11 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len)return false;/* -* If the page was cached from a memslot, make sure the memslots have -* not been re-configured. +* If the page was cached from a memslot, make sure the memslots nor +* memory attributes have not been re-configured. */ -if (!kvm_is_error_gpa (gpc->gpa) && !kvm_check_gen (gpc->kvm, gpc->generation)) +if (!kvm_is_error_gpa (gpc->gpa) && + !kvm_check_gen (gpc->kvm, gpc->slots_generation, gpc->attrs_generation))return false;if (kvm_is_error_hva (gpc->uhva))
@@ -253,6 +254,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)static int __kvm_gpc_refresh (struct gfn_to_pfn_cache *gpc,gpa_t gpa,unsigned long uhva) { +struct kvm *kvm = gpc->kvm;unsigned long page_offset;bool unmap_old = false;unsigned long old_uhva;
@@ -292,12 +294,13 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l page_offset =offset_in_page (gpa); -if (!kvm_memslots_check_gen (gpc->kvm, gpc->generation, &slots) || +if (!kvm_memslots_check_gen (gpc->kvm, gpc->slots_generation, gpc->attrs_generation, &slots) || gpc->gpa != gpa ||kvm_is_error_hva (gpc->uhva)) {gfn_t gfn =gpa_to_gfn (gpa); + gpc->attrs_generation =kvm_mem_attributes_generation (kvm); gpc->gpa = gpa; - gpc->generation = slots->generation; + gpc->slots_generation = slots->generation; gpc->memslot =__gfn_to_memslot (slots, gfn); gpc->uhva =gfn_to_hva_memslot (gpc->memslot, gfn);
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 24/45] KVM: Introduce a generation number for memory attributes
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] KVM: RCU stall in kvm_range_has_any_vm_memory_attributes() via xas_next() - [Medium] KVM…
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] KVM: RCU stall in kvm_range_has_any_vm_memory_attributes() via xas_next()
- [Medium] KVM: unconditional generation bump and error-path sync in kvm_set_vm_mem_attributes()
- [Low] KVM: loongarch: missing closing parenthesis in kvm_check_gen() conditions
- [Low] KVM: stale attrs_generation on error in __kvm_gfn_to_hva_cache_init()
--
commit c500cbe0be12ea140c87b086c8905238fbefe1ea
Author: Nicolas Saenz Julienne <[email protected]>
KVM: Introduce a generation number for memory attributes
This commit introduces a generation number for VM memory attributes so that KVM components using gfn_to_hva_cache and gfn_to_pfn_cache can detect memory attribute updates and synchronize via KVM SRCU when attributes requiring synchronization are modified.
Quoted 15 lines
arch/loongarch/kvm/vcpu.c +2
-1@@ -163,7 +163,8 @@ static void kvm_update_stolen_time(struct kvm_vcpu *vcpu)return ; gpa &=KVM_STEAL_PHYS_MASK ; -if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->slots_generation, ghc->attrs_generation) || + gpa != ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot) {if (kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, gpa,sizeof (*st))) { ghc->gpa =INVALID_GPA ;
arch/loongarch/kvm/vcpu.c static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) · line 1823
Quoted 7 lines
arch/loongarch/kvm/vcpu.c +2
-1@@ -1817,7 +1818,8 @@ static void kvm_vcpu_set_pv_preempted(struct kvm_vcpu *vcpu) gpa &=KVM_STEAL_PHYS_MASK ; ghc = &vcpu->arch.st.cache; -if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->generation) || gpa != ghc->gpa || +if (unlikely (!kvm_check_gen (vcpu->kvm, ghc->slots_generation, ghc->attrs_generation) || + gpa != ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot) {
[Severity: Low]
As reported by Sashiko on v2 [1], does this fail to build on LoongArch due to a missing closing parenthesis in both kvm_update_stolen_time() and kvm_vcpu_set_pv_preempted()?
Both conditions open two parentheses at if (unlikely( before calling !kvm_check_gen(vcpu->kvm, ghc->slots_generation, ghc->attrs_generation), but only close one parenthesis after !ghc->memslot before the opening brace.
[ ... ]
Quoted 4 lines
virt/kvm/kvm_main.c +0
-0
[ ... ]
Quoted 23 lines
virt/kvm/kvm_main.c +19
-0@@ -2558,6 +2559,53 @@ bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,return true; } +/* +* Returns true if _any_ gfns in the range [@start, @end) have attributes that +* match _any_ bit in @mask. +*/ +bool kvm_range_has_any_vm_memory_attributes (struct kvm *kvm,gfn_t start,gfn_t end, +unsigned long mask) +{ +XA_STATE (xas, &kvm->mem_attrs.array, start); +void *entry; + + mask &=kvm_supported_vm_mem_attributes (kvm); +if (!mask) +return false; + +if (end == start +1 ) +return !!(kvm_get_vm_memory_attributes (kvm, start) & mask); + +guard (rcu)(); +for (;;) {
virt/kvm/kvm_main.c bool kvm_range_has_vm_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end, · line 2590
Quoted 10 lines
virt/kvm/kvm_main.c +10
-0+do { + entry =xas_next (&xas); + }while (xas_retry (&xas, entry)); + +if (xas.xa_node ==XAS_BOUNDS || xas.xa_index >= end) +break ; + +if (xa_to_value (entry) & mask) +return true; + }
[Severity: High]
Can kvm_range_has_any_vm_memory_attributes() trigger an RCU CPU stall or soft lockup when userspace invokes KVM_SET_MEMORY_ATTRIBUTES over a large sparse or unassigned GFN range?
Unassigned GFNs are represented as NULL entries in kvm->mem_attrs.array, where xa_to_value(NULL) is 0, so NULL entries do not match mask and do not terminate the loop. However, the loop uses xas_next(&xas), which increments xas.xa_index by 1 on every call across NULL slots and empty xa_node subtrees whenever the XArray has an entry at an index >= end keeping xas.xa_node from reaching XAS_BOUNDS.
Because the entire loop runs inside guard(rcu)() without rescheduling, scanning a large sparse range will step through every empty GFN index one by one while holding the RCU read lock. Should this use xas_next_entry(&xas, end - 1) or xas_for_each(&xas, entry, end - 1) to skip empty slots?
Quoted 53 lines
virt/kvm/kvm_main.c +25
-2+ +return false; +} + +#define KVM_MEMORY_ATTRIBUTE_NEEDS_SYNC_MASK KVM_MEMORY_ATTRIBUTE_PROT + +static bool kvm_range_memory_attributes_need_sync (struct kvm *kvm, +gfn_t start,gfn_t end, +unsigned long attributes) +{ +u64 mask =KVM_MEMORY_ATTRIBUTE_NEEDS_SYNC_MASK ; + +if (attributes & mask) +return true; + +return kvm_range_has_any_vm_memory_attributes (kvm, start, end, mask); +} + static __always_inline void kvm_handle_gfn_range(struct kvm *kvm,struct kvm_mmu_notifier_range *range) {@@ -2648,20 +2696,25 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, .on_lock = kvm_mmu_invalidate_end, .may_block = true, }; +bool sync = false;unsigned long i;void *entry;int r =0 ; entry = attributes ?xa_mk_value (attributes) :NULL ; -trace_kvm_set_vm_mem_attributes (start, end, attributes); -mutex_lock (&kvm->slots_lock);/* Nothing to do if the entire range has the desired attributes. */ if (kvm_range_has_vm_memory_attributes (kvm, start, end, ~0 , attributes)) goto out_unlock; + sync =kvm_range_memory_attributes_need_sync (kvm, start, end, + attributes); + +trace_kvm_set_vm_mem_attributes (start, end, attributes, sync, + kvm->mem_attrs.generation +1 ); +/* * Reserve memory ahead of time to avoid having to deal with failures * partway through setting the new attributes. Storing NULL never @@ -2684,10 +2737,16 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,cond_resched (); }
virt/kvm/kvm_main.c static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, · line 2749
Quoted 10 lines
virt/kvm/kvm_main.c +6
-0+/* Pairs with acquire in kvm_mem_attributes_generation() */ +smp_store_release (&kvm->mem_attrs.generation, + kvm->mem_attrs.generation +1 ); +kvm_handle_gfn_range (kvm, &post_set_range); out_unlock:mutex_unlock (&kvm->slots_lock); +if (sync) +synchronize_srcu_expedited (&kvm->srcu);
[Severity: Medium]
Should kvm->mem_attrs.generation only be incremented when sync is true, and should synchronize_srcu_expedited() be skipped if xa_reserve() fails?
The commit message notes that for memory attributes that do not require synchronization, the attributes generation will always remain unchanged. However, kvm_set_vm_mem_attributes() unconditionally increments kvm->mem_attrs.generation via smp_store_release() even when sync is false, which invalidates all gfn_to_hva_cache and gfn_to_pfn_cache entries across the VM (and causes non-refreshing cache checks such as kvm_steal_time_set_preempted() to bail out) when toggling attributes like KVM_MEMORY_ATTRIBUTE_PRIVATE.
In addition, because sync is computed and trace_kvm_set_vm_mem_attributes() is called with kvm->mem_attrs.generation + 1 before the xa_reserve() loop:
for (i = start; entry && i < end; i++) {
r = xa_reserve(&kvm->mem_attrs.array, i, GFP_KERNEL_ACCOUNT);
if (r)
goto out_unlock;
cond_resched();
}if xa_reserve() fails and jumps to out_unlock while sync is true, kvm_set_vm_mem_attributes() still runs synchronize_srcu_expedited(&kvm->srcu) even though no attributes or generation numbers were updated.
[ ... ]
Quoted 22 lines
virt/kvm/kvm_main.c +4
-3@@ -3582,7 +3641,8 @@ int kvm_vcpu_write_guest(struct kvm_vcpu *vcpu, gpa_t gpa, const void *data, }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_write_guest); -static int __kvm_gfn_to_hva_cache_init (struct kvm_memslots *slots, +static int __kvm_gfn_to_hva_cache_init (struct kvm *kvm, +struct kvm_memslots *slots,struct gfn_to_hva_cache *ghc,gpa_t gpa,unsigned long len) {@@ -3592,8 +3652,8 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm_memslots *slots,gfn_t nr_pages_needed = end_gfn - start_gfn +1 ;gfn_t nr_pages_avail; -/* Update ghc->generation before performing any error checks. */ - ghc->generation = slots->generation; +/* Update ghc->slots_generation before performing any error checks. */ + ghc->slots_generation = slots->generation;if (start_gfn > end_gfn) { ghc->hva =KVM_HVA_ERR_BAD ;
virt/kvm/kvm_main.c static int __kvm_gfn_to_hva_cache_init(struct kvm_memslots *slots, · line 3675
Quoted 4 lines
virt/kvm/kvm_main.c +1
-0@@ -3612,6 +3672,8 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm_memslots *slots,return -EFAULT ; } + ghc->attrs_generation =kvm_mem_attributes_generation (kvm);
[Severity: Low]
Should ghc->attrs_generation be updated alongside ghc->slots_generation before the error checks in __kvm_gfn_to_hva_cache_init()?
ghc->slots_generation is updated before checking start_gfn > end_gfn or running the gfn_to_hva_many() loop so that when an invalid GPA sets ghc->hva to KVM_HVA_ERR_BAD, subsequent calls to kvm_read_guest_offset_cached() or kvm_write_guest_offset_cached() pass kvm_memslots_check_gen() and immediately hit the fast-path kvm_is_error_hva(ghc->hva) check.
Because ghc->attrs_generation is only updated after those error returns (which remains the case after the follow-up commit "KVM: Take memory protections into account for accesses with cached gfn->hva"), if a cached range fails in the memslot lookup loop after kvm->mem_attrs.generation has advanced, ghc->attrs_generation remains stale and every subsequent cached access will repeatedly fail kvm_memslots_check_gen() and re-run __kvm_gfn_to_hva_cache_init().
[1] https://lore.kernel.org/r/[email protected]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=24
kernel test robot
build botRe: [PATCH 24/45] KVM: Introduce a generation number for memory attributes
In reply to Paolo Bonzinikernel test robot noticed the following build errors:
Hi Paolo,
kernel test robot noticed the following build errors:
[auto build test ERROR on kvm/queue]
[also build test ERROR on kvm/next linus/master next-20261030] [cannot apply to mst-vhost/linux-next kvm/linux-next v7.3] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Paolo-Bonzini/KVM-nVMX-nSVM-check-unconditionally-whether-CR3-is-valid/20261102-124045 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20261102174512.8903-8-pbonzini%40redhat.com patch subject: [PATCH 24/45] KVM: Introduce a generation number for memory attributes config: i386-randconfig-1010-20261103 (https://download.01.org/0day-ci/archive/20261103/[email protected]/config) compiler: gcc-14 (Debian 14.2.0-19) 14.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261103/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from <command-line>:
include/linux/kvm_host.h: In function 'kvm_mem_attributes_generation.isra':
>> include/linux/compiler_types.h:702:45: error: call to '__compiletime_assert_523' declared with attribute error: Need native word sized stores/loads for atomicity.
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:683:25: note: in definition of macro '__compiletime_assert'
683 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:702:9: note: in expansion of macro '_compiletime_assert'
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/compiler_types.h:705:9: note: in expansion of macro 'compiletime_assert'
705 | compiletime_assert(__native_word(t), \
| ^~~~~~~~~~~~~~~~~~
arch/x86/include/asm/barrier.h:69:9: note: in expansion of macro 'compiletime_assert_atomic_type'
69 | compiletime_assert_atomic_type(*p); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
176 | #define smp_load_acquire(p) __smp_load_acquire(p)
| ^~~~~~~~~~~~~~~~~~
include/linux/kvm_host.h:2640:16: note: in expansion of macro 'smp_load_acquire'
2640 | return smp_load_acquire(&kvm->mem_attrs.generation);
| ^~~~~~~~~~~~~~~~
--
In file included from <command-line>:
include/linux/kvm_host.h: In function 'kvm_mem_attributes_generation.isra':
>> include/linux/compiler_types.h:702:45: error: call to '__compiletime_assert_523' declared with attribute error: Need native word sized stores/loads for atomicity.
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:683:25: note: in definition of macro '__compiletime_assert'
683 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:702:9: note: in expansion of macro '_compiletime_assert'
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/compiler_types.h:705:9: note: in expansion of macro 'compiletime_assert'
705 | compiletime_assert(__native_word(t), \
| ^~~~~~~~~~~~~~~~~~
arch/x86/include/asm/barrier.h:69:9: note: in expansion of macro 'compiletime_assert_atomic_type'
69 | compiletime_assert_atomic_type(*p); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/asm-generic/barrier.h:176:29: note: in expansion of macro '__smp_load_acquire'
176 | #define smp_load_acquire(p) __smp_load_acquire(p)
| ^~~~~~~~~~~~~~~~~~
include/linux/kvm_host.h:2640:16: note: in expansion of macro 'smp_load_acquire'
2640 | return smp_load_acquire(&kvm->mem_attrs.generation);
| ^~~~~~~~~~~~~~~~
In function 'kvm_vm_ioctl_set_mem_attributes',
inlined from 'kvm_vm_ioctl' at arch/x86/kvm/../../../virt/kvm/kvm_main.c:5624:7:
include/linux/compiler_types.h:702:45: error: call to '__compiletime_assert_804' declared with attribute error: BUILD_BUG_ON failed: sizeof(attrs->attributes) != sizeof(unsigned long)
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^
include/linux/compiler_types.h:683:25: note: in definition of macro '__compiletime_assert'
683 | prefix ## suffix(); \
| ^~~~~~
include/linux/compiler_types.h:702:9: note: in expansion of macro '_compiletime_assert'
702 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
| ^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:40:37: note: in expansion of macro 'compiletime_assert'
40 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
| ^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:51:9: note: in expansion of macro 'BUILD_BUG_ON_MSG'
51 | BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
| ^~~~~~~~~~~~~~~~
arch/x86/kvm/../../../virt/kvm/kvm_main.c:2795:9: note: in expansion of macro 'BUILD_BUG_ON'
2795 | BUILD_BUG_ON(sizeof(attrs->attributes) != sizeof(unsigned long));
| ^~~~~~~~~~~~vim +/__compiletime_assert_523 +702 include/linux/compiler_types.h
eb5c2d4b45e3d2d Will Deacon 2020-07-21 688 eb5c2d4b45e3d2d Will Deacon 2020-07-21 689 #define _compiletime_assert(condition, msg, prefix, suffix) \ eb5c2d4b45e3d2d Will Deacon 2020-07-21 690 __compiletime_assert(condition, msg, prefix, suffix) eb5c2d4b45e3d2d Will Deacon 2020-07-21 691 eb5c2d4b45e3d2d Will Deacon 2020-07-21 692 /** eb5c2d4b45e3d2d Will Deacon 2020-07-21 693 * compiletime_assert - break build and emit msg if condition is false eb5c2d4b45e3d2d Will Deacon 2020-07-21 694 * @condition: a compile-time constant condition to check eb5c2d4b45e3d2d Will Deacon 2020-07-21 695 * @msg: a message to emit if condition is false eb5c2d4b45e3d2d Will Deacon 2020-07-21 696 * eb5c2d4b45e3d2d Will Deacon 2020-07-21 697 * In tradition of POSIX assert, this macro will break the build if the eb5c2d4b45e3d2d Will Deacon 2020-07-21 698 * supplied condition is *false*, emitting the supplied error message if the eb5c2d4b45e3d2d Will Deacon 2020-07-21 699 * compiler has support to do so. eb5c2d4b45e3d2d Will Deacon 2020-07-21 700 */ eb5c2d4b45e3d2d Will Deacon 2020-07-21 701 #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2d Will Deacon 2020-07-21 @702 _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)eb5c2d4b45e3d2d Will Deacon 2020-07-21 703
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Paolo Bonzini
maintainerPATCH v4 25/45 KVM: Take memory protections into account for accesses with cached gfn->hva
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Account for memory attributes when accessing guest memory through kvm_get/put_guest().
This requires tracking the memory attributes generation as part of gfn_to_hva_cache's data, invalidate the cached information if the generation changes, and failing to refresh the cache if restrictive memory attributes are found within the GPA range.
Similar to how gfn_to_hva_cache disallows caching gfns mapped within read-only memory slots, gfns marked as read-only by memory attributes will also fail to initialize. Unsurprisingly, the same behaviour applies to gfns mapped as non-accessible (NR/NW), while non-executable mappings are okay.
--- include/linux/kvm_host.h | 19 +++++++++++++++++-- virt/kvm/kvm_main.c | 15 +++++++++++---- 2 files changed, 28 insertions(+), 6 deletions(-)
include/linux/kvm_host.h +17 -2
@@ -1371,7 +1371,8 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, typeof(v) __user *__uaddr = (typeof(__uaddr))(__addr + offset); \int __ret = -EFAULT ; \ \ -if (!kvm_is_error_hva (__addr)) \ +if (!kvm_is_error_hva (__addr) && \ +kvm_mem_attributes_may_read_gfn (kvm, gfn)) \ __ret =get_user (v, __uaddr); \ __ret; \ })
@@ -1391,7 +1392,8 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, typeof(v) __user *__uaddr = (typeof(__uaddr))(__addr + offset); \int __ret = -EFAULT ; \ \ -if (!kvm_is_error_hva (__addr)) \ +if (!kvm_is_error_hva (__addr) && \ +kvm_mem_attributes_may_write_gfn (kvm, gfn)) \ __ret =put_user (v, __uaddr); \if (!__ret) \mark_page_dirty (kvm, gfn); \
@@ -2640,6 +2642,12 @@ static inline u64 kvm_mem_attributes_generation(struct kvm *kvm) return smp_load_acquire(&kvm->mem_attrs.generation); }#else +static inline bool kvm_range_has_any_vm_memory_attributes (struct kvm *kvm, +gfn_t start,gfn_t end,unsigned long mask) +{ +return false; +} +static inline u64 kvm_supported_vm_mem_attributes (struct kvm *kvm) {return 0 ;
@@ -2698,6 +2706,13 @@ static inline int kvm_mem_attributes_may_write_gfn(struct kvm *kvm, gfn_t gfn)return kvm_mem_attributes_may_write (attrs); } +static inline bool kvm_range_has_rw_memory_protections (struct kvm *kvm, +gfn_t start,gfn_t end) +{ +return kvm_range_has_any_vm_memory_attributes (kvm, start, end, +KVM_MEMORY_ATTRIBUTE_NR |KVM_MEMORY_ATTRIBUTE_NW ); +} +static inline bool kvm_memslots_check_gen (struct kvm *kvm,u64 slots_generation,u64 attrs_generation,struct kvm_memslots **p_slots) {
virt/kvm/kvm_main.c +11 -4
@@ -3664,15 +3664,22 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm *kvm,* If the requested region crosses two memslots, we still * verify that the entire region is valid here. */ -for ( ; start_gfn <= end_gfn; start_gfn += nr_pages_avail) { - ghc->memslot =__gfn_to_memslot (slots, start_gfn); - ghc->hva =gfn_to_hva_many (ghc->memslot, start_gfn, - &nr_pages_avail); +for (gfn_t gfn = start_gfn ; gfn <= end_gfn; gfn += nr_pages_avail) { + ghc->memslot =__gfn_to_memslot (slots, gfn); + ghc->hva =gfn_to_hva_many (ghc->memslot, gfn, &nr_pages_avail);if (kvm_is_error_hva (ghc->hva))return -EFAULT ; } +/* +* RW memory attributes are incompatible with GHC. The RW protection +* check has to happen after storing the generation number. +*/ ghc->attrs_generation =kvm_mem_attributes_generation (kvm); +if (kvm_range_has_rw_memory_protections (kvm, start_gfn, end_gfn +1 )) { + ghc->hva =KVM_HVA_ERR_BAD ; +return -EFAULT ; + }/* Use the slow path for cross page reads and writes. */ if (nr_pages_needed ==1 )
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 25/45] KVM: Take memory protections into account for accesses with cached gfn->hva
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] KVM: redundant mem_attr checks and RO failure in __kvm_get/put_guest() - [Medium] KVM: unupdated…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] KVM: redundant mem_attr checks and RO failure in __kvm_get/put_guest()
- [Medium] KVM: unupdated attrs_generation on error in __kvm_gfn_to_hva_cache_init()
- [Medium] KVM: stale ghc->gpa on error in __kvm_gfn_to_hva_cache_init()
--
commit 70e5ce5ddda9132daea8ebdb4ea3879b1bca2fa1
Author: Nicolas Saenz Julienne <[email protected]>
KVM: Take memory protections into account for accesses with cached gfn->hva
This commit adds memory attribute protection checks to __kvm_get_guest(), __kvm_put_guest(), and __kvm_gfn_to_hva_cache_init(), invalidating and rejecting gfn_to_hva_cache entries when read or write restrictions apply to the GPA range.
Quoted 15 lines
include/linux/kvm_host.h +2
-1@@ -1371,7 +1371,8 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, typeof(v) __user *__uaddr = (typeof(__uaddr))(__addr + offset); \int __ret = -EFAULT ; \ \ -if (!kvm_is_error_hva (__addr)) \ +if (!kvm_is_error_hva (__addr) && \ +kvm_mem_attributes_may_read_gfn (kvm, gfn)) \ __ret =get_user (v, __uaddr); \ __ret; \ })
include/linux/kvm_host.h int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, · line 1397
Quoted 7 lines
include/linux/kvm_host.h +2
-1@@ -1391,7 +1392,8 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, typeof(v) __user *__uaddr = (typeof(__uaddr))(__addr + offset); \int __ret = -EFAULT ; \ \ -if (!kvm_is_error_hva (__addr)) \ +if (!kvm_is_error_hva (__addr) && \ +kvm_mem_attributes_may_write_gfn (kvm, gfn)) \ __ret =put_user (v, __uaddr); \
[Severity: Medium]
Both __kvm_get_guest() and __kvm_put_guest() obtain __addr via gfn_to_hva(kvm, gfn), which already checks both read and write memory attributes:
virt/kvm/kvm_main.c:gfn_to_hva() {
addr = gfn_to_hva_many(gfn_to_memslot(kvm, gfn), gfn, NULL);
if (kvm_is_error_hva(addr))
return addr;
if (!kvm_mem_attributes_may_read_gfn(kvm, gfn))
return KVM_HVA_ERR_BAD;
if (!kvm_mem_attributes_may_write_gfn(kvm, gfn))
return KVM_HVA_ERR_RO_BAD;
return addr;
}Because both KVM_HVA_ERR_BAD and KVM_HVA_ERR_RO_BAD satisfy kvm_is_error_hva(__addr), the additional calls to kvm_mem_attributes_may_read_gfn() and kvm_mem_attributes_may_write_gfn() in these macros perform a duplicate XArray lookup in kvm->mem_attrs.array and evaluate the macro arguments kvm and gfn a second time.
Furthermore, because __kvm_get_guest() calls gfn_to_hva() rather than gfn_to_hva_prot(), any page marked read-only with KVM_MEMORY_ATTRIBUTE_NW causes gfn_to_hva() to return KVM_HVA_ERR_RO_BAD, failing __kvm_get_guest() with -EFAULT before kvm_mem_attributes_may_read_gfn() is reached.
Should __kvm_get_guest() use gfn_to_hva_prot(kvm, gfn, NULL) so that read-only pages remain readable, and drop the redundant attribute check in __kvm_put_guest()? Also, the commit message mentions that tracking the memory attributes generation in gfn_to_hva_cache is required for kvm_get/put_guest(), though kvm_get_guest() and kvm_put_guest() do not use gfn_to_hva_cache.
[ ... ]
Quoted 14 lines
virt/kvm/kvm_main.c +2
-4@@ -3664,15 +3664,22 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm *kvm,* If the requested region crosses two memslots, we still * verify that the entire region is valid here. */ -for ( ; start_gfn <= end_gfn; start_gfn += nr_pages_avail) { - ghc->memslot =__gfn_to_memslot (slots, start_gfn); - ghc->hva =gfn_to_hva_many (ghc->memslot, start_gfn, - &nr_pages_avail); +for (gfn_t gfn = start_gfn ; gfn <= end_gfn; gfn += nr_pages_avail) { + ghc->memslot =__gfn_to_memslot (slots, gfn);
virt/kvm/kvm_main.c static int __kvm_gfn_to_hva_cache_init(struct kvm *kvm, · line 3678
Quoted 10 lines
virt/kvm/kvm_main.c +5
-0+ ghc->hva =gfn_to_hva_many (ghc->memslot, gfn, &nr_pages_avail);if (kvm_is_error_hva (ghc->hva))return -EFAULT ; } +/* +* RW memory attributes are incompatible with GHC. The RW protection +* check has to happen after storing the generation number. +*/ ghc->attrs_generation =kvm_mem_attributes_generation (kvm);
[Severity: Medium]
At the start of __kvm_gfn_to_hva_cache_init(), ghc->slots_generation is updated before any error checks so that if initialization fails, subsequent calls to kvm_read_guest_offset_cached() or kvm_write_guest_offset_cached() within the same generation pass kvm_memslots_check_gen() and immediately return -EFAULT on kvm_is_error_hva(ghc->hva).
Because ghc->attrs_generation is only updated after the start_gfn > end_gfn check and the memslot lookup loop, if kvm->mem_attrs.generation changes on a cache whose GPA fails one of those earlier checks, ghc->attrs_generation remains stale. Every subsequent call to kvm_read_guest_offset_cached() or kvm_write_guest_offset_cached() will then fail kvm_memslots_check_gen() and repeatedly re-run __kvm_gfn_to_hva_cache_init().
Should ghc->attrs_generation = kvm_mem_attributes_generation(kvm) be moved to the top of __kvm_gfn_to_hva_cache_init() alongside ghc->slots_generation?
virt/kvm/kvm_main.c static int __kvm_gfn_to_hva_cache_init(struct kvm *kvm, · line 3682
Quoted 4 lines
virt/kvm/kvm_main.c +4
-0+if (kvm_range_has_rw_memory_protections (kvm, start_gfn, end_gfn +1 )) { + ghc->hva =KVM_HVA_ERR_BAD ; +return -EFAULT ; + }
[Severity: Medium]
When a gfn_to_hva_cache previously initialized to a valid gpa1 is re-initialized to a new gpa2 that fails kvm_range_has_rw_memory_protections(), __kvm_gfn_to_hva_cache_init() updates ghc->slots_generation and ghc->attrs_generation and sets ghc->hva = KVM_HVA_ERR_BAD, but returns -EFAULT before updating ghc->gpa and ghc->len at the end of the function:
virt/kvm/kvm_main.c:__kvm_gfn_to_hva_cache_init() {
...
ghc->gpa = gpa;
ghc->len = len;
return 0;
}This leaves ghc->hva poisoned with KVM_HVA_ERR_BAD while ghc->gpa still holds the previous valid gpa1. Callers that guard cache re-initialization with a GPA comparison, such as nested_vmx_check_vmcs_link_ptr(), nested_cache_shadow_vmcs12(), and nested_flush_cached_shadow_vmcs12():
arch/x86/kvm/vmx/nested.c:nested_vmx_check_vmcs_link_ptr() {
if (ghc->gpa != vmcs12->vmcs_link_pointer &&
CC(kvm_gfn_to_hva_cache_init(vcpu->kvm, ghc,
vmcs12->vmcs_link_pointer, VMCS12_SIZE)))
return -EINVAL;
}or callers that keep the previous GPA active when an MSR update to a new GPA is rejected (such as kvm_lapic_set_pv_eoi() and kvm_pv_enable_async_pf()), will then see matching generations and fail valid accesses to gpa1 with -EFAULT.
Would it make sense to update ghc->gpa and ghc->len at the start of __kvm_gfn_to_hva_cache_init() before the error returns so that a failed re-initialization does not leave a stale ghc->gpa paired with KVM_HVA_ERR_BAD?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=25
Paolo Bonzini
maintainerPATCH v4 26/45 KVM: pfncache: Fail to refresh if it contains memory protections
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Similar to how pfncache disallows caching gfns mapped within read-only memory slots, gfns marked as read-only by memory attributes must fail to initialize. Unsurprisingly, the same behaviour applies to gfns mapped as non-accessible (NR/NW), while non-executable mappings are fine.
--- virt/kvm/pfncache.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
virt/kvm/pfncache.c +11 -0
@@ -298,7 +298,18 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l gpc->gpa != gpa || kvm_is_error_hva(gpc->uhva)) {gfn_t gfn =gpa_to_gfn (gpa); +/* +* RW memory attributes are incompatible with GPC. The +* RW protection check has to happen after storing the +* generation number. +*/ gpc->attrs_generation =kvm_mem_attributes_generation (kvm); +if (kvm_range_has_rw_memory_protections (kvm, gfn, gfn +1 )) { + gpc->uhva =KVM_HVA_ERR_BAD ; + ret = -EFAULT ; + goto out; + } + gpc->gpa = gpa; gpc->slots_generation = slots->generation; gpc->memslot =__gfn_to_memslot (slots, gfn);
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 26/45] KVM: pfncache: Fail to refresh if it contains memory protections
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] KVM: pfncache: __kvm_gpc_refresh() skips updating gpc->gpa on RW protection error --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] KVM: pfncache: __kvm_gpc_refresh() skips updating gpc->gpa on RW protection error --
commit 808264658bd3c3bcb182dcf95cb531c1ff5d0f1b
Author: Nicolas Saenz Julienne <[email protected]>
KVM: pfncache: Fail to refresh if it contains memory protections
This commit updates __kvm_gpc_refresh() in virt/kvm/pfncache.c to reject refreshing GPA-based caches when the target GFN has NR or NW memory protection attributes.
Quoted 13 lines
virt/kvm/pfncache.c +5
-0@@ -298,7 +298,18 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l gpc->gpa != gpa || kvm_is_error_hva(gpc->uhva)) {gfn_t gfn =gpa_to_gfn (gpa); +/* +* RW memory attributes are incompatible with GPC. The +* RW protection check has to happen after storing the +* generation number. +*/
virt/kvm/pfncache.c static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l · line 315
Quoted 10 lines
virt/kvm/pfncache.c +6
-0gpc->attrs_generation =kvm_mem_attributes_generation (kvm); +if (kvm_range_has_rw_memory_protections (kvm, gfn, gfn +1 )) { + gpc->uhva =KVM_HVA_ERR_BAD ; + ret = -EFAULT ; + goto out; + } + gpc->gpa = gpa; gpc->slots_generation = slots->generation; gpc->memslot =__gfn_to_memslot (slots, gfn);
[Severity: High]
Because this error path jumps to out before setting gpc->gpa = gpa, what happens when kvm_gpc_activate() is called on a GPA backed by NR or NW memory protections?
In virt/kvm/pfncache.c, __kvm_gpc_activate() marks the cache active before calling __kvm_gpc_refresh():
virt/kvm/pfncache.c:__kvm_gpc_activate() {
if (!gpc->active) {
...
write_lock_irq(&gpc->lock);
gpc->active = true;
write_unlock_irq(&gpc->lock);
}
return __kvm_gpc_refresh(gpc, gpa, uhva);
}Unlike the existing gfn_to_hva_memslot() failure path immediately below, jumping to out on kvm_range_has_rw_memory_protections() leaves gpc->gpa unchanged while setting gpc->uhva = KVM_HVA_ERR_BAD:
1. If the cache was freshly initialized or previously HVA-active (so gpc->gpa == INVALID_GPA), it is left with gpc->active == true, gpc->gpa == INVALID_GPA, and gpc->uhva == KVM_HVA_ERR_BAD. On a subsequent kvm_gpc_refresh() call, kvm_is_error_gpa(gpc->gpa) is true, so kvm_gpc_refresh() passes both gpa == INVALID_GPA and uhva == KVM_HVA_ERR_BAD into __kvm_gpc_refresh():
virt/kvm/pfncache.c:kvm_gpc_refresh() {
...
uhva = kvm_is_error_gpa(gpc->gpa) ? gpc->uhva : KVM_HVA_ERR_BAD;
return __kvm_gpc_refresh(gpc, gpc->gpa, uhva);
}virt/kvm/pfncache.c:__kvm_gpc_refresh() {
/* Either gpa or uhva must be valid, but not both */
if (WARN_ON_ONCE(kvm_is_error_gpa(gpa) == kvm_is_error_hva(uhva)))
return -EINVAL;
}This allows a guest (for example, by writing MSR_KVM_SYSTEM_TIME_NEW with a protected GPA) to trigger the host WARN_ON_ONCE(), and prevents kvm_gpc_refresh() from ever recovering even if userspace later clears the memory protections.
2. In that same state (gpc->active == true and gpc->gpa == INVALID_GPA), kvm_gpc_is_hva_active() evaluates to true, which causes kvm_xen_hvm_get_attr() and kvm_xen_vcpu_get_attr() in arch/x86/kvm/xen.c to copy gpc->uhva (KVM_HVA_ERR_BAD, i.e. PAGE_OFFSET) to userspace:
arch/x86/kvm/xen.c:kvm_xen_vcpu_get_attr() {
case KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO_HVA:
if (kvm_gpc_is_hva_active(&vcpu->arch.xen.vcpu_info_cache))
data->u.hva = vcpu->arch.xen.vcpu_info_cache.uhva;
}3. If the cache was already active at a valid writable GPA and is re-activated via kvm_gpc_activate() with a new GPA that has NR or NW memory protections, gpc->valid is cleared and gpc->uhva is set to KVM_HVA_ERR_BAD, but gpc->gpa remains set to the old GPA. The next kvm_gpc_refresh() call will then re-translate the stale gpc->gpa and silently resume writing to the old guest physical address.
Should gpc->gpa (along with gpc->slots_generation and gpc->memslot) be updated before checking kvm_range_has_rw_memory_protections(), matching the gfn_to_hva_memslot() error path?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=26
Paolo Bonzini
maintainerPATCH v4 27/45 KVM: x86/mmu: Do not prefetch sptes on gfns backed by memory attributes
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Prefetched SPTEs are always given full access. Do not prefetch GFNs that have memory protections applied.
--- arch/x86/kvm/mmu/mmu.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/kvm/mmu/mmu.c +5 -0
@@ -3240,6 +3240,11 @@ static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep,return false;for (i =0 ; i < nr_pages; i++, gfn++, sptep++) { +if (kvm_get_vm_memory_attributes (vcpu->kvm, gfn)) { +kvm_release_page_clean (pages[i]); +continue ; + } +mmu_set_spte (vcpu, slot, sptep, access, gfn,page_to_pfn (pages[i]),NULL );
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 28/45 KVM: x86/mmu: Take memory protection attributes into account during faults
In reply to Paolo BonziniFrom: Nicolas Saenz Julienne <[email protected]>
Take memory protection attributes when faulting guest memory. Prohibited memory accesses will cause a user-space -EFAULT exit just like private memory accesses. Userspace will either enable the access or bump it to the guest as some kind of exception (e.g. a VTL return).
Since the struct kvm_page_fault already has the access type in PFERR_* format, the check is done via the kvm_page_format permissions table. This means that it supports naturally all page table format variants, and it can even handle mode-based memory protection when the host uses MBEC/GMET. The only thing that needs some care is to build the restricted ACC_* mask with the root page's own access mask as a base (and not ACC_ALL). Otherwise, supervisor mode execution would be handled incorrectly on AMD processors with GMET.
To avoid spamming the trace buffer too much, the new trace event only kicks in if memory protection attributes are present for the faulted gfn.
--- arch/x86/kvm/mmu/mmu.c | 49 +++++++++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu_internal.h | 19 +++++++++++++ arch/x86/kvm/mmu/mmutrace.h | 36 ++++++++++++++++++++++++ arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.h | 11 ++------ 5 files changed, 107 insertions(+), 10 deletions(-)
arch/x86/kvm/mmu/mmu.c +49 -0
@@ -4701,6 +4701,50 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,return RET_PF_CONTINUE ; } +static inline unsigned kvm_get_gfn_protections (struct kvm_vcpu *vcpu,gfn_t gfn) +{ +struct kvm *kvm = vcpu->kvm; +unsigned int access = vcpu->arch.mmu->root_role.access; +unsigned long attrs =kvm_get_vm_memory_attributes (kvm, gfn); +if (!attrs) +return access; + +WARN_ON_ONCE (!kvm_vm_mem_attributes_valid (kvm, attrs)); + +if (!kvm_mem_attributes_may_read (attrs)) + access &= ~ACC_READ_MASK ; +if (!kvm_mem_attributes_may_write (attrs)) + access &= ~ACC_WRITE_MASK ; +if (!kvm_mem_attributes_may_exec (attrs)) { + access &= ~ACC_EXEC_MASK ; +if (shadow_xu_mask) + access &= ~ACC_USER_EXEC_MASK ; + } + +return access; +} + +static int kvm_faultin_memory_protections (struct kvm_vcpu *vcpu, +struct kvm_page_fault *fault) +{ +unsigned access; + +/* Memory attributes don't apply to MMIO regions */ +if (unlikely (!fault->slot)) +return RET_PF_CONTINUE ; + + access =kvm_get_gfn_protections (vcpu, fault->gfn); +if (access ==ACC_ALL ) +return RET_PF_CONTINUE ; + +trace_kvm_faultin_memory_protections (vcpu, fault, access); +if (__permission_fault (vcpu->arch.mmu, access, fault)) +return -EFAULT ; + + fault->host_access &= access; +return RET_PF_CONTINUE ; +} +static int __kvm_mmu_faultin_pfn (struct kvm_vcpu *vcpu,struct kvm_page_fault *fault) {
@@ -4781,6 +4825,11 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu,if (unlikely (!slot))return kvm_handle_noslot_fault (vcpu, fault, access); +if (kvm_faultin_memory_protections (vcpu, fault)) { +kvm_mmu_prepare_memory_fault_exit (vcpu, fault); +return -EFAULT ; + } +/* * Retry the page fault if the gfn hit a memslot that is being deleted * or moved. This ensures any existing SPTEs for the old memslot will
arch/x86/kvm/mmu/mmu_internal.h +19 -0
@@ -290,6 +290,25 @@ struct kvm_page_fault {bool write_fault_to_shadow_pgtable; }; +/* +* Returns true if the access indicated by @fault is forbidden by the existing +* SPTE protections. +*/ +static inline bool __permission_fault (struct kvm_mmu *mmu,unsigned access, +struct kvm_page_fault *fault) +{ +unsigned pfec; + +/* +* RSVD is handled elsewhere, and is used for SMAP in the context +* of accessing fmt.permissions[]. SPTEs never use PK or SS, as +* they are not supported for shadow paging and irrelevant for TDP. +*/ + pfec = fault->error_code & ( +PFERR_WRITE_MASK |PFERR_USER_MASK |PFERR_FETCH_MASK ); +return (mmu->fmt.permissions[pfec >>1 ] >> access) &1 ; +} +/* * Return values of handle_mmio_page_fault(), mmu.page_fault(), fast_page_fault(), * and of course kvm_mmu_do_page_fault().
arch/x86/kvm/mmu/mmutrace.h +36 -0
@@ -8,6 +8,13 @@#undef TRACE_SYSTEM#define TRACE_SYSTEM kvmmmu +#ifdef CREATE_TRACE_POINTS +#define tracing_kvm_rip_read (vcpu) ({ \ + typeof(vcpu) __vcpu = vcpu; \ + __vcpu->arch.guest_state_protected ? 0 : kvm_rip_read(__vcpu); \ + }) +#endif +#define KVM_MMU_PAGE_FIELDS \ __field(__u8, mmu_valid_gen) \ __field(__u64, gfn) \
@@ -447,6 +454,35 @@ TRACE_EVENT( __entry->gfn, __entry->spte, __entry->level, __entry->errno) ); +TRACE_EVENT (kvm_faultin_memory_protections, + TP_PROTO(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, + unsigned access), +TP_ARGS (vcpu, fault, access), + +TP_STRUCT__entry ( + __field(unsigned int, vcpu_id) + __field(unsigned long, guest_rip) + __field(u64, fault_address) + __field(bool, write) + __field(bool, exec) +__field (unsigned, access) + ), + +TP_fast_assign ( + __entry->vcpu_id = vcpu->vcpu_id; + __entry->guest_rip = tracing_kvm_rip_read(vcpu); + __entry->fault_address = fault->gfn; + __entry->write = fault->write; + __entry->exec = fault->exec; + __entry->access = access; + ), + +TP_printk ("vcpu %d rip 0x%lx gfn 0x%016llx access %s protections 0x%x" , + __entry->vcpu_id, __entry->guest_rip, __entry->fault_address, + __entry->exec ?"X" : (__entry->write ?"W" :"R" ), + __entry->access) +); +#endif /* _TRACE_KVMMMU_H */ #undef TRACE_INCLUDE_PATH
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
@@ -995,7 +995,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int sptep = &sp->spt[i]; spte = *sptep; - host_access =ACC_ALL ; + host_access =kvm_get_gfn_protections (vcpu, gfn);if (!(spte & shadow_host_writable_mask)) host_access &= ~ACC_WRITE_MASK ; slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn);
arch/x86/kvm/mmu/spte.h +2 -9
@@ -491,7 +491,7 @@ static inline bool is_mmu_writable_spte(u64 spte)static inline bool spte_permission_fault (struct kvm_mmu *mmu,u64 spte,struct kvm_page_fault *fault) { -unsigned pfec, pte_access; +unsigned pte_access;if (!is_shadow_present_pte (spte))return true;
@@ -511,14 +511,7 @@ static inline bool spte_permission_fault(struct kvm_mmu *mmu, u64 spte, pte_access |= spte & shadow_xu_mask ?ACC_USER_EXEC_MASK :0 ; } -/* -* RSVD is handled elsewhere, and is used for SMAP in the context -* of accessing fmt.permissions[]. SPTEs never use PK or SS, as -* they are not supported for shadow paging and irrelevant for TDP. -*/ - pfec = fault->error_code & ( -PFERR_WRITE_MASK |PFERR_USER_MASK |PFERR_FETCH_MASK ); -return (mmu->fmt.permissions[pfec >>1 ] >> pte_access) &1 ; +return __permission_fault (mmu, pte_access, fault); }/*
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 28/45] KVM: x86/mmu: Take memory protection attributes into account during faults
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] KVM: x86/mmu: make_small_spte() re-enables exec on KVM_MEMORY_ATTRIBUTE_NX - [High] KVM: x86/mmu…
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] KVM: x86/mmu: make_small_spte() re-enables exec on KVM_MEMORY_ATTRIBUTE_NX
- [High] KVM: x86/mmu: spurious -EFAULT in kvm_faultin_memory_protections() with SMEP
- [Medium] KVM: x86/mmu: broken ACC_ALL check in kvm_faultin_memory_protections()
- [Medium] KVM: x86/mmu: FNAME(sync_spte) installs 0-permission EPT SPTE on NX GFN --
commit ed18ad0e63078d1672b15a00740ac1eefc27f904
Author: Nicolas Saenz Julienne <[email protected]>
KVM: x86/mmu: Take memory protection attributes into account during faults
This commit checks per-gfn memory protection attributes during guest page faults and SPTE synchronization, restricting SPTE permissions or exiting to userspace with -EFAULT when an access is prohibited. It also adds a trace event when faulting in a gfn with restricted memory protections.
Quoted 33 lines
arch/x86/kvm/mmu/mmu.c +25
-0@@ -4705,6 +4705,50 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,return RET_PF_CONTINUE ; } +static inline unsigned kvm_get_gfn_protections (struct kvm_vcpu *vcpu,gfn_t gfn) +{ +struct kvm *kvm = vcpu->kvm; +unsigned int access = vcpu->arch.mmu->root_role.access; +unsigned long attrs =kvm_get_vm_memory_attributes (kvm, gfn); +if (!attrs) +return access; + +WARN_ON_ONCE (!kvm_vm_mem_attributes_valid (kvm, attrs)); + +if (!kvm_mem_attributes_may_read (attrs)) + access &= ~ACC_READ_MASK ; +if (!kvm_mem_attributes_may_write (attrs)) + access &= ~ACC_WRITE_MASK ; +if (!kvm_mem_attributes_may_exec (attrs)) { + access &= ~ACC_EXEC_MASK ; +if (shadow_xu_mask) + access &= ~ACC_USER_EXEC_MASK ; + } + +return access; +} + +static int kvm_faultin_memory_protections (struct kvm_vcpu *vcpu, +struct kvm_page_fault *fault)
arch/x86/kvm/mmu/mmu.c static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu, · line 4742
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +10
-0+{ +unsigned access; + +/* Memory attributes don't apply to MMIO regions */ +if (unlikely (!fault->slot)) +return RET_PF_CONTINUE ; + + access =kvm_get_gfn_protections (vcpu, fault->gfn); +if (access ==ACC_ALL ) +return RET_PF_CONTINUE ;
[Severity: Medium]
On AMD processors with GMET enabled, does this fast-path check in arch/x86/kvm/mmu/mmu.c:kvm_faultin_memory_protections() ever succeed when no memory protection attributes are present?
In arch/x86/kvm/mmu/mmu.c:kvm_calc_tdp_mmu_root_page_role(), root_role.access has ACC_USER_MASK cleared when GMET is active:
/* All TDP pages are supervisor-executable */ role.access = ACC_ALL; if (role.cr4_smep && shadow_user_mask) role.access &= ~ACC_USER_MASK;
When attrs is 0, kvm_get_gfn_protections() returns root_role.access (ACC_ALL & ~ACC_USER_MASK), so comparing access against ACC_ALL in kvm_faultin_memory_protections() evaluates to false on every TDP page fault and unconditionally triggers trace_kvm_faultin_memory_protections() and __permission_fault(). Should this compare access against vcpu->arch.mmu->root_role.access instead?
arch/x86/kvm/mmu/mmu.c static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu, · line 4742
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c +4
-0+ +trace_kvm_faultin_memory_protections (vcpu, fault, access); +if (__permission_fault (vcpu->arch.mmu, access, fault)) +return -EFAULT ;
[Severity: High]
As reported by Sashiko on v2 [1], in arch/x86/kvm/mmu/mmu.c:kvm_faultin_memory_protections(), can passing access derived from vcpu->arch.mmu->root_role.access into __permission_fault() cause spurious -EFAULT exits on valid supervisor instruction fetches when SMEP or nested GMET is enabled?
In arch/x86/kvm/mmu/mmu.c:init_kvm_shadow_mmu() (when the guest enables cr4.smep) and arch/x86/kvm/mmu/mmu.c:kvm_init_shadow_npt_mmu() (when L1 enables GMET), root_role.cr4_smep is set to 1 while root_role.access remains ACC_ALL (which includes ACC_USER_MASK):
cpu_role.base.cr4_smep = (misc_ctl & SVM_MISC_ENABLE_GMET) != 0; init_kvm_page_walk(vcpu, &vcpu->arch.ngpa_walk, cpu_role); root_role = cpu_role.base;
When arch/x86/kvm/mmu/mmu.c:__update_permission_bitmask() builds mmu->fmt.permissions for non-EPT MMUs with cr4_smep enabled, any access mask with ACC_USER_MASK set is marked as faulting on supervisor instruction fetches:
/* Disallow supervisor fetches of user code if cr4.smep */ if (cr4_smep) ff |= (pfec & PFERR_FETCH_MASK) ? kf : 0;
If a guest under shadow paging or nested shadow NPT performs a supervisor instruction fetch from a supervisor page (guest PTE U=0) whose GFN has KVM_MEMORY_ATTRIBUTE_NW set, kvm_get_gfn_protections() returns ACC_READ_MASK | ACC_EXEC_MASK | ACC_USER_MASK. Doesn't __permission_fault() then treat the access as a SMEP violation because ACC_USER_MASK is still set in access, returning a spurious -EFAULT to userspace?
arch/x86/kvm/mmu/mmu.c static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu, · line 4746
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c +4
-0+ + fault->host_access &= access; +return RET_PF_CONTINUE ; +}
[Severity: High]
When a huge page is mapped on a GFN range with uniform KVM_MEMORY_ATTRIBUTE_NX and later split into 4K pages, will arch/x86/kvm/mmu/spte.c:make_small_spte() re-enable execute permissions on the resulting child SPTEs?
During a read or write fault, kvm_faultin_memory_protections() clears ACC_EXEC_MASK in fault->host_access, and make_spte() installs a non-executable huge SPTE. However, sp->role.access retains ACC_EXEC_MASK.
When the huge page is later split via arch/x86/kvm/mmu/tdp_mmu.c:tdp_mmu_split_huge_page() or arch/x86/kvm/mmu/mmu.c:shadow_mmu_split_huge_page(), make_small_spte() restores execute permissions from role.access whenever the NX huge page mitigation is enabled:
if (is_nx_huge_page_enabled(kvm)) child_spte = change_spte_executable(child_spte, role.access);
Doesn't this make the split 4K SPTEs executable in hardware despite KVM_MEMORY_ATTRIBUTE_NX being set on the underlying GFNs?
[ ... ]
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int · line 1000
Quoted 13 lines
arch/x86/kvm/mmu/paging_tmpl.h +1
-1@@ -994,7 +994,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int sptep = &sp->spt[i]; spte = *sptep; - host_access =ACC_ALL ; + host_access =kvm_get_gfn_protections (vcpu, gfn);if (!(spte & shadow_host_writable_mask)) host_access &= ~ACC_WRITE_MASK ; slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn);
[Severity: Medium]
In arch/x86/kvm/mmu/paging_tmpl.h:FNAME(sync_spte), what happens on nested EPT with execute-only support when L1 updates an unsync EPT12 entry to be execute-only (pte_access = ACC_EXEC_MASK | ACC_USER_EXEC_MASK) on a GFN that has KVM_MEMORY_ATTRIBUTE_NX set?
Earlier in FNAME(sync_spte), the check for an effective non-present SPTE runs before host_access is applied:
if ((pte_access | shadow_present_mask) == SHADOW_NONPRESENT_VALUE ||
gfn != kvm_mmu_page_get_gfn(sp, i)) {
drop_spte(vcpu->kvm, &sp->spt[i]);
return 1;
}Because pte_access still has execute bits set at that point, the SPTE is not dropped. When make_spte() then masks pte_access with host_access:
WARN_ON_ONCE((pte_access | shadow_present_mask) == SHADOW_NONPRESENT_VALUE); ... pte_access &= host_access;
all remaining permission bits in pte_access are cleared, leaving an EPT SPTE with SPTE_MMU_PRESENT_MASK set but hardware bits 2:0 and 10 all zero. Should the SHADOW_NONPRESENT_VALUE check in FNAME(sync_spte) account for the protections from kvm_get_gfn_protections() so the SPTE is dropped instead?
[1] https://lore.kernel.org/r/[email protected]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=28
Paolo Bonzini
maintainerPATCH v4 29/45 KVM: x86/emulate: change return value of emulator_io_permitted to int
In reply to Paolo BonziniAllow emulator_io_permitted to distinguish memory attribute faults from disallowed I/O ports.
--- arch/x86/kvm/emulate.c | 50 +++++++++++++++++++++++++----------------- 1 file changed, 30 insertions(+), 20 deletions(-)
arch/x86/kvm/emulate.c +30 -20
@@ -2526,8 +2526,8 @@ static bool emulator_bad_iopl(struct x86_emulate_ctxt *ctxt)#define VMWARE_PORT_VMPORT (0x5658)#define VMWARE_PORT_VMRPC (0x5659) -static bool emulator_io_port_access_allowed (struct x86_emulate_ctxt *ctxt, -u16 port,u16 len) +static int emulator_io_port_access_allowed (struct x86_emulate_ctxt *ctxt, +u16 port,u16 len) {const struct x86_emulate_ops *ops = ctxt->ops;struct desc_struct tr_seg;
@@ -2543,43 +2543,47 @@ static bool emulator_io_port_access_allowed(struct x86_emulate_ctxt *ctxt,*/ if (enable_vmware_backdoor && ((port ==VMWARE_PORT_VMPORT ) || (port ==VMWARE_PORT_VMRPC ))) -return true; +return 0 ; ops->get_segment (ctxt, &tr, &tr_seg, &base3,VCPU_SREG_TR );if (!tr_seg.p) -return false; +return -EACCES ;if (desc_limit_scaled (&tr_seg) <103 ) -return false; +return -EACCES ; base =get_desc_base (&tr_seg);#ifdef CONFIG_X86_64 base |= ((u64 )base3) <<32 ;#endif r = ops->read_std (ctxt, base +102 , &io_bitmap_ptr,2 ,NULL , true);if (r !=X86EMUL_CONTINUE ) -return false; +return -EACCES ;if (io_bitmap_ptr + port/8 >desc_limit_scaled (&tr_seg)) -return false; +return -EACCES ; r = ops->read_std (ctxt, base + io_bitmap_ptr + port/8 , &perm,2 ,NULL , true);if (r !=X86EMUL_CONTINUE ) -return false; +return -EACCES ;if ((perm >> bit_idx) & mask) -return false; -return true; +return -EACCES ; +return 0 ; } -static bool emulator_io_permitted (struct x86_emulate_ctxt *ctxt, -u16 port,u16 len) +static int emulator_io_permitted (struct x86_emulate_ctxt *ctxt, +u16 port,u16 len) { -if (ctxt->perm_ok) -return true; +int r; -if (emulator_bad_iopl (ctxt)) -if (!emulator_io_port_access_allowed (ctxt, port, len)) -return false; +if (ctxt->perm_ok) +return 0 ; + +if (emulator_bad_iopl (ctxt)) { + r =emulator_io_port_access_allowed (ctxt, port, len); +if (r) +return r; + } ctxt->perm_ok = true; -return true; +return 0 ; }static void string_registers_quirk (struct x86_emulate_ctxt *ctxt)
@@ -3931,8 +3935,11 @@ static int check_rdpmc(struct x86_emulate_ctxt *ctxt)static int check_perm_in (struct x86_emulate_ctxt *ctxt) { +int r; + ctxt->dst.bytes =min (ctxt->dst.bytes,4u ); -if (!emulator_io_permitted (ctxt, ctxt->src.val, ctxt->dst.bytes)) + r =emulator_io_permitted (ctxt, ctxt->src.val, ctxt->dst.bytes); +if (r)return emulate_gp (ctxt,0 );return X86EMUL_CONTINUE ;
@@ -3940,8 +3947,11 @@ static int check_perm_in(struct x86_emulate_ctxt *ctxt)static int check_perm_out (struct x86_emulate_ctxt *ctxt) { +int r; + ctxt->src.bytes =min (ctxt->src.bytes,4u ); -if (!emulator_io_permitted (ctxt, ctxt->dst.val, ctxt->src.bytes)) + r =emulator_io_permitted (ctxt, ctxt->dst.val, ctxt->src.bytes); +if (r)return emulate_gp (ctxt,0 );return X86EMUL_CONTINUE ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 30/45 KVM: x86/emulate: Propagate memory faults to userspace
In reply to Paolo BonziniFull handling of memory faults is not really possible with KVM's architecture, short of adding something like a journal to complex operations like task switches or SMM enter/leave. However, KVM also should at least try to propagate faults to userspace, as an emulation failure if it is not possible to return them as -EFAULT.
Introduce new return values for both the emulator ops (X86EMUL_MEMORY_FAULT) and emulate.c (EMULATION_MEMORY_FAULT), and plumb them through the entire call chain. Terminate them with -EFAULT as a best effort.
--- arch/x86/kvm/emulate.c | 24 ++++++++++++++-- arch/x86/kvm/kvm_emulate.h | 3 ++ arch/x86/kvm/svm/svm.c | 6 +++- arch/x86/kvm/vmx/sgx.c | 4 ++- arch/x86/kvm/vmx/vmx.c | 22 +++++++++------ arch/x86/kvm/x86.c | 56 ++++++++++++++++++++++++++------------ include/linux/kvm_host.h | 3 ++ virt/kvm/kvm_main.c | 24 ++++++++++++++++ 8 files changed, 111 insertions(+), 31 deletions(-)
arch/x86/kvm/emulate.c +22 -2
@@ -2555,11 +2555,15 @@ static int emulator_io_port_access_allowed(struct x86_emulate_ctxt *ctxt, base |= ((u64 )base3) <<32 ;#endif r = ops->read_std (ctxt, base +102 , &io_bitmap_ptr,2 ,NULL , true); +if (r ==X86EMUL_MEMORY_FAULT ) +return r;if (r !=X86EMUL_CONTINUE )return -EACCES ;if (io_bitmap_ptr + port/8 >desc_limit_scaled (&tr_seg))return -EACCES ; r = ops->read_std (ctxt, base + io_bitmap_ptr + port/8 , &perm,2 ,NULL , true); +if (r ==X86EMUL_MEMORY_FAULT ) +return r;if (r !=X86EMUL_CONTINUE )return -EACCES ;if ((perm >> bit_idx) & mask)
@@ -2758,7 +2762,10 @@ static int load_state_from_tss32(struct x86_emulate_ctxt *ctxt,int ret;u8 cpl; -if (ctxt->ops->set_cr (ctxt,3 , tss->cr3)) + ret = ctxt->ops->set_cr (ctxt,3 , tss->cr3); +if (ret ==X86EMUL_MEMORY_FAULT ) +return ret; +if (ret)return emulate_gp (ctxt,0 ); ctxt->_eip = tss->eip; ctxt->eflags = tss->eflags |2 ;
@@ -2993,6 +3000,8 @@ int emulator_task_switch(struct x86_emulate_ctxt *ctxt, ctxt->eip = ctxt->_eip;writeback_registers (ctxt); } +if (rc ==X86EMUL_MEMORY_FAULT ) +return EMULATION_MEMORY_FAULT ;return (rc ==X86EMUL_UNHANDLEABLE ) ?EMULATION_FAILED :EMULATION_OK ; }
@@ -3260,7 +3269,10 @@ static int em_cr_write(struct x86_emulate_ctxt *ctxt)int cr_num = ctxt->modrm_reg;int r; -if (ctxt->ops->set_cr (ctxt, cr_num, ctxt->src.val)) + r = ctxt->ops->set_cr (ctxt, cr_num, ctxt->src.val); +if (r ==X86EMUL_MEMORY_FAULT ) +return r; +if (r)return emulate_gp (ctxt,0 );/* Disable writeback. */
@@ -3939,6 +3951,8 @@ static int check_perm_in(struct x86_emulate_ctxt *ctxt) ctxt->dst.bytes =min (ctxt->dst.bytes,4u ); r =emulator_io_permitted (ctxt, ctxt->src.val, ctxt->dst.bytes); +if (r ==X86EMUL_MEMORY_FAULT ) +return r;if (r)return emulate_gp (ctxt,0 );
@@ -3951,6 +3965,8 @@ static int check_perm_out(struct x86_emulate_ctxt *ctxt) ctxt->src.bytes =min (ctxt->src.bytes,4u ); r =emulator_io_permitted (ctxt, ctxt->dst.val, ctxt->src.bytes); +if (r ==X86EMUL_MEMORY_FAULT ) +return r;if (r)return emulate_gp (ctxt,0 );
@@ -5219,6 +5235,8 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len, int done:if (rc ==X86EMUL_PROPAGATE_FAULT ) ctxt->have_exception = true; +if (rc ==X86EMUL_MEMORY_FAULT ) +return EMULATION_MEMORY_FAULT ;return (rc !=X86EMUL_CONTINUE ) ?EMULATION_FAILED :EMULATION_OK ; }
@@ -5579,6 +5597,8 @@ int x86_emulate_insn(struct x86_emulate_ctxt *ctxt, bool check_intercepts) ctxt->eip = (u32 )ctxt->_eip; done: +if (rc ==X86EMUL_MEMORY_FAULT ) +return EMULATION_MEMORY_FAULT ;if (rc ==X86EMUL_PROPAGATE_FAULT ) {if (KVM_EMULATOR_BUG_ON (ctxt->exception.vector >0x1f , ctxt))return EMULATION_FAILED ;
arch/x86/kvm/kvm_emulate.h +3 -0
@@ -97,6 +97,8 @@ struct x86_instruction_info {#define X86EMUL_INTERCEPTED 6/* Intercepted by nested VMCB/VMCS */ /* Emulation during event vectoring is unhandleable. */ #define X86EMUL_UNHANDLEABLE_VECTORING 7 +/* Terminate emulation and return a memory fault to userspace. */ +#define X86EMUL_MEMORY_FAULT 8/* x86-specific emulation flags */ #define X86EMUL_F_WRITE BIT(0)
@@ -532,6 +534,7 @@ bool x86_page_table_writing_insn(struct x86_emulate_ctxt *ctxt);#define EMULATION_OK 0#define EMULATION_RESTART 1#define EMULATION_INTERCEPTED 2 +#define EMULATION_MEMORY_FAULT 3void init_decode_cache (struct x86_emulate_ctxt *ctxt);int x86_emulate_insn (struct x86_emulate_ctxt *ctxt,bool check_intercepts); int emulator_task_switch(struct x86_emulate_ctxt *ctxt,
arch/x86/kvm/svm/svm.c +5 -1
@@ -2363,13 +2363,17 @@ static int gp_interception(struct kvm_vcpu *vcpu)struct vcpu_svm *svm =to_svm (vcpu);u32 error_code = svm->vmcb->control.exit_info_1;u64 svm_exit_code; +int r;/* Both #GP cases have zero error_code */ if (error_code) goto reinject;/* Decode the instruction for usage later */ -if (x86_decode_emulated_instruction (vcpu,0 ,NULL ,0 ) !=EMULATION_OK ) + r =x86_decode_emulated_instruction (vcpu,0 ,NULL ,0 ); +if (r ==EMULATION_MEMORY_FAULT ) +return -EFAULT ; +if (r !=EMULATION_OK ) goto reinject;/* FIXME: Handle SVM instructions through the emulator */
arch/x86/kvm/vmx/sgx.c +3 -1
@@ -236,7 +236,9 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu)*/ r =kvm_read_guest_virt (vcpu, pageinfo_gva, &pageinfo,sizeof (pageinfo), &ex); -if (r ==X86EMUL_PROPAGATE_FAULT ) { +if (r ==X86EMUL_MEMORY_FAULT ) { +return -EFAULT ; + }else if (r ==X86EMUL_PROPAGATE_FAULT ) {kvm_inject_emulated_page_fault (vcpu, &ex);return 1 ; }else if (r !=X86EMUL_CONTINUE ) {
arch/x86/kvm/vmx/vmx.c +13 -9
@@ -5342,19 +5342,21 @@ static bool rmode_exception(struct kvm_vcpu *vcpu, int vec)static int handle_rmode_exception (struct kvm_vcpu *vcpu,int vec,u32 err_code) { +int r; +/* * Instruction with address size override prefix opcode 0x67 * Cause the #SS fault with 0 error code in VM86 mode. */ if (((vec ==GP_VECTOR ) || (vec ==SS_VECTOR )) && err_code ==0 ) { -if (kvm_emulate_instruction (vcpu,0 )) { -if (vcpu->arch.halt_request) { - vcpu->arch.halt_request =0 ; -return kvm_emulate_halt_noskip (vcpu); - } -return 1 ; + r =kvm_emulate_instruction (vcpu,0 ); +if (r <=0 ) +return r; +if (vcpu->arch.halt_request) { + vcpu->arch.halt_request =0 ; +return kvm_emulate_halt_noskip (vcpu); } -return 0 ; +return 1 ; }/*
@@ -6080,6 +6082,7 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)struct vcpu_vmx *vmx =to_vmx (vcpu);bool intr_window_requested;unsigned count =130 ; +int r; intr_window_requested =exec_controls_get (vmx) &CPU_BASED_INTR_WINDOW_EXITING ;
@@ -6098,8 +6101,9 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)*/ smp_rmb (); -if (!kvm_emulate_instruction (vcpu,0 )) -return 0 ; + r =kvm_emulate_instruction (vcpu,0 ); +if (r <=0 ) +return r;if (vmx_unhandleable_emulation_required (vcpu)) {kvm_prepare_emulation_failure_exit (vcpu);
arch/x86/kvm/x86.c +38 -18
148 lines not on this page · raw message
include/linux/kvm_host.h +3 -0
@@ -2641,6 +2641,9 @@ static inline u64 kvm_mem_attributes_generation(struct kvm *kvm)*/ return smp_load_acquire(&kvm->mem_attrs.generation); } + +bool kvm_prepare_memory_fault_exit_for_range (struct kvm_vcpu *vcpu,gpa_t gpa, +size_t size,bool write);#else static inline bool kvm_range_has_any_vm_memory_attributes (struct kvm *kvm,gfn_t start,gfn_t end,unsigned long mask)
virt/kvm/kvm_main.c +24 -0
@@ -2485,6 +2485,30 @@ static u64 kvm_supports_memory_protection_attr(struct kvm *kvm)#endif #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES +bool kvm_prepare_memory_fault_exit_for_range (struct kvm_vcpu *vcpu,gpa_t gpa, +size_t size,bool write) +{ +gfn_t gfn, last; + + last =gpa_to_gfn (gpa + size -1 ); +for (gfn =gpa_to_gfn (gpa); gfn <= last; gfn++) { +if (!kvm_mem_attributes_may_read_gfn (vcpu->kvm, gfn)) { +kvm_prepare_memory_fault_exit (vcpu,gfn_to_gpa (gfn),PAGE_SIZE , + write, false, false); +return true; + } + +if (write && !kvm_mem_attributes_may_write_gfn (vcpu->kvm, gfn)) { +kvm_prepare_memory_fault_exit (vcpu,gfn_to_gpa (gfn),PAGE_SIZE , + true, false, false); +return true; + } + } + +return false; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_prepare_memory_fault_exit_for_range); +u64 kvm_supported_vm_mem_attributes (struct kvm *kvm) {u64 supported_attrs =0 ;
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 30/45] KVM: x86/emulate: Propagate memory faults to userspace
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] KVM: x86: x86_emulate_instruction() fails to exit on EMULATION_MEMORY_FAULT - [Medium] KVM: x86…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] KVM: x86: x86_emulate_instruction() fails to exit on EMULATION_MEMORY_FAULT
- [Medium] KVM: x86: handle X86EMUL_MEMORY_FAULT in kvm_handle_memory_failure()
- [Medium] KVM: x86: emulator_read_write_onepage() faults on MMIO with mem attributes --
commit bb98c6118897ded1d558925a185b73b85f254506
Author: Paolo Bonzini <[email protected]>
KVM: x86/emulate: Propagate memory faults to userspace
This commit introduces X86EMUL_MEMORY_FAULT and EMULATION_MEMORY_FAULT to propagate guest memory attribute faults through the x86 instruction emulator and out to userspace as -EFAULT or an emulation failure.
Quoted 31 lines
arch/x86/kvm/x86.c +4
-5@@ -4865,7 +4865,6 @@ static int kvm_read_guest_virt_helper(gva_t addr, void *val, unsigned int bytes, {struct kvm_pagewalk *gva_walk = &vcpu->arch.gva_walk;void *data = val; -int r =X86EMUL_CONTINUE ;while (bytes) {gpa_t gpa = gva_walk->gva_to_gpa (vcpu, gva_walk, addr, access, exception);@@ -4878,16 +4877,16 @@ static int kvm_read_guest_virt_helper(gva_t addr, void *val, unsigned int bytes, ret =kvm_vcpu_read_guest_page (vcpu, gpa >>PAGE_SHIFT , data, offset, toread);if (ret <0 ) { - r =X86EMUL_IO_NEEDED ; - goto out; +if (kvm_prepare_memory_fault_exit_for_range (vcpu, gpa, toread, false)) +return X86EMUL_MEMORY_FAULT ; +return X86EMUL_IO_NEEDED ; } bytes -= toread; data += toread; addr += toread; } -out: -return r; +return X86EMUL_CONTINUE ; }
[ ... ]
arch/x86/kvm/x86.c static int kvm_write_guest_virt_helper(gva_t addr, void *val, unsigned int bytes · line 4981
Quoted 10 lines
arch/x86/kvm/x86.c +3
-2@@ -4969,16 +4975,16 @@ static int kvm_write_guest_virt_helper(gva_t addr, void *val, unsigned int bytesreturn X86EMUL_PROPAGATE_FAULT ; ret =kvm_vcpu_write_guest (vcpu, gpa, data, towrite);if (ret <0 ) { - r =X86EMUL_IO_NEEDED ; - goto out; +if (kvm_prepare_memory_fault_exit_for_range (vcpu, gpa, towrite, true)) +return X86EMUL_MEMORY_FAULT ; +return X86EMUL_IO_NEEDED ; }
[Severity: Medium]
Should kvm_handle_memory_failure() in arch/x86/kvm/x86.c also check for X86EMUL_MEMORY_FAULT and return -EFAULT?
When a guest executes INVPCID, VMXON, VMCLEAR, VMPTRLD, VMREAD, VMWRITE, VMPTRST, INVEPT, or INVVPID with a memory operand backed by a page that has KVM_MEMORY_ATTRIBUTE_NR or KVM_MEMORY_ATTRIBUTE_NW set, kvm_read_guest_virt() or kvm_write_guest_virt_system() calls kvm_prepare_memory_fault_exit_for_range() (setting vcpu->run->exit_reason to KVM_EXIT_MEMORY_FAULT) and returns X86EMUL_MEMORY_FAULT.
Callers such as kvm_handle_invpcid() in arch/x86/kvm/x86.c and nested_vmx_get_vmptr(), handle_vmread(), handle_vmwrite(), handle_vmptrst(), handle_invept(), and handle_invvpid() in arch/x86/kvm/vmx/nested.c pass that return value to kvm_handle_memory_failure():
arch/x86/kvm/x86.c:kvm_handle_memory_failure() {
if (r == X86EMUL_PROPAGATE_FAULT) {
if (KVM_BUG_ON(!e, vcpu->kvm))
return -EIO;
kvm_inject_emulated_page_fault(vcpu, e);
return 1;
}
kvm_prepare_emulation_failure_exit(vcpu);
return 0;
}Because kvm_handle_memory_failure() does not handle X86EMUL_MEMORY_FAULT, it calls kvm_prepare_emulation_failure_exit(), overwriting the prepared KVM_EXIT_MEMORY_FAULT in vcpu->run with KVM_EXIT_INTERNAL_ERROR and returning 0 instead of -EFAULT.
Similarly, in handle_ud() in arch/x86/kvm/x86.c, if force_emulation_prefix is enabled and reading the 5-byte prefix via kvm_read_guest_virt() hits a KVM_MEMORY_ATTRIBUTE_NR page and returns X86EMUL_MEMORY_FAULT, won't handle_ud() ignore the prepared memory fault exit and fall through to inject a #UD into the guest instead of returning -EFAULT?
[ ... ]
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0@@ -5151,8 +5159,14 @@ static int emulator_read_write_onepage(unsigned long addr, void *val,* i.e. if accessing host user memory failed, but this has been KVM's * historical ABI for decades. */
arch/x86/kvm/x86.c static int emulator_read_write_onepage(unsigned long addr, void *val, · line 5169
Quoted 10 lines
arch/x86/kvm/x86.c +8
-2-if (!ret && ops->read_write_guest (vcpu, gpa, val, bytes)) -return X86EMUL_CONTINUE ; +if (!ret) { + ret = ops->read_write_guest (vcpu, gpa, val, bytes); +if (!ret) +return X86EMUL_CONTINUE ; + } + +if (kvm_prepare_memory_fault_exit_for_range (vcpu, gpa, bytes, write)) +return X86EMUL_MEMORY_FAULT ;
[Severity: Medium]
What happens here in emulator_read_write_onepage() when a guest accesses an emulated MMIO region (such as the local APIC at APIC_DEFAULT_PHYS_BASE or a GPA with no memslot) while KVM_MEMORY_ATTRIBUTE_NR or KVM_MEMORY_ATTRIBUTE_NW is set on that GPA range?
In arch/x86/kvm/mmu/mmu.c, both kvm_mmu_faultin_pfn() and kvm_faultin_memory_protections() skip memory attribute checks when !slot, allowing MMIO SPTE creation and returning RET_PF_EMULATE.
However, in emulator_read_write_onepage(), kvm_prepare_memory_fault_exit_for_range() is called both when ret == 1 (the GPA was already identified as MMIO by vcpu_is_mmio_gpa() or vcpu_mmio_gva_to_gpa()) and when ops->read_write_guest() fails due to a missing memslot.
Because kvm_prepare_memory_fault_exit_for_range() in virt/kvm/kvm_main.c checks kvm_mem_attributes_may_read_gfn() and kvm_mem_attributes_may_write_gfn() without checking whether a memslot exists for the GFN, won't it return X86EMUL_MEMORY_FAULT and abort MMIO emulation before reaching ops->read_write_mmio()?
[ ... ]
arch/x86/kvm/x86.c int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, · line 6552
Quoted 9 lines
arch/x86/kvm/x86.c +1
-1@@ -6528,7 +6546,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,if (r ==EMULATION_INTERCEPTED )return 1 ; -if (r ==EMULATION_FAILED ) { +if (r ==EMULATION_MEMORY_FAULT || r ==EMULATION_FAILED ) {if (kvm_unprotect_and_retry_on_failure (vcpu, cr2_or_gpa, emulation_type))return 1 ;
[Severity: High]
When x86_emulate_insn() returns EMULATION_MEMORY_FAULT in x86_emulate_instruction(), routing it through kvm_unprotect_and_retry_on_failure() and handle_emulation_failure() seems to prevent the memory fault from reaching userspace in several cases:
arch/x86/kvm/x86.c:handle_emulation_failure() {
...
if (emulation_type & EMULTYPE_VMWARE_GP) {
kvm_queue_exception_e(vcpu, GP_VECTOR, 0);
return 1;
}
if (kvm->arch.exit_on_emulation_error ||
(emulation_type & EMULTYPE_SKIP)) {
prepare_emulation_ctxt_failure_exit(vcpu);
return 0;
}
kvm_queue_exception(vcpu, UD_VECTOR);
if (!is_guest_mode(vcpu) && kvm_x86_call(get_cpl)(vcpu) == 0) {
prepare_emulation_ctxt_failure_exit(vcpu);
return 0;
}
return 1;
}1. If EMULTYPE_ALLOW_RETRY_PF is set, kvm_unprotect_and_retry_on_failure() will unprotect the shadow page and return true, causing x86_emulate_instruction() to return 1 and re-enter the guest instead of exiting to userspace.
2. If kvm->arch.exit_on_emulation_error is false (the default) and !(emulation_type & EMULTYPE_SKIP), handle_emulation_failure() queues a #UD into the guest and, whenever is_guest_mode(vcpu) is true or get_cpl(vcpu) != 0, returns 1 to resume the guest without exiting to userspace. In particular, emulator_io_port_access_allowed() in arch/x86/kvm/emulate.c is only called when emulator_bad_iopl(ctxt) is true (which requires CPL > 0), so returning X86EMUL_MEMORY_FAULT from emulator_io_port_access_allowed() will always have get_cpl(vcpu) > 0 and inject #UD (or #GP for EMULTYPE_VMWARE_GP) into the guest instead of exiting to userspace.
3. Even when handle_emulation_failure() does exit to userspace by returning 0, prepare_emulation_ctxt_failure_exit() overwrites the KVM_EXIT_MEMORY_FAULT prepared by kvm_prepare_memory_fault_exit_for_range() with KVM_EXIT_INTERNAL_ERROR, and still queues #UD if !kvm->arch.exit_on_emulation_error.
Should EMULATION_MEMORY_FAULT be handled separately from EMULATION_FAILED so that it always exits to userspace (for example by returning -EFAULT when no partial state changes have occurred, or preparing the failure exit without retrying or injecting #UD/#GP into the guest)?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=30
Paolo Bonzini
maintainerPATCH v4 31/45 KVM: x86: "Handle" failed decode when skipping instructions
In reply to Paolo BonziniWhen calling into the emulator to skip an instruction, the caller has already performed the side effects of the instruction and cannot propagate -EFAULT. Mark the VM dead.
--- arch/x86/kvm/svm/svm.c | 9 +++++++-- arch/x86/kvm/vmx/vmx.c | 10 +++++++--- arch/x86/kvm/x86.c | 4 ++-- 3 files changed, 16 insertions(+), 7 deletions(-)
arch/x86/kvm/svm/svm.c +7 -2
@@ -304,6 +304,7 @@ static int __svm_skip_emulated_instruction(struct kvm_vcpu *vcpu, {struct vcpu_svm *svm =to_svm (vcpu);unsigned long old_rflags; +int r;/* * SEV-ES does not expose the next RIP. The RIP update is controlled by
@@ -321,7 +322,10 @@ static int __svm_skip_emulated_instruction(struct kvm_vcpu *vcpu,if (unlikely (!commit_side_effects)) old_rflags = svm->vmcb->save.rflags; -if (!kvm_emulate_instruction (vcpu, emul_type)) + r =kvm_emulate_instruction (vcpu, emul_type); +if (r <0 ) +kvm_vm_dead (vcpu->kvm); +if (r <=0 )return 0 ;if (unlikely (!commit_side_effects))
@@ -2128,7 +2132,8 @@ static int icebp_interception(struct kvm_vcpu *vcpu)* instead of after it (and SVM doesn't provide enough information for * KVM to detect and manually advance the pre-#DB RIP). */ -svm_skip_emulated_instruction (vcpu); +if (!svm_skip_emulated_instruction (vcpu)) +return 0 ;kvm_queue_exception (vcpu,DB_VECTOR );return 1 ; }
arch/x86/kvm/vmx/vmx.c +7 -3
@@ -1809,6 +1809,7 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu)union vmx_exit_reason exit_reason =vt_get_exit_reason (vcpu);unsigned long rip, orig_rip;u32 instr_len; +int r;/* * Using VMCS.VM_EXIT_INSTRUCTION_LEN on EPT misconfig depends on
@@ -1857,7 +1858,10 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu)#endif kvm_rip_write (vcpu, rip); }else { -if (!kvm_emulate_instruction (vcpu,EMULTYPE_SKIP )) + r =kvm_emulate_instruction (vcpu,EMULTYPE_SKIP ); +if (r <0 ) +kvm_vm_dead (vcpu->kvm); +if (r <=0 )return 0 ; }
@@ -5538,8 +5542,8 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)* higher priority. Note, skipping ICEBP still clears * STI and MOVSS blocking. */ -if (is_icebp (intr_info)) -WARN_ON (!skip_emulated_instruction (vcpu)); +if (is_icebp (intr_info) && !skip_emulated_instruction (vcpu)) +return 0 ;kvm_queue_exception_p (vcpu,DB_VECTOR , dr6);return 1 ;
arch/x86/kvm/x86.c +2 -2
@@ -6235,8 +6235,8 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu)int r; r =kvm_x86_call (skip_emulated_instruction)(vcpu); -if (unlikely (!r)) -return 0 ; +if (unlikely (r <=0 )) +return r;kvm_pmu_instruction_retired (vcpu);
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 31/45] KVM: x86: "Handle" failed decode when skipping instructions
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM: x86: skip_emulated_instruction() returns 0 with KVM_EXIT_MEMORY_FAULT - [Medium] KVM: SVM…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM: x86: skip_emulated_instruction() returns 0 with KVM_EXIT_MEMORY_FAULT
- [Medium] KVM: SVM: __svm_skip_emulated_instruction() kills VM on soft int RIP update --
commit 7e42aaabee05dd2baa14946c524ae1b2af0f0bd1
Author: Paolo Bonzini <[email protected]>
KVM: x86: "Handle" failed decode when skipping instructions
This commit marks the VM dead when kvm_emulate_instruction() returns a negative error code while skipping an emulated instruction in SVM and VMX, and updates kvm_skip_emulated_instruction() and ICEBP handlers to check for failure.
Quoted 14 lines
arch/x86/kvm/svm/svm.c +1
-0@@ -304,6 +304,7 @@ static int __svm_skip_emulated_instruction(struct kvm_vcpu *vcpu, {struct vcpu_svm *svm =to_svm (vcpu);unsigned long old_rflags; +int r;/* * SEV-ES does not expose the next RIP. The RIP update is controlled by @@ -321,7 +322,10 @@ static int __svm_skip_emulated_instruction(struct kvm_vcpu *vcpu,if (unlikely (!commit_side_effects))
arch/x86/kvm/svm/svm.c static int __svm_skip_emulated_instruction(struct kvm_vcpu *vcpu, · line 331
Quoted 10 lines
arch/x86/kvm/svm/svm.c +4
-1old_rflags = svm->vmcb->save.rflags; -if (!kvm_emulate_instruction (vcpu, emul_type)) + r =kvm_emulate_instruction (vcpu, emul_type); +if (r <0 ) +kvm_vm_dead (vcpu->kvm); +if (r <=0 )return 0 ;if (unlikely (!commit_side_effects))
[Severity: Medium]
Should __svm_skip_emulated_instruction() really mark the VM dead when called from svm_update_soft_interrupt_rip()?
Prior to VM-Entry, kvm_check_and_inject_events() invokes svm_inject_exception() or svm_inject_irq() -> svm_update_soft_interrupt_rip() to compute NextRIP before injecting a software exception or software interrupt:
arch/x86/kvm/svm/svm.c:svm_update_soft_interrupt_rip() {
const int emul_type = EMULTYPE_SKIP | EMULTYPE_SKIP_SOFT_INT |
EMULTYPE_SET_SOFT_INT_VECTOR(vector);
...
if (!__svm_skip_emulated_instruction(vcpu, emul_type, !nrips))
return -EIO;In that path, no instruction side effects have been performed yet (and when nrips is enabled, commit_side_effects is false). As noted in the comment in svm_update_soft_interrupt_rip(), decoding is not guaranteed to succeed, and a failure returns -EIO so the caller drops the event injection and lets the guest retry the instruction.
If the guest page at CS:RIP has restrictive memory attributes (such as KVM_MEMORY_ATTRIBUTE_NX), kvm_emulate_instruction() returns -EFAULT and __svm_skip_emulated_instruction() now calls kvm_vm_dead(vcpu->kvm), terminating the VM instead of aborting the injection.
[ ... ]
Quoted 24 lines
arch/x86/kvm/vmx/vmx.c +5
-1@@ -1809,6 +1809,7 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu)union vmx_exit_reason exit_reason =vt_get_exit_reason (vcpu);unsigned long rip, orig_rip;u32 instr_len; +int r;/* * Using VMCS.VM_EXIT_INSTRUCTION_LEN on EPT misconfig depends on @@ -1857,7 +1858,10 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu)#endif kvm_rip_write (vcpu, rip); }else { -if (!kvm_emulate_instruction (vcpu,EMULTYPE_SKIP )) + r =kvm_emulate_instruction (vcpu,EMULTYPE_SKIP ); +if (r <0 ) +kvm_vm_dead (vcpu->kvm); +if (r <=0 )return 0 ; }
[ ... ]
Quoted 5 lines
arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu) · line 6242
Quoted 10 lines
arch/x86/kvm/x86.c +2
-2@@ -6235,8 +6235,8 @@ int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu)int r; r =kvm_x86_call (skip_emulated_instruction)(vcpu); -if (unlikely (!r)) -return 0 ; +if (unlikely (r <=0 )) +return r;kvm_pmu_instruction_retired (vcpu);
[Severity: High]
Won't clamping r < 0 to 0 in __svm_skip_emulated_instruction() and VMX's skip_emulated_instruction() cause KVM_RUN to return 0 with exit_reason set to KVM_EXIT_MEMORY_FAULT, while also making the r < 0 case in kvm_skip_emulated_instruction() unreachable?
When an instruction fetch hits a guest page with non-executable or non-readable memory attributes during kvm_emulate_instruction() -> x86_emulate_instruction() -> x86_decode_emulated_instruction() -> x86_decode_insn() -> kvm_fetch_guest_virt(), kvm_prepare_memory_fault_exit() sets vcpu->run->exit_reason = KVM_EXIT_MEMORY_FAULT, and x86_emulate_instruction() returns -EFAULT before reaching handle_emulation_failure():
arch/x86/kvm/x86.c:x86_emulate_instruction() {
r = x86_decode_emulated_instruction(vcpu, emulation_type,
insn, insn_len);
if (r == EMULATION_MEMORY_FAULT)
return -EFAULT;In both __svm_skip_emulated_instruction() and VMX's skip_emulated_instruction(), the check:
if (r < 0) kvm_vm_dead(vcpu->kvm); if (r <= 0) return 0;
returns 0 when r is negative, so kvm_x86_call(skip_emulated_instruction)(vcpu) never returns a negative value to kvm_skip_emulated_instruction().
When 0 is propagated back to vcpu_enter_guest() -> vcpu_run() (or through vcpu->arch.complete_userspace_io in kvm_arch_vcpu_ioctl_run()), the run loop exits to userspace immediately on r <= 0 without re-entering vcpu_enter_guest() where KVM_REQ_VM_DEAD is checked and converted into -EIO. As a result, ioctl(KVM_RUN) returns 0 to userspace with vcpu->run->exit_reason left as KVM_EXIT_MEMORY_FAULT, violating the KVM_EXIT_MEMORY_FAULT UAPI contract (which requires a -1 return with errno set to EFAULT or EHWPOISON). In addition, in callers such as kvm_emulate_halt() and kvm_emulate_ap_reset_hold() with a userspace irqchip, __kvm_emulate_halt() overwrites exit_reason with KVM_EXIT_HLT or KVM_EXIT_AP_RESET_HOLD before returning 0.
Note that if the vendor callbacks are changed to return a negative error code (such as -EIO) when r < 0, callers that test the return value as a boolean -- such as !svm_skip_emulated_instruction(vcpu) in icebp_interception(), task_switch_interception(), nested_svm_vmrun(), and svm_update_soft_interrupt_rip(); !skip_emulated_instruction(vcpu) in handle_exception_nmi() and handle_task_switch(); !kvm_skip_emulated_instruction(vcpu) in kvm_emulate_wrmsr(); and ret && kvm_skip_emulated_instruction(vcpu) in kvm_fast_pio(), kvm_emulate_halt(), and kvm_emulate_ap_reset_hold() -- would also need to check <= 0 so they do not treat a negative errno as success.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=31
Paolo Bonzini
maintainerPATCH v4 32/45 KVM: vmx/sgx: make return value of sgx_gva_to_gpa consistent with its callers
In reply to Paolo BonziniReturn 1 instead of -EFAULT, in preparation for using -EFAULT for memory protection failures.
--- arch/x86/kvm/vmx/sgx.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/sgx.c +15 -9
@@ -84,7 +84,7 @@ static int sgx_gva_to_gpa(struct kvm_vcpu *vcpu, gva_t gva, bool write,if (*gpa ==INVALID_GPA ) {kvm_inject_emulated_page_fault (vcpu, &ex); -return -EFAULT ; +return 1 ; }return 0 ;
@@ -256,10 +256,13 @@ static int handle_encls_ecreate(struct kvm_vcpu *vcpu)* Translate the SECINFO, SOURCE and SECS pointers from GVA to GPA. * Resume the guest on failure to inject a #PF. */ -if (sgx_gva_to_gpa (vcpu, metadata_gva, false, &metadata_gpa) || -sgx_gva_to_gpa (vcpu, contents_gva, false, &contents_gpa) || -sgx_gva_to_gpa (vcpu, secs_gva, true, &secs_gpa)) -return 1 ; + r =sgx_gva_to_gpa (vcpu, metadata_gva, false, &metadata_gpa); +if (!r) + r =sgx_gva_to_gpa (vcpu, contents_gva, false, &contents_gpa); +if (!r) + r =sgx_gva_to_gpa (vcpu, secs_gva, true, &secs_gpa); +if (r) +return r <0 ? r :1 ;/* * ...and then to HVA. The order of accesses isn't architectural, i.e.
@@ -314,10 +317,13 @@ static int handle_encls_einit(struct kvm_vcpu *vcpu)* Translate the SIGSTRUCT, SECS and TOKEN pointers from GVA to GPA. * Resume the guest on failure to inject a #PF. */ -if (sgx_gva_to_gpa (vcpu, sig_gva, false, &sig_gpa) || -sgx_gva_to_gpa (vcpu, secs_gva, true, &secs_gpa) || -sgx_gva_to_gpa (vcpu, token_gva, false, &token_gpa)) -return 1 ; + ret =sgx_gva_to_gpa (vcpu, sig_gva, false, &sig_gpa); +if (!ret) + ret =sgx_gva_to_gpa (vcpu, secs_gva, true, &secs_gpa); +if (!ret) + ret =sgx_gva_to_gpa (vcpu, token_gva, false, &token_gpa); +if (ret) +return ret <0 ? ret :1 ;/* * ...and then to HVA. The order of accesses isn't architectural, i.e.
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 33/45 KVM: x86: Allow page fault fixup to return errors
In reply to Paolo Bonzinikvm_fixup_and_inject_pf_error() reflects intercepted hardware page faults into the guest, walking the guest page tables when necessary to obtain an error code consistent with the guest's paging rules. For example, a GPA that exceeds guest.MAXPHYADDR but fits within host.MAXPHYADDR must cause a reserved-bit fault, even though hardware does not report one.
Allow the function's callers to propagate memory attribute failures from the page table walk; subsequent changes will return from kvm_fixup_and_inject_pf_error() any error from the page-table walk.
--- arch/x86/kvm/vmx/vmx.c | 15 +++++++++++---- arch/x86/kvm/x86.c | 3 ++- arch/x86/kvm/x86.h | 2 +- 3 files changed, 14 insertions(+), 6 deletions(-)
arch/x86/kvm/vmx/vmx.c +11 -4
@@ -5418,10 +5418,14 @@ static int vmx_handle_page_fault(struct kvm_vcpu *vcpu, u32 error_code)WARN_ON_ONCE (!IS_ENABLED (CONFIG_X86_SGX_KVM ) || !cpu_feature_enabled (X86_FEATURE_SGX2 )); -if (guest_cpu_cap_has (vcpu,X86_FEATURE_SGX2 )) -kvm_fixup_and_inject_pf_error (vcpu, cr2, error_code); -else +if (guest_cpu_cap_has (vcpu,X86_FEATURE_SGX2 )) { +int r =kvm_fixup_and_inject_pf_error (vcpu, cr2, error_code); + +if (r) +return r; + }else {kvm_inject_gp (vcpu,0 ); + }return 1 ; }
@@ -5431,7 +5435,10 @@ static int vmx_handle_page_fault(struct kvm_vcpu *vcpu, u32 error_code)* to the GPA being legal with respect to host.MAXPHYADDR). */ if (enable_ept) { -kvm_fixup_and_inject_pf_error (vcpu, cr2, error_code); +int r =kvm_fixup_and_inject_pf_error (vcpu, cr2, error_code); + +if (r) +return r;return 1 ; }
arch/x86/kvm/x86.c +2 -1
@@ -10811,7 +10811,7 @@ void kvm_arch_gmem_invalidate_range(struct kvm *kvm, struct kvm_gfn_range *range#endif #endif -void kvm_fixup_and_inject_pf_error (struct kvm_vcpu *vcpu,gva_t gva,u16 error_code) +int kvm_fixup_and_inject_pf_error (struct kvm_vcpu *vcpu,gva_t gva,u16 error_code) {struct kvm_pagewalk *gva_walk = &vcpu->arch.gva_walk;struct x86_exception fault;
@@ -10833,6 +10833,7 @@ void kvm_fixup_and_inject_pf_error(struct kvm_vcpu *vcpu, gva_t gva, u16 error_c fault.async_page_fault = false; } gva_walk->inject_page_fault (vcpu, &fault, true); +return 0 ; }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_fixup_and_inject_pf_error);
arch/x86/kvm/x86.h +1 -1
@@ -368,7 +368,7 @@ void kvm_deliver_exception_payload(struct kvm_vcpu *vcpu, struct kvm_queued_exception *ex);void kvm_handle_exception_payload_quirk (struct kvm_vcpu *vcpu); -void kvm_fixup_and_inject_pf_error (struct kvm_vcpu *vcpu,gva_t gva,u16 error_code); +int kvm_fixup_and_inject_pf_error (struct kvm_vcpu *vcpu,gva_t gva,u16 error_code);int x86_decode_emulated_instruction (struct kvm_vcpu *vcpu,int emulation_type,void *insn,int insn_len); int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 34/45 KVM: x86/mmu: Adjust gva_to_gpa to allow returning an error
In reply to Paolo BonziniIn order to allow the emulator to check memory attributes and return -EFAULT to userspace, INVALID_GPA is not sufficient anymore to distinguish all possible return values of w->gva_to_gpa. Move the guest physical address to an out parameter, and use the return value for errno.
--- Documentation/virt/kvm/api.rst | 4 +- arch/x86/include/asm/kvm_host.h | 6 +-- arch/x86/kvm/mmu.h | 10 ++--- arch/x86/kvm/mmu/mmu.c | 11 +++--- arch/x86/kvm/mmu/paging_tmpl.h | 15 ++++---- arch/x86/kvm/svm/nested.c | 4 +- arch/x86/kvm/vmx/nested.c | 4 +- arch/x86/kvm/vmx/sgx.c | 7 +++- arch/x86/kvm/x86.c | 66 ++++++++++++++++++++++----------- 9 files changed, 80 insertions(+), 47 deletions(-)
Documentation/virt/kvm/api.rst +3 -1
@@ -521,7 +521,9 @@ data structures. :Architectures: x86 :Type: vcpu ioctl :Parameters: struct kvm_translation (in/out) -:Returns: 0 on success, -1 on error +:Returns: 0 on success, -1 on error. errno is EFAULT if + the parameter is invalid but also if guest memory + could not be accessed Translates a virtual address according to the vcpu's current address translation mode.
arch/x86/include/asm/kvm_host.h +3 -3
@@ -490,9 +490,9 @@ struct kvm_pagewalk {void (*inject_page_fault)(struct kvm_vcpu *vcpu,struct x86_exception *fault,bool from_hardware); -gpa_t (*gva_to_gpa)(struct kvm_vcpu *vcpu,struct kvm_pagewalk *w, -gpa_t gva_or_gpa,u64 access, -struct x86_exception *exception); +int (*gva_to_gpa)(struct kvm_vcpu *vcpu,struct kvm_pagewalk *w, +gpa_t gva_or_gpa,u64 access,gpa_t *gpa, +struct x86_exception *exception);union kvm_cpu_role cpu_role;struct kvm_page_format fmt;
arch/x86/kvm/mmu.h +5 -5
@@ -185,12 +185,12 @@ static inline bool kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu,void kvm_mmu_free_roots (struct kvm *kvm,struct kvm_mmu *mmu,ulong roots_to_free);void kvm_mmu_free_guest_mode_roots (struct kvm *kvm,struct kvm_mmu *mmu); -gpa_t kvm_mmu_gva_to_gpa_read (struct kvm_vcpu *vcpu,gva_t gva, +int kvm_mmu_gva_to_gpa_read (struct kvm_vcpu *vcpu,gva_t gva,gpa_t *gpa, +struct x86_exception *exception); +int kvm_mmu_gva_to_gpa_write (struct kvm_vcpu *vcpu,gva_t gva,gpa_t *gpa, +struct x86_exception *exception); +int kvm_mmu_gva_to_gpa_system (struct kvm_vcpu *vcpu,gva_t gva,gpa_t *gpa,struct x86_exception *exception); -gpa_t kvm_mmu_gva_to_gpa_write (struct kvm_vcpu *vcpu,gva_t gva, -struct x86_exception *exception); -gpa_t kvm_mmu_gva_to_gpa_system (struct kvm_vcpu *vcpu,gva_t gva, -struct x86_exception *exception);static inline int kvm_mmu_reload (struct kvm_vcpu *vcpu) {
arch/x86/kvm/mmu/mmu.c +6 -5
@@ -2993,7 +2993,7 @@ bool __kvm_mmu_unprotect_gfn_and_retry(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, goto out;if (!vcpu->arch.mmu->root_role.direct) { - gpa =kvm_mmu_gva_to_gpa_write (vcpu, cr2_or_gpa,NULL ); +kvm_mmu_gva_to_gpa_write (vcpu, cr2_or_gpa, &gpa,NULL );if (gpa ==INVALID_GPA ) goto out; }
@@ -4462,9 +4462,9 @@ void kvm_mmu_sync_prev_roots(struct kvm_vcpu *vcpu)kvm_mmu_free_roots (vcpu->kvm, vcpu->arch.mmu, roots_to_free); } -static gpa_t nonpaging_gva_to_gpa (struct kvm_vcpu *vcpu,struct kvm_pagewalk *w, -gpa_t vaddr,u64 access, -struct x86_exception *exception) +static int nonpaging_gva_to_gpa (struct kvm_vcpu *vcpu,struct kvm_pagewalk *w, +gpa_t vaddr,u64 access,gpa_t *gpa, +struct x86_exception *exception) {if (exception) exception->error_code =0 ;
@@ -4474,8 +4474,9 @@ static gpa_t nonpaging_gva_to_gpa(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,* user-mode address if CR0.PG=0. Therefore *include* ACC_USER_MASK in * the last argument to kvm_translate_gpa (which NPT does not use). */ -return kvm_translate_gpa (vcpu, w, vaddr, access |PFERR_GUEST_FINAL_MASK , + *gpa =kvm_translate_gpa (vcpu, w, vaddr, access |PFERR_GUEST_FINAL_MASK , exception,ACC_ALL ); +return 0 ; }static bool mmio_info_in_cache (struct kvm_vcpu *vcpu,u64 addr,bool direct)
arch/x86/kvm/mmu/paging_tmpl.h +8 -7
@@ -901,14 +901,15 @@ static gpa_t FNAME(get_level1_sp_gpa)(struct kvm_mmu_page *sp) }/* Note, @addr is a GPA when gva_to_gpa() translates an L2 GPA to an L1 GPA. */ -static gpa_t FNAME (gva_to_gpa )(struct kvm_vcpu *vcpu,struct kvm_pagewalk *w, -gpa_t addr,u64 access, -struct x86_exception *exception) +static int FNAME (gva_to_gpa )(struct kvm_vcpu *vcpu,struct kvm_pagewalk *w, +gpa_t addr,u64 access,gpa_t *gpa, +struct x86_exception *exception) {struct guest_walker walker; -gpa_t gpa =INVALID_GPA ;int r; + *gpa =INVALID_GPA ; +#ifndef CONFIG_X86_64 /* A 64-bit GVA should be impossible on 32-bit KVM. */ WARN_ON_ONCE ((addr >>32 ) && w == &vcpu->arch.gva_walk);
@@ -917,12 +918,12 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w, r =FNAME (walk_addr_generic)(&walker, vcpu, w, addr, access);if (r) { - gpa =gfn_to_gpa (walker.gfn); - gpa |= addr & ~PAGE_MASK ; + *gpa =gfn_to_gpa (walker.gfn); + *gpa |= addr & ~PAGE_MASK ; }else if (exception) *exception = walker.fault; -return gpa; +return 0 ; }/*
arch/x86/kvm/svm/nested.c +3 -1
@@ -2155,6 +2155,7 @@ static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, {struct vcpu_svm *svm =to_svm (vcpu);struct kvm_pagewalk *w = &vcpu->arch.ngpa_walk; +gpa_t result;if (WARN_ON_ONCE (!mmu_is_nested (vcpu)))return gpa;
@@ -2163,7 +2164,8 @@ static gpa_t svm_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa,if (!(svm->nested.ctl.misc_ctl &SVM_MISC_ENABLE_GMET )) access |=PFERR_USER_MASK ; -return w->gva_to_gpa (vcpu, w, gpa, access, exception); + w->gva_to_gpa (vcpu, w, gpa, access, &result, exception); +return result; }struct kvm_x86_nested_ops svm_nested_ops__initdata = {
arch/x86/kvm/vmx/nested.c +3 -1
@@ -7483,6 +7483,7 @@ static gpa_t vmx_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, u64 pte_access) {struct kvm_pagewalk *w = &vcpu->arch.ngpa_walk; +gpa_t result;if (WARN_ON_ONCE (!mmu_is_nested (vcpu)))return gpa;
@@ -7495,7 +7496,8 @@ static gpa_t vmx_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa,if ((pte_access &ACC_USER_MASK ) && (access &PFERR_GUEST_FINAL_MASK )) access |=PFERR_USER_MASK ; -return w->gva_to_gpa (vcpu, w, gpa, access, exception); + w->gva_to_gpa (vcpu, w, gpa, access, &result, exception); +return result; }struct kvm_x86_nested_ops vmx_nested_ops__initdata = {
arch/x86/kvm/vmx/sgx.c +5 -2
@@ -76,11 +76,14 @@ static int sgx_gva_to_gpa(struct kvm_vcpu *vcpu, gva_t gva, bool write, gpa_t *gpa) {struct x86_exception ex; +int r;if (write) - *gpa =kvm_mmu_gva_to_gpa_write (vcpu, gva, &ex); + r =kvm_mmu_gva_to_gpa_write (vcpu, gva, gpa, &ex);else - *gpa =kvm_mmu_gva_to_gpa_read (vcpu, gva, &ex); + r =kvm_mmu_gva_to_gpa_read (vcpu, gva, gpa, &ex); +if (r) +return r;if (*gpa ==INVALID_GPA ) {kvm_inject_emulated_page_fault (vcpu, &ex);
arch/x86/kvm/x86.c +44 -22
152 lines not on this page · raw message
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 34/45] KVM: x86/mmu: Adjust gva_to_gpa to allow returning an error
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] KVM: x86: mismatch between KVM_TRANSLATE docs and -EFAULT handling - [Medium] KVM: x86…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] KVM: x86: mismatch between KVM_TRANSLATE docs and -EFAULT handling
- [Medium] KVM: x86: unconditional gva_to_gpa() walk in kvm_fixup_and_inject_pf_error() --
commit 914d25167c243b05b87b4e07ce157df3731ce832
Author: Paolo Bonzini <[email protected]>
KVM: x86/mmu: Adjust gva_to_gpa to allow returning an error
This commit changes the gva_to_gpa callback and its helper wrappers to return an int error code while writing the translated guest physical address to an out parameter, allowing callers to distinguish -EFAULT from INVALID_GPA.
Quoted 15 lines
Documentation/virt/kvm/api.rst +3
-1@@ -521,7 +521,9 @@ data structures. :Architectures: x86 :Type: vcpu ioctl :Parameters: struct kvm_translation (in/out) -:Returns: 0 on success, -1 on error +:Returns: 0 on success, -1 on error. errno is EFAULT if + the parameter is invalid but also if guest memory + could not be accessed Translates a virtual address according to the vcpu's current address translation mode.
[ ... ]
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0
[ ... ]
Quoted 19 lines
arch/x86/kvm/x86.c +8
-3@@ -9341,19 +9354,24 @@ int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu, {unsigned long vaddr = tr->linear_address;gpa_t gpa; -int idx; +int idx, r;vcpu_load (vcpu); - idx =srcu_read_lock (&vcpu->kvm->srcu); - gpa =kvm_mmu_gva_to_gpa_system (vcpu, vaddr,NULL ); + r =kvm_mmu_gva_to_gpa_system (vcpu, vaddr, &gpa,NULL );srcu_read_unlock (&vcpu->kvm->srcu, idx); +vcpu_put (vcpu); + +if (r) { +if (r != -EFAULT ) +return r; + gpa =INVALID_GPA ;
arch/x86/kvm/x86.c int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu, · line 9376
Quoted 10 lines
arch/x86/kvm/x86.c +2
-2+ } + tr->physical_address = gpa; tr->valid = gpa !=INVALID_GPA ; tr->writeable =1 ; tr->usermode =0 ; - -vcpu_put (vcpu);return 0 ; }
[Severity: Medium]Documentation/virt/kvm/api.rst is updated above to state that KVM_TRANSLATE returns -1 with errno set to EFAULT when guest memory could not be accessed.
However, in kvm_arch_vcpu_ioctl_translate(), when kvm_mmu_gva_to_gpa_system() returns -EFAULT, the error is caught and converted into gpa = INVALID_GPA, causing the ioctl to return 0 with tr->valid = 0 instead of returning -EFAULT.
Should kvm_arch_vcpu_ioctl_translate() propagate -EFAULT to userspace, or was the intention to report an invalid translation (tr->valid = 0) and the documentation in api.rst needs to be updated to match?
Quoted 8 lines
arch/x86/kvm/x86.c +2
-0@@ -10815,11 +10833,15 @@ int kvm_fixup_and_inject_pf_error(struct kvm_vcpu *vcpu, gva_t gva, u16 error_co {struct kvm_pagewalk *gva_walk = &vcpu->arch.gva_walk;struct x86_exception fault; +gpa_t gpa; +int r;u64 access = error_code & (PFERR_WRITE_MASK |PFERR_FETCH_MASK |PFERR_USER_MASK );
arch/x86/kvm/x86.c int kvm_fixup_and_inject_pf_error(struct kvm_vcpu *vcpu, gva_t gva, u16 error_co · line 10847
Quoted 10 lines
arch/x86/kvm/x86.c +4
-2-if (!(error_code &PFERR_PRESENT_MASK ) || - gva_walk->gva_to_gpa (vcpu, gva_walk, gva, access, &fault) !=INVALID_GPA ) { + r = gva_walk->gva_to_gpa (vcpu, gva_walk, gva, access, &gpa, &fault); +if (r) +return r; +if (!(error_code &PFERR_PRESENT_MASK ) || gpa !=INVALID_GPA ) {/* * If gva_walk->gva_to_gpa succeeded, the page * tables probably do not match the TLB. Just proceed
[Severity: Medium]
Previously, kvm_fixup_and_inject_pf_error() short-circuited and skipped calling gva_walk->gva_to_gpa() whenever !(error_code & PFERR_PRESENT_MASK) was true.
With gva_walk->gva_to_gpa() hoisted before the PFERR_PRESENT_MASK check, every non-present page fault reaching kvm_fixup_and_inject_pf_error() (for instance, in nested VMX when vmx_need_pf_intercept() causes prepare_vmcs02_rare() to intercept all L2 page faults) now walks the guest and nested page tables even though both gpa and fault are overwritten in the if-block when PFERR_PRESENT_MASK is clear.
Aside from the extra page-table walk, if a guest PTE was concurrently made present or upper-level guest PTEs are walked, FNAME(walk_addr_generic)() can update guest Accessed/Dirty bits or return -EFAULT on a non-present fault.
Should gva_walk->gva_to_gpa() only be called when (error_code & PFERR_PRESENT_MASK) is set?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=34
Paolo Bonzini
maintainerPATCH v4 35/45 KVM: x86/mmu: Propagate errors from nested GPA translation
In reply to Paolo BonziniThe nested translation callbacks currently discard errors returned by gva_to_gpa(). Adjust translate_nested_gpa()'s calling convention similar to what was done to gva_to_gpa(), so that errors can be propagated from page walk as well as from users that directly care about L2 GPAs.
--- arch/x86/include/asm/kvm_host.h | 8 ++++---- arch/x86/kvm/hyperv.c | 6 ++++-- arch/x86/kvm/mmu.h | 18 ++++++++++-------- arch/x86/kvm/mmu/mmu.c | 5 ++--- arch/x86/kvm/mmu/paging_tmpl.h | 12 ++++++------ arch/x86/kvm/regs.c | 8 +++++--- arch/x86/kvm/svm/nested.c | 18 +++++++++--------- arch/x86/kvm/vmx/nested.c | 18 +++++++++--------- arch/x86/kvm/x86.c | 15 +++++++++++---- 9 files changed, 60 insertions(+), 48 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -4
@@ -1780,10 +1780,10 @@ struct kvm_x86_nested_ops { struct kvm_nested_state *kvm_state);bool (*get_nested_state_pages)(struct kvm_vcpu *vcpu);int (*write_log_dirty)(struct kvm_vcpu *vcpu,gpa_t l2_gpa); -gpa_t (*translate_nested_gpa)(struct kvm_vcpu *vcpu,gpa_t gpa, -u64 access, -struct x86_exception *exception, -u64 pte_access); +int (*translate_nested_gpa)(struct kvm_vcpu *vcpu,gpa_t gpa, +u64 access,gpa_t *translated_gpa, +struct x86_exception *exception, +u64 pte_access);int (*enable_evmcs)(struct kvm_vcpu *vcpu,uint16_t *vmcs_version);
arch/x86/kvm/hyperv.c +4 -2
@@ -2119,8 +2119,10 @@ static s64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)/* Slow direct hypercalls from L2 provide a nested GPA. */ if (!hc->fast) { - hc->ingpa =kvm_translate_gpa (vcpu, &vcpu->arch.gva_walk, hc->ingpa, -PFERR_GUEST_FINAL_MASK ,NULL ,0 ); + r =kvm_translate_gpa (vcpu, &vcpu->arch.gva_walk, hc->ingpa, +PFERR_GUEST_FINAL_MASK , &hc->ingpa,NULL ,0 ); +if (r) +return r;if (unlikely (hc->ingpa ==INVALID_GPA ))return HV_STATUS_INVALID_HYPERCALL_INPUT ; }
arch/x86/kvm/mmu.h +10 -8
@@ -377,15 +377,17 @@ static inline bool mmu_is_nested(struct kvm_vcpu *vcpu)return vcpu->arch.mmu == &vcpu->arch.guest_mmu; } -static inline gpa_t kvm_translate_gpa (struct kvm_vcpu *vcpu, -struct kvm_pagewalk *w, -gpa_t gpa,u64 access, -struct x86_exception *exception, -u64 pte_access) +static inline int kvm_translate_gpa (struct kvm_vcpu *vcpu, +struct kvm_pagewalk *w, +gpa_t gpa,u64 access,gpa_t *translated_gpa, +struct x86_exception *exception, +u64 pte_access) { -if (!mmu_is_nested (vcpu) || w == &vcpu->arch.ngpa_walk) -return gpa; -return kvm_nested_call (translate_nested_gpa)(vcpu, gpa, access, +if (!mmu_is_nested (vcpu) || w == &vcpu->arch.ngpa_walk) { + *translated_gpa = gpa; +return 0 ; + } +return kvm_nested_call (translate_nested_gpa)(vcpu, gpa, access, translated_gpa, exception, pte_access); }
arch/x86/kvm/mmu/mmu.c +2 -3
@@ -4474,9 +4474,8 @@ static int nonpaging_gva_to_gpa(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,* user-mode address if CR0.PG=0. Therefore *include* ACC_USER_MASK in * the last argument to kvm_translate_gpa (which NPT does not use). */ - *gpa =kvm_translate_gpa (vcpu, w, vaddr, access |PFERR_GUEST_FINAL_MASK , - exception,ACC_ALL ); -return 0 ; +return kvm_translate_gpa (vcpu, w, vaddr, access |PFERR_GUEST_FINAL_MASK , + gpa, exception,ACC_ALL ); }static bool mmio_info_in_cache (struct kvm_vcpu *vcpu,u64 addr,bool direct)
arch/x86/kvm/mmu/paging_tmpl.h +6 -6
@@ -398,9 +398,9 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, walker->table_gfn[walker->level -1 ] = table_gfn; walker->pte_gpa[walker->level -1 ] = pte_gpa; - real_gpa =kvm_translate_gpa (vcpu, w,gfn_to_gpa (table_gfn), - nested_access |PFERR_GUEST_PAGE_MASK , - &walker->fault,0 ); +kvm_translate_gpa (vcpu, w,gfn_to_gpa (table_gfn), + nested_access |PFERR_GUEST_PAGE_MASK , + &real_gpa, &walker->fault,0 );if (unlikely (real_gpa ==INVALID_GPA ))return 0 ;
@@ -458,9 +458,9 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, gfn +=pse36_gfn_delta (pte);#endif - real_gpa =kvm_translate_gpa (vcpu, w,gfn_to_gpa (gfn), - access |PFERR_GUEST_FINAL_MASK , - &walker->fault, walker->pte_access); +kvm_translate_gpa (vcpu, w,gfn_to_gpa (gfn), + access |PFERR_GUEST_FINAL_MASK , &real_gpa, + &walker->fault, walker->pte_access);if (real_gpa ==INVALID_GPA )return 0 ;
arch/x86/kvm/regs.c +5 -3
@@ -165,9 +165,11 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3)* If the MMU is nested, CR3 holds an L2 GPA and needs to be translated * to an L1 GPA. */ - real_gpa =kvm_translate_gpa (vcpu, w,gfn_to_gpa (pdpt_gfn), -PFERR_USER_MASK |PFERR_WRITE_MASK | -PFERR_GUEST_PAGE_MASK ,NULL ,0 ); + ret =kvm_translate_gpa (vcpu, w,gfn_to_gpa (pdpt_gfn), +PFERR_USER_MASK |PFERR_WRITE_MASK |PFERR_GUEST_PAGE_MASK , + &real_gpa,NULL ,0 ); +if (ret) +return ret;if (real_gpa ==INVALID_GPA )return 0 ;
arch/x86/kvm/svm/nested.c +9 -9
@@ -2148,24 +2148,24 @@ static bool svm_get_nested_state_pages(struct kvm_vcpu *vcpu)return true; } -static gpa_t svm_translate_nested_gpa (struct kvm_vcpu *vcpu,gpa_t gpa, -u64 access, -struct x86_exception *exception, -u64 pte_access) +static int svm_translate_nested_gpa (struct kvm_vcpu *vcpu,gpa_t gpa, +u64 access,gpa_t *translated_gpa, +struct x86_exception *exception, +u64 pte_access) {struct vcpu_svm *svm =to_svm (vcpu);struct kvm_pagewalk *w = &vcpu->arch.ngpa_walk; -gpa_t result; -if (WARN_ON_ONCE (!mmu_is_nested (vcpu))) -return gpa; +if (WARN_ON_ONCE (!mmu_is_nested (vcpu))) { + *translated_gpa = gpa; +return 0 ; + }/* Non-GMET walks are always user-walks */ if (!(svm->nested.ctl.misc_ctl &SVM_MISC_ENABLE_GMET )) access |=PFERR_USER_MASK ; - w->gva_to_gpa (vcpu, w, gpa, access, &result, exception); -return result; +return w->gva_to_gpa (vcpu, w, gpa, access, translated_gpa, exception); }struct kvm_x86_nested_ops svm_nested_ops__initdata = {
arch/x86/kvm/vmx/nested.c +9 -9
@@ -7477,16 +7477,17 @@ __init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *)) } -static gpa_t vmx_translate_nested_gpa (struct kvm_vcpu *vcpu,gpa_t gpa, -u64 access, -struct x86_exception *exception, -u64 pte_access) +static int vmx_translate_nested_gpa (struct kvm_vcpu *vcpu,gpa_t gpa, +u64 access,gpa_t *translated_gpa, +struct x86_exception *exception, +u64 pte_access) {struct kvm_pagewalk *w = &vcpu->arch.ngpa_walk; -gpa_t result; -if (WARN_ON_ONCE (!mmu_is_nested (vcpu))) -return gpa; +if (WARN_ON_ONCE (!mmu_is_nested (vcpu))) { + *translated_gpa = gpa; +return 0 ; + }/* * MBEC differentiates based on the effective U/S bit of
@@ -7496,8 +7497,7 @@ static gpa_t vmx_translate_nested_gpa(struct kvm_vcpu *vcpu, gpa_t gpa,if ((pte_access &ACC_USER_MASK ) && (access &PFERR_GUEST_FINAL_MASK )) access |=PFERR_USER_MASK ; - w->gva_to_gpa (vcpu, w, gpa, access, &result, exception); -return result; +return w->gva_to_gpa (vcpu, w, gpa, access, translated_gpa, exception); }struct kvm_x86_nested_ops vmx_nested_ops__initdata = {
arch/x86/kvm/x86.c +11 -4
@@ -7315,10 +7315,17 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, *ret = -KVM_EFAULT ;while (offset <sizeof (clock_pairing)) { -gpa_t gpa =kvm_translate_gpa (vcpu, &vcpu->arch.gva_walk, paddr + offset, -PFERR_WRITE_MASK |PFERR_GUEST_FINAL_MASK ,NULL ,0 ); -size_t len =min_t (size_t,sizeof (clock_pairing) - offset, -PAGE_SIZE -offset_in_page (gpa)); +gpa_t gpa; +size_t len; +int r; + + r =kvm_translate_gpa (vcpu, &vcpu->arch.gva_walk, paddr + offset, +PFERR_WRITE_MASK |PFERR_GUEST_FINAL_MASK , + &gpa,NULL ,0 ); +if (r) +return r; + len =min_t (size_t,sizeof (clock_pairing) - offset, +PAGE_SIZE -offset_in_page (gpa));if (gpa ==INVALID_GPA )return 0 ;
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 36/45 KVM: x86: Propagate memory faults through load_pdptrs()
In reply to Paolo Bonziniload_pdptrs() returns a 0/1 value, and therefore its callers do not distinguish memory attribute errors (including from nested GPA translation) from other failures. Change them to propagate negative errno.
--- arch/x86/kvm/regs.c | 43 ++++++++++++++++++++++++++++----------- arch/x86/kvm/svm/nested.c | 10 +++++---- arch/x86/kvm/svm/svm.c | 4 ++-- arch/x86/kvm/vmx/nested.c | 6 ++++-- 4 files changed, 43 insertions(+), 20 deletions(-)
arch/x86/kvm/regs.c +31 -12
@@ -176,7 +176,12 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3)/* Note the offset, PDPTRs are 32 byte aligned when using PAE paging. */ ret =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (real_gpa), pdpte, cr3 &GENMASK (11 ,5 ),sizeof (pdpte)); -if (ret <0 ) +if (ret == -EFAULT ) { +kvm_prepare_memory_fault_exit (vcpu, real_gpa &PAGE_MASK ,PAGE_SIZE , + false, false, false); +return ret; + } +if (ret)return 0 ;for (i =0 ; i <ARRAY_SIZE (pdpte); ++i) {
@@ -264,6 +269,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_post_set_cr0);int kvm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0) {unsigned long old_cr0 =kvm_read_cr0 (vcpu); +int r;if (!kvm_is_valid_cr0 (vcpu, cr0))return 1 ;
@@ -286,9 +292,11 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) }#endif if (!(vcpu->arch.efer &EFER_LME ) && (cr0 &X86_CR0_PG ) && -is_pae (vcpu) && ((cr0 ^ old_cr0) &X86_CR0_PDPTR_BITS ) && - !load_pdptrs (vcpu,kvm_read_cr3 (vcpu))) -return 1 ; +is_pae (vcpu) && ((cr0 ^ old_cr0) &X86_CR0_PDPTR_BITS )) { + r =load_pdptrs (vcpu,kvm_read_cr3 (vcpu)); +if (r <=0 ) +return r ?:1 ; + }if (!(cr0 &X86_CR0_PG ) && (is_64_bit_mode (vcpu) ||kvm_is_cr4_bit_set (vcpu,X86_CR4_PCIDE )))
@@ -315,6 +323,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) {bool skip_tlb_flush = false;unsigned long pcid =0 ; +int r;#ifdef CONFIG_X86_64 if (kvm_is_cr4_bit_set (vcpu,X86_CR4_PCIDE )) { skip_tlb_flush = cr3 &X86_CR3_PCID_NOFLUSH ;
@@ -335,8 +344,11 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)if (!kvm_vcpu_is_legal_cr3 (vcpu, cr3))return 1 ; -if (is_pae_paging (vcpu) && !load_pdptrs (vcpu, cr3)) -return 1 ; +if (is_pae_paging (vcpu)) { + r =load_pdptrs (vcpu, cr3); +if (r <=0 ) +return r ?:1 ; + }if (cr3 !=kvm_read_cr3 (vcpu))kvm_mmu_new_pgd (vcpu, cr3);
@@ -411,6 +423,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_post_set_cr4);int kvm_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) {unsigned long old_cr4 =kvm_read_cr4 (vcpu); +int r;if (!kvm_is_valid_cr4 (vcpu, cr4))return 1 ;
@@ -420,10 +433,12 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)return 1 ;if ((cr4 ^ old_cr4) &X86_CR4_LA57 )return 1 ; - }else if (is_paging (vcpu) && (cr4 &X86_CR4_PAE ) - && ((cr4 ^ old_cr4) &X86_CR4_PDPTR_BITS ) - && !load_pdptrs (vcpu,kvm_read_cr3 (vcpu))) -return 1 ; + }else if (is_paging (vcpu) && (cr4 &X86_CR4_PAE ) && + ((cr4 ^ old_cr4) &X86_CR4_PDPTR_BITS )) { + r =load_pdptrs (vcpu,kvm_read_cr3 (vcpu)); +if (r <=0 ) +return r ?:1 ; + }if ((cr4 &X86_CR4_PCIDE ) && !(old_cr4 &X86_CR4_PCIDE )) {/* PCID can not be enabled when cr3[11:0]!=000H or EFER.LMA=0 */
@@ -579,7 +594,7 @@ static bool kvm_is_valid_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs)static int __set_sregs_common (struct kvm_vcpu *vcpu,struct kvm_sregs *sregs,int *mmu_reset_needed,bool update_pdptrs) { -int idx; +int idx, r;struct desc_ptr dt;if (!kvm_is_valid_sregs (vcpu, sregs))
@@ -616,7 +631,11 @@ static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs,if (update_pdptrs) { idx =srcu_read_lock (&vcpu->kvm->srcu);if (is_pae_paging (vcpu)) { -load_pdptrs (vcpu,kvm_read_cr3 (vcpu)); + r =load_pdptrs (vcpu,kvm_read_cr3 (vcpu)); +if (r <0 ) { +srcu_read_unlock (&vcpu->kvm->srcu, idx); +return r; + } *mmu_reset_needed =1 ; }srcu_read_unlock (&vcpu->kvm->srcu, idx);
arch/x86/kvm/svm/nested.c +6 -4
@@ -720,7 +720,7 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,if (reload_pdptrs &&is_pae_paging (vcpu)) {if (nested_npt)kvm_register_mark_for_reload (vcpu,VCPU_REG_PDPTR ); -else if (CC (!load_pdptrs (vcpu, cr3))) +else if (CC (load_pdptrs (vcpu, cr3) <=0 ))return -EINVAL ; }
@@ -2132,11 +2132,13 @@ static bool svm_get_nested_state_pages(struct kvm_vcpu *vcpu)* nGPA translation. For !nNPT, reload eagerly unless userspace * already supplied authoritative PDPTRs via KVM_SET_SREGS2. */ -if (nested_npt_enabled (to_svm (vcpu))) +if (nested_npt_enabled (to_svm (vcpu))) {kvm_register_mark_for_reload (vcpu,VCPU_REG_PDPTR ); -else if (!vcpu->arch.pdptrs_from_userspace && -CC (!load_pdptrs (vcpu, vcpu->arch.cr3))) + }else if (!vcpu->arch.pdptrs_from_userspace && +CC (load_pdptrs (vcpu, vcpu->arch.cr3) <=0 )) { +kvm_prepare_emulation_failure_exit (vcpu);return false; + } }if (!nested_svm_merge_msrpm (vcpu))
arch/x86/kvm/svm/svm.c +2 -2
@@ -1587,8 +1587,8 @@ static void svm_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)* When !npt_enabled, vcpu->pdptrs[] is already available since * it is always updated per SDM when moving to CRs. */ -if (npt_enabled) -load_pdptrs (vcpu,kvm_read_cr3 (vcpu)); +if (npt_enabled &&load_pdptrs (vcpu,kvm_read_cr3 (vcpu)) <0 ) +kvm_vm_dead (vcpu->kvm);break ;default :KVM_BUG_ON (1 , vcpu->kvm);
arch/x86/kvm/vmx/nested.c +4 -2
@@ -1240,7 +1240,7 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,* must not be dereferenced. */ if (reload_pdptrs && !nested_ept &&is_pae_paging (vcpu) && -CC (!load_pdptrs (vcpu, cr3))) { +CC (load_pdptrs (vcpu, cr3) <=0 )) { *entry_failure_code =ENTRY_FAIL_PDPTE ;return -EINVAL ; }
@@ -3454,8 +3454,10 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)* the guest CR3 might be restored prior to setting the nested * state which can lead to a load of wrong PDPTRs. */ -if (CC (!load_pdptrs (vcpu, vcpu->arch.cr3))) +if (CC (load_pdptrs (vcpu, vcpu->arch.cr3) <=0 )) { +kvm_prepare_emulation_failure_exit (vcpu);return false; + } }
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 36/45] KVM: x86: Propagate memory faults through load_pdptrs()
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM: x86: unconditional memory fault exit on read failure in load_pdptrs() - [High] KVM: SVM…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM: x86: unconditional memory fault exit on read failure in load_pdptrs()
- [High] KVM: SVM: svm_cache_reg() kills VM when load_pdptrs() returns negative errno --
commit 780e6889d183774ef90a40deaa0094e70e71d6a2
Author: Paolo Bonzini <[email protected]>
KVM: x86: Propagate memory faults through load_pdptrs()
This commit changes load_pdptrs() and its callers to propagate negative errno values so memory attribute faults can be distinguished from other PDPTR load failures.
Quoted 6 lines
arch/x86/kvm/regs.c +0
-0@@ -176,7 +176,12 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3)/* Note the offset, PDPTRs are 32 byte aligned when using PAE paging. */
arch/x86/kvm/regs.c int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3) · line 185
Quoted 10 lines
arch/x86/kvm/regs.c +6
-1ret =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (real_gpa), pdpte, cr3 &GENMASK (11 ,5 ),sizeof (pdpte)); -if (ret <0 ) +if (ret == -EFAULT ) { +kvm_prepare_memory_fault_exit (vcpu, real_gpa &PAGE_MASK ,PAGE_SIZE , + false, false, false); +return ret; + } +if (ret)return 0 ;
[Severity: High]
Does load_pdptrs() misclassify every read failure from kvm_vcpu_read_guest_page() as a memory attribute fault?
Looking at virt/kvm/kvm_main.c, kvm_vcpu_read_guest_page() calls __kvm_read_guest_page(), which only ever returns 0 or -EFAULT:
virt/kvm/kvm_main.c:__kvm_read_guest_page() {
addr = gfn_to_hva_memslot_prot(slot, gfn, NULL);
if (kvm_is_error_hva(addr))
return -EFAULT;
attrs = kvm_get_vm_memory_attributes(kvm, gfn);
if (attrs & attr)
return -EFAULT;
r = __copy_from_user(data, (void __user *)addr + offset, len);
if (r)
return -EFAULT;
return 0;
}Because __kvm_read_guest_page() returns -EFAULT for all failure cases (including when real_gpa is unmapped and has no backing memslot), the following check in load_pdptrs():
if (ret) return 0;
becomes unreachable dead code, and any unmapped PAE CR3 GPA prepares a KVM_EXIT_MEMORY_FAULT and returns -EFAULT instead of 0.
Doesn't this cause kvm_set_cr0(), kvm_set_cr3(), and kvm_set_cr4() to exit to userspace with KVM_EXIT_MEMORY_FAULT instead of returning 1 to inject #GP(0) when a PAE guest loads an unmapped GPA into CR3, and cause __set_sregs_common() (via KVM_SET_SREGS or KVM_SET_SREGS2 without KVM_SREGS2_FLAGS_PDPTRS_VALID) to fail with -EFAULT when CR3 is not backed by a memslot?
Should load_pdptrs() check kvm_prepare_memory_fault_exit_for_range() (or !kvm_mem_attributes_may_read_gfn()) before preparing KVM_EXIT_MEMORY_FAULT and returning -EFAULT, similar to kvm_read_guest_virt_helper() and emulator_read_write_onepage()?
[ ... ]
Quoted 5 lines
arch/x86/kvm/svm/svm.c +0
-0
arch/x86/kvm/svm/svm.c static void svm_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) · line 1598
Quoted 10 lines
arch/x86/kvm/svm/svm.c +2
-2@@ -1591,8 +1591,8 @@ static void svm_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)* When !npt_enabled, vcpu->pdptrs[] is already available since * it is always updated per SDM when moving to CRs. */ -if (npt_enabled) -load_pdptrs (vcpu,kvm_read_cr3 (vcpu)); +if (npt_enabled &&load_pdptrs (vcpu,kvm_read_cr3 (vcpu)) <0 ) +kvm_vm_dead (vcpu->kvm);break ;default :KVM_BUG_ON (1 , vcpu->kvm);
[Severity: High]
Can an L1 or L2 guest in PAE paging mode trigger kvm_vm_dead() here on AMD SVM with NPT enabled?
When npt_enabled is true, init_vmcb() clears INTERCEPT_CR3_WRITE, so guest writes to CR3 are not intercepted. On VM-exit, svm_vcpu_run() copies svm->vmcb->save.cr3 to vcpu->arch.cr3 and clears VCPU_REG_PDPTR from vcpu->arch.regs_avail (and on nested VMRUN with nested NPT enabled, nested_svm_load_cr3() sets vcpu->arch.cr3 and calls kvm_register_mark_for_reload(vcpu, VCPU_REG_PDPTR)).
If the guest sets CR3 in PAE mode to an unmapped GPA or a GPA with KVM_MEMORY_ATTRIBUTE_NR (or if a nested GPA walk in kvm_translate_gpa() hits a page with KVM_MEMORY_ATTRIBUTE_NR) and then triggers a KVM page walk or userspace calls KVM_GET_SREGS2:
kvm_vcpu_ioctl_x86_get_sregs2() / walk_addr_generic()
-> kvm_pdptr_read()
-> svm_cache_reg(vcpu, VCPU_REG_PDPTR)
-> load_pdptrs()won't load_pdptrs() return -EFAULT and cause svm_cache_reg() to terminate the VM via kvm_vm_dead() instead of treating an unmapped GPA as invalid PDPTRs or propagating KVM_EXIT_MEMORY_FAULT to userspace?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=36
Paolo Bonzini
maintainerPATCH v4 37/45 KVM: x86/mmu: Issue memory fault exit if walk failed due to memory attribute
In reply to Paolo BonziniDuring execution, page table entries are subject to memory protection simply because their GPA is obtained with an EPT walk. During emulation, the checks need to be done by hand and result in an -EFAULT userspace exit.
However, if a guest PTE is read-only, do not go all the way out to userspace and just stomp out the write. This is consistent with the way KVM handles read-only memory slots during emulation, though admittedly inconsistent with everything else: NPT for example always wants nested page table entries for the guest page tables to be writable, and will fault.
--- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/mmu/mmu.c | 6 +++-- arch/x86/kvm/mmu/paging_tmpl.h | 40 ++++++++++++++++++++++++++------- arch/x86/kvm/regs.h | 6 +++++ arch/x86/kvm/svm/nested.c | 16 ++++++------- arch/x86/kvm/vmx/nested.c | 2 +- 6 files changed, 52 insertions(+), 20 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -486,7 +486,7 @@ struct kvm_page_fault;*/ struct kvm_pagewalk {unsigned long (*get_guest_pgd)(struct kvm_vcpu *vcpu); -u64 (*get_pdptr)(struct kvm_vcpu *vcpu,int index); +int (*get_pdptr)(struct kvm_vcpu *vcpu,int index,u64 *pdptr);void (*inject_page_fault)(struct kvm_vcpu *vcpu,struct x86_exception *fault,bool from_hardware);
arch/x86/kvm/mmu/mmu.c +4 -2
@@ -4204,7 +4204,9 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu)*/ if (w->cpu_role.base.level ==PT32E_ROOT_LEVEL ) {for (i =0 ; i <4 ; ++i) { - pdptrs[i] = w->get_pdptr (vcpu, i); + r = w->get_pdptr (vcpu, i, &pdptrs[i]); +if (r) +return r;if (!(pdptrs[i] &PT_PRESENT_MASK ))continue ;
@@ -6211,7 +6213,7 @@ static void init_kvm_page_walk(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w, w->cpu_role.as_u64 = cpu_role.as_u64; w->inject_page_fault = kvm_inject_page_fault; - w->get_pdptr = kvm_pdptr_read; + w->get_pdptr = kvm_pdptr_get; w->get_guest_pgd = get_guest_cr3;if (!is_cr0_pg (w))
arch/x86/kvm/mmu/paging_tmpl.h +32 -8
@@ -89,6 +89,7 @@ struct guest_walker {unsigned int pt_access[PT_MAX_FULL_LEVELS ];unsigned int pte_access;gfn_t gfn; +int error;struct x86_exception fault; };
@@ -342,6 +343,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,trace_kvm_mmu_pagetable_walk (addr, access); retry_walk: + walker->error =0 ; walker->level = w->cpu_role.base.level; pte =kvm_mmu_get_guest_pgd (vcpu, w); have_ad =PT_HAVE_ACCESSED_DIRTY (w);
@@ -349,7 +351,9 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,#if PTTYPE ==64 walk_nx_mask =1ULL <<PT64_NX_SHIFT ;if (walker->level ==PT32E_ROOT_LEVEL ) { - pte = w->get_pdptr (vcpu, (addr >>30 ) &3 ); + walker->error = w->get_pdptr (vcpu, (addr >>30 ) &3 , &pte); +if (walker->error) + goto error;trace_kvm_mmu_paging_element (pte, walker->level);if (!FNAME (is_present_gpte)(w, pte)) goto error;
@@ -398,9 +402,11 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, walker->table_gfn[walker->level -1 ] = table_gfn; walker->pte_gpa[walker->level -1 ] = pte_gpa; -kvm_translate_gpa (vcpu, w,gfn_to_gpa (table_gfn), - nested_access |PFERR_GUEST_PAGE_MASK , - &real_gpa, &walker->fault,0 ); + walker->error =kvm_translate_gpa (vcpu, w,gfn_to_gpa (table_gfn), + nested_access |PFERR_GUEST_PAGE_MASK , + &real_gpa, &walker->fault,0 ); +if (walker->error) +return 0 ;if (unlikely (real_gpa ==INVALID_GPA ))return 0 ;
@@ -414,6 +420,16 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (kvm_is_error_hva (host_addr))) goto error; +if (!kvm_mem_attributes_may_read_gfn (vcpu->kvm,gpa_to_gfn (real_gpa))) { + walker->error = -EFAULT ; +kvm_prepare_memory_fault_exit (vcpu, real_gpa &PAGE_MASK , +PAGE_SIZE , false, false, false); + goto error; + } + +if (!kvm_mem_attributes_may_write_gfn (vcpu->kvm,gpa_to_gfn (real_gpa))) + walker->pte_writable[walker->level -1 ] = false; + ptep_user = (pt_element_t __user *)((void *)host_addr + offset);if (unlikely (get_user (pte, ptep_user))) goto error;
@@ -458,9 +474,11 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, gfn +=pse36_gfn_delta (pte);#endif -kvm_translate_gpa (vcpu, w,gfn_to_gpa (gfn), - access |PFERR_GUEST_FINAL_MASK , &real_gpa, - &walker->fault, walker->pte_access); + walker->error =kvm_translate_gpa (vcpu, w,gfn_to_gpa (gfn), + access |PFERR_GUEST_FINAL_MASK , &real_gpa, + &walker->fault, walker->pte_access); +if (walker->error) +return 0 ;if (real_gpa ==INVALID_GPA )return 0 ;
@@ -823,6 +841,9 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault* The page is not mapped by the guest. Let the guest handle it. */ if (!r) { +if (walker.error) +return walker.error; +if (!fault->prefetch)__kvm_inject_emulated_page_fault (vcpu, &walker.fault, true);
@@ -920,8 +941,11 @@ static int FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, struct kvm_pagewalk *w,if (r) { *gpa =gfn_to_gpa (walker.gfn); *gpa |= addr & ~PAGE_MASK ; - }else if (exception) + }else if (walker.error) { +return walker.error; + }else if (exception) { *exception = walker.fault; + }return 0 ; }
arch/x86/kvm/regs.h +6 -0
@@ -319,6 +319,12 @@ static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index)return vcpu->arch.pdptrs[index]; } +static inline int kvm_pdptr_get (struct kvm_vcpu *vcpu,int index,u64 *pdptr) +{ + *pdptr =kvm_pdptr_read (vcpu, index); +return 0 ; +} +static inline void kvm_pdptr_write (struct kvm_vcpu *vcpu,int index,u64 value) { vcpu->arch.pdptrs[index] = value;
arch/x86/kvm/svm/nested.c +8 -8
@@ -71,22 +71,22 @@ static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu,nested_svm_vmexit (svm); } -static u64 nested_svm_get_tdp_pdptr (struct kvm_vcpu *vcpu,int index) +static int nested_svm_get_tdp_pdptr (struct kvm_vcpu *vcpu,int index,u64 *pdptr) {struct vcpu_svm *svm =to_svm (vcpu);u64 cr3 = svm->nested.ctl.nested_cr3; -u64 pdpte; -int ret; +int r;/* * Note, nCR3 is "assumed" to be 32-byte aligned, i.e. the CPU ignores * nCR3[4:0] when loading PDPTEs from memory. */ - ret =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (cr3), &pdpte, - (cr3 &GENMASK (11 ,5 )) + index *8 ,8 ); -if (ret) -return 0 ; -return pdpte; + r =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (cr3), pdptr, + (cr3 &GENMASK (11 ,5 )) + index *8 ,8 ); +if (r == -EFAULT ) +kvm_prepare_memory_fault_exit (vcpu, cr3 &PAGE_MASK ,PAGE_SIZE , + false, false, false); +return r; }static unsigned long nested_svm_get_tdp_cr3 (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/nested.c +1 -1
@@ -520,7 +520,7 @@ static void nested_ept_init_mmu_context(struct kvm_vcpu *vcpu) vcpu->arch.mmu = &vcpu->arch.guest_mmu;nested_ept_new_eptp (vcpu); vcpu->arch.ngpa_walk.get_guest_pgd = nested_ept_get_eptp; - vcpu->arch.ngpa_walk.get_pdptr = kvm_pdptr_read; + vcpu->arch.ngpa_walk.get_pdptr = kvm_pdptr_get; vcpu->arch.ngpa_walk.inject_page_fault = nested_ept_inject_page_fault; }
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 37/45] KVM: x86/mmu: Issue memory fault exit if walk failed due to memory attribute
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM: x86/svm: spurious memory fault exit in nested_svm_get_tdp_pdptr() - [High] KVM: x86: missing…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM: x86/svm: spurious memory fault exit in nested_svm_get_tdp_pdptr()
- [High] KVM: x86: missing PDPTR error propagation in kvm_pdptr_get() kills SVM VM --
commit 47564fc6c68d998cb57e74371e6dfb0d76a0b22c
Author: Paolo Bonzini <[email protected]>
KVM: x86/mmu: Issue memory fault exit if walk failed due to memory attribute
This commit updates the guest page table walker to check memory attributes on page table pages and return -EFAULT with a KVM_EXIT_MEMORY_FAULT exit when a walk fails due to unreadable memory attributes, while treating read-only attributes on guest PTEs by suppressing A/D bit writes. It also updates struct kvm_pagewalk's get_pdptr callback to return an error code so PAE PDPTR reads can propagate faults.
arch/x86/kvm/regs.h static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index) · line 326
Quoted 13 lines
arch/x86/kvm/regs.h +5
-0@@ -319,6 +319,12 @@ static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index)return vcpu->arch.pdptrs[index]; } +static inline int kvm_pdptr_get (struct kvm_vcpu *vcpu,int index,u64 *pdptr) +{ + *pdptr =kvm_pdptr_read (vcpu, index); +return 0 ; +}
[Severity: High]
Can a 32-bit PAE guest on AMD SVM with NPT enabled terminate the VM via kvm_pdptr_get() instead of propagating -EFAULT to the page table walker?
In init_kvm_page_walk(), w->get_pdptr is wired to kvm_pdptr_get(), which calls kvm_pdptr_read() and unconditionally returns 0:
FNAME(walk_addr_generic)() / mmu_alloc_shadow_roots()
-> kvm_pdptr_get()
-> kvm_pdptr_read()
-> svm_cache_reg(vcpu, VCPU_REG_PDPTR)On SVM with npt_enabled, VCPU_REG_PDPTR is lazily loaded on demand in svm_cache_reg():
arch/x86/kvm/svm/svm.c:svm_cache_reg() {
...
case VCPU_REG_PDPTR:
/*
* When !npt_enabled, vcpu->pdptrs[] is already available since
* it is always updated per SDM when moving to CRs.
*/
if (npt_enabled && load_pdptrs(vcpu, kvm_read_cr3(vcpu)) < 0)
kvm_vm_dead(vcpu->kvm);
break;
}And load_pdptrs() returns -EFAULT when translating CR3 or reading the guest page fails:
arch/x86/kvm/regs.c:load_pdptrs() {
...
ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(real_gpa), pdpte,
cr3 & GENMASK(11, 5), sizeof(pdpte));
if (ret == -EFAULT) {
kvm_prepare_memory_fault_exit(vcpu, real_gpa & PAGE_MASK, PAGE_SIZE,
false, false, false);
return ret;
}
}If the guest's CR3 points to a GPA with KVM_MEMORY_ATTRIBUTE_NR (or an unmapped GPA outside any visible memslot) when KVM walks its page tables, load_pdptrs() returns -EFAULT, causing svm_cache_reg() to call kvm_vm_dead() while kvm_pdptr_get() returns 0 with a stale vcpu->arch.pdptrs[index] value.
Quoted 23 lines
arch/x86/kvm/svm/nested.c +2
-5@@ -71,22 +71,22 @@ static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu,nested_svm_vmexit (svm); } -static u64 nested_svm_get_tdp_pdptr (struct kvm_vcpu *vcpu,int index) +static int nested_svm_get_tdp_pdptr (struct kvm_vcpu *vcpu,int index,u64 *pdptr) {struct vcpu_svm *svm =to_svm (vcpu);u64 cr3 = svm->nested.ctl.nested_cr3; -u64 pdpte; -int ret; +int r;/* * Note, nCR3 is "assumed" to be 32-byte aligned, i.e. the CPU ignores * nCR3[4:0] when loading PDPTEs from memory. */ - ret =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (cr3), &pdpte, - (cr3 &GENMASK (11 ,5 )) + index *8 ,8 );
arch/x86/kvm/svm/nested.c static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu, · line 90
Quoted 10 lines
arch/x86/kvm/svm/nested.c +6
-3-if (ret) -return 0 ; -return pdpte; + r =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (cr3), pdptr, + (cr3 &GENMASK (11 ,5 )) + index *8 ,8 ); +if (r == -EFAULT ) +kvm_prepare_memory_fault_exit (vcpu, cr3 &PAGE_MASK ,PAGE_SIZE , + false, false, false); +return r; }
[Severity: High]
Won't nested_svm_get_tdp_pdptr() cause a spurious KVM_EXIT_MEMORY_FAULT exit to host userspace instead of injecting an SVM_EXIT_NPF into L1 when L1 sets nested_cr3 to an unmapped GPA in 32-bit PAE NPT mode?
kvm_vcpu_read_guest_page() calls __kvm_read_guest_page(), which returns -EFAULT for all failure modes, including missing or invalid memslots and __copy_from_user() failures, not only memory attribute violations:
virt/kvm/kvm_main.c:__kvm_read_guest_page() {
int r;
unsigned long addr;
unsigned long attrs;
addr = gfn_to_hva_memslot_prot(slot, gfn, NULL);
if (kvm_is_error_hva(addr))
return -EFAULT;
attrs = kvm_get_vm_memory_attributes(kvm, gfn);
if (attrs & attr)
return -EFAULT;
r = __copy_from_user(data, (void __user *)addr + offset, len);
if (r)
return -EFAULT;
return 0;
}When L1 configures 32-bit PAE NPT with nested_cr3 pointing outside any visible memslot, mmu_alloc_shadow_roots() sees !kvm_vcpu_is_visible_gfn() and installs kvm_mmu_get_dummy_root(). When L2 subsequently faults on the dummy root (or KVM translates an L2 GPA via ngpa_walk.gva_to_gpa), FNAME(walk_addr_generic) calls w->get_pdptr():
arch/x86/kvm/mmu/paging_tmpl.h:FNAME(walk_addr_generic)() {
...
if (walker->level == PT32E_ROOT_LEVEL) {
walker->error = w->get_pdptr(vcpu, (addr >> 30) & 3, &pte);
if (walker->error)
goto error;
trace_kvm_mmu_paging_element(pte, walker->level);
if (!FNAME(is_present_gpte)(w, pte))
goto error;
--walker->level;
}
}Previously, nested_svm_get_tdp_pdptr() returned 0 on a failed read, which walk_addr_generic() treated as a non-present PDPTE with walker->error == 0, injecting SVM_EXIT_NPF into L1 (matching how walk_addr_generic() handles !kvm_is_visible_memslot(slot) and kvm_is_error_hva(host_addr) at all other page table levels).
With this change, nested_svm_get_tdp_pdptr() returns -EFAULT for any read failure (and load_pdptrs() in arch/x86/kvm/regs.c has the same issue), which sets walker->error = -EFAULT and causes FNAME(page_fault) and FNAME(gva_to_gpa) to exit to host userspace with KVM_EXIT_MEMORY_FAULT. Should nested_svm_get_tdp_pdptr() only prepare a memory fault exit and return -EFAULT when !kvm_mem_attributes_may_read_gfn(vcpu->kvm, gpa_to_gfn(cr3)), and otherwise set *pdptr = 0 and return 0?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=37
Paolo Bonzini
maintainerPATCH v4 38/45 KVM: x86: mark VM as dead after KVM_SET_SREGS fails to load PDPTRs
In reply to Paolo BonziniIf loading PDPTRs from guest memory fails, the KVM_SET_SREGS ioctl returns before resetting the MMU context and with stale PDPTRs. Mark the VM dead on the error path, so that the vCPU cannot run with inconsistent state; userspace should anyway use KVM_SET_SREGS2, which provides the cached PDPTRs and avoids the guest-memory read.
--- arch/x86/kvm/regs.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/regs.c +1 -0
@@ -634,6 +634,7 @@ static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs, r =load_pdptrs (vcpu,kvm_read_cr3 (vcpu));if (r <0 ) {srcu_read_unlock (&vcpu->kvm->srcu, idx); +kvm_vm_dead (vcpu->kvm);return r; } *mmu_reset_needed =1 ;
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 38/45] KVM: x86: mark VM as dead after KVM_SET_SREGS fails to load PDPTRs
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] KVM: x86: __set_sregs_common() kills VM when PAE CR3 page is unmapped --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] KVM: x86: __set_sregs_common() kills VM when PAE CR3 page is unmapped --
commit 4ba7fe2bbdbb50dd511abb52c67718a74b2c9c1c
Author: Paolo Bonzini <[email protected]>
KVM: x86: mark VM as dead after KVM_SET_SREGS fails to load PDPTRs
This commit marks the VM as dead in __set_sregs_common() if load_pdptrs() returns a negative error when setting PAE paging state, preventing the vCPU from running with stale PDPTRs and an unreset MMU context.
arch/x86/kvm/regs.c static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs, · line 638
Quoted 10 lines
arch/x86/kvm/regs.c +1
-0@@ -634,6 +634,7 @@ static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs, r =load_pdptrs (vcpu,kvm_read_cr3 (vcpu));if (r <0 ) {srcu_read_unlock (&vcpu->kvm->srcu, idx); +kvm_vm_dead (vcpu->kvm);return r;
[Severity: High]
Won't calling kvm_vm_dead() here terminate the VM whenever a PAE vCPU's CR3 points to an unmapped GPA during KVM_SET_SREGS (as well as KVM_SET_SREGS2 without KVM_SREGS2_FLAGS_PDPTRS_VALID, or KVM_RUN with KVM_SYNC_X86_SREGS)?
In arch/x86/kvm/regs.c, load_pdptrs() reads the guest page with kvm_vcpu_read_guest_page():
arch/x86/kvm/regs.c:load_pdptrs() {
...
ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(real_gpa), pdpte,
cr3 & GENMASK(11, 5), sizeof(pdpte));
if (ret == -EFAULT) {
kvm_prepare_memory_fault_exit(vcpu, real_gpa & PAGE_MASK, PAGE_SIZE,
false, false, false);
return ret;
}
if (ret)
return 0;
...
}Because kvm_vcpu_read_guest_page() -> __kvm_read_guest_page() returns 0 or -EFAULT, it returns -EFAULT not only for guest_memfd memory attribute faults, but also whenever the GFN has no memslot (for example, when userspace calls KVM_SET_SREGS before populating memslots via KVM_SET_USER_MEMORY_REGION, or on SVM with NPT when the guest loaded an unmapped GPA into CR3).
This also triggers during state restore of a nested PAE L2 guest when userspace calls KVM_SET_SREGS before KVM_SET_NESTED_STATE (such as in vcpu_load_state() in KVM selftests). At KVM_SET_SREGS time, is_guest_mode() is still false, so load_pdptrs() treats L2's CR3 as an L1 GPA without nested NPT/EPT translation; if that GPA does not map to a readable L1 memslot, load_pdptrs() returns -EFAULT.
In those cases, returning early and calling kvm_vm_dead() leaves segment registers and CR8 unloaded, skips the MMU reset, overwrites vcpu->run with KVM_EXIT_MEMORY_FAULT outside KVM_RUN, and permanently kills the VM. Should __set_sregs_common() ignore -EFAULT from load_pdptrs() (or avoid preparing a memory fault exit outside KVM_RUN) and continue loading the remaining sregs and resetting the MMU as before?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=38
Paolo Bonzini
maintainerPATCH v4 39/45 KVM: VMX: mark VM dead on shadow VMCS or eVMCS access failures
In reply to Paolo BonziniKVM ignores failures when reading or writing L1's in-memory VMCSs (shadow VMCS, eVMCS), allowing execution to continue with stale shadow VMCS values or silently losing updates. Check both the HVA cache initialization and the cached memory accesses, and mark the VM dead on failure; these failures can only occur when guest memory permissions deny the access, and terminating the VM is safe (though it precludes using memory attributes as a replacement for e.g. userfaultfd, and only leaves the feature usable for security purposes).
Likewise, treat a failed read of an enabled VP assist page as fatal too, otherwise KVM continues by disabling enlightened VMCS.
--- arch/x86/kvm/vmx/hyperv.c | 7 ++++++- arch/x86/kvm/vmx/nested.c | 21 ++++++++++++++------- 2 files changed, 20 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/hyperv.c +6 -1
@@ -18,9 +18,14 @@ u64 nested_get_evmptr(struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); -if (unlikely (kvm_hv_get_assist_page (vcpu))) +if (unlikely (!kvm_hv_assist_page_enabled (vcpu)))return EVMPTR_INVALID ; +if (unlikely (kvm_hv_get_assist_page (vcpu))) { +kvm_vm_dead (vcpu->kvm); +return EVMPTR_INVALID ; + } +if (unlikely (!hv_vcpu->vp_assist_page.enlighten_vmentry))return EVMPTR_INVALID ;
arch/x86/kvm/vmx/nested.c +14 -7
@@ -861,10 +861,13 @@ static void nested_cache_shadow_vmcs12(struct kvm_vcpu *vcpu,if (ghc->gpa != vmcs12->vmcs_link_pointer &&kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, vmcs12->vmcs_link_pointer,VMCS12_SIZE )) -return ; + goto error; -kvm_read_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), -VMCS12_SIZE ); +if (!kvm_read_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), +VMCS12_SIZE )) +return ; +error: +kvm_vm_dead (vcpu->kvm); } static void nested_flush_cached_shadow_vmcs12(struct kvm_vcpu *vcpu,
@@ -880,10 +883,13 @@ static void nested_flush_cached_shadow_vmcs12(struct kvm_vcpu *vcpu,if (ghc->gpa != vmcs12->vmcs_link_pointer &&kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, vmcs12->vmcs_link_pointer,VMCS12_SIZE )) -return ; + goto error; -kvm_write_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), -VMCS12_SIZE ); +if (!kvm_write_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), +VMCS12_SIZE )) +return ; +error: +kvm_vm_dead (vcpu->kvm); }/*
@@ -5091,7 +5097,8 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,* do that when something is forcing L2->L1 exit prior to * the first L2 run. */ - (void )nested_get_evmcs_page (vcpu); +if (!nested_get_evmcs_page (vcpu)) +kvm_vm_dead (vcpu->kvm);#endif }
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 39/45] KVM: VMX: mark VM dead on shadow VMCS or eVMCS access failures
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider: - [High] KVM: VMX: nested_get_evmcs_page() ignores EVMPTRLD_DISABLED and hits WARN_ON - [High] KVM: VMX…
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] KVM: VMX: nested_get_evmcs_page() ignores EVMPTRLD_DISABLED and hits WARN_ON
- [High] KVM: VMX: MSR_KVM_PV_EOI_EN write breaks nested_get_evmptr() and kills VM
- [Medium] KVM: VMX: nested_get_evmptr() error return disables eVMCS in callers
- [Medium] KVM: VMX: nested_vmx_run() blocks vCPU after nested_cache_shadow_vmcs12() fails
- [Medium] KVM: VMX: __nested_vmx_vmexit() marks VM dead during vmx_leave_nested() --
commit 6832da53d677d8f33bf223e6c9407f718e6f9f02
Author: Paolo Bonzini <[email protected]>
KVM: VMX: mark VM dead on shadow VMCS or eVMCS access failures
This commit marks the VM dead when initializing the HVA cache or accessing guest memory fails for shadow VMCS12, enlightened VMCS, or an enabled Hyper-V VP assist page.
Quoted 6 lines
arch/x86/kvm/vmx/hyperv.c +0
-0@@ -18,9 +18,14 @@ u64 nested_get_evmptr(struct kvm_vcpu *vcpu) {
arch/x86/kvm/vmx/hyperv.c u64 nested_get_evmptr(struct kvm_vcpu *vcpu) · line 27
Quoted 10 lines
arch/x86/kvm/vmx/hyperv.c +5
-1struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); -if (unlikely (kvm_hv_get_assist_page (vcpu))) +if (unlikely (!kvm_hv_assist_page_enabled (vcpu)))return EVMPTR_INVALID ; +if (unlikely (kvm_hv_get_assist_page (vcpu))) { +kvm_vm_dead (vcpu->kvm); +return EVMPTR_INVALID ; + }
[Severity: High]
Can an unprivileged L1 guest trigger a host kernel WARN_ON_ONCE and kill the VM here without any host memory protection attributes?
HV_X64_MSR_VP_ASSIST_PAGE and MSR_KVM_PV_EOI_EN share vcpu->arch.pv_eoi.msr_val and the vcpu->arch.pv_eoi.data cache via kvm_lapic_set_pv_eoi():
arch/x86/kvm/hyperv.c:kvm_hv_assist_page_enabled() {
if (!(hv_vcpu->hv_vapic & HV_X64_MSR_VP_ASSIST_PAGE_ENABLE))
return false;
return vcpu->arch.pv_eoi.msr_val & KVM_MSR_ENABLED;
}arch/x86/kvm/lapic.c:kvm_lapic_set_pv_eoi() {
if (data & KVM_MSR_ENABLED) {
if (addr == ghc->gpa && len <= ghc->len)
new_len = ghc->len;
else
new_len = len;
ret = kvm_gfn_to_hva_cache_init(vcpu->kvm, ghc, addr, new_len);
if (ret)
return ret;
}
}If the guest enables HV_X64_MSR_VP_ASSIST_PAGE at one GPA (initializing ghc->len to sizeof(struct hv_vp_assist_page)) and then writes MSR_KVM_PV_EOI_EN with a different valid GPA and KVM_MSR_ENABLED, kvm_lapic_set_pv_eoi() reinitializes vcpu->arch.pv_eoi.data with new_len = 1 while kvm_hv_assist_page_enabled() remains true.
Alternatively, if the guest writes MSR_KVM_PV_EOI_EN with an unmapped GPA and KVM_MSR_ENABLED, kvm_gfn_to_hva_cache_init() sets ghc->hva to KVM_HVA_ERR_BAD and returns -EFAULT without clearing KVM_MSR_ENABLED in vcpu->arch.pv_eoi.msr_val.
When the guest then executes VMCLEAR, VMLAUNCH, or VMRESUME, nested_get_evmptr() -> kvm_hv_get_assist_page() -> kvm_read_guest_offset_cached() either hits:
if (WARN_ON_ONCE(len + offset > ghc->len)) return -EINVAL;
or returns -EFAULT, and nested_get_evmptr() calls kvm_vm_dead(vcpu->kvm).
[Severity: Medium]
Also in nested_get_evmptr(), when kvm_hv_get_assist_page() fails, does returning EVMPTR_INVALID still cause callers to continue the current instruction as if enlightened VMCS were disabled before KVM_REQ_VM_DEAD is serviced?
In handle_vmclear() -> nested_evmcs_handle_vmclear():
if (!guest_cpu_cap_has_evmcs(vcpu) || !evmptr_is_valid(nested_get_evmptr(vcpu))) return false;
returning EVMPTR_INVALID causes nested_evmcs_handle_vmclear() to return false, so handle_vmclear() falls through to the non-eVMCS path and writes zero to vmptr + offsetof(struct vmcs12, launch_state) in guest memory.
Similarly, in nested_vmx_run() -> nested_vmx_handle_enlightened_vmptrld():
evmcs_gpa = nested_get_evmptr(vcpu);
if (!evmptr_is_valid(evmcs_gpa)) {
nested_release_evmcs(vcpu);
return EVMPTRLD_DISABLED;
}returning EVMPTR_INVALID releases the eVMCS and returns EVMPTRLD_DISABLED instead of EVMPTRLD_ERROR, allowing nested_vmx_run() to proceed with eVMCS disabled.
[ ... ]
Quoted 9 lines
arch/x86/kvm/vmx/nested.c +0
-1@@ -861,10 +861,13 @@ static void nested_cache_shadow_vmcs12(struct kvm_vcpu *vcpu,if (ghc->gpa != vmcs12->vmcs_link_pointer &&kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, vmcs12->vmcs_link_pointer,VMCS12_SIZE )) -return ;
arch/x86/kvm/vmx/nested.c static void nested_cache_shadow_vmcs12(struct kvm_vcpu *vcpu, · line 871
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +6
-2+ goto error; -kvm_read_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), -VMCS12_SIZE ); +if (!kvm_read_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), +VMCS12_SIZE )) +return ; +error: +kvm_vm_dead (vcpu->kvm); }
[Severity: Medium]
What happens in nested_vmx_run() when nested_cache_shadow_vmcs12() (or nested_get_evmptr() when vmx->nested.current_vmptr is valid) marks the VM dead and vmcs12->guest_activity_state is GUEST_ACTIVITY_HLT or GUEST_ACTIVITY_WAIT_SIPI?
In arch/x86/kvm/vmx/nested.c:nested_vmx_run():
nested_cache_shadow_vmcs12(vcpu, vmcs12);
switch (vmcs12->guest_activity_state) {
case GUEST_ACTIVITY_HLT:
if (!(vmcs12->vm_entry_intr_info_field & INTR_INFO_VALID_MASK) &&
!nested_cpu_has(vmcs12, CPU_BASED_NMI_WINDOW_EXITING) &&
!(nested_cpu_has(vmcs12, CPU_BASED_INTR_WINDOW_EXITING) &&
(vmcs12->guest_rflags & X86_EFLAGS_IF))) {
vcpu->arch.nested_run_pending = 0;
return kvm_emulate_halt_noskip(vcpu);
}
break;
case GUEST_ACTIVITY_WAIT_SIPI:
vcpu->arch.nested_run_pending = 0;
kvm_set_mp_state(vcpu, KVM_MP_STATE_INIT_RECEIVED);
break;Because nested_cache_shadow_vmcs12() does not return an error to abort nested_vmx_run(), the vCPU transitions to KVM_MP_STATE_HALTED or KVM_MP_STATE_INIT_RECEIVED and returns 1.
On the next iteration of vcpu_run():
if (kvm_vcpu_running(vcpu)) {
r = vcpu_enter_guest(vcpu);
} else {
r = vcpu_block(vcpu);
}vcpu_run() calls vcpu_block() instead of vcpu_enter_guest(), where KVM_REQ_VM_DEAD is checked, so could the vCPU block on a dead VM (or exit with KVM_EXIT_HLT when the LAPIC is in userspace) and later flush the uncached shadow_vmcs12 back to guest memory on wakeup?
Quoted 18 lines
arch/x86/kvm/vmx/nested.c +6
-3@@ -880,10 +883,13 @@ static void nested_flush_cached_shadow_vmcs12(struct kvm_vcpu *vcpu,if (ghc->gpa != vmcs12->vmcs_link_pointer &&kvm_gfn_to_hva_cache_init (vcpu->kvm, ghc, vmcs12->vmcs_link_pointer,VMCS12_SIZE )) -return ; + goto error; -kvm_write_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), -VMCS12_SIZE ); +if (!kvm_write_guest_cached (vcpu->kvm, ghc,get_shadow_vmcs12 (vcpu), +VMCS12_SIZE )) +return ; +error: +kvm_vm_dead (vcpu->kvm); }/*
arch/x86/kvm/vmx/nested.c void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 5105
Quoted 8 lines
arch/x86/kvm/vmx/nested.c +2
-1@@ -5093,7 +5099,8 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,* do that when something is forcing L2->L1 exit prior to * the first L2 run. */ - (void )nested_get_evmcs_page (vcpu); +if (!nested_get_evmcs_page (vcpu)) +kvm_vm_dead (vcpu->kvm);#endif }
[Severity: High]
When kvm_hv_get_assist_page() fails in nested_get_evmptr() while vmx->nested.hv_evmcs_vmptr is EVMPTR_MAP_PENDING, won't nested_get_evmcs_page() still return true and trigger a host WARN_ON?
The call chain is:
nested_get_evmcs_page()
-> nested_vmx_handle_enlightened_vmptrld()
-> nested_get_evmptr()
When nested_get_evmptr() fails and returns EVMPTR_INVALID, nested_vmx_handle_enlightened_vmptrld() resets vmx->nested.hv_evmcs_vmptr to EVMPTR_INVALID via nested_release_evmcs() and returns EVMPTRLD_DISABLED.
However, nested_get_evmcs_page() only checks for EVMPTRLD_VMFAIL and EVMPTRLD_ERROR:
if (guest_cpu_cap_has_evmcs(vcpu) &&
vmx->nested.hv_evmcs_vmptr == EVMPTR_MAP_PENDING) {
enum nested_evmptrld_status evmptrld_status =
nested_vmx_handle_enlightened_vmptrld(vcpu, false);
if (evmptrld_status == EVMPTRLD_VMFAIL ||
evmptrld_status == EVMPTRLD_ERROR)
return false;
vmx->nested.need_vmcs12_to_shadow_sync = true;
}
return true;In __nested_vmx_vmexit(), if nested_vmx_handle_enlightened_vmptrld() returns EVMPTRLD_DISABLED (either because reading the VP assist page failed, or because the VP assist page or enlighten_vmentry was disabled), nested_get_evmcs_page() returns true and bypasses the kvm_vm_dead() check.
Furthermore, in vcpu_enter_guest(), KVM_REQ_VM_DEAD is checked before KVM_REQ_GET_NESTED_STATE_PAGES:
if (kvm_check_request(KVM_REQ_VM_DEAD, vcpu))
return -EIO;
if (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu)) {
if (unlikely(!kvm_x86_call(get_nested_state_pages)(vcpu))) {
r = 0;
goto out;
}
}When vmx_get_nested_state_pages() -> nested_get_evmcs_page() sees EVMPTRLD_DISABLED, it sets vmx->nested.need_vmcs12_to_shadow_sync = true and returns true. Execution continues into vmx_prepare_switch_to_guest() -> nested_sync_vmcs12_to_shadow():
if (nested_vmx_is_evmptr12_valid(vmx)) copy_vmcs12_to_enlightened(vmx); else copy_vmcs12_to_shadow(vmx);
Because hv_evmcs_vmptr is now EVMPTR_INVALID, this falls back to copy_vmcs12_to_shadow(vmx) and hits WARN_ON(!shadow_vmcs) when shadow VMCS is not enabled. Should nested_get_evmcs_page() check evmptrld_status != EVMPTRLD_SUCCEEDED instead?
[Severity: Medium]
Also in __nested_vmx_vmexit(), could calling kvm_vm_dead() in nested_get_evmcs_page() and nested_flush_cached_shadow_vmcs12() kill the VM during host-initiated teardown via vmx_leave_nested()?
arch/x86/kvm/vmx/nested.c:vmx_leave_nested() {
if (is_guest_mode(vcpu)) {
vcpu->arch.nested_run_pending = 0;
nested_vmx_vmexit(vcpu, -1, 0, 0);
}
free_nested(vcpu);
}When vmx_leave_nested() is called (for example during vCPU reset, clearing MSR_IA32_FEAT_CTL, SMM transitions in KVM_SET_VCPU_EVENTS, or KVM_SET_NESTED_STATE), __nested_vmx_vmexit() runs with vm_exit_reason == -1 right before free_nested() discards all nested state. If the pending eVMCS or shadow VMCS GPA in guest memory is unmapped or not writable at that point, should __nested_vmx_vmexit() avoid accessing guest memory and marking the VM dead when vm_exit_reason == -1?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=39
Paolo Bonzini
maintainerPATCH v4 40/45 KVM: let kvm_arch_post_set_memory_attributes drop mmu_lock
In reply to Paolo BonziniEnsure that mmu_invalidate_in_progress is not cleared until kvm_arch_post_set_memory_attributes() returns.
kvm_arch_post_set_memory_attributes() runs with the MMU lock taken exclusively, so a concurrent page fault would be blocked anyway, and delaying the clearing makes it possible to use cond_resched() within kvm_arch_post_set_memory_attributes().
--- virt/kvm/kvm_main.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
virt/kvm/kvm_main.c +12 -4
@@ -2630,7 +2630,7 @@ static bool kvm_range_memory_attributes_need_sync(struct kvm *kvm,return kvm_range_has_any_vm_memory_attributes (kvm, start, end, mask); } -static __always_inline void kvm_handle_gfn_range(struct kvm *kvm, +static __always_inline bool kvm_handle_gfn_range(struct kvm *kvm,struct kvm_mmu_notifier_range *range) {struct kvm_gfn_range gfn_range;
@@ -2680,6 +2680,8 @@ static __always_inline void kvm_handle_gfn_range(struct kvm *kvm,if (found_memslot)KVM_MMU_UNLOCK (kvm); + +return found_memslot; } static bool kvm_pre_set_vm_memory_attributes(struct kvm *kvm,
@@ -2717,10 +2719,11 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, .end = end, .arg.attributes = attributes, .handler = kvm_arch_post_set_vm_memory_attributes, - .on_lock = kvm_mmu_invalidate_end, + .on_lock = (void *)kvm_null_fn, .may_block = true, };bool sync = false; +bool found_memslot;unsigned long i;void *entry;int r =0 ;
@@ -2752,7 +2755,7 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,cond_resched (); } -kvm_handle_gfn_range (kvm, &pre_set_range); + found_memslot =kvm_handle_gfn_range (kvm, &pre_set_range);for (i = start; i < end; i++) { r = xa_err(xa_store(&kvm->mem_attrs.array, i, entry,
@@ -2765,7 +2768,12 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,smp_store_release (&kvm->mem_attrs.generation, kvm->mem_attrs.generation +1 ); -kvm_handle_gfn_range (kvm, &post_set_range); +if (found_memslot) { +kvm_handle_gfn_range (kvm, &post_set_range); +KVM_MMU_LOCK (kvm); +kvm_mmu_invalidate_end (kvm); +KVM_MMU_UNLOCK (kvm); + } out_unlock:mutex_unlock (&kvm->slots_lock);
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 41/45 KVM: x86/mmu: Obsolete all roots if memattr contains gPTEs
In reply to Paolo BonziniNote that the zapping of obsolete roots needs to happen after mmu_lock is dropped, so introduce a separate kvm_arch callback. The two map neatly to the "front" and "back" halves of kvm_mmu_zap_all_fast().
--- arch/x86/kvm/mmu/mmu.c | 30 ++++++++++++++++++++++++++++++ include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 1 + 3 files changed, 32 insertions(+)
arch/x86/kvm/mmu/mmu.c +30 -0
@@ -8359,6 +8359,7 @@ bool kvm_arch_post_set_vm_memory_attributes(struct kvm *kvm, {unsigned long attrs = range->arg.attributes;struct kvm_memory_slot *slot = range->slot; +struct kvm_mmu_page *sp;int level;lockdep_assert_held_write (&kvm->mmu_lock);
@@ -8414,9 +8415,38 @@ bool kvm_arch_post_set_vm_memory_attributes(struct kvm *kvm,hugepage_set_mixed (slot, gfn, level); } } + +/* +* There are special considerations when applying an memory protection +* attibute against a page containing guest PTEs, because virtual +* addresses that requires traversing that page need to fault. +* +* When using shadow paging, the PTEs that the CPU actually walks +* do not have any information about attributes (unlike TDP where +* the attributes become part of the TDP permission bits). Since +* this should be a rare occasion, just invalidate all the mmu roots +* if shadow paging is in use and attributes do cover a guest PTE. +*/ +for (gfn_t gfn = range->start; gfn < range->end; gfn++) { +for_each_gfn_valid_sp_with_gptes (kvm, sp, gfn) { +__kvm_mmu_zap_all_fast_front_half (kvm); +return false; + } + }return false; } +void kvm_arch_post_set_vm_memory_attributes_unlocked (struct kvm *kvm) +{ +/* +* This may also be called if no zap was done in +* kvm_arch_post_set_memory_attributes(). It is not an issue because +* __kvm_mmu_zap_all_fast_back_half() is cheap when it does not do +* anything. +*/ +__kvm_mmu_zap_all_fast_back_half (kvm); +} +void kvm_mmu_init_memslot_memory_attributes (struct kvm *kvm,struct kvm_memory_slot *slot) {
include/linux/kvm_host.h +1 -0
@@ -2624,6 +2624,7 @@ bool kvm_arch_pre_set_vm_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range);bool kvm_arch_post_set_vm_memory_attributes (struct kvm *kvm,struct kvm_gfn_range *range); +void kvm_arch_post_set_vm_memory_attributes_unlocked (struct kvm *kvm);bool kvm_range_has_any_vm_memory_attributes (struct kvm *kvm,gfn_t start,gfn_t end,unsigned long mask);u64 kvm_supported_vm_mem_attributes (struct kvm *kvm);
virt/kvm/kvm_main.c +1 -0
@@ -2773,6 +2773,7 @@ static int kvm_set_vm_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end,KVM_MMU_LOCK (kvm);kvm_mmu_invalidate_end (kvm);KVM_MMU_UNLOCK (kvm); +kvm_arch_post_set_vm_memory_attributes_unlocked (kvm); } out_unlock:
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 42/45 KVM: x86: selftests: Introduce memory protection attributes test
In reply to Paolo BonziniIntroduce architecture-independent tests for memory attributes, using small architecture-specific snippets to handle the execution tests.
--- tools/include/uapi/linux/kvm.h | 3 + tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/kvm_util.h | 18 +- .../testing/selftests/kvm/memory_attributes.c | 344 ++++++++++++++++++ .../selftests/kvm/x86/memory_attributes.c | 43 +++ 5 files changed, 405 insertions(+), 4 deletions(-) create mode 100644 tools/testing/selftests/kvm/memory_attributes.c create mode 100644 tools/testing/selftests/kvm/x86/memory_attributes.c
tools/include/uapi/linux/kvm.h +3 -0
@@ -1649,6 +1649,9 @@ struct kvm_memory_attributes {__u64 flags; }; +#define KVM_MEMORY_ATTRIBUTE_NR (1ULL << 0) +#define KVM_MEMORY_ATTRIBUTE_NW (1ULL << 1) +#define KVM_MEMORY_ATTRIBUTE_NX (1ULL << 2)#define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)#define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest_memfd)
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -173,6 +173,7 @@ TEST_GEN_PROGS_x86 += rseq_test TEST_GEN_PROGS_x86 += steal_time TEST_GEN_PROGS_x86 += system_counter_offset_test TEST_GEN_PROGS_x86 += pre_fault_memory_test +TEST_GEN_PROGS_x86 += memory_attributes # Compiled outputs used by test targets TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_test
tools/testing/selftests/kvm/include/kvm_util.h +14 -4
@@ -473,17 +473,27 @@ static inline void vm_enable_cap(struct kvm_vm *vm, u32 cap, u64 arg0)int kvm_gpa_to_guest_memfd (struct kvm_vm *vm,gpa_t gpa,off_t *fd_offset,size_t *nr_bytes); -static inline void vm_set_memory_attributes (struct kvm_vm *vm,gpa_t gpa, -size_t size,u64 attributes) +static inline int __vm_set_memory_attributes (struct kvm_vm *vm,u64 gpa, +u64 size,u64 attributes, +u64 flags) {struct kvm_memory_attributes attr = { .attributes = attributes, .address = gpa, .size = size, - .flags =0 , + .flags = flags, }; -vm_ioctl (vm,KVM_SET_MEMORY_ATTRIBUTES , &attr); +return __vm_ioctl (vm,KVM_SET_MEMORY_ATTRIBUTES , &attr); +} + +static inline void vm_set_memory_attributes (struct kvm_vm *vm,gpa_t gpa, +u64 size,u64 attributes) +{ +int rc; + + rc =__vm_set_memory_attributes (vm, gpa, size, attributes,0 ); +TEST_ASSERT_VM_VCPU_IOCTL (!rc,KVM_SET_MEMORY_ATTRIBUTES , rc, vm); } static inline int __gmem_set_memory_attributes(int fd, u64 offset,
tools/testing/selftests/kvm/memory_attributes.c new file +344 -0
344 lines not on this page · raw message
tools/testing/selftests/kvm/x86/memory_attributes.c new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 42/45] KVM: x86: selftests: Introduce memory protection attributes test
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] KVM: selftests: unused MMIO mapping and non-static arrays in memory_attributes --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] KVM: selftests: unused MMIO mapping and non-static arrays in memory_attributes --
commit 739900eb4be23c429345b95c9dff33aecc77373c
Author: Paolo Bonzini <[email protected]>
KVM: x86: selftests: Introduce memory protection attributes test
This commit introduces a kselftest for KVM memory protection attributes with architecture-independent test logic and x86-specific helpers for read, write, and execution access checks.
Quoted 14 lines
tools/testing/selftests/kvm/memory_attributes.c new file +8
-0@@ -0,0 +1,344 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (C) 2024, Amazon.com, Inc. or its affiliates. All Rights Reserved +* +* Test for KVM_MEMORY_ATTRIBUTES +*/ +#include <pthread.h> +#include <signal.h>
[ ... ]
Quoted 21 lines
tools/testing/selftests/kvm/memory_attributes.c new file +21
-0+#define TEST_MEM_SLOT 1 +#define TEST_MEM_GPA 0x80000000 + +#define MMIO_GPA 0x700000000 +#define MMIO_GVA MMIO_GPA + +enum { +TEST_OP_NOP , +TEST_OP_READ , +TEST_OP_WRITE , +TEST_OP_EXEC , +TEST_OP_EXIT , +}; + +const char *test_op_names[] = +{ + [TEST_OP_READ ] ="Read" , + [TEST_OP_WRITE ] ="Write" , + [TEST_OP_EXEC ] ="Exec" , + [TEST_OP_EXIT ] ="Exit" , +};
[ ... ]
Quoted 5 lines
tools/testing/selftests/kvm/memory_attributes.c new file +5
-0+const struct memory_access { +const char *name; +uint64_t attrs; +int ops[5 ]; +} access_array[] = {
[ ... ]
Quoted 13 lines
tools/testing/selftests/kvm/memory_attributes.c new file +13
-0+int main (int argc,char *argv[]) +{ +uint32_t guest_page_size = vm_guest_mode_params[VM_MODE_DEFAULT ].page_size; +unsigned int ptes_per_page = guest_page_size /8 ; +size_t pages = ptes_per_page *2 ;/* 2 huge-pages */ +size_t size = guest_page_size * pages; +struct kvm_vcpu *vcpu; +gva_t test_mem; +struct kvm_vm *vm; + +TEST_REQUIRE (kvm_check_cap (KVM_CAP_MEMORY_ATTRIBUTES ) & +KVM_MEMORY_ATTRIBUTE_NO_ACCESS ); +
tools/testing/selftests/kvm/memory_attributes.c line 335
Quoted 10 lines
tools/testing/selftests/kvm/memory_attributes.c new file +10
-0+ vm =__vm_create_with_one_vcpu (&vcpu, pages, guest_code); + +/* add a few extra pages for use by tests */ +vm_userspace_mem_region_add (vm,VM_MEM_SRC_ANONYMOUS , +TEST_MEM_GPA ,TEST_MEM_SLOT , + pages +100 ,0 ); + vm->memslots[MEM_REGION_TEST_DATA ] =TEST_MEM_SLOT ; + + test_mem =vm_alloc_pages (vm, pages); +virt_map (vcpu->vm,MMIO_GVA ,MMIO_GPA ,1 );
[Severity: Low]
This isn't a bug, but even after the subsequent commit "KVM: x86: selftests:
Introduce memory attributes side-channel tests" in this series, MMIO_GPA and MMIO_GVA are never accessed anywhere in tools/testing/selftests/kvm/memory_attributes.c after main() maps them via virt_map(vcpu->vm, MMIO_GVA, MMIO_GPA, 1).
Similarly, signal.h is never used in this file, and the file-local arrays test_op_names[] and access_array[] are declared without static.
Could the unused MMIO_GPA/MMIO_GVA definitions, virt_map() call, and signal.h include be dropped, and should test_op_names[] and access_array[] be marked static?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=42
Paolo Bonzini
maintainerPATCH v4 43/45 KVM: x86: selftests: Introduce memory attributes PTE test
In reply to Paolo BonziniAdd more memory attributes tests to check that memory protection restricts accesses even when installed on a page that holds guest page table entries. Notably two cases are taken into account:
- The page is made non-accesible. In such case the next access to a virtual memory address translated by that paging structure should fault.
- The page is made read-only. In such case the next access to a virtual memory address translated by that paging structure should either fault, or succeed yet not perform any writes into the PTE (accessed and dirty bits).
--- .../testing/selftests/kvm/memory_attributes.c | 39 ++++- .../selftests/kvm/x86/memory_attributes.c | 144 +++++++++++++++++- 2 files changed, 177 insertions(+), 6 deletions(-)
tools/testing/selftests/kvm/memory_attributes.c +35 -4
@@ -25,11 +25,16 @@#define MMIO_GPA 0x700000000#define MMIO_GVA MMIO_GPA +#define PT_WRITABLE_MASK BIT_ULL(1) +#define PT_ACCESSED_MASK BIT_ULL(5) +#define PTE_VADDR 0x1000000000 +enum {TEST_OP_NOP ,TEST_OP_READ ,TEST_OP_WRITE ,TEST_OP_EXEC , +TEST_OP_INVPLG ,TEST_OP_EXIT , };
@@ -38,6 +43,7 @@ const char *test_op_names[] = [TEST_OP_READ] = "Read", [TEST_OP_WRITE] = "Write", [TEST_OP_EXEC] = "Exec", + [TEST_OP_INVPLG] = "Invplg", [TEST_OP_EXIT] = "Exit", };
@@ -45,6 +51,7 @@ struct test_data {uint8_t op;int stage;gva_t vaddr; +uint64_t expected_val;struct kvm_vcpu *vcpu; };
@@ -54,7 +61,8 @@ static struct test_data *test_data;static uint64_t arch_controlled_read (gva_t addr);static void arch_controlled_write (gva_t addr,uint64_t val);static void arch_controlled_exec (gva_t addr); -static void arch_write_return_insn (struct kvm_vm *vm,gpa_t vaddr); +static void arch_write_return_insn (struct kvm_vm *vm,gpa_t paddr); +static bool arch_test_op (struct test_data *test_data);static void guest_code (void *data) {
@@ -62,6 +70,7 @@ static void guest_code(void *data)int stage =1 ;while (true) { +uint64_t expected_val =READ_ONCE (test_data->expected_val);gva_t vaddr =READ_ONCE (test_data->vaddr);switch (READ_ONCE (test_data->op)) {
@@ -70,7 +79,7 @@ static void guest_code(void *data)GUEST_SYNC (stage++);break ;case TEST_OP_WRITE : -arch_controlled_write (vaddr,1 ); +arch_controlled_write (vaddr, expected_val);GUEST_SYNC (stage++);break ;case TEST_OP_EXEC :
@@ -78,7 +87,10 @@ static void guest_code(void *data)GUEST_SYNC (stage++);break ;default : - goto exit; +if (!arch_test_op (test_data)) + goto exit; +GUEST_SYNC (stage++); +break ; }; }
@@ -109,6 +121,21 @@ static void vcpu_run_and_inc_stage(struct kvm_vcpu *vcpu) test_data->stage++; } +static int test_page (struct kvm_vcpu *vcpu,int op,gva_t vaddr) +{ +int rc; + + test_data->op = op; + test_data->vaddr = vaddr; + + rc =_vcpu_run (vcpu); + +if (rc >=0 ) + test_data->stage++; + +return rc <0 ? -errno : rc; +} +static void test_page_restricted (struct kvm_vcpu *vcpu,int op,gva_t vaddr,gpa_t fault_paddr,uint64_t fault_reason)
@@ -125,7 +152,8 @@ static void test_page_restricted(struct kvm_vcpu *vcpu, int op, test_op_names[op], rc, errno);TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_MEMORY_FAULT );TEST_ASSERT_EQ (vcpu->run->memory_fault.gpa, fault_paddr); -TEST_ASSERT_EQ (vcpu->run->memory_fault.flags, fault_reason); +if (fault_reason) +TEST_ASSERT_EQ (vcpu->run->memory_fault.flags, fault_reason);TEST_ASSERT_EQ (vcpu->run->memory_fault.size, vm->page_size); }
@@ -337,6 +365,9 @@ int main(int argc, char *argv[])test_input_validation (vm);test_memory_access (vcpu, test_mem, size); +#ifdef __x86_64__ +arch_test_memory_access_pte (vcpu, test_mem); +#endif test_finalize (vcpu);kvm_vm_free (vm);
tools/testing/selftests/kvm/x86/memory_attributes.c +142 -2
149 lines not on this page · raw message
Signature · 1 line
2.52.0
Paolo Bonzini
maintainerPATCH v4 44/45 KVM: x86: selftests: Introduce memory attributes side-channel tests
In reply to Paolo BonziniIntroduce memory attributes selftests to catch any vulnerable side-channels.
Memory attributes access restrictions are vulnerable to side-channel attacks. This means that any KVM operation initiated by the guest that requires guest memory access (which is the case for most para-virtualised interfaces) needs to consider memory attributes.
The tests confirm this requirement is upheld for a variety of use-cases, exercising various kinds of guest memory access such as kvm_read/write_guest(), gfn_to_hva_cache and gfn_to_pfn_cache.
--- .../testing/selftests/kvm/include/kvm_util.h | 9 + .../selftests/kvm/include/x86/processor.h | 1 + .../testing/selftests/kvm/lib/x86/processor.c | 5 + .../testing/selftests/kvm/memory_attributes.c | 51 +++- .../selftests/kvm/x86/memory_attributes.c | 232 ++++++++++++++++++ 5 files changed, 297 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/include/kvm_util.h +9 -0
@@ -1058,6 +1058,15 @@ static inline int vcpu_get_stats_fd(struct kvm_vcpu *vcpu)return fd; } +static inline struct kvm_translation vcpu_translate (struct kvm_vcpu *vcpu, +u64 gva) +{ +struct kvm_translation tr = { .linear_address = gva }; + +vcpu_ioctl (vcpu,KVM_TRANSLATE , &tr); +return tr; +} +int __kvm_has_device_attr (int dev_fd,u32 group,u64 attr);static inline void kvm_has_device_attr (int dev_fd,u32 group,u64 attr)
tools/testing/selftests/kvm/include/x86/processor.h +1 -0
@@ -1493,6 +1493,7 @@ static inline bool kvm_is_nested_virtualization_enabled(void)return get_kvm_amd_param_integer ("nested" ); } +u64 *vm_get_pte_level (struct kvm_vm *vm,gva_t gva,int *level);u64 *vm_get_pte (struct kvm_vm *vm,gva_t gva);u64 kvm_hypercall (u64 nr,u64 a0,u64 a1,u64 a2,u64 a3);
tools/testing/selftests/kvm/lib/x86/processor.c +5 -0
@@ -402,6 +402,11 @@ u64 *tdp_get_pte(struct kvm_vm *vm, u64 l2_gpa)return __vm_get_page_table_entry (vm, &vm->stage2_mmu, l2_gpa, &level); } +u64 *vm_get_pte_level (struct kvm_vm *vm,gva_t gva,int *level) +{ +return __vm_get_page_table_entry (vm, &vm->mmu, gva, level); +} +u64 *vm_get_pte (struct kvm_vm *vm,gva_t gva) {int level =PG_LEVEL_4K ;
tools/testing/selftests/kvm/memory_attributes.c +50 -1
122 lines not on this page · raw message
tools/testing/selftests/kvm/x86/memory_attributes.c +232 -0
252 lines not on this page · raw message
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH 44/45] KVM: x86: selftests: Introduce memory attributes side-channel tests
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] KVM: selftests: missing vcpu_force_vapic_update() in vapic side-channel test - [Medium] KVM…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] KVM: selftests: missing vcpu_force_vapic_update() in vapic side-channel test
- [Medium] KVM: selftests: pvclock side-channel test bypasses active gpc invalidation
- [Medium] KVM: selftests: unconditional vcpu_set_hv_cpuid() in main() breaks !HYPERV --
commit d4f11e28c38cae048fc4237383806de4018441e7
Author: Paolo Bonzini <[email protected]>
KVM: x86: selftests: Introduce memory attributes side-channel tests
This commit introduces x86 selftests to verify that guest-initiated KVM operations accessing guest memory via kvm_read/write_guest(), gfn_to_hva_cache, gfn_to_pfn_cache, and the guest page walker respect memory attribute restrictions.
Quoted 4 lines
tools/testing/selftests/kvm/memory_attributes.c +0
-0
[ ... ]
tools/testing/selftests/kvm/memory_attributes.c int main(int argc, char *argv[]) · line 412
Quoted 7 lines
tools/testing/selftests/kvm/memory_attributes.c +3
-0@@ -362,11 +407,15 @@ int main(int argc, char *argv[]) test_mem =vm_alloc_pages (vm, pages);virt_map (vcpu->vm,MMIO_GVA ,MMIO_GPA ,1 ); test_data =init_test_data (vcpu); +#ifdef __x86_64__ +vcpu_set_hv_cpuid (vcpu); +#endif
[Severity: Medium]
What happens on an x86_64 kernel built with CONFIG_KVM_HYPERV=n?
In test_side_channel_hyperv_hypercall_inputs(), the test checks kvm_has_cap(KVM_CAP_HYPERV_SEND_IPI) so it can skip the Hyper-V subtest when Hyper-V support is unavailable.
However, main() unconditionally calls vcpu_set_hv_cpuid(vcpu) -> kvm_get_supported_hv_cpuid(), which invokeskvm_ioctl(kvm_fd, KVM_GET_SUPPORTED_HV_CPUID, cpuid) and asserts success. Because KVM_GET_SUPPORTED_HV_CPUID returns -EINVAL when CONFIG_KVM_HYPERV is disabled, will this abort the entire test binary in main() before any tests can run unless vcpu_set_hv_cpuid(vcpu) is guarded by a capability check such as kvm_has_cap(KVM_CAP_SYS_HYPERV_CPUID) or kvm_has_cap(KVM_CAP_HYPERV_SEND_IPI)?
Quoted 4 lines
tools/testing/selftests/kvm/x86/memory_attributes.c +0
-0
[ ... ]
Quoted 14 lines
tools/testing/selftests/kvm/x86/memory_attributes.c +14
-0+/* +* Setup the vapic address on a GPA that is write-protected. Force an vapic +* update and validate its contents were not changes. Then, lift the write +* restriction and validate the page's contents are updated. +*/ +static void test_side_channel_vapic_addr (struct kvm_vcpu *vcpu) +{ +struct kvm_vm *vm = vcpu->vm; +gva_t vaddr =vm_alloc_page (vm); +gpa_t paddr =addr_gva2gpa (vm, vaddr); + +vm_set_vapic_addr (vcpu, paddr); + test_data->op =TEST_OP_READ ; + test_data->vaddr = vaddr;
tools/testing/selftests/kvm/x86/memory_attributes.c static void arch_test_memory_access_pte(struct kvm_vcpu *vcpu, gva_t vaddr) · line 315
Quoted 10 lines
tools/testing/selftests/kvm/x86/memory_attributes.c +10
-0+ test_data->confirm_read =1 ; + test_data->expected_val = ~0ULL >>32 ; +memset (addr_gva2hva (vm, vaddr),0xff ,sizeof (uint32_t )); +vm_set_memory_attributes (vm, paddr, vm->page_size,KVM_MEMORY_ATTRIBUTE_NW ); +vcpu_run_and_inc_stage (vcpu); + +vm_set_memory_attributes (vm, paddr, vm->page_size,0 ); +vcpu_force_vapic_update (vcpu); + test_data->expected_val =0ULL ; +vcpu_run_and_inc_stage (vcpu);
[Severity: Medium]
Should vcpu_force_vapic_update(vcpu) also be called before the first vcpu_run_and_inc_stage(vcpu) while KVM_MEMORY_ATTRIBUTE_NW is set?
In arch/x86/kvm/x86.c, vcpu_enter_guest() only calls kvm_lapic_sync_to_vapic(vcpu) when KVM_REQ_EVENT (or an interrupt window) is pending:
if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win ||
kvm_xen_has_interrupt(vcpu)) {
++vcpu->stat.req_event;
...
if (kvm_lapic_enabled(vcpu)) {
kvm_lapic_update_cr8_intercept(vcpu);
kvm_lapic_sync_to_vapic(vcpu);
}
}Neither vm_set_vapic_addr() (KVM_SET_VAPIC_ADDR) nor vm_set_memory_attributes() (KVM_SET_MEMORY_ATTRIBUTES) sets KVM_REQ_EVENT on the vCPU. Without calling vcpu_force_vapic_update(vcpu) before the first vcpu_run_and_inc_stage(vcpu), won't kvm_lapic_sync_to_vapic() be skipped on that entry, allowing the write-protected check to pass even if kvm_write_guest_cached() did not honor memory attributes?
[ ... ]
Quoted 12 lines
tools/testing/selftests/kvm/x86/memory_attributes.c +12
-0+static void test_side_channel_pvclock (struct kvm_vcpu *vcpu) +{ +struct kvm_vm *vm = vcpu->vm; +gva_t vaddr =vm_alloc_page (vm); +gpa_t paddr =addr_gva2gpa (vm, vaddr); +struct pvclock_vcpu_time_info *pvclock =addr_gpa2hva (vm, paddr); + + pvclock->version =0 ; + + test_data->op =TEST_OP_MONITOR_ADDRESS ; + test_data->vaddr = vaddr + offsetof(struct pvclock_vcpu_time_info , version); + test_data->expected_val = pvclock->version;
tools/testing/selftests/kvm/x86/memory_attributes.c static void arch_test_memory_access_pte(struct kvm_vcpu *vcpu, gva_t vaddr) · line 357
Quoted 10 lines
tools/testing/selftests/kvm/x86/memory_attributes.c +10
-0+ +vcpu_set_msr (vcpu,MSR_KVM_SYSTEM_TIME_NEW , paddr |0x1 ); +pthread_create (&vcpu_thread,NULL , vcpu_worker, test_data); +usleep (msecs_to_usecs (1000 )); +TEST_ASSERT_EQ (pthread_tryjoin_np (vcpu_thread,NULL ),0 ); +vcpu_set_msr (vcpu,MSR_KVM_SYSTEM_TIME_NEW ,0 ); + + test_data->expected_val = pvclock->version; +vm_set_memory_attributes (vm, paddr, vm->page_size,KVM_MEMORY_ATTRIBUTE_NW ); +vcpu_set_msr (vcpu,MSR_KVM_SYSTEM_TIME_NEW , paddr |0x1 );
[Severity: Medium]
Does deactivating MSR_KVM_SYSTEM_TIME_NEW before setting KVM_MEMORY_ATTRIBUTE_NW prevent this test from exercising invalidation of an already-active gfn_to_pfn_cache?
Writing 0 to MSR_KVM_SYSTEM_TIME_NEW calls kvm_gpc_deactivate(&vcpu->arch.pv_time), which clears gpc->valid and gpc->active before vm_set_memory_attributes() updates the attributes. When paddr | 0x1 is subsequently written to MSR_KVM_SYSTEM_TIME_NEW, kvm_gpc_activate() -> __kvm_gpc_refresh() runs synchronously during KVM_SET_MSRS, sees KVM_MEMORY_ATTRIBUTE_NW already in place, updates gpc->attrs_generation to the current generation, and marks gpc->valid = false.
As a result, when vcpu_worker enters the guest and kvm_setup_guest_pvclock() calls kvm_gpc_check(), the cache is already invalid and its attrs_generation already matches the VM's generation. Would keeping the cache active across vm_set_memory_attributes() (or writing paddr | 0x1 to MSR_KVM_SYSTEM_TIME_NEW to queue KVM_REQ_GLOBAL_CLOCK_UPDATE before calling vm_set_memory_attributes() with KVM_MEMORY_ATTRIBUTE_NW) better test that kvm_gpc_check() detects the generation mismatch on an active, valid cache?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=44
Paolo Bonzini
maintainerPATCH v4 45/45 KVM: x86: split out parts of load_pdptrs
In reply to Paolo BonziniAdd new functions that can be used to read and validate the PDPTRs without placing them in the kvm_vcpu struct. KVM can use them to read all the information that is needed for a nested vmentry or vmexit, and fault out to userspace is needed, prior to committing to completing the vmentry/vmexit.
--- arch/x86/kvm/regs.c | 51 ++++++++++++++++++++++++++++++++------------- arch/x86/kvm/regs.h | 3 +++ 2 files changed, 40 insertions(+), 14 deletions(-)
arch/x86/kvm/regs.c +37 -14
@@ -149,17 +149,29 @@ static inline u64 pdptr_rsvd_bits(struct kvm_vcpu *vcpu)return vcpu->arch.reserved_gpa_bits |rsvd_bits (5 ,8 ) |rsvd_bits (1 ,2 ); } +int kvm_validate_pdptrs (struct kvm_vcpu *vcpu,const u64 *pdptrs) +{ +int i; + +for (i =0 ; i <ARRAY_SIZE (vcpu->arch.pdptrs); ++i) { +if ((pdptrs[i] &PT_PRESENT_MASK ) && + (pdptrs[i] &pdptr_rsvd_bits (vcpu))) +return 0 ; + } +return 1 ; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_validate_pdptrs); +/* -* Load the pae pdptrs. Return 1 if they are all valid, 0 otherwise. +* Read the PAE PDPTRs into @pdptrs. Return 1 if they are valid, 0 if they are +* invalid, and a negative value on error. */ -int load_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3) +int kvm_read_validate_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3,u64 *pdptrs) {struct kvm_pagewalk *w = &vcpu->arch.gva_walk;gfn_t pdpt_gfn = cr3 >>PAGE_SHIFT ;gpa_t real_gpa; -int i;int ret; -u64 pdpte[ARRAY_SIZE (vcpu->arch.pdptrs)];/* * If the MMU is nested, CR3 holds an L2 GPA and needs to be translated
@@ -174,8 +186,8 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3)return 0 ;/* Note the offset, PDPTRs are 32 byte aligned when using PAE paging. */ - ret =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (real_gpa), pdpte, - cr3 &GENMASK (11 ,5 ),sizeof (pdpte)); + ret =kvm_vcpu_read_guest_page (vcpu,gpa_to_gfn (real_gpa), pdptrs, + cr3 &GENMASK (11 ,5 ),sizeof (vcpu->arch.pdptrs));if (ret == -EFAULT ) {kvm_prepare_memory_fault_exit (vcpu, real_gpa &PAGE_MASK ,PAGE_SIZE , false, false, false);
@@ -184,26 +196,37 @@ int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3)if (ret)return 0 ; -for (i =0 ; i <ARRAY_SIZE (pdpte); ++i) { -if ((pdpte[i] &PT_PRESENT_MASK ) && - (pdpte[i] &pdptr_rsvd_bits (vcpu))) { -return 0 ; - } - } +return kvm_validate_pdptrs (vcpu, pdptrs); +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_read_validate_pdptrs); +void kvm_commit_pdptrs (struct kvm_vcpu *vcpu,const u64 *pdptrs) +{/* * Marking VCPU_REG_PDPTR dirty doesn't work for !tdp_enabled. * Shadow page roots need to be reconstructed instead. */ -if (!tdp_enabled &&memcmp (vcpu->arch.pdptrs, pdpte,sizeof (vcpu->arch.pdptrs))) +if (!tdp_enabled &&memcmp (vcpu->arch.pdptrs, pdptrs,sizeof (vcpu->arch.pdptrs)))kvm_mmu_free_roots (vcpu->kvm, &vcpu->arch.root_mmu,KVM_MMU_ROOT_CURRENT ); -memcpy (vcpu->arch.pdptrs, pdpte,sizeof (vcpu->arch.pdptrs)); +memcpy (vcpu->arch.pdptrs, pdptrs,sizeof (vcpu->arch.pdptrs));kvm_register_mark_dirty (vcpu,VCPU_REG_PDPTR );kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); vcpu->arch.pdptrs_from_userspace = false; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_commit_pdptrs); +int load_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3) +{ +u64 pdptrs[ARRAY_SIZE (vcpu->arch.pdptrs)]; +int r; + + r =kvm_read_validate_pdptrs (vcpu, cr3, pdptrs); +if (r <=0 ) +return r; + +kvm_commit_pdptrs (vcpu, pdptrs);return 1 ; }EXPORT_SYMBOL_FOR_KVM_INTERNAL (load_pdptrs);
arch/x86/kvm/regs.h +3 -0
@@ -63,6 +63,9 @@ unsigned long kvm_get_dr(struct kvm_vcpu *vcpu, int dr);unsigned long kvm_get_dr6_fixed_1 (struct kvm_vcpu *vcpu);unsigned long kvm_get_cr8 (struct kvm_vcpu *vcpu);void kvm_lmsw (struct kvm_vcpu *vcpu,unsigned long msw); +int kvm_read_validate_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3,u64 *pdptrs); +int kvm_validate_pdptrs (struct kvm_vcpu *vcpu,const u64 *pdptrs); +void kvm_commit_pdptrs (struct kvm_vcpu *vcpu,const u64 *pdptrs);int load_pdptrs (struct kvm_vcpu *vcpu,unsigned long cr3);static inline bool is_long_mode (struct kvm_vcpu *vcpu)
Signature · 1 line
2.52.0
syzbot ci has tested the following series
syzbot ci has tested the following series
[v4] KVM: x86: Introduce memory protection attributes
https://lore.kernel.org/all/[email protected]
* [PATCH 01/45] KVM: nVMX/nSVM: check unconditionally whether CR3 is valid
* [PATCH 02/45] KVM: x86/hyperv: do not overwrite hc->ingpa for SIGNAL_EVENT hypercall
* [PATCH 03/45] KVM: x86: apply nGPA->GPA translation to KVM_HC_CLOCK_PAIRING
* [PATCH 04/45] KVM: x86/emulate: Check TSS busy-bit writes during task switches
* [PATCH 05/45] KVM: loongarch: do full validity check on the gfn-to-hva cache
* [PATCH 06/45] KVM: selftests: Test address translation for Hyper-V direct L2 hypercalls
* [PATCH 07/45] KVM: selftests: Take into account mixed memory fault flags
* [PATCH 08/45] selftests: kvm: drop TEST_ASSERT_SUPPORTED_ATTRIBUTES
* [PATCH 09/45] KVM: Define and communicate KVM_EXIT_MEMORY_FAULT RWX flags to userspace
* [PATCH 10/45] KVM: x86: Introduce memory fault on invalid hypercalls reads/writes
* [PATCH 11/45] KVM: selftests: test hypercall memory fault exits
* [PATCH 12/45] KVM: x86/mmu: intersect writability from __kvm_faultin_pfn with fault->map_writable
* [PATCH 13/45] KVM: x86/mmu: Extend map_writable to a full ACC_* mask
* [PATCH 14/45] KVM: x86/mmu: Init memslot hugepage information for non-private_mem VMs too
* [PATCH 15/45] KVM: VMX: return error from kvm_set_cr*
* [PATCH 16/45] KVM: adjust for presence of more than one attribute
* [PATCH 17/45] KVM: Introduce NR/NW/NX memory attributes
* [PATCH 18/45] KVM: Include memory protections in result of gfn->hva conversion
* [PATCH 19/45] KVM: Introduce kvm_fetch_guest_page() and use it for x86
* [PATCH 20/45] KVM: Take memory protections into account for memory read/write/fetch
* [PATCH 21/45] KVM: Take memory protections into account for __kvm_vcpu_map
* [PATCH 22/45] KVM: Encapsulate memattrs array into anonymous struct
* [PATCH 23/45] KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen()
* [PATCH 24/45] KVM: Introduce a generation number for memory attributes
* [PATCH 25/45] KVM: Take memory protections into account for accesses with cached gfn->hva
* [PATCH 26/45] KVM: pfncache: Fail to refresh if it contains memory protections
* [PATCH 27/45] KVM: x86/mmu: Do not prefetch sptes on gfns backed by memory attributes
* [PATCH 28/45] KVM: x86/mmu: Take memory protection attributes into account during faults
* [PATCH 29/45] KVM: x86/emulate: change return value of emulator_io_permitted to int
* [PATCH 30/45] KVM: x86/emulate: Propagate memory faults to userspace
* [PATCH 31/45] KVM: x86: "Handle" failed decode when skipping instructions
* [PATCH 32/45] KVM: vmx/sgx: make return value of sgx_gva_to_gpa consistent with its callers
* [PATCH 33/45] KVM: x86: Allow page fault fixup to return errors
* [PATCH 34/45] KVM: x86/mmu: Adjust gva_to_gpa to allow returning an error
* [PATCH 35/45] KVM: x86/mmu: Propagate errors from nested GPA translation
* [PATCH 36/45] KVM: x86: Propagate memory faults through load_pdptrs()
* [PATCH 37/45] KVM: x86/mmu: Issue memory fault exit if walk failed due to memory attribute
* [PATCH 38/45] KVM: x86: mark VM as dead after KVM_SET_SREGS fails to load PDPTRs
* [PATCH 39/45] KVM: VMX: mark VM dead on shadow VMCS or eVMCS access failures
* [PATCH 40/45] KVM: let kvm_arch_post_set_memory_attributes drop mmu_lock
* [PATCH 41/45] KVM: x86/mmu: Obsolete all roots if memattr contains gPTEs
* [PATCH 42/45] KVM: x86: selftests: Introduce memory protection attributes test
* [PATCH 43/45] KVM: x86: selftests: Introduce memory attributes PTE test
* [PATCH 44/45] KVM: x86: selftests: Introduce memory attributes side-channel tests
* [PATCH 45/45] KVM: x86: split out parts of load_pdptrs
and found the following issue:
WARNING: suspicious RCU usage in __kvm_gpc_refresh
Full report is available here:
https://ci.syzbot.org/series/beb5d6f5-3128-404c-9de5-537fb4326658
***
WARNING: suspicious RCU usage in __kvm_gpc_refresh
tree: kvm-next URL: https://kernel.googlesource.com/pub/scm/virt/kvm/kvm/ base: d21eef2084a9af87ac2a783bc7c2b175ba114e4e arch: amd64 compiler: Debian clang version 22.1.8 (1~deb13u4), Debian LLD 22.1.8 config: https://ci.syzbot.org/builds/1748209f-578f-45cd-8bb4-6f6392d79a88/config syz repro: https://ci.syzbot.org/findings/d02cf2a1-ed84-4f69-ac5f-29b2075b5e1f/syz_repro
=============================
WARNING: suspicious RCU usage
syzkaller #0 Not tainted
-----------------------------./include/linux/kvm_host.h:2636 Suspicious memory attribute generation usage !
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
locks held by syz.1.189/6607: 2, last CPU#0:
#0: ffff8881bf19dc38 (&gpc->refresh_lock){+.+.}-{4:4}, at: class_mutex_constructor include/linux/mutex.h:253 [inline] #0: ffff8881bf19dc38 (&gpc->refresh_lock){+.+.}-{4:4}, at: __kvm_gpc_activate+0x91/0x3b0 virt/kvm/pfncache.c:417 #1: ffff8881bf19dbb0 (&gpc->lock){....}-{3:3}, at: __kvm_gpc_refresh+0x1af/0x1830 virt/kvm/pfncache.c:272
stack backtrace: CPU: 0 UID: 0 PID: 6607 Comm: syz.1.189 Not tainted syzkaller #0 PREEMPT(full) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 lockdep_rcu_suspicious+0x140/0x1d0 kernel/locking/lockdep.c:6972 kvm_mem_attributes_generation include/linux/kvm_host.h:2634 [inline] __kvm_gpc_refresh+0x1657/0x1830 virt/kvm/pfncache.c:306 __kvm_gpc_activate+0x289/0x3b0 virt/kvm/pfncache.c:436 kvm_set_msr_common+0x2dab/0x4320 arch/x86/kvm/msrs.c:-1 vmx_set_msr+0x15b3/0x27f0 arch/x86/kvm/vmx/vmx.c:2632 __kvm_set_msr+0x60c/0xab0 arch/x86/kvm/msrs.c:806 _kvm_set_msr arch/x86/kvm/msrs.c:812 [inline] kvm_do_msr_access arch/x86/kvm/msrs.c:549 [inline] kvm_set_msr_ignored_check+0x2a/0x390 arch/x86/kvm/msrs.c:818 nested_vmx_load_msr+0x245/0x4d0 arch/x86/kvm/vmx/nested.c:1126 load_vmcs12_host_state+0x10ca/0x1e20 arch/x86/kvm/vmx/nested.c:4935 __nested_vmx_vmexit+0x1a12/0x2c90 arch/x86/kvm/vmx/nested.c:5203 nested_vmx_vmexit arch/x86/kvm/vmx/nested.h:46 [inline] vmx_leave_nested arch/x86/kvm/vmx/nested.c:6898 [inline] nested_vmx_free_vcpu+0x8e/0xd0 arch/x86/kvm/vmx/nested.c:389 vmx_vcpu_free+0x109/0x3e0 arch/x86/kvm/vmx/vmx.c:7536 kvm_arch_vcpu_destroy+0x154/0x380 arch/x86/kvm/x86.c:9627 kvm_vcpu_destroy virt/kvm/kvm_main.c:477 [inline] kvm_destroy_vcpus+0x123/0x380 virt/kvm/kvm_main.c:497 kvm_arch_destroy_vm+0xf6/0x1d0 arch/x86/kvm/x86.c:10198 kvm_destroy_vm virt/kvm/kvm_main.c:1314 [inline] kvm_put_kvm+0x772/0xb50 virt/kvm/kvm_main.c:1351 kvm_vm_release+0x6b/0x80 virt/kvm/kvm_main.c:1376 __fput+0x40d/0xa50 fs/file_table.c:512 task_work_run+0x1d9/0x270 kernel/task_work.c:233 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline] __exit_to_user_mode_loop kernel/entry/common.c:70 [inline] exit_to_user_mode_loop+0x204/0x770 kernel/entry/common.c:101 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline] syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:230 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:343 [inline] do_syscall_64+0x328/0x520 arch/x86/entry/syscall_64.c:89 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7fe8f039f559 Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007fff5710adc8 EFLAGS: 00000246 ORIG_RAX: 00000000000001b4 RAX: 0000000000000000 RBX: 00007fe8f0627da0 RCX: 00007fe8f039f559 RDX: 0000000000000000 RSI: 000000000000001e RDI: 0000000000000003 RBP: 00007fe8f0627da0 R08: 00007fe8f0626038 R09: 0000000000000000 R10: 000000000003fd98 R11: 0000000000000246 R12: 00000000000213f0 R13: 00007fe8f062609c R14: 000000000002118b R15: 00007fe8f0626090 </TASK>
***
If these findings have caused you to resend the series or submit a separate fix, please add the following tag to your commit message:
---
This report is generated by a bot. It may contain errors. syzbot ci engineers can be reached at [email protected].
To test a fix for this bug, please reply with #syz test (on a separate line) and attach the patch to the email.
Notes:
- The patch will be applied on top of the tested series (as an incremental fix).
- To test a new version of the whole series, please send it directly to [email protected].
- Arguments like custom git repos and branches are not supported.