KVM: x86: Don't reuse the GPA of a fault on the guest's page tables
PATCH 1/1 KVM: x86: Don't reuse the GPA of a fault on the guest's page tables
When a #NPF or EPT violation leads to emulation, x86_emulate_instruction() saves the faulting GPA for direct MMUs, and emulator_read_write_onepage() uses it instead of walking the guest's page tables for an access whose page offset matches. If the fault occurred while the CPU was walking the guest's page tables (PFERR_GUEST_PAGE_MASK), the GPA is that of a guest paging structure, not of the access being emulated, and the access goes to the wrong address.
KVM selftest set_memory_region_test runs into this on AMD. test_delete_memory_region() deletes the memslot holding the guest's page tables while the guest spins on a read of 0xc0000000 that userspace handles as MMIO. If the vCPU is at the read when the memslot goes away, the CPU takes a #NPF on the guest's root page table at 0x180000. DecodeAssist provides the instruction bytes, the read of 0xc0000000 has the same page offset as 0x180000, and the emulator turns it into an MMIO read of 0x180000:
==== Test Assertion Failure ====
set_memory_region_test.c:83: run->mmio.phys_addr == MEM_REGION_GPA
...
Unexpected exit mmio address = 0x180000If the vCPU is at the next instruction instead, the #NPF happens on the code fetch, DecodeAssist provides no bytes, KVM fails to fetch the instruction itself and injects a #PF, and the guest shuts down as the test expects. KernelCI sees the failure in about half of the runs on a DL345 Gen11 (EPYC).
The CPU walks the guest's page tables with write accesses, so such faults also set EMULTYPE_PF_WRITE, and a write would be redirected the same way.
Tell the emulator when the fault occurred on the guest's page tables and don't use the GPA for the access then, so that the access is translated through the guest's page tables. KVM can't walk the encrypted page tables of SEV guests, so resume an SEV guest instead of emulating, as is done when the #NPF occurred on the instruction fetch.
---
Reproduced on an HPE DL345 Gen11 (EPYC 9655P) with KernelCI's x86 build of mainline d135ab453a74: set_memory_region_test failed in 15 of 30 runs, and in 6 of 12 runs with tracing, always with the MMIO exit at 0x180000. The traces show the #NPF at 0x180000 with error code 0x200000006 on the cmpq that reads 0xc0000000, followed by an MMIO read of 0x180000.
With this patch on the same config, the test passed 50 of 50 runs. When the #NPF hits the cmpq, KVM now injects a #PF for the read of 0xc0000000 and the guest shuts down. The rest of the KVM selftests give the same results as without the patch, with and without kvm.force_emulation_prefix=1. Not tested on SEV or Intel.
arch/x86/kvm/mmu/mmu.c | 2 ++ arch/x86/kvm/svm/svm.c | 9 +++++++++ arch/x86/kvm/x86.c | 10 ++++++++-- arch/x86/kvm/x86.h | 6 ++++++ 4 files changed, 25 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu/mmu.c +2 -0
@@ -6719,6 +6719,8 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 err emulate:if (error_code &PFERR_WRITE_MASK ) emulation_type |=EMULTYPE_PF_WRITE ; +if (error_code &PFERR_GUEST_PAGE_MASK ) + emulation_type |=EMULTYPE_PF_GUEST_PAGE ;return x86_emulate_instruction (vcpu, cr2_or_gpa, emulation_type, insn, insn_len);
arch/x86/kvm/svm/svm.c +9 -0
@@ -5324,6 +5324,15 @@ static int svm_check_emulate_instruction(struct kvm_vcpu *vcpu, int emul_type,return X86EMUL_PROPAGATE_FAULT ; } +/* +* Don't emulate if the #NPF occurred while walking the guest's page +* tables. The GPA is that of a guest paging structure, not of the +* access, and KVM can't walk the guest's encrypted page tables to +* translate the access itself. Resume the guest, see below. +*/ +if (emul_type &EMULTYPE_PF_GUEST_PAGE ) + goto resume_guest; +/* * Emulate for SEV guests if the insn buffer is not empty. The buffer * will be empty if the DecodeAssist microcode cannot fetch bytes for
arch/x86/kvm/x86.c +8 -2
@@ -6495,8 +6495,14 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,/* Save the faulting GPA (cr2) in the address field */ ctxt->exception.address = cr2_or_gpa; -/* With shadow page tables, cr2 contains a GVA or nGPA. */ -if (vcpu->arch.mmu->root_role.direct) { +/* +* With shadow page tables, cr2 contains a GVA or nGPA. If the +* fault occurred while walking the guest's page tables, the +* GPA is that of a guest paging structure, not of the access +* that needs to be emulated, and can't be used for the access. +*/ +if (vcpu->arch.mmu->root_role.direct && + !(emulation_type &EMULTYPE_PF_GUEST_PAGE )) { ctxt->gpa_access =ACC_READ_MASK ;/* * Always allow writes for guests with protected page
arch/x86/kvm/x86.h +6 -0
@@ -410,6 +410,11 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,* * EMULTYPE_PF_WRITE - Set with EMULTYPE_PF when hardware reports a write access. * +* EMULTYPE_PF_GUEST_PAGE - Set with EMULTYPE_PF when hardware reports that the +* fault occurred while walking the guest's page +* tables, i.e. the GPA is that of a guest paging +* structure, not of the access being emulated. +* * EMULTYPE_COMPLETE_USER_EXIT - Set when the emulator should update interruptibility * state and inject single-step #DBs after skipping * an instruction (after completing userspace I/O).
@@ -449,6 +454,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,#define EMULTYPE_WRITE_PF_TO_SP (1 << 8)#define EMULTYPE_SKIP_SOFT_INT (1 << 9)#define EMULTYPE_PF_WRITE (1 << 10) +#define EMULTYPE_PF_GUEST_PAGE (1 << 11)#define EMULTYPE_SET_SOFT_INT_VECTOR (v) ((u32)((v) & 0xff) << 16)#define EMULTYPE_GET_SOFT_INT_VECTOR (e) (((e) >> 16) & 0xff)
Signature · 1 line
2.53.0
Hi Sean, Paolo,
Ping. Did you get a chance to look at this?
It still applies to kvm-x86/next and to mainline, and the bug still reproduces :)
If you'd rather fix it some other way, let me know.
Signature · 2 lines
Thanks, Sasha
Paolo Bonzini
maintainerQuoted 4 lines · Sasha Levin
Hi Sean, Paolo,
Ping. Did you get a chance to look at this?
I am looking now. Sorry for the delay, I was busy with KVM Forum. :)
Paolo
Paolo Bonzini
maintainerQuoted 15 lines · Sasha Levin
DecodeAssist provides the instruction bytes, the read of
0xc0000000has the same page offset as0x180000, and the emulator turns it into an MMIO read of0x180000:==== Test Assertion Failure ==== set_memory_region_test.c:83: run->mmio.phys_addr == MEM_REGION_GPA ... Unexpected exit mmio address = 0x180000If the vCPU is at the next instruction instead, the #NPF happens on the code fetch, DecodeAssist provides no bytes, KVM fails to fetch the instruction itself and injects a #PF, and the guest shuts down as the test expects. KernelCI sees the failure in about half of the runs on a DL345 Gen11 (EPYC).
Quoted 6 lines
The CPU walks the guest's page tables with write accesses, so such faults also set
EMULTYPE_PF_WRITE, and a write would be redirected the same way.Tell the emulator when the fault occurred on the guest's page tables and don't use the GPA for the access then, so that the access is translated through the guest's page tables. KVM can't walk the encrypted page tables
^^^^^^^^^
This is a separate bug; something like "Also, KVM can't walk..." would be better - but actually a separate patch would be even better.
The non-SEV part looks good. I'll wait for Sean to shout that it isn't, or otherwise apply it and send out the SEV patch below.
Review
Quoted 8 lines
(quoted diff) +8
-0+ /* + * Don't emulate if the #NPF occurred while walking the guest's page + * tables. The GPA is that of a guest paging structure, not of the + * access, and KVM can't walk the guest's encrypted page tables to + * translate the access itself. Resume the guest, see below. + */ + if (emul_type & EMULTYPE_PF_GUEST_PAGE) + goto resume_guest;
Here I would return X86EMUL_UNHANDLEABLE, which causes an internal-error exit or #UD. Also, even though it's mostly aesthetic, SEV only gets emulation for #PF and the code is already consulting exit_info_1. In fact it is already checking PFERR_GUEST_PAGE_MASK but in the wrong order. Something like this will do:
arch/x86/kvm/svm/svm.c +18 -12
@@ -5513,19 +5513,29 @@ static int svm_check_emulate_instruction( }/* -* Emulate for SEV guests if the insn buffer is not empty. The buffer -* will be empty if the DecodeAssist microcode cannot fetch bytes for -* the faulting instruction because the code fetch itself faulted, e.g. -* the guest attempted to fetch from emulated MMIO or a guest page -* table used to translate CS:RIP resides in emulated MMIO. +* Under SEV, page tables are encrypted and cannot be read by the +* hypervisor, so there is no way to resolve a fault on page tables. */ + error_code = svm->vmcb->control.exit_info_1; +if (error_code &PFERR_GUEST_PAGE_MASK ) +return X86EMUL_UNHANDLEABLE ; +if (likely (insn_len))return X86EMUL_CONTINUE ;/* -* Detect and workaround Errata 1096 Fam_17h_00_0Fh. +* The insn buffer was empty; the DecodeAssist microcode could not fetch +* bytes for the faulting instruction because the code fetch itself faulted, +* e.g. the guest attempted to fetch from emulated MMIO or a guest page +* table used to translate CS:RIP resides in emulated MMIO. Just retry. +*/ +if (error_code &PFERR_FETCH_MASK ) + goto resume_guest; + +/* +* A data fault could actually be Errata 1096 Fam_17h_00_0Fh. +* Detect it to apply a workaround. * -* Errata: * When CPU raises #NPF on guest data access and vCPU CR4.SMAP=1, it is * possible that CPU microcode implementing DecodeAssist will fail to * read guest memory at CS:RIP and vmcb.GuestIntrBytes will incorrectly
@@ -5552,12 +5562,8 @@ static int svm_check_emulate_instruction(* retrieve the instruction bytes is legitimate (see abvoe). * * In addition, don't apply the erratum workaround if the #NPF occurred -* while translating guest page tables (see below). +* while translating guest page tables (see above). */ - error_code = svm->vmcb->control.exit_info_1; -if (error_code & (PFERR_GUEST_PAGE_MASK |PFERR_FETCH_MASK )) - goto resume_guest; - smep =kvm_is_cr4_bit_set (vcpu,X86_CR4_SMEP ); smap =kvm_is_cr4_bit_set (vcpu,X86_CR4_SMAP ); is_user =svm_get_cpl (vcpu) ==3 ;
Paolo