KVM: arm64: Fixes to early stage-2 fault handling
Will Deacon
maintainercover KVM: arm64: Fixes to early stage-2 fault handling
Hi folks,
Continuing my journey into the KVM stage-2 page-table code, here are some fixes for a bunch of issues I spotted purely by code inspection. Most of these involve really unusual scenarios, but I'm a bit worried about the stage-2 fault on stage-1 page-table walk during instruction fetch from a read-only memslot, as that feels like it might be hittable with EFI.
Anyway, feedback welcome, especially as this is a user-visible change.
Cheers,
Will
--->8
Will Deacon (7):
KVM: arm64: Update comment when skipping guest MMIO access instruction
KVM: arm64: Rename kvm_vcpu_dabt_isextabt()
KVM: arm64: Handle data and instruction external aborts the same way
KVM: arm64: Remove useless local variable
KVM: arm64: Move 'invalid syndrome' logic out of io_mem_abort()
KVM: arm64: Handle stage-2 faults on stage-1 page-table walks earlier
KVM: arm64: Separate write faults on read-only memslots from MMIO
arch/arm64/include/asm/kvm_emulate.h | 2 +- arch/arm64/kvm/hyp/switch.c | 2 +- arch/arm64/kvm/mmio.c | 29 +++------- arch/arm64/kvm/mmu.c | 87 +++++++++++++++++++--------- 4 files changed, 69 insertions(+), 51 deletions(-)
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerPATCH 1/7 KVM: arm64: Update comment when skipping guest MMIO access instruction
If a 32-bit guest accesses MMIO using a 16-bit Thumb-2 instruction that is reported to the hypervisor without a valid syndrom (for example, because of the addressing mode), then we may hand off the fault to userspace. When resuming the guest, we unconditionally advance the PC by 4 bytes, since ESR_EL2.IL is always 1 for data aborts generated without a valid syndrome. This is a bit rubbish, but it's also difficult to see how we can fix it without potentially introducing regressions in userspace MMIO fault handling.
Update the comment when skipping a guest MMIO access instruction so that this corner case is at least written down.
--- arch/arm64/kvm/mmio.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/kvm/mmio.c +7 -0
@@ -113,6 +113,13 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run)/* * The MMIO instruction is emulated and should not be re-executed * in the guest. +* +* Note: If user space handled the emulation because the abort +* symdrome information was not valid (ISV set in the ESR), then +* this will assume that the faulting instruction was 32-bit. +* If the faulting instruction was a 16-bit Thumb instruction, +* then userspace would need to rewind the PC by 2 bytes prior to +* resuming the vCPU (yuck!). */ kvm_skip_instr (vcpu,kvm_vcpu_trap_il_is32bit (vcpu));
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH 1/7] KVM: arm64: Update comment when skipping guest MMIO access instruction
Quoted 9 lines · Will Deacon
If a 32-bit guest accesses MMIO using a 16-bit Thumb-2 instruction that is reported to the hypervisor without a valid syndrom (for example, because of the addressing mode), then we may hand off the fault to userspace. When resuming the guest, we unconditionally advance the PC by 4 bytes, since ESR_EL2.IL is always 1 for data aborts generated without a valid syndrome. This is a bit rubbish, but it's also difficult to see how we can fix it without potentially introducing regressions in userspace MMIO fault handling.
Not quite, see below.
Quoted 16 lines
Update the comment when skipping a guest MMIO access instruction so that this corner case is at least written down.
Cc Marc Zyngier <[email protected]>Cc Quentin Perret <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/kvm/mmio.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kvm/mmio.c +0
-0
arch/arm64/kvm/mmio.c int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run) · line 118
Quoted 6 lines
arch/arm64/kvm/mmio.c +3
-0@@ -113,6 +113,13 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run)/* * The MMIO instruction is emulated and should not be re-executed * in the guest. +* +* Note: If user space handled the emulation because the abort +* symdrome information was not valid (ISV set in the ESR), then
nits: syndrome, ISV *clear*.
arch/arm64/kvm/mmio.c int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run) · line 125
Quoted 7 lines
arch/arm64/kvm/mmio.c +4
-0+* this will assume that the faulting instruction was 32-bit. +* If the faulting instruction was a 16-bit Thumb instruction, +* then userspace would need to rewind the PC by 2 bytes prior to +* resuming the vCPU (yuck!). */ kvm_skip_instr (vcpu,kvm_vcpu_trap_il_is32bit (vcpu));
That's not how I read it. On ESR_EL2.ISV being clear, and in the absence of KVM_CAP_ARM_NISV_TO_USER being set, we return a -ENOSYS from io_mem_abort(), exiting back to userspace *without* advertising a MMIO access. The VMM is free to do whatever it can to handle it (i.e. not much), but crucially we don't go via kvm_handle_mmio_return() on resuming the vcpu (unless the VMM sets run->exit_reason to KVM_EXIT_MMIO, but that's clearly its own decision).
Instead, the expectation is that userspace willing to handle an exit resulting in ESR_EL2.ISV being clear would instead request a KVM_EXIT_ARM_NISV exit type (by enabling KVM_CAP_ARM_NISV_TO_USER), getting extra information in the process such as as the fault IPA). KVM_EXIT_ARM_NISV clearly states in the documentation:
"Note that KVM does not skip the faulting instruction as it does for
KVM_EXIT_MMIO, but userspace has to emulate any change to the processing state if it decides to decode and emulate the instruction."
Thanks,
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerRe: [PATCH 1/7] KVM: arm64: Update comment when skipping guest MMIO access instruction
Quoted 25 lines · Marc Zyngier
Quoted 9 lines · Will Deacon
If a 32-bit guest accesses MMIO using a 16-bit Thumb-2 instruction that is reported to the hypervisor without a valid syndrom (for example, because of the addressing mode), then we may hand off the fault to userspace. When resuming the guest, we unconditionally advance the PC by 4 bytes, since ESR_EL2.IL is always 1 for data aborts generated without a valid syndrome. This is a bit rubbish, but it's also difficult to see how we can fix it without potentially introducing regressions in userspace MMIO fault handling.
Not quite, see below.
Quoted 11 lines
Update the comment when skipping a guest MMIO access instruction so that this corner case is at least written down.
Cc Marc Zyngier <[email protected]>Cc Quentin Perret <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/kvm/mmio.c | 7 +++++++ 1 file changed, 7 insertions(+)
Quoted 13 lines
arch/arm64/kvm/mmio.c int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run) · line 118
Quoted 11 lines
arch/arm64/kvm/mmio.c +3
-0@@ -113,6 +113,13 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run)/* * The MMIO instruction is emulated and should not be re-executed * in the guest. +* +* Note: If user space handled the emulation because the abort +* symdrome information was not valid (ISV set in the ESR), then nits: syndrome, ISV *clear*.
Duh, thanks.
Quoted 22 lines
arch/arm64/kvm/mmio.c int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run) · line 125
Quoted 7 lines
arch/arm64/kvm/mmio.c +4
-0+* this will assume that the faulting instruction was 32-bit. +* If the faulting instruction was a 16-bit Thumb instruction, +* then userspace would need to rewind the PC by 2 bytes prior to +* resuming the vCPU (yuck!). */ kvm_skip_instr (vcpu,kvm_vcpu_trap_il_is32bit (vcpu));That's not how I read it. On ESR_EL2.ISV being clear, and in the absence of
KVM_CAP_ARM_NISV_TO_USERbeing set, we return a-ENOSYSfromio_mem_abort(), exiting back to userspace *without* advertising a MMIO access. The VMM is free to do whatever it can to handle it (i.e. not much), but crucially we don't go viakvm_handle_mmio_return()on resuming the vcpu (unless the VMM setsrun->exit_reasontoKVM_EXIT_MMIO, but that's clearly its own decision).Instead, the expectation is that userspace willing to handle an exit resulting in ESR_EL2.ISV being clear would instead request a
KVM_EXIT_ARM_NISVexit type (by enablingKVM_CAP_ARM_NISV_TO_USER), getting extra information in the process such as as the fault IPA).KVM_EXIT_ARM_NISVclearly states in the documentation:
Quoted 3 lines
"Note that KVM does not skip the faulting instruction as it does for
KVM_EXIT_MMIO, but userspace has to emulate any change to the processing state if it decides to decode and emulate the instruction."
Thanks, I think you're right. I _thought_ we always reported EXIT_MMIO for write faults on read-only memslots (as per the documented behaviour), but actually that goes down the io_mem_abort() path too and so the skipping only ever occurs when the syndrome is valid.
I'll drop this patch.
Will
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerPATCH 2/7 KVM: arm64: Rename kvm_vcpu_dabt_isextabt()
In reply to Will Deaconkvm_vcpu_dabt_isextabt() is not specific to data aborts and has nothing to do with sign extension.
Rename it.
--- arch/arm64/include/asm/kvm_emulate.h | 2 +- arch/arm64/kvm/hyp/switch.c | 2 +- arch/arm64/kvm/mmu.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +1 -1
@@ -366,7 +366,7 @@ static __always_inline u8 kvm_vcpu_trap_get_fault_type(const struct kvm_vcpu *vcreturn kvm_vcpu_get_hsr (vcpu) & ESR_ELx_FSC_TYPE; } -static __always_inline bool kvm_vcpu_dabt_isextabt(const struct kvm_vcpu *vcpu) +static __always_inline bool kvm_vcpu_abt_is_extabt(const struct kvm_vcpu *vcpu) {switch (kvm_vcpu_trap_get_fault (vcpu)) {case FSC_SEA :
arch/arm64/kvm/hyp/switch.c +1 -1
@@ -594,7 +594,7 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) valid =kvm_vcpu_trap_get_class (vcpu) == ESR_ELx_EC_DABT_LOW &&kvm_vcpu_trap_get_fault_type (vcpu) ==FSC_FAULT &&kvm_vcpu_dabt_isvalid (vcpu) && - !kvm_vcpu_dabt_isextabt (vcpu) && + !kvm_vcpu_abt_is_extabt (vcpu) && !kvm_vcpu_dabt_iss1tw (vcpu);if (valid) {
arch/arm64/kvm/mmu.c +1 -1
@@ -2074,7 +2074,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) is_iabt =kvm_vcpu_trap_is_iabt (vcpu);/* Synchronous External Abort? */ -if (kvm_vcpu_dabt_isextabt (vcpu)) { +if (kvm_vcpu_abt_is_extabt (vcpu)) {/* * For RAS the host kernel may handle this abort. * There is no need to pass the error into the guest.
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH 2/7] KVM: arm64: Rename kvm_vcpu_dabt_isextabt()
Quoted 3 lines · Will Deacon
kvm_vcpu_dabt_isextabt()is not specific to data aborts and has nothing to do with sign extension.
Not sign extension, but external abort (it reads as "is external abort?").
This is in keeping with all the other helpers (kvm_vcpu_foo_isbar). If you want to drop the "data" part (which is indeed not correct), how about kvm_vcpu_abt_isexternal? or kvm_vcpu_abt_issea?
Thanks,
M.
Quoted 55 lines
Rename it.
Cc Marc Zyngier <[email protected]>Cc Quentin Perret <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 2 +- arch/arm64/kvm/hyp/switch.c | 2 +- arch/arm64/kvm/mmu.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)arch/arm64/include/asm/kvm_emulate.h +1
-1@@ -366,7 +366,7 @@ static __always_inline u8 kvm_vcpu_trap_get_fault_type(const struct kvm_vcpu *vcreturn kvm_vcpu_get_hsr (vcpu) & ESR_ELx_FSC_TYPE; } -static __always_inline bool kvm_vcpu_dabt_isextabt(const struct kvm_vcpu *vcpu) +static __always_inline bool kvm_vcpu_abt_is_extabt(const struct kvm_vcpu *vcpu) {switch (kvm_vcpu_trap_get_fault (vcpu)) {case FSC_SEA :arch/arm64/kvm/hyp/switch.c +1
-1@@ -594,7 +594,7 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) valid =kvm_vcpu_trap_get_class (vcpu) == ESR_ELx_EC_DABT_LOW &&kvm_vcpu_trap_get_fault_type (vcpu) ==FSC_FAULT &&kvm_vcpu_dabt_isvalid (vcpu) && - !kvm_vcpu_dabt_isextabt (vcpu) && + !kvm_vcpu_abt_is_extabt (vcpu) && !kvm_vcpu_dabt_iss1tw (vcpu);if (valid) {arch/arm64/kvm/mmu.c +1
-1@@ -2074,7 +2074,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) is_iabt =kvm_vcpu_trap_is_iabt (vcpu);/* Synchronous External Abort? */ -if (kvm_vcpu_dabt_isextabt (vcpu)) { +if (kvm_vcpu_abt_is_extabt (vcpu)) {/* * For RAS the host kernel may handle this abort. * There is no need to pass the error into the guest. Signature · 1 line
2.28.0.rc0.142.g3c755180ce-goog
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerRe: [PATCH 2/7] KVM: arm64: Rename kvm_vcpu_dabt_isextabt()
Quoted 6 lines · Marc Zyngier
Quoted 3 lines · Will Deacon
kvm_vcpu_dabt_isextabt()is not specific to data aborts and has nothing to do with sign extension.
Quoted 6 lines
Not sign extension, but external abort (it reads as "is external abort?").
This is in keeping with all the other helpers (
kvm_vcpu_foo_isbar). If you want to drop the "data" part (which is indeed not correct), how aboutkvm_vcpu_abt_isexternal? orkvm_vcpu_abt_issea?
kvm_vcpu_abt_issea() would be great -- the problem is that I confused kvm_vcpu_dabt_issext() and kvm_vcpu_dabt_isextabt() when I was hacking and if you get the wrong one then you're really hosed!
I'll update for v2.
Cheers,
Will
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerPATCH 3/7 KVM: arm64: Handle data and instruction external aborts the same way
In reply to Will DeaconIf the guest generates a synchronous external abort which is not handled by the host, we inject it back into the guest as a virtual SError, but only if the original fault was reported on the data side. Instruction faults are reported as "Unsupported FSC", causing the vCPU run loop to bail with -EFAULT.
Although synchronous external aborts from a guest are pretty unusual, treat them the same regardless of whether they are taken as data or instruction aborts by EL2.
--- arch/arm64/kvm/mmu.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
arch/arm64/kvm/mmu.c +3 -6
@@ -2079,13 +2079,10 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run)* For RAS the host kernel may handle this abort. * There is no need to pass the error into the guest. */ -if (!kvm_handle_guest_sea (fault_ipa,kvm_vcpu_get_hsr (vcpu))) -return 1 ; - -if (unlikely (!is_iabt)) { +if (kvm_handle_guest_sea (fault_ipa,kvm_vcpu_get_hsr (vcpu)))kvm_inject_vabt (vcpu); -return 1 ; - } + +return 1 ; } trace_kvm_guest_fault(*vcpu_pc(vcpu), kvm_vcpu_get_hsr(vcpu),
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerPATCH 4/7 KVM: arm64: Remove useless local variable
In reply to Will Deacon'is_iabt' is used exactly once in kvm_handle_guest_abort(). Remove it, and inlined its definition instead.
--- arch/arm64/kvm/mmu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
arch/arm64/kvm/mmu.c +2 -3
@@ -2064,14 +2064,13 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run)phys_addr_t fault_ipa;struct kvm_memory_slot *memslot;unsigned long hva; -bool is_iabt, write_fault, writable; +bool write_fault, writable;gfn_t gfn;int ret, idx; fault_status =kvm_vcpu_trap_get_fault_type (vcpu); fault_ipa =kvm_vcpu_get_fault_ipa (vcpu); - is_iabt =kvm_vcpu_trap_is_iabt (vcpu);/* Synchronous External Abort? */ if (kvm_vcpu_abt_is_extabt (vcpu)) {
@@ -2105,7 +2104,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) hva =gfn_to_hva_memslot_prot (memslot, gfn, &writable); write_fault =kvm_is_write_fault (vcpu);if (kvm_is_error_hva (hva) || (write_fault && !writable)) { -if (is_iabt) { +if (kvm_vcpu_trap_is_iabt (vcpu)) {/* Prefetch Abort on I/O address */ ret = -ENOEXEC ; goto out;
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerPATCH 5/7 KVM: arm64: Move 'invalid syndrome' logic out of io_mem_abort()
In reply to Will DeaconIn preparation for handling stage-2 faults on stage-1 page-table walks earlier, move the 'invalid syndrome' logic out of io_mem_abort() and into its own function, which can be called from kvm_handle_guest_abort() directly.
--- arch/arm64/kvm/mmio.c | 22 ---------------------- arch/arm64/kvm/mmu.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 22 deletions(-)
arch/arm64/kvm/mmio.c +0 -22
@@ -136,28 +136,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,int len;u8 data_buf[8 ]; -/* -* No valid syndrome? Ask userspace for help if it has -* volunteered to do so, and bail out otherwise. -*/ -if (!kvm_vcpu_dabt_isvalid (vcpu)) { -if (vcpu->kvm->arch.return_nisv_io_abort_to_user) { - run->exit_reason =KVM_EXIT_ARM_NISV ; - run->arm_nisv.esr_iss =kvm_vcpu_dabt_iss_nisv_sanitized (vcpu); - run->arm_nisv.fault_ipa = fault_ipa; -return 0 ; - } - -kvm_pr_unimpl ("Data abort outside memslots with no valid syndrome info\n" ); -return -ENOSYS ; - } - -/* Page table accesses IO mem: tell guest to fix its TTBR */ -if (kvm_vcpu_dabt_iss1tw (vcpu)) { -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); -return 1 ; - } -/* * Prepare MMIO operation. First decode the syndrome data we get * from the CPU. Then try if some in-kernel emulation feels
arch/arm64/kvm/mmu.c +32 -0
@@ -2046,6 +2046,20 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)kvm_set_pfn_accessed (pfn); } +static int handle_error_invalid_dabt (struct kvm_vcpu *vcpu,struct kvm_run *run, +phys_addr_t fault_ipa) +{ +if (!vcpu->kvm->arch.return_nisv_io_abort_to_user) { +kvm_pr_unimpl ("Data abort outside memslots with no valid syndrome info\n" ); +return -ENOSYS ; + } + + run->exit_reason =KVM_EXIT_ARM_NISV ; + run->arm_nisv.esr_iss =kvm_vcpu_dabt_iss_nisv_sanitized (vcpu); + run->arm_nisv.fault_ipa = fault_ipa; +return 0 ; +} +/** * kvm_handle_guest_abort - handles all 2nd stage aborts * @vcpu: the VCPU pointer
@@ -2133,6 +2147,21 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run)* of the page size. */ fault_ipa |=kvm_vcpu_get_hfar (vcpu) & ((1 <<12 ) -1 ); + +/* +* No valid syndrome? Ask userspace for help if it has +* volunteered to do so, and bail out otherwise. +*/ +if (!kvm_vcpu_dabt_isvalid (vcpu)) { + ret =handle_error_invalid_dabt (vcpu, run, fault_ipa); + goto out_unlock; + } + +if (kvm_vcpu_dabt_iss1tw (vcpu)) { + ret = -ENXIO ; + goto out; + } + ret =io_mem_abort (vcpu, run, fault_ipa); goto out_unlock; }
@@ -2153,6 +2182,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run)if (ret == -ENOEXEC ) {kvm_inject_pabt (vcpu,kvm_vcpu_get_hfar (vcpu)); ret =1 ; + }else if (ret == -ENXIO ) { +kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); + ret =1 ; } out_unlock:srcu_read_unlock (&vcpu->kvm->srcu, idx);
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH 5/7] KVM: arm64: Move 'invalid syndrome' logic out of io_mem_abort()
Quoted 52 lines · Will Deacon
In preparation for handling stage-2 faults on stage-1 page-table walks earlier, move the 'invalid syndrome' logic out of
io_mem_abort()and into its own function, which can be called fromkvm_handle_guest_abort()directly.Cc Marc Zyngier <[email protected]>Cc Quentin Perret <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/kvm/mmio.c | 22 ---------------------- arch/arm64/kvm/mmu.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 22 deletions(-)arch/arm64/kvm/mmio.c +0
-22@@ -136,28 +136,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,int len;u8 data_buf[8 ]; -/* -* No valid syndrome? Ask userspace for help if it has -* volunteered to do so, and bail out otherwise. -*/ -if (!kvm_vcpu_dabt_isvalid (vcpu)) { -if (vcpu->kvm->arch.return_nisv_io_abort_to_user) { - run->exit_reason =KVM_EXIT_ARM_NISV ; - run->arm_nisv.esr_iss =kvm_vcpu_dabt_iss_nisv_sanitized (vcpu); - run->arm_nisv.fault_ipa = fault_ipa; -return 0 ; - } - -kvm_pr_unimpl ("Data abort outside memslots with no valid syndrome info\n" ); -return -ENOSYS ; - } - -/* Page table accesses IO mem: tell guest to fix its TTBR */ -if (kvm_vcpu_dabt_iss1tw (vcpu)) { -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); -return 1 ; - } -/* * Prepare MMIO operation. First decode the syndrome data we get * from the CPU. Then try if some in-kernel emulation feels arch/arm64/kvm/mmu.c +0
-0
arch/arm64/kvm/mmu.c static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) · line 2049
Quoted 4 lines
arch/arm64/kvm/mmu.c +1
-0@@ -2046,6 +2046,20 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)kvm_set_pfn_accessed (pfn); } +static int handle_error_invalid_dabt(struct kvm_vcpu *vcpu, struct kvm_run *run,
Nit: why the "_error_"? There isn't any error here, only an awkward part of the architecture. I'd rather see something like handle_nisv_dabt(), which matches what this function actually does.
Quoted 52 lines
arch/arm64/kvm/mmu.c +31
-0+ phys_addr_t fault_ipa) +{ +if (!vcpu->kvm->arch.return_nisv_io_abort_to_user) { +kvm_pr_unimpl ("Data abort outside memslots with no valid syndrome info\n" ); +return -ENOSYS ; + } + + run->exit_reason =KVM_EXIT_ARM_NISV ; + run->arm_nisv.esr_iss =kvm_vcpu_dabt_iss_nisv_sanitized (vcpu); + run->arm_nisv.fault_ipa = fault_ipa; +return 0 ; +} +/** * kvm_handle_guest_abort - handles all 2nd stage aborts * @vcpu: the VCPU pointer @@ -2133,6 +2147,21 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run)* of the page size. */ fault_ipa |=kvm_vcpu_get_hfar (vcpu) & ((1 <<12 ) -1 ); + +/* +* No valid syndrome? Ask userspace for help if it has +* volunteered to do so, and bail out otherwise. +*/ +if (!kvm_vcpu_dabt_isvalid (vcpu)) { + ret =handle_error_invalid_dabt (vcpu, run, fault_ipa); + goto out_unlock; + } + +if (kvm_vcpu_dabt_iss1tw (vcpu)) { + ret = -ENXIO ; + goto out; + } + ret =io_mem_abort (vcpu, run, fault_ipa); goto out_unlock; }@@ -2153,6 +2182,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run)if (ret == -ENOEXEC ) {kvm_inject_pabt (vcpu,kvm_vcpu_get_hfar (vcpu)); ret =1 ; + }else if (ret == -ENXIO ) { +kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); + ret =1 ; } out_unlock:srcu_read_unlock (&vcpu->kvm->srcu, idx);Signature · 1 line
2.28.0.rc0.142.g3c755180ce-goog
Otherwise looks OK.
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerRe: [PATCH 5/7] KVM: arm64: Move 'invalid syndrome' logic out of io_mem_abort()
Quoted 10 lines · Marc Zyngier
Quoted 8 lines · Will Deacon
arch/arm64/kvm/mmu.c +0
-0@@ -2046,6 +2046,20 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)kvm_set_pfn_accessed (pfn); }
Quoted 5 lines
arch/arm64/kvm/mmu.c static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) · line 2049
Quoted 1 line
arch/arm64/kvm/mmu.c +1
-0+static int handle_error_invalid_dabt(struct kvm_vcpu *vcpu, struct kvm_run *run,Nit: why the "_error_"? There isn't any error here, only an awkward part of the architecture. I'd rather see something like
handle_nisv_dabt(), which matches what this function actually does.
I chose "_error_" because this handling the case when kvm_is_error_hva() is true (but I agree that "error" is misleading in both cases).
Will
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerPATCH 6/7 KVM: arm64: Handle stage-2 faults on stage-1 page-table walks earlier
In reply to Will DeaconStage-2 faults on stage-1 page-table walks can occur on both the I-side and the D-side. It is IMPLEMENTATATION DEFINED whether they are reported as reads or writes and, in the case that they are generated by an AT instruction, they are reported with the CM bit set.
All of this deeply confuses the logic in kvm_handle_guest_abort(); userspace may or may not see the fault, depending on whether it occurs on the data or the instruction side, and an AT instruction may be skipped if the translation tables are held in a read-only memslot.
Move the handling of stage-2 faults on stage-1 page-table walks earlier so that they consistently result in either a data or an instruction abort being re-injected back to the guest.
--- arch/arm64/kvm/mmu.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/arm64/kvm/mmu.c +5 -5
@@ -2124,6 +2124,11 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) goto out; } +if (kvm_vcpu_dabt_iss1tw (vcpu)) { + ret = -ENXIO ; + goto out; + } +/* * Check for a cache maintenance operation. Since we * ended-up here, we know it is outside of any memory
@@ -2157,11 +2162,6 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) goto out_unlock; } -if (kvm_vcpu_dabt_iss1tw (vcpu)) { - ret = -ENXIO ; - goto out; - } - ret =io_mem_abort (vcpu, run, fault_ipa); goto out_unlock; }
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH 6/7] KVM: arm64: Handle stage-2 faults on stage-1 page-table walks earlier
Quoted 10 lines · Will Deacon
Stage-2 faults on stage-1 page-table walks can occur on both the I-side and the D-side. It is IMPLEMENTATATION DEFINED whether they are reported as reads or writes and, in the case that they are generated by an AT instruction, they are reported with the CM bit set.
All of this deeply confuses the logic in
kvm_handle_guest_abort(); userspace may or may not see the fault, depending on whether it occurs on the data or the instruction side, and an AT instruction may be skipped if the translation tables are held in a read-only memslot.
Yuk, that's indeed ugly. Well spotted. I guess the saving grace is that a S2 trap caused by an ATS1 instruction will be reported as S1PTW+CM, while the fault caused by a CMO is reported as *either* S1PTW *or* CM, but never both.
Quoted 4 lines
Move the handling of stage-2 faults on stage-1 page-table walks earlier so that they consistently result in either a data or an instruction abort being re-injected back to the guest.
The instruction abort seems to be happening as the side effect of executing outside of a memslot, not really because of a S1PTW. I wonder whether these S1PTW faults should be classified as external aborts instead (because putting your page tables outside of a memslot seems a bit bonkers).
Quoted 40 lines
Cc Marc Zyngier <[email protected]>Cc Quentin Perret <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/kvm/mmu.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)arch/arm64/kvm/mmu.c +5
-5@@ -2124,6 +2124,11 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) goto out; } +if (kvm_vcpu_dabt_iss1tw (vcpu)) { + ret = -ENXIO ; + goto out; + } +/* * Check for a cache maintenance operation. Since we * ended-up here, we know it is outside of any memory @@ -2157,11 +2162,6 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) goto out_unlock; } -if (kvm_vcpu_dabt_iss1tw (vcpu)) { - ret = -ENXIO ; - goto out; - } - ret =io_mem_abort (vcpu, run, fault_ipa); goto out_unlock; }Signature · 1 line
2.28.0.rc0.142.g3c755180ce-goog
Thanks,
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerRe: [PATCH 6/7] KVM: arm64: Handle stage-2 faults on stage-1 page-table walks earlier
Quoted 13 lines · Marc Zyngier
Quoted 10 lines · Will Deacon
Stage-2 faults on stage-1 page-table walks can occur on both the I-side and the D-side. It is IMPLEMENTATATION DEFINED whether they are reported as reads or writes and, in the case that they are generated by an AT instruction, they are reported with the CM bit set.
All of this deeply confuses the logic in
kvm_handle_guest_abort(); userspace may or may not see the fault, depending on whether it occurs on the data or the instruction side, and an AT instruction may be skipped if the translation tables are held in a read-only memslot.
Quoted 4 lines
Yuk, that's indeed ugly. Well spotted. I guess the saving grace is that a S2 trap caused by an ATS1 instruction will be reported as S1PTW+CM, while the fault caused by a CMO is reported as *either* S1PTW *or* CM, but never both.
Hmm, is that right? If the translation faults at S2 for a CM instruction, wouldn't it have both bits set?
Quoted 6 lines
Quoted 3 lines
Move the handling of stage-2 faults on stage-1 page-table walks earlier so that they consistently result in either a data or an instruction abort being re-injected back to the guest.
The instruction abort seems to be happening as the side effect of executing outside of a memslot, not really because of a S1PTW.
Not sure about that. If the instruction fetch generates an S2 abort during translation, then we could be executing from within a memslot; it's the location of the page-tables that matters.
However, I think that means things still aren't quite right with my patches because we can end up calling io_mem_abort() from an instruction abort, which won't have enough syndrome information to do anything useful. Hmm.
Stepping back, here's what I _think_ we want, although see the '(?)' bits where I'm particularly unsure:
S2 instruction abort:
* Not in memslot: inject external iabt to guest
* In R/O memslot:
- S2 fault on S1 walk: either EXIT_NISV or inject external iabt
to guest (?)
S2 data abort:
* Not in memslot:
- S2 fault on S1 walk: inject external dabt to guest
- Cache maintenance: skip instr
- Syndrome valid EXIT_MMIO
- Syndrome invalid EXIT_NISV
* In R/O memslot:
- S2 fault on S1 walk: either EXIT_NISV or inject external dabt
to guest (?)
- Access is write (including cache maintenance (?)):
- Syndrome valid EXIT_MMIO
- Syndrome invalid EXIT_NISV
Everything else gets handled by handle_access_fault()/user_mem_abort().
What do you think?
Quoted 3 lines
I wonder whether these S1PTW faults should be classified as external aborts instead (because putting your page tables outside of a memslot seems a bit bonkers).
I think that's what this patch does, since we end up in kvm_inject_dabt().
Will
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Will Deacon
maintainerPATCH 7/7 KVM: arm64: Separate write faults on read-only memslots from MMIO
In reply to Will DeaconAlthough taking a write-fault on a read-only memslot triggers an MMIO exit back to userspace, lumping the handling together in kvm_handle_guest_abort() causes some of the early triage to have weird effects on userspace. For example, if a guest generates a stage-2 fault on a stage-1 translation fault when trying to fetch an instruction from a read-only memslot, it will be mistakenly treated as an attempt to execute from MMIO and a prefetch abort will be re-injected into the guest.
Separate the MMIO handling from the read-only memslot handling, so that the latter is left entirely up to userspace. Note that this _will_ result in more exits than before for read-only memslots, since userspace will now see some cache maintenance and instruction-side aborts.
--- arch/arm64/kvm/mmu.c | 45 ++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
arch/arm64/kvm/mmu.c +25 -20
@@ -2117,9 +2117,30 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) memslot =gfn_to_memslot (vcpu->kvm, gfn); hva =gfn_to_hva_memslot_prot (memslot, gfn, &writable); write_fault =kvm_is_write_fault (vcpu); -if (kvm_is_error_hva (hva) || (write_fault && !writable)) { + +/* +* The IPA is reported as [MAX:12], so we need to complement it with +* the bottom 12 bits from the faulting VA. This is always 12 bits, +* irrespective of the page size. +*/ + fault_ipa |=kvm_vcpu_get_hfar (vcpu) & ((1 <<12 ) -1 ); + +/* +* We can perform some early fault triage based purely on the memslot +* information: +* +* Faults on IPAs falling outside of any memslot are re-injected +* into the guest as external aborts if they were either signalled as +* instruction aborts or as a stage-2 fault on a translation table walk. +* If the instruction was a cache maintenance instruction then it is +* quietly skipped, otherwise we exit to userspace for MMIO emulation. +* +* Write faults on IPAs falling within a read-only memslot are reported +* to userspace as MMIO exits. This includes cache maintenance and +* stage-2 faults on translation table walks, +*/ +if (kvm_is_error_hva (hva)) {if (kvm_vcpu_trap_is_iabt (vcpu)) { -/* Prefetch Abort on I/O address */ ret = -ENOEXEC ; goto out; }
@@ -2129,30 +2150,12 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) goto out; } -/* -* Check for a cache maintenance operation. Since we -* ended-up here, we know it is outside of any memory -* slot. But we can't find out if that is for a device, -* or if the guest is just being stupid. The only thing -* we know for sure is that this range cannot be cached. -* -* So let's assume that the guest is just being -* cautious, and skip the instruction. -*/ if (kvm_vcpu_dabt_is_cm (vcpu)) {kvm_skip_instr (vcpu,kvm_vcpu_trap_il_is32bit (vcpu)); ret =1 ; goto out_unlock; } -/* -* The IPA is reported as [MAX:12], so we need to -* complement it with the bottom 12 bits from the -* faulting VA. This is always 12 bits, irrespective -* of the page size. -*/ - fault_ipa |=kvm_vcpu_get_hfar (vcpu) & ((1 <<12 ) -1 ); -/* * No valid syndrome? Ask userspace for help if it has * volunteered to do so, and bail out otherwise.
@@ -2161,7 +2164,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run) ret =handle_error_invalid_dabt (vcpu, run, fault_ipa); goto out_unlock; } + } +if (kvm_is_error_hva (hva) || (write_fault && !writable)) { ret =io_mem_abort (vcpu, run, fault_ipa); goto out_unlock; }
Signature · 6 lines
2.28.0.rc0.142.g3c755180ce-goog _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm