KVM: arm64: SCTLR2, DoubleFault2, and NV external abort fixes
v2 cover KVM: arm64: SCTLR2, DoubleFault2, and NV external abort fixes
Well... this series grew a bit. Thanks to some leading review feedback in v1, I went about implementing FEAT_RAS, FEAT_SCLTR2, and FEAT_DoubleFault2 for NV (and non-NV, where applicable).
The most annoying addition of the whole bunch is certainly TMEA, which has the effect of enabling vSErrors and potentially making 'physical' SErrors deliverable to the same context.
v1: https://lore.kernel.org/kvmarm/[email protected]/
v1 -> v2:
- Only use kvm_inject_nested_*() for literal nested exceptions (i.e. EL1&0 => EL2&0 regime) (Marc)
- Take Marc's is_nested_ctxt() helper (thanks!)
- Delegate vSError context to guest hypervisor when appropriate, in turn implementing FEAT_RAS
- Handle state for FEAT_SCTLR2
- Route "masked" aborts to EL2 when HCR_EL2.TMEA is set
- Always perform emulated SError injection when SErrors are unmasked
- Tests
Marc Zyngier (1):
KVM: arm64: Add helper to identify a nested context
Oliver Upton (26):
arm64: Detect FEAT_SCTLR2
arm64: Detect FEAT_DoubleFault2
KVM: arm64: Treat vCPU with pending SError as runnable
KVM: arm64: nv: Respect exception routing rules for SEAs
KVM: arm64: nv: Honor SError exception routing / masking
KVM: arm64: nv: Add FEAT_RAS vSError sys regs to table
KVM: arm64: nv: Use guest hypervisor's vSError state
KVM: arm64: nv: Advertise support for FEAT_RAS
KVM: arm64: nv: Describe trap behavior of SCTLR2_EL1
KVM: arm64: Wire up SCTLR2_ELx sysreg descriptors
KVM: arm64: Context switch SCTLR2_ELx when advertised to the guest
KVM: arm64: Enable SCTLR2 when advertised to the guest
KVM: arm64: Describe SCTLR2_ELx RESx masks
KVM: arm64: Factor out helper for selecting exception target EL
KVM: arm64: nv: Ensure Address size faults affect correct ESR
KVM: arm64: Route SEAs to the SError vector when EASE is set
KVM: arm64: nv: Handle effects of HCRX_EL2.TMEA on SError injection
KVM: arm64: Take "masked" SEAs to EL2 when TMEA is set
KVM: arm64: nv: Enable vSErrors when HCRX_EL2.TMEA is set
KVM: arm64: Advertise support for FEAT_SCTLR2
KVM: arm64: Advertise support for FEAT_DoubleFault2
KVM: arm64: Don't retire MMIO instruction w/ pending (emulated) SError
KVM: arm64: selftests: Add basic SError injection test
KVM: arm64: selftests: Test SEAs are taken to SError vector when
EASE=1
KVM: arm64: selftests: Add SCTLR2_EL1 to get-reg-list
KVM: arm64: selftests: Catch up set_id_regs with the kernel
arch/arm64/include/asm/kvm_emulate.h | 51 +++-
arch/arm64/include/asm/kvm_host.h | 31 ++-
arch/arm64/include/asm/kvm_nested.h | 2 +
arch/arm64/include/asm/vncr_mapping.h | 2 +
arch/arm64/kernel/cpufeature.c | 9 +
arch/arm64/kvm/arch_timer.c | 2 +-
arch/arm64/kvm/arm.c | 9 +-
arch/arm64/kvm/config.c | 28 +++
arch/arm64/kvm/emulate-nested.c | 41 +++-
arch/arm64/kvm/guest.c | 36 +--
arch/arm64/kvm/handle_exit.c | 24 +-
arch/arm64/kvm/hyp/exception.c | 10 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 46 +++-
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 31 ++-
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 6 +
arch/arm64/kvm/inject_fault.c | 222 ++++++++++++------
arch/arm64/kvm/mmio.c | 12 +-
arch/arm64/kvm/mmu.c | 17 +-
arch/arm64/kvm/nested.c | 45 +++-
arch/arm64/kvm/sys_regs.c | 31 ++-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 2 +-
arch/arm64/tools/cpucaps | 1 +
tools/testing/selftests/kvm/Makefile.kvm | 2 +-
.../arm64/{mmio_abort.c => external_aborts.c} | 159 ++++++++++++-
.../selftests/kvm/arm64/get-reg-list.c | 7 +
.../testing/selftests/kvm/arm64/set_id_regs.c | 14 +-
.../selftests/kvm/include/arm64/processor.h | 10 +
28 files changed, 683 insertions(+), 169 deletions(-)
rename tools/testing/selftests/kvm/arm64/{mmio_abort.c => external_aborts.c} (50%)base-commit: e04c78d86a9699d136910cfc0bdcf01087e3267e
Signature · 1 line
2.39.5
PATCH v2 1/27 arm64: Detect FEAT_SCTLR2
KVM is about to pick up support for SCTLR2. Add cpucap for later use in the guest/host context switch hot path.
--- arch/arm64/kernel/cpufeature.c | 8 ++++++++ arch/arm64/tools/cpucaps | 1 + 2 files changed, 9 insertions(+)
arch/arm64/kernel/cpufeature.c +8 -0
@@ -500,6 +500,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr3[] = {ARM64_FTR_BITS (FTR_VISIBLE_IF_IS_ENABLED (CONFIG_ARM64_POE ),FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_S1POE_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_S1PIE_SHIFT ,4 ,0 ), +ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_SCTLRX_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_TCRX_SHIFT ,4 ,0 ), ARM64_FTR_END, };
@@ -3061,6 +3062,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_pmuv3, },#endif + { + .desc = "SCTLR2", + .capability = ARM64_HAS_SCTLR2, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR3_EL1 ,SCTLRX ,IMP ) + }, {}, };
arch/arm64/tools/cpucaps +1 -0
@@ -49,6 +49,7 @@ HAS_PAN HAS_PMUV3 HAS_S1PIE HAS_S1POE +HAS_SCTLR2 HAS_RAS_EXTN HAS_RNG HAS_SB
Signature · 1 line
2.39.5
PATCH v2 2/27 arm64: Detect FEAT_DoubleFault2
In reply to Oliver UptonKVM will soon support FEAT_DoubleFault2. Add a descriptor for the corresponding ID register field.
--- arch/arm64/kernel/cpufeature.c | 1 + 1 file changed, 1 insertion(+)
arch/arm64/kernel/cpufeature.c +1 -0
@@ -303,6 +303,7 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { };static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = { +ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64PFR1_EL1_DF2_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_VISIBLE_IF_IS_ENABLED (CONFIG_ARM64_GCS ),FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64PFR1_EL1_GCS_SHIFT ,4 ,0 ),S_ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE , ID_AA64PFR1_EL1_MTE_frac_SHIFT,4 ,0 ),
Signature · 1 line
2.39.5
PATCH v2 3/27 KVM: arm64: Add helper to identify a nested context
In reply to Oliver UptonFrom: Marc Zyngier <[email protected]>
A common idiom in the KVM code is to check if we are currently dealing with a "nested" context, defined as having NV enabled, but being in the EL1&0 translation regime.
This is usually expressed as:
if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu) ... )
which is a mouthful and a bit hard to read, specially when followed by additional conditions.
Introduce a new helper that encapsulate these two terms, allowing the above to be written as
if (is_nested_context(vcpu) ... )
which is both shorter and easier to read, and makes more obvious the potential for simplification on some code paths.
--- arch/arm64/include/asm/kvm_emulate.h | 5 +++++ arch/arm64/kvm/arch_timer.c | 2 +- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/emulate-nested.c | 9 ++------- arch/arm64/kvm/handle_exit.c | 20 +++++++------------- arch/arm64/kvm/hyp/include/hyp/switch.h | 8 ++++---- arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +- arch/arm64/kvm/vgic/vgic-v3-nested.c | 2 +- 8 files changed, 22 insertions(+), 28 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +5 -0
@@ -224,6 +224,11 @@ static inline bool vcpu_is_host_el0(const struct kvm_vcpu *vcpu)return is_hyp_ctxt (vcpu) && !vcpu_is_el2 (vcpu); } +static inline bool is_nested_ctxt (struct kvm_vcpu *vcpu) +{ +return vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu); +} +/* * The layout of SPSR for an AArch32 state is different when observed from an * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
arch/arm64/kvm/arch_timer.c +1 -1
@@ -830,7 +830,7 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)* by the guest (either FEAT_VHE or FEAT_E2H0 is implemented, but * not both). This simplifies the handling of the EL1NV* bits. */ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) {u64 val =__vcpu_sys_reg (vcpu,CNTHCTL_EL2 );/* Use the VHE format for mental sanity */
arch/arm64/kvm/arm.c +1 -1
@@ -521,7 +521,7 @@ static void vcpu_set_pauth_traps(struct kvm_vcpu *vcpu)* Either we're running an L2 guest, and the API/APK bits come * from L1's HCR_EL2, or API/APK are both set. */ -if (unlikely (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu))) { +if (unlikely (is_nested_ctxt (vcpu))) {u64 val; val =__vcpu_sys_reg (vcpu,HCR_EL2 );
arch/arm64/kvm/emulate-nested.c +2 -7
@@ -2592,13 +2592,8 @@ bool triage_sysreg_trap(struct kvm_vcpu *vcpu, int *sr_index)static bool __forward_traps (struct kvm_vcpu *vcpu,unsigned int reg,u64 control_bit) { -bool control_bit_set; - -if (!vcpu_has_nv (vcpu)) -return false; - - control_bit_set =__vcpu_sys_reg (vcpu, reg) & control_bit; -if (!is_hyp_ctxt (vcpu) && control_bit_set) { +if (is_nested_ctxt (vcpu) && + (__vcpu_sys_reg (vcpu, reg) & control_bit)) {kvm_inject_nested_sync (vcpu,kvm_vcpu_get_esr (vcpu));return true; }
arch/arm64/kvm/handle_exit.c +7 -13
@@ -252,7 +252,7 @@ static int kvm_handle_ptrauth(struct kvm_vcpu *vcpu)return 1 ; } -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) {kvm_inject_nested_sync (vcpu,kvm_vcpu_get_esr (vcpu));return 1 ; }
@@ -311,12 +311,11 @@ static int kvm_handle_gcs(struct kvm_vcpu *vcpu)static int handle_other (struct kvm_vcpu *vcpu) { -bool is_l2 =vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu); +bool allowed, fwd =is_nested_ctxt (vcpu);u64 hcrx =__vcpu_sys_reg (vcpu,HCRX_EL2 );u64 esr =kvm_vcpu_get_esr (vcpu);u64 iss =ESR_ELx_ISS (esr);struct kvm *kvm = vcpu->kvm; -bool allowed, fwd = false;/* * We only trap for two reasons:
@@ -335,28 +334,23 @@ static int handle_other(struct kvm_vcpu *vcpu)switch (iss) {case ESR_ELx_ISS_OTHER_ST64BV: allowed =kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_V ); -if (is_l2) - fwd = !(hcrx & HCRX_EL2_EnASR); + fwd &= !(hcrx & HCRX_EL2_EnASR);break ;case ESR_ELx_ISS_OTHER_ST64BV0: allowed =kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_ACCDATA ); -if (is_l2) - fwd = !(hcrx & HCRX_EL2_EnAS0); + fwd &= !(hcrx & HCRX_EL2_EnAS0);break ;case ESR_ELx_ISS_OTHER_LDST64B: allowed =kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64 ); -if (is_l2) - fwd = !(hcrx & HCRX_EL2_EnALS); + fwd &= !(hcrx & HCRX_EL2_EnALS);break ;case ESR_ELx_ISS_OTHER_TSBCSYNC: allowed =kvm_has_feat (kvm,ID_AA64DFR0_EL1 , TraceBuffer,TRBE_V1P1 ); -if (is_l2) - fwd = (__vcpu_sys_reg (vcpu,HFGITR2_EL2 ) &HFGITR2_EL2_TSBCSYNC ); + fwd &= (__vcpu_sys_reg (vcpu,HFGITR2_EL2 ) &HFGITR2_EL2_TSBCSYNC );break ;case ESR_ELx_ISS_OTHER_PSBCSYNC: allowed =kvm_has_feat (kvm,ID_AA64DFR0_EL1 , PMSVer,V1P5 ); -if (is_l2) - fwd = (__vcpu_sys_reg (vcpu,HFGITR_EL2 ) &HFGITR_EL2_PSBCSYNC ); + fwd &= (__vcpu_sys_reg (vcpu,HFGITR_EL2 ) &HFGITR_EL2_PSBCSYNC );break ;default :/* Clearly, we're missing something. */
arch/arm64/kvm/hyp/include/hyp/switch.h +4 -4
@@ -168,7 +168,7 @@ static inline void __activate_traps_fpsimd32(struct kvm_vcpu *vcpu)u64 val; \ \ ctxt_sys_reg(hctxt, reg) = read_sysreg_s(SYS_ ## reg); \ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) \ +if (is_nested_ctxt (vcpu)) \compute_clr_set (vcpu, reg, c, s); \ \compute_undef_clr_set (vcpu, kvm, reg, c, s); \
@@ -306,7 +306,7 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)if (cpus_have_final_cap (ARM64_HAS_HCX )) {u64 hcrx = vcpu->arch.hcrx_el2; -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) {u64 val =__vcpu_sys_reg (vcpu,HCRX_EL2 ); hcrx |= val & __HCRX_EL2_MASK; hcrx &= ~(~val & __HCRX_EL2_nMASK);
@@ -401,7 +401,7 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)* nested guest, as the guest hypervisor could select a smaller VL. Slap * that into hardware before wrapping up. */ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) +if (is_nested_ctxt (vcpu))sve_cond_update_zcr_vq (__vcpu_sys_reg (vcpu,ZCR_EL2 ),SYS_ZCR_EL2 );write_sysreg_el1 (__vcpu_sys_reg (vcpu,vcpu_sve_zcr_elx (vcpu)),SYS_ZCR );
@@ -427,7 +427,7 @@ static inline void fpsimd_lazy_switch_to_guest(struct kvm_vcpu *vcpu)if (vcpu_has_sve (vcpu)) {/* A guest hypervisor may restrict the effective max VL. */ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) +if (is_nested_ctxt (vcpu)) zcr_el2 =__vcpu_sys_reg (vcpu,ZCR_EL2 );else zcr_el2 =vcpu_sve_max_vq (vcpu) -1 ;
arch/arm64/kvm/hyp/vgic-v3-sr.c +1 -1
@@ -1050,7 +1050,7 @@ static bool __vgic_v3_check_trap_forwarding(struct kvm_vcpu *vcpu, {u64 ich_hcr; -if (!vcpu_has_nv (vcpu) ||is_hyp_ctxt (vcpu)) +if (!is_nested_ctxt (vcpu))return false; ich_hcr =__vcpu_sys_reg (vcpu,ICH_HCR_EL2 );
arch/arm64/kvm/vgic/vgic-v3-nested.c +1 -1
@@ -111,7 +111,7 @@ bool vgic_state_is_nested(struct kvm_vcpu *vcpu) {u64 xmo; -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) { xmo =__vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_IMO |HCR_FMO );WARN_ONCE (xmo && xmo != (HCR_IMO |HCR_FMO ),"Separate virtual IRQ/FIQ settings not supported\n" );
Signature · 1 line
2.39.5
PATCH v2 4/27 KVM: arm64: Treat vCPU with pending SError as runnable
In reply to Oliver UptonPer R_VRLPB, a pending SError is a WFI wakeup event regardless of PSTATE.A, meaning that the vCPU is runnable. Sample VSE in addition to the other IRQ lines.
--- arch/arm64/kvm/arm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/arm64/kvm/arm.c +2 -1
@@ -740,7 +740,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,*/ int kvm_arch_vcpu_runnable (struct kvm_vcpu *v) { -bool irq_lines = *vcpu_hcr (v) & (HCR_VI |HCR_VF ); +bool irq_lines = *vcpu_hcr (v) & (HCR_VI |HCR_VF |HCR_VSE ); +return ((irq_lines ||kvm_vgic_vcpu_pending_irq (v)) && !kvm_arm_vcpu_stopped (v) && !v->arch.pause); }
Signature · 1 line
2.39.5
PATCH v2 5/27 KVM: arm64: nv: Respect exception routing rules for SEAs
In reply to Oliver UptonSynchronous external aborts are taken to EL2 if ELIsInHost() or HCR_EL2.TEA=1. Rework the SEA injection plumbing to respect the imposed routing of the guest hypervisor and opportunistically rephrase things to make their function a bit more obvious.
--- arch/arm64/include/asm/kvm_emulate.h | 14 +++++++-- arch/arm64/kvm/emulate-nested.c | 9 ++++++ arch/arm64/kvm/guest.c | 8 +++-- arch/arm64/kvm/inject_fault.c | 45 +++++++++++----------------- arch/arm64/kvm/mmio.c | 6 ++-- arch/arm64/kvm/mmu.c | 15 +++------- 6 files changed, 50 insertions(+), 47 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +12 -2
@@ -46,15 +46,25 @@ void kvm_skip_instr32(struct kvm_vcpu *vcpu);void kvm_inject_undefined (struct kvm_vcpu *vcpu);void kvm_inject_vabt (struct kvm_vcpu *vcpu); -void kvm_inject_dabt (struct kvm_vcpu *vcpu,unsigned long addr); -void kvm_inject_pabt (struct kvm_vcpu *vcpu,unsigned long addr); +int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);void kvm_inject_size_fault (struct kvm_vcpu *vcpu); +static inline int kvm_inject_sea_dabt (struct kvm_vcpu *vcpu,u64 addr) +{ +return kvm_inject_sea (vcpu, false, addr); +} + +static inline int kvm_inject_sea_iabt (struct kvm_vcpu *vcpu,u64 addr) +{ +return kvm_inject_sea (vcpu, true, addr); +} +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);int kvm_inject_nested_sync (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_inject_nested_irq (struct kvm_vcpu *vcpu); +int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);static inline void kvm_inject_nested_sve_trap (struct kvm_vcpu *vcpu) {
arch/arm64/kvm/emulate-nested.c +9 -0
@@ -2811,3 +2811,12 @@ int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)/* esr_el2 value doesn't matter for exits due to irqs. */ return kvm_inject_nested (vcpu,0 , except_type_irq); } + +int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr) +{ +u64 esr =FIELD_PREP (ESR_ELx_EC_MASK, + iabt ? ESR_ELx_EC_IABT_LOW : ESR_ELx_EC_DABT_LOW); + esr |= ESR_ELx_FSC_EXTABT | ESR_ELx_IL; + +return kvm_inject_s2_fault (vcpu, esr); +}
arch/arm64/kvm/guest.c +6 -2
@@ -839,6 +839,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,bool serror_pending = events->exception.serror_pending;bool has_esr = events->exception.serror_has_esr;bool ext_dabt_pending = events->exception.ext_dabt_pending; +int ret;if (serror_pending && has_esr) {if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN ))
@@ -852,8 +853,11 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,kvm_inject_vabt (vcpu); } -if (ext_dabt_pending) -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); +if (ext_dabt_pending) { + ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); +if (ret <0 ) +return ret; + }return 0 ; }
arch/arm64/kvm/inject_fault.c +17 -28
63 lines not on this page · raw message
arch/arm64/kvm/mmio.c +2 -4
@@ -169,10 +169,8 @@ int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)trace_kvm_mmio_nisv (*vcpu_pc (vcpu),kvm_vcpu_get_esr (vcpu),kvm_vcpu_get_hfar (vcpu), fault_ipa); -if (vcpu_is_protected (vcpu)) { -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); -return 1 ; - } +if (vcpu_is_protected (vcpu)) +return kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu));if (test_bit (KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER , &vcpu->kvm->arch.flags)) {
arch/arm64/kvm/mmu.c +4 -11
@@ -1836,11 +1836,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)if (fault_ipa >=BIT_ULL (VTCR_EL2_IPA (vcpu->arch.hw_mmu->vtcr))) { fault_ipa |=kvm_vcpu_get_hfar (vcpu) &GENMASK (11 ,0 ); -if (is_iabt) -kvm_inject_pabt (vcpu, fault_ipa); -else -kvm_inject_dabt (vcpu, fault_ipa); -return 1 ; +return kvm_inject_sea (vcpu, is_iabt, fault_ipa); } }
@@ -1912,8 +1908,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) }if (kvm_vcpu_abt_iss1tw (vcpu)) { -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); - ret =1 ; + ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); goto out_unlock; }
@@ -1958,10 +1953,8 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)if (ret ==0 ) ret =1 ; out: -if (ret == -ENOEXEC ) { -kvm_inject_pabt (vcpu,kvm_vcpu_get_hfar (vcpu)); - ret =1 ; - } +if (ret == -ENOEXEC ) + ret =kvm_inject_sea_iabt (vcpu,kvm_vcpu_get_hfar (vcpu)); out_unlock:srcu_read_unlock (&vcpu->kvm->srcu, idx);return ret;
Signature · 1 line
2.39.5
Marc Zyngier
maintainerRe: [PATCH v2 05/27] KVM: arm64: nv: Respect exception routing rules for SEAs
Quoted 54 lines · Oliver Upton
Synchronous external aborts are taken to EL2 if
ELIsInHost()or HCR_EL2.TEA=1. Rework the SEA injection plumbing to respect the imposed routing of the guest hypervisor and opportunistically rephrase things to make their function a bit more obvious.Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 14 +++++++-- arch/arm64/kvm/emulate-nested.c | 9 ++++++ arch/arm64/kvm/guest.c | 8 +++-- arch/arm64/kvm/inject_fault.c | 45 +++++++++++----------------- arch/arm64/kvm/mmio.c | 6 ++-- arch/arm64/kvm/mmu.c | 15 +++------- 6 files changed, 50 insertions(+), 47 deletions(-)arch/arm64/include/asm/kvm_emulate.h +12
-2@@ -46,15 +46,25 @@ void kvm_skip_instr32(struct kvm_vcpu *vcpu);void kvm_inject_undefined (struct kvm_vcpu *vcpu);void kvm_inject_vabt (struct kvm_vcpu *vcpu); -void kvm_inject_dabt (struct kvm_vcpu *vcpu,unsigned long addr); -void kvm_inject_pabt (struct kvm_vcpu *vcpu,unsigned long addr); +int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);void kvm_inject_size_fault (struct kvm_vcpu *vcpu); +static inline int kvm_inject_sea_dabt (struct kvm_vcpu *vcpu,u64 addr) +{ +return kvm_inject_sea (vcpu, false, addr); +} + +static inline int kvm_inject_sea_iabt (struct kvm_vcpu *vcpu,u64 addr) +{ +return kvm_inject_sea (vcpu, true, addr); +} +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);int kvm_inject_nested_sync (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_inject_nested_irq (struct kvm_vcpu *vcpu); +int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);static inline void kvm_inject_nested_sve_trap (struct kvm_vcpu *vcpu) {arch/arm64/kvm/emulate-nested.c +0
-0@@ -2811,3 +2811,12 @@ int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)/* esr_el2 value doesn't matter for exits due to irqs. */
arch/arm64/kvm/emulate-nested.c int kvm_inject_nested_irq(struct kvm_vcpu *vcpu) · line 2821
Quoted 10 lines
arch/arm64/kvm/emulate-nested.c +8
-0return kvm_inject_nested (vcpu,0 , except_type_irq); } + +int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr) +{ +u64 esr =FIELD_PREP (ESR_ELx_EC_MASK, + iabt ? ESR_ELx_EC_IABT_LOW : ESR_ELx_EC_DABT_LOW); + esr |= ESR_ELx_FSC_EXTABT | ESR_ELx_IL; + +return kvm_inject_s2_fault (vcpu, esr);
I think this may be slightly abusive. R_FKLWR gives a list of all faults that populate HPFAR_EL2, and SEA isn't one of them. I think only populating ESR/FAR should be enough, and avoid leaking stale fault addresses from arch.fault.hpfar_el2.
arch/arm64/kvm/guest.c int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, · line 842
Quoted 10 lines
arch/arm64/kvm/emulate-nested.c +1
-0+}arch/arm64/kvm/guest.c +1
-0@@ -839,6 +839,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,bool serror_pending = events->exception.serror_pending;bool has_esr = events->exception.serror_has_esr;bool ext_dabt_pending = events->exception.ext_dabt_pending; +int ret;
Initialise ret to 0...
arch/arm64/kvm/guest.c int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, · line 859
Quoted 13 lines
arch/arm64/kvm/guest.c +4
-2if (serror_pending && has_esr) {if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN ))@@ -852,8 +853,11 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,kvm_inject_vabt (vcpu); } -if (ext_dabt_pending) -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); +if (ext_dabt_pending) { + ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); +if (ret <0 ) +return ret;
... drop this test ...
arch/arm64/kvm/guest.c int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, · line 862
Quoted 3 lines
arch/arm64/kvm/guest.c +1
-0+ }return 0 ;
... and return ret?
Otherwise, LGTM.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH v2 6/27 KVM: arm64: nv: Honor SError exception routing / masking
In reply to Oliver UptonTo date KVM has used HCR_EL2.VSE to track the state of a pending SError for the guest. With this bit set, hardware respects the EL1 exception routing / masking rules and injects the vSError when appropriate.
This isn't correct for NV guests as hardware is oblivious to vEL2's intentions for SErrors. Better yet, with FEAT_NV2 the guest can change the routing behind our back as HCR_EL2 is redirected to memory. Cope with this mess by:
- Using a flag (instead of HCR_EL2.VSE) to track the pending SError state when SErrors are unconditionally masked for the current context
- Resampling the routing / masking of a pending SError on every guest entry/exit
- Emulating exception entry when SError routing implies a translation regime change
--- arch/arm64/include/asm/kvm_emulate.h | 20 +++++++++++++- arch/arm64/include/asm/kvm_host.h | 20 +++++++++++--- arch/arm64/include/asm/kvm_nested.h | 2 ++ arch/arm64/kvm/arm.c | 4 +++ arch/arm64/kvm/emulate-nested.c | 8 ++++++ arch/arm64/kvm/guest.c | 32 +++++++++++++---------- arch/arm64/kvm/handle_exit.c | 4 +-- arch/arm64/kvm/hyp/exception.c | 6 ++++- arch/arm64/kvm/inject_fault.c | 39 ++++++++++++++++------------ arch/arm64/kvm/mmu.c | 2 +- arch/arm64/kvm/nested.c | 36 +++++++++++++++++++++++++ 11 files changed, 134 insertions(+), 39 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +19 -1
@@ -45,7 +45,7 @@ bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);void kvm_skip_instr32 (struct kvm_vcpu *vcpu);void kvm_inject_undefined (struct kvm_vcpu *vcpu); -void kvm_inject_vabt (struct kvm_vcpu *vcpu); +int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr);int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);void kvm_inject_size_fault (struct kvm_vcpu *vcpu);
@@ -59,12 +59,25 @@ static inline int kvm_inject_sea_iabt(struct kvm_vcpu *vcpu, u64 addr)return kvm_inject_sea (vcpu, true, addr); } +static inline int kvm_inject_serror (struct kvm_vcpu *vcpu) +{ +/* +* ESR_ELx.ISV (later renamed to IDS) indicates whether or not +* ESR_ELx.ISS contains IMPLEMENTATION DEFINED syndrome information. +* +* Set the bit when injecting an SError w/o an ESR to indicate ISS +* does not follow the architected format. +*/ +return kvm_inject_serror_esr (vcpu, ESR_ELx_ISV); +} +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);int kvm_inject_nested_sync (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_inject_nested_irq (struct kvm_vcpu *vcpu);int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr); +int kvm_inject_nested_serror (struct kvm_vcpu *vcpu,u64 esr);static inline void kvm_inject_nested_sve_trap (struct kvm_vcpu *vcpu) {
@@ -205,6 +218,11 @@ static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu)return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_TGE ; } +static inline bool vcpu_el2_amo_is_set (const struct kvm_vcpu *vcpu) +{ +return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_AMO ; +} +static inline bool is_hyp_ctxt (const struct kvm_vcpu *vcpu) {bool e2h, tge;
arch/arm64/include/asm/kvm_host.h +17 -3
@@ -817,7 +817,7 @@ struct kvm_vcpu_arch {u8 iflags;/* State flags for kernel bookkeeping, unused by the hypervisor code */ -u8 sflags; +u16 sflags;/* * Don't run the guest (internal implementation need).
@@ -953,9 +953,23 @@ struct kvm_vcpu_arch {__vcpu_flags_preempt_enable (); \ } while (0) +#define __vcpu_test_and_clear_flag (v, flagset, f, m) \ + ({ \ + typeof(v->arch.flagset) set; \ + \ + __vcpu_flags_preempt_disable(); \ + set = __vcpu_get_flag(v, flagset, f, m); \ + __vcpu_clear_flag(v, flagset, f, m); \ + __vcpu_flags_preempt_enable(); \ + \ + set; \ + }) +#define vcpu_get_flag (v, ...) __vcpu_get_flag((v), __VA_ARGS__)#define vcpu_set_flag (v, ...) __vcpu_set_flag((v), __VA_ARGS__)#define vcpu_clear_flag (v, ...) __vcpu_clear_flag((v), __VA_ARGS__) +#define vcpu_test_and_clear_flag (v, ...) \ + __vcpu_test_and_clear_flag((v), __VA_ARGS__)/* KVM_ARM_VCPU_INIT completed */ #define VCPU_INITIALIZED __vcpu_single_flag(cflags, BIT(0))
@@ -1015,6 +1029,8 @@ struct kvm_vcpu_arch {#define IN_WFI __vcpu_single_flag(sflags, BIT(6))/* KVM is currently emulating a nested ERET */ #define IN_NESTED_ERET __vcpu_single_flag(sflags, BIT(7)) +/* SError pending for nested guest */ +#define NESTED_SERROR_PENDING __vcpu_single_flag(sflags, BIT(8))/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */
@@ -1389,8 +1405,6 @@ static inline bool kvm_arm_is_pvtime_enabled(struct kvm_vcpu_arch *vcpu_arch)return (vcpu_arch->steal.base !=INVALID_GPA ); } -void kvm_set_sei_esr (struct kvm_vcpu *vcpu,u64 syndrome); -struct kvm_vcpu *kvm_mpidr_to_vcpu (struct kvm *kvm,unsigned long mpidr); DECLARE_KVM_HYP_PER_CPU(struct kvm_host_data, kvm_host_data);
arch/arm64/include/asm/kvm_nested.h +2 -0
@@ -80,6 +80,8 @@ extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);extern void kvm_vcpu_put_hw_mmu (struct kvm_vcpu *vcpu);extern void check_nested_vcpu_requests (struct kvm_vcpu *vcpu); +extern void kvm_nested_flush_hwstate (struct kvm_vcpu *vcpu); +extern void kvm_nested_sync_hwstate (struct kvm_vcpu *vcpu);struct kvm_s2_trans {phys_addr_t output;
arch/arm64/kvm/arm.c +4 -0
@@ -1188,6 +1188,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)*/ preempt_disable (); +kvm_nested_flush_hwstate (vcpu); +if (kvm_vcpu_has_pmu (vcpu))kvm_pmu_flush_hwstate (vcpu);
@@ -1287,6 +1289,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)/* Exit types that need handling before we can be preempted */ handle_exit_early (vcpu, ret); +kvm_nested_sync_hwstate (vcpu); +preempt_enable ();/*
arch/arm64/kvm/emulate-nested.c +8 -0
@@ -2714,6 +2714,9 @@ static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,case except_type_irq:kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_IRQ );break ; +case except_type_serror: +kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SERR ); +break ;default :WARN_ONCE (1 ,"Unsupported EL2 exception injection %d\n" , type); }
@@ -2820,3 +2823,8 @@ int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr)return kvm_inject_s2_fault (vcpu, esr); } + +int kvm_inject_nested_serror (struct kvm_vcpu *vcpu,u64 esr) +{ +return kvm_inject_nested (vcpu, esr, except_type_serror); +}
arch/arm64/kvm/guest.c +18 -14
@@ -818,8 +818,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,int __kvm_arm_vcpu_get_events (struct kvm_vcpu *vcpu,struct kvm_vcpu_events *events) { - events->exception.serror_pending = !!(vcpu->arch.hcr_el2 &HCR_VSE ); events->exception.serror_has_esr =cpus_have_final_cap (ARM64_HAS_RAS_EXTN ); + events->exception.serror_pending = (vcpu->arch.hcr_el2 &HCR_VSE ) || +vcpu_get_flag (vcpu,NESTED_SERROR_PENDING );if (events->exception.serror_pending && events->exception.serror_has_esr) events->exception.serror_esr =vcpu_get_vsesr (vcpu);
@@ -839,27 +840,30 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,bool serror_pending = events->exception.serror_pending;bool has_esr = events->exception.serror_has_esr;bool ext_dabt_pending = events->exception.ext_dabt_pending; +u64 esr = events->exception.serror_esr;int ret; -if (serror_pending && has_esr) { -if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) -return -EINVAL ; - -if (!((events->exception.serror_esr) & ~ESR_ELx_ISS_MASK)) -kvm_set_sei_esr (vcpu, events->exception.serror_esr); -else -return -EINVAL ; - }else if (serror_pending) { -kvm_inject_vabt (vcpu); - } -if (ext_dabt_pending) { ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu));if (ret <0 )return ret; } -return 0 ; +if (!serror_pending) +return 0 ; + +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && has_esr) +return -EINVAL ; + +if (has_esr && (esr & ~ESR_ELx_ISS_MASK)) +return -EINVAL ; + +if (has_esr) + ret =kvm_inject_serror_esr (vcpu, esr); +else + ret =kvm_inject_serror (vcpu); + +return (ret <0 ) ? ret :0 ; } u32 __attribute_const__ kvm_target_cpu(void)
arch/arm64/kvm/handle_exit.c +2 -2
@@ -32,7 +32,7 @@ typedef int (*exit_handle_fn)(struct kvm_vcpu *);static void kvm_handle_guest_serror (struct kvm_vcpu *vcpu,u64 esr) {if (!arm64_is_ras_serror (esr) ||arm64_is_fatal_ras_serror (NULL , esr)) -kvm_inject_vabt (vcpu); +kvm_inject_serror (vcpu); }static int handle_hvc (struct kvm_vcpu *vcpu)
@@ -490,7 +490,7 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index)kvm_handle_guest_serror (vcpu,disr_to_esr (disr)); }else { -kvm_inject_vabt (vcpu); +kvm_inject_serror (vcpu); }return ;
arch/arm64/kvm/hyp/exception.c +5 -1
@@ -347,9 +347,13 @@ static void kvm_inject_exception(struct kvm_vcpu *vcpu)enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_irq);break ; +case unpack_vcpu_flag (EXCEPT_AA64_EL2_SERR ): +enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_serror); +break ; +default :/* -* Only EL1_SYNC and EL2_{SYNC,IRQ} makes +* Only EL1_SYNC and EL2_{SYNC,IRQ,SERR} makes * sense so far. Everything else gets silently * ignored. */
arch/arm64/kvm/inject_fault.c +22 -17
@@ -219,25 +219,30 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)inject_undef64 (vcpu); } -void kvm_set_sei_esr (struct kvm_vcpu *vcpu,u64 esr) +static bool kvm_serror_target_is_el2 (struct kvm_vcpu *vcpu) { -vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); - *vcpu_hcr (vcpu) |=HCR_VSE ; +return is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu); } -/** -* kvm_inject_vabt - inject an async abort / SError into the guest -* @vcpu: The VCPU to receive the exception -* -* It is assumed that this code is called from the VCPU thread and that the -* VCPU therefore is not currently executing guest code. -* -* Systems with the RAS Extensions specify an imp-def ESR (ISV/IDS = 1) with -* the remaining ISS all-zeros so that this error is not interpreted as an -* uncategorized RAS error. Without the RAS Extensions we can't specify an ESR -* value, so the CPU generates an imp-def value. -*/ -void kvm_inject_vabt (struct kvm_vcpu *vcpu) +static bool kvm_serror_undeliverable_at_el2 (struct kvm_vcpu *vcpu) { -kvm_set_sei_esr (vcpu, ESR_ELx_ISV); +return !(vcpu_el2_tge_is_set (vcpu) ||vcpu_el2_amo_is_set (vcpu)); +} + +int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr) +{ +lockdep_assert_held (&vcpu->mutex); + +if (is_nested_ctxt (vcpu) &&kvm_serror_target_is_el2 (vcpu)) +return kvm_inject_nested_serror (vcpu, esr); + +if (vcpu_is_el2 (vcpu) &&kvm_serror_undeliverable_at_el2 (vcpu)) { +vcpu_set_vsesr (vcpu, esr); +vcpu_set_flag (vcpu,NESTED_SERROR_PENDING ); +return 1 ; + } + +vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); + *vcpu_hcr (vcpu) |=HCR_VSE ; +return 1 ; }
arch/arm64/kvm/mmu.c +1 -1
@@ -1808,7 +1808,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)* There is no need to pass the error into the guest. */ if (kvm_handle_guest_sea ()) -kvm_inject_vabt (vcpu); +return kvm_inject_serror (vcpu);return 1 ; }
arch/arm64/kvm/nested.c +36 -0
@@ -1782,3 +1782,39 @@ void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_GUEST_HYP_IRQ_PENDING , vcpu))kvm_inject_nested_irq (vcpu); } + +/* +* One of the many architectural bugs in FEAT_NV2 is that the guest hypervisor +* can write to HCR_EL2 behind our back, potentially changing the exception +* routing / masking for even the host context. +* +* What follows is some slop to (1) react to exception routing / masking and (2) +* preserve the pending SError state across translation regimes. +*/ +void kvm_nested_flush_hwstate (struct kvm_vcpu *vcpu) +{ +if (!vcpu_has_nv (vcpu)) +return ; + +if (unlikely (vcpu_test_and_clear_flag (vcpu,NESTED_SERROR_PENDING ))) +kvm_inject_serror_esr (vcpu,vcpu_get_vsesr (vcpu)); +} + +void kvm_nested_sync_hwstate (struct kvm_vcpu *vcpu) +{ +unsigned long *hcr =vcpu_hcr (vcpu); + +if (!vcpu_has_nv (vcpu)) +return ; + +/* +* We previously decided that an SError was deliverable to the guest. +* Reap the pending state from HCR_EL2 and... +*/ +if (unlikely (__test_and_clear_bit (__ffs (HCR_VSE ), hcr))) +vcpu_set_flag (vcpu,NESTED_SERROR_PENDING ); + +/* Re-attempt SError injection in case the deliverability has changed */ +if (unlikely (vcpu_test_and_clear_flag (vcpu,NESTED_SERROR_PENDING ))) +kvm_inject_serror_esr (vcpu,vcpu_get_vsesr (vcpu)); +}
Signature · 1 line
2.39.5
Marc Zyngier
maintainerRe: [PATCH v2 06/27] KVM: arm64: nv: Honor SError exception routing / masking
Quoted 100 lines · Oliver Upton
To date KVM has used HCR_EL2.VSE to track the state of a pending SError for the guest. With this bit set, hardware respects the EL1 exception routing / masking rules and injects the vSError when appropriate.
This isn't correct for NV guests as hardware is oblivious to vEL2's intentions for SErrors. Better yet, with
FEAT_NV2the guest can change the routing behind our back asHCR_EL2is redirected to memory. Cope with this mess by:- Using a flag (instead of HCR_EL2.VSE) to track the pending SError state when SErrors are unconditionally masked for the current context
- Resampling the routing / masking of a pending SError on every guest entry/exit
- Emulating exception entry when SError routing implies a translation regime change
Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 20 +++++++++++++- arch/arm64/include/asm/kvm_host.h | 20 +++++++++++--- arch/arm64/include/asm/kvm_nested.h | 2 ++ arch/arm64/kvm/arm.c | 4 +++ arch/arm64/kvm/emulate-nested.c | 8 ++++++ arch/arm64/kvm/guest.c | 32 +++++++++++++---------- arch/arm64/kvm/handle_exit.c | 4 +-- arch/arm64/kvm/hyp/exception.c | 6 ++++- arch/arm64/kvm/inject_fault.c | 39 ++++++++++++++++------------ arch/arm64/kvm/mmu.c | 2 +- arch/arm64/kvm/nested.c | 36 +++++++++++++++++++++++++ 11 files changed, 134 insertions(+), 39 deletions(-)arch/arm64/include/asm/kvm_emulate.h +19
-1@@ -45,7 +45,7 @@ bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);void kvm_skip_instr32 (struct kvm_vcpu *vcpu);void kvm_inject_undefined (struct kvm_vcpu *vcpu); -void kvm_inject_vabt (struct kvm_vcpu *vcpu); +int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr);int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);void kvm_inject_size_fault (struct kvm_vcpu *vcpu);@@ -59,12 +59,25 @@ static inline int kvm_inject_sea_iabt(struct kvm_vcpu *vcpu, u64 addr)return kvm_inject_sea (vcpu, true, addr); } +static inline int kvm_inject_serror (struct kvm_vcpu *vcpu) +{ +/* +* ESR_ELx.ISV (later renamed to IDS) indicates whether or not +* ESR_ELx.ISS contains IMPLEMENTATION DEFINED syndrome information. +* +* Set the bit when injecting an SError w/o an ESR to indicate ISS +* does not follow the architected format. +*/ +return kvm_inject_serror_esr (vcpu, ESR_ELx_ISV); +} +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);int kvm_inject_nested_sync (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_inject_nested_irq (struct kvm_vcpu *vcpu);int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr); +int kvm_inject_nested_serror (struct kvm_vcpu *vcpu,u64 esr);static inline void kvm_inject_nested_sve_trap (struct kvm_vcpu *vcpu) {@@ -205,6 +218,11 @@ static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu)return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_TGE ; } +static inline bool vcpu_el2_amo_is_set (const struct kvm_vcpu *vcpu) +{ +return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_AMO ; +} +static inline bool is_hyp_ctxt (const struct kvm_vcpu *vcpu) {bool e2h, tge;arch/arm64/include/asm/kvm_host.h +1
-1@@ -817,7 +817,7 @@ struct kvm_vcpu_arch {u8 iflags;/* State flags for kernel bookkeeping, unused by the hypervisor code */ -u8 sflags; +u16 sflags;/* * Don't run the guest (internal implementation need). @@ -953,9 +953,23 @@ struct kvm_vcpu_arch {__vcpu_flags_preempt_enable (); \
arch/arm64/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 963
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +8
-0} while (0) +#define __vcpu_test_and_clear_flag (v, flagset, f, m) \ + ({ \ + typeof(v->arch.flagset) set; \ + \ + __vcpu_flags_preempt_disable(); \ + set = __vcpu_get_flag(v, flagset, f, m); \ + __vcpu_clear_flag(v, flagset, f, m); \ + __vcpu_flags_preempt_enable(); \
I have the feeling that you can drop the preemption manipulation here. as __vcpu_clear_flags() already does it.
Quoted 129 lines
arch/arm64/include/asm/kvm_host.h +8
-2+ \ + set; \ + }) +#define vcpu_get_flag (v, ...) __vcpu_get_flag((v), __VA_ARGS__)#define vcpu_set_flag (v, ...) __vcpu_set_flag((v), __VA_ARGS__)#define vcpu_clear_flag (v, ...) __vcpu_clear_flag((v), __VA_ARGS__) +#define vcpu_test_and_clear_flag (v, ...) \ + __vcpu_test_and_clear_flag((v), __VA_ARGS__)/* KVM_ARM_VCPU_INIT completed */ #define VCPU_INITIALIZED __vcpu_single_flag(cflags, BIT(0))@@ -1015,6 +1029,8 @@ struct kvm_vcpu_arch {#define IN_WFI __vcpu_single_flag(sflags, BIT(6))/* KVM is currently emulating a nested ERET */ #define IN_NESTED_ERET __vcpu_single_flag(sflags, BIT(7)) +/* SError pending for nested guest */ +#define NESTED_SERROR_PENDING __vcpu_single_flag(sflags, BIT(8))/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ @@ -1389,8 +1405,6 @@ static inline bool kvm_arm_is_pvtime_enabled(struct kvm_vcpu_arch *vcpu_arch)return (vcpu_arch->steal.base !=INVALID_GPA ); } -void kvm_set_sei_esr (struct kvm_vcpu *vcpu,u64 syndrome); -struct kvm_vcpu *kvm_mpidr_to_vcpu (struct kvm *kvm,unsigned long mpidr); DECLARE_KVM_HYP_PER_CPU(struct kvm_host_data, kvm_host_data);arch/arm64/include/asm/kvm_nested.h +2
-0@@ -80,6 +80,8 @@ extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);extern void kvm_vcpu_put_hw_mmu (struct kvm_vcpu *vcpu);extern void check_nested_vcpu_requests (struct kvm_vcpu *vcpu); +extern void kvm_nested_flush_hwstate (struct kvm_vcpu *vcpu); +extern void kvm_nested_sync_hwstate (struct kvm_vcpu *vcpu);struct kvm_s2_trans {phys_addr_t output;arch/arm64/kvm/arm.c +4
-0@@ -1188,6 +1188,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)*/ preempt_disable (); +kvm_nested_flush_hwstate (vcpu); +if (kvm_vcpu_has_pmu (vcpu))kvm_pmu_flush_hwstate (vcpu);@@ -1287,6 +1289,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)/* Exit types that need handling before we can be preempted */ handle_exit_early (vcpu, ret); +kvm_nested_sync_hwstate (vcpu); +preempt_enable ();/* arch/arm64/kvm/emulate-nested.c +8
-0@@ -2714,6 +2714,9 @@ static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,case except_type_irq:kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_IRQ );break ; +case except_type_serror: +kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SERR ); +break ;default :WARN_ONCE (1 ,"Unsupported EL2 exception injection %d\n" , type); }@@ -2820,3 +2823,8 @@ int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr)return kvm_inject_s2_fault (vcpu, esr); } + +int kvm_inject_nested_serror (struct kvm_vcpu *vcpu,u64 esr) +{ +return kvm_inject_nested (vcpu, esr, except_type_serror); +}arch/arm64/kvm/guest.c +3
-14@@ -818,8 +818,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,int __kvm_arm_vcpu_get_events (struct kvm_vcpu *vcpu,struct kvm_vcpu_events *events) { - events->exception.serror_pending = !!(vcpu->arch.hcr_el2 &HCR_VSE ); events->exception.serror_has_esr =cpus_have_final_cap (ARM64_HAS_RAS_EXTN ); + events->exception.serror_pending = (vcpu->arch.hcr_el2 &HCR_VSE ) || +vcpu_get_flag (vcpu,NESTED_SERROR_PENDING );if (events->exception.serror_pending && events->exception.serror_has_esr) events->exception.serror_esr =vcpu_get_vsesr (vcpu);@@ -839,27 +840,30 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,bool serror_pending = events->exception.serror_pending;bool has_esr = events->exception.serror_has_esr;bool ext_dabt_pending = events->exception.ext_dabt_pending; +u64 esr = events->exception.serror_esr;int ret; -if (serror_pending && has_esr) { -if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) -return -EINVAL ; - -if (!((events->exception.serror_esr) & ~ESR_ELx_ISS_MASK)) -kvm_set_sei_esr (vcpu, events->exception.serror_esr); -else -return -EINVAL ; - }else if (serror_pending) { -kvm_inject_vabt (vcpu); - } -if (ext_dabt_pending) { ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu));if (ret <0 )return ret; } -return 0 ;
arch/arm64/kvm/guest.c int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, · line 861
Quoted 10 lines
arch/arm64/kvm/guest.c +10
-0+if (!serror_pending) +return 0 ; + +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && has_esr) +return -EINVAL ; + +if (has_esr && (esr & ~ESR_ELx_ISS_MASK)) +return -EINVAL ; + +if (has_esr)
We should probably consider whether the VM itself has RAS before populating an ESR, and return an error to userspace otherwise. Unless that's yet another can of worm that we'd rather stay closed?
I have the ugly feeling that it might be the latter...
Quoted 147 lines
arch/arm64/kvm/guest.c +5
-0+ ret =kvm_inject_serror_esr (vcpu, esr); + else + ret =kvm_inject_serror (vcpu); + +return (ret <0 ) ? ret :0 ; } u32 __attribute_const__ kvm_target_cpu(void)arch/arm64/kvm/handle_exit.c +2
-2@@ -32,7 +32,7 @@ typedef int (*exit_handle_fn)(struct kvm_vcpu *);static void kvm_handle_guest_serror (struct kvm_vcpu *vcpu,u64 esr) {if (!arm64_is_ras_serror (esr) ||arm64_is_fatal_ras_serror (NULL , esr)) -kvm_inject_vabt (vcpu); +kvm_inject_serror (vcpu); }static int handle_hvc (struct kvm_vcpu *vcpu)@@ -490,7 +490,7 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index)kvm_handle_guest_serror (vcpu,disr_to_esr (disr)); }else { -kvm_inject_vabt (vcpu); +kvm_inject_serror (vcpu); }return ;arch/arm64/kvm/hyp/exception.c +5
-1@@ -347,9 +347,13 @@ static void kvm_inject_exception(struct kvm_vcpu *vcpu)enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_irq);break ; +case unpack_vcpu_flag (EXCEPT_AA64_EL2_SERR ): +enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_serror); +break ; +default :/* -* Only EL1_SYNC and EL2_{SYNC,IRQ} makes +* Only EL1_SYNC and EL2_{SYNC,IRQ,SERR} makes * sense so far. Everything else gets silently * ignored. */ arch/arm64/kvm/inject_fault.c +22
-17@@ -219,25 +219,30 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)inject_undef64 (vcpu); } -void kvm_set_sei_esr (struct kvm_vcpu *vcpu,u64 esr) +static bool kvm_serror_target_is_el2 (struct kvm_vcpu *vcpu) { -vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); - *vcpu_hcr (vcpu) |=HCR_VSE ; +return is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu); } -/** -* kvm_inject_vabt - inject an async abort / SError into the guest -* @vcpu: The VCPU to receive the exception -* -* It is assumed that this code is called from the VCPU thread and that the -* VCPU therefore is not currently executing guest code. -* -* Systems with the RAS Extensions specify an imp-def ESR (ISV/IDS = 1) with -* the remaining ISS all-zeros so that this error is not interpreted as an -* uncategorized RAS error. Without the RAS Extensions we can't specify an ESR -* value, so the CPU generates an imp-def value. -*/ -void kvm_inject_vabt (struct kvm_vcpu *vcpu) +static bool kvm_serror_undeliverable_at_el2 (struct kvm_vcpu *vcpu) { -kvm_set_sei_esr (vcpu, ESR_ELx_ISV); +return !(vcpu_el2_tge_is_set (vcpu) ||vcpu_el2_amo_is_set (vcpu)); +} + +int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr) +{ +lockdep_assert_held (&vcpu->mutex); + +if (is_nested_ctxt (vcpu) &&kvm_serror_target_is_el2 (vcpu)) +return kvm_inject_nested_serror (vcpu, esr); + +if (vcpu_is_el2 (vcpu) &&kvm_serror_undeliverable_at_el2 (vcpu)) { +vcpu_set_vsesr (vcpu, esr); +vcpu_set_flag (vcpu,NESTED_SERROR_PENDING ); +return 1 ; + } + +vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); + *vcpu_hcr (vcpu) |=HCR_VSE ; +return 1 ; }arch/arm64/kvm/mmu.c +1
-1@@ -1808,7 +1808,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)* There is no need to pass the error into the guest. */ if (kvm_handle_guest_sea ()) -kvm_inject_vabt (vcpu); +return kvm_inject_serror (vcpu);return 1 ; }arch/arm64/kvm/nested.c +25
-0@@ -1782,3 +1782,39 @@ void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_GUEST_HYP_IRQ_PENDING , vcpu))kvm_inject_nested_irq (vcpu); } + +/* +* One of the many architectural bugs in FEAT_NV2 is that the guest hypervisor +* can write to HCR_EL2 behind our back, potentially changing the exception +* routing / masking for even the host context. +* +* What follows is some slop to (1) react to exception routing / masking and (2) +* preserve the pending SError state across translation regimes. +*/ +void kvm_nested_flush_hwstate (struct kvm_vcpu *vcpu) +{ +if (!vcpu_has_nv (vcpu)) +return ; + +if (unlikely (vcpu_test_and_clear_flag (vcpu,NESTED_SERROR_PENDING ))) +kvm_inject_serror_esr (vcpu,vcpu_get_vsesr (vcpu)); +} + +void kvm_nested_sync_hwstate (struct kvm_vcpu *vcpu) +{ +unsigned long *hcr =vcpu_hcr (vcpu); + +if (!vcpu_has_nv (vcpu)) +return ; +
arch/arm64/kvm/nested.c void check_nested_vcpu_requests(struct kvm_vcpu *vcpu) · line 1819
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+/* +* We previously decided that an SError was deliverable to the guest. +* Reap the pending state from HCR_EL2 and... +*/ +if (unlikely (__test_and_clear_bit (__ffs (HCR_VSE ), hcr))) +vcpu_set_flag (vcpu,NESTED_SERROR_PENDING ); + +/* Re-attempt SError injection in case the deliverability has changed */ +if (unlikely (vcpu_test_and_clear_flag (vcpu,NESTED_SERROR_PENDING ))) +kvm_inject_serror_esr (vcpu,vcpu_get_vsesr (vcpu));
Why do we need to re-attempt the injection, given that we already do it on flush?
Another thing that might be worth considering is how this pending state is preserved across save/restore. The GIC provides this state implicitly for IRQ/FIQ, but we don't have an external component driving SError. Do we need to do anything about this here?
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v2 06/27] KVM: arm64: nv: Honor SError exception routing / masking
Quoted 104 lines · Marc Zyngier
Quoted 102 lines · Oliver Upton
To date KVM has used HCR_EL2.VSE to track the state of a pending SError for the guest. With this bit set, hardware respects the EL1 exception routing / masking rules and injects the vSError when appropriate.
This isn't correct for NV guests as hardware is oblivious to vEL2's intentions for SErrors. Better yet, with
FEAT_NV2the guest can change the routing behind our back asHCR_EL2is redirected to memory. Cope with this mess by:- Using a flag (instead of HCR_EL2.VSE) to track the pending SError state when SErrors are unconditionally masked for the current context
- Resampling the routing / masking of a pending SError on every guest entry/exit
- Emulating exception entry when SError routing implies a translation regime change
Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 20 +++++++++++++- arch/arm64/include/asm/kvm_host.h | 20 +++++++++++--- arch/arm64/include/asm/kvm_nested.h | 2 ++ arch/arm64/kvm/arm.c | 4 +++ arch/arm64/kvm/emulate-nested.c | 8 ++++++ arch/arm64/kvm/guest.c | 32 +++++++++++++---------- arch/arm64/kvm/handle_exit.c | 4 +-- arch/arm64/kvm/hyp/exception.c | 6 ++++- arch/arm64/kvm/inject_fault.c | 39 ++++++++++++++++------------ arch/arm64/kvm/mmu.c | 2 +- arch/arm64/kvm/nested.c | 36 +++++++++++++++++++++++++ 11 files changed, 134 insertions(+), 39 deletions(-)arch/arm64/include/asm/kvm_emulate.h +19
-1@@ -45,7 +45,7 @@ bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);void kvm_skip_instr32 (struct kvm_vcpu *vcpu);void kvm_inject_undefined (struct kvm_vcpu *vcpu); -void kvm_inject_vabt (struct kvm_vcpu *vcpu); +int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr);int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);void kvm_inject_size_fault (struct kvm_vcpu *vcpu);@@ -59,12 +59,25 @@ static inline int kvm_inject_sea_iabt(struct kvm_vcpu *vcpu, u64 addr)return kvm_inject_sea (vcpu, true, addr); } +static inline int kvm_inject_serror (struct kvm_vcpu *vcpu) +{ +/* +* ESR_ELx.ISV (later renamed to IDS) indicates whether or not +* ESR_ELx.ISS contains IMPLEMENTATION DEFINED syndrome information. +* +* Set the bit when injecting an SError w/o an ESR to indicate ISS +* does not follow the architected format. +*/ +return kvm_inject_serror_esr (vcpu, ESR_ELx_ISV); +} +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);int kvm_inject_nested_sync (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_inject_nested_irq (struct kvm_vcpu *vcpu);int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr); +int kvm_inject_nested_serror (struct kvm_vcpu *vcpu,u64 esr);static inline void kvm_inject_nested_sve_trap (struct kvm_vcpu *vcpu) {@@ -205,6 +218,11 @@ static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu)return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_TGE ; } +static inline bool vcpu_el2_amo_is_set (const struct kvm_vcpu *vcpu) +{ +return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_AMO ; +} +static inline bool is_hyp_ctxt (const struct kvm_vcpu *vcpu) {bool e2h, tge;arch/arm64/include/asm/kvm_host.h +1
-1@@ -817,7 +817,7 @@ struct kvm_vcpu_arch {u8 iflags;/* State flags for kernel bookkeeping, unused by the hypervisor code */ -u8 sflags; +u16 sflags;/* * Don't run the guest (internal implementation need). @@ -953,9 +953,23 @@ struct kvm_vcpu_arch {__vcpu_flags_preempt_enable (); \ }while (0 )
Quoted 11 lines
arch/arm64/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 963
Quoted 8 lines
arch/arm64/include/asm/kvm_host.h +8
-0+#define __vcpu_test_and_clear_flag (v, flagset, f, m) \ + ({ \ + typeof(v->arch.flagset) set; \ + \ + __vcpu_flags_preempt_disable(); \ + set = __vcpu_get_flag(v, flagset, f, m); \ + __vcpu_clear_flag(v, flagset, f, m); \ + __vcpu_flags_preempt_enable(); \I have the feeling that you can drop the preemption manipulation here. as
__vcpu_clear_flags()already does it.
Agreed, this was previously open-coded which is where I picked up the explicit preemption guard.
[...]
Quoted 15 lines
arch/arm64/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 973
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +10
-0+if (!serror_pending) +return 0 ; + +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && has_esr) +return -EINVAL ; + +if (has_esr && (esr & ~ESR_ELx_ISS_MASK)) +return -EINVAL ; + +if (has_esr)We should probably consider whether the VM itself has RAS before populating an ESR, and return an error to userspace otherwise. Unless that's yet another can of worm that we'd rather stay closed?
Quoted 1 line
I have the ugly feeling that it might be the latter...
Yeah, I'm rather hesitant to change things around here because we've made it ABI at this point. This was an oversight when we added support for writable ID registers.
The hardware sucks here because VSESR propagation happens unconditionally when FEAT_RAS exists...
Quoted 12 lines
Quoted 12 lines
arch/arm64/include/asm/kvm_host.h +12
-0+void kvm_nested_sync_hwstate (struct kvm_vcpu *vcpu) +{ +unsigned long *hcr =vcpu_hcr (vcpu); + +if (!vcpu_has_nv (vcpu)) +return ; + +/* +* We previously decided that an SError was deliverable to the guest. +* Reap the pending state from HCR_EL2 and... +*/ +if (unlikely (__test_and_clear_bit (__ffs (HCR_VSE ), hcr)))
Quoted 8 lines
arch/arm64/include/asm/kvm_host.h
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +5
-0+vcpu_set_flag (vcpu,NESTED_SERROR_PENDING ); + +/* Re-attempt SError injection in case the deliverability has changed */ +if (unlikely (vcpu_test_and_clear_flag (vcpu,NESTED_SERROR_PENDING ))) +kvm_inject_serror_esr (vcpu,vcpu_get_vsesr (vcpu));Why do we need to re-attempt the injection, given that we already do it on flush?
This bit needs a clarifying comment, because it is actually load-bearing. We need to make sure a pending SError (unmasked by AMO) is treated as a wakeup condition for WFI emulation. e.g. we may block indefinitely in the case of:
sysreg_clear_set(hcr_el2, HCR_EL2_AMO, 0); isb(); local_daif_mask(); assert(read_sysreg(isr_el1) & ISR_EL1_A); /* Pending SError */ sysreg_clear_set(hcr_el2, 0, HCR_EL2_AMO); isb(); wfi(); local_daif_restore();
For that to work we need to set the VSE line and can't check the vCPU flag itself (SErrors masked by AMO are *not* wakeup conditions).
Quoted 4 lines
Another thing that might be worth considering is how this pending state is preserved across save/restore. The GIC provides this state implicitly for IRQ/FIQ, but we don't have an external component driving SError. Do we need to do anything about this here?
I think that's working as intended, I updated KVM_GET_VCPU_EVENTS to test the flag in addition to VSE. I'll stick a mention in the changelog to make that a bit more obvious.
Thanks,
Oliver
PATCH v2 7/27 KVM: arm64: nv: Add FEAT_RAS vSError sys regs to table
In reply to Oliver UptonPrepare to implement RAS for NV by adding the missing EL2 sysregs for the vSError context.
--- arch/arm64/include/asm/kvm_host.h | 4 ++++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 2 ++ 3 files changed, 7 insertions(+)
arch/arm64/include/asm/kvm_host.h +4 -0
@@ -565,6 +565,10 @@ enum vcpu_sysreg {VNCR (POR_EL1 ),/* Permission Overlay Register 1 (EL1) */ +/* FEAT_RAS registers */ +VNCR (VDISR_EL2 ), +VNCR (VSESR_EL2 ), +VNCR (HFGRTR_EL2 ),VNCR (HFGWTR_EL2 ), VNCR(HFGITR_EL2),
arch/arm64/include/asm/vncr_mapping.h +1 -0
@@ -84,6 +84,7 @@#define VNCR_ICH_HCR_EL2 0x4C0#define VNCR_ICH_VMCR_EL2 0x4C8#define VNCR_VDISR_EL2 0x500 +#define VNCR_VSESR_EL2 0x508#define VNCR_PMBLIMITR_EL1 0x800#define VNCR_PMBPTR_EL1 0x810#define VNCR_PMBSR_EL1 0x820
arch/arm64/kvm/sys_regs.c +2 -0
@@ -3344,6 +3344,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {EL2_REG (AFSR0_EL2 , access_rw, reset_val,0 ),EL2_REG (AFSR1_EL2 , access_rw, reset_val,0 ),EL2_REG_REDIR (ESR_EL2 , reset_val,0 ), +EL2_REG_VNCR (VSESR_EL2 , reset_unknown,0 ), { SYS_DESC(SYS_FPEXC32_EL2), undef_access, reset_val, FPEXC32_EL2, 0x700 }, EL2_REG_REDIR(FAR_EL2, reset_val, 0),
@@ -3372,6 +3373,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {EL2_REG (VBAR_EL2 , access_rw, reset_val,0 ),EL2_REG (RVBAR_EL2 , access_rw, reset_val,0 ), { SYS_DESC(SYS_RMR_EL2), undef_access }, +EL2_REG_VNCR (VDISR_EL2 , reset_unknown,0 ),EL2_REG_VNCR (ICH_AP0R0_EL2 , reset_val,0 ), EL2_REG_VNCR(ICH_AP0R1_EL2, reset_val, 0),
Signature · 1 line
2.39.5
PATCH v2 8/27 KVM: arm64: nv: Use guest hypervisor's vSError state
In reply to Oliver UptonWhen HCR_EL2.AMO is set, physical SErrors are routed to EL2 and virtual SError injection is enabled for EL1. Conceptually treating host-initiated SErrors as 'physical', this means we can delegate control of the vSError injection context to the guest hypervisor when nesting && AMO is set.
--- arch/arm64/include/asm/kvm_emulate.h | 5 +++ arch/arm64/kvm/hyp/include/hyp/switch.h | 38 +++++++++++++++++++--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 14 ++++++-- 3 files changed, 50 insertions(+), 7 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +5 -0
@@ -257,6 +257,11 @@ static inline bool is_nested_ctxt(struct kvm_vcpu *vcpu)return vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu); } +static inline bool vserror_state_is_nested (struct kvm_vcpu *vcpu) +{ +return is_nested_ctxt (vcpu) &&vcpu_el2_amo_is_set (vcpu); +} +/* * The layout of SPSR for an AArch32 state is different when observed from an * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
arch/arm64/kvm/hyp/include/hyp/switch.h +33 -5
@@ -346,21 +346,49 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu, u64 hcr)write_sysreg_hcr (hcr); -if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && (hcr &HCR_VSE )) -write_sysreg_s (vcpu->arch.vsesr_el2,SYS_VSESR_EL2 ); +if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && (hcr &HCR_VSE )) { +u64 vsesr; + +/* +* When HCR_EL2.AMO is set, physical SErrors are taken to EL2 +* and vSError injection is enabled for EL1. Conveniently, for +* NV this means that it is never the case where a 'physical' +* SError (injected by KVM or userspace) and vSError are +* deliverable to the same context. +* +* As such, we can trivially select between the host or guest's +* VSESR_EL2. +*/ +if (vserror_state_is_nested (vcpu)) + vsesr =__vcpu_sys_reg (vcpu,VSESR_EL2 ); +else + vsesr = vcpu->arch.vsesr_el2; + +write_sysreg_s (vsesr,SYS_VSESR_EL2 ); + } }static inline void ___deactivate_traps (struct kvm_vcpu *vcpu) { +u64 *hcr; + +if (vserror_state_is_nested (vcpu)) + hcr =__ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ); +else + hcr = &vcpu->arch.hcr_el2; +/* * If we pended a virtual abort, preserve it until it gets * cleared. See D1.14.3 (Virtual Interrupts) for details, but * the crucial bit is "On taking a vSError interrupt, * HCR_EL2.VSE is cleared to 0." +* +* Additionally, when in a nested context we need to propagate the +* updated state to the guest hypervisor's HCR_EL2. */ -if (vcpu->arch.hcr_el2 &HCR_VSE ) { - vcpu->arch.hcr_el2 &= ~HCR_VSE ; - vcpu->arch.hcr_el2 |=read_sysreg (hcr_el2) &HCR_VSE ; +if (*hcr &HCR_VSE ) { + *hcr &= ~HCR_VSE ; + *hcr |=read_sysreg (hcr_el2) &HCR_VSE ; } }
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +12 -2
@@ -159,7 +159,12 @@ static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)if (!has_vhe () && ctxt->__hyp_running_vcpu) ctxt->regs.pstate =read_sysreg_el2 (SYS_SPSR ); -if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +return ; + +if (vserror_state_is_nested (ctxt_to_vcpu (ctxt))) +ctxt_sys_reg (ctxt,VDISR_EL2 ) =read_sysreg_s (SYS_VDISR_EL2 ); +else ctxt_sys_reg (ctxt,DISR_EL1 ) =read_sysreg_s (SYS_VDISR_EL2 ); }
@@ -293,7 +298,12 @@ static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxwrite_sysreg_el2 (ctxt->regs.pc,SYS_ELR );write_sysreg_el2 (pstate,SYS_SPSR ); -if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +return ; + +if (vserror_state_is_nested (ctxt_to_vcpu (ctxt))) +write_sysreg_s (ctxt_sys_reg (ctxt,VDISR_EL2 ),SYS_VDISR_EL2 ); +else write_sysreg_s (ctxt_sys_reg (ctxt,DISR_EL1 ),SYS_VDISR_EL2 ); }
Signature · 1 line
2.39.5
Marc Zyngier
maintainerRe: [PATCH v2 08/27] KVM: arm64: nv: Use guest hypervisor's vSError state
Quoted 111 lines · Oliver Upton
When HCR_EL2.AMO is set, physical SErrors are routed to EL2 and virtual SError injection is enabled for EL1. Conceptually treating host-initiated SErrors as 'physical', this means we can delegate control of the vSError injection context to the guest hypervisor when nesting && AMO is set.
Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 5 +++ arch/arm64/kvm/hyp/include/hyp/switch.h | 38 +++++++++++++++++++--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 14 ++++++-- 3 files changed, 50 insertions(+), 7 deletions(-)arch/arm64/include/asm/kvm_emulate.h +5
-0@@ -257,6 +257,11 @@ static inline bool is_nested_ctxt(struct kvm_vcpu *vcpu)return vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu); } +static inline bool vserror_state_is_nested (struct kvm_vcpu *vcpu) +{ +return is_nested_ctxt (vcpu) &&vcpu_el2_amo_is_set (vcpu); +} +/* * The layout of SPSR for an AArch32 state is different when observed from an * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32 arch/arm64/kvm/hyp/include/hyp/switch.h +33
-5@@ -346,21 +346,49 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu, u64 hcr)write_sysreg_hcr (hcr); -if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && (hcr &HCR_VSE )) -write_sysreg_s (vcpu->arch.vsesr_el2,SYS_VSESR_EL2 ); +if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && (hcr &HCR_VSE )) { +u64 vsesr; + +/* +* When HCR_EL2.AMO is set, physical SErrors are taken to EL2 +* and vSError injection is enabled for EL1. Conveniently, for +* NV this means that it is never the case where a 'physical' +* SError (injected by KVM or userspace) and vSError are +* deliverable to the same context. +* +* As such, we can trivially select between the host or guest's +* VSESR_EL2. +*/ +if (vserror_state_is_nested (vcpu)) + vsesr =__vcpu_sys_reg (vcpu,VSESR_EL2 ); +else + vsesr = vcpu->arch.vsesr_el2; + +write_sysreg_s (vsesr,SYS_VSESR_EL2 ); + } }static inline void ___deactivate_traps (struct kvm_vcpu *vcpu) { +u64 *hcr; + +if (vserror_state_is_nested (vcpu)) + hcr =__ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ); +else + hcr = &vcpu->arch.hcr_el2; +/* * If we pended a virtual abort, preserve it until it gets * cleared. See D1.14.3 (Virtual Interrupts) for details, but * the crucial bit is "On taking a vSError interrupt, * HCR_EL2.VSE is cleared to 0." +* +* Additionally, when in a nested context we need to propagate the +* updated state to the guest hypervisor's HCR_EL2. */ -if (vcpu->arch.hcr_el2 &HCR_VSE ) { - vcpu->arch.hcr_el2 &= ~HCR_VSE ; - vcpu->arch.hcr_el2 |=read_sysreg (hcr_el2) &HCR_VSE ; +if (*hcr &HCR_VSE ) { + *hcr &= ~HCR_VSE ; + *hcr |=read_sysreg (hcr_el2) &HCR_VSE ; } }arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +6
-1@@ -159,7 +159,12 @@ static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)if (!has_vhe () && ctxt->__hyp_running_vcpu) ctxt->regs.pstate =read_sysreg_el2 (SYS_SPSR ); -if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +return ; + +if (vserror_state_is_nested (ctxt_to_vcpu (ctxt))) +ctxt_sys_reg (ctxt,VDISR_EL2 ) =read_sysreg_s (SYS_VDISR_EL2 ); +else ctxt_sys_reg (ctxt,DISR_EL1 ) =read_sysreg_s (SYS_VDISR_EL2 ); }@@ -293,7 +298,12 @@ static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxwrite_sysreg_el2 (ctxt->regs.pc,SYS_ELR );write_sysreg_el2 (pstate,SYS_SPSR );
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctx · line 309
Quoted 10 lines
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +6
-1-if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +return ; + +if (vserror_state_is_nested (ctxt_to_vcpu (ctxt))) +write_sysreg_s (ctxt_sys_reg (ctxt,VDISR_EL2 ),SYS_VDISR_EL2 ); +else write_sysreg_s (ctxt_sys_reg (ctxt,DISR_EL1 ),SYS_VDISR_EL2 ); }
I have the same question as for one of the previous patches: should this also be gated on RAS being exposed to the guest?
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH v2 9/27 KVM: arm64: nv: Advertise support for FEAT_RAS
In reply to Oliver UptonNow that the missing bits for vSError injection/deferral have been added we can merrily claim support for FEAT_RAS.
--- arch/arm64/kvm/nested.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/arm64/kvm/nested.c +1 -2
@@ -1424,12 +1424,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)break ;case SYS_ID_AA64PFR0_EL1 : -/* No RME, AMU, MPAM, S-EL2, or RAS */ +/* No RME, AMU, MPAM, or S-EL2 */ val &= ~(ID_AA64PFR0_EL1_RME | ID_AA64PFR0_EL1_AMU | ID_AA64PFR0_EL1_MPAM | ID_AA64PFR0_EL1_SEL2 | - ID_AA64PFR0_EL1_RAS | ID_AA64PFR0_EL1_EL3 | ID_AA64PFR0_EL1_EL2 | ID_AA64PFR0_EL1_EL1 |
Signature · 1 line
2.39.5
PATCH v2 10/27 KVM: arm64: nv: Describe trap behavior of SCTLR2_EL1
In reply to Oliver UptonAdd the complete trap description for SCTLR2_EL1, including FGT and the inverted HCRX bit.
--- arch/arm64/kvm/emulate-nested.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/arm64/kvm/emulate-nested.c +12 -0
@@ -88,6 +88,7 @@ enum cgt_group_id { CGT_HCRX_EnFPM, CGT_HCRX_TCR2En, + CGT_HCRX_SCTLR2En,CGT_CNTHCTL_EL1TVT , CGT_CNTHCTL_EL1TVCT,
@@ -108,6 +109,7 @@ enum cgt_group_id {CGT_HCR_TTLB_TTLBOS ,CGT_HCR_TVM_TRVM , CGT_HCR_TVM_TRVM_HCRX_TCR2En, + CGT_HCR_TVM_TRVM_HCRX_SCTLR2En,CGT_HCR_TPU_TICAB ,CGT_HCR_TPU_TOCU , CGT_HCR_NV1_nNV2_ENSCXT,
@@ -398,6 +400,12 @@ static const struct trap_bits coarse_trap_bits[] = { .mask = HCRX_EL2_TCR2En, .behaviour = BEHAVE_FORWARD_RW, }, + [CGT_HCRX_SCTLR2En] = { + .index = HCRX_EL2, + .value = 0, + .mask = HCRX_EL2_SCTLR2En, + .behaviour = BEHAVE_FORWARD_RW, + }, [CGT_CNTHCTL_EL1TVT] = { .index = CNTHCTL_EL2, .value = CNTHCTL_EL1TVT,
@@ -449,6 +457,8 @@ static const enum cgt_group_id *coarse_control_combo[] = {MCB (CGT_HCR_TVM_TRVM ,CGT_HCR_TVM ,CGT_HCR_TRVM ),MCB (CGT_HCR_TVM_TRVM_HCRX_TCR2En,CGT_HCR_TVM ,CGT_HCR_TRVM , CGT_HCRX_TCR2En), +MCB (CGT_HCR_TVM_TRVM_HCRX_SCTLR2En, +CGT_HCR_TVM ,CGT_HCR_TRVM , CGT_HCRX_SCTLR2En),MCB (CGT_HCR_TPU_TICAB ,CGT_HCR_TPU ,CGT_HCR_TICAB ),MCB (CGT_HCR_TPU_TOCU ,CGT_HCR_TPU ,CGT_HCR_TOCU ), MCB(CGT_HCR_NV1_nNV2_ENSCXT, CGT_HCR_NV1_nNV2, CGT_HCR_ENSCXT),
@@ -782,6 +792,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = {SR_TRAP (OP_TLBI_RVALE1OSNXS ,CGT_HCR_TTLB_TTLBOS ),SR_TRAP (OP_TLBI_RVAALE1OSNXS ,CGT_HCR_TTLB_TTLBOS ),SR_TRAP (SYS_SCTLR_EL1 ,CGT_HCR_TVM_TRVM ), +SR_TRAP (SYS_SCTLR2_EL1 , CGT_HCR_TVM_TRVM_HCRX_SCTLR2En),SR_TRAP (SYS_TTBR0_EL1 ,CGT_HCR_TVM_TRVM ),SR_TRAP (SYS_TTBR1_EL1 ,CGT_HCR_TVM_TRVM ), SR_TRAP(SYS_TCR_EL1, CGT_HCR_TVM_TRVM),
@@ -1354,6 +1365,7 @@ static const struct encoding_to_trap_config encoding_to_fgt[] __initconst = {SR_FGT (SYS_SCXTNUM_EL0 ,HFGRTR ,SCXTNUM_EL0 ,1 ),SR_FGT (SYS_SCXTNUM_EL1 ,HFGRTR ,SCXTNUM_EL1 ,1 ),SR_FGT (SYS_SCTLR_EL1 ,HFGRTR ,SCTLR_EL1 ,1 ), +SR_FGT (SYS_SCTLR2_EL1 ,HFGRTR ,SCTLR_EL1 ,1 ),SR_FGT (SYS_REVIDR_EL1 ,HFGRTR ,REVIDR_EL1 ,1 ),SR_FGT (SYS_PAR_EL1 ,HFGRTR ,PAR_EL1 ,1 ), SR_FGT(SYS_MPIDR_EL1, HFGRTR, MPIDR_EL1, 1),
Signature · 1 line
2.39.5
PATCH v2 11/27 KVM: arm64: Wire up SCTLR2_ELx sysreg descriptors
In reply to Oliver UptonSet up the sysreg descriptors for SCTLR2_ELx, along with the associated storage and VNCR mapping.
--- arch/arm64/include/asm/kvm_host.h | 7 +++++++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 20 ++++++++++++++++++++ 3 files changed, 28 insertions(+)
arch/arm64/include/asm/kvm_host.h +7 -0
@@ -523,6 +523,7 @@ enum vcpu_sysreg {/* Anything from this can be RES0/RES1 sanitised */ MARKER (__SANITISED_REG_START__),TCR2_EL2 ,/* Extended Translation Control Register (EL2) */ +SCTLR2_EL2 ,/* System Control Register 2 (EL2) */ MDCR_EL2 ,/* Monitor Debug Configuration Register (EL2) */ CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
@@ -537,6 +538,7 @@ enum vcpu_sysreg {VNCR (TTBR1_EL1 ),/* Translation Table Base Register 1 */ VNCR (TCR_EL1 ),/* Translation Control Register */ VNCR (TCR2_EL1 ),/* Extended Translation Control Register */ +VNCR (SCTLR2_EL1 ),/* System Control Register 2 */ VNCR (ESR_EL1 ),/* Exception Syndrome Register */ VNCR (AFSR0_EL1 ),/* Auxiliary Fault Status Register 0 */ VNCR(AFSR1_EL1),/* Auxiliary Fault Status Register 1 */
@@ -1206,6 +1208,7 @@ static inline bool __vcpu_read_sys_reg_from_cpu(int reg, u64 *val)case IFSR32_EL2 : *val =read_sysreg_s (SYS_IFSR32_EL2 );break ;case DBGVCR32_EL2 : *val =read_sysreg_s (SYS_DBGVCR32_EL2 );break ;case ZCR_EL1 : *val =read_sysreg_s (SYS_ZCR_EL12 );break ; +case SCTLR2_EL1 : *val =read_sysreg_s (SYS_SCTLR2_EL12 );break ;default :return false; }
@@ -1256,6 +1259,7 @@ static inline bool __vcpu_write_sys_reg_to_cpu(u64 val, int reg)case IFSR32_EL2 :write_sysreg_s (val,SYS_IFSR32_EL2 );break ;case DBGVCR32_EL2 :write_sysreg_s (val,SYS_DBGVCR32_EL2 );break ;case ZCR_EL1 :write_sysreg_s (val,SYS_ZCR_EL12 );break ; +case SCTLR2_EL1 :write_sysreg_s (val,SYS_SCTLR2_EL12 );break ;default :return false; }
@@ -1686,6 +1690,9 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val);#define kvm_has_s1poe (k) \ (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) +#define kvm_has_sctlr2 (k) \ + (kvm_has_feat((k), ID_AA64MMFR3_EL1, SCTLRX, IMP)) +static inline bool kvm_arch_has_irq_bypass (void ) {return true;
arch/arm64/include/asm/vncr_mapping.h +1 -0
@@ -51,6 +51,7 @@#define VNCR_SP_EL1 0x240#define VNCR_VBAR_EL1 0x250#define VNCR_TCR2_EL1 0x270 +#define VNCR_SCTLR2_EL1 0x278#define VNCR_PIRE0_EL1 0x290#define VNCR_PIR_EL1 0x2A0#define VNCR_POR_EL1 0x2A8
arch/arm64/kvm/sys_regs.c +20 -0
@@ -144,6 +144,7 @@ static bool get_el2_to_el1_mapping(unsigned int reg,MAPPED_EL2_SYSREG (SPSR_EL2 ,SPSR_EL1 ,NULL );MAPPED_EL2_SYSREG (ZCR_EL2 ,ZCR_EL1 ,NULL );MAPPED_EL2_SYSREG (CONTEXTIDR_EL2 ,CONTEXTIDR_EL1 ,NULL ); +MAPPED_EL2_SYSREG (SCTLR2_EL2 ,SCTLR2_EL1 ,NULL );default :return false; }
@@ -2483,6 +2484,21 @@ static unsigned int vncr_el2_visibility(const struct kvm_vcpu *vcpu,return REG_HIDDEN ; } +static unsigned int sctlr2_visibility (const struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +if (kvm_has_sctlr2 (vcpu->kvm)) +return 0 ; + +return REG_HIDDEN ; +} + +static unsigned int sctlr2_el2_visibility (const struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +return __el2_visibility (vcpu, rd, sctlr2_visibility); +} +static bool access_zcr_el2 (struct kvm_vcpu *vcpu,struct sys_reg_params *p,const struct sys_reg_desc *r)
@@ -2955,6 +2971,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 }, { SYS_DESC(SYS_ACTLR_EL1), access_actlr, reset_actlr, ACTLR_EL1 }, { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 }, + { SYS_DESC(SYS_SCTLR2_EL1), access_vm_reg, reset_val, SCTLR2_EL1, 0, + .visibility = sctlr2_visibility },MTE_REG (RGSR_EL1 ), MTE_REG(GCR_EL1),
@@ -3302,6 +3320,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {EL2_REG_VNCR (VMPIDR_EL2 , reset_unknown,0 ),EL2_REG (SCTLR_EL2 , access_rw, reset_val,SCTLR_EL2_RES1 ),EL2_REG (ACTLR_EL2 , access_rw, reset_val,0 ), +EL2_REG_FILTERED (SCTLR2_EL2 , access_vm_reg, reset_val,0 , + sctlr2_el2_visibility),EL2_REG_VNCR (HCR_EL2 , reset_hcr,0 ),EL2_REG (MDCR_EL2 , access_mdcr, reset_mdcr,0 ), EL2_REG(CPTR_EL2, access_rw, reset_val, CPTR_NVHE_EL2_RES1),
Signature · 1 line
2.39.5
PATCH v2 12/27 KVM: arm64: Context switch SCTLR2_ELx when advertised to the guest
In reply to Oliver UptonRestore SCTLR2_EL1 with the correct value for the given context when FEAT_SCTLR2 is advertised to the guest.
--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 17 +++++++++++++++++ arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 6 ++++++ 2 files changed, 23 insertions(+)
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +17 -0
@@ -109,6 +109,17 @@ static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt)return kvm_has_s1poe (kern_hyp_va (vcpu->kvm)); } +static inline bool ctxt_has_sctlr2 (struct kvm_cpu_context *ctxt) +{ +struct kvm_vcpu *vcpu; + +if (!cpus_have_final_cap (ARM64_HAS_SCTLR2 )) +return false; + + vcpu =ctxt_to_vcpu (ctxt); +return kvm_has_sctlr2 (kern_hyp_va (vcpu->kvm)); +} +static inline void __sysreg_save_el1_state (struct kvm_cpu_context *ctxt) {ctxt_sys_reg (ctxt,SCTLR_EL1 ) =read_sysreg_el1 (SYS_SCTLR );
@@ -147,6 +158,9 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)ctxt_sys_reg (ctxt,SP_EL1 ) =read_sysreg (sp_el1);ctxt_sys_reg (ctxt,ELR_EL1 ) =read_sysreg_el1 (SYS_ELR );ctxt_sys_reg (ctxt,SPSR_EL1 ) =read_sysreg_el1 (SYS_SPSR ); + +if (ctxt_has_sctlr2 (ctxt)) +ctxt_sys_reg (ctxt,SCTLR2_EL1 ) =read_sysreg_el1 (SYS_SCTLR2 ); }static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt)
@@ -257,6 +271,9 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,write_sysreg (ctxt_sys_reg (ctxt,SP_EL1 ), sp_el1);write_sysreg_el1 (ctxt_sys_reg (ctxt,ELR_EL1 ),SYS_ELR );write_sysreg_el1 (ctxt_sys_reg (ctxt,SPSR_EL1 ),SYS_SPSR ); + +if (ctxt_has_sctlr2 (ctxt)) +write_sysreg_el1 (ctxt_sys_reg (ctxt,SCTLR2_EL1 ),SYS_SCTLR2 ); }/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +6 -0
@@ -77,6 +77,9 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)__vcpu_assign_sys_reg (vcpu,SP_EL2 ,read_sysreg (sp_el1));__vcpu_assign_sys_reg (vcpu,ELR_EL2 ,read_sysreg_el1 (SYS_ELR ));__vcpu_assign_sys_reg (vcpu,SPSR_EL2 ,read_sysreg_el1 (SYS_SPSR )); + +if (ctxt_has_sctlr2 (&vcpu->arch.ctxt)) +__vcpu_assign_sys_reg (vcpu,SCTLR2_EL2 ,read_sysreg_el1 (SYS_SCTLR2 )); }static void __sysreg_restore_vel2_state (struct kvm_vcpu *vcpu)
@@ -139,6 +142,9 @@ static void __sysreg_restore_vel2_state(struct kvm_vcpu *vcpu)write_sysreg (__vcpu_sys_reg (vcpu,SP_EL2 ), sp_el1);write_sysreg_el1 (__vcpu_sys_reg (vcpu,ELR_EL2 ),SYS_ELR );write_sysreg_el1 (__vcpu_sys_reg (vcpu,SPSR_EL2 ),SYS_SPSR ); + +if (ctxt_has_sctlr2 (&vcpu->arch.ctxt)) +write_sysreg_el1 (__vcpu_sys_reg (vcpu,SCTLR2_EL2 ),SYS_SCTLR2 ); }/*
Signature · 1 line
2.39.5
PATCH v2 13/27 KVM: arm64: Enable SCTLR2 when advertised to the guest
In reply to Oliver UptonHCRX_EL2.SCTLR2En needs to be set for SCTLR2_EL1 to take effect in hardware (in addition to disabling traps).
--- arch/arm64/include/asm/kvm_emulate.h | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +3 -0
@@ -727,6 +727,9 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)if (kvm_has_fpmr (kvm)) vcpu->arch.hcrx_el2 |= HCRX_EL2_EnFPM; + +if (kvm_has_sctlr2 (kvm)) + vcpu->arch.hcrx_el2 |= HCRX_EL2_SCTLR2En; } }#endif /* __ARM64_KVM_EMULATE_H__ */
Signature · 1 line
2.39.5
PATCH v2 14/27 KVM: arm64: Describe SCTLR2_ELx RESx masks
In reply to Oliver UptonExternal abort injection will soon rely on a sanitised view of SCTLR2_ELx to determine exception routing. Compute the RESx masks.
--- arch/arm64/kvm/config.c | 28 ++++++++++++++++++++++++++++ arch/arm64/kvm/nested.c | 6 ++++++ 2 files changed, 34 insertions(+)
arch/arm64/kvm/config.c +28 -0
@@ -131,6 +131,8 @@ struct reg_bits_to_feat_map {#define FEAT_SPMU ID_AA64DFR1_EL1, SPMU, IMP#define FEAT_SPE_nVM ID_AA64DFR2_EL1, SPE_nVM, IMP#define FEAT_STEP2 ID_AA64DFR2_EL1, STEP, IMP +#define FEAT_SYSREG128 ID_AA64ISAR2_EL1, SYSREG_128, IMP +#define FEAT_CPA2 ID_AA64ISAR3_EL1, CPA, CPA2static bool not_feat_aa64el3 (struct kvm *kvm) {
@@ -832,6 +834,23 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = { NEEDS_FEAT_FIXED(HCR_EL2_E2H, compute_hcr_e2h), }; +static const struct reg_bits_to_feat_map sctlr2_feat_map[] = { +NEEDS_FEAT (SCTLR2_EL1_NMEA | +SCTLR2_EL1_EASE , + FEAT_DoubleFault2), +NEEDS_FEAT (SCTLR2_EL1_EnADERR, feat_aderr), +NEEDS_FEAT (SCTLR2_EL1_EnANERR, feat_anerr), +NEEDS_FEAT (SCTLR2_EL1_EnIDCP128,FEAT_SYSREG128 ), +NEEDS_FEAT (SCTLR2_EL1_EnPACM | + SCTLR2_EL1_EnPACM0, + feat_pauth_lr), +NEEDS_FEAT (SCTLR2_EL1_CPTA | +SCTLR2_EL1_CPTA0 | +SCTLR2_EL1_CPTM | +SCTLR2_EL1_CPTM0 , +FEAT_CPA2 ), +}; +static void __init check_feat_map (const struct reg_bits_to_feat_map *map,int map_size,u64 res0,const char *str) {
@@ -863,6 +882,8 @@ void __init check_feature_map(void) __HCRX_EL2_RES0, "HCRX_EL2");check_feat_map (hcr_feat_map,ARRAY_SIZE (hcr_feat_map),HCR_EL2_RES0 ,"HCR_EL2" ); +check_feat_map (sctlr2_feat_map,ARRAY_SIZE (sctlr2_feat_map), +SCTLR2_EL1_RES0 ,"SCTLR2_EL1" ); }static bool idreg_feat_match (struct kvm *kvm,const struct reg_bits_to_feat_map *map)
@@ -1077,6 +1098,13 @@ void get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg, u64 *res0, u64 *r *res0 |=HCR_EL2_RES0 | (mask & ~fixed); *res1 =HCR_EL2_RES1 | (mask & fixed);break ; +case SCTLR2_EL1 : +case SCTLR2_EL2 : + *res0 =compute_res0_bits (kvm, sctlr2_feat_map, +ARRAY_SIZE (sctlr2_feat_map),0 ,0 ); + *res0 |=SCTLR2_EL1_RES0 ; + *res1 =SCTLR2_EL1_RES1 ; +break ;default :WARN_ON_ONCE (1 ); *res0 = *res1 =0 ;
arch/arm64/kvm/nested.c +6 -0
@@ -1690,6 +1690,12 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) res0 |=SCTLR_EL1_EPAN ;set_sysreg_masks (kvm,SCTLR_EL1 , res0, res1); +/* SCTLR2_ELx */ +get_reg_fixed_bits (kvm,SCTLR2_EL1 , &res0, &res1); +set_sysreg_masks (kvm,SCTLR2_EL1 , res0, res1); +get_reg_fixed_bits (kvm,SCTLR2_EL2 , &res0, &res1); +set_sysreg_masks (kvm,SCTLR2_EL2 , res0, res1); +/* MDCR_EL2 */ res0 =MDCR_EL2_RES0 ; res1 =MDCR_EL2_RES1 ;
Signature · 1 line
2.39.5
Marc Zyngier
maintainerRe: [PATCH v2 14/27] KVM: arm64: Describe SCTLR2_ELx RESx masks
Quoted 57 lines · Oliver Upton
External abort injection will soon rely on a sanitised view of
SCTLR2_ELxto determine exception routing. Compute the RESx masks.Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/kvm/config.c | 28 ++++++++++++++++++++++++++++ arch/arm64/kvm/nested.c | 6 ++++++ 2 files changed, 34 insertions(+)arch/arm64/kvm/config.c +21
-0@@ -131,6 +131,8 @@ struct reg_bits_to_feat_map {#define FEAT_SPMU ID_AA64DFR1_EL1, SPMU, IMP#define FEAT_SPE_nVM ID_AA64DFR2_EL1, SPE_nVM, IMP#define FEAT_STEP2 ID_AA64DFR2_EL1, STEP, IMP +#define FEAT_SYSREG128 ID_AA64ISAR2_EL1, SYSREG_128, IMP +#define FEAT_CPA2 ID_AA64ISAR3_EL1, CPA, CPA2static bool not_feat_aa64el3 (struct kvm *kvm) {@@ -832,6 +834,23 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = { NEEDS_FEAT_FIXED(HCR_EL2_E2H, compute_hcr_e2h), }; +static const struct reg_bits_to_feat_map sctlr2_feat_map[] = { +NEEDS_FEAT (SCTLR2_EL1_NMEA | +SCTLR2_EL1_EASE , + FEAT_DoubleFault2), +NEEDS_FEAT (SCTLR2_EL1_EnADERR, feat_aderr), +NEEDS_FEAT (SCTLR2_EL1_EnANERR, feat_anerr), +NEEDS_FEAT (SCTLR2_EL1_EnIDCP128,FEAT_SYSREG128 ), +NEEDS_FEAT (SCTLR2_EL1_EnPACM | + SCTLR2_EL1_EnPACM0, + feat_pauth_lr), +NEEDS_FEAT (SCTLR2_EL1_CPTA | +SCTLR2_EL1_CPTA0 | +SCTLR2_EL1_CPTM | +SCTLR2_EL1_CPTM0 , +FEAT_CPA2 ), +}; +static void __init check_feat_map (const struct reg_bits_to_feat_map *map,int map_size,u64 res0,const char *str) {@@ -863,6 +882,8 @@ void __init check_feature_map(void) __HCRX_EL2_RES0, "HCRX_EL2");check_feat_map (hcr_feat_map,ARRAY_SIZE (hcr_feat_map),HCR_EL2_RES0 ,"HCR_EL2" ); +check_feat_map (sctlr2_feat_map,ARRAY_SIZE (sctlr2_feat_map), +SCTLR2_EL1_RES0 ,"SCTLR2_EL1" ); }static bool idreg_feat_match (struct kvm *kvm,const struct reg_bits_to_feat_map *map)
arch/arm64/kvm/config.c void get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg, u64 *res0, u64 *r · line 1107
Quoted 10 lines
arch/arm64/kvm/config.c +7
-0@@ -1077,6 +1098,13 @@ void get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg, u64 *res0, u64 *r *res0 |=HCR_EL2_RES0 | (mask & ~fixed); *res1 =HCR_EL2_RES1 | (mask & fixed);break ; +case SCTLR2_EL1 : +case SCTLR2_EL2 : + *res0 =compute_res0_bits (kvm, sctlr2_feat_map, +ARRAY_SIZE (sctlr2_feat_map),0 ,0 ); + *res0 |=SCTLR2_EL1_RES0 ; + *res1 =SCTLR2_EL1_RES1 ; +break ;
This is potentially mis-describing SCTLR2_EL2.EMEC, which is conditioned on FEAT_MEC. This isn't a huge deal at this stage, but may become useful if MEC gets expanded to non-Realm PAS. I think it'd be worth adding a SCTLR2_EL2-specific table for this.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH v2 15/27 KVM: arm64: Factor out helper for selecting exception target EL
In reply to Oliver UptonPull out the exception target selection from pend_sync_exception() for general use. Use PSR_MODE_ELxh as a shorthand for the target EL, as SP_ELx selection is handled further along in the hyp's exception emulation.
--- arch/arm64/kvm/inject_fault.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-)
arch/arm64/kvm/inject_fault.c +14 -14
@@ -15,13 +15,11 @@#include <asm/kvm_nested.h> #include <asm/esr.h> -static void pend_sync_exception (struct kvm_vcpu *vcpu) +static unsigned int exception_target_el (struct kvm_vcpu *vcpu) {/* If not nesting, EL1 is the only possible exception target */ -if (likely (!vcpu_has_nv (vcpu))) { -kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); -return ; - } +if (likely (!vcpu_has_nv (vcpu))) +return PSR_MODE_EL1h;/* * With NV, we need to pick between EL1 and EL2. Note that we
@@ -32,23 +30,25 @@ static void pend_sync_exception(struct kvm_vcpu *vcpu)switch (*vcpu_cpsr (vcpu) &PSR_MODE_MASK ) {case PSR_MODE_EL2h:case PSR_MODE_EL2t: -kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); -break ; +return PSR_MODE_EL2h;case PSR_MODE_EL1h:case PSR_MODE_EL1t: -kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); -break ; +return PSR_MODE_EL1h;case PSR_MODE_EL0t: -if (vcpu_el2_tge_is_set (vcpu)) -kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); -else -kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); -break ; +return vcpu_el2_tge_is_set (vcpu) ? PSR_MODE_EL2h : PSR_MODE_EL1h;default :BUG (); } } +static void pend_sync_exception (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL1h) +kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); +else +kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); +} +static bool match_target_el (struct kvm_vcpu *vcpu,unsigned long target) {return (vcpu_get_flag (vcpu,EXCEPT_MASK ) == target);
Signature · 1 line
2.39.5
PATCH v2 16/27 KVM: arm64: nv: Ensure Address size faults affect correct ESR
In reply to Oliver UptonFor historical reasons, Address size faults are first injected into the guest as an SEA and ESR_EL1 is subsequently modified to reflect the correct FSC. Of course, when dealing with a vEL2 this should poke ESR_EL2.
--- arch/arm64/kvm/inject_fault.c | 39 +++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 18 deletions(-)
arch/arm64/kvm/inject_fault.c +21 -18
@@ -41,6 +41,22 @@ static unsigned int exception_target_el(struct kvm_vcpu *vcpu) } } +static enum vcpu_sysreg exception_esr_elx (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL2h) +return ESR_EL2 ; + +return ESR_EL1 ; +} + +static enum vcpu_sysreg exception_far_elx (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL2h) +return FAR_EL2 ; + +return FAR_EL1 ; +} +static void pend_sync_exception (struct kvm_vcpu *vcpu) {if (exception_target_el (vcpu) == PSR_MODE_EL1h)
@@ -49,11 +65,6 @@ static void pend_sync_exception(struct kvm_vcpu *vcpu)kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); } -static bool match_target_el (struct kvm_vcpu *vcpu,unsigned long target) -{ -return (vcpu_get_flag (vcpu,EXCEPT_MASK ) == target); -} -static void inject_abt64 (struct kvm_vcpu *vcpu,bool is_iabt,unsigned long addr) {unsigned long cpsr = *vcpu_cpsr (vcpu);
@@ -83,13 +94,8 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr esr |= ESR_ELx_FSC_EXTABT; -if (match_target_el (vcpu,unpack_vcpu_flag (EXCEPT_AA64_EL1_SYNC ))) { -vcpu_write_sys_reg (vcpu, addr,FAR_EL1 ); -vcpu_write_sys_reg (vcpu, esr,ESR_EL1 ); - }else { -vcpu_write_sys_reg (vcpu, addr,FAR_EL2 ); -vcpu_write_sys_reg (vcpu, esr,ESR_EL2 ); - } +vcpu_write_sys_reg (vcpu, addr,exception_far_elx (vcpu)); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); }static void inject_undef64 (struct kvm_vcpu *vcpu)
@@ -105,10 +111,7 @@ static void inject_undef64(struct kvm_vcpu *vcpu)if (kvm_vcpu_trap_il_is32bit (vcpu)) esr |= ESR_ELx_IL; -if (match_target_el (vcpu,unpack_vcpu_flag (EXCEPT_AA64_EL1_SYNC ))) -vcpu_write_sys_reg (vcpu, esr,ESR_EL1 ); -else -vcpu_write_sys_reg (vcpu, esr,ESR_EL2 ); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); }#define DFSR_FSC_EXTABT_LPAE 0x10
@@ -199,9 +202,9 @@ void kvm_inject_size_fault(struct kvm_vcpu *vcpu) !(vcpu_read_sys_reg(vcpu, TCR_EL1) & TTBCR_EAE)) return; - esr =vcpu_read_sys_reg (vcpu,ESR_EL1 ); + esr =vcpu_read_sys_reg (vcpu,exception_esr_elx (vcpu)); esr &= ~GENMASK_ULL (5 ,0 ); -vcpu_write_sys_reg (vcpu, esr,ESR_EL1 ); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); }/**
Signature · 1 line
2.39.5
PATCH v2 17/27 KVM: arm64: Route SEAs to the SError vector when EASE is set
In reply to Oliver UptonOne of the finest additions of FEAT_DoubleFault2 is the ability for software to request *synchronous* external aborts be taken to the SError vector, which of coure are *asynchronous* in nature.
Opinions be damned, implement the architecture and send SEAs to the SError vector if EASE is set for the target context.
--- arch/arm64/kvm/emulate-nested.c | 3 +++ arch/arm64/kvm/hyp/exception.c | 6 +++++- arch/arm64/kvm/inject_fault.c | 38 ++++++++++++++++++++++++++++++++- 3 files changed, 45 insertions(+), 2 deletions(-)
arch/arm64/kvm/emulate-nested.c +3 -0
@@ -2833,6 +2833,9 @@ int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr) iabt ? ESR_ELx_EC_IABT_LOW : ESR_ELx_EC_DABT_LOW); esr |= ESR_ELx_FSC_EXTABT | ESR_ELx_IL; +if (__vcpu_sys_reg (vcpu,SCTLR2_EL2 ) &SCTLR2_EL1_EASE ) +return kvm_inject_nested (vcpu, esr, except_type_serror); +return kvm_inject_s2_fault (vcpu, esr); }
arch/arm64/kvm/hyp/exception.c +5 -1
@@ -339,6 +339,10 @@ static void kvm_inject_exception(struct kvm_vcpu *vcpu)enter_exception64 (vcpu, PSR_MODE_EL1h, except_type_sync);break ; +case unpack_vcpu_flag (EXCEPT_AA64_EL1_SERR ): +enter_exception64 (vcpu, PSR_MODE_EL1h, except_type_serror); +break ; +case unpack_vcpu_flag (EXCEPT_AA64_EL2_SYNC ):enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_sync);break ;
@@ -353,7 +357,7 @@ static void kvm_inject_exception(struct kvm_vcpu *vcpu)default :/* -* Only EL1_SYNC and EL2_{SYNC,IRQ,SERR} makes +* Only EL1_{SYNC,SERR} and EL2_{SYNC,IRQ,SERR} makes * sense so far. Everything else gets silently * ignored. */
arch/arm64/kvm/inject_fault.c +37 -1
@@ -65,13 +65,49 @@ static void pend_sync_exception(struct kvm_vcpu *vcpu)kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); } +static void pend_serror_exception (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL1h) +kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SERR ); +else +kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SERR ); +} + +static bool __effective_sctlr2_bit (struct kvm_vcpu *vcpu,unsigned int idx) +{ +u64 sctlr2; + +if (!kvm_has_sctlr2 (vcpu->kvm)) +return false; + +if (is_nested_ctxt (vcpu) && + !(__vcpu_sys_reg (vcpu,HCRX_EL2 ) & HCRX_EL2_SCTLR2En)) +return false; + +if (exception_target_el (vcpu) == PSR_MODE_EL1h) + sctlr2 =vcpu_read_sys_reg (vcpu,SCTLR2_EL1 ); +else + sctlr2 =vcpu_read_sys_reg (vcpu,SCTLR2_EL2 ); + +return sctlr2 &BIT (idx); +} + +static bool effective_sctlr2_ease (struct kvm_vcpu *vcpu) +{ +return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_EASE_SHIFT ); +} +static void inject_abt64 (struct kvm_vcpu *vcpu,bool is_iabt,unsigned long addr) {unsigned long cpsr = *vcpu_cpsr (vcpu);bool is_aarch32 =vcpu_mode_is_32bit (vcpu);u64 esr =0 ; -pend_sync_exception (vcpu); +/* This delight is brought to you by FEAT_DoubleFault2. */ +if (effective_sctlr2_ease (vcpu)) +pend_serror_exception (vcpu); +else +pend_sync_exception (vcpu);/* * Build an {i,d}abort, depending on the level and the
Signature · 1 line
2.39.5
Marc Zyngier
maintainerRe: [PATCH v2 17/27] KVM: arm64: Route SEAs to the SError vector when EASE is set
Quoted 7 lines · Oliver Upton
One of the finest additions of
FEAT_DoubleFault2is the ability for software to request *synchronous* external aborts be taken to the SError vector, which of coure are *asynchronous* in nature.Opinions be damned, implement the architecture and send SEAs to the SError vector if EASE is set for the target context.
As they say, there is no accounting for taste! :)
Quoted 13 lines
Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/kvm/emulate-nested.c | 3 +++ arch/arm64/kvm/hyp/exception.c | 6 +++++- arch/arm64/kvm/inject_fault.c | 38 ++++++++++++++++++++++++++++++++- 3 files changed, 45 insertions(+), 2 deletions(-)arch/arm64/kvm/emulate-nested.c +0
-0
arch/arm64/kvm/emulate-nested.c int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr) · line 2837
Quoted 5 lines
arch/arm64/kvm/emulate-nested.c +2
-0@@ -2833,6 +2833,9 @@ int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr) iabt ? ESR_ELx_EC_IABT_LOW : ESR_ELx_EC_DABT_LOW); esr |= ESR_ELx_FSC_EXTABT | ESR_ELx_IL; +if (__vcpu_sys_reg (vcpu,SCTLR2_EL2 ) &SCTLR2_EL1_EASE ) +return kvm_inject_nested (vcpu, esr, except_type_serror);
Are we allowed to not set FAR_EL2 here? My reading of R_RYXCL is that only the exception vector changes, not what is reported.
But the spec is clear as mud, and I wonder if I'm reading too much into it.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v2 17/27] KVM: arm64: Route SEAs to the SError vector when EASE is set
Quoted 28 lines · Marc Zyngier
Quoted 7 lines · Oliver Upton
One of the finest additions of
FEAT_DoubleFault2is the ability for software to request *synchronous* external aborts be taken to the SError vector, which of coure are *asynchronous* in nature.Opinions be damned, implement the architecture and send SEAs to the SError vector if EASE is set for the target context.
As they say, there is no accounting for taste! :)
Quoted 16 lines
Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/kvm/emulate-nested.c | 3 +++ arch/arm64/kvm/hyp/exception.c | 6 +++++- arch/arm64/kvm/inject_fault.c | 38 ++++++++++++++++++++++++++++++++- 3 files changed, 45 insertions(+), 2 deletions(-)arch/arm64/kvm/emulate-nested.c +0
-0@@ -2833,6 +2833,9 @@ int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr) iabt ? ESR_ELx_EC_IABT_LOW : ESR_ELx_EC_DABT_LOW); esr |= ESR_ELx_FSC_EXTABT | ESR_ELx_IL;
Quoted 8 lines
arch/arm64/kvm/emulate-nested.c int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr) · line 2837
Quoted 2 lines
arch/arm64/kvm/emulate-nested.c +2
-0+if (__vcpu_sys_reg (vcpu,SCTLR2_EL2 ) &SCTLR2_EL1_EASE ) +return kvm_inject_nested (vcpu, esr, except_type_serror);Are we allowed to not set
FAR_EL2here? My reading ofR_RYXCLis that only the exception vector changes, not what is reported.But the spec is clear as mud, and I wonder if I'm reading too much into it.
This was an oversight on my part. My interpretation is similar to yours, where EASE allows software to choose the regular or spicy exception vector but preserves the rest of the exception context.
I've wound up fixing this by way of unhitching from kvm_inject_s2_fault()
Thanks,
Oliver
PATCH v2 18/27 KVM: arm64: nv: Handle effects of HCRX_EL2.TMEA on SError injection
In reply to Oliver UptonHCRX_EL2.TMEA further modifies the physical SError behavior where unmasked SErrors are taken to EL1 and masked SErrors are taken to EL2. This gets a bit hairy when considering the fact that TMEA also enables vSErrors, i.e. KVM has delegated the HW vSError context to the guest hypervisor.
We can keep the vSError context ownership by taking advantage of a couple properties:
- If SErrors are unmasked, the 'physical' SError can be taken in-context immediately. In other words, KVM can emulate the EL1 SError while preserving vEL2's ownership of the vSError context.
- If SErrors are masked, the 'physical' SError is taken to EL2 and needs the usual nested exception entry.
Note that the new in-context handling has the benign effect where unmasked SError injections are emulated even for non-nested VMs.
--- arch/arm64/kvm/inject_fault.c | 36 +++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-)
arch/arm64/kvm/inject_fault.c +34 -2
@@ -97,6 +97,11 @@ static bool effective_sctlr2_ease(struct kvm_vcpu *vcpu)return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_EASE_SHIFT ); } +static bool effective_sctlr2_nmea (struct kvm_vcpu *vcpu) +{ +return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_NMEA_SHIFT ); +} +static void inject_abt64 (struct kvm_vcpu *vcpu,bool is_iabt,unsigned long addr) {unsigned long cpsr = *vcpu_cpsr (vcpu);
@@ -258,14 +263,29 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)inject_undef64 (vcpu); } +static bool serror_is_masked (struct kvm_vcpu *vcpu) +{ +bool masked = *vcpu_cpsr (vcpu) &PSR_A_BIT ; + +if (!vcpu_mode_priv (vcpu)) + masked |=effective_sctlr2_nmea (vcpu); + +return masked; +} +static bool kvm_serror_target_is_el2 (struct kvm_vcpu *vcpu) { -return is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu); +if (is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu)) +return true; + +return serror_is_masked (vcpu) && + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }static bool kvm_serror_undeliverable_at_el2 (struct kvm_vcpu *vcpu) { -return !(vcpu_el2_tge_is_set (vcpu) ||vcpu_el2_amo_is_set (vcpu)); +return !(vcpu_el2_tge_is_set (vcpu) ||vcpu_el2_amo_is_set (vcpu) || +effective_sctlr2_nmea (vcpu)); }int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr)
@@ -281,6 +301,18 @@ int kvm_inject_serror_esr(struct kvm_vcpu *vcpu, u64 esr)return 1 ; } +/* +* Emulate the exception entry if SErrors are unmasked. This is useful if +* the vCPU is in a nested context w/ vSErrors enabled then we've already +* delegated he hardware vSError context (i.e. HCR_EL2.VSE, VSESR_EL2, +* VDISR_EL2) to the guest hypervisor. +*/ +if (!serror_is_masked (vcpu)) { +pend_serror_exception (vcpu); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); +return 1 ; + } +vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); *vcpu_hcr (vcpu) |=HCR_VSE ;return 1 ;
Signature · 1 line
2.39.5
Marc Zyngier
maintainerRe: [PATCH v2 18/27] KVM: arm64: nv: Handle effects of HCRX_EL2.TMEA on SError injection
Quoted 14 lines · Oliver Upton
HCRX_EL2.TMEA further modifies the physical SError behavior where unmasked SErrors are taken to EL1 and masked SErrors are taken to EL2. This gets a bit hairy when considering the fact that TMEA also enables vSErrors, i.e. KVM has delegated the HW vSError context to the guest hypervisor.
We can keep the vSError context ownership by taking advantage of a couple properties:
- If SErrors are unmasked, the 'physical' SError can be taken in-context immediately. In other words, KVM can emulate the EL1 SError while preserving vEL2's ownership of the vSError context.
Quoted 5 lines
- If SErrors are masked, the 'physical' SError is taken to EL2 and needs the usual nested exception entry.
Note that the new in-context handling has the benign effect where unmasked SError injections are emulated even for non-nested VMs.
This patch isn't *just* about HCRX_EL2.TMEA, right? Clearly, SCTLR2_ELx.NMEA plays a role. One is about routing, while the other is about bypassing PSTATE.A (NM stands for Non-Maskable). Also, TMEA affects both SEA and SError, while NMEA is SError only.
For the sake of making things a bit clearer, it might be worth either describing the effects of NMEA here, or split the NMEA handling to another patch.
Quoted 63 lines
Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/kvm/inject_fault.c | 36 +++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-)arch/arm64/kvm/inject_fault.c +27
-2@@ -97,6 +97,11 @@ static bool effective_sctlr2_ease(struct kvm_vcpu *vcpu)return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_EASE_SHIFT ); } +static bool effective_sctlr2_nmea (struct kvm_vcpu *vcpu) +{ +return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_NMEA_SHIFT ); +} +static void inject_abt64 (struct kvm_vcpu *vcpu,bool is_iabt,unsigned long addr) {unsigned long cpsr = *vcpu_cpsr (vcpu);@@ -258,14 +263,29 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)inject_undef64 (vcpu); } +static bool serror_is_masked (struct kvm_vcpu *vcpu) +{ +bool masked = *vcpu_cpsr (vcpu) &PSR_A_BIT ; + +if (!vcpu_mode_priv (vcpu)) + masked |=effective_sctlr2_nmea (vcpu); + +return masked; +} +static bool kvm_serror_target_is_el2 (struct kvm_vcpu *vcpu) { -return is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu); +if (is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu)) +return true; + +return serror_is_masked (vcpu) && + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }static bool kvm_serror_undeliverable_at_el2 (struct kvm_vcpu *vcpu) { -return !(vcpu_el2_tge_is_set (vcpu) ||vcpu_el2_amo_is_set (vcpu)); +return !(vcpu_el2_tge_is_set (vcpu) ||vcpu_el2_amo_is_set (vcpu) || +effective_sctlr2_nmea (vcpu)); }int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr)@@ -281,6 +301,18 @@ int kvm_inject_serror_esr(struct kvm_vcpu *vcpu, u64 esr)return 1 ; } +/* +* Emulate the exception entry if SErrors are unmasked. This is useful if +* the vCPU is in a nested context w/ vSErrors enabled then we've already +* delegated he hardware vSError context (i.e. HCR_EL2.VSE, VSESR_EL2, +* VDISR_EL2) to the guest hypervisor.
arch/arm64/kvm/inject_fault.c int kvm_inject_serror_esr(struct kvm_vcpu *vcpu, u64 esr) · line 318
Quoted 10 lines
arch/arm64/kvm/inject_fault.c +7
-0+*/ +if (!serror_is_masked (vcpu)) { +pend_serror_exception (vcpu); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); +return 1 ; + } +vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); *vcpu_hcr (vcpu) |=HCR_VSE ;return 1 ;
I see that the handling of TMEA affecting the routing of SEAs is in a follow-up patch, but it'd be good to either call out the split in the commit message, or have a single patch addressing all of the TMEA effects.
None of that affects the code, which seems correct (well, I think -- I can sense a headache coming!).
Thanks,
M.
Signature · 1 line
Jazz isn't dead. It just smells funny.
PATCH v2 19/27 KVM: arm64: Take "masked" SEAs to EL2 when TMEA is set
In reply to Oliver UptonSEAs can never actually be masked (they're synchronous after all), but the value of PSTATE.A used as a proxy for whether software might be in the middle of an SEA handler. And, when TMEA is set, these "masked" SEAs are actually routed to EL2.
--- arch/arm64/kvm/inject_fault.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
arch/arm64/kvm/inject_fault.c +8 -1
@@ -209,7 +209,14 @@ static void __kvm_inject_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr)static bool kvm_sea_target_is_el2 (struct kvm_vcpu *vcpu) { -return __vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_TGE |HCR_TEA ); +if (__vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_TGE |HCR_TEA )) +return true; + +if (!vcpu_mode_priv (vcpu)) +return false; + +return (*vcpu_cpsr (vcpu) &PSR_A_BIT ) && + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr)
Signature · 1 line
2.39.5
Marc Zyngier
maintainerRe: [PATCH v2 19/27] KVM: arm64: Take "masked" SEAs to EL2 when TMEA is set
Quoted 21 lines · Oliver Upton
SEAs can never actually be masked (they're synchronous after all), but the value of PSTATE.A used as a proxy for whether software might be in the middle of an SEA handler. And, when TMEA is set, these "masked" SEAs are actually routed to EL2.
Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/kvm/inject_fault.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)arch/arm64/kvm/inject_fault.c +1
-1@@ -209,7 +209,14 @@ static void __kvm_inject_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr)static bool kvm_sea_target_is_el2 (struct kvm_vcpu *vcpu) { -return __vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_TGE |HCR_TEA ); +if (__vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_TGE |HCR_TEA ))
arch/arm64/kvm/inject_fault.c static void __kvm_inject_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr) · line 222
Quoted 10 lines
arch/arm64/kvm/inject_fault.c +7
-0+return true; + +if (!vcpu_mode_priv (vcpu)) +return false; + +return (*vcpu_cpsr (vcpu) &PSR_A_BIT ) && + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr)
Only tangentially related to this patch and the previous one: what should a hypervisor do when inheriting an SEA/SError from its guest? I can't see a good way to handle that other than either terminating the guest, or ignoring the problem altogether.
But maybe that's what the design point is (eyes roll...).
M.
Signature · 1 line
Jazz isn't dead. It just smells funny.
PATCH v2 20/27 KVM: arm64: nv: Enable vSErrors when HCRX_EL2.TMEA is set
In reply to Oliver UptonPer R_CDCKC, vSErrors are enabled if HCRX_EL2.TMEA is set, regardless of HCR_EL2.AMO.
--- arch/arm64/include/asm/kvm_emulate.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_emulate.h +5 -1
@@ -259,7 +259,11 @@ static inline bool is_nested_ctxt(struct kvm_vcpu *vcpu)static inline bool vserror_state_is_nested (struct kvm_vcpu *vcpu) { -return is_nested_ctxt (vcpu) &&vcpu_el2_amo_is_set (vcpu); +if (!is_nested_ctxt (vcpu)) +return false; + +return vcpu_el2_amo_is_set (vcpu) || + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }/*
Signature · 1 line
2.39.5
PATCH v2 21/27 KVM: arm64: Advertise support for FEAT_SCTLR2
In reply to Oliver UptonEverything is in place to handle the additional state for SCTLR2_ELx, which is all that FEAT_SCTLR2 implies.
--- arch/arm64/kvm/sys_regs.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/arm64/kvm/sys_regs.c +5 -2
@@ -1644,8 +1644,10 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, val &= ~ID_AA64MMFR2_EL1_NV ;break ;case SYS_ID_AA64MMFR3_EL1 : - val &=ID_AA64MMFR3_EL1_TCRX |ID_AA64MMFR3_EL1_S1POE | -ID_AA64MMFR3_EL1_S1PIE ; + val &=ID_AA64MMFR3_EL1_TCRX | +ID_AA64MMFR3_EL1_SCTLRX | +ID_AA64MMFR3_EL1_S1POE | +ID_AA64MMFR3_EL1_S1PIE ;break ;case SYS_ID_MMFR4_EL1 : val &= ~ARM64_FEATURE_MASK (ID_MMFR4_EL1_CCIDX );
@@ -2961,6 +2963,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {ID_AA64MMFR2_EL1_NV | ID_AA64MMFR2_EL1_CCIDX)),ID_WRITABLE (ID_AA64MMFR3_EL1 , (ID_AA64MMFR3_EL1_TCRX | +ID_AA64MMFR3_EL1_SCTLRX |ID_AA64MMFR3_EL1_S1PIE |ID_AA64MMFR3_EL1_S1POE )), ID_WRITABLE(ID_AA64MMFR4_EL1, ID_AA64MMFR4_EL1_NV_frac),
Signature · 1 line
2.39.5
PATCH v2 22/27 KVM: arm64: Advertise support for FEAT_DoubleFault2
In reply to Oliver UptonKVM's external abort injection now respects the exception routing wreckage due to FEAT_DoubleFault2. Advertise the feature.
--- arch/arm64/kvm/sys_regs.c | 2 -- 1 file changed, 2 deletions(-)
arch/arm64/kvm/sys_regs.c +0 -2
@@ -1613,7 +1613,6 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_GCS ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_THE ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_MTEX ); - val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_DF2 ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_PFAR ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_MPAM_frac);break ;
@@ -2884,7 +2883,6 @@ static const struct sys_reg_desc sys_reg_descs[] = { ID_AA64PFR0_EL1_FP)), ID_FILTERED(ID_AA64PFR1_EL1, id_aa64pfr1_el1, ~(ID_AA64PFR1_EL1_PFAR | - ID_AA64PFR1_EL1_DF2 | ID_AA64PFR1_EL1_MTEX | ID_AA64PFR1_EL1_THE | ID_AA64PFR1_EL1_GCS |
Signature · 1 line
2.39.5
PATCH v2 23/27 KVM: arm64: Don't retire MMIO instruction w/ pending (emulated) SError
In reply to Oliver UptonKVM might have an emulated SError queued for the guest if userspace returned an abort for MMIO. Better yet, it could actually be a *synchronous* exception in disguise if SCTLR2_ELx.EASE is set.
Don't advance PC if KVM owes an emulated SError, just like the handling of emulated SEA injection.
--- arch/arm64/kvm/mmio.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm64/kvm/mmio.c +4 -2
@@ -72,7 +72,7 @@ unsigned long kvm_mmio_read_buf(const void *buf, unsigned int len)return data; } -static bool kvm_pending_sync_exception (struct kvm_vcpu *vcpu) +static bool kvm_pending_external_abort (struct kvm_vcpu *vcpu) {if (!vcpu_get_flag (vcpu,PENDING_EXCEPTION ))return false;
@@ -90,6 +90,8 @@ static bool kvm_pending_sync_exception(struct kvm_vcpu *vcpu)switch (vcpu_get_flag (vcpu,EXCEPT_MASK )) {case unpack_vcpu_flag (EXCEPT_AA64_EL1_SYNC ):case unpack_vcpu_flag (EXCEPT_AA64_EL2_SYNC ): +case unpack_vcpu_flag (EXCEPT_AA64_EL1_SERR ): +case unpack_vcpu_flag (EXCEPT_AA64_EL2_SERR ):return true;default :return false;
@@ -113,7 +115,7 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu)* Detect if the MMIO return was already handled or if userspace aborted * the MMIO access. */ -if (unlikely (!vcpu->mmio_needed ||kvm_pending_sync_exception (vcpu))) +if (unlikely (!vcpu->mmio_needed ||kvm_pending_external_abort (vcpu)))return 1 ; vcpu->mmio_needed =0 ;
Signature · 1 line
2.39.5
PATCH v2 24/27 KVM: arm64: selftests: Add basic SError injection test
In reply to Oliver UptonAdd tests for SError injection considering KVM is more directly involved in delivery:
- Pending SErrors are taken at the first CSE after SErrors are unmasked
- Pending SErrors aren't taken and remain pending if SErrors are masked
- Unmasked SErrors are taken immediately when injected (implementation detail)
---
tools/testing/selftests/kvm/Makefile.kvm | 2 +-
.../arm64/{mmio_abort.c => external_aborts.c} | 117 ++++++++++++++++--
.../selftests/kvm/include/arm64/processor.h | 10 ++
3 files changed, 121 insertions(+), 8 deletions(-)
rename tools/testing/selftests/kvm/arm64/{mmio_abort.c => external_aborts.c} (60%)tools/testing/selftests/kvm/Makefile.kvm +1 -1
@@ -156,7 +156,7 @@ TEST_GEN_PROGS_arm64 += arm64/arch_timer_edge_cases TEST_GEN_PROGS_arm64 += arm64/debug-exceptions TEST_GEN_PROGS_arm64 += arm64/host_sve TEST_GEN_PROGS_arm64 += arm64/hypercalls -TEST_GEN_PROGS_arm64 += arm64/mmio_abort +TEST_GEN_PROGS_arm64 += arm64/external_aborts TEST_GEN_PROGS_arm64 += arm64/page_fault_test TEST_GEN_PROGS_arm64 += arm64/psci_test TEST_GEN_PROGS_arm64 += arm64/set_id_regs
tools/testing/selftests/kvm/arm64/external_aborts.c renamed from tools/testing/selftests/kvm/arm64/mmio_abort.c +110 -7
161 lines not on this page · raw message
tools/testing/selftests/kvm/include/arm64/processor.h +10 -0
@@ -254,6 +254,16 @@ static inline void local_irq_disable(void) asmvolatile ("msr daifset, #3" : : :"memory" ); } +static inline void local_serror_enable (void ) +{ + asmvolatile ("msr daifclr, #4" : : :"memory" ); +} + +static inline void local_serror_disable (void ) +{ + asmvolatile ("msr daifset, #4" : : :"memory" ); +} +/** * struct arm_smccc_res - Result from SMC/HVC call * @a0-a3 result values from registers 0 to 3
Signature · 1 line
2.39.5
PATCH v2 25/27 KVM: arm64: selftests: Test SEAs are taken to SError vector when EASE=1
In reply to Oliver UptonEnsure KVM routes SEAs to the correct vector depending on SCTLR2_EL1.EASE.
--- .../selftests/kvm/arm64/external_aborts.c | 42 ++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/arm64/external_aborts.c +41 -1
@@ -86,7 +86,7 @@ static void vcpu_run_expect_sync(struct kvm_vcpu *vcpu)extern char test_mmio_abort_insn; -static void test_mmio_abort_guest (void ) +static noinline void test_mmio_abort_guest(void) {WRITE_ONCE (expected_abort_pc, (u64)&test_mmio_abort_insn);
@@ -251,6 +251,45 @@ static void test_serror_emulated(void)kvm_vm_free (vm); } +static void test_mmio_ease_guest (void ) +{ +sysreg_clear_set_s (SYS_SCTLR2_EL1 ,0 ,SCTLR2_EL1_EASE ); +isb (); + +test_mmio_abort_guest (); +} + +/* +* Test that KVM doesn't complete MMIO emulation when userspace has made an +* external abort pending for the instruction. +*/ +static void test_mmio_ease (void ) +{ +struct kvm_vcpu *vcpu; +struct kvm_vm *vm =vm_create_with_dabt_handler (&vcpu, test_mmio_ease_guest, + unexpected_dabt_handler); +struct kvm_run *run = vcpu->run; +u64 pfr1; + + pfr1 =vcpu_get_reg (vcpu,KVM_ARM64_SYS_REG (SYS_ID_AA64PFR1_EL1 )); +if (!SYS_FIELD_GET (ID_AA64PFR1_EL1 ,DF2 , pfr1)) { +pr_debug ("Skipping %s\n" , __func__); +return ; + } + +vm_install_exception_handler (vm,VECTOR_ERROR_CURRENT , expect_serror_handler); + +vcpu_run (vcpu); +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_MMIO ); +TEST_ASSERT_EQ (run->mmio.phys_addr,MMIO_ADDR ); +TEST_ASSERT_EQ (run->mmio.len,sizeof (unsigned long )); +TEST_ASSERT (!run->mmio.is_write,"Expected MMIO read" ); + +vcpu_inject_sea (vcpu); +vcpu_run_expect_done (vcpu); +kvm_vm_free (vm); +} +int main (void ) {test_mmio_abort ();
@@ -259,4 +298,5 @@ int main(void)test_serror ();test_serror_masked ();test_serror_emulated (); +test_mmio_ease (); }
Signature · 1 line
2.39.5
PATCH v2 26/27 KVM: arm64: selftests: Add SCTLR2_EL1 to get-reg-list
In reply to Oliver UptonHandle SCTLR2_EL1 specially as it is only visible to userspace when FEAT_SCTLR2 is implemented for the VM.
--- tools/testing/selftests/kvm/arm64/get-reg-list.c | 7 +++++++ 1 file changed, 7 insertions(+)
tools/testing/selftests/kvm/arm64/get-reg-list.c +7 -0
@@ -52,6 +52,12 @@ static struct feature_id_reg feat_id_regs[] = {ARM64_SYS_REG (3 ,0 ,0 ,7 ,3 ),/* ID_AA64MMFR3_EL1 */ 16 ,1 + }, + { +KVM_ARM64_SYS_REG (SYS_SCTLR2_EL1 ), +KVM_ARM64_SYS_REG (SYS_ID_AA64MMFR3_EL1 ), +ID_AA64MMFR3_EL1_SCTLRX_SHIFT , + ID_AA64MMFR3_EL1_SCTLRX_IMP } };
@@ -469,6 +475,7 @@ static __u64 base_regs[] = {ARM64_SYS_REG (3 ,0 ,1 ,0 ,0 ),/* SCTLR_EL1 */ ARM64_SYS_REG (3 ,0 ,1 ,0 ,1 ),/* ACTLR_EL1 */ ARM64_SYS_REG (3 ,0 ,1 ,0 ,2 ),/* CPACR_EL1 */ +KVM_ARM64_SYS_REG (SYS_SCTLR2_EL1 ),ARM64_SYS_REG (3 ,0 ,2 ,0 ,0 ),/* TTBR0_EL1 */ ARM64_SYS_REG (3 ,0 ,2 ,0 ,1 ),/* TTBR1_EL1 */ ARM64_SYS_REG(3, 0, 2, 0, 2), /* TCR_EL1 */
Signature · 1 line
2.39.5
PATCH v2 27/27 KVM: arm64: selftests: Catch up set_id_regs with the kernel
In reply to Oliver UptonAdd test coverage for ID_AA64MMFR3_EL1 and the recently added FEAT_DoubleFault2.
--- tools/testing/selftests/kvm/arm64/set_id_regs.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/arm64/set_id_regs.c +12 -2
@@ -139,6 +139,7 @@ static const struct reg_ftr_bits ftr_id_aa64pfr0_el1[] = { };static const struct reg_ftr_bits ftr_id_aa64pfr1_el1[] = { +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 ,DF2 ,0 ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 , CSV2_frac,0 ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 ,SSBS ,ID_AA64PFR1_EL1_SSBS_NI ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 ,BT ,0 ),
@@ -187,6 +188,14 @@ static const struct reg_ftr_bits ftr_id_aa64mmfr2_el1[] = { REG_FTR_END, }; +static const struct reg_ftr_bits ftr_id_aa64mmfr3_el1[] = { +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,S1POE ,0 ), +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,S1PIE ,0 ), +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,SCTLRX ,0 ), +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,TCRX ,0 ), +REG_FTR_END , +}; +static const struct reg_ftr_bits ftr_id_aa64zfr0_el1[] = {REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64ZFR0_EL1 ,F64MM ,0 ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64ZFR0_EL1 ,F32MM ,0 ),
@@ -217,6 +226,7 @@ static struct test_feature_reg test_regs[] = {TEST_REG (SYS_ID_AA64MMFR0_EL1 , ftr_id_aa64mmfr0_el1),TEST_REG (SYS_ID_AA64MMFR1_EL1 , ftr_id_aa64mmfr1_el1),TEST_REG (SYS_ID_AA64MMFR2_EL1 , ftr_id_aa64mmfr2_el1), +TEST_REG (SYS_ID_AA64MMFR3_EL1 , ftr_id_aa64mmfr3_el1), TEST_REG(SYS_ID_AA64ZFR0_EL1, ftr_id_aa64zfr0_el1), };
@@ -774,8 +784,8 @@ int main(void)ARRAY_SIZE (ftr_id_aa64isar2_el1) +ARRAY_SIZE (ftr_id_aa64pfr0_el1) +ARRAY_SIZE (ftr_id_aa64pfr1_el1) +ARRAY_SIZE (ftr_id_aa64mmfr0_el1) +ARRAY_SIZE (ftr_id_aa64mmfr1_el1) +ARRAY_SIZE (ftr_id_aa64mmfr2_el1) + -ARRAY_SIZE (ftr_id_aa64zfr0_el1) -ARRAY_SIZE (test_regs) +3 + -MPAM_IDREG_TEST +MTE_IDREG_TEST ; +ARRAY_SIZE (ftr_id_aa64mmfr3_el1) +ARRAY_SIZE (ftr_id_aa64zfr0_el1) - +ARRAY_SIZE (test_regs) +3 +MPAM_IDREG_TEST +MTE_IDREG_TEST ;ksft_set_plan (test_cnt);
Signature · 1 line
2.39.5
Quoted 8 lines · Oliver Upton
Well... this series grew a bit. Thanks to some leading review feedback in v1, I went about implementing
FEAT_RAS,FEAT_SCLTR2, andFEAT_DoubleFault2for NV (and non-NV, where applicable).The most annoying addition of the whole bunch is certainly TMEA, which has the effect of enabling vSErrors and potentially making 'physical' SErrors deliverable to the same context.
Apart from a small number of minor issues, this looks really good. Thanks a lot for going the extra mile and posting a consistent set of features!
With these issues addressed:
M.
Signature · 1 line
Jazz isn't dead. It just smells funny.