KVM: arm64: SCTLR2, DoubleFault2, and NV external abort fixes
v3 cover KVM: arm64: SCTLR2, DoubleFault2, and NV external abort fixes
v2: https://lore.kernel.org/kvmarm/[email protected]/
v2 -> v3:
- Fix mask computation when SCLTR2_ELx.NMEA is set
- Consolidate TMEA handling for all aborts, spin off NMEA (Marc)
- Set up the fault context correctly for the injected abort (e.g. not setting HPFAR_EL2 when UNKNOWN) (Marc)
- Don't use the guest hypervisor's VSESR when FEAT_RAS isn't implemented for the guest (Marc)
Marc Zyngier (1):
KVM: arm64: Add helper to identify a nested context
Oliver Upton (26):
arm64: Detect FEAT_SCTLR2
arm64: Detect FEAT_DoubleFault2
KVM: arm64: Treat vCPU with pending SError as runnable
KVM: arm64: nv: Respect exception routing rules for SEAs
KVM: arm64: nv: Honor SError exception routing / masking
KVM: arm64: nv: Add FEAT_RAS vSError sys regs to table
KVM: arm64: nv: Use guest hypervisor's vSError state
KVM: arm64: nv: Advertise support for FEAT_RAS
KVM: arm64: nv: Describe trap behavior of SCTLR2_EL1
KVM: arm64: Wire up SCTLR2_ELx sysreg descriptors
KVM: arm64: Context switch SCTLR2_ELx when advertised to the guest
KVM: arm64: Enable SCTLR2 when advertised to the guest
KVM: arm64: Describe SCTLR2_ELx RESx masks
KVM: arm64: Factor out helper for selecting exception target EL
KVM: arm64: nv: Ensure Address size faults affect correct ESR
KVM: arm64: Route SEAs to the SError vector when EASE is set
KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set
KVM: arm64: nv: Honor SError routing effects of SCTLR2_ELx.NMEA
KVM: arm64: nv: Enable vSErrors when HCRX_EL2.TMEA is set
KVM: arm64: Advertise support for FEAT_SCTLR2
KVM: arm64: Advertise support for FEAT_DoubleFault2
KVM: arm64: Don't retire MMIO instruction w/ pending (emulated) SError
KVM: arm64: selftests: Add basic SError injection test
KVM: arm64: selftests: Test SEAs are taken to SError vector when
EASE=1
KVM: arm64: selftests: Add SCTLR2_EL1 to get-reg-list
KVM: arm64: selftests: Catch up set_id_regs with the kernel
arch/arm64/include/asm/kvm_emulate.h | 51 +++-
arch/arm64/include/asm/kvm_host.h | 32 ++-
arch/arm64/include/asm/kvm_nested.h | 2 +
arch/arm64/include/asm/vncr_mapping.h | 2 +
arch/arm64/kernel/cpufeature.c | 9 +
arch/arm64/kvm/arch_timer.c | 2 +-
arch/arm64/kvm/arm.c | 9 +-
arch/arm64/kvm/config.c | 28 +++
arch/arm64/kvm/emulate-nested.c | 49 +++-
arch/arm64/kvm/guest.c | 33 +--
arch/arm64/kvm/handle_exit.c | 24 +-
arch/arm64/kvm/hyp/exception.c | 10 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 53 +++-
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 48 +++-
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 6 +
arch/arm64/kvm/inject_fault.c | 231 ++++++++++++------
arch/arm64/kvm/mmio.c | 12 +-
arch/arm64/kvm/mmu.c | 17 +-
arch/arm64/kvm/nested.c | 49 +++-
arch/arm64/kvm/sys_regs.c | 31 ++-
arch/arm64/kvm/vgic/vgic-v3-nested.c | 2 +-
arch/arm64/tools/cpucaps | 1 +
tools/testing/selftests/kvm/Makefile.kvm | 2 +-
.../arm64/{mmio_abort.c => external_aborts.c} | 159 +++++++++++-
.../selftests/kvm/arm64/get-reg-list.c | 7 +
.../testing/selftests/kvm/arm64/set_id_regs.c | 14 +-
.../selftests/kvm/include/arm64/processor.h | 10 +
28 files changed, 725 insertions(+), 170 deletions(-)
rename tools/testing/selftests/kvm/arm64/{mmio_abort.c => external_aborts.c} (50%)base-commit: 86731a2a651e58953fc949573895f2fa6d456841
Signature · 1 line
2.39.5
PATCH v3 1/27 arm64: Detect FEAT_SCTLR2
KVM is about to pick up support for SCTLR2. Add cpucap for later use in the guest/host context switch hot path.
--- arch/arm64/kernel/cpufeature.c | 8 ++++++++ arch/arm64/tools/cpucaps | 1 + 2 files changed, 9 insertions(+)
arch/arm64/kernel/cpufeature.c +8 -0
@@ -500,6 +500,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr3[] = {ARM64_FTR_BITS (FTR_VISIBLE_IF_IS_ENABLED (CONFIG_ARM64_POE ),FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_S1POE_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_S1PIE_SHIFT ,4 ,0 ), +ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_SCTLRX_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1_TCRX_SHIFT ,4 ,0 ), ARM64_FTR_END, };
@@ -3061,6 +3062,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_pmuv3, },#endif + { + .desc = "SCTLR2", + .capability = ARM64_HAS_SCTLR2, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR3_EL1 ,SCTLRX ,IMP ) + }, {}, };
arch/arm64/tools/cpucaps +1 -0
@@ -49,6 +49,7 @@ HAS_PAN HAS_PMUV3 HAS_S1PIE HAS_S1POE +HAS_SCTLR2 HAS_RAS_EXTN HAS_RNG HAS_SB
Signature · 1 line
2.39.5
PATCH v3 2/27 arm64: Detect FEAT_DoubleFault2
In reply to Oliver UptonKVM will soon support FEAT_DoubleFault2. Add a descriptor for the corresponding ID register field.
--- arch/arm64/kernel/cpufeature.c | 1 + 1 file changed, 1 insertion(+)
arch/arm64/kernel/cpufeature.c +1 -0
@@ -303,6 +303,7 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { };static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = { +ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64PFR1_EL1_DF2_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_VISIBLE_IF_IS_ENABLED (CONFIG_ARM64_GCS ),FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64PFR1_EL1_GCS_SHIFT ,4 ,0 ),S_ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE , ID_AA64PFR1_EL1_MTE_frac_SHIFT,4 ,0 ),
Signature · 1 line
2.39.5
PATCH v3 3/27 KVM: arm64: Add helper to identify a nested context
In reply to Oliver UptonFrom: Marc Zyngier <[email protected]>
A common idiom in the KVM code is to check if we are currently dealing with a "nested" context, defined as having NV enabled, but being in the EL1&0 translation regime.
This is usually expressed as:
if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu) ... )
which is a mouthful and a bit hard to read, specially when followed by additional conditions.
Introduce a new helper that encapsulate these two terms, allowing the above to be written as
if (is_nested_context(vcpu) ... )
which is both shorter and easier to read, and makes more obvious the potential for simplification on some code paths.
--- arch/arm64/include/asm/kvm_emulate.h | 5 +++++ arch/arm64/kvm/arch_timer.c | 2 +- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/emulate-nested.c | 9 ++------- arch/arm64/kvm/handle_exit.c | 20 +++++++------------- arch/arm64/kvm/hyp/include/hyp/switch.h | 8 ++++---- arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +- arch/arm64/kvm/vgic/vgic-v3-nested.c | 2 +- 8 files changed, 22 insertions(+), 28 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +5 -0
@@ -224,6 +224,11 @@ static inline bool vcpu_is_host_el0(const struct kvm_vcpu *vcpu)return is_hyp_ctxt (vcpu) && !vcpu_is_el2 (vcpu); } +static inline bool is_nested_ctxt (struct kvm_vcpu *vcpu) +{ +return vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu); +} +/* * The layout of SPSR for an AArch32 state is different when observed from an * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
arch/arm64/kvm/arch_timer.c +1 -1
@@ -830,7 +830,7 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)* by the guest (either FEAT_VHE or FEAT_E2H0 is implemented, but * not both). This simplifies the handling of the EL1NV* bits. */ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) {u64 val =__vcpu_sys_reg (vcpu,CNTHCTL_EL2 );/* Use the VHE format for mental sanity */
arch/arm64/kvm/arm.c +1 -1
@@ -521,7 +521,7 @@ static void vcpu_set_pauth_traps(struct kvm_vcpu *vcpu)* Either we're running an L2 guest, and the API/APK bits come * from L1's HCR_EL2, or API/APK are both set. */ -if (unlikely (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu))) { +if (unlikely (is_nested_ctxt (vcpu))) {u64 val; val =__vcpu_sys_reg (vcpu,HCR_EL2 );
arch/arm64/kvm/emulate-nested.c +2 -7
@@ -2592,13 +2592,8 @@ bool triage_sysreg_trap(struct kvm_vcpu *vcpu, int *sr_index)static bool __forward_traps (struct kvm_vcpu *vcpu,unsigned int reg,u64 control_bit) { -bool control_bit_set; - -if (!vcpu_has_nv (vcpu)) -return false; - - control_bit_set =__vcpu_sys_reg (vcpu, reg) & control_bit; -if (!is_hyp_ctxt (vcpu) && control_bit_set) { +if (is_nested_ctxt (vcpu) && + (__vcpu_sys_reg (vcpu, reg) & control_bit)) {kvm_inject_nested_sync (vcpu,kvm_vcpu_get_esr (vcpu));return true; }
arch/arm64/kvm/handle_exit.c +7 -13
@@ -252,7 +252,7 @@ static int kvm_handle_ptrauth(struct kvm_vcpu *vcpu)return 1 ; } -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) {kvm_inject_nested_sync (vcpu,kvm_vcpu_get_esr (vcpu));return 1 ; }
@@ -311,12 +311,11 @@ static int kvm_handle_gcs(struct kvm_vcpu *vcpu)static int handle_other (struct kvm_vcpu *vcpu) { -bool is_l2 =vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu); +bool allowed, fwd =is_nested_ctxt (vcpu);u64 hcrx =__vcpu_sys_reg (vcpu,HCRX_EL2 );u64 esr =kvm_vcpu_get_esr (vcpu);u64 iss =ESR_ELx_ISS (esr);struct kvm *kvm = vcpu->kvm; -bool allowed, fwd = false;/* * We only trap for two reasons:
@@ -335,28 +334,23 @@ static int handle_other(struct kvm_vcpu *vcpu)switch (iss) {case ESR_ELx_ISS_OTHER_ST64BV: allowed =kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_V ); -if (is_l2) - fwd = !(hcrx & HCRX_EL2_EnASR); + fwd &= !(hcrx & HCRX_EL2_EnASR);break ;case ESR_ELx_ISS_OTHER_ST64BV0: allowed =kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_ACCDATA ); -if (is_l2) - fwd = !(hcrx & HCRX_EL2_EnAS0); + fwd &= !(hcrx & HCRX_EL2_EnAS0);break ;case ESR_ELx_ISS_OTHER_LDST64B: allowed =kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64 ); -if (is_l2) - fwd = !(hcrx & HCRX_EL2_EnALS); + fwd &= !(hcrx & HCRX_EL2_EnALS);break ;case ESR_ELx_ISS_OTHER_TSBCSYNC: allowed =kvm_has_feat (kvm,ID_AA64DFR0_EL1 , TraceBuffer,TRBE_V1P1 ); -if (is_l2) - fwd = (__vcpu_sys_reg (vcpu,HFGITR2_EL2 ) &HFGITR2_EL2_TSBCSYNC ); + fwd &= (__vcpu_sys_reg (vcpu,HFGITR2_EL2 ) &HFGITR2_EL2_TSBCSYNC );break ;case ESR_ELx_ISS_OTHER_PSBCSYNC: allowed =kvm_has_feat (kvm,ID_AA64DFR0_EL1 , PMSVer,V1P5 ); -if (is_l2) - fwd = (__vcpu_sys_reg (vcpu,HFGITR_EL2 ) &HFGITR_EL2_PSBCSYNC ); + fwd &= (__vcpu_sys_reg (vcpu,HFGITR_EL2 ) &HFGITR_EL2_PSBCSYNC );break ;default :/* Clearly, we're missing something. */
arch/arm64/kvm/hyp/include/hyp/switch.h +4 -4
@@ -298,7 +298,7 @@ static inline void __deactivate_cptr_traps(struct kvm_vcpu *vcpu)u64 val; \ \ ctxt_sys_reg(hctxt, reg) = read_sysreg_s(SYS_ ## reg); \ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) \ +if (is_nested_ctxt (vcpu)) \compute_clr_set (vcpu, reg, c, s); \ \compute_undef_clr_set (vcpu, kvm, reg, c, s); \
@@ -436,7 +436,7 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)if (cpus_have_final_cap (ARM64_HAS_HCX )) {u64 hcrx = vcpu->arch.hcrx_el2; -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) {u64 val =__vcpu_sys_reg (vcpu,HCRX_EL2 ); hcrx |= val & __HCRX_EL2_MASK; hcrx &= ~(~val & __HCRX_EL2_nMASK);
@@ -531,7 +531,7 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)* nested guest, as the guest hypervisor could select a smaller VL. Slap * that into hardware before wrapping up. */ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) +if (is_nested_ctxt (vcpu))sve_cond_update_zcr_vq (__vcpu_sys_reg (vcpu,ZCR_EL2 ),SYS_ZCR_EL2 );write_sysreg_el1 (__vcpu_sys_reg (vcpu,vcpu_sve_zcr_elx (vcpu)),SYS_ZCR );
@@ -557,7 +557,7 @@ static inline void fpsimd_lazy_switch_to_guest(struct kvm_vcpu *vcpu)if (vcpu_has_sve (vcpu)) {/* A guest hypervisor may restrict the effective max VL. */ -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) +if (is_nested_ctxt (vcpu)) zcr_el2 =__vcpu_sys_reg (vcpu,ZCR_EL2 );else zcr_el2 =vcpu_sve_max_vq (vcpu) -1 ;
arch/arm64/kvm/hyp/vgic-v3-sr.c +1 -1
@@ -1050,7 +1050,7 @@ static bool __vgic_v3_check_trap_forwarding(struct kvm_vcpu *vcpu, {u64 ich_hcr; -if (!vcpu_has_nv (vcpu) ||is_hyp_ctxt (vcpu)) +if (!is_nested_ctxt (vcpu))return false; ich_hcr =__vcpu_sys_reg (vcpu,ICH_HCR_EL2 );
arch/arm64/kvm/vgic/vgic-v3-nested.c +1 -1
@@ -116,7 +116,7 @@ bool vgic_state_is_nested(struct kvm_vcpu *vcpu) {u64 xmo; -if (vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu)) { +if (is_nested_ctxt (vcpu)) { xmo =__vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_IMO |HCR_FMO );WARN_ONCE (xmo && xmo != (HCR_IMO |HCR_FMO ),"Separate virtual IRQ/FIQ settings not supported\n" );
Signature · 1 line
2.39.5
PATCH v3 4/27 KVM: arm64: Treat vCPU with pending SError as runnable
In reply to Oliver UptonPer R_VRLPB, a pending SError is a WFI wakeup event regardless of PSTATE.A, meaning that the vCPU is runnable. Sample VSE in addition to the other IRQ lines.
--- arch/arm64/kvm/arm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/arm64/kvm/arm.c +2 -1
@@ -740,7 +740,8 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,*/ int kvm_arch_vcpu_runnable (struct kvm_vcpu *v) { -bool irq_lines = *vcpu_hcr (v) & (HCR_VI |HCR_VF ); +bool irq_lines = *vcpu_hcr (v) & (HCR_VI |HCR_VF |HCR_VSE ); +return ((irq_lines ||kvm_vgic_vcpu_pending_irq (v)) && !kvm_arm_vcpu_stopped (v) && !v->arch.pause); }
Signature · 1 line
2.39.5
PATCH v3 5/27 KVM: arm64: nv: Respect exception routing rules for SEAs
In reply to Oliver UptonSynchronous external aborts are taken to EL2 if ELIsInHost() or HCR_EL2.TEA=1. Rework the SEA injection plumbing to respect the imposed routing of the guest hypervisor and opportunistically rephrase things to make their function a bit more obvious.
--- arch/arm64/include/asm/kvm_emulate.h | 14 +++++++-- arch/arm64/kvm/emulate-nested.c | 10 +++++++ arch/arm64/kvm/guest.c | 5 ++-- arch/arm64/kvm/inject_fault.c | 45 +++++++++++----------------- arch/arm64/kvm/mmio.c | 6 ++-- arch/arm64/kvm/mmu.c | 15 +++------- 6 files changed, 48 insertions(+), 47 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +12 -2
@@ -46,15 +46,25 @@ void kvm_skip_instr32(struct kvm_vcpu *vcpu);void kvm_inject_undefined (struct kvm_vcpu *vcpu);void kvm_inject_vabt (struct kvm_vcpu *vcpu); -void kvm_inject_dabt (struct kvm_vcpu *vcpu,unsigned long addr); -void kvm_inject_pabt (struct kvm_vcpu *vcpu,unsigned long addr); +int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);void kvm_inject_size_fault (struct kvm_vcpu *vcpu); +static inline int kvm_inject_sea_dabt (struct kvm_vcpu *vcpu,u64 addr) +{ +return kvm_inject_sea (vcpu, false, addr); +} + +static inline int kvm_inject_sea_iabt (struct kvm_vcpu *vcpu,u64 addr) +{ +return kvm_inject_sea (vcpu, true, addr); +} +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);int kvm_inject_nested_sync (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_inject_nested_irq (struct kvm_vcpu *vcpu); +int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);static inline void kvm_inject_nested_sve_trap (struct kvm_vcpu *vcpu) {
arch/arm64/kvm/emulate-nested.c +10 -0
@@ -2811,3 +2811,13 @@ int kvm_inject_nested_irq(struct kvm_vcpu *vcpu)/* esr_el2 value doesn't matter for exits due to irqs. */ return kvm_inject_nested (vcpu,0 , except_type_irq); } + +int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr) +{ +u64 esr =FIELD_PREP (ESR_ELx_EC_MASK, + iabt ? ESR_ELx_EC_IABT_LOW : ESR_ELx_EC_DABT_LOW); + esr |= ESR_ELx_FSC_EXTABT | ESR_ELx_IL; + +vcpu_write_sys_reg (vcpu,FAR_EL2 , addr); +return kvm_inject_nested_sync (vcpu, esr); +}
arch/arm64/kvm/guest.c +3 -2
@@ -839,6 +839,7 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,bool serror_pending = events->exception.serror_pending;bool has_esr = events->exception.serror_has_esr;bool ext_dabt_pending = events->exception.ext_dabt_pending; +int ret =0 ;if (serror_pending && has_esr) {if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN ))
@@ -853,9 +854,9 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, }if (ext_dabt_pending) -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); + ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); -return 0 ; +return ret; } u32 __attribute_const__ kvm_target_cpu(void)
arch/arm64/kvm/inject_fault.c +17 -28
63 lines not on this page · raw message
arch/arm64/kvm/mmio.c +2 -4
@@ -169,10 +169,8 @@ int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)trace_kvm_mmio_nisv (*vcpu_pc (vcpu),kvm_vcpu_get_esr (vcpu),kvm_vcpu_get_hfar (vcpu), fault_ipa); -if (vcpu_is_protected (vcpu)) { -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); -return 1 ; - } +if (vcpu_is_protected (vcpu)) +return kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu));if (test_bit (KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER , &vcpu->kvm->arch.flags)) {
arch/arm64/kvm/mmu.c +4 -11
@@ -1836,11 +1836,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)if (fault_ipa >=BIT_ULL (VTCR_EL2_IPA (vcpu->arch.hw_mmu->vtcr))) { fault_ipa |=kvm_vcpu_get_hfar (vcpu) &GENMASK (11 ,0 ); -if (is_iabt) -kvm_inject_pabt (vcpu, fault_ipa); -else -kvm_inject_dabt (vcpu, fault_ipa); -return 1 ; +return kvm_inject_sea (vcpu, is_iabt, fault_ipa); } }
@@ -1912,8 +1908,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) }if (kvm_vcpu_abt_iss1tw (vcpu)) { -kvm_inject_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); - ret =1 ; + ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); goto out_unlock; }
@@ -1958,10 +1953,8 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)if (ret ==0 ) ret =1 ; out: -if (ret == -ENOEXEC ) { -kvm_inject_pabt (vcpu,kvm_vcpu_get_hfar (vcpu)); - ret =1 ; - } +if (ret == -ENOEXEC ) + ret =kvm_inject_sea_iabt (vcpu,kvm_vcpu_get_hfar (vcpu)); out_unlock:srcu_read_unlock (&vcpu->kvm->srcu, idx);return ret;
Signature · 1 line
2.39.5
PATCH v3 6/27 KVM: arm64: nv: Honor SError exception routing / masking
In reply to Oliver UptonTo date KVM has used HCR_EL2.VSE to track the state of a pending SError for the guest. With this bit set, hardware respects the EL1 exception routing / masking rules and injects the vSError when appropriate.
This isn't correct for NV guests as hardware is oblivious to vEL2's intentions for SErrors. Better yet, with FEAT_NV2 the guest can change the routing behind our back as HCR_EL2 is redirected to memory. Cope with this mess by:
- Using a flag (instead of HCR_EL2.VSE) to track the pending SError state when SErrors are unconditionally masked for the current context
- Resampling the routing / masking of a pending SError on every guest entry/exit
- Emulating exception entry when SError routing implies a translation regime change
--- arch/arm64/include/asm/kvm_emulate.h | 20 +++++++++++++- arch/arm64/include/asm/kvm_host.h | 18 ++++++++++--- arch/arm64/include/asm/kvm_nested.h | 2 ++ arch/arm64/kvm/arm.c | 4 +++ arch/arm64/kvm/emulate-nested.c | 14 ++++++++++ arch/arm64/kvm/guest.c | 32 ++++++++++++---------- arch/arm64/kvm/handle_exit.c | 4 +-- arch/arm64/kvm/hyp/exception.c | 6 ++++- arch/arm64/kvm/inject_fault.c | 39 +++++++++++++++------------ arch/arm64/kvm/mmu.c | 2 +- arch/arm64/kvm/nested.c | 40 ++++++++++++++++++++++++++++ 11 files changed, 142 insertions(+), 39 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +19 -1
@@ -45,7 +45,7 @@ bool kvm_condition_valid32(const struct kvm_vcpu *vcpu);void kvm_skip_instr32 (struct kvm_vcpu *vcpu);void kvm_inject_undefined (struct kvm_vcpu *vcpu); -void kvm_inject_vabt (struct kvm_vcpu *vcpu); +int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr);int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr);void kvm_inject_size_fault (struct kvm_vcpu *vcpu);
@@ -59,12 +59,25 @@ static inline int kvm_inject_sea_iabt(struct kvm_vcpu *vcpu, u64 addr)return kvm_inject_sea (vcpu, true, addr); } +static inline int kvm_inject_serror (struct kvm_vcpu *vcpu) +{ +/* +* ESR_ELx.ISV (later renamed to IDS) indicates whether or not +* ESR_ELx.ISS contains IMPLEMENTATION DEFINED syndrome information. +* +* Set the bit when injecting an SError w/o an ESR to indicate ISS +* does not follow the architected format. +*/ +return kvm_inject_serror_esr (vcpu, ESR_ELx_ISV); +} +void kvm_vcpu_wfi (struct kvm_vcpu *vcpu);void kvm_emulate_nested_eret (struct kvm_vcpu *vcpu);int kvm_inject_nested_sync (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_inject_nested_irq (struct kvm_vcpu *vcpu);int kvm_inject_nested_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr); +int kvm_inject_nested_serror (struct kvm_vcpu *vcpu,u64 esr);static inline void kvm_inject_nested_sve_trap (struct kvm_vcpu *vcpu) {
@@ -205,6 +218,11 @@ static inline bool vcpu_el2_tge_is_set(const struct kvm_vcpu *vcpu)return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_TGE ; } +static inline bool vcpu_el2_amo_is_set (const struct kvm_vcpu *vcpu) +{ +return ctxt_sys_reg (&vcpu->arch.ctxt,HCR_EL2 ) &HCR_AMO ; +} +static inline bool is_hyp_ctxt (const struct kvm_vcpu *vcpu) {bool e2h, tge;
arch/arm64/include/asm/kvm_host.h +15 -3
@@ -817,7 +817,7 @@ struct kvm_vcpu_arch {u8 iflags;/* State flags for kernel bookkeeping, unused by the hypervisor code */ -u8 sflags; +u16 sflags;/* * Don't run the guest (internal implementation need).
@@ -953,9 +953,21 @@ struct kvm_vcpu_arch {__vcpu_flags_preempt_enable (); \ } while (0) +#define __vcpu_test_and_clear_flag (v, flagset, f, m) \ + ({ \ + typeof(v->arch.flagset) set; \ + \ + set = __vcpu_get_flag(v, flagset, f, m); \ + __vcpu_clear_flag(v, flagset, f, m); \ + \ + set; \ + }) +#define vcpu_get_flag (v, ...) __vcpu_get_flag((v), __VA_ARGS__)#define vcpu_set_flag (v, ...) __vcpu_set_flag((v), __VA_ARGS__)#define vcpu_clear_flag (v, ...) __vcpu_clear_flag((v), __VA_ARGS__) +#define vcpu_test_and_clear_flag (v, ...) \ + __vcpu_test_and_clear_flag((v), __VA_ARGS__)/* KVM_ARM_VCPU_INIT completed */ #define VCPU_INITIALIZED __vcpu_single_flag(cflags, BIT(0))
@@ -1015,6 +1027,8 @@ struct kvm_vcpu_arch {#define IN_WFI __vcpu_single_flag(sflags, BIT(6))/* KVM is currently emulating a nested ERET */ #define IN_NESTED_ERET __vcpu_single_flag(sflags, BIT(7)) +/* SError pending for nested guest */ +#define NESTED_SERROR_PENDING __vcpu_single_flag(sflags, BIT(8))/* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */
@@ -1387,8 +1401,6 @@ static inline bool kvm_arm_is_pvtime_enabled(struct kvm_vcpu_arch *vcpu_arch)return (vcpu_arch->steal.base !=INVALID_GPA ); } -void kvm_set_sei_esr (struct kvm_vcpu *vcpu,u64 syndrome); -struct kvm_vcpu *kvm_mpidr_to_vcpu (struct kvm *kvm,unsigned long mpidr); DECLARE_KVM_HYP_PER_CPU(struct kvm_host_data, kvm_host_data);
arch/arm64/include/asm/kvm_nested.h +2 -0
@@ -80,6 +80,8 @@ extern void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu);extern void kvm_vcpu_put_hw_mmu (struct kvm_vcpu *vcpu);extern void check_nested_vcpu_requests (struct kvm_vcpu *vcpu); +extern void kvm_nested_flush_hwstate (struct kvm_vcpu *vcpu); +extern void kvm_nested_sync_hwstate (struct kvm_vcpu *vcpu);struct kvm_s2_trans {phys_addr_t output;
arch/arm64/kvm/arm.c +4 -0
@@ -1188,6 +1188,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)*/ preempt_disable (); +kvm_nested_flush_hwstate (vcpu); +if (kvm_vcpu_has_pmu (vcpu))kvm_pmu_flush_hwstate (vcpu);
@@ -1287,6 +1289,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)/* Exit types that need handling before we can be preempted */ handle_exit_early (vcpu, ret); +kvm_nested_sync_hwstate (vcpu); +preempt_enable ();/*
arch/arm64/kvm/emulate-nested.c +14 -0
@@ -2714,6 +2714,9 @@ static void kvm_inject_el2_exception(struct kvm_vcpu *vcpu, u64 esr_el2,case except_type_irq:kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_IRQ );break ; +case except_type_serror: +kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SERR ); +break ;default :WARN_ONCE (1 ,"Unsupported EL2 exception injection %d\n" , type); }
@@ -2821,3 +2824,14 @@ int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr)vcpu_write_sys_reg (vcpu,FAR_EL2 , addr);return kvm_inject_nested_sync (vcpu, esr); } + +int kvm_inject_nested_serror (struct kvm_vcpu *vcpu,u64 esr) +{ +/* +* Hardware sets up the EC field when propagating ESR as a result of +* vSError injection. Manually populate EC for an emulated SError +* exception. +*/ + esr |=FIELD_PREP (ESR_ELx_EC_MASK, ESR_ELx_EC_SERROR); +return kvm_inject_nested (vcpu, esr, except_type_serror); +}
arch/arm64/kvm/guest.c +18 -14
@@ -818,8 +818,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,int __kvm_arm_vcpu_get_events (struct kvm_vcpu *vcpu,struct kvm_vcpu_events *events) { - events->exception.serror_pending = !!(vcpu->arch.hcr_el2 &HCR_VSE ); events->exception.serror_has_esr =cpus_have_final_cap (ARM64_HAS_RAS_EXTN ); + events->exception.serror_pending = (vcpu->arch.hcr_el2 &HCR_VSE ) || +vcpu_get_flag (vcpu,NESTED_SERROR_PENDING );if (events->exception.serror_pending && events->exception.serror_has_esr) events->exception.serror_esr =vcpu_get_vsesr (vcpu);
@@ -839,24 +840,27 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,bool serror_pending = events->exception.serror_pending;bool has_esr = events->exception.serror_has_esr;bool ext_dabt_pending = events->exception.ext_dabt_pending; +u64 esr = events->exception.serror_esr;int ret =0 ; -if (serror_pending && has_esr) { -if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) -return -EINVAL ; - -if (!((events->exception.serror_esr) & ~ESR_ELx_ISS_MASK)) -kvm_set_sei_esr (vcpu, events->exception.serror_esr); -else -return -EINVAL ; - }else if (serror_pending) { -kvm_inject_vabt (vcpu); - } -if (ext_dabt_pending) ret =kvm_inject_sea_dabt (vcpu,kvm_vcpu_get_hfar (vcpu)); -return ret; +if (ret || !serror_pending) +return ret; + +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && has_esr) +return -EINVAL ; + +if (has_esr && (esr & ~ESR_ELx_ISS_MASK)) +return -EINVAL ; + +if (has_esr) + ret =kvm_inject_serror_esr (vcpu, esr); +else + ret =kvm_inject_serror (vcpu); + +return (ret <0 ) ? ret :0 ; } u32 __attribute_const__ kvm_target_cpu(void)
arch/arm64/kvm/handle_exit.c +2 -2
@@ -32,7 +32,7 @@ typedef int (*exit_handle_fn)(struct kvm_vcpu *);static void kvm_handle_guest_serror (struct kvm_vcpu *vcpu,u64 esr) {if (!arm64_is_ras_serror (esr) ||arm64_is_fatal_ras_serror (NULL , esr)) -kvm_inject_vabt (vcpu); +kvm_inject_serror (vcpu); }static int handle_hvc (struct kvm_vcpu *vcpu)
@@ -490,7 +490,7 @@ void handle_exit_early(struct kvm_vcpu *vcpu, int exception_index)kvm_handle_guest_serror (vcpu,disr_to_esr (disr)); }else { -kvm_inject_vabt (vcpu); +kvm_inject_serror (vcpu); }return ;
arch/arm64/kvm/hyp/exception.c +5 -1
@@ -347,9 +347,13 @@ static void kvm_inject_exception(struct kvm_vcpu *vcpu)enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_irq);break ; +case unpack_vcpu_flag (EXCEPT_AA64_EL2_SERR ): +enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_serror); +break ; +default :/* -* Only EL1_SYNC and EL2_{SYNC,IRQ} makes +* Only EL1_SYNC and EL2_{SYNC,IRQ,SERR} makes * sense so far. Everything else gets silently * ignored. */
arch/arm64/kvm/inject_fault.c +22 -17
@@ -219,25 +219,30 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)inject_undef64 (vcpu); } -void kvm_set_sei_esr (struct kvm_vcpu *vcpu,u64 esr) +static bool kvm_serror_target_is_el2 (struct kvm_vcpu *vcpu) { -vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); - *vcpu_hcr (vcpu) |=HCR_VSE ; +return is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu); } -/** -* kvm_inject_vabt - inject an async abort / SError into the guest -* @vcpu: The VCPU to receive the exception -* -* It is assumed that this code is called from the VCPU thread and that the -* VCPU therefore is not currently executing guest code. -* -* Systems with the RAS Extensions specify an imp-def ESR (ISV/IDS = 1) with -* the remaining ISS all-zeros so that this error is not interpreted as an -* uncategorized RAS error. Without the RAS Extensions we can't specify an ESR -* value, so the CPU generates an imp-def value. -*/ -void kvm_inject_vabt (struct kvm_vcpu *vcpu) +static bool kvm_serror_undeliverable_at_el2 (struct kvm_vcpu *vcpu) { -kvm_set_sei_esr (vcpu, ESR_ELx_ISV); +return !(vcpu_el2_tge_is_set (vcpu) ||vcpu_el2_amo_is_set (vcpu)); +} + +int kvm_inject_serror_esr (struct kvm_vcpu *vcpu,u64 esr) +{ +lockdep_assert_held (&vcpu->mutex); + +if (is_nested_ctxt (vcpu) &&kvm_serror_target_is_el2 (vcpu)) +return kvm_inject_nested_serror (vcpu, esr); + +if (vcpu_is_el2 (vcpu) &&kvm_serror_undeliverable_at_el2 (vcpu)) { +vcpu_set_vsesr (vcpu, esr); +vcpu_set_flag (vcpu,NESTED_SERROR_PENDING ); +return 1 ; + } + +vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); + *vcpu_hcr (vcpu) |=HCR_VSE ; +return 1 ; }
arch/arm64/kvm/mmu.c +1 -1
@@ -1808,7 +1808,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)* There is no need to pass the error into the guest. */ if (kvm_handle_guest_sea ()) -kvm_inject_vabt (vcpu); +return kvm_inject_serror (vcpu);return 1 ; }
arch/arm64/kvm/nested.c +40 -0
@@ -1782,3 +1782,43 @@ void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_GUEST_HYP_IRQ_PENDING , vcpu))kvm_inject_nested_irq (vcpu); } + +/* +* One of the many architectural bugs in FEAT_NV2 is that the guest hypervisor +* can write to HCR_EL2 behind our back, potentially changing the exception +* routing / masking for even the host context. +* +* What follows is some slop to (1) react to exception routing / masking and (2) +* preserve the pending SError state across translation regimes. +*/ +void kvm_nested_flush_hwstate (struct kvm_vcpu *vcpu) +{ +if (!vcpu_has_nv (vcpu)) +return ; + +if (unlikely (vcpu_test_and_clear_flag (vcpu,NESTED_SERROR_PENDING ))) +kvm_inject_serror_esr (vcpu,vcpu_get_vsesr (vcpu)); +} + +void kvm_nested_sync_hwstate (struct kvm_vcpu *vcpu) +{ +unsigned long *hcr =vcpu_hcr (vcpu); + +if (!vcpu_has_nv (vcpu)) +return ; + +/* +* We previously decided that an SError was deliverable to the guest. +* Reap the pending state from HCR_EL2 and... +*/ +if (unlikely (__test_and_clear_bit (__ffs (HCR_VSE ), hcr))) +vcpu_set_flag (vcpu,NESTED_SERROR_PENDING ); + +/* +* Re-attempt SError injection in case the deliverability has changed, +* which is necessary to faithfully emulate WFI the case of a pending +* SError being a wakeup condition. +*/ +if (unlikely (vcpu_test_and_clear_flag (vcpu,NESTED_SERROR_PENDING ))) +kvm_inject_serror_esr (vcpu,vcpu_get_vsesr (vcpu)); +}
Signature · 1 line
2.39.5
PATCH v3 7/27 KVM: arm64: nv: Add FEAT_RAS vSError sys regs to table
In reply to Oliver UptonPrepare to implement RAS for NV by adding the missing EL2 sysregs for the vSError context.
--- arch/arm64/include/asm/kvm_host.h | 4 ++++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 2 ++ 3 files changed, 7 insertions(+)
arch/arm64/include/asm/kvm_host.h +4 -0
@@ -565,6 +565,10 @@ enum vcpu_sysreg {VNCR (POR_EL1 ),/* Permission Overlay Register 1 (EL1) */ +/* FEAT_RAS registers */ +VNCR (VDISR_EL2 ), +VNCR (VSESR_EL2 ), +VNCR (HFGRTR_EL2 ),VNCR (HFGWTR_EL2 ), VNCR(HFGITR_EL2),
arch/arm64/include/asm/vncr_mapping.h +1 -0
@@ -84,6 +84,7 @@#define VNCR_ICH_HCR_EL2 0x4C0#define VNCR_ICH_VMCR_EL2 0x4C8#define VNCR_VDISR_EL2 0x500 +#define VNCR_VSESR_EL2 0x508#define VNCR_PMBLIMITR_EL1 0x800#define VNCR_PMBPTR_EL1 0x810#define VNCR_PMBSR_EL1 0x820
arch/arm64/kvm/sys_regs.c +2 -0
@@ -3344,6 +3344,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {EL2_REG (AFSR0_EL2 , access_rw, reset_val,0 ),EL2_REG (AFSR1_EL2 , access_rw, reset_val,0 ),EL2_REG_REDIR (ESR_EL2 , reset_val,0 ), +EL2_REG_VNCR (VSESR_EL2 , reset_unknown,0 ), { SYS_DESC(SYS_FPEXC32_EL2), undef_access, reset_val, FPEXC32_EL2, 0x700 }, EL2_REG_REDIR(FAR_EL2, reset_val, 0),
@@ -3372,6 +3373,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {EL2_REG (VBAR_EL2 , access_rw, reset_val,0 ),EL2_REG (RVBAR_EL2 , access_rw, reset_val,0 ), { SYS_DESC(SYS_RMR_EL2), undef_access }, +EL2_REG_VNCR (VDISR_EL2 , reset_unknown,0 ),EL2_REG_VNCR (ICH_AP0R0_EL2 , reset_val,0 ), EL2_REG_VNCR(ICH_AP0R1_EL2, reset_val, 0),
Signature · 1 line
2.39.5
PATCH v3 8/27 KVM: arm64: nv: Use guest hypervisor's vSError state
In reply to Oliver UptonWhen HCR_EL2.AMO is set, physical SErrors are routed to EL2 and virtual SError injection is enabled for EL1. Conceptually treating host-initiated SErrors as 'physical', this means we can delegate control of the vSError injection context to the guest hypervisor when nesting && AMO is set.
--- arch/arm64/include/asm/kvm_emulate.h | 5 +++ arch/arm64/include/asm/kvm_host.h | 3 ++ arch/arm64/kvm/hyp/include/hyp/switch.h | 45 +++++++++++++++++++--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 31 +++++++++++++-- 4 files changed, 76 insertions(+), 8 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +5 -0
@@ -257,6 +257,11 @@ static inline bool is_nested_ctxt(struct kvm_vcpu *vcpu)return vcpu_has_nv (vcpu) && !is_hyp_ctxt (vcpu); } +static inline bool vserror_state_is_nested (struct kvm_vcpu *vcpu) +{ +return is_nested_ctxt (vcpu) &&vcpu_el2_amo_is_set (vcpu); +} +/* * The layout of SPSR for an AArch32 state is different when observed from an * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
arch/arm64/include/asm/kvm_host.h +3 -0
@@ -1682,6 +1682,9 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val);#define kvm_has_s1poe (k) \ (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) +#define kvm_has_ras (k) \ + (kvm_has_feat((k), ID_AA64PFR0_EL1, RAS, IMP)) +static inline bool kvm_arch_has_irq_bypass (void ) {return true;
arch/arm64/kvm/hyp/include/hyp/switch.h +40 -5
61 lines not on this page · raw message
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +28 -3
@@ -109,6 +109,17 @@ static inline bool ctxt_has_s1poe(struct kvm_cpu_context *ctxt)return kvm_has_s1poe (kern_hyp_va (vcpu->kvm)); } +static inline bool ctxt_has_ras (struct kvm_cpu_context *ctxt) +{ +struct kvm_vcpu *vcpu; + +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +return false; + + vcpu =ctxt_to_vcpu (ctxt); +return kvm_has_ras (kern_hyp_va (vcpu->kvm)); +} +static inline void __sysreg_save_el1_state (struct kvm_cpu_context *ctxt) {ctxt_sys_reg (ctxt,SCTLR_EL1 ) =read_sysreg_el1 (SYS_SCTLR );
@@ -159,8 +170,13 @@ static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)if (!has_vhe () && ctxt->__hyp_running_vcpu) ctxt->regs.pstate =read_sysreg_el2 (SYS_SPSR ); -if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +return ; + +if (!vserror_state_is_nested (ctxt_to_vcpu (ctxt)))ctxt_sys_reg (ctxt,DISR_EL1 ) =read_sysreg_s (SYS_VDISR_EL2 ); +else if (ctxt_has_ras (ctxt)) +ctxt_sys_reg (ctxt,VDISR_EL2 ) =read_sysreg_s (SYS_VDISR_EL2 ); }static inline void __sysreg_restore_common_state (struct kvm_cpu_context *ctxt)
@@ -293,8 +309,17 @@ static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxwrite_sysreg_el2 (ctxt->regs.pc,SYS_ELR );write_sysreg_el2 (pstate,SYS_SPSR ); -if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) -write_sysreg_s (ctxt_sys_reg (ctxt,DISR_EL1 ),SYS_VDISR_EL2 ); +if (!cpus_have_final_cap (ARM64_HAS_RAS_EXTN )) +return ; + +if (!vserror_state_is_nested (ctxt_to_vcpu (ctxt))) + vdisr =ctxt_sys_reg (ctxt,DISR_EL1 ); +else if (ctxt_has_ras (ctxt)) + vdisr =ctxt_sys_reg (ctxt,VDISR_EL2 ); +else + vdisr =0 ; + +write_sysreg_s (vdisr,SYS_VDISR_EL2 ); }static inline void __sysreg32_save_state (struct kvm_vcpu *vcpu)
Signature · 1 line
2.39.5
Re: [PATCH v3 08/27] KVM: arm64: nv: Use guest hypervisor's vSError state
Quoted 8 lines · Oliver Upton
When HCR_EL2.AMO is set, physical SErrors are routed to EL2 and virtual SError injection is enabled for EL1. Conceptually treating host-initiated SErrors as 'physical', this means we can delegate control of the vSError injection context to the guest hypervisor when nesting && AMO is set.
Reviewed-by Marc Zyngier <[email protected]>Signed-off-by Oliver Upton <[email protected]>
Grr, I scrapped a fixup by accident for this patch. Squashing this into the series:
From 568c69139ced56362028b50fc4751295cea36985 Mon Sep 17 00:00:00 2001
From: Oliver Upton <[email protected]>
Date: Tue, 8 Jul 2025 10:37:16 -0700
Subject: [PATCH] fixup! KVM: arm64: nv: Use guest hypervisor's vSError state
--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 1 + 1 file changed, 1 insertion(+)
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +1 -0
@@ -308,6 +308,7 @@ static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctx {u64 pstate =to_hw_pstate (ctxt);u64 mode = pstate &PSR_AA32_MODE_MASK ; +u64 vdisr;/* * Safety check to ensure we're setting the CPU up to enter the guest
Signature · 1 line
2.39.5
PATCH v3 9/27 KVM: arm64: nv: Advertise support for FEAT_RAS
In reply to Oliver UptonNow that the missing bits for vSError injection/deferral have been added we can merrily claim support for FEAT_RAS.
--- arch/arm64/kvm/nested.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/arm64/kvm/nested.c +1 -2
@@ -1424,12 +1424,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)break ;case SYS_ID_AA64PFR0_EL1 : -/* No RME, AMU, MPAM, S-EL2, or RAS */ +/* No RME, AMU, MPAM, or S-EL2 */ val &= ~(ID_AA64PFR0_EL1_RME | ID_AA64PFR0_EL1_AMU | ID_AA64PFR0_EL1_MPAM | ID_AA64PFR0_EL1_SEL2 | - ID_AA64PFR0_EL1_RAS | ID_AA64PFR0_EL1_EL3 | ID_AA64PFR0_EL1_EL2 | ID_AA64PFR0_EL1_EL1 |
Signature · 1 line
2.39.5
PATCH v3 10/27 KVM: arm64: nv: Describe trap behavior of SCTLR2_EL1
In reply to Oliver UptonAdd the complete trap description for SCTLR2_EL1, including FGT and the inverted HCRX bit.
--- arch/arm64/kvm/emulate-nested.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/arm64/kvm/emulate-nested.c +12 -0
@@ -88,6 +88,7 @@ enum cgt_group_id { CGT_HCRX_EnFPM, CGT_HCRX_TCR2En, + CGT_HCRX_SCTLR2En,CGT_CNTHCTL_EL1TVT , CGT_CNTHCTL_EL1TVCT,
@@ -108,6 +109,7 @@ enum cgt_group_id {CGT_HCR_TTLB_TTLBOS ,CGT_HCR_TVM_TRVM , CGT_HCR_TVM_TRVM_HCRX_TCR2En, + CGT_HCR_TVM_TRVM_HCRX_SCTLR2En,CGT_HCR_TPU_TICAB ,CGT_HCR_TPU_TOCU , CGT_HCR_NV1_nNV2_ENSCXT,
@@ -398,6 +400,12 @@ static const struct trap_bits coarse_trap_bits[] = { .mask = HCRX_EL2_TCR2En, .behaviour = BEHAVE_FORWARD_RW, }, + [CGT_HCRX_SCTLR2En] = { + .index = HCRX_EL2, + .value = 0, + .mask = HCRX_EL2_SCTLR2En, + .behaviour = BEHAVE_FORWARD_RW, + }, [CGT_CNTHCTL_EL1TVT] = { .index = CNTHCTL_EL2, .value = CNTHCTL_EL1TVT,
@@ -449,6 +457,8 @@ static const enum cgt_group_id *coarse_control_combo[] = {MCB (CGT_HCR_TVM_TRVM ,CGT_HCR_TVM ,CGT_HCR_TRVM ),MCB (CGT_HCR_TVM_TRVM_HCRX_TCR2En,CGT_HCR_TVM ,CGT_HCR_TRVM , CGT_HCRX_TCR2En), +MCB (CGT_HCR_TVM_TRVM_HCRX_SCTLR2En, +CGT_HCR_TVM ,CGT_HCR_TRVM , CGT_HCRX_SCTLR2En),MCB (CGT_HCR_TPU_TICAB ,CGT_HCR_TPU ,CGT_HCR_TICAB ),MCB (CGT_HCR_TPU_TOCU ,CGT_HCR_TPU ,CGT_HCR_TOCU ), MCB(CGT_HCR_NV1_nNV2_ENSCXT, CGT_HCR_NV1_nNV2, CGT_HCR_ENSCXT),
@@ -782,6 +792,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = {SR_TRAP (OP_TLBI_RVALE1OSNXS ,CGT_HCR_TTLB_TTLBOS ),SR_TRAP (OP_TLBI_RVAALE1OSNXS ,CGT_HCR_TTLB_TTLBOS ),SR_TRAP (SYS_SCTLR_EL1 ,CGT_HCR_TVM_TRVM ), +SR_TRAP (SYS_SCTLR2_EL1 , CGT_HCR_TVM_TRVM_HCRX_SCTLR2En),SR_TRAP (SYS_TTBR0_EL1 ,CGT_HCR_TVM_TRVM ),SR_TRAP (SYS_TTBR1_EL1 ,CGT_HCR_TVM_TRVM ), SR_TRAP(SYS_TCR_EL1, CGT_HCR_TVM_TRVM),
@@ -1354,6 +1365,7 @@ static const struct encoding_to_trap_config encoding_to_fgt[] __initconst = {SR_FGT (SYS_SCXTNUM_EL0 ,HFGRTR ,SCXTNUM_EL0 ,1 ),SR_FGT (SYS_SCXTNUM_EL1 ,HFGRTR ,SCXTNUM_EL1 ,1 ),SR_FGT (SYS_SCTLR_EL1 ,HFGRTR ,SCTLR_EL1 ,1 ), +SR_FGT (SYS_SCTLR2_EL1 ,HFGRTR ,SCTLR_EL1 ,1 ),SR_FGT (SYS_REVIDR_EL1 ,HFGRTR ,REVIDR_EL1 ,1 ),SR_FGT (SYS_PAR_EL1 ,HFGRTR ,PAR_EL1 ,1 ), SR_FGT(SYS_MPIDR_EL1, HFGRTR, MPIDR_EL1, 1),
Signature · 1 line
2.39.5
PATCH v3 11/27 KVM: arm64: Wire up SCTLR2_ELx sysreg descriptors
In reply to Oliver UptonSet up the sysreg descriptors for SCTLR2_ELx, along with the associated storage and VNCR mapping.
--- arch/arm64/include/asm/kvm_host.h | 7 +++++++ arch/arm64/include/asm/vncr_mapping.h | 1 + arch/arm64/kvm/sys_regs.c | 20 ++++++++++++++++++++ 3 files changed, 28 insertions(+)
arch/arm64/include/asm/kvm_host.h +7 -0
@@ -523,6 +523,7 @@ enum vcpu_sysreg {/* Anything from this can be RES0/RES1 sanitised */ MARKER (__SANITISED_REG_START__),TCR2_EL2 ,/* Extended Translation Control Register (EL2) */ +SCTLR2_EL2 ,/* System Control Register 2 (EL2) */ MDCR_EL2 ,/* Monitor Debug Configuration Register (EL2) */ CNTHCTL_EL2, /* Counter-timer Hypervisor Control register */
@@ -537,6 +538,7 @@ enum vcpu_sysreg {VNCR (TTBR1_EL1 ),/* Translation Table Base Register 1 */ VNCR (TCR_EL1 ),/* Translation Control Register */ VNCR (TCR2_EL1 ),/* Extended Translation Control Register */ +VNCR (SCTLR2_EL1 ),/* System Control Register 2 */ VNCR (ESR_EL1 ),/* Exception Syndrome Register */ VNCR (AFSR0_EL1 ),/* Auxiliary Fault Status Register 0 */ VNCR(AFSR1_EL1),/* Auxiliary Fault Status Register 1 */
@@ -1204,6 +1206,7 @@ static inline bool __vcpu_read_sys_reg_from_cpu(int reg, u64 *val)case IFSR32_EL2 : *val =read_sysreg_s (SYS_IFSR32_EL2 );break ;case DBGVCR32_EL2 : *val =read_sysreg_s (SYS_DBGVCR32_EL2 );break ;case ZCR_EL1 : *val =read_sysreg_s (SYS_ZCR_EL12 );break ; +case SCTLR2_EL1 : *val =read_sysreg_s (SYS_SCTLR2_EL12 );break ;default :return false; }
@@ -1254,6 +1257,7 @@ static inline bool __vcpu_write_sys_reg_to_cpu(u64 val, int reg)case IFSR32_EL2 :write_sysreg_s (val,SYS_IFSR32_EL2 );break ;case DBGVCR32_EL2 :write_sysreg_s (val,SYS_DBGVCR32_EL2 );break ;case ZCR_EL1 :write_sysreg_s (val,SYS_ZCR_EL12 );break ; +case SCTLR2_EL1 :write_sysreg_s (val,SYS_SCTLR2_EL12 );break ;default :return false; }
@@ -1685,6 +1689,9 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val);#define kvm_has_ras (k) \ (kvm_has_feat((k), ID_AA64PFR0_EL1, RAS, IMP)) +#define kvm_has_sctlr2 (k) \ + (kvm_has_feat((k), ID_AA64MMFR3_EL1, SCTLRX, IMP)) +static inline bool kvm_arch_has_irq_bypass (void ) {return true;
arch/arm64/include/asm/vncr_mapping.h +1 -0
@@ -51,6 +51,7 @@#define VNCR_SP_EL1 0x240#define VNCR_VBAR_EL1 0x250#define VNCR_TCR2_EL1 0x270 +#define VNCR_SCTLR2_EL1 0x278#define VNCR_PIRE0_EL1 0x290#define VNCR_PIR_EL1 0x2A0#define VNCR_POR_EL1 0x2A8
arch/arm64/kvm/sys_regs.c +20 -0
@@ -144,6 +144,7 @@ static bool get_el2_to_el1_mapping(unsigned int reg,MAPPED_EL2_SYSREG (SPSR_EL2 ,SPSR_EL1 ,NULL );MAPPED_EL2_SYSREG (ZCR_EL2 ,ZCR_EL1 ,NULL );MAPPED_EL2_SYSREG (CONTEXTIDR_EL2 ,CONTEXTIDR_EL1 ,NULL ); +MAPPED_EL2_SYSREG (SCTLR2_EL2 ,SCTLR2_EL1 ,NULL );default :return false; }
@@ -2483,6 +2484,21 @@ static unsigned int vncr_el2_visibility(const struct kvm_vcpu *vcpu,return REG_HIDDEN ; } +static unsigned int sctlr2_visibility (const struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +if (kvm_has_sctlr2 (vcpu->kvm)) +return 0 ; + +return REG_HIDDEN ; +} + +static unsigned int sctlr2_el2_visibility (const struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +return __el2_visibility (vcpu, rd, sctlr2_visibility); +} +static bool access_zcr_el2 (struct kvm_vcpu *vcpu,struct sys_reg_params *p,const struct sys_reg_desc *r)
@@ -2955,6 +2971,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 }, { SYS_DESC(SYS_ACTLR_EL1), access_actlr, reset_actlr, ACTLR_EL1 }, { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 }, + { SYS_DESC(SYS_SCTLR2_EL1), access_vm_reg, reset_val, SCTLR2_EL1, 0, + .visibility = sctlr2_visibility },MTE_REG (RGSR_EL1 ), MTE_REG(GCR_EL1),
@@ -3302,6 +3320,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {EL2_REG_VNCR (VMPIDR_EL2 , reset_unknown,0 ),EL2_REG (SCTLR_EL2 , access_rw, reset_val,SCTLR_EL2_RES1 ),EL2_REG (ACTLR_EL2 , access_rw, reset_val,0 ), +EL2_REG_FILTERED (SCTLR2_EL2 , access_vm_reg, reset_val,0 , + sctlr2_el2_visibility),EL2_REG_VNCR (HCR_EL2 , reset_hcr,0 ),EL2_REG (MDCR_EL2 , access_mdcr, reset_mdcr,0 ), EL2_REG(CPTR_EL2, access_rw, reset_val, CPTR_NVHE_EL2_RES1),
Signature · 1 line
2.39.5
PATCH v3 12/27 KVM: arm64: Context switch SCTLR2_ELx when advertised to the guest
In reply to Oliver UptonRestore SCTLR2_EL1 with the correct value for the given context when FEAT_SCTLR2 is advertised to the guest.
--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 17 +++++++++++++++++ arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 6 ++++++ 2 files changed, 23 insertions(+)
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +17 -0
@@ -120,6 +120,17 @@ static inline bool ctxt_has_ras(struct kvm_cpu_context *ctxt)return kvm_has_ras (kern_hyp_va (vcpu->kvm)); } +static inline bool ctxt_has_sctlr2 (struct kvm_cpu_context *ctxt) +{ +struct kvm_vcpu *vcpu; + +if (!cpus_have_final_cap (ARM64_HAS_SCTLR2 )) +return false; + + vcpu =ctxt_to_vcpu (ctxt); +return kvm_has_sctlr2 (kern_hyp_va (vcpu->kvm)); +} +static inline void __sysreg_save_el1_state (struct kvm_cpu_context *ctxt) {ctxt_sys_reg (ctxt,SCTLR_EL1 ) =read_sysreg_el1 (SYS_SCTLR );
@@ -158,6 +169,9 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)ctxt_sys_reg (ctxt,SP_EL1 ) =read_sysreg (sp_el1);ctxt_sys_reg (ctxt,ELR_EL1 ) =read_sysreg_el1 (SYS_ELR );ctxt_sys_reg (ctxt,SPSR_EL1 ) =read_sysreg_el1 (SYS_SPSR ); + +if (ctxt_has_sctlr2 (ctxt)) +ctxt_sys_reg (ctxt,SCTLR2_EL1 ) =read_sysreg_el1 (SYS_SCTLR2 ); }static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt)
@@ -268,6 +282,9 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,write_sysreg (ctxt_sys_reg (ctxt,SP_EL1 ), sp_el1);write_sysreg_el1 (ctxt_sys_reg (ctxt,ELR_EL1 ),SYS_ELR );write_sysreg_el1 (ctxt_sys_reg (ctxt,SPSR_EL1 ),SYS_SPSR ); + +if (ctxt_has_sctlr2 (ctxt)) +write_sysreg_el1 (ctxt_sys_reg (ctxt,SCTLR2_EL1 ),SYS_SCTLR2 ); }/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +6 -0
@@ -77,6 +77,9 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)__vcpu_assign_sys_reg (vcpu,SP_EL2 ,read_sysreg (sp_el1));__vcpu_assign_sys_reg (vcpu,ELR_EL2 ,read_sysreg_el1 (SYS_ELR ));__vcpu_assign_sys_reg (vcpu,SPSR_EL2 ,read_sysreg_el1 (SYS_SPSR )); + +if (ctxt_has_sctlr2 (&vcpu->arch.ctxt)) +__vcpu_assign_sys_reg (vcpu,SCTLR2_EL2 ,read_sysreg_el1 (SYS_SCTLR2 )); }static void __sysreg_restore_vel2_state (struct kvm_vcpu *vcpu)
@@ -139,6 +142,9 @@ static void __sysreg_restore_vel2_state(struct kvm_vcpu *vcpu)write_sysreg (__vcpu_sys_reg (vcpu,SP_EL2 ), sp_el1);write_sysreg_el1 (__vcpu_sys_reg (vcpu,ELR_EL2 ),SYS_ELR );write_sysreg_el1 (__vcpu_sys_reg (vcpu,SPSR_EL2 ),SYS_SPSR ); + +if (ctxt_has_sctlr2 (&vcpu->arch.ctxt)) +write_sysreg_el1 (__vcpu_sys_reg (vcpu,SCTLR2_EL2 ),SYS_SCTLR2 ); }/*
Signature · 1 line
2.39.5
PATCH v3 13/27 KVM: arm64: Enable SCTLR2 when advertised to the guest
In reply to Oliver UptonHCRX_EL2.SCTLR2En needs to be set for SCTLR2_EL1 to take effect in hardware (in addition to disabling traps).
--- arch/arm64/include/asm/kvm_emulate.h | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +3 -0
@@ -665,6 +665,9 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)if (kvm_has_fpmr (kvm)) vcpu->arch.hcrx_el2 |= HCRX_EL2_EnFPM; + +if (kvm_has_sctlr2 (kvm)) + vcpu->arch.hcrx_el2 |= HCRX_EL2_SCTLR2En; } }#endif /* __ARM64_KVM_EMULATE_H__ */
Signature · 1 line
2.39.5
PATCH v3 14/27 KVM: arm64: Describe SCTLR2_ELx RESx masks
In reply to Oliver UptonExternal abort injection will soon rely on a sanitised view of SCTLR2_ELx to determine exception routing. Compute the RESx masks.
--- arch/arm64/kvm/config.c | 28 ++++++++++++++++++++++++++++ arch/arm64/kvm/nested.c | 6 ++++++ 2 files changed, 34 insertions(+)
arch/arm64/kvm/config.c +28 -0
@@ -131,6 +131,8 @@ struct reg_bits_to_feat_map {#define FEAT_SPMU ID_AA64DFR1_EL1, SPMU, IMP#define FEAT_SPE_nVM ID_AA64DFR2_EL1, SPE_nVM, IMP#define FEAT_STEP2 ID_AA64DFR2_EL1, STEP, IMP +#define FEAT_SYSREG128 ID_AA64ISAR2_EL1, SYSREG_128, IMP +#define FEAT_CPA2 ID_AA64ISAR3_EL1, CPA, CPA2static bool not_feat_aa64el3 (struct kvm *kvm) {
@@ -832,6 +834,23 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = { NEEDS_FEAT_FIXED(HCR_EL2_E2H, compute_hcr_e2h), }; +static const struct reg_bits_to_feat_map sctlr2_feat_map[] = { +NEEDS_FEAT (SCTLR2_EL1_NMEA | +SCTLR2_EL1_EASE , + FEAT_DoubleFault2), +NEEDS_FEAT (SCTLR2_EL1_EnADERR, feat_aderr), +NEEDS_FEAT (SCTLR2_EL1_EnANERR, feat_anerr), +NEEDS_FEAT (SCTLR2_EL1_EnIDCP128,FEAT_SYSREG128 ), +NEEDS_FEAT (SCTLR2_EL1_EnPACM | + SCTLR2_EL1_EnPACM0, + feat_pauth_lr), +NEEDS_FEAT (SCTLR2_EL1_CPTA | +SCTLR2_EL1_CPTA0 | +SCTLR2_EL1_CPTM | +SCTLR2_EL1_CPTM0 , +FEAT_CPA2 ), +}; +static void __init check_feat_map (const struct reg_bits_to_feat_map *map,int map_size,u64 res0,const char *str) {
@@ -863,6 +882,8 @@ void __init check_feature_map(void) __HCRX_EL2_RES0, "HCRX_EL2");check_feat_map (hcr_feat_map,ARRAY_SIZE (hcr_feat_map),HCR_EL2_RES0 ,"HCR_EL2" ); +check_feat_map (sctlr2_feat_map,ARRAY_SIZE (sctlr2_feat_map), +SCTLR2_EL1_RES0 ,"SCTLR2_EL1" ); }static bool idreg_feat_match (struct kvm *kvm,const struct reg_bits_to_feat_map *map)
@@ -1077,6 +1098,13 @@ void get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg, u64 *res0, u64 *r *res0 |=HCR_EL2_RES0 | (mask & ~fixed); *res1 =HCR_EL2_RES1 | (mask & fixed);break ; +case SCTLR2_EL1 : +case SCTLR2_EL2 : + *res0 =compute_res0_bits (kvm, sctlr2_feat_map, +ARRAY_SIZE (sctlr2_feat_map),0 ,0 ); + *res0 |=SCTLR2_EL1_RES0 ; + *res1 =SCTLR2_EL1_RES1 ; +break ;default :WARN_ON_ONCE (1 ); *res0 = *res1 =0 ;
arch/arm64/kvm/nested.c +6 -0
@@ -1690,6 +1690,12 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) res0 |=SCTLR_EL1_EPAN ;set_sysreg_masks (kvm,SCTLR_EL1 , res0, res1); +/* SCTLR2_ELx */ +get_reg_fixed_bits (kvm,SCTLR2_EL1 , &res0, &res1); +set_sysreg_masks (kvm,SCTLR2_EL1 , res0, res1); +get_reg_fixed_bits (kvm,SCTLR2_EL2 , &res0, &res1); +set_sysreg_masks (kvm,SCTLR2_EL2 , res0, res1); +/* MDCR_EL2 */ res0 =MDCR_EL2_RES0 ; res1 =MDCR_EL2_RES1 ;
Signature · 1 line
2.39.5
PATCH v3 15/27 KVM: arm64: Factor out helper for selecting exception target EL
In reply to Oliver UptonPull out the exception target selection from pend_sync_exception() for general use. Use PSR_MODE_ELxh as a shorthand for the target EL, as SP_ELx selection is handled further along in the hyp's exception emulation.
--- arch/arm64/kvm/inject_fault.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-)
arch/arm64/kvm/inject_fault.c +14 -14
@@ -15,13 +15,11 @@#include <asm/kvm_nested.h> #include <asm/esr.h> -static void pend_sync_exception (struct kvm_vcpu *vcpu) +static unsigned int exception_target_el (struct kvm_vcpu *vcpu) {/* If not nesting, EL1 is the only possible exception target */ -if (likely (!vcpu_has_nv (vcpu))) { -kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); -return ; - } +if (likely (!vcpu_has_nv (vcpu))) +return PSR_MODE_EL1h;/* * With NV, we need to pick between EL1 and EL2. Note that we
@@ -32,23 +30,25 @@ static void pend_sync_exception(struct kvm_vcpu *vcpu)switch (*vcpu_cpsr (vcpu) &PSR_MODE_MASK ) {case PSR_MODE_EL2h:case PSR_MODE_EL2t: -kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); -break ; +return PSR_MODE_EL2h;case PSR_MODE_EL1h:case PSR_MODE_EL1t: -kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); -break ; +return PSR_MODE_EL1h;case PSR_MODE_EL0t: -if (vcpu_el2_tge_is_set (vcpu)) -kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); -else -kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); -break ; +return vcpu_el2_tge_is_set (vcpu) ? PSR_MODE_EL2h : PSR_MODE_EL1h;default :BUG (); } } +static void pend_sync_exception (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL1h) +kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SYNC ); +else +kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); +} +static bool match_target_el (struct kvm_vcpu *vcpu,unsigned long target) {return (vcpu_get_flag (vcpu,EXCEPT_MASK ) == target);
Signature · 1 line
2.39.5
PATCH v3 16/27 KVM: arm64: nv: Ensure Address size faults affect correct ESR
In reply to Oliver UptonFor historical reasons, Address size faults are first injected into the guest as an SEA and ESR_EL1 is subsequently modified to reflect the correct FSC. Of course, when dealing with a vEL2 this should poke ESR_EL2.
--- arch/arm64/kvm/inject_fault.c | 39 +++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 18 deletions(-)
arch/arm64/kvm/inject_fault.c +21 -18
@@ -41,6 +41,22 @@ static unsigned int exception_target_el(struct kvm_vcpu *vcpu) } } +static enum vcpu_sysreg exception_esr_elx (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL2h) +return ESR_EL2 ; + +return ESR_EL1 ; +} + +static enum vcpu_sysreg exception_far_elx (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL2h) +return FAR_EL2 ; + +return FAR_EL1 ; +} +static void pend_sync_exception (struct kvm_vcpu *vcpu) {if (exception_target_el (vcpu) == PSR_MODE_EL1h)
@@ -49,11 +65,6 @@ static void pend_sync_exception(struct kvm_vcpu *vcpu)kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); } -static bool match_target_el (struct kvm_vcpu *vcpu,unsigned long target) -{ -return (vcpu_get_flag (vcpu,EXCEPT_MASK ) == target); -} -static void inject_abt64 (struct kvm_vcpu *vcpu,bool is_iabt,unsigned long addr) {unsigned long cpsr = *vcpu_cpsr (vcpu);
@@ -83,13 +94,8 @@ static void inject_abt64(struct kvm_vcpu *vcpu, bool is_iabt, unsigned long addr esr |= ESR_ELx_FSC_EXTABT; -if (match_target_el (vcpu,unpack_vcpu_flag (EXCEPT_AA64_EL1_SYNC ))) { -vcpu_write_sys_reg (vcpu, addr,FAR_EL1 ); -vcpu_write_sys_reg (vcpu, esr,ESR_EL1 ); - }else { -vcpu_write_sys_reg (vcpu, addr,FAR_EL2 ); -vcpu_write_sys_reg (vcpu, esr,ESR_EL2 ); - } +vcpu_write_sys_reg (vcpu, addr,exception_far_elx (vcpu)); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); }static void inject_undef64 (struct kvm_vcpu *vcpu)
@@ -105,10 +111,7 @@ static void inject_undef64(struct kvm_vcpu *vcpu)if (kvm_vcpu_trap_il_is32bit (vcpu)) esr |= ESR_ELx_IL; -if (match_target_el (vcpu,unpack_vcpu_flag (EXCEPT_AA64_EL1_SYNC ))) -vcpu_write_sys_reg (vcpu, esr,ESR_EL1 ); -else -vcpu_write_sys_reg (vcpu, esr,ESR_EL2 ); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); }#define DFSR_FSC_EXTABT_LPAE 0x10
@@ -199,9 +202,9 @@ void kvm_inject_size_fault(struct kvm_vcpu *vcpu) !(vcpu_read_sys_reg(vcpu, TCR_EL1) & TTBCR_EAE)) return; - esr =vcpu_read_sys_reg (vcpu,ESR_EL1 ); + esr =vcpu_read_sys_reg (vcpu,exception_esr_elx (vcpu)); esr &= ~GENMASK_ULL (5 ,0 ); -vcpu_write_sys_reg (vcpu, esr,ESR_EL1 ); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); }/**
Signature · 1 line
2.39.5
PATCH v3 17/27 KVM: arm64: Route SEAs to the SError vector when EASE is set
In reply to Oliver UptonOne of the finest additions of FEAT_DoubleFault2 is the ability for software to request *synchronous* external aborts be taken to the SError vector, which of coure are *asynchronous* in nature.
Opinions be damned, implement the architecture and send SEAs to the SError vector if EASE is set for the target context.
--- arch/arm64/kvm/emulate-nested.c | 4 ++++ arch/arm64/kvm/hyp/exception.c | 6 +++++- arch/arm64/kvm/inject_fault.c | 38 ++++++++++++++++++++++++++++++++- 3 files changed, 46 insertions(+), 2 deletions(-)
arch/arm64/kvm/emulate-nested.c +4 -0
@@ -2834,6 +2834,10 @@ int kvm_inject_nested_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr) esr |= ESR_ELx_FSC_EXTABT | ESR_ELx_IL;vcpu_write_sys_reg (vcpu,FAR_EL2 , addr); + +if (__vcpu_sys_reg (vcpu,SCTLR2_EL2 ) &SCTLR2_EL1_EASE ) +return kvm_inject_nested (vcpu, esr, except_type_serror); +return kvm_inject_nested_sync (vcpu, esr); }
arch/arm64/kvm/hyp/exception.c +5 -1
@@ -339,6 +339,10 @@ static void kvm_inject_exception(struct kvm_vcpu *vcpu)enter_exception64 (vcpu, PSR_MODE_EL1h, except_type_sync);break ; +case unpack_vcpu_flag (EXCEPT_AA64_EL1_SERR ): +enter_exception64 (vcpu, PSR_MODE_EL1h, except_type_serror); +break ; +case unpack_vcpu_flag (EXCEPT_AA64_EL2_SYNC ):enter_exception64 (vcpu, PSR_MODE_EL2h, except_type_sync);break ;
@@ -353,7 +357,7 @@ static void kvm_inject_exception(struct kvm_vcpu *vcpu)default :/* -* Only EL1_SYNC and EL2_{SYNC,IRQ,SERR} makes +* Only EL1_{SYNC,SERR} and EL2_{SYNC,IRQ,SERR} makes * sense so far. Everything else gets silently * ignored. */
arch/arm64/kvm/inject_fault.c +37 -1
@@ -65,13 +65,49 @@ static void pend_sync_exception(struct kvm_vcpu *vcpu)kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SYNC ); } +static void pend_serror_exception (struct kvm_vcpu *vcpu) +{ +if (exception_target_el (vcpu) == PSR_MODE_EL1h) +kvm_pend_exception (vcpu,EXCEPT_AA64_EL1_SERR ); +else +kvm_pend_exception (vcpu,EXCEPT_AA64_EL2_SERR ); +} + +static bool __effective_sctlr2_bit (struct kvm_vcpu *vcpu,unsigned int idx) +{ +u64 sctlr2; + +if (!kvm_has_sctlr2 (vcpu->kvm)) +return false; + +if (is_nested_ctxt (vcpu) && + !(__vcpu_sys_reg (vcpu,HCRX_EL2 ) & HCRX_EL2_SCTLR2En)) +return false; + +if (exception_target_el (vcpu) == PSR_MODE_EL1h) + sctlr2 =vcpu_read_sys_reg (vcpu,SCTLR2_EL1 ); +else + sctlr2 =vcpu_read_sys_reg (vcpu,SCTLR2_EL2 ); + +return sctlr2 &BIT (idx); +} + +static bool effective_sctlr2_ease (struct kvm_vcpu *vcpu) +{ +return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_EASE_SHIFT ); +} +static void inject_abt64 (struct kvm_vcpu *vcpu,bool is_iabt,unsigned long addr) {unsigned long cpsr = *vcpu_cpsr (vcpu);bool is_aarch32 =vcpu_mode_is_32bit (vcpu);u64 esr =0 ; -pend_sync_exception (vcpu); +/* This delight is brought to you by FEAT_DoubleFault2. */ +if (effective_sctlr2_ease (vcpu)) +pend_serror_exception (vcpu); +else +pend_sync_exception (vcpu);/* * Build an {i,d}abort, depending on the level and the
Signature · 1 line
2.39.5
PATCH v3 18/27 KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set
In reply to Oliver UptonHCRX_EL2.TMEA further modifies the external abort behavior where unmasked aborts are taken to EL1 and masked aborts are taken to EL2. It's rather weird when you consider that SEAs are, well, *synchronous* and therefore not actually maskable. However, for the purposes of exception routing, they're considered "masked" if the A flag is set.
This gets a bit hairier when considering the fact that TMEA also enables vSErrors, i.e. KVM has delegated the HW vSError context to the guest hypervisor. We can keep the vSError context delegation as-is by taking advantage of a couple properties:
- If SErrors are unmasked, the 'physical' SError can be taken in-context immediately. In other words, KVM can emulate the EL1 SError while preserving vEL2's ownership of the vSError context.
- If SErrors are masked, the 'physical' SError is taken to EL2 immediately and needs the usual nested exception entry.
Note that the new in-context handling has the benign effect where unmasked SError injections are emulated even for non-nested VMs.
--- arch/arm64/kvm/inject_fault.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)
arch/arm64/kvm/inject_fault.c +32 -2
@@ -204,7 +204,14 @@ static void __kvm_inject_sea(struct kvm_vcpu *vcpu, bool iabt, u64 addr)static bool kvm_sea_target_is_el2 (struct kvm_vcpu *vcpu) { -return __vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_TGE |HCR_TEA ); +if (__vcpu_sys_reg (vcpu,HCR_EL2 ) & (HCR_TGE |HCR_TEA )) +return true; + +if (!vcpu_mode_priv (vcpu)) +return false; + +return (*vcpu_cpsr (vcpu) &PSR_A_BIT ) && + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }int kvm_inject_sea (struct kvm_vcpu *vcpu,bool iabt,u64 addr)
@@ -258,9 +265,20 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)inject_undef64 (vcpu); } +static bool serror_is_masked (struct kvm_vcpu *vcpu) +{ +return *vcpu_cpsr (vcpu) &PSR_A_BIT ; +} +static bool kvm_serror_target_is_el2 (struct kvm_vcpu *vcpu) { -return is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu); +if (is_hyp_ctxt (vcpu) ||vcpu_el2_amo_is_set (vcpu)) +return true; + +if (!(__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA )) +return false; + +return serror_is_masked (vcpu); }static bool kvm_serror_undeliverable_at_el2 (struct kvm_vcpu *vcpu)
@@ -281,6 +299,18 @@ int kvm_inject_serror_esr(struct kvm_vcpu *vcpu, u64 esr)return 1 ; } +/* +* Emulate the exception entry if SErrors are unmasked. This is useful if +* the vCPU is in a nested context w/ vSErrors enabled then we've already +* delegated he hardware vSError context (i.e. HCR_EL2.VSE, VSESR_EL2, +* VDISR_EL2) to the guest hypervisor. +*/ +if (!serror_is_masked (vcpu)) { +pend_serror_exception (vcpu); +vcpu_write_sys_reg (vcpu, esr,exception_esr_elx (vcpu)); +return 1 ; + } +vcpu_set_vsesr (vcpu, esr & ESR_ELx_ISS_MASK); *vcpu_hcr (vcpu) |=HCR_VSE ;return 1 ;
Signature · 1 line
2.39.5
Re: [PATCH v3 18/27] KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set
Quoted 5 lines · Oliver Upton
HCRX_EL2.TMEA further modifies the external abort behavior where unmasked aborts are taken to EL1 and masked aborts are taken to EL2. It's rather weird when you consider that SEAs are, well, *synchronous* and therefore not actually maskable. However, for the purposes of exception routing, they're considered "masked" if the A flag is set.
For the past few days the external_aborts KVM selftest has been failing in -next on a number of platforms with:
# selftests: kvm: external_aborts
# Random seed: 0x6b8b4567
# ==== Test Assertion Failure ====
# arm64/external_aborts.c:19: regs->pc == expected_abort_pc
# pid=2598 tid=2598 errno=4 - Interrupted system call
# 1 0x0000000000402f93: __vcpu_run_expect at external_aborts.c:85 # 2 0x0000000000402197: vcpu_run_expect_done at external_aborts.c:97 # 3 (inlined by) test_mmio_abort at external_aborts.c:136 # 4 (inlined by) main at external_aborts.c:323 # 5 0x0000ffffacbd7543: ?? ??:0 # 6 0x0000ffffacbd7617: ?? ??:0 # 7 0x000000000040272f: _start at ??:?
# 0x0 != 0x4028f8 (regs->pc != expected_abort_pc)
not ok 14 selftests: kvm: external_aborts # exit=254
This appears to be happening on many, possibly all, VHE platforms being tested - nVHE appears fine. I ran a bisect, fixing the selftests version at the one in -next due to the renaming of this test, which pointed at this commit.
Full log:
https://lava.sirena.org.uk/scheduler/job/1585350#L3418
bisect log:
git bisect start
# status: waiting for both good and bad commits
# bad: [d086c886ceb9f59dea6c3a9dae7eb89e780a20c9] Add linux-next specific files for 20250718
git bisect bad d086c886ceb9f59dea6c3a9dae7eb89e780a20c9
# status: waiting for good commit(s), bad commit known
# good: [04991f9d52f128b90c39e4e26c27a2bb03f82eba] Merge branch 'for-linux-next-fixes' of https://gitlab.freedesktop.org/drm/misc/kernel.git
git bisect good 04991f9d52f128b90c39e4e26c27a2bb03f82eba
# good: [04df9b7af77fe92aac33c006e8d48dfcdf12bd47] Merge branch 'main' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git
git bisect good 04df9b7af77fe92aac33c006e8d48dfcdf12bd47
# good: [ef3a80d9cf6739dcb358b47052470a474ca9202e] Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git
git bisect good ef3a80d9cf6739dcb358b47052470a474ca9202e
# bad: [a4ff72f2a847bbec65a94387fd64e9673fd3d997] Merge branch 'usb-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git
git bisect bad a4ff72f2a847bbec65a94387fd64e9673fd3d997
# good: [eb7455bf812f1ce0f91f3e6d64726d821bab0f15] Merge branch 'edac-for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras.git
git bisect good eb7455bf812f1ce0f91f3e6d64726d821bab0f15
# bad: [31a02958732db752412d656c35689a1bac43b86d] Merge branch 'next' of https://github.com/kvm-x86/linux.git
git bisect bad 31a02958732db752412d656c35689a1bac43b86d
# good: [c10f79ba55517270cb35953ac2f82a56cf9dbf74] Merge branch 'selftests'
git bisect good c10f79ba55517270cb35953ac2f82a56cf9dbf74
# good: [b7ba58c6c588667934ca3738f84b439ec0c4a075] Merge branch 'non-rcu/next' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
git bisect good b7ba58c6c588667934ca3738f84b439ec0c4a075
# bad: [a826c1169d283ed62a5eaa7872d763545d263b60] Merge remote-tracking branch 'kernel/maz/irq/gic-v5-host' into kvmarm/next git bisect bad a826c1169d283ed62a5eaa7872d763545d263b60
# bad: [da53a2014ebb996e07218d1491ac8e12ed770780] Merge branch 'kvm-arm64/doublefault2' into kvmarm/next
git bisect bad da53a2014ebb996e07218d1491ac8e12ed770780
# good: [fff97df2a0bd215979ae224b97e3e4075030a953] KVM: arm64: Route SEAs to the SError vector when EASE is set
git bisect good fff97df2a0bd215979ae224b97e3e4075030a953
# bad: [a90aac55324920de988cc447f4227c04ad769351] KVM: arm64: selftests: Test SEAs are taken to SError vector when EASE=1
git bisect bad a90aac55324920de988cc447f4227c04ad769351
# bad: [075c2dc7367e7e80d83adae8db597e48ceb7ba94] KVM: arm64: Advertise support for FEAT_SCTLR2
git bisect bad 075c2dc7367e7e80d83adae8db597e48ceb7ba94
# bad: [59b6d08666f0424f3e0ade08ab664fac65a09c5d] KVM: arm64: nv: Honor SError routing effects of SCTLR2_ELx.NMEA
git bisect bad 59b6d08666f0424f3e0ade08ab664fac65a09c5d
# bad: [ce66109cec867c9afd2ee1ecf1aff8d73cdb2f89] KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set
git bisect bad ce66109cec867c9afd2ee1ecf1aff8d73cdb2f89
# first bad commit: [ce66109cec867c9afd2ee1ecf1aff8d73cdb2f89] KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set
Marc Zyngier
maintainerRe: [PATCH v3 18/27] KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set
[Adding Zenghui to the list. In the future, please CC all reviewers
listed in MAINTAINERS, not just an arbitrary selection]
Quoted 23 lines · Mark Brown
Quoted 5 lines · Oliver Upton
HCRX_EL2.TMEA further modifies the external abort behavior where unmasked aborts are taken to EL1 and masked aborts are taken to EL2. It's rather weird when you consider that SEAs are, well, *synchronous* and therefore not actually maskable. However, for the purposes of exception routing, they're considered "masked" if the A flag is set.
For the past few days the
external_abortsKVM selftest has been failing in -next on a number of platforms with:# selftests: kvm:
external_aborts
# Random seed:0x6b8b4567
# ==== Test Assertion Failure ====
#arm64/external_aborts.c:19:regs->pc==expected_abort_pc
# pid=2598 tid=2598 errno=4 - Interrupted system call# 1 0x0000000000402f93: __vcpu_run_expect at external_aborts.c:85 # 2 0x0000000000402197: vcpu_run_expect_done at external_aborts.c:97 # 3 (inlined by) test_mmio_abort at external_aborts.c:136 # 4 (inlined by) main at external_aborts.c:323 # 5 0x0000ffffacbd7543: ?? ??:0 # 6 0x0000ffffacbd7617: ?? ??:0 # 7 0x000000000040272f: _start at ??:?
Quoted 7 lines
#
0x0!=0x4028f8(regs->pc!=expected_abort_pc)
not ok 14 selftests: kvm:external_aborts# exit=254This appears to be happening on many, possibly all, VHE platforms being tested - nVHE appears fine. I ran a bisect, fixing the selftests version at the one in -next due to the renaming of this test, which pointed at this commit.
Thanks for reporting this.
It turns out that the exception entry code fully expects everything to be loaded on the CPU when processing the exception. However, this is no longer true when thins are injected from userspace. But really, this is a pretty fragile expectation, and this absolutely needs fixing.
I've posted a potential fix at [1], but it appears that this series as further issues. If I run this very test in a nested guest (patch applied on both L0 and L1), I get this:
bash-5.2# /host/home/maz/external_aborts
Random seed: 0x6b8b4567
[ 5.936631] PC = 402764 [ 5.942221] PC = 4027f4 [ 5.961351] SError Interrupt on CPU1, code 0x00000000be000000 -- SError [ 5.961355] CPU: 1 UID: 0 PID: 64 Comm: external_aborts Not tainted 6.16.0-rc6-00163-ga03b7055c54b-dirty #4690 PREEMPT [ 5.961357] Hardware name: linux,dummy-virt (DT) [ 5.961358] pstate: 604000c9 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 5.961359] pc : __kvm_vcpu_run+0x30/0x70 [ 5.961364] lr : __kvm_vcpu_run+0x24/0x70 [ 5.961365] sp : ffff800080633ad0 [ 5.961366] x29: ffff800080633ad0 x28: ffff000004391280 x27: 0000000000000000 [ 5.961368] x26: 0000000000000000 x25: 0000000000000000 x24: ffff000004438048 [ 5.961369] x23: 0000000000000000 x22: ffff000002efc000 x21: 0000000000402820 [ 5.961371] x20: ffff000004391280 x19: ffff000004438000 x18: 0000000000000000 [ 5.961372] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 5.961373] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 [ 5.961374] x11: 0000000000000000 x10: 0000000000000000 x9 : ffffbf8cfc0a78f0 [ 5.961375] x8 : 0000000000000000 x7 : 0000000000000000 x6 : 0000000000000000 [ 5.961377] x5 : 0000000000001000 x4 : ffff407342580000 x3 : 0000000000000000 [ 5.961378] x2 : 0000000000000000 x1 : 00000000000000c0 x0 : 0000000000000000 [ 5.961379] Kernel panic - not syncing: Asynchronous SError Interrupt [ 5.961381] CPU: 1 UID: 0 PID: 64 Comm: external_aborts Not tainted 6.16.0-rc6-00163-ga03b7055c54b-dirty #4690 PREEMPT [ 5.961382] Hardware name: linux,dummy-virt (DT) [ 5.961383] Call trace: [ 5.961384] show_stack+0x20/0x38 (C) [ 5.961388] dump_stack_lvl+0xc8/0xf8 [ 5.961392] dump_stack+0x18/0x28 [ 5.961393] panic+0x380/0x3e8 [ 5.961395] nmi_panic+0x48/0xa0 [ 5.961396] arm64_serror_panic+0x6c/0x88 [ 5.961398] arm64_is_fatal_ras_serror+0x8c/0x98 [ 5.961399] do_serror+0x3c/0x68 [ 5.961401] el1h_64_error_handler+0x38/0x60 [ 5.961404] el1h_64_error+0x80/0x88 [ 5.961405] __kvm_vcpu_run+0x30/0x70 (P) [ 5.961406] kvm_arm_vcpu_enter_exit+0x64/0x98 [ 5.961408] kvm_arch_vcpu_ioctl_run+0x208/0x620 [ 5.961411] kvm_vcpu_ioctl+0x14c/0x9f8 [ 5.961414] __arm64_sys_ioctl+0x9c/0x100 [ 5.961416] invoke_syscall+0x50/0x120 [ 5.961417] el0_svc_common.constprop.0+0x48/0xf0 [ 5.961418] do_el0_svc+0x24/0x38 [ 5.961419] el0_svc+0x34/0xd8 [ 5.961421] el0t_64_sync_handler+0x10c/0x138 [ 5.961422] el0t_64_sync+0x1ac/0x1b0 [ 5.961425] SMP: stopping secondary CPUs [ 5.961432] Kernel Offset: 0x3f8c7c000000 from 0xffff800080000000 [ 5.961433] PHYS_OFFSET: 0x80000000 [ 5.961433] CPU features: 0x01000,00000700,094f0d61,556ffea7 [ 5.961436] Memory Limit: none [ 5.995295] ---[ end Kernel panic - not syncing: Asynchronous SError Interrupt ]---
where the L1 crashes due to L0 having reinjected the exception in L1.
Another thing is that a E2H==0 L1 never completes this test at all. Nothing bad happens, but this is another indication of something being off.
I'll try to investigate both issues if I get the time.
M.
[1] https://lore.kernel.org/r/[email protected]
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerRe: [PATCH v3 18/27] KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set
On Sun, 20 Jul 2025 11:36:30 +0100,
Marc Zyngier <[email protected]> wrote:
[...]
Quoted 56 lines
If I run this very test in a nested guest (patch applied on both L0 and L1), I get this:
bash-5.2# /host/home/maz/external_aborts
Random seed:0x6b8b4567[ 5.936631] PC = 402764 [ 5.942221] PC = 4027f4 [ 5.961351] SError Interrupt on CPU1, code 0x00000000be000000 -- SError [ 5.961355] CPU: 1 UID: 0 PID: 64 Comm: external_aborts Not tainted 6.16.0-rc6-00163-ga03b7055c54b-dirty #4690 PREEMPT [ 5.961357] Hardware name: linux,dummy-virt (DT) [ 5.961358] pstate: 604000c9 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 5.961359] pc : __kvm_vcpu_run+0x30/0x70 [ 5.961364] lr : __kvm_vcpu_run+0x24/0x70 [ 5.961365] sp : ffff800080633ad0 [ 5.961366] x29: ffff800080633ad0 x28: ffff000004391280 x27: 0000000000000000 [ 5.961368] x26: 0000000000000000 x25: 0000000000000000 x24: ffff000004438048 [ 5.961369] x23: 0000000000000000 x22: ffff000002efc000 x21: 0000000000402820 [ 5.961371] x20: ffff000004391280 x19: ffff000004438000 x18: 0000000000000000 [ 5.961372] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 5.961373] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 [ 5.961374] x11: 0000000000000000 x10: 0000000000000000 x9 : ffffbf8cfc0a78f0 [ 5.961375] x8 : 0000000000000000 x7 : 0000000000000000 x6 : 0000000000000000 [ 5.961377] x5 : 0000000000001000 x4 : ffff407342580000 x3 : 0000000000000000 [ 5.961378] x2 : 0000000000000000 x1 : 00000000000000c0 x0 : 0000000000000000 [ 5.961379] Kernel panic - not syncing: Asynchronous SError Interrupt [ 5.961381] CPU: 1 UID: 0 PID: 64 Comm: external_aborts Not tainted 6.16.0-rc6-00163-ga03b7055c54b-dirty #4690 PREEMPT [ 5.961382] Hardware name: linux,dummy-virt (DT) [ 5.961383] Call trace: [ 5.961384] show_stack+0x20/0x38 (C) [ 5.961388] dump_stack_lvl+0xc8/0xf8 [ 5.961392] dump_stack+0x18/0x28 [ 5.961393] panic+0x380/0x3e8 [ 5.961395] nmi_panic+0x48/0xa0 [ 5.961396] arm64_serror_panic+0x6c/0x88 [ 5.961398] arm64_is_fatal_ras_serror+0x8c/0x98 [ 5.961399] do_serror+0x3c/0x68 [ 5.961401] el1h_64_error_handler+0x38/0x60 [ 5.961404] el1h_64_error+0x80/0x88 [ 5.961405] __kvm_vcpu_run+0x30/0x70 (P) [ 5.961406] kvm_arm_vcpu_enter_exit+0x64/0x98 [ 5.961408] kvm_arch_vcpu_ioctl_run+0x208/0x620 [ 5.961411] kvm_vcpu_ioctl+0x14c/0x9f8 [ 5.961414] __arm64_sys_ioctl+0x9c/0x100 [ 5.961416] invoke_syscall+0x50/0x120 [ 5.961417] el0_svc_common.constprop.0+0x48/0xf0 [ 5.961418] do_el0_svc+0x24/0x38 [ 5.961419] el0_svc+0x34/0xd8 [ 5.961421] el0t_64_sync_handler+0x10c/0x138 [ 5.961422] el0t_64_sync+0x1ac/0x1b0 [ 5.961425] SMP: stopping secondary CPUs [ 5.961432] Kernel Offset: 0x3f8c7c000000 from 0xffff800080000000 [ 5.961433] PHYS_OFFSET: 0x80000000 [ 5.961433] CPU features: 0x01000,00000700,094f0d61,556ffea7 [ 5.961436] Memory Limit: none [ 5.995295] ---[ end Kernel panic - not syncing: Asynchronous SError Interrupt ]---
Quoted 5 lines
where the L1 crashes due to L0 having reinjected the exception in L1.
Another thing is that a E2H==0 L1 never completes this test at all. Nothing bad happens, but this is another indication of something being off.
Having dug into this, the bug is rather obvious: we unconditionally take the L1's HCR_EL2.VSE and merge it into the host's, instead of only merging it when running the L2. As a result, it is extremely likely that the L1 guest will observe the SError instead of the L2, taking it as if it was a physical one. All it takes is a trap, and there is no shortage of those.
I've posted the equally obvious fix at [1].
With this and the previous fix, the external_aborts test runs correctly in both VHE and nVHE guests on my setup (tested on the QC X1E box).
M.
[1] https://lore.kernel.org/r/[email protected]
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH v3 19/27 KVM: arm64: nv: Honor SError routing effects of SCTLR2_ELx.NMEA
In reply to Oliver UptonAs the name might imply, when NMEA is set SErrors are non-maskable and can be taken regardless of PSTATE.A. As is the recurring theme with DoubleFault2, the effects on SError routing are entirely backwards to this.
If at EL1, NMEA is *not* considered for SError routing when TMEA is set and the exception is taken to EL2 when PSTATE.A is set.
--- arch/arm64/kvm/inject_fault.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)
arch/arm64/kvm/inject_fault.c +19 -1
@@ -97,6 +97,11 @@ static bool effective_sctlr2_ease(struct kvm_vcpu *vcpu)return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_EASE_SHIFT ); } +static bool effective_sctlr2_nmea (struct kvm_vcpu *vcpu) +{ +return __effective_sctlr2_bit (vcpu,SCTLR2_EL1_NMEA_SHIFT ); +} +static void inject_abt64 (struct kvm_vcpu *vcpu,bool is_iabt,unsigned long addr) {unsigned long cpsr = *vcpu_cpsr (vcpu);
@@ -267,7 +272,7 @@ void kvm_inject_undefined(struct kvm_vcpu *vcpu)static bool serror_is_masked (struct kvm_vcpu *vcpu) { -return *vcpu_cpsr (vcpu) &PSR_A_BIT ; +return (*vcpu_cpsr (vcpu) &PSR_A_BIT ) && !effective_sctlr2_nmea (vcpu); }static bool kvm_serror_target_is_el2 (struct kvm_vcpu *vcpu)
@@ -278,6 +283,19 @@ static bool kvm_serror_target_is_el2(struct kvm_vcpu *vcpu)if (!(__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ))return false; +/* +* In another example where FEAT_DoubleFault2 is entirely backwards, +* "masked" as it relates to the routing effects of HCRX_EL2.TMEA +* doesn't consider SCTLR2_EL1.NMEA. That is to say, even if EL1 asked +* for non-maskable SErrors, the EL2 bit takes priority if A is set. +*/ +if (vcpu_mode_priv (vcpu)) +return *vcpu_cpsr (vcpu) &PSR_A_BIT ; + +/* +* Otherwise SErrors are considered unmasked when taken from EL0 and +* NMEA is set. +*/ return serror_is_masked (vcpu); }
Signature · 1 line
2.39.5
PATCH v3 20/27 KVM: arm64: nv: Enable vSErrors when HCRX_EL2.TMEA is set
In reply to Oliver UptonPer R_CDCKC, vSErrors are enabled if HCRX_EL2.TMEA is set, regardless of HCR_EL2.AMO.
--- arch/arm64/include/asm/kvm_emulate.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_emulate.h +5 -1
@@ -259,7 +259,11 @@ static inline bool is_nested_ctxt(struct kvm_vcpu *vcpu)static inline bool vserror_state_is_nested (struct kvm_vcpu *vcpu) { -return is_nested_ctxt (vcpu) &&vcpu_el2_amo_is_set (vcpu); +if (!is_nested_ctxt (vcpu)) +return false; + +return vcpu_el2_amo_is_set (vcpu) || + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }/*
Signature · 1 line
2.39.5
PATCH v3 21/27 KVM: arm64: Advertise support for FEAT_SCTLR2
In reply to Oliver UptonEverything is in place to handle the additional state for SCTLR2_ELx, which is all that FEAT_SCTLR2 implies.
--- arch/arm64/kvm/sys_regs.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/arm64/kvm/sys_regs.c +5 -2
@@ -1644,8 +1644,10 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, val &= ~ID_AA64MMFR2_EL1_NV ;break ;case SYS_ID_AA64MMFR3_EL1 : - val &=ID_AA64MMFR3_EL1_TCRX |ID_AA64MMFR3_EL1_S1POE | -ID_AA64MMFR3_EL1_S1PIE ; + val &=ID_AA64MMFR3_EL1_TCRX | +ID_AA64MMFR3_EL1_SCTLRX | +ID_AA64MMFR3_EL1_S1POE | +ID_AA64MMFR3_EL1_S1PIE ;break ;case SYS_ID_MMFR4_EL1 : val &= ~ARM64_FEATURE_MASK (ID_MMFR4_EL1_CCIDX );
@@ -2961,6 +2963,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {ID_AA64MMFR2_EL1_NV | ID_AA64MMFR2_EL1_CCIDX)),ID_WRITABLE (ID_AA64MMFR3_EL1 , (ID_AA64MMFR3_EL1_TCRX | +ID_AA64MMFR3_EL1_SCTLRX |ID_AA64MMFR3_EL1_S1PIE |ID_AA64MMFR3_EL1_S1POE )), ID_WRITABLE(ID_AA64MMFR4_EL1, ID_AA64MMFR4_EL1_NV_frac),
Signature · 1 line
2.39.5
PATCH v3 22/27 KVM: arm64: Advertise support for FEAT_DoubleFault2
In reply to Oliver UptonKVM's external abort injection now respects the exception routing wreckage due to FEAT_DoubleFault2. Advertise the feature.
--- arch/arm64/kvm/sys_regs.c | 2 -- 1 file changed, 2 deletions(-)
arch/arm64/kvm/sys_regs.c +0 -2
@@ -1613,7 +1613,6 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_GCS ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_THE ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_MTEX ); - val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_DF2 ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_PFAR ); val &= ~ARM64_FEATURE_MASK (ID_AA64PFR1_EL1_MPAM_frac);break ;
@@ -2884,7 +2883,6 @@ static const struct sys_reg_desc sys_reg_descs[] = { ID_AA64PFR0_EL1_FP)), ID_FILTERED(ID_AA64PFR1_EL1, id_aa64pfr1_el1, ~(ID_AA64PFR1_EL1_PFAR | - ID_AA64PFR1_EL1_DF2 | ID_AA64PFR1_EL1_MTEX | ID_AA64PFR1_EL1_THE | ID_AA64PFR1_EL1_GCS |
Signature · 1 line
2.39.5
PATCH v3 23/27 KVM: arm64: Don't retire MMIO instruction w/ pending (emulated) SError
In reply to Oliver UptonKVM might have an emulated SError queued for the guest if userspace returned an abort for MMIO. Better yet, it could actually be a *synchronous* exception in disguise if SCTLR2_ELx.EASE is set.
Don't advance PC if KVM owes an emulated SError, just like the handling of emulated SEA injection.
--- arch/arm64/kvm/mmio.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm64/kvm/mmio.c +4 -2
@@ -72,7 +72,7 @@ unsigned long kvm_mmio_read_buf(const void *buf, unsigned int len)return data; } -static bool kvm_pending_sync_exception (struct kvm_vcpu *vcpu) +static bool kvm_pending_external_abort (struct kvm_vcpu *vcpu) {if (!vcpu_get_flag (vcpu,PENDING_EXCEPTION ))return false;
@@ -90,6 +90,8 @@ static bool kvm_pending_sync_exception(struct kvm_vcpu *vcpu)switch (vcpu_get_flag (vcpu,EXCEPT_MASK )) {case unpack_vcpu_flag (EXCEPT_AA64_EL1_SYNC ):case unpack_vcpu_flag (EXCEPT_AA64_EL2_SYNC ): +case unpack_vcpu_flag (EXCEPT_AA64_EL1_SERR ): +case unpack_vcpu_flag (EXCEPT_AA64_EL2_SERR ):return true;default :return false;
@@ -113,7 +115,7 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu)* Detect if the MMIO return was already handled or if userspace aborted * the MMIO access. */ -if (unlikely (!vcpu->mmio_needed ||kvm_pending_sync_exception (vcpu))) +if (unlikely (!vcpu->mmio_needed ||kvm_pending_external_abort (vcpu)))return 1 ; vcpu->mmio_needed =0 ;
Signature · 1 line
2.39.5
PATCH v3 24/27 KVM: arm64: selftests: Add basic SError injection test
In reply to Oliver UptonAdd tests for SError injection considering KVM is more directly involved in delivery:
- Pending SErrors are taken at the first CSE after SErrors are unmasked
- Pending SErrors aren't taken and remain pending if SErrors are masked
- Unmasked SErrors are taken immediately when injected (implementation detail)
---
tools/testing/selftests/kvm/Makefile.kvm | 2 +-
.../arm64/{mmio_abort.c => external_aborts.c} | 117 ++++++++++++++++--
.../selftests/kvm/include/arm64/processor.h | 10 ++
3 files changed, 121 insertions(+), 8 deletions(-)
rename tools/testing/selftests/kvm/arm64/{mmio_abort.c => external_aborts.c} (60%)tools/testing/selftests/kvm/Makefile.kvm +1 -1
@@ -156,7 +156,7 @@ TEST_GEN_PROGS_arm64 += arm64/arch_timer_edge_cases TEST_GEN_PROGS_arm64 += arm64/debug-exceptions TEST_GEN_PROGS_arm64 += arm64/host_sve TEST_GEN_PROGS_arm64 += arm64/hypercalls -TEST_GEN_PROGS_arm64 += arm64/mmio_abort +TEST_GEN_PROGS_arm64 += arm64/external_aborts TEST_GEN_PROGS_arm64 += arm64/page_fault_test TEST_GEN_PROGS_arm64 += arm64/psci_test TEST_GEN_PROGS_arm64 += arm64/set_id_regs
tools/testing/selftests/kvm/arm64/external_aborts.c renamed from tools/testing/selftests/kvm/arm64/mmio_abort.c +110 -7
161 lines not on this page · raw message
tools/testing/selftests/kvm/include/arm64/processor.h +10 -0
@@ -254,6 +254,16 @@ static inline void local_irq_disable(void) asmvolatile ("msr daifset, #3" : : :"memory" ); } +static inline void local_serror_enable (void ) +{ + asmvolatile ("msr daifclr, #4" : : :"memory" ); +} + +static inline void local_serror_disable (void ) +{ + asmvolatile ("msr daifset, #4" : : :"memory" ); +} +/** * struct arm_smccc_res - Result from SMC/HVC call * @a0-a3 result values from registers 0 to 3
Signature · 1 line
2.39.5
PATCH v3 25/27 KVM: arm64: selftests: Test SEAs are taken to SError vector when EASE=1
In reply to Oliver UptonEnsure KVM routes SEAs to the correct vector depending on SCTLR2_EL1.EASE.
--- .../selftests/kvm/arm64/external_aborts.c | 42 ++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/arm64/external_aborts.c +41 -1
@@ -86,7 +86,7 @@ static void vcpu_run_expect_sync(struct kvm_vcpu *vcpu)extern char test_mmio_abort_insn; -static void test_mmio_abort_guest (void ) +static noinline void test_mmio_abort_guest(void) {WRITE_ONCE (expected_abort_pc, (u64)&test_mmio_abort_insn);
@@ -251,6 +251,45 @@ static void test_serror_emulated(void)kvm_vm_free (vm); } +static void test_mmio_ease_guest (void ) +{ +sysreg_clear_set_s (SYS_SCTLR2_EL1 ,0 ,SCTLR2_EL1_EASE ); +isb (); + +test_mmio_abort_guest (); +} + +/* +* Test that KVM doesn't complete MMIO emulation when userspace has made an +* external abort pending for the instruction. +*/ +static void test_mmio_ease (void ) +{ +struct kvm_vcpu *vcpu; +struct kvm_vm *vm =vm_create_with_dabt_handler (&vcpu, test_mmio_ease_guest, + unexpected_dabt_handler); +struct kvm_run *run = vcpu->run; +u64 pfr1; + + pfr1 =vcpu_get_reg (vcpu,KVM_ARM64_SYS_REG (SYS_ID_AA64PFR1_EL1 )); +if (!SYS_FIELD_GET (ID_AA64PFR1_EL1 ,DF2 , pfr1)) { +pr_debug ("Skipping %s\n" , __func__); +return ; + } + +vm_install_exception_handler (vm,VECTOR_ERROR_CURRENT , expect_serror_handler); + +vcpu_run (vcpu); +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_MMIO ); +TEST_ASSERT_EQ (run->mmio.phys_addr,MMIO_ADDR ); +TEST_ASSERT_EQ (run->mmio.len,sizeof (unsigned long )); +TEST_ASSERT (!run->mmio.is_write,"Expected MMIO read" ); + +vcpu_inject_sea (vcpu); +vcpu_run_expect_done (vcpu); +kvm_vm_free (vm); +} +int main (void ) {test_mmio_abort ();
@@ -259,4 +298,5 @@ int main(void)test_serror ();test_serror_masked ();test_serror_emulated (); +test_mmio_ease (); }
Signature · 1 line
2.39.5
PATCH v3 26/27 KVM: arm64: selftests: Add SCTLR2_EL1 to get-reg-list
In reply to Oliver UptonHandle SCTLR2_EL1 specially as it is only visible to userspace when FEAT_SCTLR2 is implemented for the VM.
--- tools/testing/selftests/kvm/arm64/get-reg-list.c | 7 +++++++ 1 file changed, 7 insertions(+)
tools/testing/selftests/kvm/arm64/get-reg-list.c +7 -0
@@ -52,6 +52,12 @@ static struct feature_id_reg feat_id_regs[] = {ARM64_SYS_REG (3 ,0 ,0 ,7 ,3 ),/* ID_AA64MMFR3_EL1 */ 16 ,1 + }, + { +KVM_ARM64_SYS_REG (SYS_SCTLR2_EL1 ), +KVM_ARM64_SYS_REG (SYS_ID_AA64MMFR3_EL1 ), +ID_AA64MMFR3_EL1_SCTLRX_SHIFT , + ID_AA64MMFR3_EL1_SCTLRX_IMP } };
@@ -469,6 +475,7 @@ static __u64 base_regs[] = {ARM64_SYS_REG (3 ,0 ,1 ,0 ,0 ),/* SCTLR_EL1 */ ARM64_SYS_REG (3 ,0 ,1 ,0 ,1 ),/* ACTLR_EL1 */ ARM64_SYS_REG (3 ,0 ,1 ,0 ,2 ),/* CPACR_EL1 */ +KVM_ARM64_SYS_REG (SYS_SCTLR2_EL1 ),ARM64_SYS_REG (3 ,0 ,2 ,0 ,0 ),/* TTBR0_EL1 */ ARM64_SYS_REG (3 ,0 ,2 ,0 ,1 ),/* TTBR1_EL1 */ ARM64_SYS_REG(3, 0, 2, 0, 2), /* TCR_EL1 */
Signature · 1 line
2.39.5
PATCH v3 27/27 KVM: arm64: selftests: Catch up set_id_regs with the kernel
In reply to Oliver UptonAdd test coverage for ID_AA64MMFR3_EL1 and the recently added FEAT_DoubleFault2.
--- tools/testing/selftests/kvm/arm64/set_id_regs.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/arm64/set_id_regs.c +12 -2
@@ -139,6 +139,7 @@ static const struct reg_ftr_bits ftr_id_aa64pfr0_el1[] = { };static const struct reg_ftr_bits ftr_id_aa64pfr1_el1[] = { +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 ,DF2 ,0 ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 , CSV2_frac,0 ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 ,SSBS ,ID_AA64PFR1_EL1_SSBS_NI ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64PFR1_EL1 ,BT ,0 ),
@@ -187,6 +188,14 @@ static const struct reg_ftr_bits ftr_id_aa64mmfr2_el1[] = { REG_FTR_END, }; +static const struct reg_ftr_bits ftr_id_aa64mmfr3_el1[] = { +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,S1POE ,0 ), +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,S1PIE ,0 ), +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,SCTLRX ,0 ), +REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64MMFR3_EL1 ,TCRX ,0 ), +REG_FTR_END , +}; +static const struct reg_ftr_bits ftr_id_aa64zfr0_el1[] = {REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64ZFR0_EL1 ,F64MM ,0 ),REG_FTR_BITS (FTR_LOWER_SAFE ,ID_AA64ZFR0_EL1 ,F32MM ,0 ),
@@ -217,6 +226,7 @@ static struct test_feature_reg test_regs[] = {TEST_REG (SYS_ID_AA64MMFR0_EL1 , ftr_id_aa64mmfr0_el1),TEST_REG (SYS_ID_AA64MMFR1_EL1 , ftr_id_aa64mmfr1_el1),TEST_REG (SYS_ID_AA64MMFR2_EL1 , ftr_id_aa64mmfr2_el1), +TEST_REG (SYS_ID_AA64MMFR3_EL1 , ftr_id_aa64mmfr3_el1), TEST_REG(SYS_ID_AA64ZFR0_EL1, ftr_id_aa64zfr0_el1), };
@@ -774,8 +784,8 @@ int main(void)ARRAY_SIZE (ftr_id_aa64isar2_el1) +ARRAY_SIZE (ftr_id_aa64pfr0_el1) +ARRAY_SIZE (ftr_id_aa64pfr1_el1) +ARRAY_SIZE (ftr_id_aa64mmfr0_el1) +ARRAY_SIZE (ftr_id_aa64mmfr1_el1) +ARRAY_SIZE (ftr_id_aa64mmfr2_el1) + -ARRAY_SIZE (ftr_id_aa64zfr0_el1) -ARRAY_SIZE (test_regs) +3 + -MPAM_IDREG_TEST +MTE_IDREG_TEST ; +ARRAY_SIZE (ftr_id_aa64mmfr3_el1) +ARRAY_SIZE (ftr_id_aa64zfr0_el1) - +ARRAY_SIZE (test_regs) +3 +MPAM_IDREG_TEST +MTE_IDREG_TEST ;ksft_set_plan (test_cnt);
Signature · 1 line
2.39.5
Oliver Upton
AppliedQuoted 11 lines · Oliver Upton
v2: https://lore.kernel.org/kvmarm/[email protected]/
v2 -> v3:
- Fix mask computation when SCLTR2_ELx.NMEA is set
- Consolidate TMEA handling for all aborts, spin off NMEA (Marc)
- Set up the fault context correctly for the injected abort (e.g. not settingHPFAR_EL2when UNKNOWN) (Marc)
- Don't use the guest hypervisor's VSESR whenFEAT_RASisn't implemented for the guest (Marc)[...]
Applied to next, thanks!
[01/27] arm64: Detect FEAT_SCTLR2
https://git.kernel.org/kvmarm/kvmarm/c/bf49e73dde7d [02/27] arm64: Detect FEAT_DoubleFault2 https://git.kernel.org/kvmarm/kvmarm/c/e3fd66620f10 [03/27] KVM: arm64: Add helper to identify a nested context https://git.kernel.org/kvmarm/kvmarm/c/1d6fea7663b2 [04/27] KVM: arm64: Treat vCPU with pending SError as runnable https://git.kernel.org/kvmarm/kvmarm/c/aae35f4ffbf2 [05/27] KVM: arm64: nv: Respect exception routing rules for SEAs https://git.kernel.org/kvmarm/kvmarm/c/9aba641b9ec2 [06/27] KVM: arm64: nv: Honor SError exception routing / masking https://git.kernel.org/kvmarm/kvmarm/c/77ee70a07357 [07/27] KVM: arm64: nv: Add FEAT_RAS vSError sys regs to table https://git.kernel.org/kvmarm/kvmarm/c/211fced460f2 [08/27] KVM: arm64: nv: Use guest hypervisor's vSError state https://git.kernel.org/kvmarm/kvmarm/c/18fbc24707db [09/27] KVM: arm64: nv: Advertise support for FEAT_RAS https://git.kernel.org/kvmarm/kvmarm/c/a99456abd834 [10/27] KVM: arm64: nv: Describe trap behavior of SCTLR2_EL1 https://git.kernel.org/kvmarm/kvmarm/c/0ead48acc935 [11/27] KVM: arm64: Wire up SCTLR2_ELx sysreg descriptors https://git.kernel.org/kvmarm/kvmarm/c/81fbef164766 [12/27] KVM: arm64: Context switch SCTLR2_ELx when advertised to the guest https://git.kernel.org/kvmarm/kvmarm/c/02dd33ec8831 [13/27] KVM: arm64: Enable SCTLR2 when advertised to the guest https://git.kernel.org/kvmarm/kvmarm/c/7fb7660b9c0b [14/27] KVM: arm64: Describe SCTLR2_ELx RESx masks https://git.kernel.org/kvmarm/kvmarm/c/abc693fef30c [15/27] KVM: arm64: Factor out helper for selecting exception target EL https://git.kernel.org/kvmarm/kvmarm/c/720ef4611c46 [16/27] KVM: arm64: nv: Ensure Address size faults affect correct ESR https://git.kernel.org/kvmarm/kvmarm/c/178ec0ae35f8 [17/27] KVM: arm64: Route SEAs to the SError vector when EASE is set https://git.kernel.org/kvmarm/kvmarm/c/fff97df2a0bd [18/27] KVM: arm64: nv: Take "masked" aborts to EL2 when HCRX_EL2.TMEA is set https://git.kernel.org/kvmarm/kvmarm/c/ce66109cec86 [19/27] KVM: arm64: nv: Honor SError routing effects of SCTLR2_ELx.NMEA https://git.kernel.org/kvmarm/kvmarm/c/59b6d08666f0 [20/27] KVM: arm64: nv: Enable vSErrors when HCRX_EL2.TMEA is set https://git.kernel.org/kvmarm/kvmarm/c/1f1c08d9896c [21/27] KVM: arm64: Advertise support for FEAT_SCTLR2 https://git.kernel.org/kvmarm/kvmarm/c/075c2dc7367e [22/27] KVM: arm64: Advertise support for FEAT_DoubleFault2 https://git.kernel.org/kvmarm/kvmarm/c/a3c4a00dbe20 [23/27] KVM: arm64: Don't retire MMIO instruction w/ pending (emulated) SError https://git.kernel.org/kvmarm/kvmarm/c/bfb7a30b1986 [24/27] KVM: arm64: selftests: Add basic SError injection test https://git.kernel.org/kvmarm/kvmarm/c/2858ea3083f0 [25/27] KVM: arm64: selftests: Test SEAs are taken to SError vector when EASE=1 https://git.kernel.org/kvmarm/kvmarm/c/a90aac553249 [26/27] KVM: arm64: selftests: Add SCTLR2_EL1 to get-reg-list https://git.kernel.org/kvmarm/kvmarm/c/55ea75f5b273 [27/27] KVM: arm64: selftests: Catch up set_id_regs with the kernel https://git.kernel.org/kvmarm/kvmarm/c/0b593ef12afc
--
Best,
Oliver