KVM: arm64: A510 errata workaround and fixes for fixup_guest_exit()
Subsystems (3)
v2 cover KVM: arm64: A510 errata workaround and fixes for fixup_guest_exit()
Hello!
Changes since v1:
* Moved the SPSR_EL2 fixup into a helper called earlier
* Use final cap
* Dropped the IS_ENABLED() check
---
Early Cortex-A510 parts have a nasty erratum where two ERETs, pointer-auth and software step conspire to corrupt SPSR_EL2. A guest can only trigger this when it is being stepped by EL2, which gives EL2 the opportunity to work around the erratum. Patch 4 does this, the SDEN is available from:
https://developer.arm.com/documentation/SDEN2397239/900
Patches 2 and 3 fix two issues with the adjacent code where a stale esr value could be used to alter the ELR_EL2 when an IRQ synchronises an SError, and when an HVC synchronises an SError, the HVC may be handled twice, (not just execute twice).
There are three series that would add the Cortex-A510 part macros. I've picked Anshuman's patch that does this, on the assumption that makes someone's life easier. I haven't spotted that patch on the arm64/for-next/fixes branch, so I've not included the hash in the prerequisite field of the CC-stable.
Let me know if you want this reposted once that value is known.
This series is based on v5.17-rc1 and can be retrieved from:
https://git.gitlab.arm.com/linux-arm/linux-jm.git a510_errata/kvm_bits/v2
Thanks,
James
Anshuman Khandual (1):
arm64: Add Cortex-A510 CPU part definition
James Morse (3):
KVM: arm64: Avoid consuming a stale esr value when SError occur
KVM: arm64: Stop handle_exit() from handling HVC twice when an SError
occurs
KVM: arm64: Workaround Cortex-A510's single-step and PAC trap errata
Documentation/arm64/silicon-errata.rst | 2 ++ arch/arm64/Kconfig | 16 ++++++++++++++++ arch/arm64/include/asm/cputype.h | 2 ++ arch/arm64/kernel/cpu_errata.c | 8 ++++++++ arch/arm64/kvm/handle_exit.c | 8 ++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +++++++++++++++++++++-- arch/arm64/tools/cpucaps | 1 + 7 files changed, 58 insertions(+), 2 deletions(-)
Signature · 6 lines
2.30.2 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 1/4 arm64: Add Cortex-A510 CPU part definition
From: Anshuman Khandual <[email protected]>
Add the CPU Partnumbers for the new Arm designs.
--- arch/arm64/include/asm/cputype.h | 2 ++ 1 file changed, 2 insertions(+)
arch/arm64/include/asm/cputype.h +2 -0
@@ -73,6 +73,7 @@#define ARM_CPU_PART_CORTEX_A76 0xD0B#define ARM_CPU_PART_NEOVERSE_N1 0xD0C#define ARM_CPU_PART_CORTEX_A77 0xD0D +#define ARM_CPU_PART_CORTEX_A510 0xD46#define ARM_CPU_PART_CORTEX_A710 0xD47#define ARM_CPU_PART_NEOVERSE_N2 0xD49
@@ -115,6 +116,7 @@#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)#define MIDR_CORTEX_A77 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A77) +#define MIDR_CORTEX_A510 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A510)#define MIDR_CORTEX_A710 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A710)#define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2)#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
Signature · 6 lines
2.30.2 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 2/4 KVM: arm64: Avoid consuming a stale esr value when SError occur
In reply to James MorseWhen any exception other than an IRQ occurs, the CPU updates the ESR_EL2 register with the exception syndrome. An SError may also become pending, and will be synchronised by KVM. KVM notes the exception type, and whether an SError was synchronised in exit_code.
When an exception other than an IRQ occurs, fixup_guest_exit() updates vcpu->arch.fault.esr_el2 from the hardware register. When an SError was synchronised, the vcpu esr value is used to determine if the exception was due to an HVC. If so, ELR_EL2 is moved back one instruction. This is so that KVM can process the SError first, and re-execute the HVC if the guest survives the SError.
But if an IRQ synchronises an SError, the vcpu's esr value is stale. If the previous non-IRQ exception was an HVC, KVM will corrupt ELR_EL2, causing an unrelated guest instruction to be executed twice.
Check ARM_EXCEPTION_CODE() before messing with ELR_EL2, IRQs don't update this register so don't need to check.
--- arch/arm64/kvm/hyp/include/hyp/switch.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +2 -1
@@ -424,7 +424,8 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)if (ARM_EXCEPTION_CODE (*exit_code) !=ARM_EXCEPTION_IRQ ) vcpu->arch.fault.esr_el2 =read_sysreg_el2 (SYS_ESR ); -if (ARM_SERROR_PENDING (*exit_code)) { +if (ARM_SERROR_PENDING (*exit_code) && +ARM_EXCEPTION_CODE (*exit_code) !=ARM_EXCEPTION_IRQ ) {u8 esr_ec =kvm_vcpu_trap_get_class (vcpu);/*
Signature · 6 lines
2.30.2 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 3/4 KVM: arm64: Stop handle_exit() from handling HVC twice when an SError occurs
In reply to James MorsePrior to commit defe21f49bc9 ("KVM: arm64: Move PC rollback on SError to HYP"), when an SError is synchronised due to another exception, KVM handles the SError first. If the guest survives, the instruction that triggered the original exception is re-exectued to handle the first exception. HVC is treated as a special case as the instruction wouldn't normally be re-exectued, as its not a trap.
Commit defe21f49bc9 didn't preserve the behaviour of the 'return 1' that skips the rest of handle_exit().
Since commit defe21f49bc9, KVM will try to handle the SError and the original exception at the same time. When the exception was an HVC, fixup_guest_exit() has already rolled back ELR_EL2, meaning if the guest has virtual SError masked, it will execute and handle the HVC twice.
Restore the original behaviour.
---
It may be possible to remove both this patch, and the HVC handling code in fixup_guest_exit(). This means KVM would always handle the exception and the SError. This may result in unnecessary work if the guest takes the virtual SError when it is next restarted, but should be harmless if SError are always re-injected using HCR_EL2.VSE.
--- arch/arm64/kvm/handle_exit.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/arm64/kvm/handle_exit.c +8 -0
@@ -228,6 +228,14 @@ int handle_exit(struct kvm_vcpu *vcpu, int exception_index) {struct kvm_run *run = vcpu->run; +if (ARM_SERROR_PENDING (exception_index)) { +/* +* The SError is handled by handle_exit_early(). If the guest +* survives it will re-execute the original instruction. +*/ +return 1 ; + } + exception_index =ARM_EXCEPTION_CODE (exception_index);switch (exception_index) {
Signature · 6 lines
2.30.2 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v2 4/4 KVM: arm64: Workaround Cortex-A510's single-step and PAC trap errata
In reply to James MorseCortex-A510's erratum #2077057 causes SPSR_EL2 to be corrupted when single-stepping authenticated ERET instructions. A single step is expected, but a pointer authentication trap is taken instead. The erratum causes SPSR_EL1 to be copied to SPSR_EL2, which could allow EL1 to cause a return to EL2 with a guest controlled ELR_EL2.
Because the conditions require an ERET into active-not-pending state, this is only a problem for the EL2 when EL2 is stepping EL1. In this case the previous SPSR_EL2 value is preserved in struct kvm_vcpu, and can be restored.
---
Changes since v1:
* Moved the SPSR_EL2 fixup into a helper called earlier
* Use final cap
* Dropped the IS_ENABLED() check
Documentation/arm64/silicon-errata.rst | 2 ++ arch/arm64/Kconfig | 16 ++++++++++++++++ arch/arm64/kernel/cpu_errata.c | 8 ++++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 20 +++++++++++++++++++- arch/arm64/tools/cpucaps | 1 + 5 files changed, 46 insertions(+), 1 deletion(-)
Documentation/arm64/silicon-errata.rst +2 -0
@@ -92,6 +92,8 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A77 | #1508412 | ARM64_ERRATUM_1508412 | +----------------+-----------------+-----------------+-----------------------------+ +| ARM | Cortex-A510 | #2077057 | ARM64_ERRATUM_2077057 | ++----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A710 | #2119858 | ARM64_ERRATUM_2119858 | +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A710 | #2054223 | ARM64_ERRATUM_2054223 |
arch/arm64/Kconfig +16 -0
@@ -670,6 +670,22 @@ config ARM64_ERRATUM_1508412 config ARM64_WORKAROUND_TRBE_OVERWRITE_FILL_MODE bool +config ARM64_ERRATUM_2077057 + bool "Cortex-A510: 2077057: workaround software-step corrupting SPSR_EL2" + help + This option adds the workaround for ARM Cortex-A510 erratum 2077057. + Affected Cortex-A510 may corrupt SPSR_EL2 when the a step exception is + expected, but a Pointer Authentication trap is taken instead. The + erratum causes SPSR_EL1 to be copied to SPSR_EL2, which could allow + EL1 to cause a return to EL2 with a guest controlled ELR_EL2. + + This can only happen when EL2 is stepping EL1. + + When these conditions occur, the SPSR_EL2 value is unchanged from the + previous guest entry, and can be restored from the in-memory copy. + + If unsure, say Y. + config ARM64_ERRATUM_2119858 bool "Cortex-A710: 2119858: workaround TRBE overwriting trace data in FILL mode" default y
arch/arm64/kernel/cpu_errata.c +8 -0
@@ -597,6 +597,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE, CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus), }, +#endif +#ifdef CONFIG_ARM64_ERRATUM_2077057 + { + .desc = "ARM erratum 2077057", + .capability = ARM64_WORKAROUND_2077057, + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, + ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A510, 0, 0, 2), + }, #endif { }
arch/arm64/kvm/hyp/include/hyp/switch.h +19 -1
@@ -402,6 +402,24 @@ static inline bool kvm_hyp_handle_exit(struct kvm_vcpu *vcpu, u64 *exit_code)return false; } +static inline void synchronize_vcpu_pstate (struct kvm_vcpu *vcpu,u64 *exit_code) +{ +/* +* Check for the conditions of Cortex-A510's #2077057. When these occur +* SPSR_EL2 can't be trusted, but isn't needed either as it is +* unchanged from the value in vcpu_gp_regs(vcpu)->pstate. +* Are we single-stepping the guest, and took a PAC exception from the +* active-not-pending state? +*/ +if (cpus_have_final_cap (ARM64_WORKAROUND_2077057 ) && + vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP && + *vcpu_cpsr (vcpu) &DBG_SPSR_SS && +ESR_ELx_EC (read_sysreg_el2 (SYS_ESR )) == ESR_ELx_EC_PAC) +write_sysreg_el2 (*vcpu_cpsr (vcpu),SYS_SPSR ); + + vcpu->arch.ctxt.regs.pstate =read_sysreg_el2 (SYS_SPSR ); +} +/* * Return true when we were able to fixup the guest exit and should return to * the guest, false when we should restore the host state and return to the
@@ -413,7 +431,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)* Save PSTATE early so that we can evaluate the vcpu mode * early on. */ - vcpu->arch.ctxt.regs.pstate =read_sysreg_el2 (SYS_SPSR ); +synchronize_vcpu_pstate (vcpu, exit_code);/* * Check whether we want to repaint the state one way or
arch/arm64/tools/cpucaps +1 -0
@@ -55,6 +55,7 @@ WORKAROUND_1418040 WORKAROUND_1463225 WORKAROUND_1508412 WORKAROUND_1542419 +WORKAROUND_2077057 WORKAROUND_TRBE_OVERWRITE_FILL_MODE WORKAROUND_TSB_FLUSH_FAILURE WORKAROUND_TRBE_WRITE_OUT_OF_RANGE
Signature · 6 lines
2.30.2 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Quoted 4 lines · James Morse
Changes since v1:
* Moved theSPSR_EL2fixup into a helper called earlier
* Use final cap
* Dropped theIS_ENABLED()check
Applied to fixes, thanks!
[2/4] KVM: arm64: Avoid consuming a stale esr value when SError occur
commit: 1c71dbc8a179d99dd9bb7e7fc1888db613cf85de
[3/4] KVM: arm64: Stop handle_exit() from handling HVC twice when an SError occurs
commit: 1229630af88620f6e3a621a1ebd1ca14d9340df7
[4/4] KVM: arm64: Workaround Cortex-A510's single-step and PAC trap errata
commit: 1dd498e5e26ad71e3e9130daf72cfb6a693fee03Cheers,
M.
Signature · 7 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm