arm64: mte: move register initialization to C
PATCH v4 1/1 arm64: mte: move register initialization to C
If FEAT_MTE2 is disabled via the arm64.nomte command line argument on a CPU that claims to support FEAT_MTE2, the kernel will use Tagged Normal in the MAIR. If we interpret arm64.nomte to mean that the CPU does not in fact implement FEAT_MTE2, setting the system register like this may lead to UNSPECIFIED behavior. Fix it by arranging for MAIR to be set in the C function cpu_enable_mte which is called based on the sanitized version of the system register.
There is no need for the rest of the MTE-related system register initialization to happen from assembly, with the exception of TCR_EL1, which must be set to include at least TBI1 because the secondary CPUs access KASan-allocated data structures early. Therefore, make the TCR_EL1 initialization unconditional and move the rest of the initialization to cpu_enable_mte so that we no longer have a dependency on the unsanitized ID register value.
---
Changelong since v3:
- Removed extra isb barrier.
- Fixed build without CONFIG_ARM64_MTE.
Changelog since v2:
- Fixed register initialization on cpu_resume code path.
Changelog since v1:
- Keep TBI1 off unless CONFIG_ARM64_MTE
- Fixed mask application in the RGSR_EL1 computation (bug found by Kenny Root).
- Changed code formatting
arch/arm64/include/asm/mte.h | 5 ++++ arch/arm64/kernel/cpufeature.c | 3 +- arch/arm64/kernel/mte.c | 51 ++++++++++++++++++++++++++++++++++ arch/arm64/kernel/suspend.c | 2 ++ arch/arm64/mm/proc.S | 46 ++++-------------------------- 5 files changed, 65 insertions(+), 42 deletions(-)
arch/arm64/include/asm/mte.h +5 -0
@@ -42,7 +42,9 @@ void mte_sync_tags(pte_t old_pte, pte_t pte);void mte_copy_page_tags (void *kto,const void *kfrom);void mte_thread_init_user (void );void mte_thread_switch (struct task_struct *next); +void mte_cpu_setup (void );void mte_suspend_enter (void ); +void mte_suspend_exit (void );long set_mte_ctrl (struct task_struct *task,unsigned long arg);long get_mte_ctrl (struct task_struct *task); int mte_ptrace_copy_tags(struct task_struct *child, long request,
@@ -72,6 +74,9 @@ static inline void mte_thread_switch(struct task_struct *next)static inline void mte_suspend_enter (void ) { } +static inline void mte_suspend_exit (void ) +{ +}static inline long set_mte_ctrl (struct task_struct *task,unsigned long arg) {return 0 ;
arch/arm64/kernel/cpufeature.c +2 -1
@@ -2034,7 +2034,8 @@ static void bti_enable(const struct arm64_cpu_capabilities *__unused)static void cpu_enable_mte (struct arm64_cpu_capabilities const *cap) {sysreg_clear_set (sctlr_el1,0 , SCTLR_ELx_ATA |SCTLR_EL1_ATA0 ); -isb (); + +mte_cpu_setup ();/* * Clear the tags in the zero page. This needs to be done via the
arch/arm64/kernel/mte.c +51 -0
63 lines not on this page · raw message
arch/arm64/kernel/suspend.c +2 -0
@@ -43,6 +43,8 @@ void notrace __cpu_suspend_exit(void) {unsigned int cpu =smp_processor_id (); +mte_suspend_exit (); +/* * We are resuming from reset with the idmap active in TTBR0_EL1. * We must uninstall the idmap and restore the expected MMU
arch/arm64/mm/proc.S +5 -41
68 lines not on this page · raw message
Signature · 1 line
2.37.2.789.g6183377224-goog
Quoted 9 lines · Evgenii Stepanov
If
FEAT_MTE2is disabled via the arm64.nomte command line argument on a CPU that claims to supportFEAT_MTE2, the kernel will use Tagged Normal in the MAIR. If we interpret arm64.nomte to mean that the CPU does not in fact implementFEAT_MTE2, setting the system register like this may lead to UNSPECIFIED behavior. Fix it by arranging for MAIR to be set in the C functioncpu_enable_mtewhich is called based on the sanitized version of the system register.
Quoted 7 lines
There is no need for the rest of the MTE-related system register initialization to happen from assembly, with the exception of
TCR_EL1, which must be set to include at least TBI1 because the secondary CPUs access KASan-allocated data structures early. Therefore, make theTCR_EL1initialization unconditional and move the rest of the initialization tocpu_enable_mteso that we no longer have a dependency on the unsanitized ID register value.
Moving the register initialization to C also fixes a bug where the kernel's zeroing of TFSR_EL1 has no practical effect when the kernel is started in VHE mode because the register is currently being zeroed prior to the kernel enabling the redirect of TFSR_EL2 to TFSR_EL1 when it enables VHE. As a result, without this patch it is possible to get a spurious KASAN error report if TFSR_EL2 is non-zero out of reset.
Peter
Catalin Marinas
maintainerQuoted 17 lines · Peter Collingbourne
Quoted 8 lines · Evgenii Stepanov
If
FEAT_MTE2is disabled via the arm64.nomte command line argument on a CPU that claims to supportFEAT_MTE2, the kernel will use Tagged Normal in the MAIR. If we interpret arm64.nomte to mean that the CPU does not in fact implementFEAT_MTE2, setting the system register like this may lead to UNSPECIFIED behavior. Fix it by arranging for MAIR to be set in the C functioncpu_enable_mtewhich is called based on the sanitized version of the system register.Quoted 7 lines
There is no need for the rest of the MTE-related system register initialization to happen from assembly, with the exception of
TCR_EL1, which must be set to include at least TBI1 because the secondary CPUs access KASan-allocated data structures early. Therefore, make theTCR_EL1initialization unconditional and move the rest of the initialization tocpu_enable_mteso that we no longer have a dependency on the unsanitized ID register value.
Quoted 6 lines
Moving the register initialization to C also fixes a bug where the kernel's zeroing of
TFSR_EL1has no practical effect when the kernel is started in VHE mode because the register is currently being zeroed prior to the kernel enabling the redirect ofTFSR_EL2toTFSR_EL1when it enables VHE. As a result, without this patch it is possible to get a spurious KASAN error report ifTFSR_EL2is non-zero out of reset.
Oh, I think this is a side-effect of the nVHE patches. We added MTE in 5.10 and __cpu_setup() was called at EL2 if the kernel was entered at EL2 - 3b714d24ef17 ("arm64: mte: CPU feature detection and initial sysreg configuration"). When nVHE turned up in 5.12, this was changed to to run __cpu_setup at EL1 and this only initialises TFSR_EL1. __finalise_el2 should have transferred TFSR_EL12.
I don't think there other registers we missed in __cpu_setup() but I haven't looked in detail.
So for this, we either move the reg initialisation to C or we fix __finalise_el2. I'm tempted to go with the former as long as the kernel doesn't read that register up to that point and complain of a spurious asynchronous fault.
Signature · 1 line
Catalin
Quoted 33 lines · Catalin Marinas
Quoted 17 lines · Peter Collingbourne
Quoted 8 lines · Evgenii Stepanov
If
FEAT_MTE2is disabled via the arm64.nomte command line argument on a CPU that claims to supportFEAT_MTE2, the kernel will use Tagged Normal in the MAIR. If we interpret arm64.nomte to mean that the CPU does not in fact implementFEAT_MTE2, setting the system register like this may lead to UNSPECIFIED behavior. Fix it by arranging for MAIR to be set in the C functioncpu_enable_mtewhich is called based on the sanitized version of the system register.Quoted 7 lines
There is no need for the rest of the MTE-related system register initialization to happen from assembly, with the exception of
TCR_EL1, which must be set to include at least TBI1 because the secondary CPUs access KASan-allocated data structures early. Therefore, make theTCR_EL1initialization unconditional and move the rest of the initialization tocpu_enable_mteso that we no longer have a dependency on the unsanitized ID register value.Quoted 6 lines
Moving the register initialization to C also fixes a bug where the kernel's zeroing of
TFSR_EL1has no practical effect when the kernel is started in VHE mode because the register is currently being zeroed prior to the kernel enabling the redirect ofTFSR_EL2toTFSR_EL1when it enables VHE. As a result, without this patch it is possible to get a spurious KASAN error report ifTFSR_EL2is non-zero out of reset.Oh, I think this is a side-effect of the nVHE patches. We added MTE in 5.10 and
__cpu_setup()was called at EL2 if the kernel was entered at EL2 -3b714d24ef17("arm64: mte: CPU feature detection and initial sysreg configuration"). When nVHE turned up in 5.12, this was changed to to run__cpu_setupat EL1 and this only initialisesTFSR_EL1.__finalise_el2should have transferredTFSR_EL12.
Quoted 7 lines
I don't think there other registers we missed in
__cpu_setup()but I haven't looked in detail.So for this, we either move the reg initialisation to C or we fix
__finalise_el2. I'm tempted to go with the former as long as the kernel doesn't read that register up to that point and complain of a spurious asynchronous fault.
Yes, I don't think we can generally expect the kernel to read the register up to that point. The register is generally only read in response to an exception, and the early kernel initialization code runs with interrupts disabled and only enables them well after the call to smp_prepare_boot_cpu() for the primary CPU (see the calls to local_irq_disable() and local_irq_enable() in start_kernel()) and check_local_cpu_capabilities() for the secondaries (see the call to local_daif_restore() in secondary_start_kernel()), so unless something has gone seriously wrong that would cause us to get a synchronous exception that early (in which case the spurious async fault report is the least of our worries) I don't think we should expect to see one of these reports as a result of moving the initialization code to C.
Peter
Catalin Marinas
maintainerQuoted 16 lines · Evgenii Stepanov
If
FEAT_MTE2is disabled via the arm64.nomte command line argument on a CPU that claims to supportFEAT_MTE2, the kernel will use Tagged Normal in the MAIR. If we interpret arm64.nomte to mean that the CPU does not in fact implementFEAT_MTE2, setting the system register like this may lead to UNSPECIFIED behavior. Fix it by arranging for MAIR to be set in the C functioncpu_enable_mtewhich is called based on the sanitized version of the system register.There is no need for the rest of the MTE-related system register initialization to happen from assembly, with the exception of
TCR_EL1, which must be set to include at least TBI1 because the secondary CPUs access KASan-allocated data structures early. Therefore, make theTCR_EL1initialization unconditional and move the rest of the initialization tocpu_enable_mteso that we no longer have a dependency on the unsanitized ID register value.
Quoted 2 lines
Signed-off-by Peter Collingbourne <[email protected]>Signed-off-by Evgenii Stepanov <[email protected]>
Does this need a Co-developed-by from Peter? The sign-off on its own doesn't make sense unless Peter sent the patch.
If we want this as a fix (though we have an alternative to transfer mair_el12 to mair_el2), I'd also add:
together with some comment what it fixes.
arch/arm64/kernel/cpufeature.c static void bti_enable(const struct arm64_cpu_capabilities *__unused) · line 2036
Quoted 8 lines
arch/arm64/kernel/cpufeature.c +0
-0@@ -2034,7 +2034,8 @@ static void bti_enable(const struct arm64_cpu_capabilities *__unused)static void cpu_enable_mte (struct arm64_cpu_capabilities const *cap) {sysreg_clear_set (sctlr_el1,0 , SCTLR_ELx_ATA |SCTLR_EL1_ATA0 );
I wonder if we need to keep this here on its own rather than moving it to mte_cpu_setup(). I guess we avoid writing this register the third time on the resume path.
arch/arm64/kernel/cpufeature.c static void bti_enable(const struct arm64_cpu_capabilities *__unused) · line 2039
Quoted 4 lines
arch/arm64/kernel/cpufeature.c +2
-1-isb (); + +mte_cpu_setup ();
So in principle I'm fine with moving the initialisation to C. However, a concern I have is that we may no longer spot potential SoC issues that trigger SErrors on tag accesses. The architecture does allow any Normal Cacheable mapping to fetch tags, even if marked as untagged, though most current implementations won't do this. In addition, if you run this in a virtualised environment, a guest without this patch (or a malicious one) can simply enable tagging in MAIR_EL1. HCR_EL2.ATA==0 does not have any effect, so you haven't solved the problem.
So if part of RAM in your SoC does not support MTE, I don't think you should rely on any kernel command-line options, just make sure no untrusted guest can access memory that may trigger SErrors even if MTE is disabled and the control registers level. Current behaviour (without this patch) comes in handy to detect such issues.
Signature · 1 line
Catalin