arm64: sme: Use STR P to clear FFR context field in streaming SVE mode
Will Deacon
maintainerPATCH 1/1 arm64: sme: Use STR P to clear FFR context field in streaming SVE mode
The FFR is a predicate register which can vary between 16 and 256 bits in size depending upon the configured vector length. When saving the SVE state in streaming SVE mode, the FFR register is inaccessible and so commit 9f5848665788 ("arm64/sve: Make access to FFR optional") simply clears the FFR field of the in-memory context structure. Unfortunately, it achieves this using an unconditional 8-byte store and so if the SME vector length is anything other than 64 bytes in size we will either fail to clear the entire field or, worse, we will corrupt memory immediately following the structure. This has led to intermittent kfence splats in CI [1] and can trigger kmalloc Redzone corruption messages when running the 'fp-stress' kselftest:
| ============================================================================= | BUG kmalloc-1k (Not tainted): kmalloc Redzone overwritten | ----------------------------------------------------------------------------- | | 0xffff000809bf1e22-0xffff000809bf1e27 @offset=7714. First byte 0x0 instead of 0xcc | Allocated in do_sme_acc+0x9c/0x220 age=2613 cpu=1 pid=531 | __kmalloc+0x8c/0xcc | do_sme_acc+0x9c/0x220 | ...
Replace the 8-byte store with a store of a predicate register which has been zero-initialised with PFALSE, ensuring that the entire field is cleared in memory.
[1] https://lore.kernel.org/r/CA+G9fYtU7HsV0R0dp4XEH5xXHSJFw8KyDf5VQrLLfMxWfxQkag@mail.gmail.com
--- arch/arm64/include/asm/fpsimdmacros.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/arm64/include/asm/fpsimdmacros.h +3 -3
@@ -316,12 +316,12 @@ _for n, 0, 15, _sve_str_p \n, \nxbase, \n - 16 cbz \save_ffr, 921f _sve_rdffr 0 - _sve_str_p 0, \nxbase - _sve_ldr_p 0, \nxbase, -16 b 922f 921: - str xzr, [x\nxbase] // Zero out FFR + _sve_pfalse 0 // Zero out FFR 922: + _sve_str_p 0, \nxbase + _sve_ldr_p 0, \nxbase, -16 mrs x\nxtmp, fpsr str w\nxtmp, [\xpfpsr] mrs x\nxtmp, fpcr
Signature · 7 lines
2.41.0.162.gfafddb0af9-goog _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 11 lines · Will Deacon
The FFR is a predicate register which can vary between 16 and 256 bits in size depending upon the configured vector length. When saving the SVE state in streaming SVE mode, the FFR register is inaccessible and so commit
9f5848665788("arm64/sve: Make access to FFR optional") simply clears the FFR field of the in-memory context structure. Unfortunately, it achieves this using an unconditional 8-byte store and so if the SME vector length is anything other than 64 bytes in size we will either fail to clear the entire field or, worse, we will corrupt memory immediately following the structure. This has led to intermittent kfence splats in CI [1] and can trigger kmalloc Redzone corruption messages when running the 'fp-stress' kselftest:
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 12 lines · Will Deacon
The FFR is a predicate register which can vary between 16 and 256 bits in size depending upon the configured vector length. When saving the SVE state in streaming SVE mode, the FFR register is inaccessible and so commit
9f5848665788("arm64/sve: Make access to FFR optional") simply clears the FFR field of the in-memory context structure. Unfortunately, it achieves this using an unconditional 8-byte store and so if the SME vector length is anything other than 64 bytes in size we will either fail to clear the entire field or, worse, we will corrupt memory immediately following the structure. This has led to intermittent kfence splats in CI [1] and can trigger kmalloc Redzone corruption messages when running the 'fp-stress' kselftest:
Quoted 1 line
[...]
Applied to arm64 (for-next/core), thanks!
[1/1] arm64: sme: Use STR P to clear FFR context field in streaming SVE mode
https://git.kernel.org/arm64/c/1c297ec19245
Signature · 7 lines
Catalin _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 12 lines · Will Deacon
The FFR is a predicate register which can vary between 16 and 256 bits in size depending upon the configured vector length. When saving the SVE state in streaming SVE mode, the FFR register is inaccessible and so commit
9f5848665788("arm64/sve: Make access to FFR optional") simply clears the FFR field of the in-memory context structure. Unfortunately, it achieves this using an unconditional 8-byte store and so if the SME vector length is anything other than 64 bytes in size we will either fail to clear the entire field or, worse, we will corrupt memory immediately following the structure. This has led to intermittent kfence splats in CI [1] and can trigger kmalloc Redzone corruption messages when running the 'fp-stress' kselftest:
I applied your patch on next-20230628 and ran the kselftest-arm64 tests over night...
I've not been able to reproduce "BUG: KFENCE: memory corruption in fpsimd_release_task".
Cheers,
Anders
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel