arm64/simd: Avoid pointless clearing of FP/SIMD buffer
Subsystems (2)
PATCH v2 1/1 arm64/simd: Avoid pointless clearing of FP/SIMD buffer
The buffer provided to kernel_neon_begin() is only used if the task is scheduled out while the FP/SIMD is in use by the kernel, or when such a section is interrupted by a softirq that also uses the FP/SIMD.
IOW, this happens rarely, and even if it happened often, there is still no reason for this buffer to be cleared beforehand, which happens unconditionally, due to the use of a compound literal expression.
So define that buffer variable explicitly, and mark it as __uninitialized so that it will not get cleared, even when -ftrivial-auto-var-init is in effect.
This requires some preprocessor gymnastics, due to the fact that the variable must be defined throughout the entire guarded scope, and the expression
({ struct user_fpsimd_state __uninitialized st; &st; })
is problematic in that regard, even though the compilers seem to permit it. So instead, repeat for 'for ()' trick that is also used in the implementation of the guarded scope helpers.
--- arch/arm64/include/asm/simd.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
arch/arm64/include/asm/simd.h +8 -1
@@ -48,6 +48,13 @@ DEFINE_LOCK_GUARD_1(ksimd, kernel_neon_begin(_T->lock), kernel_neon_end(_T->lock)) -#define scoped_ksimd () scoped_guard(ksimd, &(struct user_fpsimd_state){}) +#define __scoped_ksimd (_label) \ + for (struct user_fpsimd_state __uninitialized __st; \ + true; ({ goto _label; })) \ + if (0) { \ +_label: break; \ + } else scoped_guard(ksimd, &__st) + +#define scoped_ksimd () __scoped_ksimd(__UNIQUE_ID(label))#endif
base-commit: 3f9f0252130e7dd60d41be0802bf58f6471c691d
Signature · 1 line
2.47.3
Eric Biggers
AppliedQuoted 41 lines · Ard Biesheuvel
The buffer provided to
kernel_neon_begin()is only used if the task is scheduled out while the FP/SIMD is in use by the kernel, or when such a section is interrupted by a softirq that also uses the FP/SIMD.IOW, this happens rarely, and even if it happened often, there is still no reason for this buffer to be cleared beforehand, which happens unconditionally, due to the use of a compound literal expression.
So define that buffer variable explicitly, and mark it as
__uninitializedso that it will not get cleared, even when -ftrivial-auto-var-init is in effect.This requires some preprocessor gymnastics, due to the fact that the variable must be defined throughout the entire guarded scope, and the expression
({ struct
user_fpsimd_state__uninitializedst; &st; })is problematic in that regard, even though the compilers seem to permit it. So instead, repeat for 'for ()' trick that is also used in the implementation of the guarded scope helpers.
Cc Will Deacon <[email protected]>,Cc Catalin Marinas <[email protected]>,Cc Kees Cook <[email protected]>Cc Eric Biggers <[email protected]>Signed-off-by Ard Biesheuvel <[email protected]>--- arch/arm64/include/asm/simd.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)arch/arm64/include/asm/simd.h +0
-1@@ -48,6 +48,13 @@ DEFINE_LOCK_GUARD_1(ksimd,kernel_neon_begin (_T->lock), kernel_neon_end(_T->lock)) -#define scoped_ksimd () scoped_guard(ksimd, &(struct user_fpsimd_state){})
arch/arm64/include/asm/simd.h DEFINE_LOCK_GUARD_1(ksimd, · line 60
Quoted 10 lines
arch/arm64/include/asm/simd.h +8
-0+#define __scoped_ksimd (_label) \ + for (struct user_fpsimd_state __uninitialized __st; \ + true; ({ goto _label; })) \ + if (0) { \ +_label: break; \ + } else scoped_guard(ksimd, &__st) + +#define scoped_ksimd () __scoped_ksimd(__UNIQUE_ID(label))#endif
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-fixes
I added:
- Eric