Rework READ_ONCE() to improve codegen
Subsystems (16)
Will Deacon
maintainerv3 cover Rework READ_ONCE() to improve codegen
Hi everyone,
This is version three of the patches I previously posted for improving the code generation of READ_ONCE() and moving the minimum GCC version to 4.8:
RFC: https://lore.kernel.org/lkml/[email protected] v2: https://lore.kernel.org/lkml/[email protected]
Although v2 was queued up by Peter in -tip, it was found to break the build for m68k and sparc32. We fixed m68k during the merge window and I've since posted patches to fix sparc32 here:
https://lore.kernel.org/lkml/[email protected]
This series is a refresh on top of 5.7-rc1, the main changes being:
* Fix another issue where 'const' is assigned to non-const via WRITE_ONCE(), this time in the tls code
* Fix READ_ONCE_NOCHECK() abuse in arm64 checksum code
* Added Reviewed-bys and Acks from v2
Hopefully this can be considered for 5.8, along with the sparc32 changes.
Cheers,
Will
--->8
Will Deacon (12):
compiler/gcc: Emit build-time warning for GCC prior to version 4.8
netfilter: Avoid assigning 'const' pointer to non-const pointer
net: tls: Avoid assigning 'const' pointer to non-const pointer
fault_inject: Don't rely on "return value" from WRITE_ONCE()
arm64: csum: Disable KASAN for do_csum()
READ_ONCE: Simplify implementations of {READ,WRITE}_ONCE()
READ_ONCE: Enforce atomicity for {READ,WRITE}_ONCE() memory accesses
READ_ONCE: Drop pointer qualifiers when reading from scalar types locking/barriers: Use '__unqual_scalar_typeof' for load-acquire macros
arm64: barrier: Use '__unqual_scalar_typeof' for acquire/release
macros
compiler/gcc: Raise minimum GCC version for kernel builds to 4.8
gcov: Remove old GCC 3.4 support
Documentation/process/changes.rst | 2 +- arch/arm/crypto/Kconfig | 12 +- arch/arm64/include/asm/barrier.h | 16 +- arch/arm64/lib/csum.c | 20 +- crypto/Kconfig | 1 - drivers/xen/time.c | 2 +- include/asm-generic/barrier.h | 16 +- include/linux/compiler-gcc.h | 5 +- include/linux/compiler.h | 129 +++---- include/linux/compiler_types.h | 21 ++ init/Kconfig | 5 +- kernel/gcov/Kconfig | 24 -- kernel/gcov/Makefile | 3 +- kernel/gcov/gcc_3_4.c | 573 ------------------------------ lib/fault-inject.c | 4 +- net/netfilter/core.c | 2 +- net/tls/tls_main.c | 2 +- scripts/Kconfig.include | 6 + scripts/gcc-plugins/Kconfig | 2 +- 19 files changed, 126 insertions(+), 719 deletions(-) delete mode 100644 kernel/gcov/gcc_3_4.c
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 1/12 compiler/gcc: Emit build-time warning for GCC prior to version 4.8
Prior to version 4.8, GCC may miscompile READ_ONCE() by erroneously discarding the 'volatile' qualifier:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145
We've been working around this using some nasty hacks which make READ_ONCE() both horribly complicated and also prevent us from enforcing that it is only used on scalar types. Since GCC 4.8 is pretty old for kernel builds now, emit a warning if we detect it during the build.
--- init/Kconfig | 4 ++-- scripts/Kconfig.include | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-)
init/Kconfig +2 -2
@@ -9,11 +9,11 @@ config DEFCONFIG_LIST default "arch/$(SRCARCH)/configs/$(KBUILD_DEFCONFIG)" config CC_IS_GCC - def_bool $(success,$(CC) --version | head -n 1 | grep -q gcc) + def_bool $(cc-is-gcc) config GCC_VERSION int - default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC + default $(gcc-version) if CC_IS_GCC default 0 config LD_VERSION
scripts/Kconfig.include +9 -0
@@ -54,3 +54,12 @@ $(error-if,$(success, $(LD) -v | grep -q gold), gold linker '$(LD)' not supporte cc-option-bit = $(if-success,$(CC) -Werror $(1) -E -x c /dev/null -o /dev/null,$(1)) m32-flag := $(cc-option-bit,-m32) m64-flag := $(cc-option-bit,-m64) + +# gcc version including patch level +gcc-version := $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) + +# Return y if the compiler is GCC, n otherwise +cc-is-gcc := $(success,$(CC) --version | head -n 1 | grep -q gcc) + +# Warn if the compiler is GCC prior to 4.8 +$(warning-if,$(if-success,[ $(gcc-version) -lt 40800 ],$(cc-is-gcc),n),"Your compiler is old and may miscompile the kernel due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145 - please upgrade it.")
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Re: [PATCH v3 01/12] compiler/gcc: Emit build-time warning for GCC prior to version 4.8
Quoted 10 lines · Will Deacon
Prior to version 4.8, GCC may miscompile
READ_ONCE()by erroneously discarding the 'volatile' qualifier:https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145
We've been working around this using some nasty hacks which make
READ_ONCE()both horribly complicated and also prevent us from enforcing that it is only used on scalar types. Since GCC 4.8 is pretty old for kernel builds now, emit a warning if we detect it during the build.
This patch is unneeded since you will remove GCC 4.8 support in 11/12.
Please move 11/12 to the head of this series,
and remove all noise changes.
Thanks.
Quoted 50 lines
Suggested-by Linus Torvalds <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Michael Ellerman <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Masahiro Yamada <[email protected]>Signed-off-by Will Deacon <[email protected]>--- init/Kconfig | 4 ++-- scripts/Kconfig.include | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-)init/Kconfig +2
-2@@ -9,11 +9,11 @@ config DEFCONFIG_LIST default "arch/$(SRCARCH)/configs/$(KBUILD_DEFCONFIG)" config CC_IS_GCC - def_bool $(success,$(CC) --version | head -n 1 | grep -q gcc) + def_bool $(cc-is-gcc) config GCC_VERSION int - default $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) if CC_IS_GCC + default $(gcc-version) if CC_IS_GCC default 0 config LD_VERSIONscripts/Kconfig.include +9
-1@@ -54,3 +54,12 @@ $(error-if,$(success, $(LD) -v | grep -q gold), gold linker '$(LD)' not supporte cc-option-bit = $(if-success,$(CC) -Werror $(1) -E -x c /dev/null -o /dev/null,$(1)) m32-flag := $(cc-option-bit,-m32) m64-flag := $(cc-option-bit,-m64) + +# gcc version including patch level +gcc-version := $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) + +# Return y if the compiler is GCC, n otherwise +cc-is-gcc := $(success,$(CC) --version | head -n 1 | grep -q gcc) + +# Warn if the compiler is GCC prior to 4.8 +$(warning-if,$(if-success,[ $(gcc-version) -lt 40800 ],$(cc-is-gcc),n),"Your compiler is old and may miscompile the kernel due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145 - please upgrade it.") --2.26.0.110.g2183baf09c-goog
Signature · 2 lines
Best Regards Masahiro Yamada
Will Deacon
maintainerPATCH v3 2/12 netfilter: Avoid assigning 'const' pointer to non-const pointer
In reply to Will Deaconnf_remove_net_hook() uses WRITE_ONCE() to assign a 'const' pointer to a 'non-const' pointer. Cleanups to the implementation of WRITE_ONCE() mean that this will give rise to a compiler warning, just like a plain old assignment would do:
| In file included from ./include/linux/export.h:43, | from ./include/linux/linkage.h:7, | from ./include/linux/kernel.h:8, | from net/netfilter/core.c:9: | net/netfilter/core.c: In function ‘nf_remove_net_hook’: | ./include/linux/compiler.h:216:30: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers] | *(volatile typeof(x) *)&(x) = (val); \ | ^ | net/netfilter/core.c:379:3: note: in expansion of macro ‘WRITE_ONCE’ | WRITE_ONCE(orig_ops[i], &dummy_ops); | ^~~~~~~~~~
Follow the pattern used elsewhere in this file and add a cast to 'void *' to squash the warning.
--- net/netfilter/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/netfilter/core.c +1 -1
@@ -376,7 +376,7 @@ static bool nf_remove_net_hook(struct nf_hook_entries *old,if (orig_ops[i] != unreg)continue ;WRITE_ONCE (old->hooks[i].hook, accept_all); -WRITE_ONCE (orig_ops[i], &dummy_ops); +WRITE_ONCE (orig_ops[i], (void *)&dummy_ops);return true; }
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 3/12 net: tls: Avoid assigning 'const' pointer to non-const pointer
In reply to Will Deacontls_build_proto() uses WRITE_ONCE() to assign a 'const' pointer to a 'non-const' pointer. Cleanups to the implementation of WRITE_ONCE() mean that this will give rise to a compiler warning, just like a plain old assignment would do:
| net/tls/tls_main.c: In function ‘tls_build_proto’: | ./include/linux/compiler.h:229:30: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers] | net/tls/tls_main.c:640:4: note: in expansion of macro ‘smp_store_release’ | 640 | smp_store_release(&saved_tcpv6_prot, prot); | | ^~~~~~~~~~~~~~~~~
Drop the const qualifier from the local 'prot' variable, as it isn't needed.
--- net/tls/tls_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/tls/tls_main.c +1 -1
@@ -629,7 +629,7 @@ struct tls_context *tls_ctx_create(struct sock *sk)static void tls_build_proto (struct sock *sk) {int ip_ver = sk->sk_family ==AF_INET6 ?TLSV6 :TLSV4 ; -const struct proto *prot =READ_ONCE (sk->sk_prot); +struct proto *prot =READ_ONCE (sk->sk_prot);/* Build IPv6 TLS whenever the address of tcpv6 _prot changes */ if (ip_ver == TLSV6 &&
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 4/12 fault_inject: Don't rely on "return value" from WRITE_ONCE()
In reply to Will DeaconIt's a bit weird that WRITE_ONCE() evaluates to the value it stores and it's different to smp_store_release(), which can't be used this way.
In preparation for preventing this in WRITE_ONCE(), change the fault injection code to use a local variable instead.
--- lib/fault-inject.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
lib/fault-inject.c +3 -1
@@ -106,7 +106,9 @@ bool should_fail(struct fault_attr *attr, ssize_t size)unsigned int fail_nth =READ_ONCE (current->fail_nth);if (fail_nth) { -if (!WRITE_ONCE (current->fail_nth, fail_nth -1 )) + fail_nth--; +WRITE_ONCE (current->fail_nth, fail_nth); +if (!fail_nth) goto fail;return false;
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 5/12 arm64: csum: Disable KASAN for do_csum()
In reply to Will Deacondo_csum() over-reads the source buffer and therefore abuses READ_ONCE_NOCHECK() to avoid tripping up KASAN. In preparation for READ_ONCE_NOCHECK() becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotate do_csum() explicitly and fall back to normal loads.
--- arch/arm64/lib/csum.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
arch/arm64/lib/csum.c +12 -8
@@ -14,7 +14,11 @@ static u64 accumulate(u64 sum, u64 data)return tmp + (tmp >>64 ); } -unsigned int do_csum (const unsigned char *buff,int len) +/* +* We over-read the buffer and this makes KASAN unhappy. Instead, disable +* instrumentation and call kasan explicitly. +*/ +unsigned int __no_sanitize_address do_csum(const unsigned char *buff, int len) {unsigned int offset, shift, sum;const u64 *ptr;
@@ -42,7 +46,7 @@ unsigned int do_csum(const unsigned char *buff, int len)* odd/even alignment, and means we can ignore it until the very end. */ shift = offset *8 ; - data =READ_ONCE_NOCHECK (*ptr++); + data = *ptr++;#ifdef __LITTLE_ENDIAN data = (data >> shift) << shift;#else
@@ -58,10 +62,10 @@ unsigned int do_csum(const unsigned char *buff, int len)while (unlikely (len >64 )) {__uint128_t tmp1, tmp2, tmp3, tmp4; - tmp1 =READ_ONCE_NOCHECK (*(__uint128_t *)ptr); - tmp2 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +2 )); - tmp3 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +4 )); - tmp4 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +6 )); + tmp1 = *(__uint128_t *)ptr; + tmp2 = *(__uint128_t *)(ptr +2 ); + tmp3 = *(__uint128_t *)(ptr +4 ); + tmp4 = *(__uint128_t *)(ptr +6 ); len -=64 ; ptr +=8 ;
@@ -85,7 +89,7 @@ unsigned int do_csum(const unsigned char *buff, int len)__uint128_t tmp; sum64 =accumulate (sum64, data); - tmp =READ_ONCE_NOCHECK (*(__uint128_t *)ptr); + tmp = *(__uint128_t *)ptr; len -=16 ; ptr +=2 ;
@@ -100,7 +104,7 @@ unsigned int do_csum(const unsigned char *buff, int len) }if (len >0 ) { sum64 =accumulate (sum64, data); - data =READ_ONCE_NOCHECK (*ptr); + data = *ptr; len -=8 ; }/*
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Mark Rutland
reviewerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
Hi Will,
Quoted 5 lines · Will Deacon
do_csum()over-reads the source buffer and therefore abusesREAD_ONCE_NOCHECK()to avoid tripping up KASAN. In preparation forREAD_ONCE_NOCHECK()becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotatedo_csum()explicitly and fall back to normal loads.
I'm confused by this. The whole point of READ_ONCE_NOCHECK() is that it isn't checked by KASAN, so if that semantic is removed it has no reason to exist.
Changing that will break the unwind/stacktrace code across multiple architectures. IIRC they use READ_ONCE_NOCHECK() for two reasons:
1. Races with concurrent modification, as might happen when a thread's stack is corrupted. Allowing the unwinder to bail out after a sanity check means the resulting report is more useful than a KASAN splat in the unwinder. I made the arm64 unwinder robust to this case.
2. I believe that the frame record itself /might/ be poisoned by KASAN, since it's not meant to be an accessible object at the C langauge level. I could be wrong about this, and would have to check.
I would like to keep the unwinding robust in the first case, even if the second case doesn't apply, and I'd prefer to not mark the entirety of the unwinding code as unchecked as that's sufficiently large an subtle that it could have nasty bugs.
Is there any way we keep something like READ_ONCE_NOCHECK() around even if we have to give it reduced functionality relative to READ_ONCE()?
I'm not enirely sure why READ_ONCE_NOCHECK() had to go, so if there's a particular pain point I'm happy to take a look.
Thanks,
Mark.
Quoted 70 lines
Cc Mark Rutland <[email protected]>Cc Robin Murphy <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/lib/csum.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)arch/arm64/lib/csum.c +12
-8@@ -14,7 +14,11 @@ static u64 accumulate(u64 sum, u64 data)return tmp + (tmp >>64 ); } -unsigned int do_csum (const unsigned char *buff,int len) +/* +* We over-read the buffer and this makes KASAN unhappy. Instead, disable +* instrumentation and call kasan explicitly. +*/ +unsigned int __no_sanitize_address do_csum(const unsigned char *buff, int len) {unsigned int offset, shift, sum;const u64 *ptr;@@ -42,7 +46,7 @@ unsigned int do_csum(const unsigned char *buff, int len)* odd/even alignment, and means we can ignore it until the very end. */ shift = offset *8 ; - data =READ_ONCE_NOCHECK (*ptr++); + data = *ptr++;#ifdef __LITTLE_ENDIAN data = (data >> shift) << shift;#else @@ -58,10 +62,10 @@ unsigned int do_csum(const unsigned char *buff, int len)while (unlikely (len >64 )) {__uint128_t tmp1, tmp2, tmp3, tmp4; - tmp1 =READ_ONCE_NOCHECK (*(__uint128_t *)ptr); - tmp2 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +2 )); - tmp3 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +4 )); - tmp4 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +6 )); + tmp1 = *(__uint128_t *)ptr; + tmp2 = *(__uint128_t *)(ptr +2 ); + tmp3 = *(__uint128_t *)(ptr +4 ); + tmp4 = *(__uint128_t *)(ptr +6 ); len -=64 ; ptr +=8 ;@@ -85,7 +89,7 @@ unsigned int do_csum(const unsigned char *buff, int len)__uint128_t tmp; sum64 =accumulate (sum64, data); - tmp =READ_ONCE_NOCHECK (*(__uint128_t *)ptr); + tmp = *(__uint128_t *)ptr; len -=16 ; ptr +=2 ;@@ -100,7 +104,7 @@ unsigned int do_csum(const unsigned char *buff, int len) }if (len >0 ) { sum64 =accumulate (sum64, data); - data =READ_ONCE_NOCHECK (*ptr); + data = *ptr; len -=8 ; }/* Signature · 1 line
2.26.0.110.g2183baf09c-goog
Arnd Bergmann
maintainerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
Quoted 17 lines · Mark Rutland
Hi Will,
Quoted 5 lines · Will Deacon
do_csum()over-reads the source buffer and therefore abusesREAD_ONCE_NOCHECK()to avoid tripping up KASAN. In preparation forREAD_ONCE_NOCHECK()becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotatedo_csum()explicitly and fall back to normal loads.I'm confused by this. The whole point of
READ_ONCE_NOCHECK()is that it isn't checked by KASAN, so if that semantic is removed it has no reason to exist.Changing that will break the unwind/stacktrace code across multiple architectures. IIRC they use
READ_ONCE_NOCHECK()for two reasons:
Quoted 8 lines
1. Races with concurrent modification, as might happen when a thread's stack is corrupted. Allowing the unwinder to bail out after a sanity check means the resulting report is more useful than a KASAN splat in the unwinder. I made the arm64 unwinder robust to this case.
2. I believe that the frame record itself /might/ be poisoned by KASAN, since it's not meant to be an accessible object at the C langauge level. I could be wrong about this, and would have to check.
I thought the main reason was deadlocks when a READ_ONCE() is called inside of code that is part of the KASAN handling. If READ_ONCE() ends up recursively calling itself, the kernel tends to crash once it overflows its stack.
Quoted 10 lines
I would like to keep the unwinding robust in the first case, even if the second case doesn't apply, and I'd prefer to not mark the entirety of the unwinding code as unchecked as that's sufficiently large an subtle that it could have nasty bugs.
Is there any way we keep something like
READ_ONCE_NOCHECK()around even if we have to give it reduced functionality relative toREAD_ONCE()?I'm not enirely sure why
READ_ONCE_NOCHECK()had to go, so if there's a particular pain point I'm happy to take a look.
As I understood, only this particular instance was removed, not all of them.
Arnd
Will Deacon
maintainerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
Quoted 20 lines · Arnd Bergmann
Quoted 19 lines · Mark Rutland
Quoted 5 lines · Will Deacon
do_csum()over-reads the source buffer and therefore abusesREAD_ONCE_NOCHECK()to avoid tripping up KASAN. In preparation forREAD_ONCE_NOCHECK()becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotatedo_csum()explicitly and fall back to normal loads.I'm confused by this. The whole point of
READ_ONCE_NOCHECK()is that it isn't checked by KASAN, so if that semantic is removed it has no reason to exist.Changing that will break the unwind/stacktrace code across multiple architectures. IIRC they use
READ_ONCE_NOCHECK()for two reasons:1. Races with concurrent modification, as might happen when a thread's stack is corrupted. Allowing the unwinder to bail out after a sanity check means the resulting report is more useful than a KASAN splat in the unwinder. I made the arm64 unwinder robust to this case.
Quoted 8 lines
Quoted 3 lines
2. I believe that the frame record itself /might/ be poisoned by KASAN, since it's not meant to be an accessible object at the C langauge level. I could be wrong about this, and would have to check.
I thought the main reason was deadlocks when a
READ_ONCE()is called inside of code that is part of the KASAN handling. IfREAD_ONCE()ends up recursively calling itself, the kernel tends to crash once it overflows its stack.
That was also my understanding.
Quoted 5 lines
Quoted 5 lines
I would like to keep the unwinding robust in the first case, even if the second case doesn't apply, and I'd prefer to not mark the entirety of the unwinding code as unchecked as that's sufficiently large an subtle that it could have nasty bugs.
Quoted 8 lines
Quoted 5 lines
Is there any way we keep something like
READ_ONCE_NOCHECK()around even if we have to give it reduced functionality relative toREAD_ONCE()?I'm not enirely sure why
READ_ONCE_NOCHECK()had to go, so if there's a particular pain point I'm happy to take a look.As I understood, only this particular instance was removed, not all of them.
Right, but the problem is that whether the NOCHECK version gets checked or not now depends on the caller, since it's all just a macro. If we want to fix this, then we could force the nocheck variant to return unsigned long, which simplifies things a lot (completely untested):
#define READ_ONCE(x) \
({ \
compiletime_assert_rwonce_type(x); \
__READ_ONCE_SCALAR(x); \
})unsigned long __no_sanitise_address kasan_nocheck_read_once_ul(const volatile void *p)
{
return READ_ONCE(*p);
}/* Please don't use this */
#define READ_ONCE_NOCHECK(x) kasan_nocheck_read_once_ul(&x)
which would make sense for the unwinders, where there is concurrency involved, but I'd be inclined to have them call kasan_nocheck_read_once_ul() directly and ditch READ_ONCE_NOCHECK() so that it doesn't get used for single-threaded code as a convenience to avoid annotation.
What do you think?
Will
Will Deacon
maintainerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
Quoted 58 lines · Will Deacon
Quoted 20 lines · Arnd Bergmann
Quoted 19 lines · Mark Rutland
Quoted 5 lines · Will Deacon
do_csum()over-reads the source buffer and therefore abusesREAD_ONCE_NOCHECK()to avoid tripping up KASAN. In preparation forREAD_ONCE_NOCHECK()becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotatedo_csum()explicitly and fall back to normal loads.I'm confused by this. The whole point of
READ_ONCE_NOCHECK()is that it isn't checked by KASAN, so if that semantic is removed it has no reason to exist.Changing that will break the unwind/stacktrace code across multiple architectures. IIRC they use
READ_ONCE_NOCHECK()for two reasons:1. Races with concurrent modification, as might happen when a thread's stack is corrupted. Allowing the unwinder to bail out after a sanity check means the resulting report is more useful than a KASAN splat in the unwinder. I made the arm64 unwinder robust to this case.
Quoted 8 lines
Quoted 3 lines
2. I believe that the frame record itself /might/ be poisoned by KASAN, since it's not meant to be an accessible object at the C langauge level. I could be wrong about this, and would have to check.
I thought the main reason was deadlocks when a
READ_ONCE()is called inside of code that is part of the KASAN handling. IfREAD_ONCE()ends up recursively calling itself, the kernel tends to crash once it overflows its stack.That was also my understanding.
Quoted 5 lines
Quoted 5 lines
I would like to keep the unwinding robust in the first case, even if the second case doesn't apply, and I'd prefer to not mark the entirety of the unwinding code as unchecked as that's sufficiently large an subtle that it could have nasty bugs.
Quoted 8 lines
Quoted 5 lines
Is there any way we keep something like
READ_ONCE_NOCHECK()around even if we have to give it reduced functionality relative toREAD_ONCE()?I'm not enirely sure why
READ_ONCE_NOCHECK()had to go, so if there's a particular pain point I'm happy to take a look.As I understood, only this particular instance was removed, not all of them.
Right, but the problem is that whether the NOCHECK version gets checked or not now depends on the caller, since it's all just a macro. If we want to fix this, then we could force the nocheck variant to return unsigned long, which simplifies things a lot (completely untested):
#define
READ_ONCE(x)\({ \ compiletime_assert_rwonce_type(x); \ __READ_ONCE_SCALAR(x); \ })
Quoted 9 lines
unsigned long
__no_sanitise_addresskasan_nocheck_read_once_ul(const volatile void *p){ return READ_ONCE(*p); }/* Please don't use this */
#defineREAD_ONCE_NOCHECK(x)kasan_nocheck_read_once_ul(&x)
Urgh, scratch that. Trying to instantiate READ_ONCE() in compiler.h causes a circular header-file dependency between linux/compiler.h and asm-generic/barrier.h thanks to smp_read_barrier_depends().
Time to dust off that patch I had splitting up compiler.h.
Will
Will Deacon
maintainerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
In reply to Mark RutlandQuoted 10 lines · Mark Rutland
Quoted 5 lines · Will Deacon
do_csum()over-reads the source buffer and therefore abusesREAD_ONCE_NOCHECK()to avoid tripping up KASAN. In preparation forREAD_ONCE_NOCHECK()becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotatedo_csum()explicitly and fall back to normal loads.I'm confused by this. The whole point of
READ_ONCE_NOCHECK()is that it isn't checked by KASAN, so if that semantic is removed it has no reason to exist.
Oh, I thought it was there to be used by things like KASAN itself and because READ_ONCE() was implemented using a static function, then that function had to be marked as __no_sanitize_address when used in these cases. Now that it's just a macro, that's not necessary so it's just the same as normal READ_ONCE().
Do we have a "nocheck" version where we don't require the READ_ONCE() semantics? I think abusing a relaxed concurrency primitive for this is not the right thing to do, particularly when the __no_sanitize_address annotation is available. I fact, it's almost an argument in favour of removing READ_ONCE_NOCHECK() so that people use the annotation instead!
Quoted 11 lines
Changing that will break the unwind/stacktrace code across multiple architectures. IIRC they use
READ_ONCE_NOCHECK()for two reasons:1. Races with concurrent modification, as might happen when a thread's stack is corrupted. Allowing the unwinder to bail out after a sanity check means the resulting report is more useful than a KASAN splat in the unwinder. I made the arm64 unwinder robust to this case.
2. I believe that the frame record itself /might/ be poisoned by KASAN, since it's not meant to be an accessible object at the C langauge level. I could be wrong about this, and would have to check.
Ok.
Quoted 4 lines
I would like to keep the unwinding robust in the first case, even if the second case doesn't apply, and I'd prefer to not mark the entirety of the unwinding code as unchecked as that's sufficiently large an subtle that it could have nasty bugs.
Hmm, maybe. I don't really see what's wrong with annotating the unwinding code, though. You can still tell kasan about the accesses you're making, like we do in the checksumming code here, and it's not hard to move the frame-pointer chasing code into a separate function.
Quoted 5 lines
Is there any way we keep something like
READ_ONCE_NOCHECK()around even if we have to give it reduced functionality relative toREAD_ONCE()?I'm not enirely sure why
READ_ONCE_NOCHECK()had to go, so if there's a particular pain point I'm happy to take a look.
I got rid if it because I thought it wasn't required now that it's implemented entirely as a macro. I'd be reluctant to bring it back if there isn't a non-ONCE version of the helper.
Will
Mark Rutland
reviewerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
Quoted 12 lines · Will Deacon
Quoted 10 lines · Mark Rutland
Quoted 5 lines · Will Deacon
do_csum()over-reads the source buffer and therefore abusesREAD_ONCE_NOCHECK()to avoid tripping up KASAN. In preparation forREAD_ONCE_NOCHECK()becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotatedo_csum()explicitly and fall back to normal loads.I'm confused by this. The whole point of
READ_ONCE_NOCHECK()is that it isn't checked by KASAN, so if that semantic is removed it has no reason to exist.
Quoted 5 lines
Oh, I thought it was there to be used by things like KASAN itself and because
READ_ONCE()was implemented using a static function, then that function had to be marked as__no_sanitize_addresswhen used in these cases. Now that it's just a macro, that's not necessary so it's just the same as normalREAD_ONCE().
I believe that the KASAN core files are compiled without instrumentation, so they can use either without issue.
Quoted 2 lines
Do we have a "nocheck" version where we don't require the
READ_ONCE()semantics?
For the unwind code we rely on the ONCE semantic (but arguably don't need single-copy-atomicity) so that we operate on a consistent snapshot.
Quoted 4 lines
I think abusing a relaxed concurrency primitive for this is not the right thing to do, particularly when the
__no_sanitize_addressannotation is available. I fact, it's almost an argument in favour of removingREAD_ONCE_NOCHECK()so that people use the annotation instead!
Arguably we *are* using it as a relaxed concurrency primitive, to get a snapshot of a varaible undergoing concurrent modification.
FWIW, for the arm64 unwind code we could add a helper to snapshot the frame record, and mark that as __no_sanitize_address, e.g.
/*
* Get a snapshot of a frame record that might be undergoing concurrent
* modification (and hence we must also avoid a KASAN splat). */
static __no_sanitize_address snapshot_frame(struct stackframe *frame,
unsigned long fp)
{
frame->fp = READ_ONCE(*(unsigned long *)(fp));
frame->pc = READ_ONCE(*(unsigned long *)(fp + 8));
}... we'd need to do likewied in a few bits of unwind code:
arch/s390/kernel/unwind_bc.c: READ_ONCE_NOCHECK(regs->psw.mask) & PSW_MASK_PSTATE;arch/s390/kernel/unwind_bc.c: ip = READ_ONCE_NOCHECK(sf->gprs[8]);arch/s390/kernel/unwind_bc.c: sp = READ_ONCE_NOCHECK(sf->back_chain);arch/s390/kernel/unwind_bc.c: ip = READ_ONCE_NOCHECK(sf->gprs[8]);arch/s390/kernel/unwind_bc.c: ip = READ_ONCE_NOCHECK(regs->psw.addr);arch/s390/kernel/unwind_bc.c: sp = READ_ONCE_NOCHECK(regs->gprs[15]);arch/s390/kernel/unwind_bc.c: ip = READ_ONCE_NOCHECK(sf->gprs[8]);arch/x86/include/asm/atomic.h: * Note for KASAN: we deliberately don't use READ_ONCE_NOCHECK() here,arch/x86/include/asm/unwind.h: val = READ_ONCE_NOCHECK(x); \arch/x86/kernel/dumpstack.c: unsigned long addr = READ_ONCE_NOCHECK(*stack);arch/x86/kernel/process.c: fp = READ_ONCE_NOCHECK(((struct inactive_task_frame *)sp)->bp);arch/x86/kernel/process.c: ip = READ_ONCE_NOCHECK(*(unsigned long *)(fp + sizeof(unsigned long)));arch/x86/kernel/process.c: fp = READ_ONCE_NOCHECK(*(unsigned long *)fp);arch/x86/kernel/unwind_frame.c: word = READ_ONCE_NOCHECK(*sp);arch/x86/kernel/unwind_guess.c: addr = READ_ONCE_NOCHECK(*state->sp);arch/x86/kernel/unwind_guess.c: unsigned long addr = READ_ONCE_NOCHECK(*state->sp);arch/x86/kernel/unwind_orc.c: *val = READ_ONCE_NOCHECK(*(unsigned long *)addr);arch/x86/kernel/unwind_orc.c: state->bp = READ_ONCE_NOCHECK(frame->bp);arch/x86/kernel/unwind_orc.c: state->ip = READ_ONCE_NOCHECK(frame->ret_addr);include/linux/compiler.h: * Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need include/linux/compiler.h:#define READ_ONCE_NOCHECK(x) __READ_ONCE(x, 0)kernel/trace/trace_stack.c: * The READ_ONCE_NOCHECK is used to let KASAN know thatkernel/trace/trace_stack.c: if ((READ_ONCE_NOCHECK(*p)) == stack_dump_trace[i]) {
Quoted 9 lines
Quoted 4 lines
I would like to keep the unwinding robust in the first case, even if the second case doesn't apply, and I'd prefer to not mark the entirety of the unwinding code as unchecked as that's sufficiently large an subtle that it could have nasty bugs.
Hmm, maybe. I don't really see what's wrong with annotating the unwinding code, though. You can still tell kasan about the accesses you're making, like we do in the checksumming code here, and it's not hard to move the frame-pointer chasing code into a separate function.
Sure; agreed as above. We just need to fix up a number of places.
Quoted 9 lines
Quoted 5 lines
Is there any way we keep something like
READ_ONCE_NOCHECK()around even if we have to give it reduced functionality relative toREAD_ONCE()?I'm not enirely sure why
READ_ONCE_NOCHECK()had to go, so if there's a particular pain point I'm happy to take a look.I got rid if it because I thought it wasn't required now that it's implemented entirely as a macro. I'd be reluctant to bring it back if there isn't a non-ONCE version of the helper.
As above, I think that we *do* care about the ONCE-ness for the unwind code, but I'm happy for those to be dealt with by special helpers.
Thanks,
Mark.
Will Deacon
maintainerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
Quoted 8 lines · Mark Rutland
Quoted 4 lines · Will Deacon
I think abusing a relaxed concurrency primitive for this is not the right thing to do, particularly when the
__no_sanitize_addressannotation is available. I fact, it's almost an argument in favour of removingREAD_ONCE_NOCHECK()so that people use the annotation instead!Arguably we *are* using it as a relaxed concurrency primitive, to get a snapshot of a varaible undergoing concurrent modification.
That's fair, so it's only the checksum code that was abusing this, which I've fixed.
Quoted 31 lines
FWIW, for the arm64 unwind code we could add a helper to snapshot the frame record, and mark that as
__no_sanitize_address, e.g./*
* Get a snapshot of a frame record that might be undergoing concurrent
* modification (and hence we must also avoid a KASAN splat). */
static__no_sanitize_addresssnapshot_frame(struct stackframe *frame,
unsigned long fp){ frame->fp = READ_ONCE(*(unsigned long *)(fp)); frame->pc = READ_ONCE(*(unsigned long *)(fp + 8)); }... we'd need to do likewied in a few bits of unwind code:
arch/s390/kernel/unwind_bc.c:READ_ONCE_NOCHECK(regs->psw.mask)&PSW_MASK_PSTATE;arch/s390/kernel/unwind_bc.c: ip =READ_ONCE_NOCHECK(sf->gprs[8]);arch/s390/kernel/unwind_bc.c: sp =READ_ONCE_NOCHECK(sf->back_chain);arch/s390/kernel/unwind_bc.c: ip =READ_ONCE_NOCHECK(sf->gprs[8]);arch/s390/kernel/unwind_bc.c: ip =READ_ONCE_NOCHECK(regs->psw.addr);arch/s390/kernel/unwind_bc.c: sp =READ_ONCE_NOCHECK(regs->gprs[15]);arch/s390/kernel/unwind_bc.c: ip =READ_ONCE_NOCHECK(sf->gprs[8]);arch/x86/include/asm/atomic.h: * Note for KASAN: we deliberately don't useREAD_ONCE_NOCHECK()here,arch/x86/include/asm/unwind.h: val =READ_ONCE_NOCHECK(x); \arch/x86/kernel/dumpstack.c: unsigned long addr =READ_ONCE_NOCHECK(*stack);arch/x86/kernel/process.c: fp =READ_ONCE_NOCHECK(((struct inactive_task_frame *)sp)->bp);arch/x86/kernel/process.c: ip =READ_ONCE_NOCHECK(*(unsigned long *)(fp + sizeof(unsigned long)));arch/x86/kernel/process.c: fp =READ_ONCE_NOCHECK(*(unsigned long *)fp);arch/x86/kernel/unwind_frame.c: word =READ_ONCE_NOCHECK(*sp);arch/x86/kernel/unwind_guess.c: addr =READ_ONCE_NOCHECK(*state->sp);
Quoted 8 lines
arch/x86/kernel/unwind_guess.c: unsigned long addr =READ_ONCE_NOCHECK(*state->sp);arch/x86/kernel/unwind_orc.c: *val =READ_ONCE_NOCHECK(*(unsigned long *)addr);arch/x86/kernel/unwind_orc.c:state->bp=READ_ONCE_NOCHECK(frame->bp);arch/x86/kernel/unwind_orc.c:state->ip=READ_ONCE_NOCHECK(frame->ret_addr);include/linux/compiler.h: * UseREAD_ONCE_NOCHECK()instead ofREAD_ONCE()if you need include/linux/compiler.h:#defineREAD_ONCE_NOCHECK(x)__READ_ONCE(x, 0)kernel/trace/trace_stack.c: * TheREAD_ONCE_NOCHECKis used to let KASAN know thatkernel/trace/trace_stack.c: if ((READ_ONCE_NOCHECK(*p)) == stack_dump_trace[i]) {
Indeed. For now, I'm going to keep this simple with the change below, but I'll revisit this later on because I have another series removing smp_read_barrier_depends() which makes this a lot simpler.
Will
--->8
include/linux/compiler.h +19 -7
@@ -212,18 +212,12 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, (typeof (x ))__x; \ }) -/* -* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need -* to hide memory access from KASAN. -*/ -#define READ_ONCE_NOCHECK (x) \ +#define READ_ONCE (x) \ ({ \ compiletime_assert_rwonce_type(x); \ __READ_ONCE_SCALAR(x); \ }) -#define READ_ONCE (x) READ_ONCE_NOCHECK(x) -#define __WRITE_ONCE (x, val) \ do { \ *(volatile typeof(x) *)&(x) = (val); \
@@ -247,6 +241,24 @@ do { \# define __no_kasan_or_inline __always_inline#endif +static __no_kasan_or_inline +unsigned long __read_once_word_nocheck (const void *addr) +{ +return __READ_ONCE (*(unsigned long *)addr); +} + +/* +* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need to load a +* word from memory atomically but without telling KASAN. This is usually +* used by unwinding code when walking the stack of a running process. +*/ +#define READ_ONCE_NOCHECK (x) \ +({ \ + unsigned long __x = __read_once_word_nocheck(&(x)); \ + smp_read_barrier_depends(); \ + __x; \ +}) +static __no_kasan_or_inline unsigned long read_word_at_a_time (const void *addr) {
Mark Rutland
reviewerRe: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
Quoted 3 lines · Will Deacon
Quoted 2 lines · Mark Rutland
FWIW, for the arm64 unwind code we could add a helper to snapshot the frame record, and mark that as
__no_sanitize_address, e.g.
[...]
Quoted 1 line
Quoted 1 line
... we'd need to do likewied in a few bits of unwind code:
[...]
Quoted 5 lines
Indeed. For now, I'm going to keep this simple with the change below, but I'll revisit this later on because I have another series removing
smp_read_barrier_depends()which makes this a lot simpler.Will
The below looks good to me; thanks for putting that together!
Mark.
Quoted 52 lines
--->8
include/linux/compiler.h +19
-7@@ -212,18 +212,12 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, (typeof (x ))__x; \ }) -/* -* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need -* to hide memory access from KASAN. -*/ -#define READ_ONCE_NOCHECK (x) \ +#define READ_ONCE (x) \ ({ \ compiletime_assert_rwonce_type(x); \ __READ_ONCE_SCALAR(x); \ }) -#define READ_ONCE (x) READ_ONCE_NOCHECK(x) -#define __WRITE_ONCE (x, val) \ do { \ *(volatile typeof(x) *)&(x) = (val); \@@ -247,6 +241,24 @@ do { \# define __no_kasan_or_inline __always_inline#endif +static __no_kasan_or_inline +unsigned long __read_once_word_nocheck (const void *addr) +{ +return __READ_ONCE (*(unsigned long *)addr); +} + +/* +* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need to load a +* word from memory atomically but without telling KASAN. This is usually +* used by unwinding code when walking the stack of a running process. +*/ +#define READ_ONCE_NOCHECK (x) \ +({ \ + unsigned long __x = __read_once_word_nocheck(&(x)); \ + smp_read_barrier_depends(); \ + __x; \ +}) +static __no_kasan_or_inline unsigned long read_word_at_a_time (const void *addr) {
Re: [PATCH v3 05/12] arm64: csum: Disable KASAN for do_csum()
In reply to Will DeaconQuoted 5 lines · Will Deacon
do_csum()over-reads the source buffer and therefore abusesREAD_ONCE_NOCHECK()to avoid tripping up KASAN. In preparation forREAD_ONCE_NOCHECK()becoming a macro, and therefore losing its '__no_sanitize_address' annotation, just annotatedo_csum()explicitly and fall back to normal loads.
FWIW with most compilers I played with, the read-once-ness *was* also important to ensure the uint128_t accesses compose to nice efficient LDPs rather than being split into a motley mess of individual LDRs.
The buffer loads that aren't the first or potentially the last didn't strictly need to be nocheck, however since the whole range gets explicitly checked up-front they may as well avoid further unnecessary KASAN penalty, plus it made things look nice and consistent :)
Robin.
Quoted 67 lines
Cc Mark Rutland <[email protected]>Cc Robin Murphy <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/lib/csum.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)arch/arm64/lib/csum.c +12
-8@@ -14,7 +14,11 @@ static u64 accumulate(u64 sum, u64 data)return tmp + (tmp >>64 ); } -unsigned int do_csum (const unsigned char *buff,int len) +/* +* We over-read the buffer and this makes KASAN unhappy. Instead, disable +* instrumentation and call kasan explicitly. +*/ +unsigned int __no_sanitize_address do_csum(const unsigned char *buff, int len) {unsigned int offset, shift, sum;const u64 *ptr;@@ -42,7 +46,7 @@ unsigned int do_csum(const unsigned char *buff, int len)* odd/even alignment, and means we can ignore it until the very end. */ shift = offset *8 ; - data =READ_ONCE_NOCHECK (*ptr++); + data = *ptr++;#ifdef __LITTLE_ENDIAN data = (data >> shift) << shift;#else @@ -58,10 +62,10 @@ unsigned int do_csum(const unsigned char *buff, int len)while (unlikely (len >64 )) {__uint128_t tmp1, tmp2, tmp3, tmp4; - tmp1 =READ_ONCE_NOCHECK (*(__uint128_t *)ptr); - tmp2 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +2 )); - tmp3 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +4 )); - tmp4 =READ_ONCE_NOCHECK (*(__uint128_t *)(ptr +6 )); + tmp1 = *(__uint128_t *)ptr; + tmp2 = *(__uint128_t *)(ptr +2 ); + tmp3 = *(__uint128_t *)(ptr +4 ); + tmp4 = *(__uint128_t *)(ptr +6 ); len -=64 ; ptr +=8 ;@@ -85,7 +89,7 @@ unsigned int do_csum(const unsigned char *buff, int len)__uint128_t tmp; sum64 =accumulate (sum64, data); - tmp =READ_ONCE_NOCHECK (*(__uint128_t *)ptr); + tmp = *(__uint128_t *)ptr; len -=16 ; ptr +=2 ;@@ -100,7 +104,7 @@ unsigned int do_csum(const unsigned char *buff, int len) }if (len >0 ) { sum64 =accumulate (sum64, data); - data =READ_ONCE_NOCHECK (*ptr); + data = *ptr; len -=8 ; }/*
Will Deacon
maintainerPATCH v3 6/12 READ_ONCE: Simplify implementations of {READ,WRITE}_ONCE()
In reply to Will DeaconThe implementations of {READ,WRITE}_ONCE() suffer from a significant amount of indirection and complexity due to a historic GCC bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145
which was originally worked around by 230fa253df63 ("kernel: Provide READ_ONCE and ASSIGN_ONCE").
Since GCC 4.8 is fairly vintage at this point and we emit a warning if we detect it during the build, return {READ,WRITE}_ONCE() to their former glory with an implementation that is easier to understand and, crucially, more amenable to optimisation. A side effect of this simplification is that WRITE_ONCE() no longer returns a value, but nobody seems to be relying on that and the new behaviour is aligned with smp_store_release().
--- include/linux/compiler.h | 104 ++++++++++----------------------------- 1 file changed, 25 insertions(+), 79 deletions(-)
include/linux/compiler.h +25 -79
@@ -177,60 +177,6 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,# define __UNIQUE_ID (prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __LINE__)#endif -#include <uapi/linux/types.h> - -#define __READ_ONCE_SIZE \ -({ \ - switch (size) { \ - case 1: *(__u8 *)res = *(volatile __u8 *)p; break; \ - case 2: *(__u16 *)res = *(volatile __u16 *)p; break; \ - case 4: *(__u32 *)res = *(volatile __u32 *)p; break; \ - case 8: *(__u64 *)res = *(volatile __u64 *)p; break; \ - default: \ - barrier(); \ - __builtin_memcpy((void *)res, (const void *)p, size); \ - barrier(); \ - } \ -}) - -static __always_inline -void __read_once_size(const volatile void *p, void *res, int size) -{ - __READ_ONCE_SIZE; -} - -#ifdef CONFIG_KASAN -/* -* We can't declare function 'inline' because __no_sanitize_address confilcts -* with inlining. Attempt to inline it may cause a build failure. -* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 -* '__maybe_unused' allows us to avoid defined-but-not-used warnings. -*/ -# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused -#else -# define __no_kasan_or_inline __always_inline -#endif - -static __no_kasan_or_inline -void __read_once_size_nocheck(const volatile void *p, void *res, int size) -{ - __READ_ONCE_SIZE; -} - -static __always_inline void __write_once_size(volatile void *p, void *res, int size) -{ -switch (size) { -case 1 : *(volatile __u8 *)p = *(__u8 *)res;break ; -case 2 : *(volatile __u16 *)p = *(__u16 *)res;break ; -case 4 : *(volatile __u32 *)p = *(__u32 *)res;break ; -case 8 : *(volatile __u64 *)p = *(__u64 *)res;break ; -default : -barrier (); -__builtin_memcpy ((void *)p, (const void *)res, size); -barrier (); - } -} -/* * Prevent the compiler from merging or refetching reads or writes. The * compiler is also forbidden from reordering successive instances of
@@ -240,11 +186,7 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s* statements. * * These two macros will also work on aggregate data types like structs or -* unions. If the size of the accessed data type exceeds the word size of -* the machine (e.g., 32 bits or 64 bits) READ_ONCE() and WRITE_ONCE() will -* fall back to memcpy(). There's at least two memcpy()s: one for the -* __builtin_memcpy() and then one for the macro doing the copy of variable -* - '__u' allocated on the stack. +* unions. * * Their two major use cases are: (1) Mediating communication between * process-level code and irq/NMI handlers, all running on the same CPU,
@@ -256,23 +198,35 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s#include <asm/barrier.h> #include <linux/kasan-checks.h> -#define __READ_ONCE (x, check) \ +/* +* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need +* to hide memory access from KASAN. +*/ +#define READ_ONCE_NOCHECK (x) \ ({ \ - union { typeof(x) __val; char __c[1]; } __u; \ - if (check) \ - __read_once_size(&(x), __u.__c, sizeof(x)); \ - else \ - __read_once_size_nocheck(&(x), __u.__c, sizeof(x)); \ - smp_read_barrier_depends();/* Enforce dependency ordering from x */ \ - __u.__val; \ + typeof(x) __x = *(volatile typeof(x) *)&(x); \ + smp_read_barrier_depends(); \ + __x; \ }) -#define READ_ONCE (x) __READ_ONCE(x, 1) +#define READ_ONCE (x) READ_ONCE_NOCHECK(x) + +#define WRITE_ONCE (x, val) \ +do { \ + *(volatile typeof(x) *)&(x) = (val); \ +} while (0) + +#ifdef CONFIG_KASAN /* -* Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need -* to hide memory access from KASAN. +* We can't declare function 'inline' because __no_sanitize_address conflicts +* with inlining. Attempt to inline it may cause a build failure. +* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 +* '__maybe_unused' allows us to avoid defined-but-not-used warnings. */ -#define READ_ONCE_NOCHECK (x) __READ_ONCE(x, 0) +# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused +#else +# define __no_kasan_or_inline __always_inline +#endif static __no_kasan_or_inline unsigned long read_word_at_a_time (const void *addr)
@@ -281,14 +235,6 @@ unsigned long read_word_at_a_time(const void *addr) return *(unsigned long *)addr; } -#define WRITE_ONCE (x, val) \ -({ \ - union { typeof(x) __val; char __c[1]; } __u = \ - { .__val = (__force typeof(x)) (val) }; \ - __write_once_size(&(x), __u.__c, sizeof(x)); \ - __u.__val; \ -}) -#endif /* __KERNEL__ */ /*
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 7/12 READ_ONCE: Enforce atomicity for {READ,WRITE}_ONCE() memory accesses
In reply to Will Deacon{READ,WRITE}_ONCE() cannot guarantee atomicity for arbitrary data sizes. This can be surprising to callers that might incorrectly be expecting atomicity for accesses to aggregate structures, although there are other callers where tearing is actually permissable (e.g. if they are using something akin to sequence locking to protect the access).
Linus sayeth:
| We could also look at being stricter for the normal READ/WRITE_ONCE(), | and require that they are | | (a) regular integer types | | (b) fit in an atomic word | | We actually did (b) for a while, until we noticed that we do it on | loff_t's etc and relaxed the rules. But maybe we could have a | "non-atomic" version of READ/WRITE_ONCE() that is used for the | questionable cases?
The slight snag is that we also have to support 64-bit accesses on 32-bit architectures, as these appear to be widespread and tend to work out ok if either the architecture supports atomic 64-bit accesses (x86, armv7) or if the variable being accesses represents a virtual address and therefore only requires 32-bit atomicity in practice.
Take a step in that direction by introducing a variant of 'compiletime_assert_atomic_type()' and use it to check the pointer argument to {READ,WRITE}_ONCE(). Expose __{READ,WRITE_ONCE}() variants which are allowed to tear and convert the one broken caller over to the new macros.
--- drivers/xen/time.c | 2 +- include/linux/compiler.h | 37 +++++++++++++++++++++++++++++++++---- 2 files changed, 34 insertions(+), 5 deletions(-)
drivers/xen/time.c +1 -1
@@ -64,7 +64,7 @@ static void xen_get_runstate_snapshot_cpu_delta(do { state_time =get64 (&state->state_entry_time);rmb ();/* Hypervisor might update data. */ - *res =READ_ONCE (*state); + *res =__READ_ONCE (*state);rmb ();/* Hypervisor might update data. */ }while (get64 (&state->state_entry_time) != state_time || (state_time &XEN_RUNSTATE_UPDATE ));
include/linux/compiler.h +33 -4
@@ -198,24 +198,43 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,#include <asm/barrier.h> #include <linux/kasan-checks.h> +/* +* Use __READ_ONCE() instead of READ_ONCE() if you do not require any +* atomicity or dependency ordering guarantees. Note that this may result +* in tears! +*/ +#define __READ_ONCE (x) (*(const volatile typeof(x) *)&(x)) + +#define __READ_ONCE_SCALAR (x) \ +({ \ + typeof(x) __x = __READ_ONCE(x); \ + smp_read_barrier_depends(); \ + __x; \ +}) +/* * Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need * to hide memory access from KASAN. */ #define READ_ONCE_NOCHECK (x) \ ({ \ - typeof(x) __x = *(volatile typeof(x) *)&(x); \ - smp_read_barrier_depends(); \ - __x; \ + compiletime_assert_rwonce_type(x); \ + __READ_ONCE_SCALAR(x); \ })#define READ_ONCE (x) READ_ONCE_NOCHECK(x) -#define WRITE_ONCE (x, val) \ +#define __WRITE_ONCE (x, val) \ do { \ *(volatile typeof(x) *)&(x) = (val); \ } while (0) +#define WRITE_ONCE (x, val) \ +do { \ + compiletime_assert_rwonce_type(x); \ + __WRITE_ONCE(x, val); \ +} while (0) +#ifdef CONFIG_KASAN /* * We can't declare function 'inline' because __no_sanitize_address conflicts
@@ -299,6 +318,16 @@ static inline void *offset_to_ptr(const int *off)compiletime_assert (__native_word (t), \"Need native word sized stores/loads for atomicity." ) +/* +* Yes, this permits 64-bit accesses on 32-bit architectures. These will +* actually be atomic in many cases (namely x86), but for others we rely on +* the access being split into 2x32-bit accesses for a 32-bit quantity (e.g. +* a virtual address) and a strong prevailing wind. + */ +#define compiletime_assert_rwonce_type (t) \ + compiletime_assert(__native_word(t) || sizeof(t) == sizeof(long long), \ + "Unsupported access size for {READ,WRITE}_ONCE().") +/* &a[0] degrades to a pointer: a different type from an array */ #define __must_be_array (a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 8/12 READ_ONCE: Drop pointer qualifiers when reading from scalar types
In reply to Will DeaconPassing a volatile-qualified pointer to READ_ONCE() is an absolute trainwreck for code generation: the use of 'typeof()' to define a temporary variable inside the macro means that the final evaluation in macro scope ends up forcing a read back from the stack. When stack protector is enabled (the default for arm64, at least), this causes the compiler to vomit up all sorts of junk.
Unfortunately, dropping pointer qualifiers inside the macro poses quite a challenge, especially since the pointed-to type is permitted to be an aggregate, and this is relied upon by mm/ code accessing things like 'pmd_t'. Based on numerous hacks and discussions on the mailing list, this is the best I've managed to come up with.
Introduce '__unqual_scalar_typeof()' which takes an expression and, if the expression is an optionally qualified 8, 16, 32 or 64-bit scalar type, evaluates to the unqualified type. Other input types, including aggregates, remain unchanged. Hopefully READ_ONCE() on volatile aggregate pointers isn't something we do on a fast-path.
--- include/linux/compiler.h | 6 +++--- include/linux/compiler_types.h | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+), 3 deletions(-)
include/linux/compiler.h +3 -3
@@ -203,13 +203,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,* atomicity or dependency ordering guarantees. Note that this may result * in tears! */ -#define __READ_ONCE (x) (*(const volatile typeof(x) *)&(x)) +#define __READ_ONCE (x) (*(const volatile __unqual_scalar_typeof(x) *)&(x))#define __READ_ONCE_SCALAR (x) \ ({ \ - typeof(x) __x = __READ_ONCE(x); \ + __unqual_scalar_typeof(x) __x = __READ_ONCE(x); \ smp_read_barrier_depends(); \ - __x; \ + (typeof(x))__x; \ })/*
include/linux/compiler_types.h +21 -0
@@ -210,6 +210,27 @@ struct ftrace_likely_data {/* Are two types/vars the same type (ignoring qualifiers)? */ #define __same_type (a, b) __builtin_types_compatible_p(typeof(a), typeof(b)) +/* +* __unqual_scalar_typeof(x) - Declare an unqualified scalar type, leaving +* non-scalar types unchanged. +* +* We build this out of a couple of helper macros in a vain attempt to +* help you keep your lunch down while reading it. +*/ +#define __pick_scalar_type (x, type, otherwise) \ + __builtin_choose_expr(__same_type(x, type), (type)0, otherwise) + +#define __pick_integer_type (x, type, otherwise) \ + __pick_scalar_type(x, unsigned type, \ + __pick_scalar_type(x, signed type, otherwise)) + +#define __unqual_scalar_typeof (x) typeof( \ + __pick_integer_type(x, char, \ + __pick_integer_type(x, short, \ + __pick_integer_type(x, int, \ + __pick_integer_type(x, long, \ + __pick_integer_type(x, long long, x)))))) +/* Is this type a native word size -- useful for atomic operations */ #define __native_word (t) \ (sizeof(t) == sizeof(char) || sizeof(t) == sizeof(short) || \
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 9/12 locking/barriers: Use '__unqual_scalar_typeof' for load-acquire macros
In reply to Will DeaconPassing volatile-qualified pointers to the asm-generic implementations of the load-acquire macros results in a re-load from the stack due to the temporary result variable inheriting the volatile semantics thanks to the use of 'typeof()'.
Define these temporary variables using 'unqual_scalar_typeof' to drop the volatile qualifier in the case that they are scalar types.
--- include/asm-generic/barrier.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
include/asm-generic/barrier.h +8 -8
@@ -128,10 +128,10 @@ do { \#ifndef __smp_load_acquire#define __smp_load_acquire (p) \ ({ \ - typeof(*p) ___p1 = READ_ONCE(*p); \ + __unqual_scalar_typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ __smp_mb(); \ - ___p1; \ + (typeof(*p))___p1; \ })#endif
@@ -183,10 +183,10 @@ do { \#ifndef smp_load_acquire#define smp_load_acquire (p) \ ({ \ - typeof(*p) ___p1 = READ_ONCE(*p); \ + __unqual_scalar_typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ barrier(); \ - ___p1; \ + (typeof(*p))___p1; \ })#endif
@@ -229,14 +229,14 @@ do { \#ifndef smp_cond_load_relaxed#define smp_cond_load_relaxed (ptr, cond_expr) ({ \ typeof(ptr) __PTR = (ptr); \ - typeof(*ptr) VAL; \ + __unqual_scalar_typeof(*ptr) VAL; \ for (;;) { \ VAL = READ_ONCE(*__PTR); \ if (cond_expr) \ break; \ cpu_relax(); \ } \ - VAL; \ + (typeof(*ptr))VAL; \ })#endif
@@ -250,10 +250,10 @@ do { \*/ #ifndef smp_cond_load_acquire#define smp_cond_load_acquire (ptr, cond_expr) ({ \ - typeof(*ptr) _val; \ + __unqual_scalar_typeof(*ptr) _val; \ _val = smp_cond_load_relaxed(ptr, cond_expr); \ smp_acquire__after_ctrl_dep(); \ - _val; \ + (typeof(*ptr))_val; \ })#endif
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 10/12 arm64: barrier: Use '__unqual_scalar_typeof' for acquire/release macros
In reply to Will DeaconPassing volatile-qualified pointers to the arm64 implementations of the load-acquire/store-release macros results in a re-load from the stack and a bunch of associated stack-protector churn due to the temporary result variable inheriting the volatile semantics thanks to the use of 'typeof()'.
Define these temporary variables using 'unqual_scalar_typeof' to drop the volatile qualifier in the case that they are scalar types.
--- arch/arm64/include/asm/barrier.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
arch/arm64/include/asm/barrier.h +8 -8
@@ -76,8 +76,8 @@ static inline unsigned long array_index_mask_nospec(unsigned long idx,#define __smp_store_release (p, v) \ do { \ typeof(p) __p = (p); \ - union { typeof(*p) __val; char __c[1]; } __u = \ - { .__val = (__force typeof(*p)) (v) }; \ + union { __unqual_scalar_typeof(*p) __val; char __c[1]; } __u = \ + { .__val = (__force __unqual_scalar_typeof(*p)) (v) }; \ compiletime_assert_atomic_type(*p); \ kasan_check_write(__p, sizeof(*p)); \ switch (sizeof(*p)) { \
@@ -110,7 +110,7 @@ do { \#define __smp_load_acquire (p) \ ({ \ - union { typeof(*p) __val; char __c[1]; } __u; \ + union { __unqual_scalar_typeof(*p) __val; char __c[1]; } __u; \ typeof(p) __p = (p); \ compiletime_assert_atomic_type(*p); \ kasan_check_read(__p, sizeof(*p)); \
@@ -136,33 +136,33 @@ do { \ : "Q" (*__p) : "memory"); \break ; \ } \ - __u.__val; \ + (typeof(*p))__u.__val; \ })#define smp_cond_load_relaxed (ptr, cond_expr) \ ({ \ typeof(ptr) __PTR = (ptr); \ - typeof(*ptr) VAL; \ + __unqual_scalar_typeof(*ptr) VAL; \ for (;;) { \ VAL = READ_ONCE(*__PTR); \ if (cond_expr) \ break; \ __cmpwait_relaxed(__PTR, VAL); \ } \ - VAL; \ + (typeof(*ptr))VAL; \ })#define smp_cond_load_acquire (ptr, cond_expr) \ ({ \ typeof(ptr) __PTR = (ptr); \ - typeof(*ptr) VAL; \ + __unqual_scalar_typeof(*ptr) VAL; \ for (;;) { \ VAL = smp_load_acquire(__PTR); \ if (cond_expr) \ break; \ __cmpwait_relaxed(__PTR, VAL); \ } \ - VAL; \ + (typeof(*ptr))VAL; \ })#include <asm-generic/barrier.h>
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Will Deacon
maintainerPATCH v3 11/12 compiler/gcc: Raise minimum GCC version for kernel builds to 4.8
In reply to Will DeaconIt is very rare to see versions of GCC prior to 4.8 being used to build the mainline kernel. These old compilers are also know to have codegen issues which can lead to silent miscompilation:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145
Raise the minimum GCC version for kernel build to 4.8 and remove some tautological Kconfig dependencies as a consequence.
--- Documentation/process/changes.rst | 2 +- arch/arm/crypto/Kconfig | 12 ++++++------ crypto/Kconfig | 1 - include/linux/compiler-gcc.h | 5 ++--- init/Kconfig | 1 - scripts/Kconfig.include | 3 --- scripts/gcc-plugins/Kconfig | 2 +- 7 files changed, 10 insertions(+), 16 deletions(-)
Documentation/process/changes.rst +1 -1
@@ -29,7 +29,7 @@ you probably needn't concern yourself with pcmciautils. ====================== =============== ======================================== Program Minimal version Command to check the version ====================== =============== ======================================== -GNU C 4.6 gcc --version +GNU C 4.8 gcc --version GNU make 3.81 make --version binutils 2.23 ld -v flex 2.5.35 flex --version
arch/arm/crypto/Kconfig +6 -6
@@ -30,7 +30,7 @@ config CRYPTO_SHA1_ARM_NEON config CRYPTO_SHA1_ARM_CE tristate "SHA1 digest algorithm (ARM v8 Crypto Extensions)" - depends on KERNEL_MODE_NEON && (CC_IS_CLANG || GCC_VERSION >= 40800) + depends on KERNEL_MODE_NEON select CRYPTO_SHA1_ARM select CRYPTO_HASH help
@@ -39,7 +39,7 @@ config CRYPTO_SHA1_ARM_CE config CRYPTO_SHA2_ARM_CE tristate "SHA-224/256 digest algorithm (ARM v8 Crypto Extensions)" - depends on KERNEL_MODE_NEON && (CC_IS_CLANG || GCC_VERSION >= 40800) + depends on KERNEL_MODE_NEON select CRYPTO_SHA256_ARM select CRYPTO_HASH help
@@ -96,7 +96,7 @@ config CRYPTO_AES_ARM_BS config CRYPTO_AES_ARM_CE tristate "Accelerated AES using ARMv8 Crypto Extensions" - depends on KERNEL_MODE_NEON && (CC_IS_CLANG || GCC_VERSION >= 40800) + depends on KERNEL_MODE_NEON select CRYPTO_SKCIPHER select CRYPTO_LIB_AES select CRYPTO_SIMD
@@ -106,7 +106,7 @@ config CRYPTO_AES_ARM_CE config CRYPTO_GHASH_ARM_CE tristate "PMULL-accelerated GHASH using NEON/ARMv8 Crypto Extensions" - depends on KERNEL_MODE_NEON && (CC_IS_CLANG || GCC_VERSION >= 40800) + depends on KERNEL_MODE_NEON select CRYPTO_HASH select CRYPTO_CRYPTD select CRYPTO_GF128MUL
@@ -118,13 +118,13 @@ config CRYPTO_GHASH_ARM_CE config CRYPTO_CRCT10DIF_ARM_CE tristate "CRCT10DIF digest algorithm using PMULL instructions" - depends on KERNEL_MODE_NEON && (CC_IS_CLANG || GCC_VERSION >= 40800) + depends on KERNEL_MODE_NEON depends on CRC_T10DIF select CRYPTO_HASH config CRYPTO_CRC32_ARM_CE tristate "CRC32(C) digest algorithm using CRC and/or PMULL instructions" - depends on KERNEL_MODE_NEON && (CC_IS_CLANG || GCC_VERSION >= 40800) + depends on KERNEL_MODE_NEON depends on CRC32 select CRYPTO_HASH
crypto/Kconfig +0 -1
@@ -316,7 +316,6 @@ config CRYPTO_AEGIS128 config CRYPTO_AEGIS128_SIMD bool "Support SIMD acceleration for AEGIS-128" depends on CRYPTO_AEGIS128 && ((ARM || ARM64) && KERNEL_MODE_NEON) - depends on !ARM || CC_IS_CLANG || GCC_VERSION >= 40800 default y config CRYPTO_AEGIS128_AESNI_SSE2
include/linux/compiler-gcc.h +2 -3
@@ -10,7 +10,8 @@ + __GNUC_MINOR__ * 100 \ + __GNUC_PATCHLEVEL__) -#if GCC_VERSION <40600 +/* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145 */ +#if GCC_VERSION <40800 # error Sorry, your compiler is too old - please upgrade it.#endif
@@ -126,9 +127,7 @@#if defined(CONFIG_ARCH_USE_BUILTIN_BSWAP ) && !defined(__CHECKER__)#define __HAVE_BUILTIN_BSWAP32__#define __HAVE_BUILTIN_BSWAP64__ -#if GCC_VERSION >=40800 #define __HAVE_BUILTIN_BSWAP16__ -#endif #endif /* CONFIG_ARCH_USE_BUILTIN_BSWAP && !__CHECKER__ */ #if GCC_VERSION >=70000
init/Kconfig +0 -1
@@ -1285,7 +1285,6 @@ config LD_DEAD_CODE_DATA_ELIMINATION bool "Dead code and data elimination (EXPERIMENTAL)" depends on HAVE_LD_DEAD_CODE_DATA_ELIMINATION depends on EXPERT - depends on !(FUNCTION_TRACER && CC_IS_GCC && GCC_VERSION < 40800) depends on $(cc-option,-ffunction-sections -fdata-sections) depends on $(ld-option,--gc-sections) help
scripts/Kconfig.include +0 -3
@@ -60,6 +60,3 @@ gcc-version := $(shell,$(srctree)/scripts/gcc-version.sh $(CC)) # Return y if the compiler is GCC, n otherwise cc-is-gcc := $(success,$(CC) --version | head -n 1 | grep -q gcc) - -# Warn if the compiler is GCC prior to 4.8 -$(warning-if,$(if-success,[ $(gcc-version) -lt 40800 ],$(cc-is-gcc),n),"Your compiler is old and may miscompile the kernel due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145 - please upgrade it.")
scripts/gcc-plugins/Kconfig +1 -1
@@ -8,7 +8,7 @@ config HAVE_GCC_PLUGINS menuconfig GCC_PLUGINS bool "GCC plugins" depends on HAVE_GCC_PLUGINS - depends on CC_IS_GCC && GCC_VERSION >= 40800 + depends on CC_IS_GCC depends on $(success,$(srctree)/scripts/gcc-plugin.sh $(CC)) default y help
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Re: [PATCH v3 11/12] compiler/gcc: Raise minimum GCC version for kernel builds to 4.8
Quoted 7 lines · Will Deacon
It is very rare to see versions of GCC prior to 4.8 being used to build the mainline kernel. These old compilers are also know to have codegen issues which can lead to silent miscompilation:
Quoted 7 lines
Raise the minimum GCC version for kernel build to 4.8 and remove some tautological
Kconfigdependencies as a consequence.Cc Arnd Bergmann <[email protected]>Cc Masahiro Yamada <[email protected]>Reviewed-by Nick Desaulniers <[email protected]>Signed-off-by Will Deacon <[email protected]>
On a related note, I've updated the cross-compilers on kernel.org and done some randconfig testing on all major versions from v4.8 to 9.3. There were a couple of minor regressions with 4.8 (mostly harmless warnings), but overall gcc-4.8 is still working well. I did not try older compilers.
Arnd
Will Deacon
maintainerPATCH v3 12/12 gcov: Remove old GCC 3.4 support
In reply to Will DeaconThe kernel requires at least GCC 4.8 in order to build, and so there is no need to cater for the pre-4.7 gcov format.
Remove the obsolete code.
--- kernel/gcov/Kconfig | 24 -- kernel/gcov/Makefile | 3 +- kernel/gcov/gcc_3_4.c | 573 ------------------------------------------ 3 files changed, 1 insertion(+), 599 deletions(-) delete mode 100644 kernel/gcov/gcc_3_4.c
kernel/gcov/Kconfig +0 -24
@@ -51,28 +51,4 @@ config GCOV_PROFILE_ALL larger and run slower. Also be sure to exclude files from profiling which are not linked to the kernel image to prevent linker errors. -choice - prompt "Specify GCOV format" - depends on GCOV_KERNEL - depends on CC_IS_GCC - ---help--- - The gcov format is usually determined by the GCC version, and the - default is chosen according to your GCC version. However, there are - exceptions where format changes are integrated in lower-version GCCs. - In such a case, change this option to adjust the format used in the - kernel accordingly. - -config GCOV_FORMAT_3_4 - bool "GCC 3.4 format" - depends on GCC_VERSION < 40700 - ---help--- - Select this option to use the format defined by GCC 3.4. - -config GCOV_FORMAT_4_7 - bool "GCC 4.7 format" - ---help--- - Select this option to use the format defined by GCC 4.7. - -endchoice - endmenu
kernel/gcov/Makefile +1 -2
@@ -2,6 +2,5 @@ ccflags-y := -DSRCTREE='"$(srctree)"' -DOBJTREE='"$(objtree)"' obj-y := base.o fs.o -obj-$(CONFIG_GCOV_FORMAT_3_4) += gcc_base.o gcc_3_4.o -obj-$(CONFIG_GCOV_FORMAT_4_7) += gcc_base.o gcc_4_7.o +obj-$(CONFIG_CC_IS_GCC) += gcc_base.o gcc_4_7.o obj-$(CONFIG_CC_IS_CLANG) += clang.o
kernel/gcov/gcc_3_4.c deleted +0 -573
573 lines not on this page · raw message
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Quoted 30 lines · Will Deacon
Hi everyone,
This is version three of the patches I previously posted for improving the code generation of
READ_ONCE()and moving the minimum GCC version to 4.8:RFC: https://lore.kernel.org/lkml/[email protected] v2: https://lore.kernel.org/lkml/[email protected]Although v2 was queued up by Peter in -tip, it was found to break the build for m68k and sparc32. We fixed m68k during the merge window and I've since posted patches to fix sparc32 here:
https://lore.kernel.org/lkml/[email protected]
This series is a refresh on top of 5.7-rc1, the main changes being:
* Fix another issue where 'const' is assigned to non-const via
WRITE_ONCE(), this time in the tls code* Fix
READ_ONCE_NOCHECK()abuse in arm64 checksum code* Added Reviewed-bys and Acks from v2
Hopefully this can be considered for 5.8, along with the sparc32 changes.
Cheers,
Will
Quoted 11 lines
Cc Michael Ellerman <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Linus Torvalds <[email protected]>Cc Segher Boessenkool <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Luc Van Oostenryck <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Peter Oberparleiter <[email protected]>Cc Masahiro Yamada <[email protected]>Cc Nick Desaulniers <[email protected]>
I gave this series a try and s390 seems to compile fine and it also seems to properly compile the the ipte_unlock_siif function in arch/s390/kvm/gaccess.c This function was miscompiled with gcc4.6 and the trigger for replacing ACCESS_ONCE with READ_ONCE
Will Deacon
maintainerQuoted 44 lines · Christian Borntraeger
Quoted 30 lines · Will Deacon
Hi everyone,
This is version three of the patches I previously posted for improving the code generation of
READ_ONCE()and moving the minimum GCC version to 4.8:RFC: https://lore.kernel.org/lkml/[email protected] v2: https://lore.kernel.org/lkml/[email protected]Although v2 was queued up by Peter in -tip, it was found to break the build for m68k and sparc32. We fixed m68k during the merge window and I've since posted patches to fix sparc32 here:
https://lore.kernel.org/lkml/[email protected]
This series is a refresh on top of 5.7-rc1, the main changes being:
* Fix another issue where 'const' is assigned to non-const via
WRITE_ONCE(), this time in the tls code* Fix
READ_ONCE_NOCHECK()abuse in arm64 checksum code* Added Reviewed-bys and Acks from v2
Hopefully this can be considered for 5.8, along with the sparc32 changes.
Cheers,
Will
Quoted 11 lines
Cc Michael Ellerman <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Linus Torvalds <[email protected]>Cc Segher Boessenkool <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Luc Van Oostenryck <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Peter Oberparleiter <[email protected]>Cc Masahiro Yamada <[email protected]>Cc Nick Desaulniers <[email protected]>
Quoted 4 lines
I gave this series a try and s390 seems to compile fine and it also seems to properly compile the the
ipte_unlock_siiffunction inarch/s390/kvm/gaccess.cThis function was miscompiled with gcc4.6 and the trigger for replacingACCESS_ONCEwithREAD_ONCE
That's good to hear, thanks Christian!
Will