Kconfig: Bump required compiler version of KASAN and UBSAN
Subsystems (6)
Marco Elver
maintainerPATCH 1/2 Kconfig: Bump required compiler version of KASAN and UBSAN
Adds config variable CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due to no_sanitize functions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.
For example, we require that __always_inline functions used from no_sanitize functions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.
For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
---
Apply after:
https://lkml.kernel.org/r/[email protected]
--- init/Kconfig | 3 +++ lib/Kconfig.kasan | 1 + lib/Kconfig.kcsan | 1 + lib/Kconfig.ubsan | 1 + 4 files changed, 6 insertions(+)
init/Kconfig +3 -0
@@ -39,6 +39,9 @@ config TOOLS_SUPPORT_RELR config CC_HAS_ASM_INLINE def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x c - -c -o /dev/null) +config CC_HAS_WORKING_NOSANITIZE + def_bool !CC_IS_GCC || GCC_VERSION >= 80000 + config CONSTRUCTORS bool depends on !UML
lib/Kconfig.kasan +1 -0
@@ -20,6 +20,7 @@ config KASAN depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \ (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS) depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + depends on CC_HAS_WORKING_NOSANITIZE help Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, designed to find out-of-bounds accesses and use-after-free bugs.
lib/Kconfig.kcsan +1 -0
@@ -5,6 +5,7 @@ config HAVE_ARCH_KCSAN config HAVE_KCSAN_COMPILER def_bool CC_IS_CLANG && $(cc-option,-fsanitize=thread -mllvm -tsan-distinguish-volatile=1) + depends on CC_HAS_WORKING_NOSANITIZE help For the list of compilers that support KCSAN, please see <file:Documentation/dev-tools/kcsan.rst>.
lib/Kconfig.ubsan +1 -0
@@ -4,6 +4,7 @@ config ARCH_HAS_UBSAN_SANITIZE_ALL menuconfig UBSAN bool "Undefined behaviour sanity checker" + depends on CC_HAS_WORKING_NOSANITIZE help This option enables the Undefined Behaviour sanity checker. Compile-time instrumentation is used to detect various undefined
Signature · 1 line
2.27.0.rc2.251.g90737beb825-goog
Marco Elver
maintainerPATCH 2/2 compiler_types.h: Add __no_sanitize_{address,undefined} to noinstr
Adds the portable definitions for __no_sanitize_address, and __no_sanitize_undefined, and subsequently changes noinstr to use the attributes to disable instrumentation via KASAN or UBSAN.
---
Note: __no_sanitize_coverage (for KCOV) isn't possible right now, because neither GCC nor Clang support such an attribute. This means going and changing the compilers again (for Clang it's fine, for GCC, it'll take a while).
However, it looks like that KCOV_INSTRUMENT := n is currently in all the right places. Short-term, this should be reasonable.
--- include/linux/compiler-clang.h | 8 ++++++++ include/linux/compiler-gcc.h | 6 ++++++ include/linux/compiler_types.h | 3 ++- 3 files changed, 16 insertions(+), 1 deletion(-)
include/linux/compiler-clang.h +8 -0
@@ -33,6 +33,14 @@#define __no_sanitize_thread#endif +#if __has_feature (undefined_behavior_sanitizer) +/* GCC does not have __SANITIZE_UNDEFINED__ */ +#define __no_sanitize_undefined \ + __attribute__((no_sanitize("undefined"))) +#else +#define __no_sanitize_undefined +#endif +/* * Not all versions of clang implement the the type-generic versions * of the builtin overflow checkers. Fortunately, clang implements
include/linux/compiler-gcc.h +6 -0
@@ -150,6 +150,12 @@#define __no_sanitize_thread#endif +#if __has_attribute (__no_sanitize_undefined__) +#define __no_sanitize_undefined __attribute__((no_sanitize_undefined)) +#else +#define __no_sanitize_undefined +#endif +#if GCC_VERSION >=50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1#endif
include/linux/compiler_types.h +2 -1
@@ -198,7 +198,8 @@ struct ftrace_likely_data {/* Section for code which can't be instrumented at all */ #define noinstr \ - noinline notrace __attribute((__section__(".noinstr.text"))) __no_kcsan + noinline notrace __attribute((__section__(".noinstr.text"))) \ + __no_kcsan __no_sanitize_address __no_sanitize_undefined#endif /* __KERNEL__ */
Signature · 1 line
2.27.0.rc2.251.g90737beb825-goog
Nick Desaulniers
reviewerRe: [PATCH -tip 2/2] compiler_types.h: Add __no_sanitize_{address,undefined} to noinstr
Quoted 8 lines · Marco Elver' via Clang Built Linux
Adds the portable definitions for
__no_sanitize_address, and__no_sanitize_undefined, and subsequently changes noinstr to use the attributes to disable instrumentation via KASAN or UBSAN.Reported-by [email protected]Signed-off-by Marco Elver <[email protected]>
Currently most of our compiler attribute detection is done in include/linux/compiler_attributes.h; I think this should be handled there. +Miguel Ojeda
Quoted 66 lines
---
Note:
__no_sanitize_coverage(for KCOV) isn't possible right now, because neither GCC nor Clang support such an attribute. This means going and changing the compilers again (for Clang it's fine, for GCC, it'll take a while).However, it looks like that
KCOV_INSTRUMENT:= n is currently in all the right places. Short-term, this should be reasonable.--- include/linux/compiler-clang.h | 8 ++++++++ include/linux/compiler-gcc.h | 6 ++++++ include/linux/compiler_types.h | 3 ++- 3 files changed, 16 insertions(+), 1 deletion(-)include/linux/compiler-clang.h +8
-0@@ -33,6 +33,14 @@#define __no_sanitize_thread#endif +#if __has_feature (undefined_behavior_sanitizer) +/* GCC does not have __SANITIZE_UNDEFINED__ */ +#define __no_sanitize_undefined \ + __attribute__((no_sanitize("undefined"))) +#else +#define __no_sanitize_undefined +#endif +/* * Not all versions of clang implement the the type-generic versions * of the builtin overflow checkers. Fortunately, clang implements include/linux/compiler-gcc.h +6
-0@@ -150,6 +150,12 @@#define __no_sanitize_thread#endif +#if __has_attribute (__no_sanitize_undefined__) +#define __no_sanitize_undefined __attribute__((no_sanitize_undefined)) +#else +#define __no_sanitize_undefined +#endif +#if GCC_VERSION >=50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1#endif include/linux/compiler_types.h +2
-2@@ -198,7 +198,8 @@ struct ftrace_likely_data {/* Section for code which can't be instrumented at all */ #define noinstr \ - noinline notrace __attribute((__section__(".noinstr.text"))) __no_kcsan + noinline notrace __attribute((__section__(".noinstr.text"))) \ + __no_kcsan __no_sanitize_address __no_sanitize_undefined#endif /* __KERNEL__ */ --
Signature · 2 lines
Thanks, ~Nick Desaulniers
Marco Elver
maintainerRe: [PATCH -tip 2/2] compiler_types.h: Add __no_sanitize_{address,undefined} to noinstr
Quoted 8 lines · Nick Desaulniers' via kasan-dev
Quoted 5 lines · Marco Elver' via Clang Built Linux
Adds the portable definitions for
__no_sanitize_address, and__no_sanitize_undefined, and subsequently changes noinstr to use the attributes to disable instrumentation via KASAN or UBSAN.
Quoted 7 lines
Quoted 3 lines
Reported-by [email protected]Signed-off-by Marco Elver <[email protected]>Currently most of our compiler attribute detection is done in
include/linux/compiler_attributes.h; I think this should be handled there. +Miguel Ojeda
GCC and Clang define these very differently, and the way to query for them is different too. All we want is a portable __no_sanitize, and compiler-{gcc,clang}.h is the right place for that. Similar to why we define the other __no_sanitize above the places they were added.
Thanks,
-- Marco
Miguel Ojeda
Acked-byRe: [PATCH -tip 2/2] compiler_types.h: Add __no_sanitize_{address,undefined} to noinstr
In reply to Nick DesaulniersQuoted 4 lines · Nick Desaulniers
Currently most of our compiler attribute detection is done in
include/linux/compiler_attributes.h; I think this should be handled there. +Miguel Ojeda
Thanks a lot for the CC Nick! Marco is right, since this attribute is different per-compiler, we don't want them in compiler_attributes.h (for the moment -- we'll see if they end up with the same syntax/behavior in the future).
Cheers,
Miguel
Re: [PATCH -tip 2/2] compiler_types.h: Add __no_sanitize_{address,undefined} to noinstr
In reply to Marco ElverQuoted 8 lines · Marco Elver
Adds the portable definitions for
__no_sanitize_address, and__no_sanitize_undefined, and subsequently changes noinstr to use the attributes to disable instrumentation via KASAN or UBSAN.Reported-by [email protected]Signed-off-by Marco Elver <[email protected]>
Quoted 68 lines
---
Note:
__no_sanitize_coverage(for KCOV) isn't possible right now, because neither GCC nor Clang support such an attribute. This means going and changing the compilers again (for Clang it's fine, for GCC, it'll take a while).However, it looks like that
KCOV_INSTRUMENT:= n is currently in all the right places. Short-term, this should be reasonable.--- include/linux/compiler-clang.h | 8 ++++++++ include/linux/compiler-gcc.h | 6 ++++++ include/linux/compiler_types.h | 3 ++- 3 files changed, 16 insertions(+), 1 deletion(-)include/linux/compiler-clang.h +8
-0@@ -33,6 +33,14 @@#define __no_sanitize_thread#endif +#if __has_feature (undefined_behavior_sanitizer) +/* GCC does not have __SANITIZE_UNDEFINED__ */ +#define __no_sanitize_undefined \ + __attribute__((no_sanitize("undefined"))) +#else +#define __no_sanitize_undefined +#endif +/* * Not all versions of clang implement the the type-generic versions * of the builtin overflow checkers. Fortunately, clang implements include/linux/compiler-gcc.h +6
-0@@ -150,6 +150,12 @@#define __no_sanitize_thread#endif +#if __has_attribute (__no_sanitize_undefined__) +#define __no_sanitize_undefined __attribute__((no_sanitize_undefined)) +#else +#define __no_sanitize_undefined +#endif +#if GCC_VERSION >=50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1#endif include/linux/compiler_types.h +2
-2@@ -198,7 +198,8 @@ struct ftrace_likely_data {/* Section for code which can't be instrumented at all */ #define noinstr \ - noinline notrace __attribute((__section__(".noinstr.text"))) __no_kcsan + noinline notrace __attribute((__section__(".noinstr.text"))) \ + __no_kcsan __no_sanitize_address __no_sanitize_undefined#endif /* __KERNEL__ */ --2.27.0.rc2.251.g90737beb825-goog
Quoted 9 lines · Marco Elver
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.
Could you also update KASAN docs to mention this requirement? As a separate patch or in v2, up to you.
Quoted 70 lines
For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Marco Elver <[email protected]>---
Apply after:
https://lkml.kernel.org/r/[email protected]--- init/Kconfig | 3 +++ lib/Kconfig.kasan | 1 + lib/Kconfig.kcsan | 1 + lib/Kconfig.ubsan | 1 + 4 files changed, 6 insertions(+)init/Kconfig +3
-0@@ -39,6 +39,9 @@ config TOOLS_SUPPORT_RELR config CC_HAS_ASM_INLINE def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x c - -c -o /dev/null) +config CC_HAS_WORKING_NOSANITIZE + def_bool !CC_IS_GCC || GCC_VERSION >= 80000 + config CONSTRUCTORS bool depends on !UMLlib/Kconfig.kasan +1
-0@@ -20,6 +20,7 @@ config KASAN depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \ (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS) depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + depends on CC_HAS_WORKING_NOSANITIZE help Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, designed to find out-of-bounds accesses and use-after-free bugs.lib/Kconfig.kcsan +1
-0@@ -5,6 +5,7 @@ config HAVE_ARCH_KCSAN config HAVE_KCSAN_COMPILER def_bool CC_IS_CLANG && $(cc-option,-fsanitize=thread -mllvm -tsan-distinguish-volatile=1) + depends on CC_HAS_WORKING_NOSANITIZE help For the list of compilers that support KCSAN, please see <file:Documentation/dev-tools/kcsan.rst>.lib/Kconfig.ubsan +1
-1@@ -4,6 +4,7 @@ config ARCH_HAS_UBSAN_SANITIZE_ALL menuconfig UBSAN bool "Undefined behaviour sanity checker" + depends on CC_HAS_WORKING_NOSANITIZE help This option enables the Undefined Behaviour sanity checker. Compile-time instrumentation is used to detect various undefined --2.27.0.rc2.251.g90737beb825-goog
Marco Elver
maintainerQuoted 8 lines · Andrey Konovalov
Quoted 6 lines · Marco Elver
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.
Quoted 6 lines
Quoted 3 lines
For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.Could you also update KASAN docs to mention this requirement? As a separate patch or in v2, up to you.
I can do a v2 tomorrow. But all this is once again tangled up with KCSAN, so I was hoping to keep changes minimal. ;-)
Thanks,
-- Marco
Quoted 14 lines · Marco Elver
Quoted 12 lines · Andrey Konovalov
Quoted 9 lines · Marco Elver
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.
Quoted 5 lines
Quoted 2 lines
Could you also update KASAN docs to mention this requirement? As a separate patch or in v2, up to you.
I can do a v2 tomorrow. But all this is once again tangled up with KCSAN, so I was hoping to keep changes minimal. ;-)
OK, we can do a separate patch after all this is merged.
Nick Desaulniers
reviewerQuoted 10 lines · Marco Elver' via Clang Built Linux
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.
Quoted 6 lines
For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Marco Elver <[email protected]>
Is this a problem only for x86? If so, that's quite a jump in minimal compiler versions for a feature that I don't think is currently problematic for other architectures? (Based on
https://lore.kernel.org/lkml/[email protected]/
)
Quoted 61 lines
---
Apply after:
https://lkml.kernel.org/r/[email protected]--- init/Kconfig | 3 +++ lib/Kconfig.kasan | 1 + lib/Kconfig.kcsan | 1 + lib/Kconfig.ubsan | 1 + 4 files changed, 6 insertions(+)init/Kconfig +3
-0@@ -39,6 +39,9 @@ config TOOLS_SUPPORT_RELR config CC_HAS_ASM_INLINE def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x c - -c -o /dev/null) +config CC_HAS_WORKING_NOSANITIZE + def_bool !CC_IS_GCC || GCC_VERSION >= 80000 + config CONSTRUCTORS bool depends on !UMLlib/Kconfig.kasan +1
-0@@ -20,6 +20,7 @@ config KASAN depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \ (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS) depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + depends on CC_HAS_WORKING_NOSANITIZE help Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, designed to find out-of-bounds accesses and use-after-free bugs.lib/Kconfig.kcsan +1
-0@@ -5,6 +5,7 @@ config HAVE_ARCH_KCSAN config HAVE_KCSAN_COMPILER def_bool CC_IS_CLANG && $(cc-option,-fsanitize=thread -mllvm -tsan-distinguish-volatile=1) + depends on CC_HAS_WORKING_NOSANITIZE help For the list of compilers that support KCSAN, please see <file:Documentation/dev-tools/kcsan.rst>.lib/Kconfig.ubsan +1
-1@@ -4,6 +4,7 @@ config ARCH_HAS_UBSAN_SANITIZE_ALL menuconfig UBSAN bool "Undefined behaviour sanity checker" + depends on CC_HAS_WORKING_NOSANITIZE help This option enables the Undefined Behaviour sanity checker. Compile-time instrumentation is used to detect various undefined --
Signature · 2 lines
Thanks, ~Nick Desaulniers
Marco Elver
maintainerQuoted 20 lines · Nick Desaulniers
Quoted 10 lines · Marco Elver' via Clang Built Linux
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.Quoted 6 lines
For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Marco Elver <[email protected]>
Quoted 5 lines
Is this a problem only for x86? If so, that's quite a jump in minimal compiler versions for a feature that I don't think is currently problematic for other architectures? (Based on
https://lore.kernel.org/lkml/[email protected]/
)
__always_inline void foo(void) {}__no_sanitize_address void bar(void) { foo(); }
where __no_sanitize_address is implied by 'noinstr' now, and 'noinstr' is no longer just x86.
Therefore, it's broken on *all* architectures. The compiler will just break the build with an error. I don't think we can fix that.
Thanks,
-- Marco
Quoted 19 lines · Nick Desaulniers
Quoted 10 lines · Marco Elver' via Clang Built Linux
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.Quoted 6 lines
For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Marco Elver <[email protected]>
Quoted 5 lines
Is this a problem only for x86? If so, that's quite a jump in minimal compiler versions for a feature that I don't think is currently problematic for other architectures? (Based on
https://lore.kernel.org/lkml/[email protected]/
)
Currently x86 only, but I know other arch maintainers are planning to have a hard look at their code based on our findings.
Marco Elver
maintainerQuoted 21 lines · Peter Zijlstra
Quoted 19 lines · Nick Desaulniers
Quoted 16 lines · Marco Elver' via Clang Built Linux
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Marco Elver <[email protected]>
Quoted 8 lines
Quoted 5 lines
Is this a problem only for x86? If so, that's quite a jump in minimal compiler versions for a feature that I don't think is currently problematic for other architectures? (Based on
https://lore.kernel.org/lkml/[email protected]/
)Currently x86 only, but I know other arch maintainers are planning to have a hard look at their code based on our findings.
I've already spotted a bunch of 'noinstr' outside arch/x86 e.g. in kernel/{locking,rcu}, and a bunch of these functions use atomic_*, all of which are __always_inline. The noinstr uses outside arch/x86 would break builds on all architecture with GCC <= 7 when using sanitizers. At least that's what led me to conclude we need this for all architectures.
Thanks,
-- Marco
Quoted 11 lines · Marco Elver
Quoted 2 lines · Peter Zijlstra
Currently x86 only, but I know other arch maintainers are planning to have a hard look at their code based on our findings.
I've already spotted a bunch of 'noinstr' outside
arch/x86e.g. in kernel/{locking,rcu}, and a bunch of these functions use atomic_*, all of which are__always_inline. The noinstr uses outsidearch/x86would break builds on all architecture with GCC <= 7 when using sanitizers. At least that's what led me to conclude we need this for all architectures.
True; but !x86 could, probably, get away with not fully respecting noinstr at this time. But that'd make a mess of things again, so my preference is as you did, unilaterally raise the min version for *SAN.
That said; noinstr's __no_sanitize combined with atomic_t might be 'interesting', because the regular atomic things have explicit annotations in them. That should give validation warnings for the right .config, I'll have to go try -- so far I've made sure to never enable the *SAN stuff.
Quoted 14 lines · Peter Zijlstra
Quoted 11 lines · Marco Elver
Quoted 2 lines · Peter Zijlstra
Currently x86 only, but I know other arch maintainers are planning to have a hard look at their code based on our findings.
I've already spotted a bunch of 'noinstr' outside
arch/x86e.g. in kernel/{locking,rcu}, and a bunch of these functions use atomic_*, all of which are__always_inline. The noinstr uses outsidearch/x86would break builds on all architecture with GCC <= 7 when using sanitizers. At least that's what led me to conclude we need this for all architectures.
Quoted 3 lines
True; but !x86 could, probably, get away with not fully respecting noinstr at this time. But that'd make a mess of things again, so my preference is as you did, unilaterally raise the min version for *SAN.
Fair, thought I'd ask. (I prefer people use newer hopefully-less-buggier-but-maybe-not-really-suprise-they're-actually-worse tools anyways)
---
Thanks,
~Nick Desaulniers
PATCH 1/1 rcu: Fixup noinstr warnings
In reply to Peter ZijlstraQuoted 5 lines · Peter Zijlstra
That said; noinstr's
__no_sanitizecombined withatomic_tmight be 'interesting', because the regular atomic things have explicit annotations in them. That should give validation warnings for the right .config, I'll have to go try -- so far I've made sure to never enable the *SAN stuff.
---
Subject: rcu: Fixup noinstr warnings
A KCSAN build revealed we have explicit annoations through atomic_t usage, switch to arch_atomic_*() for the respective functions.
vmlinux.o: warning: objtool: rcu_nmi_exit()+0x4d: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_enter()+0x25: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_nmi_enter()+0x4f: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_exit()+0x2a: call to __kcsan_check_access() leaves .noinstr.text section vmlinux.o: warning: objtool: __rcu_is_watching()+0x25: call to __kcsan_check_access() leaves .noinstr.text section
--- kernel/rcu/tree.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
kernel/rcu/tree.c +5 -6
@@ -250,7 +250,7 @@ static noinstr void rcu_dynticks_eqs_enter(void)* next idle sojourn. */ rcu_dynticks_task_trace_enter ();// Before ->dynticks update! - seq =atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks); + seq =arch_atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks);// RCU is no longer watching. Better be in extended quiescent state! WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && (seq &RCU_DYNTICK_CTRL_CTR ));
@@ -274,13 +274,13 @@ static noinstr void rcu_dynticks_eqs_exit(void)* and we also must force ordering with the next RCU read-side * critical section. */ - seq =atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks); + seq =arch_atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks);// RCU is now watching. Better not be in an extended quiescent state! rcu_dynticks_task_trace_exit ();// After ->dynticks update! WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !(seq &RCU_DYNTICK_CTRL_CTR ));if (seq &RCU_DYNTICK_CTRL_MASK ) { -atomic_andnot (RCU_DYNTICK_CTRL_MASK , &rdp->dynticks); +arch_atomic_andnot (RCU_DYNTICK_CTRL_MASK , &rdp->dynticks);smp_mb__after_atomic ();/* _exit after clearing mask. */ } }
@@ -313,7 +313,7 @@ static __always_inline bool rcu_dynticks_curr_cpu_in_eqs(void) {struct rcu_data *rdp =this_cpu_ptr (&rcu_data); -return !(atomic_read (&rdp->dynticks) &RCU_DYNTICK_CTRL_CTR ); +return !(arch_atomic_read (&rdp->dynticks) &RCU_DYNTICK_CTRL_CTR ); }/*
@@ -692,6 +692,7 @@ noinstr void rcu_nmi_exit(void) {struct rcu_data *rdp =this_cpu_ptr (&rcu_data); +instrumentation_begin ();/* * Check for ->dynticks_nmi_nesting underflow and bad ->dynticks. * (We are exiting an NMI handler, so RCU better be paying attention
@@ -705,7 +706,6 @@ noinstr void rcu_nmi_exit(void)* leave it in non-RCU-idle state. */ if (rdp->dynticks_nmi_nesting !=1 ) { -instrumentation_begin ();trace_rcu_dyntick (TPS ("--=" ), rdp->dynticks_nmi_nesting, rdp->dynticks_nmi_nesting -2 ,atomic_read (&rdp->dynticks)); WRITE_ONCE(rdp->dynticks_nmi_nesting, /* No store tearing. */
@@ -714,7 +714,6 @@ noinstr void rcu_nmi_exit(void)return ; } -instrumentation_begin ();/* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */ trace_rcu_dyntick (TPS ("Startirq" ), rdp->dynticks_nmi_nesting,0 ,atomic_read (&rdp->dynticks));WRITE_ONCE (rdp->dynticks_nmi_nesting,0 );/* Avoid store tearing. */
Paul E. McKenney
maintainerRe: [PATCH] rcu: Fixup noinstr warnings
Quoted 14 lines · Peter Zijlstra
Quoted 5 lines · Peter Zijlstra
That said; noinstr's
__no_sanitizecombined withatomic_tmight be 'interesting', because the regular atomic things have explicit annotations in them. That should give validation warnings for the right .config, I'll have to go try -- so far I've made sure to never enable the *SAN stuff.---
Subject: rcu: Fixup noinstr warningsA KCSAN build revealed we have explicit annoations through
atomic_tusage, switch to arch_atomic_*() for the respective functions.
Quoted 7 lines
vmlinux.o: warning: objtool: rcu_nmi_exit()+0x4d: call to
__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_enter()+0x25: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_nmi_enter()+0x4f: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_exit()+0x2a: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: __rcu_is_watching()+0x25: call to__kcsan_check_access()leaves .noinstr.text sectionSigned-off-by Peter Zijlstra (Intel) <[email protected]>
This one does not apply cleanly onto the -rcu tree's "dev" branch, so I am guessing that it is intended to be carried in -tip with yours and Thomas's patch series.
If you do instead want this in -rcu, please let me know.
Thanx, Paul
Quoted 66 lines
--- kernel/rcu/tree.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)kernel/rcu/tree.c +5
-6@@ -250,7 +250,7 @@ static noinstr void rcu_dynticks_eqs_enter(void)* next idle sojourn. */ rcu_dynticks_task_trace_enter ();// Before ->dynticks update! - seq =atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks); + seq =arch_atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks);// RCU is no longer watching. Better be in extended quiescent state! WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && (seq &RCU_DYNTICK_CTRL_CTR ));@@ -274,13 +274,13 @@ static noinstr void rcu_dynticks_eqs_exit(void)* and we also must force ordering with the next RCU read-side * critical section. */ - seq =atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks); + seq =arch_atomic_add_return (RCU_DYNTICK_CTRL_CTR , &rdp->dynticks);// RCU is now watching. Better not be in an extended quiescent state! rcu_dynticks_task_trace_exit ();// After ->dynticks update! WARN_ON_ONCE (IS_ENABLED (CONFIG_RCU_EQS_DEBUG ) && !(seq &RCU_DYNTICK_CTRL_CTR ));if (seq &RCU_DYNTICK_CTRL_MASK ) { -atomic_andnot (RCU_DYNTICK_CTRL_MASK , &rdp->dynticks); +arch_atomic_andnot (RCU_DYNTICK_CTRL_MASK , &rdp->dynticks);smp_mb__after_atomic ();/* _exit after clearing mask. */ } }@@ -313,7 +313,7 @@ static __always_inline bool rcu_dynticks_curr_cpu_in_eqs(void) {struct rcu_data *rdp =this_cpu_ptr (&rcu_data); -return !(atomic_read (&rdp->dynticks) &RCU_DYNTICK_CTRL_CTR ); +return !(arch_atomic_read (&rdp->dynticks) &RCU_DYNTICK_CTRL_CTR ); }/* @@ -692,6 +692,7 @@ noinstr void rcu_nmi_exit(void) {struct rcu_data *rdp =this_cpu_ptr (&rcu_data); +instrumentation_begin ();/* * Check for ->dynticks_nmi_nesting underflow and bad ->dynticks. * (We are exiting an NMI handler, so RCU better be paying attention @@ -705,7 +706,6 @@ noinstr void rcu_nmi_exit(void)* leave it in non-RCU-idle state. */ if (rdp->dynticks_nmi_nesting !=1 ) { -instrumentation_begin ();trace_rcu_dyntick (TPS ("--=" ), rdp->dynticks_nmi_nesting, rdp->dynticks_nmi_nesting -2 ,atomic_read (&rdp->dynticks)); WRITE_ONCE(rdp->dynticks_nmi_nesting, /* No store tearing. */@@ -714,7 +714,6 @@ noinstr void rcu_nmi_exit(void)return ; } -instrumentation_begin ();/* This NMI interrupted an RCU-idle CPU, restore RCU-idleness. */ trace_rcu_dyntick (TPS ("Startirq" ), rdp->dynticks_nmi_nesting,0 ,atomic_read (&rdp->dynticks));WRITE_ONCE (rdp->dynticks_nmi_nesting,0 );/* Avoid store tearing. */
Re: [PATCH] rcu: Fixup noinstr warnings
Quoted 21 lines · Paul E. McKenney
Quoted 20 lines · Peter Zijlstra
Quoted 5 lines · Peter Zijlstra
That said; noinstr's
__no_sanitizecombined withatomic_tmight be 'interesting', because the regular atomic things have explicit annotations in them. That should give validation warnings for the right .config, I'll have to go try -- so far I've made sure to never enable the *SAN stuff.---
Subject: rcu: Fixup noinstr warningsA KCSAN build revealed we have explicit annoations through
atomic_tusage, switch to arch_atomic_*() for the respective functions.vmlinux.o: warning: objtool: rcu_nmi_exit()+0x4d: call to
__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_enter()+0x25: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_nmi_enter()+0x4f: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_exit()+0x2a: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: __rcu_is_watching()+0x25: call to__kcsan_check_access()leaves .noinstr.text section
Quoted 5 lines
Quoted 1 line
Signed-off-by Peter Zijlstra (Intel) <[email protected]>This one does not apply cleanly onto the -rcu tree's "dev" branch, so I am guessing that it is intended to be carried in -tip with yours and Thomas's patch series.
Right, I've not played patch tetris yet so see how it should all fit together. I also didn't know you feel about loosing the instrumentation in these functions.
One option would be do add explicit: instrument_atomic_write() calls before instrument_end() / after instrument_begin() in
the respective callers that have that.
Anyway, I'll shortly be posting a pile of patches resulting from various KCSAN and KASAN builds. The good news is that GCC-KASAN seems to behave quite well with Marco's patches, the bad news is that GCC-KASAN is retarded wrt inline and needs a bunch of kicks.
That is, it out-of-lines:
static inline bool foo(..)
{
return false;
}just because..
Paul E. McKenney
maintainerRe: [PATCH] rcu: Fixup noinstr warnings
Quoted 24 lines · Peter Zijlstra
Quoted 23 lines · Paul E. McKenney
Quoted 21 lines · Peter Zijlstra
Quoted 5 lines · Peter Zijlstra
That said; noinstr's
__no_sanitizecombined withatomic_tmight be 'interesting', because the regular atomic things have explicit annotations in them. That should give validation warnings for the right .config, I'll have to go try -- so far I've made sure to never enable the *SAN stuff.---
Subject: rcu: Fixup noinstr warningsA KCSAN build revealed we have explicit annoations through
atomic_tusage, switch to arch_atomic_*() for the respective functions.vmlinux.o: warning: objtool: rcu_nmi_exit()+0x4d: call to
__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_enter()+0x25: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_nmi_enter()+0x4f: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: rcu_dynticks_eqs_exit()+0x2a: call to__kcsan_check_access()leaves .noinstr.text section vmlinux.o: warning: objtool: __rcu_is_watching()+0x25: call to__kcsan_check_access()leaves .noinstr.text sectionSigned-off-by Peter Zijlstra (Intel) <[email protected]>
Quoted 7 lines
Quoted 3 lines
This one does not apply cleanly onto the -rcu tree's "dev" branch, so I am guessing that it is intended to be carried in -tip with yours and Thomas's patch series.
Right, I've not played patch tetris yet so see how it should all fit together. I also didn't know you feel about loosing the instrumentation in these functions.
It would be very good for KCSAN to be able to detect misuse of ->dynticks!
Quoted 3 lines
One option would be do add explicit:
instrument_atomic_write()calls beforeinstrument_end()/ afterinstrument_begin()in
the respective callers that have that.
One good thing: The atomic_andnot() goes away in -rcu patches slated for v5.9. However, the others remain.
So if today's -next is any guide, this instrument_atomic_write() would be added (for one example) in the functions that invoke rcu_dynticks_eqs_enter(), since it is noinstr. Rather annoying, and will require careful commenting. But there are only two such calls and they are both in the same file and it is very low-level code, so this should be doable.
I should also add some commentary to the RCU requirements document say why all of this is happening.
Quoted 5 lines
Anyway, I'll shortly be posting a pile of patches resulting from various KCSAN and KASAN builds. The good news is that GCC-KASAN seems to behave quite well with Marco's patches, the bad news is that GCC-KASAN is retarded wrt inline and needs a bunch of kicks.
Quoted 8 lines
That is, it out-of-lines:
static inline bool foo(..)
{ return false; }just because..
Compilers!!! :-/
Thanx, Paul
Quoted 10 lines · Marco Elver
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.
Quoted 6 lines
For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Marco Elver <[email protected]>
Quoted 63 lines
---
Apply after:
https://lkml.kernel.org/r/[email protected]--- init/Kconfig | 3 +++ lib/Kconfig.kasan | 1 + lib/Kconfig.kcsan | 1 + lib/Kconfig.ubsan | 1 + 4 files changed, 6 insertions(+)init/Kconfig +3
-0@@ -39,6 +39,9 @@ config TOOLS_SUPPORT_RELR config CC_HAS_ASM_INLINE def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x c - -c -o /dev/null) +config CC_HAS_WORKING_NOSANITIZE + def_bool !CC_IS_GCC || GCC_VERSION >= 80000 + config CONSTRUCTORS bool depends on !UMLlib/Kconfig.kasan +1
-0@@ -20,6 +20,7 @@ config KASAN depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \ (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS) depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + depends on CC_HAS_WORKING_NOSANITIZE help Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, designed to find out-of-bounds accesses and use-after-free bugs.lib/Kconfig.kcsan +1
-0@@ -5,6 +5,7 @@ config HAVE_ARCH_KCSAN config HAVE_KCSAN_COMPILER def_bool CC_IS_CLANG && $(cc-option,-fsanitize=thread -mllvm -tsan-distinguish-volatile=1) + depends on CC_HAS_WORKING_NOSANITIZE help For the list of compilers that support KCSAN, please see <file:Documentation/dev-tools/kcsan.rst>.lib/Kconfig.ubsan +1
-1@@ -4,6 +4,7 @@ config ARCH_HAS_UBSAN_SANITIZE_ALL menuconfig UBSAN bool "Undefined behaviour sanity checker" + depends on CC_HAS_WORKING_NOSANITIZE help This option enables the Undefined Behaviour sanity checker. Compile-time instrumentation is used to detect various undefined --2.27.0.rc2.251.g90737beb825-goog
Marco Elver
maintainerQuoted 12 lines · Andrey Konovalov' via kasan-dev
Quoted 10 lines · Marco Elver
Adds config variable
CC_HAS_WORKING_NOSANITIZE, which will be true if we have a compiler that does not fail builds due tono_sanitizefunctions. This does not yet mean they work as intended, but for automated build-tests, this is the minimum requirement.For example, we require that
__always_inlinefunctions used fromno_sanitizefunctions do not generate instrumentation. On GCC <= 7 this fails to build entirely, therefore we make the minimum version GCC 8.
Quoted 8 lines
Quoted 6 lines
For KCSAN this is a non-functional change, however, we should add it in case this variable changes in future.
Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Marco Elver <[email protected]>Acked-by Andrey Konovalov <[email protected]>
I've sent a v2 of this, which limits the compiler-bump to KASAN only. It appears no_sanitize_undefined (for UBSAN) is not broken GCC <= 7, and in general the no_sanitize attributes seem to behave differently from sanitizer to sanitizer as we discovered for UBSAN.