bpf: don't rely on GCC __attribute__((optimize)) to disable GCSE
Subsystems (5)
PATCH 1/1 bpf: don't rely on GCC __attribute__((optimize)) to disable GCSE
Commit 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()") introduced a __no_fgcse macro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.
However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case, it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
This reverts commit 3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.
--- include/linux/compiler-gcc.h | 2 -- include/linux/compiler_types.h | 4 ---- kernel/bpf/Makefile | 4 +++- kernel/bpf/core.c | 2 +- 4 files changed, 4 insertions(+), 8 deletions(-)
include/linux/compiler-gcc.h +0 -2
@@ -175,5 +175,3 @@#else #define __diag_GCC_8 (s)#endif - -#define __no_fgcse __attribute__((optimize("-fno-gcse")))
include/linux/compiler_types.h +0 -4
@@ -247,10 +247,6 @@ struct ftrace_likely_data {#define asm_inline asm#endif -#ifndef __no_fgcse -# define __no_fgcse -#endif -/* Are two types/vars the same type (ignoring qualifiers)? */ #define __same_type (a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
kernel/bpf/Makefile +3 -1
@@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 obj-y := core.o -CFLAGS_core.o += $(call cc-disable-warning, override-init) +# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details +cflags-core-$(CONFIG_X86) := -fno-gcse +CFLAGS_core.o += $(call cc-disable-warning, override-init) $(cflags-core-y) obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o bpf_iter.o map_iter.o task_iter.o prog_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o
kernel/bpf/core.c +1 -1
@@ -1369,7 +1369,7 @@ u64 __weak bpf_probe_read_kernel(void *dst, u32 size, const void *unsafe_ptr)* * Decode and execute eBPF instructions. */ -static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) +static u64 ___bpf_prog_run (u64 *regs,const struct bpf_insn *insn,u64 *stack) {#define BPF_INSN_2_LBL (x, y) [BPF_##x | BPF_##y] = &&x##_##y#define BPF_INSN_3_LBL (x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
Signature · 1 line
2.17.1
Quoted 68 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case, it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
This reverts commit
3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.Cc Nick Desaulniers <[email protected]>Cc Arvind Sankar <[email protected]>Cc Randy Dunlap <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Peter Zijlstra (Intel) <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Kees Cook <[email protected]>Fixes 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")Signed-off-by Ard Biesheuvel <[email protected]>--- include/linux/compiler-gcc.h | 2 -- include/linux/compiler_types.h | 4 ---- kernel/bpf/Makefile | 4 +++- kernel/bpf/core.c | 2 +- 4 files changed, 4 insertions(+), 8 deletions(-)include/linux/compiler-gcc.h +0
-2@@ -175,5 +175,3 @@#else #define __diag_GCC_8 (s)#endif - -#define __no_fgcse __attribute__((optimize("-fno-gcse")))include/linux/compiler_types.h +0
-4@@ -247,10 +247,6 @@ struct ftrace_likely_data {#define asm_inline asm#endif -#ifndef __no_fgcse -# define __no_fgcse -#endif -/* Are two types/vars the same type (ignoring qualifiers)? */ #define __same_type (a, b) __builtin_types_compatible_p(typeof(a), typeof(b))kernel/bpf/Makefile +0
-0
kernel/bpf/Makefile line 4
Quoted 5 lines
kernel/bpf/Makefile +2
-1@@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 obj-y := core.o -CFLAGS_core.o += $(call cc-disable-warning, override-init) +# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details +cflags-core-$(CONFIG_X86) := -fno-gcse
-fno-gcse is not recognized by clang and will produce -Wignored-optimization-argument. It should at least be wrapped in cc-option, though since it's unlikely to ever not be compiler specific, I think it might be ok to guard with ifdef CONFIG_CC_IS_GCC. Also, might we want to only do this for ifndef CONFIG_RETPOLINE, based on 3193c0836f203?
Finally, this is going to disable GCSE for the whole translation unit,
which may be overkill. Previously it was isolated to one function
definition. You could lower the definition of the preprocessor define into kernel/bpf/core.c to keep its use isolated as far as possible.
I'm fine with either approach, but we should avoid new warnings for clang. Thanks for the patch!
Quoted 20 lines
kernel/bpf/Makefile +1
-0+CFLAGS_core.o += $(call cc-disable-warning, override-init) $(cflags-core-y) obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o bpf_iter.o map_iter.o task_iter.o prog_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.okernel/bpf/core.c +1
-2@@ -1369,7 +1369,7 @@ u64 __weak bpf_probe_read_kernel(void *dst, u32 size, const void *unsafe_ptr)* * Decode and execute eBPF instructions. */ -static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) +static u64 ___bpf_prog_run (u64 *regs,const struct bpf_insn *insn,u64 *stack) {#define BPF_INSN_2_LBL (x, y) [BPF_##x | BPF_##y] = &&x##_##y#define BPF_INSN_3_LBL (x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z --2.17.1
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 83 lines · Nick Desaulniers
Quoted 73 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case, it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
This reverts commit
3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.Cc Nick Desaulniers <[email protected]>Cc Arvind Sankar <[email protected]>Cc Randy Dunlap <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Peter Zijlstra (Intel) <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Kees Cook <[email protected]>Fixes 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")Signed-off-by Ard Biesheuvel <[email protected]>--- include/linux/compiler-gcc.h | 2 -- include/linux/compiler_types.h | 4 ---- kernel/bpf/Makefile | 4 +++- kernel/bpf/core.c | 2 +- 4 files changed, 4 insertions(+), 8 deletions(-)include/linux/compiler-gcc.h +0
-2@@ -175,5 +175,3 @@#else #define __diag_GCC_8 (s)#endif - -#define __no_fgcse __attribute__((optimize("-fno-gcse")))include/linux/compiler_types.h +0
-4@@ -247,10 +247,6 @@ struct ftrace_likely_data {#define asm_inline asm#endif -#ifndef __no_fgcse -# define __no_fgcse -#endif -/* Are two types/vars the same type (ignoring qualifiers)? */ #define __same_type (a, b) __builtin_types_compatible_p(typeof(a), typeof(b))kernel/bpf/Makefile +2
-1@@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 obj-y := core.o -CFLAGS_core.o += $(call cc-disable-warning, override-init) +# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details +cflags-core-$(CONFIG_X86) := -fno-gcse(quoted diff) +0
-2-fno-gcse is not recognized by clang and will produce -Wignored-optimization-argument. It should at least be wrapped incc-option, though since it's unlikely to ever not be compiler specific, I think it might be ok to guard with
ifdef CONFIG_CC_IS_GCC. Also, might we want to only do this forifndef CONFIG_RETPOLINE, based on3193c0836f203?
Quoted 5 lines
Finally, this is going to disable GCSE for the whole translation unit,
which may be overkill. Previously it was isolated to one function
definition. You could lower the definition of the preprocessor define intokernel/bpf/core.cto keep its use isolated as far as possible.
Which preprocessor define?
Quoted 28 lines
I'm fine with either approach, but we should avoid new warnings for clang. Thanks for the patch!
Quoted 18 lines
kernel/bpf/Makefile +1
-0+CFLAGS_core.o += $(call cc-disable-warning, override-init) $(cflags-core-y) obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o bpf_iter.o map_iter.o task_iter.o prog_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.okernel/bpf/core.c +1
-2@@ -1369,7 +1369,7 @@ u64 __weak bpf_probe_read_kernel(void *dst, u32 size, const void *unsafe_ptr)* * Decode and execute eBPF instructions. */ -static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) +static u64 ___bpf_prog_run (u64 *regs,const struct bpf_insn *insn,u64 *stack) {#define BPF_INSN_2_LBL (x, y) [BPF_##x | BPF_##y] = &&x##_##y#define BPF_INSN_3_LBL (x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z --Quoted 2 lines
2.17.1
--
Thanks,
~Nick Desaulniers
Quoted 23 lines · Ard Biesheuvel
Quoted 11 lines · Nick Desaulniers
Quoted 9 lines · Ard Biesheuvel
include/linux/compiler_types.h +0
-0@@ -247,10 +247,6 @@ struct ftrace_likely_data {#define asm_inline asm#endif Quoted 9 lines
include/linux/compiler_types.h struct ftrace_likely_data { · line 253
Quoted 4 lines
include/linux/compiler_types.h +0
-4-#ifndef __no_fgcse -# define __no_fgcse -#endif -Finally, this is going to disable GCSE for the whole translation unit,
which may be overkill. Previously it was isolated to one function
definition. You could lower the definition of the preprocessor define intokernel/bpf/core.cto keep its use isolated as far as possible.
Quoted 1 line
Which preprocessor define?
__no_fgcse
Quoted 3 lines
Quoted 3 lines
I'm fine with either approach, but we should avoid new warnings for clang. Thanks for the patch!
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 25 lines · Nick Desaulniers
Quoted 23 lines · Ard Biesheuvel
Quoted 20 lines · Nick Desaulniers
include/linux/compiler_types.h struct ftrace_likely_data { · line 253
Quoted 13 lines · Ard Biesheuvel
include/linux/compiler_types.h +0
-4@@ -247,10 +247,6 @@ struct ftrace_likely_data {#define asm_inline asm#endif -#ifndef __no_fgcse -# define __no_fgcse -#endif -Finally, this is going to disable GCSE for the whole translation unit,
which may be overkill. Previously it was isolated to one function
definition. You could lower the definition of the preprocessor define intokernel/bpf/core.cto keep its use isolated as far as possible.
Quoted 4 lines
Quoted 1 line
Which preprocessor define?
__no_fgcse
But we can't use that, that is the whole point of this patch.
Quoted 27 lines · Ard Biesheuvel
Quoted 25 lines · Nick Desaulniers
Quoted 23 lines · Ard Biesheuvel
Quoted 20 lines · Nick Desaulniers
include/linux/compiler_types.h struct ftrace_likely_data { · line 253
Quoted 13 lines · Ard Biesheuvel
include/linux/compiler_types.h +0
-4@@ -247,10 +247,6 @@ struct ftrace_likely_data {#define asm_inline asm#endif -#ifndef __no_fgcse -# define __no_fgcse -#endif -Finally, this is going to disable GCSE for the whole translation unit,
which may be overkill. Previously it was isolated to one function
definition. You could lower the definition of the preprocessor define intokernel/bpf/core.cto keep its use isolated as far as possible.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Which preprocessor define?
__no_fgcseBut we can't use that, that is the whole point of this patch.
Ah, right because the attribute drops other command line flags...ok, then -fno-gcse the whole translation unit it is then.
Still need to avoid new warnings with clang though.
Signature · 2 lines
Thanks, ~Nick Desaulniers
Daniel Borkmann
maintainerQuoted 10 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case,
Looks like there are couple more as well aside from __no_fgcse, are you also planning to fix them?
arch/powerpc/kernel/setup.h:14:#define __nostackprotector __attribute__((__optimize__("no-stack-protector")))
tools/include/linux/compiler-gcc.h:37:#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls")))
Quoted 3 lines
it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
Would have been useful to add a pointer to the original discussion with Link tag.
Quoted 16 lines
This reverts commit
3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.Cc Nick Desaulniers <[email protected]>Cc Arvind Sankar <[email protected]>Cc Randy Dunlap <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Peter Zijlstra (Intel) <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Kees Cook <[email protected]>Fixes 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")Signed-off-by Ard Biesheuvel <[email protected]>
[...]
kernel/bpf/Makefile line 5
Quoted 11 lines
kernel/bpf/Makefile +3
-1@@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 obj-y := core.o -CFLAGS_core.o += $(call cc-disable-warning, override-init) +# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details +cflags-core-$(CONFIG_X86) := -fno-gcse +CFLAGS_core.o += $(call cc-disable-warning, override-init) $(cflags-core-y)
Also, this needs to be guarded behind !CONFIG_RETPOLINE and !CONFIG_BPF_JIT_ALWAYS_ON in particular the latter since only in this case interpreter is compiled in ... most distros have the CONFIG_BPF_JIT_ALWAYS_ON set these days for x86.
Do you have an analysis for the commit log on what else this penalizes in core.c if it's now for the entire translation unit?
Quoted 16 lines
kernel/bpf/Makefile +0
-0obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o bpf_iter.o map_iter.o task_iter.o prog_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.okernel/bpf/core.c +1
-1@@ -1369,7 +1369,7 @@ u64 __weak bpf_probe_read_kernel(void *dst, u32 size, const void *unsafe_ptr)* * Decode and execute eBPF instructions. */ -static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) +static u64 ___bpf_prog_run (u64 *regs,const struct bpf_insn *insn,u64 *stack) {#define BPF_INSN_2_LBL (x, y) [BPF_##x | BPF_##y] = &&x##_##y#define BPF_INSN_3_LBL (x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
Quoted 13 lines · Daniel Borkmann
Quoted 10 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case,
Quoted 4 lines
Looks like there are couple more as well aside from
__no_fgcse, are you also planning to fix them?arch/powerpc/kernel/setup.h:14:#define __nostackprotector __attribute__((__optimize__("no-stack-protector")))
GCC literally just landed support for __attribute__((no_stack_protector)) a few days ago. I was planning on sending a patch adding it to compiler_attributes.h, but we won't be able to rely on it for a while. Now I see I'll have to clean up ppc a bit. Surely they've had bugs related to optimize attribute unexpectedly dropping flags.
Quoted 1 line
tools/include/linux/compiler-gcc.h:37:#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls")))Only used in perf?tools/perf/tests/dwarf-unwind.c
Quoted 63 lines
Quoted 4 lines
it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
Would have been useful to add a pointer to the original discussion with Link tag.
Quoted 16 lines
This reverts commit
3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.Cc Nick Desaulniers <[email protected]>Cc Arvind Sankar <[email protected]>Cc Randy Dunlap <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Peter Zijlstra (Intel) <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Kees Cook <[email protected]>Fixes 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")Signed-off-by Ard Biesheuvel <[email protected]>[...]
kernel/bpf/Makefile line 5
Quoted 11 lines
kernel/bpf/Makefile +3
-1@@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 obj-y := core.o -CFLAGS_core.o += $(call cc-disable-warning, override-init) +# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details +cflags-core-$(CONFIG_X86) := -fno-gcse +CFLAGS_core.o += $(call cc-disable-warning, override-init) $(cflags-core-y)Also, this needs to be guarded behind !CONFIG_RETPOLINE and !CONFIG_BPF_JIT_ALWAYS_ON in particular the latter since only in this case interpreter is compiled in ... most distros have the
CONFIG_BPF_JIT_ALWAYS_ONset these days for x86.Do you have an analysis for the commit log on what else this penalizes in
core.cif it's now for the entire translation unit?Quoted 16 lines
kernel/bpf/Makefile +0
-0obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o bpf_iter.o map_iter.o task_iter.o prog_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.okernel/bpf/core.c +1
-1@@ -1369,7 +1369,7 @@ u64 __weak bpf_probe_read_kernel(void *dst, u32 size, const void *unsafe_ptr)* * Decode and execute eBPF instructions. */ -static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) +static u64 ___bpf_prog_run (u64 *regs,const struct bpf_insn *insn,u64 *stack) {#define BPF_INSN_2_LBL (x, y) [BPF_##x | BPF_##y] = &&x##_##y#define BPF_INSN_3_LBL (x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
Signature · 2 lines
Thanks, ~Nick Desaulniers
Hi Ard,
Quoted 18 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case, it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
This reverts commit
3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.
Quoted 12 lines
Cc Nick Desaulniers <[email protected]>Cc Arvind Sankar <[email protected]>Cc Randy Dunlap <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Peter Zijlstra (Intel) <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Kees Cook <[email protected]>Fixes 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")Signed-off-by Ard Biesheuvel <[email protected]>
Thanks, this gets rid of the following warning, which you may want to quote in the patch description:
aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' fromkernel/bpf/core.o' being placed in section .eh_frame'
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH] tools/perf: Remove broken __no_tail_call attribute
In reply to Peter ZijlstraQuoted 2 lines · Peter Zijlstra
Subject:
tools/perf: Remove broken__no_tail_callattribute
Cheers,
Miguel
Peter Zijlstra
maintainerPATCH 1/1 tools/perf: Remove broken __no_tail_call attribute
In reply to Nick DesaulniersQuoted 26 lines · Nick Desaulniers
Quoted 13 lines · Daniel Borkmann
Quoted 10 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case,
Quoted 4 lines
Looks like there are couple more as well aside from
__no_fgcse, are you also planning to fix them?arch/powerpc/kernel/setup.h:14:#define __nostackprotector __attribute__((__optimize__("no-stack-protector")))GCC literally just landed support for __attribute__((no_stack_protector)) a few days ago. I was planning on sending a patch adding it to
compiler_attributes.h, but we won't be able to rely on it for a while. Now I see I'll have to clean up ppc a bit. Surely they've had bugs related to optimize attribute unexpectedly dropping flags.
Quoted 4 lines
Quoted 1 line
tools/include/linux/compiler-gcc.h:37:#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls")))Only used in perf?
tools/perf/tests/dwarf-unwind.c
Right, that should probably be fixed. It also probably doesn't matter too much since its an unwinder tests, but still, having that attribute is dangerous.
The only cross-compiler way of doing this is like in commit a9a3ed1eff360.
---
Subject: tools/perf: Remove broken __no_tail_call attribute
The GCC specific __attribute__((optimize)) attribute does not what is commonly expected and is explicitly recommended against using in production code by the GCC people.
Unlike what is often expected, it doesn't add to the optimization flags, but it fully replaces them, loosing any and all optimization flags provided by the compiler commandline.
The only guaranteed upon means of inhibiting tail-calls is by placing a volatile asm with side-effects after the call such that the tail-call simply cannot be done.
Given the original commit wasn't specific on which calls were the problem, this removal might re-introduce the problem, which can then be re-analyzed and cured properly.
--- tools/include/linux/compiler-gcc.h | 12 ------------ tools/include/linux/compiler.h | 3 --- tools/perf/tests/dwarf-unwind.c | 10 +++++----- 3 files changed, 5 insertions(+), 20 deletions(-)
tools/include/linux/compiler-gcc.h +0 -12
@@ -27,18 +27,6 @@#define __pure __attribute__((pure))#endif #define noinline __attribute__((noinline)) -#ifdef __has_attribute -#if __has_attribute (disable_tail_calls) -#define __no_tail_call __attribute__((disable_tail_calls)) -#endif -#endif -#ifndef __no_tail_call -#if GCC_VERSION >40201 -#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls"))) -#else -#define __no_tail_call -#endif -#endif #ifndef __packed#define __packed __attribute__((packed))#endif
tools/include/linux/compiler.h +0 -3
@@ -47,9 +47,6 @@#ifndef noinline#define noinline#endif -#ifndef __no_tail_call -#define __no_tail_call -#endif /* Are two types/vars the same type (ignoring qualifiers)? */ #ifndef __same_type
tools/perf/tests/dwarf-unwind.c +5 -5
@@ -95,7 +95,7 @@ static int unwind_entry(struct unwind_entry *entry, void *arg)return strcmp ((const char *) symbol, funcs[idx]); } -__no_tail_call noinline int test_dwarf_unwind__thread(struct thread *thread) +noinline int test_dwarf_unwind__thread(struct thread *thread) {struct perf_sample sample;unsigned long cnt =0 ;
@@ -126,7 +126,7 @@ __no_tail_call noinline int test_dwarf_unwind__thread(struct thread *thread)static int global_unwind_retval = -INT_MAX ; -__no_tail_call noinline int test_dwarf_unwind__compare(void *p1, void *p2) +noinline int test_dwarf_unwind__compare(void *p1, void *p2) {/* Any possible value should be 'thread' */ struct thread *thread = *(struct thread **)p1;
@@ -145,7 +145,7 @@ __no_tail_call noinline int test_dwarf_unwind__compare(void *p1, void *p2)return p1 - p2; } -__no_tail_call noinline int test_dwarf_unwind__krava_3(struct thread *thread) +noinline int test_dwarf_unwind__krava_3(struct thread *thread) {struct thread *array[2 ] = {thread, thread};void *fp = &bsearch;
@@ -164,12 +164,12 @@ __no_tail_call noinline int test_dwarf_unwind__krava_3(struct thread *thread)return global_unwind_retval; } -__no_tail_call noinline int test_dwarf_unwind__krava_2(struct thread *thread) +noinline int test_dwarf_unwind__krava_2(struct thread *thread) {return test_dwarf_unwind__krava_3 (thread); } -__no_tail_call noinline int test_dwarf_unwind__krava_1(struct thread *thread) +noinline int test_dwarf_unwind__krava_1(struct thread *thread) {return test_dwarf_unwind__krava_2 (thread); }
Quoted 70 lines · Ard Biesheuvel
Quoted 13 lines · Daniel Borkmann
Quoted 10 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case,
Quoted 6 lines
Looks like there are couple more as well aside from
__no_fgcse, are you also planning to fix them?arch/powerpc/kernel/setup.h:14:#define __nostackprotector __attribute__((__optimize__("no-stack-protector"))) tools/include/linux/compiler-gcc.h:37:#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls")))No, but we can notify the respective maintainers.
Quoted 9 lines
Quoted 3 lines
it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
Would have been useful to add a pointer to the original discussion with Link tag.
Agreed.
Quoted 17 lines
Quoted 16 lines
This reverts commit
3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.Cc Nick Desaulniers <[email protected]>Cc Arvind Sankar <[email protected]>Cc Randy Dunlap <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Peter Zijlstra (Intel) <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Kees Cook <[email protected]>Fixes 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")Signed-off-by Ard Biesheuvel <[email protected]>[...]
Quoted 16 lines
kernel/bpf/Makefile line 5
Quoted 11 lines
kernel/bpf/Makefile +3
-1@@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 obj-y := core.o -CFLAGS_core.o += $(call cc-disable-warning, override-init) +# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details +cflags-core-$(CONFIG_X86) := -fno-gcse +CFLAGS_core.o += $(call cc-disable-warning, override-init) $(cflags-core-y)Also, this needs to be guarded behind !CONFIG_RETPOLINE and !CONFIG_BPF_JIT_ALWAYS_ON in particular the latter since only in this case interpreter is compiled in ... most distros have the
CONFIG_BPF_JIT_ALWAYS_ONset these days for x86.
Quoted 3 lines
Is that a new requirement? Because before this patch, -fno-gcse was applied unconditionally.
Ah never mind. You are saying ___bpf_prog_run() does not even exist if CONFIG_BPF_JIT_ALWAYS_ON=y, right?
Quoted 29 lines
Quoted 3 lines
Do you have an analysis for the commit log on what else this penalizes in
core.cif it's now for the entire translation unit?No, I simply observed the regression this caused on non-x86 architectures, and proposed a way to fix it.
Do you have any concerns in particular regarding other things in
core.c? Would you prefer___bpf_prog_run()to be moved into a separate .c file?Quoted 17 lines
Quoted 16 lines
kernel/bpf/Makefile +0
-0obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o bpf_iter.o map_iter.o task_iter.o prog_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.okernel/bpf/core.c +1
-1@@ -1369,7 +1369,7 @@ u64 __weak bpf_probe_read_kernel(void *dst, u32 size, const void *unsafe_ptr)* * Decode and execute eBPF instructions. */ -static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) +static u64 ___bpf_prog_run (u64 *regs,const struct bpf_insn *insn,u64 *stack) {#define BPF_INSN_2_LBL (x, y) [BPF_##x | BPF_##y] = &&x##_##y#define BPF_INSN_3_LBL (x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
Re: [PATCH] tools/perf: Remove broken __no_tail_call attribute
In reply to Peter ZijlstraQuoted 59 lines · Peter Zijlstra
Quoted 26 lines · Nick Desaulniers
Quoted 13 lines · Daniel Borkmann
Quoted 10 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case,
Quoted 4 lines
Looks like there are couple more as well aside from
__no_fgcse, are you also planning to fix them?arch/powerpc/kernel/setup.h:14:#define __nostackprotector __attribute__((__optimize__("no-stack-protector")))GCC literally just landed support for __attribute__((no_stack_protector)) a few days ago. I was planning on sending a patch adding it to
compiler_attributes.h, but we won't be able to rely on it for a while. Now I see I'll have to clean up ppc a bit. Surely they've had bugs related to optimize attribute unexpectedly dropping flags.Quoted 4 lines
Quoted 1 line
tools/include/linux/compiler-gcc.h:37:#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls")))Only used in perf?
tools/perf/tests/dwarf-unwind.cRight, that should probably be fixed. It also probably doesn't matter too much since its an unwinder tests, but still, having that attribute is dangerous.
The only cross-compiler way of doing this is like in commit
a9a3ed1eff360.---
Subject:tools/perf: Remove broken__no_tail_callattributeThe GCC specific __attribute__((optimize)) attribute does not what is commonly expected and is explicitly recommended against using in production code by the GCC people.
Unlike what is often expected, it doesn't add to the optimization flags, but it fully replaces them, loosing any and all optimization flags provided by the compiler commandline.
The only guaranteed upon means of inhibiting tail-calls is by placing a volatile asm with side-effects after the call such that the tail-call simply cannot be done.
Given the original commit wasn't specific on which calls were the problem, this removal might re-introduce the problem, which can then be re-analyzed and cured properly.
Quoted 7 lines
Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- tools/include/linux/compiler-gcc.h | 12 ------------ tools/include/linux/compiler.h | 3 --- tools/perf/tests/dwarf-unwind.c | 10 +++++----- 3 files changed, 5 insertions(+), 20 deletions(-)
Quoted 83 lines
tools/include/linux/compiler-gcc.h +0
-12@@ -27,18 +27,6 @@#define __pure __attribute__((pure))#endif #define noinline __attribute__((noinline)) -#ifdef __has_attribute -#if __has_attribute (disable_tail_calls) -#define __no_tail_call __attribute__((disable_tail_calls)) -#endif -#endif -#ifndef __no_tail_call -#if GCC_VERSION >40201 -#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls"))) -#else -#define __no_tail_call -#endif -#endif #ifndef __packed#define __packed __attribute__((packed))#endif tools/include/linux/compiler.h +0
-3@@ -47,9 +47,6 @@#ifndef noinline#define noinline#endif -#ifndef __no_tail_call -#define __no_tail_call -#endif /* Are two types/vars the same type (ignoring qualifiers)? */ #ifndef __same_typetools/perf/tests/dwarf-unwind.c +5
-5@@ -95,7 +95,7 @@ static int unwind_entry(struct unwind_entry *entry, void *arg)return strcmp ((const char *) symbol, funcs[idx]); } -__no_tail_call noinline int test_dwarf_unwind__thread(struct thread *thread) +noinline int test_dwarf_unwind__thread(struct thread *thread) {struct perf_sample sample;unsigned long cnt =0 ;@@ -126,7 +126,7 @@ __no_tail_call noinline int test_dwarf_unwind__thread(struct thread *thread)static int global_unwind_retval = -INT_MAX ; -__no_tail_call noinline int test_dwarf_unwind__compare(void *p1, void *p2) +noinline int test_dwarf_unwind__compare(void *p1, void *p2) {/* Any possible value should be 'thread' */ struct thread *thread = *(struct thread **)p1;@@ -145,7 +145,7 @@ __no_tail_call noinline int test_dwarf_unwind__compare(void *p1, void *p2)return p1 - p2; } -__no_tail_call noinline int test_dwarf_unwind__krava_3(struct thread *thread) +noinline int test_dwarf_unwind__krava_3(struct thread *thread) {struct thread *array[2 ] = {thread, thread};void *fp = &bsearch;@@ -164,12 +164,12 @@ __no_tail_call noinline int test_dwarf_unwind__krava_3(struct thread *thread)return global_unwind_retval; } -__no_tail_call noinline int test_dwarf_unwind__krava_2(struct thread *thread) +noinline int test_dwarf_unwind__krava_2(struct thread *thread) {return test_dwarf_unwind__krava_3 (thread); } -__no_tail_call noinline int test_dwarf_unwind__krava_1(struct thread *thread) +noinline int test_dwarf_unwind__krava_1(struct thread *thread) {return test_dwarf_unwind__krava_2 (thread); }
Quoted 13 lines · Daniel Borkmann
Quoted 10 lines · Ard Biesheuvel
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.However, as the GCC manual documents, __attribute__((optimize)) is not for production use, and results in all other optimization options to be forgotten for the function in question. This can cause all kinds of trouble, but in one particular reported case,
Quoted 6 lines
Looks like there are couple more as well aside from
__no_fgcse, are you also planning to fix them?arch/powerpc/kernel/setup.h:14:#define __nostackprotector __attribute__((__optimize__("no-stack-protector"))) tools/include/linux/compiler-gcc.h:37:#define __no_tail_call __attribute__((optimize("no-optimize-sibling-calls")))
No, but we can notify the respective maintainers.
Quoted 9 lines
Quoted 3 lines
it causes -fno-asynchronous-unwind-tables to be disregarded, resulting in .eh_frame info to be emitted for the function inadvertently.
Would have been useful to add a pointer to the original discussion with Link tag.
Agreed.
Quoted 17 lines
Quoted 16 lines
This reverts commit
3193c0836f203, and instead, it disables the -fgcse optimization for the entire source file, but only when building for X86.Cc Nick Desaulniers <[email protected]>Cc Arvind Sankar <[email protected]>Cc Randy Dunlap <[email protected]>Cc Josh Poimboeuf <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Alexei Starovoitov <[email protected]>Cc Daniel Borkmann <[email protected]>Cc Peter Zijlstra (Intel) <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Kees Cook <[email protected]>Fixes 3193c0836f203 ("bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()")Signed-off-by Ard Biesheuvel <[email protected]>[...]
Quoted 16 lines
kernel/bpf/Makefile line 5
Quoted 11 lines
kernel/bpf/Makefile +3
-1@@ -1,6 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 obj-y := core.o -CFLAGS_core.o += $(call cc-disable-warning, override-init) +# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details +cflags-core-$(CONFIG_X86) := -fno-gcse +CFLAGS_core.o += $(call cc-disable-warning, override-init) $(cflags-core-y)Also, this needs to be guarded behind !CONFIG_RETPOLINE and !CONFIG_BPF_JIT_ALWAYS_ON in particular the latter since only in this case interpreter is compiled in ... most distros have the
CONFIG_BPF_JIT_ALWAYS_ONset these days for x86.
Is that a new requirement? Because before this patch, -fno-gcse was applied unconditionally.
Quoted 3 lines
Do you have an analysis for the commit log on what else this penalizes in
core.cif it's now for the entire translation unit?
No, I simply observed the regression this caused on non-x86 architectures, and proposed a way to fix it.
Do you have any concerns in particular regarding other things in core.c? Would you prefer ___bpf_prog_run() to be moved into a separate .c file?
Quoted 17 lines
Quoted 16 lines
kernel/bpf/Makefile +0
-0obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o bpf_iter.o map_iter.o task_iter.o prog_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.okernel/bpf/core.c +1
-1@@ -1369,7 +1369,7 @@ u64 __weak bpf_probe_read_kernel(void *dst, u32 size, const void *unsafe_ptr)* * Decode and execute eBPF instructions. */ -static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) +static u64 ___bpf_prog_run (u64 *regs,const struct bpf_insn *insn,u64 *stack) {#define BPF_INSN_2_LBL (x, y) [BPF_##x | BPF_##y] = &&x##_##y#define BPF_INSN_3_LBL (x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
From: Ard Biesheuvel
Quoted 7 lines
Sent: 27 October 2020 20:57
Commit
3193c0836f203("bpf: Disable GCC -fgcse optimization for___bpf_prog_run()") introduced a__no_fgcsemacro that expands to a function scope __attribute__((optimize("-fno-gcse"))), to disable a GCC specific optimization that was causing trouble on x86 builds, and was not expected to have any positive effect in the first place.
Surely it is possibly to 'adjust' the bpf code so that gcc doesn't apply (and can't apply) the gcse optimisation?
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)