bpf,ftrace: bpf dispatcher function fix
Subsystems (6)
Jiri Olsa
reviewercover bpf,ftrace: bpf dispatcher function fix
hi,
as discussed [1] sending fix that moves bpf dispatcher function of out ftrace locations together with Peter's HAVE_DYNAMIC_FTRACE_NO_PATCHABLE dependency change.
thanks,
jirka
[1] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (1):
bpf: Move bpf_dispatcher function out of ftrace locations
Peter Zijlstra (Intel) (1):
ftrace: Add HAVE_DYNAMIC_FTRACE_NO_PATCHABLE
arch/x86/Kconfig | 1 + include/asm-generic/vmlinux.lds.h | 11 ++++++++++- include/linux/bpf.h | 2 ++ kernel/trace/Kconfig | 6 ++++++ tools/objtool/check.c | 3 ++- 5 files changed, 21 insertions(+), 2 deletions(-)
Jiri Olsa
reviewerPATCH 1/2 ftrace: Add HAVE_DYNAMIC_FTRACE_NO_PATCHABLE
From: "Peter Zijlstra (Intel)" <[email protected]>
x86 will shortly start using -fpatchable-function-entry for purposes other than ftrace, make sure the __patchable_function_entry section isn't merged in the mcount_loc section.
--- include/asm-generic/vmlinux.lds.h | 11 ++++++++++- kernel/trace/Kconfig | 6 ++++++ tools/objtool/check.c | 3 ++- 3 files changed, 18 insertions(+), 2 deletions(-)
include/asm-generic/vmlinux.lds.h +10 -1
@@ -154,6 +154,14 @@#define MEM_DISCARD (sec) *(.mem##sec)#endif +#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_NO_PATCHABLE +#define KEEP_PATCHABLE KEEP(*(__patchable_function_entries)) +#define PATCHABLE_DISCARDS +#else +#define KEEP_PATCHABLE +#define PATCHABLE_DISCARDS *(__patchable_function_entries) +#endif +#ifdef CONFIG_FTRACE_MCOUNT_RECORD /* * The ftrace call sites are logged to a section whose name depends on the
@@ -172,7 +180,7 @@#define MCOUNT_REC () . = ALIGN(8); \ __start_mcount_loc = .; \ KEEP(*(__mcount_loc)) \ - KEEP(*(__patchable_function_entries)) \ + KEEP_PATCHABLE \ __stop_mcount_loc = .; \ ftrace_stub_graph = ftrace_stub; \ ftrace_ops_list_func = arch_ftrace_ops_list_func;
@@ -1024,6 +1032,7 @@#define COMMON_DISCARDS \ SANITIZER_DISCARDS \ + PATCHABLE_DISCARDS \ *(.discard) \ *(.discard.*) \ *(.modinfo) \
kernel/trace/Kconfig +6 -0
@@ -51,6 +51,12 @@ config HAVE_DYNAMIC_FTRACE_WITH_ARGS This allows for use of regs_get_kernel_argument() and kernel_stack_pointer(). +config HAVE_DYNAMIC_FTRACE_NO_PATCHABLE + bool + help + If the architecture generates __patchable_function_entries sections + but does not want them included in the ftrace locations. + config HAVE_FTRACE_MCOUNT_RECORD bool help
tools/objtool/check.c +2 -1
@@ -4110,7 +4110,8 @@ static int validate_ibt(struct objtool_file *file) !strcmp (sec->name,"__bug_table" ) || !strcmp (sec->name,"__ex_table" ) || !strcmp (sec->name,"__jump_table" ) || - !strcmp(sec->name, "__mcount_loc")) + !strcmp (sec->name,"__mcount_loc" ) || + strstr(sec->name, "__patchable_function_entries")) continue;list_for_each_entry (reloc, &sec->reloc->reloc_list, list)
Signature · 1 line
2.37.2
Jiri Olsa
reviewerPATCH 2/2 bpf: Move bpf_dispatcher function out of ftrace locations
In reply to Jiri OlsaThe dispatcher function is attached/detached to trampoline by dispatcher update function. At the same time it's available as ftrace attachable function.
After discussion [1] the proposed solution is to use compiler attributes to alter bpf_dispatcher_##name##_func function:
- remove it from being instrumented with __no_instrument_function__ attribute, so ftrace has no track of it
- but still generate 5 nop instructions with patchable_function_entry(5) attribute, which are expected by bpf_arch_text_poke used by dispatcher update function
Enabling HAVE_DYNAMIC_FTRACE_NO_PATCHABLE option for x86, so __patchable_function_entries functions are not part of ftrace/mcount locations.
The dispatcher code is generated and attached only for x86 so it's safe to keep bpf_dispatcher func in patchable_function_entry locations for other archs.
[1] https://lore.kernel.org/bpf/[email protected]/
--- arch/x86/Kconfig | 1 + include/linux/bpf.h | 2 ++ 2 files changed, 3 insertions(+)
arch/x86/Kconfig +1 -0
@@ -284,6 +284,7 @@ config X86 select PROC_PID_ARCH_STATUS if PROC_FS select HAVE_ARCH_NODE_DEV_GROUP if X86_SGX imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI + select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE config INSTRUCTION_DECODER def_bool y
include/linux/bpf.h +2 -0
@@ -925,6 +925,8 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); }#define DEFINE_BPF_DISPATCHER (name) \ + __attribute__((__no_instrument_function__)) \ + __attribute__((patchable_function_entry(5))) \ noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \ const void *ctx, \ const struct bpf_insn *insnsi, \
Signature · 1 line
2.37.2
Re: [PATCH bpf-next 2/2] bpf: Move bpf_dispatcher function out of ftrace locations
Quoted 19 lines · Jiri Olsa
The dispatcher function is attached/detached to trampoline by dispatcher update function. At the same time it's available as ftrace attachable function.
After discussion [1] the proposed solution is to use compiler attributes to alter bpf_dispatcher_##name##_func function:
- remove it from being instrumented with
__no_instrument_function__attribute, so ftrace has no track of it- but still generate 5 nop instructions with
patchable_function_entry(5)attribute, which are expected bybpf_arch_text_pokeused by dispatcher update functionEnabling
HAVE_DYNAMIC_FTRACE_NO_PATCHABLEoption for x86, so__patchable_function_entriesfunctions are not part of ftrace/mcount locations.
Quoted 7 lines
The dispatcher code is generated and attached only for x86 so it's safe to keep
bpf_dispatcherfunc inpatchable_function_entrylocations for other archs.[1] https://lore.kernel.org/bpf/[email protected]/
Suggested-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
I read the whole discussion in
https://lore.kernel.org/bpf/CAADnVQLMY9Gbipx6Tc4dsRN3YSXo2CF8T=KwAvjU7nRDjYxRgQ@mail.gmail.com/
and this LGTM
Quoted 33 lines
--- arch/x86/Kconfig | 1 + include/linux/bpf.h | 2 ++ 2 files changed, 3 insertions(+)arch/x86/Kconfig +1
-0@@ -284,6 +284,7 @@ config X86 select PROC_PID_ARCH_STATUS if PROC_FS select HAVE_ARCH_NODE_DEV_GROUP if X86_SGX imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI + select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE config INSTRUCTION_DECODER def_bool yinclude/linux/bpf.h +2
-1@@ -925,6 +925,8 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); }#define DEFINE_BPF_DISPATCHER (name) \ + __attribute__((__no_instrument_function__)) \ + __attribute__((patchable_function_entry(5))) \ noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \ const void *ctx, \ const struct bpf_insn *insnsi, \ --2.37.2
Peter Zijlstra
maintainerRe: [PATCH bpf-next 2/2] bpf: Move bpf_dispatcher function out of ftrace locations
In reply to Jiri OlsaQuoted 9 lines · Jiri Olsa
The dispatcher function is attached/detached to trampoline by dispatcher update function. At the same time it's available as ftrace attachable function.
After discussion [1] the proposed solution is to use compiler attributes to alter bpf_dispatcher_##name##_func function:
- remove it from being instrumented with
__no_instrument_function__attribute, so ftrace has no track of it
This is typically spelled like: 'notrace' in the kernel.
Quoted 37 lines
- but still generate 5 nop instructions with
patchable_function_entry(5)attribute, which are expected bybpf_arch_text_pokeused by dispatcher update functionEnabling
HAVE_DYNAMIC_FTRACE_NO_PATCHABLEoption for x86, so__patchable_function_entriesfunctions are not part of ftrace/mcount locations.The dispatcher code is generated and attached only for x86 so it's safe to keep
bpf_dispatcherfunc inpatchable_function_entrylocations for other archs.[1] https://lore.kernel.org/bpf/[email protected]/
Suggested-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- arch/x86/Kconfig | 1 + include/linux/bpf.h | 2 ++ 2 files changed, 3 insertions(+)arch/x86/Kconfig +1
-0@@ -284,6 +284,7 @@ config X86 select PROC_PID_ARCH_STATUS if PROC_FS select HAVE_ARCH_NODE_DEV_GROUP if X86_SGX imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI + select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE config INSTRUCTION_DECODER def_bool yinclude/linux/bpf.h +0
-0
include/linux/bpf.h int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); · line 932
Quoted 8 lines
include/linux/bpf.h +2
-0@@ -925,6 +925,8 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); }#define DEFINE_BPF_DISPATCHER (name) \ + __attribute__((__no_instrument_function__)) \ + __attribute__((patchable_function_entry(5))) \ noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \ const void *ctx, \ const struct bpf_insn *insnsi, \
What makes that whole dispatcher thing x86 only? AFAICT it is only under BPF_JIT here and could be used by anyone.
ARM64 for instance has BPG_JIT and builds net/core/filter.c. And ARM64 very much does use patchable_function_entry() for its ftrace implementation.
Re: [PATCH bpf-next 2/2] bpf: Move bpf_dispatcher function out of ftrace locations
Quoted 52 lines · Peter Zijlstra
Quoted 9 lines · Jiri Olsa
The dispatcher function is attached/detached to trampoline by dispatcher update function. At the same time it's available as ftrace attachable function.
After discussion [1] the proposed solution is to use compiler attributes to alter bpf_dispatcher_##name##_func function:
- remove it from being instrumented with
__no_instrument_function__attribute, so ftrace has no track of itThis is typically spelled like: 'notrace' in the kernel.
Quoted 39 lines
- but still generate 5 nop instructions with
patchable_function_entry(5)attribute, which are expected bybpf_arch_text_pokeused by dispatcher update functionEnabling
HAVE_DYNAMIC_FTRACE_NO_PATCHABLEoption for x86, so__patchable_function_entriesfunctions are not part of ftrace/mcount locations.The dispatcher code is generated and attached only for x86 so it's safe to keep
bpf_dispatcherfunc inpatchable_function_entrylocations for other archs.[1] https://lore.kernel.org/bpf/[email protected]/
Suggested-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- arch/x86/Kconfig | 1 + include/linux/bpf.h | 2 ++ 2 files changed, 3 insertions(+)arch/x86/Kconfig +1
-0@@ -284,6 +284,7 @@ config X86 select PROC_PID_ARCH_STATUS if PROC_FS select HAVE_ARCH_NODE_DEV_GROUP if X86_SGX imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI + select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE config INSTRUCTION_DECODER def_bool yinclude/linux/bpf.h +0
-0@@ -925,6 +925,8 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); }
Quoted 9 lines
include/linux/bpf.h int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); · line 932
Quoted 6 lines
include/linux/bpf.h +2
-0#define DEFINE_BPF_DISPATCHER (name) \ + __attribute__((__no_instrument_function__)) \ + __attribute__((patchable_function_entry(5))) \ noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \ const void *ctx, \ const struct bpf_insn *insnsi, \What makes that whole dispatcher thing x86 only? AFAICT it is only under
BPF_JIThere and could be used by anyone.
it is just optimalization on x86, that transform default indirect calls to direct call, described in changelog in here:
75ccbef6369e bpf: Introduce BPF dispatcher
other archs just make the call to bpf_func
jirka
Quoted 4 lines
ARM64 for instance has
BPG_JITand buildsnet/core/filter.c. And ARM64 very much does usepatchable_function_entry()for its ftrace implementation.
Daniel Borkmann
maintainerQuoted 4 lines · Jiri Olsa
hi,
as discussed [1] sending fix that moves bpf dispatcher function of out ftrace locations together with Peter'sHAVE_DYNAMIC_FTRACE_NO_PATCHABLEdependency change.
Looks like the series breaks s390x builds; BPF CI link:
https://github.com/kernel-patches/bpf/runs/8079411784?check_suite_focus=true
[...]
CC net/xfrm/xfrm_state.o
CC net/packet/af_packet.o
{standard input}: Assembler messages:
{standard input}:16055: Error: bad expression
{standard input}:16056: Error: bad expression
{standard input}:16057: Error: bad expression
{standard input}:16058: Error: bad expression
{standard input}:16059: Error: bad expression
CC drivers/s390/char/raw3270.o
CC net/ipv6/ip6_output.o
[...]
CC net/xfrm/xfrm_output.o
CC net/ipv6/ip6_input.o
{standard input}:16055: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16056: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16057: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16058: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16059: Error: invalid operands (*ABS* and *UND* sections) for `%'
make[3]: *** [scripts/Makefile.build:249: net/core/filter.o] Error 1
make[2]: *** [scripts/Makefile.build:465: net/core] Error 2
make[2]: *** Waiting for unfinished jobs....
CC net/ipv4/tcp_fastopen.o
[...]
CC lib/percpu-refcount.o
make[1]: *** [Makefile:1855: net] Error 2
CC lib/rhashtable.o
make[1]: *** Waiting for unfinished jobs....
CC lib/base64.o
[...]
AR lib/built-in.a
CC kernel/kheaders.o
AR kernel/built-in.a make: *** [Makefile:353: __build_one_by_one] Error 2
Error: Process completed with exit code 2.
Quoted 15 lines
[1] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (1):
bpf: Movebpf_dispatcherfunction out of ftrace locationsPeter Zijlstra (Intel) (1):
ftrace: AddHAVE_DYNAMIC_FTRACE_NO_PATCHABLEarch/x86/Kconfig | 1 + include/asm-generic/vmlinux.lds.h | 11 ++++++++++- include/linux/bpf.h | 2 ++ kernel/trace/Kconfig | 6 ++++++ tools/objtool/check.c | 3 ++- 5 files changed, 21 insertions(+), 2 deletions(-)
Quoted 43 lines · Daniel Borkmann
Quoted 4 lines · Jiri Olsa
hi,
as discussed [1] sending fix that moves bpf dispatcher function of out ftrace locations together with Peter'sHAVE_DYNAMIC_FTRACE_NO_PATCHABLEdependency change.Looks like the series breaks s390x builds; BPF CI link:
https://github.com/kernel-patches/bpf/runs/8079411784?check_suite_focus=true
[...]
CCnet/xfrm/xfrm_state.o
CCnet/packet/af_packet.o
{standard input}: Assembler messages:
{standard input}:16055: Error: bad expression
{standard input}:16056: Error: bad expression
{standard input}:16057: Error: bad expression
{standard input}:16058: Error: bad expression
{standard input}:16059: Error: bad expressionCC drivers/s390/char/raw3270.o CC net/ipv6/ip6_output.o [...] CC net/xfrm/xfrm_output.o CC net/ipv6/ip6_input.o {standard input}:16055: Error: invalid operands (*ABS* and *UND* sections) for `%' {standard input}:16056: Error: invalid operands (*ABS* and *UND* sections) for `%' {standard input}:16057: Error: invalid operands (*ABS* and *UND* sections) for `%' {standard input}:16058: Error: invalid operands (*ABS* and *UND* sections) for `%' {standard input}:16059: Error: invalid operands (*ABS* and *UND* sections) for `%' make[3]: *** [scripts/Makefile.build:249: net/core/filter.o] Error 1 make[2]: *** [scripts/Makefile.build:465: net/core] Error 2 make[2]: *** Waiting for unfinished jobs.... CC net/ipv4/tcp_fastopen.o [...] CC lib/percpu-refcount.o make[1]: *** [Makefile:1855: net] Error 2 CC lib/rhashtable.o make[1]: *** Waiting for unfinished jobs.... CC lib/base64.o [...] AR lib/built-in.a CC kernel/kheaders.o AR kernel/built-in.a
Quoted 2 lines
make: *** [Makefile:353:
__build_one_by_one] Error 2
Error: Process completed with exit code 2.
it does not break on my cross build with gcc 12, but I can reproduce with gcc 8 (CI seems to be on gcc 9)
the problem seems to be wrong assembler code with extra '%' that's generated for patchable_function_entry(5)
gcc 8 generates:
.LPFE1:
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0and gcc 12 generates:
.LPFE1:
nopr %r0
nopr %r0
nopr %r0
nopr %r0
nopr %r0perhaps we need to upgrade gcc in CI? cc-ing Ilya, any idea?
thanks,
jirka
Quoted 17 lines
Quoted 16 lines
[1] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (1):
bpf: Movebpf_dispatcherfunction out of ftrace locationsPeter Zijlstra (Intel) (1):
ftrace: AddHAVE_DYNAMIC_FTRACE_NO_PATCHABLEarch/x86/Kconfig | 1 + include/asm-generic/vmlinux.lds.h | 11 ++++++++++- include/linux/bpf.h | 2 ++ kernel/trace/Kconfig | 6 ++++++ tools/objtool/check.c | 3 ++- 5 files changed, 21 insertions(+), 2 deletions(-)
Quoted 80 lines · Jiri Olsa
Quoted 45 lines · Daniel Borkmann
Quoted 6 lines · Jiri Olsa
hi,
as discussed [1] sending fix that moves bpf dispatcher function of out
ftrace locations together with Peter'sHAVE_DYNAMIC_FTRACE_NO_PATCHABLE
dependency change.Looks like the series breaks s390x builds; BPF CI link:
https://github.com/kernel-patches/bpf/runs/8079411784?check_suite_focus=true
[...]
CCnet/xfrm/xfrm_state.o
CCnet/packet/af_packet.o
{standard input}: Assembler messages:
{standard input}:16055: Error: bad expression {standard input}:16056: Error: bad expression {standard input}:16057: Error: bad expression {standard input}:16058: Error: bad expression {standard input}:16059: Error: bad expression
CCdrivers/s390/char/raw3270.o
CCnet/ipv6/ip6_output.o
[...]
CCnet/xfrm/xfrm_output.o
CCnet/ipv6/ip6_input.o
{standard input}:16055: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16056: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16057: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16058: Error: invalid operands (*ABS* and *UND* sections) for `%'
{standard input}:16059: Error: invalid operands (*ABS* and *UND* sections) for `%'
make[3]: *** [scripts/Makefile.build:249:net/core/filter.o] Error 1
make[2]: *** [scripts/Makefile.build:465: net/core] Error 2 make[2]: *** Waiting for unfinished jobs....
CCnet/ipv4/tcp_fastopen.o
[...]
CClib/percpu-refcount.o
make[1]: *** [Makefile:1855: net] Error 2
CClib/rhashtable.oQuoted 8 lines
make[1]: *** Waiting for unfinished jobs.... CC
lib/base64.o
[...]
ARlib/built-in.a
CCkernel/kheaders.o
ARkernel/built-in.a
make: *** [Makefile:353:__build_one_by_one] Error 2
Error: Process completed with exit code 2.it does not break on my cross build with gcc 12, but I can reproduce with gcc 8 (CI seems to be on gcc 9)
the problem seems to be wrong assembler code with extra '%' that's generated for
patchable_function_entry(5)gcc 8 generates:
.LPFE1:
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0and gcc 12 generates:
.LPFE1:
nopr %r0
nopr %r0
nopr %r0
nopr %r0
nopr %r0
Quoted 4 lines
perhaps we need to upgrade gcc in CI? cc-ing Ilya, any idea?
thanks,
jirka
It's not obvious to me which gcc commit fixed this; I will bisect and find out. This will take some time.
However, officially, the kernel must be buildable by gcc 5.1+. Whatever I find, it's unlikely that we'll be able to backport it that far.
Therefore I think we need to find a way to conditionally do something else when using broken gccs. Or maybe just keep this x86-only after all.
Best regards,
Ilya
Quoted 98 lines · Ilya Leoshkevich
Quoted 85 lines · Jiri Olsa
Quoted 50 lines · Daniel Borkmann
Quoted 6 lines · Jiri Olsa
hi,
as discussed [1] sending fix that moves bpf dispatcher function of out
ftrace locations together with Peter'sHAVE_DYNAMIC_FTRACE_NO_PATCHABLE
dependency change.Looks like the series breaks s390x builds; BPF CI link:
https://github.com/kernel-patches/bpf/runs/8079411784?check_suite_focus=true
[...]
CCnet/xfrm/xfrm_state.o
CCnet/packet/af_packet.o
{standard input}: Assembler messages:
{standard input}:16055: Error: bad expression {standard input}:16056: Error: bad expression {standard input}:16057: Error: bad expression {standard input}:16058: Error: bad expression {standard input}:16059: Error: bad expression
CCdrivers/s390/char/raw3270.o
CCnet/ipv6/ip6_output.o
[...]
CCnet/xfrm/xfrm_output.o
CCnet/ipv6/ip6_input.o
{standard input}:16055: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16056: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16057: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16058: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16059: Error: invalid operands (*ABS* and *UND*
sections) for `%'
make[3]: *** [scripts/Makefile.build:249:net/core/filter.o] Error 1
make[2]: *** [scripts/Makefile.build:465: net/core] Error 2 make[2]: *** Waiting for unfinished jobs....
CCnet/ipv4/tcp_fastopen.o
[...]
CClib/percpu-refcount.o
make[1]: *** [Makefile:1855: net] Error 2
CClib/rhashtable.oQuoted 8 lines
make[1]: *** Waiting for unfinished jobs.... CC
lib/base64.o
[...]
ARlib/built-in.a
CCkernel/kheaders.o
ARkernel/built-in.a
make: *** [Makefile:353:__build_one_by_one] Error 2
Error: Process completed with exit code 2.it does not break on my cross build with gcc 12, but I can reproduce with gcc 8 (CI seems to be on gcc 9)
the problem seems to be wrong assembler code with extra '%' that's generated for
patchable_function_entry(5)gcc 8 generates:
.LPFE1:
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0and gcc 12 generates:
.LPFE1:
nopr %r0
nopr %r0
nopr %r0
nopr %r0
nopr %r0Quoted 4 lines
perhaps we need to upgrade gcc in CI? cc-ing Ilya, any idea?
thanks,
jirkaIt's not obvious to me which gcc commit fixed this; I will bisect and find out. This will take some time.
However, officially, the kernel must be buildable by gcc 5.1+. Whatever I find, it's unlikely that we'll be able to backport it that far.
Quoted 6 lines
Therefore I think we need to find a way to conditionally do something else when using broken gccs. Or maybe just keep this x86-only after all.
Best regards,
Ilya
FWIW, bisect points to
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=45d06a4045bebc3dbaaf0b1c676f4e22b7c6aca1
which makes perfect sense. Still, as I mentioned above, it's probably worth tolerating brokens gccs instead of spending time backporting this everywhere. And upgrading the CI machine will only paper over the issue.
At a closer look, it looks weird to me that we have patchable_function_entry(5) in a common header. If this optimization is ever implemented for another architecture, a different number will be required.
For simplicity, would it make sense to hide this under an #ifdef? Something like this (untested):
#ifdef CONFIG_X86
#define BPF_DISPATCHER_ATTRIBUTES
__attribute__((patchable_function_entry(5)))
#else
#define BPF_DISPATCHER_ATTRIBUTES
#endif
Best regards,
Ilya
Quoted 103 lines · Ilya Leoshkevich
Quoted 102 lines · Ilya Leoshkevich
Quoted 85 lines · Jiri Olsa
Quoted 50 lines · Daniel Borkmann
Quoted 6 lines · Jiri Olsa
hi,
as discussed [1] sending fix that moves bpf dispatcher function of out
ftrace locations together with Peter'sHAVE_DYNAMIC_FTRACE_NO_PATCHABLE
dependency change.Looks like the series breaks s390x builds; BPF CI link:
https://github.com/kernel-patches/bpf/runs/8079411784?check_suite_focus=true
[...]
CCnet/xfrm/xfrm_state.o
CCnet/packet/af_packet.o
{standard input}: Assembler messages:
{standard input}:16055: Error: bad expression {standard input}:16056: Error: bad expression {standard input}:16057: Error: bad expression {standard input}:16058: Error: bad expression {standard input}:16059: Error: bad expression
CCdrivers/s390/char/raw3270.o
CCnet/ipv6/ip6_output.o
[...]
CCnet/xfrm/xfrm_output.o
CCnet/ipv6/ip6_input.o
{standard input}:16055: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16056: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16057: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16058: Error: invalid operands (*ABS* and *UND*
sections) for `%'
{standard input}:16059: Error: invalid operands (*ABS* and *UND*
sections) for `%'
make[3]: *** [scripts/Makefile.build:249:net/core/filter.o] Error 1
make[2]: *** [scripts/Makefile.build:465: net/core] Error 2 make[2]: *** Waiting for unfinished jobs....
CCnet/ipv4/tcp_fastopen.o
[...]
CClib/percpu-refcount.o
make[1]: *** [Makefile:1855: net] Error 2
CClib/rhashtable.oQuoted 8 lines
make[1]: *** Waiting for unfinished jobs.... CC
lib/base64.o
[...]
ARlib/built-in.a
CCkernel/kheaders.o
ARkernel/built-in.a
make: *** [Makefile:353:__build_one_by_one] Error 2
Error: Process completed with exit code 2.it does not break on my cross build with gcc 12, but I can reproduce with gcc 8 (CI seems to be on gcc 9)
the problem seems to be wrong assembler code with extra '%' that's generated for
patchable_function_entry(5)gcc 8 generates:
.LPFE1:
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0
nopr %%r0and gcc 12 generates:
.LPFE1:
nopr %r0
nopr %r0
nopr %r0
nopr %r0
nopr %r0Quoted 4 lines
perhaps we need to upgrade gcc in CI? cc-ing Ilya, any idea?
thanks,
jirkaIt's not obvious to me which gcc commit fixed this; I will bisect and find out. This will take some time.
However, officially, the kernel must be buildable by gcc 5.1+. Whatever I find, it's unlikely that we'll be able to backport it that far.
Therefore I think we need to find a way to conditionally do something else when using broken gccs. Or maybe just keep this x86-only after all.
Quoted 6 lines
Quoted 2 lines
Best regards,
IlyaFWIW, bisect points to
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=45d06a4045bebc3dbaaf0b1c676f4e22b7c6aca1
great, thanks for doing that
Quoted 14 lines
which makes perfect sense. Still, as I mentioned above, it's probably worth tolerating brokens gccs instead of spending time backporting this everywhere. And upgrading the CI machine will only paper over the issue.
At a closer look, it looks weird to me that we have
patchable_function_entry(5)in a common header. If this optimization is ever implemented for another architecture, a different number will be required.For simplicity, would it make sense to hide this under an #ifdef? Something like this (untested):
Quoted 6 lines
#ifdef
CONFIG_X86
#defineBPF_DISPATCHER_ATTRIBUTES
__attribute__((patchable_function_entry(5)))
#else
#defineBPF_DISPATCHER_ATTRIBUTES
#endif
right, I think we can limit it directly to x86_64 like below
jirka
---
arch/x86/Kconfig +1 -0
@@ -284,6 +284,7 @@ config X86 select PROC_PID_ARCH_STATUS if PROC_FS select HAVE_ARCH_NODE_DEV_GROUP if X86_SGX imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI + select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE config INSTRUCTION_DECODER def_bool y
include/linux/bpf.h +8 -0
@@ -924,7 +924,15 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); }, \ } +#ifdef CONFIG_X86_64 +#define BPF_DISPATCHER_ATTRIBUTES __attribute__((__no_instrument_function__)) \ + __attribute__((patchable_function_entry(5))) +#else +#define BPF_DISPATCHER_ATTRIBUTES +#endif +#define DEFINE_BPF_DISPATCHER (name) \ + BPF_DISPATCHER_ATTRIBUTES \ noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \ const void *ctx, \ const struct bpf_insn *insnsi, \