Skip to contentSkip to conversation

bpf,ftrace: bpf dispatcher function fix

PATCH Superseded Jiri Olsa 3 participants 7 messages Sep 1, 2022 – Sep 3, 2022

Subsystems (6)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

v2 cover bpf,ftrace: bpf dispatcher function fix

hi,
as discussed [1] sending fix that moves bpf dispatcher function of out ftrace locations together with Peter's HAVE_DYNAMIC_FTRACE_NO_PATCHABLE dependency change.

v2 changes:
- fixing s390x CI build failure by enabling attributes only for x86_64 [Ilya Leoshkevic]

thanks,
jirka

[1] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (1):
bpf: Move bpf_dispatcher function out of ftrace locations

Peter Zijlstra (Intel) (1):
ftrace: Add HAVE_DYNAMIC_FTRACE_NO_PATCHABLE

 arch/x86/Kconfig                  |  1 +
 include/asm-generic/vmlinux.lds.h | 11 ++++++++++-
 include/linux/bpf.h               |  8 ++++++++
 kernel/trace/Kconfig              |  6 ++++++
 tools/objtool/check.c             |  3 ++-
 5 files changed, 27 insertions(+), 2 deletions(-)
author reviewer
#1 lore raw

PATCH v2 1/2 ftrace: Add HAVE_DYNAMIC_FTRACE_NO_PATCHABLE

From: "Peter Zijlstra (Intel)" <[email protected]>

x86 will shortly start using -fpatchable-function-entry for purposes other than ftrace, make sure the __patchable_function_entry section isn't merged in the mcount_loc section.

---
 include/asm-generic/vmlinux.lds.h | 11 ++++++++++-
 kernel/trace/Kconfig              |  6 ++++++
 tools/objtool/check.c             |  3 ++-
 3 files changed, 18 insertions(+), 2 deletions(-)
include/asm-generic/vmlinux.lds.h +10 -1
index 7515a465ec03..13b197ef0d63 100644
@@ -154,6 +154,14 @@
 #define MEM_DISCARD(sec) *(.mem##sec)
 #endif
 
+#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_NO_PATCHABLE
+#define KEEP_PATCHABLE		KEEP(*(__patchable_function_entries))
+#define PATCHABLE_DISCARDS
+#else
+#define KEEP_PATCHABLE
+#define PATCHABLE_DISCARDS	*(__patchable_function_entries)
+#endif
+
 #ifdef CONFIG_FTRACE_MCOUNT_RECORD
 /*
  * The ftrace call sites are logged to a section whose name depends on the
@@ -172,7 +180,7 @@
 #define MCOUNT_REC()	. = ALIGN(8);				\
 			__start_mcount_loc = .;			\
 			KEEP(*(__mcount_loc))			\
-			KEEP(*(__patchable_function_entries))	\
+			KEEP_PATCHABLE				\
 			__stop_mcount_loc = .;			\
 			ftrace_stub_graph = ftrace_stub;	\
 			ftrace_ops_list_func = arch_ftrace_ops_list_func;
@@ -1024,6 +1032,7 @@
 
 #define COMMON_DISCARDS							\
 	SANITIZER_DISCARDS						\
+	PATCHABLE_DISCARDS						\
 	*(.discard)							\
 	*(.discard.*)							\
 	*(.modinfo)							\
kernel/trace/Kconfig +6 -0
index 1052126bdca2..e9e95c790b8e 100644
@@ -51,6 +51,12 @@ config HAVE_DYNAMIC_FTRACE_WITH_ARGS
 	 This allows for use of regs_get_kernel_argument() and
 	 kernel_stack_pointer().
 
+config HAVE_DYNAMIC_FTRACE_NO_PATCHABLE
+	bool
+	help
+	  If the architecture generates __patchable_function_entries sections
+	  but does not want them included in the ftrace locations.
+
 config HAVE_FTRACE_MCOUNT_RECORD
 	bool
 	help
tools/objtool/check.c +2 -1
index 0cec74da7ffe..f23e8d11f6d4 100644
@@ -4110,7 +4110,8 @@ static int validate_ibt(struct objtool_file *file)
 		    !strcmp(sec->name, "__bug_table")			||
 		    !strcmp(sec->name, "__ex_table")			||
 		    !strcmp(sec->name, "__jump_table")			||
-		    !strcmp(sec->name, "__mcount_loc"))
+		    !strcmp(sec->name, "__mcount_loc")			||
+		    strstr(sec->name, "__patchable_function_entries"))
 			continue;
 
 		list_for_each_entry(reloc, &sec->reloc->reloc_list, list)
Signature · 1 line
2.37.2
author reviewer
#2 lore raw

PATCH v2 2/2 bpf: Move bpf_dispatcher function out of ftrace locations

In reply to Jiri Olsa

The dispatcher function is attached/detached to trampoline by dispatcher update function. At the same time it's available as ftrace attachable function.

After discussion [1] the proposed solution is to use compiler attributes to alter bpf_dispatcher_##name##_func function:

- remove it from being instrumented with __no_instrument_function__ attribute, so ftrace has no track of it

- but still generate 5 nop instructions with patchable_function_entry(5) attribute, which are expected by bpf_arch_text_poke used by dispatcher update function

Enabling HAVE_DYNAMIC_FTRACE_NO_PATCHABLE option for x86, so __patchable_function_entries functions are not part of ftrace/mcount locations.

Adding attributes to bpf_dispatcher_XXX function on x86_64 so it's kept out of ftrace locations and has 5 byte nop generated at entry.

These attributes need to be arch specific as pointer out by Ilya Leoshkevic in here [2].

The dispatcher image is generated only for x86_64 arch, so the code can stay as is for other archs.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

---
 arch/x86/Kconfig    | 1 +
 include/linux/bpf.h | 8 ++++++++
 2 files changed, 9 insertions(+)
arch/x86/Kconfig +1 -0
index f9920f1341c8..089c20cefd2b 100644
@@ -284,6 +284,7 @@ config X86
 	select PROC_PID_ARCH_STATUS		if PROC_FS
 	select HAVE_ARCH_NODE_DEV_GROUP		if X86_SGX
 	imply IMA_SECURE_AND_OR_TRUSTED_BOOT    if EFI
+	select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE
 
 config INSTRUCTION_DECODER
 	def_bool y
include/linux/bpf.h +8 -0
index 9c1674973e03..4ab4b0a1beb8 100644
@@ -924,7 +924,15 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs);
 	},							\
 }
 
+#ifdef CONFIG_X86_64
+#define BPF_DISPATCHER_ATTRIBUTES __attribute__((__no_instrument_function__)) \
+				   __attribute__((patchable_function_entry(5)))
+#else
+#define BPF_DISPATCHER_ATTRIBUTES
+#endif
+
 #define DEFINE_BPF_DISPATCHER(name)					\
+	BPF_DISPATCHER_ATTRIBUTES					\
 	noinline __nocfi unsigned int bpf_dispatcher_##name##_func(	\
 		const void *ctx,					\
 		const struct bpf_insn *insnsi,				\
Signature · 1 line
2.37.2
maintainer
#3 lore raw

Re: [PATCHv2 bpf-next 2/2] bpf: Move bpf_dispatcher function out of ftrace locations

Quoted 10 lines · Jiri Olsa
include/linux/bpf.h +2 -0
index 9c1674973e03..4ab4b0a1beb8 100644
@@ -924,7 +924,15 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs);
 	},							\
 }
 
+#ifdef CONFIG_X86_64
+#define BPF_DISPATCHER_ATTRIBUTES __attribute__((__no_instrument_function__)) \

include/linux/bpf.h int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); · line 938

Quoted 10 lines
include/linux/bpf.h +6 -0
+				   __attribute__((patchable_function_entry(5)))
+#else
+#define BPF_DISPATCHER_ATTRIBUTES
+#endif
+
 #define DEFINE_BPF_DISPATCHER(name)					\
+	BPF_DISPATCHER_ATTRIBUTES					\
 	noinline __nocfi unsigned int bpf_dispatcher_##name##_func(	\
 		const void *ctx,					\
 		const struct bpf_insn *insnsi,				\

Are you sure you want the notrace x86_64 only?

That is, perhaps something like this...

+#ifdef CONFIG_X86_64
+#define BPF_DISPATCHER_ATTRIBUTES __attribute__((patchable_function_entry(5)))
+#else
+#define BPF_DISPATCHER_ATTRIBUTES
+#endif
+
#define DEFINE_BPF_DISPATCHER(name) \
+ notrace BPF_DISPATCHER_ATTRIBUTES \
noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \

#4 lore raw

Re: [PATCHv2 bpf-next 2/2] bpf: Move bpf_dispatcher function out of ftrace locations

Quoted 28 lines · Peter Zijlstra
Quoted 10 lines · Jiri Olsa
include/linux/bpf.h +2 -0
index 9c1674973e03..4ab4b0a1beb8 100644
@@ -924,7 +924,15 @@ int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs);
 	},							\
 }
 
+#ifdef CONFIG_X86_64
+#define BPF_DISPATCHER_ATTRIBUTES __attribute__((__no_instrument_function__)) \

include/linux/bpf.h int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); · line 938

Quoted 10 lines
include/linux/bpf.h +6 -0
+				   __attribute__((patchable_function_entry(5)))
+#else
+#define BPF_DISPATCHER_ATTRIBUTES
+#endif
+
 #define DEFINE_BPF_DISPATCHER(name)					\
+	BPF_DISPATCHER_ATTRIBUTES					\
 	noinline __nocfi unsigned int bpf_dispatcher_##name##_func(	\
 		const void *ctx,					\
 		const struct bpf_insn *insnsi,				\

Are you sure you want the notrace x86_64 only?

That is, perhaps something like this...

include/linux/bpf.h int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); · line 936

Quoted 9 lines
include/linux/bpf.h +7 -0
+#ifdef CONFIG_X86_64
+#define BPF_DISPATCHER_ATTRIBUTES	   __attribute__((patchable_function_entry(5)))
+#else
+#define BPF_DISPATCHER_ATTRIBUTES
+#endif
+
 #define DEFINE_BPF_DISPATCHER(name)					\
+	notrace BPF_DISPATCHER_ATTRIBUTES				\
 	noinline __nocfi unsigned int bpf_dispatcher_##name##_func(	\

that's also an option.. but I don't this it's big deal that the function is traceable on other arches, because the dispatcher image is generated only on x86, so no other arch is touching that function entry, so it's safe for ftrace to attach

jirka

maintainer
#5 lore raw

Re: [PATCHv2 bpf-next 2/2] bpf: Move bpf_dispatcher function out of ftrace locations

Quoted 4 lines · Jiri Olsa
Quoted 4 lines

Are you sure you want the notrace x86_64 only?

That is, perhaps something like this...

Quoted 15 lines

include/linux/bpf.h int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); · line 936

Quoted 10 lines
include/linux/bpf.h +7 -0
+#ifdef CONFIG_X86_64
+#define BPF_DISPATCHER_ATTRIBUTES	   __attribute__((patchable_function_entry(5)))
+#else
+#define BPF_DISPATCHER_ATTRIBUTES
+#endif
+
 #define DEFINE_BPF_DISPATCHER(name)					\
+	notrace BPF_DISPATCHER_ATTRIBUTES				\
 	noinline __nocfi unsigned int bpf_dispatcher_##name##_func(	\

that's also an option.. but I don't this it's big deal that the function is traceable on other arches, because the dispatcher image is generated only on x86, so no other arch is touching that function entry, so it's safe for ftrace to attach

It just seems like a pointless difference.

From a code-gen POV you don't strictly need the notrace; without it it'll generate:

bpf_dispatcher_name_func:

	nop
	nop
	nop
	nop
	nop
	call __fentry__
	RET

It'll just function 'weird', but it'll 'work'.

#6 lore raw

Re: [PATCHv2 bpf-next 2/2] bpf: Move bpf_dispatcher function out of ftrace locations

Quoted 21 lines · Peter Zijlstra
Quoted 4 lines · Jiri Olsa
Quoted 4 lines

Are you sure you want the notrace x86_64 only?

That is, perhaps something like this...

Quoted 15 lines

include/linux/bpf.h int arch_prepare_bpf_dispatcher(void *image, s64 *funcs, int num_funcs); · line 936

Quoted 10 lines
include/linux/bpf.h +7 -0
+#ifdef CONFIG_X86_64
+#define BPF_DISPATCHER_ATTRIBUTES	   __attribute__((patchable_function_entry(5)))
+#else
+#define BPF_DISPATCHER_ATTRIBUTES
+#endif
+
 #define DEFINE_BPF_DISPATCHER(name)					\
+	notrace BPF_DISPATCHER_ATTRIBUTES				\
 	noinline __nocfi unsigned int bpf_dispatcher_##name##_func(	\

that's also an option.. but I don't this it's big deal that the function is traceable on other arches, because the dispatcher image is generated only on x86, so no other arch is touching that function entry, so it's safe for ftrace to attach

Quoted 1 line

It just seems like a pointless difference.

ok, I'll use notrace as default

Quoted 13 lines

From a code-gen POV you don't strictly need the notrace; without it it'll generate:

bpf_dispatcher_name_func:

	nop
	nop
	nop
	nop
	nop
	call __fentry__
	RET
Quoted 1 line

It'll just function 'weird', but it'll 'work'.

I'll use notrace for all archs as you suggests, so there's less surprise in future if some other arch implements dispatcher image

thanks,
jirka