bpf: Speed up symbol resolving in kprobe multi link
Subsystems (8)
Jiri Olsa
reviewerv2 cover bpf: Speed up symbol resolving in kprobe multi link
hi,
sending additional fix for symbol resolving in kprobe multi link requested by Alexei and Andrii [1].
This speeds up bpftrace kprobe attachment, when using pure symbols (3344 symbols) to attach:
Before:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }'
...
6.5681 +- 0.0225 seconds time elapsed ( +- 0.34% )After:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }'
...
0.5661 +- 0.0275 seconds time elapsed ( +- 4.85% )v2 changes (first version [2]):
- removed the 2 seconds check [Alexei]
- moving/forcing symbols sorting out of kallsyms_lookup_names function [Alexei]
- skipping one array allocation and copy_from_user [Andrii]
- several small fixes [Masami,Andrii]
- build fix [kernel test robot]
thanks,
jirka
[1] https://lore.kernel.org/bpf/CAEf4BzZtQaiUxQ-sm_hH2qKPRaqGHyOfEsW96DxtBHRaKLoL3Q@mail.gmail.com/
[2] https://lore.kernel.org/bpf/[email protected]/
---
Jiri Olsa (4):
kallsyms: Add kallsyms_lookup_names function
fprobe: Resolve symbols with kallsyms_lookup_names
bpf: Resolve symbols with kallsyms_lookup_names for kprobe multi link selftests/bpf: Add attach bench test
include/linux/kallsyms.h | 6 ++++++ kernel/kallsyms.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- kernel/trace/bpf_trace.c | 113 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------- kernel/trace/fprobe.c | 32 ++++++++++++-------------------- tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 136 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/kprobe_multi_empty.c | 12 ++++++++++++ 6 files changed, 302 insertions(+), 67 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_empty.c
Jiri Olsa
reviewerPATCH v2 1/4 kallsyms: Add kallsyms_lookup_names function
Adding kallsyms_lookup_names function that resolves array of symbols with single pass over kallsyms.
The user provides array of string pointers with count and pointer to allocated array for resolved values.
int kallsyms_lookup_names(const char **syms, size_t cnt,
unsigned long *addrs)
It iterates all kalsyms symbols and tries to loop up each in provided symbols array with bsearch. The symbols array needs to be sorted by name for this reason.
We also check each symbol to pass ftrace_location, because this API will be used for fprobe symbols resolving. This can be optional in future if there's a need.
We need kallsyms_on_each_symbol function, so enabling it and also the new function for CONFIG_FPROBE option.
--- include/linux/kallsyms.h | 6 ++++ kernel/kallsyms.c | 70 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-)
include/linux/kallsyms.h +6 -0
@@ -72,6 +72,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *,#ifdef CONFIG_KALLSYMS /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name (const char *name); +int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs); extern int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize,
@@ -103,6 +104,11 @@ static inline unsigned long kallsyms_lookup_name(const char *name)return 0 ; } +static inline int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs) +{ +return -ERANGE ; +} +static inline int kallsyms_lookup_size_offset (unsigned long addr,unsigned long *symbolsize,unsigned long *offset)
kernel/kallsyms.c +69 -1
88 lines not on this page · raw message
Signature · 1 line
2.35.1
Masami Hiramatsu
maintainerRe: [PATCHv2 bpf-next 1/4] kallsyms: Add kallsyms_lookup_names function
Quoted 8 lines · On Mon, 18 Apr 2022 14:48:31 +0200 Jiri Olsa <[email protected]> wrote:
Adding
kallsyms_lookup_namesfunction that resolves array of symbols with single pass over kallsyms.The user provides array of string pointers with count and pointer to allocated array for resolved values.
int kallsyms_lookup_names(const char **syms,
size_tcnt,
unsigned long *addrs)
What about renaming the 'syms' argument to 'sorted_syms' so that user is easily notice what is required?
Or renaming the function as kallsyms_lookup_sorted_names()?
Quoted 85 lines
It iterates all kalsyms symbols and tries to loop up each in provided symbols array with bsearch. The symbols array needs to be sorted by name for this reason.
We also check each symbol to pass
ftrace_location, because this API will be used for fprobe symbols resolving. This can be optional in future if there's a need.We need
kallsyms_on_each_symbolfunction, so enabling it and also the new function forCONFIG_FPROBEoption.Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- include/linux/kallsyms.h | 6 ++++ kernel/kallsyms.c | 70 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-)include/linux/kallsyms.h +6
-0@@ -72,6 +72,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *,#ifdef CONFIG_KALLSYMS /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name (const char *name); +int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs); extern int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize,@@ -103,6 +104,11 @@ static inline unsigned long kallsyms_lookup_name(const char *name)return 0 ; } +static inline int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs) +{ +return -ERANGE ; +} +static inline int kallsyms_lookup_size_offset (unsigned long addr,unsigned long *symbolsize,unsigned long *offset)kernel/kallsyms.c +19
-1@@ -29,6 +29,7 @@#include <linux/compiler.h> #include <linux/module.h> #include <linux/kernel.h> +#include <linux/bsearch.h> /* * These will be re-linked against their real values @@ -228,7 +229,7 @@ unsigned long kallsyms_lookup_name(const char *name)return module_kallsyms_lookup_name (name); } -#ifdef CONFIG_LIVEPATCH +#if defined(CONFIG_LIVEPATCH ) || defined(CONFIG_FPROBE )/* * Iterate over all symbols in vmlinux. For symbols from modules use * module_kallsyms_on_each_symbol instead. @@ -572,6 +573,73 @@ int sprint_backtrace_build_id(char *buffer, unsigned long address)return __sprint_symbol (buffer, address,-1 ,1 ,1 ); } +#ifdef CONFIG_FPROBE +static int symbols_cmp (const void *a,const void *b) +{ +const char **str_a = (const char **) a; +const char **str_b = (const char **) b; + +return strcmp (*str_a, *str_b); +} + +struct kallsyms_data { +unsigned long *addrs; +const char **syms; +size_t cnt; +size_t found; +}; + +static int kallsyms_callback(void *data, const char *name,
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 602
Quoted 10 lines
kernel/kallsyms.c +10
-0+ struct module *mod, unsigned long addr) +{ +struct kallsyms_data *args = data; + +if (!bsearch (&name, args->syms, args->cnt,sizeof (*args->syms), symbols_cmp)) +return 0 ; + + addr =ftrace_location (addr); +if (!addr) +return 0 ;
Ooops, wait. Did you do this last version? I missed this point. This changes the meanings of the kernel function.
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 609
Quoted 7 lines
kernel/kallsyms.c +7
-0+ + args->addrs[args->found++] = addr; +return args->found == args->cnt ?1 :0 ; +} + +/** +* kallsyms_lookup_names - Lookup addresses for array of symbols
More correctly "Lookup 'ftraced' addresses for array of sorted symbols", right?
I'm not sure, we can call it as a 'kallsyms' API, since this is using kallsyms but doesn't return symbol address, but ftrace address. I think this name misleads user to expect returning symbol address.
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 619
Quoted 10 lines
kernel/kallsyms.c +10
-0+* +* @syms: array of symbols pointers symbols to resolve, must be +* alphabetically sorted +* @cnt: number of symbols/addresses in @syms/@addrs arrays +* @addrs: array for storing resulting addresses +* +* This function looks up addresses for array of symbols provided in +* @syms array (must be alphabetically sorted) and stores them in +* @addrs array, which needs to be big enough to store at least @cnt +* addresses.
Hmm, sorry I changed my mind. I rather like to expose kallsyms_on_each_symbol() and provide this API from fprobe or ftrace, because this returns ftrace address and thus this is only used from fprobe.
Thank you,
Quoted 29 lines
kernel/kallsyms.c +23
-0+* +* This function returns 0 if all provided symbols are found, +* -ESRCH otherwise. +*/ +int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs) +{ +struct kallsyms_data args; + + args.addrs = addrs; + args.syms = syms; + args.cnt = cnt; + args.found =0 ; +kallsyms_on_each_symbol (kallsyms_callback, &args); + +return args.found == args.cnt ?0 : -ESRCH ; +} +#else +int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs) +{ +return -ERANGE ; +} +#endif /* CONFIG_FPROBE */ +/* To avoid using get_symbol_offset for every symbol, we carry prefix along. */ struct kallsym_iter {loff_t pos;Signature · 1 line
2.35.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
Re: [PATCHv2 bpf-next 1/4] kallsyms: Add kallsyms_lookup_names function
Quoted 107 lines · Masami Hiramatsu
Quoted 8 lines · On Mon, 18 Apr 2022 14:48:31 +0200 Jiri Olsa <[email protected]> wrote:
Adding
kallsyms_lookup_namesfunction that resolves array of symbols with single pass over kallsyms.The user provides array of string pointers with count and pointer to allocated array for resolved values.
int kallsyms_lookup_names(const char **syms,
size_tcnt,
unsigned long *addrs)What about renaming the 'syms' argument to '
sorted_syms' so that user is easily notice what is required?
Or renaming the function askallsyms_lookup_sorted_names()?Quoted 90 lines
It iterates all kalsyms symbols and tries to loop up each in provided symbols array with bsearch. The symbols array needs to be sorted by name for this reason.
We also check each symbol to pass
ftrace_location, because this API will be used for fprobe symbols resolving. This can be optional in future if there's a need.We need
kallsyms_on_each_symbolfunction, so enabling it and also the new function forCONFIG_FPROBEoption.Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- include/linux/kallsyms.h | 6 ++++ kernel/kallsyms.c | 70 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-)include/linux/kallsyms.h +6
-0@@ -72,6 +72,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *,#ifdef CONFIG_KALLSYMS /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name (const char *name); +int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs); extern int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize,@@ -103,6 +104,11 @@ static inline unsigned long kallsyms_lookup_name(const char *name)return 0 ; } +static inline int kallsyms_lookup_names (const char **syms,size_t cnt,unsigned long *addrs) +{ +return -ERANGE ; +} +static inline int kallsyms_lookup_size_offset (unsigned long addr,unsigned long *symbolsize,unsigned long *offset)kernel/kallsyms.c +24
-1@@ -29,6 +29,7 @@#include <linux/compiler.h> #include <linux/module.h> #include <linux/kernel.h> +#include <linux/bsearch.h> /* * These will be re-linked against their real values @@ -228,7 +229,7 @@ unsigned long kallsyms_lookup_name(const char *name)return module_kallsyms_lookup_name (name); } -#ifdef CONFIG_LIVEPATCH +#if defined(CONFIG_LIVEPATCH ) || defined(CONFIG_FPROBE )/* * Iterate over all symbols in vmlinux. For symbols from modules use * module_kallsyms_on_each_symbol instead. @@ -572,6 +573,73 @@ int sprint_backtrace_build_id(char *buffer, unsigned long address)return __sprint_symbol (buffer, address,-1 ,1 ,1 ); } +#ifdef CONFIG_FPROBE +static int symbols_cmp (const void *a,const void *b) +{ +const char **str_a = (const char **) a; +const char **str_b = (const char **) b; + +return strcmp (*str_a, *str_b); +} + +struct kallsyms_data { +unsigned long *addrs; +const char **syms; +size_t cnt; +size_t found; +}; + +static int kallsyms_callback (void *data,const char *name, +struct module *mod,unsigned long addr) +{ +struct kallsyms_data *args = data; + +if (!bsearch (&name, args->syms, args->cnt,sizeof (*args->syms), symbols_cmp))
Quoted 8 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 602
Quoted 5 lines
kernel/kallsyms.c +5
-0+return 0 ; + + addr =ftrace_location (addr); +if (!addr) +return 0 ;Ooops, wait. Did you do this last version? I missed this point. This changes the meanings of the kernel function.
yes, it was there before ;-) and you're right.. so some archs can return different address, I did not realize that
Quoted 26 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 609
Quoted 8 lines
kernel/kallsyms.c +7
-0+ + args->addrs[args->found++] = addr; +return args->found == args->cnt ?1 :0 ; +} + +/** +* kallsyms_lookup_names - Lookup addresses for array of symbols More correctly "Lookup 'ftraced' addresses for array of sorted symbols", right?
I'm not sure, we can call it as a 'kallsyms' API, since this is using kallsyms but doesn't return symbol address, but ftrace address. I think this name misleads user to expect returning symbol address.
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 619
Quoted 10 lines
kernel/kallsyms.c +10
-0+* +* @syms: array of symbols pointers symbols to resolve, must be +* alphabetically sorted +* @cnt: number of symbols/addresses in @syms/@addrs arrays +* @addrs: array for storing resulting addresses +* +* This function looks up addresses for array of symbols provided in +* @syms array (must be alphabetically sorted) and stores them in +* @addrs array, which needs to be big enough to store at least @cnt +* addresses.
Quoted 3 lines
Hmm, sorry I changed my mind. I rather like to expose
kallsyms_on_each_symbol()and provide this API from fprobe or ftrace, because this returns ftrace address and thus this is only used from fprobe.
ok, so how about:
int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs);
thanks,
jirka
Re: [PATCHv2 bpf-next 1/4] kallsyms: Add kallsyms_lookup_names function
On Tue, Apr 19, 2022 at 10:26:05AM +0200, Jiri Olsa wrote:
SNIP
Quoted 44 lines
Quoted 6 lines
Quoted 6 lines
(quoted diff) +6
-0+static int kallsyms_callback(void *data, const char *name, + struct module *mod, unsigned long addr) +{ + struct kallsyms_data *args = data; + + if (!bsearch(&name, args->syms, args->cnt, sizeof(*args->syms), symbols_cmp))Quoted 8 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ return 0; + + addr = ftrace_location(addr); + if (!addr) + return 0;Ooops, wait. Did you do this last version? I missed this point. This changes the meanings of the kernel function.
yes, it was there before ;-) and you're right.. so some archs can return different address, I did not realize that
Quoted 26 lines
Review
Quoted 8 lines
(quoted diff) +7
-0+ + args->addrs[args->found++] = addr; + return args->found == args->cnt ? 1 : 0; +} + +/** + * kallsyms_lookup_names - Lookup addresses for array of symbolsMore correctly "Lookup 'ftraced' addresses for array of sorted symbols", right?
I'm not sure, we can call it as a 'kallsyms' API, since this is using kallsyms but doesn't return symbol address, but ftrace address. I think this name misleads user to expect returning symbol address.
Quoted 10 lines
+ *
+ * @syms: array of symbols pointers symbols to resolve, must be
+ * alphabetically sorted
+ * @cnt: number of symbols/addresses in @syms/@addrs arrays
+ * @addrs: array for storing resulting addresses
+ *
+ * This function looks up addresses for array of symbols provided in
+ * @syms array (must be alphabetically sorted) and stores them in
+ * @addrs array, which needs to be big enough to store at least @cnt
+ * addresses.
Quoted 7 lines
Quoted 3 lines
Hmm, sorry I changed my mind. I rather like to expose
kallsyms_on_each_symbol()and provide this API from fprobe or ftrace, because this returns ftrace address and thus this is only used from fprobe.ok, so how about:
int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs);
quick question.. is it ok if it stays in kalsyms.c object?
so we don't need to expose kallsyms_on_each_symbol, and it stays in 'kalsyms' place
jirka
include/linux/kallsyms.h +6 -0
@@ -72,6 +72,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *,#ifdef CONFIG_KALLSYMS /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name (const char *name); +int ftrace_lookup_symbols (const char **sorted_syms,size_t cnt,unsigned long *addrs); extern int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize,
@@ -103,6 +104,11 @@ static inline unsigned long kallsyms_lookup_name(const char *name)return 0 ; } +static inline int ftrace_lookup_symbols (const char **sorted_syms,size_t cnt,unsigned long *addrs); +{ +return -ERANGE ; +} +static inline int kallsyms_lookup_size_offset (unsigned long addr,unsigned long *symbolsize,unsigned long *offset)
kernel/kallsyms.c +69 -1
88 lines not on this page · raw message
Masami Hiramatsu
maintainerRe: [PATCHv2 bpf-next 1/4] kallsyms: Add kallsyms_lookup_names function
Hi Jiri,
Sorry for replying late.
Quoted 52 lines · On Fri, 22 Apr 2022 08:47:13 +0200 Jiri Olsa <[email protected]> wrote:
On Tue, Apr 19, 2022 at 10:26:05AM +0200, Jiri Olsa wrote:
SNIP
Quoted 48 lines
Quoted 6 lines
Quoted 6 lines
kernel/kallsyms.c +6
-0+static int kallsyms_callback (void *data,const char *name, +struct module *mod,unsigned long addr) +{ +struct kallsyms_data *args = data; + +if (!bsearch (&name, args->syms, args->cnt,sizeof (*args->syms), symbols_cmp))Quoted 8 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 602
Quoted 5 lines
kernel/kallsyms.c +5
-0+return 0 ; + + addr =ftrace_location (addr); +if (!addr) +return 0 ;Ooops, wait. Did you do this last version? I missed this point. This changes the meanings of the kernel function.
yes, it was there before ;-) and you're right.. so some archs can return different address, I did not realize that
Quoted 29 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address)
Quoted 8 lines
kernel/kallsyms.c +7
-0+ + args->addrs[args->found++] = addr; +return args->found == args->cnt ?1 :0 ; +} + +/** +* kallsyms_lookup_names - Lookup addresses for array of symbols More correctly "Lookup 'ftraced' addresses for array of sorted symbols", right?
I'm not sure, we can call it as a 'kallsyms' API, since this is using kallsyms but doesn't return symbol address, but ftrace address. I think this name misleads user to expect returning symbol address.
Quoted 10 lines
+ *
+ * @syms: array of symbols pointers symbols to resolve, must be
+ * alphabetically sorted
+ * @cnt: number of symbols/addresses in @syms/@addrs arrays
+ * @addrs: array for storing resulting addresses
+ *
+ * This function looks up addresses for array of symbols provided in
+ * @syms array (must be alphabetically sorted) and stores them in
+ * @addrs array, which needs to be big enough to store at least @cnt
+ * addresses.Hmm, sorry I changed my mind. I rather like to expose
kallsyms_on_each_symbol()and provide this API from fprobe or ftrace, because this returns ftrace address and thus this is only used from fprobe.
Quoted 5 lines
Quoted 3 lines
ok, so how about:
int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs);quick question.. is it ok if it stays in
kalsyms.cobject?
I think if this is for the ftrace API, I think it should be in the ftrace.c, and it can remove unneeded #ifdefs in C code.
Quoted 3 lines
so we don't need to expose
kallsyms_on_each_symbol, and it stays in 'kalsyms' place
We don't need to expose it to modules, but just make it into a global scope. I don't think that doesn't cause a problem.
Thank you,
Quoted 124 lines
jirka
include/linux/kallsyms.h +6
-0@@ -72,6 +72,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *,#ifdef CONFIG_KALLSYMS /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name (const char *name); +int ftrace_lookup_symbols (const char **sorted_syms,size_t cnt,unsigned long *addrs); extern int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize,@@ -103,6 +104,11 @@ static inline unsigned long kallsyms_lookup_name(const char *name)return 0 ; } +static inline int ftrace_lookup_symbols (const char **sorted_syms,size_t cnt,unsigned long *addrs); +{ +return -ERANGE ; +} +static inline int kallsyms_lookup_size_offset (unsigned long addr,unsigned long *symbolsize,unsigned long *offset)kernel/kallsyms.c +69
-1@@ -29,6 +29,7 @@#include <linux/compiler.h> #include <linux/module.h> #include <linux/kernel.h> +#include <linux/bsearch.h> /* * These will be re-linked against their real values @@ -228,7 +229,7 @@ unsigned long kallsyms_lookup_name(const char *name)return module_kallsyms_lookup_name (name); } -#ifdef CONFIG_LIVEPATCH +#if defined(CONFIG_LIVEPATCH ) || defined(CONFIG_FPROBE )/* * Iterate over all symbols in vmlinux. For symbols from modules use * module_kallsyms_on_each_symbol instead. @@ -572,6 +573,73 @@ int sprint_backtrace_build_id(char *buffer, unsigned long address)return __sprint_symbol (buffer, address,-1 ,1 ,1 ); } +#ifdef CONFIG_FPROBE +static int symbols_cmp (const void *a,const void *b) +{ +const char **str_a = (const char **) a; +const char **str_b = (const char **) b; + +return strcmp (*str_a, *str_b); +} + +struct kallsyms_data { +unsigned long *addrs; +const char **syms; +size_t cnt; +size_t found; +}; + +static int kallsyms_callback (void *data,const char *name, +struct module *mod,unsigned long addr) +{ +struct kallsyms_data *args = data; + +if (!bsearch (&name, args->syms, args->cnt,sizeof (*args->syms), symbols_cmp)) +return 0 ; + + addr =ftrace_location (addr); +if (!addr) +return 0 ; + + args->addrs[args->found++] = addr; +return args->found == args->cnt ?1 :0 ; +} + +/** +* ftrace_lookup_symbols - Lookup addresses for array of symbols +* +* @sorted_syms: array of symbols pointers symbols to resolve, +* must be alphabetically sorted +* @cnt: number of symbols/addresses in @syms/@addrs arrays +* @addrs: array for storing resulting addresses +* +* This function looks up addresses for array of symbols provided in +* @syms array (must be alphabetically sorted) and stores them in +* @addrs array, which needs to be big enough to store at least @cnt +* addresses. +* +* This function returns 0 if all provided symbols are found, +* -ESRCH otherwise. +*/ +int ftrace_lookup_symbols (const char **sorted_syms,size_t cnt,unsigned long *addrs) +{ +struct kallsyms_data args; + + args.addrs = addrs; + args.syms = sorted_syms; + args.cnt = cnt; + args.found =0 ; +kallsyms_on_each_symbol (kallsyms_callback, &args); + +return args.found == args.cnt ?0 : -ESRCH ; +} +#else +int ftrace_lookup_symbols (const char **sorted_syms,size_t cnt,unsigned long *addrs) +{ +return -ERANGE ; +} +#endif /* CONFIG_FPROBE */ +/* To avoid using get_symbol_offset for every symbol, we carry prefix along. */ struct kallsym_iter {loff_t pos;
Signature · 1 line
Masami Hiramatsu <[email protected]>
Re: [PATCHv2 bpf-next 1/4] kallsyms: Add kallsyms_lookup_names function
Quoted 69 lines · Masami Hiramatsu
Hi Jiri,
Sorry for replying late.
Quoted 52 lines · On Fri, 22 Apr 2022 08:47:13 +0200 Jiri Olsa <[email protected]> wrote:
On Tue, Apr 19, 2022 at 10:26:05AM +0200, Jiri Olsa wrote:
SNIP
Quoted 48 lines
Quoted 6 lines
Quoted 6 lines
kernel/kallsyms.c +6
-0+static int kallsyms_callback (void *data,const char *name, +struct module *mod,unsigned long addr) +{ +struct kallsyms_data *args = data; + +if (!bsearch (&name, args->syms, args->cnt,sizeof (*args->syms), symbols_cmp))Quoted 8 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 602
Quoted 5 lines
kernel/kallsyms.c +5
-0+return 0 ; + + addr =ftrace_location (addr); +if (!addr) +return 0 ;Ooops, wait. Did you do this last version? I missed this point. This changes the meanings of the kernel function.
yes, it was there before ;-) and you're right.. so some archs can return different address, I did not realize that
Quoted 29 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address)
Quoted 8 lines
kernel/kallsyms.c +7
-0+ + args->addrs[args->found++] = addr; +return args->found == args->cnt ?1 :0 ; +} + +/** +* kallsyms_lookup_names - Lookup addresses for array of symbols More correctly "Lookup 'ftraced' addresses for array of sorted symbols", right?
I'm not sure, we can call it as a 'kallsyms' API, since this is using kallsyms but doesn't return symbol address, but ftrace address. I think this name misleads user to expect returning symbol address.
Quoted 10 lines
+ *
+ * @syms: array of symbols pointers symbols to resolve, must be
+ * alphabetically sorted
+ * @cnt: number of symbols/addresses in @syms/@addrs arrays
+ * @addrs: array for storing resulting addresses
+ *
+ * This function looks up addresses for array of symbols provided in
+ * @syms array (must be alphabetically sorted) and stores them in
+ * @addrs array, which needs to be big enough to store at least @cnt
+ * addresses.Hmm, sorry I changed my mind. I rather like to expose
kallsyms_on_each_symbol()and provide this API from fprobe or ftrace, because this returns ftrace address and thus this is only used from fprobe.Quoted 5 lines
Quoted 3 lines
ok, so how about:
int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs);quick question.. is it ok if it stays in
kalsyms.cobject?I think if this is for the ftrace API, I think it should be in the
ftrace.c, and it can remove unneeded #ifdefs in C code.Quoted 1 line
Quoted 5 lines
Quoted 2 lines
so we don't need to expose
kallsyms_on_each_symbol, and it stays in 'kalsyms' placeWe don't need to expose it to modules, but just make it into a global scope. I don't think that doesn't cause a problem.
np, will move it to ftrace
thanks,
jirka
Masami Hiramatsu
maintainerRe: [PATCHv2 bpf-next 1/4] kallsyms: Add kallsyms_lookup_names function
Quoted 70 lines · On Tue, 26 Apr 2022 14:27:04 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 69 lines · Masami Hiramatsu
Hi Jiri,
Sorry for replying late.
Quoted 52 lines · On Fri, 22 Apr 2022 08:47:13 +0200 Jiri Olsa <[email protected]> wrote:
On Tue, Apr 19, 2022 at 10:26:05AM +0200, Jiri Olsa wrote:
SNIP
Quoted 48 lines
Quoted 6 lines
Quoted 6 lines
kernel/kallsyms.c +6
-0+static int kallsyms_callback (void *data,const char *name, +struct module *mod,unsigned long addr) +{ +struct kallsyms_data *args = data; + +if (!bsearch (&name, args->syms, args->cnt,sizeof (*args->syms), symbols_cmp))Quoted 8 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address) · line 602
Quoted 5 lines
kernel/kallsyms.c +5
-0+return 0 ; + + addr =ftrace_location (addr); +if (!addr) +return 0 ;Ooops, wait. Did you do this last version? I missed this point. This changes the meanings of the kernel function.
yes, it was there before ;-) and you're right.. so some archs can return different address, I did not realize that
Quoted 29 lines
kernel/kallsyms.c int sprint_backtrace_build_id(char *buffer, unsigned long address)
Quoted 8 lines
kernel/kallsyms.c +7
-0+ + args->addrs[args->found++] = addr; +return args->found == args->cnt ?1 :0 ; +} + +/** +* kallsyms_lookup_names - Lookup addresses for array of symbols More correctly "Lookup 'ftraced' addresses for array of sorted symbols", right?
I'm not sure, we can call it as a 'kallsyms' API, since this is using kallsyms but doesn't return symbol address, but ftrace address. I think this name misleads user to expect returning symbol address.
Quoted 10 lines
+ *
+ * @syms: array of symbols pointers symbols to resolve, must be
+ * alphabetically sorted
+ * @cnt: number of symbols/addresses in @syms/@addrs arrays
+ * @addrs: array for storing resulting addresses
+ *
+ * This function looks up addresses for array of symbols provided in
+ * @syms array (must be alphabetically sorted) and stores them in
+ * @addrs array, which needs to be big enough to store at least @cnt
+ * addresses.Hmm, sorry I changed my mind. I rather like to expose
kallsyms_on_each_symbol()and provide this API from fprobe or ftrace, because this returns ftrace address and thus this is only used from fprobe.Quoted 5 lines
Quoted 3 lines
ok, so how about:
int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs);quick question.. is it ok if it stays in
kalsyms.cobject?I think if this is for the ftrace API, I think it should be in the
ftrace.c, and it can remove unneeded #ifdefs in C code.Quoted 1 line
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
so we don't need to expose
kallsyms_on_each_symbol, and it stays in 'kalsyms' placeWe don't need to expose it to modules, but just make it into a global scope. I don't think that doesn't cause a problem.
Oops, I meant "I don't think that cause any problem."
Quoted 2 lines
np, will move it to ftrace
Thank you!
Quoted 3 lines
thanks,
jirka
Signature · 1 line
Masami Hiramatsu <[email protected]>
Jiri Olsa
reviewerPATCH v2 2/4 fprobe: Resolve symbols with kallsyms_lookup_names
In reply to Jiri OlsaUsing kallsyms_lookup_names to speed up symbols lookup in register_fprobe_syms API.
This requires syms array to be alphabetically sorted.
--- kernel/trace/fprobe.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-)
kernel/trace/fprobe.c +12 -20
@@ -85,39 +85,31 @@ static void fprobe_exit_handler(struct rethook_node *rh, void *data, }NOKPROBE_SYMBOL (fprobe_exit_handler); +static int symbols_cmp (const void *a,const void *b) +{ +const char **str_a = (const char **) a; +const char **str_b = (const char **) b; + +return strcmp (*str_a, *str_b); +} +/* Convert ftrace location address from symbols */ static unsigned long *get_ftrace_locations (const char **syms,int num) { -unsigned long addr, size;unsigned long *addrs; -int i;/* Convert symbols to symbol address */ addrs =kcalloc (num,sizeof (*addrs),GFP_KERNEL );if (!addrs)return ERR_PTR (-ENOMEM ); -for (i =0 ; i < num; i++) { - addr =kallsyms_lookup_name (syms[i]); -if (!addr)/* Maybe wrong symbol */ - goto error; - -/* Convert symbol address to ftrace location. */ -if (!kallsyms_lookup_size_offset (addr, &size,NULL ) || !size) - goto error; +/* kallsyms_lookup_names expects sorted symbols */ +sort (syms, num,sizeof (*syms), symbols_cmp,NULL ); - addr =ftrace_location_range (addr, addr + size -1 ); -if (!addr)/* No dynamic ftrace there. */ - goto error; +if (!kallsyms_lookup_names (syms, num, addrs)) +return addrs; - addrs[i] = addr; - } - -return addrs; - -error:kfree (addrs); -return ERR_PTR (-ENOENT ); }
Signature · 1 line
2.35.1
Masami Hiramatsu
maintainerRe: [PATCHv2 bpf-next 2/4] fprobe: Resolve symbols with kallsyms_lookup_names
Hi Jiri,
Quoted 4 lines · On Mon, 18 Apr 2022 14:48:32 +0200 Jiri Olsa <[email protected]> wrote:
Using
kallsyms_lookup_namesto speed up symbols lookup inregister_fprobe_symsAPI.This requires syms array to be alphabetically sorted.
This looks good to me. But we may need to update this for changing the function name "kallsyms_lookup_names()". I think "ftrace_lookup_sorted_syms()" will be better to understand.
What would you think?
Thank you,
Quoted 65 lines
Signed-off-by Jiri Olsa <[email protected]>--- kernel/trace/fprobe.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-)kernel/trace/fprobe.c +12
-20@@ -85,39 +85,31 @@ static void fprobe_exit_handler(struct rethook_node *rh, void *data, }NOKPROBE_SYMBOL (fprobe_exit_handler); +static int symbols_cmp (const void *a,const void *b) +{ +const char **str_a = (const char **) a; +const char **str_b = (const char **) b; + +return strcmp (*str_a, *str_b); +} +/* Convert ftrace location address from symbols */ static unsigned long *get_ftrace_locations (const char **syms,int num) { -unsigned long addr, size;unsigned long *addrs; -int i;/* Convert symbols to symbol address */ addrs =kcalloc (num,sizeof (*addrs),GFP_KERNEL );if (!addrs)return ERR_PTR (-ENOMEM ); -for (i =0 ; i < num; i++) { - addr =kallsyms_lookup_name (syms[i]); -if (!addr)/* Maybe wrong symbol */ - goto error; - -/* Convert symbol address to ftrace location. */ -if (!kallsyms_lookup_size_offset (addr, &size,NULL ) || !size) - goto error; +/* kallsyms_lookup_names expects sorted symbols */ +sort (syms, num,sizeof (*syms), symbols_cmp,NULL ); - addr =ftrace_location_range (addr, addr + size -1 ); -if (!addr)/* No dynamic ftrace there. */ - goto error; +if (!kallsyms_lookup_names (syms, num, addrs)) +return addrs; - addrs[i] = addr; - } - -return addrs; - -error:kfree (addrs); -return ERR_PTR (-ENOENT ); }Signature · 1 line
2.35.1
Signature · 1 line
Masami Hiramatsu <[email protected]>
Jiri Olsa
reviewerPATCH v2 3/4 bpf: Resolve symbols with kallsyms_lookup_names for kprobe multi link
In reply to Jiri OlsaUsing kallsyms_lookup_names function to speed up symbols lookup in kprobe multi link attachment and replacing with it the current kprobe_multi_resolve_syms function.
This speeds up bpftrace kprobe attachment:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }'
...
6.5681 +- 0.0225 seconds time elapsed ( +- 0.34% )After:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }'
...
0.5661 +- 0.0275 seconds time elapsed ( +- 4.85% )--- kernel/trace/bpf_trace.c | 113 +++++++++++++++++++++++---------------- 1 file changed, 67 insertions(+), 46 deletions(-)
kernel/trace/bpf_trace.c +67 -46
@@ -2226,6 +2226,60 @@ struct bpf_kprobe_multi_run_ctx {unsigned long entry_ip; }; +struct user_syms { +const char **syms; +char *buf; +}; + +static int copy_user_syms (struct user_syms *us,unsigned long __user *usyms,u32 cnt) +{ +unsigned long __user usymbol; +const char **syms =NULL ; +char *buf =NULL , *p; +int err = -EFAULT ; +unsigned int i; + + err = -ENOMEM ; + syms =kvmalloc (cnt *sizeof (*syms),GFP_KERNEL ); +if (!syms) + goto error; + + buf =kvmalloc (cnt *KSYM_NAME_LEN ,GFP_KERNEL ); +if (!buf) + goto error; + +for (p = buf, i =0 ; i < cnt; i++) { +if (__get_user (usymbol, usyms + i)) { + err = -EFAULT ; + goto error; + } + err =strncpy_from_user (p, (const char __user *) usymbol,KSYM_NAME_LEN ); +if (err ==KSYM_NAME_LEN ) + err = -E2BIG ; +if (err <0 ) + goto error; + syms[i] = p; + p += err +1 ; + } + + err =0 ; + us->syms = syms; + us->buf = buf; + +error: +if (err) { +kvfree (syms); +kvfree (buf); + } +return err; +} + +static void free_user_syms (struct user_syms *us) +{ +kvfree (us->syms); +kvfree (us->buf); +} +static void bpf_kprobe_multi_link_release (struct bpf_link *link) {struct bpf_kprobe_multi_link *kmulti_link;
@@ -2346,53 +2400,12 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long entry_ip,kprobe_multi_link_prog_run (link, entry_ip, regs); } -static int -kprobe_multi_resolve_syms (const void __user *usyms,u32 cnt, -unsigned long *addrs) +static int symbols_cmp (const void *a,const void *b) { -unsigned long addr, size; -const char __user **syms; -int err = -ENOMEM ; -unsigned int i; -char *func; - - size = cnt *sizeof (*syms); - syms =kvzalloc (size,GFP_KERNEL ); -if (!syms) -return -ENOMEM ; - - func =kmalloc (KSYM_NAME_LEN ,GFP_KERNEL ); -if (!func) - goto error; - -if (copy_from_user (syms, usyms, size)) { - err = -EFAULT ; - goto error; - } - -for (i =0 ; i < cnt; i++) { - err =strncpy_from_user (func, syms[i],KSYM_NAME_LEN ); -if (err ==KSYM_NAME_LEN ) - err = -E2BIG ; -if (err <0 ) - goto error; - err = -EINVAL ; - addr =kallsyms_lookup_name (func); -if (!addr) - goto error; -if (!kallsyms_lookup_size_offset (addr, &size,NULL )) - goto error; - addr =ftrace_location_range (addr, addr + size -1 ); -if (!addr) - goto error; - addrs[i] = addr; - } +const char **str_a = (const char **) a; +const char **str_b = (const char **) b; - err =0 ; -error: -kvfree (syms); -kfree (func); -return err; +return strcmp (*str_a, *str_b); }int bpf_kprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog)
@@ -2438,7 +2451,15 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr goto error; } }else { - err =kprobe_multi_resolve_syms (usyms, cnt, addrs); +struct user_syms us; + + err =copy_user_syms (&us, usyms, cnt); +if (err) + goto error; + +sort (us.syms, cnt,sizeof (*us.syms), symbols_cmp,NULL ); + err =kallsyms_lookup_names (us.syms, cnt, addrs); +free_user_syms (&us);if (err) goto error; }
Signature · 1 line
2.35.1
Re: [PATCHv2 bpf-next 3/4] bpf: Resolve symbols with kallsyms_lookup_names for kprobe multi link
Quoted 11 lines · Jiri Olsa
Using
kallsyms_lookup_namesfunction to speed up symbols lookup in kprobe multi link attachment and replacing with it the currentkprobe_multi_resolve_symsfunction.This speeds up bpftrace kprobe attachment:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 6.5681 +- 0.0225 seconds time elapsed ( +- 0.34% )
Quoted 8 lines
After:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 0.5661 +- 0.0275 seconds time elapsed ( +- 4.85% )Signed-off-by Jiri Olsa <[email protected]>---
LGTM.
Quoted 40 lines
kernel/trace/bpf_trace.c | 113 +++++++++++++++++++++++---------------- 1 file changed, 67 insertions(+), 46 deletions(-)kernel/trace/bpf_trace.c +29
-0@@ -2226,6 +2226,60 @@ struct bpf_kprobe_multi_run_ctx {unsigned long entry_ip; }; +struct user_syms { +const char **syms; +char *buf; +}; + +static int copy_user_syms (struct user_syms *us,unsigned long __user *usyms,u32 cnt) +{ +unsigned long __user usymbol; +const char **syms =NULL ; +char *buf =NULL , *p; +int err = -EFAULT ; +unsigned int i; + + err = -ENOMEM ; + syms =kvmalloc (cnt *sizeof (*syms),GFP_KERNEL ); +if (!syms) + goto error; + + buf =kvmalloc (cnt *KSYM_NAME_LEN ,GFP_KERNEL ); +if (!buf) + goto error; + +for (p = buf, i =0 ; i < cnt; i++) { +if (__get_user (usymbol, usyms + i)) { + err = -EFAULT ; + goto error; + } + err =strncpy_from_user (p, (const char __user *) usymbol,KSYM_NAME_LEN ); +if (err ==KSYM_NAME_LEN )
kernel/trace/bpf_trace.c struct bpf_kprobe_multi_run_ctx { · line 2267
Quoted 10 lines
kernel/trace/bpf_trace.c +10
-0+ err = -E2BIG ; +if (err <0 ) + goto error; + syms[i] = p; + p += err +1 ; + } + + err =0 ; + us->syms = syms; + us->buf = buf;
return 0 here instead of falling through into error: block?
Quoted 18 lines
kernel/trace/bpf_trace.c +15
-0+ +error: +if (err) { +kvfree (syms); +kvfree (buf); + } +return err; +} + +static void free_user_syms (struct user_syms *us) +{ +kvfree (us->syms); +kvfree (us->buf); +} +static void bpf_kprobe_multi_link_release (struct bpf_link *link) {struct bpf_kprobe_multi_link *kmulti_link;
[...]
Re: [PATCHv2 bpf-next 3/4] bpf: Resolve symbols with kallsyms_lookup_names for kprobe multi link
Quoted 69 lines · Andrii Nakryiko
Quoted 11 lines · Jiri Olsa
Using
kallsyms_lookup_namesfunction to speed up symbols lookup in kprobe multi link attachment and replacing with it the currentkprobe_multi_resolve_symsfunction.This speeds up bpftrace kprobe attachment:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 6.5681 +- 0.0225 seconds time elapsed ( +- 0.34% )Quoted 8 lines
After:
# perf stat -r 5 -e cycles ./src/bpftrace -e 'kprobe:x* { } i:ms:1 { exit(); }' ... 0.5661 +- 0.0275 seconds time elapsed ( +- 4.85% )Signed-off-by Jiri Olsa <[email protected]>---
LGTM.
Acked-by Andrii Nakryiko <[email protected]>Quoted 44 lines
kernel/trace/bpf_trace.c | 113 +++++++++++++++++++++++---------------- 1 file changed, 67 insertions(+), 46 deletions(-)kernel/trace/bpf_trace.c +33
-0@@ -2226,6 +2226,60 @@ struct bpf_kprobe_multi_run_ctx {unsigned long entry_ip; }; +struct user_syms { +const char **syms; +char *buf; +}; + +static int copy_user_syms (struct user_syms *us,unsigned long __user *usyms,u32 cnt) +{ +unsigned long __user usymbol; +const char **syms =NULL ; +char *buf =NULL , *p; +int err = -EFAULT ; +unsigned int i; + + err = -ENOMEM ; + syms =kvmalloc (cnt *sizeof (*syms),GFP_KERNEL ); +if (!syms) + goto error; + + buf =kvmalloc (cnt *KSYM_NAME_LEN ,GFP_KERNEL ); +if (!buf) + goto error; + +for (p = buf, i =0 ; i < cnt; i++) { +if (__get_user (usymbol, usyms + i)) { + err = -EFAULT ; + goto error; + } + err =strncpy_from_user (p, (const char __user *) usymbol,KSYM_NAME_LEN ); +if (err ==KSYM_NAME_LEN ) + err = -E2BIG ; +if (err <0 ) + goto error; + syms[i] = p;
Quoted 8 lines
kernel/trace/bpf_trace.c struct bpf_kprobe_multi_run_ctx { · line 2267
Quoted 6 lines
kernel/trace/bpf_trace.c +6
-0+ p += err +1 ; + } + + err =0 ; + us->syms = syms; + us->buf = buf;return 0 here instead of falling through into error: block?
ok, will change
jirka
Quoted 21 lines
Quoted 19 lines
kernel/trace/bpf_trace.c +15
-0+ +error: +if (err) { +kvfree (syms); +kvfree (buf); + } +return err; +} + +static void free_user_syms (struct user_syms *us) +{ +kvfree (us->syms); +kvfree (us->buf); +} +static void bpf_kprobe_multi_link_release (struct bpf_link *link) {struct bpf_kprobe_multi_link *kmulti_link;[...]
Jiri Olsa
reviewerPATCH v2 4/4 selftests/bpf: Add attach bench test
In reply to Jiri OlsaAdding test that reads all functions from ftrace available_filter_functions file and attach them all through kprobe_multi API.
It also prints stats info with -v option, like on my setup:
test_bench_attach: found 48712 functions
test_bench_attach: attached in 1.069s
test_bench_attach: detached in 0.373s
--- .../bpf/prog_tests/kprobe_multi_test.c | 136 ++++++++++++++++++ .../selftests/bpf/progs/kprobe_multi_empty.c | 12 ++ 2 files changed, 148 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_empty.c
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +136 -0
@@ -2,6 +2,9 @@#include <test_progs.h> #include "kprobe_multi.skel.h" #include "trace_helpers.h" +#include "kprobe_multi_empty.skel.h" +#include "bpf/libbpf_internal.h" +#include "bpf/hashmap.h" static void kprobe_multi_test_run (struct kprobe_multi *skel,bool test_return) {
@@ -301,6 +304,137 @@ static void test_attach_api_fails(void)kprobe_multi__destroy (skel); } +static inline __u64 get_time_ns (void ) +{ +struct timespec t; + +clock_gettime (CLOCK_MONOTONIC , &t); +return (__u64 ) t.tv_sec *1000000000 + t.tv_nsec; +} + +static size_t symbol_hash (const void *key,void *ctx__maybe_unused ) +{ +return str_hash ((const char *) key); +} + +static bool symbol_equal (const void *key1,const void *key2,void *ctx__maybe_unused ) +{ +return strcmp ((const char *) key1, (const char *) key2) ==0 ; +} + +#define DEBUGFS "/sys/kernel/debug/tracing/" + +static int get_syms (char ***symsp,size_t *cntp) +{ +size_t cap =0 , cnt =0 , i; +char *name, **syms =NULL ; +struct hashmap *map; +char buf[256 ]; +FILE *f; +int err; + +/* +* The available_filter_functions contains many duplicates, +* but other than that all symbols are usable in kprobe multi +* interface. +* Filtering out duplicates by using hashmap__add, which won't +* add existing entry. +*/ + f =fopen (DEBUGFS "available_filter_functions" ,"r" ); +if (!f) +return -EINVAL ; + + map =hashmap__new (symbol_hash, symbol_equal,NULL ); + err =libbpf_get_error (map); +if (err) + goto error; + +while (fgets (buf,sizeof (buf), f)) { +/* skip modules */ +if (strchr (buf,'[' )) +continue ; +if (sscanf (buf,"%ms$*[^\n]\n" , &name) !=1 ) +continue ; + err =hashmap__add (map, name,NULL ); +if (err) { +free (name); +if (err == -EEXIST ) +continue ; + goto error; + } + err =libbpf_ensure_mem ((void **) &syms, &cap, +sizeof (*syms), cnt +1 ); +if (err) { +free (name); + goto error; + } + syms[cnt] = name; + cnt++; + } + + *symsp = syms; + *cntp = cnt; + +error: +fclose (f); +hashmap__free (map); +if (err) { +for (i =0 ; i < cnt; i++) +free (syms[cnt]); +free (syms); + } +return err; +} + +static void test_bench_attach (void ) +{ +LIBBPF_OPTS (bpf_kprobe_multi_opts, opts); +struct kprobe_multi_empty *skel =NULL ; +long attach_start_ns, attach_end_ns; +long detach_start_ns, detach_end_ns; +double attach_delta, detach_delta; +struct bpf_link *link =NULL ; +char **syms =NULL ; +size_t cnt, i; + +if (!ASSERT_OK (get_syms (&syms, &cnt),"get_syms" )) +return ; + + skel =kprobe_multi_empty__open_and_load (); +if (!ASSERT_OK_PTR (skel,"kprobe_multi_empty__open_and_load" )) + goto cleanup; + + opts.syms = (const char **) syms; + opts.cnt = cnt; + + attach_start_ns =get_time_ns (); + link =bpf_program__attach_kprobe_multi_opts (skel->progs.test_kprobe_empty, +NULL , &opts); + attach_end_ns =get_time_ns (); + +if (!ASSERT_OK_PTR (link,"bpf_program__attach_kprobe_multi_opts" )) + goto cleanup; + + detach_start_ns =get_time_ns (); +bpf_link__destroy (link); + detach_end_ns =get_time_ns (); + + attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +fprintf (stderr,"%s: found %lu functions\n" , __func__, cnt); +fprintf (stderr,"%s: attached in %7.3lfs\n" , __func__, attach_delta); +fprintf (stderr,"%s: detached in %7.3lfs\n" , __func__, detach_delta); + +cleanup: +kprobe_multi_empty__destroy (skel); +if (syms) { +for (i =0 ; i < cnt; i++) +free (syms[i]); +free (syms); + } +} +void test_kprobe_multi_test (void ) {if (!ASSERT_OK (load_kallsyms (),"load_kallsyms" ))
@@ -320,4 +454,6 @@ void test_kprobe_multi_test(void)test_attach_api_syms ();if (test__start_subtest ("attach_api_fails" ))test_attach_api_fails (); +if (test__start_subtest ("bench_attach" )) +test_bench_attach (); }
tools/testing/selftests/bpf/progs/kprobe_multi_empty.c new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +char _license[] SEC("license") = "GPL"; + +SEC("kprobe.multi/*") +int test_kprobe_empty (struct pt_regs *ctx) +{ +return 0 ; +}
Signature · 1 line
2.35.1
Re: [PATCHv2 bpf-next 4/4] selftests/bpf: Add attach bench test
Quoted 62 lines · Jiri Olsa
Adding test that reads all functions from ftrace
available_filter_functionsfile and attach them all throughkprobe_multiAPI.It also prints stats info with -v option, like on my setup:
test_bench_attach: found 48712 functions
test_bench_attach: attached in 1.069s
test_bench_attach: detached in 0.373sSigned-off-by Jiri Olsa <[email protected]>--- .../bpf/prog_tests/kprobe_multi_test.c | 136 ++++++++++++++++++ .../selftests/bpf/progs/kprobe_multi_empty.c | 12 ++ 2 files changed, 148 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/kprobe_multi_empty.ctools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +30
-0@@ -2,6 +2,9 @@#include <test_progs.h> #include "kprobe_multi.skel.h" #include "trace_helpers.h" +#include "kprobe_multi_empty.skel.h" +#include "bpf/libbpf_internal.h" +#include "bpf/hashmap.h" static void kprobe_multi_test_run (struct kprobe_multi *skel,bool test_return) {@@ -301,6 +304,137 @@ static void test_attach_api_fails(void)kprobe_multi__destroy (skel); } +static inline __u64 get_time_ns (void ) +{ +struct timespec t; + +clock_gettime (CLOCK_MONOTONIC , &t); +return (__u64 ) t.tv_sec *1000000000 + t.tv_nsec; +} + +static size_t symbol_hash (const void *key,void *ctx__maybe_unused ) +{ +return str_hash ((const char *) key); +} + +static bool symbol_equal (const void *key1,const void *key2,void *ctx__maybe_unused ) +{ +return strcmp ((const char *) key1, (const char *) key2) ==0 ; +} + +#define DEBUGFS "/sys/kernel/debug/tracing/" + +static int get_syms (char ***symsp,size_t *cntp) +{ +size_t cap =0 , cnt =0 , i; +char *name, **syms =NULL ; +struct hashmap *map; +char buf[256 ]; +FILE *f;
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 343
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +10
-0+int err; + +/* +* The available_filter_functions contains many duplicates, +* but other than that all symbols are usable in kprobe multi +* interface. +* Filtering out duplicates by using hashmap__add, which won't +* add existing entry. +*/ + f =fopen (DEBUGFS "available_filter_functions" ,"r" );
nit: DEBUGFS "constant" just makes it harder to follow the code and doesn't add anything, please just use the full path here directly
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 348
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +5
-0+if (!f) +return -EINVAL ; + + map =hashmap__new (symbol_hash, symbol_equal,NULL ); + err =libbpf_get_error (map);
hashmap__new() is an internal API, so please use IS_ERR() directly here. libbpf_get_error() should be used for public libbpf APIs, and preferably not in libbpf 1.0 mode
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +7
-0+if (err) + goto error; + +while (fgets (buf,sizeof (buf), f)) { +/* skip modules */ +if (strchr (buf,'[' )) +continue ;
[...]
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 427
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +6
-0+ attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +fprintf (stderr,"%s: found %lu functions\n" , __func__, cnt); +fprintf (stderr,"%s: attached in %7.3lfs\n" , __func__, attach_delta); +fprintf (stderr,"%s: detached in %7.3lfs\n" , __func__, detach_delta);
why stderr? just do printf() and let test_progs handle output
Quoted 26 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +12
-0+ +cleanup: +kprobe_multi_empty__destroy (skel); +if (syms) { +for (i =0 ; i < cnt; i++) +free (syms[i]); +free (syms); + } +} +void test_kprobe_multi_test (void ) {if (!ASSERT_OK (load_kallsyms (),"load_kallsyms" ))@@ -320,4 +454,6 @@ void test_kprobe_multi_test(void)test_attach_api_syms ();if (test__start_subtest ("attach_api_fails" ))test_attach_api_fails (); +if (test__start_subtest ("bench_attach" )) +test_bench_attach (); }tools/testing/selftests/bpf/progs/kprobe_multi_empty.c new file +0
-0
tools/testing/selftests/bpf/progs/kprobe_multi_empty.c line 8
Quoted 8 lines
tools/testing/selftests/bpf/progs/kprobe_multi_empty.c new file +8
-0@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +char _license[] SEC("license") = "GPL"; + +SEC ("kprobe.multi/*" )
use SEC("kprobe.multi") to make it clear that we are attaching it "manually"?
Quoted 7 lines
tools/testing/selftests/bpf/progs/kprobe_multi_empty.c new file +4
-1+int test_kprobe_empty (struct pt_regs *ctx) +{ +return 0 ; +} --2.35.1
Re: [PATCHv2 bpf-next 4/4] selftests/bpf: Add attach bench test
On Wed, Apr 20, 2022 at 02:56:53PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 14 lines
Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +14
-0+#define DEBUGFS "/sys/kernel/debug/tracing/" + +static int get_syms (char ***symsp,size_t *cntp) +{ +size_t cap =0 , cnt =0 , i; +char *name, **syms =NULL ; +struct hashmap *map; +char buf[256 ]; +FILE *f; +int err; + +/* +* The available_filter_functions contains many duplicates, +* but other than that all symbols are usable in kprobe multi
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 343
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +5
-0+* interface. +* Filtering out duplicates by using hashmap__add, which won't +* add existing entry. +*/ + f =fopen (DEBUGFS "available_filter_functions" ,"r" );nit: DEBUGFS "constant" just makes it harder to follow the code and doesn't add anything, please just use the full path here directly
there's another one DEBUGFS in trace_helpers.c,
we could put it in trace_helpers.h
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 348
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +5
-0+if (!f) +return -EINVAL ; + + map =hashmap__new (symbol_hash, symbol_equal,NULL ); + err =libbpf_get_error (map);
hashmap__new()is an internal API, so please useIS_ERR()directly here.libbpf_get_error()should be used for public libbpf APIs, and preferably not in libbpf 1.0 mode
ok
Quoted 19 lines
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +7
-0+if (err) + goto error; + +while (fgets (buf,sizeof (buf), f)) { +/* skip modules */ +if (strchr (buf,'[' )) +continue ;[...]
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c static void test_attach_api_fails(void) · line 427
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +6
-0+ attach_delta = (attach_end_ns - attach_start_ns) /1000000000.0 ; + detach_delta = (detach_end_ns - detach_start_ns) /1000000000.0 ; + +fprintf (stderr,"%s: found %lu functions\n" , __func__, cnt); +fprintf (stderr,"%s: attached in %7.3lfs\n" , __func__, attach_delta); +fprintf (stderr,"%s: detached in %7.3lfs\n" , __func__, detach_delta);
Quoted 1 line
why stderr? just do
printf()and lettest_progshandle output
ok
Quoted 30 lines
Quoted 30 lines
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +12
-0+ +cleanup: +kprobe_multi_empty__destroy (skel); +if (syms) { +for (i =0 ; i < cnt; i++) +free (syms[i]); +free (syms); + } +} +void test_kprobe_multi_test (void ) {if (!ASSERT_OK (load_kallsyms (),"load_kallsyms" ))@@ -320,4 +454,6 @@ void test_kprobe_multi_test(void)test_attach_api_syms ();if (test__start_subtest ("attach_api_fails" ))test_attach_api_fails (); +if (test__start_subtest ("bench_attach" )) +test_bench_attach (); }tools/testing/selftests/bpf/progs/kprobe_multi_empty.c new file +2
-0@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/bpf.h>
Quoted 8 lines
tools/testing/selftests/bpf/progs/kprobe_multi_empty.c line 8
Quoted 6 lines
tools/testing/selftests/bpf/progs/kprobe_multi_empty.c new file +6
-0+#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +char _license[] SEC("license") = "GPL"; + +SEC ("kprobe.multi/*" )use
SEC("kprobe.multi")to make it clear that we are attaching it "manually"?
yep, will do
thanks,
jirka