selftests/bpf: make use of PROCMAP_QUERY ioctl if available
Subsystems (3)
Andrii Nakryiko
maintainerPATCH 1/1 selftests/bpf: make use of PROCMAP_QUERY ioctl if available
Instead of parsing text-based /proc/<pid>/maps file, try to use PROCMAP_QUERY ioctl() to simplify and speed up data fetching. This logic is used to do uprobe file offset calculation, so any bugs in this logic would manifest as failing uprobe BPF selftests.
This also serves as a simple demonstration of one of the intended uses.
--- tools/testing/selftests/bpf/test_progs.c | 3 + tools/testing/selftests/bpf/test_progs.h | 2 + tools/testing/selftests/bpf/trace_helpers.c | 104 +++++++++++++++++--- 3 files changed, 94 insertions(+), 15 deletions(-)
tools/testing/selftests/bpf/test_progs.c +3 -0
@@ -33,6 +33,8 @@ __weak void backtrace_symbols_fd(void *const *buffer, int size, int fd)dprintf (fd,"<backtrace not supported>\n" ); } +int env_verbosity =0 ; +static bool verbose (void ) {return env.verbosity >VERBOSE_NONE ;
@@ -862,6 +864,7 @@ static error_t parse_arg(int key, char *arg, struct argp_state *state)return -EINVAL ; } } + env_verbosity = env->verbosity;if (verbose ()) {if (setenv ("SELFTESTS_VERBOSE" ,"1" ,1 ) ==-1 ) {
tools/testing/selftests/bpf/test_progs.h +2 -0
@@ -95,6 +95,8 @@ struct test_state {FILE *stdout_saved; }; +extern int env_verbosity; +struct test_env {struct test_selector test_selector;struct test_selector subtest_selector;
tools/testing/selftests/bpf/trace_helpers.c +89 -15
139 lines not on this page · raw message
Signature · 1 line
2.43.5
On Tue, Aug 06, 2024 at 04:03:19PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 26 lines
ssize_tget_uprobe_offset(const void *addr)
{tools/testing/selftests/bpf/trace_helpers.c +11
-7-size_t start, end, base; -char buf[256 ]; -bool found = false; +size_t start, base, end;FILE *f; +char buf[256 ]; +int err, flags; f =fopen ("/proc/self/maps" ,"r" );if (!f)return -errno; -while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { -if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { - found = true; -break ; +/* requested executable VMA only */ + err =procmap_query (fileno (f), addr,PROCMAP_QUERY_VMA_EXECUTABLE , &start, &base, &flags); +if (err == -EOPNOTSUPP ) { +bool found = false; + +while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { +if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { + found = true;
tools/testing/selftests/bpf/trace_helpers.c int kallsyms_find(const char *sym, unsigned long long *addr) · line 330
Quoted 10 lines
tools/testing/selftests/bpf/trace_helpers.c +9
-0+break ; + } + } +if (!found) { +fclose (f); +return -ESRCH ; } + }else if (err) { +fclose (f); +return err;
I feel like I commented on this before, so feel free to ignore me, but this seems similar to the code below, could be in one function
anyway it's good for follow up
there was another selftest in the original patchset adding benchmark for the procfs query interface, is it coming in as well?
jirka
Quoted 44 lines
tools/testing/selftests/bpf/trace_helpers.c +14
-8} -fclose (f); -if (!found) -return -ESRCH ; -#if defined(__powerpc64__) && defined(_CALL_ELF) && _CALL_ELF ==2 #define OP_RT_RA_MASK 0xffff0000UL@@ -307,15 +371,25 @@ ssize_t get_rel_offset(uintptr_t addr)size_t start, end, offset;char buf[256 ];FILE *f; +int err, flags; f =fopen ("/proc/self/maps" ,"r" );if (!f)return -errno; -while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &offset) ==4 ) { -if (addr >= start && addr < end) { -fclose (f); -return (size_t )addr - start + offset; + err =procmap_query (fileno (f), (const void *)addr,0 , &start, &offset, &flags); +if (err ==0 ) { +fclose (f); +return (size_t )addr - start + offset; + }else if (err != -EOPNOTSUPP ) { +fclose (f); +return err; + }else if (err) { +while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &offset) ==4 ) { +if (addr >= start && addr < end) { +fclose (f); +return (size_t )addr - start + offset; + } } }Signature · 1 line
2.43.5
Quoted 42 lines · Jiri Olsa
On Tue, Aug 06, 2024 at 04:03:19PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 26 lines
ssize_tget_uprobe_offset(const void *addr)
{tools/testing/selftests/bpf/trace_helpers.c +11
-7-size_t start, end, base; -char buf[256 ]; -bool found = false; +size_t start, base, end;FILE *f; +char buf[256 ]; +int err, flags; f =fopen ("/proc/self/maps" ,"r" );if (!f)return -errno; -while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { -if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { - found = true; -break ; +/* requested executable VMA only */ + err =procmap_query (fileno (f), addr,PROCMAP_QUERY_VMA_EXECUTABLE , &start, &base, &flags); +if (err == -EOPNOTSUPP ) { +bool found = false; + +while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { +if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { + found = true;tools/testing/selftests/bpf/trace_helpers.c int kallsyms_find(const char *sym, unsigned long long *addr) · line 330
Quoted 10 lines
tools/testing/selftests/bpf/trace_helpers.c +9
-0+break ; + } + } +if (!found) { +fclose (f); +return -ESRCH ; } + }else if (err) { +fclose (f); +return err;
Quoted 2 lines
I feel like I commented on this before, so feel free to ignore me, but this seems similar to the code below, could be in one function
Do you mean get_rel_offset()? That one is for data symbols (USDT semaphores), so it a) doesn't do arch-specific adjustments and b) doesn't filter by executable flag. So while the logic of parsing and finding VMA is similar, conditions and adjustments are different. It feels not worth combining them, tbh.
Quoted 5 lines
anyway it's good for follow up
there was another selftest in the original patchset adding benchmark for the procfs query interface, is it coming in as well?
I didn't plan to send it, given it's not really a test. But I can put it on Github somewhere, probably, if it's useful.
Quoted 49 lines
Acked-by Jiri Olsa <[email protected]>jirka
Quoted 44 lines
}tools/testing/selftests/bpf/trace_helpers.c +14
-9-fclose (f); -if (!found) -return -ESRCH ; -#if defined(__powerpc64__) && defined(_CALL_ELF) && _CALL_ELF ==2 #define OP_RT_RA_MASK 0xffff0000UL@@ -307,15 +371,25 @@ ssize_t get_rel_offset(uintptr_t addr)size_t start, end, offset;char buf[256 ];FILE *f; +int err, flags; f =fopen ("/proc/self/maps" ,"r" );if (!f)return -errno; -while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &offset) ==4 ) { -if (addr >= start && addr < end) { -fclose (f); -return (size_t )addr - start + offset; + err =procmap_query (fileno (f), (const void *)addr,0 , &start, &offset, &flags); +if (err ==0 ) { +fclose (f); +return (size_t )addr - start + offset; + }else if (err != -EOPNOTSUPP ) { +fclose (f); +return err; + }else if (err) { +while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &offset) ==4 ) { +if (addr >= start && addr < end) { +fclose (f); +return (size_t )addr - start + offset; + } } } --2.43.5
Quoted 54 lines · Andrii Nakryiko
Quoted 42 lines · Jiri Olsa
On Tue, Aug 06, 2024 at 04:03:19PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 26 lines
ssize_tget_uprobe_offset(const void *addr)
{tools/testing/selftests/bpf/trace_helpers.c +11
-7-size_t start, end, base; -char buf[256 ]; -bool found = false; +size_t start, base, end;FILE *f; +char buf[256 ]; +int err, flags; f =fopen ("/proc/self/maps" ,"r" );if (!f)return -errno; -while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { -if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { - found = true; -break ; +/* requested executable VMA only */ + err =procmap_query (fileno (f), addr,PROCMAP_QUERY_VMA_EXECUTABLE , &start, &base, &flags); +if (err == -EOPNOTSUPP ) { +bool found = false; + +while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { +if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { + found = true;tools/testing/selftests/bpf/trace_helpers.c int kallsyms_find(const char *sym, unsigned long long *addr) · line 330
Quoted 10 lines
tools/testing/selftests/bpf/trace_helpers.c +9
-0+break ; + } + } +if (!found) { +fclose (f); +return -ESRCH ; } + }else if (err) { +fclose (f); +return err;Quoted 2 lines
I feel like I commented on this before, so feel free to ignore me, but this seems similar to the code below, could be in one function
Do you mean
get_rel_offset()? That one is for data symbols (USDT semaphores), so it a) doesn't do arch-specific adjustments and b) doesn't filter by executable flag. So while the logic of parsing and finding VMA is similar, conditions and adjustments are different. It feels not worth combining them, tbh.Quoted 1 line
Quoted 7 lines
Quoted 4 lines
anyway it's good for follow up
there was another selftest in the original patchset adding benchmark for the procfs query interface, is it coming in as well?
I didn't plan to send it, given it's not really a test. But I can put it on Github somewhere, probably, if it's useful.
With and without this selftest applied I see:
./test_progs -t uprobe
#416 uprobe:OK
#417 uprobe_autoattach:OK
[ 47.448908] ref_ctr_offset mismatch. inode: 0x16b5f921 offset:
0x2d4297 ref_ctr_offset(old): 0x45e8b56 ref_ctr_offset(new): 0x45e8b54
#418/1 uprobe_multi_test/skel_api:OK
Is this a known issue?
Applied anyway.
Quoted 69 lines · Alexei Starovoitov
Quoted 54 lines · Andrii Nakryiko
Quoted 42 lines · Jiri Olsa
On Tue, Aug 06, 2024 at 04:03:19PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 26 lines
ssize_tget_uprobe_offset(const void *addr)
{tools/testing/selftests/bpf/trace_helpers.c +11
-7-size_t start, end, base; -char buf[256 ]; -bool found = false; +size_t start, base, end;FILE *f; +char buf[256 ]; +int err, flags; f =fopen ("/proc/self/maps" ,"r" );if (!f)return -errno; -while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { -if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { - found = true; -break ; +/* requested executable VMA only */ + err =procmap_query (fileno (f), addr,PROCMAP_QUERY_VMA_EXECUTABLE , &start, &base, &flags); +if (err == -EOPNOTSUPP ) { +bool found = false; + +while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { +if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { + found = true;tools/testing/selftests/bpf/trace_helpers.c int kallsyms_find(const char *sym, unsigned long long *addr) · line 330
Quoted 10 lines
tools/testing/selftests/bpf/trace_helpers.c +9
-0+break ; + } + } +if (!found) { +fclose (f); +return -ESRCH ; } + }else if (err) { +fclose (f); +return err;Quoted 2 lines
I feel like I commented on this before, so feel free to ignore me, but this seems similar to the code below, could be in one function
Do you mean
get_rel_offset()? That one is for data symbols (USDT semaphores), so it a) doesn't do arch-specific adjustments and b) doesn't filter by executable flag. So while the logic of parsing and finding VMA is similar, conditions and adjustments are different. It feels not worth combining them, tbh.Quoted 1 line
Quoted 7 lines
Quoted 4 lines
anyway it's good for follow up
there was another selftest in the original patchset adding benchmark for the procfs query interface, is it coming in as well?
I didn't plan to send it, given it's not really a test. But I can put it on Github somewhere, probably, if it's useful.
With and without this selftest applied I see:
./test_progs -t uprobe
#416 uprobe:OK
#417 uprobe_autoattach:OK
Quoted 5 lines
[ 47.448908] ref_ctr_offset mismatch. inode: 0x16b5f921 offset:
0x2d4297ref_ctr_offset(old):0x45e8b56ref_ctr_offset(new):0x45e8b54
#418/1 uprobe_multi_test/skel_api:OKIs this a known issue?
Yeah, that's not due to my changes. It's an old warning in uprobe internals, but I think we should remove it, because it can trivially be triggered by a user. Which is what Jiri is doing intentionally in one of selftests to test uprobe failure handling.
Jiri, maybe let's get rid of this warning?
Quoted 2 lines
Applied anyway.
Thanks! I just found another auto-archived patch of mine, the one adding multi-uprobe benchmarks (see patchworks). Please take a look and maybe apply, when you get a chance.
Quoted 75 lines · Andrii Nakryiko
Quoted 73 lines · Alexei Starovoitov
Quoted 54 lines · Andrii Nakryiko
Quoted 42 lines · Jiri Olsa
On Tue, Aug 06, 2024 at 04:03:19PM -0700, Andrii Nakryiko wrote:
SNIP
Quoted 26 lines
ssize_tget_uprobe_offset(const void *addr)
{tools/testing/selftests/bpf/trace_helpers.c +11
-7-size_t start, end, base; -char buf[256 ]; -bool found = false; +size_t start, base, end;FILE *f; +char buf[256 ]; +int err, flags; f =fopen ("/proc/self/maps" ,"r" );if (!f)return -errno; -while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { -if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { - found = true; -break ; +/* requested executable VMA only */ + err =procmap_query (fileno (f), addr,PROCMAP_QUERY_VMA_EXECUTABLE , &start, &base, &flags); +if (err == -EOPNOTSUPP ) { +bool found = false; + +while (fscanf (f,"%zx-%zx %s %zx %*[^\n]\n" , &start, &end, buf, &base) ==4 ) { +if (buf[2 ] =='x' && (uintptr_t )addr >= start && (uintptr_t )addr < end) { + found = true;tools/testing/selftests/bpf/trace_helpers.c int kallsyms_find(const char *sym, unsigned long long *addr) · line 330
Quoted 10 lines
tools/testing/selftests/bpf/trace_helpers.c +9
-0+break ; + } + } +if (!found) { +fclose (f); +return -ESRCH ; } + }else if (err) { +fclose (f); +return err;Quoted 2 lines
I feel like I commented on this before, so feel free to ignore me, but this seems similar to the code below, could be in one function
Do you mean
get_rel_offset()? That one is for data symbols (USDT semaphores), so it a) doesn't do arch-specific adjustments and b) doesn't filter by executable flag. So while the logic of parsing and finding VMA is similar, conditions and adjustments are different. It feels not worth combining them, tbh.Quoted 1 line
Quoted 7 lines
Quoted 4 lines
anyway it's good for follow up
there was another selftest in the original patchset adding benchmark for the procfs query interface, is it coming in as well?
I didn't plan to send it, given it's not really a test. But I can put it on Github somewhere, probably, if it's useful.
With and without this selftest applied I see:
./test_progs -t uprobe
#416 uprobe:OK
#417 uprobe_autoattach:OK[ 47.448908] ref_ctr_offset mismatch. inode: 0x16b5f921 offset:
0x2d4297ref_ctr_offset(old):0x45e8b56ref_ctr_offset(new):0x45e8b54
#418/1 uprobe_multi_test/skel_api:OK
Quoted 8 lines
Quoted 1 line
Is this a known issue?
Yeah, that's not due to my changes. It's an old warning in uprobe internals, but I think we should remove it, because it can trivially be triggered by a user. Which is what Jiri is doing intentionally in one of selftests to test uprobe failure handling.
Jiri, maybe let's get rid of this warning?
fine with me.. or change to pr_debug if that makes sense for anyone, Oleg, any idea/preference?
jirka
Quoted 7 lines
Quoted 3 lines
Applied anyway.
Thanks! I just found another auto-archived patch of mine, the one adding multi-uprobe benchmarks (see patchworks). Please take a look and maybe apply, when you get a chance.
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Tue, 6 Aug 2024 16:03:19 -0700 you wrote:
Instead of parsing text-based /proc/<pid>/maps file, try to use
PROCMAP_QUERYioctl()to simplify and speed up data fetching. This logic is used to do uprobe file offset calculation, so any bugs in this logic would manifest as failing uprobe BPF selftests.This also serves as a simple demonstration of one of the intended uses.
[...]
Here is the summary with links:
- [bpf-next] selftests/bpf: make use of PROCMAP_QUERY ioctl if available
https://git.kernel.org/bpf/bpf-next/c/4e9e07603ecd
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html