selftests/bpf: Add Memory Sanitizer support
Subsystems (5)
cover selftests/bpf: Add Memory Sanitizer support
Hi,
This series adds support for building selftests with Memory Sanitizer
[1] - a compiler instrumentation for detecting usages of undefined memory.
The primary motivation is to make sure that such usages do not occur during testing, since the ones that have not been caught yet are likely to affect the CI results on s390x. The secondary motivation is to be able to use libbpf in applications instrumented with MSan (it requires all code running in a process to be instrumented).
MSan has found one real issue (fixed by patch 6), and of course a number of false positives. This rest of this series deals with preparing the build infrastructure and adding MSan annotations to libbpf and selftests.
The setup I'm using is as follows:
- Instrumented zlib-ng and patched elfutils [2].
- Patched LLVM [3, 4].
- Clang-built kernel.
- Building tests with MSan:
make \
-C tools/testing/selftests/bpf \
CC="ccache clang-17" \
LD=ld \
HOSTCC="ccache clang-17" \
HOSTLD=ld \
LLVM=1 \
LLVM_SUFFIX=-17 \
OBJCOPY=objcopy \
CLANG="ccache clang-17" \
LLVM_STRIP=llvm-strip-17 \
LLC=llc-17 \
BPF_GCC= \
SAN_CFLAGS="-fsanitize=memory \
-fsanitize-memory-track-origins \
-I$PWD/../zlib-ng/dist/include \
-I$PWD/../elfutils/dist/include" \
SAN_LDFLAGS="-fsanitize=memory \
-fsanitize-memory-track-origins \
-L$PWD/../zlib-ng/dist/lib \
-L$PWD/../elfutils/dist/lib" \
CROSS_COMPILE=s390x-linux-gnu- It's a lot of options, but most of them are trivial: setting up LLVM toolchain, taking in account s390x quirks, setting up MSan and disabling bpf-gcc. The CROSS_COMPILE one is a hack: instrumenting bpftool turned out to be too complicated from the build system perspective, so having CROSS_COMPILE forces compiling the host libbpf uninstrumented and guest libbpf instrumented.
- Running tests with MSan on s390x:
LD_LIBRARY_PATH=<instrumented libs> ./test_progs
...
Summary: 282/1624 PASSED, 23 SKIPPED, 4 FAILEDThe 4 failures happen without MSan too, they are already known and denylisted.
Best regards,
Ilya
[1] https://clang.llvm.org/docs/MemorySanitizer.html
[2] https://sourceware.org/pipermail/elfutils-devel/2023q1/005831.html
[3] https://reviews.llvm.org/D143296
[4] https://reviews.llvm.org/D143330
Ilya Leoshkevich (9):
selftests/bpf: Quote host tools
tools: runqslower: Add EXTRA_CFLAGS and EXTRA_LDFLAGS support selftests/bpf: Split SAN_CFLAGS and SAN_LDFLAGS selftests/bpf: Forward SAN_CFLAGS and SAN_LDFLAGS to runqslower and
libbpf
selftests/bpf: Attach to fopen()/fclose() in uprobe_autoattach selftests/bpf: Attach to fopen()/fclose() in attach_probe
libbpf: Fix alen calculation in libbpf_nla_dump_errormsg()
libbpf: Add MSan annotations
selftests/bpf: Add MSan annotations
tools/bpf/runqslower/Makefile | 6 ++ tools/lib/bpf/bpf.c | 70 +++++++++++++++++-- tools/lib/bpf/btf.c | 1 + tools/lib/bpf/libbpf.c | 1 + tools/lib/bpf/libbpf_internal.h | 14 ++++ tools/lib/bpf/nlattr.c | 2 +- tools/testing/selftests/bpf/Makefile | 17 +++-- tools/testing/selftests/bpf/cap_helpers.c | 3 + .../selftests/bpf/prog_tests/attach_probe.c | 10 +-- .../selftests/bpf/prog_tests/bpf_obj_id.c | 10 +++ .../selftests/bpf/prog_tests/bpf_tcp_ca.c | 3 + tools/testing/selftests/bpf/prog_tests/btf.c | 11 +++ .../selftests/bpf/prog_tests/send_signal.c | 2 + .../bpf/prog_tests/tp_attach_query.c | 6 ++ .../bpf/prog_tests/uprobe_autoattach.c | 12 ++-- .../selftests/bpf/prog_tests/xdp_bonding.c | 3 + .../selftests/bpf/progs/test_attach_probe.c | 8 ++- .../bpf/progs/test_uprobe_autoattach.c | 10 +-- tools/testing/selftests/bpf/xdp_synproxy.c | 2 + 19 files changed, 161 insertions(+), 30 deletions(-)
Signature · 1 line
2.39.1
PATCH 2/9 tools: runqslower: Add EXTRA_CFLAGS and EXTRA_LDFLAGS support
This makes it possible to add sanitizer flags.
--- tools/bpf/runqslower/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)
tools/bpf/runqslower/Makefile +6 -0
@@ -13,6 +13,12 @@ BPF_DESTDIR := $(BPFOBJ_OUTPUT) BPF_INCLUDE := $(BPF_DESTDIR)/include INCLUDES := -I$(OUTPUT) -I$(BPF_INCLUDE) -I$(abspath ../../include/uapi) CFLAGS := -g -Wall $(CLANG_CROSS_FLAGS) +ifneq ($(EXTRA_CFLAGS),) +CFLAGS += $(EXTRA_CFLAGS) +endif +ifneq ($(EXTRA_LDFLAGS),) +LDFLAGS += $(EXTRA_LDFLAGS) +endif # Try to detect best kernel BTF source KERNEL_REL := $(shell uname -r)
Signature · 1 line
2.39.1
PATCH 3/9 selftests/bpf: Split SAN_CFLAGS and SAN_LDFLAGS
In reply to Ilya LeoshkevichMemory Sanitizer requires passing different options to CFLAGS and
LDFLAGS: besides the mandatory -fsanitize=memory, one needs to pass header and library paths, and passing -L to a compilation step triggers -Wunused-command-line-argument. So introduce a separate variable for linker flags. Use $(SAN_CFLAGS) as a default in order to avoid complicating the ASan usage.
--- tools/testing/selftests/bpf/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/Makefile +2 -1
@@ -22,10 +22,11 @@ endif BPF_GCC ?= $(shell command -v bpf-gcc;) SAN_CFLAGS ?= +SAN_LDFLAGS ?= $(SAN_CFLAGS) CFLAGS += -g -O0 -rdynamic -Wall -Werror $(GENFLAGS) $(SAN_CFLAGS) \ -I$(CURDIR) -I$(INCLUDE_DIR) -I$(GENDIR) -I$(LIBDIR) \ -I$(TOOLSINCDIR) -I$(APIDIR) -I$(OUTPUT) -LDFLAGS += $(SAN_CFLAGS) +LDFLAGS += $(SAN_LDFLAGS) LDLIBS += -lelf -lz -lrt -lpthread # Silence some warnings when compiled with clang
Signature · 1 line
2.39.1
PATCH 1/9 selftests/bpf: Quote host tools
In reply to Ilya LeoshkevichUsing HOSTCC="ccache clang" breaks building the tests, since, when it's forwarded to e.g. bpftool, the child make sees HOSTCC=ccache and "clang" is considered a target. Fix by quoting it, and also HOSTLD and HOSTAR for consistency.
--- tools/testing/selftests/bpf/Makefile | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/Makefile +4 -3
@@ -248,7 +248,7 @@ BPFTOOL ?= $(DEFAULT_BPFTOOL) $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) \ $(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \ - ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD) \ + ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD="$(HOSTLD)" \ EXTRA_CFLAGS='-g -O0' \ OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \ LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \
@@ -280,7 +280,8 @@ $(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ | $(HOST_BUILD_DIR)/libbpf $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \ EXTRA_CFLAGS='-g -O0' ARCH= CROSS_COMPILE= \ - OUTPUT=$(HOST_BUILD_DIR)/libbpf/ CC=$(HOSTCC) LD=$(HOSTLD) \ + OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \ + CC="$(HOSTCC)" LD="$(HOSTLD)" \ DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers endif
@@ -301,7 +302,7 @@ $(RESOLVE_BTFIDS): $(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/resolve_btfids \ $(TOOLSDIR)/lib/ctype.c \ $(TOOLSDIR)/lib/str_error_r.c $(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/resolve_btfids \ - CC=$(HOSTCC) LD=$(HOSTLD) AR=$(HOSTAR) \ + CC="$(HOSTCC)" LD="$(HOSTLD)" AR="$(HOSTAR)" \ LIBBPF_INCLUDE=$(HOST_INCLUDE_DIR) \ OUTPUT=$(HOST_BUILD_DIR)/resolve_btfids/ BPFOBJ=$(HOST_BPFOBJ)
Signature · 1 line
2.39.1
PATCH 4/9 selftests/bpf: Forward SAN_CFLAGS and SAN_LDFLAGS to runqslower and libbpf
In reply to Ilya LeoshkevichTo get useful results from the Memory Sanitizer, all code running in a process needs to be instrumented. When building tests with other sanitizers, it's not strictly necessary, but is also helpful. So make sure runqslower and libbpf are compiled with SAN_CFLAGS and linked with SAN_LDFLAGS.
--- tools/testing/selftests/bpf/Makefile | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/Makefile +5 -2
@@ -215,7 +215,9 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT) OUTPUT=$(RUNQSLOWER_OUTPUT) VMLINUX_BTF=$(VMLINUX_BTF) \ BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \ BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf \ - BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) && \ + BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) \ + EXTRA_CFLAGS='-g -O0 $(SAN_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS)' && \ cp $(RUNQSLOWER_OUTPUT)runqslower $@ TEST_GEN_PROGS_EXTENDED += $(DEFAULT_BPFTOOL)
@@ -272,7 +274,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ $(APIDIR)/linux/bpf.h \ | $(BUILD_DIR)/libbpf $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \ - EXTRA_CFLAGS='-g -O0' \ + EXTRA_CFLAGS='-g -O0 $(SAN_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS)' \ DESTDIR=$(SCRATCH_DIR) prefix= all install_headers ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
Signature · 1 line
2.39.1
PATCH 6/9 selftests/bpf: Attach to fopen()/fclose() in attach_probe
In reply to Ilya Leoshkevichmalloc() and free() may be completely replaced by sanitizers, use fopen() and fclose() instead.
--- tools/testing/selftests/bpf/prog_tests/attach_probe.c | 10 +++++----- tools/testing/selftests/bpf/progs/test_attach_probe.c | 8 +++++--- 2 files changed, 10 insertions(+), 8 deletions(-)
tools/testing/selftests/bpf/prog_tests/attach_probe.c +5 -5
@@ -33,8 +33,8 @@ void test_attach_probe(void)struct test_attach_probe * skel;ssize_t uprobe_offset, ref_ctr_offset;struct bpf_link *uprobe_err_link; +FILE *devnull;bool legacy; -char *mem;/* Check if new-style kprobe/uprobe API is supported. * Kernels that support new FD-based kprobe and uprobe BPF attachment
@@ -147,7 +147,7 @@ void test_attach_probe(void)/* test attach by name for a library function, using the library * as the binary argument. libc.so.6 will be resolved via dlopen()/dlinfo(). */ - uprobe_opts.func_name ="malloc" ; + uprobe_opts.func_name ="fopen" ; uprobe_opts.retprobe = false; skel->links.handle_uprobe_byname2 = bpf_program__attach_uprobe_opts(skel->progs.handle_uprobe_byname2,
@@ -157,7 +157,7 @@ void test_attach_probe(void)if (!ASSERT_OK_PTR (skel->links.handle_uprobe_byname2,"attach_uprobe_byname2" )) goto cleanup; - uprobe_opts.func_name ="free" ; + uprobe_opts.func_name ="fclose" ; uprobe_opts.retprobe = true; skel->links.handle_uretprobe_byname2 = bpf_program__attach_uprobe_opts(skel->progs.handle_uretprobe_byname2,
@@ -195,8 +195,8 @@ void test_attach_probe(void)usleep (1 );/* trigger & validate shared library u[ret]probes attached by name */ - mem =malloc (1 ); -free (mem); + devnull =fopen ("/dev/null" ,"r" ); +fclose (devnull);/* trigger & validate uprobe & uretprobe */ trigger_func ();
tools/testing/selftests/bpf/progs/test_attach_probe.c +5 -3
@@ -94,10 +94,12 @@ int handle_uretprobe_byname(struct pt_regs *ctx) SEC("uprobe")int handle_uprobe_byname2 (struct pt_regs *ctx) { -unsigned int size =PT_REGS_PARM1 (ctx); +void *mode_ptr = (void *)(long )PT_REGS_PARM2 (ctx); +char mode[2 ] = {}; -/* verify malloc size */ -if (size ==1 ) +/* verify fopen mode */ +bpf_probe_read_user (mode,sizeof (mode), mode_ptr); +if (mode[0 ] =='r' && mode[1 ] ==0 ) uprobe_byname2_res =7 ;return 0 ; }
Signature · 1 line
2.39.1
PATCH 5/9 selftests/bpf: Attach to fopen()/fclose() in uprobe_autoattach
In reply to Ilya Leoshkevichmalloc() and free() may be completely replaced by sanitizers, use fopen() and fclose() instead.
--- .../selftests/bpf/prog_tests/uprobe_autoattach.c | 12 ++++++------ .../selftests/bpf/progs/test_uprobe_autoattach.c | 10 +++++----- 2 files changed, 11 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/prog_tests/uprobe_autoattach.c +6 -6
@@ -16,10 +16,10 @@ static noinline int autoattach_trigger_func(int arg1, int arg2, int arg3,void test_uprobe_autoattach (void ) { +const char *devnull_str ="/dev/null" ;struct test_uprobe_autoattach *skel;int trigger_ret; -size_t malloc_sz =1 ; -char *mem; +FILE *devnull; skel =test_uprobe_autoattach__open_and_load ();if (!ASSERT_OK_PTR (skel,"skel_open" ))
@@ -36,16 +36,16 @@ void test_uprobe_autoattach(void) skel->bss->test_pid =getpid ();/* trigger & validate shared library u[ret]probes attached by name */ - mem =malloc (malloc_sz); + devnull =fopen (devnull_str,"r" );ASSERT_EQ (skel->bss->uprobe_byname_parm1,1 ,"check_uprobe_byname_parm1" );ASSERT_EQ (skel->bss->uprobe_byname_ran,1 ,"check_uprobe_byname_ran" );ASSERT_EQ (skel->bss->uretprobe_byname_rc, trigger_ret,"check_uretprobe_byname_rc" );ASSERT_EQ (skel->bss->uretprobe_byname_ret, trigger_ret,"check_uretprobe_byname_ret" );ASSERT_EQ (skel->bss->uretprobe_byname_ran,2 ,"check_uretprobe_byname_ran" ); -ASSERT_EQ (skel->bss->uprobe_byname2_parm1, malloc_sz,"check_uprobe_byname2_parm1" ); +ASSERT_EQ (skel->bss->uprobe_byname2_parm1, devnull_str,"check_uprobe_byname2_parm1" );ASSERT_EQ (skel->bss->uprobe_byname2_ran,3 ,"check_uprobe_byname2_ran" ); -ASSERT_EQ (skel->bss->uretprobe_byname2_rc, mem,"check_uretprobe_byname2_rc" ); +ASSERT_EQ (skel->bss->uretprobe_byname2_rc, (void *)devnull,"check_uretprobe_byname2_rc" );ASSERT_EQ (skel->bss->uretprobe_byname2_ran,4 ,"check_uretprobe_byname2_ran" );ASSERT_EQ (skel->bss->a[0 ],1 ,"arg1" );
@@ -67,7 +67,7 @@ void test_uprobe_autoattach(void)ASSERT_EQ (skel->bss->a[7 ],8 ,"arg8" );#endif -free (mem); +fclose (devnull); cleanup:test_uprobe_autoattach__destroy (skel); }
tools/testing/selftests/bpf/progs/test_uprobe_autoattach.c +5 -5
@@ -13,9 +13,9 @@ int uprobe_byname_ran = 0;int uretprobe_byname_rc =0 ;int uretprobe_byname_ret =0 ;int uretprobe_byname_ran =0 ; -size_t uprobe_byname2_parm1 =0 ; +void *uprobe_byname2_parm1 =NULL ;int uprobe_byname2_ran =0 ; -char *uretprobe_byname2_rc =NULL ; +void *uretprobe_byname2_rc =NULL ;int uretprobe_byname2_ran =0 ;int test_pid;
@@ -88,7 +88,7 @@ int BPF_URETPROBE(handle_uretprobe_byname, int ret) } -SEC("uprobe/libc.so.6:malloc") +SEC("uprobe/libc.so.6:fopen")int handle_uprobe_byname2 (struct pt_regs *ctx) {int pid =bpf_get_current_pid_tgid () >>32 ;
@@ -96,12 +96,12 @@ int handle_uprobe_byname2(struct pt_regs *ctx)/* ignore irrelevant invocations */ if (test_pid != pid)return 0 ; - uprobe_byname2_parm1 =PT_REGS_PARM1_CORE (ctx); + uprobe_byname2_parm1 = (void *)(long )PT_REGS_PARM1_CORE (ctx); uprobe_byname2_ran =3 ;return 0 ; } -SEC("uretprobe/libc.so.6:malloc") +SEC("uretprobe/libc.so.6:fopen")int handle_uretprobe_byname2 (struct pt_regs *ctx) {int pid =bpf_get_current_pid_tgid () >>32 ;
Signature · 1 line
2.39.1
PATCH 7/9 libbpf: Fix alen calculation in libbpf_nla_dump_errormsg()
In reply to Ilya LeoshkevichThe code assumes that everything that comes after nlmsgerr are nlattrs. When calculating their size, it does not account for the initial nlmsghdr. This may lead to accessing uninitialized memory.
--- tools/lib/bpf/nlattr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/bpf/nlattr.c +1 -1
@@ -178,7 +178,7 @@ int libbpf_nla_dump_errormsg(struct nlmsghdr *nlh) hlen +=nlmsg_len (&err->msg); attr = (struct nlattr *) ((void *) err + hlen); - alen = nlh->nlmsg_len - hlen; + alen = (char *)nlh + nlh->nlmsg_len - (char *)attr;if (libbpf_nla_parse (tb,NLMSGERR_ATTR_MAX , attr, alen, extack_policy) !=0 ) {
Signature · 1 line
2.39.1
PATCH 8/9 libbpf: Add MSan annotations
In reply to Ilya LeoshkevichMSan runs into a few false positives in libbpf. They all come from the fact that MSan does not know anything about the bpf syscall, particularly, what it writes to.
Add libbpf_mark_defined() function to mark memory modified by the bpf syscall. Use the abstract name (it could be e.g. libbpf_msan_unpoison()), because it can be used for valgrind in the future as well.
--- tools/lib/bpf/bpf.c | 70 +++++++++++++++++++++++++++++++-- tools/lib/bpf/btf.c | 1 + tools/lib/bpf/libbpf.c | 1 + tools/lib/bpf/libbpf_internal.h | 14 +++++++ 4 files changed, 82 insertions(+), 4 deletions(-)
tools/lib/bpf/bpf.c +66 -4
@@ -92,6 +92,10 @@ int sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts) fd =sys_bpf_fd (BPF_PROG_LOAD , attr, size); }while (fd <0 && errno ==EAGAIN && --attempts >0 ); +if (attr->log_buf) +libbpf_mark_defined ((void *)(unsigned long )attr->log_buf, + attr->log_size); +return fd; }
@@ -395,6 +399,33 @@ int bpf_map_update_elem(int fd, const void *key, const void *value,return libbpf_err_errno (ret); } +static void mark_map_value_defined (int fd,void *value) +{ +#ifdef HAVE_LIBBPF_MARK_DEFINED +struct bpf_map_info info; +size_t size =0 ; +__u32 info_len; +int num_cpus; +int err; + + info_len =sizeof (info); + err =bpf_obj_get_info_by_fd (fd, &info, &info_len); +if (!err) { + size = info.value_size; +if (info.type ==BPF_MAP_TYPE_PERCPU_ARRAY || + info.type ==BPF_MAP_TYPE_PERCPU_HASH || + info.type ==BPF_MAP_TYPE_LRU_PERCPU_HASH || + info.type ==BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE ) { + num_cpus =libbpf_num_possible_cpus (); +if (num_cpus >0 ) + size *= num_cpus; + } + } +if (size) +libbpf_mark_defined (value, size); +#endif +} +int bpf_map_lookup_elem (int fd,const void *key,void *value) { const size_t attr_sz = offsetofend(union bpf_attr, flags);
@@ -407,6 +438,8 @@ int bpf_map_lookup_elem(int fd, const void *key, void *value) attr.value =ptr_to_u64 (value); ret =sys_bpf (BPF_MAP_LOOKUP_ELEM , &attr, attr_sz); +if (!ret) +mark_map_value_defined (fd, value);return libbpf_err_errno (ret); }
@@ -423,6 +456,8 @@ int bpf_map_lookup_elem_flags(int fd, const void *key, void *value, __u64 flags) attr.flags = flags; ret =sys_bpf (BPF_MAP_LOOKUP_ELEM , &attr, attr_sz); +if (!ret) +mark_map_value_defined (fd, value);return libbpf_err_errno (ret); }
@@ -438,6 +473,8 @@ int bpf_map_lookup_and_delete_elem(int fd, const void *key, void *value) attr.value =ptr_to_u64 (value); ret =sys_bpf (BPF_MAP_LOOKUP_AND_DELETE_ELEM , &attr, attr_sz); +if (!ret) +mark_map_value_defined (fd, value);return libbpf_err_errno (ret); }
@@ -454,6 +491,8 @@ int bpf_map_lookup_and_delete_elem_flags(int fd, const void *key, void *value, _ attr.flags = flags; ret =sys_bpf (BPF_MAP_LOOKUP_AND_DELETE_ELEM , &attr, attr_sz); +if (!ret) +mark_map_value_defined (fd, value);return libbpf_err_errno (ret); }
@@ -823,10 +862,12 @@ int bpf_prog_query_opts(int target_fd, { const size_t attr_sz = offsetofend(union bpf_attr, query);union bpf_attr attr; +__u32 *prog_ids;int ret;if (!OPTS_VALID (opts, bpf_prog_query_opts))return libbpf_err (-EINVAL ); + prog_ids =OPTS_GET (opts, prog_ids,NULL );memset (&attr,0 , attr_sz);
@@ -834,11 +875,15 @@ int bpf_prog_query_opts(int target_fd, attr.query.attach_type = type; attr.query.query_flags =OPTS_GET (opts, query_flags,0 ); attr.query.prog_cnt =OPTS_GET (opts, prog_cnt,0 ); - attr.query.prog_ids =ptr_to_u64 (OPTS_GET (opts, prog_ids,NULL )); + attr.query.prog_ids =ptr_to_u64 (prog_ids); attr.query.prog_attach_flags =ptr_to_u64 (OPTS_GET (opts, prog_attach_flags,NULL )); ret =sys_bpf (BPF_PROG_QUERY , &attr, attr_sz); +if (!ret && prog_ids) +libbpf_mark_defined (prog_ids, + attr.query.prog_cnt *sizeof (*prog_ids)); +OPTS_SET (opts, attach_flags, attr.query.attach_flags);OPTS_SET (opts, prog_cnt, attr.query.prog_cnt);
@@ -868,10 +913,14 @@ int bpf_prog_test_run_opts(int prog_fd, struct bpf_test_run_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, test);union bpf_attr attr; +void *data_out; +void *ctx_out;int ret;if (!OPTS_VALID (opts, bpf_test_run_opts))return libbpf_err (-EINVAL ); + data_out =OPTS_GET (opts, data_out,NULL ); + ctx_out =OPTS_GET (opts, ctx_out,NULL );memset (&attr,0 , attr_sz); attr.test.prog_fd = prog_fd;
@@ -885,12 +934,19 @@ int bpf_prog_test_run_opts(int prog_fd, struct bpf_test_run_opts *opts) attr.test.data_size_in =OPTS_GET (opts, data_size_in,0 ); attr.test.data_size_out =OPTS_GET (opts, data_size_out,0 ); attr.test.ctx_in =ptr_to_u64 (OPTS_GET (opts, ctx_in,NULL )); - attr.test.ctx_out =ptr_to_u64 (OPTS_GET (opts, ctx_out,NULL )); + attr.test.ctx_out =ptr_to_u64 (ctx_out); attr.test.data_in =ptr_to_u64 (OPTS_GET (opts, data_in,NULL )); - attr.test.data_out =ptr_to_u64 (OPTS_GET (opts, data_out,NULL )); + attr.test.data_out =ptr_to_u64 (data_out); ret =sys_bpf (BPF_PROG_TEST_RUN , &attr, attr_sz); +if (!ret) { +if (data_out) +libbpf_mark_defined (data_out, attr.test.data_size_out); +if (ctx_out) +libbpf_mark_defined (ctx_out, attr.test.ctx_size_out); + } +OPTS_SET (opts, data_size_out, attr.test.data_size_out);OPTS_SET (opts, ctx_size_out, attr.test.ctx_size_out);OPTS_SET (opts, duration, attr.test.duration);
@@ -1039,8 +1095,10 @@ int bpf_obj_get_info_by_fd(int bpf_fd, void *info, __u32 *info_len) attr.info.info =ptr_to_u64 (info); err =sys_bpf (BPF_OBJ_GET_INFO_BY_FD , &attr, attr_sz); -if (!err) +if (!err) { *info_len = attr.info.info_len; +libbpf_mark_defined (info, attr.info.info_len); + }return libbpf_err_errno (err); }
@@ -1103,6 +1161,8 @@ int bpf_btf_load(const void *btf_data, size_t btf_size, const struct bpf_btf_loa attr.btf_log_level =1 ; fd =sys_bpf_fd (BPF_BTF_LOAD , &attr, attr_sz); } +if (log_buf) +libbpf_mark_defined (log_buf, attr.btf_log_size);return libbpf_err_errno (fd); }
@@ -1122,6 +1182,8 @@ int bpf_task_fd_query(int pid, int fd, __u32 flags, char *buf, __u32 *buf_len, attr.task_fd_query.buf_len = *buf_len; err =sys_bpf (BPF_TASK_FD_QUERY , &attr, attr_sz); +if (!err && buf) +libbpf_mark_defined (buf, attr.task_fd_query.buf_len +1 ); *buf_len = attr.task_fd_query.buf_len; *prog_id = attr.task_fd_query.prog_id;
tools/lib/bpf/btf.c +1 -0
@@ -1388,6 +1388,7 @@ struct btf *btf_get_from_fd(int btf_fd, struct btf *base_btf) goto exit_free; } +libbpf_mark_defined (ptr, btf_info.btf_size); btf =btf_new (ptr, btf_info.btf_size, base_btf); exit_free:
tools/lib/bpf/libbpf.c +1 -0
@@ -5443,6 +5443,7 @@ static int load_module_btfs(struct bpf_object *obj)pr_warn ("failed to get BTF object #%d info: %d\n" , id, err); goto err_out; } +libbpf_mark_defined (name, info.name_len +1 );/* ignore non-module BTFs */ if (!info.kernel_btf ||strcmp (name,"vmlinux" ) ==0 ) {
tools/lib/bpf/libbpf_internal.h +14 -0
@@ -577,4 +577,18 @@ static inline bool is_pow_of_2(size_t x)#define PROG_LOAD_ATTEMPTS 5int sys_bpf_prog_load (union bpf_attr *attr,unsigned int size,int attempts); +#if defined(__has_feature) +#if __has_feature (memory_sanitizer) +#define LIBBPF_MSAN +#endif +#endif + +#ifdef LIBBPF_MSAN +#define HAVE_LIBBPF_MARK_DEFINED +#include <sanitizer/msan_interface.h> +#define libbpf_mark_defined __msan_unpoison +#else +static inline void libbpf_mark_defined (void *s,size_t n) {} +#endif +#endif /* __LIBBPF_LIBBPF_INTERNAL_H */
Signature · 1 line
2.39.1
PATCH 9/9 selftests/bpf: Add MSan annotations
In reply to Ilya LeoshkevicheBPF selftests produce a few false positives with MSan. These can be divided in two classes:
- Sending uninitalized data via a socket.
- bpf_obj_get_info_by_fd() calls.
The first class is trivial; the second should ideally be handled by libbpf, but it doesn't look possible at the moment, since we don't know the type of the eBPF object referred to by fd, and therefore the structure of the output data.
--- tools/testing/selftests/bpf/cap_helpers.c | 3 +++ tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c | 10 ++++++++++ tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c | 3 +++ tools/testing/selftests/bpf/prog_tests/btf.c | 11 +++++++++++ tools/testing/selftests/bpf/prog_tests/send_signal.c | 2 ++ .../selftests/bpf/prog_tests/tp_attach_query.c | 6 ++++++ tools/testing/selftests/bpf/prog_tests/xdp_bonding.c | 3 +++ tools/testing/selftests/bpf/xdp_synproxy.c | 2 ++ 8 files changed, 40 insertions(+)
tools/testing/selftests/bpf/cap_helpers.c +3 -0
@@ -1,4 +1,5 @@// SPDX-License-Identifier: GPL-2.0 +#include <bpf/libbpf_internal.h> #include "cap_helpers.h" /* Avoid including <sys/capability.h> from the libcap-devel package,
@@ -20,6 +21,7 @@ int cap_enable_effective(__u64 caps, __u64 *old_caps) err =capget (&hdr, data);if (err)return err; +libbpf_mark_defined (data,sizeof (data));if (old_caps) *old_caps = (__u64)(data[1 ].effective) <<32 | data[0 ].effective;
@@ -50,6 +52,7 @@ int cap_disable_effective(__u64 caps, __u64 *old_caps) err =capget (&hdr, data);if (err)return err; +libbpf_mark_defined (data,sizeof (data));if (old_caps) *old_caps = (__u64)(data[1 ].effective) <<32 | data[0 ].effective;
tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c +10 -0
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0 #include <test_progs.h> +#include <bpf/libbpf_internal.h> #define nr_iters 2
@@ -31,6 +32,7 @@ void serial_test_bpf_obj_id(void)__u64 array_value;uid_t my_uid =getuid ();time_t now, load_time; +int tp_name_len; err =bpf_prog_get_fd_by_id (0 ); CHECK(err >= 0 || errno != ENOENT,
@@ -122,6 +124,10 @@ void serial_test_bpf_obj_id(void) &info_len); load_time = (real_time_ts.tv_sec - boot_time_ts.tv_sec) + (prog_infos[i].load_time / nsec_per_sec); +if (!err) +libbpf_mark_defined (&map_ids[i], + prog_infos[i].nr_map_ids * +sizeof (map_ids[0 ])); if (CHECK(err || prog_infos[i].type != BPF_PROG_TYPE_RAW_TRACEPOINT || info_len != sizeof(struct bpf_prog_info) ||
@@ -163,6 +169,10 @@ void serial_test_bpf_obj_id(void) link_infos[i].raw_tracepoint.tp_name_len =sizeof (tp_name); err =bpf_obj_get_info_by_fd (bpf_link__fd (links[i]), &link_infos[i], &info_len); +if (!err) { + tp_name_len = link_infos[i].raw_tracepoint.tp_name_len; +libbpf_mark_defined (tp_name, tp_name_len +1 ); + } if (CHECK(err || link_infos[i].type != BPF_LINK_TYPE_RAW_TRACEPOINT || link_infos[i].prog_id != prog_infos[i].id ||
tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c +3 -0
@@ -4,6 +4,7 @@#include <linux/err.h> #include <netinet/tcp.h> #include <test_progs.h> +#include <bpf/libbpf_internal.h> #include "network_helpers.h" #include "bpf_dctcp.skel.h" #include "bpf_cubic.skel.h"
@@ -39,6 +40,8 @@ static void *server(void *arg)ssize_t nr_sent =0 , bytes =0 ;char batch[1500 ]; +libbpf_mark_defined (batch,sizeof (batch)); + fd =accept (lfd,NULL ,NULL );while (fd ==-1 ) {if (errno ==EINTR )
tools/testing/selftests/bpf/prog_tests/btf.c +11 -0
@@ -20,6 +20,7 @@#include <assert.h> #include <bpf/libbpf.h> #include <bpf/btf.h> +#include <bpf/libbpf_internal.h> #include "bpf_util.h" #include "../test_btf.h"
@@ -4500,6 +4501,8 @@ static int test_btf_id(unsigned int test_num)/* Test BPF_OBJ_GET_INFO_BY_ID on btf_id */ info_len =sizeof (info[0 ]); err =bpf_obj_get_info_by_fd (btf_fd[0 ], &info[0 ], &info_len); +if (!err) +libbpf_mark_defined (user_btf[0 ], info[0 ].btf_size);if (CHECK (err,"errno:%d" , errno)) { err =-1 ; goto done;
@@ -4513,6 +4516,8 @@ static int test_btf_id(unsigned int test_num) ret =0 ; err =bpf_obj_get_info_by_fd (btf_fd[1 ], &info[1 ], &info_len); +if (!err) +libbpf_mark_defined (user_btf[1 ], info[1 ].btf_size); if (CHECK(err || info[0].id != info[1].id || info[0].btf_size != info[1].btf_size || (ret = memcmp(user_btf[0], user_btf[1], info[0].btf_size)),
@@ -4639,6 +4644,8 @@ static void do_test_get_info(unsigned int test_num) ret =0 ; err =bpf_obj_get_info_by_fd (btf_fd, &info, &info_len); +if (!err) +libbpf_mark_defined (user_btf, info.btf_size); if (CHECK(err || !info.id || info_len != sizeof(info) || info.btf_size != raw_btf_size || (ret = memcmp(raw_btf, user_btf, expected_nbytes)),
@@ -4788,6 +4795,8 @@ static void do_test_file(unsigned int test_num) info.func_info =ptr_to_u64 (func_info); err =bpf_obj_get_info_by_fd (prog_fd, &info, &info_len); +if (!err) +libbpf_mark_defined (func_info, info.nr_func_info * rec_size);if (CHECK (err <0 ,"invalid get info (2nd) errno:%d" , errno)) {fprintf (stderr,"%s\n" , btf_log_buf);
@@ -6436,6 +6445,8 @@ static int test_get_finfo(const struct prog_info_raw_test *test, info.func_info_rec_size = rec_size; info.func_info =ptr_to_u64 (func_info); err =bpf_obj_get_info_by_fd (prog_fd, &info, &info_len); +if (!err) +libbpf_mark_defined (func_info, info.nr_func_info * rec_size);if (CHECK (err <0 ,"invalid get info (2nd) errno:%d" , errno)) {fprintf (stderr,"%s\n" , btf_log_buf); err =-1 ;
tools/testing/selftests/bpf/prog_tests/send_signal.c +2 -0
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0 #include <test_progs.h> +#include <bpf/libbpf_internal.h> #include <sys/time.h> #include <sys/resource.h> #include "test_send_signal_kern.skel.h"
@@ -58,6 +59,7 @@ static void test_send_signal_common(struct perf_event_attr *attr,ASSERT_OK (setpriority (PRIO_PROCESS ,0 ,-20 ),"setpriority" );/* notify parent signal handler is installed */ +libbpf_mark_defined (buf,1 );ASSERT_EQ (write (pipe_c2p[1 ], buf,1 ),1 ,"pipe_write" );/* make sure parent enabled bpf program to send_signal */
tools/testing/selftests/bpf/prog_tests/tp_attach_query.c +6 -0
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0 #include <test_progs.h> +#include <bpf/libbpf_internal.h> void serial_test_tp_attach_query (void ) {
@@ -65,6 +66,7 @@ void serial_test_tp_attach_query(void)if (i ==0 ) {/* check NULL prog array query */ query->ids_len = num_progs; + query->prog_cnt =0 ; err =ioctl (pmu_fd[i],PERF_EVENT_IOC_QUERY_BPF , query); if (CHECK(err || query->prog_cnt != 0, "perf_event_ioc_query_bpf",
@@ -109,6 +111,10 @@ void serial_test_tp_attach_query(void) query->ids_len = num_progs; err =ioctl (pmu_fd[i],PERF_EVENT_IOC_QUERY_BPF , query); +if (!err) +libbpf_mark_defined (query->ids, + query->prog_cnt * +sizeof (query->ids[0 ])); if (CHECK(err || query->prog_cnt != (i + 1), "perf_event_ioc_query_bpf", "err %d errno %d query->prog_cnt %u\n",
tools/testing/selftests/bpf/prog_tests/xdp_bonding.c +3 -0
@@ -14,6 +14,7 @@#include <net/if.h> #include <linux/if_link.h> #include "test_progs.h" +#include "bpf/libbpf_internal.h" #include "network_helpers.h" #include <linux/if_bonding.h> #include <linux/limits.h>
@@ -224,6 +225,8 @@ static int send_udp_packets(int vary_dst_ip)int i, s =-1 ;int ifindex; +libbpf_mark_defined (buf,sizeof (buf)); + s =socket (AF_PACKET ,SOCK_RAW ,IPPROTO_RAW );if (!ASSERT_GE (s,0 ,"socket" )) goto err;
tools/testing/selftests/bpf/xdp_synproxy.c +2 -0
@@ -12,6 +12,7 @@#include <sys/types.h> #include <bpf/bpf.h> #include <bpf/libbpf.h> +#include <bpf/libbpf_internal.h> #include <net/if.h> #include <linux/if_link.h> #include <linux/limits.h>
@@ -297,6 +298,7 @@ static int syncookie_open_bpf_maps(__u32 prog_id, int *values_map_fd, int *portsfprintf (stderr,"Error: bpf_obj_get_info_by_fd: %s\n" ,strerror (-err)); goto out; } +libbpf_mark_defined (map_ids, prog_info.nr_map_ids *sizeof (map_ids[0 ]));if (prog_info.nr_map_ids <2 ) { fprintf(stderr, "Error: Found %u BPF maps, expected at least 2\n",
Signature · 1 line
2.39.1
Re: [PATCH bpf-next 2/9] tools: runqslower: Add EXTRA_CFLAGS and EXTRA_LDFLAGS support
In reply to Ilya LeoshkevichQuoted 13 lines · Ilya Leoshkevich
This makes it possible to add sanitizer flags.
Signed-off-by Ilya Leoshkevich <[email protected]>--- tools/bpf/runqslower/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)tools/bpf/runqslower/Makefile +0
-0
tools/bpf/runqslower/Makefile BPF_DESTDIR := $(BPFOBJ_OUTPUT) · line 22
Quoted 10 lines
tools/bpf/runqslower/Makefile +6
-0@@ -13,6 +13,12 @@ BPF_DESTDIR := $(BPFOBJ_OUTPUT) BPF_INCLUDE := $(BPF_DESTDIR)/include INCLUDES := -I$(OUTPUT) -I$(BPF_INCLUDE) -I$(abspath ../../include/uapi) CFLAGS := -g -Wall $(CLANG_CROSS_FLAGS) +ifneq ($(EXTRA_CFLAGS),) +CFLAGS += $(EXTRA_CFLAGS) +endif +ifneq ($(EXTRA_LDFLAGS),) +LDFLAGS += $(EXTRA_LDFLAGS) +endif
can't we do this unconditionally? If those variables are not defined, Makefile will substitute empty string, no?
Quoted 5 lines
tools/bpf/runqslower/Makefile +0
-1# Try to detect best kernel BTF source KERNEL_REL := $(shell uname -r) --2.39.1
Re: [PATCH bpf-next 4/9] selftests/bpf: Forward SAN_CFLAGS and SAN_LDFLAGS to runqslower and libbpf
In reply to Ilya LeoshkevichQuoted 17 lines · Ilya Leoshkevich
To get useful results from the Memory Sanitizer, all code running in a process needs to be instrumented. When building tests with other sanitizers, it's not strictly necessary, but is also helpful. So make sure runqslower and libbpf are compiled with
SAN_CFLAGSand linked withSAN_LDFLAGS.Signed-off-by Ilya Leoshkevich <[email protected]>--- tools/testing/selftests/bpf/Makefile | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/Makefile +0
-0
tools/testing/selftests/bpf/Makefile $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT) · line 222
Quoted 9 lines
tools/testing/selftests/bpf/Makefile +3
-1@@ -215,7 +215,9 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT) OUTPUT=$(RUNQSLOWER_OUTPUT) VMLINUX_BTF=$(VMLINUX_BTF) \ BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \ BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf \ - BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) && \ + BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) \ + EXTRA_CFLAGS='-g -O0 $(SAN_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS)' && \ cp $(RUNQSLOWER_OUTPUT)runqslower $@
I wouldn't do it for runqslower, we just make sure that it compiles, we don't really run it at all. No need to complicate its build, IMO.
Quoted 14 lines
tools/testing/selftests/bpf/Makefile +2
-2TEST_GEN_PROGS_EXTENDED += $(DEFAULT_BPFTOOL)@@ -272,7 +274,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ $(APIDIR)/linux/bpf.h \ | $(BUILD_DIR)/libbpf $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \ - EXTRA_CFLAGS='-g -O0' \ + EXTRA_CFLAGS='-g -O0 $(SAN_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS)' \ DESTDIR=$(SCRATCH_DIR) prefix= all install_headers ifneq ($(BPFOBJ),$(HOST_BPFOBJ)) --2.39.1
Re: [PATCH bpf-next 5/9] selftests/bpf: Attach to fopen()/fclose() in uprobe_autoattach
In reply to Ilya LeoshkevichQuoted 61 lines · Ilya Leoshkevich
malloc()andfree()may be completely replaced by sanitizers, usefopen()andfclose()instead.Signed-off-by Ilya Leoshkevich <[email protected]>--- .../selftests/bpf/prog_tests/uprobe_autoattach.c | 12 ++++++------ .../selftests/bpf/progs/test_uprobe_autoattach.c | 10 +++++----- 2 files changed, 11 insertions(+), 11 deletions(-)tools/testing/selftests/bpf/prog_tests/uprobe_autoattach.c +6
-6@@ -16,10 +16,10 @@ static noinline int autoattach_trigger_func(int arg1, int arg2, int arg3,void test_uprobe_autoattach (void ) { +const char *devnull_str ="/dev/null" ;struct test_uprobe_autoattach *skel;int trigger_ret; -size_t malloc_sz =1 ; -char *mem; +FILE *devnull; skel =test_uprobe_autoattach__open_and_load ();if (!ASSERT_OK_PTR (skel,"skel_open" ))@@ -36,16 +36,16 @@ void test_uprobe_autoattach(void) skel->bss->test_pid =getpid ();/* trigger & validate shared library u[ret]probes attached by name */ - mem =malloc (malloc_sz); + devnull =fopen (devnull_str,"r" );ASSERT_EQ (skel->bss->uprobe_byname_parm1,1 ,"check_uprobe_byname_parm1" );ASSERT_EQ (skel->bss->uprobe_byname_ran,1 ,"check_uprobe_byname_ran" );ASSERT_EQ (skel->bss->uretprobe_byname_rc, trigger_ret,"check_uretprobe_byname_rc" );ASSERT_EQ (skel->bss->uretprobe_byname_ret, trigger_ret,"check_uretprobe_byname_ret" );ASSERT_EQ (skel->bss->uretprobe_byname_ran,2 ,"check_uretprobe_byname_ran" ); -ASSERT_EQ (skel->bss->uprobe_byname2_parm1, malloc_sz,"check_uprobe_byname2_parm1" ); +ASSERT_EQ (skel->bss->uprobe_byname2_parm1, devnull_str,"check_uprobe_byname2_parm1" );ASSERT_EQ (skel->bss->uprobe_byname2_ran,3 ,"check_uprobe_byname2_ran" ); -ASSERT_EQ (skel->bss->uretprobe_byname2_rc, mem,"check_uretprobe_byname2_rc" ); +ASSERT_EQ (skel->bss->uretprobe_byname2_rc, (void *)devnull,"check_uretprobe_byname2_rc" );ASSERT_EQ (skel->bss->uretprobe_byname2_ran,4 ,"check_uretprobe_byname2_ran" );ASSERT_EQ (skel->bss->a[0 ],1 ,"arg1" );@@ -67,7 +67,7 @@ void test_uprobe_autoattach(void)ASSERT_EQ (skel->bss->a[7 ],8 ,"arg8" );#endif -free (mem); +fclose (devnull); cleanup:test_uprobe_autoattach__destroy (skel); }tools/testing/selftests/bpf/progs/test_uprobe_autoattach.c +0
-0
tools/testing/selftests/bpf/progs/test_uprobe_autoattach.c int uprobe_byname_ran = 0; · line 18
Quoted 8 lines
tools/testing/selftests/bpf/progs/test_uprobe_autoattach.c +2
-2@@ -13,9 +13,9 @@ int uprobe_byname_ran = 0;int uretprobe_byname_rc =0 ;int uretprobe_byname_ret =0 ;int uretprobe_byname_ran =0 ; -size_t uprobe_byname2_parm1 =0 ; +void *uprobe_byname2_parm1 =NULL ;int uprobe_byname2_ran =0 ; -char *uretprobe_byname2_rc =NULL ; +void *uretprobe_byname2_rc =NULL ;
ugh... shall we use u64 and avoid problems with 32-bit host vs 64-bit BPF mismatch? Maybe it will never bite us, but why risking?
Quoted 30 lines
tools/testing/selftests/bpf/progs/test_uprobe_autoattach.c +3
-4int uretprobe_byname2_ran =0 ;int test_pid;@@ -88,7 +88,7 @@ int BPF_URETPROBE(handle_uretprobe_byname, int ret) } -SEC("uprobe/libc.so.6:malloc") +SEC("uprobe/libc.so.6:fopen")int handle_uprobe_byname2 (struct pt_regs *ctx) {int pid =bpf_get_current_pid_tgid () >>32 ;@@ -96,12 +96,12 @@ int handle_uprobe_byname2(struct pt_regs *ctx)/* ignore irrelevant invocations */ if (test_pid != pid)return 0 ; - uprobe_byname2_parm1 =PT_REGS_PARM1_CORE (ctx); + uprobe_byname2_parm1 = (void *)(long )PT_REGS_PARM1_CORE (ctx); uprobe_byname2_ran =3 ;return 0 ; } -SEC("uretprobe/libc.so.6:malloc") +SEC("uretprobe/libc.so.6:fopen")int handle_uretprobe_byname2 (struct pt_regs *ctx) {int pid =bpf_get_current_pid_tgid () >>32 ; --2.39.1
Re: [PATCH bpf-next 6/9] selftests/bpf: Attach to fopen()/fclose() in attach_probe
In reply to Ilya LeoshkevichQuoted 58 lines · Ilya Leoshkevich
malloc()andfree()may be completely replaced by sanitizers, usefopen()andfclose()instead.Signed-off-by Ilya Leoshkevich <[email protected]>--- tools/testing/selftests/bpf/prog_tests/attach_probe.c | 10 +++++----- tools/testing/selftests/bpf/progs/test_attach_probe.c | 8 +++++--- 2 files changed, 10 insertions(+), 8 deletions(-)tools/testing/selftests/bpf/prog_tests/attach_probe.c +5
-5@@ -33,8 +33,8 @@ void test_attach_probe(void)struct test_attach_probe * skel;ssize_t uprobe_offset, ref_ctr_offset;struct bpf_link *uprobe_err_link; +FILE *devnull;bool legacy; -char *mem;/* Check if new-style kprobe/uprobe API is supported. * Kernels that support new FD-based kprobe and uprobe BPF attachment @@ -147,7 +147,7 @@ void test_attach_probe(void)/* test attach by name for a library function, using the library * as the binary argument. libc.so.6 will be resolved via dlopen()/dlinfo(). */ - uprobe_opts.func_name ="malloc" ; + uprobe_opts.func_name ="fopen" ; uprobe_opts.retprobe = false; skel->links.handle_uprobe_byname2 = bpf_program__attach_uprobe_opts(skel->progs.handle_uprobe_byname2,@@ -157,7 +157,7 @@ void test_attach_probe(void)if (!ASSERT_OK_PTR (skel->links.handle_uprobe_byname2,"attach_uprobe_byname2" )) goto cleanup; - uprobe_opts.func_name ="free" ; + uprobe_opts.func_name ="fclose" ; uprobe_opts.retprobe = true; skel->links.handle_uretprobe_byname2 = bpf_program__attach_uprobe_opts(skel->progs.handle_uretprobe_byname2,@@ -195,8 +195,8 @@ void test_attach_probe(void)usleep (1 );/* trigger & validate shared library u[ret]probes attached by name */ - mem =malloc (1 ); -free (mem); + devnull =fopen ("/dev/null" ,"r" ); +fclose (devnull);/* trigger & validate uprobe & uretprobe */ trigger_func ();tools/testing/selftests/bpf/progs/test_attach_probe.c +0
-0
tools/testing/selftests/bpf/progs/test_attach_probe.c int handle_uretprobe_byname(struct pt_regs *ctx) · line 97
Quoted 5 lines
tools/testing/selftests/bpf/progs/test_attach_probe.c +1
-1@@ -94,10 +94,12 @@ int handle_uretprobe_byname(struct pt_regs *ctx) SEC("uprobe")int handle_uprobe_byname2 (struct pt_regs *ctx) { -unsigned int size =PT_REGS_PARM1 (ctx); +void *mode_ptr = (void *)(long )PT_REGS_PARM2 (ctx);
let's use BPF_UPROBE() macro instead of PT_REGS_xxx() calls?
Quoted 13 lines
tools/testing/selftests/bpf/progs/test_attach_probe.c +4
-3+char mode[2 ] = {}; -/* verify malloc size */ -if (size ==1 ) +/* verify fopen mode */ +bpf_probe_read_user (mode,sizeof (mode), mode_ptr); +if (mode[0 ] =='r' && mode[1 ] ==0 ) uprobe_byname2_res =7 ;return 0 ; } --2.39.1
Re: [PATCH bpf-next 7/9] libbpf: Fix alen calculation in libbpf_nla_dump_errormsg()
In reply to Ilya LeoshkevichQuoted 15 lines · Ilya Leoshkevich
The code assumes that everything that comes after nlmsgerr are nlattrs. When calculating their size, it does not account for the initial nlmsghdr. This may lead to accessing uninitialized memory.
Signed-off-by Ilya Leoshkevich <[email protected]>--- tools/lib/bpf/nlattr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/lib/bpf/nlattr.c +0
-0
tools/lib/bpf/nlattr.c int libbpf_nla_dump_errormsg(struct nlmsghdr *nlh) · line 181
Quoted 5 lines
tools/lib/bpf/nlattr.c +1
-1@@ -178,7 +178,7 @@ int libbpf_nla_dump_errormsg(struct nlmsghdr *nlh) hlen +=nlmsg_len (&err->msg); attr = (struct nlattr *) ((void *) err + hlen); - alen = nlh->nlmsg_len - hlen; + alen = (char *)nlh + nlh->nlmsg_len - (char *)attr;
we use (void *) for pointer manipulations, let's be consistent? Otherwise looks good (I think, this whole nlattr stuff is very cryptic to me).
Quoted 6 lines
tools/lib/bpf/nlattr.c +0
-1if (libbpf_nla_parse (tb,NLMSGERR_ATTR_MAX , attr, alen, extack_policy) !=0 ) { --2.39.1
Re: [PATCH bpf-next 8/9] libbpf: Add MSan annotations
In reply to Ilya LeoshkevichQuoted 5 lines · Ilya Leoshkevich
MSan runs into a few false positives in libbpf. They all come from the fact that MSan does not know anything about the bpf syscall, particularly, what it writes to.
Quoted 7 lines
Add
libbpf_mark_defined()function to mark memory modified by the bpf syscall. Use the abstract name (it could be e.g.libbpf_msan_unpoison()), because it can be used for valgrind in the future as well.Signed-off-by Ilya Leoshkevich <[email protected]>---
This is a lot to satisfy MSan, especially mark_map_value_defined which has to do bpf_obj_get_info_by_fd()... Is there any other way? What happens if we don't do it?
As for libbpf_mark_defined, wouldn't it be cleaner to teach it to handle NULL pointer and/or zero size transparently? Also, we can have libbpf_mark_defined_if(void *ptr, size_t sz, bool cond), so in code we don't have to have those explicit if conditions. Instead of:
tools/lib/bpf/bpf.c int bpf_prog_query_opts(int target_fd, · line 885
Quoted 3 lines
tools/lib/bpf/bpf.c +3
-0+if (!ret && prog_ids) +libbpf_mark_defined (prog_ids, + attr.query.prog_cnt *sizeof (*prog_ids));
we can write just
libbpf_mark_defined_if(prog_ids, attr.query.prog_cnt * sizeof(*prog_ids), ret == 0);
?
Should we also call a helper something like 'libbpf_mark_mem_written', because that's what we are saying here, right?
Quoted 6 lines
tools/lib/bpf/bpf.c | 70 +++++++++++++++++++++++++++++++-- tools/lib/bpf/btf.c | 1 + tools/lib/bpf/libbpf.c | 1 + tools/lib/bpf/libbpf_internal.h | 14 +++++++ 4 files changed, 82 insertions(+), 4 deletions(-)
[...]
Quoted 5 lines
tools/lib/bpf/libbpf_internal.h +0
-0
tools/lib/bpf/libbpf_internal.h static inline bool is_pow_of_2(size_t x) · line 586
Quoted 10 lines
tools/lib/bpf/libbpf_internal.h +7
-0@@ -577,4 +577,18 @@ static inline bool is_pow_of_2(size_t x)#define PROG_LOAD_ATTEMPTS 5int sys_bpf_prog_load (union bpf_attr *attr,unsigned int size,int attempts); +#if defined(__has_feature) +#if __has_feature (memory_sanitizer) +#define LIBBPF_MSAN +#endif +#endif + +#ifdef LIBBPF_MSAN
would below work:
#if defined(__has_feature) && __has_feature(memory_sanitizer)
?
Quoted 11 lines
tools/lib/bpf/libbpf_internal.h +7
-1+#define HAVE_LIBBPF_MARK_DEFINED +#include <sanitizer/msan_interface.h> +#define libbpf_mark_defined __msan_unpoison +#else +static inline void libbpf_mark_defined (void *s,size_t n) {} +#endif +#endif /* __LIBBPF_LIBBPF_INTERNAL_H */ --2.39.1
Re: [PATCH bpf-next 9/9] selftests/bpf: Add MSan annotations
In reply to Ilya LeoshkevichQuoted 11 lines · Ilya Leoshkevich
eBPF selftests produce a few false positives with MSan. These can be divided in two classes:
- Sending uninitalized data via a socket.
-bpf_obj_get_info_by_fd()calls.The first class is trivial; the second should ideally be handled by libbpf, but it doesn't look possible at the moment, since we don't know the type of the eBPF object referred to by fd, and therefore the structure of the output data.
yeah, bpf_obj_get_info_by_fd() is quite bad from usability standpoint. I think we should add bpf_get_{map,prog,link,btf}_info_by_fd() wrappers and try to use them. That will allow to specify correct expected struct types, we'll be able to mark initialized memory properly. We already have bpf_{map,prog,btf,link}_get_fd_by_id() family, so having similar for getting info seems fitting (even if underlying bpf() command is generic).
Thoughts?
Quoted 13 lines
Signed-off-by Ilya Leoshkevich <[email protected]>--- tools/testing/selftests/bpf/cap_helpers.c | 3 +++ tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c | 10 ++++++++++ tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c | 3 +++ tools/testing/selftests/bpf/prog_tests/btf.c | 11 +++++++++++ tools/testing/selftests/bpf/prog_tests/send_signal.c | 2 ++ .../selftests/bpf/prog_tests/tp_attach_query.c | 6 ++++++ tools/testing/selftests/bpf/prog_tests/xdp_bonding.c | 3 +++ tools/testing/selftests/bpf/xdp_synproxy.c | 2 ++ 8 files changed, 40 insertions(+)
[...]
Re: [PATCH bpf-next 4/9] selftests/bpf: Forward SAN_CFLAGS and SAN_LDFLAGS to runqslower and libbpf
In reply to Andrii NakryikoQuoted 33 lines · Andrii Nakryiko
Quoted 31 lines · Ilya Leoshkevich
To get useful results from the Memory Sanitizer, all code running in a
process needs to be instrumented. When building tests with other sanitizers, it's not strictly necessary, but is also helpful. So make sure runqslower and libbpf are compiled withSAN_CFLAGSand linked withSAN_LDFLAGS.Signed-off-by Ilya Leoshkevich <[email protected]>---
tools/testing/selftests/bpf/Makefile | 7 +++++--1 file changed, 5 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/Makefile +0
-0@@ -215,7 +215,9 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT) OUTPUT=$(RUNQSLOWER_OUTPUT) VMLINUX_BTF=$(VMLINUX_BTF) \
BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \
BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf \
- BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) && \
+ BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) \
+ EXTRA_CFLAGS='-g -O0
Quoted 8 lines
Quoted 5 lines
$(SAN_CFLAGS)' \
+ EXTRA_LDFLAGS='$(SAN_LDFLAGS)' && \
cp $(RUNQSLOWER_OUTPUT)runqslower $@I wouldn't do it for runqslower, we just make sure that it compiles, we don't really run it at all. No need to complicate its build, IMO.
runqslower is linked with target libbpf, which is instrumented. This produces undefined symbol errors, since MSan runtime is expected to be a part of an executable.
[...]
Re: [PATCH bpf-next 8/9] libbpf: Add MSan annotations
In reply to Andrii NakryikoQuoted 14 lines · Andrii Nakryiko
Quoted 12 lines · Ilya Leoshkevich
MSan runs into a few false positives in libbpf. They all come from the
fact that MSan does not know anything about the bpf syscall, particularly, what it writes to.Add
libbpf_mark_defined()function to mark memory modified by the bpf
syscall. Use the abstract name (it could be e.g.libbpf_msan_unpoison()), because it can be used for valgrind in the future as well.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Ilya Leoshkevich <[email protected]>---
This is a lot to satisfy MSan, especially
mark_map_value_definedwhich
has to dobpf_obj_get_info_by_fd()... Is there any other way? What happens if we don't do it?
It would generate false positives when accessing the resulting map values, which is not nice. The main complexity in this case comes not from mark_map_value_defined() per se, but from the fact that we don't know the value size, especially for percpu maps.
I toyed with the idea to extend the BPF_MAP_LOOKUP_ELEM interface to return the number of bytes written, but decided against it - the only user of this would be MSan, and at least for the beginning it's better to have extra complexity in userspace, rather than in kernel.
Quoted 13 lines
As for
libbpf_mark_defined, wouldn't it be cleaner to teach it to handle NULL pointer and/or zero size transparently? Also, we can havelibbpf_mark_defined_if(void *ptr, size_t sz, bool cond), so in code we
don't have to have those explicit if conditions. Instead of:Quoted 4 lines
+ if (!ret &&
prog_ids)
+ libbpf_mark_defined(prog_ids,
+ attr.query.prog_cnt *sizeof(*prog_ids));we can write just
Quoted 8 lines
libbpf_mark_defined_if(prog_ids, attr.query.prog_cnt * sizeof(*prog_ids), ret == 0);?
Should we also call a helper something like
'libbpf_mark_mem_written',
because that's what we are saying here, right?
Sure, all this sounds reasonable. Will do.
Quoted 46 lines
Quoted 8 lines
tools/lib/bpf/bpf.c | 70
+++++++++++++++++++++++++++++++--
tools/lib/bpf/btf.c | 1 +
tools/lib/bpf/libbpf.c | 1 +
tools/lib/bpf/libbpf_internal.h | 14 +++++++4 files changed, 82 insertions(+), 4 deletions(-)[...]
Quoted 10 lines
tools/lib/bpf/libbpf_internal.h +0
-0@@ -577,4 +577,18 @@ static inline bool is_pow_of_2(size_t x) #define PROG_LOAD_ATTEMPTS 5int
sys_bpf_prog_load(union bpf_attr *attr, unsigned int size, int attempts);tools/lib/bpf/libbpf_internal.h static inline bool is_pow_of_2(size_t x) · line 586
Quoted 7 lines
tools/lib/bpf/libbpf_internal.h +7
-0+#if defined(__has_feature) +#if __has_feature (memory_sanitizer) +#define LIBBPF_MSAN +#endif +#endif + +#ifdef LIBBPF_MSAN would below work:
#if
defined(__has_feature)&&__has_feature(memory_sanitizer)?
Quoted 11 lines
tools/lib/bpf/libbpf_internal.h +7
-0+#define HAVE_LIBBPF_MARK_DEFINED +#include <sanitizer/msan_interface.h> +#define libbpf_mark_defined __msan_unpoison +#else +static inline void libbpf_mark_defined (void *s,size_t n) {} +#endif +#endif /*
__LIBBPF_LIBBPF_INTERNAL_H*/
--
2.39.1
Re: [PATCH bpf-next 9/9] selftests/bpf: Add MSan annotations
In reply to Andrii NakryikoQuoted 25 lines · Andrii Nakryiko
Quoted 5 lines · Ilya Leoshkevich
eBPF selftests produce a few false positives with MSan. These can be
divided in two classes:Quoted 8 lines
- Sending uninitalized data via a socket.
-bpf_obj_get_info_by_fd()calls.The first class is trivial; the second should ideally be handled by libbpf, but it doesn't look possible at the moment, since we don't know the type of the eBPF object referred to by fd, and therefore the
structure of the output data.yeah,
bpf_obj_get_info_by_fd()is quite bad from usability standpoint.
I think we should add bpf_get_{map,prog,link,btf}_info_by_fd() wrappers and try to use them. That will allow to specify correct expected struct types, we'll be able to mark initialized memory properly. We already have bpf_{map,prog,btf,link}_get_fd_by_id() family, so having similar for getting info seems fitting (even if underlyingbpf()command is generic).
Quoted 1 line
Thoughts?
Sounds good to me, I will give it a try.
Quoted 17 lines
Quoted 15 lines
Signed-off-by Ilya Leoshkevich <[email protected]>---
tools/testing/selftests/bpf/cap_helpers.c | 3 +++ tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c | 10
++++++++++
tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c | 3 +++ tools/testing/selftests/bpf/prog_tests/btf.c | 11
+++++++++++
tools/testing/selftests/bpf/prog_tests/send_signal.c | 2 ++ .../selftests/bpf/prog_tests/tp_attach_query.c | 6 ++++++ tools/testing/selftests/bpf/prog_tests/xdp_bonding.c | 3 +++ tools/testing/selftests/bpf/xdp_synproxy.c | 2 ++8 files changed, 40 insertions(+)[...]
Re: [PATCH bpf-next 4/9] selftests/bpf: Forward SAN_CFLAGS and SAN_LDFLAGS to runqslower and libbpf
In reply to Ilya LeoshkevichQuoted 44 lines · Ilya Leoshkevich
Quoted 27 lines · Andrii Nakryiko
Quoted 25 lines · Ilya Leoshkevich
To get useful results from the Memory Sanitizer, all code running in a
process needs to be instrumented. When building tests with other sanitizers, it's not strictly necessary, but is also helpful. So make sure runqslower and libbpf are compiled withSAN_CFLAGSand linked withSAN_LDFLAGS.Signed-off-by Ilya Leoshkevich <[email protected]>--- tools/testing/selftests/bpf/Makefile | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/Makefile +0
-0@@ -215,7 +215,9 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT) OUTPUT=$(RUNQSLOWER_OUTPUT) VMLINUX_BTF=$(VMLINUX_BTF) \ BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \Quoted 14 lines
Quoted 11 lines
tools/testing/selftests/bpf/Makefile +0
-1BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf \ - BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR)&& \
+ BPFOBJ=$(BPFOBJ)
BPF_INCLUDE=$(INCLUDE_DIR) \
+ EXTRA_CFLAGS='-g -O0
$(SAN_CFLAGS)' \
+ EXTRA_LDFLAGS='$(SAN_LDFLAGS)'&& \ cp $(RUNQSLOWER_OUTPUT)runqslower $@I wouldn't do it for runqslower, we just make sure that it compiles, we don't really run it at all. No need to complicate its build, IMO.
Quoted 3 lines
runqslower is linked with target libbpf, which is instrumented. This produces undefined symbol errors, since MSan runtime is expected to be a part of an executable.
ah, ok then, never mind
Quoted 2 lines
[...]
Re: [PATCH bpf-next 8/9] libbpf: Add MSan annotations
In reply to Ilya LeoshkevichQuoted 29 lines · Ilya Leoshkevich
Quoted 14 lines · Andrii Nakryiko
Quoted 12 lines · Ilya Leoshkevich
MSan runs into a few false positives in libbpf. They all come from the
fact that MSan does not know anything about the bpf syscall, particularly, what it writes to.Add
libbpf_mark_defined()function to mark memory modified by the bpf
syscall. Use the abstract name (it could be e.g.libbpf_msan_unpoison()), because it can be used for valgrind in the future as well.Quoted 7 lines
Quoted 2 lines
Signed-off-by Ilya Leoshkevich <[email protected]>---
This is a lot to satisfy MSan, especially
mark_map_value_definedwhich
has to dobpf_obj_get_info_by_fd()... Is there any other way? What happens if we don't do it?It would generate false positives when accessing the resulting map values, which is not nice. The main complexity in this case comes not from
mark_map_value_defined()per se, but from the fact that we don't know the value size, especially for percpu maps.
Quoted 4 lines
I toyed with the idea to extend the
BPF_MAP_LOOKUP_ELEMinterface to return the number of bytes written, but decided against it - the only user of this would be MSan, and at least for the beginning it's better to have extra complexity in userspace, rather than in kernel.
yep, I know, it's a source of very confusing problems in some cases. Which is why bpf_map__lookup_elem() and such expect user to provide total key/value size (and checks it against struct bpf_map's knowledge of key/value size, taking into account if map is per-CPU)
Ok, I don't see any other way around this as well. Just please extract this check of whether the map is per-cpu or not into a separate helper, so we can maintain this list in only one place. Currently we have this check in validate_map_op() and I don't want to duplicate this.
Quoted 72 lines
Quoted 14 lines
As for
libbpf_mark_defined, wouldn't it be cleaner to teach it to handle NULL pointer and/or zero size transparently? Also, we can havelibbpf_mark_defined_if(void *ptr, size_t sz, bool cond), so in code we
don't have to have those explicit if conditions. Instead of:Quoted 4 lines
tools/lib/bpf/bpf.c +3
-0+if (!ret && prog_ids) + libbpf_mark_defined(prog_ids, + attr.query.prog_cnt *
sizeof(*prog_ids));we can write just
Quoted 8 lines
libbpf_mark_defined_if(prog_ids, attr.query.prog_cnt * sizeof(*prog_ids), ret == 0);?
Should we also call a helper something like
'libbpf_mark_mem_written',
because that's what we are saying here, right?Sure, all this sounds reasonable. Will do.
Quoted 46 lines
Quoted 8 lines
tools/lib/bpf/bpf.c | 70 +++++++++++++++++++++++++++++++-- tools/lib/bpf/btf.c | 1 + tools/lib/bpf/libbpf.c | 1 + tools/lib/bpf/libbpf_internal.h | 14 +++++++ 4 files changed, 82 insertions(+), 4 deletions(-)[...]
Quoted 7 lines
tools/lib/bpf/libbpf_internal.h +0
-0@@ -577,4 +577,18 @@ static inline bool is_pow_of_2(size_t x)#define PROG_LOAD_ATTEMPTS 5tools/lib/bpf/libbpf_internal.h static inline bool is_pow_of_2(size_t x) · line 587
Quoted 10 lines
tools/lib/bpf/libbpf_internal.h +7
-0int sys_bpf_prog_load (union bpf_attr *attr,unsigned int size,int attempts); +#if defined(__has_feature) +#if __has_feature (memory_sanitizer) +#define LIBBPF_MSAN +#endif +#endif + +#ifdef LIBBPF_MSAN would below work:
#if
defined(__has_feature)&&__has_feature(memory_sanitizer)?
Quoted 11 lines
tools/lib/bpf/libbpf_internal.h +7
-0+#define HAVE_LIBBPF_MARK_DEFINED +#include <sanitizer/msan_interface.h> +#define libbpf_mark_defined __msan_unpoison +#else +static inline void libbpf_mark_defined (void *s,size_t n) {} +#endif +#endif /*
__LIBBPF_LIBBPF_INTERNAL_H*/
--
2.39.1