Enhance and rework logging controls in libbpf
Subsystems (5)
Andrii Nakryiko
maintainercover Enhance and rework logging controls in libbpf
Add new open options and per-program setters to control BTF and program loading log verboseness and allow providing custom log buffers to capture logs of interest. Note how custom log_buf and log_level are orthogonal, which matches previous (alas less customizable) behavior of libbpf, even though it sort of worked by accident: if someone specified log_level = 1 in bpf_object__load_xattr(), first attempt to load any BPF program resulted in wasted bpf() syscall with -EINVAL due to !!log_buf != !!log_level. Then on retry libbpf would allocated log_buffer and try again, after which prog loading would succeed and libbpf would print verbose program loading log through its print callback.
This behavior is now documented and made more efficient, not wasting unnecessary syscall. But additionally, log_level can be controlled globally on a per-bpf_object level through bpf_object_open_opts, as well as on a per-program basis with bpf_program__set_log_buf() and bpf_program__set_log_level() APIs.
Now that we have a more future-proof way to set log_level, deprecate bpf_object__load_xattr().
v1->v2:
- fix log_level == 0 handling of bpf_prog_load, add as patch #1 (Alexei);
- add comments explaining log_buf_size overflow prevention (Alexei).
Andrii Nakryiko (12):
libbpf: fix bpf_prog_load() log_buf logic for log_level 0
libbpf: add OPTS-based bpf_btf_load() API
libbpf: allow passing preallocated log_buf when loading BTF into
kernel
libbpf: allow passing user log setting through bpf_object_open_opts
libbpf: improve logging around BPF program loading
libbpf: preserve kernel error code and remove kprobe prog type
guessing
libbpf: add per-program log buffer setter and getter
libbpf: deprecate bpf_object__load_xattr()
selftests/bpf: replace all uses of bpf_load_btf() with bpf_btf_load() selftests/bpf: add test for libbpf's custom log_buf behavior selftests/bpf: remove the only use of deprecated
bpf_object__load_xattr()
bpftool: switch bpf_object__load_xattr() to bpf_object__load()
tools/bpf/bpftool/gen.c | 11 +- tools/bpf/bpftool/prog.c | 24 +-- tools/bpf/bpftool/struct_ops.c | 15 +- tools/lib/bpf/bpf.c | 88 ++++++-- tools/lib/bpf/bpf.h | 22 +- tools/lib/bpf/btf.c | 78 ++++--- tools/lib/bpf/libbpf.c | 194 ++++++++++++------ tools/lib/bpf/libbpf.h | 49 ++++- tools/lib/bpf/libbpf.map | 3 + tools/lib/bpf/libbpf_internal.h | 1 + tools/lib/bpf/libbpf_probes.c | 2 +- .../selftests/bpf/map_tests/sk_storage_map.c | 2 +- .../selftests/bpf/prog_tests/bpf_tcp_ca.c | 6 +- tools/testing/selftests/bpf/prog_tests/btf.c | 50 +++-- .../selftests/bpf/prog_tests/log_buf.c | 137 +++++++++++++ .../selftests/bpf/progs/test_log_buf.c | 24 +++ tools/testing/selftests/bpf/test_verifier.c | 2 +- tools/testing/selftests/bpf/testing_helpers.c | 10 +- 18 files changed, 544 insertions(+), 174 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/log_buf.c create mode 100644 tools/testing/selftests/bpf/progs/test_log_buf.c
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 1/12 libbpf: fix bpf_prog_load() log_buf logic for log_level 0
To unify libbpf APIs behavior w.r.t. log_buf and log_level, fix bpf_prog_load() to follow the same logic as bpf_btf_load() and high-level bpf_object__load() API will follow in the subsequent patches:
- if log_level is 0 and non-NULL log_buf is provided by a user, attempt
load operation initially with no log_buf and log_level set;
- if successful, we are done, return new FD;
- on error, retry the load operation with log_level bumped to 1 and log_buf set; this way verbose logging will be requested only when we are sure that there is a failure, but will be fast in the common/expected success case.
Of course, user can still specify log_level > 0 from the very beginning to force log collection.
--- tools/lib/bpf/bpf.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-)
tools/lib/bpf/bpf.c +16 -13
@@ -303,10 +303,6 @@ int bpf_prog_load_v0_6_0(enum bpf_prog_type prog_type,if (log_level && !log_buf)return libbpf_err (-EINVAL ); - attr.log_level = log_level; - attr.log_buf =ptr_to_u64 (log_buf); - attr.log_size = log_size; - func_info_rec_size =OPTS_GET (opts, func_info_rec_size,0 ); func_info =OPTS_GET (opts, func_info,NULL ); attr.func_info_rec_size = func_info_rec_size;
@@ -321,6 +317,12 @@ int bpf_prog_load_v0_6_0(enum bpf_prog_type prog_type, attr.fd_array =ptr_to_u64 (OPTS_GET (opts, fd_array,NULL )); +if (log_level) { + attr.log_buf =ptr_to_u64 (log_buf); + attr.log_size = log_size; + attr.log_level = log_level; + } + fd =sys_bpf_prog_load (&attr,sizeof (attr), attempts);if (fd >=0 )return fd;
@@ -366,16 +368,17 @@ int bpf_prog_load_v0_6_0(enum bpf_prog_type prog_type, goto done; } -if (log_level || !log_buf) - goto done; +if (log_level ==0 && !log_buf) { +/* log_level == 0 with non-NULL log_buf requires retrying on error +* with log_level == 1 and log_buf/log_buf_size set, to get details of +* failure +*/ + attr.log_buf =ptr_to_u64 (log_buf); + attr.log_size = log_size; + attr.log_level =1 ; -/* Try again with log */ - log_buf[0 ] =0 ; - attr.log_buf =ptr_to_u64 (log_buf); - attr.log_size = log_size; - attr.log_level =1 ; - - fd =sys_bpf_prog_load (&attr,sizeof (attr), attempts); + fd =sys_bpf_prog_load (&attr,sizeof (attr), attempts); + } done:/* free() doesn't affect errno, so we don't need to restore it */ free (finfo);
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 2/12 libbpf: add OPTS-based bpf_btf_load() API
In reply to Andrii NakryikoSimilar to previous bpf_prog_load() and bpf_map_create() APIs, add bpf_btf_load() API which is taking optional OPTS struct. Schedule bpf_load_btf() for deprecation in v0.8 ([0]).
This makes naming consistent with BPF_BTF_LOAD command, sets up an API for extensibility in the future, moves options parameters (log-related fields) into optional options, and also allows to pass log_level directly.
It also removes log buffer auto-allocation logic from low-level API (consistent with bpf_prog_load() behavior), but preserves a special treatment of log_level == 0 with non-NULL log_buf, which matches low-level bpf_prog_load() and high-level libbpf APIs for BTF and program loading behaviors.
[0] Closes: https://github.com/libbpf/libbpf/issues/419
--- tools/lib/bpf/bpf.c | 59 +++++++++++++++++++++++++++++------ tools/lib/bpf/bpf.h | 19 +++++++++-- tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_probes.c | 2 +- 4 files changed, 69 insertions(+), 12 deletions(-)
tools/lib/bpf/bpf.c +50 -9
@@ -1047,24 +1047,65 @@ int bpf_raw_tracepoint_open(const char *name, int prog_fd)return libbpf_err_errno (fd); } -int bpf_load_btf (const void *btf,__u32 btf_size,char *log_buf,__u32 log_buf_size, -bool do_log) +int bpf_btf_load (const void *btf_data,size_t btf_size,const struct bpf_btf_load_opts *opts) { -union bpf_attr attr = {}; + const size_t attr_sz = offsetofend(union bpf_attr, btf_log_level); +union bpf_attr attr; +char *log_buf; +size_t log_size; +__u32 log_level;int fd; - attr.btf =ptr_to_u64 (btf); +memset (&attr,0 , attr_sz); + +if (!OPTS_VALID (opts, bpf_btf_load_opts)) +return libbpf_err (-EINVAL ); + + log_buf =OPTS_GET (opts, log_buf,NULL ); + log_size =OPTS_GET (opts, log_size,0 ); + log_level =OPTS_GET (opts, log_level,0 ); + +if (log_size >UINT_MAX ) +return libbpf_err (-EINVAL ); +if (log_size && !log_buf) +return libbpf_err (-EINVAL ); + + attr.btf =ptr_to_u64 (btf_data); attr.btf_size = btf_size; +/* log_level == 0 and log_buf != NULL means "try loading without +* log_buf, but retry with log_buf and log_level=1 on error", which is +* consistent across low-level and high-level BTF and program loading +* APIs within libbpf and provides a sensible behavior in practice +*/ +if (log_level) { + attr.btf_log_buf =ptr_to_u64 (log_buf); + attr.btf_log_size = (__u32 )log_size; + attr.btf_log_level = log_level; + } -retry: -if (do_log && log_buf && log_buf_size) { - attr.btf_log_level =1 ; - attr.btf_log_size = log_buf_size; + fd =sys_bpf_fd (BPF_BTF_LOAD , &attr, attr_sz); +if (fd <0 && log_buf && log_level ==0 ) { attr.btf_log_buf =ptr_to_u64 (log_buf); + attr.btf_log_size = (__u32 )log_size; + attr.btf_log_level = log_level; + fd =sys_bpf_fd (BPF_BTF_LOAD , &attr, attr_sz); } +return libbpf_err_errno (fd); +} + +int bpf_load_btf (const void *btf,__u32 btf_size,char *log_buf,__u32 log_buf_size,bool do_log) +{ +LIBBPF_OPTS (bpf_btf_load_opts, opts); +int fd; - fd =sys_bpf_fd (BPF_BTF_LOAD , &attr,sizeof (attr)); +retry: +if (do_log && log_buf && log_buf_size) { + opts.log_buf = log_buf; + opts.log_size = log_buf_size; + opts.log_level =1 ; + } + fd =bpf_btf_load (btf, btf_size, &opts);if (fd <0 && !do_log && log_buf && log_buf_size) { do_log = true; goto retry;
tools/lib/bpf/bpf.h +17 -2
@@ -213,6 +213,23 @@ LIBBPF_API int bpf_verify_program(enum bpf_prog_type type, char *log_buf, size_t log_buf_sz, int log_level); +struct bpf_btf_load_opts { +size_t sz;/* size of this struct for forward/backward compatibility */ + +/* kernel log options */ +char *log_buf; +__u32 log_level; +__u32 log_size; +}; +#define bpf_btf_load_opts__last_field log_size + +LIBBPF_API int bpf_btf_load(const void *btf_data, size_t btf_size, +const struct bpf_btf_load_opts *opts); + +LIBBPF_DEPRECATED_SINCE(0, 8, "use bpf_btf_load() instead") +LIBBPF_API int bpf_load_btf(const void *btf, __u32 btf_size, char *log_buf, +__u32 log_buf_size,bool do_log); + LIBBPF_API int bpf_map_update_elem(int fd, const void *key, const void *value,__u64 flags);
@@ -340,8 +357,6 @@ LIBBPF_API int bpf_prog_query(int target_fd, enum bpf_attach_type type, __u32 query_flags, __u32 *attach_flags, __u32 *prog_ids, __u32 *prog_cnt); LIBBPF_API int bpf_raw_tracepoint_open(const char *name, int prog_fd); -LIBBPF_API int bpf_load_btf(const void *btf, __u32 btf_size, char *log_buf, -__u32 log_buf_size,bool do_log); LIBBPF_API int bpf_task_fd_query(int pid, int fd, __u32 flags, char *buf,__u32 *buf_len,__u32 *prog_id,__u32 *fd_type,__u64 *probe_offset,__u64 *probe_addr);
tools/lib/bpf/libbpf.map +1 -0
@@ -422,6 +422,7 @@ LIBBPF_0.6.0 { LIBBPF_0.7.0 { global: + bpf_btf_load; bpf_program__log_level; bpf_program__set_log_level; };
tools/lib/bpf/libbpf_probes.c +1 -1
@@ -164,7 +164,7 @@ int libbpf__load_raw_btf(const char *raw_types, size_t types_len,memcpy (raw_btf + hdr.hdr_len, raw_types, hdr.type_len);memcpy (raw_btf + hdr.hdr_len + hdr.type_len, str_sec, hdr.str_len); - btf_fd =bpf_load_btf (raw_btf, btf_len,NULL ,0 , false); + btf_fd =bpf_btf_load (raw_btf, btf_len,NULL );free (raw_btf);return btf_fd;
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 3/12 libbpf: allow passing preallocated log_buf when loading BTF into kernel
In reply to Andrii NakryikoAdd libbpf-internal btf_load_into_kernel() that allows to pass preallocated log_buf and custom log_level to be passed into kernel during BPF_BTF_LOAD call. When custom log_buf is provided, btf_load_into_kernel() won't attempt an retry with automatically allocated internal temporary buffer to capture BTF validation log.
It's important to note the relation between log_buf and log_level, which slightly deviates from stricter kernel logic. From kernel's POV, if log_buf is specified, log_level has to be > 0, and vice versa. While kernel has good reasons to request such "sanity, this, in practice, is a bit unconvenient and restrictive for libbpf's high-level bpf_object APIs.
So libbpf will allow to set non-NULL log_buf and log_level == 0. This is fine and means to attempt to load BTF without logging requested, but if it failes, retry the load with custom log_buf and log_level 1. Similar logic will be implemented for program loading. In practice this means that users can provide custom log buffer just in case error happens, but not really request slower verbose logging all the time. This is also consistent with libbpf behavior when custom log_buf is not set: libbpf first tries to load everything with log_level=0, and only if error happens allocates internal log buffer and retries with log_level=1.
Also, while at it, make BTF validation log more obvious and follow the log pattern libbpf is using for dumping BPF verifier log during BPF_PROG_LOAD. BTF loading resulting in an error will look like this:
libbpf: BTF loading error: -22
libbpf: -- BEGIN BTF LOAD LOG ---
magic: 0xeb9f
version: 1
flags: 0x0hdr_len: 24type_off: 0type_len: 1040str_off: 1040str_len: 2063598257btf_total_size: 1753
Total section length too long
-- END BTF LOAD LOG --
libbpf: Error loading .BTF into kernel: -22. BTF is optional, ignoring.
This makes it much easier to find relevant parts in libbpf log output.
--- tools/lib/bpf/btf.c | 78 +++++++++++++++++++++++---------- tools/lib/bpf/libbpf_internal.h | 1 + 2 files changed, 56 insertions(+), 23 deletions(-)
tools/lib/bpf/btf.c +55 -23
@@ -1124,54 +1124,86 @@ struct btf *btf__parse_split(const char *path, struct btf *base_btf)static void *btf_get_raw_data (const struct btf *btf,__u32 *size,bool swap_endian); -int btf__load_into_kernel (struct btf *btf) +int btf_load_into_kernel (struct btf *btf,char *log_buf,size_t log_sz,__u32 log_level) { -__u32 log_buf_size =0 , raw_size; -char *log_buf =NULL ; +LIBBPF_OPTS (bpf_btf_load_opts, opts); +__u32 buf_sz =0 , raw_size; +char *buf =NULL , *tmp;void *raw_data;int err =0 ;if (btf->fd >=0 )return libbpf_err (-EEXIST ); +if (log_sz && !log_buf) +return libbpf_err (-EINVAL ); -retry_load: -if (log_buf_size) { - log_buf =malloc (log_buf_size); -if (!log_buf) -return libbpf_err (-ENOMEM ); - - *log_buf =0 ; - } - +/* cache native raw data representation */ raw_data =btf_get_raw_data (btf, &raw_size, false);if (!raw_data) { err = -ENOMEM ; goto done; } -/* cache native raw data representation */ btf->raw_size = raw_size; btf->raw_data = raw_data; - btf->fd =bpf_load_btf (raw_data, raw_size, log_buf, log_buf_size, false); +retry_load: +/* if log_level is 0, we won't provide log_buf/log_size to the kernel, +* initially. Only if BTF loading fails, we bump log_level to 1 and +* retry, using either auto-allocated or custom log_buf. This way +* non-NULL custom log_buf provides a buffer just in case, but hopes +* for successful load and no need for log_buf. +*/ +if (log_level) { +/* if caller didn't provide custom log_buf, we'll keep +* allocating our own progressively bigger buffers for BTF +* verification log +*/ +if (!log_buf) { + buf_sz =max ((__u32 )BPF_LOG_BUF_SIZE , buf_sz *2 ); + tmp =realloc (buf, buf_sz); +if (!tmp) { + err = -ENOMEM ; + goto done; + } + buf = tmp; + buf[0 ] ='\0' ; + } + + opts.log_buf = log_buf ? log_buf : buf; + opts.log_size = log_buf ? log_sz : buf_sz; + opts.log_level = log_level; + } + + btf->fd =bpf_btf_load (raw_data, raw_size, &opts);if (btf->fd <0 ) { -if (!log_buf || errno ==ENOSPC ) { - log_buf_size =max ((__u32 )BPF_LOG_BUF_SIZE , - log_buf_size <<1 ); -free (log_buf); +/* time to turn on verbose mode and try again */ +if (log_level ==0 ) { + log_level =1 ; goto retry_load; } +/* only retry if caller didn't provide custom log_buf, but +* make sure we can never overflow buf_sz +*/ +if (!log_buf && errno ==ENOSPC && buf_sz <=UINT_MAX /2 ) + goto retry_load; err = -errno; -pr_warn ("Error loading BTF: %s(%d)\n" ,strerror (errno), errno); -if (*log_buf) -pr_warn ("%s\n" , log_buf); - goto done; +pr_warn ("BTF loading error: %d\n" , err); +/* don't print out contents of custom log_buf */ +if (!log_buf && buf[0 ]) +pr_warn ("-- BEGIN BTF LOAD LOG ---\n%s\n-- END BTF LOAD LOG --\n" , buf); } done: -free (log_buf); +free (buf);return libbpf_err (err); } + +int btf__load_into_kernel (struct btf *btf) +{ +return btf_load_into_kernel (btf,NULL ,0 ,0 ); +} +int btf__load (struct btf *) __attribute__((alias ("btf__load_into_kernel" )));int btf__fd (const struct btf *btf)
tools/lib/bpf/libbpf_internal.h +1 -0
@@ -277,6 +277,7 @@ int parse_cpu_mask_str(const char *s, bool **mask, int *mask_sz);int parse_cpu_mask_file (const char *fcpu,bool **mask,int *mask_sz);int libbpf__load_raw_btf (const char *raw_types,size_t types_len,const char *str_sec,size_t str_len); +int btf_load_into_kernel (struct btf *btf,char *log_buf,size_t log_sz,__u32 log_level);struct btf *btf_get_from_fd (int btf_fd,struct btf *base_btf); void btf_get_kernel_prefix_kind(enum bpf_attach_type attach_type,
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 4/12 libbpf: allow passing user log setting through bpf_object_open_opts
In reply to Andrii NakryikoAllow users to provide their own custom log_buf, log_size, and log_level at bpf_object level through bpf_object_open_opts. This log_buf will be used during BTF loading. Subsequent patch will use same log_buf during BPF program loading, unless overriden at per-bpf_program level.
When such custom log_buf is provided, libbpf won't be attempting retrying loading of BTF to try to provide its own log buffer to capture kernel's error log output. User is responsible to provide big enough buffer, otherwise they run a risk of getting -ENOSPC error from the bpf() syscall.
See also comments in bpf_object_open_opts regarding log_level and log_buf interactions.
--- tools/lib/bpf/bpf.h | 3 ++- tools/lib/bpf/libbpf.c | 24 +++++++++++++++++++++++- tools/lib/bpf/libbpf.h | 41 ++++++++++++++++++++++++++++++++++++++++- 3 files changed, 65 insertions(+), 3 deletions(-)
tools/lib/bpf/bpf.h +2 -1
@@ -195,8 +195,9 @@ struct bpf_load_program_attr {/* Flags to direct loading requirements */ #define MAPS_RELAX_COMPAT 0x01 -/* Recommend log buffer size */ +/* Recommended log buffer size */ #define BPF_LOG_BUF_SIZE (UINT32_MAX >> 8)/* verifier maximum in kernels <= 5.1 */ + LIBBPF_DEPRECATED_SINCE(0, 7, "use bpf_prog_load() instead") LIBBPF_API int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,char *log_buf,size_t log_buf_sz);
tools/lib/bpf/libbpf.c +23 -1
@@ -573,6 +573,11 @@ struct bpf_object {size_t btf_module_cnt;size_t btf_module_cap; +/* optional log settings passed to BPF_BTF_LOAD and BPF_PROG_LOAD commands */ +char *log_buf; +size_t log_size; +__u32 log_level; +void *priv;bpf_object_clear_priv_t clear_priv;
@@ -3017,7 +3022,9 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj)*/ btf__set_fd (kern_btf,0 ); }else { - err =btf__load_into_kernel (kern_btf); +/* currently BPF_BTF_LOAD only supports log_level 1 */ + err =btf_load_into_kernel (kern_btf, obj->log_buf, obj->log_size, + obj->log_level ?1 :0 ); }if (sanitize) {if (!err) {
@@ -6932,6 +6939,9 @@ __bpf_object__open(const char *path, const void *obj_buf, size_t obj_buf_sz,struct bpf_object *obj;char tmp_name[64 ];int err; +char *log_buf; +size_t log_size; +__u32 log_level;if (elf_version (EV_CURRENT ) ==EV_NONE ) { pr_warn("failed to init libelf for %s\n",
@@ -6954,10 +6964,22 @@ __bpf_object__open(const char *path, const void *obj_buf, size_t obj_buf_sz,pr_debug ("loading object '%s' from buffer\n" , obj_name); } + log_buf =OPTS_GET (opts, kernel_log_buf,NULL ); + log_size =OPTS_GET (opts, kernel_log_size,0 ); + log_level =OPTS_GET (opts, kernel_log_level,0 ); +if (log_size >UINT_MAX ) +return ERR_PTR (-EINVAL ); +if (log_size && !log_buf) +return ERR_PTR (-EINVAL ); + obj =bpf_object__new (path, obj_buf, obj_buf_sz, obj_name);if (IS_ERR (obj))return obj; + obj->log_buf = log_buf; + obj->log_size = log_size; + obj->log_level = log_level; + btf_tmp_path =OPTS_GET (opts, btf_custom_path,NULL );if (btf_tmp_path) {if (strlen (btf_tmp_path) >=PATH_MAX ) {
tools/lib/bpf/libbpf.h +40 -1
48 lines not on this page · raw message
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 5/12 libbpf: improve logging around BPF program loading
In reply to Andrii NakryikoAdd missing "prog '%s': " prefixes in few places and use consistently markers for beginning and end of program load logs. Here's an example of log output:
libbpf: prog 'handler': BPF program load failed: Permission denied
libbpf: -- BEGIN PROG LOAD LOG ---
arg#0 reference type('UNKNOWN ') size cannot be determined: -22 ; out1 = in1;
0: (18) r1 = 0xffffc9000cdcc000
2: (61) r1 = *(u32 *)(r1 +0)
...
81: (63) *(u32 *)(r4 +0) = r5
R1_w=map_value(id=0,off=16,ks=4,vs=20,imm=0) R4=map_value(id=0,off=400,ks=4,vs=16,imm=0)
invalid access to map value, value_size=16 off=400 size=4
R4 min value is outside of the allowed memory range
processed 63 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
-- END PROG LOAD LOG --
libbpf: failed to load program 'handler'
libbpf: failed to load object 'test_skeleton'
The entire verifier log, including BEGIN and END markers are now always youtput during a single print callback call. This should make it much easier to post-process or parse it, if necessary. It's not an explicit API guarantee, but it can be reasonably expected to stay like that.
Also __bpf_object__open is renamed to bpf_object_open() as it's always an adventure to find the exact function that implements bpf_object's open phase, so drop the double underscored and use internal libbpf naming convention.
--- tools/lib/bpf/libbpf.c | 38 +++++++++---------- .../selftests/bpf/prog_tests/bpf_tcp_ca.c | 6 ++- 2 files changed, 23 insertions(+), 21 deletions(-)
tools/lib/bpf/libbpf.c +19 -19
@@ -6662,8 +6662,10 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_prog ret =bpf_prog_load (prog->type, prog_name, license, insns, insns_cnt, &load_attr);if (ret >=0 ) { -if (log_buf && load_attr.log_level) -pr_debug ("verifier log:\n%s" , log_buf); +if (log_buf && load_attr.log_level) { +pr_debug ("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n" , + prog->name, log_buf); + }if (obj->has_rodata &&kernel_supports (obj,FEAT_PROG_BIND_MAP )) {struct bpf_map *map;
@@ -6676,8 +6678,8 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_progif (bpf_prog_bind_map (ret,bpf_map__fd (map),NULL )) { cp =libbpf_strerror_r (errno, errmsg,sizeof (errmsg)); -pr_warn ("prog '%s': failed to bind .rodata map: %s\n" , - prog->name, cp); +pr_warn ("prog '%s': failed to bind map '%s': %s\n" , + prog->name, map->real_name, cp);/* Don't fail hard if can't bind rodata. */ } }
@@ -6691,23 +6693,22 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_progif (!log_buf || errno ==ENOSPC ) { log_buf_size =max ((size_t )BPF_LOG_BUF_SIZE , log_buf_size <<1 ); -free (log_buf); goto retry_load; } ret = errno ? -errno : -LIBBPF_ERRNO__LOAD ; cp =libbpf_strerror_r (errno, errmsg,sizeof (errmsg)); -pr_warn ("load bpf program failed: %s\n" , cp); +pr_warn ("prog '%s': BPF program load failed: %s\n" , prog->name, cp);pr_perm_msg (ret);if (log_buf && log_buf[0 ] !='\0' ) { ret = -LIBBPF_ERRNO__VERIFY ; -pr_warn ("-- BEGIN DUMP LOG ---\n" ); -pr_warn ("\n%s\n" , log_buf); -pr_warn ("-- END LOG --\n" ); - }else if (insns_cnt >=BPF_MAXINSNS ) { -pr_warn ("Program too large (%d insns), at most %d insns\n" , - insns_cnt,BPF_MAXINSNS ); +pr_warn ("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n" , + prog->name, log_buf); + } +if (insns_cnt >=BPF_MAXINSNS ) { +pr_warn ("prog '%s': program too large (%d insns), at most %d insns\n" , + prog->name, insns_cnt,BPF_MAXINSNS ); ret = -LIBBPF_ERRNO__PROG2BIG ; }else if (prog->type !=BPF_PROG_TYPE_KPROBE ) {/* Wrong program type? */
@@ -6931,9 +6932,8 @@ static int bpf_object_init_progs(struct bpf_object *obj, const struct bpf_objectreturn 0 ; } -static struct bpf_object * -__bpf_object__open (const char *path,const void *obj_buf,size_t obj_buf_sz, -const struct bpf_object_open_opts *opts) +static struct bpf_object *bpf_object_open (const char *path,const void *obj_buf,size_t obj_buf_sz, +const struct bpf_object_open_opts *opts) {const char *obj_name, *kconfig, *btf_tmp_path;struct bpf_object *obj;
@@ -7033,7 +7033,7 @@ __bpf_object__open_xattr(struct bpf_object_open_attr *attr, int flags)return NULL ;pr_debug ("loading %s\n" , attr->file); -return __bpf_object__open (attr->file,NULL ,0 , &opts); +return bpf_object_open (attr->file,NULL ,0 , &opts); }struct bpf_object *bpf_object__open_xattr (struct bpf_object_open_attr *attr)
@@ -7059,7 +7059,7 @@ bpf_object__open_file(const char *path, const struct bpf_object_open_opts *opts)pr_debug ("loading %s\n" , path); -return libbpf_ptr (__bpf_object__open (path,NULL ,0 , opts)); +return libbpf_ptr (bpf_object_open (path,NULL ,0 , opts)); } struct bpf_object *
@@ -7069,7 +7069,7 @@ bpf_object__open_mem(const void *obj_buf, size_t obj_buf_sz,if (!obj_buf || obj_buf_sz ==0 )return libbpf_err_ptr (-EINVAL ); -return libbpf_ptr (__bpf_object__open (NULL , obj_buf, obj_buf_sz, opts)); +return libbpf_ptr (bpf_object_open (NULL , obj_buf, obj_buf_sz, opts)); } struct bpf_object *
@@ -7086,7 +7086,7 @@ bpf_object__open_buffer(const void *obj_buf, size_t obj_buf_sz,if (!obj_buf || obj_buf_sz ==0 )return errno =EINVAL ,NULL ; -return libbpf_ptr (__bpf_object__open (NULL , obj_buf, obj_buf_sz, &opts)); +return libbpf_ptr (bpf_object_open (NULL , obj_buf, obj_buf_sz, &opts)); }static int bpf_object_unload (struct bpf_object *obj)
tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c +4 -2
@@ -217,14 +217,16 @@ static bool found;static int libbpf_debug_print (enum libbpf_print_level level,const char *format,va_list args) { -char *log_buf; +const char *log_buf;if (level !=LIBBPF_WARN || -strcmp (format,"libbpf: \n%s\n" )) { + !strstr (format,"-- BEGIN PROG LOAD LOG --" )) {vprintf (format, args);return 0 ; } +/* skip prog_name */ + va_arg(args, char *); log_buf = va_arg(args, char *);if (!log_buf) goto out;
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 6/12 libbpf: preserve kernel error code and remove kprobe prog type guessing
In reply to Andrii NakryikoInstead of rewriting error code returned by the kernel of prog load with libbpf-sepcific variants pass through the original error.
There is now also no need to have a backup generic -LIBBPF_ERRNO__LOAD fallback error as bpf_prog_load() guarantees that errno will be properly set no matter what.
Also drop a completely outdated and pretty useless BPF_PROG_TYPE_KPROBE guess logic. It's not necessary and neither it's helpful in modern BPF applications.
--- tools/lib/bpf/libbpf.c | 19 ++----------------- 1 file changed, 2 insertions(+), 17 deletions(-)
tools/lib/bpf/libbpf.c +2 -17
@@ -6696,34 +6696,19 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_progfree (log_buf); goto retry_load; } - ret = errno ? -errno : -LIBBPF_ERRNO__LOAD ; + + ret = -errno; cp =libbpf_strerror_r (errno, errmsg,sizeof (errmsg));pr_warn ("prog '%s': BPF program load failed: %s\n" , prog->name, cp);pr_perm_msg (ret);if (log_buf && log_buf[0 ] !='\0' ) { - ret = -LIBBPF_ERRNO__VERIFY ;pr_warn ("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n" , prog->name, log_buf); }if (insns_cnt >=BPF_MAXINSNS ) {pr_warn ("prog '%s': program too large (%d insns), at most %d insns\n" , prog->name, insns_cnt,BPF_MAXINSNS ); - ret = -LIBBPF_ERRNO__PROG2BIG ; - }else if (prog->type !=BPF_PROG_TYPE_KPROBE ) { -/* Wrong program type? */ -int fd; - - load_attr.expected_attach_type =0 ; - load_attr.log_buf =NULL ; - load_attr.log_size =0 ; - fd =bpf_prog_load (BPF_PROG_TYPE_KPROBE , prog_name, license, - insns, insns_cnt, &load_attr); -if (fd >=0 ) { -close (fd); - ret = -LIBBPF_ERRNO__PROGTYPE ; - goto out; - } } out:
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 7/12 libbpf: add per-program log buffer setter and getter
In reply to Andrii NakryikoAllow to set user-provided log buffer on a per-program basis ([0]). This gives great deal of flexibility in terms of which programs are loaded with logging enabled and where corresponding logs go.
Log buffer set with bpf_program__set_log_buf() overrides kernel_log_buf and kernel_log_size settings set at bpf_object open time through bpf_object_open_opts, if any.
Adjust bpf_object_load_prog_instance() logic to not perform own log buf allocation and load retry if custom log buffer is provided by the user.
[0] Closes: https://github.com/libbpf/libbpf/issues/418
--- tools/lib/bpf/libbpf.c | 92 ++++++++++++++++++++++++++++++++-------- tools/lib/bpf/libbpf.h | 7 +++ tools/lib/bpf/libbpf.map | 2 + 3 files changed, 84 insertions(+), 17 deletions(-)
tools/lib/bpf/libbpf.c +75 -17
@@ -331,7 +331,11 @@ struct bpf_program {struct reloc_desc *reloc_desc;int nr_reloc; -int log_level; + +/* BPF verifier log settings */ +char *log_buf; +size_t log_size; +__u32 log_level;struct {int nr;
@@ -713,6 +717,9 @@ bpf_object__init_prog(struct bpf_object *obj, struct bpf_program *prog, prog->instances.fds =NULL ; prog->instances.nr =-1 ; +/* inherit object's log_level */ + prog->log_level = obj->log_level; + prog->sec_name =strdup (sec_name);if (!prog->sec_name) goto errout;
@@ -6591,8 +6598,10 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_progconst char *prog_name =NULL ;char *cp, errmsg[STRERR_BUFSIZE ];size_t log_buf_size =0 ; -char *log_buf =NULL ; +char *log_buf =NULL , *tmp;int btf_fd, ret, err; +bool own_log_buf = true; +__u32 log_level = prog->log_level;if (prog->type ==BPF_PROG_TYPE_UNSPEC ) {/*
@@ -6627,7 +6636,7 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_prog load_attr.line_info_rec_size = prog->line_info_rec_size; load_attr.line_info_cnt = prog->line_info_cnt; } - load_attr.log_level = prog->log_level; + load_attr.log_level = log_level; load_attr.prog_flags = prog->prog_flags; load_attr.fd_array = obj->fd_array;
@@ -6648,21 +6657,42 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_prog *prog_fd =-1 ;return 0 ; } -retry_load: -if (log_buf_size) { - log_buf =malloc (log_buf_size); -if (!log_buf) -return -ENOMEM ; - *log_buf =0 ; +retry_load: +/* if log_level is zero, we don't request logs initiallly even if +* custom log_buf is specified; if the program load fails, then we'll +* bump log_level to 1 and use either custom log_buf or we'll allocate +* our own and retry the load to get details on what failed +*/ +if (log_level) { +if (prog->log_buf) { + log_buf = prog->log_buf; + log_buf_size = prog->log_size; + own_log_buf = false; + }else if (obj->log_buf) { + log_buf = obj->log_buf; + log_buf_size = obj->log_size; + own_log_buf = false; + }else { + log_buf_size =max ((size_t )BPF_LOG_BUF_SIZE , log_buf_size *2 ); + tmp =realloc (log_buf, log_buf_size); +if (!tmp) { + ret = -ENOMEM ; + goto out; + } + log_buf = tmp; + log_buf[0 ] ='\0' ; + own_log_buf = true; + } } load_attr.log_buf = log_buf; load_attr.log_size = log_buf_size; - ret =bpf_prog_load (prog->type, prog_name, license, insns, insns_cnt, &load_attr); + load_attr.log_level = log_level; + ret =bpf_prog_load (prog->type, prog_name, license, insns, insns_cnt, &load_attr);if (ret >=0 ) { -if (log_buf && load_attr.log_level) { +if (log_level && own_log_buf) {pr_debug ("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n" , prog->name, log_buf); }
@@ -6690,19 +6720,26 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_prog goto out; } -if (!log_buf || errno ==ENOSPC ) { - log_buf_size =max ((size_t )BPF_LOG_BUF_SIZE , - log_buf_size <<1 ); -free (log_buf); +if (log_level ==0 ) { + log_level =1 ; goto retry_load; } +/* On ENOSPC, increase log buffer size and retry, unless custom +* log_buf is specified. +* Be careful to not overflow u32, though. Kernel's log buf size limit +* isn't part of UAPI so it can always be bumped to full 4GB. So don't +* multiply by 2 unless we are sure we'll fit within 32 bits. +* Currently, we'll get -EINVAL when we reach (UINT_MAX >> 2). +*/ +if (own_log_buf && errno ==ENOSPC && log_buf_size <=UINT_MAX /2 ) + goto retry_load; ret = -errno; cp =libbpf_strerror_r (errno, errmsg,sizeof (errmsg));pr_warn ("prog '%s': BPF program load failed: %s\n" , prog->name, cp);pr_perm_msg (ret); -if (log_buf && log_buf[0 ] !='\0' ) { +if (own_log_buf && log_buf && log_buf[0 ] !='\0' ) {pr_warn ("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n" , prog->name, log_buf); }
@@ -6712,7 +6749,8 @@ static int bpf_object_load_prog_instance(struct bpf_object *obj, struct bpf_prog } out: -free (log_buf); +if (own_log_buf) +free (log_buf);return ret; }
@@ -8498,6 +8536,26 @@ int bpf_program__set_log_level(struct bpf_program *prog, __u32 log_level)return 0 ; } +const char *bpf_program__log_buf (const struct bpf_program *prog,size_t *log_size) +{ + *log_size = prog->log_size; +return prog->log_buf; +} + +int bpf_program__set_log_buf (struct bpf_program *prog,char *log_buf,size_t log_size) +{ +if (log_size && !log_buf) +return -EINVAL ; +if (prog->log_size >UINT_MAX ) +return -EINVAL ; +if (prog->obj->loaded) +return -EBUSY ; + + prog->log_buf = log_buf; + prog->log_size = log_size; +return 0 ; +} +#define SEC_DEF (sec_pfx, ptype, atype, flags, ...) { \ .sec = sec_pfx, \ .prog_type = BPF_PROG_TYPE_##ptype, \
tools/lib/bpf/libbpf.h +7 -0
@@ -591,8 +591,15 @@ bpf_program__set_expected_attach_type(struct bpf_program *prog, LIBBPF_API __u32 bpf_program__flags(const struct bpf_program *prog); LIBBPF_API int bpf_program__set_flags(struct bpf_program *prog, __u32 flags); + +/* Per-program log level and log buffer getters/setters. +* See bpf_object_open_opts comments regarding log_level and log_buf +* interactions. +*/ LIBBPF_API __u32 bpf_program__log_level(const struct bpf_program *prog); LIBBPF_API int bpf_program__set_log_level(struct bpf_program *prog, __u32 log_level); +LIBBPF_API const char *bpf_program__log_buf(const struct bpf_program *prog, size_t *log_size); +LIBBPF_API int bpf_program__set_log_buf(struct bpf_program *prog, char *log_buf, size_t log_size); LIBBPF_API int bpf_program__set_attach_target(struct bpf_program *prog, int attach_prog_fd,
tools/lib/bpf/libbpf.map +2 -0
@@ -423,6 +423,8 @@ LIBBPF_0.6.0 { LIBBPF_0.7.0 { global: bpf_btf_load; + bpf_program__log_buf; bpf_program__log_level; + bpf_program__set_log_buf; bpf_program__set_log_level; };
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 8/12 libbpf: deprecate bpf_object__load_xattr()
In reply to Andrii NakryikoDeprecate non-extensible bpf_object__load_xattr() in v0.8 ([0]).
With log_level control through bpf_object_open_opts or bpf_program__set_log_level(), we are finally at the point where bpf_object__load_xattr() doesn't provide any functionality that can't be accessed through other (better) ways. The other feature, target_btf_path, is also controllable through bpf_object_open_opts.
[0] Closes: https://github.com/libbpf/libbpf/issues/289
--- tools/lib/bpf/libbpf.c | 23 ++++++++++------------- tools/lib/bpf/libbpf.h | 1 + 2 files changed, 11 insertions(+), 13 deletions(-)
tools/lib/bpf/libbpf.c +10 -13
@@ -7462,14 +7462,10 @@ static int bpf_object__resolve_externs(struct bpf_object *obj,return 0 ; } -int bpf_object__load_xattr (struct bpf_object_load_attr *attr) +static int bpf_object_load (struct bpf_object *obj,int extra_log_level,const char *target_btf_path) { -struct bpf_object *obj;int err, i; -if (!attr) -return libbpf_err (-EINVAL ); - obj = attr->obj;if (!obj)return libbpf_err (-EINVAL );
@@ -7479,7 +7475,7 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr) }if (obj->gen_loader) -bpf_gen__init (obj->gen_loader, attr->log_level); +bpf_gen__init (obj->gen_loader, extra_log_level); err =bpf_object__probe_loading (obj); err = err ? :bpf_object__load_vmlinux_btf (obj, false);
@@ -7488,8 +7484,8 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr) err = err ? :bpf_object__sanitize_maps (obj); err = err ? :bpf_object__init_kern_struct_ops_maps (obj); err = err ? :bpf_object__create_maps (obj); - err = err ? :bpf_object__relocate (obj, obj->btf_custom_path ? : attr->target_btf_path); - err = err ? :bpf_object__load_progs (obj, attr->log_level); + err = err ? :bpf_object__relocate (obj, obj->btf_custom_path ? : target_btf_path); + err = err ? :bpf_object__load_progs (obj, extra_log_level); err = err ? :bpf_object_init_prog_arrays (obj);if (obj->gen_loader) {
@@ -7534,13 +7530,14 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr)return libbpf_err (err); } -int bpf_object__load (struct bpf_object *obj) +int bpf_object__load_xattr (struct bpf_object_load_attr *attr) { -struct bpf_object_load_attr attr = { - .obj = obj, - }; +return bpf_object_load (attr->obj, attr->log_level, attr->target_btf_path); +} -return bpf_object__load_xattr (&attr); +int bpf_object__load (struct bpf_object *obj) +{ +return bpf_object_load (obj,0 ,NULL ); }static int make_parent_dir (const char *path)
tools/lib/bpf/libbpf.h +1 -0
@@ -214,6 +214,7 @@ struct bpf_object_load_attr {/* Load/unload object into/from kernel */ LIBBPF_API int bpf_object__load(struct bpf_object *obj); +LIBBPF_DEPRECATED_SINCE(0, 8, "use bpf_object__load() instead") LIBBPF_API int bpf_object__load_xattr(struct bpf_object_load_attr *attr); LIBBPF_DEPRECATED_SINCE(0, 6, "bpf_object__unload() is deprecated, use bpf_object__close() instead") LIBBPF_API int bpf_object__unload(struct bpf_object *obj);
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 9/12 selftests/bpf: replace all uses of bpf_load_btf() with bpf_btf_load()
In reply to Andrii NakryikoSwitch all selftests uses of to-be-deprecated bpf_load_btf() with equivalent bpf_btf_load() calls.
--- .../selftests/bpf/map_tests/sk_storage_map.c | 2 +- tools/testing/selftests/bpf/prog_tests/btf.c | 50 +++++++++++-------- tools/testing/selftests/bpf/test_verifier.c | 2 +- 3 files changed, 32 insertions(+), 22 deletions(-)
tools/testing/selftests/bpf/map_tests/sk_storage_map.c +1 -1
@@ -136,7 +136,7 @@ static int load_btf(void)memcpy (raw_btf +sizeof (btf_hdr) +sizeof (btf_raw_types), btf_str_sec,sizeof (btf_str_sec)); -return bpf_load_btf (raw_btf,sizeof (raw_btf),0 ,0 ,0 ); +return bpf_btf_load (raw_btf,sizeof (raw_btf),NULL ); }static int create_sk_storage_map (void )
tools/testing/selftests/bpf/prog_tests/btf.c +30 -20
100 lines not on this page · raw message
tools/testing/selftests/bpf/test_verifier.c +1 -1
@@ -641,7 +641,7 @@ static int load_btf(void)memcpy (ptr, btf_str_sec, hdr.str_len); ptr += hdr.str_len; - btf_fd =bpf_load_btf (raw_btf, ptr - raw_btf,0 ,0 ,0 ); + btf_fd =bpf_btf_load (raw_btf, ptr - raw_btf,NULL );free (raw_btf);if (btf_fd <0 )return -1 ;
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 10/12 selftests/bpf: add test for libbpf's custom log_buf behavior
In reply to Andrii NakryikoAdd a selftest that validates that per-program and per-object log_buf overrides work as expected.
--- .../selftests/bpf/prog_tests/log_buf.c | 137 ++++++++++++++++++ .../selftests/bpf/progs/test_log_buf.c | 24 +++ 2 files changed, 161 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/log_buf.c create mode 100644 tools/testing/selftests/bpf/progs/test_log_buf.c
tools/testing/selftests/bpf/prog_tests/log_buf.c new file +137 -0
@@ -0,0 +1,137 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ + +#include <test_progs.h> + +#include "test_log_buf.skel.h" + +static size_t libbpf_log_pos; +static char libbpf_log_buf[1024 *1024 ]; +static bool libbpf_log_error; + +static int libbpf_print_cb (enum libbpf_print_level level,const char *fmt,va_list args) +{ +int emitted_cnt; +size_t left_cnt; + + left_cnt =sizeof (libbpf_log_buf) - libbpf_log_pos; + emitted_cnt =vsnprintf (libbpf_log_buf + libbpf_log_pos, left_cnt, fmt, args); + +if (emitted_cnt <0 || emitted_cnt +1 > left_cnt) { + libbpf_log_error = true; +return 0 ; + } + + libbpf_log_pos += emitted_cnt; +return 0 ; +} + +void test_log_buf (void ) +{ +libbpf_print_fn_t old_print_cb =libbpf_set_print (libbpf_print_cb); +LIBBPF_OPTS (bpf_object_open_opts, opts); +const size_t log_buf_sz =1024 *1024 ; +struct test_log_buf * skel; +char *obj_log_buf, *good_log_buf, *bad_log_buf; +int err; + + obj_log_buf =malloc (3 *log_buf_sz); +if (!ASSERT_OK_PTR (obj_log_buf,"obj_log_buf" )) +return ; + + good_log_buf = obj_log_buf + log_buf_sz; + bad_log_buf = obj_log_buf +2 * log_buf_sz; + obj_log_buf[0 ] = good_log_buf[0 ] = bad_log_buf[0 ] ='\0' ; + + opts.kernel_log_buf = obj_log_buf; + opts.kernel_log_size = log_buf_sz; + opts.kernel_log_level =4 ;/* for BTF this will turn into 1 */ + +/* In the first round every prog has its own log_buf, so libbpf logs +* don't have program failure logs +*/ + skel =test_log_buf__open_opts (&opts); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +/* set very verbose level for good_prog so we always get detailed logs */ +bpf_program__set_log_buf (skel->progs.good_prog, good_log_buf, log_buf_sz); +bpf_program__set_log_level (skel->progs.good_prog,2 ); + +bpf_program__set_log_buf (skel->progs.bad_prog, bad_log_buf, log_buf_sz); +/* log_level 0 with custom log_buf means that verbose logs are not +* requested if program load is successful, but libbpf should retry +* with log_level 1 on error and put program's verbose load log into +* custom log_buf +*/ +bpf_program__set_log_level (skel->progs.bad_prog,0 ); + + err =test_log_buf__load (skel); +if (!ASSERT_ERR (err,"unexpected_load_success" )) + goto cleanup; + +ASSERT_FALSE (libbpf_log_error,"libbpf_log_error" ); + +/* there should be no prog loading log because we specified per-prog log buf */ +ASSERT_NULL (strstr (libbpf_log_buf,"-- BEGIN PROG LOAD LOG --" ),"unexp_libbpf_log" ); +ASSERT_OK_PTR (strstr (libbpf_log_buf,"prog 'bad_prog': BPF program load failed" ), +"libbpf_log_not_empty" ); +ASSERT_OK_PTR (strstr (obj_log_buf,"DATASEC license" ),"obj_log_not_empty" ); +ASSERT_OK_PTR (strstr (good_log_buf,"0: R1=ctx(id=0,off=0,imm=0) R10=fp0" ), +"good_log_verbose" ); +ASSERT_OK_PTR (strstr (bad_log_buf,"invalid access to map value, value_size=16 off=16000 size=4" ), +"bad_log_not_empty" ); + +if (env.verbosity >VERBOSE_NONE ) { +printf ("LIBBPF LOG: \n=================\n%s=================\n" , libbpf_log_buf); +printf ("OBJ LOG: \n=================\n%s=================\n" , obj_log_buf); +printf ("GOOD_PROG LOG:\n=================\n%s=================\n" , good_log_buf); +printf ("BAD_PROG LOG:\n=================\n%s=================\n" , bad_log_buf); + } + +/* reset everything */ +test_log_buf__destroy (skel); + obj_log_buf[0 ] = good_log_buf[0 ] = bad_log_buf[0 ] ='\0' ; + libbpf_log_buf[0 ] ='\0' ; + libbpf_log_pos =0 ; + libbpf_log_error = false; + +/* In the second round we let bad_prog's failure be logged through print callback */ + opts.kernel_log_buf =NULL ;/* let everything through into print callback */ + opts.kernel_log_size =0 ; + opts.kernel_log_level =1 ; + + skel =test_log_buf__open_opts (&opts); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +/* set normal verbose level for good_prog to check log_level is taken into account */ +bpf_program__set_log_buf (skel->progs.good_prog, good_log_buf, log_buf_sz); +bpf_program__set_log_level (skel->progs.good_prog,1 ); + + err =test_log_buf__load (skel); +if (!ASSERT_ERR (err,"unexpected_load_success" )) + goto cleanup; + +ASSERT_FALSE (libbpf_log_error,"libbpf_log_error" ); + +/* this time prog loading error should be logged through print callback */ +ASSERT_OK_PTR (strstr (libbpf_log_buf,"libbpf: prog 'bad_prog': -- BEGIN PROG LOAD LOG --" ), +"libbpf_log_correct" ); +ASSERT_STREQ (obj_log_buf,"" ,"obj_log__empty" ); +ASSERT_STREQ (good_log_buf,"processed 4 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0\n" , +"good_log_ok" ); +ASSERT_STREQ (bad_log_buf,"" ,"bad_log_empty" ); + +if (env.verbosity >VERBOSE_NONE ) { +printf ("LIBBPF LOG: \n=================\n%s=================\n" , libbpf_log_buf); +printf ("OBJ LOG: \n=================\n%s=================\n" , obj_log_buf); +printf ("GOOD_PROG LOG:\n=================\n%s=================\n" , good_log_buf); +printf ("BAD_PROG LOG:\n=================\n%s=================\n" , bad_log_buf); + } + +cleanup: +free (obj_log_buf); +test_log_buf__destroy (skel); +libbpf_set_print (old_print_cb); +}
tools/testing/selftests/bpf/progs/test_log_buf.c new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ + +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> + +int a[4 ]; +const volatile int off =4000 ; + +SEC("raw_tp/sys_enter") +int good_prog (const void *ctx) +{ + a[0 ] = (int )(long )ctx; +return a[1 ]; +} + +SEC("raw_tp/sys_enter") +int bad_prog (const void *ctx) +{ +/* out of bounds access */ +return a[off]; +} + +char _license[] SEC("license") = "GPL";
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 11/12 selftests/bpf: remove the only use of deprecated bpf_object__load_xattr()
In reply to Andrii NakryikoSwitch from bpf_object__load_xattr() to bpf_object__load() and kernel_log_level in bpf_object_open_opts.
--- tools/testing/selftests/bpf/testing_helpers.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/testing/selftests/bpf/testing_helpers.c +5 -5
@@ -88,13 +88,15 @@ int extra_prog_load_log_flags = 0;int bpf_prog_test_load (const char *file,enum bpf_prog_type type,struct bpf_object **pobj,int *prog_fd) { -struct bpf_object_load_attr attr = {}; +LIBBPF_OPTS (bpf_object_open_opts, opts, + .kernel_log_level = extra_prog_load_log_flags, + );struct bpf_object *obj;struct bpf_program *prog;__u32 flags;int err; - obj =bpf_object__open (file); + obj =bpf_object__open_file (file, &opts);if (!obj)return -errno;
@@ -110,9 +112,7 @@ int bpf_prog_test_load(const char *file, enum bpf_prog_type type, flags =bpf_program__flags (prog) |BPF_F_TEST_RND_HI32 ;bpf_program__set_flags (prog, flags); - attr.obj = obj; - attr.log_level = extra_prog_load_log_flags; - err =bpf_object__load_xattr (&attr); + err =bpf_object__load (obj);if (err) goto err_out;
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 12/12 bpftool: switch bpf_object__load_xattr() to bpf_object__load()
In reply to Andrii NakryikoSwitch all the uses of to-be-deprecated bpf_object__load_xattr() into a simple bpf_object__load() calls with optional log_level passed through open_opts.kernel_log_level, if -d option is specified.
--- tools/bpf/bpftool/gen.c | 11 ++++------- tools/bpf/bpftool/prog.c | 24 ++++++++++-------------- tools/bpf/bpftool/struct_ops.c | 15 +++++++-------- 3 files changed, 21 insertions(+), 29 deletions(-)
tools/bpf/bpftool/gen.c +4 -7
@@ -486,7 +486,6 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name)static int gen_trace (struct bpf_object *obj,const char *obj_name,const char *header_guard) { -struct bpf_object_load_attr load_attr = {};DECLARE_LIBBPF_OPTS (gen_loader_opts, opts);struct bpf_map *map;char ident[256 ];
@@ -496,12 +495,7 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *hif (err)return err; - load_attr.obj = obj; -if (verifier_logs) -/* log_level1 + log_level2 + stats, but not stable UAPI */ - load_attr.log_level =1 +2 +4 ; - - err =bpf_object__load_xattr (&load_attr); + err =bpf_object__load (obj);if (err) {p_err ("failed to load object file" ); goto out;
@@ -719,6 +713,9 @@ static int do_skeleton(int argc, char **argv)if (obj_name[0 ] =='\0' )get_obj_name (obj_name, file); opts.object_name = obj_name; +if (verifier_logs) +/* log_level1 + log_level2 + stats, but not stable UAPI */ + opts.kernel_log_level =1 +2 +4 ; obj =bpf_object__open_mem (obj_data, file_sz, &opts); err =libbpf_get_error (obj);if (err) {
tools/bpf/bpftool/prog.c +10 -14
@@ -1464,7 +1464,6 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)DECLARE_LIBBPF_OPTS (bpf_object_open_opts, open_opts, .relaxed_maps = relaxed_maps, ); -struct bpf_object_load_attr load_attr = {0 };enum bpf_attach_type expected_attach_type;struct map_replace *map_replace =NULL ;struct bpf_program *prog =NULL , *pos;
@@ -1598,6 +1597,10 @@ static int load_with_options(int argc, char **argv, bool first_prog_only)set_max_rlimit (); +if (verifier_logs) +/* log_level1 + log_level2 + stats, but not stable UAPI */ + open_opts.kernel_log_level =1 +2 +4 ; + obj =bpf_object__open_file (file, &open_opts);if (libbpf_get_error (obj)) {p_err ("failed to open object file" );
@@ -1677,12 +1680,7 @@ static int load_with_options(int argc, char **argv, bool first_prog_only) goto err_close_obj; } - load_attr.obj = obj; -if (verifier_logs) -/* log_level1 + log_level2 + stats, but not stable UAPI */ - load_attr.log_level =1 +2 +4 ; - - err =bpf_object__load_xattr (&load_attr); + err =bpf_object__load (obj);if (err) {p_err ("failed to load object file" ); goto err_close_obj;
@@ -1809,7 +1807,6 @@ static int do_loader(int argc, char **argv) {DECLARE_LIBBPF_OPTS (bpf_object_open_opts, open_opts);DECLARE_LIBBPF_OPTS (gen_loader_opts, gen); -struct bpf_object_load_attr load_attr = {};struct bpf_object *obj;const char *file;int err =0 ;
@@ -1818,6 +1815,10 @@ static int do_loader(int argc, char **argv)return -1 ; file =GET_ARG (); +if (verifier_logs) +/* log_level1 + log_level2 + stats, but not stable UAPI */ + open_opts.kernel_log_level =1 +2 +4 ; + obj =bpf_object__open_file (file, &open_opts);if (libbpf_get_error (obj)) {p_err ("failed to open object file" );
@@ -1828,12 +1829,7 @@ static int do_loader(int argc, char **argv)if (err) goto err_close_obj; - load_attr.obj = obj; -if (verifier_logs) -/* log_level1 + log_level2 + stats, but not stable UAPI */ - load_attr.log_level =1 +2 +4 ; - - err =bpf_object__load_xattr (&load_attr); + err =bpf_object__load (obj);if (err) {p_err ("failed to load object file" ); goto err_close_obj;
tools/bpf/bpftool/struct_ops.c +7 -8
@@ -479,7 +479,7 @@ static int do_unregister(int argc, char **argv)static int do_register (int argc,char **argv) { -struct bpf_object_load_attr load_attr = {}; +LIBBPF_OPTS (bpf_object_open_opts, open_opts);const struct bpf_map_def *def;struct bpf_map_info info = {};__u32 info_len =sizeof (info);
@@ -494,18 +494,17 @@ static int do_register(int argc, char **argv) file =GET_ARG (); - obj =bpf_object__open (file); +if (verifier_logs) +/* log_level1 + log_level2 + stats, but not stable UAPI */ + open_opts.kernel_log_level =1 +2 +4 ; + + obj =bpf_object__open_file (file, &open_opts);if (libbpf_get_error (obj))return -1 ;set_max_rlimit (); - load_attr.obj = obj; -if (verifier_logs) -/* log_level1 + log_level2 + stats, but not stable UAPI */ - load_attr.log_level =1 +2 +4 ; - -if (bpf_object__load_xattr (&load_attr)) { +if (bpf_object__load (obj)) {bpf_object__close (obj);return -1 ; }
Signature · 1 line
2.30.2
Quoted 18 lines · Andrii Nakryiko
Add new open options and per-program setters to control BTF and program loading log verboseness and allow providing custom log buffers to capture logs of interest. Note how custom
log_bufandlog_levelare orthogonal, which matches previous (alas less customizable) behavior of libbpf, even though it sort of worked by accident: if someone specifiedlog_level= 1 inbpf_object__load_xattr(), first attempt to load any BPF program resulted in wastedbpf()syscall with-EINVALdue to !!log_buf != !!log_level. Then on retry libbpf would allocatedlog_bufferand try again, after which prog loading would succeed and libbpf would print verbose program loading log through its print callback.This behavior is now documented and made more efficient, not wasting unnecessary syscall. But additionally,
log_levelcan be controlled globally on a per-bpf_object level throughbpf_object_open_opts, as well as on a per-program basis withbpf_program__set_log_buf()andbpf_program__set_log_level()APIs.
Quoted 7 lines
Now that we have a more future-proof way to set
log_level, deprecatebpf_object__load_xattr().
v1->v2:
- fixlog_level== 0 handling ofbpf_prog_load, add as patch #1 (Alexei);
- add comments explaininglog_buf_sizeoverflow prevention (Alexei).
Oh, the patch set was supposed to be marked with v2. I'll resend with proper v2 tag, sorry for spam.
Quoted 42 lines
Andrii Nakryiko (12):
libbpf: fixbpf_prog_load()log_buflogic forlog_level0
libbpf: add OPTS-basedbpf_btf_load()API
libbpf: allow passing preallocatedlog_bufwhen loading BTF into
kernel
libbpf: allow passing user log setting throughbpf_object_open_opts
libbpf: improve logging around BPF program loading
libbpf: preserve kernel error code and remove kprobe prog type
guessing
libbpf: add per-program log buffer setter and getter
libbpf: deprecatebpf_object__load_xattr()
selftests/bpf: replace all uses ofbpf_load_btf()withbpf_btf_load()selftests/bpf: add test for libbpf's customlog_bufbehavior selftests/bpf: remove the only use of deprecated
bpf_object__load_xattr()
bpftool: switchbpf_object__load_xattr()tobpf_object__load()tools/bpf/bpftool/gen.c | 11 +- tools/bpf/bpftool/prog.c | 24 +-- tools/bpf/bpftool/struct_ops.c | 15 +- tools/lib/bpf/bpf.c | 88 ++++++-- tools/lib/bpf/bpf.h | 22 +- tools/lib/bpf/btf.c | 78 ++++--- tools/lib/bpf/libbpf.c | 194 ++++++++++++------ tools/lib/bpf/libbpf.h | 49 ++++- tools/lib/bpf/libbpf.map | 3 + tools/lib/bpf/libbpf_internal.h | 1 + tools/lib/bpf/libbpf_probes.c | 2 +- .../selftests/bpf/map_tests/sk_storage_map.c | 2 +- .../selftests/bpf/prog_tests/bpf_tcp_ca.c | 6 +- tools/testing/selftests/bpf/prog_tests/btf.c | 50 +++-- .../selftests/bpf/prog_tests/log_buf.c | 137 +++++++++++++ .../selftests/bpf/progs/test_log_buf.c | 24 +++ tools/testing/selftests/bpf/test_verifier.c | 2 +- tools/testing/selftests/bpf/testing_helpers.c | 10 +- 18 files changed, 544 insertions(+), 174 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/log_buf.c create mode 100644 tools/testing/selftests/bpf/progs/test_log_buf.c--
2.30.2