bpf: Extend BPF syscall with common attributes support
Subsystems (6)
v5 cover bpf: Extend BPF syscall with common attributes support
This patch series builds upon the discussion in
"[PATCH bpf-next v4 0/4] bpf: Improve error reporting for freplace attachment failure" [1].
This patch series introduces support for *common attributes* in the BPF syscall, providing a unified mechanism for passing shared metadata across all BPF commands.
The initial set of common attributes includes:
1. 'log_buf': User-provided buffer for storing log output.
2. 'log_size': Size of the provided log buffer.
3. 'log_level': Verbosity level for logging.
4. 'log_true_size': The size of log reported by kernel.
With this extension, the BPF syscall will be able to return meaningful error messages (e.g., failures of creating map), improving debuggability and user experience.
Links:
[1] https://lore.kernel.org/bpf/[email protected]/
Changes:
v4 -> v5:
* Rework reporting 'log_true_size' for prog_load, btf_load, and map_create (per Alexei).
RFC v3 -> v4:
* Drop RFC.
* Address comments from Andrii:
* Add parentheses in 'sys_bpf_ext()'.
* Avoid creating new fd in 'probe_sys_bpf_ext()'.
* Add a new struct to wrap log fields in libbpf.
* Address comments from Alexei:
* Do not skip writing to user space when log_true_size is zero.
* Do not use 'bool' arguments.
* Drop the adding WARN_ON_ONCE()'s.
RFC v2 -> RFC v3:
* Rename probe_sys_bpf_extended to probe_sys_bpf_ext.
* Refactor reporting 'log_true_size' for prog_load.
* Refactor reporting 'btf_log_true_size' for btf_load.
* Add warnings for internal bugs in map_create.
* Check log_true_size in test cases.
* Address comment from Alexei:
* Change kvzalloc/kvfree to kzalloc/kfree.
* Address comments from Andrii:
* Move BPF_COMMON_ATTRS to 'enum bpf_cmd' alongside brief comment.
* Add bpf_check_uarg_tail_zero() for extra checks.
* Rename sys_bpf_extended to sys_bpf_ext.
* Rename sys_bpf_fd_extended to sys_bpf_ext_fd.
* Probe the new feature using NULL and -EFAULT.
* Move probe_sys_bpf_ext to libbpf_internal.h and drop LIBBPF_API.
* Return -EUSERS when log attrs are conflict between bpf_attr and bpf_common_attr.
* Avoid touching bpf_vlog_init().
* Update the reason messages in map_create.
* Finalize the log using __cleanup().
* Report log size to users.
* Change type of log_buf from '__u64' to 'const char *' and cast type
using ptr_to_u64() in bpf_map_create().
* Do not return -EOPNOTSUPP when kernel doesn't support this feature in bpf_map_create().
* Add log_level support for map creation for consistency.
* Address comment from Eduard:
* Use common_attrs->log_level instead of BPF_LOG_FIXED.
RFC v1 -> RFC v2:
* Fix build error reported by test bot.
* Address comments from Alexei:
* Drop new uapi for freplace.
* Add common attributes support for prog_load and btf_load.
* Add common attributes support for map_create.
Leon Hwang (9):
bpf: Extend BPF syscall with common attributes support
libbpf: Add support for extended bpf syscall
bpf: Refactor reporting log_true_size for prog_load
bpf: Add syscall common attributes support for prog_load
bpf: Refactor reporting btf_log_true_size for btf_load
bpf: Add syscall common attributes support for btf_load
bpf: Add syscall common attributes support for map_create
libbpf: Add common attr support for map_create selftests/bpf: Add tests to verify map create failure log
include/linux/bpf.h | 19 +- include/linux/bpf_verifier.h | 17 ++ include/linux/btf.h | 3 +- include/linux/syscalls.h | 3 +- include/uapi/linux/bpf.h | 8 + kernel/bpf/btf.c | 32 +--- kernel/bpf/log.c | 103 +++++++++++ kernel/bpf/syscall.c | 122 ++++++++++--- kernel/bpf/verifier.c | 19 +- tools/include/uapi/linux/bpf.h | 8 + tools/lib/bpf/bpf.c | 49 ++++- tools/lib/bpf/bpf.h | 17 +- tools/lib/bpf/features.c | 8 + tools/lib/bpf/libbpf_internal.h | 3 + .../selftests/bpf/prog_tests/map_init.c | 168 ++++++++++++++++++ 15 files changed, 518 insertions(+), 61 deletions(-)
--
2.52.0
PATCH v5 1/9 bpf: Extend BPF syscall with common attributes support
Extend the BPF syscall to support a set of common attributes shared across all BPF commands:
1. 'log_buf': User-provided buffer for storing logs.
2. 'log_size': Size of the log buffer.
3. 'log_level': Log verbosity level.
4. 'log_true_size': The size of log reported by kernel.
These common attributes are passed as the 4th argument to the BPF syscall, with the 5th argument specifying the size of this structure.
To indicate the use of these common attributes from userspace, a new flag 'BPF_COMMON_ATTRS' ('1 << 16') is introduced. This flag is OR-ed into the 'cmd' field of the syscall.
When 'cmd & BPF_COMMON_ATTRS' is set, the kernel will copy the common attributes from userspace into kernel space for use.
--- include/linux/syscalls.h | 3 ++- include/uapi/linux/bpf.h | 8 ++++++++ kernel/bpf/syscall.c | 25 +++++++++++++++++++++---- tools/include/uapi/linux/bpf.h | 8 ++++++++ 4 files changed, 39 insertions(+), 5 deletions(-)
include/linux/syscalls.h +2 -1
@@ -937,7 +937,8 @@ asmlinkage long sys_seccomp(unsigned int op, unsigned int flags,asmlinkage long sys_getrandom (char __user *buf,size_t count,unsigned int flags);asmlinkage long sys_memfd_create (const char __user *uname_ptr,unsigned int flags); -asmlinkage long sys_bpf (int cmd,union bpf_attr __user *attr,unsigned int size); +asmlinkage long sys_bpf (int cmd,union bpf_attr __user *attr,unsigned int size, +struct bpf_common_attr __user *attr_common,unsigned int size_common);asmlinkage long sys_execveat (int dfd,const char __user *filename,const char __user *const __user *argv,const char __user *const __user *envp,int flags);
include/uapi/linux/bpf.h +8 -0
@@ -986,6 +986,7 @@ enum bpf_cmd {BPF_PROG_STREAM_READ_BY_FD ,BPF_PROG_ASSOC_STRUCT_OPS , __MAX_BPF_CMD, + BPF_COMMON_ATTRS = 1 << 16, /* Indicate carrying bpf_common_attr. */ }; enum bpf_map_type {
@@ -1491,6 +1492,13 @@ struct bpf_stack_build_id { }; }; +struct bpf_common_attr { +__u64 log_buf; +__u32 log_size; +__u32 log_level; +__u32 log_true_size; +}; +#define BPF_OBJ_NAME_LEN 16Uenum {
kernel/bpf/syscall.c +21 -4
@@ -6163,8 +6163,10 @@ static int prog_assoc_struct_ops(union bpf_attr *attr)return ret; } -static int __sys_bpf (enum bpf_cmd cmd,bpfptr_t uattr,unsigned int size) +static int __sys_bpf (enum bpf_cmd cmd,bpfptr_t uattr,unsigned int size, +bpfptr_t uattr_common,unsigned int size_common) { +struct bpf_common_attr common_attr;union bpf_attr attr;int err;
@@ -6178,6 +6180,20 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size)if (copy_from_bpfptr (&attr, uattr, size) !=0 )return -EFAULT ; +memset (&common_attr,0 ,sizeof (common_attr)); +if (cmd &BPF_COMMON_ATTRS ) { + err =bpf_check_uarg_tail_zero (uattr_common,sizeof (common_attr), size_common); +if (err) +return err; + + cmd &= ~BPF_COMMON_ATTRS ; + size_common =min_t (u32, size_common,sizeof (common_attr)); +if (copy_from_bpfptr (&common_attr, uattr_common, size_common) !=0 ) +return -EFAULT ; + }else { + size_common =0 ; + } + err =security_bpf (cmd, &attr, size, uattr.is_kernel);if (err <0 )return err;
@@ -6313,9 +6329,10 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size)return err; } -SYSCALL_DEFINE3(bpf, int, cmd, union bpf_attr __user *, uattr, unsigned int, size) +SYSCALL_DEFINE5(bpf, int, cmd, union bpf_attr __user *, uattr, unsigned int, size, + struct bpf_common_attr __user *, uattr_common, unsigned int, size_common) { -return __sys_bpf (cmd,USER_BPFPTR (uattr), size); +return __sys_bpf (cmd,USER_BPFPTR (uattr), size,USER_BPFPTR (uattr_common), size_common); } static bool syscall_prog_is_valid_access(int off, int size,
@@ -6346,7 +6363,7 @@ BPF_CALL_3(bpf_sys_bpf, int, cmd, union bpf_attr *, attr, u32, attr_size)default :return -EINVAL ; } -return __sys_bpf (cmd,KERNEL_BPFPTR (attr), attr_size); +return __sys_bpf (cmd,KERNEL_BPFPTR (attr), attr_size,KERNEL_BPFPTR (NULL ),0 ); }
tools/include/uapi/linux/bpf.h +8 -0
@@ -986,6 +986,7 @@ enum bpf_cmd {BPF_PROG_STREAM_READ_BY_FD ,BPF_PROG_ASSOC_STRUCT_OPS , __MAX_BPF_CMD, + BPF_COMMON_ATTRS = 1 << 16, /* Indicate carrying bpf_common_attr. */ }; enum bpf_map_type {
@@ -1491,6 +1492,13 @@ struct bpf_stack_build_id { }; }; +struct bpf_common_attr { +__u64 log_buf; +__u32 log_size; +__u32 log_level; +__u32 log_true_size; +}; +#define BPF_OBJ_NAME_LEN 16Uenum {
Signature · 1 line
2.52.0
PATCH v5 2/9 libbpf: Add support for extended bpf syscall
In reply to Leon HwangTo support the extended BPF syscall introduced in the previous commit, introduce the following internal APIs:
* 'sys_bpf_ext()'
* 'sys_bpf_ext_fd()'
They wrap the raw 'syscall()' interface to support passing extended attributes.
* 'probe_sys_bpf_ext()'
Check whether current kernel supports the extended attributes.
--- tools/lib/bpf/bpf.c | 34 +++++++++++++++++++++++++++++++++ tools/lib/bpf/features.c | 8 ++++++++ tools/lib/bpf/libbpf_internal.h | 3 +++ 3 files changed, 45 insertions(+)
tools/lib/bpf/bpf.c +34 -0
@@ -69,6 +69,40 @@ static inline __u64 ptr_to_u64(const void *ptr)return (__u64 ) (unsigned long ) ptr; } +static inline int sys_bpf_ext (enum bpf_cmd cmd,union bpf_attr *attr, +unsigned int size, +struct bpf_common_attr *common_attr, +unsigned int size_common) +{ + cmd = common_attr ? (cmd |BPF_COMMON_ATTRS ) : (cmd & ~BPF_COMMON_ATTRS ); +return syscall (__NR_bpf, cmd, attr, size, common_attr, size_common); +} + +static inline int sys_bpf_ext_fd (enum bpf_cmd cmd,union bpf_attr *attr, +unsigned int size, +struct bpf_common_attr *common_attr, +unsigned int size_common) +{ +int fd; + + fd =sys_bpf_ext (cmd, attr, size, common_attr, size_common); +return ensure_good_fd (fd); +} + +int probe_sys_bpf_ext (void ) +{ + const size_t attr_sz = offsetofend(union bpf_attr, prog_token_fd); +union bpf_attr attr; +int fd; + +memset (&attr,0 , attr_sz); + fd =syscall (__NR_bpf,BPF_PROG_LOAD |BPF_COMMON_ATTRS , &attr, attr_sz,NULL , +sizeof (struct bpf_common_attr )); +if (fd >=0 ) +close (fd); +return errno ==EFAULT ; +} +static inline int sys_bpf (enum bpf_cmd cmd,union bpf_attr *attr,unsigned int size) {
tools/lib/bpf/features.c +8 -0
@@ -506,6 +506,11 @@ static int probe_kern_arg_ctx_tag(int token_fd)return probe_fd (prog_fd); } +static int probe_kern_extended_syscall (int token_fd) +{ +return probe_sys_bpf_ext (); +} +typedef int (*feature_probe_fn )(int /* token_fd */ );static struct kern_feature_cache feature_cache;
@@ -581,6 +586,9 @@ static struct kern_feature_desc { [FEAT_BTF_QMARK_DATASEC ] = {"BTF DATASEC names starting from '?'" , probe_kern_btf_qmark_datasec, }, + [FEAT_EXTENDED_SYSCALL ] = { +"Kernel supports extended syscall" , probe_kern_extended_syscall, + }, };bool feat_supported (struct kern_feature_cache *cache,enum kern_feature_id feat_id)
tools/lib/bpf/libbpf_internal.h +3 -0
@@ -392,6 +392,8 @@ enum kern_feature_id {FEAT_ARG_CTX_TAG ,/* Kernel supports '?' at the front of datasec names */ FEAT_BTF_QMARK_DATASEC , +/* Kernel supports extended syscall */ +FEAT_EXTENDED_SYSCALL , __FEAT_CNT, };
@@ -757,4 +759,5 @@ int probe_fd(int fd);#define SHA256_DWORD_SIZE SHA256_DIGEST_LENGTH / sizeof(__u64)void libbpf_sha256 (const void *data,size_t len,__u8 out[SHA256_DIGEST_LENGTH ]); +int probe_sys_bpf_ext (void );#endif /* __LIBBPF_LIBBPF_INTERNAL_H */
Signature · 1 line
2.52.0
PATCH v5 3/9 bpf: Refactor reporting log_true_size for prog_load
In reply to Leon HwangThe next commit will add support for reporting logs via extended common attributes, including 'log_true_size'.
To prepare for that, refactor the 'log_true_size' reporting logic by introducing a new struct bpf_log_attr to encapsulate log-related behavior:
* bpf_prog_load_log_attr_init(): initialize the log fields, which will support extended common attributes in the next commit.
* bpf_log_attr_finalize(): handle log finalization and write back 'log_true_size' to userspace.
--- include/linux/bpf.h | 19 ++++++++++++++++- include/linux/bpf_verifier.h | 11 ++++++++++ kernel/bpf/log.c | 40 ++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 9 +++++++- kernel/bpf/verifier.c | 19 ++++++----------- 5 files changed, 83 insertions(+), 15 deletions(-)
include/linux/bpf.h +18 -1
@@ -2867,8 +2867,25 @@ int bpf_get_file_flag(int flags);int bpf_check_uarg_tail_zero (bpfptr_t uaddr,size_t expected_size,size_t actual_size); +struct bpf_attrs { +const void *attr; +bpfptr_t uattr; +u32 size; +}; + +static inline void bpf_attrs_init (struct bpf_attrs *attrs,const void *attr,bpfptr_t uattr, +u32 size) +{ +memset (attrs,0 ,sizeof (*attrs)); + attrs->attr = attr; + attrs->uattr = uattr; + attrs->size = size; +} +/* verify correctness of eBPF program */ -int bpf_check (struct bpf_prog **fp,union bpf_attr *attr,bpfptr_t uattr,u32 uattr_size); +struct bpf_log_attr ; +int bpf_check (struct bpf_prog **fp,union bpf_attr *attr,bpfptr_t uattr, +struct bpf_log_attr *log_attr);#ifndef CONFIG_BPF_JIT_ALWAYS_ON void bpf_patch_call_args (struct bpf_insn *insn,u32 stack_depth);
include/linux/bpf_verifier.h +11 -0
@@ -631,6 +631,17 @@ static inline bool bpf_verifier_log_needed(const struct bpf_verifier_log *log)return log && log->level; } +struct bpf_log_attr { +u64 log_buf; +u32 log_size; +u32 log_level; +struct bpf_attrs *attrs; +u32 offsetof_log_true_size; +}; + +int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs); +int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log); +#define BPF_MAX_SUBPROGS 256struct bpf_subprog_arg_info {
kernel/bpf/log.c +40 -0
@@ -863,3 +863,43 @@ void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st }print_verifier_state (env, vstate, frameno, false); } + +static int bpf_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,u64 log_buf, +u32 log_size,u32 log_level,int offsetof_log_true_size) +{ +memset (log_attr,0 ,sizeof (*log_attr)); + log_attr->log_buf = log_buf; + log_attr->log_size = log_size; + log_attr->log_level = log_level; + log_attr->attrs = attrs; + log_attr->offsetof_log_true_size = offsetof_log_true_size; +return 0 ; +} + +int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs) +{ +const union bpf_attr *attr = attrs->attr; + +return bpf_log_attr_init (log_attr, attrs, attr->log_buf, attr->log_size, attr->log_level, + offsetof(union bpf_attr , log_true_size)); +} + +int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log) +{ +u32 log_true_size, off; +size_t size; +int err; + +if (!log) +return 0 ; + + err =bpf_vlog_finalize (log, &log_true_size); + + size =sizeof (log_true_size); + off = log_attr->offsetof_log_true_size; +if (log_attr->attrs && log_attr->attrs->size >= off + size && +copy_to_bpfptr_offset (log_attr->attrs->uattr, off, &log_true_size, size)) + err = -EFAULT ; + +return err; +}
kernel/bpf/syscall.c +8 -1
@@ -2871,6 +2871,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)struct bpf_prog *prog, *dst_prog =NULL ;struct btf *attach_btf =NULL ;struct bpf_token *token =NULL ; +struct bpf_log_attr log_attr; +struct bpf_attrs attrs;bool bpf_cap;int err;char license[128 ];
@@ -3082,8 +3084,13 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)if (err) goto free_prog_sec; +bpf_attrs_init (&attrs, attr, uattr, uattr_size); + err =bpf_prog_load_log_attr_init (&log_attr, &attrs); +if (err <0 ) + goto free_used_maps; +/* run eBPF verifier */ - err =bpf_check (&prog, attr, uattr, uattr_size); + err =bpf_check (&prog, attr, uattr, &log_attr);if (err <0 ) goto free_used_maps;
kernel/bpf/verifier.c +6 -13
@@ -25106,12 +25106,12 @@ static int compute_scc(struct bpf_verifier_env *env)return err; } -int bpf_check (struct bpf_prog **prog,union bpf_attr *attr,bpfptr_t uattr,__u32 uattr_size) +int bpf_check (struct bpf_prog **prog,union bpf_attr *attr,bpfptr_t uattr, +struct bpf_log_attr *log_attr) {u64 start_time =ktime_get_ns ();struct bpf_verifier_env *env;int i, len, ret = -EINVAL , err; -u32 log_true_size;bool is_priv;BTF_TYPE_EMIT (enum bpf_features );
@@ -25158,9 +25158,9 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3/* user could have requested verbose verifier output * and supplied buffer to store the verification trace */ - ret =bpf_vlog_init (&env->log, attr->log_level, - (char __user *) (unsigned long ) attr->log_buf, - attr->log_size); + ret =bpf_vlog_init (&env->log, log_attr->log_level, +u64_to_user_ptr (log_attr->log_buf), + log_attr->log_size);if (ret) goto err_unlock;
@@ -25310,17 +25310,10 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3 env->prog->aux->verified_insns = env->insn_processed;/* preserve original error even if log finalization is successful */ - err =bpf_vlog_finalize (&env->log, &log_true_size); + err =bpf_log_attr_finalize (log_attr, &env->log);if (err) ret = err; - if (uattr_size >= offsetofend(union bpf_attr, log_true_size) && -copy_to_bpfptr_offset (uattr, offsetof(union bpf_attr , log_true_size), - &log_true_size,sizeof (log_true_size))) { - ret = -EFAULT ; - goto err_release_maps; - } -if (ret) goto err_release_maps;
Signature · 1 line
2.52.0
PATCH v5 4/9 bpf: Add syscall common attributes support for prog_load
In reply to Leon HwangThe log buffer of common attributes would be confusing with the one in 'union bpf_attr' for BPF_PROG_LOAD.
In order to clarify the usage of these two log buffers, they both can be used for logging if:
* They are same, including 'log_buf', 'log_level' and 'log_size'.
* One of them is missing, then another one will be used for logging.
If they both have 'log_buf' but they are not same totally, return -EUSERS.
--- include/linux/bpf_verifier.h | 4 +++- kernel/bpf/log.c | 29 ++++++++++++++++++++++++++--- kernel/bpf/syscall.c | 9 ++++++--- 3 files changed, 35 insertions(+), 7 deletions(-)
include/linux/bpf_verifier.h +3 -1
@@ -637,9 +637,11 @@ struct bpf_log_attr {u32 log_level;struct bpf_attrs *attrs;u32 offsetof_log_true_size; +struct bpf_attrs *attrs_common; }; -int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs); +int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs, +struct bpf_attrs *attrs_common);int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log);#define BPF_MAX_SUBPROGS 256
kernel/bpf/log.c +26 -3
@@ -865,23 +865,41 @@ void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st }static int bpf_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,u64 log_buf, -u32 log_size,u32 log_level,int offsetof_log_true_size) +u32 log_size,u32 log_level,int offsetof_log_true_size, +struct bpf_attrs *attrs_common) { +const struct bpf_common_attr *common_attr = attrs_common ? attrs_common->attr :NULL ; +memset (log_attr,0 ,sizeof (*log_attr)); log_attr->log_buf = log_buf; log_attr->log_size = log_size; log_attr->log_level = log_level; log_attr->attrs = attrs; log_attr->offsetof_log_true_size = offsetof_log_true_size; + log_attr->attrs_common = attrs_common; + +if (log_buf && common_attr && common_attr->log_buf && + (log_buf != common_attr->log_buf || + log_size != common_attr->log_size || + log_level != common_attr->log_level)) +return -EUSERS ; + +if (!log_buf && common_attr && common_attr->log_buf) { + log_attr->log_buf = common_attr->log_buf; + log_attr->log_size = common_attr->log_size; + log_attr->log_level = common_attr->log_level; + } +return 0 ; } -int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs) +int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs, +struct bpf_attrs *attrs_common) {const union bpf_attr *attr = attrs->attr;return bpf_log_attr_init (log_attr, attrs, attr->log_buf, attr->log_size, attr->log_level, - offsetof(union bpf_attr , log_true_size)); + offsetof(union bpf_attr , log_true_size), attrs_common); }int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log)
@@ -901,5 +919,10 @@ int bpf_log_attr_finalize(struct bpf_log_attr *log_attr, struct bpf_verifier_log copy_to_bpfptr_offset(log_attr->attrs->uattr, off, &log_true_size, size)) err = -EFAULT ; + off = offsetof(struct bpf_common_attr , log_true_size); +if (log_attr->attrs_common && log_attr->attrs_common->size >= off + size && +copy_to_bpfptr_offset (log_attr->attrs_common->uattr, off, &log_true_size, size)) + err = -EFAULT ; +return err; }
kernel/bpf/syscall.c +6 -3
@@ -2865,7 +2865,8 @@ static int bpf_prog_mark_insn_arrays_ready(struct bpf_prog *prog)/* last field in 'union bpf_attr' used by this command */ #define BPF_PROG_LOAD_LAST_FIELD keyring_id -static int bpf_prog_load (union bpf_attr *attr,bpfptr_t uattr,u32 uattr_size) +static int bpf_prog_load (union bpf_attr *attr,bpfptr_t uattr,u32 uattr_size, +struct bpf_attrs *common_attrs) {enum bpf_prog_type type = attr->prog_type;struct bpf_prog *prog, *dst_prog =NULL ;
@@ -3085,7 +3086,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size) goto free_prog_sec;bpf_attrs_init (&attrs, attr, uattr, uattr_size); - err =bpf_prog_load_log_attr_init (&log_attr, &attrs); + err =bpf_prog_load_log_attr_init (&log_attr, &attrs, common_attrs);if (err <0 ) goto free_used_maps;
@@ -6174,6 +6175,7 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size, bpfptr_t uattr_common, unsigned int size_common) { struct bpf_common_attr common_attr; +struct bpf_attrs common_attrs;union bpf_attr attr;int err;
@@ -6225,7 +6227,8 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size, err =map_freeze (&attr);break ;case BPF_PROG_LOAD : - err =bpf_prog_load (&attr, uattr, size); +bpf_attrs_init (&common_attrs, &common_attr, uattr_common, size_common); + err =bpf_prog_load (&attr, uattr, size, &common_attrs);break ;case BPF_OBJ_PIN : err =bpf_obj_pin (&attr);
Signature · 1 line
2.52.0
PATCH v5 5/9 bpf: Refactor reporting btf_log_true_size for btf_load
In reply to Leon HwangIn the next commit, it will be able to report logs via extended common attributes, which will report 'log_true_size' via the extended common attributes meanwhile.
Therefore, refactor the way of 'btf_log_true_size' reporting in order to report 'log_true_size' via the extended common attributes easily.
--- include/linux/bpf_verifier.h | 1 + include/linux/btf.h | 3 ++- kernel/bpf/btf.c | 32 +++++++++----------------------- kernel/bpf/log.c | 9 +++++++++ kernel/bpf/syscall.c | 10 +++++++++- 5 files changed, 30 insertions(+), 25 deletions(-)
include/linux/bpf_verifier.h +1 -0
@@ -642,6 +642,7 @@ struct bpf_log_attr {int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,struct bpf_attrs *attrs_common); +int bpf_btf_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs);int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log);#define BPF_MAX_SUBPROGS 256
include/linux/btf.h +2 -1
@@ -144,7 +144,8 @@ const char *btf_get_name(const struct btf *btf);void btf_get (struct btf *btf);void btf_put (struct btf *btf);const struct btf_header *btf_header (const struct btf *btf); -int btf_new_fd (const union bpf_attr *attr,bpfptr_t uattr,u32 uattr_sz); +struct bpf_log_attr ; +int btf_new_fd (const union bpf_attr *attr,bpfptr_t uattr,struct bpf_log_attr *log_attr);struct btf *btf_get_by_fd (int fd); int btf_get_info_by_fd(const struct btf *btf, const union bpf_attr *attr,
kernel/bpf/btf.c +9 -23
@@ -5745,25 +5745,11 @@ static int btf_check_type_tags(struct btf_verifier_env *env,return 0 ; } -static int finalize_log (struct bpf_verifier_log *log,bpfptr_t uattr,u32 uattr_size) -{ -u32 log_true_size; -int err; - - err =bpf_vlog_finalize (log, &log_true_size); - - if (uattr_size >= offsetofend(union bpf_attr, btf_log_true_size) && -copy_to_bpfptr_offset (uattr, offsetof(union bpf_attr , btf_log_true_size), - &log_true_size,sizeof (log_true_size))) - err = -EFAULT ; - -return err; -} - -static struct btf *btf_parse (const union bpf_attr *attr,bpfptr_t uattr,u32 uattr_size) +static struct btf *btf_parse (const union bpf_attr *attr,bpfptr_t uattr, +struct bpf_log_attr *log_attr) {bpfptr_t btf_data =make_bpfptr (attr->btf, uattr.is_kernel); -char __user *log_ubuf =u64_to_user_ptr (attr->btf_log_buf); +char __user *log_ubuf =u64_to_user_ptr (log_attr->log_buf);struct btf_struct_metas *struct_meta_tab;struct btf_verifier_env *env =NULL ;struct btf *btf =NULL ;
@@ -5780,8 +5766,8 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat/* user could have requested verbose verifier output * and supplied buffer to store the verification trace */ - err =bpf_vlog_init (&env->log, attr->btf_log_level, - log_ubuf, attr->btf_log_size); + err =bpf_vlog_init (&env->log, log_attr->log_level, + log_ubuf, log_attr->log_size);if (err) goto errout_free;
@@ -5841,7 +5827,7 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat } } - err =finalize_log (&env->log, uattr, uattr_size); + err =bpf_log_attr_finalize (log_attr, &env->log);if (err) goto errout_free;
@@ -5853,7 +5839,7 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uatbtf_free_struct_meta_tab (btf); errout:/* overwrite err with -ENOSPC or -EFAULT */ - ret =finalize_log (&env->log, uattr, uattr_size); + ret =bpf_log_attr_finalize (log_attr, &env->log);if (ret) err = ret; errout_free:
@@ -8017,12 +8003,12 @@ static int __btf_new_fd(struct btf *btf)return anon_inode_getfd ("btf" , &btf_fops, btf,O_RDONLY |O_CLOEXEC ); } -int btf_new_fd (const union bpf_attr *attr,bpfptr_t uattr,u32 uattr_size) +int btf_new_fd (const union bpf_attr *attr,bpfptr_t uattr,struct bpf_log_attr *log_attr) {struct btf *btf;int ret; - btf =btf_parse (attr, uattr, uattr_size); + btf =btf_parse (attr, uattr, log_attr);if (IS_ERR (btf))return PTR_ERR (btf);
kernel/bpf/log.c +9 -0
@@ -902,6 +902,15 @@ int bpf_prog_load_log_attr_init(struct bpf_log_attr *log_attr, struct bpf_attrs offsetof(union bpf_attr, log_true_size), attrs_common); } +int bpf_btf_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs) +{ +const union bpf_attr *attr = attrs->attr; + +return bpf_log_attr_init (log_attr, attrs, attr->btf_log_buf, attr->btf_log_size, + attr->btf_log_level, offsetof(union bpf_attr , btf_log_true_size), +NULL ); +} +int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log) {u32 log_true_size, off;
kernel/bpf/syscall.c +9 -1
@@ -5436,6 +5436,9 @@ static int bpf_obj_get_info_by_fd(const union bpf_attr *attr,static int bpf_btf_load (const union bpf_attr *attr,bpfptr_t uattr,__u32 uattr_size) {struct bpf_token *token =NULL ; +struct bpf_log_attr log_attr; +struct bpf_attrs attrs; +int err;if (CHECK_ATTR (BPF_BTF_LOAD ))return -EINVAL ;
@@ -5443,6 +5446,11 @@ static int bpf_btf_load(const union bpf_attr *attr, bpfptr_t uattr, __u32 uattr_if (attr->btf_flags & ~BPF_F_TOKEN_FD )return -EINVAL ; +bpf_attrs_init (&attrs, attr, uattr, uattr_size); + err =bpf_btf_load_log_attr_init (&log_attr, &attrs); +if (err) +return err; +if (attr->btf_flags &BPF_F_TOKEN_FD ) { token =bpf_token_get_from_fd (attr->btf_token_fd);if (IS_ERR (token))
@@ -5460,7 +5468,7 @@ static int bpf_btf_load(const union bpf_attr *attr, bpfptr_t uattr, __u32 uattr_bpf_token_put (token); -return btf_new_fd (attr, uattr, uattr_size); +return btf_new_fd (attr, uattr, &log_attr); }#define BPF_BTF_GET_FD_BY_ID_LAST_FIELD fd_by_id_token_fd
Signature · 1 line
2.52.0
PATCH v5 6/9 bpf: Add syscall common attributes support for btf_load
In reply to Leon HwangSince bpf_log_attr_init() now supports struct bpf_common_attr, pass the common attributes to it to enable syscall common attributes support for BPF_BTF_LOAD.
--- include/linux/bpf_verifier.h | 3 ++- kernel/bpf/log.c | 5 +++-- kernel/bpf/syscall.c | 8 +++++--- 3 files changed, 10 insertions(+), 6 deletions(-)
include/linux/bpf_verifier.h +2 -1
@@ -642,7 +642,8 @@ struct bpf_log_attr {int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,struct bpf_attrs *attrs_common); -int bpf_btf_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs); +int bpf_btf_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs, +struct bpf_attrs *attrs_common);int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log);#define BPF_MAX_SUBPROGS 256
kernel/bpf/log.c +3 -2
@@ -902,13 +902,14 @@ int bpf_prog_load_log_attr_init(struct bpf_log_attr *log_attr, struct bpf_attrs offsetof(union bpf_attr, log_true_size), attrs_common); } -int bpf_btf_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs) +int bpf_btf_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs, +struct bpf_attrs *attrs_common) {const union bpf_attr *attr = attrs->attr;return bpf_log_attr_init (log_attr, attrs, attr->btf_log_buf, attr->btf_log_size, attr->btf_log_level, offsetof(union bpf_attr , btf_log_true_size), -NULL ); + attrs_common); }int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log)
kernel/bpf/syscall.c +5 -3
@@ -5433,7 +5433,8 @@ static int bpf_obj_get_info_by_fd(const union bpf_attr *attr,#define BPF_BTF_LOAD_LAST_FIELD btf_token_fd -static int bpf_btf_load (const union bpf_attr *attr,bpfptr_t uattr,__u32 uattr_size) +static int bpf_btf_load (const union bpf_attr *attr,bpfptr_t uattr,__u32 uattr_size, +struct bpf_attrs *common_attrs) {struct bpf_token *token =NULL ;struct bpf_log_attr log_attr;
@@ -5447,7 +5448,7 @@ static int bpf_btf_load(const union bpf_attr *attr, bpfptr_t uattr, __u32 uattr_return -EINVAL ;bpf_attrs_init (&attrs, attr, uattr, uattr_size); - err =bpf_btf_load_log_attr_init (&log_attr, &attrs); + err =bpf_btf_load_log_attr_init (&log_attr, &attrs, common_attrs);if (err)return err;
@@ -6281,7 +6282,8 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size, err =bpf_raw_tracepoint_open (&attr);break ;case BPF_BTF_LOAD : - err =bpf_btf_load (&attr, uattr, size); +bpf_attrs_init (&common_attrs, &common_attr, uattr_common, size_common); + err =bpf_btf_load (&attr, uattr, size, &common_attrs);break ;case BPF_BTF_GET_FD_BY_ID : err =bpf_btf_get_fd_by_id (&attr);
Signature · 1 line
2.52.0
PATCH v5 7/9 bpf: Add syscall common attributes support for map_create
In reply to Leon HwangCurrently, many BPF_MAP_CREATE failures return -EINVAL without providing any explanation to userspace.
With extended BPF syscall support, detailed error messages can now be reported via the log buffer, allowing users to understand the specific reason for a failed map creation.
--- include/linux/bpf_verifier.h | 2 ++ kernel/bpf/log.c | 30 +++++++++++++++++ kernel/bpf/syscall.c | 65 ++++++++++++++++++++++++++++++------ 3 files changed, 87 insertions(+), 10 deletions(-)
include/linux/bpf_verifier.h +2 -0
@@ -644,6 +644,8 @@ int bpf_prog_load_log_attr_init(struct bpf_log_attr *log_attr, struct bpf_attrs struct bpf_attrs *attrs_common);int bpf_btf_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,struct bpf_attrs *attrs_common); +struct bpf_verifier_log *bpf_log_attr_create_vlog (struct bpf_log_attr *log_attr, +struct bpf_attrs *attrs_common);int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log);#define BPF_MAX_SUBPROGS 256
kernel/bpf/log.c +30 -0
@@ -912,6 +912,36 @@ int bpf_btf_load_log_attr_init(struct bpf_log_attr *log_attr, struct bpf_attrs * attrs_common); } +struct bpf_verifier_log *bpf_log_attr_create_vlog (struct bpf_log_attr *log_attr, +struct bpf_attrs *attrs_common) +{ +const struct bpf_common_attr *common_attr = attrs_common->attr; +struct bpf_verifier_log *log; +int err; + +memset (log_attr,0 ,sizeof (*log_attr)); + log_attr->log_buf = common_attr->log_buf; + log_attr->log_size = common_attr->log_size; + log_attr->log_level = common_attr->log_level; + log_attr->attrs_common = attrs_common; + +if (!log_attr->log_buf) +return NULL ; + + log =kzalloc (sizeof (*log),GFP_KERNEL ); +if (!log) +return ERR_PTR (-ENOMEM ); + + err =bpf_vlog_init (log, log_attr->log_level,u64_to_user_ptr (log_attr->log_buf), + log_attr->log_size); +if (err) { +kfree (log); +return ERR_PTR (err); + } + +return log; +} +int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log) {u32 log_true_size, off;
kernel/bpf/syscall.c +55 -10
147 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v5 8/9 libbpf: Add common attr support for map_create
In reply to Leon HwangWith the previous commit adding common attribute support for BPF_MAP_CREATE, users can now retrieve detailed error messages when map creation fails via the log_buf field.
Introduce struct bpf_syscall_common_attr_opts with the following fields:log_buf, log_size, log_level, and log_true_size.
Extend bpf_map_create_opts with a new field common_attr_opts, allowing users to capture and inspect log messages on map creation failures.
--- tools/lib/bpf/bpf.c | 15 ++++++++++++++- tools/lib/bpf/bpf.h | 17 ++++++++++++++++- 2 files changed, 30 insertions(+), 2 deletions(-)
tools/lib/bpf/bpf.c +14 -1
@@ -207,6 +207,9 @@ int bpf_map_create(enum bpf_map_type map_type, const struct bpf_map_create_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, excl_prog_hash_size); +const size_t common_attr_sz =sizeof (struct bpf_common_attr ); +struct bpf_syscall_common_attr_opts *common_attr_opts; +struct bpf_common_attr common_attr;union bpf_attr attr;int fd;
@@ -240,7 +243,17 @@ int bpf_map_create(enum bpf_map_type map_type, attr.excl_prog_hash =ptr_to_u64 (OPTS_GET (opts, excl_prog_hash,NULL )); attr.excl_prog_hash_size =OPTS_GET (opts, excl_prog_hash_size,0 ); - fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz); + common_attr_opts =OPTS_GET (opts, common_attr_opts,NULL ); +if (common_attr_opts &&feat_supported (NULL ,FEAT_EXTENDED_SYSCALL )) { +memset (&common_attr,0 , common_attr_sz); + common_attr.log_buf =ptr_to_u64 (OPTS_GET (common_attr_opts, log_buf,NULL )); + common_attr.log_size =OPTS_GET (common_attr_opts, log_size,0 ); + common_attr.log_level =OPTS_GET (common_attr_opts, log_level,0 ); + fd =sys_bpf_ext_fd (BPF_MAP_CREATE , &attr, attr_sz, &common_attr, common_attr_sz); +OPTS_SET (common_attr_opts, log_true_size, common_attr.log_true_size); + }else { + fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz); + }return libbpf_err_errno (fd); }
tools/lib/bpf/bpf.h +16 -1
@@ -37,6 +37,18 @@ extern "C" { LIBBPF_API int libbpf_set_memlock_rlim(size_t memlock_bytes); +struct bpf_syscall_common_attr_opts { +size_t sz;/* size of this struct for forward/backward compatibility */ + +char *log_buf; +__u32 log_size; +__u32 log_level; +__u32 log_true_size; + + size_t :0; +}; +#define bpf_syscall_common_attr_opts__last_field log_true_size +struct bpf_map_create_opts {size_t sz;/* size of this struct for forward/backward compatibility */
@@ -57,9 +69,12 @@ struct bpf_map_create_opts {const void *excl_prog_hash;__u32 excl_prog_hash_size; + +struct bpf_syscall_common_attr_opts *common_attr_opts; + size_t :0; }; -#define bpf_map_create_opts__last_field excl_prog_hash_size +#define bpf_map_create_opts__last_field common_attr_opts LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name,
Signature · 1 line
2.52.0
PATCH v5 9/9 selftests/bpf: Add tests to verify map create failure log
In reply to Leon HwangAdd tests to verify that the kernel reports the expected error messages when map creation fails.
--- .../selftests/bpf/prog_tests/map_init.c | 168 ++++++++++++++++++ 1 file changed, 168 insertions(+)
tools/testing/selftests/bpf/prog_tests/map_init.c +168 -0
@@ -212,3 +212,171 @@ void test_map_init(void)if (test__start_subtest ("pcpu_lru_map_init" ))test_pcpu_lru_map_init (); } + +#define BPF_LOG_FIXED 8 + +static void test_map_create (enum bpf_map_type map_type,const char *map_name, +struct bpf_map_create_opts *opts,const char *exp_msg) +{ +const int key_size =4 , value_size =4 , max_entries =1 ; +char log_buf[128 ]; +int fd; +LIBBPF_OPTS (bpf_syscall_common_attr_opts, copts); + + log_buf[0 ] ='\0' ; + copts.log_buf = log_buf; + copts.log_size =sizeof (log_buf); + copts.log_level =BPF_LOG_FIXED ; + opts->common_attr_opts = &copts; + fd =bpf_map_create (map_type, map_name, key_size, value_size, max_entries, opts); +if (!ASSERT_LT (fd,0 ,"bpf_map_create" )) { +close (fd); +return ; + } + +ASSERT_STREQ (log_buf, exp_msg,"log_buf" ); +ASSERT_EQ (copts.log_true_size,strlen (exp_msg) +1 ,"log_true_size" ); +} + +static void test_map_create_array (struct bpf_map_create_opts *opts,const char *exp_msg) +{ +test_map_create (BPF_MAP_TYPE_ARRAY ,"test_map_create" , opts, exp_msg); +} + +static void test_invalid_vmlinux_value_type_id_struct_ops (void ) +{ +const char *msg ="btf_vmlinux_value_type_id can only be used with struct_ops maps.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .btf_vmlinux_value_type_id = 1, + ); + +test_map_create_array (&opts, msg); +} + +static void test_invalid_vmlinux_value_type_id_kv_type_id (void ) +{ +const char *msg ="btf_vmlinux_value_type_id is mutually exclusive with btf_key_type_id and btf_value_type_id.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .btf_vmlinux_value_type_id = 1, + .btf_key_type_id = 1, + ); + +test_map_create (BPF_MAP_TYPE_STRUCT_OPS ,"test_map_create" , &opts, msg); +} + +static void test_invalid_value_type_id (void ) +{ +const char *msg ="Invalid btf_value_type_id.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .btf_key_type_id = 1, + ); + +test_map_create_array (&opts, msg); +} + +static void test_invalid_map_extra (void ) +{ +const char *msg ="Invalid map_extra.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .map_extra = 1, + ); + +test_map_create_array (&opts, msg); +} + +static void test_invalid_numa_node (void ) +{ +const char *msg ="Invalid numa_node.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .map_flags = BPF_F_NUMA_NODE, + .numa_node = 0xFF, + ); + +test_map_create_array (&opts, msg); +} + +static void test_invalid_map_type (void ) +{ +const char *msg ="Invalid map_type.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts); + +test_map_create (__MAX_BPF_MAP_TYPE,"test_map_create" , &opts, msg); +} + +static void test_invalid_token_fd (void ) +{ +const char *msg ="Invalid map_token_fd.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .map_flags = BPF_F_TOKEN_FD, + .token_fd = 0xFF, + ); + +test_map_create_array (&opts, msg); +} + +static void test_invalid_map_name (void ) +{ +const char *msg ="Invalid map_name.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts); + +test_map_create (BPF_MAP_TYPE_ARRAY ,"test-!@#" , &opts, msg); +} + +static void test_invalid_btf_fd (void ) +{ +const char *msg ="Invalid btf_fd.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .btf_fd = -1, + .btf_key_type_id = 1, + .btf_value_type_id = 1, + ); + +test_map_create_array (&opts, msg); +} + +static void test_excl_prog_hash_size_1 (void ) +{ +const char *msg ="Invalid excl_prog_hash_size.\n" ; +const char *hash ="DEADCODE" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .excl_prog_hash = hash, + ); + +test_map_create_array (&opts, msg); +} + +static void test_excl_prog_hash_size_2 (void ) +{ +const char *msg ="Invalid excl_prog_hash_size.\n" ; +LIBBPF_OPTS (bpf_map_create_opts, opts, + .excl_prog_hash_size = 1, + ); + +test_map_create_array (&opts, msg); +} + +void test_map_create_failure (void ) +{ +if (test__start_subtest ("invalid_vmlinux_value_type_id_struct_ops" )) +test_invalid_vmlinux_value_type_id_struct_ops (); +if (test__start_subtest ("invalid_vmlinux_value_type_id_kv_type_id" )) +test_invalid_vmlinux_value_type_id_kv_type_id (); +if (test__start_subtest ("invalid_value_type_id" )) +test_invalid_value_type_id (); +if (test__start_subtest ("invalid_map_extra" )) +test_invalid_map_extra (); +if (test__start_subtest ("invalid_numa_node" )) +test_invalid_numa_node (); +if (test__start_subtest ("invalid_map_type" )) +test_invalid_map_type (); +if (test__start_subtest ("invalid_token_fd" )) +test_invalid_token_fd (); +if (test__start_subtest ("invalid_map_name" )) +test_invalid_map_name (); +if (test__start_subtest ("invalid_btf_fd" )) +test_invalid_btf_fd (); +if (test__start_subtest ("invalid_excl_prog_hash_size_1" )) +test_excl_prog_hash_size_1 (); +if (test__start_subtest ("invalid_excl_prog_hash_size_2" )) +test_excl_prog_hash_size_2 (); +}
Signature · 1 line
2.52.0
Re: [PATCH bpf-next v5 2/9] libbpf: Add support for extended bpf syscall
In reply to Leon HwangQuoted 23 lines · Leon Hwang
To support the extended BPF syscall introduced in the previous commit, introduce the following internal APIs:
* '
sys_bpf_ext()'
* 'sys_bpf_ext_fd()'
They wrap the raw 'syscall()' interface to support passing extended attributes.
* 'probe_sys_bpf_ext()'
Check whether current kernel supports the extended attributes.Signed-off-by Leon Hwang <[email protected]>--- tools/lib/bpf/bpf.c | 34 +++++++++++++++++++++++++++++++++ tools/lib/bpf/features.c | 8 ++++++++ tools/lib/bpf/libbpf_internal.h | 3 +++ 3 files changed, 45 insertions(+)tools/lib/bpf/bpf.c +0
-0
tools/lib/bpf/bpf.c static inline __u64 ptr_to_u64(const void *ptr) · line 74
Quoted 6 lines
tools/lib/bpf/bpf.c +3
-0@@ -69,6 +69,40 @@ static inline __u64 ptr_to_u64(const void *ptr)return (__u64 ) (unsigned long ) ptr; } +static inline int sys_bpf_ext(enum bpf_cmd cmd, union bpf_attr *attr, + unsigned int size, + struct bpf_common_attr *common_attr,
nit: kernel uses consistent attr_common/size_common pattern, but here you are inverting attr_common -> common_attr, let's not?
Quoted 18 lines
tools/lib/bpf/bpf.c +18
-0+ unsigned int size_common) +{ + cmd = common_attr ? (cmd |BPF_COMMON_ATTRS ) : (cmd & ~BPF_COMMON_ATTRS ); +return syscall (__NR_bpf, cmd, attr, size, common_attr, size_common); +} + +static inline int sys_bpf_ext_fd (enum bpf_cmd cmd,union bpf_attr *attr, +unsigned int size, +struct bpf_common_attr *common_attr, +unsigned int size_common) +{ +int fd; + + fd =sys_bpf_ext (cmd, attr, size, common_attr, size_common); +return ensure_good_fd (fd); +} + +int probe_sys_bpf_ext (void )
tools/lib/bpf/bpf.c static inline __u64 ptr_to_u64(const void *ptr) · line 102
Quoted 10 lines
tools/lib/bpf/bpf.c +10
-0+{ + const size_t attr_sz = offsetofend(union bpf_attr, prog_token_fd); +union bpf_attr attr; +int fd; + +memset (&attr,0 , attr_sz); + fd =syscall (__NR_bpf,BPF_PROG_LOAD |BPF_COMMON_ATTRS , &attr, attr_sz,NULL , +sizeof (struct bpf_common_attr )); +if (fd >=0 ) +close (fd);
hm... close can change errno, this is fragile. If fd >= 0, something is wrong with our detection, just return error right away?
Quoted 23 lines
tools/lib/bpf/bpf.c +3
-0+return errno ==EFAULT ; +} +static inline int sys_bpf (enum bpf_cmd cmd,union bpf_attr *attr,unsigned int size) {tools/lib/bpf/features.c +5
-0@@ -506,6 +506,11 @@ static int probe_kern_arg_ctx_tag(int token_fd)return probe_fd (prog_fd); } +static int probe_kern_extended_syscall (int token_fd) +{ +return probe_sys_bpf_ext (); +} +typedef int (*feature_probe_fn )(int /* token_fd */ );static struct kern_feature_cache feature_cache;
tools/lib/bpf/features.c static struct kern_feature_desc { · line 590
Quoted 5 lines
tools/lib/bpf/features.c +2
-0@@ -581,6 +586,9 @@ static struct kern_feature_desc { [FEAT_BTF_QMARK_DATASEC] = { "BTF DATASEC names starting from '?'", probe_kern_btf_qmark_datasec, }, + [FEAT_EXTENDED_SYSCALL] = { + "Kernel supports extended syscall", probe_kern_extended_syscall,
"extended syscall" is a bit vague... We specifically detect common attrs support, maybe say that?
Quoted 9 lines
tools/lib/bpf/features.c +1
-0+ }, };bool feat_supported (struct kern_feature_cache *cache,enum kern_feature_id feat_id)tools/lib/bpf/libbpf_internal.h +0
-0
tools/lib/bpf/libbpf_internal.h enum kern_feature_id { · line 396
Quoted 5 lines
tools/lib/bpf/libbpf_internal.h +2
-0@@ -392,6 +392,8 @@ enum kern_feature_id {FEAT_ARG_CTX_TAG ,/* Kernel supports '?' at the front of datasec names */ FEAT_BTF_QMARK_DATASEC , +/* Kernel supports extended syscall */ + FEAT_EXTENDED_SYSCALL,
FEAT_BPF_COMMON_ATTRS ?
Quoted 12 lines
tools/lib/bpf/libbpf_internal.h +1
-1__FEAT_CNT, };@@ -757,4 +759,5 @@ int probe_fd(int fd);#define SHA256_DWORD_SIZE SHA256_DIGEST_LENGTH / sizeof(__u64)void libbpf_sha256 (const void *data,size_t len,__u8 out[SHA256_DIGEST_LENGTH ]); +int probe_sys_bpf_ext (void );#endif /* __LIBBPF_LIBBPF_INTERNAL_H */ --2.52.0
Re: [PATCH bpf-next v5 4/9] bpf: Add syscall common attributes support for prog_load
In reply to Leon HwangQuoted 11 lines · Leon Hwang
The log buffer of common attributes would be confusing with the one in 'union
bpf_attr' forBPF_PROG_LOAD.In order to clarify the usage of these two log buffers, they both can be used for logging if:
* They are same, including '
log_buf', 'log_level' and 'log_size'.
* One of them is missing, then another one will be used for logging.If they both have '
log_buf' but they are not same totally, return-EUSERS.
why use this special error code that we don't seem to use in BPF subsystem at all? What's wrong with -EINVAL. This shouldn't be an easy mistake to do, tbh.
Quoted 30 lines
Signed-off-by Leon Hwang <[email protected]>--- include/linux/bpf_verifier.h | 4 +++- kernel/bpf/log.c | 29 ++++++++++++++++++++++++++--- kernel/bpf/syscall.c | 9 ++++++--- 3 files changed, 35 insertions(+), 7 deletions(-)include/linux/bpf_verifier.h +3
-1@@ -637,9 +637,11 @@ struct bpf_log_attr {u32 log_level;struct bpf_attrs *attrs;u32 offsetof_log_true_size; +struct bpf_attrs *attrs_common; }; -int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs); +int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs, +struct bpf_attrs *attrs_common);int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log);#define BPF_MAX_SUBPROGS 256kernel/bpf/log.c +0
-0
kernel/bpf/log.c void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st · line 872
Quoted 9 lines
kernel/bpf/log.c +4
-1@@ -865,23 +865,41 @@ void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st }static int bpf_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,u64 log_buf, -u32 log_size,u32 log_level,int offsetof_log_true_size) +u32 log_size,u32 log_level,int offsetof_log_true_size, +struct bpf_attrs *attrs_common) { +const struct bpf_common_attr *common_attr = attrs_common ? attrs_common->attr :NULL ; +
There is something to be said about naming choices here :) it's easy to get lost in attrs_common being actually bpf_attrs, which contains attr field, which is actually of bpf_common_attr type... It's a bit disorienting. :)
Quoted 23 lines
kernel/bpf/log.c +14
-0memset (log_attr,0 ,sizeof (*log_attr)); log_attr->log_buf = log_buf; log_attr->log_size = log_size; log_attr->log_level = log_level; log_attr->attrs = attrs; log_attr->offsetof_log_true_size = offsetof_log_true_size; + log_attr->attrs_common = attrs_common; + +if (log_buf && common_attr && common_attr->log_buf && + (log_buf != common_attr->log_buf || + log_size != common_attr->log_size || + log_level != common_attr->log_level)) +return -EUSERS ; + +if (!log_buf && common_attr && common_attr->log_buf) { + log_attr->log_buf = common_attr->log_buf; + log_attr->log_size = common_attr->log_size; + log_attr->log_level = common_attr->log_level; + } +return 0 ; }
[...]
Re: [PATCH bpf-next v5 8/9] libbpf: Add common attr support for map_create
In reply to Leon HwangQuoted 36 lines · Leon Hwang
With the previous commit adding common attribute support for
BPF_MAP_CREATE, users can now retrieve detailed error messages when map creation fails via thelog_buffield.Introduce struct
bpf_syscall_common_attr_optswith the following fields:log_buf,log_size,log_level, andlog_true_size.Extend
bpf_map_create_optswith a new fieldcommon_attr_opts, allowing users to capture and inspect log messages on map creation failures.Signed-off-by Leon Hwang <[email protected]>--- tools/lib/bpf/bpf.c | 15 ++++++++++++++- tools/lib/bpf/bpf.h | 17 ++++++++++++++++- 2 files changed, 30 insertions(+), 2 deletions(-)tools/lib/bpf/bpf.c +3
-1@@ -207,6 +207,9 @@ int bpf_map_create(enum bpf_map_type map_type, const struct bpf_map_create_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, excl_prog_hash_size); +const size_t common_attr_sz =sizeof (struct bpf_common_attr ); +struct bpf_syscall_common_attr_opts *common_attr_opts; +struct bpf_common_attr common_attr;union bpf_attr attr;int fd;@@ -240,7 +243,17 @@ int bpf_map_create(enum bpf_map_type map_type, attr.excl_prog_hash =ptr_to_u64 (OPTS_GET (opts, excl_prog_hash,NULL )); attr.excl_prog_hash_size =OPTS_GET (opts, excl_prog_hash_size,0 ); - fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz);
tools/lib/bpf/bpf.c int bpf_map_create(enum bpf_map_type map_type, · line 255
Quoted 10 lines
tools/lib/bpf/bpf.c +10
-0+ common_attr_opts =OPTS_GET (opts, common_attr_opts,NULL ); +if (common_attr_opts &&feat_supported (NULL ,FEAT_EXTENDED_SYSCALL )) { +memset (&common_attr,0 , common_attr_sz); + common_attr.log_buf =ptr_to_u64 (OPTS_GET (common_attr_opts, log_buf,NULL )); + common_attr.log_size =OPTS_GET (common_attr_opts, log_size,0 ); + common_attr.log_level =OPTS_GET (common_attr_opts, log_level,0 ); + fd =sys_bpf_ext_fd (BPF_MAP_CREATE , &attr, attr_sz, &common_attr, common_attr_sz); +OPTS_SET (common_attr_opts, log_true_size, common_attr.log_true_size); + }else { + fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz);
OPTS_SET(log_true_size) to zero here, maybe?
Quoted 13 lines
tools/lib/bpf/bpf.c +1
-0+ }return libbpf_err_errno (fd); }tools/lib/bpf/bpf.h +1
-0@@ -37,6 +37,18 @@ extern "C" { LIBBPF_API int libbpf_set_memlock_rlim(size_t memlock_bytes); +struct bpf_syscall_common_attr_opts {
tools/lib/bpf/bpf.h extern "C" { · line 50
Quoted 10 lines
tools/lib/bpf/bpf.h +10
-0+size_t sz;/* size of this struct for forward/backward compatibility */ + +char *log_buf; +__u32 log_size; +__u32 log_level; +__u32 log_true_size; + + size_t :0; +}; +#define bpf_syscall_common_attr_opts__last_field log_true_size
see below, let's drop this struct and just add these 4 fields directly to bpf_map_create_opts
tools/lib/bpf/bpf.h struct bpf_map_create_opts { · line 73
Quoted 10 lines
tools/lib/bpf/bpf.h +3
-0+struct bpf_map_create_opts {size_t sz;/* size of this struct for forward/backward compatibility */ @@ -57,9 +69,12 @@ struct bpf_map_create_opts {const void *excl_prog_hash;__u32 excl_prog_hash_size; + +struct bpf_syscall_common_attr_opts *common_attr_opts;
maybe let's just add those log_xxx fields here directly? This whole extra bpf_syscall_common_attr_opts pointer and struct seems like a cumbersome API.
Quoted 11 lines
tools/lib/bpf/bpf.h +2
-2+ size_t :0; }; -#define bpf_map_create_opts__last_field excl_prog_hash_size +#define bpf_map_create_opts__last_field common_attr_opts LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name, --2.52.0
Re: [PATCH bpf-next v5 2/9] libbpf: Add support for extended bpf syscall
In reply to Andrii NakryikoQuoted 27 lines · Andrii Nakryiko
Quoted 26 lines · Leon Hwang
To support the extended BPF syscall introduced in the previous commit, introduce the following internal APIs:
* '
sys_bpf_ext()'
* 'sys_bpf_ext_fd()'
They wrap the raw 'syscall()' interface to support passing extended attributes.
* 'probe_sys_bpf_ext()'
Check whether current kernel supports the extended attributes.Signed-off-by Leon Hwang <[email protected]>--- tools/lib/bpf/bpf.c | 34 +++++++++++++++++++++++++++++++++ tools/lib/bpf/features.c | 8 ++++++++ tools/lib/bpf/libbpf_internal.h | 3 +++ 3 files changed, 45 insertions(+)tools/lib/bpf/bpf.c +0
-0@@ -69,6 +69,40 @@ static inline __u64 ptr_to_u64(const void *ptr)return (__u64 ) (unsigned long ) ptr; }
Quoted 7 lines
tools/lib/bpf/bpf.c static inline __u64 ptr_to_u64(const void *ptr) · line 74
Quoted 3 lines
tools/lib/bpf/bpf.c +3
-0+static inline int sys_bpf_ext(enum bpf_cmd cmd, union bpf_attr *attr, + unsigned int size, + struct bpf_common_attr *common_attr,nit: kernel uses consistent attr_common/size_common pattern, but here you are inverting
attr_common->common_attr, let's not?
Ack.
I'll keep the same pattern.
Quoted 23 lines
Quoted 23 lines
tools/lib/bpf/bpf.c +23
-0+ unsigned int size_common) +{ + cmd = common_attr ? (cmd |BPF_COMMON_ATTRS ) : (cmd & ~BPF_COMMON_ATTRS ); +return syscall (__NR_bpf, cmd, attr, size, common_attr, size_common); +} + +static inline int sys_bpf_ext_fd (enum bpf_cmd cmd,union bpf_attr *attr, +unsigned int size, +struct bpf_common_attr *common_attr, +unsigned int size_common) +{ +int fd; + + fd =sys_bpf_ext (cmd, attr, size, common_attr, size_common); +return ensure_good_fd (fd); +} + +int probe_sys_bpf_ext (void ) +{ + const size_t attr_sz = offsetofend(union bpf_attr, prog_token_fd); +union bpf_attr attr; +int fd; +
Quoted 9 lines
tools/lib/bpf/bpf.c static inline __u64 ptr_to_u64(const void *ptr) · line 102
Quoted 5 lines
tools/lib/bpf/bpf.c +5
-0+memset (&attr,0 , attr_sz); + fd =syscall (__NR_bpf,BPF_PROG_LOAD |BPF_COMMON_ATTRS , &attr, attr_sz,NULL , +sizeof (struct bpf_common_attr )); +if (fd >=0 ) +close (fd);hm... close can change errno, this is fragile. If fd >= 0, something is wrong with our detection, just return error right away?
How about capture errno before closing?
err = errno;
if (fd >= 0)
close(fd);
return err = EFAULT;
Then, we can wrap all details in probe_sys_bpf_ext().
Quoted 21 lines
Quoted 21 lines
tools/lib/bpf/bpf.c +3
-0+return errno ==EFAULT ; +} +static inline int sys_bpf (enum bpf_cmd cmd,union bpf_attr *attr,unsigned int size) {tools/lib/bpf/features.c +5
-0@@ -506,6 +506,11 @@ static int probe_kern_arg_ctx_tag(int token_fd)return probe_fd (prog_fd); } +static int probe_kern_extended_syscall (int token_fd) +{ +return probe_sys_bpf_ext (); +} +typedef int (*feature_probe_fn )(int /* token_fd */ );
Quoted 11 lines
tools/lib/bpf/features.c static struct kern_feature_desc { · line 590
Quoted 7 lines
tools/lib/bpf/features.c +2
-0static struct kern_feature_cache feature_cache;@@ -581,6 +586,9 @@ static struct kern_feature_desc { [FEAT_BTF_QMARK_DATASEC] = { "BTF DATASEC names starting from '?'", probe_kern_btf_qmark_datasec, }, + [FEAT_EXTENDED_SYSCALL] = { + "Kernel supports extended syscall", probe_kern_extended_syscall,"extended syscall" is a bit vague... We specifically detect common attrs support, maybe say that?
Ack.
I'll update it to "BPF syscall common attributes support."
Quoted 17 lines
Quoted 9 lines
tools/lib/bpf/features.c +1
-0+ }, };bool feat_supported (struct kern_feature_cache *cache,enum kern_feature_id feat_id)tools/lib/bpf/libbpf_internal.h +0
-0tools/lib/bpf/libbpf_internal.h enum kern_feature_id { · line 396
Quoted 5 lines
tools/lib/bpf/libbpf_internal.h +2
-0@@ -392,6 +392,8 @@ enum kern_feature_id {FEAT_ARG_CTX_TAG ,/* Kernel supports '?' at the front of datasec names */ FEAT_BTF_QMARK_DATASEC , +/* Kernel supports extended syscall */ + FEAT_EXTENDED_SYSCALL,
FEAT_BPF_COMMON_ATTRS?
FEAT_BPF_SYSCALL_COMMON_ATTRS seems more accurate.
Thanks,
Leon
Quoted 12 lines
Quoted 12 lines
tools/lib/bpf/libbpf_internal.h +1
-1__FEAT_CNT, };@@ -757,4 +759,5 @@ int probe_fd(int fd);#define SHA256_DWORD_SIZE SHA256_DIGEST_LENGTH / sizeof(__u64)void libbpf_sha256 (const void *data,size_t len,__u8 out[SHA256_DIGEST_LENGTH ]); +int probe_sys_bpf_ext (void );#endif /* __LIBBPF_LIBBPF_INTERNAL_H */ --2.52.0
Re: [PATCH bpf-next v5 4/9] bpf: Add syscall common attributes support for prog_load
In reply to Andrii NakryikoQuoted 8 lines · Andrii Nakryiko
Quoted 7 lines · Leon Hwang
The log buffer of common attributes would be confusing with the one in 'union
bpf_attr' forBPF_PROG_LOAD.In order to clarify the usage of these two log buffers, they both can be used for logging if:
Quoted 9 lines
Quoted 4 lines
* They are same, including '
log_buf', 'log_level' and 'log_size'.
* One of them is missing, then another one will be used for logging.If they both have '
log_buf' but they are not same totally, return-EUSERS.why use this special error code that we don't seem to use in BPF subsystem at all? What's wrong with
-EINVAL. This shouldn't be an easy mistake to do, tbh.
-EUSERS was suggested by Alexei.
However, I agree with you that it is better to use -EINVAL here.
Quoted 32 lines
Quoted 32 lines
Signed-off-by Leon Hwang <[email protected]>--- include/linux/bpf_verifier.h | 4 +++- kernel/bpf/log.c | 29 ++++++++++++++++++++++++++--- kernel/bpf/syscall.c | 9 ++++++--- 3 files changed, 35 insertions(+), 7 deletions(-)include/linux/bpf_verifier.h +3
-1@@ -637,9 +637,11 @@ struct bpf_log_attr {u32 log_level;struct bpf_attrs *attrs;u32 offsetof_log_true_size; +struct bpf_attrs *attrs_common; }; -int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs); +int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs, +struct bpf_attrs *attrs_common);int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log);#define BPF_MAX_SUBPROGS 256kernel/bpf/log.c +0
-0@@ -865,23 +865,41 @@ void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st }
Quoted 13 lines
kernel/bpf/log.c void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st · line 872
Quoted 7 lines
kernel/bpf/log.c +4
-1static int bpf_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,u64 log_buf, -u32 log_size,u32 log_level,int offsetof_log_true_size) +u32 log_size,u32 log_level,int offsetof_log_true_size, +struct bpf_attrs *attrs_common) { +const struct bpf_common_attr *common_attr = attrs_common ? attrs_common->attr :NULL ; +There is something to be said about naming choices here :) it's easy to get lost in
attrs_commonbeing actuallybpf_attrs, which contains attr field, which is actually ofbpf_common_attrtype... It's a bit disorienting. :)
I see your point about the naming being confusing.
The original intent of 'struct bpf_attrs' was to provide a shared wrapper for both 'union bpf_attr' and 'struct bpf_common_attr'. However, I agree that using 'attrs_common' here makes the layering harder to follow.
If that approach is undesirable, how about introducing a dedicated structure instead, e.g.:
struct bpf_common_attrs {
const struct bpf_common_attr *attr;
bpfptr_t uattr;
u32 size;
};This should make the ownership and intent clearer.
Thanks,
Leon
[...]
Re: [PATCH bpf-next v5 8/9] libbpf: Add common attr support for map_create
In reply to Andrii NakryikoQuoted 36 lines · Andrii Nakryiko
Quoted 35 lines · Leon Hwang
With the previous commit adding common attribute support for
BPF_MAP_CREATE, users can now retrieve detailed error messages when map creation fails via thelog_buffield.Introduce struct
bpf_syscall_common_attr_optswith the following fields:log_buf,log_size,log_level, andlog_true_size.Extend
bpf_map_create_optswith a new fieldcommon_attr_opts, allowing users to capture and inspect log messages on map creation failures.Signed-off-by Leon Hwang <[email protected]>--- tools/lib/bpf/bpf.c | 15 ++++++++++++++- tools/lib/bpf/bpf.h | 17 ++++++++++++++++- 2 files changed, 30 insertions(+), 2 deletions(-)tools/lib/bpf/bpf.c +3
-0@@ -207,6 +207,9 @@ int bpf_map_create(enum bpf_map_type map_type, const struct bpf_map_create_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, excl_prog_hash_size); +const size_t common_attr_sz =sizeof (struct bpf_common_attr ); +struct bpf_syscall_common_attr_opts *common_attr_opts; +struct bpf_common_attr common_attr;union bpf_attr attr;int fd;@@ -240,7 +243,17 @@ int bpf_map_create(enum bpf_map_type map_type, attr.excl_prog_hash =ptr_to_u64 (OPTS_GET (opts, excl_prog_hash,NULL )); attr.excl_prog_hash_size =OPTS_GET (opts, excl_prog_hash_size,0 );
Quoted 14 lines
tools/lib/bpf/bpf.c int bpf_map_create(enum bpf_map_type map_type, · line 255
Quoted 11 lines
tools/lib/bpf/bpf.c +10
-1- fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz); + common_attr_opts =OPTS_GET (opts, common_attr_opts,NULL ); +if (common_attr_opts &&feat_supported (NULL ,FEAT_EXTENDED_SYSCALL )) { +memset (&common_attr,0 , common_attr_sz); + common_attr.log_buf =ptr_to_u64 (OPTS_GET (common_attr_opts, log_buf,NULL )); + common_attr.log_size =OPTS_GET (common_attr_opts, log_size,0 ); + common_attr.log_level =OPTS_GET (common_attr_opts, log_level,0 ); + fd =sys_bpf_ext_fd (BPF_MAP_CREATE , &attr, attr_sz, &common_attr, common_attr_sz); +OPTS_SET (common_attr_opts, log_true_size, common_attr.log_true_size); + }else { + fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz);
OPTS_SET(log_true_size)to zero here, maybe?
Unnecessary, but ok to do it.
Quoted 33 lines
Quoted 13 lines
tools/lib/bpf/bpf.c +1
-0+ }return libbpf_err_errno (fd); }tools/lib/bpf/bpf.h +1
-0@@ -37,6 +37,18 @@ extern "C" { LIBBPF_API int libbpf_set_memlock_rlim(size_t memlock_bytes); +struct bpf_syscall_common_attr_opts {tools/lib/bpf/bpf.h extern "C" { · line 50
Quoted 10 lines
tools/lib/bpf/bpf.h +10
-0+size_t sz;/* size of this struct for forward/backward compatibility */ + +char *log_buf; +__u32 log_size; +__u32 log_level; +__u32 log_true_size; + + size_t :0; +}; +#define bpf_syscall_common_attr_opts__last_field log_true_sizesee below, let's drop this struct and just add these 4 fields directly to
bpf_map_create_optsQuoted 6 lines
tools/lib/bpf/bpf.h +1
-0+struct bpf_map_create_opts {size_t sz;/* size of this struct for forward/backward compatibility */ @@ -57,9 +69,12 @@ struct bpf_map_create_opts {
Quoted 9 lines
tools/lib/bpf/bpf.h struct bpf_map_create_opts { · line 73
Quoted 4 lines
tools/lib/bpf/bpf.h +2
-0const void *excl_prog_hash;__u32 excl_prog_hash_size; + +struct bpf_syscall_common_attr_opts *common_attr_opts;maybe let's just add those
log_xxxfields here directly? This whole extrabpf_syscall_common_attr_optspointer and struct seems like a cumbersome API.
Oops... This struct was suggested by the v3 discussion [1].
This struct was used to report 'log_true_size' without changing 'bpf_map_create()' API.
Links
[1]
https://lore.kernel.org/bpf/CAEf4Bzaw9cboFSf1OXmD84S7pKaeyj=bcQg_diUzGwAkFsjUgg@mail.gmail.com/
Thanks,
Leon
Quoted 11 lines
Quoted 11 lines
tools/lib/bpf/bpf.h +2
-2+ size_t :0; }; -#define bpf_map_create_opts__last_field excl_prog_hash_size +#define bpf_map_create_opts__last_field common_attr_opts LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name, --2.52.0
Re: [PATCH bpf-next v5 2/9] libbpf: Add support for extended bpf syscall
In reply to Leon HwangQuoted 76 lines · Leon Hwang
Quoted 27 lines · Andrii Nakryiko
Quoted 26 lines · Leon Hwang
To support the extended BPF syscall introduced in the previous commit, introduce the following internal APIs:
* '
sys_bpf_ext()'
* 'sys_bpf_ext_fd()'
They wrap the raw 'syscall()' interface to support passing extended attributes.
* 'probe_sys_bpf_ext()'
Check whether current kernel supports the extended attributes.Signed-off-by Leon Hwang <[email protected]>--- tools/lib/bpf/bpf.c | 34 +++++++++++++++++++++++++++++++++ tools/lib/bpf/features.c | 8 ++++++++ tools/lib/bpf/libbpf_internal.h | 3 +++ 3 files changed, 45 insertions(+)tools/lib/bpf/bpf.c +0
-0@@ -69,6 +69,40 @@ static inline __u64 ptr_to_u64(const void *ptr)return (__u64 ) (unsigned long ) ptr; }Quoted 7 lines
tools/lib/bpf/bpf.c static inline __u64 ptr_to_u64(const void *ptr) · line 74
Quoted 3 lines
tools/lib/bpf/bpf.c +3
-0+static inline int sys_bpf_ext(enum bpf_cmd cmd, union bpf_attr *attr, + unsigned int size, + struct bpf_common_attr *common_attr,nit: kernel uses consistent attr_common/size_common pattern, but here you are inverting
attr_common->common_attr, let's not?Ack.
I'll keep the same pattern.
Quoted 23 lines
Quoted 23 lines
tools/lib/bpf/bpf.c +23
-0+ unsigned int size_common) +{ + cmd = common_attr ? (cmd |BPF_COMMON_ATTRS ) : (cmd & ~BPF_COMMON_ATTRS ); +return syscall (__NR_bpf, cmd, attr, size, common_attr, size_common); +} + +static inline int sys_bpf_ext_fd (enum bpf_cmd cmd,union bpf_attr *attr, +unsigned int size, +struct bpf_common_attr *common_attr, +unsigned int size_common) +{ +int fd; + + fd =sys_bpf_ext (cmd, attr, size, common_attr, size_common); +return ensure_good_fd (fd); +} + +int probe_sys_bpf_ext (void ) +{ + const size_t attr_sz = offsetofend(union bpf_attr, prog_token_fd); +union bpf_attr attr; +int fd; +Quoted 9 lines
tools/lib/bpf/bpf.c static inline __u64 ptr_to_u64(const void *ptr) · line 102
Quoted 5 lines
tools/lib/bpf/bpf.c +5
-0+memset (&attr,0 , attr_sz); + fd =syscall (__NR_bpf,BPF_PROG_LOAD |BPF_COMMON_ATTRS , &attr, attr_sz,NULL , +sizeof (struct bpf_common_attr )); +if (fd >=0 ) +close (fd);hm... close can change errno, this is fragile. If fd >= 0, something is wrong with our detection, just return error right away?
Quoted 6 lines
How about capture errno before closing?
err = errno;
if (fd >= 0)close(fd);return err = EFAULT;
not sure what this code is trying to do, but yes, preserving errno is one way to fix an immediate problem.
But fd should really not be >= 0, and if it is -- it's some problem, so I'd return an error in that case to keep us aware, which is why I'm saying I'd just return inside if (fd >= 0) { }
Quoted 6 lines
Then, we can wrap all details in
probe_sys_bpf_ext().Quoted 3 lines
Quoted 3 lines
tools/lib/bpf/bpf.c +3
-0+return errno ==EFAULT ; +} +
[...]
Re: [PATCH bpf-next v5 4/9] bpf: Add syscall common attributes support for prog_load
In reply to Leon HwangQuoted 17 lines · Leon Hwang
Quoted 13 lines · Andrii Nakryiko
Quoted 11 lines · Leon Hwang
The log buffer of common attributes would be confusing with the one in 'union
bpf_attr' forBPF_PROG_LOAD.In order to clarify the usage of these two log buffers, they both can be used for logging if:
* They are same, including '
log_buf', 'log_level' and 'log_size'.
* One of them is missing, then another one will be used for logging.If they both have '
log_buf' but they are not same totally, return-EUSERS.
Quoted 8 lines
Quoted 4 lines
why use this special error code that we don't seem to use in BPF subsystem at all? What's wrong with
-EINVAL. This shouldn't be an easy mistake to do, tbh.
-EUSERSwas suggested by Alexei.However, I agree with you that it is better to use
-EINVALhere.
I don't know what the context was, if you can find it that would be great. Maybe special error makes sense for what Alexei had in mind.
Quoted 56 lines
Quoted 33 lines
Quoted 33 lines
Signed-off-by Leon Hwang <[email protected]>--- include/linux/bpf_verifier.h | 4 +++- kernel/bpf/log.c | 29 ++++++++++++++++++++++++++--- kernel/bpf/syscall.c | 9 ++++++--- 3 files changed, 35 insertions(+), 7 deletions(-)include/linux/bpf_verifier.h +3
-1@@ -637,9 +637,11 @@ struct bpf_log_attr {u32 log_level;struct bpf_attrs *attrs;u32 offsetof_log_true_size; +struct bpf_attrs *attrs_common; }; -int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs); +int bpf_prog_load_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs, +struct bpf_attrs *attrs_common);int bpf_log_attr_finalize (struct bpf_log_attr *log_attr,struct bpf_verifier_log *log);#define BPF_MAX_SUBPROGS 256kernel/bpf/log.c +0
-0@@ -865,23 +865,41 @@ void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st }Quoted 13 lines
kernel/bpf/log.c void print_insn_state(struct bpf_verifier_env *env, const struct bpf_verifier_st · line 872
Quoted 7 lines
kernel/bpf/log.c +4
-1static int bpf_log_attr_init (struct bpf_log_attr *log_attr,struct bpf_attrs *attrs,u64 log_buf, -u32 log_size,u32 log_level,int offsetof_log_true_size) +u32 log_size,u32 log_level,int offsetof_log_true_size, +struct bpf_attrs *attrs_common) { +const struct bpf_common_attr *common_attr = attrs_common ? attrs_common->attr :NULL ; +There is something to be said about naming choices here :) it's easy to get lost in
attrs_commonbeing actuallybpf_attrs, which contains attr field, which is actually ofbpf_common_attrtype... It's a bit disorienting. :)I see your point about the naming being confusing.
The original intent of 'struct
bpf_attrs' was to provide a shared wrapper for both 'unionbpf_attr' and 'structbpf_common_attr'. However, I agree that using 'attrs_common' here makes the layering harder to follow.If that approach is undesirable, how about introducing a dedicated structure instead, e.g.:
Quoted 7 lines
struct bpf_common_attrs { const struct bpf_common_attr *attr; bpfptr_t uattr; u32 size; };This should make the ownership and intent clearer.
I don't know and it's not that important, as it's pretty content. But I'd just try to shorten some names, maybe just "common" for internal helpers would make sense. common->log_buf, seems to work.
Quoted 6 lines
Thanks,
Leon[...]
Re: [PATCH bpf-next v5 8/9] libbpf: Add common attr support for map_create
In reply to Leon HwangQuoted 100 lines · Leon Hwang
Quoted 36 lines · Andrii Nakryiko
Quoted 35 lines · Leon Hwang
With the previous commit adding common attribute support for
BPF_MAP_CREATE, users can now retrieve detailed error messages when map creation fails via thelog_buffield.Introduce struct
bpf_syscall_common_attr_optswith the following fields:log_buf,log_size,log_level, andlog_true_size.Extend
bpf_map_create_optswith a new fieldcommon_attr_opts, allowing users to capture and inspect log messages on map creation failures.Signed-off-by Leon Hwang <[email protected]>--- tools/lib/bpf/bpf.c | 15 ++++++++++++++- tools/lib/bpf/bpf.h | 17 ++++++++++++++++- 2 files changed, 30 insertions(+), 2 deletions(-)tools/lib/bpf/bpf.c +3
-0@@ -207,6 +207,9 @@ int bpf_map_create(enum bpf_map_type map_type, const struct bpf_map_create_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, excl_prog_hash_size); +const size_t common_attr_sz =sizeof (struct bpf_common_attr ); +struct bpf_syscall_common_attr_opts *common_attr_opts; +struct bpf_common_attr common_attr;union bpf_attr attr;int fd;@@ -240,7 +243,17 @@ int bpf_map_create(enum bpf_map_type map_type, attr.excl_prog_hash =ptr_to_u64 (OPTS_GET (opts, excl_prog_hash,NULL )); attr.excl_prog_hash_size =OPTS_GET (opts, excl_prog_hash_size,0 );Quoted 14 lines
tools/lib/bpf/bpf.c int bpf_map_create(enum bpf_map_type map_type, · line 255
Quoted 11 lines
tools/lib/bpf/bpf.c +10
-1- fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz); + common_attr_opts =OPTS_GET (opts, common_attr_opts,NULL ); +if (common_attr_opts &&feat_supported (NULL ,FEAT_EXTENDED_SYSCALL )) { +memset (&common_attr,0 , common_attr_sz); + common_attr.log_buf =ptr_to_u64 (OPTS_GET (common_attr_opts, log_buf,NULL )); + common_attr.log_size =OPTS_GET (common_attr_opts, log_size,0 ); + common_attr.log_level =OPTS_GET (common_attr_opts, log_level,0 ); + fd =sys_bpf_ext_fd (BPF_MAP_CREATE , &attr, attr_sz, &common_attr, common_attr_sz); +OPTS_SET (common_attr_opts, log_true_size, common_attr.log_true_size); + }else { + fd =sys_bpf_fd (BPF_MAP_CREATE , &attr, attr_sz);
OPTS_SET(log_true_size)to zero here, maybe?Unnecessary, but ok to do it.
Quoted 33 lines
Quoted 13 lines
tools/lib/bpf/bpf.c +1
-0+ }return libbpf_err_errno (fd); }tools/lib/bpf/bpf.h +1
-0@@ -37,6 +37,18 @@ extern "C" { LIBBPF_API int libbpf_set_memlock_rlim(size_t memlock_bytes); +struct bpf_syscall_common_attr_opts {tools/lib/bpf/bpf.h extern "C" { · line 50
Quoted 10 lines
tools/lib/bpf/bpf.h +10
-0+size_t sz;/* size of this struct for forward/backward compatibility */ + +char *log_buf; +__u32 log_size; +__u32 log_level; +__u32 log_true_size; + + size_t :0; +}; +#define bpf_syscall_common_attr_opts__last_field log_true_sizesee below, let's drop this struct and just add these 4 fields directly to
bpf_map_create_optsQuoted 6 lines
tools/lib/bpf/bpf.h +1
-0+struct bpf_map_create_opts {size_t sz;/* size of this struct for forward/backward compatibility */ @@ -57,9 +69,12 @@ struct bpf_map_create_opts {Quoted 9 lines
tools/lib/bpf/bpf.h struct bpf_map_create_opts { · line 73
Quoted 4 lines
tools/lib/bpf/bpf.h +2
-0const void *excl_prog_hash;__u32 excl_prog_hash_size; + +struct bpf_syscall_common_attr_opts *common_attr_opts;maybe let's just add those
log_xxxfields here directly? This whole extrabpf_syscall_common_attr_optspointer and struct seems like a cumbersome API.
Quoted 5 lines
Oops... This struct was suggested by the v3 discussion [1].
This struct was used to report '
log_true_size' without changing 'bpf_map_create()' API.
Ah, I already forgot. log_true_size being an output parameter here... Sigh. I don't like the verboseness of bpf_syscall_common_attr_opts and "common_attr_opts" and all that stuff...
What if we make it struct bpf_log_opts {} and keep it log-specific?
Quoted 19 lines
Links
[1]
https://lore.kernel.org/bpf/CAEf4Bzaw9cboFSf1OXmD84S7pKaeyj=bcQg_diUzGwAkFsjUgg@mail.gmail.com/Thanks,
LeonQuoted 11 lines
Quoted 11 lines
tools/lib/bpf/bpf.h +2
-2+ size_t :0; }; -#define bpf_map_create_opts__last_field excl_prog_hash_size +#define bpf_map_create_opts__last_field common_attr_opts LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name, --2.52.0