Add support to emit verifier warnings
Subsystems (6)
Kumar Kartikeya Dwivedi
maintainerv2 cover Add support to emit verifier warnings
Currently, there are only two ways of communicating information to the user when a program is verified, success or failure with a verbose verifier log. Some information is meant to be more discretionary, e.g. warning about use of kfuncs that are deprecated, and may be removed in future kernel releases.
An example is shown below.
$ ./test_progs -t kfunc_implicit_args/test_kfunc_implicit_arg_legacy_impl -v ... 2: (b7) r3 = 0 ; R3=0 3: (85) call bpf_kfunc_implicit_arg_legacy_impl#114683 kfunc_implicit_args.c:40 (insn #3) uses deprecated kfunc bpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfunc bpf_kfunc_implicit_arg_legacy() instead. For older kernels, choose the correct kfunc using bpf_ksym_exists(). 4: R0=scalar() 4: (95) exit ...
With verbose logging, the deprecation warnings are printed inline in the verifier log. When no log level is chosen, the warning is printed in a block, as follows:
libbpf: prog 'test_kfunc_implicit_arg_legacy_impl': -- BEGIN PROG LOAD WARNINGS -- kfunc_implicit_args.c:40 (insn #3) uses deprecated kfunc bpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfunc bpf_kfunc_implicit_arg_legacy() instead.
For older kernels, choose the correct kfunc using bpf_ksym_exists().
-- END PROG LOAD WARNINGS --
The set was tested with and without kernel changes applied, so that we could conclude that libbpf feature detection works correctly and backwards compatibility is not violated on older kernels. Still, it would be helpful if other reviewers could recheck all assumptions and make sure I didn't miss any corner case.
One case I need more opinions on: log_buf.c passes log_buf with log_level=0 and expects it to be empty, with newer libbpf that won't be true on older kernels anymore. I don't think this is something that needs to be addressed, but explicitly noting it here since I saw that test fail when testing new libbpf on older kernels. The kernel will now populate such log_buf will "processed N insns ..." log since on failure libbpf will retry with log_level=1, so unless applications are relying on log_buf to be empty for correctness it should be ok.
Changelog:
----------
v1 -> v2
v1: https://lore.kernel.org/bpf/[email protected]
* Use BTF tags and kfunc flags for deprecation warnings.
* Reuse verifier log to deliver log messages. (Alexei)
* Make the mechanism declarative without kfunc lists in verifier.c.
Kumar Kartikeya Dwivedi (6):
bpf: Add support for verifier warning messages
bpf: Extract bpf_get_linfo_file_line
bpf: Make find_linfo widely available
bpf: Use KF_DEPRECATED to emit verifier warnings
bpf: Add __bpf_kfunc_replacement() annotation
libbpf: Flush verifier warning messages by default
include/linux/bpf.h | 3 + include/linux/bpf_verifier.h | 1 + include/linux/btf.h | 6 ++ include/linux/compiler_types.h | 14 +++- kernel/bpf/core.c | 66 ++++++++++++++++--- kernel/bpf/helpers.c | 24 ++++--- kernel/bpf/log.c | 49 +------------- kernel/bpf/verifier.c | 60 ++++++++++++++++- tools/lib/bpf/bpf.c | 5 +- tools/lib/bpf/features.c | 56 ++++++++++++++++ tools/lib/bpf/libbpf.c | 20 ++++-- tools/lib/bpf/libbpf_internal.h | 2 + .../selftests/bpf/test_kmods/bpf_testmod.c | 3 +- 13 files changed, 233 insertions(+), 76 deletions(-)
base-commit: 5c662b1c1789f51f79ee9c648681abc8410dfa81
Signature · 1 line
2.52.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 1/6 bpf: Add support for verifier warning messages
Relax conditions such that we allow log_level = 0 to have an associated log buffer. Add a warn() macro that emits messages to log buffer without any restrictions, aggregate the warnings emitted, and then use it to decide whether we reset the log or not.
--- include/linux/bpf_verifier.h | 1 + kernel/bpf/log.c | 6 ++---- kernel/bpf/verifier.c | 17 +++++++++++++++-- 3 files changed, 18 insertions(+), 6 deletions(-)
include/linux/bpf_verifier.h +1 -0
@@ -795,6 +795,7 @@ struct bpf_verifier_env {bool bypass_spec_v4;bool seen_direct_write;bool seen_exception; +bool warnings;struct bpf_insn_aux_data *insn_aux_data;/* array of per-insn state */ const struct bpf_line_info *prev_linfo;struct bpf_verifier_log log;
kernel/bpf/log.c +2 -4
@@ -18,13 +18,11 @@ static bool bpf_verifier_log_attr_valid(const struct bpf_verifier_log *log)/* ubuf and len_total should both be specified (or not) together */ if (!!log->ubuf != !!log->len_total)return false; -/* log buf without log_level is meaningless */ -if (log->ubuf && log->level ==0 ) -return false;if (log->level & ~BPF_LOG_MASK )return false;if (log->len_total >UINT_MAX >>2 )return false; +/* log->ubuf may be set for log->level = 0 to get warning messages. */ return true; }
@@ -229,7 +227,7 @@ int bpf_vlog_finalize(struct bpf_verifier_log *log, u32 *log_size_actual)int err; *log_size_actual =0 ; -if (!log || log->level ==0 || log->level ==BPF_LOG_KERNEL ) +if (!log || (!log->ubuf && log->level ==0 ) || log->level ==BPF_LOG_KERNEL )return 0 ;if (!log->ubuf)
kernel/bpf/verifier.c +15 -2
@@ -47,6 +47,7 @@ static const struct bpf_verifier_ops * const bpf_verifier_ops[] = {enum bpf_features {BPF_FEAT_RDONLY_CAST_TO_VOID =0 ,BPF_FEAT_STREAMS =1 , +BPF_FEAT_VERIFIER_WARNINGS =2 , __MAX_BPF_FEAT, };
@@ -368,6 +369,17 @@ __printf(2, 3) static void verbose(void *private_data, const char *fmt, ...)va_end (args); } +__printf(2, 3) static void warn(void *private_data, const char *fmt, ...) +{ +struct bpf_verifier_env *env = private_data; +va_list args; + +va_start (args, fmt); +bpf_verifier_vlog (&env->log, fmt, args); +va_end (args); + env->warnings = true; +} + static void verbose_invalid_scalar(struct bpf_verifier_env *env, struct bpf_reg_state *reg, struct bpf_retval_range range, const char *ctx,
@@ -2128,7 +2140,8 @@ static int pop_stack(struct bpf_verifier_env *env, int *prev_insn_idx,if (err)return err; } -if (pop_log) +/* Preserve warning-only output across branch explorations. */ +if (pop_log && !(env->warnings && env->log.level ==0 ))bpf_vlog_reset (&env->log, head->log_pos);if (insn_idx) *insn_idx = head->insn_idx;
@@ -25211,7 +25224,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) ret =do_check (env); out: -if (!ret && pop_log) +if (!ret && pop_log && !env->warnings)bpf_vlog_reset (&env->log,0 );free_states (env);return ret;
Signature · 1 line
2.52.0
Re: [PATCH bpf-next v2 1/6] bpf: Add support for verifier warning messages
Quoted 84 lines · Kumar Kartikeya Dwivedi
Relax conditions such that we allow
log_level= 0 to have an associated log buffer. Add awarn()macro that emits messages to log buffer without any restrictions, aggregate the warnings emitted, and then use it to decide whether we reset the log or not.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- include/linux/bpf_verifier.h | 1 + kernel/bpf/log.c | 6 ++---- kernel/bpf/verifier.c | 17 +++++++++++++++-- 3 files changed, 18 insertions(+), 6 deletions(-)include/linux/bpf_verifier.h +1
-0@@ -795,6 +795,7 @@ struct bpf_verifier_env {bool bypass_spec_v4;bool seen_direct_write;bool seen_exception; +bool warnings;struct bpf_insn_aux_data *insn_aux_data;/* array of per-insn state */ const struct bpf_line_info *prev_linfo;struct bpf_verifier_log log;kernel/bpf/log.c +2
-4@@ -18,13 +18,11 @@ static bool bpf_verifier_log_attr_valid(const struct bpf_verifier_log *log)/* ubuf and len_total should both be specified (or not) together */ if (!!log->ubuf != !!log->len_total)return false; -/* log buf without log_level is meaningless */ -if (log->ubuf && log->level ==0 ) -return false;if (log->level & ~BPF_LOG_MASK )return false;if (log->len_total >UINT_MAX >>2 )return false; +/* log->ubuf may be set for log->level = 0 to get warning messages. */ return true; }@@ -229,7 +227,7 @@ int bpf_vlog_finalize(struct bpf_verifier_log *log, u32 *log_size_actual)int err; *log_size_actual =0 ; -if (!log || log->level ==0 || log->level ==BPF_LOG_KERNEL ) +if (!log || (!log->ubuf && log->level ==0 ) || log->level ==BPF_LOG_KERNEL )return 0 ;if (!log->ubuf)kernel/bpf/verifier.c +12
-0@@ -47,6 +47,7 @@ static const struct bpf_verifier_ops * const bpf_verifier_ops[] = {enum bpf_features {BPF_FEAT_RDONLY_CAST_TO_VOID =0 ,BPF_FEAT_STREAMS =1 , +BPF_FEAT_VERIFIER_WARNINGS =2 , __MAX_BPF_FEAT, };@@ -368,6 +369,17 @@ __printf(2, 3) static void verbose(void *private_data, const char *fmt, ...)va_end (args); } +__printf(2, 3) static void warn(void *private_data, const char *fmt, ...) +{ +struct bpf_verifier_env *env = private_data; +va_list args; + +va_start (args, fmt); +bpf_verifier_vlog (&env->log, fmt, args); +va_end (args); + env->warnings = true; +} + static void verbose_invalid_scalar(struct bpf_verifier_env *env, struct bpf_reg_state *reg, struct bpf_retval_range range, const char *ctx,
kernel/bpf/verifier.c static int pop_stack(struct bpf_verifier_env *env, int *prev_insn_idx, · line 2144
Quoted 6 lines
kernel/bpf/verifier.c +2
-1@@ -2128,7 +2140,8 @@ static int pop_stack(struct bpf_verifier_env *env, int *prev_insn_idx,if (err)return err; } -if (pop_log) +/* Preserve warning-only output across branch explorations. */ +if (pop_log && !(env->warnings && env->log.level ==0 ))
IIUC, this can drop some warnings when log_level > 0, which seems to be a reasonable
design trade-off to me. Might it be worth mentioning this in the commit message?
Quoted 16 lines
kernel/bpf/verifier.c +1
-2bpf_vlog_reset (&env->log, head->log_pos);if (insn_idx) *insn_idx = head->insn_idx;@@ -25211,7 +25224,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) ret =do_check (env); out: -if (!ret && pop_log) +if (!ret && pop_log && !env->warnings)bpf_vlog_reset (&env->log,0 );free_states (env);return ret; --2.52.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 2/6 bpf: Extract bpf_get_linfo_file_line
In reply to Kumar Kartikeya DwivediExtract bpf_get_linfo_file_line as its own function so that the logic to obtain the file, line, and line number for a given program can be shared in subsequent patches.
--- include/linux/bpf.h | 2 ++ kernel/bpf/core.c | 29 +++++++++++++++++++++-------- 2 files changed, 23 insertions(+), 8 deletions(-)
include/linux/bpf.h +2 -0
@@ -3945,6 +3945,8 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)return prog->aux->func_idx !=0 ; } +void bpf_get_linfo_file_line (struct btf *btf,const struct bpf_line_info *linfo, +const char **filep,const char **linep,int *nump);int bpf_prog_get_file_line (struct bpf_prog *prog,unsigned long ip,const char **filep,const char **linep,int *nump);struct bpf_prog *bpf_prog_find_from_stack (void );
kernel/bpf/core.c +21 -8
@@ -3315,6 +3315,26 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(xdp_bulk_tx);#ifdef CONFIG_BPF_SYSCALL +void bpf_get_linfo_file_line (struct btf *btf,const struct bpf_line_info *linfo, +const char **filep,const char **linep,int *nump) +{ +/* Get base component of the file path. */ +if (filep) { + *filep =btf_name_by_offset (btf, linfo->file_name_off); + *filep =kbasename (*filep); + } + +/* Obtain the source line, and strip whitespace in prefix. */ +if (linep) { + *linep =btf_name_by_offset (btf, linfo->line_off); +while (isspace (**linep)) + *linep +=1 ; + } + +if (nump) + *nump =BPF_LINE_INFO_LINE_NUM (linfo->line_col); +} +int bpf_prog_get_file_line (struct bpf_prog *prog,unsigned long ip,const char **filep,const char **linep,int *nump) {
@@ -3349,14 +3369,7 @@ int bpf_prog_get_file_line(struct bpf_prog *prog, unsigned long ip, const char *if (idx ==-1 )return -ENOENT ; -/* Get base component of the file path. */ - *filep =btf_name_by_offset (btf, linfo[idx].file_name_off); - *filep =kbasename (*filep); -/* Obtain the source line, and strip whitespace in prefix. */ - *linep =btf_name_by_offset (btf, linfo[idx].line_off); -while (isspace (**linep)) - *linep +=1 ; - *nump =BPF_LINE_INFO_LINE_NUM (linfo[idx].line_col); +bpf_get_linfo_file_line (btf, &linfo[idx], filep, linep, nump);return 0 ; }
Signature · 1 line
2.52.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 3/6 bpf: Make find_linfo widely available
In reply to Kumar Kartikeya DwivediMove find_linfo() as bpf_find_linfo() into core.c to allow for its use in the verifier in subsequent patches.
--- include/linux/bpf.h | 1 + kernel/bpf/core.c | 37 +++++++++++++++++++++++++++++++++++++ kernel/bpf/log.c | 43 +------------------------------------------ 3 files changed, 39 insertions(+), 42 deletions(-)
include/linux/bpf.h +1 -0
@@ -3945,6 +3945,7 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)return prog->aux->func_idx !=0 ; } +const struct bpf_line_info *bpf_find_linfo (const struct bpf_prog *prog,u32 insn_off);void bpf_get_linfo_file_line (struct btf *btf,const struct bpf_line_info *linfo,const char **filep,const char **linep,int *nump); int bpf_prog_get_file_line(struct bpf_prog *prog, unsigned long ip, const char **filep,
kernel/bpf/core.c +37 -0
@@ -3335,6 +3335,43 @@ void bpf_get_linfo_file_line(struct btf *btf, const struct bpf_line_info *linfo, *nump =BPF_LINE_INFO_LINE_NUM (linfo->line_col); } +const struct bpf_line_info *bpf_find_linfo (const struct bpf_prog *prog,u32 insn_off) +{ +const struct bpf_line_info *linfo; +u32 nr_linfo; +int l, r, m; + + nr_linfo = prog->aux->nr_linfo; +if (!nr_linfo || insn_off >= prog->len) +return NULL ; + + linfo = prog->aux->linfo; +/* Loop invariant: linfo[l].insn_off <= insns_off. +* linfo[0].insn_off == 0 which always satisfies above condition. +* Binary search is searching for rightmost linfo entry that satisfies +* the above invariant, giving us the desired record that covers given +* instruction offset. +*/ + l =0 ; + r = nr_linfo -1 ; +while (l < r) { +/* (r - l + 1) / 2 means we break a tie to the right, so if: +* l=1, r=2, linfo[l].insn_off <= insn_off, linfo[r].insn_off > insn_off, +* then m=2, we see that linfo[m].insn_off > insn_off, and so +* r becomes 1 and we exit the loop with correct l==1. +* If the tie was broken to the left, m=1 would end us up in +* an endless loop where l and m stay at 1 and r stays at 2. +*/ + m = l + (r - l +1 ) /2 ; +if (linfo[m].insn_off <= insn_off) + l = m; +else + r = m -1 ; + } + +return &linfo[l]; +} +int bpf_prog_get_file_line (struct bpf_prog *prog,unsigned long ip,const char **filep,const char **linep,int *nump) {
kernel/bpf/log.c +1 -42
55 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH bpf-next v2 3/6] bpf: Make find_linfo widely available
Quoted 32 lines · Kumar Kartikeya Dwivedi
Move
find_linfo()asbpf_find_linfo()intocore.cto allow for its use in the verifier in subsequent patches.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- include/linux/bpf.h | 1 + kernel/bpf/core.c | 37 +++++++++++++++++++++++++++++++++++++ kernel/bpf/log.c | 43 +------------------------------------------ 3 files changed, 39 insertions(+), 42 deletions(-)include/linux/bpf.h +1
-0@@ -3945,6 +3945,7 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)return prog->aux->func_idx !=0 ; } +const struct bpf_line_info *bpf_find_linfo (const struct bpf_prog *prog,u32 insn_off);void bpf_get_linfo_file_line (struct btf *btf,const struct bpf_line_info *linfo,const char **filep,const char **linep,int *nump); int bpf_prog_get_file_line(struct bpf_prog *prog, unsigned long ip, const char **filep,kernel/bpf/core.c +2
-0@@ -3335,6 +3335,43 @@ void bpf_get_linfo_file_line(struct btf *btf, const struct bpf_line_info *linfo, *nump =BPF_LINE_INFO_LINE_NUM (linfo->line_col); } +const struct bpf_line_info *bpf_find_linfo (const struct bpf_prog *prog,u32 insn_off) +{
kernel/bpf/core.c void bpf_get_linfo_file_line(struct btf *btf, const struct bpf_line_info *linfo, · line 3349
Quoted 10 lines
kernel/bpf/core.c +10
-0+const struct bpf_line_info *linfo; +u32 nr_linfo; +int l, r, m; + + nr_linfo = prog->aux->nr_linfo; +if (!nr_linfo || insn_off >= prog->len) +return NULL ; + + linfo = prog->aux->linfo; +/* Loop invariant: linfo[l].insn_off <= insns_off.
If you are touching these lines, it's worth updating to kernel style comments.
other than that the change looks straightforward, just moving a function from one place to another + minor change of the arguments and rename.
Quoted 89 lines
kernel/bpf/core.c +25
-0+* linfo[0].insn_off == 0 which always satisfies above condition. +* Binary search is searching for rightmost linfo entry that satisfies +* the above invariant, giving us the desired record that covers given +* instruction offset. +*/ + l =0 ; + r = nr_linfo -1 ; +while (l < r) { +/* (r - l + 1) / 2 means we break a tie to the right, so if: +* l=1, r=2, linfo[l].insn_off <= insn_off, linfo[r].insn_off > insn_off, +* then m=2, we see that linfo[m].insn_off > insn_off, and so +* r becomes 1 and we exit the loop with correct l==1. +* If the tie was broken to the left, m=1 would end us up in +* an endless loop where l and m stay at 1 and r stays at 2. +*/ + m = l + (r - l +1 ) /2 ; +if (linfo[m].insn_off <= insn_off) + l = m; +else + r = m -1 ; + } + +return &linfo[l]; +} +int bpf_prog_get_file_line (struct bpf_prog *prog,unsigned long ip,const char **filep,const char **linep,int *nump) {kernel/bpf/log.c +1
-42@@ -327,47 +327,6 @@ __printf(2, 3) void bpf_log(struct bpf_verifier_log *log, }EXPORT_SYMBOL_GPL (bpf_log); -static const struct bpf_line_info * -find_linfo (const struct bpf_verifier_env *env,u32 insn_off) -{ -const struct bpf_line_info *linfo; -const struct bpf_prog *prog; -u32 nr_linfo; -int l, r, m; - - prog = env->prog; - nr_linfo = prog->aux->nr_linfo; - -if (!nr_linfo || insn_off >= prog->len) -return NULL ; - - linfo = prog->aux->linfo; -/* Loop invariant: linfo[l].insn_off <= insns_off. -* linfo[0].insn_off == 0 which always satisfies above condition. -* Binary search is searching for rightmost linfo entry that satisfies -* the above invariant, giving us the desired record that covers given -* instruction offset. -*/ - l =0 ; - r = nr_linfo -1 ; -while (l < r) { -/* (r - l + 1) / 2 means we break a tie to the right, so if: -* l=1, r=2, linfo[l].insn_off <= insn_off, linfo[r].insn_off > insn_off, -* then m=2, we see that linfo[m].insn_off > insn_off, and so -* r becomes 1 and we exit the loop with correct l==1. -* If the tie was broken to the left, m=1 would end us up in -* an endless loop where l and m stay at 1 and r stays at 2. -*/ - m = l + (r - l +1 ) /2 ; -if (linfo[m].insn_off <= insn_off) - l = m; -else - r = m -1 ; - } - -return &linfo[l]; -} -static const char *ltrim (const char *s) {while (isspace (*s))@@ -388,7 +347,7 @@ __printf(3, 4) void verbose_linfo(struct bpf_verifier_env *env,return ; prev_linfo = env->prev_linfo; - linfo =find_linfo (env, insn_off); + linfo =bpf_find_linfo (env->prog, insn_off);if (!linfo || linfo == prev_linfo)return ;
Kumar Kartikeya Dwivedi
maintainerPATCH v2 4/6 bpf: Use KF_DEPRECATED to emit verifier warnings
In reply to Kumar Kartikeya DwivediAs documented in kfuncs.rst, introduce a KF_DEPRECATED annotation for kfuncs and apply it to all existing impl-suffixed kfuncs that are preserved for backwards compatibility for a few kernel releases, after we introduced versions with KF_IMPLICIT_ARGS.
Use the kfunc flag to decide whether to emit warnings to the verifier log or not. Be helpful and point out the file:line in addition to the instruction when the information is available while emitting the deprecation warning.
--- include/linux/btf.h | 1 + kernel/bpf/helpers.c | 16 ++++----- kernel/bpf/verifier.c | 34 +++++++++++++++++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 2 +- 4 files changed, 44 insertions(+), 9 deletions(-)
include/linux/btf.h +1 -0
@@ -79,6 +79,7 @@#define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ #define KF_IMPLICIT_ARGS (1 << 16)/* kfunc has implicit arguments supplied by the verifier */ +#define KF_DEPRECATED (1 << 17)/* kfunc is deprecated and should warn when used */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/helpers.c +8 -8
@@ -4685,19 +4685,19 @@ BTF_KFUNCS_START(generic_btf_ids) BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE)#endif BTF_ID_FLAGS(func, bpf_obj_new, KF_ACQUIRE | KF_RET_NULL | KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL | KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_percpu_obj_new, KF_ACQUIRE | KF_RET_NULL | KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_percpu_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_percpu_obj_new_impl, KF_ACQUIRE | KF_RET_NULL | KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_obj_drop, KF_RELEASE | KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE | KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_percpu_obj_drop, KF_RELEASE | KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_percpu_obj_drop_impl, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_percpu_obj_drop_impl, KF_RELEASE | KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_refcount_acquire, KF_ACQUIRE | KF_RET_NULL | KF_RCU | KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_refcount_acquire_impl, KF_ACQUIRE | KF_RET_NULL | KF_RCU) +BTF_ID_FLAGS(func, bpf_refcount_acquire_impl, KF_ACQUIRE | KF_RET_NULL | KF_RCU | KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_list_push_front, KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_list_push_front_impl) +BTF_ID_FLAGS(func, bpf_list_push_front_impl, KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_list_push_back, KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_list_push_back_impl) +BTF_ID_FLAGS(func, bpf_list_push_back_impl, KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)BTF_ID_FLAGS (func, bpf_list_front,KF_RET_NULL )
@@ -4706,7 +4706,7 @@ BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_rbtree_remove, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_rbtree_add, KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_rbtree_add_impl) +BTF_ID_FLAGS(func, bpf_rbtree_add_impl, KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_rbtree_first, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_rbtree_root, KF_RET_NULL)BTF_ID_FLAGS (func, bpf_rbtree_left,KF_RET_NULL )
kernel/bpf/verifier.c +34 -0
@@ -3229,6 +3229,7 @@ struct bpf_kfunc_desc {u32 func_id;s32 imm;u16 offset; +bool warned_deprecated;unsigned long addr; };
@@ -12426,6 +12427,11 @@ static bool is_kfunc_destructive(struct bpf_kfunc_call_arg_meta *meta)return meta->kfunc_flags &KF_DESTRUCTIVE ; } +static bool is_kfunc_deprecated (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_DEPRECATED ; +} +static bool is_kfunc_rcu (struct bpf_kfunc_call_arg_meta *meta) {return meta->kfunc_flags &KF_RCU ;
@@ -14583,6 +14589,32 @@ static int check_special_kfunc(struct bpf_verifier_env *env, struct bpf_kfunc_castatic int check_return_code (struct bpf_verifier_env *env,int regno,const char *reg_name); +static void warn_for_deprecated_kfuncs (struct bpf_verifier_env *env, +struct bpf_kfunc_call_arg_meta *meta, +int insn_idx,s16 offset) +{ +const struct bpf_line_info *linfo; +struct bpf_kfunc_desc *desc; +const char *file; +int line_num; + + desc =find_kfunc_desc (env->prog, meta->func_id, offset); +if (!desc || desc->warned_deprecated || !is_kfunc_deprecated (meta) || !env->prog->aux->btf) +return ; + + linfo =bpf_find_linfo (env->prog, insn_idx); +if (linfo) { +bpf_get_linfo_file_line (env->prog->aux->btf, linfo, &file,NULL , &line_num); +warn (env,"%s:%d (insn #%d) uses deprecated kfunc %s(), which will be removed.\n" , + file, line_num, insn_idx, meta->func_name); + }else { +warn (env,"(insn #%d) uses deprecated kfunc %s(), which will be removed.\n" , + insn_idx, meta->func_name); + } + + desc->warned_deprecated = true; +} +static int check_kfunc_call (struct bpf_verifier_env *env,struct bpf_insn *insn,int *insn_idx_p) {
@@ -14612,6 +14644,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, insn_aux->is_iter_next =is_iter_next_kfunc (&meta); +warn_for_deprecated_kfuncs (env, &meta, insn_idx, insn->off); +if (!insn->off && (insn->imm == special_kfunc_list[KF_bpf_res_spin_lock] || insn->imm == special_kfunc_list[KF_bpf_res_spin_lock_irqsave])) {
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +1 -1
@@ -1326,7 +1326,7 @@ BTF_ID_FLAGS(func, bpf_kfunc_multi_st_ops_test_1) BTF_ID_FLAGS(func, bpf_kfunc_multi_st_ops_test_1_assoc, KF_IMPLICIT_ARGS) BTF_ID_FLAGS(func, bpf_kfunc_implicit_arg, KF_IMPLICIT_ARGS) BTF_ID_FLAGS(func, bpf_kfunc_implicit_arg_legacy, KF_IMPLICIT_ARGS) -BTF_ID_FLAGS(func, bpf_kfunc_implicit_arg_legacy_impl) +BTF_ID_FLAGS(func, bpf_kfunc_implicit_arg_legacy_impl, KF_DEPRECATED) BTF_ID_FLAGS(func, bpf_kfunc_trigger_ctx_check)BTF_KFUNCS_END (bpf_testmod_check_kfunc_ids)
Signature · 1 line
2.52.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 5/6 bpf: Add __bpf_kfunc_replacement() annotation
In reply to Kumar Kartikeya DwivediAdd an annotation to emit more contextual messages per deprecated kfunc about which replacement kfunc can be used in its place. The macro itself takes the replacement kfunc as the argument.
This uses BTF declaration tags, and is only supplied as a warning to the user when the kernel is compiled using LLVM, as GCC lacks equivalent support. That said, in all cases KF_DEPRECATED does produce a warning, so this strictly leads to an improved message where possible.
--- include/linux/btf.h | 5 +++++ include/linux/compiler_types.h | 14 ++++++++++++-- kernel/bpf/helpers.c | 8 ++++++++ kernel/bpf/verifier.c | 9 +++++++++ .../testing/selftests/bpf/test_kmods/bpf_testmod.c | 1 + 5 files changed, 35 insertions(+), 2 deletions(-)
include/linux/btf.h +5 -0
@@ -89,6 +89,11 @@*/ #define __bpf_kfunc __used __retain __noclone noinline +#define BPF_KFUNC_DECL_TAG_REPLACEMENT "bpf:kfunc:replacement:" + +#define __bpf_kfunc_replacement (replacement) \ + BTF_DECL_TAG(BPF_KFUNC_DECL_TAG_REPLACEMENT __stringify(replacement)) +#define __bpf_kfunc_start_defs () \ __diag_push(); \ __diag_ignore_all("-Wmissing-declarations", \
include/linux/compiler_types.h +12 -2
@@ -35,10 +35,20 @@* see https://github.com/rust-lang/rust-bindgen/issues/2244. */ #if defined(CONFIG_DEBUG_INFO_BTF ) && defined(CONFIG_PAHOLE_HAS_BTF_TAG ) && \ -__has_attribute (btf_type_tag) && !defined(__BINDGEN__) -# define BTF_TYPE_TAG (value) __attribute__((btf_type_tag(#value))) + !defined(__BINDGEN__) +# if __has_attribute (btf_type_tag) +# define BTF_TYPE_TAG (value) __attribute__((btf_type_tag(#value))) +# else +# define BTF_TYPE_TAG (value)/* nothing */ +# endif +# if __has_attribute (btf_decl_tag) +# define BTF_DECL_TAG (value) __attribute__((btf_decl_tag(value))) +# else +# define BTF_DECL_TAG (value)/* nothing */ +# endif #else # define BTF_TYPE_TAG (value)/* nothing */ +# define BTF_DECL_TAG (value)/* nothing */ #endif #include <linux/compiler-context-analysis.h>
kernel/bpf/helpers.c +8 -0
@@ -2328,6 +2328,7 @@ __bpf_kfunc void *bpf_obj_new(u64 local_type_id__k, struct btf_struct_meta *metareturn p; } +__bpf_kfunc_replacement (bpf_obj_new) __bpf_kfunc void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign) {return bpf_obj_new (local_type_id__k, meta__ign);
@@ -2352,6 +2353,7 @@ __bpf_kfunc void *bpf_percpu_obj_new(u64 local_type_id__k, struct btf_struct_metreturn bpf_mem_alloc (&bpf_global_percpu_ma, size); } +__bpf_kfunc_replacement (bpf_percpu_obj_new) __bpf_kfunc void *bpf_percpu_obj_new_impl(u64 local_type_id__k, void *meta__ign) {return bpf_percpu_obj_new (local_type_id__k, meta__ign);
@@ -2395,6 +2397,7 @@ __bpf_kfunc void bpf_obj_drop(void *p__alloc, struct btf_struct_meta *meta)__bpf_obj_drop_impl (p, meta ? meta->record :NULL , false); } +__bpf_kfunc_replacement (bpf_obj_drop) __bpf_kfunc void bpf_obj_drop_impl(void *p__alloc, void *meta__ign) {return bpf_obj_drop (p__alloc, meta__ign);
@@ -2413,6 +2416,7 @@ __bpf_kfunc void bpf_percpu_obj_drop(void *p__alloc, struct btf_struct_meta *metbpf_mem_free_rcu (&bpf_global_percpu_ma, p__alloc); } +__bpf_kfunc_replacement (bpf_percpu_obj_drop) __bpf_kfunc void bpf_percpu_obj_drop_impl(void *p__alloc, void *meta__ign) {bpf_percpu_obj_drop (p__alloc, meta__ign);
@@ -2445,6 +2449,7 @@ __bpf_kfunc void *bpf_refcount_acquire(void *p__refcounted_kptr, struct btf_strureturn (void *)p__refcounted_kptr; } +__bpf_kfunc_replacement (bpf_refcount_acquire) __bpf_kfunc void *bpf_refcount_acquire_impl(void *p__refcounted_kptr, void *meta__ign) {return bpf_refcount_acquire (p__refcounted_kptr, meta__ign);
@@ -2499,6 +2504,7 @@ __bpf_kfunc int bpf_list_push_front(struct bpf_list_head *head,return __bpf_list_add (n, head, false, meta ? meta->record :NULL , off); } +__bpf_kfunc_replacement (bpf_list_push_front) __bpf_kfunc int bpf_list_push_front_impl(struct bpf_list_head *head,struct bpf_list_node *node,void *meta__ign,u64 off)
@@ -2528,6 +2534,7 @@ __bpf_kfunc int bpf_list_push_back(struct bpf_list_head *head,return __bpf_list_add (n, head, true, meta ? meta->record :NULL , off); } +__bpf_kfunc_replacement (bpf_list_push_back) __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,struct bpf_list_node *node,void *meta__ign,u64 off)
@@ -2668,6 +2675,7 @@ __bpf_kfunc int bpf_rbtree_add(struct bpf_rb_root *root,return __bpf_rbtree_add (root, n, (void *)less, meta ? meta->record :NULL , off); } +__bpf_kfunc_replacement (bpf_rbtree_add) __bpf_kfunc int bpf_rbtree_add_impl(struct bpf_rb_root *root, struct bpf_rb_node *node,bool (less )(struct bpf_rb_node *a,const struct bpf_rb_node *b),void *meta__ign,u64 off)
kernel/bpf/verifier.c +9 -0
@@ -14594,6 +14594,8 @@ static void warn_for_deprecated_kfuncs(struct bpf_verifier_env *env, int insn_idx, s16 offset) {const struct bpf_line_info *linfo; +const char *replacement; +const struct btf_type *t;struct bpf_kfunc_desc *desc;const char *file;int line_num;
@@ -14612,6 +14614,13 @@ static void warn_for_deprecated_kfuncs(struct bpf_verifier_env *env, insn_idx, meta->func_name); } + t =btf_type_by_id (meta->btf, meta->func_id); + replacement =btf_find_decl_tag_value (meta->btf, t,-1 ,BPF_KFUNC_DECL_TAG_REPLACEMENT ); +if (!IS_ERR (replacement) && !str_is_empty (replacement)) { +warn (env,"Switch to kfunc %s() instead.\n" , replacement); +warn (env,"For older kernels, choose the correct kfunc using bpf_ksym_exists().\n" ); + } + desc->warned_deprecated = true; }
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +1 -0
@@ -1252,6 +1252,7 @@ __bpf_kfunc int bpf_kfunc_multi_st_ops_test_1_assoc(struct st_ops_args *args, st __bpf_kfunc int bpf_kfunc_implicit_arg(int a, struct bpf_prog_aux *aux); __bpf_kfunc int bpf_kfunc_implicit_arg_legacy(int a, int b, struct bpf_prog_aux *aux); +__bpf_kfunc_replacement (bpf_kfunc_implicit_arg_legacy) __bpf_kfunc int bpf_kfunc_implicit_arg_legacy_impl(int a, int b, struct bpf_prog_aux *aux);/* hook targets */
Signature · 1 line
2.52.0
Kumar Kartikeya Dwivedi
maintainerPATCH v2 6/6 libbpf: Flush verifier warning messages by default
In reply to Kumar Kartikeya DwivediAllow passing a log buffer with log_level = 0, and also set the log buffer by default when loading a program, so that warnings can be captured and flushed, such that we can notify users of deprecation messages from the verifier even if program succeeds in loading.
Use the presence of enum bpf_features member to disable passing log_buf with log_level = 0 as the verifier will reject the program due to an invalid attribute. Low-level bpf_prog_load() API is relaxed, and will continue passing on older kernels as long as callers don't pass log_buf unconditionally.
--- tools/lib/bpf/bpf.c | 5 +-- tools/lib/bpf/features.c | 56 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 20 +++++++----- tools/lib/bpf/libbpf_internal.h | 2 ++ 4 files changed, 74 insertions(+), 9 deletions(-)
tools/lib/bpf/bpf.c +3 -2
@@ -315,11 +315,12 @@ int bpf_prog_load(enum bpf_prog_type prog_type, attr.fd_array =ptr_to_u64 (OPTS_GET (opts, fd_array,NULL )); attr.fd_array_cnt =OPTS_GET (opts, fd_array_cnt,0 ); -if (log_level) { +/* Pass log_buf/log_size independently so log_level=0 loads can collect warnings. */ +if (log_buf) { attr.log_buf =ptr_to_u64 (log_buf); attr.log_size = log_size; - attr.log_level = log_level; } + attr.log_level = log_level; fd =sys_bpf_prog_load (&attr, attr_sz, attempts);OPTS_SET (opts, log_true_size, attr.log_true_size);
tools/lib/bpf/features.c +56 -0
68 lines not on this page · raw message
tools/lib/bpf/libbpf.c +13 -7
@@ -7832,10 +7832,13 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog {LIBBPF_OPTS (bpf_prog_load_opts, load_attr);const char *prog_name =NULL ; +const size_t warn_log_buf_size =4096 ;size_t log_buf_size =0 ;char *log_buf =NULL , *tmp;bool own_log_buf = true;__u32 log_level = prog->log_level; +bool want_verifier_warnings = log_level ==0 && +kernel_supports (obj,FEAT_VERIFIER_WARNINGS );int ret, err;/* Be more helpful by rejecting programs that can't be validated early
@@ -7912,12 +7915,11 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog } retry_load: -/* if log_level is zero, we don't request logs initially even if -* custom log_buf is specified; if the program load fails, then we'll -* bump log_level to 1 and use either custom log_buf or we'll allocate -* our own and retry the load to get details on what failed +/* If supported, pass a buffer with log_level=0 so the kernel can report +* success-path warnings. If the program load fails, retry with +* log_level=1 to get details on what failed. */ -if (log_level) { +if (log_level || want_verifier_warnings) {if (prog->log_buf) { log_buf = prog->log_buf; log_buf_size = prog->log_size;
@@ -7927,7 +7929,8 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog log_buf_size = obj->log_size; own_log_buf = false; }else { - log_buf_size =max ((size_t )BPF_LOG_BUF_SIZE , log_buf_size *2 ); + log_buf_size =max (log_level ? (size_t )BPF_LOG_BUF_SIZE : warn_log_buf_size, + log_buf_size *2 ); tmp =realloc (log_buf, log_buf_size);if (!tmp) { ret = -ENOMEM ;
@@ -7945,7 +7948,10 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog ret =bpf_prog_load (prog->type, prog_name, license, insns, insns_cnt, &load_attr);if (ret >=0 ) { -if (log_level && own_log_buf) { +if (want_verifier_warnings && log_level ==0 && load_attr.log_true_size) { +pr_warn ("prog '%s': -- BEGIN PROG LOAD WARNINGS --\n%s-- END PROG LOAD WARNINGS --\n" , + prog->name, log_buf); + }else if (log_level && own_log_buf) {pr_debug ("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n" , prog->name, log_buf); }
tools/lib/bpf/libbpf_internal.h +2 -0
@@ -378,6 +378,8 @@ enum kern_feature_id {FEAT_BTF_DECL_TAG ,/* BTF_KIND_TYPE_TAG support */ FEAT_BTF_TYPE_TAG , +/* Kernel supports success-path verifier warnings */ +FEAT_VERIFIER_WARNINGS ,/* memcg-based accounting for BPF maps and progs */ FEAT_MEMCG_ACCOUNT,/* BPF cookie (bpf_get_attach_cookie() BPF helper) support */
Signature · 1 line
2.52.0
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v2 6/6] libbpf: Flush verifier warning messages by default
Quoted 41 lines · Kumar Kartikeya Dwivedi
Allow passing a log buffer with
log_level= 0, and also set the log buffer by default when loading a program, so that warnings can be captured and flushed, such that we can notify users of deprecation messages from the verifier even if program succeeds in loading.Use the presence of enum
bpf_featuresmember to disable passinglog_bufwithlog_level= 0 as the verifier will reject the program due to an invalid attribute. Low-levelbpf_prog_load()API is relaxed, and will continue passing on older kernels as long as callers don't passlog_bufunconditionally.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
[...]
retry_load:tools/lib/bpf/libbpf.c +6
-6-/* if log_level is zero, we don't request logs initially even if -* custom log_buf is specified; if the program load fails, then we'll -* bump log_level to 1 and use either custom log_buf or we'll allocate -* our own and retry the load to get details on what failed +/* If supported, pass a buffer with log_level=0 so the kernel can report +* success-path warnings. If the program load fails, retry with +* log_level=1 to get details on what failed. */ -if (log_level) { +if (log_level || want_verifier_warnings) {if (prog->log_buf) { log_buf = prog->log_buf; log_buf_size = prog->log_size;@@ -7927,7 +7929,8 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog log_buf_size = obj->log_size; own_log_buf = false; }else { - log_buf_size =max ((size_t )BPF_LOG_BUF_SIZE , log_buf_size *2 ); + log_buf_size =max (log_level ? (size_t )BPF_LOG_BUF_SIZE : warn_log_buf_size, + log_buf_size *2 ); tmp =realloc (log_buf, log_buf_size);if (!tmp) { ret = -ENOMEM ;
tools/lib/bpf/libbpf.c static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog · line 7951
Quoted 5 lines
tools/lib/bpf/libbpf.c +1
-1@@ -7945,7 +7948,10 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog ret =bpf_prog_load (prog->type, prog_name, license, insns, insns_cnt, &load_attr);if (ret >=0 ) { -if (log_level && own_log_buf) { +if (want_verifier_warnings && log_level ==0 && load_attr.log_true_size) {
I was wondering whether to use own_log_buf here or not, but would like a second opinion.
Quoted 22 lines
tools/lib/bpf/libbpf.c +3
-0+pr_warn ("prog '%s': -- BEGIN PROG LOAD WARNINGS --\n%s-- END PROG LOAD WARNINGS --\n" , + prog->name, log_buf); + }else if (log_level && own_log_buf) {pr_debug ("prog '%s': -- BEGIN PROG LOAD LOG --\n%s-- END PROG LOAD LOG --\n" , prog->name, log_buf); }tools/lib/bpf/libbpf_internal.h +2
-1@@ -378,6 +378,8 @@ enum kern_feature_id {FEAT_BTF_DECL_TAG ,/* BTF_KIND_TYPE_TAG support */ FEAT_BTF_TYPE_TAG , +/* Kernel supports success-path verifier warnings */ +FEAT_VERIFIER_WARNINGS ,/* memcg-based accounting for BPF maps and progs */ FEAT_MEMCG_ACCOUNT,/* BPF cookie (bpf_get_attach_cookie() BPF helper) support */ --2.52.0
Quoted 34 lines · Kumar Kartikeya Dwivedi
Currently, there are only two ways of communicating information to the user when a program is verified, success or failure with a verbose verifier log. Some information is meant to be more discretionary, e.g. warning about use of kfuncs that are deprecated, and may be removed in future kernel releases.
An example is shown below.
$ ./test_progs -t kfunc_implicit_args/test_kfunc_implicit_arg_legacy_impl -v ... 2: (b7) r3 = 0 ; R3=0 3: (85) call bpf_kfunc_implicit_arg_legacy_impl#114683 kfunc_implicit_args.c:40 (insn #3) uses deprecated kfunc bpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfunc bpf_kfunc_implicit_arg_legacy() instead. For older kernels, choose the correct kfunc using bpf_ksym_exists(). 4: R0=scalar() 4: (95) exit ...With verbose logging, the deprecation warnings are printed inline in the verifier log. When no log level is chosen, the warning is printed in a block, as follows:
libbpf: prog '
test_kfunc_implicit_arg_legacy_impl': -- BEGIN PROG LOAD WARNINGS -- kfunc_implicit_args.c:40 (insn #3) uses deprecated kfuncbpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfuncbpf_kfunc_implicit_arg_legacy()instead.
For older kernels, choose the correct kfunc usingbpf_ksym_exists().
-- END PROG LOAD WARNINGS --The set was tested with and without kernel changes applied, so that we could conclude that libbpf feature detection works correctly and backwards compatibility is not violated on older kernels. Still, it would be helpful if other reviewers could recheck all assumptions and make sure I didn't miss any corner case.
Quoted 9 lines
One case I need more opinions on:
log_buf.cpasseslog_bufwith log_level=0 and expects it to be empty, with newer libbpf that won't be true on older kernels anymore. I don't think this is something that needs to be addressed, but explicitly noting it here since I saw that test fail when testing new libbpf on older kernels. The kernel will now populate suchlog_bufwill "processed N insns ..." log since on failure libbpf will retry with log_level=1, so unless applications are relying onlog_bufto be empty for correctness it should be ok.
Since I'm going to add common attributes [1] support for BPF syscall, how about reporting the warnings via the common attributes? Even though, the original intent of common attributes support for BPF_LOAD was to report the verifier log too.
[1] https://lore.kernel.org/bpf/[email protected]/
Thanks,
Leon
Quoted 47 lines · Leon Hwang
Quoted 34 lines · Kumar Kartikeya Dwivedi
Currently, there are only two ways of communicating information to the user when a program is verified, success or failure with a verbose verifier log. Some information is meant to be more discretionary, e.g. warning about use of kfuncs that are deprecated, and may be removed in future kernel releases.
An example is shown below.
$ ./test_progs -t kfunc_implicit_args/test_kfunc_implicit_arg_legacy_impl -v ... 2: (b7) r3 = 0 ; R3=0 3: (85) call bpf_kfunc_implicit_arg_legacy_impl#114683 kfunc_implicit_args.c:40 (insn #3) uses deprecated kfunc bpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfunc bpf_kfunc_implicit_arg_legacy() instead. For older kernels, choose the correct kfunc using bpf_ksym_exists(). 4: R0=scalar() 4: (95) exit ...With verbose logging, the deprecation warnings are printed inline in the verifier log. When no log level is chosen, the warning is printed in a block, as follows:
libbpf: prog '
test_kfunc_implicit_arg_legacy_impl': -- BEGIN PROG LOAD WARNINGS -- kfunc_implicit_args.c:40 (insn #3) uses deprecated kfuncbpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfuncbpf_kfunc_implicit_arg_legacy()instead.
For older kernels, choose the correct kfunc usingbpf_ksym_exists().
-- END PROG LOAD WARNINGS --The set was tested with and without kernel changes applied, so that we could conclude that libbpf feature detection works correctly and backwards compatibility is not violated on older kernels. Still, it would be helpful if other reviewers could recheck all assumptions and make sure I didn't miss any corner case.
Quoted 9 lines
One case I need more opinions on:
log_buf.cpasseslog_bufwith log_level=0 and expects it to be empty, with newer libbpf that won't be true on older kernels anymore. I don't think this is something that needs to be addressed, but explicitly noting it here since I saw that test fail when testing new libbpf on older kernels. The kernel will now populate suchlog_bufwill "processed N insns ..." log since on failure libbpf will retry with log_level=1, so unless applications are relying onlog_bufto be empty for correctness it should be ok.
Quoted 3 lines
Since I'm going to add common attributes [1] support for BPF syscall, how about reporting the warnings via the common attributes? Even though, the original intent of common attributes support for
BPF_LOADwas
BPF_LOAD -> BPF_PROG_LOAD
Also, common attributes support for BPF_BTF_LOAD was to report logs, similar to btf_log_* in union bpf_attr.
If to report the warnings via common attributes for BPF_PROG_LOAD, we should rethink the purpose of common attributes support for BPF_BTF_LOAD.
Quoted 3 lines
report the verifier log too.
[1]
https://lore.kernel.org/bpf/[email protected]/
Quoted 4 lines
Thanks,
Leon
Kumar Kartikeya Dwivedi
maintainerQuoted 57 lines · Leon Hwang
Quoted 47 lines · Leon Hwang
Quoted 34 lines · Kumar Kartikeya Dwivedi
Currently, there are only two ways of communicating information to the user when a program is verified, success or failure with a verbose verifier log. Some information is meant to be more discretionary, e.g. warning about use of kfuncs that are deprecated, and may be removed in future kernel releases.
An example is shown below.
$ ./test_progs -t kfunc_implicit_args/test_kfunc_implicit_arg_legacy_impl -v ... 2: (b7) r3 = 0 ; R3=0 3: (85) call bpf_kfunc_implicit_arg_legacy_impl#114683 kfunc_implicit_args.c:40 (insn #3) uses deprecated kfunc bpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfunc bpf_kfunc_implicit_arg_legacy() instead. For older kernels, choose the correct kfunc using bpf_ksym_exists(). 4: R0=scalar() 4: (95) exit ...With verbose logging, the deprecation warnings are printed inline in the verifier log. When no log level is chosen, the warning is printed in a block, as follows:
libbpf: prog '
test_kfunc_implicit_arg_legacy_impl': -- BEGIN PROG LOAD WARNINGS -- kfunc_implicit_args.c:40 (insn #3) uses deprecated kfuncbpf_kfunc_implicit_arg_legacy_impl(), which will be removed. Switch to kfuncbpf_kfunc_implicit_arg_legacy()instead.
For older kernels, choose the correct kfunc usingbpf_ksym_exists().
-- END PROG LOAD WARNINGS --The set was tested with and without kernel changes applied, so that we could conclude that libbpf feature detection works correctly and backwards compatibility is not violated on older kernels. Still, it would be helpful if other reviewers could recheck all assumptions and make sure I didn't miss any corner case.
Quoted 9 lines
One case I need more opinions on:
log_buf.cpasseslog_bufwith log_level=0 and expects it to be empty, with newer libbpf that won't be true on older kernels anymore. I don't think this is something that needs to be addressed, but explicitly noting it here since I saw that test fail when testing new libbpf on older kernels. The kernel will now populate suchlog_bufwill "processed N insns ..." log since on failure libbpf will retry with log_level=1, so unless applications are relying onlog_bufto be empty for correctness it should be ok.Quoted 3 lines
Since I'm going to add common attributes [1] support for BPF syscall, how about reporting the warnings via the common attributes? Even though, the original intent of common attributes support for
BPF_LOADwas
BPF_LOAD->BPF_PROG_LOADAlso, common attributes support for
BPF_BTF_LOADwas to report logs, similar to btf_log_* in unionbpf_attr.
Quoted 7 lines
If to report the warnings via common attributes for
BPF_PROG_LOAD, we should rethink the purpose of common attributes support forBPF_BTF_LOAD.Quoted 3 lines
report the verifier log too.
[1]
It can be used when present, I will switch to a warning bit in the log level in the next version. But I don't think we need to rethink anything. If we want warnings for other objects they can use the log level's warning bit to supply those to user space as well. When you add support you already have to figure out how to merge those in bpf_attr and common attributes.
Quoted 6 lines
Quoted 4 lines
Thanks,
Leon
Quoted 4 lines · Kumar Kartikeya Dwivedi
Quoted 3 lines · Leon Hwang
Quoted 1 line · Leon Hwang
[...]
Quoted 15 lines
Quoted 15 lines
Quoted 4 lines
Since I'm going to add common attributes [1] support for BPF syscall, how about reporting the warnings via the common attributes? Even though, the original intent of common attributes support for
BPF_LOADwas
BPF_LOAD->BPF_PROG_LOADAlso, common attributes support for
BPF_BTF_LOADwas to report logs, similar to btf_log_* in unionbpf_attr.If to report the warnings via common attributes for
BPF_PROG_LOAD, we should rethink the purpose of common attributes support forBPF_BTF_LOAD.Quoted 3 lines
report the verifier log too.
[1]
Quoted 9 lines
Quoted 1 line
It can be used when present, I will switch to a warning bit in the log level in the next version. But I don't think we need to rethink anything. If we want warnings for other objects they can use the log level's warning bit to supply those to user space as well. When you add support you already have to figure out how to merge those in
bpf_attrand common attributes.
Make sense.
Thanks,
Leon
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Wed, 8 Apr 2026 04:13:51 +0200 you wrote:
Currently, there are only two ways of communicating information to the user when a program is verified, success or failure with a verbose verifier log. Some information is meant to be more discretionary, e.g. warning about use of kfuncs that are deprecated, and may be removed in future kernel releases.
An example is shown below.
[...]
Here is the summary with links:
- [bpf-next,v2,1/6] bpf: Add support for verifier warning messages (no matching commit)
- [bpf-next,v2,2/6] bpf: Extract bpf_get_linfo_file_line
https://git.kernel.org/bpf/bpf-next/c/fbb98834a922 - [bpf-next,v2,3/6] bpf: Make find_linfo widely available https://git.kernel.org/bpf/bpf-next/c/4f64d5b66418
- [bpf-next,v2,4/6] bpf: Use KF_DEPRECATED to emit verifier warnings (no matching commit)
- [bpf-next,v2,5/6] bpf: Add __bpf_kfunc_replacement() annotation (no matching commit)
- [bpf-next,v2,6/6] libbpf: Flush verifier warning messages by default (no matching commit)
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html