BPF verifier retval logic fixes
Subsystems (4)
Andrii Nakryiko
maintainerv2 cover BPF verifier retval logic fixes
This patch set fixes BPF verifier logic around validating and enforcing return values for BPF programs that have specific range of expected return values. Both sync and async callbacks have similar logic and are fixes as well. A few tests are added that would fail without the fixes in this patch set.
Also, while at it, we update retval checking logic to use umin/umax range instead of tnum, avoiding future potential issues if expected range cannot be represented precisely by tnum (e.g., [0, 2] is not representable by tnum and is treated as [0, 3]).
There is a little bit of refactoring to unify async callback and program exit logic to avoid duplication of checks as much as possible.
v1->v2:
- drop tnum from retval checks (Eduard);
- use smin/smax instead of umin/umax (Alexei).
Andrii Nakryiko (10):
bpf: provide correct register name for exception callback retval check
bpf: enforce precision of R0 on callback return
bpf: enforce exact retval range on subprog/callback exit selftests/bpf: add selftest validating callback result is enforced
bpf: enforce precise retval range on program exit
bpf: unify async callback and program retval checks
bpf: enforce precision of R0 on program/async callback return selftests/bpf: validate async callback return value check correctness selftests/bpf: adjust global_func15 test to validate prog exit
precision
bpf: simplify tnum output if a fully known constant
include/linux/bpf_verifier.h | 7 +- kernel/bpf/log.c | 13 ++ kernel/bpf/tnum.c | 6 - kernel/bpf/verifier.c | 120 ++++++++++-------- .../selftests/bpf/progs/exceptions_assert.c | 2 +- .../selftests/bpf/progs/exceptions_fail.c | 2 +- .../selftests/bpf/progs/test_global_func15.c | 34 ++++- .../selftests/bpf/progs/timer_failure.c | 36 ++++-- .../selftests/bpf/progs/user_ringbuf_fail.c | 2 +- .../bpf/progs/verifier_cgroup_inv_retcode.c | 8 +- .../bpf/progs/verifier_direct_packet_access.c | 2 +- .../selftests/bpf/progs/verifier_int_ptr.c | 2 +- .../bpf/progs/verifier_netfilter_retcode.c | 2 +- .../selftests/bpf/progs/verifier_stack_ptr.c | 4 +- .../bpf/progs/verifier_subprog_precision.c | 50 ++++++++ 15 files changed, 211 insertions(+), 79 deletions(-)
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 1/10 bpf: provide correct register name for exception callback retval check
bpf_throw() is checking R1, so let's report R1 in the log.
--- kernel/bpf/verifier.c | 12 ++++++------ .../testing/selftests/bpf/progs/exceptions_assert.c | 2 +- tools/testing/selftests/bpf/progs/exceptions_fail.c | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-)
kernel/bpf/verifier.c +6 -6
@@ -11805,7 +11805,7 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env,return 0 ; } -static int check_return_code (struct bpf_verifier_env *env,int regno); +static int check_return_code (struct bpf_verifier_env *env,int regno,const char *reg_name);static int check_kfunc_call (struct bpf_verifier_env *env,struct bpf_insn *insn,int *insn_idx_p)
@@ -11942,7 +11942,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,* to bpf_throw becomes the return value of the program. */ if (!env->exception_callback_subprog) { - err =check_return_code (env,BPF_REG_1 ); + err =check_return_code (env,BPF_REG_1 ,"R1" );if (err <0 )return err; }
@@ -14972,7 +14972,7 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)return 0 ; } -static int check_return_code (struct bpf_verifier_env *env,int regno) +static int check_return_code (struct bpf_verifier_env *env,int regno,const char *reg_name) {struct tnum enforce_attach_type_range = tnum_unknown;const struct bpf_prog *prog = env->prog;
@@ -15026,7 +15026,7 @@ static int check_return_code(struct bpf_verifier_env *env, int regno) }if (!tnum_in (const_0, reg->var_off)) { -verbose_invalid_scalar (env, reg, &const_0,"async callback" ,"R0" ); +verbose_invalid_scalar (env, reg, &const_0,"async callback" , reg_name);return -EINVAL ; }return 0 ;
@@ -15126,7 +15126,7 @@ static int check_return_code(struct bpf_verifier_env *env, int regno) }if (!tnum_in (range, reg->var_off)) { -verbose_invalid_scalar (env, reg, &range,"program exit" ,"R0" ); +verbose_invalid_scalar (env, reg, &range,"program exit" , reg_name);if (prog->expected_attach_type ==BPF_LSM_CGROUP && prog_type ==BPF_PROG_TYPE_LSM && !prog->aux->attach_func_proto->type)
@@ -17410,7 +17410,7 @@ static int do_check(struct bpf_verifier_env *env)continue ; } - err =check_return_code (env,BPF_REG_0 ); + err =check_return_code (env,BPF_REG_0 ,"R0" );if (err)return err; process_bpf_exit:
tools/testing/selftests/bpf/progs/exceptions_assert.c +1 -1
@@ -125,7 +125,7 @@ int check_assert_generic(struct __sk_buff *ctx) } SEC("?fentry/bpf_check") -__failure __msg("At program exit the register R0 has value (0x40; 0x0)") +__failure __msg("At program exit the register R1 has value (0x40; 0x0)") int check_assert_with_return(void *ctx) {bpf_assert_with (!ctx,64 );
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
@@ -308,7 +308,7 @@ int reject_set_exception_cb_bad_ret1(void *ctx) } SEC("?fentry/bpf_check") -__failure __msg("At program exit the register R0 has value (0x40; 0x0) should") +__failure __msg("At program exit the register R1 has value (0x40; 0x0) should") int reject_set_exception_cb_bad_ret2(void *ctx) {bpf_throw (64 );
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 2/10 bpf: enforce precision of R0 on callback return
In reply to Andrii NakryikoGiven verifier checks actual value, r0 has to be precise, so we need to propagate precision properly. r0 also has to be marked as read, otherwise subsequent state comparisons will ignore such register as unimportant and precision won't really help here.
--- kernel/bpf/verifier.c | 7 +++++++ 1 file changed, 7 insertions(+)
kernel/bpf/verifier.c +7 -0
@@ -9590,6 +9590,13 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)verbose (env,"R0 not a scalar value\n" );return -EACCES ; } + +/* we are going to rely on register's precise value */ + err =mark_reg_read (env, r0, r0->parent,REG_LIVE_READ64 ); + err = err ?:mark_chain_precision (env,BPF_REG_0 ); +if (err) +return err; +if (!tnum_in (range, r0->var_off)) {verbose_invalid_scalar (env, r0, &range,"callback return" ,"R0" );return -EINVAL ;
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 3/10 bpf: enforce exact retval range on subprog/callback exit
In reply to Andrii NakryikoInstead of relying on potentially imprecise tnum representation of expected return value range for callbacks and subprogs, validate that umin/umax range satisfy exact expected range of return values.
E.g., if callback would need to return [0, 2] range, tnum can't represent this precisely and instead will allow [0, 3] range. By checking umin/umax range, we can make sure that subprog/callback indeed returns only valid [0, 2] range.
--- include/linux/bpf_verifier.h | 7 ++++++- kernel/bpf/verifier.c | 40 ++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 12 deletions(-)
include/linux/bpf_verifier.h +6 -1
@@ -275,6 +275,11 @@ struct bpf_reference_state {int callback_ref; }; +struct bpf_retval_range { +s32 minval; +s32 maxval; +}; +/* state of the program: * type of all registers and stack info */
@@ -297,7 +302,7 @@ struct bpf_func_state {* void foo(void) { bpf_timer_set_callback(,foo); } */ u32 async_entry_cnt; -struct tnum callback_ret_range; +struct bpf_retval_range callback_ret_range;bool in_callback_fn;bool in_async_callback_fn;bool in_exception_callback_fn;
kernel/bpf/verifier.c +29 -11
@@ -2305,6 +2305,11 @@ static void init_reg_state(struct bpf_verifier_env *env, regs[BPF_REG_FP ].frameno = state->frameno; } +static struct bpf_retval_range retval_range (s32 minval,s32 maxval) +{ +return (struct bpf_retval_range ){ minval, maxval }; +} +#define BPF_MAIN_FUNC (-1) static void init_func_state(struct bpf_verifier_env *env, struct bpf_func_state *state,
@@ -2313,7 +2318,7 @@ static void init_func_state(struct bpf_verifier_env *env, state->callsite = callsite; state->frameno = frameno; state->subprogno = subprogno; - state->callback_ret_range =tnum_range (0 ,0 ); + state->callback_ret_range =retval_range (0 ,0 );init_reg_state (env, state);mark_verifier_state_scratched (env); }
@@ -9396,7 +9401,7 @@ static int set_map_elem_callback_state(struct bpf_verifier_env *env,return err; callee->in_callback_fn = true; - callee->callback_ret_range =tnum_range (0 ,1 ); + callee->callback_ret_range =retval_range (0 ,1 );return 0 ; }
@@ -9418,7 +9423,7 @@ static int set_loop_callback_state(struct bpf_verifier_env *env,__mark_reg_not_init (env, &callee->regs[BPF_REG_5 ]); callee->in_callback_fn = true; - callee->callback_ret_range =tnum_range (0 ,1 ); + callee->callback_ret_range =retval_range (0 ,1 );return 0 ; }
@@ -9448,7 +9453,7 @@ static int set_timer_callback_state(struct bpf_verifier_env *env,__mark_reg_not_init (env, &callee->regs[BPF_REG_4 ]);__mark_reg_not_init (env, &callee->regs[BPF_REG_5 ]); callee->in_async_callback_fn = true; - callee->callback_ret_range =tnum_range (0 ,1 ); + callee->callback_ret_range =retval_range (0 ,1 );return 0 ; }
@@ -9476,7 +9481,7 @@ static int set_find_vma_callback_state(struct bpf_verifier_env *env,__mark_reg_not_init (env, &callee->regs[BPF_REG_4 ]);__mark_reg_not_init (env, &callee->regs[BPF_REG_5 ]); callee->in_callback_fn = true; - callee->callback_ret_range =tnum_range (0 ,1 ); + callee->callback_ret_range =retval_range (0 ,1 );return 0 ; }
@@ -9499,7 +9504,7 @@ static int set_user_ringbuf_callback_state(struct bpf_verifier_env *env,__mark_reg_not_init (env, &callee->regs[BPF_REG_5 ]); callee->in_callback_fn = true; - callee->callback_ret_range =tnum_range (0 ,1 ); + callee->callback_ret_range =retval_range (0 ,1 );return 0 ; }
@@ -9531,7 +9536,7 @@ static int set_rbtree_add_callback_state(struct bpf_verifier_env *env,__mark_reg_not_init (env, &callee->regs[BPF_REG_4 ]);__mark_reg_not_init (env, &callee->regs[BPF_REG_5 ]); callee->in_callback_fn = true; - callee->callback_ret_range =tnum_range (0 ,1 ); + callee->callback_ret_range =retval_range (0 ,1 );return 0 ; }
@@ -9560,6 +9565,19 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)return is_rbtree_lock_required_kfunc (kfunc_btf_id); } +static bool retval_range_within (struct bpf_retval_range range,const struct bpf_reg_state *reg) +{ +return range.minval <= reg->umin_value && reg->umax_value <= range.maxval; +} + +static struct tnum retval_range_as_tnum (struct bpf_retval_range range) +{ +if (range.minval == range.maxval) +return tnum_const (range.minval); +else +return tnum_range (range.minval, range.maxval); +} +static int prepare_func_exit (struct bpf_verifier_env *env,int *insn_idx) {struct bpf_verifier_state *state = env->cur_state, *prev_st;
@@ -9583,9 +9601,6 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx) caller = state->frame[state->curframe -1 ];if (callee->in_callback_fn) { -/* enforce R0 return value range [0, 1]. */ -struct tnum range = callee->callback_ret_range; -if (r0->type !=SCALAR_VALUE ) {verbose (env,"R0 not a scalar value\n" );return -EACCES ;
@@ -9597,7 +9612,10 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)if (err)return err; -if (!tnum_in (range, r0->var_off)) { +/* enforce R0 return value range */ +if (!retval_range_within (callee->callback_ret_range, r0)) { +struct tnum range =retval_range_as_tnum (callee->callback_ret_range); +verbose_invalid_scalar (env, r0, &range,"callback return" ,"R0" );return -EINVAL ; }
Signature · 1 line
2.34.1
Re: [PATCH v2 bpf-next 03/10] bpf: enforce exact retval range on subprog/callback exit
Quoted 17 lines · Andrii Nakryiko
Instead of relying on potentially imprecise tnum representation of expected return value range for callbacks and subprogs, validate that umin/umax range satisfy exact expected range of return values.
E.g., if callback would need to return [0, 2] range, tnum can't represent this precisely and instead will allow [0, 3] range. By checking umin/umax range, we can make sure that subprog/callback indeed returns only valid [0, 2] range.
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>--- include/linux/bpf_verifier.h | 7 ++++++- kernel/bpf/verifier.c | 40 ++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 12 deletions(-)
[...]
Quoted 10 lines
kernel/bpf/verifier.c +1
-1@@ -9531,7 +9536,7 @@ static int set_rbtree_add_callback_state(struct bpf_verifier_env *env,__mark_reg_not_init (env, &callee->regs[BPF_REG_4 ]);__mark_reg_not_init (env, &callee->regs[BPF_REG_5 ]); callee->in_callback_fn = true; - callee->callback_ret_range =tnum_range (0 ,1 ); + callee->callback_ret_range =retval_range (0 ,1 );return 0 ; }
kernel/bpf/verifier.c static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env) · line 9570
Quoted 6 lines
kernel/bpf/verifier.c +3
-0@@ -9560,6 +9565,19 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)return is_rbtree_lock_required_kfunc (kfunc_btf_id); } +static bool retval_range_within (struct bpf_retval_range range,const struct bpf_reg_state *reg) +{ +return range.minval <= reg->umin_value && reg->umax_value <= range.maxval;
argh, I didn't update the core piece of logic to use smin/smax here.
I'll send v3 tomorrow, sorry for the spam...
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+} + +static struct tnum retval_range_as_tnum (struct bpf_retval_range range) +{ +if (range.minval == range.maxval) +return tnum_const (range.minval); +else +return tnum_range (range.minval, range.maxval); +} +
[...]
Re: [PATCH v2 bpf-next 03/10] bpf: enforce exact retval range on subprog/callback exit
In reply to Andrii NakryikoQuoted 15 lines · Andrii Nakryiko
Instead of relying on potentially imprecise tnum representation of expected return value range for callbacks and subprogs, validate that umin/umax range satisfy exact expected range of return values.
E.g., if callback would need to return [0, 2] range, tnum can't represent this precisely and instead will allow [0, 3] range. By checking umin/umax range, we can make sure that subprog/callback indeed returns only valid [0, 2] range.
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>--- include/linux/bpf_verifier.h | 7 ++++++- kernel/bpf/verifier.c | 40 ++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 12 deletions(-)
...
Quoted 8 lines
kernel/bpf/verifier.c +2
-0@@ -9560,6 +9565,19 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)return is_rbtree_lock_required_kfunc (kfunc_btf_id); } +static bool retval_range_within (struct bpf_retval_range range,const struct bpf_reg_state *reg) +{
kernel/bpf/verifier.c static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env) · line 9579
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+return range.minval <= reg->umin_value && reg->umax_value <= range.maxval; +} + +static struct tnum retval_range_as_tnum (struct bpf_retval_range range) +{ +if (range.minval == range.maxval) +return tnum_const (range.minval); +else +return tnum_range (range.minval, range.maxval); +}
Nit: find it slightly strange to have retval_range_as_tnum() added here (patch 3), only to be removed again in the patch 5. As far as I can see patch 4 doesn't require this, and it is only used once.
Perhaps just replace its use below with tnum_range() instead? (Not pretty, but will be removed anyway).
Quoted 8 lines
kernel/bpf/verifier.c +3
-1@@ -9597,7 +9612,10 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)if (err)return err; -if (!tnum_in (range, r0->var_off)) { +/* enforce R0 return value range */ +if (!retval_range_within (callee->callback_ret_range, r0)) { +struct tnum range =retval_range_as_tnum (callee->callback_ret_range);
Re: [PATCH v2 bpf-next 03/10] bpf: enforce exact retval range on subprog/callback exit
Quoted 43 lines · Shung-Hsi Yu
Quoted 15 lines · Andrii Nakryiko
Instead of relying on potentially imprecise tnum representation of expected return value range for callbacks and subprogs, validate that umin/umax range satisfy exact expected range of return values.
E.g., if callback would need to return [0, 2] range, tnum can't represent this precisely and instead will allow [0, 3] range. By checking umin/umax range, we can make sure that subprog/callback indeed returns only valid [0, 2] range.
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>--- include/linux/bpf_verifier.h | 7 ++++++- kernel/bpf/verifier.c | 40 ++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 12 deletions(-)...
Quoted 8 lines
kernel/bpf/verifier.c +2
-0@@ -9560,6 +9565,19 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)return is_rbtree_lock_required_kfunc (kfunc_btf_id); } +static bool retval_range_within (struct bpf_retval_range range,const struct bpf_reg_state *reg) +{kernel/bpf/verifier.c static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env) · line 9579
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+return range.minval <= reg->umin_value && reg->umax_value <= range.maxval; +} + +static struct tnum retval_range_as_tnum (struct bpf_retval_range range) +{ +if (range.minval == range.maxval) +return tnum_const (range.minval); +else +return tnum_range (range.minval, range.maxval); +}Nit: find it slightly strange to have
retval_range_as_tnum()added here (patch 3), only to be removed again in the patch 5. As far as I can see patch 4 doesn't require this, and it is only used once.
Quoted 3 lines
Perhaps just replace its use below with
tnum_range()instead? (Not pretty, but will be removed anyway).
I do this to delay the refactoring of verbose_invalid_scalar() which is used by another piece of logic which I refactor in a separate patch. If I don't do this temporary retval_range_as_tnum() helper, I might need to update some more tests that expect exact var_off value in logs, and I didn't want to do it. Given it's a trivial helper, it feels like it's not a big deal to keep it for a patch or two before completing the refactoring.
Quoted 8 lines
Quoted 8 lines
kernel/bpf/verifier.c +3
-1@@ -9597,7 +9612,10 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)if (err)return err; -if (!tnum_in (range, r0->var_off)) { +/* enforce R0 return value range */ +if (!retval_range_within (callee->callback_ret_range, r0)) { +struct tnum range =retval_range_as_tnum (callee->callback_ret_range);
Re: [PATCH v2 bpf-next 03/10] bpf: enforce exact retval range on subprog/callback exit
Quoted 46 lines · Andrii Nakryiko
Quoted 42 lines · Shung-Hsi Yu
Quoted 15 lines · Andrii Nakryiko
Instead of relying on potentially imprecise tnum representation of expected return value range for callbacks and subprogs, validate that umin/umax range satisfy exact expected range of return values.
E.g., if callback would need to return [0, 2] range, tnum can't represent this precisely and instead will allow [0, 3] range. By checking umin/umax range, we can make sure that subprog/callback indeed returns only valid [0, 2] range.
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>--- include/linux/bpf_verifier.h | 7 ++++++- kernel/bpf/verifier.c | 40 ++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 12 deletions(-)...
Quoted 8 lines
kernel/bpf/verifier.c +2
-0@@ -9560,6 +9565,19 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)return is_rbtree_lock_required_kfunc (kfunc_btf_id); } +static bool retval_range_within (struct bpf_retval_range range,const struct bpf_reg_state *reg) +{kernel/bpf/verifier.c static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env) · line 9579
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+return range.minval <= reg->umin_value && reg->umax_value <= range.maxval; +} + +static struct tnum retval_range_as_tnum (struct bpf_retval_range range) +{ +if (range.minval == range.maxval) +return tnum_const (range.minval); +else +return tnum_range (range.minval, range.maxval); +}Nit: find it slightly strange to have
retval_range_as_tnum()added here (patch 3), only to be removed again in the patch 5. As far as I can see patch 4 doesn't require this, and it is only used once.Quoted 2 lines
Perhaps just replace its use below with
tnum_range()instead? (Not pretty, but will be removed anyway).
Quoted 7 lines
I do this to delay the refactoring of
verbose_invalid_scalar()which is used by another piece of logic which I refactor in a separate patch. If I don't do this temporaryretval_range_as_tnum()helper, I might need to update some more tests that expect exactvar_offvalue in logs, and I didn't want to do it. Given it's a trivial helper, it feels like it's not a big deal to keep it for a patch or two before completing the refactoring.
Replace retval_range_as_tnum(callee->callback_ret_range) with
tnum_range(callee->callback_ret_range.minval,
callee->callback_ret_range.maxval)
and the verbose_invalid_scalar() signature stays the same; also no var_off changes because it is just manual inline of retval_range_as_tnum(), as tnum_range(n, n) == tnum_const(n).
Agree it really is not a big deal, so I won't insist on it.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/verifier.c +3
-1@@ -9597,7 +9612,10 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)if (err)return err; -if (!tnum_in (range, r0->var_off)) { +/* enforce R0 return value range */ +if (!retval_range_within (callee->callback_ret_range, r0)) { +struct tnum range =retval_range_as_tnum (callee->callback_ret_range);
Re: [PATCH v2 bpf-next 03/10] bpf: enforce exact retval range on subprog/callback exit
Quoted 56 lines · Shung-Hsi Yu
Quoted 46 lines · Andrii Nakryiko
Quoted 42 lines · Shung-Hsi Yu
Quoted 15 lines · Andrii Nakryiko
Instead of relying on potentially imprecise tnum representation of expected return value range for callbacks and subprogs, validate that umin/umax range satisfy exact expected range of return values.
E.g., if callback would need to return [0, 2] range, tnum can't represent this precisely and instead will allow [0, 3] range. By checking umin/umax range, we can make sure that subprog/callback indeed returns only valid [0, 2] range.
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>--- include/linux/bpf_verifier.h | 7 ++++++- kernel/bpf/verifier.c | 40 ++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 12 deletions(-)...
Quoted 8 lines
kernel/bpf/verifier.c +2
-0@@ -9560,6 +9565,19 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)return is_rbtree_lock_required_kfunc (kfunc_btf_id); } +static bool retval_range_within (struct bpf_retval_range range,const struct bpf_reg_state *reg) +{kernel/bpf/verifier.c static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env) · line 9579
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+return range.minval <= reg->umin_value && reg->umax_value <= range.maxval; +} + +static struct tnum retval_range_as_tnum (struct bpf_retval_range range) +{ +if (range.minval == range.maxval) +return tnum_const (range.minval); +else +return tnum_range (range.minval, range.maxval); +}Nit: find it slightly strange to have
retval_range_as_tnum()added here (patch 3), only to be removed again in the patch 5. As far as I can see patch 4 doesn't require this, and it is only used once.Quoted 2 lines
Perhaps just replace its use below with
tnum_range()instead? (Not pretty, but will be removed anyway).Quoted 7 lines
I do this to delay the refactoring of
verbose_invalid_scalar()which is used by another piece of logic which I refactor in a separate patch. If I don't do this temporaryretval_range_as_tnum()helper, I might need to update some more tests that expect exactvar_offvalue in logs, and I didn't want to do it. Given it's a trivial helper, it feels like it's not a big deal to keep it for a patch or two before completing the refactoring.
Quoted 8 lines
Replace
retval_range_as_tnum(callee->callback_ret_range)withtnum_range(callee->callback_ret_range.minval,
callee->callback_ret_range.maxval)and the
verbose_invalid_scalar()signature stays the same; also novar_offchanges because it is just manual inline ofretval_range_as_tnum(), astnum_range(n, n)==tnum_const(n).
I tried it locally, and I don't have to adjust any new tests, so I'll inline tnum_range() as you suggested, thanks.
Quoted 11 lines
Agree it really is not a big deal, so I won't insist on it.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/verifier.c +3
-1@@ -9597,7 +9612,10 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)if (err)return err; -if (!tnum_in (range, r0->var_off)) { +/* enforce R0 return value range */ +if (!retval_range_within (callee->callback_ret_range, r0)) { +struct tnum range =retval_range_as_tnum (callee->callback_ret_range);
Andrii Nakryiko
maintainerPATCH v2 4/10 selftests/bpf: add selftest validating callback result is enforced
In reply to Andrii NakryikoBPF verifier expects callback subprogs to return values from specified range (typically [0, 1]). This requires that r0 at exit is both precise (because we rely on specific value range) and is marked as read (otherwise state comparison will ignore such register as unimportant).
Add a simple test that validates that all these conditions are enforced.
--- .../bpf/progs/verifier_subprog_precision.c | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+)
tools/testing/selftests/bpf/progs/verifier_subprog_precision.c +50 -0
@@ -117,6 +117,56 @@ __naked int global_subprog_result_precise(void) ); } +__naked __noinline __used +static unsigned long loop_callback_bad() +{ +/* bpf_loop() callback that can return values outside of [0, 1] range */ + asmvolatile ( +"call %[bpf_get_prandom_u32];" +"if r0 > 1000 goto 1f;" +"r0 = 0;" +"1:" +"goto +0;" /* checkpoint */ +/* bpf_loop() expects [0, 1] values, so branch above skipping +* r0 = 0; should lead to a failure, but if exit instruction +* doesn't enforce r0's precision, this callback will be +* successfully verified +*/ +"exit;" + : + : __imm(bpf_get_prandom_u32) + : __clobber_common + ); +} + +SEC("?raw_tp") +__failure __log_level(2) +__flag(BPF_F_TEST_STATE_FREQ) +/* check that fallthrough code path marks r0 as precise */ +__msg("mark_precise: frame1: regs=r0 stack= before 11: (b7) r0 = 0") +/* check that we have branch code path doing its own validation */ +__msg ("from 10 to 12: frame1: R0=scalar(umin=1001) R10=fp0 cb" ) +/* check that branch code path marks r0 as precise, before failing */ +__msg("mark_precise: frame1: regs=r0 stack= before 9: (85) call bpf_get_prandom_u32#7") +__msg("At callback return the register R0 has unknown scalar value should have been in (0x0; 0x1)") +__naked int callback_precise_return_fail (void ) +{ + asmvolatile ( +"r1 = 1;" /* nr_loops */ +"r2 = %[loop_callback_bad];" /* callback_fn */ +"r3 = 0;" /* callback_ctx */ +"r4 = 0;" /* flags */ +"call %[bpf_loop];" + +"r0 = 0;" +"exit;" + : + : __imm_ptr(loop_callback_bad), +__imm (bpf_loop) + : __clobber_common + ); +} + SEC("?raw_tp") __success __log_level(2)/* First simulated path does not include callback body,
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 5/10 bpf: enforce precise retval range on program exit
In reply to Andrii NakryikoSimilarly to subprog/callback logic, enforce return value of BPF program using more precise umin/umax range.
We need to adjust a bunch of tests due to a changed format of an error message.
--- kernel/bpf/verifier.c | 63 +++++++++---------- .../selftests/bpf/progs/exceptions_assert.c | 2 +- .../selftests/bpf/progs/exceptions_fail.c | 2 +- .../selftests/bpf/progs/test_global_func15.c | 2 +- .../selftests/bpf/progs/timer_failure.c | 2 +- .../selftests/bpf/progs/user_ringbuf_fail.c | 2 +- .../bpf/progs/verifier_cgroup_inv_retcode.c | 8 +-- .../bpf/progs/verifier_netfilter_retcode.c | 2 +- .../bpf/progs/verifier_subprog_precision.c | 2 +- 9 files changed, 40 insertions(+), 45 deletions(-)
kernel/bpf/verifier.c +29 -34
@@ -362,20 +362,23 @@ __printf(2, 3) static void verbose(void *private_data, const char *fmt, ...)static void verbose_invalid_scalar (struct bpf_verifier_env *env,struct bpf_reg_state *reg, -struct tnum *range,const char *ctx, +struct bpf_retval_range range,const char *ctx,const char *reg_name) { -char tn_buf[48 ]; +bool unknown = true; -verbose (env,"At %s the register %s " , ctx, reg_name); -if (!tnum_is_unknown (reg->var_off)) { -tnum_strn (tn_buf,sizeof (tn_buf), reg->var_off); -verbose (env,"has value %s" , tn_buf); - }else { -verbose (env,"has unknown scalar value" ); +verbose (env,"At %s the register %s has" , ctx, reg_name); +if (reg->umin_value >0 ) { +verbose (env," umin=%llu" , reg->umin_value); + unknown = false; } -tnum_strn (tn_buf,sizeof (tn_buf), *range); -verbose (env," should have been in %s\n" , tn_buf); +if (reg->umax_value <U64_MAX ) { +verbose (env," umax=%llu" , reg->umax_value); + unknown = false; + } +if (unknown) +verbose (env," unknown scalar value" ); +verbose (env," should have been in [%u, %u]\n" , range.minval, range.maxval); }static bool type_may_be_null (u32 type)
@@ -9570,14 +9573,6 @@ static bool retval_range_within(struct bpf_retval_range range, const struct bpf_return range.minval <= reg->umin_value && reg->umax_value <= range.maxval; } -static struct tnum retval_range_as_tnum (struct bpf_retval_range range) -{ -if (range.minval == range.maxval) -return tnum_const (range.minval); -else -return tnum_range (range.minval, range.maxval); -} -static int prepare_func_exit (struct bpf_verifier_env *env,int *insn_idx) {struct bpf_verifier_state *state = env->cur_state, *prev_st;
@@ -9614,9 +9609,8 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)/* enforce R0 return value range */ if (!retval_range_within (callee->callback_ret_range, r0)) { -struct tnum range =retval_range_as_tnum (callee->callback_ret_range); - -verbose_invalid_scalar (env, r0, &range,"callback return" ,"R0" ); +verbose_invalid_scalar (env, r0, callee->callback_ret_range, +"callback return" ,"R0" );return -EINVAL ; }if (!calls_callback (env, callee->callsite)) {
@@ -15002,7 +14996,8 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charstruct tnum enforce_attach_type_range = tnum_unknown;const struct bpf_prog *prog = env->prog;struct bpf_reg_state *reg; -struct tnum range =tnum_range (0 ,1 ), const_0 =tnum_const (0 ); +struct bpf_retval_range range =retval_range (0 ,1 ); +struct bpf_retval_range const_0 =retval_range (0 ,0 );enum bpf_prog_type prog_type =resolve_prog_type (env->prog);int err;struct bpf_func_state *frame = env->cur_state->frame[0 ];
@@ -15050,8 +15045,8 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charreturn -EINVAL ; } -if (!tnum_in (const_0, reg->var_off)) { -verbose_invalid_scalar (env, reg, &const_0,"async callback" , reg_name); +if (!retval_range_within (const_0, reg)) { +verbose_invalid_scalar (env, reg, const_0,"async callback" , reg_name);return -EINVAL ; }return 0 ;
@@ -15077,14 +15072,14 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char env->prog->expected_attach_type == BPF_CGROUP_INET4_GETSOCKNAME || env->prog->expected_attach_type == BPF_CGROUP_INET6_GETSOCKNAME || env->prog->expected_attach_type == BPF_CGROUP_UNIX_GETSOCKNAME) - range =tnum_range (1 ,1 ); + range =retval_range (1 ,1 );if (env->prog->expected_attach_type ==BPF_CGROUP_INET4_BIND || env->prog->expected_attach_type ==BPF_CGROUP_INET6_BIND ) - range =tnum_range (0 ,3 ); + range =retval_range (0 ,3 );break ;case BPF_PROG_TYPE_CGROUP_SKB :if (env->prog->expected_attach_type ==BPF_CGROUP_INET_EGRESS ) { - range =tnum_range (0 ,3 ); + range =retval_range (0 ,3 ); enforce_attach_type_range =tnum_range (2 ,3 ); }break ;
@@ -15097,13 +15092,13 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charcase BPF_PROG_TYPE_RAW_TRACEPOINT :if (!env->prog->aux->attach_btf_id)return 0 ; - range =tnum_const (0 ); + range =retval_range (0 ,0 );break ;case BPF_PROG_TYPE_TRACING :switch (env->prog->expected_attach_type) {case BPF_TRACE_FENTRY :case BPF_TRACE_FEXIT : - range =tnum_const (0 ); + range =retval_range (0 ,0 );break ;case BPF_TRACE_RAW_TP :case BPF_MODIFY_RETURN :
@@ -15115,7 +15110,7 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char }break ;case BPF_PROG_TYPE_SK_LOOKUP : - range =tnum_range (SK_DROP ,SK_PASS ); + range =retval_range (SK_DROP ,SK_PASS );break ;case BPF_PROG_TYPE_LSM :
@@ -15129,12 +15124,12 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char/* Make sure programs that attach to void * hooks don't try to modify return value. */ - range =tnum_range (1 ,1 ); + range =retval_range (1 ,1 ); }break ;case BPF_PROG_TYPE_NETFILTER : - range =tnum_range (NF_DROP ,NF_ACCEPT ); + range =retval_range (NF_DROP ,NF_ACCEPT );break ;case BPF_PROG_TYPE_EXT :/* freplace program can return anything as its return value
@@ -15150,8 +15145,8 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charreturn -EINVAL ; } -if (!tnum_in (range, reg->var_off)) { -verbose_invalid_scalar (env, reg, &range,"program exit" , reg_name); +if (!retval_range_within (range, reg)) { +verbose_invalid_scalar (env, reg, range,"program exit" , reg_name);if (prog->expected_attach_type ==BPF_LSM_CGROUP && prog_type ==BPF_PROG_TYPE_LSM && !prog->aux->attach_func_proto->type)
tools/testing/selftests/bpf/progs/exceptions_assert.c +1 -1
@@ -125,7 +125,7 @@ int check_assert_generic(struct __sk_buff *ctx) } SEC("?fentry/bpf_check") -__failure __msg("At program exit the register R1 has value (0x40; 0x0)") +__failure __msg("At program exit the register R1 has umin=64 umax=64") int check_assert_with_return(void *ctx) {bpf_assert_with (!ctx,64 );
tools/testing/selftests/bpf/progs/exceptions_fail.c +1 -1
@@ -308,7 +308,7 @@ int reject_set_exception_cb_bad_ret1(void *ctx) } SEC("?fentry/bpf_check") -__failure __msg("At program exit the register R1 has value (0x40; 0x0) should") +__failure __msg("At program exit the register R1 has umin=64 umax=64 should") int reject_set_exception_cb_bad_ret2(void *ctx) {bpf_throw (64 );
tools/testing/selftests/bpf/progs/test_global_func15.c +1 -1
@@ -13,7 +13,7 @@ __noinline int foo(unsigned int *v) } SEC("cgroup_skb/ingress") -__failure __msg("At program exit the register R0 has value") +__failure __msg("At program exit the register R0 has ") int global_func15(struct __sk_buff *skb) {unsigned int v =1 ;
tools/testing/selftests/bpf/progs/timer_failure.c +1 -1
@@ -30,7 +30,7 @@ static int timer_cb_ret1(void *map, int *key, struct bpf_timer *timer) } SEC("fentry/bpf_fentry_test1") -__failure __msg("should have been in (0x0; 0x0)") +__failure __msg("should have been in [0, 0]") int BPF_PROG2(test_ret_1, int, a) {int key =0 ;
tools/testing/selftests/bpf/progs/user_ringbuf_fail.c +1 -1
@@ -184,7 +184,7 @@ invalid_drain_callback_return(struct bpf_dynptr *dynptr, void *context)* not be able to write to that pointer. */ SEC("?raw_tp") -__failure __msg("At callback return the register R0 has value") +__failure __msg("At callback return the register R0 has ")int user_ringbuf_callback_invalid_return (void *ctx) {bpf_user_ringbuf_drain (&user_ringbuf, invalid_drain_callback_return,NULL ,0 );
tools/testing/selftests/bpf/progs/verifier_cgroup_inv_retcode.c +4 -4
@@ -7,7 +7,7 @@ SEC("cgroup/sock") __description("bpf_exit with invalid return code. test1") -__failure __msg("R0 has value (0x0; 0xffffffff)") +__failure __msg("umax=4294967295 should have been in [0, 1]") __naked void with_invalid_return_code_test1(void) { asm volatile (" \
@@ -30,7 +30,7 @@ __naked void with_invalid_return_code_test2(void) SEC("cgroup/sock") __description("bpf_exit with invalid return code. test3") -__failure __msg("R0 has value (0x0; 0x3)") +__failure __msg("umax=3 should have been in [0, 1]") __naked void with_invalid_return_code_test3(void) { asm volatile (" \
@@ -53,7 +53,7 @@ __naked void with_invalid_return_code_test4(void) SEC("cgroup/sock") __description("bpf_exit with invalid return code. test5") -__failure __msg("R0 has value (0x2; 0x0)") +__failure __msg("umin=2 umax=2 should have been in [0, 1]") __naked void with_invalid_return_code_test5(void) { asm volatile (" \
@@ -75,7 +75,7 @@ __naked void with_invalid_return_code_test6(void) SEC("cgroup/sock") __description("bpf_exit with invalid return code. test7") -__failure __msg("R0 has unknown scalar value") +__failure __msg("R0 has unknown scalar value should have been in [0, 1]") __naked void with_invalid_return_code_test7(void) { asm volatile (" \
tools/testing/selftests/bpf/progs/verifier_netfilter_retcode.c +1 -1
@@ -39,7 +39,7 @@ __naked void with_valid_return_code_test3(void) SEC("netfilter") __description("bpf_exit with invalid return code. test4") -__failure __msg("R0 has value (0x2; 0x0)") +__failure __msg("R0 has umin=2 umax=2 should have been in [0, 1]") __naked void with_invalid_return_code_test4(void) { asm volatile (" \
tools/testing/selftests/bpf/progs/verifier_subprog_precision.c +1 -1
@@ -148,7 +148,7 @@ __msg("mark_precise: frame1: regs=r0 stack= before 11: (b7) r0 = 0")__msg ("from 10 to 12: frame1: R0=scalar(umin=1001) R10=fp0 cb" ) /* check that branch code path marks r0 as precise, before failing */ __msg("mark_precise: frame1: regs=r0 stack= before 9: (85) call bpf_get_prandom_u32#7") -__msg("At callback return the register R0 has unknown scalar value should have been in (0x0; 0x1)") +__msg("At callback return the register R0 has umin=1001 should have been in [0, 1]")__naked int callback_precise_return_fail (void ) { asm volatile (
Signature · 1 line
2.34.1
Re: [PATCH v2 bpf-next 05/10] bpf: enforce precise retval range on program exit
Quoted 8 lines · Andrii Nakryiko
Similarly to subprog/callback logic, enforce return value of BPF program using more precise umin/umax range.
We need to adjust a bunch of tests due to a changed format of an error message.
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>
...
Q: should the missing register name and values be added?
I know relatively little about selftest, but scrolling through it looks as though the expect verifier message is incomplete. (Admittedly lots of them are like this even before this patch, and this patch improves the situation already)
e.g.
tools/testing/selftests/bpf/progs/test_global_func15.c __noinline int foo(unsigned int *v) · line 19
Quoted 11 lines
tools/testing/selftests/bpf/progs/test_global_func15.c +1
-1@@ -13,7 +13,7 @@ __noinline int foo(unsigned int *v) } SEC("cgroup_skb/ingress") -__failure __msg("At program exit the register R0 has value") +__failure __msg("At program exit the register R0 has ") int global_func15(struct __sk_buff *skb) {unsigned int v =1 ;
looks like it is missing umin/umax=1
...
tools/testing/selftests/bpf/progs/verifier_cgroup_inv_retcode.c line 13
Quoted 13 lines
tools/testing/selftests/bpf/progs/verifier_cgroup_inv_retcode.c +1
-1@@ -7,7 +7,7 @@ SEC("cgroup/sock") __description("bpf_exit with invalid return code. test1") -__failure __msg("R0 has value (0x0; 0xffffffff)") +__failure __msg("umax=4294967295 should have been in [0, 1]") __naked void with_invalid_return_code_test1(void) { asm volatile (" \
looks like it is missing mention of R0, etc.
Re: [PATCH v2 bpf-next 05/10] bpf: enforce precise retval range on program exit
Quoted 8 lines · Shung-Hsi Yu
Quoted 6 lines · Andrii Nakryiko
Similarly to subprog/callback logic, enforce return value of BPF program using more precise umin/umax range.
We need to adjust a bunch of tests due to a changed format of an error message.
Quoted 7 lines
Quoted 2 lines
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>...
Q: should the missing register name and values be added?
Probably not, it makes future refactoring a bit less painful. If the important part is to check that there *was* a message about invalid return result, rather *what exact format* of that message was, then matching for a substring is enough and makes the test a bit more robust to future slight refactorings.
Quoted 5 lines
I know relatively little about selftest, but scrolling through it looks as though the expect verifier message is incomplete. (Admittedly lots of them are like this even before this patch, and this patch improves the situation already)
Often times it's actually a mistake to expect exact format, it makes for painful refactoring and improvements. I feel it every time I touch verifier log formatting logic :( So I don't want to add to that pain.
Quoted 33 lines
e.g.
tools/testing/selftests/bpf/progs/test_global_func15.c __noinline int foo(unsigned int *v) · line 19
Quoted 11 lines
tools/testing/selftests/bpf/progs/test_global_func15.c +1
-1@@ -13,7 +13,7 @@ __noinline int foo(unsigned int *v) } SEC("cgroup_skb/ingress") -__failure __msg("At program exit the register R0 has value") +__failure __msg("At program exit the register R0 has ") int global_func15(struct __sk_buff *skb) {unsigned int v =1 ;looks like it is missing umin/umax=1
...
tools/testing/selftests/bpf/progs/verifier_cgroup_inv_retcode.c line 13
Quoted 13 lines
tools/testing/selftests/bpf/progs/verifier_cgroup_inv_retcode.c +1
-1@@ -7,7 +7,7 @@ SEC("cgroup/sock") __description("bpf_exit with invalid return code. test1") -__failure __msg("R0 has value (0x0; 0xffffffff)") +__failure __msg("umax=4294967295 should have been in [0, 1]") __naked void with_invalid_return_code_test1(void) { asm volatile (" \looks like it is missing mention of R0, etc.
Re: [PATCH v2 bpf-next 05/10] bpf: enforce precise retval range on program exit
Quoted 22 lines · Andrii Nakryiko
Quoted 8 lines · Shung-Hsi Yu
Quoted 6 lines · Andrii Nakryiko
Similarly to subprog/callback logic, enforce return value of BPF program using more precise umin/umax range.
We need to adjust a bunch of tests due to a changed format of an error message.
Quoted 6 lines
Quoted 2 lines
Acked-by Eduard Zingerman <[email protected]>Signed-off-by Andrii Nakryiko <[email protected]>...
Q: should the missing register name and values be added?
Probably not, it makes future refactoring a bit less painful. If the important part is to check that there *was* a message about invalid return result, rather *what exact format* of that message was, then matching for a substring is enough and makes the test a bit more robust to future slight refactorings.
Quoted 8 lines
Quoted 4 lines
I know relatively little about selftest, but scrolling through it looks as though the expect verifier message is incomplete. (Admittedly lots of them are like this even before this patch, and this patch improves the situation already)
Often times it's actually a mistake to expect exact format, it makes for painful refactoring and improvements. I feel it every time I touch verifier log formatting logic :( So I don't want to add to that pain.
Understood. Thanks for going through the reasoning!
Quoted 33 lines
Quoted 33 lines
e.g.
tools/testing/selftests/bpf/progs/test_global_func15.c __noinline int foo(unsigned int *v) · line 19
Quoted 11 lines
tools/testing/selftests/bpf/progs/test_global_func15.c +1
-1@@ -13,7 +13,7 @@ __noinline int foo(unsigned int *v) } SEC("cgroup_skb/ingress") -__failure __msg("At program exit the register R0 has value") +__failure __msg("At program exit the register R0 has ") int global_func15(struct __sk_buff *skb) {unsigned int v =1 ;looks like it is missing umin/umax=1
...
tools/testing/selftests/bpf/progs/verifier_cgroup_inv_retcode.c line 13
Quoted 13 lines
tools/testing/selftests/bpf/progs/verifier_cgroup_inv_retcode.c +1
-1@@ -7,7 +7,7 @@ SEC("cgroup/sock") __description("bpf_exit with invalid return code. test1") -__failure __msg("R0 has value (0x0; 0xffffffff)") +__failure __msg("umax=4294967295 should have been in [0, 1]") __naked void with_invalid_return_code_test1(void) { asm volatile (" \looks like it is missing mention of R0, etc.
Andrii Nakryiko
maintainerPATCH v2 6/10 bpf: unify async callback and program retval checks
In reply to Andrii NakryikoUse common logic to verify program return values and async callback return values. This allows to avoid duplication of any extra steps necessary, like precision marking, which will be added in the next patch.
--- kernel/bpf/verifier.c | 30 ++++++++++++------------------ 1 file changed, 12 insertions(+), 18 deletions(-)
kernel/bpf/verifier.c +12 -18
@@ -367,7 +367,7 @@ static void verbose_invalid_scalar(struct bpf_verifier_env *env, {bool unknown = true; -verbose (env,"At %s the register %s has" , ctx, reg_name); +verbose (env,"%s the register %s has" , ctx, reg_name);if (reg->umin_value >0 ) {verbose (env," umin=%llu" , reg->umin_value); unknown = false;
@@ -9610,7 +9610,7 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)/* enforce R0 return value range */ if (!retval_range_within (callee->callback_ret_range, r0)) {verbose_invalid_scalar (env, r0, callee->callback_ret_range, -"callback return" ,"R0" ); +"At callback return" ,"R0" );return -EINVAL ; }if (!calls_callback (env, callee->callsite)) {
@@ -14993,11 +14993,11 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)static int check_return_code (struct bpf_verifier_env *env,int regno,const char *reg_name) { +const char *exit_ctx ="At program exit" ;struct tnum enforce_attach_type_range = tnum_unknown;const struct bpf_prog *prog = env->prog;struct bpf_reg_state *reg;struct bpf_retval_range range =retval_range (0 ,1 ); -struct bpf_retval_range const_0 =retval_range (0 ,0 );enum bpf_prog_type prog_type =resolve_prog_type (env->prog);int err;struct bpf_func_state *frame = env->cur_state->frame[0 ];
@@ -15039,17 +15039,9 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charif (frame->in_async_callback_fn) {/* enforce return zero from async callbacks like timer */ -if (reg->type !=SCALAR_VALUE ) { -verbose (env,"In async callback the register R%d is not a known value (%s)\n" , - regno,reg_type_str (env, reg->type)); -return -EINVAL ; - } - -if (!retval_range_within (const_0, reg)) { -verbose_invalid_scalar (env, reg, const_0,"async callback" , reg_name); -return -EINVAL ; - } -return 0 ; + exit_ctx ="At async callback return" ; + range =retval_range (0 ,0 ); + goto enforce_retval; }if (is_subprog && !frame->in_exception_callback_fn) {
@@ -15139,15 +15131,17 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charreturn 0 ; } +enforce_retval:if (reg->type !=SCALAR_VALUE ) { -verbose (env,"At program exit the register R%d is not a known value (%s)\n" , - regno,reg_type_str (env, reg->type)); +verbose (env,"%s the register R%d is not a known value (%s)\n" , + exit_ctx, regno,reg_type_str (env, reg->type));return -EINVAL ; }if (!retval_range_within (range, reg)) { -verbose_invalid_scalar (env, reg, range,"program exit" , reg_name); -if (prog->expected_attach_type ==BPF_LSM_CGROUP && +verbose_invalid_scalar (env, reg, range, exit_ctx, reg_name); +if (!is_subprog && + prog->expected_attach_type ==BPF_LSM_CGROUP && prog_type ==BPF_PROG_TYPE_LSM && !prog->aux->attach_func_proto->type)verbose (env,"Note, BPF_LSM_CGROUP that attach to void LSM hooks can't modify return value!\n" );
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 7/10 bpf: enforce precision of R0 on program/async callback return
In reply to Andrii NakryikoGiven we enforce a valid range for program and async callback return value, we must mark R0 as precise to avoid incorrect state pruning.
--- kernel/bpf/verifier.c | 4 ++++ 1 file changed, 4 insertions(+)
kernel/bpf/verifier.c +4 -0
@@ -15138,6 +15138,10 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const charreturn -EINVAL ; } + err =mark_chain_precision (env, regno); +if (err) +return err; +if (!retval_range_within (range, reg)) {verbose_invalid_scalar (env, reg, range, exit_ctx, reg_name); if (!is_subprog &&
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 8/10 selftests/bpf: validate async callback return value check correctness
In reply to Andrii NakryikoAdjust timer/timer_ret_1 test to validate more carefully verifier logic of enforcing async callback return value. This test will pass only if return result is marked precise and read.
--- .../selftests/bpf/progs/timer_failure.c | 36 ++++++++++++++----- 1 file changed, 28 insertions(+), 8 deletions(-)
tools/testing/selftests/bpf/progs/timer_failure.c +28 -8
@@ -21,17 +21,37 @@ struct { __type(value, struct elem); } timer_map SEC(".maps"); -static int timer_cb_ret1 (void *map,int *key,struct bpf_timer *timer) +__naked __noinline __used +static unsigned long timer_cb_ret_bad() { -if (bpf_get_smp_processor_id () %2 ) -return 1 ; -else -return 0 ; + asmvolatile ( +"call %[bpf_get_prandom_u32];" +"if r0 > 1000 goto 1f;" +"r0 = 0;" +"1:" +"goto +0;" /* checkpoint */ +/* async callback is expected to return 0, so branch above +* skipping r0 = 0; should lead to a failure, but if exit +* instruction doesn't enforce r0's precision, this callback +* will be successfully verified +*/ +"exit;" + : + : __imm(bpf_get_prandom_u32) + : __clobber_common + ); } SEC("fentry/bpf_fentry_test1") -__failure __msg("should have been in [0, 0]") -int BPF_PROG2(test_ret_1, int, a) +__log_level(2) +__flag (BPF_F_TEST_STATE_FREQ) +__failure +/* check that fallthrough code path marks r0 as precise */ +__msg ("mark_precise: frame0: regs=r0 stack= before 22: (b4) w0 = 0" ) +/* check that branch code path marks r0 as precise */ +__msg("mark_precise: frame0: regs=r0 stack= before 24: (85) call bpf_get_prandom_u32#7") +__msg("should have been in [0, 0]") +int BPF_PROG2 (test_bad_ret ,int ,a ) {int key =0 ;struct bpf_timer *timer;
@@ -39,7 +59,7 @@ int BPF_PROG2(test_ret_1, int, a) timer =bpf_map_lookup_elem (&timer_map, &key);if (timer) {bpf_timer_init (timer, &timer_map,CLOCK_BOOTTIME ); -bpf_timer_set_callback (timer, timer_cb_ret1); +bpf_timer_set_callback (timer, timer_cb_ret_bad);bpf_timer_start (timer,1000 ,0 ); }
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 9/10 selftests/bpf: adjust global_func15 test to validate prog exit precision
In reply to Andrii NakryikoAdd one more subtest to global_func15 selftest to validate that verifier properly marks r0 as precise and avoids erroneous state pruning of the branch that has return value outside of expected [0, 1] value.
--- .../selftests/bpf/progs/test_global_func15.c | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+)
tools/testing/selftests/bpf/progs/test_global_func15.c +32 -0
@@ -22,3 +22,35 @@ int global_func15(struct __sk_buff *skb)return v; } + +SEC("cgroup_skb/ingress") +__log_level(2) __flag(BPF_F_TEST_STATE_FREQ) +__failure +/* check that fallthrough code path marks r0 as precise */ +__msg ("mark_precise: frame0: regs=r0 stack= before 2: (b7) r0 = 1" ) +/* check that branch code path marks r0 as precise */ +__msg("mark_precise: frame0: regs=r0 stack= before 0: (85) call bpf_get_prandom_u32#7") +__msg("At program exit the register R0 has ") +__naked int global_func15_tricky_pruning (void ) +{ + asmvolatile ( +"call %[bpf_get_prandom_u32];" +"if r0 > 1000 goto 1f;" +"r0 = 1;" +"1:" +"goto +0;" /* checkpoint */ +/* cgroup_skb/ingress program is expected to return [0, 1] +* values, so branch above makes sure that in a fallthrough +* case we have a valid 1 stored in R0 register, but in +* a branch case we assign some random value to R0. So if +* there is something wrong with precision tracking for R0 at +* program exit, we might erronenously prune branch case, +* because R0 in fallthrough case is imprecise (and thus any +* value is valid from POV of verifier is_state_equal() logic) +*/ +"exit;" + : + : __imm(bpf_get_prandom_u32) + : __clobber_common + ); +}
Signature · 1 line
2.34.1
Andrii Nakryiko
maintainerPATCH v2 10/10 bpf: simplify tnum output if a fully known constant
In reply to Andrii NakryikoEmit tnum representation as just a constant if all bits are known. Use decimal-vs-hex logic to determine exact format of emitted constant value, just like it's done for register range values. For that move tnum_strn() to kernel/bpf/log.c to reuse decimal-vs-hex determination logic and constants.
--- kernel/bpf/log.c | 13 +++++++++++++ kernel/bpf/tnum.c | 6 ------ .../bpf/progs/verifier_direct_packet_access.c | 2 +- .../testing/selftests/bpf/progs/verifier_int_ptr.c | 2 +- .../selftests/bpf/progs/verifier_stack_ptr.c | 4 ++-- 5 files changed, 17 insertions(+), 10 deletions(-)
kernel/bpf/log.c +13 -0
@@ -539,6 +539,19 @@ static void verbose_snum(struct bpf_verifier_env *env, s64 num)verbose (env,"%#llx" , num); } +int tnum_strn (char *str,size_t size,struct tnum a) +{ +/* print as a constant, if tnum is fully known */ +if (a.mask ==0 ) { +if (is_unum_decimal (a.value)) +return snprintf (str, size,"%llu" , a.value); +else +return snprintf (str, size,"%#llx" , a.value); + } +return snprintf (str, size,"(%#llx; %#llx)" , a.value, a.mask); +} +EXPORT_SYMBOL_GPL (tnum_strn); +static void print_scalar_ranges (struct bpf_verifier_env *env,const struct bpf_reg_state *reg,const char **sep)
kernel/bpf/tnum.c +0 -6
@@ -172,12 +172,6 @@ bool tnum_in(struct tnum a, struct tnum b)return a.value == b.value; } -int tnum_strn (char *str,size_t size,struct tnum a) -{ -return snprintf (str, size,"(%#llx; %#llx)" , a.value, a.mask); -} -EXPORT_SYMBOL_GPL (tnum_strn); -int tnum_sbin (char *str,size_t size,struct tnum a) {size_t n;
tools/testing/selftests/bpf/progs/verifier_direct_packet_access.c +1 -1
@@ -411,7 +411,7 @@ l0_%=: r0 = 0; \ SEC("tc") __description("direct packet access: test17 (pruning, alignment)") -__failure __msg("misaligned packet access off 2+(0x0; 0x0)+15+-4 size 4") +__failure __msg("misaligned packet access off 2+0+15+-4 size 4")__flag (BPF_F_STRICT_ALIGNMENT) __naked void packet_access_test17_pruning_alignment(void) {
tools/testing/selftests/bpf/progs/verifier_int_ptr.c +1 -1
@@ -67,7 +67,7 @@ __naked void ptr_to_long_half_uninitialized(void) SEC("cgroup/sysctl") __description("ARG_PTR_TO_LONG misaligned") -__failure __msg("misaligned stack access off (0x0; 0x0)+-20+0 size 8") +__failure __msg("misaligned stack access off 0+-20+0 size 8") __naked void arg_ptr_to_long_misaligned(void) { asm volatile (" \
tools/testing/selftests/bpf/progs/verifier_stack_ptr.c +2 -2
@@ -37,7 +37,7 @@ __naked void ptr_to_stack_store_load(void) SEC("socket") __description("PTR_TO_STACK store/load - bad alignment on off") -__failure __msg("misaligned stack access off (0x0; 0x0)+-8+2 size 8") +__failure __msg("misaligned stack access off 0+-8+2 size 8") __failure_unpriv__naked void load_bad_alignment_on_off (void ) {
@@ -53,7 +53,7 @@ __naked void load_bad_alignment_on_off(void) SEC("socket") __description("PTR_TO_STACK store/load - bad alignment on reg") -__failure __msg("misaligned stack access off (0x0; 0x0)+-10+8 size 8") +__failure __msg("misaligned stack access off 0+-10+8 size 8") __failure_unpriv__naked void load_bad_alignment_on_reg (void ) {
Signature · 1 line
2.34.1
Quoted 6 lines · Andrii Nakryiko
This patch set fixes BPF verifier logic around validating and enforcing return values for BPF programs that have specific range of expected return values. Both sync and async callbacks have similar logic and are fixes as well. A few tests are added that would fail without the fixes in this patch set.
Also, while at it, we update retval checking logic to use umin/umax range
Looks like this should be change to smin/smax as well
Quoted 10 lines
instead of tnum, avoiding future potential issues if expected range cannot be represented precisely by tnum (e.g., [0, 2] is not representable by tnum and is treated as [0, 3]).
There is a little bit of refactoring to unify async callback and program exit logic to avoid duplication of checks as much as possible.
v1->v2:
- drop tnum from retval checks (Eduard);
- use smin/smax instead of umin/umax (Alexei).
...
Quoted 11 lines · Shung-Hsi Yu
Quoted 6 lines · Andrii Nakryiko
This patch set fixes BPF verifier logic around validating and enforcing return values for BPF programs that have specific range of expected return values. Both sync and async callbacks have similar logic and are fixes as well. A few tests are added that would fail without the fixes in this patch set.
Also, while at it, we update retval checking logic to use umin/umax range
Looks like this should be change to smin/smax as well
yep, thanks, I fixed up few more places where I missed umin/umax -> smin/smax updates
Quoted 12 lines
Quoted 10 lines
instead of tnum, avoiding future potential issues if expected range cannot be represented precisely by tnum (e.g., [0, 2] is not representable by tnum and is treated as [0, 3]).
There is a little bit of refactoring to unify async callback and program exit logic to avoid duplication of checks as much as possible.
v1->v2:
- drop tnum from retval checks (Eduard);
- use smin/smax instead of umin/umax (Alexei)....