bpf: Add bpf_rcu_read_lock() support
Subsystems (7)
v4 cover bpf: Add bpf_rcu_read_lock() support
Currently, without rcu attribute info in BTF, the verifier treats rcu tagged pointer as a normal pointer. This might be a problem for sleepable program where rcu_read_lock()/unlock() is not available. For example, for a sleepable fentry program, if rcu protected memory access is interleaved with a sleepable helper/kfunc, it is possible the memory access after the sleepable helper/kfunc might be invalid since the object might have been freed then. Even without
a sleepable helper/kfunc, without rcu_read_lock() protection, it is possible that the rcu protected object might be release in the middle of bpf program execution which may cause incorrect result.
To prevent above cases, enable btf_type_tag("rcu") attributes, introduce new bpf_rcu_read_lock/unlock() kfuncs and add verifier support.
In the rest of patch set, Patch 1 enabled btf_type_tag for __rcu attribute. Patche 2 is a refactoring patch. Patch 3 added new bpf_rcu_read_lock/unlock() kfuncs. Patch 4 added verifier support and Patch 5 enabled sleepable program support for cgrp local storage. Patch 6 added some tests for new helpers and verifier support and Patch 7 added new test to the deny list for s390x arch.
Changelogs:
v3 -> v4:
. fix selftest failures when built with gcc. gcc doesn't support
btf_type_tag yet and some tests relies on that. skip these tests if vmlinux BTF does not have btf_type_tag("rcu").
v2 -> v3:
. went back to MEM_RCU approach with invalidate rcu ptr registers
at bpf_rcu_read_unlock() place.
. remove KF_RCU_LOCK/UNLOCK flag and compare btf_id at verification
time instead.
v1 -> v2:
. use kfunc instead of helper for bpf_rcu_read_lock/unlock. . not use MEM_RCU bpf_type_flag, instead use active_rcu_lock
in reg state to identify rcu ptr's.
. Add more self tests.
. add new test to s390x deny list.
Yonghong Song (7):
compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
bpf: Abstract out functions to check sleepable helpers
bpf: Add kfunc bpf_rcu_read_lock/unlock()
bpf: Add bpf_rcu_read_lock() verifier support
bpf: Enable sleeptable support for cgrp local storage selftests/bpf: Add tests for bpf_rcu_read_lock() selftests/bpf: Add rcu_read_lock test to s390x deny list
include/linux/bpf.h | 6 + include/linux/bpf_lsm.h | 6 + include/linux/bpf_verifier.h | 4 + include/linux/compiler_types.h | 3 +- include/linux/trace_events.h | 8 + kernel/bpf/bpf_lsm.c | 20 +- kernel/bpf/btf.c | 39 +- kernel/bpf/helpers.c | 25 +- kernel/bpf/verifier.c | 125 +++++- kernel/trace/bpf_trace.c | 22 +- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/rcu_read_lock.c | 166 ++++++++ .../selftests/bpf/progs/rcu_read_lock.c | 355 ++++++++++++++++++ 13 files changed, 758 insertions(+), 22 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c
Signature · 1 line
2.30.2
PATCH v4 1/7 compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
Currently, without rcu attribute info in BTF, the verifier treats rcu tagged pointer as a normal pointer. This might be a problem for sleepable program where rcu_read_lock()/unlock() is not available. For example, for a sleepable fentry program, if rcu protected memory access is interleaved with a sleepable helper/kfunc, it is possible the memory access after the sleepable helper/kfunc might be invalid since the object might have been freed then. To prevent such cases, introducing rcu tagging for memory accesses in verifier can help to reject such programs.
To enable rcu tagging in BTF, during kernel compilation,
define __rcu as attribute btf_type_tag("rcu") so __rcu information can be preserved in dwarf and btf, and later can be used for bpf prog verification.
--- include/linux/compiler_types.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/compiler_types.h +2 -1
@@ -49,7 +49,8 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }# endif # define __iomem# define __percpu BTF_TYPE_TAG(percpu) -# define __rcu +# define __rcu BTF_TYPE_TAG(rcu) +# define __chk_user_ptr (x) (void)0# define __chk_io_ptr (x) (void)0/* context/locking */
Signature · 1 line
2.30.2
PATCH v4 2/7 bpf: Abstract out functions to check sleepable helpers
In reply to Yonghong SongAbstract out two functions to check whether a particular helper is sleepable or not for bpf_lsm and bpf_trace. These two functions will be used later to check whether a helper is sleepable or not in verifier. There is no functionality change.
--- include/linux/bpf_lsm.h | 6 ++++++ include/linux/trace_events.h | 8 ++++++++ kernel/bpf/bpf_lsm.c | 20 ++++++++++++++++---- kernel/trace/bpf_trace.c | 22 ++++++++++++++++++---- 4 files changed, 48 insertions(+), 8 deletions(-)
include/linux/bpf_lsm.h +6 -0
@@ -28,6 +28,7 @@ int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog, const struct bpf_prog *prog);bool bpf_lsm_is_sleepable_hook (u32 btf_id); +const struct bpf_func_proto *bpf_lsm_sleepable_func_proto (enum bpf_func_id func_id);static inline struct bpf_storage_blob *bpf_inode (const struct inode *inode)
@@ -50,6 +51,11 @@ static inline bool bpf_lsm_is_sleepable_hook(u32 btf_id) {return false; } +static inline const struct bpf_func_proto * +bpf_lsm_sleepable_func_proto (enum bpf_func_id func_id) +{ +return NULL ; +}static inline int bpf_lsm_verify_prog (struct bpf_verifier_log *vlog,const struct bpf_prog *prog)
include/linux/trace_events.h +8 -0
@@ -9,6 +9,7 @@#include <linux/hardirq.h> #include <linux/perf_event.h> #include <linux/tracepoint.h> +#include <uapi/linux/bpf.h> struct trace_array ;struct array_buffer ;
@@ -16,6 +17,7 @@ struct tracer;struct dentry ;struct bpf_prog ;union bpf_attr ; +struct bpf_func_proto ; const char *trace_print_flags_seq(struct trace_seq *p, const char *delim, unsigned long flags,
@@ -748,6 +750,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id, u32 *fd_type, const char **buf, u64 *probe_offset, u64 *probe_addr);int bpf_kprobe_multi_link_attach (const union bpf_attr *attr,struct bpf_prog *prog); +const struct bpf_func_proto *bpf_tracing_sleepable_func_proto (enum bpf_func_id func_id);#else static inline unsigned int trace_call_bpf (struct trace_event_call *call,void *ctx) {
@@ -794,6 +797,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) {return -EOPNOTSUPP ; } +static inline const struct bpf_func_proto * +bpf_tracing_sleepable_func_proto (enum bpf_func_id func_id) +{ +return NULL ; +}#endif enum {
kernel/bpf/bpf_lsm.c +16 -4
@@ -192,6 +192,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto = { .arg1_type = ARG_PTR_TO_CTX, }; +const struct bpf_func_proto *bpf_lsm_sleepable_func_proto (enum bpf_func_id func_id) +{ +switch (func_id) { +case BPF_FUNC_ima_inode_hash: +return &bpf_ima_inode_hash_proto; +case BPF_FUNC_ima_file_hash: +return &bpf_ima_file_hash_proto; +default : +return NULL ; + } +} +static const struct bpf_func_proto *bpf_lsm_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog) {
@@ -203,6 +215,10 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return func_proto; } + func_proto =bpf_lsm_sleepable_func_proto (func_id); +if (func_proto) +return prog->aux->sleepable ? func_proto :NULL ; +switch (func_id) {case BPF_FUNC_inode_storage_get:return &bpf_inode_storage_get_proto;
@@ -220,10 +236,6 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_spin_unlock_proto;case BPF_FUNC_bprm_opts_set:return &bpf_bprm_opts_set_proto; -case BPF_FUNC_ima_inode_hash: -return prog->aux->sleepable ? &bpf_ima_inode_hash_proto :NULL ; -case BPF_FUNC_ima_file_hash: -return prog->aux->sleepable ? &bpf_ima_file_hash_proto :NULL ;case BPF_FUNC_get_attach_cookie:return bpf_prog_has_trampoline (prog) ? &bpf_get_attach_cookie_proto :NULL ;#ifdef CONFIG_NET
kernel/trace/bpf_trace.c +18 -4
@@ -1383,9 +1383,27 @@ static int __init bpf_key_sig_kfuncs_init(void)late_initcall (bpf_key_sig_kfuncs_init);#endif /* CONFIG_KEYS */ +const struct bpf_func_proto *bpf_tracing_sleepable_func_proto (enum bpf_func_id func_id) +{ +switch (func_id) { +case BPF_FUNC_copy_from_user: +return &bpf_copy_from_user_proto; +case BPF_FUNC_copy_from_user_task: +return &bpf_copy_from_user_task_proto; +default : +return NULL ; + } +} +static const struct bpf_func_proto *bpf_tracing_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog) { +const struct bpf_func_proto *func_proto; + + func_proto =bpf_tracing_sleepable_func_proto (func_id); +if (func_proto) +return prog->aux->sleepable ? func_proto :NULL ; +switch (func_id) {case BPF_FUNC_map_lookup_elem:return &bpf_map_lookup_elem_proto;
@@ -1484,10 +1502,6 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_jiffies64_proto;case BPF_FUNC_get_task_stack:return &bpf_get_task_stack_proto; -case BPF_FUNC_copy_from_user: -return prog->aux->sleepable ? &bpf_copy_from_user_proto :NULL ; -case BPF_FUNC_copy_from_user_task: -return prog->aux->sleepable ? &bpf_copy_from_user_task_proto :NULL ;case BPF_FUNC_snprintf_btf:return &bpf_snprintf_btf_proto;case BPF_FUNC_per_cpu_ptr:
Signature · 1 line
2.30.2
PATCH v4 3/7 bpf: Add kfunc bpf_rcu_read_lock/unlock()
In reply to Yonghong SongAdd two kfunc's bpf_rcu_read_lock() and bpf_rcu_read_unlock(). These two kfunc's can be used for all program types. A new kfunc hook type BTF_KFUNC_HOOK_GENERIC is added which corresponds to prog type BPF_PROG_TYPE_UNSPEC, indicating the kfunc intends to be used for all prog types.
--- include/linux/bpf.h | 3 +++ kernel/bpf/btf.c | 8 ++++++++ kernel/bpf/helpers.c | 25 ++++++++++++++++++++++++- 3 files changed, 35 insertions(+), 1 deletion(-)
include/linux/bpf.h +3 -0
@@ -2113,6 +2113,9 @@ bool bpf_prog_has_kfunc_call(const struct bpf_prog *prog);const struct btf_func_model *bpf_jit_find_kfunc_model (const struct bpf_prog *prog,const struct bpf_insn *insn); +void bpf_rcu_read_lock (void ); +void bpf_rcu_read_unlock (void ); +struct bpf_core_ctx {struct bpf_verifier_log *log;const struct btf *btf;
kernel/bpf/btf.c +8 -0
@@ -199,6 +199,7 @@ DEFINE_IDR(btf_idr);DEFINE_SPINLOCK (btf_idr_lock);enum btf_kfunc_hook { +BTF_KFUNC_HOOK_GENERIC ,BTF_KFUNC_HOOK_XDP ,BTF_KFUNC_HOOK_TC ,BTF_KFUNC_HOOK_STRUCT_OPS ,
@@ -7499,6 +7500,8 @@ static u32 *__btf_kfunc_id_set_contains(const struct btf *btf,static int bpf_prog_type_to_kfunc_hook (enum bpf_prog_type prog_type) {switch (prog_type) { +case BPF_PROG_TYPE_UNSPEC : +return BTF_KFUNC_HOOK_GENERIC ;case BPF_PROG_TYPE_XDP :return BTF_KFUNC_HOOK_XDP ;case BPF_PROG_TYPE_SCHED_CLS :
@@ -7527,6 +7530,11 @@ u32 *btf_kfunc_id_set_contains(const struct btf *btf, u32 kfunc_btf_id) {enum btf_kfunc_hook hook; +u32 *kfunc_flags; + + kfunc_flags =__btf_kfunc_id_set_contains (btf,BTF_KFUNC_HOOK_GENERIC , kfunc_btf_id); +if (kfunc_flags) +return kfunc_flags; hook =bpf_prog_type_to_kfunc_hook (prog_type);return __btf_kfunc_id_set_contains (btf, hook, kfunc_btf_id);
kernel/bpf/helpers.c +24 -1
@@ -1717,9 +1717,32 @@ static const struct btf_kfunc_id_set tracing_kfunc_set = { .set = &tracing_btf_ids, }; +void bpf_rcu_read_lock (void ) +{ +rcu_read_lock (); +} + +void bpf_rcu_read_unlock (void ) +{ +rcu_read_unlock (); +} + +BTF_SET8_START (generic_btf_ids) +BTF_ID_FLAGS (func ,bpf_rcu_read_lock ) +BTF_ID_FLAGS (func, bpf_rcu_read_unlock) +BTF_SET8_END (generic_btf_ids) + +static const struct btf_kfunc_id_set generic_kfunc_set = { + .owner =THIS_MODULE , + .set = &generic_btf_ids, +}; +static int __init kfunc_init (void ) { -return register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &tracing_kfunc_set); +int ret; + + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &tracing_kfunc_set); +return ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_UNSPEC , &generic_kfunc_set); }late_initcall (kfunc_init);
Signature · 1 line
2.30.2
PATCH v4 4/7 bpf: Add bpf_rcu_read_lock() verifier support
In reply to Yonghong SongTo simplify the design and support the common practice, no nested bpf_rcu_read_lock() is allowed. A new bpf_type_flag MEM_RCU is added to indicate a PTR_TO_BTF_ID object access needing rcu_read_lock protection. Note that rcu protection is not needed for non-sleepable program, but it is supported to make cross-sleepable/nonsleepable development easier. For sleepable program, the following insns can be inside the rcu lock region:
- any non call insns except BPF_ABS/BPF_IND
- non sleepable helpers or kfuncs
The rcu pointer will be invalidated at bpf_rcu_read_unlock() so it cannot be used outside the current rcu read lock region. Also, bpf_*_storage_get() helper's 5th hidden argument (for memory allocation flag) should be GFP_ATOMIC.
If a pointer (PTR_TO_BTF_ID) is marked as rcu, then any use of this pointer and the load which gets this pointer needs to be protected by bpf_rcu_read_lock(). The following shows a couple of examples:
struct task_struct {
...
struct task_struct __rcu *real_parent;
struct css_set __rcu *cgroups;
...
};
struct css_set {
...
struct cgroup *dfl_cgrp;
... }
...
task = bpf_get_current_task_btf();
cgroups = task->cgroups;
dfl_cgroup = cgroups->dfl_cgrp;
... using dfl_cgroup ...
The bpf_rcu_read_lock/unlock() should be added like below to avoid verification failures.
task = bpf_get_current_task_btf(); bpf_rcu_read_lock();
cgroups = task->cgroups;
dfl_cgroup = cgroups->dfl_cgrp; bpf_rcu_read_unlock();
... using dfl_cgroup ...
The following is another example for task->real_parent.
task = bpf_get_current_task_btf(); bpf_rcu_read_lock();
real_parent = task->real_parent;
... bpf_task_storage_get(&map, real_parent, 0, 0); bpf_rcu_read_unlock();
There is another case observed in selftest bpf_iter_ipv6_route.c:
struct fib6_info *rt = ctx->rt;
...
fib6_nh = &rt->fib6_nh[0]; // Not rcu protected
...
if (rt->nh)
fib6_nh = &nh->nh_info->fib6_nh; // rcu protected
...
... using fib6_nh ...
Note that the use of fib6_nh is tag with rcu in one path but not in the other path. Current verification will fail since the same insn cannot be used with different pointer types. The above use case is a valid one so the verifier is changed to ignore MEM_RCU type tag
in such cases.
--- include/linux/bpf.h | 3 + include/linux/bpf_verifier.h | 4 ++ kernel/bpf/btf.c | 31 ++++++++- kernel/bpf/verifier.c | 122 ++++++++++++++++++++++++++++++++--- 4 files changed, 149 insertions(+), 11 deletions(-)
include/linux/bpf.h +3 -0
@@ -510,6 +510,9 @@ enum bpf_type_flag {/* Size is known at compile time. */ MEM_FIXED_SIZE =BIT (10 +BPF_BASE_TYPE_BITS ), +/* MEM is tagged with rcu and memory access needs rcu_read_lock protection. */ +MEM_RCU =BIT (11 +BPF_BASE_TYPE_BITS ), + __BPF_TYPE_FLAG_MAX, __BPF_TYPE_LAST_FLAG = __BPF_TYPE_FLAG_MAX - 1, };
include/linux/bpf_verifier.h +4 -0
@@ -325,6 +325,7 @@ struct bpf_verifier_state {u32 curframe;u32 active_spin_lock;bool speculative; +bool active_rcu_lock;/* first and last insn idx of this verifier state */ u32 first_insn_idx;
@@ -424,6 +425,7 @@ struct bpf_insn_aux_data {u32 seen;/* this insn was processed by the verifier at env->pass_cnt */ bool sanitize_stack_spill;/* subject to Spectre v4 sanitation */ bool zext_dst;/* this insn zero extends dst reg */ +bool storage_get_func_atomic;/* bpf_*_storage_get() with atomic memory alloc */ u8 alu_state;/* used in combination with alu_limit */ /* below fields are initialized once */
@@ -627,6 +629,8 @@ void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab);int mark_chain_precision (struct bpf_verifier_env *env,int regno); +void clear_all_rcu_pointers (struct bpf_verifier_env *env); +#define BPF_BASE_TYPE_MASK GENMASK(BPF_BASE_TYPE_BITS - 1, 0)/* extract base type from bpf_{arg, return, reg}_type. */
kernel/bpf/btf.c +30 -1
@@ -5850,6 +5850,9 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,/* check __percpu tag */ if (strcmp (tag_value,"percpu" ) ==0 ) tmp_flag =MEM_PERCPU ; +/* check __rcu tag */ +if (strcmp (tag_value,"rcu" ) ==0 ) + tmp_flag =MEM_RCU ; } stype =btf_type_skip_modifiers (btf, mtype->type, &id);
@@ -6309,6 +6312,9 @@ static bool btf_is_kfunc_arg_mem_size(const struct btf *btf,return true; } +BTF_ID_LIST_SINGLE(bpf_rcu_read_lock_id, func, bpf_rcu_read_lock) +BTF_ID_LIST_SINGLE(bpf_rcu_read_unlock_id, func, bpf_rcu_read_unlock) + static int btf_check_func_arg_match(struct bpf_verifier_env *env, const struct btf *btf, u32 func_id, struct bpf_reg_state *regs,
@@ -6318,7 +6324,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, {enum bpf_prog_type prog_type =resolve_prog_type (env->prog);bool rel = false, kptr_get = false, trusted_args = false; -bool sleepable = false; +bool sleepable = false, rcu_lock = false, rcu_unlock = false;struct bpf_verifier_log *log = &env->log;u32 i, nargs, ref_id, ref_obj_id =0 ;bool is_kfunc =btf_is_kernel (btf);
@@ -6327,6 +6333,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,const struct btf_param *args;int ref_regno =0 , ret; + t =btf_type_by_id (btf, func_id);if (!t || !btf_type_is_func (t)) {/* These checks were already done by the verifier while loading
@@ -6357,6 +6364,28 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, kptr_get = kfunc_meta->flags &KF_KPTR_GET ; trusted_args = kfunc_meta->flags &KF_TRUSTED_ARGS ; sleepable = kfunc_meta->flags &KF_SLEEPABLE ; + rcu_lock = func_id == *bpf_rcu_read_lock_id; + rcu_unlock = func_id == *bpf_rcu_read_unlock_id; + } + +/* checking rcu read lock/unlock */ +if (env->cur_state->active_rcu_lock) { +if (rcu_lock) { +bpf_log (log,"nested rcu read lock (kernel function %s)\n" , func_name); +return -EINVAL ; + }else if (rcu_unlock) { +clear_all_rcu_pointers (env); + env->cur_state->active_rcu_lock = false; + }else if (sleepable) { +bpf_log (log,"kernel func %s is sleepable within rcu_read_lock region\n" , + func_name); +return -EINVAL ; + } + }else if (rcu_lock) { + env->cur_state->active_rcu_lock = true; + }else if (rcu_unlock) { +bpf_log (log,"unmatched rcu read unlock (kernel function %s)\n" , func_name); +return -EINVAL ; }/* check that BTF function arguments match actual types that the
kernel/bpf/verifier.c +112 -10
243 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v4 5/7 bpf: Enable sleeptable support for cgrp local storage
In reply to Yonghong SongWith proper bpf_rcu_read_lock() support, sleepable support for cgrp local storage can be enabled as typical use case task->cgroups->dfl_cgrp can be protected with bpf_rcu_read_lock().
--- kernel/bpf/verifier.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +2 -1
@@ -12958,10 +12958,11 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env,case BPF_MAP_TYPE_INODE_STORAGE :case BPF_MAP_TYPE_SK_STORAGE :case BPF_MAP_TYPE_TASK_STORAGE : +case BPF_MAP_TYPE_CGRP_STORAGE :break ;default :verbose (env, -"Sleepable programs can only use array, hash, and ringbuf maps\n" ); +"Sleepable programs can only use array, hash, ringbuf and local storage maps\n" );return -EINVAL ; }
Signature · 1 line
2.30.2
PATCH v4 6/7 selftests/bpf: Add tests for bpf_rcu_read_lock()
In reply to Yonghong SongAdd a few positive/negative tests to test bpf_rcu_read_lock() and its corresponding verifier support.
--- .../selftests/bpf/prog_tests/rcu_read_lock.c | 166 ++++++++ .../selftests/bpf/progs/rcu_read_lock.c | 355 ++++++++++++++++++ 2 files changed, 521 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c
tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c new file +166 -0
@@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates.*/ + +#define _GNU_SOURCE +#include <unistd.h> +#include <sys/syscall.h> +#include <sys/types.h> +#include <test_progs.h> +#include <bpf/btf.h> +#include "rcu_read_lock.skel.h" + +static void test_local_storage (void ) +{ +struct rcu_read_lock *skel; +int err; + + skel =rcu_read_lock__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) +return ; + + skel->bss->target_pid =syscall (SYS_gettid); + +bpf_program__set_autoload (skel->progs.cgrp_succ, true); +bpf_program__set_autoload (skel->progs.task_succ, true); +bpf_program__set_autoload (skel->progs.two_regions, true); +bpf_program__set_autoload (skel->progs.non_sleepable_1, true); +bpf_program__set_autoload (skel->progs.non_sleepable_2, true); + err =rcu_read_lock__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto done; + + err =rcu_read_lock__attach (skel); +if (!ASSERT_OK (err,"skel_attach" )) + goto done; + +syscall (SYS_getpgid); + +ASSERT_EQ (skel->bss->result,2 ,"result" ); +done: +rcu_read_lock__destroy (skel); +} + +static void test_runtime_diff_rcu_tag (void ) +{ +struct rcu_read_lock *skel; +int err; + + skel =rcu_read_lock__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) +return ; + +bpf_program__set_autoload (skel->progs.dump_ipv6_route, true); + err =rcu_read_lock__load (skel); +ASSERT_OK (err,"skel_load" ); +rcu_read_lock__destroy (skel); +} + +static void test_negative_region (void ) +{ +#define NUM_REGION_FAILED_PROGS 6 +struct rcu_read_lock *skel; +struct bpf_program *prog; +int i, err; + +for (i =0 ; i <NUM_REGION_FAILED_PROGS ; i++) { + skel =rcu_read_lock__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) +return ; + +switch (i) { +case 0 : + prog = skel->progs.miss_lock; +break ; +case 1 : + prog = skel->progs.miss_unlock; +break ; +case 2 : + prog = skel->progs.non_sleepable_rcu_mismatch; +break ; +case 3 : + prog = skel->progs.inproper_sleepable_helper; +break ; +case 4 : + prog = skel->progs.inproper_sleepable_kfunc; +break ; +default : + prog = skel->progs.nested_rcu_region; +break ; + } + +bpf_program__set_autoload (prog, true); + err =rcu_read_lock__load (skel); +if (!ASSERT_ERR (err,"skel_load" )) { +rcu_read_lock__destroy (skel); +return ; + } + } +} + +static void test_negative_rcuptr_misuse (void ) +{ +#define NUM_RCUPTR_FAILED_PROGS 4 +struct rcu_read_lock *skel; +struct bpf_program *prog; +struct btf *vmlinux_btf; +int i, err, type_id; + + vmlinux_btf =btf__load_vmlinux_btf (); +if (!ASSERT_OK_PTR (vmlinux_btf,"could not load vmlinux BTF" )) +return ; + +/* skip the test if btf_type_tag("rcu") is not present in vmlinux */ + type_id =btf__find_by_name_kind (vmlinux_btf,"rcu" ,BTF_KIND_TYPE_TAG ); +if (type_id <0 ) { +test__skip (); +return ; + } + +for (i =0 ; i <NUM_RCUPTR_FAILED_PROGS ; i++) { + skel =rcu_read_lock__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) +return ; + +switch (i) { +case 0 : + prog = skel->progs.cgrp_incorrect_rcu_region; +break ; +case 1 : + prog = skel->progs.task_incorrect_rcu_region1; +break ; +case 2 : + prog = skel->progs.task_incorrect_rcu_region2; +break ; +default : + prog = skel->progs.cross_rcu_region; +break ; + } + +bpf_program__set_autoload (prog, true); + err =rcu_read_lock__load (skel); +if (!ASSERT_ERR (err,"skel_load" )) { +rcu_read_lock__destroy (skel); +return ; + } + } +} + +void test_rcu_read_lock (void ) +{ +int cgroup_fd; + + cgroup_fd =test__join_cgroup ("/rcu_read_lock" ); +if (!ASSERT_GE (cgroup_fd,0 ,"join_cgroup /rcu_read_lock" )) +return ; + +if (test__start_subtest ("local_storage" )) +test_local_storage (); +if (test__start_subtest ("runtime_diff_rcu_tag" )) +test_runtime_diff_rcu_tag (); +if (test__start_subtest ("negative_tests_region" )) +test_negative_region (); +if (test__start_subtest ("negative_tests_rcuptr_misuse" )) +test_negative_rcuptr_misuse (); + +close (cgroup_fd); +}
tools/testing/selftests/bpf/progs/rcu_read_lock.c new file +355 -0
355 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v4 7/7 selftests/bpf: Add rcu_read_lock test to s390x deny list
In reply to Yonghong SongThe new rcu_read_lock test will fail on s390x with the following error message:
...
test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec test_local_storage:PASS:skel_open 0 nsec
libbpf: prog 'cgrp_succ': failed to find kernel BTF type ID of '__s390x_sys_getpgid': -3
libbpf: prog 'cgrp_succ': failed to prepare load attributes: -3
libbpf: prog 'cgrp_succ': failed to load: -3
libbpf: failed to load object 'rcu_read_lock'
libbpf: failed to load BPF skeleton 'rcu_read_lock': -3 test_local_storage:FAIL:skel_load unexpected error: -3 (errno 3)
...
So add it to the s390x deny list.
--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + 1 file changed, 1 insertion(+)
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -41,6 +41,7 @@ module_attach # skel_attach skeleton attach failed: - mptcp netcnt # failed to load BPF skeleton 'netcnt_prog': -7 (?) probe_user # check_kprobe_res wrong kprobe res from probe read (?) +rcu_read_lock # failed to find kernel BTF type ID of '__x64_sys_getpgid': -3 (?) recursion # skel_attach unexpected error: -524 (trampoline) ringbuf # skel_load skeleton load failed (?) select_reuseport # intermittently fails on new s390x setup
Signature · 1 line
2.30.2
Re: [PATCH bpf-next v4 7/7] selftests/bpf: Add rcu_read_lock test to s390x deny list
Quoted 25 lines · Yonghong Song
The new
rcu_read_locktest will fail on s390x with the following error message:...
test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec test_local_storage:PASS:skel_open 0 nsec
libbpf: prog 'cgrp_succ': failed to find kernel BTF type ID of '__s390x_sys_getpgid': -3
libbpf: prog 'cgrp_succ': failed to prepare load attributes: -3
libbpf: prog 'cgrp_succ': failed to load: -3
libbpf: failed to load object 'rcu_read_lock'
libbpf: failed to load BPF skeleton 'rcu_read_lock': -3 test_local_storage:FAIL:skel_load unexpected error: -3 (errno 3)
...So add it to the s390x deny list.
Signed-off-by Yonghong Song <[email protected]>--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + 1 file changed, 1 insertion(+)tools/testing/selftests/bpf/DENYLIST.s390x +0
-0
tools/testing/selftests/bpf/DENYLIST.s390x module_attach # skel_attach skeleton attach failed: - · line 44
Quoted 4 lines
tools/testing/selftests/bpf/DENYLIST.s390x +1
-0@@ -41,6 +41,7 @@ module_attach # skel_attach skeleton attach failed: - mptcp netcnt # failed to load BPF skeleton 'netcnt_prog': -7 (?) probe_user # check_kprobe_res wrong kprobe res from probe read (?) +rcu_read_lock # failed to find kernel BTF type ID of '__x64_sys_getpgid': -3 (?)
This also seems to fail on aarch64:
``` 2022-11-10T18:39:39.2406543Z test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec 2022-11-10T18:39:39.2409781Z test_local_storage:PASS:skel_open 0 nsec 2022-11-10T18:39:39.2413002Z test_local_storage:PASS:skel_load 0 nsec 2022-11-10T18:39:39.2418758Z libbpf: prog 'cgrp_succ': failed to attach: ERROR: strerror_r(-524)=22 2022-11-10T18:39:39.2422765Z libbpf: prog 'cgrp_succ': failed to auto-attach: -524 2022-11-10T18:39:39.2428250Z test_local_storage:FAIL:skel_attach unexpected error: -524 (errno 524) 2022-11-10T18:39:39.2431555Z #145/1 rcu_read_lock/local_storage:FAIL 2022-11-10T18:39:39.2435392Z #145/2 rcu_read_lock/runtime_diff_rcu_tag:OK 2022-11-10T18:39:39.2439296Z #145/3 rcu_read_lock/negative_tests_region:OK 2022-11-10T18:39:39.2443876Z #145/4 rcu_read_lock/negative_tests_rcuptr_misuse:SKIP 2022-11-10T18:39:39.2446212Z #145 rcu_read_lock:FAIL ```
Can you add the test to DENYLIST.aarch64 also?
Quoted 6 lines
tools/testing/selftests/bpf/DENYLIST.s390x +0
-0recursion # skel_attach unexpected error: -524 (trampoline) ringbuf # skel_load skeleton load failed (?) select_reuseport # intermittently fails on new s390x setupSignature · 1 line
2.30.2
Re: [PATCH bpf-next v4 7/7] selftests/bpf: Add rcu_read_lock test to s390x deny list
Quoted 46 lines · Manu Bretelle
Quoted 25 lines · Yonghong Song
The new
rcu_read_locktest will fail on s390x with the following error message:...
test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec test_local_storage:PASS:skel_open 0 nsec
libbpf: prog 'cgrp_succ': failed to find kernel BTF type ID of '__s390x_sys_getpgid': -3
libbpf: prog 'cgrp_succ': failed to prepare load attributes: -3
libbpf: prog 'cgrp_succ': failed to load: -3
libbpf: failed to load object 'rcu_read_lock'
libbpf: failed to load BPF skeleton 'rcu_read_lock': -3 test_local_storage:FAIL:skel_load unexpected error: -3 (errno 3)
...So add it to the s390x deny list.
Signed-off-by Yonghong Song <[email protected]>--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + 1 file changed, 1 insertion(+)tools/testing/selftests/bpf/DENYLIST.s390x +0
-0tools/testing/selftests/bpf/DENYLIST.s390x module_attach # skel_attach skeleton attach failed: - · line 44
Quoted 4 lines
tools/testing/selftests/bpf/DENYLIST.s390x +1
-0@@ -41,6 +41,7 @@ module_attach # skel_attach skeleton attach failed: - mptcp netcnt # failed to load BPF skeleton 'netcnt_prog': -7 (?) probe_user # check_kprobe_res wrong kprobe res from probe read (?) +rcu_read_lock # failed to find kernel BTF type ID of '__x64_sys_getpgid': -3 (?)This also seems to fail on aarch64:
``` 2022-11-10T18:39:39.2406543Z test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec 2022-11-10T18:39:39.2409781Z test_local_storage:PASS:skel_open 0 nsec 2022-11-10T18:39:39.2413002Z test_local_storage:PASS:skel_load 0 nsec 2022-11-10T18:39:39.2418758Z libbpf: prog 'cgrp_succ': failed to attach: ERROR: strerror_r(-524)=22 2022-11-10T18:39:39.2422765Z libbpf: prog 'cgrp_succ': failed to auto-attach: -524 2022-11-10T18:39:39.2428250Z test_local_storage:FAIL:skel_attach unexpected error: -524 (errno 524) 2022-11-10T18:39:39.2431555Z #145/1 rcu_read_lock/local_storage:FAIL 2022-11-10T18:39:39.2435392Z #145/2 rcu_read_lock/runtime_diff_rcu_tag:OK 2022-11-10T18:39:39.2439296Z #145/3 rcu_read_lock/negative_tests_region:OK 2022-11-10T18:39:39.2443876Z #145/4 rcu_read_lock/negative_tests_rcuptr_misuse:SKIP 2022-11-10T18:39:39.2446212Z #145 rcu_read_lock:FAIL ```
Quoted 1 line
Can you add the test to DENYLIST.aarch64 also?
Thanks. Will add to the next revision.
Quoted 8 lines
Quoted 8 lines
tools/testing/selftests/bpf/DENYLIST.s390x +0
-0recursion # skel_attach unexpected error: -524 (trampoline) ringbuf # skel_load skeleton load failed (?) select_reuseport # intermittently fails on new s390x setupSignature · 1 line
2.30.2