Skip to contentSkip to conversation

bpf: Add bpf_rcu_read_lock() support

PATCH Applied Yonghong Song 4 participants 10 messages Nov 24, 2022 – Nov 25, 2022

Subsystems (7)
Kselftest BPF Tracing BPF selftests +3 more subsystems
lore
Share & tools
lore
#0 lore raw

v10 cover bpf: Add bpf_rcu_read_lock() support

Currently, without rcu attribute info in BTF, the verifier treats rcu tagged pointer as a normal pointer. This might be a problem for sleepable program where rcu_read_lock()/unlock() is not available. For example, for a sleepable fentry program, if rcu protected memory access is interleaved with a sleepable helper/kfunc, it is possible the memory access after the sleepable helper/kfunc might be invalid since the object might have been freed then. Even without
a sleepable helper/kfunc, without rcu_read_lock() protection, it is possible that the rcu protected object might be release in the middle of bpf program execution which may cause incorrect result.

To prevent above cases, enable btf_type_tag("rcu") attributes, introduce new bpf_rcu_read_lock/unlock() kfuncs and add verifier support.

In the rest of patch set, Patch 1 enabled btf_type_tag for __rcu attribute. Patche 2 added might_sleep in bpf_func_proto. Patch 3 added new bpf_rcu_read_lock/unlock() kfuncs and verifier support.
Patch 4 added some tests for these two new kfuncs.

Changelogs:
v9 -> v10:
. if no rcu tag support in vmlinux btf, using bpf_rcu_read_lock/unlock()
will cause verification error.
. at bpf_rcu_read_unlock(), invalidate rcu ptr to PTR_UNTRUSTED
instead of SCALAR_VALUE.
. a few other comment changes and other minor changes.
v8 -> v9:
. remove sleepable prog check for ld_abs/ind checking in rcu read
lock region.
. fix a test failure with gcc-compiled kernel.
. a couple of other minor fixes.
v7 -> v8:
. add might_sleep in bpf_func_proto so we can easily identify whether
a helper is sleepable or not.
. do not enforce rcu rules for sleepable, e.g., rcu dereference must
be in a bpf_rcu_read_lock region. This is to keep old code working fine.
. Mark 'b' in 'b = a->b' (b is tagged with __rcu) as MEM_RCU only if
'b = a->b' in rcu read region and 'a' is trusted. This adds safety guarantee for 'b' inside the rcu read region.
v6 -> v7:
. rebase on top of bpf-next.
. remove the patch which enables sleepable program using
cgrp_local_storage map. This is orthogonal to this patch set and will be addressed separately.
. mark the rcu pointer dereference result as UNTRUSTED if inside
a bpf_rcu_read_lock() region.
v5 -> v6:
. fix selftest prog miss_unlock which tested nested locking. . add comments in selftest prog cgrp_succ to explain how to handle
nested memory access after rcu memory load.
v4 -> v5:
. add new test to aarch64 deny list.
v3 -> v4:
. fix selftest failures when built with gcc. gcc doesn't support
btf_type_tag yet and some tests relies on that. skip these tests if vmlinux BTF does not have btf_type_tag("rcu").
v2 -> v3:
. went back to MEM_RCU approach with invalidate rcu ptr registers
at bpf_rcu_read_unlock() place.
. remove KF_RCU_LOCK/UNLOCK flag and compare btf_id at verification
time instead.
v1 -> v2:
. use kfunc instead of helper for bpf_rcu_read_lock/unlock. . not use MEM_RCU bpf_type_flag, instead use active_rcu_lock
in reg state to identify rcu ptr's.
. Add more self tests.
. add new test to s390x deny list.

Yonghong Song (4):
compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
bpf: Introduce might_sleep field in bpf_func_proto
bpf: Add kfunc bpf_rcu_read_lock/unlock()
selftests/bpf: Add tests for bpf_rcu_read_lock()

 include/linux/bpf.h                           |   4 +
 include/linux/bpf_verifier.h                  |   5 +-
 include/linux/compiler_types.h                |   3 +-
 kernel/bpf/bpf_lsm.c                          |   6 +-
 kernel/bpf/btf.c                              |   3 +
 kernel/bpf/helpers.c                          |  14 +
 kernel/bpf/verifier.c                         | 167 ++++++++--
 kernel/trace/bpf_trace.c                      |   4 +-
 tools/testing/selftests/bpf/DENYLIST.aarch64  |   1 +
 tools/testing/selftests/bpf/DENYLIST.s390x    |   1 +
 .../selftests/bpf/prog_tests/rcu_read_lock.c  | 158 ++++++++++
 .../selftests/bpf/progs/rcu_read_lock.c       | 290 ++++++++++++++++++
 12 files changed, 621 insertions(+), 35 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
 create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c
Signature · 1 line
2.30.2
#1 lore raw

PATCH v10 1/4 compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))

Currently, without rcu attribute info in BTF, the verifier treats rcu tagged pointer as a normal pointer. This might be a problem for sleepable program where rcu_read_lock()/unlock() is not available. For example, for a sleepable fentry program, if rcu protected memory access is interleaved with a sleepable helper/kfunc, it is possible the memory access after the sleepable helper/kfunc might be invalid since the object might have been freed then. To prevent such cases, introducing rcu tagging for memory accesses in verifier can help to reject such programs.

To enable rcu tagging in BTF, during kernel compilation,
define __rcu as attribute btf_type_tag("rcu") so __rcu information can be preserved in dwarf and btf, and later can be used for bpf prog verification.

---
 include/linux/compiler_types.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/compiler_types.h +2 -1
index eb0466236661..7c1afe0f4129 100644
@@ -49,7 +49,8 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 # endif
 # define __iomem
 # define __percpu	BTF_TYPE_TAG(percpu)
-# define __rcu
+# define __rcu		BTF_TYPE_TAG(rcu)
+
 # define __chk_user_ptr(x)	(void)0
 # define __chk_io_ptr(x)	(void)0
 /* context/locking */
Signature · 1 line
2.30.2
#2 lore raw

PATCH v10 2/4 bpf: Introduce might_sleep field in bpf_func_proto

In reply to Yonghong Song

Introduce bpf_func_proto->might_sleep to indicate a particular helper might sleep. This will make later check whether a helper might be sleepable or not easier.

---
 include/linux/bpf.h      | 1 +
 kernel/bpf/bpf_lsm.c     | 6 ++++--
 kernel/bpf/helpers.c     | 2 ++
 kernel/bpf/verifier.c    | 5 +++++
 kernel/trace/bpf_trace.c | 4 ++--
 5 files changed, 14 insertions(+), 4 deletions(-)
include/linux/bpf.h +1 -0
index c9eafa67f2a2..43fd7eeeeabb 100644
@@ -682,6 +682,7 @@ struct bpf_func_proto {
 	u64 (*func)(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
 	bool gpl_only;
 	bool pkt_access;
+	bool might_sleep;
 	enum bpf_return_type ret_type;
 	union {
 		struct {
kernel/bpf/bpf_lsm.c +4 -2
index d6c9b3705f24..ae0267f150b5 100644
@@ -151,6 +151,7 @@ BTF_ID_LIST_SINGLE(bpf_ima_inode_hash_btf_ids, struct, inode)
 static const struct bpf_func_proto bpf_ima_inode_hash_proto = {
 	.func		= bpf_ima_inode_hash,
 	.gpl_only	= false,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_BTF_ID,
 	.arg1_btf_id	= &bpf_ima_inode_hash_btf_ids[0],
@@ -169,6 +170,7 @@ BTF_ID_LIST_SINGLE(bpf_ima_file_hash_btf_ids, struct, file)
 static const struct bpf_func_proto bpf_ima_file_hash_proto = {
 	.func		= bpf_ima_file_hash,
 	.gpl_only	= false,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_BTF_ID,
 	.arg1_btf_id	= &bpf_ima_file_hash_btf_ids[0],
@@ -221,9 +223,9 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	case BPF_FUNC_bprm_opts_set:
 		return &bpf_bprm_opts_set_proto;
 	case BPF_FUNC_ima_inode_hash:
-		return prog->aux->sleepable ? &bpf_ima_inode_hash_proto : NULL;
+		return &bpf_ima_inode_hash_proto;
 	case BPF_FUNC_ima_file_hash:
-		return prog->aux->sleepable ? &bpf_ima_file_hash_proto : NULL;
+		return &bpf_ima_file_hash_proto;
 	case BPF_FUNC_get_attach_cookie:
 		return bpf_prog_has_trampoline(prog) ? &bpf_get_attach_cookie_proto : NULL;
 #ifdef CONFIG_NET
kernel/bpf/helpers.c +2 -0
index 2299bb0d89e5..9296b654dbd7 100644
@@ -661,6 +661,7 @@ BPF_CALL_3(bpf_copy_from_user, void *, dst, u32, size,
 const struct bpf_func_proto bpf_copy_from_user_proto = {
 	.func		= bpf_copy_from_user,
 	.gpl_only	= false,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_UNINIT_MEM,
 	.arg2_type	= ARG_CONST_SIZE_OR_ZERO,
@@ -691,6 +692,7 @@ BPF_CALL_5(bpf_copy_from_user_task, void *, dst, u32, size,
 const struct bpf_func_proto bpf_copy_from_user_task_proto = {
 	.func		= bpf_copy_from_user_task,
 	.gpl_only	= true,
+	.might_sleep	= true,
 	.ret_type	= RET_INTEGER,
 	.arg1_type	= ARG_PTR_TO_UNINIT_MEM,
 	.arg2_type	= ARG_CONST_SIZE_OR_ZERO,
kernel/bpf/verifier.c +5 -0
index 9528a066cfa5..068cc885903c 100644
@@ -7516,6 +7516,11 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 		return -EINVAL;
 	}
 
+	if (!env->prog->aux->sleepable && fn->might_sleep) {
+		verbose(env, "helper call might sleep in a non-sleepable prog\n");
+		return -EINVAL;
+	}
+
 	/* With LD_ABS/IND some JITs save/restore skb from r1. */
 	changes_data = bpf_helper_changes_pkt_data(fn->func);
 	if (changes_data && fn->arg1_type != ARG_PTR_TO_CTX) {
kernel/trace/bpf_trace.c +2 -2
index 5b9008bc597b..3bbd3f0c810c 100644
@@ -1485,9 +1485,9 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 	case BPF_FUNC_get_task_stack:
 		return &bpf_get_task_stack_proto;
 	case BPF_FUNC_copy_from_user:
-		return prog->aux->sleepable ? &bpf_copy_from_user_proto : NULL;
+		return &bpf_copy_from_user_proto;
 	case BPF_FUNC_copy_from_user_task:
-		return prog->aux->sleepable ? &bpf_copy_from_user_task_proto : NULL;
+		return &bpf_copy_from_user_task_proto;
 	case BPF_FUNC_snprintf_btf:
 		return &bpf_snprintf_btf_proto;
 	case BPF_FUNC_per_cpu_ptr:
Signature · 1 line
2.30.2
#3 lore raw

PATCH v10 3/4 bpf: Add kfunc bpf_rcu_read_lock/unlock()

In reply to Yonghong Song

Add two kfunc's bpf_rcu_read_lock() and bpf_rcu_read_unlock(). These two kfunc's can be used for all program types. The following is an example about how rcu pointer are used w.r.t. bpf_rcu_read_lock()/bpf_rcu_read_unlock().

  struct task_struct {
    ...
    struct task_struct              *last_wakee;
    struct task_struct __rcu        *real_parent;
    ...
  };

Let us say prog does 'task = bpf_get_current_task_btf()' to get a 'task' pointer. The basic rules are:
- 'real_parent = task->real_parent' should be inside bpf_rcu_read_lock region. This is to simulate rcu_dereference() operation. The 'real_parent' is marked as MEM_RCU only if (1). task->real_parent is inside bpf_rcu_read_lock region, and (2). task is a trusted ptr. So MEM_RCU marked ptr can be 'trusted' inside the bpf_rcu_read_lock region.
- 'last_wakee = real_parent->last_wakee' should be inside bpf_rcu_read_lock region since it tries to access rcu protected memory.
- the ptr 'last_wakee' will be marked as PTR_UNTRUSTED since in general it is not clear whether the object pointed by 'last_wakee' is valid or not even inside bpf_rcu_read_lock region.

The verifier will reset all rcu pointer register states to untrusted at bpf_rcu_read_unlock() kfunc call site, so any such rcu pointer won't be trusted any more outside the bpf_rcu_read_lock() region.

The current implementation does not support nested rcu read lock region in the prog.

---
 include/linux/bpf.h          |   3 +
 include/linux/bpf_verifier.h |   5 +-
 kernel/bpf/btf.c             |   3 +
 kernel/bpf/helpers.c         |  12 +++
 kernel/bpf/verifier.c        | 162 ++++++++++++++++++++++++++++-------
 5 files changed, 155 insertions(+), 30 deletions(-)
include/linux/bpf.h +3 -0
index 43fd7eeeeabb..c6aa6912ea16 100644
@@ -572,6 +572,9 @@ enum bpf_type_flag {
 	 */
 	PTR_TRUSTED		= BIT(12 + BPF_BASE_TYPE_BITS),
 
+	/* MEM is tagged with rcu and memory access needs rcu_read_lock protection. */
+	MEM_RCU			= BIT(13 + BPF_BASE_TYPE_BITS),
+
 	__BPF_TYPE_FLAG_MAX,
 	__BPF_TYPE_LAST_FLAG	= __BPF_TYPE_FLAG_MAX - 1,
 };
include/linux/bpf_verifier.h +4 -1
index 545152ac136c..c05aa6e1f6f5 100644
@@ -344,6 +344,7 @@ struct bpf_verifier_state {
 		u32 id;
 	} active_lock;
 	bool speculative;
+	bool active_rcu_lock;
 
 	/* first and last insn idx of this verifier state */
 	u32 first_insn_idx;
@@ -445,6 +446,7 @@ struct bpf_insn_aux_data {
 	u32 seen; /* this insn was processed by the verifier at env->pass_cnt */
 	bool sanitize_stack_spill; /* subject to Spectre v4 sanitation */
 	bool zext_dst; /* this insn zero extends dst reg */
+	bool storage_get_func_atomic; /* bpf_*_storage_get() with atomic memory alloc */
 	u8 alu_state; /* used in combination with alu_limit */
 
 	/* below fields are initialized once */
@@ -534,6 +536,7 @@ struct bpf_verifier_env {
 	bool bypass_spec_v1;
 	bool bypass_spec_v4;
 	bool seen_direct_write;
+	bool rcu_tag_supported;
 	struct bpf_insn_aux_data *insn_aux_data; /* array of per-insn state */
 	const struct bpf_line_info *prev_linfo;
 	struct bpf_verifier_log log;
@@ -680,7 +683,7 @@ static inline bool bpf_prog_check_recur(const struct bpf_prog *prog)
 	}
 }
 
-#define BPF_REG_TRUSTED_MODIFIERS (MEM_ALLOC | PTR_TRUSTED)
+#define BPF_REG_TRUSTED_MODIFIERS (MEM_ALLOC | MEM_RCU | PTR_TRUSTED)
 
 static inline bool bpf_type_has_unsafe_modifiers(u32 type)
 {
kernel/bpf/btf.c +3 -0
index 9dbfda2b5c6c..bd3369100239 100644
@@ -6238,6 +6238,9 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 				/* check __percpu tag */
 				if (strcmp(tag_value, "percpu") == 0)
 					tmp_flag = MEM_PERCPU;
+				/* check __rcu tag */
+				if (strcmp(tag_value, "rcu") == 0)
+					tmp_flag = MEM_RCU;
 			}
 
 			stype = btf_type_skip_modifiers(btf, mtype->type, &id);
kernel/bpf/helpers.c +12 -0
index 9296b654dbd7..a5a511430f2a 100644
@@ -1990,6 +1990,16 @@ void *bpf_rdonly_cast(void *obj__ign, u32 btf_id__k)
 	return obj__ign;
 }
 
+void bpf_rcu_read_lock(void)
+{
+	rcu_read_lock();
+}
+
+void bpf_rcu_read_unlock(void)
+{
+	rcu_read_unlock();
+}
+
 __diag_pop();
 
 BTF_SET8_START(generic_btf_ids)
@@ -2031,6 +2041,8 @@ BTF_ID(func, bpf_cgroup_release)
 BTF_SET8_START(common_btf_ids)
 BTF_ID_FLAGS(func, bpf_cast_to_kern_ctx)
 BTF_ID_FLAGS(func, bpf_rdonly_cast)
+BTF_ID_FLAGS(func, bpf_rcu_read_lock)
+BTF_ID_FLAGS(func, bpf_rcu_read_unlock)
 BTF_SET8_END(common_btf_ids)
 
 static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/verifier.c +133 -29

281 lines not on this page · raw message

Signature · 1 line
2.30.2
#4 lore raw

Re: [PATCH bpf-next v10 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()

kernel/bpf/verifier.c int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr) · line 16686

Quoted 7 lines · Yonghong Song
kernel/bpf/verifier.c +2 -0
@@ -16580,6 +16682,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr)
        env->bypass_spec_v1 = bpf_bypass_spec_v1();
        env->bypass_spec_v4 = bpf_bypass_spec_v4();
        env->bpf_capable = bpf_capable();
+       env->rcu_tag_supported =
+               btf_find_by_name_kind(btf_vmlinux, "rcu", BTF_KIND_TYPE_TAG) > 0;

It needs btf_vmlinux != NULL check as well,
since we error earlier only on IS_ERR(btf_vmlinux).
btf_vmlinux can be NULL at this point when CONFIG_DEBUG_INFO_BTF is not set.

In the previous discussion I thought we agreed to fix convert_ctx_accesses() vs incorrect application of BPF_PROBE_MEM for PTR_TRUSTED pointers.
But I didn't find it in this patch.
So I'm fixing both issues and planning to apply after testing.

#5 lore raw

Re: [PATCH bpf-next v10 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()

Quoted 14 lines · Alexei Starovoitov

kernel/bpf/verifier.c int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr) · line 16686

Quoted 7 lines · Yonghong Song
kernel/bpf/verifier.c +2 -0
@@ -16580,6 +16682,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr)
        env->bypass_spec_v1 = bpf_bypass_spec_v1();
        env->bypass_spec_v4 = bpf_bypass_spec_v4();
        env->bpf_capable = bpf_capable();
+       env->rcu_tag_supported =
+               btf_find_by_name_kind(btf_vmlinux, "rcu", BTF_KIND_TYPE_TAG) > 0;

It needs btf_vmlinux != NULL check as well,
since we error earlier only on IS_ERR(btf_vmlinux).
btf_vmlinux can be NULL at this point when CONFIG_DEBUG_INFO_BTF is not set.

Quoted 5 lines

In the previous discussion I thought we agreed to fix convert_ctx_accesses() vs incorrect application of BPF_PROBE_MEM for PTR_TRUSTED pointers.
But I didn't find it in this patch.
So I'm fixing both issues and planning to apply after testing.

Turned out the initial PTR_TRUSTED patch was buggy. prog_type_args_trusted() is incorrect.
It marks fentry/fexit pointers as trusted and our own selftest is crashing.
I'll send a separate fix.

So going to apply rcu set with just btf_vmlinux fix.

#6 lore raw

Re: [PATCH bpf-next v10 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()

In reply to Alexei Starovoitov
Quoted 12 lines · Alexei Starovoitov

kernel/bpf/verifier.c int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr) · line 16686

Quoted 7 lines · Yonghong Song
kernel/bpf/verifier.c +2 -0
@@ -16580,6 +16682,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr)
         env->bypass_spec_v1 = bpf_bypass_spec_v1();
         env->bypass_spec_v4 = bpf_bypass_spec_v4();
         env->bpf_capable = bpf_capable();
+       env->rcu_tag_supported =
+               btf_find_by_name_kind(btf_vmlinux, "rcu", BTF_KIND_TYPE_TAG) > 0;

It needs btf_vmlinux != NULL check as well,
since we error earlier only on IS_ERR(btf_vmlinux).
btf_vmlinux can be NULL at this point when CONFIG_DEBUG_INFO_BTF is not set.

I checked the code and it looks like btf_find_by_name_kind can handle btf_vmlinux = NULL properly. Consider this is a unlikely case so I did not add btf_vmlinux checking here.

u32 btf_nr_types(const struct btf *btf)

{
        u32 total = 0;

        while (btf) {
                total += btf->nr_types;
                btf = btf->base_btf;
        }

        return total;
}

s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)

{
        const struct btf_type *t;
        const char *tname;
        u32 i, total;

        total = btf_nr_types(btf);
        for (i = 1; i < total; i++) {
                t = btf_type_by_id(btf, i);
                if (BTF_INFO_KIND(t->info) != kind)
                        continue;

                tname = btf_name_by_offset(btf, t->name_off);
                if (!strcmp(tname, name))
                        return i;
        }

        return -ENOENT;
}

If btf_vmlinux is NULL, then btf_nr_types(...) will return 0, so btf_find_by_name_kind will return -ENOENT.

Certainly it does not hurt by adding explicit btf_vmlinux checking before doing btf_find_by_name_kind(...)

Quoted 6 lines

In the previous discussion I thought we agreed to fix convert_ctx_accesses() vs incorrect application of BPF_PROBE_MEM for PTR_TRUSTED pointers.
But I didn't find it in this patch.
So I'm fixing both issues and planning to apply after testing.

Thanks. I didn't do that since it is not really related to bpf_rcu_read_lock(). I plan to do it as a followup. But saw you just fixed the issue.

#7 lore raw

Re: [PATCH bpf-next v10 3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()

Quoted 17 lines · Yonghong Song
Quoted 12 lines · Alexei Starovoitov

kernel/bpf/verifier.c int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr) · line 16686

Quoted 7 lines · Yonghong Song
kernel/bpf/verifier.c +2 -0
@@ -16580,6 +16682,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr)
         env->bypass_spec_v1 = bpf_bypass_spec_v1();
         env->bypass_spec_v4 = bpf_bypass_spec_v4();
         env->bpf_capable = bpf_capable();
+       env->rcu_tag_supported =
+               btf_find_by_name_kind(btf_vmlinux, "rcu", BTF_KIND_TYPE_TAG) > 0;

It needs btf_vmlinux != NULL check as well,
since we error earlier only on IS_ERR(btf_vmlinux).
btf_vmlinux can be NULL at this point when CONFIG_DEBUG_INFO_BTF is not set.

Quoted 3 lines

I checked the code and it looks like btf_find_by_name_kind can handle btf_vmlinux = NULL properly. Consider this is a unlikely case so I did not add btf_vmlinux checking here.

Good point. You're right.
I got confused by the similar !btf check in bpf_find_btf_id(). There it looks to be necessary. Here it's indeed redundant. Sorry about that.

#8 lore raw

PATCH v10 4/4 selftests/bpf: Add tests for bpf_rcu_read_lock()

In reply to Yonghong Song

Add a few positive/negative tests to test bpf_rcu_read_lock() and its corresponding verifier support. The new test will fail on s390x and aarch64, so an entry is added to each of their respective deny lists.

---
 tools/testing/selftests/bpf/DENYLIST.aarch64  |   1 +
 tools/testing/selftests/bpf/DENYLIST.s390x    |   1 +
 .../selftests/bpf/prog_tests/rcu_read_lock.c  | 158 ++++++++++
 .../selftests/bpf/progs/rcu_read_lock.c       | 290 ++++++++++++++++++
 4 files changed, 450 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
 create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c
tools/testing/selftests/bpf/DENYLIST.aarch64 +1 -0
index affc5aebbf0f..8e77515d56f6 100644
@@ -45,6 +45,7 @@ modify_return                                    # modify_return__attach failed
 module_attach                                    # skel_attach skeleton attach failed: -524
 mptcp/base                                       # run_test mptcp unexpected error: -524 (errno 524)
 netcnt                                           # packets unexpected packets: actual 10001 != expected 10000
+rcu_read_lock                                    # failed to attach: ERROR: strerror_r(-524)=22
 recursion                                        # skel_attach unexpected error: -524 (errno 524)
 ringbuf                                          # skel_attach skeleton attachment failed: -1
 setget_sockopt                                   # attach_cgroup unexpected error: -524
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
index b9a3d80204c6..648a8a1b6b78 100644
@@ -43,6 +43,7 @@ module_attach                            # skel_attach skeleton attach failed: -
 mptcp
 netcnt                                   # failed to load BPF skeleton 'netcnt_prog': -7                               (?)
 probe_user                               # check_kprobe_res wrong kprobe res from probe read                           (?)
+rcu_read_lock                            # failed to find kernel BTF type ID of '__x64_sys_getpgid': -3                (?)
 recursion                                # skel_attach unexpected error: -524                                          (trampoline)
 ringbuf                                  # skel_load skeleton load failed                                              (?)
 select_reuseport                         # intermittently fails on new s390x setup
tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c new file +158 -0

158 lines not on this page · raw message

tools/testing/selftests/bpf/progs/rcu_read_lock.c new file +290 -0

290 lines not on this page · raw message

Signature · 1 line
2.30.2
#9 lore raw
In reply to Yonghong Song
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 12 lines · On Wed, 23 Nov 2022 21:32:01 -0800 you wrote:

Currently, without rcu attribute info in BTF, the verifier treats rcu tagged pointer as a normal pointer. This might be a problem for sleepable program where rcu_read_lock()/unlock() is not available. For example, for a sleepable fentry program, if rcu protected memory access is interleaved with a sleepable helper/kfunc, it is possible the memory access after the sleepable helper/kfunc might be invalid since the object might have been freed then. Even without
a sleepable helper/kfunc, without rcu_read_lock() protection, it is possible that the rcu protected object might be release in the middle of bpf program execution which may cause incorrect result.

Quoted 1 line

[...]

Here is the summary with links:
- [bpf-next,v10,1/4] compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))

    https://git.kernel.org/bpf/bpf-next/c/5a0f663f0189
  - [bpf-next,v10,2/4] bpf: Introduce might_sleep field in bpf_func_proto
    https://git.kernel.org/bpf/bpf-next/c/01685c5bddaa
  - [bpf-next,v10,3/4] bpf: Add kfunc bpf_rcu_read_lock/unlock()
    https://git.kernel.org/bpf/bpf-next/c/9bb00b2895cb
  - [bpf-next,v10,4/4] selftests/bpf: Add tests for bpf_rcu_read_lock()
    https://git.kernel.org/bpf/bpf-next/c/48671232fcb8

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html