bpf: Add bpf_rcu_read_lock() support
Status from this message: This series was applied to bpf/bpf-next.git (master)
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v10 Latest4 patches 4 repliesApplied 4/4
- v94 patches 6 repliesSuperseded
- v84 patches 9 repliesSuperseded
Earlier versions · 7
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v7 | 4 patches | 11 replies | Superseded | |
| v6 | 7 patches | 0 replies | Superseded | |
| v5 | 7 patches | 2 replies | Superseded | |
| v4 | 7 patches | 2 replies | Superseded | |
| v3 | 7 patches | 0 replies | Superseded | |
| v2 | 8 patches | 8 replies | Superseded | |
| v1 | 5 patches | 20 replies | Superseded |
Changes in v10 from the cover letter
. if no rcu tag support in vmlinux btf, using bpf_rcu_read_lock/unlock()
will cause verification error.
. at bpf_rcu_read_unlock(), invalidate rcu ptr to PTR_UNTRUSTED
instead of SCALAR_VALUE.
. a few other comment changes and other minor changes.
Patches in v10 All 4 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))include/ | +2 | 0 replies | Applied |
| 2/4 | bpf: Introduce might_sleep field in bpf_func_protoinclude/ kernel/ kernel/ +2 more | +14 | 0 replies | Applied |
| 3/4 | bpf: Add kfunc bpf_rcu_read_lock/unlock()include/ include/ kernel/ +2 more | +155 | 4 replies | Applied |
| 4/4 | selftests/bpf: Add tests for bpf_rcu_read_lock()tools/ tools/ tools/ +1 more | +450 | 0 replies | Applied |
Reviews trailers per patch
All 4 patches have a Reviewed-by or Acked-by.
| Patch | Martin KaFai Lau | KP Singh |
|---|---|---|
| 1/4 | A-b | A-b |
| 2/4 | A-b | |
| 3/4 | A-b | |
| 4/4 | A-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH bpf-next v10 0/4] bpf: Add bpf_rcu_read_lock() support
Currently, without rcu attribute info in BTF, the verifier treats rcu tagged pointer as a normal pointer. This might be a problem for sleepable program where rcu_read_lock()/unlock() is not available. For example, for a sleepable fentry program, if rcu protected memory access is interleaved with a sleepable helper/kfunc, it is possible the memory access after the sleepable helper/kfunc might be invalid since the object might have been freed then. Even without
a sleepable helper/kfunc, without rcu_read_lock() protection, it is possible that the rcu protected object might be release in the middle of bpf program execution which may cause incorrect result.
To prevent above cases, enable btf_type_tag("rcu") attributes, introduce new bpf_rcu_read_lock/unlock() kfuncs and add verifier support.
In the rest of patch set, Patch 1 enabled btf_type_tag for __rcu attribute. Patche 2 added might_sleep in bpf_func_proto. Patch 3 added new bpf_rcu_read_lock/unlock() kfuncs and verifier support.
Patch 4 added some tests for these two new kfuncs.
Changelogs:
v9 -> v10:
. if no rcu tag support in vmlinux btf, using bpf_rcu_read_lock/unlock()
will cause verification error.
. at bpf_rcu_read_unlock(), invalidate rcu ptr to PTR_UNTRUSTED
instead of SCALAR_VALUE.
. a few other comment changes and other minor changes.
v8 -> v9:
. remove sleepable prog check for ld_abs/ind checking in rcu read
lock region.
. fix a test failure with gcc-compiled kernel.
. a couple of other minor fixes.
v7 -> v8:
. add might_sleep in bpf_func_proto so we can easily identify whether
a helper is sleepable or not.
. do not enforce rcu rules for sleepable, e.g., rcu dereference must
be in a bpf_rcu_read_lock region. This is to keep old code working fine.
. Mark 'b' in 'b = a->b' (b is tagged with __rcu) as MEM_RCU only if
'b = a->b' in rcu read region and 'a' is trusted. This adds safety guarantee for 'b' inside the rcu read region.
v6 -> v7:
. rebase on top of bpf-next.
. remove the patch which enables sleepable program using
cgrp_local_storage map. This is orthogonal to this patch set and will be addressed separately.
. mark the rcu pointer dereference result as UNTRUSTED if inside
a bpf_rcu_read_lock() region.
v5 -> v6:
. fix selftest prog miss_unlock which tested nested locking. . add comments in selftest prog cgrp_succ to explain how to handle
nested memory access after rcu memory load.
v4 -> v5:
. add new test to aarch64 deny list.
v3 -> v4:
. fix selftest failures when built with gcc. gcc doesn't support
btf_type_tag yet and some tests relies on that. skip these tests if vmlinux BTF does not have btf_type_tag("rcu").
v2 -> v3:
. went back to MEM_RCU approach with invalidate rcu ptr registers
at bpf_rcu_read_unlock() place.
. remove KF_RCU_LOCK/UNLOCK flag and compare btf_id at verification
time instead.
v1 -> v2:
. use kfunc instead of helper for bpf_rcu_read_lock/unlock. . not use MEM_RCU bpf_type_flag, instead use active_rcu_lock
in reg state to identify rcu ptr's.
. Add more self tests.
. add new test to s390x deny list.
Yonghong Song (4):
compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
bpf: Introduce might_sleep field in bpf_func_proto
bpf: Add kfunc bpf_rcu_read_lock/unlock()
selftests/bpf: Add tests for bpf_rcu_read_lock()
include/linux/bpf.h | 4 + include/linux/bpf_verifier.h | 5 +- include/linux/compiler_types.h | 3 +- kernel/bpf/bpf_lsm.c | 6 +- kernel/bpf/btf.c | 3 + kernel/bpf/helpers.c | 14 + kernel/bpf/verifier.c | 167 ++++++++-- kernel/trace/bpf_trace.c | 4 +- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/rcu_read_lock.c | 158 ++++++++++ .../selftests/bpf/progs/rcu_read_lock.c | 290 ++++++++++++++++++ 12 files changed, 621 insertions(+), 35 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c
Signature · 1 line
2.30.2