Close race in freeing special fields and map value
Status from this message: This series was applied to bpf/bpf.git (master)
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest5 patches 0 repliesApplied 5/5
- v25 patches 3 repliesSuperseded
- v14 patches 4 repliesSuperseded
Changes in v3 from the cover letter
v2: https://lore.kernel.org/bpf/[email protected]
* Add syzbot Tested-by.
* Add Amery's Reviewed-by.
* Fix missing rcu_dereference_check() in __bpf_selem_free_rcu. (BPF CI Bot)
* Remove migrate_disable() in bpf_selem_free_rcu. (Alexei)
Patches in v3 All 5 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | bpf: Register dtor for freeing special fieldsinclude/ kernel/ kernel/ | +140 | 0 replies | Applied |
| 2/5 | bpf: Lose const-ness of map in map_check_btf()include/ include/ kernel/ +8 more | +15 | 0 replies | Applied |
| 3/5 | bpf: Delay freeing fields in local storagekernel/ | +21 | 0 replies | Applied |
| 4/5 | bpf: Retire rcu_trace_implies_rcu_gp() from local storagekernel/ | +19 | 0 replies | Applied |
| 5/5 | selftests/bpf: Add tests for special fields racestools/ tools/ | +415 | 0 replies | Applied |
Reviews trailers per patch
3 of 5 patches have a Reviewed-by or Acked-by.
| Patch | [email protected] | Amery Hung | Paul E. McKenney |
|---|---|---|---|
| 1/5 | T-b | ||
| 2/5 | |||
| 3/5 | R-b | ||
| 4/5 | R-b | R-b | |
| 5/5 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [PATCH bpf v3 0/5] Close race in freeing special fields and map value
There exists a race across various map types where the freeing of special fields (tw, timer, wq, kptr, etc.) can be done eagerly when a logical delete operation is done on a map value, such that the program which continues to have access to such a map value can recreate the fields and cause them to leak.
The set contains fixes for this case. It is a continuation of Mykyta's previous attempt in [0], but applies to all fields. A test is included which reproduces the bug reliably in absence of the fixes.
Local Storage Benchmarks
------------------------
Evaluation Setup: Benchmarked on a dual-socket Intel Xeon Gold 6348 (Ice Lake) @ 2.60GHz (56 cores / 112 threads), with the CPU governor set to performance. Bench was pinned to a single NUMA node throughout the test.
Benchmark comes from [1] using the following command:
./bench -p 1 local-storage-create --storage-type <socket,task> --batch-size <16,32,64>
Before the test, 10 runs of all cases ([socket|task] x 3 batch sizes x 7 iterations per batch size) are done to warm up and prime the machine.
Then, 3 runs of all cases are done (with and without the patch, across reboots).
For each comparison, we have 21 samples, i.e. per batch size (e.g. socket 16) of a given local storage, we have 3 runs x 7 iterations.
The statistics (mean, median, stddev) and t-test is done for each scenario (local storage and batch size pair) individually (21 samples for either case). All values are for local storage creations in thousand creations / sec (k/s).
Baseline (without patch) With patch Delta
Case Median Mean Std. Dev. Median Mean Std. Dev. Median %
---------------------------------------------------------------------------------------------------
socket 16 432.026 431.941 1.047 431.347 431.953 1.635 -0.679 -0.16% socket 32 432.641 432.818 1.535 432.488 432.302 1.508 -0.153 -0.04% socket 64 431.504 431.996 1.337 429.145 430.326 2.469 -2.359 -0.55% task 16 38.816 39.382 1.456 39.657 39.337 1.831 +0.841 +2.17% task 32 38.815 39.644 2.690 38.721 39.122 1.636 -0.094 -0.24% task 64 37.562 38.080 1.701 39.554 38.563 1.689 +1.992 +5.30%
The cases for socket are within the range of noise, and improvements in task local storage are due to high variance (CV ~4%-6% across batch sizes). The only statistically significant case worth mentioning is socket with batch size 64 with p-value from t-test < 0.05, but the absolute difference is small (~2k/s).
TL;DR there doesn't appear to be any significant regression or improvement.
[0]: https://lore.kernel.org/bpf/[email protected]
[1]: https://lore.kernel.org/bpf/[email protected]
Changelog:
----------
v2 -> v3
v2: https://lore.kernel.org/bpf/[email protected]
* Add syzbot Tested-by.
* Add Amery's Reviewed-by.
* Fix missing rcu_dereference_check() in __bpf_selem_free_rcu. (BPF CI Bot)
* Remove migrate_disable() in bpf_selem_free_rcu. (Alexei)
v1 -> v2
v1: https://lore.kernel.org/bpf/[email protected]
* Add Paul's Reviewed-by.
* Fix use-after-free in accessing bpf_mem_alloc embedded in map. (syzbot CI)
* Add benchmark numbers for local storage.
* Add extra test case for per-cpu hashmap coverage with up to 16 refcount leaks.
* Target bpf tree.
Kumar Kartikeya Dwivedi (5):
bpf: Register dtor for freeing special fields
bpf: Lose const-ness of map in map_check_btf()
bpf: Delay freeing fields in local storage
bpf: Retire rcu_trace_implies_rcu_gp() from local storage selftests/bpf: Add tests for special fields races
include/linux/bpf.h | 4 +- include/linux/bpf_local_storage.h | 2 +- include/linux/bpf_mem_alloc.h | 6 + kernel/bpf/arena.c | 2 +- kernel/bpf/arraymap.c | 2 +- kernel/bpf/bloom_filter.c | 2 +- kernel/bpf/bpf_insn_array.c | 2 +- kernel/bpf/bpf_local_storage.c | 75 +++--- kernel/bpf/hashtab.c | 86 +++++++ kernel/bpf/local_storage.c | 2 +- kernel/bpf/lpm_trie.c | 2 +- kernel/bpf/memalloc.c | 58 ++++- kernel/bpf/syscall.c | 2 +- .../selftests/bpf/prog_tests/map_kptr_race.c | 218 ++++++++++++++++++ .../selftests/bpf/progs/map_kptr_race.c | 197 ++++++++++++++++ 15 files changed, 603 insertions(+), 57 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/map_kptr_race.c create mode 100644 tools/testing/selftests/bpf/progs/map_kptr_race.c
base-commit: 6881af27f9ea0f5ca8f606f573ef5cc25ca31fe4
Signature · 1 line
2.47.3