bpf: Avoid locks in bpf_timer and bpf_wq
Get series with b4
b4 am [email protected] change-id 20251028-timer_nolock-457f5b9daaceVersions newest first
- v8 Latest8 patches 1 replyIn review
- v78 patches 4 repliesSuperseded
- v610 patches 3 repliesSuperseded 4 of 10 applied
Earlier versions · 5
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v5 RFC | 10 patches | 5 replies | Superseded | |
| v4 RFC | 8 patches | 0 replies | Superseded | |
| v3 RFC | 10 patches | 25 replies | Superseded | |
| v2 RFC | 5 patches | 10 replies | Superseded | |
| v1 RFC | 5 patches | 9 replies | Superseded |
Changes in v8 from the cover letter
- Return -EBUSY in bpf_async_read_op() if last_seq is failed to be set
- In bpf_async_cancel_and_free() drop bpf_async_cb ref after calling bpf_async_process()
- Link to v7: https://lore.kernel.org/r/[email protected]
Patches in v8 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/8 | bpf: Enable bpf timer and workqueue use in NMIkernel/ | +254 | 1 reply | |
| 2/8 | bpf: Add verifier support for bpf_timer argument in kfuncskernel/ | +54 | 0 replies | |
| 3/8 | bpf: Introduce bpf_timer_cancel_async() kfunckernel/ | +13 | 0 replies | |
| 4/8 | selftests/bpf: Refactor timer selfteststools/ | +36 | 0 replies | |
| 5/8 | selftests/bpf: Add stress test for timer async canceltools/ tools/ | +28 | 0 replies | |
| 6/8 | selftests/bpf: Verify bpf_timer_cancel_async workstools/ tools/ | +48 | 0 replies | |
| 7/8 | selftests/bpf: Add timer stress test in NMI contexttools/ tools/ | +231 | 0 replies | |
| 8/8 | selftests/bpf: Removed obsolete teststools/ | +0 | 0 replies |
Reviews trailers per patch
2 of 8 patches have a Reviewed-by or Acked-by.
| Patch | Kumar Kartikeya Dwivedi | Andrii Nakryiko |
|---|---|---|
| 1/8 | A-b | R-b |
| 2/8 | A-b | |
| 3/8 | ||
| 4/8 | ||
| 5/8 | ||
| 6/8 | ||
| 7/8 | ||
| 8/8 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH bpf-next v8 0/8] bpf: Avoid locks in bpf_timer and bpf_wq
This series reworks implementation of BPF timer and workqueue APIs. The goal is to make both timers and wq non-blocking, enabling their use in NMI context.
Today this code relies on a bpf_spin_lock embedded in the map element to serialize:
* init of the async object,
* setting/changing the callback and bpf_prog
* starting/cancelling the timer/work
* tearing down when the map element is deleted or the map’s user ref is dropped
The basic design approach in this series:
* Use irq_work to offload all blocking work from NMI
* Introduce refcount to guarantee lifetime of the bpf_async_cb structs deferred to potentially multiple irq_work callbacks
* Keep objects under RCU protection to make sure they are not freed while kfuncs/helpers access them (We can't use refcnt for this, as refcnt itself is part of the bpf_async_cb struct)
---
Changes in v8:
- Return -EBUSY in bpf_async_read_op() if last_seq is failed to be set
- In bpf_async_cancel_and_free() drop bpf_async_cb ref after calling bpf_async_process()
- Link to v7: https://lore.kernel.org/r/[email protected]
Changes in v7:
- Addressed Andrii's review points from the previous version - nothing very significang.
- Added NMI stress tests for bpf_timer - hit few verifier failing checks and removed them.
- Address sparse warning in the bpf_async_update_prog_callback()
- Link to v6: https://lore.kernel.org/r/[email protected]
Changes in v6:
- Reworked destruction and refcnt use:
- On cancel_and_free() set last_seq to BPF_ASYNC_DESTROY value, drop map's reference
- In irq work callback, atomically switch DESTROY to DESTROYED, cancel timer/wq
- Free bpf_async_cb on refcnt going to 0.
- Link to v5: https://lore.kernel.org/r/[email protected]
Changes in v5:
- Extracted lock-free algorithm for updating cb->prog andcb->callback_fn into a function bpf_async_update_prog_callback(),
added a new commit and introduces this function and uses it in __bpf_async_set_callback(), bpf_timer_cancel() and bpf_async_cancel_and_free().
This allows to move the change into the separate commit without breaking correctness.
- Handle NULL prog in bpf_async_update_prog_callback().
- Link to v4: https://lore.kernel.org/r/[email protected]
Changes in v4:
- Handle irq_work_queue failures in both schedule and cancel_and_free
paths: introduced bpf_async_refcnt_dec_cleanup() that decrements refcnt and makes sure if last reference is put, there is at least one irq_work scheduled to execute final cleanup.
- Additional refcnt inc/dec in set_callback() + rcu lock to make sure cleanup is not running at the same time as set_callback().
- Added READ_ONCE where it was needed.
- Squash 'bpf: Refactor __bpf_async_set_callback()' commit into 'bpf:
Add lock-free cell for NMI-safe
async operations'
- Removed mpmc_cell, use seqcount_latch_t instead.
- Link to v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- Major rework
- Introduce mpmc_cell, allowing concurrent writes and reads
- Implement irq_work deferring
- Adding selftests
- Introduces bpf_timer_cancel_async kfunc
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Move refcnt initialization and put (from cancel_and_free())
from patch 5 into the patch 4, so that patch 4 has more clear and full implementation and use of refcnt
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Mykyta Yatsenko (8):
bpf: Enable bpf timer and workqueue use in NMI
bpf: Add verifier support for bpf_timer argument in kfuncs
bpf: Introduce bpf_timer_cancel_async() kfunc selftests/bpf: Refactor timer selftests selftests/bpf: Add stress test for timer async cancel selftests/bpf: Verify bpf_timer_cancel_async works selftests/bpf: Add timer stress test in NMI context selftests/bpf: Removed obsolete tests
kernel/bpf/helpers.c | 449 ++++++++++++--------- kernel/bpf/verifier.c | 75 +++- tools/testing/selftests/bpf/prog_tests/timer.c | 250 +++++++++++- tools/testing/selftests/bpf/progs/timer.c | 118 +++++- .../bpf/progs/verifier_helper_restricted.c | 111 ----- 5 files changed, 659 insertions(+), 344 deletions(-)
---
base-commit: ba335bf3a5b8c5f47d56e9be3f96e0989dad5346
change-id: 20251028-timer_nolock-457f5b9daace
Best regards,
Signature · 1 line
Mykyta Yatsenko <[email protected]>