Skip to content

bpf: Avoid locks in bpf_timer and bpf_wq

Mykyta Yatsenko bpf 8 versions – In review

Get series with b4
b4 am [email protected] change-id 20251028-timer_nolock-457f5b9daace

Versions newest first

  1. v8 Latest8 patches 1 replyIn review
  2. v78 patches 4 repliesSuperseded
  3. v610 patches 3 repliesSuperseded 4 of 10 applied
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v5 RFC10 patches5 repliesSuperseded
v4 RFC8 patches0 repliesSuperseded
v3 RFC10 patches25 repliesSuperseded
v2 RFC5 patches10 repliesSuperseded
v1 RFC5 patches9 repliesSuperseded

Changes in v8 from the cover letter

- Return -EBUSY in bpf_async_read_op() if last_seq is failed to be set
- In bpf_async_cancel_and_free() drop bpf_async_cb ref after calling bpf_async_process()
- Link to v7: https://lore.kernel.org/r/[email protected]

Patches in v8 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8bpf: Enable bpf timer and workqueue use in NMIkernel/bpf/helpers.c+254 -1821 reply
2/8bpf: Add verifier support for bpf_timer argument in kfuncskernel/bpf/verifier.c+54 -210 replies
3/8bpf: Introduce bpf_timer_cancel_async() kfunckernel/bpf/helpers.c+13 -00 replies
4/8selftests/bpf: Refactor timer selfteststools/testing/selftests/bpf/prog_tests/timer.c+36 -190 replies
5/8selftests/bpf: Add stress test for timer async canceltools/testing/selftests/bpf/prog_tests/timer.c tools/testing/selftests/bpf/progs/timer.c+28 -40 replies
6/8selftests/bpf: Verify bpf_timer_cancel_async workstools/testing/selftests/bpf/prog_tests/timer.c tools/testing/selftests/bpf/progs/timer.c+48 -00 replies
7/8selftests/bpf: Add timer stress test in NMI contexttools/testing/selftests/bpf/prog_tests/timer.c tools/testing/selftests/bpf/progs/timer.c+231 -120 replies
8/8selftests/bpf: Removed obsolete teststools/testing/selftests/bpf/progs/verifier_helper_restricted.c+0 -1110 replies

Reviews trailers per patch

2 of 8 patches have a Reviewed-by or Acked-by.

PatchKumar Kartikeya DwivediAndrii Nakryiko
1/8A-b R-b
2/8A-b
3/8
4/8
5/8
6/8
7/8
8/8

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v8 thread

Cover letter [PATCH bpf-next v8 0/8] bpf: Avoid locks in bpf_timer and bpf_wq

This series reworks implementation of BPF timer and workqueue APIs. The goal is to make both timers and wq non-blocking, enabling their use in NMI context.
Today this code relies on a bpf_spin_lock embedded in the map element to serialize:
* init of the async object,
* setting/changing the callback and bpf_prog
* starting/cancelling the timer/work
* tearing down when the map element is deleted or the map’s user ref is dropped

The basic design approach in this series:
* Use irq_work to offload all blocking work from NMI
* Introduce refcount to guarantee lifetime of the bpf_async_cb structs deferred to potentially multiple irq_work callbacks
* Keep objects under RCU protection to make sure they are not freed while kfuncs/helpers access them (We can't use refcnt for this, as refcnt itself is part of the bpf_async_cb struct)

---
Changes in v8:
- Return -EBUSY in bpf_async_read_op() if last_seq is failed to be set
- In bpf_async_cancel_and_free() drop bpf_async_cb ref after calling bpf_async_process()
- Link to v7: https://lore.kernel.org/r/[email protected]

Changes in v7:
- Addressed Andrii's review points from the previous version - nothing very significang.
- Added NMI stress tests for bpf_timer - hit few verifier failing checks and removed them.
- Address sparse warning in the bpf_async_update_prog_callback()
- Link to v6: https://lore.kernel.org/r/[email protected]

Changes in v6:
- Reworked destruction and refcnt use:
- On cancel_and_free() set last_seq to BPF_ASYNC_DESTROY value, drop map's reference
- In irq work callback, atomically switch DESTROY to DESTROYED, cancel timer/wq
- Free bpf_async_cb on refcnt going to 0.
- Link to v5: https://lore.kernel.org/r/[email protected]

Changes in v5:
- Extracted lock-free algorithm for updating cb->prog and
cb->callback_fn into a function bpf_async_update_prog_callback(),
added a new commit and introduces this function and uses it in __bpf_async_set_callback(), bpf_timer_cancel() and bpf_async_cancel_and_free().
This allows to move the change into the separate commit without breaking correctness.
- Handle NULL prog in bpf_async_update_prog_callback().
- Link to v4: https://lore.kernel.org/r/[email protected]

Changes in v4:
- Handle irq_work_queue failures in both schedule and cancel_and_free
paths: introduced bpf_async_refcnt_dec_cleanup() that decrements refcnt and makes sure if last reference is put, there is at least one irq_work scheduled to execute final cleanup.
- Additional refcnt inc/dec in set_callback() + rcu lock to make sure cleanup is not running at the same time as set_callback().
- Added READ_ONCE where it was needed.
- Squash 'bpf: Refactor __bpf_async_set_callback()' commit into 'bpf:
Add lock-free cell for NMI-safe
async operations'
- Removed mpmc_cell, use seqcount_latch_t instead.
- Link to v3: https://lore.kernel.org/r/[email protected]

Changes in v3:
- Major rework
- Introduce mpmc_cell, allowing concurrent writes and reads
- Implement irq_work deferring
- Adding selftests
- Introduces bpf_timer_cancel_async kfunc
- Link to v2: https://lore.kernel.org/r/[email protected]

Changes in v2:
- Move refcnt initialization and put (from cancel_and_free())
from patch 5 into the patch 4, so that patch 4 has more clear and full implementation and use of refcnt
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Mykyta Yatsenko (8):
bpf: Enable bpf timer and workqueue use in NMI
bpf: Add verifier support for bpf_timer argument in kfuncs
bpf: Introduce bpf_timer_cancel_async() kfunc selftests/bpf: Refactor timer selftests selftests/bpf: Add stress test for timer async cancel selftests/bpf: Verify bpf_timer_cancel_async works selftests/bpf: Add timer stress test in NMI context selftests/bpf: Removed obsolete tests

 kernel/bpf/helpers.c                               | 449 ++++++++++++---------
 kernel/bpf/verifier.c                              |  75 +++-
 tools/testing/selftests/bpf/prog_tests/timer.c     | 250 +++++++++++-
 tools/testing/selftests/bpf/progs/timer.c          | 118 +++++-
 .../bpf/progs/verifier_helper_restricted.c         | 111 -----
 5 files changed, 659 insertions(+), 344 deletions(-)

---
base-commit: ba335bf3a5b8c5f47d56e9be3f96e0989dad5346
change-id: 20251028-timer_nolock-457f5b9daace

Best regards,

Signature · 1 line
Mykyta Yatsenko <[email protected]>