Skip to content

bpf: Fix deadlock with rq_lock in bpf_send_signal()

Yonghong Song bpf 2 versions In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest2 patches 2 repliesIn review
  2. v22 patches 4 repliesSuperseded

Changes in v3 from the cover letter

. simplify selftest send_signal_sched_switch().
The previous version has mmap/munmap inherited from Song's reproducer. They are not necessary in this context.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/2bpf: Fix deadlock with rq_lock in bpf_send_signal()kernel/trace/bpf_trace.c+4 -11 reply
2/2selftests/bpf: add send_signal_sched_switch testtools/testing/selftests/bpf/prog_tests/send_signal_sched_switch.c tools/testing/selftests/bpf/progs/test_send_signal_kern.c+66 -01 reply

Reviews trailers per patch

1 of 2 patches has a Reviewed-by or Acked-by.

PatchAndrii Nakryiko
1/2
2/2A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH bpf v3 0/2] bpf: Fix deadlock with rq_lock in bpf_send_signal()

Commit 8b401f9ed244 ("bpf: implement bpf_send_signal() helper") introduced bpf_send_signal() helper and Commit 8482941f0906 ("bpf: Add bpf_send_signal_thread() helper") added bpf_send_signal_thread() helper. Both helpers try to send a signel to current process or thread.

When bpf_prog is called with scheduler rq_lock held, a deadlock could happen since bpf_send_signal() and bpf_send_signal_thread() will call group_send_sig_info() which may ultimately want to acquire rq_lock() again. This happens in 5.2 and 4.16 based kernels in our production environment with perf_sw_event.

In a different scenario, the following is also possible in the last kernel:
cpu 1:

     do_task_stat <- holding sighand->siglock
     ...
     task_sched_runtime <- trying to grab rq_lock

cpu 2:

     __schedule <- holding rq_lock
     ...
     do_send_sig_info <- trying to grab sighand->siglock

Commit eac9153f2b58 ("bpf/stackmap: Fix deadlock with rq_lock in bpf_get_stack()") has a similar issue with above rq_lock() deadlock. This patch set addressed the issue in a similar way. Patch #1 provided kernel solution and Patch #2 added a selftest.

Changelogs:
v2 -> v3:
. simplify selftest send_signal_sched_switch().
The previous version has mmap/munmap inherited from Song's reproducer. They are not necessary in this context.
v1 -> v2:
. previous fix using task_work in nmi() is incorrect.
there is no nmi() involvement here. Using task_work in all cases might be a solution. But decided to use a similar fix as in Commit eac9153f2b58.

Yonghong Song (2):
bpf: Fix deadlock with rq_lock in bpf_send_signal() selftests/bpf: add send_signal_sched_switch test

 kernel/trace/bpf_trace.c                      |  5 +-
 .../bpf/prog_tests/send_signal_sched_switch.c | 60 +++++++++++++++++++
 .../bpf/progs/test_send_signal_kern.c         |  6 ++
 3 files changed, 70 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/send_signal_sched_switch.c
Signature · 1 line
2.17.1