Skip to contentSkip to conversation

perf/bpf: Fix lockless access to parent_event->prog in perf_event_alloc()

PATCH In review Deepanshu Kartikey 2 participants 4 messages Aug 25 – Sep 23

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 perf/bpf: Fix lockless access to parent_event->prog in perf_event_alloc()

During fork(), perf_event_alloc() reads parent_event->prog locklessly which can race with concurrent detach clearing and freeing the prog via perf_event_free_bpf_handler() or perf_event_detach_bpf_prog(). This can result in a NULL pointer dereference or use-after-free in bpf_prog_inc().

Fix by using READ_ONCE() to atomically read parent_event->prog into a local variable and bpf_prog_inc_not_zero() to safely increment the reference count only if the program is still alive. This handles both tracing and non-tracing event types without requiring any additional locking.

Fixes 85192dbf4de0 ("bpf: Convert bpf_prog refcnt to atomic64_t")

---
v2:
- Drop bpf_event_mutex approach which was wrong for non-tracing events as perf_event_free_bpf_handler() operates locklessly
- Use READ_ONCE() + bpf_prog_inc_not_zero() instead which handles both tracing and non-tracing event types safely without any lock
- Remove extern bpf_event_mutex from perf_event.h (no longer needed)
- Remove static removal from bpf_trace.c (no longer needed)

---
 kernel/events/core.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
kernel/events/core.c +7 -5
index ba5bd6a78fe7..39755bfacc55 100644
@@ -13433,11 +13433,13 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 		overflow_handler = parent_event->overflow_handler;
 		context = parent_event->overflow_handler_context;
 #if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_EVENT_TRACING)
-		if (parent_event->prog) {
-			struct bpf_prog *prog = parent_event->prog;
-
-			bpf_prog_inc(prog);
-			event->prog = prog;
+		struct bpf_prog *prog;
+
+		prog = READ_ONCE(parent_event->prog);
+		if (prog) {
+			prog = bpf_prog_inc_not_zero(prog);
+			if (!IS_ERR(prog))
+				event->prog = prog;
 		}
 #endif
 	}
Signature · 1 line
2.43.0
#1 lore raw
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [Critical] Missing RCU read lock around the lockless read and increment of parent_event->prog allows a Use-After-Free.
- [High] The BPF program reference is leaked on error paths of perf_event_alloc().
--

--- Patch [1]: [PATCH v2] perf/bpf: Fix lockless access to parent_event->prog in perf_event_alloc() --- commit 14870f355d1ed182bbc1e31accf43918dde6051a
Author: Deepanshu Kartikey <[email protected]>

perf/bpf: Fix lockless access to parent_event->prog in perf_event_alloc()

This commit attempts to fix a race condition during fork() where perf_event_alloc() locklessly reads parent_event->prog. It updates the code to use READ_ONCE() and bpf_prog_inc_not_zero() to safely increment the reference count without locks.

Quoted 8 lines
kernel/events/core.c +0 -0
index a6c8e38a31104..9d456b03c4969 100644
@@ -13465,11 +13465,13 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 		overflow_handler = parent_event->overflow_handler;
 		context = parent_event->overflow_handler_context;
 #if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_EVENT_TRACING)

kernel/events/core.c perf_event_alloc(struct perf_event_attr *attr, int cpu, · line 13473

Quoted 10 lines
kernel/events/core.c +4 -4
-		if (parent_event->prog) {
-			struct bpf_prog *prog = parent_event->prog;
 
-			bpf_prog_inc(prog);
-			event->prog = prog;
+		struct bpf_prog *prog;
 
+		prog = READ_ONCE(parent_event->prog);
+		if (prog) {
+			prog = bpf_prog_inc_not_zero(prog);

[Severity: Critical]
This isn't a bug introduced by this patch, but since it aims to fix a race here, does this code still leave a use-after-free window?

Because perf_event_alloc() executes in a sleepable context, the RCU grace period could expire between reading parent_event->prog and calling bpf_prog_inc_not_zero().

If a concurrent thread detaches the BPF program by closing the bpf_link fd, it can locklessly release the program before the atomic increment happens.

Should this section be protected by an rcu_read_lock() and rcu_read_unlock() block to ensure the memory isn't freed before the atomic increment completes?

kernel/events/core.c perf_event_alloc(struct perf_event_attr *attr, int cpu, · line 13442

Quoted 2 lines
kernel/events/core.c +2 -0
+			if (!IS_ERR(prog))
+				event->prog = prog;

[Severity: High]
This is a pre-existing issue, but does this increment leak the BPF program reference if perf_event_alloc() fails later on?

If a subsequent initialization step like attach_perf_ctx_data() or perf_cgroup_connect() fails, perf_event_alloc() returns an error pointer and triggers the cleanup via __free_event().

Unlike _free_event() which calls perf_event_free_bpf_prog(), the __free_event() error cleanup path doesn't appear to release the acquired bpf_prog reference.

Could an unprivileged user systematically exhaust memory by forcing failures during fork() on an inherited perf event?

Quoted 3 lines
kernel/events/core.c +0 -0
 		}
 #endif

}

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#2 lore raw
In reply to Deepanshu Kartikey
Quoted 51 lines · Deepanshu Kartikey

During fork(), perf_event_alloc() reads parent_event->prog locklessly which can race with concurrent detach clearing and freeing the prog via perf_event_free_bpf_handler() or perf_event_detach_bpf_prog(). This can result in a NULL pointer dereference or use-after-free in bpf_prog_inc().

Fix by using READ_ONCE() to atomically read parent_event->prog into a local variable and bpf_prog_inc_not_zero() to safely increment the reference count only if the program is still alive. This handles both tracing and non-tracing event types without requiring any additional locking.

Fixes 85192dbf4de0 ("bpf: Convert bpf_prog refcnt to atomic64_t")

---
v2:
- Drop bpf_event_mutex approach which was wrong for non-tracing events as perf_event_free_bpf_handler() operates locklessly
- Use READ_ONCE() + bpf_prog_inc_not_zero() instead which handles both tracing and non-tracing event types safely without any lock
- Remove extern bpf_event_mutex from perf_event.h (no longer needed)
- Remove static removal from bpf_trace.c (no longer needed)

---
 kernel/events/core.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
kernel/events/core.c +7 -6
index ba5bd6a78fe7..39755bfacc55 100644
@@ -13433,11 +13433,13 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
                overflow_handler = parent_event->overflow_handler;
                context = parent_event->overflow_handler_context;
 #if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_EVENT_TRACING)
-               if (parent_event->prog) {
-                       struct bpf_prog *prog = parent_event->prog;
-
-                       bpf_prog_inc(prog);
-                       event->prog = prog;
+               struct bpf_prog *prog;
+
+               prog = READ_ONCE(parent_event->prog);
+               if (prog) {
+                       prog = bpf_prog_inc_not_zero(prog);
+                       if (!IS_ERR(prog))
+                               event->prog = prog;
                }
 #endif
        }
--
Quoted 2 lines

2.43.0

Gentle Reminder. Please let me know the status of this patch.

Thanks

Deepanshu

#3 lore raw
In reply to Deepanshu Kartikey
Quoted 51 lines · Deepanshu Kartikey

During fork(), perf_event_alloc() reads parent_event->prog locklessly which can race with concurrent detach clearing and freeing the prog via perf_event_free_bpf_handler() or perf_event_detach_bpf_prog(). This can result in a NULL pointer dereference or use-after-free in bpf_prog_inc().

Fix by using READ_ONCE() to atomically read parent_event->prog into a local variable and bpf_prog_inc_not_zero() to safely increment the reference count only if the program is still alive. This handles both tracing and non-tracing event types without requiring any additional locking.

Fixes 85192dbf4de0 ("bpf: Convert bpf_prog refcnt to atomic64_t")

---
v2:
- Drop bpf_event_mutex approach which was wrong for non-tracing events as perf_event_free_bpf_handler() operates locklessly
- Use READ_ONCE() + bpf_prog_inc_not_zero() instead which handles both tracing and non-tracing event types safely without any lock
- Remove extern bpf_event_mutex from perf_event.h (no longer needed)
- Remove static removal from bpf_trace.c (no longer needed)

---
 kernel/events/core.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
kernel/events/core.c +7 -6
index ba5bd6a78fe7..39755bfacc55 100644
@@ -13433,11 +13433,13 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
                overflow_handler = parent_event->overflow_handler;
                context = parent_event->overflow_handler_context;
 #if defined(CONFIG_BPF_SYSCALL) && defined(CONFIG_EVENT_TRACING)
-               if (parent_event->prog) {
-                       struct bpf_prog *prog = parent_event->prog;
-
-                       bpf_prog_inc(prog);
-                       event->prog = prog;
+               struct bpf_prog *prog;
+
+               prog = READ_ONCE(parent_event->prog);
+               if (prog) {
+                       prog = bpf_prog_inc_not_zero(prog);
+                       if (!IS_ERR(prog))
+                               event->prog = prog;
                }
 #endif
        }
--
Quoted 2 lines

2.43.0

Gentle Reminder. Please let me know the status of this patch.

Thanks

Deepanshu