Skip to contentSkip to conversation

perf/core: Fix possible deadlock in sys_perf_event_open()

PATCH In review 李棒(伯兮) 2 participants 2 messages Sep 6, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 perf/core: Fix possible deadlock in sys_perf_event_open()

In certain scenarios, gctx and ctx may be equal in the __perf_event_ctx_lock_double() function, resulting in a deadlock.

Thread 1, thread 2 and thread 3 belong to the same process, and the process number is assumed to be M. The deadlock scenario is as follows:

1) Thread 1 creates a pure software group through the system call sys_perf_event_open() and returns an fd, assuming the value of fd is N. The parameters of sys_perf_event_open() are as follows.

   For example:
    perf_event_attr.type = PERF_TYPE_SOFTWARE;
    pid = M;
    cpu = 0;
    group_fd = -1;
    flags = 0;
    N = sys_perf_event_open(&perf_event_attr, pid, cpu, group_fd, flags);

2) Thread 2 and thread 3 call the perf_event_open() function concurrently with the same parameters on a different cpu. And use the fd generated by thread 1 as group_fd. The parameters of sys_perf_event_open() are as follows.

   For example:
    perf_event_attr.type = PERF_TYPE_HARDWARE;
    pid = M;
    cpu = 0;
    group_fd = N;
    flags = 0;
    sys_perf_event_open(&perf_event_attr, pid, cpu, group_fd, flags);

3) In the __perf_event_ctx_lock_double function, assuming that thread 2 successfully acquires gctx->mutex and ctx->mutex first, thread 3 will wait here. At the same time, thread 2 will move the pure software gruop to the hardware context and change group_leader->ctx to the hardware context.

4) When thread 2 releases gctx->mutex and ctx->mutex, thread 3 acquires gctx->mutex and ctx->mutex. And find that group_leader->ctx != gctx, then reacquires gctx. At this time, gctx is equal to the ctx of thread 3, triggering a deadlock.

Fixes 321027c1fe77 ("perf/core: Fix concurrent sys_perf_event_open() vs. 'move_group' race")
---
 kernel/events/core.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/events/core.c +10 -0
index db1065daabb6..8af980cdd3a5 100644
@@ -12161,6 +12161,11 @@ __perf_event_ctx_lock_double(struct perf_event *group_leader,
 	}
 	rcu_read_unlock();
 
+	if (gctx == ctx) {
+		put_ctx(gctx);
+		return NULL;
+	}
+
 	mutex_lock_double(&gctx->mutex, &ctx->mutex);
 
 	if (group_leader->ctx != gctx) {
@@ -12465,6 +12470,11 @@ SYSCALL_DEFINE5(perf_event_open,
 
 	if (move_group) {
 		gctx = __perf_event_ctx_lock_double(group_leader, ctx);
+		if (unlikely(!gctx)) {
+			mutex_lock(&ctx->mutex);
+			move_group = 0;
+			goto not_move_group;
+		}
 
 		if (gctx->task == TASK_TOMBSTONE) {
 			err = -ESRCH;
Signature · 1 line
2.19.1.6.gb485710b
#1 lore raw
Quoted 42 lines · 李棒(伯兮)

In certain scenarios, gctx and ctx may be equal in the __perf_event_ctx_lock_double() function, resulting in a deadlock.

Thread 1, thread 2 and thread 3 belong to the same process, and the process number is assumed to be M. The deadlock scenario is as follows:

1) Thread 1 creates a pure software group through the system call sys_perf_event_open() and returns an fd, assuming the value of fd is N. The parameters of sys_perf_event_open() are as follows.

   For example:
    perf_event_attr.type = PERF_TYPE_SOFTWARE;
    pid = M;
    cpu = 0;
    group_fd = -1;
    flags = 0;
    N = sys_perf_event_open(&perf_event_attr, pid, cpu, group_fd, flags);

2) Thread 2 and thread 3 call the perf_event_open() function concurrently with the same parameters on a different cpu. And use the fd generated by thread 1 as group_fd. The parameters of sys_perf_event_open() are as follows.

   For example:
    perf_event_attr.type = PERF_TYPE_HARDWARE;
    pid = M;
    cpu = 0;
    group_fd = N;
    flags = 0;
    sys_perf_event_open(&perf_event_attr, pid, cpu, group_fd, flags);

3) In the __perf_event_ctx_lock_double function, assuming that thread 2 successfully acquires gctx->mutex and ctx->mutex first, thread 3 will wait here. At the same time, thread 2 will move the pure software gruop to the hardware context and change group_leader->ctx to the hardware context.

4) When thread 2 releases gctx->mutex and ctx->mutex, thread 3 acquires gctx->mutex and ctx->mutex. And find that group_leader->ctx != gctx, then reacquires gctx. At this time, gctx is equal to the ctx of thread 3, triggering a deadlock.

Quoted 6 lines
Fixes 321027c1fe77 ("perf/core: Fix concurrent sys_perf_event_open() vs. 'move_group' race")
---
 kernel/events/core.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

<formletter>

This is not the correct way to submit patches for inclusion in the stable kernel tree. Please read:
https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
for how to do this properly.

</formletter>