Skip to contentSkip to conversation

Task stuck on global_ctx_data_rwsem

BUG Namhyung Kim 2 participants 8 messages Dec 22, 2025 – Jan 8 perf events

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

Hello,

I got a report that a task is stuck in perf_event_exit_task() waiting for global_ctx_data_rwsem. On large systems, it'd have performance issues when it grabs the lock to iterate all threads in the system to allocate the context data. And it'd block task exit path which is problematic especially under memory pressure.

perf_event_open
perf_event_alloc
attach_perf_ctx_data
attach_global_ctx_data
percpu_down_write (global_ctx_data_rwsem)
for_each_process_thread
alloc_task_ctx_data
do_exit
perf_event_exit_task
percpu_down_read (global_ctx_data_rwsem)

I think attach_global_ctx_data() should skip tasks with PF_EXITING and it'd be nice if perf_event_exit_task() could release the ctx_data unconditionally. But I'm not sure how to synchronize them properly.

Any thoughts?

Thanks,
Namhyung

author maintainer
#1 lore raw

Re: [BUG] perf/core: Task stuck on global_ctx_data_rwsem

Added a subject prefix and CC LKML.

Thanks,
Namhyung

Quoted 28 lines · Namhyung Kim

Hello,

I got a report that a task is stuck in perf_event_exit_task() waiting for global_ctx_data_rwsem. On large systems, it'd have performance issues when it grabs the lock to iterate all threads in the system to allocate the context data. And it'd block task exit path which is problematic especially under memory pressure.

perf_event_open
perf_event_alloc
attach_perf_ctx_data
attach_global_ctx_data
percpu_down_write (global_ctx_data_rwsem)
for_each_process_thread
alloc_task_ctx_data
do_exit
perf_event_exit_task
percpu_down_read (global_ctx_data_rwsem)

I think attach_global_ctx_data() should skip tasks with PF_EXITING and it'd be nice if perf_event_exit_task() could release the ctx_data unconditionally. But I'm not sure how to synchronize them properly.

Any thoughts?

Thanks,
Namhyung

author maintainer
#2 lore raw

Re: [BUG] perf/core: Task stuck on global_ctx_data_rwsem

Hello,

Quoted 20 lines · Namhyung Kim
Quoted 19 lines · Namhyung Kim

Hello,

I got a report that a task is stuck in perf_event_exit_task() waiting for global_ctx_data_rwsem. On large systems, it'd have performance issues when it grabs the lock to iterate all threads in the system to allocate the context data. And it'd block task exit path which is problematic especially under memory pressure.

perf_event_open
perf_event_alloc
attach_perf_ctx_data
attach_global_ctx_data
percpu_down_write (global_ctx_data_rwsem)
for_each_process_thread
alloc_task_ctx_data
do_exit
perf_event_exit_task
percpu_down_read (global_ctx_data_rwsem)

Quoted 5 lines
Quoted 5 lines

I think attach_global_ctx_data() should skip tasks with PF_EXITING and it'd be nice if perf_event_exit_task() could release the ctx_data unconditionally. But I'm not sure how to synchronize them properly.

Any thoughts?

I'm curious if this makes any sense.. I feel like it needs to check the flag again before allocation.

Thanks,
Namhyung

kernel/events/core.c +2 -1
index 376fb07d869b8b50..2a8847e95d7eb698 100644
@@ -5469,6 +5469,8 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache)
 	/* Allocate everything */
 	scoped_guard (rcu) {
 		for_each_process_thread(g, p) {
+			if (p->flags & PF_EXITING)
+				continue;
 			cd = rcu_dereference(p->perf_ctx_data);
 			if (cd && !cd->global) {
 				cd->global = 1;
@@ -14563,7 +14565,6 @@ void perf_event_exit_task(struct task_struct *task)
 	/*
 	 * Detach the perf_ctx_data for the system-wide event.
 	 */
-	guard(percpu_read)(&global_ctx_data_rwsem);
 	detach_task_ctx_data(task);
 }
 
maintainer
#3 lore raw

Re: [BUG] perf/core: Task stuck on global_ctx_data_rwsem

Quoted 41 lines · Namhyung Kim

Hello,

Quoted 20 lines · Namhyung Kim
Quoted 19 lines · Namhyung Kim

Hello,

I got a report that a task is stuck in perf_event_exit_task() waiting for global_ctx_data_rwsem. On large systems, it'd have performance issues when it grabs the lock to iterate all threads in the system to allocate the context data. And it'd block task exit path which is problematic especially under memory pressure.

perf_event_open
perf_event_alloc
attach_perf_ctx_data
attach_global_ctx_data
percpu_down_write (global_ctx_data_rwsem)
for_each_process_thread
alloc_task_ctx_data
do_exit
perf_event_exit_task
percpu_down_read (global_ctx_data_rwsem)

Quoted 5 lines
Quoted 5 lines

I think attach_global_ctx_data() should skip tasks with PF_EXITING and it'd be nice if perf_event_exit_task() could release the ctx_data unconditionally. But I'm not sure how to synchronize them properly.

Any thoughts?

I'm curious if this makes any sense.. I feel like it needs to check the flag again before allocation.

Thanks,
Namhyung

kernel/events/core.c +0 -0
index 376fb07d869b8b50..2a8847e95d7eb698 100644

kernel/events/core.c attach_global_ctx_data(struct kmem_cache *ctx_cache) · line 5476

Quoted 8 lines
kernel/events/core.c +2 -0
@@ -5469,6 +5469,8 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache)
 	/* Allocate everything */
 	scoped_guard (rcu) {
 		for_each_process_thread(g, p) {
+			if (p->flags & PF_EXITING)
+				continue;
 			cd = rcu_dereference(p->perf_ctx_data);
 			if (cd && !cd->global) {
 				cd->global = 1;

I suppose this makes sense.

kernel/events/core.c void perf_event_exit_task(struct task_struct *task) · line 14569

Quoted 7 lines
kernel/events/core.c +0 -1
@@ -14563,7 +14565,6 @@ void perf_event_exit_task(struct task_struct *task)
 	/*
 	 * Detach the perf_ctx_data for the system-wide event.
 	 */
-	guard(percpu_read)(&global_ctx_data_rwsem);
 	detach_task_ctx_data(task);
 }

This would need a comment; something like:

	/*
	 * This can be done without holding global_ctx_data_rwsem
	 * because this is done after setting PF_EXITING such that
	 * attach_global_ctx_data() will skip over this task.
	 */
	WARN_ON_ONCE(!(task->flags & PF_EXITING))

But yes, I suppose this can do. The question is however, how do you get into this predicament to begin with? Are you creating and destroying a lot of global LBR events or something?

Would it make sense to delay detach_global_ctx_data() for a second or so? That is, what is your event creation pattern?

author maintainer
#4 lore raw

Re: [BUG] perf/core: Task stuck on global_ctx_data_rwsem

Quoted 70 lines · Peter Zijlstra
Quoted 41 lines · Namhyung Kim

Hello,

Quoted 20 lines · Namhyung Kim
Quoted 19 lines · Namhyung Kim

Hello,

I got a report that a task is stuck in perf_event_exit_task() waiting for global_ctx_data_rwsem. On large systems, it'd have performance issues when it grabs the lock to iterate all threads in the system to allocate the context data. And it'd block task exit path which is problematic especially under memory pressure.

perf_event_open
perf_event_alloc
attach_perf_ctx_data
attach_global_ctx_data
percpu_down_write (global_ctx_data_rwsem)
for_each_process_thread
alloc_task_ctx_data
do_exit
perf_event_exit_task
percpu_down_read (global_ctx_data_rwsem)

Quoted 5 lines
Quoted 5 lines

I think attach_global_ctx_data() should skip tasks with PF_EXITING and it'd be nice if perf_event_exit_task() could release the ctx_data unconditionally. But I'm not sure how to synchronize them properly.

Any thoughts?

I'm curious if this makes any sense.. I feel like it needs to check the flag again before allocation.

Thanks,
Namhyung

kernel/events/core.c +0 -0
index 376fb07d869b8b50..2a8847e95d7eb698 100644

kernel/events/core.c attach_global_ctx_data(struct kmem_cache *ctx_cache) · line 5476

Quoted 8 lines
kernel/events/core.c +2 -0
@@ -5469,6 +5469,8 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache)
 	/* Allocate everything */
 	scoped_guard (rcu) {
 		for_each_process_thread(g, p) {
+			if (p->flags & PF_EXITING)
+				continue;
 			cd = rcu_dereference(p->perf_ctx_data);
 			if (cd && !cd->global) {
 				cd->global = 1;

I suppose this makes sense.

kernel/events/core.c void perf_event_exit_task(struct task_struct *task) · line 14569

Quoted 7 lines
kernel/events/core.c +0 -1
@@ -14563,7 +14565,6 @@ void perf_event_exit_task(struct task_struct *task)
 	/*
 	 * Detach the perf_ctx_data for the system-wide event.
 	 */
-	guard(percpu_read)(&global_ctx_data_rwsem);
 	detach_task_ctx_data(task);
 }

This would need a comment; something like:

	/*
	 * This can be done without holding global_ctx_data_rwsem
	 * because this is done after setting PF_EXITING such that
	 * attach_global_ctx_data() will skip over this task.
	 */
	WARN_ON_ONCE(!(task->flags & PF_EXITING))
Quoted 3 lines

But yes, I suppose this can do. The question is however, how do you get into this predicament to begin with? Are you creating and destroying a lot of global LBR events or something?

I think it's just because there are too many tasks in the system like O(100K). And any thread going to exit needs to wait for attach_global_ctx_data() to finish the iteration over every task.

Quoted 3 lines

Would it make sense to delay detach_global_ctx_data() for a second or so? That is, what is your event creation pattern?

I don't think it has a special pattern, but I'm curious how we can handle a race like below.

attach_global_ctx_data

    check p->flags & PF_EXITING
                                              do_exit
    (preemption)                                set PF_EXITING
                                                detach_task_ctx_data()
    check p->perf_ctx_data
    attach_task_ctx_data()   ---> memory leak

Thanks,
Namhyung

maintainer
#5 lore raw

Re: [BUG] perf/core: Task stuck on global_ctx_data_rwsem

Quoted 7 lines · Namhyung Kim
Quoted 3 lines

But yes, I suppose this can do. The question is however, how do you get into this predicament to begin with? Are you creating and destroying a lot of global LBR events or something?

I think it's just because there are too many tasks in the system like O(100K). And any thread going to exit needs to wait for attach_global_ctx_data() to finish the iteration over every task.

OMG, so many tasks ...

Quoted 6 lines
Quoted 2 lines

Would it make sense to delay detach_global_ctx_data() for a second or so? That is, what is your event creation pattern?

I don't think it has a special pattern, but I'm curious how we can handle a race like below.

Quoted 7 lines

attach_global_ctx_data

    check p->flags & PF_EXITING
                                              do_exit
    (preemption)                                set PF_EXITING
                                                detach_task_ctx_data()
    check p->perf_ctx_data
    attach_task_ctx_data()   ---> memory leak

Oh right. Something like so perhaps?

---

kernel/events/core.c +17 -2
index 3c2a491200c6..e5e716420eb3 100644
@@ -5421,9 +5421,19 @@ attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cache,
 		return -ENOMEM;
 
 	for (;;) {
-		if (try_cmpxchg((struct perf_ctx_data **)&task->perf_ctx_data, &old, cd)) {
+		if (try_cmpxchg(&task->perf_ctx_data, &old, cd)) {
 			if (old)
 				perf_free_ctx_data_rcu(old);
+			/*
+			 * try_cmpxchg() pairs with try_cmpxchg() from
+			 * detach_task_ctx_data() such that
+			 * if we race with perf_event_exit_task(), we must
+			 * observe PF_EXITING.
+			 */
+			if (task->flags & PF_EXITING) {
+				task->perf_ctx_data = NULL;
+				perf_free_ctx_data_rcu(cd);
+			}
 			return 0;
 		}
 
@@ -5469,6 +5479,8 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache)
 	/* Allocate everything */
 	scoped_guard (rcu) {
 		for_each_process_thread(g, p) {
+			if (p->flags & PF_EXITING)
+				continue;
 			cd = rcu_dereference(p->perf_ctx_data);
 			if (cd && !cd->global) {
 				cd->global = 1;
@@ -14568,8 +14580,11 @@ void perf_event_exit_task(struct task_struct *task)
 
 	/*
 	 * Detach the perf_ctx_data for the system-wide event.
+	 *
+	 * Done without holding global_ctx_data_rwsem; typically
+	 * attach_global_ctx_data() will skip over this task, but otherwise
+	 * attach_task_ctx_data() will observe PF_EXITING.
 	 */
-	guard(percpu_read)(&global_ctx_data_rwsem);
 	detach_task_ctx_data(task);
 }
 
maintainer
#6 lore raw

Re: [BUG] perf/core: Task stuck on global_ctx_data_rwsem

Quoted 40 lines · Peter Zijlstra
Quoted 7 lines · Namhyung Kim
Quoted 3 lines

But yes, I suppose this can do. The question is however, how do you get into this predicament to begin with? Are you creating and destroying a lot of global LBR events or something?

I think it's just because there are too many tasks in the system like O(100K). And any thread going to exit needs to wait for attach_global_ctx_data() to finish the iteration over every task.

OMG, so many tasks ...

Quoted 6 lines
Quoted 2 lines

Would it make sense to delay detach_global_ctx_data() for a second or so? That is, what is your event creation pattern?

I don't think it has a special pattern, but I'm curious how we can handle a race like below.

Quoted 7 lines

attach_global_ctx_data

    check p->flags & PF_EXITING
                                              do_exit
    (preemption)                                set PF_EXITING
                                                detach_task_ctx_data()
    check p->perf_ctx_data
    attach_task_ctx_data()   ---> memory leak

Oh right. Something like so perhaps?

---

kernel/events/core.c +1 -1
index 3c2a491200c6..e5e716420eb3 100644
@@ -5421,9 +5421,19 @@ attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cache,
 		return -ENOMEM;
 
 	for (;;) {
-		if (try_cmpxchg((struct perf_ctx_data **)&task->perf_ctx_data, &old, cd)) {
+		if (try_cmpxchg(&task->perf_ctx_data, &old, cd)) {
 			if (old)

kernel/events/core.c attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cache, · line 5435

Quoted 10 lines
kernel/events/core.c +9 -0
 				perf_free_ctx_data_rcu(old);
+			/*
+			 * try_cmpxchg() pairs with try_cmpxchg() from
+			 * detach_task_ctx_data() such that
+			 * if we race with perf_event_exit_task(), we must
+			 * observe PF_EXITING.
+			 */
+			if (task->flags & PF_EXITING) {
+				task->perf_ctx_data = NULL;
+				perf_free_ctx_data_rcu(cd);

Ugh and now it can race and do a double free, another try_cmpxchg() is needed here.

Quoted 26 lines
kernel/events/core.c +7 -1
+			}
 			return 0;
 		}
 
@@ -5469,6 +5479,8 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache)
 	/* Allocate everything */
 	scoped_guard (rcu) {
 		for_each_process_thread(g, p) {
+			if (p->flags & PF_EXITING)
+				continue;
 			cd = rcu_dereference(p->perf_ctx_data);
 			if (cd && !cd->global) {
 				cd->global = 1;
@@ -14568,8 +14580,11 @@ void perf_event_exit_task(struct task_struct *task)
 
 	/*
 	 * Detach the perf_ctx_data for the system-wide event.
+	 *
+	 * Done without holding global_ctx_data_rwsem; typically
+	 * attach_global_ctx_data() will skip over this task, but otherwise
+	 * attach_task_ctx_data() will observe PF_EXITING.
 	 */
-	guard(percpu_read)(&global_ctx_data_rwsem);
 	detach_task_ctx_data(task);
 }
 
author maintainer
#7 lore raw

Re: [BUG] perf/core: Task stuck on global_ctx_data_rwsem

Quoted 46 lines · Peter Zijlstra
Quoted 45 lines · Peter Zijlstra
Quoted 7 lines · Namhyung Kim
Quoted 3 lines

But yes, I suppose this can do. The question is however, how do you get into this predicament to begin with? Are you creating and destroying a lot of global LBR events or something?

I think it's just because there are too many tasks in the system like O(100K). And any thread going to exit needs to wait for attach_global_ctx_data() to finish the iteration over every task.

OMG, so many tasks ...

Quoted 6 lines
Quoted 2 lines

Would it make sense to delay detach_global_ctx_data() for a second or so? That is, what is your event creation pattern?

I don't think it has a special pattern, but I'm curious how we can handle a race like below.

Quoted 7 lines

attach_global_ctx_data

    check p->flags & PF_EXITING
                                              do_exit
    (preemption)                                set PF_EXITING
                                                detach_task_ctx_data()
    check p->perf_ctx_data
    attach_task_ctx_data()   ---> memory leak

Oh right. Something like so perhaps?

---

kernel/events/core.c +5 -1
index 3c2a491200c6..e5e716420eb3 100644
@@ -5421,9 +5421,19 @@ attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cache,
 		return -ENOMEM;
 
 	for (;;) {
-		if (try_cmpxchg((struct perf_ctx_data **)&task->perf_ctx_data, &old, cd)) {
+		if (try_cmpxchg(&task->perf_ctx_data, &old, cd)) {
 			if (old)
 				perf_free_ctx_data_rcu(old);
+			/*
+			 * try_cmpxchg() pairs with try_cmpxchg() from
+			 * detach_task_ctx_data() such that
+			 * if we race with perf_event_exit_task(), we must
Quoted 8 lines

kernel/events/core.c attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cache, · line 5435

Quoted 5 lines
kernel/events/core.c +5 -0
+			 * observe PF_EXITING.
+			 */
+			if (task->flags & PF_EXITING) {
+				task->perf_ctx_data = NULL;
+				perf_free_ctx_data_rcu(cd);

Ugh and now it can race and do a double free, another try_cmpxchg() is needed here.

Thanks! Something like this?

Namhyung

kernel/events/core.c +18 -2
index 376fb07d869b8b50..cf252d8f49b2b259 100644
@@ -5421,9 +5421,20 @@ attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cache,
 		return -ENOMEM;
 
 	for (;;) {
-		if (try_cmpxchg((struct perf_ctx_data **)&task->perf_ctx_data, &old, cd)) {
+		if (try_cmpxchg(&task->perf_ctx_data, &old, cd)) {
 			if (old)
 				perf_free_ctx_data_rcu(old);
+			/*
+			 * try_cmpxchg() pairs with try_cmpxchg() from
+			 * detach_task_ctx_data() such that
+			 * if we race with perf_event_exit_task(), we must
+			 * observe PF_EXITING.
+			 */
+			if (task->flags & PF_EXITING) {
+				/* detach_task_ctx_data() may free it already */
+				if (try_cmpxchg(&task->perf_ctx_data, &cd, NULL))
+					perf_free_ctx_data_rcu(cd);
+			}
 			return 0;
 		}
 
@@ -5469,6 +5480,8 @@ attach_global_ctx_data(struct kmem_cache *ctx_cache)
 	/* Allocate everything */
 	scoped_guard (rcu) {
 		for_each_process_thread(g, p) {
+			if (p->flags & PF_EXITING)
+				continue;
 			cd = rcu_dereference(p->perf_ctx_data);
 			if (cd && !cd->global) {
 				cd->global = 1;
@@ -14562,8 +14575,11 @@ void perf_event_exit_task(struct task_struct *task)
 
 	/*
 	 * Detach the perf_ctx_data for the system-wide event.
+	 *
+	 * Done without holding global_ctx_data_rwsem; typically
+	 * attach_global_ctx_data() will skip over this task, but otherwise
+	 * attach_task_ctx_data() will observe PF_EXITING.
 	 */
-	guard(percpu_read)(&global_ctx_data_rwsem);
 	detach_task_ctx_data(task);
 }