Skip to contentSkip to conversation

sched/psi: Fix psimon fork deadlock and rtpoll_timer UAF

PATCH In review Tejun Heo 5 participants 14 messages Jul 12 – Jul 15

Subsystems (5)
lore
Share & tools
lore

Tejun Heo

author maintainer
#0 lore raw

PATCH v2 1/1 sched/psi: Fix psimon fork deadlock and rtpoll_timer UAF

Hello,

v2: - Retagged sched/psi (was cgroup) (patch 1).
- Added a fix for a pre-existing rtpoll_timer use-after-free that the Sashiko AI review flagged on the previous posting (patch 2).

v1: https://lore.kernel.org/r/[email protected]

a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() fork the psimon kthread while holding cgroup_mutex, which deadlocks against the sched_ext enable path blocking forks before grabbing cgroup_mutex. Patch 1 moves the fork outside cgroup_mutex. Patch 2 fixes the use-after-free from a schedule attempt re-arming group->rtpoll_timer behind psi_trigger_destroy() by shutting down the timer when the group is freed.

Matt, patch 1 is unchanged from the previous posting except for the subject prefix, so test results against that posting still apply.

Peter, once these are reviewed and tested, how do you want them routed? I can take them through the cgroup tree if that works for you.

Patches:

[PATCH 1/2] sched/psi: Create the psimon kthread outside of cgroup_mutex [PATCH 2/2] sched/psi: Shut down rtpoll_timer in psi_cgroup_free()

Based on cgroup/for-7.2-fixes (97fef6025844).

 include/linux/psi.h    |  4 ++-
 kernel/cgroup/cgroup.c | 23 +++++++++++++++-
 kernel/sched/psi.c     | 75 +++++++++++++++++++++++++++++++++++++++-----------
 3 files changed, 84 insertions(+), 18 deletions(-)

Thanks.

--
tejun

author maintainer
#1 lore raw

PATCH 1/2 sched/psi: Create the psimon kthread outside of cgroup_mutex

a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() hold cgroup_mutex across psi_trigger_create(), which forks the psimon kthread for the first rtpoll trigger. As kthread creation depends on the whole fork path, the commit inadvertently created a lot of unwanted locking dependencies from cgroup_mutex.

sched_ext got hit by one: its enable path blocks forks and then grabs cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with every other fork piling up behind.

Fix it by splitting trigger creation so that the worker is forked with cgroup_mutex dropped and the kernfs active reference left broken. The latter matters because rmdir and cgroup.pressure writes drain active references under cgroup_mutex. Publishing the trigger last keeps error reporting synchronous and preserves the of->priv lifetime rules.

The trigger registered in the first stage pins the group's rtpoll machinery across the unlocked window, leaving only creation races to resolve. The catch-up poll on installation covers scheduling attempts dropped while there was no worker.

v2: Retagged sched/psi (was cgroup).

Fixes a5b98009f16d ("sched/psi: fix race between file release and pressure write")
Cc Johannes Weiner <[email protected]>
Cc Edward Adam Davis <[email protected]>
Cc Chen Ridong <[email protected]>
---
 include/linux/psi.h    |  4 ++-
 kernel/cgroup/cgroup.c | 23 +++++++++++++-
 kernel/sched/psi.c     | 69 ++++++++++++++++++++++++++++++++----------
 3 files changed, 78 insertions(+), 18 deletions(-)
include/linux/psi.h +3 -1
index e0745873e3f2..7966e3ac03b9 100644
@@ -25,7 +25,9 @@ void psi_memstall_leave(unsigned long *flags);
 int psi_show(struct seq_file *s, struct psi_group *group, enum psi_res res);
 struct psi_trigger *psi_trigger_create(struct psi_group *group, char *buf,
 				       enum psi_res res, struct file *file,
-				       struct kernfs_open_file *of);
+				       struct kernfs_open_file *of,
+				       bool *need_rtpoll_worker);
+int psi_trigger_create_rtpoll_worker(struct psi_group *group);
 void psi_trigger_destroy(struct psi_trigger *t);
 
 __poll_t psi_trigger_poll(void **trigger_ptr, struct file *file,
kernel/cgroup/cgroup.c +22 -1
index 38f8d9df8fbc..b5b461d4418b 100644
@@ -3996,6 +3996,7 @@ static ssize_t pressure_write(struct kernfs_open_file *of, char *buf,
 	struct psi_trigger *new;
 	struct cgroup *cgrp;
 	struct psi_group *psi;
+	bool need_rtpoll_worker;
 	ssize_t ret = 0;
 
 	cgrp = cgroup_kn_lock_live(of->kn, false);
@@ -4015,12 +4016,32 @@ static ssize_t pressure_write(struct kernfs_open_file *of, char *buf,
 	}
 
 	psi = cgroup_psi(cgrp);
-	new = psi_trigger_create(psi, buf, res, of->file, of);
+	new = psi_trigger_create(psi, buf, res, of->file, of,
+				 &need_rtpoll_worker);
 	if (IS_ERR(new)) {
 		ret = PTR_ERR(new);
 		goto out_unlock;
 	}
 
+	/*
+	 * The worker fork must run with neither cgroup_mutex nor the file's
+	 * kernfs active reference held. The latter is broken since
+	 * cgroup_kn_lock_live(). @of->priv may be released while unlocked, so
+	 * recheck before publishing @new.
+	 */
+	if (need_rtpoll_worker) {
+		cgroup_unlock();
+		ret = psi_trigger_create_rtpoll_worker(psi);
+		cgroup_lock();
+
+		if (!ret && !of->priv)
+			ret = -ENODEV;
+		if (ret) {
+			psi_trigger_destroy(new);
+			goto out_unlock;
+		}
+	}
+
 	smp_store_release(&ctx->psi.trigger, new);
 
 out_unlock:
kernel/sched/psi.c +53 -16

112 lines not on this page · raw message

Signature · 1 line
2.55.0
author maintainer
#7 lore raw

PATCH 2/2 sched/psi: Shut down rtpoll_timer in psi_cgroup_free()

In reply to Tejun Heo

psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath and can race psi_trigger_destroy() taking down the last rtpoll trigger under rtpoll_trigger_lock:

psi_schedule_rtpoll_work() psi_trigger_destroy()

  rcu_read_lock();
  task = rcu_dereference(rtpoll_task);
                                    rcu_assign_pointer(rtpoll_task, NULL);
                                    timer_delete(&rtpoll_timer);
  mod_timer(&rtpoll_timer, ...);
  rcu_read_unlock();
                                    synchronize_rcu();
                                    kthread_stop(task_to_destroy);

The group can then be freed with the re-armed timer still pending, and poll_timer_fn() runs on freed memory.

461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling mechanism") deleted the timer synchronously after the synchronize_rcu(), which prevented this but raced trigger creation instead: the deletion could cancel the timer that a new trigger set armed during the grace period and, as creation also reinitialized the timer at the time, corrupt it. 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the initialization into group_init() and the deletion into the locked section, trading the creation races for the window above.

Neither placement in the destruction path works. A pending timer firing while the group is alive is harmless though. poll_timer_fn() just wakes the rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it by then. timer_shutdown_sync() because the timer is never armed again.

Fixes 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy")
Cc [email protected] # v5.10+
---
 kernel/sched/psi.c | 6 ++++++
 1 file changed, 6 insertions(+)
kernel/sched/psi.c +6 -0
index 565ec7b80743..e2e825dcd088 100644
@@ -1134,6 +1134,12 @@ void psi_cgroup_free(struct cgroup *cgroup)
 		return;
 
 	cancel_delayed_work_sync(&cgroup->psi->avgs_work);
+	/*
+	 * A psi_schedule_rtpoll_work() call racing the last trigger's
+	 * destruction may have re-armed the timer after psi_trigger_destroy()
+	 * deleted it. Spurious firing while the group is alive is harmless.
+	 */
+	timer_shutdown_sync(&cgroup->psi->rtpoll_timer);
 	free_percpu(cgroup->psi->pcpu);
 	/* All triggers must be removed by now */
 	WARN_ONCE(cgroup->psi->rtpoll_states, "psi: trigger leak\n");
Signature · 1 line
2.55.0
#2 lore raw

Re: [PATCH 1/2] sched/psi: Create the psimon kthread outside of cgroup_mutex

In reply to Tejun Heo
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Calling psi_trigger_destroy() while holding cgroup_mutex introduces severe system-wide latency…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Calling psi_trigger_destroy() while holding cgroup_mutex introduces severe system-wide latency spikes. --

--- Patch [1]: [PATCH 1/2] sched/psi: Create the psimon kthread outside of cgroup_mutex --- commit 4990aba341f12c025b0b99ecf8869c68be847294
Author: Tejun Heo <[email protected]>
sched/psi: Create the psimon kthread outside of cgroup_mutex

This commit splits psi trigger creation into two stages to avoid locking dependencies. It drops cgroup_mutex before forking the psimon kthread, then reacquires the mutex to verify the cgroup is still alive before publishing the new trigger.

Quoted 4 lines
kernel/cgroup/cgroup.c +0 -0
index 38f8d9df8fbc4..b5b461d4418b1 100644

[ ... ]

Quoted 17 lines
kernel/cgroup/cgroup.c +7 -1
@@ -4015,12 +4016,32 @@ static ssize_t pressure_write(struct kernfs_open_file *of, char *buf,
 	}
 
 	psi = cgroup_psi(cgrp);
-	new = psi_trigger_create(psi, buf, res, of->file, of);
+	new = psi_trigger_create(psi, buf, res, of->file, of,
+				 &need_rtpoll_worker);
 	if (IS_ERR(new)) {
 		ret = PTR_ERR(new);
 		goto out_unlock;
 	}
 
+	/*
+	 * The worker fork must run with neither cgroup_mutex nor the file's
+	 * kernfs active reference held. The latter is broken since
+	 * cgroup_kn_lock_live(). @of->priv may be released while unlocked, so
+	 * recheck before publishing @new.

kernel/cgroup/cgroup.c static ssize_t pressure_write(struct kernfs_open_file *of, char *buf, · line 4040

Quoted 10 lines
kernel/cgroup/cgroup.c +10 -0
+	 */
+	if (need_rtpoll_worker) {
+		cgroup_unlock();
+		ret = psi_trigger_create_rtpoll_worker(psi);
+		cgroup_lock();
+
+		if (!ret && !of->priv)
+			ret = -ENODEV;
+		if (ret) {
+			psi_trigger_destroy(new);

[Severity: High]
Since psi_trigger_destroy() unconditionally blocks on synchronize_rcu(), will calling it here while holding cgroup_mutex cause system-wide latency spikes?

Looking at psi_trigger_destroy() in kernel/sched/psi.c, it does:

	/*
	 * Wait for psi_schedule_rtpoll_work RCU to complete its read-side
	 * critical section before destroying the trigger and optionally the
	 * rtpoll_task.
	 */
	synchronize_rcu();

If this error path is reached (e.g. by racing pressure_write() with cgroup rmdir making of->priv NULL), the calling thread would sleep for an RCU grace period while holding cgroup_mutex, stalling all other cgroup operations.

Could the trigger destruction be deferred until after cgroup_mutex is dropped, or handled asynchronously to avoid stalling the cgroup subsystem?

Quoted 7 lines
kernel/cgroup/cgroup.c +4 -0
+			goto out_unlock;
+		}
+	}
+
 	smp_store_release(&ctx->psi.trigger, new);
 
 out_unlock:
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

Re: [PATCH 1/2] sched/psi: Create the psimon kthread outside of cgroup_mutex

Hello.

Quoted 6 lines

If this error path is reached (e.g. by racing pressure_write() with cgroup rmdir making of->priv NULL), the calling thread would sleep for an RCU grace period while holding cgroup_mutex, stalling all other cgroup operations.

Could the trigger destruction be deferred until after cgroup_mutex is dropped, or handled asynchronously to avoid stalling the cgroup subsystem?

This is pre-existing - rmdir and cgroup.pressure disable already destroy triggers under cgroup_mutex when draining open fds. Maybe something to improve in the future.

Thanks.

--
tejun

maintainer Acked-by on v1 1/2
#4 lore raw

Re: [PATCH 1/2] sched/psi: Create the psimon kthread outside of cgroup_mutex

In reply to Tejun Heo
Quoted 23 lines · Tejun Heo

a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() hold cgroup_mutex across psi_trigger_create(), which forks the psimon kthread for the first rtpoll trigger. As kthread creation depends on the whole fork path, the commit inadvertently created a lot of unwanted locking dependencies from cgroup_mutex.

sched_ext got hit by one: its enable path blocks forks and then grabs cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with every other fork piling up behind.

Fix it by splitting trigger creation so that the worker is forked with cgroup_mutex dropped and the kernfs active reference left broken. The latter matters because rmdir and cgroup.pressure writes drain active references under cgroup_mutex. Publishing the trigger last keeps error reporting synchronous and preserves the of->priv lifetime rules.

The trigger registered in the first stage pins the group's rtpoll machinery across the unlocked window, leaving only creation races to resolve. The catch-up poll on installation covers scheduling attempts dropped while there was no worker.

v2: Retagged sched/psi (was cgroup).

Quoted 8 lines
Fixes a5b98009f16d ("sched/psi: fix race between file release and pressure write")
Cc Johannes Weiner <[email protected]>
Cc Edward Adam Davis <[email protected]>
Cc Chen Ridong <[email protected]>
maintainer Acked-by on v1 2/2
#8 lore raw

Re: [PATCH 2/2] sched/psi: Shut down rtpoll_timer in psi_cgroup_free()

In reply to Tejun Heo
Quoted 33 lines · Tejun Heo

psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath and can race psi_trigger_destroy() taking down the last rtpoll trigger under rtpoll_trigger_lock:

psi_schedule_rtpoll_work() psi_trigger_destroy()

  rcu_read_lock();
  task = rcu_dereference(rtpoll_task);
                                    rcu_assign_pointer(rtpoll_task, NULL);
                                    timer_delete(&rtpoll_timer);
  mod_timer(&rtpoll_timer, ...);
  rcu_read_unlock();
                                    synchronize_rcu();
                                    kthread_stop(task_to_destroy);

The group can then be freed with the re-armed timer still pending, and poll_timer_fn() runs on freed memory.

461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling mechanism") deleted the timer synchronously after the synchronize_rcu(), which prevented this but raced trigger creation instead: the deletion could cancel the timer that a new trigger set armed during the grace period and, as creation also reinitialized the timer at the time, corrupt it. 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the initialization into group_init() and the deletion into the locked section, trading the creation races for the window above.

Neither placement in the destruction path works. A pending timer firing while the group is alive is harmless though. poll_timer_fn() just wakes the rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it by then. timer_shutdown_sync() because the timer is never armed again.

Quoted 5 lines
Fixes 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy")
Cc [email protected] # v5.10+

Both these patches look good to me, but Suren can you please also take a look?

Tested-by on v1 1/2
#5 lore raw

Re: [PATCH 1/2] sched/psi: Create the psimon kthread outside of cgroup_mutex

In reply to Tejun Heo
Quoted 31 lines · Tejun Heo

a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() hold cgroup_mutex across psi_trigger_create(), which forks the psimon kthread for the first rtpoll trigger. As kthread creation depends on the whole fork path, the commit inadvertently created a lot of unwanted locking dependencies from cgroup_mutex.

sched_ext got hit by one: its enable path blocks forks and then grabs cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with every other fork piling up behind.

Fix it by splitting trigger creation so that the worker is forked with cgroup_mutex dropped and the kernfs active reference left broken. The latter matters because rmdir and cgroup.pressure writes drain active references under cgroup_mutex. Publishing the trigger last keeps error reporting synchronous and preserves the of->priv lifetime rules.

The trigger registered in the first stage pins the group's rtpoll machinery across the unlocked window, leaving only creation races to resolve. The catch-up poll on installation covers scheduling attempts dropped while there was no worker.

v2: Retagged sched/psi (was cgroup).

Fixes a5b98009f16d ("sched/psi: fix race between file release and pressure write")
Cc Johannes Weiner <[email protected]>
Cc Edward Adam Davis <[email protected]>
Cc Chen Ridong <[email protected]>
Quoted 5 lines
---
 include/linux/psi.h    |  4 ++-
 kernel/cgroup/cgroup.c | 23 +++++++++++++-
 kernel/sched/psi.c     | 69 ++++++++++++++++++++++++++++++++----------
 3 files changed, 78 insertions(+), 18 deletions(-)
Tested-by on v1 2/2
#11 lore raw

Re: [PATCH 2/2] sched/psi: Shut down rtpoll_timer in psi_cgroup_free()

In reply to Tejun Heo
Quoted 33 lines · Tejun Heo

psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath and can race psi_trigger_destroy() taking down the last rtpoll trigger under rtpoll_trigger_lock:

psi_schedule_rtpoll_work() psi_trigger_destroy()

  rcu_read_lock();
  task = rcu_dereference(rtpoll_task);
                                    rcu_assign_pointer(rtpoll_task, NULL);
                                    timer_delete(&rtpoll_timer);
  mod_timer(&rtpoll_timer, ...);
  rcu_read_unlock();
                                    synchronize_rcu();
                                    kthread_stop(task_to_destroy);

The group can then be freed with the re-armed timer still pending, and poll_timer_fn() runs on freed memory.

461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling mechanism") deleted the timer synchronously after the synchronize_rcu(), which prevented this but raced trigger creation instead: the deletion could cancel the timer that a new trigger set armed during the grace period and, as creation also reinitialized the timer at the time, corrupt it. 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the initialization into group_init() and the deletion into the locked section, trading the creation races for the window above.

Neither placement in the destruction path works. A pending timer firing while the group is alive is harmless though. poll_timer_fn() just wakes the rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it by then. timer_shutdown_sync() because the timer is never armed again.

Quoted 8 lines
Fixes 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy")
Cc [email protected] # v5.10+
---
 kernel/sched/psi.c | 6 ++++++
 1 file changed, 6 insertions(+)
#12 lore raw
In reply to Tejun Heo
Quoted 16 lines · Tejun Heo

Hello,

v2: - Retagged sched/psi (was cgroup) (patch 1).
- Added a fix for a pre-existing rtpoll_timer use-after-free that the Sashiko AI review flagged on the previous posting (patch 2).

v1: https://lore.kernel.org/r/[email protected]

a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() fork the psimon kthread while holding cgroup_mutex, which deadlocks against the sched_ext enable path blocking forks before grabbing cgroup_mutex. Patch 1 moves the fork outside cgroup_mutex. Patch 2 fixes the use-after-free from a schedule attempt re-arming group->rtpoll_timer behind psi_trigger_destroy() by shutting down the timer when the group is freed.

Quoted 2 lines

Matt, patch 1 is unchanged from the previous posting except for the subject prefix, so test results against that posting still apply.

I ended up testing both patches. This resolves the issue for me!

Thanks,
Matt

#9 lore raw

Re: [PATCH 2/2] sched/psi: Shut down rtpoll_timer in psi_cgroup_free()

In reply to Johannes Weiner
Quoted 41 lines · Johannes Weiner
Quoted 33 lines · Tejun Heo

psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath and can race psi_trigger_destroy() taking down the last rtpoll trigger under rtpoll_trigger_lock:

psi_schedule_rtpoll_work() psi_trigger_destroy()

  rcu_read_lock();
  task = rcu_dereference(rtpoll_task);
                                    rcu_assign_pointer(rtpoll_task, NULL);
                                    timer_delete(&rtpoll_timer);
  mod_timer(&rtpoll_timer, ...);
  rcu_read_unlock();
                                    synchronize_rcu();
                                    kthread_stop(task_to_destroy);

The group can then be freed with the re-armed timer still pending, and poll_timer_fn() runs on freed memory.

461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling mechanism") deleted the timer synchronously after the synchronize_rcu(), which prevented this but raced trigger creation instead: the deletion could cancel the timer that a new trigger set armed during the grace period and, as creation also reinitialized the timer at the time, corrupt it. 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the initialization into group_init() and the deletion into the locked section, trading the creation races for the window above.

Neither placement in the destruction path works. A pending timer firing while the group is alive is harmless though. poll_timer_fn() just wakes the rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it by then. timer_shutdown_sync() because the timer is never armed again.

Quoted 5 lines
Fixes 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy")
Cc [email protected] # v5.10+
Quoted 4 lines

Both these patches look good to me, but Suren can you please also take a look?

Yes, I'm on it. Need some time to remind myself of all the details of the implementation.

maintainer Acked-by
#6 lore raw

Re: [PATCH 1/2] sched/psi: Create the psimon kthread outside of cgroup_mutex

In reply to Matt Fleming
Quoted 39 lines · Matt Fleming
Quoted 31 lines · Tejun Heo

a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() hold cgroup_mutex across psi_trigger_create(), which forks the psimon kthread for the first rtpoll trigger. As kthread creation depends on the whole fork path, the commit inadvertently created a lot of unwanted locking dependencies from cgroup_mutex.

sched_ext got hit by one: its enable path blocks forks and then grabs cgroup_mutex, so a pressure write racing a scheduler enable deadlocks, with every other fork piling up behind.

Fix it by splitting trigger creation so that the worker is forked with cgroup_mutex dropped and the kernfs active reference left broken. The latter matters because rmdir and cgroup.pressure writes drain active references under cgroup_mutex. Publishing the trigger last keeps error reporting synchronous and preserves the of->priv lifetime rules.

The trigger registered in the first stage pins the group's rtpoll machinery across the unlocked window, leaving only creation races to resolve. The catch-up poll on installation covers scheduling attempts dropped while there was no worker.

v2: Retagged sched/psi (was cgroup).

Fixes a5b98009f16d ("sched/psi: fix race between file release and pressure write")
Cc Johannes Weiner <[email protected]>
Cc Edward Adam Davis <[email protected]>
Cc Chen Ridong <[email protected]>
Quoted 5 lines
---
 include/linux/psi.h    |  4 ++-
 kernel/cgroup/cgroup.c | 23 +++++++++++++-
 kernel/sched/psi.c     | 69 ++++++++++++++++++++++++++++++++----------
 3 files changed, 78 insertions(+), 18 deletions(-)
Quoted 1 line
maintainer Acked-by
#10 lore raw

Re: [PATCH 2/2] sched/psi: Shut down rtpoll_timer in psi_cgroup_free()

In reply to Suren Baghdasaryan
Quoted 37 lines · Suren Baghdasaryan
Quoted 35 lines · Johannes Weiner
Quoted 33 lines · Tejun Heo

psi_schedule_rtpoll_work() is called locklessly from the scheduler hotpath and can race psi_trigger_destroy() taking down the last rtpoll trigger under rtpoll_trigger_lock:

psi_schedule_rtpoll_work() psi_trigger_destroy()

  rcu_read_lock();
  task = rcu_dereference(rtpoll_task);
                                    rcu_assign_pointer(rtpoll_task, NULL);
                                    timer_delete(&rtpoll_timer);
  mod_timer(&rtpoll_timer, ...);
  rcu_read_unlock();
                                    synchronize_rcu();
                                    kthread_stop(task_to_destroy);

The group can then be freed with the re-armed timer still pending, and poll_timer_fn() runs on freed memory.

461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling mechanism") deleted the timer synchronously after the synchronize_rcu(), which prevented this but raced trigger creation instead: the deletion could cancel the timer that a new trigger set armed during the grace period and, as creation also reinitialized the timer at the time, corrupt it. 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy") moved the initialization into group_init() and the deletion into the locked section, trading the creation races for the window above.

Neither placement in the destruction path works. A pending timer firing while the group is alive is harmless though. poll_timer_fn() just wakes the rtpoll waitqueue and doesn't re-arm itself. Bind the timer to the group's lifetime instead and shut it down in psi_cgroup_free(). Nothing can arm it by then. timer_shutdown_sync() because the timer is never armed again.

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Fixes 8f91efd870ea ("psi: Fix race between psi_trigger_create/destroy")
Cc [email protected] # v5.10+
Quoted 6 lines
Quoted 3 lines

Both these patches look good to me, but Suren can you please also take a look?

Yes, I'm on it. Need some time to remind myself of all the details of the implementation.

author maintainer
#13 lore raw
In reply to Tejun Heo
Quoted 16 lines · Tejun Heo

Hello,

v2: - Retagged sched/psi (was cgroup) (patch 1).
- Added a fix for a pre-existing rtpoll_timer use-after-free that the Sashiko AI review flagged on the previous posting (patch 2).

v1: https://lore.kernel.org/r/[email protected]

a5b98009f16d ("sched/psi: fix race between file release and pressure write") made pressure_write() fork the psimon kthread while holding cgroup_mutex, which deadlocks against the sched_ext enable path blocking forks before grabbing cgroup_mutex. Patch 1 moves the fork outside cgroup_mutex. Patch 2 fixes the use-after-free from a schedule attempt re-arming group->rtpoll_timer behind psi_trigger_destroy() by shutting down the timer when the group is freed.

Quoted 5 lines

Matt, patch 1 is unchanged from the previous posting except for the subject prefix, so test results against that posting still apply.

Peter, once these are reviewed and tested, how do you want them routed? I can take them through the cgroup tree if that works for you.

Applying this to cgroup/for-7.2-fixes. Please holler for any objections.

Thanks.

Signature · 1 line
tejun