Skip to contentSkip to conversation

cgroup, kthread: do not allow moving kthreads out of the root cgroup

RFC In review Roman Gushchin 3 participants 5 messages Oct 12, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC 1/2 cgroup, kthread: do not allow moving kthreads out of the root cgroup

Attaching kernel threads to a non-root cgroup is generally a bad idea. Kernel threads are generally performing the work required to keep the system working and healthy, and applying various resource limits may affect system stability and performance.

Some examples of dangerous behavior are limiting CPU time available to rcu stuff, memory limits applied to almost all kthreads, etc.

To prevent this dangerous behavior, let's deny all kthread movements between cgroups. Right now only kthreads bounded to CPUs are not allowed to move, which is not sufficient.

If there are examples of kthreads which can be limited, and it's guaranteed to be safe, we can allow explicit exceptions further.

Cc Tejun Heo <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra (Intel) <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Chris Mason <[email protected]>
---
 kernel/cgroup/cgroup.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/cgroup/cgroup.c +4 -4
index c7086c8835da..a3d7a1b5720d 100644
@@ -2629,12 +2629,12 @@ struct task_struct *cgroup_procs_write_start(char *buf, bool threadgroup)
 		tsk = tsk->group_leader;
 
 	/*
-	 * kthreads may acquire PF_NO_SETAFFINITY during initialization.
-	 * If userland migrates such a kthread to a non-root cgroup, it can
+	 * If userland migrates a kthread to a non-root cgroup, it can
 	 * become trapped in a cpuset, or RT kthread may be born in a
-	 * cgroup with no rt_runtime allocated.  Just say no.
+	 * cgroup with no rt_runtime allocated, or it can suffer from
+	 * memory shortage, etc. Just say no.
 	 */
-	if (tsk->no_cgroup_migration || (tsk->flags & PF_NO_SETAFFINITY)) {
+	if (tsk->flags & (PF_KTHREAD | PF_NO_SETAFFINITY)) {
 		tsk = ERR_PTR(-EINVAL);
 		goto out_unlock_threadgroup;
 	}
Signature · 1 line
2.13.6
#1 lore raw

RFC 2/2 cgroup, kthread: cleanup after sticking kthreads to the root cgroup

Since we're not allowing to move kernel threads out of the root cgroup anymore, we don't need some infrastructure built to introduce partial moving ban by
commit 77f88796cee8 ("cgroup, kthread: close race window where new kthreads can be migrated to non-root cgroups").

This patch is just a cleanup, no functional change is performed.

Cc Tejun Heo <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra (Intel) <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Chris Mason <[email protected]>
---
 include/linux/cgroup.h | 21 ---------------------
 include/linux/sched.h  |  4 ----
 kernel/kthread.c       |  2 --
 3 files changed, 27 deletions(-)
include/linux/cgroup.h +0 -21
index 328a70ce0e23..2b2b838beaed 100644
@@ -626,25 +626,6 @@ static inline void pr_cont_cgroup_path(struct cgroup *cgrp)
 	pr_cont_kernfs_path(cgrp->kn);
 }
 
-static inline void cgroup_init_kthreadd(void)
-{
-	/*
-	 * kthreadd is inherited by all kthreads, keep it in the root so
-	 * that the new kthreads are guaranteed to stay in the root until
-	 * initialization is finished.
-	 */
-	current->no_cgroup_migration = 1;
-}
-
-static inline void cgroup_kthread_ready(void)
-{
-	/*
-	 * This kthread finished initialization.  The creator should have
-	 * set PF_NO_SETAFFINITY if this kthread should stay in the root.
-	 */
-	current->no_cgroup_migration = 0;
-}
-
 static inline union kernfs_node_id *cgroup_get_kernfs_id(struct cgroup *cgrp)
 {
 	return &cgrp->kn->id;
@@ -672,8 +653,6 @@ static inline void cgroup_free(struct task_struct *p) {}
 
 static inline int cgroup_init_early(void) { return 0; }
 static inline int cgroup_init(void) { return 0; }
-static inline void cgroup_init_kthreadd(void) {}
-static inline void cgroup_kthread_ready(void) {}
 static inline union kernfs_node_id *cgroup_get_kernfs_id(struct cgroup *cgrp)
 {
 	return NULL;
include/linux/sched.h +0 -4
index bdd6ad6fcce1..7ee7bad521ff 100644
@@ -650,10 +650,6 @@ struct task_struct {
 #ifdef CONFIG_COMPAT_BRK
 	unsigned			brk_randomized:1;
 #endif
-#ifdef CONFIG_CGROUPS
-	/* disallow userland-initiated cgroup migration */
-	unsigned			no_cgroup_migration:1;
-#endif
 
 	unsigned long			atomic_flags; /* Flags requiring atomic access. */
 
kernel/kthread.c +0 -2
index e92070878ce5..0ba173e9b5b6 100644
@@ -237,7 +237,6 @@ static int kthread(void *_create)
 
 	ret = -EINTR;
 	if (!test_bit(KTHREAD_SHOULD_STOP, &self->flags)) {
-		cgroup_kthread_ready();
 		__kthread_parkme(self);
 		ret = threadfn(data);
 	}
@@ -551,7 +550,6 @@ int kthreadd(void *unused)
 	set_mems_allowed(node_states[N_MEMORY]);
 
 	current->flags |= PF_NOFREEZE;
-	cgroup_init_kthreadd();
 
 	for (;;) {
 		set_current_state(TASK_INTERRUPTIBLE);
Signature · 1 line
2.13.6
maintainer
#2 lore raw
In reply to Roman Gushchin
Quoted 8 lines · Roman Gushchin

Attaching kernel threads to a non-root cgroup is generally a bad idea. Kernel threads are generally performing the work required to keep the system working and healthy, and applying various resource limits may affect system stability and performance.

Some examples of dangerous behavior are limiting CPU time available to rcu stuff, memory limits applied to almost all kthreads, etc.

Quoted 7 lines

To prevent this dangerous behavior, let's deny all kthread movements between cgroups. Right now only kthreads bounded to CPUs are not allowed to move, which is not sufficient.

If there are examples of kthreads which can be limited, and it's guaranteed to be safe, we can allow explicit exceptions further.

The traditional use-case is stuffing all the unbound kthreads into a system cpuset in order to limit 'crap' on the rest of the CPUs. This setup is typically found in HPC and RT environments.

So NAK. This needs to stay working in as far as it still works.

#3 lore raw
Quoted 17 lines · Peter Zijlstra
Quoted 8 lines · Roman Gushchin

Attaching kernel threads to a non-root cgroup is generally a bad idea. Kernel threads are generally performing the work required to keep the system working and healthy, and applying various resource limits may affect system stability and performance.

Some examples of dangerous behavior are limiting CPU time available to rcu stuff, memory limits applied to almost all kthreads, etc.

Quoted 7 lines

To prevent this dangerous behavior, let's deny all kthread movements between cgroups. Right now only kthreads bounded to CPUs are not allowed to move, which is not sufficient.

If there are examples of kthreads which can be limited, and it's guaranteed to be safe, we can allow explicit exceptions further.

Quoted 5 lines

The traditional use-case is stuffing all the unbound kthreads into a system cpuset in order to limit 'crap' on the rest of the CPUs. This setup is typically found in HPC and RT environments.

So NAK. This needs to stay working in as far as it still works.

Ok, thanks!

#4 lore raw
In reply to Peter Zijlstra
Quoted 16 lines · Peter Zijlstra
Quoted 8 lines

Attaching kernel threads to a non-root cgroup is generally a bad idea. Kernel threads are generally performing the work required to keep the system working and healthy, and applying various resource limits may affect system stability and performance.

Some examples of dangerous behavior are limiting CPU time available to rcu stuff, memory limits applied to almost all kthreads, etc.

Quoted 7 lines

To prevent this dangerous behavior, let's deny all kthread movements between cgroups. Right now only kthreads bounded to CPUs are not allowed to move, which is not sufficient.

If there are examples of kthreads which can be limited, and it's guaranteed to be safe, we can allow explicit exceptions further.

Quoted 6 lines

The traditional use-case is stuffing all the unbound kthreads into a system cpuset in order to limit 'crap' on the rest of the CPUs. This setup is typically found in HPC and RT environments.

So NAK. This needs to stay working in as far as it still works.

+1, I originally introduced PF_THREAD_BOUND to cpusets in 2008 to prevent migration threads and software watchdog threads from being able to access the cpus they work on. We also move unbound kthreads to a system cpuset.