cgroups: Task counter subsystem
Subsystems (8)
v4 cover cgroups: Task counter subsystem
(Sorry, I had unbalanced "<" and ">" around email addresses so I need to resend otherwise LKML wouldn't take it.)
Reminder:
The task counter is a cgroup subsystem that features an upper bound limitation on the number of tasks running inside cgroup.
It has two desired usecases, but possibly more can be found:
- Protect against forkbomb in the scope of a cgroup, this can be especially useful in the case of containers where traditional rlmit is too limited.
- Kill all tasks inside a cgroup without worrying about races against concurrent forks.
See documentation in patch 10/12 for more details.
May be I should rebase the whole against Tejun's patches that use an iterator on tasks.
Changes in v4:
- Rebase on top of "cgroups: Don't attach task to subsystem if migration failed" (https://lkml.org/lkml/2011/8/26/262), applied in -mm.
- Add comment about the use of spinlock in res_counter_write_u64() (patch 1)
- Update documentation after [can_]attach_task() arguments changes (patch 3)
- Update documentation for cancel_attach_task() new callback (patch 4)
- Cancel task attachment on migration failure in cgroup_attach_proc() (patch 4)
- Some function inlining (patch 5)
- Move "Allow charge failure pointer to be null" earlier in the set for sanity (patch 7)
- Fix error return value (patch 8 and 9)
- Whitespace fixes, comments added, improve config help section (patch 9)
- Drop subsystem early init (patch 9)
- Add subsystem documentation (patch 10)
- Remove ad-hoc fork hook and reuse existing one for cgroup (patch 11 and 12)
This can not be pulled from:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
cgroup/nr_proc-v4
Frederic Weisbecker (12):
cgroups: Add res_counter_write_u64() API
cgroups: New resource counter inheritance API
cgroups: Add previous cgroup in can_attach_task/attach_task callbacks
cgroups: New cancel_attach_task subsystem callback
cgroups: Ability to stop res charge propagation on bounded ancestor
cgroups: Add res counter common ancestor searching
res_counter: Allow charge failure pointer to be null
cgroups: Pull up res counter charge failure interpretation to caller
cgroups: Add a task counter subsystem
cgroups: Add documentation for task counter subsystem
cgroups: Allow subsystems to cancel a fork
cgroups: Convert task counter to use the subsys fork callback
Documentation/cgroups/cgroups.txt | 13 ++- Documentation/cgroups/task_counter.txt | 126 ++++++++++++++++++++ block/blk-cgroup.c | 10 +- include/linux/cgroup.h | 20 ++- include/linux/cgroup_subsys.h | 8 ++ include/linux/res_counter.h | 25 ++++- init/Kconfig | 7 + kernel/Makefile | 1 + kernel/cgroup.c | 56 +++++++-- kernel/cgroup_freezer.c | 9 +- kernel/cgroup_task_counter.c | 200 ++++++++++++++++++++++++++++++++ kernel/cpuset.c | 6 +- kernel/events/core.c | 5 +- kernel/exit.c | 2 +- kernel/fork.c | 7 +- kernel/res_counter.c | 77 ++++++++++--- kernel/sched.c | 6 +- 17 files changed, 524 insertions(+), 54 deletions(-) create mode 100644 Documentation/cgroups/task_counter.txt create mode 100644 kernel/cgroup_task_counter.c
Signature · 1 line
1.7.5.4
PATCH v4 1/12 cgroups: Add res_counter_write_u64() API
Extend the resource counter API with a mirror of res_counter_read_u64() to make it handy to update a resource counter value from a cgroup subsystem u64 value file.
--- include/linux/res_counter.h | 2 ++ kernel/res_counter.c | 25 +++++++++++++++++++------ 2 files changed, 21 insertions(+), 6 deletions(-)
include/linux/res_counter.h +2 -0
@@ -82,6 +82,8 @@ int res_counter_memparse_write_strategy(const char *buf,int res_counter_write (struct res_counter *counter,int member,const char *buffer,write_strategy_fn write_strategy); +void res_counter_write_u64 (struct res_counter *counter,int member,u64 val); +/* * the field descriptors. one for each member of res_counter */
kernel/res_counter.c +19 -6
@@ -168,12 +168,26 @@ int res_counter_memparse_write_strategy(const char *buf,return 0 ; } +void res_counter_write_u64 (struct res_counter *counter,int member,u64 val) +{ +unsigned long long *target; +unsigned long flags; + +/* +* We need the lock to protect against concurrent add/dec on 32 bits. +* No need to ifdef it's seldom used. +*/ +spin_lock_irqsave (&counter->lock, flags); + target =res_counter_member (counter, member); + *target = val; +spin_unlock_irqrestore (&counter->lock, flags); +} +int res_counter_write (struct res_counter *counter,int member,const char *buf,write_strategy_fn write_strategy) {char *end; -unsigned long flags; -unsigned long long tmp, *val; +unsigned long long tmp;if (write_strategy) {if (write_strategy (buf, &tmp))
@@ -183,9 +197,8 @@ int res_counter_write(struct res_counter *counter, int member,if (*end !='\0' )return -EINVAL ; } -spin_lock_irqsave (&counter->lock, flags); - val =res_counter_member (counter, member); - *val = tmp; -spin_unlock_irqrestore (&counter->lock, flags); + +res_counter_write_u64 (counter, member, tmp); +return 0 ; }
Signature · 1 line
1.7.5.4
PATCH v4 2/12 cgroups: New resource counter inheritance API
In reply to Frederic WeisbeckerProvide an API to inherit a counter value from a parent. This can be useful to implement cgroup.clone_children on a resource counter.
Still the resources of the children are limited by those of the parent, so this is only to provide a default setting behaviour when clone_children is set.
--- include/linux/res_counter.h | 2 ++ kernel/res_counter.c | 12 ++++++++++++ 2 files changed, 14 insertions(+), 0 deletions(-)
include/linux/res_counter.h +2 -0
@@ -84,6 +84,8 @@ int res_counter_write(struct res_counter *counter, int member,void res_counter_write_u64 (struct res_counter *counter,int member,u64 val); +void res_counter_inherit (struct res_counter *counter,int member); +/* * the field descriptors. one for each member of res_counter */
kernel/res_counter.c +12 -0
@@ -202,3 +202,15 @@ int res_counter_write(struct res_counter *counter, int member,return 0 ; } + +void res_counter_inherit (struct res_counter *counter,int member) +{ +struct res_counter *parent; +unsigned long long val; + + parent = counter->parent; +if (parent) { + val =res_counter_read_u64 (parent, member); +res_counter_write_u64 (counter, member, val); + } +}
Signature · 1 line
1.7.5.4
Andrew Morton
maintainerRe: [PATCH 02/12] cgroups: New resource counter inheritance API
kernel/res_counter.c int res_counter_write(struct res_counter *counter, int member, · line 216
Quoted 11 lines · On Tue, 6 Sep 2011 02:12:56 +0200 Frederic Weisbecker <[email protected]>…
kernel/res_counter.c +11
-0+void res_counter_inherit (struct res_counter *counter,int member) +{ +struct res_counter *parent; +unsigned long long val; + + parent = counter->parent; +if (parent) { + val =res_counter_read_u64 (parent, member); +res_counter_write_u64 (counter, member, val); + } +}
What locking protects the read-modify-write? If none, please fix, otherwise please document it.
All of kernel/res_counter.c is charmingly undocumented.
Re: [PATCH 02/12] cgroups: New resource counter inheritance API
Quoted 18 lines · Andrew Morton
kernel/res_counter.c int res_counter_write(struct res_counter *counter, int member, · line 216
Quoted 11 lines · On Tue, 6 Sep 2011 02:12:56 +0200 Frederic Weisbecker <[email protected]>…
kernel/res_counter.c +11
-0+void res_counter_inherit (struct res_counter *counter,int member) +{ +struct res_counter *parent; +unsigned long long val; + + parent = counter->parent; +if (parent) { + val =res_counter_read_u64 (parent, member); +res_counter_write_u64 (counter, member, val); + } +}What locking protects the read-modify-write? If none, please fix, otherwise please document it.
Quoted 1 line
All of
kernel/res_counter.cis charmingly undocumented.
How should I address your reviews now that you've applied these patches? Would you prefer me to send new patches on top of -mm?
Also Tejun has patches that conflict with mine. But I believe -mm is on top of -next, so perhaps I should rebase those patches when his branch reach -next? Or something else?
PATCH v4 3/12 cgroups: Add previous cgroup in can_attach_task/attach_task callbacks
In reply to Frederic WeisbeckerThis is to prepare the integration of a new max number of proc cgroup subsystem. We'll need to release some resources from the previous cgroup.
--- Documentation/cgroups/cgroups.txt | 6 ++++-- block/blk-cgroup.c | 10 ++++++---- include/linux/cgroup.h | 5 +++-- kernel/cgroup.c | 10 ++++++---- kernel/cgroup_freezer.c | 3 ++- kernel/cpuset.c | 6 ++++-- kernel/events/core.c | 5 +++-- kernel/sched.c | 6 ++++-- 8 files changed, 32 insertions(+), 19 deletions(-)
Documentation/cgroups/cgroups.txt +4 -2
@@ -605,7 +605,8 @@ called on a fork. If this method returns 0 (success) then this should remain valid while the caller holds cgroup_mutex and it is ensured that either attach() or cancel_attach() will be called in future. -int can_attach_task(struct cgroup *cgrp, struct task_struct *tsk); +int can_attach_task(struct cgroup *cgrp, struct cgroup *old_cgrp, + struct task_struct *tsk); (cgroup_mutex held by caller) As can_attach, but for operations that must be run once per task to be
@@ -635,7 +636,8 @@ void attach(struct cgroup_subsys *ss, struct cgroup *cgrp, Called after the task has been attached to the cgroup, to allow any post-attachment activity that requires memory allocations or blocking. -void attach_task(struct cgroup *cgrp, struct task_struct *tsk); +void attach_task(struct cgroup *cgrp, struct cgroup *old_cgrp, + struct task_struct *tsk); (cgroup_mutex held by caller) As attach, but for operations that must be run once per task to be attached,
block/blk-cgroup.c +6 -4
@@ -30,8 +30,8 @@ EXPORT_SYMBOL_GPL(blkio_root_cgroup);static struct cgroup_subsys_state *blkiocg_create (struct cgroup_subsys *,struct cgroup *); -static int blkiocg_can_attach_task (struct cgroup *,struct task_struct *); -static void blkiocg_attach_task (struct cgroup *,struct task_struct *); +static int blkiocg_can_attach_task (struct cgroup *,struct cgroup *,struct task_struct *); +static void blkiocg_attach_task (struct cgroup *,struct cgroup *,struct task_struct *);static void blkiocg_destroy (struct cgroup_subsys *,struct cgroup *);static int blkiocg_populate (struct cgroup_subsys *,struct cgroup *);
@@ -1614,7 +1614,8 @@ done:* of the main cic data structures. For now we allow a task to change * its cgroup only if it's the only owner of its ioc. */ -static int blkiocg_can_attach_task (struct cgroup *cgrp,struct task_struct *tsk) +static int blkiocg_can_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk) {struct io_context *ioc;int ret =0 ;
@@ -1629,7 +1630,8 @@ static int blkiocg_can_attach_task(struct cgroup *cgrp, struct task_struct *tsk)return ret; } -static void blkiocg_attach_task (struct cgroup *cgrp,struct task_struct *tsk) +static void blkiocg_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk) {struct io_context *ioc;
include/linux/cgroup.h +3 -2
@@ -468,11 +468,12 @@ struct cgroup_subsys {void (*destroy)(struct cgroup_subsys *ss,struct cgroup *cgrp);int (*can_attach)(struct cgroup_subsys *ss,struct cgroup *cgrp,struct task_struct *tsk); -int (*can_attach_task)(struct cgroup *cgrp,struct task_struct *tsk); +int (*can_attach_task)(struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk);void (*cancel_attach)(struct cgroup_subsys *ss,struct cgroup *cgrp,struct task_struct *tsk);void (*pre_attach)(struct cgroup *cgrp); -void (*attach_task)(struct cgroup *cgrp,struct task_struct *tsk); +void (*attach_task)(struct cgroup *cgrp,struct cgroup *old_cgrp,struct task_struct *tsk);void (*attach)(struct cgroup_subsys *ss,struct cgroup *cgrp,struct cgroup *old_cgrp,struct task_struct *tsk);void (*fork)(struct cgroup_subsys *ss,struct task_struct *task);
kernel/cgroup.c +6 -4
@@ -1844,7 +1844,7 @@ int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk) } }if (ss->can_attach_task) { - retval = ss->can_attach_task (cgrp, tsk); + retval = ss->can_attach_task (cgrp, oldcgrp, tsk);if (retval) { failed_ss = ss; goto out;
@@ -1860,7 +1860,7 @@ int cgroup_attach_task(struct cgroup *cgrp, struct task_struct *tsk)if (ss->pre_attach) ss->pre_attach (cgrp);if (ss->attach_task) - ss->attach_task (cgrp, tsk); + ss->attach_task (cgrp, oldcgrp, tsk);if (ss->attach) ss->attach (ss, cgrp, oldcgrp, tsk); }
@@ -2075,7 +2075,9 @@ int cgroup_attach_proc(struct cgroup *cgrp, struct task_struct *leader)/* run on each task in the threadgroup. */ for (i =0 ; i < group_size; i++) { tsk =flex_array_get_ptr (group, i); - retval = ss->can_attach_task (cgrp, tsk); + oldcgrp =task_cgroup_from_root (tsk, root); + + retval = ss->can_attach_task (cgrp, oldcgrp, tsk);if (retval) { failed_ss = ss; cancel_failed_ss = true;
@@ -2141,7 +2143,7 @@ int cgroup_attach_proc(struct cgroup *cgrp, struct task_struct *leader)/* attach each task to each subsystem */ for_each_subsys (root, ss) {if (ss->attach_task) - ss->attach_task (cgrp, tsk); + ss->attach_task (cgrp, oldcgrp, tsk); } }else {BUG_ON (retval != -ESRCH );
kernel/cgroup_freezer.c +2 -1
@@ -175,7 +175,8 @@ static int freezer_can_attach(struct cgroup_subsys *ss,return 0 ; } -static int freezer_can_attach_task (struct cgroup *cgrp,struct task_struct *tsk) +static int freezer_can_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk) {rcu_read_lock ();if (__cgroup_freezing_or_frozen (tsk)) {
kernel/cpuset.c +4 -2
@@ -1390,7 +1390,8 @@ static int cpuset_can_attach(struct cgroup_subsys *ss, struct cgroup *cont,return 0 ; } -static int cpuset_can_attach_task (struct cgroup *cgrp,struct task_struct *task) +static int cpuset_can_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *task) {return security_task_setscheduler (task); }
@@ -1418,7 +1419,8 @@ static void cpuset_pre_attach(struct cgroup *cont) }/* Per-thread attachment work. */ -static void cpuset_attach_task (struct cgroup *cont,struct task_struct *tsk) +static void cpuset_attach_task (struct cgroup *cont,struct cgroup *old, +struct task_struct *tsk) {int err;struct cpuset *cs =cgroup_cs (cont);
kernel/events/core.c +3 -2
@@ -7001,7 +7001,8 @@ static int __perf_cgroup_move(void *info) }static void -perf_cgroup_attach_task (struct cgroup *cgrp,struct task_struct *task) +perf_cgroup_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *task) {task_function_call (task, __perf_cgroup_move, task); }
@@ -7017,7 +7018,7 @@ static void perf_cgroup_exit(struct cgroup_subsys *ss, struct cgroup *cgrp,if (!(task->flags &PF_EXITING ))return ; -perf_cgroup_attach_task (cgrp, task); +perf_cgroup_attach_task (cgrp, old_cgrp, task); }struct cgroup_subsys perf_subsys = {
kernel/sched.c +4 -2
@@ -8967,7 +8967,8 @@ cpu_cgroup_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp) }static int -cpu_cgroup_can_attach_task (struct cgroup *cgrp,struct task_struct *tsk) +cpu_cgroup_can_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk) {#ifdef CONFIG_RT_GROUP_SCHED if (!sched_rt_can_attach (cgroup_tg (cgrp), tsk))
@@ -8981,7 +8982,8 @@ cpu_cgroup_can_attach_task(struct cgroup *cgrp, struct task_struct *tsk) }static void -cpu_cgroup_attach_task (struct cgroup *cgrp,struct task_struct *tsk) +cpu_cgroup_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk) {sched_move_task (tsk); }
Signature · 1 line
1.7.5.4
PATCH v4 4/12 cgroups: New cancel_attach_task subsystem callback
In reply to Frederic WeisbeckerTo cancel a process attachment on a subsystem, we only call the cancel_attach() callback once on the leader but we have no way to cancel the attachment individually for each member of the process group.
This is going to be needed for the max number of tasks susbystem that is coming.
To prepare for this integration, call a new cancel_attach_task() callback on each task of the group until we reach the member that failed to attach.
--- Documentation/cgroups/cgroups.txt | 7 +++++++ include/linux/cgroup.h | 1 + kernel/cgroup.c | 23 +++++++++++++++++++---- 3 files changed, 27 insertions(+), 4 deletions(-)
Documentation/cgroups/cgroups.txt +7 -0
@@ -623,6 +623,13 @@ function, so that the subsystem can implement a rollback. If not, not necessary. This will be called only about subsystems whose can_attach() operation have succeeded. +void cancel_attach_task(struct cgroup *cgrp, struct task_struct *tsk) +(cgroup_mutex held by caller) + +As cancel_attach, but for operations that must be cancelled once per +task that wanted to be attached. This typically revert the effect of +can_attach_task(). + void pre_attach(struct cgroup *cgrp); (cgroup_mutex held by caller)
include/linux/cgroup.h +1 -0
@@ -472,6 +472,7 @@ struct cgroup_subsys { struct task_struct *tsk);void (*cancel_attach)(struct cgroup_subsys *ss,struct cgroup *cgrp,struct task_struct *tsk); +void (*cancel_attach_task)(struct cgroup *cgrp,struct task_struct *tsk);void (*pre_attach)(struct cgroup *cgrp);void (*attach_task)(struct cgroup *cgrp,struct cgroup *old_cgrp,struct task_struct *tsk); void (*attach)(struct cgroup_subsys *ss, struct cgroup *cgrp,
kernel/cgroup.c +19 -4
@@ -1883,6 +1883,9 @@ out:* remaining subsystems. */ break ; + +if (ss->cancel_attach_task) + ss->cancel_attach_task (cgrp, tsk);if (ss->cancel_attach) ss->cancel_attach (ss, cgrp, tsk); }
@@ -1992,7 +1995,7 @@ int cgroup_attach_proc(struct cgroup *cgrp, struct task_struct *leader) {int retval, i, group_size;struct cgroup_subsys *ss, *failed_ss =NULL ; -bool cancel_failed_ss = false; +struct task_struct *failed_task =NULL ;/* guaranteed to be initialized later, but the compiler needs this */ struct cgroup *oldcgrp =NULL ;struct css_set *oldcg;
@@ -2080,7 +2083,7 @@ int cgroup_attach_proc(struct cgroup *cgrp, struct task_struct *leader) retval = ss->can_attach_task (cgrp, oldcgrp, tsk);if (retval) { failed_ss = ss; - cancel_failed_ss = true; + failed_task = tsk; goto out_cancel_attach; } }
@@ -2145,8 +2148,11 @@ int cgroup_attach_proc(struct cgroup *cgrp, struct task_struct *leader)if (ss->attach_task) ss->attach_task (cgrp, oldcgrp, tsk); } + }else if (retval == -ESRCH ) { +if (ss->cancel_attach_task) + ss->cancel_attach_task (cgrp, tsk); }else { -BUG_ON (retval != -ESRCH ); +BUG_ON (1 ); } }/* nothing is sensitive to fork() after this point. */
@@ -2178,8 +2184,17 @@ out_cancel_attach:/* same deal as in cgroup_attach_task */ if (retval) {for_each_subsys (root, ss) { +if (ss->cancel_attach_task && (ss != failed_ss || failed_task)) { +for (i =0 ; i < group_size; i++) { + tsk =flex_array_get_ptr (group, i); +if (tsk == failed_task) +break ; + ss->cancel_attach_task (cgrp, tsk); + } + } +if (ss == failed_ss) { -if (cancel_failed_ss && ss->cancel_attach) +if (failed_task && ss->cancel_attach) ss->cancel_attach (ss, cgrp, leader);break ; }
Signature · 1 line
1.7.5.4
PATCH v4 5/12 cgroups: Ability to stop res charge propagation on bounded ancestor
In reply to Frederic WeisbeckerMoving a task from a cgroup to another may require to substract its resource charge from the old cgroup and add it to the new one.
For this to happen, the uncharge/charge propagation can just stop when we reach the common ancestor for the two cgroups. Further the performance reasons, we also want to avoid to temporarily overload the common ancestors with a non-accurate resource counter usage if we charge first the new cgroup and uncharge the old one thereafter. This is going to be a requirement for the coming max number of task subsystem.
To solve this, provide a pair of new API that can charge/uncharge a resource counter until we reach a given ancestor.
--- include/linux/res_counter.h | 19 ++++++++++++++++--- kernel/res_counter.c | 13 ++++++++----- 2 files changed, 24 insertions(+), 8 deletions(-)
include/linux/res_counter.h +16 -3
@@ -117,8 +117,16 @@ void res_counter_init(struct res_counter *counter, struct res_counter *parent);int __must_check res_counter_charge_locked (struct res_counter *counter,unsigned long val); -int __must_check res_counter_charge (struct res_counter *counter, -unsigned long val,struct res_counter **limit_fail_at); +int __must_check res_counter_charge_until (struct res_counter *counter, +struct res_counter *limit, +unsigned long val, +struct res_counter **limit_fail_at); +static inline int __must_check +res_counter_charge (struct res_counter *counter,unsigned long val, +struct res_counter **limit_fail_at) +{ +return res_counter_charge_until (counter,NULL , val, limit_fail_at); +}/* * uncharge - tell that some portion of the resource is released
@@ -131,7 +139,12 @@ int __must_check res_counter_charge(struct res_counter *counter,*/ void res_counter_uncharge_locked (struct res_counter *counter,unsigned long val); -void res_counter_uncharge (struct res_counter *counter,unsigned long val); +void res_counter_uncharge_until (struct res_counter *counter,struct res_counter *limit, +unsigned long val); +static inline void res_counter_uncharge (struct res_counter *counter,unsigned long val) +{ +res_counter_uncharge_until (counter,NULL , val); +}/** * res_counter_margin - calculate chargeable space of a counter
kernel/res_counter.c +8 -5
@@ -35,8 +35,9 @@ int res_counter_charge_locked(struct res_counter *counter, unsigned long val)return 0 ; } -int res_counter_charge (struct res_counter *counter,unsigned long val, -struct res_counter **limit_fail_at) +int res_counter_charge_until (struct res_counter *counter, +struct res_counter *limit,unsigned long val, +struct res_counter **limit_fail_at) {int ret;unsigned long flags;
@@ -44,7 +45,7 @@ int res_counter_charge(struct res_counter *counter, unsigned long val, *limit_fail_at =NULL ;local_irq_save (flags); -for (c = counter; c !=NULL ; c = c->parent) { +for (c = counter; c != limit; c = c->parent) {spin_lock (&c->lock); ret =res_counter_charge_locked (c, val);spin_unlock (&c->lock);
@@ -74,13 +75,15 @@ void res_counter_uncharge_locked(struct res_counter *counter, unsigned long val) counter->usage -= val; } -void res_counter_uncharge (struct res_counter *counter,unsigned long val) +void res_counter_uncharge_until (struct res_counter *counter, +struct res_counter *limit, +unsigned long val) {unsigned long flags;struct res_counter *c;local_irq_save (flags); -for (c = counter; c !=NULL ; c = c->parent) { +for (c = counter; c != limit; c = c->parent) {spin_lock (&c->lock);res_counter_uncharge_locked (c, val);spin_unlock (&c->lock);
Signature · 1 line
1.7.5.4
PATCH v4 6/12 cgroups: Add res counter common ancestor searching
In reply to Frederic WeisbeckerAdd a new API to find the common ancestor between two resource counters. This includes the passed resource counter themselves.
--- include/linux/res_counter.h | 2 ++ kernel/res_counter.c | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+), 0 deletions(-)
include/linux/res_counter.h +2 -0
@@ -146,6 +146,8 @@ static inline void res_counter_uncharge(struct res_counter *counter, unsigned lores_counter_uncharge_until (counter,NULL , val); } +struct res_counter *res_counter_common_ancestor (struct res_counter *l,struct res_counter *r); +/** * res_counter_margin - calculate chargeable space of a counter * @cnt: the counter
kernel/res_counter.c +19 -0
@@ -92,6 +92,25 @@ void res_counter_uncharge_until(struct res_counter *counter, } +struct res_counter * +res_counter_common_ancestor (struct res_counter *r1,struct res_counter *r2) +{ +struct res_counter *iter; + +while (r1) { + iter = r2; +while (iter) { +if (iter == r1) +return iter; + iter = iter->parent; + } + + r1 = r1->parent; + } + +return NULL ; +} +static inline unsigned long long *res_counter_member (struct res_counter *counter,int member) {
Signature · 1 line
1.7.5.4
Andrew Morton
maintainerRe: [PATCH 06/12] cgroups: Add res counter common ancestor searching
Quoted 8 lines · On Tue, 6 Sep 2011 02:13:00 +0200 Frederic Weisbecker <[email protected]>…
kernel/res_counter.c +8
-0+struct res_counter * +res_counter_common_ancestor (struct res_counter *r1,struct res_counter *r2) +{ +struct res_counter *iter; + +while (r1) { + iter = r2; +while (iter) {
kernel/res_counter.c void res_counter_uncharge_until(struct res_counter *counter, · line 112
Quoted 10 lines
kernel/res_counter.c +10
-0+if (iter == r1) +return iter; + iter = iter->parent; + } + + r1 = r1->parent; + } + +return NULL ; +}
cgroup_mutex, one assumes?
Re: [PATCH 06/12] cgroups: Add res counter common ancestor searching
Quoted 15 lines · Andrew Morton
Quoted 12 lines · On Tue, 6 Sep 2011 02:13:00 +0200 Frederic Weisbecker <[email protected]>…
kernel/res_counter.c +12
-0+struct res_counter * +res_counter_common_ancestor (struct res_counter *r1,struct res_counter *r2) +{ +struct res_counter *iter; + +while (r1) { + iter = r2; +while (iter) { +if (iter == r1) +return iter; + iter = iter->parent; + }
Quoted 8 lines
kernel/res_counter.c void res_counter_uncharge_until(struct res_counter *counter, · line 112
Quoted 6 lines
kernel/res_counter.c +6
-0+ + r1 = r1->parent; + } + +return NULL ; +}
cgroup_mutex, one assumes?
No, a cgroup can't move to other parents, they remain stable as long as they have children (if I understood correctly). So this mirrors to res_counters.
In fact res counters is just a library, pretty much like lists, except they are only used by cgroups for now, so it's up to the caller to guarantee the stability of its objects.
But yeah the documentation for this function is missing. I'm adding some comments in the file.
PATCH v4 7/12 res_counter: Allow charge failure pointer to be null
In reply to Frederic WeisbeckerSo that callers of res_counter_charge() don't have to create and pass this pointer even if they aren't interested in it.
--- kernel/res_counter.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)
kernel/res_counter.c +4 -2
@@ -43,14 +43,16 @@ int res_counter_charge_until(struct res_counter *counter,unsigned long flags;struct res_counter *c, *u; - *limit_fail_at =NULL ; +if (limit_fail_at) + *limit_fail_at =NULL ;local_irq_save (flags);for (c = counter; c != limit; c = c->parent) {spin_lock (&c->lock); ret =res_counter_charge_locked (c, val);spin_unlock (&c->lock);if (ret <0 ) { - *limit_fail_at = c; +if (limit_fail_at) + *limit_fail_at = c; goto undo; } }
Signature · 1 line
1.7.5.4
PATCH v4 8/12 cgroups: Pull up res counter charge failure interpretation to caller
In reply to Frederic Weisbeckerres_counter_charge() always returns -ENOMEM when the limit is reached and the charge thus can't happen.
However it's up to the caller to interpret this failure and return the appropriate error value. The task counter subsystem will need to report the user that a fork() has been cancelled because of some limit reached, not because we are too short on memory.
Fix this by returning -1 when res_counter_charge() fails.
--- kernel/res_counter.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/res_counter.c +1 -1
@@ -26,7 +26,7 @@ int res_counter_charge_locked(struct res_counter *counter, unsigned long val) {if (counter->usage + val > counter->limit) { counter->failcnt++; -return -ENOMEM ; +return -1 ; } counter->usage += val;
Signature · 1 line
1.7.5.4
Andrew Morton
maintainerRe: [PATCH 08/12] cgroups: Pull up res counter charge failure interpretation to caller
Quoted 29 lines · On Tue, 6 Sep 2011 02:13:02 +0200 Frederic Weisbecker <[email protected]>…
res_counter_charge()always returns-ENOMEMwhen the limit is reached and the charge thus can't happen.However it's up to the caller to interpret this failure and return the appropriate error value. The task counter subsystem will need to report the user that a
fork()has been cancelled because of some limit reached, not because we are too short on memory.Fix this by returning -1 when
res_counter_charge()fails.Signed-off-by Frederic Weisbecker <[email protected]>Cc Paul Menage <[email protected]>Cc Li Zefan <[email protected]>Cc Johannes Weiner <[email protected]>Cc Aditya Kali <[email protected]>Cc Oleg Nesterov <[email protected]>Cc Andrew Morton <[email protected]>Cc Kay Sievers <[email protected]>Cc Tim Hockin <[email protected]>Cc Tejun Heo <[email protected]>--- kernel/res_counter.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)kernel/res_counter.c +0
-0
kernel/res_counter.c int res_counter_charge_locked(struct res_counter *counter, unsigned long val) · line 32
Quoted 8 lines
kernel/res_counter.c +1
-1@@ -26,7 +26,7 @@ int res_counter_charge_locked(struct res_counter *counter, unsigned long val) {if (counter->usage + val > counter->limit) { counter->failcnt++; -return -ENOMEM ; +return -1 ; } counter->usage += val;
This also affects the return value of your new and undocumented res_counter_charge_until().
That's a bit of a hand-grenade which could lead to system calls returning -1 (ie: EPERM) to userspace.
Re: [PATCH 08/12] cgroups: Pull up res counter charge failure interpretation to caller
Quoted 39 lines · Andrew Morton
Quoted 36 lines · On Tue, 6 Sep 2011 02:13:02 +0200 Frederic Weisbecker <[email protected]>…
res_counter_charge()always returns-ENOMEMwhen the limit is reached and the charge thus can't happen.However it's up to the caller to interpret this failure and return the appropriate error value. The task counter subsystem will need to report the user that a
fork()has been cancelled because of some limit reached, not because we are too short on memory.Fix this by returning -1 when
res_counter_charge()fails.Signed-off-by Frederic Weisbecker <[email protected]>Cc Paul Menage <[email protected]>Cc Li Zefan <[email protected]>Cc Johannes Weiner <[email protected]>Cc Aditya Kali <[email protected]>Cc Oleg Nesterov <[email protected]>Cc Andrew Morton <[email protected]>Cc Kay Sievers <[email protected]>Cc Tim Hockin <[email protected]>Cc Tejun Heo <[email protected]>--- kernel/res_counter.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)kernel/res_counter.c +1
-1@@ -26,7 +26,7 @@ int res_counter_charge_locked(struct res_counter *counter, unsigned long val) {if (counter->usage + val > counter->limit) { counter->failcnt++; -return -ENOMEM ; +return -1 ; }
Quoted 7 lines
kernel/res_counter.c int res_counter_charge_locked(struct res_counter *counter, unsigned long val) · line 32
Quoted 1 line
kernel/res_counter.c +0
-0counter->usage += val;This also affects the return value of your new and undocumented
res_counter_charge_until().That's a bit of a hand-grenade which could lead to system calls returning -1 (ie: EPERM) to userspace.
Right. What about making it a boolean?
Andrew Morton
maintainerRe: [PATCH 08/12] cgroups: Pull up res counter charge failure interpretation to caller
Quoted 45 lines · On Fri, 9 Sep 2011 15:33:20 +0200 Frederic Weisbecker <[email protected]>…
Quoted 44 lines · Andrew Morton
Quoted 29 lines · On Tue, 6 Sep 2011 02:13:02 +0200 Frederic Weisbecker <[email protected]>…
res_counter_charge()always returns-ENOMEMwhen the limit is reached and the charge thus can't happen.However it's up to the caller to interpret this failure and return the appropriate error value. The task counter subsystem will need to report the user that a
fork()has been cancelled because of some limit reached, not because we are too short on memory.Fix this by returning -1 when
res_counter_charge()fails.Signed-off-by Frederic Weisbecker <[email protected]>Cc Paul Menage <[email protected]>Cc Li Zefan <[email protected]>Cc Johannes Weiner <[email protected]>Cc Aditya Kali <[email protected]>Cc Oleg Nesterov <[email protected]>Cc Andrew Morton <[email protected]>Cc Kay Sievers <[email protected]>Cc Tim Hockin <[email protected]>Cc Tejun Heo <[email protected]>--- kernel/res_counter.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)kernel/res_counter.c +0
-0kernel/res_counter.c int res_counter_charge_locked(struct res_counter *counter, unsigned long val) · line 32
Quoted 8 lines
kernel/res_counter.c +1
-1@@ -26,7 +26,7 @@ int res_counter_charge_locked(struct res_counter *counter, unsigned long val) {if (counter->usage + val > counter->limit) { counter->failcnt++; -return -ENOMEM ; +return -1 ; } counter->usage += val;This also affects the return value of your new and undocumented
res_counter_charge_until().
Quoted 4 lines
Quoted 2 lines
That's a bit of a hand-grenade which could lead to system calls returning -1 (ie: EPERM) to userspace.
Right. What about making it a boolean?
mmm, not sure. 0/-1 is a reasonable return value for a function which either did or didn't succeed. Adding appropriate interface documentation is a way of reducing the opportunity for making this mistake.
PATCH v4 9/12 cgroups: Add a task counter subsystem
In reply to Frederic WeisbeckerAdd a new subsystem to limit the number of running tasks, similar to the NR_PROC rlimit but in the scope of a cgroup.
This is a step to be able to isolate a bit more a cgroup against the rest of the system and limit the global impact of a fork bomb inside a given cgroup.
--- include/linux/cgroup.h | 9 ++ include/linux/cgroup_subsys.h | 8 ++ init/Kconfig | 7 ++ kernel/Makefile | 1 + kernel/cgroup_task_counter.c | 199 +++++++++++++++++++++++++++++++++++++++++ kernel/fork.c | 4 + 6 files changed, 228 insertions(+), 0 deletions(-) create mode 100644 kernel/cgroup_task_counter.c
include/linux/cgroup.h +9 -0
@@ -659,4 +659,13 @@ static inline int cgroup_attach_task_current_cg(struct task_struct *t)#endif /* !CONFIG_CGROUPS */ +#ifdef CONFIG_CGROUP_TASK_COUNTER +int cgroup_task_counter_fork (struct task_struct *child); +#else +static inline int cgroup_task_counter_fork (struct task_struct *child) +{ +return 0 ; +} +#endif /* CONFIG_CGROUP_TASK_COUNTER */ +#endif /* _LINUX_CGROUP_H */
include/linux/cgroup_subsys.h +8 -0
@@ -59,8 +59,16 @@ SUBSYS(net_cls) SUBSYS(blkio)#endif +/* */ +#ifdef CONFIG_CGROUP_PERF SUBSYS(perf)#endif /* */ + +#ifdef CONFIG_CGROUP_TASK_COUNTER +SUBSYS(tasks) +#endif + +/* */
init/Kconfig +7 -0
@@ -690,6 +690,13 @@ config CGROUP_MEM_RES_CTLR_SWAP_ENABLED select this option (if, for some reason, they need to disable it then swapaccount=0 does the trick). +config CGROUP_TASK_COUNTER + bool "Control number of tasks in a cgroup" + depends on RESOURCE_COUNTERS + help + Let the user set up an upper bound allowed number of tasks running + in a cgroup. + config CGROUP_PERF bool "Enable perf_event per-cpu per-container group (cgroup) monitoring" depends on PERF_EVENTS && CGROUPS
kernel/Makefile +1 -0
@@ -60,6 +60,7 @@ obj-$(CONFIG_BACKTRACE_SELF_TEST) += backtracetest.o obj-$(CONFIG_COMPAT) += compat.o obj-$(CONFIG_CGROUPS) += cgroup.o obj-$(CONFIG_CGROUP_FREEZER) += cgroup_freezer.o +obj-$(CONFIG_CGROUP_TASK_COUNTER) += cgroup_task_counter.o obj-$(CONFIG_CPUSETS) += cpuset.o obj-$(CONFIG_UTS_NS) += utsname.o obj-$(CONFIG_USER_NS) += user_namespace.o
kernel/cgroup_task_counter.c new file +199 -0
199 lines not on this page · raw message
kernel/fork.c +4 -0
@@ -1309,6 +1309,10 @@ static struct task_struct *copy_process(unsigned long clone_flags, p->group_leader = p;INIT_LIST_HEAD (&p->thread_group); + retval =cgroup_task_counter_fork (p); +if (retval) + goto bad_fork_free_pid; +/* Now that the task is set up, run cgroup callbacks if * necessary. We need to run them before the task is visible * on the tasklist. */
Signature · 1 line
1.7.5.4
Andrew Morton
maintainerRe: [PATCH 09/12] cgroups: Add a task counter subsystem
Quoted 6 lines · On Tue, 6 Sep 2011 02:13:03 +0200 Frederic Weisbecker <[email protected]>…
Add a new subsystem to limit the number of running tasks, similar to the
NR_PROCrlimit but in the scope of a cgroup.This is a step to be able to isolate a bit more a cgroup against the rest of the system and limit the global impact of a fork bomb inside a given cgroup.
It would be nice to show some testing results for the putative forkbomb-control feature.
init/Kconfig config CGROUP_MEM_RES_CTLR_SWAP_ENABLED · line 698
Quoted 9 lines
...
init/Kconfig +6
-0+config CGROUP_TASK_COUNTER + bool "Control number of tasks in a cgroup" + depends on RESOURCE_COUNTERS + help + Let the user set up an upper bound allowed number of tasks running + in a cgroup.
"of the allowed"?
Perhaps this help section could be fleshed out somewhat.
kernel/cgroup_task_counter.c line 7
Quoted 11 lines
...
kernel/cgroup_task_counter.c new file +7
-0@@ -0,0 +1,199 @@ +/* +* Limits on number of tasks subsystem for cgroups +* +* Copyright (C) 2011 Red Hat, Inc., Frederic Weisbecker <[email protected]> +* +* Thanks to Andrew Morton, Johannes Weiner, Li Zefan, Oleg Nesterov and Paul Menage +* for their suggestions.
80 cols, please. (checkpatch!)
Quoted 7 lines
kernel/cgroup_task_counter.c new file +7
-0+* +*/ + +#include <linux/cgroup.h> +#include <linux/slab.h> +#include <linux/res_counter.h> +
kernel/cgroup_task_counter.c line 24
Quoted 10 lines
kernel/cgroup_task_counter.c new file +10
-0+ +struct task_counter { +struct res_counter res; +struct cgroup_subsys_state css; +}; + +/* +* The root task counter doesn't exist as it's not part of the +* whole task counting in order to optimize the trivial case +* of only one root cgroup living.
That sentence is rather hard to follow.
Quoted 4 lines
kernel/cgroup_task_counter.c new file +4
-0+*/ +static struct cgroup_subsys_state root_css; + +
kernel/cgroup_task_counter.c line 38
Quoted 10 lines
kernel/cgroup_task_counter.c new file +10
-0+static inline struct task_counter *cgroup_task_counter (struct cgroup *cgrp) +{ +if (!cgrp->parent) +return NULL ; + +return container_of (cgroup_subsys_state (cgrp, tasks_subsys_id), + struct task_counter, css); +} + +static inline struct res_counter *cgroup_task_counter_res (struct cgroup *cgrp)
"cgroup_res_counter" would be a more symmetrical name. Or perhaps cgroup_task_res_counter. Swapping the "counter" and "res" seems odd.
Quoted 4 lines
kernel/cgroup_task_counter.c new file +4
-0+{ +struct task_counter *cnt; + + cnt =cgroup_task_counter (cgrp);
kernel/cgroup_task_counter.c line 90
Quoted 10 lines
kernel/cgroup_task_counter.c new file +7
-0+if (!cnt) +return NULL ; + +return &cnt->res; +} + +/* Protected amongst can_attach_task/attach_task/cancel_attach_task by cgroup mutex */
/*
* Protected amongst can_attach_task/attach_task/cancel_attach_task by cgroup
* mutex
*/
(checkpatch)
Quoted 20 lines
...
kernel/cgroup_task_counter.c new file +17
-0+static int task_counter_can_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk) +{ +struct res_counter *res =cgroup_task_counter_res (cgrp); +struct res_counter *old_res =cgroup_task_counter_res (old_cgrp); +int err; + +/* +* When moving a task from a cgroup to another, we don't want +* to charge the common ancestors, even though they will be +* uncharged later from attach_task(), because during that +* short window between charge and uncharge, a task could fork +* in the ancestor and spuriously fail due to the temporary +* charge. +*/ + common_ancestor =res_counter_common_ancestor (res, old_res); +
kernel/cgroup_task_counter.c line 119
Quoted 10 lines
kernel/cgroup_task_counter.c new file +10
-0+/* +* If cgrp is the root then res is NULL, however in this case +* the common ancestor is NULL as well, making the below a NOP. +*/ + err =res_counter_charge_until (res, common_ancestor,1 ,NULL ); +if (err) +return -EINVAL ; + +return 0 ; +}
One would expect a "can"-named function to return a boolean. This one returns an errno which is OK, I guess. But the function is misnamed because if successful it actually alters charges. A less misleading name would be simply task_counter_attach_task(), but that's already taken. Or perhaps task_counter_try_attach_task(), but that seems unnecessary to me - many many kernel functions "try" something and back out with an errno if it failed.
I really do dislike the fact that the documentation is over in another file and another patch. For a start, it makes review harder and slower.
Quoted 3 lines
...
Re: [PATCH 09/12] cgroups: Add a task counter subsystem
Quoted 6 lines · Andrew Morton
Quoted 3 lines · On Tue, 6 Sep 2011 02:13:03 +0200 Frederic Weisbecker <[email protected]>…
Add a new subsystem to limit the number of running tasks, similar to the
NR_PROCrlimit but in the scope of a cgroup.
Quoted 6 lines
Quoted 3 lines
This is a step to be able to isolate a bit more a cgroup against the rest of the system and limit the global impact of a fork bomb inside a given cgroup.
It would be nice to show some testing results for the putative forkbomb-control feature.
I'm uploading a selftest tool that I've been using to ensure it behaves as expected. A forkbomb test is included.
Quoted 12 lines
init/Kconfig config CGROUP_MEM_RES_CTLR_SWAP_ENABLED · line 698
Quoted 9 lines
...
init/Kconfig +6
-0+config CGROUP_TASK_COUNTER + bool "Control number of tasks in a cgroup" + depends on RESOURCE_COUNTERS + help + Let the user set up an upper bound allowed number of tasks running + in a cgroup."of the allowed"?
Quoted 1 line
Perhaps this help section could be fleshed out somewhat.
I've fixed and expanded it a bit for the v5.
Quoted 4 lines
Quoted 4 lines
...
Quoted 10 lines
kernel/cgroup_task_counter.c line 7
Quoted 8 lines
kernel/cgroup_task_counter.c new file +7
-0@@ -0,0 +1,199 @@ +/* +* Limits on number of tasks subsystem for cgroups +* +* Copyright (C) 2011 Red Hat, Inc., Frederic Weisbecker <[email protected]> +* +* Thanks to Andrew Morton, Johannes Weiner, Li Zefan, Oleg Nesterov and Paul Menage +* for their suggestions. 80 cols, please. (checkpatch!)
Fixed in v5.
Quoted 12 lines
Quoted 12 lines
kernel/cgroup_task_counter.c new file +11
-0+* +*/ + +#include <linux/cgroup.h> +#include <linux/slab.h> +#include <linux/res_counter.h> + + +struct task_counter { +struct res_counter res; +struct cgroup_subsys_state css;
Quoted 8 lines
kernel/cgroup_task_counter.c line 24
Quoted 6 lines
kernel/cgroup_task_counter.c new file +6
-0+}; + +/* +* The root task counter doesn't exist as it's not part of the +* whole task counting in order to optimize the trivial case +* of only one root cgroup living. That sentence is rather hard to follow.
I've fixed it too. I mean I tried something...
Quoted 9 lines
Quoted 9 lines
kernel/cgroup_task_counter.c new file +9
-0+*/ +static struct cgroup_subsys_state root_css; + + +static inline struct task_counter *cgroup_task_counter (struct cgroup *cgrp) +{ +if (!cgrp->parent) +return NULL ; +
Quoted 8 lines
kernel/cgroup_task_counter.c line 38
Quoted 5 lines
kernel/cgroup_task_counter.c new file +5
-0+return container_of (cgroup_subsys_state (cgrp, tasks_subsys_id), + struct task_counter, css); +} + +static inline struct res_counter *cgroup_task_counter_res (struct cgroup *cgrp)"
cgroup_res_counter" would be a more symmetrical name. Or perhapscgroup_task_res_counter. Swapping the "counter" and "res" seems odd.
Indeed; fixed.
Quoted 13 lines
Quoted 13 lines
kernel/cgroup_task_counter.c new file +10
-0+{ +struct task_counter *cnt; + + cnt =cgroup_task_counter (cgrp); +if (!cnt) +return NULL ; + +return &cnt->res; +} +
Quoted 8 lines
Quoted 1 line
+/* Protected amongst can_attach_task/attach_task/cancel_attach_task by cgroup mutex */
/*
* Protected amongst can_attach_task/attach_task/cancel_attach_task by cgroup
* mutex
*/(checkpatch)
Fixed
Quoted 31 lines
Quoted 20 lines
...
kernel/cgroup_task_counter.c new file +17
-0+static int task_counter_can_attach_task (struct cgroup *cgrp,struct cgroup *old_cgrp, +struct task_struct *tsk) +{ +struct res_counter *res =cgroup_task_counter_res (cgrp); +struct res_counter *old_res =cgroup_task_counter_res (old_cgrp); +int err; + +/* +* When moving a task from a cgroup to another, we don't want +* to charge the common ancestors, even though they will be +* uncharged later from attach_task(), because during that +* short window between charge and uncharge, a task could fork +* in the ancestor and spuriously fail due to the temporary +* charge. +*/ + common_ancestor =res_counter_common_ancestor (res, old_res); +kernel/cgroup_task_counter.c line 119
Quoted 10 lines
kernel/cgroup_task_counter.c new file +10
-0+/* +* If cgrp is the root then res is NULL, however in this case +* the common ancestor is NULL as well, making the below a NOP. +*/ + err =res_counter_charge_until (res, common_ancestor,1 ,NULL ); +if (err) +return -EINVAL ; + +return 0 ; +}
Quoted 7 lines
One would expect a "can"-named function to return a boolean. This one returns an errno which is OK, I guess. But the function is misnamed because if successful it actually alters charges. A less misleading name would be simply
task_counter_attach_task(), but that's already taken. Or perhapstask_counter_try_attach_task(), but that seems unnecessary to me - many many kernel functions "try" something and back out with an errno if it failed.
Yeah, the ->can_attach_task() cgroup subsystem callbacks are more than just things that passively report if something is possible or not. They allow some side effects that can be rolled back in ->cancel_attach() callbacks.
May be they could be renamed as pre_attach() one day. ->pre_attach() callbacks already exist but are targeted for removal in Tejun's patches.
Quoted 4 lines
I really do dislike the fact that the documentation is over in another file and another patch. For a start, it makes review harder and slower.
Ok, I've merged the doc in that patch.
Thanks!
PATCH v4 10/12 cgroups: Add documentation for task counter subsystem
In reply to Frederic Weisbecker--- Documentation/cgroups/task_counter.txt | 126 ++++++++++++++++++++++++++++++++ 1 files changed, 126 insertions(+), 0 deletions(-) create mode 100644 Documentation/cgroups/task_counter.txt
Documentation/cgroups/task_counter.txt new file +126 -0
@@ -0,0 +1,126 @@ +Task counter subsystem + +1. Description + +The task counter subsystem limits the number of tasks running +inside a given cgroup. It behaves like the NR_PROC rlimit but in +the scope of a cgroup instead of a user. + +It has two typical usecases, although more can probably be found: + +- Protect against forkbombs that explode inside a container when +that container is implemented using a cgroup. The NR_PROC rlimit +is not efficient for that because if we have several containers +running in parallel under the same user, one container could starve +all the others by spawning a high number of tasks close to the +rlimit boundary. So in this case we need this limitation to be +done in a per cgroup granularity. + +- Kill all tasks inside a cgroup without races. By setting the limit +of running tasks to 0, one can prevent from any further fork inside a +cgroup and then kill all of its tasks without the need to retry an +unbound amount of time due to races between kills and forks running +in parallel (more details in "Kill a cgroup safely" paragraph). + + +2. Interface + +When a hierarchy is mounted with the task counter subsystem binded, it +adds two files into the cgroups directories, except the root one: + +- tasks.usage contains the number of tasks running inside a cgroup and +its children in the hierarchy (see paragraph about Inheritance). + +- tasks.limit contains the maximum number of tasks that can run inside +a cgroup. We check this limit when a task forks or when it is migrated +to a cgroup. + +Note that the tasks.limit value can be forced below tasks.usage, in which +case any new task in the cgroup will be rejected until the tasks.usage +value goes below tasks.limit. + +For optimization reasons, the root directory of a hierarchy doesn't have +a task counter. + + +3. Inheritance + +When a task is added to a cgroup, by way of a cgroup migration or a fork, +it increases the task counter of that cgroup and of all its ancestors. +Hence a cgroup is also subject to the limit of its ancestors. + +In the following hierarchy: + + + A + | + B + / \ + C D + + +We have 1 task running in B, one running in C and none running in D. +It means we have tasks.usage = 1 in C and tasks.usage = 2 in B because +B counts its task and those of its children. + +Now lets set tasks.limit = 2 in B and tasks.limit = 1 in D. +If we move a new task in D, it will be refused because the limit in B has +been reached already. + + +4. Kill a cgroup safely + +As explained in the description, this subsystem is also helpful to +kill all tasks in a cgroup safely, after setting tasks.limit to 0, +so that we don't race against parallel forks in an unbound numbers +of kill iterations. + +But there is a small detail to be aware of to use this feature that +way. + +Some typical way to proceed would be: + + echo 0 > tasks.limit + for TASK in $(cat cgroup.procs) + do + kill -KILL $TASK + done + +However there is a small race window where a task can be in the way to +be forked but hasn't enough completed the fork to have the PID of the +fork appearing in the cgroup.procs file. + +The only way to get it right is to run a loop that reads tasks.usage, kill +all the tasks in cgroup.procs and exit the loop only if the value in +tasks.usage was the same than the number of tasks that were in cgroup.procs, +ie: the number of tasks that were killed. + +It works because the new child appears in tasks.usage right before we check, +in the fork path, whether the parent has a pending signal, in which case the +fork is cancelled anyway. So relying on tasks.usage is fine and non-racy. + +This race window is tiny and unlikely to happen, so most of the time a single +kill iteration should be enough. But it's worth knowing about that corner +case spotted by Oleg Nesterov. + +Example of safe use would be: + + echo 0 > tasks.limit + END=false + + while [ $END == false ] + do + NR_TASKS=$(cat tasks.usage) + NR_KILLED=0 + + for TASK in $(cat cgroup.procs) + do + let NR_KILLED=NR_KILLED+1 + kill -KILL $TASK + done + + if [ "$NR_TASKS" = "$NR_KILLED" ] + then + END=true + fi + done
Signature · 1 line
1.7.5.4
Andrew Morton
maintainerRe: [PATCH 10/12] cgroups: Add documentation for task counter subsystem
Oh. And I now see that the documentation file provides no documentation for the code in cgroup_task_counter.c anyway. Really?
Re: [PATCH 10/12] cgroups: Add documentation for task counter subsystem
Quoted 2 lines · Andrew Morton
Oh. And I now see that the documentation file provides no documentation for the code in
cgroup_task_counter.canyway. Really?
I've added more comment in that file for the v5. Especially for the can_attach - attach - cancel_attach sequence that is a bit tricky.
Thanks.
RFC PATCH v4 11/12 cgroups: Allow subsystems to cancel a fork
In reply to Frederic WeisbeckerLet the subsystem's fork callback return an error value so that they can cancel a fork. This is going to be used by the task counter subsystem to implement the limit.
--- include/linux/cgroup.h | 14 +++++++++----- kernel/cgroup.c | 23 +++++++++++++++++++---- kernel/cgroup_freezer.c | 6 ++++-- kernel/exit.c | 2 +- kernel/fork.c | 7 +++++-- 5 files changed, 38 insertions(+), 14 deletions(-)
include/linux/cgroup.h +9 -5
@@ -32,9 +32,11 @@ extern int cgroup_lock_is_held(void);extern bool cgroup_lock_live_group (struct cgroup *cgrp);extern void cgroup_unlock (void );extern void cgroup_fork (struct task_struct *p); -extern void cgroup_fork_callbacks (struct task_struct *p); +extern int cgroup_fork_callbacks (struct task_struct *p, +struct cgroup_subsys **failed_ss);extern void cgroup_post_fork (struct task_struct *p); -extern void cgroup_exit (struct task_struct *p,int run_callbacks); +extern void cgroup_exit (struct task_struct *p,int run_callbacks, +struct cgroup_subsys *failed_ss);extern int cgroupstats_build (struct cgroupstats *stats,struct dentry *dentry);extern int cgroup_load_subsys (struct cgroup_subsys *ss);
@@ -477,7 +479,7 @@ struct cgroup_subsys {void (*attach_task)(struct cgroup *cgrp,struct cgroup *old_cgrp,struct task_struct *tsk);void (*attach)(struct cgroup_subsys *ss,struct cgroup *cgrp,struct cgroup *old_cgrp,struct task_struct *tsk); -void (*fork)(struct cgroup_subsys *ss,struct task_struct *task); +int (*fork)(struct cgroup_subsys *ss,struct task_struct *task);void (*exit)(struct cgroup_subsys *ss,struct cgroup *cgrp,struct cgroup *old_cgrp,struct task_struct *task); int (*populate)(struct cgroup_subsys *ss,
@@ -634,9 +636,11 @@ struct cgroup_subsys_state *cgroup_css_from_dir(struct file *f, int id);static inline int cgroup_init_early (void ) {return 0 ; }static inline int cgroup_init (void ) {return 0 ; }static inline void cgroup_fork (struct task_struct *p) {} -static inline void cgroup_fork_callbacks (struct task_struct *p) {} +static inline int cgroup_fork_callbacks (struct task_struct *p, +struct cgroup_subsys **failed_ss) {}static inline void cgroup_post_fork (struct task_struct *p) {} -static inline void cgroup_exit (struct task_struct *p,int callbacks) {} +static inline void cgroup_exit (struct task_struct *p,int callbacks, +struct cgroup_subsys **failed_ss) {}static inline void cgroup_lock (void ) {}static inline void cgroup_unlock (void ) {}
kernel/cgroup.c +19 -4
@@ -4538,8 +4538,11 @@ void cgroup_fork(struct task_struct *child)* tasklist. No need to take any locks since no-one can * be operating on this task. */ -void cgroup_fork_callbacks (struct task_struct *child) +int cgroup_fork_callbacks (struct task_struct *child, +struct cgroup_subsys **failed_ss) { +int err; +if (need_forkexit_callback) {int i;/*
@@ -4549,10 +4552,17 @@ void cgroup_fork_callbacks(struct task_struct *child)*/ for (i =0 ; i <CGROUP_BUILTIN_SUBSYS_COUNT ; i++) {struct cgroup_subsys *ss = subsys[i]; -if (ss->fork) - ss->fork (ss, child); +if (ss->fork) { + err = ss->fork (ss, child); +if (err) { + *failed_ss = ss; +return err; + } + } } } + +return 0 ; }/**
@@ -4610,7 +4620,8 @@ void cgroup_post_fork(struct task_struct *child)* which wards off any cgroup_attach_task() attempts, or task is a failed * fork, never visible to cgroup_attach_task. */ -void cgroup_exit (struct task_struct *tsk,int run_callbacks) +void cgroup_exit (struct task_struct *tsk,int run_callbacks, +struct cgroup_subsys *failed_ss) {struct css_set *cg;int i;
@@ -4639,6 +4650,10 @@ void cgroup_exit(struct task_struct *tsk, int run_callbacks)*/ for (i =0 ; i <CGROUP_BUILTIN_SUBSYS_COUNT ; i++) {struct cgroup_subsys *ss = subsys[i]; + +if (ss == failed_ss) +break ; +if (ss->exit) {struct cgroup *old_cgrp =rcu_dereference_raw (cg->subsys[i])->cgroup;
kernel/cgroup_freezer.c +4 -2
@@ -187,7 +187,7 @@ static int freezer_can_attach_task(struct cgroup *cgrp, struct cgroup *old_cgrp,return 0 ; } -static void freezer_fork (struct cgroup_subsys *ss,struct task_struct *task) +static int freezer_fork (struct cgroup_subsys *ss,struct task_struct *task) {struct freezer *freezer;
@@ -207,7 +207,7 @@ static void freezer_fork(struct cgroup_subsys *ss, struct task_struct *task)* following check. */ if (!freezer->css.cgroup->parent) -return ; +return 0 ;spin_lock_irq (&freezer->lock);BUG_ON (freezer->state ==CGROUP_FROZEN );
@@ -216,6 +216,8 @@ static void freezer_fork(struct cgroup_subsys *ss, struct task_struct *task)if (freezer->state ==CGROUP_FREEZING )freeze_task (task, true);spin_unlock_irq (&freezer->lock); + +return 0 ; }/*
kernel/exit.c +1 -1
@@ -994,7 +994,7 @@ NORET_TYPE void do_exit(long code)*/ perf_event_exit_task (tsk); -cgroup_exit (tsk,1 ); +cgroup_exit (tsk,1 ,NULL );if (group_dead)disassociate_ctty (1 );
kernel/fork.c +5 -2
@@ -1058,6 +1058,7 @@ static struct task_struct *copy_process(unsigned long clone_flags,int retval;struct task_struct *p;int cgroup_callbacks_done =0 ; +struct cgroup_subsys *cgroup_failed_ss =NULL ;if ((clone_flags & (CLONE_NEWNS |CLONE_FS )) == (CLONE_NEWNS |CLONE_FS ))return ERR_PTR (-EINVAL );
@@ -1316,8 +1317,10 @@ static struct task_struct *copy_process(unsigned long clone_flags,/* Now that the task is set up, run cgroup callbacks if * necessary. We need to run them before the task is visible * on the tasklist. */ -cgroup_fork_callbacks (p); + retval =cgroup_fork_callbacks (p, &cgroup_failed_ss); cgroup_callbacks_done =1 ; +if (retval) + goto bad_fork_free_pid;/* Need tasklist lock for parent etc handling! */ write_lock_irq (&tasklist_lock);
@@ -1423,7 +1426,7 @@ bad_fork_cleanup_cgroup:#endif if (clone_flags &CLONE_THREAD )threadgroup_fork_read_unlock (current); -cgroup_exit (p, cgroup_callbacks_done); +cgroup_exit (p, cgroup_callbacks_done, cgroup_failed_ss);delayacct_tsk_free (p);module_put (task_thread_info (p)->exec_domain->module); bad_fork_cleanup_count:
Signature · 1 line
1.7.5.4
Re: [RFC PATCH 11/12] cgroups: Allow subsystems to cancel a fork
Quoted 3 lines · On Tue, 6 Sep 2011 02:13:05 +0200 Frederic Weisbecker <[email protected]>…
Let the subsystem's fork callback return an error value so that they can cancel a fork. This is going to be used by the task counter subsystem to implement the limit.
This breaks the build quite gruesomely with CONFIG_CGROUPS=n. I looked at doing a quick fix but that would just have piled ugly on ugly.
Re: [RFC PATCH 11/12] cgroups: Allow subsystems to cancel a fork
Quoted 10 lines · Andrew Morton
Quoted 3 lines · On Tue, 6 Sep 2011 02:13:05 +0200 Frederic Weisbecker <[email protected]>…
Let the subsystem's fork callback return an error value so that they can cancel a fork. This is going to be used by the task counter subsystem to implement the limit.
This breaks the build quite gruesomely with
CONFIG_CGROUPS=n. I looked at doing a quick fix but that would just have piled ugly on ugly.
Sorry about that. I'm fixing the whole so that it builds with or without cgroups.
I'm also improving the documentation and changelog along the way.
Thanks.
RFC PATCH v4 12/12 cgroups: Convert task counter to use the subsys fork callback
In reply to Frederic WeisbeckerInstead of using an ad hoc hook on the fork path for the task counter, use the subsystem fork callback that we can now use to cancel the fork.
--- include/linux/cgroup.h | 9 --------- kernel/cgroup_task_counter.c | 3 ++- kernel/fork.c | 4 ---- 3 files changed, 2 insertions(+), 14 deletions(-)
include/linux/cgroup.h +0 -9
@@ -663,13 +663,4 @@ static inline int cgroup_attach_task_current_cg(struct task_struct *t)#endif /* !CONFIG_CGROUPS */ -#ifdef CONFIG_CGROUP_TASK_COUNTER -int cgroup_task_counter_fork (struct task_struct *child); -#else -static inline int cgroup_task_counter_fork (struct task_struct *child) -{ -return 0 ; -} -#endif /* CONFIG_CGROUP_TASK_COUNTER */ -#endif /* _LINUX_CGROUP_H */
kernel/cgroup_task_counter.c +2 -1
@@ -168,7 +168,7 @@ static int task_counter_populate(struct cgroup_subsys *ss, struct cgroup *cgrp)return cgroup_add_files (cgrp, ss, files,ARRAY_SIZE (files)); } -int cgroup_task_counter_fork (struct task_struct *child) +static int task_counter_fork (struct cgroup_subsys *ss,struct task_struct *child) {struct cgroup_subsys_state *css = child->cgroups->subsys[tasks_subsys_id];struct cgroup *cgrp = css->cgroup;
@@ -195,5 +195,6 @@ struct cgroup_subsys tasks_subsys = { .can_attach_task = task_counter_can_attach_task, .cancel_attach_task = task_counter_cancel_attach_task, .attach_task = task_counter_attach_task, + .fork = task_counter_fork, .populate = task_counter_populate, };
kernel/fork.c +0 -4
@@ -1310,10 +1310,6 @@ static struct task_struct *copy_process(unsigned long clone_flags, p->group_leader = p;INIT_LIST_HEAD (&p->thread_group); - retval =cgroup_task_counter_fork (p); -if (retval) - goto bad_fork_free_pid; -/* Now that the task is set up, run cgroup callbacks if * necessary. We need to run them before the task is visible * on the tasklist. */
Signature · 1 line
1.7.5.4