Skip to contentSkip to conversation

cgroups: support for module-loadable subsystems

PATCH In review Ben Blum 13 participants 45 messages Dec 31, 2009 – Jan 14, 2010 12 messages not in the archive

Subsystems (7)
lore
Share & tools
lore

Ben Blum

author
#0 lore raw

v4 cover cgroups: support for module-loadable subsystems

[This is a revision of http://lkml.org/lkml/2009/12/21/211 ]

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Patch #1 sets up the subsys[] array so its contents can be dynamic as modules appear and (eventually) disappear. Iterations over the array are modified to handle when subsystems are absent, and the dynamic section of the array is protected by cgroup_mutex.

Patch #2 implements an interface for modules to load subsystems, called cgroup_load_subsys, similar to cgroup_init_subsys, and adds a module pointer in struct cgroup_subsys.

Patch #3 adds a mechanism for unloading modular subsystems, which includes a more advanced rework of the rudimentary reference counting introduced in patch 2.

Patch #4 modifies the net_cls subsystem, which already had some module declarations, to be configurable as a module, which also serves as a simple proof-of-concept.

Part of implementing patches 2 and 4 involved updating css pointers in each css_set when the module appears or leaves. In doing this, it was discovered that css_sets always remain linked to the dummy cgroup, regardless of whether or not any subsystems are actually bound to it (i.e., not mounted on an actual hierarchy). The subsystem loading and unloading code therefore should keep in mind the special cases where the added subsystem is the only one in the dummy cgroup (and therefore all css_sets need to be linked back into it) and where the removed subsys was the only one in the dummy cgroup (and therefore all css_sets should be unlinked from it) - however, as all css_sets always stay attached to the dummy cgroup anyway, these cases are ignored. Any fix that addresses this issue should also make sure these cases are addressed in the subsystem loading and unloading code.

-- bblum

---
 Documentation/cgroups/cgroups.txt |    9
 include/linux/cgroup.h            |   18 +
 kernel/cgroup.c                   |  388 +++++++++++++++++++++++++++++++++-----
 net/sched/Kconfig                 |    5
 net/sched/cls_cgroup.c            |   36 ++-
 5 files changed, 400 insertions(+), 56 deletions(-)

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw

PATCH v4 1/4 cgroups: revamp subsys array

Make subsys[] able to be dynamically populated to support modular subsystems

From: Ben Blum <[email protected]>

This patch reworks the way the subsys[] array is used so that subsystems can register themselves after boot time, and enables the internals of cgroups to be able to handle when subsystems are not present or may appear/disappear.

---

 include/linux/cgroup.h |   10 ++++-
 kernel/cgroup.c        |   96 ++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 88 insertions(+), 18 deletions(-)
include/linux/cgroup.h +8 -2
index 0008dee..83da43d 100644
@@ -39,13 +39,19 @@ extern int cgroupstats_build(struct cgroupstats *stats,
 
 extern const struct file_operations proc_cgroup_operations;
 
-/* Define the enumeration of all cgroup subsystems */
+/* Define the enumeration of all builtin cgroup subsystems */
 #define SUBSYS(_x) _x ## _subsys_id,
 enum cgroup_subsys_id {
 #include <linux/cgroup_subsys.h>
-	CGROUP_SUBSYS_COUNT
+	CGROUP_BUILTIN_SUBSYS_COUNT
 };
 #undef SUBSYS
+/*
+ * This define indicates the maximum number of subsystems that can be loaded
+ * at once. We limit to this many since cgroupfs_root has subsys_bits to keep
+ * track of all of them.
+ */
+#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
 
 /* Per-subsystem/per-cgroup state maintained by the system. */
 struct cgroup_subsys_state {
kernel/cgroup.c +80 -16

238 lines not on this page · raw message

#3 lore raw

PATCH v4 2/4 cgroups: subsystem module loading interface

In reply to a message not in the archive
Quoted 43 lines · Ben Blum

[This is a revision of http://lkml.org/lkml/2009/12/21/211 ]

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Patch #1 sets up the subsys[] array so its contents can be dynamic as modules appear and (eventually) disappear. Iterations over the array are modified to handle when subsystems are absent, and the dynamic section of the array is protected by cgroup_mutex.

Patch #2 implements an interface for modules to load subsystems, called cgroup_load_subsys, similar to cgroup_init_subsys, and adds a module pointer in struct cgroup_subsys.

Patch #3 adds a mechanism for unloading modular subsystems, which includes a more advanced rework of the rudimentary reference counting introduced in patch 2.

Patch #4 modifies the net_cls subsystem, which already had some module declarations, to be configurable as a module, which also serves as a simple proof-of-concept.

Part of implementing patches 2 and 4 involved updating css pointers in each css_set when the module appears or leaves. In doing this, it was discovered that css_sets always remain linked to the dummy cgroup, regardless of whether or not any subsystems are actually bound to it (i.e., not mounted on an actual hierarchy). The subsystem loading and unloading code therefore should keep in mind the special cases where the added subsystem is the only one in the dummy cgroup (and therefore all css_sets need to be linked back into it) and where the removed subsys was the only one in the dummy cgroup (and therefore all css_sets should be unlinked from it) - however, as all css_sets always stay attached to the dummy cgroup anyway, these cases are ignored. Any fix that addresses this issue should also make sure these cases are addressed in the subsystem loading and unloading code.

-- bblum

Quoted 9 lines
---
 Documentation/cgroups/cgroups.txt |    9
 include/linux/cgroup.h            |   18 +
 kernel/cgroup.c                   |  388 +++++++++++++++++++++++++++++++++-----
 net/sched/Kconfig                 |    5
 net/sched/cls_cgroup.c            |   36 ++-
 5 files changed, 400 insertions(+), 56 deletions(-)

Add interface between cgroups subsystem management and module loading

From: Ben Blum <[email protected]>

This patch implements rudimentary module-loading support for cgroups - namely, a cgroup_load_subsys (similar to cgroup_init_subsys) for use as a module initcall, and a struct module pointer in struct cgroup_subsys.

Several functions that might be wanted by modules have had EXPORT_SYMBOL added to them, but it's unclear exactly which functions want it and which won't.

---

 Documentation/cgroups/cgroups.txt |    4 +
 include/linux/cgroup.h            |    4 +
 kernel/cgroup.c                   |  128 +++++++++++++++++++++++++++++++++++++
 3 files changed, 136 insertions(+), 0 deletions(-)
Documentation/cgroups/cgroups.txt +4 -0
index 0b33bfe..6ffcf81 100644
@@ -488,6 +488,10 @@ Each subsystem should:
 - add an entry in linux/cgroup_subsys.h
 - define a cgroup_subsys object called <name>_subsys
 
+If a subsystem can be compiled as a module, it should also have in its
+module initcall a call to cgroup_load_subsys(&its_subsys_struct). It
+should also set its_subsys.module = THIS_MODULE in its .c file.
+
 Each subsystem may export the following methods. The only mandatory
 methods are create/destroy. Any others that are null are presumed to
 be successful no-ops.
include/linux/cgroup.h +4 -0
index 83da43d..9461aed 100644
@@ -36,6 +36,7 @@ extern void cgroup_post_fork(struct task_struct *p);
 extern void cgroup_exit(struct task_struct *p, int run_callbacks);
 extern int cgroupstats_build(struct cgroupstats *stats,
 				struct dentry *dentry);
+extern int cgroup_load_subsys(struct cgroup_subsys *ss);
 
 extern const struct file_operations proc_cgroup_operations;
 
@@ -477,6 +478,9 @@ struct cgroup_subsys {
 	/* used when use_id == true */
 	struct idr idr;
 	spinlock_t id_lock;
+
+	/* should be defined only by modular subsystems */
+	struct module *module;
 };
 
 #define SUBSYS(_x) extern struct cgroup_subsys _x ## _subsys;
kernel/cgroup.c +128 -0

153 lines not on this page · raw message

_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

#4 lore raw

PATCH v4 3/4 cgroups: subsystem module unloading

In reply to a message not in the archive

Provides support for unloading modular subsystems.

From: Ben Blum <[email protected]>

This patch adds a new function cgroup_unload_subsys which is to be used for removing a loaded subsystem during module deletion. Reference counting of the subsystems' modules is moved from once (at load time) to once per attached hierarchy (in parse_cgroupfs_options and rebind_subsystems) (i.e., 0 or 1).

---

 Documentation/cgroups/cgroups.txt |    5 +
 include/linux/cgroup.h            |    4 +
 kernel/cgroup.c                   |  164 +++++++++++++++++++++++++++++++------
 3 files changed, 145 insertions(+), 28 deletions(-)
Documentation/cgroups/cgroups.txt +3 -2
index 6ffcf81..7527bac 100644
@@ -489,8 +489,9 @@ Each subsystem should:
 - define a cgroup_subsys object called <name>_subsys
 
 If a subsystem can be compiled as a module, it should also have in its
-module initcall a call to cgroup_load_subsys(&its_subsys_struct). It
-should also set its_subsys.module = THIS_MODULE in its .c file.
+module initcall a call to cgroup_load_subsys(), and in its exitcall a
+call to cgroup_unload_subsys(). It should also set its_subsys.module =
+THIS_MODULE in its .c file.
 
 Each subsystem may export the following methods. The only mandatory
 methods are create/destroy. Any others that are null are presumed to
include/linux/cgroup.h +3 -1
index 9461aed..9be4c22 100644
@@ -37,6 +37,7 @@ extern void cgroup_exit(struct task_struct *p, int run_callbacks);
 extern int cgroupstats_build(struct cgroupstats *stats,
 				struct dentry *dentry);
 extern int cgroup_load_subsys(struct cgroup_subsys *ss);
+extern void cgroup_unload_subsys(struct cgroup_subsys *ss);
 
 extern const struct file_operations proc_cgroup_operations;
 
@@ -264,7 +265,8 @@ struct css_set {
 	/*
 	 * Set of subsystem states, one for each subsystem. This array
 	 * is immutable after creation apart from the init_css_set
-	 * during subsystem registration (at boot time).
+	 * during subsystem registration (at boot time) and modular subsystem
+	 * loading/unloading.
 	 */
 	struct cgroup_subsys_state *subsys[CGROUP_SUBSYS_COUNT];
 
kernel/cgroup.c +139 -25

277 lines not on this page · raw message

#5 lore raw

PATCH v4 4/4 cgroups: net_cls as module

In reply to a message not in the archive

Allows the net_cls cgroup subsystem to be compiled as a module

From: Ben Blum <[email protected]>

This patch modifies net/sched/cls_cgroup.c to allow the net_cls subsystem to be optionally compiled as a module instead of builtin. The cgroup_subsys struct is moved around a bit to allow the subsys_id to be either declared as a compile-time constant by the cgroup_subsys.h include in cgroup.h, or, if it's a module, initialized within the struct by cgroup_load_subsys.

---

 net/sched/Kconfig      |    5 ++++-
 net/sched/cls_cgroup.c |   36 +++++++++++++++++++++++++++---------
 2 files changed, 31 insertions(+), 10 deletions(-)
net/sched/Kconfig +4 -1
index 929218a..08ff9bc 100644
@@ -328,13 +328,16 @@ config NET_CLS_FLOW
 	  module will be called cls_flow.
 
 config NET_CLS_CGROUP
-	bool "Control Group Classifier"
+	tristate "Control Group Classifier"
 	select NET_CLS
 	depends on CGROUPS
 	---help---
 	  Say Y here if you want to classify packets based on the control
 	  cgroup of their process.
 
+	  To compile this code as a module, choose M here: the
+	  module will be called cls_cgroup.
+
 config NET_EMATCH
 	bool "Extended Matches"
 	select NET_CLS
net/sched/cls_cgroup.c +27 -9
index e4877ca..7f27d2c 100644
@@ -24,6 +24,25 @@ struct cgroup_cls_state
 	u32 classid;
 };
 
+static struct cgroup_subsys_state *cgrp_create(struct cgroup_subsys *ss,
+					       struct cgroup *cgrp);
+static void cgrp_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
+static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp);
+
+struct cgroup_subsys net_cls_subsys = {
+	.name		= "net_cls",
+	.create		= cgrp_create,
+	.destroy	= cgrp_destroy,
+	.populate	= cgrp_populate,
+#ifdef CONFIG_NET_CLS_CGROUP
+	.subsys_id	= net_cls_subsys_id,
+#else
+#define net_cls_subsys_id net_cls_subsys.subsys_id
+#endif
+	.module		= THIS_MODULE,
+};
+
+
 static inline struct cgroup_cls_state *cgrp_cls_state(struct cgroup *cgrp)
 {
 	return container_of(cgroup_subsys_state(cgrp, net_cls_subsys_id),
@@ -79,14 +98,6 @@ static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
 	return cgroup_add_files(cgrp, ss, ss_files, ARRAY_SIZE(ss_files));
 }
 
-struct cgroup_subsys net_cls_subsys = {
-	.name		= "net_cls",
-	.create		= cgrp_create,
-	.destroy	= cgrp_destroy,
-	.populate	= cgrp_populate,
-	.subsys_id	= net_cls_subsys_id,
-};
-
 struct cls_cgroup_head
 {
 	u32			handle;
@@ -277,12 +288,19 @@ static struct tcf_proto_ops cls_cgroup_ops __read_mostly = {
 
 static int __init init_cgroup_cls(void)
 {
-	return register_tcf_proto_ops(&cls_cgroup_ops);
+	int ret = register_tcf_proto_ops(&cls_cgroup_ops);
+	if (ret)
+		return ret;
+	ret = cgroup_load_subsys(&net_cls_subsys);
+	if (ret)
+		unregister_tcf_proto_ops(&cls_cgroup_ops);
+	return ret;
 }
 
 static void __exit exit_cgroup_cls(void)
 {
 	unregister_tcf_proto_ops(&cls_cgroup_ops);
+	cgroup_unload_subsys(&net_cls_subsys);
 }
 
 module_init(init_cgroup_cls);
author
#6 lore raw

PATCH v4 1/4 cgroups: revamp subsys array

In reply to Ben Blum

Make subsys[] able to be dynamically populated to support modular subsystems

From: Ben Blum <[email protected]>

This patch reworks the way the subsys[] array is used so that subsystems can register themselves after boot time, and enables the internals of cgroups to be able to handle when subsystems are not present or may appear/disappear.

---

 include/linux/cgroup.h |   10 ++++-
 kernel/cgroup.c        |   96 ++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 88 insertions(+), 18 deletions(-)
include/linux/cgroup.h +8 -2
index 0008dee..83da43d 100644
@@ -39,13 +39,19 @@ extern int cgroupstats_build(struct cgroupstats *stats,
 
 extern const struct file_operations proc_cgroup_operations;
 
-/* Define the enumeration of all cgroup subsystems */
+/* Define the enumeration of all builtin cgroup subsystems */
 #define SUBSYS(_x) _x ## _subsys_id,
 enum cgroup_subsys_id {
 #include <linux/cgroup_subsys.h>
-	CGROUP_SUBSYS_COUNT
+	CGROUP_BUILTIN_SUBSYS_COUNT
 };
 #undef SUBSYS
+/*
+ * This define indicates the maximum number of subsystems that can be loaded
+ * at once. We limit to this many since cgroupfs_root has subsys_bits to keep
+ * track of all of them.
+ */
+#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
 
 /* Per-subsystem/per-cgroup state maintained by the system. */
 struct cgroup_subsys_state {
kernel/cgroup.c +80 -16

238 lines not on this page · raw message

author
#7 lore raw

PATCH v4 2/4 cgroups: subsystem module loading interface

In reply to Ben Blum
Quoted 43 lines · Ben Blum

[This is a revision of http://lkml.org/lkml/2009/12/21/211 ]

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Patch #1 sets up the subsys[] array so its contents can be dynamic as modules appear and (eventually) disappear. Iterations over the array are modified to handle when subsystems are absent, and the dynamic section of the array is protected by cgroup_mutex.

Patch #2 implements an interface for modules to load subsystems, called cgroup_load_subsys, similar to cgroup_init_subsys, and adds a module pointer in struct cgroup_subsys.

Patch #3 adds a mechanism for unloading modular subsystems, which includes a more advanced rework of the rudimentary reference counting introduced in patch 2.

Patch #4 modifies the net_cls subsystem, which already had some module declarations, to be configurable as a module, which also serves as a simple proof-of-concept.

Part of implementing patches 2 and 4 involved updating css pointers in each css_set when the module appears or leaves. In doing this, it was discovered that css_sets always remain linked to the dummy cgroup, regardless of whether or not any subsystems are actually bound to it (i.e., not mounted on an actual hierarchy). The subsystem loading and unloading code therefore should keep in mind the special cases where the added subsystem is the only one in the dummy cgroup (and therefore all css_sets need to be linked back into it) and where the removed subsys was the only one in the dummy cgroup (and therefore all css_sets should be unlinked from it) - however, as all css_sets always stay attached to the dummy cgroup anyway, these cases are ignored. Any fix that addresses this issue should also make sure these cases are addressed in the subsystem loading and unloading code.

-- bblum

Quoted 9 lines
---
 Documentation/cgroups/cgroups.txt |    9
 include/linux/cgroup.h            |   18 +
 kernel/cgroup.c                   |  388 +++++++++++++++++++++++++++++++++-----
 net/sched/Kconfig                 |    5
 net/sched/cls_cgroup.c            |   36 ++-
 5 files changed, 400 insertions(+), 56 deletions(-)

Add interface between cgroups subsystem management and module loading

From: Ben Blum <[email protected]>

This patch implements rudimentary module-loading support for cgroups - namely, a cgroup_load_subsys (similar to cgroup_init_subsys) for use as a module initcall, and a struct module pointer in struct cgroup_subsys.

Several functions that might be wanted by modules have had EXPORT_SYMBOL added to them, but it's unclear exactly which functions want it and which won't.

---

 Documentation/cgroups/cgroups.txt |    4 +
 include/linux/cgroup.h            |    4 +
 kernel/cgroup.c                   |  128 +++++++++++++++++++++++++++++++++++++
 3 files changed, 136 insertions(+), 0 deletions(-)
Documentation/cgroups/cgroups.txt +4 -0
index 0b33bfe..6ffcf81 100644
@@ -488,6 +488,10 @@ Each subsystem should:
 - add an entry in linux/cgroup_subsys.h
 - define a cgroup_subsys object called <name>_subsys
 
+If a subsystem can be compiled as a module, it should also have in its
+module initcall a call to cgroup_load_subsys(&its_subsys_struct). It
+should also set its_subsys.module = THIS_MODULE in its .c file.
+
 Each subsystem may export the following methods. The only mandatory
 methods are create/destroy. Any others that are null are presumed to
 be successful no-ops.
include/linux/cgroup.h +4 -0
index 83da43d..9461aed 100644
@@ -36,6 +36,7 @@ extern void cgroup_post_fork(struct task_struct *p);
 extern void cgroup_exit(struct task_struct *p, int run_callbacks);
 extern int cgroupstats_build(struct cgroupstats *stats,
 				struct dentry *dentry);
+extern int cgroup_load_subsys(struct cgroup_subsys *ss);
 
 extern const struct file_operations proc_cgroup_operations;
 
@@ -477,6 +478,9 @@ struct cgroup_subsys {
 	/* used when use_id == true */
 	struct idr idr;
 	spinlock_t id_lock;
+
+	/* should be defined only by modular subsystems */
+	struct module *module;
 };
 
 #define SUBSYS(_x) extern struct cgroup_subsys _x ## _subsys;
kernel/cgroup.c +128 -0

153 lines not on this page · raw message

author
#8 lore raw

PATCH v4 3/4 cgroups: subsystem module unloading

In reply to Ben Blum

Provides support for unloading modular subsystems.

From: Ben Blum <[email protected]>

This patch adds a new function cgroup_unload_subsys which is to be used for removing a loaded subsystem during module deletion. Reference counting of the subsystems' modules is moved from once (at load time) to once per attached hierarchy (in parse_cgroupfs_options and rebind_subsystems) (i.e., 0 or 1).

---

 Documentation/cgroups/cgroups.txt |    5 +
 include/linux/cgroup.h            |    4 +
 kernel/cgroup.c                   |  164 +++++++++++++++++++++++++++++++------
 3 files changed, 145 insertions(+), 28 deletions(-)
Documentation/cgroups/cgroups.txt +3 -2
index 6ffcf81..7527bac 100644
@@ -489,8 +489,9 @@ Each subsystem should:
 - define a cgroup_subsys object called <name>_subsys
 
 If a subsystem can be compiled as a module, it should also have in its
-module initcall a call to cgroup_load_subsys(&its_subsys_struct). It
-should also set its_subsys.module = THIS_MODULE in its .c file.
+module initcall a call to cgroup_load_subsys(), and in its exitcall a
+call to cgroup_unload_subsys(). It should also set its_subsys.module =
+THIS_MODULE in its .c file.
 
 Each subsystem may export the following methods. The only mandatory
 methods are create/destroy. Any others that are null are presumed to
include/linux/cgroup.h +3 -1
index 9461aed..9be4c22 100644
@@ -37,6 +37,7 @@ extern void cgroup_exit(struct task_struct *p, int run_callbacks);
 extern int cgroupstats_build(struct cgroupstats *stats,
 				struct dentry *dentry);
 extern int cgroup_load_subsys(struct cgroup_subsys *ss);
+extern void cgroup_unload_subsys(struct cgroup_subsys *ss);
 
 extern const struct file_operations proc_cgroup_operations;
 
@@ -264,7 +265,8 @@ struct css_set {
 	/*
 	 * Set of subsystem states, one for each subsystem. This array
 	 * is immutable after creation apart from the init_css_set
-	 * during subsystem registration (at boot time).
+	 * during subsystem registration (at boot time) and modular subsystem
+	 * loading/unloading.
 	 */
 	struct cgroup_subsys_state *subsys[CGROUP_SUBSYS_COUNT];
 
kernel/cgroup.c +139 -25

277 lines not on this page · raw message

author
#9 lore raw

PATCH v4 4/4 cgroups: net_cls as module

In reply to Ben Blum

Allows the net_cls cgroup subsystem to be compiled as a module

From: Ben Blum <[email protected]>

This patch modifies net/sched/cls_cgroup.c to allow the net_cls subsystem to be optionally compiled as a module instead of builtin. The cgroup_subsys struct is moved around a bit to allow the subsys_id to be either declared as a compile-time constant by the cgroup_subsys.h include in cgroup.h, or, if it's a module, initialized within the struct by cgroup_load_subsys.

---

 net/sched/Kconfig      |    5 ++++-
 net/sched/cls_cgroup.c |   36 +++++++++++++++++++++++++++---------
 2 files changed, 31 insertions(+), 10 deletions(-)
net/sched/Kconfig +4 -1
index 929218a..08ff9bc 100644
@@ -328,13 +328,16 @@ config NET_CLS_FLOW
 	  module will be called cls_flow.
 
 config NET_CLS_CGROUP
-	bool "Control Group Classifier"
+	tristate "Control Group Classifier"
 	select NET_CLS
 	depends on CGROUPS
 	---help---
 	  Say Y here if you want to classify packets based on the control
 	  cgroup of their process.
 
+	  To compile this code as a module, choose M here: the
+	  module will be called cls_cgroup.
+
 config NET_EMATCH
 	bool "Extended Matches"
 	select NET_CLS
net/sched/cls_cgroup.c +27 -9
index e4877ca..7f27d2c 100644
@@ -24,6 +24,25 @@ struct cgroup_cls_state
 	u32 classid;
 };
 
+static struct cgroup_subsys_state *cgrp_create(struct cgroup_subsys *ss,
+					       struct cgroup *cgrp);
+static void cgrp_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
+static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp);
+
+struct cgroup_subsys net_cls_subsys = {
+	.name		= "net_cls",
+	.create		= cgrp_create,
+	.destroy	= cgrp_destroy,
+	.populate	= cgrp_populate,
+#ifdef CONFIG_NET_CLS_CGROUP
+	.subsys_id	= net_cls_subsys_id,
+#else
+#define net_cls_subsys_id net_cls_subsys.subsys_id
+#endif
+	.module		= THIS_MODULE,
+};
+
+
 static inline struct cgroup_cls_state *cgrp_cls_state(struct cgroup *cgrp)
 {
 	return container_of(cgroup_subsys_state(cgrp, net_cls_subsys_id),
@@ -79,14 +98,6 @@ static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
 	return cgroup_add_files(cgrp, ss, ss_files, ARRAY_SIZE(ss_files));
 }
 
-struct cgroup_subsys net_cls_subsys = {
-	.name		= "net_cls",
-	.create		= cgrp_create,
-	.destroy	= cgrp_destroy,
-	.populate	= cgrp_populate,
-	.subsys_id	= net_cls_subsys_id,
-};
-
 struct cls_cgroup_head
 {
 	u32			handle;
@@ -277,12 +288,19 @@ static struct tcf_proto_ops cls_cgroup_ops __read_mostly = {
 
 static int __init init_cgroup_cls(void)
 {
-	return register_tcf_proto_ops(&cls_cgroup_ops);
+	int ret = register_tcf_proto_ops(&cls_cgroup_ops);
+	if (ret)
+		return ret;
+	ret = cgroup_load_subsys(&net_cls_subsys);
+	if (ret)
+		unregister_tcf_proto_ops(&cls_cgroup_ops);
+	return ret;
 }
 
 static void __exit exit_cgroup_cls(void)
 {
 	unregister_tcf_proto_ops(&cls_cgroup_ops);
+	cgroup_unload_subsys(&net_cls_subsys);
 }
 
 module_init(init_cgroup_cls);
#10 lore raw
In reply to Ben Blum
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

A message that is not in the archive (sent off-list or not yet received): 20100106160414.bd555474.akpm-de/[email protected]

#12 lore raw
Quoted 11 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

author
#13 lore raw
In reply to Andrew Morton
Quoted 11 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#15 lore raw
Quoted 12 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 11 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 8 lines
Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Hmm, do you have your own module in plan ?

Thanks,
-Kame

#16 lore raw
In reply to Ben Blum
Quoted 12 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum <[email protected]> wrote:
Quoted 11 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 8 lines
Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Hmm, do you have your own module in plan ?

Thanks,
-Kame

A message that is not in the archive (sent off-list or not yet received): 20100107120732.97d502bd.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org

#18 lore raw
Quoted 16 lines · KAMEZAWA Hiroyuki
Quoted 13 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 12 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

Quoted 8 lines
Quoted 5 lines

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

#19 lore raw
In reply to KAMEZAWA Hiroyuki
Quoted 16 lines · KAMEZAWA Hiroyuki
Quoted 13 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum <[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

Quoted 8 lines
Quoted 5 lines

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw
Quoted 23 lines · On Thu, 07 Jan 2010 14:42:19 +0800 Li Zefan…
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 5 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Hmm, I read the patch slightly. I'm not enough expert to review this patch..

I requst following as TODO.
(No objection to the direction/patch.)

1. Add documentation about load/unlod module.
It seems module unloading will not succuess while subsystem is mounted. Right ?

2. Making this to be reasonable value.
#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
I can't find why.

3. show whehter a subsys is a loadable module or not via /proc/cgroups

4. how to test ? load/unload NET_CLS is enough ?

Last one is question.

5. Is following path is safe ?

    find_css_set() {
     ....
          read_lock(&css_set_lock);
          get template including pointer
          read_unlock(&css_set_lock);

          use template to build new css_set.

Thanks,
-Kame

#22 lore raw
In reply to a message not in the archive
Quoted 23 lines · Li Zefan
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 4 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Ok, the following two patches make this happen (or at least pretend to well enough to fool me). The first one adds use_id initialization in cgroup_load_subsys, and the second rearranges config options and some code as appropriate in block/ and adds EXPORT_SYMBOLs in cgroup.c.

-- bblum

---
 block/Kconfig             |    2 -
 block/Kconfig.iosched     |    2 -
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++-----------
 block/blk-cgroup.h        |   10 ++++++--
 include/linux/iocontext.h |    2 -
 kernel/cgroup.c           |   31 +++++++++++++++++++++-----
 6 files changed, 77 insertions(+), 23 deletions(-)
#23 lore raw
In reply to Li Zefan
Quoted 23 lines · On Thu, 07 Jan 2010 14:42:19 +0800 Li Zefan <[email protected]> wrote:
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 5 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Hmm, I read the patch slightly. I'm not enough expert to review this patch..

I requst following as TODO.
(No objection to the direction/patch.)

1. Add documentation about load/unlod module.
It seems module unloading will not succuess while subsystem is mounted. Right ?

2. Making this to be reasonable value.
#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
I can't find why.

3. show whehter a subsys is a loadable module or not via /proc/cgroups

4. how to test ? load/unload NET_CLS is enough ?

Last one is question.

5. Is following path is safe ?

    find_css_set() {
     ....
          read_lock(&css_set_lock);
          get template including pointer
          read_unlock(&css_set_lock);

          use template to build new css_set.

Thanks,
-Kame

A message that is not in the archive (sent off-list or not yet received): 20100107161627.34b31e0c.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org

#25 lore raw
Quoted 32 lines · KAMEZAWA Hiroyuki
Quoted 23 lines · On Thu, 07 Jan 2010 14:42:19 +0800 Li Zefan…
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 5 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Quoted 6 lines

Hmm, I read the patch slightly. I'm not enough expert to review this patch..

I requst following as TODO.
(No objection to the direction/patch.)

1. Add documentation about load/unlod module.

could perhaps use more, i suppose.

Quoted 2 lines

It seems module unloading will not succuess while subsystem is mounted. Right ?

yes, when you mount it, it takes a reference on the module, so you get "module is in use".

Quoted 3 lines

2. Making this to be reasonable value.
#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
I can't find why.

"We limit to this many since cgroupfs_root has subsys_bits to keep track of all of them." should it be less, perhaps? the memory footprint is not great, it is true, but implementing dynamically sized subsystem tracking data structures requires much cleverer code in many places.

Quoted 1 line

3. show whehter a subsys is a loadable module or not via /proc/cgroups

with just "y" or "n"? possible, and probably easy. do note that since they are sorted by subsys_id, all the ones above a certain line will be "n" and all below will be "y".

Quoted 1 line

4. how to test ? load/unload NET_CLS is enough ?

load, mount, remount, unmount, mount with different combinations, unload was the general approach I took, plus using gdb to examine state.

Quoted 5 lines

Last one is question.

5. Is following path is safe ?

Quoted 7 lines
    find_css_set() {
     ....
          read_lock(&css_set_lock);
          get template including pointer
          read_unlock(&css_set_lock);

          use template to build new css_set.

should be, because that code is dealing with a cgrp's/css's specific ->subsys array, which state is protected by refcounts held by the already mounted hierarchy, and the other entries in the array are not cared about by the particular css in question.

Quoted 7 lines

Thanks,
-Kame

author
#26 lore raw
In reply to KAMEZAWA Hiroyuki
Quoted 32 lines · KAMEZAWA Hiroyuki
Quoted 23 lines · On Thu, 07 Jan 2010 14:42:19 +0800 Li Zefan <[email protected]> wrote:
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 5 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Quoted 6 lines

Hmm, I read the patch slightly. I'm not enough expert to review this patch..

I requst following as TODO.
(No objection to the direction/patch.)

1. Add documentation about load/unlod module.

could perhaps use more, i suppose.

Quoted 2 lines

It seems module unloading will not succuess while subsystem is mounted. Right ?

yes, when you mount it, it takes a reference on the module, so you get "module is in use".

Quoted 3 lines

2. Making this to be reasonable value.
#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
I can't find why.

"We limit to this many since cgroupfs_root has subsys_bits to keep track of all of them." should it be less, perhaps? the memory footprint is not great, it is true, but implementing dynamically sized subsystem tracking data structures requires much cleverer code in many places.

Quoted 1 line

3. show whehter a subsys is a loadable module or not via /proc/cgroups

with just "y" or "n"? possible, and probably easy. do note that since they are sorted by subsys_id, all the ones above a certain line will be "n" and all below will be "y".

Quoted 1 line

4. how to test ? load/unload NET_CLS is enough ?

load, mount, remount, unmount, mount with different combinations, unload was the general approach I took, plus using gdb to examine state.

Quoted 5 lines

Last one is question.

5. Is following path is safe ?

Quoted 7 lines
    find_css_set() {
     ....
          read_lock(&css_set_lock);
          get template including pointer
          read_unlock(&css_set_lock);

          use template to build new css_set.

should be, because that code is dealing with a cgrp's/css's specific ->subsys array, which state is protected by refcounts held by the already mounted hierarchy, and the other entries in the array are not cared about by the particular css in question.

Quoted 7 lines

Thanks,
-Kame

#27 lore raw
Quoted 6 lines · On Thu, 7 Jan 2010 02:48:12 -0500 Ben Blum…
Quoted 3 lines

2. Making this to be reasonable value.
#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
I can't find why.

"We limit to this many since cgroupfs_root has subsys_bits to keep track of all of them." should it be less, perhaps?

It's ok if it's clear that
"this decistion is done by implementation choice, not by cgroup's nature"

Quoted 4 lines

the memory footprint is not
great, it is true, but implementing dynamically sized subsystem tracking data structures requires much cleverer code in many places.

yes. I don't request that.

Quoted 6 lines
Quoted 1 line

3. show whehter a subsys is a loadable module or not via /proc/cgroups

with just "y" or "n"? possible, and probably easy. do note that since they are sorted by subsys_id, all the ones above a certain line will be "n" and all below will be "y".

yes.

#subsys_name hierarchy num_cgroups enabled module
cpuset 0 1 1 0

and 0/1 to show y/n ? (but this cause interface incompatibility...)

Thanks,
-Kame

author
#28 lore raw
Quoted 7 lines · KAMEZAWA Hiroyuki
Quoted 4 lines · On Thu, 7 Jan 2010 02:48:12 -0500 Ben Blum…
Quoted 3 lines

2. Making this to be reasonable value.
#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
I can't find why.

Quoted 5 lines
Quoted 2 lines

"We limit to this many since cgroupfs_root has subsys_bits to keep track of all of them." should it be less, perhaps?

It's ok if it's clear that
"this decistion is done by implementation choice, not by cgroup's nature"

mhm, well, it is the upper limit due to nature, but why it and not a smaller number is by choice.

Quoted 6 lines
Quoted 5 lines

the memory footprint is not
great, it is true, but implementing dynamically sized subsystem tracking data structures requires much cleverer code in many places.

yes. I don't request that.

it might be possible to have a config option as CGROUP_EXTRA_SUBSYSTEMS (with max being 64 or 32) which would add slots for subsystems outside of the kernel tree, to avoid using up a lot of blank slots in typical use cases. not entirely sure how to implement it in the scope of the configuration world, just speculation.

Quoted 6 lines
Quoted 6 lines
Quoted 1 line

3. show whehter a subsys is a loadable module or not via /proc/cgroups

with just "y" or "n"? possible, and probably easy. do note that since they are sorted by subsys_id, all the ones above a certain line will be "n" and all below will be "y".

Quoted 6 lines

yes.

#subsys_name hierarchy num_cgroups enabled module
cpuset 0 1 1 0

and 0/1 to show y/n ? (but this cause interface incompatibility...)

well, format should be agreed upon. 1/0 would be consistent with the rest of the output.

Quoted 7 lines

Thanks,
-Kame

author
#29 lore raw
In reply to Li Zefan
Quoted 23 lines · Li Zefan
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 4 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

certainly looks promising. one thing I note while looking over it is that it wants .use_id = 1, and the load_subsys interface neglects to make a call to init_idr. adding calls to cgroup_init_idr has a few possible complications... what use does blkio have for use_id?

A message that is not in the archive (sent off-list or not yet received): [email protected]

#31 lore raw
Quoted 27 lines · Ben Blum
Quoted 26 lines · Li Zefan
Quoted 24 lines · KAMEZAWA Hiroyuki
Quoted 11 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 7 lines
Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Hmm, do you have your own module in plan ?

Quoted 6 lines
Quoted 1 line

Maybe the new blkio_cgroup can also be made module-able.

certainly looks promising. one thing I note while looking over it is that it wants .use_id = 1, and the load_subsys interface neglects to make a call to init_idr. adding calls to cgroup_init_idr has a few possible complications... what use does blkio have for use_id?

oh, oops, while tracing the calls that init_idr does i ended up looking at the wrong functions and made it out to be harder than it was (in several places, even). still it would need to be included in load_subsys and the single error case handled appropriately.

author
#32 lore raw
In reply to Ben Blum
Quoted 27 lines · Ben Blum
Quoted 26 lines · Li Zefan
Quoted 24 lines · KAMEZAWA Hiroyuki
Quoted 11 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 7 lines
Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Hmm, do you have your own module in plan ?

Quoted 6 lines
Quoted 1 line

Maybe the new blkio_cgroup can also be made module-able.

certainly looks promising. one thing I note while looking over it is that it wants .use_id = 1, and the load_subsys interface neglects to make a call to init_idr. adding calls to cgroup_init_idr has a few possible complications... what use does blkio have for use_id?

oh, oops, while tracing the calls that init_idr does i ended up looking at the wrong functions and made it out to be harder than it was (in several places, even). still it would need to be included in load_subsys and the single error case handled appropriately.

author
#33 lore raw
In reply to Li Zefan
Quoted 23 lines · Li Zefan
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 4 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Ok, the following two patches make this happen (or at least pretend to well enough to fool me). The first one adds use_id initialization in cgroup_load_subsys, and the second rearranges config options and some code as appropriate in block/ and adds EXPORT_SYMBOLs in cgroup.c.

-- bblum

---
 block/Kconfig             |    2 -
 block/Kconfig.iosched     |    2 -
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++-----------
 block/blk-cgroup.h        |   10 ++++++--
 include/linux/iocontext.h |    2 -
 kernel/cgroup.c           |   31 +++++++++++++++++++++-----
 6 files changed, 77 insertions(+), 23 deletions(-)

A message that is not in the archive (sent off-list or not yet received): [email protected]

#35 lore raw

RFC PATCH 1/2 cgroups: modular subsystems support for use_id

Quoted 36 lines · Ben Blum
Quoted 23 lines · Li Zefan
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum…
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum…

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 4 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Ok, the following two patches make this happen (or at least pretend to well enough to fool me). The first one adds use_id initialization in cgroup_load_subsys, and the second rearranges config options and some code as appropriate in block/ and adds EXPORT_SYMBOLs in cgroup.c.

-- bblum

Quoted 10 lines
---
 block/Kconfig             |    2 -
 block/Kconfig.iosched     |    2 -
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++-----------
 block/blk-cgroup.h        |   10 ++++++--
 include/linux/iocontext.h |    2 -
 kernel/cgroup.c           |   31 +++++++++++++++++++++-----
 6 files changed, 77 insertions(+), 23 deletions(-)

This patch adds support for subsys.use_id in module-loadable subsystems.

From: Ben Blum <[email protected]>

---
 kernel/cgroup.c |   23 +++++++++++++++++------
 1 files changed, 17 insertions(+), 6 deletions(-)
kernel/cgroup.c +17 -6
index cc2e1f6..5af59eb 100644
@@ -240,7 +240,8 @@ struct cg_cgroup_link {
 static struct css_set init_css_set;
 static struct cg_cgroup_link init_css_set_link;
 
-static int cgroup_subsys_init_idr(struct cgroup_subsys *ss);
+static int cgroup_init_idr(struct cgroup_subsys *ss,
+			   struct cgroup_subsys_state *css);
 
 /* css_set_lock protects the list of css_set objects, and the
  * chain of tasks off each css_set.  Nests outside task->alloc_lock
@@ -3390,6 +3391,16 @@ int __init_or_module cgroup_load_subsys(struct cgroup_subsys *ss)
 		mutex_unlock(&cgroup_mutex);
 		return PTR_ERR(css);
 	}
+	/* call init_idr here because it has a failure case */
+	if (ss->use_id) {
+		int ret = cgroup_init_idr(ss, css);
+		if (ret) {
+			ss->destroy(ss, dummytop);
+			subsys[i] = NULL;
+			mutex_unlock(&cgroup_mutex);
+			return ret;
+		}
+	}
 
 	list_add(&ss->sibling, &rootnode.subsys_list);
 	ss->root = &rootnode;
@@ -3484,7 +3495,8 @@ void cgroup_unload_subsys(struct cgroup_subsys *ss)
 	/*
 	 * remove subsystem's css from the dummytop and free it - need to free
 	 * before marking as null because ss->destroy needs the cgrp->subsys
-	 * pointer to find their state.
+	 * pointer to find their state. note that this also takes care of
+	 * freeing the css_id.
 	 */
 	ss->destroy(ss, dummytop);
 	dummytop->subsys[ss->subsys_id] = NULL;
@@ -3563,7 +3575,7 @@ int __init cgroup_init(void)
 		if (!ss->early_init)
 			cgroup_init_subsys(ss);
 		if (ss->use_id)
-			cgroup_subsys_init_idr(ss);
+			cgroup_init_idr(ss, init_css_set.subsys[ss->subsys_id]);
 	}
 
 	/* Add init_css_set to the hash table */
@@ -4231,15 +4243,14 @@ err_out:
 
 }
 
-static int __init cgroup_subsys_init_idr(struct cgroup_subsys *ss)
+static int __init_or_module cgroup_init_idr(struct cgroup_subsys *ss,
+					    struct cgroup_subsys_state *rootcss)
 {
 	struct css_id *newid;
-	struct cgroup_subsys_state *rootcss;
 
 	spin_lock_init(&ss->id_lock);
 	idr_init(&ss->idr);
 
-	rootcss = init_css_set.subsys[ss->subsys_id];
 	newid = get_new_cssid(ss, 0);
 	if (IS_ERR(newid))
 		return PTR_ERR(newid);

_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

#36 lore raw

RFC PATCH 2/2 cgroups: blkio subsystem as module

In reply to a message not in the archive

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

---
 block/Kconfig             |    2 +-
 block/Kconfig.iosched     |    2 +-
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++----------
 block/blk-cgroup.h        |   10 +++++++-
 include/linux/iocontext.h |    2 +-
 kernel/cgroup.c           |    8 +++++++
 6 files changed, 60 insertions(+), 17 deletions(-)
block/Kconfig +1 -1
index e20fbde..62a5921 100644
@@ -78,7 +78,7 @@ config BLK_DEV_INTEGRITY
 	Protection.  If in doubt, say N.
 
 config BLK_CGROUP
-	bool
+	tristate
 	depends on CGROUPS
 	default n
 	---help---
block/Kconfig.iosched +1 -1
index b71abfb..fc71cf0 100644
@@ -23,6 +23,7 @@ config IOSCHED_DEADLINE
 
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
+	select BLK_CGROUP if CFQ_GROUP_IOSCHED
 	default y
 	---help---
 	  The CFQ I/O scheduler tries to distribute bandwidth equally
@@ -35,7 +36,6 @@ config IOSCHED_CFQ
 config CFQ_GROUP_IOSCHED
 	bool "CFQ Group Scheduling support"
 	depends on IOSCHED_CFQ && CGROUPS
-	select BLK_CGROUP
 	default n
 	---help---
 	  Enable group IO scheduling in CFQ.
block/blk-cgroup.c +41 -12

74 lines not on this page · raw message

block/blk-cgroup.h +8 -2
index 4d316df..57648c6 100644
@@ -15,7 +15,13 @@
 
 #include <linux/cgroup.h>
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
+
+#ifndef CONFIG_BLK_CGROUP
+/* When blk-cgroup is a module, its subsys_id isn't a compile-time constant */
+extern struct cgroup_subsys blkio_subsys;
+#define blkio_subsys_id blkio_subsys.subsys_id
+#endif
 
 struct blkio_cgroup {
 	struct cgroup_subsys_state css;
@@ -94,7 +100,7 @@ static inline void blkiocg_update_blkio_group_dequeue_stats(
 			struct blkio_group *blkg, unsigned long dequeue) {}
 #endif
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 extern struct blkio_cgroup blkio_root_cgroup;
 extern struct blkio_cgroup *cgroup_to_blkio_cgroup(struct cgroup *cgroup);
 extern void blkiocg_add_blkio_group(struct blkio_cgroup *blkcg,
include/linux/iocontext.h +1 -1
index a632359..b9f109d 100644
@@ -68,7 +68,7 @@ struct io_context {
 	unsigned short ioprio;
 	unsigned short ioprio_changed;
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 	unsigned short cgroup_changed;
 #endif
 
kernel/cgroup.c +8 -0
index 5af59eb..391ff41 100644
@@ -690,6 +690,7 @@ void cgroup_lock(void)
 {
 	mutex_lock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_lock);
 
 /**
  * cgroup_unlock - release lock on cgroup changes
@@ -700,6 +701,7 @@ void cgroup_unlock(void)
 {
 	mutex_unlock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_unlock);
 
 /*
  * A couple of forward declarations required, due to cyclic reference loop:
@@ -1762,6 +1764,7 @@ bool cgroup_lock_live_group(struct cgroup *cgrp)
 	}
 	return true;
 }
+EXPORT_SYMBOL_GPL(cgroup_lock_live_group);
 
 static int cgroup_release_agent_write(struct cgroup *cgrp, struct cftype *cft,
 				      const char *buffer)
@@ -4034,6 +4037,7 @@ void __css_put(struct cgroup_subsys_state *css)
 	rcu_read_unlock();
 	WARN_ON_ONCE(val < 1);
 }
+EXPORT_SYMBOL_GPL(__css_put);
 
 /*
  * Notify userspace when a cgroup is released, by running the
@@ -4149,6 +4153,7 @@ unsigned short css_id(struct cgroup_subsys_state *css)
 		return cssid->id;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_id);
 
 unsigned short css_depth(struct cgroup_subsys_state *css)
 {
@@ -4158,6 +4163,7 @@ unsigned short css_depth(struct cgroup_subsys_state *css)
 		return cssid->depth;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_depth);
 
 bool css_is_ancestor(struct cgroup_subsys_state *child,
 		    const struct cgroup_subsys_state *root)
@@ -4194,6 +4200,7 @@ void free_css_id(struct cgroup_subsys *ss, struct cgroup_subsys_state *css)
 	spin_unlock(&ss->id_lock);
 	call_rcu(&id->rcu_head, __free_css_id_cb);
 }
+EXPORT_SYMBOL_GPL(free_css_id);
 
 /*
  * This is called by init or create(). Then, calls to this function are
@@ -4310,6 +4317,7 @@ struct cgroup_subsys_state *css_lookup(struct cgroup_subsys *ss, int id)
 
 	return rcu_dereference(cssid->css);
 }
+EXPORT_SYMBOL_GPL(css_lookup);
 
 /**
  * css_get_next - lookup next cgroup under specified hierarchy.
author
#37 lore raw

RFC PATCH 1/2 cgroups: modular subsystems support for use_id

In reply to Ben Blum
Quoted 36 lines · Ben Blum
Quoted 23 lines · Li Zefan
Quoted 22 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Wed, 6 Jan 2010 20:26:06 -0500 Ben Blum <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 31 Dec 2009 00:10:50 -0500 Ben Blum <[email protected]> wrote:

This patch series implements support for building, loading, and unloading subsystems as modules, both within and outside the kernel source tree. It provides an interface cgroup_load_subsys() and cgroup_unload_subsys() which modular subsystems can use to register and depart during runtime. The net_cls classifier subsystem serves as the example for a subsystem which can be converted into a module using these changes.

Quoted 2 lines

What is the value in this? What are the usage scenarios? Why does the benefit of this change exceed the cost/risk/etc of merging it?

As discussed in the first posting of these patches, this provides the ability for arbitrary subsystems to be used with cgroups.. cls_cgroup would have already been a module except for a lack of support from cgroups, and the change also allows other module-loadable classifiers to add subsystems of their own.

Quoted 4 lines
Quoted 2 lines

Hmm, do you have your own module in plan ?

Maybe the new blkio_cgroup can also be made module-able.

Ok, the following two patches make this happen (or at least pretend to well enough to fool me). The first one adds use_id initialization in cgroup_load_subsys, and the second rearranges config options and some code as appropriate in block/ and adds EXPORT_SYMBOLs in cgroup.c.

-- bblum

Quoted 10 lines
---
 block/Kconfig             |    2 -
 block/Kconfig.iosched     |    2 -
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++-----------
 block/blk-cgroup.h        |   10 ++++++--
 include/linux/iocontext.h |    2 -
 kernel/cgroup.c           |   31 +++++++++++++++++++++-----
 6 files changed, 77 insertions(+), 23 deletions(-)

This patch adds support for subsys.use_id in module-loadable subsystems.

From: Ben Blum <[email protected]>

---
 kernel/cgroup.c |   23 +++++++++++++++++------
 1 files changed, 17 insertions(+), 6 deletions(-)
kernel/cgroup.c +17 -6
index cc2e1f6..5af59eb 100644
@@ -240,7 +240,8 @@ struct cg_cgroup_link {
 static struct css_set init_css_set;
 static struct cg_cgroup_link init_css_set_link;
 
-static int cgroup_subsys_init_idr(struct cgroup_subsys *ss);
+static int cgroup_init_idr(struct cgroup_subsys *ss,
+			   struct cgroup_subsys_state *css);
 
 /* css_set_lock protects the list of css_set objects, and the
  * chain of tasks off each css_set.  Nests outside task->alloc_lock
@@ -3390,6 +3391,16 @@ int __init_or_module cgroup_load_subsys(struct cgroup_subsys *ss)
 		mutex_unlock(&cgroup_mutex);
 		return PTR_ERR(css);
 	}
+	/* call init_idr here because it has a failure case */
+	if (ss->use_id) {
+		int ret = cgroup_init_idr(ss, css);
+		if (ret) {
+			ss->destroy(ss, dummytop);
+			subsys[i] = NULL;
+			mutex_unlock(&cgroup_mutex);
+			return ret;
+		}
+	}
 
 	list_add(&ss->sibling, &rootnode.subsys_list);
 	ss->root = &rootnode;
@@ -3484,7 +3495,8 @@ void cgroup_unload_subsys(struct cgroup_subsys *ss)
 	/*
 	 * remove subsystem's css from the dummytop and free it - need to free
 	 * before marking as null because ss->destroy needs the cgrp->subsys
-	 * pointer to find their state.
+	 * pointer to find their state. note that this also takes care of
+	 * freeing the css_id.
 	 */
 	ss->destroy(ss, dummytop);
 	dummytop->subsys[ss->subsys_id] = NULL;
@@ -3563,7 +3575,7 @@ int __init cgroup_init(void)
 		if (!ss->early_init)
 			cgroup_init_subsys(ss);
 		if (ss->use_id)
-			cgroup_subsys_init_idr(ss);
+			cgroup_init_idr(ss, init_css_set.subsys[ss->subsys_id]);
 	}
 
 	/* Add init_css_set to the hash table */
@@ -4231,15 +4243,14 @@ err_out:
 
 }
 
-static int __init cgroup_subsys_init_idr(struct cgroup_subsys *ss)
+static int __init_or_module cgroup_init_idr(struct cgroup_subsys *ss,
+					    struct cgroup_subsys_state *rootcss)
 {
 	struct css_id *newid;
-	struct cgroup_subsys_state *rootcss;
 
 	spin_lock_init(&ss->id_lock);
 	idr_init(&ss->idr);
 
-	rootcss = init_css_set.subsys[ss->subsys_id];
 	newid = get_new_cssid(ss, 0);
 	if (IS_ERR(newid))
 		return PTR_ERR(newid);
author
#38 lore raw

RFC PATCH 2/2 cgroups: blkio subsystem as module

In reply to Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

---
 block/Kconfig             |    2 +-
 block/Kconfig.iosched     |    2 +-
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++----------
 block/blk-cgroup.h        |   10 +++++++-
 include/linux/iocontext.h |    2 +-
 kernel/cgroup.c           |    8 +++++++
 6 files changed, 60 insertions(+), 17 deletions(-)
block/Kconfig +1 -1
index e20fbde..62a5921 100644
@@ -78,7 +78,7 @@ config BLK_DEV_INTEGRITY
 	Protection.  If in doubt, say N.
 
 config BLK_CGROUP
-	bool
+	tristate
 	depends on CGROUPS
 	default n
 	---help---
block/Kconfig.iosched +1 -1
index b71abfb..fc71cf0 100644
@@ -23,6 +23,7 @@ config IOSCHED_DEADLINE
 
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
+	select BLK_CGROUP if CFQ_GROUP_IOSCHED
 	default y
 	---help---
 	  The CFQ I/O scheduler tries to distribute bandwidth equally
@@ -35,7 +36,6 @@ config IOSCHED_CFQ
 config CFQ_GROUP_IOSCHED
 	bool "CFQ Group Scheduling support"
 	depends on IOSCHED_CFQ && CGROUPS
-	select BLK_CGROUP
 	default n
 	---help---
 	  Enable group IO scheduling in CFQ.
block/blk-cgroup.c +41 -12

74 lines not on this page · raw message

block/blk-cgroup.h +8 -2
index 4d316df..57648c6 100644
@@ -15,7 +15,13 @@
 
 #include <linux/cgroup.h>
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
+
+#ifndef CONFIG_BLK_CGROUP
+/* When blk-cgroup is a module, its subsys_id isn't a compile-time constant */
+extern struct cgroup_subsys blkio_subsys;
+#define blkio_subsys_id blkio_subsys.subsys_id
+#endif
 
 struct blkio_cgroup {
 	struct cgroup_subsys_state css;
@@ -94,7 +100,7 @@ static inline void blkiocg_update_blkio_group_dequeue_stats(
 			struct blkio_group *blkg, unsigned long dequeue) {}
 #endif
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 extern struct blkio_cgroup blkio_root_cgroup;
 extern struct blkio_cgroup *cgroup_to_blkio_cgroup(struct cgroup *cgroup);
 extern void blkiocg_add_blkio_group(struct blkio_cgroup *blkcg,
include/linux/iocontext.h +1 -1
index a632359..b9f109d 100644
@@ -68,7 +68,7 @@ struct io_context {
 	unsigned short ioprio;
 	unsigned short ioprio_changed;
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 	unsigned short cgroup_changed;
 #endif
 
kernel/cgroup.c +8 -0
index 5af59eb..391ff41 100644
@@ -690,6 +690,7 @@ void cgroup_lock(void)
 {
 	mutex_lock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_lock);
 
 /**
  * cgroup_unlock - release lock on cgroup changes
@@ -700,6 +701,7 @@ void cgroup_unlock(void)
 {
 	mutex_unlock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_unlock);
 
 /*
  * A couple of forward declarations required, due to cyclic reference loop:
@@ -1762,6 +1764,7 @@ bool cgroup_lock_live_group(struct cgroup *cgrp)
 	}
 	return true;
 }
+EXPORT_SYMBOL_GPL(cgroup_lock_live_group);
 
 static int cgroup_release_agent_write(struct cgroup *cgrp, struct cftype *cft,
 				      const char *buffer)
@@ -4034,6 +4037,7 @@ void __css_put(struct cgroup_subsys_state *css)
 	rcu_read_unlock();
 	WARN_ON_ONCE(val < 1);
 }
+EXPORT_SYMBOL_GPL(__css_put);
 
 /*
  * Notify userspace when a cgroup is released, by running the
@@ -4149,6 +4153,7 @@ unsigned short css_id(struct cgroup_subsys_state *css)
 		return cssid->id;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_id);
 
 unsigned short css_depth(struct cgroup_subsys_state *css)
 {
@@ -4158,6 +4163,7 @@ unsigned short css_depth(struct cgroup_subsys_state *css)
 		return cssid->depth;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_depth);
 
 bool css_is_ancestor(struct cgroup_subsys_state *child,
 		    const struct cgroup_subsys_state *root)
@@ -4194,6 +4200,7 @@ void free_css_id(struct cgroup_subsys *ss, struct cgroup_subsys_state *css)
 	spin_unlock(&ss->id_lock);
 	call_rcu(&id->rcu_head, __free_css_id_cb);
 }
+EXPORT_SYMBOL_GPL(free_css_id);
 
 /*
  * This is called by init or create(). Then, calls to this function are
@@ -4310,6 +4317,7 @@ struct cgroup_subsys_state *css_lookup(struct cgroup_subsys *ss, int id)
 
 	return rcu_dereference(cssid->css);
 }
+EXPORT_SYMBOL_GPL(css_lookup);
 
 /**
  * css_get_next - lookup next cgroup under specified hierarchy.
#39 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

Quoted 9 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Hi Ben,

I will give this patch a try.

So from blk-cgroup perspective, the advantage of allowing it as module will be that we can save some memory if we are not using the controller?

Quoted 65 lines
---
 block/Kconfig             |    2 +-
 block/Kconfig.iosched     |    2 +-
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++----------
 block/blk-cgroup.h        |   10 +++++++-
 include/linux/iocontext.h |    2 +-
 kernel/cgroup.c           |    8 +++++++
 6 files changed, 60 insertions(+), 17 deletions(-)
block/Kconfig +1 -1
index e20fbde..62a5921 100644
@@ -78,7 +78,7 @@ config BLK_DEV_INTEGRITY
 	Protection.  If in doubt, say N.
 
 config BLK_CGROUP
-	bool
+	tristate
 	depends on CGROUPS
 	default n
 	---help---
block/Kconfig.iosched +1 -1
index b71abfb..fc71cf0 100644
@@ -23,6 +23,7 @@ config IOSCHED_DEADLINE
 
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
+	select BLK_CGROUP if CFQ_GROUP_IOSCHED
 	default y
 	---help---
 	  The CFQ I/O scheduler tries to distribute bandwidth equally
@@ -35,7 +36,6 @@ config IOSCHED_CFQ
 config CFQ_GROUP_IOSCHED
 	bool "CFQ Group Scheduling support"
 	depends on IOSCHED_CFQ && CGROUPS
-	select BLK_CGROUP
 	default n
 	---help---
 	  Enable group IO scheduling in CFQ.
block/blk-cgroup.c +14 -0
index 1fa2654..6c73380 100644
@@ -23,6 +23,31 @@ static LIST_HEAD(blkio_list);
 struct blkio_cgroup blkio_root_cgroup = { .weight = 2*BLKIO_WEIGHT_DEFAULT };
 EXPORT_SYMBOL_GPL(blkio_root_cgroup);
 
+static struct cgroup_subsys_state *blkiocg_create(struct cgroup_subsys *,
+						  struct cgroup *);
+static int blkiocg_can_attach(struct cgroup_subsys *, struct cgroup *,
+			      struct task_struct *, bool);
+static void blkiocg_attach(struct cgroup_subsys *, struct cgroup *,
+			   struct cgroup *, struct task_struct *, bool);
+static void blkiocg_destroy(struct cgroup_subsys *, struct cgroup *);
+static int blkiocg_populate(struct cgroup_subsys *, struct cgroup *);
+
+struct cgroup_subsys blkio_subsys = {
+	.name = "blkio",
+	.create = blkiocg_create,
+	.can_attach = blkiocg_can_attach,
+	.attach = blkiocg_attach,

block/blk-cgroup.c static LIST_HEAD(blkio_list); · line 49

Quoted 10 lines
block/blk-cgroup.c +10 -0
+	.destroy = blkiocg_destroy,
+	.populate = blkiocg_populate,
+#ifdef CONFIG_BLK_CGROUP
+	/* note: blkio_subsys_id is otherwise defined in blk-cgroup.h */
+	.subsys_id = blkio_subsys_id,
+#endif
+	.use_id = 1,
+	.module = THIS_MODULE,
+};
+EXPORT_SYMBOL_GPL(blkio_subsys);

Why are you moving blkio_subsys declaration up in the file. That way now you have to declare all the functions before you can instanciate blkio_subsys?

Quoted 57 lines
block/blk-cgroup.c +17 -12
+
 bool blkiocg_css_tryget(struct blkio_cgroup *blkcg)
 {
 	if (!css_tryget(&blkcg->css))
@@ -267,7 +292,8 @@ remove_entry:
 done:
 	free_css_id(&blkio_subsys, &blkcg->css);
 	rcu_read_unlock();
-	kfree(blkcg);
+	if (blkcg != &blkio_root_cgroup)
+		kfree(blkcg);
 }
 
 static struct cgroup_subsys_state *
@@ -333,17 +359,6 @@ static void blkiocg_attach(struct cgroup_subsys *subsys, struct cgroup *cgroup,
 	task_unlock(tsk);
 }
 
-struct cgroup_subsys blkio_subsys = {
-	.name = "blkio",
-	.create = blkiocg_create,
-	.can_attach = blkiocg_can_attach,
-	.attach = blkiocg_attach,
-	.destroy = blkiocg_destroy,
-	.populate = blkiocg_populate,
-	.subsys_id = blkio_subsys_id,
-	.use_id = 1,
-};
-
 void blkio_policy_register(struct blkio_policy_type *blkiop)
 {
 	spin_lock(&blkio_list_lock);
@@ -359,3 +374,17 @@ void blkio_policy_unregister(struct blkio_policy_type *blkiop)
 	spin_unlock(&blkio_list_lock);
 }
 EXPORT_SYMBOL_GPL(blkio_policy_unregister);
+
+static int __init init_cgroup_blkio(void)
+{
+	return cgroup_load_subsys(&blkio_subsys);
+}
+
+static void __exit exit_cgroup_blkio(void)
+{
+	cgroup_unload_subsys(&blkio_subsys);
+}
+
+module_init(init_cgroup_blkio);
+module_exit(exit_cgroup_blkio);
+MODULE_LICENSE("GPL");
block/blk-cgroup.h +0 -0
index 4d316df..57648c6 100644
@@ -15,7 +15,13 @@
 
 #include <linux/cgroup.h>

block/blk-cgroup.h line 25

Quoted 10 lines
block/blk-cgroup.h +7 -1
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
+
+#ifndef CONFIG_BLK_CGROUP
+/* When blk-cgroup is a module, its subsys_id isn't a compile-time constant */
+extern struct cgroup_subsys blkio_subsys;
+#define blkio_subsys_id blkio_subsys.subsys_id
+#endif
 

Could above if conditions be simplified to just check for BLK_CGROUP_MODULE.

#ifdef CONFIG_BLK_CGROUP_MODULE
extern struct cgroup_subsys blkio_subsys;
#define blkio_subsys_id blkio_subsys.subsys_id
#endif

Thanks
Vivek

Quoted 92 lines
block/blk-cgroup.h +1 -1
 struct blkio_cgroup {
 	struct cgroup_subsys_state css;
@@ -94,7 +100,7 @@ static inline void blkiocg_update_blkio_group_dequeue_stats(
 			struct blkio_group *blkg, unsigned long dequeue) {}
 #endif
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 extern struct blkio_cgroup blkio_root_cgroup;
 extern struct blkio_cgroup *cgroup_to_blkio_cgroup(struct cgroup *cgroup);
 extern void blkiocg_add_blkio_group(struct blkio_cgroup *blkcg,
include/linux/iocontext.h +1 -1
index a632359..b9f109d 100644
@@ -68,7 +68,7 @@ struct io_context {
 	unsigned short ioprio;
 	unsigned short ioprio_changed;
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 	unsigned short cgroup_changed;
 #endif
 
kernel/cgroup.c +8 -0
index 5af59eb..391ff41 100644
@@ -690,6 +690,7 @@ void cgroup_lock(void)
 {
 	mutex_lock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_lock);
 
 /**
  * cgroup_unlock - release lock on cgroup changes
@@ -700,6 +701,7 @@ void cgroup_unlock(void)
 {
 	mutex_unlock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_unlock);
 
 /*
  * A couple of forward declarations required, due to cyclic reference loop:
@@ -1762,6 +1764,7 @@ bool cgroup_lock_live_group(struct cgroup *cgrp)
 	}
 	return true;
 }
+EXPORT_SYMBOL_GPL(cgroup_lock_live_group);
 
 static int cgroup_release_agent_write(struct cgroup *cgrp, struct cftype *cft,
 				      const char *buffer)
@@ -4034,6 +4037,7 @@ void __css_put(struct cgroup_subsys_state *css)
 	rcu_read_unlock();
 	WARN_ON_ONCE(val < 1);
 }
+EXPORT_SYMBOL_GPL(__css_put);
 
 /*
  * Notify userspace when a cgroup is released, by running the
@@ -4149,6 +4153,7 @@ unsigned short css_id(struct cgroup_subsys_state *css)
 		return cssid->id;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_id);
 
 unsigned short css_depth(struct cgroup_subsys_state *css)
 {
@@ -4158,6 +4163,7 @@ unsigned short css_depth(struct cgroup_subsys_state *css)
 		return cssid->depth;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_depth);
 
 bool css_is_ancestor(struct cgroup_subsys_state *child,
 		    const struct cgroup_subsys_state *root)
@@ -4194,6 +4200,7 @@ void free_css_id(struct cgroup_subsys *ss, struct cgroup_subsys_state *css)
 	spin_unlock(&ss->id_lock);
 	call_rcu(&id->rcu_head, __free_css_id_cb);
 }
+EXPORT_SYMBOL_GPL(free_css_id);
 
 /*
  * This is called by init or create(). Then, calls to this function are
@@ -4310,6 +4317,7 @@ struct cgroup_subsys_state *css_lookup(struct cgroup_subsys *ss, int id)
 
 	return rcu_dereference(cssid->css);
 }
+EXPORT_SYMBOL_GPL(css_lookup);
 
 /**
  * css_get_next - lookup next cgroup under specified hierarchy.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#41 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

Quoted 11 lines · On Fri, 8 Jan 2010 10:10:38 -0500 Vivek Goyal…
Quoted 9 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Quoted 7 lines

Hi Ben,

I will give this patch a try.

So from blk-cgroup perspective, the advantage of allowing it as module will be that we can save some memory if we are not using the controller?

Is "moduled" blkio cgroup safe after page-tracking by page_cgroup is introduced ?

Thanks,
-Kame

#42 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

In reply to Vivek Goyal
Quoted 11 lines · On Fri, 8 Jan 2010 10:10:38 -0500 Vivek Goyal <[email protected]> wrote:
Quoted 9 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Quoted 7 lines

Hi Ben,

I will give this patch a try.

So from blk-cgroup perspective, the advantage of allowing it as module will be that we can save some memory if we are not using the controller?

Is "moduled" blkio cgroup safe after page-tracking by page_cgroup is introduced ?

Thanks,
-Kame

A message that is not in the archive (sent off-list or not yet received): 20100112092106.9ac4e535.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org

#44 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

Quoted 16 lines · KAMEZAWA Hiroyuki
Quoted 13 lines · On Fri, 8 Jan 2010 10:10:38 -0500 Vivek Goyal…
Quoted 9 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Hi Ben,

Quoted 8 lines
Quoted 5 lines

I will give this patch a try.

So from blk-cgroup perspective, the advantage of allowing it as module will be that we can save some memory if we are not using the controller?

Is "moduled" blkio cgroup safe after page-tracking by page_cgroup is introduced ?

My guess is it won't be, unless we start exposing page_cgroup API and then make the module depend on memcg.

Balbir

#45 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

In reply to KAMEZAWA Hiroyuki
Quoted 16 lines · KAMEZAWA Hiroyuki
Quoted 13 lines · On Fri, 8 Jan 2010 10:10:38 -0500 Vivek Goyal <[email protected]> wrote:
Quoted 9 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Hi Ben,

Quoted 8 lines
Quoted 5 lines

I will give this patch a try.

So from blk-cgroup perspective, the advantage of allowing it as module will be that we can save some memory if we are not using the controller?

Is "moduled" blkio cgroup safe after page-tracking by page_cgroup is introduced ?

My guess is it won't be, unless we start exposing page_cgroup API and then make the module depend on memcg.

Balbir

A message that is not in the archive (sent off-list or not yet received): 661de9471001140132o23f44a4fjb1849d865f96d19a-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#47 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

Quoted 23 lines · Balbir Singh
Quoted 21 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Fri, 8 Jan 2010 10:10:38 -0500 Vivek Goyal…
Quoted 9 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Hi Ben,

I will give this patch a try.

So from blk-cgroup perspective, the advantage of allowing it as module will be that we can save some memory if we are not using the controller?

Quoted 6 lines
Quoted 3 lines

Is "moduled" blkio cgroup safe after page-tracking by page_cgroup is introduced ?

My guess is it won't be, unless we start exposing page_cgroup API and then make the module depend on memcg.

I think I agree. When we introduce page_cgroup based page tracking, either we need to export page_cgroup API or we can force blkio controller to compile as in-kernel if user selects the CONFIG_PAGE_TRACKING option.

So as of now, I can't think why we should not we allow compiling blkio as module as long as core cgroup functionality supports it safely.

Thanks
Vivek

#48 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

In reply to Balbir Singh
Quoted 23 lines · Balbir Singh
Quoted 21 lines · KAMEZAWA Hiroyuki
Quoted 18 lines · On Fri, 8 Jan 2010 10:10:38 -0500 Vivek Goyal <[email protected]> wrote:
Quoted 9 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Hi Ben,

I will give this patch a try.

So from blk-cgroup perspective, the advantage of allowing it as module will be that we can save some memory if we are not using the controller?

Quoted 6 lines
Quoted 3 lines

Is "moduled" blkio cgroup safe after page-tracking by page_cgroup is introduced ?

My guess is it won't be, unless we start exposing page_cgroup API and then make the module depend on memcg.

I think I agree. When we introduce page_cgroup based page tracking, either we need to export page_cgroup API or we can force blkio controller to compile as in-kernel if user selects the CONFIG_PAGE_TRACKING option.

So as of now, I can't think why we should not we allow compiling blkio as module as long as core cgroup functionality supports it safely.

Thanks
Vivek

#49 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

In reply to Ben Blum
Quoted 10 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Two quick observations with testing.

You need to EXPORT cgroup_path.

Second, after loading the module, I mounted the blkio controller. But creating a cgroup directory crashed.

Vivek

BUG: unable to handle kernel NULL pointer dereference at 0000000000000020
IP: [<ffffffff8106bcb4>] cgroup_mkdir+0x17e/0x350
PGD 1a7fe0067 PUD 1a0f19067 PMD 0 
Oops: 0000 [#1] SMP 
last sysfs file: /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
CPU 3 
Pid: 3984, comm: mkdir Not tainted 2.6.33-rc3-cgroup-module #2 /ProLiant DL380 G5
RIP: 0010:[<ffffffff8106bcb4>]  [<ffffffff8106bcb4>] cgroup_mkdir+0x17e/0x350
RSP: 0018:ffff8801a0f17e38  EFLAGS: 00010203
RAX: ffff8801a8cbde00 RBX: ffffffffa02cc8e0 RCX: ffff8801a8cbde00
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffff8801a770c000 R08: 0000000000000040 R09: ffff8801a0f17d48
R10: ffff8801a0f17ec8 R11: ffffffff8113efb4 R12: 0000000000000001
R13: ffff8801a0f22000 R14: ffff8801948aa980 R15: ffff8801a0f22030
FS:  00007ff9ecf5a710(0000) GS:ffff8800282c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000020 CR3: 00000001a257e000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process mkdir (pid: 3984, threadinfo ffff8801a0f16000, task ffff8801a8e2c040)
Stack:
 0000000000000003 000001ed948a8db0 ffff8801a0e1e800 0000000000000000
<0> ffff8801a7745940 ffff8801948a8db0 ffff8801948aa980 0000000000000000
<0> 00000000000001ed 00007fff661cb6eb 0000000000000000 ffffffff810d837e
Call Trace:
 [<ffffffff810d837e>] ? vfs_mkdir+0xd5/0x164
 [<ffffffff810da743>] ? sys_mkdirat+0x85/0xd1
 [<ffffffff810773be>] ? audit_syscall_entry+0x1b9/0x1e4
 [<ffffffff8100286b>] ? system_call_fastpath+0x16/0x1b
Code: 4c 24 18 e8 82 f3 ff ff 31 f6 48 3d 00 f0 ff ff 48 89 c1 76 20 85 c0 74 35 45 31 e4 e9 5a 01 00 00 48 8b 7c 24 18 48 63 d6 ff c6 <66> 8b 44 57 20 66 89 44 51 20 44 39 e6 7c e7 8b 51 08 49 63 c4 
RIP  [<ffffffff8106bcb4>] cgroup_mkdir+0x17e/0x350
 RSP <ffff8801a0f17e38>
CR2: 0000000000000020
---[ end trace 09377bc5e5c2b563 ]---
Quoted 233 lines
---
 block/Kconfig             |    2 +-
 block/Kconfig.iosched     |    2 +-
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++----------
 block/blk-cgroup.h        |   10 +++++++-
 include/linux/iocontext.h |    2 +-
 kernel/cgroup.c           |    8 +++++++
 6 files changed, 60 insertions(+), 17 deletions(-)
block/Kconfig +1 -1
index e20fbde..62a5921 100644
@@ -78,7 +78,7 @@ config BLK_DEV_INTEGRITY
 	Protection.  If in doubt, say N.
 
 config BLK_CGROUP
-	bool
+	tristate
 	depends on CGROUPS
 	default n
 	---help---
block/Kconfig.iosched +1 -1
index b71abfb..fc71cf0 100644
@@ -23,6 +23,7 @@ config IOSCHED_DEADLINE
 
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
+	select BLK_CGROUP if CFQ_GROUP_IOSCHED
 	default y
 	---help---
 	  The CFQ I/O scheduler tries to distribute bandwidth equally
@@ -35,7 +36,6 @@ config IOSCHED_CFQ
 config CFQ_GROUP_IOSCHED
 	bool "CFQ Group Scheduling support"
 	depends on IOSCHED_CFQ && CGROUPS
-	select BLK_CGROUP
 	default n
 	---help---
 	  Enable group IO scheduling in CFQ.
block/blk-cgroup.c +41 -12
index 1fa2654..6c73380 100644
@@ -23,6 +23,31 @@ static LIST_HEAD(blkio_list);
 struct blkio_cgroup blkio_root_cgroup = { .weight = 2*BLKIO_WEIGHT_DEFAULT };
 EXPORT_SYMBOL_GPL(blkio_root_cgroup);
 
+static struct cgroup_subsys_state *blkiocg_create(struct cgroup_subsys *,
+						  struct cgroup *);
+static int blkiocg_can_attach(struct cgroup_subsys *, struct cgroup *,
+			      struct task_struct *, bool);
+static void blkiocg_attach(struct cgroup_subsys *, struct cgroup *,
+			   struct cgroup *, struct task_struct *, bool);
+static void blkiocg_destroy(struct cgroup_subsys *, struct cgroup *);
+static int blkiocg_populate(struct cgroup_subsys *, struct cgroup *);
+
+struct cgroup_subsys blkio_subsys = {
+	.name = "blkio",
+	.create = blkiocg_create,
+	.can_attach = blkiocg_can_attach,
+	.attach = blkiocg_attach,
+	.destroy = blkiocg_destroy,
+	.populate = blkiocg_populate,
+#ifdef CONFIG_BLK_CGROUP
+	/* note: blkio_subsys_id is otherwise defined in blk-cgroup.h */
+	.subsys_id = blkio_subsys_id,
+#endif
+	.use_id = 1,
+	.module = THIS_MODULE,
+};
+EXPORT_SYMBOL_GPL(blkio_subsys);
+
 bool blkiocg_css_tryget(struct blkio_cgroup *blkcg)
 {
 	if (!css_tryget(&blkcg->css))
@@ -267,7 +292,8 @@ remove_entry:
 done:
 	free_css_id(&blkio_subsys, &blkcg->css);
 	rcu_read_unlock();
-	kfree(blkcg);
+	if (blkcg != &blkio_root_cgroup)
+		kfree(blkcg);
 }
 
 static struct cgroup_subsys_state *
@@ -333,17 +359,6 @@ static void blkiocg_attach(struct cgroup_subsys *subsys, struct cgroup *cgroup,
 	task_unlock(tsk);
 }
 
-struct cgroup_subsys blkio_subsys = {
-	.name = "blkio",
-	.create = blkiocg_create,
-	.can_attach = blkiocg_can_attach,
-	.attach = blkiocg_attach,
-	.destroy = blkiocg_destroy,
-	.populate = blkiocg_populate,
-	.subsys_id = blkio_subsys_id,
-	.use_id = 1,
-};
-
 void blkio_policy_register(struct blkio_policy_type *blkiop)
 {
 	spin_lock(&blkio_list_lock);
@@ -359,3 +374,17 @@ void blkio_policy_unregister(struct blkio_policy_type *blkiop)
 	spin_unlock(&blkio_list_lock);
 }
 EXPORT_SYMBOL_GPL(blkio_policy_unregister);
+
+static int __init init_cgroup_blkio(void)
+{
+	return cgroup_load_subsys(&blkio_subsys);
+}
+
+static void __exit exit_cgroup_blkio(void)
+{
+	cgroup_unload_subsys(&blkio_subsys);
+}
+
+module_init(init_cgroup_blkio);
+module_exit(exit_cgroup_blkio);
+MODULE_LICENSE("GPL");
block/blk-cgroup.h +8 -2
index 4d316df..57648c6 100644
@@ -15,7 +15,13 @@
 
 #include <linux/cgroup.h>
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
+
+#ifndef CONFIG_BLK_CGROUP
+/* When blk-cgroup is a module, its subsys_id isn't a compile-time constant */
+extern struct cgroup_subsys blkio_subsys;
+#define blkio_subsys_id blkio_subsys.subsys_id
+#endif
 
 struct blkio_cgroup {
 	struct cgroup_subsys_state css;
@@ -94,7 +100,7 @@ static inline void blkiocg_update_blkio_group_dequeue_stats(
 			struct blkio_group *blkg, unsigned long dequeue) {}
 #endif
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 extern struct blkio_cgroup blkio_root_cgroup;
 extern struct blkio_cgroup *cgroup_to_blkio_cgroup(struct cgroup *cgroup);
 extern void blkiocg_add_blkio_group(struct blkio_cgroup *blkcg,
include/linux/iocontext.h +1 -1
index a632359..b9f109d 100644
@@ -68,7 +68,7 @@ struct io_context {
 	unsigned short ioprio;
 	unsigned short ioprio_changed;
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 	unsigned short cgroup_changed;
 #endif
 
kernel/cgroup.c +8 -0
index 5af59eb..391ff41 100644
@@ -690,6 +690,7 @@ void cgroup_lock(void)
 {
 	mutex_lock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_lock);
 
 /**
  * cgroup_unlock - release lock on cgroup changes
@@ -700,6 +701,7 @@ void cgroup_unlock(void)
 {
 	mutex_unlock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_unlock);
 
 /*
  * A couple of forward declarations required, due to cyclic reference loop:
@@ -1762,6 +1764,7 @@ bool cgroup_lock_live_group(struct cgroup *cgrp)
 	}
 	return true;
 }
+EXPORT_SYMBOL_GPL(cgroup_lock_live_group);
 
 static int cgroup_release_agent_write(struct cgroup *cgrp, struct cftype *cft,
 				      const char *buffer)
@@ -4034,6 +4037,7 @@ void __css_put(struct cgroup_subsys_state *css)
 	rcu_read_unlock();
 	WARN_ON_ONCE(val < 1);
 }
+EXPORT_SYMBOL_GPL(__css_put);
 
 /*
  * Notify userspace when a cgroup is released, by running the
@@ -4149,6 +4153,7 @@ unsigned short css_id(struct cgroup_subsys_state *css)
 		return cssid->id;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_id);
 
 unsigned short css_depth(struct cgroup_subsys_state *css)
 {
@@ -4158,6 +4163,7 @@ unsigned short css_depth(struct cgroup_subsys_state *css)
 		return cssid->depth;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_depth);
 
 bool css_is_ancestor(struct cgroup_subsys_state *child,
 		    const struct cgroup_subsys_state *root)
@@ -4194,6 +4200,7 @@ void free_css_id(struct cgroup_subsys *ss, struct cgroup_subsys_state *css)
 	spin_unlock(&ss->id_lock);
 	call_rcu(&id->rcu_head, __free_css_id_cb);
 }
+EXPORT_SYMBOL_GPL(free_css_id);
 
 /*
  * This is called by init or create(). Then, calls to this function are
@@ -4310,6 +4317,7 @@ struct cgroup_subsys_state *css_lookup(struct cgroup_subsys *ss, int id)
 
 	return rcu_dereference(cssid->css);
 }
+EXPORT_SYMBOL_GPL(css_lookup);
 
 /**
  * css_get_next - lookup next cgroup under specified hierarchy.
author
#50 lore raw

Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

Quoted 12 lines · Vivek Goyal
Quoted 10 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

Quoted 8 lines

Two quick observations with testing.

You need to EXPORT cgroup_path.

Second, after loading the module, I mounted the blkio controller. But creating a cgroup directory crashed.

Vivek

argh, good catches on both of them. didn't test with DEBUG_CFQ_IOSCHED (for cgroup_path) or with making a sub-cgroup (for the crash); shame on me. turns out it crashed because I had init_idr before init_css_set, and init_css_set sets css->id = NULL explicitly. fixed patches forthcoming.

-- bblum

---
 block/Kconfig             |    2 -
 block/Kconfig.iosched     |    2 -
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++-----------
 block/blk-cgroup.h        |   10 ++++++--
 include/linux/iocontext.h |    2 -
 kernel/cgroup.c           |   34 ++++++++++++++++++++++++-----
 6 files changed, 80 insertions(+), 23 deletions(-)
author
#51 lore raw

PATCH v2 1/2 cgroups: modular subsystems support for use_id

This patch adds support for subsys.use_id in module-loadable subsystems.

From: Ben Blum <[email protected]>

---
 kernel/cgroup.c |   25 +++++++++++++++++++------
 1 files changed, 19 insertions(+), 6 deletions(-)
kernel/cgroup.c +19 -6
index cc2e1f6..b4ae6ef 100644
@@ -240,7 +240,8 @@ struct cg_cgroup_link {
 static struct css_set init_css_set;
 static struct cg_cgroup_link init_css_set_link;
 
-static int cgroup_subsys_init_idr(struct cgroup_subsys *ss);
+static int cgroup_init_idr(struct cgroup_subsys *ss,
+			   struct cgroup_subsys_state *css);
 
 /* css_set_lock protects the list of css_set objects, and the
  * chain of tasks off each css_set.  Nests outside task->alloc_lock
@@ -3396,6 +3397,18 @@ int __init_or_module cgroup_load_subsys(struct cgroup_subsys *ss)
 
 	/* our new subsystem will be attached to the dummy hierarchy. */
 	init_cgroup_css(css, ss, dummytop);
+	/* init_idr must be after init_cgroup_css because it sets css->id. */
+	if (ss->use_id) {
+		int ret = cgroup_init_idr(ss, css);
+		if (ret) {
+			dummytop->subsys[ss->subsys_id] = NULL;
+			ss->destroy(ss, dummytop);
+			subsys[i] = NULL;
+			mutex_unlock(&cgroup_mutex);
+			return ret;
+		}
+	}
+
 	/*
 	 * Now we need to entangle the css into the existing css_sets. unlike
 	 * in cgroup_init_subsys, there are now multiple css_sets, so each one
@@ -3484,7 +3497,8 @@ void cgroup_unload_subsys(struct cgroup_subsys *ss)
 	/*
 	 * remove subsystem's css from the dummytop and free it - need to free
 	 * before marking as null because ss->destroy needs the cgrp->subsys
-	 * pointer to find their state.
+	 * pointer to find their state. note that this also takes care of
+	 * freeing the css_id.
 	 */
 	ss->destroy(ss, dummytop);
 	dummytop->subsys[ss->subsys_id] = NULL;
@@ -3563,7 +3577,7 @@ int __init cgroup_init(void)
 		if (!ss->early_init)
 			cgroup_init_subsys(ss);
 		if (ss->use_id)
-			cgroup_subsys_init_idr(ss);
+			cgroup_init_idr(ss, init_css_set.subsys[ss->subsys_id]);
 	}
 
 	/* Add init_css_set to the hash table */
@@ -4231,15 +4245,14 @@ err_out:
 
 }
 
-static int __init cgroup_subsys_init_idr(struct cgroup_subsys *ss)
+static int __init_or_module cgroup_init_idr(struct cgroup_subsys *ss,
+					    struct cgroup_subsys_state *rootcss)
 {
 	struct css_id *newid;
-	struct cgroup_subsys_state *rootcss;
 
 	spin_lock_init(&ss->id_lock);
 	idr_init(&ss->idr);
 
-	rootcss = init_css_set.subsys[ss->subsys_id];
 	newid = get_new_cssid(ss, 0);
 	if (IS_ERR(newid))
 		return PTR_ERR(newid);
author
#52 lore raw

PATCH v2 2/2 cgroups: blkio subsystem as module

In reply to Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

---
 block/Kconfig             |    2 +-
 block/Kconfig.iosched     |    2 +-
 block/blk-cgroup.c        |   53 +++++++++++++++++++++++++++++++++++----------
 block/blk-cgroup.h        |   10 +++++++-
 include/linux/iocontext.h |    2 +-
 kernel/cgroup.c           |    9 ++++++++
 6 files changed, 61 insertions(+), 17 deletions(-)
block/Kconfig +1 -1
index e20fbde..62a5921 100644
@@ -78,7 +78,7 @@ config BLK_DEV_INTEGRITY
 	Protection.  If in doubt, say N.
 
 config BLK_CGROUP
-	bool
+	tristate
 	depends on CGROUPS
 	default n
 	---help---
block/Kconfig.iosched +1 -1
index b71abfb..fc71cf0 100644
@@ -23,6 +23,7 @@ config IOSCHED_DEADLINE
 
 config IOSCHED_CFQ
 	tristate "CFQ I/O scheduler"
+	select BLK_CGROUP if CFQ_GROUP_IOSCHED
 	default y
 	---help---
 	  The CFQ I/O scheduler tries to distribute bandwidth equally
@@ -35,7 +36,6 @@ config IOSCHED_CFQ
 config CFQ_GROUP_IOSCHED
 	bool "CFQ Group Scheduling support"
 	depends on IOSCHED_CFQ && CGROUPS
-	select BLK_CGROUP
 	default n
 	---help---
 	  Enable group IO scheduling in CFQ.
block/blk-cgroup.c +41 -12

74 lines not on this page · raw message

block/blk-cgroup.h +8 -2
index 4d316df..57648c6 100644
@@ -15,7 +15,13 @@
 
 #include <linux/cgroup.h>
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
+
+#ifndef CONFIG_BLK_CGROUP
+/* When blk-cgroup is a module, its subsys_id isn't a compile-time constant */
+extern struct cgroup_subsys blkio_subsys;
+#define blkio_subsys_id blkio_subsys.subsys_id
+#endif
 
 struct blkio_cgroup {
 	struct cgroup_subsys_state css;
@@ -94,7 +100,7 @@ static inline void blkiocg_update_blkio_group_dequeue_stats(
 			struct blkio_group *blkg, unsigned long dequeue) {}
 #endif
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 extern struct blkio_cgroup blkio_root_cgroup;
 extern struct blkio_cgroup *cgroup_to_blkio_cgroup(struct cgroup *cgroup);
 extern void blkiocg_add_blkio_group(struct blkio_cgroup *blkcg,
include/linux/iocontext.h +1 -1
index a632359..b9f109d 100644
@@ -68,7 +68,7 @@ struct io_context {
 	unsigned short ioprio;
 	unsigned short ioprio_changed;
 
-#ifdef CONFIG_BLK_CGROUP
+#if defined(CONFIG_BLK_CGROUP) || defined(CONFIG_BLK_CGROUP_MODULE)
 	unsigned short cgroup_changed;
 #endif
 
kernel/cgroup.c +9 -0
index b4ae6ef..845a2e7 100644
@@ -690,6 +690,7 @@ void cgroup_lock(void)
 {
 	mutex_lock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_lock);
 
 /**
  * cgroup_unlock - release lock on cgroup changes
@@ -700,6 +701,7 @@ void cgroup_unlock(void)
 {
 	mutex_unlock(&cgroup_mutex);
 }
+EXPORT_SYMBOL_GPL(cgroup_unlock);
 
 /*
  * A couple of forward declarations required, due to cyclic reference loop:
@@ -1622,6 +1624,7 @@ int cgroup_path(const struct cgroup *cgrp, char *buf, int buflen)
 	memmove(buf, start, buf + buflen - start);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(cgroup_path);
 
 /**
  * cgroup_attach_task - attach task 'tsk' to cgroup 'cgrp'
@@ -1762,6 +1765,7 @@ bool cgroup_lock_live_group(struct cgroup *cgrp)
 	}
 	return true;
 }
+EXPORT_SYMBOL_GPL(cgroup_lock_live_group);
 
 static int cgroup_release_agent_write(struct cgroup *cgrp, struct cftype *cft,
 				      const char *buffer)
@@ -4036,6 +4040,7 @@ void __css_put(struct cgroup_subsys_state *css)
 	rcu_read_unlock();
 	WARN_ON_ONCE(val < 1);
 }
+EXPORT_SYMBOL_GPL(__css_put);
 
 /*
  * Notify userspace when a cgroup is released, by running the
@@ -4151,6 +4156,7 @@ unsigned short css_id(struct cgroup_subsys_state *css)
 		return cssid->id;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_id);
 
 unsigned short css_depth(struct cgroup_subsys_state *css)
 {
@@ -4160,6 +4166,7 @@ unsigned short css_depth(struct cgroup_subsys_state *css)
 		return cssid->depth;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(css_depth);
 
 bool css_is_ancestor(struct cgroup_subsys_state *child,
 		    const struct cgroup_subsys_state *root)
@@ -4196,6 +4203,7 @@ void free_css_id(struct cgroup_subsys *ss, struct cgroup_subsys_state *css)
 	spin_unlock(&ss->id_lock);
 	call_rcu(&id->rcu_head, __free_css_id_cb);
 }
+EXPORT_SYMBOL_GPL(free_css_id);
 
 /*
  * This is called by init or create(). Then, calls to this function are
@@ -4312,6 +4320,7 @@ struct cgroup_subsys_state *css_lookup(struct cgroup_subsys *ss, int id)
 
 	return rcu_dereference(cssid->css);
 }
+EXPORT_SYMBOL_GPL(css_lookup);
 
 /**
  * css_get_next - lookup next cgroup under specified hierarchy.

A message that is not in the archive (sent off-list or not yet received): 20100112233709.GD5098-/[email protected]

#54 lore raw

Re: [PATCH v2 2/2] cgroups: blkio subsystem as module

Quoted 8 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

I'd like to add this for 2.6.34, but I'd like some sign-off(s) on the cgroup side of things (for patch 1/2).

Signature · 1 line
Jens Axboe
#55 lore raw

Re: [PATCH v2 2/2] cgroups: blkio subsystem as module

In reply to Ben Blum
Quoted 8 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

I'd like to add this for 2.6.34, but I'd like some sign-off(s) on the cgroup side of things (for patch 1/2).

Signature · 1 line
Jens Axboe
#56 lore raw

Re: [PATCH v2 2/2] cgroups: blkio subsystem as module

Quoted 5 lines · Jens Axboe
Quoted 4 lines · Ben Blum

Convert blk-cgroup to be buildable as a module

From: Ben Blum <[email protected]>

Quoted 8 lines
Quoted 4 lines

This patch modifies the Block I/O cgroup subsystem to be able to be built as a module. As the CFQ disk scheduler optionally depends on blk-cgroup, config options in block/Kconfig, block/Kconfig.iosched, and block/blk-cgroup.h are enhanced to support the new module dependency.

I'd like to add this for 2.6.34, but I'd like some sign-off(s) on the cgroup side of things (for patch 1/2).

Yes, it's for .34. But these 2 patches can't be applied to block-tree, because the patches that implement cgroup modular feature are in -mm, and they will go into .34.