Skip to contentSkip to conversation

cgroups: support for module-loadable subsystems

RFC PATCH In review Ben Blum 8 participants 15 messages Nov 2, 2009 – Nov 4, 2009 5 messages not in the archive

Subsystems (4)
lore
Share & tools
lore

Ben Blum

author
#0 lore raw

cover cgroups: support for module-loadable subsystems

This patch series implements simple support for building and loading subsystems as modules, both within and outside the kernel source tree. Module unloading is as yet unimplemented - it will require more advanced reference counting in the mount/unmount code, and I plan to work on this soon.

Patch #1 sets up the subsys[] array so its contents can be dynamic as modules appear and (eventually) disappear. I introduce an rwsem called subsys_mutex to protect against concurrent loads/unloads/reads, and modify iterations over the array to handle when subsystems are absent.

Patch #2 implements an interface for modules to load subsystems, called cgroup_load_subsys, similar to cgroup_init_subsys, and adds a module pointer in struct cgroup_subsys.

Patch #3 modifies the net_cls subsystem, which already had some module declarations, to be configurable as a module, which also serves as a simple proof-of-concept.

-- bblum

author
#1 lore raw

RFC PATCH 1/3 cgroups: revamp subsys array

Make subsys[] able to be dynamically populated to support modular subsystems

From: Ben Blum <[email protected]>

This patch reworks the way the subsys[] array is used so that subsystems can register themselves after boot time, and enables the internals of cgroups to be able to handle when subsystems are not present or may appear/disappear.

---

 include/linux/cgroup.h |    8 +++-
 kernel/cgroup.c        |   91 ++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 81 insertions(+), 18 deletions(-)
include/linux/cgroup.h +6 -2
index 642a47f..d7f1545 100644
@@ -41,13 +41,17 @@ extern int cgroupstats_build(struct cgroupstats *stats,
 
 extern struct file_operations proc_cgroup_operations;
 
-/* Define the enumeration of all cgroup subsystems */
+/* Define the enumeration of all builtin cgroup subsystems */
 #define SUBSYS(_x) _x ## _subsys_id,
 enum cgroup_subsys_id {
 #include <linux/cgroup_subsys.h>
-	CGROUP_SUBSYS_COUNT
+	CGROUP_BUILTIN_SUBSYS_COUNT
 };
 #undef SUBSYS
+/* This define indicates the maximum number of subsystems that can be loaded
+ * at once. We limit to this many since cgroupfs_root has subsys_bits to keep
+ * track of all of them. */
+#define CGROUP_SUBSYS_COUNT (BITS_PER_BYTE*sizeof(unsigned long))
 
 /* Per-subsystem/per-cgroup state maintained by the system. */
 struct cgroup_subsys_state {
kernel/cgroup.c +75 -16

278 lines not on this page · raw message

author
#2 lore raw

RFC PATCH 2/3 cgroups: subsystem module interface

In reply to Ben Blum

Add interface between cgroups subsystem management and module loading

From: Ben Blum <[email protected]>

This patch implements rudimentary module-loading support for cgroups - namely, a cgroup_load_subsys (similar to cgroup_init_subsys) for use as a module initcall, and a struct module pointer in struct cgroup_subsys. Support for subsystem unloading not here, yet to be done (requires more advanced reference counting, for one thing).

I tested compiling a subsystem outside of the kernel source tree, and it was apparent that I wanted the EXPORT_SYMBOL for several functions, but I'm not sure what else it might be good to put them on. Suggestions welcome.

---

 include/linux/cgroup.h |    4 ++
 kernel/cgroup.c        |   92 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+), 0 deletions(-)
include/linux/cgroup.h +4 -0
index d7f1545..c8474c4 100644
@@ -38,6 +38,7 @@ extern void cgroup_fork_failed(struct task_struct *p, int run_callbacks,
 			       unsigned long clone_flags);
 extern int cgroupstats_build(struct cgroupstats *stats,
 				struct dentry *dentry);
+extern int cgroup_load_subsys(struct cgroup_subsys *ss);
 
 extern struct file_operations proc_cgroup_operations;
 
@@ -477,6 +478,9 @@ struct cgroup_subsys {
 	/* used when use_id == true */
 	struct idr idr;
 	spinlock_t id_lock;
+
+	/* should be defined only by modular subsystems */
+	struct module *module;
 };
 
 #define SUBSYS(_x) extern struct cgroup_subsys _x ## _subsys;
kernel/cgroup.c +92 -0

111 lines not on this page · raw message

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Re: [RFC][PATCH 2/3] cgroups: subsystem module interface

Quoted 133 lines · Ben Blum

Add interface between cgroups subsystem management and module loading

From: Ben Blum <[email protected]>

This patch implements rudimentary module-loading support for cgroups - namely, a cgroup_load_subsys (similar to cgroup_init_subsys) for use as a module initcall, and a struct module pointer in struct cgroup_subsys. Support for subsystem unloading not here, yet to be done (requires more advanced reference counting, for one thing).

I tested compiling a subsystem outside of the kernel source tree, and it was apparent that I wanted the EXPORT_SYMBOL for several functions, but I'm not sure what else it might be good to put them on. Suggestions welcome.

---

 include/linux/cgroup.h |    4 ++
 kernel/cgroup.c        |   92 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+), 0 deletions(-)
include/linux/cgroup.h +4 -0
index d7f1545..c8474c4 100644
@@ -38,6 +38,7 @@ extern void cgroup_fork_failed(struct task_struct *p, int run_callbacks,
 			       unsigned long clone_flags);
 extern int cgroupstats_build(struct cgroupstats *stats,
 				struct dentry *dentry);
+extern int cgroup_load_subsys(struct cgroup_subsys *ss);
 
 extern struct file_operations proc_cgroup_operations;
 
@@ -477,6 +478,9 @@ struct cgroup_subsys {
 	/* used when use_id == true */
 	struct idr idr;
 	spinlock_t id_lock;
+
+	/* should be defined only by modular subsystems */
+	struct module *module;
 };
 
 #define SUBSYS(_x) extern struct cgroup_subsys _x ## _subsys;
kernel/cgroup.c +67 -0
index 9966dfe..8eba8a5 100644
@@ -2491,6 +2491,7 @@ int cgroup_add_file(struct cgroup *cgrp,
 		error = PTR_ERR(dentry);
 	return error;
 }
+EXPORT_SYMBOL_GPL(cgroup_add_file);
 
 int cgroup_add_files(struct cgroup *cgrp,
 			struct cgroup_subsys *subsys,
@@ -2505,6 +2506,7 @@ int cgroup_add_files(struct cgroup *cgrp,
 	}
 	return 0;
 }
+EXPORT_SYMBOL_GPL(cgroup_add_files);
 
 /**
  * cgroup_task_count - count the number of tasks in a cgroup.
@@ -3650,7 +3652,97 @@ static void __init cgroup_init_subsys(struct cgroup_subsys *ss)
 	mutex_init(&ss->hierarchy_mutex);
 	lockdep_set_class(&ss->hierarchy_mutex, &ss->subsys_key);
 	ss->active = 1;
+
+	/* this function shouldn't be used with modular subsystems, since they
+	 * need to register a subsys_id, among other things */
+	BUG_ON(ss->module);
+}
+
+/**
+ * cgroup_load_subsys: load and register a modular subsystem at runtime
+ * @ss: the subsystem to load
+ *
+ * This function should be called in a modular subsystem's initcall. If the
+ * subsytem is built as a module, it will be assigned a new subsys_id and set
+ * up for use. If the subsystem is built-in anyway, work is delegated to the
+ * simpler cgroup_init_subsys.
+ */
+int __init_or_module cgroup_load_subsys(struct cgroup_subsys *ss)
+{
+	int i;
+	struct cgroup_subsys_state *css;
+
+	/* check name validity */
+	if (ss->name == NULL || strlen(ss->name) > MAX_CGROUP_TYPE_NAMELEN)
+		return -EINVAL;
+
+	/* we don't support callbacks in modular subsystems. this check is
+	 * before the ss->module check for consistency - a module that *could*
+	 * be a module should still have no callbacks for consistency. */
+	if (ss->fork || ss->exit)
+		return -EINVAL;
+
+	/* an optionally modular subsystem is built-in: we want to do nothing,
+	 * since cgroup_init_subsys will take care of it. */
+	if (ss->module == NULL) {
+		/* sanity: ss->module NULL only if the subsys is built-in and
+		 * appears in subsys[] already. */
+		BUG_ON(ss->subsys_id >= CGROUP_BUILTIN_SUBSYS_COUNT);
+		BUG_ON(subsys[ss->subsys_id] != ss);
+		return 0;
+	}
+
+	/* need to register a subsys id before anything else - for example,
+	 * init_cgroup_css needs it. also, subsys_mutex needs to nest outside
+	 * cgroup_mutex. */
+	down_write(&subsys_mutex);
+	/* find the first empty slot in the array */
+	for (i = CGROUP_BUILTIN_SUBSYS_COUNT; i < CGROUP_SUBSYS_COUNT; i++) {
+		if (subsys[i] == NULL)
+			break;
+	}
+	if (i == CGROUP_SUBSYS_COUNT) {
+		/* maximum number of subsystems already registered! */
+		up_write(&subsys_mutex);
+		return -EBUSY;
+	}
+	/* assign ourselves the subsys_id */
+	ss->subsys_id = i;
+	subsys[i] = ss;
+
+	mutex_lock(&cgroup_mutex);
+	/* no ss->create seems to need anything important in the ss struct, so
+	 * this can happen first (i.e. before the rootnode attachment). */
+	css = ss->create(ss, dummytop);
+	if (IS_ERR(css)) {
+		/* failure case - need to deassign the subsys[] slot. */
+		mutex_unlock(&cgroup_mutex);

kernel/cgroup.c static void __init cgroup_init_subsys(struct cgroup_subsys *ss) · line 3729

Quoted 10 lines
kernel/cgroup.c +10 -0
+		subsys[i] = NULL;
+		up_write(&subsys_mutex);
+		return PTR_ERR(css);
+	}
+
+	list_add(&ss->sibling, &rootnode.subsys_list);
+	ss->root = &rootnode;
+
+	init_cgroup_css(css, ss, dummytop);
+	init_css_set.subsys[ss->subsys_id] = dummytop->subsys[ss->subsys_id];

There is a problem here; namely, the init_css_set is supposed to be immutable after boot. Putting a new subsystem state in it seems like it will be impossible to get right, since a hierarchy might be mounted and already using the init_css_set. Perhaps it would be better to use find_css_set for this?

Quoted 20 lines
kernel/cgroup.c +15 -0
+
+	mutex_init(&ss->hierarchy_mutex);
+	lockdep_set_class(&ss->hierarchy_mutex, &ss->subsys_key);
+	ss->active = 1;
+
+	/* pin the subsystem's module so it doesn't go away. this shouldn't
+	 * fail, since the module's initcall calls us.
+	 * TODO: with module unloading, move this elsewhere */
+	BUG_ON(!try_module_get(ss->module));
+
+	/* success! */
+	mutex_unlock(&cgroup_mutex);
+	up_write(&subsys_mutex);
+	return 0;
 }
+EXPORT_SYMBOL_GPL(cgroup_load_subsys);
 
 /**
  * cgroup_init_early - cgroup initialization at system boot
author
#5 lore raw

Re: [RFC][PATCH 2/3] cgroups: subsystem module interface

In reply to Ben Blum
Quoted 133 lines · Ben Blum

Add interface between cgroups subsystem management and module loading

From: Ben Blum <[email protected]>

This patch implements rudimentary module-loading support for cgroups - namely, a cgroup_load_subsys (similar to cgroup_init_subsys) for use as a module initcall, and a struct module pointer in struct cgroup_subsys. Support for subsystem unloading not here, yet to be done (requires more advanced reference counting, for one thing).

I tested compiling a subsystem outside of the kernel source tree, and it was apparent that I wanted the EXPORT_SYMBOL for several functions, but I'm not sure what else it might be good to put them on. Suggestions welcome.

---

 include/linux/cgroup.h |    4 ++
 kernel/cgroup.c        |   92 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+), 0 deletions(-)
include/linux/cgroup.h +4 -0
index d7f1545..c8474c4 100644
@@ -38,6 +38,7 @@ extern void cgroup_fork_failed(struct task_struct *p, int run_callbacks,
 			       unsigned long clone_flags);
 extern int cgroupstats_build(struct cgroupstats *stats,
 				struct dentry *dentry);
+extern int cgroup_load_subsys(struct cgroup_subsys *ss);
 
 extern struct file_operations proc_cgroup_operations;
 
@@ -477,6 +478,9 @@ struct cgroup_subsys {
 	/* used when use_id == true */
 	struct idr idr;
 	spinlock_t id_lock;
+
+	/* should be defined only by modular subsystems */
+	struct module *module;
 };
 
 #define SUBSYS(_x) extern struct cgroup_subsys _x ## _subsys;
kernel/cgroup.c +67 -0
index 9966dfe..8eba8a5 100644
@@ -2491,6 +2491,7 @@ int cgroup_add_file(struct cgroup *cgrp,
 		error = PTR_ERR(dentry);
 	return error;
 }
+EXPORT_SYMBOL_GPL(cgroup_add_file);
 
 int cgroup_add_files(struct cgroup *cgrp,
 			struct cgroup_subsys *subsys,
@@ -2505,6 +2506,7 @@ int cgroup_add_files(struct cgroup *cgrp,
 	}
 	return 0;
 }
+EXPORT_SYMBOL_GPL(cgroup_add_files);
 
 /**
  * cgroup_task_count - count the number of tasks in a cgroup.
@@ -3650,7 +3652,97 @@ static void __init cgroup_init_subsys(struct cgroup_subsys *ss)
 	mutex_init(&ss->hierarchy_mutex);
 	lockdep_set_class(&ss->hierarchy_mutex, &ss->subsys_key);
 	ss->active = 1;
+
+	/* this function shouldn't be used with modular subsystems, since they
+	 * need to register a subsys_id, among other things */
+	BUG_ON(ss->module);
+}
+
+/**
+ * cgroup_load_subsys: load and register a modular subsystem at runtime
+ * @ss: the subsystem to load
+ *
+ * This function should be called in a modular subsystem's initcall. If the
+ * subsytem is built as a module, it will be assigned a new subsys_id and set
+ * up for use. If the subsystem is built-in anyway, work is delegated to the
+ * simpler cgroup_init_subsys.
+ */
+int __init_or_module cgroup_load_subsys(struct cgroup_subsys *ss)
+{
+	int i;
+	struct cgroup_subsys_state *css;
+
+	/* check name validity */
+	if (ss->name == NULL || strlen(ss->name) > MAX_CGROUP_TYPE_NAMELEN)
+		return -EINVAL;
+
+	/* we don't support callbacks in modular subsystems. this check is
+	 * before the ss->module check for consistency - a module that *could*
+	 * be a module should still have no callbacks for consistency. */
+	if (ss->fork || ss->exit)
+		return -EINVAL;
+
+	/* an optionally modular subsystem is built-in: we want to do nothing,
+	 * since cgroup_init_subsys will take care of it. */
+	if (ss->module == NULL) {
+		/* sanity: ss->module NULL only if the subsys is built-in and
+		 * appears in subsys[] already. */
+		BUG_ON(ss->subsys_id >= CGROUP_BUILTIN_SUBSYS_COUNT);
+		BUG_ON(subsys[ss->subsys_id] != ss);
+		return 0;
+	}
+
+	/* need to register a subsys id before anything else - for example,
+	 * init_cgroup_css needs it. also, subsys_mutex needs to nest outside
+	 * cgroup_mutex. */
+	down_write(&subsys_mutex);
+	/* find the first empty slot in the array */
+	for (i = CGROUP_BUILTIN_SUBSYS_COUNT; i < CGROUP_SUBSYS_COUNT; i++) {
+		if (subsys[i] == NULL)
+			break;
+	}
+	if (i == CGROUP_SUBSYS_COUNT) {
+		/* maximum number of subsystems already registered! */
+		up_write(&subsys_mutex);
+		return -EBUSY;
+	}
+	/* assign ourselves the subsys_id */
+	ss->subsys_id = i;
+	subsys[i] = ss;
+
+	mutex_lock(&cgroup_mutex);
+	/* no ss->create seems to need anything important in the ss struct, so
+	 * this can happen first (i.e. before the rootnode attachment). */
+	css = ss->create(ss, dummytop);
+	if (IS_ERR(css)) {
+		/* failure case - need to deassign the subsys[] slot. */
+		mutex_unlock(&cgroup_mutex);

kernel/cgroup.c static void __init cgroup_init_subsys(struct cgroup_subsys *ss) · line 3729

Quoted 10 lines
kernel/cgroup.c +10 -0
+		subsys[i] = NULL;
+		up_write(&subsys_mutex);
+		return PTR_ERR(css);
+	}
+
+	list_add(&ss->sibling, &rootnode.subsys_list);
+	ss->root = &rootnode;
+
+	init_cgroup_css(css, ss, dummytop);
+	init_css_set.subsys[ss->subsys_id] = dummytop->subsys[ss->subsys_id];

There is a problem here; namely, the init_css_set is supposed to be immutable after boot. Putting a new subsystem state in it seems like it will be impossible to get right, since a hierarchy might be mounted and already using the init_css_set. Perhaps it would be better to use find_css_set for this?

Quoted 20 lines
kernel/cgroup.c +15 -0
+
+	mutex_init(&ss->hierarchy_mutex);
+	lockdep_set_class(&ss->hierarchy_mutex, &ss->subsys_key);
+	ss->active = 1;
+
+	/* pin the subsystem's module so it doesn't go away. this shouldn't
+	 * fail, since the module's initcall calls us.
+	 * TODO: with module unloading, move this elsewhere */
+	BUG_ON(!try_module_get(ss->module));
+
+	/* success! */
+	mutex_unlock(&cgroup_mutex);
+	up_write(&subsys_mutex);
+	return 0;
 }
+EXPORT_SYMBOL_GPL(cgroup_load_subsys);
 
 /**
  * cgroup_init_early - cgroup initialization at system boot
#6 lore raw

Re: [RFC][PATCH 2/3] cgroups: subsystem module interface

Quoted 8 lines · Ben Blum
Quoted 2 lines

+     init_cgroup_css(css, ss, dummytop);
+     init_css_set.subsys[ss->subsys_id] = dummytop->subsys[ss->subsys_id];

There is a problem here; namely, the init_css_set is supposed to be immutable after boot. Putting a new subsystem state in it seems like it will be impossible to get right, since a hierarchy might be mounted and already using the init_css_set. Perhaps it would be better to use find_css_set for this?

Hmm, it's not just init_css_set that needs to get a pointer to the new css, it's all existing css_sets - since every task has to start off in the root cgroup for the new subsystem. A css_set is just a flattened cache of the css pointers from all the cgroups that a task is a member of, so when we add a css pointer to the unmount cgroup, we need to either update all css_set objects, or else run through every task in the system and update its cgroups pointer to a new css_set that's equivalent to its old css_set plus the new css pointer.

Given the races inherent in trying to find and update all tasks in the system, I think the cleanest way to solve this is:

- copy the css_set_table hash table and clear the original table.
- for each bucket in the hash table:
- for each css_set in the bucket:
- add the new root css to the css_set
- rehash the css_set into the hash table

Unfortunately this issue also makes it a bit harder to support modular-subsystem unloading.

Paul

author
#7 lore raw

RFC PATCH 3/3 cgroups: net_cls subsys as module

In reply to Ben Blum

Allows the net_cls cgroup subsystem to be compiled as a module

From: Ben Blum <[email protected]>

This patch modifies net/sched/cls_cgroup.c to allow the net_cls subsystem to be optionally compiled as a module instead of builtin. The cgroup_subsys struct is moved around a bit to allow the subsys_id to be either declared as a compile-time constant by the cgroup_subsys.h include in cgroup.h, or, if it's a module, initialized within the struct by cgroup_load_subsys.

---

 net/sched/Kconfig      |    2 +-
 net/sched/cls_cgroup.c |   37 ++++++++++++++++++++++++++++---------
 2 files changed, 29 insertions(+), 10 deletions(-)
net/sched/Kconfig +1 -1
index 929218a..8489951 100644
@@ -328,7 +328,7 @@ config NET_CLS_FLOW
 	  module will be called cls_flow.
 
 config NET_CLS_CGROUP
-	bool "Control Group Classifier"
+	tristate "Control Group Classifier"
 	select NET_CLS
 	depends on CGROUPS
 	---help---
net/sched/cls_cgroup.c +28 -9
index e4877ca..df9723b 100644
@@ -24,6 +24,25 @@ struct cgroup_cls_state
 	u32 classid;
 };
 
+static struct cgroup_subsys_state *cgrp_create(struct cgroup_subsys *ss,
+					       struct cgroup *cgrp);
+static void cgrp_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
+static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp);
+
+struct cgroup_subsys net_cls_subsys = {
+	.name		= "net_cls",
+	.create		= cgrp_create,
+	.destroy	= cgrp_destroy,
+	.populate	= cgrp_populate,
+#ifdef CONFIG_NET_CLS_CGROUP
+	.subsys_id	= net_cls_subsys_id,
+#else
+#define net_cls_subsys_id net_cls_subsys.subsys_id
+#endif
+	.module		= THIS_MODULE,
+};
+
+
 static inline struct cgroup_cls_state *cgrp_cls_state(struct cgroup *cgrp)
 {
 	return container_of(cgroup_subsys_state(cgrp, net_cls_subsys_id),
@@ -79,14 +98,6 @@ static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
 	return cgroup_add_files(cgrp, ss, ss_files, ARRAY_SIZE(ss_files));
 }
 
-struct cgroup_subsys net_cls_subsys = {
-	.name		= "net_cls",
-	.create		= cgrp_create,
-	.destroy	= cgrp_destroy,
-	.populate	= cgrp_populate,
-	.subsys_id	= net_cls_subsys_id,
-};
-
 struct cls_cgroup_head
 {
 	u32			handle;
@@ -277,12 +288,20 @@ static struct tcf_proto_ops cls_cgroup_ops __read_mostly = {
 
 static int __init init_cgroup_cls(void)
 {
-	return register_tcf_proto_ops(&cls_cgroup_ops);
+	int ret = register_tcf_proto_ops(&cls_cgroup_ops);
+	if (ret)
+		return ret;
+	ret = cgroup_load_subsys(&net_cls_subsys);
+	if (ret)
+		unregister_tcf_proto_ops(&cls_cgroup_ops);
+	return ret;
 }
 
 static void __exit exit_cgroup_cls(void)
 {
 	unregister_tcf_proto_ops(&cls_cgroup_ops);
+	/* TODO: unload subsystem. for now, the try_module_get in load_subsys
+	 * prevents us from getting here. */
 }
 
 module_init(init_cgroup_cls);
#8 lore raw

Re: [RFC][PATCH 3/3] cgroups: net_cls subsys as module

Quoted 1 line · On Mon, 2 Nov 2009 16:31:03 -0500 Ben Blum wrote:

Allows the net_cls cgroup subsystem to be compiled as a module

Please cc: netdev on networking patches. [done]

Quoted 97 lines

From: Ben Blum <[email protected]>

This patch modifies net/sched/cls_cgroup.c to allow the net_cls subsystem to be optionally compiled as a module instead of builtin. The cgroup_subsys struct is moved around a bit to allow the subsys_id to be either declared as a compile-time constant by the cgroup_subsys.h include in cgroup.h, or, if it's a module, initialized within the struct by cgroup_load_subsys.

---

 net/sched/Kconfig      |    2 +-
 net/sched/cls_cgroup.c |   37 ++++++++++++++++++++++++++++---------
 2 files changed, 29 insertions(+), 10 deletions(-)
net/sched/Kconfig +1 -1
index 929218a..8489951 100644
@@ -328,7 +328,7 @@ config NET_CLS_FLOW
 	  module will be called cls_flow.
 
 config NET_CLS_CGROUP
-	bool "Control Group Classifier"
+	tristate "Control Group Classifier"
 	select NET_CLS
 	depends on CGROUPS
 	---help---
net/sched/cls_cgroup.c +28 -10
index e4877ca..df9723b 100644
@@ -24,6 +24,25 @@ struct cgroup_cls_state
 	u32 classid;
 };
 
+static struct cgroup_subsys_state *cgrp_create(struct cgroup_subsys *ss,
+					       struct cgroup *cgrp);
+static void cgrp_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
+static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp);
+
+struct cgroup_subsys net_cls_subsys = {
+	.name		= "net_cls",
+	.create		= cgrp_create,
+	.destroy	= cgrp_destroy,
+	.populate	= cgrp_populate,
+#ifdef CONFIG_NET_CLS_CGROUP
+	.subsys_id	= net_cls_subsys_id,
+#else
+#define net_cls_subsys_id net_cls_subsys.subsys_id
+#endif
+	.module		= THIS_MODULE,
+};
+
+
 static inline struct cgroup_cls_state *cgrp_cls_state(struct cgroup *cgrp)
 {
 	return container_of(cgroup_subsys_state(cgrp, net_cls_subsys_id),
@@ -79,14 +98,6 @@ static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
 	return cgroup_add_files(cgrp, ss, ss_files, ARRAY_SIZE(ss_files));
 }
 
-struct cgroup_subsys net_cls_subsys = {
-	.name		= "net_cls",
-	.create		= cgrp_create,
-	.destroy	= cgrp_destroy,
-	.populate	= cgrp_populate,
-	.subsys_id	= net_cls_subsys_id,
-};
-
 struct cls_cgroup_head
 {
 	u32			handle;
@@ -277,12 +288,20 @@ static struct tcf_proto_ops cls_cgroup_ops __read_mostly = {
 
 static int __init init_cgroup_cls(void)
 {
-	return register_tcf_proto_ops(&cls_cgroup_ops);
+	int ret = register_tcf_proto_ops(&cls_cgroup_ops);
+	if (ret)
+		return ret;
+	ret = cgroup_load_subsys(&net_cls_subsys);
+	if (ret)
+		unregister_tcf_proto_ops(&cls_cgroup_ops);
+	return ret;
 }
 
 static void __exit exit_cgroup_cls(void)
 {
 	unregister_tcf_proto_ops(&cls_cgroup_ops);
+	/* TODO: unload subsystem. for now, the try_module_get in load_subsys
+	 * prevents us from getting here. */
 }
 
 module_init(init_cgroup_cls);
--
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

---
~Randy

A message that is not in the archive (sent off-list or not yet received): [email protected]

#10 lore raw

Re: [RFC][PATCH 3/3] cgroups: net_cls subsys as module

Quoted 1 line · On Mon, 2 Nov 2009 16:31:03 -0500 Ben Blum wrote:

Allows the net_cls cgroup subsystem to be compiled as a module

Please cc: netdev on networking patches. [done]

Quoted 97 lines

From: Ben Blum <[email protected]>

This patch modifies net/sched/cls_cgroup.c to allow the net_cls subsystem to be optionally compiled as a module instead of builtin. The cgroup_subsys struct is moved around a bit to allow the subsys_id to be either declared as a compile-time constant by the cgroup_subsys.h include in cgroup.h, or, if it's a module, initialized within the struct by cgroup_load_subsys.

---

 net/sched/Kconfig      |    2 +-
 net/sched/cls_cgroup.c |   37 ++++++++++++++++++++++++++++---------
 2 files changed, 29 insertions(+), 10 deletions(-)
net/sched/Kconfig +1 -1
index 929218a..8489951 100644
@@ -328,7 +328,7 @@ config NET_CLS_FLOW
 	  module will be called cls_flow.
 
 config NET_CLS_CGROUP
-	bool "Control Group Classifier"
+	tristate "Control Group Classifier"
 	select NET_CLS
 	depends on CGROUPS
 	---help---
net/sched/cls_cgroup.c +28 -10
index e4877ca..df9723b 100644
@@ -24,6 +24,25 @@ struct cgroup_cls_state
 	u32 classid;
 };
 
+static struct cgroup_subsys_state *cgrp_create(struct cgroup_subsys *ss,
+					       struct cgroup *cgrp);
+static void cgrp_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
+static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp);
+
+struct cgroup_subsys net_cls_subsys = {
+	.name		= "net_cls",
+	.create		= cgrp_create,
+	.destroy	= cgrp_destroy,
+	.populate	= cgrp_populate,
+#ifdef CONFIG_NET_CLS_CGROUP
+	.subsys_id	= net_cls_subsys_id,
+#else
+#define net_cls_subsys_id net_cls_subsys.subsys_id
+#endif
+	.module		= THIS_MODULE,
+};
+
+
 static inline struct cgroup_cls_state *cgrp_cls_state(struct cgroup *cgrp)
 {
 	return container_of(cgroup_subsys_state(cgrp, net_cls_subsys_id),
@@ -79,14 +98,6 @@ static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp)
 	return cgroup_add_files(cgrp, ss, ss_files, ARRAY_SIZE(ss_files));
 }
 
-struct cgroup_subsys net_cls_subsys = {
-	.name		= "net_cls",
-	.create		= cgrp_create,
-	.destroy	= cgrp_destroy,
-	.populate	= cgrp_populate,
-	.subsys_id	= net_cls_subsys_id,
-};
-
 struct cls_cgroup_head
 {
 	u32			handle;
@@ -277,12 +288,20 @@ static struct tcf_proto_ops cls_cgroup_ops __read_mostly = {
 
 static int __init init_cgroup_cls(void)
 {
-	return register_tcf_proto_ops(&cls_cgroup_ops);
+	int ret = register_tcf_proto_ops(&cls_cgroup_ops);
+	if (ret)
+		return ret;
+	ret = cgroup_load_subsys(&net_cls_subsys);
+	if (ret)
+		unregister_tcf_proto_ops(&cls_cgroup_ops);
+	return ret;
 }
 
 static void __exit exit_cgroup_cls(void)
 {
 	unregister_tcf_proto_ops(&cls_cgroup_ops);
+	/* TODO: unload subsystem. for now, the try_module_get in load_subsys
+	 * prevents us from getting here. */
 }
 
 module_init(init_cgroup_cls);
--
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

---
~Randy

A message that is not in the archive (sent off-list or not yet received): [email protected]

#12 lore raw
Quoted 6 lines · Ben Blum

This patch series implements simple support for building and loading subsystems as modules, both within and outside the kernel source tree. Module unloading is as yet unimplemented - it will require more advanced reference counting in the mount/unmount code, and I plan to work on this soon.

I doubt the value of module-loadable subsystems. A cgroup subsystem is usually a kernel resource controller, and normally it needs to add some hooks in some in-kernel structures and functions, which makes it impossible to be a module.

In fact, net_cls is the only subsystem that can be a module, and make it a module doesn't seem to have real benefit?

Quoted 15 lines

Patch #1 sets up the subsys[] array so its contents can be dynamic as modules appear and (eventually) disappear. I introduce an rwsem called subsys_mutex to protect against concurrent loads/unloads/reads, and modify iterations over the array to handle when subsystems are absent.

Patch #2 implements an interface for modules to load subsystems, called cgroup_load_subsys, similar to cgroup_init_subsys, and adds a module pointer in struct cgroup_subsys.

Patch #3 modifies the net_cls subsystem, which already had some module declarations, to be configurable as a module, which also serves as a simple proof-of-concept.

-- bblum

#13 lore raw
In reply to Ben Blum
Quoted 6 lines · Ben Blum

This patch series implements simple support for building and loading subsystems as modules, both within and outside the kernel source tree. Module unloading is as yet unimplemented - it will require more advanced reference counting in the mount/unmount code, and I plan to work on this soon.

I doubt the value of module-loadable subsystems. A cgroup subsystem is usually a kernel resource controller, and normally it needs to add some hooks in some in-kernel structures and functions, which makes it impossible to be a module.

In fact, net_cls is the only subsystem that can be a module, and make it a module doesn't seem to have real benefit?

Quoted 15 lines

Patch #1 sets up the subsys[] array so its contents can be dynamic as modules appear and (eventually) disappear. I introduce an rwsem called subsys_mutex to protect against concurrent loads/unloads/reads, and modify iterations over the array to handle when subsystems are absent.

Patch #2 implements an interface for modules to load subsystems, called cgroup_load_subsys, similar to cgroup_init_subsys, and adds a module pointer in struct cgroup_subsys.

Patch #3 modifies the net_cls subsystem, which already had some module declarations, to be configurable as a module, which also serves as a simple proof-of-concept.

-- bblum

A message that is not in the archive (sent off-list or not yet received): [email protected]

#15 lore raw
Quoted 5 lines · Li Zefan

I doubt the value of module-loadable subsystems. A cgroup subsystem is usually a kernel resource controller, and normally it needs to add some hooks in some in-kernel structures and functions, which makes it impossible to be a module.

Not true - as the classifier example shows, if there's already a framework that supports hooking loadable modules into some kernel system, a resource controller can be a module.

Quoted 3 lines

In fact, net_cls is the only subsystem that can be a module, and make it a module doesn't seem to have real benefit?

Wouldn't it be just as useful as any other module-loadable classifier, of which there are quite a few?

It's true that net_cls is currently the only subsystem that makes sense to use as a module, but its existence shows that the concept isn't too outlandish.

Paul

#16 lore raw
In reply to Li Zefan
Quoted 5 lines · Li Zefan

I doubt the value of module-loadable subsystems. A cgroup subsystem is usually a kernel resource controller, and normally it needs to add some hooks in some in-kernel structures and functions, which makes it impossible to be a module.

Not true - as the classifier example shows, if there's already a framework that supports hooking loadable modules into some kernel system, a resource controller can be a module.

Quoted 3 lines

In fact, net_cls is the only subsystem that can be a module, and make it a module doesn't seem to have real benefit?

Wouldn't it be just as useful as any other module-loadable classifier, of which there are quite a few?

It's true that net_cls is currently the only subsystem that makes sense to use as a module, but its existence shows that the concept isn't too outlandish.

Paul

A message that is not in the archive (sent off-list or not yet received): 6599ad830911021811n7ccf7a70y6939ac581381c22f-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#18 lore raw
Quoted 10 lines · Paul Menage
Quoted 4 lines · Li Zefan

I doubt the value of module-loadable subsystems. A cgroup subsystem is usually a kernel resource controller, and normally it needs to add some hooks in some in-kernel structures and functions, which makes it impossible to be a module.

Not true - as the classifier example shows, if there's already a framework that supports hooking loadable modules into some kernel system, a resource controller can be a module.

True. Just reminds me a pure cfq-based io controller can be a module too, though the io controller discussed in the mini summit won't be a pure cfq-based one.

Quoted 10 lines
Quoted 2 lines

In fact, net_cls is the only subsystem that can be a module, and make it a module doesn't seem to have real benefit?

Wouldn't it be just as useful as any other module-loadable classifier, of which there are quite a few?

It's true that net_cls is currently the only subsystem that makes sense to use as a module, but its existence shows that the concept isn't too outlandish.

Yes, sounds reasonable.

#19 lore raw
In reply to Paul Menage
Quoted 10 lines · Paul Menage
Quoted 4 lines · Li Zefan

I doubt the value of module-loadable subsystems. A cgroup subsystem is usually a kernel resource controller, and normally it needs to add some hooks in some in-kernel structures and functions, which makes it impossible to be a module.

Not true - as the classifier example shows, if there's already a framework that supports hooking loadable modules into some kernel system, a resource controller can be a module.

True. Just reminds me a pure cfq-based io controller can be a module too, though the io controller discussed in the mini summit won't be a pure cfq-based one.

Quoted 10 lines
Quoted 2 lines

In fact, net_cls is the only subsystem that can be a module, and make it a module doesn't seem to have real benefit?

Wouldn't it be just as useful as any other module-loadable classifier, of which there are quite a few?

It's true that net_cls is currently the only subsystem that makes sense to use as a module, but its existence shows that the concept isn't too outlandish.

Yes, sounds reasonable.