Skip to contentSkip to conversation

cgroup: use static_keys for subsystem enabled and on_dfl tests

PATCH In review Tejun Heo 3 participants 10 messages Sep 16, 2015 – Sep 18, 2015

Subsystems (6)
lore
Share & tools
lore

Tejun Heo

author maintainer
#0 lore raw

PATCH 1/1 cgroup: use static_keys for subsystem enabled and on_dfl tests

cgroup_subsys->disabled and cgroup_on_dfl() tests are likely to be used in hot paths and seldom change. The former is set once during boot and the latter only when a controller is migrated between the default hierarchy and traditional ones.

This patchset makes these tests static_key based and contains the following four patches.

0001-jump_label-make-static_key_enabled-work-on-static_ke.patch 0002-cgroup-implement-static_key-based-cgroup_subsys_enab.patch 0003-cgroup-replace-cgroup_subsys-disabled-tests-with-cgr.patch 0004-cgroup-replace-cgroup_on_dfl-tests-in-controllers-wi.patch

0001 is a prep patch in jump_label. 0002 adds the needed static_keys. 0003-0004 convert the existing usages and drop the old tests.

This patchset is on top of v4.3-rc1 and is availalbe in the following git branch.

git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git review-jump-labels

diffstat follows. Thanks.

 block/blk-throttle.c           |    2 
 block/cfq-iosched.c            |    4 -
 include/linux/cgroup-defs.h    |    1 
 include/linux/cgroup.h         |   79 +++++++---------------------
 include/linux/hugetlb_cgroup.h |    4 -
 include/linux/jump_label.h     |   18 +++---
 include/linux/memcontrol.h     |    4 -
 kernel/cgroup.c                |  113 ++++++++++++++++++++++++++++++++++++++---
 kernel/cpuset.c                |   23 ++++----
 mm/memcontrol.c                |    4 -
 10 files changed, 157 insertions(+), 95 deletions(-)

--
tejun

author maintainer
#1 lore raw

PATCH 1/4 jump_label: make static_key_enabled() work on static_key_true/false types too

static_key_enabled() can be used on struct static_key but not on its wrapper types static_key_true and static_key_false. The function is useful for debugging and management of static keys. Update it so that it can be used for the wrapper types too.

Cc Peter Zijlstra <[email protected]>
Cc Andrew Morton <[email protected]>

---
Hello,

If this patch is acceptable, please let me know how it should be routed.

Thanks.

 include/linux/jump_label.h | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
include/linux/jump_label.h +11 -7
index 7f653e8..c9ca050 100644
@@ -216,11 +216,6 @@ static inline int jump_label_apply_nops(struct module *mod)
 #define STATIC_KEY_INIT STATIC_KEY_INIT_FALSE
 #define jump_label_enabled static_key_enabled
 
-static inline bool static_key_enabled(struct static_key *key)
-{
-	return static_key_count(key) > 0;
-}
-
 static inline void static_key_enable(struct static_key *key)
 {
 	int count = static_key_count(key);
@@ -267,6 +262,17 @@ struct static_key_false {
 #define DEFINE_STATIC_KEY_FALSE(name)	\
 	struct static_key_false name = STATIC_KEY_FALSE_INIT
 
+extern bool ____wrong_branch_error(void);
+
+#define static_key_enabled(x)							\
+({										\
+	if (!__builtin_types_compatible_p(typeof(*x), struct static_key) &&	\
+	    !__builtin_types_compatible_p(typeof(*x), struct static_key_true) &&\
+	    !__builtin_types_compatible_p(typeof(*x), struct static_key_false))	\
+		____wrong_branch_error();					\
+	static_key_count((struct static_key *)x) > 0;				\
+})
+
 #ifdef HAVE_JUMP_LABEL
 
 /*
@@ -325,8 +331,6 @@ struct static_key_false {
  * See jump_label_type() / jump_label_init_type().
  */
 
-extern bool ____wrong_branch_error(void);
-
 #define static_branch_likely(x)							\
 ({										\
 	bool branch;								\
Signature · 1 line
2.4.3
maintainer
#2 lore raw

Re: [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too

Quoted 5 lines · Tejun Heo

static_key_enabled() can be used on struct static_key but not on its wrapper types static_key_true and static_key_false. The function is useful for debugging and management of static keys. Update it so that it can be used for the wrapper types too.

Quoted 9 lines
Cc Peter Zijlstra <[email protected]>
Cc Andrew Morton <akpm-de/[email protected]>

---
Hello,

If this patch is acceptable, please let me know how it should be routed.

Yeah, no problem with this, I'm assuming there's dependencies in the patches you didn't send me?

author maintainer
#3 lore raw

Re: [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too

Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Tejun Heo

static_key_enabled() can be used on struct static_key but not on its wrapper types static_key_true and static_key_false. The function is useful for debugging and management of static keys. Update it so that it can be used for the wrapper types too.

Cc Peter Zijlstra <[email protected]>
Cc Andrew Morton <akpm-de/[email protected]>

---
Hello,

Quoted 6 lines
Quoted 3 lines

If this patch is acceptable, please let me know how it should be routed.

Yeah, no problem with this, I'm assuming there's dependencies in the patches you didn't send me?

Yeap, later patches add static_keys for testing whether a controller is enabled and whether it's on the new or old hierarchy and management path in cgroup core needs to index the keys and test them.

Thanks.

Signature · 1 line
tejun
maintainer Acked-by
#4 lore raw

Re: [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too

Quoted 17 lines · Tejun Heo
Quoted 16 lines · Peter Zijlstra
Quoted 14 lines · Tejun Heo

static_key_enabled() can be used on struct static_key but not on its wrapper types static_key_true and static_key_false. The function is useful for debugging and management of static keys. Update it so that it can be used for the wrapper types too.

Cc Peter Zijlstra <[email protected]>
Cc Andrew Morton <[email protected]>

---
Hello,

If this patch is acceptable, please let me know how it should be routed.

Quoted 6 lines
Quoted 2 lines

Yeah, no problem with this, I'm assuming there's dependencies in the patches you didn't send me?

Yeap, later patches add static_keys for testing whether a controller is enabled and whether it's on the new or old hierarchy and management path in cgroup core needs to index the keys and test them.

Ok, take it through you tree then, its a small enough patch.

author maintainer
#5 lore raw

PATCH 2/4 cgroup: implement static_key based cgroup_subsys_enabled() and cgroup_subsys_on_dfl()

In reply to Tejun Heo

Whether a subsys is enabled and attached to the default hierarchy seldom changes and may be tested in the hot paths. This patch implements static_key based cgroup_subsys_enabled() and cgroup_subsys_on_dfl() tests.

The following patches will update the users and remove duplicate mechanisms.

---
 include/linux/cgroup.h | 21 +++++++++++++++++++++
 kernel/cgroup.c        | 27 ++++++++++++++++++++++++++-
 2 files changed, 47 insertions(+), 1 deletion(-)
include/linux/cgroup.h +21 -0
index eb7ca55..c3a9f1e 100644
@@ -17,6 +17,7 @@
 #include <linux/fs.h>
 #include <linux/seq_file.h>
 #include <linux/kernfs.h>
+#include <linux/jump_label.h>
 
 #include <linux/cgroup-defs.h>
 
@@ -50,6 +51,26 @@ extern struct css_set init_css_set;
 #include <linux/cgroup_subsys.h>
 #undef SUBSYS
 
+#define SUBSYS(_x)								\
+	extern struct static_key_true _x ## _cgrp_subsys_enabled_key;		\
+	extern struct static_key_true _x ## _cgrp_subsys_on_dfl_key;
+#include <linux/cgroup_subsys.h>
+#undef SUBSYS
+
+/**
+ * cgroup_subsys_enabled - fast test on whether a subsys is enabled
+ * @ss: subsystem in question
+ */
+#define cgroup_subsys_enabled(ss)						\
+	static_branch_likely(&ss ## _enabled_key)
+
+/**
+ * cgroup_subsys_on_dfl - fast test on whether a subsys is on default hierarchy
+ * @ss: subsystem in question
+ */
+#define cgroup_subsys_on_dfl(ss)						\
+	static_branch_likely(&ss ## _on_dfl_key)
+
 bool css_has_online_children(struct cgroup_subsys_state *css);
 struct cgroup_subsys_state *css_from_id(int id, struct cgroup_subsys *ss);
 struct cgroup_subsys_state *cgroup_get_e_css(struct cgroup *cgroup,
kernel/cgroup.c +26 -1
index 2cf0f79..3619389 100644
@@ -139,6 +139,27 @@ static const char *cgroup_subsys_name[] = {
 };
 #undef SUBSYS
 
+/* array of static_keys for cgroup_subsys_enabled() and cgroup_subsys_on_dfl() */
+#define SUBSYS(_x)								\
+	DEFINE_STATIC_KEY_TRUE(_x ## _cgrp_subsys_enabled_key);			\
+	DEFINE_STATIC_KEY_TRUE(_x ## _cgrp_subsys_on_dfl_key);			\
+	EXPORT_SYMBOL_GPL(_x ## _cgrp_subsys_enabled_key);			\
+	EXPORT_SYMBOL_GPL(_x ## _cgrp_subsys_on_dfl_key);
+#include <linux/cgroup_subsys.h>
+#undef SUBSYS
+
+#define SUBSYS(_x) [_x ## _cgrp_id] = &_x ## _cgrp_subsys_enabled_key,
+static struct static_key_true *cgroup_subsys_enabled_key[] = {
+#include <linux/cgroup_subsys.h>
+};
+#undef SUBSYS
+
+#define SUBSYS(_x) [_x ## _cgrp_id] = &_x ## _cgrp_subsys_on_dfl_key,
+static struct static_key_true *cgroup_subsys_on_dfl_key[] = {
+#include <linux/cgroup_subsys.h>
+};
+#undef SUBSYS
+
 /*
  * The default hierarchy, reserved for the subsystems that are otherwise
  * unattached - it never has more than a single cgroup, and all tasks are
@@ -1319,9 +1340,12 @@ static int rebind_subsystems(struct cgroup_root *dst_root,
 
 		/* default hierarchy doesn't enable controllers by default */
 		dst_root->subsys_mask |= 1 << ssid;
-		if (dst_root != &cgrp_dfl_root) {
+		if (dst_root == &cgrp_dfl_root) {
+			static_branch_enable(cgroup_subsys_on_dfl_key[ssid]);
+		} else {
 			dst_root->cgrp.subtree_control |= 1 << ssid;
 			cgroup_refresh_child_subsys_mask(&dst_root->cgrp);
+			static_branch_disable(cgroup_subsys_on_dfl_key[ssid]);
 		}
 
 		if (ss->bind)
@@ -5483,6 +5507,7 @@ static int __init cgroup_disable(char *str)
 			    strcmp(token, ss->legacy_name))
 				continue;
 
+			static_branch_disable(cgroup_subsys_enabled_key[i]);
 			ss->disabled = 1;
 			printk(KERN_INFO "Disabling %s control group subsystem\n",
 			       ss->name);
Signature · 1 line
2.4.3
author maintainer
#6 lore raw

PATCH 3/4 cgroup: replace cgroup_subsys->disabled tests with cgroup_subsys_enabled()

In reply to Tejun Heo

Replace cgroup_subsys->disabled tests in controllers with cgroup_subsys_enabled(). cgroup_subsys_enabled() requires literal subsys name as its parameter and thus can't be used for cgroup core which iterates through controllers. For cgroup core, introduce and use cgroup_ssid_enabled() which uses slower static_key_enabled() test and can be indexed by subsys ID.

This leaves cgroup_subsys->disabled unused. Removed.

Cc Li Zefan <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michal Hocko <[email protected]>
---
 include/linux/cgroup-defs.h    |  1 -
 include/linux/hugetlb_cgroup.h |  4 +---
 include/linux/memcontrol.h     |  4 +---
 kernel/cgroup.c                | 28 +++++++++++++++++++++-------
 4 files changed, 23 insertions(+), 14 deletions(-)
include/linux/cgroup-defs.h +0 -1
index 4d8fcf2..c5d41c3 100644
@@ -419,7 +419,6 @@ struct cgroup_subsys {
 		     struct task_struct *task);
 	void (*bind)(struct cgroup_subsys_state *root_css);
 
-	int disabled;
 	int early_init;
 
 	/*
include/linux/hugetlb_cgroup.h +1 -3
index bcc853e..7edd305 100644
@@ -48,9 +48,7 @@ int set_hugetlb_cgroup(struct page *page, struct hugetlb_cgroup *h_cg)
 
 static inline bool hugetlb_cgroup_disabled(void)
 {
-	if (hugetlb_cgrp_subsys.disabled)
-		return true;
-	return false;
+	return !cgroup_subsys_enabled(hugetlb_cgrp_subsys);
 }
 
 extern int hugetlb_cgroup_charge_cgroup(int idx, unsigned long nr_pages,
include/linux/memcontrol.h +1 -3
index ad800e6..9aa7820 100644
@@ -347,9 +347,7 @@ ino_t page_cgroup_ino(struct page *page);
 
 static inline bool mem_cgroup_disabled(void)
 {
-	if (memory_cgrp_subsys.disabled)
-		return true;
-	return false;
+	return !cgroup_subsys_enabled(memory_cgrp_subsys);
 }
 
 /*
kernel/cgroup.c +21 -7
index 3619389..5703ba7 100644
@@ -224,6 +224,19 @@ static void kill_css(struct cgroup_subsys_state *css);
 static int cgroup_addrm_files(struct cgroup *cgrp, struct cftype cfts[],
 			      bool is_add);
 
+/**
+ * cgroup_ssid_enabled - cgroup subsys enabled test by subsys ID
+ * @ssid: subsys ID of interest
+ *
+ * cgroup_subsys_enabled() can only be used with literal subsys names which
+ * is fine for individual subsystems but unsuitable for cgroup core.  This
+ * is slower static_key_enabled() based test indexed by @ssid.
+ */
+static bool cgroup_ssid_enabled(int ssid)
+{
+	return static_key_enabled(cgroup_subsys_enabled_key[ssid]);
+}
+
 /* IDR wrappers which synchronize using cgroup_idr_lock */
 static int cgroup_idr_alloc(struct idr *idr, void *ptr, int start, int end,
 			    gfp_t gfp_mask)
@@ -1482,7 +1495,7 @@ static int parse_cgroupfs_options(char *data, struct cgroup_sb_opts *opts)
 		for_each_subsys(ss, i) {
 			if (strcmp(token, ss->legacy_name))
 				continue;
-			if (ss->disabled)
+			if (!cgroup_ssid_enabled(i))
 				continue;
 
 			/* Mutually exclusive option 'all' + subsystem name */
@@ -1513,7 +1526,7 @@ static int parse_cgroupfs_options(char *data, struct cgroup_sb_opts *opts)
 	 */
 	if (all_ss || (!one_ss && !opts->none && !opts->name))
 		for_each_subsys(ss, i)
-			if (!ss->disabled)
+			if (cgroup_ssid_enabled(i))
 				opts->subsys_mask |= (1 << i);
 
 	/*
@@ -2762,7 +2775,8 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
 		if (tok[0] == '\0')
 			continue;
 		for_each_subsys_which(ss, ssid, &tmp_ss_mask) {
-			if (ss->disabled || strcmp(tok + 1, ss->name))
+			if (!cgroup_ssid_enabled(ssid) ||
+			    strcmp(tok + 1, ss->name))
 				continue;
 
 			if (*tok == '+') {
@@ -3320,7 +3334,7 @@ static int cgroup_add_cftypes(struct cgroup_subsys *ss, struct cftype *cfts)
 {
 	int ret;
 
-	if (ss->disabled)
+	if (!cgroup_ssid_enabled(ss->id))
 		return 0;
 
 	if (!cfts || cfts[0].name[0] == '\0')
@@ -5082,7 +5096,7 @@ int __init cgroup_init(void)
 		 * disabled flag and cftype registration needs kmalloc,
 		 * both of which aren't available during early_init.
 		 */
-		if (ss->disabled)
+		if (!cgroup_ssid_enabled(ssid))
 			continue;
 
 		cgrp_dfl_root.subsys_mask |= 1 << ss->id;
@@ -5217,7 +5231,8 @@ static int proc_cgroupstats_show(struct seq_file *m, void *v)
 	for_each_subsys(ss, i)
 		seq_printf(m, "%s\t%d\t%d\t%d\n",
 			   ss->legacy_name, ss->root->hierarchy_id,
-			   atomic_read(&ss->root->nr_cgrps), !ss->disabled);
+			   atomic_read(&ss->root->nr_cgrps),
+			   cgroup_ssid_enabled(i));
 
 	mutex_unlock(&cgroup_mutex);
 	return 0;
@@ -5508,7 +5523,6 @@ static int __init cgroup_disable(char *str)
 				continue;
 
 			static_branch_disable(cgroup_subsys_enabled_key[i]);
-			ss->disabled = 1;
 			printk(KERN_INFO "Disabling %s control group subsystem\n",
 			       ss->name);
 			break;
Signature · 1 line
2.4.3
author maintainer
#7 lore raw

PATCH 4/4 cgroup: replace cgroup_on_dfl() tests in controllers with cgroup_subsys_on_dfl()

In reply to Tejun Heo

cgroup_on_dfl() tests whether the cgroup's root is the default hierarchy; however, an individual controller is only interested in whether the controller is attached to the default hierarchy and never tests a cgroup which doesn't belong to the hierarchy that the controller is attached to.

This patch replaces cgroup_on_dfl() tests in controllers with faster static_key based cgroup_subsys_on_dfl(). This leaves cgroup core as the only user of cgroup_on_dfl() and the function is moved from the header file to cgroup.c.

Cc Vivek Goyal <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Li Zefan <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michal Hocko <[email protected]>
---
 block/blk-throttle.c   |  2 +-
 block/cfq-iosched.c    |  4 ++--
 include/linux/cgroup.h | 58 --------------------------------------------------
 kernel/cgroup.c        | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/cpuset.c        | 23 +++++++++++---------
 mm/memcontrol.c        |  4 ++--
 6 files changed, 76 insertions(+), 73 deletions(-)
block/blk-throttle.c +1 -1
index c75a263..2149a1d 100644
@@ -369,7 +369,7 @@ static void throtl_pd_init(struct blkg_policy_data *pd)
 	 * regardless of the position of the group in the hierarchy.
 	 */
 	sq->parent_sq = &td->service_queue;
-	if (cgroup_on_dfl(blkg->blkcg->css.cgroup) && blkg->parent)
+	if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent)
 		sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue;
 	tg->td = td;
 }
block/cfq-iosched.c +2 -2
index 04de884..1f9093e 100644
@@ -1581,7 +1581,7 @@ static struct blkcg_policy_data *cfq_cpd_alloc(gfp_t gfp)
 static void cfq_cpd_init(struct blkcg_policy_data *cpd)
 {
 	struct cfq_group_data *cgd = cpd_to_cfqgd(cpd);
-	unsigned int weight = cgroup_on_dfl(blkcg_root.css.cgroup) ?
+	unsigned int weight = cgroup_subsys_on_dfl(io_cgrp_subsys) ?
 			      CGROUP_WEIGHT_DFL : CFQ_WEIGHT_LEGACY_DFL;
 
 	if (cpd_to_blkcg(cpd) == &blkcg_root)
@@ -1599,7 +1599,7 @@ static void cfq_cpd_free(struct blkcg_policy_data *cpd)
 static void cfq_cpd_bind(struct blkcg_policy_data *cpd)
 {
 	struct blkcg *blkcg = cpd_to_blkcg(cpd);
-	bool on_dfl = cgroup_on_dfl(blkcg_root.css.cgroup);
+	bool on_dfl = cgroup_subsys_on_dfl(io_cgrp_subsys);
 	unsigned int weight = on_dfl ? CGROUP_WEIGHT_DFL : CFQ_WEIGHT_LEGACY_DFL;
 
 	if (blkcg == &blkcg_root)
include/linux/cgroup.h +0 -58

64 lines not on this page · raw message

kernel/cgroup.c +58 -0

64 lines not on this page · raw message

kernel/cpuset.c +13 -10
index f0acff0..20eedd8 100644
@@ -473,7 +473,8 @@ static int validate_change(struct cpuset *cur, struct cpuset *trial)
 
 	/* On legacy hiearchy, we must be a subset of our parent cpuset. */
 	ret = -EACCES;
-	if (!cgroup_on_dfl(cur->css.cgroup) && !is_cpuset_subset(trial, par))
+	if (!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+	    !is_cpuset_subset(trial, par))
 		goto out;
 
 	/*
@@ -879,7 +880,8 @@ static void update_cpumasks_hier(struct cpuset *cs, struct cpumask *new_cpus)
 		 * If it becomes empty, inherit the effective mask of the
 		 * parent, which is guaranteed to have some CPUs.
 		 */
-		if (cgroup_on_dfl(cp->css.cgroup) && cpumask_empty(new_cpus))
+		if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+		    cpumask_empty(new_cpus))
 			cpumask_copy(new_cpus, parent->effective_cpus);
 
 		/* Skip the whole subtree if the cpumask remains the same. */
@@ -896,7 +898,7 @@ static void update_cpumasks_hier(struct cpuset *cs, struct cpumask *new_cpus)
 		cpumask_copy(cp->effective_cpus, new_cpus);
 		spin_unlock_irq(&callback_lock);
 
-		WARN_ON(!cgroup_on_dfl(cp->css.cgroup) &&
+		WARN_ON(!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
 			!cpumask_equal(cp->cpus_allowed, cp->effective_cpus));
 
 		update_tasks_cpumask(cp);
@@ -1135,7 +1137,8 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)
 		 * If it becomes empty, inherit the effective mask of the
 		 * parent, which is guaranteed to have some MEMs.
 		 */
-		if (cgroup_on_dfl(cp->css.cgroup) && nodes_empty(*new_mems))
+		if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+		    nodes_empty(*new_mems))
 			*new_mems = parent->effective_mems;
 
 		/* Skip the whole subtree if the nodemask remains the same. */
@@ -1152,7 +1155,7 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)
 		cp->effective_mems = *new_mems;
 		spin_unlock_irq(&callback_lock);
 
-		WARN_ON(!cgroup_on_dfl(cp->css.cgroup) &&
+		WARN_ON(!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
 			!nodes_equal(cp->mems_allowed, cp->effective_mems));
 
 		update_tasks_nodemask(cp);
@@ -1440,7 +1443,7 @@ static int cpuset_can_attach(struct cgroup_subsys_state *css,
 
 	/* allow moving tasks into an empty cpuset if on default hierarchy */
 	ret = -ENOSPC;
-	if (!cgroup_on_dfl(css->cgroup) &&
+	if (!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
 	    (cpumask_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed)))
 		goto out_unlock;
 
@@ -1952,7 +1955,7 @@ static int cpuset_css_online(struct cgroup_subsys_state *css)
 	cpuset_inc();
 
 	spin_lock_irq(&callback_lock);
-	if (cgroup_on_dfl(cs->css.cgroup)) {
+	if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys)) {
 		cpumask_copy(cs->effective_cpus, parent->effective_cpus);
 		cs->effective_mems = parent->effective_mems;
 	}
@@ -2029,7 +2032,7 @@ static void cpuset_bind(struct cgroup_subsys_state *root_css)
 	mutex_lock(&cpuset_mutex);
 	spin_lock_irq(&callback_lock);
 
-	if (cgroup_on_dfl(root_css->cgroup)) {
+	if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys)) {
 		cpumask_copy(top_cpuset.cpus_allowed, cpu_possible_mask);
 		top_cpuset.mems_allowed = node_possible_map;
 	} else {
@@ -2210,7 +2213,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs)
 	cpus_updated = !cpumask_equal(&new_cpus, cs->effective_cpus);
 	mems_updated = !nodes_equal(new_mems, cs->effective_mems);
 
-	if (cgroup_on_dfl(cs->css.cgroup))
+	if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys))
 		hotplug_update_tasks(cs, &new_cpus, &new_mems,
 				     cpus_updated, mems_updated);
 	else
@@ -2241,7 +2244,7 @@ static void cpuset_hotplug_workfn(struct work_struct *work)
 	static cpumask_t new_cpus;
 	static nodemask_t new_mems;
 	bool cpus_updated, mems_updated;
-	bool on_dfl = cgroup_on_dfl(top_cpuset.css.cgroup);
+	bool on_dfl = cgroup_subsys_on_dfl(cpuset_cgrp_subsys);
 
 	mutex_lock(&cpuset_mutex);
 
mm/memcontrol.c +2 -2
index 6ddaeba..b35c4cc 100644
@@ -434,7 +434,7 @@ struct cgroup_subsys_state *mem_cgroup_css_from_page(struct page *page)
 
 	memcg = page->mem_cgroup;
 
-	if (!memcg || !cgroup_on_dfl(memcg->css.cgroup))
+	if (!memcg || !cgroup_subsys_on_dfl(memory_cgrp_subsys))
 		memcg = root_mem_cgroup;
 
 	rcu_read_unlock();
@@ -5059,7 +5059,7 @@ static void mem_cgroup_bind(struct cgroup_subsys_state *root_css)
 	 * guarantees that @root doesn't have any children, so turning it
 	 * on for the root memcg is enough.
 	 */
-	if (cgroup_on_dfl(root_css->cgroup))
+	if (cgroup_subsys_on_dfl(memory_cgrp_subsys))
 		root_mem_cgroup->use_hierarchy = true;
 	else
 		root_mem_cgroup->use_hierarchy = false;
Signature · 1 line
2.4.3
Acked-by on v1 1/1
#8 lore raw
In reply to Tejun Heo
Quoted 23 lines · Tejun Heo

cgroup_subsys->disabled and cgroup_on_dfl() tests are likely to be used in hot paths and seldom change. The former is set once during boot and the latter only when a controller is migrated between the default hierarchy and traditional ones.

This patchset makes these tests static_key based and contains the following four patches.

0001-jump_label-make-static_key_enabled-work-on-static_ke.patch 0002-cgroup-implement-static_key-based-cgroup_subsys_enab.patch 0003-cgroup-replace-cgroup_subsys-disabled-tests-with-cgr.patch 0004-cgroup-replace-cgroup_on_dfl-tests-in-controllers-wi.patch

0001 is a prep patch in jump_label. 0002 adds the needed static_keys. 0003-0004 convert the existing usages and drop the old tests.

This patchset is on top of v4.3-rc1 and is availalbe in the following git branch.

git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git review-jump-labels

diffstat follows. Thanks.

Quoted 12 lines
  block/blk-throttle.c           |    2
  block/cfq-iosched.c            |    4 -
  include/linux/cgroup-defs.h    |    1
  include/linux/cgroup.h         |   79 +++++++---------------------
  include/linux/hugetlb_cgroup.h |    4 -
  include/linux/jump_label.h     |   18 +++---
  include/linux/memcontrol.h     |    4 -
  kernel/cgroup.c                |  113 ++++++++++++++++++++++++++++++++++++++---
  kernel/cpuset.c                |   23 ++++----
  mm/memcontrol.c                |    4 -
  10 files changed, 157 insertions(+), 95 deletions(-)
author maintainer
#9 lore raw
In reply to Tejun Heo
Quoted 4 lines · Tejun Heo

cgroup_subsys->disabled and cgroup_on_dfl() tests are likely to be used in hot paths and seldom change. The former is set once during boot and the latter only when a controller is migrated between the default hierarchy and traditional ones.

Applying to cgroup/for-4.4.

Thanks.

Signature · 1 line
tejun