Skip to contentSkip to conversation

cpuset, sched: Fix cpuset sched_relax_domain_level

RFC PATCH Superseded Jason Low 2 participants 3 messages Jan 28, 2015 – Jan 30, 2015 cgroup

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Jason Low

author
#0 lore raw

RFC PATCH 1/1 cpuset, sched: Fix cpuset sched_relax_domain_level

The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.

The reason this occurred was because the update_domain_attr_tree() traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying the sched_relax_domain_level parameter.

This patch was able to address that problem by having update_domain_attr_tree() allowing updates for the root (top_cpuset) in the cpuset traversal.

---
 kernel/cpuset.c |   12 +++++++-----
 1 files changed, 7 insertions(+), 5 deletions(-)
kernel/cpuset.c +7 -5
index 64b257f..0f58c54 100644
@@ -541,15 +541,17 @@ update_domain_attr(struct sched_domain_attr *dattr, struct cpuset *c)
 }
 
 static void update_domain_attr_tree(struct sched_domain_attr *dattr,
-				    struct cpuset *root_cs)
+				    struct cpuset *root_cs, bool update_root)
 {
 	struct cpuset *cp;
 	struct cgroup_subsys_state *pos_css;
 
 	rcu_read_lock();
 	cpuset_for_each_descendant_pre(cp, pos_css, root_cs) {
-		if (cp == root_cs)
-			continue;
+		if (cp == root_cs) {
+			if (!update_root)
+				continue;
+		}
 
 		/* skip the whole subtree if @cp doesn't have any CPU */
 		if (cpumask_empty(cp->cpus_allowed)) {
@@ -644,7 +646,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
 		dattr = kmalloc(sizeof(struct sched_domain_attr), GFP_KERNEL);
 		if (dattr) {
 			*dattr = SD_ATTR_INIT;
-			update_domain_attr_tree(dattr, &top_cpuset);
+			update_domain_attr_tree(dattr, &top_cpuset, true);
 		}
 		cpumask_copy(doms[0], top_cpuset.effective_cpus);
 
@@ -752,7 +754,7 @@ restart:
 			if (apn == b->pn) {
 				cpumask_or(dp, dp, b->effective_cpus);
 				if (dattr)
-					update_domain_attr_tree(dattr + nslot, b);
+					update_domain_attr_tree(dattr + nslot, b, false);
 
 				/* Done with this partition */
 				b->pn = -1;
Signature · 1 line
1.7.1
#1 lore raw
Quoted 5 lines · Jason Low

The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.

Quoted 8 lines

The reason this occurred was because the update_domain_attr_tree() traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying the sched_relax_domain_level parameter.

This patch was able to address that problem by having update_domain_attr_tree() allowing updates for the root (top_cpuset) in the cpuset traversal.

Thanks for finding this bug!

Please Add:

Cc <[email protected]> # 3.9+
Fixes fc560a26acce ("cpuset: replace cpuset->stack_list with cpuset_for_each_descendant_pre()")

I'll prepare a different fix for 3.10.y when this patch hits mainline.

Quoted 12 lines
---
 kernel/cpuset.c |   12 +++++++-----
 1 files changed, 7 insertions(+), 5 deletions(-)
kernel/cpuset.c +0 -0
index 64b257f..0f58c54 100644
@@ -541,15 +541,17 @@ update_domain_attr(struct sched_domain_attr *dattr, struct cpuset *c)
 }
 
 static void update_domain_attr_tree(struct sched_domain_attr *dattr,

kernel/cpuset.c update_domain_attr(struct sched_domain_attr *dattr, struct cpuset *c) · line 552

Quoted 10 lines
kernel/cpuset.c +1 -3
-				    struct cpuset *root_cs)
+				    struct cpuset *root_cs, bool update_root)
 {
 	struct cpuset *cp;
 	struct cgroup_subsys_state *pos_css;
 
 	rcu_read_lock();
 	cpuset_for_each_descendant_pre(cp, pos_css, root_cs) {
-		if (cp == root_cs)
-			continue;

I don't think this fix is correct. We should simply remove these two lines, and no other changes are needed.

Quoted 26 lines
kernel/cpuset.c +6 -2
+		if (cp == root_cs) {
+			if (!update_root)
+				continue;
+		}
 
 		/* skip the whole subtree if @cp doesn't have any CPU */
 		if (cpumask_empty(cp->cpus_allowed)) {
@@ -644,7 +646,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
 		dattr = kmalloc(sizeof(struct sched_domain_attr), GFP_KERNEL);
 		if (dattr) {
 			*dattr = SD_ATTR_INIT;
-			update_domain_attr_tree(dattr, &top_cpuset);
+			update_domain_attr_tree(dattr, &top_cpuset, true);
 		}
 		cpumask_copy(doms[0], top_cpuset.effective_cpus);
 
@@ -752,7 +754,7 @@ restart:
 			if (apn == b->pn) {
 				cpumask_or(dp, dp, b->effective_cpus);
 				if (dattr)
-					update_domain_attr_tree(dattr + nslot, b);
+					update_domain_attr_tree(dattr + nslot, b, false);
 
 				/* Done with this partition */
 				b->pn = -1;
author
#2 lore raw
Quoted 13 lines · Zefan Li
Quoted 12 lines · Jason Low

The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.

The reason this occurred was because the update_domain_attr_tree() traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying the sched_relax_domain_level parameter.

This patch was able to address that problem by having update_domain_attr_tree() allowing updates for the root (top_cpuset) in the cpuset traversal.

Quoted 8 lines
Quoted 1 line

Thanks for finding this bug!

Please Add:

Cc <[email protected]> # 3.9+
Fixes fc560a26acce ("cpuset: replace cpuset->stack_list with cpuset_for_each_descendant_pre()")

Okay, I will send out a new version.

Thanks,
Jason