cpuset, sched: Fix cpuset sched_relax_domain_level
RFC PATCH 1/1 cpuset, sched: Fix cpuset sched_relax_domain_level
The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.
The reason this occurred was because the update_domain_attr_tree() traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying the sched_relax_domain_level parameter.
This patch was able to address that problem by having update_domain_attr_tree() allowing updates for the root (top_cpuset) in the cpuset traversal.
--- kernel/cpuset.c | 12 +++++++----- 1 files changed, 7 insertions(+), 5 deletions(-)
kernel/cpuset.c +7 -5
@@ -541,15 +541,17 @@ update_domain_attr(struct sched_domain_attr *dattr, struct cpuset *c) }static void update_domain_attr_tree (struct sched_domain_attr *dattr, -struct cpuset *root_cs) +struct cpuset *root_cs,bool update_root) {struct cpuset *cp;struct cgroup_subsys_state *pos_css;rcu_read_lock ();cpuset_for_each_descendant_pre (cp, pos_css, root_cs) { -if (cp == root_cs) -continue ; +if (cp == root_cs) { +if (!update_root) +continue ; + }/* skip the whole subtree if @cp doesn't have any CPU */ if (cpumask_empty (cp->cpus_allowed)) {
@@ -644,7 +646,7 @@ static int generate_sched_domains(cpumask_var_t **domains, dattr =kmalloc (sizeof (struct sched_domain_attr ),GFP_KERNEL );if (dattr) { *dattr =SD_ATTR_INIT ; -update_domain_attr_tree (dattr, &top_cpuset); +update_domain_attr_tree (dattr, &top_cpuset, true); }cpumask_copy (doms[0 ], top_cpuset.effective_cpus);
@@ -752,7 +754,7 @@ restart:if (apn == b->pn) {cpumask_or (dp, dp, b->effective_cpus);if (dattr) -update_domain_attr_tree (dattr + nslot, b); +update_domain_attr_tree (dattr + nslot, b, false);/* Done with this partition */ b->pn =-1 ;
Signature · 1 line
1.7.1
Quoted 5 lines · Jason Low
The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.
Quoted 8 lines
The reason this occurred was because the
update_domain_attr_tree()traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying thesched_relax_domain_levelparameter.This patch was able to address that problem by having
update_domain_attr_tree()allowing updates for the root (top_cpuset) in the cpuset traversal.Signed-off-by Jason Low <[email protected]>
Thanks for finding this bug!
Please Add:
I'll prepare a different fix for 3.10.y when this patch hits mainline.
Quoted 12 lines
--- kernel/cpuset.c | 12 +++++++----- 1 files changed, 7 insertions(+), 5 deletions(-)kernel/cpuset.c +0
-0@@ -541,15 +541,17 @@ update_domain_attr(struct sched_domain_attr *dattr, struct cpuset *c) } static void update_domain_attr_tree(struct sched_domain_attr *dattr,
kernel/cpuset.c update_domain_attr(struct sched_domain_attr *dattr, struct cpuset *c) · line 552
Quoted 10 lines
kernel/cpuset.c +1
-3- struct cpuset *root_cs) + struct cpuset *root_cs, bool update_root) {struct cpuset *cp;struct cgroup_subsys_state *pos_css;rcu_read_lock ();cpuset_for_each_descendant_pre (cp, pos_css, root_cs) { -if (cp == root_cs) -continue ;
I don't think this fix is correct. We should simply remove these two lines, and no other changes are needed.
Quoted 26 lines
kernel/cpuset.c +6
-2+if (cp == root_cs) { +if (!update_root) +continue ; + }/* skip the whole subtree if @cp doesn't have any CPU */ if (cpumask_empty (cp->cpus_allowed)) {@@ -644,7 +646,7 @@ static int generate_sched_domains(cpumask_var_t **domains, dattr =kmalloc (sizeof (struct sched_domain_attr ),GFP_KERNEL );if (dattr) { *dattr =SD_ATTR_INIT ; -update_domain_attr_tree (dattr, &top_cpuset); +update_domain_attr_tree (dattr, &top_cpuset, true); }cpumask_copy (doms[0 ], top_cpuset.effective_cpus);@@ -752,7 +754,7 @@ restart:if (apn == b->pn) {cpumask_or (dp, dp, b->effective_cpus);if (dattr) -update_domain_attr_tree (dattr + nslot, b); +update_domain_attr_tree (dattr + nslot, b, false);/* Done with this partition */ b->pn =-1 ;
Quoted 13 lines · Zefan Li
Quoted 12 lines · Jason Low
The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.
The reason this occurred was because the
update_domain_attr_tree()traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying thesched_relax_domain_levelparameter.This patch was able to address that problem by having
update_domain_attr_tree()allowing updates for the root (top_cpuset) in the cpuset traversal.
Quoted 8 lines
Quoted 1 line
Signed-off-by Jason Low <[email protected]>Thanks for finding this bug!
Please Add:
Cc <[email protected]> # 3.9+Fixes fc560a26acce ("cpuset: replace cpuset->stack_list with cpuset_for_each_descendant_pre()")
Okay, I will send out a new version.
Thanks,
Jason