Skip to contentSkip to conversation

cpuset: initialize effective masks when clone_children is enabled

PATCH In review Zefan Li 3 participants 7 messages Feb 13, 2015 – Mar 2, 2015 cgroup

lore
Share & tools
lore

Zefan Li

author
#0 lore raw

PATCH 1/3 cpuset: initialize effective masks when clone_children is enabled

If clone_children is enabled, effective masks won't be initialized due to the bug:

  # mount -t cgroup -o cpuset xxx /mnt
  # echo 1 > cgroup.clone_children
  # mkdir /mnt/tmp
  # cat /mnt/tmp/
  # cat cpuset.effective_cpus
  # cat cpuset.cpus
  0-15

And then this cpuset won't constrain the tasks in it.

Either the bug or the fix has no effect on unified hierarchy, as there's no clone_chidren flag there any more.

---
 kernel/cpuset.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/cpuset.c +2 -0
index 64b257f..7e9d711 100644
@@ -1992,7 +1992,9 @@ static int cpuset_css_online(struct cgroup_subsys_state *css)
 
 	spin_lock_irq(&callback_lock);
 	cs->mems_allowed = parent->mems_allowed;
+	cs->effective_mems = parent->mems_allowed;
 	cpumask_copy(cs->cpus_allowed, parent->cpus_allowed);
+	cpumask_copy(cs->effective_cpus, parent->cpus_allowed);
 	spin_unlock_irq(&callback_lock);
 out_unlock:
 	mutex_unlock(&cpuset_mutex);
Signature · 1 line
1.8.0.2
author
#1 lore raw

PATCH 2/3 cpuset: fix a warning when clearing configured masks in old hierarchy

When we clear cpuset.cpus, cpuset.effective_cpus won't be cleared:

  # mount -t cgroup -o cpuset xxx /mnt
  # mkdir /mnt/tmp
  # echo 0 > /mnt/tmp/cpuset.cpus
  # echo > /mnt/tmp/cpuset.cpus
  # cat cpuset.cpus
  # cat cpuset.effective_cpus
  0-15

And a kernel warning in update_cpumasks_hier() is triggered:

 ------------[ cut here ]------------
 WARNING: CPU: 0 PID: 4028 at kernel/cpuset.c:894 update_cpumasks_hier+0x471/0x650()
---
 kernel/cpuset.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/cpuset.c +2 -2
index 7e9d711..29463c2 100644
@@ -873,7 +873,7 @@ static void update_cpumasks_hier(struct cpuset *cs, struct cpumask *new_cpus)
 		 * If it becomes empty, inherit the effective mask of the
 		 * parent, which is guaranteed to have some CPUs.
 		 */
-		if (cpumask_empty(new_cpus))
+		if (cgroup_on_dfl(cp->css.cgroup) && cpumask_empty(new_cpus))
 			cpumask_copy(new_cpus, parent->effective_cpus);
 
 		/* Skip the whole subtree if the cpumask remains the same. */
@@ -1129,7 +1129,7 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)
 		 * If it becomes empty, inherit the effective mask of the
 		 * parent, which is guaranteed to have some MEMs.
 		 */
-		if (nodes_empty(*new_mems))
+		if (cgroup_on_dfl(cp->css.cgroup) && nodes_empty(*new_mems))
 			*new_mems = parent->effective_mems;
 
 		/* Skip the whole subtree if the nodemask remains the same. */
Signature · 1 line
1.8.0.2
author
#2 lore raw
In reply to Zefan Li

From: Jason Low <[email protected]>

The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.

The reason this occurred was because the update_domain_attr_tree() traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying the sched_relax_domain_level parameter.

This patch is able to address that problem by having update_domain_attr_tree() allow updates for the root in the cpuset traversal.

Fixes fc560a26acce ("cpuset: replace cpuset->stack_list with cpuset_for_each_descendant_pre()")
Cc <[email protected]> # 3.9+
---
 kernel/cpuset.c | 3 ---
 1 file changed, 3 deletions(-)
kernel/cpuset.c +0 -3
index 29463c2..9e25599 100644
@@ -548,9 +548,6 @@ static void update_domain_attr_tree(struct sched_domain_attr *dattr,
 
 	rcu_read_lock();
 	cpuset_for_each_descendant_pre(cp, pos_css, root_cs) {
-		if (cp == root_cs)
-			continue;
-
 		/* skip the whole subtree if @cp doesn't have any CPU */
 		if (cpumask_empty(cp->cpus_allowed)) {
 			pos_css = css_rightmost_descendant(pos_css);
Signature · 1 line
1.8.0.2
author
#3 lore raw

PATCH 3/3 cpuset: Fix cpuset sched_relax_domain_level

In reply to Zefan Li

From: Jason Low <[email protected]>

The cpuset.sched_relax_domain_level can control how far we do immediate load balancing on a system. However, it was found on recent kernels that echo'ing a value into cpuset.sched_relax_domain_level did not reduce any immediate load balancing.

The reason this occurred was because the update_domain_attr_tree() traversal did not update for the "top_cpuset". This resulted in nothing being changed when modifying the sched_relax_domain_level parameter.

This patch is able to address that problem by having update_domain_attr_tree() allow updates for the root in the cpuset traversal.

Fixes fc560a26acce ("cpuset: replace cpuset->stack_list with cpuset_for_each_descendant_pre()")
Cc <[email protected]> # 3.9+

---

This is a resend. I forgot to edit the subject when sending this patch...

---
 kernel/cpuset.c | 3 ---
 1 file changed, 3 deletions(-)
kernel/cpuset.c +0 -3
index 29463c2..9e25599 100644
@@ -548,9 +548,6 @@ static void update_domain_attr_tree(struct sched_domain_attr *dattr,
 
 	rcu_read_lock();
 	cpuset_for_each_descendant_pre(cp, pos_css, root_cs) {
-		if (cp == root_cs)
-			continue;
-
 		/* skip the whole subtree if @cp doesn't have any CPU */
 		if (cpumask_empty(cp->cpus_allowed)) {
 			pos_css = css_rightmost_descendant(pos_css);

-- 1.8.0.2

Tested-by Serge Hallyn on v1 1/3
#4 lore raw
In reply to Zefan Li
Quoted 12 lines · Zefan Li

If clone_children is enabled, effective masks won't be initialized due to the bug:

  # mount -t cgroup -o cpuset xxx /mnt
  # echo 1 > cgroup.clone_children
  # mkdir /mnt/tmp
  # cat /mnt/tmp/
  # cat cpuset.effective_cpus
  # cat cpuset.cpus
  0-15
Quoted 7 lines

And then this cpuset won't constrain the tasks in it.

Either the bug or the fix has no effect on unified hierarchy, as there's no clone_chidren flag there any more.

Thanks - this give sme the correct output in /proc/self/status and cpuest.cpus. (I didn't do a stress test but that seems unlikely to be broken)

Quoted 28 lines
---
 kernel/cpuset.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/cpuset.c +2 -0
index 64b257f..7e9d711 100644
@@ -1992,7 +1992,9 @@ static int cpuset_css_online(struct cgroup_subsys_state *css)
 
 	spin_lock_irq(&callback_lock);
 	cs->mems_allowed = parent->mems_allowed;
+	cs->effective_mems = parent->mems_allowed;
 	cpumask_copy(cs->cpus_allowed, parent->cpus_allowed);
+	cpumask_copy(cs->effective_cpus, parent->cpus_allowed);
 	spin_unlock_irq(&callback_lock);
 out_unlock:
 	mutex_unlock(&cpuset_mutex);
Signature · 7 lines
1.8.0.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
author
#5 lore raw
In reply to Zefan Li

Hi Tejun,

Could you pick up those bug fixes? Or should I ask Ingo/akpm to take care of them?

Quoted 40 lines · Zefan Li

If clone_children is enabled, effective masks won't be initialized due to the bug:

  # mount -t cgroup -o cpuset xxx /mnt
  # echo 1 > cgroup.clone_children
  # mkdir /mnt/tmp
  # cat /mnt/tmp/
  # cat cpuset.effective_cpus
  # cat cpuset.cpus
  0-15

And then this cpuset won't constrain the tasks in it.

Either the bug or the fix has no effect on unified hierarchy, as there's no clone_chidren flag there any more.

---
 kernel/cpuset.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/cpuset.c +2 -0
index 64b257f..7e9d711 100644
@@ -1992,7 +1992,9 @@ static int cpuset_css_online(struct cgroup_subsys_state *css)
 
 	spin_lock_irq(&callback_lock);
 	cs->mems_allowed = parent->mems_allowed;
+	cs->effective_mems = parent->mems_allowed;
 	cpumask_copy(cs->cpus_allowed, parent->cpus_allowed);
+	cpumask_copy(cs->effective_cpus, parent->cpus_allowed);
 	spin_unlock_irq(&callback_lock);
 out_unlock:
 	mutex_unlock(&cpuset_mutex);
maintainer
#6 lore raw
Quoted 4 lines · Zefan Li

Hi Tejun,

Could you pick up those bug fixes? Or should I ask Ingo/akpm to take care of them?

Sorry about the delay. Applied to cgroup/for-4.0-fixes. Will push out in a week.

Thanks.

Signature · 1 line
tejun