Skip to contentSkip to conversation

cpusets,isolcpus: exclude isolcpus from load balancing in cpusets

PATCH RESEND In review [email protected] 3 participants 14 messages Mar 9, 2015 – Mar 19, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v4 cover cpusets,isolcpus: exclude isolcpus from load balancing in cpusets

Ensure that cpus specified with the isolcpus= boot commandline option stay outside of the load balancing in the kernel scheduler.

Operations like load balancing can introduce unwanted latencies, which is exactly what the isolcpus= commandline is there to prevent.

Previously, simply creating a new cpuset, without even touching the cpuset.cpus field inside the new cpuset, would undo the effects of isolcpus=, by creating a scheduler domain spanning the whole system, and setting up load balancing inside that domain. The cpuset root cpuset.cpus file is read-only, so there was not even a way to undo that effect.

This does not impact the majority of cpusets users, since isolcpus= is a fairly specialized feature used for realtime purposes.

This version fixes the UP compilation issue, in the same way done for the other cpumasks.

#1 lore raw

PATCH v4 1/4 sched,isolcpu: make cpu_isolated_map visible outside scheduler

From: Rik van Riel <[email protected]>

Needed by the next patch. Also makes cpu_isolated_map present when compiled without SMP and/or with CONFIG_NR_CPUS=1, like the other cpu masks.

At some point we may want to clean things up so cpumasks do not exist in UP kernels. Maybe something for the CONFIG_TINY crowd.

Cc Peter Zijlstra <[email protected]>
Cc Clark Williams <[email protected]>
Cc Li Zefan <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc David Rientjes <[email protected]>
Cc Mike Galbraith <[email protected]>
---
 include/linux/sched.h | 2 ++
 kernel/sched/core.c   | 6 +++---
 2 files changed, 5 insertions(+), 3 deletions(-)
include/linux/sched.h +2 -0
index 6d77432e14ff..ca365d79480c 100644
@@ -329,6 +329,8 @@ extern asmlinkage void schedule_tail(struct task_struct *prev);
 extern void init_idle(struct task_struct *idle, int cpu);
 extern void init_idle_bootup_task(struct task_struct *idle);
 
+extern cpumask_var_t cpu_isolated_map;
+
 extern int runqueue_is_locked(int cpu);
 
 #if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
kernel/sched/core.c +3 -3
index f0f831e8a345..b578bb23410b 100644
@@ -306,6 +306,9 @@ __read_mostly int scheduler_running;
  */
 int sysctl_sched_rt_runtime = 950000;
 
+/* cpus with isolated domains */
+cpumask_var_t cpu_isolated_map;
+
 /*
  * this_rq_lock - lock this runqueue and disable interrupts.
  */
@@ -5811,9 +5814,6 @@ cpu_attach_domain(struct sched_domain *sd, struct root_domain *rd, int cpu)
 	update_top_cache_domain(cpu);
 }
 
-/* cpus with isolated domains */
-static cpumask_var_t cpu_isolated_map;
-
 /* Setup the mask of cpus configured for isolated domains */
 static int __init isolated_cpu_setup(char *str)
 {
Signature · 1 line
2.1.0
#2 lore raw

PATCH v4 2/4 cpusets,isolcpus: exclude isolcpus from load balancing in cpusets

In reply to [email protected]

From: Rik van Riel <[email protected]>

Ensure that cpus specified with the isolcpus= boot commandline option stay outside of the load balancing in the kernel scheduler.

Operations like load balancing can introduce unwanted latencies, which is exactly what the isolcpus= commandline is there to prevent.

Previously, simply creating a new cpuset, without even touching the cpuset.cpus field inside the new cpuset, would undo the effects of isolcpus=, by creating a scheduler domain spanning the whole system, and setting up load balancing inside that domain. The cpuset root cpuset.cpus file is read-only, so there was not even a way to undo that effect.

This does not impact the majority of cpusets users, since isolcpus= is a fairly specialized feature used for realtime purposes.

Cc Peter Zijlstra <[email protected]>
Cc Clark Williams <[email protected]>
Cc Li Zefan <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
---
 kernel/cpuset.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
kernel/cpuset.c +11 -2
index 1d1fe9361d29..b544e5229d99 100644
@@ -625,6 +625,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
 	int csn;		/* how many cpuset ptrs in csa so far */
 	int i, j, k;		/* indices for partition finding loops */
 	cpumask_var_t *doms;	/* resulting partition; i.e. sched domains */
+	cpumask_var_t non_isolated_cpus;  /* load balanced CPUs */
 	struct sched_domain_attr *dattr;  /* attributes for custom domains */
 	int ndoms = 0;		/* number of sched domains in result */
 	int nslot;		/* next empty doms[] struct cpumask slot */
@@ -634,6 +635,10 @@ static int generate_sched_domains(cpumask_var_t **domains,
 	dattr = NULL;
 	csa = NULL;
 
+	if (!alloc_cpumask_var(&non_isolated_cpus, GFP_KERNEL))
+		goto done;
+	cpumask_andnot(non_isolated_cpus, cpu_possible_mask, cpu_isolated_map);
+
 	/* Special case for the 99% of systems with one, full, sched domain */
 	if (is_sched_load_balance(&top_cpuset)) {
 		ndoms = 1;
@@ -646,7 +651,8 @@ static int generate_sched_domains(cpumask_var_t **domains,
 			*dattr = SD_ATTR_INIT;
 			update_domain_attr_tree(dattr, &top_cpuset);
 		}
-		cpumask_copy(doms[0], top_cpuset.effective_cpus);
+		cpumask_and(doms[0], top_cpuset.effective_cpus,
+				     non_isolated_cpus);
 
 		goto done;
 	}
@@ -669,7 +675,8 @@ static int generate_sched_domains(cpumask_var_t **domains,
 		 * the corresponding sched domain.
 		 */
 		if (!cpumask_empty(cp->cpus_allowed) &&
-		    !is_sched_load_balance(cp))
+		    !(is_sched_load_balance(cp) &&
+		      cpumask_intersects(cp->cpus_allowed, non_isolated_cpus)))
 			continue;
 
 		if (is_sched_load_balance(cp))
@@ -751,6 +758,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
 
 			if (apn == b->pn) {
 				cpumask_or(dp, dp, b->effective_cpus);
+				cpumask_and(dp, dp, non_isolated_cpus);
 				if (dattr)
 					update_domain_attr_tree(dattr + nslot, b);
 
@@ -763,6 +771,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
 	BUG_ON(nslot != ndoms);
 
 done:
+	free_cpumask_var(non_isolated_cpus);
 	kfree(csa);
 
 	/*
Signature · 1 line
2.1.0
#3 lore raw

PATCH v4 3/4 cpusets,isolcpus: add file to show isolated cpus in cpuset

In reply to [email protected]

From: Rik van Riel <[email protected]>

The previous patch makes it so the code skips over isolcpus when building scheduler load balancing domains. This makes it hard to see for a user which of the CPUs in a cpuset are participating in load balancing, and which ones are isolated cpus.

Add a cpuset.isolcpus file with info on which cpus in a cpuset are isolated CPUs.

This file is read-only for now. In the future we could extend things so isolcpus can be changed at run time, for the root (system wide) cpuset only.

Cc Peter Zijlstra <[email protected]>
Cc Clark Williams <[email protected]>
Cc Li Zefan <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc David Rientjes <[email protected]>
Cc Mike Galbraith <[email protected]>
---
 kernel/cpuset.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
kernel/cpuset.c +22 -0
index b544e5229d99..5462e1ca90bd 100644
@@ -1563,6 +1563,7 @@ typedef enum {
 	FILE_MEMORY_PRESSURE,
 	FILE_SPREAD_PAGE,
 	FILE_SPREAD_SLAB,
+	FILE_ISOLCPUS,
 } cpuset_filetype_t;
 
 static int cpuset_write_u64(struct cgroup_subsys_state *css, struct cftype *cft,
@@ -1704,6 +1705,16 @@ static ssize_t cpuset_write_resmask(struct kernfs_open_file *of,
 	return retval ?: nbytes;
 }
 
+/* protected by the lock in cpuset_common_seq_show */
+static cpumask_var_t print_isolated_cpus;
+
+static void cpuset_seq_print_isolcpus(struct seq_file *sf, struct cpuset *cs)
+{
+	cpumask_and(print_isolated_cpus, cs->cpus_allowed, cpu_isolated_map);
+
+	seq_printf(sf, "%*pbl\n", cpumask_pr_args(print_isolated_cpus));
+}
+
 /*
  * These ascii lists should be read in a single call, by using a user
  * buffer large enough to hold the entire map.  If read in smaller
@@ -1733,6 +1744,9 @@ static int cpuset_common_seq_show(struct seq_file *sf, void *v)
 	case FILE_EFFECTIVE_MEMLIST:
 		seq_printf(sf, "%*pbl\n", nodemask_pr_args(&cs->effective_mems));
 		break;
+	case FILE_ISOLCPUS:
+		cpuset_seq_print_isolcpus(sf, cs);
+		break;
 	default:
 		ret = -EINVAL;
 	}
@@ -1893,6 +1907,12 @@ static struct cftype files[] = {
 		.private = FILE_MEMORY_PRESSURE_ENABLED,
 	},
 
+	{
+		.name = "isolcpus",
+		.seq_show = cpuset_common_seq_show,
+		.private = FILE_ISOLCPUS,
+	},
+
 	{ }	/* terminate */
 };
 
@@ -2070,6 +2090,8 @@ int __init cpuset_init(void)
 		BUG();
 	if (!alloc_cpumask_var(&top_cpuset.effective_cpus, GFP_KERNEL))
 		BUG();
+	if (!alloc_cpumask_var(&print_isolated_cpus, GFP_KERNEL))
+		BUG();
 
 	cpumask_setall(top_cpuset.cpus_allowed);
 	nodes_setall(top_cpuset.mems_allowed);
Signature · 1 line
2.1.0
maintainer
#4 lore raw

Re: [PATCH 3/4] cpusets,isolcpus: add file to show isolated cpus in cpuset

Quoted 7 lines · On Mon, Mar 09, 2015 at 12:12:09PM -0400…

From: Rik van Riel <[email protected]>

The previous patch makes it so the code skips over isolcpus when building scheduler load balancing domains. This makes it hard to see for a user which of the CPUs in a cpuset are participating in load balancing, and which ones are isolated cpus.

Quoted 6 lines

Add a cpuset.isolcpus file with info on which cpus in a cpuset are isolated CPUs.

This file is read-only for now. In the future we could extend things so isolcpus can be changed at run time, for the root (system wide) cpuset only.

Didn't Li say that this is trivially computable from userland? I'm not sure this knob actually belongs to cpuset.

Thanks.

Signature · 1 line
tejun
#5 lore raw

Re: [PATCH 3/4] cpusets,isolcpus: add file to show isolated cpus in cpuset

Quoted 11 lines · Tejun Heo
Quoted 10 lines · On Mon, Mar 09, 2015 at 12:12:09PM -0400…

From: Rik van Riel <[email protected]>

The previous patch makes it so the code skips over isolcpus when building scheduler load balancing domains. This makes it hard to see for a user which of the CPUs in a cpuset are participating in load balancing, and which ones are isolated cpus.

Add a cpuset.isolcpus file with info on which cpus in a cpuset are isolated CPUs.

Quoted 6 lines
Quoted 3 lines

This file is read-only for now. In the future we could extend things so isolcpus can be changed at run time, for the root (system wide) cpuset only.

Didn't Li say that this is trivially computable from userland? I'm not sure this knob actually belongs to cpuset.

I don't know whether the information to compute this is always visible from userland. I am happy to drop this patch if Li prefers things that way, though.

Li?

Signature · 1 line
All rights reversed
#6 lore raw

Re: [PATCH 3/4] cpusets,isolcpus: add file to show isolated cpus in cpuset

Quoted 16 lines · Rik van Riel
Quoted 15 lines · Tejun Heo
Quoted 13 lines · On Mon, Mar 09, 2015 at 12:12:09PM -0400, [email protected] wrote:

From: Rik van Riel <[email protected]>

The previous patch makes it so the code skips over isolcpus when building scheduler load balancing domains. This makes it hard to see for a user which of the CPUs in a cpuset are participating in load balancing, and which ones are isolated cpus.

Add a cpuset.isolcpus file with info on which cpus in a cpuset are isolated CPUs.

This file is read-only for now. In the future we could extend things so isolcpus can be changed at run time, for the root (system wide) cpuset only.

Quoted 7 lines
Quoted 2 lines

Didn't Li say that this is trivially computable from userland? I'm not sure this knob actually belongs to cpuset.

I don't know whether the information to compute this is always visible from userland. I am happy to drop this patch if Li prefers things that way, though.

What I proposed is adding /sys/devices/system/cpu/isolated. Sysfs is visible in containers, unless specially configured not so.

#7 lore raw

Re: [PATCH 3/4] cpusets,isolcpus: add file to show isolated cpus in cpuset

Quoted 20 lines · Zefan Li
Quoted 19 lines · Rik van Riel
Quoted 17 lines · Tejun Heo
Quoted 7 lines · On Mon, Mar 09, 2015 at 12:12:09PM -0400…

From: Rik van Riel <[email protected]>

The previous patch makes it so the code skips over isolcpus when building scheduler load balancing domains. This makes it hard to see for a user which of the CPUs in a cpuset are participating in load balancing, and which ones are isolated cpus.

Quoted 6 lines

Add a cpuset.isolcpus file with info on which cpus in a cpuset are isolated CPUs.

This file is read-only for now. In the future we could extend things so isolcpus can be changed at run time, for the root (system wide) cpuset only.

Didn't Li say that this is trivially computable from userland? I'm not sure this knob actually belongs to cpuset.

Quoted 7 lines
Quoted 4 lines

I don't know whether the information to compute this is always visible from userland. I am happy to drop this patch if Li prefers things that way, though.

What I proposed is adding /sys/devices/system/cpu/isolated. Sysfs is visible in containers, unless specially configured not so.

OK, are you willing to take patches 1, 2, and the first hunk of patch 4 now? I can submit a patch to add /sys/devices/system/cpu/isolated on Friday, to the appropriate maintainer.

(taking tomorrow off to go hiking on the last full day of winter)

Signature · 1 line
All rights reversed
#8 lore raw

Re: [PATCH 3/4] cpusets,isolcpus: add file to show isolated cpus in cpuset

Quoted 26 lines · Rik van Riel
Quoted 25 lines · Zefan Li
Quoted 23 lines · Rik van Riel
Quoted 11 lines · Tejun Heo
Quoted 10 lines · On Mon, Mar 09, 2015 at 12:12:09PM -0400…

From: Rik van Riel <[email protected]>

The previous patch makes it so the code skips over isolcpus when building scheduler load balancing domains. This makes it hard to see for a user which of the CPUs in a cpuset are participating in load balancing, and which ones are isolated cpus.

Add a cpuset.isolcpus file with info on which cpus in a cpuset are isolated CPUs.

Quoted 6 lines
Quoted 3 lines

This file is read-only for now. In the future we could extend things so isolcpus can be changed at run time, for the root (system wide) cpuset only.

Didn't Li say that this is trivially computable from userland? I'm not sure this knob actually belongs to cpuset.

I don't know whether the information to compute this is always visible from userland. I am happy to drop this patch if Li prefers things that way, though.

Quoted 7 lines
Quoted 2 lines

What I proposed is adding /sys/devices/system/cpu/isolated. Sysfs is visible in containers, unless specially configured not so.

OK, are you willing to take patches 1, 2, and the first hunk of patch 4 now? I can submit a patch to add /sys/devices/system/cpu/isolated on Friday, to the appropriate maintainer.

Other patches look good to me.

Quoted 2 lines

(taking tomorrow off to go hiking on the last full day of winter)

Have a good day!

#9 lore raw

Re: [PATCH 3/4] cpusets,isolcpus: add file to show isolated cpus in cpuset

In reply to Tejun Heo
Quoted 11 lines · Tejun Heo
Quoted 10 lines · On Mon, Mar 09, 2015 at 12:12:09PM -0400…

From: Rik van Riel <[email protected]>

The previous patch makes it so the code skips over isolcpus when building scheduler load balancing domains. This makes it hard to see for a user which of the CPUs in a cpuset are participating in load balancing, and which ones are isolated cpus.

Add a cpuset.isolcpus file with info on which cpus in a cpuset are isolated CPUs.

Quoted 6 lines
Quoted 3 lines

This file is read-only for now. In the future we could extend things so isolcpus can be changed at run time, for the root (system wide) cpuset only.

Didn't Li say that this is trivially computable from userland? I'm not sure this knob actually belongs to cpuset.

Going back through my email, it looks like Li
previously acked this patch.

That explains why I'm confused about your comment :)

Signature · 1 line
All rights reversed
maintainer
#10 lore raw

Re: [PATCH 3/4] cpusets,isolcpus: add file to show isolated cpus in cpuset

Hello,

Quoted 2 lines · Rik van Riel

Going back through my email, it looks like Li
previously acked this patch.

Ooh, the patch didn't have acked-by. Sorry about the confusion.

Quoted 1 line

That explains why I'm confused about your comment :)

I still don't think cpuset is the right place to put this knob. This is a system-wide boot paramter, isn't it? There gotta be a place better suited for this, be that a sysctl knob or a file under sys.

Thanks.

Signature · 1 line
tejun
#11 lore raw

PATCH v4 4/4 cpuset,isolcpus: document relationship between cpusets & isolcpus

In reply to [email protected]

From: Rik van Riel <[email protected]>

Document the subtly changed relationship between cpusets and isolcpus. Turns out the old documentation did not match the code...

---
 Documentation/cgroups/cpusets.txt | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
Documentation/cgroups/cpusets.txt +8 -2
index f2235a162529..fdf7dff3f607 100644
@@ -392,8 +392,10 @@ Put simply, it costs less to balance between two smaller sched domains
 than one big one, but doing so means that overloads in one of the
 two domains won't be load balanced to the other one.
 
-By default, there is one sched domain covering all CPUs, except those
-marked isolated using the kernel boot time "isolcpus=" argument.
+By default, there is one sched domain covering all CPUs, including those
+marked isolated using the kernel boot time "isolcpus=" argument. However,
+the isolated CPUs will not participate in load balancing, and will not
+have tasks running on them unless explicitly assigned.
 
 This default load balancing across all CPUs is not well suited for
 the following two situations:
@@ -465,6 +467,10 @@ such partially load balanced cpusets, as they may be artificially
 constrained to some subset of the CPUs allowed to them, for lack of
 load balancing to the other CPUs.
 
+CPUs in "cpuset.isolcpus" were excluded from load balancing by the
+isolcpus= kernel boot option, and will never be load balanced regardless
+of the value of "cpuset.sched_load_balance" in any cpuset.
+
 1.7.1 sched_load_balance implementation details.
 ------------------------------------------------
 
Signature · 1 line
2.1.0
#12 lore raw
In reply to [email protected]
Quoted 13 lines · On 03/09/2015 12:12 PM, [email protected] wrote:

Ensure that cpus specified with the isolcpus= boot commandline option stay outside of the load balancing in the kernel scheduler.

Operations like load balancing can introduce unwanted latencies, which is exactly what the isolcpus= commandline is there to prevent.

Previously, simply creating a new cpuset, without even touching the cpuset.cpus field inside the new cpuset, would undo the effects of isolcpus=, by creating a scheduler domain spanning the whole system, and setting up load balancing inside that domain. The cpuset root cpuset.cpus file is read-only, so there was not even a way to undo that effect.

Quoted 5 lines

This does not impact the majority of cpusets users, since isolcpus= is a fairly specialized feature used for realtime purposes.

This version fixes the UP compilation issue, in the same way done for the other cpumasks.

Ping?

Tejun? Li?

maintainer
#13 lore raw
In reply to [email protected]
Quoted 13 lines · On Mon, Mar 09, 2015 at 12:12:06PM -0400, [email protected] wrote:

Ensure that cpus specified with the isolcpus= boot commandline option stay outside of the load balancing in the kernel scheduler.

Operations like load balancing can introduce unwanted latencies, which is exactly what the isolcpus= commandline is there to prevent.

Previously, simply creating a new cpuset, without even touching the cpuset.cpus field inside the new cpuset, would undo the effects of isolcpus=, by creating a scheduler domain spanning the whole system, and setting up load balancing inside that domain. The cpuset root cpuset.cpus file is read-only, so there was not even a way to undo that effect.

Quoted 5 lines

This does not impact the majority of cpusets users, since isolcpus= is a fairly specialized feature used for realtime purposes.

This version fixes the UP compilation issue, in the same way done for the other cpumasks.

Applied 1, 2, and 4 to cgroup/for-4.1. I collected the acks from the previous threads. Can you please collect them when posting new revisions in the future?

Thanks.

Signature · 1 line
tejun