Dynamic sched domains: sched changes
Subsystems (2)
PATCH 1/3 Dynamic sched domains: sched changes
Hello Andrew,
The following patches add dynamic sched domains functionality that was extensively discussed on lkml and lse-tech.
I would like to see this added to -mm
o The main advantage with this feature is that it ensures that the scheduler load balacing code only balances against the cpus that are in the sched domain as defined by an exclusive cpuset and not all of the cpus in the system. This removes any overhead due to load balancing code trying to pull tasks outside of the cpu exclusive cpuset only to be prevented by the tasks' cpus_allowed mask.
o cpu exclusive cpusets are useful for servers running orthogonal workloads such as RT applications requiring low latency and HPC applications that are throughput sensitive
o It provides a new API partition_sched_domains in sched.c that makes dynamic sched domains possible.
o cpu_exclusive cpusets sets are now associated with a sched domain. Which means that the users can dynamically modify the sched domains through the cpuset file system interface
o ia64 sched domain code has been updated to support this feature as well
o Currently, this does not support hotplug. (However some of my tests indicate hotplug+preempt is currently broken)
o I have tested it extensively on x86.
o This should have very minimal impact on performance as none of the fast paths are affected
-Dinakar
linux-2.6.12-rc4-mm1-1/include/linux/sched.h | 2 linux-2.6.12-rc4-mm1-1/kernel/sched.c | 130 +++++++++++++++-------- linux-2.6.12-rc4-mm1-2/Documentation/cpusets.txt | 16 ++ linux-2.6.12-rc4-mm1-2/kernel/cpuset.c | 89 +++++++++++++-- linux-2.6.12-rc4-mm1-3/arch/ia64/kernel/domain.c | 76 +++++++------ 5 files changed, 225 insertions(+), 88 deletions(-)
include/linux/sched.h +2 -0
@@ -561,6 +561,8 @@ struct sched_domain {#endif }; +extern void partition_sched_domains (cpumask_t *partition1, +cpumask_t *partition2);#ifdef ARCH_HAS_SCHED_DOMAIN /* Useful helpers that arch setup code may use. Defined in kernel/sched.c */ extern cpumask_t cpu_isolated_map;
kernel/sched.c +86 -44
274 lines not on this page · raw message
PATCH 2/3 Dynamic sched domains: cpuset changes
Adds the core update_cpu_domains code and updated cpusets documentation
Documentation/cpusets.txt +16 -0
@@ -51,6 +51,14 @@ mems_allowed vector. If a cpuset is cpu or mem exclusive, no other cpuset, other than a direct ancestor or descendent, may share any of the same CPUs or Memory Nodes. +A cpuset that is cpu exclusive has a sched domain associated with it. +The sched domain consists of all cpus in the current cpuset that are not +part of any exclusive child cpusets. +This ensures that the scheduler load balacing code only balances +against the cpus that are in the sched domain as defined above and not +all of the cpus in the system. This removes any overhead due to +load balancing code trying to pull tasks outside of the cpu exclusive +cpuset only to be prevented by the tasks' cpus_allowed mask. User level code may create and destroy cpusets by name in the cpuset virtual file system, manage the attributes and permissions of these
@@ -84,6 +92,9 @@ This can be especially valuable on: and a database), or * NUMA systems running large HPC applications with demanding performance characteristics. + * Also cpu_exclusive cpusets are useful for servers running orthogonal + workloads such as RT applications requiring low latency and HPC + applications that are throughput sensitive These subsets, or "soft partitions" must be able to be dynamically adjusted, as the job mix changes, without impacting other concurrently
@@ -125,6 +136,8 @@ Cpusets extends these two mechanisms as - A cpuset may be marked exclusive, which ensures that no other cpuset (except direct ancestors and descendents) may contain any overlapping CPUs or Memory Nodes. + Also a cpu_exclusive cpuset would be associated with a sched + domain. - You can list all the tasks (by pid) attached to any cpuset. The implementation of cpusets requires a few, simple hooks
@@ -136,6 +149,9 @@ into the rest of the kernel, none in per allowed in that tasks cpuset. - in sched.c migrate_all_tasks(), to keep migrating tasks within the CPUs allowed by their cpuset, if possible. + - in sched.c, a new API partition_sched_domains for handling + sched domain changes associated with cpu_exclusive cpusets + and related changes in both sched.c and arch/ia64/kernel/domain.c - in the mbind and set_mempolicy system calls, to mask the requested Memory Nodes by what's allowed in that tasks cpuset. - in page_alloc, to restrict memory to allowed nodes.
kernel/cpuset.c +76 -13
127 lines not on this page · raw message
PATCH 3/3 Dynamic sched domains: ia64 changes
In reply to Dinakar Guniguntala ia64 changes similar to kernel/sched.c. This patch compiles ok, but has not been tested
arch/ia64/kernel/domain.c +45 -31
@@ -27,7 +27,7 @@* * Should use nodemask_t. */ -static int __devinit find_next_best_node (int node,unsigned long *used_nodes) +static int find_next_best_node (int node,unsigned long *used_nodes) {int i, n, val, min_val, best_node =0 ;
@@ -66,7 +66,7 @@ static int __devinit find_next_best_node* should be one that prevents unnecessary balancing, but also spreads tasks * out optimally. */ -static cpumask_t __devinit sched_domain_node_span (int node) +static cpumask_t sched_domain_node_span (int node) {int i;cpumask_t span, nodemask;
@@ -96,7 +96,7 @@ static cpumask_t __devinit sched_domain_#ifdef CONFIG_SCHED_SMT static DEFINE_PER_CPU(struct sched_domain, cpu_domains);static struct sched_group sched_group_cpus[NR_CPUS ]; -static int __devinit cpu_to_cpu_group (int cpu) +static int cpu_to_cpu_group (int cpu) {return cpu; }
@@ -104,7 +104,7 @@ static int __devinit cpu_to_cpu_group(in static DEFINE_PER_CPU(struct sched_domain, phys_domains);static struct sched_group sched_group_phys[NR_CPUS ]; -static int __devinit cpu_to_phys_group (int cpu) +static int cpu_to_phys_group (int cpu) {#ifdef CONFIG_SCHED_SMT return first_cpu (cpu_sibling_map[cpu]);
@@ -125,44 +125,36 @@ static struct sched_group *sched_group_n static DEFINE_PER_CPU(struct sched_domain, allnodes_domains);static struct sched_group sched_group_allnodes[MAX_NUMNODES ]; -static int __devinit cpu_to_allnodes_group (int cpu) +static int cpu_to_allnodes_group (int cpu) {return cpu_to_node (cpu); }#endif /* -* Set up scheduler domains and groups. Callers must hold the hotplug lock. +* Build sched domains for a given set of cpus and attach the sched domains +* to the individual cpus */ -void __devinit arch_init_sched_domains (void ) +void build_sched_domains (const cpumask_t *cpu_map) {int i; -cpumask_t cpu_default_map;/* -* Setup mask for cpus without special case scheduling requirements. -* For now this just excludes isolated cpus, but could be used to -* exclude other special cases in the future. +* Set up domains for cpus specified by the cpu_map. */ -cpus_complement (cpu_default_map, cpu_isolated_map); -cpus_and (cpu_default_map, cpu_default_map, cpu_online_map); - -/* -* Set up domains. Isolated domains just stay on the dummy domain. -*/ -for_each_cpu_mask (i, cpu_default_map) { +for_each_cpu_mask (i, *cpu_map) {int group;struct sched_domain *sd =NULL , *p;cpumask_t nodemask =node_to_cpumask (cpu_to_node (i)); -cpus_and (nodemask, nodemask, cpu_default_map); +cpus_and (nodemask, nodemask, *cpu_map);#ifdef CONFIG_NUMA if (num_online_cpus () >SD_NODES_PER_DOMAIN *cpus_weight (nodemask)) { sd = &per_cpu (allnodes_domains, i); *sd =SD_ALLNODES_INIT ; - sd->span = cpu_default_map; + sd->span = *cpu_map; group =cpu_to_allnodes_group (i); sd->groups = &sched_group_allnodes[group]; p = sd;
@@ -173,7 +165,7 @@ void __devinit arch_init_sched_domains(v *sd =SD_NODE_INIT ; sd->span =sched_domain_node_span (cpu_to_node (i)); sd->parent = p; -cpus_and (sd->span, sd->span, cpu_default_map); +cpus_and (sd->span, sd->span, *cpu_map);#endif p = sd;
@@ -190,7 +182,7 @@ void __devinit arch_init_sched_domains(v group =cpu_to_cpu_group (i); *sd =SD_SIBLING_INIT ; sd->span = cpu_sibling_map[i]; -cpus_and (sd->span, sd->span, cpu_default_map); +cpus_and (sd->span, sd->span, *cpu_map); sd->parent = p; sd->groups = &sched_group_cpus[group];#endif
@@ -198,9 +190,9 @@ void __devinit arch_init_sched_domains(v#ifdef CONFIG_SCHED_SMT /* Set up CPU (sibling) groups */ -for_each_cpu_mask (i, cpu_default_map) { +for_each_cpu_mask (i, *cpu_map) {cpumask_t this_sibling_map = cpu_sibling_map[i]; -cpus_and (this_sibling_map, this_sibling_map, cpu_default_map); +cpus_and (this_sibling_map, this_sibling_map, *cpu_map);if (i !=first_cpu (this_sibling_map))continue ;
@@ -213,7 +205,7 @@ void __devinit arch_init_sched_domains(vfor (i =0 ; i <MAX_NUMNODES ; i++) {cpumask_t nodemask =node_to_cpumask (i); -cpus_and (nodemask, nodemask, cpu_default_map); +cpus_and (nodemask, nodemask, *cpu_map);if (cpus_empty (nodemask))continue ;
@@ -222,7 +214,7 @@ void __devinit arch_init_sched_domains(v }#ifdef CONFIG_NUMA -init_sched_build_groups (sched_group_allnodes, cpu_default_map, +init_sched_build_groups (sched_group_allnodes, *cpu_map, &cpu_to_allnodes_group);for (i =0 ; i <MAX_NUMNODES ; i++) {
@@ -233,12 +225,12 @@ void __devinit arch_init_sched_domains(vcpumask_t covered =CPU_MASK_NONE ;int j; -cpus_and (nodemask, nodemask, cpu_default_map); +cpus_and (nodemask, nodemask, *cpu_map);if (cpus_empty (nodemask))continue ; domainspan =sched_domain_node_span (i); -cpus_and (domainspan, domainspan, cpu_default_map); +cpus_and (domainspan, domainspan, *cpu_map); sg =kmalloc (sizeof (struct sched_group ),GFP_KERNEL ); sched_group_nodes[i] = sg;
@@ -266,7 +258,7 @@ void __devinit arch_init_sched_domains(vint n = (i + j) %MAX_NUMNODES ;cpus_complement (notcovered, covered); -cpus_and (tmp, notcovered, cpu_default_map); +cpus_and (tmp, notcovered, *cpu_map);cpus_and (tmp, tmp, domainspan);if (cpus_empty (tmp))break ;
@@ -293,7 +285,7 @@ void __devinit arch_init_sched_domains(v#endif /* Calculate CPU power for physical packages and nodes */ -for_each_cpu_mask (i, cpu_default_map) { +for_each_cpu_mask (i, *cpu_map) {int power;struct sched_domain *sd;#ifdef CONFIG_SCHED_SMT
@@ -359,13 +351,35 @@ next_sg:cpu_attach_domain (sd, i); } } +/* +* Set up scheduler domains and groups. Callers must hold the hotplug lock. +*/ +void arch_init_sched_domains (const cpumask_t *cpu_map) +{ +cpumask_t cpu_default_map; + +/* +* Setup mask for cpus without special case scheduling requirements. +* For now this just excludes isolated cpus, but could be used to +* exclude other special cases in the future. +*/ +cpus_andnot (cpu_default_map, *cpu_map, cpu_isolated_map); + +build_sched_domains (&cpu_default_map); +} -void __devinit arch_destroy_sched_domains (void ) +void arch_destroy_sched_domains (const cpumask_t *cpu_map) {#ifdef CONFIG_NUMA int i;for (i =0 ; i <MAX_NUMNODES ; i++) { +cpumask_t nodemask =node_to_cpumask (i);struct sched_group *oldsg, *sg = sched_group_nodes[i]; + +cpus_and (nodemask, nodemask, *cpu_map); +if (cpus_empty (nodemask)) +continue ; +if (sg ==NULL )continue ; sg = sg->next;
Quoted 34 lines · Dinakar Guniguntala
Hello Andrew,
The following patches add dynamic sched domains functionality that was extensively discussed on lkml and lse-tech.
I would like to see this added to -mmo The main advantage with this feature is that it ensures that the scheduler load balacing code only balances against the cpus that are in the sched domain as defined by an exclusive cpuset and not all of the cpus in the system. This removes any overhead due to load balancing code trying to pull tasks outside of the cpu exclusive cpuset only to be prevented by the tasks'
cpus_allowedmask.
o cpu exclusive cpusets are useful for servers running orthogonal workloads such as RT applications requiring low latency and HPC applications that are throughput sensitiveo It provides a new API
partition_sched_domainsinsched.cthat makes dynamic sched domains possible.
ocpu_exclusivecpusets sets are now associated with a sched domain. Which means that the users can dynamically modify the sched domains through the cpuset file system interface
o ia64 sched domain code has been updated to support this feature as well
o Currently, this does not support hotplug. (However some of my tests indicate hotplug+preempt is currently broken)
o I have tested it extensively on x86.
o This should have very minimal impact on performance as none of the fast paths are affected-Dinakar
Signed-off-by Dinakar Guniguntala <[email protected]>Acked-by Paul Jackson <[email protected]>Acked-by Nick Piggin <[email protected]>
Quoted 6 lines
linux-2.6.12-rc4-mm1-1/include/linux/sched.h | 2 linux-2.6.12-rc4-mm1-1/kernel/sched.c | 130 +++++++++++++++-------- linux-2.6.12-rc4-mm1-2/Documentation/cpusets.txt | 16 ++ linux-2.6.12-rc4-mm1-2/kernel/cpuset.c | 89 +++++++++++++-- linux-2.6.12-rc4-mm1-3/arch/ia64/kernel/domain.c | 76 +++++++------ 5 files changed, 225 insertions(+), 88 deletions(-)
Looks good to me, Dinakar!
-Matt