Coregroup support on Powerpc
v4 cover Coregroup support on Powerpc
Changelog v3 ->v4:
v3: https://lore.kernel.org/lkml/[email protected]/t/#u
powerpc/smp: Create coregroup domain
if coregroup_support doesn't exist, update MC mask to the next smaller domain mask.
Changelog v2 -> v3:
v2: https://lore.kernel.org/linuxppc-dev/[email protected]/t/#u
powerpc/smp: Cache node for reuse
Removed node caching part. Rewrote the Commit msg (Michael Ellerman) Renamed to powerpc/smp: Fix a warning under !NEED_MULTIPLE_NODES
powerpc/smp: Enable small core scheduling sooner
Rewrote changelog (Gautham)
Renamed to powerpc/smp: Move topology fixups into a new function
powerpc/smp: Create coregroup domain
Add optimization for mask updation under coregroup_support
Changelog v1 -> v2:
v1: https://lore.kernel.org/linuxppc-dev/[email protected]/t/#u
powerpc/smp: Merge Power9 topology with Power topology
Replaced a reference to cpu_smt_mask with per_cpu(cpu_sibling_map, cpu) since cpu_smt_mask is only defined under CONFIG_SCHED_SMT
powerpc/smp: Enable small core scheduling sooner
Restored the previous info msg (Jordan)
Moved big core topology fixup to fixup_topology (Gautham)
powerpc/smp: Dont assume l2-cache to be superset of sibling
Set cpumask after verifying l2-cache. (Gautham)
powerpc/smp: Generalize 2nd sched domain
Moved shared_cache topology fixup to fixup_topology (Gautham)
Powerpc/numa: Detect support for coregroup
Explained Coregroup in commit msg (Michael Ellerman)
Powerpc/smp: Create coregroup domain
Moved coregroup topology fixup to fixup_topology (Gautham)
powerpc/smp: Implement cpu_to_coregroup_id
Move coregroup_enabled before getting associativity (Gautham)
powerpc/smp: Provide an ability to disable coregroup
Patch dropped (Michael Ellerman)
Cleanup of existing powerpc topologies and add coregroup support on Powerpc. Coregroup is a group of (subset of) cores of a DIE that share a resource.
Patch 7 of this patch series: "Powerpc/numa: Detect support for coregroup" depends on
https://lore.kernel.org/linuxppc-dev/[email protected]/t/#u
However it should be easy to rebase the patch without the above patch.
This patch series is based on top of current powerpc/next tree + the above patch.
On Power 8 Systems
------------------
$ tail /proc/cpuinfo processor : 255 cpu : POWER8 (architected), altivec supported clock : 3724.000000MHz revision : 2.1 (pvr 004b 0201)
timebase : 512000000
platform : pSeries
model : IBM,8408-E8E
machine : CHRP IBM,8408-E8E
MMU : Hash
Before the patchset
-------------------
$ cat /proc/sys/kernel/sched_domain/cpu0/domain*/name SMT DIE NUMA NUMA $ head /proc/schedstat version 15 timestamp 4295534931 cpu0 0 0 0 0 0 0 41389823338 17682779896 14117 domain0 00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain1 00000000,00000000,00000000,00000000,00000000,00000000,00000000,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain2 00000000,00000000,00000000,00000000,00000000,00000000,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain3 ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cpu1 0 0 0 0 0 0 27087859050 152273672 10396 domain0 00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain1 00000000,00000000,00000000,00000000,00000000,00000000,00000000,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
After the patchset
------------------
$ cat /proc/sys/kernel/sched_domain/cpu0/domain*/name SMT DIE NUMA NUMA $ head /proc/schedstat version 15 timestamp 4295534931 cpu0 0 0 0 0 0 0 41389823338 17682779896 14117 domain0 00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain1 00000000,00000000,00000000,00000000,00000000,00000000,00000000,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain2 00000000,00000000,00000000,00000000,00000000,00000000,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain3 ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cpu1 0 0 0 0 0 0 27087859050 152273672 10396 domain0 00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain1 00000000,00000000,00000000,00000000,00000000,00000000,00000000,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
On Power 9 (with device-tree enablement to show coregroups). (hunks for mimicing a coregroup was posted at
https://lore.kernel.org/linuxppc-dev/[email protected]/t/#m2cb09bb11c7a93257d6123d1d27edb8212f8af21)
-----------------------------------------------------------
$ tail /proc/cpuinfo processor : 127 cpu : POWER9 (architected), altivec supported clock : 3000.000000MHz revision : 2.2 (pvr 004e 0202)
timebase : 512000000
platform : pSeries
model : IBM,9008-22L
machine : CHRP IBM,9008-22L
MMU : Hash
Before patchset
--------------
$ cat /proc/sys/kernel/sched_domain/cpu0/domain*/name SMT CACHE DIE NUMA
$ head /proc/schedstat version 15 timestamp 4318242208 cpu0 0 0 0 0 0 0 28077107004 4773387362 78205 domain0 00000000,00000000,00000000,00000055 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain1 00000000,00000000,00000000,000000ff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain2 00000000,00000000,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain3 ffffffff,ffffffff,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cpu1 0 0 0 0 0 0 24177439200 413887604 75393 domain0 00000000,00000000,00000000,000000aa 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain1 00000000,00000000,00000000,000000ff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
After patchset
--------------
$ cat /proc/sys/kernel/sched_domain/cpu0/domain*/name SMT CACHE MC DIE NUMA
$ head /proc/schedstat version 15 timestamp 4318242208 cpu0 0 0 0 0 0 0 28077107004 4773387362 78205 domain0 00000000,00000000,00000000,00000055 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain1 00000000,00000000,00000000,000000ff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain2 00000000,00000000,00000000,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain3 00000000,00000000,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 domain4 ffffffff,ffffffff,ffffffff,ffffffff 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cpu1 0 0 0 0 0 0 24177439200 413887604 75393 domain0 00000000,00000000,00000000,000000aa 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Srikar Dronamraju (10):
powerpc/smp: Fix a warning under !NEED_MULTIPLE_NODES powerpc/smp: Merge Power9 topology with Power topology powerpc/smp: Move powerpc_topology above
powerpc/smp: Move topology fixups into a new function powerpc/smp: Dont assume l2-cache to be superset of sibling powerpc/smp: Generalize 2nd sched domain powerpc/numa: Detect support for coregroup powerpc/smp: Allocate cpumask only after searching thread group powerpc/smp: Create coregroup domain
powerpc/smp: Implement cpu_to_coregroup_id
arch/powerpc/include/asm/smp.h | 1 + arch/powerpc/include/asm/topology.h | 10 ++ arch/powerpc/kernel/smp.c | 246 +++++++++++++++++----------- arch/powerpc/mm/numa.c | 59 +++++-- 4 files changed, 210 insertions(+), 106 deletions(-)
Signature · 1 line
2.17.1
PATCH v4 1/10 powerpc/smp: Fix a warning under !NEED_MULTIPLE_NODES
Fix a build warning in a non CONFIG_NEED_MULTIPLE_NODES "error: _numa_cpu_lookup_table_ undeclared"
---
Changelog v2 -> v3:
Removed node caching part. Rewrote the Commit msg (Michael Ellerman) Renamed to powerpc/smp: Fix a warning under !NEED_MULTIPLE_NODES
arch/powerpc/kernel/smp.c | 2 ++ 1 file changed, 2 insertions(+)
arch/powerpc/kernel/smp.c +2 -0
@@ -860,6 +860,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus) GFP_KERNEL, cpu_to_node(cpu));zalloc_cpumask_var_node (&per_cpu (cpu_core_map, cpu),GFP_KERNEL ,cpu_to_node (cpu)); +#ifdef CONFIG_NEED_MULTIPLE_NODES /* * numa_node_id() works after this. */
@@ -868,6 +869,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)set_cpu_numa_mem (cpu, local_memory_node(numa_cpu_lookup_table[cpu])); } +#endif }/* Init the cpumasks so the boot CPU is related to itself */
Signature · 1 line
2.17.1
PATCH v4 2/10 powerpc/smp: Merge Power9 topology with Power topology
In reply to Srikar DronamrajuA new sched_domain_topology_level was added just for Power9. However the same can be achieved by merging powerpc_topology with power9_topology and makes the code more simpler especially when adding a new sched domain.
---
Changelog v1 -> v2:
Replaced a reference to cpu_smt_mask with per_cpu(cpu_sibling_map, cpu) since cpu_smt_mask is only defined under CONFIG_SCHED_SMT
arch/powerpc/kernel/smp.c | 33 ++++++++++----------------------- 1 file changed, 10 insertions(+), 23 deletions(-)
arch/powerpc/kernel/smp.c +10 -23
@@ -1313,7 +1313,7 @@ int setup_profiling_timer(unsigned int multiplier) }#ifdef CONFIG_SCHED_SMT -/* cpumask of CPUs with asymetric SMT dependancy */ +/* cpumask of CPUs with asymmetric SMT dependency */ static int powerpc_smt_flags (void ) {int flags =SD_SHARE_CPUCAPACITY |SD_SHARE_PKG_RESOURCES ;
@@ -1326,14 +1326,6 @@ static int powerpc_smt_flags(void) }#endif -static struct sched_domain_topology_level powerpc_topology[] = { -#ifdef CONFIG_SCHED_SMT - { cpu_smt_mask, powerpc_smt_flags,SD_INIT_NAME (SMT ) }, -#endif - { cpu_cpu_mask,SD_INIT_NAME (DIE ) }, - {NULL , }, -}; -/* * P9 has a slightly odd architecture where pairs of cores share an L2 cache. * This topology makes it *much* cheaper to migrate tasks between adjacent cores
@@ -1351,7 +1343,13 @@ static int powerpc_shared_cache_flags(void)*/ static const struct cpumask *shared_cache_mask (int cpu) { -return cpu_l2_cache_mask (cpu); +if (shared_caches) +return cpu_l2_cache_mask (cpu); + +if (has_big_cores) +return cpu_smallcore_mask (cpu); + +return per_cpu (cpu_sibling_map, cpu); }#ifdef CONFIG_SCHED_SMT
@@ -1361,7 +1359,7 @@ static const struct cpumask *smallcore_smt_mask(int cpu) }#endif -static struct sched_domain_topology_level power9_topology[] = { +static struct sched_domain_topology_level powerpc_topology[] = { #ifdef CONFIG_SCHED_SMT { cpu_smt_mask, powerpc_smt_flags,SD_INIT_NAME (SMT ) }, #endif
@@ -1386,21 +1384,10 @@ void __init smp_cpus_done(unsigned int max_cpus)#ifdef CONFIG_SCHED_SMT if (has_big_cores) {pr_info ("Big cores detected but using small core scheduling\n" ); - power9_topology[0 ].mask = smallcore_smt_mask; powerpc_topology[0 ].mask = smallcore_smt_mask; }#endif -/* -* If any CPU detects that it's sharing a cache with another CPU then -* use the deeper topology that is aware of this sharing. -*/ -if (shared_caches) { -pr_info ("Using shared cache scheduler topology\n" ); -set_sched_topology (power9_topology); - }else { -pr_info ("Using standard scheduler topology\n" ); -set_sched_topology (powerpc_topology); - } +set_sched_topology (powerpc_topology); }#ifdef CONFIG_HOTPLUG_CPU
Signature · 1 line
2.17.1
PATCH v4 3/10 powerpc/smp: Move powerpc_topology above
In reply to Srikar DronamrajuJust moving the powerpc_topology description above.
This will help in using functions in this file and avoid declarations.
No other functional changes
--- arch/powerpc/kernel/smp.c | 116 +++++++++++++++++++------------------- 1 file changed, 58 insertions(+), 58 deletions(-)
arch/powerpc/kernel/smp.c +58 -58
@@ -818,6 +818,64 @@ static int init_cpu_l1_cache_map(int cpu)return err; } +static bool shared_caches; + +#ifdef CONFIG_SCHED_SMT +/* cpumask of CPUs with asymmetric SMT dependency */ +static int powerpc_smt_flags (void ) +{ +int flags =SD_SHARE_CPUCAPACITY |SD_SHARE_PKG_RESOURCES ; + +if (cpu_has_feature (CPU_FTR_ASYM_SMT )) { +printk_once (KERN_INFO "Enabling Asymmetric SMT scheduling\n" ); + flags |=SD_ASYM_PACKING ; + } +return flags; +} +#endif + +/* +* P9 has a slightly odd architecture where pairs of cores share an L2 cache. +* This topology makes it *much* cheaper to migrate tasks between adjacent cores +* since the migrated task remains cache hot. We want to take advantage of this +* at the scheduler level so an extra topology level is required. +*/ +static int powerpc_shared_cache_flags (void ) +{ +return SD_SHARE_PKG_RESOURCES ; +} + +/* +* We can't just pass cpu_l2_cache_mask() directly because +* returns a non-const pointer and the compiler barfs on that. +*/ +static const struct cpumask *shared_cache_mask (int cpu) +{ +if (shared_caches) +return cpu_l2_cache_mask (cpu); + +if (has_big_cores) +return cpu_smallcore_mask (cpu); + +return per_cpu (cpu_sibling_map, cpu); +} + +#ifdef CONFIG_SCHED_SMT +static const struct cpumask *smallcore_smt_mask (int cpu) +{ +return cpu_smallcore_mask (cpu); +} +#endif + +static struct sched_domain_topology_level powerpc_topology[] = { +#ifdef CONFIG_SCHED_SMT + { cpu_smt_mask, powerpc_smt_flags,SD_INIT_NAME (SMT ) }, +#endif + { shared_cache_mask, powerpc_shared_cache_flags,SD_INIT_NAME (CACHE ) }, + { cpu_cpu_mask,SD_INIT_NAME (DIE ) }, + {NULL , }, +}; +static int init_big_cores (void ) {int cpu;
@@ -1247,8 +1305,6 @@ static void add_cpu_to_masks(int cpu)set_cpus_related (cpu, i, cpu_core_mask); } -static bool shared_caches; -/* Activate a secondary processor. */ void start_secondary (void *unused) {
@@ -1312,62 +1368,6 @@ int setup_profiling_timer(unsigned int multiplier)return 0 ; } -#ifdef CONFIG_SCHED_SMT -/* cpumask of CPUs with asymmetric SMT dependency */ -static int powerpc_smt_flags (void ) -{ -int flags =SD_SHARE_CPUCAPACITY |SD_SHARE_PKG_RESOURCES ; - -if (cpu_has_feature (CPU_FTR_ASYM_SMT )) { -printk_once (KERN_INFO "Enabling Asymmetric SMT scheduling\n" ); - flags |=SD_ASYM_PACKING ; - } -return flags; -} -#endif - -/* -* P9 has a slightly odd architecture where pairs of cores share an L2 cache. -* This topology makes it *much* cheaper to migrate tasks between adjacent cores -* since the migrated task remains cache hot. We want to take advantage of this -* at the scheduler level so an extra topology level is required. -*/ -static int powerpc_shared_cache_flags (void ) -{ -return SD_SHARE_PKG_RESOURCES ; -} - -/* -* We can't just pass cpu_l2_cache_mask() directly because -* returns a non-const pointer and the compiler barfs on that. -*/ -static const struct cpumask *shared_cache_mask (int cpu) -{ -if (shared_caches) -return cpu_l2_cache_mask (cpu); - -if (has_big_cores) -return cpu_smallcore_mask (cpu); - -return per_cpu (cpu_sibling_map, cpu); -} - -#ifdef CONFIG_SCHED_SMT -static const struct cpumask *smallcore_smt_mask (int cpu) -{ -return cpu_smallcore_mask (cpu); -} -#endif - -static struct sched_domain_topology_level powerpc_topology[] = { -#ifdef CONFIG_SCHED_SMT - { cpu_smt_mask, powerpc_smt_flags,SD_INIT_NAME (SMT ) }, -#endif - { shared_cache_mask, powerpc_shared_cache_flags,SD_INIT_NAME (CACHE ) }, - { cpu_cpu_mask,SD_INIT_NAME (DIE ) }, - {NULL , }, -}; -void __init smp_cpus_done (unsigned int max_cpus) {/*
Signature · 1 line
2.17.1
PATCH v4 4/10 powerpc/smp: Move topology fixups into a new function
In reply to Srikar DronamrajuMove topology fixup based on the platform attributes into its own function which is called just before set_sched_topology.
---
Changelog v2 -> v3:
Rewrote changelog (Gautham)
Renamed to powerpc/smp: Move topology fixups into a new function
arch/powerpc/kernel/smp.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)
arch/powerpc/kernel/smp.c +11 -6
@@ -1368,6 +1368,16 @@ int setup_profiling_timer(unsigned int multiplier)return 0 ; } +static void fixup_topology (void ) +{ +#ifdef CONFIG_SCHED_SMT +if (has_big_cores) { +pr_info ("Big cores detected but using small core scheduling\n" ); + powerpc_topology[0 ].mask = smallcore_smt_mask; + } +#endif +} +void __init smp_cpus_done (unsigned int max_cpus) {/*
@@ -1381,12 +1391,7 @@ void __init smp_cpus_done(unsigned int max_cpus)dump_numa_cpu_topology (); -#ifdef CONFIG_SCHED_SMT -if (has_big_cores) { -pr_info ("Big cores detected but using small core scheduling\n" ); - powerpc_topology[0 ].mask = smallcore_smt_mask; - } -#endif +fixup_topology ();set_sched_topology (powerpc_topology); }
Signature · 1 line
2.17.1
PATCH v4 5/10 powerpc/smp: Dont assume l2-cache to be superset of sibling
In reply to Srikar DronamrajuCurrent code assumes that cpumask of cpus sharing a l2-cache mask will always be a superset of cpu_sibling_mask.
Lets stop that assumption. cpu_l2_cache_mask is a superset of cpu_sibling_mask if and only if shared_caches is set.
---
Changelog v1 -> v2:
Set cpumask after verifying l2-cache. (Gautham)
arch/powerpc/kernel/smp.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-)
arch/powerpc/kernel/smp.c +15 -13
@@ -1194,6 +1194,7 @@ static bool update_mask_by_l2(int cpu, struct cpumask *(*mask_fn)(int))if (!l2_cache)return false; +cpumask_set_cpu (cpu,mask_fn (cpu));for_each_cpu (i, cpu_online_mask) {/* * when updating the marks the current CPU has not been marked
@@ -1276,29 +1277,30 @@ static void add_cpu_to_masks(int cpu)* add it to it's own thread sibling mask. */ cpumask_set_cpu (cpu,cpu_sibling_mask (cpu)); +cpumask_set_cpu (cpu,cpu_core_mask (cpu));for (i = first_thread; i < first_thread + threads_per_core; i++)if (cpu_online (i))set_cpus_related (i, cpu, cpu_sibling_mask);add_cpu_to_smallcore_masks (cpu); -/* -* Copy the thread sibling mask into the cache sibling mask -* and mark any CPUs that share an L2 with this CPU. -*/ -for_each_cpu (i,cpu_sibling_mask (cpu)) -set_cpus_related (cpu, i, cpu_l2_cache_mask);update_mask_by_l2 (cpu, cpu_l2_cache_mask); -/* -* Copy the cache sibling mask into core sibling mask and mark -* any CPUs on the same chip as this CPU. -*/ -for_each_cpu (i,cpu_l2_cache_mask (cpu)) -set_cpus_related (cpu, i, cpu_core_mask); +if (pkg_id ==-1 ) { +struct cpumask *(*mask)(int ) = cpu_sibling_mask; + +/* +* Copy the sibling mask into core sibling mask and +* mark any CPUs on the same chip as this CPU. +*/ +if (shared_caches) + mask = cpu_l2_cache_mask; + +for_each_cpu (i,mask (cpu)) +set_cpus_related (cpu, i, cpu_core_mask); -if (pkg_id ==-1 )return ; + }for_each_cpu (i, cpu_online_mask)if (get_physical_package_id (i) == pkg_id)
Signature · 1 line
2.17.1
PATCH v4 6/10 powerpc/smp: Generalize 2nd sched domain
In reply to Srikar DronamrajuCurrently "CACHE" domain happens to be the 2nd sched domain as per powerpc_topology. This domain will collapse if cpumask of l2-cache is same as SMT domain. However we could generalize this domain such that it could mean either be a "CACHE" domain or a "BIGCORE" domain.
While setting up the "CACHE" domain, check if shared_cache is already set.
---
Changelog v1 -> v2:
Moved shared_cache topology fixup to fixup_topology (Gautham)
arch/powerpc/kernel/smp.c | 49 ++++++++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 14 deletions(-)
arch/powerpc/kernel/smp.c +34 -14
@@ -85,6 +85,14 @@ EXPORT_PER_CPU_SYMBOL(cpu_l2_cache_map);EXPORT_PER_CPU_SYMBOL (cpu_core_map);EXPORT_SYMBOL_GPL (has_big_cores); +enum { +#ifdef CONFIG_SCHED_SMT + smt_idx, +#endif + bigcore_idx, + die_idx, +}; +#define MAX_THREAD_LIST_SIZE 8#define THREAD_GROUP_SHARE_L1 1struct thread_groups {
@@ -851,13 +859,7 @@ static int powerpc_shared_cache_flags(void)*/ static const struct cpumask *shared_cache_mask (int cpu) { -if (shared_caches) -return cpu_l2_cache_mask (cpu); - -if (has_big_cores) -return cpu_smallcore_mask (cpu); - -return per_cpu (cpu_sibling_map, cpu); +return per_cpu (cpu_l2_cache_map, cpu); }#ifdef CONFIG_SCHED_SMT
@@ -867,11 +869,16 @@ static const struct cpumask *smallcore_smt_mask(int cpu) }#endif +static const struct cpumask *cpu_bigcore_mask (int cpu) +{ +return per_cpu (cpu_sibling_map, cpu); +} +static struct sched_domain_topology_level powerpc_topology[] = { #ifdef CONFIG_SCHED_SMT { cpu_smt_mask, powerpc_smt_flags,SD_INIT_NAME (SMT ) }, #endif - { shared_cache_mask, powerpc_shared_cache_flags,SD_INIT_NAME (CACHE ) }, + { cpu_bigcore_mask,SD_INIT_NAME (BIGCORE ) }, { cpu_cpu_mask,SD_INIT_NAME (DIE ) }, {NULL , }, };
@@ -1311,7 +1318,6 @@ static void add_cpu_to_masks(int cpu)void start_secondary (void *unused) {unsigned int cpu =smp_processor_id (); -struct cpumask *(*sibling_mask)(int ) = cpu_sibling_mask;mmgrab (&init_mm); current->active_mm = &init_mm;
@@ -1337,14 +1343,20 @@ void start_secondary(void *unused)/* Update topology CPU masks */ add_cpu_to_masks (cpu); -if (has_big_cores) - sibling_mask = cpu_smallcore_mask;/* * Check for any shared caches. Note that this must be done on a * per-core basis because one core in the pair might be disabled. */ -if (!cpumask_equal (cpu_l2_cache_mask (cpu),sibling_mask (cpu))) - shared_caches = true; +if (!shared_caches) { +struct cpumask *(*sibling_mask)(int ) = cpu_sibling_mask; +struct cpumask *mask =cpu_l2_cache_mask (cpu); + +if (has_big_cores) + sibling_mask = cpu_smallcore_mask; + +if (cpumask_weight (mask) >cpumask_weight (sibling_mask (cpu))) + shared_caches = true; + }set_numa_node (numa_cpu_lookup_table[cpu]);set_numa_mem (local_memory_node (numa_cpu_lookup_table[cpu]));
@@ -1375,9 +1387,17 @@ static void fixup_topology(void)#ifdef CONFIG_SCHED_SMT if (has_big_cores) {pr_info ("Big cores detected but using small core scheduling\n" ); - powerpc_topology[0 ].mask = smallcore_smt_mask; + powerpc_topology[smt_idx].mask = smallcore_smt_mask; }#endif +if (shared_caches) { +pr_info ("Using shared cache scheduler topology\n" ); + powerpc_topology[bigcore_idx].mask = shared_cache_mask; + powerpc_topology[bigcore_idx].sd_flags = powerpc_shared_cache_flags; +#ifdef CONFIG_SCHED_DEBUG + powerpc_topology[bigcore_idx].name ="CACHE" ; +#endif + } }void __init smp_cpus_done (unsigned int max_cpus)
Signature · 1 line
2.17.1
PATCH v4 7/10 Powerpc/numa: Detect support for coregroup
In reply to Srikar DronamrajuAdd support for grouping cores based on the device-tree classification.
- The last domain in the associativity domains always refers to the core.
- If primary reference domain happens to be the penultimate domain in the associativity domains device-tree property, then there are no coregroups. However if its not a penultimate domain, then there are coregroups. There can be more than one coregroup. For now we would be interested in the last or the smallest coregroups.
---
Changelog v1 -> v2:
Explained Coregroup in commit msg (Michael Ellerman)
arch/powerpc/include/asm/smp.h | 1 + arch/powerpc/kernel/smp.c | 1 + arch/powerpc/mm/numa.c | 34 +++++++++++++++++++++------------- 3 files changed, 23 insertions(+), 13 deletions(-)
arch/powerpc/include/asm/smp.h +1 -0
@@ -28,6 +28,7 @@extern int boot_cpuid;extern int spinning_secondaries;extern u32 *cpu_to_phys_id; +extern bool coregroup_enabled;extern void cpu_die (void );extern int cpu_to_chip_id (int cpu);
arch/powerpc/kernel/smp.c +1 -0
@@ -74,6 +74,7 @@ static DEFINE_PER_CPU(int, cpu_state) = { 0 };struct task_struct *secondary_current;bool has_big_cores; +bool coregroup_enabled;DEFINE_PER_CPU (cpumask_var_t, cpu_sibling_map);DEFINE_PER_CPU (cpumask_var_t, cpu_smallcore_map);
arch/powerpc/mm/numa.c +21 -13
@@ -886,7 +886,9 @@ static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn)static void __init find_possible_nodes (void ) {struct device_node *rtas; -u32 numnodes, i; +const __be32 *domains; +int prop_length, max_nodes; +u32 i;if (!numa_enabled)return ;
@@ -895,25 +897,31 @@ static void __init find_possible_nodes(void)if (!rtas)return ; -if (of_property_read_u32_index (rtas,"ibm,current-associativity-domains" , - min_common_depth, &numnodes)) { -/* -* ibm,current-associativity-domains is a fairly recent -* property. If it doesn't exist, then fallback on -* ibm,max-associativity-domains. Current denotes what the -* platform can support compared to max which denotes what the -* Hypervisor can support. -*/ -if (of_property_read_u32_index (rtas,"ibm,max-associativity-domains" , - min_common_depth, &numnodes)) +/* +* ibm,current-associativity-domains is a fairly recent property. If +* it doesn't exist, then fallback on ibm,max-associativity-domains. +* Current denotes what the platform can support compared to max +* which denotes what the Hypervisor can support. +*/ + domains =of_get_property (rtas,"ibm,current-associativity-domains" , + &prop_length); +if (!domains) { + domains =of_get_property (rtas,"ibm,max-associativity-domains" , + &prop_length); +if (!domains) goto out; } -for (i =0 ; i < numnodes; i++) { + max_nodes =of_read_number (&domains[min_common_depth],1 ); +for (i =0 ; i < max_nodes; i++) {if (!node_possible (i))node_set (i, node_possible_map); } + prop_length /=sizeof (int ); +if (prop_length > min_common_depth +2 ) + coregroup_enabled =1 ; + out:of_node_put (rtas); }
Signature · 1 line
2.17.1
PATCH v4 8/10 powerpc/smp: Allocate cpumask only after searching thread group
In reply to Srikar DronamrajuIf allocated earlier and the search fails, then cpumask need to be freed. However cpu_l1_cache_map can be allocated after we search thread group.
--- arch/powerpc/kernel/smp.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
arch/powerpc/kernel/smp.c +3 -4
@@ -797,10 +797,6 @@ static int init_cpu_l1_cache_map(int cpu)if (err) goto out; -zalloc_cpumask_var_node (&per_cpu (cpu_l1_cache_map, cpu), -GFP_KERNEL , -cpu_to_node (cpu)); - cpu_group_start =get_cpu_thread_group_start (cpu, &tg);if (unlikely (cpu_group_start ==-1 )) {
@@ -809,6 +805,9 @@ static int init_cpu_l1_cache_map(int cpu) goto out; } +zalloc_cpumask_var_node (&per_cpu (cpu_l1_cache_map, cpu), +GFP_KERNEL ,cpu_to_node (cpu)); +for (i = first_thread; i < first_thread + threads_per_core; i++) {int i_group_start =get_cpu_thread_group_start (i, &tg);
Signature · 1 line
2.17.1
PATCH v4 9/10 Powerpc/smp: Create coregroup domain
In reply to Srikar DronamrajuAdd percpu coregroup maps and masks to create coregroup domain. If a coregroup doesn't exist, the coregroup domain will be degenerated in favour of SMT/CACHE domain.
---
Changelog v3 ->v4:
if coregroup_support doesn't exist, update MC mask to the next smaller domain mask.
Changelog v2 -> v3:
Add optimization for mask updation under coregroup_support
Changelog v1 -> v2:
Moved coregroup topology fixup to fixup_topology (Gautham)
arch/powerpc/include/asm/topology.h | 10 ++++++ arch/powerpc/kernel/smp.c | 48 +++++++++++++++++++++++++++++ arch/powerpc/mm/numa.c | 5 +++ 3 files changed, 63 insertions(+)
arch/powerpc/include/asm/topology.h +10 -0
@@ -88,12 +88,22 @@ static inline int cpu_distance(__be32 *cpu1_assoc, __be32 *cpu2_assoc)#if defined(CONFIG_NUMA ) && defined(CONFIG_PPC_SPLPAR )extern int find_and_online_cpu_nid (int cpu); +extern int cpu_to_coregroup_id (int cpu);#else static inline int find_and_online_cpu_nid (int cpu) {return 0 ; } +static inline int cpu_to_coregroup_id (int cpu) +{ +#ifdef CONFIG_SMP +return cpu_to_core_id (cpu); +#else +return 0 ; +#endif +} +#endif /* CONFIG_NUMA && CONFIG_PPC_SPLPAR */ #include <asm-generic/topology.h>
arch/powerpc/kernel/smp.c +44 -0
98 lines not on this page · raw message
arch/powerpc/mm/numa.c +5 -0
@@ -1216,6 +1216,11 @@ int find_and_online_cpu_nid(int cpu)return new_nid; } +int cpu_to_coregroup_id (int cpu) +{ +return cpu_to_core_id (cpu); +} +static int topology_update_init (void ) { topology_inited =1 ;
Signature · 1 line
2.17.1
PATCH v4 10/10 powerpc/smp: Implement cpu_to_coregroup_id
In reply to Srikar DronamrajuLookup the coregroup id from the associativity array.
If unable to detect the coregroup id, fallback on the core id. This way, ensure sched_domain degenerates and an extra sched domain is not created.
Ideally this function should have been implemented in arch/powerpc/kernel/smp.c. However if its implemented in mm/numa.c, we don't need to find the primary domain again.
If the device-tree mentions more than one coregroup, then kernel implements only the last or the smallest coregroup, which currently corresponds to the penultimate domain in the device-tree.
---
Changelog v1 -> v2:
Move coregroup_enabled before getting associativity (Gautham)
arch/powerpc/mm/numa.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/powerpc/mm/numa.c +20 -0
@@ -1218,6 +1218,26 @@ int find_and_online_cpu_nid(int cpu)int cpu_to_coregroup_id (int cpu) { +__be32 associativity[VPHN_ASSOC_BUFSIZE ] = {0 }; +int index; + +if (cpu <0 || cpu > nr_cpu_ids) +return -1 ; + +if (!coregroup_enabled) + goto out; + +if (!firmware_has_feature (FW_FEATURE_VPHN )) + goto out; + +if (vphn_get_associativity (cpu, associativity)) + goto out; + + index =of_read_number (associativity,1 ); +if (index > min_common_depth +1 ) +return of_read_number (&associativity[index -1 ],1 ); + +out:return cpu_to_core_id (cpu); }
Signature · 1 line
2.17.1
Quoted 7 lines · Srikar Dronamraju
Changelog v3 ->v4:
v3: https://lore.kernel.org/lkml/[email protected]/t/#upowerpc/smp: Create coregroup domain
ifcoregroup_supportdoesn't exist, update MC mask to the next smaller domain mask.
Sorry for the double post of v4.
Please follow the other thread.
http://lore.kernel.org/lkml/[email protected]/t/#u
Quoted 1 line
Signature · 2 lines
Thanks and Regards Srikar Dronamraju