x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled
PATCH RESEND 1/1 x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled
From: Li Chen <[email protected]>
Currently, the SMT domain is added into sched_domain_topology by default if CONFIG_SCHED_SMT is enabled.
If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight is just 1, it will destroy_sched_domain it.
On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.
We can avoid these unnecessary operations by simply checking cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT is not enabled.
---
[RESEND] because I forgot to add any mailing list previously.
arch/x86/kernel/smpboot.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kernel/smpboot.c +0 -0
@@ -485,9 +485,11 @@ static void __init build_sched_topology(void)int i =0 ;
#ifdef CONFIG_SCHED_SMT
- x86_topology[i++] = (struct sched_domain_topology_level){
- cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
- };
+ if (cpu_smt_num_threads > 1) {
+ x86_topology[i++] = (struct sched_domain_topology_level){
+ cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
+ };
+ }
#endif
#ifdef CONFIG_SCHED_CLUSTER
x86_topology[i++] = (struct sched_domain_topology_level){--
2.48.1
Gentle ping.
---- On Tue, 22 Apr 2025 16:47:18 +0800 Li Chen <[email protected]> wrote ---
Quoted 43 lines
From: Li Chen <[email protected]>
Currently, the SMT domain is added into
sched_domain_topologyby default ifCONFIG_SCHED_SMTis enabled.If
cpu_attach_domainfinds that the CPU SMT domain’scpumask_weightis just 1, it willdestroy_sched_domainit.On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.
We can avoid these unnecessary operations by simply checking
cpu_smt_num_threadsand not inserting SMT domain intox86_topologyif SMT is not enabled.Signed-off-by Li Chen <[email protected]>---
[RESEND] because I forgot to add any mailing list previously.
arch/x86/kernel/smpboot.c| 8 +++++---1 file changed, 5 insertions(+), 3 deletions(-)arch/x86/kernel/smpboot.c +5
-4@@ -485,9 +485,11 @@ static void __init build_sched_topology(void)int i =0 ;#ifdef CONFIG_SCHED_SMT - x86_topology[i++] = (struct sched_domain_topology_level ){ - cpu_smt_mask, cpu_smt_flags,SD_INIT_NAME (SMT ) - }; +if (cpu_smt_num_threads >1 ) { + x86_topology[i++] = (struct sched_domain_topology_level ){ + cpu_smt_mask, cpu_smt_flags,SD_INIT_NAME (SMT ) + }; + }#endif #ifdef CONFIG_SCHED_CLUSTER x86_topology[i++] = (struct sched_domain_topology_level){ --
Quoted 3 lines
2.48.1
Regards,
Li
Quoted 5 lines · Li Chen
Currently, the SMT domain is added into
sched_domain_topologyby default ifCONFIG_SCHED_SMTis enabled.If
cpu_attach_domainfinds that the CPU SMT domain’scpumask_weightis just 1, it willdestroy_sched_domainit.
If cpu_attach_domain() ..., it will destroy it.
Quoted 4 lines
On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.
We can avoid these unnecessary operations by simply checking
s/We can avoid/Avoid/
Quoted 1 line
cpu_smt_num_threadsand not inserting SMT domain intox86_topologyif SMT
the SMT domain
Review
Quoted 11 lines
is not enabled.
#ifdef
CONFIG_SCHED_SMT(quoted diff) +5
-3- x86_topology[i++] = (struct sched_domain_topology_level){ - cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) - }; + if (cpu_smt_num_threads > 1) { + x86_topology[i++] = (struct sched_domain_topology_level){ + cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) + }; + }
Looks about right, though I really detest this coding style. I'm not blaming you, as you just followed the already existing bad taste...
Thanks,
tglx
PATCH 1/1 x86/smpboot: Decrapify build_sched_topology()
The #ifdeffery and the initializers in build_sched_topology() are just disgusting. The SCHED_SMT #ifdef is also pointless because SCHED_SMT is unconditionally enabled when SMP is enabled.
Statically initialize the domain levels in the topology array and let build_sched_topology() invalidate the package domain level when NUMA in package is available.
---
Note: This allows to remove the SMT level with a simple memmove()
--- arch/x86/kernel/smpboot.c | 45 +++++++++++++++++---------------------------- 1 file changed, 17 insertions(+), 28 deletions(-)
arch/x86/kernel/smpboot.c +17 -28
@@ -478,43 +478,32 @@ static int x86_cluster_flags(void)*/ static bool x86_has_numa_in_package; -static struct sched_domain_topology_level x86_topology[6 ]; +#define DOMAIN (maskfn, flagsfn, dname) { .mask = maskfn, .sd_flags = flagsfn, .name = #dname } -static void __init build_sched_topology (void ) -{ -int i =0 ; - -#ifdef CONFIG_SCHED_SMT - x86_topology[i++] = (struct sched_domain_topology_level ){ - cpu_smt_mask, cpu_smt_flags,SD_INIT_NAME (SMT ) - }; -#endif +static struct sched_domain_topology_level x86_topology[] = { +DOMAIN (cpu_smt_mask, cpu_smt_flags,SMT ), #ifdef CONFIG_SCHED_CLUSTER - x86_topology[i++] = (struct sched_domain_topology_level ){ - cpu_clustergroup_mask, x86_cluster_flags,SD_INIT_NAME (CLS ) - }; +DOMAIN (cpu_clustergroup_mask, x86_cluster_flags,CLS ), #endif #ifdef CONFIG_SCHED_MC - x86_topology[i++] = (struct sched_domain_topology_level ){ - cpu_coregroup_mask, x86_core_flags,SD_INIT_NAME (MC ) - }; +DOMAIN (cpu_coregroup_mask, x86_core_flags,MC ), #endif -/* -* When there is NUMA topology inside the package skip the PKG domain -* since the NUMA domains will auto-magically create the right spanning -* domains based on the SLIT. -*/ -if (!x86_has_numa_in_package) { - x86_topology[i++] = (struct sched_domain_topology_level ){ - cpu_cpu_mask, x86_sched_itmt_flags,SD_INIT_NAME (PKG ) - }; - } +DOMAIN (cpu_cpu_mask, x86_sched_itmt_flags,PKG ), + {NULL }, +}; +static void __init build_sched_topology (void ) +{/* -* There must be one trailing NULL entry left. +* When there is NUMA topology inside the package invalidate the +* PKG domain since the NUMA domains will auto-magically create the +* right spanning domains based on the SLIT. */ -BUG_ON (i >=ARRAY_SIZE (x86_topology)-1 ); +if (x86_has_numa_in_package) { +unsigned int pkgdom =ARRAY_SIZE (x86_topology) -2 ; +memset (&x86_topology[pkgdom],0 ,sizeof (x86_topology[pkgdom])); + }set_sched_topology (x86_topology); }
Hi Thomas,
---- On Fri, 20 Jun 2025 21:54:27 +0800 Thomas Gleixner <[email protected]> wrote ---
Quoted 32 lines · Li Chen
Quoted 5 lines
Currently, the SMT domain is added into
sched_domain_topologyby default ifCONFIG_SCHED_SMTis enabled.If
cpu_attach_domainfinds that the CPU SMT domain’scpumask_weightis just 1, it willdestroy_sched_domainit.If
cpu_attach_domain()..., it will destroy it.Quoted 4 lines
On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.
We can avoid these unnecessary operations by simply checking
s/We can avoid/Avoid/
Quoted 1 line
cpu_smt_num_threadsand not inserting SMT domain intox86_topologyif SMTthe SMT domain
Review
Quoted 11 lines
is not enabled.
#ifdef
CONFIG_SCHED_SMT(quoted diff) +5
-3- x86_topology[i++] = (struct sched_domain_topology_level){ - cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) - }; + if (cpu_smt_num_threads > 1) { + x86_topology[i++] = (struct sched_domain_topology_level){ + cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT) + }; + }
Quoted 7 lines
Looks about right, though I really detest this coding style. I'm not blaming you, as you just followed the already existing bad taste...
Thanks,
tglx
Thanks for your review, I have fixed the wording issues here: https://lore.kernel.org/all/[email protected]/T/#t
Regards,
Li