Skip to contentSkip to conversation

x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled

PATCH RESEND In review Li Chen 2 participants 5 messages Apr 22, 2025 – Jun 24, 2025 x86

lore
Share & tools
lore

Li Chen

author
#0 lore raw

PATCH RESEND 1/1 x86/smpboot: avoid SMT domain attach/destroy if SMT is not enabled

From: Li Chen <[email protected]>

Currently, the SMT domain is added into sched_domain_topology by default if CONFIG_SCHED_SMT is enabled.

If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight is just 1, it will destroy_sched_domain it.

On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.

We can avoid these unnecessary operations by simply checking cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT is not enabled.

---

[RESEND] because I forgot to add any mailing list previously.

arch/x86/kernel/smpboot.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kernel/smpboot.c +0 -0
index d6cf1e23c2a3..da6192e1af12 100644
@@ -485,9 +485,11 @@ static void __init build_sched_topology(void)
    int i = 0;
 

#ifdef CONFIG_SCHED_SMT
- x86_topology[i++] = (struct sched_domain_topology_level){
- cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
- };
+ if (cpu_smt_num_threads > 1) {
+ x86_topology[i++] = (struct sched_domain_topology_level){
+ cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
+ };
+ }
#endif

#ifdef CONFIG_SCHED_CLUSTER
    x86_topology[i++] = (struct sched_domain_topology_level){

--
2.48.1

author
#1 lore raw

Gentle ping.

---- On Tue, 22 Apr 2025 16:47:18 +0800 Li Chen <[email protected]> wrote ---

Quoted 43 lines

From: Li Chen <[email protected]>

Currently, the SMT domain is added into sched_domain_topology by default if CONFIG_SCHED_SMT is enabled.

If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight is just 1, it will destroy_sched_domain it.

On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.

We can avoid these unnecessary operations by simply checking cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT is not enabled.

---

[RESEND] because I forgot to add any mailing list previously.

arch/x86/kernel/smpboot.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kernel/smpboot.c +5 -4
index d6cf1e23c2a3..da6192e1af12 100644
@@ -485,9 +485,11 @@ static void __init build_sched_topology(void)
    int i = 0;
 
 #ifdef CONFIG_SCHED_SMT
-    x86_topology[i++] = (struct sched_domain_topology_level){
-        cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
-    };
+    if (cpu_smt_num_threads > 1) {
+        x86_topology[i++] = (struct sched_domain_topology_level){
+            cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
+        };
+    }
 #endif
 #ifdef CONFIG_SCHED_CLUSTER
    x86_topology[i++] = (struct sched_domain_topology_level){
--
Quoted 3 lines

2.48.1

Regards,
Li

#2 lore raw
In reply to Li Chen
Quoted 5 lines · Li Chen

Currently, the SMT domain is added into sched_domain_topology by default if CONFIG_SCHED_SMT is enabled.

If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight is just 1, it will destroy_sched_domain it.

If cpu_attach_domain() ..., it will destroy it.

Quoted 4 lines

On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.

We can avoid these unnecessary operations by simply checking

s/We can avoid/Avoid/

Quoted 1 line

cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT

the SMT domain

Review

Quoted 11 lines

is not enabled.

#ifdef CONFIG_SCHED_SMT

(quoted diff) +5 -3
-    x86_topology[i++] = (struct sched_domain_topology_level){
-        cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
-    };
+    if (cpu_smt_num_threads > 1) {
+        x86_topology[i++] = (struct sched_domain_topology_level){
+            cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
+        };
+    }

Looks about right, though I really detest this coding style. I'm not blaming you, as you just followed the already existing bad taste...

Thanks,

tglx

#3 lore raw

PATCH 1/1 x86/smpboot: Decrapify build_sched_topology()

The #ifdeffery and the initializers in build_sched_topology() are just disgusting. The SCHED_SMT #ifdef is also pointless because SCHED_SMT is unconditionally enabled when SMP is enabled.

Statically initialize the domain levels in the topology array and let build_sched_topology() invalidate the package domain level when NUMA in package is available.

---
Note: This allows to remove the SMT level with a simple memmove()

---
 arch/x86/kernel/smpboot.c |   45 +++++++++++++++++----------------------------
 1 file changed, 17 insertions(+), 28 deletions(-)
arch/x86/kernel/smpboot.c +17 -28
@@ -478,43 +478,32 @@ static int x86_cluster_flags(void)
  */
 static bool x86_has_numa_in_package;
 
-static struct sched_domain_topology_level x86_topology[6];
+#define DOMAIN(maskfn, flagsfn, dname) { .mask = maskfn, .sd_flags = flagsfn, .name = #dname }
 
-static void __init build_sched_topology(void)
-{
-	int i = 0;
-
-#ifdef CONFIG_SCHED_SMT
-	x86_topology[i++] = (struct sched_domain_topology_level){
-		cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
-	};
-#endif
+static struct sched_domain_topology_level x86_topology[] = {
+	DOMAIN(cpu_smt_mask, cpu_smt_flags, SMT),
 #ifdef CONFIG_SCHED_CLUSTER
-	x86_topology[i++] = (struct sched_domain_topology_level){
-		cpu_clustergroup_mask, x86_cluster_flags, SD_INIT_NAME(CLS)
-	};
+	DOMAIN(cpu_clustergroup_mask, x86_cluster_flags, CLS),
 #endif
 #ifdef CONFIG_SCHED_MC
-	x86_topology[i++] = (struct sched_domain_topology_level){
-		cpu_coregroup_mask, x86_core_flags, SD_INIT_NAME(MC)
-	};
+	DOMAIN(cpu_coregroup_mask, x86_core_flags, MC),
 #endif
-	/*
-	 * When there is NUMA topology inside the package skip the PKG domain
-	 * since the NUMA domains will auto-magically create the right spanning
-	 * domains based on the SLIT.
-	 */
-	if (!x86_has_numa_in_package) {
-		x86_topology[i++] = (struct sched_domain_topology_level){
-			cpu_cpu_mask, x86_sched_itmt_flags, SD_INIT_NAME(PKG)
-		};
-	}
+	DOMAIN(cpu_cpu_mask, x86_sched_itmt_flags, PKG),
+	{ NULL },
+};
 
+static void __init build_sched_topology(void)
+{
 	/*
-	 * There must be one trailing NULL entry left.
+	 * When there is NUMA topology inside the package invalidate the
+	 * PKG domain since the NUMA domains will auto-magically create the
+	 * right spanning domains based on the SLIT.
 	 */
-	BUG_ON(i >= ARRAY_SIZE(x86_topology)-1);
+	if (x86_has_numa_in_package) {
+		unsigned int pkgdom = ARRAY_SIZE(x86_topology) - 2;
 
+		memset(&x86_topology[pkgdom], 0, sizeof(x86_topology[pkgdom]));
+	}
 	set_sched_topology(x86_topology);
 }
 
author
#4 lore raw
In reply to Thomas Gleixner

Hi Thomas,

---- On Fri, 20 Jun 2025 21:54:27 +0800 Thomas Gleixner <[email protected]> wrote ---

Quoted 32 lines · Li Chen
Quoted 5 lines

Currently, the SMT domain is added into sched_domain_topology by default if CONFIG_SCHED_SMT is enabled.

If cpu_attach_domain finds that the CPU SMT domain’s cpumask_weight is just 1, it will destroy_sched_domain it.

If cpu_attach_domain() ..., it will destroy it.

Quoted 4 lines

On a large machine, such as one with 512 cores, this results in 512 redundant domain attach/destroy operations.

We can avoid these unnecessary operations by simply checking

s/We can avoid/Avoid/

Quoted 1 line

cpu_smt_num_threads and not inserting SMT domain into x86_topology if SMT

the SMT domain

Review

Quoted 11 lines

is not enabled.

#ifdef CONFIG_SCHED_SMT

(quoted diff) +5 -3
-    x86_topology[i++] = (struct sched_domain_topology_level){
-        cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
-    };
+    if (cpu_smt_num_threads > 1) {
+        x86_topology[i++] = (struct sched_domain_topology_level){
+            cpu_smt_mask, cpu_smt_flags, SD_INIT_NAME(SMT)
+        };
+    }
Quoted 7 lines

Looks about right, though I really detest this coding style. I'm not blaming you, as you just followed the already existing bad taste...

Thanks,

tglx

Thanks for your review, I have fixed the wording issues here: https://lore.kernel.org/all/[email protected]/T/#t

Regards,
Li