Skip to contentSkip to conversation

sched/x86: Calculate booted cores after construction of sibling_mask

PATCH In review Kamalesh Babulal 2 participants 5 messages May 31, 2012 – Jun 6, 2012 x86

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched/x86: Calculate booted cores after construction of sibling_mask

While booting tip (1d06e6d354) on qemu with -smp sockets=1,cores=2,threads=2 topology. There is a mismatch between the number of cores actually present and cores reported.

Changes related to calculating the number of booted cores was introduced by 316ad248307fb (sched/x86: Rewrite set_cpu_sibling_map()). With the clean up, the calculation of booted cores per package is done on an incompletely constructed sibling_mask per cpu.

sched/x86: Calculate booted cores after construction of sibling_mask

This patch re-introduces the old behaviour of constructing the complete sibling mask of the cpu, before calculating the number of cores booted.

On qemu booted with -smp sockets=1,cores=2,threads=2, Before:

$ cat /proc/cpuinfo |grep cores
cpu cores       : 2
cpu cores       : 1
cpu cores       : 4
cpu cores       : 3

With the patch:

$ cat /proc/cpuinfo |grep cores
cpu cores       : 2
cpu cores       : 2
cpu cores       : 2
cpu cores       : 2
---
 arch/x86/kernel/smpboot.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)
arch/x86/kernel/smpboot.c +6 -1
index fd019d7..a8d78f3 100644
@@ -382,7 +382,12 @@ void __cpuinit set_cpu_sibling_map(int cpu)
 		if ((i == cpu) || (has_mc && match_llc(c, o)))
 			link_mask(llc_shared, cpu, i);
 
-		if ((i == cpu) || (has_mc && match_mc(c, o))) {
+	}
+
+	for_each_cpu(i, cpu_sibling_setup_mask) {
+		o = &cpu_data(i);
+
+	if ((i == cpu) || (has_mc && match_mc(c, o))) {
 			link_mask(core, cpu, i);
 
 			/*
#1 lore raw
Quoted 10 lines · Kamalesh Babulal

While booting tip (1d06e6d354) on qemu with -smp sockets=1,cores=2,threads=2 topology. There is a mismatch between the number of cores actually present and cores reported.

Changes related to calculating the number of booted cores was introduced by 316ad248307fb (sched/x86: Rewrite set_cpu_sibling_map()). With the clean up, the calculation of booted cores per package is done on an incompletely constructed sibling_mask per cpu.

sched/x86: Calculate booted cores after construction of sibling_mask

Why repeat this subject again?

Quoted 16 lines

This patch re-introduces the old behaviour of constructing the complete sibling mask of the cpu, before calculating the number of cores booted.

---
 arch/x86/kernel/smpboot.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)
arch/x86/kernel/smpboot.c +0 -0
index fd019d7..a8d78f3 100644
@@ -382,7 +382,12 @@ void __cpuinit set_cpu_sibling_map(int cpu)
 		if ((i == cpu) || (has_mc && match_llc(c, o)))
 			link_mask(llc_shared, cpu, i);
 

arch/x86/kernel/smpboot.c void __cpuinit set_cpu_sibling_map(int cpu) · line 393

Quoted 10 lines
arch/x86/kernel/smpboot.c +6 -1
-		if ((i == cpu) || (has_mc && match_mc(c, o))) {
+	}
+
+	for_each_cpu(i, cpu_sibling_setup_mask) {
+		o = &cpu_data(i);
+
+	if ((i == cpu) || (has_mc && match_mc(c, o))) {
 			link_mask(core, cpu, i);
 
 			/*

This patch actually horribly mangles the indenting. I did the below to it:

---
Subject: sched/x86: Calculate booted cores after construction of sibling_mask
From: Kamalesh Babulal <[email protected]>
Date: Thu, 31 May 2012 13:07:38 +0530

Commit 316ad248307fb ("sched/x86: Rewrite set_cpu_sibling_map()") broke the booted_cores accounting.

The problem is that the booted_cores accounting needs all the sibling links set up. So restore the second loop and add a comment as to why its needed.

On qemu booted with -smp sockets=1,cores=2,threads=2; Before:

$ grep cores /proc/cpuinfo
cpu cores       : 2
cpu cores       : 1
cpu cores       : 4
cpu cores       : 3

With the patch:

$ grep cores /proc/cpuinfo
cpu cores       : 2
cpu cores       : 2
cpu cores       : 2
cpu cores       : 2
---
 arch/x86/kernel/smpboot.c |    9 +++++++++
 1 file changed, 9 insertions(+)
arch/x86/kernel/smpboot.c +9 -0
@@ -382,6 +382,15 @@ void __cpuinit set_cpu_sibling_map(int c
 		if ((i == cpu) || (has_mc && match_llc(c, o)))
 			link_mask(llc_shared, cpu, i);
 
+	}
+
+	/*
+	 * This needs a separate iteration over the cpus because we rely on all
+	 * cpu_sibling_mask links to be set-up.
+	 */
+	for_each_cpu(i, cpu_sibling_setup_mask) {
+		o = &cpu_data(i);
+
 		if ((i == cpu) || (has_mc && match_mc(c, o))) {
 			link_mask(core, cpu, i);
 
#2 lore raw

* Peter Zijlstra <[email protected]> [2012-05-31 22:30:57]:

(snip)

Quoted 5 lines
Quoted 2 lines

sched/x86: Calculate booted cores after construction of sibling_mask

Why repeat this subject again?

Will take care of it from next posting.

(snip)

Quoted 17 lines
Quoted 17 lines
arch/x86/kernel/smpboot.c +6 -1
index fd019d7..a8d78f3 100644
@@ -382,7 +382,12 @@ void __cpuinit set_cpu_sibling_map(int cpu)
 		if ((i == cpu) || (has_mc && match_llc(c, o)))
 			link_mask(llc_shared, cpu, i);
 
-		if ((i == cpu) || (has_mc && match_mc(c, o))) {
+	}
+
+	for_each_cpu(i, cpu_sibling_setup_mask) {
+		o = &cpu_data(i);
+
+	if ((i == cpu) || (has_mc && match_mc(c, o))) {
 			link_mask(core, cpu, i);
 
Quoted 4 lines

arch/x86/kernel/smpboot.c void __cpuinit set_cpu_sibling_map(int cpu) · line 393

Quoted 1 line
arch/x86/kernel/smpboot.c +0 -0
 			/*

This patch actually horribly mangles the indenting. I did the below to it:

Sorry about the indentation. Thanks for fixing it.

#3 lore raw
Quoted 1 line · Kamalesh Babulal

Will take care of it from next posting.

No need, I queued the fixup...

#4 lore raw
In reply to Kamalesh Babulal
Commit-ID:  ceb1cbac8eda66cf0f889def226b4e82f8ff857b
Gitweb:     http://git.kernel.org/tip/ceb1cbac8eda66cf0f889def226b4e82f8ff857b
Author:     Kamalesh Babulal <[email protected]>
AuthorDate: Thu, 31 May 2012 13:07:38 +0530
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 6 Jun 2012 16:37:59 +0200

sched/x86: Calculate booted cores after construction of sibling_mask

Commit 316ad248307fb ("sched/x86: Rewrite set_cpu_sibling_map()") broke the booted_cores accounting.

The problem is that the booted_cores accounting needs all the sibling links set up. So restore the second loop and add a comment as to why its needed.

On qemu booted with -smp sockets=1,cores=2,threads=2; Before:

 $ grep cores /proc/cpuinfo
 cpu cores       : 2
 cpu cores       : 1
 cpu cores       : 4
 cpu cores       : 3

With the patch:

 $ grep cores /proc/cpuinfo
 cpu cores       : 2
 cpu cores       : 2
 cpu cores       : 2
 cpu cores       : 2
---
 arch/x86/kernel/smpboot.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
arch/x86/kernel/smpboot.c +9 -0
index fd019d7..3fab55b 100644
@@ -382,6 +382,15 @@ void __cpuinit set_cpu_sibling_map(int cpu)
 		if ((i == cpu) || (has_mc && match_llc(c, o)))
 			link_mask(llc_shared, cpu, i);
 
+	}
+
+	/*
+	 * This needs a separate iteration over the cpus because we rely on all
+	 * cpu_sibling_mask links to be set-up.
+	 */
+	for_each_cpu(i, cpu_sibling_setup_mask) {
+		o = &cpu_data(i);
+
 		if ((i == cpu) || (has_mc && match_mc(c, o))) {
 			link_mask(core, cpu, i);