Skip to contentSkip to conversation

x86/numa: Allow specifying node_distance() for numa=fake

tip:sched/core tip-bot for Peter Zijlstra 3 participants 3 messages May 9, 2012 – Jun 6, 2012

Subsystems (2)
x86 x86 MM
lore
Share & tools
lore

tip-bot for Peter Zijlstra

author
#0 lore raw
Commit-ID:  94c0dd3278dd3eae52eabf0fb77d472d0dd3e373
Gitweb:     http://git.kernel.org/tip/94c0dd3278dd3eae52eabf0fb77d472d0dd3e373
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Wed, 18 Apr 2012 19:04:17 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 9 May 2012 13:28:59 +0200

x86/numa: Allow specifying node_distance() for numa=fake

Allows emulating more interesting NUMA configurations like a quad socket AMD Magny-Cour:

"numa=fake=8:10,16,16,22,16,22,16,22,

              16,10,22,16,22,16,22,16,
              16,22,10,16,16,22,16,22,
              22,16,16,10,22,16,22,16,
              16,22,16,22,10,16,16,22,
              22,16,22,16,16,10,22,16,
              16,22,16,22,16,22,10,16,
              22,16,22,16,22,16,16,10"

Which has a non-fully-connected topology.

Signed-off-by Peter Zijlstra <[email protected]>
Cc Tejun Heo <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc [email protected]
Link http://lkml.kernel.org/n/[email protected] open here
Signed-off-by Ingo Molnar <[email protected]>
---
 arch/x86/mm/numa_emulation.c |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)
arch/x86/mm/numa_emulation.c +6 -2
index 53489ff..871dd88 100644
@@ -339,9 +339,11 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
 	} else {
 		unsigned long n;
 
-		n = simple_strtoul(emu_cmdline, NULL, 0);
+		n = simple_strtoul(emu_cmdline, &emu_cmdline, 0);
 		ret = split_nodes_interleave(&ei, &pi, 0, max_addr, n);
 	}
+	if (*emu_cmdline == ':')
+		emu_cmdline++;
 
 	if (ret < 0)
 		goto no_emu;

@@ -418,7 +420,9 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
 			int physj = emu_nid_to_phys[j];
 			int dist;
 
-			if (physi >= numa_dist_cnt || physj >= numa_dist_cnt)
+			if (get_option(&emu_cmdline, &dist) == 2)
+				;
+			else if (physi >= numa_dist_cnt || physj >= numa_dist_cnt)
 				dist = physi == physj ?
 					LOCAL_DISTANCE : REMOTE_DISTANCE;
 			else
#1 lore raw
Quoted 21 lines · tip-bot for Peter Zijlstra
Commit-ID:  94c0dd3278dd3eae52eabf0fb77d472d0dd3e373
Gitweb:     http://git.kernel.org/tip/94c0dd3278dd3eae52eabf0fb77d472d0dd3e373
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Wed, 18 Apr 2012 19:04:17 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 9 May 2012 13:28:59 +0200

x86/numa: Allow specifying node_distance() for numa=fake

Allows emulating more interesting NUMA configurations like a quad socket AMD Magny-Cour:

"numa=fake=8:10,16,16,22,16,22,16,22,

              16,10,22,16,22,16,22,16,
              16,22,10,16,16,22,16,22,
              22,16,16,10,22,16,22,16,
              16,22,16,22,10,16,16,22,
              22,16,22,16,16,10,22,16,
              16,22,16,22,16,22,10,16,
              22,16,22,16,22,16,16,10"
Quoted 2 lines

Which has a non-fully-connected topology.

I like this support and I'm pretty sure you used it to reproduce my problems with sched/numa locally, but I think it would be better to seperate it out as a different parameter such as slit=fake so that we can still use it to fake the SLIT of our NUMA machines without requiring numa=fake which provides no guarantees to break the nodes along physical boundaries.

So without seperating it out into slit=fake, we can't change this information without changing the SLIT itself and that makes debugging harder.

#2 lore raw
Quoted 34 lines · David Rientjes
Quoted 21 lines · tip-bot for Peter Zijlstra
Commit-ID:  94c0dd3278dd3eae52eabf0fb77d472d0dd3e373
Gitweb:     http://git.kernel.org/tip/94c0dd3278dd3eae52eabf0fb77d472d0dd3e373
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Wed, 18 Apr 2012 19:04:17 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 9 May 2012 13:28:59 +0200

x86/numa: Allow specifying node_distance() for numa=fake

Allows emulating more interesting NUMA configurations like a quad socket AMD Magny-Cour:

"numa=fake=8:10,16,16,22,16,22,16,22,

              16,10,22,16,22,16,22,16,
              16,22,10,16,16,22,16,22,
              22,16,16,10,22,16,22,16,
              16,22,16,22,10,16,16,22,
              22,16,22,16,16,10,22,16,
              16,22,16,22,16,22,10,16,
              22,16,22,16,22,16,16,10"
Quoted 2 lines

Which has a non-fully-connected topology.

I like this support and I'm pretty sure you used it to reproduce my problems with sched/numa locally, but I think it would be better to seperate it out as a different parameter such as slit=fake so that we can still use it to fake the SLIT of our NUMA machines without requiring numa=fake which provides no guarantees to break the nodes along physical boundaries.

Quoted 3 lines

So without seperating it out into slit=fake, we can't change this information without changing the SLIT itself and that makes debugging harder.

Makes sense - wanna send a patch?

Thanks,

Ingo