Skip to contentSkip to conversation

NUMA-aware per-cpu allocation

PATCH New Rusty Russell 1 participant 1 message Jun 4, 2004

lore
Share & tools
lore

Rusty Russell

author
#0 lore raw

PATCH 1/1 NUMA-aware per-cpu allocation

Stolen from x86_64: if we have NUMA, then use alloc_bootmem_node.

Can't hurt, I figure.

In 2.7, my cunning plan is to require contiguous sections, and allocate similar sections for the dynamic per-cpu allocations, so this won't work (archs will need to use tricks to create virtually-contiguous memory from different NUMA nodes). I have a bitrotted implementation.

Meanwhile, this compiles: does it help performance?

Name: Per-CPU Memory On Right Nodes
Status: Untested

Stolen from x86_64: for NUMA machines, allocate the per-cpu memory using alloc_bootmem_node so it's optimally placed.

include/asm-ppc64/percpu.h +2 -0
diff -urpN --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal .1784-linux-2.6.7-rc2-bk4/include/asm-ppc64/percpu.h .1784-linux-2.6.7-rc2-bk4.updated/include/asm-ppc64/percpu.h
@@ -1,6 +1,8 @@
 #ifndef __ARCH_PPC64_PERCPU__
 #define __ARCH_PPC64_PERCPU__
 
+/* Big sloppy arch needs more per-cpu room than usual */
+#define PERCPU_ENOUGH_ROOM 65536
 #include <asm-generic/percpu.h>
 
 #endif /* __ARCH_PPC64_PERCPU__ */
init/main.c +14 -0
diff -urpN --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal .1784-linux-2.6.7-rc2-bk4/init/main.c .1784-linux-2.6.7-rc2-bk4.updated/init/main.c
@@ -338,12 +338,26 @@ static void __init setup_per_cpu_areas(v
 		size = PERCPU_ENOUGH_ROOM;
 #endif
 
+	/* Multiple allocs for non-NUMA just wastes memory. */
+#ifdef CONFIG_NUMA
+	for (i = 0; i < NR_CPUS; i++) { 
+		pg_data_t *pgdat;
+
+		pgdat = NODE_DATA(cpu_to_node(i));
+		ptr = alloc_bootmem_node(pgdat, size);
+		if (!ptr)
+			panic("Cannot allocate cpu data for CPU %ld\n", i);
+		__per_cpu_offset[i] = ptr - __per_cpu_start;
+		memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start);
+	}
+#else
 	ptr = alloc_bootmem(size * NR_CPUS);
 
 	for (i = 0; i < NR_CPUS; i++, ptr += size) {
 		__per_cpu_offset[i] = ptr - __per_cpu_start;
 		memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start);
 	}
+#endif /* CONFIG_NUMA */
 }
 #endif /* !__GENERIC_PER_CPU */
Signature · 1 line
Anyone who quotes me in their signature is an idiot -- Rusty Russell