Manfreds patch to distribute boot allocations across nodes
Subsystems (2)
Hi,
Manfred had a patch to distribute kmallocs across nodes during boot. I took it for a spin.
buddyinfo before:
Node 7, 0 2 1 1 0 2 1 2 1 2 1 2 741 Node 6, 0 0 0 2 0 2 1 1 2 2 2 2 1002 Node 5, 0 0 0 2 0 2 1 2 1 2 2 2 2006 Node 4, 0 0 0 2 0 2 1 2 1 2 2 2 2006 Node 3, 0 0 0 2 0 2 1 2 1 2 2 2 2006 Node 2, 0 0 0 2 0 2 1 2 1 2 2 2 2006 Node 1, 0 0 0 2 0 2 1 1 2 2 2 2 1002 Node 0, 0 0 38 7 0 1 1 1 0 0 0 0 1998
buddyinfo after:
Node 7, 0 1 0 1 1 1 1 0 0 0 1 2 738 Node 6, 0 1 0 1 1 1 0 1 0 0 2 2 1002 Node 5, 0 0 0 1 1 1 1 0 0 0 2 2 2006 Node 4, 0 1 0 1 0 1 1 0 0 0 2 2 2006 Node 3, 0 0 0 1 0 1 1 0 0 0 2 2 2005 Node 2, 0 1 0 0 0 0 0 1 0 0 2 2 2006 Node 1, 0 2 1 1 0 1 1 1 0 0 2 2 1002 Node 0, 0 20 45 8 3 0 1 1 1 1 0 1 2004
Change in free memory due to patch:
Node 7 -54.08 MB
Node 6 -6.33 MB Node 5 -6.09 MB Node 4 -6.14 MB
Node 3 -22.15 MB
Node 2 -6.05 MB
Node 1 -6.12 MB
Node 0 107.35 MB
As you can see we gained over 100MB on node 0. Spreading boot time allocations around also helps us to avoid node 0 becoming the hot node.
--
Manfred Spraul <[email protected]>
Distribute the memory allocations that happen during boot to all nodes. The memory will be touched by all cpus, binding all allocs to the boot node is wrong.
--
mm/page_alloc.c +40 -0
@@ -681,6 +681,42 @@EXPORT_SYMBOL (__alloc_pages); +#ifdef CONFIG_NUMA +/* Early boot: Everything is done by one cpu, but the data structures will be +* used by all cpus - spread them on all nodes. +*/ +static __init unsigned long get_boot_pages (unsigned int gfp_mask,unsigned int order) +{ +static int nodenr; +int i = nodenr; +struct page *page; + +for (;;) { +if (i > nodenr + numnodes) +return 0 ; +if (node_present_pages (i%numnodes)) { +struct zone **z; +/* The node contains memory. Check that there is +* memory in the intended zonelist. +*/ + z =NODE_DATA (i%numnodes)->node_zonelists[gfp_mask &GFP_ZONEMASK ].zones; +while (*z) { +if ( (*z)->free_pages > (1UL <<order)) + goto found_node; + z++; + } + } + i++; + } +found_node: + nodenr = i+1 ; + page =alloc_pages_node (i%numnodes, gfp_mask, order); +if (!page) +return 0 ; +return (unsigned long )page_address (page); +} +#endif +/* * Common helper functions. */
@@ -688,6 +724,10 @@ {struct page * page; +#ifdef CONFIG_NUMA +if (unlikely (!system_running)) +return get_boot_pages (gfp_mask, order); +#endif page =alloc_pages (gfp_mask, order);if (!page)return 0 ;
Quoted 2 lines · Anton Blanchard
Manfred had a patch to distribute kmallocs across nodes during boot.
He's a handy guy.
Quoted 4 lines
...
Change in free memory due to patch:
Quoted 8 lines
Node 7 -54.08 MB
Node 6 -6.33 MB Node 5 -6.09 MB Node 4 -6.14 MBNode 3 -22.15 MB
Node 2 -6.05 MB
Node 1 -6.12 MB
Node 0 107.35 MB
OK.
Quoted 1 line
+#ifdef
CONFIG_NUMA
Is this a thing which all NUMA machines want to be doing?
Quoted 21 lines
mm/page_alloc.c +21
-0+static __init unsigned long get_boot_pages (unsigned int gfp_mask,unsigned int order) +{ +static int nodenr; +int i = nodenr; +struct page *page; + +for (;;) { +if (i > nodenr + numnodes) +return 0 ; +if (node_present_pages (i%numnodes)) { +struct zone **z; +/* The node contains memory. Check that there is +* memory in the intended zonelist. +*/ + z =NODE_DATA (i%numnodes)->node_zonelists[gfp_mask &GFP_ZONEMASK ].zones; +while (*z) { +if ( (*z)->free_pages > (1UL <<order)) + goto found_node; + z++; + } + }
mm/page_alloc.c line 718
Quoted 10 lines
mm/page_alloc.c +10
-0+ i++; + } +found_node: + nodenr = i+1 ; + page =alloc_pages_node (i%numnodes, gfp_mask, order); +if (!page) +return 0 ; +return (unsigned long )page_address (page); +} +#endif
Should this not search for the emptiest node?
mm/page_alloc.c line 730
Quoted 8 lines
mm/page_alloc.c +4
-0@@ -688,6 +724,10 @@ {struct page * page; +#ifdef CONFIG_NUMA +if (unlikely (!system_running)) +return get_boot_pages (gfp_mask, order); +#endif
Is non-__init code allowed to call __init code? I thought that caused linkage errors on some setups. Pretty sure about that. I think, maybe.
Quoted 2 lines
Is this a thing which all NUMA machines want to be doing?
So far we have ppc64 and x86-64. I suspect the others will be OK with it.
Quoted 1 line
Should this not search for the emptiest node?
Allocating things round robin avoids a hot node where everything ends up being allocated.
Quoted 2 lines
Is non-__init code allowed to call
__initcode? I thought that caused linkage errors on some setups. Pretty sure about that. I think, maybe.
Maybe. Its news to me.
Anton
Quoted 5 lines · Anton Blanchard
Quoted 2 lines
Should this not search for the emptiest node?
Allocating things round robin avoids a hot node where everything ends up being allocated.
Have you any performance measurements for this patch?
Quoted 4 lines
Quoted 2 lines
Is non-__init code allowed to call
__initcode? I thought that caused linkage errors on some setups. Pretty sure about that. I think, maybe.Maybe. Its news to me.
I'll check.
Quoted 8 lines · Andrew Morton
Quoted 5 lines · Anton Blanchard
Quoted 2 lines
Should this not search for the emptiest node?
Allocating things round robin avoids a hot node where everything ends up being allocated.
Have you any performance measurements for this patch?
Any suggestions on what benchmark to run?
I tried the patch on Altix and saw similar balancing that Anton showed. The machine was a 64-way (32 node) machine with 256GB RAM.
mh
Signature · 2 lines
Martin Hicks Wild Open Source Inc. [email protected] 613-266-2296
Quoted 6 lines · Martin Hicks
Quoted 2 lines · Andrew Morton
Quoted 1 line · Anton Blanchard
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Should this not search for the emptiest node?
Allocating things round robin avoids a hot node where everything ends up being allocated.
Have you any performance measurements for this patch?
Any suggestions on what benchmark to run?
I guess SDET is the closest thing we have to a "mixed workload".
On Mon, 9 Feb 2004 09:56:29 -0800 Andrew Morton <[email protected]> wrote:
| Martin Hicks <[email protected]> wrote: | > | > | > | > On Sat, Feb 07, 2004 at 11:07:32AM -0800, Andrew Morton wrote: | > > Anton Blanchard <[email protected]> wrote: | > > > | > > > > Should this not search for the emptiest node? | > > > | > > > Allocating things round robin avoids a hot node where everything ends up | > > > being allocated. | > > | > > Have you any performance measurements for this patch? | > | > Any suggestions on what benchmark to run? | | I guess SDET is the closest thing we have to a "mixed workload". | -
Cliff White says that re-aim should also work for this.
https://sourceforge.net/projects/re-aim-7/
or use the OSDL STP interface.
--
~Randy