x86, NUMA: Unify 32 and 64bit NUMA initialization
PATCH 1/1 x86, NUMA: Unify 32 and 64bit NUMA initialization
Hello,
This patchset, finally, unifies 32 and 64bit NUMA initialization. It gradually moves 64bit stuff to common code and replaces 32bit code with it. Once the unification is complete, amdtopology and emulation are enabled for 32bit too (there's no reason not to).
This patchset contains the following 25 patches.
0001-x86-64-NUMA-Simplify-hotadd-memory-handling.patch 0002-x86-64-NUMA-trivial-cleanups-for-setup_node_bootmem.patch 0003-x86-64-NUMA-simplify-nodedata-allocation.patch 0004-x86-32-NUMA-Automatically-set-apicid-node-in-setup_l.patch 0005-x86-NUMA-Unify-32-64bit-numa_cpu_node-implementation.patch 0006-x86-32-NUMA-Make-apic-x86_32_numa_cpu_node-optional.patch 0007-x86-32-NUMA-use-sparse_memory_present_with_active_re.patch 0008-x86-NUMA-trivial-cleanups.patch
0009-x86-NUMA-rename-srat_64.c-to-srat.c.patch 0010-x86-NUMA-make-srat.c-32bit-safe.patch 0011-x86-32-NUMA-Move-get_memcfg_numa-into-numa_32.c.patch 0012-x86-NUMA-Move-numa_nodes_parsed-to-numa.-hc.patch 0013-x86-32-NUMA-implement-temporary-NUMA-init-shims.patch 0014-x86-32-NUMA-Replace-srat_32.c-with-srat.c.patch
0015-x86-32-NUMA-Update-numaq-to-use-new-NUMA-init-protoc.patch
0016-x86-NUMA-Move-NUMA-init-logic-from-numa_64.c-to-numa.patch 0017-x86-NUMA-Enable-build-of-generic-NUMA-init-code-on-3.patch 0018-x86-NUMA-Remove-long-64bit-assumption-from-numa.c.patch 0019-x86-32-NUMA-Add-start-and-end-to-init_alloc_remap.patch 0020-x86-NUMA-Initialize-and-use-remap-allocator-from-set.patch 0021-x86-NUMA-Make-32bit-use-common-NUMA-init-path.patch 0022-x86-NUMA-Make-numa_init_array-static.patch
0023-x86-NUMA-Rename-amdtopology_64.c-to-amdtopology.c.patch 0024-x86-NUMA-Enable-CONFIG_AMD_NUMA-on-32bit-too.patch 0025-x86-NUMA-Enable-emulation-on-32bit-too.patch
Patches can be grouped as follows.
0001-0008 are preparation patches. Please note that 0001 was posted before.
0009-0014 rename srat_64.c to srat.c and replace srat_32.c with it.
0015 converts NUMAQ to follow the new init protocol.
0016-0022 move 64bit NUMA init code from numa_64.c to numa.c and replace 32bit init code in numa_32.c with it.
0023-0025 enable amdtoplogy and emulation for 32bit too.
I've tested ACPI and amdtopology with and without emulation on 32 and 64bit with several different configurations. It all looks dandy but wider scope testing is definitely required. Also, I couldn't test NUMAQ at all.
Due to dependencies, the base commit this patchset is based on is a bit complex. This patchset is based on,
tip:x86/mm c7a7b814c9dca9ee01b38e63b4a46de87156d3b6
+ tip:x86/numa 993ba1585cbb03fab012e41d1a5d24330a283b31
+ Cherry pick of 765af22da8 (x86-32, NUMA: Fix ACPI NUMA init broken by recent x86-64 change)
The tip:x86/numa requirement is because of the "x86-32, NUMA: Clean up alloc_remap" patchset[2] which got committed there. I think it would have been better if they got routed together through x86/mm. Well, anyways, we can sort it out.
So, for review, please use the following git branch as base.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.git review-unify-numa-base
The whole patchset on top of the above base is available in the following git branch.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.git review-unify-numa
diffstat follows. 610 lines removed and 32bit NUMA got much better! :)
arch/x86/Kconfig | 4 arch/x86/include/asm/acpi.h | 2 arch/x86/include/asm/amd_nb.h | 1 arch/x86/include/asm/apic.h | 9 arch/x86/include/asm/dma.h | 12 arch/x86/include/asm/mmzone_32.h | 20 - arch/x86/include/asm/numa.h | 32 ++ arch/x86/include/asm/numa_32.h | 10 arch/x86/include/asm/numa_64.h | 36 -- arch/x86/include/asm/numaq.h | 7 arch/x86/include/asm/srat.h | 39 -- arch/x86/include/asm/topology.h | 7 arch/x86/kernel/apic/apic.c | 26 - arch/x86/kernel/apic/apic_noop.c | 9 arch/x86/kernel/apic/bigsmp_32.c | 1 arch/x86/kernel/apic/es7000_32.c | 7 arch/x86/kernel/apic/numaq_32.c | 30 -- arch/x86/kernel/apic/probe_32.c | 1 arch/x86/kernel/apic/summit_32.c | 1 arch/x86/mm/Makefile | 4 arch/x86/mm/amdtopology.c | 197 ++++++++++++++ arch/x86/mm/amdtopology_64.c | 196 -------------- arch/x86/mm/init_32.c | 1 arch/x86/mm/init_64.c | 8 arch/x86/mm/numa.c | 545 ++++++++++++++++++++++++++++++++++++++- arch/x86/mm/numa_32.c | 165 ----------- arch/x86/mm/numa_64.c | 519 ------------------------------------- arch/x86/mm/numa_emulation.c | 16 - arch/x86/mm/numa_internal.h | 8 arch/x86/mm/srat.c | 184 +++++++++++++ arch/x86/mm/srat_32.c | 287 -------------------- arch/x86/mm/srat_64.c | 260 ------------------ 32 files changed, 1017 insertions(+), 1627 deletions(-)
--
tejun
PATCH 1/25 x86-64, NUMA: Simplify hotadd memory handling
The only special handling NUMA needs to do for hotadd memory is determining the node for the hotadd memory given the address of it and there's nothing specific to specific config method used.
srat_64.c does somewhat elaborate error checking on ACPI_SRAT_MEM_HOT_PLUGGABLE regions, remembers them and implements memory_add_physaddr_to_nid() which determines the node for given hotadd address.
This is almost completely redundant. All the information is already available to the generic NUMA code which already performs all the sanity checking and merging. All that's necessary is not using __initdata from numa_meminfo and providing a function which uses it to map address to node.
Drop the specific implementation from srat_64.c and add generic memory_add_physaddr_to_nid() in numa_64.c, which is enabled if CONFIG_MEMORY_HOTPLUG is set. Other than dropping the code, srat_64.c doesn't need any change as it already calls numa_add_memblk() for hot pluggable regions which is enough.
While at it, change CONFIG_MEMORY_HOTPLUG_SPARSE in srat_64.c to CONFIG_MEMORY_HOTPLUG, for NUMA on x86-64, the two are always the same.
--- arch/x86/mm/init_64.c | 8 ----- arch/x86/mm/numa_64.c | 22 +++++++++++++- arch/x86/mm/srat_64.c | 78 +------------------------------------------------ 3 files changed, 22 insertions(+), 86 deletions(-)
arch/x86/mm/init_64.c +0 -8
@@ -679,14 +679,6 @@ int arch_add_memory(int nid, u64 start, u64 size) }EXPORT_SYMBOL_GPL (arch_add_memory); -#if !defined(CONFIG_ACPI_NUMA ) && defined(CONFIG_NUMA ) -int memory_add_physaddr_to_nid (u64 start) -{ -return 0 ; -} -EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid); -#endif -#endif /* CONFIG_MEMORY_HOTPLUG */ static struct kcore_list kcore_vsyscall;
arch/x86/mm/numa_64.c +21 -1
@@ -28,7 +28,12 @@ EXPORT_SYMBOL(node_data);nodemask_t numa_nodes_parsed__initdata ; -static struct numa_meminfo numa_meminfo__initdata ; +static struct numa_meminfo numa_meminfo +#ifndef CONFIG_MEMORY_HOTPLUG +__initdata +#endif +; +static int numa_distance_cnt;static u8 *numa_distance;
@@ -540,3 +545,18 @@ int __cpuinit numa_cpu_node(int cpu)return __apicid_to_node[apicid];return NUMA_NO_NODE ; } + +#ifdef CONFIG_MEMORY_HOTPLUG +int memory_add_physaddr_to_nid (u64 start) +{ +struct numa_meminfo *mi = &numa_meminfo; +int nid = mi->blk[0 ].nid; +int i; + +for (i =0 ; i < mi->nr_blks; i++) +if (mi->blk[i].start <= start && mi->blk[i].end > start) + nid = mi->blk[i].nid; +return nid; +} +EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid); +#endif
arch/x86/mm/srat_64.c +1 -77
109 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 2/25 x86-64, NUMA: trivial cleanups for setup_node_bootmem()
In reply to Tejun HeoMake the following trivial changes in preparation for further updates.
* nodeid -> nid, nid -> tnid
* use nd_ prefix for nodedata related variables
* remove start/end_pfn and use start/end directly
--- arch/x86/mm/numa_64.c | 52 +++++++++++++++++++++--------------------------- 1 files changed, 23 insertions(+), 29 deletions(-)
arch/x86/mm/numa_64.c +23 -29
@@ -128,14 +128,11 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)/* Initialize bootmem allocator for a node */ void __init -setup_node_bootmem (int nodeid,unsigned long start,unsigned long end) +setup_node_bootmem (int nid,unsigned long start,unsigned long end) { -unsigned long start_pfn, last_pfn, nodedata_phys; -const int pgdat_size =roundup (sizeof (pg_data_t),PAGE_SIZE ); -int nid; - -if (!end) -return ; +const size_t nd_size =roundup (sizeof (pg_data_t),PAGE_SIZE ); +unsigned long nd_pa; +int tnid;/* * Don't confuse VM with a node that doesn't have the
@@ -146,30 +143,27 @@ setup_node_bootmem(int nodeid, unsigned long start, unsigned long end) start =roundup (start,ZONE_ALIGN ); -printk (KERN_INFO "Initmem setup node %d %016lx-%016lx\n" , nodeid, - start, end); - - start_pfn = start >>PAGE_SHIFT ; - last_pfn = end >>PAGE_SHIFT ; +printk (KERN_INFO "Initmem setup node %d %016lx-%016lx\n" , + nid, start, end); - node_data[nodeid] =early_node_mem (nodeid, start, end, pgdat_size, -SMP_CACHE_BYTES ); -if (node_data[nodeid] ==NULL ) + node_data[nid] =early_node_mem (nid, start, end, nd_size, +SMP_CACHE_BYTES ); +if (node_data[nid] ==NULL )return ; - nodedata_phys =__pa (node_data[nodeid]); -memblock_x86_reserve_range (nodedata_phys, nodedata_phys + pgdat_size,"NODE_DATA" ); -printk (KERN_INFO " NODE_DATA [%016lx - %016lx]\n" , nodedata_phys, - nodedata_phys + pgdat_size -1 ); - nid =early_pfn_to_nid (nodedata_phys >>PAGE_SHIFT ); -if (nid != nodeid) -printk (KERN_INFO " NODE_DATA(%d) on node %d\n" , nodeid, nid); - -memset (NODE_DATA (nodeid),0 ,sizeof (pg_data_t)); -NODE_DATA (nodeid)->node_id = nodeid; -NODE_DATA (nodeid)->node_start_pfn = start_pfn; -NODE_DATA (nodeid)->node_spanned_pages = last_pfn - start_pfn; - -node_set_online (nodeid); + nd_pa =__pa (node_data[nid]); +memblock_x86_reserve_range (nd_pa, nd_pa + nd_size,"NODE_DATA" ); +printk (KERN_INFO " NODE_DATA [%016lx - %016lx]\n" , + nd_pa, nd_pa + nd_size -1 ); + tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); +if (tnid != nid) +printk (KERN_INFO " NODE_DATA(%d) on node %d\n" , nid, tnid); + +memset (NODE_DATA (nid),0 ,sizeof (pg_data_t)); +NODE_DATA (nid)->node_id = nid; +NODE_DATA (nid)->node_start_pfn = start >>PAGE_SHIFT ; +NODE_DATA (nid)->node_spanned_pages = (end - start) >>PAGE_SHIFT ; + +node_set_online (nid); }/**
Signature · 1 line
1.7.1
PATCH 3/25 x86-64, NUMA: simplify nodedata allocation
In reply to Tejun HeoWith top-down memblock allocation, the allocation range limits in ealry_node_mem() can be simplified - try node-local first, then any node but in any case don't allocate below DMA limit.
Remove early_node_mem() and implement simplified allocation directly in setup_node_bootmem().
--- arch/x86/mm/numa_64.c | 53 +++++++++++++++--------------------------------- 1 files changed, 17 insertions(+), 36 deletions(-)
arch/x86/mm/numa_64.c +17 -36
@@ -37,38 +37,6 @@ __initdatastatic int numa_distance_cnt;static u8 *numa_distance; -static void *__init early_node_mem (int nodeid,unsigned long start, -unsigned long end,unsigned long size, -unsigned long align) -{ -unsigned long mem; - -/* -* put it on high as possible -* something will go with NODE_DATA -*/ -if (start < (MAX_DMA_PFN <<PAGE_SHIFT )) - start =MAX_DMA_PFN <<PAGE_SHIFT ; -if (start < (MAX_DMA32_PFN <<PAGE_SHIFT ) && - end > (MAX_DMA32_PFN <<PAGE_SHIFT )) - start =MAX_DMA32_PFN <<PAGE_SHIFT ; - mem =memblock_x86_find_in_range_node (nodeid, start, end, size, align); -if (mem !=MEMBLOCK_ERROR ) -return __va (mem); - -/* extend the search scope */ - end = max_pfn_mapped <<PAGE_SHIFT ; - start =MAX_DMA_PFN <<PAGE_SHIFT ; - mem =memblock_find_in_range (start, end, size, align); -if (mem !=MEMBLOCK_ERROR ) -return __va (mem); - -printk (KERN_ERR "Cannot find %lu bytes in node %d\n" , - size, nodeid); - -return NULL ; -} -static int __init numa_add_memblk_to (int nid,u64 start,u64 end,struct numa_meminfo *mi) {
@@ -130,6 +98,8 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)void __init setup_node_bootmem (int nid,unsigned long start,unsigned long end) { +const u64 nd_low = (u64 )MAX_DMA_PFN <<PAGE_SHIFT ; +const u64 nd_high = (u64 )max_pfn_mapped <<PAGE_SHIFT ;const size_t nd_size =roundup (sizeof (pg_data_t),PAGE_SIZE );unsigned long nd_pa;int tnid;
@@ -146,18 +116,29 @@ setup_node_bootmem(int nid, unsigned long start, unsigned long end)printk (KERN_INFO "Initmem setup node %d %016lx-%016lx\n" , nid, start, end); - node_data[nid] =early_node_mem (nid, start, end, nd_size, -SMP_CACHE_BYTES ); -if (node_data[nid] ==NULL ) +/* +* Try to allocate node data on local node and then fall back to +* all nodes. Never allocate in DMA zone. +*/ + nd_pa =memblock_x86_find_in_range_node (nid, nd_low, nd_high, + nd_size,SMP_CACHE_BYTES ); +if (nd_pa ==MEMBLOCK_ERROR ) + nd_pa =memblock_find_in_range (nd_low, nd_high, + nd_size,SMP_CACHE_BYTES ); +if (nd_pa ==MEMBLOCK_ERROR ) { +pr_err ("Cannot find %lu bytes in node %d\n" , nd_size, nid);return ; - nd_pa =__pa (node_data[nid]); + }memblock_x86_reserve_range (nd_pa, nd_pa + nd_size,"NODE_DATA" ); + +/* report and initialize */ printk (KERN_INFO " NODE_DATA [%016lx - %016lx]\n" , nd_pa, nd_pa + nd_size -1 ); tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT );if (tnid != nid)printk (KERN_INFO " NODE_DATA(%d) on node %d\n" , nid, tnid); + node_data[nid] =__va (nd_pa);memset (NODE_DATA (nid),0 ,sizeof (pg_data_t));NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start >>PAGE_SHIFT ;
Signature · 1 line
1.7.1
Re: [PATCH 03/25] x86-64, NUMA: simplify nodedata allocation
Quoted 6 lines · Tejun Heo
With top-down memblock allocation, the allocation range limits in
ealry_node_mem()can be simplified - try node-local first, then any node but in any case don't allocate below DMA limit.Remove
early_node_mem()and implement simplified allocation directly insetup_node_bootmem().
keep early_node_mem would be better?
Yinghai
Quoted 97 lines
Signed-off-by Tejun Heo <[email protected]>Cc Ingo Molnar <[email protected]>Cc Yinghai Lu <[email protected]>Cc David Rientjes <[email protected]>Cc Thomas Gleixner <[email protected]>Cc "H. Peter Anvin" <[email protected]>--- arch/x86/mm/numa_64.c | 53 +++++++++++++++--------------------------------- 1 files changed, 17 insertions(+), 36 deletions(-)arch/x86/mm/numa_64.c +17
-36@@ -37,38 +37,6 @@ __initdatastatic int numa_distance_cnt;static u8 *numa_distance; -static void *__init early_node_mem (int nodeid,unsigned long start, -unsigned long end,unsigned long size, -unsigned long align) -{ -unsigned long mem; - -/* -* put it on high as possible -* something will go with NODE_DATA -*/ -if (start < (MAX_DMA_PFN <<PAGE_SHIFT )) - start =MAX_DMA_PFN <<PAGE_SHIFT ; -if (start < (MAX_DMA32_PFN <<PAGE_SHIFT ) && - end > (MAX_DMA32_PFN <<PAGE_SHIFT )) - start =MAX_DMA32_PFN <<PAGE_SHIFT ; - mem =memblock_x86_find_in_range_node (nodeid, start, end, size, align); -if (mem !=MEMBLOCK_ERROR ) -return __va (mem); - -/* extend the search scope */ - end = max_pfn_mapped <<PAGE_SHIFT ; - start =MAX_DMA_PFN <<PAGE_SHIFT ; - mem =memblock_find_in_range (start, end, size, align); -if (mem !=MEMBLOCK_ERROR ) -return __va (mem); - -printk (KERN_ERR "Cannot find %lu bytes in node %d\n" , - size, nodeid); - -return NULL ; -} -static int __init numa_add_memblk_to (int nid,u64 start,u64 end,struct numa_meminfo *mi) {@@ -130,6 +98,8 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)void __init setup_node_bootmem (int nid,unsigned long start,unsigned long end) { +const u64 nd_low = (u64 )MAX_DMA_PFN <<PAGE_SHIFT ; +const u64 nd_high = (u64 )max_pfn_mapped <<PAGE_SHIFT ;const size_t nd_size =roundup (sizeof (pg_data_t),PAGE_SIZE );unsigned long nd_pa;int tnid;@@ -146,18 +116,29 @@ setup_node_bootmem(int nid, unsigned long start, unsigned long end)printk (KERN_INFO "Initmem setup node %d %016lx-%016lx\n" , nid, start, end); - node_data[nid] =early_node_mem (nid, start, end, nd_size, -SMP_CACHE_BYTES ); -if (node_data[nid] ==NULL ) +/* +* Try to allocate node data on local node and then fall back to +* all nodes. Never allocate in DMA zone. +*/ + nd_pa =memblock_x86_find_in_range_node (nid, nd_low, nd_high, + nd_size,SMP_CACHE_BYTES ); +if (nd_pa ==MEMBLOCK_ERROR ) + nd_pa =memblock_find_in_range (nd_low, nd_high, + nd_size,SMP_CACHE_BYTES ); +if (nd_pa ==MEMBLOCK_ERROR ) { +pr_err ("Cannot find %lu bytes in node %d\n" , nd_size, nid);return ; - nd_pa =__pa (node_data[nid]); + }memblock_x86_reserve_range (nd_pa, nd_pa + nd_size,"NODE_DATA" ); + +/* report and initialize */ printk (KERN_INFO " NODE_DATA [%016lx - %016lx]\n" , nd_pa, nd_pa + nd_size -1 ); tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT );if (tnid != nid)printk (KERN_INFO " NODE_DATA(%d) on node %d\n" , nid, tnid); + node_data[nid] =__va (nd_pa);memset (NODE_DATA (nid),0 ,sizeof (pg_data_t));NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start >>PAGE_SHIFT ;
Re: [PATCH 03/25] x86-64, NUMA: simplify nodedata allocation
Quoted 9 lines · Yinghai Lu
Quoted 6 lines · Tejun Heo
With top-down memblock allocation, the allocation range limits in
ealry_node_mem()can be simplified - try node-local first, then any node but in any case don't allocate below DMA limit.Remove
early_node_mem()and implement simplified allocation directly insetup_node_bootmem().keep
early_node_memwould be better?
I don't know, maybe, maybe not. I usually find separating out linear procedural logic into a function, which is used only once, more distracting / obfuscating than helpful.
Signature · 1 line
tejun
PATCH 4/25 x86-32, NUMA: Automatically set apicid -> node in setup_local_APIC()
In reply to Tejun HeoSome x86-32 NUMA implementations (NUMAQ) don't initialize apicid -> node mapping using set_apicid_to_node() during NUMA init but implement custom apic->x86_32_numa_cpu_node() instead.
This patch automatically initializes the default apic -> node mapping table from apic->x86_32_numa_cpu_node() from setup_local_APIC() such that the mapping table is in sync with the actual mapping.
As the table isn't used by custom implementations, this doesn't make any difference at this point. This is in preparation of unifying numa_cpu_node() between x86-32 and 64.
--- arch/x86/kernel/apic/apic.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-)
arch/x86/kernel/apic/apic.c +10 -0
@@ -1237,6 +1237,16 @@ void __cpuinit setup_local_APIC(void)/* always use the value from LDR */ early_per_cpu (x86_cpu_to_logical_apicid, cpu) =logical_smp_processor_id (); + +/* +* Some NUMA implementations (NUMAQ) don't initialize apicid to +* node mapping during NUMA init. Now that logical apicid is +* guaranteed to be known, give it another chance. This is already +* a bit too late - percpu allocation has already happened without +* proper NUMA affinity. +*/ +set_apicid_to_node (early_per_cpu (x86_cpu_to_apicid, cpu), + apic->x86_32_numa_cpu_node (cpu));#endif /*
Signature · 1 line
1.7.1
PATCH 5/25 x86, NUMA: Unify 32/64bit numa_cpu_node() implementation
In reply to Tejun HeoCurrently, the only meaningful user of apic->x86_32_numa_cpu_node() is NUMAQ which returns valid mapping only after CPU is initialized during SMP bringup; thus, the previous patch to set apicid -> node in setup_local_APIC() makes __apicid_to_node[] always contain the correct mapping whether custom apic->x86_32_numa_cpu_node() is used or not.
So, there is no reason to keep separate 32bit implementation. We can always consult __apicid_to_node[]. Move 64bit implementation from numa_64.c to numa.c and remove 32bit implementation from numa_32.c.
--- arch/x86/include/asm/numa.h | 10 ++++++++++ arch/x86/include/asm/numa_32.h | 6 ------ arch/x86/include/asm/numa_64.h | 3 --- arch/x86/mm/numa.c | 9 +++++++++ arch/x86/mm/numa_32.c | 5 ----- arch/x86/mm/numa_64.c | 9 --------- 6 files changed, 19 insertions(+), 23 deletions(-)
arch/x86/include/asm/numa.h +10 -0
@@ -1,6 +1,8 @@#ifndef _ASM_X86_NUMA_H#define _ASM_X86_NUMA_H +#include <linux/nodemask.h> +#include <asm/topology.h> #include <asm/apicdef.h>
@@ -22,10 +24,18 @@ static inline void set_apicid_to_node(int apicid, s16 node) { __apicid_to_node[apicid] = node; } + +extern int __cpuinit numa_cpu_node (int cpu); +#else /* CONFIG_NUMA */ static inline void set_apicid_to_node (int apicid,s16 node) { } + +static inline int numa_cpu_node (int cpu) +{ +return NUMA_NO_NODE ; +}#endif /* CONFIG_NUMA */ #ifdef CONFIG_X86_32
arch/x86/include/asm/numa_32.h +0 -6
@@ -5,12 +5,6 @@ extern int numa_off;extern int pxm_to_nid (int pxm); -#ifdef CONFIG_NUMA -extern int __cpuinit numa_cpu_node (int cpu); -#else /* CONFIG_NUMA */ -static inline int numa_cpu_node (int cpu) {return NUMA_NO_NODE ; } -#endif /* CONFIG_NUMA */ -#ifdef CONFIG_HIGHMEM extern void set_highmem_pages_init (void );#else
arch/x86/include/asm/numa_64.h +0 -3
@@ -26,7 +26,6 @@ extern void setup_node_bootmem(int nodeid, unsigned long start,extern nodemask_t numa_nodes_parsed__initdata ; -extern int __cpuinit numa_cpu_node (int cpu);extern int __init numa_add_memblk (int nodeid,u64 start,u64 end);extern void __init numa_set_distance (int from,int to,int distance);
@@ -35,8 +34,6 @@ extern void __init numa_set_distance(int from, int to, int distance);#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL))void numa_emu_cmdline (char *);#endif /* CONFIG_NUMA_EMU */ -#else -static inline int numa_cpu_node (int cpu) {return NUMA_NO_NODE ; }#endif #endif /* _ASM_X86_NUMA_64_H */
arch/x86/mm/numa.c +9 -0
@@ -32,6 +32,15 @@ s16 __apicid_to_node[MAX_LOCAL_APIC] __cpuinitdata = { [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE }; +int __cpuinit numa_cpu_node (int cpu) +{ +int apicid =early_per_cpu (x86_cpu_to_apicid, cpu); + +if (apicid !=BAD_APICID ) +return __apicid_to_node[apicid]; +return NUMA_NO_NODE ; +} +cpumask_var_t node_to_cpumask_map[MAX_NUMNODES ];EXPORT_SYMBOL (node_to_cpumask_map);
arch/x86/mm/numa_32.c +0 -5
@@ -107,11 +107,6 @@ extern unsigned long highend_pfn, highstart_pfn;static void *node_remap_start_vaddr[MAX_NUMNODES ];void set_pmd_pfn (unsigned long vaddr,unsigned long pfn,pgprot_t flags); -int __cpuinit numa_cpu_node (int cpu) -{ -return apic->x86_32_numa_cpu_node (cpu); -} -/* * FLAT - support for basic PC memory model with discontig enabled, essentially * a single node with all available processors in it with a flat
arch/x86/mm/numa_64.c +0 -9
@@ -512,15 +512,6 @@ unsigned long __init numa_free_all_bootmem(void)return pages; } -int __cpuinit numa_cpu_node (int cpu) -{ -int apicid =early_per_cpu (x86_cpu_to_apicid, cpu); - -if (apicid !=BAD_APICID ) -return __apicid_to_node[apicid]; -return NUMA_NO_NODE ; -} -#ifdef CONFIG_MEMORY_HOTPLUG int memory_add_physaddr_to_nid (u64 start) {
Signature · 1 line
1.7.1
PATCH 6/25 x86-32, NUMA: Make apic->x86_32_numa_cpu_node() optional
In reply to Tejun HeoNUMAQ is the only meaningful user of this callback and setup_local_APIC() the only callsite. Stop torturing everyone else by making the callback optional and removing all the boilerplate implementations and assignments.
--- arch/x86/include/asm/apic.h | 9 ++++++--- arch/x86/kernel/apic/apic.c | 20 +++----------------- arch/x86/kernel/apic/apic_noop.c | 9 --------- arch/x86/kernel/apic/bigsmp_32.c | 1 - arch/x86/kernel/apic/es7000_32.c | 7 ------- arch/x86/kernel/apic/probe_32.c | 1 - arch/x86/kernel/apic/summit_32.c | 1 - 7 files changed, 9 insertions(+), 39 deletions(-)
arch/x86/include/asm/apic.h +6 -3
@@ -363,7 +363,12 @@ struct apic {*/ int (*x86_32_early_logical_apicid)(int cpu); -/* determine CPU -> NUMA node mapping */ +/* +* Optional method called from setup_local_APIC() after logical +* apicid is guaranteed to be known to initialize apicid -> node +* mapping if NUMA initialization hasn't done so already. Don't +* add new users. +*/ int (*x86_32_numa_cpu_node)(int cpu);#endif };
@@ -537,8 +542,6 @@ static inline int default_phys_pkg_id(int cpuid_apic, int index_msb)return cpuid_apic >> index_msb; } -extern int default_x86_32_numa_cpu_node (int cpu); -#endif static inline unsigned int
arch/x86/kernel/apic/apic.c +3 -17
@@ -1245,8 +1245,9 @@ void __cpuinit setup_local_APIC(void)* a bit too late - percpu allocation has already happened without * proper NUMA affinity. */ -set_apicid_to_node (early_per_cpu (x86_cpu_to_apicid, cpu), - apic->x86_32_numa_cpu_node (cpu)); +if (apic->x86_32_numa_cpu_node) +set_apicid_to_node (early_per_cpu (x86_cpu_to_apicid, cpu), + apic->x86_32_numa_cpu_node (cpu));#endif /*
@@ -2013,21 +2014,6 @@ void default_init_apic_ldr(void)apic_write (APIC_LDR , val); } -#ifdef CONFIG_X86_32 -int default_x86_32_numa_cpu_node (int cpu) -{ -#ifdef CONFIG_NUMA -int apicid =early_per_cpu (x86_cpu_to_apicid, cpu); - -if (apicid !=BAD_APICID ) -return __apicid_to_node[apicid]; -return NUMA_NO_NODE ; -#else -return 0 ; -#endif -} -#endif -/* * Power management */
arch/x86/kernel/apic/apic_noop.c +0 -9
@@ -119,14 +119,6 @@ static void noop_apic_write(u32 reg, u32 v)WARN_ON_ONCE (cpu_has_apic && !disable_apic); } -#ifdef CONFIG_X86_32 -static int noop_x86_32_numa_cpu_node (int cpu) -{ -/* we're always on node 0 */ -return 0 ; -} -#endif -struct apic apic_noop = { .name ="noop" , .probe = noop_probe,
@@ -195,6 +187,5 @@ struct apic apic_noop = {#ifdef CONFIG_X86_32 .x86_32_early_logical_apicid = noop_x86_32_early_logical_apicid, - .x86_32_numa_cpu_node = noop_x86_32_numa_cpu_node,#endif };
arch/x86/kernel/apic/bigsmp_32.c +0 -1
@@ -253,5 +253,4 @@ struct apic apic_bigsmp = { .safe_wait_icr_idle = native_safe_apic_wait_icr_idle, .x86_32_early_logical_apicid = bigsmp_early_logical_apicid, - .x86_32_numa_cpu_node = default_x86_32_numa_cpu_node, };
arch/x86/kernel/apic/es7000_32.c +0 -7
@@ -510,11 +510,6 @@ static void es7000_setup_apic_routing(void) nr_ioapics, cpumask_bits(es7000_target_cpus())[0]); } -static int es7000_numa_cpu_node (int cpu) -{ -return 0 ; -} -static int es7000_cpu_present_to_apicid (int mps_cpu) {if (!mps_cpu)
@@ -688,7 +683,6 @@ struct apic __refdata apic_es7000_cluster = { .safe_wait_icr_idle = native_safe_apic_wait_icr_idle, .x86_32_early_logical_apicid = es7000_early_logical_apicid, - .x86_32_numa_cpu_node = es7000_numa_cpu_node, };struct apic __refdata apic_es7000 = {
@@ -752,5 +746,4 @@ struct apic __refdata apic_es7000 = { .safe_wait_icr_idle = native_safe_apic_wait_icr_idle, .x86_32_early_logical_apicid = es7000_early_logical_apicid, - .x86_32_numa_cpu_node = es7000_numa_cpu_node, };
arch/x86/kernel/apic/probe_32.c +0 -1
@@ -172,7 +172,6 @@ struct apic apic_default = { .safe_wait_icr_idle = native_safe_apic_wait_icr_idle, .x86_32_early_logical_apicid = default_x86_32_early_logical_apicid, - .x86_32_numa_cpu_node = default_x86_32_numa_cpu_node, };extern struct apic apic_numaq;
arch/x86/kernel/apic/summit_32.c +0 -1
@@ -551,5 +551,4 @@ struct apic apic_summit = { .safe_wait_icr_idle = native_safe_apic_wait_icr_idle, .x86_32_early_logical_apicid = summit_early_logical_apicid, - .x86_32_numa_cpu_node = default_x86_32_numa_cpu_node, };
Signature · 1 line
1.7.1
PATCH 7/25 x86-32, NUMA: use sparse_memory_present_with_active_regions()
In reply to Tejun HeoInstead of calling memory_present() for each region from NUMA init, call sparse_memory_present_with_active_regions() from paging_init() similarly to x86-64.
For flat and numaq, this results in exactly the same memory_present() calls. For srat, if there are multiple memory chunks for a node, after this change, memory_present() will be called separately for each chunk instead of being called once to encompass the whole range, which doesn't cause any harm and actually is the better behavior.
--- arch/x86/kernel/apic/numaq_32.c | 2 -- arch/x86/mm/init_32.c | 1 + arch/x86/mm/numa_32.c | 1 - arch/x86/mm/srat_32.c | 8 +------- 4 files changed, 2 insertions(+), 10 deletions(-)
arch/x86/kernel/apic/numaq_32.c +0 -2
@@ -91,8 +91,6 @@ static inline void numaq_register_node(int node, struct sys_cfg_data *scd)memblock_x86_register_active_regions (node, node_start_pfn[node], node_end_pfn[node]); - -memory_present (node, node_start_pfn[node], node_end_pfn[node]); }/*
arch/x86/mm/init_32.c +1 -0
@@ -716,6 +716,7 @@ void __init paging_init(void)* NOTE: at this point the bootmem allocator is fully available. */ olpc_dt_build_devicetree (); +sparse_memory_present_with_active_regions (MAX_NUMNODES );sparse_init ();zone_sizes_init (); }
arch/x86/mm/numa_32.c +0 -1
@@ -119,7 +119,6 @@ int __init get_memcfg_numa_flat(void) node_start_pfn[0 ] =0 ; node_end_pfn[0 ] = max_pfn;memblock_x86_register_active_regions (0 ,0 , max_pfn); -memory_present (0 ,0 , max_pfn);/* Indicate there is one node available. */ nodes_clear (node_online_map);
arch/x86/mm/srat_32.c +1 -7
@@ -209,7 +209,7 @@ static __init int node_read_chunk(int nid, struct node_memory_chunk_s *memory_chint __init get_memcfg_from_srat (void ) { -int i, j, nid; +int i, j;if (srat_disabled ()) goto out_fail;
@@ -273,12 +273,6 @@ int __init get_memcfg_from_srat(void)/* for out of order entries in SRAT */ sort_node_map (); -for_each_online_node (nid) { -unsigned long start = node_start_pfn[nid]; -unsigned long end =min (node_end_pfn[nid], max_pfn); - -memory_present (nid, start, end); - }return 1 ; out_fail: printk(KERN_DEBUG "failed to get NUMA memory information from SRAT"
Signature · 1 line
1.7.1
PATCH 8/25 x86, NUMA: trivial cleanups
In reply to Tejun Heo* Kill no longer used struct bootnode.
* Kill dangling declaration of pxm_to_nid() in numa_32.h.
* Make setup_node_bootmem() static.
--- arch/x86/include/asm/acpi.h | 2 -- arch/x86/include/asm/amd_nb.h | 1 - arch/x86/include/asm/numa_32.h | 2 -- arch/x86/include/asm/numa_64.h | 7 ------- arch/x86/mm/numa_64.c | 2 +- 5 files changed, 1 insertions(+), 13 deletions(-)
arch/x86/include/asm/acpi.h +0 -2
@@ -183,8 +183,6 @@ static inline void disable_acpi(void) { }#define ARCH_HAS_POWER_INIT 1 -struct bootnode ; -#ifdef CONFIG_ACPI_NUMA extern int acpi_numa;extern int x86_acpi_numa_init (void );
arch/x86/include/asm/amd_nb.h +0 -1
@@ -11,7 +11,6 @@ struct amd_nb_bus_dev_range {extern const struct pci_device_id amd_nb_misc_ids[];extern const struct amd_nb_bus_dev_range amd_nb_bus_dev_ranges[]; -struct bootnode ;extern bool early_is_amd_nb (u32 value);extern int amd_cache_northbridges (void );
arch/x86/include/asm/numa_32.h +0 -2
@@ -3,8 +3,6 @@extern int numa_off; -extern int pxm_to_nid (int pxm); -#ifdef CONFIG_HIGHMEM extern void set_highmem_pages_init (void );#else
arch/x86/include/asm/numa_64.h +0 -7
@@ -3,18 +3,11 @@#include <linux/nodemask.h> -struct bootnode { -u64 start; -u64 end; -}; -#define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT))extern int numa_off;extern unsigned long numa_free_all_bootmem (void ); -extern void setup_node_bootmem (int nodeid,unsigned long start, -unsigned long end);#ifdef CONFIG_NUMA /*
arch/x86/mm/numa_64.c +1 -1
@@ -95,7 +95,7 @@ int __init numa_add_memblk(int nid, u64 start, u64 end) }/* Initialize bootmem allocator for a node */ -void __init +static void __init setup_node_bootmem (int nid,unsigned long start,unsigned long end) {const u64 nd_low = (u64 )MAX_DMA_PFN <<PAGE_SHIFT ;
Signature · 1 line
1.7.1
Re: [PATCH 08/25] x86, NUMA: trivial cleanups
Quoted 5 lines · Tejun Heo
* Kill no longer used struct bootnode.
* Kill dangling declaration of
pxm_to_nid()innuma_32.h.* Make
setup_node_bootmem()static.
first one and third one should appear in patches that I posted before.
Yinghai
Quoted 89 lines
Signed-off-by Tejun Heo <[email protected]>Cc Ingo Molnar <[email protected]>Cc Yinghai Lu <[email protected]>Cc David Rientjes <[email protected]>Cc Thomas Gleixner <[email protected]>Cc "H. Peter Anvin" <[email protected]>--- arch/x86/include/asm/acpi.h | 2 -- arch/x86/include/asm/amd_nb.h | 1 - arch/x86/include/asm/numa_32.h | 2 -- arch/x86/include/asm/numa_64.h | 7 ------- arch/x86/mm/numa_64.c | 2 +- 5 files changed, 1 insertions(+), 13 deletions(-)arch/x86/include/asm/acpi.h +0
-2@@ -183,8 +183,6 @@ static inline void disable_acpi(void) { }#define ARCH_HAS_POWER_INIT 1 -struct bootnode ; -#ifdef CONFIG_ACPI_NUMA extern int acpi_numa;extern int x86_acpi_numa_init (void );arch/x86/include/asm/amd_nb.h +0
-1@@ -11,7 +11,6 @@ struct amd_nb_bus_dev_range {extern const struct pci_device_id amd_nb_misc_ids[];extern const struct amd_nb_bus_dev_range amd_nb_bus_dev_ranges[]; -struct bootnode ;extern bool early_is_amd_nb (u32 value);extern int amd_cache_northbridges (void );arch/x86/include/asm/numa_32.h +0
-2@@ -3,8 +3,6 @@extern int numa_off; -extern int pxm_to_nid (int pxm); -#ifdef CONFIG_HIGHMEM extern void set_highmem_pages_init (void );#else arch/x86/include/asm/numa_64.h +0
-7@@ -3,18 +3,11 @@#include <linux/nodemask.h> -struct bootnode { -u64 start; -u64 end; -}; -#define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT))extern int numa_off;extern unsigned long numa_free_all_bootmem (void ); -extern void setup_node_bootmem (int nodeid,unsigned long start, -unsigned long end);#ifdef CONFIG_NUMA /* arch/x86/mm/numa_64.c +1
-1@@ -95,7 +95,7 @@ int __init numa_add_memblk(int nid, u64 start, u64 end) }/* Initialize bootmem allocator for a node */ -void __init +static void __init setup_node_bootmem (int nid,unsigned long start,unsigned long end) {const u64 nd_low = (u64 )MAX_DMA_PFN <<PAGE_SHIFT ;
Re: [PATCH 08/25] x86, NUMA: trivial cleanups
Quoted 8 lines · Yinghai Lu
Quoted 5 lines · Tejun Heo
* Kill no longer used struct bootnode.
* Kill dangling declaration of
pxm_to_nid()innuma_32.h.* Make
setup_node_bootmem()static.first one and third one should appear in patches that I posted before.
Sorry, can't understand what you mean. What are you trying to say?
Signature · 1 line
tejun
Re: [PATCH 08/25] x86, NUMA: trivial cleanups
Quoted 4 lines · Tejun Heo
Quoted 3 lines · Yinghai Lu
Quoted 2 lines · Tejun Heo
* Kill no longer used struct bootnode.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
* Kill dangling declaration of
pxm_to_nid()innuma_32.h.* Make
setup_node_bootmem()static.first one and third one should appear in patches that I posted before.
Sorry, can't understand what you mean. What are you trying to say?
I posted two patches for
Kill no longer used struct bootnode.
Make setup_node_bootmem() static
Also setup_node_bootmem now is some misleading, could change to setup_node_data() ?
Thanks
Yinghai
Re: [PATCH 08/25] x86, NUMA: trivial cleanups
Hello,
Quoted 3 lines · Yinghai Lu
I posted two patches for
Kill no longer used struct bootnode.
Makesetup_node_bootmem()static
Ah, okay. Do you mind resending them to me? I'll integrate it with the patchset.
Quoted 1 line
Also
setup_node_bootmemnow is some misleading, could change tosetup_node_data()?
Sure, I don't like the current name either. Do you mind sending a patch for that too?
Thanks.
Signature · 1 line
tejun
Re: [PATCH 08/25] x86, NUMA: trivial cleanups
Quoted 9 lines · Tejun Heo
Hello,
Quoted 3 lines · Yinghai Lu
I posted two patches for
Kill no longer used struct bootnode.
Makesetup_node_bootmem()staticAh, okay. Do you mind resending them to me? I'll integrate it with the patchset.
Never mind. Do not want to waste your time to rebase your tree.
Quoted 5 lines
Quoted 2 lines
Also
setup_node_bootmemnow is some misleading, could change tosetup_node_data()?Sure, I don't like the current name either. Do you mind sending a patch for that too?
ok, another mail.
Thanks
Yinghai Lu
PATCH 1/1 x86, numa: Rename setup_node_bootmem to setup_node_data
In reply to Tejun Heoafter using memblock to replace bootmem, that function only setup node_data now.
Change the name to reflecting the real work.
--- arch/x86/mm/numa.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/mm/numa.c +3 -3
@@ -189,8 +189,8 @@ int __init numa_add_memblk(int nid, u64return numa_add_memblk_to (nid, start, end, &numa_meminfo); } -/* Initialize bootmem allocator for a node */ -static void __init setup_node_bootmem (int nid,u64 start,u64 end) +/* Initialize NODE_DATA for a node on the local memory */ +static void __init setup_node_data (int nid,u64 start,u64 end) {const u64 nd_low =PFN_PHYS (MAX_DMA_PFN );const u64 nd_high =PFN_PHYS (max_pfn_mapped);
@@ -522,7 +522,7 @@ static int __init numa_register_memblks( }if (start < end) -setup_node_bootmem (nid, start, end); +setup_node_data (nid, start, end); }return 0 ;
PATCH 9/25 x86, NUMA: rename srat_64.c to srat.c
In reply to Tejun HeoRename srat_64.c to srat.c. This is to prepare for unification of NUMA init paths between 32 and 64bit.
--- arch/x86/mm/Makefile | 5 +- arch/x86/mm/srat.c | 184 +++++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/mm/srat_64.c | 184 ------------------------------------------------- 3 files changed, 188 insertions(+), 185 deletions(-) create mode 100644 arch/x86/mm/srat.c delete mode 100644 arch/x86/mm/srat_64.c
arch/x86/mm/Makefile +4 -1
@@ -24,7 +24,10 @@ obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o obj-$(CONFIG_NUMA) += numa.o numa_$(BITS).o obj-$(CONFIG_AMD_NUMA) += amdtopology_64.o -obj-$(CONFIG_ACPI_NUMA) += srat_$(BITS).o +ifeq ($(CONFIG_ACPI_NUMA),y) +obj-$(CONFIG_X86_64) += srat.o +obj-$(CONFIG_X86_32) += srat_32.o +endif obj-$(CONFIG_NUMA_EMU) += numa_emulation.o obj-$(CONFIG_HAVE_MEMBLOCK) += memblock.o
arch/x86/mm/srat.c new file +184 -0
184 lines not on this page · raw message
arch/x86/mm/srat_64.c deleted +0 -184
184 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 10/25 x86, NUMA: make srat.c 32bit safe
In reply to Tejun HeoMake srat.c 32bit safe by removing the assumption that unsigned long is 64bit.
--- arch/x86/mm/srat.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/mm/srat.c +2 -2
@@ -138,7 +138,7 @@ static inline int save_add_info(void) {return 0;}void __init acpi_numa_memory_affinity_init (struct acpi_srat_mem_affinity *ma) { -unsigned long start, end; +u64 start, end;int node, pxm;if (srat_disabled ())
@@ -167,7 +167,7 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)return ; } -printk (KERN_INFO "SRAT: Node %u PXM %u %lx-%lx\n" , node, pxm, +printk (KERN_INFO "SRAT: Node %u PXM %u %Lx-%Lx\n" , node, pxm, start, end); }
Signature · 1 line
1.7.1
PATCH 11/25 x86-32, NUMA: Move get_memcfg_numa() into numa_32.c
In reply to Tejun HeoThere's no reason get_memcfg_numa() to be implemented inline in mmzone_32.h. Move it to numa_32.c and also make get_memcfg_numa_flag() static.
--- arch/x86/include/asm/mmzone_32.h | 18 ------------------ arch/x86/mm/numa_32.c | 11 ++++++++++- 2 files changed, 10 insertions(+), 19 deletions(-)
arch/x86/include/asm/mmzone_32.h +0 -18
@@ -16,28 +16,10 @@ extern struct pglist_data *node_data[];/* summit or generic arch */ #include <asm/srat.h> -extern int get_memcfg_numa_flat (void ); -/* -* This allows any one NUMA architecture to be compiled -* for, and still fall back to the flat function if it -* fails. -*/ -static inline void get_memcfg_numa (void ) -{ - -if (get_memcfg_numaq ()) -return ; -if (get_memcfg_from_srat ()) -return ; -get_memcfg_numa_flat (); -} -extern void resume_map_numa_kva (pgd_t *pgd);#else /* !CONFIG_NUMA */ -#define get_memcfg_numa get_memcfg_numa_flat -static inline void resume_map_numa_kva (pgd_t *pgd) {}#endif /* CONFIG_NUMA */
arch/x86/mm/numa_32.c +10 -1
@@ -112,7 +112,7 @@ void set_pmd_pfn(unsigned long vaddr, unsigned long pfn, pgprot_t flags);* a single node with all available processors in it with a flat * memory map. */ -int __init get_memcfg_numa_flat (void ) +static int __init get_memcfg_numa_flat (void ) {printk (KERN_DEBUG "NUMA - single node, flat memory mode\n" );
@@ -332,6 +332,15 @@ static __init void init_alloc_remap(int nid) nid, node_pa, node_pa + size, remap_va, remap_va + size); } +static void get_memcfg_numa (void ) +{ +if (get_memcfg_numaq ()) +return ; +if (get_memcfg_from_srat ()) +return ; +get_memcfg_numa_flat (); +} +void __init initmem_init (void ) {int nid;
Signature · 1 line
1.7.1
PATCH 12/25 x86, NUMA: Move numa_nodes_parsed to numa.[hc]
In reply to Tejun HeoMove numa_nodes_parsed from numa_64.[hc] to numa.[hc] to prepare for NUMA init path unification.
--- arch/x86/include/asm/numa.h | 1 + arch/x86/include/asm/numa_64.h | 4 ---- arch/x86/mm/numa.c | 1 + arch/x86/mm/numa_64.c | 2 -- 4 files changed, 2 insertions(+), 6 deletions(-)
arch/x86/include/asm/numa.h +1 -0
@@ -19,6 +19,7 @@* numa_cpu_node(). */ extern s16 __apicid_to_node[MAX_LOCAL_APIC ]; +extern nodemask_t numa_nodes_parsed__initdata ;static inline void set_apicid_to_node (int apicid,s16 node) {
arch/x86/include/asm/numa_64.h +0 -4
@@ -1,8 +1,6 @@#ifndef _ASM_X86_NUMA_64_H#define _ASM_X86_NUMA_64_H -#include <linux/nodemask.h> -#define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT))extern int numa_off;
@@ -17,8 +15,6 @@ extern unsigned long numa_free_all_bootmem(void);*/ #define NODE_MIN_SIZE (4*1024*1024) -extern nodemask_t numa_nodes_parsed__initdata ; -extern int __init numa_add_memblk (int nodeid,u64 start,u64 end);extern void __init numa_set_distance (int from,int to,int distance);
arch/x86/mm/numa.c +1 -0
@@ -6,6 +6,7 @@#include <asm/acpi.h> int __initdata numa_off; +nodemask_t numa_nodes_parsed__initdata ;static __init int numa_setup (char *opt) {
arch/x86/mm/numa_64.c +0 -2
@@ -26,8 +26,6 @@struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ;EXPORT_SYMBOL (node_data); -nodemask_t numa_nodes_parsed__initdata ; - static struct numa_meminfo numa_meminfo#ifndef CONFIG_MEMORY_HOTPLUG __initdata
Signature · 1 line
1.7.1
PATCH 13/25 x86-32, NUMA: implement temporary NUMA init shims
In reply to Tejun HeoTo help transition to common NUMA init, implement temporary 32bit shims for numa_add_memblk() and numa_set_distance(). numa_add_memblk() registers the memblk and adjusts node_start/end_pfn[]. numa_set_distance() is noop.
These shims will allow using 64bit NUMA init functions on 32bit and gradual transition to common NUMA init path.
For detailed description, please read description of commits which make use of the shim functions.
--- arch/x86/include/asm/numa.h | 3 +++ arch/x86/include/asm/numa_64.h | 3 --- arch/x86/mm/numa_32.c | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 37 insertions(+), 3 deletions(-)
arch/x86/include/asm/numa.h +3 -0
@@ -21,6 +21,9 @@extern s16 __apicid_to_node[MAX_LOCAL_APIC ];extern nodemask_t numa_nodes_parsed__initdata ; +extern int __init numa_add_memblk (int nodeid,u64 start,u64 end); +extern void __init numa_set_distance (int from,int to,int distance); +static inline void set_apicid_to_node (int apicid,s16 node) { __apicid_to_node[apicid] = node;
arch/x86/include/asm/numa_64.h +0 -3
@@ -15,9 +15,6 @@ extern unsigned long numa_free_all_bootmem(void);*/ #define NODE_MIN_SIZE (4*1024*1024) -extern int __init numa_add_memblk (int nodeid,u64 start,u64 end); -extern void __init numa_set_distance (int from,int to,int distance); -#ifdef CONFIG_NUMA_EMU #define FAKE_NODE_MIN_SIZE ((u64)32 << 20)#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL))
arch/x86/mm/numa_32.c +34 -0
@@ -414,3 +414,37 @@ int memory_add_physaddr_to_nid(u64 addr)EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid);#endif +/* temporary shim, will go away soon */ +int __init numa_add_memblk (int nid,u64 start,u64 end) +{ +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long end_pfn = end >>PAGE_SHIFT ; + +printk (KERN_DEBUG "nid %d start_pfn %08lx end_pfn %08lx\n" , + nid, start_pfn, end_pfn); + +if (start >= (u64 )max_pfn <<PAGE_SHIFT ) { +printk (KERN_INFO "Ignoring SRAT pfns: %08lx - %08lx\n" , + start_pfn, end_pfn); +return 0 ; + } + +node_set_online (nid); +memblock_x86_register_active_regions (nid, start_pfn, +min (end_pfn, max_pfn)); + +if (!node_has_online_mem (nid)) { + node_start_pfn[nid] = start_pfn; + node_end_pfn[nid] = end_pfn; + }else { + node_start_pfn[nid] =min (node_start_pfn[nid], start_pfn); + node_end_pfn[nid] =max (node_end_pfn[nid], end_pfn); + } +return 0 ; +} + +/* temporary shim, will go away soon */ +void __init numa_set_distance (int from,int to,int distance) +{ +/* nada */ +}
Signature · 1 line
1.7.1
PATCH 14/25 x86-32, NUMA: Replace srat_32.c with srat.c
In reply to Tejun HeoSRAT support implementation in srat_32.c and srat.c are generally similar; however, there are some differences.
First of all, 64bit implementation supports more types of SRAT entries. 64bit supports x2apic, affinity, memory and SLIT. 32bit only supports processor and memory.
Most other differences stem from different initialization protocols employed by 64bit and 32bit NUMA init paths.
On 64bit,
* Mappings among PXM, node and apicid are directly done in each SRAT entry callback.
* Memory affinity information is passed to numa_add_memblk() which takes care of all interfacing with NUMA init.
* Doesn't directly initialize NUMA configurations. All the information is recorded in numa_nodes_parsed and memblks.
On 32bit,
* Checks numa_off.
* Things go through one more level of indirection via private tables but eventually end up initializing the same mappings.
* node_start/end_pfn[] are initialized and memblock_x86_register_active_regions() is called for each memory chunk.
* node_set_online() is called for each online node.
* sort_node_map() is called.
There are also other minor differences in sanity checking and messages but taking 64bit version should be good enough.
This patch drops the 32bit specific implementation and makes the 64bit implementation common for both 32 and 64bit.
The init protocol differences are dealt with in two places - the numa_add_memblk() shim added in the previous patch and new temporary numa_32.c:get_memcfg_from_srat() which wraps invocation of x86_acpi_numa_init().
The shim numa_add_memblk() handles the folowings.
* node_start/end_pfn[] initialization.
* node_set_online() for memory nodes.
* Invocation of memblock_x86_register_active_regions().
The shim get_memcfg_from_srat() handles the followings.
* numa_off check.
* node_set_online() for CPU nodes.
* sort_node_map() invocation.
* Clearing of numa_nodes_parsed and active_ranges on failure.
The shims are temporary and will be removed as the generic NUMA init path in 32bit is replaced with 64bit one.
--- arch/x86/include/asm/mmzone_32.h | 2 - arch/x86/include/asm/srat.h | 39 ------ arch/x86/mm/Makefile | 5 +- arch/x86/mm/numa_32.c | 23 +++ arch/x86/mm/srat_32.c | 281 -------------------------------------- 5 files changed, 24 insertions(+), 326 deletions(-) delete mode 100644 arch/x86/include/asm/srat.h delete mode 100644 arch/x86/mm/srat_32.c
arch/x86/include/asm/mmzone_32.h +0 -2
@@ -13,8 +13,6 @@ extern struct pglist_data *node_data[];#define NODE_DATA (nid) (node_data[nid])#include <asm/numaq.h> -/* summit or generic arch */ -#include <asm/srat.h> extern void resume_map_numa_kva (pgd_t *pgd);
arch/x86/include/asm/srat.h deleted +0 -39
@@ -1,39 +0,0 @@ -/* -* Some of the code in this file has been gleaned from the 64 bit -* discontigmem support code base. -* -* Copyright (C) 2002, IBM Corp. -* -* All rights reserved. -* -* This program is free software; you can redistribute it and/or modify -* it under the terms of the GNU General Public License as published by -* the Free Software Foundation; either version 2 of the License, or -* (at your option) any later version. -* -* This program is distributed in the hope that it will be useful, but -* WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or -* NON INFRINGEMENT. See the GNU General Public License for more -* details. -* -* You should have received a copy of the GNU General Public License -* along with this program; if not, write to the Free Software -* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -* -* Send feedback to Pat Gaughen <[email protected]> -*/ - -#ifndef _ASM_X86_SRAT_H -#define _ASM_X86_SRAT_H - -#ifdef CONFIG_ACPI_NUMA -extern int get_memcfg_from_srat (void ); -#else -static inline int get_memcfg_from_srat (void ) -{ -return 0 ; -} -#endif - -#endif /* _ASM_X86_SRAT_H */
arch/x86/mm/Makefile +1 -4
@@ -24,10 +24,7 @@ obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o obj-$(CONFIG_NUMA) += numa.o numa_$(BITS).o obj-$(CONFIG_AMD_NUMA) += amdtopology_64.o -ifeq ($(CONFIG_ACPI_NUMA),y) -obj-$(CONFIG_X86_64) += srat.o -obj-$(CONFIG_X86_32) += srat_32.o -endif +obj-$(CONFIG_ACPI_NUMA) += srat.o obj-$(CONFIG_NUMA_EMU) += numa_emulation.o obj-$(CONFIG_HAVE_MEMBLOCK) += memblock.o
arch/x86/mm/numa_32.c +23 -0
@@ -332,6 +332,29 @@ static __init void init_alloc_remap(int nid) nid, node_pa, node_pa + size, remap_va, remap_va + size); } +static int get_memcfg_from_srat (void ) +{ +#ifdef CONFIG_ACPI_NUMA +int nid; + +if (numa_off) +return 0 ; + +if (x86_acpi_numa_init () <0 ) { +nodes_clear (numa_nodes_parsed); +remove_all_active_ranges (); +return 0 ; + } + +for_each_node_mask (nid, numa_nodes_parsed) +node_set_online (nid); +sort_node_map (); +return 1 ; +#else +return 0 ; +#endif +} +static void get_memcfg_numa (void ) {if (get_memcfg_numaq ())
arch/x86/mm/srat_32.c deleted +0 -281
281 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 15/25 x86-32, NUMA: Update numaq to use new NUMA init protocol
In reply to Tejun HeoUpdate numaq such that it calls numa_add_memblk() and sets numa_nodes_parsed instead of directly diddling with NUMA states. The original get_memcfg_numaq() is renamed to numaq_numa_init() and new get_memcfg_numaq() is created in numa_32.c.
The shim numa_add_memblk() implementation handles node_start/end_pfn[] and node_set_online() for nodes with memory. The new get_memcfg_numaq() exactly the same with get_memcfg_from_srat() other than calling the numaq init function. Things get_memcfgs_numaq() do are not strictly necessary for numaq but added for consistency and to help unifying NUMA init handling.
--- arch/x86/include/asm/numaq.h | 7 +------ arch/x86/kernel/apic/numaq_32.c | 28 ++++++++++------------------ arch/x86/mm/numa_32.c | 23 +++++++++++++++++++++++ 3 files changed, 34 insertions(+), 24 deletions(-)
arch/x86/include/asm/numaq.h +1 -6
@@ -29,7 +29,7 @@#ifdef CONFIG_X86_NUMAQ extern int found_numaq; -extern int get_memcfg_numaq (void ); +extern int numaq_numa_init (void );extern int pci_numaq_init (void );extern void *xquad_portio;
@@ -166,11 +166,6 @@ struct sys_cfg_data {void numaq_tsc_disable (void ); -#else -static inline int get_memcfg_numaq (void ) -{ -return 0 ; -}#endif /* CONFIG_X86_NUMAQ */ #endif /* _ASM_X86_NUMAQ_H */
arch/x86/kernel/apic/numaq_32.c +10 -18
@@ -48,8 +48,6 @@#include <asm/e820.h> #include <asm/ipi.h> -#define MB_TO_PAGES (addr) ((addr) << (20 - PAGE_SHIFT)) -int found_numaq;/*
@@ -79,25 +77,20 @@ int quad_local_to_mp_bus_id[NR_CPUS/4][4];static inline void numaq_register_node (int node,struct sys_cfg_data *scd) {struct eachquadmem *eq = scd->eq + node; +u64 start = (u64)(eq->hi_shrd_mem_start - eq->priv_mem_size) <<20 ; +u64 end = (u64)(eq->hi_shrd_mem_start + eq->hi_shrd_mem_size) <<20 ; +int ret; -node_set_online (node); - -/* Convert to pages */ - node_start_pfn[node] = -MB_TO_PAGES (eq->hi_shrd_mem_start - eq->priv_mem_size); - - node_end_pfn[node] = -MB_TO_PAGES (eq->hi_shrd_mem_start + eq->hi_shrd_mem_size); - -memblock_x86_register_active_regions (node, node_start_pfn[node], - node_end_pfn[node]); +node_set (node, numa_nodes_parsed); + ret =numa_add_memblk (node, start, end); +BUG_ON (ret <0 ); }/* * Function: smp_dump_qct() * * Description: gets memory layout from the quad config table. This -* function also updates node_online_map with the nodes (quads) present. +* function also updates numa_nodes_parsed with the nodes (quads) present. */ static void __init smp_dump_qct (void ) {
@@ -106,7 +99,6 @@ static void __init smp_dump_qct(void) scd = (void *)__va (SYS_CFG_DATA_PRIV_ADDR ); -nodes_clear (node_online_map);for_each_node (node) {if (scd->quads_present31_0 & (1 << node))numaq_register_node (node, scd);
@@ -276,14 +268,14 @@ static __init void early_check_numaq(void) } } -int __init get_memcfg_numaq (void ) +int __init numaq_numa_init (void ) {early_check_numaq ();if (!found_numaq) -return 0 ; +return -ENOENT ;smp_dump_qct (); -return 1 ; +return 0 ; }#define NUMAQ_APIC_DFR_VALUE (APIC_DFR_CLUSTER)
arch/x86/mm/numa_32.c +23 -0
@@ -332,6 +332,29 @@ static __init void init_alloc_remap(int nid) nid, node_pa, node_pa + size, remap_va, remap_va + size); } +static int get_memcfg_numaq (void ) +{ +#ifdef CONFIG_X86_NUMAQ +int nid; + +if (numa_off) +return 0 ; + +if (numaq_numa_init () <0 ) { +nodes_clear (numa_nodes_parsed); +remove_all_active_ranges (); +return 0 ; + } + +for_each_node_mask (nid, numa_nodes_parsed) +node_set_online (nid); +sort_node_map (); +return 1 ; +#else +return 0 ; +#endif +} +static int get_memcfg_from_srat (void ) {#ifdef CONFIG_ACPI_NUMA
Signature · 1 line
1.7.1
PATCH 16/25 x86, NUMA: Move NUMA init logic from numa_64.c to numa.c
In reply to Tejun HeoMove the generic 64bit NUMA init machinery from numa_64.c to numa.c.
* node_data[], numa_mem_info and numa_distance
* numa_add_memblk[_to](), numa_remove_memblk[_from]()
* numa_set_distance() and friends
* numa_init() and all the numa_meminfo handling helpers called from it
* dummy_numa_init()
* memory_add_physaddr_to_nid()
A new function x86_numa_init() is added and the content of numa_64.c::initmem_init() is moved into it. initmem_init() now simply calls x86_numa_init().
Constants and numa_off declaration are moved from numa_{32|64}.h to numa.h.
This is code reorganization and doesn't involve any functional change.
--- arch/x86/include/asm/numa.h | 16 ++ arch/x86/include/asm/numa_32.h | 2 - arch/x86/include/asm/numa_64.h | 19 -- arch/x86/mm/numa.c | 523 +++++++++++++++++++++++++++++++++++++++- arch/x86/mm/numa_64.c | 503 +-------------------------------------- arch/x86/mm/numa_internal.h | 2 + 6 files changed, 539 insertions(+), 526 deletions(-)
arch/x86/include/asm/numa.h +16 -0
@@ -9,6 +9,16 @@#ifdef CONFIG_NUMA #define NR_NODE_MEMBLKS (MAX_NUMNODES*2) +#define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) + +/* +* Too small node sizes may confuse the VM badly. Usually they +* result from BIOS bugs. So dont recognize nodes as standalone +* NUMA entities that have less than this amount of RAM listed: +*/ +#define NODE_MIN_SIZE (4*1024*1024) + +extern int numa_off;/* * __apicid_to_node[] stores the raw mapping between physical apicid and
@@ -68,4 +78,10 @@ static inline void numa_remove_cpu(int cpu) { }struct cpumask __cpuinit *debug_cpumask_set_cpu (int cpu,int enable);#endif +#ifdef CONFIG_NUMA_EMU +#define FAKE_NODE_MIN_SIZE ((u64)32 << 20) +#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL)) +void numa_emu_cmdline (char *); +#endif /* CONFIG_NUMA_EMU */ +#endif /* _ASM_X86_NUMA_H */
arch/x86/include/asm/numa_32.h +0 -2
@@ -1,8 +1,6 @@#ifndef _ASM_X86_NUMA_32_H#define _ASM_X86_NUMA_32_H -extern int numa_off; -#ifdef CONFIG_HIGHMEM extern void set_highmem_pages_init (void );#else
arch/x86/include/asm/numa_64.h +0 -19
@@ -1,25 +1,6 @@#ifndef _ASM_X86_NUMA_64_H#define _ASM_X86_NUMA_64_H -#define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) - -extern int numa_off; -extern unsigned long numa_free_all_bootmem (void ); -#ifdef CONFIG_NUMA -/* -* Too small node sizes may confuse the VM badly. Usually they -* result from BIOS bugs. So dont recognize nodes as standalone -* NUMA entities that have less than this amount of RAM listed: -*/ -#define NODE_MIN_SIZE (4*1024*1024) - -#ifdef CONFIG_NUMA_EMU -#define FAKE_NODE_MIN_SIZE ((u64)32 << 20) -#define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL)) -void numa_emu_cmdline (char *); -#endif /* CONFIG_NUMA_EMU */ -#endif -#endif /* _ASM_X86_NUMA_64_H */
arch/x86/mm/numa.c +520 -3
548 lines not on this page · raw message
arch/x86/mm/numa_64.c +1 -502
518 lines not on this page · raw message
arch/x86/mm/numa_internal.h +2 -0
@@ -19,6 +19,8 @@ void __init numa_remove_memblk_from(int idx, struct numa_meminfo *mi);int __init numa_cleanup_meminfo (struct numa_meminfo *mi);void __init numa_reset_distance (void ); +void __init x86_numa_init (void ); +#ifdef CONFIG_NUMA_EMU void __init numa_emulation (struct numa_meminfo *numa_meminfo,int numa_dist_cnt);
Signature · 1 line
1.7.1
PATCH 17/25 x86, NUMA: Enable build of generic NUMA init code on 32bit
In reply to Tejun HeoGeneric NUMA init code was moved to numa.c from numa_64.c but is still guaraded by CONFIG_X86_64. This patch removes the compile guard and enables compiling on 32bit.
* numa_add_memblk() and numa_set_distance() clash with the shim implementation in numa_32.c and are left out.
* memory_add_physaddr_to_nid() clashes with 32bit implementation and is left out.
* MAX_DMA_PFN definition in dma.h moved out of !CONFIG_X86_32.
* node_data definition in numa_32.c removed in favor of the one in numa.c.
There are places where ulong is assumed to be 64bit. The next patch will fix them up. Note that although the code is compiled it isn't used yet and this patch doesn't cause any functional change.
--- arch/x86/include/asm/dma.h | 6 +++--- arch/x86/mm/numa.c | 10 ++++------ arch/x86/mm/numa_32.c | 3 --- 3 files changed, 7 insertions(+), 12 deletions(-)
arch/x86/include/asm/dma.h +3 -3
@@ -69,6 +69,9 @@#define MAX_DMA_CHANNELS 8 +/* 16MB ISA DMA zone */ +#define MAX_DMA_PFN ((16 * 1024 * 1024) >> PAGE_SHIFT) +#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */
@@ -76,9 +79,6 @@#else -/* 16MB ISA DMA zone */ -#define MAX_DMA_PFN ((16 * 1024 * 1024) >> PAGE_SHIFT) -/* 4GB broken PCI/AGP hardware bus master zone */ #define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT)
arch/x86/mm/numa.c +4 -6
@@ -23,7 +23,6 @@int __initdata numa_off;nodemask_t numa_nodes_parsed__initdata ; -#ifdef CONFIG_X86_64 struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ;EXPORT_SYMBOL (node_data);
@@ -35,7 +34,6 @@ __initdatastatic int numa_distance_cnt;static u8 *numa_distance; -#endif static __init int numa_setup (char *opt) {
@@ -134,7 +132,6 @@ void __init setup_node_to_cpumask_map(void)pr_debug ("Node to cpumask map for %d nodes\n" , nr_node_ids); } -#ifdef CONFIG_X86_64 static int __init numa_add_memblk_to (int nid,u64 start,u64 end,struct numa_meminfo *mi) {
@@ -176,6 +173,7 @@ void __init numa_remove_memblk_from(int idx, struct numa_meminfo *mi) (mi->nr_blks - idx) * sizeof(mi->blk[0])); } +#ifdef CONFIG_X86_64 /** * numa_add_memblk - Add one numa_memblk to numa_meminfo * @nid: NUMA node ID of the new memblk
@@ -191,6 +189,7 @@ int __init numa_add_memblk(int nid, u64 start, u64 end) {return numa_add_memblk_to (nid, start, end, &numa_meminfo); } +#endif /* Initialize bootmem allocator for a node */ static void __init
@@ -402,6 +401,7 @@ static int __init numa_alloc_distance(void)return 0 ; } +#ifdef CONFIG_X86_64 /** * numa_set_distance - Set NUMA distance from one NUMA to another * @from: the 'from' node to set distance
@@ -440,6 +440,7 @@ void __init numa_set_distance(int from, int to, int distance) numa_distance[from * numa_distance_cnt + to] = distance; } +#endif int __node_distance (int from,int to) {
@@ -518,7 +519,6 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)return 0 ; } -#endif /* * There are unfortunately some poorly designed mainboards around that
@@ -542,7 +542,6 @@ void __init numa_init_array(void) } } -#ifdef CONFIG_X86_64 static int __init numa_init (int (*init_func)(void )) {int i;
@@ -627,7 +626,6 @@ void __init x86_numa_init(void)numa_init (dummy_numa_init); } -#endif static __init int find_near_online_node (int node) {
arch/x86/mm/numa_32.c +0 -3
@@ -41,9 +41,6 @@#include <asm/bios_ebda.h> #include <asm/proto.h> -struct pglist_data *node_data[MAX_NUMNODES ]__read_mostly ; -EXPORT_SYMBOL (node_data); -/* * numa interface - we expect the numa architecture specific code to have * populated the following initialisation.
Signature · 1 line
1.7.1
PATCH 18/25 x86, NUMA: Remove long 64bit assumption from numa.c
In reply to Tejun HeoCode moved from numa_64.c has assumption that long is 64bit in several places. This patch removes the assumption by using {s|u}64_t explicity, using PFN_PHYS() for page number -> addr conversions and adjusting printf formats.
--- arch/x86/mm/numa.c | 45 ++++++++++++++++++++++----------------------- 1 files changed, 22 insertions(+), 23 deletions(-)
arch/x86/mm/numa.c +22 -23
@@ -192,13 +192,12 @@ int __init numa_add_memblk(int nid, u64 start, u64 end)#endif /* Initialize bootmem allocator for a node */ -static void __init -setup_node_bootmem (int nid,unsigned long start,unsigned long end) +static void __init setup_node_bootmem (int nid,u64 start,u64 end) { -const u64 nd_low = (u64 )MAX_DMA_PFN <<PAGE_SHIFT ; -const u64 nd_high = (u64 )max_pfn_mapped <<PAGE_SHIFT ; +const u64 nd_low =PFN_PHYS (MAX_DMA_PFN ); +const u64 nd_high =PFN_PHYS (max_pfn_mapped);const size_t nd_size =roundup (sizeof (pg_data_t),PAGE_SIZE ); -unsigned long nd_pa; +u64 nd_pa;int tnid;/*
@@ -210,7 +209,7 @@ setup_node_bootmem(int nid, unsigned long start, unsigned long end) start =roundup (start,ZONE_ALIGN ); -printk (KERN_INFO "Initmem setup node %d %016lx-%016lx\n" , +printk (KERN_INFO "Initmem setup node %d %016Lx-%016Lx\n" , nid, start, end);/*
@@ -223,13 +222,13 @@ setup_node_bootmem(int nid, unsigned long start, unsigned long end) nd_pa =memblock_find_in_range (nd_low, nd_high, nd_size,SMP_CACHE_BYTES );if (nd_pa ==MEMBLOCK_ERROR ) { -pr_err ("Cannot find %lu bytes in node %d\n" , nd_size, nid); +pr_err ("Cannot find %zu bytes in node %d\n" , nd_size, nid);return ; }memblock_x86_reserve_range (nd_pa, nd_pa + nd_size,"NODE_DATA" );/* report and initialize */ -printk (KERN_INFO " NODE_DATA [%016lx - %016lx]\n" , +printk (KERN_INFO " NODE_DATA [%016Lx - %016Lx]\n" , nd_pa, nd_pa + nd_size -1 ); tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT );if (tnid != nid)
@@ -257,7 +256,7 @@ setup_node_bootmem(int nid, unsigned long start, unsigned long end)int __init numa_cleanup_meminfo (struct numa_meminfo *mi) {const u64 low =0 ; -const u64 high = (u64 )max_pfn <<PAGE_SHIFT ; +const u64 high =PFN_PHYS (max_pfn);int i, j, k;for (i =0 ; i < mi->nr_blks; i++) {
@@ -275,7 +274,7 @@ int __init numa_cleanup_meminfo(struct numa_meminfo *mi)for (j = i +1 ; j < mi->nr_blks; j++) {struct numa_memblk *bj = &mi->blk[j]; -unsigned long start, end; +u64 start, end;/* * See whether there are overlapping blocks. Whine
@@ -313,7 +312,7 @@ int __init numa_cleanup_meminfo(struct numa_meminfo *mi) }if (k < mi->nr_blks)continue ; -printk (KERN_INFO "NUMA: Node %d [%Lx,%Lx) + [%Lx,%Lx) -> [%lx,%lx)\n" , +printk (KERN_INFO "NUMA: Node %d [%Lx,%Lx) + [%Lx,%Lx) -> [%Lx,%Lx)\n" , bi->nid, bi->start, bi->end, bj->start, bj->end, start, end); bi->start = start;
@@ -378,7 +377,7 @@ static int __init numa_alloc_distance(void) cnt++; size = cnt * cnt *sizeof (numa_distance[0 ]); - phys =memblock_find_in_range (0 , (u64 )max_pfn_mapped <<PAGE_SHIFT , + phys =memblock_find_in_range (0 ,PFN_PHYS (max_pfn_mapped), size,PAGE_SIZE );if (phys ==MEMBLOCK_ERROR ) {pr_warning ("NUMA: Warning: can't allocate distance table!\n" );
@@ -456,24 +455,24 @@ EXPORT_SYMBOL(__node_distance);*/ static bool __init numa_meminfo_cover_memory (const struct numa_meminfo *mi) { -unsigned long numaram, e820ram; +u64 numaram, e820ram;int i; numaram =0 ;for (i =0 ; i < mi->nr_blks; i++) { -unsigned long s = mi->blk[i].start >>PAGE_SHIFT ; -unsigned long e = mi->blk[i].end >>PAGE_SHIFT ; +u64 s = mi->blk[i].start >>PAGE_SHIFT ; +u64 e = mi->blk[i].end >>PAGE_SHIFT ; numaram += e - s; numaram -=__absent_pages_in_range (mi->blk[i].nid, s, e); -if ((long )numaram <0 ) +if ((s64 )numaram <0 ) numaram =0 ; } e820ram = max_pfn - (memblock_x86_hole_size (0 , - max_pfn <<PAGE_SHIFT ) >>PAGE_SHIFT ); +PFN_PHYS (max_pfn)) >>PAGE_SHIFT );/* We seem to lose 3 pages somewhere. Allow 1M of slack. */ -if ((long )(e820ram - numaram) >= (1 << (20 -PAGE_SHIFT ))) { -printk (KERN_ERR "NUMA: nodes only cover %luMB of your %luMB e820 RAM. Not used.\n" , +if ((s64)(e820ram - numaram) >= (1 << (20 -PAGE_SHIFT ))) { +printk (KERN_ERR "NUMA: nodes only cover %LuMB of your %LuMB e820 RAM. Not used.\n" , (numaram <<PAGE_SHIFT ) >>20 , (e820ram <<PAGE_SHIFT ) >>20 );return false;
@@ -503,7 +502,7 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)/* Finally register nodes. */ for_each_node_mask (nid, node_possible_map) { -u64 start = (u64 )max_pfn <<PAGE_SHIFT ; +u64 start =PFN_PHYS (max_pfn);u64 end =0 ;for (i =0 ; i < mi->nr_blks; i++) {
@@ -595,11 +594,11 @@ static int __init dummy_numa_init(void) {printk (KERN_INFO "%s\n" , numa_off ?"NUMA turned off" :"No NUMA configuration found" ); -printk (KERN_INFO "Faking a node at %016lx-%016lx\n" , -0LU , max_pfn <<PAGE_SHIFT ); +printk (KERN_INFO "Faking a node at %016Lx-%016Lx\n" , +0LLU ,PFN_PHYS (max_pfn));node_set (0 , numa_nodes_parsed); -numa_add_memblk (0 ,0 , (u64 )max_pfn <<PAGE_SHIFT ); +numa_add_memblk (0 ,0 ,PFN_PHYS (max_pfn));return 0 ; }
Signature · 1 line
1.7.1
PATCH 19/25 x86-32, NUMA: Add @start and @end to init_alloc_remap()
In reply to Tejun HeoInstead of dereferencing node_start/end_pfn[] directly, make init_alloc_remap() take @start and @end and let the caller be responsible for making sure the range is sane. This is to prepare for use from unified NUMA init code.
--- arch/x86/mm/numa_32.c | 29 ++++++++++++++--------------- 1 files changed, 14 insertions(+), 15 deletions(-)
arch/x86/mm/numa_32.c +14 -15
@@ -265,8 +265,10 @@ void resume_map_numa_kva(pgd_t *pgd_base)* opportunistically and the callers will fall back to other memory * allocation mechanisms on failure. */ -static __init void init_alloc_remap (int nid) +static __init void init_alloc_remap (int nid,u64 start,u64 end) { +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long end_pfn = end >>PAGE_SHIFT ;unsigned long size, pfn;u64 node_pa, remap_pa;void *remap_va;
@@ -276,24 +278,15 @@ static __init void init_alloc_remap(int nid)* memory could be added but not currently present. */ printk (KERN_DEBUG "node %d pfn: [%lx - %lx]\n" , - nid, node_start_pfn[nid], node_end_pfn[nid]); -if (node_start_pfn[nid] > max_pfn) -return ; -if (!node_end_pfn[nid]) -return ; -if (node_end_pfn[nid] > max_pfn) - node_end_pfn[nid] = max_pfn; + nid, start_pfn, end_pfn);/* calculate the necessary space aligned to large page size */ - size =node_memmap_size_bytes (nid, node_start_pfn[nid], -min (node_end_pfn[nid], max_pfn)); + size =node_memmap_size_bytes (nid, start_pfn, end_pfn); size +=ALIGN (sizeof (pg_data_t),PAGE_SIZE ); size =ALIGN (size,LARGE_PAGE_BYTES );/* allocate node memory and the lowmem remap area */ - node_pa =memblock_find_in_range (node_start_pfn[nid] <<PAGE_SHIFT , - (u64 )node_end_pfn[nid] <<PAGE_SHIFT , - size,LARGE_PAGE_BYTES ); + node_pa =memblock_find_in_range (start, end, size,LARGE_PAGE_BYTES );if (node_pa ==MEMBLOCK_ERROR ) {pr_warning ("remap_alloc: failed to allocate %lu bytes for node %d\n" , size, nid);
@@ -391,8 +384,14 @@ void __init initmem_init(void)get_memcfg_numa ();numa_init_array (); -for_each_online_node (nid) -init_alloc_remap (nid); +for_each_online_node (nid) { +u64 start = (u64 )node_start_pfn[nid] <<PAGE_SHIFT ; +u64 end =min ((u64 )node_end_pfn[nid] <<PAGE_SHIFT , + (u64 )max_pfn <<PAGE_SHIFT ); + +if (start < end) +init_alloc_remap (nid, start, end); + }#ifdef CONFIG_HIGHMEM highstart_pfn = highend_pfn = max_pfn;
Signature · 1 line
1.7.1
PATCH 20/25 x86, NUMA: Initialize and use remap allocator from setup_node_bootmem()
In reply to Tejun Heosetup_node_bootmem() is taken from 64bit and doesn't use remap allocator. It's about to be shared with 32bit so add support for it. If NODE_DATA is remapped, it's noted in the debug message and node locality check is skipped as the __pa() of the remapped address doesn't reflect the actual physical address.
On 64bit, remap allocator becomes noop and doesn't affect the behavior.
--- arch/x86/mm/numa.c | 41 +++++++++++++++++++++++++++-------------- arch/x86/mm/numa_32.c | 2 +- arch/x86/mm/numa_internal.h | 6 ++++++ 3 files changed, 34 insertions(+), 15 deletions(-)
arch/x86/mm/numa.c +27 -14
@@ -197,7 +197,9 @@ static void __init setup_node_bootmem(int nid, u64 start, u64 end)const u64 nd_low =PFN_PHYS (MAX_DMA_PFN );const u64 nd_high =PFN_PHYS (max_pfn_mapped);const size_t nd_size =roundup (sizeof (pg_data_t),PAGE_SIZE ); +bool remapped = false;u64 nd_pa; +void *nd;int tnid;/*
@@ -207,34 +209,45 @@ static void __init setup_node_bootmem(int nid, u64 start, u64 end)if (end && (end - start) <NODE_MIN_SIZE )return ; +/* initialize remap allocator before aligning to ZONE_ALIGN */ +init_alloc_remap (nid, start, end); + start =roundup (start,ZONE_ALIGN );printk (KERN_INFO "Initmem setup node %d %016Lx-%016Lx\n" , nid, start, end);/* -* Try to allocate node data on local node and then fall back to -* all nodes. Never allocate in DMA zone. +* Allocate node data. Try remap allocator first, node-local +* memory and then any node. Never allocate in DMA zone. */ - nd_pa =memblock_x86_find_in_range_node (nid, nd_low, nd_high, + nd =alloc_remap (nid, nd_size); +if (nd) { + nd_pa =__pa (nd); + remapped = true; + }else { + nd_pa =memblock_x86_find_in_range_node (nid, nd_low, nd_high, nd_size,SMP_CACHE_BYTES ); -if (nd_pa ==MEMBLOCK_ERROR ) - nd_pa =memblock_find_in_range (nd_low, nd_high, - nd_size,SMP_CACHE_BYTES ); -if (nd_pa ==MEMBLOCK_ERROR ) { -pr_err ("Cannot find %zu bytes in node %d\n" , nd_size, nid); -return ; +if (nd_pa ==MEMBLOCK_ERROR ) + nd_pa =memblock_find_in_range (nd_low, nd_high, + nd_size,SMP_CACHE_BYTES ); +if (nd_pa ==MEMBLOCK_ERROR ) { +pr_err ("Cannot find %zu bytes in node %d\n" , + nd_size, nid); +return ; + } +memblock_x86_reserve_range (nd_pa, nd_pa + nd_size,"NODE_DATA" ); + nd =__va (nd_pa); } -memblock_x86_reserve_range (nd_pa, nd_pa + nd_size,"NODE_DATA" );/* report and initialize */ -printk (KERN_INFO " NODE_DATA [%016Lx - %016Lx]\n" , - nd_pa, nd_pa + nd_size -1 ); +printk (KERN_INFO " NODE_DATA [%016Lx - %016Lx]%s\n" , + nd_pa, nd_pa + nd_size -1 , remapped ?" (remapped)" :"" ); tnid =early_pfn_to_nid (nd_pa >>PAGE_SHIFT ); -if (tnid != nid) +if (!remapped && tnid != nid)printk (KERN_INFO " NODE_DATA(%d) on node %d\n" , nid, tnid); - node_data[nid] =__va (nd_pa); + node_data[nid] = nd;memset (NODE_DATA (nid),0 ,sizeof (pg_data_t));NODE_DATA (nid)->node_id = nid;NODE_DATA (nid)->node_start_pfn = start >>PAGE_SHIFT ;
arch/x86/mm/numa_32.c +1 -1
@@ -265,7 +265,7 @@ void resume_map_numa_kva(pgd_t *pgd_base)* opportunistically and the callers will fall back to other memory * allocation mechanisms on failure. */ -static __init void init_alloc_remap (int nid,u64 start,u64 end) +void __init init_alloc_remap (int nid,u64 start,u64 end) {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long end_pfn = end >>PAGE_SHIFT ;
arch/x86/mm/numa_internal.h +6 -0
@@ -21,6 +21,12 @@ void __init numa_reset_distance(void);void __init x86_numa_init (void ); +#ifdef CONFIG_X86_64 +static inline void init_alloc_remap (int nid,u64 start,u64 end) { } +#else +void __init init_alloc_remap (int nid,u64 start,u64 end); +#endif +#ifdef CONFIG_NUMA_EMU void __init numa_emulation (struct numa_meminfo *numa_meminfo,int numa_dist_cnt);
Signature · 1 line
1.7.1
PATCH 21/25 x86, NUMA: Make 32bit use common NUMA init path
In reply to Tejun HeoWith both _numa_init() methods converted and the rest of init code adjusted, numa_32.c now can switch from the 32bit only init code to the common one in numa.c.
* Shim get_memcfg_*()'s are dropped and initmem_init() calls x86_numa_init(), which is updated to handle NUMAQ.
* All boilerplate operations including node range limiting, pgdat alloc/init are handled by numa_init(). 32bit only implementation is removed.
* 32bit numa_add_memblk(), numa_set_distance() and memory_add_physaddr_to_nid() removed and common versions in numa_32.c enabled for 32bit.
This change causes the following behavior changes.
* NODE_DATA()->node_start_pfn/node_spanned_pages properly initialized for 32bit too.
* Much more sanity checks and configuration cleanups.
* Proper handling of node distances.
* The same NUMA init messages as 64bit.
--- arch/x86/include/asm/topology.h | 7 - arch/x86/mm/numa.c | 10 +- arch/x86/mm/numa_32.c | 232 +-------------------------------------- 3 files changed, 7 insertions(+), 242 deletions(-)
arch/x86/include/asm/topology.h +0 -7
@@ -93,18 +93,11 @@ extern void setup_node_to_cpumask_map(void);#define pcibus_to_node (bus) __pcibus_to_node(bus)#ifdef CONFIG_X86_32 -extern unsigned long node_start_pfn[]; -extern unsigned long node_end_pfn[]; -#define node_has_online_mem (nid) (node_start_pfn[nid] != node_end_pfn[nid]) -# define SD_CACHE_NICE_TRIES 1# define SD_IDLE_IDX 1 -#else -# define SD_CACHE_NICE_TRIES 2# define SD_IDLE_IDX 2 -#endif /* sched_domains SD_NODE_INIT for NUMA machines */
arch/x86/mm/numa.c +5 -5
@@ -173,7 +173,6 @@ void __init numa_remove_memblk_from(int idx, struct numa_meminfo *mi) (mi->nr_blks - idx) * sizeof(mi->blk[0])); } -#ifdef CONFIG_X86_64 /** * numa_add_memblk - Add one numa_memblk to numa_meminfo * @nid: NUMA node ID of the new memblk
@@ -189,7 +188,6 @@ int __init numa_add_memblk(int nid, u64 start, u64 end) {return numa_add_memblk_to (nid, start, end, &numa_meminfo); } -#endif /* Initialize bootmem allocator for a node */ static void __init setup_node_bootmem (int nid,u64 start,u64 end)
@@ -413,7 +411,6 @@ static int __init numa_alloc_distance(void)return 0 ; } -#ifdef CONFIG_X86_64 /** * numa_set_distance - Set NUMA distance from one NUMA to another * @from: the 'from' node to set distance
@@ -452,7 +449,6 @@ void __init numa_set_distance(int from, int to, int distance) numa_distance[from * numa_distance_cnt + to] = distance; } -#endif int __node_distance (int from,int to) {
@@ -626,6 +622,10 @@ static int __init dummy_numa_init(void)void __init x86_numa_init (void ) {if (!numa_off) { +#ifdef CONFIG_X86_NUMAQ +if (!numa_init (numaq_numa_init)) +return ; +#endif #ifdef CONFIG_ACPI_NUMA if (!numa_init (x86_acpi_numa_init))return ;
@@ -810,7 +810,7 @@ EXPORT_SYMBOL(cpumask_of_node);#endif /* !CONFIG_DEBUG_PER_CPU_MAPS */ -#if defined(CONFIG_X86_64 ) && defined(CONFIG_MEMORY_HOTPLUG ) +#ifdef CONFIG_MEMORY_HOTPLUG int memory_add_physaddr_to_nid (u64 start) {struct numa_meminfo *mi = &numa_meminfo;
arch/x86/mm/numa_32.c +2 -230
270 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 22/25 x86, NUMA: Make numa_init_array() static
In reply to Tejun Heonuma_init_array() no longer has users outside of numa.c. Make it static.
--- arch/x86/include/asm/numa.h | 2 -- arch/x86/mm/numa.c | 2 +- 2 files changed, 1 insertions(+), 3 deletions(-)
arch/x86/include/asm/numa.h +0 -2
@@ -61,14 +61,12 @@ static inline int numa_cpu_node(int cpu)#ifdef CONFIG_NUMA extern void __cpuinit numa_set_node (int cpu,int node);extern void __cpuinit numa_clear_node (int cpu); -extern void __init numa_init_array (void );extern void __init init_cpu_to_node (void );extern void __cpuinit numa_add_cpu (int cpu);extern void __cpuinit numa_remove_cpu (int cpu);#else /* CONFIG_NUMA */ static inline void numa_set_node (int cpu,int node) { }static inline void numa_clear_node (int cpu) { } -static inline void numa_init_array (void ) { }static inline void init_cpu_to_node (void ) { }static inline void numa_add_cpu (int cpu) { }static inline void numa_remove_cpu (int cpu) { }
arch/x86/mm/numa.c +1 -1
@@ -535,7 +535,7 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)* as the number of CPUs is not known yet. We round robin the existing * nodes. */ -void __init numa_init_array (void ) +static void __init numa_init_array (void ) {int rr, i;
Signature · 1 line
1.7.1
PATCH 23/25 x86, NUMA: Rename amdtopology_64.c to amdtopology.c
In reply to Tejun Heoamdtopology is going to be used by 32bit too drop _64 suffix. This is pure rename.
--- arch/x86/mm/Makefile | 2 +- arch/x86/mm/amdtopology.c | 196 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/mm/amdtopology_64.c | 196 ------------------------------------------ 3 files changed, 197 insertions(+), 197 deletions(-) create mode 100644 arch/x86/mm/amdtopology.c delete mode 100644 arch/x86/mm/amdtopology_64.c
arch/x86/mm/Makefile +1 -1
@@ -23,7 +23,7 @@ mmiotrace-y := kmmio.o pf_in.o mmio-mod.o obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o obj-$(CONFIG_NUMA) += numa.o numa_$(BITS).o -obj-$(CONFIG_AMD_NUMA) += amdtopology_64.o +obj-$(CONFIG_AMD_NUMA) += amdtopology.o obj-$(CONFIG_ACPI_NUMA) += srat.o obj-$(CONFIG_NUMA_EMU) += numa_emulation.o
arch/x86/mm/amdtopology.c new file +196 -0
196 lines not on this page · raw message
arch/x86/mm/amdtopology_64.c deleted +0 -196
196 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 24/25 x86, NUMA: Enable CONFIG_AMD_NUMA on 32bit too
In reply to Tejun HeoNow that NUMA init path is unified, amdtopology can be enabled on 32bit. Make amdtopology.c safe on 32bit by explicitly using u64 and drop X86_64 dependency from Kconfig.
Inclusion of bootmem.h is added for max_pfn declaration.
--- arch/x86/Kconfig | 2 +- arch/x86/mm/amdtopology.c | 21 +++++++++++---------- 2 files changed, 12 insertions(+), 11 deletions(-)
arch/x86/Kconfig +1 -1
@@ -1174,7 +1174,7 @@ comment "NUMA (Summit) requires SMP, 64GB highmem support, ACPI" config AMD_NUMA def_bool y prompt "Old style AMD Opteron NUMA detection" - depends on X86_64 && NUMA && PCI + depends on NUMA && PCI ---help--- Enable AMD NUMA node topology detection. You should say Y here if you have a multi processor AMD system. This uses an old method to
arch/x86/mm/amdtopology.c +11 -10
@@ -12,6 +12,7 @@#include <linux/module.h> #include <linux/nodemask.h> #include <linux/memblock.h> +#include <linux/bootmem.h> #include <asm/io.h> #include <linux/pci_ids.h>
@@ -69,10 +70,10 @@ static __init void early_get_boot_cpu_id(void)int __init amd_numa_init (void ) { -unsigned long start =PFN_PHYS (0 ); -unsigned long end =PFN_PHYS (max_pfn); +u64 start =PFN_PHYS (0 ); +u64 end =PFN_PHYS (max_pfn);unsigned numnodes; -unsigned long prevbase; +u64 prevbase;int i, j, nb;u32 nodeid, reg;unsigned int bits, cores, apicid_base;
@@ -95,7 +96,7 @@ int __init amd_numa_init(void) prevbase =0 ;for (i =0 ; i <8 ; i++) { -unsigned long base, limit; +u64 base, limit; base =read_pci_config (0 , nb,1 ,0x40 + i*8 ); limit =read_pci_config (0 , nb,1 ,0x44 + i*8 );
@@ -107,18 +108,18 @@ int __init amd_numa_init(void)continue ; }if (nodeid >= numnodes) { -pr_info ("Ignoring excess node %d (%lx:%lx)\n" , nodeid, +pr_info ("Ignoring excess node %d (%Lx:%Lx)\n" , nodeid, base, limit);continue ; }if (!limit) { -pr_info ("Skipping node entry %d (base %lx)\n" , +pr_info ("Skipping node entry %d (base %Lx)\n" , i, base);continue ; }if ((base >>8 ) &3 || (limit >>8 ) &3 ) { -pr_err ("Node %d using interleaving mode %lx/%lx\n" , +pr_err ("Node %d using interleaving mode %Lx/%Lx\n" , nodeid, (base >>8 ) &3 , (limit >>8 ) &3 );return -EINVAL ; }
@@ -150,19 +151,19 @@ int __init amd_numa_init(void)continue ; }if (limit < base) { -pr_err ("Node %d bogus settings %lx-%lx.\n" , +pr_err ("Node %d bogus settings %Lx-%Lx.\n" , nodeid, base, limit);continue ; }/* Could sort here, but pun for now. Should not happen anyroads. */ if (prevbase > base) { -pr_err ("Node map not sorted %lx,%lx\n" , +pr_err ("Node map not sorted %Lx,%Lx\n" , prevbase, base);return -EINVAL ; } -pr_info ("Node %d MemBase %016lx Limit %016lx\n" , +pr_info ("Node %d MemBase %016Lx Limit %016Lx\n" , nodeid, base, limit); prevbase = base;
Signature · 1 line
1.7.1
PATCH 25/25 x86, NUMA: Enable emulation on 32bit too
In reply to Tejun HeoNow that NUMA init path is unified, NUMA emulation can be enabled on 32bit. Make numa_emluation.c safe on 32bit by doing the followings.
* Define MAX_DMA32_PFN on 32bit too.
* Include bootmem.h for max_pfn declaration.
* Use u64 explicitly and always use PFN_PHYS() when converting page number to address.
* Avoid __udivdi3() generation on 32bit by doing number of pages calculation instead in split_nodes_interleave().
And drop X86_64 dependency from Kconfig.
--- arch/x86/Kconfig | 2 +- arch/x86/include/asm/dma.h | 10 +++------- arch/x86/mm/numa_emulation.c | 16 +++++++++++----- 3 files changed, 15 insertions(+), 13 deletions(-)
arch/x86/Kconfig +1 -1
@@ -1201,7 +1201,7 @@ config NODES_SPAN_OTHER_NODES config NUMA_EMU bool "NUMA emulation" - depends on X86_64 && NUMA + depends on NUMA ---help--- Enable NUMA emulation. A flat machine will be split into virtual nodes when booted with "numa=fake=N", where N is the
arch/x86/include/asm/dma.h +3 -7
@@ -72,19 +72,15 @@/* 16MB ISA DMA zone */ #define MAX_DMA_PFN ((16 * 1024 * 1024) >> PAGE_SHIFT) -#ifdef CONFIG_X86_32 +/* 4GB broken PCI/AGP hardware bus master zone */ +#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) +#ifdef CONFIG_X86_32 /* The maximum address that we can perform a DMA transfer to on this platform */ #define MAX_DMA_ADDRESS (PAGE_OFFSET + 0x1000000) -#else - -/* 4GB broken PCI/AGP hardware bus master zone */ -#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT) -/* Compat define for old dma zone */ #define MAX_DMA_ADDRESS ((unsigned long)__va(MAX_DMA_PFN << PAGE_SHIFT)) -#endif /* 8237 DMA controllers */
arch/x86/mm/numa_emulation.c +11 -5
@@ -5,6 +5,7 @@#include <linux/errno.h> #include <linux/topology.h> #include <linux/memblock.h> +#include <linux/bootmem.h> #include <asm/dma.h> #include "numa_internal.h"
@@ -84,7 +85,13 @@ static int __init split_nodes_interleave(struct numa_meminfo *ei, nr_nodes =MAX_NUMNODES ; } - size = (max_addr - addr -memblock_x86_hole_size (addr, max_addr)) / nr_nodes; +/* +* Calculate target node size. x86_32 freaks on __udivdi3() so do +* the division in ulong number of pages and convert back. +*/ + size = max_addr - addr -memblock_x86_hole_size (addr, max_addr); + size =PFN_PHYS ((unsigned long )(size >>PAGE_SHIFT ) / nr_nodes); +/* * Calculate the number of big nodes that can be allocated as a result * of consolidating the remainder.
@@ -226,7 +233,7 @@ static int __init split_nodes_size_interleave(struct numa_meminfo *ei,*/ while (nodes_weight (physnode_mask)) {for_each_node_mask (i, physnode_mask) { -u64 dma32_end =MAX_DMA32_PFN <<PAGE_SHIFT ; +u64 dma32_end =PFN_PHYS (MAX_DMA32_PFN );u64 start, limit, end;int phys_blk;
@@ -298,7 +305,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt) {static struct numa_meminfo ei__initdata ;static struct numa_meminfo pi__initdata ; -const u64 max_addr = max_pfn <<PAGE_SHIFT ; +const u64 max_addr =PFN_PHYS (max_pfn);u8 *phys_dist =NULL ;size_t phys_size = numa_dist_cnt * numa_dist_cnt *sizeof (phys_dist[0 ]);int max_emu_nid, dfl_phys_nid;
@@ -342,8 +349,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)if (numa_dist_cnt) {u64 phys; - phys =memblock_find_in_range (0 , - (u64 )max_pfn_mapped <<PAGE_SHIFT , + phys =memblock_find_in_range (0 ,PFN_PHYS (max_pfn_mapped), phys_size,PAGE_SIZE );if (phys ==MEMBLOCK_ERROR ) {pr_warning ("NUMA: Warning: can't allocate copy of distance table, disabling emulation\n" );
Signature · 1 line
1.7.1
Quoted 3 lines · Tejun Heo
diffstat follows. 610 lines removed and 32bit NUMA got much better! :)
32 files changed, 1017 insertions(+), 1627 deletions(-)
impressive! :-)
Thanks,
Ingo
Quoted 6 lines · Tejun Heo
Hello,
This patchset, finally, unifies 32 and 64bit NUMA initialization. It gradually moves 64bit stuff to common code and replaces 32bit code with it. Once the unification is complete, amdtopology and emulation are enabled for 32bit too (there's no reason not to).
got:
SRAT: Node 0 PXM 0 0-a0000
SRAT: Node 0 PXM 0 100000-80000000
SRAT: Node 0 PXM 0 100000000-880000000
SRAT: Node 1 PXM 1 880000000-1080000000
SRAT: Node 2 PXM 2 1080000000-1880000000
SRAT: Node 3 PXM 3 1880000000-2080000000
SRAT: Node 4 PXM 4 2080000000-2880000000
SRAT: Node 5 PXM 5 2880000000-3080000000
SRAT: Node 6 PXM 6 3080000000-3880000000
SRAT: Node 7 PXM 7 3880000000-4080000000
NUMA: Initialized distance table, cnt=8
NUMA: Node 0 [0,a0000) + [100000,80000000) -> [0,80000000)
NUMA: Node 0 [0,80000000) + [100000000,880000000) -> [0,880000000) Adding active range (0, 0x10, 0x95) 0 entries of 3200 used Adding active range (0, 0x100, 0x7f750) 1 entries of 3200 used Adding active range (0, 0x100000, 0x880000) 2 entries of 3200 used Adding active range (1, 0x880000, 0x1000000) 3 entries of 3200 used Adding active range (2, 0x1080000, 0x1000000) 4 entries of 3200 used
------------[ cut here ]------------ WARNING: at mm/sparse.c:170 mminit_validate_memmodel_limits+0x29/0x69() Hardware name: Sun Fire X4800 M2 Modules linked in: Pid: 0, comm: swapper Not tainted 2.6.39-rc5-tip-03938-gda7ba4e-dirty #896 Call Trace: [<40261c57>] warn_slowpath_common+0x65/0x7a [<40e30020>] ? mminit_validate_memmodel_limits+0x29/0x69 [<40261c7b>] warn_slowpath_null+0xf/0x13 [<40e30020>] mminit_validate_memmodel_limits+0x29/0x69 [<40e10a2e>] add_active_range+0x34/0xdb [<40e0c281>] memblock_x86_register_active_regions+0x92/0xb3 [<40e0a8c1>] numa_init+0xf1/0x5e9 [<408cad47>] ? printk+0xf/0x11 [<40e0af98>] x86_numa_init+0x16/0x34 [<40e0b2dd>] initmem_init+0x8/0xbc [<40dfc3ce>] setup_arch+0xa27/0xad7 [<40df85cb>] start_kernel+0x71/0x2ed [<40df80c4>] i386_start_kernel+0xc4/0xcb ---[ end trace 4eaa2a86a8e2da22 ]---
Adding active range (3, 0x1880000, 0x1000000) 5 entries of 3200 used Adding active range (4, 0x2080000, 0x1000000) 6 entries of 3200 used Adding active range (5, 0x2880000, 0x1000000) 7 entries of 3200 used Adding active range (6, 0x3080000, 0x1000000) 8 entries of 3200 used Adding active range (7, 0x3880000, 0x1000000) 9 entries of 3200 used
NUMA: nodes only cover 0MB of your 63478MB e820 RAM. Not used. No NUMA configuration found
Faking a node at 0000000000000000-0000001000000000
Adding active range (0, 0x10, 0x95) 0 entries of 3200 used Adding active range (0, 0x100, 0x7f750) 1 entries of 3200 used Adding active range (0, 0x100000, 0x1000000) 2 entries of 3200 used node 0 pfn: [0 - 1000000]remap_alloc: node 0 [fffe00000-1000000000) -> [bde00000-be000000) Initmem setup node 0 [0000000000000000-0000000fffffffff]
NODE_DATA [#000000007de00000 - 0x0000007de02fff] (remapped)
62606MB HIGHMEM available.
2929MB LOWMEM available.max_low_pfn = b71fe, highstart_pfn = b71fe
need following patch.
Thanks
Yinghai
[PATCH] x86, numa: Trim numa meminfo correctly
During testing 32bit numa unifying code from tj, found one system with more than 64g fail to use numa.
It turn out we do not trim that numa meminfo correctly with max_pfn. start could be bigger than 64g too.
Also need to make the checking in seperated loop.
--- arch/x86/mm/numa.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
arch/x86/mm/numa.c +8 -5
@@ -272,6 +272,7 @@ int __init numa_cleanup_meminfo(struct nconst u64 high =PFN_PHYS (max_pfn);int i, j, k; +/* trim all entries at first */ for (i =0 ; i < mi->nr_blks; i++) {struct numa_memblk *bi = &mi->blk[i];
@@ -280,10 +281,12 @@ int __init numa_cleanup_meminfo(struct n bi->end =min (bi->end, high);/* and there's no empty block */ -if (bi->start == bi->end) { +if (bi->start >= bi->end)numa_remove_memblk_from (i--, mi); -continue ; - } + } + +for (i =0 ; i < mi->nr_blks; i++) { +struct numa_memblk *bi = &mi->blk[i];for (j = i +1 ; j < mi->nr_blks; j++) {struct numa_memblk *bj = &mi->blk[j];
@@ -313,8 +316,8 @@ int __init numa_cleanup_meminfo(struct n*/ if (bi->nid != bj->nid)continue ; - start =max (min (bi->start, bj->start), low); - end =min (max (bi->end, bj->end), high); + start =min (bi->start, bj->start); + end =max (bi->end, bj->end);for (k =0 ; k < mi->nr_blks; k++) {struct numa_memblk *bk = &mi->blk[k];
Hello, Yinghai.
Nice catch, but,
Quoted 9 lines · Yinghai Lu
[PATCH] x86, numa: Trim numa meminfo correctly
During testing 32bit numa unifying code from tj, found one system with more than 64g fail to use numa.
It turn out we do not trim that numa meminfo correctly with
max_pfn. start could be bigger than 64g too.Also need to make the checking in seperated loop.
Why?
Isn't all that necessary the following?
--- arch/x86/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/numa.c +1 -1
@@ -278,7 +278,7 @@ int __init numa_cleanup_meminfo(struct n bi->end =min (bi->end, high);/* and there's no empty block */ -if (bi->start == bi->end) { +if (bi->start >= bi->end) {numa_remove_memblk_from (i--, mi);continue ; }
PATCH 1/1 x86, NUMA: Fix empty memblk detection in numa_cleanup_meminfo()
From: Yinghai Lu <[email protected]>
numa_cleanup_meminfo() trims each memblk between low (0) and high (max_pfn) limits and discard empty ones. However, the emptiness detection incorrectly used equality test. If the start of a memblk is higher than max_pfn, it is empty but fails the equality test and doesn't get discarded.
Fix it by using >= instead of ==.
---
So, something like this. Does this fix the problem you see?
Thanks.
arch/x86/mm/numa_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/numa_64.c +1 -1
@@ -191,7 +191,7 @@ int __init numa_cleanup_meminfo(struct n bi->end =min (bi->end, high);/* and there's no empty block */ -if (bi->start == bi->end) { +if (bi->start >= bi->end) {numa_remove_memblk_from (i--, mi);continue ; }
Re: [PATCH] x86, NUMA: Fix empty memblk detection in numa_cleanup_meminfo()
Quoted 8 lines · Tejun Heo
From: Yinghai Lu <[email protected]>
numa_cleanup_meminfo()trims each memblk between low (0) and high (max_pfn) limits and discard empty ones. However, the emptiness detection incorrectly used equality test. If the start of a memblk is higher thanmax_pfn, it is empty but fails the equality test and doesn't get discarded.
Quoted 6 lines
Fix it by using >= instead of ==.
Signed-off-by Yinghai Lu <[email protected]>Signed-off-by Tejun Heo <[email protected]>---
So, something like this. Does this fix the problem you see?
Ooh, this is before the code is moved to numa.c, so please test the previous patch against numa.c.
Thanks.
Signature · 1 line
tejun
Re: [PATCH] x86, NUMA: Fix empty memblk detection in numa_cleanup_meminfo()
In reply to Tejun HeoQuoted 25 lines · Tejun Heo
From: Yinghai Lu <[email protected]>
numa_cleanup_meminfo()trims each memblk between low (0) and high (max_pfn) limits and discard empty ones. However, the emptiness detection incorrectly used equality test. If the start of a memblk is higher thanmax_pfn, it is empty but fails the equality test and doesn't get discarded.Fix it by using >= instead of ==.
Signed-off-by Yinghai Lu <[email protected]>Signed-off-by Tejun Heo <[email protected]>---
So, something like this. Does this fix the problem you see?Thanks.
arch/x86/mm/numa_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Index:
work/arch/x86/mm/numa.c
===================================================================
arch/x86/mm/numa.c int __init numa_cleanup_meminfo(struct n · line 197
Quoted 8 lines
arch/x86/mm/numa.c +1
-1@@ -191,7 +191,7 @@ int __init numa_cleanup_meminfo(struct n bi->end =min (bi->end, high);/* and there's no empty block */ -if (bi->start == bi->end) { +if (bi->start >= bi->end) {numa_remove_memblk_from (i--, mi);continue ; }
this one works too
but print out is some strange
on 512g system got:
SRAT: Node 0 PXM 0 0-a0000
SRAT: Node 0 PXM 0 100000-80000000
SRAT: Node 0 PXM 0 100000000-1080000000
SRAT: Node 1 PXM 1 1080000000-2080000000
SRAT: Node 2 PXM 2 2080000000-3080000000
SRAT: Node 3 PXM 3 3080000000-4080000000
SRAT: Node 4 PXM 4 4080000000-5080000000
SRAT: Node 5 PXM 5 5080000000-6080000000
SRAT: Node 6 PXM 6 6080000000-7080000000
SRAT: Node 7 PXM 7 7080000000-8080000000
NUMA: Initialized distance table, cnt=8
NUMA: Node 0 [0,a0000) + [100000,80000000) -> [0,80000000)
NUMA: Node 0 [0,80000000) + [100000000,1080000000) -> [0,1000000000)
first patch on 512g system got
NUMA: Node 0 [0,a0000) + [100000,80000000) -> [0,80000000)
NUMA: Node 0 [0,80000000) + [100000000,1000000000) -> [0,1000000000)
still thinking first one is more clean.
Thanks
Yinghai
Re: [PATCH] x86, NUMA: Fix empty memblk detection in numa_cleanup_meminfo()
Hello,
Quoted 9 lines · Yinghai Lu
this one works too
but print out is some strange
on 512g system got:NUMA: Initialized distance table, cnt=8
NUMA: Node 0 [0,a0000) + [100000,80000000) -> [0,80000000)
NUMA: Node 0 [0,80000000) + [100000000,1080000000) -> [0,1000000000)
Quoted 5 lines
first patch on 512g system got
NUMA: Node 0 [0,a0000) + [100000,80000000) -> [0,80000000)
NUMA: Node 0 [0,80000000) + [100000000,1000000000) -> [0,1000000000)still thinking first one is more clean.
Yeah, I don't object to the change (it's easier to understand too) but it should be a separate patch because it's unrelated change to the fix itself. If you don't mind sending the splitting part separately, I'll put it on top of the patchset.
Thanks.
Signature · 1 line
tejun
PATCH 1/1 x86, numa: Trim numa meminfo with max_pfn in separated loop
During testing 32bit numa unifying code from tj, found one system with more than 64g fail to use numa.
It turns out we do not trim that numa meminfo correctly with max_pfn. Because start could be bigger than 64g too.
Bug fix (checking correctly) already made it to tip tree.
This one move the checking and trimming to separated loop. So We don't need to compare low/high in following merge loops. It makes the code more readable.
Also make one 512g numa system with 32bit get not strange print out. befrore:
Quoted 2 lines
NUMA: Node 0 [0,a0000) + [100000,80000000) -> [0,80000000)
NUMA: Node 0 [0,80000000) + [100000000,1080000000) -> [0,1000000000)
after:
Quoted 2 lines
NUMA: Node 0 [0,a0000) + [100000,80000000) -> [0,80000000)
NUMA: Node 0 [0,80000000) + [100000000,1000000000) -> [0,1000000000)
--- arch/x86/mm/numa.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
arch/x86/mm/numa.c +8 -5
@@ -272,6 +272,7 @@ int __init numa_cleanup_meminfo(struct nconst u64 high =PFN_PHYS (max_pfn);int i, j, k; +/* Trim all entries at first */ for (i =0 ; i < mi->nr_blks; i++) {struct numa_memblk *bi = &mi->blk[i];
@@ -280,10 +281,12 @@ int __init numa_cleanup_meminfo(struct n bi->end =min (bi->end, high);/* and there's no empty block */ -if (bi->start >= bi->end) { +if (bi->start >= bi->end)numa_remove_memblk_from (i--, mi); -continue ; - } + } + +for (i =0 ; i < mi->nr_blks; i++) { +struct numa_memblk *bi = &mi->blk[i];for (j = i +1 ; j < mi->nr_blks; j++) {struct numa_memblk *bj = &mi->blk[j];
@@ -313,8 +316,8 @@ int __init numa_cleanup_meminfo(struct n*/ if (bi->nid != bj->nid)continue ; - start =max (min (bi->start, bj->start), low); - end =min (max (bi->end, bj->end), high); + start =min (bi->start, bj->start); + end =max (bi->end, bj->end);for (k =0 ; k < mi->nr_blks; k++) {struct numa_memblk *bk = &mi->blk[k];
Quoted 10 lines · Tejun Heo
Hello, Yinghai.
Nice catch, but,
Quoted 5 lines · Yinghai Lu
[PATCH] x86, numa: Trim numa meminfo correctly
During testing 32bit numa unifying code from tj, found one system with more than 64g fail to use numa.
Quoted 6 lines
Quoted 4 lines
It turn out we do not trim that numa meminfo correctly with
max_pfn. start could be bigger than 64g too.Also need to make the checking in seperated loop.
Why?
so do not need to compare them with low/high in following inner loop.
Quoted 20 lines
Isn't all that necessary the following?
--- arch/x86/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Index:
work/arch/x86/mm/numa.c
===================================================================arch/x86/mm/numa.c +1
-1@@ -278,7 +278,7 @@ int __init numa_cleanup_meminfo(struct n bi->end =min (bi->end, high);/* and there's no empty block */ -if (bi->start == bi->end) { +if (bi->start >= bi->end) {numa_remove_memblk_from (i--, mi);continue ; }