x86: memblock related cleanup
Subsystems (7)
cover x86: memblock related cleanup
1. allocate bootmem with memblock will follow top-down
2. make page table cold be as high and on local node.
3. seperate bootmem.c into nobootmem.c.
[PATCH 01/11] memblock: Don't adjust size in memblock_find_base() [PATCH 02/11] x86, mm, 64bit: Put early page table high. [PATCH 03/11] x86, 64bit, gart: Fix allocation with memblock [PATCH 04/11] memblock: Make find_memory_core_early() find from top-down [PATCH 05/11] x86: Change get_max_mapped() to inline
[PATCH 06/11] x86, 64bit, numa: Allocate memnodemap under max_pfn_mapped [PATCH 07/11] x86, 64bit, numa: Put pgtable to local node memory [PATCH 08/11] x86: Rename e820_table_* to pgt_buf_*
[PATCH 09/11] bootmem: Add nobootmem.c to reduce the #ifdef [PATCH 10/11] mm: Move contig_page_data define to bootmem.c/nobootmem.c [PATCH 11/11] memblock: Move __alloc_memory_core_early() to nobootmem.c
First patch is already in -mm, but better to go with this patchset as patch 3 need that. and patch 4 is depending patch 3.
Thanks
Yinghai Lu
PATCH 1/11 memblock: Don't adjust size in memblock_find_base()
While applying patch to use memblock to find aperture for 64bit x86. Ingo found system with 1g + force_iommu
Quoted 7 lines
No AGP bridge found
Node 0: aperture @ 38000000 size 32 MB Aperture pointing to e820 RAM. Ignoring. Your BIOS doesn't leave a aperture memory hole Please enable the IOMMU option in the BIOS setup This costs you 64 MB of RAM
Cannot allocate aperture memory hole (0,65536K)
the corresponding code:
addr = memblock_find_in_range(0, 1ULL<<32, aper_size, 512ULL<<20);
if (addr == MEMBLOCK_ERROR || addr + aper_size > 0xffffffff) {
printk(KERN_ERR
"Cannot allocate aperture memory hole (%lx,%uK)\n",
addr, aper_size>>10);
return 0;
}
memblock_x86_reserve_range(addr, addr + aper_size, "aperture64")it failes because memblock core code align the size with 512M. that could make size way too big.
So don't align the size in that case.
acctually __memblock_alloc_base, the another caller already align that before calling that function.
BTW. x86 does not use __memblock_alloc_base...
--- mm/memblock.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-)
mm/memblock.c +0 -2
@@ -137,8 +137,6 @@ static phys_addr_t __init_memblock memblock_find_base(phys_addr_t size,BUG_ON (0 == size); - size =memblock_align_up (size, align); -/* Pump up max_addr */ if (end ==MEMBLOCK_ALLOC_ACCESSIBLE ) end = memblock.current_limit;
Signature · 1 line
1.7.1
PATCH 2/11 x86, mm, 64bit: Put early page table high.
In reply to Yinghai LuWhile dubug kdump, found current kernel will have problem with crashkernel=512M.
It turns out that initial mapping is to 512M, and later initial mapping to 4G (acutally is 2040M in my platform), will put page table near 512M. then initial mapping to 128g will be near 2g.
before this patch:
[ 0.000000] initial memory mapped : 0 - 20000000 [ 0.000000] init_memory_mapping: [0x00000000000000-0x0000007f74ffff] [ 0.000000] 0000000000 - 007f600000 page 2M [ 0.000000] 007f600000 - 007f750000 page 4k [ 0.000000] kernel direct mapping tables up to 7f750000 @ [0x1fffc000-0x1fffffff] [ 0.000000] memblock_x86_reserve_range: [0x1fffc000-0x1fffdfff] PGTABLE [ 0.000000] init_memory_mapping: [0x00000100000000-0x0000207fffffff] [ 0.000000] 0100000000 - 2080000000 page 2M [ 0.000000] kernel direct mapping tables up to 2080000000 @ [0x7bc01000-0x7bc83fff] [ 0.000000] memblock_x86_reserve_range: [0x7bc01000-0x7bc7efff] PGTABLE [ 0.000000] RAMDISK: 7bc84000 - 7f745000 [ 0.000000] crashkernel reservation failed - No suitable area found.
after patch:
[ 0.000000] initial memory mapped : 0 - 20000000 [ 0.000000] init_memory_mapping: [0x00000000000000-0x0000007f74ffff] [ 0.000000] 0000000000 - 007f600000 page 2M [ 0.000000] 007f600000 - 007f750000 page 4k [ 0.000000] kernel direct mapping tables up to 7f750000 @ [0x7f74c000-0x7f74ffff] [ 0.000000] memblock_x86_reserve_range: [0x7f74c000-0x7f74dfff] PGTABLE [ 0.000000] init_memory_mapping: [0x00000100000000-0x0000207fffffff] [ 0.000000] 0100000000 - 2080000000 page 2M [ 0.000000] kernel direct mapping tables up to 2080000000 @ [0x207ff7d000-0x207fffffff] [ 0.000000] memblock_x86_reserve_range: [0x207ff7d000-0x207fffafff] PGTABLE [ 0.000000] RAMDISK: 7bc84000 - 7f745000 [ 0.000000] memblock_x86_reserve_range: [0x17000000-0x36ffffff] CRASH KERNEL [ 0.000000] Reserving 512MB of memory at 368MB for crashkernel (System RAM: 133120MB)
It means with the patch, page table for [0, 2g) will need 2g, instead of under 512M,page table for [4g, 128g) will be near 128g, instead of under 2g.
That would be good, if we have lots of memory above 4g, like 1024g, or 2048g or 16T, will not put related page table under 2g.
the code change will use add map_low_page() and update unmap_low_page() for 64bit, and use them to get access the corresponding high memory for page table setting.
--- arch/x86/mm/init.c | 9 ++---- arch/x86/mm/init_64.c | 63 +++++++++++++++++++++---------------------------- 2 files changed, 30 insertions(+), 42 deletions(-)
arch/x86/mm/init.c +3 -6
@@ -33,7 +33,7 @@ int direct_gbpagesstatic void __init find_early_table_space (unsigned long end,int use_pse,int use_gbpages) { -unsigned long puds, pmds, ptes, tables, start; +unsigned long puds, pmds, ptes, tables, start =0 , good_end = end;phys_addr_t base; puds = (end +PUD_SIZE -1 ) >>PUD_SHIFT ;
@@ -73,12 +73,9 @@ static void __init find_early_table_space(unsigned long end, int use_pse,* need roughly 0.5KB per GB. */ #ifdef CONFIG_X86_32 - start =0x7000 ; -#else - start =0x8000 ; + good_end = max_pfn_mapped <<PAGE_SHIFT ;#endif - base =memblock_find_in_range (start, max_pfn_mapped<<PAGE_SHIFT , - tables,PAGE_SIZE ); + base =memblock_find_in_range (start, good_end, tables,PAGE_SIZE );if (base ==MEMBLOCK_ERROR )panic ("Cannot find space for the kernel page tables" );
arch/x86/mm/init_64.c +27 -36
113 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 3/11 x86, 64bit, gart: Fix allocation with memblock
In reply to Yinghai LuWhen trying to change alloc_bootmem with memblock to go with real top-down Found one old system:
[ 0.000000] Node 0: aperture @ ac000000 size 64 MB [ 0.000000] Aperture pointing to e820 RAM. Ignoring. [ 0.000000] Your BIOS doesn't leave a aperture memory hole [ 0.000000] Please enable the IOMMU option in the BIOS setup [ 0.000000] This costs you 64 MB of RAM [ 0.000000] memblock_x86_reserve_range: [0x2020000000-0x2023ffffff] aperture64 [ 0.000000] Cannot allocate aperture memory hole (ffff882020000000,65536K) [ 0.000000] memblock_x86_free_range: [0x2020000000-0x2023ffffff] [ 0.000000] Kernel panic - not syncing: Not enough memory for aperture [ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.37-rc5-tip-yh-06229-gb792dc2-dirty #331 [ 0.000000] Call Trace: [ 0.000000] [<ffffffff81cf50fe>] ? panic+0x91/0x1a3 [ 0.000000] [<ffffffff827c66b2>] ? gart_iommu_hole_init+0x3d7/0x4a3 [ 0.000000] [<ffffffff81d026a9>] ? _etext+0x0/0x3 [ 0.000000] [<ffffffff827ba940>] ? pci_iommu_alloc+0x47/0x71 [ 0.000000] [<ffffffff827c820b>] ? mem_init+0x19/0xec [ 0.000000] [<ffffffff827b3c40>] ? start_kernel+0x20a/0x3e8 [ 0.000000] [<ffffffff827b32cc>] ? x86_64_start_reservations+0x9c/0xa0 [ 0.000000] [<ffffffff827b33e4>] ? x86_64_start_kernel+0x114/0x11b
it means __alloc_bootmem_nopanic() get too high for that aperture.
Use memblock_find_in_range() with limit directly.
--- arch/x86/kernel/aperture_64.c | 33 ++++++++++++++++----------------- 1 files changed, 16 insertions(+), 17 deletions(-)
arch/x86/kernel/aperture_64.c +16 -17
@@ -13,7 +13,7 @@#include <linux/kernel.h> #include <linux/types.h> #include <linux/init.h> -#include <linux/bootmem.h> +#include <linux/memblock.h> #include <linux/mmzone.h> #include <linux/pci_ids.h> #include <linux/pci.h>
@@ -57,7 +57,7 @@ static void __init insert_aperture_resource(u32 aper_base, u32 aper_size)static u32 __init allocate_aperture (void ) {u32 aper_size; -void *p; +unsigned long addr;/* aper_size should <= 1G */ if (fallback_aper_order >5 )
@@ -83,27 +83,26 @@ static u32 __init allocate_aperture(void)* so don't use 512M below as gart iommu, leave the space for kernel * code for safe */ - p =__alloc_bootmem_nopanic (aper_size, aper_size,512ULL <<20 ); + addr =memblock_find_in_range (0 ,1ULL <<32 , aper_size,512ULL <<20 ); +if (addr ==MEMBLOCK_ERROR || addr + aper_size >0xffffffff ) { +printk (KERN_ERR +"Cannot allocate aperture memory hole (%lx,%uK)\n" , + addr, aper_size>>10 ); +return 0 ; + } +memblock_x86_reserve_range (addr, addr + aper_size,"aperture64" );/* * Kmemleak should not scan this block as it may not be mapped via the * kernel direct mapping. */ -kmemleak_ignore (p); -if (!p ||__pa (p)+aper_size >0xffffffff ) { -printk (KERN_ERR -"Cannot allocate aperture memory hole (%p,%uK)\n" , - p, aper_size>>10 ); -if (p) -free_bootmem (__pa (p), aper_size); -return 0 ; - } +kmemleak_ignore (phys_to_virt (addr));printk (KERN_INFO "Mapping aperture over %d KB of RAM @ %lx\n" , - aper_size >>10 ,__pa (p)); -insert_aperture_resource ((u32 )__pa (p), aper_size); -register_nosave_region ((u32 )__pa (p) >>PAGE_SHIFT , - (u32 )__pa (p+aper_size) >>PAGE_SHIFT ); + aper_size >>10 , addr); +insert_aperture_resource ((u32 )addr, aper_size); +register_nosave_region (addr >>PAGE_SHIFT , + (addr+aper_size) >>PAGE_SHIFT ); -return (u32 )__pa (p); +return (u32 )addr; }
Signature · 1 line
1.7.1
PATCH 4/11 memblock: Make find_memory_core_early() find from top-down
In reply to Yinghai LuThat is used for find ram in node or bootmem type.
We should make it top-down so it will be consistent to memblock_find.
--- mm/page_alloc.c | 34 +++++++++++++++++++++++++++++++++- 1 files changed, 33 insertions(+), 1 deletions(-)
mm/page_alloc.c +33 -1
@@ -3616,6 +3616,34 @@ static int __meminit next_active_region_index_in_nid(int index, int nid)return -1 ; } +/* +* Basic iterator support. Return the last range of PFNs for a node +* Note: nid == MAX_NUMNODES returns last region regardless of node +*/ +static int __meminit last_active_region_index_in_nid (int nid) +{ +int i; + +for (i = nr_nodemap_entries -1 ; i >=0 ; i--) +if (nid ==MAX_NUMNODES || early_node_map[i].nid == nid) +return i; + +return -1 ; +} + +/* +* Basic iterator support. Return the previous active range of PFNs for a node +* Note: nid == MAX_NUMNODES returns next region regardless of node +*/ +static int __meminit previous_active_region_index_in_nid (int index,int nid) +{ +for (index = index -1 ; index >=0 ; index--) +if (nid ==MAX_NUMNODES || early_node_map[index].nid == nid) +return index; + +return -1 ; +} +#ifndef CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID /* * Required by SPARSEMEM. Given a PFN, return what node the PFN is on.
@@ -3667,6 +3695,10 @@ bool __meminit early_pfn_in_nid(unsigned long pfn, int node)for (i =first_active_region_index_in_nid (nid); i !=-1 ; \ i = next_active_region_index_in_nid(i, nid)) +#define for_each_active_range_index_in_nid_reverse (i, nid) \ + for (i = last_active_region_index_in_nid(nid); i != -1; \ + i = previous_active_region_index_in_nid(i, nid)) +/** * free_bootmem_with_active_regions - Call free_bootmem_node for each active range * @nid: The node to free memory on. If MAX_NUMNODES, all nodes are freed.
@@ -3705,7 +3737,7 @@ u64 __init find_memory_core_early(int nid, u64 size, u64 align,int i;/* Need to go over early_node_map to find out good range for node */ -for_each_active_range_index_in_nid (i, nid) { +for_each_active_range_index_in_nid_reverse (i, nid) {u64 addr;u64 ei_start, ei_last;u64 final_start, final_end;
Signature · 1 line
1.7.1
PATCH 5/11 x86: Change get_max_mapped() to inline
In reply to Yinghai LuMove it into head file. to prepare use it in other files.
-v2: hpa pointed out that u64 should not be used here.
Actually We could unsigned long here. because for 32bit it will under 4g.
--- arch/x86/include/asm/page_types.h | 5 +++++ arch/x86/kernel/setup.c | 9 --------- 2 files changed, 5 insertions(+), 9 deletions(-)
arch/x86/include/asm/page_types.h +5 -0
@@ -45,6 +45,11 @@ extern int devmem_is_allowed(unsigned long pagenr);extern unsigned long max_low_pfn_mapped;extern unsigned long max_pfn_mapped; +static inline unsigned long get_max_mapped (void ) +{ +return max_pfn_mapped<<PAGE_SHIFT ; +} +extern unsigned long init_memory_mapping (unsigned long start,unsigned long end);
arch/x86/kernel/setup.c +0 -9
@@ -680,15 +680,6 @@ static int __init parse_reservelow(char *p)early_param ("reservelow" , parse_reservelow); -static u64 __init get_max_mapped (void ) -{ -u64 end = max_pfn_mapped; - - end <<=PAGE_SHIFT ; - -return end; -} -/* * Determine if we were loaded by an EFI loader. If so, then we have also been * passed the efi memmap, systab, etc., so we should use these data structures
Signature · 1 line
1.7.1
PATCH 6/11 x86, 64bit, numa: Allocate memnodemap under max_pfn_mapped
In reply to Yinghai LuWe need to access it right way, so make sure that it is mapped already.
Prepare to put page table on local node, and nodemap is used before that.
--- arch/x86/mm/numa_64.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/mm/numa_64.c +1 -1
@@ -86,7 +86,7 @@ static int __init allocate_cachealigned_memnodemap(void) addr =0x8000 ; nodemap_size =roundup (sizeof (s16) * memnodemapsize,L1_CACHE_BYTES ); - nodemap_addr =memblock_find_in_range (addr, max_pfn<<PAGE_SHIFT , + nodemap_addr =memblock_find_in_range (addr,get_max_mapped (), nodemap_size,L1_CACHE_BYTES );if (nodemap_addr ==MEMBLOCK_ERROR ) { printk(KERN_ERR
Signature · 1 line
1.7.1
PATCH 7/11 x86, 64bit, numa: Put pgtable to local node memory
In reply to Yinghai LuIntroduce init_memory_mapping_high(), and use it with 64bit.
It will go with every memory segment above 4g to create page table to the memory range itself.
before this patch all page tables was on one node.
with this patch, one RED-PEN is killed
debug out for 8 sockets system after patch
[ 0.000000] initial memory mapped : 0 - 20000000 [ 0.000000] init_memory_mapping: [0x00000000000000-0x0000007f74ffff] [ 0.000000] 0000000000 - 007f600000 page 2M [ 0.000000] 007f600000 - 007f750000 page 4k [ 0.000000] kernel direct mapping tables up to 7f750000 @ [0x7f74c000-0x7f74ffff] [ 0.000000] RAMDISK: 7bc84000 - 7f745000 .... [ 0.000000] Adding active range (0, 0x10, 0x95) 0 entries of 3200 used [ 0.000000] Adding active range (0, 0x100, 0x7f750) 1 entries of 3200 used [ 0.000000] Adding active range (0, 0x100000, 0x1080000) 2 entries of 3200 used [ 0.000000] Adding active range (1, 0x1080000, 0x2080000) 3 entries of 3200 used [ 0.000000] Adding active range (2, 0x2080000, 0x3080000) 4 entries of 3200 used [ 0.000000] Adding active range (3, 0x3080000, 0x4080000) 5 entries of 3200 used [ 0.000000] Adding active range (4, 0x4080000, 0x5080000) 6 entries of 3200 used [ 0.000000] Adding active range (5, 0x5080000, 0x6080000) 7 entries of 3200 used [ 0.000000] Adding active range (6, 0x6080000, 0x7080000) 8 entries of 3200 used [ 0.000000] Adding active range (7, 0x7080000, 0x8080000) 9 entries of 3200 used [ 0.000000] init_memory_mapping: [0x00000100000000-0x0000107fffffff] [ 0.000000] 0100000000 - 1080000000 page 2M [ 0.000000] kernel direct mapping tables up to 1080000000 @ [0x107ffbd000-0x107fffffff] [ 0.000000] memblock_x86_reserve_range: [0x107ffc2000-0x107fffffff] PGTABLE [ 0.000000] init_memory_mapping: [0x00001080000000-0x0000207fffffff] [ 0.000000] 1080000000 - 2080000000 page 2M [ 0.000000] kernel direct mapping tables up to 2080000000 @ [0x207ff7d000-0x207fffffff] [ 0.000000] memblock_x86_reserve_range: [0x207ffc0000-0x207fffffff] PGTABLE [ 0.000000] init_memory_mapping: [0x00002080000000-0x0000307fffffff] [ 0.000000] 2080000000 - 3080000000 page 2M [ 0.000000] kernel direct mapping tables up to 3080000000 @ [0x307ff3d000-0x307fffffff] [ 0.000000] memblock_x86_reserve_range: [0x307ffc0000-0x307fffffff] PGTABLE [ 0.000000] init_memory_mapping: [0x00003080000000-0x0000407fffffff] [ 0.000000] 3080000000 - 4080000000 page 2M [ 0.000000] kernel direct mapping tables up to 4080000000 @ [0x407fefd000-0x407fffffff] [ 0.000000] memblock_x86_reserve_range: [0x407ffc0000-0x407fffffff] PGTABLE [ 0.000000] init_memory_mapping: [0x00004080000000-0x0000507fffffff] [ 0.000000] 4080000000 - 5080000000 page 2M [ 0.000000] kernel direct mapping tables up to 5080000000 @ [0x507febd000-0x507fffffff] [ 0.000000] memblock_x86_reserve_range: [0x507ffc0000-0x507fffffff] PGTABLE [ 0.000000] init_memory_mapping: [0x00005080000000-0x0000607fffffff] [ 0.000000] 5080000000 - 6080000000 page 2M [ 0.000000] kernel direct mapping tables up to 6080000000 @ [0x607fe7d000-0x607fffffff] [ 0.000000] memblock_x86_reserve_range: [0x607ffc0000-0x607fffffff] PGTABLE [ 0.000000] init_memory_mapping: [0x00006080000000-0x0000707fffffff] [ 0.000000] 6080000000 - 7080000000 page 2M [ 0.000000] kernel direct mapping tables up to 7080000000 @ [0x707fe3d000-0x707fffffff] [ 0.000000] memblock_x86_reserve_range: [0x707ffc0000-0x707fffffff] PGTABLE [ 0.000000] init_memory_mapping: [0x00007080000000-0x0000807fffffff] [ 0.000000] 7080000000 - 8080000000 page 2M [ 0.000000] kernel direct mapping tables up to 8080000000 @ [0x807fdfc000-0x807fffffff] [ 0.000000] memblock_x86_reserve_range: [0x807ffbf000-0x807fffffff] PGTABLE [ 0.000000] Initmem setup node 0 [0000000000000000-000000107fffffff] [ 0.000000] NODE_DATA [0x0000107ffbd000-0x0000107ffc1fff] [ 0.000000] Initmem setup node 1 [0000001080000000-000000207fffffff] [ 0.000000] NODE_DATA [0x0000207ffbb000-0x0000207ffbffff] [ 0.000000] Initmem setup node 2 [0000002080000000-000000307fffffff] [ 0.000000] NODE_DATA [0x0000307ffbb000-0x0000307ffbffff] [ 0.000000] Initmem setup node 3 [0000003080000000-000000407fffffff] [ 0.000000] NODE_DATA [0x0000407ffbb000-0x0000407ffbffff] [ 0.000000] Initmem setup node 4 [0000004080000000-000000507fffffff] [ 0.000000] NODE_DATA [0x0000507ffbb000-0x0000507ffbffff] [ 0.000000] Initmem setup node 5 [0000005080000000-000000607fffffff] [ 0.000000] NODE_DATA [0x0000607ffbb000-0x0000607ffbffff] [ 0.000000] Initmem setup node 6 [0000006080000000-000000707fffffff] [ 0.000000] NODE_DATA [0x0000707ffbb000-0x0000707ffbffff] [ 0.000000] Initmem setup node 7 [0000007080000000-000000807fffffff] [ 0.000000] NODE_DATA [0x0000807ffba000-0x0000807ffbefff]
-v2: update it to recent numa-emulation change
--- arch/x86/include/asm/page_types.h | 2 + arch/x86/kernel/setup.c | 8 ----- arch/x86/mm/amdtopology_64.c | 8 +++-- arch/x86/mm/init.c | 8 +----- arch/x86/mm/init_64.c | 54 +++++++++++++++++++++++++++++++++++++ arch/x86/mm/numa_64.c | 6 +++- arch/x86/mm/srat_64.c | 2 + 7 files changed, 68 insertions(+), 20 deletions(-)
arch/x86/include/asm/page_types.h +2 -0
@@ -53,6 +53,8 @@ static inline unsigned long get_max_mapped(void)extern unsigned long init_memory_mapping (unsigned long start,unsigned long end); +void init_memory_mapping_high (void ); +extern void initmem_init (unsigned long start_pfn,unsigned long end_pfn,int acpi,int k8);extern void free_initmem (void );
arch/x86/kernel/setup.c +0 -8
@@ -941,14 +941,6 @@ void __init setup_arch(char **cmdline_p) max_low_pfn_mapped =init_memory_mapping (0 , max_low_pfn<<PAGE_SHIFT ); max_pfn_mapped = max_low_pfn_mapped; -#ifdef CONFIG_X86_64 -if (max_pfn > max_low_pfn) { - max_pfn_mapped =init_memory_mapping (1UL <<32 , - max_pfn<<PAGE_SHIFT ); -/* can we preseve max_low_pfn ?*/ - max_low_pfn = max_pfn; - } -#endif memblock.current_limit =get_max_mapped ();/*
arch/x86/mm/amdtopology_64.c +5 -3
@@ -278,12 +278,14 @@ int __init amd_scan_nodes(void) apicid_base = boot_cpu_physical_apicid; } -for_each_node_mask (i, node_possible_map) { -int j; - +for_each_node_mask (i, node_possible_map)memblock_x86_register_active_regions (i, nodes[i].start >>PAGE_SHIFT , nodes[i].end >>PAGE_SHIFT ); +init_memory_mapping_high (); +for_each_node_mask (i, node_possible_map) { +int j; +for (j = apicid_base; j < cores + apicid_base; j++) apicid_to_node[(i << bits) + j] = i;setup_node_bootmem (i, nodes[i].start, nodes[i].end);
arch/x86/mm/init.c +1 -7
@@ -65,16 +65,10 @@ static void __init find_early_table_space(unsigned long end, int use_pse,#ifdef CONFIG_X86_32 /* for fixmap */ tables +=roundup (__end_of_fixed_addresses *sizeof (pte_t),PAGE_SIZE ); -#endif -/* -* RED-PEN putting page tables only on node 0 could -* cause a hotspot and fill up ZONE_DMA. The page tables -* need roughly 0.5KB per GB. -*/ -#ifdef CONFIG_X86_32 good_end = max_pfn_mapped <<PAGE_SHIFT ;#endif + base =memblock_find_in_range (start, good_end, tables,PAGE_SIZE );if (base ==MEMBLOCK_ERROR )panic ("Cannot find space for the kernel page tables" );
arch/x86/mm/init_64.c +54 -0
63 lines not on this page · raw message
arch/x86/mm/numa_64.c +4 -2
@@ -598,11 +598,12 @@ static int __init numa_emulation(unsigned long start_pfn,* the e820 memory map. */ remove_all_active_ranges (); -for_each_node_mask (i, node_possible_map) { +for_each_node_mask (i, node_possible_map)memblock_x86_register_active_regions (i, nodes[i].start >>PAGE_SHIFT , nodes[i].end >>PAGE_SHIFT ); +init_memory_mapping_high (); +for_each_node_mask (i, node_possible_map)setup_node_bootmem (i, nodes[i].start, nodes[i].end); - }setup_physnodes (addr, max_addr, acpi, amd);fake_physnodes (acpi, amd, num_nodes);numa_init_array ();
@@ -658,6 +659,7 @@ void __init initmem_init(unsigned long start_pfn, unsigned long last_pfn,for (i =0 ; i < nr_cpu_ids; i++)numa_set_node (i,0 );memblock_x86_register_active_regions (0 , start_pfn, last_pfn); +init_memory_mapping_high ();setup_node_bootmem (0 , start_pfn <<PAGE_SHIFT , last_pfn <<PAGE_SHIFT ); }
arch/x86/mm/srat_64.c +2 -0
@@ -444,6 +444,8 @@ int __init acpi_scan_nodes(unsigned long start, unsigned long end)return -1 ; } +init_memory_mapping_high (); +/* Account for nodes with cpus and no memory */ nodes_or (node_possible_map, nodes_parsed, cpu_nodes_parsed);
Signature · 1 line
1.7.1
PATCH 8/11 x86: Rename e820_table_* to pgt_buf_*
In reply to Yinghai LuNow it is found from memblock way.
Change the name to purpose related.
-v2: Ingo found "4/6 x86: Use early pre-allocated page table buffer top-down"
cause 32bit crash.
and need to drop it, So update this one accordingly.
--- arch/x86/include/asm/init.h | 6 +++--- arch/x86/mm/init.c | 20 ++++++++++---------- arch/x86/mm/init_32.c | 8 ++++---- arch/x86/mm/init_64.c | 4 ++-- arch/x86/xen/mmu.c | 2 +- 5 files changed, 20 insertions(+), 20 deletions(-)
arch/x86/include/asm/init.h +3 -3
@@ -11,8 +11,8 @@ kernel_physical_mapping_init(unsigned long start, unsigned long page_size_mask); -extern unsigned long __initdata e820_table_start; -extern unsigned long __meminitdata e820_table_end; -extern unsigned long __meminitdata e820_table_top; +extern unsigned long __initdata pgt_buf_start; +extern unsigned long __meminitdata pgt_buf_end; +extern unsigned long __meminitdata pgt_buf_top;#endif /* _ASM_X86_INIT_32_H */
arch/x86/mm/init.c +10 -10
@@ -18,9 +18,9 @@ DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); -unsigned long __initdata e820_table_start; -unsigned long __meminitdata e820_table_end; -unsigned long __meminitdata e820_table_top; +unsigned long __initdata pgt_buf_start; +unsigned long __meminitdata pgt_buf_end; +unsigned long __meminitdata pgt_buf_top;int after_bootmem;
@@ -73,12 +73,12 @@ static void __init find_early_table_space(unsigned long end, int use_pse,if (base ==MEMBLOCK_ERROR )panic ("Cannot find space for the kernel page tables" ); - e820_table_start = base >>PAGE_SHIFT ; - e820_table_end = e820_table_start; - e820_table_top = e820_table_start + (tables >>PAGE_SHIFT ); + pgt_buf_start = base >>PAGE_SHIFT ; + pgt_buf_end = pgt_buf_start; + pgt_buf_top = pgt_buf_start + (tables >>PAGE_SHIFT );printk (KERN_DEBUG "kernel direct mapping tables up to %lx @ %lx-%lx\n" , - end, e820_table_start <<PAGE_SHIFT , e820_table_top <<PAGE_SHIFT ); + end, pgt_buf_start <<PAGE_SHIFT , pgt_buf_top <<PAGE_SHIFT ); }struct map_range {
@@ -291,9 +291,9 @@ unsigned long __init_refok init_memory_mapping(unsigned long start,#endif __flush_tlb_all (); -if (!after_bootmem && e820_table_end > e820_table_start) -memblock_x86_reserve_range (e820_table_start <<PAGE_SHIFT , - e820_table_end <<PAGE_SHIFT ,"PGTABLE" ); +if (!after_bootmem && pgt_buf_end > pgt_buf_start) +memblock_x86_reserve_range (pgt_buf_start <<PAGE_SHIFT , + pgt_buf_end <<PAGE_SHIFT ,"PGTABLE" );if (!after_bootmem)early_memtest (start, end);
arch/x86/mm/init_32.c +4 -4
@@ -62,10 +62,10 @@ bool __read_mostly __vmalloc_start_set = false;static __init void *alloc_low_page (void ) { -unsigned long pfn = e820_table_end++; +unsigned long pfn = pgt_buf_end++;void *adr; -if (pfn >= e820_table_top) +if (pfn >= pgt_buf_top)panic ("alloc_low_page: ran out of memory" ); adr =__va (pfn *PAGE_SIZE );
@@ -163,8 +163,8 @@ static pte_t *__init page_table_kmap_check(pte_t *pte, pmd_t *pmd,if (pmd_idx_kmap_begin != pmd_idx_kmap_end && (vaddr >>PMD_SHIFT ) >= pmd_idx_kmap_begin && (vaddr >>PMD_SHIFT ) <= pmd_idx_kmap_end - && ((__pa (pte) >>PAGE_SHIFT ) < e820_table_start - || (__pa (pte) >>PAGE_SHIFT ) >= e820_table_end)) { + && ((__pa (pte) >>PAGE_SHIFT ) < pgt_buf_start + || (__pa (pte) >>PAGE_SHIFT ) >= pgt_buf_end)) {pte_t *newpte;int i;
arch/x86/mm/init_64.c +2 -2
@@ -314,7 +314,7 @@ void __init cleanup_highmap(void)static __ref void *alloc_low_page (unsigned long *phys) { -unsigned long pfn = e820_table_end++; +unsigned long pfn = pgt_buf_end++;void *adr;if (after_bootmem) {
@@ -324,7 +324,7 @@ static __ref void *alloc_low_page(unsigned long *phys)return adr; } -if (pfn >= e820_table_top) +if (pfn >= pgt_buf_top)panic ("alloc_low_page: ran out of memory" ); adr =early_memremap (pfn *PAGE_SIZE ,PAGE_SIZE );
arch/x86/xen/mmu.c +1 -1
@@ -1443,7 +1443,7 @@ static __init pte_t mask_rw_pte(pte_t *ptep, pte_t pte)* early_ioremap fixmap slot, make sure it is RO. */ if (!is_early_ioremap_ptep (ptep) && - pfn >= e820_table_start && pfn < e820_table_end) + pfn >= pgt_buf_start && pfn < pgt_buf_end) pte =pte_wrprotect (pte);return pte;
Signature · 1 line
1.7.1
PATCH 9/11 bootmem: Add nobootmem.c to reduce the #ifdef
In reply to Yinghai LuIntroduce nobootmem.c to hold wrapper for CONFIG_NO_BOOTMEM=y.
that will remove related #ifdef in bootmem.c
--- mm/Makefile | 8 +- mm/bootmem.c | 173 +------------------------ mm/nobootmem.c | 405 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 415 insertions(+), 171 deletions(-) create mode 100644 mm/nobootmem.c
mm/Makefile +7 -1
@@ -7,7 +7,7 @@ mmu-$(CONFIG_MMU) := fremap.o highmem.o madvise.o memory.o mincore.o \ mlock.o mmap.o mprotect.o mremap.o msync.o rmap.o \ vmalloc.o pagewalk.o pgtable-generic.o -obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ +obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page_alloc.o page-writeback.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \
@@ -15,6 +15,12 @@ obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ $(mmu-y) obj-y += init-mm.o +ifdef CONFIG_NO_BOOTMEM + obj-y += nobootmem.o +else + obj-y += bootmem.o +endif + obj-$(CONFIG_HAVE_MEMBLOCK) += memblock.o obj-$(CONFIG_BOUNCE) += bounce.o
mm/bootmem.c +3 -170
361 lines not on this page · raw message
mm/nobootmem.c new file +405 -0
405 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 10/11 mm: Move contig_page_data define to bootmem.c/nobootmem.c
In reply to Yinghai LuWe can remove #ifdef in mm/page_alloc.c
--- mm/bootmem.c | 7 +++++++ mm/nobootmem.c | 5 +++++ mm/page_alloc.c | 9 --------- 3 files changed, 12 insertions(+), 9 deletions(-)
mm/bootmem.c +7 -0
@@ -23,6 +23,13 @@#include "internal.h" +#ifndef CONFIG_NEED_MULTIPLE_NODES +struct pglist_data __refdata contig_page_data = { + .bdata = &bootmem_node_data[0 ] + }; +EXPORT_SYMBOL (contig_page_data); +#endif +unsigned long max_low_pfn;unsigned long min_low_pfn;unsigned long max_pfn;
mm/nobootmem.c +5 -0
@@ -23,6 +23,11 @@#include "internal.h" +#ifndef CONFIG_NEED_MULTIPLE_NODES +struct pglist_data __refdata contig_page_data; +EXPORT_SYMBOL (contig_page_data); +#endif +unsigned long max_low_pfn;unsigned long min_low_pfn;unsigned long max_pfn;
mm/page_alloc.c +0 -9
@@ -4841,15 +4841,6 @@ void __init set_dma_reserve(unsigned long new_dma_reserve) dma_reserve = new_dma_reserve; } -#ifndef CONFIG_NEED_MULTIPLE_NODES -struct pglist_data __refdata contig_page_data = { -#ifndef CONFIG_NO_BOOTMEM - .bdata = &bootmem_node_data[0 ] -#endif - }; -EXPORT_SYMBOL (contig_page_data); -#endif -void __init free_area_init (unsigned long *zones_size) { free_area_init_node(0, zones_size,
Signature · 1 line
1.7.1
Andrew Morton
maintainerRe: [PATCH 10/11] mm: Move contig_page_data define to bootmem.c/nobootmem.c
Quoted 14 lines · On Mon, 7 Feb 2011 15:17:11 -0800 Yinghai Lu <[email protected]> wrote:
We can remove #ifdef in
mm/page_alloc.cSigned-off-by Yinghai Lu <[email protected]>--- mm/bootmem.c | 7 +++++++ mm/nobootmem.c | 5 +++++ mm/page_alloc.c | 9 --------- 3 files changed, 12 insertions(+), 9 deletions(-)mm/bootmem.c +0
-0
mm/bootmem.c line 31
Quoted 9 lines
mm/bootmem.c +6
-0@@ -23,6 +23,13 @@#include "internal.h" +#ifndef CONFIG_NEED_MULTIPLE_NODES +struct pglist_data __refdata contig_page_data = { + .bdata = &bootmem_node_data[0 ] + }; +EXPORT_SYMBOL (contig_page_data); +#endif
Please fix the layout when moving this code - it's quite busted and as we're touching the code anyway, we might as well fix it up.
Patches 9, 10 and 11 look good to me and won't conflict with ther pending work - please merge them up via the appropriate ingotree.
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/3 bootmem: Add nobootmem.c to reduce the #ifdef
Introduce nobootmem.c to hold wrapper for CONFIG_NO_BOOTMEM=y.
that will remove related #ifdef in bootmem.c
--- mm/Makefile | 8 - mm/bootmem.c | 173 ------------------------ mm/nobootmem.c | 405 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 415 insertions(+), 171 deletions(-)
mm/Makefile +7 -1
@@ -7,7 +7,7 @@ mmu-$(CONFIG_MMU) := fremap.o highmem.o mlock.o mmap.o mprotect.o mremap.o msync.o rmap.o \ vmalloc.o pagewalk.o pgtable-generic.o -obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ +obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page_alloc.o page-writeback.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \
@@ -15,6 +15,12 @@ obj-y := bootmem.o filemap.o mempool.o $(mmu-y) obj-y += init-mm.o +ifdef CONFIG_NO_BOOTMEM + obj-y += nobootmem.o +else + obj-y += bootmem.o +endif + obj-$(CONFIG_HAVE_MEMBLOCK) += memblock.o obj-$(CONFIG_BOUNCE) += bounce.o
mm/bootmem.c +3 -170
361 lines not on this page · raw message
mm/nobootmem.c new file +405 -0
405 lines not on this page · raw message
PATCH 1/1 bootmem: Separate out CONFIG_NO_BOOTMEM code into nobootmem.c
Quoted 1 line
From
0932587328d9bd5b500a640fbaff3290c8d4cabfMon Sep 17 00:00:00 2001
From: Yinghai Lu <[email protected]>
Date: Thu, 24 Feb 2011 14:43:05 +0100
mm/bootmem.c contained code paths for both bootmem and no bootmem configurations. They implement about the same set of APIs in different ways and as a result bootmem.c contains massive amount of #ifdef CONFIG_NO_BOOTMEM.
Separate out CONFIG_NO_BOOTMEM code into mm/nobootmem.c. As the common part is relatively small, duplicate them in nobootmem.c instead of creating a common file or ifdef'ing in bootmem.c.
The followings are duplicated.
* {min|max}_low_pfn, max_pfn, saved_max_pfn
* free_bootmem_late()
* ___alloc_bootmem()
* __alloc_bootmem_low()
The followings are applicable only to nobootmem and moved verbatim.
* __free_pages_memory()
* free_all_memory_core_early()
The followings are not applicable to nobootmem and omitted in nobootmem.c.
* reserve_bootmem_node()
* reserve_bootmem()
The rest split function bodies according to CONFIG_NO_BOOTMEM.
Makefile is updated so that only either bootmem.c or nobootmem.c is built according to CONFIG_NO_BOOTMEM.
This patch doesn't introduce any behavior change.
-tj: Rewrote commit description.
---
Comitted to misc:x86-numa with patch description rewritten. Thanks.
mm/Makefile | 8 +- mm/bootmem.c | 173 +------------------------ mm/nobootmem.c | 405 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 415 insertions(+), 171 deletions(-) create mode 100644 mm/nobootmem.c
mm/Makefile +7 -1
@@ -7,7 +7,7 @@ mmu-$(CONFIG_MMU) := fremap.o highmem.o madvise.o memory.o mincore.o \ mlock.o mmap.o mprotect.o mremap.o msync.o rmap.o \ vmalloc.o pagewalk.o pgtable-generic.o -obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ +obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page_alloc.o page-writeback.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \
@@ -15,6 +15,12 @@ obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ $(mmu-y) obj-y += init-mm.o +ifdef CONFIG_NO_BOOTMEM + obj-y += nobootmem.o +else + obj-y += bootmem.o +endif + obj-$(CONFIG_HAVE_MEMBLOCK) += memblock.o obj-$(CONFIG_BOUNCE) += bounce.o
mm/bootmem.c +3 -170
361 lines not on this page · raw message
mm/nobootmem.c new file +405 -0
405 lines not on this page · raw message
Signature · 1 line
1.7.1
PATCH 2/3 mm: Move contig_page_data define to bootmem.c/nobootmem.c
In reply to a message not in the archiveWe can remove #ifdef in mm/page_alloc.c
-v2: According to Andrew, fixed the struct layout.
--- mm/bootmem.c | 7 +++++++ mm/nobootmem.c | 5 +++++ mm/page_alloc.c | 9 --------- 3 files changed, 12 insertions(+), 9 deletions(-)
mm/bootmem.c +7 -0
@@ -23,6 +23,13 @@#include "internal.h" +#ifndef CONFIG_NEED_MULTIPLE_NODES +struct pglist_data __refdata contig_page_data = { + .bdata = &bootmem_node_data[0 ] +}; +EXPORT_SYMBOL (contig_page_data); +#endif +unsigned long max_low_pfn;unsigned long min_low_pfn;unsigned long max_pfn;
mm/nobootmem.c +5 -0
@@ -23,6 +23,11 @@#include "internal.h" +#ifndef CONFIG_NEED_MULTIPLE_NODES +struct pglist_data __refdata contig_page_data; +EXPORT_SYMBOL (contig_page_data); +#endif +unsigned long max_low_pfn;unsigned long min_low_pfn;unsigned long max_pfn;
mm/page_alloc.c +0 -9
@@ -4841,15 +4841,6 @@ void __init set_dma_reserve(unsigned lon dma_reserve = new_dma_reserve; } -#ifndef CONFIG_NEED_MULTIPLE_NODES -struct pglist_data __refdata contig_page_data = { -#ifndef CONFIG_NO_BOOTMEM - .bdata = &bootmem_node_data[0 ] -#endif - }; -EXPORT_SYMBOL (contig_page_data); -#endif -void __init free_area_init (unsigned long *zones_size) { free_area_init_node(0, zones_size,
PATCH 1/1 bootmem: Move contig_page_data definition to bootmem.c/nobootmem.c
Quoted 1 line
From
e782ab421bbba1912c87934bd0e8998630736418Mon Sep 17 00:00:00 2001
From: Yinghai Lu <[email protected]>
Date: Thu, 24 Feb 2011 14:43:06 +0100
Now that bootmem.c and nobootmem.c are separate, it's cleaner to define contig_page_data in each file than in page_alloc.c with #ifdef. Move it.
This patch doesn't introduce any behavior change.
-v2: According to Andrew, fixed the struct layout. -tj: Updated commit description.
---
Committed to misc:x86-numa with description updates. Thanks.
mm/bootmem.c | 7 +++++++ mm/nobootmem.c | 5 +++++ mm/page_alloc.c | 9 --------- 3 files changed, 12 insertions(+), 9 deletions(-)
mm/bootmem.c +7 -0
@@ -23,6 +23,13 @@#include "internal.h" +#ifndef CONFIG_NEED_MULTIPLE_NODES +struct pglist_data __refdata contig_page_data = { + .bdata = &bootmem_node_data[0 ] +}; +EXPORT_SYMBOL (contig_page_data); +#endif +unsigned long max_low_pfn;unsigned long min_low_pfn;unsigned long max_pfn;
mm/nobootmem.c +5 -0
@@ -23,6 +23,11 @@#include "internal.h" +#ifndef CONFIG_NEED_MULTIPLE_NODES +struct pglist_data __refdata contig_page_data; +EXPORT_SYMBOL (contig_page_data); +#endif +unsigned long max_low_pfn;unsigned long min_low_pfn;unsigned long max_pfn;
mm/page_alloc.c +0 -9
@@ -4841,15 +4841,6 @@ void __init set_dma_reserve(unsigned long new_dma_reserve) dma_reserve = new_dma_reserve; } -#ifndef CONFIG_NEED_MULTIPLE_NODES -struct pglist_data __refdata contig_page_data = { -#ifndef CONFIG_NO_BOOTMEM - .bdata = &bootmem_node_data[0 ] -#endif - }; -EXPORT_SYMBOL (contig_page_data); -#endif -void __init free_area_init (unsigned long *zones_size) { free_area_init_node(0, zones_size,
Signature · 1 line
1.7.1
PATCH 3/3 memblock: Move __alloc_memory_core_early() to nobootmem.c
In reply to a message not in the archiveWe can remove #ifdef in mm/page_alloc.c
and change that function to static
--- include/linux/mm.h | 2 -- mm/nobootmem.c | 25 +++++++++++++++++++++++++ mm/page_alloc.c | 28 ---------------------------- 3 files changed, 25 insertions(+), 30 deletions(-)
include/linux/mm.h +0 -2
@@ -1309,8 +1309,6 @@ int add_from_early_node_map(struct range int nr_range, int nid);u64 __init find_memory_core_early (int nid,u64 size,u64 align,u64 goal,u64 limit); -void *__alloc_memory_core_early (int nodeid,u64 size,u64 align, -u64 goal,u64 limit);typedef int (*work_fn_t )(unsigned long ,unsigned long ,void *);extern void work_with_active_regions (int nid,work_fn_t work_fn,void *data);extern void sparse_memory_present_with_active_regions (int nid);
mm/nobootmem.c +25 -0
@@ -40,6 +40,31 @@ unsigned long max_pfn;unsigned long saved_max_pfn;#endif +static void *__init __alloc_memory_core_early (int nid,u64 size,u64 align, +u64 goal,u64 limit) +{ +void *ptr; +u64 addr; + +if (limit > memblock.current_limit) + limit = memblock.current_limit; + + addr =find_memory_core_early (nid, size, align, goal, limit); + +if (addr ==MEMBLOCK_ERROR ) +return NULL ; + + ptr =phys_to_virt (addr); +memset (ptr,0 , size); +memblock_x86_reserve_range (addr, addr + size,"BOOTMEM" ); +/* +* The min_count is set to 0 so that bootmem allocated blocks +* are never reported as leaks. +*/ +kmemleak_alloc (ptr, size,0 ,0 ); +return ptr; +} +/* * free_bootmem_late - free bootmem pages directly to page allocator * @addr: starting address of the range
mm/page_alloc.c +0 -28
@@ -3780,34 +3780,6 @@ int __init add_from_early_node_map(strucreturn nr_range; } -#ifdef CONFIG_NO_BOOTMEM -void *__init __alloc_memory_core_early (int nid,u64 size,u64 align, -u64 goal,u64 limit) -{ -void *ptr; -u64 addr; - -if (limit > memblock.current_limit) - limit = memblock.current_limit; - - addr =find_memory_core_early (nid, size, align, goal, limit); - -if (addr ==MEMBLOCK_ERROR ) -return NULL ; - - ptr =phys_to_virt (addr); -memset (ptr,0 , size); -memblock_x86_reserve_range (addr, addr + size,"BOOTMEM" ); -/* -* The min_count is set to 0 so that bootmem allocated blocks -* are never reported as leaks. -*/ -kmemleak_alloc (ptr, size,0 ,0 ); -return ptr; -} -#endif - -void __init work_with_active_regions (int nid,work_fn_t work_fn,void *data) {int i;
PATCH 1/1 bootmem: Move __alloc_memory_core_early() to nobootmem.c
Quoted 1 line
From
8bc1f91e1f0e977fb95b11d8fa686f5091888110Mon Sep 17 00:00:00 2001
From: Yinghai Lu <[email protected]>
Date: Thu, 24 Feb 2011 14:43:06 +0100
Now that bootmem.c and nobootmem.c are separate, there's no reason to define __alloc_memory_core_early(), which is used only by nobootmem, inside #ifdef in page_alloc.c. Move it to nobootmem.c and make it static.
This patch doesn't introduce any behavior change.
-tj: Updated commit description.
---
Committed to misc:x86-numa with description updated. Thanks.
include/linux/mm.h | 2 -- mm/nobootmem.c | 25 +++++++++++++++++++++++++ mm/page_alloc.c | 28 ---------------------------- 3 files changed, 25 insertions(+), 30 deletions(-)
include/linux/mm.h +0 -2
@@ -1309,8 +1309,6 @@ int add_from_early_node_map(struct range *range, int az, int nr_range, int nid);u64 __init find_memory_core_early (int nid,u64 size,u64 align,u64 goal,u64 limit); -void *__alloc_memory_core_early (int nodeid,u64 size,u64 align, -u64 goal,u64 limit);typedef int (*work_fn_t )(unsigned long ,unsigned long ,void *);extern void work_with_active_regions (int nid,work_fn_t work_fn,void *data);extern void sparse_memory_present_with_active_regions (int nid);
mm/nobootmem.c +25 -0
@@ -40,6 +40,31 @@ unsigned long max_pfn;unsigned long saved_max_pfn;#endif +static void *__init __alloc_memory_core_early (int nid,u64 size,u64 align, +u64 goal,u64 limit) +{ +void *ptr; +u64 addr; + +if (limit > memblock.current_limit) + limit = memblock.current_limit; + + addr =find_memory_core_early (nid, size, align, goal, limit); + +if (addr ==MEMBLOCK_ERROR ) +return NULL ; + + ptr =phys_to_virt (addr); +memset (ptr,0 , size); +memblock_x86_reserve_range (addr, addr + size,"BOOTMEM" ); +/* +* The min_count is set to 0 so that bootmem allocated blocks +* are never reported as leaks. +*/ +kmemleak_alloc (ptr, size,0 ,0 ); +return ptr; +} +/* * free_bootmem_late - free bootmem pages directly to page allocator * @addr: starting address of the range
mm/page_alloc.c +0 -28
@@ -3780,34 +3780,6 @@ int __init add_from_early_node_map(struct range *range, int az,return nr_range; } -#ifdef CONFIG_NO_BOOTMEM -void *__init __alloc_memory_core_early (int nid,u64 size,u64 align, -u64 goal,u64 limit) -{ -void *ptr; -u64 addr; - -if (limit > memblock.current_limit) - limit = memblock.current_limit; - - addr =find_memory_core_early (nid, size, align, goal, limit); - -if (addr ==MEMBLOCK_ERROR ) -return NULL ; - - ptr =phys_to_virt (addr); -memset (ptr,0 , size); -memblock_x86_reserve_range (addr, addr + size,"BOOTMEM" ); -/* -* The min_count is set to 0 so that bootmem allocated blocks -* are never reported as leaks. -*/ -kmemleak_alloc (ptr, size,0 ,0 ); -return ptr; -} -#endif - -void __init work_with_active_regions (int nid,work_fn_t work_fn,void *data) {int i;
Signature · 1 line
1.7.1
cover mm: Split bootmem.c for nobootmem support
In reply to Andrew MortonResubmit for put into tip tree, with Andrew's Acked-by.
Those three patches could be applied to current Linus tree cleanly. So could be in separated tip/mm branch?
Thanks
Yinghai
PATCH 11/11 memblock: Move __alloc_memory_core_early() to nobootmem.c
In reply to Yinghai LuWe can remove #ifdef in mm/page_alloc.c
and change that function to static
--- include/linux/mm.h | 2 -- mm/nobootmem.c | 25 +++++++++++++++++++++++++ mm/page_alloc.c | 28 ---------------------------- 3 files changed, 25 insertions(+), 30 deletions(-)
include/linux/mm.h +0 -2
@@ -1309,8 +1309,6 @@ int add_from_early_node_map(struct range *range, int az, int nr_range, int nid);u64 __init find_memory_core_early (int nid,u64 size,u64 align,u64 goal,u64 limit); -void *__alloc_memory_core_early (int nodeid,u64 size,u64 align, -u64 goal,u64 limit);typedef int (*work_fn_t )(unsigned long ,unsigned long ,void *);extern void work_with_active_regions (int nid,work_fn_t work_fn,void *data);extern void sparse_memory_present_with_active_regions (int nid);
mm/nobootmem.c +25 -0
@@ -40,6 +40,31 @@ unsigned long max_pfn;unsigned long saved_max_pfn;#endif +static void *__init __alloc_memory_core_early (int nid,u64 size,u64 align, +u64 goal,u64 limit) +{ +void *ptr; +u64 addr; + +if (limit > memblock.current_limit) + limit = memblock.current_limit; + + addr =find_memory_core_early (nid, size, align, goal, limit); + +if (addr ==MEMBLOCK_ERROR ) +return NULL ; + + ptr =phys_to_virt (addr); +memset (ptr,0 , size); +memblock_x86_reserve_range (addr, addr + size,"BOOTMEM" ); +/* +* The min_count is set to 0 so that bootmem allocated blocks +* are never reported as leaks. +*/ +kmemleak_alloc (ptr, size,0 ,0 ); +return ptr; +} +/* * free_bootmem_late - free bootmem pages directly to page allocator * @addr: starting address of the range
mm/page_alloc.c +0 -28
@@ -3780,34 +3780,6 @@ int __init add_from_early_node_map(struct range *range, int az,return nr_range; } -#ifdef CONFIG_NO_BOOTMEM -void *__init __alloc_memory_core_early (int nid,u64 size,u64 align, -u64 goal,u64 limit) -{ -void *ptr; -u64 addr; - -if (limit > memblock.current_limit) - limit = memblock.current_limit; - - addr =find_memory_core_early (nid, size, align, goal, limit); - -if (addr ==MEMBLOCK_ERROR ) -return NULL ; - - ptr =phys_to_virt (addr); -memset (ptr,0 , size); -memblock_x86_reserve_range (addr, addr + size,"BOOTMEM" ); -/* -* The min_count is set to 0 so that bootmem allocated blocks -* are never reported as leaks. -*/ -kmemleak_alloc (ptr, size,0 ,0 ); -return ptr; -} -#endif - -void __init work_with_active_regions (int nid,work_fn_t work_fn,void *data) {int i;
Signature · 1 line
1.7.1
Quoted 16 lines · Yinghai Lu
1. allocate bootmem with memblock will follow top-down
2. make page table cold be as high and on local node.
3. seperatebootmem.cintonobootmem.c.[PATCH 01/11] memblock: Don't adjust size in
memblock_find_base()[PATCH 02/11] x86, mm, 64bit: Put early page table high. [PATCH 03/11] x86, 64bit, gart: Fix allocation with memblock [PATCH 04/11] memblock: Makefind_memory_core_early()find from top-down [PATCH 05/11] x86: Changeget_max_mapped()to inline
[PATCH 06/11] x86, 64bit, numa: Allocate memnodemap undermax_pfn_mapped[PATCH 07/11] x86, 64bit, numa: Put pgtable to local node memory [PATCH 08/11] x86: Rename e820_table_* to pgt_buf_*
[PATCH 09/11] bootmem: Addnobootmem.cto reduce the #ifdef [PATCH 10/11] mm: Movecontig_page_datadefine tobootmem.c/nobootmem.c[PATCH 11/11] memblock: Move__alloc_memory_core_early()tonobootmem.c
Quoted 3 lines
First patch is already in -mm, but better to go with this patchset as patch 3 need that. and patch 4 is depending patch 3.
Ingo,
patch1 is in mainline already via Andrew path.
patch2 to patch7 are in tip/x86/bootmem, and now it is in tip/master.
please check
git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git
memblock
for last four patches.
main change is creating nobootmem.c for memblock bootmem wrapper.
arch/x86/include/asm/init.h | 6 +- arch/x86/mm/init.c | 20 +- arch/x86/mm/init_32.c | 8 +- arch/x86/mm/init_64.c | 4 +- arch/x86/xen/mmu.c | 2 +- include/linux/mm.h | 2 - mm/Makefile | 8 +- mm/bootmem.c | 180 +----------------- mm/nobootmem.c | 435 +++++++++++++++++++++++++++++++++++++++++++ mm/page_alloc.c | 37 ---- 10 files changed, 472 insertions(+), 230 deletions(-) create mode 100644 mm/nobootmem.c
Thanks
Yinghai Lu