early_node_mem()'s memory allocation policy
We're seeing problems under Xen where large portions of the memory
could be reserved (because they're not yet physically present, even though the appear in E820), and the 'start' and 'end' early_node_mem() is choosing is entirely within that reserved range.
Also, the code seems dubious because it adjusts start and end without regarding how much space it is trying to allocate:
/* extend the search scope */ end = max_pfn_mapped << PAGE_SHIFT; if (end > (MAX_DMA32_PFN<<PAGE_SHIFT)) start = MAX_DMA32_PFN<<PAGE_SHIFT; else start = MAX_DMA_PFN<<PAGE_SHIFT;
what if max_pfn_mapped is only a few pages larger than MAX_DMA32_PFN, and that is smaller than the size it is trying to allocate?
I tried just removing the start and end adjustments in early_node_mem() and the kernel booted fine under Xen, but it seemed to allocate at a very low address. Should the for_each_active_range_index_in_nid() loop in find_memory_core_early() be iterating from high to low addresses? If the allocation could be relied on to be top-down, then you wouldn't need to adjust start at all, and it would return the highest available memory in a natural way.
Thanks,
J
Quoted 18 lines · Jeremy Fitzhardinge
We're seeing problems under Xen where large portions of the memory
could be reserved (because they're not yet physically present, even though the appear in E820), and the 'start' and 'end'early_node_mem()is choosing is entirely within that reserved range.Also, the code seems dubious because it adjusts start and end without regarding how much space it is trying to allocate:
/* extend the search scope */ end = max_pfn_mapped << PAGE_SHIFT; if (end > (MAX_DMA32_PFN<<PAGE_SHIFT)) start = MAX_DMA32_PFN<<PAGE_SHIFT; else start = MAX_DMA_PFN<<PAGE_SHIFT;what if
max_pfn_mappedis only a few pages larger thanMAX_DMA32_PFN, and that is smaller than the size it is trying to allocate?
Quoted 7 lines
I tried just removing the start and end adjustments in
early_node_mem()and the kernel booted fine under Xen, but it seemed to allocate at a very low address. Should thefor_each_active_range_index_in_nid()loop infind_memory_core_early()be iterating from high to low addresses? If the allocation could be relied on to be top-down, then you wouldn't need to adjust start at all, and it would return the highest available memory in a natural way.
please check
[PATCH] x86, memblock: Fix early_node_mem with big reserved region.
Jeremy said Xen could reserve huge mem but still show as ram in e820.
early_node_mem could not find range because of start/end adjusting.
Let's use memblock_find_in_range instead ***_node. So get real top down in fallback path.
arch/x86/mm/numa_64.c +2 -5
@@ -178,11 +178,8 @@ static void * __init early_node_mem(int nodeid, unsigned long start,/* extend the search scope */ end = max_pfn_mapped <<PAGE_SHIFT ; -if (end > (MAX_DMA32_PFN <<PAGE_SHIFT )) - start =MAX_DMA32_PFN <<PAGE_SHIFT ; -else - start =MAX_DMA_PFN <<PAGE_SHIFT ; - mem =memblock_x86_find_in_range_node (nodeid, start, end, size, align); + start =MAX_DMA_PFN <<PAGE_SHIFT ; + mem =memblock_find_in_range (start, end, size, align);if (mem !=MEMBLOCK_ERROR )return __va (mem);
Konrad Rzeszutek Wilk
Tested-byQuoted 27 lines · Yinghai Lu
Quoted 18 lines · Jeremy Fitzhardinge
We're seeing problems under Xen where large portions of the memory
could be reserved (because they're not yet physically present, even though the appear in E820), and the 'start' and 'end'early_node_mem()is choosing is entirely within that reserved range.Also, the code seems dubious because it adjusts start and end without regarding how much space it is trying to allocate:
/* extend the search scope */ end = max_pfn_mapped << PAGE_SHIFT; if (end > (MAX_DMA32_PFN<<PAGE_SHIFT)) start = MAX_DMA32_PFN<<PAGE_SHIFT; else start = MAX_DMA_PFN<<PAGE_SHIFT;what if
max_pfn_mappedis only a few pages larger thanMAX_DMA32_PFN, and that is smaller than the size it is trying to allocate?Quoted 7 lines
I tried just removing the start and end adjustments in
early_node_mem()and the kernel booted fine under Xen, but it seemed to allocate at a very low address. Should thefor_each_active_range_index_in_nid()loop infind_memory_core_early()be iterating from high to low addresses? If the allocation could be relied on to be top-down, then you wouldn't need to adjust start at all, and it would return the highest available memory in a natural way.
Quoted 1 line
please check
It definitly gets us across that hump. Thanks.
Quoted 10 lines
[PATCH] x86, memblock: Fix
early_node_memwith big reserved region.Jeremy said Xen could reserve huge mem but still show as ram in e820.
early_node_memcould not find range because of start/end adjusting.Let's use
memblock_find_in_rangeinstead ***_node. So get real top down in fallback path.Signed-off-by Yinghai Lu <[email protected]>
Quoted 19 lines
arch/x86/mm/numa_64.c +2
-5@@ -178,11 +178,8 @@ static void * __init early_node_mem(int nodeid, unsigned long start,/* extend the search scope */ end = max_pfn_mapped <<PAGE_SHIFT ; -if (end > (MAX_DMA32_PFN <<PAGE_SHIFT )) - start =MAX_DMA32_PFN <<PAGE_SHIFT ; -else - start =MAX_DMA_PFN <<PAGE_SHIFT ; - mem =memblock_x86_find_in_range_node (nodeid, start, end, size, align); + start =MAX_DMA_PFN <<PAGE_SHIFT ; + mem =memblock_find_in_range (start, end, size, align);if (mem !=MEMBLOCK_ERROR )return __va (mem);
Quoted 4 lines · Yinghai Lu
please check
[PATCH] x86, memblock: Fix
early_node_memwith big reserved region.
Quoted 8 lines
Jeremy said Xen could reserve huge mem but still show as ram in e820.
early_node_memcould not find range because of start/end adjusting.Let's use
memblock_find_in_rangeinstead ***_node. So get real top down in fallback path.Signed-off-by Yinghai Lu <[email protected]>
Yes, that works. Could you queue it for upstream? Without this we see Xen crashes with certain memory configurations.
Thanks,
J
Quoted 19 lines
arch/x86/mm/numa_64.c +2
-5@@ -178,11 +178,8 @@ static void * __init early_node_mem(int nodeid, unsigned long start,/* extend the search scope */ end = max_pfn_mapped <<PAGE_SHIFT ; -if (end > (MAX_DMA32_PFN <<PAGE_SHIFT )) - start =MAX_DMA32_PFN <<PAGE_SHIFT ; -else - start =MAX_DMA_PFN <<PAGE_SHIFT ; - mem =memblock_x86_find_in_range_node (nodeid, start, end, size, align); + start =MAX_DMA_PFN <<PAGE_SHIFT ; + mem =memblock_find_in_range (start, end, size, align);if (mem !=MEMBLOCK_ERROR )return __va (mem);
PATCH 1/1 x86, memblock: Fix early_node_mem with big reserved region.
Jeremy said Xen could reserve huge mem but still show as ram in e820. early_node_mem could not find range because of start/end adjusting. and will go through fallback path. but fallback is still using memblock_x86_find_range_node(), and it is partially top-down because it go through active_range entries from low to high.
Let's use memblock_find_in_range instead memblock_x86_find_range_node. So get real top down in fallback path.
This is for 2.6.37.
And We may still need to make memblock_x86_find_range_node to do overall top_down work.
--- arch/x86/mm/numa_64.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
arch/x86/mm/numa_64.c +2 -5
@@ -178,11 +178,8 @@ static void * __init early_node_mem(int/* extend the search scope */ end = max_pfn_mapped <<PAGE_SHIFT ; -if (end > (MAX_DMA32_PFN <<PAGE_SHIFT )) - start =MAX_DMA32_PFN <<PAGE_SHIFT ; -else - start =MAX_DMA_PFN <<PAGE_SHIFT ; - mem =memblock_x86_find_in_range_node (nodeid, start, end, size, align); + start =MAX_DMA_PFN <<PAGE_SHIFT ; + mem =memblock_find_in_range (start, end, size, align);if (mem !=MEMBLOCK_ERROR )return __va (mem);
[tip:x86/urgent] x86, memblock: Fix early_node_mem with big reserved region.
Commit-ID: 419db274bed4269f475a8e78cbe9c917192cfe8b Gitweb: http://git.kernel.org/tip/419db274bed4269f475a8e78cbe9c917192cfe8b Author: Yinghai Lu <[email protected]> AuthorDate: Thu, 28 Oct 2010 09:50:17 -0700 Committer: H. Peter Anvin <[email protected]> CommitDate: Thu, 28 Oct 2010 15:52:36 -0700
x86, memblock: Fix early_node_mem with big reserved region.
Xen can reserve huge amounts of memory for pre-ballooning, but that still shows as RAM in the e820 memory map. early_node_mem could not find range because of start/end adjusting, and will go through the fallback path. However, the fallback patch is still using memblock_x86_find_range_node(), and it is partially top-down because it go through active_range entries from low to high.
Let's use memblock_find_in_range instead memblock_x86_find_range_node. So get real top down in fallback path.
We may still need to make memblock_x86_find_range_node to do overall top_down work.
LKML-Reference: <[email protected]>
--- arch/x86/mm/numa_64.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-)
arch/x86/mm/numa_64.c +2 -5
@@ -178,11 +178,8 @@ static void * __init early_node_mem(int nodeid, unsigned long start,/* extend the search scope */ end = max_pfn_mapped <<PAGE_SHIFT ; -if (end > (MAX_DMA32_PFN <<PAGE_SHIFT )) - start =MAX_DMA32_PFN <<PAGE_SHIFT ; -else - start =MAX_DMA_PFN <<PAGE_SHIFT ; - mem =memblock_x86_find_in_range_node (nodeid, start, end, size, align); + start =MAX_DMA_PFN <<PAGE_SHIFT ; + mem =memblock_find_in_range (start, end, size, align);if (mem !=MEMBLOCK_ERROR )return __va (mem);