percpu: make pcpu_build_alloc_info() clear static buffers
Tejun Heo
maintainerPATCH 1/3 percpu: make pcpu_build_alloc_info() clear static buffers
pcpu_build_alloc_info() may be called multiple times when percpu is falling back to different first chunk allocator. Make it clear static buffers so that they don't contain values from previous runs.
---
These three patches are scheduled for mainline and aim to work around the cases where distance between the two farthest units is too large compared to vmalloc area size. This happens on sparc64 because vmalloc area size is relatively small there and nodes can easily be placed such that they are too far apart.
This patchset implements page mapping first chunk allocator for sparc64 and make embedding allocator fallback to it when vmalloc area doesn't seem large enough. This should make percpu allocator more robust on other archs which implement page mapping allocator (only x86 currently) and diagnosing problems easier on other archs.
Thanks.
mm/percpu.c | 4 ++++ 1 file changed, 4 insertions(+)
mm/percpu.c +4 -0
@@ -1347,6 +1347,10 @@ struct pcpu_alloc_info * __init pcpu_buistruct pcpu_alloc_info *ai;unsigned int *cpu_map; +/* this function may be called multiple times */ +memset (group_map,0 ,sizeof (group_map)); +memset (group_cnt,0 ,sizeof (group_map)); +/* * Determine min_unit_size, alloc_size and max_upa such that * alloc_size is multiple of atom_size and is the smallest
Tejun Heo
maintainerPATCH 2/3 sparc64: implement page mapping percpu first chunk allocator
Implement page mapping percpu first chunk allocator as a fallback to the embedding allocator. The next patch will make the embedding allocator check distances between units to determine whether it fits within the vmalloc area so that this fallback can be used on such cases.
sparc64 currently has relatively small vmalloc area which makes it impossible to create any dynamic chunks on certain configurations leading to percpu allocation failures. This and the next patch should allow those configurations to keep working until proper solution is found.
---
David, can you please ack this after reviewing?
Thanks.
arch/sparc/Kconfig | 3 ++ arch/sparc/kernel/smp_64.c | 51 +++++++++++++++++++++++++++++++++++++-------- 2 files changed, 46 insertions(+), 8 deletions(-)
arch/sparc/Kconfig +3 -0
@@ -102,6 +102,9 @@ config HAVE_SETUP_PER_CPU_AREA config NEED_PER_CPU_EMBED_FIRST_CHUNK def_bool y if SPARC64 +config NEED_PER_CPU_PAGE_FIRST_CHUNK + def_bool y if SPARC64 + config GENERIC_HARDIRQS_NO__DO_IRQ bool def_bool y if SPARC64
arch/sparc/kernel/smp_64.c +43 -8
68 lines not on this page · raw message
Tejun Heo
maintainerPATCH 3/3 percpu: make embedding first chunk allocator check vmalloc space size
Embedding first chunk allocator maintains the distances between units in the vmalloc area and thus needs vmalloc space to be larger than the maximum distances between units; otherwise, it wouldn't be able to create any dynamic chunks. This patch makes the embedding first chunk allocator check vmalloc space size and if the maximum distance between units is larger than 75% of it, print warning and, if page mapping allocator is available, fail initialization so that the system falls back onto it.
This should work around percpu allocation failure problems on certain sparc64 configurations where distances between NUMA nodes are larger than the vmalloc area and makes percpu allocator more robust for future configurations.
--- mm/percpu.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
mm/percpu.c +18 -2
@@ -1786,7 +1786,7 @@ int __init pcpu_embed_first_chunk(size_tvoid *base = (void *)ULONG_MAX ;void **areas =NULL ;struct pcpu_alloc_info *ai; -size_t size_sum, areas_size; +size_t size_sum, areas_size, max_distance;int group, i, rc; ai = pcpu_build_alloc_info(reserved_size, dyn_size, atom_size,
@@ -1836,8 +1836,24 @@ int __init pcpu_embed_first_chunk(size_t }/* base address is now known, determine group base offsets */ -for (group =0 ; group < ai->nr_groups; group++) + max_distance =0 ; +for (group =0 ; group < ai->nr_groups; group++) { ai->groups[group].base_offset = areas[group] - base; + max_distance =max (max_distance, ai->groups[group].base_offset); + } + max_distance += ai->unit_size; + +/* warn if maximum distance is further than 75% of vmalloc space */ +if (max_distance > (VMALLOC_END -VMALLOC_START ) *3 /4 ) { +pr_warning ("PERCPU: max_distance=0x%lx too large for vmalloc " +"space 0x%lx\n" , + max_distance,VMALLOC_END -VMALLOC_START ); +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK +/* and fail if we have fallback */ + rc = -EINVAL ; + goto out_free; +#endif + } pr_info("PERCPU: Embedded %zu pages/cpu @%p s%zu r%zu d%zu u%zu\n", PFN_DOWN(size_sum), base, ai->static_size, ai->reserved_size,
Re: [PATCH 2/3] sparc64: implement page mapping percpu first chunk allocator
In reply to Tejun HeoFrom: Tejun Heo <[email protected]>
Date: Thu, 24 Sep 2009 21:56:32 +0900
Quoted 12 lines
Implement page mapping percpu first chunk allocator as a fallback to the embedding allocator. The next patch will make the embedding allocator check distances between units to determine whether it fits within the vmalloc area so that this fallback can be used on such cases.
sparc64 currently has relatively small vmalloc area which makes it impossible to create any dynamic chunks on certain configurations leading to percpu allocation failures. This and the next patch should allow those configurations to keep working until proper solution is found.
Quoted 3 lines
Signed-off-by Tejun Heo <[email protected]>---
David, can you please ack this after reviewing?
This looks fine to me:
David Miller
maintainerRe: [PATCH 2/3] sparc64: implement page mapping percpu first chunk allocator
In reply to Tejun HeoFrom: Tejun Heo <[email protected]>
Date: Thu, 24 Sep 2009 21:56:32 +0900
Quoted 12 lines
Implement page mapping percpu first chunk allocator as a fallback to the embedding allocator. The next patch will make the embedding allocator check distances between units to determine whether it fits within the vmalloc area so that this fallback can be used on such cases.
sparc64 currently has relatively small vmalloc area which makes it impossible to create any dynamic chunks on certain configurations leading to percpu allocation failures. This and the next patch should allow those configurations to keep working until proper solution is found.
Quoted 3 lines
Signed-off-by Tejun Heo <[email protected]>---
David, can you please ack this after reviewing?
Tejun I am testing out the following patch which will make these patches of your's basically unnecessary:
arch/sparc/include/asm/pgtable_64.h +2 -2
@@ -41,8 +41,8 @@#define LOW_OBP_ADDRESS _AC(0x00000000f0000000,UL)#define HI_OBP_ADDRESS _AC(0x0000000100000000,UL)#define VMALLOC_START _AC(0x0000000100000000,UL) -#define VMALLOC_END _AC(0x0000000200000000,UL) -#define VMEMMAP_BASE _AC(0x0000000200000000,UL) +#define VMALLOC_END _AC(0x0000010000000000,UL) +#define VMEMMAP_BASE _AC(0x0000010000000000,UL)#define vmemmap ((struct page *)VMEMMAP_BASE)
arch/sparc/kernel/ktlb.S +4 -4
@@ -280,8 +280,8 @@ kvmap_dtlb_nonlinear: #ifdef CONFIG_SPARSEMEM_VMEMMAP /* Do not use the TSB for vmemmap. */ - mov (VMEMMAP_BASE >> 24), %g5 - sllx %g5, 24, %g5 + mov (VMEMMAP_BASE >> 40), %g5 + sllx %g5, 40, %g5 cmp %g4,%g5 bgeu,pn %xcc, kvmap_vmemmap nop
@@ -293,8 +293,8 @@ kvmap_dtlb_tsbmiss: sethi %hi(MODULES_VADDR), %g5 cmp %g4, %g5 blu,pn %xcc, kvmap_dtlb_longpath - mov (VMALLOC_END >> 24), %g5 - sllx %g5, 24, %g5 + mov (VMALLOC_END >> 40), %g5 + sllx %g5, 40, %g5 cmp %g4, %g5 bgeu,pn %xcc, kvmap_dtlb_longpath nop
Tejun Heo
maintainerRe: [PATCH 2/3] sparc64: implement page mapping percpu first chunk allocator
Hello, David.
Quoted 15 lines · David Miller
From: Tejun Heo <[email protected]>
Date: Thu, 24 Sep 2009 21:56:32 +0900Quoted 12 lines
Implement page mapping percpu first chunk allocator as a fallback to the embedding allocator. The next patch will make the embedding allocator check distances between units to determine whether it fits within the vmalloc area so that this fallback can be used on such cases.
sparc64 currently has relatively small vmalloc area which makes it impossible to create any dynamic chunks on certain configurations leading to percpu allocation failures. This and the next patch should allow those configurations to keep working until proper solution is found.
Quoted 6 lines
Quoted 3 lines
Signed-off-by Tejun Heo <[email protected]>---
David, can you please ack this after reviewing?Tejun I am testing out the following patch which will make these patches of your's basically unnecessary:
Ah... great but unless you object, I think it would be better to push it out anyway just to make things a bit more robust and ease tracking and debugging when something goes wrong. The added code is small and ditched once boot is complete. What do you think?
Thanks.
Signature · 1 line
tejun
Re: [PATCH 2/3] sparc64: implement page mapping percpu first chunk allocator
From: Tejun Heo <[email protected]>
Date: Tue, 29 Sep 2009 09:14:58 +0900
Quoted 4 lines
Ah... great but unless you object, I think it would be better to push it out anyway just to make things a bit more robust and ease tracking and debugging when something goes wrong. The added code is small and ditched once boot is complete. What do you think?
That's fine with me: