percpu: convert ia64 to dynamic percpu and drop the old one, take#2
Subsystems (2)
Tejun Heo
maintainerPATCH 1/1 percpu: convert ia64 to dynamic percpu and drop the old one, take#2
Hello, all.
This is the second take of convert-ia64-to-dynamic-percpu patchset. Changes from the last take[L] are
* 0001 now updates ia64 to not define VMALLOC_END as a macro to vmalloc_end instead of disallowing vmalloc_end as a variable name as suggested by Christoph.
* 0002 added to initialize cpu maps early. This is necessary to get contig memory model working.
* 0004 updated so that dyn_size is calculated correctly for contig model.
This patchset contains the following five patches.
0001-ia64-don-t-alias-VMALLOC_END-to-vmalloc_end.patch 0002-ia64-initialize-cpu-maps-early.patch 0003-ia64-allocate-percpu-area-for-cpu0-like-percpu-areas.patch 0004-ia64-convert-to-dynamic-percpu-allocator.patch 0005-percpu-kill-legacy-percpu-allocator.patch
0001 is misc prep to avoid macro / local variable collision. 0002 makes ia64 arch code initialize cpu possible and present maps before memory initialization. 0003 makes ia64 allocate percpu area for cpu0 in the same way it does for other cpus. 0004 converts ia64 to dynamic percpu allocator and 0005 drops now unused legacy allocator.
Contig memory model was tested on a 16p Tiger4 machine. Discontig and sparse tested on 4-way SGI altix. ski seems to be happy with contig up/smp.
This patchset is available in the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git convert-ia64
The new commit ID is dcc91f19c6662b24f1f4e5878d773244f1079724 and it's on top of today's Linus 7fa07729e439a6184bd824746d06a49cca553f15.
Thanks.
arch/ia64/Kconfig | 3 arch/ia64/kernel/setup.c | 12 -- arch/ia64/kernel/vmlinux.lds.S | 11 +- arch/ia64/mm/contig.c | 87 ++++++++++++++++---- arch/ia64/mm/discontig.c | 120 +++++++++++++++++++++++++-- include/linux/percpu.h | 24 ----- kernel/module.c | 150 ---------------------------------- mm/Makefile | 4 mm/allocpercpu.c | 177 ----------------------------------------- mm/percpu.c | 2 mm/vmalloc.c | 16 +-- 11 files changed, 193 insertions(+), 413 deletions(-)
--
tejun
Tejun Heo
maintainerPATCH 1/5 ia64: don't alias VMALLOC_END to vmalloc_end
If CONFIG_VIRTUAL_MEM_MAP is enabled, ia64 defines macro VMALLOC_END as unsigned long variable vmalloc_end which is adjusted to prepare room for vmemmap. This becomes probnlematic if a local variables vmalloc_end is defined in some function (not very unlikely) and VMALLOC_END is used in the function - the function thinks its referencing the global VMALLOC_END value but would be referencing its own local vmalloc_end variable.
There's no reason VMALLOC_END should be a macro. Just define it as an unsigned long variable if CONFIG_VIRTUAL_MEM_MAP is set to avoid nasty surprises.
--- arch/ia64/include/asm/meminit.h | 2 +- arch/ia64/include/asm/pgtable.h | 3 +-- arch/ia64/mm/contig.c | 4 ++-- arch/ia64/mm/discontig.c | 4 ++-- arch/ia64/mm/init.c | 4 ++-- 5 files changed, 8 insertions(+), 9 deletions(-)
arch/ia64/include/asm/meminit.h +1 -1
@@ -61,7 +61,7 @@ extern int register_active_ranges(u64 start, u64 len, int nid);#ifdef CONFIG_VIRTUAL_MEM_MAP # define LARGE_GAP 0x40000000/* Use virtual mem map if hole is > than this */ -extern unsigned long vmalloc_end; +extern unsigned long VMALLOC_END ;extern struct page *vmem_map;extern int find_largest_hole (u64 start,u64 end,void *arg);extern int create_mem_map_page_table (u64 start,u64 end,void *arg);
arch/ia64/include/asm/pgtable.h +1 -2
@@ -228,8 +228,7 @@ ia64_phys_addr_valid (unsigned long addr)#define VMALLOC_START (RGN_BASE(RGN_GATE) + 0x200000000UL)#ifdef CONFIG_VIRTUAL_MEM_MAP # define VMALLOC_END_INIT (RGN_BASE(RGN_GATE) + (1UL << (4*PAGE_SHIFT - 9))) -# define VMALLOC_END vmalloc_end -extern unsigned long vmalloc_end; +extern unsigned long VMALLOC_END ;#else #if defined(CONFIG_SPARSEMEM ) && defined(CONFIG_SPARSEMEM_VMEMMAP )/* SPARSEMEM_VMEMMAP uses half of vmalloc... */
arch/ia64/mm/contig.c +2 -2
@@ -270,8 +270,8 @@ paging_init (void) map_size =PAGE_ALIGN (ALIGN (max_low_pfn,MAX_ORDER_NR_PAGES ) *sizeof (struct page )); - vmalloc_end -= map_size; - vmem_map = (struct page *) vmalloc_end; +VMALLOC_END -= map_size; + vmem_map = (struct page *)VMALLOC_END ;efi_memmap_walk (create_mem_map_page_table,NULL );/*
arch/ia64/mm/discontig.c +2 -2
@@ -666,9 +666,9 @@ void __init paging_init(void)sparse_init ();#ifdef CONFIG_VIRTUAL_MEM_MAP - vmalloc_end -=PAGE_ALIGN (ALIGN (max_low_pfn,MAX_ORDER_NR_PAGES ) * +VMALLOC_END -=PAGE_ALIGN (ALIGN (max_low_pfn,MAX_ORDER_NR_PAGES ) *sizeof (struct page )); - vmem_map = (struct page *) vmalloc_end; + vmem_map = (struct page *)VMALLOC_END ;efi_memmap_walk (create_mem_map_page_table,NULL );printk ("Virtual mem_map starts at 0x%p\n" , vmem_map);#endif
arch/ia64/mm/init.c +2 -2
@@ -44,8 +44,8 @@ extern void ia64_tlb_init (void);unsigned long MAX_DMA_ADDRESS =PAGE_OFFSET +0x100000000UL ;#ifdef CONFIG_VIRTUAL_MEM_MAP -unsigned long vmalloc_end =VMALLOC_END_INIT ; -EXPORT_SYMBOL (vmalloc_end); +unsigned long VMALLOC_END =VMALLOC_END_INIT ; +EXPORT_SYMBOL (VMALLOC_END );struct page *vmem_map;EXPORT_SYMBOL (vmem_map);#endif
Signature · 1 line
1.6.4.2
Tejun Heo
maintainerPATCH 2/5 ia64: initialize cpu maps early
In reply to Tejun HeoAll information necessary to initialize cpu possible and present maps are available once early_acpi_boot_init() is complete. Reorganize setup_arch() and acpi init functions such that,
* CPU information is printed after LAPIC entries are parsed in early_acpi_boot_init().
* smp_build_cpu_map() is called by setup_arch() instead of acpi functions.
* smp_build_cpu_map() is called once all CPU related information is available before memory is initialized.
This is primarily to allow find_memory() to use cpu maps but is also a general cleanup. Please note that with this change, the somewhat ad-hoc early_cpu_possible_map defined and used for NUMA configurations is probably unnecessary. Something to clean up another day.
--- arch/ia64/kernel/acpi.c | 33 +++++++++++++++------------------ arch/ia64/kernel/setup.c | 11 +++++------ 2 files changed, 20 insertions(+), 24 deletions(-)
arch/ia64/kernel/acpi.c +14 -17
@@ -702,11 +702,23 @@ int __init early_acpi_boot_init(void) printk(KERN_ERR PREFIX "Error parsing MADT - no LAPIC entries\n"); +#ifdef CONFIG_SMP +if (available_cpus =0 ) { +printk (KERN_INFO "ACPI: Found 0 CPUS; assuming 1\n" ); +printk (KERN_INFO "CPU 0 (0x%04x)" ,hard_smp_processor_id ()); + smp_boot_data.cpu_phys_id[available_cpus] +hard_smp_processor_id (); + available_cpus =1 ;/* We've got at least one of these, no? */ + } + smp_boot_data.cpu_count = available_cpus; +#endif +/* Make boot-up look pretty */ +printk (KERN_INFO "%d CPUs available, %d CPUs total\n" , available_cpus, + total_cpus); +return 0 ; } - -int __init acpi_boot_init (void ) {
@@ -769,18 +781,8 @@ int __init acpi_boot_init(void)if (acpi_table_parse (ACPI_SIG_FADT , acpi_parse_fadt)) printk(KERN_ERR PREFIX "Can't find FADT\n"); +#ifdef CONFIG_ACPI_NUMA #ifdef CONFIG_SMP - if (available_cpus = 0) { - printk(KERN_INFO "ACPI: Found 0 CPUS; assuming 1\n"); - printk(KERN_INFO "CPU 0 (0x%04x)", hard_smp_processor_id()); - smp_boot_data.cpu_phys_id[available_cpus] - hard_smp_processor_id(); - available_cpus = 1; /* We've got at least one of these, no? */ - } - smp_boot_data.cpu_count = available_cpus; - - smp_build_cpu_map(); -# ifdef CONFIG_ACPI_NUMA if (srat_num_cpus = 0) { int cpu, i = 1; for (cpu = 0; cpu < smp_boot_data.cpu_count; cpu++)
@@ -789,14 +791,9 @@ int __init acpi_boot_init(void) node_cpuid[i++].phys_id smp_boot_data.cpu_phys_id[cpu]; } -# endif #endif -#ifdef CONFIG_ACPI_NUMA build_cpu_to_node_map ();#endif -/* Make boot-up look pretty */ -printk (KERN_INFO "%d CPUs available, %d CPUs total\n" , available_cpus, - total_cpus);return 0 ; }
arch/ia64/kernel/setup.c +5 -6
@@ -566,19 +566,18 @@ setup_arch (char **cmdline_p)early_acpi_boot_init ();# ifdef CONFIG_ACPI_NUMA acpi_numa_init (); -#ifdef CONFIG_ACPI_HOTPLUG_CPU +# ifdef CONFIG_ACPI_HOTPLUG_CPU prefill_possible_map (); -#endif +# endif per_cpu_scan_finalize ((cpus_weight (early_cpu_possible_map) =0 ?32 :cpus_weight (early_cpu_possible_map)), additional_cpus >0 ? additional_cpus :0 );# endif -#else -# ifdef CONFIG_SMP -smp_build_cpu_map ();/* happens, e.g., with the Ski simulator */ -# endif #endif /* CONFIG_APCI_BOOT */ +#ifdef CONFIG_SMP +smp_build_cpu_map (); +#endif find_memory ();/* process SAL system table: */
Signature · 1 line
1.6.4.2
Tejun Heo
maintainerPATCH 3/5 ia64: allocate percpu area for cpu0 like percpu areas for other cpus
In reply to Tejun Heocpu0 used special percpu area reserved by the linker, __cpu0_per_cpu, which is set up early in boot by head.S. However, this doesn't guarantee that the area will be on the same node as cpu0 and the percpu area for cpu0 ends up very far away from percpu areas for other cpus which cause problems for congruent percpu allocator.
This patch makes percpu area initialization allocate percpu area for cpu0 like any other cpus and copy it from __cpu0_per_cpu which now resides in the __init area. This means that for cpu0, percpu area is first setup at __cpu0_per_cpu early by head.S and then moved to an area in the linear mapping during memory initialization and it's not allowed to take a pointer to percpu variables between head.S and memory initialization.
--- arch/ia64/kernel/vmlinux.lds.S | 11 +++++---- arch/ia64/mm/contig.c | 47 +++++++++++++++++++++++++-------------- arch/ia64/mm/discontig.c | 35 ++++++++++++++++++++--------- 3 files changed, 60 insertions(+), 33 deletions(-)
arch/ia64/kernel/vmlinux.lds.S +6 -5
@@ -166,6 +166,12 @@ SECTIONS } #endif +#ifdef CONFIG_SMP + . = ALIGN(PERCPU_PAGE_SIZE); + __cpu0_per_cpu = .; + . = . + PERCPU_PAGE_SIZE; /* cpu0 per-cpu space */ +#endif + . = ALIGN(PAGE_SIZE); __init_end = .;
@@ -198,11 +204,6 @@ SECTIONS data : { } :data .data : AT(ADDR(.data) - LOAD_OFFSET) { -#ifdef CONFIG_SMP - . = ALIGN(PERCPU_PAGE_SIZE); - __cpu0_per_cpu = .; - . = . + PERCPU_PAGE_SIZE; /* cpu0 per-cpu space */ -#endif INIT_TASK_DATA(PAGE_SIZE) CACHELINE_ALIGNED_DATA(SMP_CACHE_BYTES) READ_MOSTLY_DATA(SMP_CACHE_BYTES)
arch/ia64/mm/contig.c +27 -14
63 lines not on this page · raw message
arch/ia64/mm/discontig.c +24 -11
@@ -143,17 +143,30 @@ static void *per_cpu_node_setup(void *cpu_data, int node)int cpu;for_each_possible_early_cpu (cpu) { -if (cpu =0 ) { -void *cpu0_data = __cpu0_per_cpu; - __per_cpu_offset[cpu] = (char *)cpu0_data - - __per_cpu_start; - }else if (node = node_cpuid[cpu].nid) { -memcpy (__va (cpu_data), __phys_per_cpu_start, - __per_cpu_end - __per_cpu_start); - __per_cpu_offset[cpu] = (char *)__va (cpu_data) - - __per_cpu_start; - cpu_data +=PERCPU_PAGE_SIZE ; - } +void *src = cpu =0 ? __cpu0_per_cpu : __phys_per_cpu_start; + +if (node != node_cpuid[cpu].nid) +continue ; + +memcpy (__va (cpu_data), src, __per_cpu_end - __per_cpu_start); + __per_cpu_offset[cpu] = (char *)__va (cpu_data) - + __per_cpu_start; + +/* +* percpu area for cpu0 is moved from the __init area +* which is setup by head.S and used till this point. +* Update ar.k3. This move is ensures that percpu +* area for cpu0 is on the correct node and its +* virtual address isn't insanely far from other +* percpu areas which is important for congruent +* percpu allocator. +*/ +if (cpu =0 ) +ia64_set_kr (IA64_KR_PER_CPU_DATA , + (unsigned long )cpu_data - + (unsigned long )__per_cpu_start); + + cpu_data +=PERCPU_PAGE_SIZE ; }#endif return cpu_data;
Signature · 1 line
1.6.4.2
Tejun Heo
maintainerPATCH 4/5 ia64: convert to dynamic percpu allocator
In reply to Tejun HeoUnlike other archs, ia64 reserves space for percpu areas during early memory initialization. These areas occupy a contiguous region indexed by cpu number on contiguous memory model or are grouped by node on discontiguous memory model.
As allocation and initialization are done by the arch code, all that setup_per_cpu_areas() needs to do is communicating the determined layout to the percpu allocator. This patch implements setup_per_cpu_areas() for both contig and discontig memory models and drops HAVE_LEGACY_PER_CPU_AREA.
Please note that for contig model, the allocation itself is modified only to allocate for possible cpus instead of NR_CPUS. As dynamic percpu allocator can handle non-direct mapping, there's no reason to allocate memory for cpus which aren't possible.
--- arch/ia64/Kconfig | 3 -- arch/ia64/kernel/setup.c | 12 ------ arch/ia64/mm/contig.c | 58 ++++++++++++++++++++++++++++--- arch/ia64/mm/discontig.c | 85 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 138 insertions(+), 20 deletions(-)
arch/ia64/Kconfig +0 -3
@@ -89,9 +89,6 @@ config GENERIC_TIME_VSYSCALL bool default y -config HAVE_LEGACY_PER_CPU_AREA - def_bool y - config HAVE_SETUP_PER_CPU_AREA def_bool y
arch/ia64/kernel/setup.c +0 -12
@@ -855,18 +855,6 @@ identify_cpu (struct cpuinfo_ia64 *c) }/* -* In UP configuration, setup_per_cpu_areas() is defined in -* include/linux/percpu.h -*/ -#ifdef CONFIG_SMP -void __init -setup_per_cpu_areas (void ) -{ -/* start_kernel() requires this... */ -} -#endif - -/* * Do the following calculations: * * 1. the max. cache line size.
arch/ia64/mm/contig.c +53 -5
73 lines not on this page · raw message
arch/ia64/mm/discontig.c +85 -0
91 lines not on this page · raw message
Signature · 1 line
1.6.4.2
Tejun Heo
maintainerPATCH 5/5 percpu: kill legacy percpu allocator
In reply to Tejun HeoWith ia64 converted, there's no arch left which still uses legacy percpu allocator. Kill it.
--- include/linux/percpu.h | 24 ------- kernel/module.c | 150 ---------------------------------------- mm/Makefile | 4 - mm/allocpercpu.c | 177 ------------------------------------------------ mm/percpu.c | 2 - 5 files changed, 0 insertions(+), 357 deletions(-) delete mode 100644 mm/allocpercpu.c
include/linux/percpu.h +0 -24
@@ -34,8 +34,6 @@#ifdef CONFIG_SMP -#ifndef CONFIG_HAVE_LEGACY_PER_CPU_AREA -/* minimum unit size, also is the maximum supported allocation size */ #define PCPU_MIN_UNIT_SIZE PFN_ALIGN(64 << 10)
@@ -130,28 +128,6 @@ extern int __init pcpu_page_first_chunk(size_t reserved_size,#define per_cpu_ptr (ptr, cpu) SHIFT_PERCPU_PTR((ptr), per_cpu_offset((cpu)))extern void *__alloc_reserved_percpu (size_t size,size_t align); - -#else /* CONFIG_HAVE_LEGACY_PER_CPU_AREA */ - -struct percpu_data { -void *ptrs[1 ]; -}; - -/* pointer disguising messes up the kmemleak objects tracking */ -#ifndef CONFIG_DEBUG_KMEMLEAK -#define __percpu_disguise (pdata) (struct percpu_data *)~(unsigned long)(pdata) -#else -#define __percpu_disguise (pdata) (struct percpu_data *)(pdata) -#endif - -#define per_cpu_ptr (ptr, cpu) \ -({ \ - struct percpu_data *__p = __percpu_disguise(ptr); \ - (__typeof__(ptr))__p->ptrs[(cpu)]; \ -}) - -#endif /* CONFIG_HAVE_LEGACY_PER_CPU_AREA */ -extern void *__alloc_percpu (size_t size,size_t align);extern void free_percpu (void *__pdata);
kernel/module.c +0 -150
162 lines not on this page · raw message
mm/Makefile +0 -4
@@ -34,11 +34,7 @@ obj-$(CONFIG_FAILSLAB) += failslab.o obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o obj-$(CONFIG_FS_XIP) += filemap_xip.o obj-$(CONFIG_MIGRATION) += migrate.o -ifndef CONFIG_HAVE_LEGACY_PER_CPU_AREA obj-$(CONFIG_SMP) += percpu.o -else -obj-$(CONFIG_SMP) += allocpercpu.o -endif obj-$(CONFIG_QUICKLIST) += quicklist.o obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o page_cgroup.o obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o
mm/allocpercpu.c deleted +0 -177
177 lines not on this page · raw message
mm/percpu.c +0 -2
@@ -46,8 +46,6 @@* * To use this allocator, arch code should do the followings. * -* - drop CONFIG_HAVE_LEGACY_PER_CPU_AREA -* * - define __addr_to_pcpu_ptr() and __pcpu_ptr_to_addr() to translate * regular address to percpu pointer and back if they need to be * different from the default
Signature · 1 line
1.6.4.2
Re: [PATCH 5/5] percpu: kill legacy percpu allocator
Quoted 7 lines · Tejun Heo
With ia64 converted, there's no arch left which still uses legacy percpu allocator. Kill it.
Signed-off-by Tejun Heo <[email protected]>Cc Ingo Molnar <[email protected]>Cc Rusty Russell <[email protected]>Cc Christoph Lameter <[email protected]>
Thanks!
Rusty.
Quoted 33 lines · Tejun Heo
Hello, all.
This is the second take of convert-ia64-to-dynamic-percpu patchset. Changes from the last take[L] are
* 0001 now updates ia64 to not define
VMALLOC_ENDas a macro tovmalloc_endinstead of disallowingvmalloc_endas a variable name as suggested by Christoph.* 0002 added to initialize cpu maps early. This is necessary to get contig memory model working.
* 0004 updated so that
dyn_sizeis calculated correctly for contig model.This patchset contains the following five patches.
0001-ia64-don-t-alias-VMALLOC_END-to-vmalloc_end.patch 0002-ia64-initialize-cpu-maps-early.patch 0003-ia64-allocate-percpu-area-for-cpu0-like-percpu-areas.patch 0004-ia64-convert-to-dynamic-percpu-allocator.patch 0005-percpu-kill-legacy-percpu-allocator.patch
0001 is misc prep to avoid macro / local variable collision. 0002 makes ia64 arch code initialize cpu possible and present maps before memory initialization. 0003 makes ia64 allocate percpu area for cpu0 in the same way it does for other cpus. 0004 converts ia64 to dynamic percpu allocator and 0005 drops now unused legacy allocator.
Contig memory model was tested on a 16p Tiger4 machine. Discontig and sparse tested on 4-way SGI altix. ski seems to be happy with contig up/smp.
Quoted 6 lines
This patchset is available in the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.gitconvert-ia64The new commit ID is
dcc91f19c6662b24f1f4e5878d773244f1079724and it's on top of today's Linus7fa07729e439a6184bd824746d06a49cca553f15.
Tony, can you please ack ia64 part?
Thanks.
Signature · 1 line
tejun
PATCH 3/5 ia64: allocate percpu area for cpu0 like percpu areas for other cpus
In reply to Tejun Heocpu0 used special percpu area reserved by the linker, __cpu0_per_cpu, which is set up early in boot by head.S. However, this doesn't guarantee that the area will be on the same node as cpu0 and the percpu area for cpu0 ends up very far away from percpu areas for other cpus which cause problems for congruent percpu allocator.
This patch makes percpu area initialization allocate percpu area for cpu0 like any other cpus and copy it from __cpu0_per_cpu which now resides in the __init area. This means that for cpu0, percpu area is first setup at __cpu0_per_cpu early by head.S and then moved to an area in the linear mapping during memory initialization and it's not allowed to take a pointer to percpu variables between head.S and memory initialization.
---
Tony Luck didn't receive this one and couldn't find it in archives either. Repost. Unchanged from the original posting.
arch/ia64/kernel/vmlinux.lds.S | 11 +++++---- arch/ia64/mm/contig.c | 47 +++++++++++++++++++++++++-------------- arch/ia64/mm/discontig.c | 35 ++++++++++++++++++++--------- 3 files changed, 60 insertions(+), 33 deletions(-)
arch/ia64/kernel/vmlinux.lds.S +6 -5
@@ -166,6 +166,12 @@ SECTIONS } #endif +#ifdef CONFIG_SMP + . = ALIGN(PERCPU_PAGE_SIZE); + __cpu0_per_cpu = .; + . = . + PERCPU_PAGE_SIZE; /* cpu0 per-cpu space */ +#endif + . = ALIGN(PAGE_SIZE); __init_end = .;
@@ -198,11 +204,6 @@ SECTIONS data : { } :data .data : AT(ADDR(.data) - LOAD_OFFSET) { -#ifdef CONFIG_SMP - . = ALIGN(PERCPU_PAGE_SIZE); - __cpu0_per_cpu = .; - . = . + PERCPU_PAGE_SIZE; /* cpu0 per-cpu space */ -#endif INIT_TASK_DATA(PAGE_SIZE) CACHELINE_ALIGNED_DATA(SMP_CACHE_BYTES) READ_MOSTLY_DATA(SMP_CACHE_BYTES)
arch/ia64/mm/contig.c +27 -14
63 lines not on this page · raw message
arch/ia64/mm/discontig.c +24 -11
@@ -143,17 +143,30 @@ static void *per_cpu_node_setup(void *cpu_data, int node)int cpu;for_each_possible_early_cpu (cpu) { -if (cpu =0 ) { -void *cpu0_data = __cpu0_per_cpu; - __per_cpu_offset[cpu] = (char *)cpu0_data - - __per_cpu_start; - }else if (node = node_cpuid[cpu].nid) { -memcpy (__va (cpu_data), __phys_per_cpu_start, - __per_cpu_end - __per_cpu_start); - __per_cpu_offset[cpu] = (char *)__va (cpu_data) - - __per_cpu_start; - cpu_data +=PERCPU_PAGE_SIZE ; - } +void *src = cpu =0 ? __cpu0_per_cpu : __phys_per_cpu_start; + +if (node != node_cpuid[cpu].nid) +continue ; + +memcpy (__va (cpu_data), src, __per_cpu_end - __per_cpu_start); + __per_cpu_offset[cpu] = (char *)__va (cpu_data) - + __per_cpu_start; + +/* +* percpu area for cpu0 is moved from the __init area +* which is setup by head.S and used till this point. +* Update ar.k3. This move is ensures that percpu +* area for cpu0 is on the correct node and its +* virtual address isn't insanely far from other +* percpu areas which is important for congruent +* percpu allocator. +*/ +if (cpu =0 ) +ia64_set_kr (IA64_KR_PER_CPU_DATA , + (unsigned long )cpu_data - + (unsigned long )__per_cpu_start); + + cpu_data +=PERCPU_PAGE_SIZE ; }#endif return cpu_data;
Signature · 1 line
1.6.4.2
Quoted 1 line
Tony, can you please ack ia64 part?
Tejun,
Ok. The new patchset builda and boots for both the contig.c and discontig.c versions of the kernel.
Acked-by; Tony Luck <[email protected]>
Great. Ill post a new rev of the per cpu ops patchset.
Tony: Could we use a global register for the per cpu address? That would make IA64 work similar to sparc.
Quoted 2 lines
Tony: Could we use a global register for the per cpu address? That would make IA64 work similar to sparc.
Would that be a useful trade of resources for convenience? We've already hard-wired r13 for "current". Grabbing another one would require fixing (since user code will have clobbered it). Possibly re-working any existing code that already uses whatever register you choose.
How would that compare with using [r13]?
We might have a krN free ... but kregs are a lot slower than real registers.
-Tony
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Tony" = Tony Luck <Luck> writes:
Tony: Could we use a global register for the per cpu address? That would make IA64 work similar to sparc.
Tony> Would that be a useful trade of resources for convenience? Tony> We've already hard-wired r13 for "current". Grabbing another one Tony> would require fixing (since user code will have clobbered it). Tony> Possibly re-working any existing code that already uses whatever Tony> register you choose.
r3, r4 and r5 are currently unused by the kernel, and unused
by GCC and ICC. Only hand-written assembler and weird compilers use
those registers(and my virtual-machine monitor :-(). If you wanted to experiment, that'd be a starting place.
I'm not sure of the advantage though -- TLB mapping is relatively cheap, and we're no longer hard-wiring the translation register.
You';d have to do somne careful benchmarking on a wide variety of workloads and machines to get a definitive answer.
---
Dr Peter Chubb www.nicta.com.au peter DOT chubb AT nicta.com.au
http://www.ertos.nicta.com.au ERTOS within National ICT Australia
From Imagination to Impact Imagining the (ICT) Future
Quoted 7 lines · Peter Chubb
r3, r4 and r5 are currently unused by the kernel, and unused
by GCC and ICC. Only hand-written assembler and weird compilers use
those registers(and my virtual-machine monitor :-(). If you wanted to experiment, that'd be a starting place.I'm not sure of the advantage though -- TLB mapping is relatively cheap, and we're no longer hard-wiring the translation register.
Dynamic and static per cpu variables could use relative access to that register. This would reduce code size, avoid the use of a TLB entry.
Quoted 2 lines
You';d have to do somne careful benchmarking on a wide variety of workloads and machines to get a definitive answer.
I have some patches here that make heavy use of dynamic percpu allocations in the allocators to optimize the alloc / free paths.
Quoted 4 lines · Tejun Heo
Hello, all.
This is the second take of convert-ia64-to-dynamic-percpu patchset. Changes from the last take[L] are
Patchset pushed out for linux-next with Rusty and Tony's acks added.
Thanks.
Signature · 1 line
tejun