linux-next: build failure after merge of the final tree (slab tree related)
Subsystems (6)
Hi all,
After merging the final tree, today's linux-next build (powerpc allnoconfig) failed like this:
mm/slub.c: In function 'alloc_kmem_cache_cpus':mm/slub.c:2094: error: 'PERCPU_DYNAMIC_EARLY_SIZE' undeclared (first use in this function) mm/slub.c:2094: error: (Each undeclared identifier is reported only once mm/slub.c:2094: error: for each function it appears in.)mm/slub.c:2094: error: bit-field '<anonymous>' width not an integer constant
Caused by commit e18d65f0500b95d8724b17d8ea9f1116cf390bbe ("slub: Remove static kmem_cache_cpu array for boot"). PERCPU_DYNAMIC_EARLY_SIZE is only defined for SMP (and only in linux/percpu.h which is not explicitly included). This build does not have CONFIG_SMP set.
I have used the version of the slab tree from next-20100823 for today.
Signature · 3 lines
Cheers, Stephen Rothwell [email protected] http://www.canb.auug.org.au/~sfr/
Quoted 11 lines · Stephen Rothwell
Hi all,
After merging the final tree, today's linux-next build (powerpc allnoconfig) failed like this:
mm/slub.c: In function 'alloc_kmem_cache_cpus':mm/slub.c:2094: error: 'PERCPU_DYNAMIC_EARLY_SIZE' undeclared (first use in this function)mm/slub.c:2094: error: (Each undeclared identifier is reported only oncemm/slub.c:2094: error: for each function it appears in.)mm/slub.c:2094: error: bit-field '<anonymous>' width not an integer constant
Quoted 6 lines
Caused by commit
e18d65f0500b95d8724b17d8ea9f1116cf390bbe("slub: Remove statickmem_cache_cpuarray for boot").PERCPU_DYNAMIC_EARLY_SIZEis only defined for SMP (and only inlinux/percpu.hwhich is not explicitly included). This build does not haveCONFIG_SMPset.I have used the version of the slab tree from next-20100823 for today.
Thanks for the report. The problem should be fixed by this commit:
Quoted 1 line · Pekka Enberg
Thanks for the report. The problem should be fixed by this commit:
Its not that easy. __alloc_percpu falls back to kzalloc() on UP and this can result in unique bootstrap problems with UP since the bootstrap array is no longer there. Does the UP kernel boot?
Why did this ever build on my UP configuration tests? Hmmm... I only tested x86_64 UP. This was 32 bit UP I guess.
Quoted 7 lines · Christoph Lameter
Quoted 1 line · Pekka Enberg
Thanks for the report. The problem should be fixed by this commit:
Its not that easy.
__alloc_percpufalls back tokzalloc()on UP and this can result in unique bootstrap problems with UP since the bootstrap array is no longer there. Does the UP kernel boot?
No, I get this under kvm:
[ 0.000000] Linux version 2.6.36-rc2+ (penberg@tiger) (gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9) ) #103 Tue Aug 24 21:27:28 EEST 2010 [ 0.000000] Command line: notsc nolapic nosmp noacpi pci=conf1 earlyprintk=ttyS0,keep [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) [ 0.000000] BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) [ 0.000000] BIOS-e820: 00000000000f0000 - 00000000000fffff (reserved) [ 0.000000] BIOS-e820: 0000000000100000 - 0000000004000000 (usable) [ 0.000000] console [earlyser0] enabled [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] No AGP bridge found [ 0.000000] last_pfn = 0x4000 max_arch_pfn = 0x400000000 [ 0.000000] x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106 [ 0.000000] CPU MTRRs all blank - virtualized system. [ 0.000000] Scanning 1 areas for low memory corruption [ 0.000000] modified physical RAM map: [ 0.000000] modified: 0000000000000000 - 0000000000010000 (reserved) [ 0.000000] modified: 0000000000010000 - 000000000009fc00 (usable) [ 0.000000] modified: 000000000009fc00 - 00000000000a0000 (reserved) [ 0.000000] modified: 00000000000f0000 - 00000000000fffff (reserved) [ 0.000000] modified: 0000000000100000 - 0000000004000000 (usable) [ 0.000000] init_memory_mapping: 0000000000000000-0000000004000000 [ 0.000000] ACPI Error: A valid RSDP was not found (20100702/tbxfroot-219) [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: cpu 0, msr 0:1945621, boot clock [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] DMA32 0x00001000 -> 0x00100000 [ 0.000000] Normal empty [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x0000009f [ 0.000000] 0: 0x00000100 -> 0x00004000 [ 0.000000] No local APIC present [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000 [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000 [ 0.000000] PM: Registered nosave memory: 00000000000f0000 - 00000000000ff000 [ 0.000000] PM: Registered nosave memory: 00000000000ff000 - 0000000000100000 [ 0.000000] Allocating PCI resources starting at 4000000 (gap: 4000000:fc000000) [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 16047 [ 0.000000] Kernel command line: notsc nolapic nosmp noacpi pci=conf1 earlyprintk=ttyS0,keep [ 0.000000] notsc: Kernel compiled with CONFIG_X86_TSC, cannot disable TSC completely. [ 0.000000] PID hash table entries: 256 (order: -1, 2048 bytes) [ 0.000000] Dentry cache hash table entries: 8192 (order: 4, 65536 bytes) [ 0.000000] Inode-cache hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] Checking aperture... [ 0.000000] No AGP bridge found [ 0.000000] Subtract (23 early reservations) [ 0.000000] #0 [0001000000 - 0001b751a0] TEXT DATA BSS [ 0.000000] #1 [000009f000 - 0000100000] BIOS reserved [ 0.000000] #2 [0000010000 - 0000012000] TRAMPOLINE [ 0.000000] #3 [0000012000 - 0000016000] ACPI WAKEUP [ 0.000000] #4 [0001b751c0 - 0001b761c0] BOOTMEM [ 0.000000] #5 [0001f761c0 - 0001f761d8] BOOTMEM [ 0.000000] #6 [0002377000 - 0002378000] BOOTMEM [ 0.000000] #7 [0002378000 - 0002379000] BOOTMEM [ 0.000000] #8 [0002400000 - 0002600000] MEMMAP 0 [ 0.000000] #9 [0001b761c0 - 0001b762c0] BOOTMEM [ 0.000000] #10 [0001b762c0 - 0001b766c0] BOOTMEM [ 0.000000] #11 [0001b77000 - 0001b78000] BOOTMEM [ 0.000000] #12 [0001b766c0 - 0001b767d8] BOOTMEM [ 0.000000] #13 [0001b76800 - 0001b76868] BOOTMEM [ 0.000000] #14 [0001b76880 - 0001b768e8] BOOTMEM [ 0.000000] #15 [0001b76900 - 0001b76968] BOOTMEM [ 0.000000] #16 [0001b76980 - 0001b769e8] BOOTMEM [ 0.000000] #17 [0001b76a00 - 0001b76a20] BOOTMEM [ 0.000000] #18 [0001b76a40 - 0001b76a7d] BOOTMEM [ 0.000000] #19 [0001b76a80 - 0001b76abd] BOOTMEM [ 0.000000] #20 [0001b78000 - 0001b78800] BOOTMEM [ 0.000000] #21 [0001b78800 - 0001b88800] BOOTMEM [ 0.000000] #22 [0001b88800 - 0001b90800] BOOTMEM [ 0.000000] Memory: 51156k/65536k available (6550k kernel code, 452k absent, 13928k reserved, 3761k data, 764k init) [ 0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache size=232 realsize=256 order=0 offset=0 flags=42000 [ 0.000000] [ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc2+ #103 [ 0.000000] Call Trace: [ 0.000000] [<ffffffff81659a35>] panic+0x91/0x19b [ 0.000000] [<ffffffff81126e59>] kmem_cache_open+0x1e9/0x200 [ 0.000000] [<ffffffff81a3141a>] kmem_cache_init+0x57/0x300 [ 0.000000] [<ffffffff81a14a97>] start_kernel+0x1ca/0x396 [ 0.000000] [<ffffffff81a14325>] x86_64_start_reservations+0x12c/0x130 [ 0.000000] [<ffffffff81a14423>] x86_64_start_kernel+0xfa/0x109
Quoted 2 lines
Why did this ever build on my UP configuration tests? Hmmm... I only tested
x86_64UP. This was 32 bit UP I guess.
I could reproduce the build problem with x86-64 and !CONFIG_SMP so it's not related to 32-bit.
Quoted 5 lines · Pekka Enberg
Quoted 4 lines · Christoph Lameter
Quoted 1 line · Pekka Enberg
Thanks for the report. The problem should be fixed by this commit:
Quoted 8 lines
Quoted 3 lines
Its not that easy.
__alloc_percpufalls back tokzalloc()on UP and this can result in unique bootstrap problems with UP since the bootstrap array is no longer there. Does the UP kernel boot?No, I get this under kvm:
[ 0.000000] Kernel panic - not syncing: Cannot create slab kmem_cachesize=232 realsize=256 order=0 offset=0 flags=42000
alloc per cpu result in kmalloc which fails.
Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?
Tejun Heo
maintainerHello,
Quoted 7 lines · Christoph Lameter
Quoted 2 lines
[ 0.000000] Kernel panic - not syncing: Cannot create slab kmem_cachesize=232 realsize=256 order=0 offset=0 flags=42000
alloc per cpu result in kmalloc which fails.
Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?
Eh... nasty. Maybe we can create a alloc_percpu_early() function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?
Signature · 1 line
tejun
Quoted 6 lines · Tejun Heo
Hello,
Quoted 3 lines · Christoph Lameter
Quoted 2 lines
[ 0.000000] Kernel panic - not syncing: Cannot create slab kmem_cachesize=232 realsize=256 order=0 offset=0 flags=42000
Quoted 8 lines
Quoted 4 lines
alloc per cpu result in kmalloc which fails.
Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?
Eh... nasty. Maybe we can create a
alloc_percpu_early()function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?
Yeah, I was thinking about that too.
Pekka
Quoted 9 lines · Pekka Enberg
Quoted 8 lines · Tejun Heo
Hello,
Quoted 5 lines · Christoph Lameter
Quoted 2 lines
[ 0.000000] Kernel panic - not syncing: Cannot create slab kmem_cachesize=232 realsize=256 order=0 offset=0 flags=42000
alloc per cpu result in kmalloc which fails.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?
Eh... nasty. Maybe we can create a
alloc_percpu_early()function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?Yeah, I was thinking about that too.
Another solution is to allocate an order 1 (compound) page for each early cache. Then resize the kmalloc array after everything is up. kfree() will then redirect to the page allocator.
But this is a slab allocator specific solution.
We now have the situation that alloc_percpu can only be used on early boot on SMP machines. A general solution would be better I think.
If the early alloc_percpu stuff would work consistently then it could also be used to avoid the boot_pageset in the page allocator f.e.
Can we just get rid of the special UP case and just run the percpu subsystem even for UP?
Tejun Heo
maintainerHello,
Quoted 2 lines · Christoph Lameter
Can we just get rid of the special UP case and just run the percpu subsystem even for UP?
Yeah, maybe. Then we also can guarantee that percpu allocator always honors alignment (which wq code currently requires and papers over with similarly ugly workaround). It would add a mostly redundant allocator code tho. I'll look into how easily it can be done.
Thanks.
Signature · 1 line
tejun
Tejun Heo
maintainerPATCH 1/3 vmalloc: pcpu_get/free_vm_areas() aren't needed on UP
These functions are used only by percpu memory allocator on SMP. Don't build them on UP.
---
So, something like these three patches.
Nick, can I route this one through percpu tree?
Thanks.
include/linux/vmalloc.h | 2 ++ mm/vmalloc.c | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-)
include/linux/vmalloc.h +2 -0
@@ -115,10 +115,12 @@ extern rwlock_t vmlist_lock;extern struct vm_struct *vmlist;extern __init void vm_area_register_early (struct vm_struct *vm,size_t align); +#ifdef CONFIG_SMP struct vm_struct **pcpu_get_vm_areas (const unsigned long *offsets,const size_t *sizes,int nr_vms,size_t align,gfp_t gfp_mask);void pcpu_free_vm_areas (struct vm_struct **vms,int nr_vms); +#endif #endif /* _LINUX_VMALLOC_H */
mm/vmalloc.c +2 -0
@@ -2052,6 +2052,7 @@ void free_vm_area(struct vm_struct *area) }EXPORT_SYMBOL_GPL (free_vm_area); +#ifdef CONFIG_SMP static struct vmap_area *node_to_va (struct rb_node *n) { return n ? rb_entry(n, struct vmap_area, rb_node) : NULL;
@@ -2332,6 +2333,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)free_vm_area (vms[i]); kfree(vms); } +#endif /* CONFIG_SMP */ #ifdef CONFIG_PROC_FS static void *s_start (struct seq_file *m,loff_t *pos)
Signature · 1 line
1.7.1
Re: [PATCH 1/3] vmalloc: pcpu_get/free_vm_areas() aren't needed on UP
Tejun Heo
maintainerPATCH 2/3 percpu: reduce PCPU_MIN_UNIT_SIZE to 32k
In reply to Tejun HeoIn preparation of enabling percpu allocator for UP, reduce PCPU_MIN_UNIT_SIZE to 32k. On UP, the first chunk doesn't have to include static percpu variables and chunk size can be smaller which is important as UP percpu allocator will use contiguous kernel memory to populate chunks.
PCPU_MIN_UNIT_SIZE also determines the maximum supported allocation size but 32k should still be enough.
--- include/linux/percpu.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
include/linux/percpu.h +1 -1
@@ -42,7 +42,7 @@#ifdef CONFIG_SMP /* minimum unit size, also is the maximum supported allocation size */ -#define PCPU_MIN_UNIT_SIZE PFN_ALIGN(64 << 10) +#define PCPU_MIN_UNIT_SIZE PFN_ALIGN(32 << 10)/* * Percpu allocator can serve percpu allocations before slab is
Signature · 1 line
1.7.1
Re: [PATCH 2/3] percpu: reduce PCPU_MIN_UNIT_SIZE to 32k
Tejun Heo
maintainerPATCH 3/3 percpu: use percpu allocator on UP too
In reply to Tejun HeoOn UP, percpu allocations were redirected to kmalloc. This has the following problems.
* For certain amount of allocations (determined by PERCPU_DYNAMIC_EARLY_SLOTS and PERCPU_DYNAMIC_EARLY_SIZE), percpu allocator can be used before the usual kernel memory allocator is brought online. On SMP, this is used to initialize the kernel memory allocator.
* percpu allocator honors alignment upto PAGE_SIZE but kmalloc() doesn't. For example, workqueue makes use of larger alignments for cpu_workqueues.
Currently, users of percpu allocators need to handle UP differently, which is somewhat fragile and ugly. Other than small amount of memory, there isn't much to lose by enabling percpu allocator on UP. It can simply use kernel memory based chunk allocation which was added for SMP archs w/o MMUs.
This patch removes mm/percpu_up.c, builds mm/percpu.c on UP too and makes UP build use percpu-km. As percpu addresses and kernel addresses are always identity mapped and static percpu variables don't need any special treatment, nothing is arch dependent and mm/percpu.c implements generic setup_per_cpu_areas() for UP.
--- include/linux/percpu.h | 29 ++++------------------- mm/Kconfig | 8 ++++++ mm/Makefile | 7 +---- mm/percpu-km.c | 2 +- mm/percpu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++--- mm/percpu_up.c | 30 ------------------------ 6 files changed, 71 insertions(+), 65 deletions(-) delete mode 100644 mm/percpu_up.c
include/linux/percpu.h +5 -24
@@ -39,8 +39,6 @@preempt_enable (); \ }while (0 ) -#ifdef CONFIG_SMP - /* minimum unit size, also is the maximum supported allocation size */#define PCPU_MIN_UNIT_SIZE PFN_ALIGN(32 << 10)
@@ -137,37 +135,20 @@ extern int __init pcpu_page_first_chunk(size_t reserved_size,* dynamically allocated. Non-atomic access to the current CPU's * version should probably be combined with get_cpu()/put_cpu(). */ +#ifdef CONFIG_SMP #define per_cpu_ptr (ptr, cpu) SHIFT_PERCPU_PTR((ptr), per_cpu_offset((cpu))) +#else +#define per_cpu_ptr (ptr, cpu) ({ (void)(cpu); VERIFY_PERCPU_PTR((ptr)); }) +#endif extern void __percpu *__alloc_reserved_percpu (size_t size,size_t align);extern bool is_kernel_percpu_address (unsigned long addr); -#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA +#if !defined(CONFIG_SMP ) || !defined(CONFIG_HAVE_SETUP_PER_CPU_AREA )extern void __init setup_per_cpu_areas (void );#endif extern void __init percpu_init_late (void ); -#else /* CONFIG_SMP */ - -#define per_cpu_ptr (ptr, cpu) ({ (void)(cpu); VERIFY_PERCPU_PTR((ptr)); }) - -/* can't distinguish from other static vars, always false */ -static inline bool is_kernel_percpu_address (unsigned long addr) -{ -return false; -} - -static inline void __init setup_per_cpu_areas (void ) { } - -static inline void __init percpu_init_late (void ) { } - -static inline void *pcpu_lpage_remapped (void *kaddr) -{ -return NULL ; -} - -#endif /* CONFIG_SMP */ -extern void __percpu *__alloc_percpu (size_t size,size_t align);extern void free_percpu (void __percpu *__pdata);extern phys_addr_t per_cpu_ptr_to_phys (void *addr);
mm/Kconfig +8 -0
@@ -301,3 +301,11 @@ config NOMMU_INITIAL_TRIM_EXCESS of 1 says that all excess pages should be trimmed. See Documentation/nommu-mmap.txt for more information. + +# +# UP and nommu archs use km based percpu allocator +# +config NEED_PER_CPU_KM + depends on !SMP + bool + default y
mm/Makefile +1 -6
@@ -11,7 +11,7 @@ obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page_alloc.o page-writeback.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \ - page_isolation.o mm_init.o mmu_context.o \ + page_isolation.o mm_init.o mmu_context.o percpu.o \ $(mmu-y) obj-y += init-mm.o
@@ -36,11 +36,6 @@ obj-$(CONFIG_FAILSLAB) += failslab.o obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o obj-$(CONFIG_FS_XIP) += filemap_xip.o obj-$(CONFIG_MIGRATION) += migrate.o -ifdef CONFIG_SMP -obj-y += percpu.o -else -obj-y += percpu_up.o -endif obj-$(CONFIG_QUICKLIST) += quicklist.o obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o page_cgroup.o obj-$(CONFIG_MEMORY_FAILURE) += memory-failure.o
mm/percpu-km.c +1 -1
@@ -27,7 +27,7 @@* chunk size is not aligned. percpu-km code will whine about it. */ -#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK +#if defined(CONFIG_SMP ) && defined(CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK )#error "contiguous percpu allocation is incompatible with paged first chunk"#endif
mm/percpu.c +56 -4
133 lines not on this page · raw message
mm/percpu_up.c deleted +0 -30
@@ -1,30 +0,0 @@ -/* -* mm/percpu_up.c - dummy percpu memory allocator implementation for UP -*/ - -#include <linux/module.h> -#include <linux/percpu.h> -#include <linux/slab.h> - -void __percpu *__alloc_percpu (size_t size,size_t align) -{ -/* -* Can't easily make larger alignment work with kmalloc. WARN -* on it. Larger alignment should only be used for module -* percpu sections on SMP for which this path isn't used. -*/ -WARN_ON_ONCE (align >SMP_CACHE_BYTES ); -return (void __percpu __force *)kzalloc (size,GFP_KERNEL ); -} -EXPORT_SYMBOL_GPL (__alloc_percpu); - -void free_percpu (void __percpu *p) -{ -kfree (this_cpu_ptr (p)); -} -EXPORT_SYMBOL_GPL (free_percpu); - -phys_addr_t per_cpu_ptr_to_phys (void *addr) -{ -return __pa (addr); -}
Signature · 1 line
1.7.1
Re: [PATCH 3/3] percpu: use percpu allocator on UP too
As much as I can review looks okay.
Re: [PATCH 3/3] percpu: use percpu allocator on UP too
In reply to Tejun HeoQuoted 25 lines · Tejun Heo
On UP, percpu allocations were redirected to kmalloc. This has the following problems.
* For certain amount of allocations (determined by
PERCPU_DYNAMIC_EARLY_SLOTSandPERCPU_DYNAMIC_EARLY_SIZE), percpu allocator can be used before the usual kernel memory allocator is brought online. On SMP, this is used to initialize the kernel memory allocator.* percpu allocator honors alignment upto
PAGE_SIZEbutkmalloc()doesn't. For example, workqueue makes use of larger alignments forcpu_workqueues.Currently, users of percpu allocators need to handle UP differently, which is somewhat fragile and ugly. Other than small amount of memory, there isn't much to lose by enabling percpu allocator on UP. It can simply use kernel memory based chunk allocation which was added for SMP archs w/o MMUs.
This patch removes
mm/percpu_up.c, buildsmm/percpu.con UP too and makes UP build use percpu-km. As percpu addresses and kernel addresses are always identity mapped and static percpu variables don't need any special treatment, nothing is arch dependent andmm/percpu.cimplements genericsetup_per_cpu_areas()for UP.
Quoted 3 lines
Signed-off-by Tejun Heo<[email protected]>Cc Christoph Lameter<[email protected]>Cc Pekka Enberg<[email protected]>
Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?
Pekka
Tejun Heo
maintainerRe: [PATCH 3/3] percpu: use percpu allocator on UP too
Hello,
Quoted 2 lines · Pekka Enberg
Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?
I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.
Thanks.
Signature · 1 line
tejun
Tejun Heo
maintainerRe: [PATCH 3/3] percpu: use percpu allocator on UP too
Quoted 8 lines · Tejun Heo
Hello,
Quoted 2 lines · Pekka Enberg
Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?
I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.
Alright, now in percpu#for-next tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git for-next
The branch currenly contains only those three patches, so please feel free to pull from it.
Thanks.
Signature · 1 line
tejun
Tejun Heo
maintainerRe: [PATCH 3/3] percpu: use percpu allocator on UP too
Quoted 10 lines · Tejun Heo
Quoted 8 lines · Tejun Heo
Hello,
Quoted 2 lines · Pekka Enberg
Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?
I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.
Quoted 6 lines
Alright, now in percpu#for-next tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.gitfor-nextThe branch currenly contains only those three patches, so please feel free to pull from it.
Branch updated to fix build for s390 and add missing memory clearing for km- allocator. You'll at least want to pull upto commit fc1481a956181d0360d3eb129965302489895a1b.
Thanks.
Signature · 1 line
tejun
Re: [PATCH 3/3] percpu: use percpu allocator on UP too
Quoted 13 lines · Tejun Heo
Quoted 12 lines · Tejun Heo
Quoted 8 lines · Tejun Heo
Hello,
Quoted 2 lines · Pekka Enberg
Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?
I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.
Alright, now in percpu#for-next tree.
Quoted 8 lines
Quoted 4 lines
git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.gitfor-nextThe branch currenly contains only those three patches, so please feel free to pull from it.
Branch updated to fix build for s390 and add missing memory clearing for km- allocator. You'll at least want to pull upto commit
fc1481a956181d0360d3eb129965302489895a1b.
I cherry-picked patches from the branch and reverted SLUB bandaid. SLUB works fine here on UP now so I'm putting it in linux-next. Thanks Tejun!
Pekka
Tejun Heo
maintainerPATCH RESEND 1/3 vmalloc: pcpu_get/free_vm_areas() aren't needed on UP
In reply to Tejun HeoThese functions are used only by percpu memory allocator on SMP. Don't build them on UP.
---
(resending w/ Nick's new email address)
So, something like these three patches. Also available in the following git tree.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git review-up
Nick, can I route this one through percpu tree?
Thanks.
include/linux/vmalloc.h | 2 ++ mm/vmalloc.c | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-)
include/linux/vmalloc.h +2 -0
@@ -115,10 +115,12 @@ extern rwlock_t vmlist_lock;extern struct vm_struct *vmlist;extern __init void vm_area_register_early (struct vm_struct *vm,size_t align); +#ifdef CONFIG_SMP struct vm_struct **pcpu_get_vm_areas (const unsigned long *offsets,const size_t *sizes,int nr_vms,size_t align,gfp_t gfp_mask);void pcpu_free_vm_areas (struct vm_struct **vms,int nr_vms); +#endif #endif /* _LINUX_VMALLOC_H */
mm/vmalloc.c +2 -0
@@ -2052,6 +2052,7 @@ void free_vm_area(struct vm_struct *area) }EXPORT_SYMBOL_GPL (free_vm_area); +#ifdef CONFIG_SMP static struct vmap_area *node_to_va (struct rb_node *n) { return n ? rb_entry(n, struct vmap_area, rb_node) : NULL;
@@ -2332,6 +2333,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)free_vm_area (vms[i]); kfree(vms); } +#endif /* CONFIG_SMP */ #ifdef CONFIG_PROC_FS static void *s_start (struct seq_file *m,loff_t *pos)
Signature · 1 line
1.7.1
Quoted 3 lines · Tejun Heo
Eh... nasty. Maybe we can create a
alloc_percpu_early()function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?
The whole early alloc stuff does also does not go over too well with the page allocator (never had trouble on KVM):
Decompressing Linux... Parsing ELF... done.
Booting the kernel.
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.36-rc2 (root@rd-rsync) (gcc version 4.4.4 (Debian 4.4.4-5) ) #1 SMP Wed Aug 25 13:26:59 CDT 2010
Command line: ro root=/dev/mapper/vgubuntu-root console=tty0 console=ttyS1,57600 idle=mwait earlyprintk=ttyS1,57600 BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
BIOS-e820: 0000000000100000 - 00000000cf699000 (usable)
BIOS-e820: 00000000cf699000 - 00000000cf6af000 (reserved)
BIOS-e820: 00000000cf6af000 - 00000000cf6ce000 (ACPI data)
BIOS-e820: 00000000cf6ce000 - 00000000d0000000 (reserved)
BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
BIOS-e820: 00000000fe000000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 00000001b0000000 (usable)
bootconsole [earlyser0] enabled
NX (Execute Disable) protection: active
DMI 2.6 present.
No AGP bridge foundlast_pfn = 0x1b0000 max_arch_pfn = 0x400000000last_pfn = 0xcf699 max_arch_pfn = 0x400000000
found SMP MP-table at [ffff8800000fe710] fe710init_memory_mapping: 0000000000000000-00000000cf699000init_memory_mapping: 0000000100000000-00000001b0000000
RAMDISK: 37b98000 - 37ff0000
ACPI: RSDP 00000000000f1630 00024 (v02 DELL )
ACPI: XSDT 00000000000f1734 0009C (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: FACP 00000000cf6c3f9c 000F4 (v03 DELL PE_SC3 00000001 DELL
00000001)
ACPI: DSDT 00000000cf6af000 0320F (v01 DELL PE_SC3 00000001 INTL
20050624)
ACPI: FACS 00000000cf6c6000 00040
ACPI: APIC 00000000cf6c3478 0015E (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SPCR 00000000cf6c35d8 00050 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: HPET 00000000cf6c362c 00038 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: DM__ 00000000cf6c3668 001A8 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: MCFG 00000000cf6c38c4 0003C (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: WD__ 00000000cf6c3904 00134 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SLIC 00000000cf6c3a3c 00176 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: ERST 00000000cf6b2390 00270 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: HEST 00000000cf6b2600 0027C (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: BERT 00000000cf6b2210 00030 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: EINJ 00000000cf6b2240 00150 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SRAT 00000000cf6c3bc0 00370 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: TCPA 00000000cf6c3f34 00064 (v02 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SSDT 00000000cf6c7000 04BE4 (v01 INTEL PPM RCM 80000001 INTL 20061109)
SRAT: PXM 1 -> APIC 0x10 -> Node 0
SRAT: PXM 2 -> APIC 0x00 -> Node 1
SRAT: PXM 1 -> APIC 0x12 -> Node 0
SRAT: PXM 2 -> APIC 0x02 -> Node 1
SRAT: PXM 1 -> APIC 0x14 -> Node 0
SRAT: PXM 2 -> APIC 0x04 -> Node 1
SRAT: PXM 1 -> APIC 0x16 -> Node 0
SRAT: PXM 2 -> APIC 0x06 -> Node 1
SRAT: PXM 1 -> APIC 0x11 -> Node 0
SRAT: PXM 2 -> APIC 0x01 -> Node 1
SRAT: PXM 1 -> APIC 0x13 -> Node 0
SRAT: PXM 2 -> APIC 0x03 -> Node 1
SRAT: PXM 1 -> APIC 0x15 -> Node 0
SRAT: PXM 2 -> APIC 0x05 -> Node 1
SRAT: PXM 1 -> APIC 0x17 -> Node 0
SRAT: PXM 2 -> APIC 0x07 -> Node 1
SRAT: Node 1 PXM 2 0-c0000000
SRAT: Node 0 PXM 1 c0000000-d0000000
SRAT: Node 0 PXM 1 100000000-1b0000000
SRAT: Node 0 [c0000000,d0000000) + [100000000,1b0000000) -> [c0000000,1b0000000)
Initmem setup node 0 00000000c0000000-00000001b0000000
NODE_DATA [0000000100000000 - 0000000100004fff]
Initmem setup node 1 0000000000000000-00000000c0000000
NODE_DATA [00000000016ee780 - 00000000016f377f]
Zone PFN ranges:
DMA 0x00000001 -> 0x00001000
DMA32 0x00001000 -> 0x00100000
Normal 0x00100000 -> 0x001b0000
Movable zone start PFN for each node
early_node_map[4] active PFN ranges
1: 0x00000001 -> 0x000000a0
1: 0x00000100 -> 0x000c0000
0: 0x000c0000 -> 0x000cf699
0: 0x00100000 -> 0x001b0000
------------[ cut here ]------------
WARNING: at mm/percpu.c:285 pcpu_mem_alloc+0x31/0x78()
Hardware name: PowerEdge R610
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.36-rc2 #1
Call Trace:
[<ffffffff810305fa>] warn_slowpath_common+0x80/0x98
[<ffffffff81030627>] warn_slowpath_null+0x15/0x17
[<ffffffff810a226c>] pcpu_mem_alloc+0x31/0x78
[<ffffffff810a2fe1>] pcpu_alloc+0x243/0x8d9
[<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
[<ffffffff810a3692>] __alloc_percpu+0xb/0xd
[<ffffffff81601d6f>] free_area_init_node+0x235/0x350
[<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
[<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
[<ffffffff815e2d50>] paging_init+0x53/0x5a
[<ffffffff815d642f>] setup_arch+0x82c/0x8b8
[<ffffffff813ad816>] ? printk+0x3c/0x3e
[<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
[<ffffffff815d29df>] start_kernel+0x88/0x351
[<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
[<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
---[ end trace 4eaa2a86a8e2da22 ]---
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G W 2.6.36-rc2 #1
Call Trace:
[<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
[<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
[<ffffffff810a3692>] __alloc_percpu+0xb/0xd
[<ffffffff81601d6f>] free_area_init_node+0x235/0x350
[<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
[<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
[<ffffffff815e2d50>] paging_init+0x53/0x5a
[<ffffffff815d642f>] setup_arch+0x82c/0x8b8
[<ffffffff813ad816>] ? printk+0x3c/0x3e
[<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
[<ffffffff815d29df>] start_kernel+0x88/0x351
[<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
[<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
---[ end trace 4eaa2a86a8e2da22 ]---
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G W 2.6.36-rc2 #1
Call Trace:
[<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
[<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
[<ffffffff810a3692>] __alloc_percpu+0xb/0xd
[<ffffffff81601d6f>] free_area_init_node+0x235/0x350
[<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
[<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
[<ffffffff815e2d50>] paging_init+0x53/0x5a
[<ffffffff815d642f>] setup_arch+0x82c/0x8b8
[<ffffffff813ad816>] ? printk+0x3c/0x3e
[<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
[<ffffffff815d29df>] start_kernel+0x88/0x351
[<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
[<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G W 2.6.36-rc2 #1
Call Trace:
[<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
[<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
[<ffffffff810a3692>] __alloc_percpu+0xb/0xd
[<ffffffff81601d6f>] free_area_init_node+0x235/0x350
[<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
[<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
[<ffffffff815e2d50>] paging_init+0x53/0x5a
[<ffffffff815d642f>] setup_arch+0x82c/0x8b8
[<ffffffff813ad816>] ? printk+0x3c/0x3e
[<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4aErr... Sorry there was patch under testing in there (remove the boot pageset from the page allocator and use early allocpercpu) which resulted in the reserve area becoming a bit tight. Sorry.
Hi Pekka,
Quoted 4 lines · On Tue, 24 Aug 2010 20:41:36 +0300 Pekka Enberg <[email protected]> wrote:
Thanks for the report. The problem should be fixed by this commit:
A small point: I did *not* review that patch. I *did* report the bug (and seemingly provided the commit message).
Thanks for the build fix.
Signature · 3 lines
Cheers, Stephen Rothwell [email protected] http://www.canb.auug.org.au/~sfr/
Quoted 10 lines · Stephen Rothwell
Hi Pekka,
Quoted 4 lines · On Tue, 24 Aug 2010 20:41:36 +0300 Pekka Enberg<[email protected]> wrote:
Thanks for the report. The problem should be fixed by this commit:
A small point: I did *not* review that patch. I *did* report the bug (and seemingly provided the commit message).
Oh, sorry about that.
Quoted 1 line
Thanks for the build fix.
I dropped slub cleanups from -next until we've resolved the UP boot time crash problem.
Quoted 3 lines · Pekka Enberg
I dropped slub cleanups from -next until we've resolved the UP boot time crash problem.
It works here with this patch. I'd rather see a general solution so that we can use allocpercpu for bootstrapping any allocator without such special bandaids.
Subject: Slub: UP bandaid
Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().
Compound pages will be released when the cpu caches are resized.
--- mm/slub.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
mm/slub.c +16 -0
@@ -2064,8 +2064,24 @@static inline int alloc_kmem_cache_cpus (struct kmem_cache *s) { +#ifdef CONFIG_SMP +/* +* Will use reserve that does not require slab operation during +* early boot. +*/ BUILD_BUG_ON (PERCPU_DYNAMIC_EARLY_SIZE <SLUB_PAGE_SHIFT *sizeof (struct kmem_cache_cpu )); +#else +/* +* Special hack for UP mode. allocpercpu() falls back to kmalloc +* operations. So we cannot use that before the slab allocator is up +* Simply get the smallest possible compound page. The page will be +* released via kfree() when the cpu caches are resized later. +*/ +if (slab_state <UP ) + s->cpu_slab = (__percpu void *)__get_free_page (GFP_NOWAIT ,1 ); + else +#endif s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
David Rientjes
reviewerQuoted 26 lines · Christoph Lameter
Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use
__get_free_page()to improvise a compound page allocation that can be later freed viakfree().Compound pages will be released when the cpu caches are resized.
Signed-off-by Christoph Lameter <[email protected]>--- mm/slub.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)Index:
linux-2.6/mm/slub.c
===================================================================mm/slub.c +5
-0@@ -2064,8 +2064,24 @@static inline int alloc_kmem_cache_cpus (struct kmem_cache *s) { +#ifdef CONFIG_SMP +/* +* Will use reserve that does not require slab operation during +* early boot. +*/ BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
mm/slub.c line 2082
Quoted 10 lines
mm/slub.c +9
-0SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu)); +#else +/* +* Special hack for UP mode. allocpercpu() falls back to kmalloc +* operations. So we cannot use that before the slab allocator is up +* Simply get the smallest possible compound page. The page will be +* released via kfree() when the cpu caches are resized later. +*/ +if (slab_state <UP ) + s->cpu_slab = (__percpu void *)__get_free_page (GFP_NOWAIT ,1 );
__get_free_pages() takes an order argument.
Quoted 5 lines
mm/slub.c +2
-0+ else +#endif s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
Quoted 9 lines · David Rientjes
Review
Quoted 7 lines
(quoted diff) +7
-0+ * Special hack for UP mode. allocpercpu() falls back to kmalloc + * operations. So we cannot use that before the slab allocator is up + * Simply get the smallest possible compound page. The page will be + * released via kfree() when the cpu caches are resized later. + */ + if (slab_state < UP) + s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);
__get_free_pages()takes an order argument.
Right. Patch not refreshed after the last tinzy winzy change. Seems that I got a bit rusty.
Subject: Slub: UP bandaid
Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().
Compound pages will be released when the cpu caches are resized.
--- mm/slub.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
mm/slub.c +16 -0
@@ -2091,8 +2091,24 @@ init_kmem_cache_node(struct kmem_cache_nstatic inline int alloc_kmem_cache_cpus (struct kmem_cache *s) { +#ifdef CONFIG_SMP +/* +* Will use reserve that does not require slab operation during +* early boot. +*/ BUILD_BUG_ON (PERCPU_DYNAMIC_EARLY_SIZE <SLUB_PAGE_SHIFT *sizeof (struct kmem_cache_cpu )); +#else +/* +* Special hack for UP mode. allocpercpu() falls back to kmalloc +* operations. So we cannot use that before the slab allocator is up +* Simply get the smallest possible compound page. The page will be +* released via kfree() when the cpu caches are resized later. +*/ +if (slab_state <UP ) + s->cpu_slab = (__percpu void *)__get_free_pages (GFP_NOWAIT ,1 ); + else +#endif s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
Quoted 15 lines · Christoph Lameter
Quoted 9 lines
mm/slub.c init_kmem_cache_node(struct kmem_cache_n
Quoted 7 lines
mm/slub.c +7
-0+* Special hack for UP mode. allocpercpu() falls back to kmalloc +* operations. So we cannot use that before the slab allocator is up +* Simply get the smallest possible compound page. The page will be +* released via kfree() when the cpu caches are resized later. +*/ +if (slab_state <UP ) + s->cpu_slab = (__percpu void *)__get_free_page (GFP_NOWAIT ,1 );
__get_free_pages()takes an order argument.Right. Patch not refreshed after the last tinzy winzy change. Seems that I got a bit rusty.
Subject: Slub: UP bandaid
Quoted 7 lines
Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use
__get_free_page()to improvise a compound page allocation that can be later freed viakfree().Compound pages will be released when the cpu caches are resized.
Signed-off-by Christoph Lameter <[email protected]>
I'm really hoping that we can remove this hack soon when the percpu allocator can handle these allocations on UP without any specialized slab behavior.
Christoph Lameter
Acked-by David RientjesQuoted 3 lines · David Rientjes
I'm really hoping that we can remove this hack soon when the percpu allocator can handle these allocations on UP without any specialized slab behavior.
So do I. Here is a slightly less hacky version through using kmalloc_large instead:
Subject: Slub: UP bandaid
Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().
Compound pages will be released when the cpu caches are resized.
--- mm/slub.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
mm/slub.c +16 -0
@@ -2103,8 +2103,24 @@ init_kmem_cache_node(struct kmem_cache_nstatic inline int alloc_kmem_cache_cpus (struct kmem_cache *s) { +#ifdef CONFIG_SMP +/* +* Will use reserve that does not require slab operation during +* early boot. +*/ BUILD_BUG_ON (PERCPU_DYNAMIC_EARLY_SIZE <SLUB_PAGE_SHIFT *sizeof (struct kmem_cache_cpu )); +#else +/* +* Special hack for UP mode. allocpercpu() falls back to kmalloc +* operations. So we cannot use that before the slab allocator is up +* Simply get the smallest possible compound page. The page will be +* released via kfree() when the cpu caches are resized later. +*/ +if (slab_state <UP ) + s->cpu_slab = (__percpu void *)kmalloc_large (PAGE_SIZE <<1 ,GFP_NOWAIT ); + else +#endif s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · David Rientjes
I'm really hoping that we can remove this hack soon when the percpu allocator can handle these allocations on UP without any specialized slab behavior.
So do I. Here is a slightly less hacky version through using
kmalloc_largeinstead:
Applied and queued to -next.