Skip to contentSkip to conversation

linux-next: build failure after merge of the final tree (slab tree related)

PATCH Superseded Stephen Rothwell 5 participants 31 messages Aug 24, 2010 – Sep 18, 2010

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

Hi all,

After merging the final tree, today's linux-next build (powerpc allnoconfig) failed like this:

mm/slub.c: In function 'alloc_kmem_cache_cpus':
mm/slub.c:2094: error: 'PERCPU_DYNAMIC_EARLY_SIZE' undeclared (first use in this function) mm/slub.c:2094: error: (Each undeclared identifier is reported only once mm/slub.c:2094: error: for each function it appears in.)
mm/slub.c:2094: error: bit-field '<anonymous>' width not an integer constant

Caused by commit e18d65f0500b95d8724b17d8ea9f1116cf390bbe ("slub: Remove static kmem_cache_cpu array for boot"). PERCPU_DYNAMIC_EARLY_SIZE is only defined for SMP (and only in linux/percpu.h which is not explicitly included). This build does not have CONFIG_SMP set.

I have used the version of the slab tree from next-20100823 for today.

Signature · 3 lines
Cheers,
Stephen Rothwell                    [email protected]
http://www.canb.auug.org.au/~sfr/
#1 lore raw
Quoted 11 lines · Stephen Rothwell

Hi all,

After merging the final tree, today's linux-next build (powerpc allnoconfig) failed like this:

mm/slub.c: In function 'alloc_kmem_cache_cpus':
mm/slub.c:2094: error: 'PERCPU_DYNAMIC_EARLY_SIZE' undeclared (first use in this function) mm/slub.c:2094: error: (Each undeclared identifier is reported only once mm/slub.c:2094: error: for each function it appears in.)
mm/slub.c:2094: error: bit-field '<anonymous>' width not an integer constant

Quoted 6 lines

Caused by commit e18d65f0500b95d8724b17d8ea9f1116cf390bbe ("slub: Remove static kmem_cache_cpu array for boot"). PERCPU_DYNAMIC_EARLY_SIZE is only defined for SMP (and only in linux/percpu.h which is not explicitly included). This build does not have CONFIG_SMP set.

I have used the version of the slab tree from next-20100823 for today.

Thanks for the report. The problem should be fixed by this commit:

http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commitdiff;h=5792949c540c9fd719c3fc0d6edc97374222e792

#2 lore raw
Quoted 1 line · Pekka Enberg

Thanks for the report. The problem should be fixed by this commit:

Its not that easy. __alloc_percpu falls back to kzalloc() on UP and this can result in unique bootstrap problems with UP since the bootstrap array is no longer there. Does the UP kernel boot?

Why did this ever build on my UP configuration tests? Hmmm... I only tested x86_64 UP. This was 32 bit UP I guess.

#3 lore raw
Quoted 7 lines · Christoph Lameter
Quoted 1 line · Pekka Enberg

Thanks for the report. The problem should be fixed by this commit:

Its not that easy. __alloc_percpu falls back to kzalloc() on UP and this can result in unique bootstrap problems with UP since the bootstrap array is no longer there. Does the UP kernel boot?

No, I get this under kvm:

[    0.000000] Linux version 2.6.36-rc2+ (penberg@tiger) (gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9) ) #103 Tue Aug 24 21:27:28 EEST 2010
[    0.000000] Command line: notsc nolapic nosmp noacpi pci=conf1 earlyprintk=ttyS0,keep
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
[    0.000000]  BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000f0000 - 00000000000fffff (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 0000000004000000 (usable)
[    0.000000] console [earlyser0] enabled
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x4000 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106
[    0.000000] CPU MTRRs all blank - virtualized system.
[    0.000000] Scanning 1 areas for low memory corruption
[    0.000000] modified physical RAM map:
[    0.000000]  modified: 0000000000000000 - 0000000000010000 (reserved)
[    0.000000]  modified: 0000000000010000 - 000000000009fc00 (usable)
[    0.000000]  modified: 000000000009fc00 - 00000000000a0000 (reserved)
[    0.000000]  modified: 00000000000f0000 - 00000000000fffff (reserved)
[    0.000000]  modified: 0000000000100000 - 0000000004000000 (usable)
[    0.000000] init_memory_mapping: 0000000000000000-0000000004000000
[    0.000000] ACPI Error: A valid RSDP was not found (20100702/tbxfroot-219)
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:1945621, boot clock
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009f
[    0.000000]     0: 0x00000100 -> 0x00004000
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
[    0.000000] PM: Registered nosave memory: 00000000000f0000 - 00000000000ff000
[    0.000000] PM: Registered nosave memory: 00000000000ff000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 4000000 (gap: 4000000:fc000000)
[    0.000000] Booting paravirtualized kernel on KVM
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 16047
[    0.000000] Kernel command line: notsc nolapic nosmp noacpi pci=conf1 earlyprintk=ttyS0,keep
[    0.000000] notsc: Kernel compiled with CONFIG_X86_TSC, cannot disable TSC completely.
[    0.000000] PID hash table entries: 256 (order: -1, 2048 bytes)
[    0.000000] Dentry cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.000000] Inode-cache hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Subtract (23 early reservations)
[    0.000000]   #0 [0001000000 - 0001b751a0]   TEXT DATA BSS
[    0.000000]   #1 [000009f000 - 0000100000]   BIOS reserved
[    0.000000]   #2 [0000010000 - 0000012000]      TRAMPOLINE
[    0.000000]   #3 [0000012000 - 0000016000]     ACPI WAKEUP
[    0.000000]   #4 [0001b751c0 - 0001b761c0]         BOOTMEM
[    0.000000]   #5 [0001f761c0 - 0001f761d8]         BOOTMEM
[    0.000000]   #6 [0002377000 - 0002378000]         BOOTMEM
[    0.000000]   #7 [0002378000 - 0002379000]         BOOTMEM
[    0.000000]   #8 [0002400000 - 0002600000]        MEMMAP 0
[    0.000000]   #9 [0001b761c0 - 0001b762c0]         BOOTMEM
[    0.000000]   #10 [0001b762c0 - 0001b766c0]         BOOTMEM
[    0.000000]   #11 [0001b77000 - 0001b78000]         BOOTMEM
[    0.000000]   #12 [0001b766c0 - 0001b767d8]         BOOTMEM
[    0.000000]   #13 [0001b76800 - 0001b76868]         BOOTMEM
[    0.000000]   #14 [0001b76880 - 0001b768e8]         BOOTMEM
[    0.000000]   #15 [0001b76900 - 0001b76968]         BOOTMEM
[    0.000000]   #16 [0001b76980 - 0001b769e8]         BOOTMEM
[    0.000000]   #17 [0001b76a00 - 0001b76a20]         BOOTMEM
[    0.000000]   #18 [0001b76a40 - 0001b76a7d]         BOOTMEM
[    0.000000]   #19 [0001b76a80 - 0001b76abd]         BOOTMEM
[    0.000000]   #20 [0001b78000 - 0001b78800]         BOOTMEM
[    0.000000]   #21 [0001b78800 - 0001b88800]         BOOTMEM
[    0.000000]   #22 [0001b88800 - 0001b90800]         BOOTMEM
[    0.000000] Memory: 51156k/65536k available (6550k kernel code, 452k absent, 13928k reserved, 3761k data, 764k init)
[    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache size=232 realsize=256 order=0 offset=0 flags=42000
[    0.000000]
[    0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc2+ #103
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff81659a35>] panic+0x91/0x19b
[    0.000000]  [<ffffffff81126e59>] kmem_cache_open+0x1e9/0x200
[    0.000000]  [<ffffffff81a3141a>] kmem_cache_init+0x57/0x300
[    0.000000]  [<ffffffff81a14a97>] start_kernel+0x1ca/0x396
[    0.000000]  [<ffffffff81a14325>] x86_64_start_reservations+0x12c/0x130
[    0.000000]  [<ffffffff81a14423>] x86_64_start_kernel+0xfa/0x109
Quoted 2 lines

Why did this ever build on my UP configuration tests? Hmmm... I only tested x86_64 UP. This was 32 bit UP I guess.

I could reproduce the build problem with x86-64 and !CONFIG_SMP so it's not related to 32-bit.

#4 lore raw
Quoted 5 lines · Pekka Enberg
Quoted 4 lines · Christoph Lameter
Quoted 1 line · Pekka Enberg

Thanks for the report. The problem should be fixed by this commit:

Quoted 8 lines
Quoted 3 lines

Its not that easy. __alloc_percpu falls back to kzalloc() on UP and this can result in unique bootstrap problems with UP since the bootstrap array is no longer there. Does the UP kernel boot?

No, I get this under kvm:

[    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache

size=232 realsize=256 order=0 offset=0 flags=42000

alloc per cpu result in kmalloc which fails.

Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?

maintainer
#5 lore raw

Hello,

Quoted 7 lines · Christoph Lameter
Quoted 2 lines
[    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache

size=232 realsize=256 order=0 offset=0 flags=42000

alloc per cpu result in kmalloc which fails.

Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?

Eh... nasty. Maybe we can create a alloc_percpu_early() function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?

Signature · 1 line
tejun
#6 lore raw
Quoted 6 lines · Tejun Heo

Hello,

Quoted 3 lines · Christoph Lameter
Quoted 2 lines
[    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache

size=232 realsize=256 order=0 offset=0 flags=42000

Quoted 8 lines
Quoted 4 lines

alloc per cpu result in kmalloc which fails.

Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?

Eh... nasty. Maybe we can create a alloc_percpu_early() function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?

Yeah, I was thinking about that too.

Pekka

#7 lore raw
Quoted 9 lines · Pekka Enberg
Quoted 8 lines · Tejun Heo

Hello,

Quoted 5 lines · Christoph Lameter
Quoted 2 lines
[    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache

size=232 realsize=256 order=0 offset=0 flags=42000

alloc per cpu result in kmalloc which fails.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Tejon: Is there some way we could get a reserved per cpu area under UP instead of fallback to slab allocations during bootup?

Eh... nasty. Maybe we can create a alloc_percpu_early() function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?

Yeah, I was thinking about that too.

Another solution is to allocate an order 1 (compound) page for each early cache. Then resize the kmalloc array after everything is up. kfree() will then redirect to the page allocator.

But this is a slab allocator specific solution.

We now have the situation that alloc_percpu can only be used on early boot on SMP machines. A general solution would be better I think.

If the early alloc_percpu stuff would work consistently then it could also be used to avoid the boot_pageset in the page allocator f.e.

Can we just get rid of the special UP case and just run the percpu subsystem even for UP?

maintainer
#8 lore raw

Hello,

Quoted 2 lines · Christoph Lameter

Can we just get rid of the special UP case and just run the percpu subsystem even for UP?

Yeah, maybe. Then we also can guarantee that percpu allocator always honors alignment (which wq code currently requires and papers over with similarly ugly workaround). It would add a mostly redundant allocator code tho. I'll look into how easily it can be done.

Thanks.

Signature · 1 line
tejun
maintainer
#9 lore raw

PATCH 1/3 vmalloc: pcpu_get/free_vm_areas() aren't needed on UP

These functions are used only by percpu memory allocator on SMP. Don't build them on UP.

---
So, something like these three patches.

Nick, can I route this one through percpu tree?

Thanks.

 include/linux/vmalloc.h |    2 ++
 mm/vmalloc.c            |    2 ++
 2 files changed, 4 insertions(+), 0 deletions(-)
include/linux/vmalloc.h +2 -0
index de05e96..3d510e8 100644
@@ -115,10 +115,12 @@ extern rwlock_t vmlist_lock;
 extern struct vm_struct *vmlist;
 extern __init void vm_area_register_early(struct vm_struct *vm, size_t align);
 
+#ifdef CONFIG_SMP
 struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,
 				     const size_t *sizes, int nr_vms,
 				     size_t align, gfp_t gfp_mask);
 
 void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms);
+#endif
 
 #endif /* _LINUX_VMALLOC_H */
mm/vmalloc.c +2 -0
index b7e314b..eb57a63 100644
@@ -2052,6 +2052,7 @@ void free_vm_area(struct vm_struct *area)
 }
 EXPORT_SYMBOL_GPL(free_vm_area);
 
+#ifdef CONFIG_SMP
 static struct vmap_area *node_to_va(struct rb_node *n)
 {
 	return n ? rb_entry(n, struct vmap_area, rb_node) : NULL;
@@ -2332,6 +2333,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 		free_vm_area(vms[i]);
 	kfree(vms);
 }
+#endif	/* CONFIG_SMP */
 
 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)
Signature · 1 line
1.7.1
maintainer
#11 lore raw

PATCH 2/3 percpu: reduce PCPU_MIN_UNIT_SIZE to 32k

In reply to Tejun Heo

In preparation of enabling percpu allocator for UP, reduce PCPU_MIN_UNIT_SIZE to 32k. On UP, the first chunk doesn't have to include static percpu variables and chunk size can be smaller which is important as UP percpu allocator will use contiguous kernel memory to populate chunks.

PCPU_MIN_UNIT_SIZE also determines the maximum supported allocation size but 32k should still be enough.

---
 include/linux/percpu.h |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
include/linux/percpu.h +1 -1
index 49466b1..fc8130a 100644
@@ -42,7 +42,7 @@
 #ifdef CONFIG_SMP
 
 /* minimum unit size, also is the maximum supported allocation size */
-#define PCPU_MIN_UNIT_SIZE		PFN_ALIGN(64 << 10)
+#define PCPU_MIN_UNIT_SIZE		PFN_ALIGN(32 << 10)
 
 /*
  * Percpu allocator can serve percpu allocations before slab is
Signature · 1 line
1.7.1
maintainer
#13 lore raw

PATCH 3/3 percpu: use percpu allocator on UP too

In reply to Tejun Heo

On UP, percpu allocations were redirected to kmalloc. This has the following problems.

* For certain amount of allocations (determined by PERCPU_DYNAMIC_EARLY_SLOTS and PERCPU_DYNAMIC_EARLY_SIZE), percpu allocator can be used before the usual kernel memory allocator is brought online. On SMP, this is used to initialize the kernel memory allocator.

* percpu allocator honors alignment upto PAGE_SIZE but kmalloc() doesn't. For example, workqueue makes use of larger alignments for cpu_workqueues.

Currently, users of percpu allocators need to handle UP differently, which is somewhat fragile and ugly. Other than small amount of memory, there isn't much to lose by enabling percpu allocator on UP. It can simply use kernel memory based chunk allocation which was added for SMP archs w/o MMUs.

This patch removes mm/percpu_up.c, builds mm/percpu.c on UP too and makes UP build use percpu-km. As percpu addresses and kernel addresses are always identity mapped and static percpu variables don't need any special treatment, nothing is arch dependent and mm/percpu.c implements generic setup_per_cpu_areas() for UP.

Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
---
 include/linux/percpu.h |   29 ++++-------------------
 mm/Kconfig             |    8 ++++++
 mm/Makefile            |    7 +----
 mm/percpu-km.c         |    2 +-
 mm/percpu.c            |   60 ++++++++++++++++++++++++++++++++++++++++++++---
 mm/percpu_up.c         |   30 ------------------------
 6 files changed, 71 insertions(+), 65 deletions(-)
 delete mode 100644 mm/percpu_up.c
include/linux/percpu.h +5 -24
index fc8130a..aeeeef1 100644
@@ -39,8 +39,6 @@
 	preempt_enable();				\
 } while (0)
 
-#ifdef CONFIG_SMP
-
 /* minimum unit size, also is the maximum supported allocation size */
 #define PCPU_MIN_UNIT_SIZE		PFN_ALIGN(32 << 10)
 
@@ -137,37 +135,20 @@ extern int __init pcpu_page_first_chunk(size_t reserved_size,
  * dynamically allocated. Non-atomic access to the current CPU's
  * version should probably be combined with get_cpu()/put_cpu().
  */
+#ifdef CONFIG_SMP
 #define per_cpu_ptr(ptr, cpu)	SHIFT_PERCPU_PTR((ptr), per_cpu_offset((cpu)))
+#else
+#define per_cpu_ptr(ptr, cpu)	({ (void)(cpu); VERIFY_PERCPU_PTR((ptr)); })
+#endif
 
 extern void __percpu *__alloc_reserved_percpu(size_t size, size_t align);
 extern bool is_kernel_percpu_address(unsigned long addr);
 
-#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA
+#if !defined(CONFIG_SMP) || !defined(CONFIG_HAVE_SETUP_PER_CPU_AREA)
 extern void __init setup_per_cpu_areas(void);
 #endif
 extern void __init percpu_init_late(void);
 
-#else /* CONFIG_SMP */
-
-#define per_cpu_ptr(ptr, cpu) ({ (void)(cpu); VERIFY_PERCPU_PTR((ptr)); })
-
-/* can't distinguish from other static vars, always false */
-static inline bool is_kernel_percpu_address(unsigned long addr)
-{
-	return false;
-}
-
-static inline void __init setup_per_cpu_areas(void) { }
-
-static inline void __init percpu_init_late(void) { }
-
-static inline void *pcpu_lpage_remapped(void *kaddr)
-{
-	return NULL;
-}
-
-#endif /* CONFIG_SMP */
-
 extern void __percpu *__alloc_percpu(size_t size, size_t align);
 extern void free_percpu(void __percpu *__pdata);
 extern phys_addr_t per_cpu_ptr_to_phys(void *addr);
mm/Kconfig +8 -0
index f4e516e..01a5744 100644
@@ -301,3 +301,11 @@ config NOMMU_INITIAL_TRIM_EXCESS
 	  of 1 says that all excess pages should be trimmed.
 
 	  See Documentation/nommu-mmap.txt for more information.
+
+#
+# UP and nommu archs use km based percpu allocator
+#
+config NEED_PER_CPU_KM
+	depends on !SMP
+	bool
+	default y
mm/Makefile +1 -6
index 34b2546..f73f75a 100644
@@ -11,7 +11,7 @@ obj-y			:= bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \
 			   maccess.o page_alloc.o page-writeback.o \
 			   readahead.o swap.o truncate.o vmscan.o shmem.o \
 			   prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \
-			   page_isolation.o mm_init.o mmu_context.o \
+			   page_isolation.o mm_init.o mmu_context.o percpu.o \
 			   $(mmu-y)
 obj-y += init-mm.o
 
@@ -36,11 +36,6 @@ obj-$(CONFIG_FAILSLAB) += failslab.o
 obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o
 obj-$(CONFIG_FS_XIP) += filemap_xip.o
 obj-$(CONFIG_MIGRATION) += migrate.o
-ifdef CONFIG_SMP
-obj-y += percpu.o
-else
-obj-y += percpu_up.o
-endif
 obj-$(CONFIG_QUICKLIST) += quicklist.o
 obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o page_cgroup.o
 obj-$(CONFIG_MEMORY_FAILURE) += memory-failure.o
mm/percpu-km.c +1 -1
index df68085..7037bc7 100644
@@ -27,7 +27,7 @@
  *   chunk size is not aligned.  percpu-km code will whine about it.
  */
 
-#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
+#if defined(CONFIG_SMP) && defined(CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK)
 #error "contiguous percpu allocation is incompatible with paged first chunk"
 #endif
 
mm/percpu.c +56 -4

133 lines not on this page · raw message

mm/percpu_up.c deleted +0 -30
deleted file mode 100644
index db884fa..0000000
@@ -1,30 +0,0 @@
-/*
- * mm/percpu_up.c - dummy percpu memory allocator implementation for UP
- */
-
-#include <linux/module.h>
-#include <linux/percpu.h>
-#include <linux/slab.h>
-
-void __percpu *__alloc_percpu(size_t size, size_t align)
-{
-	/*
-	 * Can't easily make larger alignment work with kmalloc.  WARN
-	 * on it.  Larger alignment should only be used for module
-	 * percpu sections on SMP for which this path isn't used.
-	 */
-	WARN_ON_ONCE(align > SMP_CACHE_BYTES);
-	return (void __percpu __force *)kzalloc(size, GFP_KERNEL);
-}
-EXPORT_SYMBOL_GPL(__alloc_percpu);
-
-void free_percpu(void __percpu *p)
-{
-	kfree(this_cpu_ptr(p));
-}
-EXPORT_SYMBOL_GPL(free_percpu);
-
-phys_addr_t per_cpu_ptr_to_phys(void *addr)
-{
-	return __pa(addr);
-}
Signature · 1 line
1.7.1
Acked-by on v1 3/3
#15 lore raw

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

In reply to Tejun Heo
Quoted 25 lines · Tejun Heo

On UP, percpu allocations were redirected to kmalloc. This has the following problems.

* For certain amount of allocations (determined by PERCPU_DYNAMIC_EARLY_SLOTS and PERCPU_DYNAMIC_EARLY_SIZE), percpu allocator can be used before the usual kernel memory allocator is brought online. On SMP, this is used to initialize the kernel memory allocator.

* percpu allocator honors alignment upto PAGE_SIZE but kmalloc() doesn't. For example, workqueue makes use of larger alignments for cpu_workqueues.

Currently, users of percpu allocators need to handle UP differently, which is somewhat fragile and ugly. Other than small amount of memory, there isn't much to lose by enabling percpu allocator on UP. It can simply use kernel memory based chunk allocation which was added for SMP archs w/o MMUs.

This patch removes mm/percpu_up.c, builds mm/percpu.c on UP too and makes UP build use percpu-km. As percpu addresses and kernel addresses are always identity mapped and static percpu variables don't need any special treatment, nothing is arch dependent and mm/percpu.c implements generic setup_per_cpu_areas() for UP.

Quoted 3 lines
Cc Christoph Lameter<[email protected]>
Cc Pekka Enberg<[email protected]>

Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?

Pekka

maintainer
#16 lore raw

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

Hello,

Quoted 2 lines · Pekka Enberg

Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?

I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.

Thanks.

Signature · 1 line
tejun
maintainer
#17 lore raw

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

Quoted 8 lines · Tejun Heo

Hello,

Quoted 2 lines · Pekka Enberg

Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?

I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.

Alright, now in percpu#for-next tree.

git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git for-next

The branch currenly contains only those three patches, so please feel free to pull from it.

Thanks.

Signature · 1 line
tejun
maintainer
#18 lore raw

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

Quoted 10 lines · Tejun Heo
Quoted 8 lines · Tejun Heo

Hello,

Quoted 2 lines · Pekka Enberg

Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?

I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.

Quoted 6 lines

Alright, now in percpu#for-next tree.

git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git for-next

The branch currenly contains only those three patches, so please feel free to pull from it.

Branch updated to fix build for s390 and add missing memory clearing for km- allocator. You'll at least want to pull upto commit fc1481a956181d0360d3eb129965302489895a1b.

Thanks.

Signature · 1 line
tejun
#19 lore raw

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

Quoted 13 lines · Tejun Heo
Quoted 12 lines · Tejun Heo
Quoted 8 lines · Tejun Heo

Hello,

Quoted 2 lines · Pekka Enberg

Is this going into some public append-only branch I could cherry-pick the changeset from to my 'slub/cleanups' branch?

I'll put it into percpu#for-next once Linus pulls in the currently pending set of fixes. I'll let you know.

Alright, now in percpu#for-next tree.

Quoted 8 lines
Quoted 4 lines

git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git for-next

The branch currenly contains only those three patches, so please feel free to pull from it.

Branch updated to fix build for s390 and add missing memory clearing for km- allocator. You'll at least want to pull upto commit fc1481a956181d0360d3eb129965302489895a1b.

I cherry-picked patches from the branch and reverted SLUB bandaid. SLUB works fine here on UP now so I'm putting it in linux-next. Thanks Tejun!

Pekka

maintainer
#20 lore raw

PATCH RESEND 1/3 vmalloc: pcpu_get/free_vm_areas() aren't needed on UP

In reply to Tejun Heo

These functions are used only by percpu memory allocator on SMP. Don't build them on UP.

---
(resending w/ Nick's new email address)

So, something like these three patches. Also available in the following git tree.

git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git review-up

Nick, can I route this one through percpu tree?

Thanks.

 include/linux/vmalloc.h |    2 ++
 mm/vmalloc.c            |    2 ++
 2 files changed, 4 insertions(+), 0 deletions(-)
include/linux/vmalloc.h +2 -0
index de05e96..3d510e8 100644
@@ -115,10 +115,12 @@ extern rwlock_t vmlist_lock;
 extern struct vm_struct *vmlist;
 extern __init void vm_area_register_early(struct vm_struct *vm, size_t align);
 
+#ifdef CONFIG_SMP
 struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,
 				     const size_t *sizes, int nr_vms,
 				     size_t align, gfp_t gfp_mask);
 
 void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms);
+#endif
 
 #endif /* _LINUX_VMALLOC_H */
mm/vmalloc.c +2 -0
index b7e314b..eb57a63 100644
@@ -2052,6 +2052,7 @@ void free_vm_area(struct vm_struct *area)
 }
 EXPORT_SYMBOL_GPL(free_vm_area);
 
+#ifdef CONFIG_SMP
 static struct vmap_area *node_to_va(struct rb_node *n)
 {
 	return n ? rb_entry(n, struct vmap_area, rb_node) : NULL;
@@ -2332,6 +2333,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 		free_vm_area(vms[i]);
 	kfree(vms);
 }
+#endif	/* CONFIG_SMP */
 
 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)
Signature · 1 line
1.7.1
#21 lore raw
In reply to Tejun Heo
Quoted 3 lines · Tejun Heo

Eh... nasty. Maybe we can create a alloc_percpu_early() function which doesn't allow freeing of allocate memory and just redirect to bootmem on UP?

The whole early alloc stuff does also does not go over too well with the page allocator (never had trouble on KVM):

Decompressing Linux... Parsing ELF... done.
Booting the kernel.
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.36-rc2 (root@rd-rsync) (gcc version 4.4.4 (Debian 4.4.4-5) ) #1 SMP Wed Aug 25 13:26:59 CDT 2010
Command line: ro root=/dev/mapper/vgubuntu-root console=tty0 console=ttyS1,57600 idle=mwait earlyprintk=ttyS1,57600 BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
BIOS-e820: 0000000000100000 - 00000000cf699000 (usable)
BIOS-e820: 00000000cf699000 - 00000000cf6af000 (reserved)
BIOS-e820: 00000000cf6af000 - 00000000cf6ce000 (ACPI data)
BIOS-e820: 00000000cf6ce000 - 00000000d0000000 (reserved)
BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
BIOS-e820: 00000000fe000000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 00000001b0000000 (usable)
bootconsole [earlyser0] enabled
NX (Execute Disable) protection: active
DMI 2.6 present.
No AGP bridge found
last_pfn = 0x1b0000 max_arch_pfn = 0x400000000
last_pfn = 0xcf699 max_arch_pfn = 0x400000000
found SMP MP-table at [ffff8800000fe710] fe710
init_memory_mapping: 0000000000000000-00000000cf699000
init_memory_mapping: 0000000100000000-00000001b0000000
RAMDISK: 37b98000 - 37ff0000
ACPI: RSDP 00000000000f1630 00024 (v02 DELL )
ACPI: XSDT 00000000000f1734 0009C (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: FACP 00000000cf6c3f9c 000F4 (v03 DELL PE_SC3 00000001 DELL
00000001)
ACPI: DSDT 00000000cf6af000 0320F (v01 DELL PE_SC3 00000001 INTL
20050624)
ACPI: FACS 00000000cf6c6000 00040
ACPI: APIC 00000000cf6c3478 0015E (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SPCR 00000000cf6c35d8 00050 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: HPET 00000000cf6c362c 00038 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: DM__ 00000000cf6c3668 001A8 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: MCFG 00000000cf6c38c4 0003C (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: WD__ 00000000cf6c3904 00134 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SLIC 00000000cf6c3a3c 00176 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: ERST 00000000cf6b2390 00270 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: HEST 00000000cf6b2600 0027C (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: BERT 00000000cf6b2210 00030 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: EINJ 00000000cf6b2240 00150 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SRAT 00000000cf6c3bc0 00370 (v01 DELL PE_SC3 00000001 DELL
00000001)
ACPI: TCPA 00000000cf6c3f34 00064 (v02 DELL PE_SC3 00000001 DELL
00000001)
ACPI: SSDT 00000000cf6c7000 04BE4 (v01 INTEL PPM RCM 80000001 INTL 20061109)
SRAT: PXM 1 -> APIC 0x10 -> Node 0
SRAT: PXM 2 -> APIC 0x00 -> Node 1
SRAT: PXM 1 -> APIC 0x12 -> Node 0
SRAT: PXM 2 -> APIC 0x02 -> Node 1
SRAT: PXM 1 -> APIC 0x14 -> Node 0
SRAT: PXM 2 -> APIC 0x04 -> Node 1
SRAT: PXM 1 -> APIC 0x16 -> Node 0
SRAT: PXM 2 -> APIC 0x06 -> Node 1
SRAT: PXM 1 -> APIC 0x11 -> Node 0
SRAT: PXM 2 -> APIC 0x01 -> Node 1
SRAT: PXM 1 -> APIC 0x13 -> Node 0
SRAT: PXM 2 -> APIC 0x03 -> Node 1
SRAT: PXM 1 -> APIC 0x15 -> Node 0
SRAT: PXM 2 -> APIC 0x05 -> Node 1
SRAT: PXM 1 -> APIC 0x17 -> Node 0
SRAT: PXM 2 -> APIC 0x07 -> Node 1
SRAT: Node 1 PXM 2 0-c0000000
SRAT: Node 0 PXM 1 c0000000-d0000000
SRAT: Node 0 PXM 1 100000000-1b0000000
SRAT: Node 0 [c0000000,d0000000) + [100000000,1b0000000) -> [c0000000,1b0000000)
Initmem setup node 0 00000000c0000000-00000001b0000000
NODE_DATA [0000000100000000 - 0000000100004fff]
Initmem setup node 1 0000000000000000-00000000c0000000
NODE_DATA [00000000016ee780 - 00000000016f377f]
Zone PFN ranges:

  DMA      0x00000001 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x001b0000
Movable zone start PFN for each node
early_node_map[4] active PFN ranges
    1: 0x00000001 -> 0x000000a0
    1: 0x00000100 -> 0x000c0000
    0: 0x000c0000 -> 0x000cf699
    0: 0x00100000 -> 0x001b0000
------------[ cut here ]------------
WARNING: at mm/percpu.c:285 pcpu_mem_alloc+0x31/0x78()
Hardware name: PowerEdge R610
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.36-rc2 #1
Call Trace:
 [<ffffffff810305fa>] warn_slowpath_common+0x80/0x98
 [<ffffffff81030627>] warn_slowpath_null+0x15/0x17
 [<ffffffff810a226c>] pcpu_mem_alloc+0x31/0x78
 [<ffffffff810a2fe1>] pcpu_alloc+0x243/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
 [<ffffffff815d29df>] start_kernel+0x88/0x351
 [<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
 [<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
---[ end trace 4eaa2a86a8e2da22 ]---
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G        W   2.6.36-rc2 #1
Call Trace:
 [<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
 [<ffffffff815d29df>] start_kernel+0x88/0x351
 [<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
 [<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
---[ end trace 4eaa2a86a8e2da22 ]---
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G        W   2.6.36-rc2 #1
Call Trace:
 [<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
 [<ffffffff815d29df>] start_kernel+0x88/0x351
 [<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
 [<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G        W   2.6.36-rc2 #1
Call Trace:
 [<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
#22 lore raw

Err... Sorry there was patch under testing in there (remove the boot pageset from the page allocator and use early allocpercpu) which resulted in the reserve area becoming a bit tight. Sorry.

#23 lore raw
In reply to Pekka Enberg

Hi Pekka,

Quoted 4 lines · On Tue, 24 Aug 2010 20:41:36 +0300 Pekka Enberg <[email protected]> wrote:

Thanks for the report. The problem should be fixed by this commit:

http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commitdiff;h=5792949c540c9fd719c3fc0d6edc97374222e792

A small point: I did *not* review that patch. I *did* report the bug (and seemingly provided the commit message).

Thanks for the build fix.

Signature · 3 lines
Cheers,
Stephen Rothwell                    [email protected]
http://www.canb.auug.org.au/~sfr/
#24 lore raw
Quoted 10 lines · Stephen Rothwell

Hi Pekka,

Quoted 4 lines · On Tue, 24 Aug 2010 20:41:36 +0300 Pekka Enberg<[email protected]> wrote:

Thanks for the report. The problem should be fixed by this commit:

http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commitdiff;h=5792949c540c9fd719c3fc0d6edc97374222e792

A small point: I did *not* review that patch. I *did* report the bug (and seemingly provided the commit message).

Oh, sorry about that.

Quoted 1 line

Thanks for the build fix.

I dropped slub cleanups from -next until we've resolved the UP boot time crash problem.

#25 lore raw
Quoted 3 lines · Pekka Enberg

I dropped slub cleanups from -next until we've resolved the UP boot time crash problem.

It works here with this patch. I'd rather see a general solution so that we can use allocpercpu for bootstrapping any allocator without such special bandaids.

Subject: Slub: UP bandaid

Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

---
 mm/slub.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
mm/slub.c +16 -0
Index: linux-2.6/mm/slub.c
===================================================================
@@ -2064,8 +2064,24 @@
 
 static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
 {
+#ifdef CONFIG_SMP
+	/*
+	 * Will use reserve that does not require slab operation during
+	 * early boot.
+	 */
 	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
 			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
+#else
+	/*
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);
+	else
+#endif
 
 	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
#26 lore raw
Quoted 26 lines · Christoph Lameter

Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

---
 mm/slub.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Index: linux-2.6/mm/slub.c
===================================================================

mm/slub.c +5 -0
@@ -2064,8 +2064,24 @@
 
 static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
 {
+#ifdef CONFIG_SMP
+	/*
+	 * Will use reserve that does not require slab operation during
+	 * early boot.
+	 */
 	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <

mm/slub.c line 2082

Quoted 10 lines
mm/slub.c +9 -0
 			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
+#else
+	/*
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);

__get_free_pages() takes an order argument.

Quoted 5 lines
mm/slub.c +2 -0
+	else
+#endif
 
 	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
 
#27 lore raw
Quoted 9 lines · David Rientjes

Review

Quoted 7 lines
(quoted diff) +7 -0
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);

__get_free_pages() takes an order argument.

Right. Patch not refreshed after the last tinzy winzy change. Seems that I got a bit rusty.

Subject: Slub: UP bandaid

Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

---
 mm/slub.c |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)
mm/slub.c +16 -0
Index: linux-2.6/mm/slub.c
===================================================================
@@ -2091,8 +2091,24 @@ init_kmem_cache_node(struct kmem_cache_n
 
 static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
 {
+#ifdef CONFIG_SMP
+	/*
+	 * Will use reserve that does not require slab operation during
+	 * early boot.
+	 */
 	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
 			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
+#else
+	/*
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)__get_free_pages(GFP_NOWAIT, 1);
+	else
+#endif
 
 	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
reviewer Acked-by
#28 lore raw
Quoted 15 lines · Christoph Lameter
Quoted 9 lines

mm/slub.c init_kmem_cache_node(struct kmem_cache_n

Quoted 7 lines
mm/slub.c +7 -0
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);

__get_free_pages() takes an order argument.

Right. Patch not refreshed after the last tinzy winzy change. Seems that I got a bit rusty.

Subject: Slub: UP bandaid

Quoted 7 lines

Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

I'm really hoping that we can remove this hack soon when the percpu allocator can handle these allocations on UP without any specialized slab behavior.

Acked-by David Rientjes
#29 lore raw
Quoted 3 lines · David Rientjes

I'm really hoping that we can remove this hack soon when the percpu allocator can handle these allocations on UP without any specialized slab behavior.

So do I. Here is a slightly less hacky version through using kmalloc_large instead:

Subject: Slub: UP bandaid

Since the percpu allocator does not provide early allocation in UP mode (only in SMP configurations) use __get_free_page() to improvise a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

---
 mm/slub.c |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)
mm/slub.c +16 -0
Index: linux-2.6/mm/slub.c
===================================================================
@@ -2103,8 +2103,24 @@ init_kmem_cache_node(struct kmem_cache_n
 
 static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
 {
+#ifdef CONFIG_SMP
+	/*
+	 * Will use reserve that does not require slab operation during
+	 * early boot.
+	 */
 	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
 			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
+#else
+	/*
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)kmalloc_large(PAGE_SIZE << 1, GFP_NOWAIT);
+	else
+#endif
 
 	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
#30 lore raw
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · David Rientjes

I'm really hoping that we can remove this hack soon when the percpu allocator can handle these allocations on UP without any specialized slab behavior.

So do I. Here is a slightly less hacky version through using kmalloc_large instead:

Applied and queued to -next.