build error: sparsemem + SLOB

LD .tmp_vmlinux1
mm/built-in.o: In function `sparse_index_init':
sparse.c:(.text.sparse_index_init+0x19): undefined reference to `slab_is_available'
make: *** [.tmp_vmlinux1] Error 1

mm/sparse.c: line 35 uses slab_is_available() but SLAB=n, SLOB=y.

Quoted 7 lines

LD .tmp_vmlinux1
mm/built-in.o: In function `sparse_index_init':
sparse.c:(.text.sparse_index_init+0x19): undefined reference to `slab_is_available'
make: *** [.tmp_vmlinux1] Error 1

mm/sparse.c: line 35 uses slab_is_available() but SLAB=n, SLOB=y.

Hmm.
To be honest, I don't know that embedded people would like to use sparsemem. mem_section[] array might be too big for them.
But, they may expect reduce power consumption of memory by memory hotplug.

Anyway, this is fix.

------------
This patch is to fix compile error when SLOB is on with sparsemem.

LD .tmp_vmlinux1
mm/built-in.o: In function `sparse_index_init':
sparse.c:(.text.sparse_index_init+0x19): undefined reference to `slab_is_available'
make: *** [.tmp_vmlinux1] Error 1

mm/sparse.c: line 35 uses slab_is_available() but SLAB=n, SLOB=y.

This patch is for 2.6.19-rc5-mm2.

Signed-off-by Yasunori Goto <[email protected]>

------------

 include/linux/slab.h |    9 +++++++++
 mm/slob.c            |    4 ++++
 mm/sparse.c          |    2 +-
 3 files changed, 14 insertions(+), 1 deletion(-)

Index: 19-rc5-mm2/include/linux/slab.h
===================================================================

include/linux/slab.h +9 -0

@@ -210,6 +210,8 @@ static inline void *kcalloc(size_t n, si
 	return kzalloc(n * size, flags);
 }
 
+#define kmem_is_available() slab_is_available()
+
 extern void kfree(const void *);
 extern unsigned int ksize(const void *);
 extern int slab_is_available(void);

@@ -260,6 +262,13 @@ extern void kmem_set_shrinker(kmem_cache
 #else /* CONFIG_SLOB */
 
 /* SLOB allocator routines */
+extern int slob_initialized;
+
+#define kmem_is_available() slob_is_available()
+static inline int slob_is_available(void)
+{
+	return slob_initialized;
+}
 
 void kmem_cache_init(void);
 struct kmem_cache *kmem_cache_create(const char *c, size_t, size_t,

Index: 19-rc5-mm2/mm/slob.c
===================================================================

mm/slob.c +4 -0

@@ -61,6 +61,8 @@ static DEFINE_SPINLOCK(block_lock);
 
 static void slob_free(void *b, int size);
 
+int slob_initialized= 0;
+
 static void *slob_alloc(size_t size, gfp_t gfp, int align)
 {
 	slob_t *prev, *cur, *aligned = 0;

@@ -337,4 +339,6 @@ void kmem_cache_init(void)
 		free_page((unsigned long)p);
 
 	mod_timer(&slob_timer, jiffies + HZ);
+	slob_initialized = 1;
+
 }

Index: 19-rc5-mm2/mm/sparse.c
===================================================================

mm/sparse.c +1 -1

@@ -50,7 +50,7 @@ static struct mem_section *sparse_index_
 	unsigned long array_size = SECTIONS_PER_ROOT *
 				   sizeof(struct mem_section);
 
-	if (slab_is_available())
+	if (kmem_is_available())
 		section = kmalloc_node(array_size, GFP_KERNEL, nid);
 	else
 		section = alloc_bootmem_node(NODE_DATA(nid), array_size);

mm/sparse.c: line 35 uses slab_is_available() but SLAB=n, SLOB=y.

Quoted 1 line · Randy Dunlap

mm/sparse.c: line 35 uses slab_is_available() but SLAB=n, SLOB=y.

I wonder if its worth bothering about SLOB?

As far as I can tell SLOB is fundamentally racy since it does not support SLAB_DESTROY_BY_RCU correctly. F.e. The constructor for the anon_vma will be called on alloc without regard for RCU, we free an item and reuse it without regard to RCU. This can potentially mess up the anon_vma locking state while we access it.

Is SLOB used at all or have we been lucky so far?

As far as I can tell SLOB is fundamentally racy since it does not support SLAB_DESTROY_BY_RCU correctly. F.e. The constructor for the anon_vma will be called on alloc without regard for RCU, we free an item and reuse it without regard to RCU. This can potentially mess up the anon_vma locking state while we access it.

Is SLOB used at all or have we been lucky so far?

Quoted 6 lines · Christoph Lameter

As far as I can tell SLOB is fundamentally racy since it does not support SLAB_DESTROY_BY_RCU correctly. F.e. The constructor for the anon_vma will be called on alloc without regard for RCU, we free an item and reuse it without regard to RCU. This can potentially mess up the anon_vma locking state while we access it.

Good find!

Quoted 1 line

Is SLOB used at all or have we been lucky so far?

Lucky so far. Well, we'd actually have to be quite unlucky to ever see what page_lock_anon_vma/SLAB_DESTROY_BY_RCU are guarding against.

But you're absolutely right that users should not be exposed to such unsafety. I'd say SLOB should be disallowed if SMP.

Quoted 15 lines · Christoph Lameter

Quoted 6 lines

As far as I can tell SLOB is fundamentally racy since it does not support SLAB_DESTROY_BY_RCU correctly. F.e. The constructor for the anon_vma will be called on alloc without regard for RCU, we free an item and reuse it without regard to RCU. This can potentially mess up the anon_vma locking state while we access it.

Good find!

Quoted 1 line

Is SLOB used at all or have we been lucky so far?

Lucky so far. Well, we'd actually have to be quite unlucky to ever see what page_lock_anon_vma/SLAB_DESTROY_BY_RCU are guarding against.

Quoted 2 lines

But you're absolutely right that users should not be exposed to such unsafety. I'd say SLOB should be disallowed if SMP.

SLOB is an O(N) allocator and is pretty poorly suited to running on anything like a modern desktop. Disallowing if SMP is probably reasonable, as even machines with multicore ARM or MIPS will probably have enough memory to make SLOB a bit painful.

Quoted 16 lines · Hugh Dickins

Quoted 15 lines · Christoph Lameter

Quoted 6 lines

As far as I can tell SLOB is fundamentally racy since it does not support SLAB_DESTROY_BY_RCU correctly. F.e. The constructor for the anon_vma will be called on alloc without regard for RCU, we free an item and reuse it without regard to RCU. This can potentially mess up the anon_vma locking state while we access it.

Good find!

Quoted 1 line

Is SLOB used at all or have we been lucky so far?

Lucky so far. Well, we'd actually have to be quite unlucky to ever see what page_lock_anon_vma/SLAB_DESTROY_BY_RCU are guarding against.

Quoted 7 lines

Quoted 2 lines

But you're absolutely right that users should not be exposed to such unsafety. I'd say SLOB should be disallowed if SMP.

SLOB is an O(N) allocator and is pretty poorly suited to running on anything like a modern desktop. Disallowing if SMP is probably reasonable, as even machines with multicore ARM or MIPS will probably have enough memory to make SLOB a bit painful.

Ok. It's simple. This is fix.

----

This is to disallow to make SLOB with SMP or SPARSEMEM. This avoids latent troubles of SLOB with SLAB_DESTROY_BY_RCU. And fix compile error.

This patch is for 2.6.19-rc5-mm2.

Signed-off-by Yasunori Goto <[email protected]>

----

 init/Kconfig |    2 +-
 1 files changed, 1 insertion(+), 1 deletion(-)

Index: 19-rc5-mm2/init/Kconfig
===================================================================

init/Kconfig +1 -1

@@ -465,7 +465,7 @@ config SHMEM
 
 config SLAB
 	default y
-	bool "Use full SLAB allocator" if EMBEDDED
+	bool "Use full SLAB allocator" if (EMBEDDED && !SMP && !SPARSEMEM)
 	help
 	  Disabling this replaces the advanced SLAB allocator and
 	  kmalloc support with the drastically simpler SLOB allocator.

Signature · 1 line

-- 

Ok. It's simple. This is fix.

Quoted 19 lines · Hugh Dickins

Quoted 17 lines · Christoph Lameter

Quoted 6 lines

As far as I can tell SLOB is fundamentally racy since it does not support SLAB_DESTROY_BY_RCU correctly. F.e. The constructor for the anon_vma will be called on alloc without regard for RCU, we free an item and reuse it without regard to RCU. This can potentially mess up the anon_vma locking state while we access it.

Good find!

Quoted 1 line

Is SLOB used at all or have we been lucky so far?

Lucky so far. Well, we'd actually have to be quite unlucky to ever see what page_lock_anon_vma/SLAB_DESTROY_BY_RCU are guarding against.

But you're absolutely right that users should not be exposed to such unsafety. I'd say SLOB should be disallowed if SMP.

Quoted 4 lines

SLOB is an O(N) allocator and is pretty poorly suited to running on anything like a modern desktop. Disallowing if SMP is probably reasonable, as even machines with multicore ARM or MIPS will probably have enough memory to make SLOB a bit painful.

Ok. It's simple. This is fix.

Are there any implications for preemptible kernels here?

Quoted 1 line · Matt Mackall

Are there any implications for preemptible kernels here?

Matt: Would you mind if I replace SLOB with my new slab design? It is as memory efficient as yours (maybe even more since we do not have the header for each allocation). It can work efficiently in an SMP system (NUMA still under test). The code is more complex though. Still needs to mature.

Are there any implications for preemptible kernels here?

Lucky so far. Well, we'd actually have to be quite unlucky to ever see what page_lock_anon_vma/SLAB_DESTROY_BY_RCU are guarding against.

Quoted 2 lines · Hugh Dickins

Lucky so far. Well, we'd actually have to be quite unlucky to ever see what page_lock_anon_vma/SLAB_DESTROY_BY_RCU are guarding against.

Hmmm... I had to repeatedly fix my new slab code when I broke DESTROY_BY_RCU. The machine wont even boot if that is not done right.