slqb: fix small zero size alloc bug

Fix a problem where SLQB did not correctly return ZERO_SIZE_PTR for a zero sized allocation.

---
 include/linux/slqb_def.h |    2 +-
 mm/slqb.c                |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

Index: linux-2.6/include/linux/slqb_def.h
===================================================================
@@ -237,7 +237,7 @@ static __always_inline struct kmem_cache
 
 	index = kmalloc_index(size);
 	if (unlikely(index == 0))
-		return NULL;
+		return ZERO_SIZE_PTR;
 
 	if (likely(!(flags & SLQB_DMA)))
 		return &kmalloc_caches[index];

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Well I promised to improve this situation before slqb can go upstream, so here it is.

It would be nice to keep this changeset in history if it gets merged upstream. I don't know for sure if there won't be a performance impact.

--

Implement dynamic allocation for SLQB per-cpu and per-node arrays. This should hopefully have minimal runtime performance impact, because although there is an extra level of indirection to do allocations, the pointer should be in the cache hot area of the struct kmem_cache.

It's not quite possible to use dynamic percpu allocator for this: firstly, that subsystem uses the slab allocator. Secondly, it doesn't have good support for per-node data. If those problems were improved, we could use it. For now, just implement a very very simple allocator until the kmalloc caches are up.

On x86-64 with a NUMA MAXCPUS config, sizes look like this:

   text    data     bss     dec     hex filename
  29960  259565     100  289625   46b59 mm/slab.o
  34130  497130     696  531956   81df4 mm/slub.o

24575 1634267 111136 1769978 1b01fa mm/slqb.o
24845 13959 712 39516 9a5c mm/slqb.o + this patch

SLQB is now 2 orders of magnitude smaller than it was, and an order of magnitude smaller than SLAB or SLUB (in total size -- text size has always been smaller). So it should now be very suitable for distro-type configs in this respect.

As a side-effect the UP version of cpu_slab (which is embedded directly in the kmem_cache struct) moves up to the hot cachelines, so it need no longer be cacheline aligned on UP. The overall result should be a reduction in cacheline footprint on UP kernels.

---
 include/linux/slqb_def.h |   21 ++++----
 mm/slqb.c                |  117 +++++++++++++++++++++++++++++++++++------------
 2 files changed, 99 insertions(+), 39 deletions(-)

Index: linux-2.6/include/linux/slqb_def.h
===================================================================
@@ -111,7 +111,7 @@ struct kmem_cache_cpu {
 	struct kmlist		rlist;
 	struct kmem_cache_list	*remote_cache_list;
 #endif
-} ____cacheline_aligned;
+} ____cacheline_aligned_in_smp;
 
 /*
  * Per-node, per-kmem_cache structure. Used for node-specific allocations.

@@ -128,10 +128,19 @@ struct kmem_cache {
 	unsigned long	flags;
 	int		hiwater;	/* LIFO list high watermark */
 	int		freebatch;	/* LIFO freelist batch flush size */
+#ifdef CONFIG_SMP
+	struct kmem_cache_cpu	**cpu_slab; /* dynamic per-cpu structures */
+#else
+	struct kmem_cache_cpu	cpu_slab;
+#endif
 	int		objsize;	/* Size of object without meta data */
 	int		offset;		/* Free pointer offset. */
 	int		objects;	/* Number of objects in slab */
 
+#ifdef CONFIG_NUMA
+	struct kmem_cache_node	**node_slab; /* dynamic per-node structures */
+#endif
+
 	int		size;		/* Size of object including meta data */
 	int		order;		/* Allocation order */
 	gfp_t		allocflags;	/* gfp flags to use on allocation */

@@ -148,15 +157,7 @@ struct kmem_cache {
 #ifdef CONFIG_SLQB_SYSFS
 	struct kobject	kobj;		/* For sysfs */
 #endif
-#ifdef CONFIG_NUMA
-	struct kmem_cache_node	*node[MAX_NUMNODES];
-#endif
-#ifdef CONFIG_SMP
-	struct kmem_cache_cpu	*cpu_slab[NR_CPUS];
-#else
-	struct kmem_cache_cpu	cpu_slab;
-#endif
-};
+} ____cacheline_aligned;
 
 /*
  * Kmalloc subsystem.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Hi Nick,

The 'topic/slqb/core' branch in slab.git is append-only so when you want to merge this with Linus, we can always just ask him to pull. I don't see much reason to fiddle with the history. After all, it's hidden under a new config option so it's not like it's going to break git bisect or anything.

I think Christoph's percpu allocator is supposed to fix it up for SLUB. We can probably switch SLQB over as well when that work hits mainline.

Applied, thanks!

Pekka

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Applied, thanks!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>