Skip to contentSkip to conversation

slab.c cleanup

PATCH Manfred Spraul 2 participants 4 messages Mar 23, 2003 Memory management

lore
Share & tools
lore
#0 lore raw
Quoted 8 lines · Anton
Quoted 1 line

- Don't create caches that are not multiples of L1_CACHE_BYTES.

Nice idea, I often see the list walk (of the cache sizes) in kmalloc in kernel profiles. eg a bunch of kmalloc(2k) for network drivers.

Since we have a 128byte cacheline on ppc64 this patch should reduce that.

No, the patch is a bad thing: It means that everyone who does kmalloc(32,) now allocates 128 bytes, i.e. 3/4 wasted. IMHO not acceptable.

I agree that the list walk in kmalloc is a problem - but the right solution is a quicker lookup.
Perhaps something like the attached patch.
For networking: I bet that virtually all allocations have the same size - what about calling kmem_find_general_cachep, and caching the last request/result? Then the next calls can allocate directly from the appropriate cache, without any lookup.

--
Manfred

mm/slab.c +19 -1
@@ -424,6 +430,7 @@
 	CN("size-131072")
 }; 
 #undef CN
+static struct cache_sizes *malloc_hints[sizeof(size_t)*8];
 
 struct arraycache_init initarray_cache __initdata = { { 0, BOOT_CPUCACHE_ENTRIES, 1, 0} };
 struct arraycache_init initarray_generic __initdata = { { 0, BOOT_CPUCACHE_ENTRIES, 1, 0} };
@@ -587,6 +594,7 @@
 void __init kmem_cache_init(void)
 {
 	size_t left_over;
+	int i;
 
 	init_MUTEX(&cache_chain_sem);
 	INIT_LIST_HEAD(&cache_chain);
@@ -604,6 +612,18 @@
 	 * that initializes ac_data for all new cpus
 	 */
 	register_cpu_notifier(&cpucache_notifier);
+
+	for (i=0;i<sizeof(size_t)*8;i++) {
+		struct cache_sizes *csizep = malloc_sizes;
+		int size = (1<<i)+1;
+
+		for ( ; csizep->cs_size; csizep++) {
+			if (size > csizep->cs_size)
+				continue;
+			break;
+		}
+		malloc_hints[i] = csizep;
+	}
 }
 
 
@@ -1796,7 +1816,11 @@
  */
 void * kmalloc (size_t size, int flags)
 {
-	struct cache_sizes *csizep = malloc_sizes;
+	struct cache_sizes *csizep;
+
+	if (unlikely(size < 2))
+		size = 2;
+	csizep = malloc_hints[fls((size-1))-1];
 
 	for (; csizep->cs_size; csizep++) {
 		if (size > csizep->cs_size)
#1 lore raw
Quoted 5 lines · Manfred Spraul
Quoted 3 lines · Anton
Quoted 1 line

- Don't create caches that are not multiples of L1_CACHE_BYTES.

Quoted 7 lines
Quoted 6 lines

Nice idea, I often see the list walk (of the cache sizes) in kmalloc in kernel profiles. eg a bunch of kmalloc(2k) for network drivers.

Since we have a 128byte cacheline on ppc64 this patch should reduce that.

No, the patch is a bad thing: It means that everyone who does kmalloc(32,) now allocates 128 bytes, i.e. 3/4 wasted. IMHO not acceptable.

Perhaps, but it currently is already allocating 128 bytes for smaller caches, because the cache is created with SLAB_HWCACHE_ALIGN. So we ended up with redundantly sized caches.

--
Brian Gerst

#2 lore raw
Quoted 3 lines · Brian Gerst

Perhaps, but it currently is already allocating 128 bytes for smaller caches, because the cache is created with SLAB_HWCACHE_ALIGN. So we ended up with redundantly sized caches.

linux/mm/slab.c:

Quoted 9 lines
	if (flags & SLAB_HWCACHE_ALIGN) {
		/* Need to adjust size so that objs are cache aligned. */
		/* Small obj size, can get at least two per cache line. */
		while (size < align/2)
			align /= 2;
		size = (size+align-1)&(~(align-1));
	}

HWALIGN is just a hint, the implementation ignores it if it results in unreasonable wasting of memory.

--
Manfred

#3 lore raw
Quoted 17 lines · Manfred Spraul
Quoted 3 lines · Brian Gerst

Perhaps, but it currently is already allocating 128 bytes for smaller caches, because the cache is created with SLAB_HWCACHE_ALIGN. So we ended up with redundantly sized caches.

linux/mm/slab.c:

Quoted 9 lines
    if (flags & SLAB_HWCACHE_ALIGN) {
        /* Need to adjust size so that objs are cache aligned. */
        /* Small obj size, can get at least two per cache line. */
        while (size < align/2)
            align /= 2;
        size = (size+align-1)&(~(align-1));
    }
Quoted 1 line

HWALIGN is just a hint, the implementation ignores it if it results in unreasonable wasting of memory.

I think I see what was causing be to believe it was always rounding up to the cache size. The while test should be while (size <= align/2). On my machine (athlon, 64 byte cache), the size-32 cache was rounded up to 64 bytes because of this.

size-128            1416   1470    128   49   49    1 :  248  124
size-64              351    413     64    7    7    1 :  248  124
size-32              649    649     64   11   11    1 :  248  124
                                   ^^^^

--
Brian Gerst