slab.c cleanup
Quoted 8 lines · Anton
Quoted 1 line
- Don't create caches that are not multiples of
L1_CACHE_BYTES.Nice idea, I often see the list walk (of the cache sizes) in kmalloc in kernel profiles. eg a bunch of kmalloc(2k) for network drivers.
Since we have a 128byte cacheline on ppc64 this patch should reduce that.
No, the patch is a bad thing: It means that everyone who does kmalloc(32,) now allocates 128 bytes, i.e. 3/4 wasted. IMHO not acceptable.
I agree that the list walk in kmalloc is a problem - but the right solution is a quicker lookup.
Perhaps something like the attached patch.
For networking: I bet that virtually all allocations have the same size - what about calling kmem_find_general_cachep, and caching the last request/result? Then the next calls can allocate directly from the appropriate cache, without any lookup.
--
Manfred
mm/slab.c +19 -1
@@ -424,6 +430,7 @@ CN("size-131072") };#undef CN +static struct cache_sizes *malloc_hints[sizeof (size_t )*8 ];struct arraycache_init initarray_cache__initdata = { {0 ,BOOT_CPUCACHE_ENTRIES ,1 ,0 } };struct arraycache_init initarray_generic__initdata = { {0 ,BOOT_CPUCACHE_ENTRIES ,1 ,0 } };
@@ -587,6 +594,7 @@void __init kmem_cache_init (void ) {size_t left_over; +int i;init_MUTEX (&cache_chain_sem);INIT_LIST_HEAD (&cache_chain);
@@ -604,6 +612,18 @@* that initializes ac_data for all new cpus */ register_cpu_notifier (&cpucache_notifier); + +for (i=0 ;i<sizeof (size_t )*8 ;i++) { +struct cache_sizes *csizep = malloc_sizes; +int size = (1 <<i)+1 ; + +for ( ; csizep->cs_size; csizep++) { +if (size > csizep->cs_size) +continue ; +break ; + } + malloc_hints[i] = csizep; + } }
@@ -1796,7 +1816,11 @@*/ void *kmalloc (size_t size,int flags) { -struct cache_sizes *csizep = malloc_sizes; +struct cache_sizes *csizep; + +if (unlikely (size <2 )) + size =2 ; + csizep = malloc_hints[fls ((size-1 ))-1 ];for (; csizep->cs_size; csizep++) {if (size > csizep->cs_size)
Quoted 5 lines · Manfred Spraul
Quoted 3 lines · Anton
Quoted 1 line
- Don't create caches that are not multiples of
L1_CACHE_BYTES.
Quoted 7 lines
Quoted 6 lines
Nice idea, I often see the list walk (of the cache sizes) in kmalloc in kernel profiles. eg a bunch of kmalloc(2k) for network drivers.
Since we have a 128byte cacheline on ppc64 this patch should reduce that.
No, the patch is a bad thing: It means that everyone who does
kmalloc(32,)now allocates 128 bytes, i.e. 3/4 wasted. IMHO not acceptable.
Perhaps, but it currently is already allocating 128 bytes for smaller caches, because the cache is created with SLAB_HWCACHE_ALIGN. So we ended up with redundantly sized caches.
--
Brian Gerst
Quoted 3 lines · Brian Gerst
Perhaps, but it currently is already allocating 128 bytes for smaller caches, because the cache is created with
SLAB_HWCACHE_ALIGN. So we ended up with redundantly sized caches.
linux/mm/slab.c:
Quoted 9 lines
if (flags & SLAB_HWCACHE_ALIGN) {
/* Need to adjust size so that objs are cache aligned. */
/* Small obj size, can get at least two per cache line. */
while (size < align/2)
align /= 2;
size = (size+align-1)&(~(align-1));
}HWALIGN is just a hint, the implementation ignores it if it results in unreasonable wasting of memory.
--
Manfred
Quoted 17 lines · Manfred Spraul
Quoted 3 lines · Brian Gerst
Perhaps, but it currently is already allocating 128 bytes for smaller caches, because the cache is created with
SLAB_HWCACHE_ALIGN. So we ended up with redundantly sized caches.
linux/mm/slab.c:Quoted 9 lines
if (flags & SLAB_HWCACHE_ALIGN) { /* Need to adjust size so that objs are cache aligned. */ /* Small obj size, can get at least two per cache line. */ while (size < align/2) align /= 2; size = (size+align-1)&(~(align-1)); }
Quoted 1 line
HWALIGN is just a hint, the implementation ignores it if it results in unreasonable wasting of memory.
I think I see what was causing be to believe it was always rounding up to the cache size. The while test should be while (size <= align/2). On my machine (athlon, 64 byte cache), the size-32 cache was rounded up to 64 bytes because of this.
size-128 1416 1470 128 49 49 1 : 248 124
size-64 351 413 64 7 7 1 : 248 124
size-32 649 649 64 11 11 1 : 248 124
^^^^--
Brian Gerst