The Enhanced(hopefully) Slab Allocator
Subsystems (3)
v2 cover The Enhanced(hopefully) Slab Allocator
(V2 some more work as time permitted this week)
SLEB is a merging of SLUB with some queuing concepts from SLAB and a new way of managing objects in the slabs using bitmaps. It uses a percpu queue so that free operations can be properly buffered and a bitmap for managing the free/allocated state in the slabs. It is slightly more inefficient than SLUB (due to the need to place large bitmaps --sized a few words--in some slab pages if there are more than BITS_PER_LONG objects in a slab page) but in general does compete well with SLUB (and therefore also with SLOB) in terms of memory wastage.
It does not have the excessive memory requirements of SLAB because there is no slab management structure nor alien caches. Under NUMA the remote shared caches are used instead (which may have its issues).
The SLAB scheme of not touching the object during management is adopted. SLEB can efficiently free and allocate cache cold objects without causing cache misses.
There are numerous SLAB schemes that are not supported. Those could be added if needed and if they really make a difference.
WARNING: This only ran successfully using hackbench in kvm instances so far. But works with NUMA, SMP and UP there.
V1->V2 Add NUMA capabilities. Refine queue size configurations (not complete).
Test in UP, SMP, NUMA
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 1/14 slab: Introduce a constant for a unspecified node.
kmalloc_node() and friends can be passed a constant -1 to indicate that no choice was made for the node from which the object needs to come.
Add a constant for this.
--- include/linux/slab.h | 2 ++ mm/slub.c | 10 +++++----- 2 files changed, 7 insertions(+), 5 deletions(-)
include/linux/slab.h +2 -0
@@ -92,6 +92,8 @@#define ZERO_OR_NULL_PTR (x) ((unsigned long)(x) <= \ (unsigned long)ZERO_SIZE_PTR) +#define SLAB_NODE_UNSPECIFIED (-1L) +/* * struct kmem_cache related prototypes */
mm/slub.c +5 -5
@@ -1081,7 +1081,7 @@ static inline struct page *alloc_slab_pa flags |= __GFP_NOTRACK; -if (node ==-1 ) +if (node ==SLAB_NODE_UNSPECIFIED )return alloc_pages (flags, order);else return alloc_pages_node (node, flags, order);
@@ -1731,7 +1731,7 @@ static __always_inline void *slab_alloc(void *kmem_cache_alloc (struct kmem_cache *s,gfp_t gfpflags) { -void *ret =slab_alloc (s, gfpflags,-1 , _RET_IP_); +void *ret =slab_alloc (s, gfpflags,SLAB_NODE_UNSPECIFIED , _RET_IP_);trace_kmem_cache_alloc (_RET_IP_, ret, s->objsize, s->size, gfpflags);
@@ -1742,7 +1742,7 @@ EXPORT_SYMBOL(kmem_cache_alloc);#ifdef CONFIG_TRACING void *kmem_cache_alloc_notrace (struct kmem_cache *s,gfp_t gfpflags) { -return slab_alloc (s, gfpflags,-1 , _RET_IP_); +return slab_alloc (s, gfpflags,SLAB_NODE_UNSPECIFIED , _RET_IP_); }EXPORT_SYMBOL (kmem_cache_alloc_notrace);#endif
@@ -2740,7 +2740,7 @@ void *__kmalloc(size_t size, gfp_t flagsif (unlikely (ZERO_OR_NULL_PTR (s)))return s; - ret =slab_alloc (s, flags,-1 , _RET_IP_); + ret =slab_alloc (s, flags,SLAB_NODE_UNSPECIFIED , _RET_IP_);trace_kmalloc (_RET_IP_, ret, size, s->size, flags);
@@ -3324,7 +3324,7 @@ void *__kmalloc_track_caller(size_t sizeif (unlikely (ZERO_OR_NULL_PTR (s)))return s; - ret =slab_alloc (s, gfpflags,-1 , caller); + ret =slab_alloc (s, gfpflags,SLAB_NODE_UNSPECIFIED , caller);/* Honor the call site pointer we recieved. */ trace_kmalloc (caller, ret, size, s->size, gfpflags);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerRe: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Quoted 6 lines · Christoph Lameter
kmalloc_node()and friends can be passed a constant -1 to indicate that no choice was made for the node from which the object needs to come.Add a constant for this.
I think it would be better to simply use the generic NUMA_NO_NODE for this purpose, which is identical to how hugetlb, pxm mappings, etc, use it to specify no specific node affinity.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Quoted 6 lines · David Rientjes
Quoted 4 lines · Christoph Lameter
kmalloc_node()and friends can be passed a constant -1 to indicate that no choice was made for the node from which the object needs to come.
Quoted 6 lines
Quoted 2 lines
Add a constant for this.
I think it would be better to simply use the generic
NUMA_NO_NODEfor this purpose, which is identical to how hugetlb, pxm mappings, etc, use it to specify no specific node affinity.
Ok will do that in the next release.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Quoted 8 lines · Christoph Lameter
Quoted 6 lines · David Rientjes
Quoted 4 lines · Christoph Lameter
kmalloc_node()and friends can be passed a constant -1 to indicate that no choice was made for the node from which the object needs to come.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Add a constant for this.
I think it would be better to simply use the generic
NUMA_NO_NODEfor this purpose, which is identical to how hugetlb, pxm mappings, etc, use it to specify no specific node affinity.Ok will do that in the next release.
Patches 1-5 are queued for 2.6.36 so please send an incremental patch on top of 'slub/cleanups' branch of slab.git.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerRe: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Quoted 5 lines · Pekka Enberg
Quoted 1 line
Ok will do that in the next release.
Patches 1-5 are queued for 2.6.36 so please send an incremental patch on top of 'slub/cleanups' branch of slab.git.
An incremental patch in this case would change everything that the original patch did, so it'd probably be best to simply revert and queue the updated version.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Hi David,
Quoted 4 lines · Pekka Enberg
Quoted 4 lines
Quoted 1 line
Ok will do that in the next release.
Patches 1-5 are queued for 2.6.36 so please send an incremental patch on top of 'slub/cleanups' branch of slab.git.
Quoted 3 lines · David Rientjes
An incremental patch in this case would change everything that the original patch did, so it'd probably be best to simply revert and queue the updated version.
If I revert it, we end up with two commits instead of one. And I really prefer not to *rebase* a topic branch even though it might be doable for a small tree like slab.git.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerRe: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Quoted 8 lines · Pekka Enberg
Quoted 3 lines
An incremental patch in this case would change everything that the original patch did, so it'd probably be best to simply revert and queue the updated version.
If I revert it, we end up with two commits instead of one. And I really prefer not to *rebase* a topic branch even though it might be doable for a small tree like slab.git.
I commented on improvements for three of the five patches you've added as slub cleanups and Christoph has shown an interest in proposing them again (perhaps seperating patches 1-5 out as a seperate set of cleanups?), so it's probably cleaner to just reset and reapply with the revisions.
Let me know if my suggested changes should be add-on patches to Christoph's first five and I'll come up with a three patch series to do just that.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Hi David,
(I'm LKML and Ingo to CC.)
Quoted 7 lines · Pekka Enberg
Quoted 7 lines
Quoted 3 lines
An incremental patch in this case would change everything that the original patch did, so it'd probably be best to simply revert and queue the updated version.
If I revert it, we end up with two commits instead of one. And I really prefer not to *rebase* a topic branch even though it might be doable for a small tree like slab.git.
Quoted 4 lines · David Rientjes
I commented on improvements for three of the five patches you've added as slub cleanups and Christoph has shown an interest in proposing them again (perhaps seperating patches 1-5 out as a seperate set of cleanups?), so it's probably cleaner to just reset and reapply with the revisions.
As I said, we can probably get away with that in slab.git because we're so small but that doesn't work in general.
If we ignore the fact how painful the actual rebase operation is (there's a 'sleb/core' branch that shares the commits), I don't think the revised history is 'cleaner' by any means. The current patches are known to be good (I've tested them) but if I just replace them, all the testing effort was basically wasted. So if I need to do a git-bisect, for example, I didn't benefit one bit from testing the original patches.
The other issue is patch metadata. If I just nuke the existing patches, I'm also could be dropping important stuff like Tested-by or Reported-by tags. Yes, I realize that in this particular case, there's none but the approach works only as long as you remember exactly what you merged.
There are probably other benefits for larger trees but those two are enough for me to keep my published branches append-only.
Quoted 3 lines · David Rientjes
Let me know if my suggested changes should be add-on patches to Christoph's first five and I'll come up with a three patch series to do just that.
Yes, I really would prefer incremental patches on top of the 'slub/cleanups' branch.
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerRe: [RFC V2 SLEB 01/14] slab: Introduce a constant for a unspecified node.
Quoted 11 lines · Pekka Enberg
As I said, we can probably get away with that in slab.git because we're so small but that doesn't work in general.
If we ignore the fact how painful the actual rebase operation is (there's a 'sleb/core' branch that shares the commits), I don't think the revised history is 'cleaner' by any means. The current patches are known to be good (I've tested them) but if I just replace them, all the testing effort was basically wasted. So if I need to do a git-bisect, for example, I didn't benefit one bit from testing the original patches.
Quoted 9 lines
The other issue is patch metadata. If I just nuke the existing patches, I'm also could be dropping important stuff like Tested-by or Reported-by tags. Yes, I realize that in this particular case, there's none but the approach works only as long as you remember exactly what you merged.
There are probably other benefits for larger trees but those two are enough for me to keep my published branches append-only.
I wasn't really trying to suggest an alternative way to do it for all git trees, I just thought that since Christoph wanted to repropose these changes in another set and given there's no harm in doing it within slab.git right now that you'd have no problem making an exception in this case just for a cleaner history later.
If you'd like to keep a commit that is then completely obsoleted by another commit when it's on the tip of your tree right now and could be reverted with minimal work simply to follow this general principle, that's fine :)
Quoted 7 lines
Quoted 3 lines
Let me know if my suggested changes should be add-on patches to Christoph's first five and I'll come up with a three patch series to do just that.
Yes, I really would prefer incremental patches on top of the 'slub/cleanups' branch.
Ok then, I'll send incremental changes based on my feedback of patches 1-5. Thanks!
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 2/14 SLUB: Constants need UL
In reply to Christoph LameterUL suffix is missing in some constants. Conform to how slab.h uses constants.
--- mm/slub.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/slub.c +2 -2
@@ -170,8 +170,8 @@#define MAX_OBJS_PER_PAGE 65535/* since page.objects is u16 */ /* Internal SLUB flags */ -#define __OBJECT_POISON 0x80000000/* Poison object */ -#define __SYSFS_ADD_DEFERRED 0x40000000/* Not yet visible via sysfs */ +#define __OBJECT_POISON 0x80000000UL/* Poison object */ +#define __SYSFS_ADD_DEFERRED 0x40000000UL/* Not yet visible via sysfs */ static int kmem_size =sizeof (struct kmem_cache );
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 3/14 SLUB: Use kmem_cache flags to detect if Slab is in debugging mode.
In reply to Christoph LameterThe cacheline with the flags is reachable from the hot paths after the percpu allocator changes went in. So there is no need anymore to put a flag into each slab page. Get rid of the SlubDebug flag and use the flags in kmem_cache instead.
--- include/linux/page-flags.h | 1 - mm/slub.c | 33 ++++++++++++--------------------- 2 files changed, 12 insertions(+), 22 deletions(-)
include/linux/page-flags.h +0 -1
@@ -215,7 +215,6 @@ PAGEFLAG(SwapBacked, swapbacked) __CLEAR __PAGEFLAG(SlobFree, slob_free)__PAGEFLAG (SlubFrozen, slub_frozen) -__PAGEFLAG (SlubDebug, slub_debug)/* * Private page markings that may be used by the filesystem that owns the page
mm/slub.c +12 -21
@@ -107,11 +107,17 @@* the fast path and disables lockless freelists. */ +#define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ + SLAB_TRACE | SLAB_DEBUG_FREE) + +static inline int debug_on (struct kmem_cache *s) +{#ifdef CONFIG_SLUB_DEBUG -#define SLABDEBUG 1 +return unlikely (s->flags &SLAB_DEBUG_FLAGS );#else -#define SLABDEBUG 0 +return 0 ;#endif +}/* * Issues still to be resolved:
@@ -1165,9 +1171,6 @@ static struct page *new_slab(struct kmeminc_slabs_node (s,page_to_nid (page), page->objects); page->slab = s; page->flags |=1 << PG_slab; -if (s->flags & (SLAB_DEBUG_FREE |SLAB_RED_ZONE |SLAB_POISON | -SLAB_STORE_USER |SLAB_TRACE )) -__SetPageSlubDebug (page); start =page_address (page);
@@ -1194,14 +1197,13 @@ static void __free_slab(struct kmem_cachint order =compound_order (page);int pages =1 << order; -if (unlikely (SLABDEBUG &&PageSlubDebug (page))) { +if (debug_on (s)) {void *p;slab_pad_check (s, page);for_each_object (p, s,page_address (page), page->objects)check_object (s, page, p,0 ); -__ClearPageSlubDebug (page); }kmemcheck_free_shadow (page,compound_order (page));
@@ -1419,8 +1421,7 @@ static void unfreeze_slab(struct kmem_castat (s, tail ?DEACTIVATE_TO_TAIL :DEACTIVATE_TO_HEAD ); }else {stat (s,DEACTIVATE_FULL ); -if (SLABDEBUG &&PageSlubDebug (page) && - (s->flags &SLAB_STORE_USER )) +if (debug_on (s) && (s->flags &SLAB_STORE_USER ))add_full (n, page); }slab_unlock (page);
@@ -1628,7 +1629,7 @@ load_freelist: object = c->page->freelist;if (unlikely (!object)) goto another_slab; -if (unlikely (SLABDEBUG &&PageSlubDebug (c->page))) +if (debug_on (s)) goto debug; c->freelist =get_freepointer (s, object);
@@ -1787,7 +1788,7 @@ static void __slab_free(struct kmem_cachstat (s,FREE_SLOWPATH );slab_lock (page); -if (unlikely (SLABDEBUG &&PageSlubDebug (page))) +if (debug_on (s)) goto debug; checks_ok:
@@ -3400,16 +3401,6 @@ static void validate_slab_slab(struct km }else printk (KERN_INFO "SLUB %s: Skipped busy slab 0x%p\n" , s->name, page); - -if (s->flags &DEBUG_DEFAULT_FLAGS ) { -if (!PageSlubDebug (page)) -printk (KERN_ERR "SLUB %s: SlubDebug not set " -"on slab 0x%p\n" , s->name, page); - }else { -if (PageSlubDebug (page)) -printk (KERN_ERR "SLUB %s: SlubDebug set on " -"slab 0x%p\n" , s->name, page); - } } static int validate_slab_node(struct kmem_cache *s,
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerRe: [RFC V2 SLEB 03/14] SLUB: Use kmem_cache flags to detect if Slab is in debugging mode.
Quoted 14 lines · Christoph Lameter
Index:
linux-2.6/mm/slub.c
===================================================================mm/slub.c +5
-0@@ -107,11 +107,17 @@* the fast path and disables lockless freelists. */ +#define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ + SLAB_TRACE | SLAB_DEBUG_FREE) + +static inline int debug_on (struct kmem_cache *s) +{#ifdef CONFIG_SLUB_DEBUG
mm/slub.c line 123
Quoted 10 lines
mm/slub.c +3
-2-#define SLABDEBUG 1 +return unlikely (s->flags &SLAB_DEBUG_FLAGS );#else -#define SLABDEBUG 0 +return 0 ;#endif +}/* * Issues still to be resolved:
Nice optimization! I'd recommend a non-generic name for this check, though, such as cache_debug_on().
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 4/14 SLUB: discard_slab_unlock
In reply to Christoph LameterThe sequence of unlocking a slab and freeing occurs multiple times. Put the common into a single function.
--- mm/slub.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
mm/slub.c +10 -6
@@ -1268,6 +1268,13 @@ static __always_inline int slab_trylock(return rc; } +static void discard_slab_unlock (struct kmem_cache *s, +struct page *page) +{ +slab_unlock (page); +discard_slab (s, page); +} +/* * Management of partially allocated slabs */
@@ -1441,9 +1448,8 @@ static void unfreeze_slab(struct kmem_caadd_partial (n, page,1 );slab_unlock (page); }else { -slab_unlock (page);stat (s,FREE_SLAB ); -discard_slab (s, page); +discard_slab_unlock (s, page); } } }
@@ -1826,9 +1832,8 @@ slab_empty:remove_partial (s, page);stat (s,FREE_REMOVE_PARTIAL ); } -slab_unlock (page);stat (s,FREE_SLAB ); -discard_slab (s, page); +discard_slab_unlock (s, page);return ; debug:
@@ -2905,8 +2910,7 @@ int kmem_cache_shrink(struct kmem_cache*/ list_del (&page->lru); n->nr_partial--; -slab_unlock (page); -discard_slab (s, page); +discard_slab_unlock (s, page); }else {list_move (&page->lru, slabs_by_inuse + page->inuse);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 5/14 SLUB: is_kmalloc_cache
In reply to Christoph LameterThe determination if a slab is a kmalloc slab is occurring multiple times.
--- mm/slub.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
mm/slub.c +7 -3
@@ -312,6 +312,11 @@ static inline int oo_objects(struct kmemreturn x.x &OO_MASK ; } +static int is_kmalloc_cache (struct kmem_cache *s) +{ +return (s < kmalloc_caches +KMALLOC_CACHES && s >= kmalloc_caches); +} +#ifdef CONFIG_SLUB_DEBUG /* * Debug settings:
@@ -2076,7 +2081,7 @@ static DEFINE_PER_CPU(struct kmem_cache_static inline int alloc_kmem_cache_cpus (struct kmem_cache *s,gfp_t flags) { -if (s < kmalloc_caches +KMALLOC_CACHES && s >= kmalloc_caches) +if (is_kmalloc_cache (s))/* * Boot time creation of the kmalloc array. Use static per cpu data * since the per cpu allocator is not available yet.
@@ -2158,8 +2163,7 @@ static int init_kmem_cache_nodes(structint node;int local_node; -if (slab_state >=UP && (s < kmalloc_caches || - s >= kmalloc_caches +KMALLOC_CACHES )) +if (slab_state >=UP && !is_kmalloc_cache (s)) local_node =page_to_nid (virt_to_page (s));else local_node =0 ;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerRe: [RFC V2 SLEB 05/14] SLUB: is_kmalloc_cache
Quoted 26 lines · Christoph Lameter
Index:
linux-2.6/mm/slub.c
===================================================================mm/slub.c +6
-1@@ -312,6 +312,11 @@ static inline int oo_objects(struct kmemreturn x.x &OO_MASK ; } +static int is_kmalloc_cache (struct kmem_cache *s) +{ +return (s < kmalloc_caches +KMALLOC_CACHES && s >= kmalloc_caches); +} +#ifdef CONFIG_SLUB_DEBUG /* * Debug settings: @@ -2076,7 +2081,7 @@ static DEFINE_PER_CPU(struct kmem_cache_static inline int alloc_kmem_cache_cpus (struct kmem_cache *s,gfp_t flags) { -if (s < kmalloc_caches +KMALLOC_CACHES && s >= kmalloc_caches) +if (is_kmalloc_cache (s))/* * Boot time creation of the kmalloc array. Use static per cpu data * since the per cpu allocator is not available yet.
mm/slub.c static int init_kmem_cache_nodes(struct · line 2169
Quoted 9 lines
mm/slub.c +1
-2@@ -2158,8 +2163,7 @@ static int init_kmem_cache_nodes(structint node;int local_node; -if (slab_state >=UP && (s < kmalloc_caches || - s >= kmalloc_caches +KMALLOC_CACHES )) +if (slab_state >=UP && !is_kmalloc_cache (s)) local_node =page_to_nid (virt_to_page (s));else local_node =0 ;
Looks good, how about extending it to dma_kmalloc_cache() as well?
---
mm/slub.c +1 -2
@@ -2641,13 +2641,12 @@ static noinline struct kmem_cache *dma_kmalloc_cache(int index, gfp_t flags) text =kasprintf (flags & ~SLUB_DMA ,"kmalloc_dma-%d" , (unsigned int )realsize); - s =NULL ;for (i =0 ; i <KMALLOC_CACHES ; i++)if (!kmalloc_caches[i].size)break ; -BUG_ON (i >=KMALLOC_CACHES ); s = kmalloc_caches + i; +BUG_ON (!is_kmalloc_cache (s));/* * Must defer sysfs creation to a workqueue because we don't know
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 6/14 SLUB: Get rid of the kmalloc_node slab
In reply to Christoph LameterCurrently bootstrap works with the kmalloc_node slab. We can avoid creating that slab and boot using allocation from a kmalloc array slab instead. This is necessary for the future if we want to dynamically size kmem_cache structures.
--- mm/slub.c | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-)
mm/slub.c +24 -15
@@ -2111,10 +2111,11 @@ static void early_kmem_cache_node_alloc(struct page *page;struct kmem_cache_node *n;unsigned long flags; +int i =kmalloc_index (sizeof (struct kmem_cache_node )); -BUG_ON (kmalloc_caches->size <sizeof (struct kmem_cache_node )); +BUG_ON (kmalloc_caches[i].size <sizeof (struct kmem_cache_node )); - page =new_slab (kmalloc_caches, gfpflags, node); + page =new_slab (kmalloc_caches + i, gfpflags, node);BUG_ON (!page);if (page_to_nid (page) != node) {
@@ -2126,15 +2127,15 @@ static void early_kmem_cache_node_alloc( n = page->freelist;BUG_ON (!n); - page->freelist =get_freepointer (kmalloc_caches, n); + page->freelist =get_freepointer (kmalloc_caches + i, n); page->inuse++; - kmalloc_caches->node[node] = n; + kmalloc_caches[i].node[node] = n;#ifdef CONFIG_SLUB_DEBUG -init_object (kmalloc_caches, n,1 ); -init_tracking (kmalloc_caches, n); +init_object (kmalloc_caches + i, n,1 ); +init_tracking (kmalloc_caches + i, n);#endif -init_kmem_cache_node (n, kmalloc_caches); -inc_slabs_node (kmalloc_caches, node, page->objects); +init_kmem_cache_node (n, kmalloc_caches + i); +inc_slabs_node (kmalloc_caches + i, node, page->objects);/* * lockdep requires consistent irq usage for each lock
@@ -2152,8 +2153,9 @@ static void free_kmem_cache_nodes(structfor_each_node_state (node,N_NORMAL_MEMORY ) {struct kmem_cache_node *n = s->node[node]; +if (n && n != &s->local_node) -kmem_cache_free (kmalloc_caches, n); +kfree (n); s->node[node] =NULL ; } }
@@ -2178,8 +2180,8 @@ static int init_kmem_cache_nodes(structearly_kmem_cache_node_alloc (gfpflags, node);continue ; } - n =kmem_cache_alloc_node (kmalloc_caches, - gfpflags, node); + n =kmalloc_node (sizeof (struct kmem_cache_node ), gfpflags, + node);if (!n) {free_kmem_cache_nodes (s);
@@ -2574,6 +2576,12 @@ static struct kmem_cache *create_kmalloc {unsigned int flags =0 ; +if (s->size) { + s->name = name; +/* Already created */ +return s; + } +if (gfp_flags &SLUB_DMA ) flags =SLAB_CACHE_DMA ;
@@ -2978,7 +2986,7 @@ static void slab_mem_offline_callback(voBUG_ON (slabs_node (s, offline_node)); s->node[offline_node] =NULL ; -kmem_cache_free (kmalloc_caches, n); +kfree (n); } }up_read (&slub_lock);
@@ -3011,7 +3019,7 @@ static int slab_mem_going_online_callbac* since memory is not yet available from the node that * is brought up. */ - n =kmem_cache_alloc (kmalloc_caches,GFP_KERNEL ); + n =kmalloc (sizeof (struct kmem_cache_node ),GFP_KERNEL );if (!n) { ret = -ENOMEM ; goto out;
@@ -3068,9 +3076,10 @@ void __init kmem_cache_init(void)* struct kmem_cache_node's. There is special bootstrap code in * kmem_cache_open for slab_state == DOWN. */ -create_kmalloc_cache (&kmalloc_caches[0 ],"kmem_cache_node" , + i =kmalloc_index (sizeof (struct kmem_cache_node )); +create_kmalloc_cache (&kmalloc_caches[i],"bootstrap" ,sizeof (struct kmem_cache_node ),GFP_NOWAIT ); - kmalloc_caches[0 ].refcount =-1 ; + kmalloc_caches[i].refcount =-1 ; caches++;hotplug_memory_notifier (slab_memory_callback,SLAB_CALLBACK_PRI );
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerRe: [RFC V2 SLEB 06/14] SLUB: Get rid of the kmalloc_node slab
Quoted 1 line · Christoph Lameter
Currently bootstrap works with the
kmalloc_nodeslab.
s/kmalloc_node/kmem_cache_node/
Quoted 16 lines
We can avoid
creating that slab and boot using allocation from a kmalloc array slab instead. This is necessary for the future if we want to dynamically sizekmem_cachestructures.Signed-off-by Christoph Lameter <[email protected]>--- mm/slub.c | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-)Index:
linux-2.6/mm/slub.c
===================================================================
mm/slub.c static void early_kmem_cache_node_alloc( · line 2115
Quoted 5 lines
mm/slub.c +1
-0@@ -2111,10 +2111,11 @@ static void early_kmem_cache_node_alloc(struct page *page;struct kmem_cache_node *n;unsigned long flags; +int i =kmalloc_index (sizeof (struct kmem_cache_node ));
const int?
Maybe even better would be
struct kmem_cache *s = kmalloc_caches + i;
to make the rest of this easier?
mm/slub.c static void early_kmem_cache_node_alloc( · line 2129
Quoted 12 lines
mm/slub.c +2
-2-BUG_ON (kmalloc_caches->size <sizeof (struct kmem_cache_node )); +BUG_ON (kmalloc_caches[i].size <sizeof (struct kmem_cache_node )); - page =new_slab (kmalloc_caches, gfpflags, node); + page =new_slab (kmalloc_caches + i, gfpflags, node);BUG_ON (!page);if (page_to_nid (page) != node) {@@ -2126,15 +2127,15 @@ static void early_kmem_cache_node_alloc( n = page->freelist;BUG_ON (!n);
I don't think we need this BUG_ON() anymore, but that's a seperate issue.
Quoted 41 lines
mm/slub.c +10
-9- page->freelist =get_freepointer (kmalloc_caches, n); + page->freelist =get_freepointer (kmalloc_caches + i, n); page->inuse++; - kmalloc_caches->node[node] = n; + kmalloc_caches[i].node[node] = n;#ifdef CONFIG_SLUB_DEBUG -init_object (kmalloc_caches, n,1 ); -init_tracking (kmalloc_caches, n); +init_object (kmalloc_caches + i, n,1 ); +init_tracking (kmalloc_caches + i, n);#endif -init_kmem_cache_node (n, kmalloc_caches); -inc_slabs_node (kmalloc_caches, node, page->objects); +init_kmem_cache_node (n, kmalloc_caches + i); +inc_slabs_node (kmalloc_caches + i, node, page->objects);/* * lockdep requires consistent irq usage for each lock @@ -2152,8 +2153,9 @@ static void free_kmem_cache_nodes(structfor_each_node_state (node,N_NORMAL_MEMORY ) {struct kmem_cache_node *n = s->node[node]; +if (n && n != &s->local_node) -kmem_cache_free (kmalloc_caches, n); +kfree (n); s->node[node] =NULL ; } }@@ -2178,8 +2180,8 @@ static int init_kmem_cache_nodes(structearly_kmem_cache_node_alloc (gfpflags, node);continue ; } - n =kmem_cache_alloc_node (kmalloc_caches, - gfpflags, node); + n =kmalloc_node (sizeof (struct kmem_cache_node ), gfpflags, + node);if (!n) {free_kmem_cache_nodes (s);
mm/slub.c static struct kmem_cache *create_kmalloc · line 2580
Quoted 5 lines
mm/slub.c +2
-0@@ -2574,6 +2576,12 @@ static struct kmem_cache *create_kmalloc {unsigned int flags =0 ; +if (s->size) { + s->name = name;
Do we need this? The iteration at the end of kmem_cache_init() should reset this kmalloc cache to have the standard kmalloc-<size> name, so I don't think we need to reset "bootstrap" here.
Quoted 28 lines
mm/slub.c +6
-2+/* Already created */ +return s; + } +if (gfp_flags &SLUB_DMA ) flags =SLAB_CACHE_DMA ;@@ -2978,7 +2986,7 @@ static void slab_mem_offline_callback(voBUG_ON (slabs_node (s, offline_node)); s->node[offline_node] =NULL ; -kmem_cache_free (kmalloc_caches, n); +kfree (n); } }up_read (&slub_lock);@@ -3011,7 +3019,7 @@ static int slab_mem_going_online_callbac* since memory is not yet available from the node that * is brought up. */ - n =kmem_cache_alloc (kmalloc_caches,GFP_KERNEL ); + n =kmalloc (sizeof (struct kmem_cache_node ),GFP_KERNEL );if (!n) { ret = -ENOMEM ; goto out;@@ -3068,9 +3076,10 @@ void __init kmem_cache_init(void)* struct kmem_cache_node's. There is special bootstrap code in * kmem_cache_open for slab_state == DOWN.
mm/slub.c void __init kmem_cache_init(void) · line 3085
Quoted 10 lines
mm/slub.c +3
-2*/ -create_kmalloc_cache (&kmalloc_caches[0 ],"kmem_cache_node" , + i =kmalloc_index (sizeof (struct kmem_cache_node )); +create_kmalloc_cache (&kmalloc_caches[i],"bootstrap" ,sizeof (struct kmem_cache_node ),GFP_NOWAIT ); - kmalloc_caches[0 ].refcount =-1 ; + kmalloc_caches[i].refcount =-1 ; caches++;hotplug_memory_notifier (slab_memory_callback,SLAB_CALLBACK_PRI );
So kmalloc_caches[0] will never be used after this change, then?
We could also remove the gfp_t argument to create_kmalloc_cache(), it's not used for anything other than GFP_NOWAIT anymore.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC V2 SLEB 06/14] SLUB: Get rid of the kmalloc_node slab
The patch needs a rework since it sometimes calculates the wrong kmalloc slab. Value needs to be rounded up to the next kmalloc slab size. This problem shows up if CONFIG_SLUB_DEBUG is enabled.
Please do not merge patches that are marked "RFC". That usually means that I am not satisfied with their quality yet.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC V2 SLEB 06/14] SLUB: Get rid of the kmalloc_node slab
Hi Christoph,
Quoted 6 lines · Christoph Lameter
The patch needs a rework since it sometimes calculates the wrong kmalloc slab. Value needs to be rounded up to the next kmalloc slab size. This problem shows up if
CONFIG_SLUB_DEBUGis enabled.Please do not merge patches that are marked "RFC". That usually means that I am not satisfied with their quality yet.
I actually _asked_ you whether or not it's OK to merge patches 1-5. Do you want to guess what you said? I have it all in writing stashed in my mailbox if you want to refresh your memory. ;-)
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC V2 SLEB 06/14] SLUB: Get rid of the kmalloc_node slab
On Wed, Jun 9, 2010 at 7:14 PM, Christoph Lameter
Quoted 7 lines
Quoted 6 lines · <[email protected]> wrote:
The patch needs a rework since it sometimes calculates the wrong kmalloc slab. Value needs to be rounded up to the next kmalloc slab size. This problem shows up if
CONFIG_SLUB_DEBUGis enabled.Please do not merge patches that are marked "RFC". That usually means that I am not satisfied with their quality yet.
Quoted 3 lines · Pekka Enberg
I actually _asked_ you whether or not it's OK to merge patches 1-5. Do you want to guess what you said? I have it all in writing stashed in my mailbox if you want to refresh your memory. ;-)
I only now realized you were talking about patch number six which shouldn't have been merged. Sorry for that, Christoph.
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 7/14 SLEB: The Enhanced Slab Allocator
In reply to Christoph LameterSLEB is a merging of SLUB with some queuing concepts from SLAB and a new way of managing objects in the slabs using bitmaps. It uses a percpu queue so that free operations can be properly buffered and a bitmap for managing the free/allocated state in the slabs. It is slightly more inefficient than SLUB (due to the need to place large bitmaps --sized a few words--in some slab pages) but in general does compete well with SLUBs space use. The storage format avoids the SLAB management structure that SLAB needs for each slab page and therefore the metadata is more compact and easily fits into a cacheline.
The SLAB scheme of not touching the object during management is adopted. SLEB can efficiently free and allocate cache cold objects.
There are numerous SLAB schemes that are not supported. Those could be added if needed and if they really make a difference.
WARNING: This only ran successfully in a kvm instance so far.
--- include/linux/slub_def.h | 11 mm/slub.c | 912 +++++++++++++++++++++-------------------------- 2 files changed, 415 insertions(+), 508 deletions(-)
include/linux/slub_def.h +6 -5
@@ -34,13 +34,16 @@ enum stat_item {ORDER_FALLBACK ,/* Number of times fallback was necessary */ NR_SLUB_STAT_ITEMS }; +#define BOOT_QUEUE_SIZE 50 +#define BOOT_BATCH_SIZE 25 +struct kmem_cache_cpu { -void **freelist;/* Pointer to first free per cpu object */ -struct page *page;/* The slab from which we are allocating */ -int node;/* The node of the page (or -1 for debug) */ #ifdef CONFIG_SLUB_STATS unsigned stat[NR_SLUB_STAT_ITEMS ];#endif +int objects;/* Number of objects available */ +int node;/* The node of the page (or -1 for debug) */ +void *object[BOOT_QUEUE_SIZE ];/* List of objects */ }; struct kmem_cache_node {
@@ -72,9 +75,7 @@ struct kmem_cache {unsigned long flags;int size;/* The size of an object including meta data */ int objsize;/* The size of an object without meta data */ -int offset;/* Free pointer offset. */ struct kmem_cache_order_objects oo; -/* * Avoid an extra cache line for UP, SMP and for the node local to * struct kmem_cache.
mm/slub.c +409 -503
1,417 lines not on this page · raw message
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 8/14 SLEB: Resize cpu queue
In reply to Christoph LameterAllow resizing of cpu queue and batch sizes. Resizing queues is only possible for non kmalloc slabs since kmalloc slabs have statically allocated per cpu queues (avoid bootstap issues) and involves reallocating the per cpu structures. This is done by replicating the basic steps of how SLAB does it.
Careful: This means that the ->cpu_slab pointer is only guaranteed to be stable if interrupts are disabled.
--- include/linux/slub_def.h | 2 mm/slub.c | 152 +++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 143 insertions(+), 11 deletions(-)
mm/slub.c +141 -11
@@ -1521,6 +1521,11 @@ static void flush_cpu_objects(struct kmestat (s,CPUSLAB_FLUSH ); } +struct flush_control { +struct kmem_cache *s; +struct kmem_cache_cpu *c; +}; +/* * Flush cpu objects. *
@@ -1528,24 +1533,77 @@ static void flush_cpu_objects(struct kme*/ static void __flush_cpu_objects (void *d) { -struct kmem_cache *s = d; -struct kmem_cache_cpu *c =__this_cpu_ptr (s->cpu_slab); +struct flush_control *f = d; +struct kmem_cache_cpu *c =__this_cpu_ptr (f->c);if (c->objects) -flush_cpu_objects (s, c); +flush_cpu_objects (f->s, c); }static void flush_all (struct kmem_cache *s) { -on_each_cpu (__flush_cpu_objects, s,1 ); +struct flush_control f = { s, s->cpu_slab}; + +on_each_cpu (__flush_cpu_objects, &f,1 ); }struct kmem_cache_cpu *alloc_kmem_cache_cpu (struct kmem_cache *s,int n) { -return __alloc_percpu (sizeof (struct kmem_cache_cpu ), +return __alloc_percpu ( +sizeof (struct kmem_cache_cpu ) +sizeof (void *) * (n -BOOT_QUEUE_SIZE ), __alignof__(struct kmem_cache_cpu )); } +static void resize_cpu_queue (struct kmem_cache *s,int queue) +{ + +if (is_kmalloc_cache (s)) { +if (queue <BOOT_QUEUE_SIZE ) { + s->queue = queue; +if (s->batch > queue) + s->batch = queue; + }else { +/* More than max. Go to max allowed */ + s->queue =BOOT_QUEUE_SIZE ; + s->batch =BOOT_BATCH_SIZE ; + } + }else { +struct kmem_cache_cpu *n =alloc_kmem_cache_cpu (s, queue); +struct flush_control f; + +/* Create the new cpu queue and then free the old one */ +down_write (&slub_lock); + f.s = s; + f.c = s->cpu_slab; + +/* We can only shrink the queue here since the new +* queue size may be smaller and there may be concurrent +* slab operations. The upate of the queue must be seen +* before the change of the location of the percpu queue. +* +* Note that the queue may contain more object than the +* queue size after this operation. +*/ +if (queue < s->queue) { + s->queue = queue; +barrier (); + } + s->cpu_slab = n; +on_each_cpu (__flush_cpu_objects, &f,1 ); + +/* +* If the queue needs to be extended then we deferred +* the update until now when the larger sized queue +* has been allocated and is working. +*/ +if (queue > s->queue) + s->queue = queue; + +up_write (&slub_lock); +free_percpu (f.c); + } +} +/* * Check if the objects in a per cpu structure fit numa * locality expectations.
@@ -1678,7 +1736,7 @@ redo: c->node = node; } -while (c->objects <BOOT_BATCH_SIZE ) { +while (c->objects < s->batch) {struct page *new;int d;
@@ -1706,7 +1764,7 @@ redo: }else stat (s,ALLOC_FROM_PARTIAL ); - d =min (BOOT_BATCH_SIZE - c->objects,available (new)); + d =min (s->batch - c->objects,available (new));retrieve_objects (s, new, c->object + c->objects, d); c->objects += d;
@@ -1806,9 +1864,9 @@ static void slab_free(struct kmem_cacheif (!(s->flags &SLAB_DEBUG_OBJECTS ))debug_check_no_obj_freed (object, s->objsize); -if (unlikely (c->objects >=BOOT_QUEUE_SIZE )) { +if (unlikely (c->objects >= s->queue)) { -int t =min (BOOT_BATCH_SIZE , c->objects); +int t =min (s->batch, c->objects);drain_objects (s, c->object, t);
@@ -2028,7 +2086,7 @@ static inline int alloc_kmem_cache_cpus( s->cpu_slab = kmalloc_percpu + (s - kmalloc_caches); else - s->cpu_slab = alloc_kmem_cache_cpu(s, BOOT_QUEUE_SIZE); + s->cpu_slab = alloc_kmem_cache_cpu(s, s->queue);if (!s->cpu_slab)return 0 ;
@@ -2263,6 +2321,26 @@ static int calculate_sizes(struct kmem_c } +/* Autotuning of the per cpu queueing */ +void initial_cpu_queue_setup (struct kmem_cache *s) +{ +if (s->size >PAGE_SIZE ) + s->queue =8 ; +else if (s->size >1024 ) + s->queue =24 ; +else if (s->size >256 ) + s->queue =54 ; +else + s->queue =120 ; + +if (is_kmalloc_cache (s) && s->queue >BOOT_QUEUE_SIZE ) { +/* static so cap it */ + s->queue =BOOT_QUEUE_SIZE ; + } + + s->batch = (s->queue +1 ) /2 ; +} + static int kmem_cache_open(struct kmem_cache *s, gfp_t gfpflags, const char *name, size_t size, size_t align, unsigned long flags,
@@ -2298,6 +2376,7 @@ static int kmem_cache_open(struct kmem_c#ifdef CONFIG_NUMA s->remote_node_defrag_ratio =1000 ;#endif +initial_cpu_queue_setup (s);if (!init_kmem_cache_nodes (s, gfpflags & ~SLUB_DMA )) goto error;
@@ -3855,6 +3934,55 @@ static ssize_t min_partial_store(struct }SLAB_ATTR (min_partial); +static ssize_t cpu_queue_size_show (struct kmem_cache *s,char *buf) +{ +return sprintf (buf,"%u\n" , s->queue); +} + +static ssize_t cpu_queue_size_store (struct kmem_cache *s, +const char *buf,size_t length) +{ +unsigned long queue; +int err; + + err =strict_strtoul (buf,10 , &queue); +if (err) +return err; + +if (queue >10000 || queue <4 ) +return -EINVAL ; + +if (s->batch > queue) + s->batch = queue; + +resize_cpu_queue (s, queue); +return length; +} +SLAB_ATTR (cpu_queue_size); + +static ssize_t cpu_batch_size_show (struct kmem_cache *s,char *buf) +{ +return sprintf (buf,"%u\n" , s->batch); +} + +static ssize_t cpu_batch_size_store (struct kmem_cache *s, +const char *buf,size_t length) +{ +unsigned long batch; +int err; + + err =strict_strtoul (buf,10 , &batch); +if (err) +return err; + +if (batch < s->queue || batch <4 ) +return -EINVAL ; + + s->batch = batch; +return length; +} +SLAB_ATTR (cpu_batch_size); +static ssize_t ctor_show (struct kmem_cache *s,char *buf) {if (s->ctor) {
@@ -4204,6 +4332,8 @@ static struct attribute *slab_attrs[] = &objs_per_slab_attr.attr, &order_attr.attr, &min_partial_attr.attr, + &cpu_queue_size_attr.attr, + &cpu_batch_size_attr.attr, &objects_attr.attr, &objects_partial_attr.attr, &total_objects_attr.attr,
@@ -4561,7 +4691,7 @@ static int s_show(struct seq_file *m, voseq_printf (m,"%-17s %6lu %6lu %6u %4u %4d" , s->name, nr_inuse, nr_objs, s->size,oo_objects (s->oo), (1 <<oo_order (s->oo))); -seq_printf (m," : tunables %4u %4u %4u" ,0 ,0 ,0 ); +seq_printf (m," : tunables %4u %4u %4u" , s->queue, s->batch,0 );seq_printf (m," : slabdata %6lu %6lu %6lu" , nr_slabs, nr_slabs,0UL );seq_putc (m,'\n' );
include/linux/slub_def.h +2 -0
@@ -76,6 +76,8 @@ struct kmem_cache {int size;/* The size of an object including meta data */ int objsize;/* The size of an object without meta data */ struct kmem_cache_order_objects oo; +int queue;/* per cpu queue size */ +int batch;/* batch size */ /* * Avoid an extra cache line for UP, SMP and for the node local to * struct kmem_cache.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 9/14 SLED: Get rid of useless function
In reply to Christoph Lametercount_free() == available()
--- mm/slub.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)
mm/slub.c +3 -8
@@ -1617,11 +1617,6 @@ static inline int node_match(struct kmemreturn 1 ; } -static int count_free (struct page *page) -{ -return available (page); -} -static unsigned long count_partial (struct kmem_cache_node *n,int (*get_count)(struct page *)) {
@@ -1670,7 +1665,7 @@ slab_out_of_memory(struct kmem_cache *s,if (!n)continue ; - nr_free =count_partial (n, count_free); + nr_free =count_partial (n, available); nr_slabs =node_nr_slabs (n); nr_objs =node_nr_objs (n);
@@ -3802,7 +3797,7 @@ static ssize_t show_slab_objects(struct x =atomic_long_read (&n->total_objects); else if (flags & SO_OBJECTS) x =atomic_long_read (&n->total_objects) - -count_partial (n, count_free); +count_partial (n, available); else x =atomic_long_read (&n->nr_slabs);
@@ -4683,7 +4678,7 @@ static int s_show(struct seq_file *m, vo nr_partials += n->nr_partial; nr_slabs +=atomic_long_read (&n->nr_slabs); nr_objs +=atomic_long_read (&n->total_objects); - nr_free +=count_partial (n, count_free); + nr_free +=count_partial (n, available); } nr_inuse = nr_objs - nr_free;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 10/14 SLEB: Remove MAX_OBJS limitation
In reply to Christoph LameterThere is no need anymore for the "inuse" field in the page struct. Extend the objects field to 32 bit allowing a practically unlimited number of objects.
--- include/linux/mm_types.h | 5 +---- mm/slub.c | 7 ------- 2 files changed, 1 insertion(+), 11 deletions(-)
include/linux/mm_types.h +1 -4
@@ -40,10 +40,7 @@ struct page {* to show when page is mapped * & limit reverse map searches. */ -struct {/* SLUB */ -u16 inuse; -u16 objects; - }; +u32 objects;/* SLEB */ }; union { struct {
mm/slub.c +0 -7
@@ -152,7 +152,6 @@ static inline int debug_on(struct kmem_c#define OO_SHIFT 16#define OO_MASK ((1 << OO_SHIFT) - 1) -#define MAX_OBJS_PER_PAGE 65535/* since page.objects is u16 */ /* Internal SLUB flags */ #define __OBJECT_POISON 0x80000000UL/* Poison object */
@@ -771,9 +770,6 @@ static int verify_slab(struct kmem_cache max_objects = ((void *)page->freelist - start) / s->size; } -if (max_objects >MAX_OBJS_PER_PAGE ) - max_objects =MAX_OBJS_PER_PAGE ; -if (page->objects != max_objects) {slab_err (s, page,"Wrong number of objects. Found %d but " "should be %d" , page->objects, max_objects);
@@ -1959,9 +1955,6 @@ static inline int slab_order(int size, iint rem;int min_order = slub_min_order; -if ((PAGE_SIZE << min_order) / size >MAX_OBJS_PER_PAGE ) -return get_order (size *MAX_OBJS_PER_PAGE ) -1 ; -for (order =max (min_order,fls (min_objects * size -1 ) -PAGE_SHIFT ); order <= max_order; order++) {
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 11/14 SLEB: Add per node cache (with a fixed size for now)
In reply to Christoph LameterThe per node cache has the function of the shared cache in SLAB. However, it will also perform the role of the alien cache in the future.
If the per cpu queues are exhausted then the shared cache will be consulted first before acquiring objects directly from the slab pages.
On free objects will be first pushed into the shared cache before freeing them directly to the slab pages.
Both methods allows other processes running on the same node to pickup the freed objects that may be cache hot in shared caches. No approximation of the actual topology is done though. It is simply assumed that all processors on a node derive some benefit from acquiring an object that has been used on another processor.
This is an initial static size of the shared cache.
--- include/linux/slub_def.h | 3 +++ mm/slub.c | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+)
include/linux/slub_def.h +3 -0
@@ -55,6 +55,9 @@ struct kmem_cache_node {atomic_long_t total_objects;struct list_head full;#endif +int objects;/* Objects in the per node cache */ +spinlock_t shared_lock;/* Serialization for per node cache */ +void *object[BOOT_QUEUE_SIZE ]; };/*
mm/slub.c +41 -0
59 lines not on this page · raw message
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 12/14 SLEB: Make the size of the shared cache configurable
In reply to Christoph LameterThis makes the size of the shared array configurable. Not that this is a bit problematic and there are likely unresolved race conditions. The kmem_cache->node[x] pointers become unstable if interrupts are allowed.
--- include/linux/slub_def.h | 3 + mm/slub.c | 133 +++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 116 insertions(+), 20 deletions(-)
include/linux/slub_def.h +3 -0
@@ -81,11 +81,14 @@ struct kmem_cache {struct kmem_cache_order_objects oo;int queue;/* per cpu queue size */ int batch;/* batch size */ +int shared;/* Shared queue size */ +#ifndef CONFIG_NUMA /* * Avoid an extra cache line for UP, SMP and for the node local to * struct kmem_cache. */ struct kmem_cache_node local_node; +#endif /* Allocation and freeing of slabs */ struct kmem_cache_order_objects max;
mm/slub.c +113 -20
182 lines not on this page · raw message
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 13/14 SLEB: Enhanced NUMA support
In reply to Christoph LameterBefore this patch all queues in SLEB may contain mixed objects (from any node). This will continue even with this patch unless the SLAB has SLAB_MEM_SPREAD set.
For SLAB_MEM_SPREAD slabs an ordering by locality is enforced and objects are managed per NUMA node (like SLAB). Cpu queues only contain objects from the local node. Alien Objects (from non local nodes) are freed into the shared cache of the remote node (avoids alien caches but introduces cold cache objects into the shared cache).
This also adds object level NUMA functionality like in SLAB that can be managed via cpusets or memory policies.
--- mm/slub.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+)
mm/slub.c +70 -0
@@ -1718,6 +1718,24 @@ void retrieve_objects(struct kmem_cache } } +static inline int find_numa_node (struct kmem_cache *s,int selected_node) +{ +#ifdef CONFIG_NUMA +if (s->flags &SLAB_MEM_SPREAD && + !in_interrupt () && + selected_node ==SLAB_NODE_UNSPECIFIED ) { + +if (cpuset_do_slab_mem_spread ()) +return cpuset_mem_spread_node (); + +if (current->mempolicy) +return slab_node (current->mempolicy); + } +#endif +return selected_node; +} + +static void *slab_alloc (struct kmem_cache *s,gfp_t gfpflags,int node,unsigned long addr) {
@@ -1732,6 +1750,7 @@ static void *slab_alloc(struct kmem_cachreturn NULL ; redo: + node =find_numa_node (s, node);local_irq_save (flags); c =__this_cpu_ptr (s->cpu_slab);if (unlikely (!c->objects || !node_match (c, node))) {
@@ -1877,6 +1896,54 @@ void *kmem_cache_alloc_node_notrace(struEXPORT_SYMBOL (kmem_cache_alloc_node_notrace);#endif +int numa_off_node_free (struct kmem_cache *s,void *x) +{ +#ifdef CONFIG_NUMA +if (s->flags &SLAB_MEM_SPREAD ) { +int node =page_to_nid (virt_to_page (x)); +/* +* Slab requires object level control of locality. We can only +* keep objects from the local node in the per cpu queue other +* foreign object must not be freed to the queue. +* +* If we enconter a free of an off node object then we free +* it to the shared cache of that node. This places a cache +* cold object into that queue though. But using the queue +* is much more effective than going directly into the slab. +* +* Alternate approach: Call drain_objects directly for a single +* object. (Drain objects would have to be fixed to not save +* to the local shared mem cache by default). +*/ +if (node !=numa_node_id ()) { +struct kmem_cache_node *n =get_node (s, node); +redo: +if (n->objects >= s->shared) { +int t =min (s->batch, n->objects); + +drain_objects (s, n->object, t); + + n->objects -= t; +if (n->objects) +memcpy (n->object, n->object + t, + n->objects *sizeof (void *)); + } +spin_lock (&n->shared_lock); +if (n->objects < s->shared) { + n->object[n->objects++] = x; + x =NULL ; + } +spin_unlock (&n->shared_lock); +if (x) + goto redo; +return 1 ; + } + } +#endif +return 0 ; +} + +static void slab_free (struct kmem_cache *s,void *x,unsigned long addr) {
@@ -1895,6 +1962,9 @@ static void slab_free(struct kmem_cacheif (!(s->flags &SLAB_DEBUG_OBJECTS ))debug_check_no_obj_freed (object, s->objsize); +if (numa_off_node_free (s, x)) + goto out; +if (unlikely (c->objects >= s->queue)) {int t =min (s->batch, c->objects);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v2 14/14 SLEB: Allocate off node objects from remote shared caches
In reply to Christoph LameterThis is in a draft state.
Leave the cpu queue alone for off node accesses and go directly to the remote shared cache for alloations.
--- include/linux/slub_def.h | 1 mm/slub.c | 184 ++++++++++++++++++++++++++++++++++++----------- 2 files changed, 142 insertions(+), 43 deletions(-)
include/linux/slub_def.h +0 -1
@@ -42,7 +42,6 @@ struct kmem_cache_cpu {unsigned stat[NR_SLUB_STAT_ITEMS ];#endif int objects;/* Number of objects available */ -int node;/* The node of the page (or -1 for debug) */ void *object[BOOT_QUEUE_SIZE ];/* List of objects */ };
mm/slub.c +142 -42
247 lines not on this page · raw message
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Christoph Lameter
SLEB is a merging of SLUB with some queuing concepts from SLAB and a new way of managing objects in the slabs using bitmaps. It uses a percpu queue so that free operations can be properly buffered and a bitmap for managing the free/allocated state in the slabs. It is slightly more inefficient than SLUB (due to the need to place large bitmaps --sized a few words--in some slab pages if there are more than
BITS_PER_LONGobjects in a slab page) but in general does compete well with SLUB (and therefore also with SLOB) in terms of memory wastage.
I merged patches 1-7 to "sleb/core" branch of slab.git if people want to test them:
http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=shortlog;h=refs/heads/sleb/core
I didn't put them in linux-next for obvious reasons.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Well I'm glad you've conceded that queues are useful for high performance computing, and that higher order allocations are not a free and unlimited resource.
I hope we can move forward now with some objective, testable comparisons and criteria for selecting one main slab allocator.
Quoted 25 lines · Christoph Lameter
(V2 some more work as time permitted this week)
SLEB is a merging of SLUB with some queuing concepts from SLAB and a new way of managing objects in the slabs using bitmaps. It uses a percpu queue so that free operations can be properly buffered and a bitmap for managing the free/allocated state in the slabs. It is slightly more inefficient than SLUB (due to the need to place large bitmaps --sized a few words--in some slab pages if there are more than
BITS_PER_LONGobjects in a slab page) but in general does compete well with SLUB (and therefore also with SLOB) in terms of memory wastage.It does not have the excessive memory requirements of SLAB because there is no slab management structure nor alien caches. Under NUMA the remote shared caches are used instead (which may have its issues).
The SLAB scheme of not touching the object during management is adopted. SLEB can efficiently free and allocate cache cold objects without causing cache misses.
There are numerous SLAB schemes that are not supported. Those could be added if needed and if they really make a difference.
WARNING: This only ran successfully using hackbench in kvm instances so far. But works with NUMA, SMP and UP there.
Quoted 8 lines
V1->V2Add NUMA capabilities. Refine queue size configurations (not complete).
Test in UP, SMP, NUMA--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Nick Piggin
Well I'm glad you've conceded that queues are useful for high performance computing, and that higher order allocations are not a free and unlimited resource.
Ahem. I have never made any such claim and would never make them. And "conceding" something ???
The "unqueueing" was the result of excessive queue handling in SLAB due and the higher order allocations are a natural move in HPC to gain performance.
Quoted 2 lines
I hope we can move forward now with some objective, testable comparisons and criteria for selecting one main slab allocator.
If can find criteria that are universally agreed upon then yes but that is doubtful.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · Nick Piggin
Well I'm glad you've conceded that queues are useful for high performance computing, and that higher order allocations are not a free and unlimited resource.
Ahem. I have never made any such claim and would never make them. And "conceding" something ???
Well, you were quite vocal about the subject.
Quoted 2 lines
The "unqueueing" was the result of excessive queue handling in SLAB due and the higher order allocations are a natural move in HPC to gain performance.
This is the kind of handwavings that need to be put into a testable form. I repeatedly asked you for examples of where the jitter is excessive or where the TLB improvements help, but you never provided any testable case. I'm not saying they don't exist, but we have to be reational about this.
Quoted 5 lines
Quoted 2 lines
I hope we can move forward now with some objective, testable comparisons and criteria for selecting one main slab allocator.
If can find criteria that are universally agreed upon then yes but that is doubtful.
I think we can agree that perfect is the enemy of good, and that no allocator will do the perfect thing for everybody. I think we have to come up with a way to a single allocator.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Nick Piggin
Quoted 2 lines
If can find criteria that are universally agreed upon then yes but that is doubtful.
I think we can agree that perfect is the enemy of good, and that no allocator will do the perfect thing for everybody. I think we have to come up with a way to a single allocator.
Yes. The interesting most interesting bit about SLEB for me is the freelist handling as bitmaps, not necessarily the "queuing" part. If the latter also helps some workloads, it's a bonus for sure.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · Pekka Enberg
Quoted 6 lines · Nick Piggin
Quoted 2 lines
If can find criteria that are universally agreed upon then yes but that is doubtful.
I think we can agree that perfect is the enemy of good, and that no allocator will do the perfect thing for everybody. I think we have to come up with a way to a single allocator.
Yes. The interesting most interesting bit about SLEB for me is the freelist handling as bitmaps, not necessarily the "queuing" part. If the latter also helps some workloads, it's a bonus for sure.
Agreed it is all interesting, but I think we have to have a rational path toward having just one.
There is nothing to stop incremental changes or tweaks on top of that allocator, even to the point of completely changing the allocation scheme. It is inevitable that with changes in workloads, SMP/NUMA, and cache/memory costs and hierarchies, the best slab allocation schemes will change over time.
I think it is more important to have one allocator than trying to get the absolute most perfect one for everybody. That way changes are carefully and slowly reviewed and merged, with results to justify the change. This way everybody is testing the same thing, and bisection will work. The situation with SLUB is already a nightmare because now each allocator has half the testing and half the work put into it.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Nick,
Quoted 5 lines · Nick Piggin
There is nothing to stop incremental changes or tweaks on top of that allocator, even to the point of completely changing the allocation scheme. It is inevitable that with changes in workloads, SMP/NUMA, and cache/memory costs and hierarchies, the best slab allocation schemes will change over time.
Agreed.
Quoted 6 lines · Nick Piggin
I think it is more important to have one allocator than trying to get the absolute most perfect one for everybody. That way changes are carefully and slowly reviewed and merged, with results to justify the change. This way everybody is testing the same thing, and bisection will work. The situation with SLUB is already a nightmare because now each allocator has half the testing and half the work put into it.
I wouldn't say it's a nightmare, but yes, it could be better. From my point of view SLUB is the base of whatever the future will be because the code is much cleaner and simpler than SLAB. That's why I find Christoph's work on SLEB more interesting than SLQB, for example, because it's building on top of something that's mature and stable.
That said, are you proposing that even without further improvements to SLUB, we should go ahead and, for example, remove SLAB from Kconfig for v2.6.36 and see if we can just delete the whole thing from, say, v2.6.38?
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 19 lines · Pekka Enberg
Hi Nick,
Quoted 5 lines · Nick Piggin
There is nothing to stop incremental changes or tweaks on top of that allocator, even to the point of completely changing the allocation scheme. It is inevitable that with changes in workloads, SMP/NUMA, and cache/memory costs and hierarchies, the best slab allocation schemes will change over time.
Agreed.
Quoted 6 lines · Nick Piggin
I think it is more important to have one allocator than trying to get the absolute most perfect one for everybody. That way changes are carefully and slowly reviewed and merged, with results to justify the change. This way everybody is testing the same thing, and bisection will work. The situation with SLUB is already a nightmare because now each allocator has half the testing and half the work put into it.
Quoted 5 lines
I wouldn't say it's a nightmare, but yes, it could be better. From my point of view SLUB is the base of whatever the future will be because the code is much cleaner and simpler than SLAB. That's why I find Christoph's work on SLEB more interesting than SLQB, for example, because it's building on top of something that's mature and stable.
I don't think SLUB ever proved itself very well. The selling points were some untestable handwaving about how queueing is bad and jitter is bad, ignoring the fact that queues could be shortened and periodic reaping disabled at runtime with SLAB style of allocator. It also has relied heavily on higher order allocations which put great strain on hugepage allocations and page reclaim (witness the big slowdown in low memory conditions when tmpfs was using higher order allocations via SLUB).
Quoted 4 lines
That said, are you proposing that even without further improvements to SLUB, we should go ahead and, for example, remove SLAB from
Kconfigfor v2.6.36 and see if we can just delete the whole thing from, say, v2.6.38?
SLUB has not been able to displace SLAB for a long timedue to performance and higher order allocation problems.
I think "clean code" is very important, but by far the hardest thing to get right by far is the actual allocation and freeing strategies. So it's crazy to base such a choice on code cleanliness. If that's the deciding factor, then I can provide a patch to modernise SLAB and then we can remove SLUB and start incremental improvements from there.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Nick,
Quoted 8 lines · Nick Piggin
I don't think SLUB ever proved itself very well. The selling points were some untestable handwaving about how queueing is bad and jitter is bad, ignoring the fact that queues could be shortened and periodic reaping disabled at runtime with SLAB style of allocator. It also has relied heavily on higher order allocations which put great strain on hugepage allocations and page reclaim (witness the big slowdown in low memory conditions when tmpfs was using higher order allocations via SLUB).
The main selling point for SLUB was NUMA. Has the situation changed? Reliance on higher order allocations isn't that relevant if we're anyway discussing ways to change allocation strategy.
Quoted 8 lines · Nick Piggin
SLUB has not been able to displace SLAB for a long timedue to performance and higher order allocation problems.
I think "clean code" is very important, but by far the hardest thing to get right by far is the actual allocation and freeing strategies. So it's crazy to base such a choice on code cleanliness. If that's the deciding factor, then I can provide a patch to modernise SLAB and then we can remove SLUB and start incremental improvements from there.
I'm more than happy to take in patches to clean up SLAB but I think you're underestimating the required effort. What SLUB has going for it:
- No NUMA alien caches
- No special lockdep handling required
- Debugging support is better
- Cpuset interractions are simpler
- Memory hotplug is more mature
- Much more contributors to SLUB than to SLAB
I was one of the people cleaning up SLAB when SLUB was merged and based on that experience I'm strongly in favor of SLUB as a base.
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 12 lines · Pekka Enberg
Hi Nick,
Quoted 8 lines · Nick Piggin
I don't think SLUB ever proved itself very well. The selling points were some untestable handwaving about how queueing is bad and jitter is bad, ignoring the fact that queues could be shortened and periodic reaping disabled at runtime with SLAB style of allocator. It also has relied heavily on higher order allocations which put great strain on hugepage allocations and page reclaim (witness the big slowdown in low memory conditions when tmpfs was using higher order allocations via SLUB).
Quoted 1 line
The main selling point for SLUB was NUMA. Has the situation changed?
Well one problem with SLAB was really just those alien caches. AFAIK they were added by Christoph Lameter (maybe wrong), and I didn't ever actually see much justification for them in the changelog. noaliencache can be and is used on bigger machines, and SLES and RHEL kernels are using SLAB on production NUMA systems up to thousands of CPU Altixes, and have been looking at working on SGI's UV, and hundreds of cores POWER7 etc.
I have not seen NUMA benchmarks showing SLUB is significantly better. I haven't done much testing myself, mind you. But from indications, we could probably quite easily drop the alien caches setup and do like a simpler single remote freeing queue per CPU or something like that.
Quoted 2 lines
Reliance on higher order allocations isn't that relevant if we're anyway discussing ways to change allocation strategy.
Then it's just going through more churn and adding untested code to get where SLAB already is (top performance without higher order allocations). So it is very relevant if we're considering how to get to a single allocator.
Quoted 14 lines · Nick Piggin
Quoted 8 lines
SLUB has not been able to displace SLAB for a long timedue to performance and higher order allocation problems.
I think "clean code" is very important, but by far the hardest thing to get right by far is the actual allocation and freeing strategies. So it's crazy to base such a choice on code cleanliness. If that's the deciding factor, then I can provide a patch to modernise SLAB and then we can remove SLUB and start incremental improvements from there.
I'm more than happy to take in patches to clean up SLAB but I think you're underestimating the required effort. What SLUB has going for it:
Quoted 5 lines
- No NUMA alien caches
- No special lockdep handling required
- Debugging support is better
- Cpuset interractions are simpler
- Memory hotplug is more mature
All this I don't think is much problem. It was only a problem because we put in SLUB and so half these new features were added to it and people weren't adding them to SLAB.
Quoted 1 line
- Much more contributors to SLUB than to SLAB
In large part because it is less mature. But also because it seems to be seen as the allocator of the future.
Problem is that SLUB was never able to prove why it should be merged. The code cleanliness issue is really trivial in comparison to how much head scratching and work goes into analysing the performance.
It *really* is not required to completely replace a whole subsystem like this to make progress. Even if we make relatively large changes, everyone gets to use and test them, and it's so easy to bisect and work out how changes interact and change behaviour. Compare that with the problems we have when someone says that SLUB has a performance regression against SLAB.
Quoted 2 lines
I was one of the people cleaning up SLAB when SLUB was merged and based on that experience I'm strongly in favor of SLUB as a base.
I think we should: modernise SLAB code, add missing debug features, possibly turn off alien caches by default, chuck out SLUB, and then require that future changes have some reasonable bar set to justify them.
I would not be at all against adding changes that transform SLAB to SLUB or SLEB or SLQB. That's how it really should be done in the first place.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Nick,
Quoted 9 lines · Nick Piggin
Quoted 1 line
The main selling point for SLUB was NUMA. Has the situation changed?
Well one problem with SLAB was really just those alien caches. AFAIK they were added by Christoph Lameter (maybe wrong), and I didn't ever actually see much justification for them in the changelog. noaliencache can be and is used on bigger machines, and SLES and RHEL kernels are using SLAB on production NUMA systems up to thousands of CPU Altixes, and have been looking at working on SGI's UV, and hundreds of cores POWER7 etc.
Yes, Christoph and some other people introduced alien caches IIRC for big iron SGI boxes. As for benchmarks, commit e498be7dafd72fd68848c1eef1575aa7c5d658df ("Numa-aware slab allocator V5") mentions AIM.
Quoted 4 lines · Nick Piggin
I have not seen NUMA benchmarks showing SLUB is significantly better. I haven't done much testing myself, mind you. But from indications, we could probably quite easily drop the alien caches setup and do like a simpler single remote freeing queue per CPU or something like that.
Commit 81819f0fc8285a2a5a921c019e3e3d7b6169d225 ("SLUB core") mentions kernbench improvements.
Other than these two data points, I unfortunately don't have any as I wasn't involved with merging of either of the patches. If other NUMA people know better, please feel free to share the data.
Quoted 8 lines · Nick Piggin
I think we should: modernise SLAB code, add missing debug features, possibly turn off alien caches by default, chuck out SLUB, and then require that future changes have some reasonable bar set to justify them.
I would not be at all against adding changes that transform SLAB to SLUB or SLEB or SLQB. That's how it really should be done in the first place.
Like I said, as a maintainer I'm happy to merge patches to modernize SLAB but I still think you're underestimating the effort especially considering the fact that we can't afford many performance regressions there either. I guess trying to get rid of alien caches would be the first logical step there.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 13 lines · Pekka Enberg
Hi Nick,
Quoted 9 lines · Nick Piggin
Quoted 1 line
The main selling point for SLUB was NUMA. Has the situation changed?
Well one problem with SLAB was really just those alien caches. AFAIK they were added by Christoph Lameter (maybe wrong), and I didn't ever actually see much justification for them in the changelog. noaliencache can be and is used on bigger machines, and SLES and RHEL kernels are using SLAB on production NUMA systems up to thousands of CPU Altixes, and have been looking at working on SGI's UV, and hundreds of cores POWER7 etc.
Quoted 4 lines
Yes, Christoph and some other people introduced alien caches IIRC for big iron SGI boxes. As for benchmarks, commit
e498be7dafd72fd68848c1eef1575aa7c5d658df("Numa-aware slab allocator V5") mentions AIM.
It's quite a change with a lot of things. But there are definitely other ways we can improve this without having a huge dumb crossbar for remote frees.
Quoted 8 lines · Nick Piggin
Quoted 4 lines
I have not seen NUMA benchmarks showing SLUB is significantly better. I haven't done much testing myself, mind you. But from indications, we could probably quite easily drop the alien caches setup and do like a simpler single remote freeing queue per CPU or something like that.
Commit
81819f0fc8285a2a5a921c019e3e3d7b6169d225("SLUB core") mentions kernbench improvements.
I haven't measured anything like that. Kernbench for me has never had slab show anywhere near the profiles (it's always page fault, teardown, page allocator paths).
Must have been a pretty specific configuration, but anyway I don't know that it is realistic.
Quoted 3 lines
Other than these two data points, I unfortunately don't have any as I wasn't involved with merging of either of the patches. If other NUMA people know better, please feel free to share the data.
A lot of people are finding SLAB is still required for performance reasons. We did not want to change in SLES11 for example because of performance concerns. Not sure about RHEL6?
Quoted 6 lines · Nick Piggin
Quoted 5 lines
I think we should: modernise SLAB code, add missing debug features, possibly turn off alien caches by default, chuck out SLUB, and then require that future changes have some reasonable bar set to justify them.
Quoted 6 lines
Quoted 3 lines
I would not be at all against adding changes that transform SLAB to SLUB or SLEB or SLQB. That's how it really should be done in the first place.
Like I said, as a maintainer I'm happy to merge patches to modernize SLAB
I think that would be most productive at this point. I will volunteer to do it.
As much as I would like to see SLQB be merged :) I think the best option is to go with SLAB because it is very well tested and very very well performing.
If Christoph or you or I or anyone have genuine improvements to make to the core algorithms, then the best thing to do will just be do make incremental changes to SLAB.
Quoted 4 lines
but I still think you're underestimating the effort especially considering the fact that we can't afford many performance regressions there either. I guess trying to get rid of alien caches would be the first logical step there.
There are several aspects to this. I think the first one will be to actually modernize the code style, simplify the bootstrap process and static memory allocations (SLQB goes even further than SLUB in this regard), and to pull in debug features from SLUB.
These steps should be made without any changes to core algorithms. Alien caches can easily be disabled and at present they are really only a problem for big Altixes where it is a known parameter to tune.
Quoted 1 line
From that point, I think we should concede that SLUB has not fulfilled
performance promises, and make SLAB the default.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Nick,
Quoted 5 lines · Nick Piggin
Quoted 2 lines
Like I said, as a maintainer I'm happy to merge patches to modernize SLAB
I think that would be most productive at this point. I will volunteer to do it.
OK, great!
Quoted 3 lines
As much as I would like to see SLQB be merged :) I think the best option is to go with SLAB because it is very well tested and very very well performing.
I would have liked to see SLQB merged as well but it just didn't happen.
Quoted 3 lines
If Christoph or you or I or anyone have genuine improvements to make to the core algorithms, then the best thing to do will just be do make incremental changes to SLAB.
I don't see the problem in improving SLUB even if we start modernizing SLAB. Do you? I'm obviously biased towards SLUB still for the reasons I already mentioned. I don't want to be a blocker for progress so if I turn out to be a problem, we should consider changing the maintainer(s). ;-)
Quoted 11 lines
There are several aspects to this. I think the first one will be to actually modernize the code style, simplify the bootstrap process and static memory allocations (SLQB goes even further than SLUB in this regard), and to pull in debug features from SLUB.
These steps should be made without any changes to core algorithms. Alien caches can easily be disabled and at present they are really only a problem for big Altixes where it is a known parameter to tune.
From that point, I think we should concede that SLUB has not fulfilled performance promises, and make SLAB the default.
Sure. I don't care which allocator "wins" if we actually are able to get there.
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Pekka Enberg
Hi Nick,
Quoted 5 lines · Nick Piggin
Quoted 2 lines
Like I said, as a maintainer I'm happy to merge patches to modernize SLAB
I think that would be most productive at this point. I will volunteer to do it.
Quoted 7 lines
OK, great!
Quoted 3 lines
As much as I would like to see SLQB be merged :) I think the best option is to go with SLAB because it is very well tested and very very well performing.
I would have liked to see SLQB merged as well but it just didn't happen.
It seemed a bit counter productive if the goal is to have one allocator. I think it still has merit but I should really practice what I preach and propose incremental improvements to SLAB.
Quoted 9 lines
Quoted 3 lines
If Christoph or you or I or anyone have genuine improvements to make to the core algorithms, then the best thing to do will just be do make incremental changes to SLAB.
I don't see the problem in improving SLUB even if we start modernizing SLAB. Do you? I'm obviously biased towards SLUB still for the reasons I already mentioned. I don't want to be a blocker for progress so if I turn out to be a problem, we should consider changing the maintainer(s). ;-)
I think it just has not proven itself at this point, we have most production kernels (at least, the performance sensitive ones that I'm aware of) running on SLAB, and if it is conceded that lack of queueing and reliance on higher order allocations is a problem then I think it is far better just to bite the bullet now, drop it so we can have a single allocator. Rather than adding SLAB-like queueing to it and other big changes. Then make incremental improvements to SLAB.
I have no problems at all with trying new ideas, but really, they should be done in SLAB as incremental improvements. Everywhere we take that approach, things seem to work better than when we do wholesale rip and replacements.
I don't want Christoph (or myself, or you) to stop testing new ideas, but really there are almost no good reasons as to why they can be done as incremental patches.
With SLAB code cleaned up, there will be even fewer reasons.
Quoted 5 lines
Quoted 5 lines
There are several aspects to this. I think the first one will be to actually modernize the code style, simplify the bootstrap process and static memory allocations (SLQB goes even further than SLUB in this regard), and to pull in debug features from SLUB.
Quoted 8 lines
Quoted 6 lines
These steps should be made without any changes to core algorithms. Alien caches can easily be disabled and at present they are really only a problem for big Altixes where it is a known parameter to tune.
From that point, I think we should concede that SLUB has not fulfilled performance promises, and make SLAB the default.
Sure. I don't care which allocator "wins" if we actually are able to get there.
SLUB is already behind the 8 ball here. So is SLQB I don't mind saying because it has had much much less testing.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 2 lines · Pekka Enberg
I would have liked to see SLQB merged as well but it just didn't happen.
And it's not going to. I'm not going to merge YASA that will stay around for years, not improve on anything, and will just mean that there are some bugs that developers don't see because they depend on some subtle interaction with the sl*b allocator.
We've got three. That's at least one too many. We're not adding any new ones until we've gotten rid of at least one old one.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Linus Torvalds
Quoted 2 lines · Pekka Enberg
I would have liked to see SLQB merged as well but it just didn't happen.
Quoted 7 lines
And it's not going to. I'm not going to merge YASA that will stay around for years, not improve on anything, and will just mean that there are some bugs that developers don't see because they depend on some subtle interaction with the sl*b allocator.
We've got three. That's at least one too many. We're not adding any new ones until we've gotten rid of at least one old one.
No agree and realized that a while back (hence stop pushing SLQB). SLAB is simply a good allocator that is very very hard to beat. The fact that a lot of places are still using SLAB despite the real secondary advantages of SLUB (cleaner code, better debugging support) indicate to me that we should go back and start from there.
What is sad is all this duplicate (and unsynchronized and not always complete) work implementing things in both the allocators[*] and split testing base.
As far as I can see, there was never a good reason to replace SLAB rather than clean up its code and make incremental improvements.
Hi Nick,
Quoted 2 lines · Nick Piggin
As far as I can see, there was never a good reason to replace SLAB rather than clean up its code and make incremental improvements.
I'm not totally convinced but I guess we're about to find that out. How do you propose we benchmark SLAB while we clean it up and change things to make sure we don't make the same mistakes as we did with SLUB (i.e. miss an important workload like TPC-C)?
Pekka
Quoted 8 lines · Pekka Enberg
Hi Nick,
Quoted 2 lines · Nick Piggin
As far as I can see, there was never a good reason to replace SLAB rather than clean up its code and make incremental improvements.
I'm not totally convinced but I guess we're about to find that out. How do you propose we benchmark SLAB while we clean it up
Well the first pass will be code cleanups, bootstrap simplifications. Then looking at what debugging features were implemented in SLUB but not SLAB and what will be useful to bring over from there.
At this point the aim would be for actual allocation behaviour with non-debug settings to be unchanged. Hopefully this removes everyone's (apparently) largest gripe that code is crufty.
Next would be to add some options to tweak queue sizes and disable cache reaping at runtime, for the benfit of the low jitter crowd, see if any further hotplug fixes are required.
Then would be to propose incremental improvements to actual algorithm. For example, replacing the alien cache crossbar with a lighter weight or more scalable structure.
Quoted 3 lines
and change
things to make sure we don't make the same mistakes as we did with SLUB (i.e. miss an important workload like TPC-C)?
Obviously it is impossible to make forward progress and also catch all regressions before release. This fact means that we have to be able to cope with them as well as possible.
We get two benefits from starting with SLAB. Firstly, we get a larger testing base. Secondly, we get a simple (ie. git revert) formula of how to get from good behaviour to bad behaviour.
I don't anticipate a huge number of functional changes to SLAB here though. It's surprisingly hard to do better than it. alien caches are one area, maybe configurable higher order allocation support, jitter reduction.
If we do get a big proposed change in the pipeline, then we have to eat it somehow, but AFAIKS we've still got a better foundation than starting with a completely new allocator and feeling around in the dark trying to move it past SLAB in terms of performance.
Quoted 6 lines · Nick Piggin
Quoted 2 lines
I'm not totally convinced but I guess we're about to find that out. How do you propose we benchmark SLAB while we clean it up
Well the first pass will be code cleanups, bootstrap simplifications. Then looking at what debugging features were implemented in SLUB but not SLAB and what will be useful to bring over from there.
Bootstrap might be easy to clean up but the biggest source of cruft comes from the deeply inlined, complex allocation paths. Cleaning those up is bound to cause performance regressions if you're not careful.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Pekka Enberg
Quoted 3 lines · Nick Piggin
Quoted 2 lines
I'm not totally convinced but I guess we're about to find that out. How do you propose we benchmark SLAB while we clean it up
Quoted 8 lines
Quoted 3 lines
Well the first pass will be code cleanups, bootstrap simplifications. Then looking at what debugging features were implemented in SLUB but not SLAB and what will be useful to bring over from there.
Bootstrap might be easy to clean up but the biggest source of cruft comes from the deeply inlined, complex allocation paths. Cleaning those up is bound to cause performance regressions if you're not careful.
Oh I see what you mean, just straight-line code speed regressions could bite us when doing cleanups.
That's possible. I'll keep a close eye on generated asm.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerQuoted 9 lines · Nick Piggin
I don't think SLUB ever proved itself very well. The selling points were some untestable handwaving about how queueing is bad and jitter is bad, ignoring the fact that queues could be shortened and periodic reaping disabled at runtime with SLAB style of allocator. It also has relied heavily on higher order allocations which put great strain on hugepage allocations and page reclaim (witness the big slowdown in low memory conditions when tmpfs was using higher order allocations via SLUB).
I agree that the higher order allocations is a major problem and slub relies heavily on them for being able to utilize both the allocation and freeing fastpaths for a number of caches. For systems with a very large amount of memory that isn't fully utilized and fragmentation isn't an issue, this works fine, but for users who use all their memory and do some amount of reclaim it comes at a significant cost. The cpu slab thrashing problem that I identified with the netperf TCP_RR benchmark can be heavily reduced by tuning certain kmalloc caches to allocate higher order slabs, but that makes it very difficult to run with hugepages and the allocation slowpath even slower. There are commandline workarounds to prevent slub from using these higher order allocations, but the performance of the allocator then suffers as a result.
Quoted 3 lines
SLUB has not been able to displace SLAB for a long timedue to performance and higher order allocation problems.
Completely agreed.
David Rientjes
reviewerQuoted 3 lines · Pekka Enberg
I wouldn't say it's a nightmare, but yes, it could be better. From my point of view SLUB is the base of whatever the future will be because the code is much cleaner and simpler than SLAB.
The code may be much cleaner and simpler than slab, but nobody (to date) has addressed the significant netperf TCP_RR regression that slub has, for example. I worked on a patchset to do that for a while but it wasn't popular because it added some increments to the fastpath for tracking data.
I think it's great to have clean and simple code, but even considering its use is a non-starter when the entire kernel is significantly slower for certain networking loads.
Quoted 9 lines
That's why I find
Christoph's work on SLEB more interesting than SLQB, for example, because it's building on top of something that's mature and stable.That said, are you proposing that even without further improvements to SLUB, we should go ahead and, for example, remove SLAB from
Kconfigfor v2.6.36 and see if we can just delete the whole thing from, say, v2.6.38?
We use slab internally specifically because of the slub regressions. Removing it from the kernel at this point would be the equivalent of saying that Linux cares about certain workloads more than others since there are clearly benchmarks that show slub to be inferior in pure performance numbers. I'd love for us to switch to slub but we can't take the performance hit.
Hi David,
Quoted 9 lines · David Rientjes
Quoted 3 lines
I wouldn't say it's a nightmare, but yes, it could be better. From my point of view SLUB is the base of whatever the future will be because the code is much cleaner and simpler than SLAB.
The code may be much cleaner and simpler than slab, but nobody (to date) has addressed the significant netperf
TCP_RRregression that slub has, for example. I worked on a patchset to do that for a while but it wasn't popular because it added some increments to the fastpath for tracking data.
Yes and IIRC I asked you to resend the series because while I care a lot about performance regressions, I simply don't have the time or the hardware to reproduce and fix the weird cases you're seeing.
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
David Rientjes
reviewerQuoted 10 lines · Pekka Enberg
Quoted 5 lines
The code may be much cleaner and simpler than slab, but nobody (to date) has addressed the significant netperf
TCP_RRregression that slub has, for example. I worked on a patchset to do that for a while but it wasn't popular because it added some increments to the fastpath for tracking data.Yes and IIRC I asked you to resend the series because while I care a lot about performance regressions, I simply don't have the time or the hardware to reproduce and fix the weird cases you're seeing.
My patchset still never attained parity with slab even though it improved slub's performance for that specific benchmark on my 16-core machine with 64G of memory:
# threads SLAB SLUB SLUB+patchset 16 69892 71592 69505 32 126490 95373 119731 48 138050 113072 125014 64 169240 149043 158919 80 192294 172035 179679 96 197779 187849 192154 112 217283 204962 209988 128 229848 217547 223507 144 238550 232369 234565 160 250333 239871 244789 176 256878 242712 248971 192 261611 243182 255596
CONFIG_SLUB_STATS demonstrates that the kmalloc-256 and kmalloc-2048 are performing quite poorly without the changes:
cache ALLOC_FASTPATH ALLOC_SLOWPATH kmalloc-256 98125871 31585955 kmalloc-2048 77243698 52347453 cache FREE_FASTPATH FREE_SLOWPATH kmalloc-256 173624 129538000 kmalloc-2048 90520 129500630
When you have these type of results, it's obvious why slub is failing to achieve the same performance as slab. With the slub fastpath percpu work that has been done recently, it might be possible to resurrect my patchset and get more positive feedback because the penalty won't be as a significant, but the point is that slub still fails to achieve the same results that slab can with heavy networking loads. Thus, I think any discussion about removing slab is premature until it's no longer shown to be a clear winner in comparison to its replacement, whether that is slub, slqb, sleb, or another allocator. I agree that slub is clearly better in terms of maintainability, but we simply can't use it because of its performance for networking loads.
If you want to duplicate these results on machines with a larger number of cores, just download netperf, run with CONFIG_SLUB on both netserver and netperf machines, and use this script:
#!/bin/bash
TIME=60 # seconds
HOSTNAME=<hostname> # netserver
NR_CPUS=$(grep ^processor /proc/cpuinfo | wc -l)
echo NR_CPUS=$NR_CPUS
run_netperf() {
for i in $(seq 1 $1); do
netperf -H $HOSTNAME -t TCP_RR -l $TIME &
done
}ITERATIONS=0
while [ $ITERATIONS -lt 12 ]; do
RATE=0
ITERATIONS=$[$ITERATIONS + 1]
THREADS=$[$NR_CPUS * $ITERATIONS]
RESULTS=$(run_netperf $THREADS | grep -v '[a-zA-Z]' | awk '{ print $6 }')
for j in $RESULTS; do
RATE=$[$RATE + ${j/.*}]
done
echo threads=$THREADS rate=$RATE
done
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · Nick Piggin
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · Nick Piggin
Well I'm glad you've conceded that queues are useful for high performance computing, and that higher order allocations are not a free and unlimited resource.
Ahem. I have never made any such claim and would never make them. And "conceding" something ???
Well, you were quite vocal about the subject.
I was always vocal about the huge amounts of queues and the complexity coming with alien caches etc. The alien caches were introduced against my objections on the development team that did the NUMA slab. But even SLUB has "queues" as many have repeatedly pointed out. The queuing is different though in order to minimize excessive NUMA queueing. IMHO the NUMA design of SLAB has fundamental problems because it implements its own "NUMAness" aside from the page allocator. I had to put lots of band aid on the NUMA functionality in SLAB to make it correct.
One of the key things in SLEB is the question how to deal with the alien issue. So far I think the best compromise would be to use the shared caches of the remote node as a stand in for the alien cache. Problem is that we will then free cache cold objects to the remote shared cache. Maybe that can be addressed by freeing to the end of the queue instead of freeing to the top.
Quoted 8 lines
Quoted 2 lines
The "unqueueing" was the result of excessive queue handling in SLAB due and the higher order allocations are a natural move in HPC to gain performance.
This is the kind of handwavings that need to be put into a testable form. I repeatedly asked you for examples of where the jitter is excessive or where the TLB improvements help, but you never provided any testable case. I'm not saying they don't exist, but we have to be reational about this.
The initial test that showed the improvements was on IA64 (16K page size) and that was the measurement that was accepted for the initial merge. Mel was able to verify those numbers.
While it will be easily possible to have less higher order allocations with SLEB I still think that higher order allocations are desirable to increase data locality and TLB pressure. Its easy though to set the defaults to order 1 (like SLAB) though and then allow manual override if desired.
Fundamentally it is still the case that memory sizes are increasing and that management overhead of 4K pages will therefore increasingly become an issue. Support for larger page sizes and huge pages is critical for all kernel components to compete in the future.
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
I hope we can move forward now with some objective, testable comparisons and criteria for selecting one main slab allocator.
If can find criteria that are universally agreed upon then yes but that is doubtful.
I think we can agree that perfect is the enemy of good, and that no allocator will do the perfect thing for everybody. I think we have to come up with a way to a single allocator.
Yes but SLAB is not really the way to go. The code is too messy. Thats why I think the best way to go at this point is to merge the clean SLUB design and add the SLAB features needed and try to keep the NUMA stuff cleaner.
I am not entirely sure that I want to get rid of SLUB. Certainly if you want minimal latency (like in my field) and more determinism then you would want a SLUB design instead of periodic queue handling. Also SLUB has a minimal memory footprint due to the linked list architecture.
The queues sacrifice a lot there. The linked list does not allow managing cache cold objects like SLAB does because you always need to touch the object and this will cause regressions against SLAB. I think this is also one of the weaknesses of SLQB.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Christoph Lameter
Quoted 6 lines · Nick Piggin
On Mon, May 24, 2010 at 10:06:08AM -0500, Christoph Lameter wrote:
This is the kind of handwavings that need to be put into a testable form. I repeatedly asked you for examples of where the jitter is excessive or where the TLB improvements help, but you never provided any testable case. I'm not saying they don't exist, but we have to be reational about this.
Quoted 3 lines
The initial test that showed the improvements was on IA64 (16K page size) and that was the measurement that was accepted for the initial merge. Mel was able to verify those numbers.
And there is nothing to prevent a SLAB type allocator from using higher order allocations, except for the fact that it usually wouldn't because far more often than not it is a bad idea.
Also, people actually want to use hugepages in userspace. The more that other allocations use them, the worse problems with fragmentation and reclaim become.
Quoted 10 lines
While it will be easily possible to have less higher order allocations with SLEB I still think that higher order allocations are desirable to increase data locality and TLB pressure. Its easy though to set the defaults to order 1 (like SLAB) though and then allow manual override if desired.
Fundamentally it is still the case that memory sizes are increasing and that management overhead of 4K pages will therefore increasingly become an issue. Support for larger page sizes and huge pages is critical for all kernel components to compete in the future.
Numbers haven't really shown that SLUB is better because of higher order allocations. Besides, as I said, higher order allocations can be used by others.
Quoted 11 lines
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
I hope we can move forward now with some objective, testable comparisons and criteria for selecting one main slab allocator.
If can find criteria that are universally agreed upon then yes but that is doubtful.
I think we can agree that perfect is the enemy of good, and that no allocator will do the perfect thing for everybody. I think we have to come up with a way to a single allocator.
Yes but SLAB is not really the way to go. The code is too messy. Thats why
That's a weak reason. SLUB has taken years to prove that it's not a suitable replacement, so more big changes to it is not make it more suitable now. We should just admit the rip and replace idea has failed, and go with more reasonable incremental improvements rather than subject everyone to another round of testing.
This is why I stopped pushing SLQB TBH, even though it showed some promise.
The hard part is clearly NOT the code cleanup. It is the design and all the testing and tuning.
Quoted 2 lines
I think the best way to go at this point is to merge the clean SLUB design and add the SLAB features needed and try to keep the NUMA stuff cleaner.
I think it is to get rid of SLUB and add SLUB features gradually to SLAB if/when they prove themselves.
Quoted 4 lines
I am not entirely sure that I want to get rid of SLUB. Certainly if you want minimal latency (like in my field) and more determinism then you would want a SLUB design instead of periodic queue handling. Also SLUB has a minimal memory footprint due to the linked list architecture.
I disagree completely. The queues can be shrunk to a similar size as the SLUB queues (which are just implicit by design), and periodic shrinking can be disabled like SLUB. It's not a fundamental design property.
Also, there were no numbers or test cases, simply handwaving. I don't disagree it might be a problem, but the way to solve problems is to provide a test case or numbers.
Quoted 4 lines
The queues sacrifice a lot there. The linked list does not allow managing cache cold objects like SLAB does because you always need to touch the object and this will cause regressions against SLAB. I think this is also one of the weaknesses of SLQB.
But this is just more handwaving. That's what got us into this situation we are in now.
What we know is that SLAB is still used by all high performance enterprise distros (and google). And it is used by Altixes in production as well as all other large NUMA machines that Linux runs on.
Given that information, how can you still say that SLUB+more big changes is the right way to proceed?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Nick Piggin
Quoted 4 lines · Christoph Lameter
The queues sacrifice a lot there. The linked list does not allow managing cache cold objects like SLAB does because you always need to touch the object and this will cause regressions against SLAB. I think this is also one of the weaknesses of SLQB.
But this is just more handwaving. That's what got us into this situation we are in now.
Quoted 6 lines
What we know is that SLAB is still used by all high performance enterprise distros (and google). And it is used by Altixes in production as well as all other large NUMA machines that Linux runs on.
Given that information, how can you still say that SLUB+more big changes is the right way to proceed?
Might I add that once SLAB code is cleaned up, you can always propose improvements from SLUB or any other ideas for it which we can carefully test and merge in slowly as bisectable changes to our benchmark performance slab allocator.
In fact, if you have better ideas in SLEB, I would encourage it.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Nick Piggin
Quoted 3 lines
The initial test that showed the improvements was on IA64 (16K page size) and that was the measurement that was accepted for the initial merge. Mel was able to verify those numbers.
And there is nothing to prevent a SLAB type allocator from using higher order allocations, except for the fact that it usually wouldn't because far more often than not it is a bad idea.
16K is the base page size on IA64. Higher order allocations are a pressing issue for the kernel given growing memory sizes and we are slowly but surely making progress with defrag etc.
Quoted 8 lines
Quoted 4 lines
Fundamentally it is still the case that memory sizes are increasing and that management overhead of 4K pages will therefore increasingly become an issue. Support for larger page sizes and huge pages is critical for all kernel components to compete in the future.
Numbers haven't really shown that SLUB is better because of higher order allocations. Besides, as I said, higher order allocations can be used by others.
Boot with huge page support (slub_min_order=9) and you will see a performance increase on many loads.
Quoted 3 lines
Also, there were no numbers or test cases, simply handwaving. I don't disagree it might be a problem, but the way to solve problems is to provide a test case or numbers.
The reason that the alien caches made it into SLAB were performance numbers that showed that the design "must" be this way. I prefer a clear maintainable design over some numbers (that invariably show the bias of the tester for certain loads).
Quoted 2 lines
Given that information, how can you still say that SLUB+more big changes is the right way to proceed?
Have you looked at the SLAB code?
Also please stop exaggerating. There are no immediate plans to replace SLAB. We are exploring a possible solution.
If the SLEB idea pans out and we can replicate SLAB (and SLUB) performance then we will have to think about replacing SLAB / SLUB at some point. So far this is just a riggedy thing that barely works where there is some hope that the SLAB - SLUB conumdrum may be solved by the approach.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Christoph Lameter
Quoted 4 lines · Nick Piggin
Quoted 3 lines
The initial test that showed the improvements was on IA64 (16K page size) and that was the measurement that was accepted for the initial merge. Mel was able to verify those numbers.
Quoted 7 lines
Quoted 3 lines
And there is nothing to prevent a SLAB type allocator from using higher order allocations, except for the fact that it usually wouldn't because far more often than not it is a bad idea.
16K is the base page size on IA64. Higher order allocations are a pressing issue for the kernel given growing memory sizes and we are slowly but surely making progress with defrag etc.
You do not understand. There is nothing *preventing* other designs of allocators from using higher order allocations. The problem is that SLUB is *forced* to use them due to it's limited queueing capabilities.
You keep spinning this as a good thing for SLUB design when it is not.
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
Fundamentally it is still the case that memory sizes are increasing and that management overhead of 4K pages will therefore increasingly become an issue. Support for larger page sizes and huge pages is critical for all kernel components to compete in the future.
Numbers haven't really shown that SLUB is better because of higher order allocations. Besides, as I said, higher order allocations can be used by others.
Boot with huge page support (slub_min_order=9) and you will see a performance increase on many loads.
Pretty ridiculous.
Quoted 8 lines
Quoted 3 lines
Also, there were no numbers or test cases, simply handwaving. I don't disagree it might be a problem, but the way to solve problems is to provide a test case or numbers.
The reason that the alien caches made it into SLAB were performance numbers that showed that the design "must" be this way. I prefer a clear maintainable design over some numbers (that invariably show the bias of the tester for certain loads).
I don't really agree. There are a number of other possible ways to improve it, including fewer remote freeing queues.
For the slab allocator, if anything, I'm pretty sure that numbers actually is the most important criteria. A few thousand lines of self contained code that services almost all the rest of the kernel we are talking about.
Quoted 4 lines
Quoted 2 lines
Given that information, how can you still say that SLUB+more big changes is the right way to proceed?
Have you looked at the SLAB code?
Of course. Have you had a look at the SLUB numbers and reports of failures?
Quoted 2 lines
Also please stop exaggerating. There are no immediate plans to replace SLAB. We are exploring a possible solution.
Good, because it cannot be replaced, I am proposing to replace SLUB in fact. I have heard no good reasons why not.
Quoted 4 lines
If the SLEB idea pans out and we can replicate SLAB (and SLUB) performance then we will have to think about replacing SLAB / SLUB at some point. So far this is just a riggedy thing that barely works where there is some hope that the SLAB - SLUB conumdrum may be solved by the approach.
SLUB has gone back to the drawing board because its original design cannot support high enough performance to replace SLAB. This gives us the opportunity to do what we should have done from the start, and incrementally improve SLAB code.
I repeat for the Nth time that there is nothing stopping you from adding SLUB ideas into SLAB. This is how it should have been done from the start.
How is it possibly better to instead start from the known suboptimal code and make changes to it? What exactly is your concern with making incremental changes to SLAB?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Nick Piggin
You do not understand. There is nothing *preventing* other designs of allocators from using higher order allocations. The problem is that SLUB is *forced* to use them due to it's limited queueing capabilities.
SLUBs use of higher order allocation is *optional*. The limited queuing is advantageous within the framework of SLUB because NUMA locality checks are simplified and locking is localized to a single page increasing concurrency.
Quoted 1 line
You keep spinning this as a good thing for SLUB design when it is not.
It is a good design decision. You have an irrational fear of higher order allocations.
Quoted 7 lines
Quoted 4 lines
The reason that the alien caches made it into SLAB were performance numbers that showed that the design "must" be this way. I prefer a clear maintainable design over some numbers (that invariably show the bias of the tester for certain loads).
I don't really agree. There are a number of other possible ways to improve it, including fewer remote freeing queues.
You disagree with the history of the allocator?
Quoted 3 lines
How is it possibly better to instead start from the known suboptimal code and make changes to it? What exactly is your concern with making incremental changes to SLAB?
I am not sure why you want me to repeat what I already said. Guess we should stop this conversation since it is deteriorating.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Christoph Lameter
Quoted 3 lines · Nick Piggin
You do not understand. There is nothing *preventing* other designs of allocators from using higher order allocations. The problem is that SLUB is *forced* to use them due to it's limited queueing capabilities.
SLUBs use of higher order allocation is *optional*. The limited queuing is advantageous within the framework of SLUB because NUMA locality checks are simplified and locking is localized to a single page increasing concurrency.
It's not optional if performance sucks without it. People want to have a well performing slab allocator and also not have the downsides of it using higher order allocations.
Look at what David said about Google's kernel for a concrete example.
Quoted 4 lines
Quoted 1 line
You keep spinning this as a good thing for SLUB design when it is not.
It is a good design decision. You have an irrational fear of higher order allocations.
No.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
The reason that the alien caches made it into SLAB were performance numbers that showed that the design "must" be this way. I prefer a clear maintainable design over some numbers (that invariably show the bias of the tester for certain loads).
I don't really agree. There are a number of other possible ways to improve it, including fewer remote freeing queues.
You disagree with the history of the allocator?
I don't agree with you saying that it "must" be that way. There are other ways to improve things there.
Quoted 6 lines
Quoted 3 lines
How is it possibly better to instead start from the known suboptimal code and make changes to it? What exactly is your concern with making incremental changes to SLAB?
I am not sure why you want me to repeat what I already said. Guess we should stop this conversation since it is deteriorating.
You never answered these questions adequately. These are the 2 most important things because if I can address your concerns with them, then we can go ahead and throw out SLUB and make incremental improvements from there instead.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 5 lines · Nick Piggin
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
The reason that the alien caches made it into SLAB were performance numbers that showed that the design "must" be this way. I prefer a clear maintainable design over some numbers (that invariably show the bias of the tester for certain loads).
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
I don't really agree. There are a number of other possible ways to improve it, including fewer remote freeing queues.
You disagree with the history of the allocator?
I don't agree with you saying that it "must" be that way. There are other ways to improve things there.
People told me that it "must" be this way. Could not convince them otherwise at the time. I never wanted it to be that way and have been looking for other ways ever since. SLUB is a result of trying something different.
Quoted 2 lines
then we can go ahead and throw out SLUB and make incremental improvements from there instead.
I am just amazed at the tosses and turns by you. Didnt you write SLQB on the basis of SLUB? And then it was abandoned? If you really believe ths and want to get this done then please invest some time in SLAB to get it cleaned up. I have some doubt that you are aware of the difficulties that you will encounter.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Christoph Lameter
Quoted 8 lines · Nick Piggin
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
The reason that the alien caches made it into SLAB were performance numbers that showed that the design "must" be this way. I prefer a clear maintainable design over some numbers (that invariably show the bias of the tester for certain loads).
I don't really agree. There are a number of other possible ways to improve it, including fewer remote freeing queues.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
You disagree with the history of the allocator?
I don't agree with you saying that it "must" be that way. There are other ways to improve things there.
People told me that it "must" be this way. Could not convince them otherwise at the time.
So again there was no numbers just handwaving?
Quoted 9 lines
I never wanted it to be that way and have been
looking for other ways ever since. SLUB is a result of trying something different.Quoted 2 lines
then we can go ahead and throw out SLUB and make incremental improvements from there instead.
I am just amazed at the tosses and turns by you. Didnt you write SLQB on the basis of SLUB? And then it was abandoned? If you really believe ths
Sure I hoped it would be able to conclusively beat SLAB, and I'd thought it might be a good idea. I stopped pushing it because I realized that incremental improvements to SLAB would likely be a far better idea.
Quoted 3 lines
and want to get this done then please invest some time in SLAB to get it cleaned up. I have some doubt that you are aware of the difficulties that you will encounter.
I am working on it. We'll see.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Nick Piggin
Quoted 2 lines
I am just amazed at the tosses and turns by you. Didnt you write SLQB on the basis of SLUB? And then it was abandoned? If you really believe ths
Sure I hoped it would be able to conclusively beat SLAB, and I'd thought it might be a good idea. I stopped pushing it because I realized that incremental improvements to SLAB would likely be a far better idea.
It looked to me as if there was a major conceptual issue with the linked lists used for objects that impacted performance plus unresolved issues with crashes on boot. I did not see you work on SLAB improvements. Seemed that other things had higher priority. The work on slab allocators in general is not well funded, not high priority and is a side issue. The time that I can spend on this is also limited.
Quoted 5 lines
Quoted 3 lines
and want to get this done then please invest some time in SLAB to get it cleaned up. I have some doubt that you are aware of the difficulties that you will encounter.
I am working on it. We'll see.
I think we agreee on one thing regardless of SLAB or SLUB as a base: It would be good to put the best approaches together to form a superior slab allocator. I just think its much easier to do give a mature and clean code base in SLUB. If we both work on this then this may coalesce at some point.
The main gripes with SLAB
- Code base difficult to maintain. Has grown over almost 2 decades.
- Alien caches need to be kept under control. Various hacky ways are implemented to bypass that problem.
- Locking issues because of long hold times of per node lock. SLUB has locking on per page level. This is important for high number of threads per node. Westmere has already 12. EX 24 and it way grow from there.
- Debugging features and recovery mechanisms.
- Off or on page slab metadata causes space wastage, complex allocation and locking and alignment issues. SLED replaces that metadata structure with a bitfield in the page struct. This may also save access to additional cacheline and maybe allow freeing of objects to a slab page without taking locks.
- Variable and function naming is confusing.
- OS noise caused by periodic cache cleaning (which requires scans over all caches of all slabs on every processor).
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 5 lines · Christoph Lameter
Quoted 3 lines · Nick Piggin
Quoted 2 lines
I am just amazed at the tosses and turns by you. Didnt you write SLQB on the basis of SLUB? And then it was abandoned? If you really believe ths
Quoted 7 lines
Quoted 4 lines
Sure I hoped it would be able to conclusively beat SLAB, and I'd thought it might be a good idea. I stopped pushing it because I realized that incremental improvements to SLAB would likely be a far better idea.
It looked to me as if there was a major conceptual issue with the linked lists used for objects that impacted performance
With SLQB's linked list? No. Single threaded cache hot performance was the same (+/- a couple of cycles IIRC) as SLUB on your microbenchmark. On Intel's OLTP workload it was as good as SLAB.
The linked lists were similar to SLOB/SLUB IIRC.
Quoted 2 lines
plus unresolved issues
with crashes on boot.
Was due to a hack using per_cpu definition for a node field (some systems would have nodes not equal to a CPU number). I don't think there were any problems left.
Quoted 4 lines
I did not see you work on SLAB improvements. Seemed
that other things had higher priority. The work on slab allocators in general is not well funded, not high priority and is a side issue. The time that I can spend on this is also limited.
I heard SLUB was just about to get there with new per-cpu accessors. That didn't seem to help too much in real world. I would have liked more time on SLAB but unfortunately have not until now.
It seems that it is *still* the best and most mature allocator we have for most users, and the most widely deployed one. So AFAIKS it still makes sense to incrementally improve it rather than take something else.
Quoted 11 lines
Quoted 5 lines
Quoted 3 lines
and want to get this done then please invest some time in SLAB to get it cleaned up. I have some doubt that you are aware of the difficulties that you will encounter.
I am working on it. We'll see.
I think we agreee on one thing regardless of SLAB or SLUB as a base: It would be good to put the best approaches together to form a superior slab allocator. I just think its much easier to do give a mature and clean code base in SLUB. If we both work on this then this may coalesce at some point.
And I've listed my gripes with SLUB countless times, so I won't any more.
Quoted 18 lines
The main gripes with SLAB
- Code base difficult to maintain. Has grown over almost 2 decades.
- Alien caches need to be kept under control. Various hacky ways are implemented to bypass that problem.
- Locking issues because of long hold times of per node lock. SLUB has locking on per page level. This is important for high number of threads per node. Westmere has already 12. EX 24 and it way grow from there.
- Debugging features and recovery mechanisms.
- Off or on page slab metadata causes space wastage, complex allocation and locking and alignment issues. SLED replaces that metadata structure with a bitfield in the page struct. This may also save access to additional cacheline and maybe allow freeing of objects to a slab page without taking locks.
- Variable and function naming is confusing.
- OS noise caused by periodic cache cleaning (which requires scans over all caches of all slabs on every processor).
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · Nick Piggin
Quoted 5 lines
Quoted 2 lines
realized that incremental improvements to SLAB would likely be a far better idea.
It looked to me as if there was a major conceptual issue with the linked lists used for objects that impacted performance
With SLQB's linked list? No. Single threaded cache hot performance was the same (+/- a couple of cycles IIRC) as SLUB on your microbenchmark. On Intel's OLTP workload it was as good as SLAB.
The linked lists were similar to SLOB/SLUB IIRC.
Yes that is the problem. So it did not address the cache cold regressions in SLUB. SLQB mostly addressed the slow path frequency on free.
The design of SLAB is superior for cache cold objects since SLAB does not touch the objects on alloc and free (if one requires similar cache cold performance from other slab allocators) thats why I cleaned up the per cpu queueing concept in SLAB (easy now with the percpu allocator and operations) and came up with SLEB. At the same time this also addresses the slowpath issues on free. I am not entirely sure how to deal with the NUMAness but I want to focus more on machines with low node counts.
The problem with SLAB was that so far the "incremental improvements" have lead to more deteriorations in the maintainability of the code. There are multiple people who have tried going this route that you propose.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 12 lines · Christoph Lameter
Quoted 10 lines · Nick Piggin
Quoted 5 lines
Quoted 2 lines
realized that incremental improvements to SLAB would likely be a far better idea.
It looked to me as if there was a major conceptual issue with the linked lists used for objects that impacted performance
With SLQB's linked list? No. Single threaded cache hot performance was the same (+/- a couple of cycles IIRC) as SLUB on your microbenchmark. On Intel's OLTP workload it was as good as SLAB.
Quoted 5 lines
Quoted 1 line
The linked lists were similar to SLOB/SLUB IIRC.
Yes that is the problem. So it did not address the cache cold regressions in SLUB. SLQB mostly addressed the slow path frequency on free.
This is going a bit off topic considering that I'm not pushing SLQB or any concept from SLQB (just yet at least). As far as I know there were no cache cold regressions in SLQB.
Quoted 9 lines
The design of SLAB is superior for cache cold objects since SLAB does not touch the objects on alloc and free (if one requires similar cache cold performance from other slab allocators) thats why I cleaned up the per cpu queueing concept in SLAB (easy now with the percpu allocator and operations) and came up with SLEB. At the same time this also addresses the slowpath issues on free. I am not entirely sure how to deal with the NUMAness but I want to focus more on machines with low node counts.
Quoted 3 lines
The problem with SLAB was that so far the "incremental improvements" have lead to more deteriorations in the maintainability of the code. There are multiple people who have tried going this route that you propose.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 14 lines · Christoph Lameter
Quoted 11 lines · Nick Piggin
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · Nick Piggin
Well I'm glad you've conceded that queues are useful for high performance computing, and that higher order allocations are not a free and unlimited resource.
Ahem. I have never made any such claim and would never make them. And "conceding" something ???
Well, you were quite vocal about the subject.
Quoted 7 lines
I was always vocal about the huge amounts of queues and the complexity coming with alien caches etc. The alien caches were introduced against my objections on the development team that did the NUMA slab. But even SLUB has "queues" as many have repeatedly pointed out. The queuing is different though in order to minimize excessive NUMA queueing. IMHO the NUMA design of SLAB has fundamental problems because it implements its own "NUMAness" aside from the page allocator.
And by the way I disagreed completely that this is a problem. And you never demonstrated that it is a problem.
It's totally unproductive to say things like it implements its own "NUMAness" aside from the page allocator. I can say SLUB implements its own "numaness" because it is checking for objects matching NUMA requirements too.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Nick Piggin
And by the way I disagreed completely that this is a problem. And you never demonstrated that it is a problem.
It's totally unproductive to say things like it implements its own "NUMAness" aside from the page allocator. I can say SLUB implements its own "numaness" because it is checking for objects matching NUMA requirements too.
SLAB implement numa policies etc in the SLAB logic. It has its own rotor now.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 5 lines · Christoph Lameter
Quoted 3 lines · Nick Piggin
And by the way I disagreed completely that this is a problem. And you never demonstrated that it is a problem.
Quoted 7 lines
Quoted 4 lines
It's totally unproductive to say things like it implements its own "NUMAness" aside from the page allocator. I can say SLUB implements its own "numaness" because it is checking for objects matching NUMA requirements too.
SLAB implement numa policies etc in the SLAB logic. It has its own rotor now.
We all know. I am saying it is unproductive because you just claim that it is some fundamental problem without why it is a problem.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>