SLUB: Optimize slab_free() debug check
Subsystems (2)
This patch optimizes slab_free() debug check to use "c->node != NUMA_NO_NODE" instead of "c->node >= 0" because the former generates smaller code on x86-64:
Before:
4736: 48 39 70 08 cmp %rsi,0x8(%rax)
473a: 75 26 jne 4762 <kfree+0xa2>
473c: 44 8b 48 10 mov 0x10(%rax),%r9d
4740: 45 85 c9 test %r9d,%r9d
4743: 78 1d js 4762 <kfree+0xa2>After:
4736: 48 39 70 08 cmp %rsi,0x8(%rax)
473a: 75 23 jne 475f <kfree+0x9f>
473c: 83 78 10 ff cmpl $0xffffffffffffffff,0x10(%rax)
4740: 74 1d je 475f <kfree+0x9f>This patch also cleans up __slab_alloc() to use NUMA_NO_NODE instead of "-1" for enabling debugging for a per-CPU cache.
Cc Christoph Lameter <[email protected]>
Cc David Rientjes <[email protected]>
Signed-off-by Pekka Enberg <[email protected]>
--- mm/slub.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
mm/slub.c +2 -2
@@ -1718,7 +1718,7 @@ debug: c->page->inuse++; c->page->freelist =get_freepointer (s, object); - c->node =-1 ; + c->node =NUMA_NO_NODE ; goto unlock_out; }
@@ -1895,7 +1895,7 @@ static __always_inline void slab_free(struct kmem_cache *s,slab_free_hook_irq (s, x); -if (likely (page == c->page && c->node >=0 )) { +if (likely (page == c->page && c->node !=NUMA_NO_NODE )) {set_freepointer (s, object, c->freelist); c->freelist = object;stat (s,FREE_FASTPATH );
Signature · 1 line
1.6.3.3
Quoted 18 lines · Pekka Enberg
This patch optimizes
slab_free()debug check to use "c->node!=NUMA_NO_NODE" instead of "c->node>= 0" because the former generates smaller code on x86-64:Before:
4736: 48 39 70 08 cmp %rsi,0x8(%rax) 473a: 75 26 jne 4762 <kfree+0xa2> 473c: 44 8b 48 10 mov 0x10(%rax),%r9d 4740: 45 85 c9 test %r9d,%r9d 4743: 78 1d js 4762 <kfree+0xa2>After:
4736: 48 39 70 08 cmp %rsi,0x8(%rax) 473a: 75 23 jne 475f <kfree+0x9f> 473c: 83 78 10 ff cmpl $0xffffffffffffffff,0x10(%rax) 4740: 74 1d je 475f <kfree+0x9f>
Quoted 6 lines
This patch also cleans up
__slab_alloc()to useNUMA_NO_NODEinstead of "-1" for enabling debugging for a per-CPU cache.Cc Christoph Lameter <[email protected]>Cc David Rientjes <[email protected]>Signed-off-by Pekka Enberg <[email protected]>
Acked-by David Rientjes <[email protected]>
You could compare to NUMA_NO_NODE in show_slab_objects(), as well, for the same optimization.