mm/slub: disallow obj's allocation on page with mismatched pfmemalloc purpose
Subsystems (2)
PATCH 1/1 mm/slub: disallow obj's allocation on page with mismatched pfmemalloc purpose
new_slab_objects() always return c->page matching the required gfpflags, but the current code is misleading and ___slab_alloc->deactivate_slab seems to allow not-pfmemalloc purpose obj to be allocated from pfmemalloc-purpose page. This patch re-organize the code to eliminate the confusion.
--- mm/slub.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
mm/slub.c +5 -7
@@ -2551,23 +2551,21 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, } freelist =new_slab_objects (s, gfpflags, node, &c); -if (unlikely (!freelist)) {slab_out_of_memory (s, gfpflags, node);return NULL ; } +VM_BUG_ON (!pfmemalloc_match (page, gfpflags)); page = c->page; -if (likely (!kmem_cache_debug (s) &&pfmemalloc_match (page, gfpflags))) + if (likely(!kmem_cache_debug(s)) goto load_freelist;/* Only entered in the debug case */ -if (kmem_cache_debug (s) && - !alloc_debug_processing (s, page, freelist, addr)) +if (!alloc_debug_processing (s, page, freelist, addr)) goto new_slab;/* Slab failed checks. Next slab needed */ - -deactivate_slab (s, page,get_freepointer (s, freelist), c); -return freelist; +else + goto load_freelist; }/*
Signature · 1 line
2.7.4
Quoted 9 lines · Pingfan Liu
mm/slub.c +1
-2-if (unlikely (!freelist)) {slab_out_of_memory (s, gfpflags, node);return NULL ; } +VM_BUG_ON (!pfmemalloc_match (page, gfpflags)); page = c->page; -if (likely (!kmem_cache_debug (s) &&pfmemalloc_match (page, gfpflags)))
mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2569
Quoted 10 lines
mm/slub.c +2
-4+ if (likely(!kmem_cache_debug(s)) goto load_freelist;/* Only entered in the debug case */ -if (kmem_cache_debug (s) && - !alloc_debug_processing (s, page, freelist, addr)) +if (!alloc_debug_processing (s, page, freelist, addr)) goto new_slab;/* Slab failed checks. Next slab needed */ - -deactivate_slab (s, page,get_freepointer (s, freelist), c);
In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.
Quoted 7 lines
mm/slub.c +2
-1-return freelist; +else + goto load_freelist; }/*
Quoted 15 lines · Christopher Lameter
Quoted 12 lines · Pingfan Liu
mm/slub.c +2
-2-if (unlikely (!freelist)) {slab_out_of_memory (s, gfpflags, node);return NULL ; } +VM_BUG_ON (!pfmemalloc_match (page, gfpflags)); page = c->page; -if (likely (!kmem_cache_debug (s) &&pfmemalloc_match (page, gfpflags))) + if (likely(!kmem_cache_debug(s)) goto load_freelist;
Quoted 12 lines
mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2569
Quoted 7 lines
/* Only entered in the debug case */mm/slub.c +1
-4-if (kmem_cache_debug (s) && - !alloc_debug_processing (s, page, freelist, addr)) +if (!alloc_debug_processing (s, page, freelist, addr)) goto new_slab;/* Slab failed checks. Next slab needed */ - -deactivate_slab (s, page,get_freepointer (s, freelist), c);In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.
Got it.
Thanks,
Pingfan
Quoted 17 lines · Pingfan Liu
Quoted 15 lines · Christopher Lameter
Quoted 12 lines · Pingfan Liu
mm/slub.c +2
-2-if (unlikely (!freelist)) {slab_out_of_memory (s, gfpflags, node);return NULL ; } +VM_BUG_ON (!pfmemalloc_match (page, gfpflags)); page = c->page; -if (likely (!kmem_cache_debug (s) &&pfmemalloc_match (page, gfpflags))) + if (likely(!kmem_cache_debug(s)) goto load_freelist;
Quoted 12 lines
Quoted 8 lines
mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2569
Quoted 7 lines
/* Only entered in the debug case */mm/slub.c +1
-4-if (kmem_cache_debug (s) && - !alloc_debug_processing (s, page, freelist, addr)) +if (!alloc_debug_processing (s, page, freelist, addr)) goto new_slab;/* Slab failed checks. Next slab needed */ - -deactivate_slab (s, page,get_freepointer (s, freelist), c);Quoted 4 lines
In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.
After taking a more closely look at the debug code, I consider whether the alloc_debug_processing() can be also called after get_freelist(s, page), then deactivate_slab() is not required . My justification is the debug code will take the same code path as the non-debug, hence the page will experience the same transition on different list like the non-debug code, and help to detect the bug, also it will improve scalability on SMP.
Besides this, I found the debug code is not scalable well, is it worth to work on it?
Thanks,
Pingfan
Quoted 4 lines · Pingfan Liu
Quoted 4 lines
Quoted 4 lines
In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.
Quoted 9 lines
After taking a more closely look at the debug code, I consider whether the
alloc_debug_processing()can be also called afterget_freelist(s, page), thendeactivate_slab()is not required . My justification is the debug code will take the same code path as the non-debug, hence the page will experience the same transition on different list like the non-debug code, and help to detect the bug, also it will improve scalability on SMP.
Besides this, I found the debug code is not scalable well, is it worth to work on it?
The debug code is kept out of the hot path intentionally because it does not scale well and reduces performance. Its compiled in in case we have to track down a nasty memory corruption bug on a prod kernel that cannot be easily rebuilt.