Skip to contentSkip to conversation

mm/slub: disallow obj's allocation on page with mismatched pfmemalloc purpose

PATCH In review Pingfan Liu 2 participants 5 messages Sep 26, 2018 – Oct 2, 2018

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/slub: disallow obj's allocation on page with mismatched pfmemalloc purpose

new_slab_objects() always return c->page matching the required gfpflags, but the current code is misleading and ___slab_alloc->deactivate_slab seems to allow not-pfmemalloc purpose obj to be allocated from pfmemalloc-purpose page. This patch re-organize the code to eliminate the confusion.

Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
---
 mm/slub.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)
mm/slub.c +5 -7
index a68c2ae..e152634 100644
@@ -2551,23 +2551,21 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node,
 	}
 
 	freelist = new_slab_objects(s, gfpflags, node, &c);
-
 	if (unlikely(!freelist)) {
 		slab_out_of_memory(s, gfpflags, node);
 		return NULL;
 	}
 
+	VM_BUG_ON(!pfmemalloc_match(page, gfpflags));
 	page = c->page;
-	if (likely(!kmem_cache_debug(s) && pfmemalloc_match(page, gfpflags)))
+	if (likely(!kmem_cache_debug(s))
 		goto load_freelist;
 
 	/* Only entered in the debug case */
-	if (kmem_cache_debug(s) &&
-			!alloc_debug_processing(s, page, freelist, addr))
+	if (!alloc_debug_processing(s, page, freelist, addr))
 		goto new_slab;	/* Slab failed checks. Next slab needed */
-
-	deactivate_slab(s, page, get_freepointer(s, freelist), c);
-	return freelist;
+	else
+		goto load_freelist;
 }
 
 /*
Signature · 1 line
2.7.4
#1 lore raw
Quoted 9 lines · Pingfan Liu
mm/slub.c +1 -2
-
 	if (unlikely(!freelist)) {
 		slab_out_of_memory(s, gfpflags, node);
 		return NULL;
 	}
 
+	VM_BUG_ON(!pfmemalloc_match(page, gfpflags));
 	page = c->page;
-	if (likely(!kmem_cache_debug(s) && pfmemalloc_match(page, gfpflags)))

mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2569

Quoted 10 lines
mm/slub.c +2 -4
+	if (likely(!kmem_cache_debug(s))
 		goto load_freelist;
 
 	/* Only entered in the debug case */
-	if (kmem_cache_debug(s) &&
-			!alloc_debug_processing(s, page, freelist, addr))
+	if (!alloc_debug_processing(s, page, freelist, addr))
 		goto new_slab;	/* Slab failed checks. Next slab needed */
-
-	deactivate_slab(s, page, get_freepointer(s, freelist), c);

In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.

Quoted 7 lines
mm/slub.c +2 -1
-	return freelist;
+	else
+		goto load_freelist;
 }
 
 /*
#2 lore raw
Quoted 15 lines · Christopher Lameter
Quoted 12 lines · Pingfan Liu
mm/slub.c +2 -2
-
      if (unlikely(!freelist)) {
              slab_out_of_memory(s, gfpflags, node);
              return NULL;
      }
 
+     VM_BUG_ON(!pfmemalloc_match(page, gfpflags));
      page = c->page;
-     if (likely(!kmem_cache_debug(s) && pfmemalloc_match(page, gfpflags)))
+     if (likely(!kmem_cache_debug(s))
              goto load_freelist;
Quoted 12 lines

mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2569

Quoted 7 lines
      /* Only entered in the debug case */
mm/slub.c +1 -4
-     if (kmem_cache_debug(s) &&
-                     !alloc_debug_processing(s, page, freelist, addr))
+     if (!alloc_debug_processing(s, page, freelist, addr))
              goto new_slab;  /* Slab failed checks. Next slab needed */
-
-     deactivate_slab(s, page, get_freepointer(s, freelist), c);

In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.

Got it.

Thanks,
Pingfan

#3 lore raw
Quoted 17 lines · Pingfan Liu
Quoted 15 lines · Christopher Lameter
Quoted 12 lines · Pingfan Liu
mm/slub.c +2 -2
-
      if (unlikely(!freelist)) {
              slab_out_of_memory(s, gfpflags, node);
              return NULL;
      }
 
+     VM_BUG_ON(!pfmemalloc_match(page, gfpflags));
      page = c->page;
-     if (likely(!kmem_cache_debug(s) && pfmemalloc_match(page, gfpflags)))
+     if (likely(!kmem_cache_debug(s))
              goto load_freelist;
Quoted 12 lines
Quoted 8 lines

mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2569

Quoted 7 lines
      /* Only entered in the debug case */
mm/slub.c +1 -4
-     if (kmem_cache_debug(s) &&
-                     !alloc_debug_processing(s, page, freelist, addr))
+     if (!alloc_debug_processing(s, page, freelist, addr))
              goto new_slab;  /* Slab failed checks. Next slab needed */
-
-     deactivate_slab(s, page, get_freepointer(s, freelist), c);
Quoted 4 lines

In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.

After taking a more closely look at the debug code, I consider whether the alloc_debug_processing() can be also called after get_freelist(s, page), then deactivate_slab() is not required . My justification is the debug code will take the same code path as the non-debug, hence the page will experience the same transition on different list like the non-debug code, and help to detect the bug, also it will improve scalability on SMP.
Besides this, I found the debug code is not scalable well, is it worth to work on it?

Thanks,
Pingfan

#4 lore raw
Quoted 4 lines · Pingfan Liu
Quoted 4 lines
Quoted 4 lines

In the debug case the slab needs to be deactivated. Otherwise the slowpath will not be used and debug checks on the following objects will not be done.

Quoted 9 lines

After taking a more closely look at the debug code, I consider whether the alloc_debug_processing() can be also called after get_freelist(s, page), then deactivate_slab() is not required . My justification is the debug code will take the same code path as the non-debug, hence the page will experience the same transition on different list like the non-debug code, and help to detect the bug, also it will improve scalability on SMP.
Besides this, I found the debug code is not scalable well, is it worth to work on it?

The debug code is kept out of the hot path intentionally because it does not scale well and reduces performance. Its compiled in in case we have to track down a nasty memory corruption bug on a prod kernel that cannot be easily rebuilt.