Make slab use alloc_pages directly

__get_free_pages() calls alloc_pages, finds the page_address() and throws away the struct page *. Slab then calls virt_to_page to get it back again. Much more efficient for slab to call alloc_pages itself, as well as making the NUMA and non-NUMA cases more similarr to each other.

__get_free_pages() calls alloc_pages, finds the page_address() and throws away the struct page *. Slab then calls virt_to_page to get it back again. Much more efficient for slab to call alloc_pages itself, as well as making the NUMA and non-NUMA cases more similarr to each other.

Signed-off-by Matthew Wilcox <[email protected]>

Index: linux-2.6/mm/slab.c
===================================================================
RCS file: /var/cvs/linux-2.6/mm/slab.c,v
retrieving revision 1.29
diff -u -p -r1.29 slab.c

mm/slab.c +1 -4

@@ -894,16 +894,13 @@ static void *kmem_getpages(kmem_cache_t 
 
 	flags |= cachep->gfpflags;
 	if (likely(nodeid == -1)) {
-		addr = (void*)__get_free_pages(flags, cachep->gfporder);
-		if (!addr)
-			return NULL;
-		page = virt_to_page(addr);
+		page = alloc_pages(flags, cachep->gfporder);
 	} else {
 		page = alloc_pages_node(nodeid, flags, cachep->gfporder);

__get_free_pages() calls alloc_pages, finds the page_address() and throws away the struct page *. Slab then calls virt_to_page to get it back again. Much more efficient for slab to call alloc_pages itself, as well as making the NUMA and non-NUMA cases more similarr to each other.

Quoted 4 lines · Matthew Wilcox

__get_free_pages() calls alloc_pages, finds the page_address() and throws away the struct page *. Slab then calls virt_to_page to get it back again. Much more efficient for slab to call alloc_pages itself, as well as making the NUMA and non-NUMA cases more similarr to each other.

Here is a better patch:

Remove the special case of nodeid == -1. Instead, use numa_node_id() when calling cache_grow().

Signed-off-by Brian Gerst <[email protected]>

diff -urN linux-2.6.11-rc2-bk/mm/slab.c linux/mm/slab.c

mm/slab.c +6 -12

@@ -893,17 +893,11 @@
 	int i;
 
 	flags |= cachep->gfpflags;
-	if (likely(nodeid == -1)) {
-		addr = (void*)__get_free_pages(flags, cachep->gfporder);
-		if (!addr)
-			return NULL;
-		page = virt_to_page(addr);
-	} else {
-		page = alloc_pages_node(nodeid, flags, cachep->gfporder);
-		if (!page)
-			return NULL;
-		addr = page_address(page);
-	}
+
+	page = alloc_pages_node(nodeid, flags, cachep->gfporder);
+	if (!page)
+		return NULL;
+	addr = page_address(page);
 
 	i = (1 << cachep->gfporder);
 	if (cachep->flags & SLAB_RECLAIM_ACCOUNT)

@@ -2065,7 +2059,7 @@
 
 	if (unlikely(!ac->avail)) {
 		int x;
-		x = cache_grow(cachep, flags, -1);
+		x = cache_grow(cachep, flags, numa_node_id());
 
 		// cache_grow can reenable interrupts, then ac could change.
 		ac = ac_data(cachep);