HUGETLB pages on multiple nodes
Seth, It looks like at boot time all the hugepages will get allocated on the boot node. We'd like to spread the pages across multiple nodes on NUMA systems. Here's a simple patch to round-robin the pages during kernel init. Comments?
Nick
=== arch/ia64/mm/hugetlbpage.c 1.2 vs edited ==--- 1.2/arch/ia64/mm/hugetlbpage.c Mon May 19 09:18:44 2003
+++ edited/arch/ia64/mm/hugetlbpage.c Thu Jul 10 11:35:05 2003
@@ -419,7 +419,12 @@
struct page *page;
for (i = 0; i < htlbpage_max; ++i) {
+#ifdef CONFIG_NUMA
+ int node = i % num_memblks;
+ page = alloc_pages_node(node, __GFP_HIGHMEM, HUGETLB_PAGE_ORDER);
+#else
page = alloc_pages(__GFP_HIGHMEM, HUGETLB_PAGE_ORDER);
+#endif /* CONFIG_NUMA */
if (!page)
break;
for (j = 0; j < HPAGE_SIZE/PAGE_SIZE; ++j)
Quoted 4 lines · Nick Pollitt
Seth, It looks like at boot time all the hugepages will get allocated on the boot node. We'd like to spread the pages across multiple nodes on NUMA systems. Here's a simple patch to round-robin the pages during kernel init. Comments?
I shipped some stuff for arch/i386/ that does node-local allocations at runtime, too.
-- wli
Review
Quoted 6 lines · Nick Pollitt
(quoted diff) +5
-0+#ifdef CONFIG_NUMA + int node = i % num_memblks; + page = alloc_pages_node(node, __GFP_HIGHMEM, HUGETLB_PAGE_ORDER); +#else page = alloc_pages(__GFP_HIGHMEM, HUGETLB_PAGE_ORDER); +#endif /* CONFIG_NUMA */
Why the num_memblks? We care for nodes, not memblks, don't we?
Also you can just use alloc_pages_node directly in the non-NUMA code.
See how alloc_pages is implemented in 2.5:
#define alloc_pages(gfp_mask, order) \
alloc_pages_node(numa_node_id(), gfp_mask, order)