mm: Set HASHDIST_DEFAULT to 1 for x86_64 NUMA
PATCH 1/1 mm: Set HASHDIST_DEFAULT to 1 for x86_64 NUMA
Enable system hashtable memory to be distributed among nodes on x86_64 NUMA
Forcing the kernel to use node interleaved vmalloc instead of bootmem for the system hashtable memory (alloc_large_system_hash) reduces the memory imbalance on node 0 by around 40MB on a 8 node x86_64 NUMA box:
Before the following patch, on bootup of a 8 node box:
Node 0 MemTotal: 3407488 kB Node 0 MemFree: 3206296 kB Node 0 MemUsed: 201192 kB Node 0 Active: 7012 kB Node 0 Inactive: 512 kB Node 0 Dirty: 0 kB Node 0 Writeback: 0 kB Node 0 FilePages: 1912 kB Node 0 Mapped: 420 kB Node 0 AnonPages: 5612 kB Node 0 PageTables: 468 kB Node 0 NFS_Unstable: 0 kB Node 0 Bounce: 0 kB Node 0 Slab: 5408 kB Node 0 SReclaimable: 644 kB Node 0 SUnreclaim: 4764 kB
After the patch (or using hashdist=1 on the kernel command line):
Node 0 MemTotal: 3407488 kB Node 0 MemFree: 3247608 kB Node 0 MemUsed: 159880 kB Node 0 Active: 3012 kB Node 0 Inactive: 616 kB Node 0 Dirty: 0 kB Node 0 Writeback: 0 kB Node 0 FilePages: 2424 kB Node 0 Mapped: 380 kB Node 0 AnonPages: 1200 kB Node 0 PageTables: 396 kB Node 0 NFS_Unstable: 0 kB Node 0 Bounce: 0 kB Node 0 Slab: 6304 kB Node 0 SReclaimable: 1596 kB Node 0 SUnreclaim: 4708 kB
I guess it is a good idea to keep HASHDIST_DEFAULT "on" for x86_64 NUMA since x86_64 has no dearth of vmalloc space? Or maybe enable hash distribution for all 64bit NUMA arches? The following patch does it only for x86_64.
Signed-off-by Pravin B. Shelar <[email protected]>
Signed-off-by Ravikiran Thirumalai <[email protected]>
Signed-off-by Shai Fultheim <[email protected]>
include/linux/bootmem.h +2 -2
@@ -122,9 +122,9 @@ extern void *alloc_large_system_hash(con#define HASH_EARLY 0x00000001/* Allocating during early boot? */ /* Only NUMA needs hash distribution. -* IA64 is known to have sufficient vmalloc space. +* IA64 and x86_64 have sufficient vmalloc space. */ -#if defined(CONFIG_NUMA ) && defined(CONFIG_IA64 ) +#if defined(CONFIG_NUMA ) && (defined(CONFIG_IA64 ) || defined(CONFIG_X86_64 ))#define HASHDIST_DEFAULT 1#else #define HASHDIST_DEFAULT 0