Skip to contentSkip to conversation

Hugetlb Fungibility for page metadata savings and network performance

RFC PATCH New Sourav Panda 2 participants 4 messages Mar 18 – Mar 21

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

cover Hugetlb Fungibility for page metadata savings and network performance

The purpose of this RFC is to supplement our discussion in LSF/MM-26.

This is sent as a proof of concept. It applies on top of v7.0-rc3.

In VM environments, the guest frequently utilizes 1GB HugeTLB pages to reduce TLB misses and minimize page table walk overhead for network-functions. This has the added benefit that it reduces redundant struct page metadata overhead through Hugepage Vmemmap Optimization (HVO).

While this saves significant overhead memory, this memory is not available for other purposes; such as the hosted VM-workload or pagecache.

The guest must balance two competing memory requirements:
HugeTLB Pool: 1GB pages reserved for high-performance NFV applications and yielding page metadata savings.
Buddy Allocator: 4KB pages required for business logic, system services, and page cache.

Current kernel limitations prevent fungibility of memory between these two pools. As a starting point we propose a hugetlb-shrinker to provide one-way fungibility wherein ~90% guest memory is allocated as 1G hugepages at boot and then converted to buddy pages on demand.

Sourav Panda (2):
mm: add hugepage shrinker for frozen memory mm/hugetlb: skip hugetlb shrinking for proactive reclaim

 include/linux/shrinker.h |   3 +
 mm/Kconfig               |   8 +++
 mm/hugetlb.c             | 131 +++++++++++++++++++++++++++++++++++++++
 mm/internal.h            |   2 +-
 mm/shrinker.c            |  12 ++--
 mm/vmscan.c              |   6 +-
 6 files changed, 154 insertions(+), 8 deletions(-)
Signature · 1 line
2.53.0.959.g497ff81fa9-goog
#1 lore raw

RFC PATCH 1/2 mm: add hugepage shrinker for frozen memory

Implement a shrinker for the hugetlbfs subsystem to provide one-way fungibility, converting unused persistent huge pages back to the buddy system. One Huge page at a time.

This is designed for virtualization user cases, where
a large pool of huge pages is reserved but kept free, acting as a "frozen" memory reservoir. When the host experiences memory pressure, this shrinker thaws the memory by reclaiming huge pages on-demand.

Pass the hugetlb_shrinker_enabled=1 kernel command line param to enable. Please note the nr_huge_pages will change without user intervention.

Both kswapd and direct reclaim can shrink gigantic hugepages when the system is under memory pressure. To safely support concurrent reclaimers (e.g., kswapd and multiple direct reclaim tasks), a new mutex hugepage_shrink_mutex is introduced.

---
 include/linux/shrinker.h |   2 +
 mm/Kconfig               |   9 +++
 mm/hugetlb.c             | 125 +++++++++++++++++++++++++++++++++++++++
 mm/shrinker.c            |   2 +
 4 files changed, 138 insertions(+)
include/linux/shrinker.h +2 -0
index 1a00be90d93a..5374c251ee9e 100644
@@ -51,6 +51,8 @@ struct shrink_control {
 	 */
 	unsigned long nr_scanned;
 
+	s8 priority;
+
 	/* current memcg being shrunk (for memcg aware shrinkers) */
 	struct mem_cgroup *memcg;
 };
mm/Kconfig +9 -0
index ebd8ea353687..a88f370c7485 100644
@@ -769,6 +769,15 @@ config NOMMU_INITIAL_TRIM_EXCESS
 config ARCH_WANT_GENERAL_HUGETLB
 	bool
 
+config HUGETLB_FROZEN_MEMORY_SHRINKER
+	bool "HugeTLB Frozen Memory Shrinker"
+	depends on HUGETLBFS
+	help
+	  Enables a shrinker for the hugetlb subsystem that allows
+	  unused huge pages to be released back to the buddy
+	  system under memory pressure. One huge page at a time.
+	  Further gated by kernel cmdline hugetlb_shrinker_enabled.
+
 config ARCH_WANTS_THP_SWAP
 	def_bool n
 
mm/hugetlb.c +125 -0

143 lines not on this page · raw message

mm/shrinker.c +2 -0
index 7b61fc0ee78f..8a7a05182465 100644
@@ -529,6 +529,7 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
 				.gfp_mask = gfp_mask,
 				.nid = nid,
 				.memcg = memcg,
+				.priority = priority,
 			};
 			struct shrinker *shrinker;
 			int shrinker_id = calc_shrinker_id(index, offset);
@@ -654,6 +655,7 @@ unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg,
 			.gfp_mask = gfp_mask,
 			.nid = nid,
 			.memcg = memcg,
+			.priority = priority,
 		};
 
 		if (!shrinker_try_get(shrinker))
Signature · 1 line
2.53.0.983.g0bb29b3bc5-goog
#2 lore raw

RFC PATCH 2/2 mm/hugetlb: skip hugetlb shrinking for proactive reclaim

In reply to Sourav Panda

Scan control can indicate if we are in the proactive reclaim mode.

Pass that to shrinker control and preclude frozen memory hugetlb shrinking if set.

---
 include/linux/shrinker.h |  1 +
 mm/hugetlb.c             |  6 ++++++
 mm/internal.h            |  2 +-
 mm/shrinker.c            | 10 ++++++----
 mm/vmscan.c              |  6 +++---
 5 files changed, 17 insertions(+), 8 deletions(-)
include/linux/shrinker.h +1 -0
index 5374c251ee9e..973d5fd68803 100644
@@ -52,6 +52,7 @@ struct shrink_control {
 	unsigned long nr_scanned;
 
 	s8 priority;
+	bool proactive;
 
 	/* current memcg being shrunk (for memcg aware shrinkers) */
 	struct mem_cgroup *memcg;
mm/hugetlb.c +6 -0
index d4953ff1dda1..a70aed7c8665 100644
@@ -4145,6 +4145,9 @@ static unsigned long hugepage_shrinker_count(struct shrinker *s,
 	if (sc->priority >= DEF_PRIORITY - 6)
 		return 0;
 
+	if (sc->proactive)
+		return 0;
+
 	if (!gigantic_page_runtime_supported())
 		return 0;
 
@@ -4193,6 +4196,9 @@ static unsigned long hugepage_shrinker_scan(struct shrinker *s,
 	if (sc->nr_to_scan == 0)
 		return SHRINK_STOP;
 
+	if (sc->proactive)
+		return SHRINK_STOP;
+
 	if (!gigantic_page_runtime_supported())
 		return SHRINK_STOP;
 
mm/internal.h +1 -1
index cb0af847d7d9..cccb68d723d4 100644
@@ -1660,7 +1660,7 @@ void __meminit __init_page_from_nid(unsigned long pfn, int nid);
 
 /* shrinker related functions */
 unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg,
-			  int priority);
+			  int priority, bool proactive);
 
 int shmem_add_to_page_cache(struct folio *folio,
 			    struct address_space *mapping,
mm/shrinker.c +6 -4
index 8a7a05182465..21b8f0b9d092 100644
@@ -467,7 +467,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
 
 #ifdef CONFIG_MEMCG
 static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
-			struct mem_cgroup *memcg, int priority)
+			struct mem_cgroup *memcg, int priority, bool proactive)
 {
 	struct shrinker_info *info;
 	unsigned long ret, freed = 0;
@@ -530,6 +530,7 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
 				.nid = nid,
 				.memcg = memcg,
 				.priority = priority,
+				.proactive = proactive,
 			};
 			struct shrinker *shrinker;
 			int shrinker_id = calc_shrinker_id(index, offset);
@@ -586,7 +587,7 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
 }
 #else /* !CONFIG_MEMCG */
 static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
-			struct mem_cgroup *memcg, int priority)
+			struct mem_cgroup *memcg, int priority, bool proactive)
 {
 	return 0;
 }
@@ -613,7 +614,7 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid,
  * Returns the number of reclaimed slab objects.
  */
 unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg,
-			  int priority)
+			  int priority, bool proactive)
 {
 	unsigned long ret, freed = 0;
 	struct shrinker *shrinker;
@@ -626,7 +627,7 @@ unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg,
 	 * oom.
 	 */
 	if (!mem_cgroup_disabled() && !mem_cgroup_is_root(memcg))
-		return shrink_slab_memcg(gfp_mask, nid, memcg, priority);
+		return shrink_slab_memcg(gfp_mask, nid, memcg, priority, proactive);
 
 	/*
 	 * lockless algorithm of global shrink.
@@ -656,6 +657,7 @@ unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg,
 			.nid = nid,
 			.memcg = memcg,
 			.priority = priority,
+			.proactive = proactive,
 		};
 
 		if (!shrinker_try_get(shrinker))
mm/vmscan.c +3 -3
index 0fc9373e8251..39151d1edeff 100644
@@ -432,7 +432,7 @@ static unsigned long drop_slab_node(int nid)
 
 	memcg = mem_cgroup_iter(NULL, NULL, NULL);
 	do {
-		freed += shrink_slab(GFP_KERNEL, nid, memcg, 0);
+		freed += shrink_slab(GFP_KERNEL, nid, memcg, 0, false);
 	} while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL);
 
 	return freed;
@@ -4925,7 +4925,7 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc)
 
 	success = try_to_shrink_lruvec(lruvec, sc);
 
-	shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, sc->priority);
+	shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, sc->priority, sc->proactive);
 
 	if (!sc->proactive)
 		vmpressure(sc->gfp_mask, memcg, false, sc->nr_scanned - scanned,
@@ -6020,7 +6020,7 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc)
 		shrink_lruvec(lruvec, sc);
 
 		shrink_slab(sc->gfp_mask, pgdat->node_id, memcg,
-			    sc->priority);
+			    sc->priority, sc->proactive);
 
 		/* Record the group's reclaim efficiency */
 		if (!sc->proactive)
Signature · 1 line
2.53.0.983.g0bb29b3bc5-goog
#3 lore raw

Re: [LSF/MM/BPF TOPIC][RFC PATCH 2/2] mm/hugetlb: skip hugetlb shrinking for proactive reclaim

kernel test robot noticed the following build warnings:

Hi Sourav,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on akpm-mm/mm-everything]
[also build test WARNING on linus/master v7.0-rc4 next-20260320] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sourav-Panda/mm-add-hugepage-shrinker-for-frozen-memory/20260319-080721
base:   https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything
patch link:    https://lore.kernel.org/r/20260318234126.3216529-3-souravpanda%40google.com
patch subject: [LSF/MM/BPF TOPIC][RFC PATCH 2/2] mm/hugetlb: skip hugetlb shrinking for proactive reclaim
config: nios2-allnoconfig (https://download.01.org/0day-ci/archive/20260321/[email protected]/config)
compiler: nios2-linux-gcc (GCC) 11.5.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260321/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> Warning: mm/shrinker.c:620 function parameter 'proactive' not described in 'shrink_slab'
>> Warning: mm/shrinker.c:620 function parameter 'proactive' not described in 'shrink_slab'
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki