Skip to contentSkip to conversation

mm: multi-gen LRU scanning for page promotion

RFC PATCH New Kinsey Ho 1 participant 3 messages Sep 10, 2025

Subsystems (4)
lore
Share & tools
lore

Kinsey Ho

author
#0 lore raw

v2 cover mm: multi-gen LRU scanning for page promotion

This patch series introduces a software-based approach to identifying hot pages for promotion in tiered memory systems, particularly those leveraging CXL-attached memory, by utilizing the Multi-Generational LRU (MGLRU) framework. This method is designed to complement hardware-based hotness detection mechanisms like Intel PMU sampling, AMD IBS, or dedicated CXL memory monitoring units, providing a more comprehensive view of page access patterns, similar to kmmscand [1].

We propose to utilize MGLRU's existing infrastructure to provide hot page information. A key benefit here is the reuse of the MGLRU page table walk code, thus avoiding the overhead and duplication of effort involved in implementing a separate page table scanning mechanism. The working set reporting proposal [2] also reuses MGLRU's infrastructure, but focuses on cold page detection. It provides its own aging daemon, which could additionally provide hot page information by integrating this proof-of-concept.

This series is based on and relies on [3] as the aggregation and promotion engine to implement the promotion policies. This is still in the RFC stage as a proof-of-concept and requires more rigorous testing.

---
v2: Fixed a correctness issue during page table walk, where, on resume, one page found with the access-bit set was incorrectly not added to the pghot data structure provided by [3].

v1: https://lore.kernel.org/linux-mm/[email protected]/
---

[1] [RFC PATCH V3 00/17] mm: slowtier page promotion based on PTE A bit
https://lore.kernel.org/linux-mm/[email protected]/

[2] [PATCH v4 0/9] mm: workingset reporting
https://lore.kernel.org/linux-mm/[email protected]/

[3] [RFC PATCH v2 0/8] mm: Hot page tracking and promotion infrastructure
https://lore.kernel.org/linux-mm/[email protected]/

Kinsey Ho (2):
mm: mglru: generalize page table walk
mm: klruscand: use mglru scanning for page promotion

 include/linux/mmzone.h |   5 ++
 mm/Kconfig             |   8 ++
 mm/Makefile            |   1 +
 mm/internal.h          |   4 +
 mm/klruscand.c         | 118 +++++++++++++++++++++++++++
 mm/vmscan.c            | 181 +++++++++++++++++++++++++++++++----------
 6 files changed, 272 insertions(+), 45 deletions(-)
 create mode 100644 mm/klruscand.c
Signature · 1 line
2.51.0.384.g4c02a37b29-goog
author
#1 lore raw

RFC PATCH v2 1/2 mm: mglru: generalize page table walk

Refactor the existing MGLRU page table walking logic to make it resumable.

Additionally, introduce two hooks into the MGLRU page table walk:
accessed callback and flush callback. The accessed callback is called for each accessed page detected via the scanned accessed bit. The flush callback is called when the accessed callback reports that a flush is required. This allows for processing pages in batches for efficiency.

With a generalised page table walk, introduce a new scan function which repeatedly scans on the same young generation and does not add a new young generation.

---
 include/linux/mmzone.h |   5 ++
 mm/internal.h          |   4 +
 mm/vmscan.c            | 181 +++++++++++++++++++++++++++++++----------
 3 files changed, 145 insertions(+), 45 deletions(-)
include/linux/mmzone.h +5 -0
index f7094babed10..a30fbdbb1a57 100644
@@ -533,6 +533,8 @@ struct lru_gen_mm_walk {
 	unsigned long seq;
 	/* the next address within an mm to scan */
 	unsigned long next_addr;
+	/* called for each accessed pte/pmd */
+	bool (*accessed_cb)(unsigned long pfn);
 	/* to batch promoted pages */
 	int nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES];
 	/* to batch the mm stats */
@@ -540,6 +542,9 @@ struct lru_gen_mm_walk {
 	/* total batched items */
 	int batched;
 	int swappiness;
+	/* for the pmd under scanning */
+	int nr_young_pte;
+	int nr_total_pte;
 	bool force_scan;
 };
 
mm/internal.h +4 -0
index 45b725c3dc03..f8ca128f61ab 100644
@@ -538,6 +538,10 @@ extern unsigned long highest_memmap_pfn;
 bool folio_isolate_lru(struct folio *folio);
 void folio_putback_lru(struct folio *folio);
 extern void reclaim_throttle(pg_data_t *pgdat, enum vmscan_throttle_state reason);
+void set_task_reclaim_state(struct task_struct *task,
+				   struct reclaim_state *rs);
+void lru_gen_scan_lruvec(struct lruvec *lruvec, unsigned long seq,
+			 bool (*accessed_cb)(unsigned long), void (*flush_cb)(void));
 #ifdef CONFIG_NUMA
 int user_proactive_reclaim(char *buf,
 			   struct mem_cgroup *memcg, pg_data_t *pgdat);
mm/vmscan.c +136 -45

361 lines not on this page · raw message

Signature · 1 line
2.51.0.384.g4c02a37b29-goog
author
#2 lore raw

RFC PATCH v2 2/2 mm: klruscand: use mglru scanning for page promotion

In reply to Kinsey Ho

Introduce a new kernel daemon, klruscand, that periodically invokes the MGLRU page table walk. It leverages the new callbacks to gather access information and forwards it to the kpromoted daemon for promotion decisions.

This benefits from reusing the existing MGLRU page table walk infrastructure, which is optimized with features such as hierarchical scanning and bloom filters to reduce CPU overhead.

As an additional optimization to be added in the future, we can tune the scan intervals for each memcg.

---
 mm/Kconfig     |   8 ++++
 mm/Makefile    |   1 +
 mm/klruscand.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 127 insertions(+)
 create mode 100644 mm/klruscand.c
mm/Kconfig +8 -0
index 8b236eb874cf..6d53c1208729 100644
@@ -1393,6 +1393,14 @@ config PGHOT
 	  by various sources. Asynchronous promotion is done by per-node
 	  kernel threads.
 
+config KLRUSCAND
+	bool "Kernel lower tier access scan daemon"
+	default y
+	depends on PGHOT && LRU_GEN_WALKS_MMU
+	help
+	  Scan for accesses from lower tiers by invoking MGLRU to perform
+	  page table walks.
+
 source "mm/damon/Kconfig"
 
 endmenu
mm/Makefile +1 -0
index ecdd5241bea8..05a96ec35aa3 100644
@@ -148,3 +148,4 @@ obj-$(CONFIG_EXECMEM) += execmem.o
 obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o
 obj-$(CONFIG_PT_RECLAIM) += pt_reclaim.o
 obj-$(CONFIG_PGHOT) += pghot.o
+obj-$(CONFIG_KLRUSCAND) += klruscand.o
mm/klruscand.c new file +118 -0

118 lines not on this page · raw message

Signature · 1 line
2.51.0.384.g4c02a37b29-goog