Skip to contentSkip to conversation

mm/mglru: dynamically protect readahead fault folios under refault pressure

RFC PATCH In review Ehab Ababneh 3 participants 15 messages Sep 1 – Oct 1

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 mm/mglru: dynamically protect readahead fault folios under refault pressure

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from it. The goal is to keep the optimization without keeping readahead pages alive forever.

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually helping.

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the original behavior stand
- decay/reset the credit as generations advance and when an lruvec is initialized

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages around unnecessarily.

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

The fix restores the readahead protection lost by the regression while preserving the original intent of the optimization: do not keep readahead pages around if the workload does not need them.

Fixes 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active")
---
 include/linux/mmzone.h |  2 ++
 mm/swap.c              | 82 ++++++++++++++++++++++++++++++++++++++----
 mm/vmscan.c            |  7 ++++
 mm/workingset.c        | 18 ++++++++++
 4 files changed, 102 insertions(+), 7 deletions(-)
include/linux/mmzone.h +2 -0
index ca2712187147..c998b1e0b8a7 100644
@@ -578,6 +578,8 @@ struct lru_gen_folio {
 	/* can be modified without holding the LRU lock */
 	atomic_long_t evicted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS];
 	atomic_long_t refaulted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS];
+	/* credit: file refaults indicate fault-path file folios need protection */
+	atomic_long_t ra_refaults;
 	/* whether the multi-gen LRU is enabled */
 	bool enabled;
 	/* the memcg generation this lru_gen_folio belongs to */
mm/swap.c +75 -7

99 lines not on this page · raw message

mm/vmscan.c +7 -0
index 56708d1d2dfd..767311593296 100644
@@ -3931,6 +3931,7 @@ static bool inc_max_seq(struct lruvec *lruvec, unsigned long seq, int swappiness
 	bool success;
 	int prev, next;
 	int type, zone;
+	long old, new;
 	struct lru_gen_folio *lrugen = &lruvec->lrugen;
 restart:
 	if (seq < READ_ONCE(lrugen->max_seq))
@@ -3983,6 +3984,11 @@ static bool inc_max_seq(struct lruvec *lruvec, unsigned long seq, int swappiness
 		reset_ctrl_pos(lruvec, type, false);
 
 	WRITE_ONCE(lrugen->timestamps[next], jiffies);
+	/* decay readahead protection credit so stale signal doesn't persist */
+	old = atomic_long_read(&lrugen->ra_refaults);
+	do {
+		new = (old * 3) / 4;
+	} while (!atomic_long_try_cmpxchg(&lrugen->ra_refaults, &old, new));
 	/* make sure preceding modifications appear */
 	smp_store_release(&lrugen->max_seq, lrugen->max_seq + 1);
 unlock:
@@ -5784,6 +5790,7 @@ void lru_gen_init_lruvec(struct lruvec *lruvec)
 
 	lrugen->max_seq = MIN_NR_GENS + 1;
 	lrugen->enabled = lru_gen_enabled();
+	atomic_long_set(&lrugen->ra_refaults, 0);
 
 	for (i = 0; i <= MIN_NR_GENS + 1; i++)
 		lrugen->timestamps[i] = jiffies;
mm/workingset.c +18 -0
index f351798e723a..63baa7220136 100644
@@ -319,6 +319,24 @@ static void lru_gen_refault(struct folio *folio, void *shadow)
 
 	atomic_long_add(delta, &lrugen->refaulted[hist][type][tier]);
 
+	if (type == LRU_GEN_FILE) {
+		/*
+		 * Cap credit at total file pages to avoid runaway while
+		 * allowing sustained protection.
+		 */
+		long cap = lruvec_page_state(lruvec, NR_LRU_BASE + LRU_INACTIVE_FILE) +
+			lruvec_page_state(lruvec, NR_LRU_BASE + LRU_ACTIVE_FILE);
+		long add = (long)delta * VM_READAHEAD_PAGES;
+		long old, new;
+
+		old = atomic_long_read(&lrugen->ra_refaults);
+		do {
+			if (old >= cap)
+				break;
+			new = min(cap, old + add);
+		} while (!atomic_long_try_cmpxchg(&lrugen->ra_refaults, &old, new));
+	}
+
 	if (workingset) {
 		/*
 		 * see folio_add_lru(), where folio_set_active() is
Signature · 1 line
2.43.0
#1 lore raw
Quoted 37 lines · Ehab Ababneh

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from it. The goal is to keep the optimization without keeping readahead pages alive forever.

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually helping.

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the original behavior stand
- decay/reset the credit as generations advance and when an lruvec is initialized

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages around unnecessarily.

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

Quoted 7 lines

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

Hello Ehab,

We ran into the same issue on our side too. I hesitated to report or fix that as I'm working on MGLRU-FG which fixed the problem on my side:

https://lore.kernel.org/linux-mm/[email protected]/

I especially mentioned it, see the parts after:
"recent change in lru_gen_folio_seq that bumps new folios with refs == 1"

Latest version still being tested which you can use directly:
https://github.com/ryncsn/linux/commits/b4/mglru-fg-v1.8/

Do you mind have a look of that as well? I think in the long term that is the right direction. With our test the regression is gone and performance is even better.

And is there any easy way to reproduce the specific case you are reporting?

Quoted 88 lines

The fix restores the readahead protection lost by the regression while preserving the original intent of the optimization: do not keep readahead pages around if the workload does not need them.

Fixes 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active")
---
 include/linux/mmzone.h |  2 ++
 mm/swap.c              | 82 ++++++++++++++++++++++++++++++++++++++----
 mm/vmscan.c            |  7 ++++
 mm/workingset.c        | 18 ++++++++++
 4 files changed, 102 insertions(+), 7 deletions(-)
include/linux/mmzone.h +2 -0
index ca2712187147..c998b1e0b8a7 100644
@@ -578,6 +578,8 @@ struct lru_gen_folio {
 	/* can be modified without holding the LRU lock */
 	atomic_long_t evicted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS];
 	atomic_long_t refaulted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS];
+	/* credit: file refaults indicate fault-path file folios need protection */
+	atomic_long_t ra_refaults;
 	/* whether the multi-gen LRU is enabled */
 	bool enabled;
 	/* the memcg generation this lru_gen_folio belongs to */
mm/swap.c +54 -0
index 588f50d8f1a8..a31c9000868a 100644
@@ -70,6 +70,70 @@ static DEFINE_PER_CPU(struct cpu_fbatches, cpu_fbatches) = {
 	.lock_irq = INIT_LOCAL_LOCK(lock_irq),
 };
 
+#ifdef CONFIG_LRU_GEN
+/* Refill two default readahead windows to amortize shared-counter updates. */
+#define RA_REFAULT_LOCAL_BATCH	(VM_READAHEAD_PAGES * 2)
+
+struct ra_credit_cache {
+	/* Batch shared credit per CPU to avoid a contended atomic RMW per folio. */
+	/* only compared for identity, never dereferenced */
+	struct lru_gen_folio *lrugen;
+	long credit;
+};
+
+static DEFINE_PER_CPU(struct ra_credit_cache, ra_credit_cache);
+
+/*
+ * Spend readahead protection credit from a per-CPU bucket, refilled in batches
+ * from the shared per-lruvec counter, so the fault path avoids an atomic RMW on
+ * a contended cacheline for every folio.
+ */
+static bool lru_gen_take_ra_credit(struct folio *folio)
+{
+	struct lru_gen_folio *lrugen;
+	long nr_pages = folio_nr_pages(folio);
+	struct ra_credit_cache *cache;
+	bool taken = false;
+	long old, new;
+
+	rcu_read_lock();
+	lrugen = &folio_lruvec(folio)->lrugen;
+	cache = get_cpu_ptr(&ra_credit_cache);
+
+	/* credit cached for a different lruvec is forfeited, bounded by the batch */
+	if (cache->lrugen != lrugen) {
+		cache->lrugen = lrugen;
+		cache->credit = 0;
+	}
+
+	if (cache->credit < nr_pages) {
+		old = atomic_long_read(&lrugen->ra_refaults);
+		while (old > 0) {
+			new = old - min_t(long, old, RA_REFAULT_LOCAL_BATCH);
+			if (atomic_long_try_cmpxchg(&lrugen->ra_refaults, &old, new)) {
+				cache->credit += old - new;
+				break;
+			}
+		}
+	}
+
+	if (cache->credit >= nr_pages) {
+		cache->credit -= nr_pages;
+		taken = true;
+	}
+
+	put_cpu_ptr(&ra_credit_cache);
+	rcu_read_unlock();

mm/swap.c static DEFINE_PER_CPU(struct cpu_fbatches, cpu_fbatches) = { · line 136

Quoted 10 lines
mm/swap.c +10 -0
+
+	return taken;
+}
+#else
+static bool lru_gen_take_ra_credit(struct folio *folio)
+{
+	return false;
+}
+#endif /* CONFIG_LRU_GEN */
+

Just an idea. For an short term and easy fix, what if we simply revert than, then only protect in_fault && folio_test_swapbacked folios with PG_active?

#2 lore raw
Quoted 47 lines · Kairui Song
Quoted 44 lines · Ehab Ababneh

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from it. The goal is to keep the optimization without keeping readahead pages alive forever.

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually helping.

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the original behavior stand
- decay/reset the credit as generations advance and when an lruvec is initialized

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages around unnecessarily.

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
  p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
  p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

[...]

Quoted 4 lines

Just an idea. For an short term and easy fix, what if we simply revert than, then only protect in_fault && folio_test_swapbacked folios with PG_active?

Hi Ehab and Kairui,

Thanks very much for your reports and discussion.

I'm not quite sure we want to revert it entirely, as that would immediately regress the workloads improved by the commit, such as the kernel build.
Also, for example, Kairui's cover letter mentioned the LevelDB benchmark:
 "I also retested the LevelDB benchmark from the cache_ext paper [5]. Interestingly, mainline MGLRU already beats CLRU on this one after a recent change in lru_gen_folio_seq that bumps new folios with refs == 1 to the second-oldest generation."

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case actually different from Kairui's?
(The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further gains.)

include/linux/mm_inline.h +3 -2
index 621c8653d8f7..91396c796a34 100644
@@ -239,7 +239,8 @@ static inline unsigned long lru_gen_folio_seq(const struct lruvec *lruvec,
 	 * |<---------------------------- MAX_NR_GENS ---------------------------->|
 	 */
 	if (folio_test_active(folio))
-		gen = MIN_NR_GENS - folio_test_workingset(folio);
+		gen = MIN_NR_GENS - folio_test_workingset(folio) +
+		      (type ? !folio_test_workingset(folio) : 0);
 	else if (reclaiming)
 		gen = MAX_NR_GENS;
 	else if ((!folio_is_file_lru(folio) && !folio_test_swapcache(folio)) ||
@@ -247,7 +248,7 @@ static inline unsigned long lru_gen_folio_seq(const struct lruvec *lruvec,
 		  (folio_test_dirty(folio) || folio_test_writeback(folio))))
 		gen = MIN_NR_GENS;
 	else
-		gen = MAX_NR_GENS - (folio_test_workingset(folio) || folio_test_referenced(folio));
+		gen = MAX_NR_GENS - folio_test_workingset(folio);
 
 	return max(READ_ONCE(lrugen->max_seq) - gen + 1, READ_ONCE(lrugen->min_seq[type]));
 }
mm/folio.c +3 -13
index c02dcea9c03c..2fd835b3b50c 100644
@@ -470,20 +470,10 @@ void folio_add_lru(struct folio *folio)
 			folio_test_unevictable(folio), folio);
 	VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
 
-	/*
-	 * For refaulted workingset folios, set PG_active so they
-	 * can be added to active generations.
-	 * For prefaulted file folios, folio_mark_accessed() sets
-	 * PG_referenced so lru_gen_folio_seq() places them into
-	 * the second oldest generation.
-	 */
+	/* see the comment in lru_gen_folio_seq() */
 	if (lru_gen_enabled() && !folio_test_unevictable(folio) &&
-	    lru_gen_in_fault() && !(current->flags & PF_MEMALLOC)) {
-		if (folio_test_workingset(folio))
-			folio_set_active(folio);
-		else if (!folio_test_referenced(folio))
-			folio_mark_accessed(folio);
-	}
+	    lru_gen_in_fault() && !(current->flags & PF_MEMALLOC))
+		folio_set_active(folio);
 
 	folio_batch_add_and_move(folio, lru_add);
 }
mm/vmscan.c +1 -5
index f11491ee9ed5..aa500ae9371a 100644
@@ -851,11 +851,7 @@ static bool lru_gen_set_refs(struct folio *folio, const vma_flags_t *vma_flags)
 		return false;
 	}
 
-	/* Promote on second access */
-	if (folio_lru_refs(folio) > 1)
-		set_mask_bits(&folio->flags.f, LRU_REFS_FLAGS, BIT(PG_workingset));
-	else
-		folio_mark_accessed(folio);
+	set_mask_bits(&folio->flags.f, LRU_REFS_FLAGS, BIT(PG_workingset));
 	return true;
 }
 #else
mm/workingset.c +4 -6
index 7ac2b88c80ae..a79b73ee9762 100644
@@ -319,13 +319,11 @@ static void lru_gen_refault(struct folio *folio, void *shadow)
 
 	atomic_long_add(delta, &lrugen->refaulted[hist][type][tier]);
 
+	/* see folio_add_lru() where folio_set_active() will be called */
+	if (lru_gen_in_fault())
+		mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta);
+
 	if (workingset) {
-		/*
-		 * see folio_add_lru(), where folio_set_active() is
-		 * called for workingset folios
-		 */
-		if (lru_gen_in_fault())
-			mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta);
 		folio_set_workingset(folio);
 		mod_lruvec_state(lruvec, WORKINGSET_RESTORE_BASE + type, delta);
 	} else
Signature · 1 line
2.34.1
#3 lore raw

Hi Barry, Kairui,

Thanks for the feedback and suggestions.

Quoted 83 lines

-----Original Message-----
From: Barry Song (Xiaomi) <[email protected]>
Sent: Tuesday, September 1, 2026 3:05 PM
To: [email protected]; Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 13 lines · Kairui Song
Quoted 11 lines · Ehab Ababneh

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Quoted 3 lines
Quoted 3 lines

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from it. The goal is to keep the optimization without keeping readahead pages alive

forever.

Quoted 17 lines
Quoted 17 lines

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually helping.

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the
  original behavior stand
- decay/reset the credit as generations advance and when an lruvec is
  initialized

Quoted 4 lines
Quoted 4 lines

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages

around unnecessarily.

Quoted 13 lines
Quoted 12 lines

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
  p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to
  replace folio_set_active"):
  p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

[...]

Quoted 4 lines

Just an idea. For an short term and easy fix, what if we simply revert than, then only protect in_fault && folio_test_swapbacked folios with PG_active?

Hi Ehab and Kairui,

Thanks very much for your reports and discussion.

I'm not quite sure we want to revert it entirely, as that would immediately regress the workloads improved by the commit, such as the kernel build. Also, for example, Kairui's cover letter mentioned the LevelDB benchmark:
 "I also retested the LevelDB benchmark from the cache_ext paper [5]. Interestingly, mainline MGLRU already beats CLRU on this one after a recent change in lru_gen_folio_seq that bumps new folios with refs == 1 to the second-oldest generation."

Quoted 6 lines

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case actually different from Kairui's? (The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further gains.)

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Thanks,
Ehab

Quoted 101 lines
include/linux/mm_inline.h index renamed from include/linux/mm_inline.h +0 -0

621c8653d8f7..91396c796a34 100644

include/linux/mm_inline.h +2 -1
@@ -239,7 +239,8 @@ static inline unsigned long lru_gen_folio_seq(const
 struct lruvec *lruvec,
 	 * |<---------------------------- MAX_NR_GENS ---------------------------->|
 	 */
 	if (folio_test_active(folio))
-		gen = MIN_NR_GENS - folio_test_workingset(folio);
+		gen = MIN_NR_GENS - folio_test_workingset(folio) +
+		      (type ? !folio_test_workingset(folio) : 0);
 	else if (reclaiming)
 		gen = MAX_NR_GENS;
include/linux/mm_inline.h +1 -1
 	else if ((!folio_is_file_lru(folio) && !folio_test_swapcache(folio)) ||
@@ -247,7 +248,7 @@ static inline unsigned long lru_gen_folio_seq(const
 struct lruvec *lruvec,
 		  (folio_test_dirty(folio) || folio_test_writeback(folio))))
 		gen = MIN_NR_GENS;
 	else
-		gen = MAX_NR_GENS - (folio_test_workingset(folio) ||
 folio_test_referenced(folio));
+		gen = MAX_NR_GENS - folio_test_workingset(folio);
 

return max(READ_ONCE(lrugen->max_seq) - gen + 1,
READ_ONCE(lrugen->min_seq[type]));
}

mm/folio.c +3 -13
index c02dcea9c03c..2fd835b3b50c 100644
@@ -470,20 +470,10 @@ void folio_add_lru(struct folio *folio)
 			folio_test_unevictable(folio), folio);
 	VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
 
-	/*
-	 * For refaulted workingset folios, set PG_active so they
-	 * can be added to active generations.
-	 * For prefaulted file folios, folio_mark_accessed() sets
-	 * PG_referenced so lru_gen_folio_seq() places them into
-	 * the second oldest generation.
-	 */
+	/* see the comment in lru_gen_folio_seq() */
 	if (lru_gen_enabled() && !folio_test_unevictable(folio) &&
-	    lru_gen_in_fault() && !(current->flags & PF_MEMALLOC)) {
-		if (folio_test_workingset(folio))
-			folio_set_active(folio);
-		else if (!folio_test_referenced(folio))
-			folio_mark_accessed(folio);
-	}
+	    lru_gen_in_fault() && !(current->flags & PF_MEMALLOC))
+		folio_set_active(folio);
 
 	folio_batch_add_and_move(folio, lru_add);  } diff --git

a/mm/vmscan.c b/mm/vmscan.c index f11491ee9ed5..aa500ae9371a 100644

mm/vmscan.c +1 -5
@@ -851,11 +851,7 @@ static bool lru_gen_set_refs(struct folio *folio, const
 vma_flags_t *vma_flags)
 		return false;
 	}
 
-	/* Promote on second access */
-	if (folio_lru_refs(folio) > 1)
-		set_mask_bits(&folio->flags.f, LRU_REFS_FLAGS,
 BIT(PG_workingset));
-	else
-		folio_mark_accessed(folio);
+	set_mask_bits(&folio->flags.f, LRU_REFS_FLAGS, BIT(PG_workingset));
 	return true;

}
#else

mm/workingset.c index renamed from mm/workingset.c +0 -0

7ac2b88c80ae..a79b73ee9762 100644

mm/workingset.c +4 -6
@@ -319,13 +319,11 @@ static void lru_gen_refault(struct folio *folio, void
 *shadow)
 
 	atomic_long_add(delta, &lrugen->refaulted[hist][type][tier]);
 
+	/* see folio_add_lru() where folio_set_active() will be called */
+	if (lru_gen_in_fault())
+		mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE +
 type, delta);
+
 	if (workingset) {
-		/*
-		 * see folio_add_lru(), where folio_set_active() is
-		 * called for workingset folios
-		 */
-		if (lru_gen_in_fault())
-			mod_lruvec_state(lruvec,
 WORKINGSET_ACTIVATE_BASE + type, delta);
mm/workingset.c +0 -0
 		folio_set_workingset(folio);
 		mod_lruvec_state(lruvec, WORKINGSET_RESTORE_BASE +

type, delta);
} else
--
2.34.1

reviewer
#4 lore raw
Quoted 95 lines · Ehab

Hi Barry, Kairui,

Thanks for the feedback and suggestions.

Quoted 83 lines

-----Original Message-----
From: Barry Song (Xiaomi) <[email protected]>
Sent: Tuesday, September 1, 2026 3:05 PM
To: [email protected]; Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 13 lines · Kairui Song
Quoted 11 lines · Ehab Ababneh

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Quoted 3 lines
Quoted 3 lines

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from it. The goal is to keep the optimization without keeping readahead pages alive

forever.

Quoted 17 lines
Quoted 17 lines

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually helping.

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the
original behavior stand
- decay/reset the credit as generations advance and when an lruvec is
initialized

Quoted 4 lines
Quoted 4 lines

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages

around unnecessarily.

Quoted 13 lines
Quoted 12 lines

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to
replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

[...]

Quoted 4 lines

Just an idea. For an short term and easy fix, what if we simply revert than, then only protect in_fault && folio_test_swapbacked folios with PG_active?

Hi Ehab and Kairui,

Thanks very much for your reports and discussion.

I'm not quite sure we want to revert it entirely, as that would immediately regress the workloads improved by the commit, such as the kernel build. Also, for example, Kairui's cover letter mentioned the LevelDB benchmark:
"I also retested the LevelDB benchmark from the cache_ext paper [5].
Interestingly, mainline MGLRU already beats CLRU on this one after a recent change in lru_gen_folio_seq that bumps new folios with refs == 1 to the second-oldest generation."

Quoted 6 lines

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case actually different from Kairui's? (The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further gains.)

Quoted 9 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Thanks very much, Ehab. Could you please also run my proposed quick fix:

https://lore.kernel.org/linux-mm/[email protected]/

Best Regards
Barry

#5 lore raw
Quoted 127 lines

-----Original Message-----
From: Barry Song <[email protected]>
Sent: Wednesday, September 2, 2026 2:58 PM
To: Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 85 lines · Ehab

Hi Barry, Kairui,

Thanks for the feedback and suggestions.

Quoted 80 lines

-----Original Message-----
From: Barry Song (Xiaomi) <[email protected]>
Sent: Tuesday, September 1, 2026 3:05 PM
To: [email protected]; Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 14 lines · Kairui Song
Quoted 12 lines · Ehab Ababneh

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace
folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Quoted 4 lines
Quoted 4 lines

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from it. The goal is to keep the optimization without keeping readahead pages alive

forever.

Quoted 17 lines
Quoted 17 lines

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually helping.

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the
original behavior stand
- decay/reset the credit as generations advance and when an lruvec is
initialized

Quoted 5 lines
Quoted 5 lines

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages

around unnecessarily.

Quoted 13 lines
Quoted 12 lines

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to
replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

[...]

Quoted 4 lines

Just an idea. For an short term and easy fix, what if we simply revert than, then only protect in_fault && folio_test_swapbacked folios with PG_active?

Hi Ehab and Kairui,

Thanks very much for your reports and discussion.

I'm not quite sure we want to revert it entirely, as that would immediately regress the workloads improved by the commit, such as the

kernel build.

Quoted 9 lines
Quoted 9 lines

Also, for example, Kairui's cover letter mentioned the LevelDB benchmark:
"I also retested the LevelDB benchmark from the cache_ext paper [5].
Interestingly, mainline MGLRU already beats CLRU on this one after a recent change in lru_gen_folio_seq that bumps new folios with refs == 1 to the second-oldest generation."

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case

actually different from Kairui's?

Quoted 5 lines
Quoted 4 lines

(The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further gains.)

Quoted 9 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Quoted 5 lines

Thanks very much, Ehab. Could you please also run my proposed quick fix:

https://lore.kernel.org/linux-mm/20260901220430.79810-1-
[email protected]/

Sure, I will do. I will report back when I have the results.

Thanks,
Ehab

Quoted 2 lines

Best Regards
Barry

#6 lore raw

Hi Kairui, Barry,

Apologies for the delay in getting back to you with these results — my test environment got corrupted and I had to spend some time recovering it before I could re-run the benchmark.

Quoted 114 lines

-----Original Message-----
From: Ababneh, Ehab
Sent: Wednesday, September 2, 2026 3:02 PM
To: 'Barry Song' <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 12 lines

-----Original Message-----
From: Barry Song <[email protected]>
Sent: Wednesday, September 2, 2026 2:58 PM
To: Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 1 line

On Thu, Sep 3, 2026 at 5:52 AM Ababneh, Ehab

<[email protected]>

Quoted 19 lines
Quoted 18 lines · wrote:

Hi Barry, Kairui,

Thanks for the feedback and suggestions.

Quoted 13 lines

-----Original Message-----
From: Barry Song (Xiaomi) <[email protected]>
Sent: Tuesday, September 1, 2026 3:05 PM
To: [email protected]; Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

On Wed, Sep 2, 2026 at 2:21 AM Kairui Song <[email protected]>

Quoted 14 lines · wrote:
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines · Ehab Ababneh
Quoted 12 lines

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace
folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from

it.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

The goal is to keep the optimization without keeping readahead pages alive

forever.

Quoted 4 lines
Quoted 4 lines

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually

helping.

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the
original behavior stand
- decay/reset the credit as generations advance and when an lruvec is
initialized

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

around unnecessarily.

Quoted 5 lines
Quoted 5 lines

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed

to

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
Quoted 7 lines

replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to
replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

[...]

Quoted 4 lines

Just an idea. For an short term and easy fix, what if we simply revert than, then only protect in_fault && folio_test_swapbacked folios with PG_active?

Hi Ehab and Kairui,

Quoted 33 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

Thanks very much for your reports and discussion.

I'm not quite sure we want to revert it entirely, as that would immediately regress the workloads improved by the commit, such as the

kernel build.

Quoted 1 line
Quoted 1 line

Also, for example, Kairui's cover letter mentioned the LevelDB

benchmark:

Quoted 25 lines
Quoted 9 lines
Quoted 9 lines

"I also retested the LevelDB benchmark from the cache_ext paper [5].
Interestingly, mainline MGLRU already beats CLRU on this one after a recent change in lru_gen_folio_seq that bumps new folios with refs == 1 to the second-oldest generation."

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case

actually different from Kairui's?

Quoted 15 lines
Quoted 5 lines

(The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further
gains.)

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Thanks for the pointer — I gave your MGLRU-FG fix a try against the same Cassandra read benchmark I used for my patch (4 nodes, 720s, 100 readers).

Results with MGLRU-FG:
Op rate: 55.0k - 56.5k op/s
Latency 99th percentile: 6.8 - 6.9 ms

For reference, here's where the other variants landed on the same setup:

Regression (6cbdd9726fb5, "mm/mglru: use folio_mark_accessed to replace folio_set_active"):

      p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s

Revert of 6cbdd9726fb5:

      p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s

My dynamic readahead-credit fix:

      p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

So MGLRU-FG recovers most of the latency regression, but the revert and my fix still recover more of it: p99 with MGLRU-FG is noticeably higher than with the revert and my fix (6.8-6.9 ms vs ~5.5-5.8 ms), though it's still a big improvement over the regression's 9.2-9.5 ms.

Happy to dig further if useful — let me know if you'd like more detail on the latency distribution or a longer run to confirm these numbers are stable.

Thanks,
Ehab

Quoted 14 lines
Quoted 6 lines

Thanks very much, Ehab. Could you please also run my proposed quick fix:

https://lore.kernel.org/linux-mm/20260901220430.79810-1-
[email protected]/

Sure, I will do. I will report back when I have the results.

Thanks,
Ehab

Quoted 2 lines

Best Regards
Barry

reviewer
#7 lore raw
Quoted 6 lines · Ehab

Hi Kairui, Barry,

Apologies for the delay in getting back to you with these results — my test environment got corrupted and I had to spend some time recovering it before I could re-run the benchmark.

No worries, Ehab. And thanks very much for your testing!

[...]

Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Quoted 5 lines

Thanks for the pointer — I gave your MGLRU-FG fix a try against the same Cassandra read benchmark I used for my patch (4 nodes, 720s, 100 readers).

Results with MGLRU-FG:
Op rate: 55.0k - 56.5k op/s

I noticed this is even higher than your dynamic
readahead fix: "throughput ~51.9k-52.7k op/s"

Is this a run-to-run variation, or is the improvement stable across runs?

Quoted 11 lines

Latency 99th percentile: 6.8 - 6.9 ms

For reference, here's where the other variants landed on the same setup:

Regression (6cbdd9726fb5, "mm/mglru: use folio_mark_accessed to replace folio_set_active"):

      p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s

Revert of 6cbdd9726fb5:

      p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
Quoted 7 lines

My dynamic readahead-credit fix:

      p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

So MGLRU-FG recovers most of the latency regression, but the revert and my fix still recover more of it: p99 with MGLRU-FG is noticeably higher than with the revert and my fix (6.8-6.9 ms vs ~5.5-5.8 ms), though it's still a big improvement over the regression's 9.2-9.5 ms.

We also have this :
https://lore.kernel.org/linux-mm/[email protected]/

you once reported it as :
"throughput was approximately
49.4K to 50.3K ops/s, averaging about 50.0K ops/s, while the 99th-percentile latency remained around 7.0 to 7.1 ms"

Quoted 5 lines

Happy to dig further if useful — let me know if you'd like more detail on the latency distribution or a longer run to confirm these numbers are stable.

Thanks
Barry

#8 lore raw

Hi Barry,

Quoted 39 lines

-----Original Message-----
From: Barry Song <[email protected]>
Sent: Wednesday, September 23, 2026 9:08 PM
To: Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 6 lines · Ehab

Hi Kairui, Barry,

Apologies for the delay in getting back to you with these results — my test environment got corrupted and I had to spend some time recovering it before I could re-run the benchmark.

No worries, Ehab. And thanks very much for your testing!

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Thanks for the pointer — I gave your MGLRU-FG fix a try against the same Cassandra read benchmark I used for my patch (4 nodes, 720s, 100

readers).

Quoted 1 line
Quoted 8 lines
Quoted 2 lines

Results with MGLRU-FG:
Op rate: 55.0k - 56.5k op/s

I noticed this is even higher than your dynamic readahead fix: "throughput ~51.9k-52.7k op/s"

Is this a run-to-run variation, or is the improvement stable across runs?

Yes, I can't explain the slight gain in throughput but loss in latency compared to my fix. I'll probably study this in more detail when I can dedicate some time to it.

Typically, I get a unique result immediately after a reboot, but subsequent runs are very consistent. The variation is usually around 0.1 ms gain/loss in latency and approximately 100-200 requests per second in throughput. I discard the results from the first run.

Quoted 19 lines
Quoted 11 lines

Latency 99th percentile: 6.8 - 6.9 ms

For reference, here's where the other variants landed on the same setup:

Regression (6cbdd9726fb5, "mm/mglru: use folio_mark_accessed to replace folio_set_active"):

      p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s

Revert of 6cbdd9726fb5:

      p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
Quoted 7 lines

My dynamic readahead-credit fix:

      p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

So MGLRU-FG recovers most of the latency regression, but the revert and my fix still recover more of it: p99 with MGLRU-FG is noticeably higher than with the revert and my fix (6.8-6.9 ms vs ~5.5-5.8 ms), though it's still a big improvement over the regression's 9.2-9.5 ms.

Quoted 9 lines

We also have this :
https://lore.kernel.org/linux-mm/20260901220430.79810-1-
[email protected]/

you once reported it as :
"throughput was approximately
49.4K to 50.3K ops/s, averaging about 50.0K ops/s, while the 99th-percentile latency remained around 7.0 to 7.1 ms"

Yes, this is from before the data was corrupted. I get slightly different results with my fix and with the revert, but the overall pattern remains consistent.

Thanks,
Ehab

Quoted 8 lines
Quoted 5 lines

Happy to dig further if useful — let me know if you'd like more detail on the latency distribution or a longer run to confirm these numbers are stable.

Thanks
Barry

#9 lore raw
Quoted 49 lines · Ababneh, Ehab <[email protected]> 于 2026年9月25日周五 01:54写道:

Hi Barry,

Quoted 46 lines

-----Original Message-----
From: Barry Song <[email protected]>
Sent: Wednesday, September 23, 2026 9:08 PM
To: Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 6 lines · Ehab

Hi Kairui, Barry,

Apologies for the delay in getting back to you with these results — my test environment got corrupted and I had to spend some time recovering it before I could re-run the benchmark.

No worries, Ehab. And thanks very much for your testing!

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Thanks for the pointer — I gave your MGLRU-FG fix a try against the same Cassandra read benchmark I used for my patch (4 nodes, 720s, 100

readers).

Quoted 3 lines

Results with MGLRU-FG:
Op rate: 55.0k - 56.5k op/s

I noticed this is even higher than your dynamic readahead fix: "throughput ~51.9k-52.7k op/s"

Quoted 6 lines
Quoted 2 lines

Is this a run-to-run variation, or is the improvement stable across runs?

Yes, I can't explain the slight gain in throughput but loss in latency compared to my fix. I'll probably study this in more detail when I can dedicate some time to it.

Hi Ehab,

That's very interesting, I think it means we are getting a bit better at identifying the working set. But maybe a slightly high cost of aging? And P99 is usually very sensitive to noises, one long tailing aging event could make it look a lot worse.

Quoted 4 lines

Typically, I get a unique result immediately after a reboot, but subsequent runs are very consistent. The variation is usually around 0.1 ms gain/loss in latency and approximately 100-200 requests per second in throughput. I discard the results from the first run.

Any way I can reproduce this? E.g. what is the test bench, database size, benchmark, machine spec, etc?

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Latency 99th percentile: 6.8 - 6.9 ms

For reference, here's where the other variants landed on the same setup:

Regression (6cbdd9726fb5, "mm/mglru: use folio_mark_accessed to replace folio_set_active"):

      p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s

Revert of 6cbdd9726fb5:

      p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

My dynamic readahead-credit fix:

      p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

So MGLRU-FG recovers most of the latency regression, but the revert and my fix still recover more of it: p99 with MGLRU-FG is noticeably higher than with the revert and my fix (6.8-6.9 ms vs ~5.5-5.8 ms), though it's still a big improvement over the regression's 9.2-9.5 ms

Did you test the V2 or V1 of MGLRU-FG? V2 includes Barry's aging optimization; I'm not sure if this latency is caused by aging or maybe some other change.

And is there any swap device you used, or is the test using the same base commit? MM also changes significantly between different baselines.

#10 lore raw

Hi Kairui,

Quoted 109 lines

-----Original Message-----
From: Kairui Song <[email protected]>
Sent: Saturday, September 26, 2026 12:06 AM
To: Ababneh, Ehab <[email protected]>
Cc: Barry Song <[email protected]>; [email protected]; Axel Rasmussen <[email protected]>; Lance Yang <[email protected]>; LKML <[email protected]>; linux-mm <[email protected]>; Qi Zheng <[email protected]>; Shakeel Butt <[email protected]>; Wei Xu <[email protected]>; Yuanchu Xie <[email protected]>; Yu Zhao <[email protected]>
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Ababneh, Ehab <[email protected]> 于 2026年9月25日周五

Quoted 9 lines · 01:54写道:

Hi Barry,

Quoted 6 lines

-----Original Message-----
From: Barry Song <[email protected]>
Sent: Wednesday, September 23, 2026 9:08 PM
To: Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected];
[email protected]; [email protected];

[email protected];

Quoted 15 lines
Quoted 15 lines

[email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 6 lines · Ehab

Hi Kairui, Barry,

Apologies for the delay in getting back to you with these results — my test environment got corrupted and I had to spend some time recovering it before I could re-run the benchmark.

Quoted 8 lines
Quoted 8 lines

No worries, Ehab. And thanks very much for your testing!

[...]

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those

workloads.

Quoted 18 lines
Quoted 18 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Thanks for the pointer — I gave your MGLRU-FG fix a try against the same Cassandra read benchmark I used for my patch (4 nodes, 720s, 100

readers).

Quoted 3 lines

Results with MGLRU-FG:
Op rate: 55.0k - 56.5k op/s

I noticed this is even higher than your dynamic readahead fix:
"throughput ~51.9k-52.7k op/s"

Quoted 6 lines
Quoted 2 lines

Is this a run-to-run variation, or is the improvement stable across runs?

Yes, I can't explain the slight gain in throughput but loss in latency compared to my fix. I'll probably study this in more detail when I can dedicate some time to it.

Hi Ehab,

That's very interesting, I think it means we are getting a bit better at identifying the working set. But maybe a slightly high cost of aging? And P99 is usually very sensitive to noises, one long tailing aging event could make it look a lot worse.

Quoted 4 lines

Typically, I get a unique result immediately after a reboot, but subsequent runs are very consistent. The variation is usually around 0.1 ms gain/loss in latency and approximately 100-200 requests per second in throughput. I discard the results from the first run.

Any way I can reproduce this? E.g. what is the test bench, database size, benchmark, machine spec, etc?

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Latency 99th percentile: 6.8 - 6.9 ms

For reference, here's where the other variants landed on the same setup:

Regression (6cbdd9726fb5, "mm/mglru: use folio_mark_accessed to replace folio_set_active"):

      p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s

Revert of 6cbdd9726fb5:

      p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

My dynamic readahead-credit fix:

      p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

So MGLRU-FG recovers most of the latency regression, but the revert and my fix still recover more of it: p99 with MGLRU-FG is noticeably higher than with the revert and my fix (6.8-6.9 ms vs ~5.5-5.8 ms), though it's still a big improvement over the regression's 9.2-9.5 ms

Quoted 6 lines

Did you test the V2 or V1 of MGLRU-FG? V2 includes Barry's aging optimization; I'm not sure if this latency is caused by aging or maybe some other change.

And is there any swap device you used, or is the test using the same base commit? MM also changes significantly between different baselines.

My setup runs Cassandra 4.1.5 on Java 11. The host has two Intel Xeon Platinum 8468 sockets, 96 physical CPU cores (192 logical CPUs), two NUMA nodes, and 440 GiB of system-visible RAM. Cassandra instances use CPU affinity, with memory interleaved across NUMA nodes.

I run four Cassandra instances, each backed by a dedicated 3.5 TB NVMe drive. Each drive has a separate mount, and holds its instance’s data files, commit log, and saved caches.

The stress workload uses a pre-populated table. Its schema, partitioning, replication, compaction, and compression settings, along with the stress profile and concurrency, are recorded because they can affect performance.

I used more than one kernel version across my test runs. The most recent was Linux 7.3.0-rc5. Within each set of setups being compared, I kept the kernel version consistent.

For the MGLRU-FG tests, I used the ryncsn/b4/mglru-fg-v1.8 branch. The MGLRU-FG implementation is commit 6bcf4ae70b5e; the branch tip is a619ab0105d9.

#11 lore raw
Quoted 2 lines · Ehab

Hi Kairui,

...

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Results with MGLRU-FG:
Op rate: 55.0k - 56.5k op/s

...

Quoted 40 lines
Quoted 20 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Latency 99th percentile: 6.8 - 6.9 ms

For reference, here's where the other variants landed on the same setup:

Regression (6cbdd9726fb5, "mm/mglru: use folio_mark_accessed to replace folio_set_active"):

      p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s

Revert of 6cbdd9726fb5:

      p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

My dynamic readahead-credit fix:

      p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

So MGLRU-FG recovers most of the latency regression, but the revert and my fix still recover more of it: p99 with MGLRU-FG is noticeably higher than with the revert and my fix (6.8-6.9 ms vs ~5.5-5.8 ms), though it's still a big improvement over the regression's 9.2-9.5 ms

Quoted 6 lines

Did you test the V2 or V1 of MGLRU-FG? V2 includes Barry's aging optimization; I'm not sure if this latency is caused by aging or maybe some other change.

And is there any swap device you used, or is the test using the same base commit? MM also changes significantly between different baselines.

My setup runs Cassandra 4.1.5 on Java 11. The host has two Intel Xeon Platinum 8468 sockets, 96 physical CPU cores (192 logical CPUs), two NUMA nodes, and 440 GiB of system-visible RAM. Cassandra instances use CPU affinity, with memory interleaved across NUMA nodes.

I run four Cassandra instances, each backed by a dedicated 3.5 TB NVMe drive. Each drive has a separate mount, and holds its instance’s data files, commit log, and saved caches.

The stress workload uses a pre-populated table. Its schema, partitioning, replication, compaction, and compression settings, along with the stress profile and concurrency, are recorded because they can affect performance.

Quoted 7 lines

I used more than one kernel version across my test runs. The most recent was Linux 7.3.0-rc5. Within each set of setups being compared, I kept the kernel version consistent.

For the MGLRU-FG tests, I used the ryncsn/b4/mglru-fg-v1.8 branch. The MGLRU-FG implementation is commit 6bcf4ae70b5e; the branch tip is a619ab0105d9.

Thanks for the info! That v1.8 is missing recent upstream changes indeed, so aging could be a bit slower, causing the jitter. I also found a few other minor improvements doable, I'll also try to run some Cassandra benchmarks too. Higher throughput and worse P99 is an interesting effect, I think it means the LRU is doing better at protecting the working set and maybe spending a bit more effort on reclaim / aging, which could be optimized away. I will try to reproduce and optimize for that as well while keep the series updated.

#12 lore raw
In reply to Barry Song
Quoted 125 lines

-----Original Message-----
From: Barry Song <[email protected]>
Sent: Wednesday, September 2, 2026 2:58 PM
To: Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 85 lines · Ehab

Hi Barry, Kairui,

Thanks for the feedback and suggestions.

Quoted 80 lines

-----Original Message-----
From: Barry Song (Xiaomi) <[email protected]>
Sent: Tuesday, September 1, 2026 3:05 PM
To: [email protected]; Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

Quoted 14 lines · Kairui Song
Quoted 12 lines · Ehab Ababneh

Commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace
folio_set_active") introduced a regression for workloads that rely on readahead to keep sequential file access efficient.

The problem is that MGLRU can place fault-path file folios in older generations, so memory pressure can reclaim readahead folios before the workload touches them. In our Cassandra read benchmark, this raised p99 latency to about 9.2-9.5 ms and cut throughput to roughly 41.8k-43.6k op/s; the revert restored the workload to about 5.5-5.6 ms and 51.9k-53.1k op/s.

Quoted 4 lines
Quoted 4 lines

Readahead is important for sequential I/O and mmap scans, but it should not be retained when the workload does not benefit from it. The goal is to keep the optimization without keeping readahead pages alive

forever.

Quoted 17 lines
Quoted 17 lines

This patch provides a middle ground: keep the original behavior by default, but temporarily protect fault-path file folios when repeated file refaults show that readahead is actually helping.

The mechanism is dynamic and self-tuning:

- add a per-lruvec readahead/refault credit
- accumulate credit on file refaults in the MGLRU refault path
- consume credit in folio_add_lru() for fault-path file folios
- keep the folio active while credit is available, and otherwise let the
original behavior stand
- decay/reset the credit as generations advance and when an lruvec is
initialized

Quoted 5 lines
Quoted 5 lines

This means we only protect fault-path file folios when refault pressure shows that the workload is actively benefiting from readahead. If the workload does not need that protection, the original optimization remains intact and we do not keep readahead pages

around unnecessarily.

Quoted 13 lines
Quoted 12 lines

Benchmark results for the Cassandra read workload
(4 nodes, 720s, 100 readers):

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to
replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

[...]

Quoted 4 lines

Just an idea. For an short term and easy fix, what if we simply revert than, then only protect in_fault && folio_test_swapbacked folios with PG_active?

Hi Ehab and Kairui,

Thanks very much for your reports and discussion.

I'm not quite sure we want to revert it entirely, as that would immediately regress the workloads improved by the commit, such as the

kernel build.

Quoted 9 lines
Quoted 9 lines

Also, for example, Kairui's cover letter mentioned the LevelDB benchmark:
"I also retested the LevelDB benchmark from the cache_ext paper [5].
Interestingly, mainline MGLRU already beats CLRU on this one after a recent change in lru_gen_folio_seq that bumps new folios with refs == 1 to the second-oldest generation."

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case

actually different from Kairui's?

Quoted 5 lines
Quoted 4 lines

(The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further gains.)

Quoted 9 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Quoted 5 lines

Thanks very much, Ehab. Could you please also run my proposed quick fix:

https://lore.kernel.org/linux-mm/20260901220430.79810-1-
[email protected]/

Hi Barry,

Thanks for sending the suggested fix. I tested it with the Cassandra benchmark, and the results look promising. Across the runs, throughput was approximately 49.4K to 50.3K ops/s, averaging about 50.0K ops/s, while the 99th-percentile latency remained around 7.0 to 7.1 ms. This represents an improvement over the baseline of approximately 43.98K ops/s and 9.025 ms p99 latency.

The underlying idea in both solutions is very similar: A credit-based system protects file folios when readahead proves useful. In my version, I further tuned credit accounting and used per-CPU batching to reduce atomic contention on multicore systems.

Thanks,
Ehab

Quoted 2 lines

Best Regards
Barry

reviewer
#13 lore raw

On Fri, Sep 4, 2026 at 3:46 AM Ababneh, Ehab <[email protected]> wrote:
[...]

Quoted 25 lines
Quoted 19 lines
Quoted 3 lines
Quoted 3 lines

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case

actually different from Kairui's?

Quoted 5 lines
Quoted 4 lines

(The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further gains.)

Quoted 9 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Quoted 5 lines

Thanks very much, Ehab. Could you please also run my proposed quick fix:

https://lore.kernel.org/linux-mm/20260901220430.79810-1-
[email protected]/

Quoted 8 lines

Hi Barry,

Thanks for sending the suggested fix. I tested it with the Cassandra benchmark, and the results look promising. Across the runs, throughput was approximately 49.4K to 50.3K ops/s, averaging about 50.0K ops/s, while the 99th-percentile latency remained around 7.0 to 7.1 ms. This represents an improvement over the baseline of approximately 43.98K ops/s and 9.025 ms p99 latency.

I am happy to see it is much better, but it still doesn't seem as good as it was at the very beginning?

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

Quoted 5 lines

The underlying idea in both solutions is very similar: A credit-based system protects file folios when readahead proves useful. In my version, I further tuned credit accounting and used per-CPU batching to reduce atomic contention on multicore systems.

Your benchmark seems quite likely to access readahead folios. Let me dig into your patch a bit deeper and see if we can find a combination of the two approaches.
BTW, does Kairui's MGLRU-FG bring your benchmark back to the performance you had at the very beginning?

Best Regards
Barry

#14 lore raw
Quoted 51 lines

-----Original Message-----
From: Barry Song <[email protected]>
Sent: Thursday, September 3, 2026 2:38 PM
To: Ababneh, Ehab <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH] mm/mglru: dynamically protect readahead fault folios under refault pressure

On Fri, Sep 4, 2026 at 3:46 AM Ababneh, Ehab <[email protected]> wrote:
[...]

Quoted 27 lines
Quoted 21 lines
Quoted 4 lines
Quoted 4 lines

I guess we could instead try to mitigate the cases where some workloads are negatively affected while preserving the original intention. Does the fix below address both of your cases, or is Ehab's case

actually different from Kairui's?

Quoted 6 lines
Quoted 5 lines

(The kernel-build test on my machine looks quite positive. It not only preserves the original optimization, but even provides further
gains.)

Quoted 9 lines

I agree that reverting the commit that caused the regression is not the optimal path. I expect there are many workloads and scenarios that benefit from the behavior introduced by that commit, so reverting it could unnecessarily regress those workloads.

I will run the Cassandra benchmark with Kairui's MGLRU-FG patches to see whether they address the issue I am seeing. I will send the results when they are ready.

Quoted 5 lines

Thanks very much, Ehab. Could you please also run my proposed quick fix:

https://lore.kernel.org/linux-mm/20260901220430.79810-1-
[email protected]/

Quoted 7 lines

Hi Barry,

Thanks for sending the suggested fix. I tested it with the Cassandra benchmark, and the results look promising. Across the runs, throughput was approximately 49.4K to 50.3K ops/s, averaging about 50.0K ops/s, while the 99th-percentile latency remained around 7.0 to 7.1 ms. This represents an improvement over the baseline of approximately 43.98K

Quoted 6 lines

ops/s and 9.025 ms p99 latency.

Quoted 1 line

I am happy to see it is much better, but it still doesn't seem as good as it was at the very beginning?

Hi Barry, All,

It is not as performant as its best, but it recovers most of the regression. During the regression, latency had spiked to around 9 ms. After applying the patch, it has come back down to 5.8 ms. Overall, this is still better than where it was about two months ago, when I was typically seeing 6.2 to 6.3 ms.

Quoted 15 lines

- with commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to replace folio_set_active"):
p99 ~9.2-9.5 ms, throughput ~41.8k-43.6k op/s
- with revert of commit 6cbdd9726fb5 ("mm/mglru: use folio_mark_accessed to
replace folio_set_active"):
p99 ~5.5-5.6 ms, throughput ~51.9k-53.1k op/s
- with this fix: p99 ~5.8 ms, throughput ~51.9k-52.7k op/s

Quoted 5 lines

The underlying idea in both solutions is very similar: A credit-based system protects file folios when readahead proves useful. In my version, I further tuned credit accounting and used per-CPU batching to reduce atomic contention on multicore systems.

Quoted 3 lines

Your benchmark seems quite likely to access readahead folios.
Let me dig into your patch a bit deeper and see if we can find a combination of the two approaches.

Yes, it does seem to access readahead folios. I also think this benchmark exercises an access pattern that is generic enough that similar behavior could show up in a wide range of real-world applications, rather than being something specific to this particular workload.

Quoted 3 lines

BTW, does Kairui's MGLRU-FG bring your benchmark back to the performance you had at the very beginning?

Not yet, but I'll be trying Kairui's MGLRU-FG today or tomorrow and will report Back once I have some results.

Thanks,
Ehab

Quoted 2 lines

Best Regards
Barry