mm: enable lru cache for smaller large folios
Subsystems (4)
Barry Song (Xiaomi)
reviewerv2 cover mm: enable lru cache for smaller large folios
This patchset enables the per-CPU LRU cache for large folios smaller than COSTLY_ORDER. It is particularly beneficial on systems that use relatively small large folios. For larger folios, the benefit is likely to be smaller because far fewer folios are expected to contend for the LRU cache.
Patch 1/2 enables the per-CPU LRU cache for eligible large folios. Patch 2/2 updates __wp_can_reuse_large_anon_folio() to account for the fact that large folios may now also reside in the per-CPU LRU cache before they are reused.
-RFC v2:
* Make __wp_can_reuse_large_anon_folio() aware of LRU-cached large folios. As David pointed out, it currently does not account for large folios residing in the per-CPU LRU cache.
Barry Song (Xiaomi) (2):
mm: allow smaller large folios to use lru_cache
mm: improve large folio reuse for LRU-cached folios
include/linux/swap.h | 4 ++-- mm/memory.c | 13 +++++++++++++ 2 files changed, 15 insertions(+), 2 deletions(-)
Signature · 1 line
2.39.3 (Apple Git-146)
Barry Song (Xiaomi)
reviewerRFC PATCH v2 1/2 mm: allow smaller large folios to use lru_cache
For a system which primarily uses smaller orders for large folios, it could be beneficial to enable lru_cache to avoid lock contention.
For large folios with higher orders, the number of folios involved would be small, so the contention is less significant.
The following small program runs on a system with 16 KiB mTHP enabled.
#include <pthread.h>
#include <sys/mman.h>
#include <string.h>
#define NUM_THREADS 20 #define MEM_SIZE (16 * 1024 * 1024) #define LOOP_COUNT 1000
void* thread_worker(void* arg) {
void *addr = mmap(NULL, MEM_SIZE, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
for (int i = 0; i < LOOP_COUNT; i++) {
memset(addr, 0x55, MEM_SIZE);
madvise(addr, MEM_SIZE, MADV_DONTNEED);
}
munmap(addr, MEM_SIZE);
pthread_exit(NULL);}
int main() {
pthread_t threads[NUM_THREADS];
for (long t = 0; t < NUM_THREADS; t++) {
pthread_create(&threads[t], NULL, thread_worker, (void*)t);
}
for (int t = 0; t < NUM_THREADS; t++) {
pthread_join(threads[t], NULL);
}
return 0;}
Before patch:
root@barry-desktop:/home/barry# time ./a.out
real 0m20.233s
user 0m21.739s
sys 6m21.143s
Perf lock report:
Name acquired contended avg wait total wait max wait min wait
21158662 21158662 29.20 us 10.30 m 67.83 us 884 ns
13547 13547 14.68 us 198.93 ms 135.39 us 982 ns
rcu_state 41 41 1.83 us 75.21 us 3.46 us 1.27 us
rcu_state 34 34 1.83 us 62.09 us 2.60 us 1.16 us
5 5 21.31 us 106.57 us 25.77 us 18.17 us
5 5 3.01 us 15.07 us 5.19 us 1.53 us
3 3 6.42 us 19.26 us 9.63 us 4.22 us
3 3 4.20 us 12.61 us 7.45 us 2.10 us
2 2 5.03 us 10.06 us 7.88 us 2.18 us
2 2 2.79 us 5.59 us 3.34 us 2.25 us
1 1 4.70 us 4.70 us 4.70 us 4.70 us
1 1 9.77 us 9.77 us 9.77 us 9.77 usAfter patch:
root@barry-desktop:/home/barry# time ./a.out
real 0m17.796s
user 0m24.962s
sys 5m28.774s
Perf lock report:
Name acquired contended avg wait total wait max wait min wait
1407013 1407013 25.44 us 35.80 s 127.83 us 972 ns
845113 845113 47.96 us 40.54 s 152.86 us 1.21 us
rcu_state 1920 1920 5.87 us 11.27 ms 25.02 us 1.76 us
rcu_state 1889 1889 5.80 us 10.96 ms 30.35 us 1.61 us
140 140 3.97 us 555.52 us 12.51 us 1.20 us
10 10 61.34 us 613.42 us 96.90 us 44.48 us
8 8 24.03 us 192.26 us 37.64 us 6.95 us
8 8 12.32 us 98.56 us 25.55 us 4.16 usBoth system time and lock contention are noticeably reduced.
--- include/linux/swap.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/swap.h +2 -2
@@ -316,9 +316,9 @@ static inline bool folio_may_be_lru_cached(struct folio *folio)/* * Holding PMD-sized folios in per-CPU LRU cache unbalances accounting. * Holding small numbers of low-order mTHP folios in per-CPU LRU cache -* will be sensible, but nobody has implemented and tested that yet. +* will be sensible. */ -return !folio_test_large (folio); +return folio_order (folio) <PAGE_ALLOC_COSTLY_ORDER ; }extern atomic_t lru_disable_count;
Signature · 1 line
2.39.3 (Apple Git-146)
David Hildenbrand (Arm)
maintainerRe: [RFC PATCH v2 1/2] mm: allow smaller large folios to use lru_cache
Quoted 99 lines · Barry Song (Xiaomi)
For a system which primarily uses smaller orders for large folios, it could be beneficial to enable
lru_cacheto avoid lock contention.For large folios with higher orders, the number of folios involved would be small, so the contention is less significant.
The following small program runs on a system with 16 KiB mTHP enabled.
#include <
pthread.h>
#include <sys/mman.h>
#include <string.h>#define NUM_THREADS 20 #define MEM_SIZE (16 * 1024 * 1024) #define LOOP_COUNT 1000void* thread_worker(void* arg) { void *addr = mmap(NULL, MEM_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); for (int i = 0; i < LOOP_COUNT; i++) { memset(addr, 0x55, MEM_SIZE); madvise(addr, MEM_SIZE, MADV_DONTNEED); } munmap(addr, MEM_SIZE); pthread_exit(NULL);}
int main() { pthread_t threads[NUM_THREADS]; for (long t = 0; t < NUM_THREADS; t++) { pthread_create(&threads[t], NULL, thread_worker, (void*)t); } for (int t = 0; t < NUM_THREADS; t++) { pthread_join(threads[t], NULL); } return 0;}
Before patch:
root@barry-desktop:/home/barry# time ./a.outreal 0m20.233s
user 0m21.739s
sys 6m21.143sPerf lock report:
Name acquired contended avg wait total wait max wait min wait 21158662 21158662 29.20 us 10.30 m 67.83 us 884 ns 13547 13547 14.68 us 198.93 ms 135.39 us 982 ns rcu_state 41 41 1.83 us 75.21 us 3.46 us 1.27 us rcu_state 34 34 1.83 us 62.09 us 2.60 us 1.16 us 5 5 21.31 us 106.57 us 25.77 us 18.17 us 5 5 3.01 us 15.07 us 5.19 us 1.53 us 3 3 6.42 us 19.26 us 9.63 us 4.22 us 3 3 4.20 us 12.61 us 7.45 us 2.10 us 2 2 5.03 us 10.06 us 7.88 us 2.18 us 2 2 2.79 us 5.59 us 3.34 us 2.25 us 1 1 4.70 us 4.70 us 4.70 us 4.70 us 1 1 9.77 us 9.77 us 9.77 us 9.77 usAfter patch:
root@barry-desktop:/home/barry# time ./a.outreal 0m17.796s
user 0m24.962s
sys 5m28.774sPerf lock report:
Name acquired contended avg wait total wait max wait min wait 1407013 1407013 25.44 us 35.80 s 127.83 us 972 ns 845113 845113 47.96 us 40.54 s 152.86 us 1.21 us rcu_state 1920 1920 5.87 us 11.27 ms 25.02 us 1.76 us rcu_state 1889 1889 5.80 us 10.96 ms 30.35 us 1.61 us 140 140 3.97 us 555.52 us 12.51 us 1.20 us 10 10 61.34 us 613.42 us 96.90 us 44.48 us 8 8 24.03 us 192.26 us 37.64 us 6.95 us 8 8 12.32 us 98.56 us 25.55 us 4.16 usBoth system time and lock contention are noticeably reduced.
Signed-off-by Barry Song (Xiaomi) <[email protected]>--- include/linux/swap.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/swap.h +0
-0@@ -316,9 +316,9 @@ static inline bool folio_may_be_lru_cached(struct folio *folio)/*
include/linux/swap.h static inline bool folio_may_be_lru_cached(struct folio *folio) · line 324
Quoted 10 lines
include/linux/swap.h +2
-2* Holding PMD-sized folios in per-CPU LRU cache unbalances accounting. * Holding small numbers of low-order mTHP folios in per-CPU LRU cache -* will be sensible, but nobody has implemented and tested that yet. +* will be sensible. */ -return !folio_test_large (folio); +return folio_order (folio) <PAGE_ALLOC_COSTLY_ORDER ; }extern atomic_t lru_disable_count;
Sashiko rightfully raises that split_folio() can now fail more easily.
So we might want to proactively drain (earlier?) on some more of the split paths.
Signature · 3 lines
Cheers, David
Barry Song (Xiaomi)
reviewerRe: [RFC PATCH v2 1/2] mm: allow smaller large folios to use lru_cache
On Wed, Jul 29, 2026 at 8:13 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]
Quoted 15 lines
Quoted 15 lines
include/linux/swap.h +2
-2@@ -316,9 +316,9 @@ static inline bool folio_may_be_lru_cached(struct folio *folio)/* * Holding PMD-sized folios in per-CPU LRU cache unbalances accounting. * Holding small numbers of low-order mTHP folios in per-CPU LRU cache -* will be sensible, but nobody has implemented and tested that yet. +* will be sensible. */ -return !folio_test_large (folio); +return folio_order (folio) <PAGE_ALLOC_COSTLY_ORDER ; }
Quoted 5 lines
include/linux/swap.h static inline bool folio_may_be_lru_cached(struct folio *folio) · line 324
Quoted 1 line
include/linux/swap.h +0
-0extern atomic_t lru_disable_count;Sashiko rightfully raises that
split_folio()can now fail more easily.So we might want to proactively drain (earlier?) on some more of the split paths.
Yes, this is a pain. I haven't tested it, but perhaps a conceptual model could be something like this?
mm/huge_memory.c +11 -1
@@ -4069,6 +4069,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order,struct folio *new_folio, *next;int nr_shmem_dropped =0 ;enum ttu_flags ttu_flags =0 ; +bool maybe_in_lru_cache; +int expected_ref_count;int ret;pgoff_t end =0 ;
@@ -4154,7 +4156,15 @@ static int __folio_split(struct folio *folio, unsigned int new_order,* Racy check if we can split the page, before unmap_folio() will * split PMDs */ -if (folio_expected_ref_count (folio) !=folio_ref_count (folio) -1 ) { + maybe_in_lru_cache =folio_may_be_lru_cached (folio) && !folio_test_lru (folio); + expected_ref_count =folio_expected_ref_count (folio); +if (expected_ref_count + maybe_in_lru_cache <folio_ref_count (folio) -1 ) { + ret = -EAGAIN ; + goto out_unlock; + } +if (maybe_in_lru_cache) +lru_add_drain_all (); +if (expected_ref_count !=folio_ref_count (folio) -1 ) { ret = -EAGAIN ; goto out_unlock; }
Best Regards
Barry
David Hildenbrand (Arm)
maintainerRe: [RFC PATCH v2 1/2] mm: allow smaller large folios to use lru_cache
Quoted 42 lines · Barry Song (Xiaomi)
On Wed, Jul 29, 2026 at 8:13 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]Quoted 15 lines
Quoted 15 lines
include/linux/swap.h +2
-2@@ -316,9 +316,9 @@ static inline bool folio_may_be_lru_cached(struct folio *folio)/* * Holding PMD-sized folios in per-CPU LRU cache unbalances accounting. * Holding small numbers of low-order mTHP folios in per-CPU LRU cache -* will be sensible, but nobody has implemented and tested that yet. +* will be sensible. */ -return !folio_test_large (folio); +return folio_order (folio) <PAGE_ALLOC_COSTLY_ORDER ; }Quoted 5 lines
include/linux/swap.h static inline bool folio_may_be_lru_cached(struct folio *folio) · line 324
Quoted 1 line
include/linux/swap.h +0
-0extern atomic_t lru_disable_count;Sashiko rightfully raises that
split_folio()can now fail more easily.So we might want to proactively drain (earlier?) on some more of the split paths.
Yes, this is a pain. I haven't tested it, but perhaps a conceptual model could be something like this?
mm/huge_memory.c +2
-0@@ -4069,6 +4069,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order,struct folio *new_folio, *next;int nr_shmem_dropped =0 ;enum ttu_flags ttu_flags =0 ; +bool maybe_in_lru_cache; +int expected_ref_count;int ret;pgoff_t end =0 ;@@ -4154,7 +4156,15 @@ static int __folio_split(struct folio *folio, unsigned int new_order,* Racy check if we can split the page, before unmap_folio() will * split PMDs
mm/huge_memory.c static int __folio_split(struct folio *folio, unsigned int new_order, · line 4166
Quoted 10 lines
mm/huge_memory.c +8
-1*/ -if (folio_expected_ref_count (folio) !=folio_ref_count (folio) -1 ) { + maybe_in_lru_cache =folio_may_be_lru_cached (folio) && !folio_test_lru (folio); + expected_ref_count =folio_expected_ref_count (folio); +if (expected_ref_count + maybe_in_lru_cache <folio_ref_count (folio) -1 ) { + ret = -EAGAIN ; + goto out_unlock; + } +if (maybe_in_lru_cache) +lru_add_drain_all ();
The drain-all is nasty. In collect_longterm_unpinnable_folios() we escalate the drainining.
I think we recently stumbled into a similar problem in guest_memfd.
Ah, yes, there it is:
https://lore.kernel.org/r/[email protected]
Signature · 3 lines
Cheers, David
Zi Yan
reviewerRe: [RFC PATCH v2 1/2] mm: allow smaller large folios to use lru_cache
Quoted 57 lines · David Hildenbrand (Arm)
Quoted 42 lines · Barry Song (Xiaomi)
On Wed, Jul 29, 2026 at 8:13 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]Quoted 15 lines
Quoted 15 lines
include/linux/swap.h +2
-2@@ -316,9 +316,9 @@ static inline bool folio_may_be_lru_cached(struct folio *folio)/* * Holding PMD-sized folios in per-CPU LRU cache unbalances accounting. * Holding small numbers of low-order mTHP folios in per-CPU LRU cache -* will be sensible, but nobody has implemented and tested that yet. +* will be sensible. */ -return !folio_test_large (folio); +return folio_order (folio) <PAGE_ALLOC_COSTLY_ORDER ; }Quoted 5 lines
include/linux/swap.h static inline bool folio_may_be_lru_cached(struct folio *folio) · line 324
Quoted 1 line
include/linux/swap.h +0
-0extern atomic_t lru_disable_count;Sashiko rightfully raises that
split_folio()can now fail more easily.So we might want to proactively drain (earlier?) on some more of the split paths.
Yes, this is a pain. I haven't tested it, but perhaps a conceptual model could be something like this?
mm/huge_memory.c +2
-0@@ -4069,6 +4069,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order,struct folio *new_folio, *next;int nr_shmem_dropped =0 ;enum ttu_flags ttu_flags =0 ; +bool maybe_in_lru_cache; +int expected_ref_count;int ret;pgoff_t end =0 ;@@ -4154,7 +4156,15 @@ static int __folio_split(struct folio *folio, unsigned int new_order,* Racy check if we can split the page, before unmap_folio() will * split PMDs mm/huge_memory.c static int __folio_split(struct folio *folio, unsigned int new_order, · line 4166
Quoted 10 lines
mm/huge_memory.c +8
-1*/ -if (folio_expected_ref_count (folio) !=folio_ref_count (folio) -1 ) { + maybe_in_lru_cache =folio_may_be_lru_cached (folio) && !folio_test_lru (folio); + expected_ref_count =folio_expected_ref_count (folio); +if (expected_ref_count + maybe_in_lru_cache <folio_ref_count (folio) -1 ) { + ret = -EAGAIN ; + goto out_unlock; + } +if (maybe_in_lru_cache) +lru_add_drain_all ();The drain-all is nasty. In
collect_longterm_unpinnable_folios()we escalate the drainining.
Quoted 5 lines
I think we recently stumbled into a similar problem in
guest_memfd.Ah, yes, there it is:
Page migration has this issue for a long time.
I wonder if we can re-design per-CPU LRU cache so that we can remove a folio from per-CPU LRU cache without drainining any to LRU lists.
Something like:
1. have a folio on per-CPU LRU cache to store a pointer to its fb_batch's folio array position;
2. zero that position when we want to drain that specific folio.
But I have not looked into all the details to make sure that is feasible yet.
Signature · 2 lines
Best Regards, Yan, Zi
Barry Song (Xiaomi)
reviewerRFC PATCH v2 2/2 mm: improve large folio reuse for LRU-cached folios
In reply to Barry Song (Xiaomi)Large folios may now reside in the per-CPU LRU cache. Before attempting to reuse them, drain the local LRU cache, which can still be beneficial in cases where the folios are likely to remain in this CPU's LRU cache:
int main(int argc, char *argv[])
{
int i;
while (1) {
volatile int *p = mmap(0, SIZE, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0);
for (int i = 0; i < SIZE / sizeof(int); i++)
p[i] = i;
madvise((void *)p, SIZE, MADV_PAGEOUT);
if (!fork())
_exit(0);
for (int i = 0; i < SIZE / sizeof(int); i++)
p[i] = i;
munmap((void *)p, SIZE);
}
return 0;
}--- mm/memory.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
mm/memory.c +13 -0
@@ -4136,6 +4136,19 @@ static bool __wp_can_reuse_large_anon_folio(struct folio *folio,folio_unlock (folio); } +if (folio_may_be_lru_cached (folio) && !folio_test_lru (folio)) { +if (folio_ref_count (folio) >folio_large_mapcount (folio) +1 ) +return false; +/* +* A global LRU drain is too expensive, but a local drain +* can still be beneficial. For example, large folios that +* have just been swapped in and fall back from +* do_swap_page() to do_wp_page() are likely still in this +* CPU's LRU cache. +*/ +lru_add_drain (); + } +if (folio_large_mapcount (folio) !=folio_ref_count (folio))return false;
Signature · 1 line
2.39.3 (Apple Git-146)
David Hildenbrand (Arm)
maintainerRe: [RFC PATCH v2 2/2] mm: improve large folio reuse for LRU-cached folios
Quoted 34 lines · Barry Song (Xiaomi)
Large folios may now reside in the per-CPU LRU cache. Before attempting to reuse them, drain the local LRU cache, which can still be beneficial in cases where the folios are likely to remain in this CPU's LRU cache:
int
main(int argc, char *argv[]){ int i; while (1) { volatile int *p = mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); for (int i = 0; i < SIZE / sizeof(int); i++) p[i] = i; madvise((void *)p, SIZE, MADV_PAGEOUT); if (!fork()) _exit(0); for (int i = 0; i < SIZE / sizeof(int); i++) p[i] = i; munmap((void *)p, SIZE); } return 0; }Signed-off-by Barry Song (Xiaomi) <[email protected]>--- mm/memory.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)mm/memory.c +0
-0
mm/memory.c static bool __wp_can_reuse_large_anon_folio(struct folio *folio, · line 4140
Quoted 5 lines
mm/memory.c +2
-0@@ -4136,6 +4136,19 @@ static bool __wp_can_reuse_large_anon_folio(struct folio *folio,folio_unlock (folio); } +if (folio_may_be_lru_cached (folio) && !folio_test_lru (folio)) { +if (folio_ref_count (folio) >folio_large_mapcount (folio) +1 )
I assume that can just be a "!=".
mm/memory.c static bool __wp_can_reuse_large_anon_folio(struct folio *folio, · line 4148
Quoted 8 lines
mm/memory.c +8
-0+return false; +/* +* A global LRU drain is too expensive, but a local drain +* can still be beneficial. For example, large folios that +* have just been swapped in and fall back from +* do_swap_page() to do_wp_page() are likely still in this +* CPU's LRU cache. +*/
Do we really need that comment? It's just the same as we have in wp_can_reuse_anon_folio(). So either drop it or use the same comment.
I was wondering whether we could unify some logic with wp_can_reuse_anon_folio(), but the checks are just to different.
Apart from that LGTM. I'm sure there are some micro-optimizations to be had, but this here should suffice.
Signature · 3 lines
Cheers, David
Barry Song
reviewerRe: [RFC PATCH v2 2/2] mm: improve large folio reuse for LRU-cached folios
Quoted 39 lines · David Hildenbrand (Arm)
Quoted 37 lines · Barry Song (Xiaomi)
Large folios may now reside in the per-CPU LRU cache. Before attempting to reuse them, drain the local LRU cache, which can still be beneficial in cases where the folios are likely to remain in this CPU's LRU cache:
int
main(int argc, char *argv[]){ int i; while (1) { volatile int *p = mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); for (int i = 0; i < SIZE / sizeof(int); i++) p[i] = i; madvise((void *)p, SIZE, MADV_PAGEOUT); if (!fork()) _exit(0); for (int i = 0; i < SIZE / sizeof(int); i++) p[i] = i; munmap((void *)p, SIZE); } return 0; }Signed-off-by Barry Song (Xiaomi) <[email protected]>--- mm/memory.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)mm/memory.c +0
-0@@ -4136,6 +4136,19 @@ static bool __wp_can_reuse_large_anon_folio(struct folio *folio,folio_unlock (folio); }
Quoted 4 lines
mm/memory.c static bool __wp_can_reuse_large_anon_folio(struct folio *folio, · line 4140
Quoted 2 lines
mm/memory.c +2
-0+if (folio_may_be_lru_cached (folio) && !folio_test_lru (folio)) { +if (folio_ref_count (folio) >folio_large_mapcount (folio) +1 )I assume that can just be a "!=".
I was trying to guard against the case where a folio is not on the LRU but also does not hold an extra reference. For example, the folio is off the LRU, yetfolio_ref_count(folio) == folio_large_mapcount(folio).
In that case, we could simply reuse !folio_test_lru(folio).
Maybe such a case cannot actually happen, since paths like isolate_folio() take an extra reference before removing the folio from the LRU.
So we should be ok to just use folio_ref_count(folio) != folio_large_mapcount(folio) +1
Thanks
barry
David Hildenbrand (Arm)
maintainerRe: [RFC PATCH v2 2/2] mm: improve large folio reuse for LRU-cached folios
Quoted 51 lines · Barry Song
Quoted 39 lines · David Hildenbrand (Arm)
Quoted 37 lines · Barry Song (Xiaomi)
Large folios may now reside in the per-CPU LRU cache. Before attempting to reuse them, drain the local LRU cache, which can still be beneficial in cases where the folios are likely to remain in this CPU's LRU cache:
int
main(int argc, char *argv[]){ int i; while (1) { volatile int *p = mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); for (int i = 0; i < SIZE / sizeof(int); i++) p[i] = i; madvise((void *)p, SIZE, MADV_PAGEOUT); if (!fork()) _exit(0); for (int i = 0; i < SIZE / sizeof(int); i++) p[i] = i; munmap((void *)p, SIZE); } return 0; }Signed-off-by Barry Song (Xiaomi) <[email protected]>--- mm/memory.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)mm/memory.c +0
-0@@ -4136,6 +4136,19 @@ static bool __wp_can_reuse_large_anon_folio(struct folio *folio,folio_unlock (folio); }Quoted 4 lines
mm/memory.c static bool __wp_can_reuse_large_anon_folio(struct folio *folio, · line 4140
Quoted 2 lines
mm/memory.c +2
-0+if (folio_may_be_lru_cached (folio) && !folio_test_lru (folio)) { +if (folio_ref_count (folio) >folio_large_mapcount (folio) +1 )I assume that can just be a "!=".
I was trying to guard against the case where a folio is not on the LRU but also does not hold an extra reference. For example, the folio is off the LRU, yet
folio_ref_count(folio)==folio_large_mapcount(folio).
Quoted 8 lines
In that case, we could simply reuse !folio_test_lru(folio).
Maybe such a case cannot actually happen, since paths like
isolate_folio()take an extra reference before removing the folio from the LRU.So we should be ok to just use
folio_ref_count(folio)!=folio_large_mapcount(folio)+1
I just stumbled into something nasty last week:
https://lore.kernel.org/r/20260731-check_and_migrate_movable_folios-v1-1-e0002d7b791e@kernel.org
For an mTHP, if we end up pinning multiple pages, calculating pin_refs is problematic and might require us to batch over multiple pages.
But there are some nasty corner cases when just batching contiguous pages.
Signature · 3 lines
Cheers, David