mm: multi-gen LRU: replace get_lruvec() with mem_cgroup_lruvec()
Subsystems (3)
v2 cover mm: multi-gen LRU: replace get_lruvec() with mem_cgroup_lruvec()
This series cleans up MGLRU's get_lruvec() helper in mm/vmscan.c.
Patch 1 updates get_lruvec() to delegate to mem_cgroup_lruvec() rather than returning &pgdat->__lruvec directly when memcg is disabled on the kernel command line.
Patch 2 removes get_lruvec() completely and replaces all 15 call sites with mem_cgroup_lruvec().
Siddharth Chintamaneni (2):
mm: multi-gen LRU: use mem_cgroup_lruvec() in get_lruvec()
mm: multi-gen LRU: remove get_lruvec()
mm/vmscan.c | 50 +++++++++++++++----------------------------------- 1 file changed, 15 insertions(+), 35 deletions(-)
Signature · 1 line
2.45.4
PATCH v2 1/2 mm: multi-gen LRU: use mem_cgroup_lruvec() in get_lruvec()
Andrew Morton noted that get_lruvec() has the same pattern as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly instead of going through mem_cgroup_lruvec(), which initializes lruvec->pgdat.
In practice all callers of get_lruvec() run after kswapd_init() has already initialized pgdat->__lruvec.pgdat on every N_MEMORY node, so this is not a runtime bug. Have get_lruvec() delegate to mem_cgroup_lruvec() directly.
--- mm/vmscan.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
mm/vmscan.c +2 -13
@@ -2745,20 +2745,9 @@ static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) {struct pglist_data *pgdat =NODE_DATA (nid); -#ifdef CONFIG_MEMCG -if (memcg) { -struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec; - -/* see the comment in mem_cgroup_lruvec() */ -if (!lruvec->pgdat) - lruvec->pgdat = pgdat; - -return lruvec; - } -#endif -VM_WARN_ON_ONCE (!mem_cgroup_disabled ()); +VM_WARN_ON_ONCE (!memcg && !mem_cgroup_disabled ()); -return &pgdat->__lruvec; +return mem_cgroup_lruvec (memcg, pgdat); }static int get_swappiness (struct lruvec *lruvec,struct scan_control *sc)
Signature · 1 line
2.45.4
Re: [PATCH v2 1/2] mm: multi-gen LRU: use mem_cgroup_lruvec() in get_lruvec()
Quoted 10 lines · Siddharth Chintamaneni
Andrew Morton noted that
get_lruvec()has the same pattern aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly instead of going throughmem_cgroup_lruvec(), which initializeslruvec->pgdat.In practice all callers of
get_lruvec()run afterkswapd_init()has already initializedpgdat->__lruvec.pgdaton everyN_MEMORYnode, so this is not a runtime bug. Haveget_lruvec()delegate tomem_cgroup_lruvec()directly.
Somehow, I feel this changelog is a bit weird. The earlier description doesn't really justify the final sentence:
"Have get_lruvec() delegate to mem_cgroup_lruvec() directly."
Can we change it to the below?
The patch was initially developed to address a possible NULL reference [1].
Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.
In practice, we found that this cannot happen, as all callers of get_lruvec() run after kswapd_init() has initialized pgdat->__lruvec.pgdat on every N_MEMORY node. Therefore, this is not a runtime bug.
However, it would still be good to clean up get_lruvec() by removing the MGLRU-specific code and having it delegate directly to mem_cgroup_lruvec().
[1] https://lore.kernel.org/linux-mm/[email protected]/
Quoted 5 lines
Reviewed-by Baolin Wang <[email protected]>Signed-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>---
With the changelog updated as suggested,
Re: [PATCH v2 1/2] mm: multi-gen LRU: use mem_cgroup_lruvec() in get_lruvec()
Quoted 35 lines · On Tue, 3 Nov 2026 06:51:19 +0800 Barry Song <[email protected]> wrote:
Quoted 10 lines · Siddharth Chintamaneni
Andrew Morton noted that
get_lruvec()has the same pattern aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly instead of going throughmem_cgroup_lruvec(), which initializeslruvec->pgdat.In practice all callers of
get_lruvec()run afterkswapd_init()has already initializedpgdat->__lruvec.pgdaton everyN_MEMORYnode, so this is not a runtime bug. Haveget_lruvec()delegate tomem_cgroup_lruvec()directly.Somehow, I feel this changelog is a bit weird. The earlier description doesn't really justify the final sentence:
"Haveget_lruvec()delegate tomem_cgroup_lruvec()directly."Can we change it to the below?
The patch was initially developed to address a possible NULL reference [1].
Andrew Morton reported that
get_lruvec()has the same problem aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly and so bypasses the initialization oflruvec->pgdat. The MGLRU debugfs interface callsget_lruvec(), then dereferences the pgdat pointer inlru_gen_seq_show()and other paths, so these can hit the same NULL dereference.In practice, we found that this cannot happen, as all callers of
get_lruvec()run afterkswapd_init()has initializedpgdat->__lruvec.pgdaton everyN_MEMORYnode. Therefore, this is not a runtime bug.
Quoted 5 lines
However, it would still be good to clean up
get_lruvec()by removing the MGLRU-specific code and having it delegate directly tomem_cgroup_lruvec().
Thanks, I locally made that change.
From: Siddharth Chintamaneni <[email protected]>
Subject: mm: multi-gen LRU: use mem_cgroup_lruvec() in get_lruvec()
Date: Mon, 26 Oct 2026 20:23:12 +0000
Patch series "mm: multi-gen LRU: replace get_lruvec() with mem_cgroup_lruvec()", v2.
This series cleans up MGLRU's get_lruvec() helper in mm/vmscan.c.
Patch 1 updates get_lruvec() to delegate to mem_cgroup_lruvec() rather than returning &pgdat->__lruvec directly when memcg is disabled on the kernel command line.
Patch 2 removes get_lruvec() completely and replaces all 15 call sites with mem_cgroup_lruvec().
This patch (of 2):
The patch was initially developed to address a possible NULL reference [1].
Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.
In practice, we found that this cannot happen, as all callers of get_lruvec() run after kswapd_init() has initialized pgdat->__lruvec.pgdat on every N_MEMORY node. Therefore, this is not a runtime bug.
However, it would still be good to clean up get_lruvec() by removing the MGLRU-specific code and having it delegate directly to mem_cgroup_lruvec().
---
mm/vmscan.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
mm/vmscan.c renamed from mm/vmscan.c~mm-multi-gen-lru-use-mem_cgroup_lruvec-in-get_lruvec +2 -13
@@ -2866,20 +2866,9 @@ static struct lruvec *get_lruvec(struct {struct pglist_data *pgdat =NODE_DATA (nid); -#ifdef CONFIG_MEMCG -if (memcg) { -struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec; +VM_WARN_ON_ONCE (!memcg && !mem_cgroup_disabled ()); -/* see the comment in mem_cgroup_lruvec() */ -if (!lruvec->pgdat) - lruvec->pgdat = pgdat; - -return lruvec; - } -#endif -VM_WARN_ON_ONCE (!mem_cgroup_disabled ()); - -return &pgdat->__lruvec; +return mem_cgroup_lruvec (memcg, pgdat); }static int get_swappiness (struct lruvec *lruvec,struct scan_control *sc)
_
PATCH v2 2/2 mm: multi-gen LRU: remove get_lruvec()
In reply to Siddharth Chintamaneniget_lruvec() was originally introduced in commit ec1c86b25f4b ("mm:
multi-gen LRU: groundwork") to guard against NULL NODE_DATA(nid) on uninitialized NUMA nodes (return pgdat ? &pgdat->__lruvec : NULL;). After commit 09f49dca570a ("mm: handle uninitialized numa nodes gracefully") and commit 931b6a8b36a2 ("mm: multi-gen LRU: remove NULL checks on NODE_DATA()"), that NULL check was removed, leaving get_lruvec() as a thin wrapper around mem_cgroup_lruvec().
Remove get_lruvec() and call mem_cgroup_lruvec() directly at all call sites.
--- mm/vmscan.c | 39 +++++++++++++++------------------------ 1 file changed, 15 insertions(+), 24 deletions(-)
mm/vmscan.c +15 -24
@@ -2741,15 +2741,6 @@ static bool should_clear_pmd_young(void)#define get_memcg_gen (seq) ((seq) % MEMCG_NR_GENS)#define get_memcg_bin (bin) ((bin) % MEMCG_NR_BINS) -static struct lruvec *get_lruvec (struct mem_cgroup *memcg,int nid) -{ -struct pglist_data *pgdat =NODE_DATA (nid); - -VM_WARN_ON_ONCE (!memcg && !mem_cgroup_disabled ()); - -return mem_cgroup_lruvec (memcg, pgdat); -} -static int get_swappiness (struct lruvec *lruvec,struct scan_control *sc) {struct mem_cgroup *memcg =lruvec_memcg (lruvec);
@@ -2942,7 +2933,7 @@ void lru_gen_add_mm(struct mm_struct *mm)spin_lock (&mm_list->lock);for_each_node_state (nid,N_MEMORY ) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));struct lru_gen_mm_state *mm_state =get_mm_state (lruvec);/* the first addition since the last iteration */
@@ -2972,7 +2963,7 @@ void lru_gen_del_mm(struct mm_struct *mm)spin_lock (&mm_list->lock);for_each_node (nid) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));struct lru_gen_mm_state *mm_state =get_mm_state (lruvec);/* where the current iteration continues after */
@@ -4424,7 +4415,7 @@ void lru_gen_online_memcg(struct mem_cgroup *memcg)for_each_node (nid) {struct pglist_data *pgdat =NODE_DATA (nid); -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg, pgdat);spin_lock_irq (&pgdat->memcg_lru.lock);
@@ -4446,7 +4437,7 @@ void lru_gen_offline_memcg(struct mem_cgroup *memcg)int nid;for_each_node (nid) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));lru_gen_rotate_memcg (lruvec,MEMCG_LRU_OLD ); }
@@ -4459,7 +4450,7 @@ void lru_gen_release_memcg(struct mem_cgroup *memcg)for_each_node (nid) {struct pglist_data *pgdat =NODE_DATA (nid); -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg, pgdat);spin_lock_irq (&pgdat->memcg_lru.lock);
@@ -4480,7 +4471,7 @@ void lru_gen_release_memcg(struct mem_cgroup *memcg)void lru_gen_soft_reclaim (struct mem_cgroup *memcg,int nid) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));/* see the comment on MEMCG_NR_GENS */ if (READ_ONCE (lruvec->lrugen.seg) !=MEMCG_LRU_HEAD )
@@ -4489,7 +4480,7 @@ void lru_gen_soft_reclaim(struct mem_cgroup *memcg, int nid)bool recheck_lru_gen_max_memcg (struct mem_cgroup *memcg,int nid) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));int type;for (type =0 ; type <ANON_AND_FILE ; type++) {
@@ -4536,7 +4527,7 @@ static void try_to_inc_max_seq_nowalk(struct mem_cgroup *memcg,*/ void max_lru_gen_memcg (struct mem_cgroup *memcg,int nid) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));int type;for (type =0 ; type <ANON_AND_FILE ; type++) {
@@ -4613,8 +4604,8 @@ void lru_gen_reparent_memcg(struct mem_cgroup *memcg, struct mem_cgroup *parent,struct zone *zone;enum lru_list lru; - child_lruvec =get_lruvec (memcg, nid); - parent_lruvec =get_lruvec (parent, nid); + child_lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid)); + parent_lruvec =mem_cgroup_lruvec (parent,NODE_DATA (nid));for_each_managed_zone_pgdat (zone,NODE_DATA (nid), zid,MAX_NR_ZONES -1 )for (type =0 ; type <ANON_AND_FILE ; type++)
@@ -5370,7 +5361,7 @@ static void lru_gen_change_state(bool enabled)int nid;for_each_node (nid) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));lruvec_lock_irq (lruvec);
@@ -5501,7 +5492,7 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos)for_each_node_state (nid,N_MEMORY ) {if (!nr_to_skip--) -return get_lruvec (memcg, nid); +return mem_cgroup_lruvec (memcg,NODE_DATA (nid)); } }while ((memcg =mem_cgroup_iter (NULL , memcg,NULL )));
@@ -5533,7 +5524,7 @@ static void *lru_gen_seq_next(struct seq_file *m, void *v, loff_t *pos) nid = first_memory_node; } -return get_lruvec (memcg, nid); +return mem_cgroup_lruvec (memcg,NODE_DATA (nid)); } static void lru_gen_seq_show_full(struct seq_file *m, struct lruvec *lruvec,
@@ -5715,7 +5706,7 @@ static int run_cmd(char cmd, u64 memcg_id, int nid, unsigned long seq, goto done; sc->target_mem_cgroup = memcg; - lruvec =get_lruvec (memcg, nid); + lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));if (swappiness <MIN_SWAPPINESS ) swappiness =get_swappiness (lruvec, sc);
@@ -5899,7 +5890,7 @@ void lru_gen_exit_memcg(struct mem_cgroup *memcg)VM_WARN_ON_ONCE (mm_list && !list_empty (&mm_list->fifo));for_each_node (nid) { -struct lruvec *lruvec =get_lruvec (memcg, nid); +struct lruvec *lruvec =mem_cgroup_lruvec (memcg,NODE_DATA (nid));struct lru_gen_mm_state *mm_state =get_mm_state (lruvec); VM_WARN_ON_ONCE(memchr_inv(lruvec->lrugen.nr_pages, 0,
Signature · 1 line
2.45.4
Re: [PATCH v2 2/2] mm: multi-gen LRU: remove get_lruvec()
Quoted 9 lines · Siddharth Chintamaneni
get_lruvec()was originally introduced in commitec1c86b25f4b("mm:
multi-gen LRU: groundwork") to guard against NULLNODE_DATA(nid)on uninitialized NUMA nodes (return pgdat ? &pgdat->__lruvec : NULL;). After commit09f49dca570a("mm: handle uninitialized numa nodes gracefully") and commit931b6a8b36a2("mm: multi-gen LRU: remove NULL checks onNODE_DATA()"), that NULL check was removed, leavingget_lruvec()as a thin wrapper aroundmem_cgroup_lruvec().
Quoted 8 lines
Remove
get_lruvec()and callmem_cgroup_lruvec()directly at all call sites.Suggested-by Barry Song <[email protected]>Reviewed-by Baolin Wang <[email protected]>Signed-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>---
It's great to see the MGLRU-specific get_lruvec() gone. Thanks! Hopefully, we can get rid of more MGLRU-specific code, such as get_swappiness().
Re: [PATCH v2 2/2] mm: multi-gen LRU: remove get_lruvec()
Quoted 21 lines · Barry Song
Quoted 9 lines · Siddharth Chintamaneni
get_lruvec()was originally introduced in commitec1c86b25f4b("mm:
multi-gen LRU: groundwork") to guard against NULLNODE_DATA(nid)on uninitialized NUMA nodes (return pgdat ? &pgdat->__lruvec : NULL;). After commit09f49dca570a("mm: handle uninitialized numa nodes gracefully") and commit931b6a8b36a2("mm: multi-gen LRU: remove NULL checks onNODE_DATA()"), that NULL check was removed, leavingget_lruvec()as a thin wrapper aroundmem_cgroup_lruvec().Quoted 8 lines
Remove
get_lruvec()and callmem_cgroup_lruvec()directly at all call sites.Suggested-by Barry Song <[email protected]>Reviewed-by Baolin Wang <[email protected]>Signed-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>---
Quoted 4 lines
It's great to see the MGLRU-specific
get_lruvec()gone. Thanks! Hopefully, we can get rid of more MGLRU-specific code, such asget_swappiness().
I just looked into removing get_swappiness() in favor of its LRU variant (sc_swappiness()). Do you mean moving the additional conditionals directly into the callsites?
something like this -
<AI Suggested>
mm/vmscan.c +0 -0
@@ -2761,25 +2761,6 @@ static struct lruvec *get_lruvec(struct
mem_cgroup *memcg, int nid)
return &pgdat->__lruvec;
}
-static int get_swappiness(struct lruvec *lruvec, struct scan_control *sc) -{
- struct mem_cgroup *memcg = lruvec_memcg(lruvec);
- struct pglist_data *pgdat = lruvec_pgdat(lruvec);
- int swappiness = sc_swappiness(sc, memcg);
-
- if (swappiness == SWAPPINESS_ANON_ONLY)
- return swappiness;
-
- if (!sc->may_swap)
- return 0;
-
- if (!can_demote(pgdat->node_id, sc, memcg) &&
- mem_cgroup_get_nr_swap_pages(memcg) < MIN_LRU_BATCH)
- return 0;
-
- return swappiness;
-}
-
static int get_nr_gens(struct lruvec *lruvec, int type)
{
return lruvec->lrugen.max_seq - lruvec->lrugen.min_seq[type] + 1;
@@ -4173,8 +4154,13 @@ static unsigned long lruvec_evictable_size(struct lruvec *lruvec, int swappiness
static bool lruvec_is_sizable(struct lruvec *lruvec, struct scan_control *sc) {
unsigned long total;
- int swappiness = get_swappiness(lruvec, sc);
struct mem_cgroup *memcg = lruvec_memcg(lruvec);
+ struct pglist_data *pgdat = lruvec_pgdat(lruvec);
+ int swappiness = sc_swappiness(sc, memcg);
+
+ if (swappiness != SWAPPINESS_ANON_ONLY &&
+ (!sc->may_swap || !can_reclaim_anon_pages(memcg, pgdat->node_id, sc)))
+ swappiness = 0;
Quoted 1 line
Reviewed-by Barry Song <[email protected]>
Barry Song
reviewerRe: [PATCH v2 2/2] mm: multi-gen LRU: remove get_lruvec()
Quoted 23 lines · Siddharth Chintamaneni
Quoted 21 lines · Barry Song
Quoted 17 lines · Siddharth Chintamaneni
get_lruvec()was originally introduced in commitec1c86b25f4b("mm:
multi-gen LRU: groundwork") to guard against NULLNODE_DATA(nid)on uninitialized NUMA nodes (return pgdat ? &pgdat->__lruvec : NULL;). After commit09f49dca570a("mm: handle uninitialized numa nodes gracefully") and commit931b6a8b36a2("mm: multi-gen LRU: remove NULL checks onNODE_DATA()"), that NULL check was removed, leavingget_lruvec()as a thin wrapper aroundmem_cgroup_lruvec().Remove
get_lruvec()and callmem_cgroup_lruvec()directly at all call sites.Suggested-by Barry Song <[email protected]>Reviewed-by Baolin Wang <[email protected]>Signed-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>---
Quoted 8 lines
Quoted 4 lines
It's great to see the MGLRU-specific
get_lruvec()gone. Thanks! Hopefully, we can get rid of more MGLRU-specific code, such asget_swappiness().I just looked into removing
get_swappiness()in favor of its LRU variant (sc_swappiness()). Do you mean moving the additional conditionals directly into the callsites?
Nop. In MGLRU, swappiness has values: 0, 1–200, and 201.
With swappiness=0, MGLRU only reclaims file pages and doesn't care about the hotness of anon pages.
With swappiness=201, MGLRU only reclaims anon pages and doesn't care about the hotness of file pages.
In both cases, we allow cold/hot inversions in inc_min_seq(), which can help reduce CPU usage:
static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness) {
/* For file type, skip the check if swappiness is anon only */
if (type && (swappiness == SWAPPINESS_ANON_ONLY))
goto done;
/* For anon type, skip the check if swappiness is zero (file only) */
if (!type && !swappiness)
goto done;
...
}Then we could have cases where swappiness is 1 ~ 200, but we still cannot reclaim anon due to:
1. Swap is full, or the memcg's swap quota is exhausted.
2. The allocation uses GFP_NOIO, but the swap cache is low [1].
So right now, we have to hack get_swappiness() to return 0 to ensure that reclaim only happens on file pages, or we at least need to hack reclamation path to use swappiness=0.
This is quite bad, as it can cause cold/hot inversions for anon pages. I believe this is already a bug in the current get_swappiness() implementation when swap is full.
Ideally, swappiness values from 1 to 200 and the reclaimable type should be orthogonal; they should not be coupled.
In other words, we shouldn't need to force swappiness=0 to reclaim only file pages. If swappiness is set to 60, it should remain 60 across all MGLRU reclaim paths. The reclaimable type should be handled separately, without forcing swappiness to 0 and introducing confusion and bugs.
Best,
Barry
Re: [PATCH v2 2/2] mm: multi-gen LRU: remove get_lruvec()
Quoted 50 lines · Barry Song
Quoted 23 lines · Siddharth Chintamaneni
Quoted 21 lines · Barry Song
Quoted 17 lines · Siddharth Chintamaneni
get_lruvec()was originally introduced in commitec1c86b25f4b("mm:
multi-gen LRU: groundwork") to guard against NULLNODE_DATA(nid)on uninitialized NUMA nodes (return pgdat ? &pgdat->__lruvec : NULL;). After commit09f49dca570a("mm: handle uninitialized numa nodes gracefully") and commit931b6a8b36a2("mm: multi-gen LRU: remove NULL checks onNODE_DATA()"), that NULL check was removed, leavingget_lruvec()as a thin wrapper aroundmem_cgroup_lruvec().Remove
get_lruvec()and callmem_cgroup_lruvec()directly at all call sites.Suggested-by Barry Song <[email protected]>Reviewed-by Baolin Wang <[email protected]>Signed-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>---
Quoted 8 lines
Quoted 4 lines
It's great to see the MGLRU-specific
get_lruvec()gone. Thanks! Hopefully, we can get rid of more MGLRU-specific code, such asget_swappiness().I just looked into removing
get_swappiness()in favor of its LRU variant (sc_swappiness()). Do you mean moving the additional conditionals directly into the callsites?Nop. In MGLRU, swappiness has values: 0, 1–200, and 201.
With swappiness=0, MGLRU only reclaims file pages and doesn't care about the hotness of anon pages.
With swappiness=201, MGLRU only reclaims anon pages and doesn't care about the hotness of file pages.
In both cases, we allow cold/hot inversions in
inc_min_seq(), which can help reduce CPU usage:static bool
inc_min_seq(struct lruvec *lruvec, int type, int swappiness){
Quoted 10 lines
/* For file type, skip the check if swappiness is anon only */
if (type && (swappiness == SWAPPINESS_ANON_ONLY))
goto done;
/* For anon type, skip the check if swappiness is zero (file only) */
if (!type && !swappiness)
goto done;
...
}Interesting!
Quoted 16 lines
Then we could have cases where swappiness is 1 ~ 200, but we still cannot reclaim anon due to:
1. Swap is full, or the memcg's swap quota is exhausted.
2. The allocation uses
GFP_NOIO, but the swap cache is low [1].So right now, we have to hack
get_swappiness()to return 0 to ensure that reclaim only happens on file pages, or we at least need to hack reclamation path to use swappiness=0.This is quite bad, as it can cause cold/hot inversions for anon pages. I believe this is already a bug in the current
get_swappiness()implementation when swap is full.
Quoted 9 lines
Ideally, swappiness values from 1 to 200 and the reclaimable type should be orthogonal; they should not be coupled.
In other words, we shouldn't need to force swappiness=0 to reclaim only file pages. If swappiness is set to 60, it should remain 60 across all MGLRU reclaim paths. The reclaimable type should be handled separately, without forcing swappiness to 0 and introducing confusion and bugs.
This is more involved than I initially thought. Let me play around with it a bit and come back.
If you have any suggestions with experiments that I could run to get a good understanding on this problem, let me know.
Quoted 4 lines
Best,
Barry