Skip to contentSkip to conversation

mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline

PATCH In review Siddharth Chintamaneni 4 participants 16 messages Oct 17 – Oct 21

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline

When CONFIG_MEMCG=y, lruvec_pgdat() returns lruvec->pgdat, which for the per-node pgdat->__lruvec is left NULL by lruvec_init() and only filled in lazily by mem_cgroup_lruvec(). Normally that happens early, e.g. on the first folio_add_lru().

With cgroup_disable=memory, lru_gen_shrink_node() passes &pgdat->__lruvec directly to shrink_one(). If direct reclaim runs before anything has called mem_cgroup_lruvec() for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel), get_swappiness() ends up dereferencing a NULL pgdat:

  Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
  pc : get_swappiness+0x68/0xb0
  lr : try_to_shrink_lruvec+0x54/0x2c0
  Call trace:
   get_swappiness+0x68/0xb0 (P)
   try_to_shrink_lruvec+0x54/0x2c0
   shrink_one+0x8c/0x240
   shrink_node+0xc6c/0xe08
   do_try_to_free_pages+0xdc/0x570
   try_to_free_pages+0x124/0x278
   __alloc_frozen_pages_noprof+0x6e8/0xf78
   alloc_pages_mpol+0xc4/0x1c0
   alloc_frozen_pages_noprof+0x54/0xe8
   alloc_pages_noprof+0x1c/0x40
   atomic_pool_expand+0x98/0x300
   __dma_atomic_pool_init+0x58/0xd8
   dma_atomic_pool_init+0x154/0x170
   do_one_initcall+0x64/0x350
   kernel_init_freeable+0x214/0x430
   kernel_init+0x2c/0x208
   ret_from_fork+0x10/0x20

The classic LRU path and MGLRU before per-node lru_gen_folio lists obtained the lruvec through mem_cgroup_lruvec(), which sets lruvec->pgdat.

The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with CONFIG_ARM64_64K_PAGES=y, CONFIG_MEMCG=y, CONFIG_LRU_GEN_ENABLED=y and:

mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory

These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:

  Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
  pc : get_swappiness+0xd4/0x150
  Call trace:
   get_swappiness+0xd4/0x150 (P)
   try_to_shrink_lruvec+0x44/0x2b8
   shrink_one+0x80/0x1a0
   shrink_node+0xd98/0xea8
   do_try_to_free_pages+0xf4/0x4b0
   try_to_free_pages+0x110/0x360
   __alloc_frozen_pages_noprof+0x8b0/0x1718
   ...
   __filemap_get_folio_mpol+0x2ac/0x548
   simple_write_begin+0x44/0xf8
   page_symlink+0x178/0x198
   ...
   do_populate_rootfs+0x13c/0x200
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
Assisted-by LLM
---
 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmscan.c +1 -1
index f11491ee9ed5..0c57e4f1eb15 100644
@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control *
 		sc->nr_reclaimed = 0;
 
 	if (mem_cgroup_disabled())
-		shrink_one(&pgdat->__lruvec, sc);
+		shrink_one(mem_cgroup_lruvec(NULL, pgdat), sc);
 	else
 		shrink_many(pgdat, sc);
 
Signature · 1 line
2.45.4
#1 lore raw

PATCH v2 2/2 mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.

Fixes ec1c86b25f4b ("mm: multi-gen LRU: groundwork")
Assisted-by LLM
---
 mm/vmscan.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)
mm/vmscan.c +2 -13
index 0c57e4f1eb15..99e9f07fa2e1 100644
@@ -2745,20 +2745,9 @@ static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid)
 {
 	struct pglist_data *pgdat = NODE_DATA(nid);
 
-#ifdef CONFIG_MEMCG
-	if (memcg) {
-		struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec;
-
-		/* see the comment in mem_cgroup_lruvec() */
-		if (!lruvec->pgdat)
-			lruvec->pgdat = pgdat;
-
-		return lruvec;
-	}
-#endif
-	VM_WARN_ON_ONCE(!mem_cgroup_disabled());
+	VM_WARN_ON_ONCE(!memcg && !mem_cgroup_disabled());
 
-	return &pgdat->__lruvec;
+	return mem_cgroup_lruvec(memcg, pgdat);
 }
 
 static int get_swappiness(struct lruvec *lruvec, struct scan_control *sc)
Signature · 1 line
2.45.4
reviewer
#2 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 8 lines · Siddharth Chintamaneni

Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.

So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?

Quoted 25 lines
Fixes ec1c86b25f4b ("mm: multi-gen LRU: groundwork")
Assisted-by LLM
---
 mm/vmscan.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)
mm/vmscan.c +0 -7
index 0c57e4f1eb15..99e9f07fa2e1 100644
@@ -2745,20 +2745,9 @@ static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid)
 {
        struct pglist_data *pgdat = NODE_DATA(nid);
 
-#ifdef CONFIG_MEMCG
-       if (memcg) {
-               struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec;
-
-               /* see the comment in mem_cgroup_lruvec() */
-               if (!lruvec->pgdat)
-                       lruvec->pgdat = pgdat;

mm/vmscan.c static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) · line 2751

Quoted 10 lines
mm/vmscan.c +2 -6
-
-               return lruvec;
-       }
-#endif
-       VM_WARN_ON_ONCE(!mem_cgroup_disabled());
+       VM_WARN_ON_ONCE(!memcg && !mem_cgroup_disabled());
 
-       return &pgdat->__lruvec;
+       return mem_cgroup_lruvec(memcg, pgdat);
 }

I've been struggling to understand why MGLRU needs its own get_lruvec() when mem_cgroup_lruvec() already exists. With your patch, it feels like we're one step closer to getting rid of MGLRU's own variant in the future.

Quoted 4 lines
mm/vmscan.c +0 -1
 static int get_swappiness(struct lruvec *lruvec, struct scan_control *sc)
--

2.45.4

Best Regards
Barry

#3 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni

Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.

Quoted 5 lines

So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?

That is my high level understanding. I'll let Andrew to comment if I am missing something,

Quoted 32 lines
Quoted 32 lines
Fixes ec1c86b25f4b ("mm: multi-gen LRU: groundwork")
Assisted-by LLM
---
 mm/vmscan.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)
mm/vmscan.c +1 -12
index 0c57e4f1eb15..99e9f07fa2e1 100644
@@ -2745,20 +2745,9 @@ static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid)
 {
        struct pglist_data *pgdat = NODE_DATA(nid);
 
-#ifdef CONFIG_MEMCG
-       if (memcg) {
-               struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec;
-
-               /* see the comment in mem_cgroup_lruvec() */
-               if (!lruvec->pgdat)
-                       lruvec->pgdat = pgdat;
-
-               return lruvec;
-       }
-#endif
-       VM_WARN_ON_ONCE(!mem_cgroup_disabled());
+       VM_WARN_ON_ONCE(!memcg && !mem_cgroup_disabled());
 
Quoted 9 lines

mm/vmscan.c static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) · line 2751

Quoted 3 lines
mm/vmscan.c +1 -1
-       return &pgdat->__lruvec;
+       return mem_cgroup_lruvec(memcg, pgdat);
 }

I've been struggling to understand why MGLRU needs its own get_lruvec() when mem_cgroup_lruvec() already exists. With your patch, it feels like we're one step closer to getting rid of MGLRU's own variant in the future.

I agree with that! I'm happy to send a follow-up cleanup patch to remove get_lruvec if that makes sense. Not sure how it will affect stable backports.

Quoted 7 lines
Quoted 4 lines
mm/vmscan.c +0 -1
 static int get_swappiness(struct lruvec *lruvec, struct scan_control *sc)
--

2.45.4

Best Regards
Barry

reviewer
#4 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni

Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.

Quoted 9 lines
Quoted 5 lines

So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?

That is my high level understanding. I'll let Andrew to comment if I am missing something,

I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:

qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 \
-m 1G \
-object memory-backend-ram,id=ram0,size=1048572K \
-object memory-backend-ram,id=ram1,size=4K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel arch/arm64/boot/Image \
-nographic -append "cgroup_disable=memory
...

Then I got a machine where node1 is very difficult to have a page in lru.

then i ran the debugfs for mglru:

/ # cat /sys/kernel/debug/lru_gen
memcg 0
node 0

          0      14972          0         953
          1      14972          0           1
          2      14972         18        3308
          3      14972          0           0
 node     1
          0      14972          0           0
          1      14972          0           0
          2      14972          0           0
          3      14972          0           0

now you see node 1 has no page in lru but yet i didn't have any crash.

i think i will need more time to figure out what is the problem and how to reproduce it.

#5 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 58 lines · Barry Song
Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni

Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.

Quoted 9 lines
Quoted 5 lines

So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?

That is my high level understanding. I'll let Andrew to comment if I am missing something,

I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:

qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \

(quoted diff) +0 -6
-smp 4 \
-m 1G \
-object memory-backend-ram,id=ram0,size=1048572K \
-object memory-backend-ram,id=ram1,size=4K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
  -kernel arch/arm64/boot/Image \
 -nographic -append "cgroup_disable=memory
...

Then I got a machine where node1 is very difficult to have a page in lru.

then i ran the debugfs for mglru:

/ # cat /sys/kernel/debug/lru_gen
memcg 0
node 0

          0      14972          0         953
          1      14972          0           1
          2      14972         18        3308
          3      14972          0           0
 node     1
          0      14972          0           0
          1      14972          0           0
          2      14972          0           0
          3      14972          0           0
Quoted 4 lines

now you see node 1 has no page in lru but yet i didn't have any crash.

i think i will need more time to figure out what is the problem and how to reproduce it.

I've asked AI to repro the issue with the similar setup

qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 -m 1G \
-object memory-backend-ram,id=ram0,size=1048512K \
-object memory-backend-ram,id=ram1,size=64K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel "$KERNEL" \
-initrd "$INITRD" \
-no-reboot \
-append "console=ttyAMA0 cgroup_disable=memory
initcall_blacklist=kswapd_init panic=-1"

and it was able to repo it. AI identified that kswapd is initializing the pgdat during boot, so blacklisting kswap_init was able to repro the crash when reading /sys/kernel/debug/lru_gen

[    1.347868] Unable to handle kernel access to user memory outside
uaccess routines at virtual address 0000000000005040
[    1.347920] Mem abort info:
[    1.347929]   ESR = 0x0000000096000005
[    1.347950]   EC = 0x25: DABT (current EL), IL = 32 bits
[    1.347964]   SET = 0, FnV = 0
[    1.347974]   EA = 0, S1PTW = 0
[    1.347986]   FSC = 0x05: level 1 translation fault
[    1.348249] Data abort info:
[    1.348277]   ISV = 0, ISS = 0x00000005, ISS2 = 0x00000000
[    1.348291]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[    1.348302]   GCS = 0, Overlay = 0, DirtyBit = 0
[    1.348345] user pgtable: 64k pages, 48-bit VAs, pgdp=00000000485f6600
[    1.348369] [0000000000005040] pgd=0800000043640403,
p4d=0800000043640403, pud=0800000043640403, pmd=0800000043620403,
pte=0000000000000000
[    1.349148] Internal error: Oops: 0000000096000005 [#1]  SMP
[    1.350414] Modules linked in:
[    1.350896] CPU: 0 UID: 0 PID: 1 Comm: init Not tainted 7.3.0-rc7+
#9 PREEMPT(full)
[    1.351111] Hardware name: linux,dummy-virt (DT)
[    1.351411] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    1.351581] pc : lru_gen_seq_show+0x5c/0x460
[    1.351800] lr : lru_gen_seq_show+0x4c/0x460
[    1.351898] sp : ffff80008034f9a0
[    1.351975] x29: ffff80008034fa20 x28: ffff0000035e67e8 x27: 0000000000400cc0
[    1.352297] x26: 000000007fff0000 x25: ffff0000035e67d8 x24: 0000000000000000
[    1.352454] x23: ffff80008034fb00 x22: ffff80008034fb28 x21: 0000000000000000
[    1.352626] x20: ffff0000035e67b0 x19: ffff00003ffc2f20 x18: ffff800080360060
[    1.352775] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[    1.352915] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[    1.353058] x11: 0000000000000000 x10: 0000000000000000 x9 : ffffa3468a020c5c
[    1.353226] x8 : ffff000008b07000 x7 : 0000000000000000 x6 : 000000000000003f
[    1.353367] x5 : 00000000ffffffff x4 : 0000000000000003 x3 : 0000000000001000
[    1.353511] x2 : ffff00003ffc2f20 x1 : 0000000000004000 x0 : 0000000000000000
[    1.353732] Call trace:
[    1.353925]  lru_gen_seq_show+0x5c/0x460 (P)
[    1.354110]  seq_read_iter+0xec/0x4c0
[    1.354213]  seq_read+0xfc/0x148
[    1.354290]  full_proxy_read+0x78/0xc0
[    1.354377]  vfs_read+0xd0/0x338
[    1.354453]  ksys_read+0x7c/0x120
[    1.354528]  __arm64_sys_read+0x24/0x40
[    1.354611]  invoke_syscall+0xa8/0x120
[    1.354695]  el0_svc_common.constprop.0+0x4c/0x100
[    1.354791]  do_el0_svc+0x28/0x40
[    1.354865]  el0_svc+0x48/0x2b0
[    1.354939]  el0t_64_sync_handler+0xa8/0xf0
[    1.355026]  el0t_64_sync+0x1b8/0x1c0
[    1.355250] Code: f9000fe0 f9432661 aa1303e2 91401021 (b9504033)
[    1.355647] ---[ end trace 0000000000000000 ]---
[    1.356555] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[    1.356888] SMP: stopping secondary CPUs
[    1.357525] Kernel Offset: 0x234609c00000 from 0xffff800080000000
[    1.357675] PHYS_OFFSET: 0x40000000
[    1.357762] CPU features: 0x0,00000000,09000020,00100002,0800821b
[    1.357901] Memory Limit: none
reviewer
#6 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 79 lines · Siddharth Chintamaneni
Quoted 58 lines · Barry Song
Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni

Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.

Quoted 9 lines
Quoted 5 lines

So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?

That is my high level understanding. I'll let Andrew to comment if I am missing something,

I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:

qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \

(quoted diff) +0 -6
-smp 4 \
-m 1G \
-object memory-backend-ram,id=ram0,size=1048572K \
-object memory-backend-ram,id=ram1,size=4K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
  -kernel arch/arm64/boot/Image \
 -nographic -append "cgroup_disable=memory
...

Then I got a machine where node1 is very difficult to have a page in lru.

then i ran the debugfs for mglru:

/ # cat /sys/kernel/debug/lru_gen
memcg 0
node 0

          0      14972          0         953
          1      14972          0           1
          2      14972         18        3308
          3      14972          0           0
 node     1
          0      14972          0           0
          1      14972          0           0
          2      14972          0           0
          3      14972          0           0
Quoted 4 lines

now you see node 1 has no page in lru but yet i didn't have any crash.

i think i will need more time to figure out what is the problem and how to reproduce it.

I've asked AI to repro the issue with the similar setup

qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 -m 1G \
-object memory-backend-ram,id=ram0,size=1048512K \
-object memory-backend-ram,id=ram1,size=64K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel "$KERNEL" \
-initrd "$INITRD" \
-no-reboot \
-append "console=ttyAMA0 cgroup_disable=memory
initcall_blacklist=kswapd_init panic=-1"

Quoted 3 lines

and it was able to repo it. AI identified that kswapd is initializing the pgdat during boot, so blacklisting kswap_init was able to repro the crash when reading /sys/kernel/debug/lru_gen

Thanks!
I guess you mean the following:

-module_init(kswapd_init)
+//module_init(kswapd_init)

Yes, I can reproduce it with the above as well.

But kswapd_init() is definitely called before we enter userspace, so this doesn't seem to be an issue here?

So I feel there is no bug here, but this cleanup is still welcome. Am I missing any other scenarios?

Best Regards
Barry

#7 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 92 lines · Barry Song
Quoted 79 lines · Siddharth Chintamaneni
Quoted 58 lines · Barry Song
Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni

Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.

Quoted 9 lines
Quoted 5 lines

So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?

That is my high level understanding. I'll let Andrew to comment if I am missing something,

I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:

qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \

(quoted diff) +0 -6
-smp 4 \
-m 1G \
-object memory-backend-ram,id=ram0,size=1048572K \
-object memory-backend-ram,id=ram1,size=4K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
  -kernel arch/arm64/boot/Image \
 -nographic -append "cgroup_disable=memory
...

Then I got a machine where node1 is very difficult to have a page in lru.

then i ran the debugfs for mglru:

/ # cat /sys/kernel/debug/lru_gen
memcg 0
node 0

          0      14972          0         953
          1      14972          0           1
          2      14972         18        3308
          3      14972          0           0
 node     1
          0      14972          0           0
          1      14972          0           0
          2      14972          0           0
          3      14972          0           0
Quoted 4 lines

now you see node 1 has no page in lru but yet i didn't have any crash.

i think i will need more time to figure out what is the problem and how to reproduce it.

I've asked AI to repro the issue with the similar setup

qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 -m 1G \
-object memory-backend-ram,id=ram0,size=1048512K \
-object memory-backend-ram,id=ram1,size=64K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel "$KERNEL" \
-initrd "$INITRD" \
-no-reboot \
-append "console=ttyAMA0 cgroup_disable=memory
initcall_blacklist=kswapd_init panic=-1"

Quoted 3 lines

and it was able to repo it. AI identified that kswapd is initializing the pgdat during boot, so blacklisting kswap_init was able to repro the crash when reading /sys/kernel/debug/lru_gen

Thanks!
I guess you mean the following:

(quoted diff) +1 -1
-module_init(kswapd_init)
+//module_init(kswapd_init)
Quoted 5 lines

Yes, I can reproduce it with the above as well.

But kswapd_init() is definitely called before we enter userspace, so this doesn't seem to be an issue here?

Looking at the callee sites of get_lruvec, all of them happen after boot. So it should be fine. Anyway code clean up will address the issue.

Quoted 1 line

So I feel there is no bug here, but this cleanup is still welcome.

Are you ok with the patch as is, or would you recommend changing the callsite of get_lruvec to mem_cgroup_lruvec?

Quoted 4 lines

Am I missing any other scenarios?

Best Regards
Barry

reviewer
#8 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 1 line · Siddharth Chintamaneni

[...]

Quoted 17 lines
Quoted 7 lines

Thanks!
I guess you mean the following:

(quoted diff) +1 -1
-module_init(kswapd_init)
+//module_init(kswapd_init)
Quoted 5 lines

Yes, I can reproduce it with the above as well.

But kswapd_init() is definitely called before we enter userspace, so this doesn't seem to be an issue here?

Looking at the callee sites of get_lruvec, all of them happen after boot. So it should be fine. Anyway code clean up will address the issue.

Quoted 4 lines
Quoted 1 line

So I feel there is no bug here, but this cleanup is still welcome.

Are you ok with the patch as is, or would you recommend changing the callsite of get_lruvec to mem_cgroup_lruvec?

Overall, I'd be happy to see MGLRU-specific code being removed.

If possible, could we split this into a two-patch series? The first patch would be this one, and the second would drop get_lruvec().

In any case, if this isn't a bug fix, there's no need for a Fixes tag or a stable backport.

BTW, I've been struggling to understand why MGLRU introduced its own version in the first place. Maybe your follow-up patchset will help clarify that :-)

Best Regards
Barry

#9 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

Quoted 27 lines · Barry Song
Quoted 1 line · Siddharth Chintamaneni

[...]

Quoted 17 lines
Quoted 7 lines

Thanks!
I guess you mean the following:

(quoted diff) +1 -1
-module_init(kswapd_init)
+//module_init(kswapd_init)
Quoted 5 lines

Yes, I can reproduce it with the above as well.

But kswapd_init() is definitely called before we enter userspace, so this doesn't seem to be an issue here?

Looking at the callee sites of get_lruvec, all of them happen after boot. So it should be fine. Anyway code clean up will address the issue.

Quoted 4 lines
Quoted 1 line

So I feel there is no bug here, but this cleanup is still welcome.

Are you ok with the patch as is, or would you recommend changing the callsite of get_lruvec to mem_cgroup_lruvec?

Quoted 9 lines

Overall, I'd be happy to see MGLRU-specific code being removed.

If possible, could we split this into a two-patch series? The first patch would be this one, and the second would drop get_lruvec().

In any case, if this isn't a bug fix, there's no need for a Fixes tag or a stable backport.

If that is the case, do we still need this patch?

Quoted 4 lines

BTW, I've been struggling to understand why MGLRU introduced its own version in the first place. Maybe your follow-up patchset will help clarify that :-)

I have a theory -
when MGLRU was introduced lru_vec also returns NULL
https://elixir.bootlin.com/linux/v6.1.190/source/mm/vmscan.c#L3163
that is not the case anymore. I haven't looked at the refactoring that happened later around it yet.
That might be the reason but I could be wrong.

Quoted 2 lines

Best Regards
Barry

reviewer
#10 lore raw

Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()

On Tue, Oct 20, 2026 at 9:37 AM Siddharth Chintamaneni <[email protected]> wrote:
[..]

Quoted 3 lines

If that is the case, do we still need this patch?

yes. We need it as cleanup patches. not bug fixes.

Best Regards
Barry

maintainer
#11 lore raw
In reply to Siddharth Chintamaneni
Quoted 10 lines · On Sat, 17 Oct 2026 05:09:27 +0000 Siddharth Chintamaneni…

When CONFIG_MEMCG=y, lruvec_pgdat() returns lruvec->pgdat, which for the per-node pgdat->__lruvec is left NULL by lruvec_init() and only filled in lazily by mem_cgroup_lruvec(). Normally that happens early, e.g. on the first folio_add_lru().

With cgroup_disable=memory, lru_gen_shrink_node() passes &pgdat->__lruvec directly to shrink_one(). If direct reclaim runs before anything has called mem_cgroup_lruvec() for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel), get_swappiness() ends up dereferencing a NULL pgdat:

Thanks, v2 lgtm.

Maintainers: please review and let us know whether you agree with the cc:stable on both.

reviewer
#12 lore raw
In reply to Siddharth Chintamaneni
Quoted 50 lines · Siddharth Chintamaneni

When CONFIG_MEMCG=y, lruvec_pgdat() returns lruvec->pgdat, which for the per-node pgdat->__lruvec is left NULL by lruvec_init() and only filled in lazily by mem_cgroup_lruvec(). Normally that happens early, e.g. on the first folio_add_lru().

With cgroup_disable=memory, lru_gen_shrink_node() passes &pgdat->__lruvec directly to shrink_one(). If direct reclaim runs before anything has called mem_cgroup_lruvec() for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel), get_swappiness() ends up dereferencing a NULL pgdat:

  Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
  pc : get_swappiness+0x68/0xb0
  lr : try_to_shrink_lruvec+0x54/0x2c0
  Call trace:
   get_swappiness+0x68/0xb0 (P)
   try_to_shrink_lruvec+0x54/0x2c0
   shrink_one+0x8c/0x240
   shrink_node+0xc6c/0xe08
   do_try_to_free_pages+0xdc/0x570
   try_to_free_pages+0x124/0x278
   __alloc_frozen_pages_noprof+0x6e8/0xf78
   alloc_pages_mpol+0xc4/0x1c0
   alloc_frozen_pages_noprof+0x54/0xe8
   alloc_pages_noprof+0x1c/0x40
   atomic_pool_expand+0x98/0x300
   __dma_atomic_pool_init+0x58/0xd8
   dma_atomic_pool_init+0x154/0x170
   do_one_initcall+0x64/0x350
   kernel_init_freeable+0x214/0x430
   kernel_init+0x2c/0x208
   ret_from_fork+0x10/0x20

The classic LRU path and MGLRU before per-node lru_gen_folio lists obtained the lruvec through mem_cgroup_lruvec(), which sets lruvec->pgdat.

The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with CONFIG_ARM64_64K_PAGES=y, CONFIG_MEMCG=y, CONFIG_LRU_GEN_ENABLED=y and:

mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory

These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:

Quoted 17 lines
  Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
  pc : get_swappiness+0xd4/0x150
  Call trace:
   get_swappiness+0xd4/0x150 (P)
   try_to_shrink_lruvec+0x44/0x2b8
   shrink_one+0x80/0x1a0
   shrink_node+0xd98/0xea8
   do_try_to_free_pages+0xf4/0x4b0
   try_to_free_pages+0x110/0x360
   __alloc_frozen_pages_noprof+0x8b0/0x1718
   ...
   __filemap_get_folio_mpol+0x2ac/0x548
   simple_write_begin+0x44/0xf8
   page_symlink+0x178/0x198
   ...
   do_populate_rootfs+0x13c/0x200

So this happens before any folios are added to the LRU and before the userspace init process starts?

Quoted 14 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
Assisted-by LLM
---
 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmscan.c +0 -0
index f11491ee9ed5..0c57e4f1eb15 100644

mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5256

Quoted 5 lines
mm/vmscan.c +1 -1
@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control *
                sc->nr_reclaimed = 0;
 
        if (mem_cgroup_disabled())
-               shrink_one(&pgdat->__lruvec, sc);
+               shrink_one(mem_cgroup_lruvec(NULL, pgdat), sc);

What's a bit weird is that even if we fix this here, with an empty lruvec, we still won't be able to reclaim a single page. But it's hard to see where we could break out early, right?

mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5258

Quoted 2 lines
mm/vmscan.c +0 -0
        else
                shrink_many(pgdat, sc);

Best Regards
Barry

#13 lore raw
Quoted 71 lines · Barry Song
Quoted 50 lines · Siddharth Chintamaneni

When CONFIG_MEMCG=y, lruvec_pgdat() returns lruvec->pgdat, which for the per-node pgdat->__lruvec is left NULL by lruvec_init() and only filled in lazily by mem_cgroup_lruvec(). Normally that happens early, e.g. on the first folio_add_lru().

With cgroup_disable=memory, lru_gen_shrink_node() passes &pgdat->__lruvec directly to shrink_one(). If direct reclaim runs before anything has called mem_cgroup_lruvec() for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel), get_swappiness() ends up dereferencing a NULL pgdat:

  Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
  pc : get_swappiness+0x68/0xb0
  lr : try_to_shrink_lruvec+0x54/0x2c0
  Call trace:
   get_swappiness+0x68/0xb0 (P)
   try_to_shrink_lruvec+0x54/0x2c0
   shrink_one+0x8c/0x240
   shrink_node+0xc6c/0xe08
   do_try_to_free_pages+0xdc/0x570
   try_to_free_pages+0x124/0x278
   __alloc_frozen_pages_noprof+0x6e8/0xf78
   alloc_pages_mpol+0xc4/0x1c0
   alloc_frozen_pages_noprof+0x54/0xe8
   alloc_pages_noprof+0x1c/0x40
   atomic_pool_expand+0x98/0x300
   __dma_atomic_pool_init+0x58/0xd8
   dma_atomic_pool_init+0x154/0x170
   do_one_initcall+0x64/0x350
   kernel_init_freeable+0x214/0x430
   kernel_init+0x2c/0x208
   ret_from_fork+0x10/0x20

The classic LRU path and MGLRU before per-node lru_gen_folio lists obtained the lruvec through mem_cgroup_lruvec(), which sets lruvec->pgdat.

The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with CONFIG_ARM64_64K_PAGES=y, CONFIG_MEMCG=y, CONFIG_LRU_GEN_ENABLED=y and:

mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory

These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:

Quoted 17 lines
  Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
  pc : get_swappiness+0xd4/0x150
  Call trace:
   get_swappiness+0xd4/0x150 (P)
   try_to_shrink_lruvec+0x44/0x2b8
   shrink_one+0x80/0x1a0
   shrink_node+0xd98/0xea8
   do_try_to_free_pages+0xf4/0x4b0
   try_to_free_pages+0x110/0x360
   __alloc_frozen_pages_noprof+0x8b0/0x1718
   ...
   __filemap_get_folio_mpol+0x2ac/0x548
   simple_write_begin+0x44/0xf8
   page_symlink+0x178/0x198
   ...
   do_populate_rootfs+0x13c/0x200
Quoted 3 lines

So this happens before any folios are added to the LRU and before the userspace init process starts?

Yes!

Quoted 16 lines
Quoted 16 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
Assisted-by LLM
---
 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmscan.c +0 -0
index f11491ee9ed5..0c57e4f1eb15 100644
@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control *
                sc->nr_reclaimed = 0;
 
Quoted 8 lines

mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5256

Quoted 3 lines
mm/vmscan.c +1 -1
        if (mem_cgroup_disabled())
-               shrink_one(&pgdat->__lruvec, sc);
+               shrink_one(mem_cgroup_lruvec(NULL, pgdat), sc);

What's a bit weird is that even if we fix this here, with an empty lruvec, we still won't be able to reclaim a single page. But it's hard to see where we could break out early, right?

I am not really sure about simple bailout logic. Please let me know if anyone have any thoughts.

Quoted 5 lines

mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5258

Quoted 2 lines
mm/vmscan.c +0 -0
        else
                shrink_many(pgdat, sc);

Best Regards
Barry

reviewer Reviewed-by
#14 lore raw

On Tue, Oct 20, 2026 at 4:37 AM Siddharth Chintamaneni <[email protected]> wrote:
[...]

Quoted 7 lines
Quoted 4 lines

So this happens before any folios are added to the LRU and before the userspace init process starts?

Yes!

Thanks for confirming.

Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
Assisted-by LLM
---
 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmscan.c +0 -0
index f11491ee9ed5..0c57e4f1eb15 100644
@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control *
                sc->nr_reclaimed = 0;
 
Quoted 8 lines

mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5256

Quoted 3 lines
mm/vmscan.c +1 -1
        if (mem_cgroup_disabled())
-               shrink_one(&pgdat->__lruvec, sc);
+               shrink_one(mem_cgroup_lruvec(NULL, pgdat), sc);

What's a bit weird is that even if we fix this here, with an empty lruvec, we still won't be able to reclaim a single page. But it's hard to see where we could break out early, right?

Quoted 2 lines

I am not really sure about simple bailout logic. Please let me know if anyone have any thoughts.

No idea. I also feel it might not be worth doing, as this only affects very uncommon scenarios. Some unnecessary reclaiming is unlikely to hurt real-world workloads, so there's no need to escalate this into a fix.

For the patch,

reviewer Reviewed-by on v2 1/2
#15 lore raw
In reply to Siddharth Chintamaneni
Quoted 66 lines · Siddharth Chintamaneni

When CONFIG_MEMCG=y, lruvec_pgdat() returns lruvec->pgdat, which for the per-node pgdat->__lruvec is left NULL by lruvec_init() and only filled in lazily by mem_cgroup_lruvec(). Normally that happens early, e.g. on the first folio_add_lru().

With cgroup_disable=memory, lru_gen_shrink_node() passes &pgdat->__lruvec directly to shrink_one(). If direct reclaim runs before anything has called mem_cgroup_lruvec() for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel), get_swappiness() ends up dereferencing a NULL pgdat:

   Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
   pc : get_swappiness+0x68/0xb0
   lr : try_to_shrink_lruvec+0x54/0x2c0
   Call trace:
    get_swappiness+0x68/0xb0 (P)
    try_to_shrink_lruvec+0x54/0x2c0
    shrink_one+0x8c/0x240
    shrink_node+0xc6c/0xe08
    do_try_to_free_pages+0xdc/0x570
    try_to_free_pages+0x124/0x278
    __alloc_frozen_pages_noprof+0x6e8/0xf78
    alloc_pages_mpol+0xc4/0x1c0
    alloc_frozen_pages_noprof+0x54/0xe8
    alloc_pages_noprof+0x1c/0x40
    atomic_pool_expand+0x98/0x300
    __dma_atomic_pool_init+0x58/0xd8
    dma_atomic_pool_init+0x154/0x170
    do_one_initcall+0x64/0x350
    kernel_init_freeable+0x214/0x430
    kernel_init+0x2c/0x208
    ret_from_fork+0x10/0x20

The classic LRU path and MGLRU before per-node lru_gen_folio lists obtained the lruvec through mem_cgroup_lruvec(), which sets lruvec->pgdat.

The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with CONFIG_ARM64_64K_PAGES=y, CONFIG_MEMCG=y, CONFIG_LRU_GEN_ENABLED=y and:

mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory

These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:

   Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
   pc : get_swappiness+0xd4/0x150
   Call trace:
    get_swappiness+0xd4/0x150 (P)
    try_to_shrink_lruvec+0x44/0x2b8
    shrink_one+0x80/0x1a0
    shrink_node+0xd98/0xea8
    do_try_to_free_pages+0xf4/0x4b0
    try_to_free_pages+0x110/0x360
    __alloc_frozen_pages_noprof+0x8b0/0x1718
    ...
    __filemap_get_folio_mpol+0x2ac/0x548
    simple_write_begin+0x44/0xf8
    page_symlink+0x178/0x198
    ...
    do_populate_rootfs+0x13c/0x200
Quoted 6 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
Assisted-by LLM

---

Good catch. LGTM.