mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline
Subsystems (4)
PATCH v2 1/2 mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline
When CONFIG_MEMCG=y, lruvec_pgdat() returns lruvec->pgdat, which for the per-node pgdat->__lruvec is left NULL by lruvec_init() and only filled in lazily by mem_cgroup_lruvec(). Normally that happens early, e.g. on the first folio_add_lru().
With cgroup_disable=memory, lru_gen_shrink_node() passes &pgdat->__lruvec directly to shrink_one(). If direct reclaim runs before anything has called mem_cgroup_lruvec() for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel), get_swappiness() ends up dereferencing a NULL pgdat:
Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0x68/0xb0 lr : try_to_shrink_lruvec+0x54/0x2c0 Call trace: get_swappiness+0x68/0xb0 (P) try_to_shrink_lruvec+0x54/0x2c0 shrink_one+0x8c/0x240 shrink_node+0xc6c/0xe08 do_try_to_free_pages+0xdc/0x570 try_to_free_pages+0x124/0x278 __alloc_frozen_pages_noprof+0x6e8/0xf78 alloc_pages_mpol+0xc4/0x1c0 alloc_frozen_pages_noprof+0x54/0xe8 alloc_pages_noprof+0x1c/0x40 atomic_pool_expand+0x98/0x300 __dma_atomic_pool_init+0x58/0xd8 dma_atomic_pool_init+0x154/0x170 do_one_initcall+0x64/0x350 kernel_init_freeable+0x214/0x430 kernel_init+0x2c/0x208 ret_from_fork+0x10/0x20
The classic LRU path and MGLRU before per-node lru_gen_folio lists obtained the lruvec through mem_cgroup_lruvec(), which sets lruvec->pgdat.
The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with CONFIG_ARM64_64K_PAGES=y, CONFIG_MEMCG=y, CONFIG_LRU_GEN_ENABLED=y and:
mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory
These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:
Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0xd4/0x150 Call trace: get_swappiness+0xd4/0x150 (P) try_to_shrink_lruvec+0x44/0x2b8 shrink_one+0x80/0x1a0 shrink_node+0xd98/0xea8 do_try_to_free_pages+0xf4/0x4b0 try_to_free_pages+0x110/0x360 __alloc_frozen_pages_noprof+0x8b0/0x1718 ... __filemap_get_folio_mpol+0x2ac/0x548 simple_write_begin+0x44/0xf8 page_symlink+0x178/0x198 ... do_populate_rootfs+0x13c/0x200
--- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmscan.c +1 -1
@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * sc->nr_reclaimed =0 ;if (mem_cgroup_disabled ()) -shrink_one (&pgdat->__lruvec, sc); +shrink_one (mem_cgroup_lruvec (NULL , pgdat), sc);else shrink_many (pgdat, sc);
Signature · 1 line
2.45.4
PATCH v2 2/2 mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Andrew Morton reported that get_lruvec() has the same problem as lru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns &pgdat->__lruvec directly and so bypasses the initialization of lruvec->pgdat. The MGLRU debugfs interface calls get_lruvec(), then dereferences the pgdat pointer in lru_gen_seq_show() and other paths, so these can hit the same NULL dereference.
--- mm/vmscan.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
mm/vmscan.c +2 -13
@@ -2745,20 +2745,9 @@ static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) {struct pglist_data *pgdat =NODE_DATA (nid); -#ifdef CONFIG_MEMCG -if (memcg) { -struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec; - -/* see the comment in mem_cgroup_lruvec() */ -if (!lruvec->pgdat) - lruvec->pgdat = pgdat; - -return lruvec; - } -#endif -VM_WARN_ON_ONCE (!mem_cgroup_disabled ()); +VM_WARN_ON_ONCE (!memcg && !mem_cgroup_disabled ()); -return &pgdat->__lruvec; +return mem_cgroup_lruvec (memcg, pgdat); }static int get_swappiness (struct lruvec *lruvec,struct scan_control *sc)
Signature · 1 line
2.45.4
Barry Song
reviewerRe: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 8 lines · Siddharth Chintamaneni
Andrew Morton reported that
get_lruvec()has the same problem aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly and so bypasses the initialization oflruvec->pgdat. The MGLRU debugfs interface callsget_lruvec(), then dereferences the pgdat pointer inlru_gen_seq_show()and other paths, so these can hit the same NULL dereference.
So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?
Quoted 25 lines
Fixes ec1c86b25f4b ("mm: multi-gen LRU: groundwork")Cc [email protected] # 6.2+Assisted-by LLMSigned-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>--- mm/vmscan.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)mm/vmscan.c +0
-7@@ -2745,20 +2745,9 @@ static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) {struct pglist_data *pgdat =NODE_DATA (nid); -#ifdef CONFIG_MEMCG -if (memcg) { -struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec; - -/* see the comment in mem_cgroup_lruvec() */ -if (!lruvec->pgdat) - lruvec->pgdat = pgdat;
mm/vmscan.c static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) · line 2751
Quoted 10 lines
mm/vmscan.c +2
-6- - return lruvec; - } -#endif -VM_WARN_ON_ONCE (!mem_cgroup_disabled ()); +VM_WARN_ON_ONCE (!memcg && !mem_cgroup_disabled ()); -return &pgdat->__lruvec; +return mem_cgroup_lruvec (memcg, pgdat); }
I've been struggling to understand why MGLRU needs its own get_lruvec() when mem_cgroup_lruvec() already exists. With your patch, it feels like we're one step closer to getting rid of MGLRU's own variant in the future.
Quoted 4 lines
mm/vmscan.c +0
-1static int get_swappiness (struct lruvec *lruvec,struct scan_control *sc) --2.45.4
Best Regards
Barry
Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni
Andrew Morton reported that
get_lruvec()has the same problem aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly and so bypasses the initialization oflruvec->pgdat. The MGLRU debugfs interface callsget_lruvec(), then dereferences the pgdat pointer inlru_gen_seq_show()and other paths, so these can hit the same NULL dereference.
Quoted 5 lines
So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?
That is my high level understanding. I'll let Andrew to comment if I am missing something,
Quoted 32 lines
Quoted 32 lines
Fixes ec1c86b25f4b ("mm: multi-gen LRU: groundwork")Cc [email protected] # 6.2+Assisted-by LLMSigned-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>--- mm/vmscan.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)mm/vmscan.c +1
-12@@ -2745,20 +2745,9 @@ static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) {struct pglist_data *pgdat =NODE_DATA (nid); -#ifdef CONFIG_MEMCG -if (memcg) { -struct lruvec *lruvec = &memcg->nodeinfo[nid]->lruvec; - -/* see the comment in mem_cgroup_lruvec() */ -if (!lruvec->pgdat) - lruvec->pgdat = pgdat; - -return lruvec; - } -#endif -VM_WARN_ON_ONCE (!mem_cgroup_disabled ()); +VM_WARN_ON_ONCE (!memcg && !mem_cgroup_disabled ());
Quoted 9 lines
mm/vmscan.c static struct lruvec *get_lruvec(struct mem_cgroup *memcg, int nid) · line 2751
Quoted 3 lines
mm/vmscan.c +1
-1-return &pgdat->__lruvec; +return mem_cgroup_lruvec (memcg, pgdat); }I've been struggling to understand why MGLRU needs its own
get_lruvec()whenmem_cgroup_lruvec()already exists. With your patch, it feels like we're one step closer to getting rid of MGLRU's own variant in the future.
I agree with that! I'm happy to send a follow-up cleanup patch to remove get_lruvec if that makes sense. Not sure how it will affect stable backports.
Quoted 7 lines
Quoted 4 lines
mm/vmscan.c +0
-1static int get_swappiness (struct lruvec *lruvec,struct scan_control *sc) --2.45.4
Best Regards
Barry
Barry Song
reviewerRe: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni
Andrew Morton reported that
get_lruvec()has the same problem aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly and so bypasses the initialization oflruvec->pgdat. The MGLRU debugfs interface callsget_lruvec(), then dereferences the pgdat pointer inlru_gen_seq_show()and other paths, so these can hit the same NULL dereference.
Quoted 9 lines
Quoted 5 lines
So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?
That is my high level understanding. I'll let Andrew to comment if I am missing something,
I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:
qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 \
-m 1G \
-object memory-backend-ram,id=ram0,size=1048572K \
-object memory-backend-ram,id=ram1,size=4K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel arch/arm64/boot/Image \
-nographic -append "cgroup_disable=memory
...
Then I got a machine where node1 is very difficult to have a page in lru.
then i ran the debugfs for mglru:
/ # cat /sys/kernel/debug/lru_gen
memcg 0
node 0
0 14972 0 953
1 14972 0 1
2 14972 18 3308
3 14972 0 0
node 1
0 14972 0 0
1 14972 0 0
2 14972 0 0
3 14972 0 0now you see node 1 has no page in lru but yet i didn't have any crash.
i think i will need more time to figure out what is the problem and how to reproduce it.
Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 58 lines · Barry Song
Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni
Andrew Morton reported that
get_lruvec()has the same problem aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly and so bypasses the initialization oflruvec->pgdat. The MGLRU debugfs interface callsget_lruvec(), then dereferences the pgdat pointer inlru_gen_seq_show()and other paths, so these can hit the same NULL dereference.Quoted 9 lines
Quoted 5 lines
So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?
That is my high level understanding. I'll let Andrew to comment if I am missing something,
I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:
qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
(quoted diff) +0
-6-smp 4 \ -m 1G \ -object memory-backend-ram,id=ram0,size=1048572K \ -object memory-backend-ram,id=ram1,size=4K \ -numa node,nodeid=0,memdev=ram0,cpus=0-1 \ -numa node,nodeid=1,memdev=ram1,cpus=2-3 \ -kernel arch/arm64/boot/Image \ -nographic -append "cgroup_disable=memoryThen I got a machine where node1 is very difficult to have a page in lru.
then i ran the debugfs for mglru:
/ # cat
/sys/kernel/debug/lru_gen
memcg 0
node 00 14972 0 953 1 14972 0 1 2 14972 18 3308 3 14972 0 0 node 1 0 14972 0 0 1 14972 0 0 2 14972 0 0 3 14972 0 0
Quoted 4 lines
now you see node 1 has no page in lru but yet i didn't have any crash.
i think i will need more time to figure out what is the problem and how to reproduce it.
I've asked AI to repro the issue with the similar setup
qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 -m 1G \
-object memory-backend-ram,id=ram0,size=1048512K \
-object memory-backend-ram,id=ram1,size=64K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel "$KERNEL" \
-initrd "$INITRD" \
-no-reboot \
-append "console=ttyAMA0 cgroup_disable=memory
initcall_blacklist=kswapd_init panic=-1"
and it was able to repo it. AI identified that kswapd is initializing the pgdat during boot, so blacklisting kswap_init was able to repro the crash when reading /sys/kernel/debug/lru_gen
[ 1.347868] Unable to handle kernel access to user memory outside uaccess routines at virtual address 0000000000005040 [ 1.347920] Mem abort info: [ 1.347929] ESR = 0x0000000096000005 [ 1.347950] EC = 0x25: DABT (current EL), IL = 32 bits [ 1.347964] SET = 0, FnV = 0 [ 1.347974] EA = 0, S1PTW = 0 [ 1.347986] FSC = 0x05: level 1 translation fault [ 1.348249] Data abort info: [ 1.348277] ISV = 0, ISS = 0x00000005, ISS2 = 0x00000000 [ 1.348291] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 [ 1.348302] GCS = 0, Overlay = 0, DirtyBit = 0 [ 1.348345] user pgtable: 64k pages, 48-bit VAs, pgdp=00000000485f6600 [ 1.348369] [0000000000005040] pgd=0800000043640403, p4d=0800000043640403, pud=0800000043640403, pmd=0800000043620403, pte=0000000000000000 [ 1.349148] Internal error: Oops: 0000000096000005 [#1] SMP [ 1.350414] Modules linked in: [ 1.350896] CPU: 0 UID: 0 PID: 1 Comm: init Not tainted 7.3.0-rc7+ #9 PREEMPT(full) [ 1.351111] Hardware name: linux,dummy-virt (DT) [ 1.351411] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1.351581] pc : lru_gen_seq_show+0x5c/0x460 [ 1.351800] lr : lru_gen_seq_show+0x4c/0x460 [ 1.351898] sp : ffff80008034f9a0 [ 1.351975] x29: ffff80008034fa20 x28: ffff0000035e67e8 x27: 0000000000400cc0 [ 1.352297] x26: 000000007fff0000 x25: ffff0000035e67d8 x24: 0000000000000000 [ 1.352454] x23: ffff80008034fb00 x22: ffff80008034fb28 x21: 0000000000000000 [ 1.352626] x20: ffff0000035e67b0 x19: ffff00003ffc2f20 x18: ffff800080360060 [ 1.352775] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 1.352915] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 [ 1.353058] x11: 0000000000000000 x10: 0000000000000000 x9 : ffffa3468a020c5c [ 1.353226] x8 : ffff000008b07000 x7 : 0000000000000000 x6 : 000000000000003f [ 1.353367] x5 : 00000000ffffffff x4 : 0000000000000003 x3 : 0000000000001000 [ 1.353511] x2 : ffff00003ffc2f20 x1 : 0000000000004000 x0 : 0000000000000000 [ 1.353732] Call trace: [ 1.353925] lru_gen_seq_show+0x5c/0x460 (P) [ 1.354110] seq_read_iter+0xec/0x4c0 [ 1.354213] seq_read+0xfc/0x148 [ 1.354290] full_proxy_read+0x78/0xc0 [ 1.354377] vfs_read+0xd0/0x338 [ 1.354453] ksys_read+0x7c/0x120 [ 1.354528] __arm64_sys_read+0x24/0x40 [ 1.354611] invoke_syscall+0xa8/0x120 [ 1.354695] el0_svc_common.constprop.0+0x4c/0x100 [ 1.354791] do_el0_svc+0x28/0x40 [ 1.354865] el0_svc+0x48/0x2b0 [ 1.354939] el0t_64_sync_handler+0xa8/0xf0 [ 1.355026] el0t_64_sync+0x1b8/0x1c0 [ 1.355250] Code: f9000fe0 f9432661 aa1303e2 91401021 (b9504033) [ 1.355647] ---[ end trace 0000000000000000 ]--- [ 1.356555] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b [ 1.356888] SMP: stopping secondary CPUs [ 1.357525] Kernel Offset: 0x234609c00000 from 0xffff800080000000 [ 1.357675] PHYS_OFFSET: 0x40000000 [ 1.357762] CPU features: 0x0,00000000,09000020,00100002,0800821b [ 1.357901] Memory Limit: none
Barry Song
reviewerRe: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 79 lines · Siddharth Chintamaneni
Quoted 58 lines · Barry Song
Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni
Andrew Morton reported that
get_lruvec()has the same problem aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly and so bypasses the initialization oflruvec->pgdat. The MGLRU debugfs interface callsget_lruvec(), then dereferences the pgdat pointer inlru_gen_seq_show()and other paths, so these can hit the same NULL dereference.Quoted 9 lines
Quoted 5 lines
So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?
That is my high level understanding. I'll let Andrew to comment if I am missing something,
I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:
qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
(quoted diff) +0
-6-smp 4 \ -m 1G \ -object memory-backend-ram,id=ram0,size=1048572K \ -object memory-backend-ram,id=ram1,size=4K \ -numa node,nodeid=0,memdev=ram0,cpus=0-1 \ -numa node,nodeid=1,memdev=ram1,cpus=2-3 \ -kernel arch/arm64/boot/Image \ -nographic -append "cgroup_disable=memoryThen I got a machine where node1 is very difficult to have a page in lru.
then i ran the debugfs for mglru:
/ # cat
/sys/kernel/debug/lru_gen
memcg 0
node 00 14972 0 953 1 14972 0 1 2 14972 18 3308 3 14972 0 0 node 1 0 14972 0 0 1 14972 0 0 2 14972 0 0 3 14972 0 0Quoted 4 lines
now you see node 1 has no page in lru but yet i didn't have any crash.
i think i will need more time to figure out what is the problem and how to reproduce it.
I've asked AI to repro the issue with the similar setup
qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 -m 1G \
-object memory-backend-ram,id=ram0,size=1048512K \
-object memory-backend-ram,id=ram1,size=64K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel "$KERNEL" \
-initrd "$INITRD" \
-no-reboot \
-append "console=ttyAMA0 cgroup_disable=memory
initcall_blacklist=kswapd_init panic=-1"
Quoted 3 lines
and it was able to repo it. AI identified that kswapd is initializing the pgdat during boot, so blacklisting
kswap_initwas able to repro the crash when reading/sys/kernel/debug/lru_gen
Thanks!
I guess you mean the following:
-module_init(kswapd_init)
+//module_init(kswapd_init)
Yes, I can reproduce it with the above as well.
But kswapd_init() is definitely called before we enter userspace, so this doesn't seem to be an issue here?
So I feel there is no bug here, but this cleanup is still welcome. Am I missing any other scenarios?
Best Regards
Barry
Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 92 lines · Barry Song
Quoted 79 lines · Siddharth Chintamaneni
Quoted 58 lines · Barry Song
Quoted 14 lines · Siddharth Chintamaneni
Quoted 12 lines · Barry Song
Quoted 8 lines · Siddharth Chintamaneni
Andrew Morton reported that
get_lruvec()has the same problem aslru_gen_shrink_node(): when memcg is disabled on the command line (cgroup_disable=memory), it returns&pgdat->__lruvecdirectly and so bypasses the initialization oflruvec->pgdat. The MGLRU debugfs interface callsget_lruvec(), then dereferences the pgdat pointer inlru_gen_seq_show()and other paths, so these can hit the same NULL dereference.Quoted 9 lines
Quoted 5 lines
So userspace must have booted, since otherwise we couldn't read debugfs. The reachable case is therefore a memory node with no LRU folios when debugfs is read? So this could happen on a multi-node system where folios are landed on other nodes but not on this node?
That is my high level understanding. I'll let Andrew to comment if I am missing something,
I made a silly toy setup by running two numa nodes with one of them has only 4KB memory, the other one has 1G-4K:
qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
(quoted diff) +0
-6-smp 4 \ -m 1G \ -object memory-backend-ram,id=ram0,size=1048572K \ -object memory-backend-ram,id=ram1,size=4K \ -numa node,nodeid=0,memdev=ram0,cpus=0-1 \ -numa node,nodeid=1,memdev=ram1,cpus=2-3 \ -kernel arch/arm64/boot/Image \ -nographic -append "cgroup_disable=memoryThen I got a machine where node1 is very difficult to have a page in lru.
then i ran the debugfs for mglru:
/ # cat
/sys/kernel/debug/lru_gen
memcg 0
node 00 14972 0 953 1 14972 0 1 2 14972 18 3308 3 14972 0 0 node 1 0 14972 0 0 1 14972 0 0 2 14972 0 0 3 14972 0 0Quoted 4 lines
now you see node 1 has no page in lru but yet i didn't have any crash.
i think i will need more time to figure out what is the problem and how to reproduce it.
I've asked AI to repro the issue with the similar setup
qemu-system-aarch64 -M virt,gic-version=3,mte=off -cpu cortex-a53 -nographic \
-smp 4 -m 1G \
-object memory-backend-ram,id=ram0,size=1048512K \
-object memory-backend-ram,id=ram1,size=64K \
-numa node,nodeid=0,memdev=ram0,cpus=0-1 \
-numa node,nodeid=1,memdev=ram1,cpus=2-3 \
-kernel "$KERNEL" \
-initrd "$INITRD" \
-no-reboot \
-append "console=ttyAMA0 cgroup_disable=memory
initcall_blacklist=kswapd_init panic=-1"Quoted 3 lines
and it was able to repo it. AI identified that kswapd is initializing the pgdat during boot, so blacklisting
kswap_initwas able to repro the crash when reading/sys/kernel/debug/lru_genThanks!
I guess you mean the following:(quoted diff) +1
-1-module_init(kswapd_init) +//module_init(kswapd_init)
Quoted 5 lines
Yes, I can reproduce it with the above as well.
But
kswapd_init()is definitely called before we enter userspace, so this doesn't seem to be an issue here?
Looking at the callee sites of get_lruvec, all of them happen after boot. So it should be fine. Anyway code clean up will address the issue.
Quoted 1 line
So I feel there is no bug here, but this cleanup is still welcome.
Are you ok with the patch as is, or would you recommend changing the callsite of get_lruvec to mem_cgroup_lruvec?
Quoted 4 lines
Am I missing any other scenarios?
Best Regards
Barry
Barry Song
reviewerRe: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 1 line · Siddharth Chintamaneni
[...]
Quoted 17 lines
Quoted 7 lines
Thanks!
I guess you mean the following:(quoted diff) +1
-1-module_init(kswapd_init) +//module_init(kswapd_init)Quoted 5 lines
Yes, I can reproduce it with the above as well.
But
kswapd_init()is definitely called before we enter userspace, so this doesn't seem to be an issue here?Looking at the callee sites of
get_lruvec, all of them happen after boot. So it should be fine. Anyway code clean up will address the issue.
Quoted 4 lines
Quoted 1 line
So I feel there is no bug here, but this cleanup is still welcome.
Are you ok with the patch as is, or would you recommend changing the callsite of
get_lruvectomem_cgroup_lruvec?
Overall, I'd be happy to see MGLRU-specific code being removed.
If possible, could we split this into a two-patch series? The first patch would be this one, and the second would drop get_lruvec().
In any case, if this isn't a bug fix, there's no need for a Fixes tag or a stable backport.
BTW, I've been struggling to understand why MGLRU introduced its own version in the first place. Maybe your follow-up patchset will help clarify that :-)
Best Regards
Barry
Re: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
Quoted 27 lines · Barry Song
Quoted 1 line · Siddharth Chintamaneni
[...]
Quoted 17 lines
Quoted 7 lines
Thanks!
I guess you mean the following:(quoted diff) +1
-1-module_init(kswapd_init) +//module_init(kswapd_init)Quoted 5 lines
Yes, I can reproduce it with the above as well.
But
kswapd_init()is definitely called before we enter userspace, so this doesn't seem to be an issue here?Looking at the callee sites of
get_lruvec, all of them happen after boot. So it should be fine. Anyway code clean up will address the issue.Quoted 4 lines
Quoted 1 line
So I feel there is no bug here, but this cleanup is still welcome.
Are you ok with the patch as is, or would you recommend changing the callsite of
get_lruvectomem_cgroup_lruvec?
Quoted 9 lines
Overall, I'd be happy to see MGLRU-specific code being removed.
If possible, could we split this into a two-patch series? The first patch would be this one, and the second would drop
get_lruvec().In any case, if this isn't a bug fix, there's no need for a Fixes tag or a stable backport.
If that is the case, do we still need this patch?
Quoted 4 lines
BTW, I've been struggling to understand why MGLRU introduced its own version in the first place. Maybe your follow-up patchset will help clarify that :-)
I have a theory -
when MGLRU was introduced lru_vec also returns NULL
https://elixir.bootlin.com/linux/v6.1.190/source/mm/vmscan.c#L3163
that is not the case anymore. I haven't looked at the refactoring that happened later around it yet.
That might be the reason but I could be wrong.
Quoted 2 lines
Best Regards
Barry
Barry Song
reviewerRe: [PATCH v2 2/2] mm: multi-gen LRU: fix NULL lruvec->pgdat deref in get_lruvec()
On Tue, Oct 20, 2026 at 9:37 AM Siddharth Chintamaneni <[email protected]> wrote:
[..]
Quoted 3 lines
If that is the case, do we still need this patch?
yes. We need it as cleanup patches. not bug fixes.
Best Regards
Barry
Andrew Morton
maintainerQuoted 10 lines · On Sat, 17 Oct 2026 05:09:27 +0000 Siddharth Chintamaneni…
When
CONFIG_MEMCG=y,lruvec_pgdat()returnslruvec->pgdat, which for the per-nodepgdat->__lruvecis left NULL bylruvec_init()and only filled in lazily bymem_cgroup_lruvec(). Normally that happens early, e.g. on the firstfolio_add_lru().With cgroup_disable=memory,
lru_gen_shrink_node()passes&pgdat->__lruvecdirectly toshrink_one(). If direct reclaim runs before anything has calledmem_cgroup_lruvec()for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel),get_swappiness()ends up dereferencing a NULL pgdat:
Thanks, v2 lgtm.
Maintainers: please review and let us know whether you agree with the cc:stable on both.
Barry Song
reviewerQuoted 50 lines · Siddharth Chintamaneni
When
CONFIG_MEMCG=y,lruvec_pgdat()returnslruvec->pgdat, which for the per-nodepgdat->__lruvecis left NULL bylruvec_init()and only filled in lazily bymem_cgroup_lruvec(). Normally that happens early, e.g. on the firstfolio_add_lru().With cgroup_disable=memory,
lru_gen_shrink_node()passes&pgdat->__lruvecdirectly toshrink_one(). If direct reclaim runs before anything has calledmem_cgroup_lruvec()for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel),get_swappiness()ends up dereferencing a NULL pgdat:Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0x68/0xb0 lr : try_to_shrink_lruvec+0x54/0x2c0 Call trace: get_swappiness+0x68/0xb0 (P) try_to_shrink_lruvec+0x54/0x2c0 shrink_one+0x8c/0x240 shrink_node+0xc6c/0xe08 do_try_to_free_pages+0xdc/0x570 try_to_free_pages+0x124/0x278 __alloc_frozen_pages_noprof+0x6e8/0xf78 alloc_pages_mpol+0xc4/0x1c0 alloc_frozen_pages_noprof+0x54/0xe8 alloc_pages_noprof+0x1c/0x40 atomic_pool_expand+0x98/0x300 __dma_atomic_pool_init+0x58/0xd8 dma_atomic_pool_init+0x154/0x170 do_one_initcall+0x64/0x350 kernel_init_freeable+0x214/0x430 kernel_init+0x2c/0x208 ret_from_fork+0x10/0x20The classic LRU path and MGLRU before per-node
lru_gen_foliolists obtained the lruvec throughmem_cgroup_lruvec(), which setslruvec->pgdat.The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with
CONFIG_ARM64_64K_PAGES=y,CONFIG_MEMCG=y,CONFIG_LRU_GEN_ENABLED=yand:mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory
These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:
Quoted 17 lines
Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0xd4/0x150 Call trace: get_swappiness+0xd4/0x150 (P) try_to_shrink_lruvec+0x44/0x2b8 shrink_one+0x80/0x1a0 shrink_node+0xd98/0xea8 do_try_to_free_pages+0xf4/0x4b0 try_to_free_pages+0x110/0x360 __alloc_frozen_pages_noprof+0x8b0/0x1718 ... __filemap_get_folio_mpol+0x2ac/0x548 simple_write_begin+0x44/0xf8 page_symlink+0x178/0x198 ... do_populate_rootfs+0x13c/0x200
So this happens before any folios are added to the LRU and before the userspace init process starts?
Quoted 14 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")Cc [email protected] # 6.3+Assisted-by LLMSigned-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>--- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/vmscan.c +0
-0
mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5256
Quoted 5 lines
mm/vmscan.c +1
-1@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * sc->nr_reclaimed =0 ;if (mem_cgroup_disabled ()) -shrink_one (&pgdat->__lruvec, sc); +shrink_one (mem_cgroup_lruvec (NULL , pgdat), sc);
What's a bit weird is that even if we fix this here, with an empty lruvec, we still won't be able to reclaim a single page. But it's hard to see where we could break out early, right?
mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5258
Quoted 2 lines
mm/vmscan.c +0
-0else shrink_many (pgdat, sc);
Best Regards
Barry
Quoted 71 lines · Barry Song
Quoted 50 lines · Siddharth Chintamaneni
When
CONFIG_MEMCG=y,lruvec_pgdat()returnslruvec->pgdat, which for the per-nodepgdat->__lruvecis left NULL bylruvec_init()and only filled in lazily bymem_cgroup_lruvec(). Normally that happens early, e.g. on the firstfolio_add_lru().With cgroup_disable=memory,
lru_gen_shrink_node()passes&pgdat->__lruvecdirectly toshrink_one(). If direct reclaim runs before anything has calledmem_cgroup_lruvec()for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel),get_swappiness()ends up dereferencing a NULL pgdat:Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0x68/0xb0 lr : try_to_shrink_lruvec+0x54/0x2c0 Call trace: get_swappiness+0x68/0xb0 (P) try_to_shrink_lruvec+0x54/0x2c0 shrink_one+0x8c/0x240 shrink_node+0xc6c/0xe08 do_try_to_free_pages+0xdc/0x570 try_to_free_pages+0x124/0x278 __alloc_frozen_pages_noprof+0x6e8/0xf78 alloc_pages_mpol+0xc4/0x1c0 alloc_frozen_pages_noprof+0x54/0xe8 alloc_pages_noprof+0x1c/0x40 atomic_pool_expand+0x98/0x300 __dma_atomic_pool_init+0x58/0xd8 dma_atomic_pool_init+0x154/0x170 do_one_initcall+0x64/0x350 kernel_init_freeable+0x214/0x430 kernel_init+0x2c/0x208 ret_from_fork+0x10/0x20The classic LRU path and MGLRU before per-node
lru_gen_foliolists obtained the lruvec throughmem_cgroup_lruvec(), which setslruvec->pgdat.The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with
CONFIG_ARM64_64K_PAGES=y,CONFIG_MEMCG=y,CONFIG_LRU_GEN_ENABLED=yand:mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory
These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:
Quoted 17 lines
Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0xd4/0x150 Call trace: get_swappiness+0xd4/0x150 (P) try_to_shrink_lruvec+0x44/0x2b8 shrink_one+0x80/0x1a0 shrink_node+0xd98/0xea8 do_try_to_free_pages+0xf4/0x4b0 try_to_free_pages+0x110/0x360 __alloc_frozen_pages_noprof+0x8b0/0x1718 ... __filemap_get_folio_mpol+0x2ac/0x548 simple_write_begin+0x44/0xf8 page_symlink+0x178/0x198 ... do_populate_rootfs+0x13c/0x200
Quoted 3 lines
So this happens before any folios are added to the LRU and before the userspace init process starts?
Yes!
Quoted 16 lines
Quoted 16 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")Cc [email protected] # 6.3+Assisted-by LLMSigned-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>--- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/vmscan.c +0
-0@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * sc->nr_reclaimed =0 ;
Quoted 8 lines
mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5256
Quoted 3 lines
mm/vmscan.c +1
-1if (mem_cgroup_disabled ()) -shrink_one (&pgdat->__lruvec, sc); +shrink_one (mem_cgroup_lruvec (NULL , pgdat), sc);What's a bit weird is that even if we fix this here, with an empty lruvec, we still won't be able to reclaim a single page. But it's hard to see where we could break out early, right?
I am not really sure about simple bailout logic. Please let me know if anyone have any thoughts.
Quoted 5 lines
mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5258
Quoted 2 lines
mm/vmscan.c +0
-0else shrink_many (pgdat, sc);Best Regards
Barry
On Tue, Oct 20, 2026 at 4:37 AM Siddharth Chintamaneni <[email protected]> wrote:
[...]
Quoted 7 lines
Quoted 4 lines
So this happens before any folios are added to the LRU and before the userspace init process starts?
Yes!
Thanks for confirming.
Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")Cc [email protected] # 6.3+Assisted-by LLMSigned-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>--- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/vmscan.c +0
-0@@ -5253,7 +5253,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * sc->nr_reclaimed =0 ;Quoted 8 lines
mm/vmscan.c static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control * · line 5256
Quoted 3 lines
mm/vmscan.c +1
-1if (mem_cgroup_disabled ()) -shrink_one (&pgdat->__lruvec, sc); +shrink_one (mem_cgroup_lruvec (NULL , pgdat), sc);What's a bit weird is that even if we fix this here, with an empty lruvec, we still won't be able to reclaim a single page. But it's hard to see where we could break out early, right?
Quoted 2 lines
I am not really sure about simple bailout logic. Please let me know if anyone have any thoughts.
No idea. I also feel it might not be worth doing, as this only affects very uncommon scenarios. Some unnecessary reclaiming is unlikely to hurt real-world workloads, so there's no need to escalate this into a fix.
For the patch,
Quoted 66 lines · Siddharth Chintamaneni
When
CONFIG_MEMCG=y,lruvec_pgdat()returnslruvec->pgdat, which for the per-nodepgdat->__lruvecis left NULL bylruvec_init()and only filled in lazily bymem_cgroup_lruvec(). Normally that happens early, e.g. on the firstfolio_add_lru().With cgroup_disable=memory,
lru_gen_shrink_node()passes&pgdat->__lruvecdirectly toshrink_one(). If direct reclaim runs before anything has calledmem_cgroup_lruvec()for the node, which can happen during early boot under memory pressure (e.g. a kdump kernel),get_swappiness()ends up dereferencing a NULL pgdat:Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0x68/0xb0 lr : try_to_shrink_lruvec+0x54/0x2c0 Call trace: get_swappiness+0x68/0xb0 (P) try_to_shrink_lruvec+0x54/0x2c0 shrink_one+0x8c/0x240 shrink_node+0xc6c/0xe08 do_try_to_free_pages+0xdc/0x570 try_to_free_pages+0x124/0x278 __alloc_frozen_pages_noprof+0x6e8/0xf78 alloc_pages_mpol+0xc4/0x1c0 alloc_frozen_pages_noprof+0x54/0xe8 alloc_pages_noprof+0x1c/0x40 atomic_pool_expand+0x98/0x300 __dma_atomic_pool_init+0x58/0xd8 dma_atomic_pool_init+0x154/0x170 do_one_initcall+0x64/0x350 kernel_init_freeable+0x214/0x430 kernel_init+0x2c/0x208 ret_from_fork+0x10/0x20The classic LRU path and MGLRU before per-node
lru_gen_foliolists obtained the lruvec throughmem_cgroup_lruvec(), which setslruvec->pgdat.The above was hit on an arm64 kdump kernel (6.18-based) booted with cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an arm64 VM with
CONFIG_ARM64_64K_PAGES=y,CONFIG_MEMCG=y,CONFIG_LRU_GEN_ENABLED=yand:mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory
These options only serve to force direct reclaim early in boot; with 160M the boot eventually ends in OOM regardless. On mainline the first LRU reclaim happens while unpacking the initramfs, and faults the same way:
Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040 pc : get_swappiness+0xd4/0x150 Call trace: get_swappiness+0xd4/0x150 (P) try_to_shrink_lruvec+0x44/0x2b8 shrink_one+0x80/0x1a0 shrink_node+0xd98/0xea8 do_try_to_free_pages+0xf4/0x4b0 try_to_free_pages+0x110/0x360 __alloc_frozen_pages_noprof+0x8b0/0x1718 ... __filemap_get_folio_mpol+0x2ac/0x548 simple_write_begin+0x44/0xf8 page_symlink+0x178/0x198 ... do_populate_rootfs+0x13c/0x200
Quoted 6 lines
Fixes e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")Cc [email protected] # 6.3+Assisted-by LLMSigned-off-by Siddharth Chintamaneni <[email protected]>Signed-off-by Hardik Garg <[email protected]>---
Good catch. LGTM.