mm/lruvec: preemptively free dead folios during lru_add drain
PATCH 1/1 mm/lruvec: preemptively free dead folios during lru_add drain
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in lru_add batches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed via folios_put_refs(), incurring two unnecessary lock acquisitions.
Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use folio_ref_freeze() to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.
During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the mm_lru_insertion tracepoint shows the effectiveness of the patch:
Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)
dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%
Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
--- mm/swap.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-)
mm/swap.c +35 -1
@@ -160,13 +160,36 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)int i;struct lruvec *lruvec =NULL ;unsigned long flags =0 ; +struct folio_batch free_fbatch; +bool is_lru_add = (move_fn == lru_add); + +/* +* If we're adding to the LRU, preemptively filter dead folios. Use +* this dedicated folio batch for temp storage and deferred cleanup. +*/ +if (is_lru_add) +folio_batch_init (&free_fbatch);for (i =0 ; i <folio_batch_count (fbatch); i++) {struct folio *folio = fbatch->folios[i];/* block memcg migration while the folio moves between lru */ -if (move_fn != lru_add && !folio_test_clear_lru (folio)) +if (!is_lru_add && !folio_test_clear_lru (folio)) +continue ; + +/* +* Filter dead folios by moving them from the add batch to the temp +* batch for freeing after this loop. +* +* Since the folio may be part of a huge page, unqueue from +* deferred split list to avoid a dangling list entry. +*/ +if (is_lru_add &&folio_ref_freeze (folio,1 )) { +folio_unqueue_deferred_split (folio); + fbatch->folios[i] =NULL ; +folio_batch_add (&free_fbatch, folio);continue ; + }folio_lruvec_relock_irqsave (folio, &lruvec, &flags);move_fn (lruvec, folio);
@@ -176,6 +199,13 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)if (lruvec)lruvec_unlock_irqrestore (lruvec, flags); + +/* Cleanup filtered dead folios. */ +if (is_lru_add) { +mem_cgroup_uncharge_folios (&free_fbatch); +free_unref_folios (&free_fbatch); + } +folios_put (fbatch); }
@@ -964,6 +994,10 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs)struct folio *folio = folios->folios[i];unsigned int nr_refs = refs ? refs[i] :1 ; +/* Folio batch entry may have been preemptively removed during drain. */ +if (!folio) +continue ; +if (is_huge_zero_folio (folio))continue ;
Signature · 1 line
2.52.0
Quoted 5 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.
Well, this is a lovely patch with no obvious downsides. Nicely done.
Quoted 60 lines
Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>--- mm/swap.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-)mm/swap.c +11
-1@@ -160,13 +160,36 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)int i;struct lruvec *lruvec =NULL ;unsigned long flags =0 ; +struct folio_batch free_fbatch; +bool is_lru_add = (move_fn == lru_add); + +/* +* If we're adding to the LRU, preemptively filter dead folios. Use +* this dedicated folio batch for temp storage and deferred cleanup. +*/ +if (is_lru_add) +folio_batch_init (&free_fbatch);for (i =0 ; i <folio_batch_count (fbatch); i++) {struct folio *folio = fbatch->folios[i];/* block memcg migration while the folio moves between lru */ -if (move_fn != lru_add && !folio_test_clear_lru (folio)) +if (!is_lru_add && !folio_test_clear_lru (folio)) +continue ;
mm/swap.c static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn) · line 188
Quoted 10 lines
mm/swap.c +10
-0+ +/* +* Filter dead folios by moving them from the add batch to the temp +* batch for freeing after this loop. +* +* Since the folio may be part of a huge page, unqueue from +* deferred split list to avoid a dangling list entry. +*/ +if (is_lru_add &&folio_ref_freeze (folio,1 )) { +folio_unqueue_deferred_split (folio);
Would it be better to do this outside the lru lock; it's just that we don't have a convenient batched version to do it? It seems like there are a few places that could use a batched version in vmscan.c and swap.c. Not that I think we should hold up this patch to investigate that micro-optimisation! Just something you couldlook at as a follow-up.
Quoted 8 lines · Matthew Wilcox
Quoted 5 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Well, this is a lovely patch with no obvious downsides. Nicely done.
Thanks for the kind words and review :)
[...]
Quoted 34 lines
Quoted 34 lines
mm/swap.c +20
-1@@ -160,13 +160,36 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)int i;struct lruvec *lruvec =NULL ;unsigned long flags =0 ; +struct folio_batch free_fbatch; +bool is_lru_add = (move_fn == lru_add); + +/* +* If we're adding to the LRU, preemptively filter dead folios. Use +* this dedicated folio batch for temp storage and deferred cleanup. +*/ +if (is_lru_add) +folio_batch_init (&free_fbatch);for (i =0 ; i <folio_batch_count (fbatch); i++) {struct folio *folio = fbatch->folios[i];/* block memcg migration while the folio moves between lru */ -if (move_fn != lru_add && !folio_test_clear_lru (folio)) +if (!is_lru_add && !folio_test_clear_lru (folio)) +continue ; + +/* +* Filter dead folios by moving them from the add batch to the temp +* batch for freeing after this loop. +* +* Since the folio may be part of a huge page, unqueue from +* deferred split list to avoid a dangling list entry. +*/ +if (is_lru_add &&folio_ref_freeze (folio,1 )) {
Quoted 8 lines
mm/swap.c static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn) · line 188
Quoted 1 line
mm/swap.c +1
-0+folio_unqueue_deferred_split (folio);Would it be better to do this outside the lru lock; it's just that we don't have a convenient batched version to do it? It seems like there are a few places that could use a batched version in
vmscan.candswap.c. Not that I think we should hold up this patch to investigate that micro-optimisation! Just something you couldlook at as a follow-up.
Good call. I'll leave this patch as-is (unless other feedback), then pursue the batched version of unqueuing the split in a separate follow-up patch.
Quoted 2 lines
Reviewed-by Matthew Wilcox (Oracle) <[email protected]>
Quoted 2 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time.
So, when they were added to the percpu lru cache, they were alive but during their stay in lru cache, they were freed (last non-lrucache ref dropped) or somehow we are adding folio where the caller drops the reference just after adding to percpu lru cache e.g. folio_putback_lru() ?
Quoted 31 lines
This
is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%
Quoted 5 lines
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>
Overall the code looks good but I do wonder if we can add something similar to folio_add_lru() and if that would be enough.
Quoted 8 lines · Shakeel Butt
Quoted 2 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time.So, when they were added to the percpu lru cache, they were alive but during their stay in lru cache, they were freed (last non-lrucache ref dropped) or somehow we are adding folio where the caller drops the reference just after adding to percpu lru cache e.g.
folio_putback_lru()?
Both scenarios can occur. Whether all callers put the folio while it is on the per-cpu batch or putback drops ref from 2 to 1, the batch ref is what remains.
[...]
Quoted 3 lines
Overall the code looks good but I do wonder if we can add something similar to
folio_add_lru()and if that would be enough.
folio_add_lru() is how it gets onto the batch. But it's still alive at that point - at least one caller ref.
Shakeel Butt
Acked-byQuoted 5 lines · JP Kobryn (Meta)
Quoted 4 lines · Shakeel Butt
Quoted 2 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time.
Quoted 8 lines
Quoted 4 lines
So, when they were added to the percpu lru cache, they were alive but during their stay in lru cache, they were freed (last non-lrucache ref dropped) or somehow we are adding folio where the caller drops the reference just after adding to percpu lru cache e.g.
folio_putback_lru()?Both scenarios can occur. Whether all callers put the folio while it is on the per-cpu batch or putback drops ref from 2 to 1, the batch ref is what remains.
I was wondering which one is dominant.
Quoted 8 lines
[...]
Quoted 3 lines
Overall the code looks good but I do wonder if we can add something similar to
folio_add_lru()and if that would be enough.
folio_add_lru()is how it gets onto the batch. But it's still alive at that point - at least one caller ref.
Yeah, I was thinking if we do special checking at folio_putback_lru() to avoid folio_add_lru() at all (only if folio_putback_lru() is the one causing such scenario most of the time).
Anyways, that analysis can be done later. So, for the path:
Quoted 33 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%
Quoted 5 lines
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>
Hi JP,
I’m seeing a large number of "BAD page" bugs. Not sure if it’s related, but reverting this patch seems to fix the issue.
[ 2869.365978] BUG: Bad page state in process uname pfn:3a5417
[ 2869.365981] page: refcount:0 mapcount:0 mapping:0000000000000000
index:0x724884c20 pfn:0x3a5417
[ 2869.365983] flags:
0x17ffffc0020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
[ 2869.365985] raw: 0017ffffc0020908 0000000000000000 dead000000000122
0000000000000000
[ 2869.365986] raw: 0000000724884c20 0000000000000000 00000000ffffffff
0000000000000000
[ 2869.365986] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[ 2869.366037] CPU: 11 UID: 0 PID: 179413 Comm: uname Tainted: G S B
7.0.0mglruswappiness+ #401 PREEMPT(full)
[ 2869.366039] Tainted: [S]=CPU_OUT_OF_SPEC, [B]=BAD_PAGE
[ 2869.366039] Hardware name: To Be Filled By O.E.M. To Be Filled By
O.E.M./H470D4-P1, BIOS P2.00 04/01/2021
[ 2869.366040] Call Trace:
[ 2869.366041] <TASK>
[ 2869.366041] dump_stack_lvl+0x76/0xa0
[ 2869.366043] dump_stack+0x10/0x20
[ 2869.366044] bad_page+0x79/0x120
[ 2869.366046] free_unref_folios+0x85b/0x960
[ 2869.366048] ? __pfx_lru_add+0x10/0x10
[ 2869.366049] folio_batch_move_lru+0x260/0x2d0
[ 2869.366051] __folio_batch_add_and_move+0x7f/0x110
[ 2869.366052] folio_add_lru+0x49/0x60
[ 2869.366053] folio_add_lru_vma+0x7b/0xb0
[ 2869.366054] map_anon_folio_pte_nopf+0xb8/0x170
[ 2869.366055] do_anonymous_page+0x5fa/0x960
[ 2869.366056] ? __pte_offset_map+0x1c/0x140
[ 2869.366058] __handle_mm_fault+0xbb6/0x1010
[ 2869.366059] ? mt_find+0xe8/0x560
[ 2869.366061] handle_mm_fault+0x1b0/0x370
[ 2869.366063] do_user_addr_fault+0x2c8/0x870
[ 2869.366064] exc_page_fault+0x7d/0x1d0
[ 2869.366065] asm_exc_page_fault+0x27/0x30
[ 2869.366066] RIP: 0010:rep_movs_alternative+0x11/0x90
[ 2869.366068] Code: c3 cc cc cc cc 0f 1f 40 00 90 90 90 90 90 90 90
90 90 90 90 90 90 90 90 90 48 83 f9 40 73 44 83 f9 08 73 25 85 c9 74
0f 8a 06 <88> 07 48 ff c7 48 ff c6 48 ff c9 75 f1 c3 cc cc cc cc 66 66
2e 0f
[ 2869.366069] RSP: 0018:ffffccb46a837a78 EFLAGS: 00050202
[ 2869.366070] RAX: 0000000000000078 RBX: 00007ffee44638d0 RCX: 0000000000000007
[ 2869.366071] RDX: 0000000000000000 RSI: ffffffff8d0d1d90 RDI: 00007ffee44638c9
[ 2869.366072] RBP: ffffccb46a837a80 R08: 00007ffffffff000 R09: 0000000000000000
[ 2869.366072] R10: ffff8c7769873600 R11: 0000000000000000 R12: ffff8c7786563800
[ 2869.366073] R13: ffff8c76c6668480 R14: ffff8c76cfe3ba40 R15: ffff8c770f419c00
[ 2869.366074] ? arch_align_stack+0x3f/0x60
[ 2869.366076] ? _copy_to_user+0x31/0x60
[ 2869.366078] load_elf_binary+0xda1/0x1800
[ 2869.366080] bprm_execve+0x2d7/0x590
[ 2869.366083] do_execveat_common.isra.0+0x15c/0x1c0
[ 2869.366084] __x64_sys_execve+0x3e/0x70
[ 2869.366085] x64_sys_call+0xd91/0x26e0
[ 2869.366087] do_syscall_64+0xe7/0x560
[ 2869.366088] ? folio_add_lru+0x49/0x60
[ 2869.366089] ? folio_add_lru_vma+0x7b/0xb0
[ 2869.366090] ? set_ptes.isra.0+0x3b/0x90
[ 2869.366091] ? do_wp_page+0x85d/0xed0
[ 2869.366092] ? __handle_mm_fault+0xafe/0x1010
[ 2869.366094] ? do_sigaction+0x165/0x4c0
[ 2869.366096] ? _copy_to_user+0x31/0x60
[ 2869.366098] ? __x64_sys_rt_sigaction+0xb7/0x110
[ 2869.366100] ? x64_sys_call+0x880/0x26e0
[ 2869.366101] ? do_syscall_64+0x11b/0x560
[ 2869.366102] ? do_syscall_64+0x9c/0x560
[ 2869.366103] ? clear_bhb_loop+0x30/0x80
[ 2869.366104] entry_SYSCALL_64_after_hwframe+0x76/0x7eThanks
Barry
Quoted 45 lines · Barry Song
Quoted 33 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%Quoted 5 lines
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>Hi JP,
I’m seeing a large number of "BAD page" bugs. Not sure if it’s related, but reverting this patch seems to fix the issue.
Quoted 5 lines
[ 2869.365978] BUG: Bad page state in process uname pfn:3a5417 [ 2869.365981] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x724884c20 pfn:0x3a5417 [ 2869.365983] flags:0x17ffffc0020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
Hi Barry, are you using MGLRU? It seems like MGLRU set active flag in folio_add_lru().
JP, we need to clean active flag but let's check what else can be set before folio_add_lru().
Quoted 47 lines · Shakeel Butt
Quoted 45 lines · Barry Song
Quoted 33 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%Quoted 5 lines
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>Hi JP,
I’m seeing a large number of "BAD page" bugs. Not sure if it’s related, but reverting this patch seems to fix the issue.
Quoted 8 lines
Quoted 5 lines
[ 2869.365978] BUG: Bad page state in process uname pfn:3a5417 [ 2869.365981] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x724884c20 pfn:0x3a5417 [ 2869.365983] flags:0x17ffffc0020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
Hi Barry, are you using MGLRU? It seems like MGLRU set active flag in
folio_add_lru().
Yes. If you are referring to this set_active, I think it is incorrect, so I have fixed it here and am waiting for review:
https://lore.kernel.org/linux-mm/[email protected]/
Quoted 3 lines
JP, we need to clean active flag but let's check what else can be set before
folio_add_lru().
Best Regards
Barry
Quoted 41 lines · Barry Song
Quoted 40 lines · Shakeel Butt
Quoted 38 lines · Barry Song
Quoted 37 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Signed-off-by JP Kobryn (Meta) <[email protected]>Hi JP,
I’m seeing a large number of "BAD page" bugs. Not sure if it’s related, but reverting this patch seems to fix the issue.
It seems this was missed since classic LRU was used in testing.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
[ 2869.365978] BUG: Bad page state in process uname pfn:3a5417 [ 2869.365981] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x724884c20 pfn:0x3a5417 [ 2869.365983] flags:0x17ffffc0020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
Quoted 11 lines
Quoted 2 lines
Hi Barry, are you using MGLRU? It seems like MGLRU set active flag in
folio_add_lru().Yes. If you are referring to this
set_active, I think it is incorrect, so I have fixed it here and am waiting for review:https://lore.kernel.org/linux-mm/[email protected]/
Quoted 3 lines
JP, we need to clean active flag but let's check what else can be set before
folio_add_lru().
Looks like only active is the problem. If we start manually clearing flags it starts to feel messy. I get that some fix is needed though. I don't see this patch in mm-new yet so maybe we can hold off on merging there to avoid the MGLRU case. But if Barry's patch is accepted, could we re-apply?
Let me know if you're thinking there are any implications beyond the active flag.
Quoted 54 lines · Barry Song
Quoted 53 lines · Shakeel Butt
Quoted 50 lines · Barry Song
Quoted 33 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%Quoted 5 lines
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>Hi JP,
I’m seeing a large number of "BAD page" bugs. Not sure if it’s related, but reverting this patch seems to fix the issue.[ 2869.365978] BUG: Bad page state in process uname pfn:3a5417 [ 2869.365981] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x724884c20 pfn:0x3a5417 [ 2869.365983] flags:0x17ffffc0020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
Quoted 11 lines
Quoted 2 lines
Hi Barry, are you using MGLRU? It seems like MGLRU set active flag in
folio_add_lru().Yes. If you are referring to this
set_active, I think it is incorrect, so I have fixed it here and am waiting for review:https://lore.kernel.org/linux-mm/[email protected]/
Quoted 3 lines
JP, we need to clean active flag but let's check what else can be set before
folio_add_lru().
Barry/Shakeel,
We can do something like this as a special case for MGLRU:
mm/swap.c +2 -0
@@ -185,6 +185,8 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)* deferred split list to avoid a dangling list entry. */ if (is_lru_add &&folio_ref_freeze (folio,1 )) { +if (lru_gen_enabled ()) +__folio_clear_active (folio);folio_unqueue_deferred_split (folio); fbatch->folios[i] =NULL ;folio_batch_add (&free_fbatch, folio);
Unless Barry's patch works out... Any thoughts?
Quoted 86 lines · JP Kobryn (Meta)
Quoted 54 lines · Barry Song
Quoted 53 lines · Shakeel Butt
Quoted 50 lines · Barry Song
Quoted 33 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%Quoted 5 lines
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>Hi JP,
I’m seeing a large number of "BAD page" bugs. Not sure if it’s related, but reverting this patch seems to fix the issue.[ 2869.365978] BUG: Bad page state in process uname pfn:3a5417 [ 2869.365981] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x724884c20 pfn:0x3a5417 [ 2869.365983] flags:0x17ffffc0020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x1fffff)
Quoted 11 lines
Quoted 2 lines
Hi Barry, are you using MGLRU? It seems like MGLRU set active flag in
folio_add_lru().Yes. If you are referring to this
set_active, I think it is incorrect, so I have fixed it here and am waiting for review:https://lore.kernel.org/linux-mm/[email protected]/
Quoted 3 lines
JP, we need to clean active flag but let's check what else can be set before
folio_add_lru().Barry/Shakeel,
We can do something like this as a special case for MGLRU:
mm/swap.c +2
-0@@ -185,6 +185,8 @@ static void folio_batch_move_lru(struct folio_batch*fbatch, move_fn_t move_fn) * deferred split list to avoid a dangling list entry. */ if (is_lru_add &&folio_ref_freeze (folio,1 )) { +if (lru_gen_enabled ()) +__folio_clear_active (folio);folio_unqueue_deferred_split (folio); fbatch->folios[i] =NULL ;
folio_batch_add(&free_fbatch, folio);
Quoted 1 line
Unless Barry's patch works out... Any thoughts?
I think this is fine. We can remove this later when Barry's patch is settled in a followup path.
Quoted 1 line
syzbot ci has tested the following series
syzbot ci has tested the following series
[v1] mm/lruvec: preemptively free dead folios during lru_add drain
https://lore.kernel.org/all/[email protected]
* [PATCH] mm/lruvec: preemptively free dead folios during lru_add drain
and found the following issues:
* BUG: Bad page state in do_pte_missing
* BUG: Bad page state in do_wp_page
Full report is available here:
https://ci.syzbot.org/series/d16e663b-bcf5-49dd-937d-24a22e0c8d6a
***
BUG: Bad page state in do_pte_missing
tree: mm-new URL: https://kernel.googlesource.com/pub/scm/linux/kernel/git/akpm/mm.git base: c9183ec6e2e3bd26a017392d6c3eaa40c580f153 arch: amd64 compiler: Debian clang version 21.1.8 (++20251221033036+2078da43e25a-1~exp1~20251221153213.50), Debian LLD 21.1.8 config: https://ci.syzbot.org/builds/4d1a53f4-f506-4ba9-a8c2-a48196f95abc/config syz repro: https://ci.syzbot.org/findings/19ba977e-39f2-4f40-b936-a8642aaab537/syz_repro
BUG: Bad page state: 15292 messages suppressed BUG: Bad page state in process syz.2.1435 pfn:1af40e page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c9e16 pfn:0x1af40e flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c9e16 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO|__GFP_COMP), pid 8846, tgid 8846 (syz.0.1431), ts 86965635591, free_ts 69887114766 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] alloc_anon_folio mm/memory.c:5282 [inline] do_anonymous_page mm/memory.c:5376 [inline] do_pte_missing+0x159d/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5831 tgid 5831 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 unmap_region+0x2a5/0x330 mm/vma.c:491 vms_clear_ptes mm/vma.c:1303 [inline] vms_complete_munmap_vmas+0x493/0xc60 mm/vma.c:1345 do_vmi_align_munmap+0x3b7/0x4b0 mm/vma.c:1604 do_vmi_munmap+0x252/0x2d0 mm/vma.c:1652 __vm_munmap+0x22c/0x3d0 mm/vma.c:3285 __do_sys_munmap mm/mmap.c:1079 [inline] __se_sys_munmap mm/mmap.c:1076 [inline] __x64_sys_munmap+0x60/0x70 mm/mmap.c:1076 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af40f page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c9e17 pfn:0x1af40f flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c9e17 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8846, tgid 8846 (syz.0.1431), ts 86965651367, free_ts 69887118923 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5831 tgid 5831 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 unmap_region+0x2a5/0x330 mm/vma.c:491 vms_clear_ptes mm/vma.c:1303 [inline] vms_complete_munmap_vmas+0x493/0xc60 mm/vma.c:1345 do_vmi_align_munmap+0x3b7/0x4b0 mm/vma.c:1604 do_vmi_munmap+0x252/0x2d0 mm/vma.c:1652 __vm_munmap+0x22c/0x3d0 mm/vma.c:3285 __do_sys_munmap mm/mmap.c:1079 [inline] __se_sys_munmap mm/mmap.c:1076 [inline] __x64_sys_munmap+0x60/0x70 mm/mmap.c:1076 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af410 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c91fd pfn:0x1af410 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c91fd 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO|__GFP_COMP), pid 8849, tgid 8846 (syz.0.1431), ts 86965683544, free_ts 69887122392 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] alloc_anon_folio mm/memory.c:5282 [inline] do_anonymous_page mm/memory.c:5376 [inline] do_pte_missing+0x159d/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5831 tgid 5831 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 unmap_region+0x2a5/0x330 mm/vma.c:491 vms_clear_ptes mm/vma.c:1303 [inline] vms_complete_munmap_vmas+0x493/0xc60 mm/vma.c:1345 do_vmi_align_munmap+0x3b7/0x4b0 mm/vma.c:1604 do_vmi_munmap+0x252/0x2d0 mm/vma.c:1652 __vm_munmap+0x22c/0x3d0 mm/vma.c:3285 __do_sys_munmap mm/mmap.c:1079 [inline] __se_sys_munmap mm/mmap.c:1076 [inline] __x64_sys_munmap+0x60/0x70 mm/mmap.c:1076 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af411 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c9e12 pfn:0x1af411 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c9e12 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8846, tgid 8846 (syz.0.1431), ts 86965789200, free_ts 69887126279 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5831 tgid 5831 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 unmap_region+0x2a5/0x330 mm/vma.c:491 vms_clear_ptes mm/vma.c:1303 [inline] vms_complete_munmap_vmas+0x493/0xc60 mm/vma.c:1345 do_vmi_align_munmap+0x3b7/0x4b0 mm/vma.c:1604 do_vmi_munmap+0x252/0x2d0 mm/vma.c:1652 __vm_munmap+0x22c/0x3d0 mm/vma.c:3285 __do_sys_munmap mm/mmap.c:1079 [inline] __se_sys_munmap mm/mmap.c:1076 [inline] __x64_sys_munmap+0x60/0x70 mm/mmap.c:1076 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af40b page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x555573b0d pfn:0x1af40b flags: 0x57ff0000002090c(referenced|uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff0000002090c 0000000000000000 dead000000000122 0000000000000000 raw: 0000000555573b0d 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8851, tgid 8851 (syz-executor), ts 86975719417, free_ts 86967133373 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0x75b/0x1340 arch/x86/mm/fault.c:1385 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 8849 tgid 8846 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 exit_mmap+0x498/0x9e0 mm/mmap.c:1313 __mmput+0x118/0x430 kernel/fork.c:1178 exit_mm+0x18e/0x250 kernel/exit.c:581 do_exit+0x6a2/0x22c0 kernel/exit.c:963 do_group_exit+0x21b/0x2d0 kernel/exit.c:1117 get_signal+0x1284/0x1330 kernel/signal.c:3037 arch_do_signal_or_restart+0xbc/0x830 arch/x86/kernel/signal.c:337 __exit_to_user_mode_loop kernel/entry/common.c:64 [inline] exit_to_user_mode_loop+0x86/0x480 kernel/entry/common.c:98 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline] syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:238 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:328 [inline] do_syscall_64+0x33e/0xf80 arch/x86/entry/syscall_64.c:100 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af409 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x3ec pfn:0x1af409 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000000000003ec 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8851, tgid 8851 (syz-executor), ts 86976014509, free_ts 86967128194 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 8849 tgid 8846 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 exit_mmap+0x498/0x9e0 mm/mmap.c:1313 __mmput+0x118/0x430 kernel/fork.c:1178 exit_mm+0x18e/0x250 kernel/exit.c:581 do_exit+0x6a2/0x22c0 kernel/exit.c:963 do_group_exit+0x21b/0x2d0 kernel/exit.c:1117 get_signal+0x1284/0x1330 kernel/signal.c:3037 arch_do_signal_or_restart+0xbc/0x830 arch/x86/kernel/signal.c:337 __exit_to_user_mode_loop kernel/entry/common.c:64 [inline] exit_to_user_mode_loop+0x86/0x480 kernel/entry/common.c:98 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline] syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:238 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:328 [inline] do_syscall_64+0x33e/0xf80 arch/x86/entry/syscall_64.c:100 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af571 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14ca945 pfn:0x1af571 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14ca945 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8851, tgid 8851 (syz-executor), ts 86978306736, free_ts 86967122819 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 8849 tgid 8846 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 exit_mmap+0x498/0x9e0 mm/mmap.c:1313 __mmput+0x118/0x430 kernel/fork.c:1178 exit_mm+0x18e/0x250 kernel/exit.c:581 do_exit+0x6a2/0x22c0 kernel/exit.c:963 do_group_exit+0x21b/0x2d0 kernel/exit.c:1117 get_signal+0x1284/0x1330 kernel/signal.c:3037 arch_do_signal_or_restart+0xbc/0x830 arch/x86/kernel/signal.c:337 __exit_to_user_mode_loop kernel/entry/common.c:64 [inline] exit_to_user_mode_loop+0x86/0x480 kernel/entry/common.c:98 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline] syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:238 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:328 [inline] do_syscall_64+0x33e/0xf80 arch/x86/entry/syscall_64.c:100 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af40c page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c9e15 pfn:0x1af40c flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c9e15 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO|__GFP_COMP), pid 8851, tgid 8851 (syz.0.1434), ts 86978464147, free_ts 86967036690 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 8849 tgid 8846 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 exit_mmap+0x498/0x9e0 mm/mmap.c:1313 __mmput+0x118/0x430 kernel/fork.c:1178 exit_mm+0x18e/0x250 kernel/exit.c:581 do_exit+0x6a2/0x22c0 kernel/exit.c:963 do_group_exit+0x21b/0x2d0 kernel/exit.c:1117 get_signal+0x1284/0x1330 kernel/signal.c:3037 arch_do_signal_or_restart+0xbc/0x830 arch/x86/kernel/signal.c:337 __exit_to_user_mode_loop kernel/entry/common.c:64 [inline] exit_to_user_mode_loop+0x86/0x480 kernel/entry/common.c:98 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline] syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:238 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:328 [inline] do_syscall_64+0x33e/0xf80 arch/x86/entry/syscall_64.c:100 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af40d page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c91fe pfn:0x1af40d flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c91fe 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO|__GFP_COMP), pid 8851, tgid 8851 (syz.0.1434), ts 86979644815, free_ts 86967032365 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] alloc_anon_folio mm/memory.c:5282 [inline] do_anonymous_page mm/memory.c:5376 [inline] do_pte_missing+0x159d/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 8849 tgid 8846 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 exit_mmap+0x498/0x9e0 mm/mmap.c:1313 __mmput+0x118/0x430 kernel/fork.c:1178 exit_mm+0x18e/0x250 kernel/exit.c:581 do_exit+0x6a2/0x22c0 kernel/exit.c:963 do_group_exit+0x21b/0x2d0 kernel/exit.c:1117 get_signal+0x1284/0x1330 kernel/signal.c:3037 arch_do_signal_or_restart+0xbc/0x830 arch/x86/kernel/signal.c:337 __exit_to_user_mode_loop kernel/entry/common.c:64 [inline] exit_to_user_mode_loop+0x86/0x480 kernel/entry/common.c:98 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline] syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:238 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:328 [inline] do_syscall_64+0x33e/0xf80 arch/x86/entry/syscall_64.c:100 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af408 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x555573b20 pfn:0x1af408 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 0000000555573b20 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8851, tgid 8851 (syz.0.1434), ts 86979662330, free_ts 86967025854 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 8849 tgid 8846 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 exit_mmap+0x498/0x9e0 mm/mmap.c:1313 __mmput+0x118/0x430 kernel/fork.c:1178 exit_mm+0x18e/0x250 kernel/exit.c:581 do_exit+0x6a2/0x22c0 kernel/exit.c:963 do_group_exit+0x21b/0x2d0 kernel/exit.c:1117 get_signal+0x1284/0x1330 kernel/signal.c:3037 arch_do_signal_or_restart+0xbc/0x830 arch/x86/kernel/signal.c:337 __exit_to_user_mode_loop kernel/entry/common.c:64 [inline] exit_to_user_mode_loop+0x86/0x480 kernel/entry/common.c:98 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline] syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:238 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:328 [inline] do_syscall_64+0x33e/0xf80 arch/x86/entry/syscall_64.c:100 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af412 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c9e16 pfn:0x1af412 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c9e16 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO|__GFP_COMP), pid 8851, tgid 8851 (syz.0.1434), ts 86981021345, free_ts 69887129694 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] alloc_anon_folio mm/memory.c:5282 [inline] do_anonymous_page mm/memory.c:5376 [inline] do_pte_missing+0x159d/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5831 tgid 5831 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 unmap_region+0x2a5/0x330 mm/vma.c:491 vms_clear_ptes mm/vma.c:1303 [inline] vms_complete_munmap_vmas+0x493/0xc60 mm/vma.c:1345 do_vmi_align_munmap+0x3b7/0x4b0 mm/vma.c:1604 do_vmi_munmap+0x252/0x2d0 mm/vma.c:1652 __vm_munmap+0x22c/0x3d0 mm/vma.c:3285 __do_sys_munmap mm/mmap.c:1079 [inline] __se_sys_munmap mm/mmap.c:1076 [inline] __x64_sys_munmap+0x60/0x70 mm/mmap.c:1076 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af413 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c9e17 pfn:0x1af413 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c9e17 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8851, tgid 8851 (syz.0.1434), ts 86981071632, free_ts 69887133292 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5831 tgid 5831 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 unmap_region+0x2a5/0x330 mm/vma.c:491 vms_clear_ptes mm/vma.c:1303 [inline] vms_complete_munmap_vmas+0x493/0xc60 mm/vma.c:1345 do_vmi_align_munmap+0x3b7/0x4b0 mm/vma.c:1604 do_vmi_munmap+0x252/0x2d0 mm/vma.c:1652 __vm_munmap+0x22c/0x3d0 mm/vma.c:3285 __do_sys_munmap mm/mmap.c:1079 [inline] __se_sys_munmap mm/mmap.c:1076 [inline] __x64_sys_munmap+0x60/0x70 mm/mmap.c:1076 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK> BUG: Bad page state in process syz.2.1435 pfn:1af414 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f14c9e12 pfn:0x1af414 flags: 0x57ff00000020908(uptodate|active|owner_2|swapbacked|node=1|zone=2|lastcpupid=0x7ff) raw: 057ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f14c9e12 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 8851, tgid 8851 (syz.0.1434), ts 86982688821, free_ts 69887137521 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5831 tgid 5831 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] free_unref_folios+0xcec/0x1480 mm/page_alloc.c:3004 folios_put_refs+0xa3d/0xb80 mm/swap.c:1042 free_pages_and_swap_cache+0x41d/0x490 mm/swap_state.c:404 __tlb_batch_free_encoded_pages mm/mmu_gather.c:138 [inline] tlb_batch_pages_flush mm/mmu_gather.c:151 [inline] tlb_flush_mmu_free mm/mmu_gather.c:417 [inline] tlb_flush_mmu+0x6d3/0xa30 mm/mmu_gather.c:424 tlb_finish_mmu+0xf9/0x230 mm/mmu_gather.c:549 unmap_region+0x2a5/0x330 mm/vma.c:491 vms_clear_ptes mm/vma.c:1303 [inline] vms_complete_munmap_vmas+0x493/0xc60 mm/vma.c:1345 do_vmi_align_munmap+0x3b7/0x4b0 mm/vma.c:1604 do_vmi_munmap+0x252/0x2d0 mm/vma.c:1652 __vm_munmap+0x22c/0x3d0 mm/vma.c:3285 __do_sys_munmap mm/mmap.c:1079 [inline] __se_sys_munmap mm/mmap.c:1076 [inline] __x64_sys_munmap+0x60/0x70 mm/mmap.c:1076 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x15f/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f Modules linked in: CPU: 1 UID: 0 PID: 8855 Comm: syz.2.1435 Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 map_anon_folio_pte_nopf+0x2ee/0x5e0 mm/memory.c:5301 map_anon_folio_pte_pf+0xbe/0x260 mm/memory.c:5311 do_anonymous_page mm/memory.c:5413 [inline] do_pte_missing+0x2d48/0x33f0 mm/memory.c:4548 handle_pte_fault mm/memory.c:6411 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x1bd7/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7fca082696d2 Code: 48 89 ca 48 c1 e2 04 48 29 ca 48 8d 0d 5f c9 3a 00 48 c1 e2 04 48 01 c2 8b 44 24 1c c6 42 20 01 89 42 24 8b 44 24 24 89 6a 28 <89> 42 78 0f b6 44 24 43 89 72 2c 88 44 19 04 8b 44 24 20 31 c9 89 RSP: 002b:00007ffc5adb04c0 EFLAGS: 00010206 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007fca08616018 RDX: 00007fca08615fa0 RSI: 0000000000000003 RDI: 0000000000000000 RBP: 0000000000001cd6 R08: 00007fca08615fa0 R09: 00007ffc5adb0357 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fca08615fac R14: 00007fca08615fa8 R15: 00007fca08615fa0 </TASK>
***
BUG: Bad page state in do_wp_page
tree: mm-new URL: https://kernel.googlesource.com/pub/scm/linux/kernel/git/akpm/mm.git base: c9183ec6e2e3bd26a017392d6c3eaa40c580f153 arch: amd64 compiler: Debian clang version 21.1.8 (++20251221033036+2078da43e25a-1~exp1~20251221153213.50), Debian LLD 21.1.8 config: https://ci.syzbot.org/builds/4d1a53f4-f506-4ba9-a8c2-a48196f95abc/config syz repro: https://ci.syzbot.org/findings/5db96cfa-fea0-4d9d-9b97-6924026375b7/syz_repro
BUG: Bad page state: 5753 messages suppressed BUG: Bad page state in process syz-executor pfn:11e4a5 page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x7f4658e12 pfn:0x11e4a5 flags: 0x17ff00000020908(uptodate|active|owner_2|swapbacked|node=0|zone=2|lastcpupid=0x7ff) raw: 017ff00000020908 0000000000000000 dead000000000122 0000000000000000 raw: 00000007f4658e12 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set page_owner tracks the page as allocated page last allocated via order 0, migratetype Movable, gfp_mask 0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), pid 6083, tgid 6083 (syz.1.50), ts 86214702085, free_ts 66376409073 set_page_owner include/linux/page_owner.h:32 [inline] post_alloc_hook+0x22d/0x280 mm/page_alloc.c:1858 prep_new_page mm/page_alloc.c:1866 [inline] get_page_from_freelist+0x24ba/0x2540 mm/page_alloc.c:3946 __alloc_frozen_pages_noprof+0x18d/0x380 mm/page_alloc.c:5226 alloc_pages_mpol+0x235/0x490 mm/mempolicy.c:2490 folio_alloc_mpol_noprof+0x39/0x160 mm/mempolicy.c:2509 vma_alloc_folio_noprof+0xe1/0x1e0 mm/mempolicy.c:2544 folio_prealloc mm/memory.c:-1 [inline] wp_page_copy mm/memory.c:3859 [inline] do_wp_page+0x118a/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 page last free pid 5817 tgid 5790 stack trace: reset_page_owner include/linux/page_owner.h:25 [inline] __free_pages_prepare mm/page_alloc.c:1402 [inline] __free_frozen_pages+0xbc7/0xd30 mm/page_alloc.c:2943 __folio_put+0x4a2/0x580 mm/swap.c:112 folio_put include/linux/mm.h:2090 [inline] migrate_folio_done mm/migrate.c:1201 [inline] migrate_folio_move mm/migrate.c:1432 [inline] migrate_folios_move mm/migrate.c:1740 [inline] migrate_pages_batch+0x3dac/0x4ca0 mm/migrate.c:1996 migrate_pages+0x1e02/0x2a10 mm/migrate.c:2130 migrate_misplaced_folio+0x273/0x720 mm/migrate.c:2751 do_numa_page mm/memory.c:6199 [inline] handle_pte_fault mm/memory.c:6417 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x20ea/0x3170 mm/memory.c:6718 do_user_addr_fault+0x75b/0x1340 arch/x86/mm/fault.c:1385 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 Modules linked in: CPU: 0 UID: 0 PID: 6089 Comm: syz-executor Tainted: G B syzkaller #0 PREEMPT(full) Tainted: [B]=BAD_PAGE Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 bad_page+0x17f/0x1c0 mm/page_alloc.c:632 free_page_is_bad mm/page_alloc.c:1073 [inline] __free_pages_prepare mm/page_alloc.c:1393 [inline] free_unref_folios+0xdcc/0x1480 mm/page_alloc.c:3004 folio_batch_move_lru+0x816/0x9e0 mm/swap.c:206 __folio_batch_add_and_move+0x510/0xc50 mm/swap.c:226 folio_add_lru_vma+0x196/0x210 mm/swap.c:566 wp_page_copy mm/memory.c:3927 [inline] do_wp_page+0x3deb/0x4cc0 mm/memory.c:4320 handle_pte_fault mm/memory.c:6427 [inline] __handle_mm_fault mm/memory.c:6549 [inline] handle_mm_fault+0x151d/0x3170 mm/memory.c:6718 do_user_addr_fault+0xa73/0x1340 arch/x86/mm/fault.c:1334 handle_page_fault arch/x86/mm/fault.c:1474 [inline] exc_page_fault+0x6a/0xc0 arch/x86/mm/fault.c:1527 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:618 RIP: 0033:0x7f4658b61da7 Code: 00 00 00 80 3d 51 aa de 00 00 0f 84 b4 00 00 00 48 c7 c0 e0 ff ff ff c7 05 4a aa de 00 00 00 00 00 64 48 8b 08 48 85 c9 74 0b <48> c7 81 80 08 00 00 01 00 00 00 48 c7 05 23 aa de 00 00 00 00 00 RSP: 002b:00007ffc5850f508 EFLAGS: 00010202 RAX: ffffffffffffffe0 RBX: 0000000000000000 RCX: 00007f4658dece20 RDX: 0000000000000000 RSI: 00007f4658c52ee0 RDI: 00007f4659948060 RBP: 00007ffc5850f66c R08: 00007f4659948060 R09: 0000000000000000 R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000001 R13: 00000000000927c0 R14: 0000000000015068 R15: 00007ffc5850f6c0 </TASK>
***
If these findings have caused you to resend the series or submit a separate fix, please add the following tag to your commit message:
---
This report is generated by a bot. It may contain errors. syzbot ci engineers can be reached at [email protected].
To test a patch for this bug, please reply with #syz test (should be on a separate line).
The patch should be attached to the email.
Note: arguments like custom git repos and branches are not supported.
Quoted 32 lines · JP Kobryn (Meta)
Of all observable lruvec lock contention in our fleet, we find that ~24% occurs when dead folios are present in
lru_addbatches at drain time. This is wasteful in the sense that the folio is added to the LRU just to be immediately removed viafolios_put_refs(), incurring two unnecessary lock acquisitions.Eliminate this overhead by preemptively cleaning up dead folios before they make it into the LRU. Use
folio_ref_freeze()to filter folios whose only remaining refcount is the batch ref. When dead folios are found, move them off the add batch and onto a temporary batch to be freed.During A/B testing on one of our prod instagram workloads (high-frequency short-lived requests), the patch intercepted almost all dead folios before they entered the LRU. Data collected using the
mm_lru_insertiontracepoint shows the effectiveness of the patch:Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097% patched: 14 19,039,996 0.0001%Within this workload, we save ~2.6M lock acquisitions per minute per host as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%
Quoted 5 lines
No regressions were observed in requests served per second or request tail latency (p99). Both metrics showed directional improvement at higher CPU utilization (comparing 85% to 95%).
Signed-off-by JP Kobryn (Meta) <[email protected]>
Thanks!
Quoted 73 lines
--- mm/swap.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-)mm/swap.c +35
-1@@ -160,13 +160,36 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)int i;struct lruvec *lruvec =NULL ;unsigned long flags =0 ; +struct folio_batch free_fbatch; +bool is_lru_add = (move_fn == lru_add); + +/* +* If we're adding to the LRU, preemptively filter dead folios. Use +* this dedicated folio batch for temp storage and deferred cleanup. +*/ +if (is_lru_add) +folio_batch_init (&free_fbatch);for (i =0 ; i <folio_batch_count (fbatch); i++) {struct folio *folio = fbatch->folios[i];/* block memcg migration while the folio moves between lru */ -if (move_fn != lru_add && !folio_test_clear_lru (folio)) +if (!is_lru_add && !folio_test_clear_lru (folio)) +continue ; + +/* +* Filter dead folios by moving them from the add batch to the temp +* batch for freeing after this loop. +* +* Since the folio may be part of a huge page, unqueue from +* deferred split list to avoid a dangling list entry. +*/ +if (is_lru_add &&folio_ref_freeze (folio,1 )) { +folio_unqueue_deferred_split (folio); + fbatch->folios[i] =NULL ; +folio_batch_add (&free_fbatch, folio);continue ; + }folio_lruvec_relock_irqsave (folio, &lruvec, &flags);move_fn (lruvec, folio);@@ -176,6 +199,13 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn)if (lruvec)lruvec_unlock_irqrestore (lruvec, flags); + +/* Cleanup filtered dead folios. */ +if (is_lru_add) { +mem_cgroup_uncharge_folios (&free_fbatch); +free_unref_folios (&free_fbatch); + } +folios_put (fbatch); }@@ -964,6 +994,10 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs)struct folio *folio = folios->folios[i];unsigned int nr_refs = refs ? refs[i] :1 ; +/* Folio batch entry may have been preemptively removed during drain. */ +if (!folio) +continue ; +if (is_huge_zero_folio (folio))continue ;Signature · 1 line
2.52.0
Signature · 2 lines
Michal Hocko SUSE Labs