mm/vmscan: flush deferred TLB before freeing large folios in reclaim

Hi,

We were chasing random user-space segfaults on our production kernels. They were fully non-deterministic, and the fault address reported for each SIGSEGV was itself random and had nothing to do with the code that was actually running -- which smells like a stale TLB entry, not corrupted data.

We managed to reproduce it with stress-ng: under memory pressure the workers take random SIGSEGV/SIGILL too, again at random fault addresses.

It turns out we're missing a TLB flush. In reclaim, shrink_folio_list() tears down the PTEs with a deferred, batched flush. Order-0 folios are collected and only freed after that batch is flushed by try_to_unmap_flush(). Large folios, though, are freed right away at the free_it label via destroy_large_folio(), which runs *before* the flush. So a large folio's pages can go back to the allocator and get reused while some other CPU still has a stale TLB entry pointing at them -- and that CPU then reads or executes through the old translation into a page that now belongs to someone else. When it's executable text mapped as a large folio, the CPU literally fetches instructions out of a reused page, which is where the random crashes come from. (This is about file-backed large folios, not anonymous THP, so transparent_hugepage=never doesn't help.)

How we reproduce it:
- Make a cgroup and set memory.high.
- Run ~45 stress-ng workers in it (e.g. --cpu N --cpu-method all).
- Alongside, run a tiny program that keeps allocating anonymous memory to push the cgroup over memory.high and keep reclaim busy.
Dropping caches first (echo 3 > /proc/sys/vm/drop_caches) makes the text refault as large folios and reproduces it much faster.

To be 100% sure about the mechanism, we filled every reclaimed large folio with 0xCC (INT3) and held onto it instead of freeing it. Under the repro, stress-ng workers immediately hit INT3 at instruction pointers inside their own text -- i.e. CPUs were fetching instructions through stale TLB entries from freed, poisoned pages. stress-ng has no INT3 in its binary, so the only way to execute one is through a stale translation into a freed page. Several CPUs hit it within the same microsecond, which lines up nicely with a single batched unmap whose flush was skipped on more than one CPU.

Upstream this got fixed as a side effect of
commit bc2ff4cbc329 ("mm: free folios in a batch in shrink_folio_list()") which sends large folios down the same flush-before-free batch path.

The patch below is the minimal fix: just flush the deferred batch before freeing a large folio inline.

Jiayuan Chen (1):
mm/vmscan: flush deferred TLB before freeing large folios

 mm/vmscan.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

2.43.0

From: Jiayuan Chen <[email protected]>

In reclaim, shrink_folio_list() unmaps PTEs with a deferred, batched TLB flush. The batch is only flushed by try_to_unmap_flush() near the end of the function, just before the order-0 folios collected in @free_folios are handed back to the allocator.

Large folios don't go through @free_folios -- they're freed inline at the free_it label via destroy_large_folio(), which runs before that flush. So a large folio's pages can be returned to the buddy allocator and reused while another CPU still holds a stale TLB entry for them, and that CPU then reads or executes through the stale translation into the reused page. For file-backed large folios (e.g. executable text) this shows up as random SIGSEGV/SIGILL in user space, with fault addresses that don't match the code being run.

Flush the deferred batch before freeing a large folio inline, the same way the order-0 path already waits for the flush.

Upstream this is fixed as a side effect of commit bc2ff4cbc329 ("mm: free folios in a batch in shrink_folio_list()"), which is a larger change; this is the minimal fix for -stable.

---
destroy_compound_page was recently renamed to destroy_large_folio. So it would be conflict when this patch was applied to 5.15/6.1

---
 mm/vmscan.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

index aba757e5c597..8eb498351d9b 100644
@@ -2123,10 +2123,12 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 		 * Is there need to periodically free_folio_list? It would
 		 * appear not as the counts should be low
 		 */
-		if (unlikely(folio_test_large(folio)))
+		if (unlikely(folio_test_large(folio))) {
+			try_to_unmap_flush();
 			destroy_large_folio(folio);
-		else
+		} else {
 			list_add(&folio->lru, &free_folios);
+		}
 		continue;
 
 activate_locked_split:

2.43.0

I agree that it would be wrong to backport bc2ff4cbc329 and all its dependencies. And free_unref_page_list() only handles order-0 folios, so we can't do this:

- if (unlikely(folio_test_large(folio)))
- destroy_large_folio(folio);
- else
- list_add(&folio->lru, &free_folios);
+ list_add(&folio->lru, &free_folios);
continue;

I think this is the right fix.

Queued for 6.6 and 6.1 with Matthew's Reviewed-by added, thanks.

It actually applied cleanly to the current 6.1.y (that tree already uses folio_test_large()/destroy_large_folio() there), but it does not apply to 5.15.y, which is still page-based (PageTransHuge()/destroy_compound_page()). Could you send a tested per-branch version for 5.15.y? 5.10.y has the identical vulnerable code shape, so a 5.10.y version would be welcome as well.

Thanks,
Sasha

Thanks Sasha.

I will try to reproduce and fix in 5.15 and 5.10.