Skip to contentSkip to conversation

mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

PATCH Superseded James Houghton 3 participants 8 messages Aug 28 – Aug 29

Subsystems (6)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/khugepaged: Don't install PMDs in uffd-minor-registered VMAs

Userfaultfd minor faults provides userspace with the ability to manually install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can map holes in the VMA when a naturally-aligned THP is present without explicit action from userspace.

This is a problem, as it bypasses userfaultfd minor faults that userspace is expecting to handle.

If userspace implements post-copy live migration using userfaultfd minor faults, this situation is currently possible:
1. The VMA for guest memory is userfaultfd-minor-registered and nothing is mapped in the page tables.
2. A stale copy of a page is present in a naturally-aligned THP (from pre-copy live migration).
3. khugepaged collapses the mapping of the THP, installs a PMD.
4. The VM now has access to the stale contents => VM is broken.
5. After installing the correct contents, userspace attempts to map the page with UFFDIO_CONTINUE; it gets EEXIST, indicating that something unexpectedly mapped the page.

The naturally-aligned THP case is the only case where this is a problem. khugepaged otherwise requires all PTEs to be present for userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is correct. This check is essentially bypassed for naturally-aligned THPs.

No changes are needed for file_backed_vma_is_retractable(), as zapping PTEs is safe. Userspace must already handle cases where PTEs are zapped without explicit action (e.g. due to reclaim).

Fixes 58ac9a8993a1 ("mm/khugepaged: attempt to map file/shmem-backed pte-mapped THPs by pmds")

---
v1->v2:
- Applied Lance's change to move the userfaultfd_minor() check to the right place.
- Suggested-by: and Tested-by: Lance. Thank you!
- Adjusted the commit description.

v1: https://lore.kernel.org/linux-mm/[email protected]/

---
 mm/khugepaged.c | 7 +++++++
 1 file changed, 7 insertions(+)
mm/khugepaged.c +7 -0
index b237f6e7662a..22356229c1e4 100644
@@ -1899,6 +1899,13 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
 	if (userfaultfd_protected(vma))
 		return SCAN_PTE_UFFD;
 
+	/*
+	 * Userfaultfd-minor-registered VMAs should not be collapsed, as
+	 * userspace is expecting to explicitly install PTEs.
+	 */
+	if (userfaultfd_minor(vma))
+		return SCAN_PTE_UFFD;
+
 	folio = filemap_lock_folio(vma->vm_file->f_mapping,
 			       linear_page_index(vma, haddr));
 	if (IS_ERR(folio))

base-commit: 26260251022fbc2f248a3d747a9b2b961b18d2d8

Signature · 1 line
2.55.0.897.gb25b4bd76c-goog
maintainer
#1 lore raw
Quoted 8 lines · On Fri, 28 Aug 2026 22:26:40 +0000 James Houghton <[email protected]> wrote:

Userfaultfd minor faults provides userspace with the ability to manually install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can map holes in the VMA when a naturally-aligned THP is present without explicit action from userspace.

This is a problem, as it bypasses userfaultfd minor faults that userspace is expecting to handle.

Quoted 11 lines

If userspace implements post-copy live migration using userfaultfd minor faults, this situation is currently possible:
1. The VMA for guest memory is userfaultfd-minor-registered and nothing is mapped in the page tables.
2. A stale copy of a page is present in a naturally-aligned THP (from pre-copy live migration).
3. khugepaged collapses the mapping of the THP, installs a PMD.
4. The VM now has access to the stale contents => VM is broken.
5. After installing the correct contents, userspace attempts to map the page with UFFDIO_CONTINUE; it gets EEXIST, indicating that something unexpectedly mapped the page.

Did I read somewhere that this it biting you in a real-life scenario? That would use useful info to spell out.

Or is this a theoretical thing?

Quoted 11 lines

The naturally-aligned THP case is the only case where this is a problem. khugepaged otherwise requires all PTEs to be present for userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is correct. This check is essentially bypassed for naturally-aligned THPs.

No changes are needed for file_backed_vma_is_retractable(), as zapping PTEs is safe. Userspace must already handle cases where PTEs are zapped without explicit action (e.g. due to reclaim).

Fixes 58ac9a8993a1 ("mm/khugepaged: attempt to map file/shmem-backed pte-mapped THPs by pmds")

From the changelog it isn't clear why a -stable backport is proposed? I assume the userspace-visible runtime effect is a rare and bogus -EEXIST from UFFDIO_CONTINUE?

Quoted 2 lines

Wondering what Lance tested. Was he able to reproduce the bug?

Sashiko might have found another race in there:

	https://sashiko.dev/#/patchset/[email protected]
#2 lore raw
Quoted 23 lines · Andrew Morton
Quoted 8 lines · On Fri, 28 Aug 2026 22:26:40 +0000 James Houghton <[email protected]> wrote:

Userfaultfd minor faults provides userspace with the ability to manually install PTEs with UFFDIO_CONTINUE. Right now, khugepaged collapse can map holes in the VMA when a naturally-aligned THP is present without explicit action from userspace.

This is a problem, as it bypasses userfaultfd minor faults that userspace is expecting to handle.

Quoted 11 lines

If userspace implements post-copy live migration using userfaultfd minor faults, this situation is currently possible:
1. The VMA for guest memory is userfaultfd-minor-registered and nothing is mapped in the page tables.
2. A stale copy of a page is present in a naturally-aligned THP (from pre-copy live migration).
3. khugepaged collapses the mapping of the THP, installs a PMD.
4. The VM now has access to the stale contents => VM is broken.
5. After installing the correct contents, userspace attempts to map the page with UFFDIO_CONTINUE; it gets EEXIST, indicating that something unexpectedly mapped the page.

Quoted 4 lines

Did I read somewhere that this it biting you in a real-life scenario? That would use useful info to spell out.

Or is this a theoretical thing?

So far it is only theoretical (but please do keep reading). I caught this while debugging a related, real problem.

The real problem was that userspace was doing MADV_COLLAPSE (*without* userfaultfd registered) and assuming that page tables wouldn't be created. This is of course incorrect, but for some reason we only started hitting this after moving from a 6.6-based kernel to a 6.18-based kernel. I don't know what change caused this (most likely a downstream change), but the real fix was for userspace to instead do something like:
1. MADV_COLLAPSE (let's assume this must be done)
2. UFFD_REGISTER_MODE_MINOR
3. MADV_DONTNEED
and we're good to go, right? Wrong, in fact!

I noticed that there was indeed a circumstance where khugepaged could create PMDs even when userfaultfd *was* registered, making the userspace fix technically incomplete. That's what this patch fixes.

I haven't yet seen this in our production environment (as far as I am aware, hence "theoretical"); we don't actually have that many VMs running a userfaultfd-based live migration stack anymore though (maybe tens of thousands).

I hope that extra context is helpful.

Quoted 9 lines
Quoted 9 lines

The naturally-aligned THP case is the only case where this is a problem. khugepaged otherwise requires all PTEs to be present for userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is correct. This check is essentially bypassed for naturally-aligned THPs.

No changes are needed for file_backed_vma_is_retractable(), as zapping PTEs is safe. Userspace must already handle cases where PTEs are zapped without explicit action (e.g. due to reclaim).

Quoted 6 lines
Quoted 2 lines
Fixes 58ac9a8993a1 ("mm/khugepaged: attempt to map file/shmem-backed pte-mapped THPs by pmds")

From the changelog it isn't clear why a -stable backport is proposed? I assume the userspace-visible runtime effect is a rare and bogus -EEXIST from UFFDIO_CONTINUE?

Although this bug is likely pretty inconsequential (I'm not sure how many users of userfaultfd minor faults there are), but rarely getting EEXIST back when UFFDIO_CONTINUE is a userspace-visible bug that doesn't really have a great workaround.

Because this bug essentially breaks userfaultfd minor faults, *technically* live migration implemented with it can sometimes corrupt guest memory, and the hypervisor (like QEMU, but QEMU doesn't use minor faults) only learns about it after the fact, after we get EEXIST back. IMO this is enough to backport the patch to stable. But if others disagree, I don't really mind.

Quoted 5 lines
Quoted 3 lines

Wondering what Lance tested. Was he able to reproduce the bug?

I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance, what did you test?

Quoted 3 lines

Sashiko might have found another race in there:

        https://sashiko.dev/#/patchset/[email protected]

Let me get back to you on this. Best case, I'll send another (separate) patch. :)

#3 lore raw
Quoted 6 lines · James Houghton
Quoted 1 line · Andrew Morton

Wondering what Lance tested. Was he able to reproduce the bug?

I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance, what did you test?

Ah, just to be clear, I linked a repro on v1. Here[1] is that repro. Sorry, slipped my mind to also include it in the notes for this v2.

[1] https://gist.github.com/48ca/d399bf534158e80241fb4937ef1ff664

maintainer
#4 lore raw
Quoted 5 lines · On Fri, 28 Aug 2026 18:18:11 -0700 James Houghton <[email protected]> wrote:
Quoted 4 lines · James Houghton
Quoted 1 line · Andrew Morton

Wondering what Lance tested. Was he able to reproduce the bug?

Quoted 7 lines
Quoted 2 lines

I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance, what did you test?

Ah, just to be clear, I linked a repro on v1. Here[1] is that repro. Sorry, slipped my mind to also include it in the notes for this v2.

[1] https://gist.github.com/48ca/d399bf534158e80241fb4937ef1ff664

Cool thanks, I'll slap this into the changelog.

# ./a.out
TAP version 13
1..1
madvise(MADV_COLLAPSE) succeeded
ioctl(UFFDIO_CONTINUE) failed: File exists (errno=17)
not ok 1 - memfd userfaultfd MINOR continue test

We do have a ton of uffd selftesting code in there. Adding this case to uffd-unit-tests.c would presumably be pretty simple. We don't really have a policy on adding a test-case after fixing something - it's common practice but why? We already fixed it!

#5 lore raw
Quoted 17 lines · Andrew Morton
Quoted 4 lines · On Fri, 28 Aug 2026 18:18:11 -0700 James Houghton <[email protected]> wrote:

Ah, just to be clear, I linked a repro on v1. Here[1] is that repro. Sorry, slipped my mind to also include it in the notes for this v2.

[1] https://gist.github.com/48ca/d399bf534158e80241fb4937ef1ff664

Cool thanks, I'll slap this into the changelog.

# ./a.out
TAP version 13
1..1
madvise(MADV_COLLAPSE) succeeded
ioctl(UFFDIO_CONTINUE) failed: File exists (errno=17)
not ok 1 - memfd userfaultfd MINOR continue test
Quoted 4 lines

We do have a ton of uffd selftesting code in there. Adding this case to uffd-unit-tests.c would presumably be pretty simple. We don't really have a policy on adding a test-case after fixing something - it's common practice but why? We already fixed it!

Good idea. The below patch should work; I'll send it properly on Monday.

tools/testing/selftests/mm/uffd-unit-tests.c +57 -5
index ef9b3956bdcf..227bcd6aef9d 100644
@@ -518,19 +518,34 @@ static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uff
 	uffd_wp_fork_pin_test_common(gopts, args, true);
 }
 
-static void check_memory_contents(uffd_global_test_opts_t *gopts, char *p)
+static int __check_memory_contents(unsigned long offset,
+				   unsigned long nr_pages,
+				   uffd_global_test_opts_t *gopts,
+				   char *p)
 {
 	unsigned long i, j;
 	uint8_t expected_byte;
 
-	for (i = 0; i < gopts->nr_pages; ++i) {
+	if (nr_pages + offset < nr_pages)
+		err("overflow in memory check");
+	if (nr_pages + offset > gopts->nr_pages)
+		err("out of bounds memory check");
+
+	for (i = offset; i < nr_pages; ++i) {
 		expected_byte = ~((uint8_t)(i % ((uint8_t)-1)));
 		for (j = 0; j < gopts->page_size; j++) {
 			uint8_t v = *(uint8_t *)(p + (i * gopts->page_size) + j);
 			if (v != expected_byte)
-				err("unexpected page contents");
+				return 1;
 		}
 	}
+
+	return 0;
+}
+
+static int check_memory_contents(uffd_global_test_opts_t *gopts, char *p)
+{
+	return __check_memory_contents(0, gopts->nr_pages, gopts, p);
 }
 
 static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_collapse, bool test_wp)
@@ -539,6 +554,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 	pthread_t uffd_mon;
 	char c = '\0';
 	struct uffd_args args = { 0 };
+	unsigned long checked = 0;
+	bool bad_contents;
 	args.gopts = gopts;
 
 	/*
@@ -564,19 +581,51 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 	if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args))
 		err("uffd_poll_thread create");
 
+	if (test_collapse) {
+		/*
+		 * Read just a single page and try collapsing. The collapse
+		 * should either be rejected or be a no-op.
+		 */
+		if (__check_memory_contents(0, 1, gopts, gopts->area_dst_alias))
+			err("unexpected memory contents before collapse");
+
+		madvise(gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,
+			MADV_COLLAPSE);
+		/*
+		 * If the above collapse mapped pages that were not explicitly
+		 * CONTINUE'd, the below __check_memory_contents() will not
+		 * fault on some pages, resulting in incorrect contents. The
+		 * PTE for the first page may get retracted, so avoid checking
+		 * that page, as we might take a second fault, flipping the
+		 * contents a second time.
+		 */
+		checked = 1;
+	}
+
 	/*
 	 * Read each of the pages back using the UFFD-registered mapping. We
 	 * expect that the first time we touch a page, it will result in a minor
 	 * fault. uffd_poll_thread will resolve the fault by bit-flipping the
 	 * page's contents, and then issuing a CONTINUE ioctl.
 	 */
-	check_memory_contents(gopts, gopts->area_dst_alias);
+	bad_contents = !!__check_memory_contents(checked, gopts->nr_pages - checked,
+						    gopts, gopts->area_dst_alias);
 
 	if (write(gopts->pipefd[1], &c, sizeof(c)) != sizeof(c))
 		err("pipe write");
 	if (pthread_join(uffd_mon, NULL))
 		err("join() failed");
 
+	if (bad_contents && test_collapse) {
+		uffd_test_fail("unexpected memory contents after collapse");
+		return;
+	}
+
+	if (bad_contents) {
+		uffd_test_fail("unexpected memory contents");
+		return;
+	}
+
 	if (test_collapse) {
 		if (madvise(gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,
 			    MADV_COLLAPSE)) {
@@ -593,7 +642,10 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 		 * This won't cause uffd-fault - it purely just makes sure there
 		 * was no corruption.
 		 */
-		check_memory_contents(gopts, gopts->area_dst_alias);
+		if (check_memory_contents(gopts, gopts->area_dst_alias)) {
+			uffd_test_fail("unexpected memory contents");
+			return;
+		}
 	}
 
 	if (args.missing_faults != 0 || args.minor_faults != gopts->nr_pages)
maintainer
#6 lore raw
In reply to James Houghton
Quoted 2 lines · On Fri, 28 Aug 2026 18:09:39 -0700 James Houghton <[email protected]> wrote:

[lots of good stuff]

OK, thanks.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Wondering what Lance tested. Was he able to reproduce the bug?

I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance, what did you test?

Wait, there was a reproducer?

Oh yeah, found it. You changed the patch Subject: between versions, dangnabbit.

I asked Gemini to write me a reproducer and it basically said "sorry andrew, I can't do that". ChatGPT was much more accommodating but after a few iterations it was still pretty mind-bending. I came away thinking "nobody has ever hit this".

But that's not the point. Now that it's out there and known about, Black Hats might be working away to find and utilize a reproducer, so yes, we backport it.

Quoted 6 lines
Quoted 3 lines

Sashiko might have found another race in there:

        https://sashiko.dev/#/patchset/[email protected]

Let me get back to you on this. Best case, I'll send another (separate) patch. :)

Great, another patch ;)

reviewer
#7 lore raw
In reply to James Houghton
Quoted 1 line · James Houghton

On Fri, Aug 28, 2026 at 5:33 PM Andrew Morton <[email protected]> wrote:

[...]

Quoted 1 line
Quoted 1 line

Wondering what Lance tested. Was he able to reproduce the bug?

Yes!

Quoted 2 lines

I assume he just re-ran my repro, but yeah I'm not sure, heh. Lance, what did you test?

In general, I prefer to write my own reproducer from scratch, using whatever approach works. That's what I did here.

On the parent, MADV_COLLAPSE installed a PMD, and a later UFFDIO_CONTINUE failed with EEXIST. With v1 applied, MADV_COLLAPSE was rejected and UFFDIO_CONTINUE succeeded when registration completed before collapse.

I then extended the reproducer to register the VMA during the mmap lock drop/reacquire window in collapse. With v1, a PMD was still installed, and the later UFFDIO_CONTINUE again failed with EEXIST. Adding a userfaultfd_minor() recheck after reacquiring the lock closed that race.

Before reviewing v2, let's go back to v1 [1] and settle the expected semantics there first. The check depends on which policy we choose, so I'd hold off on reviewing v2 until that is resolved :)

[1] https://lore.kernel.org/lkml/[email protected]/

Quoted 6 lines
Quoted 3 lines

Sashiko might have found another race in there:

        https://sashiko.dev/#/patchset/[email protected]

Let me get back to you on this. Best case, I'll send another (separate) patch. :)

@James please don't send another revision or a separate patch for now.

Cheers, Lance