mm/khugepaged: Never install PMDs in uffd-minor-registered VMAs
Subsystems (5)
PATCH v3 1/2 mm/khugepaged: Never install PMDs in uffd-minor-registered VMAs
Userfaultfd minor faults provides userspace with the ability to manually install PTEs with UFFDIO_CONTINUE. Right now, MADV_COLLAPSE can map holes in the VMA when a naturally-aligned THP is present. This is not true for khugepaged collapse: the PTEs will be retracted, but a PMD will not be installed.
When MADV_COLLAPSE installs a PMD that mapped holes in the VMA, userspace is likely to expect UFFDIO_CONTINUE to succeed on the should-be holes. UFFDIO_CONTINUE will fail and return EEXIST.
This is not inherently a problem, as MADV_COLLAPSE is an explicit userspace action. But, especially because MADV_COLLAPSE can be invoked by an external process via process_madvise(), a rogue caller could break a userfaultfd-minor resolver thread. If khugepaged is later updated to install PMDs for khugepaged collapsing, that would be a genuine problem.
Userspace cannot generally use MADV_COLLAPSE to resolve userfault minor faults, as MADV_COLLAPSE will only resolve such faults if a naturally-aligned THP is present.
The naturally-aligned THP case is the only case where this quirk exists. Collapsing otherwise requires all PTEs to be present for userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is correct. This check is essentially bypassed for naturally-aligned THPs.
---
v2: https://lore.kernel.org/linux-mm/[email protected]/
Changes since v2:
- Drop cc:stable and update the changelog for the main patch. khugepaged doesn't actually install PMDs, so it is impossible for the kernel to silently install mappings that userspace didn't ask for. Therefore this patch does not need to be backported.
- Update the uffd-unit-tests selftest to account for the new behavior (new patch). This patched selftest is equivalent to the reproducer I provided in v1.
v1: https://lore.kernel.org/linux-mm/[email protected]/
--- mm/khugepaged.c | 7 +++++++ 1 file changed, 7 insertions(+)
mm/khugepaged.c +7 -0
@@ -1894,6 +1894,13 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsignif (userfaultfd_protected (vma))return SCAN_PTE_UFFD ; +/* +* Userfaultfd-minor-registered VMAs should not be collapsed, as +* userspace is expecting to explicitly install PTEs. +*/ +if (userfaultfd_minor (vma)) +return SCAN_PTE_UFFD ; + folio =filemap_lock_folio (vma->vm_file->f_mapping,linear_page_index (vma, haddr));if (IS_ERR (folio))
base-commit: 5acbae5f7eb3d5275120abfe698c394b7325dcec
Signature · 1 line
2.55.0.1007.g17ff1f9808-goog
PATCH v3 2/2 mm: selftests: Adjust the MADV_COLLAPSE uffd-minor selftests
The behavior of MADV_COLLAPSE with uffd-minor VMAs has changed:MADV_COLLAPSE will no longer install PTEs where none existed before. Update the selftest to demonstrate this new behavior.
On an unpatched kernel, the test will hit "unexpected memory contents after collapse".
If the selftest is left unpatched but the kernel is patched, the selftest will SKIP when it gets EINVAL back from MADV_COLLAPSE.
--- tools/testing/selftests/mm/uffd-unit-tests.c | 75 ++++++++++++++++++-- 1 file changed, 68 insertions(+), 7 deletions(-)
tools/testing/selftests/mm/uffd-unit-tests.c +68 -7
@@ -518,19 +518,34 @@ static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uffuffd_wp_fork_pin_test_common (gopts, args, true); } -static void check_memory_contents (uffd_global_test_opts_t *gopts,char *p) +static int __check_memory_contents (unsigned long offset, +unsigned long nr_pages, +uffd_global_test_opts_t *gopts, +char *p) {unsigned long i, j;uint8_t expected_byte; -for (i =0 ; i < gopts->nr_pages; ++i) { +if (nr_pages + offset < nr_pages) +err ("overflow in memory check" ); +if (nr_pages + offset > gopts->nr_pages) +err ("out of bounds memory check" ); + +for (i = offset; i < offset + nr_pages; ++i) { expected_byte = ~((uint8_t )(i % ((uint8_t )-1 )));for (j =0 ; j < gopts->page_size; j++) {uint8_t v = *(uint8_t *)(p + (i * gopts->page_size) + j);if (v != expected_byte) -err ("unexpected page contents" ); +return 1 ; } } + +return 0 ; +} + +static int check_memory_contents (uffd_global_test_opts_t *gopts,char *p) +{ +return __check_memory_contents (0 , gopts->nr_pages, gopts, p); }static void uffd_minor_test_common (uffd_global_test_opts_t *gopts,bool test_collapse,bool test_wp)
@@ -539,6 +554,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_colpthread_t uffd_mon;char c ='\0' ;struct uffd_args args = {0 }; +unsigned long checked =0 ; +bool bad_contents; args.gopts = gopts;/*
@@ -564,24 +581,65 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_colif (pthread_create (&uffd_mon,NULL , uffd_poll_thread, &args))err ("uffd_poll_thread create" ); +if (test_collapse) { +/* +* Read just a single page and try collapsing. The collapse +* should either be rejected or be a no-op. +*/ +if (__check_memory_contents (0 ,1 , gopts, gopts->area_dst_alias)) +err ("unexpected memory contents before collapse" ); + +/* MADV_COLLAPSE might return EINVAL for uffd-minor VMAs. */ +madvise (gopts->area_dst_alias, gopts->nr_pages * gopts->page_size, +MADV_COLLAPSE ); +/* +* If the above collapse mapped pages that were not explicitly +* CONTINUE'd, the below __check_memory_contents() will not +* fault on some pages, resulting in incorrect contents. The +* PTE for the first page may get retracted, so avoid checking +* that page, as we might take a second fault, flipping the +* contents a second time. +*/ + checked =1 ; + } +/* * Read each of the pages back using the UFFD-registered mapping. We * expect that the first time we touch a page, it will result in a minor * fault. uffd_poll_thread will resolve the fault by bit-flipping the * page's contents, and then issuing a CONTINUE ioctl. */ -check_memory_contents (gopts, gopts->area_dst_alias); + bad_contents = !!__check_memory_contents (checked, gopts->nr_pages - checked, + gopts, gopts->area_dst_alias);if (write (gopts->pipefd[1 ], &c,sizeof (c)) !=sizeof (c))err ("pipe write" );if (pthread_join (uffd_mon,NULL ))err ("join() failed" ); +if (bad_contents && test_collapse) { +uffd_test_fail ("unexpected memory contents after collapse" ); +return ; + } + +if (bad_contents) { +uffd_test_fail ("unexpected memory contents" ); +return ; + } +if (test_collapse) { +/* +* MADV_COLLAPSE will fail unless userfaultfd-minor is +* unregistered. +*/ +if (uffd_unregister (gopts->uffd, gopts->area_dst_alias, + gopts->nr_pages * gopts->page_size)) +err ("uffd_unregister before MADV_COLLAPSE failed" ); + +/* MADV_COLLAPSE should succeed with userfaultfd unregistered. */ if (madvise (gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,MADV_COLLAPSE )) { -/* It's fine to fail for this one... */ -uffd_test_skip ("MADV_COLLAPSE failed" ); +uffd_test_fail ("MADV_COLLAPSE failed" );return ; }
@@ -593,7 +651,10 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col* This won't cause uffd-fault - it purely just makes sure there * was no corruption. */ -check_memory_contents (gopts, gopts->area_dst_alias); +if (check_memory_contents (gopts, gopts->area_dst_alias)) { +uffd_test_fail ("unexpected memory contents" ); +return ; + } }if (args.missing_faults !=0 || args.minor_faults != gopts->nr_pages)
Signature · 1 line
2.55.0.1007.g17ff1f9808-goog
David Hildenbrand (Arm)
maintainerRe: [PATCH v3 2/2] mm: selftests: Adjust the MADV_COLLAPSE uffd-minor selftests
Quoted 2 lines · James Houghton
The behavior of
MADV_COLLAPSEwith uffd-minor VMAs has changed:MADV_COLLAPSEwill no longer install PTEs where none existed before.
Misleading: MADV_COLLAPSE installed a PMD not PTEs?
Quoted 18 lines
Update the selftest to demonstrate this new behavior.
On an unpatched kernel, the test will hit "unexpected memory contents after collapse".
If the selftest is left unpatched but the kernel is patched, the selftest will SKIP when it gets EINVAL back from
MADV_COLLAPSE.Signed-off-by James Houghton <[email protected]>--- tools/testing/selftests/mm/uffd-unit-tests.c | 75 ++++++++++++++++++-- 1 file changed, 68 insertions(+), 7 deletions(-)tools/testing/selftests/mm/uffd-unit-tests.c +0
-0
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uff · line 524
Quoted 8 lines
tools/testing/selftests/mm/uffd-unit-tests.c +4
-1@@ -518,19 +518,34 @@ static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uffuffd_wp_fork_pin_test_common (gopts, args, true); } -static void check_memory_contents (uffd_global_test_opts_t *gopts,char *p) +static int __check_memory_contents (unsigned long offset, +unsigned long nr_pages, +uffd_global_test_opts_t *gopts, +char *p)
Two tabs look better ;)
Quoted 30 lines
tools/testing/selftests/mm/uffd-unit-tests.c +14
-2{unsigned long i, j;uint8_t expected_byte; -for (i =0 ; i < gopts->nr_pages; ++i) { +if (nr_pages + offset < nr_pages) +err ("overflow in memory check" ); +if (nr_pages + offset > gopts->nr_pages) +err ("out of bounds memory check" ); + +for (i = offset; i < offset + nr_pages; ++i) { expected_byte = ~((uint8_t )(i % ((uint8_t )-1 )));for (j =0 ; j < gopts->page_size; j++) {uint8_t v = *(uint8_t *)(p + (i * gopts->page_size) + j);if (v != expected_byte) -err ("unexpected page contents" ); +return 1 ; } } + +return 0 ; +} + +static int check_memory_contents (uffd_global_test_opts_t *gopts,char *p) +{ +return __check_memory_contents (0 , gopts->nr_pages, gopts, p); }static void uffd_minor_test_common (uffd_global_test_opts_t *gopts,bool test_collapse,bool test_wp)
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col · line 558
Quoted 5 lines
tools/testing/selftests/mm/uffd-unit-tests.c +2
-0@@ -539,6 +554,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_colpthread_t uffd_mon;char c ='\0' ;struct uffd_args args = {0 }; +unsigned long checked =0 ; +bool bad_contents;
Through in an empty line. Or better
struct uffd_args args = { .gopts = gopts };
Quoted 7 lines
tools/testing/selftests/mm/uffd-unit-tests.c +1
-0/* @@ -564,24 +581,65 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_colif (pthread_create (&uffd_mon,NULL , uffd_poll_thread, &args))err ("uffd_poll_thread create" ); +if (test_collapse) {
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col · line 594
Quoted 10 lines
tools/testing/selftests/mm/uffd-unit-tests.c +10
-0+/* +* Read just a single page and try collapsing. The collapse +* should either be rejected or be a no-op. +*/ +if (__check_memory_contents (0 ,1 , gopts, gopts->area_dst_alias)) +err ("unexpected memory contents before collapse" ); + +/* MADV_COLLAPSE might return EINVAL for uffd-minor VMAs. */ +madvise (gopts->area_dst_alias, gopts->nr_pages * gopts->page_size, +MADV_COLLAPSE );
Add an explicit check for expected errnos?
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col · line 603
Quoted 9 lines
tools/testing/selftests/mm/uffd-unit-tests.c +9
-0+/* +* If the above collapse mapped pages that were not explicitly +* CONTINUE'd, the below __check_memory_contents() will not +* fault on some pages, resulting in incorrect contents. The +* PTE for the first page may get retracted, so avoid checking +* that page, as we might take a second fault, flipping the +* contents a second time. +*/ + checked =1 ;
I'm not an expert on this code. But a trivial check to see whether there is something in the page tables that shouldn't be there is through the help of pagemap_is_populated().
Maybe that would help to simplify things, not sure.
Signature · 3 lines
Cheers, David
Re: [PATCH v3 2/2] mm: selftests: Adjust the MADV_COLLAPSE uffd-minor selftests
Quoted 6 lines · David Hildenbrand (Arm)
Quoted 2 lines · James Houghton
The behavior of
MADV_COLLAPSEwith uffd-minor VMAs has changed:MADV_COLLAPSEwill no longer install PTEs where none existed before.Misleading:
MADV_COLLAPSEinstalled a PMD not PTEs?
Yeah, that's fair. I've updated it to this:
The behavior of MADV_COLLAPSE with uffd-minor VMAs has changed. Previously, MADV_COLLAPSE would install PMDs in parts of a VMA where PTEs may be missing, essentially suppressing userfaultfd minor faults. Now, MADV_COLLAPSE returns -EINVAL for uffd-minor VMAs. Update the selftest to demonstrate this new behavior.
Quoted 22 lines
Quoted 22 lines
Update the selftest to demonstrate this new behavior.
On an unpatched kernel, the test will hit "unexpected memory contents after collapse".
If the selftest is left unpatched but the kernel is patched, the selftest will SKIP when it gets EINVAL back from
MADV_COLLAPSE.Signed-off-by James Houghton <[email protected]>--- tools/testing/selftests/mm/uffd-unit-tests.c | 75 ++++++++++++++++++-- 1 file changed, 68 insertions(+), 7 deletions(-)tools/testing/selftests/mm/uffd-unit-tests.c +0
-0@@ -518,19 +518,34 @@ static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uffuffd_wp_fork_pin_test_common (gopts, args, true); }
Quoted 7 lines
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uff · line 524
Quoted 5 lines
tools/testing/selftests/mm/uffd-unit-tests.c +4
-1-static void check_memory_contents (uffd_global_test_opts_t *gopts,char *p) +static int __check_memory_contents (unsigned long offset, +unsigned long nr_pages, +uffd_global_test_opts_t *gopts, +char *p)Two tabs look better ;)
Fixed (I think).
Quoted 39 lines
Quoted 31 lines
tools/testing/selftests/mm/uffd-unit-tests.c +14
-2{unsigned long i, j;uint8_t expected_byte; -for (i =0 ; i < gopts->nr_pages; ++i) { +if (nr_pages + offset < nr_pages) +err ("overflow in memory check" ); +if (nr_pages + offset > gopts->nr_pages) +err ("out of bounds memory check" ); + +for (i = offset; i < offset + nr_pages; ++i) { expected_byte = ~((uint8_t )(i % ((uint8_t )-1 )));for (j =0 ; j < gopts->page_size; j++) {uint8_t v = *(uint8_t *)(p + (i * gopts->page_size) + j);if (v != expected_byte) -err ("unexpected page contents" ); +return 1 ; } } + +return 0 ; +} + +static int check_memory_contents (uffd_global_test_opts_t *gopts,char *p) +{ +return __check_memory_contents (0 , gopts->nr_pages, gopts, p); }static void uffd_minor_test_common (uffd_global_test_opts_t *gopts,bool test_collapse,bool test_wp)tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col · line 558
Quoted 5 lines
tools/testing/selftests/mm/uffd-unit-tests.c +2
-0@@ -539,6 +554,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_colpthread_t uffd_mon;char c ='\0' ;struct uffd_args args = {0 }; +unsigned long checked =0 ; +bool bad_contents;Through in an empty line. Or better
Quoted 1 line
struct
uffd_argsargs = { .gopts = gopts };
Done. Thanks.
Quoted 6 lines
Quoted 6 lines
tools/testing/selftests/mm/uffd-unit-tests.c +0
-0/* @@ -564,24 +581,65 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_colif (pthread_create (&uffd_mon,NULL , uffd_poll_thread, &args))err ("uffd_poll_thread create" );
Quoted 13 lines
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col · line 594
Quoted 11 lines
tools/testing/selftests/mm/uffd-unit-tests.c +11
-0+if (test_collapse) { +/* +* Read just a single page and try collapsing. The collapse +* should either be rejected or be a no-op. +*/ +if (__check_memory_contents (0 ,1 , gopts, gopts->area_dst_alias)) +err ("unexpected memory contents before collapse" ); + +/* MADV_COLLAPSE might return EINVAL for uffd-minor VMAs. */ +madvise (gopts->area_dst_alias, gopts->nr_pages * gopts->page_size, +MADV_COLLAPSE );Add an explicit check for expected errnos?
Done (just EINVAL).
Quoted 11 lines
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col · line 603
Quoted 10 lines
tools/testing/selftests/mm/uffd-unit-tests.c +9
-0+/* +* If the above collapse mapped pages that were not explicitly +* CONTINUE'd, the below __check_memory_contents() will not +* fault on some pages, resulting in incorrect contents. The +* PTE for the first page may get retracted, so avoid checking +* that page, as we might take a second fault, flipping the +* contents a second time. +*/ + checked =1 ;
Quoted 5 lines
I'm not an expert on this code. But a trivial check to see whether there is something in the page tables that shouldn't be there is through the help of
pagemap_is_populated().Maybe that would help to simplify things, not sure.
I've reworded the explanation here. I think it's better just to have this checked stuff rather than checking pagemap_is_populated(). Either way, we need to conditionally skip checking the first page, that's the real annoyance.
New explanation/comment:
If the above collapse mapped pages that were not explicitly CONTINUE'd, the below __check_memory_contents() will not fault on some pages, resulting in incorrect contents.
In the correct case, when MADV_COLLAPSE doesn't map pages that were not mapped before, it may still retract existing PTEs for the region. Reaccesses to previously userfault'ed pages will trigger a *second* userfault and flip the contents again, which appears as a missed userfault. Ignore the page we already accessed to avoid this.
Andrew Morton
maintainerQuoted 25 lines · On Thu, 10 Sep 2026 02:34:10 +0000 James Houghton <[email protected]> wrote:
Userfaultfd minor faults provides userspace with the ability to manually install PTEs with
UFFDIO_CONTINUE. Right now,MADV_COLLAPSEcan map holes in the VMA when a naturally-aligned THP is present. This is not true for khugepaged collapse: the PTEs will be retracted, but a PMD will not be installed.When
MADV_COLLAPSEinstalls a PMD that mapped holes in the VMA, userspace is likely to expectUFFDIO_CONTINUEto succeed on the should-be holes.UFFDIO_CONTINUEwill fail and return EEXIST.This is not inherently a problem, as
MADV_COLLAPSEis an explicit userspace action. But, especially becauseMADV_COLLAPSEcan be invoked by an external process viaprocess_madvise(), a rogue caller could break a userfaultfd-minor resolver thread. If khugepaged is later updated to install PMDs for khugepaged collapsing, that would be a genuine problem.Userspace cannot generally use
MADV_COLLAPSEto resolve userfault minor faults, asMADV_COLLAPSEwill only resolve such faults if a naturally-aligned THP is present.The naturally-aligned THP case is the only case where this quirk exists. Collapsing otherwise requires all PTEs to be present for userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is correct. This check is essentially bypassed for naturally-aligned THPs.
Quoted 6 lines
...
- Drop cc:stable and update the changelog for the main patch. khugepaged doesn't actually install PMDs, so it is impossible for the kernel to silently install mappings that userspace didn't ask for. Therefore this patch does not need to be backported.
Thanks for explaining that. It helps. My life :(
Sashiko had a stupid niggle about the selftest change and made quite disturbing claims about the existing try_collapse_pte_mapped_thp() code:
https://sashiko.dev/#/patchset/[email protected]
I'll queue the series for test-n-review.
Quoted 4 lines · Andrew Morton
Sashiko had a stupid niggle about the selftest change and made quite disturbing claims about the existing
try_collapse_pte_mapped_thp()code:https://sashiko.dev/#/patchset/[email protected]
Thanks, it looks like the note it wrote about the selftest is genuine; it is okay for MADV_COLLAPSE to fail. So I shouldn't have changed uffd_test_skip() to uffd_test_fail().
Andrew, if you can, please add this diff as part of the patch to queue. (It's a partial revert.) Or I could send the a new version of the patch as a reply to patch #2 email. Or a whole v4, whatever's easiest.
tools/testing/selftests/mm/uffd-unit-tests.c +2 -1
@@ -639,7 +639,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col/* MADV_COLLAPSE should succeed with userfaultfd unregistered. */ if (madvise (gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,MADV_COLLAPSE )) { -uffd_test_fail ("MADV_COLLAPSE failed" ); +/* It's fine to fail for this one... */ +uffd_test_skip ("MADV_COLLAPSE failed" );return ; }
With respect to the note about try_collapse_pte_mapped_thp() breaking POSIX SIGBUS semantics, it definitely seems plausible. I'll look into it.
Quoted 1 line
I'll queue the series for test-n-review.
Thanks!
Quoted 20 lines · James Houghton
Quoted 4 lines · Andrew Morton
Sashiko had a stupid niggle about the selftest change and made quite disturbing claims about the existing
try_collapse_pte_mapped_thp()code:https://sashiko.dev/#/patchset/[email protected]Thanks, it looks like the note it wrote about the selftest is genuine; it is okay for
MADV_COLLAPSEto fail. So I shouldn't have changeduffd_test_skip()touffd_test_fail().Andrew, if you can, please add this diff as part of the patch to queue. (It's a partial revert.) Or I could send the a new version of the patch as a reply to patch #2 email. Or a whole v4, whatever's easiest.
tools/testing/selftests/mm/uffd-unit-tests.c +0
-0
tools/testing/selftests/mm/uffd-unit-tests.c static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col · line 645
Quoted 8 lines
tools/testing/selftests/mm/uffd-unit-tests.c +2
-1@@ -639,7 +639,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col/* MADV_COLLAPSE should succeed with userfaultfd unregistered. */ if (madvise (gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,MADV_COLLAPSE )) { -uffd_test_fail ("MADV_COLLAPSE failed" ); +/* It's fine to fail for this one... */ +uffd_test_skip ("MADV_COLLAPSE failed" );return ; }
Ah, the
/* MADV_COLLAPSE should succeed with userfaultfd unregistered. */
comment at the top of that hunk should also be removed.
Okay so it appears to be a real bug. Here's a diff of what I think would fix it, though I haven't tested it. I've also included an AI-generated selftest that checks the behavior when you MADV_COLLAPSE a VMA that maps a shmem file that had a THP and got truncated.
I'm happy to send this patch on its own to see what others think. I'll test it properly tomorrow.
mm/khugepaged.c +6 -0
@@ -1863,6 +1863,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsignpte_t *start_pte, *pte;pmd_t *pmd, pgt_pmd;spinlock_t *pml =NULL , *ptl; +pgoff_t file_end;int i;mmap_assert_locked (mm);
@@ -1901,6 +1902,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsignif (userfaultfd_minor (vma))return SCAN_PTE_UFFD ; +/* Do not map pages past the end of the file. */ + file_end =DIV_ROUND_UP (i_size_read (file_inode (vma->vm_file)),PAGE_SIZE ); +if (linear_page_index (vma, haddr) +HPAGE_PMD_NR > file_end) +return SCAN_TRUNCATED ; + folio =filemap_lock_folio (vma->vm_file->f_mapping,linear_page_index (vma, haddr));if (IS_ERR (folio))
tools/testing/selftests/mm/Makefile +1 -0
@@ -104,6 +104,7 @@ TEST_GEN_FILES += guard-regions TEST_GEN_FILES += merge TEST_GEN_FILES += rmap TEST_GEN_FILES += folio_split_race_test +TEST_GEN_FILES += collapse_truncate ifneq ($(ARCH),arm64) TEST_GEN_FILES += soft-dirty
tools/testing/selftests/mm/collapse_truncate.c new file +276 -0
276 lines not on this page · raw message
David Hildenbrand (Arm)
maintainerQuoted 5 lines · James Houghton
Userfaultfd minor faults provides userspace with the ability to manually install PTEs with
UFFDIO_CONTINUE. Right now,MADV_COLLAPSEcan map holes in the VMA when a naturally-aligned THP is present. This is not true for khugepaged collapse: the PTEs will be retracted, but a PMD will not be installed.
You might want to add a breadcrumb here that collapse_file() never returns SCAN_PTE_MAPPED_HUGEPAGE for khugepaged.
The whole thing makes me wonder why that khugepaged check is buried so deep down in that function. Makes me wonder whether that check can actually get pulled further out (maybe Kiryl's patches do that).
try_collapse_pte_mapped_thp(mm, addr, !cc->is_khugepaged);
is rather confusing when we never end up there with khugepaged ...
Quoted 44 lines
When
MADV_COLLAPSEinstalls a PMD that mapped holes in the VMA, userspace is likely to expectUFFDIO_CONTINUEto succeed on the should-be holes.UFFDIO_CONTINUEwill fail and return EEXIST.This is not inherently a problem, as
MADV_COLLAPSEis an explicit userspace action. But, especially becauseMADV_COLLAPSEcan be invoked by an external process viaprocess_madvise(), a rogue caller could break a userfaultfd-minor resolver thread. If khugepaged is later updated to install PMDs for khugepaged collapsing, that would be a genuine problem.Userspace cannot generally use
MADV_COLLAPSEto resolve userfault minor faults, asMADV_COLLAPSEwill only resolve such faults if a naturally-aligned THP is present.The naturally-aligned THP case is the only case where this quirk exists. Collapsing otherwise requires all PTEs to be present for userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is correct. This check is essentially bypassed for naturally-aligned THPs.
Suggested-by Lance Yang <[email protected]>Tested-by Lance Yang <[email protected]>Signed-off-by James Houghton <[email protected]>---
v2: https://lore.kernel.org/linux-mm/[email protected]/
Changes since v2:
- Drop cc:stable and update the changelog for the main patch. khugepaged doesn't actually install PMDs, so it is impossible for the kernel to silently install mappings that userspace didn't ask for. Therefore this patch does not need to be backported.
- Update the uffd-unit-tests selftest to account for the new behavior (new patch). This patched selftest is equivalent to the reproducer I provided in v1.v1: https://lore.kernel.org/linux-mm/[email protected]/
--- mm/khugepaged.c | 7 +++++++ 1 file changed, 7 insertions(+)mm/khugepaged.c +0
-0
mm/khugepaged.c static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign · line 1902
Quoted 9 lines
mm/khugepaged.c +6
-0@@ -1894,6 +1894,13 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsignif (userfaultfd_protected (vma))return SCAN_PTE_UFFD ; +/* +* Userfaultfd-minor-registered VMAs should not be collapsed, as +* userspace is expecting to explicitly install PTEs. +*/ +if (userfaultfd_minor (vma)) +return SCAN_PTE_UFFD ;
Can we just combine the two uffd checks to have less uffd noise? :)
"Don't collapse if we might have UFFD markers or if userspace fills the page tables through uffd minor faults".
Overall LGTM
Signature · 3 lines
Cheers, David
Quoted 10 lines · David Hildenbrand (Arm)
Quoted 5 lines · James Houghton
Userfaultfd minor faults provides userspace with the ability to manually install PTEs with
UFFDIO_CONTINUE. Right now,MADV_COLLAPSEcan map holes in the VMA when a naturally-aligned THP is present. This is not true for khugepaged collapse: the PTEs will be retracted, but a PMD will not be installed.You might want to add a breadcrumb here that
collapse_file()never returnsSCAN_PTE_MAPPED_HUGEPAGEfor khugepaged.
collapse_file() does return SCAN_PTE_MAPPED_HUGEPAGE even when called by khugepaged though. (right?)
Quoted 8 lines
The whole thing makes me wonder why that khugepaged check is buried so deep down in that function. Makes me wonder whether that check can actually get pulled further out (maybe Kiryl's patches do that).
try_collapse_pte_mapped_thp(mm, addr, !cc->is_khugepaged);is rather confusing when we never end up there with khugepaged ...
Hmm... it's not confusing to me; khugepaged does in fact call it. It's just that, for khugepaged, a PMD is never installed.
Kiryl's series made the "do not install a PMD" choice part of the collapse_policy, so I think it's a bit clearer now.
Quoted 57 lines
Quoted 45 lines
When
MADV_COLLAPSEinstalls a PMD that mapped holes in the VMA, userspace is likely to expectUFFDIO_CONTINUEto succeed on the should-be holes.UFFDIO_CONTINUEwill fail and return EEXIST.This is not inherently a problem, as
MADV_COLLAPSEis an explicit userspace action. But, especially becauseMADV_COLLAPSEcan be invoked by an external process viaprocess_madvise(), a rogue caller could break a userfaultfd-minor resolver thread. If khugepaged is later updated to install PMDs for khugepaged collapsing, that would be a genuine problem.Userspace cannot generally use
MADV_COLLAPSEto resolve userfault minor faults, asMADV_COLLAPSEwill only resolve such faults if a naturally-aligned THP is present.The naturally-aligned THP case is the only case where this quirk exists. Collapsing otherwise requires all PTEs to be present for userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is correct. This check is essentially bypassed for naturally-aligned THPs.
Suggested-by Lance Yang <[email protected]>Tested-by Lance Yang <[email protected]>Signed-off-by James Houghton <[email protected]>---
v2: https://lore.kernel.org/linux-mm/[email protected]/
Changes since v2:
- Drop cc:stable and update the changelog for the main patch. khugepaged doesn't actually install PMDs, so it is impossible for the kernel to silently install mappings that userspace didn't ask for. Therefore this patch does not need to be backported.
- Update the uffd-unit-tests selftest to account for the new behavior (new patch). This patched selftest is equivalent to the reproducer I provided in v1.v1: https://lore.kernel.org/linux-mm/[email protected]/
--- mm/khugepaged.c | 7 +++++++ 1 file changed, 7 insertions(+)mm/khugepaged.c +0
-0mm/khugepaged.c static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign · line 1902
Quoted 9 lines
mm/khugepaged.c +6
-0@@ -1894,6 +1894,13 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsignif (userfaultfd_protected (vma))return SCAN_PTE_UFFD ; +/* +* Userfaultfd-minor-registered VMAs should not be collapsed, as +* userspace is expecting to explicitly install PTEs. +*/ +if (userfaultfd_minor (vma)) +return SCAN_PTE_UFFD ;Can we just combine the two uffd checks to have less uffd noise? :)
Quoted 2 lines
"Don't collapse if we might have UFFD markers or if userspace fills the page tables through uffd minor faults".
I've gone with this wording:
/*
* Don't collapse if there might be PTE markers for userfaultfd-based
* access protection or if collapsing might bypass userfaultfd minor
* faults.
*/
if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
return SCAN_PTE_UFFD;Quoted 2 lines
Overall LGTM
Thanks David. :) Sorry for the delay. I'll send a new version tomorrow (on top of Kiryl's patches / mm-unstable).