mm/ksm: keep soft-dirty and uffd-wp when merging a page
Subsystems (4)
PATCH v2 1/1 mm/ksm: keep soft-dirty and uffd-wp when merging a page
replace_page() maps the KSM page with a PTE built from scratch, so the soft-dirty and uffd bits of the PTE it replaces are lost. Due to this, a write before the merge no longer shows as soft-dirty, the next write to a uffd-wp page raises no write-protect fault, and because the page is mapped with the VMA's normal permissions, on a uffd-rwp VMA the next access raises no RWP fault.
Copy the soft-dirty and uffd bits from the original PTE to the new PTE, and restore PAGE_NONE for RWP VMAs, as try_to_map_unused_to_zeropage() does.
---
Notes:
v1: https://lore.kernel.org/all/[email protected]/
Changes since v1 (thanks, Lance):
- Fixes: point the uffd-wp and RWP tags at the commits that make the registration modes reachable (63b2d4174c4a, 2d427bb01639).
- Shorten the comment; the uffd bit carries RWP state too.
- Add Lance's Tested-by.
mm/ksm.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
mm/ksm.c +12 -0
@@ -34,6 +34,7 @@#include <linux/memory.h> #include <linux/mmu_notifier.h> #include <linux/swap.h> +#include <linux/userfaultfd_k.h> #include <linux/ksm.h> #include <linux/hashtable.h> #include <linux/freezer.h>
@@ -1463,6 +1464,17 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,dec_mm_counter (mm,MM_ANONPAGES ); } +/* +* Preserve soft-dirty and uffd bits when replacing the page. +* Restore PAGE_NONE for RWP-protected PTEs. +*/ +if (pte_soft_dirty (orig_pte)) + newpte =pte_mksoft_dirty (newpte); +if (pte_uffd (orig_pte)) + newpte =pte_mkuffd (newpte); +if (pte_uffd (orig_pte) &&userfaultfd_rwp (vma)) + newpte =pte_modify (newpte,PAGE_NONE ); +flush_cache_page (vma, addr,pte_pfn (ptep_get (ptep)));/* * No need to notify as we are replacing a read only page with another
base-commit: 48fe11c8d0f1e4e59426a3606f20ac1130b6d8dc
Signature · 1 line
2.53.0
Quoted 20 lines · Samarth Narang
replace_page()maps the KSM page with a PTE built from scratch, so the soft-dirty and uffd bits of the PTE it replaces are lost. Due to this, a write before the merge no longer shows as soft-dirty, the next write to a uffd-wp page raises no write-protect fault, and because the page is mapped with the VMA's normal permissions, on a uffd-rwp VMA the next access raises no RWP fault.Copy the soft-dirty and uffd bits from the original PTE to the new PTE, and restore
PAGE_NONEfor RWP VMAs, astry_to_map_unused_to_zeropage()does.Fixes 0f8975ec4db2 ("mm: soft-dirty bits for user memory changes tracking")Fixes 63b2d4174c4a ("userfaultfd: wp: add the writeprotect API to userfaultfd ioctl")Fixes 2d427bb01639 ("mm/userfaultfd: add RWP fault delivery and expose UFFDIO_REGISTER_MODE_RWP")Tested-by Lance Yang <[email protected]>Assisted-by LLMSigned-off-by Samarth Narang <[email protected]>---
Quoted 8 lines
Notes:
v1: https://lore.kernel.org/all/[email protected]/Changes since v1 (thanks, Lance):
- Fixes: point the uffd-wp and RWP tags at the commits that make the registration modes reachable (63b2d4174c4a,2d427bb01639).
- Shorten the comment; the uffd bit carries RWP state too.
- Add Lance's Tested-by.
Generally, no need to rush the next version out, please. Let it soak for a few days so folks have time to review properly, instead of making everyone start over every time. Just saying :)
Cheers, Lance
👍
Samarth Narang reacted via Gmail
<https://www.google.com/gmail/about/?utm_source=gmail-in-product&utm_medium=et&utm_campaign=emojireactionemail#app>
{
"emoji": "👍",
"version": 1
}
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] mm/ksm: installing PAGE_NONE in replace_page() breaks break_ksm() on RWP PTEs - [Medium] mm/ksm…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] mm/ksm: installing PAGE_NONE in replace_page() breaks break_ksm() on RWP PTEs
- [Medium] mm/ksm: replace_page() sets soft-dirty on clean KSM zero-page PTEs --
commit e9f8ba7271894cc9b005a46048d0bcc61cff0fff
Author: Samarth Narang <[email protected]>
mm/ksm: keep soft-dirty and uffd-wp when merging a page
This commit preserves the soft-dirty and userfaultfd bits when replace_page() installs a KSM PTE in place of an anonymous page. For VMAs registered with userfaultfd RWP, it also modifies the new PTE with PAGE_NONE when the uffd bit is set.
Quoted 4 lines
mm/ksm.c +0
-0
[ ... ]
mm/ksm.c static int replace_page(struct vm_area_struct *vma, struct page *page, · line 1472
Quoted 10 lines
mm/ksm.c +6
-0@@ -1463,6 +1464,17 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,dec_mm_counter (mm,MM_ANONPAGES ); } +/* +* Preserve soft-dirty and uffd bits when replacing the page. +* Restore PAGE_NONE for RWP-protected PTEs. +*/ +if (pte_soft_dirty (orig_pte)) + newpte =pte_mksoft_dirty (newpte);
[Severity: Medium]
When ksm_use_zero_pages is enabled and replace_page() merges a zero-filled page whose soft-dirty bit was cleared, earlier in replace_page() the zero-page PTE is constructed with pte_mkdirty():
newpte = pte_mkdirty(pte_mkspecial(pfn_pte(page_to_pfn(kpage), vma->vm_page_prot)));
On architectures such as x86, pte_mkdirty() sets both _PAGE_DIRTY and _PAGE_SOFT_DIRTY:
arch/x86/include/asm/pgtable.h:pte_mkdirty() {
pte = pte_set_flags(pte, _PAGE_DIRTY | _PAGE_SOFT_DIRTY);
return pte_mksaveddirty(pte);
}Without an else branch calling pte_clear_soft_dirty(newpte) (similar to remove_migration_pte() in mm/migrate.c), won't clean pages merged with the KSM zero page end up falsely marked soft-dirty in /proc/<pid>/pagemap?
mm/ksm.c static int replace_page(struct vm_area_struct *vma, struct page *page, · line 1476
Quoted 4 lines
mm/ksm.c +4
-0+if (pte_uffd (orig_pte)) + newpte =pte_mkuffd (newpte); +if (pte_uffd (orig_pte) &&userfaultfd_rwp (vma)) + newpte =pte_modify (newpte,PAGE_NONE );
[Severity: High]
How does installing a KSM PTE with PAGE_NONE and the uffd bit interact with break_ksm() when unmerging KSM pages (via ksm_madvise() for MADV_UNMERGEABLE, prep_move_vma() during mremap(), __ksm_del_vma(), unmerge_and_remove_all_rmap_items(), or break_cow())?
break_ksm() invokes handle_mm_fault() with FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE (and without FAULT_FLAG_ALLOW_RETRY):
mm/ksm.c:break_ksm() {
do {
...
ret = handle_mm_fault(vma, addr,
FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE,
NULL);
} while (!(ret & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV | VM_FAULT_OOM)));
}In handle_pte_fault(), the pte_protnone() check runs before the FAULT_FLAG_UNSHARE check and dispatches to do_uffd_rwp():
mm/memory.c:handle_pte_fault() {
if (pte_protnone(vmf->orig_pte) && vma_is_accessible(vmf->vma)) {
if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte))
return do_uffd_rwp(vmf);
return do_numa_page(vmf);
}
}1. In synchronous RWP mode (!userfaultfd_rwp_async(vmf->vma)), do_uffd_rwp() calls handle_userfault(vmf, VM_UFFD_RWP). Because FAULT_FLAG_ALLOW_RETRY is not set, handle_userfault() triggers the CONFIG_DEBUG_VM warning for missing FAULT_FLAG_ALLOW_RETRY and returns VM_FAULT_SIGBUS. break_ksm() then exits its loop and returns 0, leaving the KSM page (and any subsequent KSM pages in the range) still mapped after MADV_UNMERGEABLE, mremap(), or VMA teardown.
2. In asynchronous RWP mode (userfaultfd_rwp_async(vmf->vma)), the first iteration of break_ksm() runs do_uffd_rwp(), which restores vma->vm_page_prot, clears the uffd bit, and marks the PTE young, and the second iteration unshares the page via do_wp_page() -> wp_page_copy(), silently stripping RWP protection during an internal KSM unmerge.
3. Even if handle_pte_fault() were to reach wp_page_copy() directly for FAULT_FLAG_UNSHARE, wp_page_copy() copies pte_uffd(vmf->orig_pte) onto entry built from vma->vm_page_prot without restoring PAGE_NONE when userfaultfd_rwp(vma) is true:
mm/memory.c:wp_page_copy() {
if (unlikely(unshare)) {
if (pte_soft_dirty(vmf->orig_pte))
entry = pte_mksoft_dirty(entry);
if (pte_uffd(vmf->orig_pte))
entry = pte_mkuffd(entry);
}
}Does FAULT_FLAG_UNSHARE in handle_pte_fault() and wp_page_copy() need to handle RWP-protected PTEs before KSM merges them with PAGE_NONE?
mm/ksm.c static int replace_page(struct vm_area_struct *vma, struct page *page, · line 1479
Quoted 3 lines
mm/ksm.c +0
-0flush_cache_page (vma, addr,pte_pfn (ptep_get (ptep)));/*
[]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
David Hildenbrand (Arm)
maintainerQuoted 7 lines · Samarth Narang
replace_page()maps the KSM page with a PTE built from scratch, so the soft-dirty and uffd bits of the PTE it replaces are lost. Due to this, a write before the merge no longer shows as soft-dirty, the next write to a uffd-wp page raises no write-protect fault, and because the page is mapped with the VMA's normal permissions, on a uffd-rwp VMA the next access raises no RWP fault.
Quoted 7 lines
Copy the soft-dirty and uffd bits from the original PTE to the new PTE, and restore
PAGE_NONEfor RWP VMAs, astry_to_map_unused_to_zeropage()does.Fixes 0f8975ec4db2 ("mm: soft-dirty bits for user memory changes tracking")Fixes 63b2d4174c4a ("userfaultfd: wp: add the writeprotect API to userfaultfd ioctl")Fixes 2d427bb01639 ("mm/userfaultfd: add RWP fault delivery and expose UFFDIO_REGISTER_MODE_RWP")
I would actually split that up into separate patches.
Over-indicating soft-dirty (as raised by Sashiko) is in general not a problem.
Losing WP bits is.
UFFD, the gift that keeps on giving.
How to deal RWP properly is a good question (see Sashiko review). I feel like some more stuff might be missing there to handle this correctly.
Signature · 3 lines
Cheers, David
Quoted 12 lines · David Hildenbrand (Arm)
Quoted 11 lines · Samarth Narang
replace_page()maps the KSM page with a PTE built from scratch, so the soft-dirty and uffd bits of the PTE it replaces are lost. Due to this, a write before the merge no longer shows as soft-dirty, the next write to a uffd-wp page raises no write-protect fault, and because the page is mapped with the VMA's normal permissions, on a uffd-rwp VMA the next access raises no RWP fault.Copy the soft-dirty and uffd bits from the original PTE to the new PTE, and restore
PAGE_NONEfor RWP VMAs, astry_to_map_unused_to_zeropage()does.
Quoted 7 lines
Quoted 3 lines
Fixes 0f8975ec4db2 ("mm: soft-dirty bits for user memory changes tracking")Fixes 63b2d4174c4a ("userfaultfd: wp: add the writeprotect API to userfaultfd ioctl")Fixes 2d427bb01639 ("mm/userfaultfd: add RWP fault delivery and expose UFFDIO_REGISTER_MODE_RWP")I would actually split that up into separate patches.
Over-indicating soft-dirty (as raised by Sashiko) is in general not a problem.
Right.
Quoted 3 lines
Losing WP bits is.
UFFD, the gift that keeps on giving.
Haha, yeah, I stared at the code a bit. Say we arm RWP on an existing KSM mapping, then do MADV_UNMERGEABLE. We're just replacing the shared page with a private copy. That shouldn't count as an access to the mapping ... no?
But the protnone check sends us to do_uffd_rwp() before the FAULT_FLAG_UNSHARE handling. No FAULT_FLAG_ALLOW_RETRY here, so sync RWP gives us VM_FAULT_SIGBUS. Then break_ksm() does this:
static int break_ksm(struct vm_area_struct *vma, unsigned long addr,
unsigned long end, bool lock_vma)
{
vm_fault_t ret = 0;
...
do {
...
ret = handle_mm_fault(vma, addr,
FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE,
NULL);
} while (!(ret & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV | VM_FAULT_OOM)));
...
return (ret & VM_FAULT_OOM) ? -ENOMEM : 0;
}So we report success with the page still merged. Async RWP does unshare it, but only after clearing the uffd bit.
I'd let FAULT_FLAG_UNSHARE skip do_uffd_rwp(). We'd still need to carry PAGE_NONE over in wp_page_copy(), though. That PTE is rebuilt from vma->vm_page_prot, so the uffd bit alone won't stop reads.
Quoted 2 lines
How to deal RWP properly is a good question (see Sashiko review). I feel like some more stuff might be missing there to handle this correctly.
Tested, this is what I had in mind:
---8<---
mm/memory.c +10 -4
@@ -4119,8 +4119,11 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)if (unlikely (unshare)) {if (pte_soft_dirty (vmf->orig_pte)) entry =pte_mksoft_dirty (entry); -if (pte_uffd (vmf->orig_pte)) +if (pte_uffd (vmf->orig_pte)) { entry =pte_mkuffd (entry); +if (userfaultfd_rwp (vma)) + entry =pte_modify (entry,PAGE_NONE ); + } }else { entry =maybe_mkwrite (pte_mkdirty (entry), vma); }
@@ -6656,9 +6659,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)* VM_UFFD_RWP VMA, a protnone PTE without the uffd bit is * NUMA hinting and must still fall through to do_numa_page(). */ -if (userfaultfd_pte_rwp (vmf->vma, vmf->orig_pte)) -return do_uffd_rwp (vmf); -return do_numa_page (vmf); +if (userfaultfd_pte_rwp (vmf->vma, vmf->orig_pte)) { +if (!(vmf->flags &FAULT_FLAG_UNSHARE )) +return do_uffd_rwp (vmf); + }else { +return do_numa_page (vmf); + } }spin_lock (vmf->ptl);
--
Cheers, Lance
David Hildenbrand (Arm)
maintainerQuoted 86 lines · Lance Yang
Quoted 12 lines · David Hildenbrand (Arm)
Quoted 11 lines · Samarth Narang
replace_page()maps the KSM page with a PTE built from scratch, so the soft-dirty and uffd bits of the PTE it replaces are lost. Due to this, a write before the merge no longer shows as soft-dirty, the next write to a uffd-wp page raises no write-protect fault, and because the page is mapped with the VMA's normal permissions, on a uffd-rwp VMA the next access raises no RWP fault.Copy the soft-dirty and uffd bits from the original PTE to the new PTE, and restore
PAGE_NONEfor RWP VMAs, astry_to_map_unused_to_zeropage()does.Quoted 7 lines
Quoted 3 lines
Fixes 0f8975ec4db2 ("mm: soft-dirty bits for user memory changes tracking")Fixes 63b2d4174c4a ("userfaultfd: wp: add the writeprotect API to userfaultfd ioctl")Fixes 2d427bb01639 ("mm/userfaultfd: add RWP fault delivery and expose UFFDIO_REGISTER_MODE_RWP")I would actually split that up into separate patches.
Over-indicating soft-dirty (as raised by Sashiko) is in general not a problem.
Right.
Quoted 3 lines
Losing WP bits is.
UFFD, the gift that keeps on giving.
Haha, yeah, I stared at the code a bit. Say we arm RWP on an existing KSM mapping, then do
MADV_UNMERGEABLE. We're just replacing the shared page with a private copy. That shouldn't count as an access to the mapping ... no?But the protnone check sends us to
do_uffd_rwp()before theFAULT_FLAG_UNSHAREhandling. NoFAULT_FLAG_ALLOW_RETRYhere, so sync RWP gives usVM_FAULT_SIGBUS. Thenbreak_ksm()does this:static int break_ksm(struct
vm_area_struct*vma, unsigned long addr,
unsigned long end, boollock_vma){ vm_fault_t ret = 0; ... do { ... ret = handle_mm_fault(vma, addr, FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE, NULL); } while (!(ret & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV | VM_FAULT_OOM))); ... return (ret & VM_FAULT_OOM) ? -ENOMEM : 0; }So we report success with the page still merged. Async RWP does unshare it, but only after clearing the uffd bit.
I'd let
FAULT_FLAG_UNSHAREskipdo_uffd_rwp(). We'd still need to carryPAGE_NONEover inwp_page_copy(), though. That PTE is rebuilt fromvma->vm_page_prot, so the uffd bit alone won't stop reads.Quoted 2 lines
How to deal RWP properly is a good question (see Sashiko review). I feel like some more stuff might be missing there to handle this correctly.
Tested, this is what I had in mind:
---8<---
mm/memory.c +4
-1@@ -4119,8 +4119,11 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)if (unlikely (unshare)) {if (pte_soft_dirty (vmf->orig_pte)) entry =pte_mksoft_dirty (entry); -if (pte_uffd (vmf->orig_pte)) +if (pte_uffd (vmf->orig_pte)) { entry =pte_mkuffd (entry); +if (userfaultfd_rwp (vma)) + entry =pte_modify (entry,PAGE_NONE ); + } }else { entry =maybe_mkwrite (pte_mkdirty (entry), vma); }@@ -6656,9 +6659,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)* VM_UFFD_RWP VMA, a protnone PTE without the uffd bit is * NUMA hinting and must still fall through to do_numa_page(). */
mm/memory.c static vm_fault_t handle_pte_fault(struct vm_fault *vmf) · line 6668
Quoted 10 lines
mm/memory.c +6
-3-if (userfaultfd_pte_rwp (vmf->vma, vmf->orig_pte)) -return do_uffd_rwp (vmf); -return do_numa_page (vmf); +if (userfaultfd_pte_rwp (vmf->vma, vmf->orig_pte)) { +if (!(vmf->flags &FAULT_FLAG_UNSHARE )) +return do_uffd_rwp (vmf); + }else { +return do_numa_page (vmf); + } }
Looks like a reasonable direction (without looking into all the details).
I've been also wondering whether KSM should simply skip these uffd regions, they it never worked properly ... but I'm sure we'd run into issue with leftover uffd pte bits :(
Signature · 3 lines
Cheers, David
Quoted 100 lines · David Hildenbrand (Arm)
Quoted 86 lines · Lance Yang
Quoted 12 lines · David Hildenbrand (Arm)
Quoted 11 lines · Samarth Narang
replace_page()maps the KSM page with a PTE built from scratch, so the soft-dirty and uffd bits of the PTE it replaces are lost. Due to this, a write before the merge no longer shows as soft-dirty, the next write to a uffd-wp page raises no write-protect fault, and because the page is mapped with the VMA's normal permissions, on a uffd-rwp VMA the next access raises no RWP fault.Copy the soft-dirty and uffd bits from the original PTE to the new PTE, and restore
PAGE_NONEfor RWP VMAs, astry_to_map_unused_to_zeropage()does.Quoted 7 lines
Quoted 3 lines
Fixes 0f8975ec4db2 ("mm: soft-dirty bits for user memory changes tracking")Fixes 63b2d4174c4a ("userfaultfd: wp: add the writeprotect API to userfaultfd ioctl")Fixes 2d427bb01639 ("mm/userfaultfd: add RWP fault delivery and expose UFFDIO_REGISTER_MODE_RWP")I would actually split that up into separate patches.
Over-indicating soft-dirty (as raised by Sashiko) is in general not a problem.
Right.
Quoted 3 lines
Losing WP bits is.
UFFD, the gift that keeps on giving.
Haha, yeah, I stared at the code a bit. Say we arm RWP on an existing KSM mapping, then do
MADV_UNMERGEABLE. We're just replacing the shared page with a private copy. That shouldn't count as an access to the mapping ... no?But the protnone check sends us to
do_uffd_rwp()before theFAULT_FLAG_UNSHAREhandling. NoFAULT_FLAG_ALLOW_RETRYhere, so sync RWP gives usVM_FAULT_SIGBUS. Thenbreak_ksm()does this:static int break_ksm(struct
vm_area_struct*vma, unsigned long addr,
unsigned long end, boollock_vma){ vm_fault_t ret = 0; ... do { ... ret = handle_mm_fault(vma, addr, FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE, NULL); } while (!(ret & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV | VM_FAULT_OOM))); ... return (ret & VM_FAULT_OOM) ? -ENOMEM : 0; }So we report success with the page still merged. Async RWP does unshare it, but only after clearing the uffd bit.
I'd let
FAULT_FLAG_UNSHAREskipdo_uffd_rwp(). We'd still need to carryPAGE_NONEover inwp_page_copy(), though. That PTE is rebuilt fromvma->vm_page_prot, so the uffd bit alone won't stop reads.Quoted 2 lines
How to deal RWP properly is a good question (see Sashiko review). I feel like some more stuff might be missing there to handle this correctly.
Tested, this is what I had in mind:
---8<---
mm/memory.c +4
-1@@ -4119,8 +4119,11 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)if (unlikely (unshare)) {if (pte_soft_dirty (vmf->orig_pte)) entry =pte_mksoft_dirty (entry); -if (pte_uffd (vmf->orig_pte)) +if (pte_uffd (vmf->orig_pte)) { entry =pte_mkuffd (entry); +if (userfaultfd_rwp (vma)) + entry =pte_modify (entry,PAGE_NONE ); + } }else { entry =maybe_mkwrite (pte_mkdirty (entry), vma); }@@ -6656,9 +6659,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)* VM_UFFD_RWP VMA, a protnone PTE without the uffd bit is * NUMA hinting and must still fall through to do_numa_page(). */ mm/memory.c static vm_fault_t handle_pte_fault(struct vm_fault *vmf) · line 6668
Quoted 10 lines
mm/memory.c +6
-3-if (userfaultfd_pte_rwp (vmf->vma, vmf->orig_pte)) -return do_uffd_rwp (vmf); -return do_numa_page (vmf); +if (userfaultfd_pte_rwp (vmf->vma, vmf->orig_pte)) { +if (!(vmf->flags &FAULT_FLAG_UNSHARE )) +return do_uffd_rwp (vmf); + }else { +return do_numa_page (vmf); + } }Looks like a reasonable direction (without looking into all the details).
Quoted 2 lines
I've been also wondering whether KSM should simply skip these uffd regions, they it never worked properly ... but I'm sure we'd run into issue with leftover uffd
+1. Did it ever actually work there :) Why not just skip them and call it a day :D
Quoted 2 lines
pte bits :(
David Hildenbrand (Arm)
maintainerQuoted 78 lines · Lance Yang
Quoted 75 lines · David Hildenbrand (Arm)
Quoted 61 lines · Lance Yang
Right.
Haha, yeah, I stared at the code a bit. Say we arm RWP on an existing KSM mapping, then do
MADV_UNMERGEABLE. We're just replacing the shared page with a private copy. That shouldn't count as an access to the mapping ... no?But the protnone check sends us to
do_uffd_rwp()before theFAULT_FLAG_UNSHAREhandling. NoFAULT_FLAG_ALLOW_RETRYhere, so sync RWP gives usVM_FAULT_SIGBUS. Thenbreak_ksm()does this:static int
break_ksm(struct vm_area_struct *vma, unsigned long addr, unsigned long end, bool lock_vma)
{
vm_fault_t ret = 0;
...
do {
...
ret = handle_mm_fault(vma, addr,
FAULT_FLAG_UNSHARE|FAULT_FLAG_REMOTE, NULL);
} while (!(ret & (VM_FAULT_SIGBUS|VM_FAULT_SIGSEGV|VM_FAULT_OOM)));
...
return (ret &VM_FAULT_OOM) ?-ENOMEM: 0;
}So we report success with the page still merged. Async RWP does unshare it, but only after clearing the uffd bit.
I'd let
FAULT_FLAG_UNSHAREskipdo_uffd_rwp(). We'd still need to carryPAGE_NONEover inwp_page_copy(), though. That PTE is rebuilt fromvma->vm_page_prot, so the uffd bit alone won't stop reads.Tested, this is what I had in mind:
---8<---
mm/memory.c +4
-1@@ -4119,8 +4119,11 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (unlikely(unshare)) { if (pte_soft_dirty(vmf->orig_pte)) entry = pte_mksoft_dirty(entry); - if (pte_uffd(vmf->orig_pte)) + if (pte_uffd(vmf->orig_pte)) { entry = pte_mkuffd(entry); + if (userfaultfd_rwp(vma)) + entry = pte_modify(entry, PAGE_NONE); + } } else { entry = maybe_mkwrite(pte_mkdirty(entry), vma); }@@ -6656,9 +6659,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)* VM_UFFD_RWP VMA, a protnone PTE without the uffd bit is * NUMA hinting and must still fall through to do_numa_page(). */ mm/memory.c static vm_fault_t handle_pte_fault(struct vm_fault *vmf) · line 6668
Quoted 10 lines
mm/memory.c +6
-3- if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte)) - return do_uffd_rwp(vmf); - return do_numa_page(vmf); + if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte)) { + if (!(vmf->flags & FAULT_FLAG_UNSHARE)) + return do_uffd_rwp(vmf); + } else { + return do_numa_page(vmf); + } }Looks like a reasonable direction (without looking into all the details).
Quoted 5 lines
Quoted 2 lines
I've been also wondering whether KSM should simply skip these uffd regions, they it never worked properly ... but I'm sure we'd run into issue with leftover uffd
+1. Did it ever actually work there :) Why not just skip them and call it a day :D
I think QEMU just always sets MADV_MERGABLE and later activates uffd. So that needs some care to not disable them. So let's rather try to get it fixed properly.
Can you send out proper patches? Thanks!
Signature · 3 lines
Cheers, David
Quoted 7 lines · David Hildenbrand (Arm)
Quoted 2 lines
+1. Did it ever actually work there :) Why not just skip them and call it a day :D
I think QEMU just always sets
MADV_MERGABLEand later activates uffd. So that needs some care to not disable them. So let's rather try to get it fixed properly.Can you send out proper patches? Thanks!
+1.
Lance's direction is right. Unsharing is not an access: do_wp_page() already skips the uffd-wp fault on FAULT_FLAG_UNSHARE, and RWP should do the same.
Signature · 1 line
Kiryl Shutsemau / Kirill A. Shutemov
Quoted 8 lines · Kiryl Shutsemau
Quoted 7 lines · David Hildenbrand (Arm)
Quoted 2 lines
+1. Did it ever actually work there :) Why not just skip them and call it a day :D
I think QEMU just always sets
MADV_MERGABLEand later activates uffd. So that needs some care to not disable them. So let's rather try to get it fixed properly.Can you send out proper patches? Thanks!
Thanks David! Will take it over next week and get proper patches out :)
Quoted 6 lines
+1.
Lance's direction is right. Unsharing is not an access:
do_wp_page()already skips the uffd-wp fault onFAULT_FLAG_UNSHARE, and RWP should do the same.
Thanks Kiryl! Will take a close look at that :)
Cheers, Lance
Hi Samarth,
Could you send a v3 with just the soft-dirty fix, as David suggested? Leave the uffd-wp/RWP bits to me (I'll take a closer look).
Quoted 25 lines · Lance Yang
Quoted 8 lines · Kiryl Shutsemau
Quoted 7 lines · David Hildenbrand (Arm)
Quoted 2 lines
+1. Did it ever actually work there :) Why not just skip them and call it a day :D
I think QEMU just always sets
MADV_MERGABLEand later activates uffd. So that needs some care to not disable them. So let's rather try to get it fixed properly.Can you send out proper patches? Thanks!
Thanks David! Will take it over next week and get proper patches out :)
Quoted 6 lines
+1.
Lance's direction is right. Unsharing is not an access:
do_wp_page()already skips the uffd-wp fault onFAULT_FLAG_UNSHARE, and RWP should do the same.Thanks Kiryl! Will take a close look at that :)
Cheers, Lance
Hi Lance,
Sure, sounds good. I’ll send a patch out today.
Thanks!
Quoted 36 lines · On Wed, Oct 21, 2026 at 12:12 Lance Yang <[email protected]> wrote:
Hi Samarth,
Could you send a v3 with just the soft-dirty fix, as David suggested? Leave the uffd-wp/RWP bits to me (I'll take a closer look).
Quoted 27 lines · Lance Yang
Quoted 10 lines · Kiryl Shutsemau
Quoted 9 lines · David Hildenbrand (Arm)
Quoted 2 lines
+1. Did it ever actually work there :) Why not just skip them and call it a day :D
I think QEMU just always sets
MADV_MERGABLEand later activates uffd. So that
needs some care to not disable them. So let's rather try to get it fixed properly.Can you send out proper patches? Thanks!
Thanks David! Will take it over next week and get proper patches out :)
Quoted 6 lines
+1.
Lance's direction is right. Unsharing is not an access:
do_wp_page()already skips the uffd-wp fault onFAULT_FLAG_UNSHARE, and RWP should do the same.Thanks Kiryl! Will take a close look at that :)
Cheers, Lance
Mohammed EL Kadiri
Tested-byThis works for me, on b306deee4cba (mm-testing), x86_64, CONFIG_KSM=y, CONFIG_USERFAULTFD=y, CONFIG_DEBUG_VM=y, KASAN enabled, THP disabled.
Reproducer: merge 64 identical anon pages via KSM, arm synchronous userfaultfd-wp on one page, then MADV_UNMERGEABLE it without touching it. The page should come out private with WP semantics intact.
On the same commit, before applying your wp_page_copy()/handle_pte_fault() fix:
UNMERGE ret=0 errno=0
AFTER_UNMERGE target_pfn=1153393 peer_pfn=1153393 target_KPF_KSM=1
FAIL: MADV_UNMERGEABLE returned success but target remains shared/KSM
and every run also hits this in dmesg:
FAULT_FLAG_ALLOW_RETRY missing c80
Call Trace: handle_userfault.cold+0x30/0x3f __handle_mm_fault+0xde9/0x1c30 handle_mm_fault+0x13b/0x440 break_ksm+0x8a/0x100 ksm_madvise+0x467/0x780 madvise_vma_behavior+0x1d1/0x1e40 madvise_do_behavior+0x431/0x14b0 do_madvise+0x162/0x210
break_ksm()'s internal fault-in to unshare the page is landing in the uffd-wp fault handler, which isn't expecting it. 8/8 runs across two boots, same trace every time.
After applying the fix:
AFTER_UNMERGE target_pfn=1063999 peer_pfn=1064929 target_KPF_KSM=0
PASS: target is private and no longer KSM-backed
FAULT event=18 flags=0x8 address=0x7f3dcb727000
PASS: read generated RWP fault and remained blocked
No warnings or call traces in dmesg. 8/8 runs across two boots.