mm: Fix uffd-wp bit loss when batching file folio unmapping
Subsystems (4)
PATCH 1/1 mm: Fix uffd-wp bit loss when batching file folio unmapping
The recently added file folio unmap batching support forgets to update pte_install_uffd_wp_if_needed(), which still updates a single pte. We end up jumping to the end of the folio in page_vma_mapped_walk(), thus setting the uffd-wp marker only on a single pte in the batch. Fix this by passing nr_pages into the function, and set the uffd-wp marker on all ptes.
Note that, since the nr_pages passed to this function is always derived by some sort of batching, it is guaranteed that the set of old ptevals of the batch have uffd-wp bit on all ptes or no ptes, therefore it is safe to derive the value of the local variable "arm_uffd_pte" from only the particular pteval passed to this function, but apply the result on all ptes of the batch.
Use set_pte_at() in a loop to set the markers - we cannot use set_ptes() as that will increment the PFN, but we don't have any PFN to update here.
The userspace visible effect of the bug is inaccuracy observed by workloads relying on uffd-wp regions to install their own pages.
---
Patch applies on mm-unstable, commit f8ed52ac0cfb.
I observed this bug during code inspection, but it turns out that the uffd-wp-mremap selftest will skip some tests with a bogus complain that "MADV_PAGEOUT didn't work, is swap enabled?" even when swap is enabled. It first sets the region uffd-wp, then swaps it out, then checks through pagemap whether it got swapped out. For file folios, this check makes no sense since the ptes are simply cleared, but in this particular case, because of uffd-wp preservation, we need to store PTE_MARKER_UFFD_WP, which is stored as a swap entry, that is why the test works out on a non-buggy kernel.
include/linux/mm_inline.h | 7 ++++--- mm/memory.c | 14 +------------- mm/rmap.c | 2 +- 3 files changed, 6 insertions(+), 17 deletions(-)
include/linux/mm_inline.h +4 -3
@@ -568,7 +568,7 @@ static inline pte_marker copy_pte_marker(*/ static inline bool pte_install_uffd_wp_if_needed (struct vm_area_struct *vma,unsigned long addr, -pte_t *pte,pte_t pteval) +pte_t *pte,pte_t pteval,unsigned int nr_pages) {bool arm_uffd_pte = false;
@@ -599,8 +599,9 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, arm_uffd_pte = true;if (unlikely (arm_uffd_pte)) { -set_pte_at (vma->vm_mm, addr, pte, -make_pte_marker (PTE_MARKER_UFFD_WP )); +for (int i =0 ; i < nr_pages; ++i, ++pte, addr +=PAGE_SIZE ) +set_pte_at (vma->vm_mm, addr, pte, +make_pte_marker (PTE_MARKER_UFFD_WP ));return true; }
mm/memory.c +1 -13
@@ -1588,8 +1588,6 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, pte_t *pte, int nr, struct zap_details *details, pte_t pteval) { - bool was_installed = false; - if (!uffd_supports_wp_marker()) return false;
@@ -1600,17 +1598,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,if (zap_drop_markers (details))return false; -for (;;) { -/* the PFN in the PTE is irrelevant. */ -if (pte_install_uffd_wp_if_needed (vma, addr, pte, pteval)) - was_installed = true; -if (--nr ==0 ) -break ; - pte++; - addr +=PAGE_SIZE ; - } - -return was_installed; +return pte_install_uffd_wp_if_needed (vma, addr, pte, pteval, nr); } static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,
mm/rmap.c +1 -1
@@ -2171,7 +2171,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,* we may want to replace a none pte with a marker pte if * it's file-backed, so we don't lose the tracking info. */ -pte_install_uffd_wp_if_needed (vma, address, pvmw.pte, pteval); +pte_install_uffd_wp_if_needed (vma, address, pvmw.pte, pteval, nr_pages);/* Update high watermark before we lower rss */ update_hiwater_rss (mm);
Signature · 1 line
2.34.1
Quoted 12 lines · Dev Jain
The recently added file folio unmap batching support forgets to update
pte_install_uffd_wp_if_needed(), which still updates a single pte. We end up jumping to the end of the folio inpage_vma_mapped_walk(), thus setting the uffd-wp marker only on a single pte in the batch. Fix this by passingnr_pagesinto the function, and set the uffd-wp marker on all ptes.Note that, since the
nr_pagespassed to this function is always derived by some sort of batching, it is guaranteed that the set of old ptevals of the batch have uffd-wp bit on all ptes or no ptes, therefore it is safe to derive the value of the local variable "arm_uffd_pte" from only the particular pteval passed to this function, but apply the result on all ptes of the batch.
Quoted 7 lines
Use
set_pte_at()in a loop to set the markers - we cannot useset_ptes()as that will increment the PFN, but we don't have any PFN to update here.The userspace visible effect of the bug is inaccuracy observed by workloads relying on uffd-wp regions to install their own pages.
Fixes 8798e255b5ec ("mm: rmap: support batched unmapping for file large folios")
Hmm this patch isn't upstream yet, so wouldn't this patch be better as a comment replying to the series so it can be respun?
You only do a fixes tag for either upstream things that obviously have to stay in place or perhaps things in mm-stable immediately prior to merge window that can't be adjusted.
Quoted 1 line
Signed-off-by Dev Jain <[email protected]>
Let's replace this with a comment on the series at [0] please.
[0]: https://lore.kernel.org/linux-mm/[email protected]/
Also the original series really shouldn't go in without signoff from me/David.
Thanks, Lorenzo
Quoted 89 lines
---
Patch applies on mm-unstable, commitf8ed52ac0cfb.I observed this bug during code inspection, but it turns out that the uffd-wp-mremap selftest will skip some tests with a bogus complain that "
MADV_PAGEOUTdidn't work, is swap enabled?" even when swap is enabled. It first sets the region uffd-wp, then swaps it out, then checks through pagemap whether it got swapped out. For file folios, this check makes no sense since the ptes are simply cleared, but in this particular case, because of uffd-wp preservation, we need to storePTE_MARKER_UFFD_WP, which is stored as a swap entry, that is why the test works out on a non-buggy kernel.include/linux/mm_inline.h | 7 ++++--- mm/memory.c | 14 +------------- mm/rmap.c | 2 +- 3 files changed, 6 insertions(+), 17 deletions(-)include/linux/mm_inline.h +4
-3@@ -568,7 +568,7 @@ static inline pte_marker copy_pte_marker(*/ static inline bool pte_install_uffd_wp_if_needed (struct vm_area_struct *vma,unsigned long addr, -pte_t *pte,pte_t pteval) +pte_t *pte,pte_t pteval,unsigned int nr_pages) {bool arm_uffd_pte = false;@@ -599,8 +599,9 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, arm_uffd_pte = true;if (unlikely (arm_uffd_pte)) { -set_pte_at (vma->vm_mm, addr, pte, -make_pte_marker (PTE_MARKER_UFFD_WP )); +for (int i =0 ; i < nr_pages; ++i, ++pte, addr +=PAGE_SIZE ) +set_pte_at (vma->vm_mm, addr, pte, +make_pte_marker (PTE_MARKER_UFFD_WP ));return true; }mm/memory.c +1
-13@@ -1588,8 +1588,6 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, pte_t *pte, int nr, struct zap_details *details, pte_t pteval) { - bool was_installed = false; - if (!uffd_supports_wp_marker()) return false;@@ -1600,17 +1598,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,if (zap_drop_markers (details))return false; -for (;;) { -/* the PFN in the PTE is irrelevant. */ -if (pte_install_uffd_wp_if_needed (vma, addr, pte, pteval)) - was_installed = true; -if (--nr ==0 ) -break ; - pte++; - addr +=PAGE_SIZE ; - } - -return was_installed; +return pte_install_uffd_wp_if_needed (vma, addr, pte, pteval, nr); } static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,mm/rmap.c +1
-2@@ -2171,7 +2171,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,* we may want to replace a none pte with a marker pte if * it's file-backed, so we don't lose the tracking info. */ -pte_install_uffd_wp_if_needed (vma, address, pvmw.pte, pteval); +pte_install_uffd_wp_if_needed (vma, address, pvmw.pte, pteval, nr_pages);/* Update high watermark before we lower rss */ update_hiwater_rss (mm); --2.34.1
Quoted 19 lines · Lorenzo Stoakes
Quoted 18 lines · Dev Jain
The recently added file folio unmap batching support forgets to update
pte_install_uffd_wp_if_needed(), which still updates a single pte. We end up jumping to the end of the folio inpage_vma_mapped_walk(), thus setting the uffd-wp marker only on a single pte in the batch. Fix this by passingnr_pagesinto the function, and set the uffd-wp marker on all ptes.Note that, since the
nr_pagespassed to this function is always derived by some sort of batching, it is guaranteed that the set of old ptevals of the batch have uffd-wp bit on all ptes or no ptes, therefore it is safe to derive the value of the local variable "arm_uffd_pte" from only the particular pteval passed to this function, but apply the result on all ptes of the batch.Use
set_pte_at()in a loop to set the markers - we cannot useset_ptes()as that will increment the PFN, but we don't have any PFN to update here.The userspace visible effect of the bug is inaccuracy observed by workloads relying on uffd-wp regions to install their own pages.
Quoted 7 lines
Quoted 1 line
Fixes 8798e255b5ec ("mm: rmap: support batched unmapping for file large folios")Hmm this patch isn't upstream yet, so wouldn't this patch be better as a comment replying to the series so it can be respun?
You only do a fixes tag for either upstream things that obviously have to stay in place or perhaps things in mm-stable immediately prior to merge window that can't be adjusted.
I saw that the last comment on that series was more than a week back, so best thought to just do a folded fix on top of it - and I had formed the impression (from the conversations on list) that akpm prefers fixes over respins : )
If a respin is preferred here then I am fine by that.
Quoted 99 lines
Quoted 2 lines
Signed-off-by Dev Jain <[email protected]>Let's replace this with a comment on the series at [0] please.
[0]: https://lore.kernel.org/linux-mm/[email protected]/
Also the original series really shouldn't go in without signoff from me/David.
Thanks, Lorenzo
Quoted 89 lines
---
Patch applies on mm-unstable, commitf8ed52ac0cfb.I observed this bug during code inspection, but it turns out that the uffd-wp-mremap selftest will skip some tests with a bogus complain that "
MADV_PAGEOUTdidn't work, is swap enabled?" even when swap is enabled. It first sets the region uffd-wp, then swaps it out, then checks through pagemap whether it got swapped out. For file folios, this check makes no sense since the ptes are simply cleared, but in this particular case, because of uffd-wp preservation, we need to storePTE_MARKER_UFFD_WP, which is stored as a swap entry, that is why the test works out on a non-buggy kernel.include/linux/mm_inline.h | 7 ++++--- mm/memory.c | 14 +------------- mm/rmap.c | 2 +- 3 files changed, 6 insertions(+), 17 deletions(-)include/linux/mm_inline.h +4
-3@@ -568,7 +568,7 @@ static inline pte_marker copy_pte_marker(*/ static inline bool pte_install_uffd_wp_if_needed (struct vm_area_struct *vma,unsigned long addr, -pte_t *pte,pte_t pteval) +pte_t *pte,pte_t pteval,unsigned int nr_pages) {bool arm_uffd_pte = false;@@ -599,8 +599,9 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, arm_uffd_pte = true;if (unlikely (arm_uffd_pte)) { -set_pte_at (vma->vm_mm, addr, pte, -make_pte_marker (PTE_MARKER_UFFD_WP )); +for (int i =0 ; i < nr_pages; ++i, ++pte, addr +=PAGE_SIZE ) +set_pte_at (vma->vm_mm, addr, pte, +make_pte_marker (PTE_MARKER_UFFD_WP ));return true; }mm/memory.c +1
-13@@ -1588,8 +1588,6 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, pte_t *pte, int nr, struct zap_details *details, pte_t pteval) { - bool was_installed = false; - if (!uffd_supports_wp_marker()) return false;@@ -1600,17 +1598,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,if (zap_drop_markers (details))return false; -for (;;) { -/* the PFN in the PTE is irrelevant. */ -if (pte_install_uffd_wp_if_needed (vma, addr, pte, pteval)) - was_installed = true; -if (--nr ==0 ) -break ; - pte++; - addr +=PAGE_SIZE ; - } - -return was_installed; +return pte_install_uffd_wp_if_needed (vma, addr, pte, pteval, nr); } static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,mm/rmap.c +1
-2@@ -2171,7 +2171,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,* we may want to replace a none pte with a marker pte if * it's file-backed, so we don't lose the tracking info. */ -pte_install_uffd_wp_if_needed (vma, address, pvmw.pte, pteval); +pte_install_uffd_wp_if_needed (vma, address, pvmw.pte, pteval, nr_pages);/* Update high watermark before we lower rss */ update_hiwater_rss (mm); --2.34.1
Quoted 8 lines · Dev Jain
On 16/01/26 2:09 pm, Lorenzo Stoakes wrote:
I saw that the last comment on that series was more than a week back, so best thought to just do a folded fix on top of it - and I had formed the impression (from the conversations on list) that akpm prefers fixes over respins : )If a respin is preferred here then I am fine by that.
Generally we prefer fix-patches, sent in reply to the patch being altered and sent by the series author.
Sending a patch with a Fixes: tag is never the correct way to fixup a patch unless they're upstream or unchangeably-bound-for-upstream with a commit hash that will be the same in Linus's tree.
You can by all means suggest a patch to an author by replying to the broken patch, but then it's up to them whether to take it. Also then the courteous way is to raise the issue in that reply and say something like 'it seems that the below fixes the issue, can you check it?' or something like this.
But the correct course is to the respond to the series in all cases like this.
Thanks, Lorenzo
Quoted 15 lines · Lorenzo Stoakes
Quoted 7 lines · Dev Jain
On 16/01/26 2:09 pm, Lorenzo Stoakes wrote:
I saw that the last comment on that series was more than a week back, so best thought to just do a folded fix on top of it - and I had formed the impression (from the conversations on list) that akpm prefers fixes over respins : )If a respin is preferred here then I am fine by that.
Generally we prefer fix-patches, sent in reply to the patch being altered and sent by the series author.
Sending a patch with a Fixes: tag is never the correct way to fixup a patch unless they're upstream or unchangeably-bound-for-upstream with a commit hash that will be the same in Linus's tree.
Quoted 6 lines
You can by all means suggest a patch to an author by replying to the broken patch, but then it's up to them whether to take it. Also then the courteous way is to raise the issue in that reply and say something like 'it seems that the below fixes the issue, can you check it?' or something like this.
But the correct course is to the respond to the series in all cases like this.
Alright! Thanks for your kind explanation.
Quoted 2 lines
Thanks, Lorenzo
David Hildenbrand (Red Hat)
maintainerQuoted 25 lines · Dev Jain
Quoted 23 lines · Lorenzo Stoakes
Quoted 12 lines · Dev Jain
The recently added file folio unmap batching support forgets to update
pte_install_uffd_wp_if_needed(), which still updates a single pte. We end up jumping to the end of the folio inpage_vma_mapped_walk(), thus setting the uffd-wp marker only on a single pte in the batch. Fix this by passingnr_pagesinto the function, and set the uffd-wp marker on all ptes.Note that, since the
nr_pagespassed to this function is always derived by some sort of batching, it is guaranteed that the set of old ptevals of the batch have uffd-wp bit on all ptes or no ptes, therefore it is safe to derive the value of the local variable "arm_uffd_pte" from only the particular pteval passed to this function, but apply the result on all ptes of the batch.Quoted 7 lines
Use
set_pte_at()in a loop to set the markers - we cannot useset_ptes()as that will increment the PFN, but we don't have any PFN to update here.The userspace visible effect of the bug is inaccuracy observed by workloads relying on uffd-wp regions to install their own pages.
Fixes 8798e255b5ec ("mm: rmap: support batched unmapping for file large folios")Hmm this patch isn't upstream yet, so wouldn't this patch be better as a comment replying to the series so it can be respun?
Quoted 7 lines
Quoted 3 lines
You only do a fixes tag for either upstream things that obviously have to stay in place or perhaps things in mm-stable immediately prior to merge window that can't be adjusted.
I saw that the last comment on that series was more than a week back, so best thought to just do a folded fix on top of it - and I had formed the impression (from the conversations on list) that akpm prefers fixes over respins : )
That series is still on my TODO list and I hope Andrew will keep it out of stable until I get to it.
Hope is all I got ;)
Signature · 3 lines
Cheers David
Andrew Morton
maintainerQuoted 6 lines · On Fri, 16 Jan 2026 11:51:17 +0100 "David Hildenbrand (Red Hat)"…
Quoted 3 lines
I saw that the last comment on that series was more than a week back, so best thought to just do a folded fix on top of it - and I had formed the impression (from the conversations on list) that akpm prefers fixes over respins : )
That series is still on my TODO list and I hope Andrew will keep it out of stable until I get to it.
No probs, I added a note against this series.
Quoted 1 line
Hope is all I got ;)
There's also "tell Andrew stuff". It's very helpful!
In the ordinary course of things I expect I'd have judged this series to be adequately tested and sufficiently well reviewed by sufficiently familiar names so I'd have moved it into mm-stable.
But after learning of your concerns I've added a note and I moved the series to tail of mm-unstable where it will be easier to drop or defer should the need arise.
If drop-or-defer is to be the decision then please let's make that as early as possible - we don't want non-merge-window material in there perturbing testing or causing integration issues.
David Hildenbrand (Red Hat)
maintainerQuoted 81 lines · Dev Jain
The recently added file folio unmap batching support forgets to update
pte_install_uffd_wp_if_needed(), which still updates a single pte. We end up jumping to the end of the folio inpage_vma_mapped_walk(), thus setting the uffd-wp marker only on a single pte in the batch. Fix this by passingnr_pagesinto the function, and set the uffd-wp marker on all ptes.Note that, since the
nr_pagespassed to this function is always derived by some sort of batching, it is guaranteed that the set of old ptevals of the batch have uffd-wp bit on all ptes or no ptes, therefore it is safe to derive the value of the local variable "arm_uffd_pte" from only the particular pteval passed to this function, but apply the result on all ptes of the batch.Use
set_pte_at()in a loop to set the markers - we cannot useset_ptes()as that will increment the PFN, but we don't have any PFN to update here.The userspace visible effect of the bug is inaccuracy observed by workloads relying on uffd-wp regions to install their own pages.
Fixes 8798e255b5ec ("mm: rmap: support batched unmapping for file large folios")Signed-off-by Dev Jain <[email protected]>---
Patch applies on mm-unstable, commitf8ed52ac0cfb.I observed this bug during code inspection, but it turns out that the uffd-wp-mremap selftest will skip some tests with a bogus complain that "
MADV_PAGEOUTdidn't work, is swap enabled?" even when swap is enabled. It first sets the region uffd-wp, then swaps it out, then checks through pagemap whether it got swapped out. For file folios, this check makes no sense since the ptes are simply cleared, but in this particular case, because of uffd-wp preservation, we need to storePTE_MARKER_UFFD_WP, which is stored as a swap entry, that is why the test works out on a non-buggy kernel.include/linux/mm_inline.h | 7 ++++--- mm/memory.c | 14 +------------- mm/rmap.c | 2 +- 3 files changed, 6 insertions(+), 17 deletions(-)include/linux/mm_inline.h +4
-3@@ -568,7 +568,7 @@ static inline pte_marker copy_pte_marker(*/ static inline bool pte_install_uffd_wp_if_needed (struct vm_area_struct *vma,unsigned long addr, -pte_t *pte,pte_t pteval) +pte_t *pte,pte_t pteval,unsigned int nr_pages) {bool arm_uffd_pte = false;@@ -599,8 +599,9 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, arm_uffd_pte = true;if (unlikely (arm_uffd_pte)) { -set_pte_at (vma->vm_mm, addr, pte, -make_pte_marker (PTE_MARKER_UFFD_WP )); +for (int i =0 ; i < nr_pages; ++i, ++pte, addr +=PAGE_SIZE ) +set_pte_at (vma->vm_mm, addr, pte, +make_pte_marker (PTE_MARKER_UFFD_WP ));return true; }mm/memory.c +0
-5@@ -1588,8 +1588,6 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, pte_t *pte, int nr, struct zap_details *details, pte_t pteval) { - bool was_installed = false; - if (!uffd_supports_wp_marker()) return false;@@ -1600,17 +1598,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,if (zap_drop_markers (details))return false; -for (;;) { -/* the PFN in the PTE is irrelevant. */ -if (pte_install_uffd_wp_if_needed (vma, addr, pte, pteval))
mm/memory.c zap_install_uffd_wp_if_needed(struct vm_area_struct *vma, · line 1602
Quoted 10 lines
mm/memory.c +1
-8- was_installed = true; -if (--nr ==0 ) -break ; - pte++; - addr +=PAGE_SIZE ; - } - -return was_installed; +return pte_install_uffd_wp_if_needed (vma, addr, pte, pteval, nr);
That should likely be factored out in an independent patch.
And while we do that, we should likely rename the function to indicate that we consume ptes.
That patch should also describe why the change is ok; it's not just simple code movement. :)
Signature · 3 lines
Cheers David
Quoted 8 lines · David Hildenbrand (Red Hat)
That should likely be factored out in an independent patch.
And while we do that, we should likely rename the function to indicate that we consume ptes.
That patch should also describe why the change is ok; it's not just simple code movement. :)
Right, but this patch is to correct something in a series that's not yet landed, it's not a hotfix, rather this should be a report on Baolin's series rather than splitting the converastion into the actual thread where Dev found an issue and an invalid patch submission here.
Can we try to get the conversation over there? Feel like maybe it's already messed up... sigh.
Quoted 5 lines
--
CheersDavid
Thanks, Lorenzo