mm: A few cleanup patches around zap, shmem and uffd
Subsystems (4)
Peter Xu
reviewerv3 cover mm: A few cleanup patches around zap, shmem and uffd
[Based on linus/master, commit ac08b1c68d1b which should contain the recent -mm
pull, if not applicable, I can repost anytime]
v3:
- Patch "mm: Add zap_skip_check_mapping() helper"
- In zap_skip_check_mapping() check zap_mapping first [Alistair]
- Patch "mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags":
- Fix English errors in commit message [David]
- Drop paragraph mentioning commit 22061a1ffabd in commit msg
- Set ZAP_FLAG_SKIP_SWAP for unmap_mapping_page() too
- Pick up r-bs
v2:
- Patch "mm: Clear vmf->pte after pte_unmap_same() returns"
- Remove one comment [David]
- Collect r-b for patch 2/3
- Rewrite the last two patches to drop ZAP_FLAG_CHECK_MAPPING, dropping Alistair's r-b on patch 5 because it changed [David, Matthew]
===== v1 cover letter =====
I picked up these patches from uffd-wp v5 series here:
https://lore.kernel.org/lkml/[email protected]/
IMHO all of them are very nice cleanups to existing code already, they're all small and self-contained. They'll be needed by uffd-wp coming series. I would appreciate if they can be accepted earlier, so as to not carry them over always with the uffd-wp series.
I removed some CC from the uffd-wp v5 series to reduce the noise, and added a few more into it.
Reviews are greatly welcomed, thanks.
Peter Xu (5):
mm/shmem: Unconditionally set pte dirty in mfill_atomic_install_pte
mm: Clear vmf->pte after pte_unmap_same() returns
mm: Drop first_index/last_index in zap_details
mm: Add zap_skip_check_mapping() helper
mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags
include/linux/mm.h | 34 +++++++++++++++++++-- mm/memory.c | 76 +++++++++++++++++++--------------------------- mm/shmem.c | 1 - mm/userfaultfd.c | 3 +- 4 files changed, 63 insertions(+), 51 deletions(-)
Signature · 1 line
2.31.1
Peter Xu
reviewerPATCH v3 1/5 mm/shmem: Unconditionally set pte dirty in mfill_atomic_install_pte
It was conditionally done previously, as there's one shmem special case that we use SetPageDirty() instead. However that's not necessary and it should be easier and cleaner to do it unconditionally in mfill_atomic_install_pte().
The most recent discussion about this is here, where Hugh explained the history of SetPageDirty() and why it's possible that it's not required at all:
https://lore.kernel.org/lkml/[email protected]/
Currently mfill_atomic_install_pte() has three callers:
1. shmem_mfill_atomic_pte
2. mcopy_atomic_pte
3. mcontinue_atomic_pte
After the change: case (1) should have its SetPageDirty replaced by the dirty bit on pte (so we unify them together, finally), case (2) should have no functional change at all as it has page_in_cache==false, case (3) may add a dirty bit to the pte. However since case (3) is UFFDIO_CONTINUE for shmem, it's merely 100% sure the page is dirty after all, so should not make a real difference either.
This should make it much easier to follow on which case will set dirty for uffd, as we'll simply set it all now for all uffd related ioctls. Meanwhile, no special handling of SetPageDirty() if there's no need.
--- mm/shmem.c | 1 - mm/userfaultfd.c | 3 +-- 2 files changed, 1 insertion(+), 3 deletions(-)
mm/shmem.c +0 -1
@@ -2424,7 +2424,6 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,shmem_recalc_inode (inode);spin_unlock_irq (&info->lock); -SetPageDirty (page);unlock_page (page);return 0 ; out_delete_from_cache:
mm/userfaultfd.c +1 -2
@@ -69,10 +69,9 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd,pgoff_t offset, max_off; _dst_pte =mk_pte (page, dst_vma->vm_page_prot); + _dst_pte =pte_mkdirty (_dst_pte);if (page_in_cache && !vm_shared) writable = false; -if (writable || !page_in_cache) - _dst_pte =pte_mkdirty (_dst_pte);if (writable) {if (wp_copy) _dst_pte =pte_mkuffd_wp (_dst_pte);
Signature · 1 line
2.31.1
Peter Xu
reviewerPATCH v3 2/5 mm: Clear vmf->pte after pte_unmap_same() returns
In reply to Peter Xupte_unmap_same() will always unmap the pte pointer. After the unmap, vmf->pte will not be valid any more, we should clear it.
It was safe only because no one is accessing vmf->pte after pte_unmap_same() returns, since the only caller of pte_unmap_same() (so far) is do_swap_page(), where vmf->pte will in most cases be overwritten very soon.
Directly pass in vmf into pte_unmap_same() and then we can also avoid the long parameter list too, which should be a nice cleanup.
--- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
mm/memory.c +6 -6
@@ -2724,19 +2724,19 @@ EXPORT_SYMBOL_GPL(apply_to_existing_page_range);* proceeding (but do_wp_page is only called after already making such a check; * and do_anonymous_page can safely check later on). */ -static inline int pte_unmap_same (struct mm_struct *mm,pmd_t *pmd, -pte_t *page_table,pte_t orig_pte) +static inline int pte_unmap_same (struct vm_fault *vmf) {int same =1 ;#if defined(CONFIG_SMP ) || defined(CONFIG_PREEMPTION )if (sizeof (pte_t) >sizeof (unsigned long )) { -spinlock_t *ptl =pte_lockptr (mm, pmd); +spinlock_t *ptl =pte_lockptr (vmf->vma->vm_mm, vmf->pmd);spin_lock (ptl); - same =pte_same (*page_table, orig_pte); + same =pte_same (*vmf->pte, vmf->orig_pte);spin_unlock (ptl); }#endif -pte_unmap (page_table); +pte_unmap (vmf->pte); + vmf->pte =NULL ;return same; }
@@ -3487,7 +3487,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)vm_fault_t ret =0 ;void *shadow =NULL ; -if (!pte_unmap_same (vma->vm_mm, vmf->pmd, vmf->pte, vmf->orig_pte)) +if (!pte_unmap_same (vmf)) goto out; entry =pte_to_swp_entry (vmf->orig_pte);
Signature · 1 line
2.31.1
Peter Xu
reviewerPATCH v3 3/5 mm: Drop first_index/last_index in zap_details
In reply to Peter XuThe first_index/last_index parameters in zap_details are actually only used in unmap_mapping_range_tree(). At the meantime, this function is only called by unmap_mapping_pages() once. Instead of passing these two variables through the whole stack of page zapping code, remove them from zap_details and let them simply be parameters of unmap_mapping_range_tree(), which is inlined.
--- include/linux/mm.h | 2 -- mm/memory.c | 29 ++++++++++++++++------------- 2 files changed, 16 insertions(+), 15 deletions(-)
include/linux/mm.h +0 -2
@@ -1722,8 +1722,6 @@ extern void user_shm_unlock(size_t, struct ucounts *);*/ struct zap_details {struct address_space *check_mapping;/* Check page->mapping if set */ -pgoff_t first_index;/* Lowest page->index to unmap */ -pgoff_t last_index;/* Highest page->index to unmap */ struct page *single_page;/* Locked page to be unmapped */ };
mm/memory.c +16 -13
@@ -3321,20 +3321,20 @@ static void unmap_mapping_range_vma(struct vm_area_struct *vma, }static inline void unmap_mapping_range_tree (struct rb_root_cached *root, +pgoff_t first_index, +pgoff_t last_index,struct zap_details *details) {struct vm_area_struct *vma;pgoff_t vba, vea, zba, zea; -vma_interval_tree_foreach (vma, root, - details->first_index, details->last_index) { - + vma_interval_tree_foreach(vma, root, first_index, last_index) { vba = vma->vm_pgoff; vea = vba +vma_pages (vma) -1 ; - zba = details->first_index; + zba = first_index;if (zba < vba) zba = vba; - zea = details->last_index; + zea = last_index;if (zea > vea) zea = vea;
@@ -3360,18 +3360,21 @@ void unmap_mapping_page(struct page *page) {struct address_space *mapping = page->mapping;struct zap_details details = { }; +pgoff_t first_index, last_index;VM_BUG_ON (!PageLocked (page));VM_BUG_ON (PageTail (page)); + first_index = page->index; + last_index = page->index +thp_nr_pages (page) -1 ; + details.check_mapping = mapping; - details.first_index = page->index; - details.last_index = page->index +thp_nr_pages (page) -1 ; details.single_page = page;i_mmap_lock_write (mapping);if (unlikely (!RB_EMPTY_ROOT (&mapping->i_mmap.rb_root))) -unmap_mapping_range_tree (&mapping->i_mmap, &details); +unmap_mapping_range_tree (&mapping->i_mmap, first_index, + last_index, &details);i_mmap_unlock_write (mapping); }
@@ -3390,17 +3393,17 @@ void unmap_mapping_page(struct page *page)void unmap_mapping_pages (struct address_space *mapping,pgoff_t start,pgoff_t nr,bool even_cows) { +pgoff_t first_index = start, last_index = start + nr -1 ;struct zap_details details = { }; details.check_mapping = even_cows ?NULL : mapping; - details.first_index = start; - details.last_index = start + nr -1 ; -if (details.last_index < details.first_index) - details.last_index =ULONG_MAX ; +if (last_index < first_index) + last_index =ULONG_MAX ;i_mmap_lock_write (mapping);if (unlikely (!RB_EMPTY_ROOT (&mapping->i_mmap.rb_root))) -unmap_mapping_range_tree (&mapping->i_mmap, &details); +unmap_mapping_range_tree (&mapping->i_mmap, first_index, + last_index, &details);i_mmap_unlock_write (mapping); }
Signature · 1 line
2.31.1
Re: [PATCH v3 3/5] mm: Drop first_index/last_index in zap_details
Quoted 83 lines · Peter Xu
The first_index/last_index parameters in
zap_detailsare actually only used inunmap_mapping_range_tree(). At the meantime, this function is only called byunmap_mapping_pages()once. Instead of passing these two variables through the whole stack of page zapping code, remove them fromzap_detailsand let them simply be parameters ofunmap_mapping_range_tree(), which is inlined.Reviewed-by Alistair Popple <[email protected]>Reviewed-by David Hildenbrand <[email protected]>Signed-off-by Peter Xu <[email protected]>--- include/linux/mm.h | 2 -- mm/memory.c | 29 ++++++++++++++++------------- 2 files changed, 16 insertions(+), 15 deletions(-)include/linux/mm.h +0
-2@@ -1722,8 +1722,6 @@ extern void user_shm_unlock(size_t, struct ucounts *);*/ struct zap_details {struct address_space *check_mapping;/* Check page->mapping if set */ -pgoff_t first_index;/* Lowest page->index to unmap */ -pgoff_t last_index;/* Highest page->index to unmap */ struct page *single_page;/* Locked page to be unmapped */ };mm/memory.c +11
-8@@ -3321,20 +3321,20 @@ static void unmap_mapping_range_vma(struct vm_area_struct *vma, }static inline void unmap_mapping_range_tree (struct rb_root_cached *root, +pgoff_t first_index, +pgoff_t last_index,struct zap_details *details) {struct vm_area_struct *vma;pgoff_t vba, vea, zba, zea; -vma_interval_tree_foreach (vma, root, - details->first_index, details->last_index) { - + vma_interval_tree_foreach(vma, root, first_index, last_index) { vba = vma->vm_pgoff; vea = vba +vma_pages (vma) -1 ; - zba = details->first_index; + zba = first_index;if (zba < vba) zba = vba; - zea = details->last_index; + zea = last_index;if (zea > vea) zea = vea;@@ -3360,18 +3360,21 @@ void unmap_mapping_page(struct page *page) {struct address_space *mapping = page->mapping;struct zap_details details = { }; +pgoff_t first_index, last_index;VM_BUG_ON (!PageLocked (page));VM_BUG_ON (PageTail (page)); + first_index = page->index; + last_index = page->index +thp_nr_pages (page) -1 ; + details.check_mapping = mapping; - details.first_index = page->index; - details.last_index = page->index +thp_nr_pages (page) -1 ; details.single_page = page;i_mmap_lock_write (mapping);if (unlikely (!RB_EMPTY_ROOT (&mapping->i_mmap.rb_root))) -unmap_mapping_range_tree (&mapping->i_mmap, &details); +unmap_mapping_range_tree (&mapping->i_mmap, first_index, + last_index, &details);i_mmap_unlock_write (mapping); }
mm/memory.c void unmap_mapping_page(struct page *page) · line 3396
Quoted 4 lines
mm/memory.c +1
-0@@ -3390,17 +3393,17 @@ void unmap_mapping_page(struct page *page)void unmap_mapping_pages (struct address_space *mapping,pgoff_t start,pgoff_t nr,bool even_cows) { +pgoff_t first_index = start, last_index = start + nr -1 ;
Nit: If you respin, can first_index and last_index be two lines please?
mm/memory.c void unmap_mapping_page(struct page *page) · line 3399
Quoted 7 lines
mm/memory.c +0
-4struct zap_details details = { }; details.check_mapping = even_cows ?NULL : mapping; - details.first_index = start; - details.last_index = start + nr -1 ; -if (details.last_index < details.first_index) - details.last_index =ULONG_MAX ;
Nit: Maybe throw a comment about this being overflow check, if you respin.
Quoted 14 lines
mm/memory.c +4
-1+if (last_index < first_index) + last_index =ULONG_MAX ;i_mmap_lock_write (mapping);if (unlikely (!RB_EMPTY_ROOT (&mapping->i_mmap.rb_root))) -unmap_mapping_range_tree (&mapping->i_mmap, &details); +unmap_mapping_range_tree (&mapping->i_mmap, first_index, + last_index, &details);i_mmap_unlock_write (mapping); }Signature · 1 line
2.31.1
Peter Xu
reviewerRe: [PATCH v3 3/5] mm: Drop first_index/last_index in zap_details
Quoted 7 lines · Liam Howlett
mm/memory.c void unmap_mapping_page(struct page *page) · line 3396
Quoted 5 lines
mm/memory.c +1
-0@@ -3390,17 +3393,17 @@ void unmap_mapping_page(struct page *page)void unmap_mapping_pages (struct address_space *mapping,pgoff_t start,pgoff_t nr,bool even_cows) { +pgoff_t first_index = start, last_index = start + nr -1 ;Nit: If you respin, can
first_indexandlast_indexbe two lines please?
Sure.
Quoted 11 lines
mm/memory.c void unmap_mapping_page(struct page *page) · line 3399
Quoted 8 lines
mm/memory.c +0
-4struct zap_details details = { }; details.check_mapping = even_cows ?NULL : mapping; - details.first_index = start; - details.last_index = start + nr -1 ; -if (details.last_index < details.first_index) - details.last_index =ULONG_MAX ;Nit: Maybe throw a comment about this being overflow check, if you respin.
It may not be "only" an overflow check, e.g., both unmap_mapping_range() and unmap_mapping_pages() allows taking the npages to be zero:
For unmap_mapping_range:
* @holelen: size of prospective hole in bytes. This will be rounded
* up to a PAGE_SIZE boundary. A holelen of zero truncates to the
* end of the file.
For unmap_mapping_pages:
* @nr: Number of pages to be unmapped. 0 to unmap to end of file.
So we must set it to ULONG_MAX to make sure nr==0 will work like that.
I won't bother adding a comment, but if to add it I'll probably also mention about that part on allowing a nr==0 use case, please let me know if you insist.
Quoted 12 lines
Quoted 12 lines
mm/memory.c +4
-1+if (last_index < first_index) + last_index =ULONG_MAX ;i_mmap_lock_write (mapping);if (unlikely (!RB_EMPTY_ROOT (&mapping->i_mmap.rb_root))) -unmap_mapping_range_tree (&mapping->i_mmap, &details); +unmap_mapping_range_tree (&mapping->i_mmap, first_index, + last_index, &details);i_mmap_unlock_write (mapping); }
Quoted 5 lines
Quoted 3 lines
Signature · 1 line
2.31.1Reviewed-by Liam R. Howlett <[email protected]>
Thanks for reviewing.
Signature · 1 line
Peter Xu
Peter Xu
reviewerPATCH v3 4/5 mm: Add zap_skip_check_mapping() helper
In reply to Peter XuUse the helper for the checks. Rename "check_mapping" into "zap_mapping" because "check_mapping" looks like a bool but in fact it stores the mapping itself. When it's set, we check the mapping (it must be non-NULL). When it's cleared we skip the check, which works like the old way.
Move the duplicated comments to the helper too.
--- include/linux/mm.h | 16 +++++++++++++++- mm/memory.c | 29 ++++++----------------------- 2 files changed, 21 insertions(+), 24 deletions(-)
include/linux/mm.h +15 -1
@@ -1721,10 +1721,24 @@ extern void user_shm_unlock(size_t, struct ucounts *);* Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { -struct address_space *check_mapping;/* Check page->mapping if set */ +struct address_space *zap_mapping;/* Check page->mapping if set */ struct page *single_page;/* Locked page to be unmapped */ }; +/* +* We set details->zap_mappings when we want to unmap shared but keep private +* pages. Return true if skip zapping this page, false otherwise. +*/ +static inline bool +zap_skip_check_mapping (struct zap_details *details,struct page *page) +{ +if (!details || !page) +return false; + +return details->zap_mapping && + (details->zap_mapping !=page_rmapping (page)); +} +struct page *vm_normal_page (struct vm_area_struct *vma,unsigned long addr,pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr,
mm/memory.c +6 -23
@@ -1333,16 +1333,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,struct page *page; page =vm_normal_page (vma, addr, ptent); -if (unlikely (details) && page) { -/* -* unmap_shared_mapping_pages() wants to -* invalidate cache without truncating: -* unmap shared but keep private pages. -*/ -if (details->check_mapping && - details->check_mapping !=page_rmapping (page)) -continue ; - } +if (unlikely (zap_skip_check_mapping (details, page))) +continue ; ptent =ptep_get_and_clear_full (mm, addr, pte, tlb->fullmm);tlb_remove_tlb_entry (tlb, pte, addr);
@@ -1375,17 +1367,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, is_device_exclusive_entry(entry)) {struct page *page =pfn_swap_entry_to_page (entry); -if (unlikely (details && details->check_mapping)) { -/* -* unmap_shared_mapping_pages() wants to -* invalidate cache without truncating: -* unmap shared but keep private pages. -*/ -if (details->check_mapping != -page_rmapping (page)) -continue ; - } - +if (unlikely (zap_skip_check_mapping (details, page))) +continue ;pte_clear_not_present_full (mm, addr, pte, tlb->fullmm); rss[mm_counter (page)]--;
@@ -3368,7 +3351,7 @@ void unmap_mapping_page(struct page *page) first_index = page->index; last_index = page->index +thp_nr_pages (page) -1 ; - details.check_mapping = mapping; + details.zap_mapping = mapping; details.single_page = page;i_mmap_lock_write (mapping);
@@ -3396,7 +3379,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start,pgoff_t first_index = start, last_index = start + nr -1 ;struct zap_details details = { }; - details.check_mapping = even_cows ?NULL : mapping; + details.zap_mapping = even_cows ?NULL : mapping;if (last_index < first_index) last_index =ULONG_MAX ;
Signature · 1 line
2.31.1
Re: [PATCH v3 4/5] mm: Add zap_skip_check_mapping() helper
Quoted 105 lines · AEST Peter Xu
Use the helper for the checks. Rename "
check_mapping" into "zap_mapping" because "check_mapping" looks like a bool but in fact it stores the mapping itself. When it's set, we check the mapping (it must be non-NULL). When it's cleared we skip the check, which works like the old way.Move the duplicated comments to the helper too.
Signed-off-by Peter Xu <[email protected]>--- include/linux/mm.h | 16 +++++++++++++++- mm/memory.c | 29 ++++++----------------------- 2 files changed, 21 insertions(+), 24 deletions(-)include/linux/mm.h +15
-1@@ -1721,10 +1721,24 @@ extern void user_shm_unlock(size_t, struct ucounts *);* Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details { -struct address_space *check_mapping;/* Check page->mapping if set */ +struct address_space *zap_mapping;/* Check page->mapping if set */ struct page *single_page;/* Locked page to be unmapped */ }; +/* +* We set details->zap_mappings when we want to unmap shared but keep private +* pages. Return true if skip zapping this page, false otherwise. +*/ +static inline bool +zap_skip_check_mapping (struct zap_details *details,struct page *page) +{ +if (!details || !page) +return false; + +return details->zap_mapping && + (details->zap_mapping !=page_rmapping (page)); +} +struct page *vm_normal_page (struct vm_area_struct *vma,unsigned long addr,pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr,mm/memory.c +6
-23@@ -1333,16 +1333,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,struct page *page; page =vm_normal_page (vma, addr, ptent); -if (unlikely (details) && page) { -/* -* unmap_shared_mapping_pages() wants to -* invalidate cache without truncating: -* unmap shared but keep private pages. -*/ -if (details->check_mapping && - details->check_mapping !=page_rmapping (page)) -continue ; - } +if (unlikely (zap_skip_check_mapping (details, page))) +continue ; ptent =ptep_get_and_clear_full (mm, addr, pte, tlb->fullmm);tlb_remove_tlb_entry (tlb, pte, addr);@@ -1375,17 +1367,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, is_device_exclusive_entry(entry)) {struct page *page =pfn_swap_entry_to_page (entry); -if (unlikely (details && details->check_mapping)) { -/* -* unmap_shared_mapping_pages() wants to -* invalidate cache without truncating: -* unmap shared but keep private pages. -*/ -if (details->check_mapping != -page_rmapping (page)) -continue ; - } - +if (unlikely (zap_skip_check_mapping (details, page))) +continue ;pte_clear_not_present_full (mm, addr, pte, tlb->fullmm); rss[mm_counter (page)]--;@@ -3368,7 +3351,7 @@ void unmap_mapping_page(struct page *page) first_index = page->index; last_index = page->index +thp_nr_pages (page) -1 ; - details.check_mapping = mapping; + details.zap_mapping = mapping; details.single_page = page;i_mmap_lock_write (mapping);@@ -3396,7 +3379,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start,pgoff_t first_index = start, last_index = start + nr -1 ;struct zap_details details = { }; - details.check_mapping = even_cows ?NULL : mapping; + details.zap_mapping = even_cows ?NULL : mapping;if (last_index < first_index) last_index =ULONG_MAX ;
Peter Xu
reviewerPATCH v3 5/5 mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags
In reply to Peter XuFirstly, the comment in zap_pte_range() is misleading because it checks against details rather than check_mappings, so it's against what the code did.
Meanwhile, there's no explicit reason why passing in the details pointer should mean to skip all swap entries. New user of zap_details could very possibly miss this fact if they don't read deep until zap_pte_range() because there's no comment at zap_details talking about it at all, so swap entries could be erroneously skipped without being noticed.
This partly reverts 3e8715fdc03e ("mm: drop zap_details::check_swap_entries"), but introduce ZAP_FLAG_SKIP_SWAP flag, which means the opposite of previous "details" parameter: the caller should explicitly set this to skip swap entries, otherwise swap entries will always be considered (which should still be the major case here).
We may want to look into when exactly we need ZAP_FLAG_SKIP_SWAP and we should have it in a synchronous manner, e.g., currently even if ZAP_FLAG_SKIP_SWAP is set we'll still look into swap pmds no matter what. But that should be a separate effort of this patch.
The flag introduced in this patch will be a preparation for more bits defined in the future, e.g., for a new bit in flag to show whether to persist the upcoming uffd-wp bit in pgtable entries.
--- include/linux/mm.h | 16 ++++++++++++++++ mm/memory.c | 6 +++--- 2 files changed, 19 insertions(+), 3 deletions(-)
include/linux/mm.h +16 -0
@@ -1717,12 +1717,18 @@ static inline bool can_do_mlock(void) { return false; }extern int user_shm_lock (size_t ,struct ucounts *);extern void user_shm_unlock (size_t ,struct ucounts *); +typedef unsigned int __bitwise zap_flags_t ; + +/* Whether to skip zapping swap entries */ +#define ZAP_FLAG_SKIP_SWAP ((__force zap_flags_t) BIT(0)) +/* * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details {struct address_space *zap_mapping;/* Check page->mapping if set */ struct page *single_page;/* Locked page to be unmapped */ +zap_flags_t zap_flags;/* Extra flags for zapping */ };/*
@@ -1739,6 +1745,16 @@ zap_skip_check_mapping(struct zap_details *details, struct page *page) (details->zap_mapping !=page_rmapping (page)); } +/* Return true if skip swap entries, false otherwise */ +static inline bool +zap_skip_swap (struct zap_details *details) +{ +if (!details) +return false; + +return details->zap_flags &ZAP_FLAG_SKIP_SWAP ; +} +struct page *vm_normal_page (struct vm_area_struct *vma,unsigned long addr,pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr,
mm/memory.c +3 -3
@@ -1379,8 +1379,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,continue ; } -/* If details->check_mapping, we leave swap entries. */ -if (unlikely (details)) +if (unlikely (zap_skip_swap (details)))continue ;if (!non_swap_entry (entry))
@@ -3353,6 +3352,7 @@ void unmap_mapping_page(struct page *page) details.zap_mapping = mapping; details.single_page = page; + details.zap_flags =ZAP_FLAG_SKIP_SWAP ;i_mmap_lock_write (mapping);if (unlikely (!RB_EMPTY_ROOT (&mapping->i_mmap.rb_root)))
@@ -3377,7 +3377,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) {pgoff_t first_index = start, last_index = start + nr -1 ; -struct zap_details details = { }; +struct zap_details details = { .zap_flags =ZAP_FLAG_SKIP_SWAP }; details.zap_mapping = even_cows ?NULL : mapping;if (last_index < first_index)
Signature · 1 line
2.31.1
Re: [PATCH v3 5/5] mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags
Quoted 15 lines · AEST Peter Xu
Firstly, the comment in
zap_pte_range()is misleading because it checks against details rather thancheck_mappings, so it's against what the code did.Meanwhile, there's no explicit reason why passing in the details pointer should mean to skip all swap entries. New user of
zap_detailscould very possibly miss this fact if they don't read deep untilzap_pte_range()because there's no comment atzap_detailstalking about it at all, so swap entries could be erroneously skipped without being noticed.This partly reverts
3e8715fdc03e("mm: dropzap_details::check_swap_entries"), but introduceZAP_FLAG_SKIP_SWAPflag, which means the opposite of previous "details" parameter: the caller should explicitly set this to skip swap entries, otherwise swap entries will always be considered (which should still be the major case here).
Quoted 4 lines
We may want to look into when exactly we need
ZAP_FLAG_SKIP_SWAPand we should have it in a synchronous manner, e.g., currently even ifZAP_FLAG_SKIP_SWAPis set we'll still look into swap pmds no matter what. But that should be a separate effort of this patch.
I didn't really follow what you mean by "synchronous" here, although the explanation about pmds makes sense so it's probably just terminology.
Quoted 3 lines
The flag introduced in this patch will be a preparation for more bits defined in the future, e.g., for a new bit in flag to show whether to persist the upcoming uffd-wp bit in pgtable entries.
That's kind of the problem. The patch itself looks correct to me however as mentioned it is mostly reverting a previous cleanup and it's hard to tell why that's justified without the subsequent patches. Perhaps it makes the usage of zap_details a bit clearer, but a comment also would with less code.
I know you want to try and shrink the uffd-wp series but I think this patch might be easier to review if it was included as part of that series.
Quoted 80 lines
Cc Kirill A. Shutemov <[email protected]>Cc Hugh Dickins <[email protected]>Signed-off-by Peter Xu <[email protected]>--- include/linux/mm.h | 16 ++++++++++++++++ mm/memory.c | 6 +++--- 2 files changed, 19 insertions(+), 3 deletions(-)include/linux/mm.h +16
-0@@ -1717,12 +1717,18 @@ static inline bool can_do_mlock(void) { return false; }extern int user_shm_lock (size_t ,struct ucounts *);extern void user_shm_unlock (size_t ,struct ucounts *); +typedef unsigned int __bitwise zap_flags_t ; + +/* Whether to skip zapping swap entries */ +#define ZAP_FLAG_SKIP_SWAP ((__force zap_flags_t) BIT(0)) +/* * Parameter block passed down to zap_pte_range in exceptional cases. */ struct zap_details {struct address_space *zap_mapping;/* Check page->mapping if set */ struct page *single_page;/* Locked page to be unmapped */ +zap_flags_t zap_flags;/* Extra flags for zapping */ };/* @@ -1739,6 +1745,16 @@ zap_skip_check_mapping(struct zap_details *details, struct page *page) (details->zap_mapping !=page_rmapping (page)); } +/* Return true if skip swap entries, false otherwise */ +static inline bool +zap_skip_swap (struct zap_details *details) +{ +if (!details) +return false; + +return details->zap_flags &ZAP_FLAG_SKIP_SWAP ; +} +struct page *vm_normal_page (struct vm_area_struct *vma,unsigned long addr,pte_t pte); struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr,mm/memory.c +3
-3@@ -1379,8 +1379,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,continue ; } -/* If details->check_mapping, we leave swap entries. */ -if (unlikely (details)) +if (unlikely (zap_skip_swap (details)))continue ;if (!non_swap_entry (entry))@@ -3353,6 +3352,7 @@ void unmap_mapping_page(struct page *page) details.zap_mapping = mapping; details.single_page = page; + details.zap_flags =ZAP_FLAG_SKIP_SWAP ;i_mmap_lock_write (mapping);if (unlikely (!RB_EMPTY_ROOT (&mapping->i_mmap.rb_root)))@@ -3377,7 +3377,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows) {pgoff_t first_index = start, last_index = start + nr -1 ; -struct zap_details details = { }; +struct zap_details details = { .zap_flags =ZAP_FLAG_SKIP_SWAP }; details.zap_mapping = even_cows ?NULL : mapping;if (last_index < first_index)
Peter Xu
reviewerRe: [PATCH v3 5/5] mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags
Hi, Alistair,
Quoted 16 lines · Alistair Popple
Quoted 15 lines · AEST Peter Xu
Firstly, the comment in
zap_pte_range()is misleading because it checks against details rather thancheck_mappings, so it's against what the code did.Meanwhile, there's no explicit reason why passing in the details pointer should mean to skip all swap entries. New user of
zap_detailscould very possibly miss this fact if they don't read deep untilzap_pte_range()because there's no comment atzap_detailstalking about it at all, so swap entries could be erroneously skipped without being noticed.This partly reverts
3e8715fdc03e("mm: dropzap_details::check_swap_entries"), but introduceZAP_FLAG_SKIP_SWAPflag, which means the opposite of previous "details" parameter: the caller should explicitly set this to skip swap entries, otherwise swap entries will always be considered (which should still be the major case here).
Quoted 7 lines
Quoted 4 lines
We may want to look into when exactly we need
ZAP_FLAG_SKIP_SWAPand we should have it in a synchronous manner, e.g., currently even ifZAP_FLAG_SKIP_SWAPis set we'll still look into swap pmds no matter what. But that should be a separate effort of this patch.I didn't really follow what you mean by "synchronous" here, although the explanation about pmds makes sense so it's probably just terminology.
Yes, maybe I should use "aligned manner", or please suggest anything that sounds better; sorry for my awkward English.
Quoted 5 lines
Quoted 4 lines
The flag introduced in this patch will be a preparation for more bits defined in the future, e.g., for a new bit in flag to show whether to persist the upcoming uffd-wp bit in pgtable entries.
Quoted 7 lines
That's kind of the problem. The patch itself looks correct to me however as mentioned it is mostly reverting a previous cleanup and it's hard to tell why that's justified without the subsequent patches. Perhaps it makes the usage of
zap_detailsa bit clearer, but a comment also would with less code.I know you want to try and shrink the uffd-wp series but I think this patch might be easier to review if it was included as part of that series.
I posted it because I think it's suitable to have it even without uffd-wp.
I tried to explain it above on two things this patch wanted to fix:
Firstly the comment is wrong; we've moved back and forth on changing the zap_details flags but the comment is not changing along the way and it's not matching the code right now.
Secondly I do think we should have a flag showing explicit willingness to skip swap entries. Yes, uffd-wp is the planned new one, but my point is anyone who will introduce a new user of zap_details pointer could overlook this fact. The new flag helps us to make sure someone will at least read the flags and know what'll happen with it.
For the 2nd reasoning, I also explicitly CCed Kirill too, so Kirill can provide any comment if he disagrees. For now, I still think we should keep having such a flag otherwise it could be error-prone.
Could you buy-in above reasoning?
Basically above is what I wanted to express in my commit message. I hope that can justify that this patch (even if extremly simple) can still be considered as acceptable upstream even without uffd-wp series.
If you still insist on this patch not suitable for standalone merging and especially if some other reviewer would think the same, I can move it back to uffd-wp series for sure. Then I'll repost this series with 4 patches only.
In all cases, thanks for looking at the series.
Signature · 1 line
Peter Xu
Re: [PATCH v3 5/5] mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags
Quoted 26 lines · AEST Peter Xu
Quoted 11 lines
Quoted 3 lines
The flag introduced in this patch will be a preparation for more bits defined in the future, e.g., for a new bit in flag to show whether to persist the upcoming uffd-wp bit in pgtable entries.
That's kind of the problem. The patch itself looks correct to me however as mentioned it is mostly reverting a previous cleanup and it's hard to tell why that's justified without the subsequent patches. Perhaps it makes the usage of
zap_detailsa bit clearer, but a comment also would with less code.I know you want to try and shrink the uffd-wp series but I think this patch might be easier to review if it was included as part of that series.
I posted it because I think it's suitable to have it even without uffd-wp.
I tried to explain it above on two things this patch wanted to fix:
Firstly the comment is wrong; we've moved back and forth on changing the
zap_detailsflags but the comment is not changing along the way and it's not matching the code right now.Secondly I do think we should have a flag showing explicit willingness to skip swap entries. Yes, uffd-wp is the planned new one, but my point is anyone who will introduce a new user of
zap_detailspointer could overlook this fact. The new flag helps us to make sure someone will at least read the flags and know what'll happen with it.
Quoted 5 lines
For the 2nd reasoning, I also explicitly CCed Kirill too, so Kirill can provide any comment if he disagrees. For now, I still think we should keep having such a flag otherwise it could be error-prone.
Could you buy-in above reasoning?
Kind of, I do think it makes the usage of details a bit clearer or at least harder to miss. It is just that if that was the sole aim of this patch I think there might be simpler (less code) ways of doing so.
Quoted 7 lines
Basically above is what I wanted to express in my commit message. I hope that can justify that this patch (even if extremly simple) can still be considered as acceptable upstream even without uffd-wp series.
If you still insist on this patch not suitable for standalone merging and especially if some other reviewer would think the same, I can move it back to uffd-wp series for sure. Then I'll repost this series with 4 patches only.
I won't insist, the code looks correct and it doesn't make things any less clear so you can put my Reviewed-by on it and perhaps leave it to Andrew or another reviewer to determine if this should be taken in this series or as part of a future uffd-wp series.
- Alistair
Quoted 3 lines
In all cases, thanks for looking at the series.
Peter Xu
reviewerRe: [PATCH v3 5/5] mm: Add ZAP_FLAG_SKIP_SWAP and zap_flags
Quoted 32 lines · Alistair Popple
Quoted 30 lines · AEST Peter Xu
Quoted 11 lines
Quoted 3 lines
The flag introduced in this patch will be a preparation for more bits defined in the future, e.g., for a new bit in flag to show whether to persist the upcoming uffd-wp bit in pgtable entries.
That's kind of the problem. The patch itself looks correct to me however as mentioned it is mostly reverting a previous cleanup and it's hard to tell why that's justified without the subsequent patches. Perhaps it makes the usage of
zap_detailsa bit clearer, but a comment also would with less code.I know you want to try and shrink the uffd-wp series but I think this patch might be easier to review if it was included as part of that series.
I posted it because I think it's suitable to have it even without uffd-wp.
I tried to explain it above on two things this patch wanted to fix:
Firstly the comment is wrong; we've moved back and forth on changing the
zap_detailsflags but the comment is not changing along the way and it's not matching the code right now.Secondly I do think we should have a flag showing explicit willingness to skip swap entries. Yes, uffd-wp is the planned new one, but my point is anyone who will introduce a new user of
zap_detailspointer could overlook this fact. The new flag helps us to make sure someone will at least read the flags and know what'll happen with it.For the 2nd reasoning, I also explicitly CCed Kirill too, so Kirill can provide any comment if he disagrees. For now, I still think we should keep having such a flag otherwise it could be error-prone.
Quoted 5 lines
Quoted 1 line
Could you buy-in above reasoning?
Kind of, I do think it makes the usage of details a bit clearer or at least harder to miss. It is just that if that was the sole aim of this patch I think there might be simpler (less code) ways of doing so.
Yes you're right, we can add a big enough comment above zap_details to state that, but then it'll be reverted when adding the uffd-wp flag in the other series, because uffd-wp will still needs a way to specify !SKIP_SWAP and KEEP_UFFD_WP. Then it'll make the "series split" make less sense as you said.
I split the series only because I hope it could ease the reviewers, and also that's probably the only thing I can do now to still try to smooth the process of having a complete uffd-wp finally got proper reviewed and merged.
Quoted 5 lines
Quoted 5 lines
Basically above is what I wanted to express in my commit message. I hope that can justify that this patch (even if extremly simple) can still be considered as acceptable upstream even without uffd-wp series.
Quoted 8 lines
Quoted 3 lines
If you still insist on this patch not suitable for standalone merging and especially if some other reviewer would think the same, I can move it back to uffd-wp series for sure. Then I'll repost this series with 4 patches only.
I won't insist, the code looks correct and it doesn't make things any less clear so you can put my Reviewed-by on it and perhaps leave it to Andrew or another reviewer to determine if this should be taken in this series or as part of a future uffd-wp series.
Will do; thanks.
Signature · 1 line
Peter Xu
Peter Xu
reviewerQuoted 11 lines · Peter Xu
[Based on linus/master, commit
ac08b1c68d1bwhich should contain the recent -mm
pull, if not applicable, I can repost anytime]v3:
- Patch "mm: Addzap_skip_check_mapping()helper"
- Inzap_skip_check_mapping()checkzap_mappingfirst [Alistair]
- Patch "mm: AddZAP_FLAG_SKIP_SWAPandzap_flags":
- Fix English errors in commit message [David]
- Drop paragraph mentioning commit22061a1ffabdin commit msg
- SetZAP_FLAG_SKIP_SWAPforunmap_mapping_page()too
- Pick up r-bs
Ping - patches 1-4 have at least 1 ack already, the last patch is missing ack.
The change is quite straightforward as right now we apply SKIP_SWAP to all "details" users of zapping process, so no functional change intended. It'll be important for uffd-wp in the future because uffd-wp will start to use "details" without setting SKIP_SWAP.
It would be great if anyone could help have a look at the last patch, thanks.
Signature · 1 line
Peter Xu