Skip to content

mm: COW fixes part 2: reliable GUP pins of anonymous pages

David Hildenbrand linux-mm lkml 4 versions – Applied

Status from this message: [merged mm-stable] mm-rmap-remove-do_page_add_anon_rmap.patch removed from -mm tree

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest17 patches 8 repliesApplied 17/17
  2. v316 patches 34 repliesSuperseded
  3. v215 patches 22 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v115 patches23 repliesSuperseded

Changes in v4 from the cover letter

* Minor changes/fixes in patch descriptions
* "mm/rmap: drop "compound" parameter from page_add_new_anon_rmap()" -> Remove VM_BUG_ON_PAGE(PageTransCompound(page), page);
* "mm/rmap: fail try_to_migrate() early when setting a PMD migration entry fails"
-> Added
* "mm: support GUP-triggered unsharing of anonymous pages"
-> unlikely(unshare) -> likely(!unshare) in wp_huge_pmd()
-> "page_copied && !unshare" -> "(page_copied && !unshare)"
* "mm/gup: sanity-check with CONFIG_DEBUG_VM that anonymous pages are exclusive when (un)pinning"
-> Use VM_BUG_ON_PAGE() instead of VM_BUG_ON()

Patches in v4 All 17 applied

#Subject and filesSizeRepliesApplied
1/17mm/rmap: fix missing swap_free() in try_to_unmap() after arch_unmap_one() failedmm/rmap.c+1 -00 repliesApplied
2/17mm/hugetlb: take src_mm->write_protect_seq in copy_hugetlb_page_range()include/linux/mm.h mm/hugetlb.c+10 -20 repliesApplied
3/17mm/memory: slightly simplify copy_present_pte()mm/memory.c+16 -370 repliesApplied
4/17mm/rmap: split page_dup_rmap() into page_dup_file_rmap() and page_try_dup_anon_rmap()include/linux/mm.h include/linux/rmap.h mm/huge_memory.c +3 more+79 -370 repliesApplied
5/17mm/rmap: convert RMAP flags to a proper distinct rmap_t typeinclude/linux/rmap.h mm/memory.c mm/rmap.c+25 -100 repliesApplied
6/17mm/rmap: remove do_page_add_anon_rmap()include/linux/rmap.h mm/huge_memory.c mm/ksm.c +4 more+8 -210 repliesApplied
7/17mm/rmap: pass rmap flags to hugepage_add_anon_rmap()include/linux/rmap.h mm/migrate.c mm/rmap.c+9 -50 repliesApplied
8/17mm/rmap: drop "compound" parameter from page_add_new_anon_rmap()include/linux/rmap.h kernel/events/uprobes.c mm/huge_memory.c +6 more+19 -170 repliesApplied
9/17mm/rmap: use page_move_anon_rmap() when reusing a mapped PageAnon() page exclusivelymm/huge_memory.c mm/memory.c+3 -00 repliesApplied
10/17mm/huge_memory: remove outdated VM_WARN_ON_ONCE_PAGE from unmap_page()mm/huge_memory.c+0 -20 repliesApplied
11/17mm/page-flags: reuse PG_mappedtodisk as PG_anon_exclusive for PageAnon() pagesinclude/linux/page-flags.h mm/hugetlb.c mm/memory.c +3 more+71 -20 repliesApplied
12/17mm: remember exclusively mapped anonymous pages with PG_anon_exclusiveinclude/linux/rmap.h include/linux/swap.h include/linux/swapops.h +8 more+289 -347 repliesApplied
13/17mm/rmap: fail try_to_migrate() early when setting a PMD migration entry failsinclude/linux/swapops.h mm/huge_memory.c mm/rmap.c+12 -60 repliesApplied
14/17mm/gup: disallow follow_page(FOLL_PIN)mm/gup.c mm/hugetlb.c+8 -30 repliesApplied
15/17mm: support GUP-triggered unsharing of anonymous pagesinclude/linux/mm_types.h mm/huge_memory.c mm/hugetlb.c +1 more+126 -550 repliesApplied
16/17mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared anonymous pageinclude/linux/mm.h mm/gup.c mm/huge_memory.c +1 more+105 -60 repliesApplied
17/17mm/gup: sanity-check with CONFIG_DEBUG_VM that anonymous pages are exclusive when (un)pinningmm/gup.c mm/huge_memory.c mm/hugetlb.c+66 -10 repliesApplied

Reviews trailers per patch

15 of 17 patches have a Reviewed-by or Acked-by.

PatchKhalid AzizVlastimil BabkaYang Shi
1/17R-b A-b
2/17A-b
3/17A-b
4/17A-b
5/17A-b
6/17A-b
7/17A-b
8/17A-b
9/17A-b
10/17A-b R-b
11/17A-b
12/17
13/17
14/17A-b
15/17A-b
16/17A-b
17/17A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v4 thread

Cover letter [PATCH v4 00/17] mm: COW fixes part 2: reliable GUP pins of anonymous pages

This is roughly what we have in -mm and -next, however, includes one additional patch and some minor differences, especially minor fixes in the patch descriptions.

v4 is located at:
https://github.com/davidhildenbrand/linux/tree/cow_fixes_part_2_v4

Please refer to to v3 cover letter:

	https://lkml.kernel.org/r/[email protected]

v3 -> v4:
* Minor changes/fixes in patch descriptions
* "mm/rmap: drop "compound" parameter from page_add_new_anon_rmap()" -> Remove VM_BUG_ON_PAGE(PageTransCompound(page), page);
* "mm/rmap: fail try_to_migrate() early when setting a PMD migration entry fails"
-> Added
* "mm: support GUP-triggered unsharing of anonymous pages"
-> unlikely(unshare) -> likely(!unshare) in wp_huge_pmd()
-> "page_copied && !unshare" -> "(page_copied && !unshare)"
* "mm/gup: sanity-check with CONFIG_DEBUG_VM that anonymous pages are exclusive when (un)pinning"
-> Use VM_BUG_ON_PAGE() instead of VM_BUG_ON()

v2 -> v3:
* Note 1: Left the terminology "unshare" in place for now instead of switching to "make anon exclusive".
* Note 2: We might have to tackle undoing effects of arch_unmap_one() on sparc, to free some tag memory immediately instead of when tearing down the vma/mm; looks like this needs more care either way, so I'll ignore it for now.
* Rebased on top of core MM changes for v5.18-rc1 (most conflicts were due to folio and ZONE_DEVICE migration rework). No severe changes were necessary -- mostly folio conversion and code movement.
* Retested on aarch64, ppc64, s390x and x86_64
* "mm/rmap: convert RMAP flags to a proper distinct rmap_t type" -> Missed to convert one instance in restore_exclusive_pte()
* "mm/rmap: pass rmap flags to hugepage_add_anon_rmap()"
-> Use "!!(flags & RMAP_EXCLUSIVE)" to avoid sparse warnings
* "mm/huge_memory: remove outdated VM_WARN_ON_ONCE_PAGE from unmap_page()" -> Added, as we can trigger that now more frequently
* "mm: remember exclusively mapped anonymous pages with PG_anon_exclusive" -> Use subpage in VM_BUG_ON_PAGE() in try_to_migrate_one()
-> Move comment from folio_migrate_mapping() to folio_migrate_flags()
regarding PG_anon_exclusive/PG_mappedtodisk
-> s/int rmap_flags/rmap_t rmap_flags/ in remove_migration_pmd()
* "mm/gup: sanity-check with CONFIG_DEBUG_VM that anonymous pages are exclusive when (un)pinning"
-> Use IS_ENABLED(CONFIG_DEBUG_VM) instead of ifdef

v1 -> v2:
* Tested on aarch64, ppc64, s390x and x86_64
* "mm/page-flags: reuse PG_mappedtodisk as PG_anon_exclusive for PageAnon() pages"
-> Use PG_mappedtodisk instead of PG_slab (thanks Willy!), this simlifies
the patch and necessary handling a lot. Add safety BUG_ON's
-> Move most documentation to the patch description, to be placed in a
proper documentation doc in the future, once everything's in place
* ""mm: remember exclusively mapped anonymous pages with PG_anon_exclusive -> Skip check+clearing in page_try_dup_anon_rmap(), otherwise we might trigger a wrong VM_BUG_ON() for KSM pages in ClearPageAnonExclusive() -> In __split_huge_pmd_locked(), call page_try_share_anon_rmap() only for "anon_exclusive", otherwise we might trigger a wrong VM_BUG_ON() -> In __split_huge_page_tail(), drop any remaining PG_anon_exclusive on tail pages, and document why that is fine

RFC -> v1:
* Rephrased/extended some patch descriptions+comments
* Tested on aarch64, ppc64 and x86_64
* "mm/rmap: convert RMAP flags to a proper distinct rmap_t type" -> Added
* "mm/rmap: drop "compound" parameter from page_add_new_anon_rmap()" -> Added
* "mm: remember exclusively mapped anonymous pages with PG_anon_exclusive" -> Fixed __do_huge_pmd_anonymous_page() to recheck after temporarily dropping the PT lock.
-> Use "reuse" label in __do_huge_pmd_anonymous_page()
-> Slightly simplify logic in hugetlb_cow()
-> In remove_migration_pte(), remove unrelated changes around
page_remove_rmap()
* "mm: support GUP-triggered unsharing of anonymous pages"
-> In handle_pte_fault(), trigger pte_mkdirty() only with
FAULT_FLAG_WRITE
-> In __handle_mm_fault(), extend comment regarding anonymous PUDs
* "mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared anonymous page"
-> Added unsharing logic to gup_hugepte() and gup_huge_pud() -> Changed return logic in __follow_hugetlb_must_fault(), making sure
that "unshare" is always set
* "mm/gup: sanity-check with CONFIG_DEBUG_VM that anonymous pages are exclusive when (un)pinning"
-> Slightly simplified sanity_check_pinned_pages()

David Hildenbrand (17):
mm/rmap: fix missing swap_free() in try_to_unmap() after
arch_unmap_one() failed
mm/hugetlb: take src_mm->write_protect_seq in
copy_hugetlb_page_range()
mm/memory: slightly simplify copy_present_pte()
mm/rmap: split page_dup_rmap() into page_dup_file_rmap() and
page_try_dup_anon_rmap()
mm/rmap: convert RMAP flags to a proper distinct rmap_t type mm/rmap: remove do_page_add_anon_rmap()
mm/rmap: pass rmap flags to hugepage_add_anon_rmap() mm/rmap: drop "compound" parameter from page_add_new_anon_rmap() mm/rmap: use page_move_anon_rmap() when reusing a mapped PageAnon()
page exclusively
mm/huge_memory: remove outdated VM_WARN_ON_ONCE_PAGE from unmap_page() mm/page-flags: reuse PG_mappedtodisk as PG_anon_exclusive for
PageAnon() pages
mm: remember exclusively mapped anonymous pages with PG_anon_exclusive mm/rmap: fail try_to_migrate() early when setting a PMD migration
entry fails
mm/gup: disallow follow_page(FOLL_PIN)
mm: support GUP-triggered unsharing of anonymous pages mm/gup: trigger FAULT_FLAG_UNSHARE when R/O-pinning a possibly shared
anonymous page
mm/gup: sanity-check with CONFIG_DEBUG_VM that anonymous pages are
exclusive when (un)pinning

 include/linux/mm.h         |  46 +++++++-
 include/linux/mm_types.h   |   8 ++
 include/linux/page-flags.h |  39 ++++++-
 include/linux/rmap.h       | 118 +++++++++++++++++--
 include/linux/swap.h       |  15 ++-
 include/linux/swapops.h    |  29 ++++-
 kernel/events/uprobes.c    |   2 +-
 mm/gup.c                   | 106 ++++++++++++++++-
 mm/huge_memory.c           | 133 ++++++++++++++++-----
 mm/hugetlb.c               | 135 ++++++++++++++-------
 mm/khugepaged.c            |   2 +-
 mm/ksm.c                   |  15 ++-
 mm/memory.c                | 234 +++++++++++++++++++++++--------------
 mm/memremap.c              |   9 ++
 mm/migrate.c               |  18 ++-
 mm/migrate_device.c        |  23 +++-
 mm/mprotect.c              |   8 +-
 mm/rmap.c                  | 105 ++++++++++++-----
 mm/swapfile.c              |   8 +-
 mm/userfaultfd.c           |   2 +-
 tools/vm/page-types.c      |   8 +-
 21 files changed, 836 insertions(+), 227 deletions(-)

base-commit: af2d861d4cd2a4da5137f795ee3509e6f944a25b

Signature · 1 line
2.35.1