Skip to content

RFC mm: use per-VMA lock in __access_remote_vm for improved monitoring reliability

Rik van Riel bpf linux-arm-kernel linux-kselftest linux-mm linux-riscv linux-trace-kernel lkml 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 RFC Latest12 patches 23 repliesIn review
  2. v36 patches 13 repliesSuperseded

Changes in v4 from the cover letter

https://lore.kernel.org/all/[email protected]/

- Address Usama Arif's v3 review:
- p1: correct the Acked-by address to [email protected].
- p2: rewrite the riscv changelog to describe only this patch's change, and replace the false "pmlen is stable" claim with the accurate story (pmlen may change until MM_CONTEXT_LOCK_PMLEN; READ_ONCE avoids a torn value, and the remote untag is best-effort).
- p3: fix the "name space" typo and note "No functional change intended."
- Split the check_vma_flags() ignore-flags change into its own prep patch (patch 4) so the PROT_NONE PFNMAP read hole fix is bisectable, keeping the VM_READ / FOLL_FORCE / arch-key checks for a PFNMAP VMA.
- get_user_page_vma(): add the flush_anon_page()/flush_dcache_page() cache flushes __get_user_pages() does, and reword the faultin_page() kerneldoc to cover the per-VMA lock, not just the mmap lock.
- New patch 7: read remote strings (bpf_copy_from_user_task_str()) under the per-VMA lock, extending the approach to __copy_remote_vm_str().
- Factor __access_remote_vm() and the string reader onto a shared remote_vm_walk() helper, and fix a NUL-termination regression it exposed:
the partial buffer is now terminated on a mid-walk error.
- Fold the mTHP GUP batching (patches 9-12), previously a separate follow-up, into this series; the cover letter now carries its gup_test numbers.
- Correct two changelog claims. The page table lock plus the pmd recheck, not the VMA write lock, guards the walk against concurrent THP collapse; RCU only keeps the retracted PTE page valid, and the VMA write lock excludes munmap(). The x86 untag mask changes while the target is single-threaded, so the lockless read is best-effort, not "set once and never changes."
- Rebase onto the current Linus tree (v7.2-rc5).

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/12x86/mm: add untagged_addr_remote_unlocked()arch/x86/include/asm/mmu_context.h arch/x86/include/asm/uaccess_64.h arch/x86/kernel/process_64.c +1 more+24 -82 replies
2/12riscv/mm: add untagged_addr_remote_unlocked()arch/riscv/include/asm/mmu_context.h arch/riscv/include/asm/uaccess.h arch/riscv/kernel/process.c+16 -102 replies
3/12mm: rename get_user_page_vma_remote() to get_user_page_lookup_vma()arch/arm64/kernel/mte.c arch/x86/kernel/uprobes.c include/linux/mm.h +2 more+6 -61 reply
4/12mm/gup: let check_vma_flags() ignore selected VMA flagsmm/gup.c+11 -33 replies
5/12mm/gup: add get_user_page_vma() to fault in a page under a held lockmm/gup.c mm/internal.h+159 -252 replies
6/12mm: use per-VMA lock in __access_remote_vm() for single-VMA accessesmm/memory.c+213 -551 reply
7/12mm: read remote strings under the per-VMA lockmm/memory.c+47 -640 replies
8/12selftests/mm: cover /proc/pid/mem access to VM_PFNMAP memorytools/testing/selftests/mm/pfnmap.c+66 -01 reply
9/12mm/gup: build get_user_page_lookup_vma() on get_user_page_vma()include/linux/mm.h mm/gup.c+38 -290 replies
10/12mm/gup: pass an end address to follow_page_mask() and return a page countmm/gup.c+56 -460 replies
11/12mm/gup: batch contiguous PTE-mapped large folios in follow_page_mask()mm/gup.c+45 -04 replies
12/12selftests/mm: add a slow-GUP content and COW test for mTHPtools/testing/selftests/mm/Makefile tools/testing/selftests/mm/mthp_gup_cow_test.c tools/testing/selftests/mm/run_vmtests.sh+215 -04 replies

Reviews trailers per patch

4 of 12 patches have a Reviewed-by or Acked-by.

PatchUsama ArifSuren Baghdasaryan
1/12A-b R-b
2/12R-b
3/12A-b R-b
4/12R-b
5/12
6/12
7/12
8/12
9/12
10/12
11/12
12/12

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH RFC v4 0/12] mm: use per-VMA lock in __access_remote_vm for improved monitoring reliability

__access_remote_vm() holds mmap_read_lock() for the whole transfer. On large machines, with large multi-threaded applications, the mmap_lock is often contended, leading to things like reads of /proc/PID/cmdline stalling.

This results in system monitoring tools getting stuck, right when system information would be the most helpful.

That problem can be avoided by having __access_remote_vm() access user memory under the per-VMA lock. The old code also looks up the VMA for every page accessed, and walks the page tables for every page inside a large folio. Cleaning that up speeds things up nicely.

Using code that does not include its own lock dance, and does not look up the VMA twice results in a modest speedup.

	read size   baseline    per-VMA     throughput
	8 B          201.7 ns    170.6 ns    ~15% faster
	4 KB         495 ns      442 ns      +12% (8266 -> 9265 MB/s)
	64 KB       4115 ns     3355 ns      +23% (15927 -> 19536 MB/s)
	1 MB       75902 ns    63641 ns      +19% (13815 -> 16477 MB/s)

The multi-threaded reader-versus-writer contention that motivates the lock is not measured.

The rest of the series builds on this to batch per-VMA GUP. Patches 9-12 have follow_page_mask() report a contiguous PTE-mapped run of a large folio, so the slow pin_user_pages() path walks it once instead of per page. Other paths that walk a range benefit too.

That batching pays off on PTE-mapped mTHP. With mm/gup_test.c (PIN_LONGTERM_BENCHMARK, the slow pin_user_pages() path) on a 256 MB MADV_HUGEPAGE region in a 4 CPU VM, median get time over 16 iterations:

	gup_test -L -m 256 -n 65536 -r 16 -t
	                      before      after
	64 kB mTHP            3140 us      412 us   (7.6x)
	2 MB THP (control)      78 us       76 us
	4 kB base (control)   3010 us     3042 us

The PMD-mapped 2 MB THP already returns in one step, so only the PTE-mapped large folio case improves.

Reaching a page under the per-VMA lock means looking up the VMA first, which needs to untag the remote address without the mmap lock. The untag masks (x86 LAM, riscv pointer masking) change only while the target is single-threaded, so a remote reader races at most one such change.

Those masks are already read locklessly elsewhere, so patches 1 and 2 add untagged_addr_remote_unlocked() helpers with READ_ONCE()/WRITE_ONCE() annotations. A stale read sees the old or new mask, never a torn value, and the remote untag is best-effort.

With the VMA in hand we need one page from it under the lock the caller already holds. get_user_pages_remote() does not fit: it hard codes the mmap lock and re-derives the VMA internally.

Rather than reimplement GUP in yet another caller, this version follows David Hildenbrand's v2 guidance: start from a proper GUP interface that takes a locked VMA, not an mm, and decide which faults can resolve under the VMA lock.

Patch 3 renames get_user_page_vma_remote() to get_user_page_lookup_vma() to free the name. Patch 5 adds get_user_page_vma(): a simplified __get_user_pages() that walks the tables with follow_page_mask(), faults a missing page in with faultin_page(), and returns it with a reference and the caller's lock still held.

Like __get_user_pages() it runs check_vma_flags(), so callers need not pre-check the VMA.

A VM_IO or VM_PFNMAP VMA is the exception: a COWed page with a struct page is returned normally, while a raw PFN yields -EFAULT so the caller can reach vma->vm_ops->access(). The caller sets FOLL_VMA_LOCK when it holds the per-VMA lock, which reaches the fault code as FAULT_FLAG_VMA_LOCK.

Anything the per-VMA lock cannot finish releases the lock and returns -EAGAIN, so the caller retries under the mmap lock: a dropped fault, a userfaultfd VMA (which assumes current is the faulting task), a hard error, or ->access() memory. faultin_page() reports the retry the same way for both lock types.

__access_remote_vm() then uses the per-VMA lock when an access fits within one VMA, and falls back to the mmap lock for multi-VMA accesses, stack expansion, or an -EAGAIN from get_user_page_vma().

Walking page tables under only the per-VMA lock is safe against both page table freeing and THP collapse. munmap() frees a VMA's page tables under the VMA write lock that our read lock excludes, so they cannot be torn down under the walk.

THP collapse needs neither lock: it retracts a PTE page under the page table lock and frees it by RCU.

follow_page_pte() takes that same lock through pte_offset_map_lock() and rechecks the pmd, so it either walks an intact table or sees the collapsed pmd and faults the page in, while RCU keeps the retracted page valid.

A COWed page in a VM_PFNMAP mapping was previously unreachable through /proc/pid/mem because generic_access_phys() rejects ioremaps of RAM. With get_user_page_vma() returning the struct page directly, that read now succeeds, which the new selftests cover along with the raw PFN path.

Thanks to Suren for pointing out the need, to David for pushing me to just rework the whole area, to Usama for reviewing the arch patches, and Lorenzo for the changelog and cover letter cleanups.

There is a lot more work that could be done in this area, but the patch series is big enough. More changes can come in follow up work.

---
 arch/arm64/kernel/mte.c                        |   2 +-
 arch/riscv/include/asm/mmu_context.h           |   4 +-
 arch/riscv/include/asm/uaccess.h               |  10 +-
 arch/riscv/kernel/process.c                    |  12 +-
 arch/x86/include/asm/mmu_context.h             |   6 +-
 arch/x86/include/asm/uaccess_64.h              |  15 +-
 arch/x86/kernel/process_64.c                   |   4 +-
 arch/x86/kernel/uprobes.c                      |   2 +-
 include/linux/mm.h                             |  32 +--
 include/linux/uaccess.h                        |   7 +
 mm/gup.c                                       | 360 ++++++++++++++++++-----
 mm/internal.h                                  |   8 +-
 mm/memory.c                                    | 379 +++++++++++++++++--------
 mm/rmap.c                                      |   2 +-
 tools/testing/selftests/mm/Makefile            |   1 +
 tools/testing/selftests/mm/mthp_gup_cow_test.c | 213 ++++++++++++++
 tools/testing/selftests/mm/pfnmap.c            |  66 +++++
 tools/testing/selftests/mm/run_vmtests.sh      |   1 +
 18 files changed, 887 insertions(+), 237 deletions(-)

base-commit: 248951ddc14de84de3910f9b13f51491a8cd91df
---
To: Andrew Morton <[email protected]>
To: [email protected]
To: [email protected]
Cc: Dave Hansen <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Suren Baghdasaryan <[email protected]>
Cc: Lorenzo Stoakes <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: "Liam R. Howlett" <[email protected]>
Cc: Mike Rapoport <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: John Hubbard <[email protected]>
Cc: Peter Xu <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Usama Arif <[email protected]>
Cc: Rik van Riel <[email protected]>

REVIEWERS NOTES:
Series based on the current Linus tree (v7.2-rc5, commit 248951ddc14d). Patches 1-8 add per-VMA lock remote access; patches 9-12 batch per-VMA GUP over PTE-mapped large folios.

Changes since v3 (RFC v3, 2026-07-17):
https://lore.kernel.org/all/[email protected]/

- Address Usama Arif's v3 review:
- p1: correct the Acked-by address to [email protected].
- p2: rewrite the riscv changelog to describe only this patch's change, and replace the false "pmlen is stable" claim with the accurate story (pmlen may change until MM_CONTEXT_LOCK_PMLEN; READ_ONCE avoids a torn value, and the remote untag is best-effort).
- p3: fix the "name space" typo and note "No functional change intended."
- Split the check_vma_flags() ignore-flags change into its own prep patch (patch 4) so the PROT_NONE PFNMAP read hole fix is bisectable, keeping the VM_READ / FOLL_FORCE / arch-key checks for a PFNMAP VMA.
- get_user_page_vma(): add the flush_anon_page()/flush_dcache_page() cache flushes __get_user_pages() does, and reword the faultin_page() kerneldoc to cover the per-VMA lock, not just the mmap lock.
- New patch 7: read remote strings (bpf_copy_from_user_task_str()) under the per-VMA lock, extending the approach to __copy_remote_vm_str().
- Factor __access_remote_vm() and the string reader onto a shared remote_vm_walk() helper, and fix a NUL-termination regression it exposed:
the partial buffer is now terminated on a mid-walk error.
- Fold the mTHP GUP batching (patches 9-12), previously a separate follow-up, into this series; the cover letter now carries its gup_test numbers.
- Correct two changelog claims. The page table lock plus the pmd recheck, not the VMA write lock, guards the walk against concurrent THP collapse; RCU only keeps the retracted PTE page valid, and the VMA write lock excludes munmap(). The x86 untag mask changes while the target is single-threaded, so the lockless read is best-effort, not "set once and never changes."
- Rebase onto the current Linus tree (v7.2-rc5).