mm/userfaultfd: detect VMA type change after copy retry in mfill_copy_folio_retry()
Subsystems (2)
PATCH v7 1/1 mm/userfaultfd: detect VMA type change after copy retry in mfill_copy_folio_retry()
mfill_copy_folio_retry() drops mmap_lock for the copy_from_user() call. During this window, the VMA can be replaced with a different type (e.g. hugetlb), making the caller's ops pointer stale. Subsequent use of the stale ops would dispatch into the wrong per-vma handlers.
Capture the VMA's ops via vma_uffd_ops() before dropping the lock and compare against the current vma_uffd_ops() after re-acquiring it. Return -EAGAIN if they differ so the operation can be retried. This avoids comparing against the caller's ops which may have been overridden to anon_uffd_ops for MAP_PRIVATE file-backed mappings.
---
v7: (akpm review)
- update Fixes: to the current mm-unstable hash
- add Closes: link to Usama's report
- drop "kernel crash" wording; no observed reproducer
- align Reported-by address to [email protected]
- carry Mike's Ack from v5
v6: capture ops via vma_uffd_ops() before dropping the lock so
MAP_PRIVATE shmem (which overrides to anon_uffd_ops) no longer triggers spurious -EAGAIN (Usama).
Drop unused ops parameter from mfill_copy_folio_retry().
v5: initial ops-compare approach.
Tested under virtme-ng (DEBUG_VM, LOCKDEP, PROVE_LOCKING):
uffd-unit-tests: 67 pass, 0 skip, 0 fail uffd-stress {anon,shmem,shmem-private}: 4 bounces each, clean
mm/userfaultfd.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
mm/userfaultfd.c +11 -1
@@ -443,8 +443,10 @@ static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr)return ret; } -static int mfill_copy_folio_retry (struct mfill_state *state,struct folio *folio) +static int mfill_copy_folio_retry (struct mfill_state *state, +struct folio *folio) { +const struct vm_uffd_ops *orig_ops =vma_uffd_ops (state->vma);unsigned long src_addr = state->src_addr;void *kaddr;int err;
@@ -465,6 +467,14 @@ static int mfill_copy_folio_retry(struct mfill_state *state, struct folio *folioif (err)return err; +/* +* The VMA type may have changed while the lock was dropped +* (e.g. replaced with a hugetlb mapping), making the caller's +* ops pointer stale. +*/ +if (vma_uffd_ops (state->vma) != orig_ops) +return -EAGAIN ; + err =mfill_establish_pmd (state);if (err)return err;
Signature · 1 line
2.53.0