mm/userfaultfd: detect VMA replacement after copy retry in mfill_copy_folio_retry()
Subsystems (2)
PATCH v2 1/1 mm/userfaultfd: detect VMA replacement after copy retry in mfill_copy_folio_retry()
In mfill_copy_folio_retry(), all locks are dropped to retry copy_from_user() with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap + UFFDIO_REGISTER by another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.
Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with -EAGAIN.
--- mm/userfaultfd.c | 64 ++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 57 insertions(+), 7 deletions(-)
mm/userfaultfd.c +57 -7
@@ -443,33 +443,83 @@ static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr)return ret; } +struct vma_snapshot { +struct inode *inode; +vm_flags_t flags; +}; + +static void vma_snapshot_take (struct vm_area_struct *vma, +struct vma_snapshot *s) +{ + s->flags = vma->vm_flags; +if (vma->vm_file) { + s->inode = vma->vm_file->f_inode; +ihold (s->inode); + }else { + s->inode =NULL ; + } +} + +static bool vma_snapshot_changed (struct vm_area_struct *vma, +struct vma_snapshot *s) +{ +if (s->flags != vma->vm_flags) +return true; + +if (s->inode && vma->vm_file->f_inode != s->inode) +return true; + +if (!s->inode && !vma_is_anonymous (vma)) +return true; + +return false; +} + +static void vma_snapshot_release (struct vma_snapshot *s) +{ +if (s->inode) { +iput (s->inode); + s->inode =NULL ; + } +} +static int mfill_copy_folio_retry (struct mfill_state *state,struct folio *folio) {unsigned long src_addr = state->src_addr; +struct vma_snapshot s;void *kaddr;int err; +/* Take a quick snapshot of the current vma */ +vma_snapshot_take (state->vma, &s); +/* retry copying with mm_lock dropped */ mfill_put_vma (state); kaddr =kmap_local_folio (folio,0 ); err =copy_from_user (kaddr, (const void __user *) src_addr,PAGE_SIZE );kunmap_local (kaddr); -if (unlikely (err)) -return -EFAULT ; +if (unlikely (err)) { + err = -EFAULT ; + goto out; + }flush_dcache_folio (folio);/* reget VMA and PMD, they could change underneath us */ err =mfill_get_vma (state);if (err) -return err; + goto out; - err =mfill_establish_pmd (state); -if (err) -return err; +if (vma_snapshot_changed (state->vma, &s)) { + err = -EAGAIN ; + goto out; + } -return 0 ; + err =mfill_establish_pmd (state); +out: +vma_snapshot_release (&s); +return err; } static int __mfill_atomic_pte(struct mfill_state *state,
Signature · 1 line
2.53.0
Andrew Morton
maintainerQuoted 11 lines · On Mon, 30 Mar 2026 21:29:09 +0100 David Carlier <[email protected]> wrote:
In
mfill_copy_folio_retry(), all locks are dropped to retrycopy_from_user()with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap +UFFDIO_REGISTERby another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with
-EAGAIN.
Thanks. What are the userspace-visible runtime effects of the bug?
If they're serious we might be looking at a cc:stable and a
Fixes: tag?
Peter Xu
reviewermm/userfaultfd.c static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr) · line 448
Quoted 3 lines · David Carlier
mm/userfaultfd.c +3
-0+struct vma_snapshot { +struct inode *inode; +vm_flags_t flags;
Note that I used vma_flags_t / memcpy() / memcmp(), explicitly because IIUC we're moving towards deprecating vm_flags_t.
Please see vma_flags_to_legacy().
Quoted 1 line
+};
Signature · 1 line
Peter Xu
Hi
Quoted 15 lines · Andrew Morton
Quoted 11 lines · On Mon, 30 Mar 2026 21:29:09 +0100 David Carlier <[email protected]> wrote:
In
mfill_copy_folio_retry(), all locks are dropped to retrycopy_from_user()with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap +UFFDIO_REGISTERby another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with
-EAGAIN.
Quoted 5 lines
Thanks. What are the userspace-visible runtime effects of the bug?
If they're serious we might be looking at a cc:stable and a
Fixes: tag?
The bug manifests as a NULL pointer dereference in shmem_mfill_filemap_add() via file_inode(vma->vm_file) when vm_file is NULL (anonymous VMA). This is a kernel
oops/panic — so it's definitely serious enough for Cc: stable and Fixes:.
Cheers
To "mitigate" my previous answer after further digging ...
The userspace-visible effect is a kernel NULL pointer dereference. When
a shared shmem VMA gets replaced by an anonymous VMA during the
retry
window, the stale ops->filemap_add() ends up calling shmem_mfill_filemap_add() which dereferences vma->vm_file via file_inode(). Since vm_file is NULL for anonymous mappings, this is a straight kernel oops.
The window is particularly wide when copy_from_user() blocks on slow backing stores (FUSE, NFS) as it runs with page faults enabled.
The Fixes target would be 56a3706fd7f9 ("shmem, userfaultfd:
implement
shmem uffd operations using vm_uffd_ops") but that's mm-unstable only, so no Cc: stable for now.
Quoted 20 lines · Andrew Morton
Quoted 11 lines · On Mon, 30 Mar 2026 21:29:09 +0100 David Carlier <[email protected]> wrote:
In
mfill_copy_folio_retry(), all locks are dropped to retrycopy_from_user()with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap +UFFDIO_REGISTERby another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with
-EAGAIN.Thanks. What are the userspace-visible runtime effects of the bug?
If they're serious we might be looking at a cc:stable and a
Fixes: tag?
Andrew Morton
maintainerQuoted 8 lines · On Mon, 30 Mar 2026 22:32:58 +0100 David CARLIER <[email protected]> wrote:
The userspace-visible effect is a kernel NULL pointer dereference. When
a shared shmem VMA gets replaced by an anonymous VMA during the
retry
window, the staleops->filemap_add()ends up callingshmem_mfill_filemap_add()which dereferencesvma->vm_fileviafile_inode(). Sincevm_fileis NULL for anonymous mappings, this is a straight kernel oops.
Quoted 7 lines
The window is particularly wide when
copy_from_user()blocks on slow backing stores (FUSE, NFS) as it runs with page faults enabled.The Fixes target would be
56a3706fd7f9("shmem, userfaultfd:
implement
shmem uffd operations usingvm_uffd_ops") but that's mm-unstable only, so no Cc: stable for now.
Ah, OK, thanks. I'll add a note to "shmem, userfaultfd: implement shmem uffd operations using vm_uffd_ops" for now, let's see what Mike thinks.
Mike Rapoport
maintainer(added VMA folks)
Hi,
Quoted 5 lines · David Carlier
In
mfill_copy_folio_retry(), all locks are dropped to retrycopy_from_user()with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap +UFFDIO_REGISTERby another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.
Is it possible at all that after all that dance vma pointer will remain the same?
Quoted 8 lines
Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with
-EAGAIN.Suggested-by Peter Xu <[email protected]>Signed-off-by David Carlier <[email protected]>
Sashiko has comments and they seem quite relevant to me:
https://sashiko.dev/#/patchset/20260330214948.148349-1-devnexen%40gmail.com
Quoted 3 lines
--- mm/userfaultfd.c | 64 ++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 57 insertions(+), 7 deletions(-)
...
mm/userfaultfd.c static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr) · line 515
Quoted 2 lines
mm/userfaultfd.c +2
-0+if (vma_snapshot_changed (state->vma, &s)) { + err = -EAGAIN ;
Whatever we do verify the VMA this should not be EAGAIN. EINVAL or ENOENT like mfill_get_vma() returns seem more appropriate.
Quoted 2 lines
(quoted diff) +2
-0+ goto out; + }
Signature · 2 lines
Sincerely yours, Mike.
Hi Mike,
▎ Is it possible at all that after all that dance vma pointer will remain
▎ the same?
Yes, VMA structs are slab-allocated so after munmap frees the old
VMA
and mmap allocates a new one, SLUB can hand back the same address.
The
pointer matches but it's a different VMA — which is exactly why the snapshot is needed.
▎ This isn't a bug, but struct vm_area_struct uses vm_flags, not
flags.
▎ Will this cause a compilation error?
This is a false positive from Sashiko. vm_area_struct has a union at include/linux/mm_types.h:956-960:
union {
const vm_flags_t vm_flags;
vma_flags_t flags; };
Peter explicitly asked to use vma_flags_t / vma->flags sincevm_flags_t
is being deprecated (see vma_flags_to_legacy()).
▎ If the original VMA was file-backed (s->inode is non-NULL), but is ▎ concurrently replaced by an anonymous VMA during the lock-dropped ▎ window, vma->vm_file will be NULL. Does accessing
▎ vma->vm_file->f_inode here cause a NULL pointer dereference?
Good catch, this is a real bug. Will fix with a vm_file NULL guard.
▎ Filesystem eviction paths often acquire locks (like i_rwsem) that ▎ invert with the mmap lock. Can this cause an AB-BA deadlock? Should ▎ this take a reference to the struct file via get_file() and
release it
▎ with fput() instead, which defers destruction safely?
Valid concern. I'll switch to get_file()/fput() which defers the destruction safely.
▎ Whatever we do verify the VMA this should not be EAGAIN. EINVAL or ▎ ENOENT like mfill_get_vma() returns seem more appropriate.
Agreed, will change to -EINVAL to match mfill_get_vma()'s validation failures.
Will send a v2 with all three fixes later on.
Cheers !