Skip to contentSkip to conversation

mm/userfaultfd: detect VMA replacement after copy retry in mfill_copy_folio_retry()

PATCH Superseded David Carlier 4 participants 8 messages Mar 30 – Mar 31

Subsystems (2)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/userfaultfd: detect VMA replacement after copy retry in mfill_copy_folio_retry()

In mfill_copy_folio_retry(), all locks are dropped to retry copy_from_user() with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap + UFFDIO_REGISTER by another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.

Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with -EAGAIN.

---
 mm/userfaultfd.c | 64 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 57 insertions(+), 7 deletions(-)
mm/userfaultfd.c +57 -7
index 481ec7eb4442..22e82f953eb5 100644
@@ -443,33 +443,83 @@ static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr)
 	return ret;
 }
 
+struct vma_snapshot {
+	struct inode *inode;
+	vm_flags_t flags;
+};
+
+static void vma_snapshot_take(struct vm_area_struct *vma,
+			      struct vma_snapshot *s)
+{
+	s->flags = vma->vm_flags;
+	if (vma->vm_file) {
+		s->inode = vma->vm_file->f_inode;
+		ihold(s->inode);
+	} else {
+		s->inode = NULL;
+	}
+}
+
+static bool vma_snapshot_changed(struct vm_area_struct *vma,
+				 struct vma_snapshot *s)
+{
+	if (s->flags != vma->vm_flags)
+		return true;
+
+	if (s->inode && vma->vm_file->f_inode != s->inode)
+		return true;
+
+	if (!s->inode && !vma_is_anonymous(vma))
+		return true;
+
+	return false;
+}
+
+static void vma_snapshot_release(struct vma_snapshot *s)
+{
+	if (s->inode) {
+		iput(s->inode);
+		s->inode = NULL;
+	}
+}
+
 static int mfill_copy_folio_retry(struct mfill_state *state, struct folio *folio)
 {
 	unsigned long src_addr = state->src_addr;
+	struct vma_snapshot s;
 	void *kaddr;
 	int err;
 
+	/* Take a quick snapshot of the current vma */
+	vma_snapshot_take(state->vma, &s);
+
 	/* retry copying with mm_lock dropped */
 	mfill_put_vma(state);
 
 	kaddr = kmap_local_folio(folio, 0);
 	err = copy_from_user(kaddr, (const void __user *) src_addr, PAGE_SIZE);
 	kunmap_local(kaddr);
-	if (unlikely(err))
-		return -EFAULT;
+	if (unlikely(err)) {
+		err = -EFAULT;
+		goto out;
+	}
 
 	flush_dcache_folio(folio);
 
 	/* reget VMA and PMD, they could change underneath us */
 	err = mfill_get_vma(state);
 	if (err)
-		return err;
+		goto out;
 
-	err = mfill_establish_pmd(state);
-	if (err)
-		return err;
+	if (vma_snapshot_changed(state->vma, &s)) {
+		err = -EAGAIN;
+		goto out;
+	}
 
-	return 0;
+	err = mfill_establish_pmd(state);
+out:
+	vma_snapshot_release(&s);
+	return err;
 }
 
 static int __mfill_atomic_pte(struct mfill_state *state,
Signature · 1 line
2.53.0
maintainer
#1 lore raw
Quoted 11 lines · On Mon, 30 Mar 2026 21:29:09 +0100 David Carlier <[email protected]> wrote:

In mfill_copy_folio_retry(), all locks are dropped to retry copy_from_user() with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap + UFFDIO_REGISTER by another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.

Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with -EAGAIN.

Thanks. What are the userspace-visible runtime effects of the bug?

If they're serious we might be looking at a cc:stable and a
Fixes: tag?

reviewer
#5 lore raw
In reply to David Carlier

mm/userfaultfd.c static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr) · line 448

Quoted 3 lines · David Carlier
mm/userfaultfd.c +3 -0
+struct vma_snapshot {
+	struct inode *inode;
+	vm_flags_t flags;

Note that I used vma_flags_t / memcpy() / memcmp(), explicitly because IIUC we're moving towards deprecating vm_flags_t.

Please see vma_flags_to_legacy().

Quoted 1 line

+};

Signature · 1 line
Peter Xu
#2 lore raw
In reply to Andrew Morton

Hi

Quoted 15 lines · Andrew Morton
Quoted 11 lines · On Mon, 30 Mar 2026 21:29:09 +0100 David Carlier <[email protected]> wrote:

In mfill_copy_folio_retry(), all locks are dropped to retry copy_from_user() with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap + UFFDIO_REGISTER by another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.

Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with -EAGAIN.

Quoted 5 lines

Thanks. What are the userspace-visible runtime effects of the bug?

If they're serious we might be looking at a cc:stable and a
Fixes: tag?

The bug manifests as a NULL pointer dereference in shmem_mfill_filemap_add() via file_inode(vma->vm_file) when vm_file is NULL (anonymous VMA). This is a kernel
oops/panic — so it's definitely serious enough for Cc: stable and Fixes:.

Cheers

#3 lore raw
In reply to Andrew Morton

To "mitigate" my previous answer after further digging ...

The userspace-visible effect is a kernel NULL pointer dereference. When
a shared shmem VMA gets replaced by an anonymous VMA during the
retry
window, the stale ops->filemap_add() ends up calling shmem_mfill_filemap_add() which dereferences vma->vm_file via file_inode(). Since vm_file is NULL for anonymous mappings, this is a straight kernel oops.

The window is particularly wide when copy_from_user() blocks on slow backing stores (FUSE, NFS) as it runs with page faults enabled.

The Fixes target would be 56a3706fd7f9 ("shmem, userfaultfd:
implement
shmem uffd operations using vm_uffd_ops") but that's mm-unstable only, so no Cc: stable for now.

Quoted 20 lines · Andrew Morton
Quoted 11 lines · On Mon, 30 Mar 2026 21:29:09 +0100 David Carlier <[email protected]> wrote:

In mfill_copy_folio_retry(), all locks are dropped to retry copy_from_user() with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap + UFFDIO_REGISTER by another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.

Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with -EAGAIN.

Thanks. What are the userspace-visible runtime effects of the bug?

If they're serious we might be looking at a cc:stable and a
Fixes: tag?

maintainer
#4 lore raw
Quoted 8 lines · On Mon, 30 Mar 2026 22:32:58 +0100 David CARLIER <[email protected]> wrote:

The userspace-visible effect is a kernel NULL pointer dereference. When
a shared shmem VMA gets replaced by an anonymous VMA during the
retry
window, the stale ops->filemap_add() ends up calling shmem_mfill_filemap_add() which dereferences vma->vm_file via file_inode(). Since vm_file is NULL for anonymous mappings, this is a straight kernel oops.

Quoted 7 lines

The window is particularly wide when copy_from_user() blocks on slow backing stores (FUSE, NFS) as it runs with page faults enabled.

The Fixes target would be 56a3706fd7f9 ("shmem, userfaultfd:
implement
shmem uffd operations using vm_uffd_ops") but that's mm-unstable only, so no Cc: stable for now.

Ah, OK, thanks. I'll add a note to "shmem, userfaultfd: implement shmem uffd operations using vm_uffd_ops" for now, let's see what Mike thinks.

maintainer
#6 lore raw
In reply to David Carlier

(added VMA folks)

Hi,

Quoted 5 lines · David Carlier

In mfill_copy_folio_retry(), all locks are dropped to retry copy_from_user() with page faults enabled. During this window, the VMA can be replaced entirely (e.g. munmap + mmap + UFFDIO_REGISTER by another thread), but the caller proceeds with a folio allocated from the original VMA's backing store.

Is it possible at all that after all that dance vma pointer will remain the same?

Quoted 8 lines

Checking ops alone is insufficient: the replacement VMA could be the same type (e.g. shmem -> shmem) with identical flags but a different backing inode. Take a snapshot of the VMA's inode and flags before dropping locks, and compare after re-acquiring them. If anything changed, bail out with -EAGAIN.

Sashiko has comments and they seem quite relevant to me:
https://sashiko.dev/#/patchset/20260330214948.148349-1-devnexen%40gmail.com

Quoted 3 lines
---
 mm/userfaultfd.c | 64 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 57 insertions(+), 7 deletions(-)

...

mm/userfaultfd.c static int mfill_copy_folio_locked(struct folio *folio, unsigned long src_addr) · line 515

Quoted 2 lines
mm/userfaultfd.c +2 -0
+	if (vma_snapshot_changed(state->vma, &s)) {
+		err = -EAGAIN;

Whatever we do verify the VMA this should not be EAGAIN. EINVAL or ENOENT like mfill_get_vma() returns seem more appropriate.

Quoted 2 lines
(quoted diff) +2 -0
+		goto out;
+	}
Signature · 2 lines
Sincerely yours,
Mike.
#7 lore raw

Hi Mike,

▎ Is it possible at all that after all that dance vma pointer will remain
▎ the same?

Yes, VMA structs are slab-allocated so after munmap frees the old
VMA
and mmap allocates a new one, SLUB can hand back the same address.
The
pointer matches but it's a different VMA — which is exactly why the snapshot is needed.

▎ This isn't a bug, but struct vm_area_struct uses vm_flags, not
flags.
▎ Will this cause a compilation error?

This is a false positive from Sashiko. vm_area_struct has a union at include/linux/mm_types.h:956-960:

  union {
      const vm_flags_t vm_flags;
      vma_flags_t flags;

};
Peter explicitly asked to use vma_flags_t / vma->flags since
vm_flags_t
is being deprecated (see vma_flags_to_legacy()).

▎ If the original VMA was file-backed (s->inode is non-NULL), but is ▎ concurrently replaced by an anonymous VMA during the lock-dropped ▎ window, vma->vm_file will be NULL. Does accessing
▎ vma->vm_file->f_inode here cause a NULL pointer dereference?

Good catch, this is a real bug. Will fix with a vm_file NULL guard.

▎ Filesystem eviction paths often acquire locks (like i_rwsem) that ▎ invert with the mmap lock. Can this cause an AB-BA deadlock? Should ▎ this take a reference to the struct file via get_file() and
release it
▎ with fput() instead, which defers destruction safely?

Valid concern. I'll switch to get_file()/fput() which defers the destruction safely.

▎ Whatever we do verify the VMA this should not be EAGAIN. EINVAL or ▎ ENOENT like mfill_get_vma() returns seem more appropriate.

Agreed, will change to -EINVAL to match mfill_get_vma()'s validation failures.

Will send a v2 with all three fixes later on.

Cheers !