Skip to contentSkip to conversation

mm-vma-only-permit-map_private-dev-zero-to-be-mapped-anonymous.patch removed from -mm tree

merged mm-stable Andrew Morton 1 participant 1 message Oct 9

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: mm/vma: only permit MAP_PRIVATE /dev/zero to be mapped anonymous
has been removed from the -mm tree. Its filename was
mm-vma-only-permit-map_private-dev-zero-to-be-mapped-anonymous.patch

This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: "Lorenzo Stoakes (ARM)" <[email protected]>
Subject: mm/vma: only permit MAP_PRIVATE /dev/zero to be mapped anonymous
Date: Sat, 26 Sep 2026 11:41:08 +0100

In order to use mmap_prepare() with MAP_PRIVATE mappings of /dev/zero without the success_hook hack we explicitly permitted mmap_prepare handlers to set NULL vm_ops.

However this is dangerous and we really only want to allow this for MAP_PRIVATE-mapped /dev/zero.

Therefore use the newly introduced file_is_dev_zero() to uniquely identify MAP_PRIVATE-/dev/zero mappings and only permit this behaviour for them.

Then, remove all ability for mmap_prepare or mmap hooks to set a VMA anonymous and update mmap_zero_prepare() to leave it to the core mmap code to do so.

Note that this disallows nested MAP_PRIVATE-mappings of /dev/zero regions. Doing this would be broken in any case.

We therefore do not need to update the mmap_prepare() compatibility layer to reflect these changes, as the mmap hook check suffices to disallow this behaviour.

Now we're setting vma->vm_ops to NULL for an mmap_prepare-initialised MAP_PRIVATE-/dev/zero mapping, we have to avoid a subtle issue when updating user-defined fields via set_vma_user_defined_fields().

The default for vma->vm_ops for all mmap_prepare-initialised mappings is vma_dummy_vm_ops, so map->vm_ops will be set to this and setting vma->vm_ops to this will render the VMA mistakenly non-anon.

In general, we should never be setting user-defined fields for an anonymous VMA, so explicitly check for this to avoid doing so for the one case where a mapping can be both mmap_prepare and anonymous.

In the case of legacy ->mmap hooks some drivers may set vma->vm_ops NULL believing this is the equivalent of setting no VMA operations. Therefore update mmap_file() to correct this by setting dummy VMA operations if this occurs.

An example of this is drm_gem_shmem_mmap() which deliberately clears vma->vm_ops before handing the VMA to dma-buf. Cases such as this will be updated when they are converted to mmap_prepare.

Also, in order to avoid a single commit bisection hazard, add a temporary workaround to set the VMA anonymous only after vma->vm_file is assigned in __mmap_new_file_vma().

This is because vma_set_range() calls vma_set_pgoff() and assert_sane_pgoff() in turn, prior to the vma->vm_file being assigned. If we set the VMA anonymous early then this assert will fail.

This is removed in the subsequent commit.

Cc Arnd Bergmann <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Liam R. Howlett <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Jann Horn <[email protected]>
Cc Pedro Falcato <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Baolin Wang <[email protected]>
Cc Matthew Wilcox (Oracle) <[email protected]>
Cc Jan Kara <[email protected]>

---

 mm/char-mem.c |    6 +-----
 mm/internal.h |   17 ++++++++++-------
 mm/vma.c      |   33 +++++++++++++++++++++++++--------
 3 files changed, 36 insertions(+), 20 deletions(-)
mm/char-mem.c renamed from mm/char-mem.c~mm-vma-only-permit-map_private-dev-zero-to-be-mapped-anonymous +1 -5
@@ -508,11 +508,7 @@ static int mmap_zero_prepare(struct vm_a
 	if (vma_desc_test(desc, VMA_MAYSHARE_BIT))
 		return shmem_zero_setup_desc(desc);
 
-	/*
-	 * This is a highly unique situation where we mark a MAP_PRIVATE mapping
-	 * of /dev/zero anonymous, despite it not being.
-	 */
-	vma_desc_set_anonymous(desc);
+	/* MAP_PRIVATE semantics are taken care of for us by core mm. */
 	return 0;
 }
 
mm/internal.h renamed from mm/internal.h~mm-vma-only-permit-map_private-dev-zero-to-be-mapped-anonymous +10 -7
@@ -226,15 +226,18 @@ static inline int mmap_file(struct file
 {
 	int err = vfs_mmap(file, vma);
 
-	if (likely(!err))
-		return 0;
-
 	/*
-	 * OK, we tried to call the file hook for mmap(), but an error
-	 * arose. The mapping is in an inconsistent state and we must not invoke
-	 * any further hooks on it.
+	 * Either we tried to call the file hook for mmap() and an error arose
+	 * or a driver set vma->vm_ops = NULL intending there to be no VMA
+	 * operations.
+	 *
+	 * In the former case the VMA is in an inconsistent state and we mustn't
+	 * invoke any further hooks on it, in the latter case the hook actually
+	 * wanted no further hooks to be invoked, so fix both by setting dummy
+	 * VMA ops.
 	 */
-	vma->vm_ops = &vma_dummy_vm_ops;
+	if (unlikely(err || !vma->vm_ops))
+		vma->vm_ops = &vma_dummy_vm_ops;
 
 	return err;
 }
mm/vma.c renamed from mm/vma.c~mm-vma-only-permit-map_private-dev-zero-to-be-mapped-anonymous +25 -8
@@ -2644,6 +2644,19 @@ static int __mmap_new_file_vma(struct mm
 	return 0;
 }
 
+static bool map_is_private(const struct mmap_state *map)
+{
+	return !vma_flags_test(&map->vma_flags, VMA_SHARED_BIT);
+}
+
+static bool map_is_anon(const struct mmap_state *map)
+{
+	if (!map_is_private(map))
+		return false;
+
+	return !map->file || file_is_dev_zero(map->file);
+}
+
 /*
  * __mmap_new_vma() - Allocate a new VMA for the region, as merging was not
  * possible.
@@ -2657,8 +2670,7 @@ static int __mmap_new_file_vma(struct mm
 static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,
 	struct mmap_action *action)
 {
-	const bool is_anon = !map->file &&
-		!vma_flags_test(&map->vma_flags, VMA_SHARED_BIT);
+	const bool is_anon = map_is_anon(map);
 	struct vma_iterator *vmi = map->vmi;
 	int error = 0;
 	struct vm_area_struct *vma;
@@ -2674,7 +2686,7 @@ static int __mmap_new_vma(struct mmap_st
 
 	vma_iter_config(vmi, map->addr, map->end);
 
-	if (is_anon)
+	if (is_anon && !map->file)
 		vma_set_anonymous(vma);
 
 	vma_set_range(vma, map->addr, map->end, map->pgoff, map->anon_pgoff);
@@ -2692,6 +2704,10 @@ static int __mmap_new_vma(struct mmap_st
 	else if (!is_anon)
 		error = shmem_zero_setup(vma);
 
+	/* Temporary MAP_PRIVATE-/dev/zero workaround. */
+	if (is_anon && map->file)
+		vma_set_anonymous(vma);
+
 	if (error)
 		goto free_iter_vma;
 
@@ -2800,6 +2816,10 @@ static int call_mmap_prepare(struct mmap
 	if (err)
 		return err;
 
+	/* It's invalid for mmap_prepare hooks to clear vm_ops. */
+	if (!desc->vm_ops)
+		return -EINVAL;
+
 	err = call_action_prepare(map, desc);
 	if (err)
 		return err;
@@ -2822,10 +2842,7 @@ static int call_mmap_prepare(struct mmap
 static void set_vma_user_defined_fields(struct vm_area_struct *vma,
 		struct mmap_state *map)
 {
-	if (map->vm_ops)
-		vma->vm_ops = map->vm_ops;
-	else	/* Only /dev/zero should do this. */
-		vma_set_anonymous(vma);
+	vma->vm_ops = map->vm_ops;
 	vma->vm_private_data = map->vm_private_data;
 }
 
@@ -2907,7 +2924,7 @@ static unsigned long __mmap_region(struc
 		allocated_new = true;
 	}
 
-	if (have_mmap_prepare && allocated_new)
+	if (have_mmap_prepare && !map_is_anon(&map) && allocated_new)
 		set_vma_user_defined_fields(vma, &map);
 
 	__mmap_complete(&map, vma);

_

Patches currently in -mm which might be from [email protected] are

mm-vma-fix-mmap_prepare-file-handling-remove-file_doesnt_need_get.patch mm-vma-introduce-and-use-vma_can_merge.patch mm-consistently-validate-vma-state-after-mmap-hooks.patch mm-vma-ensure-mmap_prepare-doesnt-set-actions-on-a-mergeable-vma.patch mm-make-map_kernel_pages_-internal-and-unexported.patch mm-vma-tidy-up-map-kernel-pages-enum-values.patch mm-add-mmap-action-for-discontiguous-kernel-page-mapping.patch docs-filesystems-update-mmap_prepare-docs-for-discontig-kernel-pgs.patch drivers-usb-mon-update-to-use-mmap_prepare-map-kernel-pages.patch infiniband-update-hfi1-to-use-remap_vmalloc_range.patch selinux-reject-writable-opens-of-policy-file-drop-mmap-shared-write-check.patch alsa-pcm-use-vm_insert_page-to-map-pcm-status-page.patch bpf-arena-mark-arena_map_mmap-mappings-vm_mixedmap.patch mm-vma-add-vma_is_mm_managed-predicates.patch mm-vma-only-allow-mmap-to-clear-vma_maywrite_bit-if-not-mm-managed.patch mm-vma-add-and-use-vma__is_fixed_mapping.patch scsi-sg-convert-mmap-hook-to-mmap_prepare-and-rework.patch fbdev-defio-assert-fbinfo_virtfb-drop-vm_io-add-vm_mixedmap.patch hsi-cmt_speech-convert-mmap-hook-to-mmap_prepare-refactor.patch mm-gup-error-out-early-on-vma_mayread_bit-vmas.patch uprobes-remove-vm_io-set-vm_mixedmap-for-mapped-kernel-pages.patch mm-mlock-clear-vma_locked_mask-over-mmap-callback.patch mm-mlock-eliminate-weird-vma_io_bit-abuse-and-simplify.patch mm-vma-enforce-that-mm-managed-mappings-may-not-set-vma_io_bit.patch mm-remove-vma_io_bit-check-in-vma_is_mm_managed.patch mm-remove-hugetlb_inlineh.patch mm-rename-is_vm_hugetlb_page-to-vma_is_hugetlb.patch mm-drop-some-redundant-checks-around-hugetlb-vmas.patch mm-madvise-update-is_valid_guard_vma-to-use-vma_can_merge.patch mm-vma-introduce-vma_is_mm_backed.patch mm-uffd-use-predicates-for-userfaultfd-checks.patch mm-madvise-use-predicates-for-madvise-madv_dofork.patch mm-eliminate-vma_special_flags-usage-when-hugetlb-explicitly-tested.patch mm-eliminate-vma_special_flags-check-in-lru_gen_look_around.patch mm-avoid-use-of-vma_special_flags-in-migrate_vma_setup.patch mm-eliminate-vm_special-vma_special_flags.patch fuse-dax-do-not-set-vm_mixedmap.patch mm-huge_memory-remove-vma_is_special_huge.patch mm-vma-const-ify-vma_assert_stabilised-and-associated-functions.patch mm-implement-and-use-vma_has_anon_rmap-silence-kcsan.patch mm-update-comments-to-refer-to-anon-rmap-rather-than-anon_vma.patch mm-vma-dont-remove-vma-from-rmap-if-pgoff-unchanged.patch