mm/virtio: skip redundant zeroing of host-zeroed pages

The NUMA interleave index was computed as two separate terms:

    *ilx += vma->vm_pgoff >> order;
    *ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order);

This has two problems:

1. When vm_start is not aligned to the folio size, the subtraction before the shift lets low bits affect the result via borrows.

2. For file-backed VMAs, shifting vm_pgoff and the VMA offset independently loses carries between them, giving wrong chunk indices when vm_pgoff is not aligned to order.

Combine into a single expression that adds vm_pgoff and

the page-granularity VMA offset first, then shifts once:

    *ilx += (vma->vm_pgoff +
            (addr >> PAGE_SHIFT) -
            (vma->vm_start >> PAGE_SHIFT)) >> order;

For anonymous VMAs, vm_pgoff equals vm_start >> PAGE_SHIFT,

so the vm_pgoff and vm_start terms cancel and the result

reduces to addr >> (PAGE_SHIFT + order), same as before.

For file-backed VMAs, the sum vm_pgoff + (addr >> PAGE_SHIFT)
- (vm_start >> PAGE_SHIFT) gives the file page offset of addr.

Shifting by order gives the correct file chunk index.

Signed-off-by Michael S. Tsirkin <[email protected]>

Assisted-by Claude:claude-opus-4-6

Reviewed-by Gregory Price <[email protected]>

---
 mm/mempolicy.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

mm/mempolicy.c +0 -0

index 4e4421b22b59..d139b074a599 100644

mm/mempolicy.c +0 -1

 	}
 	return pol;
 }
--

MST

TestSetPageHWPoison() is called without zone->lock, so its atomic update to page->flags can race with non-atomic flag operations that run under zone->lock in the buddy allocator.

In particular, __free_pages_prepare() does:

    page->flags.f &= ~PAGE_FLAGS_CHECK_AT_PREP;

This non-atomic read-modify-write, while correctly excluding __PG_HWPOISON from the mask, can still lose a concurrent TestSetPageHWPoison if the read happens before the poison bit is set and the write happens after. Follow-up patches in this series add similar non-atomic flag operations as well.

Fix by acquiring zone->lock around TestSetPageHWPoison and around ClearPageHWPoison in the retry path. This serializes with all buddy flag manipulation. The cost is
negligible: one lock/unlock in an extremely rare path (hardware memory errors).

Note: SetPageHWPoison and TestClearPageHWPoison calls elsewhere in this file operate on pages already removed from the buddy allocator or on non-buddy pages (DAX, hugetlb), so they do not need zone->lock protection.

Acked-by Miaohe Lin <[email protected]>

Signed-off-by Michael S. Tsirkin <[email protected]>

Assisted-by Claude:claude-opus-4-6

---
 mm/memory-failure.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

mm/memory-failure.c +10 -1

index ee42d4361309..3880486028a1 100644
@@ -2348,6 +2348,8 @@ int memory_failure(unsigned long pfn, int flags)
 	unsigned long page_flags;
 	bool retry = true;
 	int hugetlb = 0;
+	struct zone *zone;
+	unsigned long mf_flags;
 
 	if (!sysctl_memory_failure_recovery)
 		panic("Memory failure on page %lx", pfn);

@@ -2390,7 +2392,11 @@ int memory_failure(unsigned long pfn, int flags)
 	if (hugetlb)
 		goto unlock_mutex;
 
+	/* Serialize with non-atomic buddy flag operations */
+	zone = page_zone(p);
+	spin_lock_irqsave(&zone->lock, mf_flags);
 	if (TestSetPageHWPoison(p)) {
+		spin_unlock_irqrestore(&zone->lock, mf_flags);
 		res = -EHWPOISON;
 		if (flags & MF_ACTION_REQUIRED)
 			res = kill_accessing_process(current, pfn, flags);

@@ -2399,6 +2405,7 @@ int memory_failure(unsigned long pfn, int flags)
 		action_result(pfn, MF_MSG_ALREADY_POISONED, MF_FAILED);
 		goto unlock_mutex;
 	}
+	spin_unlock_irqrestore(&zone->lock, mf_flags);
 
 	/*
 	 * We need/can do nothing about count=0 pages.

@@ -2420,7 +2427,10 @@ int memory_failure(unsigned long pfn, int flags)
 			} else {
 				/* We lost the race, try again */
 				if (retry) {
+					/* Serialize with non-atomic buddy flag operations */
+					spin_lock_irqsave(&zone->lock, mf_flags);
 					ClearPageHWPoison(p);
+					spin_unlock_irqrestore(&zone->lock, mf_flags);
 					retry = false;
 					goto try_again;
 				}
--

MST

Quoted 20 lines · Michael S. Tsirkin

TestSetPageHWPoison() is called without zone->lock, so its atomic update to page->flags can race with non-atomic flag operations that run under zone->lock in the buddy allocator.

In particular, __free_pages_prepare() does:

    page->flags.f &= ~PAGE_FLAGS_CHECK_AT_PREP;

This non-atomic read-modify-write, while correctly excluding __PG_HWPOISON from the mask, can still lose a concurrent TestSetPageHWPoison if the read happens before the poison bit is set and the write happens after. Follow-up patches in this series add similar non-atomic flag operations as well.

Fix by acquiring zone->lock around TestSetPageHWPoison and around ClearPageHWPoison in the retry path. This serializes with all buddy flag manipulation. The cost is
negligible: one lock/unlock in an extremely rare path (hardware memory errors).

Quoted 7 lines

Note: SetPageHWPoison and TestClearPageHWPoison calls elsewhere in this file operate on pages already removed from the buddy allocator or on non-buddy pages (DAX, hugetlb), so they do not need zone->lock protection.

Acked-by Miaohe Lin <[email protected]>

Signed-off-by Michael S. Tsirkin <[email protected]>

Can we have Fixes: and Cc: stable and also send this separately please?

These patches seem like unrelated fixups that you've discovered along the way, and don't belong as part of the already rather large series, unless I'm missing something here.

Thanks, Lorenzo

Quoted 52 lines

Assisted-by Claude:claude-opus-4-6

---
 mm/memory-failure.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

mm/memory-failure.c +10 -1

index ee42d4361309..3880486028a1 100644
@@ -2348,6 +2348,8 @@ int memory_failure(unsigned long pfn, int flags)
 	unsigned long page_flags;
 	bool retry = true;
 	int hugetlb = 0;
+	struct zone *zone;
+	unsigned long mf_flags;
 
 	if (!sysctl_memory_failure_recovery)
 		panic("Memory failure on page %lx", pfn);

@@ -2390,7 +2392,11 @@ int memory_failure(unsigned long pfn, int flags)
 	if (hugetlb)
 		goto unlock_mutex;
 
+	/* Serialize with non-atomic buddy flag operations */
+	zone = page_zone(p);
+	spin_lock_irqsave(&zone->lock, mf_flags);
 	if (TestSetPageHWPoison(p)) {
+		spin_unlock_irqrestore(&zone->lock, mf_flags);
 		res = -EHWPOISON;
 		if (flags & MF_ACTION_REQUIRED)
 			res = kill_accessing_process(current, pfn, flags);

@@ -2399,6 +2405,7 @@ int memory_failure(unsigned long pfn, int flags)
 		action_result(pfn, MF_MSG_ALREADY_POISONED, MF_FAILED);
 		goto unlock_mutex;
 	}
+	spin_unlock_irqrestore(&zone->lock, mf_flags);
 
 	/*
 	 * We need/can do nothing about count=0 pages.

@@ -2420,7 +2427,10 @@ int memory_failure(unsigned long pfn, int flags)
 			} else {
 				/* We lost the race, try again */
 				if (retry) {
+					/* Serialize with non-atomic buddy flag operations */
+					spin_lock_irqsave(&zone->lock, mf_flags);
 					ClearPageHWPoison(p);
+					spin_unlock_irqrestore(&zone->lock, mf_flags);
 					retry = false;
 					goto try_again;
 				}
--

MST

Quoted 31 lines · Lorenzo Stoakes

Quoted 20 lines · Michael S. Tsirkin

TestSetPageHWPoison() is called without zone->lock, so its atomic update to page->flags can race with non-atomic flag operations that run under zone->lock in the buddy allocator.

In particular, __free_pages_prepare() does:

    page->flags.f &= ~PAGE_FLAGS_CHECK_AT_PREP;

This non-atomic read-modify-write, while correctly excluding __PG_HWPOISON from the mask, can still lose a concurrent TestSetPageHWPoison if the read happens before the poison bit is set and the write happens after. Follow-up patches in this series add similar non-atomic flag operations as well.

Fix by acquiring zone->lock around TestSetPageHWPoison and around ClearPageHWPoison in the retry path. This serializes with all buddy flag manipulation. The cost is
negligible: one lock/unlock in an extremely rare path (hardware memory errors).

Quoted 7 lines

Note: SetPageHWPoison and TestClearPageHWPoison calls elsewhere in this file operate on pages already removed from the buddy allocator or on non-buddy pages (DAX, hugetlb), so they do not need zone->lock protection.

Acked-by Miaohe Lin <[email protected]>

Signed-off-by Michael S. Tsirkin <[email protected]>

Can we have Fixes: and Cc: stable and also send this separately please?

Quoted 5 lines

These patches seem like unrelated fixups that you've discovered along the way, and don't belong as part of the already rather large series, unless I'm missing something here.

Thanks, Lorenzo

I think you are mising that they are a dependency, not unrelated.

For example, this issue gets worse with the patchset as there are more places that manipulate flags without atomics. No?

You are welcome to send this to stable, but I think stable rules preclude theoretical bugfixes.

As for Fixes: the issue has been there for decades. I wouldn't know what to attribute it for.

I guess I could send these separately, too, why not. Not sure what this accomplishes, but hey. But is that an ack? You want this fix merged even before the feature?

Quoted 32 lines · Michael S. Tsirkin

Quoted 31 lines · Lorenzo Stoakes

Quoted 20 lines · Michael S. Tsirkin

TestSetPageHWPoison() is called without zone->lock, so its atomic update to page->flags can race with non-atomic flag operations that run under zone->lock in the buddy allocator.

In particular, __free_pages_prepare() does:

    page->flags.f &= ~PAGE_FLAGS_CHECK_AT_PREP;

This non-atomic read-modify-write, while correctly excluding __PG_HWPOISON from the mask, can still lose a concurrent TestSetPageHWPoison if the read happens before the poison bit is set and the write happens after. Follow-up patches in this series add similar non-atomic flag operations as well.

Fix by acquiring zone->lock around TestSetPageHWPoison and around ClearPageHWPoison in the retry path. This serializes with all buddy flag manipulation. The cost is
negligible: one lock/unlock in an extremely rare path (hardware memory errors).

Quoted 7 lines

Note: SetPageHWPoison and TestClearPageHWPoison calls elsewhere in this file operate on pages already removed from the buddy allocator or on non-buddy pages (DAX, hugetlb), so they do not need zone->lock protection.

Acked-by Miaohe Lin <[email protected]>

Signed-off-by Michael S. Tsirkin <[email protected]>

Can we have Fixes: and Cc: stable and also send this separately please?

Quoted 7 lines

Quoted 5 lines

These patches seem like unrelated fixups that you've discovered along the way, and don't belong as part of the already rather large series, unless I'm missing something here.

Thanks, Lorenzo

I think you are mising that they are a dependency, not unrelated.

Then say so.

Quoted 2 lines

For example, this issue gets worse with the patchset as there are more places that manipulate flags without atomics. No?

It's your job to make that case, not mine.

Quoted 4 lines

You are welcome to send this to stable, but I think stable rules preclude theoretical bugfixes.

It's a dependency but also theoretical?

Quoted 3 lines

As for Fixes: the issue has been there for decades. I wouldn't know what to attribute it for.

Again, your job.

Quoted 5 lines

I guess I could send these separately, too, why not. Not sure what this accomplishes, but hey. But is that an ack? You want this fix merged even before the feature?

I already made the case as to why, as have other maintainers.

If you need to review what an ack looks like please consult
https://docs.kernel.org/process/5.Posting.html

Thanks, Lorenzo

Quoted 5 lines

These patches seem like unrelated fixups that you've discovered along the way, and don't belong as part of the already rather large series, unless I'm missing something here.

Thanks, Lorenzo

I think you are mising that they are a dependency, not unrelated. For example, this issue gets worse with the patchset as there are more places that manipulate flags without atomics. No?

You are welcome to send this to stable, but I think stable rules preclude theoretical bugfixes.

When a reported free page is split via expand() to satisfy a smaller allocation, the sub-pages placed back on the free lists lose the PageReported flag. This means they will be unnecessarily re-reported to the hypervisor in the next reporting cycle, wasting work.

While I was unable to quantify the performance difference, it is an obvious waste, even if small.

Propagate the PageReported flag to sub-pages during expand(), both in page_del_and_expand() and try_to_claim_block(), so

split_large_buddy() also propagates PageReported via a bool
parameter: the caller saves PageReported before del_page_from_free_list() clears it, then passes the saved value. The flag is set after __free_one_page() with a PageBuddy check, matching the page_reporting_drain() pattern. Free-path callers pass false (freshly freed pages are never reported).
that they are recognized as already-reported.

Signed-off-by Michael S. Tsirkin <[email protected]>

Assisted-by Claude:claude-opus-4-6

---
 mm/page_alloc.c | 32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)

mm/page_alloc.c +2 -1

index d49c254174da..8dae5b3f5876 100644
@@ -1502,7 +1502,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 
 /* Split a multi-block free page into its individual pageblocks. */
 static void split_large_buddy(struct zone *zone, struct page *page,
-			      unsigned long pfn, int order, fpi_t fpi)
+			      unsigned long pfn, int order, fpi_t fpi,
+			      bool reported)
 {
 	unsigned long end = pfn + (1 << order);
 

mm/page_alloc.c +2 -2

 			__count_vm_events(PGFREE, 1 << p_order);
 		}
 	}
-	split_large_buddy(zone, page, pfn, order, fpi_flags);
+	split_large_buddy(zone, page, pfn, order, fpi_flags, false);
 	spin_unlock_irqrestore(&zone->lock, flags);
 
 	__count_vm_events(PGFREE, 1 << order);

@@ -1694,7 +1698,7 @@ struct page *__pageblock_pfn_to_page(unsigned long start_pfn,
  * -- nyc
  */
 static inline unsigned int expand(struct zone *zone, struct page *page, int low,
-				  int high, int migratetype)
+				  int high, int migratetype, bool reported)
 {
 	unsigned int size = 1 << high;
 	unsigned int nr_added = 0;

mm/page_alloc.c +2 -0

+		if (reported)
+			__SetPageReported(&page[size]);
 	}
 
 	return nr_added;

mm/page_alloc.c +6 -3

 	account_freepages(zone, -nr_pages, migratetype);
 }
 

@@ -2116,11 +2130,13 @@ static bool __move_freepages_block_isolate(struct zone *zone,
 	/* We're a part of a larger buddy */
 	if (PageBuddy(buddy) && buddy_order(buddy) > pageblock_order) {
 		int order = buddy_order(buddy);
+		bool reported = PageReported(buddy);
 
 		del_page_from_free_list(buddy, zone, order,
 					get_pfnblock_migratetype(buddy, buddy_pfn));
 		toggle_pageblock_isolate(page, isolate);
-		split_large_buddy(zone, buddy, buddy_pfn, order, FPI_NONE);
+		split_large_buddy(zone, buddy, buddy_pfn, order, FPI_NONE,
+				  reported);
 		return true;
 	}
 

@@ -2283,10 +2299,12 @@ try_to_claim_block(struct zone *zone, struct page *page,
 	/* Take ownership for orders >= pageblock_order */
 	if (current_order >= pageblock_order) {
 		unsigned int nr_added;
+		bool was_reported = page_reported(page);
 
 		del_page_from_free_list(page, zone, current_order, block_type);
 		change_pageblock_range(page, current_order, start_type);
-		nr_added = expand(zone, page, order, current_order, start_type);
+		nr_added = expand(zone, page, order, current_order, start_type,
+				  was_reported);
 		account_freepages(zone, nr_added, start_type);
 		return page;
 	}
--

MST

mm/page_alloc.c static void free_one_page(struct zone *zone, struct page *page, · line 1566

Quoted 3 lines

mm/page_alloc.c +2 -1

-			split_large_buddy(zone, p, page_to_pfn(p), p_order, fpi_flags);
+			split_large_buddy(zone, p, page_to_pfn(p), p_order,
+					  fpi_flags, false);

I don't love adding a mystery meat boolean parameter like this.

Remove the !CONFIG_HUGETLB_PAGE stub for alloc_hugetlb_folio().

The stub is dead code: all callers are in mm/hugetlb.c (CONFIG_HUGETLB_PAGE) or fs/hugetlbfs/inode.c (CONFIG_HUGETLBFS),

and CONFIG_HUGETLB_PAGE is def_bool HUGETLBFS with nothing selecting it independently.

The stub is also broken: it returns NULL, but all callers check IS_ERR(folio), so a NULL return would not be caught and would crash on the subsequent folio dereference.

Remove it now since follow-up patches change the signature of alloc_hugetlb_folio and would otherwise need to update the broken stub too.

Reviewed-by Gregory Price <[email protected]>

Assisted-by Claude:claude-opus-4-6

Reviewed-by Dev Jain <[email protected]>

Signed-off-by Michael S. Tsirkin <[email protected]>

---
 include/linux/hugetlb.h | 7 -------
 1 file changed, 7 deletions(-)

include/linux/hugetlb.h +0 -8

index 5957bc25efa8..1f7ae6609e51 100644
@@ -1123,13 +1123,6 @@ static inline void wait_for_freed_hugetlb_folios(void)
 {
 }
 
-static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
-					   unsigned long addr,
-					   bool cow_from_owner)
-{
-	return NULL;
-}
-
 static inline struct folio *
 alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
 			    nodemask_t *nmask, gfp_t gfp_mask)
--

MST

Move vma_alloc_folio_noprof() from an inline in gfp.h (for !NUMA) and mempolicy.c (for NUMA) to page_alloc.c.

This prepares for a subsequent patch that will thread user_addr through the allocator: having vma_alloc_folio_noprof in page_alloc.c means user_addr can be passed to the internal allocation path without changing public API signatures or duplicating plumbing in both gfp.h and mempolicy.c.

The !NUMA path gains the VM_DROPPABLE -> __GFP_NOWARN check that the NUMA path already had.

Signed-off-by Michael S. Tsirkin <[email protected]>

Assisted-by Claude:claude-opus-4-6

Assisted-by cursor-agent:GPT-5.4-xhigh

---
 include/linux/gfp.h |  9 ++-------
 mm/mempolicy.c      | 32 --------------------------------
 mm/page_alloc.c     | 43 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+), 39 deletions(-)

include/linux/gfp.h +2 -7

index 51ef13ed756e..7ccbda35b9ad 100644
@@ -318,13 +318,13 @@ static inline struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask,
 
 #define  alloc_pages_node(...)			alloc_hooks(alloc_pages_node_noprof(__VA_ARGS__))
 
+struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order,
+		struct vm_area_struct *vma, unsigned long addr);
 #ifdef CONFIG_NUMA
 struct page *alloc_pages_noprof(gfp_t gfp, unsigned int order);
 struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order);
 struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
 		struct mempolicy *mpol, pgoff_t ilx, int nid);
-struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma,
-		unsigned long addr);
 #else
 static inline struct page *alloc_pages_noprof(gfp_t gfp_mask, unsigned int order)
 {

@@ -339,11 +339,6 @@ static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int orde
 {
 	return folio_alloc_noprof(gfp, order);
 }
-static inline struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order,
-		struct vm_area_struct *vma, unsigned long addr)
-{
-	return folio_alloc_noprof(gfp, order);
-}
 #endif
 
 #define alloc_pages(...)			alloc_hooks(alloc_pages_noprof(__VA_ARGS__))

mm/mempolicy.c +0 -32

index d139b074a599..a1707ad498a8 100644
@@ -2516,38 +2516,6 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
 	return page_rmappable_folio(page);
 }
 
-/**
- * vma_alloc_folio - Allocate a folio for a VMA.
- * @gfp: GFP flags.
- * @order: Order of the folio.
- * @vma: Pointer to VMA.
- * @addr: Virtual address of the allocation.  Must be inside @vma.
- *
- * Allocate a folio for a specific address in @vma, using the appropriate
- * NUMA policy.  The caller must hold the mmap_lock of the mm_struct of the
- * VMA to prevent it from going away.  Should be used for all allocations
- * for folios that will be mapped into user space, excepting hugetlbfs, and
- * excepting where direct use of folio_alloc_mpol() is more appropriate.
- *
- * Return: The folio on success or NULL if allocation fails.
- */
-struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma,
-		unsigned long addr)
-{
-	struct mempolicy *pol;
-	pgoff_t ilx;
-	struct folio *folio;
-
-	if (vma->vm_flags & VM_DROPPABLE)
-		gfp |= __GFP_NOWARN;
-
-	pol = get_vma_policy(vma, addr, order, &ilx);
-	folio = folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id());
-	mpol_cond_put(pol);
-	return folio;
-}
-EXPORT_SYMBOL(vma_alloc_folio_noprof);
-
 struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order)
 {
 	struct mempolicy *pol = &default_policy;

mm/page_alloc.c +43 -1

index 8dae5b3f5876..6a605d05e8cd 100644
@@ -5286,6 +5286,49 @@ struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_
 }
 EXPORT_SYMBOL(__folio_alloc_noprof);
 
+#ifdef CONFIG_NUMA
+/**
+ * vma_alloc_folio - Allocate a folio for a VMA.
+ * @gfp: GFP flags.
+ * @order: Order of the folio.
+ * @vma: Pointer to VMA.
+ * @addr: Virtual address of the allocation.  Must be inside @vma.
+ *
+ * Allocate a folio for a specific address in @vma, using the appropriate
+ * NUMA policy.  The caller must hold the mmap_lock of the mm_struct of the
+ * VMA to prevent it from going away.  Should be used for all allocations
+ * for folios that will be mapped into user space, excepting hugetlbfs, and
+ * excepting where direct use of folio_alloc_mpol() is more appropriate.
+ *
+ * Return: The folio on success or NULL if allocation fails.
+ */
+struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order,
+		struct vm_area_struct *vma, unsigned long addr)
+{
+	struct mempolicy *pol;
+	pgoff_t ilx;
+	struct folio *folio;
+
+	if (vma->vm_flags & VM_DROPPABLE)
+		gfp |= __GFP_NOWARN;
+
+	pol = get_vma_policy(vma, addr, order, &ilx);
+	folio = folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id());
+	mpol_cond_put(pol);
+	return folio;
+}
+#else
+struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order,
+		struct vm_area_struct *vma, unsigned long addr)
+{
+	if (vma->vm_flags & VM_DROPPABLE)
+		gfp |= __GFP_NOWARN;
+
+	return folio_alloc_noprof(gfp, order);
+}
+#endif
+EXPORT_SYMBOL(vma_alloc_folio_noprof);
+
 /*
  * Common helper functions. Never use with __GFP_HIGHMEM because the returned
  * address cannot represent highmem pages. Use alloc_pages and then kmap if
--

MST

Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages.

The user_addr is stored in struct alloc_context and flows through:
vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook

USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping.

Signed-off-by Michael S. Tsirkin <[email protected]>

Assisted-by Claude:claude-opus-4-6

Assisted-by cursor-agent:GPT-5.4-xhigh

---
 include/linux/gfp.h |  2 +-
 mm/compaction.c     |  5 ++---
 mm/hugetlb.c        | 36 ++++++++++++++++++++----------------
 mm/internal.h       | 21 ++++++++++++++++++---
 mm/mempolicy.c      | 44 ++++++++++++++++++++++++++++++++------------
 mm/mmap.c           |  6 ++++++
 mm/page_alloc.c     | 44 +++++++++++++++++++++++++++++---------------
 mm/slub.c           |  4 ++--
 8 files changed, 110 insertions(+), 52 deletions(-)

include/linux/gfp.h +1 -1

index 7ccbda35b9ad..ee35c5367abc 100644
@@ -337,7 +337,7 @@ static inline struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order)
 static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
 		struct mempolicy *mpol, pgoff_t ilx, int nid)
 {
-	return folio_alloc_noprof(gfp, order);
+	return __folio_alloc_noprof(gfp, order, numa_node_id(), NULL);
 }
 #endif
 

mm/compaction.c +0 -0

index 3648ce22c807..72684fe81e83 100644

mm/compaction.c +1 -2

 	set_page_refcounted(page);
 	return page;
 }

@@ -1849,8 +1849,7 @@ static struct folio *compaction_alloc_noprof(struct folio *src, unsigned long da
 		set_page_private(&freepage[size], start_order);
 	}
 	dst = (struct folio *)freepage;
-
-	post_alloc_hook(&dst->page, order, __GFP_MOVABLE);
+	post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE);
 	set_page_refcounted(&dst->page);
 	if (order)
 		prep_compound_page(&dst->page, order);

mm/hugetlb.c +0 -0

index 4b80b167cc9c..f3bc15a7889a 100644

mm/hugetlb.c +7 -6

 {
 	struct folio *folio;
 	bool alloc_try_hard = true;

@@ -1803,7 +1804,7 @@ static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask,
 	if (alloc_try_hard)
 		gfp_mask |= __GFP_RETRY_MAYFAIL;
 
-	folio = (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmask);
+	folio = (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmask, addr);
 
 	/*
 	 * If we did not specify __GFP_RETRY_MAYFAIL, but still got a

@@ -1832,7 +1833,7 @@ static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask,
 
 static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h,
 		gfp_t gfp_mask, int nid, nodemask_t *nmask,
-		nodemask_t *node_alloc_noretry)
+		nodemask_t *node_alloc_noretry, unsigned long addr)
 {
 	struct folio *folio;
 	int order = huge_page_order(h);

@@ -1844,7 +1845,7 @@ static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h,
 		folio = alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask);
 	else
 		folio = alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask,
-						 node_alloc_noretry);
+						 node_alloc_noretry, addr);
 	if (folio)
 		init_new_hugetlb_folio(folio);
 	return folio;

@@ -1858,11 +1859,12 @@ static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h,
  * pages is zero, and the accounting must be done in the caller.
  */
 static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h,
-		gfp_t gfp_mask, int nid, nodemask_t *nmask)
+		gfp_t gfp_mask, int nid, nodemask_t *nmask,
+		unsigned long addr)
 {
 	struct folio *folio;
 
-	folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL);
+	folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL, addr);
 	if (folio)
 		hugetlb_vmemmap_optimize_folio(h, folio);
 	return folio;

@@ -1902,7 +1904,7 @@ static struct folio *alloc_pool_huge_folio(struct hstate *h,
 		struct folio *folio;
 
 		folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, node,
-					nodes_allowed, node_alloc_noretry);
+					nodes_allowed, node_alloc_noretry, USER_ADDR_NONE);
 		if (folio)
 			return folio;
 	}

mm/hugetlb.c +9 -8

 {
 	struct folio *folio = NULL;
 

@@ -2083,7 +2086,7 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h,
 		goto out_unlock;
 	spin_unlock_irq(&hugetlb_lock);
 
-	folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask);
+	folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, addr);
 	if (!folio)
 		return NULL;
 

@@ -2126,7 +2129,7 @@ static struct folio *alloc_migrate_hugetlb_folio(struct hstate *h, gfp_t gfp_mas
 	if (hstate_is_gigantic(h))
 		return NULL;
 
-	folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask);
+	folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, USER_ADDR_NONE);
 	if (!folio)
 		return NULL;
 

@@ -2162,14 +2165,14 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h,
 	if (mpol_is_preferred_many(mpol)) {
 		gfp_t gfp = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
 
-		folio = alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask);
+		folio = alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask, addr);
 
 		/* Fallback to all nodes if page==NULL */
 		nodemask = NULL;
 	}
 
 	if (!folio)
-		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask);
+		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask, addr);
 	mpol_cond_put(mpol);
 	return folio;
 }

@@ -2276,7 +2279,8 @@ static int gather_surplus_pages(struct hstate *h, long delta)
 		 * down the road to pick the current node if that is the case.
 		 */
 		folio = alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h),
-						    NUMA_NO_NODE, &alloc_nodemask);
+						    NUMA_NO_NODE, &alloc_nodemask,
+						    USER_ADDR_NONE);
 		if (!folio) {
 			alloc_ok = false;
 			break;

@@ -2682,7 +2686,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct folio *old_folio,
 			spin_unlock_irq(&hugetlb_lock);
 			gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE;
 			new_folio = alloc_fresh_hugetlb_folio(h, gfp_mask,
-							      nid, NULL);
+							      nid, NULL, USER_ADDR_NONE);
 			if (!new_folio)
 				return -ENOMEM;
 			goto retry;

@@ -3380,13 +3384,13 @@ static void __init hugetlb_hstate_alloc_pages_onenode(struct hstate *h, int nid)
 			gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE;
 
 			folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid,
-					&node_states[N_MEMORY], NULL);
+					&node_states[N_MEMORY], NULL, USER_ADDR_NONE);
 			if (!folio && !list_empty(&folio_list) &&
 			    hugetlb_vmemmap_optimizable_size(h)) {
 				prep_and_add_allocated_folios(h, &folio_list);
 				INIT_LIST_HEAD(&folio_list);
 				folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid,
-						&node_states[N_MEMORY], NULL);
+						&node_states[N_MEMORY], NULL, USER_ADDR_NONE);
 			}
 			if (!folio)
 				break;

mm/internal.h +18 -3

index 5a2ddcf68e0b..9d2198114510 100644
@@ -662,6 +662,16 @@ void calculate_min_free_kbytes(void);
 int __meminit init_per_zone_wmark_min(void);
 void page_alloc_sysctl_init(void);
 
+/*
+ * Sentinel for user_addr: indicates a non-user allocation.
+ * Cannot use 0 because address 0 is a valid userspace mapping.
+ * (unsigned long)-1 is safe because:
+ * 1. vm_end = addr + len <= TASK_SIZE, and vm_end is exclusive,
+ *    so -1 is never inside any VMA.
+ * 2. It will only be compared to page-aligned addresses.
+ */
+#define USER_ADDR_NONE	((unsigned long)-1)
+
 /*
  * Structure for holding the mostly immutable allocation parameters passed
  * between functions involved in allocations, including the alloc_pages*

@@ -693,6 +703,7 @@ struct alloc_context {
 	 */
 	enum zone_type highest_zoneidx;
 	bool spread_dirty_pages;
+	unsigned long user_addr;
 };
 
 /*

@@ -916,13 +927,14 @@ static inline void init_compound_tail(struct page *tail,
 	prep_compound_tail(tail, head, order);
 }
 
-void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags);
+void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags,
+		     unsigned long user_addr);
 extern bool free_pages_prepare(struct page *page, unsigned int order);
 
 extern int user_min_free_kbytes;
 
 struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int nid,
-		nodemask_t *);
+		nodemask_t *, unsigned long user_addr);
 #define __alloc_frozen_pages(...) \
 	alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__))
 void free_frozen_pages(struct page *page, unsigned int order);

@@ -930,10 +942,13 @@ void free_unref_folios(struct folio_batch *fbatch);
 
 #ifdef CONFIG_NUMA
 struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order);
+struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order,
+		struct mempolicy *pol, pgoff_t ilx, int nid,
+		unsigned long user_addr);
 #else
 static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order)
 {
-	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL);
+	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL, USER_ADDR_NONE);
 }
 #endif
 

mm/mempolicy.c +6 -3

index a1707ad498a8..f573ff32e94d 100644
@@ -2413,7 +2413,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk,
 }
 
 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
-						int nid, nodemask_t *nodemask)
+						int nid, nodemask_t *nodemask,
+						unsigned long user_addr)
 {
 	struct page *page;
 	gfp_t preferred_gfp;

@@ -2426,25 +2427,29 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
 	 */
 	preferred_gfp = gfp | __GFP_NOWARN;
 	preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
-	page = __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask);
+	page = __alloc_frozen_pages_noprof(preferred_gfp, order, nid,
+					   nodemask, user_addr);
 	if (!page)
-		page = __alloc_frozen_pages_noprof(gfp, order, nid, NULL);
+		page = __alloc_frozen_pages_noprof(gfp, order, nid, NULL,
+						   user_addr);
 
 	return page;
 }

mm/mempolicy.c +24 -8

  *
  * Return: The page on success or NULL if allocation fails.
  */
-static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
-		struct mempolicy *pol, pgoff_t ilx, int nid)
+static struct page *__alloc_pages_mpol(gfp_t gfp, unsigned int order,
+		struct mempolicy *pol, pgoff_t ilx, int nid,
+		unsigned long user_addr)
 {
 	nodemask_t *nodemask;
 	struct page *page;

@@ -2452,7 +2457,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 	nodemask = policy_nodemask(gfp, pol, ilx, &nid);
 
 	if (pol->mode == MPOL_PREFERRED_MANY)
-		return alloc_pages_preferred_many(gfp, order, nid, nodemask);
+		return alloc_pages_preferred_many(gfp, order, nid, nodemask,
+						 user_addr);
 
 	if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
 	    /* filter "hugepage" allocation, unless from alloc_pages() */

@@ -2476,7 +2482,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 			 */
 			page = __alloc_frozen_pages_noprof(
 				gfp | __GFP_THISNODE | __GFP_NORETRY, order,
-				nid, NULL);
+				nid, NULL, user_addr);
 			if (page || !(gfp & __GFP_DIRECT_RECLAIM))
 				return page;
 			/*

@@ -2488,7 +2494,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 		}
 	}
 
-	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask);
+	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, user_addr);
 
 	if (unlikely(pol->mode == MPOL_INTERLEAVE ||
 		     pol->mode == MPOL_WEIGHTED_INTERLEAVE) && page) {

@@ -2504,11 +2510,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 	return page;
 }
 
-struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
+static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 		struct mempolicy *pol, pgoff_t ilx, int nid)
 {
-	struct page *page = alloc_pages_mpol(gfp | __GFP_COMP, order, pol,
-			ilx, nid);
+	return __alloc_pages_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE);
+}
+
+struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order,
+		struct mempolicy *pol, pgoff_t ilx, int nid,
+		unsigned long user_addr)
+{
+	struct page *page = __alloc_pages_mpol(gfp | __GFP_COMP, order, pol,
+			ilx, nid, user_addr);
 	if (!page)
 		return NULL;
 

@@ -2516,6 +2529,13 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
 	return page_rmappable_folio(page);
 }
 
+struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
+		struct mempolicy *pol, pgoff_t ilx, int nid)
+{
+	return folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, nid,
+					    USER_ADDR_NONE);
+}
+
 struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order)
 {
 	struct mempolicy *pol = &default_policy;

mm/mmap.c +0 -0

index 5754d1c36462..73413cebc418 100644

mm/mmap.c +0 -0

 	if (addr > TASK_SIZE - len)
 		return -ENOMEM;
 	if (offset_in_page(addr))

mm/page_alloc.c +20 -11

index 6a605d05e8cd..21b52c879751 100644
@@ -1815,7 +1815,7 @@ static inline bool should_skip_init(gfp_t flags)
 }
 
 inline void post_alloc_hook(struct page *page, unsigned int order,
-				gfp_t gfp_flags)
+				gfp_t gfp_flags, unsigned long user_addr)
 {
 	const bool zero_tags = gfp_flags & __GFP_ZEROTAGS;
 	bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags) &&

@@ -1870,9 +1870,10 @@ inline void post_alloc_hook(struct page *page, unsigned int order,
 }
 
 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp_flags,
-							unsigned int alloc_flags)
+							unsigned int alloc_flags,
+							unsigned long user_addr)
 {
-	post_alloc_hook(page, order, gfp_flags);
+	post_alloc_hook(page, order, gfp_flags, user_addr);
 
 	if (order && (gfp_flags & __GFP_COMP))
 		prep_compound_page(page, order);

@@ -3956,7 +3957,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 		page = rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order,
 				gfp_mask, alloc_flags, ac->migratetype);
 		if (page) {
-			prep_new_page(page, order, gfp_mask, alloc_flags);
+			prep_new_page(page, order, gfp_mask, alloc_flags,
+				      ac->user_addr);
 
 			return page;
 		} else {

@@ -4184,7 +4186,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
 
 	/* Prep a captured page if available */
 	if (page)
-		prep_new_page(page, order, gfp_mask, alloc_flags);
+		prep_new_page(page, order, gfp_mask, alloc_flags,
+			      ac->user_addr);
 
 	/* Try get a page from the freelist if available */
 	if (!page)

@@ -5061,7 +5064,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	struct zoneref *z;
 	struct per_cpu_pages *pcp;
 	struct list_head *pcp_list;
-	struct alloc_context ac;
+	struct alloc_context ac = { .user_addr = USER_ADDR_NONE };
 	gfp_t alloc_gfp;
 	unsigned int alloc_flags = ALLOC_WMARK_LOW;
 	int nr_populated = 0, nr_account = 0;

@@ -5176,7 +5179,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 		}
 		nr_account++;
 
-		prep_new_page(page, 0, gfp, 0);
+		prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE);
 		set_page_refcounted(page);
 		page_array[nr_populated++] = page;
 	}

@@ -5201,12 +5204,13 @@ EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof);
  * This is the 'heart' of the zoned buddy allocator.
  */
 struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
-		int preferred_nid, nodemask_t *nodemask)
+		int preferred_nid, nodemask_t *nodemask,
+		unsigned long user_addr)
 {
 	struct page *page;
 	unsigned int alloc_flags = ALLOC_WMARK_LOW;
 	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
-	struct alloc_context ac = { };
+	struct alloc_context ac = { .user_addr = user_addr };
 
 	/*
 	 * There are several places where we assume that the order value is sane

@@ -5267,10 +5271,12 @@ EXPORT_SYMBOL(__alloc_frozen_pages_noprof);
 
 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order,
 		int preferred_nid, nodemask_t *nodemask)
+
 {
 	struct page *page;
 
-	page = __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask);
+	page = __alloc_frozen_pages_noprof(gfp, order, preferred_nid,
+					   nodemask, USER_ADDR_NONE);
 	if (page)
 		set_page_refcounted(page);
 	return page;

@@ -5313,7 +5319,8 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order,
 		gfp |= __GFP_NOWARN;
 
 	pol = get_vma_policy(vma, addr, order, &ilx);
-	folio = folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id());
+	folio = folio_alloc_mpol_user_noprof(gfp, order, pol, ilx,
+					     numa_node_id(), addr);
 	mpol_cond_put(pol);
 	return folio;
 }

@@ -5321,10 +5328,17 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order,
 struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order,
 		struct vm_area_struct *vma, unsigned long addr)
 {
+	struct page *page;
+

mm/page_alloc.c +3 -3

 }
 #endif
 EXPORT_SYMBOL(vma_alloc_folio_noprof);

@@ -6905,7 +6919,7 @@ static void split_free_frozen_pages(struct list_head *list, gfp_t gfp_mask)
 		list_for_each_entry_safe(page, next, &list[order], lru) {
 			int i;
 
-			post_alloc_hook(page, order, gfp_mask);
+			post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE);
 			if (!order)
 				continue;
 

@@ -7111,7 +7125,7 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 		struct page *head = pfn_to_page(start);
 
 		check_new_pages(head, order);
-		prep_new_page(head, order, gfp_mask, 0);
+		prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE);
 	} else {
 		ret = -EINVAL;
 		WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n",

@@ -7776,7 +7790,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned
 	gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_COMP
 			| gfp_flags;
 	unsigned int alloc_flags = ALLOC_TRYLOCK;
-	struct alloc_context ac = { };
+	struct alloc_context ac = { .user_addr = USER_ADDR_NONE };
 	struct page *page;
 
 	VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT);

mm/slub.c +2 -3

index a2bf3756ca7d..f397fa2f3f80 100644
@@ -3275,7 +3275,7 @@ static inline struct slab *alloc_slab_page(gfp_t flags, int node,
 	else if (node == NUMA_NO_NODE)
 		page = alloc_frozen_pages(flags, order);
 	else
-		page = __alloc_frozen_pages(flags, order, node, NULL);
+		page = __alloc_frozen_pages(flags, order, node, NULL, USER_ADDR_NONE);
 
 	if (!page)
 		return NULL;

@@ -5236,7 +5236,7 @@ static void *___kmalloc_large_node(size_t size, gfp_t flags, int node)
 	if (node == NUMA_NO_NODE)
 		page = alloc_frozen_pages_noprof(flags, order);
 	else
-		page = __alloc_frozen_pages_noprof(flags, order, node, NULL);
+		page = __alloc_frozen_pages_noprof(flags, order, node, NULL, USER_ADDR_NONE);
 
 	if (page) {
 		ptr = page_address(page);
--

MST

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Quoted 18 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

Quoted 3 lines

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

Quoted 23 lines · Matthew Wilcox

Quoted 21 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Quoted 3 lines

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

I've not looked at the callers closely enough to see the delta on that, but if it avoids this mess then also worth looking at yes...

If that means that we would handle __GFP_ZERO consistently in the callers of alloc_frozen_pages(), that would also do I guess. We'd still have to pass the user address down to some degree, through folio interfaces only at least.

Quoted 3 lines · David Hildenbrand (Arm)

If that means that we would handle __GFP_ZERO consistently in the callers of alloc_frozen_pages(), that would also do I guess. We'd still have to pass the user address down to some degree, through folio interfaces only at least.

What I don't understand is how the kernel page allocator needs to know the user address in order to effectively zero it, but the hypervisor is able to zero the page without knowing the user address. It feels like somebody has x86-centric thinking where cache colouring doesn't matter.

Quoted 5 lines · Matthew Wilcox

Quoted 3 lines · David Hildenbrand (Arm)

If that means that we would handle __GFP_ZERO consistently in the callers of alloc_frozen_pages(), that would also do I guess. We'd still have to pass the user address down to some degree, through folio interfaces only at least.

Quoted 4 lines

What I don't understand is how the kernel page allocator needs to know the user address in order to effectively zero it, but the hypervisor is able to zero the page without knowing the user address. It feels like somebody has x86-centric thinking where cache colouring doesn't matter.

(not commenting on the icache dache mess we have to drag along)

The thing is that with free-page-reporting the memory is already zeroed by the hypervisor as part of discarding that memory previously (e.g., MADV_DONTNEED) and allocating fresh pages on re-access.

So it's not a question of "why is the hypervisor zeroing less efficiently", as zeroing is just a side-product of reclaiming that memory in the first place.

Quoted 7 lines · David Hildenbrand (Arm)

Quoted 5 lines · Matthew Wilcox

What I don't understand is how the kernel page allocator needs to know the user address in order to effectively zero it, but the hypervisor is able to zero the page without knowing the user address. It feels like somebody has x86-centric thinking where cache colouring doesn't matter.

Quoted 1 line

(not commenting on the icache dache mess we have to drag along)

Well, that was kind of the point of this email ... I did ask the question you're answering in a different email so let me respond to that too.

Quoted 7 lines

The thing is that with free-page-reporting the memory is already zeroed by the hypervisor as part of discarding that memory previously (e.g., MADV_DONTNEED) and allocating fresh pages on re-access.

So it's not a question of "why is the hypervisor zeroing less efficiently", as zeroing is just a side-product of reclaiming that memory in the first place.

We definitely have users who don't want the guest to trust the hypervisor. So how do they disable this optimisation?

Quoted 8 lines · Matthew Wilcox

Quoted 7 lines · David Hildenbrand (Arm)

Quoted 5 lines · Matthew Wilcox

What I don't understand is how the kernel page allocator needs to know the user address in order to effectively zero it, but the hypervisor is able to zero the page without knowing the user address. It feels like somebody has x86-centric thinking where cache colouring doesn't matter.

Quoted 5 lines

Quoted 1 line

(not commenting on the icache dache mess we have to drag along)

Well, that was kind of the point of this email ... I did ask the question you're answering in a different email so let me respond to that too.

Now I'm confused :)

Quoted 8 lines

Quoted 7 lines

The thing is that with free-page-reporting the memory is already zeroed by the hypervisor as part of discarding that memory previously (e.g., MADV_DONTNEED) and allocating fresh pages on re-access.

So it's not a question of "why is the hypervisor zeroing less efficiently", as zeroing is just a side-product of reclaiming that memory in the first place.

Quoted 2 lines

We definitely have users who don't want the guest to trust the hypervisor. So how do they disable this optimisation?

Right, I don't think we currently have a toggle to disable free page reporting. So IIUC, this optimization would similarly automatically get enabled if the hypervisor advertises it.

Signature · 3 lines

Cheers,

David

Quoted 6 lines · David Hildenbrand (Arm)

Quoted 5 lines · Matthew Wilcox

Quoted 3 lines · David Hildenbrand (Arm)

If that means that we would handle __GFP_ZERO consistently in the callers of alloc_frozen_pages(), that would also do I guess. We'd still have to pass the user address down to some degree, through folio interfaces only at least.

Quoted 6 lines

Quoted 4 lines

What I don't understand is how the kernel page allocator needs to know the user address in order to effectively zero it, but the hypervisor is able to zero the page without knowing the user address. It feels like somebody has x86-centric thinking where cache colouring doesn't matter.

(not commenting on the icache dache mess we have to drag along)

Well, that was kind of the point of this email ... I did ask the question you're answering in a different email so let me respond to that too.

Quoted 4 lines

The thing is that with free-page-reporting the memory is already zeroed by the hypervisor as part of discarding that memory previously (e.g., MADV_DONTNEED) and allocating fresh pages on re-access.

Quoted 2 lines

So it's not a question of "why is the hypervisor zeroing less efficiently", as zeroing is just a side-product of reclaiming that memory in the first place.

We definitely have users who don't want the guest to trust the hypervisor. So how do they disable this optimisation?

Quoted 18 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

Quoted 3 lines

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

Quoted 23 lines · Matthew Wilcox

Quoted 21 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Quoted 3 lines

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

This sort of just implies writing the "alloc_frozen_zeroed_pages()" wrapper that does the zeroing at the end before return, and then killing the post hook nonsense associated with it in the first place.

None of this resolves the user address annoyance which is needed on some archs for cache flushing. Whether anyone agrees that the page allocator should be responsible for this particular operation - open debate.

Quoted 24 lines · Gregory Price

Quoted 23 lines · Matthew Wilcox

Quoted 21 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Quoted 7 lines

Quoted 3 lines

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

This sort of just implies writing the "alloc_frozen_zeroed_pages()" wrapper that does the zeroing at the end before return, and then killing the post hook nonsense associated with it in the first place.

This means it is going to be a multi-step optimization. This is probably step 1.

Quoted 4 lines

None of this resolves the user address annoyance which is needed on some archs for cache flushing. Whether anyone agrees that the page allocator should be responsible for this particular operation - open debate.

This is probably step 2. But does the virtio use case apply to these archs? Does the performance matter for them? If not, maybe this part can be left as a TODO.

Best Regards,
Yan, Zi

Quoted 47 lines · Zi Yan

Quoted 24 lines · Gregory Price

Quoted 23 lines · Matthew Wilcox

Quoted 21 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Quoted 7 lines

Quoted 3 lines

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

This sort of just implies writing the "alloc_frozen_zeroed_pages()" wrapper that does the zeroing at the end before return, and then killing the post hook nonsense associated with it in the first place.

This means it is going to be a multi-step optimization. This is probably step 1.

Quoted 4 lines

None of this resolves the user address annoyance which is needed on some archs for cache flushing. Whether anyone agrees that the page allocator should be responsible for this particular operation - open debate.

This is probably step 2. But does the virtio use case apply to these archs? Does the performance matter for them? If not, maybe this part can be left as a TODO.

Quoted 2 lines

Best Regards,
Yan, Zi

I doubt it. But I don't get what's proposed, the code that we have to modify is arch independent?

Quoted 51 lines · Michael S. Tsirkin

Quoted 49 lines · Zi Yan

Quoted 24 lines · Gregory Price

Quoted 23 lines · Matthew Wilcox

Quoted 21 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Quoted 7 lines

Quoted 3 lines

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

This sort of just implies writing the "alloc_frozen_zeroed_pages()" wrapper that does the zeroing at the end before return, and then killing the post hook nonsense associated with it in the first place.

This means it is going to be a multi-step optimization. This is probably step 1.

Quoted 4 lines

None of this resolves the user address annoyance which is needed on some archs for cache flushing. Whether anyone agrees that the page allocator should be responsible for this particular operation - open debate.

This is probably step 2. But does the virtio use case apply to these archs? Does the performance matter for them? If not, maybe this part can be left as a TODO.

Best Regards,
Yan, Zi

Quoted 2 lines

I doubt it. But I don't get what's proposed, the code that we have to modify is arch independent?

Change user_alloc_needs_zeroing() to only check address aliasing even if that can cause double zeroing for virtio.

Best Regards,
Yan, Zi

Quoted 59 lines · Zi Yan

Quoted 48 lines · Michael S. Tsirkin

Quoted 47 lines · Zi Yan

Quoted 24 lines · Gregory Price

Quoted 23 lines · Matthew Wilcox

Quoted 21 lines · Lorenzo Stoakes

But instead of overloading user_addr to indicate all kinds of things, instead make life easier by actually breaking things out.

Like:

enum alloc_context_type {
	KERNEL_ALLOCATION,
	USER_MAPPED_ALLOCATION,
	USER_UNMAPPED_ALLOCATION, // Maybe? Do we ever?
	/* Perhaps some other states we want to encode? */
};

struct alloc_context {
	...

	enum alloc_context_type type;
	unsigned long user_addr; // Only set if type == USER_ALLOCATION

	// Maybe something suggesting context or whether we init before in some
	// cases?
};

Quoted 7 lines

Quoted 3 lines

Ugh, please, no. As I suggested last time I commented on this trainwreck of a series, lift the zeroing functionality from alloc_frozen_pages() into its callers.

This sort of just implies writing the "alloc_frozen_zeroed_pages()" wrapper that does the zeroing at the end before return, and then killing the post hook nonsense associated with it in the first place.

This means it is going to be a multi-step optimization. This is probably step 1.

Quoted 4 lines

None of this resolves the user address annoyance which is needed on some archs for cache flushing. Whether anyone agrees that the page allocator should be responsible for this particular operation - open debate.

This is probably step 2. But does the virtio use case apply to these archs? Does the performance matter for them? If not, maybe this part can be left as a TODO.

Quoted 5 lines

Quoted 2 lines

Best Regards,
Yan, Zi

I doubt it. But I don't get what's proposed, the code that we have to modify is arch independent?

Change user_alloc_needs_zeroing() to only check address aliasing even if that can cause double zeroing for virtio.

Quoted 2 lines

Best Regards,
Yan, Zi

Ah. I started with exactly that in v1/v2. It's a simple approach.

But mm maintainers said no, user_alloc_needs_zeroing is a hack and I must not add to it.

Quoted 3 lines · Michael S. Tsirkin

Quoted 2 lines · Zi Yan

Quoted 7 lines

Quoted 5 lines

Change user_alloc_needs_zeroing() to only check address aliasing even if that can cause double zeroing for virtio.

Best Regards,
Yan, Zi

Ah. I started with exactly that in v1/v2. It's a simple approach.

Quoted 3 lines

But mm maintainers said no, user_alloc_needs_zeroing is a hack and I must not add to it.

Got it. It sounds that you now get conflicting ideas. Maybe you should start a [DISCUSSION] thread that presents the high level idea of what you want to achieve and all the ideas you got from the reviews, so that people in this thread can have the big picture and come up a consensus before you send another version.

Thank you for patiently replying my comments, since those points apparently have been discussed in prior submissions.

On 6/8/26 23:16, Zi Yan wrote:

There was Willy's comment in RFC v3 [1], which had 19 patches. Unfortunately, he no longer followed up to my initial push back and Michael's question later [2].

That would have probably been the right time to wait for more discussion.

RFC v4 had 22 patches with little replies.
v5 had 28 patches with little replies.
v6 had 30 patches with no replies.
v7 had 31 patches with little replies.
v8 had 37 patches with no replies.

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/lkml/[email protected]/

Quoted 13 lines · David Hildenbrand (Arm)

On 6/8/26 23:16, Zi Yan wrote:

There was Willy's comment in RFC v3 [1], which had 19 patches. Unfortunately, he no longer followed up to my initial push back and Michael's question later [2].

That would have probably been the right time to wait for more discussion.

RFC v4 had 22 patches with little replies.
v5 had 28 patches with little replies.
v6 had 30 patches with no replies.
v7 had 31 patches with little replies.
v8 had 37 patches with no replies.

Quoted 3 lines

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/lkml/[email protected]/

Hm, rewinding on this back to v3 here:

https://lore.kernel.org/lkml/[email protected]/

You said:

  ```
  Exactly, that's why I am saying that vma_alloc_folio() is the only
  external interface people should be using with a user address.
  ```

Going through the list of folio_zero_user references:

Called unconditionally if a folio is acquired:

   fs/hugetlbfs/inode.c:   folio_zero_user(folio, addr);
   mm/hugetlb.c:           folio_zero_user(folio, vmf->real_address);
   mm/memfd.c:             folio_zero_user(folio, 0);

Called when user_alloc_needs_zeroing() and charging passes:

   mm/huge_memory.c:       folio_zero_user(folio, addr);
   mm/memory.c:            folio_zero_user(folio, vmf->address);

No one outside mm/ should know about this interface at all.

Arguably none of these should know about this interface either.

The appropriate place for this logic appears to be:
vma_alloc_folio
alloc_hugetlb_folio alloc_hugetlb_folio_reserve

The reason to sink it into the post_alloc_hook is to let the buddy decide whether the page actually needs to be zeroed (like the virtio situation) based on PG_zeroed or whatever.

It seems like at a minimum moving the logic all the way into post_alloc_hook lets us actually delete folio_zero_user() as a published interface and move it entirely within page_alloc.c.

The catch is user_alloc_needs_zeroing() coming along with it.

I noticed this patch, again, sneaks in some additional code changes that are not mentioned in the commit message and seem irrelevant to the patch.

Not sure if the AI is doing this, but please don't do that.

Quoted 4 lines · Michael S. Tsirkin

Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages.

This feels like very weak justification for code that massively changes mm code and makes it all much worse.

Quoted 5 lines

The user_addr is stored in struct alloc_context and flows through:
vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook

Is this the only relevant code path? You're changing a TON of code here that will have multiple different code paths?

Quoted 1 line

Quoted 2 lines

USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping.

Ugh god, so now we're passing a user address through allocation paths that might not even be aware of this or be allocating memory at a point when the mapping is known?

Quoted 24 lines · Lorenzo Stoakes

I noticed this patch, again, sneaks in some additional code changes that are not mentioned in the commit message and seem irrelevant to the patch.

Not sure if the AI is doing this, but please don't do that.

Quoted 4 lines · Michael S. Tsirkin

Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages.

This feels like very weak justification for code that massively changes mm code and makes it all much worse.

Quoted 5 lines

The user_addr is stored in struct alloc_context and flows through:
vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook

Is this the only relevant code path? You're changing a TON of code here that will have multiple different code paths?

Quoted 1 line

Quoted 6 lines

Quoted 2 lines

USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping.

Ugh god, so now we're passing a user address through allocation paths that might not even be aware of this or be allocating memory at a point when the mapping is known?

The original ideas was to do this only with internal interfaces. I think I raised before to leave hugetlb alone for now.

Fundamentally, user_alloc_needs_zeroing() is something we should get rid of, to just be able use __GFP_ZERO and do zeroing at exactly one place.

The question is how to pass that information (user_addr) through internal APIs.

I previously said, that ideally we'd end up with a folio allocation interface in mm/page_alloc.c, from where we could get this done more cleanly internally.

But I don't want what the previous proposals did: use GFP flags+leak state or return magic "zeroed" flags.

Quoted 22 lines · David Hildenbrand (Arm)

Quoted 10 lines · Lorenzo Stoakes

I noticed this patch, again, sneaks in some additional code changes that are not mentioned in the commit message and seem irrelevant to the patch.

Not sure if the AI is doing this, but please don't do that.

This feels like very weak justification for code that massively changes mm code and makes it all much worse.

Quoted 7 lines

Is this the only relevant code path? You're changing a TON of code here that will have multiple different code paths?

Ugh god, so now we're passing a user address through allocation paths that might not even be aware of this or be allocating memory at a point when the mapping is known?

The original ideas was to do this only with internal interfaces. I think I raised before to leave hugetlb alone for now.

Quoted 2 lines

Fundamentally, user_alloc_needs_zeroing() is something we should get rid of, to just be able use __GFP_ZERO and do zeroing at exactly one place.

Just a reminder that user_alloc_needs_zeroing() not only checks init_on_alloc, but also some arc clearing page requirements. To do zeroing in one place, clear_page() used in post_alloc_hook() will need to learn how to handle arch-specific zeroing from clear_user_page()/clear_user_highpage().

Quoted 4 lines

The question is how to pass that information (user_addr) through internal APIs.

Or should we defer zeroing after a page is returned from allocator? So that user_addr does not need to be passed through irrelevant allocation APIs. Something like:

alloc_page_wrapper(gfp, order, user_addr)

{
	page = alloc_pages();
	if (gfp & __GFP_ZERO)
		clear_page(page);
}

Quoted 6 lines · Zi Yan

On 8 Jun 2026, at 7:08, David Hildenbrand (Arm) wrote:

Or should we defer zeroing after a page is returned from allocator? So that user_addr does not need to be passed through irrelevant allocation APIs. Something like:

Quoted 7 lines

alloc_page_wrapper(gfp, order, user_addr)

{
	page = alloc_pages();
	if (gfp & __GFP_ZERO)
		clear_page(page);
}

Not really sure what's best here. I think we'd want to limit the lifting to some internal API, so it cannot easily be messed up by random kernel code calling into the wrong API and not getting pages cleared.

Quoted 15 lines · David Hildenbrand (Arm)

Quoted 6 lines · Zi Yan

On 8 Jun 2026, at 7:08, David Hildenbrand (Arm) wrote:

Or should we defer zeroing after a page is returned from allocator? So that user_addr does not need to be passed through irrelevant allocation APIs. Something like:

Quoted 7 lines

alloc_page_wrapper(gfp, order, user_addr)

{
	page = alloc_pages();
	if (gfp & __GFP_ZERO)
		clear_page(page);
}

Quoted 4 lines

Not really sure what's best here. I think we'd want to limit the lifting to some internal API, so it cannot easily be messed up by random kernel code calling into the wrong API and not getting pages cleared.

We're a bit in circles on this. We discussed explicit interfaces a few months back and the trade off was:

a) add user_addr to the existing API and cause churn

or

b) add special interface like above
increase the buddy surface
leaves open the ability for users to get it wrong easily

If we forget VMs for a moment and break this step out separately, the core question is whether page_alloc.c is the right place to be calling the folio_user_zero() or whatever it is.

We seem to have agreed "yes", which necessitates the plumbing of the address into the allocator. The question is whether it should churn the existing interface for have its own explicit interface.

The implications of getting it wrong are: a user page doesn't get zeroed. *oof*

I think that's why we thought the churn was better. But considering we're already in the same state now (callers are responsible for calling folio_user_zero() or whatever), maybe that's not horrid?

Quoted 28 lines · Gregory Price

Quoted 15 lines · David Hildenbrand (Arm)

Quoted 6 lines · Zi Yan

On 8 Jun 2026, at 7:08, David Hildenbrand (Arm) wrote:

Or should we defer zeroing after a page is returned from allocator? So that user_addr does not need to be passed through irrelevant allocation APIs. Something like:

Quoted 7 lines

alloc_page_wrapper(gfp, order, user_addr)

{
	page = alloc_pages();
	if (gfp & __GFP_ZERO)
		clear_page(page);
}

Quoted 4 lines

Not really sure what's best here. I think we'd want to limit the lifting to some internal API, so it cannot easily be messed up by random kernel code calling into the wrong API and not getting pages cleared.

We're a bit in circles on this. We discussed explicit interfaces a few months back and the trade off was:

a) add user_addr to the existing API and cause churn

or

Quoted 7 lines

b) add special interface like above
increase the buddy surface
leaves open the ability for users to get it wrong easily

If we forget VMs for a moment and break this step out separately, the core question is whether page_alloc.c is the right place to be calling the folio_user_zero() or whatever it is.

page_alloc.c calling folio_user_zero() is fine, but my question is whether we should do the zeroing inside post_alloc_hook(), part of allocation.

What I propose is to lift __GFP_ZERO up as much as possible, so that most of allocation code does not need to care about it. We do the zeroing right before the page is returned to callers.

Quoted 38 lines · Zi Yan

Quoted 28 lines · Gregory Price

Quoted 15 lines · David Hildenbrand (Arm)

Quoted 6 lines · Zi Yan

On 8 Jun 2026, at 7:08, David Hildenbrand (Arm) wrote:

Or should we defer zeroing after a page is returned from allocator? So that user_addr does not need to be passed through irrelevant allocation APIs. Something like:

Quoted 7 lines

alloc_page_wrapper(gfp, order, user_addr)

{
	page = alloc_pages();
	if (gfp & __GFP_ZERO)
		clear_page(page);
}

Quoted 4 lines

Not really sure what's best here. I think we'd want to limit the lifting to some internal API, so it cannot easily be messed up by random kernel code calling into the wrong API and not getting pages cleared.

We're a bit in circles on this. We discussed explicit interfaces a few months back and the trade off was:

a) add user_addr to the existing API and cause churn

or

Quoted 7 lines

b) add special interface like above
increase the buddy surface
leaves open the ability for users to get it wrong easily

If we forget VMs for a moment and break this step out separately, the core question is whether page_alloc.c is the right place to be calling the folio_user_zero() or whatever it is.

Quoted 8 lines

page_alloc.c calling folio_user_zero() is fine, but my question is whether we should do the zeroing inside post_alloc_hook(), part of allocation.

What I propose is to lift __GFP_ZERO up as much as possible, so that most of allocation code does not need to care about it. We do the zeroing right before the page is returned to callers.

essentially we end up with something like

alloc_frozen_...(..., gfp)
{
folio = whatever(..., gfp);
if (gfp & __GFP_ZERO)
folio_zero(folio, -1); /* don't do cache flush part */
}

alloc_frozen_user_...(..., gfp, user_addr)
{
folio = whatever(..., gfp);
if (gfp & __GFP_ZERO)
folio_zero(folio, user_addr); /* do cache flush part */
}

The downside of this is obvious: it's easy for developers to get this wrong and call the non-user interface for user-bound allocations and miss the cache flush (that is only needed on some archs).

Not saying that's a deal breaker, but it's something to chew on.

Quoted 17 lines · Gregory Price

essentially we end up with something like

alloc_frozen_...(..., gfp)
{
folio = whatever(..., gfp);
if (gfp & __GFP_ZERO)
folio_zero(folio, -1); /* don't do cache flush part */
}

alloc_frozen_user_...(..., gfp, user_addr)
{
folio = whatever(..., gfp);
if (gfp & __GFP_ZERO)
folio_zero(folio, user_addr); /* do cache flush part */
}

Quoted 5 lines

The downside of this is obvious: it's easy for developers to get this wrong and call the non-user interface for user-bound allocations and miss the cache flush (that is only needed on some archs).

Not saying that's a deal breaker, but it's something to chew on.

I agree that misuse can cause trouble. But if we do the churn approach, what prevents developer from doing alloc_frozen(..., user_addr = -1) and using the returned page for userspace? It is possible the allocated page can be exported to userspace later.

BTW, that cache flush thing is fragile even today, you probably can do alloc_page() + vm_insert() to get a page without doing proper flush and export it to userspace. There seems to be no mechanism to prevent that.

Quoted 68 lines · Gregory Price

Quoted 38 lines · Zi Yan

Quoted 28 lines · Gregory Price

Quoted 15 lines · David Hildenbrand (Arm)

Quoted 6 lines · Zi Yan

On 8 Jun 2026, at 7:08, David Hildenbrand (Arm) wrote:

Or should we defer zeroing after a page is returned from allocator? So that user_addr does not need to be passed through irrelevant allocation APIs. Something like:

Quoted 7 lines

alloc_page_wrapper(gfp, order, user_addr)

{
	page = alloc_pages();
	if (gfp & __GFP_ZERO)
		clear_page(page);
}

Quoted 4 lines

Not really sure what's best here. I think we'd want to limit the lifting to some internal API, so it cannot easily be messed up by random kernel code calling into the wrong API and not getting pages cleared.

We're a bit in circles on this. We discussed explicit interfaces a few months back and the trade off was:

a) add user_addr to the existing API and cause churn

or

Quoted 7 lines

b) add special interface like above
increase the buddy surface
leaves open the ability for users to get it wrong easily

If we forget VMs for a moment and break this step out separately, the core question is whether page_alloc.c is the right place to be calling the folio_user_zero() or whatever it is.

Quoted 8 lines

page_alloc.c calling folio_user_zero() is fine, but my question is whether we should do the zeroing inside post_alloc_hook(), part of allocation.

What I propose is to lift __GFP_ZERO up as much as possible, so that most of allocation code does not need to care about it. We do the zeroing right before the page is returned to callers.

essentially we end up with something like

alloc_frozen_...(..., gfp)
{
folio = whatever(..., gfp);
if (gfp & __GFP_ZERO)
folio_zero(folio, -1); /* don't do cache flush part */
}

alloc_frozen_user_...(..., gfp, user_addr)
{
folio = whatever(..., gfp);
if (gfp & __GFP_ZERO)
folio_zero(folio, user_addr); /* do cache flush part */
}

The downside of this is obvious: it's easy for developers to get this wrong and call the non-user interface for user-bound allocations and miss the cache flush (that is only needed on some archs).

Quoted 1 line

Not saying that's a deal breaker, but it's something to chew on.

I agree that misuse can cause trouble. But if we do the churn approach, what prevents developer from doing alloc_frozen(..., user_addr = -1) and using the returned page for userspace? It is possible the allocated page can be exported to userspace later.

BTW, that cache flush thing is fragile even today,

you probably can
do alloc_page() + vm_insert() to get a page without doing proper flush and export it to userspace. There seems to be no mechanism to prevent that.

Best Regards,
Yan, Zi

Add a _user variant of alloc_contig_frozen_pages that accepts a user_addr parameter for cache-friendly zeroing of contiguous allocations.

No functional change; all existing callers continue to pass USER_ADDR_NONE.

Note for reviewers: non-compound contiguous allocations are

zeroed via kernel_init_pages, same as before this patch. There is no fault address because these allocations are not from the page fault path. For compound allocations, user_addr reaches post_alloc_hook() which calls folio_zero_user() with the dcache flush on cache-aliasing architectures.

Note about Sashiko (sashiko.dev) false positives: sashiko flags two issues here: (1) user_addr silently ignored for non-compound allocations, and (2) post_alloc_hook ignores user_addr. Both are false positives: (1) non-compound contiguous allocations have no fault address to pass, and
(2) post_alloc_hook does use user_addr when it is not USER_ADDR_NONE.

Signed-off-by Michael S. Tsirkin <[email protected]>

Assisted-by Claude:claude-opus-4-6

---
 include/linux/gfp.h |  6 ++++++
 mm/page_alloc.c     | 42 ++++++++++++++++++++++++++++++++----------
 2 files changed, 38 insertions(+), 10 deletions(-)

include/linux/gfp.h +6 -0

index ee35c5367abc..73109d4e31a4 100644
@@ -453,6 +453,12 @@ struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages,
 #define alloc_contig_frozen_pages(...) \
 	alloc_hooks(alloc_contig_frozen_pages_noprof(__VA_ARGS__))
 
+struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages,
+		gfp_t gfp_mask, int nid, nodemask_t *nodemask,
+		unsigned long user_addr);
+#define alloc_contig_frozen_pages_user(...) \
+	alloc_hooks(alloc_contig_frozen_pages_user_noprof(__VA_ARGS__))
+
 struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_t gfp_mask,
 		int nid, nodemask_t *nodemask);
 #define alloc_contig_pages(...)	\

mm/page_alloc.c +0 -0

index 21b52c879751..6d3f284c607d 100644
@@ -6975,13 +6975,15 @@ static void __free_contig_frozen_range(unsigned long pfn, unsigned long nr_pages
 }
 
 /**

mm/page_alloc.c +21 -9

  *
  * The PFN range does not have to be pageblock aligned. The PFN range must
  * belong to a single zone.

@@ -6997,8 +6999,9 @@ static void __free_contig_frozen_range(unsigned long pfn, unsigned long nr_pages
  *
  * Return: zero on success or negative error code.
  */
-int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
-		acr_flags_t alloc_flags, gfp_t gfp_mask)
+static int __alloc_contig_frozen_range(unsigned long start, unsigned long end,
+		acr_flags_t alloc_flags, gfp_t gfp_mask,
+		unsigned long user_addr)
 {
 	const unsigned int order = ilog2(end - start);
 	unsigned long outer_start, outer_end;

@@ -7125,7 +7128,7 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 		struct page *head = pfn_to_page(start);
 
 		check_new_pages(head, order);
-		prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE);
+		prep_new_page(head, order, gfp_mask, 0, user_addr);
 	} else {
 		ret = -EINVAL;
 		WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n",

@@ -7135,6 +7138,13 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
 	undo_isolate_page_range(start, end);
 	return ret;
 }
+
+int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
+		acr_flags_t alloc_flags, gfp_t gfp_mask)
+{
+	return __alloc_contig_frozen_range(start, end, alloc_flags, gfp_mask,
+					   USER_ADDR_NONE);
+}
 EXPORT_SYMBOL(alloc_contig_frozen_range_noprof);
 
 /**

@@ -7227,14 +7237,16 @@ static bool zone_spans_last_pfn(const struct zone *zone,
 	return zone_spans_pfn(zone, last_pfn);
 }
 
-/**
- * alloc_contig_frozen_pages() -- tries to find and allocate contiguous range of frozen pages
+/*
+ * alloc_contig_frozen_pages_user_noprof() -- allocate contiguous frozen pages with user address
  * @nr_pages:	Number of contiguous pages to allocate
  * @gfp_mask:	GFP mask. Node/zone/placement hints limit the search; only some
  *		action and reclaim modifiers are supported. Reclaim modifiers
  *		control allocation behavior during compaction/migration/reclaim.
  * @nid:	Target node
  * @nodemask:	Mask for other possible nodes
+ * @user_addr:	user virtual address for cache-friendly zeroing, or
+ *		USER_ADDR_NONE for kernel allocations.
  *
  * This routine is a wrapper around alloc_contig_frozen_range(). It scans over
  * zones on an applicable zonelist to find a contiguous pfn range which can then

@@ -7253,8 +7265,9 @@ static bool zone_spans_last_pfn(const struct zone *zone,
  *
  * Return: pointer to contiguous frozen pages on success, or NULL if not successful.
  */
-struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages,
-		gfp_t gfp_mask, int nid, nodemask_t *nodemask)
+struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages,
+		gfp_t gfp_mask, int nid, nodemask_t *nodemask,
+		unsigned long user_addr)
 {
 	unsigned long ret, pfn, flags;
 	struct zonelist *zonelist;

@@ -7282,10 +7295,11 @@ struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages,
 				 * win the race and cause allocation to fail.
 				 */
 				spin_unlock_irqrestore(&zone->lock, flags);
-				ret = alloc_contig_frozen_range_noprof(pfn,
+				ret = __alloc_contig_frozen_range(pfn,
 							pfn + nr_pages,
 							ACR_FLAGS_NONE,
-							gfp_mask);
+							gfp_mask,
+							user_addr);
 				if (!ret)
 					return pfn_to_page(pfn);
 				spin_lock_irqsave(&zone->lock, flags);

mm/page_alloc.c +7 -1

+
+struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages,
+		gfp_t gfp_mask, int nid, nodemask_t *nodemask)
+{
+	return alloc_contig_frozen_pages_user_noprof(nr_pages, gfp_mask, nid,
+						     nodemask, USER_ADDR_NONE);
+}
 EXPORT_SYMBOL(alloc_contig_frozen_pages_noprof);
 
 /**
--

MST