Skip to contentSkip to conversation

mm: Some cleanups for page allocator APIs

PATCH Superseded Brendan Jackman 11 participants 65 messages Jun 29 – Jul 1

Subsystems (26)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover mm: Some cleanups for page allocator APIs

This is based on mm-new, it depends on moving alloc_tag to mm/ [0].

[0] https://lore.kernel.org/all/aj5QBtJcphPElczI@lucifer/

Some tweaks and cleanups for page allocator entrypoint and flags. This is motivated by preparation for __GFP_UNMAPPED [1] (which will probably become ALLOC_UNMAPPED in its next iteration), but all this is supposed to be an improvement to the codebase in its own right: unifying code paths, reducing API surface, and removing GFP flags.

[1] https://lore.kernel.org/all/[email protected]/

This started with unifying __alloc_frozen_pages[_nolock]_noprof() and expanded from there.

Unifying the nolock allocator entrypoint with the normal allocator entrypoint means adding an alloc_flags argument to the later (only exposed within mm/). This presents an opportunity to take advantage of that arg to remove some GFP flags, if we add that alloc_flags arg a bit more broadly to allocator entrypoints.

To distinguish between mm-internal and "public" allocator entrypoints, it makes sense to use the __ prefix. There are already some public APIs with that prefix. For *alloc_pages*, just removing those variants seems like a nice cleanup anyway, so do that. For get_free_pages, the "__" variant is the _only_ variant and it's very widely used, so it doesn't seem worthwhile to modify that. Therefore, scope this "__" change specifically to the *alloc_pages* API, which means we leave the *folio_alloc* API untouched too, even though that could probably be cleaned up if so desired.

Tested:

- KVM, mm, and BPF selftests in a QEMU VM

- kunit.py on x86_64

- For the ALLOC_NO_CODETAG bits I just booted a VM and read /proc/allocinfo. I confirmed that if I remove ALLOC_NO_CODETAG, the kernel crashes in early boot, so I was at least booting code that depends on this logic.

I used Google's internal version of Antigravity (AI coding harness) to do the repetitive bits, those commits are marked with Assisted-by, the rest is manual.

---
Changes in v3:
- Created mm/page_alloc.h
- Fixed EXPORT_SYMBOL() issues
- Reworded commit messages per Sashiko's pointers
- Dropped rename of alloc_flags arg in prepare_alloc_pages() (Suren)
- Renamed gfp_to_alloc_flags_nonblocking() too after rebasing onto:
https://lore.kernel.org/all/[email protected]/
- Link to v2: https://patch.msgid.link/[email protected]

Changes in v2:
- Fixed up whitespace in nolock unification patch
- Introduced ALLOC_DEFAULT to replace literal 0 for alloc_flags
- All other patches are new
- Link to v1: https://patch.msgid.link/[email protected]

---
Brendan Jackman (16):
mm/page_alloc: rename ALLOC_TRYLOCK -> ALLOC_NOLOCK mm/page_alloc: some renames to clarify alloc_flags scopes
mm: name some args in a function declaration
mm: Split out internal page_alloc.h mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof() mm/page_alloc: relax GFP WARN in nolock allocs
mm: move some stuff to mm/page_alloc.h perf/x86/intel: Use higher-level allocator API
KVM: VMX: Use higher-level allocator API x86/virt: Use higher-level allocator API
sgi-xp: Use higher-level allocator API net/funeth: Switch to higher-level allocator API
mm: Remove __alloc_pages_node()
mm: Move __alloc_pages() to mm/page_alloc.h
mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG
mm: remove the __GFP_NO_OBJ_EXT flag

 Documentation/admin-guide/cgroup-v1/cpusets.rst  |   2 +-
 Documentation/admin-guide/mm/transhuge.rst       |   2 +-
 MAINTAINERS                                      |   1 +
 arch/x86/events/intel/ds.c                       |   6 +-
 arch/x86/kvm/vmx/vmx.c                           |   2 +-
 arch/x86/virt/hw.c                               |   2 +-
 drivers/misc/sgi-xp/xpc_uv.c                     |   2 +-
 drivers/net/ethernet/fungible/funeth/funeth_rx.c |   2 +-
 include/linux/gfp.h                              |  54 +---
 mm/alloc_tag.c                                   |  22 +-
 mm/compaction.c                                  |   5 +-
 mm/hugetlb.c                                     |   4 +-
 mm/internal.h                                    | 253 ------------------
 mm/khugepaged.c                                  |   1 +
 mm/memory-failure.c                              |   1 +
 mm/memory_hotplug.c                              |   1 +
 mm/mempolicy.c                                   |  11 +-
 mm/mm_init.c                                     |   1 +
 mm/page_alloc.c                                  | 259 ++++++++++---------
 mm/page_alloc.h                                  | 316 +++++++++++++++++++++++
 mm/page_frag_cache.c                             |   6 +-
 mm/page_isolation.c                              |   1 +
 mm/page_owner.c                                  |   2 +-
 mm/show_mem.c                                    |   1 +
 mm/slub.c                                        |   7 +-
 mm/swap.c                                        |   1 +
 mm/vmscan.c                                      |   1 +
 mm/vmstat.c                                      |   1 +
 tools/include/linux/gfp_types.h                  |   7 -
 29 files changed, 506 insertions(+), 468 deletions(-)

---
base-commit: ec005628e9dbcef26b761fa860f264fe6e5fd690
change-id: 20260617-alloc-trylock-14ad37dab337

Best regards,

Signature · 1 line
Brendan Jackman <[email protected]>
#1 lore raw

PATCH v3 1/16 mm/page_alloc: rename ALLOC_TRYLOCK -> ALLOC_NOLOCK

It's confusing that the function is called "nolock" but the flag is called "trylock", align them.

The function's terminology is more visible and has more mindshare so use that.

---
 mm/internal.h   |  2 +-
 mm/page_alloc.c | 10 +++++-----
 2 files changed, 6 insertions(+), 6 deletions(-)
mm/internal.h +1 -1
index 430aa72a45758..2237eee030cba 100644
@@ -1480,7 +1480,7 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone,
 #define ALLOC_NOFRAGMENT	  0x0
 #endif
 #define ALLOC_HIGHATOMIC	0x200 /* Allows access to MIGRATE_HIGHATOMIC */
-#define ALLOC_TRYLOCK		0x400 /* Only use spin_trylock in allocation path */
+#define ALLOC_NOLOCK		0x400 /* Only use spin_trylock in allocation path */
 #define ALLOC_KSWAPD		0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */
 
 /* Flags that allow allocations below the min watermark. */
mm/page_alloc.c +5 -5
index 62f71ece7ca17..421271849f291 100644
@@ -2530,7 +2530,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
 	unsigned long flags;
 	int i;
 
-	if (unlikely(alloc_flags & ALLOC_TRYLOCK)) {
+	if (unlikely(alloc_flags & ALLOC_NOLOCK)) {
 		if (!spin_trylock_irqsave(&zone->lock, flags))
 			return 0;
 	} else {
@@ -3218,7 +3218,7 @@ struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone,
 
 	do {
 		page = NULL;
-		if (unlikely(alloc_flags & ALLOC_TRYLOCK)) {
+		if (unlikely(alloc_flags & ALLOC_NOLOCK)) {
 			if (!spin_trylock_irqsave(&zone->lock, flags))
 				return NULL;
 		} else {
@@ -5059,7 +5059,7 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
 	 * Don't invoke should_fail logic, since it may call
 	 * get_random_u32() and printk() which need to spin_lock.
 	 */
-	if (!(*alloc_flags & ALLOC_TRYLOCK) &&
+	if (!(*alloc_flags & ALLOC_NOLOCK) &&
 	    should_fail_alloc_page(gfp_mask, order))
 		return false;
 
@@ -7804,7 +7804,7 @@ static bool cond_accept_memory(struct zone *zone, unsigned int order,
 		return false;
 
 	/* Bailout, since try_to_accept_memory_one() needs to take a lock */
-	if (alloc_flags & ALLOC_TRYLOCK)
+	if (alloc_flags & ALLOC_NOLOCK)
 		return false;
 
 	wmark = promo_wmark_pages(zone);
@@ -7896,7 +7896,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned
 	 */
 	gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_COMP
 			| gfp_flags;
-	unsigned int alloc_flags = ALLOC_TRYLOCK;
+	unsigned int alloc_flags = ALLOC_NOLOCK;
 	struct alloc_context ac = { };
 	struct page *page;
 
Signature · 1 line
2.54.0
#3 lore raw

PATCH v3 2/16 mm/page_alloc: some renames to clarify alloc_flags scopes

In reply to Brendan Jackman

It's pretty confusing that:

- The slowpath and fastpath have a totally distinct set of alloc_flags.

- gfp_to_alloc_flags() sounds generic but it only influences the slowpath.

Rename some variables to highlight which alloc_flags are fastpath-specific. Rename gfp_to_alloc_flags() to highlight that it's slowpath-specific.

gfp_to_alloc_flags_cma() and gfp_to_alloc_flags_nonblocking() currently have perfectly harmless names, but to keep the naming consistent also rename those to the alloc_flags_*() pattern (which already exists for alloc_flags_nofragment()).

---
 mm/page_alloc.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)
mm/page_alloc.c +14 -14
index 421271849f291..6010693861ec2 100644
@@ -3774,8 +3774,8 @@ alloc_flags_nofragment(struct zone *zone, gfp_t gfp_mask)
 }
 
 /* Must be called after current_gfp_context() which can change gfp_mask */
-static inline unsigned int gfp_to_alloc_flags_cma(gfp_t gfp_mask,
-						  unsigned int alloc_flags)
+static inline unsigned int alloc_flags_cma(gfp_t gfp_mask,
+					   unsigned int alloc_flags)
 {
 #ifdef CONFIG_CMA
 	if (gfp_migratetype(gfp_mask) == MIGRATE_MOVABLE)
@@ -4474,7 +4474,7 @@ static void wake_all_kswapds(unsigned int order, gfp_t gfp_mask,
 }
 
 static inline unsigned int
-gfp_to_alloc_flags_nonblocking(gfp_t gfp_mask, unsigned int order)
+alloc_flags_nonblocking(gfp_t gfp_mask, unsigned int order)
 {
 	unsigned int alloc_flags = 0;
 
@@ -4497,7 +4497,7 @@ gfp_to_alloc_flags_nonblocking(gfp_t gfp_mask, unsigned int order)
 }
 
 static inline unsigned int
-gfp_to_alloc_flags(gfp_t gfp_mask, unsigned int order)
+alloc_flags_slowpath(gfp_t gfp_mask, unsigned int order)
 {
 	unsigned int alloc_flags = ALLOC_WMARK_MIN | ALLOC_CPUSET;
 
@@ -4512,7 +4512,7 @@ gfp_to_alloc_flags(gfp_t gfp_mask, unsigned int order)
 	if (gfp_mask & __GFP_KSWAPD_RECLAIM)
 		alloc_flags |= ALLOC_KSWAPD;
 
-	alloc_flags |= gfp_to_alloc_flags_nonblocking(gfp_mask, order);
+	alloc_flags |= alloc_flags_nonblocking(gfp_mask, order);
 
 	if (!(gfp_mask & __GFP_DIRECT_RECLAIM)) {
 		/*
@@ -4525,7 +4525,7 @@ gfp_to_alloc_flags(gfp_t gfp_mask, unsigned int order)
 	} else if (unlikely(rt_or_dl_task(current)) && in_task())
 		alloc_flags |= ALLOC_MIN_RESERVE;
 
-	alloc_flags = gfp_to_alloc_flags_cma(gfp_mask, alloc_flags);
+	alloc_flags = alloc_flags_cma(gfp_mask, alloc_flags);
 
 	if (defrag_mode)
 		alloc_flags |= ALLOC_NOFRAGMENT;
@@ -4791,7 +4791,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	 * kswapd needs to be woken up, and to avoid the cost of setting up
 	 * alloc_flags precisely. So we do that now.
 	 */
-	alloc_flags = gfp_to_alloc_flags(gfp_mask, order);
+	alloc_flags = alloc_flags_slowpath(gfp_mask, order);
 
 	/*
 	 * We need to recalculate the starting point for the zonelist iterator
@@ -4832,7 +4832,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 
 	reserve_flags = __gfp_pfmemalloc_flags(gfp_mask);
 	if (reserve_flags)
-		alloc_flags = gfp_to_alloc_flags_cma(gfp_mask, reserve_flags) |
+		alloc_flags = alloc_flags_cma(gfp_mask, reserve_flags) |
 					  (alloc_flags & ALLOC_KSWAPD);
 
 	/*
@@ -5063,7 +5063,7 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
 	    should_fail_alloc_page(gfp_mask, order))
 		return false;
 
-	*alloc_flags = gfp_to_alloc_flags_cma(gfp_mask, *alloc_flags);
+	*alloc_flags = alloc_flags_cma(gfp_mask, *alloc_flags);
 
 	/* Dirty zone balancing only done in the fast path */
 	ac->spread_dirty_pages = (gfp_mask & __GFP_WRITE);
@@ -5277,7 +5277,7 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 		int preferred_nid, nodemask_t *nodemask)
 {
 	struct page *page;
-	unsigned int alloc_flags = ALLOC_WMARK_LOW;
+	unsigned int fastpath_alloc_flags = ALLOC_WMARK_LOW;
 	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
 	struct alloc_context ac = { };
 
@@ -5299,18 +5299,18 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 	gfp = current_gfp_context(gfp);
 	alloc_gfp = gfp;
 	if (!prepare_alloc_pages(gfp, order, preferred_nid, nodemask, &ac,
-			&alloc_gfp, &alloc_flags))
+			&alloc_gfp, &fastpath_alloc_flags))
 		return NULL;
 
 	/*
 	 * Forbid the first pass from falling back to types that fragment
 	 * memory until all local zones are considered.
 	 */
-	alloc_flags |= alloc_flags_nofragment(zonelist_zone(ac.preferred_zoneref), gfp);
-	alloc_flags |= gfp_to_alloc_flags_nonblocking(gfp, order) & ALLOC_HIGHATOMIC;
+	fastpath_alloc_flags |= alloc_flags_nofragment(zonelist_zone(ac.preferred_zoneref), gfp);
+	fastpath_alloc_flags |= alloc_flags_nonblocking(gfp, order) & ALLOC_HIGHATOMIC;
 
 	/* First allocation attempt */
-	page = get_page_from_freelist(alloc_gfp, order, alloc_flags, &ac);
+	page = get_page_from_freelist(alloc_gfp, order, fastpath_alloc_flags, &ac);
 	if (likely(page))
 		goto out;
 
Signature · 1 line
2.54.0
#7 lore raw

PATCH v3 3/16 mm: name some args in a function declaration

In reply to Brendan Jackman

Checkpatch complains about this, a later patch will move the code, fix it so that checkpatch doesn't complain about that patch. Do it in a separate patch so the "move the code" patch is trivial to review using Git's diff colouring.

---
 mm/internal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/internal.h +2 -2
index 2237eee030cba..8ce59c5664497 100644
@@ -919,8 +919,8 @@ extern bool free_pages_prepare(struct page *page, unsigned int order);
 
 extern int user_min_free_kbytes;
 
-struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int nid,
-		nodemask_t *);
+struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, int nid,
+		nodemask_t *nodemask);
 #define __alloc_frozen_pages(...) \
 	alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__))
 void free_frozen_pages(struct page *page, unsigned int order);
Signature · 1 line
2.54.0
#9 lore raw

PATCH v3 4/16 mm: Split out internal page_alloc.h

In reply to Brendan Jackman

internal.h is a bit bloated, seems like time for a page_alloc.h.

Where it wasn't obvious, the heuristic for deciding what goes into this new header was "does it support/correspond to a definition in mm/page_alloc.c?"

Only need to include it from 15 .c files out of 164 so this does seem like a genuine reduction in scopes, which is nice. And there's no circular internal.h<->page_alloc.h dependency, so it seems worthwhile to split this up before that inevitably emerges!

---
 MAINTAINERS          |   1 +
 mm/compaction.c      |   1 +
 mm/hugetlb.c         |   1 +
 mm/internal.h        | 252 -----------------------------------------------
 mm/khugepaged.c      |   1 +
 mm/memory-failure.c  |   1 +
 mm/memory_hotplug.c  |   1 +
 mm/mempolicy.c       |   1 +
 mm/mm_init.c         |   1 +
 mm/page_alloc.c      |   1 +
 mm/page_alloc.h      | 269 +++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/page_frag_cache.c |   2 +-
 mm/page_isolation.c  |   1 +
 mm/page_owner.c      |   2 +-
 mm/show_mem.c        |   1 +
 mm/slub.c            |   1 +
 mm/swap.c            |   1 +
 mm/vmscan.c          |   1 +
 18 files changed, 285 insertions(+), 254 deletions(-)
MAINTAINERS +1 -0
index f55cc75801f4c..978a04e1f7cc3 100644
@@ -17171,6 +17171,7 @@ F:	mm/debug_page_alloc.c
 F:	mm/debug_page_ref.c
 F:	mm/fail_page_alloc.c
 F:	mm/page_alloc.c
+F:	mm/page_alloc.h
 F:	mm/page_ext.c
 F:	mm/page_frag_cache.c
 F:	mm/page_isolation.c
mm/compaction.c +1 -0
index f08765ade014c..7d80735502d9a 100644
@@ -24,6 +24,7 @@
 #include <linux/page_owner.h>
 #include <linux/psi.h>
 #include <linux/cpuset.h>
+#include "page_alloc.h"
 #include "internal.h"
 
 #ifdef CONFIG_COMPACTION
mm/hugetlb.c +1 -0
index fb7ad2a4a26b4..f7925624c4d2e 100644
@@ -47,6 +47,7 @@
 #include <linux/node.h>
 #include <linux/page_owner.h>
 #include "internal.h"
+#include "page_alloc.h"
 #include "hugetlb_vmemmap.h"
 #include "hugetlb_cma.h"
 #include "hugetlb_internal.h"
mm/internal.h +0 -252

282 lines not on this page · raw message

mm/khugepaged.c +1 -0
index 617bca76db49b..58e14d1543ecb 100644
@@ -26,6 +26,7 @@
 
 #include <asm/tlb.h>
 #include "internal.h"
+#include "page_alloc.h"
 #include "mm_slot.h"
 
 enum scan_result {
mm/memory-failure.c +1 -0
index a09d85142da46..49edc37ad4324 100644
@@ -66,6 +66,7 @@
 #include <trace/events/memory-failure.h>
 
 #include "swap.h"
+#include "page_alloc.h"
 #include "internal.h"
 
 static int sysctl_memory_failure_early_kill __read_mostly;
mm/memory_hotplug.c +1 -0
index 7ac19fab22632..9539e40c478ed 100644
@@ -40,6 +40,7 @@
 #include <asm/tlbflush.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 #include "shuffle.h"
 
 enum {
mm/mempolicy.c +1 -0
index 36699fabd3c22..9c740324f9160 100644
@@ -119,6 +119,7 @@
 #include <linux/memory.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 
 /* Internal flags */
 #define MPOL_MF_DISCONTIG_OK (MPOL_MF_INTERNAL << 0)	/* Skip checks for continuous vmas */
mm/mm_init.c +1 -0
index 4026b084bd4bf..32593cca124f8 100644
@@ -33,6 +33,7 @@
 #include <linux/kexec_handover.h>
 #include <linux/hugetlb.h>
 #include "internal.h"
+#include "page_alloc.h"
 #include "slab.h"
 #include "shuffle.h"
 
mm/page_alloc.c +1 -0
index 6010693861ec2..a3ba63c7f9199 100644
@@ -56,6 +56,7 @@
 #include <linux/pgalloc_tag.h>
 #include <asm/div64.h>
 #include "internal.h"
+#include "page_alloc.h"
 #include "shuffle.h"
 #include "page_reporting.h"
 
mm/page_alloc.h new file +269 -0

269 lines not on this page · raw message

mm/page_frag_cache.c +1 -1
index d2423f30577e4..a1077cef3a791 100644
@@ -18,7 +18,7 @@
 #include <linux/init.h>
 #include <linux/mm.h>
 #include <linux/page_frag_cache.h>
-#include "internal.h"
+#include "page_alloc.h"
 
 static unsigned long encoded_page_create(struct page *page, unsigned int order,
 					 bool pfmemalloc)
mm/page_isolation.c +1 -0
index 32ce8a7d9df35..e5dfc7bf49446 100644
@@ -11,6 +11,7 @@
 #include <linux/page_owner.h>
 #include <linux/migrate.h>
 #include "internal.h"
+#include "page_alloc.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/page_isolation.h>
mm/page_owner.c +1 -1
index 74a844a86441e..6f580a64bdba3 100644
@@ -13,7 +13,7 @@
 #include <linux/memcontrol.h>
 #include <linux/sched/clock.h>
 
-#include "internal.h"
+#include "page_alloc.h"
 
 /*
  * TODO: teach PAGE_OWNER_STACK_DEPTH (__dump_page_owner and save_stack)
mm/show_mem.c +1 -0
index 1b721a8ade67d..d1288b4c2b640 100644
@@ -16,6 +16,7 @@
 #include <linux/vmstat.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 #include "swap.h"
 
 atomic_long_t _totalram_pages __read_mostly;
mm/slub.c +1 -0
index 9ec774dc70096..877021e69cc41 100644
@@ -53,6 +53,7 @@
 #include <trace/events/kmem.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 
 /*
  * Lock order:
mm/swap.c +1 -0
index 0132ed0fb76b6..5e389bcc073a9 100644
@@ -39,6 +39,7 @@
 #include <linux/buffer_head.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/pagemap.h>
mm/vmscan.c +1 -0
index 754c5f5d716aa..de1879db39160 100644
@@ -66,6 +66,7 @@
 #include <linux/sched/sysctl.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 #include "swap.h"
 
 #define CREATE_TRACE_POINTS
Signature · 1 line
2.54.0
#12 lore raw

PATCH v3 5/16 mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Brendan Jackman

Currently the core allocator code is controlled by ALLOC_NOLOCK, but the main entry point function is significantly different from the normal __alloc_frozen_pages_nolock(), this is tiring when reading the code.

Plumb the ALLOC_NOLOCK control one layer up in the call stack: create an alloc_flags argument to __alloc_frozen_pages_nolock() (which is only exposed to mm/) and then turn the nolock variant into a thin wrapper that just sets that flag (as well as handling NUMA_NO_NODE, similar to how some of the wrappers in gfp.h do).

Rationale that this doesn't change anything:

1. Simple bits: A bunch of the nolock-specific handling is just moved to the new alloc_order_allowed(), alloc_trylock_allowed() and gfp_trylock.

2. __alloc_frozen_pages_noprof() has some extra logic that wasn't previously in the nolock variant:

a. Application of gfp_allowed_mask; this only affects early boot, and
only flags that affect the slowpath get changed here.

b. Application of current_gfp_context() - also only affects the
slowpath

3. The slowpath itself: this is now just explicitly skipped under !ALLOC_TRYLOCK.

Ulterior motive: adding an alloc_flags arg to the allocator's mm-internal entrypoint can later be used to do more allocation customisation without needing to create new GFP flags.

While adding this flag to a bunch of places, create ALLOC_DEFAULT to avoid a mysterious literal 0 in most places. alloc_frozen_pages_noprof() is defined above the alloc flags so just leave that as a slightly messy exception instead of trying to fully reorder mm/internal.h for that one case.

No functional change intended.

---
 mm/hugetlb.c    |   3 +-
 mm/mempolicy.c  |  10 ++--
 mm/page_alloc.c | 178 +++++++++++++++++++++++++++++---------------------------
 mm/page_alloc.h |   6 +-
 mm/slub.c       |   6 +-
 5 files changed, 108 insertions(+), 95 deletions(-)
mm/hugetlb.c +2 -1
index f7925624c4d2e..dfcfcfa4715bf 100644
@@ -1806,7 +1806,8 @@ static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask,
 	if (alloc_try_hard)
 		gfp_mask |= __GFP_RETRY_MAYFAIL;
 
-	folio = (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmask);
+	folio = (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmask,
+						     ALLOC_DEFAULT);
 
 	/*
 	 * If we did not specify __GFP_RETRY_MAYFAIL, but still got a
mm/mempolicy.c +6 -4
index 9c740324f9160..41d630f0ea821 100644
@@ -2426,9 +2426,11 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
 	 */
 	preferred_gfp = gfp | __GFP_NOWARN;
 	preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
-	page = __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask);
+	page = __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask,
+					   ALLOC_DEFAULT);
 	if (!page)
-		page = __alloc_frozen_pages_noprof(gfp, order, nid, NULL);
+		page = __alloc_frozen_pages_noprof(gfp, order, nid, NULL,
+						   ALLOC_DEFAULT);
 
 	return page;
 }
@@ -2476,7 +2478,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 			 */
 			page = __alloc_frozen_pages_noprof(
 				gfp | __GFP_THISNODE | __GFP_NORETRY, order,
-				nid, NULL);
+				nid, NULL, ALLOC_DEFAULT);
 			if (page || !(gfp & __GFP_DIRECT_RECLAIM))
 				return page;
 			/*
@@ -2488,7 +2490,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 		}
 	}
 
-	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask);
+	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, ALLOC_DEFAULT);
 
 	if (unlikely(pol->mode == MPOL_INTERLEAVE ||
 		     pol->mode == MPOL_WEIGHTED_INTERLEAVE) && page) {
mm/page_alloc.c +92 -86

226 lines not on this page · raw message

mm/page_alloc.h +4 -2
index 3250d44f96457..e16f905f859a7 100644
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/types.h>
 
+#define ALLOC_DEFAULT		0
 /* The ALLOC_WMARK bits are used as an index to zone->watermark */
 #define ALLOC_WMARK_MIN		WMARK_MIN
 #define ALLOC_WMARK_LOW		WMARK_LOW
@@ -219,7 +220,7 @@ extern bool free_pages_prepare(struct page *page, unsigned int order);
 extern int user_min_free_kbytes;
 
 struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, int nid,
-		nodemask_t *nodemask);
+		nodemask_t *nodemask, unsigned int alloc_flags);
 #define __alloc_frozen_pages(...) \
 	alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__))
 void free_frozen_pages(struct page *page, unsigned int order);
@@ -230,7 +231,8 @@ struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order);
 #else
 static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order)
 {
-	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL);
+	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL,
+					   0 /* ALLOC_DEFAULT */);
 }
 #endif
 
mm/slub.c +4 -2
index 877021e69cc41..3989b4758ae0a 100644
@@ -3292,7 +3292,8 @@ static inline struct slab *alloc_slab_page(gfp_t flags, int node,
 	else if (node == NUMA_NO_NODE)
 		page = alloc_frozen_pages(flags, order);
 	else
-		page = __alloc_frozen_pages(flags, order, node, NULL);
+		page = __alloc_frozen_pages(flags, order, node, NULL,
+					    ALLOC_DEFAULT);
 
 	if (!page)
 		return NULL;
@@ -5302,7 +5303,8 @@ static void *___kmalloc_large_node(size_t size, gfp_t flags, int node)
 	if (node == NUMA_NO_NODE)
 		page = alloc_frozen_pages_noprof(flags, order);
 	else
-		page = __alloc_frozen_pages_noprof(flags, order, node, NULL);
+		page = __alloc_frozen_pages_noprof(flags, order, node, NULL,
+						   ALLOC_DEFAULT);
 
 	if (page) {
 		ptr = page_address(page);
Signature · 1 line
2.54.0
#24 lore raw

PATCH v3 6/16 mm/page_alloc: relax GFP WARN in nolock allocs

In reply to Brendan Jackman

This WARN forbids setting other flags than __GFP_ACCOUNT but we unconditionally set the ones in gfp_nolock so they are certainly fine for the caller to set.

There are other GFP flags that are almost certainly fine to set here; Willy noted GFP_HIGHMEM, GFP_DMA, GFP_MOVABLE and GFP_HARDWALL. But, nolock allocation is rather special, so be conservative to try and ensure we have a chance to think carefully before nontrivial new usecases arise.

---
 mm/page_alloc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/page_alloc.c +2 -1
index 8d409d075e3e9..9cb3f1665b41b 100644
@@ -5355,7 +5355,8 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 		return NULL;
 
 	if (alloc_flags & ALLOC_NOLOCK) {
-		VM_WARN_ON_ONCE(gfp & ~__GFP_ACCOUNT);
+		/* Certain other flags could be supported later if needed. */
+		VM_WARN_ON_ONCE(gfp & ~(__GFP_ACCOUNT | gfp_nolock));
 		if (!alloc_trylock_allowed())
 			return NULL;
 		gfp |= gfp_nolock;
Signature · 1 line
2.54.0
#27 lore raw

PATCH v3 7/16 mm: move some stuff to mm/page_alloc.h

In reply to Brendan Jackman

Some of this stuff in the public header is only used internally so shrink the scope to avoid silently growing new users.

drain_local_pages() is still used from kernel/power/snapshot.c so that needs to stay behind.

---
 include/linux/gfp.h | 26 --------------------------
 mm/page_alloc.h     | 28 ++++++++++++++++++++++++++++
 mm/vmstat.c         |  1 +
 3 files changed, 29 insertions(+), 26 deletions(-)
include/linux/gfp.h +0 -26
index cdf95a9f0b87c..01d6d2591f49e 100644
@@ -17,28 +17,6 @@ struct mempolicy;
 #define __default_gfp(a,b,...) b
 #define default_gfp(...) __default_gfp(,##__VA_ARGS__,GFP_KERNEL)
 
-/* Convert GFP flags to their corresponding migrate type */
-#define GFP_MOVABLE_MASK (__GFP_RECLAIMABLE|__GFP_MOVABLE)
-#define GFP_MOVABLE_SHIFT 3
-
-static inline int gfp_migratetype(const gfp_t gfp_flags)
-{
-	VM_WARN_ON((gfp_flags & GFP_MOVABLE_MASK) == GFP_MOVABLE_MASK);
-	BUILD_BUG_ON((1UL << GFP_MOVABLE_SHIFT) != ___GFP_MOVABLE);
-	BUILD_BUG_ON((___GFP_MOVABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_MOVABLE);
-	BUILD_BUG_ON((___GFP_RECLAIMABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_RECLAIMABLE);
-	BUILD_BUG_ON(((___GFP_MOVABLE | ___GFP_RECLAIMABLE) >>
-		      GFP_MOVABLE_SHIFT) != MIGRATE_HIGHATOMIC);
-
-	if (unlikely(page_group_by_mobility_disabled))
-		return MIGRATE_UNMOVABLE;
-
-	/* Group based on mobility */
-	return (__force unsigned long)(gfp_flags & GFP_MOVABLE_MASK) >> GFP_MOVABLE_SHIFT;
-}
-#undef GFP_MOVABLE_MASK
-#undef GFP_MOVABLE_SHIFT
-
 static inline bool gfpflags_allow_blocking(const gfp_t gfp_flags)
 {
 	return !!(gfp_flags & __GFP_DIRECT_RECLAIM);
@@ -395,10 +373,6 @@ extern void free_pages(unsigned long addr, unsigned int order);
 #define __free_page(page) __free_pages((page), 0)
 #define free_page(addr) free_pages((addr), 0)
 
-void page_alloc_init_cpuhp(void);
-bool decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp);
-void drain_zone_pages(struct zone *zone, struct per_cpu_pages *pcp);
-void drain_all_pages(struct zone *zone);
 void drain_local_pages(struct zone *zone);
 
 void page_alloc_init_late(void);
mm/page_alloc.h +28 -0
index e16f905f859a7..af83764788b96 100644
@@ -266,6 +266,34 @@ static inline bool free_area_empty(struct free_area *area, int migratetype)
 	return list_empty(&area->free_list[migratetype]);
 }
 
+/* Convert GFP flags to their corresponding migrate type */
+#define GFP_MOVABLE_MASK (__GFP_RECLAIMABLE|__GFP_MOVABLE)
+#define GFP_MOVABLE_SHIFT 3
+
+static inline int gfp_migratetype(const gfp_t gfp_flags)
+{
+	VM_WARN_ON((gfp_flags & GFP_MOVABLE_MASK) == GFP_MOVABLE_MASK);
+	BUILD_BUG_ON((1UL << GFP_MOVABLE_SHIFT) != ___GFP_MOVABLE);
+	BUILD_BUG_ON((___GFP_MOVABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_MOVABLE);
+	BUILD_BUG_ON((___GFP_RECLAIMABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_RECLAIMABLE);
+	BUILD_BUG_ON(((___GFP_MOVABLE | ___GFP_RECLAIMABLE) >>
+		      GFP_MOVABLE_SHIFT) != MIGRATE_HIGHATOMIC);
+
+	if (unlikely(page_group_by_mobility_disabled))
+		return MIGRATE_UNMOVABLE;
+
+	/* Group based on mobility */
+	return (__force unsigned long)(gfp_flags & GFP_MOVABLE_MASK) >> GFP_MOVABLE_SHIFT;
+}
+#undef GFP_MOVABLE_MASK
+#undef GFP_MOVABLE_SHIFT
+
+bool decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp);
+void drain_zone_pages(struct zone *zone, struct per_cpu_pages *pcp);
+void drain_all_pages(struct zone *zone);
+void drain_local_pages(struct zone *zone);
+
+void page_alloc_init_cpuhp(void);
 void page_alloc_sysctl_init(void);
 
 #endif /* __MM_PAGE_ALLOC_H */
mm/vmstat.c +1 -0
index 7b93fbf9af092..3b5cb1031f720 100644
@@ -30,6 +30,7 @@
 #include <linux/sched/isolation.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 
 #ifdef CONFIG_PROC_FS
 #ifdef CONFIG_NUMA
Signature · 1 line
2.54.0
#29 lore raw

PATCH v3 8/16 perf/x86/intel: Use higher-level allocator API

In reply to Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Ian Rogers <[email protected]>
Cc Adrian Hunter <[email protected]>
Cc James Clark <[email protected]>
Assisted-by Gemini:unknown-version
---
 arch/x86/events/intel/ds.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/events/intel/ds.c +3 -3
index 91a093d8cf2e7..70be80211d823 100644
@@ -832,7 +832,7 @@ static void *dsalloc_pages(size_t size, gfp_t flags, int cpu)
 	int node = cpu_to_node(cpu);
 	struct page *page;
 
-	page = __alloc_pages_node(node, flags | __GFP_ZERO, order);
+	page = alloc_pages_node(node, flags | __GFP_ZERO, order);
 	return page ? page_address(page) : NULL;
 }
 
@@ -1088,9 +1088,9 @@ void init_arch_pebs_on_cpu(int cpu)
 
 	/*
 	 * 4KB-aligned pointer of the output buffer
-	 * (__alloc_pages_node() return page aligned address)
+	 * (alloc_pages_node() returns page aligned address)
 	 * Buffer Size = 4KB * 2^SIZE
-	 * contiguous physical buffer (__alloc_pages_node() with order)
+	 * contiguous physical buffer (alloc_pages_node() with order)
 	 */
 	arch_pebs_base = virt_to_phys(cpuc->pebs_vaddr) | PEBS_BUFFER_SHIFT;
 	wrmsrq_on_cpu(cpu, MSR_IA32_PEBS_BASE, arch_pebs_base);
Signature · 1 line
2.54.0
#31 lore raw

PATCH v3 9/16 KVM: VMX: Use higher-level allocator API

In reply to Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Cc Sean Christopherson <[email protected]>
Cc Paolo Bonzini <[email protected]>
Assisted-by Gemini:unknown-version
---
 arch/x86/kvm/vmx/vmx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
index 2325be57d3d75..ad6a7fc6a54da 100644
@@ -3028,7 +3028,7 @@ struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
 	struct page *pages;
 	struct vmcs *vmcs;
 
-	pages = __alloc_pages_node(node, flags, 0);
+	pages = alloc_pages_node(node, flags, 0);
 	if (!pages)
 		return NULL;
 	vmcs = page_address(pages);
Signature · 1 line
2.54.0
#34 lore raw

PATCH v3 10/16 x86/virt: Use higher-level allocator API

In reply to Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Assisted-by Gemini:unknown-version
---
 arch/x86/virt/hw.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/virt/hw.c +1 -1
index 7e9091c640be0..a236447ac7a26 100644
@@ -196,7 +196,7 @@ static __init int __x86_vmx_init(void)
 		struct page *page;
 		struct vmcs *vmcs;
 
-		page = __alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
+		page = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
 		if (WARN_ON_ONCE(!page)) {
 			x86_vmx_exit();
 			return -ENOMEM;
Signature · 1 line
2.54.0
#36 lore raw

PATCH v3 11/16 sgi-xp: Use higher-level allocator API

In reply to Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Cc Robin Holt <[email protected]>
Cc Steve Wahl <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Assisted-by Gemini:unknown-model
---
 drivers/misc/sgi-xp/xpc_uv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/sgi-xp/xpc_uv.c +1 -1
index 772c787268932..aacff70204241 100644
@@ -170,7 +170,7 @@ xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name,
 	mq->mmr_blade = uv_cpu_to_blade_id(cpu);
 
 	nid = cpu_to_node(cpu);
-	page = __alloc_pages_node(nid,
+	page = alloc_pages_node(nid,
 				      GFP_KERNEL | __GFP_ZERO | __GFP_THISNODE,
 				      pg_order);
 	if (page == NULL) {
Signature · 1 line
2.54.0
#41 lore raw

PATCH v3 12/16 net/funeth: Switch to higher-level allocator API

In reply to Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Cc Dimitris Michailidis <[email protected]>
Cc Andrew Lunn <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc Jakub Kicinski <[email protected]>
Cc Paolo Abeni <[email protected]>
Assisted-by Gemini:unknown-version
---
 drivers/net/ethernet/fungible/funeth/funeth_rx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/fungible/funeth/funeth_rx.c +1 -1
index 7e2584895de39..d7000017ac2bd 100644
@@ -103,7 +103,7 @@ static int funeth_alloc_page(struct funeth_rxq *q, struct funeth_rxbuf *rb,
 	if (cache_get(q, rb))
 		return 0;
 
-	p = __alloc_pages_node(node, gfp | __GFP_NOWARN, 0);
+	p = alloc_pages_node(node, gfp | __GFP_NOWARN, 0);
 	if (unlikely(!p))
 		return -ENOMEM;
 
Signature · 1 line
2.54.0
#43 lore raw

PATCH v3 13/16 mm: Remove __alloc_pages_node()

In reply to Brendan Jackman

There were only a few users, which have been removed. The only advantage of this API over alloc_pages_node() is avoiding a single conditional branch. The disadvantages are:

1. More API surface, more sources of confusion, more maintenance.

2. Worse impact of CPU hotplug bugs: most users of __alloc_pages_node() were using the result of cpu_to_node(); if the CPU gets hotplugged out this will return NUMA_NO_NODE. If one of these paths fails to protect against a concurrent hotplug then page_alloc.c will use NUMA_NO_NODE as an index into NODE_DATA() and cause some horrible memory corruption or other. With alloc_pages_node(), the code might just work fine.

Ulterior motive: this frees up the __* variants of the allocator APIs to serve specifically for use as mm-internal API.

---
 include/linux/gfp.h | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
include/linux/gfp.h +4 -16
index 01d6d2591f49e..3bf55a5f9143e 100644
@@ -256,21 +256,6 @@ static inline void warn_if_node_offline(int this_node, gfp_t gfp_mask)
 	dump_stack();
 }
 
-/*
- * Allocate pages, preferring the node given as nid. The node must be valid and
- * online. For more general interface, see alloc_pages_node().
- */
-static inline struct page *
-__alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order)
-{
-	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
-	warn_if_node_offline(nid, gfp_mask);
-
-	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
-}
-
-#define  __alloc_pages_node(...)		alloc_hooks(__alloc_pages_node_noprof(__VA_ARGS__))
-
 static inline
 struct folio *__folio_alloc_node_noprof(gfp_t gfp, unsigned int order, int nid)
 {
@@ -293,7 +278,10 @@ static inline struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask,
 	if (nid == NUMA_NO_NODE)
 		nid = numa_mem_id();
 
-	return __alloc_pages_node_noprof(nid, gfp_mask, order);
+	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
+	warn_if_node_offline(nid, gfp_mask);
+
+	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
 }
 
 #define  alloc_pages_node(...)			alloc_hooks(alloc_pages_node_noprof(__VA_ARGS__))
Signature · 1 line
2.54.0
#46 lore raw

PATCH v3 14/16 mm: Move __alloc_pages() to mm/page_alloc.h

In reply to Brendan Jackman

It's no longer used outside of mm/.

Since this means __alloc_pages_noprof() is no longer visible from gfp.h, this also means moving the definition of alloc_pages_node_noprof into the .c file.

Also remove references to this API from the documentation tree - referring to the specific function name was already questionable but now the function is not even public it definitely seems wrong.

---
 Documentation/admin-guide/cgroup-v1/cpusets.rst |  2 +-
 Documentation/admin-guide/mm/transhuge.rst      |  2 +-
 include/linux/gfp.h                             | 16 +---------------
 mm/page_alloc.c                                 | 13 ++++++++++++-
 mm/page_alloc.h                                 |  4 ++++
 5 files changed, 19 insertions(+), 18 deletions(-)
Documentation/admin-guide/cgroup-v1/cpusets.rst +1 -1
index c7909e5ac1361..52a213aff04e5 100644
@@ -284,7 +284,7 @@ take action.
 ==>
     Unless this feature is enabled by writing "1" to the special file
     /dev/cpuset/memory_pressure_enabled, the hook in the rebalance
-    code of __alloc_pages() for this metric reduces to simply noticing
+    code of the page allocator for this metric reduces to simply noticing
     that the cpuset_memory_pressure_enabled flag is zero.  So only
     systems that enable this feature will compute the metric.
 
Documentation/admin-guide/mm/transhuge.rst +1 -1
index 23f8d13c2629d..16f37135ed80d 100644
@@ -761,7 +761,7 @@ compact_fail
 	but failed.
 
 It is possible to establish how long the stalls were using the function
-tracer to record how long was spent in __alloc_pages() and
+tracer to record how long was spent in the page allocator and
 using the mm_page_alloc tracepoint to identify which allocations were
 for huge pages.
 
include/linux/gfp.h +1 -15
index 3bf55a5f9143e..4d57e9c0bf204 100644
@@ -204,10 +204,6 @@ static inline void arch_free_page(struct page *page, int order) { }
 static inline void arch_alloc_page(struct page *page, int order) { }
 #endif
 
-struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
-		nodemask_t *nodemask);
-#define __alloc_pages(...)			alloc_hooks(__alloc_pages_noprof(__VA_ARGS__))
-
 struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
 		nodemask_t *nodemask);
 #define __folio_alloc(...)			alloc_hooks(__folio_alloc_noprof(__VA_ARGS__))
@@ -272,17 +268,7 @@ struct folio *__folio_alloc_node_noprof(gfp_t gfp, unsigned int order, int nid)
  * prefer the current CPU's closest node. Otherwise node must be valid and
  * online.
  */
-static inline struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask,
-						   unsigned int order)
-{
-	if (nid == NUMA_NO_NODE)
-		nid = numa_mem_id();
-
-	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
-	warn_if_node_offline(nid, gfp_mask);
-
-	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
-}
+struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order);
 
 #define  alloc_pages_node(...)			alloc_hooks(alloc_pages_node_noprof(__VA_ARGS__))
 
mm/page_alloc.c +12 -1
index 9cb3f1665b41b..026f33f217036 100644
@@ -5427,7 +5427,18 @@ struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order,
 		set_page_refcounted(page);
 	return page;
 }
-EXPORT_SYMBOL(__alloc_pages_noprof);
+
+struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order)
+{
+	if (nid == NUMA_NO_NODE)
+		nid = numa_mem_id();
+
+	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
+	warn_if_node_offline(nid, gfp_mask);
+
+	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
+}
+EXPORT_SYMBOL(alloc_pages_node_noprof);
 
 struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
 		nodemask_t *nodemask)
mm/page_alloc.h +4 -0
index af83764788b96..2058cbdca56e7 100644
@@ -244,6 +244,10 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned
 	alloc_hooks(alloc_frozen_pages_nolock_noprof(__VA_ARGS__))
 void free_frozen_pages_nolock(struct page *page, unsigned int order);
 
+struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
+		nodemask_t *nodemask);
+#define __alloc_pages(...)			alloc_hooks(__alloc_pages_noprof(__VA_ARGS__))
+
 extern void zone_pcp_reset(struct zone *zone);
 extern void zone_pcp_disable(struct zone *zone);
 extern void zone_pcp_enable(struct zone *zone);
Signature · 1 line
2.54.0
#48 lore raw

PATCH v3 15/16 mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to Brendan Jackman

Now that alloc_pages has an entrypoint that allows passing alloc_flags, we can take advantage of this to start removing GFP flags that are only used for mm-internal stuff.

This requires also plumbing the alloc_flags into some more of the allocator code, in particular __alloc_pages[_noprof]() gets an alloc_flags arg to go along with its callees, and we now need to pass those flags deeper into the allocator so they can reach the alloc_tag code.

No functional change intended.

---
 mm/alloc_tag.c       | 22 ++++++----------------
 mm/compaction.c      |  4 ++--
 mm/internal.h        |  1 -
 mm/page_alloc.c      | 42 ++++++++++++++++++++++++------------------
 mm/page_alloc.h      | 17 +++++++++++++++--
 mm/page_frag_cache.c |  4 ++--
 6 files changed, 49 insertions(+), 41 deletions(-)
mm/alloc_tag.c +6 -16
index d9be1cf5187d9..a32a94e759b94 100644
@@ -15,6 +15,8 @@
 #include <linux/vmalloc.h>
 #include <linux/kmemleak.h>
 
+#include "internal.h"
+
 #define ALLOCINFO_FILE_NAME		"allocinfo"
 #define MODULE_ALLOC_TAG_VMAP_SIZE	(100000UL * sizeof(struct alloc_tag))
 #define SECTION_START(NAME)		(CODETAG_SECTION_START_PREFIX NAME)
@@ -783,19 +785,6 @@ struct pfn_pool {
 
 #define PFN_POOL_SIZE			((PAGE_SIZE - offsetof(struct pfn_pool, pfns)) / \
 					 sizeof(unsigned long))
-
-/*
- * Skip early PFN recording for a page allocation.  Reuses the
- * %__GFP_NO_OBJ_EXT bit.  Used by __alloc_tag_add_early_pfn() to avoid
- * recursion when allocating pages for the early PFN tracking list
- * itself.
- *
- * Codetags of the pages allocated with __GFP_NO_CODETAG should be
- * cleared (via clear_page_tag_ref()) before freeing the pages to prevent
- * alloc_tag_sub_check() from triggering a warning.
- */
-#define __GFP_NO_CODETAG		__GFP_NO_OBJ_EXT
-
 static struct pfn_pool *current_pfn_pool __initdata;
 
 static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
@@ -806,7 +795,8 @@ static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
 	do {
 		pool = READ_ONCE(current_pfn_pool);
 		if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE) {
-			struct page *new_page = alloc_page(__GFP_HIGH | __GFP_NO_CODETAG);
+			struct page *new_page = __alloc_pages(__GFP_HIGH, 0, numa_mem_id(),
+							      NULL, ALLOC_NO_CODETAG);
 			struct pfn_pool *new;
 
 			if (!new_page) {
@@ -837,7 +827,7 @@ typedef void alloc_tag_add_func(unsigned long pfn);
 static alloc_tag_add_func __rcu *alloc_tag_add_early_pfn_ptr __refdata =
 	RCU_INITIALIZER(__alloc_tag_add_early_pfn);
 
-void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)
+void alloc_tag_add_early_pfn(unsigned long pfn, unsigned int alloc_flags)
 {
 	alloc_tag_add_func *alloc_tag_add;
 
@@ -845,7 +835,7 @@ void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)
 		return;
 
 	/* Skip allocations for the tracking list itself to avoid recursion. */
-	if (gfp_flags & __GFP_NO_CODETAG)
+	if (alloc_flags & ALLOC_NO_CODETAG)
 		return;
 
 	rcu_read_lock();
mm/compaction.c +2 -2
index 7d80735502d9a..4b2318fad4eb5 100644
@@ -83,7 +83,7 @@ static inline bool is_via_compact_memory(int order) { return false; }
 
 static struct page *mark_allocated_noprof(struct page *page, unsigned int order, gfp_t gfp_flags)
 {
-	post_alloc_hook(page, order, __GFP_MOVABLE);
+	post_alloc_hook(page, order, __GFP_MOVABLE, ALLOC_DEFAULT);
 	set_page_refcounted(page);
 	return page;
 }
@@ -1851,7 +1851,7 @@ static struct folio *compaction_alloc_noprof(struct folio *src, unsigned long da
 	}
 	dst = (struct folio *)freepage;
 
-	post_alloc_hook(&dst->page, order, __GFP_MOVABLE);
+	post_alloc_hook(&dst->page, order, __GFP_MOVABLE, ALLOC_DEFAULT);
 	set_page_refcounted(&dst->page);
 	if (order)
 		prep_compound_page(&dst->page, order);
mm/internal.h +0 -1
index c22284f04fc9e..369c656c63fa8 100644
@@ -1237,7 +1237,6 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone,
 enum ttu_flags;
 struct tlbflush_unmap_batch;
 
-
 /*
  * only for MM internal work items which do not depend on
  * any allocations or locks which might depend on allocations
mm/page_alloc.c +24 -18

141 lines not on this page · raw message

mm/page_alloc.h +15 -2
index 2058cbdca56e7..2614bff6795b0 100644
@@ -49,6 +49,16 @@
 #define ALLOC_HIGHATOMIC	0x200 /* Allows access to MIGRATE_HIGHATOMIC */
 #define ALLOC_NOLOCK		0x400 /* Only use spin_trylock in allocation path */
 #define ALLOC_KSWAPD		0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */
+/*
+ * Skip early PFN recording for a page allocation.  Used by
+ * __alloc_tag_add_early_pfn() to avoid recursion when allocating pages for the
+ * early PFN tracking list itself.
+ *
+ * Codetags of the pages allocated with __GFP_NO_CODETAG should be
+ * cleared (via clear_page_tag_ref()) before freeing the pages to prevent
+ * alloc_tag_sub_check() from triggering a warning.
+ */
+#define ALLOC_NO_CODETAG       0x1000
 
 /* Flags that allow allocations below the min watermark. */
 #define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM)
@@ -84,6 +94,8 @@ struct alloc_context {
 	 */
 	enum zone_type highest_zoneidx;
 	bool spread_dirty_pages;
+	/* Only flags that are global to the whole allocation go here. */
+	unsigned int alloc_flags;
 };
 
 /*
@@ -214,7 +226,8 @@ static inline struct page *pageblock_pfn_to_page(unsigned long start_pfn,
 extern void __free_pages_core(struct page *page, unsigned int order,
 		enum meminit_context context);
 
-void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags);
+void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags,
+		     unsigned int alloc_flags);
 extern bool free_pages_prepare(struct page *page, unsigned int order);
 
 extern int user_min_free_kbytes;
@@ -245,7 +258,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned
 void free_frozen_pages_nolock(struct page *page, unsigned int order);
 
 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
-		nodemask_t *nodemask);
+		nodemask_t *nodemask, unsigned int alloc_flags);
 #define __alloc_pages(...)			alloc_hooks(__alloc_pages_noprof(__VA_ARGS__))
 
 extern void zone_pcp_reset(struct zone *zone);
mm/page_frag_cache.c +2 -2
index a1077cef3a791..e63efe78b7d4b 100644
@@ -57,10 +57,10 @@ static struct page *__page_frag_cache_refill(struct page_frag_cache *nc,
 	gfp_mask = (gfp_mask & ~__GFP_DIRECT_RECLAIM) |  __GFP_COMP |
 		   __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC;
 	page = __alloc_pages(gfp_mask, PAGE_FRAG_CACHE_MAX_ORDER,
-			     numa_mem_id(), NULL);
+			     numa_mem_id(), NULL, ALLOC_DEFAULT);
 #endif
 	if (unlikely(!page)) {
-		page = __alloc_pages(gfp, 0, numa_mem_id(), NULL);
+		page = __alloc_pages(gfp, 0, numa_mem_id(), NULL, ALLOC_DEFAULT);
 		order = 0;
 	}
 
Signature · 1 line
2.54.0
#57 lore raw

PATCH v3 16/16 mm: remove the __GFP_NO_OBJ_EXT flag

In reply to Brendan Jackman

All users of the flag are converted to SLAB_ALLOC_NO_RECURSE or ALLOC_NO_CODETAG (from __GFP_NO_CODETAG which reused the NO_OBJ_EXT bit). Free up the flag bit.

[Rebased onto __GFP_NO_CODETAG removal]
---
 tools/include/linux/gfp_types.h | 7 -------
 1 file changed, 7 deletions(-)
tools/include/linux/gfp_types.h +0 -7
index 6c75df30a281d..a93b8bd200b76 100644
@@ -55,7 +55,6 @@ enum {
 #ifdef CONFIG_LOCKDEP
 	___GFP_NOLOCKDEP_BIT,
 #endif
-	___GFP_NO_OBJ_EXT_BIT,
 	___GFP_LAST_BIT
 };
 
@@ -96,7 +95,6 @@ enum {
 #else
 #define ___GFP_NOLOCKDEP	0
 #endif
-#define ___GFP_NO_OBJ_EXT       BIT(___GFP_NO_OBJ_EXT_BIT)
 
 /*
  * Physical address zone modifiers (see linux/mmzone.h - low four bits)
@@ -137,17 +135,12 @@ enum {
  * node with no fallbacks or placement policy enforcements.
  *
  * %__GFP_ACCOUNT causes the allocation to be accounted to kmemcg.
- *
- * %__GFP_NO_OBJ_EXT causes slab allocation to have no object extension.
- * mark_obj_codetag_empty() should be called upon freeing for objects allocated
- * with this flag to indicate that their NULL tags are expected and normal.
  */
 #define __GFP_RECLAIMABLE ((__force gfp_t)___GFP_RECLAIMABLE)
 #define __GFP_WRITE	((__force gfp_t)___GFP_WRITE)
 #define __GFP_HARDWALL   ((__force gfp_t)___GFP_HARDWALL)
 #define __GFP_THISNODE	((__force gfp_t)___GFP_THISNODE)
 #define __GFP_ACCOUNT	((__force gfp_t)___GFP_ACCOUNT)
-#define __GFP_NO_OBJ_EXT   ((__force gfp_t)___GFP_NO_OBJ_EXT)
 
 /**
  * DOC: Watermark modifiers
Signature · 1 line
2.54.0
maintainer
#62 lore raw
In reply to Brendan Jackman

Hi Brendan,

Quoted 9 lines · Brendan Jackman

Some tweaks and cleanups for page allocator entrypoint and flags. This is motivated by preparation for __GFP_UNMAPPED [1] (which will probably become ALLOC_UNMAPPED in its next iteration), but all this is supposed to be an improvement to the codebase in its own right: unifying code paths, reducing API surface, and removing GFP flags.

Tested:

Quoted 8 lines

- KVM, mm, and BPF selftests in a QEMU VM

- kunit.py on x86_64

- For the ALLOC_NO_CODETAG bits I just booted a VM and read /proc/allocinfo. I confirmed that if I remove ALLOC_NO_CODETAG, the kernel crashes in early boot, so I was at least booting code that depends on this logic.

Heads up before the full kbuild report:

CI has tested the following submission:

Status:     FAILURE
Name:       [v3,00/16] mm: Some cleanups for page allocator APIs
Patchwork:  https://patchwork.kernel.org/project/linux-mm/list/?series=1118244&state=*
Matrix:     https://github.com/linux-mm/linux-mm/actions/runs/28375636866
Quoted 6 lines

I used Google's internal version of Antigravity (AI coding harness) to do the repetitive bits, those commits are marked with Assisted-by, the rest is manual.

---

Signature · 2 lines
Sincerely yours,
Mike.
#10 lore raw

Re: [PATCH v3 04/16] mm: Split out internal page_alloc.h

In reply to Brendan Jackman
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Missing inclusion of `page_alloc.h` causes build failures for multiple configurations. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Missing inclusion of page_alloc.h causes build failures for multiple configurations. --

commit 34a7c3e538f8ec840a6b68715addef408e469ff0
Author: Brendan Jackman <[email protected]>

mm: Split out internal page_alloc.h

internal.h is a bit bloated, seems like time for a page_alloc.h. This moves definitions supporting mm/page_alloc.c into the new header.

Quoted 4 lines
mm/internal.h +0 -0
index 8ce59c5664497..c22284f04fc9e 100644

[ ... ]

mm/internal.h extern int defrag_mode; · line 708

Quoted 5 lines
mm/internal.h +0 -5
-static inline unsigned int buddy_order(struct page *page)
-{
-	/* PageBuddy() must be checked by the caller */
-	return page_private(page);
-}

[Severity: Low]
Will this cause a build failure when shuffle page allocator or page reporting features are enabled?

It looks like shuffle_valid_page() in mm/shuffle.c and page_reporting_drain() in mm/page_reporting.c use buddy_order() but don't include the new page_alloc.h.

[ ... ]

mm/internal.h extern int __isolate_free_page(struct page *page, unsigned int order); · line 835

Quoted 2 lines
mm/internal.h +0 -2
-extern void __free_pages_core(struct page *page, unsigned int order,
-		enum meminit_context context);

[Severity: Low]
Does mm/kmsan/init.c need to include page_alloc.h now?

It seems do_collection() uses __free_pages_core() and might fail to build when the kernel is configured for kmsan if the header is missing.

[ ... ]

Quoted 1 line

-#define ALLOC_CMA 0x80 /* allow allocations from CMA areas */

[Severity: Low]
Could this break the build for migration and page reporting configurations?

I am noticing that migrate_balanced_pgdat() in mm/migrate.c and page_reporting_process_zone() in mm/page_reporting.c use ALLOC_CMA but neither of these files appears to include page_alloc.h.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#13 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Brendan Jackman
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Spurious allocation failures for ALLOC_NOLOCK on fragmented systems --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Spurious allocation failures for ALLOC_NOLOCK on fragmented systems --

commit e9001ddaf8c952ebe08e5042f720b09e0b4a5da8
Author: Brendan Jackman <[email protected]>

mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

This commit unifies the core allocator entry points by pulling the ALLOC_NOLOCK control up into __alloc_frozen_pages_noprof(). It turns the nolock variant into a thin wrapper that just sets the ALLOC_NOLOCK flag and handles NUMA_NO_NODE.

Quoted 4 lines
mm/page_alloc.c +0 -0
index a3ba63c7f9199..8d409d075e3e9 100644

[ ... ]

mm/page_alloc.c struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, · line 5390

Quoted 10 lines
mm/page_alloc.c +2 -2
@@ -5310,9 +5384,9 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 	fastpath_alloc_flags |= alloc_flags_nofragment(zonelist_zone(ac.preferred_zoneref), gfp);
 	fastpath_alloc_flags |= alloc_flags_nonblocking(gfp, order) & ALLOC_HIGHATOMIC;
 
-	/* First allocation attempt */
+	/* First allocation attempt (or, for nolock, only attempt) */
 	page = get_page_from_freelist(alloc_gfp, order, fastpath_alloc_flags, &ac);
-	if (likely(page))
+	if (likely(page) || (alloc_flags & ALLOC_NOLOCK))
 		goto out;

[Severity: High]
Could this lead to spurious allocation failures for ALLOC_NOLOCK requests on fragmented systems?

In the pre-patch code, alloc_frozen_pages_nolock_noprof() bypassed alloc_flags_nofragment(), allowing it to successfully steal pages from other migratetypes when its preferred migratetype was empty.

The new unified __alloc_frozen_pages_noprof() fastpath unconditionally applies the ALLOC_NOFRAGMENT policy to fastpath_alloc_flags. When the requested migratetype is empty, __rmqueue() respects this flag by skipping __rmqueue_steal() and returning NULL.

Since normal allocations fall back to the slowpath (which clears ALLOC_NOFRAGMENT and retries), they succeed. However, ALLOC_NOLOCK explicitly bails out after the first attempt here, permanently failing the allocation. This might cause systems using bpf or tracing facilities under memory fragmentation to silently drop allocations when hit from atomic contexts.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
#63 lore raw
In reply to Mike Rapoport
Quoted 22 lines · Mike Rapoport

Hi Brendan,

Quoted 9 lines · Brendan Jackman

Some tweaks and cleanups for page allocator entrypoint and flags. This is motivated by preparation for __GFP_UNMAPPED [1] (which will probably become ALLOC_UNMAPPED in its next iteration), but all this is supposed to be an improvement to the codebase in its own right: unifying code paths, reducing API surface, and removing GFP flags.

Tested:

Quoted 8 lines

- KVM, mm, and BPF selftests in a QEMU VM

- kunit.py on x86_64

- For the ALLOC_NO_CODETAG bits I just booted a VM and read /proc/allocinfo. I confirmed that if I remove ALLOC_NO_CODETAG, the kernel crashes in early boot, so I was at least booting code that depends on this logic.

Quoted 7 lines

Heads up before the full kbuild report:

CI has tested the following submission:

Status:     FAILURE
Name:       [v3,00/16] mm: Some cleanups for page allocator APIs
Patchwork:  https://patchwork.kernel.org/project/linux-mm/list/?series=1118244&state=*
Matrix:     https://github.com/linux-mm/linux-mm/actions/runs/28375636866

Agh, thanks, I broke the build for CMA.

I thought I had this covered in my local test scripts. I will fix that first then I'll send a fixup for the patch.

#37 lore raw

Re: [PATCH v3 11/16] sgi-xp: Use higher-level allocator API

In reply to Brendan Jackman
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] xpc_init_mq_node() incorrectly passes the node ID (nid) instead of the loop variable (cpu) to xpc_create_gru_mq_uv(). --

commit 364bc05827d5201ce117b2c0d83f21a9b64f85f3
Author: Brendan Jackman <[email protected]>

sgi-xp: Use higher-level allocator API

This commit replaces __alloc_pages_node() with alloc_pages_node() in xpc_create_gru_mq_uv(). The __alloc_pages_node() function is being phased out, and alloc_pages_node() works identically here while allowing NUMA_NO_NODE if it were needed.

drivers/misc/sgi-xp/xpc_uv.c xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name, · line 172

Quoted 8 lines
drivers/misc/sgi-xp/xpc_uv.c +0 -0
index 772c787268932..aacff70204241 100644
@@ -170,7 +170,7 @@ xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name,
 	mq->mmr_blade = uv_cpu_to_blade_id(cpu);
 
 	nid = cpu_to_node(cpu);

[Severity: High]
This is a pre-existing issue, but while reviewing this code, I noticed a potential out-of-bounds access issue.

In xpc_init_mq_node(), the node ID (nid) is incorrectly passed instead of the loop variable (cpu) to xpc_create_gru_mq_uv():

drivers/misc/sgi-xp/xpc_uv.c:xpc_init_mq_node() {
    ...
		xpc_activate_mq_uv =
			xpc_create_gru_mq_uv(XPC_ACTIVATE_MQ_SIZE_UV, nid,
					     XPC_ACTIVATE_IRQ_NAME,
					     xpc_handle_activate_IRQ_uv);
    ...
}

Because xpc_create_gru_mq_uv() expects a valid CPU index, passing a NUMA node ID means the cpu parameter here contains a node ID. Since NUMA node IDs and CPU IDs are in different ID spaces, cpu_to_node() will access the numa_node per-CPU variable using a node ID instead of a CPU ID.

If the node ID is greater than or equal to nr_cpu_ids, will this result in an out-of-bounds access causing a kernel panic during module load?

Even if it is within bounds, does this result in routing interrupts and allocating memory using the topology of the wrong CPU?

Quoted 5 lines
drivers/misc/sgi-xp/xpc_uv.c +1 -1
-	page = __alloc_pages_node(nid,
+	page = alloc_pages_node(nid,
 				      GFP_KERNEL | __GFP_ZERO | __GFP_THISNODE,
 				      pg_order);
 	if (page == NULL) {
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
#64 lore raw
In reply to Brendan Jackman
Quoted 32 lines · Brendan Jackman
Quoted 22 lines · Mike Rapoport

Hi Brendan,

Quoted 9 lines · Brendan Jackman

Some tweaks and cleanups for page allocator entrypoint and flags. This is motivated by preparation for __GFP_UNMAPPED [1] (which will probably become ALLOC_UNMAPPED in its next iteration), but all this is supposed to be an improvement to the codebase in its own right: unifying code paths, reducing API surface, and removing GFP flags.

Tested:

Quoted 8 lines

- KVM, mm, and BPF selftests in a QEMU VM

- kunit.py on x86_64

- For the ALLOC_NO_CODETAG bits I just booted a VM and read /proc/allocinfo. I confirmed that if I remove ALLOC_NO_CODETAG, the kernel crashes in early boot, so I was at least booting code that depends on this logic.

Quoted 7 lines

Heads up before the full kbuild report:

CI has tested the following submission:

Status:     FAILURE
Name:       [v3,00/16] mm: Some cleanups for page allocator APIs
Patchwork:  https://patchwork.kernel.org/project/linux-mm/list/?series=1118244&state=*
Matrix:     https://github.com/linux-mm/linux-mm/actions/runs/28375636866
Quoted 4 lines

Agh, thanks, I broke the build for CMA.

I thought I had this covered in my local test scripts. I will fix that first then I'll send a fixup for the patch.

OK my scripts are indeed checking CMA, the issue is that I didn't build with NUMA_BALANCING.

I guess Suren was right[0] and I really should build allmodconfig (at least vmlinux) before sending patches. I was a bit skeptical that this was an especially useful config to build, but now I realise it just maximises coverage, even if it does so under a pretty arbitrary configuration.

[0] https://lore.kernel.org/all/CAJuCfpHAMaK2sZYSgS750CvgksCKEfOmLzZasXdBup+CrS-3Pg@mail.gmail.com/

Here are the fixes:

mm/migrate.c +1 -0
index d9b23909d716c..8e0a6fb3f6618 100644
@@ -49,6 +49,7 @@
 #include <trace/events/migrate.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 #include "swap.h"
 
 static const struct movable_operations *offline_movable_ops;
mm/page_reporting.c +1 -0
index 7418f2e500bb4..c7325704c3202 100644
@@ -8,6 +8,7 @@
 #include <linux/delay.h>
 #include <linux/scatterlist.h>
 
+#include "page_alloc.h"
 #include "page_reporting.h"
 #include "internal.h"
 
mm/shuffle.c +1 -0
index fb1393b8b3a9d..82a2c7725a08a 100644
@@ -7,6 +7,7 @@
 #include <linux/random.h>
 #include <linux/moduleparam.h>
 #include "internal.h"
+#include "page_alloc.h"
 #include "shuffle.h"
 
 DEFINE_STATIC_KEY_FALSE(page_alloc_shuffle_key);
#44 lore raw

Re: [PATCH v3 13/16] mm: Remove __alloc_pages_node()

In reply to Brendan Jackman
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Incomplete API refactoring leaves __folio_alloc_node() and its external callers vulnerable to the CPU hotplug hazard the patch intends to fix. --

commit 1c3f10897a9f9fbcd231ced2981c00e238f22315
Author: Brendan Jackman <[email protected]>

mm: Remove __alloc_pages_node()

This commit removes the __alloc_pages_node() API to reduce the API surface and prevent issues where callers using cpu_to_node() pass NUMA_NO_NODE during CPU hotplug, leading to memory corruption or panics.

Quoted 4 lines
include/linux/gfp.h +0 -0
index 01d6d2591f49e..3bf55a5f9143e 100644

[ ... ]

Quoted 12 lines
include/linux/gfp.h +0 -8
@@ -256,21 +256,6 @@ static inline void warn_if_node_offline(int this_node, gfp_t gfp_mask)
 	dump_stack();
 }
 
-/*
- * Allocate pages, preferring the node given as nid. The node must be valid and
- * online. For more general interface, see alloc_pages_node().
- */
-static inline struct page *
-__alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order)
-{
-	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);

include/linux/gfp.h static inline void warn_if_node_offline(int this_node, gfp_t gfp_mask) · line 261

Quoted 10 lines
include/linux/gfp.h +0 -7
-	warn_if_node_offline(nid, gfp_mask);
-
-	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
-}
-
-#define  __alloc_pages_node(...)		alloc_hooks(__alloc_pages_node_noprof(__VA_ARGS__))
-
 static inline
 struct folio *__folio_alloc_node_noprof(gfp_t gfp, unsigned int order, int nid)
 {

[Severity: High]
This is a pre-existing issue, but does __folio_alloc_node_noprof() and its wrappers need the same treatment?

I noticed it still lacks the NUMA_NO_NODE fallback, which can be triggered by existing external callers. For instance, in net/sunrpc/svc.c, svc_pool_map_get_node() uses cpu_to_node(), which can return NUMA_NO_NODE (-1) during CPU hotplug:

net/sunrpc/svc.c:svc_pool_map_get_node() {
    ...
    if (m->mode == SVC_POOL_PERCPU)
        return cpu_to_node(m->pool_to[pidx]);
    ...
}

This returned node ID is then passed directly to __folio_alloc_node() in svc_prepare_thread():

net/sunrpc/svc.c:svc_prepare_thread() {
    ...
    rqstp->rq_scratch_folio = __folio_alloc_node(GFP_KERNEL, 0, node);
    if (!rqstp->rq_scratch_folio)
        goto out_enomem;
    ...
}

Could this trigger the same VM_BUG_ON(nid < 0) or out-of-bounds array access that this commit aims to resolve?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
#14 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to [email protected]
Quoted 37 lines · On Mon, 29 Jun 2026 at 16:29, <[email protected]> wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Spurious allocation failures for ALLOC_NOLOCK on fragmented systems --

commit e9001ddaf8c952ebe08e5042f720b09e0b4a5da8
Author: Brendan Jackman <[email protected]>

mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

This commit unifies the core allocator entry points by pulling the ALLOC_NOLOCK control up into __alloc_frozen_pages_noprof(). It turns the nolock variant into a thin wrapper that just sets the ALLOC_NOLOCK flag and handles NUMA_NO_NODE.

Quoted 4 lines
mm/page_alloc.c +0 -0
index a3ba63c7f9199..8d409d075e3e9 100644

[ ... ]

mm/page_alloc.c struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, · line 5390

Quoted 10 lines
mm/page_alloc.c +2 -2
@@ -5310,9 +5384,9 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
      fastpath_alloc_flags |= alloc_flags_nofragment(zonelist_zone(ac.preferred_zoneref), gfp);
      fastpath_alloc_flags |= alloc_flags_nonblocking(gfp, order) & ALLOC_HIGHATOMIC;
 
-     /* First allocation attempt */
+     /* First allocation attempt (or, for nolock, only attempt) */
      page = get_page_from_freelist(alloc_gfp, order, fastpath_alloc_flags, &ac);
-     if (likely(page))
+     if (likely(page) || (alloc_flags & ALLOC_NOLOCK))
              goto out;

[Severity: High]
Could this lead to spurious allocation failures for ALLOC_NOLOCK requests on fragmented systems?

Quoted 3 lines

In the pre-patch code, alloc_frozen_pages_nolock_noprof() bypassed alloc_flags_nofragment(), allowing it to successfully steal pages from other migratetypes when its preferred migratetype was empty.

Oh, yep. Given "no functional change intended" this should not set ALLOC_NOFRAGMENT when ac.alloc_flags has ALLOC_NOLOCK.

(Is this intentional? Do we really want to allow "opportunistic" allocations, which are very much allowed to fail, to fragment memory? It doesn't look like this was discussed at when the nolock functionality was added).

Reviewed-by Suren Baghdasaryan on v3 9/16
#32 lore raw

RE: -EXT-[PATCH v3 09/16] KVM: VMX: Use higher-level allocator API

In reply to Brendan Jackman

Can someone please remove me from this list. I've requested a couple of times.

Thanks,
David

-----Original Message-----
From: Brendan Jackman <[email protected]>
Sent: Monday, June 29, 2026 7:12 AM
To: Andrew Morton <[email protected]>; Vlastimil Babka <[email protected]>; Suren Baghdasaryan <[email protected]>; Michal Hocko <[email protected]>; Johannes Weiner <[email protected]>; Zi Yan <[email protected]>; Muchun Song <[email protected]>; Oscar Salvador <[email protected]>; David Hildenbrand <[email protected]>; Lorenzo Stoakes <[email protected]>; Liam R. Howlett <[email protected]>; Mike Rapoport <[email protected]>; Matthew Brost <[email protected]>; Joshua Hahn <[email protected]>; Rakie Kim <[email protected]>; Byungchul Park <[email protected]>; Ying Huang <[email protected]>; Alistair Popple <[email protected]>; Hao Li <[email protected]>; Christoph Lameter <[email protected]>; David Rientjes <[email protected]>; Roman Gushchin <[email protected]>; Sebastian Andrzej Siewior <[email protected]>; Clark Williams <[email protected]>; Steven Rostedt <[email protected]>
Cc: Harry Yoo (Oracle) <[email protected]>; Gregory Price <[email protected]>; Johannes Weiner <[email protected]>; Alexei Starovoitov <[email protected]>; Matthew Wilcox <[email protected]>; Hao Ge <[email protected]>; [email protected]; [email protected]; [email protected]; Brendan Jackman <[email protected]>; Sean Christopherson <[email protected]>; Paolo Bonzini <[email protected]>; [email protected]
Subject: -EXT-[PATCH v3 09/16] KVM: VMX: Use higher-level allocator API

WARNING: This message is from an external source. Evaluate the message carefully BEFORE clicking on links or opening attachments.

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Cc Sean Christopherson <[email protected]>
Cc Paolo Bonzini <[email protected]>
Assisted-by Gemini:unknown-version
---
 arch/x86/kvm/vmx/vmx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -3028,7 +3028,7 @@ struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
        struct page *pages;
        struct vmcs *vmcs;
 
-       pages = __alloc_pages_node(node, flags, 0);
+       pages = alloc_pages_node(node, flags, 0);
        if (!pages)
                return NULL;
        vmcs = page_address(pages);

--
2.54.0

#49 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to Brendan Jackman
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Low] The fallback #else definition of pgalloc_tag_add was not updated to accept unsigned int alloc_flags.

Pre-existing issues:
- [Critical] TOCTOU race in __alloc_tag_add_early_pfn leading to an out-of-bounds array write. --

commit ccee7f6d4bcf9d00da38c878e97dd852c5e649b3
Author: Brendan Jackman <[email protected]>

mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

This commit replaces the mm-internal __GFP_NO_CODETAG flag with a new alloc_flags bit, ALLOC_NO_CODETAG. It plumbs alloc_flags through several allocator functions to allow passing this flag directly to the alloc_tag code.

Quoted 4 lines
mm/alloc_tag.c +0 -0
index d9be1cf5187d9..a32a94e759b94 100644

[ ... ]

mm/alloc_tag.c static void __init __alloc_tag_add_early_pfn(unsigned long pfn) · line 800

Quoted 8 lines
mm/alloc_tag.c +2 -1
@@ -806,7 +795,8 @@ static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
 	do {
 		pool = READ_ONCE(current_pfn_pool);
 		if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE) {
-			struct page *new_page = alloc_page(__GFP_HIGH | __GFP_NO_CODETAG);
+			struct page *new_page = __alloc_pages(__GFP_HIGH, 0, numa_mem_id(),
+							      NULL, ALLOC_NO_CODETAG);
 			struct pfn_pool *new;

[Severity: Critical]
This is a pre-existing issue, but is there a time-of-check to time-of-use race condition here?

If multiple CPUs are active during the deferred init window, could they concurrently observe the same remaining capacity?

CPU1:

    if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE)

CPU2:

    if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE)

If both see 1 remaining slot, won't they both proceed to call atomic_fetch_add() on the pool counter further down in this function? This could cause an out-of-bounds array write at pool->pfns[PFN_POOL_SIZE].

[ ... ]

Quoted 4 lines
mm/page_alloc.c +0 -0
index 026f33f217036..803b32e5a5e47 100644

[ ... ]

Quoted 5 lines
mm/page_alloc.c +0 -0
@@ -1263,17 +1263,17 @@ void __pgalloc_tag_add(struct page *page, struct task_struct *task,
 		alloc_tag_add_early_pfn(page_to_pfn(page), alloc_flags);
 		if (task->alloc_tag)
 			alloc_tag_set_inaccurate(task->alloc_tag);
 	}

mm/page_alloc.c void __pgalloc_tag_add(struct page *page, struct task_struct *task, · line 1274

Quoted 10 lines
mm/page_alloc.c +2 -2
 }
 
 static inline void pgalloc_tag_add(struct page *page, struct task_struct *task,
-				   unsigned int nr, gfp_t gfp_flags)
+				   unsigned int nr, unsigned int alloc_flags)
 {
 	if (mem_alloc_profiling_enabled())
-		__pgalloc_tag_add(page, task, nr, gfp_flags);
+		__pgalloc_tag_add(page, task, nr, alloc_flags);
 }

[Severity: Low]
Does this change miss updating the fallback definition of pgalloc_tag_add() when memory allocation profiling is disabled?

If the kernel is compiled without mem-alloc profiling, the dummy function signature appears to still use the old type:

static inline void pgalloc_tag_add(struct page *page, struct task_struct *task,

                   unsigned int nr, gfp_t gfp_flags) {}

Since callers like post_alloc_hook() are now passing an unsigned int for alloc_flags, won't this cause a Sparse type mismatch warning because gfp_t is a restricted bitwise type?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
#58 lore raw

Re: [PATCH v3 16/16] mm: remove the __GFP_NO_OBJ_EXT flag

In reply to Brendan Jackman
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Incomplete removal of `__GFP_NO_OBJ_EXT` flag leaves the bit reserved in the core kernel. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Incomplete removal of __GFP_NO_OBJ_EXT flag leaves the bit reserved in the core kernel. --

commit d2006cc547e302003b4c27943f81bc40cdd90223
Author: Brendan Jackman <[email protected]>

mm: remove the __GFP_NO_OBJ_EXT flag

This commit removes the __GFP_NO_OBJ_EXT flag from the tools copy of gfp_types.h. Its previous users have been converted to use SLAB_ALLOC_NO_RECURSE or ALLOC_NO_CODETAG instead.

tools/include/linux/gfp_types.h enum { · line 58

Quoted 9 lines
tools/include/linux/gfp_types.h +0 -1
index 6c75df30a281d..a93b8bd200b76 100644
@@ -55,7 +55,6 @@ enum {
 #ifdef CONFIG_LOCKDEP
 	___GFP_NOLOCKDEP_BIT,
 #endif
-	___GFP_NO_OBJ_EXT_BIT,

[Severity: Medium]
Is it possible that the core kernel header updates were missed? The changes in this patch only modify the tools copy of the header, which causes it to diverge from the actual kernel definitions.

Looking at include/linux/gfp_types.h, ___GFP_NO_OBJ_EXT_BIT remains reserved in the core kernel. Additionally, the trace event mapping TRACE_GFP_EM(NO_OBJ_EXT) is left behind as dead code in include/trace/events/mmflags.h.

Could the removal be applied to those files as well so the bit is fully freed in the core kernel page allocator as described in the commit message?

Quoted 2 lines
tools/include/linux/gfp_types.h +0 -0
 	___GFP_LAST_BIT
 };
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
Acked-by on v3 11/16
#38 lore raw

Re: [PATCH v3 11/16] sgi-xp: Use higher-level allocator API

In reply to Brendan Jackman
Quoted 36 lines · Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Cc Robin Holt <[email protected]>
Cc Steve Wahl <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Assisted-by Gemini:unknown-model
---
 drivers/misc/sgi-xp/xpc_uv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/sgi-xp/xpc_uv.c +1 -1
index 772c787268932..aacff70204241 100644
@@ -170,7 +170,7 @@ xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name,
 	mq->mmr_blade = uv_cpu_to_blade_id(cpu);
 
 	nid = cpu_to_node(cpu);
-	page = __alloc_pages_node(nid,
+	page = alloc_pages_node(nid,
 				      GFP_KERNEL | __GFP_ZERO | __GFP_THISNODE,
 				      pg_order);
 	if (page == NULL) {
Signature · 2 lines
-- 
2.54.0
Signature · 1 line
Steve Wahl, Hewlett Packard Enterprise
reviewer
#51 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to Brendan Jackman

Hi Brendan

Quoted 27 lines · Brendan Jackman

Now that alloc_pages has an entrypoint that allows passing alloc_flags, we can take advantage of this to start removing GFP flags that are only used for mm-internal stuff.

This requires also plumbing the alloc_flags into some more of the allocator code, in particular __alloc_pages[_noprof]() gets an alloc_flags arg to go along with its callees, and we now need to pass those flags deeper into the allocator so they can reach the alloc_tag code.

No functional change intended.

---
  mm/alloc_tag.c       | 22 ++++++----------------
  mm/compaction.c      |  4 ++--
  mm/internal.h        |  1 -
  mm/page_alloc.c      | 42 ++++++++++++++++++++++++------------------
  mm/page_alloc.h      | 17 +++++++++++++++--
  mm/page_frag_cache.c |  4 ++--
  6 files changed, 49 insertions(+), 41 deletions(-)
mm/alloc_tag.c +0 -0
index d9be1cf5187d9..a32a94e759b94 100644

mm/alloc_tag.c line 18

Quoted 4 lines
mm/alloc_tag.c +1 -0
@@ -15,6 +15,8 @@
  #include <linux/vmalloc.h>
  #include <linux/kmemleak.h>
 
+#include "internal.h"

Should we include page_alloc.h here, as we call __alloc_pages later in this file?

Quoted 35 lines
mm/alloc_tag.c +3 -14
+
  #define ALLOCINFO_FILE_NAME		"allocinfo"
  #define MODULE_ALLOC_TAG_VMAP_SIZE	(100000UL * sizeof(struct alloc_tag))
  #define SECTION_START(NAME)		(CODETAG_SECTION_START_PREFIX NAME)
@@ -783,19 +785,6 @@ struct pfn_pool {
 
  #define PFN_POOL_SIZE			((PAGE_SIZE - offsetof(struct pfn_pool, pfns)) / \
  					 sizeof(unsigned long))
-
-/*
- * Skip early PFN recording for a page allocation.  Reuses the
- * %__GFP_NO_OBJ_EXT bit.  Used by __alloc_tag_add_early_pfn() to avoid
- * recursion when allocating pages for the early PFN tracking list
- * itself.
- *
- * Codetags of the pages allocated with __GFP_NO_CODETAG should be
- * cleared (via clear_page_tag_ref()) before freeing the pages to prevent
- * alloc_tag_sub_check() from triggering a warning.
- */
-#define __GFP_NO_CODETAG		__GFP_NO_OBJ_EXT
-
  static struct pfn_pool *current_pfn_pool __initdata;
 
  static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
@@ -806,7 +795,8 @@ static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
  	do {
  		pool = READ_ONCE(current_pfn_pool);
  		if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE) {
-			struct page *new_page = alloc_page(__GFP_HIGH | __GFP_NO_CODETAG);
+			struct page *new_page = __alloc_pages(__GFP_HIGH, 0, numa_mem_id(),
+							      NULL, ALLOC_NO_CODETAG);
  			struct pfn_pool *new;
 
  			if (!new_page) {

mm/alloc_tag.c typedef void alloc_tag_add_func(unsigned long pfn); · line 830

Quoted 5 lines
mm/alloc_tag.c +1 -1
@@ -837,7 +827,7 @@ typedef void alloc_tag_add_func(unsigned long pfn);
  static alloc_tag_add_func __rcu *alloc_tag_add_early_pfn_ptr __refdata =
  	RCU_INITIALIZER(__alloc_tag_add_early_pfn);
 
-void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)
+void alloc_tag_add_early_pfn(unsigned long pfn, unsigned int alloc_flags)

alloc_tag_add_early_pfn() has three occurrences across the codebase:

1. Definition in mm/alloc_tag.c:830:

void alloc_tag_add_early_pfn(unsigned long pfn, unsigned int alloc_flags)

2. Declaration in include/linux/alloc_tag.h:166:

void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)

3. Static inline stub in include/linux/alloc_tag.h:170:

static inline void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags) {}

This patch updates the definition in alloc_tag.c to take unsigned int alloc_flags,

but the two declarations in alloc_tag.h are left with the old gfp_t gfp_flags signature

These should be updated to match.

Quoted 63 lines
mm/alloc_tag.c +1 -1
  {
  	alloc_tag_add_func *alloc_tag_add;
 
@@ -845,7 +835,7 @@ void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)
  		return;
 
  	/* Skip allocations for the tracking list itself to avoid recursion. */
-	if (gfp_flags & __GFP_NO_CODETAG)
+	if (alloc_flags & ALLOC_NO_CODETAG)
  		return;
 
  	rcu_read_lock();
mm/compaction.c +2 -2
index 7d80735502d9a..4b2318fad4eb5 100644
@@ -83,7 +83,7 @@ static inline bool is_via_compact_memory(int order) { return false; }
 
  static struct page *mark_allocated_noprof(struct page *page, unsigned int order, gfp_t gfp_flags)
  {
-	post_alloc_hook(page, order, __GFP_MOVABLE);
+	post_alloc_hook(page, order, __GFP_MOVABLE, ALLOC_DEFAULT);
  	set_page_refcounted(page);
  	return page;
  }
@@ -1851,7 +1851,7 @@ static struct folio *compaction_alloc_noprof(struct folio *src, unsigned long da
  	}
  	dst = (struct folio *)freepage;
 
-	post_alloc_hook(&dst->page, order, __GFP_MOVABLE);
+	post_alloc_hook(&dst->page, order, __GFP_MOVABLE, ALLOC_DEFAULT);
  	set_page_refcounted(&dst->page);
  	if (order)
  		prep_compound_page(&dst->page, order);
mm/internal.h +0 -1
index c22284f04fc9e..369c656c63fa8 100644
@@ -1237,7 +1237,6 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone,
  enum ttu_flags;
  struct tlbflush_unmap_batch;
 
-
  /*
   * only for MM internal work items which do not depend on
   * any allocations or locks which might depend on allocations
mm/page_alloc.c +1 -1
index 026f33f217036..803b32e5a5e47 100644
@@ -1249,7 +1249,7 @@ void __clear_page_tag_ref(struct page *page)
  /* Should be called only if mem_alloc_profiling_enabled() */
  static noinline
  void __pgalloc_tag_add(struct page *page, struct task_struct *task,
-		       unsigned int nr, gfp_t gfp_flags)
+		       unsigned int nr, unsigned int alloc_flags)
  {
  	union pgtag_ref_handle handle;
  	union codetag_ref ref;
@@ -1263,17 +1263,17 @@ void __pgalloc_tag_add(struct page *page, struct task_struct *task,
  		 * page_ext is not available yet, record the pfn so we can
  		 * clear the tag ref later when page_ext is initialized.
  		 */

mm/page_alloc.c void __pgalloc_tag_add(struct page *page, struct task_struct *task, · line 1273

Quoted 10 lines
mm/page_alloc.c +2 -2
-		alloc_tag_add_early_pfn(page_to_pfn(page), gfp_flags);
+		alloc_tag_add_early_pfn(page_to_pfn(page), alloc_flags);
  		if (task->alloc_tag)
  			alloc_tag_set_inaccurate(task->alloc_tag);
  	}
  }
 
  static inline void pgalloc_tag_add(struct page *page, struct task_struct *task,
-				   unsigned int nr, gfp_t gfp_flags)
+				   unsigned int nr, unsigned int alloc_flags)

Same situation as alloc_tag_add_early_pfn() — the #else stub

at mm/page_alloc.c:1309 still uses gfp_t gfp_flags instead of unsigned int

alloc_flags.

Quoted 138 lines
mm/page_alloc.c +21 -15
  {
  	if (mem_alloc_profiling_enabled())
-		__pgalloc_tag_add(page, task, nr, gfp_flags);
+		__pgalloc_tag_add(page, task, nr, alloc_flags);
  }
 
  /* Should be called only if mem_alloc_profiling_enabled() */
@@ -1810,7 +1810,7 @@ static inline bool should_skip_init(gfp_t flags)
  }
 
  inline void post_alloc_hook(struct page *page, unsigned int order,
-				gfp_t gfp_flags)
+				gfp_t gfp_flags, unsigned int alloc_flags)
  {
  	const bool zero_tags = gfp_flags & __GFP_ZEROTAGS;
  	bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags) &&
@@ -1861,13 +1861,13 @@ inline void post_alloc_hook(struct page *page, unsigned int order,
 
  	set_page_owner(page, order, gfp_flags);
  	page_table_check_alloc(page, order);
-	pgalloc_tag_add(page, current, 1 << order, gfp_flags);
+	pgalloc_tag_add(page, current, 1 << order, alloc_flags);
  }
 
  static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp_flags,
  							unsigned int alloc_flags)
  {
-	post_alloc_hook(page, order, gfp_flags);
+	post_alloc_hook(page, order, gfp_flags, alloc_flags);
 
  	if (order && (gfp_flags & __GFP_COMP))
  		prep_compound_page(page, order);
@@ -4791,8 +4791,12 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
  	 * The fast path uses conservative alloc_flags to succeed only until
  	 * kswapd needs to be woken up, and to avoid the cost of setting up
  	 * alloc_flags precisely. So we do that now.
+	 *
+	 * Can't just or alloc_flags if it contains WMARK bits, but those flags
+	 * shouldn't be set in ac->alloc_flags.
  	 */
-	alloc_flags = alloc_flags_slowpath(gfp_mask, order);
+	VM_WARN_ON(ac->alloc_flags & ALLOC_WMARK_MASK);
+	alloc_flags = ac->alloc_flags | alloc_flags_slowpath(gfp_mask, order);
 
  	/*
  	 * We need to recalculate the starting point for the zonelist iterator
@@ -4834,7 +4838,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
  	reserve_flags = __gfp_pfmemalloc_flags(gfp_mask);
  	if (reserve_flags)
  		alloc_flags = alloc_flags_cma(gfp_mask, reserve_flags) |
-					  (alloc_flags & ALLOC_KSWAPD);
+				ac->alloc_flags | (alloc_flags & ALLOC_KSWAPD);
 
  	/*
  	 * Reset the nodemask and zonelist iterators if memory policies can be
@@ -5236,7 +5240,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
  	return nr_populated;
 
  failed:
-	page = __alloc_pages_noprof(gfp, 0, preferred_nid, nodemask);
+	page = __alloc_pages_noprof(gfp, 0, preferred_nid, nodemask, ALLOC_DEFAULT);
  	if (page)
  		page_array[nr_populated++] = page;
  	goto out;
@@ -5344,11 +5348,13 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
  {
  	struct page *page;
  	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
-	struct alloc_context ac = { };
+	struct alloc_context ac = {
+		.alloc_flags = alloc_flags,
+	};
  	unsigned int fastpath_alloc_flags = alloc_flags;
 
  	/* Other flags could be supported later if needed. */
-	if (WARN_ON(alloc_flags & ~ALLOC_NOLOCK))
+	if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG)))
  		return NULL;
 
  	if (!alloc_order_allowed(gfp, order, alloc_flags))
@@ -5417,12 +5423,12 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
  EXPORT_SYMBOL(__alloc_frozen_pages_noprof);
 
  struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order,
-		int preferred_nid, nodemask_t *nodemask)
+		int preferred_nid, nodemask_t *nodemask, unsigned int alloc_flags)
  {
  	struct page *page;
 
  	page = __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask,
-					   ALLOC_DEFAULT);
+					   alloc_flags);
  	if (page)
  		set_page_refcounted(page);
  	return page;
@@ -5436,7 +5442,7 @@ struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order
  	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
  	warn_if_node_offline(nid, gfp_mask);
 
-	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
+	return __alloc_pages_noprof(gfp_mask, order, nid, NULL, ALLOC_DEFAULT);
  }
  EXPORT_SYMBOL(alloc_pages_node_noprof);
 
@@ -5444,7 +5450,7 @@ struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_
  		nodemask_t *nodemask)
  {
  	struct page *page = __alloc_pages_noprof(gfp | __GFP_COMP, order,
-					preferred_nid, nodemask);
+					preferred_nid, nodemask, ALLOC_DEFAULT);
  	return page_rmappable_folio(page);
  }
  EXPORT_SYMBOL(__folio_alloc_noprof);
@@ -7126,7 +7132,7 @@ static void split_free_frozen_pages(struct list_head *list, gfp_t gfp_mask)
  		list_for_each_entry_safe(page, next, &list[order], lru) {
  			int i;
 
-			post_alloc_hook(page, order, gfp_mask);
+			post_alloc_hook(page, order, gfp_mask, ALLOC_DEFAULT);
  			if (!order)
  				continue;
 
@@ -7331,7 +7337,7 @@ int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long end,
  		struct page *head = pfn_to_page(start);
 
  		check_new_pages(head, order);
-		prep_new_page(head, order, gfp_mask, 0);
+		prep_new_page(head, order, gfp_mask, ALLOC_DEFAULT);
  	} else {
  		ret = -EINVAL;
  		WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n",
mm/page_alloc.h +0 -0
index 2058cbdca56e7..2614bff6795b0 100644
@@ -49,6 +49,16 @@
  #define ALLOC_HIGHATOMIC	0x200 /* Allows access to MIGRATE_HIGHATOMIC */
  #define ALLOC_NOLOCK		0x400 /* Only use spin_trylock in allocation path */

mm/page_alloc.h line 60

Quoted 10 lines
mm/page_alloc.h +9 -0
  #define ALLOC_KSWAPD		0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */
+/*
+ * Skip early PFN recording for a page allocation.  Used by
+ * __alloc_tag_add_early_pfn() to avoid recursion when allocating pages for the
+ * early PFN tracking list itself.
+ *
+ * Codetags of the pages allocated with __GFP_NO_CODETAG should be
+ * cleared (via clear_page_tag_ref()) before freeing the pages to prevent
+ * alloc_tag_sub_check() from triggering a warning.
+ */

I originally wrote this lengthy comment because the logic lives inside alloc_tag.c.

I wanted to document all the context to avoid confusion when revisiting this code later on.

We've since replaced __GFP_NO_CODETAG with ALLOC_NO_CODETAG, a generic alloc_flags bit defined in page_alloc.h.

The original long comment is no longer accurate:

Given that, I suggest updating it to the following:

/*

 * Avoid alloc_tag recursion for internal allocations.

 * Callers must clear_page_tag_ref() before
 * freeing to avoid warnings from alloc_tag_sub_check().

 */

Thanks

Best Regards

Hao

Quoted 50 lines
mm/page_alloc.h +6 -2
+#define ALLOC_NO_CODETAG       0x1000
 
  /* Flags that allow allocations below the min watermark. */
  #define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM)
@@ -84,6 +94,8 @@ struct alloc_context {
  	 */
  	enum zone_type highest_zoneidx;
  	bool spread_dirty_pages;
+	/* Only flags that are global to the whole allocation go here. */
+	unsigned int alloc_flags;
  };
 
  /*
@@ -214,7 +226,8 @@ static inline struct page *pageblock_pfn_to_page(unsigned long start_pfn,
  extern void __free_pages_core(struct page *page, unsigned int order,
  		enum meminit_context context);
 
-void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags);
+void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags,
+		     unsigned int alloc_flags);
  extern bool free_pages_prepare(struct page *page, unsigned int order);
 
  extern int user_min_free_kbytes;
@@ -245,7 +258,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned
  void free_frozen_pages_nolock(struct page *page, unsigned int order);
 
  struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
-		nodemask_t *nodemask);
+		nodemask_t *nodemask, unsigned int alloc_flags);
  #define __alloc_pages(...)			alloc_hooks(__alloc_pages_noprof(__VA_ARGS__))
 
  extern void zone_pcp_reset(struct zone *zone);
mm/page_frag_cache.c +2 -2
index a1077cef3a791..e63efe78b7d4b 100644
@@ -57,10 +57,10 @@ static struct page *__page_frag_cache_refill(struct page_frag_cache *nc,
  	gfp_mask = (gfp_mask & ~__GFP_DIRECT_RECLAIM) |  __GFP_COMP |
  		   __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC;
  	page = __alloc_pages(gfp_mask, PAGE_FRAG_CACHE_MAX_ORDER,
-			     numa_mem_id(), NULL);
+			     numa_mem_id(), NULL, ALLOC_DEFAULT);
  #endif
  	if (unlikely(!page)) {
-		page = __alloc_pages(gfp, 0, numa_mem_id(), NULL);
+		page = __alloc_pages(gfp, 0, numa_mem_id(), NULL, ALLOC_DEFAULT);
  		order = 0;
  	}
 
reviewer
#50 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to [email protected]

CC Andrew and Suren

Quoted 40 lines · On 2026/6/29 23:56, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Low] The fallback #else definition of pgalloc_tag_add was not updated to accept unsigned int alloc_flags.

Pre-existing issues:
- [Critical] TOCTOU race in __alloc_tag_add_early_pfn leading to an out-of-bounds array write. --

commit ccee7f6d4bcf9d00da38c878e97dd852c5e649b3
Author: Brendan Jackman <[email protected]>

mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

This commit replaces the mm-internal __GFP_NO_CODETAG flag with a new alloc_flags bit, ALLOC_NO_CODETAG. It plumbs alloc_flags through several allocator functions to allow passing this flag directly to the alloc_tag code.

Quoted 4 lines
mm/alloc_tag.c +0 -0
index d9be1cf5187d9..a32a94e759b94 100644

[ ... ]

mm/alloc_tag.c static void __init __alloc_tag_add_early_pfn(unsigned long pfn) · line 800

Quoted 8 lines
mm/alloc_tag.c +2 -1
@@ -806,7 +795,8 @@ static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
  	do {
  		pool = READ_ONCE(current_pfn_pool);
  		if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE) {
-			struct page *new_page = alloc_page(__GFP_HIGH | __GFP_NO_CODETAG);
+			struct page *new_page = __alloc_pages(__GFP_HIGH, 0, numa_mem_id(),
+							      NULL, ALLOC_NO_CODETAG);
  			struct pfn_pool *new;

[Severity: Critical]
This is a pre-existing issue, but is there a time-of-check to time-of-use race condition here?

If multiple CPUs are active during the deferred init window, could they concurrently observe the same remaining capacity?

CPU1:

Quoted 8 lines
     if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE)

CPU2:

     if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE)

If both see 1 remaining slot, won't they both proceed to call atomic_fetch_add() on the pool counter further down in this function? This could cause an out-of-bounds array write at pool->pfns[PFN_POOL_SIZE].

It's interesting.

I didn't get this warning from Sashiko when I submitted the patch earlier,

I'm not sure what changed on its end to trigger it now.

No worries though, unless I've overlooked some detail,

I believe this is a false positive

Even if both CPUs see 1 remaining slot and both skip the pool-growing branch,

they still have to go through atomic_cmpxchg to actually claim it.

static void __init __alloc_tag_add_early_pfn(unsigned long pfn) {
    struct pfn_pool *pool;
    int idx;

    do {
        pool = READ_ONCE(current_pfn_pool);
        if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE) {
        .........
        }
        idx = atomic_read(&pool->count);
        if (idx >= PFN_POOL_SIZE)
            continue;
        if (atomic_cmpxchg(&pool->count, idx, idx + 1) == idx) break;
    } while (1);

    pool->pfns[idx] = pfn;
}

Only one wins, the other gets a CAS failure and loops back.

There's no path where both write to pfns[PFN_POOL_SIZE].

Given that, I think we can ignore this Sashiko report and

move forward with Brendan's patch.

Thanks

Best Regards

Hao

Quoted 35 lines

[ ... ]

Quoted 4 lines
mm/page_alloc.c +0 -0
index 026f33f217036..803b32e5a5e47 100644

[ ... ]

Quoted 5 lines
mm/page_alloc.c +0 -0
@@ -1263,17 +1263,17 @@ void __pgalloc_tag_add(struct page *page, struct task_struct *task,
  		alloc_tag_add_early_pfn(page_to_pfn(page), alloc_flags);
  		if (task->alloc_tag)
  			alloc_tag_set_inaccurate(task->alloc_tag);
  	}

mm/page_alloc.c void __pgalloc_tag_add(struct page *page, struct task_struct *task, · line 1274

Quoted 10 lines
mm/page_alloc.c +2 -2
  }
 
  static inline void pgalloc_tag_add(struct page *page, struct task_struct *task,
-				   unsigned int nr, gfp_t gfp_flags)
+				   unsigned int nr, unsigned int alloc_flags)
  {
  	if (mem_alloc_profiling_enabled())
-		__pgalloc_tag_add(page, task, nr, gfp_flags);
+		__pgalloc_tag_add(page, task, nr, alloc_flags);
  }

[Severity: Low]
Does this change miss updating the fallback definition of pgalloc_tag_add() when memory allocation profiling is disabled?

If the kernel is compiled without mem-alloc profiling, the dummy function signature appears to still use the old type:

static inline void pgalloc_tag_add(struct page *page, struct task_struct *task,

                    unsigned int nr, gfp_t gfp_flags) {}

Since callers like post_alloc_hook() are now passing an unsigned int for alloc_flags, won't this cause a Sparse type mismatch warning because gfp_t is a restricted bitwise type?

#59 lore raw

Re: [PATCH v3 16/16] mm: remove the __GFP_NO_OBJ_EXT flag

In reply to [email protected]
Quoted 28 lines · sashiko-bot

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Incomplete removal of __GFP_NO_OBJ_EXT flag leaves the bit reserved in the core kernel. --

commit d2006cc547e302003b4c27943f81bc40cdd90223
Author: Brendan Jackman <[email protected]>

mm: remove the __GFP_NO_OBJ_EXT flag

This commit removes the __GFP_NO_OBJ_EXT flag from the tools copy of gfp_types.h. Its previous users have been converted to use SLAB_ALLOC_NO_RECURSE or ALLOC_NO_CODETAG instead.

tools/include/linux/gfp_types.h enum { · line 58

Quoted 9 lines
tools/include/linux/gfp_types.h +0 -1
index 6c75df30a281d..a93b8bd200b76 100644
@@ -55,7 +55,6 @@ enum {
 #ifdef CONFIG_LOCKDEP
 	___GFP_NOLOCKDEP_BIT,
 #endif
-	___GFP_NO_OBJ_EXT_BIT,

[Severity: Medium]
Is it possible that the core kernel header updates were missed? The changes in this patch only modify the tools copy of the header, which causes it to diverge from the actual kernel definitions.

Quoted 2 lines

Looking at include/linux/gfp_types.h, ___GFP_NO_OBJ_EXT_BIT remains reserved in the core kernel.

LOL, Sashiko is correct, I did not actually delete __GFP_NO_OBJ_EXT in this patch.

#52 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to Hao Ge
Quoted 34 lines · Hao Ge

Hi Brendan

Quoted 30 lines · Brendan Jackman

Now that alloc_pages has an entrypoint that allows passing alloc_flags, we can take advantage of this to start removing GFP flags that are only used for mm-internal stuff.

This requires also plumbing the alloc_flags into some more of the allocator code, in particular __alloc_pages[_noprof]() gets an alloc_flags arg to go along with its callees, and we now need to pass those flags deeper into the allocator so they can reach the alloc_tag code.

No functional change intended.

---
  mm/alloc_tag.c       | 22 ++++++----------------
  mm/compaction.c      |  4 ++--
  mm/internal.h        |  1 -
  mm/page_alloc.c      | 42 ++++++++++++++++++++++++------------------
  mm/page_alloc.h      | 17 +++++++++++++++--
  mm/page_frag_cache.c |  4 ++--
  6 files changed, 49 insertions(+), 41 deletions(-)
mm/alloc_tag.c +0 -0
index d9be1cf5187d9..a32a94e759b94 100644
@@ -15,6 +15,8 @@
  #include <linux/vmalloc.h>
  #include <linux/kmemleak.h>
 
Quoted 5 lines

mm/alloc_tag.c line 18

Quoted 1 line
mm/alloc_tag.c +1 -0
+#include "internal.h"

Should we include page_alloc.h here, as we call __alloc_pages later in this file?

Yeah, there are a few build failures due to me not doing a broad enough build. From now on I will just wait for allmodconfig instead of trying to be clever with my build tests, sorry about this.

Also, this suggests that I have not actually re-tested the alloc_tag code since v3 so I must repeat the test described in my cover letter (I just manually enable the feature and check the kernel boots) for v4.

Quoted 57 lines
Quoted 35 lines
mm/alloc_tag.c +3 -14
+
  #define ALLOCINFO_FILE_NAME		"allocinfo"
  #define MODULE_ALLOC_TAG_VMAP_SIZE	(100000UL * sizeof(struct alloc_tag))
  #define SECTION_START(NAME)		(CODETAG_SECTION_START_PREFIX NAME)
@@ -783,19 +785,6 @@ struct pfn_pool {
 
  #define PFN_POOL_SIZE			((PAGE_SIZE - offsetof(struct pfn_pool, pfns)) / \
  					 sizeof(unsigned long))
-
-/*
- * Skip early PFN recording for a page allocation.  Reuses the
- * %__GFP_NO_OBJ_EXT bit.  Used by __alloc_tag_add_early_pfn() to avoid
- * recursion when allocating pages for the early PFN tracking list
- * itself.
- *
- * Codetags of the pages allocated with __GFP_NO_CODETAG should be
- * cleared (via clear_page_tag_ref()) before freeing the pages to prevent
- * alloc_tag_sub_check() from triggering a warning.
- */
-#define __GFP_NO_CODETAG		__GFP_NO_OBJ_EXT
-
  static struct pfn_pool *current_pfn_pool __initdata;
 
  static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
@@ -806,7 +795,8 @@ static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
  	do {
  		pool = READ_ONCE(current_pfn_pool);
  		if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE) {
-			struct page *new_page = alloc_page(__GFP_HIGH | __GFP_NO_CODETAG);
+			struct page *new_page = __alloc_pages(__GFP_HIGH, 0, numa_mem_id(),
+							      NULL, ALLOC_NO_CODETAG);
  			struct pfn_pool *new;
 
  			if (!new_page) {

mm/alloc_tag.c typedef void alloc_tag_add_func(unsigned long pfn); · line 830

Quoted 5 lines
mm/alloc_tag.c +1 -1
@@ -837,7 +827,7 @@ typedef void alloc_tag_add_func(unsigned long pfn);
  static alloc_tag_add_func __rcu *alloc_tag_add_early_pfn_ptr __refdata =
  	RCU_INITIALIZER(__alloc_tag_add_early_pfn);
 
-void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)
+void alloc_tag_add_early_pfn(unsigned long pfn, unsigned int alloc_flags)

alloc_tag_add_early_pfn() has three occurrences across the codebase:

1. Definition in mm/alloc_tag.c:830:

void alloc_tag_add_early_pfn(unsigned long pfn, unsigned int alloc_flags)

2. Declaration in include/linux/alloc_tag.h:166:

void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)

3. Static inline stub in include/linux/alloc_tag.h:170:

static inline void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags) {}

Quoted 7 lines

This patch updates the definition in alloc_tag.c to take unsigned int alloc_flags,

but the two declarations in alloc_tag.h are left with the old gfp_t gfp_flags signature

These should be updated to match.

Yeah ditto, sorry about this and thanks for the review.

#55 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to Hao Ge
Quoted 24 lines · Hao Ge
Quoted 9 lines

+/*
+ * Skip early PFN recording for a page allocation. Used by
+ * __alloc_tag_add_early_pfn() to avoid recursion when allocating pages for the
+ * early PFN tracking list itself.
+ *
+ * Codetags of the pages allocated with __GFP_NO_CODETAG should be
+ * cleared (via clear_page_tag_ref()) before freeing the pages to prevent
+ * alloc_tag_sub_check() from triggering a warning.
+ */

I originally wrote this lengthy comment because the logic lives inside alloc_tag.c.

I wanted to document all the context to avoid confusion when revisiting this code later on.

We've since replaced __GFP_NO_CODETAG with ALLOC_NO_CODETAG, a generic alloc_flags bit defined in page_alloc.h.

The original long comment is no longer accurate:

Given that, I suggest updating it to the following:

Quoted 8 lines

/*

 * Avoid alloc_tag recursion for internal allocations.

 * Callers must clear_page_tag_ref() before
 * freeing to avoid warnings from alloc_tag_sub_check().

 */

Thanks for the context, pasting this in for v4!

maintainer Reviewed-by on v3 1/16
#2 lore raw

Re: [PATCH v3 01/16] mm/page_alloc: rename ALLOC_TRYLOCK -> ALLOC_NOLOCK

In reply to Brendan Jackman
Quoted 10 lines · Brendan Jackman

It's confusing that the function is called "nolock" but the flag is called "trylock", align them.

The function's terminology is more visible and has more mindshare so use that.

maintainer Reviewed-by on v3 2/16
#4 lore raw

Re: [PATCH v3 02/16] mm/page_alloc: some renames to clarify alloc_flags scopes

In reply to Brendan Jackman
Quoted 7 lines · Brendan Jackman

It's pretty confusing that:

- The slowpath and fastpath have a totally distinct set of alloc_flags.

- gfp_to_alloc_flags() sounds generic but it only influences the slowpath.

Quoted 8 lines

Rename some variables to highlight which alloc_flags are fastpath-specific. Rename gfp_to_alloc_flags() to highlight that it's slowpath-specific.

gfp_to_alloc_flags_cma() and gfp_to_alloc_flags_nonblocking() currently have perfectly harmless names, but to keep the naming consistent also rename those to the alloc_flags_*() pattern (which already exists for alloc_flags_nofragment()).

How annoying that alloc_flags_nofragment() doesn't have gfp as the first parameter, unlike others.
Oh well, must resist too much OCD :)

Uh, more annoyingly, alloc_flags_cma() takes alloc_flags and returns augmented alloc flags, so there's stuff like

*alloc_flags = alloc_flags_cma(gfp_mask, *alloc_flags);

Since we're unifying, it could be make to work additively like others? Then:

*alloc_flags |= alloc_flags_cma(gfp_mask);

Quoted 1 line

Otherwise, LGTM.

maintainer Reviewed-by on v3 3/16
#8 lore raw

Re: [PATCH v3 03/16] mm: name some args in a function declaration

In reply to Brendan Jackman
Quoted 6 lines · Brendan Jackman

Checkpatch complains about this, a later patch will move the code, fix it so that checkpatch doesn't complain about that patch. Do it in a separate patch so the "move the code" patch is trivial to review using Git's diff colouring.

Quoted 20 lines
---
 mm/internal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/internal.h +2 -2
index 2237eee030cba..8ce59c5664497 100644
@@ -919,8 +919,8 @@ extern bool free_pages_prepare(struct page *page, unsigned int order);
 
 extern int user_min_free_kbytes;
 
-struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int nid,
-		nodemask_t *);
+struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, int nid,
+		nodemask_t *nodemask);
 #define __alloc_frozen_pages(...) \
 	alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__))
 void free_frozen_pages(struct page *page, unsigned int order);
maintainer
#15 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Brendan Jackman
Quoted 10 lines · Brendan Jackman

Currently the core allocator code is controlled by ALLOC_NOLOCK, but the main entry point function is significantly different from the normal __alloc_frozen_pages_nolock(), this is tiring when reading the code.

Plumb the ALLOC_NOLOCK control one layer up in the call stack: create an alloc_flags argument to __alloc_frozen_pages_nolock() (which is only exposed to mm/) and then turn the nolock variant into a thin wrapper that just sets that flag (as well as handling NUMA_NO_NODE, similar to how some of the wrappers in gfp.h do).

Quoted 5 lines

Rationale that this doesn't change anything:

1. Simple bits: A bunch of the nolock-specific handling is just moved to the new alloc_order_allowed(), alloc_trylock_allowed() and gfp_trylock.

Right.

Quoted 5 lines

2. __alloc_frozen_pages_noprof() has some extra logic that wasn't previously in the nolock variant:

a. Application of gfp_allowed_mask; this only affects early boot, and
only flags that affect the slowpath get changed here.

gfp_allowed_mask clears __GFP_RECLAIM, and that means now allocations with GFP_KERNEL during early boot would see gfpflags_allow_spinning() = false.

The helper is not used in in the page allocator, but used in memcg/stackdepot/page_owner.

Quoted 2 lines

b. Application of current_gfp_context() - also only affects the
slowpath

PF_MEMALLOC_PIN affects the fast path, but ALLOC_NOLOCK users won't be affected.

What about alloc_flags_nofragment/nonblocking()?

Quoted 2 lines

3. The slowpath itself: this is now just explicitly skipped under !ALLOC_TRYLOCK.

Right.

Quoted 8 lines

Ulterior motive: adding an alloc_flags arg to the allocator's mm-internal entrypoint can later be used to do more allocation customisation without needing to create new GFP flags.

While adding this flag to a bunch of places, create ALLOC_DEFAULT to avoid a mysterious literal 0 in most places.

alloc_frozen_pages_noprof() is defined above the alloc flags

The function is defined below the alloc flags, no?

Quoted 32 lines

so just leave that as a slightly messy
exception instead of trying to fully reorder mm/internal.h for that one case.

No functional change intended.

---
 mm/hugetlb.c    |   3 +-
 mm/mempolicy.c  |  10 ++--
 mm/page_alloc.c | 178 +++++++++++++++++++++++++++++---------------------------
 mm/page_alloc.h |   6 +-
 mm/slub.c       |   6 +-
 5 files changed, 108 insertions(+), 95 deletions(-)
mm/page_alloc.c +9 -0
index a3ba63c7f9199..8d409d075e3e9 100644
@@ -5271,24 +5271,98 @@ void free_pages_bulk(struct page **page_array, unsigned long nr_pages)
 	}
 }
 
+static inline bool alloc_trylock_allowed(void)
+{
+	/*
+	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
+	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
+	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
+	 * mark the task as the owner of another rt_spin_lock which will
+	 * confuse PI logic, so return immediately if called from hard IRQ or
+	 * NMI.

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5292

Quoted 10 lines
mm/page_alloc.c +10 -0
+	 *
+	 * Note, irqs_disabled() case is ok. This function can be called
+	 * from raw_spin_lock_irqsave region.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
+		return false;
+
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return false;

Except for deferred_pages_enabled(), it's not specific to the page allocator. SLUB has

	/*
	 * See the comment for the same check in
	 * alloc_frozen_pages_nolock_noprof()
	 */

... and repeats the same thing as above.

Perhaps let's factor it out into a helper
rather than trying not to forget to update the other place?

Quoted 6 lines
mm/page_alloc.c +6 -0
+	/* Bailout, since _deferred_grow_zone() needs to take a lock */
+	if (deferred_pages_enabled())
+		return false;
+
+	return true;
+}
Signature · 2 lines
Cheers,
Harry / Hyeonggon
maintainer Acked-by on v3 6/16
#25 lore raw

Re: [PATCH v3 06/16] mm/page_alloc: relax GFP WARN in nolock allocs

In reply to Brendan Jackman
Quoted 10 lines · Brendan Jackman

This WARN forbids setting other flags than __GFP_ACCOUNT but we unconditionally set the ones in gfp_nolock so they are certainly fine for the caller to set.

There are other GFP flags that are almost certainly fine to set here; Willy noted GFP_HIGHMEM, GFP_DMA, GFP_MOVABLE and GFP_HARDWALL. But, nolock allocation is rather special, so be conservative to try and ensure we have a chance to think carefully before nontrivial new usecases arise.

Quoted 5 lines

---

Signature · 2 lines
Cheers,
Harry / Hyeonggon
maintainer Reviewed-by on v3 4/16
#11 lore raw

Re: [PATCH v3 04/16] mm: Split out internal page_alloc.h

In reply to Brendan Jackman
Quoted 6 lines · Brendan Jackman

internal.h is a bit bloated, seems like time for a page_alloc.h.

Where it wasn't obvious, the heuristic for deciding what goes into this new header was "does it support/correspond to a definition in mm/page_alloc.c?"

Quoted 8 lines

Only need to include it from 15 .c files out of 164 so this does seem like a genuine reduction in scopes, which is nice. And there's no circular internal.h<->page_alloc.h dependency, so it seems worthwhile to split this up before that inevitably emerges!

Cool.

#16 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Harry Yoo
Quoted 24 lines · Harry Yoo
Quoted 10 lines · Brendan Jackman

Currently the core allocator code is controlled by ALLOC_NOLOCK, but the main entry point function is significantly different from the normal __alloc_frozen_pages_nolock(), this is tiring when reading the code.

Plumb the ALLOC_NOLOCK control one layer up in the call stack: create an alloc_flags argument to __alloc_frozen_pages_nolock() (which is only exposed to mm/) and then turn the nolock variant into a thin wrapper that just sets that flag (as well as handling NUMA_NO_NODE, similar to how some of the wrappers in gfp.h do).

Quoted 5 lines

Rationale that this doesn't change anything:

1. Simple bits: A bunch of the nolock-specific handling is just moved to the new alloc_order_allowed(), alloc_trylock_allowed() and gfp_trylock.

Right.

Quoted 3 lines

2. __alloc_frozen_pages_noprof() has some extra logic that wasn't previously in the nolock variant:

Quoted 6 lines
Quoted 2 lines

a. Application of gfp_allowed_mask; this only affects early boot, and
only flags that affect the slowpath get changed here.

gfp_allowed_mask clears __GFP_RECLAIM, and that means now allocations with GFP_KERNEL during early boot would see gfpflags_allow_spinning() = false.

Is it a problem though? non-nolock allocations were affected before (the masking existed for those already) and will be affected now the same, and _nolock() allocations don't pass __GFP_RECLAIM in the first place, so the masking can't affect them?

Quoted 8 lines

The helper is not used in in the page allocator, but used in memcg/stackdepot/page_owner.

Quoted 2 lines

b. Application of current_gfp_context() - also only affects the
slowpath

PF_MEMALLOC_PIN affects the fast path, but ALLOC_NOLOCK users won't be affected.

And it wouldn't be wrong if they were? It only clears __GFP_MOVABLE?

Quoted 1 line

What about alloc_flags_nofragment/nonblocking()?

ALLOC_NOFRAGMENT due to e.g. defrag_mode could be a problem indeed, if there's no slowpath. Make ALLOC_NOLOCK override it?

nonblocking() is probably fine?

Quoted 1 line
#22 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Brendan Jackman
Quoted 2 lines · Brendan Jackman

Currently the core allocator code is controlled by ALLOC_NOLOCK, but the main entry point function is significantly different from the normal

Let's mention it explicitly, alloc_frozen_pages_nolock_noprof().

Quoted 1 line

__alloc_frozen_pages_nolock(), this is tiring when reading the code.

You mean __alloc_frozen_pages_noprof()?

Quoted 3 lines

Plumb the ALLOC_NOLOCK control one layer up in the call stack: create an alloc_flags argument to __alloc_frozen_pages_nolock() (which is only

Again __alloc_frozen_pages_noprof()

Quoted 9 lines

exposed to mm/) and then turn the nolock variant into a thin wrapper that just sets that flag (as well as handling NUMA_NO_NODE, similar to how some of the wrappers in gfp.h do).

Rationale that this doesn't change anything:

1. Simple bits: A bunch of the nolock-specific handling is just moved to the new alloc_order_allowed(), alloc_trylock_allowed() and gfp_trylock.

Should be alloc_nolock_allowed() and gfp_nolock

Quoted 5 lines

2. __alloc_frozen_pages_noprof() has some extra logic that wasn't previously in the nolock variant:

a. Application of gfp_allowed_mask; this only affects early boot, and
only flags that affect the slowpath get changed here.

As discussed in reply to Harry, I'd mention the flags excluded by GFP_BOOT_MASK are not usable by _nolock() anyway.

Quoted 5 lines

b. Application of current_gfp_context() - also only affects the
slowpath

3. The slowpath itself: this is now just explicitly skipped under !ALLOC_TRYLOCK.

ALLOC_NOLOCK.

Quoted 5 lines

Ulterior motive: adding an alloc_flags arg to the allocator's mm-internal entrypoint can later be used to do more allocation customisation without needing to create new GFP flags.

Quoted 8 lines

While adding this flag to a bunch of places, create ALLOC_DEFAULT to avoid a mysterious literal 0 in most places.

alloc_frozen_pages_noprof()
is defined above the alloc flags so just leave that as a slightly messy exception instead of trying to fully reorder mm/internal.h for that one case.

This no longer applies in v3?

Quoted 45 lines

No functional change intended.

---
 mm/hugetlb.c    |   3 +-
 mm/mempolicy.c  |  10 ++--
 mm/page_alloc.c | 178 +++++++++++++++++++++++++++++---------------------------
 mm/page_alloc.h |   6 +-
 mm/slub.c       |   6 +-
 5 files changed, 108 insertions(+), 95 deletions(-)
mm/hugetlb.c +0 -0
index f7925624c4d2e..dfcfcfa4715bf 100644
mm/page_alloc.c +5 -10
index a3ba63c7f9199..8d409d075e3e9 100644
@@ -5222,7 +5222,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 		}
 		nr_account++;
 
-		prep_new_page(page, 0, gfp, 0);
+		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
 		set_page_refcounted(page);
 		page_array[nr_populated++] = page;
 	}
@@ -5271,24 +5271,98 @@ void free_pages_bulk(struct page **page_array, unsigned long nr_pages)
 	}
 }
 
-/*
- * This is the 'heart' of the zoned buddy allocator.
- */
-struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
-		int preferred_nid, nodemask_t *nodemask)
+static inline bool alloc_order_allowed(gfp_t gfp, unsigned int order,
+				       unsigned int alloc_flags)
 {
-	struct page *page;
-	unsigned int fastpath_alloc_flags = ALLOC_WMARK_LOW;
-	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
-	struct alloc_context ac = { };
+	if (alloc_flags & ALLOC_NOLOCK)
+		return pcp_allowed_order(order);

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5287

Quoted 10 lines
mm/page_alloc.c +4 -1
 
 	/*
 	 * There are several places where we assume that the order value is sane
 	 * so bail out early if the request is out of bound.
 	 */
-	if (WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp))
+	return !(WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp));
+}
+
+static inline bool alloc_trylock_allowed(void)

alloc_nolock_allowed()

Quoted 64 lines
mm/page_alloc.c +64 -0
+{
+	/*
+	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
+	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
+	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
+	 * mark the task as the owner of another rt_spin_lock which will
+	 * confuse PI logic, so return immediately if called from hard IRQ or
+	 * NMI.
+	 *
+	 * Note, irqs_disabled() case is ok. This function can be called
+	 * from raw_spin_lock_irqsave region.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
+		return false;
+
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return false;
+
+	/* Bailout, since _deferred_grow_zone() needs to take a lock */
+	if (deferred_pages_enabled())
+		return false;
+
+	return true;
+}
+
+/*
+ * GFP flags to set for ALLOC_NOLOCK i.e. alloc_pages_nolock().
+ *
+ * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed.
+ * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd
+ * is not safe in arbitrary context.
+ *
+ * These two are the conditions for gfpflags_allow_spinning() being true.
+ *
+ * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason
+ * to warn. Also warn would trigger printk() which is unsafe from
+ * various contexts. We cannot use printk_deferred_enter() to mitigate,
+ * since the running context is unknown.
+ *
+ * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below
+ * is safe in any context. Also zeroing the page is mandatory for
+ * BPF use cases.
+ *
+ * Though __GFP_NOMEMALLOC is not checked in the code path below,
+ * specify it here to highlight that alloc_pages_nolock()
+ * doesn't want to deplete reserves.
+ */
+static const gfp_t gfp_nolock = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC |
+				__GFP_COMP;
+
+/*
+ * This is the 'heart' of the zoned buddy allocator.
+ */
+struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
+		int preferred_nid, nodemask_t *nodemask, unsigned int alloc_flags)
+{
+	struct page *page;
+	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
+	struct alloc_context ac = { };
+	unsigned int fastpath_alloc_flags = alloc_flags;
+
+	/* Other flags could be supported later if needed. */
+	if (WARN_ON(alloc_flags & ~ALLOC_NOLOCK))

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5361

Quoted 10 lines
mm/page_alloc.c +8 -0
 		return NULL;
 
+	if (!alloc_order_allowed(gfp, order, alloc_flags))
+		return NULL;
+
+	if (alloc_flags & ALLOC_NOLOCK) {
+		VM_WARN_ON_ONCE(gfp & ~__GFP_ACCOUNT);
+		if (!alloc_trylock_allowed())
+			return NULL;
+		gfp |= gfp_nolock;

I think we could do a

		fastpath_alloc_flags |= ALLOC_WMARK_MIN;

to make it explicit, even though it's a no-op (the value is 0) and alloc_frozen_pages_nolock_noprof() didn't do it.

Quoted 143 lines
mm/page_alloc.c +11 -75
+	} else {
+		fastpath_alloc_flags |= ALLOC_WMARK_LOW;
+	}
+
 	gfp &= gfp_allowed_mask;
 	/*
 	 * Apply scoped allocation constraints. This is mainly about GFP_NOFS
@@ -5310,9 +5384,9 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 	fastpath_alloc_flags |= alloc_flags_nofragment(zonelist_zone(ac.preferred_zoneref), gfp);
 	fastpath_alloc_flags |= alloc_flags_nonblocking(gfp, order) & ALLOC_HIGHATOMIC;
 
-	/* First allocation attempt */
+	/* First allocation attempt (or, for nolock, only attempt) */
 	page = get_page_from_freelist(alloc_gfp, order, fastpath_alloc_flags, &ac);
-	if (likely(page))
+	if (likely(page) || (alloc_flags & ALLOC_NOLOCK))
 		goto out;
 
 	alloc_gfp = gfp;
@@ -5329,7 +5403,8 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 out:
 	if (memcg_kmem_online() && (gfp & __GFP_ACCOUNT) && page &&
 	    unlikely(__memcg_kmem_charge_page(page, gfp, order) != 0)) {
-		free_frozen_pages(page, order);
+		__free_frozen_pages(page, order,
+				    alloc_flags & ALLOC_NOLOCK ? FPI_TRYLOCK : 0);
 		page = NULL;
 	}
 
@@ -5345,7 +5420,8 @@ struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order,
 {
 	struct page *page;
 
-	page = __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask);
+	page = __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask,
+					   ALLOC_DEFAULT);
 	if (page)
 		set_page_refcounted(page);
 	return page;
@@ -7875,80 +7951,10 @@ static bool __free_unaccepted(struct page *page)
 
 struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order)
 {
-	/*
-	 * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed.
-	 * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd
-	 * is not safe in arbitrary context.
-	 *
-	 * These two are the conditions for gfpflags_allow_spinning() being true.
-	 *
-	 * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason
-	 * to warn. Also warn would trigger printk() which is unsafe from
-	 * various contexts. We cannot use printk_deferred_enter() to mitigate,
-	 * since the running context is unknown.
-	 *
-	 * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below
-	 * is safe in any context. Also zeroing the page is mandatory for
-	 * BPF use cases.
-	 *
-	 * Though __GFP_NOMEMALLOC is not checked in the code path below,
-	 * specify it here to highlight that alloc_pages_nolock()
-	 * doesn't want to deplete reserves.
-	 */
-	gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_COMP
-			| gfp_flags;
-	unsigned int alloc_flags = ALLOC_NOLOCK;
-	struct alloc_context ac = { };
-	struct page *page;
-
-	VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT);
-	/*
-	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
-	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
-	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
-	 * mark the task as the owner of another rt_spin_lock which will
-	 * confuse PI logic, so return immediately if called from hard IRQ or
-	 * NMI.
-	 *
-	 * Note, irqs_disabled() case is ok. This function can be called
-	 * from raw_spin_lock_irqsave region.
-	 */
-	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
-		return NULL;
-
-	/* On UP, spin_trylock() always succeeds even when it is locked */
-	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
-		return NULL;
-
-	if (!pcp_allowed_order(order))
-		return NULL;
-
-	/* Bailout, since _deferred_grow_zone() needs to take a lock */
-	if (deferred_pages_enabled())
-		return NULL;
-
 	if (nid == NUMA_NO_NODE)
 		nid = numa_node_id();
 
-	prepare_alloc_pages(alloc_gfp, order, nid, NULL, &ac,
-			    &alloc_gfp, &alloc_flags);
-
-	/*
-	 * Best effort allocation from percpu free list.
-	 * If it's empty attempt to spin_trylock zone->lock.
-	 */
-	page = get_page_from_freelist(alloc_gfp, order, alloc_flags, &ac);
-
-	/* Unlike regular alloc_pages() there is no __alloc_pages_slowpath(). */
-
-	if (memcg_kmem_online() && page && (gfp_flags & __GFP_ACCOUNT) &&
-	    unlikely(__memcg_kmem_charge_page(page, alloc_gfp, order) != 0)) {
-		__free_frozen_pages(page, order, FPI_TRYLOCK);
-		page = NULL;
-	}
-	trace_mm_page_alloc(page, order, alloc_gfp, ac.migratetype);
-	kmsan_alloc_page(page, order, alloc_gfp);
-	return page;
+	return __alloc_frozen_pages_noprof(gfp_flags, order, nid, NULL, ALLOC_NOLOCK);
 }
 /**
  * alloc_pages_nolock - opportunistic reentrant allocation from any context
mm/page_alloc.h +2 -1
index 3250d44f96457..e16f905f859a7 100644
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/types.h>
 
+#define ALLOC_DEFAULT		0
 /* The ALLOC_WMARK bits are used as an index to zone->watermark */
 #define ALLOC_WMARK_MIN		WMARK_MIN
 #define ALLOC_WMARK_LOW		WMARK_LOW
@@ -219,7 +220,7 @@ extern bool free_pages_prepare(struct page *page, unsigned int order);
 extern int user_min_free_kbytes;
 
 struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, int nid,
-		nodemask_t *nodemask);
+		nodemask_t *nodemask, unsigned int alloc_flags);
 #define __alloc_frozen_pages(...) \
 	alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__))
 void free_frozen_pages(struct page *page, unsigned int order);

mm/page_alloc.h struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order); · line 235

Quoted 6 lines
mm/page_alloc.h +2 -1
@@ -230,7 +231,8 @@ struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order);
 #else
 static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order)
 {
-	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL);
+	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL,
+					   0 /* ALLOC_DEFAULT */);

Can use ALLOC_DEFAULT now.

Quoted 3 lines
mm/page_alloc.h +0 -0
 }
 #endif
 
maintainer Reviewed-by on v3 6/16
#26 lore raw

Re: [PATCH v3 06/16] mm/page_alloc: relax GFP WARN in nolock allocs

In reply to Brendan Jackman
Quoted 10 lines · Brendan Jackman

This WARN forbids setting other flags than __GFP_ACCOUNT but we unconditionally set the ones in gfp_nolock so they are certainly fine for the caller to set.

There are other GFP flags that are almost certainly fine to set here; Willy noted GFP_HIGHMEM, GFP_DMA, GFP_MOVABLE and GFP_HARDWALL. But, nolock allocation is rather special, so be conservative to try and ensure we have a chance to think carefully before nontrivial new usecases arise.

Quoted 4 lines
Quoted 19 lines
---
 mm/page_alloc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/page_alloc.c +2 -1
index 8d409d075e3e9..9cb3f1665b41b 100644
@@ -5355,7 +5355,8 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 		return NULL;
 
 	if (alloc_flags & ALLOC_NOLOCK) {
-		VM_WARN_ON_ONCE(gfp & ~__GFP_ACCOUNT);
+		/* Certain other flags could be supported later if needed. */
+		VM_WARN_ON_ONCE(gfp & ~(__GFP_ACCOUNT | gfp_nolock));
 		if (!alloc_trylock_allowed())
 			return NULL;
 		gfp |= gfp_nolock;
maintainer Reviewed-by on v3 7/16
#28 lore raw

Re: [PATCH v3 07/16] mm: move some stuff to mm/page_alloc.h

In reply to Brendan Jackman
Quoted 7 lines · Brendan Jackman

Some of this stuff in the public header is only used internally so shrink the scope to avoid silently growing new users.

drain_local_pages() is still used from kernel/power/snapshot.c so that needs to stay behind.

Quoted 102 lines
---
 include/linux/gfp.h | 26 --------------------------
 mm/page_alloc.h     | 28 ++++++++++++++++++++++++++++
 mm/vmstat.c         |  1 +
 3 files changed, 29 insertions(+), 26 deletions(-)
include/linux/gfp.h +0 -26
index cdf95a9f0b87c..01d6d2591f49e 100644
@@ -17,28 +17,6 @@ struct mempolicy;
 #define __default_gfp(a,b,...) b
 #define default_gfp(...) __default_gfp(,##__VA_ARGS__,GFP_KERNEL)
 
-/* Convert GFP flags to their corresponding migrate type */
-#define GFP_MOVABLE_MASK (__GFP_RECLAIMABLE|__GFP_MOVABLE)
-#define GFP_MOVABLE_SHIFT 3
-
-static inline int gfp_migratetype(const gfp_t gfp_flags)
-{
-	VM_WARN_ON((gfp_flags & GFP_MOVABLE_MASK) == GFP_MOVABLE_MASK);
-	BUILD_BUG_ON((1UL << GFP_MOVABLE_SHIFT) != ___GFP_MOVABLE);
-	BUILD_BUG_ON((___GFP_MOVABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_MOVABLE);
-	BUILD_BUG_ON((___GFP_RECLAIMABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_RECLAIMABLE);
-	BUILD_BUG_ON(((___GFP_MOVABLE | ___GFP_RECLAIMABLE) >>
-		      GFP_MOVABLE_SHIFT) != MIGRATE_HIGHATOMIC);
-
-	if (unlikely(page_group_by_mobility_disabled))
-		return MIGRATE_UNMOVABLE;
-
-	/* Group based on mobility */
-	return (__force unsigned long)(gfp_flags & GFP_MOVABLE_MASK) >> GFP_MOVABLE_SHIFT;
-}
-#undef GFP_MOVABLE_MASK
-#undef GFP_MOVABLE_SHIFT
-
 static inline bool gfpflags_allow_blocking(const gfp_t gfp_flags)
 {
 	return !!(gfp_flags & __GFP_DIRECT_RECLAIM);
@@ -395,10 +373,6 @@ extern void free_pages(unsigned long addr, unsigned int order);
 #define __free_page(page) __free_pages((page), 0)
 #define free_page(addr) free_pages((addr), 0)
 
-void page_alloc_init_cpuhp(void);
-bool decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp);
-void drain_zone_pages(struct zone *zone, struct per_cpu_pages *pcp);
-void drain_all_pages(struct zone *zone);
 void drain_local_pages(struct zone *zone);
 
 void page_alloc_init_late(void);
mm/page_alloc.h +28 -0
index e16f905f859a7..af83764788b96 100644
@@ -266,6 +266,34 @@ static inline bool free_area_empty(struct free_area *area, int migratetype)
 	return list_empty(&area->free_list[migratetype]);
 }
 
+/* Convert GFP flags to their corresponding migrate type */
+#define GFP_MOVABLE_MASK (__GFP_RECLAIMABLE|__GFP_MOVABLE)
+#define GFP_MOVABLE_SHIFT 3
+
+static inline int gfp_migratetype(const gfp_t gfp_flags)
+{
+	VM_WARN_ON((gfp_flags & GFP_MOVABLE_MASK) == GFP_MOVABLE_MASK);
+	BUILD_BUG_ON((1UL << GFP_MOVABLE_SHIFT) != ___GFP_MOVABLE);
+	BUILD_BUG_ON((___GFP_MOVABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_MOVABLE);
+	BUILD_BUG_ON((___GFP_RECLAIMABLE >> GFP_MOVABLE_SHIFT) != MIGRATE_RECLAIMABLE);
+	BUILD_BUG_ON(((___GFP_MOVABLE | ___GFP_RECLAIMABLE) >>
+		      GFP_MOVABLE_SHIFT) != MIGRATE_HIGHATOMIC);
+
+	if (unlikely(page_group_by_mobility_disabled))
+		return MIGRATE_UNMOVABLE;
+
+	/* Group based on mobility */
+	return (__force unsigned long)(gfp_flags & GFP_MOVABLE_MASK) >> GFP_MOVABLE_SHIFT;
+}
+#undef GFP_MOVABLE_MASK
+#undef GFP_MOVABLE_SHIFT
+
+bool decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp);
+void drain_zone_pages(struct zone *zone, struct per_cpu_pages *pcp);
+void drain_all_pages(struct zone *zone);
+void drain_local_pages(struct zone *zone);
+
+void page_alloc_init_cpuhp(void);
 void page_alloc_sysctl_init(void);
 
 #endif /* __MM_PAGE_ALLOC_H */
mm/vmstat.c +1 -0
index 7b93fbf9af092..3b5cb1031f720 100644
@@ -30,6 +30,7 @@
 #include <linux/sched/isolation.h>
 
 #include "internal.h"
+#include "page_alloc.h"
 
 #ifdef CONFIG_PROC_FS
 #ifdef CONFIG_NUMA
#17 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Vlastimil Babka (SUSE)
Quoted 28 lines · Vlastimil Babka (SUSE)
Quoted 27 lines · Harry Yoo
Quoted 10 lines · Brendan Jackman

Currently the core allocator code is controlled by ALLOC_NOLOCK, but the main entry point function is significantly different from the normal __alloc_frozen_pages_nolock(), this is tiring when reading the code.

Plumb the ALLOC_NOLOCK control one layer up in the call stack: create an alloc_flags argument to __alloc_frozen_pages_nolock() (which is only exposed to mm/) and then turn the nolock variant into a thin wrapper that just sets that flag (as well as handling NUMA_NO_NODE, similar to how some of the wrappers in gfp.h do).

Quoted 5 lines

Rationale that this doesn't change anything:

1. Simple bits: A bunch of the nolock-specific handling is just moved to the new alloc_order_allowed(), alloc_trylock_allowed() and gfp_trylock.

Right.

Quoted 5 lines

2. __alloc_frozen_pages_noprof() has some extra logic that wasn't previously in the nolock variant:

a. Application of gfp_allowed_mask; this only affects early boot, and
only flags that affect the slowpath get changed here.

Quoted 8 lines
Quoted 3 lines

gfp_allowed_mask clears __GFP_RECLAIM, and that means now allocations with GFP_KERNEL during early boot would see gfpflags_allow_spinning() = false.

Is it a problem though? non-nolock allocations were affected before (the masking existed for those already) and will be affected now the same, and _nolock() allocations don't pass __GFP_RECLAIM in the first place, so the masking can't affect them?

This was my thinking too.

Quoted 9 lines
Quoted 8 lines

The helper is not used in in the page allocator, but used in memcg/stackdepot/page_owner.

Quoted 2 lines

b. Application of current_gfp_context() - also only affects the
slowpath

PF_MEMALLOC_PIN affects the fast path, but ALLOC_NOLOCK users won't be affected.

Quoted 6 lines

And it wouldn't be wrong if they were? It only clears __GFP_MOVABLE?

Quoted 1 line

What about alloc_flags_nofragment/nonblocking()?

ALLOC_NOFRAGMENT due to e.g. defrag_mode could be a problem indeed, if there's no slowpath. Make ALLOC_NOLOCK override it?

Yeah calling alloc_flags_nofragment() here is a bug in the patch, and Sashiko also complained:

https://lore.kernel.org/all/[email protected]/

Like I said in the reply to that thread I think maybe we _do_ want to set ALLOC_NOFRAGMENT for nolock allocations? But, that is a functional change, it doesn't belong in this series.

Quoted 1 line

nonblocking() is probably fine?

Yeah, I believe this is fine.

#19 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Harry Yoo
Quoted 10 lines · Harry Yoo
Quoted 8 lines

Ulterior motive: adding an alloc_flags arg to the allocator's mm-internal entrypoint can later be used to do more allocation customisation without needing to create new GFP flags.

While adding this flag to a bunch of places, create ALLOC_DEFAULT to avoid a mysterious literal 0 in most places.

alloc_frozen_pages_noprof() is defined above the alloc flags

The function is defined below the alloc flags, no?

Yep this paragraph is stale since I created mm/page_alloc.h, will remove it.

Quoted 51 lines
Quoted 32 lines

so just leave that as a slightly messy
exception instead of trying to fully reorder mm/internal.h for that one case.

No functional change intended.

---
 mm/hugetlb.c    |   3 +-
 mm/mempolicy.c  |  10 ++--
 mm/page_alloc.c | 178 +++++++++++++++++++++++++++++---------------------------
 mm/page_alloc.h |   6 +-
 mm/slub.c       |   6 +-
 5 files changed, 108 insertions(+), 95 deletions(-)
mm/page_alloc.c +9 -0
index a3ba63c7f9199..8d409d075e3e9 100644
@@ -5271,24 +5271,98 @@ void free_pages_bulk(struct page **page_array, unsigned long nr_pages)
 	}
 }
 
+static inline bool alloc_trylock_allowed(void)
+{
+	/*
+	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
+	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
+	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
+	 * mark the task as the owner of another rt_spin_lock which will
+	 * confuse PI logic, so return immediately if called from hard IRQ or
+	 * NMI.

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5292

Quoted 10 lines
mm/page_alloc.c +10 -0
+	 *
+	 * Note, irqs_disabled() case is ok. This function can be called
+	 * from raw_spin_lock_irqsave region.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
+		return false;
+
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return false;

Except for deferred_pages_enabled(), it's not specific to the page allocator. SLUB has

	/*
	 * See the comment for the same check in
	 * alloc_frozen_pages_nolock_noprof()
	 */
Quoted 4 lines

... and repeats the same thing as above.

Perhaps let's factor it out into a helper
rather than trying not to forget to update the other place?

Hm, not sure about this. I think I would say it's a "coincidence" that these two bits of code look the same? Like, page_alloc.c uses spin_trylock() so you can't do alloc_pages_nolock() from IRQ on PREEMPT_RT. slub.c ALSO uses spin_trylock(), so you ALSO can't use kmalloc_nolock() in those scenarios. But those are two different facts that just happen to be isomorphic? Putting them into a shared helper would kinda imply that these are part of a single system with inherently coupled constraints.

I dunno I'm being a bit of a ponderous philosopher there, I don't have particularly strong feelings. But I'd lean towards leaving this out of the patchset since the potential deduplication isn't really related to the other cleanups anyway.

#5 lore raw

Re: [PATCH v3 02/16] mm/page_alloc: some renames to clarify alloc_flags scopes

In reply to Vlastimil Babka (SUSE)
Quoted 21 lines · Vlastimil Babka (SUSE)
Quoted 7 lines · Brendan Jackman

It's pretty confusing that:

- The slowpath and fastpath have a totally distinct set of alloc_flags.

- gfp_to_alloc_flags() sounds generic but it only influences the slowpath.

Quoted 8 lines

Rename some variables to highlight which alloc_flags are fastpath-specific. Rename gfp_to_alloc_flags() to highlight that it's slowpath-specific.

gfp_to_alloc_flags_cma() and gfp_to_alloc_flags_nonblocking() currently have perfectly harmless names, but to keep the naming consistent also rename those to the alloc_flags_*() pattern (which already exists for alloc_flags_nofragment()).

How annoying that alloc_flags_nofragment() doesn't have gfp as the first parameter, unlike others.
Oh well, must resist too much OCD :)

Quoted 8 lines

Uh, more annoyingly, alloc_flags_cma() takes alloc_flags and returns augmented alloc flags, so there's stuff like

*alloc_flags = alloc_flags_cma(gfp_mask, *alloc_flags);

Since we're unifying, it could be make to work additively like others? Then:

*alloc_flags |= alloc_flags_cma(gfp_mask);

Sure, I can chuck this on as an extra patch.

#23 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Vlastimil Babka (SUSE)
Quoted 336 lines · Vlastimil Babka (SUSE)
Quoted 2 lines · Brendan Jackman

Currently the core allocator code is controlled by ALLOC_NOLOCK, but the main entry point function is significantly different from the normal

Let's mention it explicitly, alloc_frozen_pages_nolock_noprof().

Quoted 1 line

__alloc_frozen_pages_nolock(), this is tiring when reading the code.

You mean __alloc_frozen_pages_noprof()?

Quoted 3 lines

Plumb the ALLOC_NOLOCK control one layer up in the call stack: create an alloc_flags argument to __alloc_frozen_pages_nolock() (which is only

Again __alloc_frozen_pages_noprof()

Quoted 9 lines

exposed to mm/) and then turn the nolock variant into a thin wrapper that just sets that flag (as well as handling NUMA_NO_NODE, similar to how some of the wrappers in gfp.h do).

Rationale that this doesn't change anything:

1. Simple bits: A bunch of the nolock-specific handling is just moved to the new alloc_order_allowed(), alloc_trylock_allowed() and gfp_trylock.

Should be alloc_nolock_allowed() and gfp_nolock

Quoted 5 lines

2. __alloc_frozen_pages_noprof() has some extra logic that wasn't previously in the nolock variant:

a. Application of gfp_allowed_mask; this only affects early boot, and
only flags that affect the slowpath get changed here.

As discussed in reply to Harry, I'd mention the flags excluded by GFP_BOOT_MASK are not usable by _nolock() anyway.

Quoted 5 lines

b. Application of current_gfp_context() - also only affects the
slowpath

3. The slowpath itself: this is now just explicitly skipped under !ALLOC_TRYLOCK.

ALLOC_NOLOCK.

Quoted 5 lines

Ulterior motive: adding an alloc_flags arg to the allocator's mm-internal entrypoint can later be used to do more allocation customisation without needing to create new GFP flags.

Quoted 8 lines

While adding this flag to a bunch of places, create ALLOC_DEFAULT to avoid a mysterious literal 0 in most places.

alloc_frozen_pages_noprof()
is defined above the alloc flags so just leave that as a slightly messy exception instead of trying to fully reorder mm/internal.h for that one case.

This no longer applies in v3?

Quoted 45 lines

No functional change intended.

---
 mm/hugetlb.c    |   3 +-
 mm/mempolicy.c  |  10 ++--
 mm/page_alloc.c | 178 +++++++++++++++++++++++++++++---------------------------
 mm/page_alloc.h |   6 +-
 mm/slub.c       |   6 +-
 5 files changed, 108 insertions(+), 95 deletions(-)
mm/hugetlb.c +0 -0
index f7925624c4d2e..dfcfcfa4715bf 100644
mm/page_alloc.c +5 -10
index a3ba63c7f9199..8d409d075e3e9 100644
@@ -5222,7 +5222,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 		}
 		nr_account++;
 
-		prep_new_page(page, 0, gfp, 0);
+		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
 		set_page_refcounted(page);
 		page_array[nr_populated++] = page;
 	}
@@ -5271,24 +5271,98 @@ void free_pages_bulk(struct page **page_array, unsigned long nr_pages)
 	}
 }
 
-/*
- * This is the 'heart' of the zoned buddy allocator.
- */
-struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
-		int preferred_nid, nodemask_t *nodemask)
+static inline bool alloc_order_allowed(gfp_t gfp, unsigned int order,
+				       unsigned int alloc_flags)
 {
-	struct page *page;
-	unsigned int fastpath_alloc_flags = ALLOC_WMARK_LOW;
-	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
-	struct alloc_context ac = { };
+	if (alloc_flags & ALLOC_NOLOCK)
+		return pcp_allowed_order(order);

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5287

Quoted 10 lines
mm/page_alloc.c +4 -1
 
 	/*
 	 * There are several places where we assume that the order value is sane
 	 * so bail out early if the request is out of bound.
 	 */
-	if (WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp))
+	return !(WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp));
+}
+
+static inline bool alloc_trylock_allowed(void)

alloc_nolock_allowed()

Quoted 64 lines
mm/page_alloc.c +64 -0
+{
+	/*
+	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
+	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
+	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
+	 * mark the task as the owner of another rt_spin_lock which will
+	 * confuse PI logic, so return immediately if called from hard IRQ or
+	 * NMI.
+	 *
+	 * Note, irqs_disabled() case is ok. This function can be called
+	 * from raw_spin_lock_irqsave region.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
+		return false;
+
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return false;
+
+	/* Bailout, since _deferred_grow_zone() needs to take a lock */
+	if (deferred_pages_enabled())
+		return false;
+
+	return true;
+}
+
+/*
+ * GFP flags to set for ALLOC_NOLOCK i.e. alloc_pages_nolock().
+ *
+ * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed.
+ * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd
+ * is not safe in arbitrary context.
+ *
+ * These two are the conditions for gfpflags_allow_spinning() being true.
+ *
+ * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason
+ * to warn. Also warn would trigger printk() which is unsafe from
+ * various contexts. We cannot use printk_deferred_enter() to mitigate,
+ * since the running context is unknown.
+ *
+ * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below
+ * is safe in any context. Also zeroing the page is mandatory for
+ * BPF use cases.
+ *
+ * Though __GFP_NOMEMALLOC is not checked in the code path below,
+ * specify it here to highlight that alloc_pages_nolock()
+ * doesn't want to deplete reserves.
+ */
+static const gfp_t gfp_nolock = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC |
+				__GFP_COMP;
+
+/*
+ * This is the 'heart' of the zoned buddy allocator.
+ */
+struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
+		int preferred_nid, nodemask_t *nodemask, unsigned int alloc_flags)
+{
+	struct page *page;
+	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
+	struct alloc_context ac = { };
+	unsigned int fastpath_alloc_flags = alloc_flags;
+
+	/* Other flags could be supported later if needed. */
+	if (WARN_ON(alloc_flags & ~ALLOC_NOLOCK))

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5361

Quoted 10 lines
mm/page_alloc.c +8 -0
 		return NULL;
 
+	if (!alloc_order_allowed(gfp, order, alloc_flags))
+		return NULL;
+
+	if (alloc_flags & ALLOC_NOLOCK) {
+		VM_WARN_ON_ONCE(gfp & ~__GFP_ACCOUNT);
+		if (!alloc_trylock_allowed())
+			return NULL;
+		gfp |= gfp_nolock;

I think we could do a

		fastpath_alloc_flags |= ALLOC_WMARK_MIN;

to make it explicit, even though it's a no-op (the value is 0) and alloc_frozen_pages_nolock_noprof() didn't do it.

Quoted 136 lines
mm/page_alloc.c +11 -75
+	} else {
+		fastpath_alloc_flags |= ALLOC_WMARK_LOW;
+	}
+
 	gfp &= gfp_allowed_mask;
 	/*
 	 * Apply scoped allocation constraints. This is mainly about GFP_NOFS
@@ -5310,9 +5384,9 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 	fastpath_alloc_flags |= alloc_flags_nofragment(zonelist_zone(ac.preferred_zoneref), gfp);
 	fastpath_alloc_flags |= alloc_flags_nonblocking(gfp, order) & ALLOC_HIGHATOMIC;
 
-	/* First allocation attempt */
+	/* First allocation attempt (or, for nolock, only attempt) */
 	page = get_page_from_freelist(alloc_gfp, order, fastpath_alloc_flags, &ac);
-	if (likely(page))
+	if (likely(page) || (alloc_flags & ALLOC_NOLOCK))
 		goto out;
 
 	alloc_gfp = gfp;
@@ -5329,7 +5403,8 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 out:
 	if (memcg_kmem_online() && (gfp & __GFP_ACCOUNT) && page &&
 	    unlikely(__memcg_kmem_charge_page(page, gfp, order) != 0)) {
-		free_frozen_pages(page, order);
+		__free_frozen_pages(page, order,
+				    alloc_flags & ALLOC_NOLOCK ? FPI_TRYLOCK : 0);
 		page = NULL;
 	}
 
@@ -5345,7 +5420,8 @@ struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order,
 {
 	struct page *page;
 
-	page = __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask);
+	page = __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask,
+					   ALLOC_DEFAULT);
 	if (page)
 		set_page_refcounted(page);
 	return page;
@@ -7875,80 +7951,10 @@ static bool __free_unaccepted(struct page *page)
 
 struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned int order)
 {
-	/*
-	 * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed.
-	 * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd
-	 * is not safe in arbitrary context.
-	 *
-	 * These two are the conditions for gfpflags_allow_spinning() being true.
-	 *
-	 * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason
-	 * to warn. Also warn would trigger printk() which is unsafe from
-	 * various contexts. We cannot use printk_deferred_enter() to mitigate,
-	 * since the running context is unknown.
-	 *
-	 * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below
-	 * is safe in any context. Also zeroing the page is mandatory for
-	 * BPF use cases.
-	 *
-	 * Though __GFP_NOMEMALLOC is not checked in the code path below,
-	 * specify it here to highlight that alloc_pages_nolock()
-	 * doesn't want to deplete reserves.
-	 */
-	gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_COMP
-			| gfp_flags;
-	unsigned int alloc_flags = ALLOC_NOLOCK;
-	struct alloc_context ac = { };
-	struct page *page;
-
-	VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT);
-	/*
-	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
-	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
-	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
-	 * mark the task as the owner of another rt_spin_lock which will
-	 * confuse PI logic, so return immediately if called from hard IRQ or
-	 * NMI.
-	 *
-	 * Note, irqs_disabled() case is ok. This function can be called
-	 * from raw_spin_lock_irqsave region.
-	 */
-	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
-		return NULL;
-
-	/* On UP, spin_trylock() always succeeds even when it is locked */
-	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
-		return NULL;
-
-	if (!pcp_allowed_order(order))
-		return NULL;
-
-	/* Bailout, since _deferred_grow_zone() needs to take a lock */
-	if (deferred_pages_enabled())
-		return NULL;
-
 	if (nid == NUMA_NO_NODE)
 		nid = numa_node_id();
 
-	prepare_alloc_pages(alloc_gfp, order, nid, NULL, &ac,
-			    &alloc_gfp, &alloc_flags);
-
-	/*
-	 * Best effort allocation from percpu free list.
-	 * If it's empty attempt to spin_trylock zone->lock.
-	 */
-	page = get_page_from_freelist(alloc_gfp, order, alloc_flags, &ac);
-
-	/* Unlike regular alloc_pages() there is no __alloc_pages_slowpath(). */
-
-	if (memcg_kmem_online() && page && (gfp_flags & __GFP_ACCOUNT) &&
-	    unlikely(__memcg_kmem_charge_page(page, alloc_gfp, order) != 0)) {
-		__free_frozen_pages(page, order, FPI_TRYLOCK);
-		page = NULL;
-	}
-	trace_mm_page_alloc(page, order, alloc_gfp, ac.migratetype);
-	kmsan_alloc_page(page, order, alloc_gfp);
-	return page;
+	return __alloc_frozen_pages_noprof(gfp_flags, order, nid, NULL, ALLOC_NOLOCK);
 }
 /**
  * alloc_pages_nolock - opportunistic reentrant allocation from any context
mm/page_alloc.h +1 -0
index 3250d44f96457..e16f905f859a7 100644
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/types.h>
 
+#define ALLOC_DEFAULT		0
 /* The ALLOC_WMARK bits are used as an index to zone->watermark */
 #define ALLOC_WMARK_MIN		WMARK_MIN
 #define ALLOC_WMARK_LOW		WMARK_LOW
@@ -219,7 +220,7 @@ extern bool free_pages_prepare(struct page *page, unsigned int order);
 extern int user_min_free_kbytes;
 
Quoted 15 lines

mm/page_alloc.h struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order); · line 235

Quoted 13 lines
mm/page_alloc.h +3 -2
 struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, int nid,
-		nodemask_t *nodemask);
+		nodemask_t *nodemask, unsigned int alloc_flags);
 #define __alloc_frozen_pages(...) \
 	alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__))
 void free_frozen_pages(struct page *page, unsigned int order);
@@ -230,7 +231,8 @@ struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order);
 #else
 static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order)
 {
-	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL);
+	return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL,
+					   0 /* ALLOC_DEFAULT */);

Can use ALLOC_DEFAULT now.

Thanks and ack to all of these.

Will mention the ALLOC_WMARK_MIN thing in the commit message too.

reviewer
#53 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to Brendan Jackman

Hi Brendan

Quoted 35 lines · Brendan Jackman
Quoted 34 lines · Hao Ge

Hi Brendan

Quoted 30 lines · Brendan Jackman

Now that alloc_pages has an entrypoint that allows passing alloc_flags, we can take advantage of this to start removing GFP flags that are only used for mm-internal stuff.

This requires also plumbing the alloc_flags into some more of the allocator code, in particular __alloc_pages[_noprof]() gets an alloc_flags arg to go along with its callees, and we now need to pass those flags deeper into the allocator so they can reach the alloc_tag code.

No functional change intended.

---
   mm/alloc_tag.c       | 22 ++++++----------------
   mm/compaction.c      |  4 ++--
   mm/internal.h        |  1 -
   mm/page_alloc.c      | 42 ++++++++++++++++++++++++------------------
   mm/page_alloc.h      | 17 +++++++++++++++--
   mm/page_frag_cache.c |  4 ++--
   6 files changed, 49 insertions(+), 41 deletions(-)
mm/alloc_tag.c +0 -0
index d9be1cf5187d9..a32a94e759b94 100644
@@ -15,6 +15,8 @@
   #include <linux/vmalloc.h>
   #include <linux/kmemleak.h>
 
Quoted 7 lines
Quoted 4 lines

mm/alloc_tag.c line 18

Quoted 1 line
mm/alloc_tag.c +1 -0
+#include "internal.h"

Should we include page_alloc.h here, as we call __alloc_pages later in this file?

Yeah, there are a few build failures due to me not doing a broad enough build. From now on I will just wait for allmodconfig instead of trying to be clever with my build tests, sorry about this.

No worries at all. For the alloc_tag build error, it depends on whether

CONFIG_MEM_ALLOC_PROFILING_DEBUG is enabled.

Quoted 4 lines

Also, this suggests that I have not actually re-tested the alloc_tag code since v3 so I must repeat the test described in my cover letter (I just manually enable the feature and check the kernel boots) for v4.

Thanks a lot for your work. I merged your patch locally via my automation bot,

and verified the basic functionality of alloc_tag.

The alloc_tag feature works well so far.

By the way, you may need to rebase your patchset on top of mm-new.

This patch has been merged into mm-new and will cause a minor conflict,

assuming the mm-new tree has not made further modifications to this patch:

https://lore.kernel.org/all/20260629-free-pfn-on-alloc-contig-range-error-path-v1-1-496ff9ca22db@nvidia.com/

Thanks

Best Regards

Hao

Quoted 64 lines
Quoted 56 lines
Quoted 35 lines
mm/alloc_tag.c +3 -14
+
   #define ALLOCINFO_FILE_NAME		"allocinfo"
   #define MODULE_ALLOC_TAG_VMAP_SIZE	(100000UL * sizeof(struct alloc_tag))
   #define SECTION_START(NAME)		(CODETAG_SECTION_START_PREFIX NAME)
@@ -783,19 +785,6 @@ struct pfn_pool {
 
   #define PFN_POOL_SIZE			((PAGE_SIZE - offsetof(struct pfn_pool, pfns)) / \
   					 sizeof(unsigned long))
-
-/*
- * Skip early PFN recording for a page allocation.  Reuses the
- * %__GFP_NO_OBJ_EXT bit.  Used by __alloc_tag_add_early_pfn() to avoid
- * recursion when allocating pages for the early PFN tracking list
- * itself.
- *
- * Codetags of the pages allocated with __GFP_NO_CODETAG should be
- * cleared (via clear_page_tag_ref()) before freeing the pages to prevent
- * alloc_tag_sub_check() from triggering a warning.
- */
-#define __GFP_NO_CODETAG		__GFP_NO_OBJ_EXT
-
   static struct pfn_pool *current_pfn_pool __initdata;
 
   static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
@@ -806,7 +795,8 @@ static void __init __alloc_tag_add_early_pfn(unsigned long pfn)
   	do {
   		pool = READ_ONCE(current_pfn_pool);
   		if (!pool || atomic_read(&pool->count) >= PFN_POOL_SIZE) {
-			struct page *new_page = alloc_page(__GFP_HIGH | __GFP_NO_CODETAG);
+			struct page *new_page = __alloc_pages(__GFP_HIGH, 0, numa_mem_id(),
+							      NULL, ALLOC_NO_CODETAG);
   			struct pfn_pool *new;
 
   			if (!new_page) {

mm/alloc_tag.c typedef void alloc_tag_add_func(unsigned long pfn); · line 830

Quoted 5 lines
mm/alloc_tag.c +1 -1
@@ -837,7 +827,7 @@ typedef void alloc_tag_add_func(unsigned long pfn);
   static alloc_tag_add_func __rcu *alloc_tag_add_early_pfn_ptr __refdata =
   	RCU_INITIALIZER(__alloc_tag_add_early_pfn);
 
-void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)
+void alloc_tag_add_early_pfn(unsigned long pfn, unsigned int alloc_flags)

alloc_tag_add_early_pfn() has three occurrences across the codebase:

1. Definition in mm/alloc_tag.c:830:

void alloc_tag_add_early_pfn(unsigned long pfn, unsigned int alloc_flags)

2. Declaration in include/linux/alloc_tag.h:166:

void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags)

3. Static inline stub in include/linux/alloc_tag.h:170:

static inline void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags) {}

Quoted 7 lines

This patch updates the definition in alloc_tag.c to take unsigned int alloc_flags,

but the two declarations in alloc_tag.h are left with the old gfp_t gfp_flags signature

These should be updated to match.

Yeah ditto, sorry about this and thanks for the review.

reviewer
#54 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

Quoted 66 lines · Hao Ge

Hi Brendan

Quoted 35 lines · Brendan Jackman
Quoted 34 lines · Hao Ge

Hi Brendan

Quoted 30 lines · Brendan Jackman

Now that alloc_pages has an entrypoint that allows passing alloc_flags, we can take advantage of this to start removing GFP flags that are only used for mm-internal stuff.

This requires also plumbing the alloc_flags into some more of the allocator code, in particular __alloc_pages[_noprof]() gets an alloc_flags arg to go along with its callees, and we now need to pass those flags deeper into the allocator so they can reach the alloc_tag code.

No functional change intended.

---
   mm/alloc_tag.c       | 22 ++++++----------------
   mm/compaction.c      |  4 ++--
   mm/internal.h        |  1 -
   mm/page_alloc.c      | 42 ++++++++++++++++++++++++------------------
   mm/page_alloc.h      | 17 +++++++++++++++--
   mm/page_frag_cache.c |  4 ++--
   6 files changed, 49 insertions(+), 41 deletions(-)
mm/alloc_tag.c +0 -0
index d9be1cf5187d9..a32a94e759b94 100644
@@ -15,6 +15,8 @@
   #include <linux/vmalloc.h>
   #include <linux/kmemleak.h>
 
Quoted 7 lines
Quoted 4 lines

mm/alloc_tag.c line 18

Quoted 1 line
mm/alloc_tag.c +1 -0
+#include "internal.h"

Should we include page_alloc.h here, as we call __alloc_pages later in this file?

Yeah, there are a few build failures due to me not doing a broad enough build. From now on I will just wait for allmodconfig instead of trying to be clever with my build tests, sorry about this.

No worries at all. For the alloc_tag build error, it depends on whether

CONFIG_MEM_ALLOC_PROFILING_DEBUG is enabled.

Quoted 4 lines

Also, this suggests that I have not actually re-tested the alloc_tag code since v3 so I must repeat the test described in my cover letter (I just manually enable the feature and check the kernel boots) for v4.

Thanks a lot for your work. I merged your patch locally via my automation bot,

and verified the basic functionality of alloc_tag.

The alloc_tag feature works well so far.

Quoted 7 lines

By the way, you may need to rebase your patchset on top of mm-new.

This patch has been merged into mm-new and will cause a minor conflict,

assuming the mm-new tree has not made further modifications to this patch:

https://lore.kernel.org/all/20260629-free-pfn-on-alloc-contig-range-error-path-v1-1-496ff9ca22db@nvidia.com/

Do not worry about it. Based on the feedback, it might be dropped/rewritten soon. :)

Signature · 2 lines
Best Regards,
Yan, Zi
maintainer
#18 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Brendan Jackman
Quoted 33 lines · Brendan Jackman
Quoted 32 lines · Vlastimil Babka (SUSE)
Quoted 30 lines · Harry Yoo
Quoted 10 lines · Brendan Jackman

Currently the core allocator code is controlled by ALLOC_NOLOCK, but the main entry point function is significantly different from the normal __alloc_frozen_pages_nolock(), this is tiring when reading the code.

Plumb the ALLOC_NOLOCK control one layer up in the call stack: create an alloc_flags argument to __alloc_frozen_pages_nolock() (which is only exposed to mm/) and then turn the nolock variant into a thin wrapper that just sets that flag (as well as handling NUMA_NO_NODE, similar to how some of the wrappers in gfp.h do).

Quoted 5 lines

Rationale that this doesn't change anything:

1. Simple bits: A bunch of the nolock-specific handling is just moved to the new alloc_order_allowed(), alloc_trylock_allowed() and gfp_trylock.

Right.

Quoted 5 lines

2. __alloc_frozen_pages_noprof() has some extra logic that wasn't previously in the nolock variant:

a. Application of gfp_allowed_mask; this only affects early boot, and
only flags that affect the slowpath get changed here.

gfp_allowed_mask clears __GFP_RECLAIM, and that means now allocations with GFP_KERNEL during early boot would see gfpflags_allow_spinning() = false.

Quoted 6 lines
Quoted 4 lines

Is it a problem though? non-nolock allocations were affected before (the masking existed for those already) and will be affected now the same, and _nolock() allocations don't pass __GFP_RECLAIM in the first place, so the masking can't affect them?

This was my thinking too.

Oh! You guys are right :) I was confused.

Quoted 10 lines
Quoted 10 lines
Quoted 8 lines

The helper is not used in in the page allocator, but used in memcg/stackdepot/page_owner.

Quoted 2 lines

b. Application of current_gfp_context() - also only affects the
slowpath

PF_MEMALLOC_PIN affects the fast path, but ALLOC_NOLOCK users won't be affected.

And it wouldn't be wrong if they were? It only clears __GFP_MOVABLE?

Nothing wrong, but wanted to mention things that were not mentioned in the changelog as it argues there's no functional change intended.

Quoted 5 lines
Quoted 4 lines
Quoted 1 line

What about alloc_flags_nofragment/nonblocking()?

ALLOC_NOFRAGMENT due to e.g. defrag_mode could be a problem indeed, if there's no slowpath. Make ALLOC_NOLOCK override it?

Quoted 8 lines

Yeah calling alloc_flags_nofragment() here is a bug in the patch, and Sashiko also complained:

https://lore.kernel.org/all/[email protected]/

Like I said in the reply to that thread I think maybe we _do_ want to set ALLOC_NOFRAGMENT for nolock allocations? But, that is a functional change, it doesn't belong in this series.

I don't think it was intentional to bypass ALLOC_NOFRAGMENT for nolock allocations. But yeah that's a functional change.

Quoted 3 lines
Quoted 1 line

nonblocking() is probably fine?

Yeah, I believe this is fine.

Agreed.

Signature · 2 lines
Cheers,
Harry / Hyeonggon
maintainer
#20 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Brendan Jackman
Quoted 41 lines · Brendan Jackman
Quoted 36 lines
Quoted 17 lines
mm/page_alloc.c +9 -0
index a3ba63c7f9199..8d409d075e3e9 100644
@@ -5271,24 +5271,98 @@ void free_pages_bulk(struct page **page_array, unsigned long nr_pages)
 	}
 }
 
+static inline bool alloc_trylock_allowed(void)
+{
+	/*
+	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
+	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
+	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
+	 * mark the task as the owner of another rt_spin_lock which will
+	 * confuse PI logic, so return immediately if called from hard IRQ or
+	 * NMI.

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5292

Quoted 10 lines
mm/page_alloc.c +10 -0
+	 *
+	 * Note, irqs_disabled() case is ok. This function can be called
+	 * from raw_spin_lock_irqsave region.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
+		return false;
+
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return false;

Except for deferred_pages_enabled(), it's not specific to the page allocator. SLUB has

	/*
	 * See the comment for the same check in
	 * alloc_frozen_pages_nolock_noprof()
	 */
Quoted 4 lines

... and repeats the same thing as above.

Perhaps let's factor it out into a helper
rather than trying not to forget to update the other place?

Quoted 8 lines

Hm, not sure about this. I think I would say it's a "coincidence" that these two bits of code look the same? Like, page_alloc.c uses spin_trylock() so you can't do alloc_pages_nolock() from IRQ on PREEMPT_RT. slub.c ALSO uses spin_trylock(), so you ALSO can't use kmalloc_nolock() in those scenarios. But those are two different facts that just happen to be isomorphic? Putting them into a shared helper would kinda imply that these are part of a single system with inherently coupled constraints.

But as long as they use spinlocks and can be called in unknown contexts, they are supposed to have the same constraint?

And actually, the only reason free_pages_nolock() or kfree_nolock() don't have these checks is just because we don't allow kmalloc() -> kfree_nolock() or alloc_pages() -> free_pages_nolock(). Once we allow them, we'll need to repeat this again.

I think it doesn't even belong page/slab allocators, probably should be spin_trylock_allowed()?

Quoted 1 line

But I'd lean towards leaving this out of> the patchset since the

potential deduplication isn't really related to

Quoted 1 line

the other cleanups anyway.

Ack.

Signature · 2 lines
Cheers,
Harry / Hyeonggon
maintainer Reviewed-by on v3 8/16
#30 lore raw

Re: [PATCH v3 08/16] perf/x86/intel: Use higher-level allocator API

In reply to Brendan Jackman
Quoted 7 lines · Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Quoted 13 lines
Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Ian Rogers <[email protected]>
Cc Adrian Hunter <[email protected]>
Cc James Clark <[email protected]>
Assisted-by Gemini:unknown-version
Quoted 30 lines
---
 arch/x86/events/intel/ds.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/events/intel/ds.c +3 -3
index 91a093d8cf2e7..70be80211d823 100644
@@ -832,7 +832,7 @@ static void *dsalloc_pages(size_t size, gfp_t flags, int cpu)
 	int node = cpu_to_node(cpu);
 	struct page *page;
 
-	page = __alloc_pages_node(node, flags | __GFP_ZERO, order);
+	page = alloc_pages_node(node, flags | __GFP_ZERO, order);
 	return page ? page_address(page) : NULL;
 }
 
@@ -1088,9 +1088,9 @@ void init_arch_pebs_on_cpu(int cpu)
 
 	/*
 	 * 4KB-aligned pointer of the output buffer
-	 * (__alloc_pages_node() return page aligned address)
+	 * (alloc_pages_node() returns page aligned address)
 	 * Buffer Size = 4KB * 2^SIZE
-	 * contiguous physical buffer (__alloc_pages_node() with order)
+	 * contiguous physical buffer (alloc_pages_node() with order)
 	 */
 	arch_pebs_base = virt_to_phys(cpuc->pebs_vaddr) | PEBS_BUFFER_SHIFT;
 	wrmsrq_on_cpu(cpu, MSR_IA32_PEBS_BASE, arch_pebs_base);
maintainer Reviewed-by on v3 9/16
#33 lore raw

Re: [PATCH v3 09/16] KVM: VMX: Use higher-level allocator API

In reply to Brendan Jackman
Quoted 5 lines · Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

Quoted 8 lines

No functional change intended.

Cc Sean Christopherson <[email protected]>
Cc Paolo Bonzini <[email protected]>
Assisted-by Gemini:unknown-version
Quoted 18 lines
---
 arch/x86/kvm/vmx/vmx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
index 2325be57d3d75..ad6a7fc6a54da 100644
@@ -3028,7 +3028,7 @@ struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags)
 	struct page *pages;
 	struct vmcs *vmcs;
 
-	pages = __alloc_pages_node(node, flags, 0);
+	pages = alloc_pages_node(node, flags, 0);
 	if (!pages)
 		return NULL;
 	vmcs = page_address(pages);
maintainer Reviewed-by on v3 10/16
#35 lore raw

Re: [PATCH v3 10/16] x86/virt: Use higher-level allocator API

In reply to Brendan Jackman
Quoted 7 lines · Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Quoted 9 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Assisted-by Gemini:unknown-version
Quoted 18 lines
---
 arch/x86/virt/hw.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/virt/hw.c +1 -1
index 7e9091c640be0..a236447ac7a26 100644
@@ -196,7 +196,7 @@ static __init int __x86_vmx_init(void)
 		struct page *page;
 		struct vmcs *vmcs;
 
-		page = __alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
+		page = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
 		if (WARN_ON_ONCE(!page)) {
 			x86_vmx_exit();
 			return -ENOMEM;
maintainer Reviewed-by on v3 11/16
#39 lore raw

Re: [PATCH v3 11/16] sgi-xp: Use higher-level allocator API

In reply to Brendan Jackman
Quoted 7 lines · Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Quoted 8 lines
Cc Robin Holt <[email protected]>
Cc Steve Wahl <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Assisted-by Gemini:unknown-model

Nit:

Quoted 9 lines
---
 drivers/misc/sgi-xp/xpc_uv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/sgi-xp/xpc_uv.c +0 -0
index 772c787268932..aacff70204241 100644

drivers/misc/sgi-xp/xpc_uv.c xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name, · line 175

Quoted 7 lines
drivers/misc/sgi-xp/xpc_uv.c +1 -1
@@ -170,7 +170,7 @@ xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name,
 	mq->mmr_blade = uv_cpu_to_blade_id(cpu);
 
 	nid = cpu_to_node(cpu);
-	page = __alloc_pages_node(nid,
+	page = alloc_pages_node(nid,
 				      GFP_KERNEL | __GFP_ZERO | __GFP_THISNODE,
 				      pg_order);

gfp flags can now fit on the line above?

Quoted 2 lines
drivers/misc/sgi-xp/xpc_uv.c +0 -0
 	if (page == NULL) {
maintainer Reviewed-by on v3 12/16
#42 lore raw

Re: [PATCH v3 12/16] net/funeth: Switch to higher-level allocator API

In reply to Brendan Jackman
Quoted 7 lines · Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Quoted 9 lines
Cc Dimitris Michailidis <[email protected]>
Cc Andrew Lunn <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc Jakub Kicinski <[email protected]>
Cc Paolo Abeni <[email protected]>
Assisted-by Gemini:unknown-version
Quoted 18 lines
---
 drivers/net/ethernet/fungible/funeth/funeth_rx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/fungible/funeth/funeth_rx.c +1 -1
index 7e2584895de39..d7000017ac2bd 100644
@@ -103,7 +103,7 @@ static int funeth_alloc_page(struct funeth_rxq *q, struct funeth_rxbuf *rb,
 	if (cache_get(q, rb))
 		return 0;
 
-	p = __alloc_pages_node(node, gfp | __GFP_NOWARN, 0);
+	p = alloc_pages_node(node, gfp | __GFP_NOWARN, 0);
 	if (unlikely(!p))
 		return -ENOMEM;
 
maintainer Reviewed-by on v3 13/16
#45 lore raw

Re: [PATCH v3 13/16] mm: Remove __alloc_pages_node()

In reply to Brendan Jackman
Quoted 14 lines · Brendan Jackman

There were only a few users, which have been removed. The only advantage of this API over alloc_pages_node() is avoiding a single conditional branch. The disadvantages are:

1. More API surface, more sources of confusion, more maintenance.

2. Worse impact of CPU hotplug bugs: most users of __alloc_pages_node() were using the result of cpu_to_node(); if the CPU gets hotplugged out this will return NUMA_NO_NODE. If one of these paths fails to protect against a concurrent hotplug then page_alloc.c will use NUMA_NO_NODE as an index into NODE_DATA() and cause some horrible memory corruption or other. With alloc_pages_node(), the code might just work fine.

Quoted 5 lines

Ulterior motive: this frees up the __* variants of the allocator APIs to serve specifically for use as mm-internal API.

Quoted 43 lines
---
 include/linux/gfp.h | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
include/linux/gfp.h +4 -16
index 01d6d2591f49e..3bf55a5f9143e 100644
@@ -256,21 +256,6 @@ static inline void warn_if_node_offline(int this_node, gfp_t gfp_mask)
 	dump_stack();
 }
 
-/*
- * Allocate pages, preferring the node given as nid. The node must be valid and
- * online. For more general interface, see alloc_pages_node().
- */
-static inline struct page *
-__alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order)
-{
-	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
-	warn_if_node_offline(nid, gfp_mask);
-
-	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
-}
-
-#define  __alloc_pages_node(...)		alloc_hooks(__alloc_pages_node_noprof(__VA_ARGS__))
-
 static inline
 struct folio *__folio_alloc_node_noprof(gfp_t gfp, unsigned int order, int nid)
 {
@@ -293,7 +278,10 @@ static inline struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask,
 	if (nid == NUMA_NO_NODE)
 		nid = numa_mem_id();
 
-	return __alloc_pages_node_noprof(nid, gfp_mask, order);
+	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
+	warn_if_node_offline(nid, gfp_mask);
+
+	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
 }
 
 #define  alloc_pages_node(...)			alloc_hooks(alloc_pages_node_noprof(__VA_ARGS__))
maintainer Reviewed-by on v3 14/16
#47 lore raw

Re: [PATCH v3 14/16] mm: Move __alloc_pages() to mm/page_alloc.h

In reply to Brendan Jackman
Quoted 11 lines · Brendan Jackman

It's no longer used outside of mm/.

Since this means __alloc_pages_noprof() is no longer visible from gfp.h, this also means moving the definition of alloc_pages_node_noprof into the .c file.

Also remove references to this API from the documentation tree - referring to the specific function name was already questionable but now the function is not even public it definitely seems wrong.

Less inline bloat also good.

Quoted 108 lines
---
 Documentation/admin-guide/cgroup-v1/cpusets.rst |  2 +-
 Documentation/admin-guide/mm/transhuge.rst      |  2 +-
 include/linux/gfp.h                             | 16 +---------------
 mm/page_alloc.c                                 | 13 ++++++++++++-
 mm/page_alloc.h                                 |  4 ++++
 5 files changed, 19 insertions(+), 18 deletions(-)
Documentation/admin-guide/cgroup-v1/cpusets.rst +1 -1
index c7909e5ac1361..52a213aff04e5 100644
@@ -284,7 +284,7 @@ take action.
 ==>
     Unless this feature is enabled by writing "1" to the special file
     /dev/cpuset/memory_pressure_enabled, the hook in the rebalance
-    code of __alloc_pages() for this metric reduces to simply noticing
+    code of the page allocator for this metric reduces to simply noticing
     that the cpuset_memory_pressure_enabled flag is zero.  So only
     systems that enable this feature will compute the metric.
 
Documentation/admin-guide/mm/transhuge.rst +1 -1
index 23f8d13c2629d..16f37135ed80d 100644
@@ -761,7 +761,7 @@ compact_fail
 	but failed.
 
 It is possible to establish how long the stalls were using the function
-tracer to record how long was spent in __alloc_pages() and
+tracer to record how long was spent in the page allocator and
 using the mm_page_alloc tracepoint to identify which allocations were
 for huge pages.
 
include/linux/gfp.h +1 -15
index 3bf55a5f9143e..4d57e9c0bf204 100644
@@ -204,10 +204,6 @@ static inline void arch_free_page(struct page *page, int order) { }
 static inline void arch_alloc_page(struct page *page, int order) { }
 #endif
 
-struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
-		nodemask_t *nodemask);
-#define __alloc_pages(...)			alloc_hooks(__alloc_pages_noprof(__VA_ARGS__))
-
 struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
 		nodemask_t *nodemask);
 #define __folio_alloc(...)			alloc_hooks(__folio_alloc_noprof(__VA_ARGS__))
@@ -272,17 +268,7 @@ struct folio *__folio_alloc_node_noprof(gfp_t gfp, unsigned int order, int nid)
  * prefer the current CPU's closest node. Otherwise node must be valid and
  * online.
  */
-static inline struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask,
-						   unsigned int order)
-{
-	if (nid == NUMA_NO_NODE)
-		nid = numa_mem_id();
-
-	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
-	warn_if_node_offline(nid, gfp_mask);
-
-	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
-}
+struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order);
 
 #define  alloc_pages_node(...)			alloc_hooks(alloc_pages_node_noprof(__VA_ARGS__))
 
mm/page_alloc.c +12 -1
index 9cb3f1665b41b..026f33f217036 100644
@@ -5427,7 +5427,18 @@ struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order,
 		set_page_refcounted(page);
 	return page;
 }
-EXPORT_SYMBOL(__alloc_pages_noprof);
+
+struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order)
+{
+	if (nid == NUMA_NO_NODE)
+		nid = numa_mem_id();
+
+	VM_BUG_ON(nid < 0 || nid >= MAX_NUMNODES);
+	warn_if_node_offline(nid, gfp_mask);
+
+	return __alloc_pages_noprof(gfp_mask, order, nid, NULL);
+}
+EXPORT_SYMBOL(alloc_pages_node_noprof);
 
 struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
 		nodemask_t *nodemask)
mm/page_alloc.h +4 -0
index af83764788b96..2058cbdca56e7 100644
@@ -244,6 +244,10 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned
 	alloc_hooks(alloc_frozen_pages_nolock_noprof(__VA_ARGS__))
 void free_frozen_pages_nolock(struct page *page, unsigned int order);
 
+struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
+		nodemask_t *nodemask);
+#define __alloc_pages(...)			alloc_hooks(__alloc_pages_noprof(__VA_ARGS__))
+
 extern void zone_pcp_reset(struct zone *zone);
 extern void zone_pcp_disable(struct zone *zone);
 extern void zone_pcp_enable(struct zone *zone);
#56 lore raw

Re: [PATCH v3 15/16] mm: replace __GFP_NO_CODETAG with ALLOC_NO_CODETAG

In reply to Brendan Jackman
Quoted 10 lines · Brendan Jackman

Now that alloc_pages has an entrypoint that allows passing alloc_flags, we can take advantage of this to start removing GFP flags that are only used for mm-internal stuff.

This requires also plumbing the alloc_flags into some more of the allocator code, in particular __alloc_pages[_noprof]() gets an alloc_flags arg to go along with its callees, and we now need to pass those flags deeper into the allocator so they can reach the alloc_tag code.

Quoted 3 lines

No functional change intended.

broadly LGTM, will wait for the updated version due to Hao Ge's feedback.

#60 lore raw

Re: [PATCH v3 16/16] mm: remove the __GFP_NO_OBJ_EXT flag

In reply to Brendan Jackman
Quoted 5 lines · Brendan Jackman

All users of the flag are converted to SLAB_ALLOC_NO_RECURSE or ALLOC_NO_CODETAG (from __GFP_NO_CODETAG which reused the NO_OBJ_EXT bit). Free up the flag bit.

For a correct process, there should be also a matching From:

Thanks!
Vlastimil

Quoted 45 lines
[Rebased onto __GFP_NO_CODETAG removal]
---
 tools/include/linux/gfp_types.h | 7 -------
 1 file changed, 7 deletions(-)
tools/include/linux/gfp_types.h +0 -7
index 6c75df30a281d..a93b8bd200b76 100644
@@ -55,7 +55,6 @@ enum {
 #ifdef CONFIG_LOCKDEP
 	___GFP_NOLOCKDEP_BIT,
 #endif
-	___GFP_NO_OBJ_EXT_BIT,
 	___GFP_LAST_BIT
 };
 
@@ -96,7 +95,6 @@ enum {
 #else
 #define ___GFP_NOLOCKDEP	0
 #endif
-#define ___GFP_NO_OBJ_EXT       BIT(___GFP_NO_OBJ_EXT_BIT)
 
 /*
  * Physical address zone modifiers (see linux/mmzone.h - low four bits)
@@ -137,17 +135,12 @@ enum {
  * node with no fallbacks or placement policy enforcements.
  *
  * %__GFP_ACCOUNT causes the allocation to be accounted to kmemcg.
- *
- * %__GFP_NO_OBJ_EXT causes slab allocation to have no object extension.
- * mark_obj_codetag_empty() should be called upon freeing for objects allocated
- * with this flag to indicate that their NULL tags are expected and normal.
  */
 #define __GFP_RECLAIMABLE ((__force gfp_t)___GFP_RECLAIMABLE)
 #define __GFP_WRITE	((__force gfp_t)___GFP_WRITE)
 #define __GFP_HARDWALL   ((__force gfp_t)___GFP_HARDWALL)
 #define __GFP_THISNODE	((__force gfp_t)___GFP_THISNODE)
 #define __GFP_ACCOUNT	((__force gfp_t)___GFP_ACCOUNT)
-#define __GFP_NO_OBJ_EXT   ((__force gfp_t)___GFP_NO_OBJ_EXT)
 
 /**
  * DOC: Watermark modifiers
#21 lore raw

Re: [PATCH v3 05/16] mm/page_alloc: unify __alloc_frozen_pages[_nolock]_noprof()

In reply to Harry Yoo
Quoted 56 lines · Harry Yoo
Quoted 41 lines · Brendan Jackman
Quoted 36 lines
Quoted 17 lines
mm/page_alloc.c +9 -0
index a3ba63c7f9199..8d409d075e3e9 100644
@@ -5271,24 +5271,98 @@ void free_pages_bulk(struct page **page_array, unsigned long nr_pages)
 	}
 }
 
+static inline bool alloc_trylock_allowed(void)
+{
+	/*
+	 * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is
+	 * unsafe in NMI. If spin_trylock() is called from hard IRQ the current
+	 * task may be waiting for one rt_spin_lock, but rt_spin_trylock() will
+	 * mark the task as the owner of another rt_spin_lock which will
+	 * confuse PI logic, so return immediately if called from hard IRQ or
+	 * NMI.

mm/page_alloc.c void free_pages_bulk(struct page **page_array, unsigned long nr_pages) · line 5292

Quoted 10 lines
mm/page_alloc.c +10 -0
+	 *
+	 * Note, irqs_disabled() case is ok. This function can be called
+	 * from raw_spin_lock_irqsave region.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
+		return false;
+
+	/* On UP, spin_trylock() always succeeds even when it is locked */
+	if (!IS_ENABLED(CONFIG_SMP) && in_nmi())
+		return false;

Except for deferred_pages_enabled(), it's not specific to the page allocator. SLUB has

	/*
	 * See the comment for the same check in
	 * alloc_frozen_pages_nolock_noprof()
	 */
Quoted 4 lines

... and repeats the same thing as above.

Perhaps let's factor it out into a helper
rather than trying not to forget to update the other place?

Quoted 8 lines

Hm, not sure about this. I think I would say it's a "coincidence" that these two bits of code look the same? Like, page_alloc.c uses spin_trylock() so you can't do alloc_pages_nolock() from IRQ on PREEMPT_RT. slub.c ALSO uses spin_trylock(), so you ALSO can't use kmalloc_nolock() in those scenarios. But those are two different facts that just happen to be isomorphic? Putting them into a shared helper would kinda imply that these are part of a single system with inherently coupled constraints.

But as long as they use spinlocks and can be called in unknown contexts, they are supposed to have the same constraint?

Quoted 7 lines

And actually, the only reason free_pages_nolock() or kfree_nolock() don't have these checks is just because we don't allow kmalloc() -> kfree_nolock() or alloc_pages() -> free_pages_nolock(). Once we allow them, we'll need to repeat this again.

I think it doesn't even belong page/slab allocators, probably should be spin_trylock_allowed()?

You are right. I was imagining a function called alloc_nolock_allowed() but actually now I see the name spin_trylock_allowed() I realise it's not about the allocators (which both just happen to use spin_trylock()) at all, it makes perfect sense.

Quoted 5 lines
Quoted 1 line

But I'd lean towards leaving this out of> the patchset since the

potential deduplication isn't really related to

Quoted 1 line

the other cleanups anyway.

Ack.

Patchset is getting pretty big already but most stuff is already acked and also most of the patches are trivial so maybe I'll tack it on after all...

#40 lore raw

Re: [PATCH v3 11/16] sgi-xp: Use higher-level allocator API

In reply to Vlastimil Babka (SUSE)
Quoted 32 lines · Vlastimil Babka (SUSE)
Quoted 7 lines · Brendan Jackman

The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE.

The former is going away and the latter works fine here so switch over.

No functional change intended.

Quoted 8 lines
Cc Robin Holt <[email protected]>
Cc Steve Wahl <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Assisted-by Gemini:unknown-model

Nit:

Quoted 11 lines
---
 drivers/misc/sgi-xp/xpc_uv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/sgi-xp/xpc_uv.c +0 -0
index 772c787268932..aacff70204241 100644
@@ -170,7 +170,7 @@ xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name,
 	mq->mmr_blade = uv_cpu_to_blade_id(cpu);
 
Quoted 7 lines

drivers/misc/sgi-xp/xpc_uv.c xpc_create_gru_mq_uv(unsigned int mq_size, int cpu, char *irq_name, · line 175

Quoted 5 lines
drivers/misc/sgi-xp/xpc_uv.c +1 -1
 	nid = cpu_to_node(cpu);
-	page = __alloc_pages_node(nid,
+	page = alloc_pages_node(nid,
 				      GFP_KERNEL | __GFP_ZERO | __GFP_THISNODE,
 				      pg_order);

gfp flags can now fit on the line above?

Ack.

#61 lore raw

Re: [PATCH v3 16/16] mm: remove the __GFP_NO_OBJ_EXT flag

In reply to Vlastimil Babka (SUSE)
Quoted 8 lines · Vlastimil Babka (SUSE)
Quoted 5 lines · Brendan Jackman

All users of the flag are converted to SLAB_ALLOC_NO_RECURSE or ALLOC_NO_CODETAG (from __GFP_NO_CODETAG which reused the NO_OBJ_EXT bit). Free up the flag bit.

For a correct process, there should be also a matching From:

Hm, that's weird. Maybe whatever weird idiocy I did that caused me to not apply the whole patch, also caused me not to set the Author correctly in Git? Anyway since then I've reapplied it and the From: is showing up when I format it with b4 send -o /tmp/presend so hopefully v4 will be correct...

Acked-by on v3 2/16
#6 lore raw

Re: [PATCH v3 02/16] mm/page_alloc: some renames to clarify alloc_flags scopes

In reply to Brendan Jackman
Quoted 44 lines · Brendan Jackman

It's pretty confusing that:

- The slowpath and fastpath have a totally distinct set of alloc_flags.

- gfp_to_alloc_flags() sounds generic but it only influences the slowpath.

Rename some variables to highlight which alloc_flags are fastpath-specific. Rename gfp_to_alloc_flags() to highlight that it's slowpath-specific.

gfp_to_alloc_flags_cma() and gfp_to_alloc_flags_nonblocking() currently have perfectly harmless names, but to keep the naming consistent also rename those to the alloc_flags_*() pattern (which already exists for alloc_flags_nofragment()).

---
 mm/page_alloc.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)
mm/page_alloc.c +3 -3
index 421271849f291..6010693861ec2 100644
@@ -3774,8 +3774,8 @@ alloc_flags_nofragment(struct zone *zone, gfp_t gfp_mask)
 }
 
 /* Must be called after current_gfp_context() which can change gfp_mask */
-static inline unsigned int gfp_to_alloc_flags_cma(gfp_t gfp_mask,
-						  unsigned int alloc_flags)
+static inline unsigned int alloc_flags_cma(gfp_t gfp_mask,
+					   unsigned int alloc_flags)
 {
 #ifdef CONFIG_CMA
 	if (gfp_migratetype(gfp_mask) == MIGRATE_MOVABLE)
@@ -4474,7 +4474,7 @@ static void wake_all_kswapds(unsigned int order, gfp_t gfp_mask,
 }
 
 static inline unsigned int
-gfp_to_alloc_flags_nonblocking(gfp_t gfp_mask, unsigned int order)
+alloc_flags_nonblocking(gfp_t gfp_mask, unsigned int order)
 {
 	unsigned int alloc_flags = 0;

[...]