make vmalloc gfp flags usage more apparent
Subsystems (2)
cover make vmalloc gfp flags usage more apparent
We should do a better job at enforcing gfp flags for vmalloc. Right now, we have a kernel-doc for __vmalloc_node_range(), and hope callers pass in supported flags. If a caller were to pass in an unsupported flag, we may BUG, silently clear it, or completely ignore it.
If we are more proactive about enforcing gfp flags, we can making sure callers know when they may be asking for unsupported behavior.
This patchset lets vmalloc control the incoming gfp flags, and cleans up some hard to read gfp code.
---
Based on currnet mm-new. Linked rfc [1] and rfc v2[2] for convenience.
Changes from last rfc:
- Collected review tags (Patches 1 & 4)
- Add unlikely keyword to help the compiler
- Replace pr_warn() with WARN(1)
RFC v2:
- Whitelist supported gfp flags instead of blacklisting the unsupported
- Move the flags check up to the only exported functions that accept flags:
__vmalloc_noprof() and vmalloc_huge_node_prof()
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
Vishal Moola (Oracle) (4):
mm/vmalloc: warn on invalid vmalloc gfp flags mm/vmalloc: Add a helper to optimize vmalloc allocation gfps mm/vmalloc: cleanup large_gfp in vm_area_alloc_pages() mm/vmalloc: cleanup gfp flag use in new_vmap_block()
mm/vmalloc.c | 46 ++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 8 deletions(-)
Signature · 1 line
2.51.1
PATCH 1/4 mm/vmalloc: warn on invalid vmalloc gfp flags
Vmalloc explicitly supports a list of flags, but we never enforce them. vmalloc has been trying to handle unsupported flags by clearing and setting flags wherever necessary. This is messy and makes the code harder to understand, when we could simply check for a supported input immediately instead.
Define a helper mask and function telling callers they have passed in invalid flags, and clear those unsupported vmalloc flags.
--- mm/vmalloc.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
mm/vmalloc.c +22 -0
@@ -3911,6 +3911,24 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,return NULL ; } +/* +* See __vmalloc_node_range() for a clear list of supported vmalloc flags. +* This gfp lists all flags currently passed through vmalloc. Currently, +* __GFP_ZERO is used by BFP and __GFP_NORETRY is used by percpu. +*/ +#define GFP_VMALLOC_SUPPORTED (GFP_KERNEL | GFP_ATOMIC | GFP_NOWAIT |\ + __GFP_NOFAIL | __GFP_ZERO | __GFP_NORETRY) + +static gfp_t vmalloc_fix_flags (gfp_t flags) +{ +gfp_t invalid_mask = flags & ~GFP_VMALLOC_SUPPORTED ; + + flags &=GFP_VMALLOC_SUPPORTED ; +WARN (1 ,"Unexpected gfp: %#x (%pGg). Fixing up to gfp: %#x (%pGg). Fix your code!\n" , + invalid_mask, &invalid_mask, flags, &flags); +return flags; +} +/** * __vmalloc_node_range - allocate virtually contiguous memory * @size: allocation size
@@ -4092,6 +4110,8 @@ EXPORT_SYMBOL_GPL(__vmalloc_node_noprof);void *__vmalloc_noprof (unsigned long size,gfp_t gfp_mask) { +if (unlikely (gfp_mask & ~GFP_VMALLOC_SUPPORTED )) + gfp_mask =vmalloc_fix_flags (gfp_mask);return __vmalloc_node_noprof (size,1 , gfp_mask,NUMA_NO_NODE ,__builtin_return_address (0 )); }
@@ -4131,6 +4151,8 @@ EXPORT_SYMBOL(vmalloc_noprof);*/ void *vmalloc_huge_node_noprof (unsigned long size,gfp_t gfp_mask,int node) { +if (unlikely (gfp_mask & ~GFP_VMALLOC_SUPPORTED )) + gfp_mask =vmalloc_fix_flags (gfp_mask);return __vmalloc_node_range_noprof (size,1 ,VMALLOC_START ,VMALLOC_END , gfp_mask,PAGE_KERNEL ,VM_ALLOW_HUGE_VMAP , node,__builtin_return_address (0 ));
Signature · 1 line
2.51.1
PATCH 2/4 mm/vmalloc: Add a helper to optimize vmalloc allocation gfps
In reply to Vishal Moola (Oracle)vm_area_alloc_pages() attempts to use different gfp flags as a way to optimize allocations. This has been done inline which makes things harder to read.
Add a helper function to make the code more readable.
--- mm/vmalloc.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
mm/vmalloc.c +14 -3
@@ -3614,6 +3614,17 @@ void *vmap_pfn(unsigned long *pfns, unsigned int count, pgprot_t prot)EXPORT_SYMBOL_GPL (vmap_pfn);#endif /* CONFIG_VMAP_PFN */ +/* +* Helper for vmalloc to adjust the gfp flags for certain allocations. +*/ +static inline gfp_t vmalloc_gfp_adjust (gfp_t flags,const bool large) +{ + flags |= __GFP_NOWARN; +if (large) + flags &= ~__GFP_NOFAIL; +return flags; +} +static inline unsigned int vm_area_alloc_pages (gfp_t gfp,int nid,unsigned int order,unsigned int nr_pages,struct page **pages)
@@ -3852,9 +3863,9 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,* Please note, the __vmalloc_node_range_noprof() falls-back * to order-0 pages if high-order attempt is unsuccessful. */ - area->nr_pages =vm_area_alloc_pages ((page_order ? - gfp_mask & ~__GFP_NOFAIL : gfp_mask) | __GFP_NOWARN, - node, page_order, nr_small_pages, area->pages); + area->nr_pages =vm_area_alloc_pages ( +vmalloc_gfp_adjust (gfp_mask, page_order), node, + page_order, nr_small_pages, area->pages);atomic_long_add (area->nr_pages, &nr_vmalloc_pages);/* All pages of vm should be charged to same memcg, so use first one. */
Signature · 1 line
2.51.1
PATCH 3/4 mm/vmalloc: cleanup large_gfp in vm_area_alloc_pages()
In reply to Vishal Moola (Oracle)Now that we have already checked for unsupported flags, we can use the helper function to set the necessary gfp flags for the large order allocation optimization.
--- mm/vmalloc.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
mm/vmalloc.c +1 -3
@@ -3634,10 +3634,8 @@ vm_area_alloc_pages(gfp_t gfp, int nid,unsigned int max_attempt_order =MAX_PAGE_ORDER ;struct page *page;int i; -gfp_t large_gfp = (gfp & - ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL | __GFP_COMP)) - | __GFP_NOWARN;unsigned int large_order =ilog2 (nr_remaining); +gfp_t large_gfp =vmalloc_gfp_adjust (gfp, large_order) & ~__GFP_DIRECT_RECLAIM; large_order =min (max_attempt_order, large_order);
Signature · 1 line
2.51.1
PATCH 4/4 mm/vmalloc: cleanup gfp flag use in new_vmap_block()
In reply to Vishal Moola (Oracle)The only caller, vb_alloc(), passes GFP_KERNEL into new_vmap_block() which is a subset of GFP_RECLAIM_MASK. Since there's no reason to use this mask here, remove it.
--- mm/vmalloc.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
mm/vmalloc.c +1 -2
@@ -2699,8 +2699,7 @@ static void *new_vmap_block(unsigned int order, gfp_t gfp_mask) node =numa_node_id (); - vb =kmalloc_node (sizeof (struct vmap_block ), - gfp_mask &GFP_RECLAIM_MASK , node); + vb =kmalloc_node (sizeof (struct vmap_block ), gfp_mask, node);if (unlikely (!vb))return ERR_PTR (-ENOMEM );
Signature · 1 line
2.51.1
syzbot ci has tested the following series
syzbot ci has tested the following series
[v1] make vmalloc gfp flags usage more apparent
https://lore.kernel.org/all/[email protected]
* [PATCH 1/4] mm/vmalloc: warn on invalid vmalloc gfp flags
* [PATCH 2/4] mm/vmalloc: Add a helper to optimize vmalloc allocation gfps
* [PATCH 3/4] mm/vmalloc: cleanup large_gfp in vm_area_alloc_pages()
* [PATCH 4/4] mm/vmalloc: cleanup gfp flag use in new_vmap_block()
and found the following issue:
WARNING: kmalloc bug in bpf_prog_alloc_no_stats
Full report is available here:
https://ci.syzbot.org/series/488ab7c0-de91-4749-bbb2-ca76c3fb798b
***
WARNING: kmalloc bug in bpf_prog_alloc_no_stats
tree: mm-new URL: https://kernel.googlesource.com/pub/scm/linux/kernel/git/akpm/mm.git base: 02dafa01ec9a00c3758c1c6478d82fe601f5f1ba arch: amd64 compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 config: https://ci.syzbot.org/builds/2334ae39-552d-4ca2-8562-7adc18ce2cb0/config
can: broadcast manager protocol
can: netlink gateway - max_hops=1
can: SAE J1939
can: isotp protocol (max_pdu_size 8300)
Bluetooth: RFCOMM TTY layer initialized
Bluetooth: RFCOMM socket layer initialized
Bluetooth: RFCOMM ver 1.11
Bluetooth: BNEP (Ethernet Emulation) ver 1.3
Bluetooth: BNEP filters: protocol multicast
Bluetooth: BNEP socket layer initialized
Bluetooth: HIDP (Human Interface Emulation) ver 1.2
Bluetooth: HIDP socket layer initialized
NET: Registered PF_RXRPC protocol family
Key type rxrpc registered
Key type rxrpc_s registered
NET: Registered PF_KCM protocol family
lec:lane_module_init: lec.c: initialized
mpoa:atm_mpoa_init: mpc.c: initializedl2tp_core: L2TP core driver, V2.0l2tp_ppp: PPPoL2TP kernel driver, V2.0l2tp_ip: L2TP IP encapsulation support (L2TPv3)l2tp_netlink: L2TP netlink interfacel2tp_eth: L2TP ethernet pseudowire support (L2TPv3)l2tp_ip6: L2TP IP encapsulation support for IPv6 (L2TPv3)
NET: Registered PF_PHONET protocol family
8021q: 802.1Q VLAN Support v1.8
sctp: Hash tables configured (bind 32/56)
NET: Registered PF_RDS protocol family
Registered RDS/infiniband transport
Registered RDS/tcp transport
tipc: Activated (version 2.0.0)
NET: Registered PF_TIPC protocol family
tipc: Started in single node mode
smc: adding smcd device lo without pnetid
NET: Registered PF_SMC protocol family
9pnet: Installing 9P2000 support
NET: Registered PF_CAIF protocol family
NET: Registered PF_IEEE802154 protocol family
Key type dns_resolver registered
Key type ceph registered
libceph: loaded (mon/osd proto 15/24)batman_adv: B.A.T.M.A.N. advanced 2025.4 (compatibility version 15) loaded
openvswitch: Open vSwitch switching datapath
NET: Registered PF_VSOCK protocol familympls_gso: MPLS GSO support
IPI shorthand broadcast: enabledsched_clock: Marking stable (21550045890, 115271513)->(21677757748, -12440345) registered taskstats version 1
------------[ cut here ]------------ Unexpected gfp: 0x100000 (__GFP_HARDWALL). Fixing up to gfp: 0xdc0 (GFP_KERNEL|__GFP_ZERO). Fix your code! WARNING: CPU: 1 PID: 1 at mm/vmalloc.c:3936 vmalloc_fix_flags+0x9c/0xe0 Modules linked in: CPU: 1 UID: 0 PID: 1 Comm: swapper/0 Not tainted syzkaller #0 PREEMPT(full) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 RIP: 0010:vmalloc_fix_flags+0x9c/0xe0 Code: 81 e6 1f 52 fe ff 89 74 24 30 81 e3 e0 ad 01 00 89 5c 24 20 90 48 c7 c7 80 b9 76 8b 4c 89 fa 89 d9 4d 89 f0 e8 85 31 6e ff 90 <0f> 0b 90 90 8b 44 24 20 48 c7 04 24 0e 36 e0 45 4b c7 04 2c 00 00 RSP: 0000:ffffc90000066d60 EFLAGS: 00010246 RAX: 50a201fad922ca00 RBX: 0000000000000dc0 RCX: ffff888160a80000 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000002 RBP: ffffc90000066df8 R08: 0000000000000003 R09: 0000000000000004 R10: dffffc0000000000 R11: fffffbfff1bba678 R12: 1ffff9200000cdac R13: dffffc0000000000 R14: ffffc90000066d80 R15: ffffc90000066d90 FS: 0000000000000000(0000) GS:ffff8882a9f32000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 000000000dd38000 CR4: 00000000000006f0 Call Trace: <TASK> __vmalloc_noprof+0xf2/0x120 bpf_prog_alloc_no_stats+0x4a/0x4d0 bpf_prog_alloc+0x3c/0x1a0 bpf_prog_load+0x735/0x19e0 __sys_bpf+0x507/0x860 kern_sys_bpf+0x17d/0x6b0 load+0x39e/0x940 do_one_initcall+0x236/0x820 do_initcall_level+0x104/0x190 do_initcalls+0x59/0xa0 kernel_init_freeable+0x334/0x4b0 kernel_init+0x1d/0x1d0 ret_from_fork+0x4bc/0x870 ret_from_fork_asm+0x1a/0x30 </TASK>
***
If these findings have caused you to resend the series or submit a separate fix, please add the following tag to your commit message:
---
This report is generated by a bot. It may contain errors. syzbot ci engineers can be reached at [email protected].
Uladzislau Rezki
maintainerQuoted 74 lines · syzbot ci
syzbot ci has tested the following series
[v1] make vmalloc gfp flags usage more apparent
https://lore.kernel.org/all/[email protected]
* [PATCH 1/4] mm/vmalloc: warn on invalid vmalloc gfp flags
* [PATCH 2/4] mm/vmalloc: Add a helper to optimize vmalloc allocation gfps
* [PATCH 3/4] mm/vmalloc: cleanuplarge_gfpinvm_area_alloc_pages()
* [PATCH 4/4] mm/vmalloc: cleanup gfp flag use innew_vmap_block()and found the following issue:
WARNING: kmalloc bug inbpf_prog_alloc_no_statsFull report is available here:
https://ci.syzbot.org/series/488ab7c0-de91-4749-bbb2-ca76c3fb798b***
WARNING: kmalloc bug in
bpf_prog_alloc_no_statstree: mm-new URL: https://kernel.googlesource.com/pub/scm/linux/kernel/git/akpm/mm.git base: 02dafa01ec9a00c3758c1c6478d82fe601f5f1ba arch: amd64 compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 config: https://ci.syzbot.org/builds/2334ae39-552d-4ca2-8562-7adc18ce2cb0/configcan: broadcast manager protocol
can: netlink gateway - max_hops=1
can: SAE J1939
can: isotp protocol (max_pdu_size8300)
Bluetooth: RFCOMM TTY layer initialized
Bluetooth: RFCOMM socket layer initialized
Bluetooth: RFCOMM ver 1.11
Bluetooth: BNEP (Ethernet Emulation) ver 1.3
Bluetooth: BNEP filters: protocol multicast
Bluetooth: BNEP socket layer initialized
Bluetooth: HIDP (Human Interface Emulation) ver 1.2
Bluetooth: HIDP socket layer initialized
NET: RegisteredPF_RXRPCprotocol family
Key type rxrpc registered
Key typerxrpc_sregistered
NET: RegisteredPF_KCMprotocol family
lec:lane_module_init:lec.c: initialized
mpoa:atm_mpoa_init:mpc.c: initializedl2tp_core: L2TP core driver, V2.0l2tp_ppp: PPPoL2TP kernel driver, V2.0l2tp_ip: L2TP IP encapsulation support (L2TPv3)l2tp_netlink: L2TP netlink interfacel2tp_eth: L2TP ethernet pseudowire support (L2TPv3)l2tp_ip6: L2TP IP encapsulation support for IPv6 (L2TPv3)
NET: RegisteredPF_PHONETprotocol family
8021q: 802.1Q VLAN Support v1.8
sctp: Hash tables configured (bind 32/56)
NET: RegisteredPF_RDSprotocol family
Registered RDS/infiniband transport
Registered RDS/tcp transport
tipc: Activated (version 2.0.0)
NET: RegisteredPF_TIPCprotocol family
tipc: Started in single node mode
smc: adding smcd device lo without pnetid
NET: RegisteredPF_SMCprotocol family
9pnet: Installing 9P2000 support
NET: RegisteredPF_CAIFprotocol family
NET: RegisteredPF_IEEE802154protocol family
Key typedns_resolverregistered
Key type ceph registered
libceph: loaded (mon/osd proto 15/24)batman_adv: B.A.T.M.A.N. advanced 2025.4 (compatibility version 15) loaded
openvswitch: Open vSwitch switching datapath
NET: RegisteredPF_VSOCKprotocol familympls_gso: MPLS GSO support
IPI shorthand broadcast: enabledsched_clock: Marking stable (21550045890, 115271513)->(21677757748, -12440345) registered taskstats version 1
Quoted 3 lines
------------[ cut here ]------------ Unexpected gfp: 0x100000 (__GFP_HARDWALL). Fixing up to gfp: 0xdc0 (GFP_KERNEL|__GFP_ZERO). Fix your code!
It looks like we need to add __GFP_HARDWALL to the white-list-mask.
--
Uladzislau Rezki
Quoted 3 lines · Uladzislau Rezki
Quoted 2 lines
Unexpected gfp:
0x100000(__GFP_HARDWALL). Fixing up to gfp:0xdc0(GFP_KERNEL|__GFP_ZERO). Fix your code!It looks like we need to add
__GFP_HARDWALLto the white-list-mask.
__GFP_HARDWALL is part of GFP_USER. Doing GFP_USER vmalloc sounds like a bit of an odd idea to me, but there are a few users mostly in bpf and drm code (why do these always show up for odd API usage patterns?).
So I guess yes, we'll need to allow it for now, but I'd like to start a discussion if it really makes much sense.
Uladzislau Rezki
maintainerQuoted 12 lines · Christoph Hellwig
Quoted 3 lines · Uladzislau Rezki
Quoted 2 lines
Unexpected gfp:
0x100000(__GFP_HARDWALL). Fixing up to gfp:0xdc0(GFP_KERNEL|__GFP_ZERO). Fix your code!It looks like we need to add
__GFP_HARDWALLto the white-list-mask.
__GFP_HARDWALLis part ofGFP_USER. DoingGFP_USERvmalloc sounds like a bit of an odd idea to me, but there are a few users mostly in bpf and drm code (why do these always show up for odd API usage patterns?).So I guess yes, we'll need to allow it for now, but I'd like to start a discussion if it really makes much sense.
<snip>
/* plain bpf_prog allocation */
prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER);
if (!prog) {<snip>
I assume that was the place that triggered the splat.
Vishal, will you send the patch adding GFP_USER to address the splat?
--
Uladzislau Rezki
Quoted 14 lines · Uladzislau Rezki
Quoted 12 lines · Christoph Hellwig
Quoted 3 lines · Uladzislau Rezki
Quoted 2 lines
Unexpected gfp:
0x100000(__GFP_HARDWALL). Fixing up to gfp:0xdc0(GFP_KERNEL|__GFP_ZERO). Fix your code!It looks like we need to add
__GFP_HARDWALLto the white-list-mask.
__GFP_HARDWALLis part ofGFP_USER. DoingGFP_USERvmalloc sounds like a bit of an odd idea to me, but there are a few users mostly in bpf and drm code (why do these always show up for odd API usage patterns?).So I guess yes, we'll need to allow it for now, but I'd like to start a discussion if it really makes much sense.
<snip>
Quoted 8 lines
/* plain bpf_prog allocation */ prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER); if (!prog) {<snip>
I assume that was the place that triggered the splat.
Vishal, will you send the patch adding
GFP_USERto address the splat?
Yes, I'll send a new version including __GFP_HARDWALL in the mask and update the comment accordingly.