Skip to contentSkip to conversation

bpf, x86: enable EXECMEM_ROX_CACHE for BPF allocations

PATCH Applied Mike Rapoport (Microsoft) 7 participants 25 messages Jul 16 – Aug 17

Subsystems (5)
x86 BPF x86 MM BPF core +1 more subsystems
lore
Share & tools
lore
#0 lore raw

v3 cover bpf, x86: enable EXECMEM_ROX_CACHE for BPF allocations

Hi,

BPF allocations of executable memory on x86 are essentially read-only. Most paths that call bpf_jit_alloc_exec() immediately make it ROX with set_memory_rox().

The code generation, at least on x86, uses separately allocated writable buffers and then updates the actual text memory with text_poke().

These patches do several small adjustments to how BPF allocates executable memory and enable EXECMEM_ROX_CACHE for BPF allocations on x86.

---
v3 changes:
* replace vmalloc() with vzalloc() in bpf_dispatcher_change_prog()
* rebase on the current bpf-next

v2: https://patch.msgid.link/[email protected]
* rebase on the current bpf-next

v1: https://patch.msgid.link/[email protected]

---

---
Mike Rapoport (Microsoft) (5):
bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()
bpf: drop __weak from bpf_jit_alloc_exec() and bpf_jit_free_exec()
bpf: alloc_prog_pack(): skip ROX management for already ROX memory bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable x86/bpf: enable EXECMEM_ROX_CACHE for BPF allocations

 arch/x86/mm/init.c          |  4 ++--
 arch/x86/net/bpf_jit_comp.c |  5 ++---
 include/linux/filter.h      |  1 +
 kernel/bpf/core.c           | 30 +++++++++++++++++++++---------
 kernel/bpf/dispatcher.c     |  5 ++++-
 5 files changed, 30 insertions(+), 15 deletions(-)

---
base-commit: d1f4b56417a3dc1a0600f960b14f46bd25eda89d
change-id: 20260626-execmem-x86-rox-bpf-v0-b4241ade80df

--
Sincerely yours,
Mike.

#1 lore raw

PATCH v3 1/5 bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

bpf_dispatcher->rw_image is a temporary writable buffer that arch_prepare_bpf_dispatcher() fills and then copies into bpf_dispatcher->image using bpf_arch_text_copy().

The rel32 offsets emitted by emit_bpf_dispatcher() are calculated against ->image, so ->rw_image does not need to live in the module address range.

Allocate ->rw_image with vzalloc() to avoid permissions dance when EXECMEM_BPF will be backed by ROX caches.

Using vzalloc() rather than vmalloc() ensures that the memory that bpf_dispatcher_update() unconditionally copies into the executable buffer is zeroed, which is not ideal but still better than random memory returned by the existing bpf_jit_alloc_exec() or plain vmalloc().

Switching from bpf_jit_alloc_exec() to vzalloc() also saves a bit of space in the more scarce module address space.

---
 kernel/bpf/dispatcher.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/bpf/dispatcher.c +4 -1
index ea2d60dc1fee..79f0c222c583 100644
@@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
 		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
 		if (!d->image)
 			goto out;
-		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
+		/* d->rw_image doesn't need to be in module memory range, so we
+		 * can use vzalloc.
+		 */
+		d->rw_image = vzalloc(PAGE_SIZE);
 		if (!d->rw_image) {
 			bpf_prog_pack_free(d->image, PAGE_SIZE);
 			d->image = NULL;
Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH bpf-next v3 1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

This isn't a bug, but for multi-line comments under kernel/bpf/ the opening /* is expected to sit on its own line, so would this read better as:

kernel/bpf/dispatcher.c void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from, · line 154

Quoted 13 lines
kernel/bpf/dispatcher.c +4 -1
index ea2d60dc1fee..79f0c222c583 100644
@@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
 		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
 		if (!d->image)
 			goto out;
-		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
+		/* d->rw_image doesn't need to be in module memory range, so we
+		 * can use vzalloc.
+		 */
+		d->rw_image = vzalloc(PAGE_SIZE);

This isn't a bug, but for multi-line comments under kernel/bpf/ the opening /* is expected to sit on its own line, so would this read better as:

	/*
	 * d->rw_image doesn't need to be in module memory range, so we
	 * can use vzalloc.
	 */

kernel/bpf/dispatcher.c void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from, · line 157

Quoted 3 lines
kernel/bpf/dispatcher.c +0 -0
 		if (!d->rw_image) {
 			bpf_prog_pack_free(d->image, PAGE_SIZE);
 			d->image = NULL;

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29482648692

#3 lore raw

Re: [PATCH bpf-next v3 1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

Quoted 17 lines · On Thu, Jul 16, 2026 at 08:52:25AM +0000, [email protected] wrote:

kernel/bpf/dispatcher.c void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from, · line 154

Quoted 13 lines
kernel/bpf/dispatcher.c +4 -1
index ea2d60dc1fee..79f0c222c583 100644
@@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
 		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
 		if (!d->image)
 			goto out;
-		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
+		/* d->rw_image doesn't need to be in module memory range, so we
+		 * can use vzalloc.
+		 */
+		d->rw_image = vzalloc(PAGE_SIZE);

This isn't a bug, but for multi-line comments under kernel/bpf/ the opening /* is expected to sit on its own line, so would this read better as:

Quoted 4 lines
	/*
	 * d->rw_image doesn't need to be in module memory range, so we
	 * can use vzalloc.
	 */

This really contradicts what I see in the code :/

Quoted 10 lines

kernel/bpf/dispatcher.c void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from, · line 157

Quoted 3 lines
kernel/bpf/dispatcher.c +0 -0
 		if (!d->rw_image) {
 			bpf_prog_pack_free(d->image, PAGE_SIZE);
 			d->image = NULL;

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29482648692

Signature · 2 lines
Sincerely yours,
Mike.
#4 lore raw

Re: [PATCH bpf-next v3 1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

Quoted 18 lines · Mike Rapoport
Quoted 17 lines · On Thu, Jul 16, 2026 at 08:52:25AM +0000, [email protected] wrote:

kernel/bpf/dispatcher.c void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from, · line 154

Quoted 13 lines
kernel/bpf/dispatcher.c +4 -1
index ea2d60dc1fee..79f0c222c583 100644
@@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
 		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
 		if (!d->image)
 			goto out;
-		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
+		/* d->rw_image doesn't need to be in module memory range, so we
+		 * can use vzalloc.
+		 */
+		d->rw_image = vzalloc(PAGE_SIZE);

This isn't a bug, but for multi-line comments under kernel/bpf/ the opening /* is expected to sit on its own line, so would this read better as:

Quoted 7 lines
Quoted 4 lines
	/*
	 * d->rw_image doesn't need to be in module memory range, so we
	 * can use vzalloc.
	 */

This really contradicts what I see in the code :/

We ended up changing to the multi-line convention, but you can let it be. People are out on vacation, hence things are moving slow, but I plan to get to this series soon. Sorry about the delay.

Quoted 10 lines
Quoted 10 lines

kernel/bpf/dispatcher.c void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from, · line 157

Quoted 3 lines
kernel/bpf/dispatcher.c +0 -0
 		if (!d->rw_image) {
 			bpf_prog_pack_free(d->image, PAGE_SIZE);
 			d->image = NULL;

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29482648692

#5 lore raw

PATCH v3 2/5 bpf: drop __weak from bpf_jit_alloc_exec() and bpf_jit_free_exec()

In reply to Mike Rapoport (Microsoft)

bpf_jit_alloc_exec() and bpf_jit_free_exec() are wrappers for the corresponding execmem APIs.

Architectures define the properties of the memory range needed by BPF in their initialization of execmem and don't need to override neither of them.

Drop the __weak qualifier from bpf_jit_alloc_exec() and bpf_jit_free_exec().

---
 kernel/bpf/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/bpf/core.c +2 -2
index 47fe047ad30b..fc75625dc951 100644
@@ -1116,12 +1116,12 @@ void bpf_jit_uncharge_modmem(u32 size)
 	atomic_long_sub(size, &bpf_jit_current);
 }
 
-void *__weak bpf_jit_alloc_exec(unsigned long size)
+void *bpf_jit_alloc_exec(unsigned long size)
 {
 	return execmem_alloc(EXECMEM_BPF, size);
 }
 
-void __weak bpf_jit_free_exec(void *addr)
+void bpf_jit_free_exec(void *addr)
 {
 	execmem_free(addr);
 }
Signature · 1 line
2.53.0
#6 lore raw

PATCH v3 3/5 bpf: alloc_prog_pack(): skip ROX management for already ROX memory

In reply to Mike Rapoport (Microsoft)

execmem_alloc() can return ROX memory that is already filled with architecture defined trapping instructions.

In preparation for enabling this mode for BPF on x86, make sure that there is no redundant management of the ROX memory.

There is no need to fill allocated memory with trapping instructions, to request permissions reset on free and to set ROX permissions as this all is handled by execmem_alloc().

Add bpf_jit_mem_is_rox() wrapper for execmem_is_rox(), use it to check if execmem_alloc() returns ROX memory and skip the redundant steps in that case.

---
 kernel/bpf/core.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)
kernel/bpf/core.c +14 -7
index fc75625dc951..1b89c18cf246 100644
@@ -916,6 +916,11 @@ static LIST_HEAD(pack_list);
 
 #define BPF_PROG_CHUNK_COUNT (BPF_PROG_PACK_SIZE / BPF_PROG_CHUNK_SIZE)
 
+static bool bpf_jit_mem_is_rox(void)
+{
+	return execmem_is_rox(EXECMEM_BPF);
+}
+
 static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_insns)
 {
 	struct bpf_prog_pack *pack;
@@ -927,16 +932,18 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins
 	pack->ptr = bpf_jit_alloc_exec(BPF_PROG_PACK_SIZE);
 	if (!pack->ptr)
 		goto out;
-	bpf_fill_ill_insns(pack->ptr, BPF_PROG_PACK_SIZE);
 	bitmap_zero(pack->bitmap, BPF_PROG_PACK_SIZE / BPF_PROG_CHUNK_SIZE);
 
 	if (static_branch_unlikely(&bpf_pred_flush_enabled))
 		pack->arch_flush_needed = true;
-	set_vm_flush_reset_perms(pack->ptr);
-	err = set_memory_rox((unsigned long)pack->ptr,
-			     BPF_PROG_PACK_SIZE / PAGE_SIZE);
-	if (err)
-		goto out;
+	if (!bpf_jit_mem_is_rox()) {
+		bpf_fill_ill_insns(pack->ptr, BPF_PROG_PACK_SIZE);
+		set_vm_flush_reset_perms(pack->ptr);
+		err = set_memory_rox((unsigned long)pack->ptr,
+				     BPF_PROG_PACK_SIZE / PAGE_SIZE);
+		if (err)
+			goto out;
+	}
 	list_add_tail(&pack->list, &pack_list);
 	return pack;
 
@@ -965,7 +972,7 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool
 			pr_warn_once("BPF: Predictors not flushed for allocations greater than BPF_PROG_PACK_SIZE\n");
 		size = round_up(size, PAGE_SIZE);
 		ptr = bpf_jit_alloc_exec(size);
-		if (ptr) {
+		if (ptr && !bpf_jit_mem_is_rox()) {
 			int err;
 
 			bpf_fill_ill_insns(ptr, size);
Signature · 1 line
2.53.0
#7 lore raw

PATCH v3 4/5 bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable

In reply to Mike Rapoport (Microsoft)

arch_bpf_trampoline_size() allocates a buffer to get actual size required for a trampoline.

This buffer must be in the module address space because __arch_prepare_bpf_trampoline() calculates rel32 offsets relatively to that buffer.

In preparation for enabling ROX mode for EXECMEM_BPF make sure that the allocated memory is writable.

Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for buffer allocation in arch_bpf_trampoline_size().

---
 arch/x86/net/bpf_jit_comp.c | 5 ++---
 include/linux/filter.h      | 1 +
 kernel/bpf/core.c           | 5 +++++
 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +2 -3
index de7515ea1bea..b2feec81e231 100644
@@ -3703,13 +3703,12 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 	int ret;
 
 	/* Allocate a temporary buffer for __arch_prepare_bpf_trampoline().
-	 * This will NOT cause fragmentation in direct map, as we do not
-	 * call set_memory_*() on this buffer.
 	 *
 	 * We cannot use kvmalloc here, because we need image to be in
 	 * module memory range.
+	 * Since it must be writable use bpf_jit_alloc_exec_rw().
 	 */
-	image = bpf_jit_alloc_exec(PAGE_SIZE);
+	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);
 	if (!image)
 		return -ENOMEM;
 
include/linux/filter.h +1 -0
index 14acb2455746..32d5297c557e 100644
@@ -1333,6 +1333,7 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr,
 void bpf_jit_binary_free(struct bpf_binary_header *hdr);
 u64 bpf_jit_alloc_exec_limit(void);
 void *bpf_jit_alloc_exec(unsigned long size);
+void *bpf_jit_alloc_exec_rw(unsigned long size);
 void bpf_jit_free_exec(void *addr);
 void bpf_jit_free(struct bpf_prog *fp);
 struct bpf_binary_header *
kernel/bpf/core.c +5 -0
index 1b89c18cf246..e2076667b245 100644
@@ -1128,6 +1128,11 @@ void *bpf_jit_alloc_exec(unsigned long size)
 	return execmem_alloc(EXECMEM_BPF, size);
 }
 
+void *bpf_jit_alloc_exec_rw(unsigned long size)
+{
+	return execmem_alloc_rw(EXECMEM_BPF, size);
+}
+
 void bpf_jit_free_exec(void *addr)
 {
 	execmem_free(addr);
Signature · 1 line
2.53.0
#8 lore raw

Re: [PATCH bpf-next v3 4/5] bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable

Quoted 26 lines · Mike Rapoport (Microsoft)

arch_bpf_trampoline_size() allocates a buffer to get actual size required for a trampoline.

This buffer must be in the module address space because __arch_prepare_bpf_trampoline() calculates rel32 offsets relatively to that buffer.

In preparation for enabling ROX mode for EXECMEM_BPF make sure that the allocated memory is writable.

Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for > buffer allocation in arch_bpf_trampoline_size().

---
 arch/x86/net/bpf_jit_comp.c | 5 ++---
 include/linux/filter.h      | 1 +
 kernel/bpf/core.c           | 5 +++++
 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +0 -0
index de7515ea1bea..b2feec81e231 100644
@@ -3703,13 +3703,12 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 	int ret;
 

arch/x86/net/bpf_jit_comp.c int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags, · line 3711

Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +2 -3
 	/* Allocate a temporary buffer for __arch_prepare_bpf_trampoline().
-	 * This will NOT cause fragmentation in direct map, as we do not
-	 * call set_memory_*() on this buffer.
 	 *
 	 * We cannot use kvmalloc here, because we need image to be in
 	 * module memory range.
+	 * Since it must be writable use bpf_jit_alloc_exec_rw().
 	 */
-	image = bpf_jit_alloc_exec(PAGE_SIZE);
+	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);

hi,
this change (this particular patch plus possibly others in this set) is causing tracing_multi attachment bench slowdown

the benchmark allocates huge number of trampolines and I'm seeing extra arch_bpf_trampoline_size code paths in the attached perf profile

I'm not that familiar with the allocator, but following hack makes the benchmark ok again:

	diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
	index 6a94370a2448..bbff3c9c6681 100644
	--- a/kernel/bpf/core.c
	+++ b/kernel/bpf/core.c
	@@ -1130,7 +1130,7 @@ void *bpf_jit_alloc_exec(unsigned long size)
	 
	 void *bpf_jit_alloc_exec_rw(unsigned long size)
	 {
	-	return execmem_alloc_rw(EXECMEM_BPF, size);
	+	return execmem_alloc(EXECMEM_MODULE_DATA, size);
	 }
	 
	 void bpf_jit_free_exec(void *addr)

I still need to do more checks, but I'm wondering if we could actually fix this by not allocating image data in arch_bpf_trampoline_size at all.. and just teach __arch_prepare_bpf_trampoline to survive NULL image data and just return the size in such case

thanks,
jirka

---
11.11% test_progs [kernel.kallsyms] [k] decay_va_pool_node
|
---decay_va_pool_node
__purge_vmap_area_lazy
_vm_unmap_aliases
change_page_attr_set_clr
|
|--4.58%--set_memory_rw
| |
| |--2.33%--execmem_alloc_rw
| | arch_bpf_trampoline_size
| | bpf_trampoline_update
| | __bpf_trampoline_link_prog
| | bpf_trampoline_multi_attach
| | bpf_tracing_multi_attach
| | __sys_bpf
| | __x64_sys_bpf
| | do_syscall_64
| | entry_SYSCALL_64_after_hwframe
| | syscall
| | sys_bpf_fd
| | bpf_link_create
| | bpf_program__attach_tracing_multi
| | serial_test_tracing_multi_bench_attach
| | run_one_test
| | main
| | __libc_start_call_main
| | __libc_start_main@@GLIBC_2.34
| | _start
| |
| --2.25%--__execmem_cache_free
| execmem_free
| arch_bpf_trampoline_size
| bpf_trampoline_update
| __bpf_trampoline_link_prog
| bpf_trampoline_multi_attach
| bpf_tracing_multi_attach
| __sys_bpf
| __x64_sys_bpf
| do_syscall_64
| entry_SYSCALL_64_after_hwframe
| syscall
| sys_bpf_fd
| bpf_link_create
| bpf_program__attach_tracing_multi
| serial_test_tracing_multi_bench_attach
| run_one_test
| main
| __libc_start_call_main
| __libc_start_main@@GLIBC_2.34
| _start
|
|--4.52%--set_memory_nx
| |
| |--2.32%--execmem_alloc_rw
| | arch_bpf_trampoline_size
| | bpf_trampoline_update
| | __bpf_trampoline_link_prog
| | bpf_trampoline_multi_attach
| | bpf_tracing_multi_attach
| | __sys_bpf
| | __x64_sys_bpf
| | do_syscall_64
| | entry_SYSCALL_64_after_hwframe
| | syscall
| | sys_bpf_fd
| | bpf_link_create
| | bpf_program__attach_tracing_multi
| | serial_test_tracing_multi_bench_attach
| | run_one_test
| | main
| | __libc_start_call_main
| | __libc_start_main@@GLIBC_2.34
| | _start
| |
| --2.20%--__execmem_cache_free
| execmem_free
| arch_bpf_trampoline_size
| bpf_trampoline_update
| __bpf_trampoline_link_prog
| bpf_trampoline_multi_attach
| bpf_tracing_multi_attach
| __sys_bpf
| __x64_sys_bpf
| do_syscall_64
| entry_SYSCALL_64_after_hwframe
| syscall
| sys_bpf_fd
| bpf_link_create
| bpf_program__attach_tracing_multi
| serial_test_tracing_multi_bench_attach
| run_one_test
| main
| __libc_start_call_main
| __libc_start_main@@GLIBC_2.34
| _start
|
--2.01%--set_memory_rox
|
--2.01%--__execmem_cache_free
execmem_free arch_bpf_trampoline_size bpf_trampoline_update __bpf_trampoline_link_prog bpf_trampoline_multi_attach bpf_tracing_multi_attach __sys_bpf
__x64_sys_bpf do_syscall_64 entry_SYSCALL_64_after_hwframe syscall
sys_bpf_fd
bpf_link_create bpf_program__attach_tracing_multi serial_test_tracing_multi_bench_attach run_one_test
main __libc_start_call_main __libc_start_main@@GLIBC_2.34 _start

6.15% test_progs [kernel.kallsyms] [k] smp_call_function_many_cond
|
---smp_call_function_many_cond
on_each_cpu_cond_mask
|
|--4.80%--cpa_flush
| change_page_attr_set_clr
| |
| |--2.37%--set_memory_rox
| | __execmem_cache_free
| | execmem_free
| | arch_bpf_trampoline_size
| | bpf_trampoline_update
| | __bpf_trampoline_link_prog
| | bpf_trampoline_multi_attach
| | bpf_tracing_multi_attach
| | __sys_bpf
| | __x64_sys_bpf
| | do_syscall_64
| | entry_SYSCALL_64_after_hwframe
| | syscall
| | sys_bpf_fd
| | bpf_link_create
| | bpf_program__attach_tracing_multi
| | serial_test_tracing_multi_bench_attach
| | run_one_test
| | main
| | __libc_start_call_main
| | __libc_start_main@@GLIBC_2.34
| | _start
| |
| |--1.21%--set_memory_nx
| | execmem_alloc_rw
| | arch_bpf_trampoline_size
| | bpf_trampoline_update
| | __bpf_trampoline_link_prog
| | bpf_trampoline_multi_attach
| | bpf_tracing_multi_attach
| | __sys_bpf
| | __x64_sys_bpf
| | do_syscall_64
| | entry_SYSCALL_64_after_hwframe
| | syscall
| | sys_bpf_fd
| | bpf_link_create
| | bpf_program__attach_tracing_multi
| | serial_test_tracing_multi_bench_attach
| | run_one_test
| | main
| | __libc_start_call_main
| | __libc_start_main@@GLIBC_2.34
| | _start
| |
| --1.21%--set_memory_rw
| execmem_alloc_rw
| |
| --1.21%--arch_bpf_trampoline_size
| bpf_trampoline_update
| __bpf_trampoline_link_prog
| bpf_trampoline_multi_attach
| bpf_tracing_multi_attach
| __sys_bpf
| __x64_sys_bpf
| do_syscall_64
| entry_SYSCALL_64_after_hwframe
| syscall
| sys_bpf_fd
| bpf_link_create
| bpf_program__attach_tracing_multi
| serial_test_tracing_multi_bench_attach
| run_one_test
| main
| __libc_start_call_main
| __libc_start_main@@GLIBC_2.34
| _start
|
--1.21%--__change_page_attr_set_clr
change_page_attr_set_clr set_memory_nx
execmem_alloc_rw
|
--1.20%--arch_bpf_trampoline_size
bpf_trampoline_update __bpf_trampoline_link_prog bpf_trampoline_multi_attach bpf_tracing_multi_attach __sys_bpf
__x64_sys_bpf do_syscall_64 entry_SYSCALL_64_after_hwframe syscall
sys_bpf_fd
bpf_link_create bpf_program__attach_tracing_multi serial_test_tracing_multi_bench_attach run_one_test
main __libc_start_call_main __libc_start_main@@GLIBC_2.34 _start

5.01% test_progs [kernel.kallsyms] [k] __raw_callee_save___pv_queued_spin_unlock
|
---__raw_callee_save___pv_queued_spin_unlock
do_raw_spin_unlock
|
|--4.20%--_raw_spin_unlock
| |
| --4.10%--decay_va_pool_node
| __purge_vmap_area_lazy
| _vm_unmap_aliases
| change_page_attr_set_clr
| |
| |--1.73%--set_memory_nx
| | |
| | |--0.95%--__execmem_cache_free
| | | execmem_free
| | | arch_bpf_trampoline_size
| | | bpf_trampoline_update
| | | __bpf_trampoline_link_prog
| | | bpf_trampoline_multi_attach
| | | bpf_tracing_multi_attach
| | | __sys_bpf
| | | __x64_sys_bpf
| | | do_syscall_64
| | | entry_SYSCALL_64_after_hwframe
| | | syscall
| | | sys_bpf_fd
| | | bpf_link_create
| | | bpf_program__attach_tracing_multi
| | | serial_test_tracing_multi_bench_attach
| | | run_one_test
| | | main
| | | __libc_start_call_main
| | | __libc_start_main@@GLIBC_2.34
| | | _start
| | |
| | --0.78%--execmem_alloc_rw
| | arch_bpf_trampoline_size
| | bpf_trampoline_update
| | __bpf_trampoline_link_prog
| | bpf_trampoline_multi_attach
| | bpf_tracing_multi_attach
| | __sys_bpf
| | __x64_sys_bpf
| | do_syscall_64
| | entry_SYSCALL_64_after_hwframe
| | syscall
| | sys_bpf_fd
| | bpf_link_create
| | bpf_program__attach_tracing_multi
| | serial_test_tracing_multi_bench_attach
| | run_one_test
| | main
| | __libc_start_call_main
| | __libc_start_main@@GLIBC_2.34
| | _start
| |
| |--1.51%--set_memory_rw
| | |
| | |--0.76%--__execmem_cache_free
| | | execmem_free
| | | arch_bpf_trampoline_size
| | | bpf_trampoline_update
| | | __bpf_trampoline_link_prog
| | | bpf_trampoline_multi_attach
| | | bpf_tracing_multi_attach
| | | __sys_bpf
| | | __x64_sys_bpf
| | | do_syscall_64
| | | entry_SYSCALL_64_after_hwframe
| | | syscall
| | | sys_bpf_fd
| | | bpf_link_create
| | | bpf_program__attach_tracing_multi
| | | serial_test_tracing_multi_bench_attach
| | | run_one_test
| | | main
| | | __libc_start_call_main
| | | __libc_start_main@@GLIBC_2.34
| | | _start
| | |
| | --0.75%--execmem_alloc_rw
| | arch_bpf_trampoline_size
| | bpf_trampoline_update
| | __bpf_trampoline_link_prog
| | bpf_trampoline_multi_attach
| | bpf_tracing_multi_attach
| | __sys_bpf
| | __x64_sys_bpf
| | do_syscall_64
| | entry_SYSCALL_64_after_hwframe
| | syscall
| | sys_bpf_fd
| | bpf_link_create
| | bpf_program__attach_tracing_multi
| | serial_test_tracing_multi_bench_attach
| | run_one_test
| | main
| | __libc_start_call_main
| | __libc_start_main@@GLIBC_2.34
| | _start
| |
| --0.86%--set_memory_rox
| __execmem_cache_free
| execmem_free
| arch_bpf_trampoline_size
| bpf_trampoline_update
| __bpf_trampoline_link_prog
| bpf_trampoline_multi_attach
| bpf_tracing_multi_attach
| __sys_bpf
| __x64_sys_bpf
| do_syscall_64
| entry_SYSCALL_64_after_hwframe
| syscall
| sys_bpf_fd
| bpf_link_create
| bpf_program__attach_tracing_multi
| serial_test_tracing_multi_bench_attach
| run_one_test
| main
| __libc_start_call_main
| __libc_start_main@@GLIBC_2.34
| _start
|
--0.80%--_raw_spin_unlock_irqrestore

4.88% test_progs [kernel.kallsyms] [k] get_symbol_offset

#9 lore raw

Re: [PATCH bpf-next v3 4/5] bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable

Hi,

Quoted 48 lines · Jiri Olsa
Quoted 26 lines · Mike Rapoport (Microsoft)

arch_bpf_trampoline_size() allocates a buffer to get actual size required for a trampoline.

This buffer must be in the module address space because __arch_prepare_bpf_trampoline() calculates rel32 offsets relatively to that buffer.

In preparation for enabling ROX mode for EXECMEM_BPF make sure that the allocated memory is writable.

Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for > buffer allocation in arch_bpf_trampoline_size().

---
 arch/x86/net/bpf_jit_comp.c | 5 ++---
 include/linux/filter.h      | 1 +
 kernel/bpf/core.c           | 5 +++++
 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +0 -0
index de7515ea1bea..b2feec81e231 100644
@@ -3703,13 +3703,12 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 	int ret;
 

arch/x86/net/bpf_jit_comp.c int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags, · line 3711

Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +2 -3
 	/* Allocate a temporary buffer for __arch_prepare_bpf_trampoline().
-	 * This will NOT cause fragmentation in direct map, as we do not
-	 * call set_memory_*() on this buffer.
 	 *
 	 * We cannot use kvmalloc here, because we need image to be in
 	 * module memory range.
+	 * Since it must be writable use bpf_jit_alloc_exec_rw().
 	 */
-	image = bpf_jit_alloc_exec(PAGE_SIZE);
+	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);

hi,
this change (this particular patch plus possibly others in this set) is causing tracing_multi attachment bench slowdown

the benchmark allocates huge number of trampolines and I'm seeing extra arch_bpf_trampoline_size code paths in the attached perf profile

I'm not that familiar with the allocator, but following hack makes the benchmark ok again:

Quoted 10 lines
	diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
	index 6a94370a2448..bbff3c9c6681 100644
	--- a/kernel/bpf/core.c
	+++ b/kernel/bpf/core.c
	@@ -1130,7 +1130,7 @@ void *bpf_jit_alloc_exec(unsigned long size)
	 
	 void *bpf_jit_alloc_exec_rw(unsigned long size)
	 {
	-	return execmem_alloc_rw(EXECMEM_BPF, size);
	+	return execmem_alloc(EXECMEM_MODULE_DATA, size);

This is fine on x86 because all execmem types live in the same 2G range. Other architectures may have _BPF and MODULE_DATA in different ranges, e.g arm64 and some configurations of powerpc.

Quoted 9 lines
	 }
	 
	 void bpf_jit_free_exec(void *addr)

I still need to do more checks, but I'm wondering if we could actually fix this by not allocating image data in arch_bpf_trampoline_size at all.. and just teach __arch_prepare_bpf_trampoline to survive NULL image data and just return the size in such case

Since bpf_jit_alloc_exec_rw() is only used by x86::arch_bpf_trampoline_size() I think we can directly allocate from _MODULE_DATA there and drop bpf_jit_alloc_exec_rw(), like (build tested only) patch below does.

If/when other architectures would implement arch_bpf_trampoline_size(), they'll probably need to deal with this differently.

From 14dff78529a9204408a5c14036954df5f2f76431 Mon Sep 17 00:00:00 2001
From: "Mike Rapoport (Microsoft)" <[email protected]>
Date: Sun, 16 Aug 2026 11:27:32 +0300
Subject: [PATCH] x86/bpf: make arch_bpf_trampoline_size allocate from
EXECMEM_MODULE_DATA

Jiri Olsa reports slowdown of tracing_multi benchmark that allocates huge number of trampolines [1].

The slowdown caused by extra protection changes in execmem_alloc_rw() and execmem_free().

With ROX caches enabled, all execmem allocations except EXECMEM_MODULE_DATA are ROX after the allocation. execmem_alloc_rw() temporarily sets them to W+NX and execmem_free() resets them back to ROX.

The only user of bpf_jit_alloc_exec_rw() is x86::arch_bpf_trampoline_size() that only needs a temporary writable buffer in the modules address space.

On x86 executable memory and module data are constrained to the same address range, so x86::arch_bpf_trampoline_size() can directly use execmem_alloc(EXECMEM_MODULE_DATA)

Replace the call to bpf_jit_alloc_exec_rw() with a call to execmem_alloc(EXECMEM_MODULE_DATA) in x86::arch_bpf_trampoline_size() and drop bpf_jit_alloc_exec_rw() helper.

Fixes f0334294a428 ("bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable")
---
 arch/x86/net/bpf_jit_comp.c | 8 +++++---
 include/linux/filter.h      | 1 -
 kernel/bpf/core.c           | 5 -----
 3 files changed, 5 insertions(+), 9 deletions(-)
arch/x86/net/bpf_jit_comp.c +5 -3
index b2feec81e231..7d064d3e2788 100644
@@ -13,6 +13,7 @@
 #include <linux/bpf_verifier.h>
 #include <linux/memory.h>
 #include <linux/sort.h>
+#include <linux/execmem.h>
 #include <asm/extable.h>
 #include <asm/ftrace.h>
 #include <asm/set_memory.h>
@@ -3706,15 +3707,16 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 	 *
 	 * We cannot use kvmalloc here, because we need image to be in
 	 * module memory range.
-	 * Since it must be writable use bpf_jit_alloc_exec_rw().
+	 * Since it must be writable use execmem_alloc(EXECMEM_MODULE_DATA)
+	 * that returns writable memory in the module address space.
 	 */
-	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);
+	image = execmem_alloc(EXECMEM_MODULE_DATA, PAGE_SIZE);
 	if (!image)
 		return -ENOMEM;
 
 	ret = __arch_prepare_bpf_trampoline(&im, image, image + PAGE_SIZE, image,
 					    m, flags, tnodes, func_addr);
-	bpf_jit_free_exec(image);
+	execmem_free(image);
 	return ret;
 }
 
include/linux/filter.h +0 -1
index 32d5297c557e..14acb2455746 100644
@@ -1333,7 +1333,6 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr,
 void bpf_jit_binary_free(struct bpf_binary_header *hdr);
 u64 bpf_jit_alloc_exec_limit(void);
 void *bpf_jit_alloc_exec(unsigned long size);
-void *bpf_jit_alloc_exec_rw(unsigned long size);
 void bpf_jit_free_exec(void *addr);
 void bpf_jit_free(struct bpf_prog *fp);
 struct bpf_binary_header *
kernel/bpf/core.c +0 -5
index e2076667b245..1b89c18cf246 100644
@@ -1128,11 +1128,6 @@ void *bpf_jit_alloc_exec(unsigned long size)
 	return execmem_alloc(EXECMEM_BPF, size);
 }
 
-void *bpf_jit_alloc_exec_rw(unsigned long size)
-{
-	return execmem_alloc_rw(EXECMEM_BPF, size);
-}
-
 void bpf_jit_free_exec(void *addr)
 {
 	execmem_free(addr);
Signature · 1 line
2.53.0
Quoted 2 lines

thanks,
jirka

Signature · 2 lines
Sincerely yours,
Mike.
Tested-by
#10 lore raw

Re: [PATCH bpf-next v3 4/5] bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable

On Sun, Aug 16, 2026 at 11:57:35AM +0300, Mike Rapoport wrote:

SNIP

Quoted 23 lines

From 14dff78529a9204408a5c14036954df5f2f76431 Mon Sep 17 00:00:00 2001
From: "Mike Rapoport (Microsoft)" <[email protected]>
Date: Sun, 16 Aug 2026 11:27:32 +0300
Subject: [PATCH] x86/bpf: make arch_bpf_trampoline_size allocate from
EXECMEM_MODULE_DATA

Jiri Olsa reports slowdown of tracing_multi benchmark that allocates huge number of trampolines [1].

The slowdown caused by extra protection changes in execmem_alloc_rw() and execmem_free().

With ROX caches enabled, all execmem allocations except EXECMEM_MODULE_DATA are ROX after the allocation. execmem_alloc_rw() temporarily sets them to W+NX and execmem_free() resets them back to ROX.

The only user of bpf_jit_alloc_exec_rw() is x86::arch_bpf_trampoline_size() that only needs a temporary writable buffer in the modules address space.

On x86 executable memory and module data are constrained to the same address range, so x86::arch_bpf_trampoline_size() can directly use execmem_alloc(EXECMEM_MODULE_DATA)

Quoted 8 lines

Replace the call to bpf_jit_alloc_exec_rw() with a call to execmem_alloc(EXECMEM_MODULE_DATA) in x86::arch_bpf_trampoline_size() and drop bpf_jit_alloc_exec_rw() helper.

Fixes f0334294a428 ("bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable")

looks good, thanks

serial_test_tracing_multi_bench_attach: found 55077 functions
serial_test_tracing_multi_bench_attach: attached in 1.470s
serial_test_tracing_multi_bench_attach: detached in 0.249s

jirka

Quoted 76 lines
---
 arch/x86/net/bpf_jit_comp.c | 8 +++++---
 include/linux/filter.h      | 1 -
 kernel/bpf/core.c           | 5 -----
 3 files changed, 5 insertions(+), 9 deletions(-)
arch/x86/net/bpf_jit_comp.c +5 -3
index b2feec81e231..7d064d3e2788 100644
@@ -13,6 +13,7 @@
 #include <linux/bpf_verifier.h>
 #include <linux/memory.h>
 #include <linux/sort.h>
+#include <linux/execmem.h>
 #include <asm/extable.h>
 #include <asm/ftrace.h>
 #include <asm/set_memory.h>
@@ -3706,15 +3707,16 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 	 *
 	 * We cannot use kvmalloc here, because we need image to be in
 	 * module memory range.
-	 * Since it must be writable use bpf_jit_alloc_exec_rw().
+	 * Since it must be writable use execmem_alloc(EXECMEM_MODULE_DATA)
+	 * that returns writable memory in the module address space.
 	 */
-	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);
+	image = execmem_alloc(EXECMEM_MODULE_DATA, PAGE_SIZE);
 	if (!image)
 		return -ENOMEM;
 
 	ret = __arch_prepare_bpf_trampoline(&im, image, image + PAGE_SIZE, image,
 					    m, flags, tnodes, func_addr);
-	bpf_jit_free_exec(image);
+	execmem_free(image);
 	return ret;
 }
 
include/linux/filter.h +0 -1
index 32d5297c557e..14acb2455746 100644
@@ -1333,7 +1333,6 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr,
 void bpf_jit_binary_free(struct bpf_binary_header *hdr);
 u64 bpf_jit_alloc_exec_limit(void);
 void *bpf_jit_alloc_exec(unsigned long size);
-void *bpf_jit_alloc_exec_rw(unsigned long size);
 void bpf_jit_free_exec(void *addr);
 void bpf_jit_free(struct bpf_prog *fp);
 struct bpf_binary_header *
kernel/bpf/core.c +0 -5
index e2076667b245..1b89c18cf246 100644
@@ -1128,11 +1128,6 @@ void *bpf_jit_alloc_exec(unsigned long size)
 	return execmem_alloc(EXECMEM_BPF, size);
 }
 
-void *bpf_jit_alloc_exec_rw(unsigned long size)
-{
-	return execmem_alloc_rw(EXECMEM_BPF, size);
-}
-
 void bpf_jit_free_exec(void *addr)
 {
 	execmem_free(addr);
Signature · 1 line
2.53.0
Quoted 2 lines

thanks,
jirka

Signature · 2 lines
Sincerely yours,
Mike.
maintainer
#11 lore raw

Re: [PATCH bpf-next v3 4/5] bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable

In reply to Mike Rapoport
Quoted 74 lines · Mike Rapoport
Quoted 48 lines · Jiri Olsa
Quoted 26 lines · Mike Rapoport (Microsoft)

arch_bpf_trampoline_size() allocates a buffer to get actual size required for a trampoline.

This buffer must be in the module address space because __arch_prepare_bpf_trampoline() calculates rel32 offsets relatively to that buffer.

In preparation for enabling ROX mode for EXECMEM_BPF make sure that the allocated memory is writable.

Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for > buffer allocation in arch_bpf_trampoline_size().

---
  arch/x86/net/bpf_jit_comp.c | 5 ++---
  include/linux/filter.h      | 1 +
  kernel/bpf/core.c           | 5 +++++
  3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +0 -0
index de7515ea1bea..b2feec81e231 100644
@@ -3703,13 +3703,12 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
  	int ret;
 

arch/x86/net/bpf_jit_comp.c int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags, · line 3711

Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +2 -3
  	/* Allocate a temporary buffer for __arch_prepare_bpf_trampoline().
-	 * This will NOT cause fragmentation in direct map, as we do not
-	 * call set_memory_*() on this buffer.
  	 *
  	 * We cannot use kvmalloc here, because we need image to be in
  	 * module memory range.
+	 * Since it must be writable use bpf_jit_alloc_exec_rw().
  	 */
-	image = bpf_jit_alloc_exec(PAGE_SIZE);
+	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);

hi,
this change (this particular patch plus possibly others in this set) is causing tracing_multi attachment bench slowdown

the benchmark allocates huge number of trampolines and I'm seeing extra arch_bpf_trampoline_size code paths in the attached perf profile

I'm not that familiar with the allocator, but following hack makes the benchmark ok again:

Quoted 10 lines
	diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
	index 6a94370a2448..bbff3c9c6681 100644
	--- a/kernel/bpf/core.c
	+++ b/kernel/bpf/core.c
	@@ -1130,7 +1130,7 @@ void *bpf_jit_alloc_exec(unsigned long size)
	
	 void *bpf_jit_alloc_exec_rw(unsigned long size)
	 {
	-	return execmem_alloc_rw(EXECMEM_BPF, size);
	+	return execmem_alloc(EXECMEM_MODULE_DATA, size);

This is fine on x86 because all execmem types live in the same 2G range. Other architectures may have _BPF and MODULE_DATA in different ranges, e.g arm64 and some configurations of powerpc.

Quoted 9 lines
	 }
	
	 void bpf_jit_free_exec(void *addr)

I still need to do more checks, but I'm wondering if we could actually fix this by not allocating image data in arch_bpf_trampoline_size at all.. and just teach __arch_prepare_bpf_trampoline to survive NULL image data and just return the size in such case

Quoted 7 lines

Since bpf_jit_alloc_exec_rw() is only used by x86::arch_bpf_trampoline_size() I think we can directly allocate from _MODULE_DATA there and drop bpf_jit_alloc_exec_rw(), like (build tested only) patch below does.

If/when other architectures would implement arch_bpf_trampoline_size(), they'll probably need to deal with this differently.

Mike, could you send this as a regular patch to bpf list, so it can go through BPF CI?

Thanks,
Daniel

#12 lore raw

Re: [PATCH bpf-next v3 4/5] bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable

In reply to Jiri Olsa
Quoted 63 lines · Jiri Olsa
Quoted 26 lines · Mike Rapoport (Microsoft)

arch_bpf_trampoline_size() allocates a buffer to get actual size required for a trampoline.

This buffer must be in the module address space because __arch_prepare_bpf_trampoline() calculates rel32 offsets relatively to that buffer.

In preparation for enabling ROX mode for EXECMEM_BPF make sure that the allocated memory is writable.

Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for > buffer allocation in arch_bpf_trampoline_size().

---
 arch/x86/net/bpf_jit_comp.c | 5 ++---
 include/linux/filter.h      | 1 +
 kernel/bpf/core.c           | 5 +++++
 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +0 -0
index de7515ea1bea..b2feec81e231 100644
@@ -3703,13 +3703,12 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 	int ret;
 

arch/x86/net/bpf_jit_comp.c int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags, · line 3711

Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +2 -3
 	/* Allocate a temporary buffer for __arch_prepare_bpf_trampoline().
-	 * This will NOT cause fragmentation in direct map, as we do not
-	 * call set_memory_*() on this buffer.
 	 *
 	 * We cannot use kvmalloc here, because we need image to be in
 	 * module memory range.
+	 * Since it must be writable use bpf_jit_alloc_exec_rw().
 	 */
-	image = bpf_jit_alloc_exec(PAGE_SIZE);
+	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);

hi,
this change (this particular patch plus possibly others in this set) is causing tracing_multi attachment bench slowdown

the benchmark allocates huge number of trampolines and I'm seeing extra arch_bpf_trampoline_size code paths in the attached perf profile

I'm not that familiar with the allocator, but following hack makes the benchmark ok again:

	diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
	index 6a94370a2448..bbff3c9c6681 100644
	--- a/kernel/bpf/core.c
	+++ b/kernel/bpf/core.c
	@@ -1130,7 +1130,7 @@ void *bpf_jit_alloc_exec(unsigned long size)
	 
	 void *bpf_jit_alloc_exec_rw(unsigned long size)
	 {
	-	return execmem_alloc_rw(EXECMEM_BPF, size);
	+	return execmem_alloc(EXECMEM_MODULE_DATA, size);
	 }
	 
	 void bpf_jit_free_exec(void *addr)
Quoted 4 lines

I still need to do more checks, but I'm wondering if we could actually fix this by not allocating image data in arch_bpf_trampoline_size at all.. and just teach __arch_prepare_bpf_trampoline to survive NULL image data and just return the size in such case

fyi doing ^^^ is not so straigh forward as I hoped for, but if we just allocate dummy page at init and used that in arch_bpf_trampoline_size we get some speedup

I'll rebase that on top of your fix and send it out later

with your fix:
serial_test_tracing_multi_bench_attach: found 55077 functions
serial_test_tracing_multi_bench_attach: attached in 1.470s
serial_test_tracing_multi_bench_attach: detached in 0.249s

with change below:
serial_test_tracing_multi_bench_attach: found 55079 functions
serial_test_tracing_multi_bench_attach: attached in 0.754s
serial_test_tracing_multi_bench_attach: detached in 0.247s

thanks,
jirka

---

arch/x86/net/bpf_jit_comp.c +18 -16
index 1a9fb530adc3..3e8ba33944c9 100644
@@ -9,6 +9,7 @@
 #include <linux/filter.h>
 #include <linux/if_vlan.h>
 #include <linux/bitfield.h>
+#include <linux/init.h>
 #include <linux/bpf.h>
 #include <linux/bpf_verifier.h>
 #include <linux/memory.h>
@@ -23,6 +24,19 @@
 
 static bool all_callee_regs_used[4] = {true, true, true, true};
 
+/*
+ * Reuse a writable image in the BPF execmem range for size calculation.
+ * Its contents do not affect size calculation.
+ */
+static void *trampoline_size_image;
+
+static int __init init_trampoline_size_image(void)
+{
+	trampoline_size_image = bpf_jit_alloc_exec_rw(PAGE_SIZE);
+	return trampoline_size_image ? 0 : -ENOMEM;
+}
+late_initcall(init_trampoline_size_image);
+
 static u8 *emit_code(u8 *ptr, u32 bytes, unsigned int len)
 {
 	if (len == 1)
@@ -3811,23 +3825,11 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 			     struct bpf_tramp_nodes *tnodes, void *func_addr)
 {
 	struct bpf_tramp_image im;
-	void *image;
-	int ret;
-
-	/* Allocate a temporary buffer for __arch_prepare_bpf_trampoline().
-	 *
-	 * We cannot use kvmalloc here, because we need image to be in
-	 * module memory range.
-	 * Since it must be writable use bpf_jit_alloc_exec_rw().
-	 */
-	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);
-	if (!image)
-		return -ENOMEM;
 
-	ret = __arch_prepare_bpf_trampoline(&im, image, image + PAGE_SIZE, image,
-					    m, flags, tnodes, func_addr);
-	bpf_jit_free_exec(image);
-	return ret;
+	return __arch_prepare_bpf_trampoline(&im, trampoline_size_image,
+					     trampoline_size_image + PAGE_SIZE,
+					     trampoline_size_image, m, flags,
+					     tnodes, func_addr);
 }
 
 static int emit_bpf_dispatcher(u8 **pprog, int a, int b, s64 *progs, u8 *image, u8 *buf)
#13 lore raw

PATCH v3 5/5 x86/bpf: enable EXECMEM_ROX_CACHE for BPF allocations

In reply to Mike Rapoport (Microsoft)

BPF core and x86 JIT use text poking and temporary writable buffers and thus can handle ROX memory.

Enable ROX cache for EXECMEM_BPF when configuration and CPU features allow that.

---
 arch/x86/mm/init.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/mm/init.c +2 -2
index fb67217fddcd..079f8c7e9e3c 100644
@@ -1107,10 +1107,10 @@ struct execmem_info __init *execmem_arch_setup(void)
 				.alignment = MODULE_ALIGN,
 			},
 			[EXECMEM_BPF] = {
-				.flags	= EXECMEM_KASAN_SHADOW,
+				.flags	= flags,
 				.start	= start,
 				.end	= MODULES_END,
-				.pgprot	= PAGE_KERNEL,
+				.pgprot	= pgprot,
 				.alignment = MODULE_ALIGN,
 			},
 			[EXECMEM_MODULE_DATA] = {
Signature · 1 line
2.53.0
reviewer
#14 lore raw
In reply to Mike Rapoport (Microsoft)
Quoted 7 lines · Mike Rapoport (Microsoft)

Hi,

BPF allocations of executable memory on x86 are essentially read-only. Most paths that call bpf_jit_alloc_exec() immediately make it ROX with set_memory_rox().

Quoted 5 lines

The code generation, at least on x86, uses separately allocated writable buffers and then updates the actual text memory with text_poke().

These patches do several small adjustments to how BPF allocates executable memory and enable EXECMEM_ROX_CACHE for BPF allocations on x86.

After this set, we are still using bpf_prog_pack_alloc() from x86 code. I think the goal is to eventually remove bpf_prog_pack_alloc(). What's our plan for the next steps (toward removing bpf_prog_pack_alloc)?

Thanks,
Song

#15 lore raw
Quoted 12 lines · Song Liu
Quoted 10 lines · Mike Rapoport (Microsoft)

Hi,

BPF allocations of executable memory on x86 are essentially read-only. Most paths that call bpf_jit_alloc_exec() immediately make it ROX with set_memory_rox().

The code generation, at least on x86, uses separately allocated writable buffers and then updates the actual text memory with text_poke().

Quoted 6 lines
Quoted 2 lines

These patches do several small adjustments to how BPF allocates executable memory and enable EXECMEM_ROX_CACHE for BPF allocations on x86.

After this set, we are still using bpf_prog_pack_alloc() from x86 code. I think the goal is to eventually remove bpf_prog_pack_alloc(). What's our plan for the next steps (toward removing bpf_prog_pack_alloc)?

"It works, don't touch"? ;-)

We can add another layer for sub-page allocations to execmem.

Since BPF is the only user the easiest would be just to move prog_pack logic from BPF to execmem and call it a day.

Another option is to add a slab-like layer for sub-page allocations to execmem. This is more complex but it would allow to get rid of the rigid BPF_PROG_CHUNK_SIZE.

Maybe it would be also possible to teach SLUB to use execmem_alloc() instead of alloc_pages() but that's surely the most far fetched one :)

And since we are talking about bpf_prog_pack_alloc(), why BPF_PROG_PACK_SIZE accounts for num_possible_nodes():

#define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())

Is it an elaborate choice or it was picked to work around older vmalloc_huge() limitations?

Quoted 2 lines

Thanks,
Song

Signature · 2 lines
Sincerely yours,
Mike.
reviewer
#16 lore raw
Quoted 17 lines · Mike Rapoport
Quoted 15 lines · Song Liu
Quoted 12 lines · Mike Rapoport (Microsoft)

Hi,

BPF allocations of executable memory on x86 are essentially read-only. Most paths that call bpf_jit_alloc_exec() immediately make it ROX with set_memory_rox().

The code generation, at least on x86, uses separately allocated writable buffers and then updates the actual text memory with text_poke().

These patches do several small adjustments to how BPF allocates executable memory and enable EXECMEM_ROX_CACHE for BPF allocations on x86.

Quoted 7 lines
Quoted 3 lines

After this set, we are still using bpf_prog_pack_alloc() from x86 code. I think the goal is to eventually remove bpf_prog_pack_alloc(). What's our plan for the next steps (toward removing bpf_prog_pack_alloc)?

"It works, don't touch"? ;-)

We can add another layer for sub-page allocations to execmem.

Sub-page allocation is not a hard requirement here. Using 4kB for each small BPF program isn't too bad. We added bpf_prog_pack to avoid fragmentation of direct map page table entry (caused by W^X requirement). If execmem allocator reserves large enough ROX memory (with PMD page table entries) and reuses them properly, we shouldn't see page table fragmentation getting worse over time. Then, we can use 4kB granularity allocation for BPF programs. (I am not sure about 64kB pages..).

Quoted 3 lines

Since BPF is the only user the easiest would be just to move prog_pack logic from BPF to execmem and call it a day.

If we move to bigger page sizes, say 64kB, there will be other users that would benefit from sub page allocation, right?

Quoted 6 lines

Another option is to add a slab-like layer for sub-page allocations to execmem. This is more complex but it would allow to get rid of the rigid BPF_PROG_CHUNK_SIZE.

Maybe it would be also possible to teach SLUB to use execmem_alloc() instead of alloc_pages() but that's surely the most far fetched one :)

I was thinking some rb-tree algorithm might be useful here, something similar to vmap.

Quoted 7 lines

And since we are talking about bpf_prog_pack_alloc(), why BPF_PROG_PACK_SIZE accounts for num_possible_nodes():

#define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())

Is it an elaborate choice or it was picked to work around older vmalloc_huge() limitations?

It is a bit complicated. The goal is to get PMDs for prog_pack. We can adjust this if vmalloc_huge() changes after that.

Thanks,
Song

#17 lore raw
Quoted 25 lines · Song Liu
Quoted 16 lines · Mike Rapoport
Quoted 15 lines · Song Liu
Quoted 12 lines · Mike Rapoport (Microsoft)

Hi,

BPF allocations of executable memory on x86 are essentially read-only. Most paths that call bpf_jit_alloc_exec() immediately make it ROX with set_memory_rox().

The code generation, at least on x86, uses separately allocated writable buffers and then updates the actual text memory with text_poke().

These patches do several small adjustments to how BPF allocates executable memory and enable EXECMEM_ROX_CACHE for BPF allocations on x86.

Quoted 7 lines
Quoted 3 lines

After this set, we are still using bpf_prog_pack_alloc() from x86 code. I think the goal is to eventually remove bpf_prog_pack_alloc(). What's our plan for the next steps (toward removing bpf_prog_pack_alloc)?

"It works, don't touch"? ;-)

We can add another layer for sub-page allocations to execmem.

Quoted 8 lines

Sub-page allocation is not a hard requirement here. Using 4kB for each small BPF program isn't too bad. We added bpf_prog_pack to avoid fragmentation of direct map page table entry (caused by W^X requirement). If execmem allocator reserves large enough ROX memory (with PMD page table entries) and reuses them properly, we shouldn't see page table fragmentation getting worse over time. Then, we can use 4kB granularity allocation for BPF programs. (I am not sure about 64kB pages..).

execmem uses PMD_SIZE pages for ROX cache. Since right now it's only supported on x86, the issue with oversized large page with 64k base pages didn't come up yet.

Quoted 5 lines
Quoted 2 lines

Since BPF is the only user the easiest would be just to move prog_pack logic from BPF to execmem and call it a day.

If we move to bigger page sizes, say 64kB, there will be other users that would benefit from sub page allocation, right?

Maybe modules could, don't know TBH.
There is another caveat with sub-page allocations: they must preserve ROX and no subsystem except BPF can deal with ROX-only allocations and writable copies.

Quoted 9 lines
Quoted 6 lines

Another option is to add a slab-like layer for sub-page allocations to execmem. This is more complex but it would allow to get rid of the rigid BPF_PROG_CHUNK_SIZE.

Maybe it would be also possible to teach SLUB to use execmem_alloc() instead of alloc_pages() but that's surely the most far fetched one :)

I was thinking some rb-tree algorithm might be useful here, something similar to vmap.

Could be that or maple_tree especially considering there's an active discussion about converting vmalloc to maple_tree as well:

https://lore.kernel.org/linux-mm/[email protected]/

But the choice of allocation algorithm is anyway secondary to the decision whether execmem should have the sub-page allocator.

If you say that a page per BPF program is fine, than maybe we can just rip off bpf_prog_pack_alloc().
If BPF still needs to maintain the ability to allocate in smaller chunks, then maybe the best way is to keep bpf_prog_pack_alloc() as is because I don't foresee other users for small page allocations any time soon.

Quoted 10 lines
Quoted 7 lines

And since we are talking about bpf_prog_pack_alloc(), why BPF_PROG_PACK_SIZE accounts for num_possible_nodes():

#define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())

Is it an elaborate choice or it was picked to work around older vmalloc_huge() limitations?

It is a bit complicated. The goal is to get PMDs for prog_pack. We can adjust this if vmalloc_huge() changes after that.

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless of number of nodes.

With 64k pages it's still a lot though.

Quoted 2 lines

Thanks,
Song

Signature · 2 lines
Sincerely yours,
Mike.
reviewer
#18 lore raw

On Fri, Jul 17, 2026 at 2:29 AM Mike Rapoport <[email protected]> wrote:
[...]

Quoted 14 lines
Quoted 5 lines
Quoted 4 lines

"It works, don't touch"? ;-)

We can add another layer for sub-page allocations to execmem.

Quoted 8 lines

Sub-page allocation is not a hard requirement here. Using 4kB for each small BPF program isn't too bad. We added bpf_prog_pack to avoid fragmentation of direct map page table entry (caused by W^X requirement). If execmem allocator reserves large enough ROX memory (with PMD page table entries) and reuses them properly, we shouldn't see page table fragmentation getting worse over time. Then, we can use 4kB granularity allocation for BPF programs. (I am not sure about 64kB pages..).

Quoted 3 lines

execmem uses PMD_SIZE pages for ROX cache. Since right now it's only supported on x86, the issue with oversized large page with 64k base pages didn't come up yet.

With 64kB page size, PMD_SIZE is probably too big for execmem.

Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

Since BPF is the only user the easiest would be just to move prog_pack logic from BPF to execmem and call it a day.

If we move to bigger page sizes, say 64kB, there will be other users that would benefit from sub page allocation, right?

Maybe modules could, don't know TBH.
There is another caveat with sub-page allocations: they must preserve ROX and no subsystem except BPF can deal with ROX-only allocations and writable copies.

I think ftrace and kprobe could be good users?

Quoted 32 lines
Quoted 9 lines
Quoted 6 lines

Another option is to add a slab-like layer for sub-page allocations to execmem. This is more complex but it would allow to get rid of the rigid BPF_PROG_CHUNK_SIZE.

Maybe it would be also possible to teach SLUB to use execmem_alloc() instead of alloc_pages() but that's surely the most far fetched one :)

I was thinking some rb-tree algorithm might be useful here, something similar to vmap.

Could be that or maple_tree especially considering there's an active discussion about converting vmalloc to maple_tree as well:

https://lore.kernel.org/linux-mm/[email protected]/

But the choice of allocation algorithm is anyway secondary to the decision whether execmem should have the sub-page allocator.

If you say that a page per BPF program is fine, than maybe we can just rip off bpf_prog_pack_alloc().
If BPF still needs to maintain the ability to allocate in smaller chunks, then maybe the best way is to keep bpf_prog_pack_alloc() as is because I don't foresee other users for small page allocations any time soon.

Quoted 8 lines
Quoted 7 lines

And since we are talking about bpf_prog_pack_alloc(), why BPF_PROG_PACK_SIZE accounts for num_possible_nodes():

#define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())

Is it an elaborate choice or it was picked to work around older vmalloc_huge() limitations?

Quoted 6 lines
Quoted 2 lines

It is a bit complicated. The goal is to get PMDs for prog_pack. We can adjust this if vmalloc_huge() changes after that.

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless of number of nodes.

Before execmem can handle sub page allocations, how about we send allocations that are bigger than page size directly to execmem, and let bpf_prog_pack handle sub page allocations. Then, BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net win for x86_64. Other archs will be the same. WDYT?

Thanks,
Song

#19 lore raw
Quoted 16 lines · Song Liu

On Fri, Jul 17, 2026 at 2:29 AM Mike Rapoport <[email protected]> wrote:
[...]

Quoted 14 lines
Quoted 13 lines
Quoted 4 lines

"It works, don't touch"? ;-)

We can add another layer for sub-page allocations to execmem.

Sub-page allocation is not a hard requirement here. Using 4kB for each small BPF program isn't too bad. We added bpf_prog_pack to avoid fragmentation of direct map page table entry (caused by W^X requirement). If execmem allocator reserves large enough ROX memory (with PMD page table entries) and reuses them properly, we shouldn't see page table fragmentation getting worse over time. Then, we can use 4kB granularity allocation for BPF programs. (I am not sure about 64kB pages..).

Quoted 5 lines
Quoted 3 lines

execmem uses PMD_SIZE pages for ROX cache. Since right now it's only supported on x86, the issue with oversized large page with 64k base pages didn't come up yet.

With 64kB page size, PMD_SIZE is probably too big for execmem.

Yeah, I think so. But maybe on arm64 with 64kB pages we could use CONT_PTE.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Since BPF is the only user the easiest would be just to move prog_pack logic from BPF to execmem and call it a day.

If we move to bigger page sizes, say 64kB, there will be other users that would benefit from sub page allocation, right?

Quoted 6 lines
Quoted 4 lines

Maybe modules could, don't know TBH.
There is another caveat with sub-page allocations: they must preserve ROX and no subsystem except BPF can deal with ROX-only allocations and writable copies.

I think ftrace and kprobe could be good users?

Don't know about ftrace, but kprobes essentially has page-sized caches and uses them internally.

Quoted 39 lines
Quoted 32 lines
Quoted 9 lines
Quoted 6 lines

Another option is to add a slab-like layer for sub-page allocations to execmem. This is more complex but it would allow to get rid of the rigid BPF_PROG_CHUNK_SIZE.

Maybe it would be also possible to teach SLUB to use execmem_alloc() instead of alloc_pages() but that's surely the most far fetched one :)

I was thinking some rb-tree algorithm might be useful here, something similar to vmap.

Could be that or maple_tree especially considering there's an active discussion about converting vmalloc to maple_tree as well:

https://lore.kernel.org/linux-mm/[email protected]/

But the choice of allocation algorithm is anyway secondary to the decision whether execmem should have the sub-page allocator.

If you say that a page per BPF program is fine, than maybe we can just rip off bpf_prog_pack_alloc().
If BPF still needs to maintain the ability to allocate in smaller chunks, then maybe the best way is to keep bpf_prog_pack_alloc() as is because I don't foresee other users for small page allocations any time soon.

Quoted 8 lines
Quoted 7 lines

And since we are talking about bpf_prog_pack_alloc(), why BPF_PROG_PACK_SIZE accounts for num_possible_nodes():

#define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())

Is it an elaborate choice or it was picked to work around older vmalloc_huge() limitations?

Quoted 6 lines
Quoted 2 lines

It is a bit complicated. The goal is to get PMDs for prog_pack. We can adjust this if vmalloc_huge() changes after that.

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless of number of nodes.

Quoted 5 lines

Before execmem can handle sub page allocations, how about we send allocations that are bigger than page size directly to execmem, and let bpf_prog_pack handle sub page allocations. Then, BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net win for x86_64. Other archs will be the same. WDYT?

Makes perfect sense to me for x86.

On architectures that support large mappings in the direct map and do not support EXECMEM_CACHE_ROX this will increase direct map fragmentation.

Quoted 2 lines

Thanks,
Song

Signature · 2 lines
Sincerely yours,
Mike.
reviewer
#20 lore raw
Quoted 81 lines · Mike Rapoport
Quoted 16 lines · Song Liu

On Fri, Jul 17, 2026 at 2:29 AM Mike Rapoport <[email protected]> wrote:
[...]

Quoted 14 lines
Quoted 13 lines
Quoted 4 lines

"It works, don't touch"? ;-)

We can add another layer for sub-page allocations to execmem.

Sub-page allocation is not a hard requirement here. Using 4kB for each small BPF program isn't too bad. We added bpf_prog_pack to avoid fragmentation of direct map page table entry (caused by W^X requirement). If execmem allocator reserves large enough ROX memory (with PMD page table entries) and reuses them properly, we shouldn't see page table fragmentation getting worse over time. Then, we can use 4kB granularity allocation for BPF programs. (I am not sure about 64kB pages..).

Quoted 5 lines
Quoted 3 lines

execmem uses PMD_SIZE pages for ROX cache. Since right now it's only supported on x86, the issue with oversized large page with 64k base pages didn't come up yet.

With 64kB page size, PMD_SIZE is probably too big for execmem.

Yeah, I think so. But maybe on arm64 with 64kB pages we could use CONT_PTE.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Since BPF is the only user the easiest would be just to move prog_pack logic from BPF to execmem and call it a day.

If we move to bigger page sizes, say 64kB, there will be other users that would benefit from sub page allocation, right?

Quoted 6 lines
Quoted 4 lines

Maybe modules could, don't know TBH.
There is another caveat with sub-page allocations: they must preserve ROX and no subsystem except BPF can deal with ROX-only allocations and writable copies.

I think ftrace and kprobe could be good users?

Don't know about ftrace, but kprobes essentially has page-sized caches and uses them internally.

Quoted 39 lines
Quoted 38 lines
Quoted 9 lines
Quoted 6 lines

Another option is to add a slab-like layer for sub-page allocations to execmem. This is more complex but it would allow to get rid of the rigid BPF_PROG_CHUNK_SIZE.

Maybe it would be also possible to teach SLUB to use execmem_alloc() instead of alloc_pages() but that's surely the most far fetched one :)

I was thinking some rb-tree algorithm might be useful here, something similar to vmap.

Could be that or maple_tree especially considering there's an active discussion about converting vmalloc to maple_tree as well:

https://lore.kernel.org/linux-mm/[email protected]/

But the choice of allocation algorithm is anyway secondary to the decision whether execmem should have the sub-page allocator.

If you say that a page per BPF program is fine, than maybe we can just rip off bpf_prog_pack_alloc().
If BPF still needs to maintain the ability to allocate in smaller chunks, then maybe the best way is to keep bpf_prog_pack_alloc() as is because I don't foresee other users for small page allocations any time soon.

Quoted 10 lines
Quoted 7 lines

And since we are talking about bpf_prog_pack_alloc(), why BPF_PROG_PACK_SIZE accounts for num_possible_nodes():

#define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())

Is it an elaborate choice or it was picked to work around older vmalloc_huge() limitations?

It is a bit complicated. The goal is to get PMDs for prog_pack. We can adjust this if vmalloc_huge() changes after that.

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless of number of nodes.

Quoted 7 lines
Quoted 5 lines

Before execmem can handle sub page allocations, how about we send allocations that are bigger than page size directly to execmem, and let bpf_prog_pack handle sub page allocations. Then, BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net win for x86_64. Other archs will be the same. WDYT?

Makes perfect sense to me for x86.

Can we include this change to this patch set? This will make this change a meaningful step towards migrating to EXECMEM.

Quoted 2 lines

On architectures that support large mappings in the direct map and do not support EXECMEM_CACHE_ROX this will increase direct map fragmentation.

Architectures that support bpf_prog_pack can benefit from EXECMEM_CACHE_ROX. What does it take to also enable EXECMEM_CACHE_ROX for them?

Thanks,
Song

#21 lore raw
Quoted 12 lines · Song Liu
Quoted 11 lines · Mike Rapoport
Quoted 10 lines
Quoted 4 lines

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless of number of nodes.

Before execmem can handle sub page allocations, how about we send allocations that are bigger than page size directly to execmem, and let bpf_prog_pack handle sub page allocations. Then, BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net win for x86_64. Other archs will be the same. WDYT?

Quoted 4 lines
Quoted 1 line

Makes perfect sense to me for x86.

Can we include this change to this patch set? This will make this change a meaningful step towards migrating to EXECMEM.

I'd prefer to leave this as a separate followup.

The decision whether to use EXECMEM_ROX_CACHE is made at runtime, which means that BPF_PROG_PACK_SIZE should become a variable, that variable needs initialization at some point during boot and that in turn requires a new helper in execmem that will say if EXECMEM_CACHE_ROX is enabled.

Quoted 6 lines
Quoted 2 lines

On architectures that support large mappings in the direct map and do not support EXECMEM_CACHE_ROX this will increase direct map fragmentation.

Architectures that support bpf_prog_pack can benefit from EXECMEM_CACHE_ROX. What does it take to also enable EXECMEM_CACHE_ROX for them?

For EXECMEM_ROX_CACHE to help in reducing the fragmentation and TLB pressure an architecture should support collapsing of the split mappings at least in vmalloc/modules space.

There is an RFC for arm64 to add support for EXECMEM_ROX_CACHE there:

https://lore.kernel.org/all/[email protected]

and it looks like it'll take a couple of iterations :)

I don't how much work it would be for other architectures.

Quoted 2 lines

Thanks,
Song

Signature · 2 lines
Sincerely yours,
Mike.
reviewer Acked-by
#22 lore raw
Quoted 19 lines · Mike Rapoport
Quoted 12 lines · Song Liu
Quoted 11 lines · Mike Rapoport
Quoted 10 lines
Quoted 4 lines

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless of number of nodes.

Before execmem can handle sub page allocations, how about we send allocations that are bigger than page size directly to execmem, and let bpf_prog_pack handle sub page allocations. Then, BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net win for x86_64. Other archs will be the same. WDYT?

Quoted 4 lines
Quoted 1 line

Makes perfect sense to me for x86.

Can we include this change to this patch set? This will make this change a meaningful step towards migrating to EXECMEM.

Quoted 6 lines

I'd prefer to leave this as a separate followup.

The decision whether to use EXECMEM_ROX_CACHE is made at runtime, which means that BPF_PROG_PACK_SIZE should become a variable, that variable needs initialization at some point during boot and that in turn requires a new helper in execmem that will say if EXECMEM_CACHE_ROX is enabled.

Hmm... deciding whether to use EXECMEM_ROX_CACHE at runtime seems weird. Can we enable it all the time and decide how much memory it reserves at runtime?

Given this constraint, we can probably ship this now and do the rest in a follow up set.

For the set

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

On architectures that support large mappings in the direct map and do not support EXECMEM_CACHE_ROX this will increase direct map fragmentation.

Architectures that support bpf_prog_pack can benefit from EXECMEM_CACHE_ROX. What does it take to also enable EXECMEM_CACHE_ROX for them?

[...]

#23 lore raw
Quoted 22 lines · Song Liu
Quoted 21 lines · Mike Rapoport
Quoted 12 lines · Song Liu
Quoted 11 lines · Mike Rapoport
Quoted 10 lines
Quoted 4 lines

Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless of number of nodes.

Before execmem can handle sub page allocations, how about we send allocations that are bigger than page size directly to execmem, and let bpf_prog_pack handle sub page allocations. Then, BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net win for x86_64. Other archs will be the same. WDYT?

Quoted 4 lines
Quoted 1 line

Makes perfect sense to me for x86.

Can we include this change to this patch set? This will make this change a meaningful step towards migrating to EXECMEM.

I'd prefer to leave this as a separate followup.

Quoted 8 lines
Quoted 4 lines

The decision whether to use EXECMEM_ROX_CACHE is made at runtime, which means that BPF_PROG_PACK_SIZE should become a variable, that variable needs initialization at some point during boot and that in turn requires a new helper in execmem that will say if EXECMEM_CACHE_ROX is enabled.

Hmm... deciding whether to use EXECMEM_ROX_CACHE at runtime seems weird. Can we enable it all the time and decide how much memory it reserves at runtime?

Enabling ROX cache without using PMD_SIZE mappings is pure overhead without advantage of using larger mappings for executable memory.

Quoted 2 lines

Given this constraint, we can probably ship this now and do the rest in a follow up set.

Yeah, this is something to think more about :)

Quoted 2 lines

For the set

Thanks!

Signature · 2 lines
Sincerely yours,
Mike.
#24 lore raw
In reply to Mike Rapoport (Microsoft)
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Quoted 10 lines · On Thu, 16 Jul 2026 10:51:34 +0300 you wrote:

Hi,

BPF allocations of executable memory on x86 are essentially read-only. Most paths that call bpf_jit_alloc_exec() immediately make it ROX with set_memory_rox().

The code generation, at least on x86, uses separately allocated writable buffers and then updates the actual text memory with text_poke().

[...]

Here is the summary with links:
- [bpf-next,v3,1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

    https://git.kernel.org/bpf/bpf-next/c/810a273919df
  - [bpf-next,v3,2/5] bpf: drop __weak from bpf_jit_alloc_exec() and bpf_jit_free_exec()
    https://git.kernel.org/bpf/bpf-next/c/4946eb5d37cb
  - [bpf-next,v3,3/5] bpf: alloc_prog_pack(): skip ROX management for already ROX memory
    https://git.kernel.org/bpf/bpf-next/c/7516714947da
  - [bpf-next,v3,4/5] bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable
    https://git.kernel.org/bpf/bpf-next/c/5bf02dbf39fa
  - [bpf-next,v3,5/5] x86/bpf: enable EXECMEM_ROX_CACHE for BPF allocations
    https://git.kernel.org/bpf/bpf-next/c/6b88aaec81db

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html