Skip to contentSkip to conversation

bpf: expand BPF arena kernel fault handling

PATCH Superseded Emil Tsalapatis 5 participants 9 messages Oct 28 – Nov 3

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Emil Tsalapatis

author reviewer
#0 lore raw

cover bpf: expand BPF arena kernel fault handling

Currently, the BPF arena fault handler protects against a misbehaving BPF program passing an arena pointer to a kfunc then unmapping the backing pages, causing a kernel fault. However, it does not handle errors where the now unmapped arena address is instead accessed by a non-BPF path, or by a BPF program that has a different arena. The former case is already possible in sched-ext if a BPF program removes from the arena pages allocated from the genpool allocator.

Keep a list of all arenas in the system and attempt to match the kernel fault to all of them. If the fault does fall in an arena, handle it with the already established scratch page technique and retry the access.

Emil Tsalapatis (2):
bpf: recover page faults across all arenas selftests/bpf: add arena fault recovery tests

 kernel/bpf/arena.c                            | 54 +++++++----
 tools/testing/selftests/bpf/Makefile          |  2 +-
 .../selftests/bpf/prog_tests/arena_fault.c    | 91 +++++++++++++++++++
 .../testing/selftests/bpf/progs/arena_fault.c | 55 +++++++++++
 .../selftests/bpf/progs/arena_fault_noarena.c | 17 ++++
 .../testing/selftests/bpf/test_kmods/Makefile |  2 +-
 .../bpf/test_kmods/bpf_test_arena_fault.c     | 55 +++++++++++
 .../bpf/test_kmods/bpf_test_arena_fault.h     | 10 ++
 8 files changed, 265 insertions(+), 21 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_fault.c
 create mode 100644 tools/testing/selftests/bpf/progs/arena_fault.c
 create mode 100644 tools/testing/selftests/bpf/progs/arena_fault_noarena.c
 create mode 100644 tools/testing/selftests/bpf/test_kmods/bpf_test_arena_fault.c
 create mode 100644 tools/testing/selftests/bpf/test_kmods/bpf_test_arena_fault.h
Signature · 1 line
2.54.0
author reviewer
#1 lore raw

PATCH 1/2 bpf: recover page faults across all arenas

The current arena fault handling code accounts for misbehaving BPF programs that free arena memory while it is being accessed by a kfunc as part of another BPF program's execution. This scenario triggers a kernel page fault that is intercepted by BPF arena-specific code. The code confirms that the fault is within the BPF program's arena, installs a scratch page, and marks the fault as handled. This is necessary to prevent BPF programs from being able to trigger an oops.

However, the protection against misbehaving BPF programs is currently incomplete: Arena memory is accessed by kernel code not just as part of a kfunc, but also from non-BPF paths. This is the case with sched-ext that includes non-BPF code paths that access arena pointers. These paths do not have an arena associated with them, and a fault there causes an oops. Such a fault can be triggered by a misbehaving BPF program by unmapping the pages nominally owned by the genpool allocator used for cid management.

Extend the kernel arena fault handling to consider all arenas in the system. The handler now scans a global arena list till it finds an arena that includes the address, and fixes up the address. Retain the stream-based reporting code for when the faulting call stack includes the BPF program that owns the arena.

Fixes 9eca087deb0b ("sched_ext: Sub-allocator over kernel-claimed BPF arena pages")
Cc Tejun Heo <[email protected]>
---
 kernel/bpf/arena.c | 54 ++++++++++++++++++++++++++++++----------------
 1 file changed, 35 insertions(+), 19 deletions(-)
kernel/bpf/arena.c +35 -19
index 868589f07330..992a0b123449 100644
@@ -10,6 +10,7 @@
 #include <linux/btf_ids.h>
 #include <linux/vmalloc.h>
 #include <linux/pagemap.h>
+#include <linux/rculist.h>
 #include <asm/tlbflush.h>
 #include "range_tree.h"
 
@@ -46,10 +47,14 @@
 #define GUARD_SZ round_up(1ull << sizeof_field(struct bpf_insn, off) * 8, PAGE_SIZE << 1)
 #define KERN_VM_SZ (SZ_4G + GUARD_SZ)
 
+static LIST_HEAD(arena_list);
+static DEFINE_SPINLOCK(arena_list_lock);
+
 static void arena_free_pages(struct bpf_arena *arena, long uaddr, long page_cnt, bool sleepable);
 
 struct bpf_arena {
 	struct bpf_map map;
+	struct list_head list;
 	u64 user_vm_start;
 	u64 user_vm_end;
 	struct vm_struct *kern_vm;
@@ -332,6 +337,10 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
 	if (err)
 		goto err_destroy_rt;
 
+	spin_lock(&arena_list_lock);
+	list_add_tail_rcu(&arena->list, &arena_list);
+	spin_unlock(&arena_list_lock);
+
 	return &arena->map;
 
 err_destroy_rt:
@@ -385,6 +394,12 @@ static void arena_map_free(struct bpf_map *map)
 	if (WARN_ON_ONCE(!list_empty(&arena->vma_list)))
 		return;
 
+	spin_lock(&arena_list_lock);
+	list_del_rcu(&arena->list);
+	spin_unlock(&arena_list_lock);
+
+	synchronize_rcu();
+
 	/* Ensure no pending deferred frees */
 	irq_work_sync(&arena->free_irq);
 	flush_work(&arena->free_work);
@@ -1207,31 +1222,32 @@ bool bpf_arena_handle_page_fault(unsigned long addr, bool is_write, unsigned lon
 	struct bpf_prog *prog;
 	unsigned long kbase;
 	unsigned long page_addr = addr & PAGE_MASK;
-
-	prog = bpf_prog_find_from_stack();
-	if (!prog)
-		return false;
-
-	arena = prog->aux->arena;
-	/* a prog not using arena may be on stack, so arena can be NULL */
-	if (!arena)
-		return false;
-
-	kbase = bpf_arena_get_kern_vm_start(arena);
+	bool handled = false;
 
 	/*
 	 * Recovery covers the 4 GiB mappable band plus the upper half-guard.
 	 * Lower guard is unreachable from kfuncs; an address there indicates
 	 * a different bug class - leave it to the regular kernel oops path.
 	 */
-	if (page_addr < kbase || page_addr >= kbase + SZ_4G + GUARD_SZ / 2)
-		return false;
-
-	apply_to_page_range(&init_mm, page_addr, PAGE_SIZE,
-			    apply_range_set_scratch_cb, arena->scratch_page);
-	flush_vmap_cache(page_addr, PAGE_SIZE);
-	__bpf_prog_report_arena_violation(prog, is_write, page_addr - kbase, fault_ip);
-	return true;
+	rcu_read_lock();
+	list_for_each_entry_rcu(arena, &arena_list, list) {
+		kbase = bpf_arena_get_kern_vm_start(arena);
+		if (page_addr < kbase || page_addr >= kbase + SZ_4G + GUARD_SZ / 2)
+			continue;
+
+		apply_to_page_range(&init_mm, page_addr, PAGE_SIZE,
+				    apply_range_set_scratch_cb, arena->scratch_page);
+		flush_vmap_cache(page_addr, PAGE_SIZE);
+
+		prog = bpf_prog_find_from_stack();
+		if (prog && prog->aux->arena == arena)
+			__bpf_prog_report_arena_violation(prog, is_write,
+							  page_addr - kbase, fault_ip);
+		handled = true;
+		break;
+	}
+	rcu_read_unlock();
+	return handled;
 }
 
 void bpf_prog_report_arena_violation(bool write, unsigned long addr, unsigned long fault_ip)
Signature · 1 line
2.54.0
#2 lore raw

Re: [PATCH bpf 1/2] bpf: recover page faults across all arenas

TL;DR: Commit message only: the Fixes: tag may point at the wrong commit; the arena-fault oops looks like it first became reachable in abdc2516f100, not 9eca087deb0b.
Quoted 1 line

bpf: recover page faults across all arenas

TL;DR: Commit message only: the Fixes: tag may point at the wrong commit; the arena-fault oops looks like it first became reachable in abdc2516f100, not 9eca087deb0b.

Quoted 31 lines

bpf: recover page faults across all arenas

The current arena fault handling code accounts for misbehaving BPF programs that free arena memory while it is being accessed by a kfunc as part of another BPF program's execution. This scenario triggers a kernel page fault that is intercepted by BPF arena-specific code. The code confirms that the fault is within the BPF program's arena, installs a scratch page, and marks the fault as handled. This is necessary to prevent BPF programs from being able to trigger an oops.

However, the protection against misbehaving BPF programs is currently incomplete: Arena memory is accessed by kernel code not just as part of a kfunc, but also from non-BPF paths. This is the case with sched-ext that includes non-BPF code paths that access arena pointers. These paths do not have an arena associated with them, and a fault there causes an oops. Such a fault can be triggered by a misbehaving BPF program by unmapping the pages nominally owned by the genpool allocator used for cid management.

Extend the kernel arena fault handling to consider all arenas in the system. The handler now scans a global arena list till it finds an arena that includes the address, and fixes up the address. Retain the stream-based reporting code for when the faulting call stack includes the BPF program that owns the arena.

Quoted 3 lines
Fixes 9eca087deb0b ("sched_ext: Sub-allocator over kernel-claimed BPF arena pages")
Cc Tejun Heo <[email protected]>

Is the Fixes: tag pointing at the right commit?

9eca087deb0b only adds a gen_pool whose bookkeeping bitmap lives in ordinary kernel memory (chunk->bits), and at that commit scx_arena_alloc() has no callers. Nothing there dereferences arena memory from a non-BPF path.

The oops described above looks like it first became reachable in abdc2516f100, where scx_call_op_set_cpumask() calls scx_cpumask_to_cmask() to write into the per-CPU arena cmask through the kern_va mapping. That happens with only the rq lock held and no BPF program on the stack, so bpf_arena_handle_page_fault() returns false because bpf_prog_find_from_stack() finds no program. The pool is first used via scx_set_cmask_scratch_alloc().

The fault handler itself came from dc11a4dba246 ("bpf: Recover arena kernel faults with scratch page"), but its limitation only matters once a non-BPF path touches arena memory.

Should this be:

Fixes abdc2516f100 ("sched_ext: Convert ops.set_cmask() to arena-resident cmask")

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40619190018

#3 lore raw

Re: [PATCH bpf 1/2] bpf: recover page faults across all arenas

In reply to Emil Tsalapatis
Quoted 75 lines · Emil Tsalapatis

The current arena fault handling code accounts for misbehaving BPF programs that free arena memory while it is being accessed by a kfunc as part of another BPF program's execution. This scenario triggers a kernel page fault that is intercepted by BPF arena-specific code. The code confirms that the fault is within the BPF program's arena, installs a scratch page, and marks the fault as handled. This is necessary to prevent BPF programs from being able to trigger an oops.

However, the protection against misbehaving BPF programs is currently incomplete: Arena memory is accessed by kernel code not just as part of a kfunc, but also from non-BPF paths. This is the case with sched-ext that includes non-BPF code paths that access arena pointers. These paths do not have an arena associated with them, and a fault there causes an oops. Such a fault can be triggered by a misbehaving BPF program by unmapping the pages nominally owned by the genpool allocator used for cid management.

Extend the kernel arena fault handling to consider all arenas in the system. The handler now scans a global arena list till it finds an arena that includes the address, and fixes up the address. Retain the stream-based reporting code for when the faulting call stack includes the BPF program that owns the arena.

Fixes 9eca087deb0b ("sched_ext: Sub-allocator over kernel-claimed BPF arena pages")
Cc Tejun Heo <[email protected]>
---
 kernel/bpf/arena.c | 54 ++++++++++++++++++++++++++++++----------------
 1 file changed, 35 insertions(+), 19 deletions(-)
kernel/bpf/arena.c +9 -0
index 868589f07330..992a0b123449 100644
@@ -10,6 +10,7 @@
 #include <linux/btf_ids.h>
 #include <linux/vmalloc.h>
 #include <linux/pagemap.h>
+#include <linux/rculist.h>
 #include <asm/tlbflush.h>
 #include "range_tree.h"
 
@@ -46,10 +47,14 @@
 #define GUARD_SZ round_up(1ull << sizeof_field(struct bpf_insn, off) * 8, PAGE_SIZE << 1)
 #define KERN_VM_SZ (SZ_4G + GUARD_SZ)
 
+static LIST_HEAD(arena_list);
+static DEFINE_SPINLOCK(arena_list_lock);
+
 static void arena_free_pages(struct bpf_arena *arena, long uaddr, long page_cnt, bool sleepable);
 
 struct bpf_arena {
 	struct bpf_map map;
+	struct list_head list;
 	u64 user_vm_start;
 	u64 user_vm_end;
 	struct vm_struct *kern_vm;
@@ -332,6 +337,10 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
 	if (err)
 		goto err_destroy_rt;
 
+	spin_lock(&arena_list_lock);
+	list_add_tail_rcu(&arena->list, &arena_list);
+	spin_unlock(&arena_list_lock);
+
 	return &arena->map;
 
 err_destroy_rt:

kernel/bpf/arena.c static void arena_map_free(struct bpf_map *map) · line 401

Quoted 8 lines
kernel/bpf/arena.c +5 -0
@@ -385,6 +394,12 @@ static void arena_map_free(struct bpf_map *map)
 	if (WARN_ON_ONCE(!list_empty(&arena->vma_list)))
 		return;
 
+	spin_lock(&arena_list_lock);
+	list_del_rcu(&arena->list);
+	spin_unlock(&arena_list_lock);
+
+	synchronize_rcu();

we tried to avoid sync_rcu() as much as possible.
Find a way to do it differently.

kernel/bpf/arena.c bool bpf_arena_handle_page_fault(unsigned long addr, bool is_write, unsigned lon · line 1211

Quoted 10 lines
kernel/bpf/arena.c +1 -2
+
 	/* Ensure no pending deferred frees */
 	irq_work_sync(&arena->free_irq);
 	flush_work(&arena->free_work);
@@ -1207,31 +1222,32 @@ bool bpf_arena_handle_page_fault(unsigned long addr, bool is_write, unsigned lon
 	struct bpf_prog *prog;
 	unsigned long kbase;
 	unsigned long page_addr = addr & PAGE_MASK;
-
-	prog = bpf_prog_find_from_stack();

This has to stay. If there are thousands arena. Above should still be a fast path. Only for sched-ext arena access it should search link link.

Quoted 28 lines
kernel/bpf/arena.c +5 -17
-	if (!prog)
-		return false;
-
-	arena = prog->aux->arena;
-	/* a prog not using arena may be on stack, so arena can be NULL */
-	if (!arena)
-		return false;
-
-	kbase = bpf_arena_get_kern_vm_start(arena);
+	bool handled = false;
 
 	/*
 	 * Recovery covers the 4 GiB mappable band plus the upper half-guard.
 	 * Lower guard is unreachable from kfuncs; an address there indicates
 	 * a different bug class - leave it to the regular kernel oops path.
 	 */
-	if (page_addr < kbase || page_addr >= kbase + SZ_4G + GUARD_SZ / 2)
-		return false;
-
-	apply_to_page_range(&init_mm, page_addr, PAGE_SIZE,
-			    apply_range_set_scratch_cb, arena->scratch_page);
-	flush_vmap_cache(page_addr, PAGE_SIZE);
-	__bpf_prog_report_arena_violation(prog, is_write, page_addr - kbase, fault_ip);
-	return true;
+	rcu_read_lock();
+	list_for_each_entry_rcu(arena, &arena_list, list) {
+		kbase = bpf_arena_get_kern_vm_start(arena);
+		if (page_addr < kbase || page_addr >= kbase + SZ_4G + GUARD_SZ / 2)

kernel/bpf/arena.c bool bpf_arena_handle_page_fault(unsigned long addr, bool is_write, unsigned lon · line 1245

Quoted 10 lines
kernel/bpf/arena.c +10 -0
+			continue;
+
+		apply_to_page_range(&init_mm, page_addr, PAGE_SIZE,
+				    apply_range_set_scratch_cb, arena->scratch_page);
+		flush_vmap_cache(page_addr, PAGE_SIZE);
+
+		prog = bpf_prog_find_from_stack();
+		if (prog && prog->aux->arena == arena)
+			__bpf_prog_report_arena_violation(prog, is_write,
+							  page_addr - kbase, fault_ip);

There has to be reporting mechanism for sched-ext faults too. Just silence is not an option.

pw-bot: cr

#4 lore raw

Re: [PATCH bpf 1/2] bpf: recover page faults across all arenas

Quoted 89 lines · Alexei Starovoitov
Quoted 75 lines · Emil Tsalapatis

The current arena fault handling code accounts for misbehaving BPF programs that free arena memory while it is being accessed by a kfunc as part of another BPF program's execution. This scenario triggers a kernel page fault that is intercepted by BPF arena-specific code. The code confirms that the fault is within the BPF program's arena, installs a scratch page, and marks the fault as handled. This is necessary to prevent BPF programs from being able to trigger an oops.

However, the protection against misbehaving BPF programs is currently incomplete: Arena memory is accessed by kernel code not just as part of a kfunc, but also from non-BPF paths. This is the case with sched-ext that includes non-BPF code paths that access arena pointers. These paths do not have an arena associated with them, and a fault there causes an oops. Such a fault can be triggered by a misbehaving BPF program by unmapping the pages nominally owned by the genpool allocator used for cid management.

Extend the kernel arena fault handling to consider all arenas in the system. The handler now scans a global arena list till it finds an arena that includes the address, and fixes up the address. Retain the stream-based reporting code for when the faulting call stack includes the BPF program that owns the arena.

Fixes 9eca087deb0b ("sched_ext: Sub-allocator over kernel-claimed BPF arena pages")
Cc Tejun Heo <[email protected]>
---
 kernel/bpf/arena.c | 54 ++++++++++++++++++++++++++++++----------------
 1 file changed, 35 insertions(+), 19 deletions(-)
kernel/bpf/arena.c +9 -0
index 868589f07330..992a0b123449 100644
@@ -10,6 +10,7 @@
 #include <linux/btf_ids.h>
 #include <linux/vmalloc.h>
 #include <linux/pagemap.h>
+#include <linux/rculist.h>
 #include <asm/tlbflush.h>
 #include "range_tree.h"
 
@@ -46,10 +47,14 @@
 #define GUARD_SZ round_up(1ull << sizeof_field(struct bpf_insn, off) * 8, PAGE_SIZE << 1)
 #define KERN_VM_SZ (SZ_4G + GUARD_SZ)
 
+static LIST_HEAD(arena_list);
+static DEFINE_SPINLOCK(arena_list_lock);
+
 static void arena_free_pages(struct bpf_arena *arena, long uaddr, long page_cnt, bool sleepable);
 
 struct bpf_arena {
      struct bpf_map map;
+     struct list_head list;
      u64 user_vm_start;
      u64 user_vm_end;
      struct vm_struct *kern_vm;
@@ -332,6 +337,10 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
      if (err)
              goto err_destroy_rt;
 
+     spin_lock(&arena_list_lock);
+     list_add_tail_rcu(&arena->list, &arena_list);
+     spin_unlock(&arena_list_lock);
+
      return &arena->map;
 
 err_destroy_rt:

kernel/bpf/arena.c static void arena_map_free(struct bpf_map *map) · line 401

Quoted 8 lines
kernel/bpf/arena.c +5 -0
@@ -385,6 +394,12 @@ static void arena_map_free(struct bpf_map *map)
      if (WARN_ON_ONCE(!list_empty(&arena->vma_list)))
              return;
 
+     spin_lock(&arena_list_lock);
+     list_del_rcu(&arena->list);
+     spin_unlock(&arena_list_lock);
+
+     synchronize_rcu();

we tried to avoid sync_rcu() as much as possible.
Find a way to do it differently.

Quoted 13 lines

kernel/bpf/arena.c bool bpf_arena_handle_page_fault(unsigned long addr, bool is_write, unsigned lon · line 1211

Quoted 10 lines
kernel/bpf/arena.c +1 -2
+
      /* Ensure no pending deferred frees */
      irq_work_sync(&arena->free_irq);
      flush_work(&arena->free_work);
@@ -1207,31 +1222,32 @@ bool bpf_arena_handle_page_fault(unsigned long addr, bool is_write, unsigned lon
      struct bpf_prog *prog;
      unsigned long kbase;
      unsigned long page_addr = addr & PAGE_MASK;
-
-     prog = bpf_prog_find_from_stack();

This has to stay. If there are thousands arena. Above should still be a fast path. Only for sched-ext arena access it should search link link.

We cannot rely on this, there might be a different program in the call stack. That's why there is an equality check on the arena when actually reporting, to make sure it's going to the right program.

The better way to address the scalability problem is to use a different data structure. I presume that replacing the list with a rbtree_latch should work. Updates will be infrequent compared to walk, so RCU+seqlock should suffice for consistent look ups. That's ~10 steps for a 1000, ~20 for a million, so it should suffice.

Quoted 44 lines
Quoted 29 lines
kernel/bpf/arena.c +5 -17
-     if (!prog)
-             return false;
-
-     arena = prog->aux->arena;
-     /* a prog not using arena may be on stack, so arena can be NULL */
-     if (!arena)
-             return false;
-
-     kbase = bpf_arena_get_kern_vm_start(arena);
+     bool handled = false;
 
      /*
       * Recovery covers the 4 GiB mappable band plus the upper half-guard.
       * Lower guard is unreachable from kfuncs; an address there indicates
       * a different bug class - leave it to the regular kernel oops path.
       */
-     if (page_addr < kbase || page_addr >= kbase + SZ_4G + GUARD_SZ / 2)
-             return false;
-
-     apply_to_page_range(&init_mm, page_addr, PAGE_SIZE,
-                         apply_range_set_scratch_cb, arena->scratch_page);
-     flush_vmap_cache(page_addr, PAGE_SIZE);
-     __bpf_prog_report_arena_violation(prog, is_write, page_addr - kbase, fault_ip);
-     return true;
+     rcu_read_lock();
+     list_for_each_entry_rcu(arena, &arena_list, list) {
+             kbase = bpf_arena_get_kern_vm_start(arena);
+             if (page_addr < kbase || page_addr >= kbase + SZ_4G + GUARD_SZ / 2)

kernel/bpf/arena.c bool bpf_arena_handle_page_fault(unsigned long addr, bool is_write, unsigned lon · line 1245

Quoted 10 lines
kernel/bpf/arena.c +10 -0
+                     continue;
+
+             apply_to_page_range(&init_mm, page_addr, PAGE_SIZE,
+                                 apply_range_set_scratch_cb, arena->scratch_page);
+             flush_vmap_cache(page_addr, PAGE_SIZE);
+
+             prog = bpf_prog_find_from_stack();
+             if (prog && prog->aux->arena == arena)
+                     __bpf_prog_report_arena_violation(prog, is_write,
+                                                       page_addr - kbase, fault_ip);

There has to be reporting mechanism for sched-ext faults too. Just silence is not an option.

pw-bot: cr

#5 lore raw

Re: [PATCH bpf 1/2] bpf: recover page faults across all arenas

Quoted 8 lines · Kumar Kartikeya Dwivedi
Quoted 4 lines
Quoted 1 line

- prog = bpf_prog_find_from_stack();

This has to stay. If there are thousands arena. Above should still be a fast path. Only for sched-ext arena access it should search link link.

We cannot rely on this, there might be a different program in the call stack. That's why there is an equality check on the arena when actually reporting, to make sure it's going to the right program.

I meant it as a fast path, not as the only path. the equality check is needed, of course.
A different prog on the stack will fail that check and the handler will do latch search.

author reviewer
#6 lore raw

PATCH 2/2 selftests/bpf: add arena fault recovery tests

In reply to Emil Tsalapatis

Test the arena fault handling code with two new selftests. The first one confirms that a program without an arena (and by extension a code path that doesn't involve BPF at all) properly recovers after accessing an unbacked arena page. The second test deploys the same configuration for a program that has its own arena map.

---
 tools/testing/selftests/bpf/Makefile          |  2 +-
 .../selftests/bpf/prog_tests/arena_fault.c    | 91 +++++++++++++++++++
 .../testing/selftests/bpf/progs/arena_fault.c | 55 +++++++++++
 .../selftests/bpf/progs/arena_fault_noarena.c | 17 ++++
 .../testing/selftests/bpf/test_kmods/Makefile |  2 +-
 .../bpf/test_kmods/bpf_test_arena_fault.c     | 55 +++++++++++
 .../bpf/test_kmods/bpf_test_arena_fault.h     | 10 ++
 7 files changed, 230 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_fault.c
 create mode 100644 tools/testing/selftests/bpf/progs/arena_fault.c
 create mode 100644 tools/testing/selftests/bpf/progs/arena_fault_noarena.c
 create mode 100644 tools/testing/selftests/bpf/test_kmods/bpf_test_arena_fault.c
 create mode 100644 tools/testing/selftests/bpf/test_kmods/bpf_test_arena_fault.h
tools/testing/selftests/bpf/Makefile +1 -1
index 1e4af62e8312..97d4a266abed 100644
@@ -47,7 +47,7 @@ TEST_PROGS_EXTENDED := \
 	ima_setup.sh verify_sig_setup.sh
 
 TEST_KMODS := bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \
-	bpf_test_modorder_y.ko bpf_test_rqspinlock.ko
+	bpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_arena_fault.ko
 TEST_KMOD_TARGETS = $(addprefix $(OUTPUT)/,$(TEST_KMODS))
 
 # Compile but not part of 'make run_tests'
tools/testing/selftests/bpf/prog_tests/arena_fault.c new file +91 -0

91 lines not on this page · raw message

tools/testing/selftests/bpf/progs/arena_fault.c new file +55 -0

55 lines not on this page · raw message

tools/testing/selftests/bpf/progs/arena_fault_noarena.c new file +17 -0
new file mode 100644
index 000000000000..7fa8814808b5
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define BPF_NO_KFUNC_PROTOTYPES
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_test_arena_fault.h"
+
+volatile u64 observed;
+
+SEC("syscall")
+int fault_stashed_noarena(void *ctx)
+{
+	observed = bpf_test_arena_fault_load_stashed();
+	return observed != 0;
+}
+
+char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/test_kmods/Makefile +1 -1
index 031c7454ce65..e611e71fcef1 100644
@@ -19,7 +19,7 @@ Q = @
 endif
 
 MODULES = bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \
-	bpf_test_modorder_y.ko bpf_test_rqspinlock.ko
+	bpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_arena_fault.ko
 
 $(foreach m,$(MODULES),$(eval obj-m += $(m:.ko=.o)))
 
tools/testing/selftests/bpf/test_kmods/bpf_test_arena_fault.c new file +55 -0

55 lines not on this page · raw message

tools/testing/selftests/bpf/test_kmods/bpf_test_arena_fault.h new file +10 -0
new file mode 100644
index 000000000000..1657c445a0fa
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#ifndef __BPF_TEST_ARENA_FAULT_H
+#define __BPF_TEST_ARENA_FAULT_H
+
+void bpf_test_arena_fault_stash(__u64 *ptr__arena) __ksym;
+__u64 bpf_test_arena_fault_load_stashed(void) __ksym;
+void bpf_test_arena_fault_clear_stashed(void) __ksym;
+
+#endif /* __BPF_TEST_ARENA_FAULT_H */
Signature · 1 line
2.54.0
#7 lore raw

Re: [PATCH bpf 2/2] selftests/bpf: add arena fault recovery tests

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] selftests/bpf: unguarded arena dereference in arena_fault breaks no_alu32/bpf_gcc --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] selftests/bpf: unguarded arena dereference in arena_fault breaks no_alu32/bpf_gcc --

commit 35e23b7a99b662f29a7ebcdbbc98176bb46d1150
Author: Emil Tsalapatis <[email protected]>

selftests/bpf: add arena fault recovery tests

This commit adds two BPF selftests and a supporting test kernel module to verify arena page fault recovery when accessing an unbacked arena page from a program with no arena or a program with a different arena map.

Quoted 38 lines
tools/testing/selftests/bpf/progs/arena_fault.c new file +32 -0
new file mode 100644
index 0000000000000..1967b128fdf69
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define BPF_NO_KFUNC_PROTOTYPES
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_arena_common.h"
+#include "../test_kmods/bpf_test_arena_fault.h"
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARENA);
+	__uint(map_flags, BPF_F_MMAPABLE);
+	__uint(max_entries, 2);
+} arena SEC(".maps");
+
+u64 __arena arena_pad;
+volatile u64 observed;
+
+SEC("syscall")
+int stash_and_free(void *ctx)
+{
+	u64 __arena *val;
+
+	val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+	if (!val)
+		return 1;
+
+	*val = 42;
+	bpf_test_arena_fault_stash((u64 *)val);
+	bpf_arena_free_pages(&arena, val, 1);
+	return 0;
+}
+

tools/testing/selftests/bpf/progs/arena_fault.c line 42

Quoted 10 lines
tools/testing/selftests/bpf/progs/arena_fault.c new file +10 -0
+SEC("syscall")
+int fault_stashed_wrong_arena(void *ctx)
+{
+	u64 __arena *val;
+
+	val = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+	if (!val)
+		return 1;
+
+	*val = 1;

[Severity: Medium]
Will stash_and_free() and fault_stashed_wrong_arena() fail BPF verification under test_progs-no_alu32 and test_progs-bpf_gcc when __BPF_FEATURE_ADDR_SPACE_CAST is not defined?

In tools/testing/selftests/bpf/libarena/include/bpf_arena_common.h, __arena only includes __attribute__((address_space(1))) when __BPF_FEATURE_ADDR_SPACE_CAST is defined:

#if defined(__BPF_FEATURE_ADDR_SPACE_CAST) && !defined(BPF_ARENA_FORCE_ASM) #define __arena __attribute__((address_space(1))) __attribute__((btf_type_tag("arena")))
...
#else
#define __arena __attribute__((btf_type_tag("arena")))

When tools/testing/selftests/bpf/Makefile builds test_progs-no_alu32 (with -mcpu=v2) or test_progs-bpf_gcc, __BPF_FEATURE_ADDR_SPACE_CAST is not defined. Without __attribute__((address_space(1))), the compiler does not emit a BPF_ADDR_SPACE_CAST instruction before dereferencing val in *val = 42 and *val = 1.

Because bpf_arena_alloc_pages() returns void * in kernel BTF, check_kfunc_call() in kernel/bpf/verifier.c leaves R0 typed as SCALAR_VALUE until BPF_ADDR_SPACE_CAST converts it to PTR_TO_ARENA in check_alu_op(). Without that cast, check_mem_access() rejects the stores to *val as invalid scalar memory accesses, causing arena_fault__open_and_load() in test_wrong_arena() and test_no_arena() in tools/testing/selftests/bpf/prog_tests/arena_fault.c to fail instead of skipping.

Should the arena pointer dereferences in progs/arena_fault.c be guarded by #if defined(__BPF_FEATURE_ADDR_SPACE_CAST) (with the runner skipping when unsupported), similar to other arena selftests?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#8 lore raw

Re: [PATCH bpf 2/2] selftests/bpf: add arena fault recovery tests

In reply to Emil Tsalapatis
Quoted 4 lines · Emil Tsalapatis
 .../testing/selftests/bpf/test_kmods/Makefile |  2 +-
 .../bpf/test_kmods/bpf_test_arena_fault.c     | 55 +++++++++++
 .../bpf/test_kmods/bpf_test_arena_fault.h     | 10 ++
 7 files changed, 230 insertions(+), 2 deletions(-)

in addition to what sashiko found...

No new kmods pls.
We already have a bunch.

imo bpf-next is fine for this bug.