Skip to contentSkip to conversation

libbpf: move arena variables out of the zero

PATCH Superseded Emil Tsalapatis 3 participants 11 messages Dec 3, 2025 – Dec 8, 2025

Subsystems (5)
Kselftest BPF BPF selftests BPF core +1 more subsystems

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Emil Tsalapatis

author reviewer
#0 lore raw

v2 cover libbpf: move arena variables out of the zero

Modify libbpf to place arena globals in a small offset inside the arena mapping instead of the very beginning. This allows programs to leave the "zero page" of the arena unmapped, so that NULL arena pointer dereferences trigger a page fault and associated backtrace in BPF streams. In contrast, the current policy of placing global data in the zero pages means that NULL dereferences silently corrupt global data, e.g, arena qspinlock state. This makes arena bugs more difficult to debug.

The patchset adds code to libbpf to move global arena data to the end of the arena. At load time, libbpf adjusts each symbol's location within the arena to point to the right location in the arena. The patchset also adjusts the arena skeleton pointer to point to the arena globals, now that they are not in the beginning of the arena region.

CHANGESET
=========

v1->v2: (https://lore.kernel.org/bpf/[email protected])

- Moved globals to the end of the mapping: (Andrii)
- Removed extra parameter for offset and parameter picking logic
- Removed padding in the skeleton
- Removed additional libbpf call
- Added Reviewed-by from Eduard on patch 1

Emil Tsalapatis (4):
selftests/bpf: explicitly account for globals in verifier_arena_large bpf/verifier: do not limit maximum direct offset into arena map
libbpf: move arena globals to the end of the arena selftests/bpf: add tests for the arena offset of globals

 kernel/bpf/verifier.c                         |  8 +--
 tools/lib/bpf/libbpf.c                        | 19 ++++--
 .../selftests/bpf/prog_tests/verifier.c       |  4 ++
 .../bpf/progs/verifier_arena_globals1.c       | 58 +++++++++++++++++++
 .../bpf/progs/verifier_arena_globals2.c       | 49 ++++++++++++++++
 .../bpf/progs/verifier_arena_large.c          | 21 +++++--
 6 files changed, 147 insertions(+), 12 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_globals1.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_globals2.c
Signature · 1 line
2.49.0
author reviewer
#1 lore raw

PATCH v2 1/4 selftests/bpf: explicitly account for globals in verifier_arena_large

The big_alloc1 test in verifier_arena_large assumes that the arena base and the first page allocated by bpf_arena_alloc_pages are identical. This is not the case, because the first page in the arena is populated by global arena data. The test still passes because the code makes the tacit assumption that the first page is on offset PAGE_SIZE instead of 0.

Make this distinction explicit in the code, and adjust the page offsets requested during the test to count from the beginning of the arena instead of using the address of the first allocated page.

---
 .../selftests/bpf/progs/verifier_arena_large.c    | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/progs/verifier_arena_large.c +11 -4
index f19e15400b3e..bd430a34c3ab 100644
@@ -23,18 +23,25 @@ int big_alloc1(void *ctx)
 {
 #if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
 	volatile char __arena *page1, *page2, *no_page, *page3;
-	void __arena *base;
+	u64 base;
 
-	page1 = base = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
+	base = (u64)arena_base(&arena);
+
+	page1 = bpf_arena_alloc_pages(&arena, NULL, 1, NUMA_NO_NODE, 0);
 	if (!page1)
 		return 1;
+
+	/* Account for global arena data. */
+	if ((u64)page1 != base + PAGE_SIZE)
+		return 15;
+
 	*page1 = 1;
-	page2 = bpf_arena_alloc_pages(&arena, base + ARENA_SIZE - PAGE_SIZE * 2,
+	page2 = bpf_arena_alloc_pages(&arena, (void __arena *)(ARENA_SIZE - PAGE_SIZE),
 				      1, NUMA_NO_NODE, 0);
 	if (!page2)
 		return 2;
 	*page2 = 2;
-	no_page = bpf_arena_alloc_pages(&arena, base + ARENA_SIZE - PAGE_SIZE,
+	no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE,
 					1, NUMA_NO_NODE, 0);
 	if (no_page)
 		return 3;
Signature · 1 line
2.49.0
author reviewer
#2 lore raw

PATCH v2 2/4 bpf/verifier: do not limit maximum direct offset into arena map

In reply to Emil Tsalapatis

The verifier currently limits direct offsets into a map to 512MiB to avoid overflow during pointer arithmetic. However, this prevents arena maps from using direct addressing instructions to access data at the end of > 512MiB arena maps. This is necessary when moving arena globals to the end of the arena instead of the front.

Relax the limitation for direct offsets into arena maps to 4GiB, the maximum arena size.

---
 kernel/bpf/verifier.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/bpf/verifier.c +4 -4
index 098dd7f21c89..a64cc5caf4aa 100644
@@ -21084,13 +21084,13 @@ static int resolve_pseudo_ldimm64(struct bpf_verifier_env *env)
 			} else {
 				u32 off = insn[1].imm;
 
-				if (off >= BPF_MAX_VAR_OFF) {
-					verbose(env, "direct value offset of %u is not allowed\n", off);
+				if (!map->ops->map_direct_value_addr) {
+					verbose(env, "no direct value access support for this map type\n");
 					return -EINVAL;
 				}
 
-				if (!map->ops->map_direct_value_addr) {
-					verbose(env, "no direct value access support for this map type\n");
+				if (off >= BPF_MAX_VAR_OFF && map->map_type != BPF_MAP_TYPE_ARENA) {
+					verbose(env, "direct value offset of %u is not allowed\n", off);
 					return -EINVAL;
 				}
 
Signature · 1 line
2.49.0
#3 lore raw

Re: [PATCH v2 2/4] bpf/verifier: do not limit maximum direct offset into arena map

Quoted 26 lines · Emil Tsalapatis

The verifier currently limits direct offsets into a map to 512MiB to avoid overflow during pointer arithmetic. However, this prevents arena maps from using direct addressing instructions to access data at the end of > 512MiB arena maps. This is necessary when moving arena globals to the end of the arena instead of the front.

Relax the limitation for direct offsets into arena maps to 4GiB, the maximum arena size.

---
 kernel/bpf/verifier.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/bpf/verifier.c +1 -2
index 098dd7f21c89..a64cc5caf4aa 100644
@@ -21084,13 +21084,13 @@ static int resolve_pseudo_ldimm64(struct bpf_verifier_env *env)
                        } else {
                                u32 off = insn[1].imm;
 
-                               if (off >= BPF_MAX_VAR_OFF) {
-                                       verbose(env, "direct value offset of %u is not allowed\n", off);
+                               if (!map->ops->map_direct_value_addr) {

kernel/bpf/verifier.c static int resolve_pseudo_ldimm64(struct bpf_verifier_env *env) · line 21095

Quoted 10 lines
kernel/bpf/verifier.c +3 -2
+                                       verbose(env, "no direct value access support for this map type\n");
                                        return -EINVAL;
                                }
 
-                               if (!map->ops->map_direct_value_addr) {
-                                       verbose(env, "no direct value access support for this map type\n");
+                               if (off >= BPF_MAX_VAR_OFF && map->map_type != BPF_MAP_TYPE_ARENA) {
+                                       verbose(env, "direct value offset of %u is not allowed\n", off);
                                        return -EINVAL;
                                }

both arena and array maps validate off for correct range, not sure we even need this check here?

Quoted 4 lines
kernel/bpf/verifier.c +0 -1
--

2.49.0

author reviewer
#4 lore raw

PATCH v2 3/4 libbpf: move arena globals to the end of the arena

In reply to Emil Tsalapatis

Arena globals are currently placed at the beginning of the arena by libbpf. This is convenient, but prevents users from reserving guard pages in the beginning of the arena to identify NULL pointer dereferences. Adjust the load logic to place the globals at the end of the arena instead.

Also modify bpftool to set the arena pointer in the program's BPF skeleton to point to the globals. Users now call bpf_map__initial_value() to find the beginning of the arena mapping and use the arena pointer in the skeleton to determine which part of the mapping holds the arena globals and which part is free.

---
 tools/lib/bpf/libbpf.c                        | 19 +++++++++++++++----
 .../bpf/progs/verifier_arena_large.c          | 12 +++++++++---
 2 files changed, 24 insertions(+), 7 deletions(-)
tools/lib/bpf/libbpf.c +15 -4
index 706e7481bdf6..9642d697b690 100644
@@ -757,6 +757,7 @@ struct bpf_object {
 	int arena_map_idx;
 	void *arena_data;
 	size_t arena_data_sz;
+	__u32 arena_data_off;
 
 	void *jumptables_data;
 	size_t jumptables_data_sz;
@@ -2991,10 +2992,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
 			       void *data, size_t data_sz)
 {
 	const long page_sz = sysconf(_SC_PAGE_SIZE);
+	const size_t data_alloc_sz = roundup(data_sz, page_sz);
 	size_t mmap_sz;
 
 	mmap_sz = bpf_map_mmap_sz(map);
-	if (roundup(data_sz, page_sz) > mmap_sz) {
+	if (data_alloc_sz > mmap_sz) {
 		pr_warn("elf: sec '%s': declared ARENA map size (%zu) is too small to hold global __arena variables of size %zu\n",
 			sec_name, mmap_sz, data_sz);
 		return -E2BIG;
@@ -3006,6 +3008,9 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
 	memcpy(obj->arena_data, data, data_sz);
 	obj->arena_data_sz = data_sz;
 
+	/* place globals at the end of the arena */
+	obj->arena_data_off = mmap_sz - data_alloc_sz;
+
 	/* make bpf_map__init_value() work for ARENA maps */
 	map->mmaped = obj->arena_data;
 
@@ -4663,7 +4668,7 @@ static int bpf_program__record_reloc(struct bpf_program *prog,
 		reloc_desc->type = RELO_DATA;
 		reloc_desc->insn_idx = insn_idx;
 		reloc_desc->map_idx = obj->arena_map_idx;
-		reloc_desc->sym_off = sym->st_value;
+		reloc_desc->sym_off = sym->st_value + obj->arena_data_off;
 
 		map = &obj->maps[obj->arena_map_idx];
 		pr_debug("prog '%s': found arena map %d (%s, sec %d, off %zu) for insn %u\n",
@@ -5624,7 +5629,8 @@ bpf_object__create_maps(struct bpf_object *obj)
 					return err;
 				}
 				if (obj->arena_data) {
-					memcpy(map->mmaped, obj->arena_data, obj->arena_data_sz);
+					memcpy(map->mmaped + obj->arena_data_off, obj->arena_data,
+						obj->arena_data_sz);
 					zfree(&obj->arena_data);
 				}
 			}
@@ -14387,6 +14393,7 @@ void bpf_object__destroy_subskeleton(struct bpf_object_subskeleton *s)
 
 int bpf_object__load_skeleton(struct bpf_object_skeleton *s)
 {
+	void *mmaped;
 	int i, err;
 
 	err = bpf_object__load(*s->obj);
@@ -14402,7 +14409,11 @@ int bpf_object__load_skeleton(struct bpf_object_skeleton *s)
 		if (!map_skel->mmaped)
 			continue;
 
-		*map_skel->mmaped = map->mmaped;
+		mmaped = map->mmaped;
+		if (map->def.type == BPF_MAP_TYPE_ARENA)
+			mmaped += map->obj->arena_data_off;
+
+		*map_skel->mmaped = mmaped;
 	}
 
 	return 0;
tools/testing/selftests/bpf/progs/verifier_arena_large.c +9 -3
index bd430a34c3ab..2b8cf2a4d880 100644
@@ -31,16 +31,22 @@ int big_alloc1(void *ctx)
 	if (!page1)
 		return 1;
 
-	/* Account for global arena data. */
-	if ((u64)page1 != base + PAGE_SIZE)
+	if ((u64)page1 != base)
 		return 15;
 
 	*page1 = 1;
-	page2 = bpf_arena_alloc_pages(&arena, (void __arena *)(ARENA_SIZE - PAGE_SIZE),
+	page2 = bpf_arena_alloc_pages(&arena, (void __arena *)(ARENA_SIZE - 2 * PAGE_SIZE),
 				      1, NUMA_NO_NODE, 0);
 	if (!page2)
 		return 2;
 	*page2 = 2;
+
+	/* Test for the guard region at the end of the arena. */
+	no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE - PAGE_SIZE,
+					1, NUMA_NO_NODE, 0);
+	if (no_page)
+		return 16;
+
 	no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE,
 					1, NUMA_NO_NODE, 0);
 	if (no_page)
Signature · 1 line
2.49.0
#5 lore raw

Re: [PATCH v2 3/4] libbpf: move arena globals to the end of the arena

Here obj->arena_data_off is a __u32, but the calculation is done in size_t (64-bit). For a 4GB arena (which the test verifier_arena_large.c uses), mmap_sz is 4GB and the result can approach 4GB…
Quoted 4 lines
tools/lib/bpf/libbpf.c +0 -0
index 706e7481b..9642d697b 100644

[ ... ]

Quoted 14 lines
tools/lib/bpf/libbpf.c +2 -1
@@ -2991,10 +2992,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
 			       void *data, size_t data_sz)
 {
 	const long page_sz = sysconf(_SC_PAGE_SIZE);
+	const size_t data_alloc_sz = roundup(data_sz, page_sz);
 	size_t mmap_sz;
 
 	mmap_sz = bpf_map_mmap_sz(map);
-	if (roundup(data_sz, page_sz) > mmap_sz) {
+	if (data_alloc_sz > mmap_sz) {
 		pr_warn("elf: sec '%s': declared ARENA map size (%zu) is too small to hold global __arena variables of size %zu\n",
 			sec_name, mmap_sz, data_sz);
 		return -E2BIG;

tools/lib/bpf/libbpf.c static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map, · line 3013

Quoted 6 lines
tools/lib/bpf/libbpf.c +3 -0
@@ -3006,6 +3008,9 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
 	memcpy(obj->arena_data, data, data_sz);
 	obj->arena_data_sz = data_sz;
 
+	/* place globals at the end of the arena */
+	obj->arena_data_off = mmap_sz - data_alloc_sz;
+

Here obj->arena_data_off is a __u32, but the calculation is done in size_t (64-bit). For a 4GB arena (which the test verifier_arena_large.c uses), mmap_sz is 4GB and the result can approach 4GB - PAGE_SIZE. This fits in u32 correctly.

tools/lib/bpf/libbpf.c static int bpf_program__record_reloc(struct bpf_program *prog, · line 4671

Quoted 6 lines
tools/lib/bpf/libbpf.c +1 -1
@@ -4663,7 +4668,7 @@ static int bpf_program__record_reloc(struct bpf_program *prog,
 		reloc_desc->type = RELO_DATA;
 		reloc_desc->insn_idx = insn_idx;
 		reloc_desc->map_idx = obj->arena_map_idx;
-		reloc_desc->sym_off = sym->st_value;
+		reloc_desc->sym_off = sym->st_value + obj->arena_data_off;
                                                 ^^^^^^^^^^^^^^^^^^^^^^

Can this overflow when storing to reloc_desc->sym_off? The sym_off field is 'int' (signed 32-bit), but arena_data_off can be larger than INT_MAX for arenas approaching 4GB in size.

For example, with a 4GB arena and one page of globals:
- mmap_sz = 4GB = 0x100000000
- data_alloc_sz = 4096
- arena_data_off = 0xFFFFF000 = 4294963200

This value (4294963200) exceeds INT_MAX (2147483647), so when stored in the signed int sym_off field, it becomes negative.

Later in bpf_object__relocate_data(), this is used as:

    insn[1].imm = insn[0].imm + relo->sym_off;

With sym_off being negative due to overflow, this produces an incorrect offset for accessing arena globals.

Should sym_off be changed to an unsigned type to handle large arena offsets correctly?

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19901383089

#6 lore raw

Re: [PATCH v2 3/4] libbpf: move arena globals to the end of the arena

Quoted 54 lines · On Wed, Dec 3, 2025 at 8:52 AM <[email protected]> wrote:
Quoted 5 lines
tools/lib/bpf/libbpf.c +0 -0
index 706e7481b..9642d697b 100644

[ ... ]

Quoted 14 lines
tools/lib/bpf/libbpf.c +2 -1
@@ -2991,10 +2992,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
                             void *data, size_t data_sz)
 {
      const long page_sz = sysconf(_SC_PAGE_SIZE);
+     const size_t data_alloc_sz = roundup(data_sz, page_sz);
      size_t mmap_sz;
 
      mmap_sz = bpf_map_mmap_sz(map);
-     if (roundup(data_sz, page_sz) > mmap_sz) {
+     if (data_alloc_sz > mmap_sz) {
              pr_warn("elf: sec '%s': declared ARENA map size (%zu) is too small to hold global __arena variables of size %zu\n",
                      sec_name, mmap_sz, data_sz);
              return -E2BIG;

tools/lib/bpf/libbpf.c static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map, · line 3013

Quoted 6 lines
tools/lib/bpf/libbpf.c +3 -0
@@ -3006,6 +3008,9 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
      memcpy(obj->arena_data, data, data_sz);
      obj->arena_data_sz = data_sz;
 
+     /* place globals at the end of the arena */
+     obj->arena_data_off = mmap_sz - data_alloc_sz;
+

Here obj->arena_data_off is a __u32, but the calculation is done in size_t (64-bit). For a 4GB arena (which the test verifier_arena_large.c uses), mmap_sz is 4GB and the result can approach 4GB - PAGE_SIZE. This fits in u32 correctly.

tools/lib/bpf/libbpf.c static int bpf_program__record_reloc(struct bpf_program *prog, · line 4671

Quoted 6 lines
tools/lib/bpf/libbpf.c +1 -1
@@ -4663,7 +4668,7 @@ static int bpf_program__record_reloc(struct bpf_program *prog,
              reloc_desc->type = RELO_DATA;
              reloc_desc->insn_idx = insn_idx;
              reloc_desc->map_idx = obj->arena_map_idx;
-             reloc_desc->sym_off = sym->st_value;
+             reloc_desc->sym_off = sym->st_value + obj->arena_data_off;
                                                 ^^^^^^^^^^^^^^^^^^^^^^

Can this overflow when storing to reloc_desc->sym_off? The sym_off field is 'int' (signed 32-bit), but arena_data_off can be larger than INT_MAX for arenas approaching 4GB in size.

For example, with a 4GB arena and one page of globals:
- mmap_sz = 4GB = 0x100000000
- data_alloc_sz = 4096
- arena_data_off = 0xFFFFF000 = 4294963200

This value (4294963200) exceeds INT_MAX (2147483647), so when stored in the signed int sym_off field, it becomes negative.

Quoted 8 lines

Later in bpf_object__relocate_data(), this is used as:

    insn[1].imm = insn[0].imm + relo->sym_off;

With sym_off being negative due to overflow, this produces an incorrect offset for accessing arena globals.

Should sym_off be changed to an unsigned type to handle large arena offsets correctly?

AI has good a point, valid offset with 4GB arena will overflow int... But then ldimm64 instructions that allow direct access to arena have insn[1].imm as s32, so they cannot even address more than 2GB? Can you please double check this, maybe we actually treat insn[1].imm as u32 on the verifier side for this use case?

Quoted 9 lines

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19901383089

author reviewer
#7 lore raw

Re: [PATCH v2 3/4] libbpf: move arena globals to the end of the arena

Quoted 62 lines · Andrii Nakryiko
Quoted 60 lines · On Wed, Dec 3, 2025 at 8:52 AM <[email protected]> wrote:
Quoted 5 lines
tools/lib/bpf/libbpf.c +0 -0
index 706e7481b..9642d697b 100644

[ ... ]

Quoted 14 lines
tools/lib/bpf/libbpf.c +2 -1
@@ -2991,10 +2992,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
                             void *data, size_t data_sz)
 {
      const long page_sz = sysconf(_SC_PAGE_SIZE);
+     const size_t data_alloc_sz = roundup(data_sz, page_sz);
      size_t mmap_sz;
 
      mmap_sz = bpf_map_mmap_sz(map);
-     if (roundup(data_sz, page_sz) > mmap_sz) {
+     if (data_alloc_sz > mmap_sz) {
              pr_warn("elf: sec '%s': declared ARENA map size (%zu) is too small to hold global __arena variables of size %zu\n",
                      sec_name, mmap_sz, data_sz);
              return -E2BIG;

tools/lib/bpf/libbpf.c static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map, · line 3013

Quoted 6 lines
tools/lib/bpf/libbpf.c +3 -0
@@ -3006,6 +3008,9 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
      memcpy(obj->arena_data, data, data_sz);
      obj->arena_data_sz = data_sz;
 
+     /* place globals at the end of the arena */
+     obj->arena_data_off = mmap_sz - data_alloc_sz;
+

Here obj->arena_data_off is a __u32, but the calculation is done in size_t (64-bit). For a 4GB arena (which the test verifier_arena_large.c uses), mmap_sz is 4GB and the result can approach 4GB - PAGE_SIZE. This fits in u32 correctly.

tools/lib/bpf/libbpf.c static int bpf_program__record_reloc(struct bpf_program *prog, · line 4671

Quoted 6 lines
tools/lib/bpf/libbpf.c +1 -1
@@ -4663,7 +4668,7 @@ static int bpf_program__record_reloc(struct bpf_program *prog,
              reloc_desc->type = RELO_DATA;
              reloc_desc->insn_idx = insn_idx;
              reloc_desc->map_idx = obj->arena_map_idx;
-             reloc_desc->sym_off = sym->st_value;
+             reloc_desc->sym_off = sym->st_value + obj->arena_data_off;
                                                 ^^^^^^^^^^^^^^^^^^^^^^

Can this overflow when storing to reloc_desc->sym_off? The sym_off field is 'int' (signed 32-bit), but arena_data_off can be larger than INT_MAX for arenas approaching 4GB in size.

For example, with a 4GB arena and one page of globals:
- mmap_sz = 4GB = 0x100000000
- data_alloc_sz = 4096
- arena_data_off = 0xFFFFF000 = 4294963200

This value (4294963200) exceeds INT_MAX (2147483647), so when stored in the signed int sym_off field, it becomes negative.

Later in bpf_object__relocate_data(), this is used as:

    insn[1].imm = insn[0].imm + relo->sym_off;

With sym_off being negative due to overflow, this produces an incorrect offset for accessing arena globals.

Quoted 9 lines
Quoted 2 lines

Should sym_off be changed to an unsigned type to handle large arena offsets correctly?

AI has good a point, valid offset with 4GB arena will overflow int... But then ldimm64 instructions that allow direct access to arena have insn[1].imm as s32, so they cannot even address more than 2GB? Can you please double check this, maybe we actually treat insn[1].imm as u32 on the verifier side for this use case?

I can confirm we treat insn[1].imm as a u32, but the following relocation logic in
bpf_object__relocate_data looks suspect:

                case RELO_DATA:
                        map = &obj->maps[relo->map_idx];
                        insn[1].imm = insn[0].imm + relo->sym_off;
                        if (obj->gen_loader) {

If sym_off is interpreted as a negative, then insn[1].imm should be completely wrong and typecasting
back to u32 should do nothing. I added tests with both nonzero insn[0].imm and st_value, and none
of them are failing which unless I"m missing something is honestly kind of surprising.

I don't think sym_off should actually ever be negative. The st_value from the ELF headers
should be always positive, and elf_find_func_offset_* also returns only positive numbers.

Should we change sym_off in reloc_desc to be a u32? If anything that is more accurate than having it
as an int.

For the rest of the feedback: Ack, will incorporate it into the next patch.

Quoted 9 lines
Quoted 9 lines

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19901383089

#8 lore raw

Re: [PATCH v2 3/4] libbpf: move arena globals to the end of the arena

In reply to Emil Tsalapatis
Quoted 25 lines · Emil Tsalapatis

Arena globals are currently placed at the beginning of the arena by libbpf. This is convenient, but prevents users from reserving guard pages in the beginning of the arena to identify NULL pointer dereferences. Adjust the load logic to place the globals at the end of the arena instead.

Also modify bpftool to set the arena pointer in the program's BPF skeleton to point to the globals. Users now call bpf_map__initial_value() to find the beginning of the arena mapping and use the arena pointer in the skeleton to determine which part of the mapping holds the arena globals and which part is free.

---
 tools/lib/bpf/libbpf.c                        | 19 +++++++++++++++----
 .../bpf/progs/verifier_arena_large.c          | 12 +++++++++---
 2 files changed, 24 insertions(+), 7 deletions(-)
tools/lib/bpf/libbpf.c +0 -0
index 706e7481bdf6..9642d697b690 100644

tools/lib/bpf/libbpf.c struct bpf_object { · line 760

Quoted 4 lines
tools/lib/bpf/libbpf.c +1 -0
@@ -757,6 +757,7 @@ struct bpf_object {
        int arena_map_idx;
        void *arena_data;
        size_t arena_data_sz;
+       __u32 arena_data_off;

nit: use size_t, please

Quoted 54 lines
tools/lib/bpf/libbpf.c +9 -3
        void *jumptables_data;
        size_t jumptables_data_sz;
@@ -2991,10 +2992,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
                               void *data, size_t data_sz)
 {
        const long page_sz = sysconf(_SC_PAGE_SIZE);
+       const size_t data_alloc_sz = roundup(data_sz, page_sz);
        size_t mmap_sz;
 
        mmap_sz = bpf_map_mmap_sz(map);
-       if (roundup(data_sz, page_sz) > mmap_sz) {
+       if (data_alloc_sz > mmap_sz) {
                pr_warn("elf: sec '%s': declared ARENA map size (%zu) is too small to hold global __arena variables of size %zu\n",
                        sec_name, mmap_sz, data_sz);
                return -E2BIG;
@@ -3006,6 +3008,9 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,
        memcpy(obj->arena_data, data, data_sz);
        obj->arena_data_sz = data_sz;
 
+       /* place globals at the end of the arena */
+       obj->arena_data_off = mmap_sz - data_alloc_sz;
+
        /* make bpf_map__init_value() work for ARENA maps */
        map->mmaped = obj->arena_data;
 
@@ -4663,7 +4668,7 @@ static int bpf_program__record_reloc(struct bpf_program *prog,
                reloc_desc->type = RELO_DATA;
                reloc_desc->insn_idx = insn_idx;
                reloc_desc->map_idx = obj->arena_map_idx;
-               reloc_desc->sym_off = sym->st_value;
+               reloc_desc->sym_off = sym->st_value + obj->arena_data_off;
 
                map = &obj->maps[obj->arena_map_idx];
                pr_debug("prog '%s': found arena map %d (%s, sec %d, off %zu) for insn %u\n",
@@ -5624,7 +5629,8 @@ bpf_object__create_maps(struct bpf_object *obj)
                                        return err;
                                }
                                if (obj->arena_data) {
-                                       memcpy(map->mmaped, obj->arena_data, obj->arena_data_sz);
+                                       memcpy(map->mmaped + obj->arena_data_off, obj->arena_data,
+                                               obj->arena_data_sz);
                                        zfree(&obj->arena_data);
                                }
                        }
@@ -14387,6 +14393,7 @@ void bpf_object__destroy_subskeleton(struct bpf_object_subskeleton *s)
 
 int bpf_object__load_skeleton(struct bpf_object_skeleton *s)
 {
+       void *mmaped;
        int i, err;
 
        err = bpf_object__load(*s->obj);

tools/lib/bpf/libbpf.c int bpf_object__load_skeleton(struct bpf_object_skeleton *s) · line 14416

Quoted 9 lines
tools/lib/bpf/libbpf.c +5 -1
@@ -14402,7 +14409,11 @@ int bpf_object__load_skeleton(struct bpf_object_skeleton *s)
                if (!map_skel->mmaped)
                        continue;
 
-               *map_skel->mmaped = map->mmaped;
+               mmaped = map->mmaped;
+               if (map->def.type == BPF_MAP_TYPE_ARENA)
+                       mmaped += map->obj->arena_data_off;
+
+               *map_skel->mmaped = mmaped;

this is minor, but I think doing

if (map->def.type == BPF_MAP_TYPE_ARENA)

    /* arena data is placed at the end of arena memory region */
    *map_skel->mmaped = map->mmaped + map->obj->arena_data_off;
else
    *map_skel->mmaped = map->mmaped;

would be a bit cleaner and easier to follow and no need for extra mutable variable

Quoted 36 lines
tools/lib/bpf/libbpf.c +0 -0
        }
 
        return 0;
tools/testing/selftests/bpf/progs/verifier_arena_large.c +9 -4
index bd430a34c3ab..2b8cf2a4d880 100644
@@ -31,16 +31,22 @@ int big_alloc1(void *ctx)
        if (!page1)
                return 1;
 
-       /* Account for global arena data. */
-       if ((u64)page1 != base + PAGE_SIZE)
+       if ((u64)page1 != base)
                return 15;
 
        *page1 = 1;
-       page2 = bpf_arena_alloc_pages(&arena, (void __arena *)(ARENA_SIZE - PAGE_SIZE),
+       page2 = bpf_arena_alloc_pages(&arena, (void __arena *)(ARENA_SIZE - 2 * PAGE_SIZE),
                                      1, NUMA_NO_NODE, 0);
        if (!page2)
                return 2;
        *page2 = 2;
+
+       /* Test for the guard region at the end of the arena. */
+       no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE - PAGE_SIZE,
+                                       1, NUMA_NO_NODE, 0);
+       if (no_page)
+               return 16;
+
        no_page = bpf_arena_alloc_pages(&arena, (void __arena *)ARENA_SIZE,
                                        1, NUMA_NO_NODE, 0);
        if (no_page)
--

2.49.0

author reviewer
#9 lore raw

PATCH v2 4/4 selftests/bpf: add tests for the arena offset of globals

In reply to Emil Tsalapatis

Add tests for the new libbpf globals arena offset logic. The tests cover all three cases: The globals being small enough to be placed at the maximum possible offset, being as large as the arena itself and being placed at the very beginning, and requiring an intermediate offset into the arena.

---
 .../selftests/bpf/prog_tests/verifier.c       |  4 ++
 .../bpf/progs/verifier_arena_globals1.c       | 58 +++++++++++++++++++
 .../bpf/progs/verifier_arena_globals2.c       | 49 ++++++++++++++++
 3 files changed, 111 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_globals1.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_globals2.c
tools/testing/selftests/bpf/prog_tests/verifier.c +4 -0
index 4b4b081b46cc..5829ffd70f8f 100644
@@ -6,6 +6,8 @@
 #include "verifier_and.skel.h"
 #include "verifier_arena.skel.h"
 #include "verifier_arena_large.skel.h"
+#include "verifier_arena_globals1.skel.h"
+#include "verifier_arena_globals2.skel.h"
 #include "verifier_array_access.skel.h"
 #include "verifier_async_cb_context.skel.h"
 #include "verifier_basic_stack.skel.h"
@@ -147,6 +149,8 @@ static void run_tests_aux(const char *skel_name,
 void test_verifier_and(void)                  { RUN(verifier_and); }
 void test_verifier_arena(void)                { RUN(verifier_arena); }
 void test_verifier_arena_large(void)          { RUN(verifier_arena_large); }
+void test_verifier_arena_globals1(void)       { RUN(verifier_arena_globals1); }
+void test_verifier_arena_globals2(void)       { RUN(verifier_arena_globals2); }
 void test_verifier_basic_stack(void)          { RUN(verifier_basic_stack); }
 void test_verifier_bitfield_write(void)       { RUN(verifier_bitfield_write); }
 void test_verifier_bounds(void)               { RUN(verifier_bounds); }
tools/testing/selftests/bpf/progs/verifier_arena_globals1.c new file +58 -0

58 lines not on this page · raw message

tools/testing/selftests/bpf/progs/verifier_arena_globals2.c new file +49 -0

49 lines not on this page · raw message

Signature · 1 line
2.49.0
#10 lore raw
In reply to Emil Tsalapatis
Quoted 3 lines · Emil Tsalapatis

Modify libbpf to place arena globals in a small offset inside the arena mapping instead of the very beginning. This allows programs to leave

Description is now out of sync with the actual approach?

Quoted 44 lines

the "zero page" of the arena unmapped, so that NULL arena pointer dereferences trigger a page fault and associated backtrace in BPF streams. In contrast, the current policy of placing global data in the zero pages means that NULL dereferences silently corrupt global data, e.g, arena qspinlock state. This makes arena bugs more difficult to debug.

The patchset adds code to libbpf to move global arena data to the end of the arena. At load time, libbpf adjusts each symbol's location within the arena to point to the right location in the arena. The patchset also adjusts the arena skeleton pointer to point to the arena globals, now that they are not in the beginning of the arena region.

CHANGESET
=========

v1->v2: (https://lore.kernel.org/bpf/[email protected])

- Moved globals to the end of the mapping: (Andrii)
- Removed extra parameter for offset and parameter picking logic
- Removed padding in the skeleton
- Removed additional libbpf call
- Added Reviewed-by from Eduard on patch 1

Emil Tsalapatis (4):
selftests/bpf: explicitly account for globals in verifier_arena_large bpf/verifier: do not limit maximum direct offset into arena map
libbpf: move arena globals to the end of the arena selftests/bpf: add tests for the arena offset of globals

 kernel/bpf/verifier.c                         |  8 +--
 tools/lib/bpf/libbpf.c                        | 19 ++++--
 .../selftests/bpf/prog_tests/verifier.c       |  4 ++
 .../bpf/progs/verifier_arena_globals1.c       | 58 +++++++++++++++++++
 .../bpf/progs/verifier_arena_globals2.c       | 49 ++++++++++++++++
 .../bpf/progs/verifier_arena_large.c          | 21 +++++--
 6 files changed, 147 insertions(+), 12 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_globals1.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_arena_globals2.c

--
2.49.0