Skip to contentSkip to conversation

bpf: Expose pinned arenas as sized bpffs files

PATCH In review Andrea Righi 6 participants 23 messages Oct 10 – Oct 19

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover bpf: Expose pinned arenas as sized bpffs files

As discussed in the BPF track and the sched_ext microconference at Linux Plumbers 2026 [1][2], shared BPF arenas can provide a generic channel for bidirectional zero-copy communication between BPF programs running on a host and its guests. One potential use is to exchange scheduling hints between host and guest sched_ext schedulers through shared memory.

The goal is to let both schedulers access the same underlying pages as BPF arena memory. Exchanging hints then requires no copies between separate host and guest buffers. With the arena file interface proposed here, the setup uses existing QEMU/KVM memory backends and guest NUMA interfaces, without further kernel or hypervisor changes to implement the communication channel.

A host scheduler's userspace component can create and pin an arena map, then use an arena allocator to partition it into per-guest chunks. Userspace passes each chunk to QEMU as a shared file-backed memory backend, using a bounded offset and length in the sized bpffs file. QEMU exposes only that chunk as a NUMA node inside the corresponding guest; the rest of the host arena is not mapped into guest RAM.

The guest scheduler can create its own arena and allocate pages from that NUMA node. Those guest physical pages are backed by the host arena, so host and guest BPF programs can access the same memory through their respective arenas. After identifying the corresponding arena offsets, both sides exchange data through direct BPF loads and stores. Arena pointers are not relocated between mappings; a shared protocol can use validated slice-relative offsets instead.

QEMU is one potential consumer. The interface is generic: it exposes a pinned BPF arena as a sized bpffs file that ordinary userspace consumers can open and mmap, including consumers that require a file size when setting up shared memory.

The series separates page retention from file export:

* BPF_F_ARENA_NO_FREE makes BPF page-free operations no-ops, retaining allocated pages until the map is destroyed. This flag alone preserves ordinary arena pin behavior.
* BPF_F_ARENA_EXPORT exposes the pin as a sized file and requires BPF_F_ARENA_NO_FREE, so BPF programs cannot remove backing pages while external consumers may still use them.

The map FD retains the canonical BPF address range. Exported mappings can select different virtual addresses and map disjoint slices. Faults resolve by file offset, and page zaps cover only the overlapping range in each exported VMA. Consumers establish the canonical range through map_extra or a full-capacity map FD mapping before mapping the pin. For exported arenas, the canonical range matches the fixed file size; ordinary and retention-only arenas still allow shorter mappings.

Pin permissions distinguish readers from writers. O_RDONLY consumers can create read-only shared mappings, but cannot request writable mappings or acquire write permission through mprotect(). Private mappings and file size changes are rejected. Open files and mappings retain the arena after unpinning until the last reference is released. The capacity is initialized before inode publication and can be queried with fstat() or lseek(fd, 0, SEEK_END).

Exported mappings retain the arena's lifecycle restrictions: they are not inherited across fork and cannot be split, expanded, or relocated. The documentation describes these limits and native tests exercise them.

Access through open() and mmap() intentionally uses filesystem permissions and the applicable file, mmap, and BPF map LSM checks. olicies specific to the bpf() syscall, including seccomp filters on that syscall, do not govern this file interface.

[1] https://lpc.events/event/20/contributions/2429/
[2] https://lpc.events/event/20/contributions/2482/

Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/arighi/linux.git bpf-pinned-arena

Andrea Righi (5):
bpf: Add an arena flag to retain allocated pages
bpf: Expose pinned arenas as sized bpffs files docs/bpf: Document pinned arena file mappings selftests/bpf: Test pinned arena file mappings selftests/bpf: Test two-way arena access with two guests

 Documentation/bpf/map_arena.rst               | 131 +++++
 include/uapi/linux/bpf.h                      |   6 +
 kernel/bpf/arena.c                            |  75 ++-
 kernel/bpf/inode.c                            |  95 +++-
 tools/include/uapi/linux/bpf.h                |   6 +
 tools/testing/selftests/bpf/.gitignore        |   2 +
 tools/testing/selftests/bpf/Makefile          |  27 +-
 tools/testing/selftests/bpf/arena_kvm.py      | 448 ++++++++++++++++++
 .../selftests/bpf/arena_kvm_guest.bpf.c       |  89 ++++
 tools/testing/selftests/bpf/arena_kvm_guest.c | 224 +++++++++
 .../selftests/bpf/arena_kvm_host.bpf.c        |  89 ++++
 tools/testing/selftests/bpf/arena_kvm_host.c  | 147 ++++++
 .../testing/selftests/bpf/arena_kvm_shared.h  |  60 +++
 .../selftests/bpf/prog_tests/arena_pinned.c   | 442 +++++++++++++++++
 14 files changed, 1819 insertions(+), 22 deletions(-)
 create mode 100644 Documentation/bpf/map_arena.rst
 create mode 100755 tools/testing/selftests/bpf/arena_kvm.py
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_guest.bpf.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_guest.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_host.bpf.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_host.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_shared.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_pinned.c
Signature · 1 line
2.56.0
#1 lore raw

PATCH 1/5 bpf: Add an arena flag to retain allocated pages

Add BPF_F_ARENA_NO_FREE to retain arena pages until the map is destroyed.

Make BPF page-free operations no-ops in both sleepable and non-sleepable programs when the flag is set. Programs can manage reusable objects within the retained pages without releasing their backing memory.

This provides a page-retention policy independently of how userspace accesses the arena. It also supplies stable backing for external consumers that must not lose pages while they are still using them.

---
 include/uapi/linux/bpf.h       | 3 +++
 kernel/bpf/arena.c             | 9 ++++++---
 tools/include/uapi/linux/bpf.h | 3 +++
 3 files changed, 12 insertions(+), 3 deletions(-)
include/uapi/linux/bpf.h +3 -0
index e0ed44b1bbcb7..75101d602bd32 100644
@@ -1500,6 +1500,9 @@ enum {
 
 /* Enable BPF ringbuf overwrite mode */
 	BPF_F_RB_OVERWRITE	= (1U << 19),
+
+	/* Keep arena pages allocated until the map is destroyed. */
+	BPF_F_ARENA_NO_FREE	= (1U << 20),
 };
 
 /* Flags for BPF_PROG_QUERY. */
kernel/bpf/arena.c +6 -3
index 0ff707707da04..343e84139ec29 100644
@@ -283,7 +283,8 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
 	    /* BPF_F_MMAPABLE must be set */
 	    !(attr->map_flags & BPF_F_MMAPABLE) ||
 	    /* No unsupported flags present */
-	    (attr->map_flags & ~(BPF_F_SEGV_ON_FAULT | BPF_F_MMAPABLE | BPF_F_NO_USER_CONV)))
+	    (attr->map_flags & ~(BPF_F_SEGV_ON_FAULT | BPF_F_MMAPABLE |
+				 BPF_F_NO_USER_CONV | BPF_F_ARENA_NO_FREE)))
 		return ERR_PTR(-EINVAL);
 
 	if (attr->map_extra & ~PAGE_MASK)
@@ -1134,7 +1135,8 @@ __bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign ||
+	    (map->map_flags & BPF_F_ARENA_NO_FREE))
 		return;
 	arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
 }
@@ -1144,7 +1146,8 @@ void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_c
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign ||
+	    (map->map_flags & BPF_F_ARENA_NO_FREE))
 		return;
 	arena_free_pages(arena, (long)ptr__ign, page_cnt, false);
 }
tools/include/uapi/linux/bpf.h +3 -0
index e0ed44b1bbcb7..75101d602bd32 100644
@@ -1500,6 +1500,9 @@ enum {
 
 /* Enable BPF ringbuf overwrite mode */
 	BPF_F_RB_OVERWRITE	= (1U << 19),
+
+	/* Keep arena pages allocated until the map is destroyed. */
+	BPF_F_ARENA_NO_FREE	= (1U << 20),
 };
 
 /* Flags for BPF_PROG_QUERY. */
Signature · 1 line
2.56.0
#2 lore raw

Re: [PATCH 1/5] bpf: Add an arena flag to retain allocated pages

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] bpf: BPF_F_ARENA_NO_FREE breaks virtual address unreservation and kernel error rollbacks --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] bpf: BPF_F_ARENA_NO_FREE breaks virtual address unreservation and kernel error rollbacks --

commit 9eb2222524999c1d26f887b5ad559236c846afe7
Author: Andrea Righi <[email protected]>

bpf: Add an arena flag to retain allocated pages

This commit adds BPF_F_ARENA_NO_FREE to retain arena pages until the map is destroyed. It modifies BPF page-free operations to be no-ops in both sleepable and non-sleepable programs when the flag is set.

Quoted 4 lines
kernel/bpf/arena.c +0 -0
index 0ff707707da04..343e84139ec29 100644

[ ... ]

kernel/bpf/arena.c __bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt · line 1140

Quoted 8 lines
kernel/bpf/arena.c +2 -1
@@ -1134,7 +1135,8 @@ __bpf_kfunc void bpf_arena_free_pages(void *p__map, void *ptr__ign, u32 page_cnt
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign ||
+	    (map->map_flags & BPF_F_ARENA_NO_FREE))
 		return;

[Severity: High]
Does unconditionally skipping arena_free_pages() prevent virtual address unreservation?

When bpf_arena_free_pages() returns early, it skips range_tree_set().

This leaves the virtual addresses perpetually locked in the range tree as "allocated", causing subsequent bpf_arena_alloc_pages() calls on that address to fail with -EBUSY.

kernel/bpf/arena.c void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_c · line 1151

Quoted 10 lines
kernel/bpf/arena.c +2 -1
 	arena_free_pages(arena, (long)ptr__ign, page_cnt, true);
 }
@@ -1144,7 +1146,8 @@ void bpf_arena_free_pages_non_sleepable(void *p__map, void *ptr__ign, u32 page_c
 	struct bpf_map *map = p__map;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
 
-	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign)
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || !ptr__ign ||
+	    (map->map_flags & BPF_F_ARENA_NO_FREE))
 		return;

[Severity: High]
Does unconditionally skipping arena_free_pages() here break external kernel callers that rely on this function for error rollbacks?

For example, subsystems like sched_ext use bpf_arena_free_pages_non_sleepable() to roll back partial allocations:

kernel/sched/ext/arena.c:scx_arena_grow() {
    ...
	ret = gen_pool_add(sch->arena_pool, ...);
	if (ret) {
		bpf_arena_free_pages_non_sleepable(sch->arena_map, p, page_cnt);
		return ret;
	}
    ...
}

In this kernel error path, if gen_pool_add() fails, the rollback becomes a silent no-op.

The physical pages and virtual address range remain allocated in the arena, but the pointer is dropped, permanently leaking capacity within the arena.

Quoted 2 lines
kernel/bpf/arena.c +0 -0
 	arena_free_pages(arena, (long)ptr__ign, page_cnt, false);
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

PATCH 2/5 bpf: Expose pinned arenas as sized bpffs files

In reply to Andrea Righi

Allow userspace consumers to access pinned BPF arena memory through ordinary file-backed mappings. This enables shared-memory communication with consumers that require a sized file, such as QEMU guest RAM backends.

Add BPF_F_ARENA_EXPORT to expose pinned arenas as sized bpffs files that userspace can open and mmap. Keep the map FD as the canonical BPF mapping, while exported mappings can use their own virtual addresses and bounded file offsets and lengths. Resolve faults by file offset and zap only the overlap with each exported VMA, allowing consumers to map disjoint slices of one arena.

Require full-capacity canonical mappings for exported arenas so the initialized address range matches the size reported by the pinned file. Keep shorter canonical mappings available for arenas without export. Validate exported mappings against the map capacity and canonical range before resolving faults. Keep the pin's size fixed while allowing permission and ownership updates.

Allow read-only shared mappings through O_RDONLY pin FDs by restoring VM_SHARED when generic mmap retains VM_MAYSHARE, without granting VM_MAYWRITE. Keep private mappings rejected and preserve existing BPF map and LSM checks, allowing pin permissions to distinguish readers from writers.

Require BPF_F_ARENA_NO_FREE with BPF_F_ARENA_EXPORT, so BPF programs cannot free pages while external consumers may still use them. Retention-only arenas keep their ordinary pin behavior. File mappings and map references keep the arena and its pages alive until the last reference is released.

Initialize the pin's capacity before publishing its inode, and support bounded SEEK_SET, SEEK_CUR, and SEEK_END operations. Use the canonical range end to distinguish initialization from a valid mapping at address zero.

---
 include/uapi/linux/bpf.h       |  3 ++
 kernel/bpf/arena.c             | 68 +++++++++++++++++++-----
 kernel/bpf/inode.c             | 95 +++++++++++++++++++++++++++++++---
 tools/include/uapi/linux/bpf.h |  3 ++
 4 files changed, 150 insertions(+), 19 deletions(-)
include/uapi/linux/bpf.h +3 -0
index 75101d602bd32..40a5489ac9fea 100644
@@ -1503,6 +1503,9 @@ enum {
 
 	/* Keep arena pages allocated until the map is destroyed. */
 	BPF_F_ARENA_NO_FREE	= (1U << 20),
+
+	/* Expose a pinned arena as a sized file; requires BPF_F_ARENA_NO_FREE. */
+	BPF_F_ARENA_EXPORT	= (1U << 21),
 };
 
 /* Flags for BPF_PROG_QUERY. */
kernel/bpf/arena.c +55 -13

133 lines not on this page · raw message

kernel/bpf/inode.c +89 -6

146 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +3 -0
index 75101d602bd32..40a5489ac9fea 100644
@@ -1503,6 +1503,9 @@ enum {
 
 	/* Keep arena pages allocated until the map is destroyed. */
 	BPF_F_ARENA_NO_FREE	= (1U << 20),
+
+	/* Expose a pinned arena as a sized file; requires BPF_F_ARENA_NO_FREE. */
+	BPF_F_ARENA_EXPORT	= (1U << 21),
 };
 
 /* Flags for BPF_PROG_QUERY. */
Signature · 1 line
2.56.0
#4 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] bpf: unreachable overlap calculation logic for exported arenas in zap_pages() --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] bpf: unreachable overlap calculation logic for exported arenas in zap_pages() --

commit 0faffaea3054cd9867f7c5fb15288b7583784671
Author: Andrea Righi <[email protected]>

bpf: Expose pinned arenas as sized bpffs files

This commit introduces the BPF_F_ARENA_EXPORT flag to expose pinned BPF arenas as sized files in bpffs, allowing userspace programs to mmap them. Map fault handling and page zapping are updated to support disjoint slice mappings using file offsets. It also mandates that exported arenas use the BPF_F_ARENA_NO_FREE flag.

Quoted 3 lines
kernel/bpf/arena.c +0 -0

[ ... ]

Quoted 4 lines
kernel/bpf/arena.c +0 -0
@@ -881,8 +917,14 @@ static void zap_pages(struct bpf_arena *arena, long uaddr, long page_cnt)
 		 */
 		vma = find_vma(mm, vm_start);
 		if (vma && vma->vm_start == vm_start &&

kernel/bpf/arena.c static void zap_pages(struct bpf_arena *arena, long uaddr, long page_cnt) · line 927

Quoted 10 lines
kernel/bpf/arena.c +8 -2
-		    vma->vm_file && vma->vm_file->private_data == &arena->map)
-			zap_vma_range(vma, uaddr, size);
+		    vma->vm_file && vma->vm_file->private_data == &arena->map) {
+			vma_start = (u64)vma->vm_pgoff << PAGE_SHIFT;
+			vma_end = vma_start + vma->vm_end - vma->vm_start;
+			if (start < vma_end && end > vma_start)
+				zap_vma_range(vma, vma->vm_start +
+					      (max(start, vma_start) - vma_start),
+					      min(end, vma_end) - max(start, vma_start));
+		}

[Severity: Low]
Is this overlap calculation in zap_pages() reachable for exported arenas?

The offset calculation here uses vma->vm_pgoff to compute overlaps for partial page zaps. This specific offset math is only strictly necessary for exported mappings, because canonical arena VMAs always have a zero page offset.

However, zap_pages() is exclusively called during page freeing operations such as bpf_arena_free_pages() or via arena_free_worker().

Because arena_map_alloc() requires exported arenas to have the BPF_F_ARENA_NO_FREE flag set, and bpf_arena_free_pages() explicitly returns early when this flag is present, BPF programs can never free pages in exported arenas.

Could this make the newly added partial zap logic using vma->vm_pgoff completely dead code?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw
In reply to Andrea Righi
TL;DR: Nits only in bpf_mkmap() in kernel/bpf/inode.c: the nested ternary could be an early return, continuation indentation is off, and the arena-export predicate duplicated in bpf_map_setattr()…
Quoted 1 line

bpf: Expose pinned arenas as sized bpffs files

TL;DR: Nits only in bpf_mkmap() in kernel/bpf/inode.c: the nested ternary could be an early return, continuation indentation is off, and the arena-export predicate duplicated in bpf_map_setattr() could be shared.

Quoted 4 lines
kernel/bpf/inode.c +0 -0
index 7837968c0842..d3dc6f70e32a 100644

[ ... ]

Quoted 12 lines
kernel/bpf/inode.c +2 -1
@@ -390,16 +469,20 @@ static int bpf_mkobj_ops(struct dentry *dentry, umode_t mode, void *raw,
 static int bpf_mkprog(struct dentry *dentry, umode_t mode, void *arg)
 {
 	return bpf_mkobj_ops(dentry, mode, arg, &bpf_prog_iops,
-			     &bpffs_obj_fops);
+			     &bpffs_obj_fops, 0);
 }
 
 static int bpf_mkmap(struct dentry *dentry, umode_t mode, void *arg)
 {
 	struct bpf_map *map = arg;
+	bool shared_arena = map->map_type == BPF_MAP_TYPE_ARENA &&

kernel/bpf/inode.c static int bpf_mkobj_ops(struct dentry *dentry, umode_t mode, void *raw, · line 486

Quoted 10 lines
kernel/bpf/inode.c +5 -2
+			    (map->map_flags & BPF_F_ARENA_EXPORT);
 
 	return bpf_mkobj_ops(dentry, mode, arg, &bpf_map_iops,
-			     bpf_map_support_seq_show(map) ?
-			     &bpffs_map_fops : &bpffs_obj_fops);
+			    shared_arena ? &bpffs_arena_fops :
+			    bpf_map_support_seq_show(map) ?
+			    &bpffs_map_fops : &bpffs_obj_fops,
+			    shared_arena ? (loff_t)map->max_entries * PAGE_SIZE : 0);
 }

This isn't a bug, but would it read more simply to handle the exported arena case with an early return in bpf_mkmap(), rather than testing shared_arena twice inside a nested ternary in the argument list?

The continuation lines here are also indented one column short of the open parenthesis, unlike the matching call in bpf_mklink().

The same 'map->map_type == BPF_MAP_TYPE_ARENA && (map->map_flags & BPF_F_ARENA_EXPORT)' predicate is repeated in bpf_map_setattr(). Could it be shared through a small helper, or could bpf_map_setattr() check inode->i_fop == &bpffs_arena_fops instead?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38093513563

#6 lore raw

PATCH 3/5 docs/bpf: Document pinned arena file mappings

In reply to Andrea Righi

Describe how consumers can access pinned BPF arenas as sized bpffs files and establish the canonical BPF address range before mapping them. Explain slice bounds, access permissions, fixed file size, and mapping lifetime so consumers can use the shared-memory interface correctly.

Document fixed-size seeking and the inherited arena VMA restrictions:
no fork inheritance, splitting, expansion, or relocation.

---
 Documentation/bpf/map_arena.rst | 131 ++++++++++++++++++++++++++++++++
 1 file changed, 131 insertions(+)
 create mode 100644 Documentation/bpf/map_arena.rst
Documentation/bpf/map_arena.rst new file +131 -0
new file mode 100644
index 0000000000000..1b9eadd5fb3a1
@@ -0,0 +1,131 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+==================
+BPF_MAP_TYPE_ARENA
+==================
+
+A BPF arena provides shared memory that BPF programs and userspace can
+access directly. Create the map with zero key and value sizes and
+``BPF_F_MMAPABLE``. ``max_entries`` specifies its capacity in pages, up to
+4 GiB. The canonical BPF address range must not cross a 4 GiB boundary.
+
+Pinned arena files
+==================
+
+An arena created with ``BPF_F_ARENA_EXPORT`` can be pinned in bpffs and
+opened as a sized file. Its size is ``max_entries * PAGE_SIZE``. Export
+requires ``BPF_F_ARENA_NO_FREE`` so BPF programs cannot release backing
+pages while external consumers may still use them. Creating an arena with
+``BPF_F_ARENA_EXPORT`` alone fails with ``EINVAL``. Arenas without the
+export flag retain their existing bpffs behavior and cannot be opened
+this way, including arenas created with ``BPF_F_ARENA_NO_FREE`` alone.
+
+Establish the canonical BPF address range before mapping the pinned file:
+
+* Set ``map_extra`` to a nonzero, page-aligned address at map creation.
+  The canonical range then covers the full map capacity, even if the map
+  FD has not been mmaped.
+* Alternatively, leave ``map_extra`` zero and mmap the map FD first. This
+  mapping establishes the canonical address and must cover the full map
+  capacity. Shorter canonical mappings fail with ``EINVAL``.
+
+A canonical map FD mapping may start at address zero if the system's
+low-address mapping policy permits it.
+
+Mapping the pinned file before either step fails with ``EINVAL``. Exported
+mappings do not establish or change the canonical BPF address range.
+
+Once the canonical range is established, it covers the entire capacity
+reported by ``fstat()``. Consumers can map the full file or smaller slices.
+Establishing the full canonical mapping reserves virtual address space;
+it does not populate every arena page. Arenas without
+``BPF_F_ARENA_EXPORT`` can still establish shorter canonical mappings.
+
+Open the pin with ``O_RDONLY`` for read-only access or ``O_RDWR`` for
+writable access, and mmap it with ``MAP_SHARED``. An exported
+mapping may use a different virtual address from the canonical mapping.
+Its offset must be page-aligned, and its offset and length must fit within
+both the map capacity and the established canonical range. Consumers can
+map disjoint slices of one arena. Oversized or out-of-range mappings fail
+with ``EINVAL``. ``MAP_PRIVATE`` mappings are unsupported.
+
+Exported mappings share the bytes of the arena without relocating pointers
+stored in those bytes. Arena pointers produced by a BPF program use that
+arena's canonical address representation, based on ``user_vm_start``.
+A consumer mapping a slice at another address must translate such pointers
+before dereferencing them. A guest BPF arena has its own canonical range;
+sharing the backing pages does not make host arena pointers valid in the
+guest arena, or vice versa. Communication structures can use offsets
+relative to the shared slice, with each side validating the offsets and
+translating them to addresses in its own mapping or arena.
+
+A pin opened with ``O_RDONLY`` supports read-only shared mappings, which
+observe updates made by BPF programs and other writable mappings. Requests
+for ``PROT_WRITE`` through that FD fail with ``EACCES``. A mapping made
+without ``PROT_WRITE`` cannot subsequently acquire write permission with
+``mprotect()``, even when the pin was opened with ``O_RDWR``.
+
+Pin permissions can grant readers access independently of writers. For
+example, mode ``0640`` permits the owner to open the pin for writable
+access and the group to open it for read-only access, subject to the
+applicable LSM checks. Changing permissions does not revoke access through
+existing FDs or mappings.
+
+The file has a fixed size. ``truncate()``, ``ftruncate()``, and ``O_TRUNC``
+cannot change it. A truncate request that leaves the size unchanged is
+allowed. Permission and ownership changes remain available.
+Consumers can discover the capacity with ``fstat()`` or
+``lseek(fd, 0, SEEK_END)``. Seeking supports ``SEEK_SET``, ``SEEK_CUR``, and
+``SEEK_END`` within the file bounds; it does not enable ``read()`` or
+``write()`` access to the memory.
+
+``fsync()``, ``fdatasync()``, and ``msync(..., MS_SYNC)`` succeed without
+performing writeback: arena memory is volatile and has no persistent
+backing. These calls do not synchronize access between consumers or BPF
+programs; shared-memory protocols still need appropriate memory ordering.
+
+Exported mappings inherit the arena's mapping lifecycle restrictions.
+They are not inherited across ``fork()``, and ``MADV_DOFORK`` cannot enable
+inheritance. Consumers must create their own mappings in a child process.
+Mappings cannot be split, expanded, or relocated with ``mremap()``.
+Partial unmapping and protection changes that require splitting a mapping
+fail with ``EINVAL``. Unmapping an entire mapping remains supported, as do
+protection changes that do not require a split and satisfy the access
+restrictions above. Consumers needing independently managed regions
+should create separate slice mappings.
+
+Access control and lifetime
+===========================
+
+Opening the pin uses ordinary filesystem permissions and file LSM checks,
+as well as ``security_bpf_map()`` for the requested access mode. Mapping it
+also passes the ordinary mmap LSM checks. Writable mappings remain subject
+to the BPF map's frozen state and program-read-only restrictions.
+
+Consumers use ``open()`` and ``mmap()`` rather than ``bpf(BPF_OBJ_GET)``.
+Consequently, seccomp rules and LSM policies specific to the ``bpf()``
+syscall do not govern this file interface. Grant access through the pin's
+permissions and the applicable file, mmap, and BPF map LSM policies.
+A process prohibited from calling ``bpf()`` can still access the arena
+through this interface if those permissions and checks allow it.
+``unprivileged_bpf_disabled`` restricts unprivileged map and program
+creation; it does not generally prohibit access to existing maps.
+
+``BPF_F_ARENA_NO_FREE`` makes ``bpf_arena_free_pages()`` a no-op in both
+sleepable and non-sleepable BPF programs. The free operation returns no
+indication that pages were retained. Programs using this flag must not
+rely on freeing pages to restore arena allocation capacity. Arena pages
+are retained until the map is destroyed. Open map and pin FDs, mappings,
+and other map references retain the arena, including after the pin is
+unlinked. Removing the pin therefore does not revoke existing mappings or
+free their pages.
+
+The retention flag does not itself expose the pin as a sized file.
+``BPF_F_ARENA_EXPORT`` opts into that interface and must be combined with
+``BPF_F_ARENA_NO_FREE``. Retention-only arenas can still share memory
+through ordinary map FD mappings and manage reusable objects within
+their retained pages.
+
+For example, a consumer such as QEMU can use an exported slice as a shared
+file-backed guest RAM backend. The file interface provides shared memory;
+consumers must supply their own allocation and communication protocol.
Signature · 1 line
2.56.0
#7 lore raw

Re: [PATCH 3/5] docs/bpf: Document pinned arena file mappings

TL;DR: - Doc says a canonical arena mapping may start at address zero, but the verifier may then reject programs using it - Doc claim about shorter mappings of non-exported arenas looks wrong when…
Quoted 1 line

docs/bpf: Document pinned arena file mappings

TL;DR:
- Doc says a canonical arena mapping may start at address zero, but the verifier may then reject programs using it
- Doc claim about shorter mappings of non-exported arenas looks wrong when map_extra is nonzero
- Doc mentions an unreachable read-only restriction

Quoted 6 lines
Documentation/bpf/map_arena.rst new file +0 -0
new file mode 100644
index 000000000000..1b9eadd5fb3a
@@ -0,0 +1,131 @@

[ ... ]

Documentation/bpf/map_arena.rst line 33

Quoted 6 lines
Documentation/bpf/map_arena.rst new file +6 -0
+* Alternatively, leave ``map_extra`` zero and mmap the map FD first. This
+  mapping establishes the canonical address and must cover the full map
+  capacity. Shorter canonical mappings fail with ``EINVAL``.
+
+A canonical map FD mapping may start at address zero if the system's
+low-address mapping policy permits it.

Can a BPF program use an arena in that state? The BPF_MAP_TYPE_ARENA handling in kernel/bpf/verifier.c still treats user_vm_start == 0 as unset:

    if (!bpf_arena_get_user_vm_start(env->prog->aux->arena)) {
        verbose(env, "arena's user address must be set via map_extra or mmap()\n");
        return -EINVAL;
    }

Consider an arena created with BPF_F_MMAPABLE, BPF_F_ARENA_NO_FREE and BPF_F_ARENA_EXPORT, with map_extra == 0, followed by:

    mmap(0, capacity, ..., MAP_SHARED | MAP_FIXED_NOREPLACE, map_fd, 0)

With mmap_min_addr == 0 this succeeds, since arena_get_unmapped_area() and arena_map_mmap() in kernel/bpf/arena.c now test user_vm_end for an established range. That leaves user_vm_start = 0 and user_vm_end = capacity, and loading any BPF program that references the arena then fails with -EINVAL.

Should the verifier check use the same "established" test, or should this paragraph say that such an arena cannot be used by BPF programs?

[ ... ]

Documentation/bpf/map_arena.rst line 42

Quoted 5 lines
Documentation/bpf/map_arena.rst new file +5 -0
+Once the canonical range is established, it covers the entire capacity
+reported by ``fstat()``. Consumers can map the full file or smaller slices.
+Establishing the full canonical mapping reserves virtual address space;
+it does not populate every arena page. Arenas without
+``BPF_F_ARENA_EXPORT`` can still establish shorter canonical mappings.

Is that last sentence true for every arena without BPF_F_ARENA_EXPORT?

For a non-exported arena created with a nonzero map_extra, arena_map_alloc() sets user_vm_end = map_extra + max_entries * PAGE_SIZE at creation, so the canonical range is already the full capacity.

arena_get_unmapped_area() then takes the "if (arena->user_vm_end)" branch and rejects any len != user_vm_end - user_vm_start with -EINVAL (-E2BIG if larger), and arena_map_mmap() returns -EBUSY when user_vm_end != vma->vm_end. A shorter map FD mapping of such an arena fails, which also conflicts with the earlier bullet saying a nonzero map_extra makes the range cover the full map capacity.

Should the sentence be qualified with "when map_extra is zero"?

[ ... ]

Documentation/bpf/map_arena.rst line 103

Quoted 4 lines
Documentation/bpf/map_arena.rst new file +4 -0
+Opening the pin uses ordinary filesystem permissions and file LSM checks,
+as well as ``security_bpf_map()`` for the requested access mode. Mapping it
+also passes the ordinary mmap LSM checks. Writable mappings remain subject
+to the BPF map's frozen state and program-read-only restrictions.

Can the "program-read-only restrictions" apply to an arena?

bpf_map_mmap() does reject writable mappings of maps with BPF_F_RDONLY_PROG (-EACCES), but arena_map_alloc() only accepts BPF_F_SEGV_ON_FAULT, BPF_F_MMAPABLE, BPF_F_NO_USER_CONV, BPF_F_ARENA_NO_FREE and BPF_F_ARENA_EXPORT. Creating an arena with BPF_F_RDONLY_PROG fails with -EINVAL, and nothing sets the flag afterwards.

Only the frozen state (BPF_MAP_FREEZE, -EPERM on a writable mapping) is a live restriction for exported arena mappings. Should the phrase be dropped, or reworded to say that only the frozen state applies?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38093513563

#8 lore raw

PATCH 4/5 selftests/bpf: Test pinned arena file mappings

In reply to Andrea Righi

Add native tests for pinned arena files, covering canonical range setup via map FD mmap and map_extra, exported slice bounds, full-capacity canonical mappings, and mapping lifetime after unpinning. Check that oversized mapping requests fail cleanly and that size changes are rejected without zapping mappings, while permission updates and BPF_OBJ_GET still work.

Reject canonical mapping lengths that differ from the exported arena's capacity and map the entire size reported by fstat(). Preserve short canonical mappings for ordinary and retention-only arenas. Drop other mappings, the pin, and open FDs before checking that the exported slice retains the arena. Change the pin's mode from its creation permissions to verify that metadata updates take effect.

Test O_RDONLY pin mappings with MAP_SHARED and MAP_SHARED_VALIDATE. Verify that readers observe writer updates, retain access after closing the FD, and cannot request writable or private mappings or upgrade a mapping with mprotect().

Check that ordinary and retention-only arenas keep their existing pin behavior, while BPF_F_ARENA_EXPORT requires BPF_F_ARENA_NO_FREE.

Check fixed-size seeking, canonical setup at address zero when security policy permits it, and the exported mapping lifecycle restrictions. Verify that partial unmap and protection changes, MADV_DOFORK, and relocation fail, while fork leaves the parent's mapping intact without inheriting it in the child.

Assisted-by LLM
---
 .../selftests/bpf/prog_tests/arena_pinned.c   | 442 ++++++++++++++++++
 1 file changed, 442 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_pinned.c
tools/testing/selftests/bpf/prog_tests/arena_pinned.c new file +442 -0
new file mode 100644
index 0000000000000..8b7ca90211296
@@ -0,0 +1,442 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES
+ */
+#include <test_progs.h>
+#include <fcntl.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+
+#define ARENA_PAGES 4
+
+struct arena_file {
+	int map_fd;
+	int file_fd;
+	char pin[128];
+	void *canonical;
+	size_t len;
+	bool pinned;
+};
+
+static void cleanup(struct arena_file *file)
+{
+	if (file->canonical != MAP_FAILED)
+		munmap(file->canonical, file->len);
+	if (file->file_fd >= 0)
+		close(file->file_fd);
+	if (file->pinned)
+		unlink(file->pin);
+	if (file->map_fd >= 0)
+		close(file->map_fd);
+}
+
+static void reject_mapping(int fd, size_t len, int flags, off_t offset,
+			   const char *name)
+{
+	void *addr;
+	int err;
+
+	addr = mmap(NULL, len, PROT_READ | PROT_WRITE, flags, fd, offset);
+	err = errno;
+	if (ASSERT_EQ(addr, MAP_FAILED, name))
+		ASSERT_EQ(err, EINVAL, "mmap_errno");
+	else
+		munmap(addr, len);
+}
+
+static bool setup(struct arena_file *file, __u64 map_extra, bool zero)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts,
+		.map_flags = BPF_F_MMAPABLE | BPF_F_ARENA_NO_FREE |
+			     BPF_F_ARENA_EXPORT,
+		.map_extra = map_extra);
+
+	memset(file, 0, sizeof(*file));
+	file->map_fd = -1;
+	file->file_fd = -1;
+	file->canonical = MAP_FAILED;
+	file->len = ARENA_PAGES * getpagesize();
+	snprintf(file->pin, sizeof(file->pin), "/sys/fs/bpf/arena_pinned_%d", getpid());
+	file->map_fd = bpf_map_create(BPF_MAP_TYPE_ARENA, "arena_pinned",
+				      0, 0, ARENA_PAGES, &opts);
+	if (file->map_fd < 0 && errno == EOPNOTSUPP) {
+		test__skip();
+		return false;
+	}
+	if (!ASSERT_GE(file->map_fd, 0, "map_create"))
+		return false;
+	if (!ASSERT_OK(bpf_obj_pin(file->map_fd, file->pin), "obj_pin"))
+		return false;
+	file->pinned = true;
+	file->file_fd = open(file->pin, O_RDWR);
+	if (!ASSERT_GE(file->file_fd, 0, "open_pin"))
+		return false;
+	reject_mapping(file->map_fd, getpagesize(), MAP_SHARED, 0, "short_canonical");
+	reject_mapping(file->map_fd, file->len + getpagesize(), MAP_SHARED,
+		       0, "large_canonical");
+	if (map_extra)
+		return true;
+
+	reject_mapping(file->file_fd, getpagesize(), MAP_SHARED, 0, "uninitialized");
+	file->canonical = mmap(NULL, file->len, PROT_READ | PROT_WRITE,
+			       MAP_SHARED | (zero ? MAP_FIXED_NOREPLACE : 0),
+			       file->map_fd, 0);
+	if (zero && file->canonical == MAP_FAILED &&
+	    (errno == EPERM || errno == EACCES)) {
+		test__skip();
+		return false;
+	}
+	return ASSERT_NEQ(file->canonical, MAP_FAILED, "canonical_mmap");
+}
+
+static void test_mappings(__u64 map_extra)
+{
+	struct arena_file file;
+	size_t ps = getpagesize();
+	struct stat st;
+	char *alias = MAP_FAILED;
+	char *full = MAP_FAILED;
+	int expected = 91;
+
+	if (!setup(&file, map_extra, false))
+		goto out;
+	if (!ASSERT_OK(fstat(file.file_fd, &st), "stat_pin"))
+		goto out;
+	ASSERT_EQ(st.st_size, ARENA_PAGES * ps, "pin_size");
+	reject_mapping(file.file_fd, 8ULL << 30, MAP_SHARED, 0, "oversized");
+	reject_mapping(file.file_fd, ps, MAP_SHARED, ARENA_PAGES * ps, "past_capacity");
+	reject_mapping(file.file_fd, ps * 2, MAP_SHARED,
+		       (ARENA_PAGES - 1) * ps, "past_capacity_end");
+	reject_mapping(file.file_fd, ps, MAP_PRIVATE, 0, "private_mapping");
+	full = mmap(NULL, st.st_size, PROT_READ | PROT_WRITE, MAP_SHARED,
+		    file.file_fd, 0);
+	if (!ASSERT_NEQ(full, MAP_FAILED, "full_file_mmap"))
+		goto out;
+	full[0] = 17;
+	full[file.len - 1] = 62;
+
+	alias = mmap(NULL, ps, PROT_READ | PROT_WRITE, MAP_SHARED,
+		     file.file_fd, (ARENA_PAGES - 1) * ps);
+	if (!ASSERT_NEQ(alias, MAP_FAILED, "last_page_slice"))
+		goto out;
+	alias[0] = 91;
+	ASSERT_OK(msync(alias, ps, MS_SYNC), "msync_slice");
+	ASSERT_OK(fsync(file.file_fd), "fsync_pin");
+	ASSERT_OK(fdatasync(file.file_fd), "fdatasync_pin");
+	ASSERT_EQ(alias[ps - 1], 62, "full_file_last_page");
+	if (file.canonical != MAP_FAILED) {
+		char *canonical = file.canonical;
+
+		ASSERT_EQ(canonical[0], 17, "full_file_first_page");
+		ASSERT_EQ(canonical[(ARENA_PAGES - 1) * ps], 91, "alias_write");
+		canonical[(ARENA_PAGES - 1) * ps] = 42;
+		ASSERT_EQ(alias[0], 42, "canonical_write");
+		expected = 42;
+		if (!ASSERT_OK(munmap(file.canonical, file.len), "unmap_canonical"))
+			goto out;
+		file.canonical = MAP_FAILED;
+	}
+	if (!ASSERT_OK(munmap(full, file.len), "unmap_full_file"))
+		goto out;
+	full = MAP_FAILED;
+	if (!ASSERT_OK(unlink(file.pin), "unlink_pin"))
+		goto out;
+	file.pinned = false;
+	close(file.file_fd);
+	file.file_fd = -1;
+	close(file.map_fd);
+	file.map_fd = -1;
+	ASSERT_EQ(alias[0], expected, "unpin_lifetime");
+	alias[0] = 73;
+out:
+	if (full != MAP_FAILED)
+		munmap(full, file.len);
+	if (alias != MAP_FAILED)
+		munmap(alias, ps);
+	cleanup(&file);
+}
+
+static void test_unexported_pin(__u32 flags)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts,
+		.map_flags = BPF_F_MMAPABLE | flags);
+	char pin[128];
+	struct stat st;
+	char *canonical = MAP_FAILED;
+	size_t ps = getpagesize();
+	int map_fd, fd, err;
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_ARENA, "arena_unexported",
+			       0, 0, ARENA_PAGES, &opts);
+	if (map_fd < 0 && errno == EOPNOTSUPP) {
+		test__skip();
+		return;
+	}
+	if (!ASSERT_GE(map_fd, 0, "unexported_create"))
+		return;
+	canonical = mmap(NULL, ps, PROT_READ | PROT_WRITE, MAP_SHARED, map_fd, 0);
+	if (!ASSERT_NEQ(canonical, MAP_FAILED, "unexported_short_mmap"))
+		goto out;
+	canonical[0] = 42;
+	ASSERT_EQ(canonical[0], 42, "unexported_short_access");
+	snprintf(pin, sizeof(pin), "/sys/fs/bpf/arena_unexported_%d", getpid());
+	if (!ASSERT_OK(bpf_obj_pin(map_fd, pin), "unexported_pin"))
+		goto out;
+	if (ASSERT_OK(stat(pin, &st), "unexported_stat"))
+		ASSERT_EQ(st.st_size, 0, "unexported_size");
+	fd = open(pin, O_RDONLY);
+	err = errno;
+	if (ASSERT_EQ(fd, -1, "unexported_open"))
+		ASSERT_EQ(err, EIO, "unexported_open_errno");
+	else
+		close(fd);
+	fd = bpf_obj_get(pin);
+	if (ASSERT_GE(fd, 0, "unexported_obj_get"))
+		close(fd);
+	ASSERT_OK(unlink(pin), "unexported_unlink");
+out:
+	if (canonical != MAP_FAILED)
+		munmap(canonical, ps);
+	close(map_fd);
+}
+
+static void test_export_requires_no_free(void)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts,
+		.map_flags = BPF_F_MMAPABLE | BPF_F_ARENA_EXPORT);
+	int fd;
+
+	fd = bpf_map_create(BPF_MAP_TYPE_ARENA, "arena_export",
+			    0, 0, ARENA_PAGES, &opts);
+	if (fd == -EOPNOTSUPP) {
+		test__skip();
+		return;
+	}
+	ASSERT_EQ(fd, -EINVAL, "export_requires_no_free");
+	if (fd >= 0)
+		close(fd);
+}
+
+static void test_read_only(int flags)
+{
+	struct arena_file file;
+	size_t ps = getpagesize();
+	char *alias = MAP_FAILED;
+	void *addr;
+	int fd = -1, err, ret;
+
+	if (!setup(&file, 0, false))
+		goto out;
+	if (!ASSERT_OK(fchmod(file.file_fd, 0400), "chmod_read_only"))
+		goto out;
+	fd = open(file.pin, O_RDONLY);
+	if (!ASSERT_GE(fd, 0, "open_read_only"))
+		goto out;
+	alias = mmap(NULL, ps, PROT_READ, flags, fd, ps);
+	if (!ASSERT_NEQ(alias, MAP_FAILED, "read_only_mmap"))
+		goto out;
+	ASSERT_EQ(alias[0], 0, "read_only_fault");
+	((char *)file.canonical)[ps] = 42;
+	ASSERT_EQ(alias[0], 42, "read_only_shared_update");
+
+	addr = mmap(NULL, ps, PROT_READ | PROT_WRITE, flags, fd, ps);
+	err = errno;
+	if (ASSERT_EQ(addr, MAP_FAILED, "read_only_writable_mmap"))
+		ASSERT_EQ(err, EACCES, "read_only_writable_errno");
+	else
+		munmap(addr, ps);
+	ret = mprotect(alias, ps, PROT_READ | PROT_WRITE);
+	err = errno;
+	ASSERT_EQ(ret, -1, "read_only_mprotect");
+	ASSERT_EQ(err, EACCES, "read_only_mprotect_errno");
+	addr = mmap(NULL, ps, PROT_READ, MAP_PRIVATE, fd, ps);
+	err = errno;
+	if (ASSERT_EQ(addr, MAP_FAILED, "read_only_private_mmap"))
+		ASSERT_EQ(err, EINVAL, "read_only_private_errno");
+	else
+		munmap(addr, ps);
+
+	close(fd);
+	fd = -1;
+	((char *)file.canonical)[ps] = 73;
+	ASSERT_EQ(alias[0], 73, "read_only_after_close");
+out:
+	if (alias != MAP_FAILED)
+		munmap(alias, ps);
+	if (fd >= 0)
+		close(fd);
+	cleanup(&file);
+}
+
+static void test_fixed_size(void)
+{
+	struct arena_file file;
+	size_t ps = getpagesize();
+	struct stat st;
+	unsigned char resident;
+	char *alias = MAP_FAILED;
+	int err, fd, ret;
+
+	if (!setup(&file, 0, false))
+		goto out;
+	alias = mmap(NULL, ps, PROT_READ | PROT_WRITE, MAP_SHARED, file.file_fd, 0);
+	if (!ASSERT_NEQ(alias, MAP_FAILED, "alias_mmap"))
+		goto out;
+	alias[0] = 42;
+	ret = ftruncate(file.file_fd, 0);
+	err = errno;
+	ASSERT_EQ(ret, -1, "ftruncate_shrink");
+	ASSERT_EQ(err, EINVAL, "ftruncate_shrink_errno");
+	ret = ftruncate(file.file_fd, file.len + ps);
+	err = errno;
+	ASSERT_EQ(ret, -1, "ftruncate_grow");
+	ASSERT_EQ(err, EINVAL, "ftruncate_grow_errno");
+	ret = truncate(file.pin, 0);
+	err = errno;
+	ASSERT_EQ(ret, -1, "truncate_pin");
+	ASSERT_EQ(err, EINVAL, "truncate_pin_errno");
+	fd = open(file.pin, O_RDWR | O_TRUNC);
+	err = errno;
+	if (ASSERT_EQ(fd, -1, "open_trunc"))
+		ASSERT_EQ(err, EINVAL, "open_trunc_errno");
+	else
+		close(fd);
+	ASSERT_OK(ftruncate(file.file_fd, file.len), "ftruncate_same_size");
+	if (!ASSERT_OK(mincore(alias, ps, &resident), "mincore"))
+		goto out;
+	ASSERT_EQ(resident & 1, 1, "mapping_not_zapped");
+	ASSERT_EQ(alias[0], 42, "mapping_value");
+	ASSERT_OK(fchmod(file.file_fd, 0640), "chmod_pin");
+	if (ASSERT_OK(fstat(file.file_fd, &st), "stat_pin")) {
+		ASSERT_EQ(st.st_size, file.len, "fixed_size");
+		ASSERT_EQ(st.st_mode & 0777, 0640, "pin_mode");
+	}
+	ASSERT_EQ(lseek(file.file_fd, 0, SEEK_END), file.len, "seek_capacity");
+	ASSERT_EQ(lseek(file.file_fd, -(off_t)ps, SEEK_CUR), file.len - ps, "seek_back");
+	ASSERT_EQ(lseek(file.file_fd, ps, SEEK_SET), ps, "seek_offset");
+	errno = 0;
+	ASSERT_EQ(lseek(file.file_fd, file.len + 1, SEEK_SET), -1, "seek_past_end");
+	ASSERT_EQ(errno, EINVAL, "seek_past_end_errno");
+	errno = 0;
+	ASSERT_EQ(lseek(file.file_fd, -1, SEEK_SET), -1, "seek_before_start");
+	ASSERT_EQ(errno, EINVAL, "seek_before_start_errno");
+	fd = bpf_obj_get(file.pin);
+	if (ASSERT_GE(fd, 0, "obj_get_after_chmod"))
+		close(fd);
+out:
+	if (alias != MAP_FAILED)
+		munmap(alias, ps);
+	cleanup(&file);
+}
+
+static void test_zero_canonical(void)
+{
+	struct arena_file file;
+	size_t ps = getpagesize();
+	char *alias = MAP_FAILED;
+	void *addr;
+
+	if (!setup(&file, 0, true))
+		goto out;
+	if (!ASSERT_EQ((unsigned long)file.canonical, 0, "zero_canonical"))
+		goto out;
+	alias = mmap(NULL, ps, PROT_READ | PROT_WRITE, MAP_SHARED, file.file_fd, ps);
+	if (!ASSERT_NEQ(alias, MAP_FAILED, "zero_canonical_export"))
+		goto out;
+	alias[0] = 42;
+	ASSERT_EQ(alias[0], 42, "zero_canonical_fault");
+	addr = mmap((void *)ps, file.len, PROT_READ | PROT_WRITE,
+		    MAP_SHARED | MAP_FIXED_NOREPLACE, file.map_fd, 0);
+	if (ASSERT_EQ(addr, MAP_FAILED, "canonical_cannot_move"))
+		ASSERT_EQ(errno, EINVAL, "canonical_cannot_move_errno");
+	else
+		munmap(addr, file.len);
+out:
+	if (alias != MAP_FAILED)
+		munmap(alias, ps);
+	cleanup(&file);
+}
+
+static void test_lifecycle(void)
+{
+	struct arena_file file;
+	size_t ps = getpagesize();
+	char *alias = MAP_FAILED;
+	void *target = MAP_FAILED, *addr;
+	pid_t child;
+	int status, ret, err;
+
+	if (!setup(&file, 0, false))
+		goto out;
+	alias = mmap(NULL, ps * 2, PROT_READ | PROT_WRITE, MAP_SHARED,
+		     file.file_fd, 0);
+	if (!ASSERT_NEQ(alias, MAP_FAILED, "lifecycle_mmap"))
+		goto out;
+	alias[0] = 42;
+	ret = munmap(alias, ps);
+	err = errno;
+	ASSERT_EQ(ret, -1, "partial_unmap");
+	ASSERT_EQ(err, EINVAL, "partial_unmap_errno");
+	ret = mprotect(alias, ps, PROT_READ);
+	err = errno;
+	ASSERT_EQ(ret, -1, "partial_mprotect");
+	ASSERT_EQ(err, EINVAL, "partial_mprotect_errno");
+	ret = madvise(alias, ps * 2, MADV_DOFORK);
+	err = errno;
+	ASSERT_EQ(ret, -1, "enable_inheritance");
+	ASSERT_EQ(err, EINVAL, "enable_inheritance_errno");
+	target = mmap(NULL, ps * 2, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+	if (!ASSERT_NEQ(target, MAP_FAILED, "reserve_remap_target"))
+		goto out;
+	addr = mremap(alias, ps * 2, ps * 2, MREMAP_MAYMOVE | MREMAP_FIXED, target);
+	err = errno;
+	if (!ASSERT_EQ(addr, MAP_FAILED, "relocate_mapping")) {
+		alias = addr;
+		target = MAP_FAILED;
+	}
+	ASSERT_EQ(err, EINVAL, "relocate_mapping_errno");
+	child = fork();
+	if (!ASSERT_GE(child, 0, "fork"))
+		goto out;
+	if (!child) {
+		unsigned char resident[2];
+
+		ret = mincore(alias, ps * 2, resident);
+		_exit(ret == -1 && errno == ENOMEM ? 0 : 1);
+	}
+	if (ASSERT_EQ(waitpid(child, &status, 0), child, "wait_child") &&
+	    ASSERT_TRUE(WIFEXITED(status), "child_exited"))
+		ASSERT_EQ(WEXITSTATUS(status), 0, "mapping_not_inherited");
+	ASSERT_EQ(alias[0], 42, "parent_mapping_retained");
+out:
+	if (target != MAP_FAILED)
+		munmap(target, ps * 2);
+	if (alias != MAP_FAILED)
+		munmap(alias, ps * 2);
+	cleanup(&file);
+}
+
+void serial_test_arena_pinned(void)
+{
+	if (test__start_subtest("ordinary_pin"))
+		test_unexported_pin(0);
+	if (test__start_subtest("retained_pin"))
+		test_unexported_pin(BPF_F_ARENA_NO_FREE);
+	if (test__start_subtest("export_requires_no_free"))
+		test_export_requires_no_free();
+	if (test__start_subtest("mappings"))
+		test_mappings(0);
+	if (test__start_subtest("map_extra"))
+		test_mappings(1ULL << 32);
+	if (test__start_subtest("fixed_size"))
+		test_fixed_size();
+	if (test__start_subtest("read_only"))
+		test_read_only(MAP_SHARED);
+	if (test__start_subtest("read_only_validate"))
+		test_read_only(MAP_SHARED_VALIDATE);
+	if (test__start_subtest("zero_canonical"))
+		test_zero_canonical();
+	if (test__start_subtest("lifecycle"))
+		test_lifecycle();
+}
Signature · 1 line
2.56.0
#9 lore raw

Re: [PATCH 4/5] selftests/bpf: Test pinned arena file mappings

TL;DR: - Commit message second paragraph uses imperatives that read like kernel changes and repeats other paragraphs - Selftest skips print no reason, so address-0 and unsupported-arena skips look…
Quoted 1 line

selftests/bpf: Test pinned arena file mappings

TL;DR:
- Commit message second paragraph uses imperatives that read like kernel changes and repeats other paragraphs
- Selftest skips print no reason, so address-0 and unsupported-arena skips look identical

Quoted 24 lines

selftests/bpf: Test pinned arena file mappings

Add native tests for pinned arena files, covering canonical range setup via map FD mmap and map_extra, exported slice bounds, full-capacity canonical mappings, and mapping lifetime after unpinning. Check that oversized mapping requests fail cleanly and that size changes are rejected without zapping mappings, while permission updates and BPF_OBJ_GET still work.

Reject canonical mapping lengths that differ from the exported arena's capacity and map the entire size reported by fstat(). Preserve short canonical mappings for ordinary and retention-only arenas. Drop other mappings, the pin, and open FDs before checking that the exported slice retains the arena. Change the pin's mode from its creation permissions to verify that metadata updates take effect.

Test O_RDONLY pin mappings with MAP_SHARED and MAP_SHARED_VALIDATE. Verify that readers observe writer updates, retain access after closing the FD, and cannot request writable or private mappings or upgrade a mapping with mprotect().

Check that ordinary and retention-only arenas keep their existing pin behavior, while BPF_F_ARENA_EXPORT requires BPF_F_ARENA_NO_FREE.

Quoted 8 lines

Check fixed-size seeking, canonical setup at address zero when security policy permits it, and the exported mapping lifecycle restrictions. Verify that partial unmap and protection changes, MADV_DOFORK, and relocation fail, while fork leaves the parent's mapping intact without inheriting it in the child.

Assisted-by LLM

This isn't a bug, but the second paragraph of the changelog uses imperatives ("Reject canonical mapping lengths...", "Preserve short canonical mappings...", "Drop other mappings...", "Change the pin's mode...") that read like kernel behaviour changes, even though this patch only adds a selftest.

Could the second paragraph be reworded to say what the tests check, for example "Check that canonical mappings whose length differs from the exported capacity are rejected ...", and merged with the first paragraph? It also repeats items already listed there ("full-capacity canonical mappings", "permission updates") and in a later paragraph ("ordinary and retention-only arenas"). Any version-to-version notes could go below the --- line.

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/arena_pinned.c new file +0 -0
new file mode 100644
index 000000000000..8b7ca9021129
@@ -0,0 +1,442 @@

[ ... ]

Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/arena_pinned.c new file +2 -0
+static bool setup(struct arena_file *file, __u64 map_extra, bool zero)
+{

[ ... ]

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/arena_pinned.c new file +6 -0
+	file->map_fd = bpf_map_create(BPF_MAP_TYPE_ARENA, "arena_pinned",
+				      0, 0, ARENA_PAGES, &opts);
+	if (file->map_fd < 0 && errno == EOPNOTSUPP) {
+		test__skip();
+		return false;
+	}

[ ... ]

tools/testing/selftests/bpf/prog_tests/arena_pinned.c line 91

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/arena_pinned.c new file +10 -0
+	file->canonical = mmap(NULL, file->len, PROT_READ | PROT_WRITE,
+			       MAP_SHARED | (zero ? MAP_FIXED_NOREPLACE : 0),
+			       file->map_fd, 0);
+	if (zero && file->canonical == MAP_FAILED &&
+	    (errno == EPERM || errno == EACCES)) {
+		test__skip();
+		return false;
+	}
+	return ASSERT_NEQ(file->canonical, MAP_FAILED, "canonical_mmap");
+}

A subsystem pattern flags this as potentially concerning: a skip that prints no reason. This isn't a bug, but every skip path in this file is a bare test__skip(), so test_progs only reports SKIP for the subtest.

For the zero_canonical subtest this skip is taken whenever mmap_min_addr or an LSM refuses a mapping at address 0, and the log cannot tell that apart from the arena-unsupported skip taken after bpf_map_create() returns EOPNOTSUPP. The same bare test__skip() appears after the EOPNOTSUPP checks in test_unexported_pin() and test_export_requires_no_free().

Other arena tests print the reason first, for example arena_htab.c:

	printf("%s:SKIP:compiler doesn't support arena_cast\n", __func__);
	test__skip();

and arena_memcg.c:

	fprintf(stderr, "%s:SKIP:no memory controller or other env error\n",
		__func__);
	test__skip();

The bare test__skip() after an arena bpf_map_create() returning EOPNOTSUPP does match existing usage in arena_direct_value.c, so the main gap is the environment-dependent zero-canonical skip. Would a short reason such as "mapping at address 0 not permitted" or "arena not supported" be worth printing before each test__skip() here?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38093513563

#10 lore raw

PATCH 5/5 selftests/bpf: Test two-way arena access with two guests

In reply to Andrea Righi

Back a NUMA node in each of two simultaneous virtme-ng guests with a disjoint slice of one pinned host arena.

Create the host arena with BPF_F_ARENA_EXPORT and BPF_F_ARENA_NO_FREE. Guest arenas use only BPF_F_ARENA_NO_FREE because they do not export pins.

Exchange two values in each direction with direct BPF loads and stores. Check that a write to one slice leaves the other signal untouched. Unpin the host arena during the exchange, verify that neither BPF side can free the shared pages and keep one guest active after the other exits.

Let the host launcher own protocol deadlines, so ready guests can wait while another guest boots or completes its exchange. Bound host helper calls and register guest ownership before starting output readers. Handle SIGTERM and SIGINT through normal cleanup. Act as a child subreaper and kill and reap descendants with pidfds across sessions and process groups, checking that readers finish after process cleanup.

Inspect compiler support before loading BPF or booting guests, and skip when arena address-space conversions are unavailable. Supply the new flag definitions when building against older kernel BTF. Use the barrier-based acquire/release fallback on PowerPC, whose JIT does not support these atomic load/store instructions for arena memory.

Honor the compiler's v4 capability check and permissive BPF build mode. Give shared nodes room for boot allocations and allocator watermarks. Check the allocated page's NUMA node and skip confirmed fallback, while keeping physical-address bounds failures fatal on the expected node.

Assisted-by LLM
---
 tools/testing/selftests/bpf/.gitignore        |   2 +
 tools/testing/selftests/bpf/Makefile          |  27 +-
 tools/testing/selftests/bpf/arena_kvm.py      | 448 ++++++++++++++++++
 .../selftests/bpf/arena_kvm_guest.bpf.c       |  89 ++++
 tools/testing/selftests/bpf/arena_kvm_guest.c | 224 +++++++++
 .../selftests/bpf/arena_kvm_host.bpf.c        |  89 ++++
 tools/testing/selftests/bpf/arena_kvm_host.c  | 147 ++++++
 .../testing/selftests/bpf/arena_kvm_shared.h  |  60 +++
 8 files changed, 1085 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/bpf/arena_kvm.py
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_guest.bpf.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_guest.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_host.bpf.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_host.c
 create mode 100644 tools/testing/selftests/bpf/arena_kvm_shared.h
tools/testing/selftests/bpf/.gitignore +2 -0
index b815bf0d88774..9747ad37d698e 100644
@@ -47,3 +47,5 @@ verification_cert.h
 *.BTF.base
 usdt_1
 usdt_2
+/arena_kvm_guest-init
+/arena_kvm_host-runner
tools/testing/selftests/bpf/Makefile +26 -1
index a22be7efd1fa2..29fffa7857c19 100644
@@ -42,7 +42,13 @@ TEST_PROGS := test_kmod.sh \
 	test_bpftool_build.sh \
 	test_doc_build.sh \
 	test_xsk.sh \
-	test_xdp_features.sh
+	test_xdp_features.sh \
+	arena_kvm.py
+
+TEST_GEN_FILES += arena_kvm_guest-init arena_kvm_host-runner
+ifneq ($(CLANG_CPUV4),)
+TEST_GEN_FILES += arena_kvm_guest.bpf.o arena_kvm_host.bpf.o
+endif
 
 TEST_PROGS_EXTENDED := \
 	ima_setup.sh verify_sig_setup.sh
@@ -94,6 +100,25 @@ BPFTOOLDIR := $(TOOLSDIR)/bpf/bpftool
 HOST_BPFOBJ := $(HOST_BUILD_DIR)/libbpf/libbpf.a
 BPF_TARGET_ENDIAN:=$(if $(IS_LITTLE_ENDIAN),--target=bpfel,--target=bpfeb)
 
+# These helpers are installed beside arena_kvm.py, including for OUTPUT=.
+$(OUTPUT)/arena_kvm_%.bpf.o: arena_kvm_%.bpf.c arena_kvm_shared.h \
+		libarena/include/bpf_atomic.h \
+		libarena/include/bpf_arena_common.h \
+		$(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) | $(OUTPUT)
+	$(call msg,BPF,,$@)
+	$(Q)$(CLANG) $(BPF_CFLAGS) $(CLANG_CFLAGS) -O2 \
+		$(BPF_TARGET_ENDIAN) -mcpu=v4 -c $< -o $@ $(call skip_on_fail,BPF)
+
+$(OUTPUT)/arena_kvm_guest-init: arena_kvm_guest.c arena_kvm_shared.h \
+		$(BPFOBJ) | $(OUTPUT)
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $< $(BPFOBJ) $(LDLIBS) -lzstd -o $@
+
+$(OUTPUT)/arena_kvm_host-runner: arena_kvm_host.c arena_kvm_shared.h \
+		$(BPFOBJ) | $(OUTPUT)
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $< $(BPFOBJ) $(LDLIBS) -lzstd -o $@
+
 NON_CHECK_FEAT_TARGETS := clean docs-clean emit_tests
 CHECK_FEAT := $(filter-out $(NON_CHECK_FEAT_TARGETS),$(or $(MAKECMDGOALS), "none"))
 ifneq ($(CHECK_FEAT),)
tools/testing/selftests/bpf/arena_kvm.py new file +448 -0

448 lines not on this page · raw message

tools/testing/selftests/bpf/arena_kvm_guest.bpf.c new file +89 -0

89 lines not on this page · raw message

tools/testing/selftests/bpf/arena_kvm_guest.c new file +224 -0

224 lines not on this page · raw message

tools/testing/selftests/bpf/arena_kvm_host.bpf.c new file +89 -0

89 lines not on this page · raw message

tools/testing/selftests/bpf/arena_kvm_host.c new file +147 -0

147 lines not on this page · raw message

tools/testing/selftests/bpf/arena_kvm_shared.h new file +60 -0

60 lines not on this page · raw message

Signature · 1 line
2.56.0
#11 lore raw

Re: [PATCH 5/5] selftests/bpf: Test two-way arena access with two guests

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] selftests/bpf: unaligned cmsghdr buffer causes undefined behavior and SIGBUS - [Medium] bpf…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] selftests/bpf: unaligned cmsghdr buffer causes undefined behavior and SIGBUS
- [Medium] bpf: virtme-ng --numa argument risks unit mismatch in QEMU memory allocation --

commit f89b47f73364f408524cec1a644bd3003fa751ab
Author: Andrea Righi <[email protected]>

selftests/bpf: Test two-way arena access with two guests

This patch adds a selftest to verify two-way memory access between two simultaneous virtme-ng guests using a shared pinned host arena. It establishes mutual memory exchanges using direct BPF loads and stores.

Quoted 3 lines
tools/testing/selftests/bpf/arena_kvm.py +0 -0

[ ... ]

Quoted 7 lines
tools/testing/selftests/bpf/arena_kvm.py +7 -0
+def start_guest(pin, build, kernel, index, vng, env, guests):
+    node0_size = GUEST_MEMORY - SLICE_SIZE
+    base = GUEST_RAM_BASES[os.uname().machine] + node0_size
+    progress(f"Starting guest {index}: {SLICE_SIZE // (1024 * 1024)} MiB "
+             f"slice at file offset {index * SLICE_SIZE:#x}, "
+             f"NUMA node 1 physical base {base:#x}")
+    qemu_opts = (

tools/testing/selftests/bpf/arena_kvm.py line 219

Quoted 10 lines
tools/testing/selftests/bpf/arena_kvm.py +10 -0
+        f"-object memory-backend-file,id=signal,size={SLICE_SIZE},share=on,"
+        f"offset={index * SLICE_SIZE},mem-path={pin} "
+        "-numa node,nodeid=1,memdev=signal"
+    )
+    guest_command = [str(build / "arena_kvm_guest-init"),
+                     str(build / "arena_kvm_guest.bpf.o")]
+    command = [
+        vng, "--verbose", "--run", str(kernel), "--cpus", "2",
+        "--memory", f"{GUEST_MEMORY // (1024 * 1024)}M",
+        "--numa", str(node0_size),

[Severity: Medium]
Will this --numa argument cause a unit mismatch in QEMU's memory allocation?

The --numa argument is passed a raw byte count (node0_size), while --memory is explicitly passed in megabytes with an M suffix. Since QEMU's legacy -numa node,mem= option defaults to megabytes if no unit is specified, passing a large byte count could be interpreted as terabytes. This could potentially cause the virtme-ng VM launch to fail immediately.

[ ... ]

Quoted 3 lines
tools/testing/selftests/bpf/arena_kvm_host.c +0 -0

[ ... ]

Quoted 5 lines
tools/testing/selftests/bpf/arena_kvm_host.c +5 -0
+static int create_arena(const char *pin, int socket_fd, unsigned int pages)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts,
+		.map_flags = BPF_F_MMAPABLE | BPF_F_ARENA_NO_FREE |
+			     BPF_F_ARENA_EXPORT);

tools/testing/selftests/bpf/arena_kvm_host.c line 35

Quoted 10 lines
tools/testing/selftests/bpf/arena_kvm_host.c +10 -0
+	char control[CMSG_SPACE(sizeof(int))] = {};
+	char byte = 0;
+	struct iovec iov = { .iov_base = &byte, .iov_len = 1 };
+	struct msghdr msg = {
+		.msg_iov = &iov,
+		.msg_iovlen = 1,
+		.msg_control = control,
+		.msg_controllen = sizeof(control),
+	};
+	struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);

[Severity: Medium]
Does declaring the control buffer as a plain char array risk alignment faults?

The control array defaults to 1-byte alignment, but CMSG_FIRSTHDR casts it to a struct cmsghdr pointer. Accessing its members (such as cmsg_level and cmsg_type) requires the pointer to be aligned to the architecture's word boundary.

If the compiler places the char array at an unaligned address, this could invoke undefined behavior and cause a SIGBUS on architectures with strict alignment requirements. Using a union or proper alignment attributes would satisfy POSIX and C standards here.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
#12 lore raw

Re: [PATCH 5/5] selftests/bpf: Test two-way arena access with two guests

In reply to Andrea Righi
TL;DR: - Unconditional -lzstd in the new Makefile link rules may break the whole selftests/bpf build where libzstd is missing - arena_kvm.py doesn't use ksft.py reporting (minor; results still…
Quoted 1 line

selftests/bpf: Test two-way arena access with two guests

TL;DR:
- Unconditional -lzstd in the new Makefile link rules may break the whole selftests/bpf build where libzstd is missing
- arena_kvm.py doesn't use ksft.py reporting (minor; results still counted)

Quoted 4 lines
tools/testing/selftests/bpf/Makefile +0 -0
index a22be7efd1fa..29fffa7857c1 100644

[ ... ]

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +1 -0
@@ -94,6 +100,25 @@ BPFTOOLDIR := $(TOOLSDIR)/bpf/bpftool
 HOST_BPFOBJ := $(HOST_BUILD_DIR)/libbpf/libbpf.a
 BPF_TARGET_ENDIAN:=$(if $(IS_LITTLE_ENDIAN),--target=bpfel,--target=bpfeb)
 
+# These helpers are installed beside arena_kvm.py, including for OUTPUT=.

[ ... ]

tools/testing/selftests/bpf/Makefile BPFTOOLDIR := $(TOOLSDIR)/bpf/bpftool · line 124

Quoted 13 lines

+$(OUTPUT)/arena_kvm_guest-init: arena_kvm_guest.c arena_kvm_shared.h \

tools/testing/selftests/bpf/Makefile +9 -0
+		$(BPFOBJ) | $(OUTPUT)
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $< $(BPFOBJ) $(LDLIBS) -lzstd -o $@
+
+$(OUTPUT)/arena_kvm_host-runner: arena_kvm_host.c arena_kvm_shared.h \
+		$(BPFOBJ) | $(OUTPUT)
+	$(call msg,BINARY,,$@)
+	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $< $(BPFOBJ) $(LDLIBS) -lzstd -o $@
+
 NON_CHECK_FEAT_TARGETS := clean docs-clean emit_tests
 CHECK_FEAT := $(filter-out $(NON_CHECK_FEAT_TARGETS),$(or $(MAKECMDGOALS), "none"))
 ifneq ($(CHECK_FEAT),)

Is it intended to pass -lzstd unconditionally in these two link rules?

Neither arena_kvm_guest.c nor arena_kvm_host.c includes a zstd header or calls into zstd, and no other userspace link line in selftests/bpf uses -lzstd. test_progs, for example, links with plain $(LDLIBS), which is $(LIBELF_LIBS) -lz -lrt -lpthread.

Elsewhere in tools/, libzstd is optional. bpftool only adds it when the feature-libelf-zstd probe passes, and libelf builds without zstd are a supported configuration.

Both binaries are in TEST_GEN_FILES without a CLANG_CPUV4 or feature guard, and these link rules do not use $(call skip_on_fail,...). On a host or sysroot that has libelf and zlib development files but no linkable libzstd, wouldn't 'make -C tools/testing/selftests/bpf' now fail with "cannot find -lzstd", even with BPF_STRICT_BUILD=0? That would block all of selftests/bpf, not just this test.

Would dropping -lzstd be enough, since a dynamic libelf pulls in its own dependencies? If it is needed for a static libelf, it could be made conditional the same way bpftool does it.

Quoted 6 lines
tools/testing/selftests/bpf/arena_kvm.py new file +0 -0
new file mode 100755
index 000000000000..552bfcfb80ac
@@ -0,0 +1,448 @@

[ ... ]

Quoted 1 line

+KSFT_SKIP = 4

[ ... ]

Quoted 1 line

+ print(f"OK: two guests exchanged BPF values through arena slices {offsets}")

[ ... ]

Quoted 3 lines
tools/testing/selftests/bpf/arena_kvm.py new file +3 -0
+def skip(reason):
+    print(f"SKIP: {reason}")
+    return KSFT_SKIP

[ ... ]

Quoted 1 line

+if __name__ == "__main__":

[ ... ]

tools/testing/selftests/bpf/arena_kvm.py line 448

Quoted 2 lines
tools/testing/selftests/bpf/arena_kvm.py new file +2 -0
+        stop_children()
+    sys.exit(status)

A subsystem guideline flags this as potentially concerning: a test that prints its own ad hoc result text and exits directly, instead of going through the kselftest reporting helpers.

arena_kvm.py defines its own KSFT_SKIP, prints hand-written "SKIP: ..." and "OK: ..." lines, and then calls sys.exit(status) directly. tools/testing/selftests/kselftest/ksft.py provides KSFT_SKIP, test_result_pass(), test_result_skip(), test_result_fail(), exit_pass() and exit_fail(), and none of them are used here, so the output is not KTAP.

Should this script report through ksft.py instead?

For context, when the kselftest runner executes a TEST_PROGS entry, it prefixes the script's output with '# ' and turns the exit code into the TAP result line (exit 4 becomes '# SKIP'), so the one result this script produces is still counted correctly. The other TEST_PROGS scripts in this directory (test_kmod.sh, test_xsk.sh, test_xdp_features.sh) also print their own results. The guideline text is written about C test binaries and does not name the Python helper.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38093513563

#13 lore raw
In reply to Andrea Righi

Hi Andrea,

Thanks for sending this out, I'd really like to converge on a shared memory model that will work for both the goals of phantom tracker and for pvsched :) Been thinking about the tradeoffs of this vs the current structure of shared memory that we have in pvsched v3. I think this definitely brings some nice properties, but I have some questions/suggestions inline below.

Quoted 7 lines · Andrea Righi

As discussed in the BPF track and the sched_ext microconference at Linux Plumbers 2026 [1][2], shared BPF arenas can provide a generic channel for bidirectional zero-copy communication between BPF programs running on a host and its guests. One potential use is to exchange scheduling hints between host and guest sched_ext schedulers through shared memory.

Quoted 6 lines

The goal is to let both schedulers access the same underlying pages as BPF arena memory. Exchanging hints then requires no copies between separate host and guest buffers. With the arena file interface proposed here, the setup uses existing QEMU/KVM memory backends and guest NUMA interfaces, without further kernel or hypervisor changes to implement the communication channel.

Seems this relies on QEMU support for supplying offset+size for a virtual NUMA node; looking quickly this would seem to need additional work to support on other VMMs like CHV (and especially crosvm). Not a showstopper but just a comment that I think the PCI device model is easier to integrate in other VMMs.

Quoted 6 lines

A host scheduler's userspace component can create and pin an arena map, then use an arena allocator to partition it into per-guest chunks. Userspace passes each chunk to QEMU as a shared file-backed memory backend, using a bounded offset and length in the sized bpffs file. QEMU exposes only that chunk as a NUMA node inside the corresponding guest; the rest of the host arena is not mapped into guest RAM.

So this is the host allocates, guest uses model. Contrasted with pvsched v3's guest allocate, host pins.

Guest still needs some way of communicating back to the host which pages correspond to which data. Ie. a mapping of VCPU -> address within this arena memory that the host can then use to lookup per-VCPU info written by the guest. This comes for free with the guest allocation model in pvsched. I see you allude to this requirement below.

Another thing missing here is a doorbell, so this approach doesn't obviate the need for at least one guest BAR. If we have that, it's not a stretch to use an additional BAR for communicating the mappings (as pvsched does in v3).

Quoted 4 lines

The guest scheduler can create its own arena and allocate pages from that NUMA node. Those guest physical pages are backed by the host arena, so host and guest BPF programs can access the same memory through their respective arenas.

Two nice things here that we don't currently have with pvsched. One is that the host BPF programs can directly read/write the arena memory without needing an additional special kfunc to grab the right pointer (for pvsched specific BPF hooks in the host we intend to pass the pointer in the BPF context, but for sched_ext programs we intended to add a kfunc to e.g. lookup the pointer by VCPU). Second, the memory here is arena memory, which is quite nice. Currently pvsched just interprets it as a struct with a large data blob. Ideally we'd like to convert the pvsched model to use an arena.

Some additional drawbacks of the guest arena allocation you describe here:
- The extra NUMA node can presumably be used for unrelated allocations; there's no way to isolate it for just the purposes of our arena? Guest kernel and other guest userspace could allocate on that node.
- Memory allocations targeting that NUMA node (for the arena) could fail to find free space and fall back to another NUMA node; that at least seems solvable by forcing allocations to only come from the targetted node though
- This removes flexiblity of the guest to allocate memory how it likes. For example, with pvsched the guest can allocate the per-VCPU pages from the guest virtual nodes where those VCPUs reside.
- I don't think this works for confidential VMs? You'd end up needing the guest to share the pages back with the host, which is essentially what the guest-allocates model already does

I would definitely like to have the memory come to BPF via an arena for the flexibility of use, but I'm still a bit hung up on the host allocation part. I had been thinking about how to make the arena work using guest allocations, but nothing super concrete. Essentially the guest would create an arena and allocate pages, and stash the GPAs in the pvsched BAR. The host would need to pull those pages into its arena and deterministically allow access to them.

Quoted 10 lines

After identifying the corresponding arena offsets,
both sides exchange data through direct BPF loads and stores. Arena pointers are not relocated between mappings; a shared protocol can use validated slice-relative offsets instead.

QEMU is one potential consumer. The interface is generic: it exposes a pinned BPF arena as a sized bpffs file that ordinary userspace consumers can open and mmap, including consumers that require a file size when setting up shared memory.

Quoted 5 lines

The series separates page retention from file export:

* BPF_F_ARENA_NO_FREE makes BPF page-free operations no-ops, retaining allocated pages until the map is destroyed. This flag alone preserves ordinary arena pin behavior.

Does this imply that the arena remains allocated at its highest watermark, regardless if we have VMs leave the system (and thus the host attempt to free the arena pages that corresponded to the departed VMs)?

Smaller note: we probably want to at least make the free operation zero out the page so that no data is leftover when we reuse that page (ie. for a new VM).

Quoted 29 lines

* BPF_F_ARENA_EXPORT exposes the pin as a sized file and requires BPF_F_ARENA_NO_FREE, so BPF programs cannot remove backing pages while external consumers may still use them.

The map FD retains the canonical BPF address range. Exported mappings can select different virtual addresses and map disjoint slices. Faults resolve by file offset, and page zaps cover only the overlapping range in each exported VMA. Consumers establish the canonical range through map_extra or a full-capacity map FD mapping before mapping the pin. For exported arenas, the canonical range matches the fixed file size; ordinary and retention-only arenas still allow shorter mappings.

Pin permissions distinguish readers from writers. O_RDONLY consumers can create read-only shared mappings, but cannot request writable mappings or acquire write permission through mprotect(). Private mappings and file size changes are rejected. Open files and mappings retain the arena after unpinning until the last reference is released. The capacity is initialized before inode publication and can be queried with fstat() or lseek(fd, 0, SEEK_END).

Exported mappings retain the arena's lifecycle restrictions: they are not inherited across fork and cannot be split, expanded, or relocated. The documentation describes these limits and native tests exercise them.

Access through open() and mmap() intentionally uses filesystem permissions and the applicable file, mmap, and BPF map LSM checks. olicies specific to the bpf() syscall, including seccomp filters on that syscall, do not govern this file interface.

Quoted 4 lines

[1] https://lpc.events/event/20/contributions/2429/
[2] https://lpc.events/event/20/contributions/2482/

Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/arighi/linux.git bpf-pinned-arena

Best,
Josh

#14 lore raw

Hi Josh,

Quoted 8 lines · Josh Don

Hi Andrea,

Thanks for sending this out, I'd really like to converge on a shared memory model that will work for both the goals of phantom tracker and for pvsched :) Been thinking about the tradeoffs of this vs the current structure of shared memory that we have in pvsched v3. I think this definitely brings some nice properties, but I have some questions/suggestions inline below.

Same here, thanks for the detailed feedback. As mentioned at LPC, I think having a generic BPF arena based channel would be nice, but it doesn't exclude pvsched / phantom tracker: they can coexist, and ideally they'd share the same underlying mechanism to expose the memory to BPF as an arena.

That said, given your points and Alexei's follow-up, I'm going to drop the NUMA node / sized bpffs file approach and rework the series around arena importing external pages Some comments inline anyway, since I think a few of them still matter for the next version.

Quoted 16 lines · Andrea Righi
Quoted 7 lines

As discussed in the BPF track and the sched_ext microconference at Linux Plumbers 2026 [1][2], shared BPF arenas can provide a generic channel for bidirectional zero-copy communication between BPF programs running on a host and its guests. One potential use is to exchange scheduling hints between host and guest sched_ext schedulers through shared memory.

Quoted 6 lines

The goal is to let both schedulers access the same underlying pages as BPF arena memory. Exchanging hints then requires no copies between separate host and guest buffers. With the arena file interface proposed here, the setup uses existing QEMU/KVM memory backends and guest NUMA interfaces, without further kernel or hypervisor changes to implement the communication channel.

Quoted 5 lines

Seems this relies on QEMU support for supplying offset+size for a virtual NUMA node; looking quickly this would seem to need additional work to support on other VMMs like CHV (and especially crosvm). Not a showstopper but just a comment that I think the PCI device model is easier to integrate in other VMMs.

Agreed. The "no VMM changes" argument only really holds for QEMU. A BAR is a more portable contract between VMMs, and in QEMU it can still be done without new code via ivshmem-plain with a memfd memory backend, which is a nice fit with the memfd import Alexei proposed.

Quoted 11 lines
Quoted 7 lines

A host scheduler's userspace component can create and pin an arena map, then use an arena allocator to partition it into per-guest chunks. Userspace passes each chunk to QEMU as a shared file-backed memory backend, using a bounded offset and length in the sized bpffs file. QEMU exposes only that chunk as a NUMA node inside the corresponding guest; the rest of the host arena is not mapped into guest RAM.

So this is the host allocates, guest uses model. Contrasted with pvsched v3's guest allocate, host pins.

Quoted 6 lines

Guest still needs some way of communicating back to the host which pages correspond to which data. Ie. a mapping of VCPU -> address within this arena memory that the host can then use to lookup per-VCPU info written by the guest. This comes for free with the guest allocation model in pvsched. I see you allude to this requirement below.

I think the vCPU -> address mapping should be a protocol between the host and guest schedulers and should stay out of the kernel. What's missing in this version is a fixed rendezvous point: with the NUMA node the guest allocator decides where pages land, so the first offset has to be reported out of band (the selftest uses /proc/self/pagemap in the guest for that). With the slice mapped linearly into the guest arena from a BAR, offset 0 is known to both sides, and any per-vCPU layout can be negotiated in-band, either decided by the host or published by the guest in a directory at a known offset.

Quoted 5 lines

Another thing missing here is a doorbell, so this approach doesn't obviate the need for at least one guest BAR. If we have that, it's not a stretch to use an additional BAR for communicating the mappings (as pvsched does in v3).

For the sched_ext use case I had in mind (hints consumed opportunistically at scheduling events), polling was enough, so I didn't need a doorbell. But I agree that a generic channel will eventually need one, once there's a BAR it makes sense to use it for the shared memory as well.

Quoted 16 lines
Quoted 5 lines

The guest scheduler can create its own arena and allocate pages from that NUMA node. Those guest physical pages are backed by the host arena, so host and guest BPF programs can access the same memory through their respective arenas.

Two nice things here that we don't currently have with pvsched. One is that the host BPF programs can directly read/write the arena memory without needing an additional special kfunc to grab the right pointer (for pvsched specific BPF hooks in the host we intend to pass the pointer in the BPF context, but for sched_ext programs we intended to add a kfunc to e.g. lookup the pointer by VCPU). Second, the memory here is arena memory, which is quite nice. Currently pvsched just interprets it as a struct with a large data blob. Ideally we'd like to convert the pvsched model to use an arena.

Quoted 9 lines

Some additional drawbacks of the guest arena allocation you describe here:
- The extra NUMA node can presumably be used for unrelated allocations; there's no way to isolate it for just the purposes of our arena? Guest kernel and other guest userspace could allocate on that node.
- Memory allocations targeting that NUMA node (for the arena) could fail to find free space and fall back to another NUMA node; that at least seems solvable by forcing allocations to only come from the targetted node though

Agreed on both. There's also a worse problem that Alexei pointed out:
when the guest arena goes away the pages return to the guest buddy allocator while the host can still write to them. The NUMA node was an early prototype solution to avoid teaching arena to use pages it didn't allocate, so that's what I'm going to address next.

Quoted 3 lines

- This removes flexiblity of the guest to allocate memory how it likes. For example, with pvsched the guest can allocate the per-VCPU pages from the guest virtual nodes where those VCPUs reside.

Fair point. With a "map external pages into an arena" primitive both directions should be possible: host-allocated pages coming from a BAR in the guest, or guest-allocated pages pinned by the host and mapped into the host arena.

Quoted 3 lines

- I don't think this works for confidential VMs? You'd end up needing the guest to share the pages back with the host, which is essentially what the guest-allocates model already does

Right, with the NUMA node approach those pages would be private guest memory, so it doesn't work for CoCo guests as is. I think any model needs the memory to be explicitly shared with the host; I haven't looked at the details of how a BAR backed by struct pages would be mapped in a CoCo guest yet, so I'll keep this in mind for the rework.

Quoted 21 lines

I would definitely like to have the memory come to BPF via an arena for the flexibility of use, but I'm still a bit hung up on the host allocation part. I had been thinking about how to make the arena work using guest allocations, but nothing super concrete. Essentially the guest would create an arena and allocate pages, and stash the GPAs in the pvsched BAR. The host would need to pull those pages into its arena and deterministically allow access to them.

Quoted 12 lines

After identifying the corresponding arena offsets,
both sides exchange data through direct BPF loads and stores. Arena pointers are not relocated between mappings; a shared protocol can use validated slice-relative offsets instead.

QEMU is one potential consumer. The interface is generic: it exposes a pinned BPF arena as a sized bpffs file that ordinary userspace consumers can open and mmap, including consumers that require a file size when setting up shared memory.

The series separates page retention from file export:

Quoted 8 lines
Quoted 3 lines

* BPF_F_ARENA_NO_FREE makes BPF page-free operations no-ops, retaining allocated pages until the map is destroyed. This flag alone preserves ordinary arena pin behavior.

Does this imply that the arena remains allocated at its highest watermark, regardless if we have VMs leave the system (and thus the host attempt to free the arena pages that corresponded to the departed VMs)?

Yes, with this series memory stays at the high watermark: userspace could reuse a departed VM's chunk for a new VM, but pages are never returned until the arena is destroyed. That's not great, and I think the import model needs an explicit "release" operation per imported range for this exact reason.

Quoted 4 lines

Smaller note: we probably want to at least make the free operation zero out the page so that no data is leftover when we reuse that page (ie. for a new VM).

Ack.

Thanks,
-Andrea

#15 lore raw
In reply to Andrea Righi
Quoted 4 lines · Andrea Righi

The guest scheduler can create its own arena and allocate pages from that NUMA node. Those guest physical pages are backed by the host arena, so host and guest BPF programs can access the same memory through their respective arenas.

in addition to what Josh said...

That node is ordinary RAM for the guest.
When the guest scheduler exits arena_map_free() does __free_page() and the page goes back to the guest's buddy allocator. Nobody tells the host. The host prog keeps writing at that offset into whatever the guest allocates there next.
NO_FREE in the guest doesn't help. It covers the kfunc only.

Patch 5 needs a 16M node (256M on ppc) to get one page into the guest arena. Arena is 4G at most and it's one arena per prog, so that's 256 guests per host scheduler.

The NUMA node is a way to avoid teaching arena to use pages that it didn't allocate. I think that's the missing piece in either model:
host allocates -> guest arena is populated from a BAR, guest allocates -> host arena is populated from pinned guest pages. As Josh said the BAR is needed for the doorbell anyway.

If arena can do that, can the host side take the pages from memfd with memfd_pin_folios() like udmabuf does ?
VMM can map memfd at any address and offset already.
Then no need for NO_FREE, EXPORT, sized file in bpffs, 2nd kind of vma in arena, etc.

The loader does memfd_create() or opens a file in tmpfs, calls bpf_arena_import_memfd(&arena, addr, cnt, fd, off) from a syscall prog and gives the same file to QEMU the way patch 5 does with the pin. The imported pages stay until the arena is freed, so it's NO_FREE for the shared part only. libarena keeps working in the rest of the arena. arena_map_ext_pages() is what a BAR in the guest or pinned guest pages in the pvsched model would call.
What is missing: user space has to mmap the memfd to see these pages. The arena vma gives SIGBUS there.
Something like this (compile tested only):

kernel/bpf/arena.c +189 -13
index 0ff707707da0..3a231e3e3073 100644
@@ -4,6 +4,8 @@
 #include <linux/btf.h>
 #include <linux/cacheflush.h>
 #include <linux/err.h>
+#include <linux/file.h>
+#include <linux/memfd.h>
 #include <linux/irq_work.h>
 #include "linux/filter.h"
 #include <linux/llist.h>
@@ -55,12 +57,15 @@ struct bpf_arena {
 	struct vm_struct *kern_vm;
 	struct page *scratch_page;
 	struct range_tree rt;
-	/* protects rt and nr_pages */
+	/* set where the arena owns the pages, clear where they came from outside */
+	struct range_tree rt_own;
+	/* protects rt, rt_own and nr_pages */
 	rqspinlock_t spinlock;
 	/* number of pages currently populated in the arena */
 	u64 nr_pages;
 	struct list_head vma_list;
-	/* protects vma_list */
+	struct list_head imports;
+	/* protects vma_list and imports */
 	struct mutex lock;
 	u64 zap_gen;
 	struct mutex zap_mutex;
@@ -72,6 +77,13 @@ struct bpf_arena {
 static void arena_free_worker(struct work_struct *work);
 static void arena_free_irq(struct irq_work *iw);
 
+/* folios pinned by one bpf_arena_import_memfd() */
+struct arena_import {
+	struct list_head head;
+	struct folio **folios;
+	long nr_folios;
+};
+
 struct arena_free_span {
 	struct llist_node node;
 	unsigned long uaddr;
@@ -147,6 +159,8 @@ static long compute_pgoff(struct bpf_arena *arena, long uaddr)
 struct apply_range_data {
 	struct bpf_arena *arena;
 	struct llist_head *pages;
+	/* pages that are on LRU cannot be linked via pcp_llist */
+	struct page **ext_pages;
 	int i;
 };
 
@@ -158,17 +172,21 @@ struct clear_range_data {
 static int apply_range_set_cb(pte_t *pte, unsigned long addr, void *data)
 {
 	struct apply_range_data *d = data;
-	struct llist_node *node;
+	struct llist_node *node = NULL;
 	struct page *page;
 	pte_t pteval;
 
 	if (!data)
 		return 0;
 
-	node = READ_ONCE(d->pages->first);
-	if (WARN_ON_ONCE(!node))
-		return -EINVAL;
-	page = llist_entry(node, struct page, pcp_llist);
+	if (d->ext_pages) {
+		page = d->ext_pages[d->i];
+	} else {
+		node = READ_ONCE(d->pages->first);
+		if (WARN_ON_ONCE(!node))
+			return -EINVAL;
+		page = llist_entry(node, struct page, pcp_llist);
+	}
 	/* paranoia, similar to vmap_pages_pte_range() */
 	if (WARN_ON_ONCE(!pfn_valid(page_to_pfn(page))))
 		return -EINVAL;
@@ -201,7 +219,8 @@ static int apply_range_set_cb(pte_t *pte, unsigned long addr, void *data)
 		return -EBUSY;
 	set_pte_at(&init_mm, addr, pte, pteval);
 #endif
-	WARN_ON_ONCE(llist_del_first(d->pages) != node);
+	if (node)
+		WARN_ON_ONCE(llist_del_first(d->pages) != node);
 	d->i++;
 	WRITE_ONCE(d->arena->nr_pages, d->arena->nr_pages + 1);
 	return 0;
@@ -312,6 +331,7 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
 		arena->user_vm_end = arena->user_vm_start + vm_range;
 
 	INIT_LIST_HEAD(&arena->vma_list);
+	INIT_LIST_HEAD(&arena->imports);
 	init_llist_head(&arena->free_spans);
 	init_irq_work(&arena->free_irq, arena_free_irq);
 	INIT_WORK(&arena->free_work, arena_free_worker);
@@ -322,9 +342,12 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
 		goto err_free_arena;
 
 	range_tree_init(&arena->rt);
+	range_tree_init(&arena->rt_own);
 	err = range_tree_set(&arena->rt, 0, attr->max_entries);
+	if (!err)
+		err = range_tree_set(&arena->rt_own, 0, attr->max_entries);
 	if (err)
-		goto err_free_scratch;
+		goto err_destroy_rt;
 	mutex_init(&arena->lock);
 	mutex_init(&arena->zap_mutex);
 	raw_res_spin_lock_init(&arena->spinlock);
@@ -336,7 +359,7 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
 
 err_destroy_rt:
 	range_tree_destroy(&arena->rt);
-err_free_scratch:
+	range_tree_destroy(&arena->rt_own);
 	__free_page(arena->scratch_page);
 err_free_arena:
 	bpf_map_area_free(arena);
@@ -361,6 +384,10 @@ static int existing_page_cb(pte_t *ptep, unsigned long addr, void *data)
 	 */
 	if (page == arena->scratch_page)
 		return 0;
+	/* Not ours. arena_map_free() unpins them. */
+	if (is_range_tree_set(&arena->rt_own,
+			      compute_pgoff(arena, addr - bpf_arena_get_kern_vm_start(arena)), 1))
+		return 0;
 	/*
 	 * We do not update pte here:
 	 * 1. Nobody should be accessing bpf_arena's range outside of a kernel bug
@@ -375,6 +402,7 @@ static int existing_page_cb(pte_t *ptep, unsigned long addr, void *data)
 static void arena_map_free(struct bpf_map *map)
 {
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+	struct arena_import *imp, *tmp;
 
 	/*
 	 * Check that user vma-s are not around when bpf map is freed.
@@ -398,7 +426,13 @@ static void arena_map_free(struct bpf_map *map)
 	apply_to_existing_page_range(&init_mm, bpf_arena_get_kern_vm_start(arena),
 				     SZ_4G + GUARD_SZ / 2, existing_page_cb, arena);
 	free_vm_area(arena->kern_vm);
+	list_for_each_entry_safe(imp, tmp, &arena->imports, head) {
+		unpin_folios(imp->folios, imp->nr_folios);
+		kvfree(imp->folios);
+		kfree(imp);
+	}
 	range_tree_destroy(&arena->rt);
+	range_tree_destroy(&arena->rt_own);
 	__free_page(arena->scratch_page);
 	bpf_map_area_free(arena);
 }
@@ -488,7 +522,7 @@ static vm_fault_t arena_vm_fault(struct vm_fault *vmf)
 	struct mem_cgroup *new_memcg, *old_memcg;
 	LLIST_HEAD(pages);
 	struct page *page, *new_page = NULL;
-	struct apply_range_data data;
+	struct apply_range_data data = {};
 	vm_fault_t fault_ret;
 	long kbase, kaddr;
 	unsigned long flags;
@@ -530,6 +564,14 @@ static vm_fault_t arena_vm_fault(struct vm_fault *vmf)
 				    ? VM_FAULT_SIGSEGV : VM_FAULT_SIGBUS;
 			goto out_err_locked;
 		}
+		if (is_range_tree_set(&arena->rt_own, vmf->pgoff, 1)) {
+			/*
+			 * It's a page cache page of another file. Its rmap won't
+			 * find this vma. User space has to mmap that file.
+			 */
+			fault_ret = VM_FAULT_SIGBUS;
+			goto out_err_locked;
+		}
 		/* already have a page vmap-ed */
 		goto out;
 	}
@@ -745,7 +787,7 @@ static long arena_alloc_pages(struct bpf_arena *arena, long uaddr, long page_cnt
 	long page_cnt_max = (arena->user_vm_end - arena->user_vm_start) >> PAGE_SHIFT;
 	u64 kern_vm_start = bpf_arena_get_kern_vm_start(arena);
 	struct mem_cgroup *new_memcg, *old_memcg;
-	struct apply_range_data data;
+	struct apply_range_data data = {};
 	LLIST_HEAD(pages);
 	long mapped = 0;
 	unsigned long flags;
@@ -928,6 +970,13 @@ static void arena_free_pages(struct bpf_arena *arena, long uaddr, long page_cnt,
 	if (ret)
 		goto defer;
 
+	/* Imported pages stay where they are until the arena is freed */
+	if (is_range_tree_set(&arena->rt_own, pgoff, page_cnt)) {
+		raw_res_spin_unlock_irqrestore(&arena->spinlock, flags);
+		bpf_map_memcg_exit(old_memcg, new_memcg);
+		return;
+	}
+
 	range_tree_set(&arena->rt, pgoff, page_cnt);
 
 	init_llist_head(&free_pages);
@@ -1017,6 +1066,104 @@ static int arena_reserve_pages(struct bpf_arena *arena, long uaddr, u32 page_cnt
 	return ret;
 }
 
+/*
+ * Map pages that the arena didn't allocate at uaddr. The caller keeps them alive
+ * until arena_map_free(). A memfd is the first supplier. Pages of a guest pinned
+ * by the host or a BAR with struct pages in the guest can be the next.
+ */
+static int arena_map_ext_pages(struct bpf_arena *arena, long uaddr, struct page **pages,
+			       long page_cnt)
+{
+	long page_cnt_max = (arena->user_vm_end - arena->user_vm_start) >> PAGE_SHIFT;
+	struct apply_range_data data = { .arena = arena, .ext_pages = pages };
+	struct mem_cgroup *new_memcg, *old_memcg;
+	unsigned long flags;
+	long kaddr, pgoff;
+	int ret;
+
+	if (!uaddr || (uaddr & ~PAGE_MASK) || page_cnt > page_cnt_max)
+		return -EINVAL;
+	pgoff = compute_pgoff(arena, uaddr);
+	if (pgoff > page_cnt_max - page_cnt)
+		return -EINVAL;
+	kaddr = bpf_arena_get_kern_vm_start(arena) + (u32)uaddr;
+
+	if (raw_res_spin_lock_irqsave(&arena->spinlock, flags))
+		return -EBUSY;
+	bpf_map_memcg_enter(&arena->map, &old_memcg, &new_memcg);
+	ret = arena_adjust_tree(arena, uaddr, page_cnt, &pgoff);
+	if (ret)
+		goto out;
+	ret = range_tree_clear(&arena->rt_own, pgoff, page_cnt);
+	if (ret) {
+		range_tree_set(&arena->rt, pgoff, page_cnt);
+		goto out;
+	}
+	/* Page tables are there. It cannot fail in a range that was free. */
+	WARN_ON_ONCE(apply_to_page_range(&init_mm, kaddr, page_cnt << PAGE_SHIFT,
+					 apply_range_set_cb, &data));
+	flush_vmap_cache(kaddr, page_cnt << PAGE_SHIFT);
+out:
+	bpf_map_memcg_exit(old_memcg, new_memcg);
+	raw_res_spin_unlock_irqrestore(&arena->spinlock, flags);
+	return ret;
+}
+
+static int arena_import_memfd(struct bpf_arena *arena, long uaddr, u32 page_cnt, int fd, u64 off)
+{
+	struct page **pages = NULL;
+	struct arena_import *imp;
+	long i, cnt = 0;
+	pgoff_t pgoff;
+	int seals, ret = -ENOMEM;
+
+	CLASS(fd, f)(fd);
+	if (fd_empty(f))
+		return -EBADF;
+	/* bpf progs are going to write into it */
+	seals = memfd_get_seals(fd_file(f));
+	if (!(fd_file(f)->f_mode & FMODE_WRITE) ||
+	    (seals > 0 && (seals & (F_SEAL_WRITE | F_SEAL_FUTURE_WRITE))))
+		return -EPERM;
+	if (off & ~PAGE_MASK)
+		return -EINVAL;
+
+	imp = kzalloc_obj(*imp, GFP_KERNEL_ACCOUNT);
+	if (!imp)
+		return -ENOMEM;
+	imp->folios = kvmalloc_objs(*imp->folios, page_cnt, GFP_KERNEL_ACCOUNT);
+	pages = kvmalloc_objs(*pages, page_cnt, GFP_KERNEL);
+	if (!imp->folios || !pages)
+		goto err;
+
+	/* takes shmem and hugetlb files only, allocates what is not there yet */
+	imp->nr_folios = memfd_pin_folios(fd_file(f), off, off + ((u64)page_cnt << PAGE_SHIFT) - 1,
+					  imp->folios, page_cnt, &pgoff);
+	if (imp->nr_folios <= 0) {
+		ret = imp->nr_folios ?: -EINVAL;
+		goto err;
+	}
+	for (i = 0; i < imp->nr_folios; i++, pgoff = 0)
+		for (; pgoff < folio_size(imp->folios[i]) && cnt < page_cnt; pgoff += PAGE_SIZE)
+			pages[cnt++] = folio_page(imp->folios[i], pgoff >> PAGE_SHIFT);
+
+	ret = cnt == page_cnt ? arena_map_ext_pages(arena, uaddr, pages, page_cnt) : -EINVAL;
+	if (ret) {
+		unpin_folios(imp->folios, imp->nr_folios);
+		goto err;
+	}
+	mutex_lock(&arena->lock);
+	list_add(&imp->head, &arena->imports);
+	mutex_unlock(&arena->lock);
+	kvfree(pages);
+	return 0;
+err:
+	kvfree(pages);
+	kvfree(imp->folios);
+	kfree(imp);
+	return ret;
+}
+
 static void arena_free_worker(struct work_struct *work)
 {
 	struct bpf_arena *arena = container_of(work, struct bpf_arena, free_work);
@@ -1051,6 +1198,9 @@ static void arena_free_worker(struct work_struct *work)
 		kaddr = arena_vm_start + s->uaddr;
 		pgoff = compute_pgoff(arena, s->uaddr);
 
+		if (is_range_tree_set(&arena->rt_own, pgoff, page_cnt))
+			continue;
+
 		/* clear ptes and collect pages in free_pages llist */
 		apply_to_existing_page_range(&init_mm, kaddr, page_cnt << PAGE_SHIFT,
 					     apply_range_clear_cb, &cdata);
@@ -1162,6 +1312,22 @@ __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c
 
 	return arena_reserve_pages(arena, (long)ptr__ign, page_cnt);
 }
+
+/*
+ * Back page_cnt pages of the arena at addr with the pages of a memfd or a file
+ * in tmpfs starting at off. 'fd' belongs to the task that does BPF_PROG_RUN.
+ */
+__bpf_kfunc int bpf_arena_import_memfd(void *p__map, void *addr__ign, u32 page_cnt,
+				       int fd, u64 off)
+{
+	struct bpf_map *map = p__map;
+	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
+
+	if (map->map_type != BPF_MAP_TYPE_ARENA || !page_cnt || page_cnt > map->max_entries)
+		return -EINVAL;
+
+	return arena_import_memfd(arena, (long)addr__ign, page_cnt, fd, off);
+}
 __bpf_kfunc_end_defs();
 
 BTF_KFUNCS_START(arena_kfuncs)
@@ -1175,9 +1341,19 @@ static const struct btf_kfunc_id_set common_kfunc_set = {
 	.set   = &arena_kfuncs,
 };
 
+BTF_KFUNCS_START(arena_syscall_kfuncs)
+BTF_ID_FLAGS(func, bpf_arena_import_memfd, KF_ARENA_ARG2 | KF_SLEEPABLE)
+BTF_KFUNCS_END(arena_syscall_kfuncs)
+
+static const struct btf_kfunc_id_set syscall_kfunc_set = {
+	.owner = THIS_MODULE,
+	.set   = &arena_syscall_kfuncs,
+};
+
 static int __init kfunc_init(void)
 {
-	return register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC, &common_kfunc_set);
+	return register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC, &common_kfunc_set) ?:
+	       register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &syscall_kfunc_set);
 }
 late_initcall(kfunc_init);
 

---
pw-bot: cr

#16 lore raw
Quoted 26 lines · Alexei Starovoitov
Quoted 4 lines · Andrea Righi

The guest scheduler can create its own arena and allocate pages from that NUMA node. Those guest physical pages are backed by the host arena, so host and guest BPF programs can access the same memory through their respective arenas.

in addition to what Josh said...

That node is ordinary RAM for the guest.
When the guest scheduler exits arena_map_free() does __free_page() and the page goes back to the guest's buddy allocator. Nobody tells the host. The host prog keeps writing at that offset into whatever the guest allocates there next.
NO_FREE in the guest doesn't help. It covers the kfunc only.

Patch 5 needs a 16M node (256M on ppc) to get one page into the guest arena. Arena is 4G at most and it's one arena per prog, so that's 256 guests per host scheduler.

The NUMA node is a way to avoid teaching arena to use pages that it didn't allocate. I think that's the missing piece in either model:
host allocates -> guest arena is populated from a BAR, guest allocates -> host arena is populated from pinned guest pages. As Josh said the BAR is needed for the doorbell anyway.

Quoted 8 lines

If arena can do that, can the host side take the pages from memfd with memfd_pin_folios() like udmabuf does ?
VMM can map memfd at any address and offset already.
Then no need for NO_FREE, EXPORT, sized file in bpffs, 2nd kind of vma in arena, etc.

The loader does memfd_create() or opens a file in tmpfs, calls bpf_arena_import_memfd(&arena, addr, cnt, fd, off) from a syscall prog

As briefly discussed at LPC, I have nothing against adding support like this to the arena.

But as I was reading Josh's reply and thinking about multiple VMs and multiple vCPUs (that can also come and go), I kept thinking that it sounds like a pretty good fit of map-of-maps and ARRAY maps, where key for map-of-maps would be either entire VM or vCPU, whichever is more convenient. What is missing here is the ability to create an ARRAY map from memory that was already allocated before, instead of asking the kernel to do that. But once we have the ability to create ARRAY from a bunch of pre-existing memory (e.g., from memfd), this can be used both in a guest and in the host, and both models, guest allocates or host allocates, could be supported with the same model.

(regardless of this particular use case, I think it would be great to be able to create ARRAY from memfd for some other potential use cases)

One disadvantage of ARRAY that was mentioned before is the need to bounds check/limit access for PTR_TO_MAP_VALUE. I agree that's annoying, but what if we teach verifier to just automatically add bounds limiting sequence (e.g., masking if value size is power of two, slightly slower sequence for non-power-of-two, of course) for all such PTR_TO_MAP_VALUE accesses (e.g., if map is marked with some BPF_F_xxx flag).

Is it doable? It would be great, if so. Arena's singleton nature and 4GB limitation might not be the right fit for all cases, so it would be good to have an ARRAY-based solution available as well, IMO.

Thoughts?

Quoted 9 lines

and gives the same file to QEMU the way patch 5 does with the pin. The imported pages stay until the arena is freed, so it's NO_FREE for the shared part only. libarena keeps working in the rest of the arena. arena_map_ext_pages() is what a BAR in the guest or pinned guest pages in the pvsched model would call.
What is missing: user space has to mmap the memfd to see these pages. The arena vma gives SIGBUS there.
Something like this (compile tested only):

[...]

#17 lore raw
Quoted 4 lines · Andrii Nakryiko

kernel to do that. But once we have the ability to create ARRAY from a bunch of pre-existing memory (e.g., from memfd), this can be used both in a guest and in the host, and both models, guest allocates or host allocates, could be supported with the same model.

I don't think it will work for their use case.
With array of maps scx needs two lookups and two NULL checks on every access, and the result cannot be kept in the task ctx which is in arena today, since ptr_to_map_value becomes scalar. unless everything is converted to magically bounded array, but...

Quoted 3 lines

annoying, but what if we teach verifier to just automatically add bounds limiting sequence (e.g., masking if value size is power of two, slightly slower sequence for non-power-of-two, of course)

while working on arena ~year back I tried to add runtime masking to ptr_to_map_value and couldn't figure it out.
then in ~april I tried again with llm-s and failed again.
the main issue is that ptr_to_map_value is a pointer,
and progs keep adding random stuff to it.
There is no way to split it into 'base + offset'
and apply mask on 'offset'.
I tried alignment tricks, like align each ptr_to_map_value's base to 64k or such, but still failed. Something needs to remember upper bits to see when math overflows.
That's why Barret was using explicit mask in C on every array access (when he implemented all kinds of complex data structs on top of arrays). Every access in C was base + mask(offset).

Array from memfd is a separate topic. So far I'm failing to see a use case, but if it's not a lot of code, then why not.

#18 lore raw
Quoted 11 lines · Alexei Starovoitov
Quoted 4 lines · Andrii Nakryiko

kernel to do that. But once we have the ability to create ARRAY from a bunch of pre-existing memory (e.g., from memfd), this can be used both in a guest and in the host, and both models, guest allocates or host allocates, could be supported with the same model.

I don't think it will work for their use case.
With array of maps scx needs two lookups and two NULL checks on every access, and the result cannot be kept in the task ctx which is in arena today, since ptr_to_map_value becomes scalar. unless everything is converted to magically bounded array, but...

Agreed, from a sched_ext perspective, arena is really what we need. Most of the new schedulers already keep their per-task and per-CPU state in the arena, and an idea could be to link that state directly to the shared hints, e.g., keep a pointer to the vCPU's hint slot in the per-CPU context and dereference it in the hot paths, without any lookup.

Andrii, about the 4G / single arena limitation: for this use case I don't think it's not a concern, the shared part is a few pages per vCPU. Having the shared pages in the same arena as the rest of the scheduler state is exactly the property we want.

Thanks,
-Andrea

#19 lore raw
In reply to Alexei Starovoitov
Quoted 11 lines · Alexei Starovoitov
Quoted 4 lines · Andrii Nakryiko

kernel to do that. But once we have the ability to create ARRAY from a bunch of pre-existing memory (e.g., from memfd), this can be used both in a guest and in the host, and both models, guest allocates or host allocates, could be supported with the same model.

I don't think it will work for their use case.
With array of maps scx needs two lookups and two NULL checks on every access, and the result cannot be kept in the task ctx which is in arena today, since ptr_to_map_value becomes scalar.

fair enough

Quoted 6 lines

unless everything is converted to magically bounded array, but...

Quoted 3 lines

annoying, but what if we teach verifier to just automatically add bounds limiting sequence (e.g., masking if value size is power of two, slightly slower sequence for non-power-of-two, of course)

Quoted 7 lines

while working on arena ~year back I tried to add runtime masking to ptr_to_map_value and couldn't figure it out.
then in ~april I tried again with llm-s and failed again. the main issue is that ptr_to_map_value is a pointer, and progs keep adding random stuff to it.
There is no way to split it into 'base + offset'
and apply mask on 'offset'.

yeah, good point, some food for thought, I haven't thought much about this, will keep it on backburner for now

Quoted 9 lines

I tried alignment tricks, like align each ptr_to_map_value's base to 64k or such, but still failed. Something needs to remember upper bits to see when math overflows.
That's why Barret was using explicit mask in C on every array access (when he implemented all kinds of complex data structs on top of arrays). Every access in C was base + mask(offset).

Array from memfd is a separate topic. So far I'm failing to see a use case, but if it's not a lot of code, then why not.

Yeah, there are just so many cases where we have some memory that is special in some way (e.g., huge page-backed memory, or some IOMMU, e.g., that doorbell stuff, which, if I understand correctly, is done though writes to specially set up memory ranges), and we can't shove all this into BPF_MAP_CREATE interface if we want to have BPF program accessing that memory. So having a generic way to just bolt ARRAY on top of already created memory seems like a useful primitive.

So we'll give it a try and see how it looks.

#20 lore raw
Quoted 8 lines · Andrii Nakryiko

Yeah, there are just so many cases where we have some memory that is special in some way (e.g., huge page-backed memory, or some IOMMU, e.g., that doorbell stuff, which, if I understand correctly, is done though writes to specially set up memory ranges), and we can't shove all this into BPF_MAP_CREATE interface if we want to have BPF program accessing that memory. So having a generic way to just bolt ARRAY on top of already created memory seems like a useful primitive.

I see. In such light the use case makes sense to me.

#21 lore raw
In reply to Alexei Starovoitov

Hi Alexei,

Quoted 6 lines · Alexei Starovoitov
Quoted 4 lines · Andrea Righi

The guest scheduler can create its own arena and allocate pages from that NUMA node. Those guest physical pages are backed by the host arena, so host and guest BPF programs can access the same memory through their respective arenas.

Quoted 8 lines

in addition to what Josh said...

That node is ordinary RAM for the guest.
When the guest scheduler exits arena_map_free() does __free_page() and the page goes back to the guest's buddy allocator. Nobody tells the host. The host prog keeps writing at that offset into whatever the guest allocates there next.
NO_FREE in the guest doesn't help. It covers the kfunc only.

Right, that's a design problem, NO_FREE only protects the host side, the guest side lifetime is not covered at all, so the host can corrupt guest memory after the guest scheduler exits.

Quoted 4 lines

Patch 5 needs a 16M node (256M on ppc) to get one page into the guest arena. Arena is 4G at most and it's one arena per prog, so that's 256 guests per host scheduler.

Agreed, the memory block granularity makes the NUMA node approach wasteful and doesn't scale.

Quoted 7 lines

The NUMA node is a way to avoid teaching arena to use pages that it didn't allocate. I think that's the missing piece in either model:
host allocates -> guest arena is populated from a BAR, guest allocates -> host arena is populated from pinned guest pages. As Josh said the BAR is needed for the doorbell anyway.

Quoted 5 lines

If arena can do that, can the host side take the pages from memfd with memfd_pin_folios() like udmabuf does ?
VMM can map memfd at any address and offset already.
Then no need for NO_FREE, EXPORT, sized file in bpffs, 2nd kind of vma in arena, etc.

Agreed, this is much simpler and it covers both models. Mapping the BAR linearly into the guest arena also gives both sides a fixed rendezvous point (offset 0 of the memfd is a known address in both arenas), which removes the out-of-band offset discovery that the selftest does via /proc/self/pagemap.

I'll drop NO_FREE, EXPORT and the bpffs changes and rework the series around arena_map_ext_pages() + bpf_arena_import_memfd(), starting from your patch.

Essentially the plan is:
1) host: bpf_arena_import_memfd() (based on your patch),
2) guest: populate the guest arena from a BAR using arena_map_ext_pages() (in QEMU, ivshmem-plain with the same memfd as memory backend, so no VMM changes),
3) convert the two-guest selftest to this path.

Quoted 6 lines

The loader does memfd_create() or opens a file in tmpfs, calls bpf_arena_import_memfd(&arena, addr, cnt, fd, off) from a syscall prog and gives the same file to QEMU the way patch 5 does with the pin. The imported pages stay until the arena is freed, so it's NO_FREE for the shared part only. libarena keeps working in the rest of the arena.

That's fine for a single long-running guest, but with VMs coming and going the host arena would only grow (this was also Josh's concern about the high watermark). Would it be ok with an explicit release of an imported range (clear the ptes, flush, set rt_own back, unpin the folios)? A BPF prog touching a released range would then hit the usual arena fault handling, like any unallocated page.

Quoted 2 lines

arena_map_ext_pages() is what a BAR in the guest or pinned guest pages in the pvsched model would call.

For the BAR case the guest needs struct pages for the BAR memory (memremap_pages() / ZONE_DEVICE or similar) to go through arena_map_ext_pages(). I'll look into the least invasive way to do that.

Quoted 2 lines

What is missing: user space has to mmap the memfd to see these pages. The arena vma gives SIGBUS there.

I think that's acceptable for the hint channel: the userspace side of the scheduler can mmap the memfd (or the BAR in the guest) separately and use offsets.

Thanks,
-Andrea

#22 lore raw
Quoted 6 lines · Andrea Righi

That's fine for a single long-running guest, but with VMs coming and going the host arena would only grow (this was also Josh's concern about the high watermark). Would it be ok with an explicit release of an imported range (clear the ptes, flush, set rt_own back, unpin the folios)? A BPF prog touching a released range would then hit the usual arena fault handling, like any unallocated page.

make sense to me.
In my prototype arena_free_pages() sees the imported range in rt_own. Instead of returning early it can do the above and
give the range back to rt.