Skip to contentSkip to conversation

bpf: Populate mmap-able array maps lazily

PATCH Superseded Song Liu 4 participants 14 messages Jul 22 – Jul 29

Subsystems (4)

v1 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Song Liu

author reviewer
#0 lore raw

cover bpf: Populate mmap-able array maps lazily

An mmap-able BPF array map (BPF_F_MMAPABLE) has its backing memory vmalloc'ed up front at map creation time. array_map_mmap() then wires up the whole mapping eagerly via remap_vmalloc_range(), which calls vm_insert_page() for every page of the map. This makes every mmap() cost O(number of pages): an 8MiB map inserts 2048 PTEs on each mmap() and tears them all down again on munmap(), even when user space only touches a few pages (or none at all).

This series makes the mapping populate lazily instead, the same way the arena map already does, turning mmap()/munmap() of an mmap-able array from O(map size) into O(1). The pages are inserted on demand by a new map_mmap_fault() callback, dispatched from the shared bpf_map_default_vmops so that the existing VMA open/close accounting (VM_MAYWRITE write-active tracking, freeze handling) stays centralized.

Callers that want the pages populated up front can still request that explicitly with MAP_POPULATE, matching regular file semantics. Kernel-side access to the map (via the vmalloc address) is unaffected.

Patch 1 implements the lazy faulting. Patch 2 adds an "arraymap-mmap" benchmark that mmap()s/munmap()s an mmap-able array map from a configurable number of threads, useful to observe how the cost scales with the mapping size.

Benchmark results:

Before the patch
nr_threads: 1, map_size: 1048576
arraymap-mmap: throughput: 0.034 +/- 0.000 M ops/s, latency: 29134.211 ns/op
nr_threads: 1, map_size: 8388608
arraymap-mmap: throughput: 0.005 +/- 0.000 M ops/s, latency: 220365.807 ns/op

       nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.001 +/- 0.000 M ops/s, latency: 1749642.119 ns/op

After the patch

    nr_threads: 1, map_size: 1048576
arraymap-mmap: throughput: 0.819 +/- 0.002 M ops/s, latency: 1221.476 ns/op
    nr_threads: 1, map_size: 8388608
arraymap-mmap: throughput: 0.808 +/- 0.005 M ops/s, latency: 1236.975 ns/op
    nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.738 +/- 0.003 M ops/s, latency: 1355.100 ns/op

Song Liu (2):
bpf: Populate mmap-able array map memory lazily selftests/bpf: Add mmap/munmap benchmark for array maps

 include/linux/bpf.h                           |   1 +
 kernel/bpf/arraymap.c                         |  40 ++++-
 kernel/bpf/syscall.c                          |  17 ++
 tools/testing/selftests/bpf/Makefile          |   1 +
 tools/testing/selftests/bpf/bench.c           |   4 +
 .../bpf/benchs/bench_arraymap_mmap.c          | 146 ++++++++++++++++++
 .../bpf/benchs/run_bench_arraymap_mmap.sh     |  15 ++
 7 files changed, 221 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c
 create mode 100755 tools/testing/selftests/bpf/benchs/run_bench_arraymap_mmap.sh

--
2.53.0-Meta

author reviewer
#1 lore raw

PATCH 1/2 bpf: Populate mmap-able array map memory lazily

An mmap-able BPF array map (BPF_F_MMAPABLE) has its backing memory vmalloc'ed up front at map creation time. array_map_mmap() then wired up the whole mapping eagerly via remap_vmalloc_range(), which calls vm_insert_page() for every page of the map. For large maps this makes every mmap() O(number of pages): an 8MiB map inserts 2048 PTEs per mmap() and tears them all down again on munmap(), even when user space only touches a few pages (or none at all).

Populate the mapping lazily instead, the same way the arena map already does. array_map_mmap() now only performs the bounds check and returns, leaving the PTEs unpopulated; the pages are inserted on demand by a new array_map_mmap_fault() handler. Because the memory is already resident, the fault handler simply resolves the vmalloc page and hands it to the fault path.

To keep the existing VMA open/close accounting (VM_MAYWRITE write-active tracking, freeze handling) centralized, the fault handler is reached through a new optional ->map_mmap_fault callback dispatched from the shared bpf_map_default_vmops, rather than each map installing its own vm_operations_struct.

This turns mmap()/munmap() of an mmap-able array from O(map size) into O(1). Callers that want the pages populated up front can still request that explicitly with MAP_POPULATE, matching regular file semantics. Kernel-side access to the map (via the vmalloc address) is unaffected.

Assisted-by Claude:claude-opus-4-8
---
 include/linux/bpf.h   |  1 +
 kernel/bpf/arraymap.c | 40 +++++++++++++++++++++++++++++++++++++---
 kernel/bpf/syscall.c  | 17 +++++++++++++++++
 3 files changed, 55 insertions(+), 3 deletions(-)
include/linux/bpf.h +1 -0
index e066f44a9c05..b536f3c9e103 100644
@@ -145,6 +145,7 @@ struct bpf_map_ops {
 	int (*map_direct_value_meta)(const struct bpf_map *map,
 				     u64 imm, u32 *off);
 	int (*map_mmap)(struct bpf_map *map, struct vm_area_struct *vma);
+	vm_fault_t (*map_mmap_fault)(struct bpf_map *map, struct vm_fault *vmf);
 	__poll_t (*map_poll)(struct bpf_map *map, struct file *filp,
 			     struct poll_table_struct *pts);
 	unsigned long (*map_get_unmapped_area)(struct file *filep, unsigned long addr,
kernel/bpf/arraymap.c +37 -3
index 248b4818178c..1711be582907 100644
@@ -576,7 +576,6 @@ static int array_map_check_btf(struct bpf_map *map,
 static int array_map_mmap(struct bpf_map *map, struct vm_area_struct *vma)
 {
 	struct bpf_array *array = container_of(map, struct bpf_array, map);
-	pgoff_t pgoff = PAGE_ALIGN(sizeof(*array)) >> PAGE_SHIFT;
 
 	if (!(map->map_flags & BPF_F_MMAPABLE))
 		return -EINVAL;
@@ -585,8 +584,42 @@ static int array_map_mmap(struct bpf_map *map, struct vm_area_struct *vma)
 	    PAGE_ALIGN((u64)array->map.max_entries * array->elem_size))
 		return -EINVAL;
 
-	return remap_vmalloc_range(vma, array_map_vmalloc_addr(array),
-				   vma->vm_pgoff + pgoff);
+	/*
+	 * The backing memory is vmalloc'ed up front, so instead of wiring up
+	 * every PTE here we let array_map_mmap_fault() insert pages on demand.
+	 * remap_vmalloc_range_partial() used to set these flags for us; keep
+	 * them to preserve behavior (no VMA expansion, excluded from coredump).
+	 */
+	vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
+
+	return 0;
+}
+
+static vm_fault_t array_map_mmap_fault(struct bpf_map *map,
+				       struct vm_fault *vmf)
+{
+	struct bpf_array *array = container_of(map, struct bpf_array, map);
+	pgoff_t pgoff = PAGE_ALIGN(sizeof(*array)) >> PAGE_SHIFT;
+	struct page *page;
+	void *kaddr;
+
+	/*
+	 * vmf->pgoff is the page offset within the mapping (vma->vm_pgoff plus
+	 * the offset of the faulting address). The data pages start after the
+	 * bpf_array header, hence the extra pgoff. This mirrors the offset that
+	 * remap_vmalloc_range(vma, ..., vma->vm_pgoff + pgoff) used to apply.
+	 */
+	kaddr = array_map_vmalloc_addr(array) +
+		((u64)(vmf->pgoff + pgoff) << PAGE_SHIFT);
+
+	page = vmalloc_to_page(kaddr);
+	if (!page)
+		return VM_FAULT_SIGBUS;
+
+	get_page(page);
+	vmf->page = page;
+
+	return 0;
 }
 
 static bool array_map_meta_equal(const struct bpf_map *meta0,
@@ -812,6 +845,7 @@ const struct bpf_map_ops array_map_ops = {
 	.map_direct_value_addr = array_map_direct_value_addr,
 	.map_direct_value_meta = array_map_direct_value_meta,
 	.map_mmap = array_map_mmap,
+	.map_mmap_fault = array_map_mmap_fault,
 	.map_seq_show_elem = array_map_seq_show_elem,
 	.map_check_btf = array_map_check_btf,
 	.map_lookup_batch = generic_map_lookup_batch,
kernel/bpf/syscall.c +17 -0
index 0ff9e3aa293d..fb0e1f55c487 100644
@@ -1077,9 +1077,26 @@ static void bpf_map_mmap_close(struct vm_area_struct *vma)
 		bpf_map_write_active_dec(map);
 }
 
+/*
+ * Called for maps that populate their memory-mapped region lazily, i.e. that
+ * return without wiring up any PTEs from their ->map_mmap callback. Maps that
+ * populate the whole VMA eagerly (e.g. via remap_vmalloc_range()) never reach
+ * here.
+ */
+static vm_fault_t bpf_map_mmap_fault(struct vm_fault *vmf)
+{
+	struct bpf_map *map = vmf->vma->vm_private_data;
+
+	if (!map->ops->map_mmap_fault)
+		return VM_FAULT_SIGBUS;
+
+	return map->ops->map_mmap_fault(map, vmf);
+}
+
 static const struct vm_operations_struct bpf_map_default_vmops = {
 	.open		= bpf_map_mmap_open,
 	.close		= bpf_map_mmap_close,
+	.fault		= bpf_map_mmap_fault,
 };
 
 static int bpf_map_mmap(struct file *filp, struct vm_area_struct *vma)
Signature · 1 line
2.53.0-Meta
#2 lore raw

Re: [PATCH bpf-next 1/2] bpf: Populate mmap-able array map memory lazily

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Critical] An integer overflow in `array_map_mmap` on 32-bit architectures allows bypassing the VMA…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] An integer overflow in array_map_mmap on 32-bit architectures allows bypassing the VMA bounds check. Combined with pointer arithmetic truncation in the new array_map_mmap_fault handler, an attacker can map the struct bpf_array header into user space, leading to Local Privilege Escalation (LPE) by overwriting the BPF map's operations pointer. --

--- Patch [1]: [PATCH bpf-next 1/2] bpf: Populate mmap-able array map memory lazily --- commit 07e8d5ac0a564f432fdda75a2eb43f2790e3860c
Author: Song Liu <[email protected]>

bpf: Populate mmap-able array map memory lazily

This commit modifies mmap-able BPF array maps to populate memory lazily rather than upfront at map creation time. It adds a new map_mmap_fault callback to handle inserting pages on demand, reducing the mmap and munmap complexity to O(1) for large maps.

Quoted 12 lines
kernel/bpf/arraymap.c +0 -1
@@ -576,7 +576,6 @@ static int array_map_check_btf(struct bpf_map *map,
 static int array_map_mmap(struct bpf_map *map, struct vm_area_struct *vma)
 {
 	struct bpf_array *array = container_of(map, struct bpf_array, map);
-	pgoff_t pgoff = PAGE_ALIGN(sizeof(*array)) >> PAGE_SHIFT;
 
 	if (!(map->map_flags & BPF_F_MMAPABLE))
 		return -EINVAL;

kernel/bpf/arraymap.c static int array_map_mmap(struct bpf_map *map, struct vm_area_struct *vma) · line 585

Quoted 2 lines
kernel/bpf/arraymap.c +0 -0
@@ -585,8 +584,42 @@ static int array_map_mmap(struct bpf_map *map, struct vm_area_struct *vma)
 	    PAGE_ALIGN((u64)array->map.max_entries * array->elem_size))
 		return -EINVAL;

[Severity: Critical]
Does this bounds check in array_map_mmap overflow on 32-bit architectures?

Since the left-hand side calculation uses 32-bit unsigned long arithmetic, could an excessive page offset overflow the result to zero, thereby bypassing the check entirely?

Quoted 19 lines
kernel/bpf/arraymap.c +17 -2
-	return remap_vmalloc_range(vma, array_map_vmalloc_addr(array),
-				   vma->vm_pgoff + pgoff);
+	/*
+	 * The backing memory is vmalloc'ed up front, so instead of wiring up
+	 * every PTE here we let array_map_mmap_fault() insert pages on demand.
+	 * remap_vmalloc_range_partial() used to set these flags for us; keep
+	 * them to preserve behavior (no VMA expansion, excluded from coredump).
+	 */
+	vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
+
+	return 0;
+}
+
+static vm_fault_t array_map_mmap_fault(struct bpf_map *map,
+				       struct vm_fault *vmf)
+{
+	struct bpf_array *array = container_of(map, struct bpf_array, map);
+	pgoff_t pgoff = PAGE_ALIGN(sizeof(*array)) >> PAGE_SHIFT;
+	struct page *page;

kernel/bpf/arraymap.c static int array_map_mmap(struct bpf_map *map, struct vm_area_struct *vma) · line 613

Quoted 10 lines
kernel/bpf/arraymap.c +10 -0
+	void *kaddr;
+
+	/*
+	 * vmf->pgoff is the page offset within the mapping (vma->vm_pgoff plus
+	 * the offset of the faulting address). The data pages start after the
+	 * bpf_array header, hence the extra pgoff. This mirrors the offset that
+	 * remap_vmalloc_range(vma, ..., vma->vm_pgoff + pgoff) used to apply.
+	 */
+	kaddr = array_map_vmalloc_addr(array) +
+		((u64)(vmf->pgoff + pgoff) << PAGE_SHIFT);

[Severity: Critical]
Can this pointer arithmetic lead to truncation when calculating kaddr?

On 32-bit systems, array_map_vmalloc_addr returns a 32-bit pointer. If the 64-bit offset calculation is added to this pointer, does it truncate the upper bits? If an overflowed pgoff was passed from the mmap bounds check, could this truncation cause the mapping to wrap around and incorrectly expose the internal bpf_array header memory to user space?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author reviewer
#3 lore raw

PATCH 2/2 selftests/bpf: Add mmap/munmap benchmark for array maps

In reply to Song Liu

Add a benchmark that repeatedly mmap()s and munmap()s an mmap-able BPF array map (BPF_F_MMAPABLE) from a configurable number of threads. Both the number of worker threads and the size of the map are tunable via --nr-threads and --map-size, defaulting to 16 threads and 8MiB.

The benchmark is useful to observe how the cost of mmap()/munmap() scales with the mapping size, e.g. eager vs lazy page-table population.

Assisted-by Claude:claude-opus-4-8
---
 tools/testing/selftests/bpf/Makefile          |   1 +
 tools/testing/selftests/bpf/bench.c           |   4 +
 .../bpf/benchs/bench_arraymap_mmap.c          | 146 ++++++++++++++++++
 .../bpf/benchs/run_bench_arraymap_mmap.sh     |  15 ++
 4 files changed, 166 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c
 create mode 100755 tools/testing/selftests/bpf/benchs/run_bench_arraymap_mmap.sh
tools/testing/selftests/bpf/Makefile +1 -0
index 55d394438705..b6593fb1135b 100644
@@ -1003,6 +1003,7 @@ $(OUTPUT)/bench: $(OUTPUT)/bench.o \
 		 $(OUTPUT)/bench_bpf_timing.o \
 		 $(OUTPUT)/bench_bpf_nop.o \
 		 $(OUTPUT)/bench_xdp_lb.o \
+		 $(OUTPUT)/bench_arraymap_mmap.o \
 		 $(OUTPUT)/usdt_1.o \
 		 $(OUTPUT)/usdt_2.o \
 		 #
tools/testing/selftests/bpf/bench.c +4 -0
index 3d9d2cd7764b..192d5fcb7a74 100644
@@ -287,6 +287,7 @@ extern struct argp bench_crypto_argp;
 extern struct argp bench_sockmap_argp;
 extern struct argp bench_lpm_trie_map_argp;
 extern struct argp bench_xdp_lb_argp;
+extern struct argp bench_arraymap_mmap_argp;
 
 static const struct argp_child bench_parsers[] = {
 	{ &bench_ringbufs_argp, 0, "Ring buffers benchmark", 0 },
@@ -304,6 +305,7 @@ static const struct argp_child bench_parsers[] = {
 	{ &bench_sockmap_argp, 0, "bpf sockmap benchmark", 0 },
 	{ &bench_lpm_trie_map_argp, 0, "LPM trie map benchmark", 0 },
 	{ &bench_xdp_lb_argp, 0, "XDP load-balancer benchmark", 0 },
+	{ &bench_arraymap_mmap_argp, 0, "arraymap-mmap benchmark", 0 },
 	{},
 };
 
@@ -582,6 +584,7 @@ extern const struct bench bench_lpm_trie_delete;
 extern const struct bench bench_lpm_trie_free;
 extern const struct bench bench_bpf_nop;
 extern const struct bench bench_xdp_lb;
+extern const struct bench bench_arraymap_mmap;
 
 static const struct bench *benchs[] = {
 	&bench_count_global,
@@ -665,6 +668,7 @@ static const struct bench *benchs[] = {
 	&bench_lpm_trie_free,
 	&bench_bpf_nop,
 	&bench_xdp_lb,
+	&bench_arraymap_mmap,
 };
 
 static void find_benchmark(void)
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +146 -0

146 lines not on this page · raw message

tools/testing/selftests/bpf/benchs/run_bench_arraymap_mmap.sh new file +15 -0
new file mode 100755
index 000000000000..3efd441cd653
@@ -0,0 +1,15 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+source ./benchs/run_common.sh
+
+set -eufo pipefail
+
+for t in 1 4 8 16; do
+for s in 1048576 8388608 67108864; do
+subtitle "nr_threads: $t, map_size: $s"
+	summarize_ops "arraymap-mmap: " \
+	    "$($RUN_BENCH --nr-threads $t --map-size $s arraymap-mmap)"
+	printf "\n"
+done
+done
Signature · 1 line
2.53.0-Meta
#4 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Mutating `env.producer_cnt` inside `validate()` causes a heap buffer overflow in the core…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Mutating env.producer_cnt inside validate() causes a heap buffer overflow in the core benchmark runner framework. --

--- Patch [2]: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps --- commit 39f3ece9fcc36c46f5da11156d115c4340de663c
Author: Song Liu <[email protected]>

selftests/bpf: Add mmap/munmap benchmark for array maps

Add a benchmark that repeatedly mmap()s and munmap()s an mmap-able BPF array map (BPF_F_MMAPABLE) from a configurable number of threads. Both the number of worker threads and the size of the map are tunable via --nr-threads and --map-size, defaulting to 16 threads and 8MiB.

Quoted 6 lines
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +0 -0
new file mode 100644
index 0000000000000..356d80a85cf01
@@ -0,0 +1,146 @@

[ ... ]

Quoted 4 lines
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +4 -0
+static void validate(void)
+{
+	if (env.consumer_cnt != 0) {
+		fprintf(stderr, "benchmark doesn't support consumer!\n");

tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c line 78

Quoted 10 lines
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +10 -0
+		exit(1);
+	}
+
+	/*
+	 * The number of worker threads is controlled by --nr-threads and
+	 * drives the framework's producer machinery, so per-producer stats
+	 * are reported correctly.
+	 */
+	env.producer_cnt = args.nr_threads;
+}

[Severity: Medium]
Does mutating env.producer_cnt inside validate() cause a heap buffer overflow?

The state.producers array is allocated in setup_benchmark() based on the initial env.producer_cnt:

tools/testing/selftests/bpf/bench.c:setup_benchmark() {
    ...
    state.producers = calloc(env.producer_cnt, sizeof(*state.producers));
    ...
}

When this validate callback overwrites env.producer_cnt with args.nr_threads (which defaults to 16), the core framework later iterates over the new count when spawning threads:

tools/testing/selftests/bpf/bench.c:setup_benchmark() {
    ...
    for (i = 0; i < env.producer_cnt; i++) {
        ...
        err = pthread_create(&state.producers[i], NULL,
                             bench->producer_thread, (void *)(long)i);
    ...
}

This will write out of bounds since the array was originally sized for the unmodified env.producer_cnt.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

In reply to Song Liu
Quoted 155 lines · Song Liu

Add a benchmark that repeatedly mmap()s and munmap()s an mmap-able BPF array map (BPF_F_MMAPABLE) from a configurable number of threads. Both the number of worker threads and the size of the map are tunable via --nr-threads and --map-size, defaulting to 16 threads and 8MiB.

The benchmark is useful to observe how the cost of mmap()/munmap() scales with the mapping size, e.g. eager vs lazy page-table population.

Assisted-by Claude:claude-opus-4-8
---
 tools/testing/selftests/bpf/Makefile          |   1 +
 tools/testing/selftests/bpf/bench.c           |   4 +
 .../bpf/benchs/bench_arraymap_mmap.c          | 146 ++++++++++++++++++
 .../bpf/benchs/run_bench_arraymap_mmap.sh     |  15 ++
 4 files changed, 166 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c
 create mode 100755 tools/testing/selftests/bpf/benchs/run_bench_arraymap_mmap.sh
tools/testing/selftests/bpf/Makefile +1 -0
index 55d394438705..b6593fb1135b 100644
@@ -1003,6 +1003,7 @@ $(OUTPUT)/bench: $(OUTPUT)/bench.o \
 		 $(OUTPUT)/bench_bpf_timing.o \
 		 $(OUTPUT)/bench_bpf_nop.o \
 		 $(OUTPUT)/bench_xdp_lb.o \
+		 $(OUTPUT)/bench_arraymap_mmap.o \
 		 $(OUTPUT)/usdt_1.o \
 		 $(OUTPUT)/usdt_2.o \
 		 #
tools/testing/selftests/bpf/bench.c +4 -0
index 3d9d2cd7764b..192d5fcb7a74 100644
@@ -287,6 +287,7 @@ extern struct argp bench_crypto_argp;
 extern struct argp bench_sockmap_argp;
 extern struct argp bench_lpm_trie_map_argp;
 extern struct argp bench_xdp_lb_argp;
+extern struct argp bench_arraymap_mmap_argp;
 
 static const struct argp_child bench_parsers[] = {
 	{ &bench_ringbufs_argp, 0, "Ring buffers benchmark", 0 },
@@ -304,6 +305,7 @@ static const struct argp_child bench_parsers[] = {
 	{ &bench_sockmap_argp, 0, "bpf sockmap benchmark", 0 },
 	{ &bench_lpm_trie_map_argp, 0, "LPM trie map benchmark", 0 },
 	{ &bench_xdp_lb_argp, 0, "XDP load-balancer benchmark", 0 },
+	{ &bench_arraymap_mmap_argp, 0, "arraymap-mmap benchmark", 0 },
 	{},
 };
 
@@ -582,6 +584,7 @@ extern const struct bench bench_lpm_trie_delete;
 extern const struct bench bench_lpm_trie_free;
 extern const struct bench bench_bpf_nop;
 extern const struct bench bench_xdp_lb;
+extern const struct bench bench_arraymap_mmap;
 
 static const struct bench *benchs[] = {
 	&bench_count_global,
@@ -665,6 +668,7 @@ static const struct bench *benchs[] = {
 	&bench_lpm_trie_free,
 	&bench_bpf_nop,
 	&bench_xdp_lb,
+	&bench_arraymap_mmap,
 };
 
 static void find_benchmark(void)
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +82 -0
new file mode 100644
index 000000000000..356d80a85cf0
@@ -0,0 +1,146 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. */
+#include <argp.h>
+#include <sys/mman.h>
+#include "bench.h"
+
+/* Benchmark mmap()/munmap() of an mmap-able BPF array map from N threads. */
+
+static struct ctx {
+	int map_fd;
+	size_t mmap_sz;
+} ctx;
+
+static struct {
+	__u32 nr_threads;
+	__u64 map_size;
+} args = {
+	.nr_threads = 16,
+	.map_size = 8 * 1024 * 1024, /* 8 MiB */
+};
+
+enum {
+	ARG_NR_THREADS = 5000,
+	ARG_MAP_SIZE = 5001,
+};
+
+static const struct argp_option opts[] = {
+	{ "nr-threads", ARG_NR_THREADS, "NR_THREADS", 0,
+		"Number of threads that mmap/munmap the map (default 16)" },
+	{ "map-size", ARG_MAP_SIZE, "BYTES", 0,
+		"Size of the BPF array map in bytes (default 8388608)" },
+	{},
+};
+
+static error_t parse_arg(int key, char *arg, struct argp_state *state)
+{
+	switch (key) {
+	case ARG_NR_THREADS:
+		args.nr_threads = strtoul(arg, NULL, 0);
+		if (args.nr_threads == 0) {
+			fprintf(stderr, "Invalid nr-threads: %s\n", arg);
+			argp_usage(state);
+		}
+		break;
+	case ARG_MAP_SIZE:
+		args.map_size = strtoull(arg, NULL, 0);
+		if (args.map_size == 0) {
+			fprintf(stderr, "Invalid map-size: %s\n", arg);
+			argp_usage(state);
+		}
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+/* exported into benchmark runner */
+const struct argp bench_arraymap_mmap_argp = {
+	.options = opts,
+	.parser = parse_arg,
+};
+
+static void validate(void)
+{
+	if (env.consumer_cnt != 0) {
+		fprintf(stderr, "benchmark doesn't support consumer!\n");
+		exit(1);
+	}
+
+	/*
+	 * The number of worker threads is controlled by --nr-threads and
+	 * drives the framework's producer machinery, so per-producer stats
+	 * are reported correctly.
+	 */
+	env.producer_cnt = args.nr_threads;
+}
+
+static long hits;
+
+static void *producer(void *input)

tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c line 92

Quoted 10 lines
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +10 -0
+{
+	while (true) {
+		void *addr;
+
+		addr = mmap(NULL, ctx.mmap_sz, PROT_READ | PROT_WRITE,
+			    MAP_SHARED, ctx.map_fd, 0);
+		if (addr == MAP_FAILED) {
+			fprintf(stderr, "mmap failed: %d\n", -errno);
+			exit(1);
+		}

hi,
so now pages are populated only when accessed, benchmark shows this nicely:

before:

		nr_threads: 1, map_size: 1048576
	arraymap-mmap:       throughput: 0.006 ± 0.000 M ops/s, latency: 172703.437 ns/op

after:

		nr_threads: 1, map_size: 1048576
	arraymap-mmap:       throughput: 0.082 ± 0.002 M ops/s, latency: 12212.778 ns/op

would it make sense to add some map access in here (each page I guess) enabled by new option and measure the impact for accessed map ? I assume latency should be equal in such case

jirka

Quoted 79 lines
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +54 -0
+		if (munmap(addr, ctx.mmap_sz)) {
+			fprintf(stderr, "munmap failed: %d\n", -errno);
+			exit(1);
+		}
+		atomic_inc(&hits);
+	}
+
+	return NULL;
+}
+
+static void measure(struct bench_res *res)
+{
+	res->hits = atomic_swap(&hits, 0);
+}
+
+static void setup(void)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_MMAPABLE);
+	long page_sz = sysconf(_SC_PAGESIZE);
+	__u32 value_size = sizeof(__u64);
+	__u32 max_entries;
+
+	setup_libbpf();
+
+	/*
+	 * mmap-able array maps round the value size up to 8 bytes and expose
+	 * PAGE_ALIGN(max_entries * elem_size) bytes to user space.
+	 */
+	max_entries = (args.map_size + value_size - 1) / value_size;
+	ctx.mmap_sz = ((__u64)max_entries * value_size + page_sz - 1) &
+		      ~(page_sz - 1);
+
+	ctx.map_fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, "mmap_array",
+				    sizeof(__u32), value_size, max_entries,
+				    &opts);
+	if (ctx.map_fd < 0) {
+		fprintf(stderr, "failed to create map: %d\n", -errno);
+		exit(1);
+	}
+
+	printf("array map: %u entries, mmap size %zu bytes, %u threads\n",
+	       max_entries, ctx.mmap_sz, args.nr_threads);
+}
+
+const struct bench bench_arraymap_mmap = {
+	.name = "arraymap-mmap",
+	.argp = &bench_arraymap_mmap_argp,
+	.validate = validate,
+	.setup = setup,
+	.producer_thread = producer,
+	.measure = measure,
+	.report_progress = ops_report_progress,
+	.report_final = ops_report_final,
+};
tools/testing/selftests/bpf/benchs/run_bench_arraymap_mmap.sh new file +15 -0
new file mode 100755
index 000000000000..3efd441cd653
@@ -0,0 +1,15 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+source ./benchs/run_common.sh
+
+set -eufo pipefail
+
+for t in 1 4 8 16; do
+for s in 1048576 8388608 67108864; do
+subtitle "nr_threads: $t, map_size: $s"
+	summarize_ops "arraymap-mmap: " \
+	    "$($RUN_BENCH --nr-threads $t --map-size $s arraymap-mmap)"
+	printf "\n"
+done
+done
Signature · 1 line
2.53.0-Meta
author reviewer
#6 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

On Thu, Jul 23, 2026 at 11:46 AM Jiri Olsa <[email protected]> wrote:
[...]

Quoted 20 lines

tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c line 92

Quoted 11 lines
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +11 -0
+static void *producer(void *input)
+{
+     while (true) {
+             void *addr;
+
+             addr = mmap(NULL, ctx.mmap_sz, PROT_READ | PROT_WRITE,
+                         MAP_SHARED, ctx.map_fd, 0);
+             if (addr == MAP_FAILED) {
+                     fprintf(stderr, "mmap failed: %d\n", -errno);
+                     exit(1);
+             }

hi,
so now pages are populated only when accessed, benchmark shows this nicely:

before:

                nr_threads: 1, map_size: 1048576
        arraymap-mmap:       throughput: 0.006 ą 0.000 M ops/s, latency: 172703.437 ns/op
Quoted 8 lines

after:

                nr_threads: 1, map_size: 1048576
        arraymap-mmap:       throughput: 0.082 ą 0.002 M ops/s, latency: 12212.778 ns/op

would it make sense to add some map access in here (each page I guess) enabled by new option and measure the impact for accessed map ? I assume latency should be equal in such case.

If we add a read per page, which is the worst case:

before:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.032 ± 0.000 M ops/s, latency: 31248.846 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.003 ± 0.000 M ops/s, latency:
372565.622 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
2061083.005 ns/op

after:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.006 ± 0.000 M ops/s, latency:
170794.193 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.001 ± 0.000 M ops/s, latency:
1346389.229 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
10700389.105 ns/op

So filling all the pages in the mmap is faster than faulting one page at a time in many page faults. IOW, there is a performance regression in this case.

Thanks,
Song

#7 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

Quoted 54 lines · Song Liu

On Thu, Jul 23, 2026 at 11:46 AM Jiri Olsa <[email protected]> wrote:
[...]

Quoted 20 lines

tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c line 92

Quoted 11 lines
tools/testing/selftests/bpf/benchs/bench_arraymap_mmap.c new file +11 -0
+static void *producer(void *input)
+{
+     while (true) {
+             void *addr;
+
+             addr = mmap(NULL, ctx.mmap_sz, PROT_READ | PROT_WRITE,
+                         MAP_SHARED, ctx.map_fd, 0);
+             if (addr == MAP_FAILED) {
+                     fprintf(stderr, "mmap failed: %d\n", -errno);
+                     exit(1);
+             }

hi,
so now pages are populated only when accessed, benchmark shows this nicely:

before:

                nr_threads: 1, map_size: 1048576
        arraymap-mmap:       throughput: 0.006 ą 0.000 M ops/s, latency: 172703.437 ns/op
Quoted 8 lines

after:

                nr_threads: 1, map_size: 1048576
        arraymap-mmap:       throughput: 0.082 ą 0.002 M ops/s, latency: 12212.778 ns/op

would it make sense to add some map access in here (each page I guess) enabled by new option and measure the impact for accessed map ? I assume latency should be equal in such case.

If we add a read per page, which is the worst case:

before:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.032 ± 0.000 M ops/s, latency: 31248.846 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.003 ± 0.000 M ops/s, latency:
372565.622 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
2061083.005 ns/op

after:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.006 ± 0.000 M ops/s, latency:
170794.193 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.001 ± 0.000 M ops/s, latency:
1346389.229 ns/op

Quoted 6 lines
        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
10700389.105 ns/op

So filling all the pages in the mmap is faster than faulting one page at a time in many page faults. IOW, there is a performance regression in this case.

would it make sense to add new map flag for that?

jirka

author reviewer
#8 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

On Fri, Jul 24, 2026 at 1:54 AM Jiri Olsa <[email protected]> wrote:
[...]

Quoted 23 lines
Quoted 23 lines

If we add a read per page, which is the worst case:

before:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.032 ± 0.000 M ops/s, latency: 31248.846 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.003 ± 0.000 M ops/s, latency:
372565.622 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
2061083.005 ns/op

after:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.006 ± 0.000 M ops/s, latency:
170794.193 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.001 ± 0.000 M ops/s, latency:
1346389.229 ns/op

Quoted 8 lines
Quoted 6 lines
        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
10700389.105 ns/op

So filling all the pages in the mmap is faster than faulting one page at a time in many page faults. IOW, there is a performance regression in this case.

would it make sense to add new map flag for that?

If possible, I would avoid adding a new flag for this. I think the regression is on a very special corner case. BPF arena has similar behavior (fill in page table on fault). So it should be OK to have array map do the same. WDYT?

Thanks,
Song

#9 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

Quoted 32 lines · Song Liu

On Fri, Jul 24, 2026 at 1:54 AM Jiri Olsa <[email protected]> wrote:
[...]

Quoted 30 lines
Quoted 29 lines

If we add a read per page, which is the worst case:

before:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.032 ± 0.000 M ops/s, latency: 31248.846 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.003 ± 0.000 M ops/s, latency:
372565.622 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
2061083.005 ns/op

after:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.006 ± 0.000 M ops/s, latency:
170794.193 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.001 ± 0.000 M ops/s, latency:
1346389.229 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
10700389.105 ns/op

So filling all the pages in the mmap is faster than faulting one page at a time in many page faults. IOW, there is a performance regression in this case.

Quoted 7 lines
Quoted 1 line

would it make sense to add new map flag for that?

If possible, I would avoid adding a new flag for this. I think the regression is on a very special corner case. BPF arena has similar behavior (fill in page table on fault). So it should be OK to have array map do the same. WDYT?

I think it would make sense to compare cumulative cost of mmap+read-every-page. I am sure it would also be lower before this change, due to the added context switches for fault on every page access that would be amortized before this, but may give a better picture about the difference.

I do feel like it's a corner case realistically, and after the first fault the page remains mapped (and users who care about pages being present can go the route of passing MAP_POPULATE). I am assuming passing that before was a noop, so it can be added to code in a backwards compatible fashion if someone really depends on this?

Quoted 2 lines

Thanks,
Song

author reviewer
#10 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

Quoted 47 lines · Kumar Kartikeya Dwivedi
Quoted 32 lines · Song Liu

On Fri, Jul 24, 2026 at 1:54 AM Jiri Olsa <[email protected]> wrote:
[...]

Quoted 30 lines
Quoted 29 lines

If we add a read per page, which is the worst case:

before:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.032 ± 0.000 M ops/s, latency: 31248.846 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.003 ± 0.000 M ops/s, latency:
372565.622 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
2061083.005 ns/op

after:

        nr_threads: 1, map_size: 1048576

arraymap-mmap: throughput: 0.006 ± 0.000 M ops/s, latency:
170794.193 ns/op

        nr_threads: 1, map_size: 8388608

arraymap-mmap: throughput: 0.001 ± 0.000 M ops/s, latency:
1346389.229 ns/op

        nr_threads: 1, map_size: 67108864

arraymap-mmap: throughput: 0.000 ± 0.000 M ops/s, latency:
10700389.105 ns/op

So filling all the pages in the mmap is faster than faulting one page at a time in many page faults. IOW, there is a performance regression in this case.

Quoted 7 lines
Quoted 1 line

would it make sense to add new map flag for that?

If possible, I would avoid adding a new flag for this. I think the regression is on a very special corner case. BPF arena has similar behavior (fill in page table on fault). So it should be OK to have array map do the same. WDYT?

I think it would make sense to compare cumulative cost of mmap+read-every-page. I am sure it would also be lower before this change, due to the added context switches for fault on every page access that would be amortized before this, but may give a better picture about the difference.

Quoted 5 lines

I do feel like it's a corner case realistically, and after the first fault the page remains mapped (and users who care about pages being present can go the route of passing MAP_POPULATE). I am assuming passing that before was a noop, so it can be added to code in a backwards compatible fashion if someone really depends on this?

Given there is MAP_POPULATE flag, yes, the user can preserve the old behavior with this flag. While this requires users to make changes for this corner case, I think it is still cleaner than adding a BPF map flag for this. If there are no strong objections, I will keep this behavior in v3.

Thanks,
Song

author reviewer
#11 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

On Mon, Jul 27, 2026 at 5:19 PM Song Liu <[email protected]> wrote:
[...]

Quoted 11 lines
Quoted 10 lines

I think it would make sense to compare cumulative cost of mmap+read-every-page. I am sure it would also be lower before this change, due to the added context switches for fault on every page access that would be amortized before this, but may give a better picture about the difference.

I do feel like it's a corner case realistically, and after the first fault the page remains mapped (and users who care about pages being present can go the route of passing MAP_POPULATE). I am assuming passing that before was a noop, so it can be added to code in a backwards compatible fashion if someone really depends on this?

Quoted 4 lines

Given there is MAP_POPULATE flag, yes, the user can preserve the old behavior with this flag. While this requires users to make changes for this corner case, I think it is still cleaner than adding a BPF map flag for this. If there are no strong objections, I will keep this behavior in v3.

This turns out to be a little more complicated:

1. Bad news: MAP_POPULATE is not a noop for old kernels. The pre fault mechanism still adds some overhead (for almost no gain). For 8MiB mmap, MAP_POPULATE addes about 80us (raw data below).

2. Good news: We can further improve MAP_POPULATE with a batch pre fault (vm_operations_struct->map_pages). With batch pre fault, mmap then accessing 8MiB is about 34us slower than the best option before the set:
(before: no MAP_POPULATE => access all pages: 252us;
after: MAP_POPULATE => access all pages: 286us).

Note that, all the above are worst cases: we need to access all pages and fill the page table one way or another. If we don't need to access all pages, the change saves a lot (220us => 1.2us) for the mmap.

Overall, I think this is a net win for most use cases.

Thanks,
Song

Raw data:

All of the following are for 8MiB, access times are in microseconds (us).

Before:
no MAP_POPULATE (do not access pages) 220us
no MAP_POPULATE => access all pages: 252us
MAP_POPULATE (do not access pages) 304us
MAP_POPULATE => access all pages: 334us

After:
no MAP_POPULATE (do not access pages): 1.2us
no MAP_POPULATE => access all pages: 543us
MAP_POPULATE (do not access pages): 262us
MAP_POPULATE => access all pages: 286us

#12 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

Quoted 24 lines · Song Liu

On Mon, Jul 27, 2026 at 5:19 PM Song Liu <[email protected]> wrote:
[...]

Quoted 11 lines
Quoted 10 lines

I think it would make sense to compare cumulative cost of mmap+read-every-page. I am sure it would also be lower before this change, due to the added context switches for fault on every page access that would be amortized before this, but may give a better picture about the difference.

I do feel like it's a corner case realistically, and after the first fault the page remains mapped (and users who care about pages being present can go the route of passing MAP_POPULATE). I am assuming passing that before was a noop, so it can be added to code in a backwards compatible fashion if someone really depends on this?

Quoted 4 lines

Given there is MAP_POPULATE flag, yes, the user can preserve the old behavior with this flag. While this requires users to make changes for this corner case, I think it is still cleaner than adding a BPF map flag for this. If there are no strong objections, I will keep this behavior in v3.

This turns out to be a little more complicated:

1. Bad news: MAP_POPULATE is not a noop for old kernels. The pre fault mechanism still adds some overhead (for almost no gain). For 8MiB mmap, MAP_POPULATE addes about 80us (raw data below).

Quoted 6 lines

2. Good news: We can further improve MAP_POPULATE with a batch pre fault (vm_operations_struct->map_pages). With batch pre fault, mmap then accessing 8MiB is about 34us slower than the best option before the set:
(before: no MAP_POPULATE => access all pages: 252us;
after: MAP_POPULATE => access all pages: 286us).

Interesting, though I figure we can let it be, since repeated mmap()+MAP_POPULATE on very big array maps should not be a common occurence. We can always revisit in the future though.

Quoted 7 lines

Note that, all the above are worst cases: we need to access all pages and fill the page table one way or another. If we don't need to access all pages, the change saves a lot (220us => 1.2us) for the mmap.

Overall, I think this is a net win for most use cases.

Thanks for the analysis and data, makes a lot of sense, and agreed about going forward with the set.

Quoted 18 lines

Thanks,
Song

Raw data:

All of the following are for 8MiB, access times are in microseconds (us).

Before:
no MAP_POPULATE (do not access pages) 220us
no MAP_POPULATE => access all pages: 252us
MAP_POPULATE (do not access pages) 304us
MAP_POPULATE => access all pages: 334us

After:
no MAP_POPULATE (do not access pages): 1.2us
no MAP_POPULATE => access all pages: 543us
MAP_POPULATE (do not access pages): 262us
MAP_POPULATE => access all pages: 286us

#13 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: Add mmap/munmap benchmark for array maps

In reply to Song Liu
Quoted 31 lines · Song Liu

On Mon, Jul 27, 2026 at 5:19 PM Song Liu <[email protected]> wrote:
[...]

Quoted 11 lines
Quoted 10 lines

I think it would make sense to compare cumulative cost of mmap+read-every-page. I am sure it would also be lower before this change, due to the added context switches for fault on every page access that would be amortized before this, but may give a better picture about the difference.

I do feel like it's a corner case realistically, and after the first fault the page remains mapped (and users who care about pages being present can go the route of passing MAP_POPULATE). I am assuming passing that before was a noop, so it can be added to code in a backwards compatible fashion if someone really depends on this?

Quoted 4 lines

Given there is MAP_POPULATE flag, yes, the user can preserve the old behavior with this flag. While this requires users to make changes for this corner case, I think it is still cleaner than adding a BPF map flag for this. If there are no strong objections, I will keep this behavior in v3.

This turns out to be a little more complicated:

1. Bad news: MAP_POPULATE is not a noop for old kernels. The pre fault mechanism still adds some overhead (for almost no gain). For 8MiB mmap, MAP_POPULATE addes about 80us (raw data below).

2. Good news: We can further improve MAP_POPULATE with a batch pre fault (vm_operations_struct->map_pages). With batch pre fault, mmap then accessing 8MiB is about 34us slower than the best option before the set:
(before: no MAP_POPULATE => access all pages: 252us;
after: MAP_POPULATE => access all pages: 286us).

Quoted 5 lines

Note that, all the above are worst cases: we need to access all pages and fill the page table one way or another. If we don't need to access all pages, the change saves a lot (220us => 1.2us) for the mmap.

Overall, I think this is a net win for most use cases.

sounds good, thanks for all the details

jirka

Quoted 19 lines

Thanks,
Song

Raw data:

All of the following are for 8MiB, access times are in microseconds (us).

Before:
no MAP_POPULATE (do not access pages) 220us
no MAP_POPULATE => access all pages: 252us
MAP_POPULATE (do not access pages) 304us
MAP_POPULATE => access all pages: 334us

After:
no MAP_POPULATE (do not access pages): 1.2us
no MAP_POPULATE => access all pages: 543us
MAP_POPULATE (do not access pages): 262us
MAP_POPULATE => access all pages: 286us