bpf: Introduce global percpu data
Subsystems (6)
v12 cover bpf: Introduce global percpu data
This patch set introduces global percpu data, similar to commit 6316f78306c1 ("Merge branch 'support-global-data'"), to reduce restrictions in C for BPF programs.
With this enhancement, it becomes possible to define and use global percpu variables, like the DEFINE_PER_CPU() macro in the kernel include/linux/percpu-defs.h.
The section name for global peurcpu data is ".percpu". Even though, a one-byte percpu variable (e.g., char run SEC(".percpu") = 0;) can trigger a crash with Clang 17 [1], users are expected to use such small variables as global percpu data with newer Clang versions, which don't have the issue.
The idea stems from the bpfsnoop [2], which itself was inspired by retsnoop [3]. During testing of bpfsnoop on the v6.6 kernel, two LBR (Last Branch Record) entries were observed related to the bpf_get_smp_processor_id() helper.
Since commit 1ae6921009e5 ("bpf: inline bpf_get_smp_processor_id() helper"), the bpf_get_smp_processor_id() helper has been inlined on x86_64, reducing the overhead and consequently minimizing these two LBR records.
However, the introduction of global percpu data offers a more robust solution. By leveraging the percpu_array map and percpu instruction, global percpu data can be implemented intrinsically.
This feature also facilitates sharing percpu information between tail callers and callees or between freplace callers and callees through a shared global percpu variable. Previously, this was achieved using a 1-entry percpu_array map, which this patch set aims to improve upon.
Links:
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://github.com/bpfsnoop/bpfsnoop
[3] https://github.com/anakryiko/retsnoop
Changes:
v11 -> v12:
* Improve feature check in bpf_object__create_maps() in libbpf.
* Add percpu_array map support in bpf_map__set_value_size() in libbpf.
* Exercise bpf_map__set_value_size() in selftest.
* Drop dead warning in bpf_object__populate_internal_map() in libbpf. (Sashiko)
* v11: https://lore.kernel.org/bpf/[email protected]/
v10 -> v11:
* Drop env->prog->jit_requested check when inlining insns for global percpu data.
* Do not autocreate percpu_array map when kernel does not have global percpu data support in libbpf.
* Check map->btf_value_type_id in bpftool's is_skel_data().
* Exercise bpf_map__lookup_elem() in selftest.
* Collect Reviewed-by tags from Emil, thanks.
* Drop all duplicate blank lines in kernel/bpf/*.c. (Emil)
* Factor out check_map_mem_read() helper. (Emil)
* Check bpf_jit_supports_percpu_insn() first in percpu_array_map_direct_value_addr/meta(). (Emil)
* Add comment for 'map->libbpf_type == LIBBPF_MAP_PERCPU' in libbpf's map_is_mmapable(). (Emil)
* Init update_flags as a const var in libbpf's bpf_object__populate_internal_map(). (Emil)
* Keep is_mmapable_map() beyond is_skel_data() in bpftool. (Emil)
* Add 'run' and 'cpu_id' in selftest. (Emil)
* Drop subskel test. Verify the generated subskel manually. (Emil)
* Add comment to the raw insns in selftest. (Emil)
* v10: https://lore.kernel.org/bpf/[email protected]/
v9 -> v10:
* Rebase latest bpf-next tree to resolve code conflict in verifier in patch #1.
* v9: https://lore.kernel.org/bpf/[email protected]/
v8 -> v9:
* Use real name for percpu data maps in libbpf in patch #4.
* Add long map name test in patch #6.
* Move parse_cpu_mask_file() to test_percpu_data_on_cpus() in test in patch #6.
* Validate map type in get_map_ident() for percpu data maps in patch #5.
* Update code comment in verifier in patch #2. (per Andrii)
* Pass 'type' to internal_map_name in libbpf in patch #4. (per Andrii)
* Factor out the helper is_skel_data() in bpftool in patch #5. (per Quentin and Andrii)
* v8: https://lore.kernel.org/bpf/[email protected]/
v7 -> v8:
* Send patch #1 and #2 separately that fix interpreter fallback issues. (Andrii)
* Use 'array->elem_size' to avoid 'range' local variable in percpu_array_map_direct_value_meta(). (Andrii)
* Keep original map name for percpu data's map in libbpf. (Andrii)
* Factor out helper bpf_map_is_skel_data() in bpftool. (Andrii)
* Update commit message of direct access read-only percpu_array map. (Andrii)
* Add test to verify that it is disallowed to directly write data of read-only percpu_array map. (Andrii)
* Drop unused 'num_cpus' in test. (bot+bpf-ci)
* Factor out helper test_percpu_data_on_cpus() in test. (bot+bpf-ci)
* v7: https://lore.kernel.org/bpf/[email protected]/
v6 -> v7:
* Use tgt_endian() in bpf_gen__map_update_elem() in patch #6. (Sashiko)
* Use sizeof(args) in verifier_snprintf test in patch #10. (Sashiko)
* Drop xlated test of v6. (Alexei)
* v6: https://lore.kernel.org/bpf/[email protected]/
v5 -> v6:
* Prevent running user addr_space_cast and addr_percpu insns in interpreter. (Sashiko)
* Cast __percpu pointer to u64 with (__force unsigned long). (lkp)
* Exclude BPF_MAP_TYPE_PERCPU_ARRAY in check_mem_access() before calling bpf_map_direct_read(), and add a test to verify it.
(Sashiko, bot+bpf-ci)
* Skip percpu data variables for subskeleton in bpftool. (Sashiko)
* Protect skel->percpu using mprotect(..., PROT_READ) in light skeleton. (Sashiko, bot+bpf-ci)
* Drop roundup() in tests. (Sashiko)
* Call test_global_percpu_data_verifier_log() without test__start_subtest(). (Sashiko)
* Cast insn->imm to __u64 with (__u32) in xlated test. (Sashiko)
* Check cnt using the new idx in xlated test. (Sashiko)
* v5: https://lore.kernel.org/bpf/[email protected]/
v4 -> v5:
* Add prog->jit_requested check to prevent running percpu data in interpreter in patch #1.
* Factor out verifier log tests using its own patch.
* Address comments from Alexei:
* Move map_type check from check_mem_access() to bpf_map_direct_read() in patch #2.
* Move BPF_MAP_TYPE_INSN_ARRAY map_type check from const_reg_xfer() to bpf_map_direct_read() in patch #2.
* Add a test to verify that the off of xlated ldimm64 insn matches the off encoded in the ELF ldimm64 insn.
* Drop patch #5 of v4.
* Address reviews from Sashiko:
* Update commit message of patch #6 to indicate that maps.percpu->mmaped has been marked as read-only in libbpf.
* Lookup elem on specified CPU using BPF_F_CPU in tests.
* Drop unnecessary err == -EOPNOTSUPP in test.
* Locate target field using its offset in the iter test.
* v4: https://lore.kernel.org/bpf/[email protected]/
v3 -> v4:
* Drop duplicate blank lines in verifier.
* Add percpu data feature probe in libbpf.
* Update percpu_array map using BPF_F_ALL_CPUS flag for lskel, if no cpu flag is set.
* Add two tests to verify verifier log.
* Add a test to verify mov64_percpu_reg instruction.
* Add a test to verify bpf_iter for percpu data map.
* Update percpu_array map using BPF_F_ALL_CPUS flag in libbpf (per Alexei and Andrii).
* Address comments from Andrii:
* Use .percpu as section identifier.
* Use bpf_jit_supports_percpu_insn() instead of CONFIG_SMP.
* Drop bpf_map__is_internal_percpu() API.
* Drop unnecessary __aligned(8) in libbpf, verified by selftest.
* Make mmap data read-only after loading prog.
v3: https://lore.kernel.org/bpf/[email protected]/
v2 -> v3:
* Use ".data..percpu" as PERCPU_DATA_SEC.
* Address comment from Alexei:
* Add u8, array of ints and struct { .. } vars to selftest.
v2: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2:
* Address comments from Andrii:
* Use LIBBPF_MAP_PERCPU and SEC_PERCPU.
* Reuse mmaped of libbpf's struct bpf_map for .percpu map data.
* Set .percpu struct pointer to NULL after loading skeleton.
* Make sure value size of .percpu map is __aligned(8).
* Use raw_tp and opts.cpu to test global percpu variables on all CPUs.
* Address comments from Alexei:
* Test non-zero offset of global percpu variable.
* Test case about BPF_PSEUDO_MAP_IDX_VALUE.
v1: https://lore.kernel.org/bpf/[email protected]/
rfc -> v1:
* Address comments from Andrii:
* Keep one image of global percpu variable for all CPUs.
* Reject non-ARRAY map in bpf_map_direct_read(), check_reg_const_str(), and check_bpf_snprintf_call() in verifier.
* Split out libbpf changes from kernel-side changes.
* Use ".percpu" as PERCPU_DATA_SEC.
* Use enum libbpf_map_type to distinguish BSS, DATA, RODATA and PERCPU_DATA.
* Avoid using errno for checking err from libbpf_num_possible_cpus().
* Use "map '%s': " prefix for error message.
rfc: https://lore.kernel.org/bpf/[email protected]/
Leon Hwang (10):
bpf: Drop duplicate blank lines in kernel/bpf/
bpf: Factor out check_map_mem_read helper in verifier
bpf: Introduce global percpu data
libbpf: Probe percpu data feature
libbpf: Add support for global percpu data
bpftool: Generate skeleton for global percpu data selftests/bpf: Add tests to verify global percpu data selftests/bpf: Test direct reading/writing read-only percpu_array map selftests/bpf: Test verifier log for global percpu data selftests/bpf: Verify bpf_iter for global percpu data
kernel/bpf/arraymap.c | 38 +- kernel/bpf/backtrack.c | 2 - kernel/bpf/btf.c | 1 - kernel/bpf/cfg.c | 1 - kernel/bpf/const_fold.c | 1 - kernel/bpf/fixups.c | 38 +- kernel/bpf/hashtab.c | 2 - kernel/bpf/helpers.c | 1 - kernel/bpf/liveness.c | 2 - kernel/bpf/queue_stack_maps.c | 1 - kernel/bpf/syscall.c | 5 - kernel/bpf/verifier.c | 100 +++--- tools/bpf/bpftool/gen.c | 53 ++- tools/lib/bpf/bpf_gen_internal.h | 3 +- tools/lib/bpf/features.c | 35 ++ tools/lib/bpf/gen_loader.c | 3 +- tools/lib/bpf/libbpf.c | 81 ++++- tools/lib/bpf/libbpf_internal.h | 2 + tools/lib/bpf/skel_internal.h | 24 +- tools/testing/selftests/bpf/Makefile | 2 +- .../bpf/prog_tests/global_data_init.c | 336 ++++++++++++++++++ .../bpf/progs/test_global_percpu_data.c | 89 +++++ 22 files changed, 725 insertions(+), 95 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/test_global_percpu_data.c
--
2.55.0
PATCH v12 1/10 bpf: Drop duplicate blank lines in kernel/bpf/
There are many adjacent blank lines in kernel/bpf/ that have accumulated over time.
Drop them for cleanup.
No functional changes intended.
--- kernel/bpf/backtrack.c | 2 -- kernel/bpf/btf.c | 1 - kernel/bpf/cfg.c | 1 - kernel/bpf/fixups.c | 1 - kernel/bpf/hashtab.c | 2 -- kernel/bpf/helpers.c | 1 - kernel/bpf/liveness.c | 2 -- kernel/bpf/queue_stack_maps.c | 1 - kernel/bpf/syscall.c | 5 ----- kernel/bpf/verifier.c | 14 -------------- 10 files changed, 30 deletions(-)
kernel/bpf/backtrack.c +0 -2
@@ -214,7 +214,6 @@ static inline bool bt_is_reg_set(struct backtrack_state *bt, u32 reg)return bt->reg_masks[bt->frame] & (1 << reg); } -/* format registers bitmask, e.g., "r0,r2,r4" for 0x15 mask */ static void fmt_reg_mask (char *buf,ssize_t buf_sz,u32 reg_mask) {
@@ -254,7 +253,6 @@ void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask) } } -/* For given verifier state backtrack_insn() is called from the last insn to * the first insn. Its purpose is to compute a bitmask of registers and * stack slots that needs precision in the parent verifier state.
kernel/bpf/btf.c +0 -1
@@ -2534,7 +2534,6 @@ static void btf_bitfield_show(void *data, u8 bits_offset,btf_int128_print (show, print_num); } - static void btf_int_bits_show(const struct btf *btf, const struct btf_type *t, void *data, u8 bits_offset,
kernel/bpf/cfg.c +0 -1
@@ -47,7 +47,6 @@ enum { BRANCH = 2, }; -static void mark_subprog_changes_pkt_data (struct bpf_verifier_env *env,int off) {struct bpf_subprog_info *subprog;
kernel/bpf/fixups.c +0 -1
@@ -1466,7 +1466,6 @@ int bpf_fixup_call_args(struct bpf_verifier_env *env)return err; } -/* The function requires that first instruction in 'patch' is insnsi[prog->len - 1] */ static int add_hidden_subprog (struct bpf_verifier_env *env,struct bpf_insn *patch,int len) {
kernel/bpf/hashtab.c +0 -2
@@ -998,7 +998,6 @@ static void dec_elem_count(struct bpf_htab *htab)atomic_dec (&htab->count); } -static void free_htab_elem (struct bpf_htab *htab,struct htab_elem *l) {htab_put_fd_value (htab, l);
@@ -2970,7 +2969,6 @@ static int rhtab_delete_elem(struct bpf_rhtab *rhtab, struct rhtab_elem *elem, vreturn 0 ; } -static long rhtab_map_delete_elem (struct bpf_map *map,void *key) { struct bpf_rhtab *rhtab = container_of(map, struct bpf_rhtab, map);
kernel/bpf/helpers.c +0 -1
@@ -4871,7 +4871,6 @@ static const struct btf_kfunc_id_set generic_kfunc_set = { .set = &generic_btf_ids, }; - BTF_ID_LIST(generic_dtor_ids) BTF_ID(struct, task_struct) BTF_ID(func, bpf_task_release_dtor)
kernel/bpf/liveness.c +0 -2
@@ -269,7 +269,6 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)__diag_pop (); -static inline bool update_insn (struct bpf_verifier_env *env,struct func_instance *instance,u32 frame,u32 insn_idx) {
@@ -1862,7 +1861,6 @@ static int analyze_subprog(struct bpf_verifier_env *env,if (need_resched ())cond_resched (); -/* * When an instance is reused (must_write_initialized == true), * record into a fresh instance and merge afterward. This avoids
kernel/bpf/queue_stack_maps.c +0 -1
@@ -123,7 +123,6 @@ static long __queue_map_get(struct bpf_map *map, void *value, bool delete)return err; } -static long __stack_map_get (struct bpf_map *map,void *value,bool delete) {struct bpf_queue_stack *qs =bpf_queue_stack (map);
kernel/bpf/syscall.c +0 -5
@@ -636,7 +636,6 @@ int bpf_map_alloc_pages(const struct bpf_map *map, int nid,return ret; } -static int btf_field_cmp (const void *a,const void *b) {const struct btf_field *f1 = a, *f2 = b;
@@ -1830,7 +1829,6 @@ static int map_lookup_elem(union bpf_attr *attr)return err; } -#define BPF_MAP_UPDATE_ELEM_LAST_FIELD flagsstatic int map_update_elem (union bpf_attr *attr,bpfptr_t uattr)
@@ -3497,7 +3495,6 @@ int bpf_link_prime(struct bpf_link *link, struct bpf_link_primer *primer)if (fd <0 )return fd; - id =bpf_link_alloc_id (link);if (id <0 ) {put_unused_fd (fd);
@@ -5505,7 +5502,6 @@ static int bpf_link_get_info_by_fd(struct file *file,return 0 ; } - static int token_get_info_by_fd(struct file *file, struct bpf_token *token, const union bpf_attr *attr,
@@ -6507,7 +6503,6 @@ BPF_CALL_3(bpf_sys_bpf, int, cmd, union bpf_attr *, attr, u32, attr_size)return __sys_bpf (cmd,KERNEL_BPFPTR (attr), attr_size,KERNEL_BPFPTR (NULL ),0 ); } -/* To shut up -Wmissing-prototypes. * This function is used by the kernel light skeleton * to load bpf programs when modules are loaded or during kernel boot.
kernel/bpf/verifier.c +0 -14
@@ -635,7 +635,6 @@ static void __mark_dynptr_reg(struct bpf_reg_state *reg,enum bpf_dynptr_type type, bool first_slot, int id, int parent_id); - static void mark_dynptr_stack_regs(struct bpf_verifier_env *env, struct bpf_reg_state *sreg1, struct bpf_reg_state *sreg2,
@@ -1674,7 +1673,6 @@ static bool same_callsites(struct bpf_verifier_state *a, struct bpf_verifier_stareturn true; } -void bpf_free_backedges (struct bpf_scc_visit *visit) {struct bpf_scc_backedge *backedge, *next;
@@ -2291,7 +2289,6 @@ static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env,return &elem->st; } -static int cmp_subprogs (const void *a,const void *b) { return ((struct bpf_subprog_info *)a)->start -
@@ -3969,7 +3966,6 @@ static int check_stack_read(struct bpf_verifier_env *env,return err; } -/* check_stack_write dispatches to check_stack_write_fixed_off or * check_stack_write_var_off. *
@@ -4767,7 +4763,6 @@ static int check_sock_access(struct bpf_verifier_env *env, int insn_idx, valid = false; } -if (valid) { env->insn_aux_data[insn_idx].ctx_field_size = info.ctx_field_size;
@@ -6635,7 +6630,6 @@ static int check_stack_range_initialized(if (err)return err; -if (tnum_is_const (reg->var_off)) { min_off = max_off = reg->var_off.value + off; }else {
@@ -7347,7 +7341,6 @@ static bool is_iter_new_kfunc(struct bpf_call_arg_meta *meta)return meta->kfunc_flags &KF_ITER_NEW ; } -static bool is_iter_destroy_kfunc (struct bpf_call_arg_meta *meta) {return meta->kfunc_flags &KF_ITER_DESTROY ;
@@ -11607,7 +11600,6 @@ static int process_irq_flag(struct bpf_verifier_env *env, struct bpf_reg_state *return 0 ; } -static int ref_set_non_owning (struct bpf_verifier_env *env,struct bpf_reg_state *reg) {struct btf_record *rec =reg_btf_record (reg);
@@ -16412,7 +16404,6 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)return 0 ; } -static bool return_retval_range (struct bpf_verifier_env *env,struct bpf_retval_range *range) {enum bpf_prog_type prog_type =resolve_prog_type (env->prog);
@@ -18361,8 +18352,6 @@ static void release_insn_arrays(struct bpf_verifier_env *env)bpf_insn_array_release (env->insn_array_maps[i]); } - -/* The verifier does more data flow analysis than llvm and will not * explore branches that are dead at run time. Malicious programs can * have dead code too. Therefore replace all dead at-run-time code
@@ -18390,8 +18379,6 @@ static void sanitize_dead_code(struct bpf_verifier_env *env) } } - -static void free_states (struct bpf_verifier_env *env) {struct bpf_verifier_state_list *sl;
@@ -18678,7 +18665,6 @@ static int do_check_main(struct bpf_verifier_env *env)return ret; } -static void print_verification_stats (struct bpf_verifier_env *env) {/* Skip over hidden subprogs which are not verified. */
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v12 01/10] bpf: Drop duplicate blank lines in kernel/bpf/
This isn't a bug, but it looks like one adjacent blank-line pair is still left in verifier.c. Inside the kfunc argument loop in check_kfunc_args(), around the code that checks:
Quoted 12 lines
kernel/bpf/backtrack.c +0
-1@@ -214,7 +214,6 @@ static inline bool bt_is_reg_set(struct backtrack_state *bt, u32 reg)return bt->reg_masks[bt->frame] & (1 << reg); } -/* format registers bitmask, e.g., "r0,r2,r4" for 0x15 mask */ static void fmt_reg_mask (char *buf,ssize_t buf_sz,u32 reg_mask) {
[ ... ]
Quoted 12 lines
kernel/bpf/verifier.c +0
-1@@ -635,7 +635,6 @@ static void __mark_dynptr_reg(struct bpf_reg_state *reg,enum bpf_dynptr_type type, bool first_slot, int id, int parent_id); - static void mark_dynptr_stack_regs(struct bpf_verifier_env *env, struct bpf_reg_state *sreg1, struct bpf_reg_state *sreg2,
[ ... ]
Quoted 10 lines
kernel/bpf/verifier.c +0
-2@@ -18361,8 +18352,6 @@ static void release_insn_arrays(struct bpf_verifier_env *env)bpf_insn_array_release (env->insn_array_maps[i]); } - -/* The verifier does more data flow analysis than llvm and will not * explore branches that are dead at run time. Malicious programs can * have dead code too. Therefore replace all dead at-run-time code
kernel/bpf/verifier.c static void sanitize_dead_code(struct bpf_verifier_env *env) · line 18384
Quoted 8 lines
kernel/bpf/verifier.c +0
-2@@ -18390,8 +18379,6 @@ static void sanitize_dead_code(struct bpf_verifier_env *env) } } - -static void free_states (struct bpf_verifier_env *env) {struct bpf_verifier_state_list *sl;
This isn't a bug, but it looks like one adjacent blank-line pair is still left in verifier.c. Inside the kfunc argument loop in check_kfunc_args(), around the code that checks:
if (bpf_register_is_null(reg) && type_may_be_null(kf_arg_type))
there's a blank line pair that appears to fit the same pattern this patch removes everywhere else. The changelog states the goal is to drop the adjacent blank lines that have accumulated over time, and the patch removes 30 blank lines across the directory, but this one instance seems to have been overlooked. Was that one intentionally kept, or could it be folded into this patch so the whole directory is done in one pass?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 01/10] bpf: Drop duplicate blank lines in kernel/bpf/
Quoted 53 lines · On 14/8/26 00:26, [email protected] wrote:
Quoted 12 lines
kernel/bpf/backtrack.c +0
-1@@ -214,7 +214,6 @@ static inline bool bt_is_reg_set(struct backtrack_state *bt, u32 reg)return bt->reg_masks[bt->frame] & (1 << reg); } -/* format registers bitmask, e.g., "r0,r2,r4" for 0x15 mask */ static void fmt_reg_mask (char *buf,ssize_t buf_sz,u32 reg_mask) {[ ... ]
Quoted 12 lines
kernel/bpf/verifier.c +0
-1@@ -635,7 +635,6 @@ static void __mark_dynptr_reg(struct bpf_reg_state *reg,enum bpf_dynptr_type type, bool first_slot, int id, int parent_id); - static void mark_dynptr_stack_regs(struct bpf_verifier_env *env, struct bpf_reg_state *sreg1, struct bpf_reg_state *sreg2,[ ... ]
Quoted 10 lines
kernel/bpf/verifier.c +0
-2@@ -18361,8 +18352,6 @@ static void release_insn_arrays(struct bpf_verifier_env *env)bpf_insn_array_release (env->insn_array_maps[i]); } - -/* The verifier does more data flow analysis than llvm and will not * explore branches that are dead at run time. Malicious programs can * have dead code too. Therefore replace all dead at-run-time code kernel/bpf/verifier.c static void sanitize_dead_code(struct bpf_verifier_env *env) · line 18384
Quoted 8 lines
kernel/bpf/verifier.c +0
-2@@ -18390,8 +18379,6 @@ static void sanitize_dead_code(struct bpf_verifier_env *env) } } - -static void free_states (struct bpf_verifier_env *env) {struct bpf_verifier_state_list *sl;This isn't a bug, but it looks like one adjacent blank-line pair is still left in
verifier.c. Inside the kfunc argument loop incheck_kfunc_args(), around the code that checks:
Quoted 8 lines
if (bpf_register_is_null(reg) && type_may_be_null(kf_arg_type))there's a blank line pair that appears to fit the same pattern this patch removes everywhere else. The changelog states the goal is to drop the adjacent blank lines that have accumulated over time, and the patch removes 30 blank lines across the directory, but this one instance seems to have been overlooked. Was that one intentionally kept, or could it be folded into this patch so the whole directory is done in one pass?
Yep, a new droppable blank line.
Thanks,
Leon
Quoted 7 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
PATCH v12 2/10 bpf: Factor out check_map_mem_read helper in verifier
In reply to Leon HwangIn the next commit, percpu_array map will add map_direct_value_addr support.
IOW, it will add a map_type check in the iff condition of the bpf_map_direct_read() code block, which will reduce the code block readability.
Hence, factor out check_map_mem_read helper to improve the readability, and the maintainability for the percpu_array map case.
--- kernel/bpf/verifier.c | 75 +++++++++++++++++++++++++------------------ 1 file changed, 43 insertions(+), 32 deletions(-)
kernel/bpf/verifier.c +43 -32
@@ -6078,6 +6078,48 @@ static void add_scalar_to_reg(struct bpf_reg_state *dst_reg, s64 val)reg_bounds_sync (dst_reg); } +static int check_map_mem_read (struct bpf_verifier_env *env,struct bpf_reg_state *reg,int off, +int bpf_size,int value_regno,bool is_ldsx) +{ +struct bpf_reg_state *regs =cur_regs (env); +int size =bpf_size_to_bytes (bpf_size); +struct bpf_map *map = reg->map_ptr; + +switch (map->map_type) { +case BPF_MAP_TYPE_INSN_ARRAY : +if (bpf_size !=BPF_DW ) { +verbose (env,"Invalid read of %d bytes from insn_array\n" , size); +return -EACCES ; + } + regs[value_regno] = *reg; +add_scalar_to_reg (®s[value_regno], off); + regs[value_regno].type =PTR_TO_INSN ; +return 0 ; +default : +break ; + } + +/* If map is read-only, track its contents as scalars. */ +if (tnum_is_const (reg->var_off) && +bpf_map_is_rdonly (map) && + map->ops->map_direct_value_addr) { +int map_off = off + reg->var_off.value; +u64 val =0 ; +int err; + + err =bpf_map_direct_read (map, map_off, size, &val, is_ldsx); +if (err) +return err; + + regs[value_regno].type =SCALAR_VALUE ; +__mark_reg_known (®s[value_regno], val); +return 0 ; + } + +mark_reg_unknown (env, regs, value_regno); +return 0 ; +} +/* check whether memory at (regno + off) is accessible for t = (read | write) * if t==write, value_regno is a register which value is stored into memory * if t==read, value_regno is a register which will receive the value from memory
@@ -6132,38 +6174,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct bif (kptr_field) { err =check_map_kptr_access (env, value_regno, insn_idx, kptr_field); }else if (t ==BPF_READ && value_regno >=0 ) { -struct bpf_map *map = reg->map_ptr; - -/* -* If map is read-only, track its contents as scalars, -* unless it is an insn array (see the special case below) -*/ -if (tnum_is_const (reg->var_off) && -bpf_map_is_rdonly (map) && - map->ops->map_direct_value_addr && - map->map_type !=BPF_MAP_TYPE_INSN_ARRAY ) { -int map_off = off + reg->var_off.value; -u64 val =0 ; - - err =bpf_map_direct_read (map, map_off, size, - &val, is_ldsx); -if (err) -return err; - - regs[value_regno].type =SCALAR_VALUE ; -__mark_reg_known (®s[value_regno], val); - }else if (map->map_type ==BPF_MAP_TYPE_INSN_ARRAY ) { -if (bpf_size !=BPF_DW ) { -verbose (env,"Invalid read of %d bytes from insn_array\n" , - size); -return -EACCES ; - } - regs[value_regno] = *reg; -add_scalar_to_reg (®s[value_regno], off); - regs[value_regno].type =PTR_TO_INSN ; - }else { -mark_reg_unknown (env, regs, value_regno); - } + err =check_map_mem_read (env, reg, off, bpf_size, value_regno, is_ldsx); } }else if (base_type (reg->type) ==PTR_TO_MEM ) {bool rdonly_mem =type_is_rdonly_mem (reg->type);
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v12 02/10] bpf: Factor out check_map_mem_read helper in verifier
This isn't a bug, but did 'the iff condition' mean 'the if condition' guarding the bpf_map_direct_read() call? And since this looks like a pure code motion, would it be worth adding a 'No functional…
Quoted 7 lines
In the next commit,
percpu_arraymap will addmap_direct_value_addrsupport.
IOW, it will add amap_typecheck in the iff condition of thebpf_map_direct_read()code block, which will reduce the code block readability.
Hence, factor outcheck_map_mem_readhelper to improve the readability, and the maintainability for thepercpu_arraymap case.
This isn't a bug, but did 'the iff condition' mean 'the if condition' guarding the bpf_map_direct_read() call? And since this looks like a pure code motion, would it be worth adding a 'No functional change intended.' line so reviewers know what to expect?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 02/10] bpf: Factor out check_map_mem_read helper in verifier
Quoted 8 lines · On 14/8/26 00:26, [email protected] wrote:
Quoted 7 lines
In the next commit,
percpu_arraymap will addmap_direct_value_addrsupport.
IOW, it will add amap_typecheck in the iff condition of thebpf_map_direct_read()code block, which will reduce the code block readability.
Hence, factor outcheck_map_mem_readhelper to improve the readability, and the maintainability for thepercpu_arraymap case.
Quoted 4 lines
This isn't a bug, but did 'the iff condition' mean 'the if condition' guarding the
bpf_map_direct_read()call? And since this looks like a pure code motion, would it be worth adding a 'No functional change intended.' line so reviewers know what to expect?
'No functional change intended.' is worth in the commit msg.
Thanks,
Leon
PATCH v12 3/10 bpf: Introduce global percpu data
In reply to Leon HwangIntroduce global percpu data, inspired by the commit 6316f78306c1 ("Merge branch 'support-global-data'"). It enables the definition of global percpu variables in BPF, similar to the include/linux/percpu-defs.h::DEFINE_PER_CPU() macro.
For example, in BPF, it is able to define a global percpu variable like:
int data SEC(".percpu");
With this patch, tools like retsnoop [1] and bpfsnoop [2] can simplify their BPF code for handling LBRs. The code can be updated from
static struct perf_branch_entry lbrs[1][MAX_LBR_ENTRIES] SEC(".data.lbrs");
to
static struct perf_branch_entry lbrs[MAX_LBR_ENTRIES] SEC(".percpu.lbrs");
This eliminates the need to retrieve the CPU ID using the bpf_get_smp_processor_id() helper.
Additionally, by reusing global percpu data map, sharing information between tail callers and callees or freplace callers and callees becomes simpler compared to reusing percpu_array maps.
Links:
[1] https://github.com/anakryiko/retsnoop
[2] https://github.com/bpfsnoop/bpfsnoop
--- kernel/bpf/arraymap.c | 38 ++++++++++++++++++++++++++++++++++++-- kernel/bpf/const_fold.c | 1 - kernel/bpf/fixups.c | 37 +++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 11 +++++++++++ 4 files changed, 84 insertions(+), 3 deletions(-)
kernel/bpf/arraymap.c +36 -2
@@ -259,6 +259,37 @@ static void *percpu_array_map_lookup_elem(struct bpf_map *map, void *key)return this_cpu_ptr (array->pptrs[index & array->index_mask]); } +static int percpu_array_map_direct_value_addr (const struct bpf_map *map,u64 *imm,u32 off) +{ + struct bpf_array *array = container_of(map, struct bpf_array, map); + +if (!bpf_jit_supports_percpu_insn ()) +return -EOPNOTSUPP ; +if (map->max_entries !=1 ) +return -EOPNOTSUPP ; +if (off >= map->value_size) +return -EINVAL ; + + *imm = (u64 )(__force unsigned long ) array->pptrs[0 ]; +return 0 ; +} + +static int percpu_array_map_direct_value_meta (const struct bpf_map *map,u64 imm,u32 *off) +{ + struct bpf_array *array = container_of(map, struct bpf_array, map); +u64 base = (u64 )(__force unsigned long ) array->pptrs[0 ]; + +if (!bpf_jit_supports_percpu_insn ()) +return -EOPNOTSUPP ; +if (map->max_entries !=1 ) +return -EOPNOTSUPP ; +if (imm < base || imm >= base + array->elem_size) +return -ENOENT ; + + *off = imm - base; +return 0 ; +} +/* emit BPF instructions equivalent to C code of percpu_array_map_lookup_elem() */ static int percpu_array_map_gen_lookup (struct bpf_map *map,struct bpf_insn *insn_buf) {
@@ -551,9 +582,10 @@ static int array_map_check_btf(struct bpf_map *map, const struct btf_type *key_type, const struct btf_type *value_type) { - /* One exception for keyless BTF: .bss/.data/.rodata map */ + /* One exception for keyless BTF: .bss/.data/.rodata/.percpu map */ if (btf_type_is_void(key_type)) { - if (map->map_type != BPF_MAP_TYPE_ARRAY || + if ((map->map_type != BPF_MAP_TYPE_ARRAY && + map->map_type != BPF_MAP_TYPE_PERCPU_ARRAY) || map->max_entries != 1) return -EINVAL;
@@ -832,6 +864,8 @@ const struct bpf_map_ops percpu_array_map_ops = { .map_get_next_key = bpf_array_get_next_key, .map_lookup_elem = percpu_array_map_lookup_elem, .map_gen_lookup = percpu_array_map_gen_lookup, + .map_direct_value_addr = percpu_array_map_direct_value_addr, + .map_direct_value_meta = percpu_array_map_direct_value_meta, .map_update_elem = array_map_update_elem, .map_delete_elem = array_map_delete_elem, .map_lookup_percpu_elem = percpu_array_map_lookup_percpu_elem,
kernel/bpf/const_fold.c +0 -1
@@ -182,7 +182,6 @@ static void const_reg_xfer(struct bpf_verifier_env *env, struct const_arg_info *u64 val =0 ;if (!bpf_map_is_rdonly (map) || !map->ops->map_direct_value_addr || - map->map_type ==BPF_MAP_TYPE_INSN_ARRAY || off <0 || off + size > map->value_size ||bpf_map_direct_read (map, off, size, &val, is_ldsx)) { *dst = unknown;
kernel/bpf/fixups.c +37 -0
@@ -1834,6 +1834,43 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env) goto next_insn; } +if (bpf_jit_supports_percpu_insn () && + insn->code == (BPF_LD |BPF_IMM |BPF_DW ) && + (insn->src_reg ==BPF_PSEUDO_MAP_VALUE || + insn->src_reg ==BPF_PSEUDO_MAP_IDX_VALUE )) { +struct bpf_map *map; + + aux = &env->insn_aux_data[i + delta]; + map = env->used_maps[aux->map_index]; +if (map->map_type !=BPF_MAP_TYPE_PERCPU_ARRAY ) + goto next_insn; + + prog->jit_required = true; + +/* +* We are *skipping* first half of ld_imm64 insn +* with 'i++;', patching over second half of it +* with that same half + mov64_percpu_reg insn. +* All because bpf_patch_insn_data() can only +* replace one 8-byte insn, which does not work +* well for ld_imm64 insn. +*/ + + insn_buf[0 ] = insn[1 ]; + insn_buf[1 ] =BPF_MOV64_PERCPU_REG (insn->dst_reg, insn->dst_reg); + cnt =2 ; + + i++; + new_prog =bpf_patch_insn_data (env, i + delta, insn_buf, cnt); +if (!new_prog) +return -ENOMEM ; + + delta += cnt -1 ; + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + goto next_insn; + } +if (insn->code != (BPF_JMP |BPF_CALL )) goto next_insn;if (insn->src_reg ==BPF_PSEUDO_CALL )
kernel/bpf/verifier.c +11 -0
@@ -5582,6 +5582,8 @@ int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val,u64 addr;int err; +if (map->map_type ==BPF_MAP_TYPE_INSN_ARRAY || map->map_type ==BPF_MAP_TYPE_PERCPU_ARRAY ) +return -EINVAL ; err = map->ops->map_direct_value_addr (map, &addr, off);if (err)return err;
@@ -6095,6 +6097,8 @@ static int check_map_mem_read(struct bpf_verifier_env *env, struct bpf_reg_stateadd_scalar_to_reg (®s[value_regno], off); regs[value_regno].type =PTR_TO_INSN ;return 0 ; +case BPF_MAP_TYPE_PERCPU_ARRAY : + goto reg_unknown;default :break ; }
@@ -6116,6 +6120,7 @@ static int check_map_mem_read(struct bpf_verifier_env *env, struct bpf_reg_statereturn 0 ; } +reg_unknown:mark_reg_unknown (env, regs, value_regno);return 0 ; }
@@ -8129,6 +8134,12 @@ static int check_arg_const_str(struct bpf_verifier_env *env,return -EACCES ; } +if (map->map_type ==BPF_MAP_TYPE_PERCPU_ARRAY ) { +verbose (env,"%s points to percpu_array map which cannot be used as const string\n" , +reg_arg_name (env, argno)); +return -EACCES ; + } +if (!bpf_map_is_rdonly (map)) {verbose (env,"%s does not point to a readonly map'\n" ,reg_arg_name (env, argno));return -EACCES ;
Signature · 1 line
2.55.0
PATCH v12 4/10 libbpf: Probe percpu data feature
In reply to Leon Hwanglibbpf needs a reliable way to distinguish kernels that can support global percpu data from those that cannot.
Add a dedicated feature probe, so libbpf can make capability decisions early and fail predictably when global percpu data is unavailable.
--- tools/lib/bpf/features.c | 35 +++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 2 files changed, 37 insertions(+)
tools/lib/bpf/features.c +35 -0
@@ -620,6 +620,38 @@ static int probe_bpf_syscall_common_attrs(int token_fd)return probe_sys_bpf_ext (); } +static int probe_kern_percpu_data (int token_fd) +{ +struct bpf_insn insns[] = { +BPF_LD_MAP_VALUE (BPF_REG_1 ,0 ,0 ), +BPF_LDX_MEM (BPF_DW ,BPF_REG_0 ,BPF_REG_1 ,0 ), +BPF_EXIT_INSN (), + }; +LIBBPF_OPTS (bpf_map_create_opts, map_opts, + .token_fd = token_fd, + .map_flags = token_fd ? BPF_F_TOKEN_FD : 0, + ); +LIBBPF_OPTS (bpf_prog_load_opts, prog_opts, + .token_fd = token_fd, + .prog_flags = token_fd ? BPF_F_TOKEN_FD : 0, + ); +int ret, map, insn_cnt =ARRAY_SIZE (insns); + + map =bpf_map_create (BPF_MAP_TYPE_PERCPU_ARRAY ,"libbpf_percpu" ,sizeof (int ),8 ,1 , + &map_opts); +if (map <0 ) { +pr_warn ("Error in %s(): %s. Couldn't create simple percpu_array map.\n" , + __func__,errstr (map)); +return map; + } + + insns[0 ].imm = map; + + ret =bpf_prog_load (BPF_PROG_TYPE_SOCKET_FILTER ,NULL ,"GPL" , insns, insn_cnt, &prog_opts); +close (map); +return probe_fd (ret); +} +typedef int (*feature_probe_fn )(int /* token_fd */ );static struct kern_feature_cache feature_cache;
@@ -707,6 +739,9 @@ static struct kern_feature_desc { [FEAT_BPF_SYSCALL_COMMON_ATTRS ] = {"BPF syscall common attributes support" , probe_bpf_syscall_common_attrs, }, + [FEAT_PERCPU_DATA ] = { +"kernel supports percpu data" , probe_kern_percpu_data, + }, };bool feat_supported (struct kern_feature_cache *cache,enum kern_feature_id feat_id)
tools/lib/bpf/libbpf_internal.h +2 -0
@@ -401,6 +401,8 @@ enum kern_feature_id {FEAT_BTF_LAYOUT ,/* Kernel supports BPF syscall common attributes */ FEAT_BPF_SYSCALL_COMMON_ATTRS , +/* Kernel supports percpu data */ +FEAT_PERCPU_DATA , __FEAT_CNT, };
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v12 04/10] libbpf: Probe percpu data feature
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The feature probe `probe_kern_percpu_data` incorrectly returns a negative error code and logs a…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The feature probe probe_kern_percpu_data incorrectly returns a negative error code and logs a warning when the kernel lacks support for BPF_MAP_TYPE_PERCPU_ARRAY. Feature probes in libbpf should silently return 0 when a feature is missing to gracefully indicate its absence without polluting stderr. --
--- Patch [4]: [PATCH bpf-next v12 04/10] libbpf: Probe percpu data feature --- commit 783c3759ac5b8bbe1c5368930abe9953ff13c465
Author: Leon Hwang <[email protected]>
libbpf: Probe percpu data feature
This commit adds a feature probe for global percpu data support. It allows libbpf to detect if the kernel supports percpu data early and make capability decisions accordingly.
Quoted 4 lines
tools/lib/bpf/features.c +0
-0
[ ... ]
Quoted 14 lines
tools/lib/bpf/features.c +14
-0+static int probe_kern_percpu_data (int token_fd) +{ +struct bpf_insn insns[] = { +BPF_LD_MAP_VALUE (BPF_REG_1 ,0 ,0 ), +BPF_LDX_MEM (BPF_DW ,BPF_REG_0 ,BPF_REG_1 ,0 ), +BPF_EXIT_INSN (), + }; +LIBBPF_OPTS (bpf_map_create_opts, map_opts, + .token_fd = token_fd, + .map_flags = token_fd ? BPF_F_TOKEN_FD : 0, + ); + LIBBPF_OPTS(bpf_prog_load_opts, prog_opts, + .token_fd = token_fd, + .prog_flags = token_fd ? BPF_F_TOKEN_FD : 0,
tools/lib/bpf/features.c static int probe_bpf_syscall_common_attrs(int token_fd) · line 646
Quoted 10 lines
tools/lib/bpf/features.c +10
-0+ ); +int ret, map, insn_cnt =ARRAY_SIZE (insns); + + map =bpf_map_create (BPF_MAP_TYPE_PERCPU_ARRAY ,"libbpf_percpu" ,sizeof (int ),8 ,1 , + &map_opts); +if (map <0 ) { +pr_warn ("Error in %s(): %s. Couldn't create simple percpu_array map.\n" , + __func__,errstr (map)); +return map; + }
[Severity: Medium]
Will this cause spurious warnings when probing environments that don't support BPF_MAP_TYPE_PERCPU_ARRAY?
Since feat_supported() expects feature probes to return 0 when a feature is missing, returning a negative error code here causes feat_supported() to log an additional failure warning.
Should this return 0 instead of map (and perhaps omit the pr_warn) to gracefully indicate the feature's absence without polluting stderr?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH bpf-next v12 04/10] libbpf: Probe percpu data feature
Quoted 48 lines · On 13/8/26 23:38, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The feature probeprobe_kern_percpu_dataincorrectly returns a negative error code and logs a warning when the kernel lacks support forBPF_MAP_TYPE_PERCPU_ARRAY. Feature probes in libbpf should silently return 0 when a feature is missing to gracefully indicate its absence without polluting stderr. ----- Patch [4]: [PATCH bpf-next v12 04/10] libbpf: Probe percpu data feature --- commit
783c3759ac5b8bbe1c5368930abe9953ff13c465
Author: Leon Hwang <[email protected]>libbpf: Probe percpu data feature
This commit adds a feature probe for global percpu data support. It allows libbpf to detect if the kernel supports percpu data early and make capability decisions accordingly.
Quoted 4 lines
tools/lib/bpf/features.c +0
-0[ ... ]
Quoted 14 lines
tools/lib/bpf/features.c +14
-0+static int probe_kern_percpu_data (int token_fd) +{ +struct bpf_insn insns[] = { +BPF_LD_MAP_VALUE (BPF_REG_1 ,0 ,0 ), +BPF_LDX_MEM (BPF_DW ,BPF_REG_0 ,BPF_REG_1 ,0 ), +BPF_EXIT_INSN (), + }; +LIBBPF_OPTS (bpf_map_create_opts, map_opts, + .token_fd = token_fd, + .map_flags = token_fd ? BPF_F_TOKEN_FD : 0, + ); + LIBBPF_OPTS(bpf_prog_load_opts, prog_opts, + .token_fd = token_fd, + .prog_flags = token_fd ? BPF_F_TOKEN_FD : 0,tools/lib/bpf/features.c static int probe_bpf_syscall_common_attrs(int token_fd) · line 646
Quoted 10 lines
tools/lib/bpf/features.c +10
-0+ ); +int ret, map, insn_cnt =ARRAY_SIZE (insns); + + map =bpf_map_create (BPF_MAP_TYPE_PERCPU_ARRAY ,"libbpf_percpu" ,sizeof (int ),8 ,1 , + &map_opts); +if (map <0 ) { +pr_warn ("Error in %s(): %s. Couldn't create simple percpu_array map.\n" , + __func__,errstr (map)); +return map; + }[Severity: Medium]
Will this cause spurious warnings when probing environments that don't supportBPF_MAP_TYPE_PERCPU_ARRAY?
Quoted 7 lines
Since
feat_supported()expects feature probes to return 0 when a feature is missing, returning a negative error code here causesfeat_supported()to log an additional failure warning.Should this return 0 instead of map (and perhaps omit the
pr_warn) to gracefully indicate the feature's absence without polluting stderr?
No.
Since percpu_array map is a basic map in bpf, this map creation should likely succeed. If it fails, the kernels are too old to use such new libbpf.
Thanks,
Leon
PATCH v12 5/10 libbpf: Add support for global percpu data
In reply to Leon HwangAdd support for global percpu data in libbpf by adding a new ".percpu" section, similar to ".data". It enables efficient handling of percpu global variables in bpf programs.
When generating loader for lightweight skeleton, update the percpu_array map used for global percpu data using BPF_F_ALL_CPUS, in order to update values across all CPUs using one value slot.
Unlike global data, the mmaped data for global percpu data will be marked as read-only after populating the percpu_array map. Thereafter, users can read those initialized percpu data after loading prog. If they want to update the percpu data after loading prog, they have to update the percpu_array map using key=0 instead.
--- tools/lib/bpf/bpf_gen_internal.h | 3 +- tools/lib/bpf/gen_loader.c | 3 +- tools/lib/bpf/libbpf.c | 81 +++++++++++++++++++++++++++----- 3 files changed, 72 insertions(+), 15 deletions(-)
tools/lib/bpf/bpf_gen_internal.h +2 -1
@@ -65,7 +65,8 @@ void bpf_gen__prog_load(struct bpf_gen *gen, enum bpf_prog_type prog_type, const char *prog_name, const char *license, struct bpf_insn *insns, size_t insn_cnt, struct bpf_prog_load_opts *load_attr, int prog_idx); -void bpf_gen__map_update_elem (struct bpf_gen *gen,int map_idx,void *value,__u32 value_size); +void bpf_gen__map_update_elem (struct bpf_gen *gen,int map_idx,void *value,__u32 value_size, +__u64 flags);void bpf_gen__map_freeze (struct bpf_gen *gen,int map_idx);void bpf_gen__record_attach_target (struct bpf_gen *gen,const char *name,enum bpf_attach_type type); void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak,
tools/lib/bpf/gen_loader.c +2 -1
@@ -1128,7 +1128,7 @@ void bpf_gen__prog_load(struct bpf_gen *gen, }void bpf_gen__map_update_elem (struct bpf_gen *gen,int map_idx,void *pvalue, -__u32 value_size) +__u32 value_size,__u64 flags) { int attr_size = offsetofend(union bpf_attr, flags);int map_update_attr, value, key;
@@ -1136,6 +1136,7 @@ void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *pvalue,int zero =0 ;memset (&attr,0 , attr_size); + attr.flags =tgt_endian (flags); value =add_data (gen, pvalue, value_size); key =add_data (gen, &zero,sizeof (zero));
tools/lib/bpf/libbpf.c +68 -13
223 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] bpftool gen skeleton fails to generate fields for .percpu maps, breaking lightweight skeleton generation memory layout.
Pre-existing issues:
- [High] A massive >4GB ELF section causes a gigabyte-scale out-of-bounds memcpy buffer overflow in libbpf. --
--- Patch [5]: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data --- commit ba7ffd0a036fafab1adafd7c20bbbb6455fe6cf6
Author: Leon Hwang <[email protected]>
libbpf: Add support for global percpu data
Add support for global percpu data in libbpf by adding a new ".percpu" section, similar to ".data". It enables efficient handling of percpu global variables in bpf programs.
Quoted 3 lines
When generating loader for lightweight skeleton, update the
percpu_arraymap used for global percpu data usingBPF_F_ALL_CPUS, in order to update values across all CPUs using one value slot.
[Severity: High]
Does this change break bpftool's lightweight skeleton generation for .percpu maps?
Looking at tools/bpf/bpftool/gen.c:get_map_ident(), the hardcoded sfxs array lacks the ".percpu" suffix:
static const char *sfxs[] = { ".data", ".rodata", ".bss", ".kconfig" };This causes the .percpu map to be silently skipped during skeleton generation.
Since the BPF loader program is generated based on the true array index (map_idx = map - obj->maps), skipping .percpu in the generated C struct shifts all subsequent maps. Can this cause the loader program to read or write out of bounds of the skel->maps structure?
Quoted 13 lines
Unlike global data, the mmaped data for global percpu data will be marked as read-only after populating the
percpu_arraymap. Thereafter, users can read those initialized percpu data after loading prog. If they want to update the percpu data after loading prog, they have to update thepercpu_arraymap using key=0 instead.Signed-off-by Leon Hwang <[email protected]>tools/lib/bpf/libbpf.c +0
-0
[ ... ]
tools/lib/bpf/libbpf.c bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type, · line 2005
Quoted 8 lines
tools/lib/bpf/libbpf.c +1
-1@@ -1983,7 +2000,7 @@ bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type, } def = &map->def; - def->type =BPF_MAP_TYPE_ARRAY ; + def->type = is_percpu ?BPF_MAP_TYPE_PERCPU_ARRAY :BPF_MAP_TYPE_ARRAY ; def->key_size =sizeof (int ); def->value_size = data_sz;
[Severity: High]
This is a pre-existing issue, but does this code cause a buffer overflow if a malicious BPF ELF file contains a data section larger than 4GB?
The 64-bit size_t data_sz is truncated into the 32-bit unsigned int def->value_size. The mmap allocation will use this undersized truncated value, but the subsequent memcpy below uses the original 64-bit data_sz, overflowing the undersized map->mmaped buffer:
if (data)
memcpy(map->mmaped, data, data_sz);Can this lead to a massive out-of-bounds write in libbpf?
Quoted 1 line
tools/lib/bpf/libbpf.c +0
-0def->max_entries =1 ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Re: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data
Quoted 27 lines · On 13/8/26 23:42, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] bpftool gen skeleton fails to generate fields for .percpu maps, breaking lightweight skeleton generation memory layout.Pre-existing issues:
- [High] A massive >4GB ELF section causes a gigabyte-scale out-of-bounds memcpy buffer overflow in libbpf. ----- Patch [5]: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data --- commit
ba7ffd0a036fafab1adafd7c20bbbb6455fe6cf6
Author: Leon Hwang <[email protected]>libbpf: Add support for global percpu data
Add support for global percpu data in libbpf by adding a new ".percpu" section, similar to ".data". It enables efficient handling of percpu global variables in bpf programs.
Quoted 3 lines
When generating loader for lightweight skeleton, update the
percpu_arraymap used for global percpu data usingBPF_F_ALL_CPUS, in order to update values across all CPUs using one value slot.[Severity: High]
Does this change break bpftool's lightweight skeleton generation for .percpu maps?
Quoted 7 lines
Looking at tools/bpf/bpftool/gen.c:get_map_ident(), the hardcoded sfxs array lacks the ".percpu" suffix:
static const char *sfxs[] = { ".data", ".rodata", ".bss", ".kconfig" };This causes the .percpu map to be silently skipped during skeleton generation.
False positive.
The .percpu map has been handled correctly in get_map_ident() by checking map type.
Quoted 40 lines
Since the BPF loader program is generated based on the true array index (
map_idx= map -obj->maps), skipping .percpu in the generated C struct shifts all subsequent maps. Can this cause the loader program to read or write out of bounds of theskel->mapsstructure?Quoted 13 lines
Unlike global data, the mmaped data for global percpu data will be marked as read-only after populating the
percpu_arraymap. Thereafter, users can read those initialized percpu data after loading prog. If they want to update the percpu data after loading prog, they have to update thepercpu_arraymap using key=0 instead.Signed-off-by Leon Hwang <[email protected]>tools/lib/bpf/libbpf.c +0
-0[ ... ]
tools/lib/bpf/libbpf.c bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type, · line 2005
Quoted 8 lines
tools/lib/bpf/libbpf.c +1
-1@@ -1983,7 +2000,7 @@ bpf_object__init_internal_map(struct bpf_object *obj, enum libbpf_map_type type, } def = &map->def; - def->type =BPF_MAP_TYPE_ARRAY ; + def->type = is_percpu ?BPF_MAP_TYPE_PERCPU_ARRAY :BPF_MAP_TYPE_ARRAY ; def->key_size =sizeof (int ); def->value_size = data_sz;[Severity: High]
This is a pre-existing issue, but does this code cause a buffer overflow if a malicious BPF ELF file contains a data section larger than 4GB?The 64-bit
size_tdata_szis truncated into the 32-bit unsigned intdef->value_size. The mmap allocation will use this undersized truncated value, but the subsequent memcpy below uses the original 64-bitdata_sz, overflowing the undersizedmap->mmapedbuffer:
Quoted 7 lines
if (data) memcpy(map->mmaped, data, data_sz);Can this lead to a massive out-of-bounds write in libbpf?
Quoted 1 line
tools/lib/bpf/libbpf.c +0
-0def->max_entries =1 ;
We must trust the BPF ELF file.
See Andrii's comment in v8:
https://lore.kernel.org/bpf/CAEf4BzbBpDQOG-xUArBzZa_qZ08q=QViH-bbFmqwLNesGvJLUA@mail.gmail.com/.
Thanks,
Leon
[email protected]
AI reviewRe: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data
In reply to Leon HwangThis isn't a bug, but bpf_map_mmap_sz() is documented as returning the length to pass to mmap()/munmap()/mprotect(), and both other arms return page-granular lengths (array_map_mmap_sz() rounds up…
Quoted 4 lines
tools/lib/bpf/bpf_gen_internal.h +0
-0
[ ... ]
Quoted 4 lines
tools/lib/bpf/libbpf.c +0
-0
[ ... ]
tools/lib/bpf/libbpf.c static size_t bpf_map_mmap_sz(const struct bpf_map *map) · line 1848
Quoted 8 lines
tools/lib/bpf/libbpf.c +2
-0@@ -1839,6 +1842,8 @@ static size_t bpf_map_mmap_sz(const struct bpf_map *map)switch (map->def.type) {case BPF_MAP_TYPE_ARRAY :return array_map_mmap_sz (map->def.value_size, map->def.max_entries); +case BPF_MAP_TYPE_PERCPU_ARRAY : +return map->def.value_size;case BPF_MAP_TYPE_ARENA :return page_sz * map->def.max_entries;
This isn't a bug, but bpf_map_mmap_sz() is documented as returning the length to pass to mmap()/munmap()/mprotect(), and both other arms return page-granular lengths (array_map_mmap_sz() rounds up to page_sz, ARENA multiplies by page_sz).
The new PERCPU_ARRAY arm returns the raw, unrounded value_size. This creates an asymmetry visible in bpf_map__set_value_size(), which compares mmap_old_sz = bpf_map_mmap_sz(map) (unrounded for percpu) against mmap_new_sz = array_map_mmap_sz(size, max_entries) (page-rounded), so even a no-op resize performs a redundant mmap() + memcpy() + munmap() cycle.
Would using array_map_mmap_sz(map->def.value_size, map->def.max_entries) for the PERCPU_ARRAY case be more consistent with the ARRAY case?
[ ... ]
Quoted 14 lines
tools/lib/bpf/libbpf.c +2
-1@@ -1866,7 +1871,8 @@ static int bpf_map_mmap_resize(struct bpf_map *map, size_t old_sz, size_t new_szreturn 0 ; } -static char *internal_map_name (struct bpf_object *obj,const char *real_name) +static char *internal_map_name (struct bpf_object *obj,const char *real_name, +enum libbpf_map_type type) {char map_name[BPF_OBJ_NAME_LEN ], *p;int pfx_len, sfx_len =max ((size_t )7 ,strlen (real_name));@@ -1907,8 +1913,11 @@ static char *internal_map_name(struct bpf_object *obj, const char *real_name)if (sfx_len >=BPF_OBJ_NAME_LEN ) sfx_len =BPF_OBJ_NAME_LEN -1 ;
tools/lib/bpf/libbpf.c static char *internal_map_name(struct bpf_object *obj, const char *real_name) · line 1923
Quoted 10 lines
tools/lib/bpf/libbpf.c +5
-2-/* if there are two or more dots in map name, it's a custom dot map */ -if (strchr (real_name +1 ,'.' ) !=NULL ) +/* +* Don't prefix the bpf_object name if this is a custom dot map +* (containing two or more dots) or a percpu data map. +*/ +if (strchr (real_name +1 ,'.' ) !=NULL || type ==LIBBPF_MAP_PERCPU ) pfx_len =0 ;else pfx_len =min ((size_t )BPF_OBJ_NAME_LEN - sfx_len -1 ,strlen (obj->name));
Emil Tsalapatis raised a design question in v10: Why does .percpu get special treatment in internal_map_name() instead of following the same prefixing logic as .data and .rodata?
He noted "it seems like we're singling out the .percpu section for highly nonbvious reasons" and asked "@Andrii Wdyt?" You responded that you prefer excluding PERCPU "to avoid the legacy naming convention for new internal maps" but said "Let us wait for Andrii's comment."
Andrii didn't publicly respond to this design question in the lore threads, and v11/v12 proceeded with the same special treatment. The related map_uses_real_name() function also treats PERCPU differently (always returns true for PERCPU, while DATA/RODATA check against their base section names).
Should this naming convention uniformity concern be addressed, or is there a technical reason for .percpu's special handling that could be documented in the comment?
[ ... ]
tools/lib/bpf/libbpf.c static bool map_is_mmapable(struct bpf_object *obj, struct bpf_map *map) · line 1959
Quoted 11 lines
tools/lib/bpf/libbpf.c +7
-0@@ -1941,6 +1950,13 @@ static bool map_is_mmapable(struct bpf_object *obj, struct bpf_map *map)if (!map->btf_value_type_id)return false; +/* +* The internal PERCPU maps are not mmapble because the underlying +* percpu_array maps do not have mmap support. +*/ +if (map->libbpf_type ==LIBBPF_MAP_PERCPU ) +return false; +
This isn't a bug, but there's a typo: "mmapble" should be "mmappable".
[ ... ]
Quoted 23 lines
tools/lib/bpf/libbpf.c +4
-2@@ -5293,18 +5326,20 @@ static int bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) { enum libbpf_map_type map_type = map->libbpf_type; +bool is_percpu = map_type ==LIBBPF_MAP_PERCPU ; +const __u64 update_flags = is_percpu ?BPF_F_ALL_CPUS :0 ;int err, zero =0 ;size_t mmap_sz;if (obj->gen_loader) {bpf_gen__map_update_elem (obj->gen_loader, map - obj->maps, - map->mmaped, map->def.value_size); + map->mmaped, map->def.value_size, update_flags);if (map_type ==LIBBPF_MAP_RODATA || map_type ==LIBBPF_MAP_KCONFIG )bpf_gen__map_freeze (obj->gen_loader, map - obj->maps);return 0 ; } - err =bpf_map_update_elem (map->fd, &zero, map->mmaped,0 ); + err =bpf_map_update_elem (map->fd, &zero, map->mmaped, update_flags);if (err) { err = -errno; pr_warn("map '%s': failed to set initial contents: %s\n",
[ ... ]
Quoted 4 lines
tools/lib/bpf/libbpf.c +0
-0@@ -5349,6 +5384,13 @@ bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map)return err; } map->mmaped = mmaped;
tools/lib/bpf/libbpf.c bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) · line 5396
Quoted 10 lines
tools/lib/bpf/libbpf.c +8
-0+ }else if (is_percpu) { +if (mprotect (map->mmaped, mmap_sz,PROT_READ )) { + err = -errno; +pr_warn ("map '%s': failed to mprotect() contents: %s\n" , +bpf_map__name (map),errstr (err)); +return err; + } + }else if (map->mmaped) {munmap (map->mmaped, mmap_sz); map->mmaped =NULL ;
This isn't a bug, but unlike every other internal map, the percpu initialization image is deliberately kept mapped and turned read-only with mprotect() instead of being munmap()ed.
The skeleton pointer that bpftool generates for this map is a plain, non-const 'struct <obj>__percpu *', so a perfectly ordinary post-load assignment such as 'skel->percpu->var = 1;' now faults with SIGSEGV in the user process rather than failing in a diagnosable way.
The behaviour is intentional and described in the commit message, but would a const-qualified generated pointer (as is done for .rodata) express the restriction at compile time instead of at runtime?
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data
Quoted 69 lines · On Thu, Aug 13, 2026 at 9:26 AM <[email protected]> wrote:
Quoted 5 lines
tools/lib/bpf/bpf_gen_internal.h +0
-0[ ... ]
Quoted 4 lines
tools/lib/bpf/libbpf.c +0
-0[ ... ]
tools/lib/bpf/libbpf.c static size_t bpf_map_mmap_sz(const struct bpf_map *map) · line 1848
Quoted 8 lines
tools/lib/bpf/libbpf.c +2
-0@@ -1839,6 +1842,8 @@ static size_t bpf_map_mmap_sz(const struct bpf_map *map)switch (map->def.type) {case BPF_MAP_TYPE_ARRAY :return array_map_mmap_sz (map->def.value_size, map->def.max_entries); +case BPF_MAP_TYPE_PERCPU_ARRAY : +return map->def.value_size;case BPF_MAP_TYPE_ARENA :return page_sz * map->def.max_entries;This isn't a bug, but
bpf_map_mmap_sz()is documented as returning the length to pass to mmap()/munmap()/mprotect(), and both other arms return page-granular lengths (array_map_mmap_sz()rounds up topage_sz, ARENA multiplies bypage_sz).The new
PERCPU_ARRAYarm returns the raw, unroundedvalue_size. This creates an asymmetry visible inbpf_map__set_value_size(), which comparesmmap_old_sz=bpf_map_mmap_sz(map)(unrounded for percpu) againstmmap_new_sz=array_map_mmap_sz(size, max_entries)(page-rounded), so even a no-op resize performs a redundantmmap()+memcpy()+munmap()cycle.Would using
array_map_mmap_sz(map->def.value_size, map->def.max_entries)for thePERCPU_ARRAYcase be more consistent with the ARRAY case?[ ... ]
Quoted 14 lines
tools/lib/bpf/libbpf.c +2
-1@@ -1866,7 +1871,8 @@ static int bpf_map_mmap_resize(struct bpf_map *map, size_t old_sz, size_t new_szreturn 0 ; } -static char *internal_map_name (struct bpf_object *obj,const char *real_name) +static char *internal_map_name (struct bpf_object *obj,const char *real_name, +enum libbpf_map_type type) {char map_name[BPF_OBJ_NAME_LEN ], *p;int pfx_len, sfx_len =max ((size_t )7 ,strlen (real_name));@@ -1907,8 +1913,11 @@ static char *internal_map_name(struct bpf_object *obj, const char *real_name)if (sfx_len >=BPF_OBJ_NAME_LEN ) sfx_len =BPF_OBJ_NAME_LEN -1 ;tools/lib/bpf/libbpf.c static char *internal_map_name(struct bpf_object *obj, const char *real_name) · line 1923
Quoted 10 lines
tools/lib/bpf/libbpf.c +5
-2-/* if there are two or more dots in map name, it's a custom dot map */ -if (strchr (real_name +1 ,'.' ) !=NULL ) +/* +* Don't prefix the bpf_object name if this is a custom dot map +* (containing two or more dots) or a percpu data map. +*/ +if (strchr (real_name +1 ,'.' ) !=NULL || type ==LIBBPF_MAP_PERCPU ) pfx_len =0 ;else pfx_len =min ((size_t )BPF_OBJ_NAME_LEN - sfx_len -1 ,strlen (obj->name));Emil Tsalapatis raised a design question in v10: Why does .percpu get special treatment in
internal_map_name()instead of following the same prefixing logic as .data and .rodata?
Quoted 6 lines
He noted "it seems like we're singling out the .percpu section for highly nonbvious reasons" and asked "@Andrii Wdyt?" You responded that you prefer excluding PERCPU "to avoid the legacy naming convention for new internal maps" but said "Let us wait for Andrii's comment."
Andrii didn't publicly respond to this design question in the lore threads,
ok, wow, being shamed by AI... I did respond on v10, and yes, I think passing type and basing decisions on that is better going forward, no need for name-based checks.
Quoted 72 lines
and v11/v12 proceeded with the same special treatment. The related
map_uses_real_name()function also treats PERCPU differently (always returns true for PERCPU, while DATA/RODATA check against their base section names).Should this naming convention uniformity concern be addressed, or is there a technical reason for .percpu's special handling that could be documented in the comment?
[ ... ]
tools/lib/bpf/libbpf.c static bool map_is_mmapable(struct bpf_object *obj, struct bpf_map *map) · line 1959
Quoted 11 lines
tools/lib/bpf/libbpf.c +7
-0@@ -1941,6 +1950,13 @@ static bool map_is_mmapable(struct bpf_object *obj, struct bpf_map *map)if (!map->btf_value_type_id)return false; +/* +* The internal PERCPU maps are not mmapble because the underlying +* percpu_array maps do not have mmap support. +*/ +if (map->libbpf_type ==LIBBPF_MAP_PERCPU ) +return false; +This isn't a bug, but there's a typo: "mmapble" should be "mmappable".
[ ... ]
Quoted 23 lines
tools/lib/bpf/libbpf.c +4
-2@@ -5293,18 +5326,20 @@ static int bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) { enum libbpf_map_type map_type = map->libbpf_type; +bool is_percpu = map_type ==LIBBPF_MAP_PERCPU ; +const __u64 update_flags = is_percpu ?BPF_F_ALL_CPUS :0 ;int err, zero =0 ;size_t mmap_sz;if (obj->gen_loader) {bpf_gen__map_update_elem (obj->gen_loader, map - obj->maps, - map->mmaped, map->def.value_size); + map->mmaped, map->def.value_size, update_flags);if (map_type ==LIBBPF_MAP_RODATA || map_type ==LIBBPF_MAP_KCONFIG )bpf_gen__map_freeze (obj->gen_loader, map - obj->maps);return 0 ; } - err =bpf_map_update_elem (map->fd, &zero, map->mmaped,0 ); + err =bpf_map_update_elem (map->fd, &zero, map->mmaped, update_flags);if (err) { err = -errno; pr_warn("map '%s': failed to set initial contents: %s\n",[ ... ]
Quoted 4 lines
tools/lib/bpf/libbpf.c +0
-0@@ -5349,6 +5384,13 @@ bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map)return err; } map->mmaped = mmaped;tools/lib/bpf/libbpf.c bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) · line 5396
Quoted 10 lines
tools/lib/bpf/libbpf.c +8
-0+ }else if (is_percpu) { +if (mprotect (map->mmaped, mmap_sz,PROT_READ )) { + err = -errno; +pr_warn ("map '%s': failed to mprotect() contents: %s\n" , +bpf_map__name (map),errstr (err)); +return err; + } + }else if (map->mmaped) {munmap (map->mmaped, mmap_sz); map->mmaped =NULL ;This isn't a bug, but unlike every other internal map, the percpu initialization image is deliberately kept mapped and turned read-only with
mprotect()instead of being munmap()ed.
Quoted 7 lines
The skeleton pointer that bpftool generates for this map is a plain, non-const 'struct <obj>__percpu *', so a perfectly ordinary post-load assignment such as '
skel->percpu->var= 1;' now faults with SIGSEGV in the user process rather than failing in a diagnosable way.The behaviour is intentional and described in the commit message, but would a const-qualified generated pointer (as is done for .rodata) express the
Is it done for .rodata, really?... As far as I can see, .rodata are still non-const because they are expected to be set before load. Eh, AI...
Quoted 10 lines
restriction at compile time instead of at runtime?
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 05/10] libbpf: Add support for global percpu data
In reply to [email protected]Quoted 28 lines · On 14/8/26 00:26, [email protected] wrote:
Quoted 4 lines
tools/lib/bpf/bpf_gen_internal.h +0
-0[ ... ]
Quoted 4 lines
tools/lib/bpf/libbpf.c +0
-0[ ... ]
tools/lib/bpf/libbpf.c static size_t bpf_map_mmap_sz(const struct bpf_map *map) · line 1848
Quoted 8 lines
tools/lib/bpf/libbpf.c +2
-0@@ -1839,6 +1842,8 @@ static size_t bpf_map_mmap_sz(const struct bpf_map *map)switch (map->def.type) {case BPF_MAP_TYPE_ARRAY :return array_map_mmap_sz (map->def.value_size, map->def.max_entries); +case BPF_MAP_TYPE_PERCPU_ARRAY : +return map->def.value_size;case BPF_MAP_TYPE_ARENA :return page_sz * map->def.max_entries;This isn't a bug, but
bpf_map_mmap_sz()is documented as returning the length to pass to mmap()/munmap()/mprotect(), and both other arms return page-granular lengths (array_map_mmap_sz()rounds up topage_sz, ARENA multiplies bypage_sz).
Quoted 8 lines
The new
PERCPU_ARRAYarm returns the raw, unroundedvalue_size. This creates an asymmetry visible inbpf_map__set_value_size(), which comparesmmap_old_sz=bpf_map_mmap_sz(map)(unrounded for percpu) againstmmap_new_sz=array_map_mmap_sz(size, max_entries)(page-rounded), so even a no-op resize performs a redundantmmap()+memcpy()+munmap()cycle.Would using
array_map_mmap_sz(map->def.value_size, map->def.max_entries)for thePERCPU_ARRAYcase be more consistent with the ARRAY case?
Correct.
Will move 'case BPF_MAP_TYPE_PERCPU_ARRAY:' upward one line, and drop 'return map->def.value_size;' line.
Quoted 43 lines
[ ... ]
Quoted 14 lines
tools/lib/bpf/libbpf.c +2
-1@@ -1866,7 +1871,8 @@ static int bpf_map_mmap_resize(struct bpf_map *map, size_t old_sz, size_t new_szreturn 0 ; } -static char *internal_map_name (struct bpf_object *obj,const char *real_name) +static char *internal_map_name (struct bpf_object *obj,const char *real_name, +enum libbpf_map_type type) {char map_name[BPF_OBJ_NAME_LEN ], *p;int pfx_len, sfx_len =max ((size_t )7 ,strlen (real_name));@@ -1907,8 +1913,11 @@ static char *internal_map_name(struct bpf_object *obj, const char *real_name)if (sfx_len >=BPF_OBJ_NAME_LEN ) sfx_len =BPF_OBJ_NAME_LEN -1 ;tools/lib/bpf/libbpf.c static char *internal_map_name(struct bpf_object *obj, const char *real_name) · line 1923
Quoted 10 lines
tools/lib/bpf/libbpf.c +5
-2-/* if there are two or more dots in map name, it's a custom dot map */ -if (strchr (real_name +1 ,'.' ) !=NULL ) +/* +* Don't prefix the bpf_object name if this is a custom dot map +* (containing two or more dots) or a percpu data map. +*/ +if (strchr (real_name +1 ,'.' ) !=NULL || type ==LIBBPF_MAP_PERCPU ) pfx_len =0 ;else pfx_len =min ((size_t )BPF_OBJ_NAME_LEN - sfx_len -1 ,strlen (obj->name));Emil Tsalapatis raised a design question in v10: Why does .percpu get special treatment in
internal_map_name()instead of following the same prefixing logic as .data and .rodata?He noted "it seems like we're singling out the .percpu section for highly nonbvious reasons" and asked "@Andrii Wdyt?" You responded that you prefer excluding PERCPU "to avoid the legacy naming convention for new internal maps" but said "Let us wait for Andrii's comment."
Andrii didn't publicly respond to this design question in the lore threads, and v11/v12 proceeded with the same special treatment. The related
map_uses_real_name()function also treats PERCPU differently (always returns true for PERCPU, while DATA/RODATA check against their base section names).
Quoted 3 lines
Should this naming convention uniformity concern be addressed, or is there a technical reason for .percpu's special handling that could be documented in the comment?
See Andrii's comment.
Quoted 7 lines
[ ... ]
Quoted 4 lines
tools/lib/bpf/libbpf.c +0
-0@@ -1941,6 +1950,13 @@ static bool map_is_mmapable(struct bpf_object *obj, struct bpf_map *map)if (!map->btf_value_type_id)return false;
Quoted 9 lines
tools/lib/bpf/libbpf.c static bool map_is_mmapable(struct bpf_object *obj, struct bpf_map *map) · line 1959
Quoted 7 lines
tools/lib/bpf/libbpf.c +7
-0+/* +* The internal PERCPU maps are not mmapble because the underlying +* percpu_array maps do not have mmap support. +*/ +if (map->libbpf_type ==LIBBPF_MAP_PERCPU ) +return false; +This isn't a bug, but there's a typo: "mmapble" should be "mmappable".
Yes, it is a typo. "mmapble" should be "mmapable".
Quoted 48 lines
[ ... ]
Quoted 23 lines
tools/lib/bpf/libbpf.c +4
-2@@ -5293,18 +5326,20 @@ static int bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) { enum libbpf_map_type map_type = map->libbpf_type; +bool is_percpu = map_type ==LIBBPF_MAP_PERCPU ; +const __u64 update_flags = is_percpu ?BPF_F_ALL_CPUS :0 ;int err, zero =0 ;size_t mmap_sz;if (obj->gen_loader) {bpf_gen__map_update_elem (obj->gen_loader, map - obj->maps, - map->mmaped, map->def.value_size); + map->mmaped, map->def.value_size, update_flags);if (map_type ==LIBBPF_MAP_RODATA || map_type ==LIBBPF_MAP_KCONFIG )bpf_gen__map_freeze (obj->gen_loader, map - obj->maps);return 0 ; } - err =bpf_map_update_elem (map->fd, &zero, map->mmaped,0 ); + err =bpf_map_update_elem (map->fd, &zero, map->mmaped, update_flags);if (err) { err = -errno; pr_warn("map '%s': failed to set initial contents: %s\n",[ ... ]
Quoted 4 lines
tools/lib/bpf/libbpf.c +0
-0@@ -5349,6 +5384,13 @@ bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map)return err; } map->mmaped = mmaped;tools/lib/bpf/libbpf.c bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) · line 5396
Quoted 10 lines
tools/lib/bpf/libbpf.c +8
-0+ }else if (is_percpu) { +if (mprotect (map->mmaped, mmap_sz,PROT_READ )) { + err = -errno; +pr_warn ("map '%s': failed to mprotect() contents: %s\n" , +bpf_map__name (map),errstr (err)); +return err; + } + }else if (map->mmaped) {munmap (map->mmaped, mmap_sz); map->mmaped =NULL ;This isn't a bug, but unlike every other internal map, the percpu initialization image is deliberately kept mapped and turned read-only with
mprotect()instead of being munmap()ed.
Quoted 8 lines
The skeleton pointer that bpftool generates for this map is a plain, non-const 'struct <obj>__percpu *', so a perfectly ordinary post-load assignment such as '
skel->percpu->var= 1;' now faults with SIGSEGV in the user process rather than failing in a diagnosable way.The behaviour is intentional and described in the commit message, but would a const-qualified generated pointer (as is done for .rodata) express the restriction at compile time instead of at runtime?
See Andrii's comment.
Thanks,
Leon
Quoted 9 lines
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
PATCH v12 6/10 bpftool: Generate skeleton for global percpu data
In reply to Leon HwangEnhance bpftool to generate skeletons that properly handle global percpu variables. The generated skeleton now includes a dedicated structure for percpu data, allowing users to initialize and access percpu variables more efficiently.
For global percpu variables, the skeleton now includes a nested structure, e.g.:
struct test_global_percpu_data {
struct bpf_object_skeleton *skeleton;
struct bpf_object *obj;
struct {
struct bpf_map *percpu;
} maps;
// ...
struct test_global_percpu_data__percpu {
int data;
char run;
struct {
char set;
int i;
int nums[7];
} struct_data;
int nums[7];
} *percpu;
// ...
}; * The "struct test_global_percpu_data__percpu *percpu" points to
initialized data, which is actually "maps.percpu->mmaped".
* Before loading the skeleton, updating the
"struct test_global_percpu_data__percpu *percpu" modifies the initial
value of the corresponding global percpu variables.
* After loading the skeleton, "maps.percpu->mmaped" has been marked as
read-only in libbpf. If users want to update the global percpu
variables, they have to update the "maps.percpu" map instead.
* For lightweight skeleton, "lskel->percpu" will be protected by
"mprotect(p, sz, PROT_READ)".* For subskeleton, those variables of global percpu data will be skipped.
--- tools/bpf/bpftool/gen.c | 53 +++++++++++++++++++++++++++-------- tools/lib/bpf/skel_internal.h | 24 ++++++++++++++-- 2 files changed, 63 insertions(+), 14 deletions(-)
tools/bpf/bpftool/gen.c +41 -12
@@ -101,6 +101,12 @@ static bool get_map_ident(const struct bpf_map *map, char *buf, size_t buf_sz)return true; } +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) { +snprintf (buf, buf_sz,"%s" , name +1 ); +sanitize_identifier (buf); +return true; + } +for (i =0 , n =ARRAY_SIZE (sfxs); i < n; i++) {const char *sfx = sfxs[i], *p;
@@ -117,7 +123,7 @@ static bool get_map_ident(const struct bpf_map *map, char *buf, size_t buf_sz)static bool get_datasec_ident (const char *sec_name,char *buf,size_t buf_sz) { -static const char *pfxs[] = {".data" ,".rodata" ,".bss" ,".kconfig" }; +static const char *pfxs[] = {".data" ,".rodata" ,".bss" ,".percpu" ,".kconfig" };int i, n;/* recognize hard coded LLVM section name */
@@ -254,7 +260,7 @@ static const struct btf_type *find_type_for_map(struct btf *btf, const char *mapreturn NULL ; } -static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +static bool is_skel_data (const struct bpf_map *map,char *buf,size_t sz) {size_t tmp_sz;
@@ -263,13 +269,24 @@ static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz)return true; } -if (!bpf_map__is_internal (map) || !(bpf_map__map_flags (map) &BPF_F_MMAPABLE )) +if (!bpf_map__is_internal (map))return false;if (!get_map_ident (map, buf, sz))return false; -return true; +if (bpf_map__map_flags (map) &BPF_F_MMAPABLE ) +return true; + +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) +return bpf_map__btf_value_type_id (map) !=0 ; + +return false; +} + +static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +{ +return is_skel_data (map, buf, sz) &&bpf_map__type (map) !=BPF_MAP_TYPE_PERCPU_ARRAY ; }static int codegen_datasecs (struct bpf_object *obj,const char *obj_name)
@@ -287,7 +304,7 @@ static int codegen_datasecs(struct bpf_object *obj, const char *obj_name)bpf_object__for_each_map (map, obj) {/* only generate definitions for memory-mapped internal maps */ -if (!is_mmapable_map (map, map_ident,sizeof (map_ident))) +if (!is_skel_data (map, map_ident,sizeof (map_ident)))continue ; sec =find_type_for_map (btf, map_ident);
@@ -517,7 +534,7 @@ static void codegen_asserts(struct bpf_object *obj, const char *obj_name) ", obj_name);bpf_object__for_each_map (map, obj) { -if (!is_mmapable_map (map, map_ident,sizeof (map_ident))) +if (!is_skel_data (map, map_ident,sizeof (map_ident)))continue ; sec =find_type_for_map (btf, map_ident);
@@ -668,8 +685,7 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name)bpf_object__for_each_map (map, obj) {if (!get_map_ident (map, ident,sizeof (ident)))continue ; -if (bpf_map__is_internal (map) && - (bpf_map__map_flags (map) &BPF_F_MMAPABLE )) +if (is_skel_data (map, ident,sizeof (ident)))printf ("\tskel_free_map_data(skel->%1$s, skel->maps.%1$s.initial_value, %2$zu);\n" , ident,bpf_map_mmap_sz (map)); codegen("\
@@ -741,7 +757,7 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *hconst void *mmap_data =NULL ;size_t mmap_size =0 ; -if (!is_mmapable_map (map, ident,sizeof (ident))) +if (!is_skel_data (map, ident,sizeof (ident)))continue ; codegen("\
@@ -849,9 +865,23 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *hbpf_object__for_each_map (map, obj) {const char *mmap_flags; -if (!is_mmapable_map (map, ident,sizeof (ident))) +if (!is_skel_data (map, ident,sizeof (ident)))continue ; +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) { +codegen ("\ +\n\ +err = skel_protect_map_data(skel->%1$s, &skel->maps.%1$s.initial_value, %2$zd);\n\ +if (err) \n\ +return err; \n\ +#ifdef __KERNEL__ \n\ +skel->%1$s = NULL; \n\ +#endif \n\ +" , + ident,bpf_map_mmap_sz (map)); +continue ; + } +if (bpf_map__map_flags (map) &BPF_F_RDONLY_PROG ) mmap_flags ="PROT_READ" ;else
@@ -955,8 +985,7 @@ codegen_maps_skeleton(struct bpf_object *obj, size_t map_cnt, bool mmaped, bool map->map = &obj->maps.%s; \n\ ", i, bpf_map__name(map), ident); -/* memory-mapped internal maps */ -if (mmaped &&is_mmapable_map (map, ident,sizeof (ident))) { +if (mmaped &&is_skel_data (map, ident,sizeof (ident))) {printf ("\tmap->mmaped = (void **)&obj->%s;\n" , ident); }
tools/lib/bpf/skel_internal.h +22 -2
@@ -131,8 +131,10 @@ static inline void skel_free_map_data(void *p, __u64 addr, size_t sz) {if (addr != ~0ULL )kvfree (p); -/* When addr == ~0ULL the 'p' points to -* ((struct bpf_array *)map)->value. See skel_finalize_map_data. +/* +* When addr == ~0ULL the init buffer has already been released. +* For skel_finalize_map_data(), 'p' points to +* ((struct bpf_array *)map)->value. */ }
@@ -170,6 +172,15 @@ static inline void *skel_finalize_map_data(__u64 *init_val, size_t mmap_sz, int return addr; } +static inline int skel_protect_map_data (void *p,__u64 *init_val,size_t sz) +{ + (void )sz; + +kvfree (p); + *init_val = ~0ULL ; +return 0 ; +} +#else static inline void *skel_alloc (size_t size)
@@ -208,6 +219,15 @@ static inline void *skel_finalize_map_data(__u64 *init_val, size_t mmap_sz, intreturn NULL ; return addr; } + +static inline int skel_protect_map_data (void *p,__u64 *init_val,size_t sz) +{ + (void )init_val; + +if (mprotect (p, sz,PROT_READ )) +return -errno; +return 0 ; +}#endif static inline int skel_closenz (int fd)
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v12 06/10] bpftool: Generate skeleton for global percpu data
The comment here says "only generate definitions for memory-mapped internal maps," but is_skel_data() now also returns true for BPF_MAP_TYPE_PERCPU_ARRAY maps. Those percpu maps are never…
Quoted 4 lines
tools/bpf/bpftool/gen.c +0
-0
[ ... ]
Quoted 37 lines
tools/bpf/bpftool/gen.c +14
-3@@ -254,7 +260,7 @@ static const struct btf_type *find_type_for_map(struct btf *btf, const char *mapreturn NULL ; } -static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +static bool is_skel_data (const struct bpf_map *map,char *buf,size_t sz) {size_t tmp_sz;@@ -263,13 +269,24 @@ static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz)return true; } -if (!bpf_map__is_internal (map) || !(bpf_map__map_flags (map) &BPF_F_MMAPABLE )) +if (!bpf_map__is_internal (map))return false;if (!get_map_ident (map, buf, sz))return false; -return true; +if (bpf_map__map_flags (map) &BPF_F_MMAPABLE ) +return true; + +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) +return bpf_map__btf_value_type_id (map) !=0 ; + +return false; +} + +static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +{ +return is_skel_data (map, buf, sz) &&bpf_map__type (map) !=BPF_MAP_TYPE_PERCPU_ARRAY ; }static int codegen_datasecs (struct bpf_object *obj,const char *obj_name)
tools/bpf/bpftool/gen.c static int codegen_datasecs(struct bpf_object *obj, const char *obj_name) · line 308
Quoted 6 lines
tools/bpf/bpftool/gen.c +1
-1@@ -287,7 +304,7 @@ static int codegen_datasecs(struct bpf_object *obj, const char *obj_name)bpf_object__for_each_map (map, obj) {/* only generate definitions for memory-mapped internal maps */ -if (!is_mmapable_map (map, map_ident,sizeof (map_ident))) +if (!is_skel_data (map, map_ident,sizeof (map_ident)))continue ;
The comment here says "only generate definitions for memory-mapped internal maps," but is_skel_data() now also returns true for BPF_MAP_TYPE_PERCPU_ARRAY maps. Those percpu maps are never
memory-mapped: map_is_mmapable() in tools/lib/bpf/libbpf.c returns false for LIBBPF_MAP_PERCPU, which is why this patch had to add mprotect() for the init buffer rather than relying on shared mmap.
When you removed the identical comment from codegen_maps_skeleton() (line 988), should this one have been updated to say "skeleton data maps" or "internal maps exposed as skeleton data" to match what is_skel_data() actually checks?
The same wording also appears at line 534 in codegen_asserts():
Quoted 4 lines
tools/bpf/bpftool/gen.c +1
-1bpf_object__for_each_map (map, obj) { -if (!is_mmapable_map (map, map_ident,sizeof (map_ident))) +if (!is_skel_data (map, map_ident,sizeof (map_ident)))continue;
where the comment above (line 513) says "Emit type size asserts for all top-level fields in memory-mapped internal maps." That comment now covers percpu maps too.
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 06/10] bpftool: Generate skeleton for global percpu data
Quoted 58 lines · On 14/8/26 00:26, [email protected] wrote:
Quoted 4 lines
tools/bpf/bpftool/gen.c +0
-0[ ... ]
Quoted 37 lines
tools/bpf/bpftool/gen.c +14
-3@@ -254,7 +260,7 @@ static const struct btf_type *find_type_for_map(struct btf *btf, const char *mapreturn NULL ; } -static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +static bool is_skel_data (const struct bpf_map *map,char *buf,size_t sz) {size_t tmp_sz;@@ -263,13 +269,24 @@ static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz)return true; } -if (!bpf_map__is_internal (map) || !(bpf_map__map_flags (map) &BPF_F_MMAPABLE )) +if (!bpf_map__is_internal (map))return false;if (!get_map_ident (map, buf, sz))return false; -return true; +if (bpf_map__map_flags (map) &BPF_F_MMAPABLE ) +return true; + +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) +return bpf_map__btf_value_type_id (map) !=0 ; + +return false; +} + +static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +{ +return is_skel_data (map, buf, sz) &&bpf_map__type (map) !=BPF_MAP_TYPE_PERCPU_ARRAY ; }static int codegen_datasecs (struct bpf_object *obj,const char *obj_name)tools/bpf/bpftool/gen.c static int codegen_datasecs(struct bpf_object *obj, const char *obj_name) · line 308
Quoted 6 lines
tools/bpf/bpftool/gen.c +1
-1@@ -287,7 +304,7 @@ static int codegen_datasecs(struct bpf_object *obj, const char *obj_name)bpf_object__for_each_map (map, obj) {/* only generate definitions for memory-mapped internal maps */ -if (!is_mmapable_map (map, map_ident,sizeof (map_ident))) +if (!is_skel_data (map, map_ident,sizeof (map_ident)))continue ;The comment here says "only generate definitions for memory-mapped internal maps," but
is_skel_data()now also returns true forBPF_MAP_TYPE_PERCPU_ARRAYmaps. Those percpu maps are never
memory-mapped:map_is_mmapable()intools/lib/bpf/libbpf.creturns false forLIBBPF_MAP_PERCPU, which is why this patch had to addmprotect()for the init buffer rather than relying on shared mmap.
Quoted 6 lines
When you removed the identical comment from
codegen_maps_skeleton()(line 988), should this one have been updated to say "skeleton data maps" or "internal maps exposed as skeleton data" to match whatis_skel_data()actually checks?The same wording also appears at line 534 in
codegen_asserts():
Will update the comments.
Quoted 9 lines
Quoted 5 lines
tools/bpf/bpftool/gen.c +1
-1bpf_object__for_each_map (map, obj) { -if (!is_mmapable_map (map, map_ident,sizeof (map_ident))) +if (!is_skel_data (map, map_ident,sizeof (map_ident)))continue;
where the comment above (line 513) says "Emit type size asserts for all top-level fields in memory-mapped internal maps." That comment now covers percpu maps too.
Will update the comment to "Emit type size asserts for all top-level fields in skel data maps.".
Thanks,
Leon
Quoted 1 line
Re: [PATCH bpf-next v12 06/10] bpftool: Generate skeleton for global percpu data
In reply to Leon HwangQuoted 50 lines · Leon Hwang
Enhance bpftool to generate skeletons that properly handle global percpu variables. The generated skeleton now includes a dedicated structure for percpu data, allowing users to initialize and access percpu variables more efficiently.
For global percpu variables, the skeleton now includes a nested structure, e.g.:
struct test_global_percpu_data { struct bpf_object_skeleton *skeleton; struct bpf_object *obj; struct { struct bpf_map *percpu; } maps; // ... struct test_global_percpu_data__percpu { int data; char run; struct { char set; int i; int nums[7]; } struct_data; int nums[7]; } *percpu; // ... };* The "struct
test_global_percpu_data__percpu*percpu" points toinitialized data, which is actually "maps.percpu->mmaped". * Before loading the skeleton, updating the "struct test_global_percpu_data__percpu *percpu" modifies the initial value of the corresponding global percpu variables. * After loading the skeleton, "maps.percpu->mmaped" has been marked as read-only in libbpf. If users want to update the global percpu variables, they have to update the "maps.percpu" map instead. * For lightweight skeleton, "lskel->percpu" will be protected by "mprotect(p, sz, PROT_READ)".* For subskeleton, those variables of global percpu data will be skipped.
Acked-by Quentin Monnet <[email protected]>Assisted-by Codex:gpt-5.5-xhighSigned-off-by Leon Hwang <[email protected]>--- tools/bpf/bpftool/gen.c | 53 +++++++++++++++++++++++++++-------- tools/lib/bpf/skel_internal.h | 24 ++++++++++++++-- 2 files changed, 63 insertions(+), 14 deletions(-)
[...]
Quoted 7 lines
tools/bpf/bpftool/gen.c +1
-1@@ -263,13 +269,24 @@ static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz)return true; } -if (!bpf_map__is_internal (map) || !(bpf_map__map_flags (map) &BPF_F_MMAPABLE )) +if (!bpf_map__is_internal (map))return false;
tools/bpf/bpftool/gen.c static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz) · line 282
Quoted 10 lines
tools/bpf/bpftool/gen.c +5
-1if (!get_map_ident (map, buf, sz))return false; -return true; +if (bpf_map__map_flags (map) &BPF_F_MMAPABLE ) +return true; + +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) +return bpf_map__btf_value_type_id (map) !=0 ;
why do we need this btf_value_type_id() check? in what case it's expected that there will be no BTF info set?
Quoted 9 lines
tools/bpf/bpftool/gen.c +7
-0+ +return false; +} + +static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +{ +return is_skel_data (map, buf, sz) &&bpf_map__type (map) !=BPF_MAP_TYPE_PERCPU_ARRAY ; }
[...]
Re: [PATCH bpf-next v12 06/10] bpftool: Generate skeleton for global percpu data
Quoted 1 line · Andrii Nakryiko
On Thu, Aug 13, 2026 at 8:24 AM Leon Hwang <[email protected]> wrote:
[...]
Quoted 11 lines
Quoted 11 lines
tools/bpf/bpftool/gen.c +1
-1@@ -263,13 +269,24 @@ static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz)return true; } -if (!bpf_map__is_internal (map) || !(bpf_map__map_flags (map) &BPF_F_MMAPABLE )) +if (!bpf_map__is_internal (map))return false;if (!get_map_ident (map, buf, sz))return false;
Quoted 9 lines
tools/bpf/bpftool/gen.c static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz) · line 282
Quoted 6 lines
tools/bpf/bpftool/gen.c +5
-1-return true; +if (bpf_map__map_flags (map) &BPF_F_MMAPABLE ) +return true; + +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) +return bpf_map__btf_value_type_id (map) !=0 ;why do we need this
btf_value_type_id()check? in what case it's expected that there will be no BTF info set?
Add this check by referencing btf_value_type_id check in libbpf's map_is_mmapable(). That said, if btf_value_type_id is 0 for global/percpu data, is_mmapable_map() should return false for them.
And, if the bpf obj is compiled without -g, this check helps to avoid generating struct for percpu data explicitly.
Thanks,
Leon
Quoted 12 lines
Quoted 10 lines
tools/bpf/bpftool/gen.c +7
-0+ +return false; +} + +static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +{ +return is_skel_data (map, buf, sz) &&bpf_map__type (map) !=BPF_MAP_TYPE_PERCPU_ARRAY ; }[...]
Re: [PATCH bpf-next v12 06/10] bpftool: Generate skeleton for global percpu data
Quoted 25 lines · Leon Hwang
Quoted 1 line · Andrii Nakryiko
On Thu, Aug 13, 2026 at 8:24 AM Leon Hwang <[email protected]> wrote:
[...]
Quoted 11 lines
Quoted 11 lines
tools/bpf/bpftool/gen.c +1
-1@@ -263,13 +269,24 @@ static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz)return true; } -if (!bpf_map__is_internal (map) || !(bpf_map__map_flags (map) &BPF_F_MMAPABLE )) +if (!bpf_map__is_internal (map))return false;if (!get_map_ident (map, buf, sz))return false;Quoted 9 lines
tools/bpf/bpftool/gen.c static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz) · line 282
Quoted 6 lines
tools/bpf/bpftool/gen.c +5
-1-return true; +if (bpf_map__map_flags (map) &BPF_F_MMAPABLE ) +return true; + +if (bpf_map__type (map) ==BPF_MAP_TYPE_PERCPU_ARRAY ) +return bpf_map__btf_value_type_id (map) !=0 ;why do we need this
btf_value_type_id()check? in what case it's expected that there will be no BTF info set?
Quoted 7 lines
Add this check by referencing
btf_value_type_idcheck in libbpf'smap_is_mmapable(). That said, ifbtf_value_type_idis 0 for global/percpu data,is_mmapable_map()should return false for them.And, if the bpf obj is compiled without -g, this check helps to avoid generating struct for percpu data explicitly.
I don't think skeleton can be generated if BPF object was compiled without -g, so let's just drop that btf_value_type_id() check, it causes more questions than it's actually useful
Quoted 16 lines
Thanks,
LeonQuoted 12 lines
Quoted 10 lines
tools/bpf/bpftool/gen.c +7
-0+ +return false; +} + +static bool is_mmapable_map (const struct bpf_map *map,char *buf,size_t sz) +{ +return is_skel_data (map, buf, sz) &&bpf_map__type (map) !=BPF_MAP_TYPE_PERCPU_ARRAY ; }[...]
PATCH v12 7/10 selftests/bpf: Add tests to verify global percpu data
In reply to Leon HwangIf the arch, like s390x, does not support percpu insn, these cases won't test global percpu data by checking FEAT_PERCPU_DATA support.
The following APIs have been tested for global percpu data:
1. bpf_map__set_initial_value()
2. bpf_map__initial_value()
3. bpf_map__set_value_size()
4. generated percpu struct pointer pointing to internal map's mmaped data
5. bpf_map__lookup_elem() for global percpu data map
6. bpf_map_lookup_elem_flags() for global percpu data map
At the same time, the case is also tested with 'bpftool gen skeleton -L'.
--- tools/testing/selftests/bpf/Makefile | 2 +- .../bpf/prog_tests/global_data_init.c | 188 ++++++++++++++++++ .../bpf/progs/test_global_percpu_data.c | 41 ++++ 3 files changed, 230 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/test_global_percpu_data.c
tools/testing/selftests/bpf/Makefile +1 -1
@@ -531,7 +531,7 @@ LSKELS_SIGNED := fentry_test.c fexit_test.c atomics.c # Generate both light skeleton and libbpf skeleton for these LSKELS_EXTRA := test_ksyms_module.c test_ksyms_weak.c kfunc_call_test.c \ - kfunc_call_test_subprog.c + kfunc_call_test_subprog.c test_global_percpu_data.c SKEL_BLACKLIST += $$(LSKELS) $$(LSKELS_SIGNED) test_static_linked.skel.h-deps := test_static_linked1.bpf.o test_static_linked2.bpf.o
tools/testing/selftests/bpf/prog_tests/global_data_init.c +188 -0
196 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_global_percpu_data.c new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v12 07/10] selftests/bpf: Add tests to verify global percpu data
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The test `test_global_percpu_data` fails because `bpf_map__set_value_size()` clears the…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The test test_global_percpu_data fails because bpf_map__set_value_size() clears the .percpu map's BTF type ID when it encounters a non-array as the last variable in the datasec. --
--- Patch [7]: [PATCH bpf-next v12 07/10] selftests/bpf: Add tests to verify global percpu data --- commit 0224e1ae61dc47271ec8a72ac29b073935a25d31
Author: Leon Hwang <[email protected]>
selftests/bpf: Add tests to verify global percpu data
This patch adds a selftest to verify global percpu data map initialization and modification using APIs like bpf_map__set_value_size() and bpf_map_lookup_elem_flags(). It includes both light and standard skeleton tests.
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0
-0
[ ... ]
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2
-0+static void test_global_percpu_data_init (void ) +{
[ ... ]
tools/testing/selftests/bpf/prog_tests/global_data_init.c void test_global_data_init(void) · line 199
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +7
-0+ err =bpf_map__set_value_size (map, desired_sz); +if (!ASSERT_OK (err,"bpf_map__set_value_size" )) + goto out; +if (!ASSERT_EQ (bpf_map__value_size (map), desired_sz,"percpu value size" )) + goto out; +if (!ASSERT_NEQ (bpf_map__btf_value_type_id (map),0 ,"percpu BTF value type" )) + goto out;
[Severity: Medium]
Will this test fail here because bpf_map__set_value_size() clears the BTF type ID?
When bpf_map__set_value_size() is called, it attempts to resize the BTF datasec. This requires the last variable in the datasec to be an array. Because struct_data is defined last in the .percpu section in the BPF program, map_btf_datasec_resize() returns -EINVAL and clears the BTF type ID to 0, causing this assertion to fail.
Quoted 5 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c new file +0
-0
[ ... ]
tools/testing/selftests/bpf/progs/test_global_percpu_data.c line 24
Quoted 10 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c new file +10
-0+int nums[7] SEC(".percpu"); +bool set SEC(".percpu") = false; +struct { +char set; +int i; +int nums[7 ]; +} struct_data SEC(".percpu") = { + .set = 0, + .i = -1, +};
[Severity: Medium]
Does placing this struct at the end of the .percpu section break the BTF resizing logic?
Could the variables in the .percpu section be reordered so that an array like nums or cpu_id is the last variable?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Re: [PATCH bpf-next v12 07/10] selftests/bpf: Add tests to verify global percpu data
Quoted 62 lines · On 13/8/26 23:42, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The testtest_global_percpu_datafails becausebpf_map__set_value_size()clears the.percpumap's BTF type ID when it encounters a non-array as the last variable in the datasec. ----- Patch [7]: [PATCH bpf-next v12 07/10] selftests/bpf: Add tests to verify global percpu data --- commit
0224e1ae61dc47271ec8a72ac29b073935a25d31
Author: Leon Hwang <[email protected]>selftests/bpf: Add tests to verify global percpu data
This patch adds a selftest to verify global percpu data map initialization and modification using APIs like
bpf_map__set_value_size()andbpf_map_lookup_elem_flags(). It includes both light and standard skeleton tests.Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0
-0[ ... ]
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2
-0+static void test_global_percpu_data_init (void ) +{[ ... ]
tools/testing/selftests/bpf/prog_tests/global_data_init.c void test_global_data_init(void) · line 199
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +7
-0+ err =bpf_map__set_value_size (map, desired_sz); +if (!ASSERT_OK (err,"bpf_map__set_value_size" )) + goto out; +if (!ASSERT_EQ (bpf_map__value_size (map), desired_sz,"percpu value size" )) + goto out; +if (!ASSERT_NEQ (bpf_map__btf_value_type_id (map),0 ,"percpu BTF value type" )) + goto out;[Severity: Medium]
Will this test fail here becausebpf_map__set_value_size()clears the BTF type ID?When
bpf_map__set_value_size()is called, it attempts to resize the BTF datasec. This requires the last variable in the datasec to be an array. Becausestruct_datais defined last in the.percpusection in the BPF program,map_btf_datasec_resize()returns-EINVALand clears the BTF type ID to 0, causing this assertion to fail.Quoted 5 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c new file +0
-0[ ... ]
tools/testing/selftests/bpf/progs/test_global_percpu_data.c line 24
Quoted 10 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c new file +10
-0+int nums[7] SEC(".percpu"); +bool set SEC(".percpu") = false; +struct { +char set; +int i; +int nums[7 ]; +} struct_data SEC(".percpu") = { + .set = 0, + .i = -1, +};[Severity: Medium]
Does placing this struct at the end of the.percpusection break the BTF resizing logic?
Quoted 3 lines
Could the variables in the
.percpusection be reordered so that an array likenumsorcpu_idis the last variable?
I think I should use a dedicated section to exercise bpf_map__set_value_size(), e.g.
int arr[1] SEC(".percpu.arr");
Hence, bpf_map__set_value_size() won't fail with new value size 'sizeof(int) * 2'.
Thanks,
Leon
[email protected]
AI reviewRe: [PATCH bpf-next v12 07/10] selftests/bpf: Add tests to verify global percpu data
In reply to Leon HwangThis isn't a bug, but does the raw_tp ctx need to be passed here, given the program ignores it?
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0
-0
[ ... ]
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +3
-0@@ -60,3 +63,188 @@ void test_global_data_init(void)free (newval);bpf_object__close (obj); } + +static void test_percpu_data_on_cpus (struct bpf_map *map,int map_fd,int prog_fd,int *runp) +{
tools/testing/selftests/bpf/prog_tests/global_data_init.c void test_global_data_init(void) · line 78
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +10
-0+struct test_global_percpu_data__percpu *data =NULL ; +int i, err, key =0 , num_online, run =0 ; +__u64 args[2 ] = {0x1234ULL ,0x5678ULL }; +size_t data_sz; +bool *online; +LIBBPF_OPTS (bpf_test_run_opts, topts, + .ctx_in = args, + .ctx_size_in = sizeof(args), + .flags = BPF_F_TEST_RUN_ON_CPU, + );
This isn't a bug, but does the raw_tp ctx need to be passed here, given the program ignores it?
[ ... ]
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +9
-0+static void test_global_percpu_data_init (void ) +{ +struct test_global_percpu_data__percpu init_value = {}; +struct test_global_percpu_data__percpu *init_data; +const __u32 desired_sz =sysconf (_SC_PAGE_SIZE); +struct test_global_percpu_data *skel =NULL ; +size_t init_data_sz; +struct bpf_map *map; +int prog_fd, err;
[ ... ]
Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +11
-0+ map = skel->maps.percpu; +if (!ASSERT_EQ (bpf_map__type (map),BPF_MAP_TYPE_PERCPU_ARRAY ,"bpf_map__type" )) + goto out; + + init_value.data =2 ; + init_value.nums[6 ] =-1 ; + init_value.struct_data.i =2 ; + init_value.struct_data.nums[6 ] =-1 ; + err =bpf_map__set_initial_value (map, &init_value,sizeof (init_value)); +if (!ASSERT_OK (err,"bpf_map__set_initial_value" )) + goto out;
tools/testing/selftests/bpf/prog_tests/global_data_init.c void test_global_data_init(void) · line 182
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +10
-0+ + init_data =bpf_map__initial_value (map, &init_data_sz); +if (!ASSERT_OK_PTR (init_data,"bpf_map__initial_value" )) + goto out; + +ASSERT_EQ (init_data->data, init_value.data,"init_value data" ); +ASSERT_EQ (init_data->set, init_value.set,"init_value set" ); +ASSERT_EQ (init_data->struct_data.i, init_value.struct_data.i,"init_value struct_data.i" ); +ASSERT_EQ (init_data->struct_data.nums[6 ], init_value.struct_data.nums[6 ], +"init_value struct_data.nums[6]" );
This isn't a bug, but should init_value.set be given a non-zero value, and nums[6] checked, so these assertions can actually distinguish a broken copy?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 07/10] selftests/bpf: Add tests to verify global percpu data
Quoted 19 lines · On 14/8/26 00:26, [email protected] wrote:
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0
-0[ ... ]
Quoted 12 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +8
-0@@ -60,3 +63,188 @@ void test_global_data_init(void)free (newval);bpf_object__close (obj); } + +static void test_percpu_data_on_cpus (struct bpf_map *map,int map_fd,int prog_fd,int *runp) +{ +struct test_global_percpu_data__percpu *data =NULL ; +int i, err, key =0 , num_online, run =0 ; +__u64 args[2 ] = {0x1234ULL ,0x5678ULL }; +size_t data_sz; +bool *online;
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c void test_global_data_init(void) · line 78
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +5
-0+LIBBPF_OPTS (bpf_test_run_opts, topts, + .ctx_in = args, + .ctx_size_in = sizeof(args), + .flags = BPF_F_TEST_RUN_ON_CPU, + );This isn't a bug, but does the
raw_tpctx need to be passed here, given the program ignores it?
Good catch.
Will drop the 'args'.
Quoted 32 lines
[ ... ]
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +9
-0+static void test_global_percpu_data_init (void ) +{ +struct test_global_percpu_data__percpu init_value = {}; +struct test_global_percpu_data__percpu *init_data; +const __u32 desired_sz =sysconf (_SC_PAGE_SIZE); +struct test_global_percpu_data *skel =NULL ; +size_t init_data_sz; +struct bpf_map *map; +int prog_fd, err;[ ... ]
Quoted 17 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +17
-0+ map = skel->maps.percpu; +if (!ASSERT_EQ (bpf_map__type (map),BPF_MAP_TYPE_PERCPU_ARRAY ,"bpf_map__type" )) + goto out; + + init_value.data =2 ; + init_value.nums[6 ] =-1 ; + init_value.struct_data.i =2 ; + init_value.struct_data.nums[6 ] =-1 ; + err =bpf_map__set_initial_value (map, &init_value,sizeof (init_value)); +if (!ASSERT_OK (err,"bpf_map__set_initial_value" )) + goto out; + + init_data =bpf_map__initial_value (map, &init_data_sz); +if (!ASSERT_OK_PTR (init_data,"bpf_map__initial_value" )) + goto out; + +ASSERT_EQ (init_data->data, init_value.data,"init_value data" );
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c void test_global_data_init(void) · line 182
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +4
-0+ASSERT_EQ (init_data->set, init_value.set,"init_value set" ); +ASSERT_EQ (init_data->struct_data.i, init_value.struct_data.i,"init_value struct_data.i" ); +ASSERT_EQ (init_data->struct_data.nums[6 ], init_value.struct_data.nums[6 ], +"init_value struct_data.nums[6]" );This isn't a bug, but should init_value.set be given a non-zero value, and nums[6] checked, so these assertions can actually distinguish a broken copy?
Will update 'init_value.set = 1;' and check 'nums[6]'.
Thanks,
Leon
PATCH v12 8/10 selftests/bpf: Test direct reading/writing read-only percpu_array map
In reply to Leon HwangVerify these two cases:
1. Direct reading the data of read-only percpu data's percpu_array map is allowed.
2. Direct writing the data of read-only percpu data's percpu_array map is disallowed.
--- .../bpf/prog_tests/global_data_init.c | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +90 -0
@@ -236,6 +236,92 @@ static void test_global_percpu_data_lskel(void)test_global_percpu_data_lskel__destroy (lskel); } +static int create_rdonly_percpu_array (void ) +{ +LIBBPF_OPTS (bpf_map_create_opts, map_opts, + .map_flags = BPF_F_RDONLY_PROG, + ); +int key =0 , map_fd, err; +__u64 value =0 ; + + map_fd =bpf_map_create (BPF_MAP_TYPE_PERCPU_ARRAY ,"percpu_ro_map" ,sizeof (int ), +sizeof (__u64),1 , &map_opts); +if (!ASSERT_GE (map_fd,0 ,"bpf_map_create" )) +return -1 ; + + err =bpf_map_update_elem (map_fd, &key, &value,BPF_F_ALL_CPUS ); +if (!ASSERT_OK (err,"bpf_map_update_elem" )) + goto out; + + err =bpf_map_freeze (map_fd); +if (!ASSERT_OK (err,"bpf_map_freeze" )) + goto out; + +return map_fd; + +out: +close (map_fd); +return -1 ; +} + +static void test_global_percpu_data_rdonly_direct_read (void ) +{ +/* +* Raw instructions with manually prepared rdonly percpu_array map +* for testing direct-read global percpu data, because libbpf +* doesn't have rdonly internal percpu_array map support for +* global percpu data. +*/ +struct bpf_insn insns[] = { +BPF_LD_MAP_VALUE (BPF_REG_1 ,0 ,0 ), +BPF_LDX_MEM (BPF_DW ,BPF_REG_0 ,BPF_REG_1 ,0 ), +BPF_EXIT_INSN (), + }; +int map_fd, prog_fd; + + map_fd =create_rdonly_percpu_array (); +if (map_fd <0 ) +return ; + + insns[0 ].imm = map_fd; + prog_fd =bpf_prog_load (BPF_PROG_TYPE_SOCKET_FILTER ,"percpu_ro_prog" ,"GPL" , insns, +ARRAY_SIZE (insns),NULL ); +if (ASSERT_GE (prog_fd,0 ,"bpf_prog_load" )) +close (prog_fd); +close (map_fd); +} + +static void test_global_percpu_data_rdonly_direct_write (void ) +{ +LIBBPF_OPTS (bpf_prog_load_opts, prog_opts); +/* See the comment in test_global_percpu_data_rdonly_direct_read() */ +struct bpf_insn insns[] = { +BPF_LD_MAP_VALUE (BPF_REG_1 ,0 ,0 ), +BPF_LDX_MEM (BPF_DW ,BPF_REG_0 ,BPF_REG_1 ,0 ), +BPF_ST_MEM (BPF_DW ,BPF_REG_1 ,0 ,0 ), +BPF_EXIT_INSN (), + }; +char log_buf[256 ] = {}; +int map_fd, prog_fd; + + prog_opts.log_buf = log_buf; + prog_opts.log_size =sizeof (log_buf); + prog_opts.log_level =1 ; + + map_fd =create_rdonly_percpu_array (); +if (map_fd <0 ) +return ; + + insns[0 ].imm = map_fd; + prog_fd =bpf_prog_load (BPF_PROG_TYPE_SOCKET_FILTER ,"percpu_ro_prog" ,"GPL" , insns, +ARRAY_SIZE (insns), &prog_opts); +if (!ASSERT_LT (prog_fd,0 ,"bpf_prog_load" )) +close (prog_fd); +else +ASSERT_HAS_SUBSTR (log_buf,"write into map forbidden" ,"verifier log" ); +close (map_fd); +} +void test_global_percpu_data (void ) {if (!feat_supported (NULL ,FEAT_PERCPU_DATA )) {
@@ -247,4 +333,8 @@ void test_global_percpu_data(void)test_global_percpu_data_init ();if (test__start_subtest ("lskel" ))test_global_percpu_data_lskel (); +if (test__start_subtest ("rdonly_direct_read" )) +test_global_percpu_data_rdonly_direct_read (); +if (test__start_subtest ("rdonly_direct_write" )) +test_global_percpu_data_rdonly_direct_write (); }
Signature · 1 line
2.55.0
PATCH v12 9/10 selftests/bpf: Test verifier log for global percpu data
In reply to Leon HwangAdd two tests to verify the verifier log
"R%d points to percpu_array map which cannot be used as const string\n".
--- .../bpf/prog_tests/global_data_init.c | 6 +++++ .../bpf/progs/test_global_percpu_data.c | 23 +++++++++++++++++++ 2 files changed, 29 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +6 -0
@@ -322,6 +322,11 @@ static void test_global_percpu_data_rdonly_direct_write(void)close (map_fd); } +static void test_global_percpu_data_verifier_log (void ) +{ +RUN_TESTS (test_global_percpu_data); +} +void test_global_percpu_data (void ) {if (!feat_supported (NULL ,FEAT_PERCPU_DATA )) {
@@ -337,4 +342,5 @@ void test_global_percpu_data(void)test_global_percpu_data_rdonly_direct_read ();if (test__start_subtest ("rdonly_direct_write" ))test_global_percpu_data_rdonly_direct_write (); +test_global_percpu_data_verifier_log (); }
tools/testing/selftests/bpf/progs/test_global_percpu_data.c +23 -0
@@ -38,4 +38,27 @@ int update_percpu_data(void *ctx)return 0 ; } +static const char fmt[] SEC(".percpu.fmt") = "data %d\n"; + +SEC("?kprobe") +__failure __msg("R{{[0-9]+}} points to percpu_array map which cannot be used as const string") +int verifier_strncmp(void *ctx) +{ +return bpf_strncmp ("test" ,5 , fmt); +} + +SEC("?kprobe") +__failure __msg("R{{[0-9]+}} points to percpu_array map which cannot be used as const string") +int verifier_snprintf(void *ctx) +{ +u64 args[] = { data }; +char buf[128 ]; +int len; + + len =bpf_snprintf (buf,sizeof (buf), fmt, args,sizeof (args)); +if (len >0 ) +bpf_printk ("snprintf: %s\n" , buf); +return 0 ; +} + char _license[] SEC("license") = "GPL";
Signature · 1 line
2.55.0
PATCH v12 10/10 selftests/bpf: Verify bpf_iter for global percpu data
In reply to Leon HwangAdd a test to verify that it is OK to iter the percpu_array map used for global percpu data.
--- .../bpf/prog_tests/global_data_init.c | 52 +++++++++++++++++++ .../bpf/progs/test_global_percpu_data.c | 25 +++++++++ 2 files changed, 77 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +52 -0
@@ -327,6 +327,56 @@ static void test_global_percpu_data_verifier_log(void)RUN_TESTS (test_global_percpu_data); } +static void test_global_percpu_data_iter (void ) +{ +DECLARE_LIBBPF_OPTS (bpf_iter_attach_opts, opts); +struct test_global_percpu_data *skel; +union bpf_iter_link_info linfo = {}; +struct bpf_link *link =NULL ; +int fd, num_cpus, len, err; +char buf[16 ]; + + num_cpus =libbpf_num_possible_cpus (); +if (!ASSERT_GT (num_cpus,0 ,"libbpf_num_possible_cpus" )) +return ; + + skel =test_global_percpu_data__open (); +if (!ASSERT_OK_PTR (skel,"test_global_percpu_data__open" )) +return ; + + skel->rodata->num_cpus = num_cpus; + skel->rodata->offsetof_num = offsetof(struct test_global_percpu_data__percpu , struct_data); + skel->rodata->offsetof_num +=sizeof (skel->percpu->struct_data) -sizeof (int ); + skel->rodata->elem_sz =roundup (sizeof (struct test_global_percpu_data__percpu ),8 ); + skel->percpu->struct_data.nums[6 ] =0xc0de ; + + err =test_global_percpu_data__load (skel); +if (!ASSERT_OK (err,"test_global_percpu_data__load" )) + goto out; + + linfo.map.map_fd =bpf_map__fd (skel->maps.percpu); + opts.link_info = &linfo; + opts.link_info_len =sizeof (linfo); + link =bpf_program__attach_iter (skel->progs.dump_percpu_data, &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_iter" )) + goto out; + + fd =bpf_iter_create (bpf_link__fd (link)); +if (!ASSERT_GE (fd,0 ,"bpf_iter_create" )) + goto out; + +while ((len =read (fd, buf,sizeof (buf))) >0 ) +do { }while (0 ); +ASSERT_EQ (len,0 ,"read iter" ); +ASSERT_TRUE (skel->bss->run_iter,"run_iter" ); +ASSERT_EQ (skel->bss->percpu_data_sum,0xc0de * num_cpus,"percpu_data_sum" ); + +close (fd); +out: +bpf_link__destroy (link); +test_global_percpu_data__destroy (skel); +} +void test_global_percpu_data (void ) {if (!feat_supported (NULL ,FEAT_PERCPU_DATA )) {
@@ -343,4 +393,6 @@ void test_global_percpu_data(void)if (test__start_subtest ("rdonly_direct_write" ))test_global_percpu_data_rdonly_direct_write ();test_global_percpu_data_verifier_log (); +if (test__start_subtest ("iter" )) +test_global_percpu_data_iter (); }
tools/testing/selftests/bpf/progs/test_global_percpu_data.c +25 -0
@@ -61,4 +61,29 @@ int verifier_snprintf(void *ctx)return 0 ; } +volatile const __u32 num_cpus =0 ; +volatile const int offsetof_num; +volatile const int elem_sz; +__u32 percpu_data_sum =0 ; +bool run_iter = false; + +SEC("iter/bpf_map_elem") +__auxiliary +int dump_percpu_data (struct bpf_iter__bpf_map_elem *ctx) +{ +void *pptr = ctx->value; +int i; + +if (!pptr) +return 0 ; + + run_iter = true; + +for (i =0 ; i < num_cpus; i++) { + percpu_data_sum += *(int *) (pptr + offsetof_num); + pptr += elem_sz; + } +return 0 ; +} + char _license[] SEC("license") = "GPL";
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v12 10/10] selftests/bpf: Verify bpf_iter for global percpu data
This isn't a bug, but could these two lines collapse into a single offsetof(struct test_global_percpu_data__percpu, struct_data.nums[6]), so the offset names the field being read instead of…
Quoted 18 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +10
-0@@ -327,6 +327,56 @@ static void test_global_percpu_data_verifier_log(void)RUN_TESTS (test_global_percpu_data); } +static void test_global_percpu_data_iter (void ) +{ +DECLARE_LIBBPF_OPTS (bpf_iter_attach_opts, opts); +struct test_global_percpu_data *skel; +union bpf_iter_link_info linfo = {}; +struct bpf_link *link =NULL ; +int fd, num_cpus, len, err; +char buf[16 ]; + + num_cpus =libbpf_num_possible_cpus ();
tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_verifier_log(void) · line 349
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +10
-0+if (!ASSERT_GT (num_cpus,0 ,"libbpf_num_possible_cpus" )) +return ; + + skel =test_global_percpu_data__open (); +if (!ASSERT_OK_PTR (skel,"test_global_percpu_data__open" )) +return ; + + skel->rodata->num_cpus = num_cpus; + skel->rodata->offsetof_num = offsetof(struct test_global_percpu_data__percpu , struct_data); + skel->rodata->offsetof_num +=sizeof (skel->percpu->struct_data) -sizeof (int );
This isn't a bug, but could these two lines collapse into a single offsetof(struct test_global_percpu_data__percpu, struct_data.nums[6]), so the offset names the field being read instead of depending on nums[6] staying last in the struct?
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +10
-0+ skel->rodata->elem_sz =roundup (sizeof (struct test_global_percpu_data__percpu ),8 ); + skel->percpu->struct_data.nums[6 ] =0xc0de ; + + err =test_global_percpu_data__load (skel); +if (!ASSERT_OK (err,"test_global_percpu_data__load" )) + goto out; + + linfo.map.map_fd =bpf_map__fd (skel->maps.percpu); + opts.link_info = &linfo; + opts.link_info_len =sizeof (linfo);
tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_verifier_log(void) · line 369
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +10
-0+ link =bpf_program__attach_iter (skel->progs.dump_percpu_data, &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_iter" )) + goto out; + + fd =bpf_iter_create (bpf_link__fd (link)); +if (!ASSERT_GE (fd,0 ,"bpf_iter_create" )) + goto out; + +while ((len =read (fd, buf,sizeof (buf))) >0 ) +do { }while (0 );
This isn't a bug, but would a bare semicolon (as in bpf_iter.c's read-drain loops) be clearer here than do { } while (0)?
[ ... ]
tools/testing/selftests/bpf/progs/test_global_percpu_data.c int verifier_snprintf(void *ctx) · line 65
Quoted 10 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c +2
-0@@ -61,4 +61,29 @@ int verifier_snprintf(void *ctx)return 0 ; } +volatile const __u32 num_cpus =0 ; +volatile const int offsetof_num;
This isn't a bug, but would a name like nums6_off (set directly from offsetof(struct test_global_percpu_data__percpu, struct_data.nums[6])) make it clearer which field the iterator is summing?
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 10/10] selftests/bpf: Verify bpf_iter for global percpu data
Quoted 25 lines · On 14/8/26 00:26, [email protected] wrote:
Quoted 25 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +17
-0@@ -327,6 +327,56 @@ static void test_global_percpu_data_verifier_log(void)RUN_TESTS (test_global_percpu_data); } +static void test_global_percpu_data_iter (void ) +{ +DECLARE_LIBBPF_OPTS (bpf_iter_attach_opts, opts); +struct test_global_percpu_data *skel; +union bpf_iter_link_info linfo = {}; +struct bpf_link *link =NULL ; +int fd, num_cpus, len, err; +char buf[16 ]; + + num_cpus =libbpf_num_possible_cpus (); +if (!ASSERT_GT (num_cpus,0 ,"libbpf_num_possible_cpus" )) +return ; + + skel =test_global_percpu_data__open (); +if (!ASSERT_OK_PTR (skel,"test_global_percpu_data__open" )) +return ; +
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_verifier_log(void) · line 349
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +3
-0+ skel->rodata->num_cpus = num_cpus; + skel->rodata->offsetof_num = offsetof(struct test_global_percpu_data__percpu , struct_data); + skel->rodata->offsetof_num +=sizeof (skel->percpu->struct_data) -sizeof (int );This isn't a bug, but could these two lines collapse into a single
offsetof(struct test_global_percpu_data__percpu, struct_data.nums[6]), so the offset names the field being read instead of depending on nums[6] staying last in the struct?
Sure. Will update.
Quoted 16 lines
Quoted 16 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +15
-0+ skel->rodata->elem_sz =roundup (sizeof (struct test_global_percpu_data__percpu ),8 ); + skel->percpu->struct_data.nums[6 ] =0xc0de ; + + err =test_global_percpu_data__load (skel); +if (!ASSERT_OK (err,"test_global_percpu_data__load" )) + goto out; + + linfo.map.map_fd =bpf_map__fd (skel->maps.percpu); + opts.link_info = &linfo; + opts.link_info_len =sizeof (linfo); + link =bpf_program__attach_iter (skel->progs.dump_percpu_data, &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_iter" )) + goto out; + + fd =bpf_iter_create (bpf_link__fd (link));
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_verifier_log(void) · line 369
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +5
-0+if (!ASSERT_GE (fd,0 ,"bpf_iter_create" )) + goto out; + +while ((len =read (fd, buf,sizeof (buf))) >0 ) +do { }while (0 );This isn't a bug, but would a bare semicolon (as in bpf_iter.c's read-drain loops) be clearer here than do { } while (0)?
Prefer 'do { } while (0);' instead of a dangling ';'.
Quoted 11 lines
[ ... ]
Quoted 8 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c +0
-0@@ -61,4 +61,29 @@ int verifier_snprintf(void *ctx)return 0 ; }
Quoted 6 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c int verifier_snprintf(void *ctx) · line 65
Quoted 2 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c +2
-0+volatile const __u32 num_cpus =0 ; +volatile const int offsetof_num;This isn't a bug, but would a name like
nums6_off(set directly fromoffsetof(struct test_global_percpu_data__percpu, struct_data.nums[6])) make it clearer which field the iterator is summing?
'num_off' would be better.
In iter, it should not know the concrete number. Instead, it identifies the number by offset 'num_off'.
Thanks,
Leon
Quoted 9 lines
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858
Re: [PATCH bpf-next v12 10/10] selftests/bpf: Verify bpf_iter for global percpu data
Quoted 63 lines · Leon Hwang
Quoted 25 lines · On 14/8/26 00:26, [email protected] wrote:
Quoted 25 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +17
-0@@ -327,6 +327,56 @@ static void test_global_percpu_data_verifier_log(void)RUN_TESTS (test_global_percpu_data); } +static void test_global_percpu_data_iter (void ) +{ +DECLARE_LIBBPF_OPTS (bpf_iter_attach_opts, opts); +struct test_global_percpu_data *skel; +union bpf_iter_link_info linfo = {}; +struct bpf_link *link =NULL ; +int fd, num_cpus, len, err; +char buf[16 ]; + + num_cpus =libbpf_num_possible_cpus (); +if (!ASSERT_GT (num_cpus,0 ,"libbpf_num_possible_cpus" )) +return ; + + skel =test_global_percpu_data__open (); +if (!ASSERT_OK_PTR (skel,"test_global_percpu_data__open" )) +return ; +Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_verifier_log(void) · line 349
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +3
-0+ skel->rodata->num_cpus = num_cpus; + skel->rodata->offsetof_num = offsetof(struct test_global_percpu_data__percpu , struct_data); + skel->rodata->offsetof_num +=sizeof (skel->percpu->struct_data) -sizeof (int );This isn't a bug, but could these two lines collapse into a single
offsetof(struct test_global_percpu_data__percpu, struct_data.nums[6]), so the offset names the field being read instead of depending on nums[6] staying last in the struct?Sure. Will update.
Quoted 16 lines
Quoted 16 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +15
-0+ skel->rodata->elem_sz =roundup (sizeof (struct test_global_percpu_data__percpu ),8 ); + skel->percpu->struct_data.nums[6 ] =0xc0de ; + + err =test_global_percpu_data__load (skel); +if (!ASSERT_OK (err,"test_global_percpu_data__load" )) + goto out; + + linfo.map.map_fd =bpf_map__fd (skel->maps.percpu); + opts.link_info = &linfo; + opts.link_info_len =sizeof (linfo); + link =bpf_program__attach_iter (skel->progs.dump_percpu_data, &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_iter" )) + goto out; + + fd =bpf_iter_create (bpf_link__fd (link));Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_verifier_log(void) · line 369
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +5
-0+if (!ASSERT_GE (fd,0 ,"bpf_iter_create" )) + goto out; + +while ((len =read (fd, buf,sizeof (buf))) >0 ) +do { }while (0 );This isn't a bug, but would a bare semicolon (as in bpf_iter.c's read-drain loops) be clearer here than do { } while (0)?
Quoted 1 line
Prefer 'do { } while (0);' instead of a dangling ';'.
humans used to have {} or { /* no-op */ } for cases like this, not a dangling semicolon
do {} while (0) is for macros only
Quoted 37 lines
Quoted 12 lines
[ ... ]
Quoted 8 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c +0
-0@@ -61,4 +61,29 @@ int verifier_snprintf(void *ctx)return 0 ; }Quoted 6 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c int verifier_snprintf(void *ctx) · line 65
Quoted 2 lines
tools/testing/selftests/bpf/progs/test_global_percpu_data.c +2
-0+volatile const __u32 num_cpus =0 ; +volatile const int offsetof_num;This isn't a bug, but would a name like
nums6_off(set directly fromoffsetof(struct test_global_percpu_data__percpu, struct_data.nums[6])) make it clearer which field the iterator is summing?'
num_off' would be better.In iter, it should not know the concrete number. Instead, it identifies the number by offset '
num_off'.Thanks,
LeonQuoted 9 lines
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/31716447858