Misc libbpf fixes and small improvements
Subsystems (4)
Andrii Nakryiko
maintainercover Misc libbpf fixes and small improvements
This patch set is a mix of mostly mutually unrelated libbpf and selftests fixes and improvements. Individual patches provide details on each one.
Andrii Nakryiko (9):
selftests/bpf: prevent skeleton generation race
libbpf: make __kptr and __kptr_ref unconditionally use btf_type_tag()
attr
libbpf: improve usability of field-based CO-RE helpers selftests/bpf: use both syntaxes for field-based CO-RE helpers
libbpf: complete field-based CO-RE helpers with field offset helper selftests/bpf: add bpf_core_field_offset() tests
libbpf: provide barrier() and barrier_var() in bpf_helpers.h
libbpf: automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary selftests/bpf: test libbpf's ringbuf size fix up logic
tools/lib/bpf/bpf_core_read.h | 37 ++++++++++++++-- tools/lib/bpf/bpf_helpers.h | 29 ++++++++++--- tools/lib/bpf/libbpf.c | 42 ++++++++++++++++++- tools/testing/selftests/bpf/Makefile | 10 ++--- .../selftests/bpf/prog_tests/core_reloc.c | 13 +++++- .../selftests/bpf/prog_tests/ringbuf_multi.c | 12 ------ .../progs/btf__core_reloc_size___diff_offs.c | 3 ++ .../selftests/bpf/progs/core_reloc_types.h | 18 ++++++++ .../selftests/bpf/progs/exhandler_kern.c | 2 - tools/testing/selftests/bpf/progs/loop5.c | 1 - tools/testing/selftests/bpf/progs/profiler1.c | 1 - tools/testing/selftests/bpf/progs/pyperf.h | 2 - .../bpf/progs/test_core_reloc_existence.c | 11 +++-- .../bpf/progs/test_core_reloc_size.c | 31 ++++++++++++-- .../selftests/bpf/progs/test_pkt_access.c | 2 - .../selftests/bpf/progs/test_ringbuf_multi.c | 2 + 16 files changed, 169 insertions(+), 47 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_size___diff_offs.c
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 1/9 selftests/bpf: prevent skeleton generation race
Prevent "classic" and light skeleton generation rules from stomping on each other's toes due to the use of the same <obj>.linked{1,2,3}.o naming pattern. There is no coordination and synchronizataion between .skel.h and .lskel.h rules, so they can easily overwrite each other's intermediate object files, leading to errors like:
/bin/sh: line 1: 170928 Bus error (core dumped)
/data/users/andriin/linux/tools/testing/selftests/bpf/tools/sbin/bpftool gen skeleton /data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.linked3.o name test_ksyms_weak
Quoted 1 line
/data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.skel.h
make: *** [Makefile:507: /data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.skel.h] Error 135
make: *** Deleting file '/data/users/andriin/linux/tools/testing/selftests/bpf/test_ksyms_weak.skel.h'
Fix by using different suffix for light skeleton rule.
--- tools/testing/selftests/bpf/Makefile | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/testing/selftests/bpf/Makefile +5 -5
@@ -423,11 +423,11 @@ $(TRUNNER_BPF_SKELS): %.skel.h: %.o $(BPFTOOL) | $(TRUNNER_OUTPUT) $(TRUNNER_BPF_LSKELS): %.lskel.h: %.o $(BPFTOOL) | $(TRUNNER_OUTPUT) $$(call msg,GEN-SKEL,$(TRUNNER_BINARY),$$@) - $(Q)$$(BPFTOOL) gen object $$(<:.o=.linked1.o) $$< - $(Q)$$(BPFTOOL) gen object $$(<:.o=.linked2.o) $$(<:.o=.linked1.o) - $(Q)$$(BPFTOOL) gen object $$(<:.o=.linked3.o) $$(<:.o=.linked2.o) - $(Q)diff $$(<:.o=.linked2.o) $$(<:.o=.linked3.o) - $(Q)$$(BPFTOOL) gen skeleton -L $$(<:.o=.linked3.o) name $$(notdir $$(<:.o=_lskel)) > $$@ + $(Q)$$(BPFTOOL) gen object $$(<:.o=.llinked1.o) $$< + $(Q)$$(BPFTOOL) gen object $$(<:.o=.llinked2.o) $$(<:.o=.llinked1.o) + $(Q)$$(BPFTOOL) gen object $$(<:.o=.llinked3.o) $$(<:.o=.llinked2.o) + $(Q)diff $$(<:.o=.llinked2.o) $$(<:.o=.llinked3.o) + $(Q)$$(BPFTOOL) gen skeleton -L $$(<:.o=.llinked3.o) name $$(notdir $$(<:.o=_lskel)) > $$@ $(TRUNNER_BPF_SKELS_LINKED): $(TRUNNER_BPF_OBJS) $(BPFTOOL) | $(TRUNNER_OUTPUT) $$(call msg,LINK-BPF,$(TRUNNER_BINARY),$$(@:.skel.h=.o))
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 2/9 libbpf: make __kptr and __kptr_ref unconditionally use btf_type_tag() attr
In reply to Andrii NakryikoIt will be annoying and surprising for users of __kptr and __kptr_ref if libbpf silently ignores them just because Clang used for compilation didn't support btf_type_tag(). It's much better to get clear compiler error than debug BPF verifier failures later on.
--- tools/lib/bpf/bpf_helpers.h | 5 ----- 1 file changed, 5 deletions(-)
tools/lib/bpf/bpf_helpers.h +0 -5
@@ -149,13 +149,8 @@ enum libbpf_tristate {#define __kconfig __attribute__((section(".kconfig")))#define __ksym __attribute__((section(".ksyms"))) -#if __has_attribute (btf_type_tag)#define __kptr __attribute__((btf_type_tag("kptr")))#define __kptr_ref __attribute__((btf_type_tag("kptr_ref"))) -#else -#define __kptr -#define __kptr_ref -#endif #ifndef ___bpf_concat#define ___bpf_concat (a, b) a ## b
Signature · 1 line
2.30.2
Re: [PATCH bpf-next 2/9] libbpf: make __kptr and __kptr_ref
Hi, Andrii. It is much better to get an explicit compiler error than to debug the BPF Verifier failure later. But should we let the other selftests continue to compile?
I get the following compilation error, and the compilation is aborted:
$ make -C tools/testing/selftests/bpf/
CLNG-BPF [test_maps] cgrp_ls_attach_cgroup.bpf.o
progs/cb_refs.c:7:29: error: unknown attribute 'btf_type_tag' ignored [-Werror,-Wunknown-attributes]
struct prog_test_ref_kfunc __kptr_ref *ptr;
^~~~~~~~~~
/home/rongtao/Git/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:178:35: note: expanded from macro '__kptr_ref'
#define __kptr_ref __attribute__((btf_type_tag("kptr_ref")))
^~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [Makefile:541: /home/rongtao/Git/linux/tools/testing/selftests/bpf/cb_refs.bpf.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from progs/cgrp_kfunc_failure.c:8:
progs/cgrp_kfunc_common.h:13:16: error: unknown attribute 'btf_type_tag' ignored [-Werror,-Wunknown-attributes]
struct cgroup __kptr_ref * cgrp;
^~~~~~~~~~
/home/rongtao/Git/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:178:35: note: expanded from macro '__kptr_ref'
#define __kptr_ref __attribute__((btf_type_tag("kptr_ref")))
^~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [Makefile:541: /home/rongtao/Git/linux/tools/testing/selftests/bpf/cgrp_kfunc_failure.bpf.o] Error 1
In file included from progs/cgrp_kfunc_success.c:8:
progs/cgrp_kfunc_common.h:13:16: error: unknown attribute 'btf_type_tag' ignored [-Werror,-Wunknown-attributes]
struct cgroup __kptr_ref * cgrp;
^~~~~~~~~~
/home/rongtao/Git/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:178:35: note: expanded from macro '__kptr_ref'
#define __kptr_ref __attribute__((btf_type_tag("kptr_ref")))
^~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [Makefile:541: /home/rongtao/Git/linux/tools/testing/selftests/bpf/cgrp_kfunc_success.bpf.o] Error 1
make: Leaving directory '/home/rongtao/Git/linux/tools/testing/selftests/bpf'Best wishes.
Rong Tao
Re: [PATCH bpf-next 2/9] libbpf: make __kptr and __kptr_ref
Quoted 5 lines · Rong Tao
Hi, Andrii. It is much better to get an explicit compiler error than to debug the BPF Verifier failure later. But should we let the other selftests continue to compile?
I get the following compilation error, and the compilation is aborted:
btf_type_tag and btf_decl_tag are supported since llvm14, could you upgrade your compiler? If you want to truely test selftests, you should try to use recent compilers, otherwise, some selftests may fail and
some others may be skipped.
Quoted 35 lines
$ make -C tools/testing/selftests/bpf/ CLNG-BPF [test_maps] cgrp_ls_attach_cgroup.bpf.o progs/cb_refs.c:7:29: error: unknown attribute 'btf_type_tag' ignored [-Werror,-Wunknown-attributes] struct prog_test_ref_kfunc __kptr_ref *ptr; ^~~~~~~~~~ /home/rongtao/Git/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:178:35: note: expanded from macro '__kptr_ref' #define __kptr_ref __attribute__((btf_type_tag("kptr_ref"))) ^~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. make: *** [Makefile:541: /home/rongtao/Git/linux/tools/testing/selftests/bpf/cb_refs.bpf.o] Error 1 make: *** Waiting for unfinished jobs.... In file included from progs/cgrp_kfunc_failure.c:8: progs/cgrp_kfunc_common.h:13:16: error: unknown attribute 'btf_type_tag' ignored [-Werror,-Wunknown-attributes] struct cgroup __kptr_ref * cgrp; ^~~~~~~~~~ /home/rongtao/Git/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:178:35: note: expanded from macro '__kptr_ref' #define __kptr_ref __attribute__((btf_type_tag("kptr_ref"))) ^~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. make: *** [Makefile:541: /home/rongtao/Git/linux/tools/testing/selftests/bpf/cgrp_kfunc_failure.bpf.o] Error 1 In file included from progs/cgrp_kfunc_success.c:8: progs/cgrp_kfunc_common.h:13:16: error: unknown attribute 'btf_type_tag' ignored [-Werror,-Wunknown-attributes] struct cgroup __kptr_ref * cgrp; ^~~~~~~~~~ /home/rongtao/Git/linux/tools/testing/selftests/bpf/tools/include/bpf/bpf_helpers.h:178:35: note: expanded from macro '__kptr_ref' #define __kptr_ref __attribute__((btf_type_tag("kptr_ref"))) ^~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. make: *** [Makefile:541: /home/rongtao/Git/linux/tools/testing/selftests/bpf/cgrp_kfunc_success.bpf.o] Error 1 make: Leaving directory '/home/rongtao/Git/linux/tools/testing/selftests/bpf'Best wishes.
Rong Tao
Andrii Nakryiko
maintainerPATCH 3/9 libbpf: improve usability of field-based CO-RE helpers
In reply to Andrii NakryikoAllow to specify field reference in two ways:
- if user has variable of necessary type, they can use variable-based reference (my_var.my_field or my_var_ptr->my_field). This was the only supported syntax up till now.
- now, bpf_core_field_exists() and bpf_core_field_size() support also specifying field in a fashion similar to offsetof() macro, by specifying type of the containing struct/union separately and field name separately: bpf_core_field_exists(struct my_type, my_field). This forms is quite often more convenient in practice and it matches type-based CO-RE helpers that support specifying type by its name without requiring any variables.
--- tools/lib/bpf/bpf_core_read.h | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)
tools/lib/bpf/bpf_core_read.h +21 -4
@@ -110,21 +110,38 @@ enum bpf_enum_value_kind { val; \ }) +#define ___bpf_field_ref1 (field) (field) +#define ___bpf_field_ref2 (type, field) (((typeof(type) *)0)->field) +#define ___bpf_field_ref(args...) \ + ___bpf_apply(___bpf_field_ref, ___bpf_narg(args))(args) +/* * Convenience macro to check that field actually exists in target kernel's. * Returns: * 1, if matching field is present in target kernel; * 0, if no matching field found. +* +* Supports two forms: +* - field reference through variable access: +* bpf_core_field_exists(p->my_field); +* - field reference through type and field names: +* bpf_core_field_exists(struct my_type, my_field). */ -#define bpf_core_field_exists (field) \ - __builtin_preserve_field_info(field, BPF_FIELD_EXISTS) +#define bpf_core_field_exists(field...) \ + __builtin_preserve_field_info(___bpf_field_ref(field), BPF_FIELD_EXISTS)/* * Convenience macro to get the byte size of a field. Works for integers, * struct/unions, pointers, arrays, and enums. +* +* Supports two forms: +* - field reference through variable access: +* bpf_core_field_size(p->my_field); +* - field reference through type and field names: +* bpf_core_field_size(struct my_type, my_field). */ -#define bpf_core_field_size (field) \ - __builtin_preserve_field_info(field, BPF_FIELD_BYTE_SIZE) +#define bpf_core_field_size(field...) \ + __builtin_preserve_field_info(___bpf_field_ref(field), BPF_FIELD_BYTE_SIZE)/* * Convenience macro to get BTF type ID of a specified type, using a local BTF
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 4/9 selftests/bpf: use both syntaxes for field-based CO-RE helpers
In reply to Andrii NakryikoExcercise both supported forms of bpf_core_field_exists() and bpf_core_field_size() helpers: variable-based field reference and type/field name-based one.
--- .../selftests/bpf/progs/test_core_reloc_existence.c | 11 +++++------ .../selftests/bpf/progs/test_core_reloc_size.c | 8 ++++---- 2 files changed, 9 insertions(+), 10 deletions(-)
tools/testing/selftests/bpf/progs/test_core_reloc_existence.c +5 -6
@@ -45,35 +45,34 @@ int test_core_existence(void *ctx)struct core_reloc_existence_output *out = (void *)&data.out; out->a_exists =bpf_core_field_exists (in->a); -if (bpf_core_field_exists (in->a)) + if (bpf_core_field_exists(struct core_reloc_existence, a)) out->a_value =BPF_CORE_READ (in, a);else out->a_value =0xff000001u ; out->b_exists =bpf_core_field_exists (in->b); -if (bpf_core_field_exists (in->b)) + if (bpf_core_field_exists(struct core_reloc_existence, b)) out->b_value =BPF_CORE_READ (in, b);else out->b_value =0xff000002u ; out->c_exists =bpf_core_field_exists (in->c); -if (bpf_core_field_exists (in->c)) + if (bpf_core_field_exists(struct core_reloc_existence, c)) out->c_value =BPF_CORE_READ (in, c);else out->c_value =0xff000003u ; out->arr_exists =bpf_core_field_exists (in->arr); -if (bpf_core_field_exists (in->arr)) + if (bpf_core_field_exists(struct core_reloc_existence, arr)) out->arr_value =BPF_CORE_READ (in, arr[0 ]);else out->arr_value =0xff000004u ; out->s_exists =bpf_core_field_exists (in->s); -if (bpf_core_field_exists (in->s)) + if (bpf_core_field_exists(struct core_reloc_existence, s)) out->s_value =BPF_CORE_READ (in, s.x);else out->s_value =0xff000005u ;return 0 ; } -
tools/testing/selftests/bpf/progs/test_core_reloc_size.c +4 -4
@@ -44,10 +44,10 @@ int test_core_size(void *ctx) out->struct_sz =bpf_core_field_size (in->struct_field); out->union_sz =bpf_core_field_size (in->union_field); out->arr_sz =bpf_core_field_size (in->arr_field); - out->arr_elem_sz =bpf_core_field_size (in->arr_field[0 ]); - out->ptr_sz =bpf_core_field_size (in->ptr_field); - out->enum_sz =bpf_core_field_size (in->enum_field); - out->float_sz =bpf_core_field_size (in->float_field); + out->arr_elem_sz = bpf_core_field_size(struct core_reloc_size, arr_field[0]); + out->ptr_sz = bpf_core_field_size(struct core_reloc_size, ptr_field); + out->enum_sz = bpf_core_field_size(struct core_reloc_size, enum_field); + out->float_sz = bpf_core_field_size(struct core_reloc_size, float_field);return 0 ; }
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 5/9 libbpf: complete field-based CO-RE helpers with field offset helper
In reply to Andrii NakryikoAdd bpf_core_field_offset() helper to complete field-based CO-RE helpers. This helper can be useful for feature-detection and for some more advanced cases of field reading (e.g., reading flexible array members).
--- tools/lib/bpf/bpf_core_read.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)
tools/lib/bpf/bpf_core_read.h +12 -0
@@ -143,6 +143,18 @@ enum bpf_enum_value_kind { #define bpf_core_field_size(field...) \ __builtin_preserve_field_info(___bpf_field_ref(field), BPF_FIELD_BYTE_SIZE) +/* +* Convenience macro to get field's byte offset. +* +* Supports two forms: +* - field reference through variable access: +* bpf_core_field_offset(p->my_field); +* - field reference through type and field names: +* bpf_core_field_offset(struct my_type, my_field). +*/ +#define bpf_core_field_offset(field...) \ + __builtin_preserve_field_info(___bpf_field_ref(field), BPF_FIELD_BYTE_OFFSET) +/* * Convenience macro to get BTF type ID of a specified type, using a local BTF * information. Return 32-bit unsigned integer with type ID from program's own
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 6/9 selftests/bpf: add bpf_core_field_offset() tests
In reply to Andrii NakryikoAdd test cases for bpf_core_field_offset() helper.
--- .../selftests/bpf/prog_tests/core_reloc.c | 13 ++++++++-- .../progs/btf__core_reloc_size___diff_offs.c | 3 +++ .../selftests/bpf/progs/core_reloc_types.h | 18 +++++++++++++ .../bpf/progs/test_core_reloc_size.c | 25 ++++++++++++++++++- 4 files changed, 56 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_size___diff_offs.c
tools/testing/selftests/bpf/prog_tests/core_reloc.c +11 -2
@@ -277,13 +277,21 @@ static int duration = 0; #define SIZE_OUTPUT_DATA(type) \ STRUCT_TO_CHAR_PTR(core_reloc_size_output) { \ .int_sz = sizeof(((type *)0)->int_field), \ + .int_off = offsetof(type, int_field), \ .struct_sz = sizeof(((type *)0)->struct_field), \ + .struct_off = offsetof(type, struct_field), \ .union_sz = sizeof(((type *)0)->union_field), \ + .union_off = offsetof(type, union_field), \ .arr_sz = sizeof(((type *)0)->arr_field), \ - .arr_elem_sz = sizeof(((type *)0)->arr_field[0]), \ + .arr_off = offsetof(type, arr_field), \ + .arr_elem_sz = sizeof(((type *)0)->arr_field[1]), \ + .arr_elem_off = offsetof(type, arr_field[1]), \ .ptr_sz = 8, /* always 8-byte pointer for BPF */ \ + .ptr_off = offsetof(type, ptr_field), \ .enum_sz = sizeof(((type *)0)->enum_field), \ + .enum_off = offsetof(type, enum_field), \ .float_sz = sizeof(((type *)0)->float_field), \ + .float_off = offsetof(type, float_field), \ }#define SIZE_CASE (name) { \
@@ -714,9 +722,10 @@ static const struct core_reloc_test_case test_cases[] = { }),BITFIELDS_ERR_CASE (bitfields___err_too_big_bitfield), -/* size relocation checks */ +/* field size and offset relocation checks */ SIZE_CASE (size),SIZE_CASE (size___diff_sz), +SIZE_CASE (size___diff_offs), SIZE_ERR_CASE(size___err_ambiguous),/* validate type existence and size relocations */
tools/testing/selftests/bpf/progs/btf__core_reloc_size___diff_offs.c new file +3 -0
@@ -0,0 +1,3 @@ +#include "core_reloc_types.h" + +void f (struct core_reloc_size___diff_offs x) {}
tools/testing/selftests/bpf/progs/core_reloc_types.h +18 -0
@@ -785,13 +785,21 @@ struct core_reloc_bitfields___err_too_big_bitfield {*/ struct core_reloc_size_output {int int_sz; +int int_off;int struct_sz; +int struct_off;int union_sz; +int union_off;int arr_sz; +int arr_off;int arr_elem_sz; +int arr_elem_off;int ptr_sz; +int ptr_off;int enum_sz; +int enum_off;int float_sz; +int float_off; };struct core_reloc_size {
@@ -814,6 +822,16 @@ struct core_reloc_size___diff_sz {double float_field; }; +struct core_reloc_size___diff_offs { +float float_field; +enum {YET_OTHER_VALUE =123 } enum_field; +void *ptr_field; +int arr_field[4 ]; +union {int x; } union_field; +struct {int x; } struct_field; +int int_field; +}; +/* Error case of two candidates with the fields (int_field) at the same * offset, but with differing final relocation values: size 4 vs size 1 */
tools/testing/selftests/bpf/progs/test_core_reloc_size.c +24 -1
@@ -15,13 +15,21 @@ struct {struct core_reloc_size_output {int int_sz; +int int_off;int struct_sz; +int struct_off;int union_sz; +int union_off;int arr_sz; +int arr_off;int arr_elem_sz; +int arr_elem_off;int ptr_sz; +int ptr_off;int enum_sz; +int enum_off;int float_sz; +int float_off; };struct core_reloc_size {
@@ -41,13 +49,28 @@ int test_core_size(void *ctx)struct core_reloc_size_output *out = (void *)&data.out; out->int_sz =bpf_core_field_size (in->int_field); + out->int_off =bpf_core_field_offset (in->int_field); + out->struct_sz =bpf_core_field_size (in->struct_field); + out->struct_off =bpf_core_field_offset (in->struct_field); + out->union_sz =bpf_core_field_size (in->union_field); + out->union_off =bpf_core_field_offset (in->union_field); + out->arr_sz =bpf_core_field_size (in->arr_field); - out->arr_elem_sz = bpf_core_field_size(struct core_reloc_size, arr_field[0]); + out->arr_off =bpf_core_field_offset (in->arr_field); + + out->arr_elem_sz = bpf_core_field_size(struct core_reloc_size, arr_field[1]); + out->arr_elem_off = bpf_core_field_offset(struct core_reloc_size, arr_field[1]); + out->ptr_sz = bpf_core_field_size(struct core_reloc_size, ptr_field); + out->ptr_off = bpf_core_field_offset(struct core_reloc_size, ptr_field); + out->enum_sz = bpf_core_field_size(struct core_reloc_size, enum_field); + out->enum_off = bpf_core_field_offset(struct core_reloc_size, enum_field); + out->float_sz = bpf_core_field_size(struct core_reloc_size, float_field); + out->float_off = bpf_core_field_offset(struct core_reloc_size, float_field);return 0 ; }
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 7/9 libbpf: provide barrier() and barrier_var() in bpf_helpers.h
In reply to Andrii NakryikoAdd barrier() and barrier_var() macros into bpf_helpers.h to be used by end users. While a bit advanced and specialized instruments, they are sometimes indispensable. Instead of requiring each user to figure out exact asm volatile incantations for themselves, provide them from bpf_helpers.h.
Also remove conflicting definitions from selftests. Some tests rely on barrier_var() definition being nothing, those will still work as libbpf does the #ifndef/#endif guarding for barrier() and barrier_var(), allowing users to redefine them, if necessary.
--- tools/lib/bpf/bpf_helpers.h | 24 +++++++++++++++++++ .../selftests/bpf/progs/exhandler_kern.c | 2 -- tools/testing/selftests/bpf/progs/loop5.c | 1 - tools/testing/selftests/bpf/progs/profiler1.c | 1 - tools/testing/selftests/bpf/progs/pyperf.h | 2 -- .../selftests/bpf/progs/test_pkt_access.c | 2 -- 6 files changed, 24 insertions(+), 8 deletions(-)
tools/lib/bpf/bpf_helpers.h +24 -0
@@ -75,6 +75,30 @@ })#endif +/* +* Compiler (optimization) barrier. +*/ +#ifndef barrier +#define barrier () asm volatile("" ::: "memory") +#endif + +/* Variable-specific compiler (optimization) barrier. It's a no-op which makes +* compiler believe that there is some black box modification of a given +* variable and thus prevents compiler from making extra assumption about its +* value and potential simplifications and optimizations on this variable. +* +* E.g., compiler might often delay or even omit 32-bit to 64-bit casting of +* a variable, making some code patterns unverifiable. Putting barrier_var() +* in place will ensure that cast is performed before the barrier_var() +* invocation, because compiler has to pessimistically assume that embedded +* asm section might perform some extra operations on that variable. +* +* This is a variable-specific variant of more global barrier(). +*/ +#ifndef barrier_var +#define barrier_var (var) asm volatile("" : "=r"(var) : "0"(var)) +#endif +/* * Helper macro to throw a compilation error if __bpf_unreachable() gets * built into the resulting code. This works given BPF back end does not
tools/testing/selftests/bpf/progs/exhandler_kern.c +0 -2
@@ -7,8 +7,6 @@#include <bpf/bpf_tracing.h> #include <bpf/bpf_core_read.h> -#define barrier_var (var) asm volatile("" : "=r"(var) : "0"(var)) - char _license[] SEC("license") = "GPL";unsigned int exception_triggered;
tools/testing/selftests/bpf/progs/loop5.c +0 -1
@@ -2,7 +2,6 @@// Copyright (c) 2019 Facebook #include <linux/bpf.h> #include <bpf/bpf_helpers.h> -#define barrier () __asm__ __volatile__("": : :"memory") char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/profiler1.c +0 -1
@@ -1,6 +1,5 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -#define barrier_var (var) asm volatile("" : "=r"(var) : "0"(var))#define UNROLL #define INLINE __always_inline#include "profiler.inc.h"
tools/testing/selftests/bpf/progs/pyperf.h +0 -2
@@ -171,8 +171,6 @@ struct process_frame_ctx {bool done; }; -#define barrier_var (var) asm volatile("" : "=r"(var) : "0"(var)) -static int process_frame_callback (__u32 i,struct process_frame_ctx *ctx) {int zero =0 ;
tools/testing/selftests/bpf/progs/test_pkt_access.c +0 -2
@@ -14,8 +14,6 @@#include <bpf/bpf_helpers.h> #include <bpf/bpf_endian.h> -#define barrier () __asm__ __volatile__("": : :"memory") -/* llvm will optimize both subprograms into exactly the same BPF assembly * * Disassembly of section .text:
Signature · 1 line
2.30.2
Andrii Nakryiko
maintainerPATCH 8/9 libbpf: automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary
In reply to Andrii NakryikoKernel imposes a pretty particular restriction on ringbuf map size. It has to be a power-of-2 multiple of page size. While generally this isn't hard for user to satisfy, sometimes it's impossible to do this declaratively in BPF source code or just plain inconvenient to do at runtime.
One such example might be BPF libraries that are supposed to work on different architectures, which might not agree on what the common page size is.
Let libbpf find the right size for user instead, if it turns out to not satisfy kernel requirements. If user didn't set size at all, that's most probably a mistake so don't upsize such zero size to one full page, though. Also we need to be careful about not overflowing __u32 max_entries.
--- tools/lib/bpf/libbpf.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf.c +41 -1
@@ -4943,6 +4943,44 @@ bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map)static void bpf_map__destroy (struct bpf_map *map); +static bool is_pow_of_2 (size_t x) +{ +return x && (x & (x -1 )); +} + +static size_t adjust_ringbuf_sz (size_t sz) +{ +__u32 page_sz =sysconf (_SC_PAGE_SIZE); +__u32 i, mul; + +/* if user forgot to set any size, make sure they see error */ +if (sz ==0 ) +return 0 ; +/* Kernel expects BPF_MAP_TYPE_RINGBUF's max_entries to be +* a power-of-2 multiple of kernel's page size. If user diligently +* satisified these conditions, pass the size through. +*/ +if ((sz % page_sz) ==0 &&is_pow_of_2 (sz / page_sz)) +return sz; + +/* Otherwise find closest (page_sz * power_of_2) product bigger than +* user-set size to satisfy both user size request and kernel +* requirements and substitute correct max_entries for map creation. +*/ +for (i =0 , mul =1 ; ; i++, mul <<=1 ) { +if (mul >UINT_MAX / page_sz)/* prevent __u32 overflow */ +break ; +if (mul * page_sz > sz) +return mul * page_sz; + } + +/* if it's impossible to satisfy the conditions (i.e., user size is +* very close to UINT_MAX but is not a power-of-2 multiple of +* page_size) then just return original size and let kernel reject it +*/ +return sz; +} +static int bpf_object__create_map (struct bpf_object *obj,struct bpf_map *map,bool is_inner) {LIBBPF_OPTS (bpf_map_create_opts, create_attr);
@@ -4981,6 +5019,9 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b }switch (def->type) { +case BPF_MAP_TYPE_RINGBUF : + map->def.max_entries =adjust_ringbuf_sz (map->def.max_entries); +/* fallthrough */ case BPF_MAP_TYPE_PERF_EVENT_ARRAY :case BPF_MAP_TYPE_CGROUP_ARRAY :case BPF_MAP_TYPE_STACK_TRACE :
@@ -4994,7 +5035,6 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, bcase BPF_MAP_TYPE_SOCKHASH :case BPF_MAP_TYPE_QUEUE :case BPF_MAP_TYPE_STACK : -case BPF_MAP_TYPE_RINGBUF : create_attr.btf_fd =0 ; create_attr.btf_key_type_id =0 ; create_attr.btf_value_type_id =0 ;
Signature · 1 line
2.30.2
Re: [PATCH bpf-next 8/9] libbpf: automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary
Hi Andrii,
Quoted 17 lines · Andrii Nakryiko
Kernel imposes a pretty particular restriction on ringbuf map size. It has to be a power-of-2 multiple of page size. While generally this isn't hard for user to satisfy, sometimes it's impossible to do this declaratively in BPF source code or just plain inconvenient to do at runtime.
One such example might be BPF libraries that are supposed to work on different architectures, which might not agree on what the common page size is.
Let libbpf find the right size for user instead, if it turns out to not satisfy kernel requirements. If user didn't set size at all, that's most probably a mistake so don't upsize such zero size to one full page, though. Also we need to be careful about not overflowing
__u32max_entries.Signed-off-by Andrii Nakryiko <[email protected]>
<snip>
Quoted 22 lines
tools/lib/bpf/libbpf.c +22
-0+static size_t adjust_ringbuf_sz (size_t sz) +{ +__u32 page_sz =sysconf (_SC_PAGE_SIZE); +__u32 i, mul; + +/* if user forgot to set any size, make sure they see error */ +if (sz ==0 ) +return 0 ; +/* Kernel expects BPF_MAP_TYPE_RINGBUF's max_entries to be +* a power-of-2 multiple of kernel's page size. If user diligently +* satisified these conditions, pass the size through. +*/ +if ((sz % page_sz) ==0 &&is_pow_of_2 (sz / page_sz)) +return sz; + +/* Otherwise find closest (page_sz * power_of_2) product bigger than +* user-set size to satisfy both user size request and kernel +* requirements and substitute correct max_entries for map creation. +*/ +for (i =0 , mul =1 ; ; i++, mul <<=1 ) { +if (mul >UINT_MAX / page_sz)/* prevent __u32 overflow */ +break ;
tools/lib/bpf/libbpf.c bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) · line 4982
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+if (mul * page_sz > sz) +return mul * page_sz; + } + +/* if it's impossible to satisfy the conditions (i.e., user size is +* very close to UINT_MAX but is not a power-of-2 multiple of +* page_size) then just return original size and let kernel reject it +*/ +return sz; +}
This patch in -next as commit 0087a681fa8c ("libbpf: Automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary") breaks the build with tip of tree LLVM due to [1] strengthening -Wunused-but-set-variable:
libbpf.c:4954:8: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
__u32 i, mul;
^1 error generated.
Should i be removed or was it intended to be used somewhere that it is not?
[1]: https://github.com/llvm/llvm-project/commit/2af845a6519c9cde5c8f58db5554f8b1084ce1ed
Cheers,
Nathan
Re: [PATCH bpf-next 8/9] libbpf: automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary
Quoted 61 lines · Nathan Chancellor
Hi Andrii,
Quoted 17 lines · Andrii Nakryiko
Kernel imposes a pretty particular restriction on ringbuf map size. It has to be a power-of-2 multiple of page size. While generally this isn't hard for user to satisfy, sometimes it's impossible to do this declaratively in BPF source code or just plain inconvenient to do at runtime.
One such example might be BPF libraries that are supposed to work on different architectures, which might not agree on what the common page size is.
Let libbpf find the right size for user instead, if it turns out to not satisfy kernel requirements. If user didn't set size at all, that's most probably a mistake so don't upsize such zero size to one full page, though. Also we need to be careful about not overflowing
__u32max_entries.Signed-off-by Andrii Nakryiko <[email protected]><snip>
Quoted 22 lines
tools/lib/bpf/libbpf.c +22
-0+static size_t adjust_ringbuf_sz (size_t sz) +{ +__u32 page_sz =sysconf (_SC_PAGE_SIZE); +__u32 i, mul; + +/* if user forgot to set any size, make sure they see error */ +if (sz ==0 ) +return 0 ; +/* Kernel expects BPF_MAP_TYPE_RINGBUF's max_entries to be +* a power-of-2 multiple of kernel's page size. If user diligently +* satisified these conditions, pass the size through. +*/ +if ((sz % page_sz) ==0 &&is_pow_of_2 (sz / page_sz)) +return sz; + +/* Otherwise find closest (page_sz * power_of_2) product bigger than +* user-set size to satisfy both user size request and kernel +* requirements and substitute correct max_entries for map creation. +*/ +for (i =0 , mul =1 ; ; i++, mul <<=1 ) { +if (mul >UINT_MAX / page_sz)/* prevent __u32 overflow */ +break ;tools/lib/bpf/libbpf.c bpf_object__populate_internal_map(struct bpf_object *obj, struct bpf_map *map) · line 4982
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+if (mul * page_sz > sz) +return mul * page_sz; + } + +/* if it's impossible to satisfy the conditions (i.e., user size is +* very close to UINT_MAX but is not a power-of-2 multiple of +* page_size) then just return original size and let kernel reject it +*/ +return sz; +}This patch in -next as commit
0087a681fa8c("libbpf: Automatically fix upBPF_MAP_TYPE_RINGBUFsize, if necessary") breaks the build with tip of tree LLVM due to [1] strengthening -Wunused-but-set-variable:
Quoted 7 lines
libbpf.c:4954:8: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
__u32 i, mul; ^1 error generated.
Should i be removed or was it intended to be used somewhere that it is not?
my initial implementation had some hard limit on number of iterations which I later removed. I'll drop the i completely, thanks for pointing out.
Quoted 5 lines
[1]: https://github.com/llvm/llvm-project/commit/2af845a6519c9cde5c8f58db5554f8b1084ce1ed
Cheers,
Nathan
Andrii Nakryiko
maintainerPATCH 9/9 selftests/bpf: test libbpf's ringbuf size fix up logic
In reply to Andrii NakryikoMake sure we always excercise libbpf's ringbuf map size adjustment logic by specifying non-zero size that's definitely not a page size multiple.
--- .../testing/selftests/bpf/prog_tests/ringbuf_multi.c | 12 ------------ .../testing/selftests/bpf/progs/test_ringbuf_multi.c | 2 ++ 2 files changed, 2 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/ringbuf_multi.c +0 -12
@@ -50,18 +50,6 @@ void test_ringbuf_multi(void)if (CHECK (!skel,"skel_open" ,"skeleton open failed\n" ))return ; - err =bpf_map__set_max_entries (skel->maps.ringbuf1, page_size); -if (CHECK (err !=0 ,"bpf_map__set_max_entries" ,"bpf_map__set_max_entries failed\n" )) - goto cleanup; - - err =bpf_map__set_max_entries (skel->maps.ringbuf2, page_size); -if (CHECK (err !=0 ,"bpf_map__set_max_entries" ,"bpf_map__set_max_entries failed\n" )) - goto cleanup; - - err =bpf_map__set_max_entries (bpf_map__inner_map (skel->maps.ringbuf_arr), page_size); -if (CHECK (err !=0 ,"bpf_map__set_max_entries" ,"bpf_map__set_max_entries failed\n" )) - goto cleanup; - proto_fd =bpf_map_create (BPF_MAP_TYPE_RINGBUF ,NULL ,0 ,0 , page_size,NULL );if (CHECK (proto_fd <0 ,"bpf_map_create" ,"bpf_map_create failed\n" )) goto cleanup;
tools/testing/selftests/bpf/progs/test_ringbuf_multi.c +2 -0
@@ -15,6 +15,8 @@ struct sample {struct ringbuf_map { __uint(type, BPF_MAP_TYPE_RINGBUF); +/* libbpf will adjust to valid page size */ + __uint(max_entries, 1000); } ringbuf1 SEC(".maps"), ringbuf2 SEC(".maps");
Signature · 1 line
2.30.2
This series was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Quoted 15 lines · On Sun, 8 May 2022 17:41:39 -0700 you wrote:
This patch set is a mix of mostly mutually unrelated libbpf and selftests fixes and improvements. Individual patches provide details on each one.
Andrii Nakryiko (9):
selftests/bpf: prevent skeleton generation race
libbpf: make__kptrand__kptr_refunconditionally usebtf_type_tag()
attr
libbpf: improve usability of field-based CO-RE helpers selftests/bpf: use both syntaxes for field-based CO-RE helpers
libbpf: complete field-based CO-RE helpers with field offset helper selftests/bpf: addbpf_core_field_offset()tests
libbpf: providebarrier()andbarrier_var()inbpf_helpers.h
libbpf: automatically fix upBPF_MAP_TYPE_RINGBUFsize, if necessary selftests/bpf: test libbpf's ringbuf size fix up logic
Quoted 1 line
[...]
Here is the summary with links:
- [bpf-next,1/9] selftests/bpf: prevent skeleton generation race
https://git.kernel.org/bpf/bpf-next/c/1e2666e029e5 - [bpf-next,2/9] libbpf: make __kptr and __kptr_ref unconditionally use btf_type_tag() attr https://git.kernel.org/bpf/bpf-next/c/8e2f618e8be6 - [bpf-next,3/9] libbpf: improve usability of field-based CO-RE helpers https://git.kernel.org/bpf/bpf-next/c/73d0280f6b79 - [bpf-next,4/9] selftests/bpf: use both syntaxes for field-based CO-RE helpers https://git.kernel.org/bpf/bpf-next/c/2a4ca46b7d2a - [bpf-next,5/9] libbpf: complete field-based CO-RE helpers with field offset helper https://git.kernel.org/bpf/bpf-next/c/7715f549a9d8 - [bpf-next,6/9] selftests/bpf: add bpf_core_field_offset() tests https://git.kernel.org/bpf/bpf-next/c/785c3342cf6c - [bpf-next,7/9] libbpf: provide barrier() and barrier_var() in bpf_helpers.h https://git.kernel.org/bpf/bpf-next/c/f760d0537925 - [bpf-next,8/9] libbpf: automatically fix up BPF_MAP_TYPE_RINGBUF size, if necessary https://git.kernel.org/bpf/bpf-next/c/0087a681fa8c - [bpf-next,9/9] selftests/bpf: test libbpf's ringbuf size fix up logic https://git.kernel.org/bpf/bpf-next/c/7b3a06382442
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html