Add BTF_KIND_FLOAT support
Subsystems (6)
v5 cover Add BTF_KIND_FLOAT support
Some BPF programs compiled on s390 fail to load, because s390 arch-specific linux headers contain float and double types.
Introduce support for such types by representing them using the new BTF_KIND_FLOAT. This series deals with libbpf, bpftool, in-kernel BTF parser as well as selftests and documentation.
There are also pahole and LLVM parts:
* https://github.com/iii-i/dwarves/commit/btf-kind-float-v2
* https://reviews.llvm.org/D83289
but they should go in after the libbpf part is integrated.
---
v0: https://lore.kernel.org/bpf/[email protected]/
v0 -> v1: Per Andrii's suggestion, remove the unnecessary trailing u32.
v1: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2: John noticed that sanitization corrupts BTF, because new and
old sizes don't match. Per Yonghong's suggestion, use a modifier type (which has the same size as the float type) as a replacement.
Per Yonghong's suggestion, add size and alignment checks to the kernel BTF parser.
v2: https://lore.kernel.org/bpf/[email protected]/
v2 -> v3: Based on Yonghong's suggestions: Use BTF_KIND_CONST instead of
BTF_KIND_TYPEDEF and make sure that the C code generated from the sanitized BTF is well-formed; fix size calculation in tests and use NAME_TBD everywhere; limit allowed sizes to 2, 4, 8, 12 and 16 (this should also fix m68k and nds32le builds).
v3: https://lore.kernel.org/bpf/[email protected]/
v3 -> v4: More fixes for the Yonghong's findings: fix the outdated
comment in bpf_object__sanitize_btf() and add the error handling there (I've decided to check uint_id and uchar_id too in order to simplify debugging); add bpftool output example; use div64_u64_rem() instead of % in order to fix the linker error.
Also fix the "invalid BTF_INFO" test (new commit, #4).
v4: https://lore.kernel.org/bpf/[email protected]/
v4 -> v5: Fixes for the Andrii's findings: Use BTF_KIND_STRUCT instead
of BTF_KIND_TYPEDEF for sanitization; check byte_sz in libbpf; move btf__add_float; remove relo support; add a dedup test (new commit, #7).
Based on Alexei's feedback [1] I'm proceeding with the BTF_KIND_FLOAT approach.
[1] https://lore.kernel.org/bpf/CAADnVQKWPODWZ2RSJ5FJhfYpxkuV0cvSAL1O+FSr9oP1ercoBg@mail.gmail.com/
Ilya Leoshkevich (8):
bpf: Add BTF_KIND_FLOAT to uapi
libbpf: Add BTF_KIND_FLOAT support
tools/bpftool: Add BTF_KIND_FLOAT support
selftests/bpf: Use the 25th bit in the "invalid BTF_INFO" test
bpf: Add BTF_KIND_FLOAT support
selftest/bpf: Add BTF_KIND_FLOAT tests
selftests/bpf: Add BTF_KIND_FLOAT to the existing deduplication tests
bpf: Document BTF_KIND_FLOAT in btf.rst
Documentation/bpf/btf.rst | 17 +- include/uapi/linux/btf.h | 5 +- kernel/bpf/btf.c | 79 ++++++++- tools/bpf/bpftool/btf.c | 8 + tools/bpf/bpftool/btf_dumper.c | 1 + tools/include/uapi/linux/btf.h | 5 +- tools/lib/bpf/btf.c | 51 +++++- tools/lib/bpf/btf.h | 6 + tools/lib/bpf/btf_dump.c | 4 + tools/lib/bpf/libbpf.c | 26 ++- tools/lib/bpf/libbpf.map | 5 + tools/lib/bpf/libbpf_internal.h | 2 + tools/testing/selftests/bpf/btf_helpers.c | 4 + tools/testing/selftests/bpf/prog_tests/btf.c | 174 +++++++++++++++++-- tools/testing/selftests/bpf/test_btf.h | 3 + 15 files changed, 367 insertions(+), 23 deletions(-)
Signature · 1 line
2.29.2
PATCH v5 7/8 selftests/bpf: Add BTF_KIND_FLOAT to the existing deduplication tests
Check that floats don't interfere with struct deduplication, that they are not merged with another kinds and that floats of different sizes are not merged with each other.
--- tools/testing/selftests/bpf/prog_tests/btf.c | 43 ++++++++++++++------ 1 file changed, 31 insertions(+), 12 deletions(-)
tools/testing/selftests/bpf/prog_tests/btf.c +31 -12
@@ -6409,11 +6409,12 @@ const struct btf_dedup_test dedup_tests[] = {/* int[16] */ BTF_TYPE_ARRAY_ENC (1 ,1 ,16 ),/* [2] */ /* struct s { */ -BTF_STRUCT_ENC (NAME_NTH (2 ),4 ,84 ),/* [3] */ +BTF_STRUCT_ENC (NAME_NTH (2 ),5 ,88 ),/* [3] */ BTF_MEMBER_ENC (NAME_NTH (3 ),4 ,0 ),/* struct s *next; */ BTF_MEMBER_ENC (NAME_NTH (4 ),5 ,64 ),/* const int *a; */ BTF_MEMBER_ENC (NAME_NTH (5 ),2 ,128 ),/* int b[16]; */ BTF_MEMBER_ENC (NAME_NTH (6 ),1 ,640 ),/* int c; */ +BTF_MEMBER_ENC (NAME_NTH (8 ),13 ,672 ),/* float d; */ /* ptr -> [3] struct s */ BTF_PTR_ENC(3), /* [4] *//* ptr -> [6] const int */
@@ -6424,39 +6425,43 @@ const struct btf_dedup_test dedup_tests[] = {/* full copy of the above */ BTF_TYPE_INT_ENC (NAME_NTH (1 ),BTF_INT_SIGNED ,0 ,32 ,4 ),/* [7] */ BTF_TYPE_ARRAY_ENC (7 ,7 ,16 ),/* [8] */ -BTF_STRUCT_ENC (NAME_NTH (2 ),4 ,84 ),/* [9] */ +BTF_STRUCT_ENC (NAME_NTH (2 ),5 ,88 ),/* [9] */ BTF_MEMBER_ENC (NAME_NTH (3 ),10 ,0 ),BTF_MEMBER_ENC (NAME_NTH (4 ),11 ,64 ),BTF_MEMBER_ENC (NAME_NTH (5 ),8 ,128 ),BTF_MEMBER_ENC (NAME_NTH (6 ),7 ,640 ), +BTF_MEMBER_ENC (NAME_NTH (8 ),13 ,672 ),BTF_PTR_ENC (9 ),/* [10] */ BTF_PTR_ENC (12 ),/* [11] */ BTF_CONST_ENC (7 ),/* [12] */ +BTF_TYPE_FLOAT_ENC (NAME_NTH (7 ),4 ),/* [13] */ BTF_END_RAW , }, - BTF_STR_SEC("\0int\0s\0next\0a\0b\0c\0"), + BTF_STR_SEC("\0int\0s\0next\0a\0b\0c\0float\0d"), }, .expect = { .raw_types = {/* int */ -BTF_TYPE_INT_ENC (NAME_NTH (4 ),BTF_INT_SIGNED ,0 ,32 ,4 ),/* [1] */ +BTF_TYPE_INT_ENC (NAME_NTH (5 ),BTF_INT_SIGNED ,0 ,32 ,4 ),/* [1] */ /* int[16] */ BTF_TYPE_ARRAY_ENC (1 ,1 ,16 ),/* [2] */ /* struct s { */ -BTF_STRUCT_ENC (NAME_NTH (6 ),4 ,84 ),/* [3] */ -BTF_MEMBER_ENC (NAME_NTH (5 ),4 ,0 ),/* struct s *next; */ +BTF_STRUCT_ENC (NAME_NTH (8 ),5 ,88 ),/* [3] */ +BTF_MEMBER_ENC (NAME_NTH (7 ),4 ,0 ),/* struct s *next; */ BTF_MEMBER_ENC (NAME_NTH (1 ),5 ,64 ),/* const int *a; */ BTF_MEMBER_ENC (NAME_NTH (2 ),2 ,128 ),/* int b[16]; */ BTF_MEMBER_ENC (NAME_NTH (3 ),1 ,640 ),/* int c; */ +BTF_MEMBER_ENC (NAME_NTH (4 ),7 ,672 ),/* float d; */ /* ptr -> [3] struct s */ BTF_PTR_ENC (3 ),/* [4] */ /* ptr -> [6] const int */ BTF_PTR_ENC (6 ),/* [5] */ /* const -> [1] int */ BTF_CONST_ENC (1 ),/* [6] */ +BTF_TYPE_FLOAT_ENC (NAME_NTH (7 ),4 ),/* [7] */ BTF_END_RAW , }, - BTF_STR_SEC("\0a\0b\0c\0int\0next\0s"), + BTF_STR_SEC("\0a\0b\0c\0d\0int\0float\0next\0s"), }, .opts = { .dont_resolve_fwds = false,
@@ -6577,9 +6582,10 @@ const struct btf_dedup_test dedup_tests[] = {BTF_FUNC_PROTO_ARG_ENC (NAME_TBD ,1 ),BTF_FUNC_PROTO_ARG_ENC (NAME_TBD ,8 ),BTF_FUNC_ENC (NAME_TBD ,12 ),/* [13] func */ +BTF_TYPE_FLOAT_ENC (NAME_TBD ,2 ),/* [14] float */ BTF_END_RAW , }, - BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M"), + BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M\0N"), }, .expect = { .raw_types = {
@@ -6602,16 +6608,17 @@ const struct btf_dedup_test dedup_tests[] = {BTF_FUNC_PROTO_ARG_ENC (NAME_TBD ,1 ),BTF_FUNC_PROTO_ARG_ENC (NAME_TBD ,8 ),BTF_FUNC_ENC (NAME_TBD ,12 ),/* [13] func */ +BTF_TYPE_FLOAT_ENC (NAME_TBD ,2 ),/* [14] float */ BTF_END_RAW , }, - BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M"), + BTF_STR_SEC("\0A\0B\0C\0D\0E\0F\0G\0H\0I\0J\0K\0L\0M\0N"), }, .opts = { .dont_resolve_fwds = false, }, }, { - .descr ="dedup: no int duplicates" , + .descr ="dedup: no int/float duplicates" , .input = { .raw_types = { BTF_TYPE_INT_ENC(NAME_NTH(1), BTF_INT_SIGNED, 0, 32, 8),
@@ -6626,9 +6633,15 @@ const struct btf_dedup_test dedup_tests[] = {BTF_TYPE_INT_ENC (NAME_NTH (1 ),BTF_INT_SIGNED ,0 ,27 ,8 ),/* different byte size */ BTF_TYPE_INT_ENC (NAME_NTH (1 ),BTF_INT_SIGNED ,0 ,32 ,4 ), +/* all allowed sizes */ +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),2 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),4 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),8 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),12 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),16 ),BTF_END_RAW , }, - BTF_STR_SEC("\0int\0some other int"), + BTF_STR_SEC("\0int\0some other int\0float"), }, .expect = { .raw_types = {
@@ -6644,9 +6657,15 @@ const struct btf_dedup_test dedup_tests[] = {BTF_TYPE_INT_ENC (NAME_NTH (1 ),BTF_INT_SIGNED ,0 ,27 ,8 ),/* different byte size */ BTF_TYPE_INT_ENC (NAME_NTH (1 ),BTF_INT_SIGNED ,0 ,32 ,4 ), +/* all allowed sizes */ +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),2 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),4 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),8 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),12 ), +BTF_TYPE_FLOAT_ENC (NAME_NTH (3 ),16 ),BTF_END_RAW , }, - BTF_STR_SEC("\0int\0some other int"), + BTF_STR_SEC("\0int\0some other int\0float"), }, .opts = { .dont_resolve_fwds = false,
Signature · 1 line
2.29.2
PATCH v5 3/8 tools/bpftool: Add BTF_KIND_FLOAT support
In reply to Ilya LeoshkevichOnly dumping support needs to be adjusted, the code structure follows that of BTF_KIND_INT. Example plain and JSON outputs:
[4] FLOAT 'float' size=4
{"id":4,"kind":"FLOAT","name":"float","size":4}--- tools/bpf/bpftool/btf.c | 8 ++++++++ tools/bpf/bpftool/btf_dumper.c | 1 + 2 files changed, 9 insertions(+)
tools/bpf/bpftool/btf.c +8 -0
@@ -36,6 +36,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO", [BTF_KIND_VAR] = "VAR", [BTF_KIND_DATASEC] = "DATASEC", + [BTF_KIND_FLOAT] = "FLOAT", };struct btf_attach_table {
@@ -327,6 +328,13 @@ static int dump_btf_type(const struct btf *btf, __u32 id,jsonw_end_array (w);break ; } +case BTF_KIND_FLOAT : { +if (json_output) +jsonw_uint_field (w,"size" , t->size); +else +printf (" size=%u" , t->size); +break ; + }default :break ; }
tools/bpf/bpftool/btf_dumper.c +1 -0
@@ -596,6 +596,7 @@ static int __btf_dumper_type_only(const struct btf *btf, __u32 type_id,switch (BTF_INFO_KIND (t->info)) {case BTF_KIND_INT :case BTF_KIND_TYPEDEF : +case BTF_KIND_FLOAT :BTF_PRINT_ARG ("%s " ,btf__name_by_offset (btf, t->name_off));break ;case BTF_KIND_STRUCT :
Signature · 1 line
2.29.2
PATCH v5 5/8 bpf: Add BTF_KIND_FLOAT support
In reply to Ilya LeoshkevichOn the kernel side, introduce a new btf_kind_operations. It is similar to that of BTF_KIND_INT, however, it does not need to handle encodings and bit offsets. Do not implement printing, since the kernel does not know how to format floating-point values.
--- kernel/bpf/btf.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 77 insertions(+), 2 deletions(-)
kernel/bpf/btf.c +77 -2
@@ -173,7 +173,7 @@#define BITS_ROUNDUP_BYTES (bits) \ (BITS_ROUNDDOWN_BYTES(bits) + !!BITS_PER_BYTE_MASKED(bits)) -#define BTF_INFO_MASK 0x8f00ffff +#define BTF_INFO_MASK 0x9f00ffff#define BTF_INT_MASK 0x0fffffff#define BTF_TYPE_ID_VALID (type_id) ((type_id) <= BTF_MAX_TYPE)#define BTF_STR_OFFSET_VALID (name_off) ((name_off) <= BTF_MAX_NAME_OFFSET)
@@ -280,6 +280,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO", [BTF_KIND_VAR] = "VAR", [BTF_KIND_DATASEC] = "DATASEC", + [BTF_KIND_FLOAT] = "FLOAT", };static const char *btf_type_str (const struct btf_type *t)
@@ -574,6 +575,7 @@ static bool btf_type_has_size(const struct btf_type *t)case BTF_KIND_UNION :case BTF_KIND_ENUM :case BTF_KIND_DATASEC : +case BTF_KIND_FLOAT :return true; }
@@ -1704,6 +1706,7 @@ __btf_resolve_size(const struct btf *btf, const struct btf_type *type,case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_FLOAT : size = type->size; goto resolved;
@@ -1849,7 +1852,7 @@ static int btf_df_check_kflag_member(struct btf_verifier_env *env,return -EINVAL ; } -/* Used for ptr, array and struct/union type members. +/* Used for ptr, array struct/union and float type members. * int, enum and modifier types have their specific callback functions. */ static int btf_generic_check_kflag_member(struct btf_verifier_env *env,
@@ -3675,6 +3678,77 @@ static const struct btf_kind_operations datasec_ops = { .show = btf_datasec_show, }; +static s32 btf_float_check_meta (struct btf_verifier_env *env, +const struct btf_type *t, +u32 meta_left) +{ +if (btf_type_vlen (t)) { +btf_verifier_log_type (env, t,"vlen != 0" ); +return -EINVAL ; + } + +if (btf_type_kflag (t)) { +btf_verifier_log_type (env, t,"Invalid btf_info kind_flag" ); +return -EINVAL ; + } + +if (t->size !=2 && t->size !=4 && t->size !=8 && t->size !=12 && + t->size !=16 ) { +btf_verifier_log_type (env, t,"Invalid type_size" ); +return -EINVAL ; + } + +btf_verifier_log_type (env, t,NULL ); + +return 0 ; +} + +static int btf_float_check_member (struct btf_verifier_env *env, +const struct btf_type *struct_type, +const struct btf_member *member, +const struct btf_type *member_type) +{ +u64 start_offset_bytes; +u64 end_offset_bytes; +u64 misalign_bits; +u64 align_bytes; +u64 align_bits; + + align_bytes =min_t (u64,sizeof (void *), member_type->size); + align_bits = align_bytes *BITS_PER_BYTE ; +div64_u64_rem (member->offset, align_bits, &misalign_bits); +if (misalign_bits) { +btf_verifier_log_member (env, struct_type, member, +"Member is not properly aligned" ); +return -EINVAL ; + } + + start_offset_bytes = member->offset /BITS_PER_BYTE ; + end_offset_bytes = start_offset_bytes + member_type->size; +if (end_offset_bytes > struct_type->size) { +btf_verifier_log_member (env, struct_type, member, +"Member exceeds struct_size" ); +return -EINVAL ; + } + +return 0 ; +} + +static void btf_float_log (struct btf_verifier_env *env, +const struct btf_type *t) +{ +btf_verifier_log (env,"size=%u" , t->size); +} + +static const struct btf_kind_operations float_ops = { + .check_meta = btf_float_check_meta, + .resolve = btf_df_resolve, + .check_member = btf_float_check_member, + .check_kflag_member = btf_generic_check_kflag_member, + .log_details = btf_float_log, + .show = btf_df_show, +}; +static int btf_func_proto_check (struct btf_verifier_env *env,const struct btf_type *t) {
@@ -3808,6 +3882,7 @@ static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = { [BTF_KIND_FUNC_PROTO] = &func_proto_ops, [BTF_KIND_VAR] = &var_ops, [BTF_KIND_DATASEC] = &datasec_ops, + [BTF_KIND_FLOAT] = &float_ops, }; static s32 btf_check_meta(struct btf_verifier_env *env,
Signature · 1 line
2.29.2
PATCH v5 1/8 bpf: Add BTF_KIND_FLOAT to uapi
In reply to Ilya LeoshkevichAdd a new kind value and expand the kind bitfield.
--- include/uapi/linux/btf.h | 5 +++-- tools/include/uapi/linux/btf.h | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-)
include/uapi/linux/btf.h +3 -2
@@ -52,7 +52,7 @@ struct btf_type { }; }; -#define BTF_INFO_KIND (info) (((info) >> 24) & 0x0f) +#define BTF_INFO_KIND (info) (((info) >> 24) & 0x1f)#define BTF_INFO_VLEN (info) ((info) & 0xffff)#define BTF_INFO_KFLAG (info) ((info) >> 31)
@@ -72,7 +72,8 @@ struct btf_type {#define BTF_KIND_FUNC_PROTO 13/* Function Proto */ #define BTF_KIND_VAR 14/* Variable */ #define BTF_KIND_DATASEC 15/* Section */ -#define BTF_KIND_MAX BTF_KIND_DATASEC +#define BTF_KIND_FLOAT 16/* Floating point */ +#define BTF_KIND_MAX BTF_KIND_FLOAT#define NR_BTF_KINDS (BTF_KIND_MAX + 1)/* For some specific BTF_KIND, "struct btf_type" is immediately
tools/include/uapi/linux/btf.h +3 -2
@@ -52,7 +52,7 @@ struct btf_type { }; }; -#define BTF_INFO_KIND (info) (((info) >> 24) & 0x0f) +#define BTF_INFO_KIND (info) (((info) >> 24) & 0x1f)#define BTF_INFO_VLEN (info) ((info) & 0xffff)#define BTF_INFO_KFLAG (info) ((info) >> 31)
@@ -72,7 +72,8 @@ struct btf_type {#define BTF_KIND_FUNC_PROTO 13/* Function Proto */ #define BTF_KIND_VAR 14/* Variable */ #define BTF_KIND_DATASEC 15/* Section */ -#define BTF_KIND_MAX BTF_KIND_DATASEC +#define BTF_KIND_FLOAT 16/* Floating point */ +#define BTF_KIND_MAX BTF_KIND_FLOAT#define NR_BTF_KINDS (BTF_KIND_MAX + 1)/* For some specific BTF_KIND, "struct btf_type" is immediately
Signature · 1 line
2.29.2
PATCH v5 4/8 selftests/bpf: Use the 25th bit in the "invalid BTF_INFO" test
In reply to Ilya LeoshkevichThe bit being checked by this test is no longer reserved after introducing BTF_KIND_FLOAT, so use the next one instead.
--- tools/testing/selftests/bpf/prog_tests/btf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/btf.c +1 -1
@@ -1903,7 +1903,7 @@ static struct btf_raw_test raw_tests[] = { .raw_types = {/* int */ /* [1] */ BTF_TYPE_INT_ENC (0 ,BTF_INT_SIGNED ,0 ,32 ,4 ), -BTF_TYPE_ENC (0 ,0x10000000 ,4 ), +BTF_TYPE_ENC (0 ,0x20000000 ,4 ), BTF_END_RAW, }, .str_sec = "",
Signature · 1 line
2.29.2
PATCH v5 8/8 bpf: Document BTF_KIND_FLOAT in btf.rst
In reply to Ilya LeoshkevichAlso document the expansion of the kind bitfield.
--- Documentation/bpf/btf.rst | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
Documentation/bpf/btf.rst +15 -2
@@ -84,6 +84,7 @@ sequentially and type id is assigned to each recognized type starting from id #define BTF_KIND_FUNC_PROTO 13 /* Function Proto */ #define BTF_KIND_VAR 14 /* Variable */ #define BTF_KIND_DATASEC 15 /* Section */ + #define BTF_KIND_FLOAT 16 /* Floating point */ Note that the type section encodes debug info, not just pure types. ``BTF_KIND_FUNC`` is not a type, and it represents a defined subprogram.
@@ -95,8 +96,8 @@ Each type contains the following common data:: /* "info" bits arrangement * bits 0-15: vlen (e.g. # of struct's members) * bits 16-23: unused - * bits 24-27: kind (e.g. int, ptr, array...etc) - * bits 28-30: unused + * bits 24-28: kind (e.g. int, ptr, array...etc) + * bits 29-30: unused * bit 31: kind_flag, currently used by * struct, union and fwd */
@@ -452,6 +453,18 @@ map definition. * ``offset``: the in-section offset of the variable * ``size``: the size of the variable in bytes +2.2.16 BTF_KIND_FLOAT +~~~~~~~~~~~~~~~~~~~~~ + +``struct btf_type`` encoding requirement: + * ``name_off``: any valid offset + * ``info.kind_flag``: 0 + * ``info.kind``: BTF_KIND_FLOAT + * ``info.vlen``: 0 + * ``size``: the size of the float type in bytes: 2, 4, 8, 12 or 16. + +No additional type data follow ``btf_type``. + 3. BTF Kernel API *****************
Signature · 1 line
2.29.2
PATCH v5 6/8 selftest/bpf: Add BTF_KIND_FLOAT tests
In reply to Ilya LeoshkevichTest the good variants as well as the potential malformed ones.
--- tools/testing/selftests/bpf/btf_helpers.c | 4 + tools/testing/selftests/bpf/prog_tests/btf.c | 129 +++++++++++++++++++ tools/testing/selftests/bpf/test_btf.h | 3 + 3 files changed, 136 insertions(+)
tools/testing/selftests/bpf/btf_helpers.c +4 -0
@@ -23,6 +23,7 @@ static const char * const btf_kind_str_mapping[] = { [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO", [BTF_KIND_VAR] = "VAR", [BTF_KIND_DATASEC] = "DATASEC", + [BTF_KIND_FLOAT] = "FLOAT", };static const char *btf_kind_str (__u16 kind)
@@ -173,6 +174,9 @@ int fprintf_btf_type_raw(FILE *out, const struct btf *btf, __u32 id) }break ; } +case BTF_KIND_FLOAT : +fprintf (out," size=%u" , t->size); +break ;default :break ; }
tools/testing/selftests/bpf/prog_tests/btf.c +129 -0
141 lines not on this page · raw message
tools/testing/selftests/bpf/test_btf.h +3 -0
@@ -66,4 +66,7 @@#define BTF_FUNC_ENC (name, func_proto) \ BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_FUNC, 0, 0), func_proto) +#define BTF_TYPE_FLOAT_ENC (name, sz) \ + BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_FLOAT, 0, 0), sz) +#endif /* _TEST_BTF_H */
Signature · 1 line
2.29.2
PATCH v5 2/8 libbpf: Add BTF_KIND_FLOAT support
In reply to Ilya LeoshkevichThe logic follows that of BTF_KIND_INT most of the time. Sanitization replaces BTF_KIND_FLOATs with equally-sized empty BTF_KIND_STRUCTs on older kernels, for example, the following:
[4] FLOAT 'float' size=4
becomes the following:
[4] STRUCT '(anon)' size=4 vlen=0
--- tools/lib/bpf/btf.c | 51 ++++++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 6 ++++ tools/lib/bpf/btf_dump.c | 4 +++ tools/lib/bpf/libbpf.c | 26 ++++++++++++++++- tools/lib/bpf/libbpf.map | 5 ++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 6 files changed, 92 insertions(+), 2 deletions(-)
tools/lib/bpf/btf.c +50 -1
@@ -292,6 +292,7 @@ static int btf_type_size(const struct btf_type *t)case BTF_KIND_PTR :case BTF_KIND_TYPEDEF :case BTF_KIND_FUNC : +case BTF_KIND_FLOAT :return base_size;case BTF_KIND_INT :return base_size +sizeof (__u32);
@@ -339,6 +340,7 @@ static int btf_bswap_type_rest(struct btf_type *t)case BTF_KIND_PTR :case BTF_KIND_TYPEDEF :case BTF_KIND_FUNC : +case BTF_KIND_FLOAT :return 0 ;case BTF_KIND_INT : *(__u32 *)(t +1 ) =bswap_32 (*(__u32 *)(t +1 ));
@@ -579,6 +581,7 @@ __s64 btf__resolve_size(const struct btf *btf, __u32 type_id)case BTF_KIND_UNION :case BTF_KIND_ENUM :case BTF_KIND_DATASEC : +case BTF_KIND_FLOAT : size = t->size; goto done;case BTF_KIND_PTR :
@@ -622,6 +625,7 @@ int btf__align_of(const struct btf *btf, __u32 id)switch (kind) {case BTF_KIND_INT :case BTF_KIND_ENUM : +case BTF_KIND_FLOAT :return min (btf_ptr_sz (btf), (size_t )t->size);case BTF_KIND_PTR :return btf_ptr_sz (btf);
@@ -1783,6 +1787,47 @@ int btf__add_int(struct btf *btf, const char *name, size_t byte_sz, int encodingreturn btf_commit_type (btf, sz); } +/* +* Append new BTF_KIND_FLOAT type with: +* - *name* - non-empty, non-NULL type name; +* - *sz* - size of the type, in bytes; +* Returns: +* - >0, type ID of newly added BTF type; +* - <0, on error. +*/ +int btf__add_float (struct btf *btf,const char *name,size_t byte_sz) +{ +struct btf_type *t; +int sz, name_off; + +/* non-empty name */ +if (!name || !name[0 ]) +return -EINVAL ; + +/* byte_sz must be one of the explicitly allowed values */ +if (byte_sz !=2 && byte_sz !=4 && byte_sz !=8 && byte_sz !=12 && + byte_sz !=16 ) +return -EINVAL ; + +if (btf_ensure_modifiable (btf)) +return -ENOMEM ; + + sz =sizeof (struct btf_type ); + t =btf_add_type_mem (btf, sz); +if (!t) +return -ENOMEM ; + + name_off =btf__add_str (btf, name); +if (name_off <0 ) +return name_off; + + t->name_off = name_off; + t->info =btf_type_info (BTF_KIND_FLOAT ,0 ,0 ); + t->size = byte_sz; + +return btf_commit_type (btf, sz); +} +/* it's completely legal to append BTF types with type IDs pointing forward to * types that haven't been appended yet, so we only make sure that id looks * sane, we can't guarantee that ID will always be valid
@@ -1910,7 +1955,7 @@ static int btf_add_composite(struct btf *btf, int kind, const char *name, __u32* - *byte_sz* - size of the struct, in bytes; * * Struct initially has no fields in it. Fields can be added by -* btf__add_field() right after btf__add_struct() succeeds. +* btf__add_field() right after btf__add_struct() succeeds. * * Returns: * - >0, type ID of newly added BTF type;
@@ -3653,6 +3698,7 @@ static int btf_dedup_prep(struct btf_dedup *d)case BTF_KIND_FWD :case BTF_KIND_TYPEDEF :case BTF_KIND_FUNC : +case BTF_KIND_FLOAT : h =btf_hash_common (t);break ;case BTF_KIND_INT :
@@ -3749,6 +3795,7 @@ static int btf_dedup_prim_type(struct btf_dedup *d, __u32 type_id)break ;case BTF_KIND_FWD : +case BTF_KIND_FLOAT : h =btf_hash_common (t);for_each_dedup_cand (d, hash_entry, h) { cand_id = (__u32 )(long )hash_entry->value;
@@ -4010,6 +4057,7 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,return btf_compat_enum (cand_type, canon_type);case BTF_KIND_FWD : +case BTF_KIND_FLOAT :return btf_equal_common (cand_type, canon_type);case BTF_KIND_CONST :
@@ -4506,6 +4554,7 @@ static int btf_dedup_remap_type(struct btf_dedup *d, __u32 type_id)switch (btf_kind (t)) {case BTF_KIND_INT :case BTF_KIND_ENUM : +case BTF_KIND_FLOAT :break ;case BTF_KIND_FWD :
tools/lib/bpf/btf.h +6 -0
@@ -95,6 +95,7 @@ LIBBPF_API int btf__find_str(struct btf *btf, const char *s); LIBBPF_API int btf__add_str(struct btf *btf, const char *s); LIBBPF_API int btf__add_int(struct btf *btf, const char *name, size_t byte_sz, int encoding); +LIBBPF_API int btf__add_float(struct btf *btf, const char *name, size_t byte_sz); LIBBPF_API int btf__add_ptr(struct btf *btf, int ref_type_id); LIBBPF_API int btf__add_array(struct btf *btf,int index_type_id,int elem_type_id,__u32 nr_elems);
@@ -294,6 +295,11 @@ static inline bool btf_is_datasec(const struct btf_type *t)return btf_kind (t) ==BTF_KIND_DATASEC ; } +static inline bool btf_is_float (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_FLOAT ; +} +static inline __u8 btf_int_encoding (const struct btf_type *t) {return BTF_INT_ENCODING (*(__u32 *)(t +1 ));
tools/lib/bpf/btf_dump.c +4 -0
@@ -279,6 +279,7 @@ static int btf_dump_mark_referenced(struct btf_dump *d)case BTF_KIND_INT :case BTF_KIND_ENUM :case BTF_KIND_FWD : +case BTF_KIND_FLOAT :break ;case BTF_KIND_VOLATILE :
@@ -453,6 +454,7 @@ static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr)switch (btf_kind (t)) {case BTF_KIND_INT : +case BTF_KIND_FLOAT : tstate->order_state =ORDERED ;return 0 ;
@@ -1133,6 +1135,7 @@ static void btf_dump_emit_type_decl(struct btf_dump *d, __u32 id,case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_TYPEDEF : +case BTF_KIND_FLOAT : goto done;default : pr_warn("unexpected type in decl chain, kind:%u, id:[%u]\n",
@@ -1247,6 +1250,7 @@ static void btf_dump_emit_type_chain(struct btf_dump *d,switch (kind) {case BTF_KIND_INT : +case BTF_KIND_FLOAT :btf_dump_emit_mods (d, decls); name =btf_name_of (d, t->name_off);btf_dump_printf (d,"%s" , name);
tools/lib/bpf/libbpf.c +25 -1
@@ -178,6 +178,8 @@ enum kern_feature_id {FEAT_PROG_BIND_MAP ,/* Kernel support for module BTFs */ FEAT_MODULE_BTF , +/* BTF_KIND_FLOAT support */ +FEAT_BTF_FLOAT , __FEAT_CNT, };
@@ -1935,6 +1937,7 @@ static const char *btf_kind_str(const struct btf_type *t)case BTF_KIND_FUNC_PROTO :return "func_proto" ;case BTF_KIND_VAR :return "var" ;case BTF_KIND_DATASEC :return "datasec" ; +case BTF_KIND_FLOAT :return "float" ;default :return "unknown" ; } }
@@ -2384,15 +2387,17 @@ static bool btf_needs_sanitization(struct bpf_object *obj) {bool has_func_global =kernel_supports (FEAT_BTF_GLOBAL_FUNC );bool has_datasec =kernel_supports (FEAT_BTF_DATASEC ); +bool has_float =kernel_supports (FEAT_BTF_FLOAT );bool has_func =kernel_supports (FEAT_BTF_FUNC ); -return !has_func || !has_datasec || !has_func_global; +return !has_func || !has_datasec || !has_func_global || !has_float; }static void bpf_object__sanitize_btf (struct bpf_object *obj,struct btf *btf) {bool has_func_global =kernel_supports (FEAT_BTF_GLOBAL_FUNC );bool has_datasec =kernel_supports (FEAT_BTF_DATASEC ); +bool has_float =kernel_supports (FEAT_BTF_FLOAT );bool has_func =kernel_supports (FEAT_BTF_FUNC );struct btf_type *t;int i, j, vlen;
@@ -2445,6 +2450,10 @@ static void bpf_object__sanitize_btf(struct bpf_object *obj, struct btf *btf) }else if (!has_func_global &&btf_is_func (t)) {/* replace BTF_FUNC_GLOBAL with BTF_FUNC_STATIC */ t->info =BTF_INFO_ENC (BTF_KIND_FUNC ,0 ,0 ); + }else if (!has_float &&btf_is_float (t)) { +/* replace FLOAT with an equally-sized empty STRUCT */ + t->name_off =0 ; + t->info =BTF_INFO_ENC (BTF_KIND_STRUCT ,0 ,0 ); } } }
@@ -3882,6 +3891,18 @@ static int probe_kern_btf_datasec(void) strs, sizeof(strs))); } +static int probe_kern_btf_float (void ) +{ +static const char strs[] ="\0float" ; +__u32 types[] = { +/* float */ +BTF_TYPE_FLOAT_ENC (1 ,4 ), + }; + +return probe_fd (libbpf__load_raw_btf ((char *)types,sizeof (types), + strs,sizeof (strs))); +} +static int probe_kern_array_mmap (void ) {struct bpf_create_map_attr attr = {
@@ -4061,6 +4082,9 @@ static struct kern_feature_desc { [FEAT_MODULE_BTF ] = {"module BTF support" , probe_module_btf, }, + [FEAT_BTF_FLOAT ] = { +"BTF_KIND_FLOAT support" , probe_kern_btf_float, + }, };static bool kernel_supports (enum kern_feature_id feat_id)
tools/lib/bpf/libbpf.map +5 -0
@@ -350,3 +350,8 @@ LIBBPF_0.3.0 { xsk_setup_xdp_prog; xsk_socket__update_xskmap; } LIBBPF_0.2.0; + +LIBBPF_0.4.0 { + global: + btf__add_float; +} LIBBPF_0.3.0;
tools/lib/bpf/libbpf_internal.h +2 -0
@@ -31,6 +31,8 @@#define BTF_MEMBER_ENC (name, type, bits_offset) (name), (type), (bits_offset)#define BTF_PARAM_ENC (name, type) (name), (type)#define BTF_VAR_SECINFO_ENC (type, offset, size) (type), (offset), (size) +#define BTF_TYPE_FLOAT_ENC (name, sz) \ + BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_FLOAT, 0, 0), sz)#ifndef likely#define likely (x) __builtin_expect(!!(x), 1)
Signature · 1 line
2.29.2
Re: [PATCH v5 bpf-next 2/8] libbpf: Add BTF_KIND_FLOAT support
Quoted 5 lines · Ilya Leoshkevich
The logic follows that of
BTF_KIND_INTmost of the time. Sanitization replacesBTF_KIND_FLOATswith equally-sized emptyBTF_KIND_STRUCTson older kernels, for example, the following:
Quoted 8 lines
[4] FLOAT 'float' size=4becomes the following:
[4] STRUCT '(anon)' size=4 vlen=0Signed-off-by Ilya Leoshkevich <[email protected]>---
Quoted 8 lines
tools/lib/bpf/btf.c | 51 ++++++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 6 ++++ tools/lib/bpf/btf_dump.c | 4 +++ tools/lib/bpf/libbpf.c | 26 ++++++++++++++++- tools/lib/bpf/libbpf.map | 5 ++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 6 files changed, 92 insertions(+), 2 deletions(-)
[...]
tools/lib/bpf/btf.c static int btf_add_composite(struct btf *btf, int kind, const char *name, __u32 · line 1958
Quoted 9 lines
/* it's completely legal to append BTF types with type IDs pointing forward to
* types that haven't been appended yet, so we only make sure that id looks
* sane, we can't guarantee that ID will always be validtools/lib/bpf/btf.c +1
-1@@ -1910,7 +1955,7 @@ static int btf_add_composite(struct btf *btf, int kind, const char *name, __u32* - *byte_sz* - size of the struct, in bytes; * * Struct initially has no fields in it. Fields can be added by -* btf__add_field() right after btf__add_struct() succeeds. +* btf__add_field() right after btf__add_struct() succeeds.
Was there some whitespacing problem on this line?
Quoted 3 lines
tools/lib/bpf/btf.c +0
-0* * Returns: * - >0, type ID of newly added BTF type;
[...]
Re: [PATCH v5 bpf-next 7/8] selftests/bpf: Add BTF_KIND_FLOAT to the existing deduplication tests
In reply to Ilya LeoshkevichQuoted 7 lines · Ilya Leoshkevich
Check that floats don't interfere with struct deduplication, that they are not merged with another kinds and that floats of different sizes are not merged with each other.
Signed-off-by Ilya Leoshkevich <[email protected]>---
Thanks! LGTM.
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/btf.c | 43 ++++++++++++++------ 1 file changed, 31 insertions(+), 12 deletions(-)
[...]
Re: [PATCH v5 bpf-next 6/8] selftest/bpf: Add BTF_KIND_FLOAT tests
In reply to Ilya LeoshkevichQuoted 6 lines · Ilya Leoshkevich
Test the good variants as well as the potential malformed ones.
Signed-off-by Ilya Leoshkevich <[email protected]>Acked-by Yonghong Song <[email protected]>---
Quoted 33 lines
tools/testing/selftests/bpf/btf_helpers.c | 4 + tools/testing/selftests/bpf/prog_tests/btf.c | 129 +++++++++++++++++++ tools/testing/selftests/bpf/test_btf.h | 3 + 3 files changed, 136 insertions(+)tools/testing/selftests/bpf/btf_helpers.c +4
-0@@ -23,6 +23,7 @@ static const char * const btf_kind_str_mapping[] = { [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO", [BTF_KIND_VAR] = "VAR", [BTF_KIND_DATASEC] = "DATASEC", + [BTF_KIND_FLOAT] = "FLOAT", };static const char *btf_kind_str (__u16 kind)@@ -173,6 +174,9 @@ int fprintf_btf_type_raw(FILE *out, const struct btf *btf, __u32 id) }break ; } +case BTF_KIND_FLOAT : +fprintf (out," size=%u" , t->size); +break ;default :break ; }tools/testing/selftests/bpf/prog_tests/btf.c +0
-0@@ -3531,6 +3531,134 @@ static struct btf_raw_test raw_tests[] = { .max_entries = 1,
tools/testing/selftests/bpf/prog_tests/btf.c static struct btf_raw_test raw_tests[] = { · line 3541
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/btf.c +8
-0}, +{ + .descr = "float test #1, well-formed", + .raw_types = { +BTF_TYPE_INT_ENC (NAME_TBD ,BTF_INT_SIGNED ,0 ,32 ,4 ), +/* [1] */ +BTF_TYPE_FLOAT_ENC (NAME_TBD ,2 ),/* [2] */ +BTF_TYPE_FLOAT_ENC (NAME_TBD ,4 ),/* [3] */ + BTF_TYPE_FLOAT_ENC(NAME_TBD, 8), /* [4] */
nit: 12-byte for completeness?
Quoted 18 lines
tools/testing/selftests/bpf/prog_tests/btf.c +18
-0+ BTF_TYPE_FLOAT_ENC(NAME_TBD, 16), /* [5] */ + BTF_STRUCT_ENC(NAME_TBD, 4, 32), /* [6] */ + BTF_MEMBER_ENC(NAME_TBD, 2, 0), + BTF_MEMBER_ENC(NAME_TBD, 3, 32), + BTF_MEMBER_ENC(NAME_TBD, 4, 64), + BTF_MEMBER_ENC(NAME_TBD, 5, 128), + BTF_END_RAW, + }, + BTF_STR_SEC("\0int\0_Float16\0float\0double\0long_double\0floats" + "\0x\0y\0z\0_"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = "float_type_check_btf", + .key_size = sizeof(int), + .value_size = 32, + .key_type_id = 1, + .value_type_id = 6, + .max_entries = 1, +},
[...]
Re: [PATCH v5 bpf-next 2/8] libbpf: Add BTF_KIND_FLOAT support
In reply to Andrii NakryikoQuoted 32 lines · Andrii Nakryiko
Quoted 7 lines · Ilya Leoshkevich
The logic follows that of
BTF_KIND_INTmost of the time. Sanitization
replacesBTF_KIND_FLOATswith equally-sized emptyBTF_KIND_STRUCTson
older kernels, for example, the following:Quoted 8 lines
[4] FLOAT 'float' size=4
becomes the following:
[4] STRUCT '(anon)' size=4 vlen=0
Signed-off-by Ilya Leoshkevich <[email protected]>---
Acked-by Andrii Nakryiko <[email protected]>Quoted 9 lines
tools/lib/bpf/btf.c | 51
++++++++++++++++++++++++++++++++-
tools/lib/bpf/btf.h | 6 ++++ tools/lib/bpf/btf_dump.c | 4 +++ tools/lib/bpf/libbpf.c | 26 ++++++++++++++++- tools/lib/bpf/libbpf.map | 5 ++++tools/lib/bpf/libbpf_internal.h| 2 ++6 files changed, 92 insertions(+), 2 deletions(-)[...]
Quoted 14 lines
Quoted 6 lines
/* it's completely legal to append BTF types with type IDs pointing forward to
* types that haven't been appended yet, so we only make sure that id looks
* sane, we can't guarantee that ID will always be valid(quoted diff) +0
-0@@ -1910,7 +1955,7 @@ static int btf_add_composite(struct btf *btf,Quoted 6 lines
int kind, const char *name,
__u32
* - *byte_sz* - size of the struct, in bytes;
*
* Struct initially has no fields in it. Fields can be added by
- *btf__add_field()right afterbtf__add_struct()succeeds.
+ *btf__add_field()right afterbtf__add_struct()succeeds.Was there some whitespacing problem on this line?
Ouch, yes, I remember dropping this chunk, but my editor appears to have sneaked it back in. I will split this commit in two (hopefully it's ok to keep the ack :-)).
Re: [PATCH v5 bpf-next 2/8] libbpf: Add BTF_KIND_FLOAT support
Quoted 49 lines · Ilya Leoshkevich
Quoted 32 lines · Andrii Nakryiko
Quoted 7 lines · Ilya Leoshkevich
The logic follows that of
BTF_KIND_INTmost of the time. Sanitization
replacesBTF_KIND_FLOATswith equally-sized emptyBTF_KIND_STRUCTson
older kernels, for example, the following:Quoted 8 lines
[4] FLOAT 'float' size=4becomes the following:
[4] STRUCT '(anon)' size=4 vlen=0Signed-off-by Ilya Leoshkevich <[email protected]>---
Acked-by Andrii Nakryiko <[email protected]>Quoted 9 lines
tools/lib/bpf/btf.c | 51 ++++++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 6 ++++ tools/lib/bpf/btf_dump.c | 4 +++ tools/lib/bpf/libbpf.c | 26 ++++++++++++++++- tools/lib/bpf/libbpf.map | 5 ++++ tools/lib/bpf/libbpf_internal.h | 2 ++ 6 files changed, 92 insertions(+), 2 deletions(-)[...]
Quoted 14 lines
Quoted 6 lines
/* it's completely legal to append BTF types with type IDs
pointing forward to
* types that haven't been appended yet, so we only make sure that
id looks
* sane, we can't guarantee that ID will always be valid(quoted diff) +0
-0@@ -1910,7 +1955,7 @@ static int btf_add_composite(struct btf *btf,Quoted 6 lines
int kind, const char *name,
__u32
* - *byte_sz* - size of the struct, in bytes;
*
* Struct initially has no fields in it. Fields can be added by
- *btf__add_field()right afterbtf__add_struct()succeeds.
+ *btf__add_field()right afterbtf__add_struct()succeeds.Was there some whitespacing problem on this line?
Quoted 4 lines
Ouch, yes, I remember dropping this chunk, but my editor appears to have sneaked it back in. I will split this commit in two (hopefully it's ok to keep the ack :-)).
Don't bother just for this line.