libbpf: Support uniform BTF-defined key/value specification across all BPF maps
Subsystems (5)
cover libbpf: Support uniform BTF-defined key/value specification across all BPF maps
Currently a bunch of (usually pretty specialized) BPF maps do not support specifying BTF types for they key and value. For such maps, specifying their definition like this:
struct {
__uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
__type(key, int);
__type(value, int);
} my_perf_buf SEC(".maps");Would actually produce warnings about retrying BPF map creation without BTF. Users are forced to know such nuances and use __uint(key_size, 4) instead. This is non-uniform, annoying, and inconvenient.
This patch set teaches libbpf to recognize those specialized maps and removes BTF type IDs when creating BPF map. Also, update existing BPF selftests to exericse this change.
Hengqi Chen (2):
libbpf: Support uniform BTF-defined key/value specification across all
BPF maps
selftests/bpf: Use BTF-defined key/value for map definitions
tools/lib/bpf/libbpf.c | 24 +++++++++++++++++++ tools/testing/selftests/bpf/progs/kfree_skb.c | 4 ++-- .../selftests/bpf/progs/perf_event_stackmap.c | 4 ++-- .../bpf/progs/sockmap_verdict_prog.c | 12 +++++----- .../selftests/bpf/progs/test_btf_map_in_map.c | 14 +++++------ .../selftests/bpf/progs/test_map_in_map.c | 10 ++++---- .../bpf/progs/test_map_in_map_invalid.c | 2 +- .../bpf/progs/test_pe_preserve_elems.c | 8 +++---- .../selftests/bpf/progs/test_perf_buffer.c | 4 ++-- .../bpf/progs/test_select_reuseport_kern.c | 4 ++-- .../bpf/progs/test_stacktrace_build_id.c | 4 ++-- .../selftests/bpf/progs/test_stacktrace_map.c | 4 ++-- .../selftests/bpf/progs/test_tcpnotify_kern.c | 4 ++-- .../selftests/bpf/progs/test_xdp_bpf2bpf.c | 4 ++-- 14 files changed, 62 insertions(+), 40 deletions(-)
--
2.30.2
PATCH 1/2 libbpf: Support uniform BTF-defined key/value specification across all BPF maps
A bunch of BPF maps do not support specifying BTF types for key and value. This is non-uniform and inconvenient[0]. Currently, libbpf uses a retry logic which removes BTF type IDs when BPF map creation failed. Instead of retrying, this commit recognizes those specialized maps and removes BTF type IDs when creating BPF map.
[0] Closes: https://github.com/libbpf/libbpf/issues/355
--- tools/lib/bpf/libbpf.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
tools/lib/bpf/libbpf.c +24 -0
@@ -4643,6 +4643,30 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b create_attr.inner_map_fd = map->inner_map_fd; } +switch (def->type) { +case BPF_MAP_TYPE_PERF_EVENT_ARRAY : +case BPF_MAP_TYPE_CGROUP_ARRAY : +case BPF_MAP_TYPE_STACK_TRACE : +case BPF_MAP_TYPE_ARRAY_OF_MAPS : +case BPF_MAP_TYPE_HASH_OF_MAPS : +case BPF_MAP_TYPE_DEVMAP : +case BPF_MAP_TYPE_DEVMAP_HASH : +case BPF_MAP_TYPE_CPUMAP : +case BPF_MAP_TYPE_XSKMAP : +case BPF_MAP_TYPE_SOCKMAP : +case BPF_MAP_TYPE_SOCKHASH : +case BPF_MAP_TYPE_QUEUE : +case BPF_MAP_TYPE_STACK : +case BPF_MAP_TYPE_RINGBUF : + create_attr.btf_fd =0 ; + create_attr.btf_key_type_id =0 ; + create_attr.btf_value_type_id =0 ; + map->btf_key_type_id =0 ; + map->btf_value_type_id =0 ; +default : +break ; + } +if (obj->gen_loader) {bpf_gen__map_create (obj->gen_loader, &create_attr, is_inner ?-1 : map - obj->maps);/* Pretend to have valid FD to pass various fd >= 0 checks.
--
2.30.2
PATCH 2/2 selftests/bpf: Use BTF-defined key/value for map definitions
In reply to Hengqi ChenChange map definitions in BPF selftests to use BTF-defined key/value types. This unifies the map definitions and ensures libbpf won't emit warning about retrying map creation.
--- tools/testing/selftests/bpf/progs/kfree_skb.c | 4 ++-- .../selftests/bpf/progs/perf_event_stackmap.c | 4 ++-- .../selftests/bpf/progs/sockmap_verdict_prog.c | 12 ++++++------ .../selftests/bpf/progs/test_btf_map_in_map.c | 14 +++++++------- .../testing/selftests/bpf/progs/test_map_in_map.c | 10 ++++------ .../selftests/bpf/progs/test_map_in_map_invalid.c | 2 +- .../selftests/bpf/progs/test_pe_preserve_elems.c | 8 ++++---- .../testing/selftests/bpf/progs/test_perf_buffer.c | 4 ++-- .../bpf/progs/test_select_reuseport_kern.c | 4 ++-- .../selftests/bpf/progs/test_stacktrace_build_id.c | 4 ++-- .../selftests/bpf/progs/test_stacktrace_map.c | 4 ++-- .../selftests/bpf/progs/test_tcpnotify_kern.c | 4 ++-- .../testing/selftests/bpf/progs/test_xdp_bpf2bpf.c | 4 ++-- 13 files changed, 38 insertions(+), 40 deletions(-)
tools/testing/selftests/bpf/progs/kfree_skb.c +2 -2
@@ -9,8 +9,8 @@ char _license[] SEC("license") = "GPL";struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); } perf_buf_map SEC(".maps");#define _ (P ) (__builtin_preserve_access_index(P))
tools/testing/selftests/bpf/progs/perf_event_stackmap.c +2 -2
@@ -11,8 +11,8 @@ typedef __u64 stack_trace_t[PERF_MAX_STACK_DEPTH];struct { __uint(type, BPF_MAP_TYPE_STACK_TRACE); __uint(max_entries, 16384); - __uint(key_size, sizeof(__u32)); - __uint(value_size, sizeof(stack_trace_t)); + __type(key, __u32); + __type(value, stack_trace_t); } stackmap SEC(".maps");struct {
tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c +6 -6
@@ -7,22 +7,22 @@ int _version SEC("version") = 1;struct { __uint(type, BPF_MAP_TYPE_SOCKMAP); __uint(max_entries, 20); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); } sock_map_rx SEC(".maps");struct { __uint(type, BPF_MAP_TYPE_SOCKMAP); __uint(max_entries, 20); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); } sock_map_tx SEC(".maps");struct { __uint(type, BPF_MAP_TYPE_SOCKMAP); __uint(max_entries, 20); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); } sock_map_msg SEC(".maps");struct {
tools/testing/selftests/bpf/progs/test_btf_map_in_map.c +7 -7
@@ -21,8 +21,8 @@ struct inner_map_sz2 {struct outer_arr { __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); __uint(max_entries, 3); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int);/* it's possible to use anonymous struct as inner map definition here */ __array(values, struct { __uint(type, BPF_MAP_TYPE_ARRAY);
@@ -61,8 +61,8 @@ struct inner_map_sz4 {struct outer_arr_dyn { __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); __uint(max_entries, 3); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); __array(values, struct { __uint(type, BPF_MAP_TYPE_ARRAY); __uint(map_flags, BPF_F_INNER_MAP);
@@ -81,7 +81,7 @@ struct outer_arr_dyn {struct outer_hash { __uint(type, BPF_MAP_TYPE_HASH_OF_MAPS); __uint(max_entries, 5); - __uint(key_size, sizeof(int)); + __type(key, int);/* Here everything works flawlessly due to reuse of struct inner_map * and compiler will complain at the attempt to use non-inner_map * references below. This is great experience.
@@ -111,8 +111,8 @@ struct sockarr_sz2 {struct outer_sockarr_sz1 { __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); __uint(max_entries, 1); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); __array(values, struct sockarr_sz1); } outer_sockarr SEC(".maps") = { .values = { (void *)&sockarr_sz1 },
tools/testing/selftests/bpf/progs/test_map_in_map.c +4 -6
@@ -9,18 +9,16 @@ struct {__uint (type,BPF_MAP_TYPE_ARRAY_OF_MAPS );__uint (max_entries,1 );__uint (map_flags,0 ); -__uint (key_size,sizeof (__u32)); -/* must be sizeof(__u32) for map in map */ -__uint (value_size,sizeof (__u32)); +__type (key, __u32); +__type (value, __u32); } mim_array SEC(".maps");struct { __uint(type, BPF_MAP_TYPE_HASH_OF_MAPS); __uint(max_entries, 1); __uint(map_flags, 0); - __uint(key_size, sizeof(int)); -/* must be sizeof(__u32) for map in map */ - __uint(value_size, sizeof(__u32)); + __type(key, int); + __type(value, __u32); } mim_hash SEC(".maps");SEC ("xdp_mimtest" )
tools/testing/selftests/bpf/progs/test_map_in_map_invalid.c +1 -1
@@ -13,7 +13,7 @@ struct inner {struct { __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); __uint(max_entries, 0); /* This will make map creation to fail */ - __uint(key_size, sizeof(__u32)); + __type(key, __u32); __array(values, struct inner); } mim SEC(".maps");
tools/testing/selftests/bpf/progs/test_pe_preserve_elems.c +4 -4
@@ -7,15 +7,15 @@struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); __uint(max_entries, 1); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); } array_1 SEC(".maps");struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); __uint(max_entries, 1); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); __uint(map_flags, BPF_F_PRESERVE_ELEMS); } array_2 SEC(".maps");
tools/testing/selftests/bpf/progs/test_perf_buffer.c +2 -2
@@ -8,8 +8,8 @@struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); } perf_buf_map SEC(".maps");SEC ("tp/raw_syscalls/sys_enter" )
tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c +2 -2
@@ -24,8 +24,8 @@ int _version SEC("version") = 1;struct { __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); __uint(max_entries, 1); - __uint(key_size, sizeof(__u32)); - __uint(value_size, sizeof(__u32)); + __type(key, __u32); + __type(value, __u32); } outer_map SEC(".maps");struct {
tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c +2 -2
@@ -28,8 +28,8 @@ struct {__uint (type,BPF_MAP_TYPE_STACK_TRACE );__uint (max_entries,128 );__uint (map_flags,BPF_F_STACK_BUILD_ID ); -__uint (key_size,sizeof (__u32)); -__uint (value_size,sizeof (stack_trace_t)); +__type (key, __u32); +__type (value, stack_trace_t); } stackmap SEC(".maps");struct {
tools/testing/selftests/bpf/progs/test_stacktrace_map.c +2 -2
@@ -27,8 +27,8 @@ typedef __u64 stack_trace_t[PERF_MAX_STACK_DEPTH];struct { __uint(type, BPF_MAP_TYPE_STACK_TRACE); __uint(max_entries, 16384); - __uint(key_size, sizeof(__u32)); - __uint(value_size, sizeof(stack_trace_t)); + __type(key, __u32); + __type(value, stack_trace_t); } stackmap SEC(".maps");struct {
tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c +2 -2
@@ -24,8 +24,8 @@ struct {struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); __uint(max_entries, 2); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(__u32)); + __type(key, int); + __type(value, __u32); } perf_event_map SEC(".maps"); int _version SEC("version") = 1;
tools/testing/selftests/bpf/progs/test_xdp_bpf2bpf.c +2 -2
@@ -36,8 +36,8 @@ struct meta {struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); - __uint(key_size, sizeof(int)); - __uint(value_size, sizeof(int)); + __type(key, int); + __type(value, int); } perf_buf_map SEC(".maps");__u64 test_result_fentry =0 ;
--
2.30.2
Quoted 40 lines · Hengqi Chen
Currently a bunch of (usually pretty specialized) BPF maps do not support specifying BTF types for they key and value. For such maps, specifying their definition like this:
struct { __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY); __type(key, int); __type(value, int); } my_perf_buf SEC(".maps");Would actually produce warnings about retrying BPF map creation without BTF. Users are forced to know such nuances and use
__uint(key_size, 4)instead. This is non-uniform, annoying, and inconvenient.This patch set teaches libbpf to recognize those specialized maps and removes BTF type IDs when creating BPF map. Also, update existing BPF selftests to exericse this change.
Hengqi Chen (2):
libbpf: Support uniform BTF-defined key/value specification across all
BPF maps
selftests/bpf: Use BTF-defined key/value for map definitionstools/lib/bpf/libbpf.c | 24 +++++++++++++++++++ tools/testing/selftests/bpf/progs/kfree_skb.c | 4 ++-- .../selftests/bpf/progs/perf_event_stackmap.c | 4 ++-- .../bpf/progs/sockmap_verdict_prog.c | 12 +++++----- .../selftests/bpf/progs/test_btf_map_in_map.c | 14 +++++------ .../selftests/bpf/progs/test_map_in_map.c | 10 ++++---- .../bpf/progs/test_map_in_map_invalid.c | 2 +- .../bpf/progs/test_pe_preserve_elems.c | 8 +++---- .../selftests/bpf/progs/test_perf_buffer.c | 4 ++-- .../bpf/progs/test_select_reuseport_kern.c | 4 ++-- .../bpf/progs/test_stacktrace_build_id.c | 4 ++-- .../selftests/bpf/progs/test_stacktrace_map.c | 4 ++-- .../selftests/bpf/progs/test_tcpnotify_kern.c | 4 ++-- .../selftests/bpf/progs/test_xdp_bpf2bpf.c | 4 ++-- 14 files changed, 62 insertions(+), 40 deletions(-)
Quoted 2 lines
--
2.30.2
Looks good. Applied to bpf-next, thanks.