libbpf: Add gating for arena globals relocation feature
Emil Tsalapatis
reviewerPATCH 1/1 libbpf: Add gating for arena globals relocation feature
Add feature gating for the arena globals relocation introduced in commit c1f61171d44b. The commit depends on a previous commit in the same patchset that is absent from older kernels, causing arena programs using arenas >= 512MiB to fail to load and breaking libbpf's backwards compatibility. Introduce a libbpf feature to check whether the running kernel includes the required patch and only relocate arena globals if it does.
--- tools/include/linux/filter.h | 10 ++ tools/lib/bpf/features.c | 178 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 7 +- tools/lib/bpf/libbpf_internal.h | 2 + 4 files changed, 195 insertions(+), 2 deletions(-)
tools/include/linux/filter.h +10 -0
@@ -366,4 +366,14 @@ .off = 0, \ .imm = 0 }) +/* Program address space cast from 1 -> 0 */ + +#define BPF_ADDR_SPACE_CAST_TO_ARENA (REG ) \ + ((struct bpf_insn) { \ + .code = BPF_ALU64 | BPF_MOV | BPF_X, \ + .dst_reg = REG, \ + .src_reg = REG, \ + .off = BPF_ADDR_SPACE_CAST, \ + .imm = 1U }) +#endif /* __TOOLS_LINUX_FILTER_H */
tools/lib/bpf/features.c +178 -0
196 lines not on this page · raw message
tools/lib/bpf/libbpf.c +5 -2
@@ -3009,8 +3009,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,memcpy (obj->arena_data, data, data_sz); obj->arena_data_sz = data_sz; -/* place globals at the end of the arena */ - obj->arena_data_off = mmap_sz - data_alloc_sz; +/* place globals at the end of the arena (if supported) */ +if (kernel_supports (obj,FEAT_ARENA_GLOBAL_RELOC )) + obj->arena_data_off = mmap_sz - data_alloc_sz; +else + obj->arena_data_off =0 ;/* make bpf_map__init_value() work for ARENA maps */ map->mmaped = obj->arena_data;
tools/lib/bpf/libbpf_internal.h +2 -0
@@ -392,6 +392,8 @@ enum kern_feature_id {FEAT_ARG_CTX_TAG ,/* Kernel supports '?' at the front of datasec names */ FEAT_BTF_QMARK_DATASEC , +/* Kernel supports relocating arena globals to end of the arena */ +FEAT_ARENA_GLOBAL_RELOC , __FEAT_CNT, };
Signature · 1 line
2.49.0
Quoted 26 lines · Emil Tsalapatis
Add feature gating for the arena globals relocation introduced in commit
c1f61171d44b. The commit depends on a previous commit in the same patchset that is absent from older kernels, causing arena programs using arenas >= 512MiB to fail to load and breaking libbpf's backwards compatibility. Introduce a libbpf feature to check whether the running kernel includes the required patch and only relocate arena globals if it does.Fixes c1f61171d44b ("libbpf: Move arena globals to the end of the arena")Signed-off-by Emil Tsalapatis <[email protected]>--- tools/include/linux/filter.h | 10 ++ tools/lib/bpf/features.c | 178 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 7 +- tools/lib/bpf/libbpf_internal.h | 2 + 4 files changed, 195 insertions(+), 2 deletions(-)tools/include/linux/filter.h +0
-0@@ -366,4 +366,14 @@ .off = 0, \ .imm = 0 })
tools/include/linux/filter.h line 378
Quoted 10 lines
tools/include/linux/filter.h +10
-0+/* Program address space cast from 1 -> 0 */ + +#define BPF_ADDR_SPACE_CAST_TO_ARENA (REG ) \ + ((struct bpf_insn) { \ + .code = BPF_ALU64 | BPF_MOV | BPF_X, \ + .dst_reg = REG, \ + .src_reg = REG, \ + .off = BPF_ADDR_SPACE_CAST, \ + .imm = 1U }) +
Note that this won't be available on Github mirror side, we'd need to add this there as well (but I hope we might now need it after all)
Quoted 2 lines
tools/include/linux/filter.h +0
-0#endif /* __TOOLS_LINUX_FILTER_H */ tools/lib/bpf/features.c +0
-0
[...]
tools/lib/bpf/features.c
Quoted 13 lines
tools/lib/bpf/features.c +13
-0+ opts.prog_btf_fd = btf_fd; + opts.func_info = &func_infos; + opts.func_info_cnt =ARRAY_SIZE (func_infos); + opts.func_info_rec_size =sizeof (func_infos[0 ]); + + prog_fd =bpf_prog_load (BPF_PROG_TYPE_SYSCALL ,NULL ,"GPL" , insns, insn_cnt, &opts); +munmap (mapping, arena_size); +close (btf_fd); +close (map); + +return probe_fd (prog_fd); +} +
ok, wow, that's quite a feature detector :)
but if we take a step back, we really are just trying to detect the change in
12a1fe6e12db ("bpf/verifier: Do not limit maximum direct offset into arena map")
(btw, do mention this commit somewhere in the comment in the feature detection function in the next revision)
which just did this:
kernel/bpf/verifier.c +0 -0
@@ -21132,11 +21132,6 @@ static int resolve_pseudo_ldimm64(struct
bpf_verifier_env *env)
} else {
u32 off = insn[1].imm;- if (off >= BPF_MAX_VAR_OFF) {
- verbose(env, "direct value
offset of %u is not allowed\n", off);
- return -EINVAL;
- }
-
if (!map->ops->map_direct_value_addr) {
verbose(env, "no direct value
access support for this map type\n");
return -EINVAL;
Now, setting up ARENA map is quite bothersome (as evidenced by your code above), but ARRAY with BPF_F_MMAPABLE flag also allows direct value access, so let's use that to do the detection.
Basically, create a small ARRAY map (no BTF required, I think), and try to verify a simple program that only does some map access with fixed large offset (e.g., at 4GB - 8 offset). With old kernels you'd expect this to fail with "direct value offset of %u is not allowed" message in the verifier log (you'll need to provide a small log buffer in the feature detection function, of course). But on newer ones we'll fail on "invalid access to map value pointer, value_size=%u off=%u".
This relies on verifier log messages, which isn't great, but we can adjust one of selftests to validate that libbpf does indeed put offset at non-zero offset, as that should always be true for newer kernels.
Does it make sense?
pw-bot: cr
tools/lib/bpf/features.c static struct kern_feature_desc { · line 760
Quoted 9 lines
typedef int (*feature_probe_fn)(int /*
token_fd*/);static struct
kern_feature_cachefeature_cache;tools/lib/bpf/features.c +2
-0@@ -581,6 +756,9 @@ static struct kern_feature_desc { [FEAT_BTF_QMARK_DATASEC] = { "BTF DATASEC names starting from '?'", probe_kern_btf_qmark_datasec, }, + [FEAT_ARENA_GLOBAL_RELOC] = { + "Relocatable arena globals support", probe_kern_arena_global_reloc,
we don't detect "relocatable arena globals support", we detect if ldimm64 allows full range offsets, so call the detector as such (i.e., what you detect, not what for you are doing that).
Quoted 39 lines
tools/lib/bpf/features.c +1
-0+ }, };bool feat_supported (struct kern_feature_cache *cache,enum kern_feature_id feat_id)tools/lib/bpf/libbpf.c +5
-2@@ -3009,8 +3009,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,memcpy (obj->arena_data, data, data_sz); obj->arena_data_sz = data_sz; -/* place globals at the end of the arena */ - obj->arena_data_off = mmap_sz - data_alloc_sz; +/* place globals at the end of the arena (if supported) */ +if (kernel_supports (obj,FEAT_ARENA_GLOBAL_RELOC )) + obj->arena_data_off = mmap_sz - data_alloc_sz; +else + obj->arena_data_off =0 ;/* make bpf_map__init_value() work for ARENA maps */ map->mmaped = obj->arena_data;tools/lib/bpf/libbpf_internal.h +2
-1@@ -392,6 +392,8 @@ enum kern_feature_id {FEAT_ARG_CTX_TAG ,/* Kernel supports '?' at the front of datasec names */ FEAT_BTF_QMARK_DATASEC , +/* Kernel supports relocating arena globals to end of the arena */ +FEAT_ARENA_GLOBAL_RELOC , __FEAT_CNT, }; --2.49.0
Emil Tsalapatis
reviewerQuoted 103 lines · Andrii Nakryiko
Quoted 26 lines · Emil Tsalapatis
Add feature gating for the arena globals relocation introduced in commit
c1f61171d44b. The commit depends on a previous commit in the same patchset that is absent from older kernels, causing arena programs using arenas >= 512MiB to fail to load and breaking libbpf's backwards compatibility. Introduce a libbpf feature to check whether the running kernel includes the required patch and only relocate arena globals if it does.Fixes c1f61171d44b ("libbpf: Move arena globals to the end of the arena")Signed-off-by Emil Tsalapatis <[email protected]>--- tools/include/linux/filter.h | 10 ++ tools/lib/bpf/features.c | 178 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.c | 7 +- tools/lib/bpf/libbpf_internal.h | 2 + 4 files changed, 195 insertions(+), 2 deletions(-)tools/include/linux/filter.h +0
-0@@ -366,4 +366,14 @@ .off = 0, \ .imm = 0 })tools/include/linux/filter.h line 378
Quoted 10 lines
tools/include/linux/filter.h +10
-0+/* Program address space cast from 1 -> 0 */ + +#define BPF_ADDR_SPACE_CAST_TO_ARENA (REG ) \ + ((struct bpf_insn) { \ + .code = BPF_ALU64 | BPF_MOV | BPF_X, \ + .dst_reg = REG, \ + .src_reg = REG, \ + .off = BPF_ADDR_SPACE_CAST, \ + .imm = 1U }) +Note that this won't be available on Github mirror side, we'd need to add this there as well (but I hope we might now need it after all)
Quoted 2 lines
tools/include/linux/filter.h +0
-0#endif /* __TOOLS_LINUX_FILTER_H */ tools/lib/bpf/features.c +0
-0[...]
tools/lib/bpf/features.c
Quoted 13 lines
tools/lib/bpf/features.c +13
-0+ opts.prog_btf_fd = btf_fd; + opts.func_info = &func_infos; + opts.func_info_cnt =ARRAY_SIZE (func_infos); + opts.func_info_rec_size =sizeof (func_infos[0 ]); + + prog_fd =bpf_prog_load (BPF_PROG_TYPE_SYSCALL ,NULL ,"GPL" , insns, insn_cnt, &opts); +munmap (mapping, arena_size); +close (btf_fd); +close (map); + +return probe_fd (prog_fd); +} +ok, wow, that's quite a feature detector :)
but if we take a step back, we really are just trying to detect the change in
12a1fe6e12db("bpf/verifier: Do not limit maximum direct offset into arena map")(btw, do mention this commit somewhere in the comment in the feature detection function in the next revision)
which just did this:
kernel/bpf/verifier.c +0
-5@@ -21132,11 +21132,6 @@ static int resolve_pseudo_ldimm64(struct bpf_verifier_env *env) }else {u32 off = insn[1 ].imm; -if (off >=BPF_MAX_VAR_OFF ) { - verbose(env, "direct value offset of %u is not allowed\n", off); - return -EINVAL; - } -if (!map->ops->map_direct_value_addr) {verbose(env, "no direct value
access support for this map type\n");return -EINVAL;Now, setting up ARENA map is quite bothersome (as evidenced by your code above), but ARRAY with
BPF_F_MMAPABLEflag also allows direct value access, so let's use that to do the detection.Basically, create a small ARRAY map (no BTF required, I think), and try to verify a simple program that only does some map access with fixed large offset (e.g., at 4GB - 8 offset). With old kernels you'd expect this to fail with "direct value offset of %u is not allowed" message in the verifier log (you'll need to provide a small log buffer in the feature detection function, of course). But on newer ones we'll fail on "invalid access to map value pointer, value_size=%u off=%u".
Quoted 6 lines
This relies on verifier log messages, which isn't great, but we can adjust one of selftests to validate that libbpf does indeed put offset at non-zero offset, as that should always be true for newer kernels.
Does it make sense?
Ack, makes total sense.
Quoted 56 lines
pw-bot: cr
tools/lib/bpf/features.c static struct kern_feature_desc { · line 760
Quoted 9 lines
typedef int (*feature_probe_fn)(int /*
token_fd*/);static struct
kern_feature_cachefeature_cache;tools/lib/bpf/features.c +2
-0@@ -581,6 +756,9 @@ static struct kern_feature_desc { [FEAT_BTF_QMARK_DATASEC] = { "BTF DATASEC names starting from '?'", probe_kern_btf_qmark_datasec, }, + [FEAT_ARENA_GLOBAL_RELOC] = { + "Relocatable arena globals support", probe_kern_arena_global_reloc,we don't detect "relocatable arena globals support", we detect if ldimm64 allows full range offsets, so call the detector as such (i.e., what you detect, not what for you are doing that).
Quoted 39 lines
tools/lib/bpf/features.c +1
-0+ }, };bool feat_supported (struct kern_feature_cache *cache,enum kern_feature_id feat_id)tools/lib/bpf/libbpf.c +5
-2@@ -3009,8 +3009,11 @@ static int init_arena_map_data(struct bpf_object *obj, struct bpf_map *map,memcpy (obj->arena_data, data, data_sz); obj->arena_data_sz = data_sz; -/* place globals at the end of the arena */ - obj->arena_data_off = mmap_sz - data_alloc_sz; +/* place globals at the end of the arena (if supported) */ +if (kernel_supports (obj,FEAT_ARENA_GLOBAL_RELOC )) + obj->arena_data_off = mmap_sz - data_alloc_sz; +else + obj->arena_data_off =0 ;/* make bpf_map__init_value() work for ARENA maps */ map->mmaped = obj->arena_data;tools/lib/bpf/libbpf_internal.h +2
-1@@ -392,6 +392,8 @@ enum kern_feature_id {FEAT_ARG_CTX_TAG ,/* Kernel supports '?' at the front of datasec names */ FEAT_BTF_QMARK_DATASEC , +/* Kernel supports relocating arena globals to end of the arena */ +FEAT_ARENA_GLOBAL_RELOC , __FEAT_CNT, }; --2.49.0