libbpf: fix CO-RE relocs against .text section
Subsystems (7)
PATCH 1/1 libbpf: fix CO-RE relocs against .text section
bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.
Adjust one of bpf_iter selftest to exhibit this pattern.
--- tools/lib/bpf/libbpf.c | 8 +++++++- tools/testing/selftests/bpf/progs/bpf_iter_netlink.c | 2 +- 2 files changed, 8 insertions(+), 2 deletions(-)
tools/lib/bpf/libbpf.c +7 -1
@@ -4818,7 +4818,13 @@ bpf_core_reloc_fields(struct bpf_object *obj, const char *targ_btf_path) err = -EINVAL ; goto out; } - prog =bpf_object__find_program_by_title (obj, sec_name); + prog =NULL ; +for (i =0 ; i < obj->nr_programs; i++) { +if (!strcmp (obj->programs[i].section_name, sec_name)) { + prog = &obj->programs[i]; +break ; + } + }if (!prog) {pr_warn ("failed to find program '%s' for CO-RE offset relocation\n" , sec_name);
tools/testing/selftests/bpf/progs/bpf_iter_netlink.c +1 -1
@@ -25,7 +25,7 @@ struct bpf_iter__netlink {struct netlink_sock *sk; } __attribute__((preserve_access_index)); -static inline struct inode *SOCK_INODE (struct socket *socket) +static __attribute__((noinline))struct inode *SOCK_INODE (struct socket *socket) { return &container_of(socket, struct socket_alloc, socket)->vfs_inode; }
Signature · 1 line
2.24.1
Quoted 10 lines · Andrii Nakryiko
bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.Adjust one of
bpf_iterselftest to exhibit this pattern.Reported-by Yonghong Song <[email protected]>Fixes ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")Signed-off-by Andrii Nakryiko <[email protected]>
But the fix here only fixed the issue for interpreter mode. For jit only mode, we still have issues. The following patch can fix the jit mode issue,
=============
From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001
From: Yonghong Song <[email protected]>
Date: Fri, 19 Jun 2020 23:26:13 -0700
Subject: [PATCH bpf] bpf: set the number of exception entries properly for
subprograms
Currently, if a bpf program has more than one subprograms, each program will be jitted separately. For tracing problem, the prog->aux->num_exentries is not setup properly. For example, with bpf_iter_netlink.c modified to force one function not inlined, and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON, we will have error like below:
$ ./test_progs -n 3/3
...
libbpf: failed to load program 'iter/netlink'
libbpf: failed to load object 'bpf_iter_netlink'
libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007
test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton
open_and_load failed
#3/3 netlink:FAIL
The dmesg shows the following errors:
ex gen bug
which is triggered by the following code in arch/x86/net/bpf_jit_comp.c:
if (excnt >= bpf_prog->aux->num_exentries) {
pr_err("ex gen bug\n");
return -EFAULT;
}If the program has more than one subprograms, num_exentries is actually 0 since it is not setup.
This patch fixed the issue by setuping proper num_exentries for each subprogram before calling jit function.
--- kernel/bpf/verifier.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +11 -1
@@ -9801,7 +9801,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)int i, j, subprog_start, subprog_end =0 , len, subprog;struct bpf_insn *insn;void *old_bpf_func; -int err; +int err, num_exentries;if (env->subprog_cnt <=1 )return 0 ;
@@ -9876,6 +9876,16 @@ static int jit_subprogs(struct bpf_verifier_env *env) func[i]->aux->nr_linfo = prog->aux->nr_linfo; func[i]->aux->jited_linfo = prog->aux->jited_linfo; func[i]->aux->linfo_idx = env->subprog_info[i].linfo_idx; + + num_exentries =0 ; + insn = func[i]->insnsi; +for (j =0 ; j < func[i]->len; j++, insn++) { +if (BPF_CLASS (insn->code) ==BPF_LDX && +BPF_MODE (insn->code) ==BPF_PROBE_MEM ) + num_exentries++; + } + func[i]->aux->num_exentries = num_exentries; + func[i] =bpf_int_jit_compile (func[i]);if (!func[i]->jited) { err = -ENOTSUPP ;
Signature · 9 lines
2.24.1 ================ We need this (or similar fixes) go in together with libbpf fix to avoid bpf_iter_netlink.c test failure at jit only mode. Do we need a separate patch for the above fix? Or Andrii can fold this into his patch and resubmit? Maybe the latter is better.
Quoted 38 lines
--- tools/lib/bpf/libbpf.c | 8 +++++++- tools/testing/selftests/bpf/progs/bpf_iter_netlink.c | 2 +- 2 files changed, 8 insertions(+), 2 deletions(-)tools/lib/bpf/libbpf.c +7
-1@@ -4818,7 +4818,13 @@ bpf_core_reloc_fields(struct bpf_object *obj, const char *targ_btf_path) err = -EINVAL ; goto out; } - prog =bpf_object__find_program_by_title (obj, sec_name); + prog =NULL ; +for (i =0 ; i < obj->nr_programs; i++) { +if (!strcmp (obj->programs[i].section_name, sec_name)) { + prog = &obj->programs[i]; +break ; + } + }if (!prog) {pr_warn ("failed to find program '%s' for CO-RE offset relocation\n" , sec_name);tools/testing/selftests/bpf/progs/bpf_iter_netlink.c +1
-1@@ -25,7 +25,7 @@ struct bpf_iter__netlink {struct netlink_sock *sk; } __attribute__((preserve_access_index)); -static inline struct inode *SOCK_INODE (struct socket *socket) +static __attribute__((noinline))struct inode *SOCK_INODE (struct socket *socket) { return &container_of(socket, struct socket_alloc, socket)->vfs_inode; }
Quoted 51 lines · Yonghong Song
Quoted 10 lines · Andrii Nakryiko
bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.Adjust one of
bpf_iterselftest to exhibit this pattern.Reported-by Yonghong Song <[email protected]>Fixes ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")Signed-off-by Andrii Nakryiko <[email protected]>Acked-by Yonghong Song <[email protected]>But the fix here only fixed the issue for interpreter mode. For jit only mode, we still have issues. The following patch can fix the jit mode issue,
=============
From
4d66814513ec45b86a30a1231b8a000d4bfc6f1aMon Sep 17 00:00:00 2001
From: Yonghong Song <[email protected]>
Date: Fri, 19 Jun 2020 23:26:13 -0700
Subject: [PATCH bpf] bpf: set the number of exception entries properly for
subprogramsCurrently, if a bpf program has more than one subprograms, each program will be jitted separately. For tracing problem, the
prog->aux->num_exentriesis not setup properly. For example, withbpf_iter_netlink.cmodified to force one function not inlined, and with proper libbpf fix, withCONFIG_BPF_JIT_ALWAYS_ON, we will have error like below:$ ./test_progs -n 3/3 ... libbpf: failed to load program 'iter/netlink' libbpf: failed to load object 'bpf_iter_netlink' libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007 test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton open_and_load failed #3/3 netlink:FAIL The dmesg shows the following errors: ex gen bug which is triggered by the following code in arch/x86/net/bpf_jit_comp.c: if (excnt >= bpf_prog->aux->num_exentries) { pr_err("ex gen bug\n"); return -EFAULT; }
Quoted 7 lines
If the program has more than one subprograms,
num_exentriesis actually 0 since it is not setup.This patch fixed the issue by setuping proper
num_exentriesfor each subprogram before calling jit function.Signed-off-by Yonghong Song <[email protected]>
Thanks for fixing. Applied both to bpf tree. Yonghong, next time please submit the patch properly. It was very awkward to copy-paste it manually from the thread. I've edited the commit log a bit.
Quoted 58 lines · Alexei Starovoitov
Quoted 57 lines · Yonghong Song
Quoted 10 lines · Andrii Nakryiko
bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.Adjust one of
bpf_iterselftest to exhibit this pattern.Reported-by Yonghong Song <[email protected]>Fixes ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")Signed-off-by Andrii Nakryiko <[email protected]>Acked-by Yonghong Song <[email protected]>But the fix here only fixed the issue for interpreter mode. For jit only mode, we still have issues. The following patch can fix the jit mode issue,
=============
From
4d66814513ec45b86a30a1231b8a000d4bfc6f1aMon Sep 17 00:00:00 2001
From: Yonghong Song <[email protected]>
Date: Fri, 19 Jun 2020 23:26:13 -0700
Subject: [PATCH bpf] bpf: set the number of exception entries properly for
subprogramsCurrently, if a bpf program has more than one subprograms, each program will be jitted separately. For tracing problem, the
prog->aux->num_exentriesis not setup properly. For example, withbpf_iter_netlink.cmodified to force one function not inlined, and with proper libbpf fix, withCONFIG_BPF_JIT_ALWAYS_ON, we will have error like below:$ ./test_progs -n 3/3 ... libbpf: failed to load program 'iter/netlink' libbpf: failed to load object 'bpf_iter_netlink' libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007 test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton open_and_load failed #3/3 netlink:FAIL The dmesg shows the following errors: ex gen bug which is triggered by the following code in arch/x86/net/bpf_jit_comp.c: if (excnt >= bpf_prog->aux->num_exentries) { pr_err("ex gen bug\n"); return -EFAULT; }If the program has more than one subprograms,
num_exentriesis actually 0 since it is not setup.This patch fixed the issue by setuping proper
num_exentriesfor each subprogram before calling jit function.
Quoted 6 lines
Quoted 1 line
Signed-off-by Yonghong Song <[email protected]>Thanks for fixing. Applied both to bpf tree. Yonghong, next time please submit the patch properly. It was very awkward to copy-paste it manually from the thread. I've edited the commit log a bit.
Thanks. I posted original commit as I am not sure how to proceed as
this and Andrii's patch belongs to the same patch set to fix bpf_iter_netlink problem. I guess next time I will go ahead with patch submit with proper description in the patch, which
sounds better for review and to get notice from other people.