Skip to contentSkip to conversation

libbpf: fix CO-RE relocs against .text section

PATCH In review Andrii Nakryiko 3 participants 4 messages Jun 19, 2020 – Jun 24, 2020

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 libbpf: fix CO-RE relocs against .text section

bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.

Adjust one of bpf_iter selftest to exhibit this pattern.

Fixes ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")
---
 tools/lib/bpf/libbpf.c                               | 8 +++++++-
 tools/testing/selftests/bpf/progs/bpf_iter_netlink.c | 2 +-
 2 files changed, 8 insertions(+), 2 deletions(-)
tools/lib/bpf/libbpf.c +7 -1
index 477c679ed945..f17151d866e6 100644
@@ -4818,7 +4818,13 @@ bpf_core_reloc_fields(struct bpf_object *obj, const char *targ_btf_path)
 			err = -EINVAL;
 			goto out;
 		}
-		prog = bpf_object__find_program_by_title(obj, sec_name);
+		prog = NULL;
+		for (i = 0; i < obj->nr_programs; i++) {
+			if (!strcmp(obj->programs[i].section_name, sec_name)) {
+				prog = &obj->programs[i];
+				break;
+			}
+		}
 		if (!prog) {
 			pr_warn("failed to find program '%s' for CO-RE offset relocation\n",
 				sec_name);
tools/testing/selftests/bpf/progs/bpf_iter_netlink.c +1 -1
index e7b8753eac0b..75ecf956a2df 100644
@@ -25,7 +25,7 @@ struct bpf_iter__netlink {
 	struct netlink_sock *sk;
 } __attribute__((preserve_access_index));
 
-static inline struct inode *SOCK_INODE(struct socket *socket)
+static __attribute__((noinline)) struct inode *SOCK_INODE(struct socket *socket)
 {
 	return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
 }
Signature · 1 line
2.24.1
Acked-by on v1 1/1
#1 lore raw
Quoted 10 lines · Andrii Nakryiko

bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.

Adjust one of bpf_iter selftest to exhibit this pattern.

Fixes ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")

But the fix here only fixed the issue for interpreter mode. For jit only mode, we still have issues. The following patch can fix the jit mode issue,

=============

From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001
From: Yonghong Song <[email protected]>
Date: Fri, 19 Jun 2020 23:26:13 -0700
Subject: [PATCH bpf] bpf: set the number of exception entries properly for
subprograms

Currently, if a bpf program has more than one subprograms, each program will be jitted separately. For tracing problem, the prog->aux->num_exentries is not setup properly. For example, with bpf_iter_netlink.c modified to force one function not inlined, and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON, we will have error like below:

  $ ./test_progs -n 3/3
  ...
  libbpf: failed to load program 'iter/netlink'
  libbpf: failed to load object 'bpf_iter_netlink'
  libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007
  test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton 
open_and_load failed
  #3/3 netlink:FAIL
The dmesg shows the following errors:
  ex gen bug
which is triggered by the following code in arch/x86/net/bpf_jit_comp.c:
  if (excnt >= bpf_prog->aux->num_exentries) {
    pr_err("ex gen bug\n");
    return -EFAULT;
  }

If the program has more than one subprograms, num_exentries is actually 0 since it is not setup.

This patch fixed the issue by setuping proper num_exentries for each subprogram before calling jit function.

---
 kernel/bpf/verifier.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +11 -1
index 34cde841ab68..7d8b23ba825c 100644
@@ -9801,7 +9801,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 	int i, j, subprog_start, subprog_end = 0, len, subprog;
 	struct bpf_insn *insn;
 	void *old_bpf_func;
-	int err;
+	int err, num_exentries;
 
 	if (env->subprog_cnt <= 1)
 		return 0;
@@ -9876,6 +9876,16 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 		func[i]->aux->nr_linfo = prog->aux->nr_linfo;
 		func[i]->aux->jited_linfo = prog->aux->jited_linfo;
 		func[i]->aux->linfo_idx = env->subprog_info[i].linfo_idx;
+
+		num_exentries = 0;
+		insn = func[i]->insnsi;
+		for (j = 0; j < func[i]->len; j++, insn++) {
+			if (BPF_CLASS(insn->code) == BPF_LDX &&
+			    BPF_MODE(insn->code) == BPF_PROBE_MEM)
+				num_exentries++;
+		}
+		func[i]->aux->num_exentries = num_exentries;
+
 		func[i] = bpf_int_jit_compile(func[i]);
 		if (!func[i]->jited) {
 			err = -ENOTSUPP;
Signature · 9 lines
2.24.1

================

We need this (or similar fixes) go in together with libbpf fix
to avoid bpf_iter_netlink.c test failure at jit only mode.

Do we need a separate patch for the above fix? Or Andrii can
fold this into his patch and resubmit? Maybe the latter is better.
Quoted 38 lines
---
  tools/lib/bpf/libbpf.c                               | 8 +++++++-
  tools/testing/selftests/bpf/progs/bpf_iter_netlink.c | 2 +-
  2 files changed, 8 insertions(+), 2 deletions(-)
tools/lib/bpf/libbpf.c +7 -1
index 477c679ed945..f17151d866e6 100644
@@ -4818,7 +4818,13 @@ bpf_core_reloc_fields(struct bpf_object *obj, const char *targ_btf_path)
  			err = -EINVAL;
  			goto out;
  		}
-		prog = bpf_object__find_program_by_title(obj, sec_name);
+		prog = NULL;
+		for (i = 0; i < obj->nr_programs; i++) {
+			if (!strcmp(obj->programs[i].section_name, sec_name)) {
+				prog = &obj->programs[i];
+				break;
+			}
+		}
  		if (!prog) {
  			pr_warn("failed to find program '%s' for CO-RE offset relocation\n",
  				sec_name);
tools/testing/selftests/bpf/progs/bpf_iter_netlink.c +1 -1
index e7b8753eac0b..75ecf956a2df 100644
@@ -25,7 +25,7 @@ struct bpf_iter__netlink {
  	struct netlink_sock *sk;
  } __attribute__((preserve_access_index));
 
-static inline struct inode *SOCK_INODE(struct socket *socket)
+static __attribute__((noinline)) struct inode *SOCK_INODE(struct socket *socket)
  {
  	return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
  }
#2 lore raw
Quoted 51 lines · Yonghong Song
Quoted 10 lines · Andrii Nakryiko

bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.

Adjust one of bpf_iter selftest to exhibit this pattern.

Fixes ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")

But the fix here only fixed the issue for interpreter mode. For jit only mode, we still have issues. The following patch can fix the jit mode issue,

=============

From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001
From: Yonghong Song <[email protected]>
Date: Fri, 19 Jun 2020 23:26:13 -0700
Subject: [PATCH bpf] bpf: set the number of exception entries properly for
subprograms

Currently, if a bpf program has more than one subprograms, each program will be jitted separately. For tracing problem, the prog->aux->num_exentries is not setup properly. For example, with bpf_iter_netlink.c modified to force one function not inlined, and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON, we will have error like below:

   $ ./test_progs -n 3/3
   ...
   libbpf: failed to load program 'iter/netlink'
   libbpf: failed to load object 'bpf_iter_netlink'
   libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007
   test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton
open_and_load failed
   #3/3 netlink:FAIL
The dmesg shows the following errors:
   ex gen bug
which is triggered by the following code in arch/x86/net/bpf_jit_comp.c:
   if (excnt >= bpf_prog->aux->num_exentries) {
     pr_err("ex gen bug\n");
     return -EFAULT;
   }
Quoted 7 lines

If the program has more than one subprograms, num_exentries is actually 0 since it is not setup.

This patch fixed the issue by setuping proper num_exentries for each subprogram before calling jit function.

Thanks for fixing. Applied both to bpf tree. Yonghong, next time please submit the patch properly. It was very awkward to copy-paste it manually from the thread. I've edited the commit log a bit.

#3 lore raw
Quoted 58 lines · Alexei Starovoitov
Quoted 57 lines · Yonghong Song
Quoted 10 lines · Andrii Nakryiko

bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually.

Adjust one of bpf_iter selftest to exhibit this pattern.

Fixes ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")

But the fix here only fixed the issue for interpreter mode. For jit only mode, we still have issues. The following patch can fix the jit mode issue,

=============

From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001
From: Yonghong Song <[email protected]>
Date: Fri, 19 Jun 2020 23:26:13 -0700
Subject: [PATCH bpf] bpf: set the number of exception entries properly for
subprograms

Currently, if a bpf program has more than one subprograms, each program will be jitted separately. For tracing problem, the prog->aux->num_exentries is not setup properly. For example, with bpf_iter_netlink.c modified to force one function not inlined, and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON, we will have error like below:

    $ ./test_progs -n 3/3
    ...
    libbpf: failed to load program 'iter/netlink'
    libbpf: failed to load object 'bpf_iter_netlink'
    libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007
    test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton
open_and_load failed
    #3/3 netlink:FAIL
The dmesg shows the following errors:
    ex gen bug
which is triggered by the following code in arch/x86/net/bpf_jit_comp.c:
    if (excnt >= bpf_prog->aux->num_exentries) {
      pr_err("ex gen bug\n");
      return -EFAULT;
    }

If the program has more than one subprograms, num_exentries is actually 0 since it is not setup.

This patch fixed the issue by setuping proper num_exentries for each subprogram before calling jit function.

Quoted 6 lines
Quoted 1 line

Thanks for fixing. Applied both to bpf tree. Yonghong, next time please submit the patch properly. It was very awkward to copy-paste it manually from the thread. I've edited the commit log a bit.

Thanks. I posted original commit as I am not sure how to proceed as
this and Andrii's patch belongs to the same patch set to fix bpf_iter_netlink problem. I guess next time I will go ahead with patch submit with proper description in the patch, which
sounds better for review and to get notice from other people.