Skip to contentSkip to conversation

bpf: Introduce global functions

PATCH Superseded Alexei Starovoitov 4 participants 14 messages Jan 9, 2020 – Jan 22, 2020

Subsystems (8)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Alexei Starovoitov

author maintainer
#0 lore raw

v2 cover bpf: Introduce global functions

v1->v2:
- addressed review comments from Song, Andrii, Yonghong
- fixed memory leak in error path
- added modified ctx check
- added more tests in patch 7

v1:
Introduce static vs global functions and function by function verification. This is another step toward dynamic re-linking (or replacement) of global functions. See patch 3 for details. The rest are supporting patches.

Alexei Starovoitov (7):
libbpf: Sanitize global functions
libbpf: Collect static vs global info about functions
bpf: Introduce function-by-function verification selftests/bpf: Add fexit-to-skb test for global funcs selftests/bpf: Add a test for a large global function selftests/bpf: Modify a test to check global functions selftests/bpf: Add unit tests for global functions

 include/linux/bpf.h                           |   7 +-
 include/linux/bpf_verifier.h                  |  10 +-
 include/uapi/linux/btf.h                      |   6 +
 kernel/bpf/btf.c                              | 175 +++++++++---
 kernel/bpf/verifier.c                         | 254 ++++++++++++++----
 tools/include/uapi/linux/btf.h                |   6 +
 tools/lib/bpf/btf.h                           |   5 +
 tools/lib/bpf/libbpf.c                        | 150 ++++++++++-
 .../bpf/prog_tests/bpf_verif_scale.c          |   2 +
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |   1 +
 .../bpf/prog_tests/test_global_funcs.c        |  81 ++++++
 .../selftests/bpf/progs/fexit_bpf2bpf.c       |  15 ++
 tools/testing/selftests/bpf/progs/pyperf.h    |   9 +-
 .../selftests/bpf/progs/pyperf_global.c       |   5 +
 .../selftests/bpf/progs/test_global_func1.c   |  45 ++++
 .../selftests/bpf/progs/test_global_func2.c   |   4 +
 .../selftests/bpf/progs/test_global_func3.c   |  65 +++++
 .../selftests/bpf/progs/test_global_func4.c   |   4 +
 .../selftests/bpf/progs/test_global_func5.c   |  31 +++
 .../selftests/bpf/progs/test_global_func6.c   |  31 +++
 .../selftests/bpf/progs/test_pkt_access.c     |  28 ++
 .../selftests/bpf/progs/test_xdp_noinline.c   |   4 +-
 22 files changed, 845 insertions(+), 93 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_global_funcs.c
 create mode 100644 tools/testing/selftests/bpf/progs/pyperf_global.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func1.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func2.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func3.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func4.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func5.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func6.c
Signature · 1 line
2.23.0
author maintainer
#2 lore raw

PATCH v2 2/7 libbpf: Collect static vs global info about functions

Collect static vs global information about BPF functions from ELF file and improve BTF with this additional info if compiler is too old and doesn't emit this information its own. There is no attempt to improve extern BTF.

---
 tools/lib/bpf/btf.h    |   5 ++
 tools/lib/bpf/libbpf.c | 115 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 116 insertions(+), 4 deletions(-)
tools/lib/bpf/btf.h +5 -0
index 8d73f7f5551f..3cbb95b2616e 100644
@@ -161,6 +161,11 @@ static inline __u16 btf_vlen(const struct btf_type *t)
 	return BTF_INFO_VLEN(t->info);
 }
 
+static inline __u16 btf_func_linkage(const struct btf_type *t)
+{
+	return BTF_INFO_VLEN(t->info);
+}
+
 static inline bool btf_kflag(const struct btf_type *t)
 {
 	return BTF_INFO_KFLAG(t->info);
tools/lib/bpf/libbpf.c +111 -4

180 lines not on this page · raw message

Signature · 1 line
2.23.0
author maintainer
#1 lore raw

PATCH v2 1/7 libbpf: Sanitize global functions

In reply to Alexei Starovoitov

In case the kernel doesn't support BTF_FUNC_GLOBAL sanitize BTF produced by the compiler for global functions.

---
 tools/include/uapi/linux/btf.h |  6 ++++++
 tools/lib/bpf/libbpf.c         | 35 +++++++++++++++++++++++++++++++++-
 2 files changed, 40 insertions(+), 1 deletion(-)
tools/include/uapi/linux/btf.h +6 -0
index 1a2898c482ee..5a667107ad2c 100644
@@ -146,6 +146,12 @@ enum {
 	BTF_VAR_GLOBAL_EXTERN = 2,
 };
 
+enum btf_func_linkage {
+	BTF_FUNC_STATIC = 0,
+	BTF_FUNC_GLOBAL = 1,
+	BTF_FUNC_EXTERN = 2,
+};
+
 /* BTF_KIND_VAR is followed by a single "struct btf_var" to describe
  * additional information related to the variable such as its linkage.
  */
tools/lib/bpf/libbpf.c +34 -1
index 7513165b104f..1039362a06a9 100644
@@ -166,6 +166,8 @@ struct bpf_capabilities {
 	__u32 btf_datasec:1;
 	/* BPF_F_MMAPABLE is supported for arrays */
 	__u32 array_mmap:1;
+	/* BTF_FUNC_GLOBAL is supported */
+	__u32 btf_func_global:1;
 };
 
 enum reloc_type {
@@ -1817,13 +1819,14 @@ static bool section_have_execinstr(struct bpf_object *obj, int idx)
 
 static void bpf_object__sanitize_btf(struct bpf_object *obj)
 {
+	bool has_func_global = obj->caps.btf_func_global;
 	bool has_datasec = obj->caps.btf_datasec;
 	bool has_func = obj->caps.btf_func;
 	struct btf *btf = obj->btf;
 	struct btf_type *t;
 	int i, j, vlen;
 
-	if (!obj->btf || (has_func && has_datasec))
+	if (!obj->btf || (has_func && has_datasec && has_func_global))
 		return;
 
 	for (i = 1; i <= btf__get_nr_types(btf); i++) {
@@ -1871,6 +1874,9 @@ static void bpf_object__sanitize_btf(struct bpf_object *obj)
 		} else if (!has_func && btf_is_func(t)) {
 			/* replace FUNC with TYPEDEF */
 			t->info = BTF_INFO_ENC(BTF_KIND_TYPEDEF, 0, 0);
+		} else if (!has_func_global && btf_is_func(t)) {
+			/* replace BTF_FUNC_GLOBAL with BTF_FUNC_STATIC */
+			t->info = BTF_INFO_ENC(BTF_KIND_FUNC, 0, 0);
 		}
 	}
 }
@@ -2804,6 +2810,32 @@ static int bpf_object__probe_btf_func(struct bpf_object *obj)
 	return 0;
 }
 
+static int bpf_object__probe_btf_func_global(struct bpf_object *obj)
+{
+	static const char strs[] = "\0int\0x\0a";
+	/* static void x(int a) {} */
+	__u32 types[] = {
+		/* int */
+		BTF_TYPE_INT_ENC(1, BTF_INT_SIGNED, 0, 32, 4),  /* [1] */
+		/* FUNC_PROTO */                                /* [2] */
+		BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_FUNC_PROTO, 0, 1), 0),
+		BTF_PARAM_ENC(7, 1),
+		/* FUNC x BTF_FUNC_GLOBAL */                    /* [3] */
+		BTF_TYPE_ENC(5, BTF_INFO_ENC(BTF_KIND_FUNC, 0, BTF_FUNC_GLOBAL), 2),
+	};
+	int btf_fd;
+
+	btf_fd = libbpf__load_raw_btf((char *)types, sizeof(types),
+				      strs, sizeof(strs));
+	if (btf_fd >= 0) {
+		obj->caps.btf_func_global = 1;
+		close(btf_fd);
+		return 1;
+	}
+
+	return 0;
+}
+
 static int bpf_object__probe_btf_datasec(struct bpf_object *obj)
 {
 	static const char strs[] = "\0x\0.data";
@@ -2859,6 +2891,7 @@ bpf_object__probe_caps(struct bpf_object *obj)
 		bpf_object__probe_name,
 		bpf_object__probe_global_data,
 		bpf_object__probe_btf_func,
+		bpf_object__probe_btf_func_global,
 		bpf_object__probe_btf_datasec,
 		bpf_object__probe_array_mmap,
 	};
Signature · 1 line
2.23.0
author maintainer
#8 lore raw

PATCH v2 5/7 selftests/bpf: Add a test for a large global function

In reply to Alexei Starovoitov

test results:
pyperf50 with always_inlined the same function five times: processed 46378 insns pyperf50 with global function: processed 6102 insns

---
 tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c | 2 ++
 tools/testing/selftests/bpf/progs/pyperf.h               | 9 +++++++--
 tools/testing/selftests/bpf/progs/pyperf_global.c        | 5 +++++
 3 files changed, 14 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/pyperf_global.c
tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c +2 -0
index 9486c13af6b2..e9f2f12ba06b 100644
@@ -48,6 +48,8 @@ void test_bpf_verif_scale(void)
 		{ "test_verif_scale2.o", BPF_PROG_TYPE_SCHED_CLS },
 		{ "test_verif_scale3.o", BPF_PROG_TYPE_SCHED_CLS },
 
+		{ "pyperf_global.o", BPF_PROG_TYPE_RAW_TRACEPOINT },
+
 		/* full unroll by llvm */
 		{ "pyperf50.o", BPF_PROG_TYPE_RAW_TRACEPOINT },
 		{ "pyperf100.o", BPF_PROG_TYPE_RAW_TRACEPOINT },
tools/testing/selftests/bpf/progs/pyperf.h +7 -2
index 71d383cc9b85..e186899954e9 100644
@@ -154,7 +154,12 @@ struct {
 	__uint(value_size, sizeof(long long) * 127);
 } stackmap SEC(".maps");
 
-static __always_inline int __on_event(struct pt_regs *ctx)
+#ifdef GLOBAL_FUNC
+__attribute__((noinline))
+#else
+static __always_inline
+#endif
+int __on_event(struct bpf_raw_tracepoint_args *ctx)
 {
 	uint64_t pid_tgid = bpf_get_current_pid_tgid();
 	pid_t pid = (pid_t)(pid_tgid >> 32);
@@ -254,7 +259,7 @@ static __always_inline int __on_event(struct pt_regs *ctx)
 }
 
 SEC("raw_tracepoint/kfree_skb")
-int on_event(struct pt_regs* ctx)
+int on_event(struct bpf_raw_tracepoint_args* ctx)
 {
 	int i, ret = 0;
 	ret |= __on_event(ctx);
tools/testing/selftests/bpf/progs/pyperf_global.c new file +5 -0
new file mode 100644
index 000000000000..079e78a7562b
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2020 Facebook */
+#define STACK_MAX_LEN 50
+#define GLOBAL_FUNC
+#include "pyperf.h"
Signature · 1 line
2.23.0
author maintainer
#7 lore raw

PATCH v2 4/7 selftests/bpf: Add fexit-to-skb test for global funcs

In reply to Alexei Starovoitov

Add simple fexit prog type to skb prog type test when subprogram is a global function.

---
 .../selftests/bpf/prog_tests/fexit_bpf2bpf.c  |  1 +
 .../selftests/bpf/progs/fexit_bpf2bpf.c       | 15 ++++++++++
 .../selftests/bpf/progs/test_pkt_access.c     | 28 +++++++++++++++++++
 3 files changed, 44 insertions(+)
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +1 -0
index b426bf2f97e4..7d3740d38965 100644
@@ -98,6 +98,7 @@ static void test_target_yes_callees(void)
 		"fexit/test_pkt_access",
 		"fexit/test_pkt_access_subprog1",
 		"fexit/test_pkt_access_subprog2",
+		"fexit/test_pkt_access_subprog3",
 	};
 	test_fexit_bpf2bpf_common("./fexit_bpf2bpf.o",
 				  "./test_pkt_access.o",
tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c +15 -0
index 2d211ee98a1c..81d7b4aaf79e 100644
@@ -79,4 +79,19 @@ int test_subprog2(struct args_subprog2 *ctx)
 	test_result_subprog2 = 1;
 	return 0;
 }
+
+__u64 test_result_subprog3 = 0;
+BPF_TRACE_3("fexit/test_pkt_access_subprog3", test_subprog3,
+	    int, val, struct sk_buff *, skb, int, ret)
+{
+	int len;
+
+	__builtin_preserve_access_index(({
+		len = skb->len;
+	}));
+	if (len != 74 || ret != 74 * val || val != 3)
+		return 0;
+	test_result_subprog3 = 1;
+	return 0;
+}
 char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/test_pkt_access.c +28 -0
index 3a7b4b607ed3..b77cebf71e66 100644
@@ -47,6 +47,32 @@ int test_pkt_access_subprog2(int val, volatile struct __sk_buff *skb)
 	return skb->len * val;
 }
 
+#define MAX_STACK (512 - 2 * 32)
+
+__attribute__ ((noinline))
+int get_skb_len(struct __sk_buff *skb)
+{
+	volatile char buf[MAX_STACK] = {};
+
+	return skb->len;
+}
+
+int get_skb_ifindex(int, struct __sk_buff *skb, int);
+
+__attribute__ ((noinline))
+int test_pkt_access_subprog3(int val, struct __sk_buff *skb)
+{
+	return get_skb_len(skb) * get_skb_ifindex(val, skb, 1);
+}
+
+__attribute__ ((noinline))
+int get_skb_ifindex(int val, struct __sk_buff *skb, int var)
+{
+	volatile char buf[MAX_STACK] = {};
+
+	return skb->ifindex * val * var;
+}
+
 SEC("classifier/test_pkt_access")
 int test_pkt_access(struct __sk_buff *skb)
 {
@@ -82,6 +108,8 @@ int test_pkt_access(struct __sk_buff *skb)
 		return TC_ACT_SHOT;
 	if (test_pkt_access_subprog2(2, skb) != skb->len * 2)
 		return TC_ACT_SHOT;
+	if (test_pkt_access_subprog3(3, skb) != skb->len * 3 * skb->ifindex)
+		return TC_ACT_SHOT;
 	if (tcp) {
 		if (((void *)(tcp) + 20) > data_end || proto != 6)
 			return TC_ACT_SHOT;
Signature · 1 line
2.23.0
author maintainer
#9 lore raw

PATCH v2 6/7 selftests/bpf: Modify a test to check global functions

In reply to Alexei Starovoitov

Make two static functions in test_xdp_noinline.c global:
before: processed 2790 insns
after: processed 2598 insns

---
 tools/testing/selftests/bpf/progs/test_xdp_noinline.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/test_xdp_noinline.c +2 -2
index e88d7b9d65ab..f95bc1a17667 100644
@@ -86,7 +86,7 @@ u32 jhash(const void *key, u32 length, u32 initval)
 	return c;
 }
 
-static __attribute__ ((noinline))
+__attribute__ ((noinline))
 u32 __jhash_nwords(u32 a, u32 b, u32 c, u32 initval)
 {
 	a += initval;
@@ -96,7 +96,7 @@ u32 __jhash_nwords(u32 a, u32 b, u32 c, u32 initval)
 	return c;
 }
 
-static __attribute__ ((noinline))
+__attribute__ ((noinline))
 u32 jhash_2words(u32 a, u32 b, u32 initval)
 {
 	return __jhash_nwords(a, b, 0, initval + JHASH_INITVAL + (2 << 2));
Signature · 1 line
2.23.0
author maintainer
#3 lore raw

PATCH v2 3/7 bpf: Introduce function-by-function verification

In reply to Alexei Starovoitov

New llvm and old llvm with libbpf help produce BTF that distinguish global and static functions. Unlike arguments of static function the arguments of global functions cannot be removed or optimized away by llvm. The compiler has to use exactly the arguments specified in a function prototype. The argument type information allows the verifier validate each global function independently. For now only supported argument types are pointer to context and scalars. In the future pointers to structures, sizes, pointer to packet data can be supported as well. Consider the following example:

static int f1(int ...)
{
...
}

int f3(int b);

int f2(int a)
{
f1(a) + f3(a);
}

int f3(int b)
{
...
}

int main(...)
{
f1(...) + f2(...) + f3(...);
}

The verifier will start its safety checks from the first global function f2(). It will recursively descend into f1() because it's static. Then it will check that arguments match for the f3() invocation inside f2(). It will not descend into f3(). It will finish f2() that has to be successfully verified for all possible values of 'a'. Then it will proceed with f3(). That function also has to be safe for all possible values of 'b'. Then it will start subprog 0 (which is main() function). It will recursively descend into f1() and will skip full check of f2() and f3(), since they are global. The order of processing global functions doesn't affect safety, since all global functions must be proven safe based on their arguments only.

Such function by function verification can drastically improve speed of the verification and reduce complexity.

Note that the stack limit of 512 still applies to the call chain regardless whether functions were static or global. The nested level of 8 also still applies. The same recursion prevention checks are in place as well.

The type information and static/global kind is preserved after the verification hence in the above example global function f2() and f3() can be replaced later by equivalent functions with the same types that are loaded and verified later without affecting safety of this main() program. Such replacement (re-linking) of global functions is a subject of future patches.

---
 include/linux/bpf.h          |   7 +-
 include/linux/bpf_verifier.h |  10 +-
 include/uapi/linux/btf.h     |   6 +
 kernel/bpf/btf.c             | 175 +++++++++++++++++++-----
 kernel/bpf/verifier.c        | 254 ++++++++++++++++++++++++++++-------
 5 files changed, 368 insertions(+), 84 deletions(-)
include/linux/bpf.h +6 -1
index b14e51d56a82..f47ef5718f3e 100644
@@ -558,6 +558,7 @@ static inline void bpf_dispatcher_change_prog(struct bpf_dispatcher *d,
 #endif
 
 struct bpf_func_info_aux {
+	u16 linkage;
 	bool unreliable;
 };
 
@@ -1006,7 +1007,11 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,
 			   const char *func_name,
 			   struct btf_func_model *m);
 
-int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog);
+struct bpf_reg_state;
+int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,
+			     struct bpf_reg_state *regs);
+int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog,
+			  struct bpf_reg_state *reg);
 
 struct bpf_prog *bpf_prog_by_id(u32 id);
 
include/linux/bpf_verifier.h +8 -2
index 26e40de9ef55..5406e6e96585 100644
@@ -304,11 +304,13 @@ struct bpf_insn_aux_data {
 	u64 map_key_state; /* constant (32 bit) key tracking for maps */
 	int ctx_field_size; /* the ctx field size for load insn, maybe 0 */
 	int sanitize_stack_off; /* stack slot to be cleared */
-	bool seen; /* this insn was processed by the verifier */
+	u32 seen; /* this insn was processed by the verifier at env->pass_cnt */
 	bool zext_dst; /* this insn zero extends dst reg */
 	u8 alu_state; /* used in combination with alu_limit */
-	bool prune_point;
+
+	/* below fields are initialized once */
 	unsigned int orig_idx; /* original instruction index */
+	bool prune_point;
 };
 
 #define MAX_USED_MAPS 64 /* max number of maps accessed by one eBPF program */
@@ -379,6 +381,7 @@ struct bpf_verifier_env {
 		int *insn_stack;
 		int cur_stack;
 	} cfg;
+	u32 pass_cnt; /* number of times do_check() was called */
 	u32 subprog_cnt;
 	/* number of instructions analyzed by the verifier */
 	u32 prev_insn_processed, insn_processed;
@@ -428,4 +431,7 @@ bpf_prog_offload_replace_insn(struct bpf_verifier_env *env, u32 off,
 void
 bpf_prog_offload_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt);
 
+int check_ctx_reg(struct bpf_verifier_env *env,
+		  const struct bpf_reg_state *reg, int regno);
+
 #endif /* _LINUX_BPF_VERIFIER_H */
include/uapi/linux/btf.h +6 -0
index 1a2898c482ee..5a667107ad2c 100644
@@ -146,6 +146,12 @@ enum {
 	BTF_VAR_GLOBAL_EXTERN = 2,
 };
 
+enum btf_func_linkage {
+	BTF_FUNC_STATIC = 0,
+	BTF_FUNC_GLOBAL = 1,
+	BTF_FUNC_EXTERN = 2,
+};
+
 /* BTF_KIND_VAR is followed by a single "struct btf_var" to describe
  * additional information related to the variable such as its linkage.
  */
kernel/bpf/btf.c +144 -31

252 lines not on this page · raw message

kernel/bpf/verifier.c +204 -50

368 lines not on this page · raw message

Signature · 1 line
2.23.0
author maintainer
#10 lore raw

PATCH v2 7/7 selftests/bpf: Add unit tests for global functions

In reply to Alexei Starovoitov

test_global_func[12] - check 512 stack limit. test_global_func[34] - check 8 frame call chain limit.
test_global_func5 - check that non-ctx pointer cannot be passed into
a function that expects context.
test_global_func6 - check that ctx pointer is unmodified.

---
 .../bpf/prog_tests/test_global_funcs.c        | 81 +++++++++++++++++++
 .../selftests/bpf/progs/test_global_func1.c   | 45 +++++++++++
 .../selftests/bpf/progs/test_global_func2.c   |  4 +
 .../selftests/bpf/progs/test_global_func3.c   | 65 +++++++++++++++
 .../selftests/bpf/progs/test_global_func4.c   |  4 +
 .../selftests/bpf/progs/test_global_func5.c   | 31 +++++++
 .../selftests/bpf/progs/test_global_func6.c   | 31 +++++++
 7 files changed, 261 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_global_funcs.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func1.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func2.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func3.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func4.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func5.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func6.c
tools/testing/selftests/bpf/prog_tests/test_global_funcs.c new file +81 -0
new file mode 100644
index 000000000000..bc588fa87d65
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2020 Facebook */
+#include <test_progs.h>
+
+const char *err_str;
+bool found;
+
+static int libbpf_debug_print(enum libbpf_print_level level,
+			      const char *format, va_list args)
+{
+	char *log_buf;
+
+	if (level != LIBBPF_WARN ||
+	    strcmp(format, "libbpf: \n%s\n")) {
+		vprintf(format, args);
+		return 0;
+	}
+
+	log_buf = va_arg(args, char *);
+	if (!log_buf)
+		goto out;
+	if (strstr(log_buf, err_str) == 0)
+		found = true;
+out:
+	printf(format, log_buf);
+	return 0;
+}
+
+extern int extra_prog_load_log_flags;
+
+static int check_load(const char *file)
+{
+	struct bpf_prog_load_attr attr;
+	struct bpf_object *obj = NULL;
+	int err, prog_fd;
+
+	memset(&attr, 0, sizeof(struct bpf_prog_load_attr));
+	attr.file = file;
+	attr.prog_type = BPF_PROG_TYPE_UNSPEC;
+	attr.log_level = extra_prog_load_log_flags;
+	attr.prog_flags = BPF_F_TEST_RND_HI32;
+	found = false;
+	err = bpf_prog_load_xattr(&attr, &obj, &prog_fd);
+	bpf_object__close(obj);
+	return err;
+}
+
+struct test_def {
+	const char *file;
+	const char *err_str;
+};
+
+void test_test_global_funcs(void)
+{
+	struct test_def tests[] = {
+		{ "test_global_func1.o", "combined stack size of 4 calls is 544" },
+		{ "test_global_func2.o" },
+		{ "test_global_func3.o" , "the call stack of 8 frames" },
+		{ "test_global_func4.o" },
+		{ "test_global_func5.o" , "expected pointer to ctx, but got PTR" },
+		{ "test_global_func6.o" , "modified ctx ptr R2" },
+	};
+	libbpf_print_fn_t old_print_fn = NULL;
+	int err, i, duration = 0;
+
+	old_print_fn = libbpf_set_print(libbpf_debug_print);
+
+	for (i = 0; i < ARRAY_SIZE(tests); i++) {
+		const struct test_def *test = &tests[i];
+
+		if (!test__start_subtest(test->file))
+			continue;
+
+		err_str = test->err_str;
+		err = check_load(test->file);
+		CHECK_FAIL(!!err ^ !!err_str);
+		if (err_str)
+			CHECK(found, "", "expected string '%s'", err_str);
+	}
+	libbpf_set_print(old_print_fn);
+}
tools/testing/selftests/bpf/progs/test_global_func1.c new file +45 -0

45 lines not on this page · raw message

tools/testing/selftests/bpf/progs/test_global_func2.c new file +4 -0
new file mode 100644
index 000000000000..2c18d82923a2
@@ -0,0 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2020 Facebook */
+#define MAX_STACK (512 - 3 * 32)
+#include "test_global_func1.c"
tools/testing/selftests/bpf/progs/test_global_func3.c new file +65 -0

65 lines not on this page · raw message

tools/testing/selftests/bpf/progs/test_global_func4.c new file +4 -0
new file mode 100644
index 000000000000..610f75edf276
@@ -0,0 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2020 Facebook */
+#define NO_FN8
+#include "test_global_func3.c"
tools/testing/selftests/bpf/progs/test_global_func5.c new file +31 -0
new file mode 100644
index 000000000000..86787c03cea8
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2020 Facebook */
+#include <stddef.h>
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+__attribute__ ((noinline))
+int f1(struct __sk_buff *skb)
+{
+	return skb->len;
+}
+
+int f3(int, struct __sk_buff *skb);
+
+__attribute__ ((noinline))
+int f2(int val, struct __sk_buff *skb)
+{
+	return f1(skb) + f3(val, (void *)&val); /* type mismatch */
+}
+
+__attribute__ ((noinline))
+int f3(int val, struct __sk_buff *skb)
+{
+	return skb->ifindex * val;
+}
+
+SEC("classifier/test")
+int test_cls(struct __sk_buff *skb)
+{
+	return f1(skb) + f2(2, skb) + f3(3, skb);
+}
tools/testing/selftests/bpf/progs/test_global_func6.c new file +31 -0
new file mode 100644
index 000000000000..e215fb3e6f02
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2020 Facebook */
+#include <stddef.h>
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+__attribute__ ((noinline))
+int f1(struct __sk_buff *skb)
+{
+	return skb->len;
+}
+
+int f3(int, struct __sk_buff *skb);
+
+__attribute__ ((noinline))
+int f2(int val, struct __sk_buff *skb)
+{
+	return f1(skb) + f3(val, skb + 1); /* type mismatch */
+}
+
+__attribute__ ((noinline))
+int f3(int val, struct __sk_buff *skb)
+{
+	return skb->ifindex * val;
+}
+
+SEC("classifier/test")
+int test_cls(struct __sk_buff *skb)
+{
+	return f1(skb) + f2(2, skb) + f3(3, skb);
+}
Signature · 1 line
2.23.0
reviewer Acked-by on v2 7/7
#11 lore raw

Re: [PATCH v2 bpf-next 7/7] selftests/bpf: Add unit tests for global functions

Quoted 8 lines · Alexei Starovoitov

test_global_func[12] - check 512 stack limit. test_global_func[34] - check 8 frame call chain limit.
test_global_func5 - check that non-ctx pointer cannot be passed into
a function that expects context.
test_global_func6 - check that ctx pointer is unmodified.

Quoted 40 lines
---
 .../bpf/prog_tests/test_global_funcs.c        | 81 +++++++++++++++++++
 .../selftests/bpf/progs/test_global_func1.c   | 45 +++++++++++
 .../selftests/bpf/progs/test_global_func2.c   |  4 +
 .../selftests/bpf/progs/test_global_func3.c   | 65 +++++++++++++++
 .../selftests/bpf/progs/test_global_func4.c   |  4 +
 .../selftests/bpf/progs/test_global_func5.c   | 31 +++++++
 .../selftests/bpf/progs/test_global_func6.c   | 31 +++++++
 7 files changed, 261 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_global_funcs.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func1.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func2.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func3.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func4.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func5.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func6.c
tools/testing/selftests/bpf/prog_tests/test_global_funcs.c new file +17 -0
new file mode 100644
index 000000000000..bc588fa87d65
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2020 Facebook */
+#include <test_progs.h>
+
+const char *err_str;
+bool found;
+
+static int libbpf_debug_print(enum libbpf_print_level level,
+                             const char *format, va_list args)
+{
+       char *log_buf;
+
+       if (level != LIBBPF_WARN ||
+           strcmp(format, "libbpf: \n%s\n")) {
+               vprintf(format, args);
+               return 0;
+       }

tools/testing/selftests/bpf/prog_tests/test_global_funcs.c line 27

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/test_global_funcs.c new file +10 -0
+
+       log_buf = va_arg(args, char *);
+       if (!log_buf)
+               goto out;
+       if (strstr(log_buf, err_str) == 0)
+               found = true;
+out:
+       printf(format, log_buf);
+       return 0;
+}

libbpf_debug_print() looks very useful. Maybe we can move it to some header files?

Thanks,
Song

Acked-by on v2 3/7
#4 lore raw

Re: [PATCH v2 bpf-next 3/7] bpf: Introduce function-by-function verification

In reply to Alexei Starovoitov
Quoted 3 lines · Alexei Starovoitov

[...]

Quoted 5 lines

Note that the stack limit of 512 still applies to the call chain regardless whether functions were static or global. The nested level of 8 also still applies. The same recursion prevention checks are in place as well.

Quoted 7 lines

The type information and static/global kind is preserved after the verification hence in the above example global function f2() and f3() can be replaced later by equivalent functions with the same types that are loaded and verified later without affecting safety of this main() program. Such replacement (re-linking) of global functions is a subject of future patches.

With one nit below.

[...]

Quoted 49 lines
kernel/bpf/verifier.c +49 -0
+
+static int do_check_common(struct bpf_verifier_env *env, int subprog)
+{
+	struct bpf_verifier_state *state;
+	struct bpf_reg_state *regs;
+	int ret, i;
+
+	env->prev_linfo = NULL;
+	env->pass_cnt++;
+
+	state = kzalloc(sizeof(struct bpf_verifier_state), GFP_KERNEL);
+	if (!state)
+		return -ENOMEM;
+	state->curframe = 0;
+	state->speculative = false;
+	state->branches = 1;
+	state->frame[0] = kzalloc(sizeof(struct bpf_func_state), GFP_KERNEL);
+	if (!state->frame[0]) {
+		kfree(state);
+		return -ENOMEM;
+	}
+	env->cur_state = state;
+	init_func_state(env, state->frame[0],
+			BPF_MAIN_FUNC /* callsite */,
+			0 /* frameno */,
+			subprog);
+
+	regs = state->frame[state->curframe]->regs;
+	if (subprog) {
+		ret = btf_prepare_func_args(env, subprog, regs);
+		if (ret)
+			goto out;
+		for (i = BPF_REG_1; i <= BPF_REG_5; i++) {
+			if (regs[i].type == PTR_TO_CTX)
+				mark_reg_known_zero(env, regs, i);
+			else if (regs[i].type == SCALAR_VALUE)
+				mark_reg_unknown(env, regs, i);
+		}
+	} else {
+		/* 1st arg to a function */
+		regs[BPF_REG_1].type = PTR_TO_CTX;
+		mark_reg_known_zero(env, regs, BPF_REG_1);
+		ret = btf_check_func_arg_match(env, subprog, regs);
+		if (ret == -EFAULT)
+			/* unlikely verifier bug. abort.
+			 * ret == 0 and ret < 0 are sadly acceptable for
+			 * main() function due to backward compatibility.
+			 * Like socket filter program may be written as:
+			 * int bpf_prog(struct pt_regs *ctx)

kernel/bpf/verifier.c static void free_states(struct bpf_verifier_env *env) · line 9572

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+			 * and never dereference that ctx in the program.
+			 * 'struct pt_regs' is a type mismatch for socket
+			 * filter that should be using 'struct __sk_buff'.
+			 */
+			goto out;
+	}
+
+	ret = do_check(env);
+out:
+	if (env->cur_state) {

I think env->cur_state will never be NULL here. This check is necessary before this patch (when we allocate cur_state in do_check()).

Thanks,
Song

#12 lore raw

Re: [PATCH v2 bpf-next 7/7] selftests/bpf: Add unit tests for global functions

In reply to Song Liu
Quoted 57 lines · Song Liu
Quoted 8 lines · Alexei Starovoitov

test_global_func[12] - check 512 stack limit. test_global_func[34] - check 8 frame call chain limit.
test_global_func5 - check that non-ctx pointer cannot be passed into
a function that expects context.
test_global_func6 - check that ctx pointer is unmodified.

Quoted 45 lines
---
 .../bpf/prog_tests/test_global_funcs.c        | 81 +++++++++++++++++++
 .../selftests/bpf/progs/test_global_func1.c   | 45 +++++++++++
 .../selftests/bpf/progs/test_global_func2.c   |  4 +
 .../selftests/bpf/progs/test_global_func3.c   | 65 +++++++++++++++
 .../selftests/bpf/progs/test_global_func4.c   |  4 +
 .../selftests/bpf/progs/test_global_func5.c   | 31 +++++++
 .../selftests/bpf/progs/test_global_func6.c   | 31 +++++++
 7 files changed, 261 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_global_funcs.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func1.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func2.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func3.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func4.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func5.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_global_func6.c
tools/testing/selftests/bpf/prog_tests/test_global_funcs.c new file +22 -0
new file mode 100644
index 000000000000..bc588fa87d65
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2020 Facebook */
+#include <test_progs.h>
+
+const char *err_str;
+bool found;
+
+static int libbpf_debug_print(enum libbpf_print_level level,
+                             const char *format, va_list args)
+{
+       char *log_buf;
+
+       if (level != LIBBPF_WARN ||
+           strcmp(format, "libbpf: \n%s\n")) {
+               vprintf(format, args);
+               return 0;
+       }
+
+       log_buf = va_arg(args, char *);
+       if (!log_buf)
+               goto out;
+       if (strstr(log_buf, err_str) == 0)
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/test_global_funcs.c line 27

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/test_global_funcs.c new file +5 -0
+               found = true;
+out:
+       printf(format, log_buf);
+       return 0;
+}

libbpf_debug_print() looks very useful. Maybe we can move it to some header files?

I think it's hack that goes deep into libbpf internals that should be discouraged. It's clearly very useful for selftests, but imo libbpf's log_buf api should be redesigned instead. It's imo the worst part of the library.

#5 lore raw

Re: [PATCH v2 bpf-next 3/7] bpf: Introduce function-by-function verification

In reply to Song Liu
Quoted 79 lines · Song Liu
Quoted 3 lines · Alexei Starovoitov

[...]

Quoted 5 lines

Note that the stack limit of 512 still applies to the call chain regardless whether functions were static or global. The nested level of 8 also still applies. The same recursion prevention checks are in place as well.

Quoted 7 lines

The type information and static/global kind is preserved after the verification hence in the above example global function f2() and f3() can be replaced later by equivalent functions with the same types that are loaded and verified later without affecting safety of this main() program. Such replacement (re-linking) of global functions is a subject of future patches.

With one nit below.

[...]

Quoted 54 lines
kernel/bpf/verifier.c +54 -0
+
+static int do_check_common(struct bpf_verifier_env *env, int subprog)
+{
+	struct bpf_verifier_state *state;
+	struct bpf_reg_state *regs;
+	int ret, i;
+
+	env->prev_linfo = NULL;
+	env->pass_cnt++;
+
+	state = kzalloc(sizeof(struct bpf_verifier_state), GFP_KERNEL);
+	if (!state)
+		return -ENOMEM;
+	state->curframe = 0;
+	state->speculative = false;
+	state->branches = 1;
+	state->frame[0] = kzalloc(sizeof(struct bpf_func_state), GFP_KERNEL);
+	if (!state->frame[0]) {
+		kfree(state);
+		return -ENOMEM;
+	}
+	env->cur_state = state;
+	init_func_state(env, state->frame[0],
+			BPF_MAIN_FUNC /* callsite */,
+			0 /* frameno */,
+			subprog);
+
+	regs = state->frame[state->curframe]->regs;
+	if (subprog) {
+		ret = btf_prepare_func_args(env, subprog, regs);
+		if (ret)
+			goto out;
+		for (i = BPF_REG_1; i <= BPF_REG_5; i++) {
+			if (regs[i].type == PTR_TO_CTX)
+				mark_reg_known_zero(env, regs, i);
+			else if (regs[i].type == SCALAR_VALUE)
+				mark_reg_unknown(env, regs, i);
+		}
+	} else {
+		/* 1st arg to a function */
+		regs[BPF_REG_1].type = PTR_TO_CTX;
+		mark_reg_known_zero(env, regs, BPF_REG_1);
+		ret = btf_check_func_arg_match(env, subprog, regs);
+		if (ret == -EFAULT)
+			/* unlikely verifier bug. abort.
+			 * ret == 0 and ret < 0 are sadly acceptable for
+			 * main() function due to backward compatibility.
+			 * Like socket filter program may be written as:
+			 * int bpf_prog(struct pt_regs *ctx)
+			 * and never dereference that ctx in the program.
+			 * 'struct pt_regs' is a type mismatch for socket
+			 * filter that should be using 'struct __sk_buff'.
+			 */
+			goto out;
Quoted 8 lines

kernel/bpf/verifier.c static void free_states(struct bpf_verifier_env *env) · line 9572

Quoted 5 lines
kernel/bpf/verifier.c +5 -0
+	}
+
+	ret = do_check(env);
+out:
+	if (env->cur_state) {

I think env->cur_state will never be NULL here. This check is necessary before this patch (when we allocate cur_state in do_check()).

yeah. good catch. 'if' can be dropped. I'll follow up with a clean up patch or will fold it if respin is necessary for other reasons.

#13 lore raw

Re: [PATCH v2 bpf-next 7/7] selftests/bpf: Add unit tests for global functions

In reply to Alexei Starovoitov
Quoted 71 lines · Alexei Starovoitov
Quoted 57 lines · Song Liu
Quoted 8 lines · Alexei Starovoitov

test_global_func[12] - check 512 stack limit. test_global_func[34] - check 8 frame call chain limit.
test_global_func5 - check that non-ctx pointer cannot be passed into
a function that expects context.
test_global_func6 - check that ctx pointer is unmodified.

Quoted 45 lines

---

.../bpf/prog_tests/test_global_funcs.c        | 81 +++++++++++++++++++
.../selftests/bpf/progs/test_global_func1.c   | 45 +++++++++++
.../selftests/bpf/progs/test_global_func2.c   |  4 +
.../selftests/bpf/progs/test_global_func3.c   | 65 +++++++++++++++
.../selftests/bpf/progs/test_global_func4.c   |  4 +
.../selftests/bpf/progs/test_global_func5.c   | 31 +++++++
.../selftests/bpf/progs/test_global_func6.c   | 31 +++++++
7 files changed, 261 insertions(+)

create mode 100644 tools/testing/selftests/bpf/prog_tests/test_global_funcs.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_func1.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_func2.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_func3.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_func4.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_func5.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_func6.c

tools/testing/selftests/bpf/prog_tests/test_global_funcs.c new file +22 -0
new file mode 100644
index 000000000000..bc588fa87d65
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2020 Facebook */
+#include <test_progs.h>
+
+const char *err_str;
+bool found;
+
+static int libbpf_debug_print(enum libbpf_print_level level,
+                             const char *format, va_list args)
+{
+       char *log_buf;
+
+       if (level != LIBBPF_WARN ||
+           strcmp(format, "libbpf: \n%s\n")) {
+               vprintf(format, args);
+               return 0;
+       }
+
+       log_buf = va_arg(args, char *);
+       if (!log_buf)
+               goto out;
+       if (strstr(log_buf, err_str) == 0)
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/test_global_funcs.c line 27

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/test_global_funcs.c new file +5 -0
+               found = true;
+out:
+       printf(format, log_buf);
+       return 0;
+}

libbpf_debug_print() looks very useful. Maybe we can move it to some header files?

Quoted 3 lines

I think it's hack that goes deep into libbpf internals that should be discouraged. It's clearly very useful for selftests, but imo libbpf's log_buf api should be redesigned instead. It's imo the worst part of the library.

Yeah, those global variables don't look good.

Song

#6 lore raw

Re: [PATCH v2 bpf-next 3/7] bpf: Introduce function-by-function verification

In reply to Alexei Starovoitov
Quoted 10 lines · Alexei Starovoitov
Quoted 7 lines

kernel/bpf/verifier.c static void free_states(struct bpf_verifier_env *env) · line 9572

Quoted 4 lines
kernel/bpf/verifier.c +4 -0
+
+   ret = do_check(env);
+out:
+   if (env->cur_state) {

I think env->cur_state will never be NULL here. This check is necessary before this patch (when we allocate cur_state in do_check()).

yeah. good catch. 'if' can be dropped. I'll follow up with a clean up patch or will fold it if respin is necessary for other reasons.

that is the case of code review gone wrong.
This dropped 'if' during code move (because we both felt that it's unnecessary) is the reason for syzbot panic under fault injection:
https://lore.kernel.org/lkml/[email protected]/
I'm sending a fix for this shortly to restore that 'if' check. This time adding a comment :)