bpf: Program extensions or dynamic re-linking
Subsystems (7)
Alexei Starovoitov
maintainerv2 cover bpf: Program extensions or dynamic re-linking
The last few month BPF community has been discussing an approach to call chaining, since exiting bpt_tail_call() mechanism used in production XDP programs has plenty of downsides. The outcome of these discussion was a conclusion to implement dynamic re-linking of BPF programs. Where rootlet XDP program attached to a netdevice can programmatically define a policy of execution of other XDP programs. Such rootlet would be compiled as normal XDP program and provide a number of placeholder global functions which later can be replaced with future XDP programs. BPF trampoline, function by function verification were building blocks towards that goal. The patch 1 is a final building block. It introduces dynamic program extensions. A number of improvements like more flexible function by function verification and better libbpf api will be implemented in future patches.
v1->v2:
- addressed Andrii's comments
- rebase
Alexei Starovoitov (3):
bpf: Introduce dynamic program extensions
libbpf: Add support for program extensions
selftests/bpf: Add tests for program extensions
include/linux/bpf.h | 10 +- include/linux/bpf_types.h | 2 + include/linux/btf.h | 5 + include/uapi/linux/bpf.h | 1 + kernel/bpf/btf.c | 152 +++++++++++++++++- kernel/bpf/syscall.c | 15 +- kernel/bpf/trampoline.c | 41 ++++- kernel/bpf/verifier.c | 85 +++++++--- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/bpf.c | 3 +- tools/lib/bpf/libbpf.c | 13 +- tools/lib/bpf/libbpf.h | 2 + tools/lib/bpf/libbpf.map | 2 + tools/lib/bpf/libbpf_probes.c | 1 + .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 20 ++- .../selftests/bpf/progs/fexit_bpf2bpf.c | 57 +++++++ .../selftests/bpf/progs/test_pkt_access.c | 8 +- 17 files changed, 384 insertions(+), 34 deletions(-)
Signature · 1 line
2.23.0
Alexei Starovoitov
maintainerPATCH v2 3/3 selftests/bpf: Add tests for program extensions
Add program extension tests that build on top of fexit_bpf2bpf tests. Replace three global functions in previously loaded test_pkt_access.c program with three new implementations:
int get_skb_len(struct __sk_buff *skb);
int get_constant(long val);
int get_skb_ifindex(int val, struct __sk_buff *skb, int var);
New function return the same results as original only if arguments match.
new_get_skb_ifindex() demonstrates that 'skb' argument doesn't have to be first and only argument of BPF program. All normal skb based accesses are available.
--- .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 20 ++++++- .../selftests/bpf/progs/fexit_bpf2bpf.c | 57 +++++++++++++++++++ .../selftests/bpf/progs/test_pkt_access.c | 8 ++- 3 files changed, 83 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c +19 -1
@@ -26,7 +26,7 @@ static void test_fexit_bpf2bpf_common(const char *obj_file, link =calloc (sizeof (struct bpf_link *), prog_cnt); prog =calloc (sizeof (struct bpf_program *), prog_cnt); - result =malloc (prog_cnt *sizeof (u64)); + result =malloc ((prog_cnt +32 /* spare */ ) *sizeof (u64));if (CHECK (!link || !prog || !result,"alloc_memory" ,"failed to alloc memory" )) goto close_prog;
@@ -106,8 +106,26 @@ static void test_target_yes_callees(void) prog_name); } +static void test_func_replace (void ) +{ +const char *prog_name[] = { +"fexit/test_pkt_access" , +"fexit/test_pkt_access_subprog1" , +"fexit/test_pkt_access_subprog2" , +"fexit/test_pkt_access_subprog3" , +"freplace/get_skb_len" , +"freplace/get_skb_ifindex" , +"freplace/get_constant" , + }; +test_fexit_bpf2bpf_common ("./fexit_bpf2bpf.o" , +"./test_pkt_access.o" , +ARRAY_SIZE (prog_name), + prog_name); +} +void test_fexit_bpf2bpf (void ) {test_target_no_callees ();test_target_yes_callees (); +test_func_replace (); }
tools/testing/selftests/bpf/progs/fexit_bpf2bpf.c +57 -0
68 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_pkt_access.c +7 -1
@@ -57,12 +57,18 @@ int get_skb_len(struct __sk_buff *skb)return skb->len; } +__attribute__ ((noinline)) +int get_constant (long val) +{ +return val -122 ; +} +int get_skb_ifindex (int ,struct __sk_buff *skb,int ); __attribute__ ((noinline))int test_pkt_access_subprog3 (int val,struct __sk_buff *skb) { -return get_skb_len (skb) *get_skb_ifindex (val, skb,1 ); +return get_skb_len (skb) *get_skb_ifindex (val, skb,get_constant (123 )); } __attribute__ ((noinline))
Signature · 1 line
2.23.0
Alexei Starovoitov
maintainerPATCH v2 1/3 bpf: Introduce dynamic program extensions
In reply to Alexei StarovoitovIntroduce dynamic program extensions. The users can load additional BPF functions and replace global functions in previously loaded BPF programs while these programs are executing.
Global functions are verified individually by the verifier based on their types only. Hence the global function in the new program which types match older function can safely replace that corresponding function.
This new function/program is called 'an extension' of old program. At load time the verifier uses (attach_prog_fd, attach_btf_id) pair to identify the function to be replaced. The BPF program type is derived from the target program into extension program. Technically bpf_verifier_ops is copied from target program. The BPF_PROG_TYPE_EXT program type is a placeholder. It has empty verifier_ops. The extension program can call the same bpf helper functions as target program. Single BPF_PROG_TYPE_EXT type is used to extend XDP, SKB and all other program types. The verifier allows only one level of replacement. Meaning that the extension program cannot recursively extend an extension. That also means that the maximum stack size is increasing from 512 to 1024 bytes and maximum function nesting level from 8 to 16. The programs don't always consume that much. The stack usage is determined by the number of on-stack variables used by the program. The verifier could have enforced 512 limit for combined original plus extension program, but it makes for difficult user experience. The main use case for extensions is to provide generic mechanism to plug external programs into policy program or function call chaining.
BPF trampoline is used to track both fentry/fexit and program extensions because both are using the same nop slot at the beginning of every BPF function. Attaching fentry/fexit to a function that was replaced is not allowed. The opposite is true as well. Replacing a function that currently being analyzed with fentry/fexit is not allowed. The executable page allocated by BPF trampoline is not used by program extensions. This inefficiency will be optimized in future patches.
Function by function verification of global function supports scalars and pointer to context only. Hence program extensions are supported for such class of global functions only. In the future the verifier will be extended with support to pointers to structures, arrays with sizes, etc.
--- include/linux/bpf.h | 10 ++- include/linux/bpf_types.h | 2 + include/linux/btf.h | 5 ++ include/uapi/linux/bpf.h | 1 + kernel/bpf/btf.c | 152 +++++++++++++++++++++++++++++++++++++- kernel/bpf/syscall.c | 15 +++- kernel/bpf/trampoline.c | 41 +++++++++- kernel/bpf/verifier.c | 85 ++++++++++++++++----- 8 files changed, 283 insertions(+), 28 deletions(-)
include/linux/bpf.h +9 -1
@@ -465,7 +465,8 @@ void notrace __bpf_prog_exit(struct bpf_prog *prog, u64 start);enum bpf_tramp_prog_type {BPF_TRAMP_FENTRY ,BPF_TRAMP_FEXIT , -BPF_TRAMP_MAX +BPF_TRAMP_MAX , +BPF_TRAMP_REPLACE ,/* more than MAX */ };struct bpf_trampoline {
@@ -480,6 +481,11 @@ struct bpf_trampoline {void *addr;bool ftrace_managed; } func; +/* if !NULL this is BPF_PROG_TYPE_EXT program that extends another BPF +* program by replacing one of its functions. func.addr is the address +* of the function it replaced. +*/ +struct bpf_prog *extension_prog;/* list of BPF programs using this trampoline */ struct hlist_head progs_hlist[BPF_TRAMP_MAX ];/* Number of attached programs. A counter per kind. */
@@ -1107,6 +1113,8 @@ int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog, struct bpf_reg_state *regs);int btf_prepare_func_args (struct bpf_verifier_env *env,int subprog,struct bpf_reg_state *reg); +int btf_check_type_match (struct bpf_verifier_env *env,struct bpf_prog *prog, +struct btf *btf,const struct btf_type *t);struct bpf_prog *bpf_prog_by_id (u32 id);
include/linux/bpf_types.h +2 -0
@@ -68,6 +68,8 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_SK_REUSEPORT, sk_reuseport,#if defined(CONFIG_BPF_JIT )BPF_PROG_TYPE (BPF_PROG_TYPE_STRUCT_OPS , bpf_struct_ops, void *, void *) +BPF_PROG_TYPE(BPF_PROG_TYPE_EXT, bpf_extension, + void *, void *) #endif BPF_MAP_TYPE(BPF_MAP_TYPE_ARRAY, array_map_ops)
include/linux/btf.h +5 -0
@@ -107,6 +107,11 @@ static inline u16 btf_type_vlen(const struct btf_type *t)return BTF_INFO_VLEN (t->info); } +static inline u16 btf_func_linkage (const struct btf_type *t) +{ +return BTF_INFO_VLEN (t->info); +} +static inline bool btf_type_kflag (const struct btf_type *t) {return BTF_INFO_KFLAG (t->info);
include/uapi/linux/bpf.h +1 -0
@@ -180,6 +180,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_CGROUP_SOCKOPT ,BPF_PROG_TYPE_TRACING ,BPF_PROG_TYPE_STRUCT_OPS , + BPF_PROG_TYPE_EXT, }; enum bpf_attach_type {
kernel/bpf/btf.c +151 -1
182 lines not on this page · raw message
kernel/bpf/syscall.c +12 -3
@@ -1932,13 +1932,15 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type,switch (prog_type) {case BPF_PROG_TYPE_TRACING :case BPF_PROG_TYPE_STRUCT_OPS : +case BPF_PROG_TYPE_EXT :break ;default :return -EINVAL ; } } -if (prog_fd && prog_type !=BPF_PROG_TYPE_TRACING ) +if (prog_fd && prog_type !=BPF_PROG_TYPE_TRACING && + prog_type !=BPF_PROG_TYPE_EXT )return -EINVAL ;switch (prog_type) {
@@ -1981,6 +1983,10 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type,default :return -EINVAL ; } +case BPF_PROG_TYPE_EXT : +if (expected_attach_type) +return -EINVAL ; +/* fallthrough */ default :return 0 ; }
@@ -2183,7 +2189,8 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog)int tr_fd, err;if (prog->expected_attach_type !=BPF_TRACE_FENTRY && - prog->expected_attach_type !=BPF_TRACE_FEXIT ) { + prog->expected_attach_type !=BPF_TRACE_FEXIT && + prog->type !=BPF_PROG_TYPE_EXT ) { err = -EINVAL ; goto out_put_prog; }
@@ -2250,12 +2257,14 @@ static int bpf_raw_tracepoint_open(const union bpf_attr *attr)if (prog->type !=BPF_PROG_TYPE_RAW_TRACEPOINT && prog->type !=BPF_PROG_TYPE_TRACING && + prog->type !=BPF_PROG_TYPE_EXT && prog->type !=BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE ) { err = -EINVAL ; goto out_put_prog; } -if (prog->type ==BPF_PROG_TYPE_TRACING ) { +if (prog->type ==BPF_PROG_TYPE_TRACING || + prog->type ==BPF_PROG_TYPE_EXT ) {if (attr->raw_tracepoint.name) {/* The attach point for this category of programs * should be specified via btf_id during program load.
kernel/bpf/trampoline.c +38 -3
73 lines not on this page · raw message
kernel/bpf/verifier.c +65 -20
147 lines not on this page · raw message
Signature · 1 line
2.23.0
Alexei Starovoitov
maintainerPATCH v2 2/3 libbpf: Add support for program extensions
In reply to Alexei StarovoitovAdd minimal support for program extensions. bpf_object_open_opts() needs to be called with attach_prog_fd = target_prog_fd and BPF program extension needs to have in .c file section definition like SEC("freplace/func_to_be_replaced"). libbpf will search for "func_to_be_replaced" in the target_prog_fd's BTF and will pass it in attach_btf_id to the kernel. This approach works for tests, but more compex use case may need to request function name (and attach_btf_id that kernel sees) to be more dynamic. Such API will be added in future patches.
--- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/bpf.c | 3 ++- tools/lib/bpf/libbpf.c | 13 ++++++++++--- tools/lib/bpf/libbpf.h | 2 ++ tools/lib/bpf/libbpf.map | 2 ++ tools/lib/bpf/libbpf_probes.c | 1 + 6 files changed, 18 insertions(+), 4 deletions(-)
tools/include/uapi/linux/bpf.h +1 -0
@@ -180,6 +180,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_CGROUP_SOCKOPT ,BPF_PROG_TYPE_TRACING ,BPF_PROG_TYPE_STRUCT_OPS , + BPF_PROG_TYPE_EXT, }; enum bpf_attach_type {
tools/lib/bpf/bpf.c +2 -1
@@ -237,7 +237,8 @@ int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr, attr.expected_attach_type = load_attr->expected_attach_type;if (attr.prog_type ==BPF_PROG_TYPE_STRUCT_OPS ) { attr.attach_btf_id = load_attr->attach_btf_id; - }else if (attr.prog_type ==BPF_PROG_TYPE_TRACING ) { + }else if (attr.prog_type ==BPF_PROG_TYPE_TRACING || + attr.prog_type ==BPF_PROG_TYPE_EXT ) { attr.attach_btf_id = load_attr->attach_btf_id; attr.attach_prog_fd = load_attr->attach_prog_fd; }else {
tools/lib/bpf/libbpf.c +10 -3
@@ -4837,7 +4837,8 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt, load_attr.license = license;if (prog->type ==BPF_PROG_TYPE_STRUCT_OPS ) { load_attr.attach_btf_id = prog->attach_btf_id; - }else if (prog->type ==BPF_PROG_TYPE_TRACING ) { + }else if (prog->type ==BPF_PROG_TYPE_TRACING || + prog->type ==BPF_PROG_TYPE_EXT ) { load_attr.attach_prog_fd = prog->attach_prog_fd; load_attr.attach_btf_id = prog->attach_btf_id; }else {
@@ -4918,7 +4919,8 @@ int bpf_program__load(struct bpf_program *prog, char *license, __u32 kern_ver) {int err =0 , fd, i, btf_id; -if (prog->type ==BPF_PROG_TYPE_TRACING ) { +if (prog->type ==BPF_PROG_TYPE_TRACING || + prog->type ==BPF_PROG_TYPE_EXT ) { btf_id =libbpf_find_attach_btf_id (prog);if (btf_id <=0 )return btf_id;
@@ -5092,7 +5094,8 @@ __bpf_object__open(const char *path, const void *obj_buf, size_t obj_buf_sz,bpf_program__set_type (prog, prog_type);bpf_program__set_expected_attach_type (prog, attach_type); -if (prog_type ==BPF_PROG_TYPE_TRACING ) +if (prog_type ==BPF_PROG_TYPE_TRACING || + prog_type ==BPF_PROG_TYPE_EXT ) prog->attach_prog_fd =OPTS_GET (opts, attach_prog_fd,0 ); }
@@ -6166,6 +6169,7 @@ BPF_PROG_TYPE_FNS(xdp, BPF_PROG_TYPE_XDP);BPF_PROG_TYPE_FNS (perf_event,BPF_PROG_TYPE_PERF_EVENT );BPF_PROG_TYPE_FNS (tracing,BPF_PROG_TYPE_TRACING );BPF_PROG_TYPE_FNS (struct_ops,BPF_PROG_TYPE_STRUCT_OPS ); +BPF_PROG_TYPE_FNS (extension,BPF_PROG_TYPE_EXT );enum bpf_attach_type bpf_program__get_expected_attach_type (struct bpf_program *prog)
@@ -6265,6 +6269,9 @@ static const struct bpf_sec_def section_defs[] = { .expected_attach_type = BPF_TRACE_FEXIT, .is_attach_btf = true, .attach_fn = attach_trace), +SEC_DEF ("freplace/" ,EXT , + .is_attach_btf = true, + .attach_fn = attach_trace),BPF_PROG_SEC ("xdp" ,BPF_PROG_TYPE_XDP ),BPF_PROG_SEC ("perf_event" ,BPF_PROG_TYPE_PERF_EVENT ), BPF_PROG_SEC("lwt_in", BPF_PROG_TYPE_LWT_IN),
tools/lib/bpf/libbpf.h +2 -0
@@ -318,6 +318,7 @@ LIBBPF_API int bpf_program__set_xdp(struct bpf_program *prog); LIBBPF_API int bpf_program__set_perf_event(struct bpf_program *prog); LIBBPF_API int bpf_program__set_tracing(struct bpf_program *prog); LIBBPF_API int bpf_program__set_struct_ops(struct bpf_program *prog); +LIBBPF_API int bpf_program__set_extension(struct bpf_program *prog); LIBBPF_API enum bpf_prog_type bpf_program__get_type(struct bpf_program *prog); LIBBPF_API void bpf_program__set_type(struct bpf_program *prog,
@@ -339,6 +340,7 @@ LIBBPF_API bool bpf_program__is_xdp(const struct bpf_program *prog); LIBBPF_API bool bpf_program__is_perf_event(const struct bpf_program *prog); LIBBPF_API bool bpf_program__is_tracing(const struct bpf_program *prog); LIBBPF_API bool bpf_program__is_struct_ops(const struct bpf_program *prog); +LIBBPF_API bool bpf_program__is_extension(const struct bpf_program *prog);/* * No need for __attribute__((packed)), all members of 'bpf_map_def'
tools/lib/bpf/libbpf.map +2 -0
@@ -228,7 +228,9 @@ LIBBPF_0.0.7 { bpf_prog_attach_xattr; bpf_program__attach; bpf_program__name; + bpf_program__is_extension; bpf_program__is_struct_ops; + bpf_program__set_extension; bpf_program__set_struct_ops; btf__align_of; libbpf_find_kernel_btf;
tools/lib/bpf/libbpf_probes.c +1 -0
@@ -107,6 +107,7 @@ probe_load(enum bpf_prog_type prog_type, const struct bpf_insn *insns,case BPF_PROG_TYPE_CGROUP_SOCKOPT :case BPF_PROG_TYPE_TRACING :case BPF_PROG_TYPE_STRUCT_OPS : +case BPF_PROG_TYPE_EXT :default :break ; }
Signature · 1 line
2.23.0
John Fastabend
reviewerRE: [PATCH v2 bpf-next 1/3] bpf: Introduce dynamic program extensions
In reply to Alexei StarovoitovQuoted 40 lines · Alexei Starovoitov
Introduce dynamic program extensions. The users can load additional BPF functions and replace global functions in previously loaded BPF programs while these programs are executing.
Global functions are verified individually by the verifier based on their types only. Hence the global function in the new program which types match older function can safely replace that corresponding function.
This new function/program is called 'an extension' of old program. At load time the verifier uses (
attach_prog_fd,attach_btf_id) pair to identify the function to be replaced. The BPF program type is derived from the target program into extension program. Technicallybpf_verifier_opsis copied from target program. TheBPF_PROG_TYPE_EXTprogram type is a placeholder. It has emptyverifier_ops. The extension program can call the same bpf helper functions as target program. SingleBPF_PROG_TYPE_EXTtype is used to extend XDP, SKB and all other program types. The verifier allows only one level of replacement. Meaning that the extension program cannot recursively extend an extension. That also means that the maximum stack size is increasing from 512 to 1024 bytes and maximum function nesting level from 8 to 16. The programs don't always consume that much. The stack usage is determined by the number of on-stack variables used by the program. The verifier could have enforced 512 limit for combined original plus extension program, but it makes for difficult user experience. The main use case for extensions is to provide generic mechanism to plug external programs into policy program or function call chaining.BPF trampoline is used to track both fentry/fexit and program extensions because both are using the same nop slot at the beginning of every BPF function. Attaching fentry/fexit to a function that was replaced is not allowed. The opposite is true as well. Replacing a function that currently being analyzed with fentry/fexit is not allowed. The executable page allocated by BPF trampoline is not used by program extensions. This inefficiency will be optimized in future patches.
Function by function verification of global function supports scalars and pointer to context only. Hence program extensions are supported for such class of global functions only. In the future the verifier will be extended with support to pointers to structures, arrays with sizes, etc.
Signed-off-by Alexei Starovoitov <[email protected]>---
[...]
kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 4170
Quoted 3 lines
kernel/bpf/btf.c +3
-0+ + t1 =btf_type_skip_modifiers (btf1, t1->type,NULL ); + t2 =btf_type_skip_modifiers (btf2, t2->type,NULL );
Is it really best to skip modifiers? I would expect that if the signature is different including modifiers then we should just reject it. OTOH its not really C code here either so modifiers may not have the same meaning. With just integers and struct it may be ok but if we add pointers to ints then what would we expect from a const int*?
So whats the reasoning for skipping modifiers? Is it purely an argument that its not required for safety so solve it elsewhere? In that case then checking names of functions is also equally not required.
Otherwise LGTM.
Quoted 69 lines
kernel/bpf/btf.c +69
-0+if (t1->info != t2->info) { +bpf_log (log, +"Return type %s of %s() doesn't match type %s of %s()\n" , +btf_type_str (t1), fn1, +btf_type_str (t2), fn2); +return -EINVAL ; + } + +for (i =0 ; i < nargs1; i++) { + t1 =btf_type_skip_modifiers (btf1, args1[i].type,NULL ); + t2 =btf_type_skip_modifiers (btf2, args2[i].type,NULL ); + +if (t1->info != t2->info) { +bpf_log (log,"arg%d in %s() is %s while %s() has %s\n" , + i, fn1,btf_type_str (t1), + fn2,btf_type_str (t2)); +return -EINVAL ; + } +if (btf_type_has_size (t1) && t1->size != t2->size) { +bpf_log (log, +"arg%d in %s() has size %d while %s() has %d\n" , + i, fn1, t1->size, + fn2, t2->size); +return -EINVAL ; + } + +/* global functions are validated with scalars and pointers +* to context only. And only global functions can be replaced. +* Hence type check only those types. +*/ +if (btf_type_is_int (t1) ||btf_type_is_enum (t1)) +continue ; +if (!btf_type_is_ptr (t1)) { +bpf_log (log, +"arg%d in %s() has unrecognized type\n" , + i, fn1); +return -EINVAL ; + } + t1 =btf_type_skip_modifiers (btf1, t1->type,NULL ); + t2 =btf_type_skip_modifiers (btf2, t2->type,NULL ); +if (!btf_type_is_struct (t1)) { +bpf_log (log, +"arg%d in %s() is not a pointer to context\n" , + i, fn1); +return -EINVAL ; + } +if (!btf_type_is_struct (t2)) { +bpf_log (log, +"arg%d in %s() is not a pointer to context\n" , + i, fn2); +return -EINVAL ; + } +/* This is an optional check to make program writing easier. +* Compare names of structs and report an error to the user. +* btf_prepare_func_args() already checked that t2 struct +* is a context type. btf_prepare_func_args() will check +* later that t1 struct is a context type as well. +*/ + s1 =btf_name_by_offset (btf1, t1->name_off); + s2 =btf_name_by_offset (btf2, t2->name_off); +if (strcmp (s1, s2)) { +bpf_log (log, +"arg%d %s(struct %s *) doesn't match %s(struct %s *)\n" , + i, fn1, s1, fn2, s2); +return -EINVAL ; + } + } +return 0 ; +}
Quoted 4 lines · Alexei Starovoitov
The last few month BPF community has been discussing an approach to call chaining, since exiting
bpt_tail_call()mechanism used in production XDP programs has plenty of downsides. The outcome of these discussion was a conclusion to implement dynamic re-linking of BPF programs. Where rootlet XDP
Quoted 8 lines
program attached to a netdevice can programmatically define a policy of execution of other XDP programs. Such rootlet would be compiled as normal XDP program and provide a number of placeholder global functions which later can be replaced with future XDP programs. BPF trampoline, function by function verification were building blocks towards that goal. The patch 1 is a final building block. It introduces dynamic program extensions. A number of improvements like more flexible function by function verification and better libbpf api will be implemented in future patches.
This is great, thank you! I'll go play around with it; couldn't spot anything obvious from eye-balling the code, except that yeah, it does need a more flexible libbpf api :)
One thing that's not obvious to me: How can userspace tell which programs replace which functions after they are loaded? Is this put into prog_tags in struct bpf_prog_info, or?
For the series:
-Toke
Re: [PATCH v2 bpf-next 1/3] bpf: Introduce dynamic program extensions
In reply to John FastabendQuoted 11 lines · John Fastabend
kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 4170
Quoted 4 lines
kernel/bpf/btf.c +3
-0+ + t1 =btf_type_skip_modifiers (btf1, t1->type,NULL ); + t2 =btf_type_skip_modifiers (btf2, t2->type,NULL );Is it really best to skip modifiers? I would expect that if the signature is different including modifiers then we should just reject it. OTOH its not really C code here either so modifiers may not have the same meaning. With just integers and struct it may be ok but if we add pointers to ints then what would we expect from a const int*?
Quoted 3 lines
So whats the reasoning for skipping modifiers? Is it purely an argument that its not required for safety so solve it elsewhere? In that case then checking names of functions is also equally not required.
Function names are not checked by the kernel. It's purely libbpf and bpf_prog.c convention. The kernel operates on prog_fd+btf_id only. The names of function arguments are not compared either.
The code has to skip modifiers. Otherwise the type comparison algorithm will be quite complex, since typedef is such modifier. Like 'u32' in original program and 'u32' in extension program would have to be recursively checked.
Another reason to skip modifiers is 'volatile' modifier. I suspect we would have to use it from time to time in original placeholder functions. Yet new replacement function will be written without volatile. The placeholder may need volatile to make sure compiler doesn't optimize things away. I found cases where 'noinline' in placeholder was not enough. clang would still inline the body of the function and remove call instruction. So far I've been using volatile as a workaround. May be we will introduce new function attribute to clang.
Having said that I share your concern regarding skipping 'const'. For 'const int arg' it's totally ok to skip it, since it's meaningless from safety pov, but for 'const int *arg' and 'const struct foo *arg' I'm planning to preserve it. It will be preserved at the verifier bpf_reg_state level though. Just checking that 'const' is present in extension prog's BTF doesn't help safety. I'm planing to make the verifier enforce that bpf prog cannot write into argument which type is pointer to const struct. That part is still wip. It will be implemented for global functions first and then for extension programs. Currently the verifier rejects any pointer to struct (other than context), so no backward compatibility issues.
Quoted 15 lines · Toke Høiland-Jørgensen
Quoted 4 lines · Alexei Starovoitov
The last few month BPF community has been discussing an approach to call chaining, since exiting
bpt_tail_call()mechanism used in production XDP programs has plenty of downsides. The outcome of these discussion was a conclusion to implement dynamic re-linking of BPF programs. Where rootlet XDPQuoted 8 lines
program attached to a netdevice can programmatically define a policy of execution of other XDP programs. Such rootlet would be compiled as normal XDP program and provide a number of placeholder global functions which later can be replaced with future XDP programs. BPF trampoline, function by function verification were building blocks towards that goal. The patch 1 is a final building block. It introduces dynamic program extensions. A number of improvements like more flexible function by function verification and better libbpf api will be implemented in future patches.
Quoted 7 lines
This is great, thank you! I'll go play around with it; couldn't spot anything obvious from eye-balling the code, except that yeah, it does need a more flexible libbpf api :)
One thing that's not obvious to me: How can userspace tell which programs replace which functions after they are loaded? Is this put into
prog_tagsin structbpf_prog_info, or?
good point. Would be good to extend bpf_prog_info. Since prog-to-prog connection is unidirectional the bpf_prog_info of extension prog will be able to say which original program it's replacing. bpftool prog show will be able to print all this data. I think fenry/fexit progs would need the same bpf_prog_info extension. attach_prog_id + attach_btf_id would be enough. In the mean time I can try to hack drgn script to do the same.
Re: [PATCH v2 bpf-next 1/3] bpf: Introduce dynamic program extensions
In reply to Alexei StarovoitovQuoted 12 lines · Alexei Starovoitov
Quoted 11 lines · John Fastabend
kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 4170
Quoted 4 lines
kernel/bpf/btf.c +3
-0+ + t1 =btf_type_skip_modifiers (btf1, t1->type,NULL ); + t2 =btf_type_skip_modifiers (btf2, t2->type,NULL );Is it really best to skip modifiers? I would expect that if the signature is different including modifiers then we should just reject it. OTOH its not really C code here either so modifiers may not have the same meaning. With just integers and struct it may be ok but if we add pointers to ints then what would we expect from a const int*?
Quoted 7 lines
Quoted 3 lines
So whats the reasoning for skipping modifiers? Is it purely an argument that its not required for safety so solve it elsewhere? In that case then checking names of functions is also equally not required.
Function names are not checked by the kernel. It's purely libbpf and
bpf_prog.cconvention. The kernel operates on prog_fd+btf_id only. The names of function arguments are not compared either.
Sorry mistyped names of struct is what I meant, but that is probably nice to have per comment.
Quoted 5 lines
The code has to skip modifiers. Otherwise the type comparison algorithm will be quite complex, since typedef is such modifier. Like 'u32' in original program and 'u32' in extension program would have to be recursively checked.
Quoted 8 lines
Another reason to skip modifiers is 'volatile' modifier. I suspect we would have to use it from time to time in original placeholder functions. Yet new replacement function will be written without volatile. The placeholder may need volatile to make sure compiler doesn't optimize things away. I found cases where 'noinline' in placeholder was not enough. clang would still inline the body of the function and remove call instruction. So far I've been using volatile as a workaround. May be we will introduce new function attribute to clang.
Yes, we have various similar issue and have in the past used volatile to work around them but volatile's inside loops tends to break loop optimizations and cause clang warnings/errors. Another common one is verifier failing to track when scalars move around in registers. As an example the following is valid C for a bounded additon to array pointer but not tractable for the verifier at the moment. (made example at some point I'll dig up a collection of real-world examples)
r1 = *(u64 *)(r10 - 8)
r6 = r1
if r6 < %[const] goto %l[err]
r3 += r1
r2 = %[copy_size]
r1 = r7
call 4compiler barriers help but not always and also breaks loop optimization passes. But, thats a different discussion I only mention it because either verifier has to track above logic better or new attributes in clang could be used for these things. But the new attributes don't usually work well when mixed with optimization passes that we would actually like to keep.
Quoted 11 lines
Having said that I share your concern regarding skipping 'const'. For 'const int arg' it's totally ok to skip it, since it's meaningless from safety pov, but for 'const int *arg' and 'const struct foo *arg' I'm planning to preserve it. It will be preserved at the verifier
bpf_reg_statelevel though. Just checking that 'const' is present in extension prog's BTF doesn't help safety. I'm planing to make the verifier enforce that bpf prog cannot write into argument which type is pointer to const struct. That part is still wip. It will be implemented for global functions first and then for extension programs. Currently the verifier rejects any pointer to struct (other than context), so no backward compatibility issues.
Ah ok this will be great. In that case const will be more general then merely functions and should be applicable generally at least as an end goal IMO. There will be a slight annoyance where old extensions may not run on new kernels though. I will argue such extensions are broken though.
For this patch then,
Quoted 42 lines · Alexei Starovoitov
The last few month BPF community has been discussing an approach to call chaining, since exiting
bpt_tail_call()mechanism used in production XDP programs has plenty of downsides. The outcome of these discussion was a conclusion to implement dynamic re-linking of BPF programs. Where rootlet XDP program attached to a netdevice can programmatically define a policy of execution of other XDP programs. Such rootlet would be compiled as normal XDP program and provide a number of placeholder global functions which later can be replaced with future XDP programs. BPF trampoline, function by function verification were building blocks towards that goal. The patch 1 is a final building block. It introduces dynamic program extensions. A number of improvements like more flexible function by function verification and better libbpf api will be implemented in future patches.
v1->v2:
- addressed Andrii's comments
- rebaseAlexei Starovoitov (3):
bpf: Introduce dynamic program extensions
libbpf: Add support for program extensions
selftests/bpf: Add tests for program extensionsinclude/linux/bpf.h | 10 +- include/linux/bpf_types.h | 2 + include/linux/btf.h | 5 + include/uapi/linux/bpf.h | 1 + kernel/bpf/btf.c | 152 +++++++++++++++++- kernel/bpf/syscall.c | 15 +- kernel/bpf/trampoline.c | 41 ++++- kernel/bpf/verifier.c | 85 +++++++--- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/bpf.c | 3 +- tools/lib/bpf/libbpf.c | 13 +- tools/lib/bpf/libbpf.h | 2 + tools/lib/bpf/libbpf.map | 2 + tools/lib/bpf/libbpf_probes.c | 1 + .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 20 ++- .../selftests/bpf/progs/fexit_bpf2bpf.c | 57 +++++++ .../selftests/bpf/progs/test_pkt_access.c | 8 +- 17 files changed, 384 insertions(+), 34 deletions(-)
Quoted 3 lines
--
2.23.0
LGTM.
For the series:
RE: [PATCH v2 bpf-next 2/3] libbpf: Add support for program extensions
In reply to Alexei StarovoitovQuoted 10 lines · Alexei Starovoitov
Add minimal support for program extensions.
bpf_object_open_opts()needs to be called withattach_prog_fd=target_prog_fdand BPF program extension needs to have in .c file section definition likeSEC("freplace/func_to_be_replaced"). libbpf will search for "func_to_be_replaced" in the target_prog_fd's BTF and will pass it inattach_btf_idto the kernel. This approach works for tests, but more compex use case may need to request function name (andattach_btf_idthat kernel sees) to be more dynamic. Such API will be added in future patches.Signed-off-by Alexei Starovoitov <[email protected]>---
RE: [PATCH v2 bpf-next 3/3] selftests/bpf: Add tests for program extensions
In reply to Alexei StarovoitovQuoted 11 lines · Alexei Starovoitov
Add program extension tests that build on top of
fexit_bpf2bpftests. Replace three global functions in previously loadedtest_pkt_access.cprogram with three new implementations:
intget_skb_len(struct __sk_buff *skb);
intget_constant(long val);
intget_skb_ifindex(int val, struct __sk_buff *skb, int var);
New function return the same results as original only if arguments match.
new_get_skb_ifindex()demonstrates that 'skb' argument doesn't have to be first and only argument of BPF program. All normal skb based accesses are available.
Quoted 6 lines
Signed-off-by Alexei Starovoitov <[email protected]>--- .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 20 ++++++- .../selftests/bpf/progs/fexit_bpf2bpf.c | 57 +++++++++++++++++++ .../selftests/bpf/progs/test_pkt_access.c | 8 ++- 3 files changed, 83 insertions(+), 2 deletions(-)
Re: [PATCH v2 bpf-next 1/3] bpf: Introduce dynamic program extensions
In reply to John FastabendQuoted 17 lines · John Fastabend
Quoted 16 lines · Alexei Starovoitov
Quoted 14 lines · John Fastabend
kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 4170
Quoted 4 lines
kernel/bpf/btf.c +3
-0+ + t1 =btf_type_skip_modifiers (btf1, t1->type,NULL ); + t2 =btf_type_skip_modifiers (btf2, t2->type,NULL );Is it really best to skip modifiers? I would expect that if the signature is different including modifiers then we should just reject it. OTOH its not really C code here either so modifiers may not have the same meaning. With just integers and struct it may be ok but if we add pointers to ints then what would we expect from a const int*?
So whats the reasoning for skipping modifiers? Is it purely an argument that its not required for safety so solve it elsewhere? In that case then checking names of functions is also equally not required.
Quoted 6 lines
Quoted 3 lines
Function names are not checked by the kernel. It's purely libbpf and
bpf_prog.cconvention. The kernel operates on prog_fd+btf_id only. The names of function arguments are not compared either.Sorry mistyped names of struct is what I meant, but that is probably nice to have per comment.
I think comments in the kernel code won't be sufficient :) I'm planing to write a proper doc describing global functions and program extensions, since these concepts are coupled.
Quoted 22 lines
Quoted 5 lines
The code has to skip modifiers. Otherwise the type comparison algorithm will be quite complex, since typedef is such modifier. Like 'u32' in original program and 'u32' in extension program would have to be recursively checked.
Quoted 8 lines
Another reason to skip modifiers is 'volatile' modifier. I suspect we would have to use it from time to time in original placeholder functions. Yet new replacement function will be written without volatile. The placeholder may need volatile to make sure compiler doesn't optimize things away. I found cases where 'noinline' in placeholder was not enough. clang would still inline the body of the function and remove call instruction. So far I've been using volatile as a workaround. May be we will introduce new function attribute to clang.
Yes, we have various similar issue and have in the past used volatile to work around them but volatile's inside loops tends to break loop optimizations and cause clang warnings/errors. Another common one is verifier failing to track when scalars move around in registers. As an example the following is valid C for a bounded additon to array pointer but not tractable for the verifier at the moment. (made example at some point I'll dig up a collection of real-world examples)
Quoted 7 lines
r1 = *(u64 *)(r10 - 8)
r6 = r1
if r6 < %[const] goto %l[err]
r3 += r1
r2 = %[copy_size]
r1 = r7
call 4Is it a sign compare in the above? The verifier currently struggles with sign compares. The issue you're describing could also be related to instruction combining optimization. Yonghong is working on undoing parts of instcombine here:
https://reviews.llvm.org/D72787
Dealing with 32-bit sign compares probably will be done as a verifier improvement. It's too hacky to do it in the llvm.
Quoted 6 lines
compiler barriers help but not always and also breaks loop optimization passes. But, thats a different discussion I only mention it because either verifier has to track above logic better or new attributes in clang could be used for these things. But the new attributes don't usually work well when mixed with optimization passes that we would actually like to keep.
yep. unfortunately llvm folks didn't like the idea of disabling individual optimization passes or parts of passes based on the backend flag. Hence BPF backend has to normalize IR before generating asm. Since all of the issues related to "llvm optimizing too much and verifier is too dumb" are related to missed tracking of registers, I'm thinking that we may try virtual register approach. Instead of doing register allocation and spill/fill in the llvm the backend may generate as many virtual registers as necessary and final BPF assembly will look like SSA except phisical registers will still be used to receive and pass arguments while all other math, branches will operate on virtual registers. Then the kernel will do register alloc after verification. I hope that should help the verifier get smarter at the expense of less efficient code and more spill/fill. But that is the last resort.
Quoted 12 lines
Quoted 11 lines
Having said that I share your concern regarding skipping 'const'. For 'const int arg' it's totally ok to skip it, since it's meaningless from safety pov, but for 'const int *arg' and 'const struct foo *arg' I'm planning to preserve it. It will be preserved at the verifier
bpf_reg_statelevel though. Just checking that 'const' is present in extension prog's BTF doesn't help safety. I'm planing to make the verifier enforce that bpf prog cannot write into argument which type is pointer to const struct. That part is still wip. It will be implemented for global functions first and then for extension programs. Currently the verifier rejects any pointer to struct (other than context), so no backward compatibility issues.
Quoted 4 lines
Ah ok this will be great. In that case const will be more general then merely functions and should be applicable generally at least as an end goal IMO. There will be a slight annoyance where old extensions may not run on new kernels though. I will argue such extensions are broken though.
'old extensions' won't run? I don't see it. Strict 'const' enforcement for pointers to struct won't break things. Currently pointers to struct are not supported. Only a pointer to one specific struct which is context. That's different. I don't think there will be any compatibility issues.
Quoted 3 lines
For this patch then,
Acked-by John Fastabend <[email protected]>
Thanks for the review!
John Fastabend
reviewerRe: [PATCH v2 bpf-next 1/3] bpf: Introduce dynamic program extensions
Quoted 59 lines · Alexei Starovoitov
Quoted 17 lines · John Fastabend
Quoted 16 lines · Alexei Starovoitov
Quoted 14 lines · John Fastabend
kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 4170
Quoted 4 lines
kernel/bpf/btf.c +3
-0+ + t1 =btf_type_skip_modifiers (btf1, t1->type,NULL ); + t2 =btf_type_skip_modifiers (btf2, t2->type,NULL );Is it really best to skip modifiers? I would expect that if the signature is different including modifiers then we should just reject it. OTOH its not really C code here either so modifiers may not have the same meaning. With just integers and struct it may be ok but if we add pointers to ints then what would we expect from a const int*?
So whats the reasoning for skipping modifiers? Is it purely an argument that its not required for safety so solve it elsewhere? In that case then checking names of functions is also equally not required.
Quoted 6 lines
Quoted 3 lines
Function names are not checked by the kernel. It's purely libbpf and
bpf_prog.cconvention. The kernel operates on prog_fd+btf_id only. The names of function arguments are not compared either.Sorry mistyped names of struct is what I meant, but that is probably nice to have per comment.
I think comments in the kernel code won't be sufficient :) I'm planing to write a proper doc describing global functions and program extensions, since these concepts are coupled.
Quoted 22 lines
Quoted 5 lines
The code has to skip modifiers. Otherwise the type comparison algorithm will be quite complex, since typedef is such modifier. Like 'u32' in original program and 'u32' in extension program would have to be recursively checked.
Quoted 8 lines
Another reason to skip modifiers is 'volatile' modifier. I suspect we would have to use it from time to time in original placeholder functions. Yet new replacement function will be written without volatile. The placeholder may need volatile to make sure compiler doesn't optimize things away. I found cases where 'noinline' in placeholder was not enough. clang would still inline the body of the function and remove call instruction. So far I've been using volatile as a workaround. May be we will introduce new function attribute to clang.
Yes, we have various similar issue and have in the past used volatile to work around them but volatile's inside loops tends to break loop optimizations and cause clang warnings/errors. Another common one is verifier failing to track when scalars move around in registers. As an example the following is valid C for a bounded additon to array pointer but not tractable for the verifier at the moment. (made example at some point I'll dig up a collection of real-world examples)
Quoted 7 lines
r1 = *(u64 *)(r10 - 8) r6 = r1 if r6 < %[const] goto %l[err] r3 += r1 r2 = %[copy_size] r1 = r7 call 4
Quoted 6 lines
Is it a sign compare in the above? The verifier currently struggles with sign compares. The issue you're describing could also be related to instruction combining optimization. Yonghong is working on undoing parts of instcombine here:
https://reviews.llvm.org/D72787
Dealing with 32-bit sign compares probably will be done as a verifier improvement. It's too hacky to do it in the llvm.
hmm I think it was an unsigned compare but I'll try to come up with some simple examples of real code this week and post them. To illustrate some of the verifier complaints we've been working around.
Quoted 8 lines
Quoted 7 lines
compiler barriers help but not always and also breaks loop optimization passes. But, thats a different discussion I only mention it because either verifier has to track above logic better or new attributes in clang could be used for these things. But the new attributes don't usually work well when mixed with optimization passes that we would actually like to keep.
Quoted 12 lines
yep. unfortunately llvm folks didn't like the idea of disabling individual optimization passes or parts of passes based on the backend flag. Hence BPF backend has to normalize IR before generating asm. Since all of the issues related to "llvm optimizing too much and verifier is too dumb" are related to missed tracking of registers, I'm thinking that we may try virtual register approach. Instead of doing register allocation and spill/fill in the llvm the backend may generate as many virtual registers as necessary and final BPF assembly will look like SSA except phisical registers will still be used to receive and pass arguments while all other math, branches will operate on virtual registers. Then the kernel will do register alloc after verification. I hope that should help the verifier get smarter at the expense of less efficient code and more spill/fill. But that is the last resort.
Interesting. Or maybe follow the parent chain and push information down the chain when we see register moves and further bounds refining.
Quoted 18 lines
Quoted 13 lines
Quoted 12 lines
Having said that I share your concern regarding skipping 'const'. For 'const int arg' it's totally ok to skip it, since it's meaningless from safety pov, but for 'const int *arg' and 'const struct foo *arg' I'm planning to preserve it. It will be preserved at the verifier
bpf_reg_statelevel though. Just checking that 'const' is present in extension prog's BTF doesn't help safety. I'm planing to make the verifier enforce that bpf prog cannot write into argument which type is pointer to const struct. That part is still wip. It will be implemented for global functions first and then for extension programs. Currently the verifier rejects any pointer to struct (other than context), so no backward compatibility issues.Quoted 4 lines
Ah ok this will be great. In that case const will be more general then merely functions and should be applicable generally at least as an end goal IMO. There will be a slight annoyance where old extensions may not run on new kernels though. I will argue such extensions are broken though.
Quoted 4 lines
'old extensions' won't run? I don't see it. Strict 'const' enforcement for pointers to struct won't break things. Currently pointers to struct are not supported. Only a pointer to one specific struct which is context. That's different. I don't think there will be any compatibility issues.
Right its not an issue.
Quoted 6 lines
Quoted 4 lines
For this patch then,
Acked-by John Fastabend <[email protected]>Thanks for the review!
Re: [PATCH v2 bpf-next 1/3] bpf: Introduce dynamic program extensions
In reply to John FastabendQuoted 54 lines · John Fastabend
Quoted 12 lines · Alexei Starovoitov
Quoted 11 lines · John Fastabend
kernel/bpf/btf.c int btf_distill_func_proto(struct bpf_verifier_log *log, · line 4170
Quoted 4 lines
kernel/bpf/btf.c +3
-0+ + t1 =btf_type_skip_modifiers (btf1, t1->type,NULL ); + t2 =btf_type_skip_modifiers (btf2, t2->type,NULL );Is it really best to skip modifiers? I would expect that if the signature is different including modifiers then we should just reject it. OTOH its not really C code here either so modifiers may not have the same meaning. With just integers and struct it may be ok but if we add pointers to ints then what would we expect from a const int*?
Quoted 7 lines
Quoted 3 lines
So whats the reasoning for skipping modifiers? Is it purely an argument that its not required for safety so solve it elsewhere? In that case then checking names of functions is also equally not required.
Function names are not checked by the kernel. It's purely libbpf and
bpf_prog.cconvention. The kernel operates on prog_fd+btf_id only. The names of function arguments are not compared either.Sorry mistyped names of struct is what I meant, but that is probably nice to have per comment.
Quoted 5 lines
The code has to skip modifiers. Otherwise the type comparison algorithm will be quite complex, since typedef is such modifier. Like 'u32' in original program and 'u32' in extension program would have to be recursively checked.
Quoted 8 lines
Another reason to skip modifiers is 'volatile' modifier. I suspect we would have to use it from time to time in original placeholder functions. Yet new replacement function will be written without volatile. The placeholder may need volatile to make sure compiler doesn't optimize things away. I found cases where 'noinline' in placeholder was not enough. clang would still inline the body of the function and remove call instruction. So far I've been using volatile as a workaround. May be we will introduce new function attribute to clang.
Yes, we have various similar issue and have in the past used volatile to work around them but volatile's inside loops tends to break loop optimizations and cause clang warnings/errors. Another common one is verifier failing to track when scalars move around in registers. As an example the following is valid C for a bounded additon to array pointer but not tractable for the verifier at the moment. (made example at some point I'll dig up a collection of real-world examples)
r1 = *(u64 *)(r10 - 8) r6 = r1 if r6 < %[const] goto %l[err] r3 += r1 r2 = %[copy_size] r1 = r7 call 4
Quoted 6 lines
compiler barriers help but not always and also breaks loop optimization passes. But, thats a different discussion I only mention it because either verifier has to track above logic better or new attributes in clang could be used for these things. But the new attributes don't usually work well when mixed with optimization passes that we would actually like to keep.
John, could you send your original C code how to reproduce this? I am working on llvm side to avoid such optimizations, i.e., moving scalars around so later on you could have two scales with different verified state holding the [slightly] same value.
Quoted 22 lines
Quoted 12 lines
Having said that I share your concern regarding skipping 'const'. For 'const int arg' it's totally ok to skip it, since it's meaningless from safety pov, but for 'const int *arg' and 'const struct foo *arg' I'm planning to preserve it. It will be preserved at the verifier
bpf_reg_statelevel though. Just checking that 'const' is present in extension prog's BTF doesn't help safety. I'm planing to make the verifier enforce that bpf prog cannot write into argument which type is pointer to const struct. That part is still wip. It will be implemented for global functions first and then for extension programs. Currently the verifier rejects any pointer to struct (other than context), so no backward compatibility issues.Ah ok this will be great. In that case const will be more general then merely functions and should be applicable generally at least as an end goal IMO. There will be a slight annoyance where old extensions may not run on new kernels though. I will argue such extensions are broken though.
For this patch then,
Acked-by John Fastabend <[email protected]>
Quoted 20 lines · Alexei Starovoitov
Quoted 19 lines · Toke Høiland-Jørgensen
Quoted 4 lines · Alexei Starovoitov
The last few month BPF community has been discussing an approach to call chaining, since exiting
bpt_tail_call()mechanism used in production XDP programs has plenty of downsides. The outcome of these discussion was a conclusion to implement dynamic re-linking of BPF programs. Where rootlet XDPQuoted 8 lines
program attached to a netdevice can programmatically define a policy of execution of other XDP programs. Such rootlet would be compiled as normal XDP program and provide a number of placeholder global functions which later can be replaced with future XDP programs. BPF trampoline, function by function verification were building blocks towards that goal. The patch 1 is a final building block. It introduces dynamic program extensions. A number of improvements like more flexible function by function verification and better libbpf api will be implemented in future patches.
This is great, thank you! I'll go play around with it; couldn't spot anything obvious from eye-balling the code, except that yeah, it does need a more flexible libbpf api :)
Quoted 7 lines
Quoted 3 lines
One thing that's not obvious to me: How can userspace tell which programs replace which functions after they are loaded? Is this put into
prog_tagsin structbpf_prog_info, or?good point. Would be good to extend
bpf_prog_info. Since prog-to-prog connection is unidirectional thebpf_prog_infoof extension prog will be able to say which original program it's replacing.
Yeah, that'll do. I can live with having to enumerate all programs and backtrack to the attached XDP program to figure out its component parts.
Quoted 3 lines
bpftool prog show will be able to print all this data. I think fenry/fexit progs would need the same
bpf_prog_infoextension.attach_prog_id+attach_btf_idwould be enough.
Yes, please. I actually assumed this was already there for fentry/fexit, which is why I was puzzled I couldn't find where this series hooked into that. I'll just wait for such an extension to show up, then :)
Quoted 1 line
In the mean time I can try to hack drgn script to do the same.
That would be great, thanks!
-Toke
Quoted 13 lines · Alexei Starovoitov
The last few month BPF community has been discussing an approach to call chaining, since exiting
bpt_tail_call()mechanism used in production XDP programs has plenty of downsides. The outcome of these discussion was a conclusion to implement dynamic re-linking of BPF programs. Where rootlet XDP program attached to a netdevice can programmatically define a policy of execution of other XDP programs. Such rootlet would be compiled as normal XDP program and provide a number of placeholder global functions which later can be replaced with future XDP programs. BPF trampoline, function by function verification were building blocks towards that goal. The patch 1 is a final building block. It introduces dynamic program extensions. A number of improvements like more flexible function by function verification and better libbpf api will be implemented in future patches.
Quoted 3 lines
v1->v2:
- addressed Andrii's comments
- rebase
Applied, thanks!