bpf: Support struct argument for trampoline base progs
v4 cover bpf: Support struct argument for trampoline base progs
Currently struct arguments are not supported for trampoline based progs. One of major reason is that struct argument may pass by value which may use more than one registers. This breaks trampoline progs where each argument is assumed to take one register. bcc community reported the issue ([1]) where struct argument is not supported for fentry program.
typedef struct {
uid_t val;
} kuid_t;
typedef struct {
gid_t val;
} kgid_t;
int security_path_chown(struct path *path, kuid_t uid, kgid_t gid);
Inside Meta, we also have a use case to attach to tcp_setsockopt()
typedef struct {
union {
void *kernel;
void __user *user;
};
bool is_kernel : 1;
} sockptr_t;
int tcp_setsockopt(struct sock *sk, int level, int optname,
sockptr_t optval, unsigned int optlen);This patch added struct value support for bpf tracing programs which uses trampoline. Only <= 16 byte struct size is supported for now which covers use cases in the above. For x86/arm64/bpf, <= 16 struct value will be passed in registers instead of by reference. Only x86_64 is supported in this patch. arm64 support can be added later.
[1] https://github.com/iovisor/bcc/issues/3657
Changelog:
v3 -> v4:
- fix a test failure where no casting for bpf_get_func_arg() value as the value type is 'int'.
- add tracing_struct test in DENYLIST.s390x
- simplify macro BPF_REG_CNT for BPF_PROG2.
v2 -> v3:
- previously struct arguments (<= 16 bytes) are passed by reference for bpf programs. Suggested by Alexei, it is passed by value now.
- in order to support passing <= 16 struct value, a new macro BPF_PROG2 is invented.
rfc v1 -> v2:
- changed bpf_func_model struct info fields to arg_flags[] to make it easy to iterate arguments in arch specific {save|restore}_regs() functions.
- added fexit tests to test return values with struct arguments.
Yonghong Song (8):
bpf: Allow struct argument in trampoline based programs
bpf: x86: Support in-register struct arguments in trampoline programs
bpf: Update descriptions for helpers bpf_get_func_arg[_cnt]()
bpf: arm64: No support of struct argument in trampoline programs
libbpf: Add new BPF_PROG2 macro
selftests/bpf: Add struct argument tests with fentry/fexit programs. selftests/bpf: Use BPF_PROG2 for some fentry programs without struct
arguments
selftests/bpf: Add tracing_struct test in DENYLIST.s390x
arch/arm64/net/bpf_jit_comp.c | 8 +- arch/x86/net/bpf_jit_comp.c | 68 +++++++--- include/linux/bpf.h | 4 + include/uapi/linux/bpf.h | 9 +- kernel/bpf/btf.c | 45 ++++++- tools/include/uapi/linux/bpf.h | 9 +- tools/lib/bpf/bpf_tracing.h | 79 ++++++++++++ tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/bpf_testmod/bpf_testmod.c | 48 +++++++ .../selftests/bpf/prog_tests/tracing_struct.c | 63 +++++++++ tools/testing/selftests/bpf/progs/timer.c | 4 +- .../selftests/bpf/progs/tracing_struct.c | 120 ++++++++++++++++++ 12 files changed, 424 insertions(+), 34 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/tracing_struct.c create mode 100644 tools/testing/selftests/bpf/progs/tracing_struct.c
Signature · 1 line
2.30.2
PATCH v4 1/8 bpf: Allow struct argument in trampoline based programs
Allow struct argument in trampoline based programs where the struct size should be <= 16 bytes. In such cases, the argument will be put into up to 2 registers for bpf, x86_64 and arm64 architectures.
To support arch-specific trampoline manipulation,
add arg_flags for additional struct information about arguments in btf_func_model. Such information will be used in arch specific function arch_prepare_bpf_trampoline() to prepare argument access properly in trampoline.
--- include/linux/bpf.h | 4 ++++ kernel/bpf/btf.c | 45 +++++++++++++++++++++++++++++++++++++++------ 2 files changed, 43 insertions(+), 6 deletions(-)
include/linux/bpf.h +4 -0
@@ -727,10 +727,14 @@ enum bpf_cgroup_storage_type {*/ #define MAX_BPF_FUNC_REG_ARGS 5 +/* The argument is a structure. */ +#define BTF_FMODEL_STRUCT_ARG BIT(0) +struct btf_func_model {u8 ret_size;u8 nr_args;u8 arg_size[MAX_BPF_FUNC_ARGS ]; +u8 arg_flags[MAX_BPF_FUNC_ARGS ]; };/* Restore arguments before returning from trampoline to let original function
kernel/bpf/btf.c +39 -6
94 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v4 2/8 bpf: x86: Support in-register struct arguments in trampoline programs
In reply to Yonghong SongIn C, struct value can be passed as a function argument. For small structs, struct value may be passed in
one or more registers. For trampoline based bpf programs, this would cause complication since one-to-one mapping between function argument and arch argument register is not valid any more.
The latest llvm16 added bpf support to pass by values for struct up to 16 bytes ([1]). This is also true for x86_64 architecture where two registers will hold the struct value if the struct size is >8 and <= 16. This may not be true if one of struct member is 'double' type but in current linux source code we don't have such instance yet, so we assume all >8 && <= 16 struct holds two general purpose argument registers.
Also change on-stack nr_args value to the number of registers holding the arguments. This will permit bpf_get_func_arg() helper to get all argument values.
[1] https://reviews.llvm.org/D132144
--- arch/x86/net/bpf_jit_comp.c | 68 +++++++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 17 deletions(-)
arch/x86/net/bpf_jit_comp.c +51 -17
@@ -1751,34 +1751,60 @@ st: if (is_imm8(insn->off))static void save_regs (const struct btf_func_model *m,u8 **prog,int nr_args,int stack_size) { -int i; +int i, j, arg_size, nr_regs;/* Store function arguments to stack. * For a function that accepts two pointers the sequence will be: * mov QWORD PTR [rbp-0x10],rdi * mov QWORD PTR [rbp-0x8],rsi */ -for (i =0 ; i <min (nr_args,6 ); i++) -emit_stx (prog,bytes_to_bpf_size (m->arg_size[i]), -BPF_REG_FP , - i ==5 ?X86_REG_R9 :BPF_REG_1 + i, - -(stack_size - i *8 )); +for (i =0 , j =0 ; i <min (nr_args,6 ); i++) { +if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG ) { + nr_regs = (m->arg_size[i] +7 ) /8 ; + arg_size =8 ; + }else { + nr_regs =1 ; + arg_size = m->arg_size[i]; + } + +while (nr_regs) { +emit_stx (prog,bytes_to_bpf_size (arg_size), +BPF_REG_FP , + j ==5 ?X86_REG_R9 :BPF_REG_1 + j, + -(stack_size - j *8 )); + nr_regs--; + j++; + } + } }static void restore_regs (const struct btf_func_model *m,u8 **prog,int nr_args,int stack_size) { -int i; +int i, j, arg_size, nr_regs;/* Restore function arguments from stack. * For a function that accepts two pointers the sequence will be: * EMIT4(0x48, 0x8B, 0x7D, 0xF0); mov rdi,QWORD PTR [rbp-0x10] * EMIT4(0x48, 0x8B, 0x75, 0xF8); mov rsi,QWORD PTR [rbp-0x8] */ -for (i =0 ; i <min (nr_args,6 ); i++) -emit_ldx (prog,bytes_to_bpf_size (m->arg_size[i]), - i ==5 ?X86_REG_R9 :BPF_REG_1 + i, -BPF_REG_FP , - -(stack_size - i *8 )); +for (i =0 , j =0 ; i <min (nr_args,6 ); i++) { +if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG ) { + nr_regs = (m->arg_size[i] +7 ) /8 ; + arg_size =8 ; + }else { + nr_regs =1 ; + arg_size = m->arg_size[i]; + } + +while (nr_regs) { +emit_ldx (prog,bytes_to_bpf_size (arg_size), + j ==5 ?X86_REG_R9 :BPF_REG_1 + j, +BPF_REG_FP , + -(stack_size - j *8 )); + nr_regs--; + j++; + } + } } static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog,
@@ -2015,7 +2041,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *istruct bpf_tramp_links *tlinks, void *orig_call) { - int ret, i, nr_args = m->nr_args; + int ret, i, nr_args = m->nr_args, extra_nregs = 0;int regs_off, ip_off, args_off, stack_size = nr_args *8 , run_ctx_off;struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY ];struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT ];
@@ -2028,6 +2054,14 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *iif (nr_args >6 )return -ENOTSUPP ; +for (i =0 ; i <MAX_BPF_FUNC_ARGS ; i++) { +if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG ) + extra_nregs += (m->arg_size[i] +7 ) /8 -1 ; + } +if (nr_args + extra_nregs >6 ) +return -ENOTSUPP ; + stack_size += extra_nregs *8 ; +/* Generated trampoline stack layout: * * RBP + 8 [ return address ]
@@ -2040,7 +2074,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i* [ ... ] * RBP - regs_off [ reg_arg1 ] program's ctx pointer * -* RBP - args_off [ args count ] always +* RBP - args_off [ arg regs count ] always * * RBP - ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag *
@@ -2083,11 +2117,11 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *iEMIT4 (0x48 ,0x83 ,0xEC , stack_size);/* sub rsp, stack_size */ EMIT1 (0x53 );/* push rbx */ -/* Store number of arguments of the traced function: -* mov rax, nr_args +/* Store number of argument registers of the traced function: +* mov rax, nr_args + extra_nregs * mov QWORD PTR [rbp - args_off], rax */ -emit_mov_imm64 (&prog,BPF_REG_0 ,0 , (u32 ) nr_args); +emit_mov_imm64 (&prog,BPF_REG_0 ,0 , (u32 ) nr_args + extra_nregs);emit_stx (&prog,BPF_DW ,BPF_REG_FP ,BPF_REG_0 , -args_off);if (flags &BPF_TRAMP_F_IP_ARG ) {
Signature · 1 line
2.30.2
Re: [PATCH bpf-next v4 2/8] bpf: x86: Support in-register struct arguments in trampoline programs
Quoted 46 lines · Yonghong Song
In C, struct value can be passed as a function argument. For small structs, struct value may be passed in
one or more registers. For trampoline based bpf programs, this would cause complication since one-to-one mapping between function argument and arch argument register is not valid any more.The latest llvm16 added bpf support to pass by values for struct up to 16 bytes ([1]). This is also true for
x86_64architecture where two registers will hold the struct value if the struct size is >8 and <= 16. This may not be true if one of struct member is 'double' type but in current linux source code we don't have such instance yet, so we assume all >8 && <= 16 struct holds two general purpose argument registers.Also change on-stack
nr_argsvalue to the number of registers holding the arguments. This will permitbpf_get_func_arg()helper to get all argument values.[1] https://reviews.llvm.org/D132144
Signed-off-by Yonghong Song <[email protected]>---
arch/x86/net/bpf_jit_comp.c | 68 +++++++++++++++++++++++++++-------- --1 file changed, 51 insertions(+), 17 deletions(-)arch/x86/net/bpf_jit_comp.c +0
-0@@ -1751,34 +1751,60 @@ st: if (is_imm8(insn- >off))static void save_regs(const struct
btf_func_model*m, u8 **prog, intnr_args,
intstack_size)
{
- int i;
+ int i, j,arg_size,nr_regs;
/* Store function arguments to stack.
* For a function that accepts two pointers the sequence will be:
* mov QWORD PTR [rbp-0x10],rdi
Quoted 8 lines
* mov QWORD PTR [rbp-0x8],rsi
*/
- for (i = 0; i <min(nr_args, 6); i++)
- emit_stx(prog,bytes_to_bpf_size(m->arg_size[i]), -BPF_REG_FP,
- i == 5 ?X86_REG_R9:BPF_REG_1+ i, - -(stack_size - i * 8));
+ for (i = 0, j = 0; i <min(nr_args, 6); i++) {
Is 6 still correct since an argument can take more than one register now? Perphaps j < min(...)?
I am not sure how to deal with a corner case that a 16 bytes struct arguement happens to be at 6th place. Does that mean first 8 bytes are in a register and the reset bytes are in the stack?
Quoted 116 lines
+ if (m->arg_flags[i] &
BTF_FMODEL_STRUCT_ARG) { + nr_regs = (m->arg_size[i] + 7) / 8; + arg_size = 8;
+ } else {
+ nr_regs = 1;
+ arg_size = m->arg_size[i];
+ }
+
+ while (nr_regs) {
+ emit_stx(prog,bytes_to_bpf_size(arg_size), +BPF_REG_FP,
+ j == 5 ?X86_REG_R9:BPF_REG_1+ j, + -(stack_size - j * 8));
+ nr_regs--;
+ j++;
+ }
+ }
}
static void restore_regs(const structbtf_func_model*m, u8 **prog, intnr_args,
intstack_size)
{
- int i;
+ int i, j,arg_size,nr_regs;
/* Restore function arguments from stack.
* For a function that accepts two pointers the sequence will be:
*EMIT4(0x48, 0x8B, 0x7D, 0xF0); mov rdi,QWORD PTR [rbp-0x10]
*EMIT4(0x48, 0x8B, 0x75, 0xF8); mov rsi,QWORD PTR [rbp-0x8] */
- for (i = 0; i <min(nr_args, 6); i++)
- emit_ldx(prog,bytes_to_bpf_size(m->arg_size[i]), - i == 5 ?X86_REG_R9:BPF_REG_1+ i, -BPF_REG_FP,
- -(stack_size - i * 8));
+ for (i = 0, j = 0; i <min(nr_args, 6); i++) {
+ if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG) { + nr_regs = (m->arg_size[i] + 7) / 8; + arg_size = 8;
+ } else {
+ nr_regs = 1;
+ arg_size = m->arg_size[i];
+ }
+
+ while (nr_regs) {
+ emit_ldx(prog,bytes_to_bpf_size(arg_size), + j == 5 ?X86_REG_R9:BPF_REG_1+ j, +BPF_REG_FP,
+ -(stack_size - j * 8));
+ nr_regs--;
+ j++;
+ }
+ }
}
static int invoke_bpf_prog(const structbtf_func_model*m, u8 **pprog,arch/x86/net/bpf_jit_comp.c +0
-0@@ -2015,7 +2041,7 @@ int arch_prepare_bpf_trampoline(struct
bpf_tramp_image*im, void *image, void *i
structbpf_tramp_links*tlinks, void*orig_call)
{
- int ret, i,nr_args=m->nr_args;
+ int ret, i,nr_args=m->nr_args,extra_nregs= 0; intregs_off,ip_off,args_off,stack_size=nr_args* 8,run_ctx_off;
structbpf_tramp_links*fentry = &tlinks[BPF_TRAMP_FENTRY]; structbpf_tramp_links*fexit = &tlinks[BPF_TRAMP_FEXIT];arch/x86/net/bpf_jit_comp.c +0
-0@@ -2028,6 +2054,14 @@ int arch_prepare_bpf_trampoline(struct
bpf_tramp_image*im, void *image, void *i
if (nr_args> 6)
return-ENOTSUPP;
+ for (i = 0; i <MAX_BPF_FUNC_ARGS; i++) {
+ if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG) + extra_nregs += (m->arg_size[i] + 7) / 8 - 1; + }
+ if (nr_args+extra_nregs> 6)
+ return-ENOTSUPP;
+ stack_size +=extra_nregs* 8;
+
/* Generated trampoline stack layout:
*
* RBP + 8 [ return address ]arch/x86/net/bpf_jit_comp.c +0
-0@@ -2040,7 +2074,7 @@ int arch_prepare_bpf_trampoline(struct
bpf_tramp_image*im, void *image, void *i
* [ ... ]
* RBP - regs_off [ reg_arg1 ] program's ctx pointer *
- * RBP - args_off [ args count ] always
+ * RBP - args_off [ arg regs count ] always
*
* RBP - ip_off [ traced function ]BPF_TRAMP_F_IP_ARGflag
*arch/x86/net/bpf_jit_comp.c +0
-0@@ -2083,11 +2117,11 @@ int arch_prepare_bpf_trampoline(struct
bpf_tramp_image*im, void *image, void *i
EMIT4(0x48,0x83,0xEC,stack_size); /* sub rsp,stack_size*/
EMIT1(0x53); /* push rbx */
- /* Store number of arguments of the traced function:
- * mov rax,nr_args
+ /* Store number of argument registers of the traced function:
+ * mov rax,nr_args+extra_nregs
* mov QWORD PTR [rbp -args_off], rax
*/
- emit_mov_imm64(&prog,BPF_REG_0, 0, (u32)nr_args); + emit_mov_imm64(&prog,BPF_REG_0, 0, (u32)nr_args+extra_nregs);
emit_stx(&prog,BPF_DW,BPF_REG_FP,BPF_REG_0, -args_off);
if (flags &BPF_TRAMP_F_IP_ARG) {
Re: [PATCH bpf-next v4 2/8] bpf: x86: Support in-register struct arguments in trampoline programs
Quoted 56 lines · Kui-Feng Lee
Quoted 46 lines · Yonghong Song
In C, struct value can be passed as a function argument. For small structs, struct value may be passed in
one or more registers. For trampoline based bpf programs, this would cause complication since one-to-one mapping between function argument and arch argument register is not valid any more.The latest llvm16 added bpf support to pass by values for struct up to 16 bytes ([1]). This is also true for
x86_64architecture where two registers will hold the struct value if the struct size is >8 and <= 16. This may not be true if one of struct member is 'double' type but in current linux source code we don't have such instance yet, so we assume all >8 && <= 16 struct holds two general purpose argument registers.Also change on-stack
nr_argsvalue to the number of registers holding the arguments. This will permitbpf_get_func_arg()helper to get all argument values.[1] https://reviews.llvm.org/D132144
Signed-off-by Yonghong Song <[email protected]>--- arch/x86/net/bpf_jit_comp.c | 68 +++++++++++++++++++++++++++-------- -- 1 file changed, 51 insertions(+), 17 deletions(-)arch/x86/net/bpf_jit_comp.c +1
-1@@ -1751,34 +1751,60 @@ st: if (is_imm8(insn- > off)) static void save_regs(const struct btf_func_model *m, u8 **prog, int nr_args, int stack_size) { - int i; + int i, j, arg_size, nr_regs; /* Store function arguments to stack.* For a function that accepts two pointers the sequence will be:
* mov QWORD PTR [rbp-0x10],rdiQuoted 8 lines
* mov QWORD PTR [rbp-0x8],rsi
*/
- for (i = 0; i <min(nr_args, 6); i++)
- emit_stx(prog,bytes_to_bpf_size(m->arg_size[i]), -BPF_REG_FP,
- i == 5 ?X86_REG_R9:BPF_REG_1+ i, - -(stack_size - i * 8));
+ for (i = 0, j = 0; i <min(nr_args, 6); i++) {
Quoted 2 lines
Is 6 still correct since an argument can take more than one register now? Perphaps j < min(...)?
'min(nr_args, 6)' probably a historical artifact. It can be replaced with 'nr_args' since in the beginning of function arch_prepare_bpf_trampoline(), we have
/* x86-64 supports up to 6 arguments. 7+ can be added in the
future */
if (nr_args > 6)
return -ENOTSUPP;so nr_args <= 6 is true.
Quoted 4 lines
I am not sure how to deal with a corner case that a 16 bytes struct arguement happens to be at 6th place. Does that mean first 8 bytes are in a register and the reset bytes are in the stack?
This won't happen, see below change in arch_prepare_bpf_trampoline() where it is checked to ensure the number of registers holding all arguments mush be <= 6.
Quoted 78 lines
Quoted 78 lines
+ if (m->arg_flags[i] &
BTF_FMODEL_STRUCT_ARG) { + nr_regs = (m->arg_size[i] + 7) / 8; + arg_size = 8;
+ } else {
+ nr_regs = 1;
+ arg_size = m->arg_size[i];
+ }
+
+ while (nr_regs) {
+ emit_stx(prog,bytes_to_bpf_size(arg_size), +BPF_REG_FP,
+ j == 5 ?X86_REG_R9:BPF_REG_1+ j, + -(stack_size - j * 8));
+ nr_regs--;
+ j++;
+ }
+ }
}static void restore_regs(const struct
btf_func_model*m, u8 **prog,
intnr_args,
intstack_size)
{
- int i;
+ int i, j,arg_size,nr_regs;/* Restore function arguments from stack.
* For a function that accepts two pointers the sequence will
be:
*EMIT4(0x48, 0x8B, 0x7D, 0xF0); mov rdi,QWORD PTR [rbp-0x10]
*EMIT4(0x48, 0x8B, 0x75, 0xF8); mov rsi,QWORD PTR [rbp-0x8] */
- for (i = 0; i <min(nr_args, 6); i++)
- emit_ldx(prog,bytes_to_bpf_size(m->arg_size[i]), - i == 5 ?X86_REG_R9:BPF_REG_1+ i, -BPF_REG_FP,
- -(stack_size - i * 8));
+ for (i = 0, j = 0; i <min(nr_args, 6); i++) {
+ if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG) { + nr_regs = (m->arg_size[i] + 7) / 8; + arg_size = 8;
+ } else {
+ nr_regs = 1;
+ arg_size = m->arg_size[i];
+ }
+
+ while (nr_regs) {
+ emit_ldx(prog,bytes_to_bpf_size(arg_size), + j == 5 ?X86_REG_R9:BPF_REG_1+ j, +BPF_REG_FP,
+ -(stack_size - j * 8));
+ nr_regs--;
+ j++;
+ }
+ }
}static int invoke_bpf_prog(const struct
btf_func_model*m, u8
**pprog,arch/x86/net/bpf_jit_comp.c +0
-0@@ -2015,7 +2041,7 @@ int arch_prepare_bpf_trampoline(struct
bpf_tramp_image*im, void *image, void *i
structbpf_tramp_links*tlinks, void*orig_call)
{
- int ret, i,nr_args=m->nr_args;
+ int ret, i,nr_args=m->nr_args,extra_nregs= 0;
intregs_off,ip_off,args_off,stack_size=nr_args* 8,run_ctx_off;
structbpf_tramp_links*fentry = &tlinks[BPF_TRAMP_FENTRY]; structbpf_tramp_links*fexit = &tlinks[BPF_TRAMP_FEXIT];arch/x86/net/bpf_jit_comp.c +0
-0@@ -2028,6 +2054,14 @@ int arch_prepare_bpf_trampoline(struct
bpf_tramp_image*im, void *image, void *i
if (nr_args> 6)
return-ENOTSUPP;
Quoted 6 lines
Quoted 6 lines
+ for (i = 0; i <
MAX_BPF_FUNC_ARGS; i++) {
+ if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG) + extra_nregs += (m->arg_size[i] + 7) / 8 - 1; + }
+ if (nr_args+extra_nregs> 6)
+ return-ENOTSUPP;
Here to ensure the number of registers holding all arguments must be <= 6.
Quoted 5 lines
Quoted 5 lines
+ stack_size +=
extra_nregs* 8;
+
/* Generated trampoline stack layout:
*
* RBP + 8 [ return address ]
[...]
Re: [PATCH bpf-next v4 2/8] bpf: x86: Support in-register struct arguments in trampoline programs
In reply to Yonghong SongQuoted 47 lines · Yonghong Song
In C, struct value can be passed as a function argument. For small structs, struct value may be passed in
one or more registers. For trampoline based bpf programs, this would cause complication since one-to-one mapping between function argument and arch argument register is not valid any more.The latest llvm16 added bpf support to pass by values for struct up to 16 bytes ([1]). This is also true for
x86_64architecture where two registers will hold the struct value if the struct size is >8 and <= 16. This may not be true if one of struct member is 'double' type but in current linux source code we don't have such instance yet, so we assume all >8 && <= 16 struct holds two general purpose argument registers.Also change on-stack
nr_argsvalue to the number of registers holding the arguments. This will permitbpf_get_func_arg()helper to get all argument values.[1] https://reviews.llvm.org/D132144
Signed-off-by Yonghong Song <[email protected]>--- arch/x86/net/bpf_jit_comp.c | 68 +++++++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 17 deletions(-)arch/x86/net/bpf_jit_comp.c +1
-4@@ -1751,34 +1751,60 @@ st: if (is_imm8(insn->off))static void save_regs (const struct btf_func_model *m,u8 **prog,int nr_args,int stack_size) { -int i; +int i, j, arg_size, nr_regs;/* Store function arguments to stack. * For a function that accepts two pointers the sequence will be: * mov QWORD PTR [rbp-0x10],rdi * mov QWORD PTR [rbp-0x8],rsi */ -for (i =0 ; i <min (nr_args,6 ); i++) - emit_stx(prog, bytes_to_bpf_size(m->arg_size[i]), - BPF_REG_FP,
arch/x86/net/bpf_jit_comp.c st: if (is_imm8(insn->off)) · line 1767
Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +8
-2- i ==5 ?X86_REG_R9 :BPF_REG_1 + i, - -(stack_size - i * 8)); +for (i =0 , j =0 ; i <min (nr_args,6 ); i++) { +if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG ) { + nr_regs = (m->arg_size[i] +7 ) /8 ; + arg_size =8 ; + }else { + nr_regs =1 ; + arg_size = m->arg_size[i]; + }
This bit begs for a common helper, but I'm not sure whether it will look better, so applied as-is.
BPF_PROG2 also feels unusual as an API macro name. We probably should bikeshed a bit and follow up if a better name is found.
Re: [PATCH bpf-next v4 2/8] bpf: x86: Support in-register struct arguments in trampoline programs
Quoted 62 lines · Alexei Starovoitov
Quoted 47 lines · Yonghong Song
In C, struct value can be passed as a function argument. For small structs, struct value may be passed in
one or more registers. For trampoline based bpf programs, this would cause complication since one-to-one mapping between function argument and arch argument register is not valid any more.The latest llvm16 added bpf support to pass by values for struct up to 16 bytes ([1]). This is also true for
x86_64architecture where two registers will hold the struct value if the struct size is >8 and <= 16. This may not be true if one of struct member is 'double' type but in current linux source code we don't have such instance yet, so we assume all >8 && <= 16 struct holds two general purpose argument registers.Also change on-stack
nr_argsvalue to the number of registers holding the arguments. This will permitbpf_get_func_arg()helper to get all argument values.[1] https://reviews.llvm.org/D132144
Signed-off-by Yonghong Song <[email protected]>--- arch/x86/net/bpf_jit_comp.c | 68 +++++++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 17 deletions(-)arch/x86/net/bpf_jit_comp.c +1
-4@@ -1751,34 +1751,60 @@ st: if (is_imm8(insn->off))static void save_regs (const struct btf_func_model *m,u8 **prog,int nr_args,int stack_size) { -int i; +int i, j, arg_size, nr_regs;/* Store function arguments to stack. * For a function that accepts two pointers the sequence will be: * mov QWORD PTR [rbp-0x10],rdi * mov QWORD PTR [rbp-0x8],rsi */ -for (i =0 ; i <min (nr_args,6 ); i++) - emit_stx(prog, bytes_to_bpf_size(m->arg_size[i]), - BPF_REG_FP,arch/x86/net/bpf_jit_comp.c st: if (is_imm8(insn->off)) · line 1767
Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +8
-2- i ==5 ?X86_REG_R9 :BPF_REG_1 + i, - -(stack_size - i * 8)); +for (i =0 , j =0 ; i <min (nr_args,6 ); i++) { +if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG ) { + nr_regs = (m->arg_size[i] +7 ) /8 ; + arg_size =8 ; + }else { + nr_regs =1 ; + arg_size = m->arg_size[i]; + }This bit begs for a common helper, but I'm not sure whether it will look better, so applied as-is.
Quoted 3 lines
BPF_PROG2also feels unusual as an API macro name. We probably should bikeshed a bit and follow up if a better name is found.
I didn't come up with a better name either. But happy to change to a different name if we agreed on one.
PATCH v4 3/8 bpf: Update descriptions for helpers bpf_get_func_arg[_cnt]()
In reply to Yonghong SongNow instead of the number of arguments, the number of registers holding argument values are stored in trampoline. Update the description of bpf_get_func_arg[_cnt]() helpers. Previous programs without struct arguments should continue to work as usual.
--- include/uapi/linux/bpf.h | 9 +++++---- tools/include/uapi/linux/bpf.h | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-)
include/uapi/linux/bpf.h +5 -4
@@ -5079,12 +5079,12 @@ union bpf_attr {* * long bpf_get_func_arg(void *ctx, u32 n, u64 *value) * Description -* Get **n**-th argument (zero based) of the traced function (for tracing programs) +* Get **n**-th argument register (zero based) of the traced function (for tracing programs) * returned in **value**. * * Return * 0 on success. -* **-EINVAL** if n >= arguments count of traced function. +* **-EINVAL** if n >= argument register count of traced function. * * long bpf_get_func_ret(void *ctx, u64 *value) * Description
@@ -5097,10 +5097,11 @@ union bpf_attr {* * long bpf_get_func_arg_cnt(void *ctx) * Description -* Get number of arguments of the traced function (for tracing programs). +* Get number of registers of the traced function (for tracing programs) where +* function arguments are stored in these registers. * * Return -* The number of arguments of the traced function. +* The number of argument registers of the traced function. * * int bpf_get_retval(void) * Description
tools/include/uapi/linux/bpf.h +5 -4
@@ -5079,12 +5079,12 @@ union bpf_attr {* * long bpf_get_func_arg(void *ctx, u32 n, u64 *value) * Description -* Get **n**-th argument (zero based) of the traced function (for tracing programs) +* Get **n**-th argument register (zero based) of the traced function (for tracing programs) * returned in **value**. * * Return * 0 on success. -* **-EINVAL** if n >= arguments count of traced function. +* **-EINVAL** if n >= argument register count of traced function. * * long bpf_get_func_ret(void *ctx, u64 *value) * Description
@@ -5097,10 +5097,11 @@ union bpf_attr {* * long bpf_get_func_arg_cnt(void *ctx) * Description -* Get number of arguments of the traced function (for tracing programs). +* Get number of registers of the traced function (for tracing programs) where +* function arguments are stored in these registers. * * Return -* The number of arguments of the traced function. +* The number of argument registers of the traced function. * * int bpf_get_retval(void) * Description
Signature · 1 line
2.30.2
Re: [PATCH bpf-next v4 3/8] bpf: Update descriptions for helpers bpf_get_func_arg[_cnt]()
Quoted 17 lines · Yonghong Song
Now instead of the number of arguments, the number of registers holding argument values are stored in trampoline. Update the description of bpf_get_func_arg[_cnt]() helpers. Previous programs without struct arguments should continue to work as usual.
Signed-off-by Yonghong Song <[email protected]>--- include/uapi/linux/bpf.h | 9 +++++---- tools/include/uapi/linux/bpf.h | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-)include/uapi/linux/bpf.h +0
-0
include/uapi/linux/bpf.h union bpf_attr { · line 5082
Quoted 5 lines
include/uapi/linux/bpf.h +1
-1@@ -5079,12 +5079,12 @@ union bpf_attr {* * long bpf_get_func_arg(void *ctx, u32 n, u64 *value) * Description -* Get **n**-th argument (zero based) of the traced function (for tracing programs) +* Get **n**-th argument register (zero based) of the traced function (for tracing programs)
I'm bit worried about the confusion between args/regs we create with this, but I don't any have better idea how to solve this
keeping extra stack values for nr_args and nr_regs and have new helpers to get reg values.. but then bpf_get_func_arg still does not return the full argument value.. also given that the struct args should be rare, I guess it's fine ;-)
jirka
Quoted 59 lines
include/uapi/linux/bpf.h +4
-3* returned in **value**. * * Return * 0 on success. -* **-EINVAL** if n >= arguments count of traced function. +* **-EINVAL** if n >= argument register count of traced function. * * long bpf_get_func_ret(void *ctx, u64 *value) * Description @@ -5097,10 +5097,11 @@ union bpf_attr {* * long bpf_get_func_arg_cnt(void *ctx) * Description -* Get number of arguments of the traced function (for tracing programs). +* Get number of registers of the traced function (for tracing programs) where +* function arguments are stored in these registers. * * Return -* The number of arguments of the traced function. +* The number of argument registers of the traced function. * * int bpf_get_retval(void) * Description tools/include/uapi/linux/bpf.h +5
-4@@ -5079,12 +5079,12 @@ union bpf_attr {* * long bpf_get_func_arg(void *ctx, u32 n, u64 *value) * Description -* Get **n**-th argument (zero based) of the traced function (for tracing programs) +* Get **n**-th argument register (zero based) of the traced function (for tracing programs) * returned in **value**. * * Return * 0 on success. -* **-EINVAL** if n >= arguments count of traced function. +* **-EINVAL** if n >= argument register count of traced function. * * long bpf_get_func_ret(void *ctx, u64 *value) * Description @@ -5097,10 +5097,11 @@ union bpf_attr {* * long bpf_get_func_arg_cnt(void *ctx) * Description -* Get number of arguments of the traced function (for tracing programs). +* Get number of registers of the traced function (for tracing programs) where +* function arguments are stored in these registers. * * Return -* The number of arguments of the traced function. +* The number of argument registers of the traced function. * * int bpf_get_retval(void) * Description Signature · 1 line
2.30.2
Re: [PATCH bpf-next v4 3/8] bpf: Update descriptions for helpers bpf_get_func_arg[_cnt]()
Quoted 27 lines · Jiri Olsa
Quoted 17 lines · Yonghong Song
Now instead of the number of arguments, the number of registers holding argument values are stored in trampoline. Update the description of bpf_get_func_arg[_cnt]() helpers. Previous programs without struct arguments should continue to work as usual.
Signed-off-by Yonghong Song <[email protected]>--- include/uapi/linux/bpf.h | 9 +++++---- tools/include/uapi/linux/bpf.h | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-)include/uapi/linux/bpf.h +0
-0include/uapi/linux/bpf.h union bpf_attr { · line 5082
Quoted 5 lines
include/uapi/linux/bpf.h +1
-1@@ -5079,12 +5079,12 @@ union bpf_attr {* * long bpf_get_func_arg(void *ctx, u32 n, u64 *value) * Description -* Get **n**-th argument (zero based) of the traced function (for tracing programs) +* Get **n**-th argument register (zero based) of the traced function (for tracing programs) I'm bit worried about the confusion between args/regs we create with this, but I don't any have better idea how to solve this
Quoted 4 lines
keeping extra stack values for
nr_argsandnr_regsand have new helpers to get reg values.. but thenbpf_get_func_argstill does not return the full argument value.. also given that the struct args should be rare, I guess it's fine ;-)
You are right. Since an argument may takes two u64 space, the helper bpf_get_func_arg() might not return the *whole* argument any more. But it still returns a func arg, maybe just a partial func arg. So the 'n' here has a semantic change, instead of 'n'th argument, it becomes 'n'th argument register. To create new helpers probably not needed, so that is why I made the description change to the existing helper which maintains backward compatability too.
Quoted 26 lines
jirka
Quoted 23 lines
include/uapi/linux/bpf.h +4
-3* returned in **value**. * * Return * 0 on success. -* **-EINVAL** if n >= arguments count of traced function. +* **-EINVAL** if n >= argument register count of traced function. * * long bpf_get_func_ret(void *ctx, u64 *value) * Description @@ -5097,10 +5097,11 @@ union bpf_attr {* * long bpf_get_func_arg_cnt(void *ctx) * Description -* Get number of arguments of the traced function (for tracing programs). +* Get number of registers of the traced function (for tracing programs) where +* function arguments are stored in these registers. * * Return -* The number of arguments of the traced function. +* The number of argument registers of the traced function. * * int bpf_get_retval(void) * Description
[...]
PATCH v4 4/8 bpf: arm64: No support of struct argument in trampoline programs
In reply to Yonghong SongARM64 does not support struct argument for trampoline based bpf programs yet.
--- arch/arm64/net/bpf_jit_comp.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/net/bpf_jit_comp.c +7 -1
@@ -1970,7 +1970,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, u32 flags, struct bpf_tramp_links *tlinks, void *orig_call) { - int ret; + int i, ret;int nargs = m->nr_args;int max_insns = ((long )image_end - (long )image) /AARCH64_INSN_SIZE ; struct jit_ctx ctx = {
@@ -1982,6 +1982,12 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image,if (nargs >8 )return -ENOTSUPP ; +/* don't support struct argument */ +for (i =0 ; i <MAX_BPF_FUNC_ARGS ; i++) { +if (m->arg_flags[i] &BTF_FMODEL_STRUCT_ARG ) +return -ENOTSUPP ; + } + ret =prepare_trampoline (&ctx, im, tlinks, orig_call, nargs, flags);if (ret <0 )return ret;
Signature · 1 line
2.30.2
PATCH v4 5/8 libbpf: Add new BPF_PROG2 macro
In reply to Yonghong SongTo support struct arguments in trampoline based programs, existing BPF_PROG doesn't work any more since the type size is needed to find whether a parameter takes one or two registers. So this patch added a new BPF_PROG2 macro to support such trampoline programs.
The idea is suggested by Andrii. For example, if the to-be-traced function has signature like
typedef struct {
void *x;
int t; } sockptr;
int blah(sockptr x, char y);
In the new BPF_PROG2 macro, the argument can be
represented as
__bpf_prog_call(
({ union {
struct { __u64 x, y; } ___z;
sockptr x;
} ___tmp = { .___z = { ctx[0], ctx[1] }};
___tmp.x;
}),
({ union {
struct { __u8 x; } ___z;
char y;
} ___tmp = { .___z = { ctx[2] }};
___tmp.y;
}));In the above, the values stored on the stack are properly assigned to the actual argument type value by using 'union' magic. Note that the macro also works even if no arguments are with struct types.
Note that new BPF_PROG2 works for both llvm16 and pre-llvm16 compilers where llvm16 supports bpf target passing value with struct up to 16 byte size and pre-llvm16 will pass by reference by storing values on the stack. With static functions with struct argument as always inline, the compiler is able to optimize and remove additional stack saving of struct values.
--- tools/lib/bpf/bpf_tracing.h | 79 +++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)
tools/lib/bpf/bpf_tracing.h +79 -0
@@ -438,6 +438,85 @@ typeof(name(0)) name(unsigned long long *ctx) \ static __always_inline typeof(name(0)) \ ____##name(unsigned long long *ctx, ##args) +#ifndef ____bpf_nth +#define ____bpf_nth (_, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20, _21, _22, _23, _24,N , ...) N +#endif +#ifndef ____bpf_narg +#define ____bpf_narg (...) ____bpf_nth(_, ##__VA_ARGS__, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 0) +#endif + +#define BPF_REG_CNT (t) \ + (__builtin_choose_expr(sizeof(t) == 1 || sizeof(t) == 2 || sizeof(t) == 4 || sizeof(t) == 8, 1, \ + __builtin_choose_expr(sizeof(t) == 16, 2, \ + (void)0))) + +#define ____bpf_reg_cnt0 () (0) +#define ____bpf_reg_cnt1 (t, x) (____bpf_reg_cnt0() + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt2(t, x, args...) (____bpf_reg_cnt1(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt3(t, x, args...) (____bpf_reg_cnt2(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt4(t, x, args...) (____bpf_reg_cnt3(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt5(t, x, args...) (____bpf_reg_cnt4(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt6(t, x, args...) (____bpf_reg_cnt5(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt7(t, x, args...) (____bpf_reg_cnt6(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt8(t, x, args...) (____bpf_reg_cnt7(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt9(t, x, args...) (____bpf_reg_cnt8(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt10(t, x, args...) (____bpf_reg_cnt9(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt11(t, x, args...) (____bpf_reg_cnt10(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt12(t, x, args...) (____bpf_reg_cnt11(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt(args...) ___bpf_apply(____bpf_reg_cnt, ____bpf_narg(args))(args) + +#define ____bpf_union_arg (t, x, n) \ + __builtin_choose_expr(sizeof(t) == 1, ({ union { struct { __u8 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]}}; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 2, ({ union { struct { __u16 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 4, ({ union { struct { __u32 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 8, ({ union { struct { __u64 x; } ___z; t x; } ___tmp = {.___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 16, ({ union { struct { __u64 x, y; } ___z; t x; } ___tmp = {.___z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \ + (void)0))))) + +#define ____bpf_ctx_arg0(n, args...) +#define ____bpf_ctx_arg1 (n, t, x) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt1(t, x)) +#define ____bpf_ctx_arg2(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt2(t, x, args)) ____bpf_ctx_arg1(n, args) +#define ____bpf_ctx_arg3(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt3(t, x, args)) ____bpf_ctx_arg2(n, args) +#define ____bpf_ctx_arg4(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt4(t, x, args)) ____bpf_ctx_arg3(n, args) +#define ____bpf_ctx_arg5(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt5(t, x, args)) ____bpf_ctx_arg4(n, args) +#define ____bpf_ctx_arg6(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt6(t, x, args)) ____bpf_ctx_arg5(n, args) +#define ____bpf_ctx_arg7(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt7(t, x, args)) ____bpf_ctx_arg6(n, args) +#define ____bpf_ctx_arg8(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt8(t, x, args)) ____bpf_ctx_arg7(n, args) +#define ____bpf_ctx_arg9(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt9(t, x, args)) ____bpf_ctx_arg8(n, args) +#define ____bpf_ctx_arg10(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt10(t, x, args)) ____bpf_ctx_arg9(n, args) +#define ____bpf_ctx_arg11(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt11(t, x, args)) ____bpf_ctx_arg10(n, args) +#define ____bpf_ctx_arg12(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt12(t, x, args)) ____bpf_ctx_arg11(n, args) +#define ____bpf_ctx_arg(n, args...) ___bpf_apply(____bpf_ctx_arg, ____bpf_narg(args))(n, args) + +#define ____bpf_ctx_decl0 () +#define ____bpf_ctx_decl1 (t, x) , t x +#define ____bpf_ctx_decl2(t, x, args...) , t x ____bpf_ctx_decl1(args) +#define ____bpf_ctx_decl3(t, x, args...) , t x ____bpf_ctx_decl2(args) +#define ____bpf_ctx_decl4(t, x, args...) , t x ____bpf_ctx_decl3(args) +#define ____bpf_ctx_decl5(t, x, args...) , t x ____bpf_ctx_decl4(args) +#define ____bpf_ctx_decl6(t, x, args...) , t x ____bpf_ctx_decl5(args) +#define ____bpf_ctx_decl7(t, x, args...) , t x ____bpf_ctx_decl6(args) +#define ____bpf_ctx_decl8(t, x, args...) , t x ____bpf_ctx_decl7(args) +#define ____bpf_ctx_decl9(t, x, args...) , t x ____bpf_ctx_decl8(args) +#define ____bpf_ctx_decl10(t, x, args...) , t x ____bpf_ctx_decl9(args) +#define ____bpf_ctx_decl11(t, x, args...) , t x ____bpf_ctx_decl10(args) +#define ____bpf_ctx_decl12(t, x, args...) , t x ____bpf_ctx_decl11(args) +#define ____bpf_ctx_decl(args...) ___bpf_apply(____bpf_ctx_decl, ____bpf_narg(args))(args) + +/* +* BPF_PROG2 can handle struct arguments. +*/ +#define BPF_PROG2(name, args...) \ +name(unsigned long long *ctx); \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)); \ +typeof(name(0)) name(unsigned long long *ctx) \ +{ \ + return ____##name(ctx ____bpf_ctx_arg(____bpf_reg_cnt(args), args)); \ +} \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)) +struct pt_regs ;#define ___bpf_kprobe_args0 () ctx
Signature · 1 line
2.30.2
Re: [PATCH bpf-next v4 5/8] libbpf: Add new BPF_PROG2 macro
Quoted 52 lines · Yonghong Song
To support struct arguments in trampoline based programs, existing
BPF_PROGdoesn't work any more since the type size is needed to find whether a parameter takes one or two registers. So this patch added a newBPF_PROG2macro to support such trampoline programs.The idea is suggested by Andrii. For example, if the to-be-traced function has signature like
typedef struct { void *x; int t;} sockptr;
intblah(sockptr x, char y);In the new
BPF_PROG2macro, the argument can be
represented as
__bpf_prog_call(({ union { struct { __u64 x, y; } ___z; sockptr x; } ___tmp = { .___z = { ctx[0], ctx[1] }}; ___tmp.x; }), ({ union { struct { __u8 x; } ___z; char y; } ___tmp = { .___z = { ctx[2] }}; ___tmp.y; }));In the above, the values stored on the stack are properly assigned to the actual argument type value by using 'union' magic. Note that the macro also works even if no arguments are with struct types.
Note that new
BPF_PROG2works for both llvm16 and pre-llvm16 compilers where llvm16 supports bpf target passing value with struct up to 16 byte size and pre-llvm16 will pass by reference by storing values on the stack. With static functions with struct argument as always inline, the compiler is able to optimize and remove additional stack saving of struct values.Signed-off-by Yonghong Song <[email protected]>--- tools/lib/bpf/bpf_tracing.h | 79 +++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)tools/lib/bpf/bpf_tracing.h +0
-0
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 443
Quoted 6 lines
tools/lib/bpf/bpf_tracing.h +3
-0@@ -438,6 +438,85 @@ typeof(name(0)) name(unsigned long long *ctx) \ static __always_inline typeof(name(0)) \ ____##name(unsigned long long *ctx, ##args) +#ifndef ____bpf_nth +#define ____bpf_nth (_, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20, _21, _22, _23, _24,N , ...) N +#endif
we already have ___bpf_nth (triple underscore) variant, wouldn't extending that one to support up to 24 argument work? It's quite confusing to have ___bpf_nth and ____bpf_nth. Maybe let's consolidate?
And I'd totally wrap this long line :)
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 446
Quoted 3 lines
tools/lib/bpf/bpf_tracing.h +3
-0+#ifndef ____bpf_narg +#define ____bpf_narg (...) ____bpf_nth(_, ##__VA_ARGS__, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 0) +#endif
similar confusiong with triple underscore bpf_narg. Given this is used in BPF_PROG2, how about renaming it to bpf_narg2 to make this connection? And also note that all similar macros use triple underscore, while you added quad underscores everywhere. Can you please follow up with a rename to use triple underscore for consistency?
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 448
Quoted 2 lines
tools/lib/bpf/bpf_tracing.h +2
-0+ +#define BPF_REG_CNT (t) \
this looks like a "public API", but I don't think this was the intention, right? Let's rename it to ___bpf_reg_cnt()?
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 449
Quoted 1 line
tools/lib/bpf/bpf_tracing.h +1
-0+ (__builtin_choose_expr(sizeof(t) == 1 || sizeof(t) == 2 || sizeof(t) == 4 || sizeof(t) == 8, 1, \
nit: seeing ____bpf_union_arg implementation below I prefer one case per line there as well. How about doing one __builtin_choose_expr per each supported size?
Quoted 15 lines
tools/lib/bpf/bpf_tracing.h +15
-0+ __builtin_choose_expr(sizeof(t) == 16, 2, \ + (void)0))) + +#define ____bpf_reg_cnt0 () (0) +#define ____bpf_reg_cnt1 (t, x) (____bpf_reg_cnt0() + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt2(t, x, args...) (____bpf_reg_cnt1(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt3(t, x, args...) (____bpf_reg_cnt2(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt4(t, x, args...) (____bpf_reg_cnt3(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt5(t, x, args...) (____bpf_reg_cnt4(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt6(t, x, args...) (____bpf_reg_cnt5(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt7(t, x, args...) (____bpf_reg_cnt6(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt8(t, x, args...) (____bpf_reg_cnt7(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt9(t, x, args...) (____bpf_reg_cnt8(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt10(t, x, args...) (____bpf_reg_cnt9(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt11(t, x, args...) (____bpf_reg_cnt10(args) + BPF_REG_CNT(t))
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 474
Quoted 10 lines
tools/lib/bpf/bpf_tracing.h +10
-0+#define ____bpf_reg_cnt12(t, x, args...) (____bpf_reg_cnt11(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt(args...) ___bpf_apply(____bpf_reg_cnt, ____bpf_narg(args))(args) + +#define ____bpf_union_arg (t, x, n) \ + __builtin_choose_expr(sizeof(t) == 1, ({ union { struct { __u8 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]}}; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 2, ({ union { struct { __u16 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 4, ({ union { struct { __u32 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 8, ({ union { struct { __u64 x; } ___z; t x; } ___tmp = {.___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 16, ({ union { struct { __u64 x, y; } ___z; t x; } ___tmp = {.___z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \ + (void)0)))))
looking at this again, we can do a bit better by using arrays, please consider using that. At the very least results in shorter lines:
#define ____bpf_union_arg(t, x, n) \
- __builtin_choose_expr(sizeof(t) == 1, ({ union { struct { __u8
x; } ___z; t x; } ___tmp = { .___z = {ctx[n]}}; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 2, ({ union { struct {__u16 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 4, ({ union { struct {__u32 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 8, ({ union { struct {__u64 x; } ___z; t x; } ___tmp = {.___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 16, ({ union { struct {__u64 x, y; } ___z; t x; } ___tmp = {.___z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 1, ({ union { __u8 z[1]; t
x; } ___tmp = { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 2, ({ union { __u16 z[1]; t
x; } ___tmp = { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 4, ({ union { __u32 z[1]; t
x; } ___tmp = { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 8, ({ union { __u64 z[1]; t
x; } ___tmp = {.z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 16, ({ union { __u64 z[2];
t x; } ___tmp = {.z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \
(void)0)))))It is using one- or two-element arrays, and it also has uniform {ctx[n]} or {ctx[n], ctx[n + 1]} initialization syntax. Seems a bit nicer than union { struct { ... combo.
Quoted 23 lines
tools/lib/bpf/bpf_tracing.h +23
-0+ +#define ____bpf_ctx_arg0(n, args...) +#define ____bpf_ctx_arg1 (n, t, x) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt1(t, x)) +#define ____bpf_ctx_arg2(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt2(t, x, args)) ____bpf_ctx_arg1(n, args) +#define ____bpf_ctx_arg3(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt3(t, x, args)) ____bpf_ctx_arg2(n, args) +#define ____bpf_ctx_arg4(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt4(t, x, args)) ____bpf_ctx_arg3(n, args) +#define ____bpf_ctx_arg5(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt5(t, x, args)) ____bpf_ctx_arg4(n, args) +#define ____bpf_ctx_arg6(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt6(t, x, args)) ____bpf_ctx_arg5(n, args) +#define ____bpf_ctx_arg7(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt7(t, x, args)) ____bpf_ctx_arg6(n, args) +#define ____bpf_ctx_arg8(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt8(t, x, args)) ____bpf_ctx_arg7(n, args) +#define ____bpf_ctx_arg9(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt9(t, x, args)) ____bpf_ctx_arg8(n, args) +#define ____bpf_ctx_arg10(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt10(t, x, args)) ____bpf_ctx_arg9(n, args) +#define ____bpf_ctx_arg11(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt11(t, x, args)) ____bpf_ctx_arg10(n, args) +#define ____bpf_ctx_arg12(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt12(t, x, args)) ____bpf_ctx_arg11(n, args) +#define ____bpf_ctx_arg(n, args...) ___bpf_apply(____bpf_ctx_arg, ____bpf_narg(args))(n, args) + +#define ____bpf_ctx_decl0 () +#define ____bpf_ctx_decl1 (t, x) , t x +#define ____bpf_ctx_decl2(t, x, args...) , t x ____bpf_ctx_decl1(args) +#define ____bpf_ctx_decl3(t, x, args...) , t x ____bpf_ctx_decl2(args) +#define ____bpf_ctx_decl4(t, x, args...) , t x ____bpf_ctx_decl3(args) +#define ____bpf_ctx_decl5(t, x, args...) , t x ____bpf_ctx_decl4(args) +#define ____bpf_ctx_decl6(t, x, args...) , t x ____bpf_ctx_decl5(args)
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 507
Quoted 10 lines
tools/lib/bpf/bpf_tracing.h +10
-0+#define ____bpf_ctx_decl7(t, x, args...) , t x ____bpf_ctx_decl6(args) +#define ____bpf_ctx_decl8(t, x, args...) , t x ____bpf_ctx_decl7(args) +#define ____bpf_ctx_decl9(t, x, args...) , t x ____bpf_ctx_decl8(args) +#define ____bpf_ctx_decl10(t, x, args...) , t x ____bpf_ctx_decl9(args) +#define ____bpf_ctx_decl11(t, x, args...) , t x ____bpf_ctx_decl10(args) +#define ____bpf_ctx_decl12(t, x, args...) , t x ____bpf_ctx_decl11(args) +#define ____bpf_ctx_decl(args...) ___bpf_apply(____bpf_ctx_decl, ____bpf_narg(args))(args) + +/* +* BPF_PROG2 can handle struct arguments.
We have to expand comment here. Let's not slack on this. Point out that it's the similar use and idea as with BPF_PROG, but emphasize the difference in syntax between BPF_PROG and BPF_PROG2. I'd show two simple examples of the same function with BPF_PROG and BPF_PROG2 here. Please follow up.
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 515
Quoted 8 lines
tools/lib/bpf/bpf_tracing.h +8
-0+*/ +#define BPF_PROG2(name, args...) \ +name(unsigned long long *ctx); \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)); \ +typeof(name(0)) name(unsigned long long *ctx) \ +{ \ + return ____##name(ctx ____bpf_ctx_arg(____bpf_reg_cnt(args), args)); \
nit: you could have simplified this by doing ____bpf_reg_cnt() call inside ____bpf_ctx_decl(args...) definition. I think it's a bit more self-contained that way.
Quoted 10 lines
tools/lib/bpf/bpf_tracing.h +4
-1+} \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)) +struct pt_regs ;#define ___bpf_kprobe_args0 () ctx --2.30.2
Re: [PATCH bpf-next v4 5/8] libbpf: Add new BPF_PROG2 macro
Quoted 64 lines · Andrii Nakryiko
Quoted 52 lines · Yonghong Song
To support struct arguments in trampoline based programs, existing
BPF_PROGdoesn't work any more since the type size is needed to find whether a parameter takes one or two registers. So this patch added a newBPF_PROG2macro to support such trampoline programs.The idea is suggested by Andrii. For example, if the to-be-traced function has signature like
typedef struct { void *x; int t; } sockptr; int blah(sockptr x, char y);In the new
BPF_PROG2macro, the argument can be
represented as
__bpf_prog_call(({ union { struct { __u64 x, y; } ___z; sockptr x; } ___tmp = { .___z = { ctx[0], ctx[1] }}; ___tmp.x; }), ({ union { struct { __u8 x; } ___z; char y; } ___tmp = { .___z = { ctx[2] }}; ___tmp.y; }));In the above, the values stored on the stack are properly assigned to the actual argument type value by using 'union' magic. Note that the macro also works even if no arguments are with struct types.
Note that new
BPF_PROG2works for both llvm16 and pre-llvm16 compilers where llvm16 supports bpf target passing value with struct up to 16 byte size and pre-llvm16 will pass by reference by storing values on the stack. With static functions with struct argument as always inline, the compiler is able to optimize and remove additional stack saving of struct values.Signed-off-by Yonghong Song <[email protected]>--- tools/lib/bpf/bpf_tracing.h | 79 +++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)tools/lib/bpf/bpf_tracing.h +0
-0tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 443
Quoted 6 lines
tools/lib/bpf/bpf_tracing.h +3
-0@@ -438,6 +438,85 @@ typeof(name(0)) name(unsigned long long *ctx) \ static __always_inline typeof(name(0)) \ ____##name(unsigned long long *ctx, ##args) +#ifndef ____bpf_nth +#define ____bpf_nth (_, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20, _21, _22, _23, _24,N , ...) N +#endif we already have
___bpf_nth(triple underscore) variant, wouldn't extending that one to support up to 24 argument work? It's quite confusing to have___bpf_nthand____bpf_nth. Maybe let's consolidate?
Quoted 1 line
And I'd totally wrap this long line :)
I tried earlier and it doesn't work. I will try again. If it still not works, I will change the name to ___bpf_nth2 similar to ___bpf_narg2 below.
Quoted 12 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 446
Quoted 5 lines
tools/lib/bpf/bpf_tracing.h +3
-0+#ifndef ____bpf_narg +#define ____bpf_narg (...) ____bpf_nth(_, ##__VA_ARGS__, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 0) +#endif similar confusiong with triple underscore
bpf_narg. Given this is used inBPF_PROG2, how about renaming it tobpf_narg2to make this connection? And also note that all similar macros use triple underscore, while you added quad underscores everywhere. Can you please follow up with a rename to use triple underscore for consistency?
Sounds good. Will change to ___bpf_narg2.
Quoted 6 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 448
Quoted 3 lines
tools/lib/bpf/bpf_tracing.h +2
-0+ +#define BPF_REG_CNT (t) \this looks like a "public API", but I don't think this was the intention, right? Let's rename it to
___bpf_reg_cnt()?
Yes, I can do it.
Quoted 6 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 449
Quoted 2 lines
tools/lib/bpf/bpf_tracing.h +1
-0+ (__builtin_choose_expr(sizeof(t) == 1 || sizeof(t) == 2 || sizeof(t) == 4 || sizeof(t) == 8, 1, \nit: seeing
____bpf_union_argimplementation below I prefer one case per line there as well. How about doing one__builtin_choose_exprper each supported size?
Actually, I did have each individual case per line in the beginning. But later on I changed to the above based on Kui-Feng's comment. I can change back to each case per line in the next revision to be aligned with other usages.
Quoted 51 lines
Quoted 16 lines
tools/lib/bpf/bpf_tracing.h +15
-0+ __builtin_choose_expr(sizeof(t) == 16, 2, \ + (void)0))) + +#define ____bpf_reg_cnt0 () (0) +#define ____bpf_reg_cnt1 (t, x) (____bpf_reg_cnt0() + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt2(t, x, args...) (____bpf_reg_cnt1(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt3(t, x, args...) (____bpf_reg_cnt2(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt4(t, x, args...) (____bpf_reg_cnt3(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt5(t, x, args...) (____bpf_reg_cnt4(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt6(t, x, args...) (____bpf_reg_cnt5(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt7(t, x, args...) (____bpf_reg_cnt6(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt8(t, x, args...) (____bpf_reg_cnt7(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt9(t, x, args...) (____bpf_reg_cnt8(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt10(t, x, args...) (____bpf_reg_cnt9(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt11(t, x, args...) (____bpf_reg_cnt10(args) + BPF_REG_CNT(t))tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 474
Quoted 10 lines
tools/lib/bpf/bpf_tracing.h +10
-0+#define ____bpf_reg_cnt12(t, x, args...) (____bpf_reg_cnt11(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt(args...) ___bpf_apply(____bpf_reg_cnt, ____bpf_narg(args))(args) + +#define ____bpf_union_arg (t, x, n) \ + __builtin_choose_expr(sizeof(t) == 1, ({ union { struct { __u8 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]}}; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 2, ({ union { struct { __u16 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 4, ({ union { struct { __u32 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 8, ({ union { struct { __u64 x; } ___z; t x; } ___tmp = {.___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 16, ({ union { struct { __u64 x, y; } ___z; t x; } ___tmp = {.___z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \ + (void)0)))))looking at this again, we can do a bit better by using arrays, please consider using that. At the very least results in shorter lines:
#define
____bpf_union_arg(t, x, n)\
- __builtin_choose_expr(sizeof(t) == 1, ({ union { struct {__u8
x; }___z; t x; }___tmp= { .___z = {ctx[n]}}; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 2, ({ union { struct {__u16x; }___z; t x; }___tmp= { .___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 4, ({ union { struct {__u32x; }___z; t x; }___tmp= { .___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 8, ({ union { struct {__u64x; }___z; t x; }___tmp= {.___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 16, ({ union { struct {__u64x, y; }___z; t x; }___tmp= {.___z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 1, ({ union {__u8z[1]; t
x; }___tmp= { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 2, ({ union {__u16z[1]; t
x; }___tmp= { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 4, ({ union {__u32z[1]; t
x; }___tmp= { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 8, ({ union {__u64z[1]; t
x; }___tmp= {.z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 16, ({ union {__u64z[2];
Quoted 6 lines
t x; } ___tmp = {.z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \ (void)0)))))It is using one- or two-element arrays, and it also has uniform {ctx[n]} or {ctx[n], ctx[n + 1]} initialization syntax. Seems a bit nicer than union { struct { ... combo.
Sounds good. Will try to do this.
Quoted 32 lines
Quoted 32 lines
tools/lib/bpf/bpf_tracing.h +31
-0+ +#define ____bpf_ctx_arg0(n, args...) +#define ____bpf_ctx_arg1 (n, t, x) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt1(t, x)) +#define ____bpf_ctx_arg2(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt2(t, x, args)) ____bpf_ctx_arg1(n, args) +#define ____bpf_ctx_arg3(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt3(t, x, args)) ____bpf_ctx_arg2(n, args) +#define ____bpf_ctx_arg4(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt4(t, x, args)) ____bpf_ctx_arg3(n, args) +#define ____bpf_ctx_arg5(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt5(t, x, args)) ____bpf_ctx_arg4(n, args) +#define ____bpf_ctx_arg6(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt6(t, x, args)) ____bpf_ctx_arg5(n, args) +#define ____bpf_ctx_arg7(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt7(t, x, args)) ____bpf_ctx_arg6(n, args) +#define ____bpf_ctx_arg8(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt8(t, x, args)) ____bpf_ctx_arg7(n, args) +#define ____bpf_ctx_arg9(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt9(t, x, args)) ____bpf_ctx_arg8(n, args) +#define ____bpf_ctx_arg10(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt10(t, x, args)) ____bpf_ctx_arg9(n, args) +#define ____bpf_ctx_arg11(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt11(t, x, args)) ____bpf_ctx_arg10(n, args) +#define ____bpf_ctx_arg12(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt12(t, x, args)) ____bpf_ctx_arg11(n, args) +#define ____bpf_ctx_arg(n, args...) ___bpf_apply(____bpf_ctx_arg, ____bpf_narg(args))(n, args) + +#define ____bpf_ctx_decl0 () +#define ____bpf_ctx_decl1 (t, x) , t x +#define ____bpf_ctx_decl2(t, x, args...) , t x ____bpf_ctx_decl1(args) +#define ____bpf_ctx_decl3(t, x, args...) , t x ____bpf_ctx_decl2(args) +#define ____bpf_ctx_decl4(t, x, args...) , t x ____bpf_ctx_decl3(args) +#define ____bpf_ctx_decl5(t, x, args...) , t x ____bpf_ctx_decl4(args) +#define ____bpf_ctx_decl6(t, x, args...) , t x ____bpf_ctx_decl5(args) +#define ____bpf_ctx_decl7(t, x, args...) , t x ____bpf_ctx_decl6(args) +#define ____bpf_ctx_decl8(t, x, args...) , t x ____bpf_ctx_decl7(args) +#define ____bpf_ctx_decl9(t, x, args...) , t x ____bpf_ctx_decl8(args) +#define ____bpf_ctx_decl10(t, x, args...) , t x ____bpf_ctx_decl9(args) +#define ____bpf_ctx_decl11(t, x, args...) , t x ____bpf_ctx_decl10(args) +#define ____bpf_ctx_decl12(t, x, args...) , t x ____bpf_ctx_decl11(args) +#define ____bpf_ctx_decl(args...) ___bpf_apply(____bpf_ctx_decl, ____bpf_narg(args))(args) +
Quoted 8 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 507
Quoted 2 lines
tools/lib/bpf/bpf_tracing.h +2
-0+/* +* BPF_PROG2 can handle struct arguments. We have to expand comment here. Let's not slack on this. Point out that it's the similar use and idea as with
BPF_PROG, but emphasize the difference in syntax betweenBPF_PROGandBPF_PROG2. I'd show two simple examples of the same function withBPF_PROGandBPF_PROG2here. Please follow up.
Will add detailed comments to explain why BPF_PROG2 and its core usage and difference from BPF_PROG in the followup patch.
Quoted 13 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 515
Quoted 9 lines
tools/lib/bpf/bpf_tracing.h +8
-0+*/ +#define BPF_PROG2(name, args...) \ +name(unsigned long long *ctx); \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)); \ +typeof(name(0)) name(unsigned long long *ctx) \ +{ \ + return ____##name(ctx ____bpf_ctx_arg(____bpf_reg_cnt(args), args)); \nit: you could have simplified this by doing
____bpf_reg_cnt()call inside____bpf_ctx_decl(args...)definition. I think it's a bit more self-contained that way.
Will do this in the follow-up patch.
Quoted 11 lines
Quoted 11 lines
tools/lib/bpf/bpf_tracing.h +4
-1+} \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)) +struct pt_regs ;#define ___bpf_kprobe_args0 () ctx --2.30.2
Re: [PATCH bpf-next v4 5/8] libbpf: Add new BPF_PROG2 macro
Quoted 68 lines · Yonghong Song
Quoted 64 lines · Andrii Nakryiko
Quoted 52 lines · Yonghong Song
To support struct arguments in trampoline based programs, existing
BPF_PROGdoesn't work any more since the type size is needed to find whether a parameter takes one or two registers. So this patch added a newBPF_PROG2macro to support such trampoline programs.The idea is suggested by Andrii. For example, if the to-be-traced function has signature like
typedef struct { void *x; int t; } sockptr; int blah(sockptr x, char y);In the new
BPF_PROG2macro, the argument can be
represented as
__bpf_prog_call(({ union { struct { __u64 x, y; } ___z; sockptr x; } ___tmp = { .___z = { ctx[0], ctx[1] }}; ___tmp.x; }), ({ union { struct { __u8 x; } ___z; char y; } ___tmp = { .___z = { ctx[2] }}; ___tmp.y; }));In the above, the values stored on the stack are properly assigned to the actual argument type value by using 'union' magic. Note that the macro also works even if no arguments are with struct types.
Note that new
BPF_PROG2works for both llvm16 and pre-llvm16 compilers where llvm16 supports bpf target passing value with struct up to 16 byte size and pre-llvm16 will pass by reference by storing values on the stack. With static functions with struct argument as always inline, the compiler is able to optimize and remove additional stack saving of struct values.Signed-off-by Yonghong Song <[email protected]>--- tools/lib/bpf/bpf_tracing.h | 79 +++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)tools/lib/bpf/bpf_tracing.h +0
-0tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 443
Quoted 6 lines
tools/lib/bpf/bpf_tracing.h +3
-0@@ -438,6 +438,85 @@ typeof(name(0)) name(unsigned long long *ctx) \ static __always_inline typeof(name(0)) \ ____##name(unsigned long long *ctx, ##args) +#ifndef ____bpf_nth +#define ____bpf_nth (_, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20, _21, _22, _23, _24,N , ...) N +#endif we already have
___bpf_nth(triple underscore) variant, wouldn't extending that one to support up to 24 argument work? It's quite confusing to have___bpf_nthand____bpf_nth. Maybe let's consolidate?
Quoted 4 lines
Quoted 1 line
And I'd totally wrap this long line :)
I tried earlier and it doesn't work. I will try again. If it still not works, I will change the name to
___bpf_nth2similar to___bpf_narg2below.
Yeah, that was the fallback I had in mind. ___bpf_nth2 is fine as well.
Quoted 28 lines
Quoted 13 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 446
Quoted 6 lines
tools/lib/bpf/bpf_tracing.h +3
-0+#ifndef ____bpf_narg +#define ____bpf_narg (...) ____bpf_nth(_, ##__VA_ARGS__, 12, 12, 11, 11, 10, 10, 9, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 0) +#endif similar confusiong with triple underscore
bpf_narg. Given this is used inBPF_PROG2, how about renaming it tobpf_narg2to make this connection? And also note that all similar macros use triple underscore, while you added quad underscores everywhere. Can you please follow up with a rename to use triple underscore for consistency?Sounds good. Will change to
___bpf_narg2.Quoted 6 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 448
Quoted 3 lines
tools/lib/bpf/bpf_tracing.h +2
-0+ +#define BPF_REG_CNT (t) \this looks like a "public API", but I don't think this was the intention, right? Let's rename it to
___bpf_reg_cnt()?Yes, I can do it.
Quoted 3 lines
Quoted 2 lines
tools/lib/bpf/bpf_tracing.h +1
-0+ (__builtin_choose_expr(sizeof(t) == 1 || sizeof(t) == 2 || sizeof(t) == 4 || sizeof(t) == 8, 1, \
Quoted 9 lines
Quoted 3 lines
nit: seeing
____bpf_union_argimplementation below I prefer one case per line there as well. How about doing one__builtin_choose_exprper each supported size?Actually, I did have each individual case per line in the beginning. But later on I changed to the above based on Kui-Feng's comment. I can change back to each case per line in the next revision to be aligned with other usages.
Didn't see that conversation, sorry. But my point stands, for consistency with ___bpf_union_arg and to cut this very long line shorter it makes more sense to have one case per line.
Quoted 101 lines
Quoted 51 lines
Quoted 16 lines
tools/lib/bpf/bpf_tracing.h +15
-0+ __builtin_choose_expr(sizeof(t) == 16, 2, \ + (void)0))) + +#define ____bpf_reg_cnt0 () (0) +#define ____bpf_reg_cnt1 (t, x) (____bpf_reg_cnt0() + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt2(t, x, args...) (____bpf_reg_cnt1(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt3(t, x, args...) (____bpf_reg_cnt2(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt4(t, x, args...) (____bpf_reg_cnt3(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt5(t, x, args...) (____bpf_reg_cnt4(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt6(t, x, args...) (____bpf_reg_cnt5(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt7(t, x, args...) (____bpf_reg_cnt6(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt8(t, x, args...) (____bpf_reg_cnt7(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt9(t, x, args...) (____bpf_reg_cnt8(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt10(t, x, args...) (____bpf_reg_cnt9(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt11(t, x, args...) (____bpf_reg_cnt10(args) + BPF_REG_CNT(t))tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 474
Quoted 10 lines
tools/lib/bpf/bpf_tracing.h +10
-0+#define ____bpf_reg_cnt12(t, x, args...) (____bpf_reg_cnt11(args) + BPF_REG_CNT(t)) +#define ____bpf_reg_cnt(args...) ___bpf_apply(____bpf_reg_cnt, ____bpf_narg(args))(args) + +#define ____bpf_union_arg (t, x, n) \ + __builtin_choose_expr(sizeof(t) == 1, ({ union { struct { __u8 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]}}; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 2, ({ union { struct { __u16 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 4, ({ union { struct { __u32 x; } ___z; t x; } ___tmp = { .___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 8, ({ union { struct { __u64 x; } ___z; t x; } ___tmp = {.___z = {ctx[n]} }; ___tmp.x; }), \ + __builtin_choose_expr(sizeof(t) == 16, ({ union { struct { __u64 x, y; } ___z; t x; } ___tmp = {.___z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \ + (void)0)))))looking at this again, we can do a bit better by using arrays, please consider using that. At the very least results in shorter lines:
#define
____bpf_union_arg(t, x, n)\
- __builtin_choose_expr(sizeof(t) == 1, ({ union { struct {__u8
x; }___z; t x; }___tmp= { .___z = {ctx[n]}}; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 2, ({ union { struct {__u16x; }___z; t x; }___tmp= { .___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 4, ({ union { struct {__u32x; }___z; t x; }___tmp= { .___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 8, ({ union { struct {__u64x; }___z; t x; }___tmp= {.___z = {ctx[n]} }; ___tmp.x; }), \
- __builtin_choose_expr(sizeof(t) == 16, ({ union { struct {__u64x, y; }___z; t x; }___tmp= {.___z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 1, ({ union {__u8z[1]; t
x; }___tmp= { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 2, ({ union {__u16z[1]; t
x; }___tmp= { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 4, ({ union {__u32z[1]; t
x; }___tmp= { .z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 8, ({ union {__u64z[1]; t
x; }___tmp= {.z = {ctx[n]} }; ___tmp.x; }), \
+ __builtin_choose_expr(sizeof(t) == 16, ({ union {__u64z[2];Quoted 6 lines
t x; } ___tmp = {.z = {ctx[n], ctx[n + 1]} }; ___tmp.x; }), \ (void)0)))))It is using one- or two-element arrays, and it also has uniform {ctx[n]} or {ctx[n], ctx[n + 1]} initialization syntax. Seems a bit nicer than union { struct { ... combo.
Sounds good. Will try to do this.
Quoted 32 lines
Quoted 32 lines
tools/lib/bpf/bpf_tracing.h +31
-0+ +#define ____bpf_ctx_arg0(n, args...) +#define ____bpf_ctx_arg1 (n, t, x) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt1(t, x)) +#define ____bpf_ctx_arg2(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt2(t, x, args)) ____bpf_ctx_arg1(n, args) +#define ____bpf_ctx_arg3(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt3(t, x, args)) ____bpf_ctx_arg2(n, args) +#define ____bpf_ctx_arg4(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt4(t, x, args)) ____bpf_ctx_arg3(n, args) +#define ____bpf_ctx_arg5(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt5(t, x, args)) ____bpf_ctx_arg4(n, args) +#define ____bpf_ctx_arg6(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt6(t, x, args)) ____bpf_ctx_arg5(n, args) +#define ____bpf_ctx_arg7(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt7(t, x, args)) ____bpf_ctx_arg6(n, args) +#define ____bpf_ctx_arg8(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt8(t, x, args)) ____bpf_ctx_arg7(n, args) +#define ____bpf_ctx_arg9(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt9(t, x, args)) ____bpf_ctx_arg8(n, args) +#define ____bpf_ctx_arg10(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt10(t, x, args)) ____bpf_ctx_arg9(n, args) +#define ____bpf_ctx_arg11(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt11(t, x, args)) ____bpf_ctx_arg10(n, args) +#define ____bpf_ctx_arg12(n, t, x, args...) , ____bpf_union_arg(t, x, n - ____bpf_reg_cnt12(t, x, args)) ____bpf_ctx_arg11(n, args) +#define ____bpf_ctx_arg(n, args...) ___bpf_apply(____bpf_ctx_arg, ____bpf_narg(args))(n, args) + +#define ____bpf_ctx_decl0 () +#define ____bpf_ctx_decl1 (t, x) , t x +#define ____bpf_ctx_decl2(t, x, args...) , t x ____bpf_ctx_decl1(args) +#define ____bpf_ctx_decl3(t, x, args...) , t x ____bpf_ctx_decl2(args) +#define ____bpf_ctx_decl4(t, x, args...) , t x ____bpf_ctx_decl3(args) +#define ____bpf_ctx_decl5(t, x, args...) , t x ____bpf_ctx_decl4(args) +#define ____bpf_ctx_decl6(t, x, args...) , t x ____bpf_ctx_decl5(args) +#define ____bpf_ctx_decl7(t, x, args...) , t x ____bpf_ctx_decl6(args) +#define ____bpf_ctx_decl8(t, x, args...) , t x ____bpf_ctx_decl7(args) +#define ____bpf_ctx_decl9(t, x, args...) , t x ____bpf_ctx_decl8(args) +#define ____bpf_ctx_decl10(t, x, args...) , t x ____bpf_ctx_decl9(args) +#define ____bpf_ctx_decl11(t, x, args...) , t x ____bpf_ctx_decl10(args) +#define ____bpf_ctx_decl12(t, x, args...) , t x ____bpf_ctx_decl11(args) +#define ____bpf_ctx_decl(args...) ___bpf_apply(____bpf_ctx_decl, ____bpf_narg(args))(args) +Quoted 8 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 507
Quoted 2 lines
tools/lib/bpf/bpf_tracing.h +2
-0+/* +* BPF_PROG2 can handle struct arguments. We have to expand comment here. Let's not slack on this. Point out that it's the similar use and idea as with
BPF_PROG, but emphasize the difference in syntax betweenBPF_PROGandBPF_PROG2. I'd show two simple examples of the same function withBPF_PROGandBPF_PROG2here. Please follow up.
Quoted 3 lines
Will add detailed comments to explain why
BPF_PROG2and its core usage and difference fromBPF_PROGin the followup patch.
cool, thanks!
Quoted 27 lines
Quoted 13 lines
tools/lib/bpf/bpf_tracing.h typeof(name(0)) name(unsigned long long *ctx) \ · line 515
Quoted 9 lines
tools/lib/bpf/bpf_tracing.h +8
-0+*/ +#define BPF_PROG2(name, args...) \ +name(unsigned long long *ctx); \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)); \ +typeof(name(0)) name(unsigned long long *ctx) \ +{ \ + return ____##name(ctx ____bpf_ctx_arg(____bpf_reg_cnt(args), args)); \nit: you could have simplified this by doing
____bpf_reg_cnt()call inside____bpf_ctx_decl(args...)definition. I think it's a bit more self-contained that way.Will do this in the follow-up patch.
Quoted 11 lines
Quoted 11 lines
tools/lib/bpf/bpf_tracing.h +4
-1+} \ +static __always_inline typeof(name(0)) \ +____##name(unsigned long long *ctx ____bpf_ctx_decl(args)) +struct pt_regs ;#define ___bpf_kprobe_args0 () ctx --2.30.2
PATCH v4 6/8 selftests/bpf: Add struct argument tests with fentry/fexit programs.
In reply to Yonghong SongAdd various struct argument tests with fentry/fexit programs. Also add one test with a kernel func which does not have any argument to test BPF_PROG2 macro in such situation.
--- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 48 +++++++ .../selftests/bpf/prog_tests/tracing_struct.c | 63 +++++++++ .../selftests/bpf/progs/tracing_struct.c | 120 ++++++++++++++++++ 3 files changed, 231 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/tracing_struct.c create mode 100644 tools/testing/selftests/bpf/progs/tracing_struct.c
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +48 -0
@@ -18,6 +18,46 @@ typedef int (*func_proto_typedef_nested1)(func_proto_typedef);typedef int (*func_proto_typedef_nested2 )(func_proto_typedef_nested1 );DEFINE_PER_CPU (int, bpf_testmod_ksym_percpu) =123 ; +long bpf_testmod_test_struct_arg_result; + +struct bpf_testmod_struct_arg_1 { +int a; +}; +struct bpf_testmod_struct_arg_2 { +long a; +long b; +}; + +noinline int +bpf_testmod_test_struct_arg_1 (struct bpf_testmod_struct_arg_2 a,int b,int c) { + bpf_testmod_test_struct_arg_result = a.a + a.b + b + c; +return bpf_testmod_test_struct_arg_result; +} + +noinline int +bpf_testmod_test_struct_arg_2 (int a,struct bpf_testmod_struct_arg_2 b,int c) { + bpf_testmod_test_struct_arg_result = a + b.a + b.b + c; +return bpf_testmod_test_struct_arg_result; +} + +noinline int +bpf_testmod_test_struct_arg_3 (int a,int b,struct bpf_testmod_struct_arg_2 c) { + bpf_testmod_test_struct_arg_result = a + b + c.a + c.b; +return bpf_testmod_test_struct_arg_result; +} + +noinline int +bpf_testmod_test_struct_arg_4 (struct bpf_testmod_struct_arg_1 a,int b, +int c,int d,struct bpf_testmod_struct_arg_2 e) { + bpf_testmod_test_struct_arg_result = a.a + b + c + d + e.a + e.b; +return bpf_testmod_test_struct_arg_result; +} + +noinline int +bpf_testmod_test_struct_arg_5 (void ) { + bpf_testmod_test_struct_arg_result =1 ; +return bpf_testmod_test_struct_arg_result; +} noinline voidbpf_testmod_test_mod_kfunc (int i)
@@ -98,11 +138,19 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj, .off = off, .len = len, }; +struct bpf_testmod_struct_arg_1 struct_arg1 = {10 }; +struct bpf_testmod_struct_arg_2 struct_arg2 = {2 ,3 };int i =1 ;while (bpf_testmod_return_ptr (i)) i++; + (void )bpf_testmod_test_struct_arg_1 (struct_arg2,1 ,4 ); + (void )bpf_testmod_test_struct_arg_2 (1 , struct_arg2,4 ); + (void )bpf_testmod_test_struct_arg_3 (1 ,4 , struct_arg2); + (void )bpf_testmod_test_struct_arg_4 (struct_arg1,1 ,2 ,3 , struct_arg2); + (void )bpf_testmod_test_struct_arg_5 (); +/* This is always true. Use the check to make sure the compiler * doesn't remove bpf_testmod_loop_test. */
tools/testing/selftests/bpf/prog_tests/tracing_struct.c new file +63 -0
63 lines not on this page · raw message
tools/testing/selftests/bpf/progs/tracing_struct.c new file +120 -0
120 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v4 7/8 selftests/bpf: Use BPF_PROG2 for some fentry programs without struct arguments
In reply to Yonghong SongUse BPF_PROG2 instead of BPF_PROG for programs in progs/timer.c to test BPF_PROG2 for cases without struct arguments.
--- tools/testing/selftests/bpf/progs/timer.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/timer.c +2 -2
@@ -120,7 +120,7 @@ static int timer_cb1(void *map, int *key, struct bpf_timer *timer) } SEC("fentry/bpf_fentry_test1") -int BPF_PROG (test1 ,int a) +int BPF_PROG2 (test1 ,int ,a ) {struct bpf_timer *arr_timer, *lru_timer;struct elem init = {};
@@ -247,7 +247,7 @@ int bpf_timer_test(void) } SEC("fentry/bpf_fentry_test2") -int BPF_PROG (test2 ,int a,int b) +int BPF_PROG2 (test2 ,int ,a ,int ,b ) {struct hmap_elem init = {}, *val;int key =HTAB , key_malloc =HTAB_MALLOC ;
Signature · 1 line
2.30.2
PATCH v4 8/8 selftests/bpf: Add tracing_struct test in DENYLIST.s390x
In reply to Yonghong SongAdd tracing_struct test in DENYLIST.s390x since s390x does not support trampoline now.
--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + 1 file changed, 1 insertion(+)
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -68,3 +68,4 @@ unpriv_bpf_disabled # fentry setget_sockopt # attach unexpected error: -524 (trampoline) cb_refs # expected error message unexpected error: -524 (trampoline) cgroup_hierarchical_stats # JIT does not support calling kernel function (kfunc) +tracing_struct # failed to auto-attach: -524 (trampoline)
Signature · 1 line
2.30.2
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 23 lines · On Wed, 31 Aug 2022 08:26:41 -0700 you wrote:
Currently struct arguments are not supported for trampoline based progs. One of major reason is that struct argument may pass by value which may use more than one registers. This breaks trampoline progs where each argument is assumed to take one register. bcc community reported the issue ([1]) where struct argument is not supported for fentry program.
typedef struct { uid_t val; } kuid_t; typedef struct { gid_t val; } kgid_t; int security_path_chown(struct path *path, kuid_t uid, kgid_t gid); Inside Meta, we also have a use case to attach to tcp_setsockopt() typedef struct { union { void *kernel; void __user *user; }; bool is_kernel : 1; } sockptr_t; int tcp_setsockopt(struct sock *sk, int level, int optname, sockptr_t optval, unsigned int optlen);
Quoted 1 line
[...]
Here is the summary with links:
- [bpf-next,v4,1/8] bpf: Allow struct argument in trampoline based programs
https://git.kernel.org/bpf/bpf-next/c/720e6a435194 - [bpf-next,v4,2/8] bpf: x86: Support in-register struct arguments in trampoline programs https://git.kernel.org/bpf/bpf-next/c/a9c5ad31fbdc - [bpf-next,v4,3/8] bpf: Update descriptions for helpers bpf_get_func_arg[_cnt]() https://git.kernel.org/bpf/bpf-next/c/27ed9353aec9 - [bpf-next,v4,4/8] bpf: arm64: No support of struct argument in trampoline programs https://git.kernel.org/bpf/bpf-next/c/eb707dde264a - [bpf-next,v4,5/8] libbpf: Add new BPF_PROG2 macro https://git.kernel.org/bpf/bpf-next/c/34586d29f8df - [bpf-next,v4,6/8] selftests/bpf: Add struct argument tests with fentry/fexit programs. https://git.kernel.org/bpf/bpf-next/c/1642a3945e22 - [bpf-next,v4,7/8] selftests/bpf: Use BPF_PROG2 for some fentry programs without struct arguments https://git.kernel.org/bpf/bpf-next/c/a7c2ca3a2f69 - [bpf-next,v4,8/8] selftests/bpf: Add tracing_struct test in DENYLIST.s390x https://git.kernel.org/bpf/bpf-next/c/ae63c10fc241
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html