bpf: Use fake pt_regs when doing bpf syscall tracepoint tracing
Yonghong Song
reviewerPATCH 1/1 bpf: Use fake pt_regs when doing bpf syscall tracepoint tracing
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read { print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===
Further analysis shows that pt_regs used for bpf syscall tracepoint tracing is from the one constructed during user->kernel transition. The call stack looks like
perf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7e
The ip address stored in pt_regs is from user space hence no kernel stack is printed.
To fix the issue, we need to use kernel address from pt_regs. In kernel repo, there are already a few cases like this. For example, in kernel/trace/bpf_trace.c, several perf_fetch_caller_regs(fake_regs_ptr) instances are used to supply ip address or use ip address to construct call stack.
The patch follows the above example by using a fake pt_regs. The pt_regs is stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to a perf_fetch_caller_regs() use case in kernel/trace/trace_event_perf.c with function perf_ftrace_function_call() where a local pt_regs is used.
With this patch, for the above bpftrace script, I got the following output
===
Kernel Stack
syscall_trace_enter+407
syscall_trace_enter+407
do_syscall_64+74
entry_SYSCALL_64_after_hwframe+75===
--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/trace/trace_syscalls.c +4 -1
@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *reint syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs)); +perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i];
Signature · 1 line
2.43.5
Quoted 44 lines · Yonghong Song
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read {print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===Further analysis shows that
pt_regsused for bpf syscall tracepoint tracing is from the one constructed duringuser->kerneltransition. The call stack looks likeperf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7eThe ip address stored in
pt_regsis from user space hence no kernel stack is printed.To fix the issue, we need to use kernel address from
pt_regs. In kernel repo, there are already a few cases like this. For example, inkernel/trace/bpf_trace.c, severalperf_fetch_caller_regs(fake_regs_ptr)instances are used to supply ip address or use ip address to construct call stack.The patch follows the above example by using a fake
pt_regs. Thept_regsis stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to aperf_fetch_caller_regs()use case inkernel/trace/trace_event_perf.cwith functionperf_ftrace_function_call()where a localpt_regsis used.With this patch, for the above bpftrace script, I got the following output
===
Kernel Stacksyscall_trace_enter+407 syscall_trace_enter+407 do_syscall_64+74 entry_SYSCALL_64_after_hwframe+75
Quoted 9 lines
===
Reported-by Salvatore Benedetto <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Note, we need to solve the same for perf_call_bpf_exit().
pw-bot: cr
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
Quoted 5 lines
kernel/trace/trace_syscalls.c +0
-0@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
let's also drop struct pt_regs * argument into perf_call_bpf_{enter,exit}(), they are not actually used anymore
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re · line 568
Quoted 11 lines
kernel/trace/trace_syscalls.c +2
-1int syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs));
sizeof(struct pt_regs) == 168 on x86-64, and on arm64 it's a whopping 336 bytes, so these memset(0) calls are not free for sure.
But we don't need to do this unnecessary work all the time.
I initially was going to suggest to use get_bpf_raw_tp_regs() from kernel/trace/bpf_trace.c to get a temporary pt_regs that was already memset(0) and used to initialize these minimal "fake regs".
But, it turns out we don't need to do even that. Note perf_trace_buf_alloc(), it has struct pt_regs ** second argument, and if you pass a valid pointer there, it will return "fake regs" struct to be used. We already use that functionality in perf_trace_##call in include/trace/perf.h (i.e., non-syscall tracepoints), so this seems to be a perfect fit.
Quoted 8 lines
kernel/trace/trace_syscalls.c +2
-1+perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i]; --2.43.5
Quoted 56 lines · Andrii Nakryiko
Quoted 44 lines · Yonghong Song
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read {print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===Further analysis shows that
pt_regsused for bpf syscall tracepoint tracing is from the one constructed duringuser->kerneltransition. The call stack looks likeperf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7eThe ip address stored in
pt_regsis from user space hence no kernel stack is printed.To fix the issue, we need to use kernel address from
pt_regs. In kernel repo, there are already a few cases like this. For example, inkernel/trace/bpf_trace.c, severalperf_fetch_caller_regs(fake_regs_ptr)instances are used to supply ip address or use ip address to construct call stack.The patch follows the above example by using a fake
pt_regs. Thept_regsis stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to aperf_fetch_caller_regs()use case inkernel/trace/trace_event_perf.cwith functionperf_ftrace_function_call()where a localpt_regsis used.With this patch, for the above bpftrace script, I got the following output
===
Kernel Stacksyscall_trace_enter+407 syscall_trace_enter+407 do_syscall_64+74 entry_SYSCALL_64_after_hwframe+75Quoted 9 lines
===
Reported-by Salvatore Benedetto <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Quoted 4 lines
Note, we need to solve the same for
perf_call_bpf_exit().pw-bot: cr
BTW, we lived with this bug for years, so I suggest basing your fix on top of bpf-next/master, no bpf/master, which will give people a bit of time to validate that the fix works as expected and doesn't produce any undesirable side effects, before this makes it into the final Linux release.
Quoted 45 lines
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
Quoted 5 lines
kernel/trace/trace_syscalls.c +0
-0@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *relet's also drop struct
pt_regs* argument into perf_call_bpf_{enter,exit}(), they are not actually used anymorekernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re · line 568
Quoted 11 lines
kernel/trace/trace_syscalls.c +2
-1int syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs));
sizeof(struct pt_regs)== 168 on x86-64, and on arm64 it's a whopping 336 bytes, so these memset(0) calls are not free for sure.But we don't need to do this unnecessary work all the time.
I initially was going to suggest to use
get_bpf_raw_tp_regs()fromkernel/trace/bpf_trace.cto get a temporarypt_regsthat was already memset(0) and used to initialize these minimal "fake regs".But, it turns out we don't need to do even that. Note
perf_trace_buf_alloc(), it hasstruct pt_regs **second argument, and if you pass a valid pointer there, it will return "fake regs" struct to be used. We already use that functionality in perf_trace_##call ininclude/trace/perf.h(i.e., non-syscall tracepoints), so this seems to be a perfect fit.Quoted 8 lines
kernel/trace/trace_syscalls.c +2
-1+perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i]; --2.43.5
Yonghong Song
reviewerQuoted 57 lines · Andrii Nakryiko
Quoted 55 lines · Andrii Nakryiko
Quoted 43 lines · Yonghong Song
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read {print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===Further analysis shows that
pt_regsused for bpf syscall tracepoint tracing is from the one constructed duringuser->kerneltransition. The call stack looks likeperf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7eThe ip address stored in
pt_regsis from user space hence no kernel stack is printed.To fix the issue, we need to use kernel address from
pt_regs. In kernel repo, there are already a few cases like this. For example, inkernel/trace/bpf_trace.c, severalperf_fetch_caller_regs(fake_regs_ptr)instances are used to supply ip address or use ip address to construct call stack.The patch follows the above example by using a fake
pt_regs. Thept_regsis stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to aperf_fetch_caller_regs()use case inkernel/trace/trace_event_perf.cwith functionperf_ftrace_function_call()where a localpt_regsis used.With this patch, for the above bpftrace script, I got the following output
===
Kernel Stacksyscall_trace_enter+407 syscall_trace_enter+407 do_syscall_64+74 entry_SYSCALL_64_after_hwframe+75Quoted 9 lines
===
Reported-by Salvatore Benedetto <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Note, we need to solve the same for
perf_call_bpf_exit().
Quoted 7 lines
Quoted 2 lines
pw-bot: cr
BTW, we lived with this bug for years, so I suggest basing your fix on top of bpf-next/master, no bpf/master, which will give people a bit of time to validate that the fix works as expected and doesn't produce any undesirable side effects, before this makes it into the final Linux release.
Yes, I did. See I indeed use 'bpf-next' in subject above.
Quoted 44 lines
Quoted 44 lines
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
Quoted 6 lines
kernel/trace/trace_syscalls.c +0
-0@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *relet's also drop struct
pt_regs* argument into perf_call_bpf_{enter,exit}(), they are not actually used anymorekernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re · line 568
Quoted 11 lines
kernel/trace/trace_syscalls.c +2
-1int syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs));
sizeof(struct pt_regs)== 168 on x86-64, and on arm64 it's a whopping 336 bytes, so these memset(0) calls are not free for sure.But we don't need to do this unnecessary work all the time.
I initially was going to suggest to use
get_bpf_raw_tp_regs()fromkernel/trace/bpf_trace.cto get a temporarypt_regsthat was already memset(0) and used to initialize these minimal "fake regs".But, it turns out we don't need to do even that. Note
perf_trace_buf_alloc(), it hasstruct pt_regs **second argument, and if you pass a valid pointer there, it will return "fake regs" struct to be used. We already use that functionality in perf_trace_##call ininclude/trace/perf.h(i.e., non-syscall tracepoints), so this seems to be a perfect fit.Quoted 8 lines
kernel/trace/trace_syscalls.c +2
-1+perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i]; --2.43.5
Quoted 62 lines · Yonghong Song
Quoted 59 lines · Andrii Nakryiko
Quoted 57 lines · Andrii Nakryiko
Quoted 43 lines · Yonghong Song
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read {print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===Further analysis shows that
pt_regsused for bpf syscall tracepoint tracing is from the one constructed duringuser->kerneltransition. The call stack looks likeperf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7eThe ip address stored in
pt_regsis from user space hence no kernel stack is printed.To fix the issue, we need to use kernel address from
pt_regs. In kernel repo, there are already a few cases like this. For example, inkernel/trace/bpf_trace.c, severalperf_fetch_caller_regs(fake_regs_ptr)instances are used to supply ip address or use ip address to construct call stack.The patch follows the above example by using a fake
pt_regs. Thept_regsis stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to aperf_fetch_caller_regs()use case inkernel/trace/trace_event_perf.cwith functionperf_ftrace_function_call()where a localpt_regsis used.With this patch, for the above bpftrace script, I got the following output
===
Kernel Stacksyscall_trace_enter+407 syscall_trace_enter+407 do_syscall_64+74 entry_SYSCALL_64_after_hwframe+75Quoted 9 lines
===
Reported-by Salvatore Benedetto <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Note, we need to solve the same for
perf_call_bpf_exit().pw-bot: cr
Quoted 7 lines
Quoted 5 lines
BTW, we lived with this bug for years, so I suggest basing your fix on top of bpf-next/master, no bpf/master, which will give people a bit of time to validate that the fix works as expected and doesn't produce any undesirable side effects, before this makes it into the final Linux release.
Yes, I did. See I indeed use 'bpf-next' in subject above.
Huh, strange, I actually tried to apply your patch to bpf-next/master and it didn't apply cleanly. It did apply to bpf/master, though, which is why I assumed you based it off of bpf/master.
Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
Quoted 7 lines
kernel/trace/trace_syscalls.c +0
-0@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *relet's also drop struct
pt_regs* argument into perf_call_bpf_{enter,exit}(), they are not actually used anymorekernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re · line 568
Quoted 11 lines
kernel/trace/trace_syscalls.c +2
-1int syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs));
sizeof(struct pt_regs)== 168 on x86-64, and on arm64 it's a whopping 336 bytes, so these memset(0) calls are not free for sure.But we don't need to do this unnecessary work all the time.
I initially was going to suggest to use
get_bpf_raw_tp_regs()fromkernel/trace/bpf_trace.cto get a temporarypt_regsthat was already memset(0) and used to initialize these minimal "fake regs".But, it turns out we don't need to do even that. Note
perf_trace_buf_alloc(), it hasstruct pt_regs **second argument, and if you pass a valid pointer there, it will return "fake regs" struct to be used. We already use that functionality in perf_trace_##call ininclude/trace/perf.h(i.e., non-syscall tracepoints), so this seems to be a perfect fit.Quoted 8 lines
kernel/trace/trace_syscalls.c +2
-1+perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i]; --2.43.5
Yonghong Song
reviewerQuoted 67 lines · Andrii Nakryiko
Quoted 66 lines · Yonghong Song
Quoted 64 lines · Andrii Nakryiko
Quoted 53 lines · Andrii Nakryiko
Quoted 43 lines · Yonghong Song
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read {print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===Further analysis shows that
pt_regsused for bpf syscall tracepoint tracing is from the one constructed duringuser->kerneltransition. The call stack looks likeperf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7eThe ip address stored in
pt_regsis from user space hence no kernel stack is printed.To fix the issue, we need to use kernel address from
pt_regs. In kernel repo, there are already a few cases like this. For example, inkernel/trace/bpf_trace.c, severalperf_fetch_caller_regs(fake_regs_ptr)instances are used to supply ip address or use ip address to construct call stack.The patch follows the above example by using a fake
pt_regs. Thept_regsis stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to aperf_fetch_caller_regs()use case inkernel/trace/trace_event_perf.cwith functionperf_ftrace_function_call()where a localpt_regsis used.With this patch, for the above bpftrace script, I got the following output
===Kernel Stack syscall_trace_enter+407 syscall_trace_enter+407 do_syscall_64+74 entry_SYSCALL_64_after_hwframe+75Quoted 9 lines
===
Reported-by Salvatore Benedetto <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Quoted 4 lines
Note, we need to solve the same for
perf_call_bpf_exit().pw-bot: cr
BTW, we lived with this bug for years, so I suggest basing your fix on top of bpf-next/master, no bpf/master, which will give people a bit of time to validate that the fix works as expected and doesn't produce any undesirable side effects, before this makes it into the final Linux release.
Quoted 4 lines
Quoted 1 line
Yes, I did. See I indeed use 'bpf-next' in subject above.
Huh, strange, I actually tried to apply your patch to bpf-next/master and it didn't apply cleanly. It did apply to bpf/master, though, which is why I assumed you based it off of bpf/master.
Interesting. The following is my git history:
7b71206057440d9559ecb9cd02d891f46927b272 (HEAD -> trace_syscall) bpf: Use fake pt_regs when doing bpf syscall tracepoint tracing 41d0c4677feee1ea063e0f2c2af72dc953b1f1cc (origin/master, origin/HEAD, master) libbpf: Fix some typos in comments 72d8508ecd3b081dba03ec00930c6b07c1ad55d3 MAINTAINERS: BPF ARC JIT: Update my e-mail address bee109b7b3e50739b88252a219fa07ecd78ad628 bpf: Fix error message on kfunc arg type mismatch
...
Not sure what is going on ...
Quoted 44 lines
Quoted 44 lines
Quoted 44 lines
Quoted 44 lines
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
Quoted 6 lines
kernel/trace/trace_syscalls.c +0
-0@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *relet's also drop struct
pt_regs* argument into perf_call_bpf_{enter,exit}(), they are not actually used anymorekernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re · line 568
Quoted 11 lines
kernel/trace/trace_syscalls.c +2
-1int syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs));
sizeof(struct pt_regs)== 168 on x86-64, and on arm64 it's a whopping 336 bytes, so these memset(0) calls are not free for sure.But we don't need to do this unnecessary work all the time.
I initially was going to suggest to use
get_bpf_raw_tp_regs()fromkernel/trace/bpf_trace.cto get a temporarypt_regsthat was already memset(0) and used to initialize these minimal "fake regs".But, it turns out we don't need to do even that. Note
perf_trace_buf_alloc(), it hasstruct pt_regs **second argument, and if you pass a valid pointer there, it will return "fake regs" struct to be used. We already use that functionality in perf_trace_##call ininclude/trace/perf.h(i.e., non-syscall tracepoints), so this seems to be a perfect fit.Quoted 8 lines
kernel/trace/trace_syscalls.c +2
-1+perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i]; --2.43.5
Quoted 77 lines · Yonghong Song
Quoted 67 lines · Andrii Nakryiko
Quoted 66 lines · Yonghong Song
Quoted 64 lines · Andrii Nakryiko
Quoted 53 lines · Andrii Nakryiko
Quoted 43 lines · Yonghong Song
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read {print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===Further analysis shows that
pt_regsused for bpf syscall tracepoint tracing is from the one constructed duringuser->kerneltransition. The call stack looks likeperf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7eThe ip address stored in
pt_regsis from user space hence no kernel stack is printed.To fix the issue, we need to use kernel address from
pt_regs. In kernel repo, there are already a few cases like this. For example, inkernel/trace/bpf_trace.c, severalperf_fetch_caller_regs(fake_regs_ptr)instances are used to supply ip address or use ip address to construct call stack.The patch follows the above example by using a fake
pt_regs. Thept_regsis stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to aperf_fetch_caller_regs()use case inkernel/trace/trace_event_perf.cwith functionperf_ftrace_function_call()where a localpt_regsis used.With this patch, for the above bpftrace script, I got the following output
===Kernel Stack syscall_trace_enter+407 syscall_trace_enter+407 do_syscall_64+74 entry_SYSCALL_64_after_hwframe+75Quoted 9 lines
===
Reported-by Salvatore Benedetto <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Quoted 4 lines
Note, we need to solve the same for
perf_call_bpf_exit().pw-bot: cr
BTW, we lived with this bug for years, so I suggest basing your fix on top of bpf-next/master, no bpf/master, which will give people a bit of time to validate that the fix works as expected and doesn't produce any undesirable side effects, before this makes it into the final Linux release.
Quoted 4 lines
Quoted 1 line
Yes, I did. See I indeed use 'bpf-next' in subject above.
Huh, strange, I actually tried to apply your patch to bpf-next/master and it didn't apply cleanly. It did apply to bpf/master, though, which is why I assumed you based it off of bpf/master.
Interesting. The following is my git history:
Quoted 8 lines
7b71206057440d9559ecb9cd02d891f46927b272(HEAD ->trace_syscall) bpf: Use fakept_regswhen doing bpf syscall tracepoint tracing41d0c4677feee1ea063e0f2c2af72dc953b1f1cc(origin/master, origin/HEAD, master) libbpf: Fix some typos in comments72d8508ecd3b081dba03ec00930c6b07c1ad55d3MAINTAINERS: BPF ARC JIT: Update my e-mail addressbee109b7b3e50739b88252a219fa07ecd78ad628bpf: Fix error message on kfunc arg type mismatch
...Not sure what is going on ...
Doesn't matter, maybe my local bpf-next/master branch was screwed up. Just send a v2 when you are ready and I'll try again :)
Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
Quoted 7 lines
kernel/trace/trace_syscalls.c +0
-0@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *relet's also drop struct
pt_regs* argument into perf_call_bpf_{enter,exit}(), they are not actually used anymorekernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re · line 568
Quoted 11 lines
kernel/trace/trace_syscalls.c +2
-1int syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs));
sizeof(struct pt_regs)== 168 on x86-64, and on arm64 it's a whopping 336 bytes, so these memset(0) calls are not free for sure.But we don't need to do this unnecessary work all the time.
I initially was going to suggest to use
get_bpf_raw_tp_regs()fromkernel/trace/bpf_trace.cto get a temporarypt_regsthat was already memset(0) and used to initialize these minimal "fake regs".But, it turns out we don't need to do even that. Note
perf_trace_buf_alloc(), it hasstruct pt_regs **second argument, and if you pass a valid pointer there, it will return "fake regs" struct to be used. We already use that functionality in perf_trace_##call ininclude/trace/perf.h(i.e., non-syscall tracepoints), so this seems to be a perfect fit.Quoted 8 lines
kernel/trace/trace_syscalls.c +2
-1+perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i]; --2.43.5
Yonghong Song
reviewerQuoted 46 lines · Andrii Nakryiko
Quoted 45 lines · Yonghong Song
Salvatore Benedetto reported an issue that when doing syscall tracepoint
tracing the kernel stack is empty. For example, using the following
command line
bpftrace -e 'tracepoint:syscalls:sys_enter_read {print("Kernel Stack\n"); print(kstack()); }'
the output will be
===
Kernel Stack
===Further analysis shows that
pt_regsused for bpf syscall tracepoint tracing is from the one constructed duringuser->kerneltransition. The call stack looks likeperf_syscall_enter+0x88/0x7c0 trace_sys_enter+0x41/0x80 syscall_trace_enter+0x100/0x160 do_syscall_64+0x38/0xf0 entry_SYSCALL_64_after_hwframe+0x76/0x7eThe ip address stored in
pt_regsis from user space hence no kernel stack is printed.To fix the issue, we need to use kernel address from
pt_regs. In kernel repo, there are already a few cases like this. For example, inkernel/trace/bpf_trace.c, severalperf_fetch_caller_regs(fake_regs_ptr)instances are used to supply ip address or use ip address to construct call stack.The patch follows the above example by using a fake
pt_regs. Thept_regsis stored in local stack since the syscall tracepoint tracing is in process context and there are no possibility that different concurrent syscall tracepoint tracing could mess up with each other. This is similar to aperf_fetch_caller_regs()use case inkernel/trace/trace_event_perf.cwith functionperf_ftrace_function_call()where a localpt_regsis used.With this patch, for the above bpftrace script, I got the following output
===
Kernel Stacksyscall_trace_enter+407 syscall_trace_enter+407 do_syscall_64+74 entry_SYSCALL_64_after_hwframe+75===
Quoted 8 lines
Quoted 7 lines
Reported-by Salvatore Benedetto <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- kernel/trace/trace_syscalls.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Note, we need to solve the same for
perf_call_bpf_exit().
Sorry, missed this one! Will add in the next revision.
Quoted 10 lines
pw-bot: cr
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re
Quoted 5 lines
kernel/trace/trace_syscalls.c +0
-0@@ -559,12 +559,15 @@ static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *relet's also drop struct
pt_regs* argument into perf_call_bpf_{enter,exit}(), they are not actually used anymore
Ack.
Quoted 7 lines
Quoted 7 lines
kernel/trace/trace_syscalls.c +1
-0int syscall_nr;unsigned long args[SYSCALL_DEFINE_MAXARGS ]; }__aligned(8) param; +struct pt_regs fake_regs;int i;
Quoted 7 lines
kernel/trace/trace_syscalls.c static int perf_call_bpf_enter(struct trace_event_call *call, struct pt_regs *re · line 568
Quoted 5 lines
kernel/trace/trace_syscalls.c +1
-1BUILD_BUG_ON (sizeof (param.ent) <sizeof (void *));/* bpf prog requires 'regs' to be the first member in the ctx (a.k.a. ¶m) */ - *(struct pt_regs **)¶m = regs; +memset (&fake_regs,0 ,sizeof (fake_regs));
sizeof(struct pt_regs)== 168 on x86-64, and on arm64 it's a whopping 336 bytes, so these memset(0) calls are not free for sure.
I calculated size on x86-64 and feels it might be acceptable. But not aware that arm64 has much larger size like 336. Indeed 336 bytes on the stack is quite large.
Quoted 7 lines
But we don't need to do this unnecessary work all the time.
I initially was going to suggest to use
get_bpf_raw_tp_regs()fromkernel/trace/bpf_trace.cto get a temporarypt_regsthat was already memset(0) and used to initialize these minimal "fake regs".
Quoted 6 lines
But, it turns out we don't need to do even that. Note
perf_trace_buf_alloc(), it hasstruct pt_regs **second argument, and if you pass a valid pointer there, it will return "fake regs" struct to be used. We already use that functionality in perf_trace_##call ininclude/trace/perf.h(i.e., non-syscall tracepoints), so this seems to be a perfect fit.
I double checked and perf_call_bpf_enter() call is in atomic process context (preempt disabled), so we are safe to use perf_trace_buf_alloc() which uses per-cpu variables.
Quoted 9 lines
Quoted 9 lines
kernel/trace/trace_syscalls.c +2
-1+perf_fetch_caller_regs (&fake_regs); + *(struct pt_regs **)¶m = &fake_regs; param.syscall_nr = rec->nr;for (i =0 ; i < sys_data->nb_args; i++) param.args[i] = rec->args[i]; --2.43.5