Ideal way to read FUNC_PROTO in raw tp?
Hi,
I am building a tracing program around workqueue. But I encountered following problem when I try to record a function pointer value from trace_workqueue_execute_end on net-next kernel:
...
libbpf: prog 'workqueue_end': BPF program load failed: Permission denied
libbpf: prog 'workqueue_end': -- BEGIN PROG LOAD LOG --
reg type unsupported for arg#0 function workqueue_end#5
0: R1=ctx(off=0,imm=0) R10=fp0
; int BPF_PROG(workqueue_end, struct work_struct *w, work_func_t f) 0: (79) r3 = *(u64 *)(r1 +8)
func 'workqueue_execute_end' arg1 type FUNC_PROTO is not a struct invalid bpf_context access off=8 size=8
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
-- END PROG LOAD LOG --
libbpf: prog 'workqueue_end': failed to load: -13
libbpf: failed to load object 'configs/test.bpf.o'
Error: failed to load object file
Warning: bpftool is now running in libbpf strict mode and has more stringent requirements about BPF programs.
If it used to work for this object file but now doesn't, see --legacy option for more details.
...
A simple reproducer for me is like:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
SEC("tp_btf/workqueue_execute_end")
int BPF_PROG(workqueue_end, struct work_struct *w, work_func_t f) {
u64 addr = (u64) f;
bpf_printk("f is %lu\n", addr);
return 0;
}
char LICENSE[] SEC("license") = "GPL";
I would like to use the function address to decode the kernel symbol and track execution of these functions. Replacing raw tp to regular tp solves the problem, but I am wondering if there is any go-to approach to read the pointer value in a raw tp? Doesn't seem to find one in selftests/samples. If not, does it make sense if we allow it in the verifier for tracing programs like the attached patch?
Yan
---
kernel/bpf/btf.c +5 -0
@@ -6308,6 +6308,11 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,__btf_name_by_offset (btf, t->name_off), btf_type_str(t));return false; + }else if (prog->type ==BPF_PROG_TYPE_TRACING || prog->type ==BPF_PROG_TYPE_RAW_TRACEPOINT ) { +/* allow reading function pointer value from a tracing program */ +const struct btf_type *pointed =btf_type_by_id (btf, t->type); +if (btf_type_is_func_proto (pointed)) +return true; }/* check for PTR_TO_RDONLY_BUF_OR_NULL or PTR_TO_RDWR_BUF_OR_NULL */
Quoted 60 lines · Yan Zhai
Hi,
I am building a tracing program around workqueue. But I encountered
following problem when I try to record a function pointer value fromtrace_workqueue_execute_endon net-next kernel:...
libbpf: prog 'workqueue_end': BPF program load failed: Permission denied
libbpf: prog 'workqueue_end': -- BEGIN PROG LOAD LOG --
reg type unsupported for arg#0 function workqueue_end#5
0: R1=ctx(off=0,imm=0) R10=fp0
; intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f)0: (79) r3 = *(u64 *)(r1 +8)
func 'workqueue_execute_end' arg1 typeFUNC_PROTOis not a struct invalidbpf_contextaccess off=8 size=8
processed 1 insns (limit 1000000)max_states_per_insn0total_states0peak_states0mark_read0
-- END PROG LOAD LOG --
libbpf: prog 'workqueue_end': failed to load: -13
libbpf: failed to load object 'configs/test.bpf.o'
Error: failed to load object file
Warning: bpftool is now running in libbpf strict mode and has more stringent requirements about BPF programs.
If it used to work for this object file but now doesn't, see --legacy option for more details.
...A simple reproducer for me is like:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
SEC("tp_btf/workqueue_execute_end")
intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f){ u64 addr = (u64) f; bpf_printk("f is %lu\n", addr); return 0; }char LICENSE[]
SEC("license")= "GPL";I would like to use the function address to decode the kernel symbol and track execution of these functions. Replacing raw tp to regular tp solves the problem, but I am wondering if there is any go-to approach to read the pointer value in a raw tp? Doesn't seem to find one in selftests/samples. If not, does it make sense if we allow it in the verifier for tracing programs like the attached patch?
Yan
---
kernel/bpf/btf.c +0
-0
kernel/bpf/btf.c bool btf_ctx_access(int off, int size, enum bpf_access_type type, · line 6315
Quoted 8 lines
kernel/bpf/btf.c +5
-0@@ -6308,6 +6308,11 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,__btf_name_by_offset (btf, t->name_off), btf_type_str(t));return false; + }else if (prog->type ==BPF_PROG_TYPE_TRACING || prog->type ==BPF_PROG_TYPE_RAW_TRACEPOINT ) { +/* allow reading function pointer value from a tracing program */ +const struct btf_type *pointed =btf_type_by_id (btf, t->type); +if (btf_type_is_func_proto (pointed)) +return true;
The reason it wasn't supported in tp_btf is to avoid potential backward compat issues when the verifier will start to recognize it as a proper pointer to a function.
Since I didn't know what that support would look like I left it as an error for now.
'return true' (which would mean scalar type to the verifier) is a bit dangerous and I feel it's better to think it through right away.
Eventually it probably will be a new reg_type PTR_TO_KERN_FUNC or something. And it won't be modifiable.
Like arithmetic won't be allowed.
Passing into other helpers (like prinkt in your example) is fine. But if we do 'return true -> scalar' today then bpf prog
will be able to do math on it, including conditional jmps,
which will be disallowed once it becomes PTR_TO_KERN_FUNC.
And that would become a backward compat issue.
So pls think it through from that angle.
Hi Alexei,
Thanks for getting back.
Quoted 80 lines · Alexei Starovoitov
Quoted 60 lines · Yan Zhai
Hi,
I am building a tracing program around workqueue. But I encountered
following problem when I try to record a function pointer value fromtrace_workqueue_execute_endon net-next kernel:...
libbpf: prog 'workqueue_end': BPF program load failed: Permission denied
libbpf: prog 'workqueue_end': -- BEGIN PROG LOAD LOG --
reg type unsupported for arg#0 function workqueue_end#5
0: R1=ctx(off=0,imm=0) R10=fp0
; intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f)0: (79) r3 = *(u64 *)(r1 +8)
func 'workqueue_execute_end' arg1 typeFUNC_PROTOis not a struct invalidbpf_contextaccess off=8 size=8
processed 1 insns (limit 1000000)max_states_per_insn0total_states0peak_states0mark_read0
-- END PROG LOAD LOG --
libbpf: prog 'workqueue_end': failed to load: -13
libbpf: failed to load object 'configs/test.bpf.o'
Error: failed to load object file
Warning: bpftool is now running in libbpf strict mode and has more stringent requirements about BPF programs.
If it used to work for this object file but now doesn't, see --legacy option for more details.
...A simple reproducer for me is like:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
SEC("tp_btf/workqueue_execute_end")
intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f){ u64 addr = (u64) f; bpf_printk("f is %lu\n", addr); return 0; }char LICENSE[]
SEC("license")= "GPL";I would like to use the function address to decode the kernel symbol and track execution of these functions. Replacing raw tp to regular tp solves the problem, but I am wondering if there is any go-to approach to read the pointer value in a raw tp? Doesn't seem to find one in selftests/samples. If not, does it make sense if we allow it in the verifier for tracing programs like the attached patch?
Yan
---
kernel/bpf/btf.c +0
-0kernel/bpf/btf.c bool btf_ctx_access(int off, int size, enum bpf_access_type type, · line 6315
Quoted 8 lines
kernel/bpf/btf.c +5
-0@@ -6308,6 +6308,11 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,__btf_name_by_offset (btf, t->name_off), btf_type_str(t));return false; + }else if (prog->type ==BPF_PROG_TYPE_TRACING || prog->type ==BPF_PROG_TYPE_RAW_TRACEPOINT ) { +/* allow reading function pointer value from a tracing program */ +const struct btf_type *pointed =btf_type_by_id (btf, t->type); +if (btf_type_is_func_proto (pointed)) +return true;The reason it wasn't supported in
tp_btfis to avoid potential backward compat issues when the verifier will start to recognize it as a proper pointer to a function.
Since I didn't know what that support would look like I left it as an error for now.'return true' (which would mean scalar type to the verifier) is a bit dangerous and I feel it's better to think it through right away.
Quoted 9 lines
Eventually it probably will be a new
reg_typePTR_TO_KERN_FUNCor something. And it won't be modifiable.
Like arithmetic won't be allowed.
Passing into other helpers (like prinkt in your example) is fine. But if we do 'return true -> scalar' today then bpf prog
will be able to do math on it, including conditional jmps,
which will be disallowed once it becomesPTR_TO_KERN_FUNC.
And that would become a backward compat issue.
So pls think it through from that angle.
Having a new reg_type makes sense to me, but IMHO it could still cause a backward compatibility issue. In my example, the regular version TP has the func pointer assigned as a void * arg, which I can do arithmetics already with. So if anyone relies on this fact, then it may break if we introduce the PTR_TO_KERN_FUNC and replace void * with a proper kernel function pointer. But if we leave the void * in place, then TP/Raw-TP have different capabilities on the same argument just like today. WDYT?
Yan
Quoted 34 lines · Yan Zhai
Hi,
I am building a tracing program around workqueue. But I encountered
following problem when I try to record a function pointer value fromtrace_workqueue_execute_endon net-next kernel:...
libbpf: prog 'workqueue_end': BPF program load failed: Permission denied
libbpf: prog 'workqueue_end': -- BEGIN PROG LOAD LOG --
reg type unsupported for arg#0 function workqueue_end#5
0: R1=ctx(off=0,imm=0) R10=fp0
; intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f)0: (79) r3 = *(u64 *)(r1 +8)
func 'workqueue_execute_end' arg1 typeFUNC_PROTOis not a struct invalidbpf_contextaccess off=8 size=8
processed 1 insns (limit 1000000)max_states_per_insn0total_states0peak_states0mark_read0
-- END PROG LOAD LOG --
libbpf: prog 'workqueue_end': failed to load: -13
libbpf: failed to load object 'configs/test.bpf.o'
Error: failed to load object file
Warning: bpftool is now running in libbpf strict mode and has more stringent requirements about BPF programs.
If it used to work for this object file but now doesn't, see --legacy option for more details.
...A simple reproducer for me is like:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
Quoted 4 lines
SEC("tp_btf/workqueue_execute_end")
intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f){ u64 addr = (u64) f;
you can work around with:
bpf_probe_read_kernel(&addr, sizeof(addr), &ctx[1]); /* ctx[1] is the second argument */
Not great, but will get you past this easily.
Quoted 34 lines
bpf_printk("f is %lu\n", addr); return 0; }char LICENSE[]
SEC("license")= "GPL";I would like to use the function address to decode the kernel symbol and track execution of these functions. Replacing raw tp to regular tp solves the problem, but I am wondering if there is any go-to approach to read the pointer value in a raw tp? Doesn't seem to find one in selftests/samples. If not, does it make sense if we allow it in the verifier for tracing programs like the attached patch?
Yan
---
kernel/bpf/btf.c +5
-0@@ -6308,6 +6308,11 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,__btf_name_by_offset (btf, t->name_off), btf_type_str(t));return false; + }else if (prog->type ==BPF_PROG_TYPE_TRACING || prog->type ==BPF_PROG_TYPE_RAW_TRACEPOINT ) { +/* allow reading function pointer value from a tracing program */ +const struct btf_type *pointed =btf_type_by_id (btf, t->type); +if (btf_type_is_func_proto (pointed)) +return true; }/* check for PTR_TO_RDONLY_BUF_OR_NULL or PTR_TO_RDWR_BUF_OR_NULL */
Quoted 41 lines · Andrii Nakryiko
Quoted 34 lines · Yan Zhai
Hi,
I am building a tracing program around workqueue. But I encountered
following problem when I try to record a function pointer value fromtrace_workqueue_execute_endon net-next kernel:...
libbpf: prog 'workqueue_end': BPF program load failed: Permission denied
libbpf: prog 'workqueue_end': -- BEGIN PROG LOAD LOG --
reg type unsupported for arg#0 function workqueue_end#5
0: R1=ctx(off=0,imm=0) R10=fp0
; intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f)0: (79) r3 = *(u64 *)(r1 +8)
func 'workqueue_execute_end' arg1 typeFUNC_PROTOis not a struct invalidbpf_contextaccess off=8 size=8
processed 1 insns (limit 1000000)max_states_per_insn0total_states0peak_states0mark_read0
-- END PROG LOAD LOG --
libbpf: prog 'workqueue_end': failed to load: -13
libbpf: failed to load object 'configs/test.bpf.o'
Error: failed to load object file
Warning: bpftool is now running in libbpf strict mode and has more stringent requirements about BPF programs.
If it used to work for this object file but now doesn't, see --legacy option for more details.
...A simple reproducer for me is like:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>Quoted 4 lines
SEC("tp_btf/workqueue_execute_end")
intBPF_PROG(workqueue_end, struct work_struct *w, work_func_t f){ u64 addr = (u64) f;
Quoted 7 lines
you can work around with:
bpf_probe_read_kernel(&addr, sizeof(addr), &ctx[1]); /* ctx[1] is the second argument */Not great, but will get you past this easily.
Yes that works, too. Currently I am using the regular tp to workaround. That said, I am more interested in the verifier side of this. I am happy to add it to my queue if no one has signed up to support func_proto type in ctx. Would maintainers welcome RFC patches for this or is it something that you prefer to leave for now?
best
Yan