perf trace: failed to load -E2BIG
Namhyung Kim
Tested-by Arnaldo Carvalho de MeloHello,
I'm seeing the following error in the current perf-tools[-next]. Do you see you the same?
$ sudo ./perf trace -a sleep 1 libbpf: prog 'sys_enter': BPF program load failed: -E2BIG libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG -- 0: R1=ctx() R10=fp0 ; int sys_enter(struct syscall_enter_args *args) @ augmented_raw_syscalls.bpf.c:527 0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx() ; return bpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423 1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar() 2: (63) *(u32 *)(r10 -4) = r0 ; R0_w=scalar() R10=fp0 fp-8=mmmm???? 3: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 ; @ augmented_raw_syscalls.bpf.c:0 4: (07) r2 += -4 ; R2_w=fp-4 ; return bpf_map_lookup_elem(pids, &pid) != NULL; @ augmented_raw_syscalls.bpf.c:428 ... 140: (79) r7 = *(u64 *)(r10 -48) ; R7_w=ctx() R10=fp0 fp-48=ctx() 141: (79) r0 = *(u64 *)(r10 -56) BPF program is too large. Processed 1000001 insn processed 1000001 insns (limit 1000000) max_states_per_insn 28 total_states 37670 peak_states 330 mark_read 16 -- END PROG LOAD LOG -- libbpf: prog 'sys_enter': failed to load: -E2BIG libbpf: failed to load object 'augmented_raw_syscalls_bpf' libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -E2BIG libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) Not enough memory to run!
The git bisect found this but I couldn't find anything suspicious.
395d38419f1853decab84acc16176b3fa5c96690 is the first bad commit commit 395d38419f1853decab84acc16176b3fa5c96690
Author: Howard Chu <[email protected]> Date: Thu Oct 10 19:14:02 2024 -0700
perf trace augmented_raw_syscalls: Add more checks to pass the verifier
Add some more checks to pass the verifier in more kernels.
.../perf/util/bpf_skel/augmented_raw_syscalls.bpf.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
bisect found first bad commit
Thanks,
Namhyung
A message that is not in the archive (sent off-list or not yet received): CA+JHD90D86YC=Kn3P_B1xLamxKS9+_zOxmKxXMWyTDQGwGnNsQ@mail.gmail.com
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Namhyung Kim
Hello,
I'm seeing the following error in the current perf-tools[-next]. Do you see you the same?
Nope, what's your environment? Kernel version, clang, etc.
It's a debian derivative and the kernel is v6.9.10 and clang 16.0.6. But I cannot reproduce it anymore after the bisect. :( Maybe it's a transient kernel bug.
Quoted 2 lines
I think we should and that I'm addition to the verifier error log.
I think the below is the verifier log and I cut it out as it repeated. It seems the verifier fell into a loop for some reason and thought the program had too many instructions.
Thanks,
Namhyung
Quoted 66 lines
- Arnaldo
Quoted 63 lines
$ sudo ./perf trace -a sleep 1 libbpf: prog 'sys_enter': BPF program load failed: -E2BIG libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG -- 0: R1=ctx() R10=fp0 ; int sys_enter(struct syscall_enter_args *args) @ augmented_raw_syscalls.bpf.c:527 0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx() ; return bpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423 1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar() 2: (63) *(u32 *)(r10 -4) = r0 ; R0_w=scalar() R10=fp0 fp-8=mmmm???? 3: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 ; @ augmented_raw_syscalls.bpf.c:0 4: (07) r2 += -4 ; R2_w=fp-4 ; return bpf_map_lookup_elem(pids, &pid) != NULL; @ augmented_raw_syscalls.bpf.c:428 ... 140: (79) r7 = *(u64 *)(r10 -48) ; R7_w=ctx() R10=fp0 fp-48=ctx() 141: (79) r0 = *(u64 *)(r10 -56) BPF program is too large. Processed 1000001 insn processed 1000001 insns (limit 1000000) max_states_per_insn 28 total_states 37670 peak_states 330 mark_read 16 -- END PROG LOAD LOG -- libbpf: prog 'sys_enter': failed to load: -E2BIG libbpf: failed to load object 'augmented_raw_syscalls_bpf' libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -E2BIG libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) Not enough memory to run!The git bisect found this but I couldn't find anything suspicious.
395d38419f1853decab84acc16176b3fa5c96690is the first bad commit commit395d38419f1853decab84acc16176b3fa5c96690Author: Howard Chu <[email protected]> Date: Thu Oct 10 19:14:02 2024 -0700perf trace
augmented_raw_syscalls: Add more checks to pass the verifierAdd some more checks to pass the verifier in more kernels.
Signed-off-by Howard Chu <[email protected]>Cc Adrian Hunter <[email protected]>Cc Alan Maguire <[email protected]>Cc Alexander Shishkin <[email protected]>Cc Ian Rogers <[email protected]>Cc Ingo Molnar <[email protected]>Cc Jiri Olsa <[email protected]>Cc Kan Liang <[email protected]>Cc Mark Rutland <[email protected]>Cc Namhyung Kim <[email protected]>Cc Peter Zijlstra <[email protected]>[ Reduced the patch removing things that can be done later ]Signed-off-by Arnaldo Carvalho de Melo <[email protected]>.../perf/util/bpf_skel/augmented_raw_syscalls.bpf.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)bisect found first bad commit
Thanks,
Namhyung
We hit the similar issue on our internal version since v6.11.0 and start with commit 395d38419f1853decab84acc16176b3fa5c96690
and can easy to reproduce by 'perf trace ls' and get such error log:
------------
# perf trace ls libbpf: prog 'sys_enter': BPF program load failed: Argument list too long libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG -- 0: R1=ctx() R10=fp0 ; int sys_enter(struct syscall_enter_args *args) @ augmented_raw_syscalls.bpf.c:527 0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx() ; return bpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423 1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar() 2: (63) *(u32 *)(r10 -4) = r0 ; R0_w=scalar() R10=fp0 fp-8=mmmm???? 3: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 4: (07) r2 += -4 ; R2_w=fp-4 ; return bpf_map_lookup_elem(pids, &pid) != NULL; @ augmented_raw_syscalls.bpf.c:428 5: (18) r1 = 0xff45c8d9a56b1c00 ; R1_w=map_ptr(map=pids_filtered,ks=4,vs=1) 7: (85) call bpf_map_lookup_elem#1 ; R0=map_value_or_null(id=1,map=pids_filtered,ks=4,vs=1) 8: (bf) r1 = r0 ; R0=map_value_or_null(id=1,map=pids_filtered,ks=4,vs=1) R1_w=map_value_or_null(id=1,map=pids_filtered,ks=4,vs=1) 9: (b7) r0 = 0 ; R0_w=0 ... ... 111: (2d) if r1 > r9 goto pc-22 90: R0=map_value(map=beauty_payload_,ks=4,vs=24688) R1_w=-6 R2=1 R3=scalar(umin=1) R6=map_value(map=beauty_map_ente,ks=4,vs=24,off=20) R7=map_value(map=augmented_args_,ks=4,vs=8272,off=56) R8=6 R9=scalar(smin=0xffffffff80000000,smax=smax32=0,umin=2,umax=0xfffffffffffffff9) R10=fp0 fp-8=mmmmmmmm fp-16=map_value(map=beauty_payload_,ks=4,vs=24688,off=88) fp-24=map_value(map=augmented_args_,ks=4,vs=8272,off=16) fp-32=map_value(map=augmented_args_,ks=4,vs=8272) fp-40=16 fp-48=ctx() fp-56=map_value(map=beauty_payload_,ks=4,vs=24688) fp-64=1 fp-72=map_value(map=beauty_payload_,ks=4,vs=24688,off=80) fp-80=map_value(map=beauty_map_ente,ks=4,vs=24) 90: (07) r7 += 8 BPF program is too large. Processed 1000001 insn processed 1000001 insns (limit 1000000) max_states_per_insn 28 total_states 40283 peak_states 269 mark_read 15 -- END PROG LOAD LOG -- libbpf: prog 'sys_enter': failed to load: -7 libbpf: failed to load object 'augmented_raw_syscalls_bpf' libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -7 libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) ... ... ................ Like some bpf error is printed before the trace is printed. Hope this can help you to debug.
- Qiao
Quoted 96 lines · Namhyung Kim
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Namhyung Kim
Hello,
I'm seeing the following error in the current perf-tools[-next]. Do you see you the same?
Nope, what's your environment? Kernel version, clang, etc.
It's a debian derivative and the kernel is v6.9.10 and clang 16.0.6. But I cannot reproduce it anymore after the bisect. :( Maybe it's a transient kernel bug.
Quoted 2 lines
I think we should and that I'm addition to the verifier error log.
I think the below is the verifier log and I cut it out as it repeated. It seems the verifier fell into a loop for some reason and thought the program had too many instructions.
Thanks,
NamhyungQuoted 66 lines
- Arnaldo
Quoted 63 lines
$ sudo ./perf trace -a sleep 1 libbpf: prog 'sys_enter': BPF program load failed: -E2BIG libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG -- 0: R1=ctx() R10=fp0 ; int sys_enter(struct syscall_enter_args *args) @ augmented_raw_syscalls.bpf.c:527 0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx() ; return bpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423 1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar() 2: (63) *(u32 *)(r10 -4) = r0 ; R0_w=scalar() R10=fp0 fp-8=mmmm???? 3: (bf) r2 = r10 ; R2_w=fp0 R10=fp0 ; @ augmented_raw_syscalls.bpf.c:0 4: (07) r2 += -4 ; R2_w=fp-4 ; return bpf_map_lookup_elem(pids, &pid) != NULL; @ augmented_raw_syscalls.bpf.c:428 ... 140: (79) r7 = *(u64 *)(r10 -48) ; R7_w=ctx() R10=fp0 fp-48=ctx() 141: (79) r0 = *(u64 *)(r10 -56) BPF program is too large. Processed 1000001 insn processed 1000001 insns (limit 1000000) max_states_per_insn 28 total_states 37670 peak_states 330 mark_read 16 -- END PROG LOAD LOG -- libbpf: prog 'sys_enter': failed to load: -E2BIG libbpf: failed to load object 'augmented_raw_syscalls_bpf' libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -E2BIG libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?) Not enough memory to run!The git bisect found this but I couldn't find anything suspicious.
395d38419f1853decab84acc16176b3fa5c96690is the first bad commit commit395d38419f1853decab84acc16176b3fa5c96690Author: Howard Chu <[email protected]> Date: Thu Oct 10 19:14:02 2024 -0700perf trace
augmented_raw_syscalls: Add more checks to pass the verifierAdd some more checks to pass the verifier in more kernels.
Signed-off-by Howard Chu <[email protected]>Cc Adrian Hunter <[email protected]>Cc Alan Maguire <[email protected]>Cc Alexander Shishkin <[email protected]>Cc Ian Rogers <[email protected]>Cc Ingo Molnar <[email protected]>Cc Jiri Olsa <[email protected]>Cc Kan Liang <[email protected]>Cc Mark Rutland <[email protected]>Cc Namhyung Kim <[email protected]>Cc Peter Zijlstra <[email protected]>[ Reduced the patch removing things that can be done later ]Signed-off-by Arnaldo Carvalho de Melo <[email protected]>.../perf/util/bpf_skel/augmented_raw_syscalls.bpf.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)bisect found first bad commit
Thanks,
Namhyung
Hi Qiao, Namhyung, and Arnaldo,
Apologies. I observed the same issue and tested perf with trace BPF skel generated by clang-13 to clang-18, turns out BPF skelw generated by clang version <= clang-16 are not loadable, with clang-15 and -16 showing the same error as yours. Additionally, the BPF verifier is running longer than usual to process the instructions.
perf $ ./perf trace -e write --max-events=1
libbpf: prog 'sys_enter': BPF program load failed: Argument list too long
libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; int sys_enter(struct syscall_enter_args *args) @ augmented_raw_syscalls.bpf.c:527
0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx()
; return bpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423
...
140: (79) r7 = *(u64 *)(r10 -48) ; R7_w=ctx() R10=fp0 fp-48=ctx()
141: (79) r0 = *(u64 *)(r10 -56)
BPF program is too large. Processed 1000001 insn
processed 1000001 insns (limit 1000000) max_states_per_insn 28 total_states 37670 peak_states 330 mark_read 16
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -7
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -7
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!
For clang-14 however, I encountered an unbounded memory access:
perf $ ./perf trace -e write --max-events=1
libbpf: prog 'sys_enter': BPF program load failed: Permission denied
libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; int sys_enter(struct syscall_enter_args *args) @ augmented_raw_syscalls.bpf.c:527
0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx()
; return bpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423
1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar()
2: (63) *(u32 *)(r10 -4) = r0 ; R0_w=scalar() R10=fp0 fp-8=mmmm????
88: (79) r1 = *(u64 *)(r10 -16) ;
R1_w=map_value(map=beauty_payload_,ks=4,vs=24688,off=112) R10=fp0 fp-16=map_value(map=beauty_payload_,ks=4,vs=24688,off=112)
89: (bf) r2 = r9 ;
R2_w=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff)) R9=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))
90: (85) call bpf_probe_read_user#112
R2 unbounded memory access, use 'var &= const' or 'if (var < const)' processed 490 insns (limit 1000000) max_states_per_insn 2 total_states 23 peak_states 23 mark_read 15
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -13
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -13
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!
I thought I had tested them, but apparently I didn’t. My apologies again, and fixes are on the way.
Thanks,
Howard
Forgot to mention clang-13 gave unbounded memory access too:
ffffffff,var_off=(0x0; 0xffffffff)) R9=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff))
90: (85) call bpf_probe_read_user#112
R2 unbounded memory access, use 'var &= const' or 'if (var < const)' processed 490 insns (limit 1000000) max_states_per_insn 2 total_states 23 peak_states 23 mark_read 15
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -13
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -13
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!
Kernel:
perf $ uname -r
6.11.0-061100-generic
Thanks,
Howard
Quoted 79 lines · Howard Chu
Hi Qiao, Namhyung, and Arnaldo,
Apologies. I observed the same issue and tested perf with trace BPF skel generated by clang-13 to clang-18, turns out BPF skelw generated by clang version <= clang-16 are not loadable, with clang-15 and -16 showing the same error as yours. Additionally, the BPF verifier is running longer than usual to process the instructions.
perf $ ./perf trace -e write --max-events=1
libbpf: prog 'sys_enter': BPF program load failed: Argument list too long
libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; intsys_enter(struct syscall_enter_args *args)@ augmented_raw_syscalls.bpf.c:527
0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx()
; returnbpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423
...
140: (79) r7 = *(u64 *)(r10 -48) ; R7_w=ctx() R10=fp0 fp-48=ctx()
141: (79) r0 = *(u64 *)(r10 -56)
BPF program is too large. Processed 1000001 insn
processed 1000001 insns (limit 1000000)max_states_per_insn28total_states37670peak_states330mark_read16
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -7
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -7
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!For clang-14 however, I encountered an unbounded memory access:
perf $ ./perf trace -e write --max-events=1
libbpf: prog 'sys_enter': BPF program load failed: Permission denied
libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; intsys_enter(struct syscall_enter_args *args)@ augmented_raw_syscalls.bpf.c:527
0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx()
; returnbpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423
1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar()
2: (63) *(u32 *)(r10 -4) = r0 ; R0_w=scalar() R10=fp0 fp-8=mmmm????
88: (79) r1 = *(u64 *)(r10 -16) ;
R1_w=map_value(map=beauty_payload_,ks=4,vs=24688,off=112) R10=fp0 fp-16=map_value(map=beauty_payload_,ks=4,vs=24688,off=112)
89: (bf) r2 = r9 ;
R2_w=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0;0xffffffff)) R9=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0;0xffffffff))
90: (85) call bpf_probe_read_user#112
R2 unbounded memory access, use 'var &= const' or 'if (var < const)' processed 490 insns (limit 1000000)max_states_per_insn2total_states23peak_states23mark_read15
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -13
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -13
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!I thought I had tested them, but apparently I didn’t. My apologies again, and fixes are on the way.
Thanks,
Howard
Cc-ing bpf list.
Quoted 111 lines · Howard Chu
Forgot to mention clang-13 gave unbounded memory access too:
ffffffff,var_off=(0x0;
0xffffffff)) R9=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0;0xffffffff))
90: (85) call bpf_probe_read_user#112
R2 unbounded memory access, use 'var &= const' or 'if (var < const)' processed 490 insns (limit 1000000)max_states_per_insn2total_states23peak_states23mark_read15
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -13
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -13
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!Kernel:
perf $ uname -r
6.11.0-061100-genericThanks,
HowardQuoted 79 lines · Howard Chu
Hi Qiao, Namhyung, and Arnaldo,
Apologies. I observed the same issue and tested perf with trace BPF skel generated by clang-13 to clang-18, turns out BPF skelw generated by clang version <= clang-16 are not loadable, with clang-15 and -16 showing the same error as yours. Additionally, the BPF verifier is running longer than usual to process the instructions.
perf $ ./perf trace -e write --max-events=1
libbpf: prog 'sys_enter': BPF program load failed: Argument list too long
libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; intsys_enter(struct syscall_enter_args *args)@ augmented_raw_syscalls.bpf.c:527
0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx()
; returnbpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423
...
140: (79) r7 = *(u64 *)(r10 -48) ; R7_w=ctx() R10=fp0 fp-48=ctx()
141: (79) r0 = *(u64 *)(r10 -56)
BPF program is too large. Processed 1000001 insn
processed 1000001 insns (limit 1000000)max_states_per_insn28total_states37670peak_states330mark_read16
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -7
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -7
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!For clang-14 however, I encountered an unbounded memory access:
perf $ ./perf trace -e write --max-events=1
libbpf: prog 'sys_enter': BPF program load failed: Permission denied
libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; intsys_enter(struct syscall_enter_args *args)@ augmented_raw_syscalls.bpf.c:527
0: (bf) r7 = r1 ; R1=ctx() R7_w=ctx()
; returnbpf_get_current_pid_tgid(); @ augmented_raw_syscalls.bpf.c:423
1: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar()
2: (63) *(u32 *)(r10 -4) = r0 ; R0_w=scalar() R10=fp0 fp-8=mmmm????
88: (79) r1 = *(u64 *)(r10 -16) ;
R1_w=map_value(map=beauty_payload_,ks=4,vs=24688,off=112) R10=fp0 fp-16=map_value(map=beauty_payload_,ks=4,vs=24688,off=112)
89: (bf) r2 = r9 ;
R2_w=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0;0xffffffff)) R9=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0;0xffffffff))
90: (85) call bpf_probe_read_user#112
R2 unbounded memory access, use 'var &= const' or 'if (var < const)' processed 490 insns (limit 1000000)max_states_per_insn2total_states23peak_states23mark_read15
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -13
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -13
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!I thought I had tested them, but apparently I didn’t. My apologies again, and fixes are on the way.
Thanks,
Howard
Quoted 37 lines · Namhyung Kim
Cc-ing bpf list.
Quoted 34 lines · Howard Chu
Forgot to mention clang-13 gave unbounded memory access too:
ffffffff,var_off=(0x0;
0xffffffff)) R9=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0;0xffffffff))
90: (85) call bpf_probe_read_user#112
R2 unbounded memory access, use 'var &= const' or 'if (var < const)' processed 490 insns (limit 1000000)max_states_per_insn2total_states23peak_states23mark_read15
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -13
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -13
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!Kernel:
perf $ uname -r
6.11.0-061100-genericQuoted 5 lines · Howard Chu
Apologies. I observed the same issue and tested perf with trace BPF skel generated by clang-13 to clang-18, turns out BPF skelw generated by clang version <= clang-16 are not loadable, with clang-15 and -16 showing the same error as yours. Additionally, the BPF verifier is running longer than usual to process the instructions.
Some more datapoints, here it is working, below you'll see a 'perf trace' session tracing bpf syscalls (that use libbpf btf pretty printing routines, that has to be improved) emitted by a 'perf trace' session augmenting open* syscalls (i.e. using BPF to get the syscall pointer args, namely pathnames), and here is the environment:
⬢ [acme@toolbox perf-tools-next]$ rpm -q clang
clang-18.1.8-1.fc40.x86_64
root@number:~# uname -a Linux number 6.13.0-rc2 #1 SMP PREEMPT_DYNAMIC Mon Dec 9 12:33:35 -03 2024 x86_64 GNU/Linux root@number:~#
root@number:~# perf trace -e bpf --max-events=3 perf trace -e open* -a sleep 0.0001 |& head
0.000 ( 0.005 ms): :3117755/3117755 bpf(cmd: 36, uattr: (union bpf_attr){.batch = (struct){.in_batch = (__u64)42949672960,.out_batch = (__u64)18446744073709551448,.keys = (__u64)2,.values = (__u64)801012880,.elem_flags = (__u64)140154830285504,.flags = (__u64)11424,},(struct){.pathname = (__u64)42949672960,.bpf_fd = (__u32)4294967128,.file_flags = (__u32)4294967295,.path_fd = (__s32)2,},.raw_tracepoint = (struct){.name = (__u64)42949672960,.prog_fd = (__u32)4294967128,.cookie = (__u64)2,},(struct){.btf = (__u64)42949672960,.btf_log_buf = (__u64)18446744073709551448,.btf_size = (__u32)2,.btf_log_level = (__u32)801012880,},}, size: 8) = -1 EOPNOTSUPP (Operation not supported)
0.009 ( 0.042 ms): :3117755/3117755 bpf(cmd: PROG_LOAD, uattr: (union bpf_attr){(struct){.map_type = (__u32)1,.key_size = (__u32)2,.value_size = (__u32)2233669616,.max_entries = (__u32)32767,.map_flags = (__u32)7334203,},(struct){.map_fd = (__u32)1,.key = (__u64)140735427057648,(union){.value = (__u64)7334203,.next_key = (__u64)7334203,},},.batch = (struct){.in_batch = (__u64)8589934593,.out_batch = (__u64)140735427057648,.keys = (__u64)7334203,},(struct){.prog_type = (__u32)1,.insn_cnt = (__u32)2,.insns = (__u64)140735427057648,.license = (__u64)7334203,},(struct){.pathname = (__u64)8589934593,.bpf_fd = (__u32)2233669616,.file_flags = (__u32)32767,.path_fd = (__s32)7334203,},(struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_bpf_fd = (__u32)2,.attach_type = (__u32)2233669616,.attach_flags = (__u32)32767,.replace_bpf_fd = (__u32)7334203,},.test = (struct){.prog_fd = (__u32)1,.retval = (__u32)2,.data_size_in = (__u32)2233669616,.data_size_out = (__u32)32767,.data_in = (__u64)7334203,},(struct){(union){.start_id = (__u32)1,.prog_id = (__u32)1,.map_id = (__u32)1,.btf_id = (__u32)1,.link_id = (__u32)1,},.next_id = (__u32)2,.open_flags = (__u32)2233669616,},.info = (struct){.bpf_fd = (__u32)1,.info_len = (__u32)2,.info = (__u64)140735427057648,},.query = (struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_type = (__u32)2,.query_flags = (__u32)2233669616,.attach_flags = (__u32)32767,.prog_ids = (__u64)7334203,},.raw_tracepoint = (struct){.name = (__u64)8589934593,.prog_fd = (__u32)2233669616,.cookie = (__u64)7334203,},(struct){.btf = (__u64)8589934593,.btf_log_buf = (__u64)140735427057648,.btf_size = (__u32)7334203,},.task_fd_query = (struct){.pid = (__u32)1,.fd = (__u32)2,.flags = (__u32)2233669616,.buf_len = (__u32)32767,.buf = (__u64)7334203,},.link_create = (struct){(union){.prog_fd = (__u32)1,.map_fd = (__u32)1,},(union){.target_fd = (__u32)2,.target_ifindex = (__u32)2,},.attach_type = (__u32)2233669616,.flags = (__u32)32767,(union){.target_btf_id = (__u32)7334203,(struct){.iter_info = (__u64)7334203) = 10
0.054 ( 0.010 ms): :3117755/3117755 bpf(cmd: PROG_LOAD, uattr: (union bpf_attr){(struct){.map_type = (__u32)1,.key_size = (__u32)2,.value_size = (__u32)2233670144,.max_entries = (__u32)32767,.map_flags = (__u32)7314455,},(struct){.map_fd = (__u32)1,.key = (__u64)140735427058176,(union){.value = (__u64)7314455,.next_key = (__u64)7314455,},},.batch = (struct){.in_batch = (__u64)8589934593,.out_batch = (__u64)140735427058176,.keys = (__u64)7314455,},(struct){.prog_type = (__u32)1,.insn_cnt = (__u32)2,.insns = (__u64)140735427058176,.license = (__u64)7314455,},(struct){.pathname = (__u64)8589934593,.bpf_fd = (__u32)2233670144,.file_flags = (__u32)32767,.path_fd = (__s32)7314455,},(struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_bpf_fd = (__u32)2,.attach_type = (__u32)2233670144,.attach_flags = (__u32)32767,.replace_bpf_fd = (__u32)7314455,},.test = (struct){.prog_fd = (__u32)1,.retval = (__u32)2,.data_size_in = (__u32)2233670144,.data_size_out = (__u32)32767,.data_in = (__u64)7314455,},(struct){(union){.start_id = (__u32)1,.prog_id = (__u32)1,.map_id = (__u32)1,.btf_id = (__u32)1,.link_id = (__u32)1,},.next_id = (__u32)2,.open_flags = (__u32)2233670144,},.info = (struct){.bpf_fd = (__u32)1,.info_len = (__u32)2,.info = (__u64)140735427058176,},.query = (struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_type = (__u32)2,.query_flags = (__u32)2233670144,.attach_flags = (__u32)32767,.prog_ids = (__u64)7314455,},.raw_tracepoint = (struct){.name = (__u64)8589934593,.prog_fd = (__u32)2233670144,.cookie = (__u64)7314455,},(struct){.btf = (__u64)8589934593,.btf_log_buf = (__u64)140735427058176,.btf_size = (__u32)7314455,},.task_fd_query = (struct){.pid = (__u32)1,.fd = (__u32)2,.flags = (__u32)2233670144,.buf_len = (__u32)32767,.buf = (__u64)7314455,},.link_create = (struct){(union){.prog_fd = (__u32)1,.map_fd = (__u32)1,},(union){.target_fd = (__u32)2,.target_ifindex = (__u32)2,},.attach_type = (__u32)2233670144,.flags = (__u32)32767,(union){.target_btf_id = (__u32)7314455,(struct){.iter_info = (__u64)7314455) = 10
0.000 ( 0.007 ms): sleep/3117756 openat(dfd: CWD, filename: "/etc/ld.so.cache", flags: RDONLY|CLOEXEC) = 3
0.022 ( 0.004 ms): sleep/3117756 openat(dfd: CWD, filename: "/lib64/libc.so.6", flags: RDONLY|CLOEXEC) = 3
0.201 ( 0.006 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/lib/locale/locale-archive", flags: RDONLY|CLOEXEC) = 3
0.247 ( 0.004 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/locale.alias", flags: RDONLY|CLOEXEC) = 3
0.283 ( 0.003 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo") = -1 ENOENT (No such file or directory)
0.287 ( 0.002 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo") = -1 ENOENT (No such file or directory)
0.290 ( 0.003 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo") = -1 ENOENT (No such file or directory)
root@number:~#root@number:~# perf trace -e connect
0.000 ( 0.020 ms): pool/3112 connect(fd: 7, uservaddr: { .family: LOCAL, path: /var/run/.heim }, addrlen: 110) = 0
665.621 ( 0.029 ms): Chrome_ChildIO/3102157 connect(fd: 26, uservaddr: { .family: INET6, port: 443, addr: 2001:4860:4860::900:0, scope_id: 5767169 }, addrlen: 28) = 0
4422.069 ( 0.024 ms): DNS Res~ver #2/19644 connect(fd: 230, uservaddr: { .family: INET, port: 53, addr: 127.0.0.53 }, addrlen: 16) = 0
4422.354 ( 0.012 ms): systemd-resolv/1213 connect(fd: 14, uservaddr: { .family: INET, port: 53, addr: 192.168.86.1 }, addrlen: 16) = 0
4422.225 ( 0.017 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: LOCAL, path: /run/systemd/r }, addrlen: 42) = 0
4422.590 ( 0.010 ms): systemd-resolv/1213 connect(fd: 26, uservaddr: { .family: INET, port: 53, addr: 192.168.86.1 }, addrlen: 16) = 0
4422.642 ( 0.003 ms): systemd-resolv/1213 connect(fd: 27, uservaddr: { .family: INET, port: 53, addr: 192.168.86.1 }, addrlen: 16) = 0
4457.908 ( 0.014 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: INET6, port: 0, addr: 2800:3f0:4001:837::900:0, scope_id: 5767169 }, addrlen: 28) = 0
4457.924 ( 0.001 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: UNSPEC }, addrlen: 16) = 0
4457.926 ( 0.004 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: INET, port: 0, addr: 142.251.128.234 }, addrlen: 16) = 0
5000.793 ( 0.059 ms): pool/3112 connect(fd: 7, uservaddr: { .family: LOCAL, path: /var/run/.heim }, addrlen: 110) = 0
^Croot@number:~#I'll try with some other systems and report.
- Arnaldo
Hi Arnaldo,
Quoted 41 lines · Arnaldo Carvalho de Melo
Quoted 37 lines · Namhyung Kim
Cc-ing bpf list.
Quoted 34 lines · Howard Chu
Forgot to mention clang-13 gave unbounded memory access too:
ffffffff,var_off=(0x0;
0xffffffff)) R9=scalar(id=14,smin=umin=umin32=2,smax=umax=0xffffffff,var_off=(0x0;0xffffffff))
90: (85) call bpf_probe_read_user#112
R2 unbounded memory access, use 'var &= const' or 'if (var < const)' processed 490 insns (limit 1000000)max_states_per_insn2total_states23peak_states23mark_read15
-- END PROG LOAD LOG --
libbpf: prog 'sys_enter': failed to load: -13
libbpf: failed to load object 'augmented_raw_syscalls_bpf'
libbpf: failed to load BPF skeleton 'augmented_raw_syscalls_bpf': -13
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
libbpf: map '__augmented_syscalls__': can't use BPF map without FD (was it created?)
Not enough memory to run!Kernel:
perf $ uname -r
6.11.0-061100-genericQuoted 5 lines · Howard Chu
Apologies. I observed the same issue and tested perf with trace BPF skel generated by clang-13 to clang-18, turns out BPF skelw generated by clang version <= clang-16 are not loadable, with clang-15 and -16 showing the same error as yours. Additionally, the BPF verifier is running longer than usual to process the instructions.
Quoted 3 lines
Some more datapoints, here it is working, below you'll see a 'perf trace' session tracing bpf syscalls (that use libbpf btf pretty printing routines, that has to be improved) emitted by a 'perf trace' session
Agree, I don't like the constant malloc / free part of the btf_dump.
Quoted 5 lines
augmenting open* syscalls (i.e. using BPF to get the syscall pointer args, namely pathnames), and here is the environment:
⬢ [acme@toolbox perf-tools-next]$ rpm -q clang
clang-18.1.8-1.fc40.x86_64
This is cool, I'm wondering if you have tried to generate the BPF skel using clang version <= 16 because that's where problems occured.
Quoted 31 lines
root@number:~# uname -a Linux number 6.13.0-rc2 #1 SMP PREEMPT_DYNAMIC Mon Dec 9 12:33:35 -03 2024 x86_64 GNU/Linux root@number:~#root@number:~# perf trace -e bpf --max-events=3 perf trace -e open* -a sleep 0.0001 |& head 0.000 ( 0.005 ms): :3117755/3117755 bpf(cmd: 36, uattr: (union bpf_attr){.batch = (struct){.in_batch = (__u64)42949672960,.out_batch = (__u64)18446744073709551448,.keys = (__u64)2,.values = (__u64)801012880,.elem_flags = (__u64)140154830285504,.flags = (__u64)11424,},(struct){.pathname = (__u64)42949672960,.bpf_fd = (__u32)4294967128,.file_flags = (__u32)4294967295,.path_fd = (__s32)2,},.raw_tracepoint = (struct){.name = (__u64)42949672960,.prog_fd = (__u32)4294967128,.cookie = (__u64)2,},(struct){.btf = (__u64)42949672960,.btf_log_buf = (__u64)18446744073709551448,.btf_size = (__u32)2,.btf_log_level = (__u32)801012880,},}, size: 8) = -1 EOPNOTSUPP (Operation not supported) 0.009 ( 0.042 ms): :3117755/3117755 bpf(cmd: PROG_LOAD, uattr: (union bpf_attr){(struct){.map_type = (__u32)1,.key_size = (__u32)2,.value_size = (__u32)2233669616,.max_entries = (__u32)32767,.map_flags = (__u32)7334203,},(struct){.map_fd = (__u32)1,.key = (__u64)140735427057648,(union){.value = (__u64)7334203,.next_key = (__u64)7334203,},},.batch = (struct){.in_batch = (__u64)8589934593,.out_batch = (__u64)140735427057648,.keys = (__u64)7334203,},(struct){.prog_type = (__u32)1,.insn_cnt = (__u32)2,.insns = (__u64)140735427057648,.license = (__u64)7334203,},(struct){.pathname = (__u64)8589934593,.bpf_fd = (__u32)2233669616,.file_flags = (__u32)32767,.path_fd = (__s32)7334203,},(struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_bpf_fd = (__u32)2,.attach_type = (__u32)2233669616,.attach_flags = (__u32)32767,.replace_bpf_fd = (__u32)7334203,},.test = (struct){.prog_fd = (__u32)1,.retval = (__u32)2,.data_size_in = (__u32)2233669616,.data_size_out = (__u32)32767,.data_in = (__u64)7334203,},(struct){(union){.start_id = (__u32)1,.prog_id = (__u32)1,.map_id = (__u32)1,.btf_id = (__u32)1,.link_id = (__u32)1,},.next_id = (__u32)2,.open_flags = (__u32)2233669616,},.info = (struct){.bpf_fd = (__u32)1,.info_len = (__u32)2,.info = (__u64)140735427057648,},.query = (struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_type = (__u32)2,.query_flags = (__u32)2233669616,.attach_flags = (__u32)32767,.prog_ids = (__u64)7334203,},.raw_tracepoint = (struct){.name = (__u64)8589934593,.prog_fd = (__u32)2233669616,.cookie = (__u64)7334203,},(struct){.btf = (__u64)8589934593,.btf_log_buf = (__u64)140735427057648,.btf_size = (__u32)7334203,},.task_fd_query = (struct){.pid = (__u32)1,.fd = (__u32)2,.flags = (__u32)2233669616,.buf_len = (__u32)32767,.buf = (__u64)7334203,},.link_create = (struct){(union){.prog_fd = (__u32)1,.map_fd = (__u32)1,},(union){.target_fd = (__u32)2,.target_ifindex = (__u32)2,},.attach_type = (__u32)2233669616,.flags = (__u32)32767,(union){.target_btf_id = (__u32)7334203,(struct){.iter_info = (__u64)7334203) = 10 0.054 ( 0.010 ms): :3117755/3117755 bpf(cmd: PROG_LOAD, uattr: (union bpf_attr){(struct){.map_type = (__u32)1,.key_size = (__u32)2,.value_size = (__u32)2233670144,.max_entries = (__u32)32767,.map_flags = (__u32)7314455,},(struct){.map_fd = (__u32)1,.key = (__u64)140735427058176,(union){.value = (__u64)7314455,.next_key = (__u64)7314455,},},.batch = (struct){.in_batch = (__u64)8589934593,.out_batch = (__u64)140735427058176,.keys = (__u64)7314455,},(struct){.prog_type = (__u32)1,.insn_cnt = (__u32)2,.insns = (__u64)140735427058176,.license = (__u64)7314455,},(struct){.pathname = (__u64)8589934593,.bpf_fd = (__u32)2233670144,.file_flags = (__u32)32767,.path_fd = (__s32)7314455,},(struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_bpf_fd = (__u32)2,.attach_type = (__u32)2233670144,.attach_flags = (__u32)32767,.replace_bpf_fd = (__u32)7314455,},.test = (struct){.prog_fd = (__u32)1,.retval = (__u32)2,.data_size_in = (__u32)2233670144,.data_size_out = (__u32)32767,.data_in = (__u64)7314455,},(struct){(union){.start_id = (__u32)1,.prog_id = (__u32)1,.map_id = (__u32)1,.btf_id = (__u32)1,.link_id = (__u32)1,},.next_id = (__u32)2,.open_flags = (__u32)2233670144,},.info = (struct){.bpf_fd = (__u32)1,.info_len = (__u32)2,.info = (__u64)140735427058176,},.query = (struct){(union){.target_fd = (__u32)1,.target_ifindex = (__u32)1,},.attach_type = (__u32)2,.query_flags = (__u32)2233670144,.attach_flags = (__u32)32767,.prog_ids = (__u64)7314455,},.raw_tracepoint = (struct){.name = (__u64)8589934593,.prog_fd = (__u32)2233670144,.cookie = (__u64)7314455,},(struct){.btf = (__u64)8589934593,.btf_log_buf = (__u64)140735427058176,.btf_size = (__u32)7314455,},.task_fd_query = (struct){.pid = (__u32)1,.fd = (__u32)2,.flags = (__u32)2233670144,.buf_len = (__u32)32767,.buf = (__u64)7314455,},.link_create = (struct){(union){.prog_fd = (__u32)1,.map_fd = (__u32)1,},(union){.target_fd = (__u32)2,.target_ifindex = (__u32)2,},.attach_type = (__u32)2233670144,.flags = (__u32)32767,(union){.target_btf_id = (__u32)7314455,(struct){.iter_info = (__u64)7314455) = 10 0.000 ( 0.007 ms): sleep/3117756 openat(dfd: CWD, filename: "/etc/ld.so.cache", flags: RDONLY|CLOEXEC) = 3 0.022 ( 0.004 ms): sleep/3117756 openat(dfd: CWD, filename: "/lib64/libc.so.6", flags: RDONLY|CLOEXEC) = 3 0.201 ( 0.006 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/lib/locale/locale-archive", flags: RDONLY|CLOEXEC) = 3 0.247 ( 0.004 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/locale.alias", flags: RDONLY|CLOEXEC) = 3 0.283 ( 0.003 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo") = -1 ENOENT (No such file or directory) 0.287 ( 0.002 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo") = -1 ENOENT (No such file or directory) 0.290 ( 0.003 ms): sleep/3117756 openat(dfd: CWD, filename: "/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo") = -1 ENOENT (No such file or directory) root@number:~#root@number:~# perf trace -e connect 0.000 ( 0.020 ms): pool/3112 connect(fd: 7, uservaddr: { .family: LOCAL, path: /var/run/.heim }, addrlen: 110) = 0 665.621 ( 0.029 ms): Chrome_ChildIO/3102157 connect(fd: 26, uservaddr: { .family: INET6, port: 443, addr: 2001:4860:4860::900:0, scope_id: 5767169 }, addrlen: 28) = 0 4422.069 ( 0.024 ms): DNS Res~ver #2/19644 connect(fd: 230, uservaddr: { .family: INET, port: 53, addr: 127.0.0.53 }, addrlen: 16) = 0 4422.354 ( 0.012 ms): systemd-resolv/1213 connect(fd: 14, uservaddr: { .family: INET, port: 53, addr: 192.168.86.1 }, addrlen: 16) = 0 4422.225 ( 0.017 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: LOCAL, path: /run/systemd/r }, addrlen: 42) = 0 4422.590 ( 0.010 ms): systemd-resolv/1213 connect(fd: 26, uservaddr: { .family: INET, port: 53, addr: 192.168.86.1 }, addrlen: 16) = 0 4422.642 ( 0.003 ms): systemd-resolv/1213 connect(fd: 27, uservaddr: { .family: INET, port: 53, addr: 192.168.86.1 }, addrlen: 16) = 0 4457.908 ( 0.014 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: INET6, port: 0, addr: 2800:3f0:4001:837::900:0, scope_id: 5767169 }, addrlen: 28) = 0 4457.924 ( 0.001 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: UNSPEC }, addrlen: 16) = 0 4457.926 ( 0.004 ms): DNS Res~ver #6/3099285 connect(fd: 227, uservaddr: { .family: INET, port: 0, addr: 142.251.128.234 }, addrlen: 16) = 0 5000.793 ( 0.059 ms): pool/3112 connect(fd: 7, uservaddr: { .family: LOCAL, path: /var/run/.heim }, addrlen: 110) = 0 ^Croot@number:~#
Quoted 1 line
I'll try with some other systems and report.
Thanks a lot Arnaldo :)
Thanks,
Howard
Quoted 2 lines
- Arnaldo