perf probe can't remove probes
Masami, have you stumbled on this already?
[root@seventh ~]# perf probe security_locked_down%return 'ret=$retval' Added new event:
probe:security_locked_down__return (on security_locked_down%return with ret=$retval)
You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down__return -aR sleep 1
[root@seventh ~]# perf probe security_locked_down what Added new event:
probe:security_locked_down (on security_locked_down with what)
You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down -aR sleep 1
[root@seventh ~]#
[root@seventh ~]# uname -r
5.10.0-rc3.bpfsign+
[root@seventh ~]# perf probe -l
probe:security_locked_down (on security_locked_down@git/bpf/security/security.c with what) probe:security_locked_down__return (on security_locked_down%return@git/bpf/security/security.c with ret)
[root@seventh ~]# perf probe -D '*:*'
Semantic error :There is non-digit char in line number.
Usage: perf probe [<options>] 'PROBEDEF' ['PROBEDEF' ...]
or: perf probe [<options>] --add 'PROBEDEF' [--add 'PROBEDEF' ...]
or: perf probe [<options>] --del '[GROUP:]EVENT' ...
or: perf probe --list [GROUP:]EVENT ...
or: perf probe [<options>] --line 'LINEDESC'
or: perf probe [<options>] --vars 'PROBEPOINT'
or: perf probe [<options>] --funcs
-D, --definition <[EVENT=]FUNC[@SRC][+OFF|%return|:RL|;PT]|SRC:AL|SRC;PT [[NAME=]ARG ...]>
Show trace event definition of given traceevent for k/uprobe_events.[root@seventh ~]# perf probe probe:security_locked_down Semantic error :There is non-digit char in line number.
Error: Command Parse Error.
[root@seventh ~]# perf probe probe:security_locked_down__return Semantic error :There is non-digit char in line number.
Error: Command Parse Error.
[root@seventh ~]# cat /sys/kernel/debug/kprobes/
blacklist enabled list
[root@seventh ~]# cat /sys/kernel/debug/kprobes/list
ffffffff8248b350 k security_locked_down+0x0 [FTRACE] ffffffff8248b350 r security_locked_down+0x0 [FTRACE]
[root@seventh ~]#
[root@seventh ~]# cat /etc/fedora-release
Fedora release 33 (Thirty Three)
[root@seventh ~]# gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.1 20201016 (Red Hat 10.2.1-6) (GCC)
[root@seventh ~]# rpm -q elfutils
elfutils-0.182-1.fc33.x86_64
[root@seventh ~]#
- Arnaldo
Hi Arnaldo,
Quoted 31 lines · On Wed, 25 Nov 2020 14:27:55 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Masami, have you stumbled on this already?
[root@seventh ~]# perf probe security_locked_down%return 'ret=$retval' Added new event:
probe:security_locked_down__return (on security_locked_down%return with ret=$retval)You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down__return -aR sleep 1
[root@seventh ~]# perf probe
security_locked_downwhat Added new event:
probe:security_locked_down (onsecurity_locked_downwith what)You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down -aR sleep 1
[root@seventh ~]#
[root@seventh ~]# uname -r
5.10.0-rc3.bpfsign+
[root@seventh ~]# perf probe -l
probe:security_locked_down (onsecurity_locked_down@git/bpf/security/security.cwith what) probe:security_locked_down__return (on security_locked_down%return@git/bpf/security/security.c with ret)
[root@seventh ~]# perf probe -D '*:*'
Semantic error :There is non-digit char in line number.Usage: perf probe [<options>] 'PROBEDEF' ['PROBEDEF' ...]
Quoted 9 lines
or: perf probe [<options>] --add 'PROBEDEF' [--add 'PROBEDEF' ...]
or: perf probe [<options>] --del '[GROUP:]EVENT' ...
or: perf probe --list [GROUP:]EVENT ...
or: perf probe [<options>] --line 'LINEDESC'
or: perf probe [<options>] --vars 'PROBEPOINT'
or: perf probe [<options>] --funcs
-D, --definition <[EVENT=]FUNC[@SRC][+OFF|%return|:RL|;PT]|SRC:AL|SRC;PT [[NAME=]ARG ...]>
Show trace event definition of given traceevent for k/uprobe_events.As you can see, "-D" is showing definition. Not delete. (*) Delete is "-d" or "--del".
(*) this option is for different version of kernel, remote-machine and boot-time tracing.
Quoted 6 lines
[root@seventh ~]# perf probe probe:security_locked_down Semantic error :There is non-digit char in line number.
Error: Command Parse Error.
[root@seventh ~]# perf probe probe:security_locked_down__return Semantic error :There is non-digit char in line number.
Error: Command Parse Error.
Since you don't pass any option, both are for adding new probe event.
What happen if you run
$ perf probe -d "*:*"
?
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Em Thu, Nov 26, 2020 at 09:21:25AM +0900, Masami Hiramatsu escreveu:
Quoted 43 lines
Hi Arnaldo,
Quoted 38 lines · On Wed, 25 Nov 2020 14:27:55 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Masami, have you stumbled on this already?
[root@seventh ~]# perf probe security_locked_down%return 'ret=$retval' Added new event:
probe:security_locked_down__return (on security_locked_down%return with ret=$retval)You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down__return -aR sleep 1
[root@seventh ~]# perf probe
security_locked_downwhat Added new event:
probe:security_locked_down (onsecurity_locked_downwith what)You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down -aR sleep 1
[root@seventh ~]#
[root@seventh ~]# uname -r
5.10.0-rc3.bpfsign+
[root@seventh ~]# perf probe -l
probe:security_locked_down (onsecurity_locked_down@git/bpf/security/security.cwith what) probe:security_locked_down__return (on security_locked_down%return@git/bpf/security/security.c with ret)
[root@seventh ~]# perf probe -D '*:*'
Semantic error :There is non-digit char in line number.Usage: perf probe [<options>] 'PROBEDEF' ['PROBEDEF' ...]
or: perf probe [<options>] --add 'PROBEDEF' [--add 'PROBEDEF' ...] or: perf probe [<options>] --del '[GROUP:]EVENT' ... or: perf probe --list [GROUP:]EVENT ... or: perf probe [<options>] --line 'LINEDESC' or: perf probe [<options>] --vars 'PROBEPOINT' or: perf probe [<options>] --funcs
Quoted 5 lines
Quoted 2 lines
-D, --definition <[EVENT=]FUNC[@SRC][+OFF|%return|:RL|;PT]|SRC:AL|SRC;PT [[NAME=]ARG ...]>
Show trace event definition of given traceevent for k/uprobe_events.As you can see, "-D" is showing definition. Not delete. (*) Delete is "-d" or "--del".
Yeah, I was in a hurry and looked at just the first line right after the command, didn't want to forget reporting it so sent the "bug" report, d0h, sorry about the noise, using -d or --del works.
But having both -d and -D, in retrospect, wasn't such a good idea :-\
- Arnaldo
Quoted 22 lines
(*) this option is for different version of kernel, remote-machine and boot-time tracing.
Quoted 6 lines
[root@seventh ~]# perf probe probe:security_locked_down Semantic error :There is non-digit char in line number.
Error: Command Parse Error.
[root@seventh ~]# perf probe probe:security_locked_down__return Semantic error :There is non-digit char in line number.
Error: Command Parse Error.Since you don't pass any option, both are for adding new probe event.
What happen if you run
$ perf probe -d "*:*"?
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Signature · 1 line
- Arnaldo
Hi Masami,
Any idea why listing the source code doesn't work while simply
adding the probe works?
[root@seventh ~]# perf probe verify_pkcs7_signature Added new event:
probe:verify_pkcs7_signature (on verify_pkcs7_signature)
You can now use it in all perf tools, such as:
perf record -e probe:verify_pkcs7_signature -aR sleep 1
[root@seventh ~]#
[root@seventh ~]# readelf -wi /lib/modules/5.10.0-rc3.bpfsign+/build/vmlinux | grep DW_AT_producer -m2
<1c> DW_AT_producer : (indirect string, offset: 0x50): GNU AS 2.35
<2f> DW_AT_producer : (indirect string, offset: 0x2daf): GNU C89 10.2.1 20201016 (Red Hat 10.2.1-6) -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone -mcmodel=kernel -mindirect-branch=thunk-extern -mindirect-branch-register -mrecord-mcount -mfentry -march=x86-64 -g -O2 -std=gnu90 -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -falign-jumps=1 -falign-loops=1 -fno-asynchronous-unwind-tables -fno-jump-tables -fno-delete-null-pointer-checks -fno-allow-store-data-races -fstack-protector-strong -fno-strict-overflow -fstack-check=no -fconserve-stack -fcf-protection=none -fno-stack-protector[root@seventh ~]#
[root@seventh ~]# perf probe -L verify_pkcs7_signature
Specified source line is not found.
Error: Failed to show lines.
[root@seventh ~]# grep verify_pkcs7_signature /proc/kallsymsffffffff9f2617b0 T verify_pkcs7_signatureffffffffa047ac30 r __ksymtab_verify_pkcs7_signatureffffffffa0482e38 r __kstrtabns_verify_pkcs7_signatureffffffffa0483b3b r __kstrtab_verify_pkcs7_signature
[root@seventh ~]# perf probe -L verify_pkcs7_signature
Specified source line is not found.
Error: Failed to show lines.
[root@seventh ~]# perf probe -v -L verify_pkcs7_signature
Looking at the vmlinux_path (8 entries long)
Using /lib/modules/5.10.0-rc3.bpfsign+/build/vmlinux for symbols
Open Debuginfo file: /lib/modules/5.10.0-rc3.bpfsign+/build/vmlinux
fname: /home/acme/git/bpf/include/linux/verification.h, lineno:38
New line range: 38 to 2147483647
path: (null)
Specified source line is not found.
Error: Failed to show lines. Reason: No such file or directory (Code: -2)
[root@seventh ~]# readelf -wi /lib/modules/5.10.0-rc3.bpfsign+/build/vmlinux | grep -B2 -A8 verify_pkcs7_signature
<1><1a362a2>: Abbrev Number: 64 (DW_TAG_subprogram)
<1a362a3> DW_AT_external : 1
<1a362a3> DW_AT_name : (indirect string, offset: 0xc2d22): verify_pkcs7_signature
<1a362a7> DW_AT_decl_file : 52
<1a362a8> DW_AT_decl_line : 38
<1a362a9> DW_AT_decl_column : 12
<1a362aa> DW_AT_prototyped : 1
<1a362aa> DW_AT_type : <0x1a3333f>
<1a362ae> DW_AT_sibling : <0x1a36313>
<2><1a362b2>: Abbrev Number: 65 (DW_TAG_formal_parameter)
<1a362b3> DW_AT_name : (indirect string, offset: 0x19ee3d): data
--
<1><24fa79d>: Abbrev Number: 83 (DW_TAG_subprogram)
<24fa79e> DW_AT_external : 1
<24fa79e> DW_AT_name : (indirect string, offset: 0xc2d22): verify_pkcs7_signature
<24fa7a2> DW_AT_decl_file : 218
<24fa7a3> DW_AT_decl_line : 38
<24fa7a4> DW_AT_decl_column : 12
<24fa7a5> DW_AT_prototyped : 1
<24fa7a5> DW_AT_type : <0x24e1e05>
<24fa7a9> DW_AT_sibling : <0x24fa80e>
<2><24fa7ad>: Abbrev Number: 41 (DW_TAG_formal_parameter)
<24fa7ae> DW_AT_name : (indirect string, offset: 0x19ee3d): data
--
<26258b8> DW_AT_location : 9 byte block: 3 0 0 0 0 0 0 0 0 (DW_OP_addr: 0)
<1><26258c2>: Abbrev Number: 116 (DW_TAG_variable)
<26258c3> DW_AT_name : (indirect string, offset: 0x10523f): __UNIQUE_ID___addressable_verify_pkcs7_signature164
<26258c7> DW_AT_decl_file : 1
<26258c8> DW_AT_decl_line : 305
<26258ca> DW_AT_decl_column : 1
<26258cb> DW_AT_type : <0x2620ca6>
<26258cf> DW_AT_location : 9 byte block: 3 0 0 0 0 0 0 0 0 (DW_OP_addr: 0)
<1><26258d9>: Abbrev Number: 117 (DW_TAG_subprogram)
<26258da> DW_AT_external : 1
<26258da> DW_AT_name : (indirect string, offset: 0x1050d9): pkcs7_free_message
--
<1><2625c2e>: Abbrev Number: 120 (DW_TAG_subprogram)
<2625c2f> DW_AT_external : 1
<2625c2f> DW_AT_name : (indirect string, offset: 0xc2d22): verify_pkcs7_signature
<2625c33> DW_AT_decl_file : 1
<2625c34> DW_AT_decl_line : 282
<2625c36> DW_AT_decl_column : 5
<2625c37> DW_AT_prototyped : 1
<2625c37> DW_AT_type : <0x26209e9>
<2625c3b> DW_AT_low_pc : 0xffffffff812617b0
<2625c43> DW_AT_high_pc : 0x73
<2625c4b> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa)
--
<1><495560e>: Abbrev Number: 68 (DW_TAG_subprogram)
<495560f> DW_AT_external : 1
<495560f> DW_AT_name : (indirect string, offset: 0xc2d22): verify_pkcs7_signature
<4955613> DW_AT_decl_file : 61
<4955614> DW_AT_decl_line : 38
<4955615> DW_AT_decl_column : 12
<4955616> DW_AT_prototyped : 1
<4955616> DW_AT_type : <0x49519fd>
<495561a> DW_AT_sibling : <0x495567f>
<2><495561e>: Abbrev Number: 74 (DW_TAG_formal_parameter)
<495561f> DW_AT_name : (indirect string, offset: 0x19ee3d): dataEm Tue, Dec 01, 2020 at 10:25:17AM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 4 lines
Hi Masami,
Any idea why listing the source code doesn't work while simply
adding the probe works?
<SNIP>
Quoted 3 lines
[root@seventh ~]# perf probe -L
verify_pkcs7_signatureSpecified source line is not found.
Error: Failed to show lines.
Yeah, those fixes you sent, when cherry-picked into this bpf-next/master based branch, do the trick, so nevermind, you fixed this already, these fixes were already merged by Linus, bpf-next will rebase at some point, etc. :-)
- Arnaldo
[acme@five bpf]$ git log --oneline -40f3147557ea8143d (HEAD -> bpfsign) perf probe: Change function definition check due to broken DWARF 0e104f490d8f86b4 perf probe: Fix to die_entrypc() returns error correctly 9423e1f089643738 libbpf: Check if the kernel supports signatures before associating them 580616996c498b70 libbpf: Attach signature ELF sections to signed ELF program sections [acme@five bpf]$
[root@seventh ~]# perf probe -L verify_pkcs7_signature <verify_pkcs7_signature@/home/acme/git/bpf/certs/system_keyring.c:0>
0 int verify_pkcs7_signature(const void *data, size_t len,
const void *raw_pkcs7, size_t pkcs7_len,
struct key *trusted_keys,
enum key_being_used_for usage,
int (*view_content)(void *ctx,
const void *data, size_t len,
size_t asn1hdrlen),
void *ctx)
{
9 struct pkcs7_message *pkcs7;
int ret;
pkcs7 = pkcs7_parse_message(raw_pkcs7, pkcs7_len);
13 if (IS_ERR(pkcs7))
14 return PTR_ERR(pkcs7);
16 ret = verify_pkcs7_message_sig(data, len, pkcs7, trusted_keys, usage,
view_content, ctx);
19 pkcs7_free_message(pkcs7);
20 pr_devel("<==%s() = %d\n", __func__, ret);
return ret;
}
EXPORT_SYMBOL_GPL(verify_pkcs7_signature);[root@seventh ~]#
Quoted 47 lines · On Thu, 26 Nov 2020 14:26:03 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Em Thu, Nov 26, 2020 at 09:21:25AM +0900, Masami Hiramatsu escreveu:
Quoted 46 lines
Hi Arnaldo,
Quoted 40 lines · On Wed, 25 Nov 2020 14:27:55 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Masami, have you stumbled on this already?
[root@seventh ~]# perf probe security_locked_down%return 'ret=$retval' Added new event:
probe:security_locked_down__return (on security_locked_down%return with ret=$retval)You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down__return -aR sleep 1
[root@seventh ~]# perf probe
security_locked_downwhat Added new event:
probe:security_locked_down (onsecurity_locked_downwith what)You can now use it in all perf tools, such as:
perf record -e probe:security_locked_down -aR sleep 1
[root@seventh ~]#
[root@seventh ~]# uname -r
5.10.0-rc3.bpfsign+
[root@seventh ~]# perf probe -l
probe:security_locked_down (onsecurity_locked_down@git/bpf/security/security.cwith what) probe:security_locked_down__return (on security_locked_down%return@git/bpf/security/security.c with ret)
[root@seventh ~]# perf probe -D '*:*'
Semantic error :There is non-digit char in line number.Usage: perf probe [<options>] 'PROBEDEF' ['PROBEDEF' ...]
or: perf probe [<options>] --add 'PROBEDEF' [--add 'PROBEDEF' ...] or: perf probe [<options>] --del '[GROUP:]EVENT' ... or: perf probe --list [GROUP:]EVENT ... or: perf probe [<options>] --line 'LINEDESC' or: perf probe [<options>] --vars 'PROBEPOINT' or: perf probe [<options>] --funcs -D, --definition <[EVENT=]FUNC[@SRC][+OFF|%return|:RL|;PT]|SRC:AL|SRC;PT [[NAME=]ARG ...]> Show trace event definition of given traceevent for k/uprobe_events.
Quoted 8 lines
Quoted 2 lines
As you can see, "-D" is showing definition. Not delete. (*) Delete is "-d" or "--del".
Yeah, I was in a hurry and looked at just the first line right after the command, didn't want to forget reporting it so sent the "bug" report, d0h, sorry about the noise, using -d or --del works.
But having both -d and -D, in retrospect, wasn't such a good idea :-\
Sorry for confusing :(
Hmm, would we better to remove -D and keep only --definition? But it is already there, I think we should keep this option for backward compatibility.
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Em Wed, Dec 16, 2020 at 09:16:40AM +0900, Masami Hiramatsu escreveu:
Quoted 1 line
On Thu, 26 Nov 2020 14:26:03 -0300
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 8 lines
Em Thu, Nov 26, 2020 at 09:21:25AM +0900, Masami Hiramatsu escreveu:
Quoted 3 lines · On Wed, 25 Nov 2020 14:27:55 -0300 Arnaldo Carvalho de Melo <[email protected]>…
As you can see, "-D" is showing definition. Not delete. (*) Delete is "-d" or "--del".
Yeah, I was in a hurry and looked at just the first line right after the command, didn't want to forget reporting it so sent the "bug" report, d0h, sorry about the noise, using -d or --del works.
Quoted 7 lines
Quoted 1 line
But having both -d and -D, in retrospect, wasn't such a good idea :-\
Sorry for confusing :(
Hmm, would we better to remove -D and keep only --definition? But it is already there, I think we should keep this option for backward compatibility.
Nah, since we have it there, lets take this just as a lesson and in the future use single letter options just for core functionality, leaving things like --definition to be just a long option.
We can't change it now :-)
- Arnaldo