Skip to contentSkip to conversation

perf probe -d failing to delete probes

DISCUSSION Arnaldo Carvalho de Melo 2 participants 5 messages Nov 29, 2019 – Dec 3, 2019

lore
Share & tools
lore
#0 lore raw

Hi Masami,

Can you please take a look at this?

[root@quaco perf]# perf probe -l
probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__set_paused+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__resume+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evsel__group_desc+222@util/evsel.c in /home/acme/bin/perf) probe_perf:map__map_ip (on svg_partial_wakeline+222@util/svghelper.c in /home/acme/bin/perf)
[root@quaco perf]# perf probe -d probe_perf:map_*
"probe_perf:map_*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe_perf:*
"probe_perf:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe*:*
"probe*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d '*:*'
"*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]#

- Arnaldo

#1 lore raw

Em Fri, Nov 29, 2019 at 12:13:24PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 17 lines

Hi Masami,

Can you please take a look at this?

[root@quaco perf]# perf probe -l
probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__set_paused+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__resume+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evsel__group_desc+222@util/evsel.c in /home/acme/bin/perf) probe_perf:map__map_ip (on svg_partial_wakeline+222@util/svghelper.c in /home/acme/bin/perf)
[root@quaco perf]# perf probe -d probe_perf:map_*
"probe_perf:map_*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe_perf:*
"probe_perf:*" does not hit any event.

Quoted 8 lines

Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe*:* "probe*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d '*:*' "*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]#

[root@quaco ~]# perf probe -l
probe_perf:map__map_ip (on process_sample_event+6663@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on process_sample_event+6739@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on unwind_entry+436@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on machine__process_extra_kernel_map+31@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on __maps__insert+192@util/map.c in /home/acme/bin/perf) probe_perf:map__map_ip (on dso__process_kernel_symbol+336@util/symbol-elf.c in /home/acme/bin/perf)
[root@quaco ~]# ls -la /sys/kernel/debug/tracing/events/probe_perf/
total 0
drwxr-xr-x. 3 root root 0 Nov 29 12:09 .
drwxr-xr-x. 113 root root 0 Dec 2 15:56 ..

-rw-r--r--.   1 root root 0 Nov 29 12:09 enable
-rw-r--r--.   1 root root 0 Nov 29 12:09 filter
drwxr-xr-x.   2 root root 0 Nov 29 12:09 map__map_ip

[root@quaco ~]#

Trying to figure out how to remove it via debugfs/tracefs

#2 lore raw

Em Mon, Dec 02, 2019 at 03:59:58PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 43 lines

Em Fri, Nov 29, 2019 at 12:13:24PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 17 lines

Hi Masami,

Can you please take a look at this?

[root@quaco perf]# perf probe -l
probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__set_paused+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__resume+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evsel__group_desc+222@util/evsel.c in /home/acme/bin/perf) probe_perf:map__map_ip (on svg_partial_wakeline+222@util/svghelper.c in /home/acme/bin/perf)
[root@quaco perf]# perf probe -d probe_perf:map_*
"probe_perf:map_*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe_perf:*
"probe_perf:*" does not hit any event.

Quoted 8 lines

Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe*:* "probe*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d '*:*' "*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]#

[root@quaco ~]# perf probe -l
probe_perf:map__map_ip (on process_sample_event+6663@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on process_sample_event+6739@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on unwind_entry+436@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on machine__process_extra_kernel_map+31@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on __maps__insert+192@util/map.c in /home/acme/bin/perf) probe_perf:map__map_ip (on dso__process_kernel_symbol+336@util/symbol-elf.c in /home/acme/bin/perf)
[root@quaco ~]# ls -la /sys/kernel/debug/tracing/events/probe_perf/
total 0
drwxr-xr-x. 3 root root 0 Nov 29 12:09 .
drwxr-xr-x. 113 root root 0 Dec 2 15:56 ..

(quoted diff) +0 -2
-rw-r--r--.   1 root root 0 Nov 29 12:09 enable
-rw-r--r--.   1 root root 0 Nov 29 12:09 filter

drwxr-xr-x. 2 root root 0 Nov 29 12:09 map__map_ip
[root@quaco ~]#

Quoted 1 line

Trying to figure out how to remove it via debugfs/tracefs

[root@quaco ~]# cat /sys/kernel/debug/tracing/dynamic_events p:probe_perf/map__map_ip /home/acme/bin/perf:0x000000000005f257 p:probe_perf/map__map_ip /home/acme/bin/perf:0x000000000005f2a3 p:probe_perf/map__map_ip /home/acme/bin/perf:0x00000000000fda24 p:probe_perf/map__map_ip /home/acme/bin/perf:0x00000000000fdacf p:probe_perf/map__map_ip /home/acme/bin/perf:0x00000000001028d0 p:probe_perf/map__map_ip /home/acme/bin/perf:0x000000000016dd90 [root@quaco ~]#

[root@quaco ~]# cat /sys/kernel/debug/tracing/kprobe_events [root@quaco ~]#

[root@quaco ~]# echo > /sys/kernel/debug/tracing/kprobe_events
[root@quaco ~]# perf probe -l
probe_perf:map__map_ip (on process_sample_event+6663@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on process_sample_event+6739@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on unwind_entry+436@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on machine__process_extra_kernel_map+31@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on __maps__insert+192@util/map.c in /home/acme/bin/perf) probe_perf:map__map_ip (on dso__process_kernel_symbol+336@util/symbol-elf.c in /home/acme/bin/perf)
[root@quaco ~]#

Finally it works:

[root@quaco ~]# echo > /sys/kernel/debug/tracing/dynamic_events [root@quaco ~]# cat /sys/kernel/debug/tracing/dynamic_events [root@quaco ~]#
[root@quaco ~]# perf probe -l
[root@quaco ~]#

A workaround, now to find out why 'perf probe -d '*:*' and the other reported variants are failing...

- Arnaldo

#3 lore raw

Hi Arnaldo,

Quoted 70 lines · On Mon, 2 Dec 2019 16:04:52 -0300 Arnaldo Carvalho de Melo <[email protected]>…

Em Mon, Dec 02, 2019 at 03:59:58PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 43 lines

Em Fri, Nov 29, 2019 at 12:13:24PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 17 lines

Hi Masami,

Can you please take a look at this?

[root@quaco perf]# perf probe -l
probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on ui__warn_map_erange+222@perf/tools/perf/builtin-top.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__set_paused+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evlist__resume+222@util/evlist.c in /home/acme/bin/perf) probe_perf:map__map_ip (on perf_evsel__group_desc+222@util/evsel.c in /home/acme/bin/perf) probe_perf:map__map_ip (on svg_partial_wakeline+222@util/svghelper.c in /home/acme/bin/perf)
[root@quaco perf]# perf probe -d probe_perf:map_*
"probe_perf:map_*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe_perf:*
"probe_perf:*" does not hit any event.

Quoted 8 lines

Error: Failed to delete events.
[root@quaco perf]# perf probe -d probe*:* "probe*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]# perf probe -d '*:*' "*:*" does not hit any event.
Error: Failed to delete events.
[root@quaco perf]#

[root@quaco ~]# perf probe -l
probe_perf:map__map_ip (on process_sample_event+6663@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on process_sample_event+6739@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on unwind_entry+436@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on machine__process_extra_kernel_map+31@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on __maps__insert+192@util/map.c in /home/acme/bin/perf) probe_perf:map__map_ip (on dso__process_kernel_symbol+336@util/symbol-elf.c in /home/acme/bin/perf)
[root@quaco ~]# ls -la /sys/kernel/debug/tracing/events/probe_perf/
total 0
drwxr-xr-x. 3 root root 0 Nov 29 12:09 .
drwxr-xr-x. 113 root root 0 Dec 2 15:56 ..

(quoted diff) +0 -2
-rw-r--r--.   1 root root 0 Nov 29 12:09 enable
-rw-r--r--.   1 root root 0 Nov 29 12:09 filter

drwxr-xr-x. 2 root root 0 Nov 29 12:09 map__map_ip
[root@quaco ~]#

Quoted 1 line

Trying to figure out how to remove it via debugfs/tracefs

[root@quaco ~]# cat /sys/kernel/debug/tracing/dynamic_events p:probe_perf/map__map_ip /home/acme/bin/perf:0x000000000005f257 p:probe_perf/map__map_ip /home/acme/bin/perf:0x000000000005f2a3 p:probe_perf/map__map_ip /home/acme/bin/perf:0x00000000000fda24 p:probe_perf/map__map_ip /home/acme/bin/perf:0x00000000000fdacf p:probe_perf/map__map_ip /home/acme/bin/perf:0x00000000001028d0 p:probe_perf/map__map_ip /home/acme/bin/perf:0x000000000016dd90 [root@quaco ~]#

[root@quaco ~]# cat /sys/kernel/debug/tracing/kprobe_events [root@quaco ~]#

[root@quaco ~]# echo > /sys/kernel/debug/tracing/kprobe_events
[root@quaco ~]# perf probe -l
probe_perf:map__map_ip (on process_sample_event+6663@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on process_sample_event+6739@tools/perf/builtin-script.c in /home/acme/bin/perf) probe_perf:map__map_ip (on unwind_entry+436@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on machine__process_extra_kernel_map+31@util/machine.c in /home/acme/bin/perf) probe_perf:map__map_ip (on __maps__insert+192@util/map.c in /home/acme/bin/perf) probe_perf:map__map_ip (on dso__process_kernel_symbol+336@util/symbol-elf.c in /home/acme/bin/perf)
[root@quaco ~]#

Finally it works:

Quoted 9 lines

[root@quaco ~]# echo > /sys/kernel/debug/tracing/dynamic_events [root@quaco ~]# cat /sys/kernel/debug/tracing/dynamic_events [root@quaco ~]#
[root@quaco ~]# perf probe -l
[root@quaco ~]#

A workaround, now to find out why 'perf probe -d '*:*' and the other reported variants are failing...

Thank you for pointing it out. You can use '*' instead of '*:*' to list them all. Anyway, there seems an inssue when I introduced multiprobe support. I'll fix it soon.

Thanks,

Quoted 1 line

- Arnaldo

Signature · 1 line
Masami Hiramatsu <[email protected]>
#4 lore raw

Em Tue, Dec 03, 2019 at 07:18:54AM +0900, Masami Hiramatsu escreveu:

Quoted 10 lines

Hi Arnaldo,

Quoted 3 lines · On Mon, 2 Dec 2019 16:04:52 -0300 Arnaldo Carvalho de Melo <[email protected]>…

Em Mon, Dec 02, 2019 at 03:59:58PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 1 line

Em Fri, Nov 29, 2019 at 12:13:24PM -0300, Arnaldo Carvalho de Melo escreveu:

A workaround, now to find out why 'perf probe -d '*:*' and the other

Thank you for pointing it out. You can use '*' instead of '*:*' to list them all.

It didn't work with either '*' or '*:*' or any other variant.

Quoted 2 lines

Anyway, there seems an inssue when I introduced multiprobe support. I'll fix it soon.

Thanks!

- Arnaldo