tracing: Add a way to filter function addresses to function names
Subsystems (3)
Steven Rostedt
maintainercover tracing: Add a way to filter function addresses to function names
There's been several times where an event records a function address in its field and I needed to filter on that address for a specific function name. It required looking up the function in kallsyms, finding its size, and doing a compare of "field >= function_start && field < function_end".
But this would change from boot to boot and is unreliable in scripts. Also, it is useful to have this at boot up, where the addresses will not be known. For example, on the boot command line:
trace_trigger="initcall_finish.traceoff if func.function == acpi_init"
To implement this, add a ".function" prefix, that will check that the field is of size long, and the only operations allowed (so far) are "==" and "!=".
Changes since v1: https://lore.kernel.org/linux-trace-kernel/[email protected]/
- Fix commit log "initcall_finish.function" to "func.function" (Ross Zwisler)
- Fixed processing of address (Ross Zwisler)
- Added selftest (Masami Hiramatsu)
- Just use "trigger" instead of "strsep(&trigger, "")" (Zheng Yejian)
Steven Rostedt (Google) (2):
tracing: Add a way to filter function addresses to function names tracing/selftests: Add test for event filtering on function name
----
Documentation/trace/events.rst | 12 +++ kernel/trace/trace_events.c | 2 +- kernel/trace/trace_events_filter.c | 93 +++++++++++++++++++++- .../ftrace/test.d/filter/event-filter-function.tc | 58 ++++++++++++++ 4 files changed, 163 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc
Steven Rostedt
maintainerPATCH 1/2 tracing: Add a way to filter function addresses to function names
From: "Steven Rostedt (Google)" <[email protected]>
There's been several times where an event records a function address in its field and I needed to filter on that address for a specific function name. It required looking up the function in kallsyms, finding its size, and doing a compare of "field >= function_start && field < function_end".
But this would change from boot to boot and is unreliable in scripts. Also, it is useful to have this at boot up, where the addresses will not be known. For example, on the boot command line:
trace_trigger="initcall_finish.traceoff if func.function == acpi_init"
To implement this, add a ".function" prefix, that will check that the field is of size long, and the only operations allowed (so far) are "==" and "!=".
--- Documentation/trace/events.rst | 12 ++++ kernel/trace/trace_events.c | 2 +- kernel/trace/trace_events_filter.c | 93 +++++++++++++++++++++++++++++- 3 files changed, 105 insertions(+), 2 deletions(-)
Documentation/trace/events.rst +12 -0
@@ -207,6 +207,18 @@ field name:: As the kernel will have to know how to retrieve the memory that the pointer is at from user space. +You can convert any long type to a function address and search by function name:: + + call_site.function == security_prepare_creds + +The above will filter when the field "call_site" falls on the address within +"security_prepare_creds". That is, it will compare the value of "call_site" and +the filter will return true if it is greater than or equal to the start of +the function "security_prepare_creds" and less than the end of that function. + +The ".function" postfix can only be attached to values of size long, and can only +be compared with "==" or "!=". + 5.2 Setting filters -------------------
kernel/trace/trace_events.c +1 -1
@@ -2822,7 +2822,7 @@ static __init int setup_trace_triggers(char *str)if (!trigger)break ; bootup_triggers[i].event =strsep (&trigger,"." ); - bootup_triggers[i].trigger =strsep (&trigger,"." ); + bootup_triggers[i].trigger = trigger;if (!bootup_triggers[i].trigger)break ; }
kernel/trace/trace_events_filter.c +92 -1
143 lines not on this page · raw message
Signature · 1 line
2.35.1
Quoted 11 lines · Steven Rostedt
From: "Steven Rostedt (Google)" <[email protected]>
There's been several times where an event records a function address in its field and I needed to filter on that address for a specific function name. It required looking up the function in kallsyms, finding its size, and doing a compare of "field >=
function_start&& field <function_end".But this would change from boot to boot and is unreliable in scripts. Also, it is useful to have this at boot up, where the addresses will not be known. For example, on the boot command line:
Quoted 7 lines
trace_trigger="initcall_finish.traceoff if func.function ==
acpi_init"To implement this, add a ".function" prefix, that will check that the field is of size long, and the only operations allowed (so far) are "==" and "!=".
Signed-off-by Steven Rostedt (Google) <[email protected]>
Steven Rostedt
maintainerPATCH 2/2 tracing/selftests: Add test for event filtering on function name
In reply to Steven RostedtFrom: "Steven Rostedt (Google)" <[email protected]>
With the new filter logic of passing in the name of a function to match an instruction pointer (or the address of the function), add a test to make sure that it is functional.
This is also the first test to test plain filtering. The filtering has been tested via the trigger logic, which uses the same code, but there was nothing to test just the event filter, so this test is the first to add such a case.
--- .../test.d/filter/event-filter-function.tc | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc
tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc new file +58 -0
@@ -0,0 +1,58 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event filter function - test event filtering on functions +# requires: set_event events/kmem/kmem_cache_free/filter +# flags: instance + +fail() { #msg + echo $1 + exit_fail +} + +echo "Test event filter function name" +echo 0 > tracing_on +echo 0 > events/enable +echo > trace +echo 'call_site.function == exit_mmap' > events/kmem/kmem_cache_free/filter +echo 1 > events/kmem/kmem_cache_free/enable +echo 1 > tracing_on +ls > /dev/null +echo 0 > events/kmem/kmem_cache_free/enable + +hitcnt=`grep kmem_cache_free trace| grep exit_mmap | wc -l` +misscnt=`grep kmem_cache_free trace| grep -v exit_mmap | wc -l` + +if [ $hitcnt -eq 0 ]; then + exit_fail +fi + +if [ $misscnt -gt 0 ]; then + exit_fail +fi + +address=`grep ' exit_mmap$' /proc/kallsyms | cut -d' ' -f1` + +echo "Test event filter function address" +echo 0 > tracing_on +echo 0 > events/enable +echo > trace +echo "call_site.function == 0x$address" > events/kmem/kmem_cache_free/filter +echo 1 > events/kmem/kmem_cache_free/enable +echo 1 > tracing_on +sleep 1 +echo 0 > events/kmem/kmem_cache_free/enable + +hitcnt=`grep kmem_cache_free trace| grep exit_mmap | wc -l` +misscnt=`grep kmem_cache_free trace| grep -v exit_mmap | wc -l` + +if [ $hitcnt -eq 0 ]; then + exit_fail +fi + +if [ $misscnt -gt 0 ]; then + exit_fail +fi + +reset_events_filter + +exit 0
Signature · 1 line
2.35.1
Re: [PATCH 2/2] tracing/selftests: Add test for event filtering on function name
Quoted 11 lines · Steven Rostedt
From: "Steven Rostedt (Google)" <[email protected]>
With the new filter logic of passing in the name of a function to match an instruction pointer (or the address of the function), add a test to make sure that it is functional.
This is also the first test to test plain filtering. The filtering has been tested via the trigger logic, which uses the same code, but there was nothing to test just the event filter, so this test is the first to add such a case.
Quoted 5 lines
Cc Shuah Khan <[email protected]>Cc Shuah Khan <[email protected]>Suggested-by Masami Hiramatsu (Google) <[email protected]>Signed-off-by Steven Rostedt (Google) <[email protected]>
Shuah Khan
maintainerRe: [PATCH 2/2] tracing/selftests: Add test for event filtering on function name
Quoted 12 lines · Ross Zwisler
Quoted 11 lines · Steven Rostedt
From: "Steven Rostedt (Google)" <[email protected]>
With the new filter logic of passing in the name of a function to match an instruction pointer (or the address of the function), add a test to make sure that it is functional.
This is also the first test to test plain filtering. The filtering has been tested via the trigger logic, which uses the same code, but there was nothing to test just the event filter, so this test is the first to add such a case.
Quoted 7 lines
Quoted 5 lines
Cc Shuah Khan <[email protected]>Cc Shuah Khan <[email protected]>Suggested-by Masami Hiramatsu (Google) <[email protected]>Signed-off-by Steven Rostedt (Google) <[email protected]>Reviewed-by Ross Zwisler <[email protected]>
Thank you both. I will apply this after rc1 comes out.
thanks,
-- Shuah
Steven Rostedt
maintainerRe: [PATCH 2/2] tracing/selftests: Add test for event filtering on function name
Quoted 19 lines · On Mon, 19 Dec 2022 15:11:39 -0700 Shuah Khan <[email protected]> wrote:
Quoted 18 lines · Ross Zwisler
Quoted 16 lines · Steven Rostedt
From: "Steven Rostedt (Google)" <[email protected]>
With the new filter logic of passing in the name of a function to match an instruction pointer (or the address of the function), add a test to make sure that it is functional.
This is also the first test to test plain filtering. The filtering has been tested via the trigger logic, which uses the same code, but there was nothing to test just the event filter, so this test is the first to add such a case.
Cc Shuah Khan <[email protected]>Cc Shuah Khan <[email protected]>Suggested-by Masami Hiramatsu (Google) <[email protected]>Signed-off-by Steven Rostedt (Google) <[email protected]>
Quoted 3 lines
Quoted 1 line
Reviewed-by Ross Zwisler <[email protected]>Thank you both. I will apply this after rc1 comes out.
It's dependent on the first patch.
-- Steve
Re: [PATCH 2/2] tracing/selftests: Add test for event filtering on function name
Quoted 22 lines · Steven Rostedt
Quoted 19 lines · On Mon, 19 Dec 2022 15:11:39 -0700 Shuah Khan <[email protected]> wrote:
Quoted 18 lines · Ross Zwisler
Quoted 16 lines · Steven Rostedt
From: "Steven Rostedt (Google)" <[email protected]>
With the new filter logic of passing in the name of a function to match an instruction pointer (or the address of the function), add a test to make sure that it is functional.
This is also the first test to test plain filtering. The filtering has been tested via the trigger logic, which uses the same code, but there was nothing to test just the event filter, so this test is the first to add such a case.
Cc Shuah Khan <[email protected]>Cc Shuah Khan <[email protected]>Suggested-by Masami Hiramatsu (Google) <[email protected]>Signed-off-by Steven Rostedt (Google) <[email protected]>
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Reviewed-by Ross Zwisler <[email protected]>Thank you both. I will apply this after rc1 comes out.
It's dependent on the first patch.
In which case,
thanks,
-- Shuah
Steven Rostedt
maintainerRe: [PATCH 2/2] tracing/selftests: Add test for event filtering on function name
Quoted 25 lines · On Mon, 19 Dec 2022 17:20:39 -0700 Shuah Khan <[email protected]> wrote:
Quoted 24 lines · Steven Rostedt
Quoted 21 lines · On Mon, 19 Dec 2022 15:11:39 -0700 Shuah Khan <[email protected]> wrote:
Quoted 19 lines · Ross Zwisler
Quoted 11 lines · Steven Rostedt
From: "Steven Rostedt (Google)" <[email protected]>
With the new filter logic of passing in the name of a function to match an instruction pointer (or the address of the function), add a test to make sure that it is functional.
This is also the first test to test plain filtering. The filtering has been tested via the trigger logic, which uses the same code, but there was nothing to test just the event filter, so this test is the first to add such a case.
Quoted 5 lines
Cc Shuah Khan <[email protected]>Cc Shuah Khan <[email protected]>Suggested-by Masami Hiramatsu (Google) <[email protected]>Signed-off-by Steven Rostedt (Google) <[email protected]>Reviewed-by Ross Zwisler <[email protected]>
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
Thank you both. I will apply this after rc1 comes out.
It's dependent on the first patch.
In which case,
Acked-by Shuah Khan <[email protected]>
Thanks Shuah!
-- Steve