tracing/boot: Support eprobe, fprobe, and tprobe events and add test framework
Subsystems (3)
Masami Hiramatsu (Google)
maintainerv2 cover tracing/boot: Support eprobe, fprobe, and tprobe events and add test framework
Hi,
This is 2nd version of series to extend kernel boot-time tracing to support newly added dynamic event types (eprobes, fprobes, and tracepoint probes) and introduces an automated QEMU-based test suite under tools/tracing/boottime/.
Previous version is here:
https://lore.kernel.org/all/178597841107.123558.13065821394773752510.stgit@devnote2/
This version fixes some issues reported by Sashiko [1], simplifying code, and improve test program.
[1/2]
- Fix raw command detection logic for eprobes, fprobes, and tprobes by requiring ':' or isspace() after type prefix.
- Consolidate duplicate loop logic into trace_boot_add_probe_event() helper function.
[2/2]
- Copy shared library dependencies using ldd if busybox is dynamically linked.
- Wrap QEMU execution with timeout command and add -T/--timeout CLI option.
- Add pre-flight checks for busybox architecture compatibility and cpio tool in run_boottime_test.sh.
- Fix POSIX syntax compatibility and allow bypassing HOST_ARCH skip when custom kernel/QEMU parameters are passed.
- Add boottime/run_boottime_test.sh to TEST_PROGS_EXTENDED in selftests/ftrace/Makefile.
[1] https://sashiko.dev/#/patchset/178597841107.123558.13065821394773752510.stgit%40devnote2
Thanks,
---
base-commit: ea694cdd92b6bfa123b3f5c563e47b4cf968046a
Masami Hiramatsu (Google) (2):
tracing/boot: Add support for eprobe, fprobe, and tprobe events selftests/ftrace: Add boot-time tracing test framework
Documentation/trace/boottime-trace.rst | 18 + kernel/trace/trace_boot.c | 86 +++++++ kernel/trace/trace_dynevent.c | 5 tools/testing/selftests/ftrace/Makefile | 4 tools/testing/selftests/ftrace/boottime/Makefile | 10 + tools/testing/selftests/ftrace/boottime/README | 58 ++++ .../ftrace/boottime/bootconfigs/01-kprobe.bconf | 5 .../ftrace/boottime/bootconfigs/02-synth.bconf | 4 .../ftrace/boottime/bootconfigs/03-eprobe.bconf | 4 .../ftrace/boottime/bootconfigs/04-fprobe.bconf | 4 .../ftrace/boottime/bootconfigs/05-tprobe.bconf | 4 .../ftrace/boottime/bootconfigs/06-instance.bconf | 5 .../selftests/ftrace/boottime/run_boottime_test.sh | 264 ++++++++++++++++++++ .../selftests/ftrace/boottime/tests/01-kprobe.sh | 18 + .../selftests/ftrace/boottime/tests/02-synth.sh | 18 + .../selftests/ftrace/boottime/tests/03-eprobe.sh | 18 + .../selftests/ftrace/boottime/tests/04-fprobe.sh | 18 + .../selftests/ftrace/boottime/tests/05-tprobe.sh | 18 + .../selftests/ftrace/boottime/tests/06-instance.sh | 23 ++ tools/testing/selftests/ftrace/config | 4 20 files changed, 583 insertions(+), 5 deletions(-) create mode 100644 tools/testing/selftests/ftrace/boottime/Makefile create mode 100644 tools/testing/selftests/ftrace/boottime/README create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf create mode 100755 tools/testing/selftests/ftrace/boottime/run_boottime_test.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/02-synth.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/06-instance.sh
--
Masami Hiramatsu (Google) <[email protected]>
Masami Hiramatsu (Google)
maintainerPATCH v2 1/2 tracing/boot: Add support for eprobe, fprobe, and tprobe events
From: Masami Hiramatsu (Google) <[email protected]>
Boot-time tracing currently supports kprobe-events and synthetic-events under per-event configuration options.
Extend boot-time tracing to support newly added dynamic probe types:
- event probes (eprobe) under the "eprobes" event group
- function probes (fprobe) under the "fprobes" event group
- tracepoint probes (tprobe) under the "tracepoints" or "tprobes" event group
To support this cleanly, update dyn_event_create() in trace_dynevent.c so that passing NULL as the type parameter delegates to create_dyn_event(), allowing generic creation of any registered dynamic event type from a raw command string.
Update Documentation/trace/boottime-trace.rst accordingly to describe the new per-event bootconfig options.
---
Changes in v2:
- Fix raw command detection logic for eprobes, fprobes, and tprobes by requiring ':' or isspace() after type prefix.
- Consolidate duplicate loop logic into trace_boot_add_probe_event() helper function.
--- Documentation/trace/boottime-trace.rst | 18 ++++++- kernel/trace/trace_boot.c | 86 ++++++++++++++++++++++++++++++++ kernel/trace/trace_dynevent.c | 5 ++ 3 files changed, 106 insertions(+), 3 deletions(-)
Documentation/trace/boottime-trace.rst +15 -3
@@ -121,9 +121,21 @@ ftrace.[instance.INSTANCE.]event.synthetic.EVENT.fields = FIELD[, FIELD2[...]] Defines new synthetic event with FIELDs. Each field should be "type varname". -Note that kprobe and synthetic event definitions can be written under -instance node, but those are also visible from other instances. So please -take care for event name conflict. +ftrace.[instance.INSTANCE.]event.eprobes.EVENT.probes = PROBE[, PROBE2[...]] + Defines new event probe based on PROBEs. This option is available only + for the event which group name is "eprobes". + +ftrace.[instance.INSTANCE.]event.fprobes.EVENT.probes = PROBE[, PROBE2[...]] + Defines new fprobe event based on PROBEs. This option is available only + for the event which group name is "fprobes". + +ftrace.[instance.INSTANCE.]event.tracepoints.EVENT.probes = PROBE[, PROBE2[...]] + Defines new tracepoint probe based on PROBEs. This option is available only + for the event which group name is "tracepoints" or "tprobes". + +Note that dynamic event definitions can be written under instance node, but +those are also visible from other instances. So please take care for event +name conflict. Ftrace Histogram Options ------------------------
kernel/trace/trace_boot.c +86 -0
104 lines not on this page · raw message
kernel/trace/trace_dynevent.c +5 -0
@@ -116,6 +116,8 @@ int dyn_event_release(const char *raw_command, struct dyn_event_operations *typereturn ret; } +static int create_dyn_event (const char *raw_command); +/* * Locked version of event creation. The event creation must be protected by * dyn_event_ops_mutex because of protecting trace_probe_log.
@@ -124,6 +126,9 @@ int dyn_event_create(const char *raw_command, struct dyn_event_operations *type) {int ret; +if (!type) +return create_dyn_event (raw_command); +mutex_lock (&dyn_event_ops_mutex); ret = type->create (raw_command);mutex_unlock (&dyn_event_ops_mutex);
Masami Hiramatsu (Google)
maintainerPATCH v2 2/2 selftests/ftrace: Add boot-time tracing test framework
In reply to Masami Hiramatsu (Google)From: Masami Hiramatsu (Google) <[email protected]>
Add an automated test framework under tools/testing/selftests/ftrace/boottime/ to verify boot-time tracing configuration parsing.
The test runner (run_boottime_test.sh) sources ktap_helpers.sh to generate output in TAP version 13 format. It builds a minimal initramfs with busybox, attaches bootconfig examples using tools/bootconfig/bootconfig, boots QEMU, and runs corresponding tracefs checker scripts.
---
Changes in v2:
- Copy shared library dependencies using ldd if busybox is dynamically linked.
- Wrap QEMU execution with timeout command and add -T/--timeout CLI option.
- Add pre-flight checks for busybox architecture compatibility and cpio tool in run_boottime_test.sh.
- Fix POSIX syntax compatibility and allow bypassing HOST_ARCH skip when custom kernel/QEMU parameters are passed.
- Add boottime/run_boottime_test.sh to TEST_PROGS_EXTENDED in selftests/ftrace/Makefile.
--- tools/testing/selftests/ftrace/Makefile | 4 tools/testing/selftests/ftrace/boottime/Makefile | 10 + tools/testing/selftests/ftrace/boottime/README | 58 ++++ .../ftrace/boottime/bootconfigs/01-kprobe.bconf | 5 .../ftrace/boottime/bootconfigs/02-synth.bconf | 4 .../ftrace/boottime/bootconfigs/03-eprobe.bconf | 4 .../ftrace/boottime/bootconfigs/04-fprobe.bconf | 4 .../ftrace/boottime/bootconfigs/05-tprobe.bconf | 4 .../ftrace/boottime/bootconfigs/06-instance.bconf | 5 .../selftests/ftrace/boottime/run_boottime_test.sh | 264 ++++++++++++++++++++ .../selftests/ftrace/boottime/tests/01-kprobe.sh | 18 + .../selftests/ftrace/boottime/tests/02-synth.sh | 18 + .../selftests/ftrace/boottime/tests/03-eprobe.sh | 18 + .../selftests/ftrace/boottime/tests/04-fprobe.sh | 18 + .../selftests/ftrace/boottime/tests/05-tprobe.sh | 18 + .../selftests/ftrace/boottime/tests/06-instance.sh | 23 ++ tools/testing/selftests/ftrace/config | 4 17 files changed, 477 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/ftrace/boottime/Makefile create mode 100644 tools/testing/selftests/ftrace/boottime/README create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf create mode 100644 tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf create mode 100755 tools/testing/selftests/ftrace/boottime/run_boottime_test.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/02-synth.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh create mode 100755 tools/testing/selftests/ftrace/boottime/tests/06-instance.sh
tools/testing/selftests/ftrace/Makefile +2 -2
@@ -1,9 +1,9 @@ # SPDX-License-Identifier: GPL-2.0 all: -TEST_PROGS_EXTENDED := ftracetest +TEST_PROGS_EXTENDED := ftracetest boottime/run_boottime_test.sh TEST_PROGS := ftracetest-ktap -TEST_FILES := test.d settings +TEST_FILES := test.d settings boottime EXTRA_CLEAN := $(OUTPUT)/logs/* TEST_GEN_FILES := poll
tools/testing/selftests/ftrace/boottime/Makefile new file +10 -0
@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0 + +all: + +run_tests: + @./run_boottime_test.sh + +clean: + +.PHONY: all run_tests clean
tools/testing/selftests/ftrace/boottime/README new file +58 -0
58 lines not on this page · raw message
tools/testing/selftests/ftrace/boottime/bootconfigs/01-kprobe.bconf new file +5 -0
@@ -0,0 +1,5 @@ +ftrace.event.kprobes.vfs_read { + probes = "vfs_read $arg1 $arg2" + filter = "common_pid < 200" + enable +}
tools/testing/selftests/ftrace/boottime/bootconfigs/02-synth.bconf new file +4 -0
@@ -0,0 +1,4 @@ +ftrace.event.synthetic.boot_lat { + fields = "unsigned long id", "u64 delta" + enable +}
tools/testing/selftests/ftrace/boottime/bootconfigs/03-eprobe.bconf new file +4 -0
@@ -0,0 +1,4 @@ +ftrace.event.eprobes.ep_read { + probes = "syscalls/sys_enter_openat" + enable +}
tools/testing/selftests/ftrace/boottime/bootconfigs/04-fprobe.bconf new file +4 -0
@@ -0,0 +1,4 @@ +ftrace.event.fprobes.fp_read { + probes = "vfs_read" + enable +}
tools/testing/selftests/ftrace/boottime/bootconfigs/05-tprobe.bconf new file +4 -0
@@ -0,0 +1,4 @@ +ftrace.event.tracepoints.tp_sched { + probes = "sched_switch" + enable +}
tools/testing/selftests/ftrace/boottime/bootconfigs/06-instance.bconf new file +5 -0
@@ -0,0 +1,5 @@ +ftrace.instance.foo { + event.sched.sched_switch.enable + buffer_size = 1MB + tracing_on = 1 +}
tools/testing/selftests/ftrace/boottime/run_boottime_test.sh new file +264 -0
264 lines not on this page · raw message
tools/testing/selftests/ftrace/boottime/tests/01-kprobe.sh new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Check kprobe bootconfig settings on tracefs +TRACEDIR="/sys/kernel/tracing" + +if [ ! -d "$TRACEDIR/events/kprobes/vfs_read" ]; then + echo "FAIL: kprobe event kprobes/vfs_read does not exist" + exit 1 +fi + +ENABLE=$(cat "$TRACEDIR/events/kprobes/vfs_read/enable") +if [ "$ENABLE" != "1" ]; then + echo "FAIL: kprobe event kprobes/vfs_read is not enabled ($ENABLE)" + exit 1 +fi + +echo "PASS: 01-kprobe" +exit 0
tools/testing/selftests/ftrace/boottime/tests/02-synth.sh new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Check synthetic event bootconfig settings on tracefs +TRACEDIR="/sys/kernel/tracing" + +if [ ! -d "$TRACEDIR/events/synthetic/boot_lat" ]; then + echo "FAIL: synthetic event synthetic/boot_lat does not exist" + exit 1 +fi + +ENABLE=$(cat "$TRACEDIR/events/synthetic/boot_lat/enable") +if [ "$ENABLE" != "1" ]; then + echo "FAIL: synthetic event synthetic/boot_lat is not enabled ($ENABLE)" + exit 1 +fi + +echo "PASS: 02-synth" +exit 0
tools/testing/selftests/ftrace/boottime/tests/03-eprobe.sh new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Check eprobe event bootconfig settings on tracefs +TRACEDIR="/sys/kernel/tracing" + +if [ ! -d "$TRACEDIR/events/eprobes/ep_read" ]; then + echo "FAIL: eprobe event eprobes/ep_read does not exist" + exit 1 +fi + +ENABLE=$(cat "$TRACEDIR/events/eprobes/ep_read/enable") +if [ "$ENABLE" != "1" ]; then + echo "FAIL: eprobe event eprobes/ep_read is not enabled ($ENABLE)" + exit 1 +fi + +echo "PASS: 03-eprobe" +exit 0
tools/testing/selftests/ftrace/boottime/tests/04-fprobe.sh new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Check fprobe event bootconfig settings on tracefs +TRACEDIR="/sys/kernel/tracing" + +if [ ! -d "$TRACEDIR/events/fprobes/fp_read" ]; then + echo "FAIL: fprobe event fprobes/fp_read does not exist" + exit 1 +fi + +ENABLE=$(cat "$TRACEDIR/events/fprobes/fp_read/enable") +if [ "$ENABLE" != "1" ]; then + echo "FAIL: fprobe event fprobes/fp_read is not enabled ($ENABLE)" + exit 1 +fi + +echo "PASS: 04-fprobe" +exit 0
tools/testing/selftests/ftrace/boottime/tests/05-tprobe.sh new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Check tracepoint probe event bootconfig settings on tracefs +TRACEDIR="/sys/kernel/tracing" + +if [ ! -d "$TRACEDIR/events/tracepoints/tp_sched" ]; then + echo "FAIL: tracepoint probe event tracepoints/tp_sched does not exist" + exit 1 +fi + +ENABLE=$(cat "$TRACEDIR/events/tracepoints/tp_sched/enable") +if [ "$ENABLE" != "1" ]; then + echo "FAIL: tracepoint probe event tracepoints/tp_sched is not enabled ($ENABLE)" + exit 1 +fi + +echo "PASS: 05-tprobe" +exit 0
tools/testing/selftests/ftrace/boottime/tests/06-instance.sh new file +23 -0
@@ -0,0 +1,23 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Check instance bootconfig settings on tracefs +TRACEDIR="/sys/kernel/tracing" + +if [ ! -d "$TRACEDIR/instances/foo" ]; then + echo "FAIL: trace instance foo does not exist" + exit 1 +fi + +if [ ! -d "$TRACEDIR/instances/foo/events/sched/sched_switch" ]; then + echo "FAIL: event sched_switch does not exist in instance foo" + exit 1 +fi + +ENABLE=$(cat "$TRACEDIR/instances/foo/events/sched/sched_switch/enable") +if [ "$ENABLE" != "1" ]; then + echo "FAIL: event sched_switch is not enabled in instance foo ($ENABLE)" + exit 1 +fi + +echo "PASS: 06-instance" +exit 0
tools/testing/selftests/ftrace/config +4 -0
@@ -1,3 +1,5 @@ +CONFIG_BOOT_CONFIG=y +CONFIG_BOOTTIME_TRACING=y CONFIG_BPF_SYSCALL=y CONFIG_DEBUG_INFO_BTF=y CONFIG_DEBUG_INFO_DWARF4=y
@@ -8,6 +10,7 @@ CONFIG_FTRACE=y CONFIG_FTRACE_SYSCALLS=y CONFIG_FUNCTION_GRAPH_RETVAL=y CONFIG_FUNCTION_PROFILER=y +CONFIG_FUNCTION_TRACER=y CONFIG_HIST_TRIGGERS=y CONFIG_IRQSOFF_TRACER=y CONFIG_KALLSYMS_ALL=y
@@ -24,6 +27,7 @@ CONFIG_SAMPLE_TRACE_EVENTS=m CONFIG_SAMPLE_TRACE_PRINTK=m CONFIG_SCHED_TRACER=y CONFIG_STACK_TRACER=y +CONFIG_SYNTH_EVENTS=y CONFIG_TRACER_SNAPSHOT=y CONFIG_UPROBES=y CONFIG_UPROBE_EVENTS=y