Skip to contentSkip to conversation

perf: ftrace enhancement

PATCH Superseded Changbin Du 2 participants 17 messages Jul 10, 2020 – Jul 11, 2020 perf events

v4 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v4 cover perf: ftrace enhancement

The perf has basic kernel ftrace support but lack support of most tracing options. This serias is target to enhance the perf ftrace functionality so that we can make full use of kernel ftrace with perf.

In general, this serias be cataloged into two main changes:
1) Improve usability of existing functions. For example, we don't need to type extra option to select the tracer.
2) Add new options to support all other ftrace functions.

Here is a glance of all ftrace functions with this serias:

$ sudo perf ftrace -h

Usage: perf ftrace [<options>] [<command>]
or: perf ftrace [<options>] -- <command> [<options>]

-a, --all-cpus system-wide collection from all CPUs
-C, --cpu <cpu> list of cpus to monitor
-D, --delay <n> ms to wait before starting tracing after program start
-F, --funcs Show available functions to filter
-G, --graph-funcs <func>
trace given functions using function_graph tracer
-g, --nograph-funcs <func>
Set nograph filter on given functions
-m, --buffer-size <size>
size of per cpu buffer
-N, --notrace-funcs <func>
do not trace given functions
-p, --pid <pid> trace on existing process id
-t, --tid <tid> trace on existing thread id (exclusive to --pid)
-T, --trace-funcs <func>
trace given functions using function tracer
-t, --tracer <tracer>
tracer to use: function or function_graph (This option is deprecated)
-v, --verbose be more verbose
--func-opts <options>
function tracer options, available options: call-graph,irq-info
--graph-opts <options>
graph tracer options, available options: nosleep-time,noirqs,verbose,thresh=<n>,depth=<n>
--inherit trace children processes

v4:
o add util/parse-sublevel-options.c
O remove -D/--graph-depth
v3:
o add --func-opts and --graph-opts to set tracer specific options.
o support units as a suffix for option '-m/--buffer-size'.
v2:
o patches for option '-u/--userstacktrace' and '--no-pager' are dropped.
o update all related perf documentation.
o rename some options. Now all funcgraph tracer options are prefixed with '--graph-', while all function tracer options are prefixed with '--func-'.
o mark old options deprecated instead of removing them.

Changbin Du (17):
perf ftrace: select function/function_graph tracer automatically perf ftrace: add option '-F/--funcs' to list available functions perf ftrace: add option -t/--tid to filter by thread id perf ftrace: factor out function write_tracing_file_int() perf ftrace: add option '-m/--buffer-size' to set per-cpu buffer size perf ftrace: show trace column header
perf ftrace: add option '--inherit' to trace children processes
perf: util: add general function to parse sublevel options perf ftrace: add support for tracing option 'func_stack_trace' perf ftrace: add support for trace option sleep-time
perf ftrace: add support for trace option funcgraph-irqs perf ftrace: add support for tracing option 'irq-info' perf ftrace: add option 'verbose' to show more info for graph tracer perf ftrace: add support for trace option tracing_thresh
perf: ftrace: allow set graph depth by '--graph-opts'
perf ftrace: add option -D/--delay to delay tracing
perf ftrace: add change log

 tools/perf/Documentation/perf-config.txt |   5 -
 tools/perf/Documentation/perf-ftrace.txt |  37 ++-
 tools/perf/builtin-ftrace.c              | 367 ++++++++++++++++++++++-
 tools/perf/util/Build                    |   1 +
 tools/perf/util/debug.c                  |  61 ++--
 tools/perf/util/parse-sublevel-options.c |  63 ++++
 tools/perf/util/parse-sublevel-options.h |  11 +
 7 files changed, 477 insertions(+), 68 deletions(-)
 create mode 100644 tools/perf/util/parse-sublevel-options.c
 create mode 100644 tools/perf/util/parse-sublevel-options.h
Signature · 1 line
2.25.1
#1 lore raw

PATCH v4 1/17 perf ftrace: select function/function_graph tracer automatically

The '-g/-G' options have already implied function_graph tracer should be used instead of function tracer. So the extra option '--tracer' can be killed.

This patch changes the behavior as below:
- By default, function tracer is used.
- If '-g' or '-G' option is on, then function_graph tracer is used.
- The perf configuration item 'ftrace.tracer' is marked as deprecated.
- The option '--tracer' is marked as deprecated.

Here are some examples.

This will start tracing all functions using function tracer:

  $ sudo perf ftrace

This will trace all functions using function graph tracer:

  $ sudo perf ftrace -G

This will trace function vfs_read using function graph tracer:

  $ sudo perf ftrace -G vfs_read

---
v3: remove default '*' for -G/-T.

---
 tools/perf/Documentation/perf-config.txt |  5 -----
 tools/perf/Documentation/perf-ftrace.txt |  2 +-
 tools/perf/builtin-ftrace.c              | 15 ++++++++++-----
 3 files changed, 11 insertions(+), 11 deletions(-)
tools/perf/Documentation/perf-config.txt +0 -5
index c7d3df5798e2..a25fee7de3b2 100644
@@ -612,11 +612,6 @@ trace.*::
 		"libbeauty", the default, to use the same argument beautifiers used in the
 		strace-like sys_enter+sys_exit lines.
 
-ftrace.*::
-	ftrace.tracer::
-		Can be used to select the default tracer. Possible values are
-		'function' and 'function_graph'.
-
 llvm.*::
 	llvm.clang-path::
 		Path to clang. If omit, search it from $PATH.
tools/perf/Documentation/perf-ftrace.txt +1 -1
index b80c84307dc9..952e46669168 100644
@@ -24,7 +24,7 @@ OPTIONS
 
 -t::
 --tracer=::
-	Tracer to use: function_graph or function.
+	Tracer to use: function_graph or function. This option is deprecated.
 
 -v::
 --verbose=::
tools/perf/builtin-ftrace.c +10 -5
index 2bfc1b0db536..5f53da87040d 100644
@@ -27,7 +27,6 @@
 #include "util/cap.h"
 #include "util/config.h"
 
-#define DEFAULT_TRACER  "function_graph"
 
 struct perf_ftrace {
 	struct evlist		*evlist;
@@ -419,6 +418,7 @@ static int perf_ftrace_config(const char *var, const char *value, void *cb)
 	if (strcmp(var, "ftrace.tracer"))
 		return -1;
 
+	pr_warning("Configuration ftrace.tracer is deprecated\n");
 	if (!strcmp(value, "function_graph") ||
 	    !strcmp(value, "function")) {
 		ftrace->tracer = value;
@@ -459,7 +459,7 @@ int cmd_ftrace(int argc, const char **argv)
 {
 	int ret;
 	struct perf_ftrace ftrace = {
-		.tracer = DEFAULT_TRACER,
+		.tracer = "function",
 		.target = { .uid = UINT_MAX, },
 	};
 	const char * const ftrace_usage[] = {
@@ -469,7 +469,7 @@ int cmd_ftrace(int argc, const char **argv)
 	};
 	const struct option ftrace_options[] = {
 	OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
-		   "tracer to use: function_graph(default) or function"),
+		   "tracer to use: function or function_graph (This option is deprecated)"),
 	OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
 		   "trace on existing process id"),
 	OPT_INCR('v', "verbose", &verbose,
@@ -479,11 +479,13 @@ int cmd_ftrace(int argc, const char **argv)
 	OPT_STRING('C', "cpu", &ftrace.target.cpu_list, "cpu",
 		    "list of cpus to monitor"),
 	OPT_CALLBACK('T', "trace-funcs", &ftrace.filters, "func",
-		     "trace given functions only", parse_filter_func),
+		     "trace given functions using function tracer",
+		     parse_filter_func),
 	OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
 		     "do not trace given functions", parse_filter_func),
 	OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
-		     "Set graph filter on given functions", parse_filter_func),
+		     "trace given functions using function_graph tracer",
+		     parse_filter_func),
 	OPT_CALLBACK('g', "nograph-funcs", &ftrace.nograph_funcs, "func",
 		     "Set nograph filter on given functions", parse_filter_func),
 	OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
@@ -505,6 +507,9 @@ int cmd_ftrace(int argc, const char **argv)
 	if (!argc && target__none(&ftrace.target))
 		ftrace.target.system_wide = true;
 
+	if (!list_empty(&ftrace.graph_funcs) || !list_empty(&ftrace.nograph_funcs))
+		ftrace.tracer = "function_graph";
+
 	ret = target__validate(&ftrace.target);
 	if (ret) {
 		char errbuf[512];
Signature · 1 line
2.25.1
maintainer
#2 lore raw

Re: [PATCH v4 01/17] perf ftrace: select function/function_graph tracer automatically

Hello,

Quoted 11 lines · Changbin Du

The '-g/-G' options have already implied function_graph tracer should be used instead of function tracer. So the extra option '--tracer' can be killed.

This patch changes the behavior as below:
- By default, function tracer is used.
- If '-g' or '-G' option is on, then function_graph tracer is used.
- The perf configuration item 'ftrace.tracer' is marked as deprecated.
- The option '--tracer' is marked as deprecated.

Quoted 7 lines

Here are some examples.

This will start tracing all functions using function tracer:

  $ sudo perf ftrace

This will trace all functions using function graph tracer:

  $ sudo perf ftrace -G

If you removed the default value, this doesn't work, right?

Thanks
Namhyung

Quoted 110 lines

This will trace function vfs_read using function graph tracer:

  $ sudo perf ftrace -G vfs_read

---
v3: remove default '*' for -G/-T.

---
 tools/perf/Documentation/perf-config.txt |  5 -----
 tools/perf/Documentation/perf-ftrace.txt |  2 +-
 tools/perf/builtin-ftrace.c              | 15 ++++++++++-----
 3 files changed, 11 insertions(+), 11 deletions(-)
tools/perf/Documentation/perf-config.txt +0 -5
index c7d3df5798e2..a25fee7de3b2 100644
@@ -612,11 +612,6 @@ trace.*::
                "libbeauty", the default, to use the same argument beautifiers used in the
                strace-like sys_enter+sys_exit lines.
 
-ftrace.*::
-       ftrace.tracer::
-               Can be used to select the default tracer. Possible values are
-               'function' and 'function_graph'.
-
 llvm.*::
        llvm.clang-path::
                Path to clang. If omit, search it from $PATH.
tools/perf/Documentation/perf-ftrace.txt +1 -1
index b80c84307dc9..952e46669168 100644
@@ -24,7 +24,7 @@ OPTIONS
 
 -t::
 --tracer=::
-       Tracer to use: function_graph or function.
+       Tracer to use: function_graph or function. This option is deprecated.
 
 -v::
 --verbose=::
tools/perf/builtin-ftrace.c +10 -6
index 2bfc1b0db536..5f53da87040d 100644
@@ -27,7 +27,6 @@
 #include "util/cap.h"
 #include "util/config.h"
 
-#define DEFAULT_TRACER  "function_graph"
 
 struct perf_ftrace {
        struct evlist           *evlist;
@@ -419,6 +418,7 @@ static int perf_ftrace_config(const char *var, const char *value, void *cb)
        if (strcmp(var, "ftrace.tracer"))
                return -1;
 
+       pr_warning("Configuration ftrace.tracer is deprecated\n");
        if (!strcmp(value, "function_graph") ||
            !strcmp(value, "function")) {
                ftrace->tracer = value;
@@ -459,7 +459,7 @@ int cmd_ftrace(int argc, const char **argv)
 {
        int ret;
        struct perf_ftrace ftrace = {
-               .tracer = DEFAULT_TRACER,
+               .tracer = "function",
                .target = { .uid = UINT_MAX, },
        };
        const char * const ftrace_usage[] = {
@@ -469,7 +469,7 @@ int cmd_ftrace(int argc, const char **argv)
        };
        const struct option ftrace_options[] = {
        OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
-                  "tracer to use: function_graph(default) or function"),
+                  "tracer to use: function or function_graph (This option is deprecated)"),
        OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
                   "trace on existing process id"),
        OPT_INCR('v', "verbose", &verbose,
@@ -479,11 +479,13 @@ int cmd_ftrace(int argc, const char **argv)
        OPT_STRING('C', "cpu", &ftrace.target.cpu_list, "cpu",
                    "list of cpus to monitor"),
        OPT_CALLBACK('T', "trace-funcs", &ftrace.filters, "func",
-                    "trace given functions only", parse_filter_func),
+                    "trace given functions using function tracer",
+                    parse_filter_func),
        OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
                     "do not trace given functions", parse_filter_func),
        OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
-                    "Set graph filter on given functions", parse_filter_func),
+                    "trace given functions using function_graph tracer",
+                    parse_filter_func),
        OPT_CALLBACK('g', "nograph-funcs", &ftrace.nograph_funcs, "func",
                     "Set nograph filter on given functions", parse_filter_func),
        OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
@@ -505,6 +507,9 @@ int cmd_ftrace(int argc, const char **argv)
        if (!argc && target__none(&ftrace.target))
                ftrace.target.system_wide = true;
 
+       if (!list_empty(&ftrace.graph_funcs) || !list_empty(&ftrace.nograph_funcs))
+               ftrace.tracer = "function_graph";
+
        ret = target__validate(&ftrace.target);
        if (ret) {
                char errbuf[512];
--

2.25.1

#3 lore raw

Re: [PATCH v4 01/17] perf ftrace: select function/function_graph tracer automatically

Quoted 16 lines · Namhyung Kim

Hello,

Quoted 13 lines · Changbin Du

The '-g/-G' options have already implied function_graph tracer should be used instead of function tracer. So the extra option '--tracer' can be killed.

This patch changes the behavior as below:
- By default, function tracer is used.
- If '-g' or '-G' option is on, then function_graph tracer is used.
- The perf configuration item 'ftrace.tracer' is marked as deprecated.
- The option '--tracer' is marked as deprecated.

Here are some examples.

Quoted 8 lines
Quoted 5 lines

This will start tracing all functions using function tracer:

  $ sudo perf ftrace

This will trace all functions using function graph tracer:

  $ sudo perf ftrace -G

If you removed the default value, this doesn't work, right?

yes, I forgot to update this commit message. Thanks.

Quoted 113 lines

Thanks
Namhyung

Quoted 110 lines

This will trace function vfs_read using function graph tracer:

  $ sudo perf ftrace -G vfs_read

---
v3: remove default '*' for -G/-T.

---
 tools/perf/Documentation/perf-config.txt |  5 -----
 tools/perf/Documentation/perf-ftrace.txt |  2 +-
 tools/perf/builtin-ftrace.c              | 15 ++++++++++-----
 3 files changed, 11 insertions(+), 11 deletions(-)
tools/perf/Documentation/perf-config.txt +0 -5
index c7d3df5798e2..a25fee7de3b2 100644
@@ -612,11 +612,6 @@ trace.*::
                "libbeauty", the default, to use the same argument beautifiers used in the
                strace-like sys_enter+sys_exit lines.
 
-ftrace.*::
-       ftrace.tracer::
-               Can be used to select the default tracer. Possible values are
-               'function' and 'function_graph'.
-
 llvm.*::
        llvm.clang-path::
                Path to clang. If omit, search it from $PATH.
tools/perf/Documentation/perf-ftrace.txt +1 -1
index b80c84307dc9..952e46669168 100644
@@ -24,7 +24,7 @@ OPTIONS
 
 -t::
 --tracer=::
-       Tracer to use: function_graph or function.
+       Tracer to use: function_graph or function. This option is deprecated.
 
 -v::
 --verbose=::
tools/perf/builtin-ftrace.c +10 -6
index 2bfc1b0db536..5f53da87040d 100644
@@ -27,7 +27,6 @@
 #include "util/cap.h"
 #include "util/config.h"
 
-#define DEFAULT_TRACER  "function_graph"
 
 struct perf_ftrace {
        struct evlist           *evlist;
@@ -419,6 +418,7 @@ static int perf_ftrace_config(const char *var, const char *value, void *cb)
        if (strcmp(var, "ftrace.tracer"))
                return -1;
 
+       pr_warning("Configuration ftrace.tracer is deprecated\n");
        if (!strcmp(value, "function_graph") ||
            !strcmp(value, "function")) {
                ftrace->tracer = value;
@@ -459,7 +459,7 @@ int cmd_ftrace(int argc, const char **argv)
 {
        int ret;
        struct perf_ftrace ftrace = {
-               .tracer = DEFAULT_TRACER,
+               .tracer = "function",
                .target = { .uid = UINT_MAX, },
        };
        const char * const ftrace_usage[] = {
@@ -469,7 +469,7 @@ int cmd_ftrace(int argc, const char **argv)
        };
        const struct option ftrace_options[] = {
        OPT_STRING('t', "tracer", &ftrace.tracer, "tracer",
-                  "tracer to use: function_graph(default) or function"),
+                  "tracer to use: function or function_graph (This option is deprecated)"),
        OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
                   "trace on existing process id"),
        OPT_INCR('v', "verbose", &verbose,
@@ -479,11 +479,13 @@ int cmd_ftrace(int argc, const char **argv)
        OPT_STRING('C', "cpu", &ftrace.target.cpu_list, "cpu",
                    "list of cpus to monitor"),
        OPT_CALLBACK('T', "trace-funcs", &ftrace.filters, "func",
-                    "trace given functions only", parse_filter_func),
+                    "trace given functions using function tracer",
+                    parse_filter_func),
        OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
                     "do not trace given functions", parse_filter_func),
        OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
-                    "Set graph filter on given functions", parse_filter_func),
+                    "trace given functions using function_graph tracer",
+                    parse_filter_func),
        OPT_CALLBACK('g', "nograph-funcs", &ftrace.nograph_funcs, "func",
                     "Set nograph filter on given functions", parse_filter_func),
        OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
@@ -505,6 +507,9 @@ int cmd_ftrace(int argc, const char **argv)
        if (!argc && target__none(&ftrace.target))
                ftrace.target.system_wide = true;
 
+       if (!list_empty(&ftrace.graph_funcs) || !list_empty(&ftrace.nograph_funcs))
+               ftrace.tracer = "function_graph";
+
        ret = target__validate(&ftrace.target);
        if (ret) {
                char errbuf[512];
--

2.25.1

Signature · 2 lines
Cheers,
Changbin Du
#4 lore raw

PATCH v4 2/17 perf ftrace: add option '-F/--funcs' to list available functions

In reply to Changbin Du

This adds an option '-F/--funcs' to list all available functions to trace, which is read from tracing file 'available_filter_functions'.

$ sudo ./perf ftrace -F | head
trace_initcall_finish_cb
initcall_blacklisted
do_one_initcall
do_one_initcall
trace_initcall_start_cb
run_init_process
try_to_run_init_process
match_dev_by_label
match_dev_by_uuid
rootfs_init_fs_context

---
v2: option name '-l/--list-functions' -> '-F/--funcs'

---
 tools/perf/Documentation/perf-ftrace.txt |  4 +++
 tools/perf/builtin-ftrace.c              | 43 ++++++++++++++++++++++++
 2 files changed, 47 insertions(+)
tools/perf/Documentation/perf-ftrace.txt +4 -0
index 952e46669168..d79560dea19f 100644
@@ -30,6 +30,10 @@ OPTIONS
 --verbose=::
         Verbosity level.
 
+-F::
+--funcs::
+        List all available functions to trace.
+
 -p::
 --pid=::
 	Trace on existing process id (comma separated list).
tools/perf/builtin-ftrace.c +43 -0

67 lines not on this page · raw message

Signature · 1 line
2.25.1
#5 lore raw

PATCH v4 3/17 perf ftrace: add option -t/--tid to filter by thread id

In reply to Changbin Du

This allows us to trace single thread instead of the whole process.

---
 tools/perf/Documentation/perf-ftrace.txt | 4 ++++
 tools/perf/builtin-ftrace.c              | 2 ++
 2 files changed, 6 insertions(+)
tools/perf/Documentation/perf-ftrace.txt +4 -0
index d79560dea19f..e204bf6d50d8 100644
@@ -38,6 +38,10 @@ OPTIONS
 --pid=::
 	Trace on existing process id (comma separated list).
 
+-t::
+--tid=::
+	Trace on existing thread id (comma separated list).
+
 -a::
 --all-cpus::
 	Force system-wide collection.  Scripts run without a <command>
tools/perf/builtin-ftrace.c +2 -0
index 244cc8e6bd60..1188b82c6541 100644
@@ -515,6 +515,8 @@ int cmd_ftrace(int argc, const char **argv)
 		    "Show available functions to filter"),
 	OPT_STRING('p', "pid", &ftrace.target.pid, "pid",
 		   "trace on existing process id"),
+	OPT_STRING('t', "tid", &ftrace.target.tid, "tid",
+		   "trace on existing thread id (exclusive to --pid)"),
 	OPT_INCR('v', "verbose", &verbose,
 		 "be more verbose"),
 	OPT_BOOLEAN('a', "all-cpus", &ftrace.target.system_wide,
Signature · 1 line
2.25.1
#6 lore raw

PATCH v4 4/17 perf ftrace: factor out function write_tracing_file_int()

In reply to Changbin Du

We will reuse this function later.

---
 tools/perf/builtin-ftrace.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
tools/perf/builtin-ftrace.c +12 -5
index 1188b82c6541..342861a1d152 100644
@@ -165,6 +165,17 @@ static int read_tracing_file_to_stdout(const char *name)
 	return ret;
 }
 
+static int write_tracing_file_int(const char *name, int value)
+{
+	char buf[16];
+
+	snprintf(buf, sizeof(buf), "%d", value);
+	if (write_tracing_file(name, buf) < 0)
+		return -1;
+
+	return 0;
+}
+
 static int reset_tracing_cpu(void);
 static void reset_tracing_filters(void);
 
@@ -295,8 +306,6 @@ static void reset_tracing_filters(void)
 
 static int set_tracing_depth(struct perf_ftrace *ftrace)
 {
-	char buf[16];
-
 	if (ftrace->graph_depth == 0)
 		return 0;
 
@@ -305,9 +314,7 @@ static int set_tracing_depth(struct perf_ftrace *ftrace)
 		return -1;
 	}
 
-	snprintf(buf, sizeof(buf), "%d", ftrace->graph_depth);
-
-	if (write_tracing_file("max_graph_depth", buf) < 0)
+	if (write_tracing_file_int("max_graph_depth", ftrace->graph_depth) < 0)
 		return -1;
 
 	return 0;
Signature · 1 line
2.25.1
#7 lore raw

PATCH v4 5/17 perf ftrace: add option '-m/--buffer-size' to set per-cpu buffer size

In reply to Changbin Du

This adds an option '-m/--buffer-size' to allow us set the size of per-cpu tracing buffer.

---
v2: support units as a suffix.

---
 tools/perf/Documentation/perf-ftrace.txt |  5 +++
 tools/perf/builtin-ftrace.c              | 56 +++++++++++++++++++++++-
 2 files changed, 60 insertions(+), 1 deletion(-)
tools/perf/Documentation/perf-ftrace.txt +5 -0
index e204bf6d50d8..98fe01d354d1 100644
@@ -56,6 +56,11 @@ OPTIONS
 	Ranges of CPUs are specified with -: 0-2.
 	Default is to trace on all online CPUs.
 
+-m::
+--buffer-size::
+	Set the size of per-cpu tracing buffer, <size> is expected to
+	be a number with appended unit character - B/K/M/G.
+
 -T::
 --trace-funcs=::
 	Only trace functions given by the argument.  Multiple functions
tools/perf/builtin-ftrace.c +55 -1

92 lines not on this page · raw message

Signature · 1 line
2.25.1
#8 lore raw

PATCH v4 6/17 perf ftrace: show trace column header

In reply to Changbin Du

This makes perf-ftrace display column header before printing trace.

  $ sudo perf ftrace
  # tracer: function
  #
  # entries-in-buffer/entries-written: 0/0   #P:8
  #
  #            TASK-PID     CPU#   TIMESTAMP  FUNCTION
  #              | |         |       |         |
             <...>-9246  [006]  10726.262760: mutex_unlock <-rb_simple_write
             <...>-9246  [006]  10726.262764: __fsnotify_parent <-vfs_write
             <...>-9246  [006]  10726.262765: fsnotify <-vfs_write
             <...>-9246  [006]  10726.262766: __sb_end_write <-vfs_write
             <...>-9246  [006]  10726.262767: fpregs_assert_state_consistent <-do_syscall_64
---
 tools/perf/builtin-ftrace.c | 3 +++
 1 file changed, 3 insertions(+)
tools/perf/builtin-ftrace.c +3 -0
index 348e2d960987..887e78b23a82 100644
@@ -430,6 +430,9 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
 	fcntl(trace_fd, F_SETFL, O_NONBLOCK);
 	pollfd.fd = trace_fd;
 
+	/* display column headers */
+	read_tracing_file_to_stdout("trace");
+
 	if (write_tracing_file("tracing_on", "1") < 0) {
 		pr_err("can't enable tracing\n");
 		goto out_close_fd;
Signature · 1 line
2.25.1
#9 lore raw

PATCH v4 7/17 perf ftrace: add option '--inherit' to trace children processes

In reply to Changbin Du

This adds an option '--inherit' to allow us trace children processes spawned by our target.

---
v2: option name '--trace-children' -> '--inherit'.

---
 tools/perf/Documentation/perf-ftrace.txt |  3 ++
 tools/perf/builtin-ftrace.c              | 38 ++++++++++++++++++++++++
 2 files changed, 41 insertions(+)
tools/perf/Documentation/perf-ftrace.txt +3 -0
index 98fe01d354d1..fd632bd9b2c1 100644
@@ -61,6 +61,9 @@ OPTIONS
 	Set the size of per-cpu tracing buffer, <size> is expected to
 	be a number with appended unit character - B/K/M/G.
 
+--inherit::
+	Trace children processes spawned by our target.
+
 -T::
 --trace-funcs=::
 	Only trace functions given by the argument.  Multiple functions
tools/perf/builtin-ftrace.c +38 -0
index 887e78b23a82..4efaa7b6a906 100644
@@ -39,6 +39,7 @@ struct perf_ftrace {
 	struct list_head	nograph_funcs;
 	int			graph_depth;
 	unsigned long		percpu_buffer_size;
+	bool			inherit;
 };
 
 struct filter_entry {
@@ -177,9 +178,27 @@ static int write_tracing_file_int(const char *name, int value)
 	return 0;
 }
 
+static int write_tracing_option_file(const char *name, const char *val)
+{
+	char *file;
+	int ret;
+
+	if (asprintf(&file, "options/%s", name) < 0)
+		return -1;
+
+	ret = __write_tracing_file(file, val, false);
+	free(file);
+	return ret;
+}
+
 static int reset_tracing_cpu(void);
 static void reset_tracing_filters(void);
 
+static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
+{
+	write_tracing_option_file("function-fork", "0");
+}
+
 static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
 {
 	if (write_tracing_file("tracing_on", "0") < 0)
@@ -198,6 +217,7 @@ static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
 		return -1;
 
 	reset_tracing_filters();
+	reset_tracing_options(ftrace);
 	return 0;
 }
 
@@ -336,6 +356,17 @@ static int set_tracing_percpu_buffer_size(struct perf_ftrace *ftrace)
 	return 0;
 }
 
+static int set_tracing_trace_inherit(struct perf_ftrace *ftrace)
+{
+	if (!ftrace->inherit)
+		return 0;
+
+	if (write_tracing_option_file("function-fork", "1") < 0)
+		return -1;
+
+	return 0;
+}
+
 static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
 {
 	char *trace_file;
@@ -405,6 +436,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
 		goto out_reset;
 	}
 
+	if (set_tracing_trace_inherit(ftrace) < 0) {
+		pr_err("failed to set tracing option function-fork\n");
+		goto out_reset;
+	}
+
 	if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
 		pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
 		goto out_reset;
@@ -599,6 +635,8 @@ int cmd_ftrace(int argc, const char **argv)
 		    "Max depth for function graph tracer"),
 	OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
 		     "size of per cpu buffer", parse_buffer_size),
+	OPT_BOOLEAN(0, "inherit", &ftrace.inherit,
+		    "trace children processes"),
 	OPT_END()
 	};
 
Signature · 1 line
2.25.1
#10 lore raw

PATCH v4 8/17 perf: util: add general function to parse sublevel options

In reply to Changbin Du

This factors out a general function perf_parse_sublevel_options() to parse sublevel options. The 'sublevel' options is something like the '--debug' options which allow more sublevel options.

---
v2: add util/parse-sublevel-options.c

---
 tools/perf/util/Build                    |  1 +
 tools/perf/util/debug.c                  | 61 +++++++----------------
 tools/perf/util/parse-sublevel-options.c | 63 ++++++++++++++++++++++++
 tools/perf/util/parse-sublevel-options.h | 11 +++++
 4 files changed, 92 insertions(+), 44 deletions(-)
 create mode 100644 tools/perf/util/parse-sublevel-options.c
 create mode 100644 tools/perf/util/parse-sublevel-options.h
tools/perf/util/Build +1 -0
index 8d18380ecd10..e86607ada0b5 100644
@@ -117,6 +117,7 @@ endif
 perf-y += parse-branch-options.o
 perf-y += dump-insn.o
 perf-y += parse-regs-options.o
+perf-y += parse-sublevel-options.o
 perf-y += term.o
 perf-y += help-unknown-cmd.o
 perf-y += mem-events.o
tools/perf/util/debug.c +17 -44

88 lines not on this page · raw message

tools/perf/util/parse-sublevel-options.c new file +63 -0

63 lines not on this page · raw message

tools/perf/util/parse-sublevel-options.h new file +11 -0
new file mode 100644
index 000000000000..9b9efcc2aaad
@@ -0,0 +1,11 @@
+#ifndef _PERF_PARSE_SUBLEVEL_OPTIONS_H
+#define _PERF_PARSE_SUBLEVEL_OPTIONS_H
+
+struct sublevel_option {
+	const char *name;
+	int *value_ptr;
+};
+
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts);
+
+#endif
\ No newline at end of file
Signature · 1 line
2.25.1
maintainer
#11 lore raw

Re: [PATCH v4 08/17] perf: util: add general function to parse sublevel options

Quoted 10 lines · Changbin Du

This factors out a general function perf_parse_sublevel_options() to parse sublevel options. The 'sublevel' options is something like the '--debug' options which allow more sublevel options.

---
v2: add util/parse-sublevel-options.c
---

[snip]

Quoted 10 lines
tools/perf/util/parse-sublevel-options.c new file +4 -0
new file mode 100644
index 000000000000..39798568547c
@@ -0,0 +1,63 @@
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdio.h>

tools/perf/util/parse-sublevel-options.c line 14

Quoted 10 lines
tools/perf/util/parse-sublevel-options.c new file +10 -0
+
+#include "util/debug.h"
+#include "util/parse-sublevel-options.h"
+
+static int parse_one_sublevel_option(const char *str,
+                                    struct sublevel_option *opts)
+{
+       struct sublevel_option *opt = &opts[0];
+       char *vstr, *s = strdup(str);
+       int v = 1;

I know you just copied the code, but let's add a check for the return value of strdup().

Also I think you can just use the opts pointer..

Quoted 57 lines
tools/perf/util/parse-sublevel-options.c new file +49 -0
+
+       vstr = strchr(s, '=');
+       if (vstr)
+               *vstr++ = 0;
+
+       while (opt->name) {
+               if (!strcmp(s, opt->name))
+                       break;
+               opt++;
+       }
+
+       if (!opt->name) {
+               pr_err("Unknown option name '%s'\n", s);
+               free(s);
+               return -1;
+       }
+
+       if (vstr)
+               v = atoi(vstr);
+
+       *opt->value_ptr = v;
+       free(s);
+       return 0;
+}
+
+/* parse options like --foo a=<n>,b,c... */
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts)
+{
+       char *s = strdup(str);
+       char *p = NULL;
+       int ret;
+
+       if (!s)
+               return -1;
+
+       p = strtok(s, ",");
+       while (p) {
+               ret = parse_one_sublevel_option(p, opts);
+               if (ret) {
+                       free(s);
+                       return ret;
+               }
+
+               p = strtok(NULL, ",");
+       }
+
+       free(s);
+       return 0;
+}
tools/perf/util/parse-sublevel-options.h new file +2 -0
new file mode 100644
index 000000000000..9b9efcc2aaad
@@ -0,0 +1,11 @@
+#ifndef _PERF_PARSE_SUBLEVEL_OPTIONS_H
+#define _PERF_PARSE_SUBLEVEL_OPTIONS_H

tools/perf/util/parse-sublevel-options.h line 11

Quoted 10 lines
tools/perf/util/parse-sublevel-options.h new file +9 -0
+
+struct sublevel_option {
+       const char *name;
+       int *value_ptr;
+};
+
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts);
+
+#endif
\ No newline at end of file

Please add a newline at the end.

Thanks
Namhyung

Quoted 3 lines

--
2.25.1

#12 lore raw

Re: [PATCH v4 08/17] perf: util: add general function to parse sublevel options

Quoted 27 lines · Namhyung Kim
Quoted 10 lines · Changbin Du

This factors out a general function perf_parse_sublevel_options() to parse sublevel options. The 'sublevel' options is something like the '--debug' options which allow more sublevel options.

---
v2: add util/parse-sublevel-options.c
---

[snip]

Quoted 15 lines
tools/perf/util/parse-sublevel-options.c new file +9 -0
new file mode 100644
index 000000000000..39798568547c
@@ -0,0 +1,63 @@
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdio.h>
+
+#include "util/debug.h"
+#include "util/parse-sublevel-options.h"
+
+static int parse_one_sublevel_option(const char *str,
Quoted 9 lines

tools/perf/util/parse-sublevel-options.c line 14

Quoted 5 lines
tools/perf/util/parse-sublevel-options.c new file +5 -0
+                                    struct sublevel_option *opts)
+{
+       struct sublevel_option *opt = &opts[0];
+       char *vstr, *s = strdup(str);
+       int v = 1;

I know you just copied the code, but let's add a check for the return value of strdup().

yes, the 's' should be checked here.

Quoted 2 lines

Also I think you can just use the opts pointer..

For this, personally I prefer to opt as it's singular form. Because we are dealing with single element.

Quoted 78 lines
Quoted 58 lines
tools/perf/util/parse-sublevel-options.c new file +49 -0
+
+       vstr = strchr(s, '=');
+       if (vstr)
+               *vstr++ = 0;
+
+       while (opt->name) {
+               if (!strcmp(s, opt->name))
+                       break;
+               opt++;
+       }
+
+       if (!opt->name) {
+               pr_err("Unknown option name '%s'\n", s);
+               free(s);
+               return -1;
+       }
+
+       if (vstr)
+               v = atoi(vstr);
+
+       *opt->value_ptr = v;
+       free(s);
+       return 0;
+}
+
+/* parse options like --foo a=<n>,b,c... */
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts)
+{
+       char *s = strdup(str);
+       char *p = NULL;
+       int ret;
+
+       if (!s)
+               return -1;
+
+       p = strtok(s, ",");
+       while (p) {
+               ret = parse_one_sublevel_option(p, opts);
+               if (ret) {
+                       free(s);
+                       return ret;
+               }
+
+               p = strtok(NULL, ",");
+       }
+
+       free(s);
+       return 0;
+}
tools/perf/util/parse-sublevel-options.h new file +2 -0
new file mode 100644
index 000000000000..9b9efcc2aaad
@@ -0,0 +1,11 @@
+#ifndef _PERF_PARSE_SUBLEVEL_OPTIONS_H
+#define _PERF_PARSE_SUBLEVEL_OPTIONS_H

tools/perf/util/parse-sublevel-options.h line 11

Quoted 10 lines
tools/perf/util/parse-sublevel-options.h new file +9 -0
+
+struct sublevel_option {
+       const char *name;
+       int *value_ptr;
+};
+
+int perf_parse_sublevel_options(const char *str, struct sublevel_option *opts);
+
+#endif
\ No newline at end of file

Please add a newline at the end.

Thanks
Namhyung

Quoted 3 lines

--
2.25.1

Signature · 2 lines
Cheers,
Changbin Du
#13 lore raw

PATCH v4 9/17 perf ftrace: add support for tracing option 'func_stack_trace'

In reply to Changbin Du

This adds support to display call trace for function tracer. To do this, just specify a '--func-opts call-graph' option.

$ sudo perf ftrace -T vfs_read --func-opts call-graph
 iio-sensor-prox-855   [003]   6168.369657: vfs_read <-ksys_read
 iio-sensor-prox-855   [003]   6168.369677: <stack trace>
 => vfs_read
 => ksys_read
 => __x64_sys_read
 => do_syscall_64
 => entry_SYSCALL_64_after_hwframe
 ...

---
v3: switch to uniform option --func-opts.
v2: option name '-s' -> '--func-call-graph'

---
 tools/perf/Documentation/perf-ftrace.txt |  4 +++
 tools/perf/builtin-ftrace.c              | 42 ++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
tools/perf/Documentation/perf-ftrace.txt +4 -0
index fd632bd9b2c1..676a30cb9b5a 100644
@@ -78,6 +78,10 @@ OPTIONS
 	(or glob patterns).  It will be passed to 'set_ftrace_notrace'
 	in tracefs.
 
+--func-opts::
+	List of options allowed to set:
+	  call-graph - Display kernel stack trace for function tracer.
+
 -G::
 --graph-funcs=::
 	Set graph filter on the given function (or a glob pattern).
tools/perf/builtin-ftrace.c +42 -0

84 lines not on this page · raw message

Signature · 1 line
2.25.1
#14 lore raw

PATCH v4 10/17 perf ftrace: add support for trace option sleep-time

In reply to Changbin Du

This adds an option '--graph-opts nosleep-time' which allow us only to measure on-CPU time. This option is function_graph tracer only.

---
v3: switch to uniform option --graph-opts.
v2: option name '--nosleep-time' -> '--graph-nosleep-time'.

---
 tools/perf/Documentation/perf-ftrace.txt |  4 +++
 tools/perf/builtin-ftrace.c              | 41 ++++++++++++++++++++++++
 2 files changed, 45 insertions(+)
tools/perf/Documentation/perf-ftrace.txt +4 -0
index 676a30cb9b5a..760c2b78c305 100644
@@ -102,6 +102,10 @@ OPTIONS
 --graph-depth=::
 	Set max depth for function graph tracer to follow
 
+--graph-opts::
+	List of options allowed to set:
+	  nosleep-time - Measure on-CPU time only for function_graph tracer.
+
 SEE ALSO
 --------
 linkperf:perf-record[1], linkperf:perf-trace[1]
tools/perf/builtin-ftrace.c +41 -0

77 lines not on this page · raw message

Signature · 1 line
2.25.1
#15 lore raw

PATCH v4 11/17 perf ftrace: add support for trace option funcgraph-irqs

In reply to Changbin Du

This adds an option '--graph-opts noirqs' to filter out functions executed in irq context.

---
v2: option name '--nofuncgraph-irqs' -> '--graph-noirqs'.

---
 tools/perf/Documentation/perf-ftrace.txt |  1 +
 tools/perf/builtin-ftrace.c              | 21 ++++++++++++++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)
tools/perf/Documentation/perf-ftrace.txt +1 -0
index 760c2b78c305..2e16e1e588d9 100644
@@ -105,6 +105,7 @@ OPTIONS
 --graph-opts::
 	List of options allowed to set:
 	  nosleep-time - Measure on-CPU time only for function_graph tracer.
+	  noirqs       - Ignore functions that happen inside interrupt.
 
 SEE ALSO
 --------
tools/perf/builtin-ftrace.c +20 -1
index b4eda459ba78..179c5da678e3 100644
@@ -43,6 +43,7 @@ struct perf_ftrace {
 	bool			inherit;
 	int			func_stack_trace;
 	int			graph_nosleep_time;
+	int			graph_noirqs;
 };
 
 struct filter_entry {
@@ -202,6 +203,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
 	write_tracing_option_file("function-fork", "0");
 	write_tracing_option_file("func_stack_trace", "0");
 	write_tracing_option_file("sleep-time", "1");
+	write_tracing_option_file("funcgraph-irqs", "1");
 }
 
 static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -394,6 +396,17 @@ static int set_tracing_sleep_time(struct perf_ftrace *ftrace)
 	return 0;
 }
 
+static int set_tracing_funcgraph_irqs(struct perf_ftrace *ftrace)
+{
+	if (!ftrace->graph_noirqs)
+		return 0;
+
+	if (write_tracing_option_file("funcgraph-irqs", "0") < 0)
+		return -1;
+
+	return 0;
+}
+
 static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
 {
 	char *trace_file;
@@ -478,6 +491,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
 		goto out_reset;
 	}
 
+	if (set_tracing_funcgraph_irqs(ftrace) < 0) {
+		pr_err("failed to set tracing option funcgraph-irqs\n");
+		goto out_reset;
+	}
+
 	if (write_tracing_file("current_tracer", ftrace->tracer) < 0) {
 		pr_err("failed to set current_tracer to %s\n", ftrace->tracer);
 		goto out_reset;
@@ -658,6 +676,7 @@ static int parse_graph_tracer_opts(const struct option *opt,
 	struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
 	struct sublevel_option graph_tracer_opts[] = {
 		{ .name = "nosleep-time",	.value_ptr = &ftrace->graph_nosleep_time },
+		{ .name = "noirqs",		.value_ptr = &ftrace->graph_noirqs },
 		{ .name = NULL, }
 	};
 
@@ -714,7 +733,7 @@ int cmd_ftrace(int argc, const char **argv)
 	OPT_INTEGER('D', "graph-depth", &ftrace.graph_depth,
 		    "Max depth for function graph tracer"),
 	OPT_CALLBACK(0, "graph-opts", &ftrace, "options",
-		     "graph tracer options, available options: nosleep-time",
+		     "graph tracer options, available options: nosleep-time,noirqs",
 		     parse_graph_tracer_opts),
 	OPT_CALLBACK('m', "buffer-size", &ftrace.percpu_buffer_size, "size",
 		     "size of per cpu buffer", parse_buffer_size),
Signature · 1 line
2.25.1
#16 lore raw

PATCH v4 12/17 perf ftrace: add support for tracing option 'irq-info'

In reply to Changbin Du

This adds support to display irq context info for function tracer. To do this, just specify a '--func-opts irq-info' option.

---
 tools/perf/Documentation/perf-ftrace.txt |  1 +
 tools/perf/builtin-ftrace.c              | 21 ++++++++++++++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)
tools/perf/Documentation/perf-ftrace.txt +1 -0
index 2e16e1e588d9..e6e9564d6c2e 100644
@@ -81,6 +81,7 @@ OPTIONS
 --func-opts::
 	List of options allowed to set:
 	  call-graph - Display kernel stack trace for function tracer.
+	  irq-info   - Display irq context info for function tracer.
 
 -G::
 --graph-funcs=::
tools/perf/builtin-ftrace.c +20 -1
index 179c5da678e3..a5906258c413 100644
@@ -42,6 +42,7 @@ struct perf_ftrace {
 	unsigned long		percpu_buffer_size;
 	bool			inherit;
 	int			func_stack_trace;
+	int			func_irq_info;
 	int			graph_nosleep_time;
 	int			graph_noirqs;
 };
@@ -204,6 +205,7 @@ static void reset_tracing_options(struct perf_ftrace *ftrace __maybe_unused)
 	write_tracing_option_file("func_stack_trace", "0");
 	write_tracing_option_file("sleep-time", "1");
 	write_tracing_option_file("funcgraph-irqs", "1");
+	write_tracing_option_file("irq-info", "0");
 }
 
 static int reset_tracing_files(struct perf_ftrace *ftrace __maybe_unused)
@@ -291,6 +293,17 @@ static int set_tracing_func_stack_trace(struct perf_ftrace *ftrace)
 	return 0;
 }
 
+static int set_tracing_func_irqinfo(struct perf_ftrace *ftrace)
+{
+	if (!ftrace->func_irq_info)
+		return 0;
+
+	if (write_tracing_option_file("irq-info", "1") < 0)
+		return -1;
+
+	return 0;
+}
+
 static int reset_tracing_cpu(void)
 {
 	struct perf_cpu_map *cpumap = perf_cpu_map__new(NULL);
@@ -466,6 +479,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv)
 		goto out_reset;
 	}
 
+	if (set_tracing_func_irqinfo(ftrace) < 0) {
+		pr_err("failed to set tracing option irq-info\n");
+		goto out_reset;
+	}
+
 	if (set_tracing_filters(ftrace) < 0) {
 		pr_err("failed to set tracing filters\n");
 		goto out_reset;
@@ -656,6 +674,7 @@ static int parse_func_tracer_opts(const struct option *opt,
 	struct perf_ftrace *ftrace = (struct perf_ftrace *) opt->value;
 	struct sublevel_option func_tracer_opts[] = {
 		{ .name = "call-graph",	.value_ptr = &ftrace->func_stack_trace },
+		{ .name = "irq-info",	.value_ptr = &ftrace->func_irq_info },
 		{ .name = NULL, }
 	};
 
@@ -723,7 +742,7 @@ int cmd_ftrace(int argc, const char **argv)
 	OPT_CALLBACK('N', "notrace-funcs", &ftrace.notrace, "func",
 		     "do not trace given functions", parse_filter_func),
 	OPT_CALLBACK(0, "func-opts", &ftrace, "options",
-		     "function tracer options, available options: call-graph",
+		     "function tracer options, available options: call-graph,irq-info",
 		     parse_func_tracer_opts),
 	OPT_CALLBACK('G', "graph-funcs", &ftrace.graph_funcs, "func",
 		     "trace given functions using function_graph tracer",
Signature · 1 line
2.25.1