Skip to contentSkip to conversation

tracing: updates and fixes

GIT PULL New Steven Rostedt 2 participants 6 messages Mar 6, 2010 – Mar 11, 2010

Subsystems (2)
lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

Ingo,

Please pull the latest tip/tracing/core tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
tip/tracing/core

Arnaldo Carvalho de Melo (1):
tracing: Update the comm field in the right variable in update_max_tr

Steven Rostedt (2):
function-graph: Fix unused reference to ftrace_set_func()
function-graph: Use comment notation for func names of dangling '}'

Tim Bird (1):
function-graph: Add tracing_thresh support to function_graph tracer

----

 kernel/trace/ftrace.c                |    6 ++----
 kernel/trace/trace.c                 |   24 ++++++++++++++++++++----
 kernel/trace/trace.h                 |    3 ++-
 kernel/trace/trace_functions_graph.c |   27 ++++++++++++++++++++++++---
 4 files changed, 48 insertions(+), 12 deletions(-)
author maintainer
#1 lore raw

PATCH 1/4 function-graph: Fix unused reference to ftrace_set_func()

From: Steven Rostedt <[email protected]>

The declaration of ftrace_set_func() is at the start of the ftrace.c file and wrapped with a #ifdef CONFIG_FUNCTION_GRAPH condition. If function graph tracing is enabled but CONFIG_DYNAMIC_FTRACE is not, a warning about that function being declared static and unused is given.

This really should have been placed within the CONFIG_FUNCTION_GRAPH condition that uses ftrace_set_func().

Moving the declaration down fixes the warning and makes the code cleaner.

---
 kernel/trace/ftrace.c |    6 ++----
 1 files changed, 2 insertions(+), 4 deletions(-)
kernel/trace/ftrace.c +2 -4
index d996353..d0407c9 100644
@@ -85,10 +85,6 @@ ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
 ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
 ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
 
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-static int ftrace_set_func(unsigned long *array, int *idx, char *buffer);
-#endif
-
 static void ftrace_list_func(unsigned long ip, unsigned long parent_ip)
 {
 	struct ftrace_ops *op = ftrace_list;
@@ -2300,6 +2296,8 @@ __setup("ftrace_filter=", set_ftrace_filter);
 
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 static char ftrace_graph_buf[FTRACE_FILTER_SIZE] __initdata;
+static int ftrace_set_func(unsigned long *array, int *idx, char *buffer);
+
 static int __init set_graph_function(char *str)
 {
 	strlcpy(ftrace_graph_buf, str, FTRACE_FILTER_SIZE);
Signature · 1 line
1.7.0
author maintainer
#2 lore raw

PATCH 2/4 function-graph: Use comment notation for func names of dangling }

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

When a '}' does not have a matching function start, the name is printed within parenthesis. But this makes it confusing between ending '}' and function starts. This patch makes the function name appear in C comment notation.

Old view:

 3)   1.281 us    |            } (might_fault)
 3)   3.620 us    |          } (filldir)
 3)   5.251 us    |        } (call_filldir)
 3)               |        call_filldir() {
 3)               |          filldir() {

New view:

 3)   1.281 us    |            } /* might_fault */
 3)   3.620 us    |          } /* filldir */
 3)   5.251 us    |        } /* call_filldir */
 3)               |        call_filldir() {
 3)               |          filldir() {
---
 kernel/trace/trace_functions_graph.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/trace/trace_functions_graph.c +1 -1
index e998a82..7b1f246 100644
@@ -920,7 +920,7 @@ print_graph_return(struct ftrace_graph_ret *trace, struct trace_seq *s,
 		if (!ret)
 			return TRACE_TYPE_PARTIAL_LINE;
 	} else {
-		ret = trace_seq_printf(s, "} (%ps)\n", (void *)trace->func);
+		ret = trace_seq_printf(s, "} /* %ps */\n", (void *)trace->func);
 		if (!ret)
 			return TRACE_TYPE_PARTIAL_LINE;
 	}
Signature · 1 line
1.7.0
author maintainer
#3 lore raw

PATCH 3/4 tracing: Update the comm field in the right variable in update_max_tr

In reply to Steven Rostedt

From: Arnaldo Carvalho de Melo <[email protected]>

The latency output showed:

# | task: -3 (uid:0 nice:0 policy:1 rt_prio:99)

The comm is missing in the "task:" and it looks like a minus 3 is the output. The correct display should be:

# | task: migration/0-3 (uid:0 nice:0 policy:1 rt_prio:99)

The problem is that the comm is being stored in the wrong data structure. The max_tr.data[cpu] is what stores the comm, not the tr->data[cpu].

Before this patch the max_tr.data[cpu]->comm was zeroed and the /debug/trace ended up showing just the '-' sign followed by the pid.

Also remove a needless initialization of max_data.

LKML-Reference: <[email protected]>

---
 kernel/trace/trace.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
kernel/trace/trace.c +2 -2
index 032c57c..6efd5cb 100644
@@ -592,7 +592,7 @@ static void
 __update_max_tr(struct trace_array *tr, struct task_struct *tsk, int cpu)
 {
 	struct trace_array_cpu *data = tr->data[cpu];
-	struct trace_array_cpu *max_data = tr->data[cpu];
+	struct trace_array_cpu *max_data;
 
 	max_tr.cpu = cpu;
 	max_tr.time_start = data->preempt_timestamp;
@@ -602,7 +602,7 @@ __update_max_tr(struct trace_array *tr, struct task_struct *tsk, int cpu)
 	max_data->critical_start = data->critical_start;
 	max_data->critical_end = data->critical_end;
 
-	memcpy(data->comm, tsk->comm, TASK_COMM_LEN);
+	memcpy(max_data->comm, tsk->comm, TASK_COMM_LEN);
 	max_data->pid = tsk->pid;
 	max_data->uid = task_uid(tsk);
 	max_data->nice = tsk->static_prio - 20 - MAX_RT_PRIO;
Signature · 1 line
1.7.0
author maintainer
#4 lore raw

PATCH 4/4 function-graph: Add tracing_thresh support to function_graph tracer

In reply to Steven Rostedt

From: Tim Bird <[email protected]>

Add support for tracing_thresh to the function_graph tracer. This version of this feature isolates the checks into new entry and return functions, to avoid adding more conditional code into the main function_graph paths.

When the tracing_thresh is set and the function graph tracer is enabled, only the functions that took longer than the time in microseconds that was set in tracing_thresh are recorded. To do this efficiently, only the function exits are recorded:

[tracing]# echo 100 > tracing_thresh
[tracing]# echo function_graph > current_tracer
[tracing]# cat trace
# tracer: function_graph
#
# CPU DURATION FUNCTION CALLS
# | | | | | | |

  1) ! 119.214 us  |  } /* smp_apic_timer_interrupt */
  1)   <========== |
  0) ! 101.527 us  |              } /* __rcu_process_callbacks */
  0) ! 126.461 us  |            } /* rcu_process_callbacks */
  0) ! 145.111 us  |          } /* __do_softirq */
  0) ! 149.667 us  |        } /* do_softirq */
  0) ! 168.817 us  |      } /* irq_exit */
  0) ! 248.254 us  |    } /* smp_apic_timer_interrupt */

Also, add support for specifying tracing_thresh on the kernel command line. When used like so: "tracing_thresh=200 ftrace=function_graph" this can be used to analyse system startup. It is important to disable tracing soon after boot, in order to avoid losing the trace data.

LKML-Reference: <[email protected]>

---
 kernel/trace/trace.c                 |   20 ++++++++++++++++++--
 kernel/trace/trace.h                 |    3 ++-
 kernel/trace/trace_functions_graph.c |   25 +++++++++++++++++++++++--
 3 files changed, 43 insertions(+), 5 deletions(-)
kernel/trace/trace.c +18 -2
index 6efd5cb..ababedb 100644
@@ -374,6 +374,21 @@ static int __init set_buf_size(char *str)
 }
 __setup("trace_buf_size=", set_buf_size);
 
+static int __init set_tracing_thresh(char *str)
+{
+	unsigned long threshhold;
+	int ret;
+
+	if (!str)
+		return 0;
+	ret = strict_strtoul(str, 0, &threshhold);
+	if (ret < 0)
+		return 0;
+	tracing_thresh = threshhold * 1000;
+	return 1;
+}
+__setup("tracing_thresh=", set_tracing_thresh);
+
 unsigned long nsecs_to_usecs(unsigned long nsecs)
 {
 	return nsecs / 1000;
@@ -579,9 +594,10 @@ static ssize_t trace_seq_to_buffer(struct trace_seq *s, void *buf, size_t cnt)
 static arch_spinlock_t ftrace_max_lock =
 	(arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED;
 
+unsigned long __read_mostly	tracing_thresh;
+
 #ifdef CONFIG_TRACER_MAX_TRACE
 unsigned long __read_mostly	tracing_max_latency;
-unsigned long __read_mostly	tracing_thresh;
 
 /*
  * Copy the new maximum trace into the separate maximum-trace
@@ -4248,10 +4264,10 @@ static __init int tracer_init_debugfs(void)
 #ifdef CONFIG_TRACER_MAX_TRACE
 	trace_create_file("tracing_max_latency", 0644, d_tracer,
 			&tracing_max_latency, &tracing_max_lat_fops);
+#endif
 
 	trace_create_file("tracing_thresh", 0644, d_tracer,
 			&tracing_thresh, &tracing_max_lat_fops);
-#endif
 
 	trace_create_file("README", 0444, d_tracer,
 			NULL, &tracing_readme_fops);
kernel/trace/trace.h +2 -1
index fd05bca..1bc8cd1 100644
@@ -396,9 +396,10 @@ extern int process_new_ksym_entry(char *ksymname, int op, unsigned long addr);
 
 extern unsigned long nsecs_to_usecs(unsigned long nsecs);
 
+extern unsigned long tracing_thresh;
+
 #ifdef CONFIG_TRACER_MAX_TRACE
 extern unsigned long tracing_max_latency;
-extern unsigned long tracing_thresh;
 
 void update_max_tr(struct trace_array *tr, struct task_struct *tsk, int cpu);
 void update_max_tr_single(struct trace_array *tr,
kernel/trace/trace_functions_graph.c +23 -2
index 7b1f246..e9df04b 100644
@@ -237,6 +237,14 @@ int trace_graph_entry(struct ftrace_graph_ent *trace)
 	return ret;
 }
 
+int trace_graph_thresh_entry(struct ftrace_graph_ent *trace)
+{
+	if (tracing_thresh)
+		return 1;
+	else
+		return trace_graph_entry(trace);
+}
+
 static void __trace_graph_return(struct trace_array *tr,
 				struct ftrace_graph_ret *trace,
 				unsigned long flags,
@@ -290,13 +298,26 @@ void set_graph_array(struct trace_array *tr)
 	smp_mb();
 }
 
+void trace_graph_thresh_return(struct ftrace_graph_ret *trace)
+{
+	if (tracing_thresh &&
+	    (trace->rettime - trace->calltime < tracing_thresh))
+		return;
+	else
+		trace_graph_return(trace);
+}
+
 static int graph_trace_init(struct trace_array *tr)
 {
 	int ret;
 
 	set_graph_array(tr);
-	ret = register_ftrace_graph(&trace_graph_return,
-				    &trace_graph_entry);
+	if (tracing_thresh)
+		ret = register_ftrace_graph(&trace_graph_thresh_return,
+					    &trace_graph_thresh_entry);
+	else
+		ret = register_ftrace_graph(&trace_graph_return,
+					    &trace_graph_entry);
 	if (ret)
 		return ret;
 	tracing_start_cmdline_record();
Signature · 1 line
1.7.0
#5 lore raw
In reply to Steven Rostedt
Quoted 18 lines · Steven Rostedt

Ingo,

Please pull the latest tip/tracing/core tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
tip/tracing/core

Arnaldo Carvalho de Melo (1):
tracing: Update the comm field in the right variable in update_max_tr

Steven Rostedt (2):
function-graph: Fix unused reference to ftrace_set_func()
function-graph: Use comment notation for func names of dangling '}'

Tim Bird (1):
function-graph: Add tracing_thresh support to function_graph tracer

Quoted 6 lines

----

 kernel/trace/ftrace.c                |    6 ++----
 kernel/trace/trace.c                 |   24 ++++++++++++++++++++----
 kernel/trace/trace.h                 |    3 ++-
 kernel/trace/trace_functions_graph.c |   27 ++++++++++++++++++++++++---
 4 files changed, 48 insertions(+), 12 deletions(-)

Pulled, thanks Steve!

Ingo