Skip to contentSkip to conversation

tracing: Various tracing fixes and enhancements

GIT PULL New Steven Rostedt 2 participants 13 messages Feb 2, 2013 – Feb 3, 2013

Subsystems (4)
lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

Ingo,

Please pull the latest tip/perf/core tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
tip/perf/core

Head SHA1: d840f718d28715a9833c1a8f46c2493ff3fd219b

Hiraku Toyooka (3):
tracing: Replace static old_tracer check of tracer name
tracing: Make a snapshot feature available from userspace
tracing: Add documentation of snapshot utility

Josh Triplett (1):
tracing: Mark tracing_dentry_percpu() static

Jovi Zhang (1):
tracing: Remove second iterator initializer

Namhyung Kim (1):
tracing: Use sched_clock_cpu for trace_clock_global

Shan Wei (1):
tracing: Use __this_cpu_inc/dec operation instead of __get_cpu_var

Steven Rostedt (1):
tracing: Remove tracepoint sample code

Steven Rostedt (Red Hat) (3):
tracing/fgraph: Adjust fgraph depth before calling trace return callback
ring-buffer: Add stats field for amount read from trace ring buffer
tracing: Init current_trace to nop_trace and remove NULL checks

----

 Documentation/trace/ftrace.txt                 |   83 +++++++++
 include/linux/ftrace_event.h                   |    3 +
 include/linux/ring_buffer.h                    |    1 +
 kernel/trace/Kconfig                           |   10 ++
 kernel/trace/ring_buffer.c                     |   18 ++
 kernel/trace/trace.c                           |  222 +++++++++++++++++-------
 kernel/trace/trace.h                           |    1 +
 kernel/trace/trace_clock.c                     |    2 +-
 kernel/trace/trace_functions_graph.c           |    8 +-
 samples/Kconfig                                |    6 -
 samples/Makefile                               |    2 +-
 samples/tracepoints/Makefile                   |    6 -
 samples/tracepoints/tp-samples-trace.h         |   11 --
 samples/tracepoints/tracepoint-probe-sample.c  |   57 ------
 samples/tracepoints/tracepoint-probe-sample2.c |   44 -----
 samples/tracepoints/tracepoint-sample.c        |   57 ------
 16 files changed, 288 insertions(+), 243 deletions(-)
author maintainer
#1 lore raw

PATCH 1/11 tracing: Mark tracing_dentry_percpu() static

From: Josh Triplett <[email protected]>

Nothing outside of kernel/trace/trace.c references tracing_dentry_percpu().

---
 kernel/trace/trace.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace.c +1 -1
index d2a6583..ca9b7df 100644
@@ -4506,7 +4506,7 @@ struct dentry *tracing_init_dentry(void)
 
 static struct dentry *d_percpu;
 
-struct dentry *tracing_dentry_percpu(void)
+static struct dentry *tracing_dentry_percpu(void)
 {
 	static int once;
 	struct dentry *d_tracer;
Signature · 1 line
1.7.10.4
author maintainer
#2 lore raw

PATCH 2/11 tracing: Remove tracepoint sample code

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The tracepoint sample code was used to teach developers how to create their own tracepoints. But now the trace_events have been added as a higher level that is used directly by developers today.

Only the trace_event code should use the tracepoint interface directly and no new tracepoints should be added.

Besides, the example had a race condition with the use of the
->d_name.name dentry field, as pointed out by Al Viro.

Best just to remove the code so it wont be used by other developers.

---
 samples/Kconfig                                |    6 ---
 samples/Makefile                               |    2 +-
 samples/tracepoints/Makefile                   |    6 ---
 samples/tracepoints/tp-samples-trace.h         |   11 -----
 samples/tracepoints/tracepoint-probe-sample.c  |   57 ------------------------
 samples/tracepoints/tracepoint-probe-sample2.c |   44 ------------------
 samples/tracepoints/tracepoint-sample.c        |   57 ------------------------
 7 files changed, 1 insertion(+), 182 deletions(-)
 delete mode 100644 samples/tracepoints/Makefile
 delete mode 100644 samples/tracepoints/tp-samples-trace.h
 delete mode 100644 samples/tracepoints/tracepoint-probe-sample.c
 delete mode 100644 samples/tracepoints/tracepoint-probe-sample2.c
 delete mode 100644 samples/tracepoints/tracepoint-sample.c
samples/Kconfig +0 -6
index 7b6792a..6181c2c 100644
@@ -5,12 +5,6 @@ menuconfig SAMPLES
 
 if SAMPLES
 
-config SAMPLE_TRACEPOINTS
-	tristate "Build tracepoints examples -- loadable modules only"
-	depends on TRACEPOINTS && m
-	help
-	  This build tracepoints example modules.
-
 config SAMPLE_TRACE_EVENTS
 	tristate "Build trace_events examples -- loadable modules only"
 	depends on EVENT_TRACING && m
samples/Makefile +1 -1
index 5ef08bb..1a60c62 100644
@@ -1,4 +1,4 @@
 # Makefile for Linux samples code
 
-obj-$(CONFIG_SAMPLES)	+= kobject/ kprobes/ tracepoints/ trace_events/ \
+obj-$(CONFIG_SAMPLES)	+= kobject/ kprobes/ trace_events/ \
 			   hw_breakpoint/ kfifo/ kdb/ hidraw/ rpmsg/ seccomp/
samples/tracepoints/Makefile deleted +0 -6
deleted file mode 100644
index 36479ad..0000000
@@ -1,6 +0,0 @@
-# builds the tracepoint example kernel modules;
-# then to use one (as root):  insmod <module_name.ko>
-
-obj-$(CONFIG_SAMPLE_TRACEPOINTS) += tracepoint-sample.o
-obj-$(CONFIG_SAMPLE_TRACEPOINTS) += tracepoint-probe-sample.o
-obj-$(CONFIG_SAMPLE_TRACEPOINTS) += tracepoint-probe-sample2.o
samples/tracepoints/tp-samples-trace.h deleted +0 -11
deleted file mode 100644
index 4d46be9..0000000
@@ -1,11 +0,0 @@
-#ifndef _TP_SAMPLES_TRACE_H
-#define _TP_SAMPLES_TRACE_H
-
-#include <linux/proc_fs.h>	/* for struct inode and struct file */
-#include <linux/tracepoint.h>
-
-DECLARE_TRACE(subsys_event,
-	TP_PROTO(struct inode *inode, struct file *file),
-	TP_ARGS(inode, file));
-DECLARE_TRACE_NOARGS(subsys_eventb);
-#endif
samples/tracepoints/tracepoint-probe-sample.c deleted +0 -57

57 lines not on this page · raw message

samples/tracepoints/tracepoint-probe-sample2.c deleted +0 -44

44 lines not on this page · raw message

samples/tracepoints/tracepoint-sample.c deleted +0 -57

57 lines not on this page · raw message

Signature · 1 line
1.7.10.4
author maintainer
#3 lore raw

PATCH 3/11 tracing: Use __this_cpu_inc/dec operation instead of __get_cpu_var

In reply to Steven Rostedt

From: Shan Wei <[email protected]>

__this_cpu_inc_return() or __this_cpu_dec generates a single instruction, which is faster than __get_cpu_var operation.

---
 kernel/trace/trace.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/trace/trace.c +2 -2
index ca9b7df..07888e1 100644
@@ -1344,7 +1344,7 @@ static void __ftrace_trace_stack(struct ring_buffer *buffer,
 	 */
 	preempt_disable_notrace();
 
-	use_stack = ++__get_cpu_var(ftrace_stack_reserve);
+	use_stack = __this_cpu_inc_return(ftrace_stack_reserve);
 	/*
 	 * We don't need any atomic variables, just a barrier.
 	 * If an interrupt comes in, we don't care, because it would
@@ -1398,7 +1398,7 @@ static void __ftrace_trace_stack(struct ring_buffer *buffer,
  out:
 	/* Again, don't let gcc optimize things here */
 	barrier();
-	__get_cpu_var(ftrace_stack_reserve)--;
+	__this_cpu_dec(ftrace_stack_reserve);
 	preempt_enable_notrace();
 
 }
Signature · 1 line
1.7.10.4
author maintainer
#4 lore raw

PATCH 4/11 tracing: Remove second iterator initializer

In reply to Steven Rostedt

From: Jovi Zhang <[email protected]>

The trace iterator is already initialized by trace_init_global_iter(), so there is no need to initialize it again.

---
 kernel/trace/trace.c |    5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
kernel/trace/trace.c +1 -4
index 07888e1..d399592 100644
@@ -5030,6 +5030,7 @@ __ftrace_dump(bool disable_tracing, enum ftrace_dump_mode oops_dump_mode)
 	if (disable_tracing)
 		ftrace_kill();
 
+	/* Simulate the iterator */
 	trace_init_global_iter(&iter);
 
 	for_each_tracing_cpu(cpu) {
@@ -5041,10 +5042,6 @@ __ftrace_dump(bool disable_tracing, enum ftrace_dump_mode oops_dump_mode)
 	/* don't look at user memory in panic mode */
 	trace_flags &= ~TRACE_ITER_SYM_USEROBJ;
 
-	/* Simulate the iterator */
-	iter.tr = &global_trace;
-	iter.trace = current_trace;
-
 	switch (oops_dump_mode) {
 	case DUMP_ALL:
 		iter.cpu_file = TRACE_PIPE_ALL_CPU;
Signature · 1 line
1.7.10.4
author maintainer
#5 lore raw

PATCH 5/11 tracing/fgraph: Adjust fgraph depth before calling trace return callback

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

While debugging the virtual cputime with the function graph tracer with a max_depth of 1 (most common use of the max_depth so far), I found that I was missing kernel execution because of a race condition.

The code for the return side of the function has a slight race:

	ftrace_pop_return_trace(&trace, &ret, frame_pointer);
	trace.rettime = trace_clock_local();
	ftrace_graph_return(&trace);
	barrier();
	current->curr_ret_stack--;

The ftrace_pop_return_trace() initializes the trace structure for the callback. The ftrace_graph_return() uses the trace structure for its own use as that structure is on the stack and is local to this function. Then the curr_ret_stack is decremented which is what the trace.depth is set to.

If an interrupt comes in after the ftrace_graph_return() but before the curr_ret_stack, then the called function will get a depth of 2. If max_depth is set to 1 this function will be ignored.

The problem is that the trace has already been called, and the timestamp for that trace will not reflect the time the function was about to re-enter userspace. Calls to the interrupt will not be traced because the max_depth has prevented this.

To solve this issue, the ftrace_graph_return() can safely be moved after the current->curr_ret_stack has been updated. This way the timestamp for the return callback will reflect the actual time.

If an interrupt comes in after the curr_ret_stack update and ftrace_graph_return(), it will be traced. It may look a little confusing to see it within the other function, but at least it will not be lost.

Cc Frederic Weisbecker <[email protected]>
---
 kernel/trace/trace_functions_graph.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/trace/trace_functions_graph.c +7 -1
index 7008d2e..39ada66 100644
@@ -191,10 +191,16 @@ unsigned long ftrace_return_to_handler(unsigned long frame_pointer)
 
 	ftrace_pop_return_trace(&trace, &ret, frame_pointer);
 	trace.rettime = trace_clock_local();
-	ftrace_graph_return(&trace);
 	barrier();
 	current->curr_ret_stack--;
 
+	/*
+	 * The trace should run after decrementing the ret counter
+	 * in case an interrupt were to come in. We don't want to
+	 * lose the interrupt if max_depth is set.
+	 */
+	ftrace_graph_return(&trace);
+
 	if (unlikely(!ret)) {
 		ftrace_graph_stop();
 		WARN_ON(1);
Signature · 1 line
1.7.10.4
author maintainer
#6 lore raw

PATCH 6/11 ring-buffer: Add stats field for amount read from trace ring buffer

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

Add a stat about the number of events read from the ring buffer:

# cat /debug/tracing/per_cpu/cpu0/stats

entries: 39869
overrun: 870512
commit overrun: 0
bytes: 1449912
oldest event ts:  6561.368690
now ts:  6565.246426
dropped events: 0
read events: 112    <-- Added
---
 include/linux/ring_buffer.h |    1 +
 kernel/trace/ring_buffer.c  |   18 ++++++++++++++++++
 kernel/trace/trace.c        |    3 +++
 3 files changed, 22 insertions(+)
include/linux/ring_buffer.h +1 -0
index 519777e..1342e69 100644
@@ -167,6 +167,7 @@ unsigned long ring_buffer_entries_cpu(struct ring_buffer *buffer, int cpu);
 unsigned long ring_buffer_overrun_cpu(struct ring_buffer *buffer, int cpu);
 unsigned long ring_buffer_commit_overrun_cpu(struct ring_buffer *buffer, int cpu);
 unsigned long ring_buffer_dropped_events_cpu(struct ring_buffer *buffer, int cpu);
+unsigned long ring_buffer_read_events_cpu(struct ring_buffer *buffer, int cpu);
 
 u64 ring_buffer_time_stamp(struct ring_buffer *buffer, int cpu);
 void ring_buffer_normalize_time_stamp(struct ring_buffer *buffer,
kernel/trace/ring_buffer.c +18 -0
index 13950d9..7244acd 100644
@@ -3103,6 +3103,24 @@ ring_buffer_dropped_events_cpu(struct ring_buffer *buffer, int cpu)
 EXPORT_SYMBOL_GPL(ring_buffer_dropped_events_cpu);
 
 /**
+ * ring_buffer_read_events_cpu - get the number of events successfully read
+ * @buffer: The ring buffer
+ * @cpu: The per CPU buffer to get the number of events read
+ */
+unsigned long
+ring_buffer_read_events_cpu(struct ring_buffer *buffer, int cpu)
+{
+	struct ring_buffer_per_cpu *cpu_buffer;
+
+	if (!cpumask_test_cpu(cpu, buffer->cpumask))
+		return 0;
+
+	cpu_buffer = buffer->buffers[cpu];
+	return cpu_buffer->read;
+}
+EXPORT_SYMBOL_GPL(ring_buffer_read_events_cpu);
+
+/**
  * ring_buffer_entries - get the number of entries in a buffer
  * @buffer: The ring buffer
  *
kernel/trace/trace.c +3 -0
index d399592..90a1c71 100644
@@ -4430,6 +4430,9 @@ tracing_stats_read(struct file *filp, char __user *ubuf,
 	cnt = ring_buffer_dropped_events_cpu(tr->buffer, cpu);
 	trace_seq_printf(s, "dropped events: %ld\n", cnt);
 
+	cnt = ring_buffer_read_events_cpu(tr->buffer, cpu);
+	trace_seq_printf(s, "read events: %ld\n", cnt);
+
 	count = simple_read_from_buffer(ubuf, count, ppos, s->buffer, s->len);
 
 	kfree(s);
Signature · 1 line
1.7.10.4
author maintainer
#7 lore raw

PATCH 7/11 tracing: Use sched_clock_cpu for trace_clock_global

In reply to Steven Rostedt

From: Namhyung Kim <[email protected]>

For systems with an unstable sched_clock, all cpu_clock() does is enable/ disable local irq during the call to sched_clock_cpu(). And for stable systems they are same.

trace_clock_global() already disables interrupts, so it can call sched_clock_cpu() directly.

---
 kernel/trace/trace_clock.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace_clock.c +1 -1
index 22b638b..24bf48e 100644
@@ -84,7 +84,7 @@ u64 notrace trace_clock_global(void)
 	local_irq_save(flags);
 
 	this_cpu = raw_smp_processor_id();
-	now = cpu_clock(this_cpu);
+	now = sched_clock_cpu(this_cpu);
 	/*
 	 * If in an NMI context then dont risk lockups and return the
 	 * cpu_clock() time:
Signature · 1 line
1.7.10.4
author maintainer
#8 lore raw

PATCH 8/11 tracing: Replace static old_tracer check of tracer name

In reply to Steven Rostedt

From: Hiraku Toyooka <[email protected]>

Currently the trace buffer read functions use a static variable "old_tracer" for detecting if the current tracer changes. This was suitable for a single trace file ("trace"), but to add a snapshot feature that will use the same function for its file, a check against a static variable is not sufficient.

To use the output functions for two different files, instead of storing the current tracer in a static variable, as the trace iterator descriptor contains a pointer to the original current tracer's name, that pointer can now be used to check if the current tracer has changed between different reads of the trace file.

---
 kernel/trace/trace.c |   22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)
kernel/trace/trace.c +9 -13
index 90a1c71..2c72466 100644
@@ -1948,18 +1948,20 @@ void tracing_iter_reset(struct trace_iterator *iter, int cpu)
 static void *s_start(struct seq_file *m, loff_t *pos)
 {
 	struct trace_iterator *iter = m->private;
-	static struct tracer *old_tracer;
 	int cpu_file = iter->cpu_file;
 	void *p = NULL;
 	loff_t l = 0;
 	int cpu;
 
-	/* copy the tracer to avoid using a global lock all around */
+	/*
+	 * copy the tracer to avoid using a global lock all around.
+	 * iter->trace is a copy of current_trace, the pointer to the
+	 * name may be used instead of a strcmp(), as iter->trace->name
+	 * will point to the same string as current_trace->name.
+	 */
 	mutex_lock(&trace_types_lock);
-	if (unlikely(old_tracer != current_trace && current_trace)) {
-		old_tracer = current_trace;
+	if (unlikely(current_trace && iter->trace->name != current_trace->name))
 		*iter->trace = *current_trace;
-	}
 	mutex_unlock(&trace_types_lock);
 
 	atomic_inc(&trace_record_cmdline_disabled);
@@ -3494,7 +3496,6 @@ tracing_read_pipe(struct file *filp, char __user *ubuf,
 		  size_t cnt, loff_t *ppos)
 {
 	struct trace_iterator *iter = filp->private_data;
-	static struct tracer *old_tracer;
 	ssize_t sret;
 
 	/* return any leftover data */
@@ -3506,10 +3507,8 @@ tracing_read_pipe(struct file *filp, char __user *ubuf,
 
 	/* copy the tracer to avoid using a global lock all around */
 	mutex_lock(&trace_types_lock);
-	if (unlikely(old_tracer != current_trace && current_trace)) {
-		old_tracer = current_trace;
+	if (unlikely(current_trace && iter->trace->name != current_trace->name))
 		*iter->trace = *current_trace;
-	}
 	mutex_unlock(&trace_types_lock);
 
 	/*
@@ -3665,7 +3664,6 @@ static ssize_t tracing_splice_read_pipe(struct file *filp,
 		.ops		= &tracing_pipe_buf_ops,
 		.spd_release	= tracing_spd_release_pipe,
 	};
-	static struct tracer *old_tracer;
 	ssize_t ret;
 	size_t rem;
 	unsigned int i;
@@ -3675,10 +3673,8 @@ static ssize_t tracing_splice_read_pipe(struct file *filp,
 
 	/* copy the tracer to avoid using a global lock all around */
 	mutex_lock(&trace_types_lock);
-	if (unlikely(old_tracer != current_trace && current_trace)) {
-		old_tracer = current_trace;
+	if (unlikely(current_trace && iter->trace->name != current_trace->name))
 		*iter->trace = *current_trace;
-	}
 	mutex_unlock(&trace_types_lock);
 
 	mutex_lock(&iter->mutex);
Signature · 1 line
1.7.10.4
author maintainer
#9 lore raw

PATCH 9/11 tracing: Make a snapshot feature available from userspace

In reply to Steven Rostedt

From: Hiraku Toyooka <[email protected]>

Ftrace has a snapshot feature available from kernel space and latency tracers (e.g. irqsoff) are using it. This patch enables user applictions to take a snapshot via debugfs.

Add "snapshot" debugfs file in "tracing" directory.

snapshot:
This is used to take a snapshot and to read the output of the snapshot.

# echo 1 > snapshot

This will allocate the spare buffer for snapshot (if it is not allocated), and take a snapshot.

# cat snapshot

This will show contents of the snapshot.

# echo 0 > snapshot

This will free the snapshot if it is allocated.

Any other positive values will clear the snapshot contents if the snapshot is allocated, or return EINVAL if it is not allocated.

[
Fixed irqsoff selftest and also a conflict with a change that fixes the update_max_tr.
]

---
 include/linux/ftrace_event.h |    3 +
 kernel/trace/Kconfig         |   10 +++
 kernel/trace/trace.c         |  166 +++++++++++++++++++++++++++++++++++-------
 kernel/trace/trace.h         |    1 +
 4 files changed, 154 insertions(+), 26 deletions(-)
include/linux/ftrace_event.h +3 -0
index 6f8d0b7..13a54d0 100644
@@ -83,6 +83,9 @@ struct trace_iterator {
 	long			idx;
 
 	cpumask_var_t		started;
+
+	/* it's true when current open file is snapshot */
+	bool			snapshot;
 };
 
 enum trace_iter_flags {
kernel/trace/Kconfig +10 -0
index cdc9d28..3656756 100644
@@ -253,6 +253,16 @@ config FTRACE_SYSCALLS
 	help
 	  Basic tracer to catch the syscall entry and exit events.
 
+config TRACER_SNAPSHOT
+	bool "Create a snapshot trace buffer"
+	select TRACER_MAX_TRACE
+	help
+	  Allow tracing users to take snapshot of the current buffer using the
+	  ftrace interface, e.g.:
+
+	      echo 1 > /sys/kernel/debug/tracing/snapshot
+	      cat snapshot
+
 config TRACE_BRANCH_PROFILING
 	bool
 	select GENERIC_TRACER
kernel/trace/trace.c +140 -26

273 lines not on this page · raw message

kernel/trace/trace.h +1 -0
index 04a2c7a..57d7e53 100644
@@ -287,6 +287,7 @@ struct tracer {
 	struct tracer_flags	*flags;
 	bool			print_max;
 	bool			use_max_tr;
+	bool			allocated_snapshot;
 };
 
 
Signature · 1 line
1.7.10.4
author maintainer
#10 lore raw

PATCH 10/11 tracing: Add documentation of snapshot utility

In reply to Steven Rostedt

From: Hiraku Toyooka <[email protected]>

This patch adds snapshot description in ftrace documentation. This description includes what the snapshot is and how to use it.

---
 Documentation/trace/ftrace.txt |   83 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
Documentation/trace/ftrace.txt +83 -0
index 6f51fed..53d6a3c 100644
@@ -1842,6 +1842,89 @@ an error.
  # cat buffer_size_kb
 85
 
+Snapshot
+--------
+CONFIG_TRACER_SNAPSHOT makes a generic snapshot feature
+available to all non latency tracers. (Latency tracers which
+record max latency, such as "irqsoff" or "wakeup", can't use
+this feature, since those are already using the snapshot
+mechanism internally.)
+
+Snapshot preserves a current trace buffer at a particular point
+in time without stopping tracing. Ftrace swaps the current
+buffer with a spare buffer, and tracing continues in the new
+current (=previous spare) buffer.
+
+The following debugfs files in "tracing" are related to this
+feature:
+
+  snapshot:
+
+	This is used to take a snapshot and to read the output
+	of the snapshot. Echo 1 into this file to allocate a
+	spare buffer and to take a snapshot (swap), then read
+	the snapshot from this file in the same format as
+	"trace" (described above in the section "The File
+	System"). Both reads snapshot and tracing are executable
+	in parallel. When the spare buffer is allocated, echoing
+	0 frees it, and echoing else (positive) values clear the
+	snapshot contents.
+	More details are shown in the table below.
+
+	status\input  |     0      |     1      |    else    |
+	--------------+------------+------------+------------+
+	not allocated |(do nothing)| alloc+swap |   EINVAL   |
+	--------------+------------+------------+------------+
+	allocated     |    free    |    swap    |   clear    |
+	--------------+------------+------------+------------+
+
+Here is an example of using the snapshot feature.
+
+ # echo 1 > events/sched/enable
+ # echo 1 > snapshot
+ # cat snapshot
+# tracer: nop
+#
+# entries-in-buffer/entries-written: 71/71   #P:8
+#
+#                              _-----=> irqs-off
+#                             / _----=> need-resched
+#                            | / _---=> hardirq/softirq
+#                            || / _--=> preempt-depth
+#                            ||| /     delay
+#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
+#              | |       |   ||||       |         |
+          <idle>-0     [005] d...  2440.603828: sched_switch: prev_comm=swapper/5 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=snapshot-test-2 next_pid=2242 next_prio=120
+           sleep-2242  [005] d...  2440.603846: sched_switch: prev_comm=snapshot-test-2 prev_pid=2242 prev_prio=120 prev_state=R ==> next_comm=kworker/5:1 next_pid=60 next_prio=120
+[...]
+          <idle>-0     [002] d...  2440.707230: sched_switch: prev_comm=swapper/2 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=snapshot-test-2 next_pid=2229 next_prio=120
+
+ # cat trace
+# tracer: nop
+#
+# entries-in-buffer/entries-written: 77/77   #P:8
+#
+#                              _-----=> irqs-off
+#                             / _----=> need-resched
+#                            | / _---=> hardirq/softirq
+#                            || / _--=> preempt-depth
+#                            ||| /     delay
+#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
+#              | |       |   ||||       |         |
+          <idle>-0     [007] d...  2440.707395: sched_switch: prev_comm=swapper/7 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=snapshot-test-2 next_pid=2243 next_prio=120
+ snapshot-test-2-2229  [002] d...  2440.707438: sched_switch: prev_comm=snapshot-test-2 prev_pid=2229 prev_prio=120 prev_state=S ==> next_comm=swapper/2 next_pid=0 next_prio=120
+[...]
+
+
+If you try to use this snapshot feature when current tracer is
+one of the latency tracers, you will get the following results.
+
+ # echo wakeup > current_tracer
+ # echo 1 > snapshot
+bash: echo: write error: Device or resource busy
+ # cat snapshot
+cat: snapshot: Device or resource busy
+
 -----------
 
 More details can be found in the source code, in the
Signature · 1 line
1.7.10.4
author maintainer
#11 lore raw

PATCH 11/11 tracing: Init current_trace to nop_trace and remove NULL checks

In reply to Steven Rostedt

From: "Steven Rostedt (Red Hat)" <[email protected]>

On early boot up, when the ftrace ring buffer is initialized, the static variable current_trace is initialized to &nop_trace. Before this initialization, current_trace is NULL and will never become NULL again. It is always reassigned to a ftrace tracer.

Several places check if current_trace is NULL before it uses it, and this check is frivolous, because at the point in time when the checks are made the only way current_trace could be NULL is if ftrace failed its allocations at boot up, and the paths to these locations would probably not be possible.

By initializing current_trace to &nop_trace where it is declared, current_trace will never be NULL, and we can remove all these checks of current_trace being NULL which never needed to be checked in the first place.

Cc Dan Carpenter <[email protected]>
Cc Hiraku Toyooka <[email protected]>
---
 kernel/trace/trace.c |   30 ++++++++++++------------------
 1 file changed, 12 insertions(+), 18 deletions(-)
kernel/trace/trace.c +12 -18
index 70dce64..5d520b7 100644
@@ -249,7 +249,7 @@ static unsigned long		trace_buf_size = TRACE_BUF_SIZE_DEFAULT;
 static struct tracer		*trace_types __read_mostly;
 
 /* current_trace points to the tracer that is currently active */
-static struct tracer		*current_trace __read_mostly;
+static struct tracer		*current_trace __read_mostly = &nop_trace;
 
 /*
  * trace_types_lock is used to protect the trace_types list.
@@ -2100,8 +2100,7 @@ print_trace_header(struct seq_file *m, struct trace_iterator *iter)
 	unsigned long total;
 	const char *name = "preemption";
 
-	if (type)
-		name = type->name;
+	name = type->name;
 
 	get_total_entries(tr, &total, &entries);
 
@@ -2477,13 +2476,12 @@ __tracing_open(struct inode *inode, struct file *file, bool snapshot)
 	if (!iter->trace)
 		goto fail;
 
-	if (current_trace)
-		*iter->trace = *current_trace;
+	*iter->trace = *current_trace;
 
 	if (!zalloc_cpumask_var(&iter->started, GFP_KERNEL))
 		goto fail;
 
-	if ((current_trace && current_trace->print_max) || snapshot)
+	if (current_trace->print_max || snapshot)
 		iter->tr = &max_tr;
 	else
 		iter->tr = &global_trace;
@@ -3037,10 +3035,7 @@ tracing_set_trace_read(struct file *filp, char __user *ubuf,
 	int r;
 
 	mutex_lock(&trace_types_lock);
-	if (current_trace)
-		r = sprintf(buf, "%s\n", current_trace->name);
-	else
-		r = sprintf(buf, "\n");
+	r = sprintf(buf, "%s\n", current_trace->name);
 	mutex_unlock(&trace_types_lock);
 
 	return simple_read_from_buffer(ubuf, cnt, ppos, buf, r);
@@ -3231,10 +3226,10 @@ static int tracing_set_tracer(const char *buf)
 		goto out;
 
 	trace_branch_disable();
-	if (current_trace && current_trace->reset)
+	if (current_trace->reset)
 		current_trace->reset(tr);
 
-	had_max_tr = current_trace && current_trace->allocated_snapshot;
+	had_max_tr = current_trace->allocated_snapshot;
 	current_trace = &nop_trace;
 
 	if (had_max_tr && !t->use_max_tr) {
@@ -3373,8 +3368,7 @@ static int tracing_open_pipe(struct inode *inode, struct file *filp)
 		ret = -ENOMEM;
 		goto fail;
 	}
-	if (current_trace)
-		*iter->trace = *current_trace;
+	*iter->trace = *current_trace;
 
 	if (!alloc_cpumask_var(&iter->started, GFP_KERNEL)) {
 		ret = -ENOMEM;
@@ -3525,7 +3519,7 @@ tracing_read_pipe(struct file *filp, char __user *ubuf,
 
 	/* copy the tracer to avoid using a global lock all around */
 	mutex_lock(&trace_types_lock);
-	if (unlikely(current_trace && iter->trace->name != current_trace->name))
+	if (unlikely(iter->trace->name != current_trace->name))
 		*iter->trace = *current_trace;
 	mutex_unlock(&trace_types_lock);
 
@@ -3691,7 +3685,7 @@ static ssize_t tracing_splice_read_pipe(struct file *filp,
 
 	/* copy the tracer to avoid using a global lock all around */
 	mutex_lock(&trace_types_lock);
-	if (unlikely(current_trace && iter->trace->name != current_trace->name))
+	if (unlikely(iter->trace->name != current_trace->name))
 		*iter->trace = *current_trace;
 	mutex_unlock(&trace_types_lock);
 
@@ -4115,7 +4109,7 @@ tracing_snapshot_write(struct file *filp, const char __user *ubuf, size_t cnt,
 
 	mutex_lock(&trace_types_lock);
 
-	if (current_trace && current_trace->use_max_tr) {
+	if (current_trace->use_max_tr) {
 		ret = -EBUSY;
 		goto out;
 	}
@@ -5299,7 +5293,7 @@ __init static int tracer_alloc_buffers(void)
 	init_irq_work(&trace_work_wakeup, trace_wake_up);
 
 	register_tracer(&nop_trace);
-	current_trace = &nop_trace;
+
 	/* All seems OK, enable tracing */
 	tracing_disabled = 0;
 
Signature · 1 line
1.7.10.4
#12 lore raw
In reply to Steven Rostedt
Quoted 46 lines · Steven Rostedt

Ingo,

Please pull the latest tip/perf/core tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
tip/perf/core

Head SHA1: d840f718d28715a9833c1a8f46c2493ff3fd219b

Hiraku Toyooka (3):
tracing: Replace static old_tracer check of tracer name
tracing: Make a snapshot feature available from userspace
tracing: Add documentation of snapshot utility

Josh Triplett (1):
tracing: Mark tracing_dentry_percpu() static

Jovi Zhang (1):
tracing: Remove second iterator initializer

Namhyung Kim (1):
tracing: Use sched_clock_cpu for trace_clock_global

Shan Wei (1):
tracing: Use __this_cpu_inc/dec operation instead of __get_cpu_var

Steven Rostedt (1):
tracing: Remove tracepoint sample code

Steven Rostedt (Red Hat) (3):
tracing/fgraph: Adjust fgraph depth before calling trace return callback
ring-buffer: Add stats field for amount read from trace ring buffer
tracing: Init current_trace to nop_trace and remove NULL checks

----

 Documentation/trace/ftrace.txt                 |   83 +++++++++
 include/linux/ftrace_event.h                   |    3 +
 include/linux/ring_buffer.h                    |    1 +
 kernel/trace/Kconfig                           |   10 ++
 kernel/trace/ring_buffer.c                     |   18 ++
 kernel/trace/trace.c                           |  222 +++++++++++++++++-------
 kernel/trace/trace.h                           |    1 +
 kernel/trace/trace_clock.c                     |    2 +-
 kernel/trace/trace_functions_graph.c           |    8 +-
Quoted 8 lines
 samples/Kconfig                                |    6 -
 samples/Makefile                               |    2 +-
 samples/tracepoints/Makefile                   |    6 -
 samples/tracepoints/tp-samples-trace.h         |   11 --
 samples/tracepoints/tracepoint-probe-sample.c  |   57 ------
 samples/tracepoints/tracepoint-probe-sample2.c |   44 -----
 samples/tracepoints/tracepoint-sample.c        |   57 ------
 16 files changed, 288 insertions(+), 243 deletions(-)

Pulled, thanks Steve!

Ingo