Skip to contentSkip to conversation

for tip/tracing/ftrace

GIT PULL In review Steven Rostedt 4 participants 15 messages Feb 28, 2009 – Mar 2, 2009 Tracing

lore
Share & tools
lore
author maintainer
#1 lore raw

PATCH 1/10 tracing: move trace point formats to files in include/trace directory

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Impact: clean up

To further facilitate the ease of adding trace points for developers, this patch creates include/trace/trace_events.h and include/trace/trace_event_types.h.

The former file will hold the trace/<type>.h files and the latter will hold the trace/<type>_event_types.h files.

To create new tracepoints and to have them automatically
appear in the event tracer, a developer makes the trace/<type>.h file which includes <linux/tracepoint.h> and the trace/<type>_event_types.h file.

The trace/<type>_event_types.h file will hold the TRACE_FORMAT macros.

Then add the trace/<type>.h file to trace/trace_events.h, and add the trace/<type>_event_types.h to the trace_event_types.h file.

No need to modify files elsewhere.

---
 include/trace/trace_event_types.h |    4 ++++
 include/trace/trace_events.h      |    4 ++++
 kernel/trace/events.c             |   10 ++--------
 3 files changed, 10 insertions(+), 8 deletions(-)
 create mode 100644 include/trace/trace_event_types.h
 create mode 100644 include/trace/trace_events.h
include/trace/trace_event_types.h new file +4 -0
new file mode 100644
index 0000000..33c8ed5
@@ -0,0 +1,4 @@
+/* trace/<type>_event_types.h here */
+
+#include <trace/sched_event_types.h>
+#include <trace/irq_event_types.h>
include/trace/trace_events.h new file +4 -0
new file mode 100644
index 0000000..ea2ef20
@@ -0,0 +1,4 @@
+/* trace/<type>.h here */
+
+#include <trace/sched.h>
+#include <trace/irq.h>
kernel/trace/events.c +2 -8
index 3c75623..46e27ad 100644
@@ -1,15 +1,9 @@
 /*
  * This is the place to register all trace points as events.
- * Include the trace/<type>.h at the top.
- * Include the trace/<type>_event_types.h at the bottom.
  */
 
-/* trace/<type>.h here */
-#include <trace/sched.h>
-#include <trace/irq.h>
+#include <trace/trace_events.h>
 
 #include "trace_events.h"
 
-/* trace/<type>_event_types.h here */
-#include <trace/sched_event_types.h>
-#include <trace/irq_event_types.h>
+#include <trace/trace_event_types.h>
Signature · 3 lines
1.5.6.5

-- 

Steven Rostedt

author maintainer
#0 lore raw

Ingo,

Please pull the latest tip/tracing/ftrace tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
tip/tracing/ftrace

Steven Rostedt (10):
tracing: move trace point formats to files in include/trace directory
tracing: add subsystem level to trace events
tracing: make the set_event and available_events subsystem aware
tracing: add subsystem irq for irq events
tracing: add subsystem sched for sched events
tracing: add interface to write into current tracer buffer
tracing: add raw trace point recording infrastructure
tracing: add raw fast tracing interface for trace events
tracing: create the C style tracing for the sched subsystem
tracing: create the C style tracing for the irq subsystem

----

 include/linux/tracepoint.h          |    3 +
 include/trace/irq_event_types.h     |   24 +++-
 include/trace/sched_event_types.h   |  124 ++++++++++++---
 include/trace/trace_event_types.h   |    4 +
 include/trace/trace_events.h        |    4 +
 kernel/trace/events.c               |   20 ++-
 kernel/trace/trace.c                |   14 ++
 kernel/trace/trace.h                |   32 ++++
 kernel/trace/trace_events.c         |  294 +++++++++++++++++++++++++++++++----
 kernel/trace/trace_events.h         |   55 -------
 kernel/trace/trace_events_stage_1.h |   34 ++++
 kernel/trace/trace_events_stage_2.h |   72 +++++++++
 kernel/trace/trace_events_stage_3.h |  219 ++++++++++++++++++++++++++
 13 files changed, 776 insertions(+), 123 deletions(-)
author maintainer
#4 lore raw

PATCH 4/10 tracing: add subsystem irq for irq events

From: Steven Rostedt <[email protected]>

Add the TRACE_SYSTEM irq for the irq events.

---
 include/trace/irq_event_types.h |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
include/trace/irq_event_types.h +5 -0
index 5d0919f..47a2be1 100644
@@ -5,6 +5,9 @@
 # error Unless you know what you are doing.
 #endif
 
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM irq
+
 TRACE_FORMAT(irq_handler_entry,
 	TPPROTO(int irq, struct irqaction *action),
 	TPARGS(irq, action),
@@ -15,3 +18,5 @@ TRACE_FORMAT(irq_handler_exit,
 	TPARGS(irq, action, ret),
 	TPFMT("irq=%d handler=%s return=%s",
 		irq, action->name, ret ? "handled" : "unhandled"));
+
+#undef TRACE_SYSTEM
Signature · 3 lines
1.5.6.5

-- 
author maintainer
#2 lore raw

PATCH 2/10 tracing: add subsystem level to trace events

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

If a trace point header defines TRACE_SYSTEM, then it will add the following trace points into that event system.

If include/trace/irq_event_types.h has:

#define TRACE_SYSTEM irq

at the top and

#undef TRACE_SYSTEM

at the bottom, then a directory "irq" will be created in the /debug/tracing/events directory. Inside that directory will contain the two trace points that are defined in include/trace/irq_event_types.h.

Only adding the above to irq and not to sched, we get:

 # ls /debug/tracing/events/
irq                     sched_process_exit  sched_signal_send  sched_wakeup_new
sched_kthread_stop      sched_process_fork  sched_switch
sched_kthread_stop_ret  sched_process_free  sched_wait_task
sched_migrate_task      sched_process_wait  sched_wakeup
 # ls /debug/tracing/events/irq
irq_handler_entry  irq_handler_exit

If we add #define TRACE_SYSTEM sched to the trace/sched_event_types.h then the rest of the trace events will be put in a sched directory within the events directory.

I've been playing with this idea of the subsystem for a while, but recently Tom Zanussi posted some patches to lkml that included this method. Tom's approach was clean and got me to finally put some effort to clean up the event trace points.

Thanks to Tom Zanussi for demonstrating how nice the subsystem method is.

---
 kernel/trace/events.c       |    4 +++
 kernel/trace/trace_events.c |   48 +++++++++++++++++++++++++++++++++++++++++++
 kernel/trace/trace_events.h |    2 +
 3 files changed, 54 insertions(+), 0 deletions(-)
kernel/trace/events.c +4 -0
index 46e27ad..4e4e458 100644
@@ -2,6 +2,10 @@
  * This is the place to register all trace points as events.
  */
 
+/* someday this needs to go in a generic header */
+#define __STR(x) #x
+#define STR(x) __STR(x)
+
 #include <trace/trace_events.h>
 
 #include "trace_events.h"
kernel/trace/trace_events.c +48 -0

59 lines not on this page · raw message

kernel/trace/trace_events.h +2 -0
index deb95e5..b015d7b 100644
@@ -7,6 +7,7 @@
 
 struct ftrace_event_call {
 	char		*name;
+	char		*system;
 	struct dentry	*dir;
 	int		enabled;
 	int		(*regfunc)(void);
@@ -44,6 +45,7 @@ static struct ftrace_event_call __used					\
 __attribute__((__aligned__(4)))						\
 __attribute__((section("_ftrace_events"))) event_##call = {		\
 	.name 			= #call,				\
+	.system			= STR(TRACE_SYSTEM),			\
 	.regfunc		= ftrace_reg_event_##call,		\
 	.unregfunc		= ftrace_unreg_event_##call,		\
 }
Signature · 3 lines
1.5.6.5

-- 
author maintainer
#3 lore raw

PATCH 3/10 tracing: make the set_event and available_events subsystem aware

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

This patch makes the event files, set_event and available_events aware of the subsystem.

Now you can enable an entire subsystem with:

echo 'irq:*' > set_event

Note: the '*' is not needed.

---
 kernel/trace/trace_events.c |   45 +++++++++++++++++++++++++++++++++++++++---
 1 files changed, 41 insertions(+), 4 deletions(-)
kernel/trace/trace_events.c +40 -3
index 1933220..b811eb3 100644
@@ -12,6 +12,8 @@
 
 #include "trace_events.h"
 
+#define TRACE_SYSTEM "TRACE_SYSTEM"
+
 #define events_for_each(event)						\
 	for (event = __start_ftrace_events;				\
 	     (unsigned long)event < (unsigned long)__stop_ftrace_events; \
@@ -45,14 +47,47 @@ static void ftrace_clear_events(void)
 static int ftrace_set_clr_event(char *buf, int set)
 {
 	struct ftrace_event_call *call = __start_ftrace_events;
+	char *event = NULL, *sub = NULL, *match;
+	int ret = -EINVAL;
+
+	/*
+	 * The buf format can be <subsystem>:<event-name>
+	 *  *:<event-name> means any event by that name.
+	 *  :<event-name> is the same.
+	 *
+	 *  <subsystem>:* means all events in that subsystem
+	 *  <subsystem>: means the same.
+	 *
+	 *  <name> (no ':') means all events in a subsystem with
+	 *  the name <name> or any event that matches <name>
+	 */
+
+	match = strsep(&buf, ":");
+	if (buf) {
+		sub = match;
+		event = buf;
+		match = NULL;
 
+		if (!strlen(sub) || strcmp(sub, "*") == 0)
+			sub = NULL;
+		if (!strlen(event) || strcmp(event, "*") == 0)
+			event = NULL;
+	}
 
 	events_for_each(call) {
 
 		if (!call->name)
 			continue;
 
-		if (strcmp(buf, call->name) != 0)
+		if (match &&
+		    strcmp(match, call->name) != 0 &&
+		    strcmp(match, call->system) != 0)
+			continue;
+
+		if (sub && strcmp(sub, call->system) != 0)
+			continue;
+
+		if (event && strcmp(event, call->name) != 0)
 			continue;
 
 		if (set) {
@@ -68,9 +103,9 @@ static int ftrace_set_clr_event(char *buf, int set)
 			call->enabled = 0;
 			call->unregfunc();
 		}
-		return 0;
+		ret = 0;
 	}
-	return -EINVAL;
+	return ret;
 }
 
 /* 128 should be much more than enough */
@@ -200,6 +235,8 @@ static int t_show(struct seq_file *m, void *v)
 {
 	struct ftrace_event_call *call = v;
 
+	if (strcmp(call->system, TRACE_SYSTEM) != 0)
+		seq_printf(m, "%s:", call->system);
 	seq_printf(m, "%s\n", call->name);
 
 	return 0;
Signature · 3 lines
1.5.6.5

-- 
author maintainer
#5 lore raw

PATCH 5/10 tracing: add subsystem sched for sched events

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Add the TRACE_SYSTEM sched for the sched events.

---
 include/trace/sched_event_types.h |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
include/trace/sched_event_types.h +5 -0
index a3d3d66..2ada206 100644
@@ -5,6 +5,9 @@
 # error Unless you know what you are doing.
 #endif
 
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM sched
+
 TRACE_FORMAT(sched_kthread_stop,
 	TPPROTO(struct task_struct *t),
 	TPARGS(t),
@@ -70,3 +73,5 @@ TRACE_FORMAT(sched_signal_send,
 	TPPROTO(int sig, struct task_struct *p),
 	TPARGS(sig, p),
 	TPFMT("sig: %d   task %s:%d", sig, p->comm, p->pid));
+
+#undef TRACE_SYSTEM
Signature · 3 lines
1.5.6.5

-- 
author maintainer
#6 lore raw

PATCH 6/10 tracing: add interface to write into current tracer buffer

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Right now all tracers must manage their own trace buffers. This was to enforce tracers to be independent in case we finally decide to allow each tracer to have their own trace buffer.

But now we are adding event tracing that writes to the current tracer's buffer. This adds an interface to allow events to write to the current tracer buffer without having to manage its own. Since event tracing has no "tracer", and is just a way to hook into any other tracer.

---
 kernel/trace/trace.c |   14 ++++++++++++++
 kernel/trace/trace.h |    6 ++++++
 2 files changed, 20 insertions(+), 0 deletions(-)
kernel/trace/trace.c +14 -0
index 9c5987a..c5e39cd 100644
@@ -846,6 +846,20 @@ void trace_buffer_unlock_commit(struct trace_array *tr,
 	trace_wake_up();
 }
 
+struct ring_buffer_event *
+trace_current_buffer_lock_reserve(unsigned char type, unsigned long len,
+				  unsigned long flags, int pc)
+{
+	return trace_buffer_lock_reserve(&global_trace,
+					 type, len, flags, pc);
+}
+
+void trace_current_buffer_unlock_commit(struct ring_buffer_event *event,
+					unsigned long flags, int pc)
+{
+	return trace_buffer_unlock_commit(&global_trace, event, flags, pc);
+}
+
 void
 trace_function(struct trace_array *tr,
 	       unsigned long ip, unsigned long parent_ip, unsigned long flags,
kernel/trace/trace.h +6 -0
index 6321917..adf161f 100644
@@ -442,6 +442,12 @@ void trace_buffer_unlock_commit(struct trace_array *tr,
 				struct ring_buffer_event *event,
 				unsigned long flags, int pc);
 
+struct ring_buffer_event *
+trace_current_buffer_lock_reserve(unsigned char type, unsigned long len,
+				  unsigned long flags, int pc);
+void trace_current_buffer_unlock_commit(struct ring_buffer_event *event,
+					unsigned long flags, int pc);
+
 struct trace_entry *tracing_get_trace_entry(struct trace_array *tr,
 						struct trace_array_cpu *data);
 
Signature · 3 lines
1.5.6.5

-- 
author maintainer
#7 lore raw

PATCH 7/10 tracing: add raw trace point recording infrastructure

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Impact: lower overhead tracing

The current event tracer can automatically pick up trace points that are registered with the TRACE_FORMAT macro. But it required a printf format string and parsing. Although, this adds the ability to get guaranteed information like task names and such, it took a hit in overhead processing. This processing can add about 500-1000 nanoseconds overhead, but in some cases that too is considered too much and we want to shave off as much from this overhead as possible.

Tom Zanussi recently posted tracing patches to lkml that are based on a nice idea about capturing the data via C structs using STRUCT_ENTER, STRUCT_EXIT type of macros.

I liked that method very much, but did not like the implementation that required a developer to add data/code in several disjoint locations.

This patch extends the event_tracer macros to do a similar "raw C" approach that Tom Zanussi did. But instead of having the developers needing to tweak a bunch of code all over the place, they can do it all in one macro - preferably placed near the code that it is tracing. That makes it much more likely that tracepoints will be maintained on an ongoing basis by the code they modify.

The new macro TRACE_EVENT_FORMAT is created for this approach. (Note, a developer may still utilize the more low level DECLARE_TRACE macros if they don't care about getting their traces automatically in the event tracer.)

They can also use the existing TRACE_FORMAT if they don't need to code the tracepoint in C, but just want to use the convenience of printf.

So if the developer wants to "hardwire" a tracepoint in the fastest possible way, and wants to acquire their data via a user space utility in a raw binary format, or wants to see it in the trace output but not sacrifice any performance, then they can implement the faster but more complex TRACE_EVENT_FORMAT macro.

Here's what usage looks like:

TRACE_EVENT_FORMAT(name,

	TPPROTO(proto),
	TPARGS(args),
	TPFMT(fmt, fmt_args),
	TRACE_STUCT(
		TRACE_FIELD(type1, item1, assign1)
		TRACE_FIELD(type2, item2, assign2)
			[...]
	),
	TPRAWFMT(raw_fmt)
	);

Note name, proto, args, and fmt, are all identical to what TRACE_FORMAT uses.

name: is the unique identifier of the trace point
proto: The proto type that the trace point uses
args: the args in the proto type
fmt: printf format to use with the event printf tracer
fmt_args: the printf argments to match fmt

TRACE_STRUCT starts the ability to create a structure. Each item in the structure is defined with a TRACE_FIELD

TRACE_FIELD(type, item, assign)

type: the C type of item.
item: the name of the item in the stucture
assign: what to assign the item in the trace point callback

raw_fmt is a way to pretty print the struct. It must match
the order of the items are added in TRACE_STUCT

An example of this would be:

TRACE_EVENT_FORMAT(sched_wakeup,

	TPPROTO(struct rq *rq, struct task_struct *p, int success),
	TPARGS(rq, p, success),
	TPFMT("task %s:%d %s",
	      p->comm, p->pid, success?"succeeded":"failed"),
	TRACE_STRUCT(
		TRACE_FIELD(pid_t, pid, p->pid)
		TRACE_FIELD(int, success, success)
	),
	TPRAWFMT("task %d success=%d")
	);

This creates us a unique struct of:

 struct {
	pid_t		pid;
	int		success;

};

And the way the call back would assign these values would be:

	entry->pid = p->pid;
	entry->success = success;

The nice part about this is that the creation of the assignent is done via macro magic in the event tracer. Once the TRACE_EVENT_FORMAT is created, the developer will then have a faster method to record into the ring buffer. They do not need to worry about the tracer itself.

The developer would only need to touch the files in include/trace/*.h

Again, I would like to give special thanks to Tom Zanussi for this nice idea.

Idea-from: Tom Zanussi <[email protected]>

---
 kernel/trace/events.c               |    6 +-
 kernel/trace/trace.h                |   19 +++
 kernel/trace/trace_events.c         |    2 +-
 kernel/trace/trace_events.h         |   57 ---------
 kernel/trace/trace_events_stage_1.h |   34 ++++++
 kernel/trace/trace_events_stage_2.h |   72 ++++++++++++
 kernel/trace/trace_events_stage_3.h |  219 +++++++++++++++++++++++++++++++++++
 7 files changed, 350 insertions(+), 59 deletions(-)
 delete mode 100644 kernel/trace/trace_events.h
 create mode 100644 kernel/trace/trace_events_stage_1.h
 create mode 100644 kernel/trace/trace_events_stage_2.h
 create mode 100644 kernel/trace/trace_events_stage_3.h
kernel/trace/events.c +5 -1
index 4e4e458..f2509cb 100644
@@ -8,6 +8,10 @@
 
 #include <trace/trace_events.h>
 
-#include "trace_events.h"
+#include "trace_output.h"
+
+#include "trace_events_stage_1.h"
+#include "trace_events_stage_2.h"
+#include "trace_events_stage_3.h"
 
 #include <trace/trace_event_types.h>
kernel/trace/trace.h +19 -0
index adf161f..aa1ab0c 100644
@@ -726,4 +726,23 @@ static inline void trace_branch_disable(void)
 }
 #endif /* CONFIG_BRANCH_TRACER */
 
+struct ftrace_event_call {
+	char		*name;
+	char		*system;
+	struct dentry	*dir;
+	int		enabled;
+	int		(*regfunc)(void);
+	void		(*unregfunc)(void);
+	int		id;
+	struct dentry	*raw_dir;
+	int		raw_enabled;
+	int		(*raw_init)(void);
+	int		(*raw_reg)(void);
+	void		(*raw_unreg)(void);
+};
+
+void event_trace_printk(unsigned long ip, const char *fmt, ...);
+extern struct ftrace_event_call __start_ftrace_events[];
+extern struct ftrace_event_call __stop_ftrace_events[];
+
 #endif /* _LINUX_KERNEL_TRACE_H */
kernel/trace/trace_events.c +1 -1
index b811eb3..77a5c02 100644
@@ -10,7 +10,7 @@
 #include <linux/module.h>
 #include <linux/ctype.h>
 
-#include "trace_events.h"
+#include "trace.h"
 
 #define TRACE_SYSTEM "TRACE_SYSTEM"
 
kernel/trace/trace_events.h deleted +0 -57

57 lines not on this page · raw message

kernel/trace/trace_events_stage_1.h new file +34 -0
new file mode 100644
index 0000000..fd3bf93
@@ -0,0 +1,34 @@
+/*
+ * Stage 1 of the trace events.
+ *
+ * Override the macros in <trace/trace_event_types.h> to include the following:
+ *
+ * struct ftrace_raw_<call> {
+ *	struct trace_entry		ent;
+ *	<type>				<item>;
+ *	[...]
+ * };
+ *
+ * The <type> <item> is created by the TRACE_FIELD(type, item, assign)
+ * macro. We simply do "type item;", and that will create the fields
+ * in the structure.
+ */
+
+#undef TRACE_FORMAT
+#define TRACE_FORMAT(call, proto, args, fmt)
+
+#undef TRACE_EVENT_FORMAT
+#define TRACE_EVENT_FORMAT(name, proto, args, fmt, tstruct, tpfmt)	\
+	struct ftrace_raw_##name {					\
+		struct trace_entry	ent;				\
+		tstruct							\
+	};								\
+	static struct ftrace_event_call event_##name
+
+#undef TRACE_STRUCT
+#define TRACE_STRUCT(args...) args
+
+#define TRACE_FIELD(type, item, assign) \
+	type item;
+
+#include <trace/trace_event_types.h>
kernel/trace/trace_events_stage_2.h new file +72 -0

72 lines not on this page · raw message

kernel/trace/trace_events_stage_3.h new file +219 -0

219 lines not on this page · raw message

Signature · 3 lines
1.5.6.5

-- 
author maintainer
#13 lore raw

PATCH 10/10 tracing: create the C style tracing for the irq subsystem

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

This patch utilizes the TRACE_EVENT_FORMAT macro to enable the C style faster tracing for the irq subsystem trace points.

---
 include/trace/irq_event_types.h |   19 +++++++++++++++----
 1 files changed, 15 insertions(+), 4 deletions(-)
include/trace/irq_event_types.h +15 -4
index 47a2be1..65850bc 100644
@@ -8,15 +8,26 @@
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM irq
 
-TRACE_FORMAT(irq_handler_entry,
+TRACE_EVENT_FORMAT(irq_handler_entry,
 	TPPROTO(int irq, struct irqaction *action),
 	TPARGS(irq, action),
-	TPFMT("irq=%d handler=%s", irq, action->name));
+	TPFMT("irq=%d handler=%s", irq, action->name),
+	TRACE_STRUCT(
+		TRACE_FIELD(int, irq, irq)
+	),
+	TPRAWFMT("irq %d")
+	);
 
-TRACE_FORMAT(irq_handler_exit,
+TRACE_EVENT_FORMAT(irq_handler_exit,
 	TPPROTO(int irq, struct irqaction *action, int ret),
 	TPARGS(irq, action, ret),
 	TPFMT("irq=%d handler=%s return=%s",
-		irq, action->name, ret ? "handled" : "unhandled"));
+		irq, action->name, ret ? "handled" : "unhandled"),
+	TRACE_STRUCT(
+		TRACE_FIELD(int, irq, irq)
+		TRACE_FIELD(int, ret, ret)
+	),
+	TPRAWFMT("irq %d ret %d")
+	);
 
 #undef TRACE_SYSTEM
Signature · 3 lines
1.5.6.5

-- 
author maintainer
#11 lore raw

PATCH 8/10 tracing: add raw fast tracing interface for trace events

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

This patch adds the interface to enable the C style trace points. In the directory /debugfs/tracing/events/subsystem/event We now have three files:

enable : values 0 or 1 to enable or disable the trace event.

available_types: values 'raw' and 'printf' which indicate the tracing
types available for the trace point. If a developer does not use the TRACE_EVENT_FORMAT macro and just uses the TRACE_FORMAT macro, then only 'printf' will be available. This file is read only.

type: values 'raw' or 'printf'. This indicates which type of tracing
is active for that trace point. 'printf' is the default and if 'raw' is not available, this file is read only.

 # echo raw > /debug/tracing/events/sched/sched_wakeup/type
 # echo 1 > /debug/tracing/events/sched/sched_wakeup/enable

Will enable the C style tracing for the sched_wakeup trace point.

---
 kernel/trace/trace.h        |    7 ++
 kernel/trace/trace_events.c |  199 +++++++++++++++++++++++++++++++++++++------
 2 files changed, 181 insertions(+), 25 deletions(-)
kernel/trace/trace.h +7 -0
index aa1ab0c..f6fa0b9 100644
@@ -726,6 +726,12 @@ static inline void trace_branch_disable(void)
 }
 #endif /* CONFIG_BRANCH_TRACER */
 
+/* trace event type bit fields, not numeric */
+enum {
+	TRACE_EVENT_TYPE_PRINTF		= 1,
+	TRACE_EVENT_TYPE_RAW		= 2,
+};
+
 struct ftrace_event_call {
 	char		*name;
 	char		*system;
@@ -736,6 +742,7 @@ struct ftrace_event_call {
 	int		id;
 	struct dentry	*raw_dir;
 	int		raw_enabled;
+	int		type;
 	int		(*raw_init)(void);
 	int		(*raw_reg)(void);
 	void		(*raw_unreg)(void);
kernel/trace/trace_events.c +174 -25

254 lines not on this page · raw message

Signature · 3 lines
1.5.6.5

-- 
author maintainer
#12 lore raw

PATCH 9/10 tracing: create the C style tracing for the sched subsystem

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

This patch utilizes the TRACE_EVENT_FORMAT macro to enable the C style faster tracing for the sched subsystem trace points.

---
 include/linux/tracepoint.h        |    3 +
 include/trace/sched_event_types.h |  119 +++++++++++++++++++++++++++++--------
 2 files changed, 97 insertions(+), 25 deletions(-)
include/linux/tracepoint.h +3 -0
index 62d1339..152b2f0 100644
@@ -157,4 +157,7 @@ static inline void tracepoint_synchronize_unregister(void)
 #define TRACE_FORMAT(name, proto, args, fmt)		\
 	DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
 
+#define TRACE_EVENT_FORMAT(name, proto, args, fmt, struct, tpfmt)	\
+	TRACE_FORMAT(name, PARAMS(proto), PARAMS(args), PARAMS(fmt))
+
 #endif
include/trace/sched_event_types.h +94 -25

170 lines not on this page · raw message

Signature · 3 lines
1.5.6.5

-- 
#14 lore raw
In reply to Steven Rostedt
Quoted 8 lines · Steven Rostedt

Ingo,

Please pull the latest tip/tracing/ftrace tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
tip/tracing/ftrace

Quoted 11 lines

Steven Rostedt (10):
tracing: move trace point formats to files in include/trace directory
tracing: add subsystem level to trace events
tracing: make the set_event and available_events subsystem aware
tracing: add subsystem irq for irq events
tracing: add subsystem sched for sched events
tracing: add interface to write into current tracer buffer
tracing: add raw trace point recording infrastructure
tracing: add raw fast tracing interface for trace events
tracing: create the C style tracing for the sched subsystem
tracing: create the C style tracing for the irq subsystem

Great stuff!

Pulled into tip:tracing/ftrace, thanks Steve!

Ingo

#8 lore raw

Re: [PATCH 07/10] tracing: add raw trace point recording infrastructure

In reply to Steven Rostedt

Hi,

Quoted 102 lines · Steven Rostedt

plain text document attachment (0007-tracing-add-raw-trace-point-recording-infrastructur.patch)
From: Steven Rostedt <[email protected]>

Impact: lower overhead tracing

The current event tracer can automatically pick up trace points that are registered with the TRACE_FORMAT macro. But it required a printf format string and parsing. Although, this adds the ability to get guaranteed information like task names and such, it took a hit in overhead processing. This processing can add about 500-1000 nanoseconds overhead, but in some cases that too is considered too much and we want to shave off as much from this overhead as possible.

Tom Zanussi recently posted tracing patches to lkml that are based on a nice idea about capturing the data via C structs using STRUCT_ENTER, STRUCT_EXIT type of macros.

I liked that method very much, but did not like the implementation that required a developer to add data/code in several disjoint locations.

This patch extends the event_tracer macros to do a similar "raw C" approach that Tom Zanussi did. But instead of having the developers needing to tweak a bunch of code all over the place, they can do it all in one macro - preferably placed near the code that it is tracing. That makes it much more likely that tracepoints will be maintained on an ongoing basis by the code they modify.

The new macro TRACE_EVENT_FORMAT is created for this approach. (Note, a developer may still utilize the more low level DECLARE_TRACE macros if they don't care about getting their traces automatically in the event tracer.)

They can also use the existing TRACE_FORMAT if they don't need to code the tracepoint in C, but just want to use the convenience of printf.

So if the developer wants to "hardwire" a tracepoint in the fastest possible way, and wants to acquire their data via a user space utility in a raw binary format, or wants to see it in the trace output but not sacrifice any performance, then they can implement the faster but more complex TRACE_EVENT_FORMAT macro.

Here's what usage looks like:

TRACE_EVENT_FORMAT(name,

	TPPROTO(proto),
	TPARGS(args),
	TPFMT(fmt, fmt_args),
	TRACE_STUCT(
		TRACE_FIELD(type1, item1, assign1)
		TRACE_FIELD(type2, item2, assign2)
			[...]
	),
	TPRAWFMT(raw_fmt)
	);

Note name, proto, args, and fmt, are all identical to what TRACE_FORMAT uses.

name: is the unique identifier of the trace point
proto: The proto type that the trace point uses
args: the args in the proto type
fmt: printf format to use with the event printf tracer
fmt_args: the printf argments to match fmt

TRACE_STRUCT starts the ability to create a structure. Each item in the structure is defined with a TRACE_FIELD

TRACE_FIELD(type, item, assign)

type: the C type of item.
item: the name of the item in the stucture
assign: what to assign the item in the trace point callback

raw_fmt is a way to pretty print the struct. It must match
the order of the items are added in TRACE_STUCT

An example of this would be:

TRACE_EVENT_FORMAT(sched_wakeup,

	TPPROTO(struct rq *rq, struct task_struct *p, int success),
	TPARGS(rq, p, success),
	TPFMT("task %s:%d %s",
	      p->comm, p->pid, success?"succeeded":"failed"),
	TRACE_STRUCT(
		TRACE_FIELD(pid_t, pid, p->pid)
		TRACE_FIELD(int, success, success)
	),
	TPRAWFMT("task %d success=%d")
	);

This creates us a unique struct of:

 struct {
	pid_t		pid;
	int		success;

};

And the way the call back would assign these values would be:

Quoted 8 lines
	entry->pid = p->pid;
	entry->success = success;

The nice part about this is that the creation of the assignent is done via macro magic in the event tracer. Once the TRACE_EVENT_FORMAT is created, the developer will then have a faster method to record into the ring buffer. They do not need to worry about the tracer itself.

Nice improvements - I definitely was unhappy about having things spread around in different files unnecessarily. And I like the fact that your macros generate assignments too but am curious about what to do if you need to do something more complicated than an assignment e.g. in the block tracepoints I had to assign fields differently based on the value of blk_pc_request():

       if (blk_pc_request(rq)) {
               zed_event->sector = 0;
               zed_event->bytes = rq->data_len;
               zed_event->pdu_len = pdu_len;
               memcpy(zed_event->pdu, rq->cmd, pdu_len);
       } else {
               zed_event->sector = rq->hard_sector;
               zed_event->bytes = rq->hard_nr_sectors << 9;
               zed_event->pdu_len = 0;
       }

Is there a way to define some fields but without the assignments, and do them manually somewhere else? I guess it would be nice to be able to define all events using TRACE_EVENT_FORMAT but have a way to special case certain events/fields.

Anyway, sorry if it's already handled in the code - haven't had a chance to really peruse it.

Tom

#9 lore raw

Re: [PATCH 07/10] tracing: add raw trace point recording infrastructure

2009/3/2 Tom Zanussi <[email protected]>:

Quoted 132 lines

Hi,

Quoted 102 lines · Steven Rostedt

plain text document attachment (0007-tracing-add-raw-trace-point-recording-infrastructur.patch)
From: Steven Rostedt <[email protected]>

Impact: lower overhead tracing

The current event tracer can automatically pick up trace points that are registered with the TRACE_FORMAT macro. But it required a printf format string and parsing. Although, this adds the ability to get guaranteed information like task names and such, it took a hit in overhead processing. This processing can add about 500-1000 nanoseconds overhead, but in some cases that too is considered too much and we want to shave off as much from this overhead as possible.

Tom Zanussi recently posted tracing patches to lkml that are based on a nice idea about capturing the data via C structs using STRUCT_ENTER, STRUCT_EXIT type of macros.

I liked that method very much, but did not like the implementation that required a developer to add data/code in several disjoint locations.

This patch extends the event_tracer macros to do a similar "raw C" approach that Tom Zanussi did. But instead of having the developers needing to tweak a bunch of code all over the place, they can do it all in one macro - preferably placed near the code that it is tracing. That makes it much more likely that tracepoints will be maintained on an ongoing basis by the code they modify.

The new macro TRACE_EVENT_FORMAT is created for this approach. (Note, a developer may still utilize the more low level DECLARE_TRACE macros if they don't care about getting their traces automatically in the event tracer.)

They can also use the existing TRACE_FORMAT if they don't need to code the tracepoint in C, but just want to use the convenience of printf.

So if the developer wants to "hardwire" a tracepoint in the fastest possible way, and wants to acquire their data via a user space utility in a raw binary format, or wants to see it in the trace output but not sacrifice any performance, then they can implement the faster but more complex TRACE_EVENT_FORMAT macro.

Here's what usage looks like:

  TRACE_EVENT_FORMAT(name,
      TPPROTO(proto),
      TPARGS(args),
      TPFMT(fmt, fmt_args),
      TRACE_STUCT(
              TRACE_FIELD(type1, item1, assign1)
              TRACE_FIELD(type2, item2, assign2)
                      [...]
      ),
      TPRAWFMT(raw_fmt)
      );

Note name, proto, args, and fmt, are all identical to what TRACE_FORMAT uses.

 name: is the unique identifier of the trace point
 proto: The proto type that the trace point uses
 args: the args in the proto type
 fmt: printf format to use with the event printf tracer
 fmt_args: the printf argments to match fmt

 TRACE_STRUCT starts the ability to create a structure. Each item in the structure is defined with a TRACE_FIELD

  TRACE_FIELD(type, item, assign)

 type: the C type of item.
 item: the name of the item in the stucture
 assign: what to assign the item in the trace point callback

 raw_fmt is a way to pretty print the struct. It must match the order of the items are added in TRACE_STUCT

 An example of this would be:

 TRACE_EVENT_FORMAT(sched_wakeup,
      TPPROTO(struct rq *rq, struct task_struct *p, int success), TPARGS(rq, p, success),
      TPFMT("task %s:%d %s",
            p->comm, p->pid, success?"succeeded":"failed"), TRACE_STRUCT(
              TRACE_FIELD(pid_t, pid, p->pid) TRACE_FIELD(int, success, success)
      ),
      TPRAWFMT("task %d success=%d")
      );

 This creates us a unique struct of:

 struct {
      pid_t           pid;
      int             success;
 };

 And the way the call back would assign these values would be:

Quoted 8 lines

      entry->pid = p->pid;
      entry->success = success;

The nice part about this is that the creation of the assignent is done via macro magic in the event tracer.  Once the TRACE_EVENT_FORMAT is created, the developer will then have a faster method to record into the ring buffer. They do not need to worry about the tracer itself.

Nice improvements - I definitely was unhappy about having things spread around in different files unnecessarily.  And I like the fact that your macros generate assignments too but am curious about what to do if you need to do something more complicated than an assignment e.g. in the block tracepoints I had to assign fields differently based on the value of blk_pc_request():

      if (blk_pc_request(rq)) {
              zed_event->sector = 0; zed_event->bytes = rq->data_len; zed_event->pdu_len = pdu_len; memcpy(zed_event->pdu, rq->cmd, pdu_len); } else {
              zed_event->sector = rq->hard_sector; zed_event->bytes = rq->hard_nr_sectors << 9; zed_event->pdu_len = 0;
      }

Quoted 4 lines

Is there a way to define some fields but without the assignments, and do them manually somewhere else?  I guess it would be nice to be able to define all events using TRACE_EVENT_FORMAT but have a way to special case certain events/fields.

Note that on such case you can do a conditional assignment:

TRACE_FIELD(int, bytes, blk_pc_request(rq) ? rq->data_len :
rq->hard_nr_sectors << 9);

The drawback here is that you'll have to repeat this conditional for each field, which is annoying
and a loss of performance.

Perhaps it would be interesting to allow something more low level if the user wishes.

Quoted 8 lines

Anyway, sorry if it's already handled in the code - haven't had a chance to really peruse it.

Tom

author maintainer
#10 lore raw

Re: [PATCH 07/10] tracing: add raw trace point recording infrastructure

In reply to Tom Zanussi
Quoted 50 lines · Tom Zanussi
Quoted 23 lines

An example of this would be:

TRACE_EVENT_FORMAT(sched_wakeup,

	TPPROTO(struct rq *rq, struct task_struct *p, int success),
	TPARGS(rq, p, success),
	TPFMT("task %s:%d %s",
	      p->comm, p->pid, success?"succeeded":"failed"),
	TRACE_STRUCT(
		TRACE_FIELD(pid_t, pid, p->pid)
		TRACE_FIELD(int, success, success)
	),
	TPRAWFMT("task %d success=%d")
	);

This creates us a unique struct of:

 struct {
	pid_t		pid;
	int		success;

};

And the way the call back would assign these values would be:

Quoted 8 lines
	entry->pid = p->pid;
	entry->success = success;

The nice part about this is that the creation of the assignent is done via macro magic in the event tracer. Once the TRACE_EVENT_FORMAT is created, the developer will then have a faster method to record into the ring buffer. They do not need to worry about the tracer itself.

Nice improvements - I definitely was unhappy about having things spread around in different files unnecessarily. And I like the fact that your macros generate assignments too but am curious about what to do if you need to do something more complicated than an assignment e.g. in the block tracepoints I had to assign fields differently based on the value of blk_pc_request():

       if (blk_pc_request(rq)) {
               zed_event->sector = 0;
               zed_event->bytes = rq->data_len;
               zed_event->pdu_len = pdu_len;
               memcpy(zed_event->pdu, rq->cmd, pdu_len);
       } else {
               zed_event->sector = rq->hard_sector;
               zed_event->bytes = rq->hard_nr_sectors << 9;
               zed_event->pdu_len = 0;
       }
Quoted 7 lines

Is there a way to define some fields but without the assignments, and do them manually somewhere else? I guess it would be nice to be able to define all events using TRACE_EVENT_FORMAT but have a way to special case certain events/fields.

Anyway, sorry if it's already handled in the code - haven't had a chance to really peruse it.

Nope, you are right, it is not handled... yet ;-)

I was thinking about adding TRACE_FIELD_SPECIAL() that would allow for a different means to copy the field.

	TRACE_FIELD_SPECIAL(char*, pdu, rec,
		TPCMD(memcpy((rec)->pdu, rq->cmd, pdu_len));

Where, rec would have your "zed_event", and the fourth argument would have the way to handle that field.

I have not tried that yet, but I think this, or something similar, could work.

Thanks,

-- Steve