Skip to contentSkip to conversation

tracing: Have trace_printk()s in the events/ directory

RFC PATCH In review Steven Rostedt 5 participants 19 messages Nov 18, 2010

Subsystems (2)
lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

cover tracing: Have trace_printk()s in the events/ directory

As suggested in:

LKML-Reference: <20101116221726.GB26243@nowhere>
or
http://marc.info/?l=linux-kernel&m=128994587230629

To have trace_printks in the events directory.

This patch set just does it for ftrace, although it can also lay the ground work for both perf and trace (are they different tools?)

This patch set converts the current trace_printk() into ftrace_printk() (back to it's original name) and that stays the same.

The new trace_printk() is off by default (we can change that later with a kernel command line option perhaps), but it can be enable now with the events directory.

Each trace_printk() added will get a entry in the debugfs/tracing/events/ directory as debugfs/tracing/entry/path/to/trace_printk/

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

The format is the printk format:

# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d
"

Every directory has an "enable" file. By echoing 0 or 1 into this file, it will enable all the trace_printk's below it.

This also uses jump_label() to keep overhead down when disabled.

Note, modules are not (yet) supported.

This is just the ground work. I'm busy doing other things so if you need more, feel free to take this and carry on.

-- Steve

The following patches are in:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git

branch: rfc/event-printk

Steven Rostedt (2):
tracing: Rename trace_printk to ftrace_printk
tracing: Make event based trace_printk()

----

 include/asm-generic/vmlinux.lds.h |    7 +-
 include/linux/kernel.h            |   16 +-
 include/trace/event_printk.h      |   52 ++++++
 kernel/trace/Makefile             |    1 +
 kernel/trace/event_printk.c       |  365 +++++++++++++++++++++++++++++++++++++
 kernel/trace/trace.h              |    1 +
 kernel/trace/trace_events.c       |    2 +-
 7 files changed, 435 insertions(+), 9 deletions(-)
author maintainer
#1 lore raw

RFC PATCH 1/2 tracing: Rename trace_printk to ftrace_printk

From: Steven Rostedt <[email protected]>

In order to make a bit more generic trace_printk(), this patch simply renames the trace_printk() macro to its original ftrace_printk() name, since it only writes into the ftrace ring buffer.

---
 include/linux/kernel.h |   14 +++++++-------
 1 files changed, 7 insertions(+), 7 deletions(-)
include/linux/kernel.h +7 -7
index b6de9a6..12151e5 100644
@@ -318,11 +318,11 @@ do {									\
 } while (0)
 
 /**
- * trace_printk - printf formatting in the ftrace buffer
+ * ftrace_printk - printf formatting in the ftrace buffer
  * @fmt: the printf format for printing
  *
- * Note: __trace_printk is an internal function for trace_printk and
- *       the @ip is passed in via the trace_printk macro.
+ * Note: __trace_printk is an internal function for ftrace_printk and
+ *       the @ip is passed in via the ftrace_printk macro.
  *
  * This function allows a kernel developer to debug fast path sections
  * that printk is not appropriate for. By scattering in various
@@ -330,11 +330,11 @@ do {									\
  * where problems are occurring.
  *
  * This is intended as a debugging tool for the developer only.
- * Please refrain from leaving trace_printks scattered around in
+ * Please refrain from leaving ftrace_printks scattered around in
  * your code.
  */
 
-#define trace_printk(fmt, args...)					\
+#define ftrace_printk(fmt, args...)					\
 do {									\
 	__trace_printk_check_format(fmt, ##args);			\
 	if (__builtin_constant_p(fmt)) {				\
@@ -383,14 +383,14 @@ __ftrace_vprintk(unsigned long ip, const char *fmt, va_list ap);
 extern void ftrace_dump(enum ftrace_dump_mode oops_dump_mode);
 #else
 static inline int
-trace_printk(const char *fmt, ...) __attribute__ ((format (printf, 1, 2)));
+ftrace_printk(const char *fmt, ...) __attribute__ ((format (printf, 1, 2)));
 
 static inline void tracing_start(void) { }
 static inline void tracing_stop(void) { }
 static inline void ftrace_off_permanent(void) { }
 static inline void trace_dump_stack(void) { }
 static inline int
-trace_printk(const char *fmt, ...)
+ftrace_printk(const char *fmt, ...)
 {
 	return 0;
 }
Signature · 1 line
1.7.1
author maintainer
#2 lore raw

RFC PATCH 2/2 tracing: Make event based trace_printk()

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Add the trace_printk() that places the usage in the /debug/tracing/events/printk directory.

Adding a trace_printk() into sched.c at line 2181, we have:

[root@bxf ~]# ls -R /debug/tracing/events/printk/
/debug/tracing/events/printk/:
enable kernel

/debug/tracing/events/printk/kernel:
enable sched.c

/debug/tracing/events/printk/kernel/sched.c:
2180 enable

/debug/tracing/events/printk/kernel/sched.c/2180:
enable format

and

[root@bxf ~]# cat /debug/tracing/events/printk/kernel/sched.c/2180/format "migrate task %s:%d
"

Note, although the trace_printk's are now in the event directories, the top level events/enable file does not enable or disable trace_printk events.

---
 include/asm-generic/vmlinux.lds.h |    7 +-
 include/linux/kernel.h            |    2 +
 include/trace/event_printk.h      |   52 ++++++
 kernel/trace/Makefile             |    1 +
 kernel/trace/event_printk.c       |  365 +++++++++++++++++++++++++++++++++++++
 kernel/trace/trace.h              |    1 +
 kernel/trace/trace_events.c       |    2 +-
 7 files changed, 428 insertions(+), 2 deletions(-)
 create mode 100644 include/trace/event_printk.h
 create mode 100644 kernel/trace/event_printk.c
include/asm-generic/vmlinux.lds.h +6 -1
index bd69d79..e74dac7 100644
@@ -131,10 +131,14 @@
 #endif
 
 #ifdef CONFIG_TRACING
-#define TRACE_PRINTKS() VMLINUX_SYMBOL(__start___trace_bprintk_fmt) = .;      \
+#define FTRACE_PRINTKS() VMLINUX_SYMBOL(__start___trace_bprintk_fmt) = .;      \
 			 *(__trace_printk_fmt) /* Trace_printk fmt' pointer */ \
 			 VMLINUX_SYMBOL(__stop___trace_bprintk_fmt) = .;
+#define TRACE_PRINTKS()	VMLINUX_SYMBOL(__start_trace_printk) = .;   \
+				*(_trace_printk)		      \
+				VMLINUX_SYMBOL(__stop_trace_printk) = .;
 #else
+#define FTRACE_PRINTKS()
 #define TRACE_PRINTKS()
 #endif
 
@@ -169,6 +173,7 @@
 	LIKELY_PROFILE()		       				\
 	BRANCH_PROFILE()						\
 	TRACE_PRINTKS()							\
+	FTRACE_PRINTKS()						\
 									\
 	STRUCT_ALIGN();							\
 	FTRACE_EVENTS()							\
include/linux/kernel.h +2 -0
index 12151e5..8a376d5 100644
@@ -402,6 +402,8 @@ ftrace_vprintk(const char *fmt, va_list ap)
 static inline void ftrace_dump(enum ftrace_dump_mode oops_dump_mode) { }
 #endif /* CONFIG_TRACING */
 
+#include <trace/event_printk.h>
+
 /*
  * min()/max()/clamp() macros that also do
  * strict type-checking.. See the
include/trace/event_printk.h new file +52 -0

52 lines not on this page · raw message

kernel/trace/Makefile +1 -0
index 53f3381..44bea92 100644
@@ -29,6 +29,7 @@ obj-$(CONFIG_TRACING) += trace.o
 obj-$(CONFIG_TRACING) += trace_output.o
 obj-$(CONFIG_TRACING) += trace_stat.o
 obj-$(CONFIG_TRACING) += trace_printk.o
+obj-$(CONFIG_TRACING) += event_printk.o
 obj-$(CONFIG_CONTEXT_SWITCH_TRACER) += trace_sched_switch.o
 obj-$(CONFIG_FUNCTION_TRACER) += trace_functions.o
 obj-$(CONFIG_IRQSOFF_TRACER) += trace_irqsoff.o
kernel/trace/event_printk.c new file +365 -0

365 lines not on this page · raw message

kernel/trace/trace.h +1 -0
index 9021f8c..c99c79f 100644
@@ -295,6 +295,7 @@ struct dentry *trace_create_file(const char *name,
 				 const struct file_operations *fops);
 
 struct dentry *tracing_init_dentry(void);
+struct dentry *event_trace_events_dir(void);
 
 struct ring_buffer_event;
 
kernel/trace/trace_events.c +1 -1
index 0725eea..4e96e09 100644
@@ -975,7 +975,7 @@ static const struct file_operations ftrace_show_header_fops = {
 	.llseek = default_llseek,
 };
 
-static struct dentry *event_trace_events_dir(void)
+struct dentry *event_trace_events_dir(void)
 {
 	static struct dentry *d_tracer;
 	static struct dentry *d_events;
Signature · 1 line
1.7.1
author maintainer
#3 lore raw

Re: [RFC][PATCH 2/2] [PATCH 2/2] tracing: Make event based trace_printk()

Quoted 36 lines · Steven Rostedt
kernel/trace/event_printk.c new file +36 -0
+static __init int event_printk_init(void)
+{
+	struct event_printk_struct *print;
+	struct dentry *d_events;
+	char *topdir;
+	char *p;
+	int len;
+	int ret = -1;
+
+	/*
+	 * Do nothing if we have no trace printks.
+	 * May change in the future when we support modules.
+	 */
+	if (__start_trace_printk == __stop_trace_printk)
+		return 0;
+
+	/*
+	 * Use this file to find the path for other
+	 * files. That is, __FILE__ returns the full path,
+	 * but we don't want the top dir path, so we must
+	 * chop it off. We know the path of this file as it
+	 * is (kernel/trace/event_printk.c), so we simply
+	 * cut that off and then we have the path to cut
+	 * off of other files.
+	 */
+	topdir = kstrdup(__FILE__, GFP_KERNEL);
+	if (WARN(!topdir, "failed to allocate printk info"))
+		return -1;
+
+	p = strstr(topdir, "kernel/trace/event_printk.c");
+	if (WARN(!p, "event_printk.c not found. Did the file move?"))
+		goto out;
+	*p = '\0';
+
+	d_events = event_trace_events_dir();
+

kernel/trace/event_printk.c line 359

Quoted 10 lines
kernel/trace/event_printk.c new file +10 -0
+	printk_root = debugfs_create_dir("printk", d_events);
+	if (WARN(!printk_root, "Could not create printk entry"))
+		goto out;
+
+	trace_create_file("enable", 0644, printk_root,
+			  NULL, &trace_printk_enable_fops);
+
+	len = strlen(topdir);
+	for_each_printk(print, __start_trace_printk, __stop_trace_printk)
+		add_event_printk(topdir, len, print);

Oops, I'm missing a:

	ret = 0;

here.

-- Steve

Quoted 5 lines
kernel/trace/event_printk.c new file +5 -0
+
+ out:
+	kfree(topdir);
+	return ret;
+}
#4 lore raw

Re: [RFC][PATCH 2/2] [PATCH 2/2] tracing: Make event based trace_printk()

In reply to Steven Rostedt

include/trace/event_printk.h line 32

Quoted 7 lines · Steven Rostedt
include/trace/event_printk.h new file +7 -0
+#define trace_printk(fmt, args...)					\
+	do {								\
+		static struct event_printk_struct			\
+		__attribute__((__aligned__(4)))				\
+		__attribute__((section("_trace_printk")))		\
+		______t = {						\
+			.format = fmt,					\

if fmt is not constant, the compiler will complain...

include/trace/event_printk.h line 39

Quoted 7 lines
include/trace/event_printk.h new file +7 -0
+			.func = __func__,				\
+			.file = __FILE__,				\
+			.line = __LINE__,				\
+		};							\
+		__trace_printk_check_format(fmt, ##args);		\
+		if (!__builtin_constant_p(fmt))				\
+			trace_printk_can_only_have_constant_format();	\

so this is not need.

Quoted 3 lines
include/trace/event_printk.h new file +3 -0
+		if (unlikely(__event_printk_test(&______t.enable)))	\
+			__trace_bprintk(_THIS_IP_, fmt, ##args); \
+	} while (0)
#8 lore raw
In reply to Steven Rostedt
Quoted 10 lines · Steven Rostedt

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

The format is the printk format:

# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d"

*groan*, so you're creating a tracepoint per instance?

That's going to be massive pain for perf.. I really don't see the point in splitting all that out.

author maintainer
#9 lore raw
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Steven Rostedt

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

The format is the printk format:

Quoted 7 lines
Quoted 2 lines
# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d"

*groan*, so you're creating a tracepoint per instance?

That's going to be massive pain for perf.. I really don't see the point in splitting all that out.

a) The file directory was what was asked about in the referenced email.
b) This is just an example of a way to display it to the user, which seems to be very intuitive.
c) Perf can implement the details anyway it wants. It can make a single tracepoint callback and have the enabling of the points as a special filter.
d) This was just an RFC that Frederic asked if I would do. I thought it would be a fun challenge and did it. Let it bit rot in hell for all I care, I wasn't taking it any further anyway.

-- Steve

#6 lore raw

Re: [RFC][PATCH 2/2] [PATCH 2/2] tracing: Make event based trace_printk()

In reply to Steven Rostedt
Quoted 6 lines · * Steven Rostedt ([email protected]) wrote:

From: Steven Rostedt <[email protected]>

Add the trace_printk() that places the usage in the /debug/tracing/events/printk directory.

Adding a trace_printk() into sched.c at line 2181, we have:

I'm curious: how do you handle two trace_printk on the same line ?

It might be OK to let the interface activate both at once, but nothing should crash per se.

Thanks,

Mathieu

Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
#10 lore raw
In reply to Steven Rostedt
Quoted 18 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Steven Rostedt

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

The format is the printk format:

Quoted 7 lines
Quoted 2 lines
# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d"

*groan*, so you're creating a tracepoint per instance?

That's going to be massive pain for perf.. I really don't see the point in splitting all that out.

Quoted 9 lines

a) The file directory was what was asked about in the referenced email.
b) This is just an example of a way to display it to the user, which seems to be very intuitive.
c) Perf can implement the details anyway it wants. It can make a single tracepoint callback and have the enabling of the points as a special filter.
d) This was just an RFC that Frederic asked if I would do. I thought it would be a fun challenge and did it. Let it bit rot in hell for all I care, I wasn't taking it any further anyway.

Hi Steven,

The LTTng tree still keeps the "trace_mark()" kernel markers, which are very very similar to ftrace_printk(). I'd be happy to combine the two eventually. This one file per line approach seems very good -- rather than explicitely naming each instance, as I did in trace_mark(), you end up automatically naming them by file/line number.

How does this behave from within static inlines called multiple times and unrolled loops ?

Thanks,

Mathieu

Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
author maintainer
#11 lore raw
Quoted 31 lines · Mathieu Desnoyers
Quoted 18 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Steven Rostedt

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

The format is the printk format:

Quoted 7 lines
Quoted 2 lines
# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d"

*groan*, so you're creating a tracepoint per instance?

That's going to be massive pain for perf.. I really don't see the point in splitting all that out.

Quoted 9 lines

a) The file directory was what was asked about in the referenced email.
b) This is just an example of a way to display it to the user, which seems to be very intuitive.
c) Perf can implement the details anyway it wants. It can make a single tracepoint callback and have the enabling of the points as a special filter.
d) This was just an RFC that Frederic asked if I would do. I thought it would be a fun challenge and did it. Let it bit rot in hell for all I care, I wasn't taking it any further anyway.

Hi Steven,

Quoted 8 lines

The LTTng tree still keeps the "trace_mark()" kernel markers, which are very very similar to ftrace_printk(). I'd be happy to combine the two eventually. This one file per line approach seems very good -- rather than explicitely naming each instance, as I did in trace_mark(), you end up automatically naming them by file/line number.

How does this behave from within static inlines called multiple times and unrolled loops ?

Actually you mean macros:

You'll get a 2181-1, 2181-2, 2181-3, etc.

Although a trace_printk() in a header static inline may be interesting.

-- Steve

author maintainer
#7 lore raw

Re: [RFC][PATCH 2/2] [PATCH 2/2] tracing: Make event based trace_printk()

In reply to Mathieu Desnoyers
Quoted 6 lines · Mathieu Desnoyers
Quoted 5 lines · * Steven Rostedt ([email protected]) wrote:

From: Steven Rostedt <[email protected]>

Add the trace_printk() that places the usage in the /debug/tracing/events/printk directory.

Quoted 6 lines
Quoted 1 line

Adding a trace_printk() into sched.c at line 2181, we have:

I'm curious: how do you handle two trace_printk on the same line ?

It might be OK to let the interface activate both at once, but nothing should crash per se.

You get 2181, and 2181-1

-- Steve

author maintainer
#5 lore raw

Re: [RFC][PATCH 2/2] [PATCH 2/2] tracing: Make event based trace_printk()

In reply to Lai Jiangshan
Quoted 10 lines · Lai Jiangshan

include/trace/event_printk.h line 32

Quoted 7 lines · Steven Rostedt
include/trace/event_printk.h new file +7 -0
+#define trace_printk(fmt, args...)					\
+	do {								\
+		static struct event_printk_struct			\
+		__attribute__((__aligned__(4)))				\
+		__attribute__((section("_trace_printk")))		\
+		______t = {						\
+			.format = fmt,					\

if fmt is not constant, the compiler will complain...

Ah you are right. I added that after I wrote all this code and wanted the format to print in the file. The original code required the compile check below.

-- Steve

Quoted 14 lines

include/trace/event_printk.h line 39

Quoted 8 lines
include/trace/event_printk.h new file +7 -0
+			.func = __func__,				\
+			.file = __FILE__,				\
+			.line = __LINE__,				\
+		};							\
+		__trace_printk_check_format(fmt, ##args);		\
+		if (!__builtin_constant_p(fmt))				\
+			trace_printk_can_only_have_constant_format();	\

so this is not need.

Quoted 3 lines
include/trace/event_printk.h new file +3 -0
+		if (unlikely(__event_printk_test(&______t.enable)))	\
+			__trace_bprintk(_THIS_IP_, fmt, ##args); \
+	} while (0)
#18 lore raw
In reply to Steven Rostedt
Quoted 7 lines · Steven Rostedt

As suggested in:

LKML-Reference: <20101116221726.GB26243@nowhere>
or
http://marc.info/?l=linux-kernel&m=128994587230629

To have trace_printks in the events directory.

(Note, I don't remember who got that idea. Perhaps that was Ingo while discussing that with him like one year ago, or may be me, not sure, whatever)...

Quoted 2 lines

This patch set just does it for ftrace, although it can also lay the ground work for both perf and trace (are they different tools?)

The trace tool uses perf. But no worry about perf, I can handle the hook for it.

Quoted 6 lines

This patch set converts the current trace_printk() into ftrace_printk() (back to it's original name) and that stays the same.

The new trace_printk() is off by default (we can change that later with a kernel command line option perhaps), but it can be enable now with the events directory.

Cool. So disablement/re-enablement are nice features. That said it should be enabled by default I think, to keep the previous (and intuitive) behaviour.

Quoted 8 lines

Each trace_printk() added will get a entry in the debugfs/tracing/events/ directory as debugfs/tracing/entry/path/to/trace_printk/

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

Ok, we'll probably need to add the ID as well.

Quoted 5 lines

The format is the printk format:

# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d
"

Do you think we can make it more compatible with the common pattern?

Like:

Name: sched.c:2180
ID: 120
format:

	field:unsigned long ip;	offset:16;	size:8;	signed:0;
	field:const char * fmt;	offset:24;	size:8;	signed:0; <-- can be retrieved from printk_format
	field:u32 buf;	offset:32;	size:0;	signed:0;

print fmt: "given format", REC->ip, REC->fm

Does that look possible to you or have you run into troubles?

I don't know how much that's possible given the bprintk format. May be we should first try with the already formatted trace_printk version.

Quoted 11 lines

Every directory has an "enable" file. By echoing 0 or 1 into this file, it will enable all the trace_printk's below it.

This also uses jump_label() to keep overhead down when disabled.

Note, modules are not (yet) supported.

This is just the ground work. I'm busy doing other things so if you need more, feel free to take this and carry on.

-- Steve

Thanks a lot!

Quoted 21 lines

The following patches are in:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git

branch: rfc/event-printk

Steven Rostedt (2):
tracing: Rename trace_printk to ftrace_printk
tracing: Make event based trace_printk()

----

 include/asm-generic/vmlinux.lds.h |    7 +-
 include/linux/kernel.h            |   16 +-
 include/trace/event_printk.h      |   52 ++++++
 kernel/trace/Makefile             |    1 +
 kernel/trace/event_printk.c       |  365 +++++++++++++++++++++++++++++++++++++
 kernel/trace/trace.h              |    1 +
 kernel/trace/trace_events.c       |    2 +-
 7 files changed, 435 insertions(+), 9 deletions(-)
#13 lore raw
In reply to Peter Zijlstra
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Steven Rostedt

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

The format is the printk format:

Quoted 7 lines
Quoted 2 lines
# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d"

*groan*, so you're creating a tracepoint per instance?

That's going to be massive pain for perf.. I really don't see the point in splitting all that out.

Because it makes it very flexible, makes it easy to display the user where are his trace_printk() and which one he could interact with.

Why would it be a massive pain for perf? People are not going to play with thousands trace_printk() at once I guess.

Of course, a problem may arise if dynamic_printk() is handled into this scheme, because of the number of fds to handle. In this case probably this scheme should allow a group of trace_printk to be a (virtual) trace_event itself.

I mean if you have two trace_printk() in kernel/sched.c and some others in kernel/, you could either create once perf_event for every trace_printk() in kernel/ or one for every trace_prink() in kernel/sched.c, or one for kernel/sched.c:118

That's easy if you have an id file at each level.

An other solution, which have been talking with Thomas yesterday, would be to allow having a single fd for several perf_events at once. That would solve some problems when you have hundreds of events opened (think about wide tracing, or use of all individual syscalls).

#12 lore raw
In reply to Steven Rostedt
Quoted 38 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 37 lines · Mathieu Desnoyers
Quoted 18 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Steven Rostedt

For example, I added a trace_printk() in kernel/sched.c at line 2180 and it creates:

# ls /debug/tracing/events/printk/kernel/sched.c/2180/
enable  format

The format is the printk format:

Quoted 7 lines
Quoted 2 lines
# cat /debug/tracing/events/printk/kernel/sched.c/2180/format 
"migrate task %s:%d"

*groan*, so you're creating a tracepoint per instance?

That's going to be massive pain for perf.. I really don't see the point in splitting all that out.

Quoted 9 lines

a) The file directory was what was asked about in the referenced email.
b) This is just an example of a way to display it to the user, which seems to be very intuitive.
c) Perf can implement the details anyway it wants. It can make a single tracepoint callback and have the enabling of the points as a special filter.
d) This was just an RFC that Frederic asked if I would do. I thought it would be a fun challenge and did it. Let it bit rot in hell for all I care, I wasn't taking it any further anyway.

Hi Steven,

The LTTng tree still keeps the "trace_mark()" kernel markers, which are very very similar to ftrace_printk(). I'd be happy to combine the two eventually. This one file per line approach seems very good -- rather than explicitely naming each instance, as I did in trace_mark(), you end up automatically naming them by file/line number.

Quoted 6 lines
Quoted 2 lines

How does this behave from within static inlines called multiple times and unrolled loops ?

Actually you mean macros:

You'll get a 2181-1, 2181-2, 2181-3, etc.

This is neat, great ! :)

Quoted 2 lines

Although a trace_printk() in a header static inline may be interesting.

Yep, food for thoughts. ;-)

Thank you !

Mathieu

Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
author maintainer
#14 lore raw
In reply to Frederic Weisbecker
Quoted 4 lines · Frederic Weisbecker

n other solution, which have been talking with Thomas yesterday, would be to allow having a single fd for several perf_events at once. That would solve some problems when you have hundreds of events opened (think about wide tracing, or use of all individual syscalls).

I would highly recommend this. One type of tracing I do all the time is to enable all events (over 600). I don't think that is currently possible with trace and/or perf.

Ftrace has no issue simple because it just stores all the events into a single buffer (per cpu though). It does not care if there's one event or 1000.

You need to figure out how to do this as well with the function tracer. Of course you can just have it as a single event and filter as needed.

-- Steve

#16 lore raw
In reply to Frederic Weisbecker
Quoted 6 lines · Frederic Weisbecker

An other solution, which have been talking with Thomas yesterday, would be to allow having a single fd for several perf_events at once. That would solve some problems when you have hundreds of events opened (think about wide tracing, or use of all individual syscalls).

I'm working on that, but that doesn't make me like this crap any more.

#17 lore raw
Quoted 9 lines · Peter Zijlstra
Quoted 6 lines · Frederic Weisbecker

An other solution, which have been talking with Thomas yesterday, would be to allow having a single fd for several perf_events at once. That would solve some problems when you have hundreds of events opened (think about wide tracing, or use of all individual syscalls).

I'm working on that, but that doesn't make me like this crap any more.

No problem, we have more important things to focus on for now anyway. We may unearth it one day if the context makes it more useful.

Thanks.

#15 lore raw
In reply to Steven Rostedt
Quoted 9 lines · Steven Rostedt
Quoted 4 lines · Frederic Weisbecker

n other solution, which have been talking with Thomas yesterday, would be to allow having a single fd for several perf_events at once. That would solve some problems when you have hundreds of events opened (think about wide tracing, or use of all individual syscalls).

I would highly recommend this. One type of tracing I do all the time is to enable all events (over 600). I don't think that is currently possible with trace and/or perf.

Yep.

Quoted 5 lines

Ftrace has no issue simple because it just stores all the events into a single buffer (per cpu though). It does not care if there's one event or 1000.

You need to figure out how to do this as well with the function tracer.

Yeah for the function (and graph) tracers, I really plan to make it a single event. I mean one for the function tracer and one for the function graph. Although we may want two for the function graph tracer actually. One for entry, one for exit.