Skip to contentSkip to conversation

tracing update

GIT PULL Frederic Weisbecker 2 participants 6 messages Apr 19, 2010 – Apr 23, 2010

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

Ingo,

Please pull the tracing/core branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/core

Thanks,
Frederic
---

Frederic Weisbecker (1):
tracing: Dump either the oops's cpu source or all cpus buffers

 Documentation/kernel-parameters.txt |    6 +++-
 Documentation/trace/ftrace.txt      |    6 +++-
 drivers/char/sysrq.c                |    2 +-
 include/linux/ftrace.h              |    4 ++-
 include/linux/kernel.h              |   11 ++++++-
 kernel/trace/trace.c                |   51 ++++++++++++++++++++++++++--------
 6 files changed, 61 insertions(+), 19 deletions(-)

---
commit c7e6b2a29dd63d44c43c1dabe0a14ee4aef967cf

Author: Frederic Weisbecker <[email protected]>
Date:   Sun Apr 18 19:08:41 2010 +0200

tracing: Dump either the oops's cpu source or all cpus buffers

The ftrace_dump_on_oops kernel parameter, sysctl and sysrq let one dump every cpu buffers when an oops or panic happens.

It's nice when you have few cpus but it may take ages if have many, plus you miss the real origin of the problem in all the cpu traces.

Sometimes, all you need is to dump the cpu buffer that triggered the opps, most of the time it is our main interest.

This patch modifies ftrace_dump_on_oops to handle this choice.

The ftrace_dump_on_oops kernel parameter, when it comes alone, has the same behaviour than before. But ftrace_dump_on_oops=orig_cpu will only dump the buffer of the cpu that oops'ed.

Similarly, sysctl kernel.ftrace_dump_on_oops=1 and echo 1 > /proc/sys/kernel/ftrace_dump_on_oops keep their previous behaviour. But setting 2 jumps into cpu origin dump mode.

v2: Fix double setup
v3: Fix spelling issues reported by Randy Dunlap

Documentation/kernel-parameters.txt +5 -1
index e4cbca5..ab67b33 100644
@@ -789,8 +789,12 @@ and is between 256 and 4096 characters. It is defined in the file
 			as early as possible in order to facilitate early
 			boot debugging.
 
-	ftrace_dump_on_oops
+	ftrace_dump_on_oops[=orig_cpu]
 			[FTRACE] will dump the trace buffers on oops.
+			If no parameter is passed, ftrace will dump
+			buffers of all CPUs, but if you pass orig_cpu, it will
+			dump only the buffer of the CPU that triggered the
+			oops.
 
 	ftrace_filter=[function-list]
 			[FTRACE] Limit the functions traced by the function
Documentation/trace/ftrace.txt +4 -2
index 03485bf..5201181 100644
@@ -1337,12 +1337,14 @@ ftrace_dump_on_oops must be set. To set ftrace_dump_on_oops, one
 can either use the sysctl function or set it via the proc system
 interface.
 
-  sysctl kernel.ftrace_dump_on_oops=1
+  sysctl kernel.ftrace_dump_on_oops=n
 
 or
 
-  echo 1 > /proc/sys/kernel/ftrace_dump_on_oops
+  echo n > /proc/sys/kernel/ftrace_dump_on_oops
 
+If n = 1, ftrace will dump buffers of all CPUs, if n = 2 ftrace will
+only dump the buffer of the CPU that triggered the oops.
 
 Here's an example of such a dump after a null pointer
 dereference in a kernel module:
drivers/char/sysrq.c +1 -1
index 59de252..d4e8b21 100644
@@ -289,7 +289,7 @@ static struct sysrq_key_op sysrq_showstate_blocked_op = {
 
 static void sysrq_ftrace_dump(int key, struct tty_struct *tty)
 {
-	ftrace_dump();
+	ftrace_dump(DUMP_ALL);
 }
 static struct sysrq_key_op sysrq_ftrace_dump_op = {
 	.handler	= sysrq_ftrace_dump,
include/linux/ftrace.h +3 -1
index 01e6ade..ea5b1aa 100644
@@ -492,7 +492,9 @@ static inline int test_tsk_trace_graph(struct task_struct *tsk)
 	return tsk->trace & TSK_TRACE_FL_GRAPH;
 }
 
-extern int ftrace_dump_on_oops;
+enum ftrace_dump_mode;
+
+extern enum ftrace_dump_mode ftrace_dump_on_oops;
 
 #ifdef CONFIG_PREEMPT
 #define INIT_TRACE_RECURSION		.trace_recursion = 0,
include/linux/kernel.h +9 -2
index 9365227..9fb1c12 100644
@@ -490,6 +490,13 @@ static inline void tracing_off(void) { }
 static inline void tracing_off_permanent(void) { }
 static inline int tracing_is_on(void) { return 0; }
 #endif
+
+enum ftrace_dump_mode {
+	DUMP_NONE,
+	DUMP_ALL,
+	DUMP_ORIG,
+};
+
 #ifdef CONFIG_TRACING
 extern void tracing_start(void);
 extern void tracing_stop(void);
@@ -571,7 +578,7 @@ __ftrace_vbprintk(unsigned long ip, const char *fmt, va_list ap);
 extern int
 __ftrace_vprintk(unsigned long ip, const char *fmt, va_list ap);
 
-extern void ftrace_dump(void);
+extern void ftrace_dump(enum ftrace_dump_mode oops_dump_mode);
 #else
 static inline void
 ftrace_special(unsigned long arg1, unsigned long arg2, unsigned long arg3) { }
@@ -592,7 +599,7 @@ ftrace_vprintk(const char *fmt, va_list ap)
 {
 	return 0;
 }
-static inline void ftrace_dump(void) { }
+static inline void ftrace_dump(enum ftrace_dump_mode oops_dump_mode) { }
 #endif /* CONFIG_TRACING */
 
 /*
kernel/trace/trace.c +39 -12

104 lines not on this page · raw message

#1 lore raw
Quoted 15 lines · Frederic Weisbecker

Ingo,

Please pull the tracing/core branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/core

Thanks,
Frederic
---

Frederic Weisbecker (1):
tracing: Dump either the oops's cpu source or all cpus buffers

Quoted 7 lines
 Documentation/kernel-parameters.txt |    6 +++-
 Documentation/trace/ftrace.txt      |    6 +++-
 drivers/char/sysrq.c                |    2 +-
 include/linux/ftrace.h              |    4 ++-
 include/linux/kernel.h              |   11 ++++++-
 kernel/trace/trace.c                |   51 ++++++++++++++++++++++++++--------
 6 files changed, 61 insertions(+), 19 deletions(-)

Pulled, thanks Frederic!

Ingo

#2 lore raw
Quoted 3 lines
Quoted 3 lines

Frederic Weisbecker (1):
tracing: Dump either the oops's cpu source or all cpus buffers

FYI, it has build failures on x86:

kernel/trace/trace.c:4412: error: conflicting types for '__ftrace_dump' kernel/trace/trace_selftest.c:258: note: previous declaration of '__ftrace_dump' was here

Ingo

#3 lore raw
Quoted 8 lines · Ingo Molnar
Quoted 3 lines
Quoted 3 lines

Frederic Weisbecker (1):
tracing: Dump either the oops's cpu source or all cpus buffers

FYI, it has build failures on x86:

kernel/trace/trace.c:4412: error: conflicting types for '__ftrace_dump' kernel/trace/trace_selftest.c:258: note: previous declaration of '__ftrace_dump' was here

Oops, fix coming soon.

Thanks.

#4 lore raw

[GIT PULL] tracing update v2

In reply to Ingo Molnar

Ingo,

Please pull the tracing/core branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/core

Thanks,
Frederic
---

Frederic Weisbecker (1):
tracing: Dump either the oops's cpu source or all cpus buffers

 Documentation/kernel-parameters.txt |    6 +++-
 Documentation/trace/ftrace.txt      |    6 +++-
 drivers/char/sysrq.c                |    2 +-
 include/linux/ftrace.h              |    4 ++-
 include/linux/kernel.h              |   11 ++++++-
 kernel/trace/trace.c                |   51 ++++++++++++++++++++++++++--------
 kernel/trace/trace_selftest.c       |    5 ++-
 7 files changed, 64 insertions(+), 21 deletions(-)

---
commit cecbca96da387428e220e307a9c945e37e2f4d9e

Author: Frederic Weisbecker <[email protected]>
Date:   Sun Apr 18 19:08:41 2010 +0200

tracing: Dump either the oops's cpu source or all cpus buffers

The ftrace_dump_on_oops kernel parameter, sysctl and sysrq let one dump every cpu buffers when an oops or panic happens.

It's nice when you have few cpus but it may take ages if have many, plus you miss the real origin of the problem in all the cpu traces.

Sometimes, all you need is to dump the cpu buffer that triggered the opps, most of the time it is our main interest.

This patch modifies ftrace_dump_on_oops to handle this choice.

The ftrace_dump_on_oops kernel parameter, when it comes alone, has the same behaviour than before. But ftrace_dump_on_oops=orig_cpu will only dump the buffer of the cpu that oops'ed.

Similarly, sysctl kernel.ftrace_dump_on_oops=1 and echo 1 > /proc/sys/kernel/ftrace_dump_on_oops keep their previous behaviour. But setting 2 jumps into cpu origin dump mode.

v2: Fix double setup
v3: Fix spelling issues reported by Randy Dunlap
v4: Also update __ftrace_dump in the selftests

Documentation/kernel-parameters.txt +5 -1
index e4cbca5..ab67b33 100644
@@ -789,8 +789,12 @@ and is between 256 and 4096 characters. It is defined in the file
 			as early as possible in order to facilitate early
 			boot debugging.
 
-	ftrace_dump_on_oops
+	ftrace_dump_on_oops[=orig_cpu]
 			[FTRACE] will dump the trace buffers on oops.
+			If no parameter is passed, ftrace will dump
+			buffers of all CPUs, but if you pass orig_cpu, it will
+			dump only the buffer of the CPU that triggered the
+			oops.
 
 	ftrace_filter=[function-list]
 			[FTRACE] Limit the functions traced by the function
Documentation/trace/ftrace.txt +4 -2
index 03485bf..5201181 100644
@@ -1337,12 +1337,14 @@ ftrace_dump_on_oops must be set. To set ftrace_dump_on_oops, one
 can either use the sysctl function or set it via the proc system
 interface.
 
-  sysctl kernel.ftrace_dump_on_oops=1
+  sysctl kernel.ftrace_dump_on_oops=n
 
 or
 
-  echo 1 > /proc/sys/kernel/ftrace_dump_on_oops
+  echo n > /proc/sys/kernel/ftrace_dump_on_oops
 
+If n = 1, ftrace will dump buffers of all CPUs, if n = 2 ftrace will
+only dump the buffer of the CPU that triggered the oops.
 
 Here's an example of such a dump after a null pointer
 dereference in a kernel module:
drivers/char/sysrq.c +1 -1
index 59de252..d4e8b21 100644
@@ -289,7 +289,7 @@ static struct sysrq_key_op sysrq_showstate_blocked_op = {
 
 static void sysrq_ftrace_dump(int key, struct tty_struct *tty)
 {
-	ftrace_dump();
+	ftrace_dump(DUMP_ALL);
 }
 static struct sysrq_key_op sysrq_ftrace_dump_op = {
 	.handler	= sysrq_ftrace_dump,
include/linux/ftrace.h +3 -1
index 01e6ade..ea5b1aa 100644
@@ -492,7 +492,9 @@ static inline int test_tsk_trace_graph(struct task_struct *tsk)
 	return tsk->trace & TSK_TRACE_FL_GRAPH;
 }
 
-extern int ftrace_dump_on_oops;
+enum ftrace_dump_mode;
+
+extern enum ftrace_dump_mode ftrace_dump_on_oops;
 
 #ifdef CONFIG_PREEMPT
 #define INIT_TRACE_RECURSION		.trace_recursion = 0,
include/linux/kernel.h +9 -2
index 9365227..9fb1c12 100644
@@ -490,6 +490,13 @@ static inline void tracing_off(void) { }
 static inline void tracing_off_permanent(void) { }
 static inline int tracing_is_on(void) { return 0; }
 #endif
+
+enum ftrace_dump_mode {
+	DUMP_NONE,
+	DUMP_ALL,
+	DUMP_ORIG,
+};
+
 #ifdef CONFIG_TRACING
 extern void tracing_start(void);
 extern void tracing_stop(void);
@@ -571,7 +578,7 @@ __ftrace_vbprintk(unsigned long ip, const char *fmt, va_list ap);
 extern int
 __ftrace_vprintk(unsigned long ip, const char *fmt, va_list ap);
 
-extern void ftrace_dump(void);
+extern void ftrace_dump(enum ftrace_dump_mode oops_dump_mode);
 #else
 static inline void
 ftrace_special(unsigned long arg1, unsigned long arg2, unsigned long arg3) { }
@@ -592,7 +599,7 @@ ftrace_vprintk(const char *fmt, va_list ap)
 {
 	return 0;
 }
-static inline void ftrace_dump(void) { }
+static inline void ftrace_dump(enum ftrace_dump_mode oops_dump_mode) { }
 #endif /* CONFIG_TRACING */
 
 /*
kernel/trace/trace.c +39 -12

104 lines not on this page · raw message

kernel/trace/trace_selftest.c +3 -2
index 9398034..6a9d36d 100644
@@ -256,7 +256,8 @@ trace_selftest_startup_function(struct tracer *trace, struct trace_array *tr)
 /* Maximum number of functions to trace before diagnosing a hang */
 #define GRAPH_MAX_FUNC_TEST	100000000
 
-static void __ftrace_dump(bool disable_tracing);
+static void
+__ftrace_dump(bool disable_tracing, enum ftrace_dump_mode oops_dump_mode);
 static unsigned int graph_hang_thresh;
 
 /* Wrap the real function entry probe to avoid possible hanging */
@@ -267,7 +268,7 @@ static int trace_graph_entry_watchdog(struct ftrace_graph_ent *trace)
 		ftrace_graph_stop();
 		printk(KERN_WARNING "BUG: Function graph tracer hang!\n");
 		if (ftrace_dump_on_oops)
-			__ftrace_dump(false);
+			__ftrace_dump(false, DUMP_ALL);
 		return 0;
 	}
 
#5 lore raw

Re: [GIT PULL] tracing update v2

Quoted 15 lines · Frederic Weisbecker

Ingo,

Please pull the tracing/core branch that can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
tracing/core

Thanks,
Frederic
---

Frederic Weisbecker (1):
tracing: Dump either the oops's cpu source or all cpus buffers

Quoted 8 lines
 Documentation/kernel-parameters.txt |    6 +++-
 Documentation/trace/ftrace.txt      |    6 +++-
 drivers/char/sysrq.c                |    2 +-
 include/linux/ftrace.h              |    4 ++-
 include/linux/kernel.h              |   11 ++++++-
 kernel/trace/trace.c                |   51 ++++++++++++++++++++++++++--------
 kernel/trace/trace_selftest.c       |    5 ++-
 7 files changed, 64 insertions(+), 21 deletions(-)

Pulled, thanks Frederic!

Ingo