ftrace: add a tracepoint for __raise_softirq_irqoff()
PATCH v3 1/1 ftrace: add a tracepoint for __raise_softirq_irqoff()
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: Use TRACE_EVENT instead of DEFINE_TRACE
2: Move the tracepoint from raise_softirq_irqoff() to
__raise_softirq_irqoff()
Changelog for v2 -> v3:
Move the definition of __raise_softifq_irqoff() to .c file when CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
--- include/linux/interrupt.h | 6 ++++++ include/trace/events/irq.h | 18 ++++++++++++++++++ kernel/softirq.c | 8 ++++++++ 3 files changed, 32 insertions(+), 0 deletions(-)
include/linux/interrupt.h +6 -0
@@ -361,7 +361,13 @@ asmlinkage void do_softirq(void); asmlinkage void __do_softirq(void);extern void open_softirq (int nr,void (*action)(struct softirq_action *));extern void softirq_init (void ); + +#ifdef CONFIG_TRACEPOINTS +extern void __raise_softirq_irqoff (unsigned int nr); +#else #define __raise_softirq_irqoff (nr) do { or_softirq_pending(1UL << (nr)); } while (0) +#endif +extern void raise_softirq_irqoff (unsigned int nr);extern void raise_softirq (unsigned int nr);extern void wakeup_softirqd (void );
include/trace/events/irq.h +18 -0
@@ -128,6 +128,24 @@ TRACE_EVENT(softirq_exit,TP_printk ("softirq=%d action=%s" , __entry->vec,__get_str (name)) ); +TRACE_EVENT (irq_softirq_raise, + + TP_PROTO(unsigned int nr), + +TP_ARGS (nr), + +TP_STRUCT__entry ( + __field( unsigned int, nr ) + ), + +TP_fast_assign ( + __entry->nr = nr; + ), + +TP_printk ("softirq=%d action=%s is raised" , + __entry->nr, softirq_to_name[__entry->nr]) +); +#endif /* _TRACE_IRQ_H */ /* This part must be outside protection */
kernel/softirq.c +8 -0
@@ -310,6 +310,14 @@ void irq_exit(void)preempt_enable_no_resched (); } +#ifdef CONFIG_TRACEPOINTS +inline void __raise_softirq_irqoff (unsigned int nr) +{ +trace_irq_softirq_raise (nr); +or_softirq_pending (1UL << (nr)); +} +#endif +/* * This function must run with irqs disabled! */
Signature · 1 line
1.6.1.2
Quoted 1 line
+TRACE_EVENT(irq_softirq_raise,
I think 'softirq_raise' is better.
Quoted 5 lines
include/trace/events/irq.h +5
-0+ + TP_PROTO(unsigned int nr), + + TP_ARGS(nr), +
include/trace/events/irq.h TRACE_EVENT(softirq_exit, · line 146
Quoted 10 lines
include/trace/events/irq.h +10
-0+TP_STRUCT__entry ( +__field (unsigned int ,nr ) + ), + +TP_fast_assign ( + __entry->nr = nr; + ), + +TP_printk ("softirq=%d action=%s is raised" , + __entry->nr, softirq_to_name[__entry->nr])
"softirq=%d action=%s" is sufficient.
Please see TRACE_EVENT(softirq_entry) and TRACE_EVENT(softirq_exit).
Quoted 24 lines
include/trace/events/irq.h +2
-0+); +#endif /* _TRACE_IRQ_H */ /* This part must be outside protection */ kernel/softirq.c +8
-0@@ -310,6 +310,14 @@ void irq_exit(void)preempt_enable_no_resched (); } +#ifdef CONFIG_TRACEPOINTS +inline void __raise_softirq_irqoff (unsigned int nr) +{ +trace_irq_softirq_raise (nr); +or_softirq_pending (1UL << (nr)); +} +#endif +/* * This function must run with irqs disabled! */
Quoted 22 lines · Li Zefan
Quoted 1 line
+TRACE_EVENT(irq_softirq_raise,
I think '
softirq_raise' is better.Quoted 5 lines
include/trace/events/irq.h +5
-0+ + TP_PROTO(unsigned int nr), + + TP_ARGS(nr), +include/trace/events/irq.h TRACE_EVENT(softirq_exit, · line 146
Quoted 10 lines
include/trace/events/irq.h +10
-0+TP_STRUCT__entry ( +__field (unsigned int ,nr ) + ), + +TP_fast_assign ( + __entry->nr = nr; + ), + +TP_printk ("softirq=%d action=%s is raised" , + __entry->nr, softirq_to_name[__entry->nr])"softirq=%d action=%s" is sufficient.
Quoted 2 lines
Please see
TRACE_EVENT(softirq_entry)andTRACE_EVENT(softirq_exit).
Thanks, I will modify it as your suggestions. ;-)
Quoted 3 lines
Quoted 3 lines
include/trace/events/irq.h +2
-0+); +
Quoted 30 lines · * Xiao Guangrong ([email protected]) wrote:
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesSigned-off-by Mathieu Desnoyers <[email protected]>Signed-off-by Xiao Guangrong <[email protected]>--- include/linux/interrupt.h | 6 ++++++ include/trace/events/irq.h | 18 ++++++++++++++++++ kernel/softirq.c | 8 ++++++++ 3 files changed, 32 insertions(+), 0 deletions(-)include/linux/interrupt.h +0
-0
include/linux/interrupt.h asmlinkage void do_softirq(void); · line 368
Quoted 8 lines
include/linux/interrupt.h +4
-0@@ -361,7 +361,13 @@ asmlinkage void do_softirq(void); asmlinkage void __do_softirq(void);extern void open_softirq (int nr,void (*action)(struct softirq_action *));extern void softirq_init (void ); + +#ifdef CONFIG_TRACEPOINTS +extern void __raise_softirq_irqoff (unsigned int nr); +#else #define __raise_softirq_irqoff (nr) do { or_softirq_pending(1UL << (nr)); } while (0)
Can you put the
trace_irq_softirq_raise(nr);
directly in the define rather than adding this weird CONFIG_TRACEPOINTS? (and change the define for a static inline), something like :
static inline void __raise_softirq_irqoff(unsigned int nr)
{
trace_irq_softirq_raise(nr);
or_softirq_pending(1UL << (nr);
}This would ensure we don't add a function call on the __raise_softirq_irqoff() fast-path.
Beware of circular include dependencies though. The tracepoints are meant not to have this kind of problems (I try to keep the dependencies very minimalistic), but I wonder if Steven's TRACE_EVENT is now ok on this aspect.
If TRACE_EVENT happens to pose problems with circular header dependencies, then try moving to the DECLARE_TRACE/DEFINE_TRACE scheme which has been more thoroughly tested as a first step.
Mathieu
Quoted 56 lines
include/linux/interrupt.h +2
-0+#endif +extern void raise_softirq_irqoff (unsigned int nr);extern void raise_softirq (unsigned int nr);extern void wakeup_softirqd (void );include/trace/events/irq.h +18
-0@@ -128,6 +128,24 @@ TRACE_EVENT(softirq_exit,TP_printk ("softirq=%d action=%s" , __entry->vec,__get_str (name)) ); +TRACE_EVENT (irq_softirq_raise, + + TP_PROTO(unsigned int nr), + +TP_ARGS (nr), + +TP_STRUCT__entry ( + __field( unsigned int, nr ) + ), + +TP_fast_assign ( + __entry->nr = nr; + ), + +TP_printk ("softirq=%d action=%s is raised" , + __entry->nr, softirq_to_name[__entry->nr]) +); +#endif /* _TRACE_IRQ_H */ /* This part must be outside protection */ kernel/softirq.c +8
-0@@ -310,6 +310,14 @@ void irq_exit(void)preempt_enable_no_resched (); } +#ifdef CONFIG_TRACEPOINTS +inline void __raise_softirq_irqoff (unsigned int nr) +{ +trace_irq_softirq_raise (nr); +or_softirq_pending (1UL << (nr)); +} +#endif +/* * This function must run with irqs disabled! */ Signature · 1 line
1.6.1.2
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Quoted 15 lines · Mathieu Desnoyers
include/linux/interrupt.h asmlinkage void do_softirq(void); · line 368
Quoted 7 lines · * Xiao Guangrong ([email protected]) wrote:
From: Mathieu Desnoyers <[email protected]>
include/linux/interrupt.h +3
-0+#ifdef CONFIG_TRACEPOINTS +extern void __raise_softirq_irqoff (unsigned int nr); +#else #define __raise_softirq_irqoff (nr) do { or_softirq_pending(1UL << (nr)); } while (0)Can you put the
trace_irq_softirq_raise(nr);directly in the define rather than adding this weird
CONFIG_TRACEPOINTS? (and change the define for a static inline), something like :
Quoted 9 lines
static inline void
__raise_softirq_irqoff(unsigned int nr){ trace_irq_softirq_raise(nr); or_softirq_pending(1UL << (nr); }This would ensure we don't add a function call on the
__raise_softirq_irqoff()fast-path.
We did this in v2, and we think it is better for same reason. But ...
Quoted 5 lines
Beware of circular include dependencies though. The tracepoints are meant not to have this kind of problems (I try to keep the dependencies very minimalistic), but I wonder if Steven's
TRACE_EVENTis now ok on this aspect.
We encount this type of problem in v2.
So we move to this version(v3).
Quoted 4 lines
If
TRACE_EVENThappens to pose problems with circular header dependencies, then try moving to the DECLARE_TRACE/DEFINE_TRACE scheme which has been more thoroughly tested as a first step.
IMHO, TRACE_EVENT framework is better for its more generic as ingo said, and it also provide ftrace support which means user can view tracepoint information from /debug/tracing/events.
Although this TRACE_EVENT happens to expose problems with circular header dependencies, we should not refuse using TRACE_EVENT, instead we should try to fix it for the whole TRACE_EVENT facility later.
Thanks
Quoted 2 lines
Mathieu
Quoted 45 lines · * Xiao Guangrong ([email protected]) wrote:
Quoted 15 lines · Mathieu Desnoyers
include/linux/interrupt.h asmlinkage void do_softirq(void); · line 368
Quoted 7 lines · * Xiao Guangrong ([email protected]) wrote:
From: Mathieu Desnoyers <[email protected]>
include/linux/interrupt.h +3
-0+#ifdef CONFIG_TRACEPOINTS +extern void __raise_softirq_irqoff (unsigned int nr); +#else #define __raise_softirq_irqoff (nr) do { or_softirq_pending(1UL << (nr)); } while (0)Can you put the
trace_irq_softirq_raise(nr);directly in the define rather than adding this weird
CONFIG_TRACEPOINTS? (and change the define for a static inline), something like :Quoted 9 lines
static inline void
__raise_softirq_irqoff(unsigned int nr){ trace_irq_softirq_raise(nr); or_softirq_pending(1UL << (nr); }This would ensure we don't add a function call on the
__raise_softirq_irqoff()fast-path.We did this in v2, and we think it is better for same reason. But ...
Quoted 5 lines
Beware of circular include dependencies though. The tracepoints are meant not to have this kind of problems (I try to keep the dependencies very minimalistic), but I wonder if Steven's
TRACE_EVENTis now ok on this aspect.We encount this type of problem in v2.
So we move to this version(v3).Quoted 4 lines
If
TRACE_EVENThappens to pose problems with circular header dependencies, then try moving to the DECLARE_TRACE/DEFINE_TRACE scheme which has been more thoroughly tested as a first step.
Quoted 8 lines
IMHO,
TRACE_EVENTframework is better for its more generic as ingo said, and it also provide ftrace support which means user can view tracepoint information from /debug/tracing/events.Although this
TRACE_EVENThappens to expose problems with circular header dependencies, we should not refuse usingTRACE_EVENT, instead we should try to fix it for the wholeTRACE_EVENTfacility later.
I partially agree with you :
Yes, we should try to fix TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem with TRACE_EVENT right now.
Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
Mathieu
Quoted 4 lines
Thanks
Quoted 2 lines
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Steven Rostedt
maintainerQuoted 7 lines · Mathieu Desnoyers
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
The majority of tracepoints happen is C files. Those few cases where they are used in headers is where the issues arise.
But...
I did not want to uglify all trace event headers with:
#ifdef CREATE_FOO_TRACE_POINTS
#undef CREATE_FOO_TRACE_POINTS
#include <trace/define_trace.h>
#endif
We would only need to do that for those trace points that need to be included in header files. Then the declaration C file would need to define both CREATE_FOO_TRACE_POINTS and CREATE_TRACE_POINTS
#define CREATE_FOO_TRACE_POINTS
#define CRATE_TRACE_POINTS
#include <trace/events/foo.h>
But this is pretty trivial to solve, and I do not consider it a show stopper or a major header dependency problem.
-- Steve
Quoted 31 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 7 lines · Mathieu Desnoyers
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
The majority of tracepoints happen is C files. Those few cases where they are used in headers is where the issues arise.
But...
I did not want to uglify all trace event headers with:
#ifdef
CREATE_FOO_TRACE_POINTS
#undefCREATE_FOO_TRACE_POINTS
#include <trace/define_trace.h>
#endifWe would only need to do that for those trace points that need to be included in header files. Then the declaration C file would need to define both
CREATE_FOO_TRACE_POINTSandCREATE_TRACE_POINTS#define
CREATE_FOO_TRACE_POINTS
#defineCRATE_TRACE_POINTS
#include <trace/events/foo.h>
Quoted 5 lines
But this is pretty trivial to solve, and I do not consider it a show stopper or a major header dependency problem.
-- Steve
Hrm, is there any way to solve it elegantly ?
What we really need is to see the cases where TRACE_EVENT() is used as a declaration vs the case where it expands TP_STRUCT__entry/TP_fast_assign/TP_printk as having different dependencies. The problem comes when we bring the include dependencies of the TP_fast_assign part into the tracepoint header and it becomes a dependency of the TRACE_EVENT() declaration-only part.
Can we do the following ?
All tracepoint headers could surround the include dependencies by :
#ifdef BUILD_EVENTS
#include <veryannoyingheaderdependency.h>
#endif
And then we follow this by the TRACE_EVENT() declarations.
BUILD_EVENTS would only be defined in kernel/trace/events.c.
I think it should work, but it looks a bit too simple, so I may have missed something... ?
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Steven Rostedt
maintainerQuoted 51 lines · Mathieu Desnoyers
Quoted 31 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 7 lines · Mathieu Desnoyers
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
The majority of tracepoints happen is C files. Those few cases where they are used in headers is where the issues arise.
But...
I did not want to uglify all trace event headers with:
#ifdef
CREATE_FOO_TRACE_POINTS
#undefCREATE_FOO_TRACE_POINTS
#include <trace/define_trace.h>
#endifWe would only need to do that for those trace points that need to be included in header files. Then the declaration C file would need to define both
CREATE_FOO_TRACE_POINTSandCREATE_TRACE_POINTS#define
CREATE_FOO_TRACE_POINTS
#defineCRATE_TRACE_POINTS
#include <trace/events/foo.h>Quoted 5 lines
But this is pretty trivial to solve, and I do not consider it a show stopper or a major header dependency problem.
-- Steve
Hrm, is there any way to solve it elegantly ?
What we really need is to see the cases where
TRACE_EVENT()is used as a declaration vs the case where it expands TP_STRUCT__entry/TP_fast_assign/TP_printk as having different dependencies. The problem comes when we bring the include dependencies of theTP_fast_assignpart into the tracepoint header and it becomes a dependency of theTRACE_EVENT()declaration-only part.Can we do the following ?
All tracepoint headers could surround the include dependencies by :
Quoted 7 lines
#ifdef
BUILD_EVENTS
#include <veryannoyingheaderdependency.h>
#endifAnd then we follow this by the
TRACE_EVENT()declarations.
BUILD_EVENTSwould only be defined inkernel/trace/events.c.
That file no longer exists. All the events are created with the define_trace.h or ftrace.h file.
We could do something like your
#ifdef BUILD_EVENTS /* or better name BUILDING_EVENTS ?? */ #include <allmyannoyingheaders.h>
[...]
#endif
And then have the define_trace.h file add define it. But this again adds more annoying CPP commands to all trace_event headers.
-- Steve
Quoted 11 lines
I think it should work, but it looks a bit too simple, so I may have missed something... ?
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Quoted 69 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 51 lines · Mathieu Desnoyers
Quoted 31 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 7 lines · Mathieu Desnoyers
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
The majority of tracepoints happen is C files. Those few cases where they are used in headers is where the issues arise.
But...
I did not want to uglify all trace event headers with:
#ifdef
CREATE_FOO_TRACE_POINTS
#undefCREATE_FOO_TRACE_POINTS
#include <trace/define_trace.h>
#endifWe would only need to do that for those trace points that need to be included in header files. Then the declaration C file would need to define both
CREATE_FOO_TRACE_POINTSandCREATE_TRACE_POINTS#define
CREATE_FOO_TRACE_POINTS
#defineCRATE_TRACE_POINTS
#include <trace/events/foo.h>Quoted 5 lines
But this is pretty trivial to solve, and I do not consider it a show stopper or a major header dependency problem.
-- Steve
Hrm, is there any way to solve it elegantly ?
What we really need is to see the cases where
TRACE_EVENT()is used as a declaration vs the case where it expands TP_STRUCT__entry/TP_fast_assign/TP_printk as having different dependencies. The problem comes when we bring the include dependencies of theTP_fast_assignpart into the tracepoint header and it becomes a dependency of theTRACE_EVENT()declaration-only part.Can we do the following ?
All tracepoint headers could surround the include dependencies by :
Quoted 7 lines
#ifdef
BUILD_EVENTS
#include <veryannoyingheaderdependency.h>
#endifAnd then we follow this by the
TRACE_EVENT()declarations.
BUILD_EVENTSwould only be defined inkernel/trace/events.c.That file no longer exists. All the events are created with the
define_trace.horftrace.hfile.We could do something like your
Quoted 8 lines
#ifdef
BUILD_EVENTS/* or better nameBUILDING_EVENTS?? */ #include <allmyannoyingheaders.h>
[...]
#endifAnd then have the
define_trace.hfile add define it. But this again adds more annoying CPP commands to alltrace_eventheaders.
Yes, but this would solve most of the include dependency problems at once. We would only have dependency on preempt.h left, which alone is easier to deal with. And if headers don't need to include any annoying headers, they don't need to use the ifdef BUILDING_EVENTS. But my point is that when it's needed (and we already see two cases where it's needed, with pvops instrumentation and softirq instrumentation, and I guess we'll see much more), it's good to have this infrastructure in place, so people don't end up trying to do hacks to the kernel code changing inlines for function calls to try to deal with the circular include issue.
I think it's
a) needed
b) it does not hurt anyone who does not need it.
So I would definitely recommend adding such define.
Mathieu
Quoted 13 lines
-- Steve
Quoted 11 lines
I think it should work, but it looks a bit too simple, so I may have missed something... ?
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Quoted 22 lines · Mathieu Desnoyers
Quoted 9 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 5 lines · Mathieu Desnoyers
Quoted 4 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 3 lines · Mathieu Desnoyers
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Yes, but this would solve most of the include dependency problems at once. We would only have dependency on
preempt.hleft, which alone is easier to deal with. And if headers don't need to include any annoying headers, they don't need to use the ifdefBUILDING_EVENTS. But my point is that when it's needed (and we already see two cases where it's needed, with pvops instrumentation and softirq instrumentation, and I guess we'll see much more), it's good to have this infrastructure in place, so people don't end up trying to do hacks to the kernel code changing inlines for function calls to try to deal with the circular include issue.
Quoted 7 lines
I think it's
a) needed
b) it does not hurt anyone who does not need it.So I would definitely recommend adding such define.
Sorry for my poor English.
Problem is not only in TP_printk, but also in TP_PROTO if we add a tracer in header file.
So, it can't be solved only by "get rid of including ftrace parts"
Take v2 patch for example:
(it is at: http://marc.info/?l=linux-kernel&m=124081169727739&w=2)
In order to trace __raise_softirq_irqoff(), we should add a trace function in __raise_softirq_irqoff() and include <include/trace/irq.h> in interrupte.h.
If we put <include/trace/irq.h> on top of linux/irq.h, we will see a warning of "struct softirq_action declared inside parameter list" in compile. It is because struct irqaction's definition is bypasswd before TP_PROTO.
To say it simple:sched.c:
include interrupte.h
| |-->include irq.h | |-->include interrupte.h (bypassed) | |-->TP_PROTO(int irq, struct irqaction *action), | (but struct softirq_action is not declared before, | it raise a compile warning:struct softirq_action declared | inside parameter list. |-> ......
I can't see the solvent for this in your discuss or I understand wrong?
Xiao
Quoted 15 lines
Mathieu
Quoted 12 lines
-- Steve
Quoted 10 lines
I think it should work, but it looks a bit too simple, so I may have missed something... ?
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Quoted 41 lines · * Xiao Guangrong ([email protected]) wrote:
Quoted 22 lines · Mathieu Desnoyers
Quoted 9 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 5 lines · Mathieu Desnoyers
Quoted 4 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 3 lines · Mathieu Desnoyers
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Yes, but this would solve most of the include dependency problems at once. We would only have dependency on
preempt.hleft, which alone is easier to deal with. And if headers don't need to include any annoying headers, they don't need to use the ifdefBUILDING_EVENTS. But my point is that when it's needed (and we already see two cases where it's needed, with pvops instrumentation and softirq instrumentation, and I guess we'll see much more), it's good to have this infrastructure in place, so people don't end up trying to do hacks to the kernel code changing inlines for function calls to try to deal with the circular include issue.Quoted 7 lines
I think it's
a) needed
b) it does not hurt anyone who does not need it.So I would definitely recommend adding such define.
Sorry for my poor English.
Problem is not only inTP_printk, but also inTP_PROTOif we add a tracer in header file.
So, it can't be solved only by "get rid of including ftrace parts"Take v2 patch for example:
(it is at: http://marc.info/?l=linux-kernel&m=124081169727739&w=2)
Quoted 8 lines
In order to trace
__raise_softirq_irqoff(), we should add a trace function in__raise_softirq_irqoff()and include <include/trace/irq.h> ininterrupte.h.
If we put <include/trace/irq.h> on top oflinux/irq.h, we will see a warning of "structsoftirq_actiondeclared inside parameter list" in compile. It is because struct irqaction's definition is bypasswd beforeTP_PROTO.
Then add a forward declaration of
struct softirqaction;
At the top of trace/irq.h. I did it in quite a few places in the LTTng tree. TP_PROTO just needs a forward declaration, not the full structure declaration.
Only the ftrace-specific parts will need the #include <linux/irq.h>, which we should put in a #ifdef :
sched.c:
#include linux/interrupt.h
|
|-->include <trace/irq.h>
#ifdef BUILDING_EVENTS
/*
* Includes needed by TP_fast_assign, TP_printk, and
* TP_STRUCT__entry :
*/
# include <linux/interrupt.h>
#else
/* Forward declarations for TP_PROTO */
struct softirq_action;
#endif
TRACE_EVENT(......);
|-->TP_PROTO(int irq, struct irqaction *action),
(but struct softirq_action is not declared before,
it raise a compile warning:struct softirq_action declared
inside parameter list.
|-> ......Mathieu
Quoted 20 lines
I can't see the solvent for this in your discuss or I understand wrong?
Xiao
Quoted 15 lines
Mathieu
Quoted 12 lines
-- Steve
Quoted 10 lines
I think it should work, but it looks a bit too simple, so I may have missed something... ?
Mathieu
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
PATCH v4 1/1 ftrace: add a tracepoint for __raise_softirq_irqoff()
In reply to Xiao GuangrongFrom: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: Use TRACE_EVENT instead of DEFINE_TRACE
2: Move the tracepoint from raise_softirq_irqoff() to
__raise_softirq_irqoff()
Changelog for v2 -> v3:
Move the definition of __raise_softifq_irqoff() to .c file when CONFIG_TRACEPOINTS is enabled, to avoid recursive includes
Changelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepoint
--- include/linux/interrupt.h | 9 ++++++++- include/trace/events/irq.h | 27 ++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 2 deletions(-)
include/linux/interrupt.h +8 -1
@@ -18,6 +18,7 @@#include <asm/atomic.h> #include <asm/ptrace.h> #include <asm/system.h> +#include <trace/events/irq.h> /* * These correspond to the IORESOURCE_IRQ_* defines in
@@ -361,7 +362,13 @@ asmlinkage void do_softirq(void); asmlinkage void __do_softirq(void);extern void open_softirq (int nr,void (*action)(struct softirq_action *));extern void softirq_init (void ); -#define __raise_softirq_irqoff (nr) do { or_softirq_pending(1UL << (nr)); } while (0) + +static inline void __raise_softirq_irqoff (unsigned int nr) +{ +trace_softirq_raise (nr); +or_softirq_pending (1UL << nr); +} +extern void raise_softirq_irqoff (unsigned int nr);extern void raise_softirq (unsigned int nr);extern void wakeup_softirqd (void );
include/trace/events/irq.h +26 -1
@@ -2,11 +2,13 @@#define _TRACE_IRQ_H#include <linux/tracepoint.h> -#include <linux/interrupt.h> #undef TRACE_SYSTEM#define TRACE_SYSTEM irq +struct irqaction ; +struct softirq_action ; +/** * irq_handler_entry - called immediately before the irq action handler * @irq: irq number
@@ -128,6 +130,29 @@ TRACE_EVENT(softirq_exit,TP_printk ("softirq=%d action=%s" , __entry->vec,__get_str (name)) ); +/** +* softirq_raise - called immediately when a softirq is raised +* @nr: softirq vector number +*/ +TRACE_EVENT (softirq_raise, + + TP_PROTO(unsigned int nr), + +TP_ARGS (nr), + +TP_STRUCT__entry ( + __field( unsigned int, nr ) +__string ( name, softirq_to_name[nr] ) + ), + +TP_fast_assign ( + __entry->nr = nr; + __assign_str(name, softirq_to_name[nr]); + ), + +TP_printk ("softirq=%u action=%s" , __entry->nr,__get_str (name)) +); +#endif /* _TRACE_IRQ_H */ /* This part must be outside protection */
Signature · 1 line
1.6.1.2
Quoted 16 lines · * Xiao Guangrong ([email protected]) wrote:
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includes
Quoted 6 lines
Changelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepoint
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.
Including interrupt.h under a ifdef would allow keeping track of TRACE_EVENT specific build dependencies neatly on a per header basis.
Mathieu
Quoted 88 lines
Signed-off-by Mathieu Desnoyers <[email protected]>Signed-off-by Xiao Guangrong <[email protected]>Review-by Lai Jiangshan <[email protected]>--- include/linux/interrupt.h | 9 ++++++++- include/trace/events/irq.h | 27 ++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 2 deletions(-)include/linux/interrupt.h +8
-1@@ -18,6 +18,7 @@#include <asm/atomic.h> #include <asm/ptrace.h> #include <asm/system.h> +#include <trace/events/irq.h> /* * These correspond to the IORESOURCE_IRQ_* defines in @@ -361,7 +362,13 @@ asmlinkage void do_softirq(void); asmlinkage void __do_softirq(void);extern void open_softirq (int nr,void (*action)(struct softirq_action *));extern void softirq_init (void ); -#define __raise_softirq_irqoff (nr) do { or_softirq_pending(1UL << (nr)); } while (0) + +static inline void __raise_softirq_irqoff (unsigned int nr) +{ +trace_softirq_raise (nr); +or_softirq_pending (1UL << nr); +} +extern void raise_softirq_irqoff (unsigned int nr);extern void raise_softirq (unsigned int nr);extern void wakeup_softirqd (void );include/trace/events/irq.h +26
-1@@ -2,11 +2,13 @@#define _TRACE_IRQ_H#include <linux/tracepoint.h> -#include <linux/interrupt.h> #undef TRACE_SYSTEM#define TRACE_SYSTEM irq +struct irqaction ; +struct softirq_action ; +/** * irq_handler_entry - called immediately before the irq action handler * @irq: irq number @@ -128,6 +130,29 @@ TRACE_EVENT(softirq_exit,TP_printk ("softirq=%d action=%s" , __entry->vec,__get_str (name)) ); +/** +* softirq_raise - called immediately when a softirq is raised +* @nr: softirq vector number +*/ +TRACE_EVENT (softirq_raise, + + TP_PROTO(unsigned int nr), + +TP_ARGS (nr), + +TP_STRUCT__entry ( + __field( unsigned int, nr ) +__string ( name, softirq_to_name[nr] ) + ), + +TP_fast_assign ( + __entry->nr = nr; + __assign_str(name, softirq_to_name[nr]); + ), + +TP_printk ("softirq=%u action=%s" , __entry->nr,__get_str (name)) +); +#endif /* _TRACE_IRQ_H */ /* This part must be outside protection */ Signature · 1 line
1.6.1.2
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Steven Rostedt
maintainerQuoted 23 lines · Mathieu Desnoyers
Quoted 16 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesQuoted 6 lines
Changelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepoint
Quoted 7 lines
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.
This is all moot, the events.c file no longer exists and as not an issue.
-- Steve
Quoted 30 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 28 lines · Mathieu Desnoyers
Quoted 16 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesQuoted 6 lines
Changelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointThis is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.
Quoted 5 lines
Quoted 2 lines
Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.
OK, thanks for the update :)
Mathieu
Quoted 2 lines
-- Steve
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Quoted 28 lines · Steven Rostedt
Quoted 27 lines · Mathieu Desnoyers
Quoted 16 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesQuoted 6 lines
Changelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointThis is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.
Quoted 5 lines
Quoted 2 lines
Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.
As Steve's says, use ftrace in ftrace.h not in events.c now. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.
Thanks for your review, is great help to me. ;-)
Quoted 4 lines
-- Steve
* From: "Xiao Guangrong" <[email protected]>
Quoted 32 lines
Quoted 31 lines · Steven Rostedt
Quoted 29 lines · Mathieu Desnoyers
Quoted 16 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesQuoted 6 lines
Changelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointThis is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.
Quoted 8 lines
Quoted 2 lines
This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
Thanks
Zhaolei
Quoted 6 lines
Quoted 5 lines
-- Steve
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 40 lines · Zhaolei
* From: "Xiao Guangrong" <[email protected]>
Quoted 39 lines
Quoted 28 lines · Steven Rostedt
Quoted 27 lines · Mathieu Desnoyers
Quoted 16 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesQuoted 6 lines
Changelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointThis is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Quoted 5 lines
Quoted 2 lines
Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.
Quoted 6 lines
Quoted 1 line
Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
There's a complication: this area of the softirq code needs fixes (unrelated to tracing).
This API:
inline void raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);
/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}
is broken with RT tasks (as recently reported to lkml), as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
The proper solution would be to have a new API:
raise_softirq_check()
and to remove the wakeup_softirqd() hack from raise_softirq_irqoff()
- and put raise_softirq_check() to all places that use raise_softirq*() from process context.
raise_softirq_check() would execute softirq handlers from process context, if there's any pending ones. It has to be called outside of bh critical sections - i.e. often a bit after the raise_softirq() has been done.
__raise_softirq_irqoff() would be made private to kernel/softirq.c, and we'd only have two public APIs to trigger softirqs:raise_softirq() and raise_softirq_irqoff(). Both just set the pending flag and dont do any wakeup.
As a side-effect of these fixes, the tracepoints will be sorted out as well - there wont be any need to hack into __raise_softirq_irqoff().
Ingo
Quoted 40 lines · Ingo Molnar
Quoted 38 lines · Zhaolei
* From: "Xiao Guangrong" <[email protected]>
Quoted 37 lines
Quoted 23 lines · Steven Rostedt
Quoted 22 lines · Mathieu Desnoyers
Quoted 22 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesChangelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointQuoted 9 lines
Quoted 7 lines
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
There's a complication: this area of the softirq code needs fixes (unrelated to tracing).
Hello, Ingo
Thanks for your reply.
Since it is unrelated to tracing, why not add this tracepoint first and fix the softirq code later? The tracepoint won't add any burden to the fix work.
Thanks
Zhaolei
Quoted 48 lines
This API:
inline void
raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}is broken with RT tasks (as recently reported to lkml), as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
The proper solution would be to have a new API:
raise_softirq_check()and to remove the
wakeup_softirqd()hack fromraise_softirq_irqoff()
- and putraise_softirq_check()to all places that use raise_softirq*() from process context.
raise_softirq_check()would execute softirq handlers from process context, if there's any pending ones. It has to be called outside of bh critical sections - i.e. often a bit after theraise_softirq()has been done.
__raise_softirq_irqoff()would be made private tokernel/softirq.c, and we'd only have two public APIs to trigger softirqs:raise_softirq()andraise_softirq_irqoff(). Both just set the pending flag and dont do any wakeup.As a side-effect of these fixes, the tracepoints will be sorted out as well - there wont be any need to hack into
__raise_softirq_irqoff().Ingo
Quoted 72 lines · Ingo Molnar
Quoted 38 lines · Zhaolei
* From: "Xiao Guangrong" <[email protected]>
Quoted 37 lines
Quoted 23 lines · Steven Rostedt
Quoted 22 lines · Mathieu Desnoyers
Quoted 22 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesChangelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointQuoted 9 lines
Quoted 7 lines
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.Quoted 5 lines
Quoted 1 line
Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
There's a complication: this area of the softirq code needs fixes (unrelated to tracing).
This API:
inline void
raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}is broken with RT tasks (as recently reported to lkml), as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
Quoted 7 lines
The proper solution would be to have a new API:
raise_softirq_check()and to remove the
wakeup_softirqd()hack fromraise_softirq_irqoff()
- and putraise_softirq_check()to all places that use raise_softirq*() from process context.
It's a nice solution. But I think it would be nicer when it is changed a little.
The new API raise_softirq_check() will become a very hard _burden_ to the users of raise_softirq*(). They have to find out a proper place to place the "raise_softirq_check();". It's not an easy things, functions are complicated and hard to be determined WHERE is the process context and WHEN(a function may be called from multi kinds of context).
Instead, I prefer that raise_softirq_check() is hidden from users. We call raise_softirq_check() from schedule(), it will handle the un-handle softirqs in time(if un-handle softirqs are too much, it'll wakeup the ksoftirqd).
Lai
Quoted 19 lines
raise_softirq_check()would execute softirq handlers from process context, if there's any pending ones. It has to be called outside of bh critical sections - i.e. often a bit after theraise_softirq()has been done.
__raise_softirq_irqoff()would be made private tokernel/softirq.c, and we'd only have two public APIs to trigger softirqs:raise_softirq()andraise_softirq_irqoff(). Both just set the pending flag and dont do any wakeup.As a side-effect of these fixes, the tracepoints will be sorted out as well - there wont be any need to hack into
__raise_softirq_irqoff().Ingo
RFC PATCH 1/1 softirq: fix ksoftirq starved
Quoted 88 lines · Lai Jiangshan
Quoted 71 lines · Ingo Molnar
Quoted 38 lines · Zhaolei
* From: "Xiao Guangrong" <[email protected]>
Quoted 37 lines
Quoted 23 lines · Steven Rostedt
Quoted 22 lines · Mathieu Desnoyers
Quoted 22 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesChangelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointQuoted 9 lines
Quoted 7 lines
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.Quoted 5 lines
Quoted 1 line
Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
There's a complication: this area of the softirq code needs fixes (unrelated to tracing).
This API:
inline void
raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}is broken with RT tasks (as recently reported to lkml), as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
Quoted 7 lines
The proper solution would be to have a new API:
raise_softirq_check()and to remove the
wakeup_softirqd()hack fromraise_softirq_irqoff()
- and putraise_softirq_check()to all places that use raise_softirq*() from process context.It's a nice solution. But I think it would be nicer when it is changed a little.
The new API
raise_softirq_check()will become a very hard _burden_ to the users of raise_softirq*(). They have to find out a proper place to place the "raise_softirq_check();". It's not an easy things, functions are complicated and hard to be determined WHERE is the process context and WHEN(a function may be called from multi kinds of context).
Quoted 28 lines
Instead, I prefer that
raise_softirq_check()is hidden from users. We callraise_softirq_check()fromschedule(), it will handle the un-handle softirqs in time (if un-handle softirqs are too much, it'll wakeup the ksoftirqd).Lai
Quoted 19 lines
raise_softirq_check()would execute softirq handlers from process context, if there's any pending ones. It has to be called outside of bh critical sections - i.e. often a bit after theraise_softirq()has been done.
__raise_softirq_irqoff()would be made private tokernel/softirq.c, and we'd only have two public APIs to trigger softirqs:raise_softirq()andraise_softirq_irqoff(). Both just set the pending flag and dont do any wakeup.As a side-effect of these fixes, the tracepoints will be sorted out as well - there wont be any need to hack into
__raise_softirq_irqoff().Ingo
Subject: [PATCH RFC] softirq: fix ksoftirq starved
The API:
void raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);
/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}is broken with RT tasks, as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
To make pending softirq is called in time, we check and run the pending softirq in schedule().
---
include/linux/interrupt.h +1 -0
@@ -359,6 +359,7 @@ struct softirq_action asmlinkage void do_softirq(void); asmlinkage void __do_softirq(void); +extern void schedule_softirq_check (void );extern void open_softirq (int nr,void (*action)(struct softirq_action *));extern void softirq_init (void );#define __raise_softirq_irqoff (nr) do { or_softirq_pending(1UL << (nr)); } while (0)
kernel/sched.c +1 -0
@@ -5307,6 +5307,7 @@ need_resched:release_kernel_lock (prev); need_resched_nonpreemptible: +schedule_softirq_check ();schedule_debug (prev);if (sched_feat (HRTICK ))
kernel/softirq.c +24 -3
@@ -55,6 +55,7 @@ EXPORT_SYMBOL(irq_stat); static struct softirq_action softirq_vec[NR_SOFTIRQS] __cacheline_aligned_in_smp; static DEFINE_PER_CPU(struct task_struct *, ksoftirqd); +static DEFINE_PER_CPU(int, schedule_softirq_check);char *softirq_to_name[NR_SOFTIRQS ] = {"HI" ,"TIMER" ,"NET_TX" ,"NET_RX" ,"BLOCK" ,
@@ -76,6 +77,14 @@ void wakeup_softirqd(void)wake_up_process (tsk); } +static void set_schedule_softirq_check (void ) +{ +if (current !=__get_cpu_var (ksoftirqd)) { +__get_cpu_var (schedule_softirq_check) =1 ; +set_tsk_need_resched (current); + } +} +/* * This one is for softirq.c-internal use, * where hardirqs are disabled legitimately:
@@ -243,6 +252,7 @@ restart:lockdep_softirq_exit (); +__get_cpu_var (schedule_softirq_check) =0 ;account_system_vtime (current);_local_bh_enable (); }
@@ -269,6 +279,12 @@ asmlinkage void do_softirq(void)#endif +void schedule_softirq_check (void ) +{ +if (__get_cpu_var (schedule_softirq_check)) +do_softirq (); +} +/* * Enter an interrupt context. */
@@ -323,11 +339,16 @@ inline void raise_softirq_irqoff(unsigned int nr)* actually run the softirq once we return from * the irq or softirq. * -* Otherwise we wake up ksoftirqd to make sure we -* schedule the softirq soon. +* Otherwise we check and run the pending softirq in schedule(). +* +* NOTE: It's sometimes helpless to wakeup the ksoftirqd here. +* Because when current task is a real-time task(which +* has higher priority) or a real-time task is on standby +* in this cpu, ksoftirqd won't execute, we starve softirq +* execution. */ if (!in_interrupt ()) -wakeup_softirqd (); +set_schedule_softirq_check (); }void raise_softirq (unsigned int nr)
Re: [PATCH RFC] softirq: fix ksoftirq starved
kernel/sched.c need_resched: · line 5311
Quoted 8 lines · Lai Jiangshan
kernel/sched.c +1
-0@@ -5307,6 +5307,7 @@ need_resched:release_kernel_lock (prev); need_resched_nonpreemptible: +schedule_softirq_check ();schedule_debug (prev);
hm, this slows down the scheduler fast-path ...
Ingo
Re: [PATCH RFC] softirq: fix ksoftirq starved
Quoted 8 lines · Ingo Molnar
Quoted 6 lines · Lai Jiangshan
kernel/sched.c +0
-0@@ -5307,6 +5307,7 @@ need_resched:release_kernel_lock (prev); need_resched_nonpreemptible:
Quoted 8 lines
kernel/sched.c need_resched: · line 5311
Quoted 2 lines
kernel/sched.c +1
-0+schedule_softirq_check ();schedule_debug (prev);hm, this slows down the scheduler fast-path ...
Ingo
It's true. But:
The overheads are:
Overhead-A: the function call statement "schedule_softirq_check();" It can be gotten rid off by a macro or inline function.
Overhead-B: __get_cpu_var() and the test statement.
Overhead-C: do_softirq()
In my patch, we test a variable and then call do_softirq() when the variable is true. do_softirq() can be called from process context or from schedule() or by any other ways, but it must be called and avoids starvation in this condition.
So we need pay this overhead. It is no worse than before.
Is it a critical thing when it slows down the scheduler fast-path because of the "Overhead-B"?
Or I misunderstand something?
Thanks, Lai.
Re: [PATCH RFC] softirq: fix ksoftirq starved
Quoted 34 lines · Lai Jiangshan
Quoted 8 lines · Ingo Molnar
Quoted 6 lines · Lai Jiangshan
kernel/sched.c +0
-0@@ -5307,6 +5307,7 @@ need_resched:release_kernel_lock (prev); need_resched_nonpreemptible:Quoted 8 lines
kernel/sched.c need_resched: · line 5311
Quoted 2 lines
kernel/sched.c +1
-0+schedule_softirq_check ();schedule_debug (prev);hm, this slows down the scheduler fast-path ...
Ingo
It's true. But:
The overheads are:
Overhead-A: the function call statement "
schedule_softirq_check();" It can be gotten rid off by a macro or inline function.Overhead-B:
__get_cpu_var()and the test statement.Overhead-C:
do_softirq()
In my patch, we test a variable and then calldo_softirq()when the variable is true.do_softirq()can be called from process context or fromschedule()or by any other ways, but it must be called and avoids starvation in this condition.
So we need pay this overhead. It is no worse than before.
Quoted 4 lines
Is it a critical thing when it slows down the scheduler fast-path because of the "Overhead-B"?
Or I misunderstand something?
I think its overhead-c, actually calling do_softirq from the schedule() path that a really whacky idea.
Why not make whoemever needs the softirq to happen (eg. network ops) poll this flag, so that regular RT processes don't get penalized by random softirq muck?
It seems to me this approach basically gives softirqs higher prio than RT processes, not something to be done lightly.
Re: [PATCH RFC] softirq: fix ksoftirq starved
In reply to Lai JiangshanQuoted 34 lines · Lai Jiangshan
Quoted 8 lines · Ingo Molnar
Quoted 6 lines · Lai Jiangshan
kernel/sched.c +0
-0@@ -5307,6 +5307,7 @@ need_resched:release_kernel_lock (prev); need_resched_nonpreemptible:Quoted 8 lines
kernel/sched.c need_resched: · line 5311
Quoted 2 lines
kernel/sched.c +1
-0+schedule_softirq_check ();schedule_debug (prev);hm, this slows down the scheduler fast-path ...
Ingo
It's true. But:
The overheads are:
Overhead-A: the function call statement "
schedule_softirq_check();" It can be gotten rid off by a macro or inline function.Overhead-B:
__get_cpu_var()and the test statement.Overhead-C:
do_softirq()
In my patch, we test a variable and then calldo_softirq()when the variable is true.do_softirq()can be called from process context or fromschedule()or by any other ways, but it must be called and avoids starvation in this condition.
So we need pay this overhead. It is no worse than before.
Quoted 4 lines
Is it a critical thing when it slows down the scheduler fast-path because of the "Overhead-B"?
Or I misunderstand something?
The thing is that _any_ extra instruction in the scheduler fast-path should be avoided. I dont think it can be claimed that this problem can only be solved that way.
Ingo
Quoted 90 lines · Ingo Molnar
Quoted 38 lines · Zhaolei
* From: "Xiao Guangrong" <[email protected]>
Quoted 37 lines
Quoted 23 lines · Steven Rostedt
Quoted 22 lines · Mathieu Desnoyers
Quoted 22 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesChangelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointQuoted 9 lines
Quoted 7 lines
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.Quoted 5 lines
Quoted 1 line
Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
There's a complication: this area of the softirq code needs fixes (unrelated to tracing).
This API:
inline void
raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}is broken with RT tasks (as recently reported to lkml), as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
The proper solution would be to have a new API:
raise_softirq_check()and to remove the
wakeup_softirqd()hack fromraise_softirq_irqoff()
- and putraise_softirq_check()to all places that use raise_softirq*() from process context.
raise_softirq_check()would execute softirq handlers from process context, if there's any pending ones. It has to be called outside of bh critical sections - i.e. often a bit after theraise_softirq()has been done.
__raise_softirq_irqoff()would be made private tokernel/softirq.c, and we'd only have two public APIs to trigger softirqs:raise_softirq()andraise_softirq_irqoff(). Both just set the pending flag and dont do any wakeup.
Quoted 6 lines
As a side-effect of these fixes, the tracepoints will be sorted out as well - there wont be any need to hack into
__raise_softirq_irqoff().Ingo
Hi, Ingo
Could you give me the .config file that this bug occurs?
Lai.
Quoted 95 lines · Lai Jiangshan
Quoted 94 lines · Ingo Molnar
Quoted 38 lines · Zhaolei
* From: "Xiao Guangrong" <[email protected]>
Quoted 37 lines
Quoted 23 lines · Steven Rostedt
Quoted 22 lines · Mathieu Desnoyers
Quoted 22 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesChangelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointQuoted 9 lines
Quoted 7 lines
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.Quoted 5 lines
Quoted 1 line
Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
There's a complication: this area of the softirq code needs fixes (unrelated to tracing).
This API:
inline void
raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}is broken with RT tasks (as recently reported to lkml), as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
The proper solution would be to have a new API:
raise_softirq_check()and to remove the
wakeup_softirqd()hack fromraise_softirq_irqoff()
- and putraise_softirq_check()to all places that use raise_softirq*() from process context.
raise_softirq_check()would execute softirq handlers from process context, if there's any pending ones. It has to be called outside of bh critical sections - i.e. often a bit after theraise_softirq()has been done.
__raise_softirq_irqoff()would be made private tokernel/softirq.c, and we'd only have two public APIs to trigger softirqs:raise_softirq()andraise_softirq_irqoff(). Both just set the pending flag and dont do any wakeup.As a side-effect of these fixes, the tracepoints will be sorted out as well - there wont be any need to hack into
__raise_softirq_irqoff().
Quoted 6 lines
Quoted 2 lines
Ingo
Hi, Ingo
Could you give me the .config file that this bug occurs?
I havent reproduced this - it was reported in a recent thread on lkml. I dont have an URI for it - does anyone on the Cc: remember the details?
Ingo
Quoted 100 lines · Ingo Molnar
Quoted 98 lines · Lai Jiangshan
Quoted 89 lines · Ingo Molnar
Quoted 38 lines · Zhaolei
* From: "Xiao Guangrong" <[email protected]>
Quoted 37 lines
Quoted 23 lines · Steven Rostedt
Quoted 22 lines · Mathieu Desnoyers
Quoted 22 lines
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2
This tracepoint can trace the time stamp when softirq action is raised.Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesChangelog for v3 -> v4:
1: Come back to v2, and use forward declarations to avoid
recursive includes as Mathieu's suggestion
2: Modifiy the tracepoint name
3: Add comments for this tracepointQuoted 9 lines
Quoted 7 lines
This is a step in the right direction, but please see my email to Lai about the fact that this assumes correct and undocumented include dependencies in
kernel/trace/events.c. Not explicitely stating the include dependencies is a build error waiting to happen.Including
interrupt.hunder a ifdef would allow keeping track ofTRACE_EVENTspecific build dependencies neatly on a per header basis.This is all moot, the
events.cfile no longer exists and as not an issue.As Steve's says, use ftrace in
ftrace.hnot inevents.cnow. So, this mistake does not exist.
Dose this patch has other error? I expect for your views.Quoted 5 lines
Quoted 1 line
Thanks for your review, is great help to me. ;-)
Hello,
It seems Mathieu has no other comments on this patch now. Ingo, what is your opinion on this patch?
There's a complication: this area of the softirq code needs fixes (unrelated to tracing).
This API:
inline void
raise_softirq_irqoff(unsigned int nr)
{
__raise_softirq_irqoff(nr);/*
* If we're in an interrupt or softirq, we're done
* (this also catches softirq-disabled code). We will
* actually run the softirq once we return from
* the irq or softirq.
*
* Otherwise we wake up ksoftirqd to make sure we
* schedule the softirq soon.
*/
if (!in_interrupt())
wakeup_softirqd();
}is broken with RT tasks (as recently reported to lkml), as when a real-time task wakes up ksoftirqd (which has lower priority) it wont execute and we starve softirq execution.
The proper solution would be to have a new API:
raise_softirq_check()and to remove the
wakeup_softirqd()hack fromraise_softirq_irqoff()
- and putraise_softirq_check()to all places that use raise_softirq*() from process context.
raise_softirq_check()would execute softirq handlers from process context, if there's any pending ones. It has to be called outside of bh critical sections - i.e. often a bit after theraise_softirq()has been done.
__raise_softirq_irqoff()would be made private tokernel/softirq.c, and we'd only have two public APIs to trigger softirqs:raise_softirq()andraise_softirq_irqoff(). Both just set the pending flag and dont do any wakeup.Quoted 6 lines
As a side-effect of these fixes, the tracepoints will be sorted out as well - there wont be any need to hack into
__raise_softirq_irqoff().Ingo
Hi, Ingo
Quoted 10 lines
Quoted 1 line
Could you give me the .config file that this bug occurs?
I havent reproduced this - it was reported in a recent thread on lkml. I dont have an URI for it - does anyone on the Cc: remember the details?
Ingo
I haven't found the email you referred, I've searched it from when you pointed out this softirq problem till now. If anyone know it, please let me know.
If my analysis is correct, I will say this problem does NOT exist:
(non-PREEMPT_RT)
1) the (outmost) interrupt will call do_softirq() when it exits. softirq will not be starved when interrupts occur.
2) the clocks will provide tick-interrupt periodic.
3) even when CONFIG_NO_HZ=y, the cpu still provide periodic tick when current task is RT-task(which is non-idle task).
So any softirq will be called in time(one jiffy). Maybe my analysis is incorrect or maybe the RT task disables the irq very long time or maybe there is other bug in kernel which has influence and impacts softirq.
A) I will be still finding the email or the .config and probe this problem. The endeavors to ensure all softirq handled are not useful and it may not solve the problem. Because the tick-interrupt does ensure pending softirq handled.
B) If my thinking is correct, it's the best that we insert tracepoints at raise_softirq() functions, it's very helpful and one of its benefit is help us to probe problems which happen here.
Lai
Quoted 11 lines · Mathieu Desnoyers
I partially agree with you :
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
I partially agree with you:
Yes, Circular header dependencies is a real problem with TRACE_EVENT right now. It is also a problem with DECLARE_TRACE. It's a stubborn disease with C-Language (for complex headers). Can we fix C-Language?
o Macros in header (!CREATE_TRACE_POINTS)
When CREATE_TRACE_POINTS is not defined, TRACE_EVENT is definitely the same as DECLARE_TRACE. Actually, TRACE_EVENT is:
#define TRACE_EVENT(name, proto, args, struct, assign, print) \
DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
So TRACE_EVENT and DECLARE_TRACE are the same in header files. And so TRACE_EVENT and DECLARE_TRACE have the same advantages and disadvantages. More TRACE_EVENT equals to a known-good solution.
o Macros in c-file
tracepoint uses DEFINE_TRACE only.
ftrace uses CREATE_TRACE_POINTS + TRACE_EVENT:
#define CREATE_TRACE_POINTS #include <trace/events/sched.h> (which uses TRACE_EVENT)
ftrace generates more code which uses the tracepoints.
Quoted 9 lines
Then add a forward declaration of
struct softirqaction;
At the top of
trace/irq.h. I did it in quite a few places in the LTTng tree.TP_PROTOjust needs a forward declaration, not the full structure declaration.
Thank you for your valuable suggestions.
You are the father of tracepoint and LTTng, your experience in LTTng is very useful for ftrace.
I'm glad for your suggestions.
Xiao Guangrong, could you add forward declarations of
struct irqaction;
struct softirq_action;
at the top of trace/irq.h as Mathieu's suggestions. (and remove "#include <linux/interrupt.h>")
Lai
Quoted 59 lines · * Lai Jiangshan ([email protected]) wrote:
Quoted 11 lines · Mathieu Desnoyers
I partially agree with you :
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
I partially agree with you:
Yes, Circular header dependencies is a real problem with
TRACE_EVENTright now. It is also a problem withDECLARE_TRACE. It's a stubborn disease with C-Language (for complex headers). Can we fix C-Language?o Macros in header (!CREATE_TRACE_POINTS)
When
CREATE_TRACE_POINTSis not defined,TRACE_EVENTis definitely the same asDECLARE_TRACE. Actually,TRACE_EVENTis:#define
TRACE_EVENT(name, proto, args, struct, assign, print)\
DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))So
TRACE_EVENTandDECLARE_TRACEare the same in header files. And soTRACE_EVENTandDECLARE_TRACEhave the same advantages and disadvantages. MoreTRACE_EVENTequals to a known-good solution.o Macros in c-file
tracepoint uses
DEFINE_TRACEonly.ftrace uses
CREATE_TRACE_POINTS+TRACE_EVENT:#define CREATE_TRACE_POINTS #include <trace/events/sched.h> (which uses TRACE_EVENT)ftrace generates more code which uses the tracepoints.
Quoted 9 lines
Then add a forward declaration of
struct softirqaction;
At the top of
trace/irq.h. I did it in quite a few places in the LTTng tree.TP_PROTOjust needs a forward declaration, not the full structure declaration.Thank you for your valuable suggestions.
You are the father of tracepoint and LTTng, your experience in LTTng is very useful for ftrace.
I'm glad for your suggestions.
Quoted 8 lines
Xiao Guangrong, could you add forward declarations of
struct irqaction;
structsoftirq_action;at the top of
trace/irq.has Mathieu's suggestions. (and remove "#include <linux/interrupt.h>")
You will probably still need something like :
#ifdef CREATE_TRACE_POINTS
#include <linux/interrupt.h>
#else
struct irqaction;
struct softirq_action;
#endif
So that FTRACE has the header dependencies it needs to build.
Mathieu
Quoted 3 lines
Lai
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Quoted 65 lines · Mathieu Desnoyers
Quoted 55 lines · * Lai Jiangshan ([email protected]) wrote:
Quoted 10 lines · Mathieu Desnoyers
I partially agree with you :
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
I partially agree with you:
Yes, Circular header dependencies is a real problem with
TRACE_EVENTright now. It is also a problem withDECLARE_TRACE. It's a stubborn disease with C-Language (for complex headers). Can we fix C-Language?o Macros in header (!CREATE_TRACE_POINTS)
When
CREATE_TRACE_POINTSis not defined,TRACE_EVENTis definitely the same asDECLARE_TRACE. Actually,TRACE_EVENTis:#define
TRACE_EVENT(name, proto, args, struct, assign, print)\
DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))So
TRACE_EVENTandDECLARE_TRACEare the same in header files. And soTRACE_EVENTandDECLARE_TRACEhave the same advantages and disadvantages. MoreTRACE_EVENTequals to a known-good solution.o Macros in c-file
tracepoint uses
DEFINE_TRACEonly.ftrace uses
CREATE_TRACE_POINTS+TRACE_EVENT:#define CREATE_TRACE_POINTS #include <trace/events/sched.h> (which uses TRACE_EVENT)ftrace generates more code which uses the tracepoints.
Quoted 8 lines
Then add a forward declaration of
struct softirqaction;
At the top of
trace/irq.h. I did it in quite a few places in the LTTng tree.TP_PROTOjust needs a forward declaration, not the full structure declaration.Thank you for your valuable suggestions.
You are the father of tracepoint and LTTng, your experience in LTTng is very useful for ftrace.
I'm glad for your suggestions.
Quoted 8 lines
Xiao Guangrong, could you add forward declarations of
struct irqaction;
structsoftirq_action;at the top of
trace/irq.has Mathieu's suggestions. (and remove "#include <linux/interrupt.h>")
Quoted 9 lines
You will probably still need something like :
#ifdef
CREATE_TRACE_POINTS
#include <linux/interrupt.h>
#else
struct irqaction;
structsoftirq_action;
#endif
It's not needed for trace/events/irq.h
Yes, it's a solution.
But I don't think we have to do this, CREATE_TRACE_POINTS is needed only _once_ for every <trace/events/xxxx.h>
The .c file which defines CREATE_TRACE_POINTS can provide (had provided likely) things like "#include <linux/interrupt.h>"
See kernel/softirq.c:
#include <linux/interrupt.h>
......
......
#define CREATE_TRACE_POINTS
#include <trace/events/irq.h>
I don't think it's a problem, CREATE_TRACE_POINTS is defined only once for a <trace/events/xxxx.h>.
Lai.
Quoted 24 lines · Lai Jiangshan
Quoted 5 lines · Mathieu Desnoyers
Quoted 1 line · * Lai Jiangshan ([email protected]) wrote:
Mathieu Desnoyers wrote:
#endif
It's not needed for
trace/events/irq.hYes, it's a solution.
But I don't think we have to do this,
CREATE_TRACE_POINTSis needed only _once_ for every <trace/events/xxxx.h>The .c file which defines
CREATE_TRACE_POINTScan provide (had provided likely) things like "#include <linux/interrupt.h>"See
kernel/softirq.c:
#include <linux/interrupt.h>
......
......
#defineCREATE_TRACE_POINTS
#include <trace/events/irq.h>
Quoted 3 lines
I don't think it's a problem,
CREATE_TRACE_POINTSis defined only once for a <trace/events/xxxx.h>.
Yes, I think Lai's argument is reasonable,
the .c file which use ftrace already include the relevant head file. I will make a patch as your suggestions.
Thanks! ;-)
Quoted 4 lines
Lai.
Quoted 86 lines · * Lai Jiangshan ([email protected]) wrote:
Quoted 65 lines · Mathieu Desnoyers
Quoted 55 lines · * Lai Jiangshan ([email protected]) wrote:
Quoted 10 lines · Mathieu Desnoyers
I partially agree with you :
Yes, we should try to fix
TRACE_EVENT, but we should fix it _before_ we start using it widely. Circular header dependencies is a real problem withTRACE_EVENTright now.Until we fix this, I will be tempted to stay with a known-good solution, which is DECLARE/DEFINE_TRACE.
I partially agree with you:
Yes, Circular header dependencies is a real problem with
TRACE_EVENTright now. It is also a problem withDECLARE_TRACE. It's a stubborn disease with C-Language (for complex headers). Can we fix C-Language?o Macros in header (!CREATE_TRACE_POINTS)
When
CREATE_TRACE_POINTSis not defined,TRACE_EVENTis definitely the same asDECLARE_TRACE. Actually,TRACE_EVENTis:#define
TRACE_EVENT(name, proto, args, struct, assign, print)\
DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))So
TRACE_EVENTandDECLARE_TRACEare the same in header files. And soTRACE_EVENTandDECLARE_TRACEhave the same advantages and disadvantages. MoreTRACE_EVENTequals to a known-good solution.o Macros in c-file
tracepoint uses
DEFINE_TRACEonly.ftrace uses
CREATE_TRACE_POINTS+TRACE_EVENT:#define CREATE_TRACE_POINTS #include <trace/events/sched.h> (which uses TRACE_EVENT)ftrace generates more code which uses the tracepoints.
Quoted 8 lines
Then add a forward declaration of
struct softirqaction;
At the top of
trace/irq.h. I did it in quite a few places in the LTTng tree.TP_PROTOjust needs a forward declaration, not the full structure declaration.Thank you for your valuable suggestions.
You are the father of tracepoint and LTTng, your experience in LTTng is very useful for ftrace.
I'm glad for your suggestions.
Quoted 8 lines
Xiao Guangrong, could you add forward declarations of
struct irqaction;
structsoftirq_action;at the top of
trace/irq.has Mathieu's suggestions. (and remove "#include <linux/interrupt.h>")Quoted 9 lines
You will probably still need something like :
#ifdef
CREATE_TRACE_POINTS
#include <linux/interrupt.h>
#else
struct irqaction;
structsoftirq_action;
#endifIt's not needed for
trace/events/irq.hYes, it's a solution.
But I don't think we have to do this,
CREATE_TRACE_POINTSis needed only _once_ for every <trace/events/xxxx.h>The .c file which defines
CREATE_TRACE_POINTScan provide (had provided likely) things like "#include <linux/interrupt.h>"
Quoted 7 lines
See
kernel/softirq.c:
#include <linux/interrupt.h>
......
......
#defineCREATE_TRACE_POINTS
#include <trace/events/irq.h>
Isn't it all included under kernel/trace/events.c ? e.g. :
#include <trace/trace_events.h>
#include "trace_output.h"
#include "trace_events_stage_1.h"
#include "trace_events_stage_2.h"
#include "trace_events_stage_3.h"
Therefore, adding a #include <linux/interrupt.h> would be just weird here. We would have to do that for every tracepoint headers, and we would just lose the ability to keep track of which tracepoint header has which include dependency.
I thought the goal of TRACE_EVENT() was exactly the opposite : to declare everything in one location....
Therefore, if you agree that it's good to have TRACE_EVENT() declaring everything in one location, then I don't see why you would argue to move the include dependencies in the very remote kernel/trace/events.c file ?
Mathieu
Quoted 5 lines
I don't think it's a problem,
CREATE_TRACE_POINTSis defined only once for a <trace/events/xxxx.h>.Lai.
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Steven Rostedt
maintainerQuoted 20 lines · Mathieu Desnoyers
Quoted 12 lines
Quoted 1 line
It's not needed for
trace/events/irq.hYes, it's a solution.
But I don't think we have to do this,
CREATE_TRACE_POINTSis needed only _once_ for every <trace/events/xxxx.h>The .c file which defines
CREATE_TRACE_POINTScan provide (had provided likely) things like "#include <linux/interrupt.h>"Quoted 7 lines
See
kernel/softirq.c:
#include <linux/interrupt.h>
......
......
#defineCREATE_TRACE_POINTS
#include <trace/events/irq.h>
Quoted 1 line
Isn't it all included under
kernel/trace/events.c? e.g. :
Mathieu,
I've told you this two or three times before. That file no longer exists. Maybe it is still there for .30, but these updates are for .31 (too late for .30). The code to make TRACE_EVENT work for modules removed the events.c file and got rid of those nasty stages.
-- Steve
Quoted 33 lines
#include <
trace/trace_events.h>#include "
trace_output.h"#include "
trace_events_stage_1.h"
#include "trace_events_stage_2.h"
#include "trace_events_stage_3.h"Therefore, adding a #include <
linux/interrupt.h> would be just weird here. We would have to do that for every tracepoint headers, and we would just lose the ability to keep track of which tracepoint header has which include dependency.I thought the goal of
TRACE_EVENT()was exactly the opposite : to declare everything in one location....Therefore, if you agree that it's good to have
TRACE_EVENT()declaring everything in one location, then I don't see why you would argue to move the include dependencies in the very remotekernel/trace/events.cfile ?Mathieu
Quoted 5 lines
I don't think it's a problem,
CREATE_TRACE_POINTSis defined only once for a <trace/events/xxxx.h>.Lai.
Signature · 2 lines
Mathieu Desnoyers OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
Quoted 12 lines · Xiao Guangrong
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()
Quoted 7 lines
Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesSigned-off-by Mathieu Desnoyers <[email protected]>Signed-off-by Xiao Guangrong <[email protected]>
hi,
Could you please add 'docbook' style comments above TRACE_EVENTi()? We recently added a tracepoint docbook to the -tip and it already contains an 'irq' chapter, so you just need to add the comments for the raise softirq tracepoint. Please see:
http://marc.info/?l=linux-kernel&m=124118471411710&w=2
thanks,
-Jason
Quoted 21 lines · Jason Baron
Quoted 12 lines · Xiao Guangrong
From: Mathieu Desnoyers <[email protected]>
This patch is modified from Mathieu Desnoyers' patch. The original patch can be found here:
http://marc.info/?l=linux-kernel&m=123791201816245&w=2This tracepoint can trace the time stamp when softirq action is raised.
Changelog for v1 -> v2:
1: UseTRACE_EVENTinstead ofDEFINE_TRACE
2: Move the tracepoint fromraise_softirq_irqoff()to
__raise_softirq_irqoff()Quoted 7 lines
Changelog for v2 -> v3:
Move the definition of__raise_softifq_irqoff()to .c file whenCONFIG_TRACEPOINTSis enabled, to avoid recursive includesSigned-off-by Mathieu Desnoyers <[email protected]>Signed-off-by Xiao Guangrong <[email protected]>
Quoted 9 lines
hi,
Could you please add 'docbook' style comments above
TRACE_EVENTi()? We recently added a tracepoint docbook to the -tip and it already contains an 'irq' chapter, so you just need to add the comments for the raise softirq tracepoint. Please see:
Hi Jason:
I will add the comments for this tracepoint.
thanks!
Quoted 7 lines
thanks,
-Jason