tracing: Make sure RCU is watching before calling a stack trace
Steven Rostedt
maintainerPATCH v2 1/1 tracing: Make sure RCU is watching before calling a stack trace
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +25 -1
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,void __trace_stack (struct trace_array *tr,unsigned long flags,int skip,int pc) { -__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); +struct ring_buffer *buffer = tr->trace_buffer.buffer; + +/* +* When an NMI triggers, RCU is enabled via rcu_nmi_enter() +* Also, RCU is always enabled when interrupts are. +*/ +if (!irqs_disabled () ||in_nmi ()) { +__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +return ; + } + +/* +* It is possible that a function is being traced in a +* location that RCU is not watching. A call to +* rcu_irq_enter() will make sure that it is, but there's +* a few internal rcu functions that could be traced +* where that wont work either. In those cases, we just +* do nothing. +*/ +if (unlikely (rcu_irq_enter_disabled ())) +return ; + +rcu_irq_enter (); +__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +rcu_irq_exit (); }/**
Signature · 1 line
2.9.3
Quoted 10 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>
Quoted 8 lines
---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
Would rcu_irq_enter_irqson() help then? This is what Petr used in a live patching handler.
Your solution works too, of course. Just asking if I am not missing something.
Thanks,
Miroslav
Steven Rostedt
maintainerQuoted 16 lines · On Thu, 18 May 2017 15:48:55 +0200 (CEST) Miroslav Benes <[email protected]> wrote:
Quoted 14 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
Quoted 8 lines
Quoted 4 lines
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.Would
rcu_irq_enter_irqson()help then? This is what Petr used in a live patching handler.
Yes, that could work too, but I wanted to avoid disabling interrupts if we didn't have to.
Quoted 3 lines
Your solution works too, of course. Just asking if I am not missing something.
Nope, I was just trying to keep the overhead down. As this can be called by every event enabled, as well as functions being traced. I figured that local_save_irqs() is faster than a pair of local_irq_save()/ local_irq_restore() calls.
-- Steve
Quoted 24 lines · Steven Rostedt
Quoted 21 lines · On Thu, 18 May 2017 15:48:55 +0200 (CEST) Miroslav Benes <[email protected]> wrote:
Quoted 18 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
Quoted 6 lines
Quoted 3 lines
Would
rcu_irq_enter_irqson()help then? This is what Petr used in a live patching handler.Yes, that could work too, but I wanted to avoid disabling interrupts if we didn't have to.
Ok, that makes sense.
Quoted 8 lines
Quoted 3 lines
Your solution works too, of course. Just asking if I am not missing something.
Nope, I was just trying to keep the overhead down. As this can be called by every event enabled, as well as functions being traced. I figured that
local_save_irqs()is faster than a pair of local_irq_save()/local_irq_restore()calls.
(noticed Paul's reply)... yeah, it'd great. Damn, this is mindblowing.
Thanks,
Miroslav
Steven Rostedt
maintainerQuoted 8 lines · On Thu, 18 May 2017 09:38:09 -0400 Steven Rostedt <[email protected]> wrote:
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>
Paul,
Can you give me an ack for this version too.
Thanks!
-- Steve
Quoted 50 lines
Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
kernel/trace/trace.c| 26 +++++++++++++++++++++++++-1 file changed, 25 insertions(+), 1 deletion(-)kernel/trace/trace.c +25
-1@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,void __trace_stack (struct trace_array *tr,unsigned long flags,int skip,int pc) { -__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); +struct ring_buffer *buffer = tr->trace_buffer.buffer; + +/* +* When an NMI triggers, RCU is enabled via rcu_nmi_enter() +* Also, RCU is always enabled when interrupts are. +*/ +if (!irqs_disabled () ||in_nmi ()) { +__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +return ; + } + +/* +* It is possible that a function is being traced in a +* location that RCU is not watching. A call to +* rcu_irq_enter() will make sure that it is, but there's +* a few internal rcu functions that could be traced +* where that wont work either. In those cases, we just +* do nothing. +*/ +if (unlikely (rcu_irq_enter_disabled ())) +return ; + +rcu_irq_enter (); +__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +rcu_irq_exit (); }/**
Quoted 28 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
kernel/trace/trace.c| 26 +++++++++++++++++++++++++-1 file changed, 25 insertions(+), 1 deletion(-)kernel/trace/trace.c +0
-0@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr, void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577
Quoted 10 lines
kernel/trace/trace.c +7
-1int pc) { -__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); +struct ring_buffer *buffer = tr->trace_buffer.buffer; + +/* +* When an NMI triggers, RCU is enabled via rcu_nmi_enter() +* Also, RCU is always enabled when interrupts are. +*/ +if (!irqs_disabled () ||in_nmi ()) {
You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?
Thanx, Paul
Quoted 24 lines
kernel/trace/trace.c +18
-0+__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +return ; + } + +/* +* It is possible that a function is being traced in a +* location that RCU is not watching. A call to +* rcu_irq_enter() will make sure that it is, but there's +* a few internal rcu functions that could be traced +* where that wont work either. In those cases, we just +* do nothing. +*/ +if (unlikely (rcu_irq_enter_disabled ())) +return ; + +rcu_irq_enter (); +__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +rcu_irq_exit (); }/** Signature · 1 line
2.9.3
Steven Rostedt
maintainerQuoted 34 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 33 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
kernel/trace/trace.c| 26 +++++++++++++++++++++++++-1 file changed, 25 insertions(+), 1 deletion(-)kernel/trace/trace.c +2
-1@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,void __trace_stack (struct trace_array *tr,unsigned long flags,int skip,int pc) { -__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); +struct ring_buffer *buffer = tr->trace_buffer.buffer; +
Quoted 9 lines
kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577
Quoted 5 lines
kernel/trace/trace.c +5
-0+/* +* When an NMI triggers, RCU is enabled via rcu_nmi_enter() +* Also, RCU is always enabled when interrupts are. +*/ +if (!irqs_disabled () ||in_nmi ()) {You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?
Hmm, no, maybe I'm missing something :-/
OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(
-- Steve
Quoted 26 lines
Thanx, Paul
Quoted 24 lines
kernel/trace/trace.c +18
-0+__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +return ; + } + +/* +* It is possible that a function is being traced in a +* location that RCU is not watching. A call to +* rcu_irq_enter() will make sure that it is, but there's +* a few internal rcu functions that could be traced +* where that wont work either. In those cases, we just +* do nothing. +*/ +if (unlikely (rcu_irq_enter_disabled ())) +return ; + +rcu_irq_enter (); +__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +rcu_irq_exit (); }/** Signature · 1 line
2.9.3
Quoted 47 lines · Steven Rostedt
Quoted 34 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 33 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
kernel/trace/trace.c| 26 +++++++++++++++++++++++++-1 file changed, 25 insertions(+), 1 deletion(-)kernel/trace/trace.c +2
-1@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,void __trace_stack (struct trace_array *tr,unsigned long flags,int skip,int pc) { -__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); +struct ring_buffer *buffer = tr->trace_buffer.buffer; +Quoted 9 lines
kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577
Quoted 5 lines
kernel/trace/trace.c +5
-0+/* +* When an NMI triggers, RCU is enabled via rcu_nmi_enter() +* Also, RCU is always enabled when interrupts are. +*/ +if (!irqs_disabled () ||in_nmi ()) {You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?
Quoted 4 lines
Hmm, no, maybe I'm missing something :-/
OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the
rcu_irq_enter_irqon()instead :-(
If rcu_is_watching() returns false, yes. Of course, if rcu_is_watching() returns true, you are golden.
Thanx, Paul
Quoted 30 lines
-- Steve
Quoted 26 lines
Thanx, Paul
Quoted 24 lines
kernel/trace/trace.c +18
-0+__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +return ; + } + +/* +* It is possible that a function is being traced in a +* location that RCU is not watching. A call to +* rcu_irq_enter() will make sure that it is, but there's +* a few internal rcu functions that could be traced +* where that wont work either. In those cases, we just +* do nothing. +*/ +if (unlikely (rcu_irq_enter_disabled ())) +return ; + +rcu_irq_enter (); +__ftrace_trace_stack (buffer, flags, skip, pc,NULL ); +rcu_irq_exit (); }/** Signature · 1 line
2.9.3
Steven Rostedt
maintainerQuoted 48 lines · On Thu, 18 May 2017 11:45:28 -0700 "Paul E. McKenney"…
Quoted 47 lines · Steven Rostedt
Quoted 43 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 28 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
kernel/trace/trace.c| 26 +++++++++++++++++++++++++-1 file changed, 25 insertions(+), 1 deletion(-)kernel/trace/trace.c +0
-0@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr, void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577
Quoted 10 lines
kernel/trace/trace.c +7
-1int pc) { -__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); +struct ring_buffer *buffer = tr->trace_buffer.buffer; + +/* +* When an NMI triggers, RCU is enabled via rcu_nmi_enter() +* Also, RCU is always enabled when interrupts are. +*/ +if (!irqs_disabled () ||in_nmi ()) {You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?
Quoted 8 lines
Quoted 4 lines
Hmm, no, maybe I'm missing something :-/
OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the
rcu_irq_enter_irqon()instead :-(If
rcu_is_watching()returns false, yes. Of course, ifrcu_is_watching()returns true, you are golden.
Hmm, so I can just replace the entire if with:
if (rcu_is_watching())
because it should be watching in NMI context, correct?
-- Steve
Quoted 60 lines · Steven Rostedt
Quoted 48 lines · On Thu, 18 May 2017 11:45:28 -0700 "Paul E. McKenney"…
Quoted 47 lines · Steven Rostedt
Quoted 43 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 28 lines · Steven Rostedt
From: "Steven Rostedt (VMware)" <[email protected]>
As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Steven Rostedt (VMware) <[email protected]>---
Changes since v1:
My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.
kernel/trace/trace.c| 26 +++++++++++++++++++++++++-1 file changed, 25 insertions(+), 1 deletion(-)kernel/trace/trace.c +0
-0@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr, void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577
Quoted 10 lines
kernel/trace/trace.c +7
-1int pc) { -__ftrace_trace_stack (tr->trace_buffer.buffer, flags, skip, pc,NULL ); +struct ring_buffer *buffer = tr->trace_buffer.buffer; + +/* +* When an NMI triggers, RCU is enabled via rcu_nmi_enter() +* Also, RCU is always enabled when interrupts are. +*/ +if (!irqs_disabled () ||in_nmi ()) {You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?
Quoted 8 lines
Quoted 4 lines
Hmm, no, maybe I'm missing something :-/
OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the
rcu_irq_enter_irqon()instead :-(If
rcu_is_watching()returns false, yes. Of course, ifrcu_is_watching()returns true, you are golden.
Quoted 5 lines
Hmm, so I can just replace the entire if with:
if (rcu_is_watching())because it should be watching in NMI context, correct?
Yes, and that would be even better! ;-)
However, this assumes that there is no possibility of this being called during the time that in_nmi() returns true but RCU is not aware of the NMI. (Yes, we did go through this a bit earlier, just my usual paranoia showing...)
Thanx, Paul