Skip to contentSkip to conversation

tracing: Make sure RCU is watching before calling a stack trace

PATCH Superseded Steven Rostedt 3 participants 10 messages May 18, 2017 Tracing

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

PATCH v2 1/1 tracing: Make sure RCU is watching before calling a stack trace

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-

 1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +25 -1
index fcc9a2d..34a98ba 100644
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {
+		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+		return;
+	}
+
+	/*
+	 * It is possible that a function is being traced in a
+	 * location that RCU is not watching. A call to
+	 * rcu_irq_enter() will make sure that it is, but there's
+	 * a few internal rcu functions that could be traced
+	 * where that wont work either. In those cases, we just
+	 * do nothing.
+	 */
+	if (unlikely(rcu_irq_enter_disabled()))
+		return;
+
+	rcu_irq_enter();
+	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+	rcu_irq_exit();
 }
 
 /**
Signature · 1 line
2.9.3
#1 lore raw
Quoted 10 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

Quoted 8 lines

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

Would rcu_irq_enter_irqson() help then? This is what Petr used in a live patching handler.

Your solution works too, of course. Just asking if I am not missing something.

Thanks,
Miroslav

author maintainer
#2 lore raw
Quoted 16 lines · On Thu, 18 May 2017 15:48:55 +0200 (CEST) Miroslav Benes <[email protected]> wrote:
Quoted 14 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

Quoted 8 lines
Quoted 4 lines

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

Would rcu_irq_enter_irqson() help then? This is what Petr used in a live patching handler.

Yes, that could work too, but I wanted to avoid disabling interrupts if we didn't have to.

Quoted 3 lines

Your solution works too, of course. Just asking if I am not missing something.

Nope, I was just trying to keep the overhead down. As this can be called by every event enabled, as well as functions being traced. I figured that local_save_irqs() is faster than a pair of local_irq_save()/ local_irq_restore() calls.

-- Steve

#3 lore raw
Quoted 24 lines · Steven Rostedt
Quoted 21 lines · On Thu, 18 May 2017 15:48:55 +0200 (CEST) Miroslav Benes <[email protected]> wrote:
Quoted 18 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

Quoted 6 lines
Quoted 3 lines

Would rcu_irq_enter_irqson() help then? This is what Petr used in a live patching handler.

Yes, that could work too, but I wanted to avoid disabling interrupts if we didn't have to.

Ok, that makes sense.

Quoted 8 lines
Quoted 3 lines

Your solution works too, of course. Just asking if I am not missing something.

Nope, I was just trying to keep the overhead down. As this can be called by every event enabled, as well as functions being traced. I figured that local_save_irqs() is faster than a pair of local_irq_save()/ local_irq_restore() calls.

(noticed Paul's reply)... yeah, it'd great. Damn, this is mindblowing.

Thanks,
Miroslav

author maintainer
#4 lore raw
In reply to Steven Rostedt
Quoted 8 lines · On Thu, 18 May 2017 09:38:09 -0400 Steven Rostedt <[email protected]> wrote:

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

Cc "Paul E. McKenney" <[email protected]>

Paul,

Can you give me an ack for this version too.

Thanks!

-- Steve

Quoted 50 lines

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-

 1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +25 -1
index fcc9a2d..34a98ba 100644
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {
+		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+		return;
+	}
+
+	/*
+	 * It is possible that a function is being traced in a
+	 * location that RCU is not watching. A call to
+	 * rcu_irq_enter() will make sure that it is, but there's
+	 * a few internal rcu functions that could be traced
+	 * where that wont work either. In those cases, we just
+	 * do nothing.
+	 */
+	if (unlikely(rcu_irq_enter_disabled()))
+		return;
+
+	rcu_irq_enter();
+	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+	rcu_irq_exit();
 }
 
 /**
#5 lore raw
In reply to Steven Rostedt
Quoted 28 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-

 1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +0 -0
index fcc9a2d..34a98ba 100644
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,

kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577

Quoted 10 lines
kernel/trace/trace.c +7 -1
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {

You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?

Thanx, Paul

Quoted 24 lines
kernel/trace/trace.c +18 -0
+		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+		return;
+	}
+
+	/*
+	 * It is possible that a function is being traced in a
+	 * location that RCU is not watching. A call to
+	 * rcu_irq_enter() will make sure that it is, but there's
+	 * a few internal rcu functions that could be traced
+	 * where that wont work either. In those cases, we just
+	 * do nothing.
+	 */
+	if (unlikely(rcu_irq_enter_disabled()))
+		return;
+
+	rcu_irq_enter();
+	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+	rcu_irq_exit();
 }
 
 /**
Signature · 1 line
2.9.3
author maintainer
#6 lore raw
Quoted 34 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 33 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-

 1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +2 -1
index fcc9a2d..34a98ba 100644
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
Quoted 9 lines

kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577

Quoted 5 lines
kernel/trace/trace.c +5 -0
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {  

You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?

Hmm, no, maybe I'm missing something :-/

OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(

-- Steve

Quoted 26 lines

Thanx, Paul

Quoted 24 lines
kernel/trace/trace.c +18 -0
+		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+		return;
+	}
+
+	/*
+	 * It is possible that a function is being traced in a
+	 * location that RCU is not watching. A call to
+	 * rcu_irq_enter() will make sure that it is, but there's
+	 * a few internal rcu functions that could be traced
+	 * where that wont work either. In those cases, we just
+	 * do nothing.
+	 */
+	if (unlikely(rcu_irq_enter_disabled()))
+		return;
+
+	rcu_irq_enter();
+	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+	rcu_irq_exit();
 }
 
 /**
Signature · 1 line
2.9.3
#7 lore raw
Quoted 47 lines · Steven Rostedt
Quoted 34 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 33 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-

 1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +2 -1
index fcc9a2d..34a98ba 100644
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
Quoted 9 lines

kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577

Quoted 5 lines
kernel/trace/trace.c +5 -0
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {  

You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?

Quoted 4 lines

Hmm, no, maybe I'm missing something :-/

OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(

If rcu_is_watching() returns false, yes. Of course, if rcu_is_watching() returns true, you are golden.

Thanx, Paul

Quoted 30 lines

-- Steve

Quoted 26 lines

Thanx, Paul

Quoted 24 lines
kernel/trace/trace.c +18 -0
+		__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+		return;
+	}
+
+	/*
+	 * It is possible that a function is being traced in a
+	 * location that RCU is not watching. A call to
+	 * rcu_irq_enter() will make sure that it is, but there's
+	 * a few internal rcu functions that could be traced
+	 * where that wont work either. In those cases, we just
+	 * do nothing.
+	 */
+	if (unlikely(rcu_irq_enter_disabled()))
+		return;
+
+	rcu_irq_enter();
+	__ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+	rcu_irq_exit();
 }
 
 /**
Signature · 1 line
2.9.3
author maintainer
#8 lore raw
Quoted 48 lines · On Thu, 18 May 2017 11:45:28 -0700 "Paul E. McKenney"…
Quoted 47 lines · Steven Rostedt
Quoted 43 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 28 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-

 1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +0 -0
index fcc9a2d..34a98ba 100644
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,

kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577

Quoted 10 lines
kernel/trace/trace.c +7 -1
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {    

You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?

Quoted 8 lines
Quoted 4 lines

Hmm, no, maybe I'm missing something :-/

OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(

If rcu_is_watching() returns false, yes. Of course, if rcu_is_watching() returns true, you are golden.

Hmm, so I can just replace the entire if with:

	if (rcu_is_watching())

because it should be watching in NMI context, correct?

-- Steve

#9 lore raw
Quoted 60 lines · Steven Rostedt
Quoted 48 lines · On Thu, 18 May 2017 11:45:28 -0700 "Paul E. McKenney"…
Quoted 47 lines · Steven Rostedt
Quoted 43 lines · On Thu, 18 May 2017 08:47:11 -0700 "Paul E. McKenney"…
Quoted 28 lines · Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

As stack tracing now requires "rcu watching", force RCU to be watching when recording a stack trace.

---

Changes since v1:

My testing discovered that the stack trace can be called with interrupts enabled, which is a no no to have when calling rcu_irq_enter(). When interrupts are enabled, as with being in an NMI, RCU will also be watching.

kernel/trace/trace.c | 26 +++++++++++++++++++++++++-

 1 file changed, 25 insertions(+), 1 deletion(-)
kernel/trace/trace.c +0 -0
index fcc9a2d..34a98ba 100644
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
 void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,

kernel/trace/trace.c static inline void ftrace_trace_stack(struct trace_array *tr, · line 2577

Quoted 10 lines
kernel/trace/trace.c +7 -1
 		   int pc)
 {
-	__ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+	struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
+	/*
+	 * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+	 * Also, RCU is always enabled when interrupts are.
+	 */
+	if (!irqs_disabled() || in_nmi()) {    

You lost me on this one. RCU might not be watching if irqs are enabled, for example, in the idle loop. What am I missing here?

Quoted 8 lines
Quoted 4 lines

Hmm, no, maybe I'm missing something :-/

OK, so if we trace in the idle loop, rcu may not be watching, so I may need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(

If rcu_is_watching() returns false, yes. Of course, if rcu_is_watching() returns true, you are golden.

Quoted 5 lines

Hmm, so I can just replace the entire if with:

	if (rcu_is_watching())

because it should be watching in NMI context, correct?

Yes, and that would be even better! ;-)

However, this assumes that there is no possibility of this being called during the time that in_nmi() returns true but RCU is not aware of the NMI. (Yes, we did go through this a bit earlier, just my usual paranoia showing...)

Thanx, Paul