rcu: Allow for page faults in NMI handlers

PATCH In review Paul E. McKenney 2 participants 5 messages Oct 12, 2017 – Oct 13, 2017

Subsystems (2)

Stable branch RCU

lore

Share & tools

lore

Paul E. McKenney

author

2017-10-12 23:10 UTC #0 lore raw

PATCH 1/1 rcu: Allow for page faults in NMI handlers

A number of architecture invoke rcu_irq_enter() on exception entry in order to allow RCU read-side critical sections in the exception handler when the exception is from an idle or nohz_full CPU. This works, at least unless the exception happens in an NMI handler. In that case, rcu_nmi_enter() would already have exited the extended quiescent state, which would mean that rcu_irq_enter() would (incorrectly) cause RCU to think that it is again in an extended quiescent state. This will in turn result in lockdep splats in response to later RCU read-side critical sections.

This commit therefore causes rcu_irq_enter() and rcu_irq_exit() to take no action if there is an rcu_nmi_enter() in effect, thus avoiding the unscheduled return to RCU quiescent state. This in turn should make the kernel safe for on-demand RCU voyeurism.

Link http://lkml.kernel.org/r/[email protected] open here

Cc [email protected] # 4.2.x

Fixes 0be964be0 ("module: Sanitize RCU usage and locking")

Reported-by Steven Rostedt <[email protected]>

Signed-off-by Paul E. McKenney <[email protected]>

Signed-off-by Steven Rostedt (VMware) <[email protected]>

kernel/rcu/tree.c +12 -0

index f07343b54fe5..8a62cbfe1f2f 100644
@@ -759,6 +759,12 @@ void rcu_irq_exit(void)
 
 	local_irq_save(flags);
 	rdtp = this_cpu_ptr(&rcu_dynticks);
+
+	/* Page faults can happen in NMI handlers, so check... */
+	if (READ_ONCE(rdtp->dynticks_nmi_nesting))
+		return;
+
+	RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_irq_exit() invoked with irqs enabled!!!");
 	oldval = rdtp->dynticks_nesting;
 	rdtp->dynticks_nesting--;
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&

@@ -887,6 +893,12 @@ void rcu_irq_enter(void)
 
 	local_irq_save(flags);
 	rdtp = this_cpu_ptr(&rcu_dynticks);
+
+	/* Page faults can happen in NMI handlers, so check... */
+	if (READ_ONCE(rdtp->dynticks_nmi_nesting))
+		return;
+
+	RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_irq_enter() invoked with irqs enabled!!!");
 	oldval = rdtp->dynticks_nesting;
 	rdtp->dynticks_nesting++;
 	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) &&

Greg KH

maintainer

2017-10-13 09:52 UTC #1 lore raw

Quoted 15 lines · Paul E. McKenney

A number of architecture invoke rcu_irq_enter() on exception entry in order to allow RCU read-side critical sections in the exception handler when the exception is from an idle or nohz_full CPU. This works, at least unless the exception happens in an NMI handler. In that case, rcu_nmi_enter() would already have exited the extended quiescent state, which would mean that rcu_irq_enter() would (incorrectly) cause RCU to think that it is again in an extended quiescent state. This will in turn result in lockdep splats in response to later RCU read-side critical sections.
This commit therefore causes rcu_irq_enter() and rcu_irq_exit() to take no action if there is an rcu_nmi_enter() in effect, thus avoiding the unscheduled return to RCU quiescent state. This in turn should make the kernel safe for on-demand RCU voyeurism.

Quoted 7 lines

Link http://lkml.kernel.org/r/[email protected] open here
Cc [email protected] # 4.2.x
Fixes 0be964be0 ("module: Sanitize RCU usage and locking")
Reported-by Steven Rostedt <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Signed-off-by Steven Rostedt (VMware) <[email protected]>

This does not apply to the 4.4-stable tree, was this the correct patch I should be using for that tree?

thanks,

greg k-h

Paul E. McKenney

author

2017-10-13 16:25 UTC #2 lore raw

Quoted 18 lines · Greg KH

Quoted 17 lines · Paul E. McKenney
A number of architecture invoke rcu_irq_enter() on exception entry in order to allow RCU read-side critical sections in the exception handler when the exception is from an idle or nohz_full CPU. This works, at least unless the exception happens in an NMI handler. In that case, rcu_nmi_enter() would already have exited the extended quiescent state, which would mean that rcu_irq_enter() would (incorrectly) cause RCU to think that it is again in an extended quiescent state. This will in turn result in lockdep splats in response to later RCU read-side critical sections.
This commit therefore causes rcu_irq_enter() and rcu_irq_exit() to take no action if there is an rcu_nmi_enter() in effect, thus avoiding the unscheduled return to RCU quiescent state. This in turn should make the kernel safe for on-demand RCU voyeurism.
Link http://lkml.kernel.org/r/[email protected] open here

Quoted 8 lines

Quoted 5 lines
Cc [email protected] # 4.2.x
Fixes 0be964be0 ("module: Sanitize RCU usage and locking")
Reported-by Steven Rostedt <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Signed-off-by Steven Rostedt (VMware) <[email protected]>
This does not apply to the 4.4-stable tree, was this the correct patch I should be using for that tree?

Hmmm... Idiot here backported to v4.4 in Linus's tree, which of course might not work... But it applies to v4.4.92 of -stable:

git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git

Am I still applying it in the wrong place?

Thanx, Paul

Greg KH

maintainer

2017-10-13 16:33 UTC #3 lore raw

Quoted 28 lines · Paul E. McKenney

Quoted 18 lines · Greg KH
Quoted 17 lines · Paul E. McKenney
A number of architecture invoke rcu_irq_enter() on exception entry in order to allow RCU read-side critical sections in the exception handler when the exception is from an idle or nohz_full CPU. This works, at least unless the exception happens in an NMI handler. In that case, rcu_nmi_enter() would already have exited the extended quiescent state, which would mean that rcu_irq_enter() would (incorrectly) cause RCU to think that it is again in an extended quiescent state. This will in turn result in lockdep splats in response to later RCU read-side critical sections.
This commit therefore causes rcu_irq_enter() and rcu_irq_exit() to take no action if there is an rcu_nmi_enter() in effect, thus avoiding the unscheduled return to RCU quiescent state. This in turn should make the kernel safe for on-demand RCU voyeurism.
Link http://lkml.kernel.org/r/[email protected] open here
Quoted 8 lines
Quoted 5 lines
Cc [email protected] # 4.2.x
Fixes 0be964be0 ("module: Sanitize RCU usage and locking")
Reported-by Steven Rostedt <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Signed-off-by Steven Rostedt (VMware) <[email protected]>
This does not apply to the 4.4-stable tree, was this the correct patch I should be using for that tree?

Quoted 6 lines

Hmmm... Idiot here backported to v4.4 in Linus's tree, which of course might not work... But it applies to v4.4.92 of -stable:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Am I still applying it in the wrong place?

Ugh, no, you are doing the right thing. I don't know what failed on my end, that's what I get for trying to deal with RCU early in the morning :)

I'll go apply this now, sorry for the noise, and thanks for checking and the backport.

greg k-h

Paul E. McKenney

author

2017-10-13 16:48 UTC #4 lore raw

Quoted 32 lines · Greg KH

Quoted 31 lines · Paul E. McKenney
Quoted 18 lines · Greg KH
Quoted 17 lines · Paul E. McKenney
A number of architecture invoke rcu_irq_enter() on exception entry in order to allow RCU read-side critical sections in the exception handler when the exception is from an idle or nohz_full CPU. This works, at least unless the exception happens in an NMI handler. In that case, rcu_nmi_enter() would already have exited the extended quiescent state, which would mean that rcu_irq_enter() would (incorrectly) cause RCU to think that it is again in an extended quiescent state. This will in turn result in lockdep splats in response to later RCU read-side critical sections.
This commit therefore causes rcu_irq_enter() and rcu_irq_exit() to take no action if there is an rcu_nmi_enter() in effect, thus avoiding the unscheduled return to RCU quiescent state. This in turn should make the kernel safe for on-demand RCU voyeurism.
Link http://lkml.kernel.org/r/[email protected] open here
Quoted 8 lines
Quoted 5 lines
Cc [email protected] # 4.2.x
Fixes 0be964be0 ("module: Sanitize RCU usage and locking")
Reported-by Steven Rostedt <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Signed-off-by Steven Rostedt (VMware) <[email protected]>
This does not apply to the 4.4-stable tree, was this the correct patch I should be using for that tree?
Hmmm... Idiot here backported to v4.4 in Linus's tree, which of course might not work... But it applies to v4.4.92 of -stable:

Quoted 7 lines

Quoted 3 lines
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Am I still applying it in the wrong place?
Ugh, no, you are doing the right thing. I don't know what failed on my end, that's what I get for trying to deal with RCU early in the morning :)

Believe me, I know that feeling very well! ;-)

Thanx, Paul

Quoted 5 lines

I'll go apply this now, sorry for the noise, and thanks for checking and the backport.
greg k-h