Skip to contentSkip to conversation

rcu: idle changes v2

GIT PULL New Frederic Weisbecker 2 participants 4 messages Sep 14, 2012 – Sep 18, 2012

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Paul,

This is the second version of my rcu/idle-for-v3.7 which rebases your rcu/idle to address reviews.

It has the same modifications than rcu/idle-for-v3.7 plus an update of:

- "rcu: Exit RCU extended QS on user preemption" : update changelog and comments after a discussion with Peterz. And compile conditionally schedule_user().

- "x86: Use the new schedule_user API on userspace preemption" : build time choose between calling schedule and schedule_user.

I'm only reposting these two patches.

The new branch is:

git://github.com/fweisbec/linux-dynticks.git

	rcu/idle-v2-for-v3.7

Thanks.

#1 lore raw

PATCH 1/2 rcu: Exit RCU extended QS on user preemption

When exceptions or irq are about to resume userspace, if the task needs to be rescheduled, the arch low level code calls schedule() directly.

If we call it, it is because we have the TIF_RESCHED flag:

- It can be set after random local calls to set_need_resched() (RCU, drm, ...)

- A wake up happened and the CPU needs preemption. This can happen in several ways:

* Remotely: the remote waking CPU has set TIF_RESCHED and send the wakee an IPI to schedule the new task.
* Remotely enqueued: the remote waking CPU sends an IPI to the target and the wake up is made by the target.
* Locally: waking CPU == wakee CPU and the wakeup is done locally. set_need_resched() is called without IPI.

In the case of local and remotely enqueued wake ups, the tick can be restarted when we enqueue the new task and RCU can exit the extended quiescent state at the same time. Then by the time we reach irq exit path and we call schedule, we are not in RCU user mode.

But if we call schedule() only because something called set_need_resched(), RCU may still be in user mode when we reach schedule.

Also if a wake up is done remotely, the CPU might see the TIF_RESCHED flag and call schedule while the IPI has not yet happen to restart the tick and exit RCU user mode.

We need to manually protect against these corner cases.

Create a new API schedule_user() that calls schedule() inside rcu_user_exit()-rcu_user_enter() in order to protect it. Archs will need to rely on it now to implement user preemption safely.

Cc Alessio Igor Bogani <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Avi Kivity <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Geoff Levand <[email protected]>
Cc Gilad Ben Yossef <[email protected]>
Cc Hakan Akkan <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Max Krasnyansky <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephen Hemminger <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Sven-Thorsten Dietrich <[email protected]>
Cc Thomas Gleixner <[email protected]>
---
 kernel/sched/core.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)
kernel/sched/core.c +15 -0
index 0bd599b..7298dbf 100644
@@ -3463,6 +3463,21 @@ asmlinkage void __sched schedule(void)
 }
 EXPORT_SYMBOL(schedule);
 
+#ifdef CONFIG_RCU_USER_QS
+asmlinkage void __sched schedule_user(void)
+{
+	/*
+	 * If we come here after a random call to set_need_resched(),
+	 * or we have been woken up remotely but the IPI has not yet arrived,
+	 * we haven't yet exited the RCU idle mode. Do it here manually until
+	 * we find a better solution.
+	 */
+	rcu_user_exit();
+	schedule();
+	rcu_user_enter();
+}
+#endif
+
 /**
  * schedule_preempt_disabled - called with preemption disabled
  *
Signature · 1 line
1.7.5.4
#2 lore raw

PATCH 2/2 x86: Use the new schedule_user API on userspace preemption

In reply to Frederic Weisbecker

This way we can exit the RCU extended quiescent state before we schedule a new task from irq/exception exit.

Cc Alessio Igor Bogani <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Avi Kivity <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Geoff Levand <[email protected]>
Cc Gilad Ben Yossef <[email protected]>
Cc Hakan Akkan <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Max Krasnyansky <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephen Hemminger <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Sven-Thorsten Dietrich <[email protected]>
Cc Thomas Gleixner <[email protected]>
---
 arch/x86/include/asm/rcu.h |   12 ++++++++++++
 arch/x86/kernel/entry_64.S |    9 +++++----
 2 files changed, 17 insertions(+), 4 deletions(-)
arch/x86/include/asm/rcu.h +12 -0
index 439815b..d1ac07a 100644
@@ -1,6 +1,8 @@
 #ifndef _ASM_X86_RCU_H
 #define _ASM_X86_RCU_H
 
+#ifndef __ASSEMBLY__
+
 #include <linux/rcupdate.h>
 #include <asm/ptrace.h>
 
@@ -17,4 +19,14 @@ static inline void exception_exit(struct pt_regs *regs)
 #endif
 }
 
+#else /* __ASSEMBLY__ */
+
+#ifdef CONFIG_RCU_USER_QS
+# define SCHEDULE_USER call schedule_user
+#else
+# define SCHEDULE_USER call schedule
+#endif
+
+#endif /* !__ASSEMBLY__ */
+
 #endif
arch/x86/kernel/entry_64.S +5 -4
index 69babd8..1a8f3cb 100644
@@ -56,6 +56,7 @@
 #include <asm/ftrace.h>
 #include <asm/percpu.h>
 #include <asm/asm.h>
+#include <asm/rcu.h>
 #include <linux/err.h>
 
 /* Avoid __ASSEMBLER__'ifying <linux/audit.h> just for this.  */
@@ -565,7 +566,7 @@ sysret_careful:
 	TRACE_IRQS_ON
 	ENABLE_INTERRUPTS(CLBR_NONE)
 	pushq_cfi %rdi
-	call schedule
+	SCHEDULE_USER
 	popq_cfi %rdi
 	jmp sysret_check
 
@@ -678,7 +679,7 @@ int_careful:
 	TRACE_IRQS_ON
 	ENABLE_INTERRUPTS(CLBR_NONE)
 	pushq_cfi %rdi
-	call schedule
+	SCHEDULE_USER
 	popq_cfi %rdi
 	DISABLE_INTERRUPTS(CLBR_NONE)
 	TRACE_IRQS_OFF
@@ -974,7 +975,7 @@ retint_careful:
 	TRACE_IRQS_ON
 	ENABLE_INTERRUPTS(CLBR_NONE)
 	pushq_cfi %rdi
-	call  schedule
+	SCHEDULE_USER
 	popq_cfi %rdi
 	GET_THREAD_INFO(%rcx)
 	DISABLE_INTERRUPTS(CLBR_NONE)
@@ -1449,7 +1450,7 @@ paranoid_userspace:
 paranoid_schedule:
 	TRACE_IRQS_ON
 	ENABLE_INTERRUPTS(CLBR_ANY)
-	call schedule
+	SCHEDULE_USER
 	DISABLE_INTERRUPTS(CLBR_ANY)
 	TRACE_IRQS_OFF
 	jmp paranoid_userspace
Signature · 1 line
1.7.5.4
#3 lore raw
In reply to Frederic Weisbecker
Quoted 15 lines · Frederic Weisbecker

Paul,

This is the second version of my rcu/idle-for-v3.7 which rebases your rcu/idle to address reviews.

It has the same modifications than rcu/idle-for-v3.7 plus an update of:

- "rcu: Exit RCU extended QS on user preemption" : update changelog and comments after a discussion with Peterz. And compile conditionally schedule_user().

- "x86: Use the new schedule_user API on userspace preemption" : build time choose between calling schedule and schedule_user.

Quoted 6 lines

I'm only reposting these two patches.

The new branch is:

git://github.com/fweisbec/linux-dynticks.git

	rcu/idle-v2-for-v3.7

Pulled, thank you!

I also modified the first patch in the series to remove the exports based on Josh's patch (7ece55a4: Don't declare trace_*_rcuidle functions in modules) in -tip.

Thanx, Paul