Skip to contentSkip to conversation

nohz: Full dynticks base interface

GIT PULL New Frederic Weisbecker 3 participants 13 messages Mar 21, 2013 – Mar 27, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=
* Improve some error handling
* Improve some documentation (comments, Kconfig, ...)

Many of these changes may have been suggested by several reviewers. Sorry for the missing credits, I lost a bit track...

Thanks.
---
Frederic Weisbecker (3):
nohz: Basic full dynticks interface
nohz: Assign timekeeping duty to a CPU outside the full dynticks
range
nohz: Wake up full dynticks CPUs when a timer gets enqueued

 Documentation/kernel-parameters.txt |    6 ++
 include/linux/sched.h               |    4 +-
 include/linux/tick.h                |    7 ++
 kernel/sched/core.c                 |   20 ++++++-
 kernel/time/Kconfig                 |   19 ++++++
 kernel/time/tick-broadcast.c        |    3 +-
 kernel/time/tick-common.c           |    5 +-
 kernel/time/tick-sched.c            |  107 ++++++++++++++++++++++++++++++++++-
 kernel/timer.c                      |   12 ++--
 9 files changed, 171 insertions(+), 12 deletions(-)
Signature · 1 line
1.7.5.4
#1 lore raw

PATCH 1/3 nohz: Basic full dynticks interface

For extreme usecases such as Real Time or HPC, having the ability to shutdown the tick when a single task runs on a CPU is a desired feature:

* Reducing the amount of interrupts improves throughput for CPU-bound tasks. The CPU is less distracted from its real job, from an execution time and from the cache point of views.

* This also improve latency response as we have less critical sections.

Start with introducing a very simple interface to define full dynticks CPU: use a boot time option defined cpumask through the "nohz_extended=" kernel parameter. CPUs that are part of this range will have their tick shutdown whenever possible: provided they run a single task and they don't do kernel activity that require the periodic tick. These details will be later documented in Documentation/*

An online CPU must be kept outside this range to handle the timekeeping.

Cc Andrew Morton <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Geoff Levand <[email protected]>
Cc Gilad Ben Yossef <[email protected]>
Cc Hakan Akkan <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Li Zhong <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Paul Gortmaker <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
---
 Documentation/kernel-parameters.txt |    6 +++
 include/linux/tick.h                |    7 ++++
 kernel/time/Kconfig                 |   19 +++++++++++
 kernel/time/tick-sched.c            |   62 +++++++++++++++++++++++++++++++++++
 4 files changed, 94 insertions(+), 0 deletions(-)
Documentation/kernel-parameters.txt +6 -0
index 4609e81..231698f 100644
@@ -1913,6 +1913,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 			Valid arguments: on, off
 			Default: on
 
+	nohz_extended=  [KNL,BOOT]
+			In kernels built with CONFIG_NO_HZ_EXTENDED=y, set
+			the specified list of CPUs whose tick will be stopped
+			whenever possible. You need to keep at least one online
+			CPU outside the range to maintain the timekeeping.
+
 	noiotrap	[SH] Disables trapped I/O port accesses.
 
 	noirqdebug	[X86-32] Disables the code which attempts to detect and
include/linux/tick.h +7 -0
index 553272e..44bfa8a 100644
@@ -157,6 +157,13 @@ static inline u64 get_cpu_idle_time_us(int cpu, u64 *unused) { return -1; }
 static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
 # endif /* !NO_HZ */
 
+#ifdef CONFIG_NO_HZ_EXTENDED
+extern int tick_nohz_extended_cpu(int cpu);
+#else
+static inline int tick_nohz_extended_cpu(int cpu) { return 0; }
+#endif
+
+
 # ifdef CONFIG_CPU_IDLE_GOV_MENU
 extern void menu_hrtimer_cancel(void);
 # else
kernel/time/Kconfig +19 -0
index 24510d8..5a87c03 100644
@@ -79,6 +79,25 @@ config NO_HZ
 	  only trigger on an as-needed basis both when the system is
 	  busy and when the system is idle.
 
+config NO_HZ_EXTENDED
+       bool "Full dynticks system"
+       depends on NO_HZ && RCU_USER_QS && VIRT_CPU_ACCOUNTING_GEN && RCU_NOCB_CPU && SMP
+       select CONTEXT_TRACKING_FORCE
+       help
+         Adaptively try to shutdown the tick whenever possible, even when
+	 the CPU is running tasks. Typically this requires running a single
+	 task on the CPU. Chances for running tickless are maximized when
+	 the task mostly runs in userspace and has few kernel activity.
+
+	 You need to fill up the nohz_extended boot parameter with the
+	 desired range of dynticks CPUs.
+
+	 This is implemented at the expense of some overhead in user <-> kernel
+	 transitions: syscalls, exceptions and interrupts. Even when it's
+	 dynamically off.
+
+	 Say N.
+
 config HIGH_RES_TIMERS
 	bool "High Resolution Timer Support"
 	depends on !ARCH_USES_GETTIMEOFFSET && GENERIC_CLOCKEVENTS
kernel/time/tick-sched.c +62 -0

68 lines not on this page · raw message

Signature · 1 line
1.7.5.4
#2 lore raw

PATCH 2/3 nohz: Assign timekeeping duty to a CPU outside the full dynticks range

In reply to Frederic Weisbecker

This way the full nohz CPUs can safely run with the tick stopped with a guarantee that somebody else is taking care of the jiffies and GTOD progression.

Once the duty is attributed to a CPU, it won't change. Also that CPU can't enter into dyntick idle mode or be hot unplugged.

This may later be improved from a power consumption POV. At least we should be able to share the duty amongst all CPUs outside the full dynticks range. Then the duty could even be shared with full dynticks CPUs when those can't stop their tick for any reason.

But let's start with that very simple approach first.

Cc Andrew Morton <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Geoff Levand <[email protected]>
Cc Gilad Ben Yossef <[email protected]>
Cc Hakan Akkan <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Li Zhong <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Paul Gortmaker <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
[fix have_nohz_full_mask offcase]
---
 kernel/time/tick-broadcast.c |    3 +-
 kernel/time/tick-common.c    |    5 +++-
 kernel/time/tick-sched.c     |   47 ++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 51 insertions(+), 4 deletions(-)
kernel/time/tick-broadcast.c +2 -1
index 2fb8cb8..8a6875c 100644
@@ -573,7 +573,8 @@ void tick_broadcast_setup_oneshot(struct clock_event_device *bc)
 		bc->event_handler = tick_handle_oneshot_broadcast;
 
 		/* Take the do_timer update */
-		tick_do_timer_cpu = cpu;
+		if (!tick_nohz_extended_cpu(cpu))
+			tick_do_timer_cpu = cpu;
 
 		/*
 		 * We must be careful here. There might be other CPUs
kernel/time/tick-common.c +4 -1
index b1600a6..b7dc0cb 100644
@@ -163,7 +163,10 @@ static void tick_setup_device(struct tick_device *td,
 		 * this cpu:
 		 */
 		if (tick_do_timer_cpu == TICK_DO_TIMER_BOOT) {
-			tick_do_timer_cpu = cpu;
+			if (!tick_nohz_extended_cpu(cpu))
+				tick_do_timer_cpu = cpu;
+			else
+				tick_do_timer_cpu = TICK_DO_TIMER_NONE;
 			tick_next_period = ktime_get();
 			tick_period = ktime_set(0, NSEC_PER_SEC / HZ);
 		}
kernel/time/tick-sched.c +45 -2

81 lines not on this page · raw message

Signature · 1 line
1.7.5.4
#3 lore raw

PATCH 3/3 nohz: Wake up full dynticks CPUs when a timer gets enqueued

In reply to Frederic Weisbecker

Wake up a CPU when a timer list timer is enqueued there and the target is part of the full dynticks range. Sending an IPI to it makes it reconsidering the next timer to program on top of recent updates.

This may later be improved by checking if the tick is really stopped on the target. This would need some careful synchronization though. So deal with such optimization later and start simple.

Cc Andrew Morton <[email protected]>
Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Geoff Levand <[email protected]>
Cc Gilad Ben Yossef <[email protected]>
Cc Hakan Akkan <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Kevin Hilman <[email protected]>
Cc Li Zhong <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Paul Gortmaker <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Thomas Gleixner <[email protected]>
---
 include/linux/sched.h |    4 ++--
 kernel/sched/core.c   |   20 +++++++++++++++++++-
 kernel/timer.c        |   12 ++++++------
 3 files changed, 27 insertions(+), 9 deletions(-)
include/linux/sched.h +2 -2
index 9004f6e..10626e2e 100644
@@ -1851,9 +1851,9 @@ static inline void idle_task_exit(void) {}
 #endif
 
 #if defined(CONFIG_NO_HZ) && defined(CONFIG_SMP)
-extern void wake_up_idle_cpu(int cpu);
+extern void wake_up_nohz_cpu(int cpu);
 #else
-static inline void wake_up_idle_cpu(int cpu) { }
+static inline void wake_up_nohz_cpu(int cpu) { }
 #endif
 
 #ifdef CONFIG_SCHED_AUTOGROUP
kernel/sched/core.c +19 -1
index 849deb9..e91ee58 100644
@@ -587,7 +587,7 @@ unlock:
  * account when the CPU goes back to idle and evaluates the timer
  * wheel for the next timer event.
  */
-void wake_up_idle_cpu(int cpu)
+static void wake_up_idle_cpu(int cpu)
 {
 	struct rq *rq = cpu_rq(cpu);
 
@@ -617,6 +617,24 @@ void wake_up_idle_cpu(int cpu)
 		smp_send_reschedule(cpu);
 }
 
+static bool wake_up_extended_nohz_cpu(int cpu)
+{
+	if (tick_nohz_extended_cpu(cpu)) {
+		if (cpu != smp_processor_id() ||
+		    tick_nohz_tick_stopped())
+			smp_send_reschedule(cpu);
+		return true;
+	}
+
+	return false;
+}
+
+void wake_up_nohz_cpu(int cpu)
+{
+	if (!wake_up_extended_nohz_cpu(cpu))
+		wake_up_idle_cpu(cpu);
+}
+
 static inline bool got_nohz_idle_kick(void)
 {
 	int cpu = smp_processor_id();
kernel/timer.c +6 -6
index dbf7a78..4e3040b 100644
@@ -930,14 +930,14 @@ void add_timer_on(struct timer_list *timer, int cpu)
 	debug_activate(timer, timer->expires);
 	internal_add_timer(base, timer);
 	/*
-	 * Check whether the other CPU is idle and needs to be
-	 * triggered to reevaluate the timer wheel when nohz is
-	 * active. We are protected against the other CPU fiddling
+	 * Check whether the other CPU is in dynticks mode and needs
+	 * to be triggered to reevaluate the timer wheel.
+	 * We are protected against the other CPU fiddling
 	 * with the timer by holding the timer base lock. This also
-	 * makes sure that a CPU on the way to idle can not evaluate
-	 * the timer wheel.
+	 * makes sure that a CPU on the way to stop its tick can not
+	 * evaluate the timer wheel.
 	 */
-	wake_up_idle_cpu(cpu);
+	wake_up_nohz_cpu(cpu);
 	spin_unlock_irqrestore(&base->lock, flags);
 }
 EXPORT_SYMBOL_GPL(add_timer_on);
Signature · 1 line
1.7.5.4
maintainer
#4 lore raw
In reply to Frederic Weisbecker
Quoted 14 lines · Frederic Weisbecker

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Quoted 7 lines

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=

Note that boot parameters suck for pretty much any purpose but quirks - please also add a (default off!) Kconfig option to easily enable nohz_extended for all CPUs.

That way I will be able to test it automatically via randconfig and such.

My next question/request after that would be: could we make sure that enabling this option does not break any applications or kernel functionality, ASAP? Once that is offered, it becomes pushable to v3.10 I think.

Meanwhile I have pulled your tree into tip:sched/nohz to stage and test it, in the hope that it becomes mergable quickly! :-)

Thanks,

Ingo

#5 lore raw

2013/3/24 Ingo Molnar <[email protected]>:

Quoted 25 lines · Frederic Weisbecker
Quoted 15 lines

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Quoted 7 lines

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=

Quoted 5 lines

Note that boot parameters suck for pretty much any purpose but quirks - please also add a (default off!) Kconfig option to easily enable nohz_extended for all CPUs.

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Quoted 5 lines

My next question/request after that would be: could we make sure that enabling this option does not break any applications or kernel functionality, ASAP? Once that is offered, it becomes pushable to v3.10 I think.

So that's already the case. I'm careful that, as we iterate, the changes aren't supposed to break something in the middle. the final code that eventually shuts the tick down should be the last one in the
series: once we know it is safe to do so.

Quoted 3 lines

Meanwhile I have pulled your tree into tip:sched/nohz to stage and test it, in the hope that it becomes mergable quickly! :-)

Thanks! :)

#6 lore raw
Quoted 26 lines · Frederic Weisbecker

2013/3/24 Ingo Molnar <[email protected]>:

Quoted 25 lines · Frederic Weisbecker
Quoted 22 lines

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=

Quoted 7 lines
Quoted 5 lines

Note that boot parameters suck for pretty much any purpose but quirks - please also add a (default off!) Kconfig option to easily enable nohz_extended for all CPUs.

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Hmmm... This would be an option to make all but one CPU an adaptive-ticks CPU, right? If so, this leads to the question of whether I should add a matching no-CBs Kconfig option. My guess is "no", because the existing CONFIG_RCU_NOCB_CPU_ALL should work just fine -- there would be a CPU that was not an adaptive-ticks CPU, but does have its RCU callbacks offloaded.

Or am I missing something here?

Thanx, Paul

Quoted 16 lines
Quoted 4 lines

My next question/request after that would be: could we make sure that enabling this option does not break any applications or kernel functionality, ASAP? Once that is offered, it becomes pushable to v3.10 I think.

So that's already the case. I'm careful that, as we iterate, the changes aren't supposed to break something in the middle. the final code that eventually shuts the tick down should be the last one in the
series: once we know it is safe to do so.

Quoted 3 lines

Meanwhile I have pulled your tree into tip:sched/nohz to stage and test it, in the hope that it becomes mergable quickly! :-)

Thanks! :)

#7 lore raw

2013/3/25 Paul E. McKenney <[email protected]>:

Quoted 35 lines · Frederic Weisbecker
Quoted 26 lines

2013/3/24 Ingo Molnar <[email protected]>:

Quoted 25 lines · Frederic Weisbecker
Quoted 22 lines

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=

Quoted 7 lines
Quoted 5 lines

Note that boot parameters suck for pretty much any purpose but quirks - please also add a (default off!) Kconfig option to easily enable nohz_extended for all CPUs.

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Quoted 7 lines

Hmmm... This would be an option to make all but one CPU an adaptive-ticks CPU, right? If so, this leads to the question of whether I should add a matching no-CBs Kconfig option. My guess is "no", because the existing CONFIG_RCU_NOCB_CPU_ALL should work just fine -- there would be a CPU that was not an adaptive-ticks CPU, but does have its RCU callbacks offloaded.

Or am I missing something here?

No that looks right. Now I wonder if I should select CONFIG_RCU_NOCB_CPU_ALL at the same time. Probably.

#8 lore raw
Quoted 42 lines · Frederic Weisbecker

2013/3/25 Paul E. McKenney <[email protected]>:

Quoted 41 lines · Frederic Weisbecker
Quoted 26 lines

2013/3/24 Ingo Molnar <[email protected]>:

Quoted 25 lines · Frederic Weisbecker
Quoted 22 lines

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=

Quoted 7 lines
Quoted 5 lines

Note that boot parameters suck for pretty much any purpose but quirks - please also add a (default off!) Kconfig option to easily enable nohz_extended for all CPUs.

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Hmmm... This would be an option to make all but one CPU an adaptive-ticks CPU, right? If so, this leads to the question of whether I should add a matching no-CBs Kconfig option. My guess is "no", because the existing CONFIG_RCU_NOCB_CPU_ALL should work just fine -- there would be a CPU that was not an adaptive-ticks CPU, but does have its RCU callbacks offloaded.

Quoted 4 lines
Quoted 1 line

Or am I missing something here?

No that looks right. Now I wonder if I should select CONFIG_RCU_NOCB_CPU_ALL at the same time. Probably.

Sounds like a good initial position to me. If it somehow causes problems, we can always change it later.

Thanx, Paul

#9 lore raw

2013/3/25 Paul E. McKenney <[email protected]>:

Quoted 43 lines · Frederic Weisbecker
Quoted 42 lines

2013/3/25 Paul E. McKenney <[email protected]>:

Quoted 41 lines · Frederic Weisbecker
Quoted 26 lines

2013/3/24 Ingo Molnar <[email protected]>:

Quoted 25 lines · Frederic Weisbecker
Quoted 22 lines

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=

Quoted 7 lines
Quoted 5 lines

Note that boot parameters suck for pretty much any purpose but quirks - please also add a (default off!) Kconfig option to easily enable nohz_extended for all CPUs.

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Hmmm... This would be an option to make all but one CPU an adaptive-ticks CPU, right? If so, this leads to the question of whether I should add a matching no-CBs Kconfig option. My guess is "no", because the existing CONFIG_RCU_NOCB_CPU_ALL should work just fine -- there would be a CPU that was not an adaptive-ticks CPU, but does have its RCU callbacks offloaded.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Or am I missing something here?

No that looks right. Now I wonder if I should select CONFIG_RCU_NOCB_CPU_ALL at the same time. Probably.

Sounds like a good initial position to me. If it somehow causes problems, we can always change it later.

Ah "rcu: Provide compile-time control for no-CBs CPUs" is not yet in -tip so I can't do that yet. Ok for now I'm going to add CONFIG_NO_HZ_EXTENDED_ALL and will select the matching RCU config once it's visible upstream.

#10 lore raw
Quoted 49 lines · Frederic Weisbecker

2013/3/25 Paul E. McKenney <[email protected]>:

Quoted 48 lines · Frederic Weisbecker
Quoted 46 lines

2013/3/25 Paul E. McKenney <[email protected]>:

Quoted 35 lines · Frederic Weisbecker
Quoted 26 lines

2013/3/24 Ingo Molnar <[email protected]>:

Quoted 25 lines · Frederic Weisbecker
Quoted 22 lines

Hi Ingo,

This settles the initial ground to start a special full dynticks tree in -tip that we can iterate incrementally to accelerate the development. It is based on tip:sched/core.

I tried to rearrange a bit the naming. We are probably not yet done with that but I guess we can fix it along with the rest.

Please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
full-dynticks-for-mingo

Changes on these commits since they were part of 3.9-rc1-nohz1:

* Force a timekeeping CPU over the full dynticks range
* Rename CONFIG_NO_HZ_FULL to CONFIG_NO_HZ_EXTENDED
* Following *_nohz_extended_* APIs renames
* Handle CPU hotplug for timekeeping
* Rename full_nohz= kernel parameter to nohz_extended=

Quoted 7 lines
Quoted 5 lines

Note that boot parameters suck for pretty much any purpose but quirks - please also add a (default off!) Kconfig option to easily enable nohz_extended for all CPUs.

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Quoted 7 lines

Hmmm... This would be an option to make all but one CPU an adaptive-ticks CPU, right? If so, this leads to the question of whether I should add a matching no-CBs Kconfig option. My guess is "no", because the existing CONFIG_RCU_NOCB_CPU_ALL should work just fine -- there would be a CPU that was not an adaptive-ticks CPU, but does have its RCU callbacks offloaded.

Or am I missing something here?

No that looks right. Now I wonder if I should select CONFIG_RCU_NOCB_CPU_ALL at the same time. Probably.

Quoted 7 lines
Quoted 2 lines

Sounds like a good initial position to me. If it somehow causes problems, we can always change it later.

Ah "rcu: Provide compile-time control for no-CBs CPUs" is not yet in -tip so I can't do that yet. Ok for now I'm going to add CONFIG_NO_HZ_EXTENDED_ALL and will select the matching RCU config once it's visible upstream.

Good point... I expect to be sending a pull request in a day or two.

Thanx, Paul

maintainer
#11 lore raw
In reply to Frederic Weisbecker
Quoted 10 lines · Frederic Weisbecker
Quoted 2 lines

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Quoted 4 lines

My next question/request after that would be: could we make sure that enabling this option does not break any applications or kernel functionality, ASAP? Once that is offered, it becomes pushable to v3.10 I think.

Quoted 4 lines

So that's already the case. I'm careful that, as we iterate, the changes aren't supposed to break something in the middle. the final code that eventually shuts the tick down should be the last one in the series:
once we know it is safe to do so.

I mean, with the .config (or boot) option enabled, with tip:timers/nohz as-is, will everything work? We cannot push a feature to Linus that doesn't work reliably when enabled ...

Thanks,

Ingo

#12 lore raw

2013/3/26 Ingo Molnar <[email protected]>:

Quoted 13 lines · Frederic Weisbecker
Quoted 11 lines
Quoted 3 lines

That way I will be able to test it automatically via randconfig and such.

Sure, I'm adding such an option.

Quoted 4 lines

My next question/request after that would be: could we make sure that enabling this option does not break any applications or kernel functionality, ASAP? Once that is offered, it becomes pushable to v3.10 I think.

Quoted 8 lines
Quoted 4 lines

So that's already the case. I'm careful that, as we iterate, the changes aren't supposed to break something in the middle. the final code that eventually shuts the tick down should be the last one in the series:
once we know it is safe to do so.

I mean, with the .config (or boot) option enabled, with tip:timers/nohz as-is, will everything work? We cannot push a feature to Linus that doesn't work reliably when enabled ...

Right. So indeed the feature is not implemented yet in tip:timers/nohz, ie: the tick won't be stopped. But the current halfway state is not supposed to break existing things.

But I agree.