Skip to contentSkip to conversation

isolation: 1Hz residual tick offloading v6

PATCH In review Frederic Weisbecker 3 participants 16 messages Feb 15, 2018 – Feb 20, 2018

Subsystems (4)
lore
Share & tools
lore

Frederic Weisbecker

author maintainer
#0 lore raw

cover isolation: 1Hz residual tick offloading v6

The version addresses the reviews from Ingo on v5:

* Start changelog titles with verbs.
* Improve/reword several comments.
* Improve/reword documentation.
* Various cosmetic changes.
* Use BUG_ON() on boot allocation failure.

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git

	sched/0hz-v6

HEAD: 7011ba38353ad8ec398ade7533c73828d27462ad

Thanks,
Frederic
---

Frederic Weisbecker (6):
sched: Rename init_rq_hrtick to hrtick_rq_init
nohz: Allow to check if remote CPU tick is stopped sched/isolation: Isolate workqueues when "nohz_full=" is set sched/isolation: Offload residual 1Hz scheduler tick sched/nohz: Remove the 1 Hz tick code
sched/isolation: Update nohz documentation to explain tick offload

 Documentation/admin-guide/kernel-parameters.txt |  11 +++
 include/linux/sched/isolation.h                 |   1 +
 include/linux/sched/nohz.h                      |   4 -
 include/linux/tick.h                            |   2 +
 kernel/sched/core.c                             | 117 ++++++++++++++++++------
 kernel/sched/idle_task.c                        |   1 -
 kernel/sched/isolation.c                        |   7 +-
 kernel/sched/sched.h                            |  13 +--
 kernel/time/tick-sched.c                        |  13 +--
 kernel/workqueue.c                              |   3 +-
 10 files changed, 122 insertions(+), 50 deletions(-)
author maintainer
#1 lore raw

PATCH 1/6 sched: Rename init_rq_hrtick to hrtick_rq_init

Do that rename in order to normalize the hrtick namespace.

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 kernel/sched/core.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/sched/core.c +3 -3
index bf724c1..f09ec5c 100644
@@ -333,7 +333,7 @@ void hrtick_start(struct rq *rq, u64 delay)
 }
 #endif /* CONFIG_SMP */
 
-static void init_rq_hrtick(struct rq *rq)
+static void hrtick_rq_init(struct rq *rq)
 {
 #ifdef CONFIG_SMP
 	rq->hrtick_csd_pending = 0;
@@ -351,7 +351,7 @@ static inline void hrtick_clear(struct rq *rq)
 {
 }
 
-static inline void init_rq_hrtick(struct rq *rq)
+static inline void hrtick_rq_init(struct rq *rq)
 {
 }
 #endif	/* CONFIG_SCHED_HRTICK */
@@ -6023,7 +6023,7 @@ void __init sched_init(void)
 		rq->last_sched_tick = 0;
 #endif
 #endif /* CONFIG_SMP */
-		init_rq_hrtick(rq);
+		hrtick_rq_init(rq);
 		atomic_set(&rq->nr_iowait, 0);
 	}
 
Signature · 1 line
2.7.4
author maintainer
#3 lore raw

PATCH 2/6 nohz: Allow to check if remote CPU tick is stopped

In reply to Frederic Weisbecker

This check is racy but provides a good heuristic to determine whether a CPU may need a remote tick or not.

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 include/linux/tick.h     | 2 ++
 kernel/time/tick-sched.c | 7 +++++++
 2 files changed, 9 insertions(+)
include/linux/tick.h +2 -0
index 7cc3592..944c829 100644
@@ -114,6 +114,7 @@ enum tick_dep_bits {
 #ifdef CONFIG_NO_HZ_COMMON
 extern bool tick_nohz_enabled;
 extern int tick_nohz_tick_stopped(void);
+extern int tick_nohz_tick_stopped_cpu(int cpu);
 extern void tick_nohz_idle_enter(void);
 extern void tick_nohz_idle_exit(void);
 extern void tick_nohz_irq_exit(void);
@@ -125,6 +126,7 @@ extern u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_time);
 #else /* !CONFIG_NO_HZ_COMMON */
 #define tick_nohz_enabled (0)
 static inline int tick_nohz_tick_stopped(void) { return 0; }
+static inline int tick_nohz_tick_stopped_cpu(int cpu) { return 0; }
 static inline void tick_nohz_idle_enter(void) { }
 static inline void tick_nohz_idle_exit(void) { }
 
kernel/time/tick-sched.c +7 -0
index 29a5733..b517485 100644
@@ -486,6 +486,13 @@ int tick_nohz_tick_stopped(void)
 	return __this_cpu_read(tick_cpu_sched.tick_stopped);
 }
 
+int tick_nohz_tick_stopped_cpu(int cpu)
+{
+	struct tick_sched *ts = per_cpu_ptr(&tick_cpu_sched, cpu);
+
+	return ts->tick_stopped;
+}
+
 /**
  * tick_nohz_update_jiffies - update jiffies when idle was interrupted
  *
Signature · 1 line
2.7.4
author maintainer
#7 lore raw

PATCH 4/6 sched/isolation: Offload residual 1Hz scheduler tick

In reply to Frederic Weisbecker

When a CPU runs in full dynticks mode, a 1Hz tick remains in order to keep the scheduler stats alive. However this residual tick is a burden for bare metal tasks that can't stand any interruption at all, or want to minimize them.

The usual boot parameters "nohz_full=" or "isolcpus=nohz" will now outsource these scheduler ticks to the global workqueue so that a housekeeping CPU handles those remotely. The sched_class::task_tick() implementations have been audited and look safe to be called remotely as the target runqueue and its current task are passed in parameter and don't seem to be accessed locally.

Note that in the case of using isolcpus, it's still up to the user to affine the global workqueues to the housekeeping CPUs through /sys/devices/virtual/workqueue/cpumask or domains isolation "isolcpus=nohz,domain".

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 kernel/sched/core.c      | 92 ++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/isolation.c |  4 +++
 kernel/sched/sched.h     |  2 ++
 3 files changed, 98 insertions(+)
kernel/sched/core.c +92 -0
index f09ec5c..86eefc4 100644
@@ -3120,6 +3120,96 @@ u64 scheduler_tick_max_deferment(void)
 
 	return jiffies_to_nsecs(next - now);
 }
+
+struct tick_work {
+	int			cpu;
+	struct delayed_work	work;
+};
+
+static struct tick_work __percpu *tick_work_cpu;
+
+static void sched_tick_remote(struct work_struct *work)
+{
+	struct delayed_work *dwork = to_delayed_work(work);
+	struct tick_work *twork = container_of(dwork, struct tick_work, work);
+	int cpu = twork->cpu;
+	struct rq *rq = cpu_rq(cpu);
+	struct rq_flags rf;
+
+	/*
+	 * Handle the tick only if it appears the remote CPU is running in full
+	 * dynticks mode. The check is racy by nature, but missing a tick or
+	 * having one too much is no big deal because the scheduler tick updates
+	 * statistics and checks timeslices in a time-independent way, regardless
+	 * of when exactly it is running.
+	 */
+	if (!idle_cpu(cpu) && tick_nohz_tick_stopped_cpu(cpu)) {
+		struct task_struct *curr;
+		u64 delta;
+
+		rq_lock_irq(rq, &rf);
+		update_rq_clock(rq);
+		curr = rq->curr;
+		delta = rq_clock_task(rq) - curr->se.exec_start;
+
+		/*
+		 * Make sure the next tick runs within a reasonable
+		 * amount of time.
+		 */
+		WARN_ON_ONCE(delta > (u64)NSEC_PER_SEC * 3);
+		curr->sched_class->task_tick(rq, curr, 0);
+		rq_unlock_irq(rq, &rf);
+	}
+
+	/*
+	 * Run the remote tick once per second (1Hz). This arbitrary
+	 * frequency is large enough to avoid overload but short enough
+	 * to keep scheduler internal stats reasonably up to date.
+	 */
+	queue_delayed_work(system_unbound_wq, dwork, HZ);
+}
+
+static void sched_tick_start(int cpu)
+{
+	struct tick_work *twork;
+
+	if (housekeeping_cpu(cpu, HK_FLAG_TICK))
+		return;
+
+	WARN_ON_ONCE(!tick_work_cpu);
+
+	twork = per_cpu_ptr(tick_work_cpu, cpu);
+	twork->cpu = cpu;
+	INIT_DELAYED_WORK(&twork->work, sched_tick_remote);
+	queue_delayed_work(system_unbound_wq, &twork->work, HZ);
+}
+
+#ifdef CONFIG_HOTPLUG_CPU
+static void sched_tick_stop(int cpu)
+{
+	struct tick_work *twork;
+
+	if (housekeeping_cpu(cpu, HK_FLAG_TICK))
+		return;
+
+	WARN_ON_ONCE(!tick_work_cpu);
+
+	twork = per_cpu_ptr(tick_work_cpu, cpu);
+	cancel_delayed_work_sync(&twork->work);
+}
+#endif /* CONFIG_HOTPLUG_CPU */
+
+int __init sched_tick_offload_init(void)
+{
+	tick_work_cpu = alloc_percpu(struct tick_work);
+	BUG_ON(!tick_work_cpu);
+
+	return 0;
+}
+
+#else /* !CONFIG_NO_HZ_FULL */
+static inline void sched_tick_start(int cpu) { }
+static inline void sched_tick_stop(int cpu) { }
 #endif
 
 #if defined(CONFIG_PREEMPT) && (defined(CONFIG_DEBUG_PREEMPT) || \
@@ -5781,6 +5871,7 @@ int sched_cpu_starting(unsigned int cpu)
 {
 	set_cpu_rq_start_time(cpu);
 	sched_rq_cpu_starting(cpu);
+	sched_tick_start(cpu);
 	return 0;
 }
 
@@ -5792,6 +5883,7 @@ int sched_cpu_dying(unsigned int cpu)
 
 	/* Handle pending wakeups and then migrate everything off */
 	sched_ttwu_pending();
+	sched_tick_stop(cpu);
 
 	rq_lock_irqsave(rq, &rf);
 	if (rq->rd) {
kernel/sched/isolation.c +4 -0
index a2500c4..39f340d 100644
@@ -13,6 +13,7 @@
 #include <linux/kernel.h>
 #include <linux/static_key.h>
 #include <linux/ctype.h>
+#include "sched.h"
 
 DEFINE_STATIC_KEY_FALSE(housekeeping_overriden);
 EXPORT_SYMBOL_GPL(housekeeping_overriden);
@@ -61,6 +62,9 @@ void __init housekeeping_init(void)
 
 	static_branch_enable(&housekeeping_overriden);
 
+	if (housekeeping_flags & HK_FLAG_TICK)
+		sched_tick_offload_init();
+
 	/* We need at least one CPU to handle housekeeping work */
 	WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
 }
kernel/sched/sched.h +2 -0
index fb5fc458..c1c7c78 100644
@@ -1574,6 +1574,7 @@ extern void post_init_entity_util_avg(struct sched_entity *se);
 
 #ifdef CONFIG_NO_HZ_FULL
 extern bool sched_can_stop_tick(struct rq *rq);
+extern int __init sched_tick_offload_init(void);
 
 /*
  * Tick may be needed by tasks in the runqueue depending on their policy and
@@ -1598,6 +1599,7 @@ static inline void sched_update_tick_dependency(struct rq *rq)
 		tick_nohz_dep_set_cpu(cpu, TICK_DEP_BIT_SCHED);
 }
 #else
+static inline int sched_tick_offload_init(void) { return 0; }
 static inline void sched_update_tick_dependency(struct rq *rq) { }
 #endif
 
Signature · 1 line
2.7.4
author maintainer
#12 lore raw

PATCH 6/6 sched/isolation: Update nohz documentation to explain tick offload

In reply to Frederic Weisbecker

Update the documentation to reflect the 1Hz tick offload changes.

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 Documentation/admin-guide/kernel-parameters.txt | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +11 -0
index 1d1d53f..50b9837 100644
@@ -1766,6 +1766,17 @@
 
 			nohz
 			  Disable the tick when a single task runs.
+
+			  A residual 1Hz tick is offloaded to workqueues, which you
+			  need to affine to housekeeping through the global
+			  workqueue's affinity configured via the
+			  /sys/devices/virtual/workqueue/cpumask sysfs file, or
+			  by using the 'domain' flag described below.
+
+			  NOTE: by default the global workqueue runs on all CPUs,
+			  so to protect individual CPUs the 'cpumask' file has to
+			  be configured manually after bootup.
+
 			domain
 			  Isolate from the general SMP balancing and scheduling
 			  algorithms. Note that performing domain isolation this way
Signature · 1 line
2.7.4
author maintainer
#10 lore raw

PATCH 5/6 sched/nohz: Remove the 1 Hz tick code

In reply to Frederic Weisbecker

Now that the 1Hz tick is offloaded to workqueues, we can safely remove the residual code that used to handle it locally.

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 include/linux/sched/nohz.h |  4 ----
 kernel/sched/core.c        | 29 -----------------------------
 kernel/sched/idle_task.c   |  1 -
 kernel/sched/sched.h       | 11 +----------
 kernel/time/tick-sched.c   |  6 ------
 5 files changed, 1 insertion(+), 50 deletions(-)
include/linux/sched/nohz.h +0 -4
index 3d3a97d..0942172 100644
@@ -37,8 +37,4 @@ extern void wake_up_nohz_cpu(int cpu);
 static inline void wake_up_nohz_cpu(int cpu) { }
 #endif
 
-#ifdef CONFIG_NO_HZ_FULL
-extern u64 scheduler_tick_max_deferment(void);
-#endif
-
 #endif /* _LINUX_SCHED_NOHZ_H */
kernel/sched/core.c +0 -29
index 86eefc4..099c9cb 100644
@@ -3091,35 +3091,9 @@ void scheduler_tick(void)
 	rq->idle_balance = idle_cpu(cpu);
 	trigger_load_balance(rq);
 #endif
-	rq_last_tick_reset(rq);
 }
 
 #ifdef CONFIG_NO_HZ_FULL
-/**
- * scheduler_tick_max_deferment
- *
- * Keep at least one tick per second when a single
- * active task is running because the scheduler doesn't
- * yet completely support full dynticks environment.
- *
- * This makes sure that uptime, CFS vruntime, load
- * balancing, etc... continue to move forward, even
- * with a very low granularity.
- *
- * Return: Maximum deferment in nanoseconds.
- */
-u64 scheduler_tick_max_deferment(void)
-{
-	struct rq *rq = this_rq();
-	unsigned long next, now = READ_ONCE(jiffies);
-
-	next = rq->last_sched_tick + HZ;
-
-	if (time_before_eq(next, now))
-		return 0;
-
-	return jiffies_to_nsecs(next - now);
-}
 
 struct tick_work {
 	int			cpu;
@@ -6111,9 +6085,6 @@ void __init sched_init(void)
 		rq->last_load_update_tick = jiffies;
 		rq->nohz_flags = 0;
 #endif
-#ifdef CONFIG_NO_HZ_FULL
-		rq->last_sched_tick = 0;
-#endif
 #endif /* CONFIG_SMP */
 		hrtick_rq_init(rq);
 		atomic_set(&rq->nr_iowait, 0);
kernel/sched/idle_task.c +0 -1
index d518664..a64fc92 100644
@@ -48,7 +48,6 @@ dequeue_task_idle(struct rq *rq, struct task_struct *p, int flags)
 
 static void put_prev_task_idle(struct rq *rq, struct task_struct *prev)
 {
-	rq_last_tick_reset(rq);
 }
 
 static void task_tick_idle(struct rq *rq, struct task_struct *curr, int queued)
kernel/sched/sched.h +1 -10
index c1c7c78..dc6c8b5 100644
@@ -727,9 +727,7 @@ struct rq {
 #endif /* CONFIG_SMP */
 	unsigned long nohz_flags;
 #endif /* CONFIG_NO_HZ_COMMON */
-#ifdef CONFIG_NO_HZ_FULL
-	unsigned long last_sched_tick;
-#endif
+
 	/* capture load from *all* tasks on this cpu: */
 	struct load_weight load;
 	unsigned long nr_load_updates;
@@ -1626,13 +1624,6 @@ static inline void sub_nr_running(struct rq *rq, unsigned count)
 	sched_update_tick_dependency(rq);
 }
 
-static inline void rq_last_tick_reset(struct rq *rq)
-{
-#ifdef CONFIG_NO_HZ_FULL
-	rq->last_sched_tick = jiffies;
-#endif
-}
-
 extern void update_rq_clock(struct rq *rq);
 
 extern void activate_task(struct rq *rq, struct task_struct *p, int flags);
kernel/time/tick-sched.c +0 -6
index b517485..0c615a0 100644
@@ -748,12 +748,6 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
 		delta = KTIME_MAX;
 	}
 
-#ifdef CONFIG_NO_HZ_FULL
-	/* Limit the tick delta to the maximum scheduler deferment */
-	if (!ts->inidle)
-		delta = min(delta, scheduler_tick_max_deferment());
-#endif
-
 	/* Calculate the next expiry time */
 	if (delta < (KTIME_MAX - basemono))
 		expires = basemono + delta;
Signature · 1 line
2.7.4
author maintainer
#5 lore raw

PATCH 3/6 sched/isolation: Isolate workqueues when "nohz_full=" is set

In reply to Frederic Weisbecker

As we prepare for offloading the residual 1hz scheduler ticks to workqueue, let's affine those to housekeepers so that they don't interrupt the CPUs that don't want to be disturbed.

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 include/linux/sched/isolation.h | 1 +
 kernel/sched/isolation.c        | 3 ++-
 kernel/workqueue.c              | 3 ++-
 3 files changed, 5 insertions(+), 2 deletions(-)
include/linux/sched/isolation.h +1 -0
index d849431..4a6582c 100644
@@ -12,6 +12,7 @@ enum hk_flags {
 	HK_FLAG_SCHED		= (1 << 3),
 	HK_FLAG_TICK		= (1 << 4),
 	HK_FLAG_DOMAIN		= (1 << 5),
+	HK_FLAG_WQ		= (1 << 6),
 };
 
 #ifdef CONFIG_CPU_ISOLATION
kernel/sched/isolation.c +2 -1
index b71b436..a2500c4 100644
@@ -3,6 +3,7 @@
  *  any CPU: unbound workqueues, timers, kthreads and any offloadable work.
  *
  * Copyright (C) 2017 Red Hat, Inc., Frederic Weisbecker
+ * Copyright (C) 2017-2018 SUSE, Frederic Weisbecker
  *
  */
 
@@ -119,7 +120,7 @@ static int __init housekeeping_nohz_full_setup(char *str)
 {
 	unsigned int flags;
 
-	flags = HK_FLAG_TICK | HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
+	flags = HK_FLAG_TICK | HK_FLAG_WQ | HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
 
 	return housekeeping_setup(str, flags);
 }
kernel/workqueue.c +2 -1
index 017044c..593dbe7 100644
@@ -5565,12 +5565,13 @@ static void __init wq_numa_init(void)
 int __init workqueue_init_early(void)
 {
 	int std_nice[NR_STD_WORKER_POOLS] = { 0, HIGHPRI_NICE_LEVEL };
+	int hk_flags = HK_FLAG_DOMAIN | HK_FLAG_WQ;
 	int i, cpu;
 
 	WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long));
 
 	BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL));
-	cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(HK_FLAG_DOMAIN));
+	cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(hk_flags));
 
 	pwq_cache = KMEM_CACHE(pool_workqueue, SLAB_PANIC);
 
Signature · 1 line
2.7.4
maintainer
#15 lore raw
In reply to Frederic Weisbecker
Quoted 8 lines · Frederic Weisbecker

The version addresses the reviews from Ingo on v5:

* Start changelog titles with verbs.
* Improve/reword several comments.
* Improve/reword documentation.
* Various cosmetic changes.
* Use BUG_ON() on boot allocation failure.

Thanks!

Reviewed-by on v1 2/6
#4 lore raw

Re: [PATCH 2/6] nohz: Allow to check if remote CPU tick is stopped

In reply to Frederic Weisbecker
Quoted 5 lines · Frederic Weisbecker

This check is racy but provides a good heuristic to determine whether a CPU may need a remote tick or not.

+int tick_nohz_tick_stopped_cpu(int cpu)

s/int/bool/ but I'm not religious about that.

kernel/time/tick-sched.c int tick_nohz_tick_stopped(void) · line 494

Quoted 5 lines
kernel/time/tick-sched.c +5 -0
+{
+	struct tick_sched *ts = per_cpu_ptr(&tick_cpu_sched, cpu);
+
+	return ts->tick_stopped;
+}

Other than that nit:

Reviewed-by on v1 3/6
#6 lore raw

Re: [PATCH 3/6] sched/isolation: Isolate workqueues when "nohz_full=" is set

In reply to Frederic Weisbecker
Quoted 5 lines · Frederic Weisbecker

As we prepare for offloading the residual 1hz scheduler ticks to workqueue, let's affine those to housekeepers so that they don't interrupt the CPUs that don't want to be disturbed.

Reviewed-by on v1 4/6
#8 lore raw

Re: [PATCH 4/6] sched/isolation: Offload residual 1Hz scheduler tick

In reply to Frederic Weisbecker
Quoted 11 lines · Frederic Weisbecker

When a CPU runs in full dynticks mode, a 1Hz tick remains in order to keep the scheduler stats alive. However this residual tick is a burden for bare metal tasks that can't stand any interruption at all, or want to minimize them.

The usual boot parameters "nohz_full=" or "isolcpus=nohz" will now outsource these scheduler ticks to the global workqueue so that a housekeeping CPU handles those remotely. The sched_class::task_tick() implementations have been audited and look safe to be called remotely as the target runqueue and its current task are passed in parameter and don't seem to be accessed locally.

That scares me a bit. Not for the current state of affairs, but we want to ensure that this still works in 2 years from now

So at least you want to add a comment to task_tick() which explains the constraints which come with the remote tick.

Other than that this looks good!

#13 lore raw

Re: [PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

In reply to Frederic Weisbecker
Quoted 25 lines · Frederic Weisbecker

Update the documentation to reflect the 1Hz tick offload changes.

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 Documentation/admin-guide/kernel-parameters.txt | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +0 -0
index 1d1d53f..50b9837 100644
@@ -1766,6 +1766,17 @@
 
 			nohz
 			  Disable the tick when a single task runs.

Documentation/admin-guide/kernel-parameters.txt line 1778

Quoted 10 lines
Documentation/admin-guide/kernel-parameters.txt +10 -0
+
+			  A residual 1Hz tick is offloaded to workqueues, which you
+			  need to affine to housekeeping through the global
+			  workqueue's affinity configured via the
+			  /sys/devices/virtual/workqueue/cpumask sysfs file, or
+			  by using the 'domain' flag described below.
+
+			  NOTE: by default the global workqueue runs on all CPUs,
+			  so to protect individual CPUs the 'cpumask' file has to
+			  be configured manually after bootup.

Hmm. Why don't you set that to the housekeeping cpus by default?

Thanks,

tglx

author maintainer
#9 lore raw

Re: [PATCH 4/6] sched/isolation: Offload residual 1Hz scheduler tick

In reply to Thomas Gleixner
Quoted 14 lines · Thomas Gleixner
Quoted 11 lines · Frederic Weisbecker

When a CPU runs in full dynticks mode, a 1Hz tick remains in order to keep the scheduler stats alive. However this residual tick is a burden for bare metal tasks that can't stand any interruption at all, or want to minimize them.

The usual boot parameters "nohz_full=" or "isolcpus=nohz" will now outsource these scheduler ticks to the global workqueue so that a housekeeping CPU handles those remotely. The sched_class::task_tick() implementations have been audited and look safe to be called remotely as the target runqueue and its current task are passed in parameter and don't seem to be accessed locally.

Quoted 5 lines

That scares me a bit. Not for the current state of affairs, but we want to ensure that this still works in 2 years from now

So at least you want to add a comment to task_tick() which explains the constraints which come with the remote tick.

Good point, I'm adding that.

Quoted 4 lines

Other than that this looks good!

Thanks!

author maintainer
#14 lore raw

Re: [PATCH 6/6] sched/isolation: Update nohz documentation to explain tick offload

In reply to Thomas Gleixner
Quoted 25 lines · Thomas Gleixner
Quoted 23 lines · Frederic Weisbecker

Update the documentation to reflect the 1Hz tick offload changes.

Cc Chris Metcalf <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Luiz Capitulino <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 Documentation/admin-guide/kernel-parameters.txt | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +0 -0
index 1d1d53f..50b9837 100644
@@ -1766,6 +1766,17 @@
 
Quoted 14 lines

Documentation/admin-guide/kernel-parameters.txt line 1778

Quoted 12 lines
Documentation/admin-guide/kernel-parameters.txt +10 -0
 			nohz
 			  Disable the tick when a single task runs.
+
+			  A residual 1Hz tick is offloaded to workqueues, which you
+			  need to affine to housekeeping through the global
+			  workqueue's affinity configured via the
+			  /sys/devices/virtual/workqueue/cpumask sysfs file, or
+			  by using the 'domain' flag described below.
+
+			  NOTE: by default the global workqueue runs on all CPUs,
+			  so to protect individual CPUs the 'cpumask' file has to
+			  be configured manually after bootup.

Hmm. Why don't you set that to the housekeeping cpus by default?

"nohz_full=" does affine to housekeeping by default but "isolcpus=nohz" doesn't as it is designed for more finegrained settings.

In fact nohz_full= is what people want to use right now as isolcpus= is still under construction. I'm waiting to be able to make its effects mutable through cpusets before extending it further. But in the end I would like to have things such as:

isolcpus=nohz,async

This would affine the tick and everything that is async (workqueues, unbound timers, unbound kthreads, etc...) to housekeeping.

And an even simpler toggle to fully isolate a CPU:

isolcpus=all

This is to avoid that nohz alone implies side effects. For example if we affine workqueues right now and later the 1hz offloaded tick isn't necessary anymore, we'll stick with affined workqueues as a side effect without reason, and reverting that would end up into regression.

Thanks.