sched: concurrency managed workqueue related sched patches
PATCH 1/1 sched: concurrency managed workqueue related sched patches
Hello, all.
This patchset contains sched patches necessary for concurrency managed workqueue and is composed of two parts. The first four patches are to be applied to sched/core and other than the implementation of force_cpus_allowed(), it makes no functional changes. The purpose of this part is to reduce unnecessary conflicts between ongoing scheduler developements and the temporary scheduler notifiers updates added by the next three patches.
The next three patches are to be applied to a separate branch. The latter part adds sched_notifiers for wakeup and sleep and try_to_wake_up_local() which can be called from those notifiers. These changes are not acked by Ingo. For these changes to go upstream, scheduler notifier framework needs to be unified and cleaned up. However, merging these scheduler changes in a separate stable branch which will only be pulled into linux-next and other testing trees allows c-m-wq a stable base to be developed upon and tested.
This patchset contains the following patches.
0001-sched-revert-498657a478c60be092208422fefa9c7b248729c.patch 0002-sched-rename-preempt_notifiers-to-sched_notifiers-an.patch 0003-sched-refactor-try_to_wake_up.patch 0004-sched-implement-force_cpus_allowed.patch 0005-sched-make-sched_notifiers-unconditional.patch 0006-sched-add-wakeup-sleep-sched_notifiers-and-allow-NUL.patch 0007-sched-implement-try_to_wake_up_local.patch
0001 reverts the mis-fix to sched_in notifier. 0002 and 0003 rename and move code around. These two patches either don't cause or cause very minute difference in the final binary output. 0004 implements force_cpus_allowed(). Although force_cpus_allowed() won't be used in upstream yet, the implementation touches various parts of sched.c so I think it's better to include it into upstream code earlier.
0005-0007 always enables sched_notifiers, add wakeup/sleep and implement try_to_wake_up_local(). As said above, these three can live in a separate branch which won't be pushed to upstream until sched notifier framework is improved.
These patches are available in the following git trees.
First four:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git sched-core-for-ingo
Rest:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git sched-wq-for-ingo
and contains the following changes.
First four:
arch/ia64/kvm/Kconfig | 2 arch/powerpc/kvm/Kconfig | 2 arch/s390/kvm/Kconfig | 2 arch/x86/kvm/Kconfig | 2 include/linux/kvm_host.h | 4 include/linux/preempt.h | 43 ------ include/linux/sched.h | 60 ++++++++- init/Kconfig | 2 kernel/sched.c | 309 +++++++++++++++++++++++++---------------------- virt/kvm/kvm_main.c | 26 +-- 10 files changed, 243 insertions(+), 209 deletions(-)
Rest:
arch/ia64/kvm/Kconfig | 1 arch/powerpc/kvm/Kconfig | 1 arch/s390/kvm/Kconfig | 1 arch/x86/kvm/Kconfig | 1 include/linux/kvm_host.h | 2 - include/linux/sched.h | 14 ++++---- init/Kconfig | 4 -- kernel/sched.c | 80 +++++++++++++++++++++++++++++++++++++---------- 8 files changed, 72 insertions(+), 32 deletions(-)
Thanks.
--
tejun
PATCH 1/7 sched: revert 498657a478c60be092208422fefa9c7b248729c2
498657a478c60be092208422fefa9c7b248729c2 incorrectly assumed that preempt wasn't disabled around context_switch() and thus was fixing imaginary problem. It also broke kvm because it depended on ->sched_in() to be called with irq enabled so that it can do smp calls from there.
Revert the incorrect commit and add comment describing different contexts under with the two callbacks are invoked.
Avi: spotted transposed in/out in the added comment.
--- include/linux/preempt.h | 5 +++++ kernel/sched.c | 2 +- 2 files changed, 6 insertions(+), 1 deletions(-)
include/linux/preempt.h +5 -0
@@ -105,6 +105,11 @@ struct preempt_notifier;* @sched_out: we've just been preempted * notifier: struct preempt_notifier for the task being preempted * next: the task that's kicking us out +* +* Please note that sched_in and out are called under different +* contexts. sched_out is called with rq lock held and irq disabled +* while sched_in is called without rq lock and irq enabled. This +* difference is intentional and depended upon by its users. */ struct preempt_ops {void (*sched_in)(struct preempt_notifier *notifier,int cpu);
kernel/sched.c +1 -1
@@ -2768,9 +2768,9 @@ static void finish_task_switch(struct rq *rq, struct task_struct *prev) prev_state = prev->state;finish_arch_switch (prev);perf_event_task_sched_in (current,cpu_of (rq)); -fire_sched_in_preempt_notifiers (current);finish_lock_switch (rq, prev); +fire_sched_in_preempt_notifiers (current);if (mm)mmdrop (mm);if (unlikely (prev_state ==TASK_DEAD )) {
Signature · 1 line
1.6.4.2
[tip:sched/core] sched: Revert 498657a478c60be092208422fefa9c7b248729c2
Commit-ID: 8592e6486a177a02f048567cb928bc3a1f9a86c3 Gitweb: http://git.kernel.org/tip/8592e6486a177a02f048567cb928bc3a1f9a86c3 Author: Tejun Heo <[email protected]> AuthorDate: Wed, 2 Dec 2009 12:56:46 +0900 Committer: Ingo Molnar <[email protected]> CommitDate: Wed, 2 Dec 2009 09:55:33 +0100
sched: Revert 498657a478c60be092208422fefa9c7b248729c2
498657a478c60be092208422fefa9c7b248729c2 incorrectly assumed that preempt wasn't disabled around context_switch() and thus was fixing imaginary problem. It also broke KVM because it depended on ->sched_in() to be called with irq enabled so that it can do smp calls from there.
Revert the incorrect commit and add comment describing different contexts under with the two callbacks are invoked.
Avi: spotted transposed in/out in the added comment.
LKML-Reference: <[email protected]>
--- include/linux/preempt.h | 5 +++++ kernel/sched.c | 2 +- 2 files changed, 6 insertions(+), 1 deletions(-)
include/linux/preempt.h +5 -0
@@ -105,6 +105,11 @@ struct preempt_notifier;* @sched_out: we've just been preempted * notifier: struct preempt_notifier for the task being preempted * next: the task that's kicking us out +* +* Please note that sched_in and out are called under different +* contexts. sched_out is called with rq lock held and irq disabled +* while sched_in is called without rq lock and irq enabled. This +* difference is intentional and depended upon by its users. */ struct preempt_ops {void (*sched_in)(struct preempt_notifier *notifier,int cpu);
kernel/sched.c +1 -1
@@ -2768,9 +2768,9 @@ static void finish_task_switch(struct rq *rq, struct task_struct *prev) prev_state = prev->state;finish_arch_switch (prev);perf_event_task_sched_in (current,cpu_of (rq)); -fire_sched_in_preempt_notifiers (current);finish_lock_switch (rq, prev); +fire_sched_in_preempt_notifiers (current);if (mm)mmdrop (mm);if (unlikely (prev_state ==TASK_DEAD )) {
PATCH 2/7 sched: rename preempt_notifiers to sched_notifiers and refactor implementation
In reply to Tejun HeoRename preempt_notifiers to sched_notifiers and move it to sched.h. Also, refactor implementation in sched.c such that adding new callbacks is easier.
This patch does not introduce any functional change and in fact generates the same binary at least with my configuration (x86_64 SMP, kvm and some debug options).
--- arch/ia64/kvm/Kconfig | 2 +- arch/powerpc/kvm/Kconfig | 2 +- arch/s390/kvm/Kconfig | 2 +- arch/x86/kvm/Kconfig | 2 +- include/linux/kvm_host.h | 4 +- include/linux/preempt.h | 48 -------------------- include/linux/sched.h | 53 +++++++++++++++++++++- init/Kconfig | 2 +- kernel/sched.c | 108 +++++++++++++++++++--------------------------- virt/kvm/kvm_main.c | 26 +++++------ 10 files changed, 113 insertions(+), 136 deletions(-)
arch/ia64/kvm/Kconfig +1 -1
@@ -22,7 +22,7 @@ config KVM depends on HAVE_KVM && MODULES && EXPERIMENTAL # for device assignment: depends on PCI - select PREEMPT_NOTIFIERS + select SCHED_NOTIFIERS select ANON_INODES select HAVE_KVM_IRQCHIP select KVM_APIC_ARCHITECTURE
arch/powerpc/kvm/Kconfig +1 -1
@@ -18,7 +18,7 @@ if VIRTUALIZATION config KVM bool - select PREEMPT_NOTIFIERS + select SCHED_NOTIFIERS select ANON_INODES config KVM_440
arch/s390/kvm/Kconfig +1 -1
@@ -18,7 +18,7 @@ if VIRTUALIZATION config KVM tristate "Kernel-based Virtual Machine (KVM) support" depends on HAVE_KVM && EXPERIMENTAL - select PREEMPT_NOTIFIERS + select SCHED_NOTIFIERS select ANON_INODES select S390_SWITCH_AMODE ---help---
arch/x86/kvm/Kconfig +1 -1
@@ -22,7 +22,7 @@ config KVM depends on HAVE_KVM # for device assignment: depends on PCI - select PREEMPT_NOTIFIERS + select SCHED_NOTIFIERS select MMU_NOTIFIER select ANON_INODES select HAVE_KVM_IRQCHIP
include/linux/kvm_host.h +2 -2
@@ -74,8 +74,8 @@ void kvm_io_bus_unregister_dev(struct kvm *kvm, struct kvm_io_bus *bus,struct kvm_vcpu {struct kvm *kvm; -#ifdef CONFIG_PREEMPT_NOTIFIERS -struct preempt_notifier preempt_notifier; +#ifdef CONFIG_SCHED_NOTIFIERS +struct sched_notifier sched_notifier;#endif int vcpu_id;struct mutex mutex;
include/linux/preempt.h +0 -48
52 lines not on this page · raw message
include/linux/sched.h +50 -3
65 lines not on this page · raw message
init/Kconfig +1 -1
@@ -1207,6 +1207,6 @@ config STOP_MACHINE source "block/Kconfig" -config PREEMPT_NOTIFIERS +config SCHED_NOTIFIERS bool
kernel/sched.c +44 -64
144 lines not on this page · raw message
virt/kvm/kvm_main.c +12 -14
@@ -79,7 +79,7 @@ static cpumask_var_t cpus_hardware_enabled;struct kmem_cache *kvm_vcpu_cache;EXPORT_SYMBOL_GPL (kvm_vcpu_cache); -static __read_mostly struct preempt_ops kvm_preempt_ops; +static __read_mostly struct sched_notifier_ops kvm_sched_notifier_ops;struct dentry *kvm_debugfs_dir;
@@ -713,7 +713,7 @@ void vcpu_load(struct kvm_vcpu *vcpu)mutex_lock (&vcpu->mutex); cpu =get_cpu (); -preempt_notifier_register (&vcpu->preempt_notifier); +sched_notifier_register (&vcpu->sched_notifier);kvm_arch_vcpu_load (vcpu, cpu);put_cpu (); }
@@ -722,7 +722,7 @@ void vcpu_put(struct kvm_vcpu *vcpu) {preempt_disable ();kvm_arch_vcpu_put (vcpu); -preempt_notifier_unregister (&vcpu->preempt_notifier); +sched_notifier_unregister (&vcpu->sched_notifier);preempt_enable ();mutex_unlock (&vcpu->mutex); }
@@ -1772,7 +1772,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)if (IS_ERR (vcpu))return PTR_ERR (vcpu); -preempt_notifier_init (&vcpu->preempt_notifier, &kvm_preempt_ops); +sched_notifier_init (&vcpu->sched_notifier, &kvm_sched_notifier_ops); r =kvm_arch_vcpu_setup (vcpu);if (r)
@@ -2690,23 +2690,21 @@ static struct sys_device kvm_sysdev = {struct page *bad_page;pfn_t bad_pfn; -static inline -struct kvm_vcpu *preempt_notifier_to_vcpu (struct preempt_notifier *pn) +static inline struct kvm_vcpu *sched_notifier_to_vcpu (struct sched_notifier *sn) { - return container_of(pn, struct kvm_vcpu, preempt_notifier); + return container_of(sn, struct kvm_vcpu, sched_notifier); } -static void kvm_sched_in (struct preempt_notifier *pn,int cpu) +static void kvm_sched_in (struct sched_notifier *sn,int cpu) { -struct kvm_vcpu *vcpu =preempt_notifier_to_vcpu (pn); +struct kvm_vcpu *vcpu =sched_notifier_to_vcpu (sn);kvm_arch_vcpu_load (vcpu, cpu); } -static void kvm_sched_out (struct preempt_notifier *pn, -struct task_struct *next) +static void kvm_sched_out (struct sched_notifier *sn,struct task_struct *next) { -struct kvm_vcpu *vcpu =preempt_notifier_to_vcpu (pn); +struct kvm_vcpu *vcpu =sched_notifier_to_vcpu (sn);kvm_arch_vcpu_put (vcpu); }
@@ -2780,8 +2778,8 @@ int kvm_init(void *opaque, unsigned int vcpu_size, goto out_free; } - kvm_preempt_ops.sched_in = kvm_sched_in; - kvm_preempt_ops.sched_out = kvm_sched_out; + kvm_sched_notifier_ops.in = kvm_sched_in; + kvm_sched_notifier_ops.out = kvm_sched_out;kvm_init_debug ();
Signature · 1 line
1.6.4.2
PATCH 3/7 sched: refactor try_to_wake_up()
In reply to Tejun HeoFactor ttwu_activate() and ttwu_woken_up() out of try_to_wake_up(). The factoring out doesn't affect try_to_wake_up() much code-generation-wise. Depending on configuration options, it ends up generating the same object code as before or slightly different one due to different register assignment.
This is to help future implementation of try_to_wake_up_local().
--- kernel/sched.c | 112 +++++++++++++++++++++++++++++++------------------------ 1 files changed, 63 insertions(+), 49 deletions(-)
kernel/sched.c +63 -49
@@ -2361,11 +2361,67 @@ void task_oncpu_function_call(struct task_struct *p,preempt_enable (); } -/*** +static inline void ttwu_activate (struct task_struct *p,struct rq *rq, +bool is_sync,bool is_migrate,bool is_local) +{ +schedstat_inc (p, se.nr_wakeups); +if (is_sync) +schedstat_inc (p, se.nr_wakeups_sync); +if (is_migrate) +schedstat_inc (p, se.nr_wakeups_migrate); +if (is_local) +schedstat_inc (p, se.nr_wakeups_local); +else +schedstat_inc (p, se.nr_wakeups_remote); + +activate_task (rq, p,1 ); + +/* +* Only attribute actual wakeups done by this task. +*/ +if (!in_interrupt ()) { +struct sched_entity *se = ¤t->se; +u64 sample = se->sum_exec_runtime; + +if (se->last_wakeup) + sample -= se->last_wakeup; +else + sample -= se->start_runtime; +update_avg (&se->avg_wakeup, sample); + + se->last_wakeup = se->sum_exec_runtime; + } +} + +static inline void ttwu_woken_up (struct task_struct *p,struct rq *rq, +int wake_flags,bool success) +{ +trace_sched_wakeup (rq, p, success); +check_preempt_curr (rq, p, wake_flags); + + p->state =TASK_RUNNING ; +#ifdef CONFIG_SMP +if (p->sched_class->task_wake_up) + p->sched_class->task_wake_up (rq, p); + +if (unlikely (rq->idle_stamp)) { +u64 delta = rq->clock - rq->idle_stamp; +u64 max =2 *sysctl_sched_migration_cost; + +if (delta > max) + rq->avg_idle = max; +else +update_avg (&rq->avg_idle, delta); + rq->idle_stamp =0 ; + } +#endif +} + +/** * try_to_wake_up - wake up a thread * @p: the to-be-woken-up thread * @state: the mask of task states that can be woken -* @sync: do a synchronous wakeup? +* @wake_flags: wake modifier flags (WF_*) * * Put it on the run-queue if it's not already there. The "current" * thread is always on the run-queue (except when the actual
@@ -2373,7 +2429,8 @@ void task_oncpu_function_call(struct task_struct *p,* the simpler "current->state = TASK_RUNNING" to mark yourself * runnable without the overhead of this. * -* returns failure only if the task is already active. +* Returns %true if @p was woken up, %false if it was already running +* or @state didn't match @p's state. */ static int try_to_wake_up (struct task_struct *p,unsigned int state,int wake_flags)
@@ -2444,54 +2501,11 @@ static int try_to_wake_up(struct task_struct *p, unsigned int state, out_activate:#endif /* CONFIG_SMP */ -schedstat_inc (p, se.nr_wakeups); -if (wake_flags &WF_SYNC ) -schedstat_inc (p, se.nr_wakeups_sync); -if (orig_cpu != cpu) -schedstat_inc (p, se.nr_wakeups_migrate); -if (cpu == this_cpu) -schedstat_inc (p, se.nr_wakeups_local); -else -schedstat_inc (p, se.nr_wakeups_remote); -activate_task (rq, p,1 ); +ttwu_activate (p, rq, wake_flags &WF_SYNC , orig_cpu != cpu, + cpu == this_cpu); success =1 ; - -/* -* Only attribute actual wakeups done by this task. -*/ -if (!in_interrupt ()) { -struct sched_entity *se = ¤t->se; -u64 sample = se->sum_exec_runtime; - -if (se->last_wakeup) - sample -= se->last_wakeup; -else - sample -= se->start_runtime; -update_avg (&se->avg_wakeup, sample); - - se->last_wakeup = se->sum_exec_runtime; - } - out_running: -trace_sched_wakeup (rq, p, success); -check_preempt_curr (rq, p, wake_flags); - - p->state =TASK_RUNNING ; -#ifdef CONFIG_SMP -if (p->sched_class->task_wake_up) - p->sched_class->task_wake_up (rq, p); - -if (unlikely (rq->idle_stamp)) { -u64 delta = rq->clock - rq->idle_stamp; -u64 max =2 *sysctl_sched_migration_cost; - -if (delta > max) - rq->avg_idle = max; -else -update_avg (&rq->avg_idle, delta); - rq->idle_stamp =0 ; - } -#endif +ttwu_woken_up (p, rq, wake_flags, success); out:task_rq_unlock (rq, &flags);put_cpu ();
Signature · 1 line
1.6.4.2
Re: [PATCH 3/7] sched: refactor try_to_wake_up()
Quoted 1 line · Tejun Heo
Factor
ttwu_activate()andttwu_woken_up()out oftry_to_wake_up().
Nit: ttwu_woken_up() sounds decidedly strange to my ear. Perhaps ttwu_post_activation()?
As a $.02 comment, factoring here doesn't look nice, reader scrolls around whereas he currently sees all the why/wherefore at a glance. Needing to pass three booleans for stats also looks bad. I think it would _look_ better with the thing just duplicated/stripped down and called what it is, sched_notifier_wakeup() or such.
Which leaves growth in it's wake though...
Quoted 3 lines
+/**
*try_to_wake_up- wake up a thread
* @p: the to-be-woken-up thread
Nit: thread to be awakened sounds better.
-Mike
Re: [PATCH 3/7] sched: refactor try_to_wake_up()
Hello,
Quoted 5 lines · Mike Galbraith
Quoted 1 line · Tejun Heo
Factor
ttwu_activate()andttwu_woken_up()out oftry_to_wake_up().Nit:
ttwu_woken_up()sounds decidedly strange to my ear. Perhapsttwu_post_activation()?
Sure, I can rename it.
Quoted 3 lines
As a $.02 comment, factoring here doesn't look nice, reader scrolls around whereas he currently sees all the why/wherefore at a glance. Needing to pass three booleans for stats also looks bad.
The three bools aren't the prettiest thing in the world but I couldn't prevent gcc from re-evaluating expressions without those.
Quoted 3 lines
I think it would _look_ better with the thing just duplicated/stripped down and called what it is,
sched_notifier_wakeup()or such.
Sorry, I'm not following. Can you elaborate a bit?
Quoted 7 lines
Which leaves growth in it's wake though...
Quoted 3 lines
+/**
*try_to_wake_up- wake up a thread
* @p: the to-be-woken-up threadNit: thread to be awakened sounds better.
Will update.
Thanks.
Signature · 1 line
tejun
Re: [PATCH 3/7] sched: refactor try_to_wake_up()
Quoted 1 line · Tejun Heo
Hello,
Greetings,
Quoted 5 lines
Quoted 3 lines
I think it would _look_ better with the thing just duplicated/stripped down and called what it is,
sched_notifier_wakeup()or such.Sorry, I'm not following. Can you elaborate a bit?
I mean copying ttwu, stripping out locking etc, and renaming the result would look better. No functional difference though, and binary growth would be left in the wake of such a duplication.
-Mike
Re: [PATCH 3/7] sched: refactor try_to_wake_up()
Quoted 9 lines · Mike Galbraith
Quoted 5 lines
Quoted 3 lines
I think it would _look_ better with the thing just duplicated/stripped down and called what it is,
sched_notifier_wakeup()or such.Sorry, I'm not following. Can you elaborate a bit?
I mean copying ttwu, stripping out locking etc, and renaming the result would look better. No functional difference though, and binary growth would be left in the wake of such a duplication.
Ah... okay. Well, that was the first approach (with some missing parts). Peter wasn't particularly happy with code duplication and suggested refactoring ttwu and reuse common parts.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 3/7] sched: refactor try_to_wake_up()
Quoted 7 lines · Tejun Heo
Quoted 6 lines · Mike Galbraith
Quoted 5 lines
Quoted 3 lines
I think it would _look_ better with the thing just duplicated/stripped down and called what it is,
sched_notifier_wakeup()or such.Sorry, I'm not following. Can you elaborate a bit?
Quoted 7 lines
Quoted 3 lines
I mean copying ttwu, stripping out locking etc, and renaming the result would look better. No functional difference though, and binary growth would be left in the wake of such a duplication.
Ah... okay. Well, that was the first approach (with some missing parts). Peter wasn't particularly happy with code duplication and suggested refactoring ttwu and reuse common parts.
Right, esp since it missed some parts.
I haven't yet had time to look at this patch, but yeah, sharing code seems like a good idea since its tricky and fragile code.
PATCH 3/7 sched: refactor try_to_wake_up()
In reply to Tejun HeoFactor ttwu_activate() and ttwu_woken_up() out of try_to_wake_up(). The factoring out doesn't affect try_to_wake_up() much code-generation-wise. Depending on configuration options, it ends up generating the same object code as before or slightly different one due to different register assignment.
This is to help future implementation of try_to_wake_up_local().
Mike Galbraith suggested rename to ttwu_post_activation() from ttwu_woken_up() and comment update in try_to_wake_up().
---
Updated to reflect Mike Galbraith's comment. The only difference is function rename and comment change.
Updated tree available on top of current sched/core available in the following git trees.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git sched-core-for-ingo
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git sched-wq-for-ingo
Thanks.
kernel/sched.c | 114 ++++++++++++++++++++++++++++++++------------------------- 1 file changed, 64 insertions(+), 50 deletions(-)
kernel/sched.c +64 -50
@@ -2361,11 +2361,67 @@ void task_oncpu_function_call(struct taspreempt_enable (); } -/*** +static inline void ttwu_activate (struct task_struct *p,struct rq *rq, +bool is_sync,bool is_migrate,bool is_local) +{ +schedstat_inc (p, se.nr_wakeups); +if (is_sync) +schedstat_inc (p, se.nr_wakeups_sync); +if (is_migrate) +schedstat_inc (p, se.nr_wakeups_migrate); +if (is_local) +schedstat_inc (p, se.nr_wakeups_local); +else +schedstat_inc (p, se.nr_wakeups_remote); + +activate_task (rq, p,1 ); + +/* +* Only attribute actual wakeups done by this task. +*/ +if (!in_interrupt ()) { +struct sched_entity *se = ¤t->se; +u64 sample = se->sum_exec_runtime; + +if (se->last_wakeup) + sample -= se->last_wakeup; +else + sample -= se->start_runtime; +update_avg (&se->avg_wakeup, sample); + + se->last_wakeup = se->sum_exec_runtime; + } +} + +static inline void ttwu_post_activation (struct task_struct *p,struct rq *rq, +int wake_flags,bool success) +{ +trace_sched_wakeup (rq, p, success); +check_preempt_curr (rq, p, wake_flags); + + p->state =TASK_RUNNING ; +#ifdef CONFIG_SMP +if (p->sched_class->task_wake_up) + p->sched_class->task_wake_up (rq, p); + +if (unlikely (rq->idle_stamp)) { +u64 delta = rq->clock - rq->idle_stamp; +u64 max =2 *sysctl_sched_migration_cost; + +if (delta > max) + rq->avg_idle = max; +else +update_avg (&rq->avg_idle, delta); + rq->idle_stamp =0 ; + } +#endif +} + +/** * try_to_wake_up - wake up a thread -* @p: the to-be-woken-up thread +* @p: the thread to be awakened * @state: the mask of task states that can be woken -* @sync: do a synchronous wakeup? +* @wake_flags: wake modifier flags (WF_*) * * Put it on the run-queue if it's not already there. The "current" * thread is always on the run-queue (except when the actual
@@ -2373,7 +2429,8 @@ void task_oncpu_function_call(struct tas* the simpler "current->state = TASK_RUNNING" to mark yourself * runnable without the overhead of this. * -* returns failure only if the task is already active. +* Returns %true if @p was woken up, %false if it was already running +* or @state didn't match @p's state. */ static int try_to_wake_up (struct task_struct *p,unsigned int state,int wake_flags)
@@ -2444,54 +2501,11 @@ static int try_to_wake_up(struct task_st out_activate:#endif /* CONFIG_SMP */ -schedstat_inc (p, se.nr_wakeups); -if (wake_flags &WF_SYNC ) -schedstat_inc (p, se.nr_wakeups_sync); -if (orig_cpu != cpu) -schedstat_inc (p, se.nr_wakeups_migrate); -if (cpu == this_cpu) -schedstat_inc (p, se.nr_wakeups_local); -else -schedstat_inc (p, se.nr_wakeups_remote); -activate_task (rq, p,1 ); +ttwu_activate (p, rq, wake_flags &WF_SYNC , orig_cpu != cpu, + cpu == this_cpu); success =1 ; - -/* -* Only attribute actual wakeups done by this task. -*/ -if (!in_interrupt ()) { -struct sched_entity *se = ¤t->se; -u64 sample = se->sum_exec_runtime; - -if (se->last_wakeup) - sample -= se->last_wakeup; -else - sample -= se->start_runtime; -update_avg (&se->avg_wakeup, sample); - - se->last_wakeup = se->sum_exec_runtime; - } - out_running: -trace_sched_wakeup (rq, p, success); -check_preempt_curr (rq, p, wake_flags); - - p->state =TASK_RUNNING ; -#ifdef CONFIG_SMP -if (p->sched_class->task_wake_up) - p->sched_class->task_wake_up (rq, p); - -if (unlikely (rq->idle_stamp)) { -u64 delta = rq->clock - rq->idle_stamp; -u64 max =2 *sysctl_sched_migration_cost; - -if (delta > max) - rq->avg_idle = max; -else -update_avg (&rq->avg_idle, delta); - rq->idle_stamp =0 ; - } -#endif +ttwu_post_activation (p, rq, wake_flags, success); out:task_rq_unlock (rq, &flags);put_cpu ();
PATCH 4/7 sched: implement force_cpus_allowed()
In reply to Tejun Heoset_cpus_allowed_ptr() modifies the allowed cpu mask of a task. The function performs the following checks before applying new mask.
* Check whether PF_THREAD_BOUND is set. This is set for bound kthreads so that they can't be moved around.
* Check whether the target cpu is still marked active - cpu_active(). Active state is cleared early while downing a cpu.
This patch adds force_cpus_allowed() which bypasses the above two checks. The caller is responsible for guaranteeing that the destination cpu doesn't go down until force_cpus_allowed() finishes.
The first check is bypassed by factoring out actual migration part into __set_cpus_allowed() from set_cpus_allowed_ptr() and calling the inner function from force_cpus_allowed().
The second check is buried deep down in __migrate_task() which is executed by migration threads. @force parameter is added to __migrate_task(). As the only way to pass parameters from __set_cpus_allowed() is through migration_req, migration_req->force is added and the @force parameter is passed down to __migrate_task().
Please note the naming discrepancy between set_cpus_allowed_ptr() and the new functions. The _ptr suffix is from the days when cpumask api wasn't mature and future changes should drop it from set_cpus_allowed_ptr() too.
force_cpus_allowed() will be used for concurrency-managed workqueue.
--- include/linux/sched.h | 7 ++++ kernel/sched.c | 89 +++++++++++++++++++++++++++++++++---------------- 2 files changed, 67 insertions(+), 29 deletions(-)
include/linux/sched.h +7 -0
@@ -1854,6 +1854,8 @@ static inline void rcu_copy_process(struct task_struct *p)#ifdef CONFIG_SMP extern int set_cpus_allowed_ptr (struct task_struct *p,const struct cpumask *new_mask); +extern int force_cpus_allowed (struct task_struct *p, +const struct cpumask *new_mask);#else static inline int set_cpus_allowed_ptr (struct task_struct *p,const struct cpumask *new_mask)
@@ -1862,6 +1864,11 @@ static inline int set_cpus_allowed_ptr(struct task_struct *p,return -EINVAL ; return 0; } +static inline int force_cpus_allowed (struct task_struct *p, +const struct cpumask *new_mask) +{ +return set_cpus_allowed_ptr (p, new_mask); +}#endif #ifndef CONFIG_CPUMASK_OFFSTACK
kernel/sched.c +60 -29
172 lines not on this page · raw message
Signature · 1 line
1.6.4.2
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 23 lines · Tejun Heo
set_cpus_allowed_ptr()modifies the allowed cpu mask of a task. The function performs the following checks before applying new mask.* Check whether
PF_THREAD_BOUNDis set. This is set for bound kthreads so that they can't be moved around.* Check whether the target cpu is still marked active -
cpu_active(). Active state is cleared early while downing a cpu.This patch adds
force_cpus_allowed()which bypasses the above two checks. The caller is responsible for guaranteeing that the destination cpu doesn't go down untilforce_cpus_allowed()finishes.The first check is bypassed by factoring out actual migration part into
__set_cpus_allowed()fromset_cpus_allowed_ptr()and calling the inner function fromforce_cpus_allowed().The second check is buried deep down in
__migrate_task()which is executed by migration threads. @force parameter is added to__migrate_task(). As the only way to pass parameters from__set_cpus_allowed()is throughmigration_req,migration_req->forceis added and the @force parameter is passed down to__migrate_task().
Quoted 6 lines
Please note the naming discrepancy between
set_cpus_allowed_ptr()and the new functions. The_ptrsuffix is from the days when cpumask api wasn't mature and future changes should drop it fromset_cpus_allowed_ptr()too.
force_cpus_allowed()will be used for concurrency-managed workqueue.
Would still like to know why all this is needed.
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 24 lines · Peter Zijlstra
Quoted 23 lines · Tejun Heo
set_cpus_allowed_ptr()modifies the allowed cpu mask of a task. The function performs the following checks before applying new mask.* Check whether
PF_THREAD_BOUNDis set. This is set for bound kthreads so that they can't be moved around.* Check whether the target cpu is still marked active -
cpu_active(). Active state is cleared early while downing a cpu.This patch adds
force_cpus_allowed()which bypasses the above two checks. The caller is responsible for guaranteeing that the destination cpu doesn't go down untilforce_cpus_allowed()finishes.The first check is bypassed by factoring out actual migration part into
__set_cpus_allowed()fromset_cpus_allowed_ptr()and calling the inner function fromforce_cpus_allowed().The second check is buried deep down in
__migrate_task()which is executed by migration threads. @force parameter is added to__migrate_task(). As the only way to pass parameters from__set_cpus_allowed()is throughmigration_req,migration_req->forceis added and the @force parameter is passed down to__migrate_task().
Quoted 8 lines
Quoted 6 lines
Please note the naming discrepancy between
set_cpus_allowed_ptr()and the new functions. The_ptrsuffix is from the days when cpumask api wasn't mature and future changes should drop it fromset_cpus_allowed_ptr()too.
force_cpus_allowed()will be used for concurrency-managed workqueue.Would still like to know why all this is needed.
That is, what problem do these new-fangled workqueues have and why is this a good solution.
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello, Peter.
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines
Quoted 1 line
force_cpus_allowed()will be used for concurrency-managed workqueue.Would still like to know why all this is needed.
That is, what problem do these new-fangled workqueues have and why is this a good solution.
This is the original RFC posting of cmwq which includes the whole thing. I'm a few days away from posting a new version but the usage of force_cpus_allowed() remains the same.
http://thread.gmane.org/gmane.linux.kernel/896268/focus=896294
There are two tests which are bypassed by the force_ variant.
* PF_THREAD_BOUND. This is used to mark tasks which are bound to a cpu using kthread_bind() to be bound permanently. However, new trustee based workqueue hotplugging decouples per-cpu workqueue flushing with cpu hot plug/unplugging. This is necessary because with cmwq, long running works can be served by regular workqueues, so delaying completion of hot plug/unplugging till certain works are flushed isn't feasible. So, what becomes necessary is the ability to re-bind tasks which has PF_THREAD_BOUND set but unbound from its now offline cpu which is coming online again.
* cpu_active() test. CPU activeness encloses cpu online status which is the actual on/offline state. Workqueues need to keep running while a cpu is going down and with cmwq keeping workqueues running involves creation of new workers (it consists part of forward-progress guarantee and one of cpu down callbacks might end up waiting on completion of certain works).
The problem with active state is that during cpu down, active going off doesn't mean all tasks have been migrated off the cpu, so without a migration interface which is synchronized with the actual offline migration, it is difficult to guarantee that all works are either running on the designated cpu if the cpu is online or all work on other cpus if the cpu is offline.
Another related problem is that there's no way to monitor the cpu activeness change notifications.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 18 lines · Tejun Heo
Hello, Peter.
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines
Quoted 1 line
force_cpus_allowed()will be used for concurrency-managed workqueue.Would still like to know why all this is needed.
That is, what problem do these new-fangled workqueues have and why is this a good solution.
This is the original RFC posting of cmwq which includes the whole thing. I'm a few days away from posting a new version but the usage of
force_cpus_allowed()remains the same.http://thread.gmane.org/gmane.linux.kernel/896268/focus=896294
There are two tests which are bypassed by the
force_variant.
Quoted 9 lines
*
PF_THREAD_BOUND. This is used to mark tasks which are bound to a cpu usingkthread_bind()to be bound permanently. However, new trustee based workqueue hotplugging decouples per-cpu workqueue flushing with cpu hot plug/unplugging. This is necessary because with cmwq, long running works can be served by regular workqueues, so delaying completion of hot plug/unplugging till certain works are flushed isn't feasible. So, what becomes necessary is the ability to re-bind tasks which hasPF_THREAD_BOUNDset but unbound from its now offline cpu which is coming online again.
I'm not at all sure I like that. I'd be perfectly happy with delaying the hot-unplug.
The whole cpu hotplug mess is tricky enough as it is and I see no compelling reason to further complicate it. If people are really going to enqueue strict per-cpu worklets (queue_work_on()) that takes seconds to complete, then they get to keep the results of that, which includes slow hot unplug.
Having an off-line cpu still process code like it was online is asking for trouble, don't go there.
Quoted 14 lines
*
cpu_active()test. CPU activeness encloses cpu online status which is the actual on/offline state. Workqueues need to keep running while a cpu is going down and with cmwq keeping workqueues running involves creation of new workers (it consists part of forward-progress guarantee and one of cpu down callbacks might end up waiting on completion of certain works).The problem with active state is that during cpu down, active going off doesn't mean all tasks have been migrated off the cpu, so without a migration interface which is synchronized with the actual offline migration, it is difficult to guarantee that all works are either running on the designated cpu if the cpu is online or all work on other cpus if the cpu is offline.
Quoted 2 lines
Another related problem is that there's no way to monitor the cpu activeness change notifications.
cpu_active() is basically meant for the scheduler to not stick new tasks on a dying cpu.
So on hot-unplug you'd want to splice your worklets to another cpu, except maybe those strictly enqueued to the dying cpu, and since there was work on the dying cpu, you already had a task processing them, so you don't need new tasks, right?
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 13 lines · Peter Zijlstra
Quoted 9 lines
*
PF_THREAD_BOUND. This is used to mark tasks which are bound to a cpu usingkthread_bind()to be bound permanently. However, new trustee based workqueue hotplugging decouples per-cpu workqueue flushing with cpu hot plug/unplugging. This is necessary because with cmwq, long running works can be served by regular workqueues, so delaying completion of hot plug/unplugging till certain works are flushed isn't feasible. So, what becomes necessary is the ability to re-bind tasks which hasPF_THREAD_BOUNDset but unbound from its now offline cpu which is coming online again.I'm not at all sure I like that. I'd be perfectly happy with delaying the hot-unplug.
Quoted 8 lines
The whole cpu hotplug mess is tricky enough as it is and I see no compelling reason to further complicate it. If people are really going to enqueue strict per-cpu worklets (
queue_work_on()) that takes seconds to complete, then they get to keep the results of that, which includes slow hot unplug.Having an off-line cpu still process code like it was online is asking for trouble, don't go there.
We're already there. Users of workqueue which require strict CPU affinity are required to flush respective works from CPU down notifiers and fire them as necessary on up notifiers; otherwise, works will continue to run until they're done after the cpu went down regardless of whether explicit queue_work_on() was used or not.
The only difference is that now the ability to create new workers is necessary to guarantee forward progress while cpu is going down, so it's not that different from what we're doing now.
Quoted 10 lines
Quoted 2 lines
Another related problem is that there's no way to monitor the cpu activeness change notifications.
cpu_active()is basically meant for the scheduler to not stick new tasks on a dying cpu.So on hot-unplug you'd want to splice your worklets to another cpu, except maybe those strictly enqueued to the dying cpu, and since there was work on the dying cpu, you already had a task processing them, so you don't need new tasks, right?
We need to be able to fork to guarantee forward progress at any point which means that to guarantee flush_work() or flush_workqueue() from cpu down notification doesn't block indefinitely, we need to be able to create new workers after cpu_active() is clear.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 22 lines · Tejun Heo
Hello,
Quoted 19 lines · Peter Zijlstra
Quoted 9 lines
*
PF_THREAD_BOUND. This is used to mark tasks which are bound to a cpu usingkthread_bind()to be bound permanently. However, new trustee based workqueue hotplugging decouples per-cpu workqueue flushing with cpu hot plug/unplugging. This is necessary because with cmwq, long running works can be served by regular workqueues, so delaying completion of hot plug/unplugging till certain works are flushed isn't feasible. So, what becomes necessary is the ability to re-bind tasks which hasPF_THREAD_BOUNDset but unbound from its now offline cpu which is coming online again.I'm not at all sure I like that. I'd be perfectly happy with delaying the hot-unplug.
The whole cpu hotplug mess is tricky enough as it is and I see no compelling reason to further complicate it. If people are really going to enqueue strict per-cpu worklets (
queue_work_on()) that takes seconds to complete, then they get to keep the results of that, which includes slow hot unplug.
Quoted 8 lines
Quoted 2 lines
Having an off-line cpu still process code like it was online is asking for trouble, don't go there.
We're already there. Users of workqueue which require strict CPU affinity are required to flush respective works from CPU down notifiers and fire them as necessary on up notifiers; otherwise, works will continue to run until they're done after the cpu went down regardless of whether explicit
queue_work_on()was used or not.
So we seem to do cleanup_workqueue_thread() from CPU_POST_DEAD, but at that time any thread that might still be around will most certainly not be running on the offlined cpu anymore.
If you really want to ensure you remain on the cpu, you have to complete from CPU_DOWN_PREPARE.
We're not running things from offline CPUs.
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 8 lines · Peter Zijlstra
So we seem to do
cleanup_workqueue_thread()fromCPU_POST_DEAD, but at that time any thread that might still be around will most certainly not be running on the offlined cpu anymore.If you really want to ensure you remain on the cpu, you have to complete from
CPU_DOWN_PREPARE.We're not running things from offline CPUs.
Oh, no, we're not doing that. We can't do that. What we're doing is to continue to process works which were queued on the now offline cpu unless it has been flushed/cancled from one of the cpu down notifications and the reason why we need to be able to fork after active is clear is to guarantee those flush/cancels don't deadlock.
Thanks.
Signature · 1 line
tejun
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 8 lines · Tejun Heo
Quoted 7 lines · Peter Zijlstra
So we seem to do
cleanup_workqueue_thread()fromCPU_POST_DEAD, but at that time any thread that might still be around will most certainly not be running on the offlined cpu anymore.If you really want to ensure you remain on the cpu, you have to complete from
CPU_DOWN_PREPARE.
Quoted 7 lines
Quoted 1 line
We're not running things from offline CPUs.
Oh, no, we're not doing that. We can't do that. What we're doing is to continue to process works which were queued on the now offline cpu unless it has been flushed/cancled from one of the cpu down notifications and the reason why we need to be able to fork after active is clear is to guarantee those flush/cancels don't deadlock.
Does my explanation justify the patch?
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 13 lines · Tejun Heo
Hello,
Quoted 10 lines · Tejun Heo
Quoted 8 lines · Peter Zijlstra
So we seem to do
cleanup_workqueue_thread()fromCPU_POST_DEAD, but at that time any thread that might still be around will most certainly not be running on the offlined cpu anymore.If you really want to ensure you remain on the cpu, you have to complete from
CPU_DOWN_PREPARE.We're not running things from offline CPUs.
Quoted 7 lines
Quoted 5 lines
Oh, no, we're not doing that. We can't do that. What we're doing is to continue to process works which were queued on the now offline cpu unless it has been flushed/cancled from one of the cpu down notifications and the reason why we need to be able to fork after active is clear is to guarantee those flush/cancels don't deadlock.
Does my explanation justify the patch?
So its only needed in order to flush a workqueue from CPU_DOWN_PREPARE? And all you need it to place a new kthread on a !active cpu?
Or is this in order to allow migrate_live_tasks() to move the worker threads away from the dead cpu?
I'm really not thrilled by the whole fork-fest workqueue design.
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 2 lines · Peter Zijlstra
So its only needed in order to flush a workqueue from
CPU_DOWN_PREPARE? And all you need it to place a new kthread on a !active cpu?
Yes, that's all I need.
Quoted 2 lines
Or is this in order to allow
migrate_live_tasks()to move the worker threads away from the dead cpu?
Nope, that's left to sched cpu callbacks.
Quoted 1 line
I'm really not thrilled by the whole fork-fest workqueue design.
Shared work pool kind of needs forks unless it's gonna create all the workers upfront. Most issues arise from corner cases - deadlocks, cpu hotplugging, freeze/thaw and so on. During usual operation, the thread pool would remain largely stable.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 4 lines · Tejun Heo
Quoted 2 lines
So its only needed in order to flush a workqueue from
CPU_DOWN_PREPARE? And all you need it to place a new kthread on a !active cpu?Yes, that's all I need.
Then you don't need most of that patch, you don't need to touch the migration stuff since a new kthread isn't running.
All you need to do is make a kthread_bind/set_cpus_allowed variant that checks against cpu_online_mask instead of cpu_active_mask.
It might even make sense to have kthread_bind() always check with cpu_online_mask as the kernel really ought to know what its doing anyway.
You also don't need to play trickery with PF_THREAD_BOUND, since a new kthread will not have that set.
In fact, your patch is against a tree that still has cpu_online_mask in all those places, so you wouldn't have needed any of that, confused.. ?!
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 16 lines · Peter Zijlstra
Quoted 4 lines · Tejun Heo
Quoted 2 lines
So its only needed in order to flush a workqueue from
CPU_DOWN_PREPARE? And all you need it to place a new kthread on a !active cpu?Yes, that's all I need.
Then you don't need most of that patch, you don't need to touch the migration stuff since a new kthread isn't running.
All you need to do is make a kthread_bind/set_cpus_allowed variant that checks against
cpu_online_maskinstead ofcpu_active_mask.It might even make sense to have
kthread_bind()always check withcpu_online_maskas the kernel really ought to know what its doing anyway.
Quoted 5 lines
You also don't need to play trickery with
PF_THREAD_BOUND, since a new kthread will not have that set.In fact, your patch is against a tree that still has
cpu_online_maskin all those places, so you wouldn't have needed any of that, confused.. ?!
More confusion, kthread_bind() doesn't even use set_cpus_allowed() so it wouldn't have hit any of that crap.
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 2 lines · Peter Zijlstra
More confusion,
kthread_bind()doesn't even useset_cpus_allowed()so it wouldn't have hit any of that crap.
kthread_bind() can't be used safely unless cpu up/down is synchronized out some way.
Thanks.
Signature · 1 line
tejun
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
In reply to Peter ZijlstraHello,
Quoted 5 lines · Peter Zijlstra
Quoted 4 lines · Tejun Heo
Quoted 2 lines
So its only needed in order to flush a workqueue from
CPU_DOWN_PREPARE? And all you need it to place a new kthread on a !active cpu?Yes, that's all I need.
Let me augment the above sentence. Yes, that's all I need *during CPU_DOWN*. During CPU_UP, I need to migrate back left running workers which survived from the last CPU_DOWN. In the original patch, the down path is worker_maybe_bind_and_lock() and the latter path is trustee_unset_rogue().
Quoted 5 lines
Then you don't need most of that patch, you don't need to touch the migration stuff since a new kthread isn't running.
All you need to do is make a kthread_bind/set_cpus_allowed variant that checks against
cpu_online_maskinstead ofcpu_active_mask.
kthread_bind() doesn't check against cpu_online_mask. Isn't set_cpus_allowed() variant which checks against cpu_online_mask is what's implemented by the patch (+ PF_THREAD_BOUND bypass)?
Quoted 3 lines
It might even make sense to have
kthread_bind()always check withcpu_online_maskas the kernel really ought to know what its doing anyway.
Oh... yeah, it was a bit strange that the function doesn't check against cpu onliness but if that is removed what would be the point of kthread_bind() when set_cpus_allowed() provides pretty much the same capability?
Quoted 2 lines
You also don't need to play trickery with
PF_THREAD_BOUND, since a new kthread will not have that set.
Yeap, but when the cpu comes back online, those kthreads need to be rebound to the cpu.
Quoted 2 lines
In fact, your patch is against a tree that still has
cpu_online_maskin all those places, so you wouldn't have needed any of that, confused.. ?!
To migrate back the workers from CPU_ONLINE callback.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 6 lines · Tejun Heo
Let me augment the above sentence. Yes, that's all I need *during CPU_DOWN*. During
CPU_UP, I need to migrate back left running workers which survived from the lastCPU_DOWN. In the original patch, the down path isworker_maybe_bind_and_lock()and the latter path istrustee_unset_rogue().
Why bother with that?
workqueue's CPU_POST_DEAD will flush the workqueue and destroy all threads under cpu_add_remove_lock, which excludes the cpu from coming back up before its fully destroyed.
So there's no remaining tasks to be migrated back.
Changing that semantics is not worthwhile.
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 9 lines · Peter Zijlstra
Why bother with that?
workqueue's
CPU_POST_DEADwill flush the workqueue and destroy all threads undercpu_add_remove_lock, which excludes the cpu from coming back up before its fully destroyed.So there's no remaining tasks to be migrated back.
Changing that semantics is not worthwhile.
It is worthwhile because the goal is to unify all worker pool mechanisms. So, slow_work or whatnot (scsi EHs, FS specific worker pools, async workers used for parallel probing) will all be converted to use workqueue instead and with that we can't afford to wait for all works to flush to down a cpu. All that's necessary to implement that is migrating back the unbound workers which can be implemented as a separate piece of code apart from regular operation. It would even benefit the current implementation as it makes cpu up/down operations much more deterministic.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 13 lines · Tejun Heo
Hello,
Quoted 9 lines · Peter Zijlstra
Why bother with that?
workqueue's
CPU_POST_DEADwill flush the workqueue and destroy all threads undercpu_add_remove_lock, which excludes the cpu from coming back up before its fully destroyed.So there's no remaining tasks to be migrated back.
Changing that semantics is not worthwhile.
Quoted 9 lines
It is worthwhile because the goal is to unify all worker pool mechanisms. So,
slow_workor whatnot (scsi EHs, FS specific worker pools, async workers used for parallel probing) will all be converted to use workqueue instead and with that we can't afford to wait for all works to flush to down a cpu. All that's necessary to implement that is migrating back the unbound workers which can be implemented as a separate piece of code apart from regular operation. It would even benefit the current implementation as it makes cpu up/down operations much more deterministic.
Hotplug and deterministic are not to be used in the same sentence, its an utter slow path and I'd much rather have simple code than clever code there -- there's been way too many 'interesting' hotplug problems.
If there is work being enqueued that takes more than a few seconds to complete then I'm thinking there's something seriously wrong and up to that point its perfectly fine to simply wait for it.
Furthermore if it's objective is to cater to generic thread pools then I think its an utter fail simply because it mandates strict cpu affinity, that basically requires you to write a work scheduler to balance work load etc.. Much easier is a simple unbounded thread pool that gets balanced by the regular scheduler.
/me liking this stuff less and less :/
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 3 lines · Peter Zijlstra
Hotplug and deterministic are not to be used in the same sentence, its an utter slow path and I'd much rather have simple code than clever code there -- there's been way too many 'interesting' hotplug problems.
Slow and indeterminism comes in different magnitudes.
Quoted 9 lines
If there is work being enqueued that takes more than a few seconds to complete then I'm thinking there's something seriously wrong and up to that point its perfectly fine to simply wait for it.
Furthermore if it's objective is to cater to generic thread pools then I think its an utter fail simply because it mandates strict cpu affinity, that basically requires you to write a work scheduler to balance work load etc.. Much easier is a simple unbounded thread pool that gets balanced by the regular scheduler.
The observation was that for most long running async jobs, most time is spent sleeping instead of burning cpu cycles and long running ones are relatively few compared to short ones so the strict affinity would be more helpful. That is the basis of whole design and why it has scheduler callbacks to regulate concurrency instead of creating a bunch of active workers and letting the scheduler take care of it. Works wouldn't be competing for cpu cycles.
In short, the target workload is the current short works + long running mostly sleeping async works, which cover most of worker pools we have in kernel. I thought about adding an unbound pool of workers for cpu intensive works for completeness but I really couldn't find much use for that. If enough number of users would need something like that, we can add an anonymous pool but for now I really don't see the need to worry about that.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 6 lines · Tejun Heo
Quoted 3 lines · Peter Zijlstra
Hotplug and deterministic are not to be used in the same sentence, its an utter slow path and I'd much rather have simple code than clever code there -- there's been way too many 'interesting' hotplug problems.
Slow and indeterminism comes in different magnitudes.
Determinism does _not_ come in magnitudes, its a very binary property, either it is or it is not.
As to the order of slowness for unplug, that is about maximal, its _the_ slowest path in the whole kernel.
Quoted 14 lines
Quoted 5 lines
Furthermore if it's objective is to cater to generic thread pools then I think its an utter fail simply because it mandates strict cpu affinity, that basically requires you to write a work scheduler to balance work load etc.. Much easier is a simple unbounded thread pool that gets balanced by the regular scheduler.
The observation was that for most long running async jobs, most time is spent sleeping instead of burning cpu cycles and long running ones are relatively few compared to short ones so the strict affinity would be more helpful. That is the basis of whole design and why it has scheduler callbacks to regulate concurrency instead of creating a bunch of active workers and letting the scheduler take care of it. Works wouldn't be competing for cpu cycles.
Quoted 3 lines
In short, the target workload is the current short works + long running mostly sleeping async works, which cover most of worker pools we have in kernel.
Ok, maybe, but that is not what I would call a generic thread pool.
So the reason we have tons of idle workqueues around are purely because of deadlock scenarios? Or is there other crap about?
So why not start simple and only have one thread per cpu (lets call it events/#) and run all works there. Then when you enqueue a work and events/# is already busy with a work from anther wq, hand the work to a global event thread which will spawn a special single shot kthread for it, with a second exception for those reclaim wq's, for which you'll have this rescue thread which you'll bind to the right cpu for that work.
That should get rid of all these gazillion threads we have, preserve the queue property and not be as invasive as your current thing.
If they're really as idle as reported you'll never need the fork-fest you currently propose, simply because there's not enough work.
So basically, have events/# service the first non-empty cwq, when there's more non empty cwqs spawn them single shot threads, or use a rescue thread.
Quoted 5 lines
I thought about adding an unbound pool of workers
for cpu intensive works for completeness but I really couldn't find much use for that. If enough number of users would need something like that, we can add an anonymous pool but for now I really don't see the need to worry about that.
And I though I'd heard multiple parties express interesting in exactly that, btrfs, bdi and pohmelfs come to mind, also crypto looks like one that could actually do some work.
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 4 lines · Peter Zijlstra
Quoted 1 line
Slow and indeterminism comes in different magnitudes.
Determinism does _not_ come in magnitudes, its a very binary property, either it is or it is not.
It's semantics, right? The whole process is made closer to being deterministic by making each component deterministic, so in that way "more deterministic" is a meaningful expression or it also can describe how deterministic it feels to human perception.
Quoted 2 lines
As to the order of slowness for unplug, that is about maximal, its _the_ slowest path in the whole kernel.
Long running works may run for minutes. They're slow to wait for as perceived by human beings, so we're talking about pretty different scales.
Quoted 1 line
Ok, maybe, but that is not what I would call a generic thread pool.
Sure, it's not completely generic, not yet anyway. The main focus is to solve concurrency issues with the current workqueue implementation. With concurrency issues solved, accomodating long running works becomes quite easy - all we need to do is to migrate back unbound workers during cup up - and as we have considerable number of users which require such usage, it's reasonable to implement it.
Quoted 2 lines
So the reason we have tons of idle workqueues around are purely because of deadlock scenarios? Or is there other crap about?
No, that's part of concurrency issues. Works don't devour and compete for CPU cycles but they overlap each other frequently. Works are often used to provide task context so that the users can use sleeping synchronization constructs or wait for events.
Quoted 10 lines
So why not start simple and only have one thread per cpu (lets call it events/#) and run all works there. Then when you enqueue a work and events/# is already busy with a work from anther wq, hand the work to a global event thread which will spawn a special single shot kthread for it, with a second exception for those reclaim wq's, for which you'll have this rescue thread which you'll bind to the right cpu for that work.
That should get rid of all these gazillion threads we have, preserve the queue property and not be as invasive as your current thing.
That doesn't solve concurrency problem at all and we'll be ending up bouncing around a LOT of works.
Quoted 6 lines
If they're really as idle as reported you'll never need the fork-fest you currently propose, simply because there's not enough work.
So basically, have events/# service the first non-empty cwq, when there's more non empty cwqs spawn them single shot threads, or use a rescue thread.
Idle doesn't mean they don't overlap. They aren't cpu cycle hungry but are context hungry.
It just isn't possible to implement shared worker pool which can scale according to necessary level of concurrency without doing some sort of dynamic worker management which necessarily involves forking new ones when in need and killing some of them off when there are more than enough.
As for the force_cpus_allowed() bit, I think it's a rather natural interface to have and maybe we can replace kthread_bind() with it or make kthread_bind() in terms of it. It's the basic migration function which adheres to the cpu hot plug/unplug synchronization rules.
Quoted 9 lines
Quoted 5 lines
I thought about adding an unbound pool of workers
for cpu intensive works for completeness but I really couldn't find much use for that. If enough number of users would need something like that, we can add an anonymous pool but for now I really don't see the need to worry about that.And I though I'd heard multiple parties express interesting in exactly that, btrfs, bdi and pohmelfs come to mind, also crypto looks like one that could actually do some work.
Yeah, anything cryptography related or crunches large chunk of data would be a good candidate but still compared to the works and kthreads we have just to have context and be able to wait for things, they're minority. Plus, it's something we can continue to work on if there are enough reasons to do so. If accomodating long running works there makes more sense, we'll do that but for most of them whether bound to certain cpu or not just isn't intersting at all and all we need to serve them is the ability to migrate back the threads during CPU UP. It's pretty isolated path.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 4/7] sched: implement force_cpus_allowed()
Quoted 4 lines · Tejun Heo
As for the
force_cpus_allowed()bit, I think it's a rather natural interface to have and maybe we can replacekthread_bind()with it or makekthread_bind()in terms of it. It's the basic migration function which adheres to the cpu hot plug/unplug synchronization rules.
I quite disagree, its quite unnatural to be adding threads to a cpu that is about to go down or about to come up for that matter, and we most certainly don't want to add to the hotplug rules, there are quite enough of them already.
You could always do it from CPU_ONLINE, since they don't care about cpu-affinity anyway (the thing was down for crying out loud), it really doesn't matter when they're moved back if at all.
I still think its utter insanity to even consider moving them back, or for that matter to have worklets that take minutes to complete, that's just daft.
I think I'm going to NAK all this, it looks quite ill conceived.
Re: [PATCH 4/7] sched: implement force_cpus_allowed()
Hello,
Quoted 10 lines · Peter Zijlstra
Quoted 4 lines · Tejun Heo
As for the
force_cpus_allowed()bit, I think it's a rather natural interface to have and maybe we can replacekthread_bind()with it or makekthread_bind()in terms of it. It's the basic migration function which adheres to the cpu hot plug/unplug synchronization rules.I quite disagree, its quite unnatural to be adding threads to a cpu that is about to go down or about to come up for that matter, and we most certainly don't want to add to the hotplug rules, there are quite enough of them already.
I don't think it adds to hotplug rules in any way. While cpu is going down, there's this single atomic moment when all threads are taken off the cpu and that's the natural synchronization point. You can add more restrictions on top of that like the cpu activeness test to avoid certain desirable behavior but in the end the natural ultimate sync point is the final migration off the cpu. That said, let's agree to disagree here. We're talking about what's being 'natural' and it depends largely on personal POVs, right?
Quoted 9 lines
You could always do it from
CPU_ONLINE, since they don't care about cpu-affinity anyway (the thing was down for crying out loud), it really doesn't matter when they're moved back if at all.I still think its utter insanity to even consider moving them back, or for that matter to have worklets that take minutes to complete, that's just daft.
I think I'm going to NAK all this, it looks quite ill conceived.
Alright, fair enough. So, I get that you NACK the whole concurrency managed workqueue thing. The reason why this part was split out and sent separately for scheduler tree was that because I thought there was general consensus toward having concurrency managed workqueues and its basic design. It looks like we'll need to have another round about that.
Here's what I'll do. I'm about done with the second round, properly split cmwq patches. It's feature complete and should be able to replace existing workqueue implementation without any problem (well, theoretically). I'll send them as a whole series with these scheduler patches and cc all the interested people and explain Ingo's concern about notification framework and your general objection to the whole thing. Let's continue there, okay?
Thanks.
Signature · 1 line
tejun
PATCH 5/7 sched: make sched_notifiers unconditional
In reply to Tejun Heosched_notifiers will be used by workqueue which is always there. Always enable sched_notifiers.
--- arch/ia64/kvm/Kconfig | 1 - arch/powerpc/kvm/Kconfig | 1 - arch/s390/kvm/Kconfig | 1 - arch/x86/kvm/Kconfig | 1 - include/linux/kvm_host.h | 2 -- include/linux/sched.h | 6 ------ init/Kconfig | 4 ---- kernel/sched.c | 13 ------------- 8 files changed, 0 insertions(+), 29 deletions(-)
arch/ia64/kvm/Kconfig +0 -1
@@ -22,7 +22,6 @@ config KVM depends on HAVE_KVM && MODULES && EXPERIMENTAL # for device assignment: depends on PCI - select SCHED_NOTIFIERS select ANON_INODES select HAVE_KVM_IRQCHIP select KVM_APIC_ARCHITECTURE
arch/powerpc/kvm/Kconfig +0 -1
@@ -18,7 +18,6 @@ if VIRTUALIZATION config KVM bool - select SCHED_NOTIFIERS select ANON_INODES config KVM_440
arch/s390/kvm/Kconfig +0 -1
@@ -18,7 +18,6 @@ if VIRTUALIZATION config KVM tristate "Kernel-based Virtual Machine (KVM) support" depends on HAVE_KVM && EXPERIMENTAL - select SCHED_NOTIFIERS select ANON_INODES select S390_SWITCH_AMODE ---help---
arch/x86/kvm/Kconfig +0 -1
@@ -22,7 +22,6 @@ config KVM depends on HAVE_KVM # for device assignment: depends on PCI - select SCHED_NOTIFIERS select MMU_NOTIFIER select ANON_INODES select HAVE_KVM_IRQCHIP
include/linux/kvm_host.h +0 -2
@@ -74,9 +74,7 @@ void kvm_io_bus_unregister_dev(struct kvm *kvm, struct kvm_io_bus *bus,struct kvm_vcpu {struct kvm *kvm; -#ifdef CONFIG_SCHED_NOTIFIERS struct sched_notifier sched_notifier; -#endif int vcpu_id;struct mutex mutex;int cpu;
include/linux/sched.h +0 -6
@@ -1210,8 +1210,6 @@ struct sched_rt_entity {#endif }; -#ifdef CONFIG_SCHED_NOTIFIERS -struct sched_notifier ;/**
@@ -1255,8 +1253,6 @@ static inline void sched_notifier_init(struct sched_notifier *notifier, notifier->ops = ops; } -#endif /* CONFIG_SCHED_NOTIFIERS */ -struct rcu_node ;struct task_struct {
@@ -1280,10 +1276,8 @@ struct task_struct {struct sched_entity se;struct sched_rt_entity rt; -#ifdef CONFIG_SCHED_NOTIFIERS /* list of struct sched_notifier: */ struct hlist_head sched_notifiers; -#endif /* * fpu_counter contains the number of consecutive context switches
init/Kconfig +0 -4
@@ -1206,7 +1206,3 @@ config STOP_MACHINE Need stop_machine() primitive. source "block/Kconfig" - -config SCHED_NOTIFIERS - bool -
kernel/sched.c +0 -13
@@ -1433,8 +1433,6 @@ static inline void cpuacct_update_stats(struct task_struct *tsk, enum cpuacct_stat_index idx, cputime_t val) {}#endif -#ifdef CONFIG_SCHED_NOTIFIERS - #define fire_sched_notifiers(p, callback, args...) do { \ struct sched_notifier *__sn; \ struct hlist_node *__pos; \
@@ -1465,12 +1463,6 @@ void sched_notifier_unregister(struct sched_notifier *notifier) }EXPORT_SYMBOL_GPL (sched_notifier_unregister); -#else /* !CONFIG_SCHED_NOTIFIERS */ - -#define fire_sched_notifiers(p, callback, args...) do { } while (0) - -#endif /* CONFIG_SCHED_NOTIFIERS */ -static inline void inc_cpu_load (struct rq *rq,unsigned long load) {update_load_add (&rq->load, load);
@@ -2592,10 +2584,7 @@ static void __sched_fork(struct task_struct *p)INIT_LIST_HEAD (&p->rt.run_list); p->se.on_rq =0 ;INIT_LIST_HEAD (&p->se.group_node); - -#ifdef CONFIG_SCHED_NOTIFIERS INIT_HLIST_HEAD (&p->sched_notifiers); -#endif /* * We mark the process as running here, but have not actually
@@ -9594,9 +9583,7 @@ void __init sched_init(void)set_load_weight (&init_task); -#ifdef CONFIG_SCHED_NOTIFIERS INIT_HLIST_HEAD (&init_task.sched_notifiers); -#endif #ifdef CONFIG_SMP open_softirq (SCHED_SOFTIRQ , run_rebalance_domains);
Signature · 1 line
1.6.4.2
PATCH 6/7 sched: add wakeup/sleep sched_notifiers and allow NULL notifier ops
In reply to Tejun HeoAdd wakeup and sleep notifiers to sched_notifiers and allow omitting some of the notifiers in the ops table. These will be used by concurrency managed workqueue.
--- include/linux/sched.h | 6 ++++++ kernel/sched.c | 11 ++++++++--- 2 files changed, 14 insertions(+), 3 deletions(-)
include/linux/sched.h +6 -0
@@ -1214,6 +1214,10 @@ struct sched_notifier;/** * sched_notifier_ops - notifiers called for scheduling events +* @wakeup: we're waking up +* notifier: struct sched_notifier for the task being woken up +* @sleep: we're going to bed +* notifier: struct sched_notifier for the task sleeping * @in: we're about to be rescheduled: * notifier: struct sched_notifier for the task being scheduled * cpu: cpu we're scheduled on
@@ -1227,6 +1231,8 @@ struct sched_notifier;* and depended upon by its users. */ struct sched_notifier_ops { +void (*wakeup)(struct sched_notifier *notifier); +void (*sleep)(struct sched_notifier *notifier);void (*in)(struct sched_notifier *notifier,int cpu);void (*out)(struct sched_notifier *notifier,struct task_struct *next); };
kernel/sched.c +8 -3
@@ -1438,7 +1438,8 @@ static inline void cpuacct_update_stats(struct task_struct *tsk,struct hlist_node *__pos; \ \ hlist_for_each_entry(__sn, __pos, &(p)->sched_notifiers, link) \ - __sn->ops->callback(__sn , ##args); \ +if (__sn->ops->callback) \ + __sn->ops->callback(__sn , ##args); \ }while (0 )/**
@@ -2409,6 +2410,8 @@ static inline void ttwu_woken_up(struct task_struct *p, struct rq *rq, rq->idle_stamp = 0; }#endif +if (success) +fire_sched_notifiers (p, wakeup); }/**
@@ -5431,10 +5434,12 @@ need_resched_nonpreemptible:clear_tsk_need_resched (prev);if (prev->state && !(preempt_count () &PREEMPT_ACTIVE )) { -if (unlikely (signal_pending_state (prev->state, prev))) +if (unlikely (signal_pending_state (prev->state, prev))) { prev->state =TASK_RUNNING ; -else + }else { +fire_sched_notifiers (prev, sleep);deactivate_task (rq, prev,1 ); + } switch_count = &prev->nvcsw; }
Signature · 1 line
1.6.4.2
PATCH 7/7 sched: implement try_to_wake_up_local()
In reply to Tejun HeoImplement try_to_wake_up_local(). try_to_wake_up_local() is similar to try_to_wake_up() but it assumes the caller has this_rq() locked and the target task is bound to this_rq(). try_to_wake_up_local() can be called from wakeup and sleep scheduler notifiers.
--- include/linux/sched.h | 2 + kernel/sched.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 0 deletions(-)
include/linux/sched.h +2 -0
@@ -2067,6 +2067,8 @@ extern void release_uids(struct user_namespace *ns);extern void do_timer (unsigned long ticks); +extern bool try_to_wake_up_local (struct task_struct *p,unsigned int state, +int wake_flags);extern int wake_up_state (struct task_struct *tsk,unsigned int state);extern int wake_up_process (struct task_struct *tsk); extern void wake_up_new_task(struct task_struct *tsk,
kernel/sched.c +56 -0
74 lines not on this page · raw message
Signature · 1 line
1.6.4.2
PATCH 7/7 sched: implement try_to_wake_up_local()
Implement try_to_wake_up_local(). try_to_wake_up_local() is similar to try_to_wake_up() but it assumes the caller has this_rq() locked and the target task is bound to this_rq(). try_to_wake_up_local() can be called from wakeup and sleep scheduler notifiers.
---
The only change is to accomodate function name change of earlier patch.
Updated tree available on top of current sched/core available in the following git trees.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git sched-core-for-ingo
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git sched-wq-for-ingo
Thanks.
include/linux/sched.h | 2 + kernel/sched.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+)
include/linux/sched.h +2 -0
@@ -2067,6 +2067,8 @@ extern void release_uids(struct user_namextern void do_timer (unsigned long ticks); +extern bool try_to_wake_up_local (struct task_struct *p,unsigned int state, +int wake_flags);extern int wake_up_state (struct task_struct *tsk,unsigned int state);extern int wake_up_process (struct task_struct *tsk); extern void wake_up_new_task(struct task_struct *tsk,
kernel/sched.c +56 -0
74 lines not on this page · raw message
Peter Zijlstra
maintainerRe: [PATCH UPDATED 7/7] sched: implement try_to_wake_up_local()
Quoted 5 lines · Tejun Heo
Updated tree available on top of current sched/core available in the following git trees.
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.gitsched-core-for-ingo
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.gitsched-wq-for-ingo
Would you have that in quilt format? git sucks for reviewing.
Re: [PATCH UPDATED 7/7] sched: implement try_to_wake_up_local()
Hello,
Quoted 1 line · Peter Zijlstra
Would you have that in quilt format? git sucks for reviewing.
Here it is. I'll post full series again after this review pass is complete.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 7/7] sched: implement try_to_wake_up_local()
In reply to Tejun HeoQuoted 15 lines · Tejun Heo
+++
b/kernel/sched.ckernel/sched.c +4
-0@@ -2410,6 +2410,10 @@ static inline void ttwu_woken_up(struct task_struct *p, struct rq *rq, rq->idle_stamp = 0; }#endif +/* +* Wake up is complete, fire wake up notifier. This allows +* try_to_wake_up_local() to be called from wake up notifiers. +*/ if (success)fire_sched_notifiers (p, wakeup); }@@ -5437,6 +5488,11 @@ need_resched_nonpreemptible:if (unlikely (signal_pending_state (prev->state, prev))) {
kernel/sched.c need_resched_nonpreemptible: · line 5498
Quoted 10 lines
kernel/sched.c +5
-0prev->state =TASK_RUNNING ; }else { +/* +* Fire sleep notifier before changing any scheduler +* state. This allows try_to_wake_up_local() to be +* called from sleep notifiers. +*/ fire_sched_notifiers (prev, sleep);deactivate_task (rq, prev,1 ); }
These two hunks seem to belong to patch 6
Re: [PATCH 7/7] sched: implement try_to_wake_up_local()
Hello,
Quoted 15 lines · Peter Zijlstra
Quoted 12 lines · Tejun Heo
+++
b/kernel/sched.ckernel/sched.c +4
-0@@ -2410,6 +2410,10 @@ static inline void ttwu_woken_up(struct task_struct *p, struct rq *rq, rq->idle_stamp = 0; }#endif +/* +* Wake up is complete, fire wake up notifier. This allows +* try_to_wake_up_local() to be called from wake up notifiers. +*/ if (success)fire_sched_notifiers (p, wakeup); }
Quoted 14 lines
kernel/sched.c need_resched_nonpreemptible: · line 5498
Quoted 12 lines
kernel/sched.c +5
-0@@ -5437,6 +5488,11 @@ need_resched_nonpreemptible:if (unlikely (signal_pending_state (prev->state, prev))) { prev->state =TASK_RUNNING ; }else { +/* +* Fire sleep notifier before changing any scheduler +* state. This allows try_to_wake_up_local() to be +* called from sleep notifiers. +*/ fire_sched_notifiers (prev, sleep);deactivate_task (rq, prev,1 ); }These two hunks seem to belong to patch 6
Hmmm... it was intentional as, before this patch, there's no try_to_wake_up_local() so it was strange to mention it in the comment. I can move the comments but I don't think it's particularly better that way.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 7/7] sched: implement try_to_wake_up_local()
Quoted 33 lines · Tejun Heo
Hello,
Quoted 15 lines · Peter Zijlstra
Quoted 12 lines · Tejun Heo
+++
b/kernel/sched.ckernel/sched.c +4
-0@@ -2410,6 +2410,10 @@ static inline void ttwu_woken_up(struct task_struct *p, struct rq *rq, rq->idle_stamp = 0; }#endif +/* +* Wake up is complete, fire wake up notifier. This allows +* try_to_wake_up_local() to be called from wake up notifiers. +*/ if (success)fire_sched_notifiers (p, wakeup); }Quoted 14 lines
kernel/sched.c need_resched_nonpreemptible: · line 5498
Quoted 12 lines
kernel/sched.c +5
-0@@ -5437,6 +5488,11 @@ need_resched_nonpreemptible:if (unlikely (signal_pending_state (prev->state, prev))) { prev->state =TASK_RUNNING ; }else { +/* +* Fire sleep notifier before changing any scheduler +* state. This allows try_to_wake_up_local() to be +* called from sleep notifiers. +*/ fire_sched_notifiers (prev, sleep);deactivate_task (rq, prev,1 ); }These two hunks seem to belong to patch 6
Quoted 4 lines
Hmmm... it was intentional as, before this patch, there's no
try_to_wake_up_local()so it was strange to mention it in the comment. I can move the comments but I don't think it's particularly better that way.
/me reads the comments and goes ah!
OK, maybe you've got a point there ;-)
Peter Zijlstra
maintainerRe: [PATCH 7/7] sched: implement try_to_wake_up_local()
Quoted 10 lines · Peter Zijlstra
Quoted 4 lines · Tejun Heo
Hmmm... it was intentional as, before this patch, there's no
try_to_wake_up_local()so it was strange to mention it in the comment. I can move the comments but I don't think it's particularly better that way./me reads the comments and goes ah!
OK, maybe you've got a point there ;-)
OK, so you fork and wakeup a new thread when an existing one goes to sleep, but do you also limit the concurrency on wakeup?
Otherwise we can end up with say 100 workqueue tasks running, simply because they all ran into a contended lock and then woke up again.
Where does that fork happen? Having to do memory allocations and all that while holding the rq->lock doesn't seem like a very good idea.
What happens when you run out of memory and the workqueue progress is needed to get free memory?
Re: [PATCH 7/7] sched: implement try_to_wake_up_local()
Hello,
Quoted 5 lines · Peter Zijlstra
OK, so you fork and wakeup a new thread when an existing one goes to sleep, but do you also limit the concurrency on wakeup?
Otherwise we can end up with say 100 workqueue tasks running, simply because they all ran into a contended lock and then woke up again.
Yes, the total number of workers is limited.
Quoted 2 lines
Where does that fork happen? Having to do memory allocations and all that while holding the
rq->lockdoesn't seem like a very good idea.
There always is guaranteed to be at least one idle worker and when the last running worker goes to sleep, one idle worker is woken up which assumes the role of manager which if necessary forks another worker, so all that happens under rq->lock is waking up another worker.
Quoted 2 lines
What happens when you run out of memory and the workqueue progress is needed to get free memory?
That's when the "rescuer" workers kick in. All workqueues which might be used during allocation paths are required to have one. When manager fails to fork a new worker, it summons the rescuers. They migrate to the distressed cpu and processes only the works from their specific workqueues thus guaranteeing forward progress.
Thanks.
Signature · 1 line
tejun
Peter Zijlstra
maintainerRe: [PATCH 7/7] sched: implement try_to_wake_up_local()
Quoted 9 lines · Tejun Heo
Quoted 3 lines
What happens when you run out of memory and the workqueue progress is needed to get free memory?
That's when the "rescuer" workers kick in. All workqueues which might be used during allocation paths are required to have one. When manager fails to fork a new worker, it summons the rescuers. They migrate to the distressed cpu and processes only the works from their specific workqueues thus guaranteeing forward progress.
What happens if the rescue worker is tied up on another cpu in a deadlock avoidance game, which requires progress on this cpu in order to resolve it?
Re: [PATCH 7/7] sched: implement try_to_wake_up_local()
Hello,
Quoted 11 lines · Peter Zijlstra
Quoted 9 lines · Tejun Heo
Quoted 3 lines
What happens when you run out of memory and the workqueue progress is needed to get free memory?
That's when the "rescuer" workers kick in. All workqueues which might be used during allocation paths are required to have one. When manager fails to fork a new worker, it summons the rescuers. They migrate to the distressed cpu and processes only the works from their specific workqueues thus guaranteeing forward progress.
Quoted 3 lines
What happens if the rescue worker is tied up on another cpu in a deadlock avoidance game, which requires progress on this cpu in order to resolve it?
Hmmm.... I think one rescuer thread per a workqueue is enough to guarantee forward-progress. If there are workqueues which have inter-cpu dependency, we'll probably need per-cpu rescuers but is there any?
Thanks.
Signature · 1 line
tejun