Skip to contentSkip to conversation

sched: prepare for cmwq

PATCH In review Tejun Heo 3 participants 29 messages May 13, 2010 – May 31, 2010

Subsystems (3)
lore
Share & tools
lore

Tejun Heo

author
#0 lore raw

PATCH 1/1 sched: prepare for cmwq

Hello, Ingo, Peter.

These four patches are the scheduler modifications necessary for cmwq and contains the following four patches.

0001-sched-consult-online-mask-instead-of-active-in-selec.patch 0002-sched-implement-__set_cpus_allowed.patch 0003-sched-refactor-try_to_wake_up.patch 0004-sched-add-hooks-for-workqueue.patch

The first three have been posted multiple times as part of cmwq patchset multiple times. Other than description updates to explain why they are needed, these three patches haven't changed from the last posting[L].

The last patch adds two hardcoded workqueue hooks as suggested in in the unify-tracers-in-sched thread[U]. These new hardcoded hooks are much simpler and no scheduler behavior change is leaked outside of sched.c. Workqueue workers are marked with PF_WQ_WORKER and hooks are called only for them. Instead of exporting try_to_wake_up_local(), the sleeping hook now returns a pointer to local task which is woken up by scheduler. The hooks are defined as noop for now and will be filled later by cmwq implementation.

The change from sched_notifier to the hardcoded hooks doesn't make any notable difference to cmwq implementation itself.

This patchset is available in the following git tree.

git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git sched-wq

and contains the following changes.

 include/linux/sched.h    |   15 ++-
 kernel/fork.c            |    2 
 kernel/sched.c           |  191 +++++++++++++++++++++++++++++++++--------------
 kernel/workqueue_sched.h |   16 +++
 4 files changed, 163 insertions(+), 61 deletions(-)

Thanks.

--
tejun

[L] http://thread.gmane.org/gmane.linux.kernel/954759
[U] http://thread.gmane.org/gmane.linux.kernel/980857

author
#1 lore raw

PATCH 1/4 sched: consult online mask instead of active in select_fallback_rq()

If called after sched_class chooses a CPU which isn't in a task's cpus_allowed mask, select_fallback_rq() can end up migrating a task which is bound to an !active but online cpu to an active cpu. This is dangerous because active is cleared before CPU_DOWN_PREPARE is called and subsystems expect affinities of kthreads and other tasks to be maintained till their CPU_DOWN_PREPARE callbacks are complete.

Consult cpu_online_mask instead.

This problem is triggered by cmwq. During CPU_DOWN_PREPARE, hotplug callback creates the trustee kthread and kthread_bind()s it to the target cpu, and the trustee is expected to run on that cpu.

Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 kernel/sched.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
kernel/sched.c +2 -2
index b531d79..aca4a20 100644
@@ -2262,12 +2262,12 @@ static int select_fallback_rq(int cpu, struct task_struct *p)
 	const struct cpumask *nodemask = cpumask_of_node(cpu_to_node(cpu));
 
 	/* Look for allowed, online CPU in same node. */
-	for_each_cpu_and(dest_cpu, nodemask, cpu_active_mask)
+	for_each_cpu_and(dest_cpu, nodemask, cpu_online_mask)
 		if (cpumask_test_cpu(dest_cpu, &p->cpus_allowed))
 			return dest_cpu;
 
 	/* Any allowed, online CPU? */
-	dest_cpu = cpumask_any_and(&p->cpus_allowed, cpu_active_mask);
+	dest_cpu = cpumask_any_and(&p->cpus_allowed, cpu_online_mask);
 	if (dest_cpu < nr_cpu_ids)
 		return dest_cpu;
 
Signature · 1 line
1.6.4.2
maintainer
#2 lore raw

Re: [PATCH 1/4] sched: consult online mask instead of active in select_fallback_rq()

Quoted 6 lines · Tejun Heo

If called after sched_class chooses a CPU which isn't in a task's cpus_allowed mask, select_fallback_rq() can end up migrating a task which is bound to an !active but online cpu to an active cpu. This is dangerous because active is cleared before CPU_DOWN_PREPARE is called and subsystems expect affinities of kthreads and other tasks to be maintained till their CPU_DOWN_PREPARE callbacks are complete.

So 6ad4c188 (sched: Fix balance vs hotplug race) moved it that early because it was done too late.

Could we not instead do it explicitly after CPU_DOWN_PREPARE? It would of course mean removing the partition_sched_domain() and generate_sched_domains() calls from these callbacks and doing it explicitly.

So we need to do it before we take the CPU down, but I think we can do it after DOWN_PREPARE.

Quoted 5 lines

Consult cpu_online_mask instead.

This problem is triggered by cmwq. During CPU_DOWN_PREPARE, hotplug callback creates the trustee kthread and kthread_bind()s it to the target cpu, and the trustee is expected to run on that cpu.

It doesn't explain wth a trustee kthread is, which pretty much renders the whole paragraph useless.

author
#3 lore raw

Re: [PATCH 1/4] sched: consult online mask instead of active in select_fallback_rq()

Hello,

Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · Tejun Heo

If called after sched_class chooses a CPU which isn't in a task's cpus_allowed mask, select_fallback_rq() can end up migrating a task which is bound to an !active but online cpu to an active cpu. This is dangerous because active is cleared before CPU_DOWN_PREPARE is called and subsystems expect affinities of kthreads and other tasks to be maintained till their CPU_DOWN_PREPARE callbacks are complete.

So 6ad4c188 (sched: Fix balance vs hotplug race) moved it that early because it was done too late.

Quoted 7 lines

Could we not instead do it explicitly after CPU_DOWN_PREPARE? It would of course mean removing the partition_sched_domain() and generate_sched_domains() calls from these callbacks and doing it explicitly.

So we need to do it before we take the CPU down, but I think we can do it after DOWN_PREPARE.

Hmm.... yeah, I suppose that would be cleaner. Maybe doing it from the lowest priority DOWN_PREPARE notifier would do the trick. I'll give it a shot.

Quoted 8 lines
Quoted 5 lines

Consult cpu_online_mask instead.

This problem is triggered by cmwq. During CPU_DOWN_PREPARE, hotplug callback creates the trustee kthread and kthread_bind()s it to the target cpu, and the trustee is expected to run on that cpu.

It doesn't explain wth a trustee kthread is, which pretty much renders the whole paragraph useless.

Come on. The paragraph makes perfect sense even if you remove the words "trustee" completely. It's saying that it's creating a kthread CPU_DOWN_PREPARE and binds it to the cpu and expects it to stay there. If you're not completely comfortable with the use of "trustee" without proper introduction, I'll be happy to remove it, but limited amount of forward reference is helpful when searching the history backwards.

Thanks.

Signature · 1 line
tejun
author
#4 lore raw

PATCH 2/4 sched: implement __set_cpus_allowed()

In reply to Tejun Heo

Concurrency managed workqueue needs to be able to migrate tasks to a cpu which is online but !active for the following two purposes.

p1. To guarantee forward progress during cpu down sequence. Each
workqueue which could be depended upon during memory allocation has an emergency worker task which is summoned when a pending work on such workqueue can't be serviced immediately. cpu hotplug callbacks expect workqueues to work during cpu down sequence (usually so that they can flush them), so, to guarantee forward progress, it should be possible to summon emergency workers to !active but online cpus.

p2. To migrate back unbound workers when a cpu comes back online.
When a cpu goes down, existing workers are unbound from the cpu and allowed to run on other cpus if there still are pending or running works. If the cpu comes back online while those workers are still around, those workers are migrated back and re-bound to the cpu. This isn't strictly required for correctness as long as those unbound workers don't execute works which are newly scheduled after the cpu comes back online; however, migrating back the workers has the advantage of making the behavior more consistent thus avoiding surprises which are difficult to expect and reproduce, and being actually cleaner and easier to implement.

To implement this, __set_cpus_allowed() is factored out from set_cpus_allowed_ptr() and @force parameter is added to it. The latter is now a wrapper around the former with @force set to %false. When @force is %false, the following original behaviors are maintained.

c1. Check whether PF_THREAD_BOUND is set. This is set for bound
kthreads so that they can't be moved around.

c2. Check whether the target cpu is still marked active -
cpu_active(). Active state is cleared early while downing a cpu.

When @force parameter is %true, __set_cpus_allowed() ignores c1 and uses cpu online state instead of active state for c2.

Due to the way migration is implemented, the @force parameter needs to be passed over to the migration cpu_stop callback. @force parameter is added to struct migration_req and passed to __migrate_task().

Please note the naming discrepancy between set_cpus_allowed_ptr() and the new functions. The _ptr suffix is from the days when cpumask API wasn't mature and future changes should drop it from set_cpus_allowed_ptr() too.

Cc Rusty Russell <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 include/linux/sched.h |   14 +++++++++---
 kernel/sched.c        |   51 +++++++++++++++++++++++++++++++-----------------
 2 files changed, 43 insertions(+), 22 deletions(-)
include/linux/sched.h +10 -4
index dfea405..ef6067c 100644
@@ -1776,11 +1776,11 @@ static inline void rcu_copy_process(struct task_struct *p)
 #endif
 
 #ifdef CONFIG_SMP
-extern int set_cpus_allowed_ptr(struct task_struct *p,
-				const struct cpumask *new_mask);
+extern int __set_cpus_allowed(struct task_struct *p,
+			      const struct cpumask *new_mask, bool force);
 #else
-static inline int set_cpus_allowed_ptr(struct task_struct *p,
-				       const struct cpumask *new_mask)
+static inline int __set_cpus_allowed(struct task_struct *p,
+				     const struct cpumask *new_mask, bool force)
 {
 	if (!cpumask_test_cpu(0, new_mask))
 		return -EINVAL;
@@ -1788,6 +1788,12 @@ static inline int set_cpus_allowed_ptr(struct task_struct *p,
 }
 #endif
 
+static inline int set_cpus_allowed_ptr(struct task_struct *p,
+				       const struct cpumask *new_mask)
+{
+	return __set_cpus_allowed(p, new_mask, false);
+}
+
 #ifndef CONFIG_CPUMASK_OFFSTACK
 static inline int set_cpus_allowed(struct task_struct *p, cpumask_t new_mask)
 {
kernel/sched.c +33 -18

117 lines not on this page · raw message

Signature · 1 line
1.6.4.2
maintainer
#5 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Quoted 11 lines · Tejun Heo

Concurrency managed workqueue needs to be able to migrate tasks to a cpu which is online but !active for the following two purposes.

p1. To guarantee forward progress during cpu down sequence. Each
workqueue which could be depended upon during memory allocation has an emergency worker task which is summoned when a pending work on such workqueue can't be serviced immediately. cpu hotplug callbacks expect workqueues to work during cpu down sequence (usually so that they can flush them), so, to guarantee forward progress, it should be possible to summon emergency workers to !active but online cpus.

If we do the thing suggested in the previous patch, that is move clearing active and rebuilding the sched domains until right after DOWN_PREPARE, this goes away, right?

Quoted 11 lines

p2. To migrate back unbound workers when a cpu comes back online.
When a cpu goes down, existing workers are unbound from the cpu and allowed to run on other cpus if there still are pending or running works. If the cpu comes back online while those workers are still around, those workers are migrated back and re-bound to the cpu. This isn't strictly required for correctness as long as those unbound workers don't execute works which are newly scheduled after the cpu comes back online; however, migrating back the workers has the advantage of making the behavior more consistent thus avoiding surprises which are difficult to expect and reproduce, and being actually cleaner and easier to implement.

I still don't like this much, if you mark these tasks to simply die when the queue is exhausted, and flush the queue explicitly on CPU_UP_PREPARE, you should never need to do this.

After which I think you don't need this patch anymore..

author
#6 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Hello,

Quoted 13 lines · Peter Zijlstra
Quoted 11 lines · Tejun Heo

Concurrency managed workqueue needs to be able to migrate tasks to a cpu which is online but !active for the following two purposes.

p1. To guarantee forward progress during cpu down sequence. Each
workqueue which could be depended upon during memory allocation has an emergency worker task which is summoned when a pending work on such workqueue can't be serviced immediately. cpu hotplug callbacks expect workqueues to work during cpu down sequence (usually so that they can flush them), so, to guarantee forward progress, it should be possible to summon emergency workers to !active but online cpus.

Quoted 3 lines

If we do the thing suggested in the previous patch, that is move clearing active and rebuilding the sched domains until right after DOWN_PREPARE, this goes away, right?

Hmmm... yeah, if the usual set_cpus_allowed_ptr() keeps working throughout CPU_DOWN_PREPARE, this probably goes away. I'll give it a shot.

Quoted 12 lines
Quoted 11 lines

p2. To migrate back unbound workers when a cpu comes back online.
When a cpu goes down, existing workers are unbound from the cpu and allowed to run on other cpus if there still are pending or running works. If the cpu comes back online while those workers are still around, those workers are migrated back and re-bound to the cpu. This isn't strictly required for correctness as long as those unbound workers don't execute works which are newly scheduled after the cpu comes back online; however, migrating back the workers has the advantage of making the behavior more consistent thus avoiding surprises which are difficult to expect and reproduce, and being actually cleaner and easier to implement.

Quoted 3 lines

I still don't like this much, if you mark these tasks to simply die when the queue is exhausted, and flush the queue explicitly on CPU_UP_PREPARE, you should never need to do this.

I don't think flushing from CPU_UP_PREPARE would be a good idea. There is no guarantee that those works will finish in short (human scale) time, but we can update cpu_active mask before other CPU_UP_PREPARE notifiers are executed so that it's symmetrical to cpu down path and then this problem goes away the exact same way, right?

Thanks.

Signature · 1 line
tejun
maintainer
#7 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Quoted 3 lines · Tejun Heo

I don't think flushing from CPU_UP_PREPARE would be a good idea. There is no guarantee that those works will finish in short (human scale) time,

If not, we should cure it by ensuring these works won't be left lingering I think.

Also, I really don't much care about how fast we can hotplug cycle things -- its an utter slow path.

Quoted 3 lines

but we can update cpu_active mask before other
CPU_UP_PREPARE notifiers are executed so that it's symmetrical to cpu down path and then this problem goes away the exact same way, right?

Ah, no, we cannot mark it active before its actually up, because at that time we'll actually try and run stuff on it, which clearly won't work when its not there to run stuff.

maintainer
#8 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

but we can update cpu_active mask before other
CPU_UP_PREPARE notifiers are executed so that it's symmetrical to cpu down path and then this problem goes away the exact same way, right?

Ah, no, we cannot mark it active before its actually up, because at that time we'll actually try and run stuff on it, which clearly won't work when its not there to run stuff.

So we should clear it _before_ we go down, and set it _after_ we're up.

author
#9 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Quoted 11 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

but we can update cpu_active mask before other
CPU_UP_PREPARE notifiers are executed so that it's symmetrical to cpu down path and then this problem goes away the exact same way, right?

Ah, no, we cannot mark it active before its actually up, because at that time we'll actually try and run stuff on it, which clearly won't work when its not there to run stuff.

So we should clear it _before_ we go down, and set it _after_ we're up.

Yeah, sure, I misspoke. I meant CPU_ONLINE not CPU_UP_PREPARE. So, we can mark a cpu active before other CPU_ONLINE callbacks are run.

Thanks.

Signature · 1 line
tejun
maintainer
#10 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Quoted 2 lines · Tejun Heo

I meant CPU_ONLINE not CPU_UP_PREPARE. So,
we can mark a cpu active before other CPU_ONLINE callbacks are run.

OK, that should work.

author
#11 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Hello,

Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Tejun Heo

I meant CPU_ONLINE not CPU_UP_PREPARE. So,
we can mark a cpu active before other CPU_ONLINE callbacks are run.

OK, that should work.

The only remaining problemw is PF_THREAD_BOUND. It needs to be set for wq workers (including rescuers). Would it be okay to add an interface which only ignores PF_THREAD_BOUND?

Thanks.

Signature · 1 line
tejun
maintainer
#12 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Quoted 4 lines · Tejun Heo

Hello,

Quoted 1 line · Peter Zijlstra

On Mon, 2010-05-31 at 12:06 +0200, Tejun Heo wrote:

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

I meant CPU_ONLINE not CPU_UP_PREPARE. So,
we can mark a cpu active before other CPU_ONLINE callbacks are run.

OK, that should work.

The only remaining problemw is PF_THREAD_BOUND. It needs to be set for wq workers (including rescuers). Would it be okay to add an interface which only ignores PF_THREAD_BOUND?

So workers _are_ thread bound and don't migrate, so for those I don't see the problem, that is of course until we have to break affinity when their CPU goes down, at which point we should clear PF_THREAD_BOUND I think.

For the rescue thread, why not set PF_THREAD_BOUND when its running worklets and clear it when done? That way we get extra checking that people won't migrate it when its not allowed.

Does that work, or did I miss something?

author
#13 lore raw

Re: [PATCH 2/4] sched: implement __set_cpus_allowed()

Hello,

Quoted 10 lines · Peter Zijlstra

So workers _are_ thread bound and don't migrate, so for those I don't see the problem, that is of course until we have to break affinity when their CPU goes down, at which point we should clear PF_THREAD_BOUND I think.

For the rescue thread, why not set PF_THREAD_BOUND when its running worklets and clear it when done? That way we get extra checking that people won't migrate it when its not allowed.

Does that work, or did I miss something?

The rescuers migrate themselves so should be okay. I still think it would be better to migrate back detached workers. It's a slow path for sure but there's a big gaping hole between machine-slow and human-slow and works can be human-slow (ie. hotplug operation not finished in several minutes). I'll see if that's implementable w/o adding an additional interface.

Thanks.

Signature · 1 line
tejun
author
#14 lore raw

PATCH 3/4 sched: refactor try_to_wake_up()

In reply to Tejun Heo

Factor ttwu_activate() and ttwu_woken_up() out of try_to_wake_up(). The factoring out doesn't affect try_to_wake_up() much code-generation-wise. Depending on configuration options, it ends up generating the same object code as before or slightly different one due to different register assignment.

This is to help future implementation of try_to_wake_up_local().

Mike Galbraith suggested rename to ttwu_post_activation() from ttwu_woken_up() and comment update in try_to_wake_up().

Cc Peter Zijlstra <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 kernel/sched.c |   83 +++++++++++++++++++++++++++++++++-----------------------
 1 files changed, 49 insertions(+), 34 deletions(-)
kernel/sched.c +49 -34
index ecf024d..0b6da84 100644
@@ -2322,11 +2322,52 @@ static void update_avg(u64 *avg, u64 sample)
 }
 #endif
 
-/***
+static inline void ttwu_activate(struct task_struct *p, struct rq *rq,
+				 bool is_sync, bool is_migrate, bool is_local,
+				 unsigned long en_flags)
+{
+	schedstat_inc(p, se.statistics.nr_wakeups);
+	if (is_sync)
+		schedstat_inc(p, se.statistics.nr_wakeups_sync);
+	if (is_migrate)
+		schedstat_inc(p, se.statistics.nr_wakeups_migrate);
+	if (is_local)
+		schedstat_inc(p, se.statistics.nr_wakeups_local);
+	else
+		schedstat_inc(p, se.statistics.nr_wakeups_remote);
+
+	activate_task(rq, p, en_flags);
+}
+
+static inline void ttwu_post_activation(struct task_struct *p, struct rq *rq,
+					int wake_flags, bool success)
+{
+	trace_sched_wakeup(p, success);
+	check_preempt_curr(rq, p, wake_flags);
+
+	p->state = TASK_RUNNING;
+#ifdef CONFIG_SMP
+	if (p->sched_class->task_woken)
+		p->sched_class->task_woken(rq, p);
+
+	if (unlikely(rq->idle_stamp)) {
+		u64 delta = rq->clock - rq->idle_stamp;
+		u64 max = 2*sysctl_sched_migration_cost;
+
+		if (delta > max)
+			rq->avg_idle = max;
+		else
+			update_avg(&rq->avg_idle, delta);
+		rq->idle_stamp = 0;
+	}
+#endif
+}
+
+/**
  * try_to_wake_up - wake up a thread
- * @p: the to-be-woken-up thread
+ * @p: the thread to be awakened
  * @state: the mask of task states that can be woken
- * @sync: do a synchronous wakeup?
+ * @wake_flags: wake modifier flags (WF_*)
  *
  * Put it on the run-queue if it's not already there. The "current"
  * thread is always on the run-queue (except when the actual
@@ -2334,7 +2375,8 @@ static void update_avg(u64 *avg, u64 sample)
  * the simpler "current->state = TASK_RUNNING" to mark yourself
  * runnable without the overhead of this.
  *
- * returns failure only if the task is already active.
+ * Returns %true if @p was woken up, %false if it was already running
+ * or @state didn't match @p's state.
  */
 static int try_to_wake_up(struct task_struct *p, unsigned int state,
 			  int wake_flags)
@@ -2414,38 +2456,11 @@ static int try_to_wake_up(struct task_struct *p, unsigned int state,
 
 out_activate:
 #endif /* CONFIG_SMP */
-	schedstat_inc(p, se.statistics.nr_wakeups);
-	if (wake_flags & WF_SYNC)
-		schedstat_inc(p, se.statistics.nr_wakeups_sync);
-	if (orig_cpu != cpu)
-		schedstat_inc(p, se.statistics.nr_wakeups_migrate);
-	if (cpu == this_cpu)
-		schedstat_inc(p, se.statistics.nr_wakeups_local);
-	else
-		schedstat_inc(p, se.statistics.nr_wakeups_remote);
-	activate_task(rq, p, en_flags);
+	ttwu_activate(p, rq, wake_flags & WF_SYNC, orig_cpu != cpu,
+		      cpu == this_cpu, en_flags);
 	success = 1;
-
 out_running:
-	trace_sched_wakeup(p, success);
-	check_preempt_curr(rq, p, wake_flags);
-
-	p->state = TASK_RUNNING;
-#ifdef CONFIG_SMP
-	if (p->sched_class->task_woken)
-		p->sched_class->task_woken(rq, p);
-
-	if (unlikely(rq->idle_stamp)) {
-		u64 delta = rq->clock - rq->idle_stamp;
-		u64 max = 2*sysctl_sched_migration_cost;
-
-		if (delta > max)
-			rq->avg_idle = max;
-		else
-			update_avg(&rq->avg_idle, delta);
-		rq->idle_stamp = 0;
-	}
-#endif
+	ttwu_post_activation(p, rq, wake_flags, success);
 out:
 	task_rq_unlock(rq, &flags);
 	put_cpu();
Signature · 1 line
1.6.4.2
author
#15 lore raw

PATCH 4/4 sched: add hooks for workqueue

In reply to Tejun Heo

Concurrency managed workqueue needs to know when workers are going to sleep and waking up, and, when a worker goes to sleep, be able to wake up another worker to maintain adequate concurrency. This patch introduces PF_WQ_WORKER to identify workqueue workers and adds the following two hooks.

* wq_worker_waking_up(): called when a worker is woken up.

* wq_worker_sleeping(): called when a worker is going to sleep and may return a pointer to a local task which should be woken up. The returned task is woken up using try_to_wake_up_local() which is simplified ttwu which is called under rq lock and can only wake up local tasks.

Both hooks are currently defined as noop in kernel/workqueue_sched.h. Later cmwq implementation will replace them with proper implementation.

These hooks are hard coded as they'll always be enabled.

Cc Peter Zijlstra <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Ingo Molnar <[email protected]>
---
 include/linux/sched.h    |    1 +
 kernel/fork.c            |    2 +-
 kernel/sched.c           |   53 ++++++++++++++++++++++++++++++++++++++++++++-
 kernel/workqueue_sched.h |   16 +++++++++++++
 4 files changed, 69 insertions(+), 3 deletions(-)
 create mode 100644 kernel/workqueue_sched.h
include/linux/sched.h +1 -0
index ef6067c..553ffae 100644
@@ -1702,6 +1702,7 @@ extern void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t *
 #define PF_EXITING	0x00000004	/* getting shut down */
 #define PF_EXITPIDONE	0x00000008	/* pi exit done on shut down */
 #define PF_VCPU		0x00000010	/* I'm a virtual CPU */
+#define PF_WQ_WORKER	0x00000020	/* I'm a workqueue worker */
 #define PF_FORKNOEXEC	0x00000040	/* forked but didn't exec */
 #define PF_MCE_PROCESS  0x00000080      /* process policy on mce errors */
 #define PF_SUPERPRIV	0x00000100	/* used super-user privileges */
kernel/fork.c +1 -1
index 44b0791..1440f90 100644
@@ -900,7 +900,7 @@ static void copy_flags(unsigned long clone_flags, struct task_struct *p)
 {
 	unsigned long new_flags = p->flags;
 
-	new_flags &= ~PF_SUPERPRIV;
+	new_flags &= ~(PF_SUPERPRIV | PF_WQ_WORKER);
 	new_flags |= PF_FORKNOEXEC;
 	new_flags |= PF_STARTING;
 	p->flags = new_flags;
kernel/sched.c +51 -2

79 lines not on this page · raw message

kernel/workqueue_sched.h new file +16 -0
new file mode 100644
index 0000000..af040ba
@@ -0,0 +1,16 @@
+/*
+ * kernel/workqueue_sched.h
+ *
+ * Scheduler hooks for concurrency managed workqueue.  Only to be
+ * included from sched.c and workqueue.c.
+ */
+static inline void wq_worker_waking_up(struct task_struct *task,
+				       unsigned int cpu)
+{
+}
+
+static inline struct task_struct *wq_worker_sleeping(struct task_struct *task,
+						     unsigned int cpu)
+{
+	return NULL;
+}
Signature · 1 line
1.6.4.2
maintainer
#16 lore raw

Re: [PATCH 4/4] sched: add hooks for workqueue

Quoted 6 lines · Tejun Heo

Concurrency managed workqueue needs to know when workers are going to sleep and waking up, and, when a worker goes to sleep, be able to wake up another worker to maintain adequate concurrency. This patch introduces PF_WQ_WORKER to identify workqueue workers and adds the following two hooks.

Quoted 7 lines

* wq_worker_waking_up(): called when a worker is woken up.

* wq_worker_sleeping(): called when a worker is going to sleep and may return a pointer to a local task which should be woken up. The returned task is woken up using try_to_wake_up_local() which is simplified ttwu which is called under rq lock and can only wake up local tasks.

This changelog seems to lack explanation for why you need the wakeup callback.

author
#17 lore raw

Re: [PATCH 4/4] sched: add hooks for workqueue

Hello, Peter.

Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Tejun Heo

Concurrency managed workqueue needs to know when workers are going to sleep and waking up, and, when a worker goes to sleep, be able to wake up another worker to maintain adequate concurrency. This patch introduces PF_WQ_WORKER to identify workqueue workers and adds the following two hooks.

* wq_worker_waking_up(): called when a worker is woken up.

Quoted 8 lines
Quoted 5 lines

* wq_worker_sleeping(): called when a worker is going to sleep and may return a pointer to a local task which should be woken up. The returned task is woken up using try_to_wake_up_local() which is simplified ttwu which is called under rq lock and can only wake up local tasks.

This changelog seems to lack explanation for why you need the wakeup callback.

Because cmwq "needs to know when workers are going to sleep and waking up, and, when a worker goes to sleep, be able to wake up another worker to maintain adequate concurrency". Sure, I can add more but again forward reference here would work pretty well when digging through history.

Signature · 1 line
tejun
maintainer
#18 lore raw

Re: [PATCH 4/4] sched: add hooks for workqueue

Quoted 18 lines · Tejun Heo

Hello, Peter.

Quoted 15 lines · Peter Zijlstra
Quoted 13 lines · Tejun Heo

Concurrency managed workqueue needs to know when workers are going to sleep and waking up, and, when a worker goes to sleep, be able to wake up another worker to maintain adequate concurrency. This patch introduces PF_WQ_WORKER to identify workqueue workers and adds the following two hooks.

* wq_worker_waking_up(): called when a worker is woken up.

* wq_worker_sleeping(): called when a worker is going to sleep and may return a pointer to a local task which should be woken up. The returned task is woken up using try_to_wake_up_local() which is simplified ttwu which is called under rq lock and can only wake up local tasks.

Quoted 6 lines
Quoted 2 lines

This changelog seems to lack explanation for why you need the wakeup callback.

Because cmwq "needs to know when workers are going to sleep and waking up, and, when a worker goes to sleep, be able to wake up another worker to maintain adequate concurrency".

That again only explains what you use the sleep hook for, not what you want to use the wakeup hook for (putting a worker to sleep when there are now 2 runnable seems like a good use).

author
#19 lore raw

Re: [PATCH 4/4] sched: add hooks for workqueue

Quoted 3 lines · Peter Zijlstra

That again only explains what you use the sleep hook for, not what you want to use the wakeup hook for (putting a worker to sleep when there are now 2 runnable seems like a good use).

Alright, will add that. Thanks.

Signature · 1 line
tejun
author
#20 lore raw
In reply to Tejun Heo
Quoted 2 lines · Tejun Heo

These four patches are the scheduler modifications necessary for cmwq and contains the following four patches.

Ping.

Signature · 1 line
tejun
author
#21 lore raw

Hello, Peter, Ingo.

Quoted 6 lines · Tejun Heo
Quoted 2 lines · Tejun Heo

These four patches are the scheduler modifications necessary for cmwq and contains the following four patches.

Ping.

I know we're inside -rc1 merge window but I'm really hoping to push cmwq through linux-next during this devel cycle as the whole thing basically has been blocked unchanged on scheduler changes for a couple of months now. So, can we please get this thing moving?

Thanks.

Signature · 1 line
tejun
#22 lore raw
Quoted 7 lines · Tejun Heo

Hello, Peter, Ingo.

Quoted 4 lines · Tejun Heo
Quoted 2 lines · Tejun Heo

These four patches are the scheduler modifications necessary for cmwq and contains the following four patches.

Quoted 8 lines
Quoted 2 lines

Ping.

I know we're inside -rc1 merge window but I'm really hoping to push cmwq through linux-next during this devel cycle as the whole thing basically has been blocked unchanged on scheduler changes for a couple of months now. So, can we please get this thing moving?

IIRC these changes are for some sort of speedup for btrfs, right?

I would really like to see before/after numbers on some important FS workload. How much of a speedup does it give?

Thanks,

Ingo

author
#23 lore raw

Hello, Ingo.

Quoted 2 lines · Ingo Molnar

IIRC these changes are for some sort of speedup for btrfs, right?

Eh.... What are you talking about? These are for cmwq. The "must unify scheduler notification hooks", "nah, if always enabled, go with hard coded hooks" stuff.

Thanks.

Signature · 1 line
tejun
#24 lore raw
Quoted 9 lines · Tejun Heo

Hello, Ingo.

Quoted 2 lines · Ingo Molnar

IIRC these changes are for some sort of speedup for btrfs, right?

Eh.... What are you talking about? These are for cmwq. The "must unify scheduler notification hooks", "nah, if always enabled, go with hard coded hooks" stuff.

... which is then used for the concurrency-managed workqueue feature, which is then used for a BTRFS speedup, right?

( If that's not the plan then color me confused :)

Thanks,

Ingo

author
#25 lore raw

Hello,

Quoted 5 lines · Ingo Molnar

... which is then used for the concurrency-managed workqueue feature, which is then used for a BTRFS speedup, right?

( If that's not the plan then color me confused :)

Yeap, you're officially confused. :-) It doesn't have much to do with btrfs at this point and performance test result was posted earlier this year (on your request too) where it performed quite similarly to the workqueue code on the favorable side while providing much easier API for its users.

The whole thing has been sitting in my queue for months now blocked on these scheduler changes. I've requested multiple times to push this forward on at least a separate sched devel branch and make incremental changes from there if scheduler side is not satisfactory so that the whole code base can be tested in wider scope and fix and improvement history can be kept in git, about which you always have had a pretty strong opinion about. There is no outstanding major opposition to cmwq itself (fscache conversion needs more performance testing tho). The whole thing is blocked on these scheduler changes. What's up?

Thanks.

Signature · 1 line
tejun
#26 lore raw
Quoted 4 lines · Tejun Heo

Hello,

On 05/23/2010 11:13 AM, Ingo Molnar wrote:

Quoted 8 lines
Quoted 5 lines

... which is then used for the concurrency-managed workqueue feature, which is then used for a BTRFS speedup, right?

( If that's not the plan then color me confused :)

Yeap, you're officially confused. :-) It doesn't have much to do with btrfs at this point [...]

Not exclusive to btrfs of course, but IIRC one of the examples given in one of those threads was btrfs scalability, right?

In case it's not obvious, i'm trying to find a tangible improvement to examine. No need to resend stuff, do you have a link to that old performance measurement by any chance?

Thanks,

Ingo

author
#27 lore raw

Hello, Ingo.

Quoted 2 lines · Ingo Molnar

Not exclusive to btrfs of course, but IIRC one of the examples given in one of those threads was btrfs scalability, right?

Hmmm... I don't think so. If btrfs ever was mentioned, it must have been pretty tangential. Nothing rings a bell to me.

Quoted 3 lines

In case it's not obvious, i'm trying to find a tangible improvement to examine. No need to resend stuff, do you have a link to that old performance measurement by any chance?

Sure.

http://thread.gmane.org/gmane.linux.kernel/939353

Thanks.

Signature · 1 line
tejun
author
#28 lore raw
Quoted 9 lines · Tejun Heo

Hello, Ingo.

Quoted 2 lines · Ingo Molnar

Not exclusive to btrfs of course, but IIRC one of the examples given in one of those threads was btrfs scalability, right?

Hmmm... I don't think so. If btrfs ever was mentioned, it must have been pretty tangential. Nothing rings a bell to me.

Quoted 7 lines
Quoted 3 lines

In case it's not obvious, i'm trying to find a tangible improvement to examine. No need to resend stuff, do you have a link to that old performance measurement by any chance?

Sure.

http://thread.gmane.org/gmane.linux.kernel/939353

Ingo, any update?

Thanks.

Signature · 1 line
tejun