Skip to contentSkip to conversation

workqueue: break affinity initiatively

PATCH Superseded Lai Jiangshan 4 participants 25 messages Dec 14, 2020 – Dec 16, 2020 Workqueue

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Lai Jiangshan

author reviewer
#0 lore raw

cover workqueue: break affinity initiatively

From: Lai Jiangshan <[email protected]>

06249738a41a ("workqueue: Manually break affinity on hotplug") said that scheduler will not force break affinity for us.

But workqueue highly depends on the old behavior. Many parts of the codes relies on it, 06249738a41a ("workqueue: Manually break affinity on hotplug") is not enough to change it, and the commit has flaws in itself too.

We need to thoroughly update the way workqueue handles affinity in cpu hot[un]plug, what is this patchset intends to do and replace the Valentin Schneider's patch [1].

Patch 1 fixes a flaw reported by Hillf Danton <[email protected]>. I have to include this fix because later patches depends on it.

The patchset is based on tip/master rather than workqueue tree, because the patchset is a complement for 06249738a41a ("workqueue:
Manually break affinity on hotplug") which is only in tip/master by now.

[1]: https://lore.kernel.org/r/[email protected]

Cc Hillf Danton <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc Qian Cai <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Vincent Donnefort <[email protected]>
Cc Tejun Heo <[email protected]>

Lai Jiangshan (10):
workqueue: restore unbound_workers' cpumask correctly
workqueue: use cpu_possible_mask instead of cpu_active_mask to break
affinity
workqueue: Manually break affinity on pool detachment
workqueue: don't set the worker's cpumask when kthread_bind_mask()
workqueue: introduce wq_online_cpumask
workqueue: use wq_online_cpumask in restore_unbound_workers_cpumask()
workqueue: Manually break affinity on hotplug for unbound pool
workqueue: reorganize workqueue_online_cpu()
workqueue: reorganize workqueue_offline_cpu() unbind_workers()
workqueue: Fix affinity of kworkers when attaching into pool

 kernel/workqueue.c | 212 +++++++++++++++++++++++++++------------------
 1 file changed, 130 insertions(+), 82 deletions(-)
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#2 lore raw

PATCH 2/10 workqueue: use cpu_possible_mask instead of cpu_active_mask to break affinity

From: Lai Jiangshan <[email protected]>

There might be other CPU online. The workers losing binding on its CPU should have chance to work on those later onlined CPUs.

Fixes 06249738a41a ("workqueue: Manually break affinity on hotplug")
---
 kernel/workqueue.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/workqueue.c +2 -1
index aba71ab359dd..1f5b8385c0cf 100644
@@ -4909,8 +4909,9 @@ static void unbind_workers(int cpu)
 
 		raw_spin_unlock_irq(&pool->lock);
 
+		/* don't rely on the scheduler to force break affinity for us. */
 		for_each_pool_worker(worker, pool)
-			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_active_mask) < 0);
+			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);
 
 		mutex_unlock(&wq_pool_attach_mutex);
 
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#7 lore raw

PATCH 3/10 workqueue: Manually break affinity on pool detachment

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

Don't rely on the scheduler to force break affinity for us -- it will stop doing that for per-cpu-kthreads.

Fixes 06249738a41a ("workqueue: Manually break affinity on hotplug")
---
 kernel/workqueue.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/workqueue.c +10 -0
index 1f5b8385c0cf..1f6cb83e0bc5 100644
@@ -1885,6 +1885,16 @@ static void worker_detach_from_pool(struct worker *worker)
 
 	if (list_empty(&pool->workers))
 		detach_completion = pool->detach_completion;
+
+	/*
+	 * The cpus of pool->attrs->cpumask might all go offline after
+	 * detachment, and the scheduler may not force break affinity
+	 * for us, so we do it on our own and unbind this worker which
+	 * can't be unbound by workqueue_offline_cpu() since it doesn't
+	 * belong to any pool after it.
+	 */
+	set_cpus_allowed_ptr(worker->task, cpu_possible_mask);
+
 	mutex_unlock(&wq_pool_attach_mutex);
 
 	/* clear leftover flags without pool->lock after it is detached */
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#15 lore raw

PATCH 9/10 workqueue: reorganize workqueue_offline_cpu() unbind_workers()

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

Just move around the code, no functionality changed. Only wq_pool_attach_mutex protected region become a little larger.

It prepares for later patch protecting wq_online_cpumask in wq_pool_attach_mutex.

---
 kernel/workqueue.c | 92 +++++++++++++++++++++++-----------------------
 1 file changed, 46 insertions(+), 46 deletions(-)
kernel/workqueue.c +46 -46
index fa29b7a083a6..5ef41c567c2b 100644
@@ -4901,62 +4901,58 @@ void wq_worker_comm(char *buf, size_t size, struct task_struct *task)
  * cpu comes back online.
  */
 
-static void unbind_workers(int cpu)
+static void unbind_workers(struct worker_pool *pool)
 {
-	struct worker_pool *pool;
 	struct worker *worker;
 
-	for_each_cpu_worker_pool(pool, cpu) {
-		mutex_lock(&wq_pool_attach_mutex);
-		raw_spin_lock_irq(&pool->lock);
+	lockdep_assert_held(&wq_pool_attach_mutex);
 
-		/*
-		 * We've blocked all attach/detach operations. Make all workers
-		 * unbound and set DISASSOCIATED.  Before this, all workers
-		 * except for the ones which are still executing works from
-		 * before the last CPU down must be on the cpu.  After
-		 * this, they may become diasporas.
-		 */
-		for_each_pool_worker(worker, pool)
-			worker->flags |= WORKER_UNBOUND;
+	raw_spin_lock_irq(&pool->lock);
 
-		pool->flags |= POOL_DISASSOCIATED;
+	/*
+	 * We've blocked all attach/detach operations. Make all workers
+	 * unbound and set DISASSOCIATED.  Before this, all workers
+	 * except for the ones which are still executing works from
+	 * before the last CPU down must be on the cpu.  After
+	 * this, they may become diasporas.
+	 */
+	for_each_pool_worker(worker, pool)
+		worker->flags |= WORKER_UNBOUND;
 
-		raw_spin_unlock_irq(&pool->lock);
+	pool->flags |= POOL_DISASSOCIATED;
 
-		/* don't rely on the scheduler to force break affinity for us. */
-		for_each_pool_worker(worker, pool)
-			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);
+	raw_spin_unlock_irq(&pool->lock);
 
-		mutex_unlock(&wq_pool_attach_mutex);
+	/* don't rely on the scheduler to force break affinity for us. */
+	for_each_pool_worker(worker, pool)
+		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);
 
-		/*
-		 * Call schedule() so that we cross rq->lock and thus can
-		 * guarantee sched callbacks see the %WORKER_UNBOUND flag.
-		 * This is necessary as scheduler callbacks may be invoked
-		 * from other cpus.
-		 */
-		schedule();
+	/*
+	 * Call schedule() so that we cross rq->lock and thus can
+	 * guarantee sched callbacks see the %WORKER_UNBOUND flag.
+	 * This is necessary as scheduler callbacks may be invoked
+	 * from other cpus.
+	 */
+	schedule();
 
-		/*
-		 * Sched callbacks are disabled now.  Zap nr_running.
-		 * After this, nr_running stays zero and need_more_worker()
-		 * and keep_working() are always true as long as the
-		 * worklist is not empty.  This pool now behaves as an
-		 * unbound (in terms of concurrency management) pool which
-		 * are served by workers tied to the pool.
-		 */
-		atomic_set(&pool->nr_running, 0);
+	/*
+	 * Sched callbacks are disabled now.  Zap nr_running.
+	 * After this, nr_running stays zero and need_more_worker()
+	 * and keep_working() are always true as long as the
+	 * worklist is not empty.  This pool now behaves as an
+	 * unbound (in terms of concurrency management) pool which
+	 * are served by workers tied to the pool.
+	 */
+	atomic_set(&pool->nr_running, 0);
 
-		/*
-		 * With concurrency management just turned off, a busy
-		 * worker blocking could lead to lengthy stalls.  Kick off
-		 * unbound chain execution of currently pending work items.
-		 */
-		raw_spin_lock_irq(&pool->lock);
-		wake_up_worker(pool);
-		raw_spin_unlock_irq(&pool->lock);
-	}
+	/*
+	 * With concurrency management just turned off, a busy
+	 * worker blocking could lead to lengthy stalls.  Kick off
+	 * unbound chain execution of currently pending work items.
+	 */
+	raw_spin_lock_irq(&pool->lock);
+	wake_up_worker(pool);
+	raw_spin_unlock_irq(&pool->lock);
 }
 
 /**
@@ -5119,7 +5115,11 @@ int workqueue_offline_cpu(unsigned int cpu)
 	if (WARN_ON(cpu != smp_processor_id()))
 		return -1;
 
-	unbind_workers(cpu);
+	for_each_cpu_worker_pool(pool, cpu) {
+		mutex_lock(&wq_pool_attach_mutex);
+		unbind_workers(pool);
+		mutex_unlock(&wq_pool_attach_mutex);
+	}
 
 	mutex_lock(&wq_pool_mutex);
 	cpumask_clear_cpu(cpu, wq_online_cpumask);
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#12 lore raw

PATCH 7/10 workqueue: Manually break affinity on hotplug for unbound pool

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

When all of the CPUs of the unbound pool go down, the scheduler will break affinity on the workers for us. We can do it by our own and don't rely on the scheduler to force break affinity for us.

Fixes 06249738a41a ("workqueue: Manually break affinity on hotplug")
---
 kernel/workqueue.c | 49 ++++++++++++++++++++++++++++++++--------------
 1 file changed, 34 insertions(+), 15 deletions(-)
kernel/workqueue.c +34 -15
index 878ed83e5908..eea58f77a37b 100644
@@ -5025,16 +5025,16 @@ static void rebind_workers(struct worker_pool *pool)
 }
 
 /**
- * restore_unbound_workers_cpumask - restore cpumask of unbound workers
+ * update_unbound_workers_cpumask - update cpumask of unbound workers
  * @pool: unbound pool of interest
- * @cpu: the CPU which is coming up
+ * @cpu: the CPU which is coming up or going down
  *
  * An unbound pool may end up with a cpumask which doesn't have any online
- * CPUs.  When a worker of such pool get scheduled, the scheduler resets
- * its cpus_allowed.  If @cpu is in @pool's cpumask which didn't have any
- * online CPU before, cpus_allowed of all its workers should be restored.
+ * CPUs.  We have to reset workers' cpus_allowed of such pool.  And we
+ * restore the workers' cpus_allowed when the pool's cpumask has online
+ * CPU at the first time after reset.
  */
-static void restore_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
+static void update_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
 {
 	static cpumask_t cpumask;
 	struct worker *worker;
@@ -5048,13 +5048,19 @@ static void restore_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
 
 	cpumask_and(&cpumask, pool->attrs->cpumask, wq_online_cpumask);
 
-	/* is @cpu the first one onlined for the @pool? */
-	if (cpumask_weight(&cpumask) > 1)
-		return;
-
-	/* as we're called from CPU_ONLINE, the following shouldn't fail */
-	for_each_pool_worker(worker, pool)
-		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask) < 0);
+	switch (cpumask_weight(&cpumask)) {
+	case 0: /* @cpu is the last one going down for the @pool. */
+		for_each_pool_worker(worker, pool)
+			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);
+		break;
+	case 1: /* @cpu is the first one onlined for the @pool. */
+		/* as we're called from CPU_ONLINE, the following shouldn't fail */
+		for_each_pool_worker(worker, pool)
+			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask) < 0);
+		break;
+	default: /* other cases, nothing to do */
+		break;
+	}
 }
 
 int workqueue_prepare_cpu(unsigned int cpu)
@@ -5085,7 +5091,7 @@ int workqueue_online_cpu(unsigned int cpu)
 		if (pool->cpu == cpu)
 			rebind_workers(pool);
 		else if (pool->cpu < 0)
-			restore_unbound_workers_cpumask(pool, cpu);
+			update_unbound_workers_cpumask(pool, cpu);
 
 		mutex_unlock(&wq_pool_attach_mutex);
 	}
@@ -5100,7 +5106,9 @@ int workqueue_online_cpu(unsigned int cpu)
 
 int workqueue_offline_cpu(unsigned int cpu)
 {
+	struct worker_pool *pool;
 	struct workqueue_struct *wq;
+	int pi;
 
 	/* unbinding per-cpu workers should happen on the local CPU */
 	if (WARN_ON(cpu != smp_processor_id()))
@@ -5108,9 +5116,20 @@ int workqueue_offline_cpu(unsigned int cpu)
 
 	unbind_workers(cpu);
 
-	/* update NUMA affinity of unbound workqueues */
 	mutex_lock(&wq_pool_mutex);
 	cpumask_clear_cpu(cpu, wq_online_cpumask);
+
+	/* update CPU affinity of workers of unbound pools */
+	for_each_pool(pool, pi) {
+		mutex_lock(&wq_pool_attach_mutex);
+
+		if (pool->cpu < 0)
+			update_unbound_workers_cpumask(pool, cpu);
+
+		mutex_unlock(&wq_pool_attach_mutex);
+	}
+
+	/* update NUMA affinity of unbound workqueues */
 	list_for_each_entry(wq, &workqueues, list)
 		wq_update_unbound_numa(wq, cpu);
 	mutex_unlock(&wq_pool_mutex);
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#16 lore raw

PATCH 10/10 workqueue: Fix affinity of kworkers when attaching into pool

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

When worker_attach_to_pool() is called, we should not put the workers to pool->attrs->cpumask when there is not CPU online in it.

We have to use wq_online_cpumask in worker_attach_to_pool() to check if pool->attrs->cpumask is valid rather than cpu_online_mask or cpu_active_mask due to gaps between stages in cpu hot[un]plug.

To use wq_online_cpumask in worker_attach_to_pool(), we need to protect wq_online_cpumask in wq_pool_attach_mutex and we modify workqueue_online_cpu() and workqueue_offline_cpu() to enlarge wq_pool_attach_mutex protected region. We also put updating wq_online_cpumask and [re|un]bind_workers() in the same wq_pool_attach_mutex protected region to make the update for percpu workqueue atomically.

---
 kernel/workqueue.c | 32 +++++++++++++++-----------------
 1 file changed, 15 insertions(+), 17 deletions(-)
kernel/workqueue.c +15 -17
index 5ef41c567c2b..7a04cef90c1c 100644
@@ -310,7 +310,7 @@ static bool workqueue_freezing;		/* PL: have wqs started freezing? */
 /* PL: allowable cpus for unbound wqs and work items */
 static cpumask_var_t wq_unbound_cpumask;
 
-/* PL: online cpus (cpu_online_mask with the going-down cpu cleared) */
+/* PL&A: online cpus (cpu_online_mask with the going-down cpu cleared) */
 static cpumask_var_t wq_online_cpumask;
 
 /* CPU where unbound work was last round robin scheduled from this CPU */
@@ -1848,11 +1848,11 @@ static void worker_attach_to_pool(struct worker *worker,
 {
 	mutex_lock(&wq_pool_attach_mutex);
 
-	/*
-	 * set_cpus_allowed_ptr() will fail if the cpumask doesn't have any
-	 * online CPUs.  It'll be re-applied when any of the CPUs come up.
-	 */
-	set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask);
+	/* Is there any cpu in pool->attrs->cpumask online? */
+	if (cpumask_any_and(pool->attrs->cpumask, wq_online_cpumask) < nr_cpu_ids)
+		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask) < 0);
+	else
+		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);
 
 	/*
 	 * The wq_pool_attach_mutex ensures %POOL_DISASSOCIATED remains
@@ -5079,13 +5079,12 @@ int workqueue_online_cpu(unsigned int cpu)
 	int pi;
 
 	mutex_lock(&wq_pool_mutex);
-	cpumask_set_cpu(cpu, wq_online_cpumask);
 
-	for_each_cpu_worker_pool(pool, cpu) {
-		mutex_lock(&wq_pool_attach_mutex);
+	mutex_lock(&wq_pool_attach_mutex);
+	cpumask_set_cpu(cpu, wq_online_cpumask);
+	for_each_cpu_worker_pool(pool, cpu)
 		rebind_workers(pool);
-		mutex_unlock(&wq_pool_attach_mutex);
-	}
+	mutex_unlock(&wq_pool_attach_mutex);
 
 	/* update CPU affinity of workers of unbound pools */
 	for_each_pool(pool, pi) {
@@ -5115,14 +5114,13 @@ int workqueue_offline_cpu(unsigned int cpu)
 	if (WARN_ON(cpu != smp_processor_id()))
 		return -1;
 
-	for_each_cpu_worker_pool(pool, cpu) {
-		mutex_lock(&wq_pool_attach_mutex);
-		unbind_workers(pool);
-		mutex_unlock(&wq_pool_attach_mutex);
-	}
-
 	mutex_lock(&wq_pool_mutex);
+
+	mutex_lock(&wq_pool_attach_mutex);
 	cpumask_clear_cpu(cpu, wq_online_cpumask);
+	for_each_cpu_worker_pool(pool, cpu)
+		unbind_workers(pool);
+	mutex_unlock(&wq_pool_attach_mutex);
 
 	/* update CPU affinity of workers of unbound pools */
 	for_each_pool(pool, pi) {
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#11 lore raw

PATCH 6/10 workqueue: use wq_online_cpumask in restore_unbound_workers_cpumask()

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

restore_unbound_workers_cpumask() is called when CPU_ONLINE, where wq_online_cpumask equals to cpu_online_mask. So no fucntionality changed.

---
 kernel/workqueue.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/workqueue.c +2 -1
index 8aca3afc88aa..878ed83e5908 100644
@@ -5039,13 +5039,14 @@ static void restore_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
 	static cpumask_t cpumask;
 	struct worker *worker;
 
+	lockdep_assert_held(&wq_pool_mutex);
 	lockdep_assert_held(&wq_pool_attach_mutex);
 
 	/* is @cpu allowed for @pool? */
 	if (!cpumask_test_cpu(cpu, pool->attrs->cpumask))
 		return;
 
-	cpumask_and(&cpumask, pool->attrs->cpumask, cpu_online_mask);
+	cpumask_and(&cpumask, pool->attrs->cpumask, wq_online_cpumask);
 
 	/* is @cpu the first one onlined for the @pool? */
 	if (cpumask_weight(&cpumask) > 1)
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#14 lore raw

PATCH 8/10 workqueue: reorganize workqueue_online_cpu()

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

Just move around the code, no functionality changed.

It prepares for later patch protecting wq_online_cpumask in wq_pool_attach_mutex.

---
 kernel/workqueue.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
kernel/workqueue.c +8 -3
index eea58f77a37b..fa29b7a083a6 100644
@@ -5085,12 +5085,17 @@ int workqueue_online_cpu(unsigned int cpu)
 	mutex_lock(&wq_pool_mutex);
 	cpumask_set_cpu(cpu, wq_online_cpumask);
 
+	for_each_cpu_worker_pool(pool, cpu) {
+		mutex_lock(&wq_pool_attach_mutex);
+		rebind_workers(pool);
+		mutex_unlock(&wq_pool_attach_mutex);
+	}
+
+	/* update CPU affinity of workers of unbound pools */
 	for_each_pool(pool, pi) {
 		mutex_lock(&wq_pool_attach_mutex);
 
-		if (pool->cpu == cpu)
-			rebind_workers(pool);
-		else if (pool->cpu < 0)
+		if (pool->cpu < 0)
 			update_unbound_workers_cpumask(pool, cpu);
 
 		mutex_unlock(&wq_pool_attach_mutex);
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#10 lore raw

PATCH 5/10 workqueue: introduce wq_online_cpumask

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

wq_online_cpumask is the cached result of cpu_online_mask with the going-down cpu cleared. It is needed for later patches for setting correct cpumask for workers and break affinity initiatively.

---
 kernel/workqueue.c | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)
kernel/workqueue.c +18 -16
index f679c599a70b..8aca3afc88aa 100644
@@ -310,6 +310,9 @@ static bool workqueue_freezing;		/* PL: have wqs started freezing? */
 /* PL: allowable cpus for unbound wqs and work items */
 static cpumask_var_t wq_unbound_cpumask;
 
+/* PL: online cpus (cpu_online_mask with the going-down cpu cleared) */
+static cpumask_var_t wq_online_cpumask;
+
 /* CPU where unbound work was last round robin scheduled from this CPU */
 static DEFINE_PER_CPU(int, wq_rr_cpu_last);
 
@@ -3830,12 +3833,10 @@ static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *wq,
  * wq_calc_node_cpumask - calculate a wq_attrs' cpumask for the specified node
  * @attrs: the wq_attrs of the default pwq of the target workqueue
  * @node: the target NUMA node
- * @cpu_going_down: if >= 0, the CPU to consider as offline
  * @cpumask: outarg, the resulting cpumask
  *
- * Calculate the cpumask a workqueue with @attrs should use on @node.  If
- * @cpu_going_down is >= 0, that cpu is considered offline during
- * calculation.  The result is stored in @cpumask.
+ * Calculate the cpumask a workqueue with @attrs should use on @node.
+ * The result is stored in @cpumask.
  *
  * If NUMA affinity is not enabled, @attrs->cpumask is always used.  If
  * enabled and @node has online CPUs requested by @attrs, the returned
@@ -3849,15 +3850,14 @@ static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *wq,
  * %false if equal.
  */
 static bool wq_calc_node_cpumask(const struct workqueue_attrs *attrs, int node,
-				 int cpu_going_down, cpumask_t *cpumask)
+				 cpumask_t *cpumask)
 {
 	if (!wq_numa_enabled || attrs->no_numa)
 		goto use_dfl;
 
 	/* does @node have any online CPUs @attrs wants? */
 	cpumask_and(cpumask, cpumask_of_node(node), attrs->cpumask);
-	if (cpu_going_down >= 0)
-		cpumask_clear_cpu(cpu_going_down, cpumask);
+	cpumask_and(cpumask, cpumask, wq_online_cpumask);
 
 	if (cpumask_empty(cpumask))
 		goto use_dfl;
@@ -3966,7 +3966,7 @@ apply_wqattrs_prepare(struct workqueue_struct *wq,
 		goto out_free;
 
 	for_each_node(node) {
-		if (wq_calc_node_cpumask(new_attrs, node, -1, tmp_attrs->cpumask)) {
+		if (wq_calc_node_cpumask(new_attrs, node, tmp_attrs->cpumask)) {
 			ctx->pwq_tbl[node] = alloc_unbound_pwq(wq, tmp_attrs);
 			if (!ctx->pwq_tbl[node])
 				goto out_free;
@@ -4091,7 +4091,6 @@ int apply_workqueue_attrs(struct workqueue_struct *wq,
  * wq_update_unbound_numa - update NUMA affinity of a wq for CPU hot[un]plug
  * @wq: the target workqueue
  * @cpu: the CPU coming up or going down
- * @online: whether @cpu is coming up or going down
  *
  * This function is to be called from %CPU_DOWN_PREPARE, %CPU_ONLINE and
  * %CPU_DOWN_FAILED.  @cpu is being hot[un]plugged, update NUMA affinity of
@@ -4109,11 +4108,9 @@ int apply_workqueue_attrs(struct workqueue_struct *wq,
  * affinity, it's the user's responsibility to flush the work item from
  * CPU_DOWN_PREPARE.
  */
-static void wq_update_unbound_numa(struct workqueue_struct *wq, int cpu,
-				   bool online)
+static void wq_update_unbound_numa(struct workqueue_struct *wq, int cpu)
 {
 	int node = cpu_to_node(cpu);
-	int cpu_off = online ? -1 : cpu;
 	struct pool_workqueue *old_pwq = NULL, *pwq;
 	struct workqueue_attrs *target_attrs;
 	cpumask_t *cpumask;
@@ -4141,7 +4138,7 @@ static void wq_update_unbound_numa(struct workqueue_struct *wq, int cpu,
 	 * and create a new one if they don't match.  If the target cpumask
 	 * equals the default pwq's, the default pwq should be used.
 	 */
-	if (wq_calc_node_cpumask(wq->dfl_pwq->pool->attrs, node, cpu_off, cpumask)) {
+	if (wq_calc_node_cpumask(wq->dfl_pwq->pool->attrs, node, cpumask)) {
 		if (cpumask_equal(cpumask, pwq->pool->attrs->cpumask))
 			return;
 	} else {
@@ -5079,6 +5076,7 @@ int workqueue_online_cpu(unsigned int cpu)
 	int pi;
 
 	mutex_lock(&wq_pool_mutex);
+	cpumask_set_cpu(cpu, wq_online_cpumask);
 
 	for_each_pool(pool, pi) {
 		mutex_lock(&wq_pool_attach_mutex);
@@ -5093,7 +5091,7 @@ int workqueue_online_cpu(unsigned int cpu)
 
 	/* update NUMA affinity of unbound workqueues */
 	list_for_each_entry(wq, &workqueues, list)
-		wq_update_unbound_numa(wq, cpu, true);
+		wq_update_unbound_numa(wq, cpu);
 
 	mutex_unlock(&wq_pool_mutex);
 	return 0;
@@ -5111,8 +5109,9 @@ int workqueue_offline_cpu(unsigned int cpu)
 
 	/* update NUMA affinity of unbound workqueues */
 	mutex_lock(&wq_pool_mutex);
+	cpumask_clear_cpu(cpu, wq_online_cpumask);
 	list_for_each_entry(wq, &workqueues, list)
-		wq_update_unbound_numa(wq, cpu, false);
+		wq_update_unbound_numa(wq, cpu);
 	mutex_unlock(&wq_pool_mutex);
 
 	return 0;
@@ -5949,6 +5948,9 @@ void __init workqueue_init_early(void)
 
 	BUILD_BUG_ON(__alignof__(struct pool_workqueue) < __alignof__(long long));
 
+	BUG_ON(!alloc_cpumask_var(&wq_online_cpumask, GFP_KERNEL));
+	cpumask_copy(wq_online_cpumask, cpu_online_mask);
+
 	BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL));
 	cpumask_copy(wq_unbound_cpumask, housekeeping_cpumask(hk_flags));
 
@@ -6045,7 +6047,7 @@ void __init workqueue_init(void)
 	}
 
 	list_for_each_entry(wq, &workqueues, list) {
-		wq_update_unbound_numa(wq, smp_processor_id(), true);
+		wq_update_unbound_numa(wq, smp_processor_id());
 		WARN(init_rescuer(wq),
 		     "workqueue: failed to create early rescuer for %s",
 		     wq->name);
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#8 lore raw

PATCH 4/10 workqueue: don't set the worker's cpumask when kthread_bind_mask()

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

There might be no online cpu in the pool->attrs->cpumask. We will set the worker's cpumask later in worker_attach_to_pool().

---
 kernel/workqueue.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
kernel/workqueue.c +9 -1
index 1f6cb83e0bc5..f679c599a70b 100644
@@ -1945,7 +1945,15 @@ static struct worker *create_worker(struct worker_pool *pool)
 		goto fail;
 
 	set_user_nice(worker->task, pool->attrs->nice);
-	kthread_bind_mask(worker->task, pool->attrs->cpumask);
+
+	/*
+	 * Set PF_NO_SETAFFINITY via kthread_bind_mask().  We use
+	 * cpu_possible_mask other than pool->attrs->cpumask, because
+	 * there might be no online cpu in the pool->attrs->cpumask.
+	 * The cpumask of the worker will be set properly later in
+	 * worker_attach_to_pool().
+	 */
+	kthread_bind_mask(worker->task, cpu_possible_mask);
 
 	/* successful, attach the worker to the pool */
 	worker_attach_to_pool(worker, pool);
Signature · 1 line
2.19.1.6.gb485710b
author reviewer
#1 lore raw

PATCH 1/10 workqueue: restore unbound_workers' cpumask correctly

In reply to Lai Jiangshan

From: Lai Jiangshan <[email protected]>

When we restore workers' cpumask, we should restore them to the designed pool->attrs->cpumask. And we need to only do it at the first time.

---
 kernel/workqueue.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/workqueue.c +5 -1
index c71da2a59e12..aba71ab359dd 100644
@@ -5031,9 +5031,13 @@ static void restore_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
 
 	cpumask_and(&cpumask, pool->attrs->cpumask, cpu_online_mask);
 
+	/* is @cpu the first one onlined for the @pool? */
+	if (cpumask_weight(&cpumask) > 1)
+		return;
+
 	/* as we're called from CPU_ONLINE, the following shouldn't fail */
 	for_each_pool_worker(worker, pool)
-		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, &cpumask) < 0);
+		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask) < 0);
 }
 
 int workqueue_prepare_cpu(unsigned int cpu)
Signature · 1 line
2.19.1.6.gb485710b
#3 lore raw

Re: [PATCH 02/10] workqueue: use cpu_possible_mask instead of cpu_active_mask to break affinity

In reply to Lai Jiangshan
Quoted 16 lines · Lai Jiangshan

From: Lai Jiangshan <[email protected]>

There might be other CPU online. The workers losing binding on its CPU should have chance to work on those later onlined CPUs.

Fixes 06249738a41a ("workqueue: Manually break affinity on hotplug")
---
 kernel/workqueue.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/workqueue.c +0 -0
index aba71ab359dd..1f5b8385c0cf 100644

kernel/workqueue.c static void unbind_workers(int cpu) · line 4914

Quoted 7 lines
kernel/workqueue.c +2 -1
@@ -4909,8 +4909,9 @@ static void unbind_workers(int cpu)
 
 		raw_spin_unlock_irq(&pool->lock);
 
+		/* don't rely on the scheduler to force break affinity for us. */
 		for_each_pool_worker(worker, pool)
-			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_active_mask) < 0);
+			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);

Please explain this one.. it's not making sense. Also the Changelog doesn't seem remotely related to the actual change.

Afaict this is actively wrong.

Also, can you please not Cc me parts of a series? That's bloody annoying.

#18 lore raw
In reply to Lai Jiangshan
Quoted 5 lines · Lai Jiangshan

From: Lai Jiangshan <[email protected]>

06249738a41a ("workqueue: Manually break affinity on hotplug") said that scheduler will not force break affinity for us.

Quoted 7 lines

But workqueue highly depends on the old behavior. Many parts of the codes relies on it, 06249738a41a ("workqueue: Manually break affinity on hotplug") is not enough to change it, and the commit has flaws in itself too.

We need to thoroughly update the way workqueue handles affinity in cpu hot[un]plug, what is this patchset intends to do and replace the Valentin Schneider's patch [1].

So the actual problem is with per-cpu kthreads, the new assumption is that hot-un-plug will make all per-cpu kthreads for the dying CPU go away.

Workqueues violated that. I fixed the obvious site, and Valentin's patch avoids workqueues from quickly creating new ones while we're not looking.

What other problems did you find?

author reviewer
#19 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 9 lines · Lai Jiangshan

From: Lai Jiangshan <[email protected]>

06249738a41a ("workqueue: Manually break affinity on hotplug") said that scheduler will not force break affinity for us.

But workqueue highly depends on the old behavior. Many parts of the codes relies on it, 06249738a41a ("workqueue: Manually break affinity on hotplug") is not enough to change it, and the commit has flaws in itself too.

Quoted 7 lines
Quoted 3 lines

We need to thoroughly update the way workqueue handles affinity in cpu hot[un]plug, what is this patchset intends to do and replace the Valentin Schneider's patch [1].

So the actual problem is with per-cpu kthreads, the new assumption is that hot-un-plug will make all per-cpu kthreads for the dying CPU go away.

Hello, Peter

"new assumption" is all needed to be aligned. I haven't read the code. I thought I understood to some extent which is enough for me to know that workqueue does violate that.

Workqueue does not break affinity for all per-cpu kthreads in several cases such as hot-un-plug and workers detaching from pool (those workers will not be searchable from pools and should be handled alike to hot-un-plug).

But workqueue has not only per-cpu kthreads but also per-node threads. And per-node threads may be bound to multiple CPUs or may be bound to a single CPU. I don't know how the scheduler distinguishes all these different cases under the "new assumption". But at least workqueue handle these different cases at the same few places. Since workqueue have to "break affinity" for per-cpu kthreads, it can also "break affinity" for other cases. Making workqueue totally do not rely on scheduler's work to "break affinity" is worth doing since we have to do it for the most parts.

I haven't read the code about "new assumption", if possible, I'll first try to find out how will scheduler handle these cases:

If a per-node thread has only cpu 4, and when it goes down, does workqueue need to "break affinity" for it?

If a per-node thread has only cpu 41,42, and when both go down, does workqueue need to "break affinity" for it?

Thanks
Lai

Quoted 6 lines

Workqueues violated that. I fixed the obvious site, and Valentin's patch avoids workqueues from quickly creating new ones while we're not looking.

What other problems did you find?

#20 lore raw
Quoted 2 lines · Lai Jiangshan

I don't know how the scheduler distinguishes all these different cases under the "new assumption".

The special case is:

  (p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1
author reviewer
#21 lore raw
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines · Lai Jiangshan

I don't know how the scheduler distinguishes all these different cases under the "new assumption".

The special case is:

  (p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1

So unbound per-node workers can possibly match this test. So there is code needed to handle for unbound workers/pools which is done by this patchset.

Is this the code of is_per_cpu_kthread()? I think I should have also used this function in workqueue and don't break affinity for unbound workers have more than 1 cpu.

author reviewer
#4 lore raw

Re: [PATCH 02/10] workqueue: use cpu_possible_mask instead of cpu_active_mask to break affinity

In reply to Peter Zijlstra
Quoted 21 lines · Peter Zijlstra
Quoted 19 lines · Lai Jiangshan

From: Lai Jiangshan <[email protected]>

There might be other CPU online. The workers losing binding on its CPU should have chance to work on those later onlined CPUs.

Fixes 06249738a41a ("workqueue: Manually break affinity on hotplug")
---
 kernel/workqueue.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/workqueue.c +0 -0
index aba71ab359dd..1f5b8385c0cf 100644
@@ -4909,8 +4909,9 @@ static void unbind_workers(int cpu)
 
              raw_spin_unlock_irq(&pool->lock);
 
Quoted 7 lines

kernel/workqueue.c static void unbind_workers(int cpu) · line 4914

Quoted 4 lines
kernel/workqueue.c +2 -1
+             /* don't rely on the scheduler to force break affinity for us. */
              for_each_pool_worker(worker, pool)
-                     WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_active_mask) < 0);
+                     WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);

Please explain this one.. it's not making sense. Also the Changelog doesn't seem remotely related to the actual change.

If the scheduler doesn't break affinity for us any more, I hope that we can "emulate" previous behavior when the scheduler did breaks affinity for us. The behavior is "changing the cpumask to cpu_possible_mask".

And there might be some other CPUs online later while the worker is still running with the pending work items. I hope the worker can also use the later online CPUs as before. If we use cpu_active_mask here,
we can't achieve this. This is what the changelog said. I don't know
which wording is better, I will combine both if this reason stands.

Quoted 5 lines

Afaict this is actively wrong.

Also, can you please not Cc me parts of a series? That's bloody annoying.

Sorry about it. I was taught "once don't send the whole series to someone" and very probably I missed the conditions about it. I think in this case, I should Cc you the whole series. May I?

Thanks
Lai

#5 lore raw

Re: [PATCH 02/10] workqueue: use cpu_possible_mask instead of cpu_active_mask to break affinity

In reply to Peter Zijlstra
Quoted 28 lines · Peter Zijlstra
Quoted 16 lines · Lai Jiangshan

From: Lai Jiangshan <[email protected]>

There might be other CPU online. The workers losing binding on its CPU should have chance to work on those later onlined CPUs.

Fixes 06249738a41a ("workqueue: Manually break affinity on hotplug")
---
 kernel/workqueue.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/workqueue.c +0 -0
index aba71ab359dd..1f5b8385c0cf 100644

kernel/workqueue.c static void unbind_workers(int cpu) · line 4914

Quoted 7 lines
kernel/workqueue.c +2 -1
@@ -4909,8 +4909,9 @@ static void unbind_workers(int cpu)
 
 		raw_spin_unlock_irq(&pool->lock);
 
+		/* don't rely on the scheduler to force break affinity for us. */
 		for_each_pool_worker(worker, pool)
-			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_active_mask) < 0);
+			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);

Please explain this one.. it's not making sense. Also the Changelog doesn't seem remotely related to the actual change.

Quoted 1 line

Afaict this is actively wrong.

I think I was too tired, I see what you're doing now and it should work fine, I still think the changelog could use help though.

#22 lore raw
In reply to Lai Jiangshan
Quoted 6 lines · Lai Jiangshan
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Lai Jiangshan

I don't know how the scheduler distinguishes all these different cases under the "new assumption".

Quoted 8 lines
Quoted 5 lines

The special case is:

  (p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1

So unbound per-node workers can possibly match this test. So there is code needed to handle for unbound workers/pools which is done by this patchset.

Curious; how could a per-node worker match this? Only if the node is a single CPU, or otherwise too?

Quoted 3 lines

Is this the code of is_per_cpu_kthread()? I think I should have also used this function in workqueue and don't break affinity for unbound workers have more than 1 cpu.

Yes, that function captures it. If you want to use it, feel free to move it to include/linux/sched.h.

This class of threads is 'special', since it needs to violate the regular hotplug rules, and migrate_disable() made it just this little bit more special. It basically comes down to how we need certain per-cpu kthreads to run on a CPU while it's brought up, before userspace is allowed on, and similarly they need to run on the CPU after userspace is no longer allowed on in order to bring it down.

(IOW, they must be allowed to violate the active mask)

Due to migrate_disable() we had to move the migration code from the very last cpu-down stage, to earlier. This in turn brought the expectation (which is normally met) that per-cpu kthreads will stop/park or otherwise make themselves scarce when the CPU goes down. We can no longer force migrate them.

Workqueues are the sole exception to that, they've got some really 'dodgy' hotplug behaviour.

author reviewer
#23 lore raw
Quoted 14 lines · Peter Zijlstra
Quoted 12 lines · Lai Jiangshan
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines · Lai Jiangshan

I don't know how the scheduler distinguishes all these different cases under the "new assumption".

The special case is:

  (p->flags & PF_KTHREAD) && p->nr_cpus_allowed == 1
Quoted 5 lines
Quoted 2 lines

So unbound per-node workers can possibly match this test. So there is code needed to handle for unbound workers/pools which is done by this patchset.

Curious; how could a per-node worker match this? Only if the node is a single CPU, or otherwise too?

We have /sys/devices/virtual/workqueue/cpumask which can be read/written to access to wq_unbound_cpumask.

A per-node worker's cpumask is wq_unbound_cpumask&possible_cpumask_of_the_node. Since wq_unbound_cpumask can be changed by system adim, so a per-node worker's cpumask is possible to be single CPU.

wq_unbound_cpumask is used when a system adim wants to isolate some CPUs from unbound workqueques. But I think it is rare case when the admin causes a per-node worker's cpumask to be single CPU.

Even it is a rare case, we have to handle it.

Quoted 7 lines
Quoted 4 lines

Is this the code of is_per_cpu_kthread()? I think I should have also used this function in workqueue and don't break affinity for unbound workers have more than 1 cpu.

Yes, that function captures it. If you want to use it, feel free to move it to include/linux/sched.h.

I will. "single CPU" for unbound workers/pools is the rare case and enough to bring the code to break affinity for unbound workers. If we optimize for the common cases (multiple CPUs for unbound workers), the optimization seems like additional code works only in the slow path (hotunplug).

I will try it and see if it is worth.

Quoted 8 lines

This class of threads is 'special', since it needs to violate the regular hotplug rules, and migrate_disable() made it just this little bit more special. It basically comes down to how we need certain per-cpu kthreads to run on a CPU while it's brought up, before userspace is allowed on, and similarly they need to run on the CPU after userspace is no longer allowed on in order to bring it down.

Quoted 7 lines

(IOW, they must be allowed to violate the active mask)

Due to migrate_disable() we had to move the migration code from the very last cpu-down stage, to earlier. This in turn brought the expectation (which is normally met) that per-cpu kthreads will stop/park or otherwise make themselves scarce when the CPU goes down. We can no longer force migrate them.

Thanks for explaining the rationale.

Quoted 4 lines

Workqueues are the sole exception to that, they've got some really 'dodgy' hotplug behaviour.

Indeed. No one want to wait for workqueue when hotunplug, so we have to do something after the fact.

#17 lore raw

Re: [PATCH 10/10] workqueue: Fix affinity of kworkers when attaching into pool

In reply to Lai Jiangshan

kernel/workqueue.c static void worker_attach_to_pool(struct worker *worker, · line 1852

Quoted 11 lines · Lai Jiangshan
kernel/workqueue.c +2 -5
@@ -1848,11 +1848,11 @@ static void worker_attach_to_pool(struct worker *worker,
 {
      mutex_lock(&wq_pool_attach_mutex);
 
-	/*
-	 * set_cpus_allowed_ptr() will fail if the cpumask doesn't have any
-	 * online CPUs.  It'll be re-applied when any of the CPUs come up.
-	 */
-	set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask);
+	/* Is there any cpu in pool->attrs->cpumask online? */
+	if (cpumask_any_and(pool->attrs->cpumask, wq_online_cpumask) < nr_cpu_ids)

if (cpumask_intersects(pool->attrs->cpumask, wq_online_cpumask))

kernel/workqueue.c static void worker_attach_to_pool(struct worker *worker, · line 1855

Quoted 3 lines
kernel/workqueue.c +3 -0
+		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, pool->attrs->cpumask) < 0);
+	else
+		WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);

So for that late-spawned per-CPU kworker case: the outgoing CPU should have already been cleared from wq_online_cpumask, so it gets its affinity reset to the possible mask and the subsequent wakeup will ensure it's put on an active CPU.

Seems alright to me.

Quoted 3 lines
kernel/workqueue.c +0 -0
      /*
       * The wq_pool_attach_mutex ensures %POOL_DISASSOCIATED remains
maintainer Acked-by on the v1 series
#24 lore raw
In reply to Lai Jiangshan
Quoted 13 lines · Lai Jiangshan

From: Lai Jiangshan <[email protected]>

06249738a41a ("workqueue: Manually break affinity on hotplug") said that scheduler will not force break affinity for us.

But workqueue highly depends on the old behavior. Many parts of the codes relies on it, 06249738a41a ("workqueue: Manually break affinity on hotplug") is not enough to change it, and the commit has flaws in itself too.

We need to thoroughly update the way workqueue handles affinity in cpu hot[un]plug, what is this patchset intends to do and replace the Valentin Schneider's patch [1].

Quoted 8 lines

Patch 1 fixes a flaw reported by Hillf Danton <[email protected]>. I have to include this fix because later patches depends on it.

The patchset is based on tip/master rather than workqueue tree, because the patchset is a complement for 06249738a41a ("workqueue:
Manually break affinity on hotplug") which is only in tip/master by now.

[1]: https://lore.kernel.org/r/[email protected]

Generally looks good to me. Please feel free to add

and route the series through tip.

Thanks.

Signature · 1 line
tejun
maintainer
#6 lore raw

Re: [PATCH 02/10] workqueue: use cpu_possible_mask instead of cpu_active_mask to break affinity

In reply to Lai Jiangshan

Hello,

kernel/workqueue.c static void unbind_workers(int cpu) · line 4912

Quoted 5 lines · Lai Jiangshan
kernel/workqueue.c +1 -0
@@ -4909,8 +4909,9 @@ static void unbind_workers(int cpu)
 
 		raw_spin_unlock_irq(&pool->lock);
 
+		/* don't rely on the scheduler to force break affinity for us. */

I'm not sure this comment is helpful. The comment may make sense right now while the scheduler behavior is changing but down the line it's not gonna make whole lot of sense.

kernel/workqueue.c static void unbind_workers(int cpu) · line 4916

Quoted 5 lines
kernel/workqueue.c +1 -1
 		for_each_pool_worker(worker, pool)
-			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_active_mask) < 0);
+			WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task, cpu_possible_mask) < 0);
 
 		mutex_unlock(&wq_pool_attach_mutex);

Thanks.

Signature · 1 line
tejun
maintainer
#9 lore raw

Re: [PATCH 04/10] workqueue: don't set the worker's cpumask when kthread_bind_mask()

In reply to Lai Jiangshan

kernel/workqueue.c static struct worker *create_worker(struct worker_pool *pool) · line 1951

Quoted 9 lines · Lai Jiangshan
kernel/workqueue.c +4 -1
@@ -1945,7 +1945,15 @@ static struct worker *create_worker(struct worker_pool *pool)
 		goto fail;
 
 	set_user_nice(worker->task, pool->attrs->nice);
-	kthread_bind_mask(worker->task, pool->attrs->cpumask);
+
+	/*
+	 * Set PF_NO_SETAFFINITY via kthread_bind_mask().  We use
+	 * cpu_possible_mask other than pool->attrs->cpumask, because
                             ^
                             instead of

kernel/workqueue.c static struct worker *create_worker(struct worker_pool *pool) · line 1952

Quoted 1 line
kernel/workqueue.c +1 -0
+	 * there might be no online cpu in the pool->attrs->cpumask.

^
might not be any

kernel/workqueue.c static struct worker *create_worker(struct worker_pool *pool) · line 1956

Quoted 4 lines
kernel/workqueue.c +4 -0
+	 * The cpumask of the worker will be set properly later in
+	 * worker_attach_to_pool().
+	 */
+	kthread_bind_mask(worker->task, cpu_possible_mask);

This is a bit ugly but not the end of the world. Maybe we can move it to the start of worker_thread() but that'd require an extra handshake. Oh well...

Thanks.

Signature · 1 line
tejun
maintainer
#13 lore raw

Re: [PATCH 07/10] workqueue: Manually break affinity on hotplug for unbound pool

In reply to Lai Jiangshan
Quoted 7 lines · Lai Jiangshan

* An unbound pool may end up with a cpumask which doesn't have any online
- * CPUs. When a worker of such pool get scheduled, the scheduler resets
- * its cpus_allowed. If @cpu is in @pool's cpumask which didn't have any
- * online CPU before, cpus_allowed of all its workers should be restored.
+ * CPUs. We have to reset workers' cpus_allowed of such pool. And we
+ * restore the workers' cpus_allowed when the pool's cpumask has online
+ * CPU at the first time after reset.

^
for the first time

Signature · 1 line
tejun