Skip to contentSkip to conversation

Patch "workqueue: handle NUMA_NO_NODE for unbound pool_workqueue lookup" has been added to the 4.4-stable tree

PATCH [email protected] 1 participant 1 message Mar 1, 2016

Subsystems (2)
lore
Share & tools
lore

[email protected]

author maintainer Applied
#0 lore raw

This is a note to let you know that I've just added the patch titled

workqueue: handle NUMA_NO_NODE for unbound pool_workqueue lookup

to the 4.4-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
workqueue-handle-numa_no_node-for-unbound-pool_workqueue-lookup.patch
and it can be found in the queue-4.4 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From d6e022f1d207a161cd88e08ef0371554680ffc46 Mon Sep 17 00:00:00 2001

From: Tejun Heo <[email protected]>
Date: Wed, 3 Feb 2016 13:54:25 -0500
Subject: workqueue: handle NUMA_NO_NODE for unbound pool_workqueue lookup

From: Tejun Heo <[email protected]>

commit d6e022f1d207a161cd88e08ef0371554680ffc46 upstream.

When looking up the pool_workqueue to use for an unbound workqueue, workqueue assumes that the target CPU is always bound to a valid NUMA node. However, currently, when a CPU goes offline, the mapping is destroyed and cpu_to_node() returns NUMA_NO_NODE.

This has always been broken but hasn't triggered often enough before 874bbfe600a6 ("workqueue: make sure delayed work run in local cpu"). After the commit, workqueue forcifully assigns the local CPU for delayed work items without explicit target CPU to fix a different issue. This widens the window where CPU can go offline while a delayed work item is pending causing delayed work items dispatched with target CPU set to an already offlined CPU. The resulting NUMA_NO_NODE mapping makes workqueue try to queue the work item on a NULL pool_workqueue and thus crash.

While 874bbfe600a6 has been reverted for a different reason making the bug less visible again, it can still happen. Fix it by mapping NUMA_NO_NODE to the default pool_workqueue from unbound_pwq_by_node(). This is a temporary workaround. The long term solution is keeping CPU -> NODE mapping stable across CPU off/online cycles which is being worked on.

---
 kernel/workqueue.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/workqueue.c +10 -0
@@ -568,6 +568,16 @@ static struct pool_workqueue *unbound_pw
 						  int node)
 {
 	assert_rcu_or_wq_mutex_or_pool_mutex(wq);
+
+	/*
+	 * XXX: @node can be NUMA_NO_NODE if CPU goes offline while a
+	 * delayed item is pending.  The plan is to keep CPU -> NODE
+	 * mapping valid and stable across CPU on/offlines.  Once that
+	 * happens, this workaround can be removed.
+	 */
+	if (unlikely(node == NUMA_NO_NODE))
+		return wq->dfl_pwq;
+
 	return rcu_dereference_raw(wq->numa_pwq_tbl[node]);
 }
 

Patches currently in stable-queue which might be from [email protected] are

queue-4.4/workqueue-handle-numa_no_node-for-unbound-pool_workqueue-lookup.patch queue-4.4/libata-fix-sff-host-state-machine-locking-while-polling.patch queue-4.4/revert-workqueue-make-sure-delayed-work-run-in-local-cpu.patch