Skip to contentSkip to conversation

workqueue: respect isolated cpus when queueing an unbound work

PATCH Superseded Tal Shorer 2 participants 6 messages Oct 19, 2017 – Oct 21, 2017 2 messages not in the archive Workqueue

lore
Share & tools
lore

Tal Shorer

author
#0 lore raw

PATCH 1/1 workqueue: respect isolated cpus when queueing an unbound work

Initialize wq_unbound_cpumask to exclude cpus that were isolated by the cmdline's isolcpus parameter.

---
 kernel/workqueue.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/workqueue.c +1 -1
index ca937b0..25b351d 100644
@@ -5546,7 +5546,7 @@ int __init workqueue_init_early(void)
 	WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long));
 
 	BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL));
-	cpumask_copy(wq_unbound_cpumask, cpu_possible_mask);
+	cpumask_andnot(wq_unbound_cpumask, cpu_possible_mask, cpu_isolated_map);
 
 	pwq_cache = KMEM_CACHE(pool_workqueue, SLAB_PANIC);
 
Signature · 1 line
2.7.4
maintainer
#1 lore raw
Quoted 13 lines · Tal Shorer

Initialize wq_unbound_cpumask to exclude cpus that were isolated by the cmdline's isolcpus parameter.

---
 kernel/workqueue.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/workqueue.c +0 -0
index ca937b0..25b351d 100644

kernel/workqueue.c int __init workqueue_init_early(void) · line 5549

Quoted 5 lines
kernel/workqueue.c +1 -1
@@ -5546,7 +5546,7 @@ int __init workqueue_init_early(void)
 	WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long));
 
 	BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL));
-	cpumask_copy(wq_unbound_cpumask, cpu_possible_mask);
+	cpumask_andnot(wq_unbound_cpumask, cpu_possible_mask, cpu_isolated_map);

Don't we also wanna apply the same masking when wq_unbound_cpumask is being updated after boot?

Thanks.

Signature · 1 line
tejun

A message that is not in the archive (sent off-list or not yet received): CAFDyS3Mgt=Vt-sJXVL_pY_DwrC2dDTDQ0jMNyAbCwvLptu88Sw@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#4 lore raw
Quoted 25 lines · Tejun Heo
Quoted 24 lines · Tal Shorer
Quoted 23 lines · Tejun Heo
Quoted 13 lines · Tal Shorer

Initialize wq_unbound_cpumask to exclude cpus that were isolated by the cmdline's isolcpus parameter.

---
 kernel/workqueue.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/workqueue.c +0 -0
index ca937b0..25b351d 100644

kernel/workqueue.c int __init workqueue_init_early(void) · line 5549

Quoted 5 lines
kernel/workqueue.c +1 -1
@@ -5546,7 +5546,7 @@ int __init workqueue_init_early(void)
      WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long));
 
      BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL));
-     cpumask_copy(wq_unbound_cpumask, cpu_possible_mask);
+     cpumask_andnot(wq_unbound_cpumask, cpu_possible_mask, cpu_isolated_map);

Don't we also wanna apply the same masking when wq_unbound_cpumask is being updated after boot?

Quoted 9 lines
Quoted 2 lines

I think that if the user actively enables unbound works on isolated cpus, the kernel should allow that.

(Please restore cc list when you reply.)

Do we have a consistent behavior around this? Are there different examples where isolcpus interact with other dynamically configurable parameters?

sched_setaffinity comes to mind, which obviously bypasses isolcpus. Running git grep, other uses of cpu_isolated_map are just its initialization and its use and the initialization of non_isolated_cpus.

Quoted 4 lines

Thanks.

--
tejun

maintainer
#5 lore raw

Hello,

Quoted 8 lines · Tal Shorer
Quoted 4 lines

Do we have a consistent behavior around this? Are there different examples where isolcpus interact with other dynamically configurable parameters?

sched_setaffinity comes to mind, which obviously bypasses isolcpus. Running git grep, other uses of cpu_isolated_map are just its initialization and its use and the initialization of non_isolated_cpus.

Can you add a comment explaining that it's intentional that we only follow isolcpus during boot and allow later overrides?

Thanks.

Signature · 1 line
tejun
author
#6 lore raw

PATCH v2 1/1 workqueue: respect isolated cpus when queueing an unbound work

Initialize wq_unbound_cpumask to exclude cpus that were isolated by the cmdline's isolcpus parameter.

---
 kernel/workqueue.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/workqueue.c +5 -1
index ca937b0..0add5d4 100644
@@ -4947,6 +4947,10 @@ int workqueue_set_unbound_cpumask(cpumask_var_t cpumask)
 	if (!zalloc_cpumask_var(&saved_cpumask, GFP_KERNEL))
 		return -ENOMEM;
 
+	/*
+	 * Not excluding isolated cpus on purpose.
+	 * If the user wishes to include them, we allow that.
+	 */
 	cpumask_and(cpumask, cpumask, cpu_possible_mask);
 	if (!cpumask_empty(cpumask)) {
 		apply_wqattrs_lock();
@@ -5546,7 +5550,7 @@ int __init workqueue_init_early(void)
 	WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long));
 
 	BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL));
-	cpumask_copy(wq_unbound_cpumask, cpu_possible_mask);
+	cpumask_andnot(wq_unbound_cpumask, cpu_possible_mask, cpu_isolated_map);
 
 	pwq_cache = KMEM_CACHE(pool_workqueue, SLAB_PANIC);
 
Signature · 1 line
2.7.4
maintainer Applied
#7 lore raw
Quoted 4 lines · Tal Shorer

Initialize wq_unbound_cpumask to exclude cpus that were isolated by the cmdline's isolcpus parameter.

Applied to wq/for-4.15.

Thanks.

Signature · 1 line
tejun