workqueues: change cancel_work_sync() to clear work->data
PATCH 1/3 workqueues: change cancel_work_sync() to clear work->data
In short: change cancel_work_sync(work) to mark this work as "never queued" upon return.
When cancel_work_sync(work) succeeds, we know that this work can't be queued or running, and since we own WORK_STRUCT_PENDING nobody can change the bits in work->data under us. This means we can also clear the "cwq" part along with _PENDING bit lockless before return, unless the work is queued nobody can assume get_wq_data() is stable even under cwq->lock.
This change can speedup the subsequent cancel/flush requests, and as Dmitry pointed out this simplifies the usage of work_struct's which can be queued on different workqueues. Consider this pseudo code from the input subsystem:
struct workqueue_struct *WQ;
struct work_struct *WORK;
for (;;) {
WQ = create_workqueue();
...
if (condition())
queue_work(WQ, WORK);
...
cancel_work_sync(WORK);
destroy_workqueue(WQ);
}If condition() returns T and then F, cancel_work_sync() will crash the kernel because WORK->data still points to the already destroyed workqueue. With this patch the code like above becomes correct.
---
kernel/workqueue.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
kernel/workqueue.c renamed from kernel/workqueue.c~1_CANCEL_CLEAR_WQ +11 -1
@@ -229,6 +229,16 @@ static inline void set_wq_data(struct woatomic_long_set (&work->data, new); } +/* +* Clear WORK_STRUCT_PENDING and the workqueue on which it was queued. +*/ +static inline void clear_wq_data (struct work_struct *work) +{ +unsigned long flags = *work_data_bits (work) & + (1UL <<WORK_STRUCT_STATIC ); +atomic_long_set (&work->data, flags); +} +static inline struct cpu_workqueue_struct *get_wq_data (struct work_struct *work) {
@@ -671,7 +681,7 @@ static int __cancel_work_timer(struct wowait_on_work (work); }while (unlikely (ret <0 )); -work_clear_pending (work); +clear_wq_data (work);return ret; }
Tejun Heo
maintainerHello,
Quoted 27 lines · Oleg Nesterov
In short: change
cancel_work_sync(work)to mark this work as "never queued" upon return.When
cancel_work_sync(work)succeeds, we know that this work can't be queued or running, and since we ownWORK_STRUCT_PENDINGnobody can change the bits inwork->dataunder us. This means we can also clear the "cwq" part along with_PENDINGbit lockless before return, unless the work is queued nobody can assumeget_wq_data()is stable even undercwq->lock.This change can speedup the subsequent cancel/flush requests, and as Dmitry pointed out this simplifies the usage of work_struct's which can be queued on different workqueues. Consider this pseudo code from the input subsystem:
struct workqueue_struct *WQ; struct work_struct *WORK; for (;;) { WQ = create_workqueue(); ... if (condition()) queue_work(WQ, WORK); ... cancel_work_sync(WORK); destroy_workqueue(WQ); }
Quoted 6 lines
If
condition()returns T and then F,cancel_work_sync()will crash the kernel becauseWORK->datastill points to the already destroyed workqueue. With this patch the code like above becomes correct.Suggested-by Dmitry Torokhov <[email protected]>Signed-off-by Oleg Nesterov <[email protected]>
Yeap, generally looks good to me and I've been doing similar things to implement non-reentrant workqueues (records the last cpu a work was on in work->dataa after the work is dispatched to a worker thread). I'll add this to the series.
Thank you.
Signature · 1 line
tejun
Quoted 27 lines · Oleg Nesterov
In short: change
cancel_work_sync(work)to mark this work as "never queued" upon return.When
cancel_work_sync(work)succeeds, we know that this work can't be queued or running, and since we ownWORK_STRUCT_PENDINGnobody can change the bits inwork->dataunder us. This means we can also clear the "cwq" part along with_PENDINGbit lockless before return, unless the work is queued nobody can assumeget_wq_data()is stable even undercwq->lock.This change can speedup the subsequent cancel/flush requests, and as Dmitry pointed out this simplifies the usage of work_struct's which can be queued on different workqueues. Consider this pseudo code from the input subsystem:
struct workqueue_struct *WQ; struct work_struct *WORK; for (;;) { WQ = create_workqueue(); ... if (condition()) queue_work(WQ, WORK); ... cancel_work_sync(WORK); destroy_workqueue(WQ); }
Quoted 4 lines
If
condition()returns T and then F,cancel_work_sync()will crash the kernel becauseWORK->datastill points to the already destroyed workqueue. With this patch the code like above becomes correct.
Very nice, thank you for making the change.
Quoted 36 lines
Suggested-by Dmitry Torokhov <[email protected]>Signed-off-by Oleg Nesterov <[email protected]>---
kernel/workqueue.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)kernel/workqueue.c renamed from kernel/workqueue.c~1_CANCEL_CLEAR_WQ +11
-1@@ -229,6 +229,16 @@ static inline void set_wq_data(struct woatomic_long_set (&work->data, new); } +/* +* Clear WORK_STRUCT_PENDING and the workqueue on which it was queued. +*/ +static inline void clear_wq_data (struct work_struct *work) +{ +unsigned long flags = *work_data_bits (work) & + (1UL <<WORK_STRUCT_STATIC ); +atomic_long_set (&work->data, flags); +} +static inline struct cpu_workqueue_struct *get_wq_data (struct work_struct *work) {@@ -671,7 +681,7 @@ static int __cancel_work_timer(struct wowait_on_work (work); }while (unlikely (ret <0 )); -work_clear_pending (work); +clear_wq_data (work);return ret; }
Signature · 1 line
Dmitry