Skip to contentSkip to conversation

workqueue: Document (some) memory-ordering properties of {queue,schedule}_work()

PATCH Superseded Andrea Parri 3 participants 4 messages Jan 18, 2020 – Jan 20, 2020 Workqueue

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 workqueue: Document (some) memory-ordering properties of {queue,schedule}_work()

It's desirable to be able to rely on the following property: All stores preceding (in program order) a call to a successful queue_work() will be visible from the CPU which will execute the queued work by the time such work executes, e.g.,

{ x is initially 0 }

CPU0 CPU1

WRITE_ONCE(x, 1); [ "work" is being executed ]
r0 = queue_work(wq, work); r1 = READ_ONCE(x);

  Forbids: r0 == true && r1 == 0

The current implementation of queue_work() provides such memory-ordering property:

- In __queue_work(), the ->lock spinlock is acquired.

- On the other side, in worker_thread(), this same ->lock is held when dequeueing work.

So the locking ordering makes things work out.

Add this property to the DocBook headers of {queue,schedule}_work().

---
 include/linux/workqueue.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
include/linux/workqueue.h +16 -0
index 4261d1c6e87b1..4fef6c38b0536 100644
@@ -487,6 +487,19 @@ extern void wq_worker_comm(char *buf, size_t size, struct task_struct *task);
  *
  * We queue the work to the CPU on which it was submitted, but if the CPU dies
  * it can be processed by another CPU.
+ *
+ * Memory-ordering properties:  If it returns %true, guarantees that all stores
+ * preceding the call to queue_work() in the program order will be visible from
+ * the CPU which will execute @work by the time such work executes, e.g.,
+ *
+ * { x is initially 0 }
+ *
+ *   CPU0				CPU1
+ *
+ *   WRITE_ONCE(x, 1);			[ @work is being executed ]
+ *   r0 = queue_work(wq, work);		  r1 = READ_ONCE(x);
+ *
+ * Forbids: r0 == true && r1 == 0
  */
 static inline bool queue_work(struct workqueue_struct *wq,
 			      struct work_struct *work)
@@ -546,6 +559,9 @@ static inline bool schedule_work_on(int cpu, struct work_struct *work)
  * This puts a job in the kernel-global workqueue if it was not already
  * queued and leaves it in the same position on the kernel-global
  * workqueue otherwise.
+ *
+ * Shares the same memory-ordering properties of queue_work(), c.f., the
+ * DocBook header of queue_work().
  */
 static inline bool schedule_work(struct work_struct *work)
 {
Signature · 1 line
2.24.0
#1 lore raw

Hi,

Quoted 57 lines · Andrea Parri

It's desirable to be able to rely on the following property: All stores preceding (in program order) a call to a successful queue_work() will be visible from the CPU which will execute the queued work by the time such work executes, e.g.,

{ x is initially 0 }

CPU0 CPU1

WRITE_ONCE(x, 1); [ "work" is being executed ]
r0 = queue_work(wq, work); r1 = READ_ONCE(x);

  Forbids: r0 == true && r1 == 0

The current implementation of queue_work() provides such memory-ordering property:

- In __queue_work(), the ->lock spinlock is acquired.

- On the other side, in worker_thread(), this same ->lock is held when dequeueing work.

So the locking ordering makes things work out.

Add this property to the DocBook headers of {queue,schedule}_work().

---
 include/linux/workqueue.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
include/linux/workqueue.h +13 -0
index 4261d1c6e87b1..4fef6c38b0536 100644
@@ -487,6 +487,19 @@ extern void wq_worker_comm(char *buf, size_t size, struct task_struct *task);
  *
  * We queue the work to the CPU on which it was submitted, but if the CPU dies
  * it can be processed by another CPU.
+ *
+ * Memory-ordering properties:  If it returns %true, guarantees that all stores
+ * preceding the call to queue_work() in the program order will be visible from
+ * the CPU which will execute @work by the time such work executes, e.g.,
+ *
+ * { x is initially 0 }
+ *
+ *   CPU0				CPU1
+ *
+ *   WRITE_ONCE(x, 1);			[ @work is being executed ]
+ *   r0 = queue_work(wq, work);		  r1 = READ_ONCE(x);
+ *
+ * Forbids: r0 == true && r1 == 0
  */
 static inline bool queue_work(struct workqueue_struct *wq,
 			      struct work_struct *work)

include/linux/workqueue.h static inline bool schedule_work_on(int cpu, struct work_struct *work) · line 563

Quoted 5 lines
include/linux/workqueue.h +2 -0
@@ -546,6 +559,9 @@ static inline bool schedule_work_on(int cpu, struct work_struct *work)
  * This puts a job in the kernel-global workqueue if it was not already
  * queued and leaves it in the same position on the kernel-global
  * workqueue otherwise.
+ *
+ * Shares the same memory-ordering properties of queue_work(), c.f., the

nit: cf. the

Quoted 5 lines
include/linux/workqueue.h +1 -0
+ * DocBook header of queue_work().
  */
 static inline bool schedule_work(struct work_struct *work)
 {
Signature · 1 line
~Randy
Acked-by on v1 1/1
#2 lore raw
In reply to Andrea Parri
Quoted 22 lines · Andrea Parri

It's desirable to be able to rely on the following property: All stores preceding (in program order) a call to a successful queue_work() will be visible from the CPU which will execute the queued work by the time such work executes, e.g.,

{ x is initially 0 }

CPU0 CPU1

WRITE_ONCE(x, 1); [ "work" is being executed ]
r0 = queue_work(wq, work); r1 = READ_ONCE(x);

  Forbids: r0 == true && r1 == 0

The current implementation of queue_work() provides such memory-ordering property:

- In __queue_work(), the ->lock spinlock is acquired.

- On the other side, in worker_thread(), this same ->lock is held when dequeueing work.

Quoted 6 lines

So the locking ordering makes things work out.

Add this property to the DocBook headers of {queue,schedule}_work().

An alternative to Randy's suggestion of dropping the comma following the "cf." is to just drop that whole phrase. I will let you and Randy work that one out, though. ;-)

Quoted 41 lines
---
 include/linux/workqueue.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
include/linux/workqueue.h +16 -0
index 4261d1c6e87b1..4fef6c38b0536 100644
@@ -487,6 +487,19 @@ extern void wq_worker_comm(char *buf, size_t size, struct task_struct *task);
  *
  * We queue the work to the CPU on which it was submitted, but if the CPU dies
  * it can be processed by another CPU.
+ *
+ * Memory-ordering properties:  If it returns %true, guarantees that all stores
+ * preceding the call to queue_work() in the program order will be visible from
+ * the CPU which will execute @work by the time such work executes, e.g.,
+ *
+ * { x is initially 0 }
+ *
+ *   CPU0				CPU1
+ *
+ *   WRITE_ONCE(x, 1);			[ @work is being executed ]
+ *   r0 = queue_work(wq, work);		  r1 = READ_ONCE(x);
+ *
+ * Forbids: r0 == true && r1 == 0
  */
 static inline bool queue_work(struct workqueue_struct *wq,
 			      struct work_struct *work)
@@ -546,6 +559,9 @@ static inline bool schedule_work_on(int cpu, struct work_struct *work)
  * This puts a job in the kernel-global workqueue if it was not already
  * queued and leaves it in the same position on the kernel-global
  * workqueue otherwise.
+ *
+ * Shares the same memory-ordering properties of queue_work(), c.f., the
+ * DocBook header of queue_work().
  */
 static inline bool schedule_work(struct work_struct *work)
 {
Signature · 1 line
2.24.0
#3 lore raw
Quoted 23 lines · Paul E. McKenney
Quoted 22 lines · Andrea Parri

It's desirable to be able to rely on the following property: All stores preceding (in program order) a call to a successful queue_work() will be visible from the CPU which will execute the queued work by the time such work executes, e.g.,

{ x is initially 0 }

CPU0 CPU1

WRITE_ONCE(x, 1); [ "work" is being executed ]
r0 = queue_work(wq, work); r1 = READ_ONCE(x);

  Forbids: r0 == true && r1 == 0

The current implementation of queue_work() provides such memory-ordering property:

- In __queue_work(), the ->lock spinlock is acquired.

- On the other side, in worker_thread(), this same ->lock is held when dequeueing work.

Quoted 8 lines
Quoted 6 lines

So the locking ordering makes things work out.

Add this property to the DocBook headers of {queue,schedule}_work().

Thanks!

Quoted 4 lines

An alternative to Randy's suggestion of dropping the comma following the "cf." is to just drop that whole phrase. I will let you and Randy work that one out, though. ;-)

Either way works for me.

I'd give Tejun and Lai some more time to review this and send a non-RFC with your Ack and this nit fixed later this week (unless I hear some objections).

Thanks,
Andrea