Skip to contentSkip to conversation

workqueues: implement flush_work()

PATCH In review Oleg Nesterov 2 participants 6 messages Jun 13, 2008 – Jun 15, 2008 Workqueue

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 workqueues: implement flush_work()

(on top of [PATCH] workqueues: insert_work: use "list_head *" instead of "int tail"
http://marc.info/?l=linux-kernel&m=121328944230175)

Most of users of flush_workqueue() can be changed to use cancel_work_sync(), but sometimes we really need to wait for the completion and cancelling is not an option. schedule_on_each_cpu() is good example.

Add the new helper, flush_work(work), which waits for the completion of the specific work_struct.

By its nature it requires that this work must not be re-queued, and thus its usage is limited. For example, this code

	queue_work(wq, work);
	/* WINDOW */
	queue_work(wq, work);

	flush_work(work);

is not right. What can happen in the WINDOW above is

- wq starts the execution of work->func()

- the caller migrates to another CPU

now, after the 2nd queue_work() this work is active on the previous CPU, and at the same time it is queued on another. We can fix this limitation, but in that case we have to iterate over all CPUs like wait_on_work() does, this will depreciate the advantages of this helper.

include/linux/workqueue.h renamed from include/linux/workqueue.h~WQ_2_FLUSH_WORK +2 -0
@@ -198,6 +198,8 @@ extern int keventd_up(void);
 extern void init_workqueues(void);
 int execute_in_process_context(work_func_t fn, struct execute_work *);
 
+extern int flush_work(struct work_struct *work);
+
 extern int cancel_work_sync(struct work_struct *work);
 
 /*
kernel/workqueue.c renamed from kernel/workqueue.c~WQ_2_FLUSH_WORK +46 -0

52 lines not on this page · raw message

#1 lore raw
Quoted 13 lines · Oleg Nesterov

Add the new helper, flush_work(work), which waits for the completion of the specific work_struct.

By its nature it requires that this work must not be re-queued, and thus its usage is limited. For example, this code

	queue_work(wq, work);
	/* WINDOW */
	queue_work(wq, work);

	flush_work(work);
Quoted 8 lines

is not right. What can happen in the WINDOW above is

- wq starts the execution of work->func()

- the caller migrates to another CPU

now, after the 2nd queue_work() this work is active on the previous CPU, and at the same time it is queued on another.

To clarify, in the scenario above, flush_work() behaves correctly. It "flushes" the result of the last queue_work() which is visible to the caller. We can add another simple helper,

	int flush_work_sync(work)
	{
		return flush_work(work) || wait_on_work(work);
	}

but I think it is not needed.

In short: flush_work() should be use with care when it is really needed.

Oleg.

#2 lore raw
In reply to Oleg Nesterov
Quoted 9 lines · Oleg Nesterov

(on top of [PATCH] workqueues: insert_work: use "list_head *" instead of "int tail"
http://marc.info/?l=linux-kernel&m=121328944230175)

Most of users of flush_workqueue() can be changed to use cancel_work_sync(), but sometimes we really need to wait for the completion and cancelling is not an option. schedule_on_each_cpu() is good example.

Add the new helper, flush_work(work), which waits for the completion of the specific work_struct.

This all looks right and better than current flush_, but... the main problem is that probably in 90% cases cancel_ + self-running a work function (if cancelled) should be both more efficient and safer wrt locking (what you convince me to, BTW). But, since there is no visible macro/wrapper for this, people will probably mostly choose what they can see in workqueue.h, so not necessarily the best thing.

Another question is if schedule_on_each_cpu() is really such a good example here: it seems these "xxx && yyy" examples could be faster, but I've lost track of this earlier thread.

BTW, flush_work() probably needs a lockdep annotation similar to flush_workqueue().

Otherwise this all looks OK to me.

Thanks,
Jarek P.

#3 lore raw
Quoted 9 lines · Jarek Poplawski
Quoted 7 lines · Oleg Nesterov

(on top of [PATCH] workqueues: insert_work: use "list_head *" instead of "int tail"
http://marc.info/?l=linux-kernel&m=121328944230175)

Most of users of flush_workqueue() can be changed to use cancel_work_sync(), but sometimes we really need to wait for the completion and cancelling is not an option. schedule_on_each_cpu() is good example.

Quoted 7 lines
Quoted 2 lines

Add the new helper, flush_work(work), which waits for the completion of the specific work_struct.

This all looks right and better than current flush_, but... the main problem is that probably in 90% cases cancel_ + self-running a work function (if cancelled) should be both more efficient and safer wrt locking (what you convince me to, BTW).

Yes, in most cases cancel_ is enough. And it is safer, note the limitations of flush_work(). Basically, flush_work(work) should be used when this work_struct can be queued only once.

Quoted 3 lines

Another question is if schedule_on_each_cpu() is really such a good example here: it seems these "xxx && yyy" examples could be faster, but I've lost track of this earlier thread.

schedule_on_each_cpu() can't use cancel_ + ->func(), the code should be executed on the remote CPU.

And note that flush_work() doesn't iterate over all CPUs, this is the reason why it is limited, but this also means it is faster than flush_work_sync() == flush_work() + wait_on_work().

Quoted 2 lines

BTW, flush_work() probably needs a lockdep annotation similar to flush_workqueue().

Yes I know... but I'd prefer to send another patch, I'm a bit paranoid when it comes to copy-and-pasting the code.

Quoted 1 line

Otherwise this all looks OK to me.

Thanks for review!

Oleg.

#4 lore raw
Quoted 1 line · Oleg Nesterov

On 06/14, Jarek Poplawski wrote:

...

Quoted 6 lines
Quoted 3 lines

Another question is if schedule_on_each_cpu() is really such a good example here: it seems these "xxx && yyy" examples could be faster, but I've lost track of this earlier thread.

schedule_on_each_cpu() can't use cancel_ + ->func(), the code should be executed on the remote CPU.

???
Actually I meant this "xxx && yyy" schedule_on_each_cpu():

http://marc.info/?l=linux-kernel&m=121329259203187&w=2

Regards,
Jarek P.

#5 lore raw
Quoted 8 lines · Jarek Poplawski
Quoted 1 line · Oleg Nesterov

On 06/14, Jarek Poplawski wrote:

...

Quoted 4 lines
Quoted 3 lines

Another question is if schedule_on_each_cpu() is really such a good example here: it seems these "xxx && yyy" examples could be faster, but I've lost track of this earlier thread.

Quoted 7 lines
Quoted 2 lines

schedule_on_each_cpu() can't use cancel_ + ->func(), the code should be executed on the remote CPU.

???
Actually I meant this "xxx && yyy" schedule_on_each_cpu():

http://marc.info/?l=linux-kernel&m=121329259203187&w=2

Ah, sorry, I misunderstood.

Yes, "xxx && yyy" needs only 1 wakeup, so it needs less CPU. But it also more complex, and it is not easy to generalize this.

Oleg.