slow_work_execute() needs mb() before test_bit(SLOW_WORK_PENDING)
PATCH 1/1 slow_work_execute() needs mb() before test_bit(SLOW_WORK_PENDING)
slow_work_execute:
clear_bit_unlock(SLOW_WORK_EXECUTING, &work->flags);
if (test_bit(SLOW_WORK_PENDING, &work->flags) {clear_bit_unlock() implies release semantics, iow we have a one-way barrier before clear_bit(). But we need the mb() semantics after clear_bit(), before we test SLOW_WORK_PENDING. Otherwise we can miss SLOW_WORK_ENQ_DEFERRED if we race slow_work_enqueue().
kernel/slow-work.c renamed from kernel/slow-work.c~2_BITS_MB +2 -1
@@ -198,7 +198,8 @@ static bool slow_work_execute(void)if (very_slow)atomic_dec (&vslow_work_executing_count); -clear_bit_unlock (SLOW_WORK_EXECUTING , &work->flags); +clear_bit (SLOW_WORK_EXECUTING , &work->flags); +smp_mb__after_clear_bit ();/* if someone tried to enqueue the item whilst we were executing it, * then it'll be left unenqueued to avoid multiple threads trying to
Quoted 11 lines · Oleg Nesterov
slow_work_execute:clear_bit_unlock(SLOW_WORK_EXECUTING, &work->flags); if (test_bit(SLOW_WORK_PENDING, &work->flags) {
clear_bit_unlock()implies release semantics, iow we have a one-way barrier beforeclear_bit(). But we need themb()semantics afterclear_bit(), before we testSLOW_WORK_PENDING. Otherwise we can missSLOW_WORK_ENQ_DEFERREDif we raceslow_work_enqueue().
However, given that both clear_bit() and set_bit() use the same word, perhaps this is not possible.
But in that case I don't understand why do we need clear_bit_unlock(), not just clear_bit(), and how "mb is not needeed" could be derived from documentation.
Oleg.
Quoted 2 lines · Oleg Nesterov
clear_bit_unlock()implies release semantics, iow we have a one-way barrier beforeclear_bit().
That is correct. The stuff before the clear_bit_unlock() must not leak out past it. SLOW_WORK_EXECUTING is a lock around the execution of the work item.
Quoted 3 lines
But we need the
mb()semantics afterclear_bit(), before we testSLOW_WORK_PENDING. Otherwise we can missSLOW_WORK_ENQ_DEFERREDif we raceslow_work_enqueue().
But if you move the mb() to the other side, and reduce to clear_bit() you break the above non-reentrancy guarantee.
Quoted 2 lines
However, given that both
clear_bit()andset_bit()use the same word, perhaps this is not possible.
I would assume clear_bit() and set_bit() on different bits of the same word must interact as if they're on the same bit, with regard to atomiticity, but I don't know for certain that it is guaranteed.
Quoted 3 lines
But in that case I don't understand why do we need
clear_bit_unlock(), not justclear_bit(), and how "mb is not needeed" could be derived from documentation.
As mentioned above, SLOW_WORK_EXECUTING is a lock against multiple entrance to the execution function.
Perhaps I should amend the comments to make this clearer.
David
Quoted 9 lines · David Howells
Quoted 2 lines · Oleg Nesterov
clear_bit_unlock()implies release semantics, iow we have a one-way barrier beforeclear_bit().That is correct. The stuff before the
clear_bit_unlock()must not leak out past it.SLOW_WORK_EXECUTINGis a lock around the execution of the work item.
Quoted 6 lines
Quoted 3 lines
But we need the
mb()semantics afterclear_bit(), before we testSLOW_WORK_PENDING. Otherwise we can missSLOW_WORK_ENQ_DEFERREDif we raceslow_work_enqueue().But if you move the
mb()to the other side, and reduce toclear_bit()you break the above non-reentrancy guarantee.
Hmm. Can't understand... please see below.
Quoted 6 lines
Quoted 2 lines
However, given that both
clear_bit()andset_bit()use the same word, perhaps this is not possible.I would assume
clear_bit()andset_bit()on different bits of the same word must interact as if they're on the same bit, with regard to atomiticity, but I don't know for certain that it is guaranteed.
Yes, this was my concern. Reading the docs, I can't see how it is possible to "prove" that clear_bit(x, WORD) + test_bit(y, WORD) can't be "reversed".
Quoted 6 lines
Quoted 3 lines
But in that case I don't understand why do we need
clear_bit_unlock(), not justclear_bit(), and how "mb is not needeed" could be derived from documentation.As mentioned above,
SLOW_WORK_EXECUTINGis a lock against multiple entrance to the execution function.
Yes I see.
But why do we need the barrier before clear_bit(SLOW_WORK_EXECUTING) ? We do have one after test_and_clear_bit(SLOW_WORK_PENDING) above, and it should be enough, no?
Oleg.
Quoted 3 lines · Oleg Nesterov
But why do we need the barrier before
clear_bit(SLOW_WORK_EXECUTING)? We do have one aftertest_and_clear_bit(SLOW_WORK_PENDING)above, and it should be enough, no?
No. There lock is covering work->ops->execute(work) too, and that is after the clearance of SLOW_WORK_PENDING. The UNLOCK-class barrier must go between execution and clearance of the execution lock bit.
David
Quoted 10 lines · David Howells
Quoted 3 lines · Oleg Nesterov
But why do we need the barrier before
clear_bit(SLOW_WORK_EXECUTING)? We do have one aftertest_and_clear_bit(SLOW_WORK_PENDING)above, and it should be enough, no?No. There lock is covering
work->ops->execute(work)too, and that is after the clearance ofSLOW_WORK_PENDING. The UNLOCK-class barrier must go between execution and clearance of the execution lock bit.
Ah. Now I see.
It is still not clear to me whether we need a barrier after clear_bit_unlock.
Thanks David!
Oleg.