dma-buf/dma-fence: Prevent misordering for error checks
Subsystems (2)
Philipp Stanner
maintainerPATCH v4 1/1 dma-buf/dma-fence: Prevent misordering for error checks
Many users of dma_fence do things like:
// Thread Adma_fence_set_error(f, -EINVAL);dma_fence_signal(f); // <- test_and_set_bit() orders against the above.
// Thread B
if (dma_fence_is_signaled(f)) // <- Does test_bit(), no ordering.
pr_info("err: %i\n", f->error); // <- Can be loaded first.There is no memory ordering between reading of the signaled-bit and reading of the error code. This can result in outdated, wrong error codes being evaluated by the user. That's a significant bug when we consider that fences' purpose is to propagate the state of jobs running on hardware.
dma_fence_get_status() does take the fence lock, presumably to ensure ordering, but it is broken, too, because the read misordering can simply happen within the critical section.
It's not entirely clear since when this bug exists, but at least since dma_fence_get_status() was added, which is why its birth commit is marked as the Fixes commit here.
Ensure ordering for everyone evaluating the error code centrally by using test_bit_acquire() in dma_fence_test_signaled_flag().
---
Changes in v4:
- Keep the hint at the function being opportunistic. (Christian)
Changes in v3:
- Be more precise in code comments and commit message about how the ordering works. (Gary)
- Update the function's documentation to inform about ordering semantics.
--- include/linux/dma-fence.h | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
include/linux/dma-fence.h +11 -4
@@ -487,16 +487,23 @@ const char __rcu *dma_fence_timeline_name(struct dma_fence *fence);* with the fence in any way. The user must, therefore, ensure through other * means that fences get signaled eventually. * -* This function uses test_bit(), which is thread-safe. Naturally, this function -* should be used opportunistically; a fence could get signaled at any moment -* after the check is done. +* Naturally, this function should be used opportunistically; a fence could get +* signaled at any moment after the check is done. +* +* This function provides acquire ordering. This means that all stores +* preceding the signaling of a fence are visible in correct order to users of +* this function. * * Return: true if signaled, false otherwise. */ static inline bool dma_fence_test_signaled_flag (struct dma_fence *fence) { -return test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); +/* +* The release counterpart is the test_and_set_bit() in +* dma_fence_signal_timestamp_locked(). +*/ +return test_bit_acquire (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); }/**
Signature · 1 line
2.56.0
Quoted 24 lines · Philipp Stanner
Many users of
dma_fencedo things like:// Thread A
dma_fence_set_error(f, -EINVAL);dma_fence_signal(f); // <-test_and_set_bit()orders against the above.// Thread B
if (dma_fence_is_signaled(f)) // <- Doestest_bit(), no ordering.pr_info("err: %i\n", f->error); // <- Can be loaded first.There is no memory ordering between reading of the signaled-bit and reading of the error code. This can result in outdated, wrong error codes being evaluated by the user. That's a significant bug when we consider that fences' purpose is to propagate the state of jobs running on hardware.
dma_fence_get_status()does take the fence lock, presumably to ensure ordering, but it is broken, too, because the read misordering can simply happen within the critical section.It's not entirely clear since when this bug exists, but at least since
dma_fence_get_status()was added, which is why its birth commit is marked as the Fixes commit here.
Quoted 6 lines
Ensure ordering for everyone evaluating the error code centrally by using
test_bit_acquire()indma_fence_test_signaled_flag().Fixes d6c99f4bf093 ("dma-fence: Wrap querying the fence->status")Signed-off-by Philipp Stanner <[email protected]>
Quoted 45 lines
---
Changes in v4:
- Keep the hint at the function being opportunistic. (Christian)Changes in v3:
- Be more precise in code comments and commit message about how the ordering works. (Gary)
- Update the function's documentation to inform about ordering semantics.--- include/linux/dma-fence.h | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)include/linux/dma-fence.h +11
-4@@ -487,16 +487,23 @@ const char __rcu *dma_fence_timeline_name(struct dma_fence *fence);* with the fence in any way. The user must, therefore, ensure through other * means that fences get signaled eventually. * -* This function uses test_bit(), which is thread-safe. Naturally, this function -* should be used opportunistically; a fence could get signaled at any moment -* after the check is done. +* Naturally, this function should be used opportunistically; a fence could get +* signaled at any moment after the check is done. +* +* This function provides acquire ordering. This means that all stores +* preceding the signaling of a fence are visible in correct order to users of +* this function. * * Return: true if signaled, false otherwise. */ static inline bool dma_fence_test_signaled_flag (struct dma_fence *fence) { -return test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); +/* +* The release counterpart is the test_and_set_bit() in +* dma_fence_signal_timestamp_locked(). +*/ +return test_bit_acquire (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); }/**
Quoted 9 lines · Christian König
Quoted 6 lines · Philipp Stanner
Ensure ordering for everyone evaluating the error code centrally by using
test_bit_acquire()indma_fence_test_signaled_flag().Fixes d6c99f4bf093 ("dma-fence: Wrap querying the fence->status")Signed-off-by Philipp Stanner <[email protected]>Reviewed-by Christian König <[email protected]>
Pushed it to misc-fixes. Thx.
P.