dma-buf/dma-fence: Prevent misordering for error checks
Subsystems (3)
Philipp Stanner
maintainerPATCH 1/1 dma-buf/dma-fence: Prevent misordering for error checks
Many users of dma_fence do things like:
// Thread Adma_fence_set_error(f, -EINVAL);dma_fence_signal(f); // <- Takes lock, which orders against the above.
// Thread B
if (dma_fence_is_signaled(f)) // <- Does test_bit(), no ordering.
pr_info("err: %i\n", f->error); // <- Can be loaded first.There is no memory ordering between reading of the signaled-bit and reading of the error code. This can result in outdated, wrong error codes being evaluated by the user.
Though dma_fence_get_status() does take the fence lock, it is equally broken because the misordering can simply happen within the critical section.
It's not entirely clear since when this bug exists, but at least since dma_fence_get_status() was added, which is why its birth commit is marked as the Fixes commit here.
Ensure ordering for everyone evaluating the error code centrally by using test_bit_acquire() in dma_fence_test_signaled_flag().
---
Changes since the RFC:
- Order with test_bit_acquire(). (Gary)
- ^ also makes it easy to backport this for now.
- Try to document roughly what is even going on.
--- include/linux/dma-fence.h | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
include/linux/dma-fence.h +10 -1
@@ -496,7 +496,16 @@ const char __rcu *dma_fence_timeline_name(struct dma_fence *fence);static inline bool dma_fence_test_signaled_flag (struct dma_fence *fence) { -return test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); +// Best-effort ordering of the locklessly evaluated signaled-bit, +// especially so that reads of fence->error are not misordered. +// +// The release counterpart is typically ensured like this: +// a) Error codes must only be set before a fence signals and +// b) The fence's spinlock's barriers effective in dma_fence_signal() +// ensure ordering. +// c) For users of dma_fence_signal_locked(), the barrier implied in +// test_and_set_bit() should do the trick. +return test_bit_acquire (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); }/**
Signature · 1 line
2.56.0
Christian König
maintainerQuoted 45 lines · Philipp Stanner
Many users of
dma_fencedo things like:// Thread A
dma_fence_set_error(f, -EINVAL);dma_fence_signal(f); // <- Takes lock, which orders against the above.// Thread B
if (dma_fence_is_signaled(f)) // <- Doestest_bit(), no ordering.pr_info("err: %i\n", f->error); // <- Can be loaded first.There is no memory ordering between reading of the signaled-bit and reading of the error code. This can result in outdated, wrong error codes being evaluated by the user.
Though
dma_fence_get_status()does take the fence lock, it is equally broken because the misordering can simply happen within the critical section.It's not entirely clear since when this bug exists, but at least since
dma_fence_get_status()was added, which is why its birth commit is marked as the Fixes commit here.Ensure ordering for everyone evaluating the error code centrally by using
test_bit_acquire()indma_fence_test_signaled_flag().Fixes d6c99f4bf093 ("dma-fence: Wrap querying the fence->status")Signed-off-by Philipp Stanner <[email protected]>---
Changes since the RFC:
- Order withtest_bit_acquire(). (Gary)
- ^ also makes it easy to backport this for now.
- Try to document roughly what is even going on.--- include/linux/dma-fence.h | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)include/linux/dma-fence.h +0
-0@@ -496,7 +496,16 @@ const char __rcu *dma_fence_timeline_name(struct dma_fence *fence);static inline bool dma_fence_test_signaled_flag (struct dma_fence *fence) {
include/linux/dma-fence.h const char __rcu *dma_fence_timeline_name(struct dma_fence *fence); · line 507
Quoted 10 lines
include/linux/dma-fence.h +9
-1-return test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); +// Best-effort ordering of the locklessly evaluated signaled-bit, +// especially so that reads of fence->error are not misordered. +// +// The release counterpart is typically ensured like this: +// a) Error codes must only be set before a fence signals and +// b) The fence's spinlock's barriers effective in dma_fence_signal() +// ensure ordering. +// c) For users of dma_fence_signal_locked(), the barrier implied in +// test_and_set_bit() should do the trick.
That should probably be a C comment, e.g. /* ... */.
Apart from that really good catch.
Regards,
Christian.
Quoted 4 lines
include/linux/dma-fence.h +1
-0+return test_bit_acquire (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags); }/**
Quoted 10 lines · Christian König
Quoted 9 lines · Philipp Stanner
Many users of
dma_fencedo things like:// Thread A
dma_fence_set_error(f, -EINVAL);dma_fence_signal(f); // <- Takes lock, which orders against the above.// Thread B
if (dma_fence_is_signaled(f)) // <- Doestest_bit(), no ordering.pr_info("err: %i\n", f->error); // <- Can be loaded first.
[…]
Quoted 3 lines
include/linux/dma-fence.h const char __rcu *dma_fence_timeline_name(struct dma_fence *fence); · line 507
Quoted 1 line
include/linux/dma-fence.h +1
-0+// test_and_set_bit() should do the trick. That should probably be a C comment, e.g. /* ... */.
huh, how did that happen…-ETOOMUCHRUST
I provide a v2
Quoted 1 line
Apart from that really good catch.
Found without any LLM 8-)
I saw it while reworking Rust-fence to force users to always evaluate a fence's error code, i.e., you can only check whether a fence is signaled when you also check for the error.
dma_fence_is_signaled() incentivizes wrong behavior.
But btw. we really want to carefully rethink our beloved signaled-bit and its semantics. Mind all our past discussions about it.
This patch is more like a temporary hotfix. Much of our API is quite fragile.
Regards
Philipp