io_uring/rsrc: skip atomic refcount for uncloned buffers
PATCH 1/1 io_uring/rsrc: skip atomic refcount for uncloned buffers
io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's reference count in case it has been cloned into another io_ring_ctx's registered buffer table. This is an expensive operation and unnecessary in the common case that the io_mapped_ubuf is only registered once. Load the reference count first and check whether it's 1. In that case, skip the atomic decrement and immediately free the io_mapped_ubuf.
--- io_uring/rsrc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
io_uring/rsrc.c +4 -2
@@ -133,12 +133,14 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)kvfree (imu); }static void io_buffer_unmap (struct io_ring_ctx *ctx,struct io_mapped_ubuf *imu) { -if (!refcount_dec_and_test (&imu->refs)) -return ; +if (unlikely (refcount_read (&imu->refs) >1 )) { +if (!refcount_dec_and_test (&imu->refs)) +return ; + }if (imu->acct_pages)io_unaccount_mem (ctx, imu->acct_pages); imu->release (imu->priv);io_free_imu (ctx, imu);
Signature · 1 line
2.45.2
Hi Jens,
Any concerns with this one? I thought it was a fairly straightforward optimization in the ublk zero-copy I/O path.
Thanks,
Caleb
Quoted 37 lines · Caleb Sander Mateos
io_buffer_unmap()performs an atomic decrement of the io_mapped_ubuf's reference count in case it has been cloned into another io_ring_ctx's registered buffer table. This is an expensive operation and unnecessary in the common case that theio_mapped_ubufis only registered once. Load the reference count first and check whether it's 1. In that case, skip the atomic decrement and immediately free theio_mapped_ubuf.Signed-off-by Caleb Sander Mateos <[email protected]>--- io_uring/rsrc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)io_uring/rsrc.c +4
-3@@ -133,12 +133,14 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)kvfree (imu); }static void io_buffer_unmap (struct io_ring_ctx *ctx,struct io_mapped_ubuf *imu) { -if (!refcount_dec_and_test (&imu->refs)) -return ; +if (unlikely (refcount_read (&imu->refs) >1 )) { +if (!refcount_dec_and_test (&imu->refs)) +return ; + }if (imu->acct_pages)io_unaccount_mem (ctx, imu->acct_pages); imu->release (imu->priv);io_free_imu (ctx, imu); --2.45.2
Jens Axboe
maintainerQuoted 3 lines · Caleb Sander Mateos
Hi Jens,
Any concerns with this one? I thought it was a fairly straightforward optimization in the ublk zero-copy I/O path.
Nope looks fine, I just have a largish backlog from being gone for 10 days. I'll queue it up for 6.17.
Signature · 1 line
Jens Axboe
Quoted 8 lines · Jens Axboe
Quoted 3 lines · Caleb Sander Mateos
Hi Jens,
Any concerns with this one? I thought it was a fairly straightforward optimization in the ublk zero-copy I/O path.Nope looks fine, I just have a largish backlog from being gone for 10 days. I'll queue it up for 6.17.
No worries, I was away for a week too. Just wanted to make sure it hadn't fallen off the radar.
Thanks,
Caleb
Quoted 8 lines · Caleb Sander Mateos
io_buffer_unmap()performs an atomic decrement of the io_mapped_ubuf's reference count in case it has been cloned into another io_ring_ctx's registered buffer table. This is an expensive operation and unnecessary in the common case that theio_mapped_ubufis only registered once. Load the reference count first and check whether it's 1. In that case, skip the atomic decrement and immediately free theio_mapped_ubuf.[...]
Applied, thanks!
[1/1] io_uring/rsrc: skip atomic refcount for uncloned buffers
commit: daa01d954b13a178c216b6a91f8451a7b83b3bf6
Best regards,
Signature · 1 line
Jens Axboe