Skip to contentSkip to conversation

io_uring/rsrc: skip atomic refcount for uncloned buffers

PATCH Applied Caleb Sander Mateos 2 participants 5 messages Jun 19, 2025 – Jul 3, 2025 io_uring

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 io_uring/rsrc: skip atomic refcount for uncloned buffers

io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's reference count in case it has been cloned into another io_ring_ctx's registered buffer table. This is an expensive operation and unnecessary in the common case that the io_mapped_ubuf is only registered once. Load the reference count first and check whether it's 1. In that case, skip the atomic decrement and immediately free the io_mapped_ubuf.

---
 io_uring/rsrc.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
io_uring/rsrc.c +4 -2
index 94a9db030e0e..9a1f24a43035 100644
@@ -133,12 +133,14 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
 		kvfree(imu);
 }
 
 static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
 {
-	if (!refcount_dec_and_test(&imu->refs))
-		return;
+	if (unlikely(refcount_read(&imu->refs) > 1)) {
+		if (!refcount_dec_and_test(&imu->refs))
+			return;
+	}
 
 	if (imu->acct_pages)
 		io_unaccount_mem(ctx, imu->acct_pages);
 	imu->release(imu->priv);
 	io_free_imu(ctx, imu);
Signature · 1 line
2.45.2
#1 lore raw

Hi Jens,
Any concerns with this one? I thought it was a fairly straightforward optimization in the ublk zero-copy I/O path.

Thanks,
Caleb

Quoted 37 lines · Caleb Sander Mateos

io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's reference count in case it has been cloned into another io_ring_ctx's registered buffer table. This is an expensive operation and unnecessary in the common case that the io_mapped_ubuf is only registered once. Load the reference count first and check whether it's 1. In that case, skip the atomic decrement and immediately free the io_mapped_ubuf.

---
 io_uring/rsrc.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
io_uring/rsrc.c +4 -3
index 94a9db030e0e..9a1f24a43035 100644
@@ -133,12 +133,14 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
                kvfree(imu);
 }
 
 static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
 {
-       if (!refcount_dec_and_test(&imu->refs))
-               return;
+       if (unlikely(refcount_read(&imu->refs) > 1)) {
+               if (!refcount_dec_and_test(&imu->refs))
+                       return;
+       }
 
        if (imu->acct_pages)
                io_unaccount_mem(ctx, imu->acct_pages);
        imu->release(imu->priv);
        io_free_imu(ctx, imu);
--

2.45.2

maintainer
#2 lore raw
Quoted 3 lines · Caleb Sander Mateos

Hi Jens,
Any concerns with this one? I thought it was a fairly straightforward optimization in the ublk zero-copy I/O path.

Nope looks fine, I just have a largish backlog from being gone for 10 days. I'll queue it up for 6.17.

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 8 lines · Jens Axboe
Quoted 3 lines · Caleb Sander Mateos

Hi Jens,
Any concerns with this one? I thought it was a fairly straightforward optimization in the ublk zero-copy I/O path.

Nope looks fine, I just have a largish backlog from being gone for 10 days. I'll queue it up for 6.17.

No worries, I was away for a week too. Just wanted to make sure it hadn't fallen off the radar.

Thanks,
Caleb

maintainer Applied
#4 lore raw
In reply to Caleb Sander Mateos
Quoted 8 lines · Caleb Sander Mateos

io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's reference count in case it has been cloned into another io_ring_ctx's registered buffer table. This is an expensive operation and unnecessary in the common case that the io_mapped_ubuf is only registered once. Load the reference count first and check whether it's 1. In that case, skip the atomic decrement and immediately free the io_mapped_ubuf.

[...]

Applied, thanks!

[1/1] io_uring/rsrc: skip atomic refcount for uncloned buffers

      commit: daa01d954b13a178c216b6a91f8451a7b83b3bf6

Best regards,

Signature · 1 line
Jens Axboe