Skip to contentSkip to conversation

io_uring: add kernel-managed buffer rings

PATCH Superseded Joanne Koong 6 participants 58 messages Feb 10 – Mar 2

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover io_uring: add kernel-managed buffer rings

Currently, io_uring buffer rings require the application to allocate and manage the backing buffers. This series introduces kernel-managed buffer rings, where the kernel allocates and manages the buffers on behalf of the application.

This is split out from the fuse over io_uring series in [1], which needs the kernel to own and manage buffers shared between the fuse server and the kernel.

This series is on top of the for-next branch in Jens' io-uring tree. The corresponding liburing changes are in [2] and will be submitted after the changes in this patchset are accepted.

Thanks,
Joanne

[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
[2] https://github.com/joannekoong/liburing/tree/kmbuf

Changelog
---------
Changes since [1]:
* add "if (bl)" check for recycling API (Bernd)
* check mul overflow, use GFP_USER, use PTR as return type (Christoph)
* fix bl->ring leak (me)

Joanne Koong (11):
io_uring/kbuf: refactor io_register_pbuf_ring() logic into generic
helpers
io_uring/kbuf: rename io_unregister_pbuf_ring() to
io_unregister_buf_ring()
io_uring/kbuf: add support for kernel-managed buffer rings io_uring/kbuf: add mmap support for kernel-managed buffer rings io_uring/kbuf: support kernel-managed buffer rings in buffer selection io_uring/kbuf: add buffer ring pinning/unpinning io_uring/kbuf: add recycling for kernel managed buffer rings io_uring/kbuf: add io_uring_is_kmbuf_ring()
io_uring/kbuf: export io_ring_buffer_select() io_uring/kbuf: return buffer id in buffer selection io_uring/cmd: set selected buffer index in __io_uring_cmd_done()

 include/linux/io_uring/cmd.h   |  53 ++++-
 include/linux/io_uring_types.h |  10 +-
 include/uapi/linux/io_uring.h  |  17 +-
 io_uring/kbuf.c                | 365 +++++++++++++++++++++++++++------
 io_uring/kbuf.h                |  19 +-
 io_uring/memmap.c              | 116 ++++++++++-
 io_uring/memmap.h              |   4 +
 io_uring/register.c            |   9 +-
 io_uring/uring_cmd.c           |   6 +-
 9 files changed, 526 insertions(+), 73 deletions(-)
Signature · 1 line
2.47.3
#1 lore raw

PATCH 1/11 io_uring/kbuf: refactor io_register_pbuf_ring() logic into generic helpers

Refactor the logic in io_register_pbuf_ring() into generic helpers:
- io_copy_and_validate_buf_reg(): Copy out user arg and validate user arg and buffer registration parameters
- io_alloc_new_buffer_list(): Allocate and initialize a new buffer list for the given buffer group ID
- io_setup_pbuf_ring(): Sets up the physical buffer ring region and handles memory mapping for provided buffer rings

This is a preparatory change for upcoming kernel-managed buffer ring support which will need to reuse some of these helpers.

---
 io_uring/kbuf.c | 129 +++++++++++++++++++++++++++++++-----------------
 1 file changed, 85 insertions(+), 44 deletions(-)
io_uring/kbuf.c +85 -44
index 67d4fe576473..850b836f32ee 100644
@@ -596,55 +596,73 @@ int io_manage_buffers_legacy(struct io_kiocb *req, unsigned int issue_flags)
 	return IOU_COMPLETE;
 }
 
-int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
+static int io_copy_and_validate_buf_reg(const void __user *arg,
+					struct io_uring_buf_reg *reg,
+					unsigned int permitted_flags)
 {
-	struct io_uring_buf_reg reg;
-	struct io_buffer_list *bl;
-	struct io_uring_region_desc rd;
-	struct io_uring_buf_ring *br;
-	unsigned long mmap_offset;
-	unsigned long ring_size;
-	int ret;
-
-	lockdep_assert_held(&ctx->uring_lock);
-
-	if (copy_from_user(&reg, arg, sizeof(reg)))
+	if (copy_from_user(reg, arg, sizeof(*reg)))
 		return -EFAULT;
-	if (!mem_is_zero(reg.resv, sizeof(reg.resv)))
+
+	if (!mem_is_zero(reg->resv, sizeof(reg->resv)))
 		return -EINVAL;
-	if (reg.flags & ~(IOU_PBUF_RING_MMAP | IOU_PBUF_RING_INC))
+	if (reg->flags & ~permitted_flags)
 		return -EINVAL;
-	if (!is_power_of_2(reg.ring_entries))
+	if (!is_power_of_2(reg->ring_entries))
 		return -EINVAL;
 	/* cannot disambiguate full vs empty due to head/tail size */
-	if (reg.ring_entries >= 65536)
+	if (reg->ring_entries >= 65536)
 		return -EINVAL;
+	return 0;
+}
 
-	bl = io_buffer_get_list(ctx, reg.bgid);
-	if (bl) {
+static struct io_buffer_list *
+io_alloc_new_buffer_list(struct io_ring_ctx *ctx,
+			 const struct io_uring_buf_reg *reg)
+{
+	struct io_buffer_list *list;
+
+	list = io_buffer_get_list(ctx, reg->bgid);
+	if (list) {
 		/* if mapped buffer ring OR classic exists, don't allow */
-		if (bl->flags & IOBL_BUF_RING || !list_empty(&bl->buf_list))
-			return -EEXIST;
-		io_destroy_bl(ctx, bl);
+		if (list->flags & IOBL_BUF_RING || !list_empty(&list->buf_list))
+			return ERR_PTR(-EEXIST);
+		io_destroy_bl(ctx, list);
 	}
 
-	bl = kzalloc(sizeof(*bl), GFP_KERNEL_ACCOUNT);
-	if (!bl)
-		return -ENOMEM;
+	list = kzalloc(sizeof(*list), GFP_KERNEL_ACCOUNT);
+	if (!list)
+		return ERR_PTR(-ENOMEM);
+
+	list->nr_entries = reg->ring_entries;
+	list->mask = reg->ring_entries - 1;
+	list->flags = IOBL_BUF_RING;
+
+	return list;
+}
+
+static int io_setup_pbuf_ring(struct io_ring_ctx *ctx,
+			      const struct io_uring_buf_reg *reg,
+			      struct io_buffer_list *bl)
+{
+	struct io_uring_region_desc rd;
+	unsigned long mmap_offset;
+	unsigned long ring_size;
+	int ret;
 
-	mmap_offset = (unsigned long)reg.bgid << IORING_OFF_PBUF_SHIFT;
-	ring_size = flex_array_size(br, bufs, reg.ring_entries);
+	mmap_offset = (unsigned long)reg->bgid << IORING_OFF_PBUF_SHIFT;
+	ring_size = flex_array_size(bl->buf_ring, bufs, reg->ring_entries);
 
 	memset(&rd, 0, sizeof(rd));
 	rd.size = PAGE_ALIGN(ring_size);
-	if (!(reg.flags & IOU_PBUF_RING_MMAP)) {
-		rd.user_addr = reg.ring_addr;
+	if (!(reg->flags & IOU_PBUF_RING_MMAP)) {
+		rd.user_addr = reg->ring_addr;
 		rd.flags |= IORING_MEM_REGION_TYPE_USER;
 	}
+
 	ret = io_create_region(ctx, &bl->region, &rd, mmap_offset);
 	if (ret)
-		goto fail;
-	br = io_region_get_ptr(&bl->region);
+		return ret;
+	bl->buf_ring = io_region_get_ptr(&bl->region);
 
 #ifdef SHM_COLOUR
 	/*
@@ -656,25 +674,48 @@ int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
 	 * should use IOU_PBUF_RING_MMAP instead, and liburing will handle
 	 * this transparently.
 	 */
-	if (!(reg.flags & IOU_PBUF_RING_MMAP) &&
-	    ((reg.ring_addr | (unsigned long)br) & (SHM_COLOUR - 1))) {
-		ret = -EINVAL;
-		goto fail;
+	if (!(reg->flags & IOU_PBUF_RING_MMAP) &&
+	    ((reg->ring_addr | (unsigned long)bl->buf_ring) &
+	     (SHM_COLOUR - 1))) {
+		io_free_region(ctx->user, &bl->region);
+		return -EINVAL;
 	}
 #endif
 
-	bl->nr_entries = reg.ring_entries;
-	bl->mask = reg.ring_entries - 1;
-	bl->flags |= IOBL_BUF_RING;
-	bl->buf_ring = br;
-	if (reg.flags & IOU_PBUF_RING_INC)
+	if (reg->flags & IOU_PBUF_RING_INC)
 		bl->flags |= IOBL_INC;
+
+	return 0;
+}
+
+int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
+{
+	unsigned int permitted_flags;
+	struct io_uring_buf_reg reg;
+	struct io_buffer_list *bl;
+	int ret;
+
+	lockdep_assert_held(&ctx->uring_lock);
+
+	permitted_flags = IOU_PBUF_RING_MMAP | IOU_PBUF_RING_INC;
+	ret = io_copy_and_validate_buf_reg(arg, &reg, permitted_flags);
+	if (ret)
+		return ret;
+
+	bl = io_alloc_new_buffer_list(ctx, &reg);
+	if (IS_ERR(bl))
+		return PTR_ERR(bl);
+
+	ret = io_setup_pbuf_ring(ctx, &reg, bl);
+	if (ret) {
+		kfree(bl);
+		return ret;
+	}
+
 	ret = io_buffer_add_list(ctx, bl, reg.bgid);
-	if (!ret)
-		return 0;
-fail:
-	io_free_region(ctx->user, &bl->region);
-	kfree(bl);
+	if (ret)
+		io_put_bl(ctx, bl);
+
 	return ret;
 }
 
Signature · 1 line
2.47.3
#2 lore raw

PATCH 2/11 io_uring/kbuf: rename io_unregister_pbuf_ring() to io_unregister_buf_ring()

In reply to Joanne Koong

Use the more generic name io_unregister_buf_ring() as this function will be used for unregistering both provided buffer rings and kernel-managed buffer rings.

This is a preparatory change for upcoming kernel-managed buffer ring support.

---
 io_uring/kbuf.c     | 2 +-
 io_uring/kbuf.h     | 2 +-
 io_uring/register.c | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
io_uring/kbuf.c +1 -1
index 850b836f32ee..aa9b70b72db4 100644
@@ -719,7 +719,7 @@ int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
 	return ret;
 }
 
-int io_unregister_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
+int io_unregister_buf_ring(struct io_ring_ctx *ctx, void __user *arg)
 {
 	struct io_uring_buf_reg reg;
 	struct io_buffer_list *bl;
io_uring/kbuf.h +1 -1
index bf15e26520d3..40b44f4fdb15 100644
@@ -74,7 +74,7 @@ int io_provide_buffers_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe
 int io_manage_buffers_legacy(struct io_kiocb *req, unsigned int issue_flags);
 
 int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg);
-int io_unregister_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg);
+int io_unregister_buf_ring(struct io_ring_ctx *ctx, void __user *arg);
 int io_register_pbuf_status(struct io_ring_ctx *ctx, void __user *arg);
 
 bool io_kbuf_recycle_legacy(struct io_kiocb *req, unsigned issue_flags);
io_uring/register.c +1 -1
index 594b1f2ce875..0882cb34f851 100644
@@ -841,7 +841,7 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode,
 		ret = -EINVAL;
 		if (!arg || nr_args != 1)
 			break;
-		ret = io_unregister_pbuf_ring(ctx, arg);
+		ret = io_unregister_buf_ring(ctx, arg);
 		break;
 	case IORING_REGISTER_SYNC_CANCEL:
 		ret = -EINVAL;
Signature · 1 line
2.47.3
#3 lore raw

PATCH 3/11 io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Joanne Koong

Add support for kernel-managed buffer rings (kmbuf rings), which allow the kernel to allocate and manage the backing buffers for a buffer ring, rather than requiring the application to provide and manage them.

This introduces two new registration opcodes:
- IORING_REGISTER_KMBUF_RING: Register a kernel-managed buffer ring
- IORING_UNREGISTER_KMBUF_RING: Unregister a kernel-managed buffer ring

The existing io_uring_buf_reg structure is extended with a union to support both application-provided buffer rings (pbuf) and kernel-managed buffer rings (kmbuf):
- For pbuf rings: ring_addr specifies the user-provided ring address
- For kmbuf rings: buf_size specifies the size of each buffer. buf_size must be non-zero and page-aligned.

The implementation follows the same pattern as pbuf ring registration, reusing the validation and buffer list allocation helpers introduced in earlier refactoring. The IOBL_KERNEL_MANAGED flag marks buffer lists as kernel-managed for appropriate handling in the I/O path.

---
 include/uapi/linux/io_uring.h |  15 ++++-
 io_uring/kbuf.c               |  81 ++++++++++++++++++++++++-
 io_uring/kbuf.h               |   7 ++-
 io_uring/memmap.c             | 111 ++++++++++++++++++++++++++++++++++
 io_uring/memmap.h             |   4 ++
 io_uring/register.c           |   7 +++
 6 files changed, 219 insertions(+), 6 deletions(-)
include/uapi/linux/io_uring.h +13 -2
index fc473af6feb4..a0889c1744bd 100644
@@ -715,6 +715,10 @@ enum io_uring_register_op {
 	/* register bpf filtering programs */
 	IORING_REGISTER_BPF_FILTER		= 37,
 
+	/* register/unregister kernel-managed ring buffer group */
+	IORING_REGISTER_KMBUF_RING		= 38,
+	IORING_UNREGISTER_KMBUF_RING		= 39,
+
 	/* this goes last */
 	IORING_REGISTER_LAST,
 
@@ -891,9 +895,16 @@ enum io_uring_register_pbuf_ring_flags {
 	IOU_PBUF_RING_INC	= 2,
 };
 
-/* argument for IORING_(UN)REGISTER_PBUF_RING */
+/* argument for IORING_(UN)REGISTER_PBUF_RING and
+ * IORING_(UN)REGISTER_KMBUF_RING
+ */
 struct io_uring_buf_reg {
-	__u64	ring_addr;
+	union {
+		/* used for pbuf rings */
+		__u64	ring_addr;
+		/* used for kmbuf rings */
+		__u32   buf_size;
+	};
 	__u32	ring_entries;
 	__u16	bgid;
 	__u16	flags;
io_uring/kbuf.c +79 -2

92 lines not on this page · raw message

io_uring/kbuf.h +5 -2
index 40b44f4fdb15..62c80a1ebf03 100644
@@ -7,9 +7,11 @@
 
 enum {
 	/* ring mapped provided buffers */
-	IOBL_BUF_RING	= 1,
+	IOBL_BUF_RING		= 1,
 	/* buffers are consumed incrementally rather than always fully */
-	IOBL_INC	= 2,
+	IOBL_INC		= 2,
+	/* buffers are kernel managed */
+	IOBL_KERNEL_MANAGED	= 4,
 };
 
 struct io_buffer_list {
@@ -74,6 +76,7 @@ int io_provide_buffers_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe
 int io_manage_buffers_legacy(struct io_kiocb *req, unsigned int issue_flags);
 
 int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg);
+int io_register_kmbuf_ring(struct io_ring_ctx *ctx, void __user *arg);
 int io_unregister_buf_ring(struct io_ring_ctx *ctx, void __user *arg);
 int io_register_pbuf_status(struct io_ring_ctx *ctx, void __user *arg);
 
io_uring/memmap.c +111 -0

141 lines not on this page · raw message

io_uring/memmap.h +4 -0
index f4cfbb6b9a1f..3aa1167462ae 100644
@@ -22,6 +22,10 @@ int io_create_region(struct io_ring_ctx *ctx, struct io_mapped_region *mr,
 		     struct io_uring_region_desc *reg,
 		     unsigned long mmap_offset);
 
+int io_create_region_multi_buf(struct io_ring_ctx *ctx,
+			       struct io_mapped_region *mr,
+			       unsigned int nr_bufs, unsigned int buf_size);
+
 static inline void *io_region_get_ptr(struct io_mapped_region *mr)
 {
 	return mr->ptr;
io_uring/register.c +7 -0
index 0882cb34f851..2db8daaf8fde 100644
@@ -837,7 +837,14 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode,
 			break;
 		ret = io_register_pbuf_ring(ctx, arg);
 		break;
+	case IORING_REGISTER_KMBUF_RING:
+		ret = -EINVAL;
+		if (!arg || nr_args != 1)
+			break;
+		ret = io_register_kmbuf_ring(ctx, arg);
+		break;
 	case IORING_UNREGISTER_PBUF_RING:
+	case IORING_UNREGISTER_KMBUF_RING:
 		ret = -EINVAL;
 		if (!arg || nr_args != 1)
 			break;
Signature · 1 line
2.47.3
#4 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 50 lines · Joanne Koong

Add support for kernel-managed buffer rings (kmbuf rings), which allow the kernel to allocate and manage the backing buffers for a buffer ring, rather than requiring the application to provide and manage them.

This introduces two new registration opcodes:
- IORING_REGISTER_KMBUF_RING: Register a kernel-managed buffer ring
- IORING_UNREGISTER_KMBUF_RING: Unregister a kernel-managed buffer ring

The existing io_uring_buf_reg structure is extended with a union to support both application-provided buffer rings (pbuf) and kernel-managed buffer rings (kmbuf):
- For pbuf rings: ring_addr specifies the user-provided ring address
- For kmbuf rings: buf_size specifies the size of each buffer. buf_size must be non-zero and page-aligned.

The implementation follows the same pattern as pbuf ring registration, reusing the validation and buffer list allocation helpers introduced in earlier refactoring. The IOBL_KERNEL_MANAGED flag marks buffer lists as kernel-managed for appropriate handling in the I/O path.

---
  include/uapi/linux/io_uring.h |  15 ++++-
  io_uring/kbuf.c               |  81 ++++++++++++++++++++++++-
  io_uring/kbuf.h               |   7 ++-
  io_uring/memmap.c             | 111 ++++++++++++++++++++++++++++++++++
  io_uring/memmap.h             |   4 ++
  io_uring/register.c           |   7 +++
  6 files changed, 219 insertions(+), 6 deletions(-)
include/uapi/linux/io_uring.h +4 -1
index fc473af6feb4..a0889c1744bd 100644
@@ -715,6 +715,10 @@ enum io_uring_register_op {
  	/* register bpf filtering programs */
  	IORING_REGISTER_BPF_FILTER		= 37,
 
+	/* register/unregister kernel-managed ring buffer group */
+	IORING_REGISTER_KMBUF_RING		= 38,
+	IORING_UNREGISTER_KMBUF_RING		= 39,
+
  	/* this goes last */
  	IORING_REGISTER_LAST,
 
@@ -891,9 +895,16 @@ enum io_uring_register_pbuf_ring_flags {
  	IOU_PBUF_RING_INC	= 2,
  };
 
-/* argument for IORING_(UN)REGISTER_PBUF_RING */

include/uapi/linux/io_uring.h enum io_uring_register_pbuf_ring_flags { · line 906

Quoted 10 lines
include/uapi/linux/io_uring.h +8 -1
+/* argument for IORING_(UN)REGISTER_PBUF_RING and
+ * IORING_(UN)REGISTER_KMBUF_RING
+ */
  struct io_uring_buf_reg {
-	__u64	ring_addr;
+	union {
+		/* used for pbuf rings */
+		__u64	ring_addr;
+		/* used for kmbuf rings */
+		__u32   buf_size;

If you're creating a region, there should be no reason why it can't work with user passed memory. You're fencing yourself off optimisations that are already there like huge pages.

Quoted 8 lines
include/uapi/linux/io_uring.h +1 -0
+	};
  	__u32	ring_entries;
  	__u16	bgid;
  	__u16	flags;
io_uring/kbuf.c +0 -0
index aa9b70b72db4..9bc36451d083 100644

...

Quoted 8 lines
io_uring/kbuf.c +8 -0
+static int io_setup_kmbuf_ring(struct io_ring_ctx *ctx,
+			       struct io_buffer_list *bl,
+			       struct io_uring_buf_reg *reg)
+{
+	struct io_uring_buf_ring *ring;
+	unsigned long ring_size;
+	void *buf_region;
+	unsigned int i;

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 803

Quoted 10 lines
io_uring/kbuf.c +10 -0
+	int ret;
+
+	/* allocate pages for the ring structure */
+	ring_size = flex_array_size(ring, bufs, bl->nr_entries);
+	ring = kzalloc(ring_size, GFP_KERNEL_ACCOUNT);
+	if (!ring)
+		return -ENOMEM;
+
+	ret = io_create_region_multi_buf(ctx, &bl->region, bl->nr_entries,
+					 reg->buf_size);

Please use io_create_region(), the new function does nothing new and only violates abstractions.

Provided buffer rings with kernel addresses could be an interesting abstraction, but why is it also responsible for allocating buffers? What I'd do:

1. Strip buffer allocation from IORING_REGISTER_KMBUF_RING.
2. Replace *_REGISTER_KMBUF_RING with *_REGISTER_PBUF_RING + a new flag. Or maybe don't expose it to the user at all and create it from fuse via internal API.
3. Require the user to register a memory region of appropriate size, see IORING_REGISTER_MEM_REGION, ctx->param_region. Make fuse populating the buffer ring using the memory region.

I wanted to make regions shareable anyway (need it for other purposes), I can toss patches for that tomorrow.

A separate question is whether extending buffer rings is the right approach as it seems like you're only using it for fuse requests and not for passing buffers to normal requests, but I don't see the big picture here.

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 810

Quoted 7 lines
io_uring/kbuf.c +7 -0
+	if (ret) {
+		kfree(ring);
+		return ret;
+	}
+
+	/* initialize ring buf entries to point to the buffers */
+	buf_region = bl->region.ptr;

io_region_get_ptr()

Quoted 20 lines
io_uring/kbuf.c +20 -0
+	for (i = 0; i < bl->nr_entries; i++) {
+		struct io_uring_buf *buf = &ring->bufs[i];
+
+		buf->addr = (u64)(uintptr_t)buf_region;
+		buf->len = reg->buf_size;
+		buf->bid = i;
+
+		buf_region += reg->buf_size;
+	}
+	ring->tail = bl->nr_entries;
+
+	bl->buf_ring = ring;
+	bl->flags |= IOBL_KERNEL_MANAGED;
+
+	return 0;
+}
+
+int io_register_kmbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
+{
+	struct io_uring_buf_reg reg;

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 840

Quoted 10 lines
io_uring/kbuf.c +10 -0
+	struct io_buffer_list *bl;
+	int ret;
+
+	lockdep_assert_held(&ctx->uring_lock);
+
+	ret = io_copy_and_validate_buf_reg(arg, &reg, 0);
+	if (ret)
+		return ret;
+
+	if (!reg.buf_size || !PAGE_ALIGNED(reg.buf_size))

With io_create_region_multi_buf() gone, you shouldn't need to align every buffer, that could be a lot of wasted memory (thinking about 64KB pages).

Quoted 17 lines
io_uring/kbuf.c +17 -0
+		return -EINVAL;
+
+	bl = io_alloc_new_buffer_list(ctx, &reg);
+	if (IS_ERR(bl))
+		return PTR_ERR(bl);
+
+	ret = io_setup_kmbuf_ring(ctx, bl, &reg);
+	if (ret) {
+		kfree(bl);
+		return ret;
+	}
+
+	ret = io_buffer_add_list(ctx, bl, reg.bgid);
+	if (ret)
+		io_put_bl(ctx, bl);
+
+	return ret;
Signature · 1 line
Pavel Begunkov
#5 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 51 lines · Pavel Begunkov
Quoted 49 lines · Joanne Koong

Add support for kernel-managed buffer rings (kmbuf rings), which allow the kernel to allocate and manage the backing buffers for a buffer ring, rather than requiring the application to provide and manage them.

This introduces two new registration opcodes:
- IORING_REGISTER_KMBUF_RING: Register a kernel-managed buffer ring
- IORING_UNREGISTER_KMBUF_RING: Unregister a kernel-managed buffer ring

The existing io_uring_buf_reg structure is extended with a union to support both application-provided buffer rings (pbuf) and kernel-managed buffer rings (kmbuf):
- For pbuf rings: ring_addr specifies the user-provided ring address
- For kmbuf rings: buf_size specifies the size of each buffer. buf_size must be non-zero and page-aligned.

The implementation follows the same pattern as pbuf ring registration, reusing the validation and buffer list allocation helpers introduced in earlier refactoring. The IOBL_KERNEL_MANAGED flag marks buffer lists as kernel-managed for appropriate handling in the I/O path.

---
  include/uapi/linux/io_uring.h |  15 ++++-
  io_uring/kbuf.c               |  81 ++++++++++++++++++++++++-
  io_uring/kbuf.h               |   7 ++-
  io_uring/memmap.c             | 111 ++++++++++++++++++++++++++++++++++
  io_uring/memmap.h             |   4 ++
  io_uring/register.c           |   7 +++
  6 files changed, 219 insertions(+), 6 deletions(-)
include/uapi/linux/io_uring.h +4 -0
index fc473af6feb4..a0889c1744bd 100644
@@ -715,6 +715,10 @@ enum io_uring_register_op {
      /* register bpf filtering programs */
      IORING_REGISTER_BPF_FILTER              = 37,
 
+     /* register/unregister kernel-managed ring buffer group */
+     IORING_REGISTER_KMBUF_RING              = 38,
+     IORING_UNREGISTER_KMBUF_RING            = 39,
+
      /* this goes last */
      IORING_REGISTER_LAST,
 
@@ -891,9 +895,16 @@ enum io_uring_register_pbuf_ring_flags {
      IOU_PBUF_RING_INC       = 2,
  };
 
Quoted 15 lines

include/uapi/linux/io_uring.h enum io_uring_register_pbuf_ring_flags { · line 906

Quoted 11 lines
include/uapi/linux/io_uring.h +8 -2
-/* argument for IORING_(UN)REGISTER_PBUF_RING */
+/* argument for IORING_(UN)REGISTER_PBUF_RING and
+ * IORING_(UN)REGISTER_KMBUF_RING
+ */
  struct io_uring_buf_reg {
-     __u64   ring_addr;
+     union {
+             /* used for pbuf rings */
+             __u64   ring_addr;
+             /* used for kmbuf rings */
+             __u32   buf_size;

If you're creating a region, there should be no reason why it can't work with user passed memory. You're fencing yourself off optimisations that are already there like huge pages.

Are there any optimizations with user-allocated buffers that wouldn't be possible with kernel-allocated buffers? For huge pages, can't the kernel do this as well (eg I see in io_mem_alloc_compound(), it calls into alloc_pages() with order > 0)?

Quoted 23 lines
Quoted 9 lines
include/uapi/linux/io_uring.h +1 -0
+     };
      __u32   ring_entries;
      __u16   bgid;
      __u16   flags;
io_uring/kbuf.c +0 -0
index aa9b70b72db4..9bc36451d083 100644

...

Quoted 13 lines
io_uring/kbuf.c +13 -0
+static int io_setup_kmbuf_ring(struct io_ring_ctx *ctx,
+                            struct io_buffer_list *bl,
+                            struct io_uring_buf_reg *reg)
+{
+     struct io_uring_buf_ring *ring;
+     unsigned long ring_size;
+     void *buf_region;
+     unsigned int i;
+     int ret;
+
+     /* allocate pages for the ring structure */
+     ring_size = flex_array_size(ring, bufs, bl->nr_entries);
+     ring = kzalloc(ring_size, GFP_KERNEL_ACCOUNT);
Quoted 8 lines

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 803

Quoted 5 lines
io_uring/kbuf.c +5 -0
+     if (!ring)
+             return -ENOMEM;
+
+     ret = io_create_region_multi_buf(ctx, &bl->region, bl->nr_entries,
+                                      reg->buf_size);

Please use io_create_region(), the new function does nothing new and only violates abstractions.

There's separate checks needed between io_create_region() and io_create_region_multi_buf() (eg IORING_MEM_REGION_TYPE_USER flag checking) and different allocation calls (eg io_region_allocate_pages() vs io_region_allocate_pages_multi_buf()). Maybe I'm misinterpreting your comment (or the code), but I'm not seeing how this can just use io_create_region().

Quoted 3 lines

Provided buffer rings with kernel addresses could be an interesting abstraction, but why is it also responsible for allocating buffers?

Conceptually, I think it makes the interface and lifecycle management simpler/cleaner. With registering it from userspace, imo there's additional complications with no tangible benefits, eg it's not guaranteed that the memory regions registered for the buffers are the same size, with allocating it from the kernel-side we can guarantee that the pages are allocated physically contiguously, userspace setup with user-allocated buffers is less straightforward, etc. In general, I'm just not really seeing what advantages there are in allocating the buffers from userspace. Could you elaborate on that part more?

Quoted 6 lines

What I'd do:

1. Strip buffer allocation from IORING_REGISTER_KMBUF_RING.
2. Replace *_REGISTER_KMBUF_RING with *_REGISTER_PBUF_RING + a new flag. Or maybe don't expose it to the user at all and create it from fuse via internal API.

If kmbuf rings are squashed into pbuf rings, then pbuf rings will need to support pinning. In fuse, there are some contexts where you can't grab the uring mutex because you're running in atomic context and this can be encountered while recycling the buffer. I originally had a patch adding pinning to pbuf rings (to mitigate the overhead of registered buffers lookups) but dropped it when Jens and Caleb didn't like the idea. But for kmbuf rings, pinning will be necessary for fuse.

Quoted 10 lines

3. Require the user to register a memory region of appropriate size, see IORING_REGISTER_MEM_REGION, ctx->param_region. Make fuse populating the buffer ring using the memory region.

I wanted to make regions shareable anyway (need it for other purposes), I can toss patches for that tomorrow.

A separate question is whether extending buffer rings is the right approach as it seems like you're only using it for fuse requests and not for passing buffers to normal requests, but I don't see the

What are 'normal requests'? For fuse's use case, there are only fuse requests.

Thanks,
Joanne

Quoted 68 lines

big picture here.

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 810

Quoted 7 lines
io_uring/kbuf.c +7 -0
+     if (ret) {
+             kfree(ring);
+             return ret;
+     }
+
+     /* initialize ring buf entries to point to the buffers */
+     buf_region = bl->region.ptr;

io_region_get_ptr()

Quoted 20 lines
io_uring/kbuf.c +20 -0
+     for (i = 0; i < bl->nr_entries; i++) {
+             struct io_uring_buf *buf = &ring->bufs[i];
+
+             buf->addr = (u64)(uintptr_t)buf_region;
+             buf->len = reg->buf_size;
+             buf->bid = i;
+
+             buf_region += reg->buf_size;
+     }
+     ring->tail = bl->nr_entries;
+
+     bl->buf_ring = ring;
+     bl->flags |= IOBL_KERNEL_MANAGED;
+
+     return 0;
+}
+
+int io_register_kmbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
+{
+     struct io_uring_buf_reg reg;

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 840

Quoted 10 lines
io_uring/kbuf.c +10 -0
+     struct io_buffer_list *bl;
+     int ret;
+
+     lockdep_assert_held(&ctx->uring_lock);
+
+     ret = io_copy_and_validate_buf_reg(arg, &reg, 0);
+     if (ret)
+             return ret;
+
+     if (!reg.buf_size || !PAGE_ALIGNED(reg.buf_size))

With io_create_region_multi_buf() gone, you shouldn't need to align every buffer, that could be a lot of wasted memory (thinking about 64KB pages).

Quoted 7 lines
io_uring/kbuf.c +7 -0
+             return -EINVAL;
+
+     bl = io_alloc_new_buffer_list(ctx, &reg);
+     if (IS_ERR(bl))
+             return PTR_ERR(bl);
+
+     ret = io_setup_kmbuf_ring(ctx, bl, &reg);

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 857

Quoted 10 lines
io_uring/kbuf.c +10 -0
+     if (ret) {
+             kfree(bl);
+             return ret;
+     }
+
+     ret = io_buffer_add_list(ctx, bl, reg.bgid);
+     if (ret)
+             io_put_bl(ctx, bl);
+
+     return ret;

--
Pavel Begunkov

#6 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 1 line · Joanne Koong

On Tue, Feb 10, 2026 at 8:34 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
include/uapi/linux/io_uring.h +4 -2
-/* argument for IORING_(UN)REGISTER_PBUF_RING */
+/* argument for IORING_(UN)REGISTER_PBUF_RING and
+ * IORING_(UN)REGISTER_KMBUF_RING
+ */
   struct io_uring_buf_reg {
-     __u64   ring_addr;
+     union {
Quoted 8 lines

include/uapi/linux/io_uring.h enum io_uring_register_pbuf_ring_flags { · line 906

Quoted 4 lines
include/uapi/linux/io_uring.h +4 -0
+             /* used for pbuf rings */
+             __u64   ring_addr;
+             /* used for kmbuf rings */
+             __u32   buf_size;

If you're creating a region, there should be no reason why it can't work with user passed memory. You're fencing yourself off optimisations that are already there like huge pages.

Quoted 4 lines

Are there any optimizations with user-allocated buffers that wouldn't be possible with kernel-allocated buffers? For huge pages, can't the kernel do this as well (eg I see in io_mem_alloc_compound(), it calls into alloc_pages() with order > 0)?

Yes, there is handful of differences. To name one, 1MB allocation won't get you a PMD mappable huge page, while user space can allocate 2MB, register the first 1MB and reuse the rest for other purposes.

Quoted 31 lines
Quoted 22 lines
Quoted 8 lines
include/uapi/linux/io_uring.h +1 -0
+     };
       __u32   ring_entries;
       __u16   bgid;
       __u16   flags;
io_uring/kbuf.c +0 -0
index aa9b70b72db4..9bc36451d083 100644

...

Quoted 13 lines
io_uring/kbuf.c +13 -0
+static int io_setup_kmbuf_ring(struct io_ring_ctx *ctx,
+                            struct io_buffer_list *bl,
+                            struct io_uring_buf_reg *reg)
+{
+     struct io_uring_buf_ring *ring;
+     unsigned long ring_size;
+     void *buf_region;
+     unsigned int i;
+     int ret;
+
+     /* allocate pages for the ring structure */
+     ring_size = flex_array_size(ring, bufs, bl->nr_entries);
+     ring = kzalloc(ring_size, GFP_KERNEL_ACCOUNT);
Quoted 8 lines

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 803

Quoted 5 lines
io_uring/kbuf.c +5 -0
+     if (!ring)
+             return -ENOMEM;
+
+     ret = io_create_region_multi_buf(ctx, &bl->region, bl->nr_entries,
+                                      reg->buf_size);

Please use io_create_region(), the new function does nothing new and only violates abstractions.

Quoted 2 lines

There's separate checks needed between io_create_region() and io_create_region_multi_buf() (eg IORING_MEM_REGION_TYPE_USER flag

If io_create_region() is too strict, let's discuss that in examples if there are any, but it's likely not a good idea changing that. If it's too lax, filter arguments in the caller. IOW, don't pass IORING_MEM_REGION_TYPE_USER if it's not used.

Quoted 2 lines

checking) and different allocation calls (eg io_region_allocate_pages() vs io_region_allocate_pages_multi_buf()).

I saw that and saying that all memmap.c changes can get dropped. You're using it as one big virtually contig kernel memory range then chunked into buffers, and that's pretty much what you're getting with normal io_create_region(). I get that you only need it to be contiguous within a single buffer, but that's not what you're doing, and it'll be only worse than default io_create_region() e.g. effectively disabling any usefulness of io_mem_alloc_compound(), and ultimately you don't need to care.

Regions shouldn't know anything about your buffers, how it's subdivided after, etc.

Quoted 2 lines

Maybe I'm misinterpreting your comment (or the code), but I'm not seeing how this can just use io_create_region().

struct io_uring_region_desc rd = {};
total_size = nr_bufs * buf_size;
rd.size = PAGE_ALIGN(total_size);
io_create_region(&region, &rd);

Add something like this for user provided memory:

if (use_user_memory) {
	rd.user_addr = uaddr;
	rd.flags |= IORING_MEM_REGION_TYPE_USER;
}
Quoted 12 lines
Quoted 2 lines

Provided buffer rings with kernel addresses could be an interesting abstraction, but why is it also responsible for allocating buffers?

Conceptually, I think it makes the interface and lifecycle management simpler/cleaner. With registering it from userspace, imo there's additional complications with no tangible benefits, eg it's not guaranteed that the memory regions registered for the buffers are the same size, with allocating it from the kernel-side we can guarantee that the pages are allocated physically contiguously, userspace setup with user-allocated buffers is less straightforward, etc. In general, I'm just not really seeing what advantages there are in allocating the buffers from userspace. Could you elaborate on that part more?

I don't think I follow. I'm saying that it might be interesting to separate rings from how and with what they're populated on the kernel API level, but the fuse kernel module can do the population and get exactly same layout as you currently have:

int fuse_create_ring(size_t region_offset /* user space argument */) {
	struct io_mapped_region *mr = get_mem_region(ctx);
	// that can take full control of the ring
	ring = grab_empty_ring(io_uring_ctx);

	size = nr_bufs * buf_size;
	if (region_offset + size > get_size(mr)) // + other validation
		return error;

	buf = mr_get_ptr(mr) + offset;
	for (i = 0; i < nr_bufs; i++) {
		ring_push_buffer(ring, buf, buf_size);
		buf += buf_size;
	}
}

fuse might not care, but with empty rings other users will get a channel they can use to do IO (e.g. read requests) using their kernel addresses in the future.

Quoted 9 lines
Quoted 6 lines

What I'd do:

1. Strip buffer allocation from IORING_REGISTER_KMBUF_RING.
2. Replace *_REGISTER_KMBUF_RING with *_REGISTER_PBUF_RING + a new flag. Or maybe don't expose it to the user at all and create it from fuse via internal API.

If kmbuf rings are squashed into pbuf rings, then pbuf rings will need to support pinning. In fuse, there are some contexts where you can't

It'd change uapi but not internals, you already piggy back it on pbuf implementation and differentiate with a flag.

It could basically be:

if (flags & IOU_PBUF_RING_KM)

	bl->flags |= IOBL_KERNEL_MANAGED;

Pinning can be gated on that flag as well. Pretty likely uapi and internals will be a bit cleaner, but that's not a huge deal, just don't see why would you roll out a separate set of uapi ([un]register, offsets, etc.) when essentially it can be treated as the same thing.

Quoted 4 lines

grab the uring mutex because you're running in atomic context and this can be encountered while recycling the buffer. I originally had a patch adding pinning to pbuf rings (to mitigate the overhead of registered buffers lookups)

IIRC, you was pinning the registered buffer table and not provided buffer rings? Which would indeed be a bad idea. Thinking about it, fwiw, instead of creating multiple registered buffers and trying to lock the entire table, you could've kept all memory in one larger registered buffer and pinned only it. It's already refcounted, so shouldn't have been much of a problem.

Quoted 7 lines

but dropped it when Jens and Caleb didn't
like the idea. But for kmbuf rings, pinning will be necessary for fuse.

Quoted 3 lines

3. Require the user to register a memory region of appropriate size, see IORING_REGISTER_MEM_REGION, ctx->param_region. Make fuse populating the buffer ring using the memory region.

To explain why, I don't think that creating many small regions is a good direction going forward. In case of kernel allocation, it's extra mmap()s, extra user space management, and wasted space. For user provided memory it's over-accounting and extra memory footprint. It'll also give you better lifecycle guarantees, i.e. you won't be able to free buffers while there are requests for the context. I'm not so sure about ring bound memory, let's say I have my suspicions, and you'd need to be extra careful about buffer lifetimes even after a fuse instance dies.

Quoted 8 lines
Quoted 6 lines

I wanted to make regions shareable anyway (need it for other purposes), I can toss patches for that tomorrow.

A separate question is whether extending buffer rings is the right approach as it seems like you're only using it for fuse requests and not for passing buffers to normal requests, but I don't see the

What are 'normal requests'? For fuse's use case, there are only fuse requests.

Any kind of read/recv/etc. that can use provided buffers. It's where kernel memory filled rings would shine, as you'd be able to use them together without changing any opcode specific code. I.e. not changes in read request implementation, only kbuf.c

Signature · 1 line
Pavel Begunkov
#7 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 71 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Tue, Feb 10, 2026 at 8:34 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
include/uapi/linux/io_uring.h +4 -2
-/* argument for IORING_(UN)REGISTER_PBUF_RING */
+/* argument for IORING_(UN)REGISTER_PBUF_RING and
+ * IORING_(UN)REGISTER_KMBUF_RING
+ */
   struct io_uring_buf_reg {
-     __u64   ring_addr;
+     union {
Quoted 8 lines

include/uapi/linux/io_uring.h enum io_uring_register_pbuf_ring_flags { · line 906

Quoted 4 lines
include/uapi/linux/io_uring.h +4 -0
+             /* used for pbuf rings */
+             __u64   ring_addr;
+             /* used for kmbuf rings */
+             __u32   buf_size;

If you're creating a region, there should be no reason why it can't work with user passed memory. You're fencing yourself off optimisations that are already there like huge pages.

Quoted 4 lines

Are there any optimizations with user-allocated buffers that wouldn't be possible with kernel-allocated buffers? For huge pages, can't the kernel do this as well (eg I see in io_mem_alloc_compound(), it calls into alloc_pages() with order > 0)?

Yes, there is handful of differences. To name one, 1MB allocation won't get you a PMD mappable huge page, while user space can allocate 2MB, register the first 1MB and reuse the rest for other purposes.

Quoted 31 lines
Quoted 22 lines
Quoted 8 lines
include/uapi/linux/io_uring.h +1 -0
+     };
       __u32   ring_entries;
       __u16   bgid;
       __u16   flags;
io_uring/kbuf.c +0 -0
index aa9b70b72db4..9bc36451d083 100644

...

Quoted 13 lines
io_uring/kbuf.c +13 -0
+static int io_setup_kmbuf_ring(struct io_ring_ctx *ctx,
+                            struct io_buffer_list *bl,
+                            struct io_uring_buf_reg *reg)
+{
+     struct io_uring_buf_ring *ring;
+     unsigned long ring_size;
+     void *buf_region;
+     unsigned int i;
+     int ret;
+
+     /* allocate pages for the ring structure */
+     ring_size = flex_array_size(ring, bufs, bl->nr_entries);
+     ring = kzalloc(ring_size, GFP_KERNEL_ACCOUNT);
Quoted 8 lines

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 803

Quoted 5 lines
io_uring/kbuf.c +5 -0
+     if (!ring)
+             return -ENOMEM;
+
+     ret = io_create_region_multi_buf(ctx, &bl->region, bl->nr_entries,
+                                      reg->buf_size);

Please use io_create_region(), the new function does nothing new and only violates abstractions.

Quoted 2 lines

There's separate checks needed between io_create_region() and io_create_region_multi_buf() (eg IORING_MEM_REGION_TYPE_USER flag

If io_create_region() is too strict, let's discuss that in examples if there are any, but it's likely not a good idea changing that. If it's too lax, filter arguments in the caller. IOW, don't pass IORING_MEM_REGION_TYPE_USER if it's not used.

Quoted 2 lines

checking) and different allocation calls (eg io_region_allocate_pages() vs io_region_allocate_pages_multi_buf()).

Quoted 8 lines

I saw that and saying that all memmap.c changes can get dropped. You're using it as one big virtually contig kernel memory range then chunked into buffers, and that's pretty much what you're getting with normal io_create_region(). I get that you only need it to be contiguous within a single buffer, but that's not what you're doing, and it'll be only worse than default io_create_region() e.g. effectively disabling any usefulness of io_mem_alloc_compound(), and ultimately you don't need to care.

When I originally implemented it, I had it use io_region_allocate_pages() but this fails because it's allocating way too much memory at once. For fuse's use case, each buffer is usually at least 1 MB if not more. Allocating the memory one buffer a time in io_region_allocate_pages_multi_buf() bypasses the allocation errors I was seeing. That's the main reason I don't think this can just use io_create_region().

Quoted 33 lines

Regions shouldn't know anything about your buffers, how it's subdivided after, etc.

Quoted 2 lines

Maybe I'm misinterpreting your comment (or the code), but I'm not seeing how this can just use io_create_region().

struct io_uring_region_desc rd = {};
total_size = nr_bufs * buf_size;
rd.size = PAGE_ALIGN(total_size);
io_create_region(&region, &rd);

Add something like this for user provided memory:

if (use_user_memory) {
        rd.user_addr = uaddr;
        rd.flags |= IORING_MEM_REGION_TYPE_USER;
}
Quoted 12 lines
Quoted 2 lines

Provided buffer rings with kernel addresses could be an interesting abstraction, but why is it also responsible for allocating buffers?

Conceptually, I think it makes the interface and lifecycle management simpler/cleaner. With registering it from userspace, imo there's additional complications with no tangible benefits, eg it's not guaranteed that the memory regions registered for the buffers are the same size, with allocating it from the kernel-side we can guarantee that the pages are allocated physically contiguously, userspace setup with user-allocated buffers is less straightforward, etc. In general, I'm just not really seeing what advantages there are in allocating the buffers from userspace. Could you elaborate on that part more?

Quoted 3 lines

I don't think I follow. I'm saying that it might be interesting to separate rings from how and with what they're populated on the kernel API level, but the fuse kernel module can do the population

Oh okay, from your first message I (and I think christoph too) thought what you were saying is that the user should be responsible for allocating the buffers with complete ownership over them, and then just pass those allocated to the kernel to use. But what you're saying is that just use a different way for getting the kernel to allocate the buffers (eg through the IORING_REGISTER_MEM_REGION interface). Am I reading this correctly?

Quoted 37 lines

and get exactly same layout as you currently have:

int fuse_create_ring(size_t region_offset /* user space argument */) {
        struct io_mapped_region *mr = get_mem_region(ctx);
        // that can take full control of the ring
        ring = grab_empty_ring(io_uring_ctx);

        size = nr_bufs * buf_size;
        if (region_offset + size > get_size(mr)) // + other validation
                return error;

        buf = mr_get_ptr(mr) + offset;
        for (i = 0; i < nr_bufs; i++) {
                ring_push_buffer(ring, buf, buf_size);
                buf += buf_size;
        }
}

fuse might not care, but with empty rings other users will get a channel they can use to do IO (e.g. read requests) using their kernel addresses in the future.

Quoted 9 lines
Quoted 6 lines

What I'd do:

1. Strip buffer allocation from IORING_REGISTER_KMBUF_RING.
2. Replace *_REGISTER_KMBUF_RING with *_REGISTER_PBUF_RING + a new flag. Or maybe don't expose it to the user at all and create it from fuse via internal API.

If kmbuf rings are squashed into pbuf rings, then pbuf rings will need to support pinning. In fuse, there are some contexts where you can't

It'd change uapi but not internals, you already piggy back it on pbuf implementation and differentiate with a flag.

It could basically be:

Quoted 8 lines

if (flags & IOU_PBUF_RING_KM)

        bl->flags |= IOBL_KERNEL_MANAGED;

Pinning can be gated on that flag as well. Pretty likely uapi and internals will be a bit cleaner, but that's not a huge deal, just don't see why would you roll out a separate set of uapi ([un]register, offsets, etc.) when essentially it can be treated as the same thing.

imo, it looked cleaner as a separate api because it has different expectations and behaviors and squashing kmbuf into the pbuf api makes the pbuf api needlessly more complex. Though I guess from the userspace pov, liburing could have a wrapper that takes care of setting up the pbuf details for kernel-managed pbufs. But in my head, having pbufs vs. kmbufs makes it clearer what each one does vs regular pbufs vs. pbufs that are kernel-managed.

Especially with now having kmbufs go through the ioring mem region interface, it makes things more confusing imo if they're combined, eg pbufs that are kernel-managed are created empty and then populated from the kernel side by whatever subsystem is using them. Right now there's only one mem region supported per ring, but in the future if there's the possibility that multiple mem regions can be registered (eg if userspace doesn't know upfront what mem region length they'll need), then we should also probably add in a region id param for the registration arg, which if kmbuf rings go through the pbuf ring registration api, is not possible to do.

But I'm happy to combine the interfaces and go with your suggestion. I'll make this change for v2 unless someone else objects.

Quoted 7 lines
Quoted 5 lines

grab the uring mutex because you're running in atomic context and this can be encountered while recycling the buffer. I originally had a patch adding pinning to pbuf rings (to mitigate the overhead of registered buffers lookups)

IIRC, you was pinning the registered buffer table and not provided

Yeah, you're right I misremembered and the objections / patch I dropped was pinning the registered buffer table, not the pbuf ring

Quoted 5 lines

buffer rings? Which would indeed be a bad idea. Thinking about it, fwiw, instead of creating multiple registered buffers and trying to lock the entire table, you could've kept all memory in one larger registered buffer and pinned only it. It's already refcounted, so shouldn't have been much of a problem.

Hmm, I'm not sure this idea would work for sparse buffers populated by the kernel, unless those are automatically pinned too but then from the user POV for unregistration they'd need to unregister buffers individually instead of just calling IORING_UNREGISTER_BUFFERS but it might be annoying for them to now need to know which buffers are pinned vs not. When i benchmarked the fuse code with vs without pinned registered buffers, it didn't seem to make much of a difference performance-wise thankfully, so I just dropped it.

Quoted 5 lines
Quoted 5 lines

but dropped it when Jens and Caleb didn't
like the idea. But for kmbuf rings, pinning will be necessary for fuse.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

3. Require the user to register a memory region of appropriate size, see IORING_REGISTER_MEM_REGION, ctx->param_region. Make fuse populating the buffer ring using the memory region.

To explain why, I don't think that creating many small regions is a good direction going forward. In case of kernel allocation, it's extra mmap()s, extra user space management, and wasted space.

To clarify, is this in reply to why the individual buffers shouldn't be allocated separately by the kernel?
I added a comment about this above in the discussion about io_region_allocate_pages_multi_buf(), and if the memory allocation issue I was seeing is bypassable and the region can be allocated all at once, I'm happy to make that change. With having the allocation be separate buffers though, I'm not sure I agree that there are extra mmaps / userspace management. All the pages across the buffers are vmapped together and the userspace just needs to do 1 mmap call for them. On the userspace side, I don't think there's more management since the mmapped address represents the range across all the buffers. I'm not seeing how there's wasted space either since the only requirement is that the buffer size is page aligned. I think also there's a higher chance of the entire buffer region being physically contiguous if each buffer is allocated separately vs. all the buffers are allocated as 1 region. I don't feel strongly about this either way and I'm happy to allocate the entire region at once if that's possible.

Quoted 2 lines

For user provided memory it's over-accounting and extra memory footprint. It'll also give you better lifecycle guarantees, i.e.

Just out of curiosity, could you elaborate on the over-accounting and extra memory footprint? I was under the impression it would be the same since the accounting gets adjusted by the total bytes allocated? For the extra memory footprint, is the extra footprint from the metadata to describe each buffer region, or are you referring to something else?

Quoted 12 lines

you won't be able to free buffers while there are requests for the context. I'm not so sure about ring bound memory, let's say I have my suspicions, and you'd need to be extra careful about buffer lifetimes even after a fuse instance dies.

Quoted 7 lines
Quoted 6 lines

I wanted to make regions shareable anyway (need it for other purposes), I can toss patches for that tomorrow.

A separate question is whether extending buffer rings is the right approach as it seems like you're only using it for fuse requests and not for passing buffers to normal requests, but I don't see the

Quoted 7 lines
Quoted 1 line

What are 'normal requests'? For fuse's use case, there are only fuse requests.

Any kind of read/recv/etc. that can use provided buffers. It's where kernel memory filled rings would shine, as you'd be able to use them together without changing any opcode specific code. I.e. not changes in read request implementation, only kbuf.c

Thanks for your input on the series. To iterate / sum up, these are changes for v2 I'll be making:
- api-wise from userspace/liburing: get rid of KMBUF_RING api interface and have users go through PBUF_RING api instead with a flag indicating the ring is kernel-managed
- have kernel buffer allocation go through IORING_REGISTER_MEM_REGION instead, which means when the pbuf ring is created and the kernel-managed flag is set, the ring will be empty. The memory region will need to be registered before the mmap call to the ring fd.
- add apis for subsystems to populate a kernel-managed buffer ring with addresses from the registered mem region

Does this align with your understanding of the conversation as well or is there anything I'm missing?

And Christoph, do these changes for v2 work for your use case as well?

Thanks,
Joanne

Quoted 3 lines

--
Pavel Begunkov

#8 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 11 lines · Joanne Koong
Quoted 3 lines

I don't think I follow. I'm saying that it might be interesting to separate rings from how and with what they're populated on the kernel API level, but the fuse kernel module can do the population

Oh okay, from your first message I (and I think christoph too) thought what you were saying is that the user should be responsible for allocating the buffers with complete ownership over them, and then just pass those allocated to the kernel to use. But what you're saying is that just use a different way for getting the kernel to allocate the buffers (eg through the IORING_REGISTER_MEM_REGION interface). Am I reading this correctly?

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love to be able to piggy back than onto your work.

#9 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Joanne Koong
Quoted 3 lines

I don't think I follow. I'm saying that it might be interesting to separate rings from how and with what they're populated on the kernel API level, but the fuse kernel module can do the population

Oh okay, from your first message I (and I think christoph too) thought what you were saying is that the user should be responsible for allocating the buffers with complete ownership over them, and then just pass those allocated to the kernel to use. But what you're saying is that just use a different way for getting the kernel to allocate the buffers (eg through the IORING_REGISTER_MEM_REGION interface). Am I reading this correctly?

Quoted 4 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love to be able to piggy back than onto your work.

IORING_REGISTER_MEM_REGION supports both types of allocations. It can have a new registration flag for read-only, and then you either make the bounce avoidance optional or reject binding fuse to unsupported setups during init. Any arguments against that? I need to go over Joanne's reply, but I don't see any contradiction in principal with your use case.

Signature · 1 line
Pavel Begunkov
#10 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 15 lines · Pavel Begunkov
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Joanne Koong
Quoted 3 lines

I don't think I follow. I'm saying that it might be interesting to separate rings from how and with what they're populated on the kernel API level, but the fuse kernel module can do the population

Oh okay, from your first message I (and I think christoph too) thought what you were saying is that the user should be responsible for allocating the buffers with complete ownership over them, and then just pass those allocated to the kernel to use. But what you're saying is that just use a different way for getting the kernel to allocate the buffers (eg through the IORING_REGISTER_MEM_REGION interface). Am I reading this correctly?

Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

Quoted 8 lines
Quoted 1 line

to be able to piggy back than onto your work.

IORING_REGISTER_MEM_REGION supports both types of allocations. It can have a new registration flag for read-only, and then you either make the bounce avoidance optional or reject binding fuse to unsupported setups during init. Any arguments against that? I need to go over Joanne's reply, but I don't see any contradiction in principal with your use case.

So i guess the flow would have to be:
a) user calls io_uring_register_region(&ring, &mem_region_reg) with mem_region_reg.region_uptr's size field set to the total buffer size (and mem_region_reg.flags read-only bit set if needed) kernel allocates region
b) user calls mmap() to get the address of the region. If read-only bit was set, it gets a read-only address
c) user calls io_uring_register_buf_ring(&ring, &buf_reg, flags) with buf_reg.flags |= IOU_PBUF_RING_KERNEL_MANAGED
kernel creates an empty kernel-managed ring. None of the buffers
are populated
d) user tells X subsystem to populate the ring starting from offset Z in the registered mem region
e) on the kernel side, the subsystem populates the ring starting from offset Z, filling it up using the buf_size and ring_entries values that the user registered the ring with in c)

To be completely honest, the more I look at this the more this feels like overkill / over-engineered to me. I get that now the user can do the PMD optimization, but does that actually lead to noticeable performance benefits? It seems especially confusing with them going through the same pbuf ring interface but having totally different expectations.

What about adding a straightforward kmbuf ring that goes through the pbuf interface (eg the design in this patchset) and then in the future adding an interface for pbuf rings (both kernel-managed and non-kernel-managed) to go through IORING_REGISTERED_MEM_REGIONS if users end up needing/wanting to have their rings populated that way?

Thanks,
Joanne

Quoted 4 lines

--
Pavel Begunkov

#11 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 4 lines · Joanne Koong
Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

Quoted 12 lines

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

The idea is that the application tells the kernel that it wants to use a fixed buffer pool for reads. Right now the application does this using io_uring_register_buffers(). The problem with that is that io_uring_register_buffers ends up just doing a pin of the memory, but the application or, in case of shared memory, someone else could still modify the memory. If the underlying file system or storage device needs verify checksums, or worse rebuild data from parity (or uncompress), it needs to ensure that the memory it is operating on can't be modified by someone else.

So I've been thinking of a version of io_uring_register_buffers where the buffers are not provided by the application, but instead by the kernel and mapped into the application address space read-only for a while, and I thought I could implement this on top of your series, but I have to admit I haven't really looked into the details all that much.

Quoted 7 lines

To be completely honest, the more I look at this the more this feels like overkill / over-engineered to me. I get that now the user can do the PMD optimization, but does that actually lead to noticeable performance benefits? It seems especially confusing with them going through the same pbuf ring interface but having totally different expectations.

Yes. The PMD mapping also is not that relevant. Both AMD (implicit) and ARM (explicit) have optimizations for contiguous PTEs that are almost as valuable.

Quoted 5 lines

What about adding a straightforward kmbuf ring that goes through the pbuf interface (eg the design in this patchset) and then in the future adding an interface for pbuf rings (both kernel-managed and non-kernel-managed) to go through IORING_REGISTERED_MEM_REGIONS if users end up needing/wanting to have their rings populated that way?

That feels much simpler to me as well.

#12 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 28 lines · Christoph Hellwig
Quoted 4 lines · Joanne Koong
Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

Quoted 12 lines

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

The idea is that the application tells the kernel that it wants to use a fixed buffer pool for reads. Right now the application does this using io_uring_register_buffers(). The problem with that is that io_uring_register_buffers ends up just doing a pin of the memory, but the application or, in case of shared memory, someone else could still modify the memory. If the underlying file system or storage device needs verify checksums, or worse rebuild data from parity (or uncompress), it needs to ensure that the memory it is operating on can't be modified by someone else.

Quoted 6 lines

So I've been thinking of a version of io_uring_register_buffers where the buffers are not provided by the application, but instead by the kernel and mapped into the application address space read-only for a while, and I thought I could implement this on top of your series, but I have to admit I haven't really looked into the details all that much.

There is nothing about registered buffers in this series. And even if you try to reuse buffer allocation out of it, it'll come with a circular buffer you'll have no need for. And I'm pretty much arguing about separating those for io_uring.

Signature · 1 line
Pavel Begunkov
#13 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 36 lines · Pavel Begunkov
Quoted 28 lines · Christoph Hellwig
Quoted 4 lines · Joanne Koong
Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this.  For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it.  I'd love

Quoted 12 lines

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

The idea is that the application tells the kernel that it wants to use a fixed buffer pool for reads.  Right now the application does this using io_uring_register_buffers().  The problem with that is that io_uring_register_buffers ends up just doing a pin of the memory, but the application or, in case of shared memory, someone else could still modify the memory.  If the underlying file system or storage device needs verify checksums, or worse rebuild data from parity (or uncompress), it needs to ensure that the memory it is operating on can't be modified by someone else.

Quoted 6 lines

So I've been thinking of a version of io_uring_register_buffers where the buffers are not provided by the application, but instead by the kernel and mapped into the application address space read-only for a while, and I thought I could implement this on top of your series, but I have to admit I haven't really looked into the details all that much.

Quoted 4 lines

There is nothing about registered buffers in this series. And even if you try to reuse buffer allocation out of it, it'll come with a circular buffer you'll have no need for. And I'm pretty much arguing about separating those for io_uring.

FWIW, the easiest solution is to internally reuse regions for allocations and mmap()'ing and wrap it into a registered buffer. It just need to make vmap'ing optional as it won't be needed.

Signature · 1 line
Pavel Begunkov
#14 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Pavel Begunkov
Quoted 37 lines · Pavel Begunkov
Quoted 28 lines · Christoph Hellwig
Quoted 4 lines · Joanne Koong
Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

Quoted 12 lines

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

The idea is that the application tells the kernel that it wants to use a fixed buffer pool for reads. Right now the application does this using io_uring_register_buffers(). The problem with that is that io_uring_register_buffers ends up just doing a pin of the memory, but the application or, in case of shared memory, someone else could still modify the memory. If the underlying file system or storage device needs verify checksums, or worse rebuild data from parity (or uncompress), it needs to ensure that the memory it is operating on can't be modified by someone else.

Quoted 6 lines

So I've been thinking of a version of io_uring_register_buffers where the buffers are not provided by the application, but instead by the kernel and mapped into the application address space read-only for a while, and I thought I could implement this on top of your series, but I have to admit I haven't really looked into the details all that much.

Quoted 3 lines

There is nothing about registered buffers in this series. And even if you try to reuse buffer allocation out of it, it'll come with a circular buffer you'll have no need for. And I'm pretty much

I think the circular buffer will be useful for Christoph's use case in the same way it'll be useful for fuse's. The read payload could be differently sized across requests, so it's a lot of wasted space to have to allocate a buffer large enough to support the max-size request per entry in the io_ring. With using a circular buffer, buffers have a way to be shared across entries, which means we can significantly reduce how much memory needs to be allocated.

Thanks,
Joanne

Quoted 5 lines

arguing about separating those for io_uring.

--
Pavel Begunkov

#15 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 42 lines · Joanne Koong
Quoted 37 lines · Pavel Begunkov
Quoted 28 lines · Christoph Hellwig
Quoted 4 lines · Joanne Koong
Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

Quoted 12 lines

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

The idea is that the application tells the kernel that it wants to use a fixed buffer pool for reads. Right now the application does this using io_uring_register_buffers(). The problem with that is that io_uring_register_buffers ends up just doing a pin of the memory, but the application or, in case of shared memory, someone else could still modify the memory. If the underlying file system or storage device needs verify checksums, or worse rebuild data from parity (or uncompress), it needs to ensure that the memory it is operating on can't be modified by someone else.

Quoted 6 lines

So I've been thinking of a version of io_uring_register_buffers where the buffers are not provided by the application, but instead by the kernel and mapped into the application address space read-only for a while, and I thought I could implement this on top of your series, but I have to admit I haven't really looked into the details all that much.

Quoted 3 lines

There is nothing about registered buffers in this series. And even if you try to reuse buffer allocation out of it, it'll come with a circular buffer you'll have no need for. And I'm pretty much

Quoted 7 lines

I think the circular buffer will be useful for Christoph's use case in the same way it'll be useful for fuse's. The read payload could be differently sized across requests, so it's a lot of wasted space to have to allocate a buffer large enough to support the max-size request per entry in the io_ring. With using a circular buffer, buffers have a way to be shared across entries, which means we can significantly reduce how much memory needs to be allocated.

Dunno, what we actually want is requests of multiple sizes. Sharing buffers across entries sounds like just reducing the ring size - I personally don't see the point here.

Thanks,
Bernd

#16 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 54 lines · Bernd Schubert
Quoted 42 lines · Joanne Koong
Quoted 37 lines · Pavel Begunkov
Quoted 28 lines · Christoph Hellwig
Quoted 4 lines · Joanne Koong
Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

Quoted 12 lines

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

The idea is that the application tells the kernel that it wants to use a fixed buffer pool for reads. Right now the application does this using io_uring_register_buffers(). The problem with that is that io_uring_register_buffers ends up just doing a pin of the memory, but the application or, in case of shared memory, someone else could still modify the memory. If the underlying file system or storage device needs verify checksums, or worse rebuild data from parity (or uncompress), it needs to ensure that the memory it is operating on can't be modified by someone else.

Quoted 6 lines

So I've been thinking of a version of io_uring_register_buffers where the buffers are not provided by the application, but instead by the kernel and mapped into the application address space read-only for a while, and I thought I could implement this on top of your series, but I have to admit I haven't really looked into the details all that much.

Quoted 3 lines

There is nothing about registered buffers in this series. And even if you try to reuse buffer allocation out of it, it'll come with a circular buffer you'll have no need for. And I'm pretty much

Quoted 7 lines

I think the circular buffer will be useful for Christoph's use case in the same way it'll be useful for fuse's. The read payload could be differently sized across requests, so it's a lot of wasted space to have to allocate a buffer large enough to support the max-size request per entry in the io_ring. With using a circular buffer, buffers have a way to be shared across entries, which means we can significantly reduce how much memory needs to be allocated.

Quoted 3 lines

Dunno, what we actually want is requests of multiple sizes. Sharing buffers across entries sounds like just reducing the ring size - I personally don't see the point here.

By "sharing buffers across entries" what I mean is different regions of the buffer can now be used concurrently by multiple entries.

Thanks,
Joanne

Quoted 4 lines

Thanks,
Bernd

#17 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Joanne Koong
Quoted 6 lines · Joanne Koong

I think the circular buffer will be useful for Christoph's use case in the same way it'll be useful for fuse's. The read payload could be differently sized across requests, so it's a lot of wasted space to have to allocate a buffer large enough to support the max-size request per entry in the io_ring.

Yes.

Quoted 3 lines

With using a circular buffer, buffers have a
way to be shared across entries, which means we can significantly reduce how much memory needs to be allocated.

Or enable such flexible use cases at all.

#18 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Christoph Hellwig
Quoted 19 lines · Christoph Hellwig
Quoted 4 lines · Joanne Koong
Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

Quoted 12 lines

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

Quoted 9 lines

The idea is that the application tells the kernel that it wants to use a fixed buffer pool for reads. Right now the application does this using io_uring_register_buffers(). The problem with that is that io_uring_register_buffers ends up just doing a pin of the memory, but the application or, in case of shared memory, someone else could still modify the memory. If the underlying file system or storage device needs verify checksums, or worse rebuild data from parity (or uncompress), it needs to ensure that the memory it is operating on can't be modified by someone else.

(resending because I hit reply instead of reply-all)

I think we have the exact same use case, except your buffers need to be read-only. I think your use case benefits from the same memory wins we'll get with incremental buffer consumption, which is the primary reason fuse is using a bufring instead of fixed buffers.

Quoted 7 lines

So I've been thinking of a version of io_uring_register_buffers where the buffers are not provided by the application, but instead by the kernel and mapped into the application address space read-only for a while, and I thought I could implement this on top of your series, but I have to admit I haven't really looked into the details all that much.

I think you can and it'll be very easy to do so. All that would be needed is to pass in a read-only flag from the userspace side when it registers the bufring, and then when userspace makes the mmap call to the bufring, the kernel checks if that read-only flag is set on the bufring and if so returns a read-only mapping. I'm happy to add that patch to this series if that would make things easier for you. The io_uring_register_buffers() api registers fixed buffers (which have to be user-allocated memory) so you would need to go through the io_uring_register_buf_ring() api once kmbufs are squashed into the pbuf interface.

With going through IORING_MEM_REGION, this would work for your use case as well. The user would have to register the mem region with io_uring_register_region() and pass in a read-only flag, and then the kernel will allocate the memory region. Then userspace would mmap the memory region and on the kernel side, it would set the mapping to be read-only. When the kmbufring then gets registered, the buffers in it will be empty. The filesystem will then have to populate the buffers in it from the mem region that was previously registered.

Thanks,
Joanne

Quoted 21 lines
Quoted 8 lines

To be completely honest, the more I look at this the more this feels like overkill / over-engineered to me. I get that now the user can do the PMD optimization, but does that actually lead to noticeable performance benefits? It seems especially confusing with them going through the same pbuf ring interface but having totally different expectations.

Yes. The PMD mapping also is not that relevant. Both AMD (implicit) and ARM (explicit) have optimizations for contiguous PTEs that are almost as valuable.

Quoted 5 lines

What about adding a straightforward kmbuf ring that goes through the pbuf interface (eg the design in this patchset) and then in the future adding an interface for pbuf rings (both kernel-managed and non-kernel-managed) to go through IORING_REGISTERED_MEM_REGIONS if users end up needing/wanting to have their rings populated that way?

That feels much simpler to me as well.

#19 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 4 lines · Joanne Koong

I think we have the exact same use case, except your buffers need to be read-only. I think your use case benefits from the same memory wins we'll get with incremental buffer consumption, which is the primary reason fuse is using a bufring instead of fixed buffers.

Yeah.

Quoted 5 lines

I think you can and it'll be very easy to do so. All that would be needed is to pass in a read-only flag from the userspace side when it registers the bufring, and then when userspace makes the mmap call to the bufring, the kernel checks if that read-only flag is set on the bufring and if so returns a read-only mapping.

Yes, tat's what I though. But Pavel seems to disagree?

#20 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Joanne Koong

I think we have the exact same use case, except your buffers need to be read-only. I think your use case benefits from the same memory wins we'll get with incremental buffer consumption, which is the primary reason fuse is using a bufring instead of fixed buffers.

Yeah.

Provided buffer rings are not useful for storage read/write requests because they bind to a buffer right away, that's in contrast to some recv request, where io_uring will first poll the socket to confirm the data is there, and only then take a buffer from the buffer ring and copy into it. With storage rw it makes more sense to specify the buffer directly gain control over where exactly data lands IOW, instead of the usual "read data into a given pointer" request semantics like what read(2) gives you, buffer rings are rather "read data somewhere and return a pointer to where you placed it".

Another problem is that someone needs to return buffers back into the buffer ring, and it's a kernel private ring. For this patchset it's assumed the fuse driver is going to be doing that, but there is no one for normal rw requests.

Quoted 7 lines
Quoted 5 lines

I think you can and it'll be very easy to do so. All that would be needed is to pass in a read-only flag from the userspace side when it registers the bufring, and then when userspace makes the mmap call to the bufring, the kernel checks if that read-only flag is set on the bufring and if so returns a read-only mapping.

Yes, tat's what I though. But Pavel seems to disagree?

Yes. You only need buffers, and it'll be better to base on sth that gives you buffers/memory without extra semantics, i.e. IORING_MEM_REGION. Or it can be a standalone registered buffer extension, likely reusing regions internally. That might even yield a finer API.

Signature · 1 line
Pavel Begunkov
#21 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Joanne Koong
Quoted 59 lines · Joanne Koong
Quoted 15 lines · Pavel Begunkov
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Joanne Koong
Quoted 3 lines

I don't think I follow. I'm saying that it might be interesting to separate rings from how and with what they're populated on the kernel API level, but the fuse kernel module can do the population

Oh okay, from your first message I (and I think christoph too) thought what you were saying is that the user should be responsible for allocating the buffers with complete ownership over them, and then just pass those allocated to the kernel to use. But what you're saying is that just use a different way for getting the kernel to allocate the buffers (eg through the IORING_REGISTER_MEM_REGION interface). Am I reading this correctly?

Quoted 3 lines
Quoted 3 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love

By "control the allocation fully" do you mean for your use case, the allocation/setup isn't triggered by userspace but is initiated by the kernel (eg user never explicitly registers any kbuf ring, the kernel just uses the kbuf ring data structure internally and users can read the buffer contents)? If userspace initiates the setup of the kbuf ring, going through IORING_REGISTER_MEM_REGION would be semantically the same, except the buffer allocation by the kernel now happens before the ring is created and then later populated into the ring. userspace would still need to make an mmap call to the region and the kernel could enforce that as read-only. But if userspace doesn't initiate the setup, then going through IORING_REGISTER_MEM_REGION gets uglier.

Quoted 8 lines
Quoted 1 line

to be able to piggy back than onto your work.

IORING_REGISTER_MEM_REGION supports both types of allocations. It can have a new registration flag for read-only, and then you either make the bounce avoidance optional or reject binding fuse to unsupported setups during init. Any arguments against that? I need to go over Joanne's reply, but I don't see any contradiction in principal with your use case.

So i guess the flow would have to be:
a) user calls io_uring_register_region(&ring, &mem_region_reg) with mem_region_reg.region_uptr's size field set to the total buffer size (and mem_region_reg.flags read-only bit set if needed) kernel allocates region
b) user calls mmap() to get the address of the region. If read-only bit was set, it gets a read-only address
c) user calls io_uring_register_buf_ring(&ring, &buf_reg, flags) with buf_reg.flags |= IOU_PBUF_RING_KERNEL_MANAGED
kernel creates an empty kernel-managed ring. None of the buffers
are populated
d) user tells X subsystem to populate the ring starting from offset Z in the registered mem region
e) on the kernel side, the subsystem populates the ring starting from offset Z, filling it up using the buf_size and ring_entries values that the user registered the ring with in c)

Quoted 4 lines

To be completely honest, the more I look at this the more this feels like overkill / over-engineered to me. I get that now the user can do the PMD optimization, but does that actually lead to noticeable performance benefits? It seems especially confusing with them going

No, it's mainly about not keeping payload buffers and rings in the same object from the io_uring uapi perspective.

1. If it's an io_uring uapi, it shouldn't be fuse specific or with a bunch of use case specific expectations attached. Why does it require all buffers to be uniform in size? Why does it require the ring size to match the number of buffers? Why does it require buffers to be allocated by io_uring in the first place? Maybe some subsystem got memory from somewhere else and wants to do use it with io_uring. Why does it need to know the total size at creation, and what would you do if you want to add more memory at runtime while using the same ring?

2. If it's meant to be fuse specific and _not_ used with other requests like recv/read/etc., then what's the point of having it as an io_uring uapi? Which also adds additional trouble like the once you're solving with pinning.

If it's supposed to be used with other requests, then buffers and rings will have different in-kernel lifetime expectations imposed by io_uring, so having them together won't even help with management.

I have a strong opinion about the memmap.c change. For the rest, if you believe it's fine, just send it out and let Jens decide.

Quoted 2 lines

through the same pbuf ring interface but having totally different expectations.

It's predicated on separating buffers from rings, see above, and assuming that I'm not sure what expectations are different apart from one being in-kernel with kernel addresses and the other user visible with user addresses.

Signature · 1 line
Pavel Begunkov
#22 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Pavel Begunkov
Quoted 7 lines · Pavel Begunkov
Quoted 4 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love to be able to piggy back than onto your work.

IORING_REGISTER_MEM_REGION supports both types of allocations. It can have a new registration flag for read-only, and then you either make

IORING_REGISTER_MEM_REGION seems to be all about cqs from both your commit message and the public documentation. I'm confused.

Quoted 4 lines

the bounce avoidance optional or reject binding fuse to unsupported setups during init. Any arguments against that? I need to go over Joanne's reply, but I don't see any contradiction in principal with your use case.

My use case is not about fuse, but good old block and file system I/O.

#23 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 11 lines · Christoph Hellwig
Quoted 7 lines · Pavel Begunkov
Quoted 4 lines

I'm arguing exactly against this. For my use case I need a setup where the kernel controls the allocation fully and guarantees user processes can only read the memory but never write to it. I'd love to be able to piggy back than onto your work.

IORING_REGISTER_MEM_REGION supports both types of allocations. It can have a new registration flag for read-only, and then you either make

IORING_REGISTER_MEM_REGION seems to be all about cqs from both your commit message and the public documentation. I'm confused.

Think of it as an area of memory for kernel-user communication. Used for syscall parameters passing to avoid copy_from_user, but I added it for a bunch of use cases. We'll hopefully get support at some point for passing request arguments like struct iovec. BPF patches use it for communication. I need to respin patches placing SQ/CQ onto it (avoid some memory waste).

Tbh, I never meant it nor io_uring regions to be used for huge payload buffers, but this series already uses regions for that.

Quoted 7 lines
Quoted 4 lines

the bounce avoidance optional or reject binding fuse to unsupported setups during init. Any arguments against that? I need to go over Joanne's reply, but I don't see any contradiction in principal with your use case.

My use case is not about fuse, but good old block and file system I/O.

Then I'm confused. Take a look at the other reply, this series is about buffer rings with kernel memory, it can't work without a kernel component returning buffers into the ring, and io_uring doesn't do that. But maybe you're thinking about adding some more elaborate API.

IIUC, Joanne also wants to add support for fuse installing registered buffers, which would allow zero-copy, but those got split out of this series.

Signature · 1 line
Pavel Begunkov
#24 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Joanne Koong
Quoted 3 lines · Joanne Koong
Quoted 2 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Tue, Feb 10, 2026 at 8:34 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 12 lines
Quoted 11 lines
Quoted 2 lines

checking) and different allocation calls (eg io_region_allocate_pages() vs io_region_allocate_pages_multi_buf()).

I saw that and saying that all memmap.c changes can get dropped. You're using it as one big virtually contig kernel memory range then chunked into buffers, and that's pretty much what you're getting with normal io_create_region(). I get that you only need it to be contiguous within a single buffer, but that's not what you're doing, and it'll be only worse than default io_create_region() e.g. effectively disabling any usefulness of io_mem_alloc_compound(), and ultimately you don't need to care.

Quoted 7 lines

When I originally implemented it, I had it use io_region_allocate_pages() but this fails because it's allocating way too much memory at once. For fuse's use case, each buffer is usually at least 1 MB if not more. Allocating the memory one buffer a time in io_region_allocate_pages_multi_buf() bypasses the allocation errors I was seeing. That's the main reason I don't think this can just use io_create_region().

Let's fix that then. For now, just work it around by wrapping into a loop.

Btw, I thought you're going to use it for metadata like some fuse headers and payloads would be zero copied by installing it as registered buffers.

...

Quoted 17 lines
Quoted 13 lines
Quoted 12 lines
Quoted 2 lines

Provided buffer rings with kernel addresses could be an interesting abstraction, but why is it also responsible for allocating buffers?

Conceptually, I think it makes the interface and lifecycle management simpler/cleaner. With registering it from userspace, imo there's additional complications with no tangible benefits, eg it's not guaranteed that the memory regions registered for the buffers are the same size, with allocating it from the kernel-side we can guarantee that the pages are allocated physically contiguously, userspace setup with user-allocated buffers is less straightforward, etc. In general, I'm just not really seeing what advantages there are in allocating the buffers from userspace. Could you elaborate on that part more?

Quoted 3 lines

I don't think I follow. I'm saying that it might be interesting to separate rings from how and with what they're populated on the kernel API level, but the fuse kernel module can do the population

Quoted 7 lines

Oh okay, from your first message I (and I think christoph too) thought what you were saying is that the user should be responsible for allocating the buffers with complete ownership over them, and then just pass those allocated to the kernel to use. But what you're saying is that just use a different way for getting the kernel to allocate the buffers (eg through the IORING_REGISTER_MEM_REGION interface). Am I reading this correctly?

The main point is disentangling memory allocation from ring creation in the io_uring uapi, and moving ring population into fuse instead of doing it at creation. And it'll still be populated by the kernel (fuse), user space doesn't have access to the ring. IORING_REGISTER_MEM_REGION is just the easiest way to achieve that without any extra uapi.

...

Quoted 9 lines
Quoted 5 lines

Pinning can be gated on that flag as well. Pretty likely uapi and internals will be a bit cleaner, but that's not a huge deal, just don't see why would you roll out a separate set of uapi ([un]register, offsets, etc.) when essentially it can be treated as the same thing.

imo, it looked cleaner as a separate api because it has different expectations and behaviors and squashing kmbuf into the pbuf api makes the pbuf api needlessly more complex. Though I guess from the

It appeared to me that they're different because of special region path and embedded buffer allocations, and otherwise differences would be minimal. But if you think it's still better to be made as a separate opcode, I'm not opposing it, go for it.

Quoted 11 lines

userspace pov, liburing could have a wrapper that takes care of setting up the pbuf details for kernel-managed pbufs. But in my head, having pbufs vs. kmbufs makes it clearer what each one does vs regular pbufs vs. pbufs that are kernel-managed.

Especially with now having kmbufs go through the ioring mem region interface, it makes things more confusing imo if they're combined, eg pbufs that are kernel-managed are created empty and then populated from the kernel side by whatever subsystem is using them. Right now there's only one mem region supported per ring, but in the future if there's the possibility that multiple mem regions can be registered

That shouldn't be a problem

Quoted 4 lines

(eg if userspace doesn't know upfront what mem region length they'll need), then we should also probably add in a region id param for the registration arg, which if kmbuf rings go through the pbuf ring registration api, is not possible to do.

Not having patches using the functionality is inconvenient. How fuse looks up the buffer ring from io_uring? I could imagine you have some control path io-uring command:

case FUSE_CMD_BIND_BUFFER_RING:

	return bind_queue(params);

Then you can pass all necessary parameters to it, pseudo code:

struct fuse_bind_kmbuf_ring_params {
	region_id;
	buf_ring_id;
	...
};

bind_queue(cmd, struct fuse_bind_kmbuf_ring_params *p)

{
	region = io_uring_get_region(cmd, p->region_id);
	// get exclusive access:
	buf_ring = io_uring_get_buf_ring(cmd, p->buf_ring_id);

	if (!validate_buf_ring(buf_ring))
		return NOTSUPPORTED;

	io_uring_pin(buf_ring);
	fuse_populate_buf_ring(buf_ring, region, ...);
}

Does that match expectations? I don't think you even need the ring part exposed as an io_uring uapi, tbh, as it stays completely in fuse and doesn't meaningfully interact with the rest of io_uring.

...

Quoted 10 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

3. Require the user to register a memory region of appropriate size, see IORING_REGISTER_MEM_REGION, ctx->param_region. Make fuse populating the buffer ring using the memory region.

To explain why, I don't think that creating many small regions is a good direction going forward. In case of kernel allocation, it's extra mmap()s, extra user space management, and wasted space.

To clarify, is this in reply to why the individual buffers shouldn't be allocated separately by the kernel?

That was about an argument for using IORING_REGISTER_MEM_REGION instead a separate region. And it's separate from whether buffers should be bound to the ring.

Quoted 10 lines

I added a comment about this above in the discussion about io_region_allocate_pages_multi_buf(), and if the memory allocation issue I was seeing is bypassable and the region can be allocated all at once, I'm happy to make that change. With having the allocation be separate buffers though, I'm not sure I agree that there are extra mmaps / userspace management. All the pages across the buffers are vmapped together and the userspace just needs to do 1 mmap call for them. On the userspace side, I don't think there's more management since the mmapped address represents the range across all the buffers. I'm not seeing how there's wasted space either since the only

I shouldn't affect you much since you have such large buffers, but imagine the total allocation size is not being pow2, and the kernel allocating it as a single folio. E.g. 3 buffers, 0.5 MB each, total = 1.5MB, and the kernel allocates a 2MB huge page.

Signature · 1 line
Pavel Begunkov
#25 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Joanne Koong
Quoted 53 lines · Joanne Koong
Quoted 42 lines · Pavel Begunkov
Quoted 29 lines · Joanne Koong
Quoted 18 lines · Pavel Begunkov
Quoted 4 lines
io_uring/kbuf.c +0 -0
index aa9b70b72db4..9bc36451d083 100644

...

Quoted 13 lines
io_uring/kbuf.c +13 -0
+static int io_setup_kmbuf_ring(struct io_ring_ctx *ctx,
+                            struct io_buffer_list *bl,
+                            struct io_uring_buf_reg *reg)
+{
+     struct io_uring_buf_ring *ring;
+     unsigned long ring_size;
+     void *buf_region;
+     unsigned int i;
+     int ret;
+
+     /* allocate pages for the ring structure */
+     ring_size = flex_array_size(ring, bufs, bl->nr_entries);
+     ring = kzalloc(ring_size, GFP_KERNEL_ACCOUNT);
Quoted 8 lines

io_uring/kbuf.c struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx, · line 803

Quoted 5 lines
io_uring/kbuf.c +5 -0
+     if (!ring)
+             return -ENOMEM;
+
+     ret = io_create_region_multi_buf(ctx, &bl->region, bl->nr_entries,
+                                      reg->buf_size);

Please use io_create_region(), the new function does nothing new and only violates abstractions.

Quoted 2 lines

There's separate checks needed between io_create_region() and io_create_region_multi_buf() (eg IORING_MEM_REGION_TYPE_USER flag

If io_create_region() is too strict, let's discuss that in examples if there are any, but it's likely not a good idea changing that. If it's too lax, filter arguments in the caller. IOW, don't pass IORING_MEM_REGION_TYPE_USER if it's not used.

Quoted 2 lines

checking) and different allocation calls (eg io_region_allocate_pages() vs io_region_allocate_pages_multi_buf()).

Quoted 8 lines

I saw that and saying that all memmap.c changes can get dropped. You're using it as one big virtually contig kernel memory range then chunked into buffers, and that's pretty much what you're getting with normal io_create_region(). I get that you only need it to be contiguous within a single buffer, but that's not what you're doing, and it'll be only worse than default io_create_region() e.g. effectively disabling any usefulness of io_mem_alloc_compound(), and ultimately you don't need to care.

Quoted 12 lines

When I originally implemented it, I had it use io_region_allocate_pages() but this fails because it's allocating way too much memory at once. For fuse's use case, each buffer is usually at least 1 MB if not more. Allocating the memory one buffer a time in io_region_allocate_pages_multi_buf() bypasses the allocation errors I was seeing. That's the main reason I don't think this can just use io_create_region().

Quoted 4 lines

Regions shouldn't know anything about your buffers, how it's subdivided after, etc.

I still think the memory for the buffers should be tied to the ring itself and allocated physically contiguously per buffer. Per-buffer contiguity will enable the most efficient DMA path for servers to send read/write data to local storage or the network. If the buffers for the bufring have to be allocated as one single memory region, the io_mem_alloc_compound() call will fail for this large allocation size. Even if io_mem_alloc_compound() did succeed, this is a waste as the buffer pool as an entity doesn't need to be physically contiguous, just the individual buffers themselves. For fuse, the server configures what buffer pool size it wants to use, depending on what queue depth and max request size it needs. So for most use cases, at least for high-performance servers, allocation will have to fall back to alloc_pages_bulk_node(), which doesn't allocate contiguously. You mentioned in an earlier comment that this "only violates abstractions"
- which abstractions does this break? The pre-existing behavior already defaults to allocating pages non-contiguously if the mem region can't be allocated fully contiguously.

Going through registered buffers doesn't help either. Fuse servers can be unprivileged and it's not guaranteed that there are enough huge pages reserved or that another process hasn't taken them or that the server has privileges to pre-reserve pages for the allocation. Also the 2 MB granularity is inflexible while 1 GB is too much.

I'm not really seeing a way where we can honor the physical contiguity requirements for the buffers without going through kernel-managed bufrings with the allocation done on a per-buffer basis. Or am I missing something here?

Thanks,
Joanne

#26 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

On 2/27/26 01:12, Joanne Koong wrote:
...

Quoted 12 lines
Quoted 2 lines
Quoted 2 lines

Regions shouldn't know anything about your buffers, how it's subdivided after, etc.

I still think the memory for the buffers should be tied to the ring itself and allocated physically contiguously per buffer. Per-buffer contiguity will enable the most efficient DMA path for servers to send read/write data to local storage or the network. If the buffers for the bufring have to be allocated as one single memory region, the io_mem_alloc_compound() call will fail for this large allocation size. Even if io_mem_alloc_compound() did succeed, this is a waste as the buffer pool as an entity doesn't need to be physically contiguous, just the individual buffers themselves. For fuse, the server

Quoted 8 lines

configures what buffer pool size it wants to use, depending on what queue depth and max request size it needs. So for most use cases, at least for high-performance servers, allocation will have to fall back to alloc_pages_bulk_node(), which doesn't allocate contiguously. You mentioned in an earlier comment that this "only violates abstractions"
- which abstractions does this break? The pre-existing behavior already defaults to allocating pages non-contiguously if the mem region can't be allocated fully contiguously.

Regions has uapi (see struct io_uring_region_desc) so that users can operate with them in a unified manner. If you want regions to be allocated in some special way, just extend it.

Quoted 4 lines

Going through registered buffers doesn't help either. Fuse servers can be unprivileged and it's not guaranteed that there are enough huge pages reserved or that another process hasn't taken them or that the server has privileges to pre-reserve pages for the allocation. Also

There is THP these days. And FWIW, we should be vigilant about not using io_uring to work around capabilities and mm policies. If user can't do it, io_uring shouldn't either. It's also all accounted against mlock, if the limit is not high enough, you won't be able to use this feature at all.

Quoted 1 line

the 2 MB granularity is inflexible while 1 GB is too much.

Signature · 1 line
Pavel Begunkov
#27 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 24 lines · Pavel Begunkov

On 2/27/26 01:12, Joanne Koong wrote:
...

Quoted 12 lines
Quoted 2 lines
Quoted 2 lines

Regions shouldn't know anything about your buffers, how it's subdivided after, etc.

I still think the memory for the buffers should be tied to the ring itself and allocated physically contiguously per buffer. Per-buffer contiguity will enable the most efficient DMA path for servers to send read/write data to local storage or the network. If the buffers for the bufring have to be allocated as one single memory region, the io_mem_alloc_compound() call will fail for this large allocation size. Even if io_mem_alloc_compound() did succeed, this is a waste as the buffer pool as an entity doesn't need to be physically contiguous, just the individual buffers themselves. For fuse, the server

Quoted 8 lines

configures what buffer pool size it wants to use, depending on what queue depth and max request size it needs. So for most use cases, at least for high-performance servers, allocation will have to fall back to alloc_pages_bulk_node(), which doesn't allocate contiguously. You mentioned in an earlier comment that this "only violates abstractions"
- which abstractions does this break? The pre-existing behavior already defaults to allocating pages non-contiguously if the mem region can't be allocated fully contiguously.

Quoted 3 lines

Regions has uapi (see struct io_uring_region_desc) so that users can operate with them in a unified manner. If you want regions to be allocated in some special way, just extend it.

You can't say "regions shouldn't know anything about your buffers, how it's subdivided, etc" and then also say "extend the region uapi for special allocation to make it buffer-compatible". If we extend the region uapi to specify contiguous chunks of size X starting at offset Y for len Z, that is basically encoding buffer layout information into the region. The buffer ring already knows buffer sizes and count - it is the natural place to express contiguity requirements.

Pushing this into the region abstraction muddies the uapi and forces awkward indirection where callers now need to manually synchronize region chunk specifications with their buffer layout. Memory regions are generic and will be used for purposes beyond kmbufs. forcing buffer-specific allocation semantics into the region UAPI pollutues a general abstraction with domain-specific details.

Quoted 7 lines
Quoted 5 lines

Going through registered buffers doesn't help either. Fuse servers can be unprivileged and it's not guaranteed that there are enough huge pages reserved or that another process hasn't taken them or that the server has privileges to pre-reserve pages for the allocation. Also

There is THP these days. And FWIW, we should be vigilant about not

THP is opportunistic and not guaranteed. It depends on external factors like fragmentation, memory pressure, system settings, etc. For high-performance FUSE servers where deterministic DMA efficiency is required, this doesn't suffice.

Quoted 1 line

using io_uring to work around capabilities and mm policies. If user

This isn't working around capabilities / mm policies. The user isn't getting contiguous physical memory to use freely, the kernel is allocating it internally to service I/O efficiently. Providing infrastructure for efficient DMA isn't a capability / mm bypass, this is standard kernel behavior. When userspace does i/o through sockets or block devices, the kernel routinely allocates contiguous memory with dma_alloc_coherent() or alloc_pages() with order > 0. That's exactly the point I'm trying to make - users shouldn't have to do this themselves (eg going through registered buffers with user-allocated buffers). The kernel should handle it internally.

Quoted 3 lines

can't do it, io_uring shouldn't either. It's also all accounted against mlock, if the limit is not high enough, you won't be able to use this feature at all.

The mlock point is orthogonal. it restricts how much memory a user can pin, but contiguous and noncontiguous allocations of the same size consume the same mlock budget.

Thanks,
Joanne

Quoted 6 lines
Quoted 2 lines

the 2 MB granularity is inflexible while 1 GB is too much.

--
Pavel Begunkov

#28 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

In reply to Pavel Begunkov
Quoted 9 lines · Pavel Begunkov

include/uapi/linux/io_uring.h enum io_uring_register_pbuf_ring_flags { · line 906

Quoted 5 lines
include/uapi/linux/io_uring.h +5 -0
+	union {
+		/* used for pbuf rings */
+		__u64	ring_addr;
+		/* used for kmbuf rings */
+		__u32   buf_size;

If you're creating a region, there should be no reason why it can't work with user passed memory. You're fencing yourself off optimisations that are already there like huge pages.

Any pages mapped to userspace can be allocated in the kernel as well.

And I really do like this design, because it means we can have a buffer ring that is only mapped read-only into userspace. That way we can still do zero-copy raids if the device requires stable pages for checksumming or raid. I was going to implement this as soon as this series lands upstream.

#29 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Pavel Begunkov

include/uapi/linux/io_uring.h enum io_uring_register_pbuf_ring_flags { · line 906

Quoted 5 lines
include/uapi/linux/io_uring.h +5 -0
+	union {
+		/* used for pbuf rings */
+		__u64	ring_addr;
+		/* used for kmbuf rings */
+		__u32   buf_size;

If you're creating a region, there should be no reason why it can't work with user passed memory. You're fencing yourself off optimisations that are already there like huge pages.

Quoted 1 line

Any pages mapped to userspace can be allocated in the kernel as well.

pow2 round ups will waste memory. 1MB allocations will never become 2MB huge pages. And there is a separate question of 1GB huge pages. The user can be smarter about all placement decisions.

Quoted 5 lines

And I really do like this design, because it means we can have a buffer ring that is only mapped read-only into userspace. That way we can still do zero-copy raids if the device requires stable pages for checksumming or raid. I was going to implement this as soon as this series lands upstream.

That's an interesting case. To be clear, user provided memory is an optional feature for pbuf rings / regions / etc., and I think the io_uring uapi should leave fields for the feature. However, I have nothing against fuse refusing to bind to buffer rings it doesn't like.

Signature · 1 line
Pavel Begunkov
#30 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 7 lines · Pavel Begunkov
Quoted 2 lines

Any pages mapped to userspace can be allocated in the kernel as well.

pow2 round ups will waste memory. 1MB allocations will never become 2MB huge pages. And there is a separate question of 1GB huge pages. The user can be smarter about all placement decisions.

Sure. But if the application cares that much about TLB pressure I'd just round up to nice multtiple of PTE levels.

Quoted 7 lines
Quoted 6 lines

And I really do like this design, because it means we can have a buffer ring that is only mapped read-only into userspace. That way we can still do zero-copy raids if the device requires stable pages for checksumming or raid. I was going to implement this as soon as this series lands upstream.

Quoted 5 lines

That's an interesting case. To be clear, user provided memory is an optional feature for pbuf rings / regions / etc., and I think the io_uring uapi should leave fields for the feature. However, I have nothing against fuse refusing to bind to buffer rings it doesn't like.

Can you clarify what you mean with 'pbuf'? The only fixed buffer API I know is io_uring_register_buffers* which always takes user provided buffers, so I have a hard time parsing what you're saying there. But that might just be sign that I'm no expert in io_uring APIs, and that web searches have degraded to the point of not being very useful anymore.

#31 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 25 lines · Christoph Hellwig
Quoted 7 lines · Pavel Begunkov
Quoted 2 lines

Any pages mapped to userspace can be allocated in the kernel as well.

pow2 round ups will waste memory. 1MB allocations will never become 2MB huge pages. And there is a separate question of 1GB huge pages. The user can be smarter about all placement decisions.

Sure. But if the application cares that much about TLB pressure I'd just round up to nice multtiple of PTE levels.

Quoted 7 lines
Quoted 6 lines

And I really do like this design, because it means we can have a buffer ring that is only mapped read-only into userspace. That way we can still do zero-copy raids if the device requires stable pages for checksumming or raid. I was going to implement this as soon as this series lands upstream.

Quoted 5 lines

That's an interesting case. To be clear, user provided memory is an optional feature for pbuf rings / regions / etc., and I think the io_uring uapi should leave fields for the feature. However, I have nothing against fuse refusing to bind to buffer rings it doesn't like.

Quoted 6 lines

Can you clarify what you mean with 'pbuf'? The only fixed buffer API I know is io_uring_register_buffers* which always takes user provided buffers, so I have a hard time parsing what you're saying there. But that might just be sign that I'm no expert in io_uring APIs, and that web searches have degraded to the point of not being very useful anymore.

Registered, aka fixed, buffers are the ones you pass to IORING_OP_[READ,WRITE]_FIXED and some other requests. It's normally created by io_uring_register_buffers*() / IORING_REGISTER_BUFFERS* with user memory, but there are special cases when it's installed internally by other kernel components, e.g. ublk.
This series has nothing to do with them, and relevant parts of the discussion here don't mention them either.

Provided buffer rings, a.k.a pbuf rings, IORING_REGISTER_PBUF_RING is a kernel-user shared ring. The entries are user buffers {uaddr, size}. The user space adds entries, the kernel (io_uring requests) consumes them and issues I/O using the user addresses. E.g. you can issue a IORING_OP_RECV request (+IOSQE_BUFFER_SELECT) and it'll grab a buffer from the ring instead of using sqe->addr.

pbuf rings, IORING_REGISTER_MEM_REGION, completion/submission queues and all other kernel-user rings/etc. are internally based on so called regions. All of them support both user allocated memory and kernel allocations + mmap.

This series essentially creates provided buffer rings, where
1. the ring now contains kernel addresses
2. the ring itself is in-kernel only and not shared with user space
3. it also allocates kernel buffers (as a region), populates the ring with them, and allows mapping the buffers into the user space.

Fuse is doing both adding (kernel) buffers to the ring and consuming them. At which point it's not clear:

1. Why it even needs io_uring provided buffer rings, it can be all contained in fuse. Maybe it's trying to reuse pbuf ring code as basically an internal memory allocator, but then why expose buffer rings as an io_uring uapi instead of keeping it internally.

That's also why I mentioned whether those buffers are supposed to be used with other types of io_uring requests like recv, etc.

2. Why making io_uring to allocate payload memory. The answer to which is probably to reuse the region api with mmap and so on. And why payload buffers are inseparably created together with the ring and via a new io_uring uapi.

And yes, I believe in the current form it's inflexible, it requires a new io_uring uapi. It requires the number of buffers to match the number of ring entries, which are related but not the same thing. You can't easily add more memory as it's bound to the ring object. The buffer memory won't even have same lifetime as the ring object -- allow using that km buffer ring with recv requests and highly likely I'll most likely give you a way to crash the kernel.

But hey, I'm tired. I don't have any beef here and am only trying to make it a bit cleaner and flexible for fuse in the first place without even questioning the I/O path. If everyone believes everything is right, just ask Jens to merge it.

Signature · 1 line
Pavel Begunkov
#32 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 31 lines · Pavel Begunkov
Quoted 8 lines · Christoph Hellwig
Quoted 1 line · Pavel Begunkov
Quoted 1 line

Can you clarify what you mean with 'pbuf'? The only fixed buffer API I know is io_uring_register_buffers* which always takes user provided buffers, so I have a hard time parsing what you're saying there. But that might just be sign that I'm no expert in io_uring APIs, and that web searches have degraded to the point of not being very useful anymore.

Registered, aka fixed, buffers are the ones you pass to IORING_OP_[READ,WRITE]_FIXED and some other requests. It's normally created by io_uring_register_buffers*() / IORING_REGISTER_BUFFERS* with user memory, but there are special cases when it's installed internally by other kernel components, e.g. ublk.
This series has nothing to do with them, and relevant parts of the discussion here don't mention them either.

Provided buffer rings, a.k.a pbuf rings, IORING_REGISTER_PBUF_RING is a kernel-user shared ring. The entries are user buffers {uaddr, size}. The user space adds entries, the kernel (io_uring requests) consumes them and issues I/O using the user addresses. E.g. you can issue a IORING_OP_RECV request (+IOSQE_BUFFER_SELECT) and it'll grab a buffer from the ring instead of using sqe->addr.

pbuf rings, IORING_REGISTER_MEM_REGION, completion/submission queues and all other kernel-user rings/etc. are internally based on so called regions. All of them support both user allocated memory and kernel allocations + mmap.

Quoted 5 lines

This series essentially creates provided buffer rings, where
1. the ring now contains kernel addresses
2. the ring itself is in-kernel only and not shared with user space
3. it also allocates kernel buffers (as a region), populates the ring with them, and allows mapping the buffers into the user space.

The most important part and the whole reason fuse needs the buffer ring to be kernel-managed is because the kernel needs to control when buffers get recycled back into the ring. For fuse's use case, the buffer is used for passing data between the kernel and the server. We can't have the server recycle the buffer because the server writes back data to the kernel in that buffer when it submits the sqe. After fuse receives the sqe and reads the reply from the server, it then needs to recycle that buffer back into the ring so it can be reused for a future cqe (eg sending a future request).

Quoted 11 lines

Fuse is doing both adding (kernel) buffers to the ring and consuming them. At which point it's not clear:

1. Why it even needs io_uring provided buffer rings, it can be all contained in fuse. Maybe it's trying to reuse pbuf ring code as basically an internal memory allocator, but then why expose buffer rings as an io_uring uapi instead of keeping it internally.

That's also why I mentioned whether those buffers are supposed to be used with other types of io_uring requests like recv, etc.

On the userspace/server side, it uses the buffers for other io-uring operations (eg reading or writing the contents from/to a locally-backed file).

Quoted 4 lines

2. Why making io_uring to allocate payload memory. The answer to which is probably to reuse the region api with mmap and so on. And why payload buffers are inseparably created together with the ring

My main motivation for this is simplicity. I see (and thanks for explaining) that using a registered mem region allows the use of some optimizations (the only one I know of right now is the PMD one you mentioned but maybe there's more I'm missing) that could be useful for some workloads, but I don't think (and this could just be my lack of understanding of what more optimizations there are) most use cases of kmbufs benefit from those optimizations, so to me it feels like we're adding non-trivial complexity for no noticeable benefit.

I feel like we get the best of both worlds by letting users have both:
the simple kernel-managed pbuf where the kernel allocates the buffers and the buffers are tied to the lifecycle of the ring, and the more advanced kernel-managed pbuf where buffers are tied to a registered memory region that the subsystem is responsible for later populating the ring with.

Quoted 1 line

and via a new io_uring uapi.

imo it felt cleaner to have a new uapi for it because kmbufs and pbufs have different expectations and behaviors (eg pbufs only work with user-provided buffers and requires userspace to populate the ring before using it, whereas for kmbufs the kernel allocates the buffers and populates it for you; pbufs require userspace to recycle back the buffer, whereas for kmbufs the kernel is the one in control of recycling) and from the user pov it seemed confusing to have kmbufs as part of the pbuf ring uapi, instead of separating it out as a different type of ringbuffer with a different expectation and behavior. I was trying to make the point that combining the interface if we go with IORING_MEM_REGION gets even more confusing because now pbufs that are kernel-managed are also empty at initialization and only can point to areas inside a registered mem region and the responsibility of populating it is now on whatever subsystem is using it.

I still have this opinion but I also think in general, you likely know better than I do what kind of io-uring uapi is best for io-uring's users. For v2 I'll have kmbufs go through the pbuf uapi.

Quoted 4 lines

And yes, I believe in the current form it's inflexible, it requires a new io_uring uapi. It requires the number of buffers to match the number of ring entries, which are related but not the same

I'm not really seeing what the purpose of having a ring entry with no buffer associated with it is. In the existing code for non-kernel managed pbuf rings, there's the same tie between reg->ring_entries being used as the marker for how many buffers the ring supports. But if the number of buffers should be different than the number of ring entries, this can be easily fixed by passing in the number of buffers from the uapi for kernel-managed pbuf rings.

Quoted 2 lines

thing. You can't easily add more memory as it's bound to the ring object. The buffer memory won't even have same lifetime as the

To play devil's advocate, we also can't easily add more memory to the mem region once it's been registered. I think there's also a worse penalty where the user needs to know upfront how much memory to allocate for the mem region for the lifetime of the ring, which imo may be hard to do (eg if a kernel-managed buf ring only needs to be registered for some code paths and not others, the mem region registration would still have to allocate the memory a potential kbuf ring would use).

Quoted 3 lines

ring object -- allow using that km buffer ring with recv requests and highly likely I'll most likely give you a way to crash the kernel.

I'm a bit confused by this part. The buffer memory does have the same lifetime as the ring object, no? The buffers only get freed when the ring itself is freed.

Quoted 4 lines

But hey, I'm tired. I don't have any beef here and am only trying to make it a bit cleaner and flexible for fuse in the first place without even questioning the I/O path. If everyone believes

I appreciate you looking at this and giving your feedback and insight. Thank you for doing so. I don't want to merge in something you're unhappy with.

Are you open to having support for both a simple kernel-managed pbuf interface and later on if/when the need arises, a kernel-managed pbuf interface that goes through a registered memory region? If the answer is no, then I'll make the change to have kmbufs go through the registered memory region.

Thanks,
Joanne

Quoted 5 lines

everything is right, just ask Jens to merge it.

--
Pavel Begunkov

#33 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 1 line · Joanne Koong

On Fri, Feb 13, 2026 at 4:41 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 8 lines
Quoted 8 lines

Fuse is doing both adding (kernel) buffers to the ring and consuming them. At which point it's not clear:

1. Why it even needs io_uring provided buffer rings, it can be all contained in fuse. Maybe it's trying to reuse pbuf ring code as basically an internal memory allocator, but then why expose buffer rings as an io_uring uapi instead of keeping it internally.

Quoted 6 lines
Quoted 2 lines

That's also why I mentioned whether those buffers are supposed to be used with other types of io_uring requests like recv, etc.

On the userspace/server side, it uses the buffers for other io-uring operations (eg reading or writing the contents from/to a locally-backed file).

Oops, typo. I was asking whether the buffer rings (not buffers) are supposed to be used with other requests. E.g. submitting a IORING_OP_RECV with IOSQE_BUFFER_SELECT set and the bgid specifying your kernel-managed buffer ring.

Quoted 4 lines
Quoted 3 lines

2. Why making io_uring to allocate payload memory. The answer to which is probably to reuse the region api with mmap and so on. And why payload buffers are inseparably created together with the ring

Quoted 8 lines

My main motivation for this is simplicity. I see (and thanks for explaining) that using a registered mem region allows the use of some optimizations (the only one I know of right now is the PMD one you mentioned but maybe there's more I'm missing) that could be useful for some workloads, but I don't think (and this could just be my lack of understanding of what more optimizations there are) most use cases of kmbufs benefit from those optimizations, so to me it feels like we're adding non-trivial complexity for no noticeable benefit.

There are two separate arguments. The first is about not making buffers inseparable from buffer rings in the io_uring user API. Whether it's IORING_REGISTER_MEM_REGION or something else is not that important. I have no objection if it's a part of fuse instead though, e.g. if fuse binds two objects together when you register it with fuse, or even if fuse create a buffer ring internally (assuming it doesn't indirectly leak into io_uring uapi).

And the second was about optionally allowing user memory for buffer creation as you're reusing the region abstraction. You can find pros and cons for both modes, and funnily enough, SQ/CQ were first kernel allocated and then people asked for backing it by user memory, and IIRC it was in the reverse order for pbuf rings.

Implementing this is trivial as well, you just need to pass an argument while creating a region. All new region users use struct io_uring_region_desc for uapi and forward it to io_create_region() without caring if it's user or kernel allocated memory.

Quoted 10 lines

I feel like we get the best of both worlds by letting users have both:
the simple kernel-managed pbuf where the kernel allocates the buffers and the buffers are tied to the lifecycle of the ring, and the more advanced kernel-managed pbuf where buffers are tied to a registered memory region that the subsystem is responsible for later populating the ring with.

Quoted 1 line

and via a new io_uring uapi.

imo it felt cleaner to have a new uapi for it because kmbufs and pbufs

The stress is on why it's an _io_uring_ API. It doesn't matter to me whether it's a separate opcode or not. Currently, buffer rings don't give you anything that can't be pure fuse, and it might be simpler to have it implemented in fuse than binding to some io_uring object. Or it could create buffer rings internally to reuse code but it doesn't become an io_uring uapi but rather implementation detail. And that predicates on whether km rings are intended to be used with other / non-fuse requests.

Quoted 8 lines

have different expectations and behaviors (eg pbufs only work with user-provided buffers and requires userspace to populate the ring before using it, whereas for kmbufs the kernel allocates the buffers and populates it for you; pbufs require userspace to recycle back the buffer, whereas for kmbufs the kernel is the one in control of recycling) and from the user pov it seemed confusing to have kmbufs as part of the pbuf ring uapi, instead of separating it out as a different type of ringbuffer with a different expectation and

I believe the source of disagreement is that you're thinking about how it's going to look like for fuse specifically, and I believe you that it'll be nicer for the fuse use case. However, on the other hand it's an io_uring uapi, and if it is an io_uring uapi, we need reusable blocks that are not specific to particular users.

If it km rings has to stay an io_uring uapi, I guess a middle ground would be to allow registering km rings together with memory, but make it a pure region without a notion of a buffer, and let fuse to chunk it. Later, we can make payload memory allocation optional.

Quoted 6 lines

behavior. I was trying to make the point that combining the interface if we go with IORING_MEM_REGION gets even more confusing because now pbufs that are kernel-managed are also empty at initialization and only can point to areas inside a registered mem region and the responsibility of populating it is now on whatever subsystem is using it.

Right, intentionally so, because otherwise it's a fuse uapi that pretends to be a generic io_uring uapi but it's not because of all assumptions in different places.

Quoted 5 lines

I still have this opinion but I also think in general, you likely know better than I do what kind of io-uring uapi is best for io-uring's users. For v2 I'll have kmbufs go through the pbuf uapi.

Quoted 1 line
Quoted 8 lines
Quoted 3 lines

And yes, I believe in the current form it's inflexible, it requires a new io_uring uapi. It requires the number of buffers to match the number of ring entries, which are related but not the same

I'm not really seeing what the purpose of having a ring entry with no buffer associated with it is. In the existing code for non-kernel managed pbuf rings, there's the same tie between reg->ring_entries being used as the marker for how many buffers the ring supports. But

Not really, it tells the buffer ring depth but says nothing about how much memory user space allocated and how it's pushed. It's a reasonable default but they could be different. For example, if you expect adding more memory at runtime, you might create the buffer ring a bit larger. Or when server processing takes a while and you can't recycle until it finishes, you might have more buffers than you need ring entries. Or you might might decide to split buffers and as you mentioned incremental consumption, which is an entire separate topic because it doesn't do de-fragmentation and you'd need to have it in fuse, just like user space does with pbufs.

Quoted 3 lines

if the number of buffers should be different than the number of ring entries, this can be easily fixed by passing in the number of buffers from the uapi for kernel-managed pbuf rings.

My entire point is that we're making lots of assumptions for io_uring uapi, and if it's moved to fuse because it knows better what it needs, it should be a win.

IOW, it sounds better if instead of passing the number of buffers to io_uring, you just ask it to create a large chunk of memory, and then fuse chunks it up and puts into the ring.

Quoted 11 lines
Quoted 2 lines

thing. You can't easily add more memory as it's bound to the ring object. The buffer memory won't even have same lifetime as the

To play devil's advocate, we also can't easily add more memory to the mem region once it's been registered. I think there's also a worse penalty where the user needs to know upfront how much memory to allocate for the mem region for the lifetime of the ring, which imo may be hard to do (eg if a kernel-managed buf ring only needs to be registered for some code paths and not others, the mem region registration would still have to allocate the memory a potential kbuf ring would use).

I agree, and you'd need something new in either case to add more memory, and it doesn't need to be IORING_REGISTER_MEM_REGION specifically.

Quoted 7 lines
Quoted 3 lines

ring object -- allow using that km buffer ring with recv requests and highly likely I'll most likely give you a way to crash the kernel.

I'm a bit confused by this part. The buffer memory does have the same lifetime as the ring object, no? The buffers only get freed when the ring itself is freed.

Unregistering a buffer ring doesn't guarantee that there are no inflight requests that are still using buffers that came out of the buffer ring. The fuse driver can wait/terminate its requests before unregisteration, but allow userspace issued IORING_OP_RECV to use this km buffer ring, and you'll need to somehow synchronise with all other io_uring requests.

Signature · 1 line
Pavel Begunkov
#34 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 12 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Fri, Feb 13, 2026 at 4:41 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 8 lines
Quoted 8 lines

Fuse is doing both adding (kernel) buffers to the ring and consuming them. At which point it's not clear:

1. Why it even needs io_uring provided buffer rings, it can be all contained in fuse. Maybe it's trying to reuse pbuf ring code as basically an internal memory allocator, but then why expose buffer rings as an io_uring uapi instead of keeping it internally.

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

That's also why I mentioned whether those buffers are supposed to be used with other types of io_uring requests like recv, etc.

On the userspace/server side, it uses the buffers for other io-uring operations (eg reading or writing the contents from/to a locally-backed file).

Sorry, I submitted v2 last night thinking the conversation on this thread had died. After reading through your reply, I'll modify v2.

Quoted 4 lines

Oops, typo. I was asking whether the buffer rings (not buffers) are supposed to be used with other requests. E.g. submitting a IORING_OP_RECV with IOSQE_BUFFER_SELECT set and the bgid specifying your kernel-managed buffer ring.

Yes the buffer rings are intended to be used with other io-uring requests. The ideal scenario is that the user can then do the equivalent of IORING_OP_READ/WRITE_FIXED operations on the kernel-managed buffers and avoid the per-i/o page pinning overhead costs.

Quoted 61 lines
Quoted 5 lines
Quoted 4 lines

2. Why making io_uring to allocate payload memory. The answer to which is probably to reuse the region api with mmap and so on. And why payload buffers are inseparably created together with the ring

Quoted 8 lines

My main motivation for this is simplicity. I see (and thanks for explaining) that using a registered mem region allows the use of some optimizations (the only one I know of right now is the PMD one you mentioned but maybe there's more I'm missing) that could be useful for some workloads, but I don't think (and this could just be my lack of understanding of what more optimizations there are) most use cases of kmbufs benefit from those optimizations, so to me it feels like we're adding non-trivial complexity for no noticeable benefit.

There are two separate arguments. The first is about not making buffers inseparable from buffer rings in the io_uring user API. Whether it's IORING_REGISTER_MEM_REGION or something else is not that important. I have no objection if it's a part of fuse instead though, e.g. if fuse binds two objects together when you register it with fuse, or even if fuse create a buffer ring internally (assuming it doesn't indirectly leak into io_uring uapi).

And the second was about optionally allowing user memory for buffer creation as you're reusing the region abstraction. You can find pros and cons for both modes, and funnily enough, SQ/CQ were first kernel allocated and then people asked for backing it by user memory, and IIRC it was in the reverse order for pbuf rings.

Implementing this is trivial as well, you just need to pass an argument while creating a region. All new region users use struct io_uring_region_desc for uapi and forward it to io_create_region() without caring if it's user or kernel allocated memory.

Quoted 10 lines

I feel like we get the best of both worlds by letting users have both:
the simple kernel-managed pbuf where the kernel allocates the buffers and the buffers are tied to the lifecycle of the ring, and the more advanced kernel-managed pbuf where buffers are tied to a registered memory region that the subsystem is responsible for later populating the ring with.

Quoted 1 line

and via a new io_uring uapi.

imo it felt cleaner to have a new uapi for it because kmbufs and pbufs

The stress is on why it's an _io_uring_ API. It doesn't matter to me whether it's a separate opcode or not. Currently, buffer rings don't give you anything that can't be pure fuse, and it might be simpler to have it implemented in fuse than binding to some io_uring object. Or it could create buffer rings internally to reuse code but it doesn't become an io_uring uapi but rather implementation detail. And that predicates on whether km rings are intended to be used with other / non-fuse requests.

Quoted 8 lines

have different expectations and behaviors (eg pbufs only work with user-provided buffers and requires userspace to populate the ring before using it, whereas for kmbufs the kernel allocates the buffers and populates it for you; pbufs require userspace to recycle back the buffer, whereas for kmbufs the kernel is the one in control of recycling) and from the user pov it seemed confusing to have kmbufs as part of the pbuf ring uapi, instead of separating it out as a different type of ringbuffer with a different expectation and

Quoted 6 lines

I believe the source of disagreement is that you're thinking about how it's going to look like for fuse specifically, and I believe you that it'll be nicer for the fuse use case. However, on the other hand it's an io_uring uapi, and if it is an io_uring uapi, we need reusable blocks that are not specific to particular users.

I agree 100%. The api we add should be what's best for io-uring, not fuse.

For the majority of use cases, it seemed to me that having the buffers separated from the buffer rings didn't yield perceptible benefits but added complexity and more restrictions like having to statically know up front how big the mem region needs to be across the lifetime of the io-uring for anything the io-uring might use the mem region for. It seems more generically useful as a concept to have the buffers owned by the ring and tied to the lifetime of the ring. I like how with this design everything is self-contained and multiple subsystems can use it without having to reimplement functionality locally in the subsystem. On the other hand, I see your point about how it might be something users want in the future if they want complete control over which parts of the mem region get used as the backing buffers to do stuff like PMD optimizations.

I think this is a matter of opinion/preference and I think in general for anything io-uring related, yours should take precedence.

With it going through a mem region, I don't think it should even go through the "pbuf ring" interface then if it's not going to specify the number of entries and buffer sizes upfront, if support is added for io-uring normal requests (eg IORING_OP_READ/WRITE) to use the backing pages from a memory region and if we're able to guarantee that the registered memory region will never be able to be unregistered by the user. I think if we repurpose the

union {
__u64 addr; /* pointer to buffer or iovecs */
__u64 splice_off_in;
};

fields in the struct io_uring_sqe to

union {
__u64 addr; /* pointer to buffer or iovecs */ __u64 splice_off_in;
__u64 offset; /* offset into registered mem region */
};

and add some IOSQE_ flag to indicate it should find the pages from the registered mem region, then that should work for normal requests. Where on the kernel side, it looks up the associated pages stored in the io_mapped_region's pages array for the offset passed in.

Right now there's only a uapi to register a memory region and none to unregister one. Is it guaranteed that io-uring will never add something in the future that will let userspace unregister the memory region or at least unregister it while it's being used (eg if we add future refcounting to it to track active uses of it)?

If so, then end-to-end, with it going through the mem region, it would be something like:
* user creates a mem region for the io-uring
* user mmaps the mem region
* user passes in offset into region, length of each buffer, and number of entries in the ring to the subsystem
* subsystem creates a locally managed bufring and adds buffers to that ring from the mem region
* on the cqe side, it sends the buffer id of the registered mem region through the same "IORING_CQE_F_BUFFER | (buf_id << IORING_CQE_BUFFER_SHIFT)" mechanism

Does this design match what you had in mind / prefer?

I think the above works for Christoph's use case too (as his and my use case are the same) but if not, please let me know.

Thanks,
Joanne

#35 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 16 lines · Joanne Koong
Quoted 15 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Fri, Feb 13, 2026 at 4:41 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 11 lines
Quoted 10 lines

Fuse is doing both adding (kernel) buffers to the ring and consuming them. At which point it's not clear:

1. Why it even needs io_uring provided buffer rings, it can be all contained in fuse. Maybe it's trying to reuse pbuf ring code as basically an internal memory allocator, but then why expose buffer rings as an io_uring uapi instead of keeping it internally.

That's also why I mentioned whether those buffers are supposed to be used with other types of io_uring requests like recv, etc.

Quoted 7 lines
Quoted 4 lines
Quoted 3 lines

On the userspace/server side, it uses the buffers for other io-uring operations (eg reading or writing the contents from/to a locally-backed file).

Sorry, I submitted v2 last night thinking the conversation on this thread had died. After reading through your reply, I'll modify v2.

No worries at all, and sorry I'm a bit slow to reply

Quoted 10 lines
Quoted 4 lines

Oops, typo. I was asking whether the buffer rings (not buffers) are supposed to be used with other requests. E.g. submitting a IORING_OP_RECV with IOSQE_BUFFER_SELECT set and the bgid specifying your kernel-managed buffer ring.

Yes the buffer rings are intended to be used with other io-uring requests. The ideal scenario is that the user can then do the equivalent of IORING_OP_READ/WRITE_FIXED operations on the kernel-managed buffers and avoid the per-i/o page pinning overhead costs.

You mention OP_READ_FIXED and below agreed not exposing km rings an io_uring uapi, which makes me believe we're still talking about different things.

Correct me if I'm wrong. Currently, only fuse cmds use the buffer ring itself, I'm not talking about buffer, i.e. fuse cmds consume entries from the ring (!!! that's the part I'm interested in), then process them and tell the server "this offset in the region has user data to process or should be populated with data".

Naturally, the server should be able to use the buffers to issue some I/O and process it in other ways, whether it's a normal OP_READ to which you pass the user space address (you can since it's mmap()'ed by the server) or something else is important but a separate question than the one I'm trying to understand.

So I'm asking whether you expect that a server or other user space program should be able to issue a READ_OP_RECV, READ_OP_READ or any other similar request, which would consume buffers/entries from the km ring without any fuse kernel code involved? Do you have some use case for that in mind?

Understanding that is the key in deciding whether km rings should be exposed as io_uring uapi or not, regardless of where buffers to populate the ring come from.

...

Quoted 21 lines

With it going through a mem region, I don't think it should even go through the "pbuf ring" interface then if it's not going to specify the number of entries and buffer sizes upfront, if support is added for io-uring normal requests (eg IORING_OP_READ/WRITE) to use the backing pages from a memory region and if we're able to guarantee that the registered memory region will never be able to be unregistered by the user. I think if we repurpose the

union {
__u64 addr; /* pointer to buffer or iovecs */
__u64 splice_off_in;
};

fields in the struct io_uring_sqe to

union {
__u64 addr; /* pointer to buffer or iovecs */ __u64 splice_off_in;
__u64 offset; /* offset into registered mem region */
};

Quoted 4 lines

and add some IOSQE_ flag to indicate it should find the pages from the registered mem region, then that should work for normal requests. Where on the kernel side, it looks up the associated pages stored in the io_mapped_region's pages array for the offset passed in.

So you already can do all that using the mmap()'ed region user pointer, and you just want it to be more efficient, right? For that let's just reuse registered buffers, we don't need a new mechanism that needs to be propagated to all request types. And registered buffer are already optimised for I/O in a bunch of ways. And as a bonus, it'll be similar to the zero-copy internally registered buffers if you still plan to add them.

The simplest way to do that is to create a registered buffer out of the mmap'ed region pointer. Pseudo code:

// mmap'ed if it's kernel allocated.
{region_ptr, region_size} = create_region();

struct iovec iov;
iov.iov_base = region_ptr;
iov.iov_len = region_size;
io_uring_register_buffers(ring, &iov, 1);

// later instead of this:
ptr = region_ptr + off;
io_uring_prep_read(sqe, fd, ptr, ...);

// you use registered buffers as usual:
io_uring_prep_read_fixed(sqe, fd, off, regbuf_idx, ...);

IIRC the registration would fail because it doesn't allow file backed pages, but it should be fine if we know it's io_uring region memory, so that would need to be patched.

There might be a bunch of other ways you can do that like create a kernel allocated registered buffer like what Cristoph wants, and then register it as a region. Or allow creating registered buffers out of a region. etc.

I wanted to unify registered buffers and regions internally at some point, but then drifted away from active io_uring core infrastructure development, so I guess that could've been useful.

Quoted 5 lines

Right now there's only a uapi to register a memory region and none to unregister one. Is it guaranteed that io-uring will never add something in the future that will let userspace unregister the memory region or at least unregister it while it's being used (eg if we add future refcounting to it to track active uses of it)?

Let's talk about it when it's needed or something changes, but if you do registered buffers instead as per above, they'll be holding page references and or have to pin the region in some other way.

Quoted 4 lines

If so, then end-to-end, with it going through the mem region, it would be something like:
* user creates a mem region for the io-uring
* user mmaps the mem region

FWIW, we should just add a liburing helper, so that fuse server doesn't need to deal with mmap'ing.

Quoted 4 lines

* user passes in offset into region, length of each buffer, and number of entries in the ring to the subsystem
* subsystem creates a locally managed bufring and adds buffers to that ring from the mem region

That's sounds clean to me _if_ it allows you to achieve all (fast path) optimisations you want to have. I hope it does?

Signature · 1 line
Pavel Begunkov
#36 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 55 lines · Pavel Begunkov
Quoted 16 lines · Joanne Koong
Quoted 15 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Fri, Feb 13, 2026 at 4:41 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 11 lines
Quoted 10 lines

Fuse is doing both adding (kernel) buffers to the ring and consuming them. At which point it's not clear:

1. Why it even needs io_uring provided buffer rings, it can be all contained in fuse. Maybe it's trying to reuse pbuf ring code as basically an internal memory allocator, but then why expose buffer rings as an io_uring uapi instead of keeping it internally.

That's also why I mentioned whether those buffers are supposed to be used with other types of io_uring requests like recv, etc.

Quoted 7 lines
Quoted 4 lines
Quoted 3 lines

On the userspace/server side, it uses the buffers for other io-uring operations (eg reading or writing the contents from/to a locally-backed file).

Sorry, I submitted v2 last night thinking the conversation on this thread had died. After reading through your reply, I'll modify v2.

No worries at all, and sorry I'm a bit slow to reply

Quoted 10 lines
Quoted 4 lines

Oops, typo. I was asking whether the buffer rings (not buffers) are supposed to be used with other requests. E.g. submitting a IORING_OP_RECV with IOSQE_BUFFER_SELECT set and the bgid specifying your kernel-managed buffer ring.

Yes the buffer rings are intended to be used with other io-uring requests. The ideal scenario is that the user can then do the equivalent of IORING_OP_READ/WRITE_FIXED operations on the kernel-managed buffers and avoid the per-i/o page pinning overhead costs.

You mention OP_READ_FIXED and below agreed not exposing km rings an io_uring uapi, which makes me believe we're still talking about different things.

Correct me if I'm wrong. Currently, only fuse cmds use the buffer ring itself, I'm not talking about buffer, i.e. fuse cmds consume entries from the ring (!!! that's the part I'm interested in), then process them and tell the server "this offset in the region has user data to process or should be populated with data".

Naturally, the server should be able to use the buffers to issue some I/O and process it in other ways, whether it's a normal OP_READ to which you pass the user space address (you can since it's mmap()'ed by the server) or something else is important but a separate question than the one I'm trying to understand.

Quoted 5 lines

So I'm asking whether you expect that a server or other user space program should be able to issue a READ_OP_RECV, READ_OP_READ or any other similar request, which would consume buffers/entries from the km ring without any fuse kernel code involved? Do you have some use case for that in mind?

Thanks for clarifying your question. Yes, this would be a useful optimization in the future for fuse servers with certain workload characteristics (eg network-backed servers with high concurrency and unpredictable latencies). I don't think the concept of kmbufrings is exclusively fuse-specific though (for example, Christoph's use case being a recent instance); I think other subsystems/users that'll use kmbuf rings would also generically find it useful to have the option of READ_OP_RECV/READ_OP_READ operating directly on the ring.

Quoted 51 lines

Understanding that is the key in deciding whether km rings should be exposed as io_uring uapi or not, regardless of where buffers to populate the ring come from.

...

Quoted 21 lines

With it going through a mem region, I don't think it should even go through the "pbuf ring" interface then if it's not going to specify the number of entries and buffer sizes upfront, if support is added for io-uring normal requests (eg IORING_OP_READ/WRITE) to use the backing pages from a memory region and if we're able to guarantee that the registered memory region will never be able to be unregistered by the user. I think if we repurpose the

union {
__u64 addr; /* pointer to buffer or iovecs */
__u64 splice_off_in;
};

fields in the struct io_uring_sqe to

union {
__u64 addr; /* pointer to buffer or iovecs */ __u64 splice_off_in;
__u64 offset; /* offset into registered mem region */
};

Quoted 4 lines

and add some IOSQE_ flag to indicate it should find the pages from the registered mem region, then that should work for normal requests. Where on the kernel side, it looks up the associated pages stored in the io_mapped_region's pages array for the offset passed in.

So you already can do all that using the mmap()'ed region user pointer, and you just want it to be more efficient, right? For that let's just reuse registered buffers, we don't need a new mechanism that needs to be propagated to all request types. And registered buffer are already optimised for I/O in a bunch of ways. And as a bonus, it'll be similar to the zero-copy internally registered buffers if you still plan to add them.

The simplest way to do that is to create a registered buffer out of the mmap'ed region pointer. Pseudo code:

// mmap'ed if it's kernel allocated.
{region_ptr, region_size} = create_region();

struct iovec iov;
iov.iov_base = region_ptr;
iov.iov_len = region_size;
io_uring_register_buffers(ring, &iov, 1);

Quoted 7 lines

// later instead of this:
ptr = region_ptr + off;
io_uring_prep_read(sqe, fd, ptr, ...);

// you use registered buffers as usual:
io_uring_prep_read_fixed(sqe, fd, off, regbuf_idx, ...);

I feel like this design makes the interface more convoluted and now muddies different concepts together by adding new complexity / relationships between them whereas they were otherwise cleanly isolated. Maybe I'm just not seeing/understanding the overarching vision for why conceptually it makes sense for them to be tied together besides as a mechanism to tell io-uring requests where to copy from by reusing what exists for fixed buffer ids. There's more complexity now on the kernel side (eg having to detect if the buffer passed in is kernel-allocated to know whether to pin the pages / charge it against the user's RLIMIT_MEMLOCK limit) but I'm not understanding what we gain from it. I got the sense from your previous comments that memory regions are the de facto way to go and should be decoupled from other structures, so if that's the case, why doesn't it make sense for io-uring to add native support for using memory regions for io-uring requests? I feel like from the userspace side it makes things more confusing with this extra layer of indirection that now has to go through a fixed buffer.

Quoted 20 lines

IIRC the registration would fail because it doesn't allow file backed pages, but it should be fine if we know it's io_uring region memory, so that would need to be patched.

There might be a bunch of other ways you can do that like create a kernel allocated registered buffer like what Cristoph wants, and then register it as a region. Or allow creating registered buffers out of a region. etc.

I wanted to unify registered buffers and regions internally at some point, but then drifted away from active io_uring core infrastructure development, so I guess that could've been useful.

Quoted 5 lines

Right now there's only a uapi to register a memory region and none to unregister one. Is it guaranteed that io-uring will never add something in the future that will let userspace unregister the memory region or at least unregister it while it's being used (eg if we add future refcounting to it to track active uses of it)?

Quoted 3 lines

Let's talk about it when it's needed or something changes, but if you do registered buffers instead as per above, they'll be holding page references and or have to pin the region in some other way.

I don't think we can guarantee that the caller will register the memory region as a fixed buffer (eg if it doesn't need/want to use the buffer for normal io-uring requests). On the kernel side, the internal buffer entry uses the kaddr of the registered memory region buffer for any memcpys. If it's not guaranteed that registered memory regions persist for the lifetime of the ring, there'll have to be extra overhead for every I/O (eg grab the io-uring lock, checking if the mem region is still registered, grab a refcount to that mem region, unlock the ring, do the memcpy to the kaddr, then grab the io-uring lock again, decrement the refcount, and unlock). Or I guess we could add pinning to a registered memory region.

Thanks,
Joanne

Quoted 20 lines
Quoted 5 lines

If so, then end-to-end, with it going through the mem region, it would be something like:
* user creates a mem region for the io-uring
* user mmaps the mem region

FWIW, we should just add a liburing helper, so that fuse server doesn't need to deal with mmap'ing.

Quoted 4 lines

* user passes in offset into region, length of each buffer, and number of entries in the ring to the subsystem
* subsystem creates a locally managed bufring and adds buffers to that ring from the mem region

That's sounds clean to me _if_ it allows you to achieve all (fast path) optimisations you want to have. I hope it does?

--
Pavel Begunkov

#37 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 1 line · Joanne Koong

On Fri, Feb 20, 2026 at 4:53 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 6 lines
Quoted 5 lines

So I'm asking whether you expect that a server or other user space program should be able to issue a READ_OP_RECV, READ_OP_READ or any other similar request, which would consume buffers/entries from the km ring without any fuse kernel code involved? Do you have some use case for that in mind?

Quoted 6 lines

Thanks for clarifying your question. Yes, this would be a useful optimization in the future for fuse servers with certain workload characteristics (eg network-backed servers with high concurrency and unpredictable latencies). I don't think the concept of kmbufrings is exclusively fuse-specific though (for example, Christoph's use case being a recent instance);

Sorry, I don't see relevance b/w km rings and what Christoph wants. I explained why in some sub-thread, but maybe someone can tell what I'm missing.

Quoted 3 lines

I think other subsystems/users that'll use
kmbuf rings would also generically find it useful to have the option of READ_OP_RECV/READ_OP_READ operating directly on the ring.

Yep, it could be, potentially, it's just the patchset doesn't plumb it to other requests and uses it within fuse. It's just cases like that always make me wonder, here it was why what is basically an internal kernel fuse API is exposed as an io_uring uapi. Maybe there was a discussion about it I missed?

Quoted 27 lines
Quoted 19 lines

So you already can do all that using the mmap()'ed region user pointer, and you just want it to be more efficient, right? For that let's just reuse registered buffers, we don't need a new mechanism that needs to be propagated to all request types. And registered buffer are already optimised for I/O in a bunch of ways. And as a bonus, it'll be similar to the zero-copy internally registered buffers if you still plan to add them.

The simplest way to do that is to create a registered buffer out of the mmap'ed region pointer. Pseudo code:

// mmap'ed if it's kernel allocated.
{region_ptr, region_size} = create_region();

struct iovec iov;
iov.iov_base = region_ptr;
iov.iov_len = region_size;
io_uring_register_buffers(ring, &iov, 1);

Quoted 7 lines

// later instead of this:
ptr = region_ptr + off;
io_uring_prep_read(sqe, fd, ptr, ...);

// you use registered buffers as usual:
io_uring_prep_read_fixed(sqe, fd, off, regbuf_idx, ...);

Quoted 11 lines

I feel like this design makes the interface more convoluted and now muddies different concepts together by adding new complexity / relationships between them whereas they were otherwise cleanly isolated. Maybe I'm just not seeing/understanding the overarching vision for why conceptually it makes sense for them to be tied together besides as a mechanism to tell io-uring requests where to copy from by reusing what exists for fixed buffer ids. There's more complexity now on the kernel side (eg having to detect if the buffer passed in is kernel-allocated to know whether to pin the pages / charge it against the user's RLIMIT_MEMLOCK limit) but I'm not understanding what we gain from it.

That would avoid doing a large revamp of uapi and plumbing it to each every request type when there is already a uapi that does what you want, does it well and have lots of things figured out. Keeping the I/O path sane is important, io_uring already has 3 different ways of passing buffers, let's not add a 4th one unless it achieves something meaningful.

Quoted 2 lines

I got the sense from your previous
comments that memory regions are the de facto way to go and should be

Sorry, maybe I wasn't clear. With what I see you're trying to do, i.e. copying client's data into user space (server), I think registered buffers would be a better abstraction. However, I just went with your design on top of regions, since it's not the first iteration of the series and I wasn't following previous ones, and IIRC you was already using registered buffers in previous revisions but moved from that for some reason. IOW, I was taking you main I/O path and was trying to make the setup path a bit more flexible and reusable.

Quoted 5 lines

decoupled from other structures, so if that's the case, why doesn't it make sense for io-uring to add native support for using memory regions for io-uring requests? I feel like from the userspace side it makes things more confusing with this extra layer of indirection that now has to go through a fixed buffer.

There is a high bar for adding a new interface for passing buffers that needs to be propagated to a good number of request handlers, and there is already one that gives you all you need to write efficient user space.

Quoted 19 lines
Quoted 19 lines

IIRC the registration would fail because it doesn't allow file backed pages, but it should be fine if we know it's io_uring region memory, so that would need to be patched.

There might be a bunch of other ways you can do that like create a kernel allocated registered buffer like what Cristoph wants, and then register it as a region. Or allow creating registered buffers out of a region. etc.

I wanted to unify registered buffers and regions internally at some point, but then drifted away from active io_uring core infrastructure development, so I guess that could've been useful.

Quoted 5 lines

Right now there's only a uapi to register a memory region and none to unregister one. Is it guaranteed that io-uring will never add something in the future that will let userspace unregister the memory region or at least unregister it while it's being used (eg if we add future refcounting to it to track active uses of it)?

Quoted 7 lines
Quoted 3 lines

Let's talk about it when it's needed or something changes, but if you do registered buffers instead as per above, they'll be holding page references and or have to pin the region in some other way.

I don't think we can guarantee that the caller will register the memory region as a fixed buffer (eg if it doesn't need/want to use the buffer for normal io-uring requests). On the kernel side, the internal

It's up to the user (i.e. fuse server) to either use OP_READ/etc. using user addresses that you have in your design from mmap()ing regions, or registering it and using OP_READ_FIXED.

Quoted 8 lines

buffer entry uses the kaddr of the registered memory region buffer for any memcpys. If it's not guaranteed that registered memory regions persist for the lifetime of the ring, there'll have to be extra overhead for every I/O (eg grab the io-uring lock, checking if the mem region is still registered, grab a refcount to that mem region, unlock the ring, do the memcpy to the kaddr, then grab the io-uring lock again, decrement the refcount, and unlock). Or I guess we could add pinning to a registered memory region.

Signature · 1 line
Pavel Begunkov
#38 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 21 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Fri, Feb 20, 2026 at 4:53 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 6 lines
Quoted 5 lines

So I'm asking whether you expect that a server or other user space program should be able to issue a READ_OP_RECV, READ_OP_READ or any other similar request, which would consume buffers/entries from the km ring without any fuse kernel code involved? Do you have some use case for that in mind?

Quoted 6 lines

Thanks for clarifying your question. Yes, this would be a useful optimization in the future for fuse servers with certain workload characteristics (eg network-backed servers with high concurrency and unpredictable latencies). I don't think the concept of kmbufrings is exclusively fuse-specific though (for example, Christoph's use case being a recent instance);

Sorry, I don't see relevance b/w km rings and what Christoph wants. I explained why in some sub-thread, but maybe someone can tell what I'm missing.

Quoted 6 lines
Quoted 3 lines

I think other subsystems/users that'll use
kmbuf rings would also generically find it useful to have the option of READ_OP_RECV/READ_OP_READ operating directly on the ring.

Yep, it could be, potentially, it's just the patchset doesn't plumb it to other requests and uses it within fuse. It's just cases like

This patchset just represents the most basic foundation. The optimization patches (eg incremental buffer consumption, plumbing it to other io-uring requests, etc) were to be follow-up patchsets that would be on top of this.

Quoted 2 lines

that always make me wonder, here it was why what is basically an internal kernel fuse API is exposed as an io_uring uapi. Maybe there

It's not really an internal kernel fuse API. There's nothing fuse-specific about it - the infrastructure that's added is the infrastructure for a generic buffer ring.

The memory that backs the buffers for the buf ring needs to be io-uring specific. io-uring already has all the infrastructure for buffer rings. So I'm not really fully understanding why it's better in this case to just have the fuse kernel code re-implement all the logic for a buffer ring and go through these layers of indirection to use registered buffers, instead of just leveraging what's already in io-uring.

Quoted 95 lines

was a discussion about it I missed?

Quoted 27 lines
Quoted 19 lines

So you already can do all that using the mmap()'ed region user pointer, and you just want it to be more efficient, right? For that let's just reuse registered buffers, we don't need a new mechanism that needs to be propagated to all request types. And registered buffer are already optimised for I/O in a bunch of ways. And as a bonus, it'll be similar to the zero-copy internally registered buffers if you still plan to add them.

The simplest way to do that is to create a registered buffer out of the mmap'ed region pointer. Pseudo code:

// mmap'ed if it's kernel allocated.
{region_ptr, region_size} = create_region();

struct iovec iov;
iov.iov_base = region_ptr;
iov.iov_len = region_size;
io_uring_register_buffers(ring, &iov, 1);

Quoted 7 lines

// later instead of this:
ptr = region_ptr + off;
io_uring_prep_read(sqe, fd, ptr, ...);

// you use registered buffers as usual:
io_uring_prep_read_fixed(sqe, fd, off, regbuf_idx, ...);

Quoted 11 lines

I feel like this design makes the interface more convoluted and now muddies different concepts together by adding new complexity / relationships between them whereas they were otherwise cleanly isolated. Maybe I'm just not seeing/understanding the overarching vision for why conceptually it makes sense for them to be tied together besides as a mechanism to tell io-uring requests where to copy from by reusing what exists for fixed buffer ids. There's more complexity now on the kernel side (eg having to detect if the buffer passed in is kernel-allocated to know whether to pin the pages / charge it against the user's RLIMIT_MEMLOCK limit) but I'm not understanding what we gain from it.

That would avoid doing a large revamp of uapi and plumbing it to each every request type when there is already a uapi that does what you want, does it well and have lots of things figured out. Keeping the I/O path sane is important, io_uring already has 3 different ways of passing buffers, let's not add a 4th one unless it achieves something meaningful.

Quoted 2 lines

I got the sense from your previous
comments that memory regions are the de facto way to go and should be

Sorry, maybe I wasn't clear. With what I see you're trying to do, i.e. copying client's data into user space (server), I think registered buffers would be a better abstraction. However, I just went with your design on top of regions, since it's not the first iteration of the series and I wasn't following previous ones, and IIRC you was already using registered buffers in previous revisions but moved from that for some reason. IOW, I was taking you main I/O path and was trying to make the setup path a bit more flexible and reusable.

Quoted 5 lines

decoupled from other structures, so if that's the case, why doesn't it make sense for io-uring to add native support for using memory regions for io-uring requests? I feel like from the userspace side it makes things more confusing with this extra layer of indirection that now has to go through a fixed buffer.

There is a high bar for adding a new interface for passing buffers that needs to be propagated to a good number of request handlers, and there is already one that gives you all you need to write efficient user space.

Quoted 23 lines
Quoted 22 lines

IIRC the registration would fail because it doesn't allow file backed pages, but it should be fine if we know it's io_uring region memory, so that would need to be patched.

There might be a bunch of other ways you can do that like create a kernel allocated registered buffer like what Cristoph wants, and then register it as a region. Or allow creating registered buffers out of a region. etc.

I wanted to unify registered buffers and regions internally at some point, but then drifted away from active io_uring core infrastructure development, so I guess that could've been useful.

Quoted 5 lines

Right now there's only a uapi to register a memory region and none to unregister one. Is it guaranteed that io-uring will never add something in the future that will let userspace unregister the memory region or at least unregister it while it's being used (eg if we add future refcounting to it to track active uses of it)?

Let's talk about it when it's needed or something changes, but if you do registered buffers instead as per above, they'll be holding page references and or have to pin the region in some other way.

Quoted 7 lines
Quoted 3 lines

I don't think we can guarantee that the caller will register the memory region as a fixed buffer (eg if it doesn't need/want to use the buffer for normal io-uring requests). On the kernel side, the internal

It's up to the user (i.e. fuse server) to either use OP_READ/etc. using user addresses that you have in your design from mmap()ing regions, or registering it and using OP_READ_FIXED.

Yes but I don't think this solves the concern of userspace being able to unregister the memory region at any time (eg while not doing io-uring requests) while the kernel still points to those addresses for the backing buffers of the bufring, since there's no callback that gets triggered in the subsystem when a memory region is unregistered, which means there will need to be extra per I/O overhead for having to ensure the memory region is still valid. Though since there's no uapi for unregistering a memory region this is not a concern, unless this is planned to be added in the future.

Thanks,
Joanne

Quoted 15 lines
Quoted 9 lines

buffer entry uses the kaddr of the registered memory region buffer for any memcpys. If it's not guaranteed that registered memory regions persist for the lifetime of the ring, there'll have to be extra overhead for every I/O (eg grab the io-uring lock, checking if the mem region is still registered, grab a refcount to that mem region, unlock the ring, do the memcpy to the kaddr, then grab the io-uring lock again, decrement the refcount, and unlock). Or I guess we could add pinning to a registered memory region.

--
Pavel Begunkov

#39 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 26 lines · Joanne Koong
Quoted 25 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Fri, Feb 20, 2026 at 4:53 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 6 lines
Quoted 5 lines

So I'm asking whether you expect that a server or other user space program should be able to issue a READ_OP_RECV, READ_OP_READ or any other similar request, which would consume buffers/entries from the km ring without any fuse kernel code involved? Do you have some use case for that in mind?

Quoted 6 lines

Thanks for clarifying your question. Yes, this would be a useful optimization in the future for fuse servers with certain workload characteristics (eg network-backed servers with high concurrency and unpredictable latencies). I don't think the concept of kmbufrings is exclusively fuse-specific though (for example, Christoph's use case being a recent instance);

Sorry, I don't see relevance b/w km rings and what Christoph wants. I explained why in some sub-thread, but maybe someone can tell what I'm missing.

Quoted 3 lines

I think other subsystems/users that'll use
kmbuf rings would also generically find it useful to have the option of READ_OP_RECV/READ_OP_READ operating directly on the ring.

Quoted 7 lines
Quoted 2 lines

Yep, it could be, potentially, it's just the patchset doesn't plumb it to other requests and uses it within fuse. It's just cases like

This patchset just represents the most basic foundation. The optimization patches (eg incremental buffer consumption, plumbing it to other io-uring requests, etc) were to be follow-up patchsets that would be on top of this.

Got it. Any understanding how the work flow would look like if used with non-cmd io_uring requests? Is there some particular use case you have in mind for fuse servers?

Quoted 7 lines
Quoted 2 lines

that always make me wonder, here it was why what is basically an internal kernel fuse API is exposed as an io_uring uapi. Maybe there

It's not really an internal kernel fuse API. There's nothing fuse-specific about it - the infrastructure that's added is the infrastructure for a generic buffer ring.

Quoted 7 lines

The memory that backs the buffers for the buf ring needs to be io-uring specific. io-uring already has all the infrastructure for buffer rings. So I'm not really fully understanding why it's better in this case to just have the fuse kernel code re-implement all the logic for a buffer ring and go through these layers of indirection to use registered buffers, instead of just leveraging what's already in io-uring.

It's simple. If user space (i.e. fuse server) knows the buffer address prior to request submission, then it should either use plain user addresses or registered buffers. Introducing a major io_uring uapi extension that does the same thing as registered buffers but for regions, as you suggested, is not the right approach.

...

Quoted 8 lines
Quoted 3 lines

It's up to the user (i.e. fuse server) to either use OP_READ/etc. using user addresses that you have in your design from mmap()ing regions, or registering it and using OP_READ_FIXED.

Yes but I don't think this solves the concern of userspace being able to unregister the memory region at any time (eg while not doing io-uring requests) while the kernel still points to those addresses for the backing buffers of the bufring, since there's no callback that

If you allow normal requests to use it, a fuse callback on region unregistration wouldn't help anyway.

Quoted 5 lines

gets triggered in the subsystem when a memory region is unregistered, which means there will need to be extra per I/O overhead for having to ensure the memory region is still valid. Though since there's no uapi for unregistering a memory region this is not a concern, unless this is planned to be added in the future.

You can assume it's not going to be unregistered.

Signature · 1 line
Pavel Begunkov
#40 lore raw

Re: [PATCH v1 03/11] io_uring/kbuf: add support for kernel-managed buffer rings

Quoted 31 lines · Pavel Begunkov
Quoted 29 lines · Joanne Koong
Quoted 27 lines · Pavel Begunkov
Quoted 1 line · Joanne Koong

On Fri, Feb 20, 2026 at 4:53 AM Pavel Begunkov <[email protected]> wrote:

...

Quoted 6 lines
Quoted 5 lines

So I'm asking whether you expect that a server or other user space program should be able to issue a READ_OP_RECV, READ_OP_READ or any other similar request, which would consume buffers/entries from the km ring without any fuse kernel code involved? Do you have some use case for that in mind?

Quoted 6 lines

Thanks for clarifying your question. Yes, this would be a useful optimization in the future for fuse servers with certain workload characteristics (eg network-backed servers with high concurrency and unpredictable latencies). I don't think the concept of kmbufrings is exclusively fuse-specific though (for example, Christoph's use case being a recent instance);

Sorry, I don't see relevance b/w km rings and what Christoph wants. I explained why in some sub-thread, but maybe someone can tell what I'm missing.

Quoted 3 lines

I think other subsystems/users that'll use
kmbuf rings would also generically find it useful to have the option of READ_OP_RECV/READ_OP_READ operating directly on the ring.

Yep, it could be, potentially, it's just the patchset doesn't plumb it to other requests and uses it within fuse. It's just cases like

Quoted 8 lines
Quoted 4 lines

This patchset just represents the most basic foundation. The optimization patches (eg incremental buffer consumption, plumbing it to other io-uring requests, etc) were to be follow-up patchsets that would be on top of this.

Got it. Any understanding how the work flow would look like if used with non-cmd io_uring requests? Is there some particular use case you have in mind for fuse servers?

One use case is for network-backed servers with high concurrency and unpredictable latencies where with kmbuf rings we can submit multipel recvs and have the buffer assignment be done as the completion comes in rather than having to preassign the buffers before submission when we don't know which requests will complete first.

Thanks,
Joanne

#41 lore raw

PATCH 4/11 io_uring/kbuf: add mmap support for kernel-managed buffer rings

In reply to Joanne Koong

Add support for mmapping kernel-managed buffer rings (kmbuf) to userspace, allowing applications to access the kernel-allocated buffers.

Similar to application-provided buffer rings (pbuf), kmbuf rings use the buffer group ID encoded in the mmap offset to identify which buffer ring to map. The implementation follows the same pattern as pbuf rings.

New mmap offset constants are introduced:
- IORING_OFF_KMBUF_RING (0x88000000): Base offset for kmbuf mappings
- IORING_OFF_KMBUF_SHIFT (16): Shift value to encode buffer group ID

The mmap offset encodes the bgid shifted by IORING_OFF_KMBUF_SHIFT. The io_buf_get_region() helper retrieves the appropriate region.

This allows userspace to mmap the kernel-allocated buffer region and access the buffers directly.

---
 include/uapi/linux/io_uring.h |  2 ++
 io_uring/kbuf.c               | 11 +++++++++--
 io_uring/kbuf.h               |  5 +++--
 io_uring/memmap.c             |  5 ++++-
 4 files changed, 18 insertions(+), 5 deletions(-)
include/uapi/linux/io_uring.h +2 -0
index a0889c1744bd..42a2812c9922 100644
@@ -545,6 +545,8 @@ struct io_uring_cqe {
 #define IORING_OFF_SQES			0x10000000ULL
 #define IORING_OFF_PBUF_RING		0x80000000ULL
 #define IORING_OFF_PBUF_SHIFT		16
+#define IORING_OFF_KMBUF_RING		0x88000000ULL
+#define IORING_OFF_KMBUF_SHIFT		16
 #define IORING_OFF_MMAP_MASK		0xf8000000ULL
 
 /*
io_uring/kbuf.c +9 -2
index 9bc36451d083..ccf5b213087b 100644
@@ -770,16 +770,23 @@ int io_register_pbuf_status(struct io_ring_ctx *ctx, void __user *arg)
 	return 0;
 }
 
-struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx,
-					    unsigned int bgid)
+struct io_mapped_region *io_buf_get_region(struct io_ring_ctx *ctx,
+					   unsigned int bgid,
+					   bool kernel_managed)
 {
 	struct io_buffer_list *bl;
+	bool is_kernel_managed;
 
 	lockdep_assert_held(&ctx->mmap_lock);
 
 	bl = xa_load(&ctx->io_bl_xa, bgid);
 	if (!bl || !(bl->flags & IOBL_BUF_RING))
 		return NULL;
+
+	is_kernel_managed = !!(bl->flags & IOBL_KERNEL_MANAGED);
+	if (is_kernel_managed != kernel_managed)
+		return NULL;
+
 	return &bl->region;
 }
 
io_uring/kbuf.h +3 -2
index 62c80a1ebf03..11d165888b8e 100644
@@ -88,8 +88,9 @@ unsigned int __io_put_kbufs(struct io_kiocb *req, struct io_buffer_list *bl,
 bool io_kbuf_commit(struct io_kiocb *req,
 		    struct io_buffer_list *bl, int len, int nr);
 
-struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx,
-					    unsigned int bgid);
+struct io_mapped_region *io_buf_get_region(struct io_ring_ctx *ctx,
+					   unsigned int bgid,
+					   bool kernel_managed);
 
 static inline bool io_kbuf_recycle_ring(struct io_kiocb *req,
 					struct io_buffer_list *bl)
io_uring/memmap.c +4 -1
index 8d37e93c0433..916315122323 100644
@@ -356,7 +356,10 @@ static struct io_mapped_region *io_mmap_get_region(struct io_ring_ctx *ctx,
 		return &ctx->sq_region;
 	case IORING_OFF_PBUF_RING:
 		id = (offset & ~IORING_OFF_MMAP_MASK) >> IORING_OFF_PBUF_SHIFT;
-		return io_pbuf_get_region(ctx, id);
+		return io_buf_get_region(ctx, id, false);
+	case IORING_OFF_KMBUF_RING:
+		id = (offset & ~IORING_OFF_MMAP_MASK) >> IORING_OFF_KMBUF_SHIFT;
+		return io_buf_get_region(ctx, id, true);
 	case IORING_MAP_OFF_PARAM_REGION:
 		return &ctx->param_region;
 	case IORING_MAP_OFF_ZCRX_REGION:
Signature · 1 line
2.47.3
maintainer
#42 lore raw

Re: [PATCH v1 04/11] io_uring/kbuf: add mmap support for kernel-managed buffer rings

Quoted 57 lines · Joanne Koong

Add support for mmapping kernel-managed buffer rings (kmbuf) to userspace, allowing applications to access the kernel-allocated buffers.

Similar to application-provided buffer rings (pbuf), kmbuf rings use the buffer group ID encoded in the mmap offset to identify which buffer ring to map. The implementation follows the same pattern as pbuf rings.

New mmap offset constants are introduced:
- IORING_OFF_KMBUF_RING (0x88000000): Base offset for kmbuf mappings
- IORING_OFF_KMBUF_SHIFT (16): Shift value to encode buffer group ID

The mmap offset encodes the bgid shifted by IORING_OFF_KMBUF_SHIFT. The io_buf_get_region() helper retrieves the appropriate region.

This allows userspace to mmap the kernel-allocated buffer region and access the buffers directly.

---
 include/uapi/linux/io_uring.h |  2 ++
 io_uring/kbuf.c               | 11 +++++++++--
 io_uring/kbuf.h               |  5 +++--
 io_uring/memmap.c             |  5 ++++-
 4 files changed, 18 insertions(+), 5 deletions(-)
include/uapi/linux/io_uring.h +2 -0
index a0889c1744bd..42a2812c9922 100644
@@ -545,6 +545,8 @@ struct io_uring_cqe {
 #define IORING_OFF_SQES			0x10000000ULL
 #define IORING_OFF_PBUF_RING		0x80000000ULL
 #define IORING_OFF_PBUF_SHIFT		16
+#define IORING_OFF_KMBUF_RING		0x88000000ULL
+#define IORING_OFF_KMBUF_SHIFT		16
 #define IORING_OFF_MMAP_MASK		0xf8000000ULL
 
 /*
io_uring/kbuf.c +4 -2
index 9bc36451d083..ccf5b213087b 100644
@@ -770,16 +770,23 @@ int io_register_pbuf_status(struct io_ring_ctx *ctx, void __user *arg)
 	return 0;
 }
 
-struct io_mapped_region *io_pbuf_get_region(struct io_ring_ctx *ctx,
-					    unsigned int bgid)
+struct io_mapped_region *io_buf_get_region(struct io_ring_ctx *ctx,
+					   unsigned int bgid,
+					   bool kernel_managed)
 {
 	struct io_buffer_list *bl;
+	bool is_kernel_managed;
 
 	lockdep_assert_held(&ctx->mmap_lock);
 

io_uring/kbuf.c int io_register_pbuf_status(struct io_ring_ctx *ctx, void __user *arg) · line 791

Quoted 10 lines
io_uring/kbuf.c +5 -0
 	bl = xa_load(&ctx->io_bl_xa, bgid);
 	if (!bl || !(bl->flags & IOBL_BUF_RING))
 		return NULL;
+
+	is_kernel_managed = !!(bl->flags & IOBL_KERNEL_MANAGED);
+	if (is_kernel_managed != kernel_managed)
+		return NULL;
+
 	return &bl->region;
 }

For this, I think just add another helper - leave io_pbuf_get_region() and add a bl->flags & IOBL_KERNEL_MANAGED error check in there, and add a io_kbuf_get_region() or similar and have a !(bl->flags & IOBL_KERNEL_MANAGED) error check in that one.

That's easier to read, and there's little reason to avoid duplicating the xa_load() part.

Minor nit, but imho it's more readable that way.

Signature · 1 line
Jens Axboe
#43 lore raw

PATCH 5/11 io_uring/kbuf: support kernel-managed buffer rings in buffer selection

In reply to Joanne Koong

Allow kernel-managed buffers to be selected. This requires modifying the io_br_sel struct to separate the fields for address and val, since a kernel address cannot be distinguished from a negative val when error checking.

Auto-commit any selected kernel-managed buffer.

---
 include/linux/io_uring_types.h |  8 ++++----
 io_uring/kbuf.c                | 16 ++++++++++++----
 2 files changed, 16 insertions(+), 8 deletions(-)
include/linux/io_uring_types.h +4 -4
index 3e4a82a6f817..36cc2e0346d9 100644
@@ -93,13 +93,13 @@ struct io_mapped_region {
  */
 struct io_br_sel {
 	struct io_buffer_list *buf_list;
-	/*
-	 * Some selection parts return the user address, others return an error.
-	 */
 	union {
+		/* for classic/ring provided buffers */
 		void __user *addr;
-		ssize_t val;
+		/* for kernel-managed buffers */
+		void *kaddr;
 	};
+	ssize_t val;
 };
 
 
io_uring/kbuf.c +12 -4
index ccf5b213087b..1e8395270227 100644
@@ -155,7 +155,8 @@ static int io_provided_buffers_select(struct io_kiocb *req, size_t *len,
 	return 1;
 }
 
-static bool io_should_commit(struct io_kiocb *req, unsigned int issue_flags)
+static bool io_should_commit(struct io_kiocb *req, struct io_buffer_list *bl,
+			     unsigned int issue_flags)
 {
 	/*
 	* If we came in unlocked, we have no choice but to consume the
@@ -170,7 +171,11 @@ static bool io_should_commit(struct io_kiocb *req, unsigned int issue_flags)
 	if (issue_flags & IO_URING_F_UNLOCKED)
 		return true;
 
-	/* uring_cmd commits kbuf upfront, no need to auto-commit */
+	/* kernel-managed buffers are auto-committed */
+	if (bl->flags & IOBL_KERNEL_MANAGED)
+		return true;
+
+	/* multishot uring_cmd commits kbuf upfront, no need to auto-commit */
 	if (!io_file_can_poll(req) && req->opcode != IORING_OP_URING_CMD)
 		return true;
 	return false;
@@ -200,9 +205,12 @@ static struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 	req->flags |= REQ_F_BUFFER_RING | REQ_F_BUFFERS_COMMIT;
 	req->buf_index = READ_ONCE(buf->bid);
 	sel.buf_list = bl;
-	sel.addr = u64_to_user_ptr(READ_ONCE(buf->addr));
+	if (bl->flags & IOBL_KERNEL_MANAGED)
+		sel.kaddr = (void *)(uintptr_t)READ_ONCE(buf->addr);
+	else
+		sel.addr = u64_to_user_ptr(READ_ONCE(buf->addr));
 
-	if (io_should_commit(req, issue_flags)) {
+	if (io_should_commit(req, bl, issue_flags)) {
 		io_kbuf_commit(req, sel.buf_list, *len, 1);
 		sel.buf_list = NULL;
 	}
Signature · 1 line
2.47.3
#44 lore raw

PATCH 6/11 io_uring/kbuf: add buffer ring pinning/unpinning

In reply to Joanne Koong

Add kernel APIs to pin and unpin buffer rings, preventing userspace from unregistering a buffer ring while it is pinned by the kernel.

This provides a mechanism for kernel subsystems to safely access buffer ring contents while ensuring the buffer ring remains valid. A pinned buffer ring cannot be unregistered until explicitly unpinned. On the userspace side, trying to unregister a pinned buffer will return -EBUSY.

This is a preparatory change for upcoming fuse usage of kernel-managed buffer rings. It is necessary for fuse to pin the buffer ring because fuse may need to select a buffer in atomic contexts, which it can only do so by using the underlying buffer list pointer.

---
 include/linux/io_uring/cmd.h | 17 +++++++++++++
 io_uring/kbuf.c              | 48 ++++++++++++++++++++++++++++++++++++
 io_uring/kbuf.h              |  5 ++++
 3 files changed, 70 insertions(+)
include/linux/io_uring/cmd.h +17 -0
index 375fd048c4cb..702b1903e6ee 100644
@@ -84,6 +84,10 @@ struct io_br_sel io_uring_cmd_buffer_select(struct io_uring_cmd *ioucmd,
 bool io_uring_mshot_cmd_post_cqe(struct io_uring_cmd *ioucmd,
 				 struct io_br_sel *sel, unsigned int issue_flags);
 
+int io_uring_buf_ring_pin(struct io_uring_cmd *cmd, unsigned buf_group,
+			  unsigned issue_flags, struct io_buffer_list **bl);
+int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd, unsigned buf_group,
+			    unsigned issue_flags);
 #else
 static inline int
 io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
@@ -126,6 +130,19 @@ static inline bool io_uring_mshot_cmd_post_cqe(struct io_uring_cmd *ioucmd,
 {
 	return true;
 }
+static inline int io_uring_buf_ring_pin(struct io_uring_cmd *cmd,
+					unsigned buf_group,
+					unsigned issue_flags,
+					struct io_buffer_list **bl)
+{
+	return -EOPNOTSUPP;
+}
+static inline int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd,
+					  unsigned buf_group,
+					  unsigned issue_flags)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 
 static inline struct io_uring_cmd *io_uring_cmd_from_tw(struct io_tw_req tw_req)
io_uring/kbuf.c +48 -0

66 lines not on this page · raw message

io_uring/kbuf.h +5 -0
index 11d165888b8e..781630c2cc10 100644
@@ -12,6 +12,11 @@ enum {
 	IOBL_INC		= 2,
 	/* buffers are kernel managed */
 	IOBL_KERNEL_MANAGED	= 4,
+	/*
+	 * buffer ring is pinned and cannot be unregistered by userspace until
+	 * it has been unpinned
+	 */
+	IOBL_PINNED		= 8,
 };
 
 struct io_buffer_list {
Signature · 1 line
2.47.3
maintainer
#45 lore raw

Re: [PATCH v1 06/11] io_uring/kbuf: add buffer ring pinning/unpinning

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 246

Quoted 6 lines · Joanne Koong
io_uring/kbuf.c +6 -0
+int io_uring_buf_ring_pin(struct io_uring_cmd *cmd, unsigned buf_group,
+			  unsigned issue_flags, struct io_buffer_list **bl)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	struct io_buffer_list *buffer_list;
+	int ret = -EINVAL;

Probably use the usual struct io_buffer_list *bl here and either use an ERR_PTR return, or rename the passed on **bl to **blret or something.

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 276

Quoted 11 lines
io_uring/kbuf.c +11 -0
+int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd, unsigned buf_group,
+		       unsigned issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	struct io_buffer_list *bl;
+	int ret = -EINVAL;
+
+	io_ring_submit_lock(ctx, issue_flags);
+
+	bl = io_buffer_get_list(ctx, buf_group);
+	if (bl && (bl->flags & IOBL_BUF_RING) && (bl->flags & IOBL_PINNED)) {

Usually done as:

	if ((bl->flags & (IOBL_BUF_RING|IOBL_PINNED)) == (IOBL_BUF_RING|IOBL_PINNED))

and maybe then just have an earlier

	if (!bl)
		goto err;

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 279

Quoted 3 lines
io_uring/kbuf.c +3 -0
+		bl->flags &= ~IOBL_PINNED;
+		ret = 0;
+	}

err:

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 263

Quoted 3 lines
io_uring/kbuf.c +3 -0
+	io_ring_submit_unlock(ctx, issue_flags);
+	return ret;
+}

to avoid making it way too long. For io_uring, it's fine to exceed 80 chars where it makes sense.

Signature · 1 line
Jens Axboe
#46 lore raw

Re: [PATCH v1 06/11] io_uring/kbuf: add buffer ring pinning/unpinning

Quoted 26 lines · Jens Axboe

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 246

Quoted 6 lines · Joanne Koong
io_uring/kbuf.c +6 -0
+int io_uring_buf_ring_pin(struct io_uring_cmd *cmd, unsigned buf_group,
+                       unsigned issue_flags, struct io_buffer_list **bl)
+{
+     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+     struct io_buffer_list *buffer_list;
+     int ret = -EINVAL;

Probably use the usual struct io_buffer_list *bl here and either use an ERR_PTR return, or rename the passed on **bl to **blret or something.

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 276

Quoted 11 lines
io_uring/kbuf.c +11 -0
+int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd, unsigned buf_group,
+                    unsigned issue_flags)
+{
+     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+     struct io_buffer_list *bl;
+     int ret = -EINVAL;
+
+     io_ring_submit_lock(ctx, issue_flags);
+
+     bl = io_buffer_get_list(ctx, buf_group);
+     if (bl && (bl->flags & IOBL_BUF_RING) && (bl->flags & IOBL_PINNED)) {

Usually done as:

Quoted 1 line
        if ((bl->flags & (IOBL_BUF_RING|IOBL_PINNED)) == (IOBL_BUF_RING|IOBL_PINNED))

FWIW, modern compilers will perform this optimization automatically. They'll even optimize it further to !(~bl->flags & (IOBL_BUF_RING|IOBL_PINNED)): https://godbolt.org/z/xGoP4TfhP

Best,
Caleb

Quoted 19 lines

and maybe then just have an earlier

        if (!bl)
                goto err;

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 279

Quoted 3 lines
io_uring/kbuf.c +3 -0
+             bl->flags &= ~IOBL_PINNED;
+             ret = 0;
+     }

err:

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 263

Quoted 3 lines
io_uring/kbuf.c +3 -0
+     io_ring_submit_unlock(ctx, issue_flags);
+     return ret;
+}

to avoid making it way too long. For io_uring, it's fine to exceed 80 chars where it makes sense.

--
Jens Axboe

maintainer
#47 lore raw

Re: [PATCH v1 06/11] io_uring/kbuf: add buffer ring pinning/unpinning

Quoted 27 lines · Caleb Sander Mateos
Quoted 26 lines · On Mon, Feb 9, 2026 at 5:07?PM Jens Axboe <[email protected]> wrote:

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 246

Quoted 6 lines · Joanne Koong
io_uring/kbuf.c +6 -0
+int io_uring_buf_ring_pin(struct io_uring_cmd *cmd, unsigned buf_group,
+                       unsigned issue_flags, struct io_buffer_list **bl)
+{
+     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+     struct io_buffer_list *buffer_list;
+     int ret = -EINVAL;

Probably use the usual struct io_buffer_list *bl here and either use an ERR_PTR return, or rename the passed on **bl to **blret or something.

io_uring/kbuf.c struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len, · line 276

Quoted 11 lines
io_uring/kbuf.c +11 -0
+int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd, unsigned buf_group,
+                    unsigned issue_flags)
+{
+     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+     struct io_buffer_list *bl;
+     int ret = -EINVAL;
+
+     io_ring_submit_lock(ctx, issue_flags);
+
+     bl = io_buffer_get_list(ctx, buf_group);
+     if (bl && (bl->flags & IOBL_BUF_RING) && (bl->flags & IOBL_PINNED)) {

Usually done as:

Quoted 5 lines
Quoted 1 line
        if ((bl->flags & (IOBL_BUF_RING|IOBL_PINNED)) == (IOBL_BUF_RING|IOBL_PINNED))

FWIW, modern compilers will perform this optimization automatically. They'll even optimize it further to !(~bl->flags & (IOBL_BUF_RING|IOBL_PINNED)): https://godbolt.org/z/xGoP4TfhP

Sure, it's not about that, it's more about the common way of doing it, which makes it easier to read for people. FWIW, your example is easier to read too than the original.

Signature · 1 line
Jens Axboe
#48 lore raw

PATCH 7/11 io_uring/kbuf: add recycling for kernel managed buffer rings

In reply to Joanne Koong

Add an interface for buffers to be recycled back into a kernel-managed buffer ring.

This is a preparatory patch for fuse over io-uring.

---
 include/linux/io_uring/cmd.h | 11 +++++++++
 io_uring/kbuf.c              | 44 ++++++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)
include/linux/io_uring/cmd.h +11 -0
index 702b1903e6ee..a488e945f883 100644
@@ -88,6 +88,10 @@ int io_uring_buf_ring_pin(struct io_uring_cmd *cmd, unsigned buf_group,
 			  unsigned issue_flags, struct io_buffer_list **bl);
 int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd, unsigned buf_group,
 			    unsigned issue_flags);
+
+int io_uring_kmbuf_recycle(struct io_uring_cmd *cmd, unsigned int buf_group,
+			   u64 addr, unsigned int len, unsigned int bid,
+			   unsigned int issue_flags);
 #else
 static inline int
 io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
@@ -143,6 +147,13 @@ static inline int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd,
 {
 	return -EOPNOTSUPP;
 }
+static inline int io_uring_kmbuf_recycle(struct io_uring_cmd *cmd,
+					 unsigned int buf_group, u64 addr,
+					 unsigned int len, unsigned int bid,
+					 unsigned int issue_flags)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 
 static inline struct io_uring_cmd *io_uring_cmd_from_tw(struct io_tw_req tw_req)
io_uring/kbuf.c +44 -0

50 lines not on this page · raw message

Signature · 1 line
2.47.3
maintainer
#49 lore raw

Re: [PATCH v1 07/11] io_uring/kbuf: add recycling for kernel managed buffer rings

Quoted 16 lines · Joanne Koong
io_uring/kbuf.c +16 -0
+int io_uring_kmbuf_recycle(struct io_uring_cmd *cmd, unsigned int buf_group,
+			   u64 addr, unsigned int len, unsigned int bid,
+			   unsigned int issue_flags)
+{
+	struct io_kiocb *req = cmd_to_io_kiocb(cmd);
+	struct io_ring_ctx *ctx = req->ctx;
+	struct io_uring_buf_ring *br;
+	struct io_uring_buf *buf;
+	struct io_buffer_list *bl;
+	int ret = -EINVAL;
+
+	if (WARN_ON_ONCE(req->flags & REQ_F_BUFFERS_COMMIT))
+		return ret;
+
+	io_ring_submit_lock(ctx, issue_flags);
+

io_uring/kbuf.c void io_kbuf_drop_legacy(struct io_kiocb *req) · line 130

Quoted 10 lines
io_uring/kbuf.c +10 -0
+	bl = io_buffer_get_list(ctx, buf_group);
+
+	if (!bl || WARN_ON_ONCE(!(bl->flags & IOBL_BUF_RING)) ||
+	    WARN_ON_ONCE(!(bl->flags & IOBL_KERNEL_MANAGED)))
+		goto done;
+
+	br = bl->buf_ring;
+
+	if (WARN_ON_ONCE((br->tail - bl->head) >= bl->nr_entries))
+		goto done;

I think you want:

	if (WARN_ON_ONCE((__u16)(br->tail - bl->head) >= bl->nr_entries))

here to avoid int promotion from messing this up if tail has wrapped.

In general, across the patches for the WARN_ON_ONCE(), it's not a huge issue to have a litter of them for now. Hopefully we can prune some of these down the line, however.

Signature · 1 line
Jens Axboe
#50 lore raw

PATCH 8/11 io_uring/kbuf: add io_uring_is_kmbuf_ring()

In reply to Joanne Koong

io_uring_is_kmbuf_ring() returns true if there is a kernel-managed buffer ring at the specified buffer group.

This is a preparatory patch for upcoming fuse kernel-managed buffer support, which needs to ensure the buffer ring registered by the server is a kernel-managed buffer ring.

---
 include/linux/io_uring/cmd.h |  9 +++++++++
 io_uring/kbuf.c              | 20 ++++++++++++++++++++
 2 files changed, 29 insertions(+)
include/linux/io_uring/cmd.h +9 -0
index a488e945f883..04a937f6f4d3 100644
@@ -92,6 +92,9 @@ int io_uring_buf_ring_unpin(struct io_uring_cmd *cmd, unsigned buf_group,
 int io_uring_kmbuf_recycle(struct io_uring_cmd *cmd, unsigned int buf_group,
 			   u64 addr, unsigned int len, unsigned int bid,
 			   unsigned int issue_flags);
+
+bool io_uring_is_kmbuf_ring(struct io_uring_cmd *cmd, unsigned int buf_group,
+			    unsigned int issue_flags);
 #else
 static inline int
 io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
@@ -154,6 +157,12 @@ static inline int io_uring_kmbuf_recycle(struct io_uring_cmd *cmd,
 {
 	return -EOPNOTSUPP;
 }
+static inline bool io_uring_is_kmbuf_ring(struct io_uring_cmd *cmd,
+					  unsigned int buf_group,
+					  unsigned int issue_flags)
+{
+	return false;
+}
 #endif
 
 static inline struct io_uring_cmd *io_uring_cmd_from_tw(struct io_tw_req tw_req)
io_uring/kbuf.c +20 -0
index 17b6178be4ce..797cc2f0a5e9 100644
@@ -963,3 +963,23 @@ int io_register_kmbuf_ring(struct io_ring_ctx *ctx, void __user *arg)
 
 	return ret;
 }
+
+bool io_uring_is_kmbuf_ring(struct io_uring_cmd *cmd, unsigned int buf_group,
+			    unsigned int issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	struct io_buffer_list *bl;
+	bool is_kmbuf_ring = false;
+
+	io_ring_submit_lock(ctx, issue_flags);
+
+	bl = io_buffer_get_list(ctx, buf_group);
+	if (likely(bl) && (bl->flags & IOBL_KERNEL_MANAGED)) {
+		WARN_ON_ONCE(!(bl->flags & IOBL_BUF_RING));
+		is_kmbuf_ring = true;
+	}
+
+	io_ring_submit_unlock(ctx, issue_flags);
+	return is_kmbuf_ring;
+}
+EXPORT_SYMBOL_GPL(io_uring_is_kmbuf_ring);
Signature · 1 line
2.47.3
#51 lore raw

PATCH 9/11 io_uring/kbuf: export io_ring_buffer_select()

In reply to Joanne Koong

Export io_ring_buffer_select() so that it may be used by callers who pass in a pinned bufring without needing to grab the io_uring mutex.

This is a preparatory patch that will be needed by fuse io-uring, which will need to select a buffer from a kernel-managed bufring while the uring mutex may already be held by in-progress commits, and may need to select a buffer in atomic contexts.

---
 include/linux/io_uring/cmd.h | 14 ++++++++++++++
 io_uring/kbuf.c              |  7 ++++---
 2 files changed, 18 insertions(+), 3 deletions(-)
include/linux/io_uring/cmd.h +14 -0
index 04a937f6f4d3..d4b5943bdeb1 100644
@@ -95,6 +95,10 @@ int io_uring_kmbuf_recycle(struct io_uring_cmd *cmd, unsigned int buf_group,
 
 bool io_uring_is_kmbuf_ring(struct io_uring_cmd *cmd, unsigned int buf_group,
 			    unsigned int issue_flags);
+
+struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
+				       struct io_buffer_list *bl,
+				       unsigned int issue_flags);
 #else
 static inline int
 io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
@@ -163,6 +167,16 @@ static inline bool io_uring_is_kmbuf_ring(struct io_uring_cmd *cmd,
 {
 	return false;
 }
+static inline struct io_br_sel io_ring_buffer_select(struct io_kiocb *req,
+						     size_t *len,
+						     struct io_buffer_list *bl,
+						     unsigned int issue_flags)
+{
+	struct io_br_sel sel = {
+		.val = -EOPNOTSUPP,
+	};
+	return sel;
+}
 #endif
 
 static inline struct io_uring_cmd *io_uring_cmd_from_tw(struct io_tw_req tw_req)
io_uring/kbuf.c +4 -3
index 797cc2f0a5e9..9a93f10d3214 100644
@@ -226,9 +226,9 @@ static bool io_should_commit(struct io_kiocb *req, struct io_buffer_list *bl,
 	return false;
 }
 
-static struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
-					      struct io_buffer_list *bl,
-					      unsigned int issue_flags)
+struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
+				       struct io_buffer_list *bl,
+				       unsigned int issue_flags)
 {
 	struct io_uring_buf_ring *br = bl->buf_ring;
 	__u16 tail, head = bl->head;
@@ -261,6 +261,7 @@ static struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 	}
 	return sel;
 }
+EXPORT_SYMBOL_GPL(io_ring_buffer_select);
 
 struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len,
 				  unsigned buf_group, unsigned int issue_flags)
Signature · 1 line
2.47.3
#52 lore raw

PATCH 10/11 io_uring/kbuf: return buffer id in buffer selection

In reply to Joanne Koong

Return the id of the selected buffer in io_buffer_select(). This is needed for kernel-managed buffer rings to later recycle the selected buffer.

---
 include/linux/io_uring/cmd.h   | 2 +-
 include/linux/io_uring_types.h | 2 ++
 io_uring/kbuf.c                | 7 +++++--
 3 files changed, 8 insertions(+), 3 deletions(-)
include/linux/io_uring/cmd.h +1 -1
index d4b5943bdeb1..94df2bdebe77 100644
@@ -71,7 +71,7 @@ void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd);
 
 /*
  * Select a buffer from the provided buffer group for multishot uring_cmd.
- * Returns the selected buffer address and size.
+ * Returns the selected buffer address, size, and id.
  */
 struct io_br_sel io_uring_cmd_buffer_select(struct io_uring_cmd *ioucmd,
 					    unsigned buf_group, size_t *len,
include/linux/io_uring_types.h +2 -0
index 36cc2e0346d9..5a56bb341337 100644
@@ -100,6 +100,8 @@ struct io_br_sel {
 		void *kaddr;
 	};
 	ssize_t val;
+	/* id of the selected buffer */
+	unsigned buf_id;
 };
 
 
io_uring/kbuf.c +5 -2
index 9a93f10d3214..24c1e34ea23e 100644
@@ -250,6 +250,7 @@ struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 	req->flags |= REQ_F_BUFFER_RING | REQ_F_BUFFERS_COMMIT;
 	req->buf_index = READ_ONCE(buf->bid);
 	sel.buf_list = bl;
+	sel.buf_id = req->buf_index;
 	if (bl->flags & IOBL_KERNEL_MANAGED)
 		sel.kaddr = (void *)(uintptr_t)READ_ONCE(buf->addr);
 	else
@@ -274,10 +275,12 @@ struct io_br_sel io_buffer_select(struct io_kiocb *req, size_t *len,
 
 	bl = io_buffer_get_list(ctx, buf_group);
 	if (likely(bl)) {
-		if (bl->flags & IOBL_BUF_RING)
+		if (bl->flags & IOBL_BUF_RING) {
 			sel = io_ring_buffer_select(req, len, bl, issue_flags);
-		else
+		} else {
 			sel.addr = io_provided_buffer_select(req, len, bl);
+			sel.buf_id = req->buf_index;
+		}
 	}
 	io_ring_submit_unlock(req->ctx, issue_flags);
 	return sel;
Signature · 1 line
2.47.3
maintainer
#53 lore raw

Re: [PATCH v1 10/11] io_uring/kbuf: return buffer id in buffer selection

Quoted 29 lines · Joanne Koong

Return the id of the selected buffer in io_buffer_select(). This is needed for kernel-managed buffer rings to later recycle the selected buffer.

---
 include/linux/io_uring/cmd.h   | 2 +-
 include/linux/io_uring_types.h | 2 ++
 io_uring/kbuf.c                | 7 +++++--
 3 files changed, 8 insertions(+), 3 deletions(-)
include/linux/io_uring/cmd.h +1 -1
index d4b5943bdeb1..94df2bdebe77 100644
@@ -71,7 +71,7 @@ void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd);
 
 /*
  * Select a buffer from the provided buffer group for multishot uring_cmd.
- * Returns the selected buffer address and size.
+ * Returns the selected buffer address, size, and id.
  */
 struct io_br_sel io_uring_cmd_buffer_select(struct io_uring_cmd *ioucmd,
 					    unsigned buf_group, size_t *len,
include/linux/io_uring_types.h +0 -0
index 36cc2e0346d9..5a56bb341337 100644

include/linux/io_uring_types.h struct io_br_sel { · line 105

Quoted 6 lines
include/linux/io_uring_types.h +2 -0
@@ -100,6 +100,8 @@ struct io_br_sel {
 		void *kaddr;
 	};
 	ssize_t val;
+	/* id of the selected buffer */
+	unsigned buf_id;
 };

I'm probably missing something here, but why can't the caller just use req->buf_index for this?

Signature · 1 line
Jens Axboe
#54 lore raw

Re: [PATCH v1 10/11] io_uring/kbuf: return buffer id in buffer selection

Quoted 32 lines · Jens Axboe
Quoted 30 lines · Joanne Koong

Return the id of the selected buffer in io_buffer_select(). This is needed for kernel-managed buffer rings to later recycle the selected buffer.

---
 include/linux/io_uring/cmd.h   | 2 +-
 include/linux/io_uring_types.h | 2 ++
 io_uring/kbuf.c                | 7 +++++--
 3 files changed, 8 insertions(+), 3 deletions(-)
include/linux/io_uring/cmd.h +1 -1
index d4b5943bdeb1..94df2bdebe77 100644
@@ -71,7 +71,7 @@ void io_uring_cmd_issue_blocking(struct io_uring_cmd *ioucmd);
 
 /*
  * Select a buffer from the provided buffer group for multishot uring_cmd.
- * Returns the selected buffer address and size.
+ * Returns the selected buffer address, size, and id.
  */
 struct io_br_sel io_uring_cmd_buffer_select(struct io_uring_cmd *ioucmd,
                                          unsigned buf_group, size_t *len,
include/linux/io_uring_types.h +0 -0
index 36cc2e0346d9..5a56bb341337 100644
@@ -100,6 +100,8 @@ struct io_br_sel {
              void *kaddr;
Quoted 8 lines

include/linux/io_uring_types.h struct io_br_sel { · line 105

Quoted 5 lines
include/linux/io_uring_types.h +2 -0
      };
      ssize_t val;
+     /* id of the selected buffer */
+     unsigned buf_id;
 };

I'm probably missing something here, but why can't the caller just use req->buf_index for this?

The caller can, but from the caller side they only have access to the cmd so they would need to do something like

struct io_kiocb *req = cmd_to_iocb_kiocb(ent->cmd); buf_id = req->buf_index;

which may be kind of ugly with looking inside io-uring internals. Maybe a helper here would be nicer, something like io_uring_cmd_buf_id() or io_uring_req_buf_id(). It seemed cleaner to me to just return the buf id as part of the io_br_sel struct, but I'm happy to do it another way if you have a preference.

Thanks,
Joanne

Quoted 3 lines

--
Jens Axboe

#55 lore raw

PATCH 11/11 io_uring/cmd: set selected buffer index in __io_uring_cmd_done()

In reply to Joanne Koong

When uring_cmd operations select a buffer, the completion queue entry should indicate which buffer was selected.

Set IORING_CQE_F_BUFFER on the completed entry and encode the buffer index if a buffer was selected.

This will be needed for fuse, which needs to relay to userspace which selected buffer contains the data.

---
 io_uring/uring_cmd.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
io_uring/uring_cmd.c +5 -1
index ee7b49f47cb5..6d38df1a812d 100644
@@ -151,6 +151,7 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2,
 		       unsigned issue_flags, bool is_cqe32)
 {
 	struct io_kiocb *req = cmd_to_io_kiocb(ioucmd);
+	u32 cflags = 0;
 
 	if (WARN_ON_ONCE(req->flags & REQ_F_APOLL_MULTISHOT))
 		return;
@@ -160,7 +161,10 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, s32 ret, u64 res2,
 	if (ret < 0)
 		req_set_fail(req);
 
-	io_req_set_res(req, ret, 0);
+	if (req->flags & (REQ_F_BUFFER_SELECTED | REQ_F_BUFFER_RING))
+		cflags |= IORING_CQE_F_BUFFER |
+			(req->buf_index << IORING_CQE_BUFFER_SHIFT);
+	io_req_set_res(req, ret, cflags);
 	if (is_cqe32) {
 		if (req->ctx->flags & IORING_SETUP_CQE_MIXED)
 			req->cqe.flags |= IORING_CQE_F_32;
Signature · 1 line
2.47.3
maintainer
#56 lore raw
In reply to Joanne Koong
Quoted 5 lines · Joanne Koong

Currently, io_uring buffer rings require the application to allocate and manage the backing buffers. This series introduces kernel-managed buffer rings, where the kernel allocates and manages the buffers on behalf of the application.

Quoted 7 lines

This is split out from the fuse over io_uring series in [1], which needs the kernel to own and manage buffers shared between the fuse server and the kernel.

This series is on top of the for-next branch in Jens' io-uring tree. The corresponding liburing changes are in [2] and will be submitted after the changes in this patchset are accepted.

Generally looks pretty good - for context, do you have a branch with these patches and the users on top too? Makes it a bit easier for cross referencing, as some of these really do need an exposed user to make a good judgement on the helpers.

I know there's the older series, but I'm assuming the latter patches changed somewhat too, and it'd be nicer to look at a current set rather than go back to the older ones.

Signature · 1 line
Jens Axboe
#57 lore raw
Quoted 11 lines · Jens Axboe
Quoted 9 lines · Joanne Koong

Currently, io_uring buffer rings require the application to allocate and manage the backing buffers. This series introduces kernel-managed buffer rings, where the kernel allocates and manages the buffers on behalf of the application.

This is split out from the fuse over io_uring series in [1], which needs the kernel to own and manage buffers shared between the fuse server and the kernel.

Quoted 8 lines
Quoted 3 lines

This series is on top of the for-next branch in Jens' io-uring tree. The corresponding liburing changes are in [2] and will be submitted after the changes in this patchset are accepted.

Generally looks pretty good - for context, do you have a branch with these patches and the users on top too? Makes it a bit easier for cross referencing, as some of these really do need an exposed user to make a good judgement on the helpers.

Thanks for reviewing the patches. The branch containing the userside changes on top of these patches is in [1]. I'll make the changes you pointed out in your other comments as part of v2. Once the discussion with Pavel is resolved / figured out with the changes he wants for v2, I'll submit v2.

Thanks,
Joanne

[1] https://github.com/joannekoong/linux/commits/fuse_zero_copy/

Quoted 7 lines

I know there's the older series, but I'm assuming the latter patches changed somewhat too, and it'd be nicer to look at a current set rather than go back to the older ones.

--
Jens Axboe