Skip to contentSkip to conversation

io_uring: extend bvec registration and add mem region lookup

PATCH Superseded Joanne Koong 2 participants 12 messages Mar 24 – Mar 25

Subsystems (4)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v3 cover io_uring: extend bvec registration and add mem region lookup

This series refactors and extends the io_uring registered buffers infrastructure to allow external subsystems to register pre-existing bvec arrays directly and obtain a pointer to the registered memory region.

The motivation for the patches in this series is to make fuse zero-copy possible.

These patches are split out from a previous larger fuse-over-io_uring series [1]. The remaining fuse patches will be submitted separately and linked to.

Thanks,
Joanne

[1] https://lore.kernel.org/linux-fsdevel/[email protected]/

Changelog:
v2: https://lore.kernel.org/io-uring/[email protected]/
v2 -> v3:
* drop patch that makes buffer release callback optional
* add patch for renaming/exporting IO_IMU_DEST / IO_IMU_SOURCE

v1: https://lore.kernel.org/io-uring/[email protected]/
v1 -> v2:
* update io_kernel_buffer_init() to take bitmasked dir directly so callers can set both dest and source

Joanne Koong (5):
io_uring/rsrc: rename
io_buffer_register_bvec()/io_buffer_unregister_bvec()
io_uring/rsrc: split io_buffer_register_request() logic io_uring/rsrc: add io_buffer_register_bvec() io_uring/rsrc: rename and export IO_IMU_DEST / IO_IMU_SOURCE io_uring/rsrc: add io_uring_registered_mem_region_get()

 Documentation/block/ublk.rst   |  14 ++--
 drivers/block/ublk_drv.c       |  22 ++---
 include/linux/io_uring/cmd.h   |  48 +++++++++--
 include/linux/io_uring_types.h |   5 ++
 io_uring/io_uring.c            |   2 +-
 io_uring/rsrc.c                | 145 ++++++++++++++++++++++++---------
 io_uring/rsrc.h                |   5 --
 7 files changed, 174 insertions(+), 67 deletions(-)
Signature · 1 line
2.52.0
#1 lore raw

PATCH v3 1/5 io_uring/rsrc: rename io_buffer_register_bvec()/io_buffer_unregister_bvec()

Currently, io_buffer_register_bvec() takes in a request. In preparation for supporting kernel-populated buffers in fuse io-uring (which will need to register bvecs directly, not through a struct request), rename this to io_buffer_register_request().

A subsequent patch will commandeer the "io_buffer_register_bvec()" function name to support registering bvecs directly.

Rename io_buffer_unregister_bvec() to a more generic name, io_buffer_unregister(), as both io_buffer_register_request() and io_buffer_register_bvec() callers will use it for unregistration.

---
 Documentation/block/ublk.rst | 14 +++++++-------
 drivers/block/ublk_drv.c     | 22 +++++++++++-----------
 include/linux/io_uring/cmd.h | 26 ++++++++++++++++++++------
 io_uring/rsrc.c              | 14 +++++++-------
 4 files changed, 45 insertions(+), 31 deletions(-)
Documentation/block/ublk.rst +7 -7
index 6ad28039663d..f014d1d69019 100644
@@ -382,17 +382,17 @@ Zero copy
 ---------
 
 ublk zero copy relies on io_uring's fixed kernel buffer, which provides
-two APIs: `io_buffer_register_bvec()` and `io_buffer_unregister_bvec`.
+two APIs: `io_buffer_register_request()` and `io_buffer_unregister`.
 
 ublk adds IO command of `UBLK_IO_REGISTER_IO_BUF` to call
-`io_buffer_register_bvec()` for ublk server to register client request
+`io_buffer_register_request()` for ublk server to register client request
 buffer into io_uring buffer table, then ublk server can submit io_uring
 IOs with the registered buffer index. IO command of `UBLK_IO_UNREGISTER_IO_BUF`
-calls `io_buffer_unregister_bvec()` to unregister the buffer, which is
-guaranteed to be live between calling `io_buffer_register_bvec()` and
-`io_buffer_unregister_bvec()`. Any io_uring operation which supports this
-kind of kernel buffer will grab one reference of the buffer until the
-operation is completed.
+calls `io_buffer_unregister()` to unregister the buffer, which is guaranteed
+to be live between calling `io_buffer_register_request()` and
+`io_buffer_unregister()`. Any io_uring operation which supports this kind of
+kernel buffer will grab one reference of the buffer until the operation is
+completed.
 
 ublk server implementing zero copy or user copy has to be CAP_SYS_ADMIN and
 be trusted, because it is ublk server's responsibility to make sure IO buffer
drivers/block/ublk_drv.c +11 -11
index 004f367243b6..b9f293261240 100644
@@ -1627,8 +1627,8 @@ ublk_auto_buf_register(const struct ublk_queue *ubq, struct request *req,
 {
 	int ret;
 
-	ret = io_buffer_register_bvec(cmd, req, ublk_io_release,
-				      io->buf.auto_reg.index, issue_flags);
+	ret = io_buffer_register_request(cmd, req, ublk_io_release,
+					 io->buf.auto_reg.index, issue_flags);
 	if (ret) {
 		if (io->buf.auto_reg.flags & UBLK_AUTO_BUF_REG_FALLBACK) {
 			ublk_auto_buf_reg_fallback(ubq, req->tag);
@@ -1868,7 +1868,7 @@ static int __ublk_batch_dispatch(struct ublk_queue *ubq,
 			ublk_io_unlock(io);
 
 			if (index != -1)
-				io_buffer_unregister_bvec(data->cmd, index,
+				io_buffer_unregister(data->cmd, index,
 						data->issue_flags);
 		}
 
@@ -3091,8 +3091,8 @@ static int ublk_register_io_buf(struct io_uring_cmd *cmd,
 	if (!req)
 		return -EINVAL;
 
-	ret = io_buffer_register_bvec(cmd, req, ublk_io_release, index,
-				      issue_flags);
+	ret = io_buffer_register_request(cmd, req, ublk_io_release, index,
+					 issue_flags);
 	if (ret) {
 		ublk_put_req_ref(io, req);
 		return ret;
@@ -3123,8 +3123,8 @@ ublk_daemon_register_io_buf(struct io_uring_cmd *cmd,
 	if (!ublk_dev_support_zero_copy(ub) || !ublk_rq_has_data(req))
 		return -EINVAL;
 
-	ret = io_buffer_register_bvec(cmd, req, ublk_io_release, index,
-				      issue_flags);
+	ret = io_buffer_register_request(cmd, req, ublk_io_release, index,
+					 issue_flags);
 	if (ret)
 		return ret;
 
@@ -3139,7 +3139,7 @@ static int ublk_unregister_io_buf(struct io_uring_cmd *cmd,
 	if (!(ub->dev_info.flags & UBLK_F_SUPPORT_ZERO_COPY))
 		return -EINVAL;
 
-	return io_buffer_unregister_bvec(cmd, index, issue_flags);
+	return io_buffer_unregister(cmd, index, issue_flags);
 }
 
 static int ublk_check_fetch_buf(const struct ublk_device *ub, __u64 buf_addr)
@@ -3280,7 +3280,7 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
 		goto out;
 
 	/*
-	 * io_buffer_unregister_bvec() doesn't access the ubq or io,
+	 * io_buffer_unregister() doesn't access the ubq or io,
 	 * so no need to validate the q_id, tag, or task
 	 */
 	if (_IOC_NR(cmd_op) == UBLK_IO_UNREGISTER_IO_BUF)
@@ -3347,7 +3347,7 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
 		req = ublk_fill_io_cmd(io, cmd);
 		ret = ublk_config_io_buf(ub, io, cmd, addr, &buf_idx);
 		if (buf_idx != UBLK_INVALID_BUF_IDX)
-			io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
+			io_buffer_unregister(cmd, buf_idx, issue_flags);
 		compl = ublk_need_complete_req(ub, io);
 
 		if (req_op(req) == REQ_OP_ZONE_APPEND)
@@ -3682,7 +3682,7 @@ static int ublk_batch_commit_io(struct ublk_queue *ubq,
 	}
 
 	if (buf_idx != UBLK_INVALID_BUF_IDX)
-		io_buffer_unregister_bvec(data->cmd, buf_idx, data->issue_flags);
+		io_buffer_unregister(data->cmd, buf_idx, data->issue_flags);
 	if (req_op(req) == REQ_OP_ZONE_APPEND)
 		req->__sector = ublk_batch_zone_lba(uc, elem);
 	if (compl)
include/linux/io_uring/cmd.h +20 -6
index 89e1a80d9f5f..165d436073a4 100644
@@ -106,6 +106,12 @@ bool io_uring_is_kmbuf_ring(struct io_uring_cmd *cmd, unsigned int buf_group,
 struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 				       struct io_buffer_list *bl,
 				       unsigned int issue_flags);
+
+int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
+			       void (*release)(void *), unsigned int index,
+			       unsigned int issue_flags);
+int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
+			 unsigned int issue_flags);
 #else
 static inline int
 io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
@@ -184,6 +190,20 @@ static inline struct io_br_sel io_ring_buffer_select(struct io_kiocb *req,
 	};
 	return sel;
 }
+static inline int io_buffer_register_request(struct io_uring_cmd *cmd,
+					     struct request *rq,
+					     void (*release)(void *),
+					     unsigned int index,
+					     unsigned int issue_flags)
+{
+	return -EOPNOTSUPP;
+}
+static inline int io_buffer_unregister(struct io_uring_cmd *cmd,
+				       unsigned int index,
+				       unsigned int issue_flags)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 
 static inline struct io_uring_cmd *io_uring_cmd_from_tw(struct io_tw_req tw_req)
@@ -233,10 +253,4 @@ static inline void io_uring_cmd_done32(struct io_uring_cmd *ioucmd, s32 ret,
 	return __io_uring_cmd_done(ioucmd, ret, res2, issue_flags, true);
 }
 
-int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
-			    void (*release)(void *), unsigned int index,
-			    unsigned int issue_flags);
-int io_buffer_unregister_bvec(struct io_uring_cmd *cmd, unsigned int index,
-			      unsigned int issue_flags);
-
 #endif /* _LINUX_IO_URING_CMD_H */
io_uring/rsrc.c +7 -7
index 52554ed89b11..7579f6992a25 100644
@@ -924,9 +924,9 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg,
 	return ret;
 }
 
-int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
-			    void (*release)(void *), unsigned int index,
-			    unsigned int issue_flags)
+int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
+			       void (*release)(void *), unsigned int index,
+			       unsigned int issue_flags)
 {
 	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
 	struct io_rsrc_data *data = &ctx->buf_table;
@@ -986,10 +986,10 @@ int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,
 	io_ring_submit_unlock(ctx, issue_flags);
 	return ret;
 }
-EXPORT_SYMBOL_GPL(io_buffer_register_bvec);
+EXPORT_SYMBOL_GPL(io_buffer_register_request);
 
-int io_buffer_unregister_bvec(struct io_uring_cmd *cmd, unsigned int index,
-			      unsigned int issue_flags)
+int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
+			 unsigned int issue_flags)
 {
 	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
 	struct io_rsrc_data *data = &ctx->buf_table;
@@ -1019,7 +1019,7 @@ int io_buffer_unregister_bvec(struct io_uring_cmd *cmd, unsigned int index,
 	io_ring_submit_unlock(ctx, issue_flags);
 	return ret;
 }
-EXPORT_SYMBOL_GPL(io_buffer_unregister_bvec);
+EXPORT_SYMBOL_GPL(io_buffer_unregister);
 
 static int validate_fixed_range(u64 buf_addr, size_t len,
 				const struct io_mapped_ubuf *imu)
Signature · 1 line
2.52.0
#2 lore raw

PATCH v3 2/5 io_uring/rsrc: split io_buffer_register_request() logic

In reply to Joanne Koong

Split the main initialization logic in io_buffer_register_request() into a helper function.

This is a preparatory patch for supporting kernel-populated buffers in fuse io-uring, which will be reusing this logic.

---
 io_uring/rsrc.c | 84 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 51 insertions(+), 33 deletions(-)
io_uring/rsrc.c +51 -33
index 7579f6992a25..01c3619e5f07 100644
@@ -924,64 +924,82 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg,
 	return ret;
 }
 
-int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
-			       void (*release)(void *), unsigned int index,
-			       unsigned int issue_flags)
+static struct io_mapped_ubuf *io_kernel_buffer_init(struct io_ring_ctx *ctx,
+						    unsigned int nr_bvecs,
+						    unsigned int total_bytes,
+						    u8 dir,
+						    void (*release)(void *),
+						    void *priv,
+						    unsigned int index)
 {
-	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
 	struct io_rsrc_data *data = &ctx->buf_table;
-	struct req_iterator rq_iter;
 	struct io_mapped_ubuf *imu;
 	struct io_rsrc_node *node;
-	struct bio_vec bv;
-	unsigned int nr_bvecs = 0;
-	int ret = 0;
 
-	io_ring_submit_lock(ctx, issue_flags);
-	if (index >= data->nr) {
-		ret = -EINVAL;
-		goto unlock;
-	}
+	if (index >= data->nr)
+		return ERR_PTR(-EINVAL);
 	index = array_index_nospec(index, data->nr);
 
-	if (data->nodes[index]) {
-		ret = -EBUSY;
-		goto unlock;
-	}
+	if (data->nodes[index])
+		return ERR_PTR(-EBUSY);
 
 	node = io_rsrc_node_alloc(ctx, IORING_RSRC_BUFFER);
-	if (!node) {
-		ret = -ENOMEM;
-		goto unlock;
-	}
+	if (!node)
+		return ERR_PTR(-ENOMEM);
 
-	/*
-	 * blk_rq_nr_phys_segments() may overestimate the number of bvecs
-	 * but avoids needing to iterate over the bvecs
-	 */
-	imu = io_alloc_imu(ctx, blk_rq_nr_phys_segments(rq));
+	imu = io_alloc_imu(ctx, nr_bvecs);
 	if (!imu) {
 		kfree(node);
-		ret = -ENOMEM;
-		goto unlock;
+		return ERR_PTR(-ENOMEM);
 	}
 
 	imu->ubuf = 0;
-	imu->len = blk_rq_bytes(rq);
+	imu->len = total_bytes;
 	imu->acct_pages = 0;
 	imu->folio_shift = PAGE_SHIFT;
+	imu->nr_bvecs = nr_bvecs;
 	refcount_set(&imu->refs, 1);
 	imu->release = release;
-	imu->priv = rq;
+	imu->priv = priv;
+	imu->dir = dir;
 	imu->flags = IO_REGBUF_F_KBUF;
-	imu->dir = 1 << rq_data_dir(rq);
 
+	node->buf = imu;
+	data->nodes[index] = node;
+
+	return imu;
+}
+
+int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
+			       void (*release)(void *), unsigned int index,
+			       unsigned int issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	struct req_iterator rq_iter;
+	struct io_mapped_ubuf *imu;
+	struct bio_vec bv;
+	/*
+	 * blk_rq_nr_phys_segments() may overestimate the number of bvecs
+	 * but avoids needing to iterate over the bvecs
+	 */
+	unsigned int nr_bvecs = blk_rq_nr_phys_segments(rq);
+	unsigned int total_bytes = blk_rq_bytes(rq);
+	int ret = 0;
+
+	io_ring_submit_lock(ctx, issue_flags);
+
+	imu = io_kernel_buffer_init(ctx, nr_bvecs, total_bytes,
+				    1 << rq_data_dir(rq), release, rq, index);
+	if (IS_ERR(imu)) {
+		ret = PTR_ERR(imu);
+		goto unlock;
+	}
+
+	nr_bvecs = 0;
 	rq_for_each_bvec(bv, rq, rq_iter)
 		imu->bvec[nr_bvecs++] = bv;
 	imu->nr_bvecs = nr_bvecs;
 
-	node->buf = imu;
-	data->nodes[index] = node;
 unlock:
 	io_ring_submit_unlock(ctx, issue_flags);
 	return ret;
Signature · 1 line
2.52.0
#3 lore raw

PATCH v3 3/5 io_uring/rsrc: add io_buffer_register_bvec()

In reply to Joanne Koong

Add io_buffer_register_bvec() for registering a bvec array.

This is a preparatory patch for fuse-over-io-uring zero-copy.

---
 include/linux/io_uring/cmd.h | 13 +++++++++++++
 io_uring/rsrc.c              | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)
include/linux/io_uring/cmd.h +13 -0
index 165d436073a4..b4d1d5e8e851 100644
@@ -110,6 +110,10 @@ struct io_br_sel io_ring_buffer_select(struct io_kiocb *req, size_t *len,
 int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
 			       void (*release)(void *), unsigned int index,
 			       unsigned int issue_flags);
+int io_buffer_register_bvec(struct io_uring_cmd *cmd, const struct bio_vec *bvs,
+			    unsigned int nr_bvecs, void (*release)(void *),
+			    void *priv, u8 dir, unsigned int index,
+			    unsigned int issue_flags);
 int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
 			 unsigned int issue_flags);
 #else
@@ -198,6 +202,15 @@ static inline int io_buffer_register_request(struct io_uring_cmd *cmd,
 {
 	return -EOPNOTSUPP;
 }
+static inline int io_buffer_register_bvec(struct io_uring_cmd *cmd,
+					  const struct bio_vec *bvs,
+					  unsigned int nr_bvecs,
+					  void (*release)(void *), void *priv,
+					  u8 dir, unsigned int index,
+					  unsigned int issue_flags)
+{
+	return -EOPNOTSUPP;
+}
 static inline int io_buffer_unregister(struct io_uring_cmd *cmd,
 				       unsigned int index,
 				       unsigned int issue_flags)
io_uring/rsrc.c +35 -0
index 01c3619e5f07..039e3b30ff60 100644
@@ -1006,6 +1006,41 @@ int io_buffer_register_request(struct io_uring_cmd *cmd, struct request *rq,
 }
 EXPORT_SYMBOL_GPL(io_buffer_register_request);
 
+/*
+ * bvs is copied internally. caller may free it on return.
+ */
+int io_buffer_register_bvec(struct io_uring_cmd *cmd, const struct bio_vec *bvs,
+			    unsigned int nr_bvecs, void (*release)(void *),
+			    void *priv, u8 dir, unsigned int index,
+			    unsigned int issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	struct io_mapped_ubuf *imu;
+	struct bio_vec *bvec;
+	unsigned int i, total_bytes = 0;
+	int ret = 0;
+
+	for (i = 0; i < nr_bvecs; i++)
+		total_bytes += bvs[i].bv_len;
+
+	io_ring_submit_lock(ctx, issue_flags);
+	imu = io_kernel_buffer_init(ctx, nr_bvecs, total_bytes, dir, release,
+				    priv, index);
+	if (IS_ERR(imu)) {
+		ret = PTR_ERR(imu);
+		goto unlock;
+	}
+
+	bvec = imu->bvec;
+	for (i = 0; i < nr_bvecs; i++)
+		bvec[i] = bvs[i];
+
+unlock:
+	io_ring_submit_unlock(ctx, issue_flags);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(io_buffer_register_bvec);
+
 int io_buffer_unregister(struct io_uring_cmd *cmd, unsigned int index,
 			 unsigned int issue_flags)
 {
Signature · 1 line
2.52.0
#4 lore raw

PATCH v3 4/5 io_uring/rsrc: rename and export IO_IMU_DEST / IO_IMU_SOURCE

In reply to Joanne Koong

Rename IO_IMU_DEST and IO_IMU_SOURCE to IO_BUF_DEST and IO_BUF_SOURCE and export it so subsystems may use it.

This is needed by the io_buffer_register_bvec() path for callers who may need the buffer to be both readable and writable.

---
 include/linux/io_uring_types.h | 5 +++++
 io_uring/io_uring.c            | 2 +-
 io_uring/rsrc.c                | 2 +-
 io_uring/rsrc.h                | 5 -----
 4 files changed, 7 insertions(+), 7 deletions(-)
include/linux/io_uring_types.h +5 -0
index 328c3c1e2a31..0f41f9351a78 100644
@@ -44,6 +44,11 @@ enum io_uring_cmd_flags {
 	IO_URING_F_COMPAT		= (1 << 12),
 };
 
+enum {
+	IO_BUF_DEST	= 1 << ITER_DEST,
+	IO_BUF_SOURCE	= 1 << ITER_SOURCE,
+};
+
 struct iou_loop_params;
 
 struct io_wq_work_node {
io_uring/io_uring.c +1 -1
index 16122f877aed..b5debc615657 100644
@@ -3215,7 +3215,7 @@ static int __init io_uring_init(void)
 	io_uring_optable_init();
 
 	/* imu->dir is u8 */
-	BUILD_BUG_ON((IO_IMU_DEST | IO_IMU_SOURCE) > U8_MAX);
+	BUILD_BUG_ON((IO_BUF_DEST | IO_BUF_SOURCE) > U8_MAX);
 
 	/*
 	 * Allow user copy in the per-command field, which starts after the
io_uring/rsrc.c +1 -1
index 039e3b30ff60..cf5638406a0c 100644
@@ -820,7 +820,7 @@ static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx,
 	imu->release = io_release_ubuf;
 	imu->priv = imu;
 	imu->flags = 0;
-	imu->dir = IO_IMU_DEST | IO_IMU_SOURCE;
+	imu->dir = IO_BUF_DEST | IO_BUF_SOURCE;
 	if (coalesced)
 		imu->folio_shift = data.folio_shift;
 	refcount_set(&imu->refs, 1);
io_uring/rsrc.h +0 -5
index cff0f8834c35..8d48195faf9d 100644
@@ -23,11 +23,6 @@ struct io_rsrc_node {
 	};
 };
 
-enum {
-	IO_IMU_DEST	= 1 << ITER_DEST,
-	IO_IMU_SOURCE	= 1 << ITER_SOURCE,
-};
-
 enum {
 	IO_REGBUF_F_KBUF		= 1,
 };
Signature · 1 line
2.52.0
#5 lore raw

PATCH v3 5/5 io_uring/rsrc: add io_uring_registered_mem_region_get()

In reply to Joanne Koong

Add io_uring_registered_mem_region_get() helper to allow io_uring command handlers to retrieve the vmapped pointer to the ring's registered memory region's pages as well as the size of the region. This provides a way for uring cmd implementations to directly access pre-registered memory for passing data.

This will be used by fuse for reading/writing header data between the kernel and the server.

---
 include/linux/io_uring/cmd.h |  9 +++++++++
 io_uring/rsrc.c              | 18 ++++++++++++++++++
 2 files changed, 27 insertions(+)
include/linux/io_uring/cmd.h +9 -0
index b4d1d5e8e851..ade0eb807da6 100644
@@ -51,6 +51,9 @@ int io_uring_cmd_import_fixed_vec(struct io_uring_cmd *ioucmd,
 				  size_t uvec_segs,
 				  int ddir, struct iov_iter *iter,
 				  unsigned issue_flags);
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+					 unsigned *nr_pages,
+					 unsigned issue_flags);
 
 /*
  * Completes the request, i.e. posts an io_uring CQE and deallocates @ioucmd
@@ -132,6 +135,12 @@ static inline int io_uring_cmd_import_fixed_vec(struct io_uring_cmd *ioucmd,
 {
 	return -EOPNOTSUPP;
 }
+static inline void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+						       unsigned *nr_pages,
+						       unsigned issue_flags)
+{
+	return NULL;
+}
 static inline void __io_uring_cmd_done(struct io_uring_cmd *cmd, s32 ret,
 		u64 ret2, unsigned issue_flags, bool is_cqe32)
 {
io_uring/rsrc.c +18 -0
index cf5638406a0c..c706324fd66d 100644
@@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
 	return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
 }
 
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+					 unsigned *nr_pages,
+					 unsigned issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	void *ptr;
+
+	io_ring_submit_lock(ctx, issue_flags);
+
+	ptr = ctx->param_region.ptr;
+	*nr_pages = ctx->param_region.nr_pages;
+
+	io_ring_submit_unlock(ctx, issue_flags);
+
+	return ptr;
+}
+EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);
+
 /* Lock two rings at once. The rings must be different! */
 static void lock_two_rings(struct io_ring_ctx *ctx1, struct io_ring_ctx *ctx2)
 {
Signature · 1 line
2.52.0
maintainer
#6 lore raw

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

Quoted 15 lines · Joanne Koong
io_uring/rsrc.c +7 -0
index cf5638406a0c..c706324fd66d 100644
@@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
 	return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
 }
 
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+					 unsigned *nr_pages,
+					 unsigned issue_flags)
+{
+	struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+	void *ptr;
+

io_uring/rsrc.c int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, · line 1201

Quoted 10 lines
io_uring/rsrc.c +10 -0
+	io_ring_submit_lock(ctx, issue_flags);
+
+	ptr = ctx->param_region.ptr;
+	*nr_pages = ctx->param_region.nr_pages;
+
+	io_ring_submit_unlock(ctx, issue_flags);
+
+	return ptr;
+}
+EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);

This looks suspicious, but I actually think it looks suspicious because you add the submit locking around it. For patterns like that, it makes the brain go "hmm, what protects this from going invalid the instant io_ring_submit_unlock() is called??". But this should be stable for the duration of the ring, hence the locking should not be needed at all?

I'd probably also prefer wrapping this in some kind of return struct, so that you just return that rather than having one part returned directly and one stuffed in a pointer to an unsigned for number of pages.

Signature · 1 line
Jens Axboe
#7 lore raw

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

Quoted 25 lines · Jens Axboe
Quoted 23 lines · Joanne Koong
io_uring/rsrc.c +15 -0
index cf5638406a0c..c706324fd66d 100644
@@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
      return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
 }
 
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+                                      unsigned *nr_pages,
+                                      unsigned issue_flags)
+{
+     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+     void *ptr;
+
+     io_ring_submit_lock(ctx, issue_flags);
+
+     ptr = ctx->param_region.ptr;
+     *nr_pages = ctx->param_region.nr_pages;
+
+     io_ring_submit_unlock(ctx, issue_flags);
+
+     return ptr;
Quoted 8 lines

io_uring/rsrc.c int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, · line 1201

Quoted 2 lines
io_uring/rsrc.c +2 -0
+}
+EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);

This looks suspicious, but I actually think it looks suspicious because you add the submit locking around it. For patterns like that, it makes the brain go "hmm, what protects this from going invalid the instant io_ring_submit_unlock() is called??". But this should be stable for the duration of the ring, hence the locking should not be needed at all?

My understanding is that once a memory region is registered to the ring, it's registered for the ring's lifetime. There's no uapi to unregister a memory region and my interpretation of the last paragraph in this thread [1] is that unregistration is not intended to be added/supported. I think the submit locking is needed in case another thread is currently registering it so we don't see partially initialized state between ptr and nr_pages (eg if the caller calls this from a task work callback).

[1] https://lore.kernel.org/linux-fsdevel/[email protected]/

Quoted 4 lines

I'd probably also prefer wrapping this in some kind of return struct, so that you just return that rather than having one part returned directly and one stuffed in a pointer to an unsigned for number of pages.

Sounds good, I will make this change.

Thanks,
Joanne

Quoted 3 lines

--
Jens Axboe

maintainer
#8 lore raw

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

Quoted 35 lines · Joanne Koong
Quoted 25 lines · On Wed, Mar 25, 2026 at 7:56?AM Jens Axboe <[email protected]> wrote:
Quoted 23 lines · Joanne Koong
io_uring/rsrc.c +15 -0
index cf5638406a0c..c706324fd66d 100644
@@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
      return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
 }
 
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+                                      unsigned *nr_pages,
+                                      unsigned issue_flags)
+{
+     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+     void *ptr;
+
+     io_ring_submit_lock(ctx, issue_flags);
+
+     ptr = ctx->param_region.ptr;
+     *nr_pages = ctx->param_region.nr_pages;
+
+     io_ring_submit_unlock(ctx, issue_flags);
+
+     return ptr;
Quoted 8 lines

io_uring/rsrc.c int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, · line 1201

Quoted 2 lines
io_uring/rsrc.c +2 -0
+}
+EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);

This looks suspicious, but I actually think it looks suspicious because you add the submit locking around it. For patterns like that, it makes the brain go "hmm, what protects this from going invalid the instant io_ring_submit_unlock() is called??". But this should be stable for the duration of the ring, hence the locking should not be needed at all?

Quoted 8 lines

My understanding is that once a memory region is registered to the ring, it's registered for the ring's lifetime. There's no uapi to unregister a memory region and my interpretation of the last paragraph in this thread [1] is that unregistration is not intended to be added/supported. I think the submit locking is needed in case another thread is currently registering it so we don't see partially initialized state between ptr and nr_pages (eg if the caller calls this from a task work callback).

Yes good point - can you please add a comment to that effect? Both why it's safe to return this state outside the lock, and also why the lock is actually required to ensure we see a sane state (either region fully registered, or not there)?

Signature · 1 line
Jens Axboe
#9 lore raw

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

Quoted 46 lines · Jens Axboe
Quoted 35 lines · Joanne Koong
Quoted 25 lines · On Wed, Mar 25, 2026 at 7:56?AM Jens Axboe <[email protected]> wrote:
Quoted 23 lines · Joanne Koong
io_uring/rsrc.c +15 -0
index cf5638406a0c..c706324fd66d 100644
@@ -1182,6 +1182,24 @@ int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter,
      return io_import_fixed(ddir, iter, node->buf, buf_addr, len);
 }
 
+void *io_uring_registered_mem_region_get(struct io_uring_cmd *cmd,
+                                      unsigned *nr_pages,
+                                      unsigned issue_flags)
+{
+     struct io_ring_ctx *ctx = cmd_to_io_kiocb(cmd)->ctx;
+     void *ptr;
+
+     io_ring_submit_lock(ctx, issue_flags);
+
+     ptr = ctx->param_region.ptr;
+     *nr_pages = ctx->param_region.nr_pages;
+
+     io_ring_submit_unlock(ctx, issue_flags);
+
+     return ptr;
Quoted 8 lines

io_uring/rsrc.c int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, · line 1201

Quoted 2 lines
io_uring/rsrc.c +2 -0
+}
+EXPORT_SYMBOL_GPL(io_uring_registered_mem_region_get);

This looks suspicious, but I actually think it looks suspicious because you add the submit locking around it. For patterns like that, it makes the brain go "hmm, what protects this from going invalid the instant io_ring_submit_unlock() is called??". But this should be stable for the duration of the ring, hence the locking should not be needed at all?

Quoted 8 lines

My understanding is that once a memory region is registered to the ring, it's registered for the ring's lifetime. There's no uapi to unregister a memory region and my interpretation of the last paragraph in this thread [1] is that unregistration is not intended to be added/supported. I think the submit locking is needed in case another thread is currently registering it so we don't see partially initialized state between ptr and nr_pages (eg if the caller calls this from a task work callback).

Quoted 4 lines

Yes good point - can you please add a comment to that effect? Both why it's safe to return this state outside the lock, and also why the lock is actually required to ensure we see a sane state (either region fully registered, or not there)?

Good idea, I will add a comment about this to make this more clear, something like:
/*
* The submit lock ensures we don't see partially initialized state
* if another thread is currently registering the region. Once registered,
* the region is stable for the ring's lifetime (no unregister API exists),
* so it's safe to access the returned pointer outside the lock. */

I will wait a day or two to send out v4 in case additional comments come in.

Thanks,
Joanne

Quoted 3 lines

--
Jens Axboe

maintainer
#10 lore raw

Re: [PATCH v3 5/5] io_uring/rsrc: add io_uring_registered_mem_region_get()

Quoted 8 lines · Joanne Koong

Good idea, I will add a comment about this to make this more clear, something like:
/*
* The submit lock ensures we don't see partially initialized state
* if another thread is currently registering the region. Once registered,
* the region is stable for the ring's lifetime (no unregister API exists),
* so it's safe to access the returned pointer outside the lock. */

Perfect, thanks!

Signature · 1 line
Jens Axboe
#11 lore raw
In reply to Joanne Koong
Quoted 10 lines · Joanne Koong

This series refactors and extends the io_uring registered buffers infrastructure to allow external subsystems to register pre-existing bvec arrays directly and obtain a pointer to the registered memory region.

The motivation for the patches in this series is to make fuse zero-copy possible.

These patches are split out from a previous larger fuse-over-io_uring series [1]. The remaining fuse patches will be submitted separately and linked to.

The fuse patches that build on this series can be found here:
https://lore.kernel.org/linux-fsdevel/[email protected]/

Thanks,
Joanne