Skip to contentSkip to conversation

scsi: bsg: release io_uring requests from end-I/O

PATCH In review Yingjie Wang 3 participants 11 messages Nov 18 – Nov 20

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v4 cover scsi: bsg: release io_uring requests from end-I/O

This is the reworked bsg half of the series "scsi: release completed passthrough requests before user copyback" (RFC v3, 2026-11-07). The four synchronous patches have been posted separately as a non-RFC v4 series. These two patches apply on top of that series:

https://lore.kernel.org/r/[email protected]

Original RFC v3 series:
https://lore.kernel.org/r/[email protected]

Yang's suggestion:
https://lore.kernel.org/r/[email protected]

Compared to the RFC v3 version of this half:

- RFC v3 5/6 exposed a new io_uring helper that handed the copied-SQE storage of a uring command to the driver. Yang Xiuwei pointed out that storage owned by bsg itself avoids coupling the BSG driver to the io_uring layout and lifetime, with a per-file or per-queue pool and -EAGAIN on exhaustion instead of sleeping. That removes the need for the io_uring contract ruling that RFC v3 asked for; the io_uring helper is dropped, and the reworked series no longer changes io_uring code.

- Patch 1 adds a per-file pool of 96-byte completion buffers to bsg, allocated at open() and freed at release(). Every open() of a SCSI bsg device creates a pool, so it is charged to the opener's memory cgroup; transport bsg devices and kernels built without io_uring allocate no pool. Patch 2 checks a buffer out at issue time, snapshots the sense data in end-I/O, and returns the buffer in task work before posting the CQE. Only commands that pass a response buffer and a nonzero response length can deliver sense bytes, so only those consume a pool buffer. The buffer is taken before the user data buffer is mapped, because unmapping a bounce-mapped READ copies zero-filled bounce pages back to the caller: a command rejected after mapping would overwrite the caller's data.

- When the pool is exhausted, submission returns -EAGAIN without waiting for a buffer. An initial nonblocking attempt may be retried through io-wq; if the pool is still empty in the worker, the command completes with -EAGAIN and userspace must resubmit it to retry. This introduces a per-file limit of 64 commands that pass both a response buffer and a nonzero response length and are in flight or waiting for task work.
64 is BSG_DEFAULT_CMDS, the default bsg reports through SG_GET_COMMAND_Q, but the old io_uring path enforced no such limit, and SG_SET_COMMAND_Q does not resize the pool. The depth can be raised in the code if deeper queues on one file matter; each further 64 slots cost about another 8.6 KiB per open file.

I chose a per-file pool over a per-queue one so that one opener cannot exhaust the buffers available to other openers and make their commands fail with -EAGAIN, for example with many commands in flight, or with a DEFER_TASKRUN ring whose task has not entered the kernel to wait for completions, so its task work has not run. It also makes the 64-command limit apply per file rather than to the whole device. The cost is one pool per open file, about 8.6 KiB of slab on x86-64, including for openers that only use SG_IO, on kernels built with io_uring.

Tested on a KASAN+lockdep kernel in QEMU with a virtio-scsi disk: the existing regression from the synchronous series passes (960 assertions, 740 of them on the BSG io_uring path), and its 20 deferred io_uring trace windows now retain no request, where the synchronous series alone leaves one in each. The canceled-task-work regression passes. A new pool test submits one 256-deep batch on a DEFER_TASKRUN ring, which runs task work only when the task waits for completions:
the 192 commands beyond the pool depth complete with -EAGAIN instead of sleeping, all of them complete on resubmission with the expected status and sense length, and all 64 buffers are available afterwards. With all 64 buffers held, an unaligned READ is rejected before its data buffer is mapped, so the caller's data is untouched. No kernel diagnostics. Both patches build with W=1 on x86-64; the final objects also build on i386 and with IO_URING=n and SCSI=m.

An earlier memcg test of the same allocation code, before this rebase, confirmed the accounting on a non-debug kernel: each bsg open was charged 8832 bytes more than an open of /dev/null, and with memory.max at 32 MiB, repeated opens triggered the memcg OOM killer after about 3600 opens.

To check the overhead of this series, earlier tests of the same implementation, before this rebase, timed batches of INQUIRY commands through a QEMU virtio-scsi disk on a non-debug x86-64 kernel (one fd, normal and DEFER_TASKRUN rings; batch time divided by the number of commands, not per-command latency). Up to 64 commands per batch, the time per command was within about 2% of the kernel without these two patches, with or without a response buffer. At 96 with a response buffer, the 32 commands beyond the pool depth completed with -EAGAIN and were resubmitted after the batch, adding about 17% per command. These numbers come from one VM, not real hardware.

Both patches are tagged for stable. Patch 2 fixes 7b6d3255e7f8 ("scsi: bsg: add io_uring passthrough handler") and does not build without patch 1, so a backport needs both, in order. Both are larger than the 100 lines the stable rules ask for; if that rules them out, I am fine with dropping the tags.

Yingjie Wang (2):
block: bsg: add a per-file pool of uring_cmd completion buffers
scsi: bsg: release io_uring requests from end-I/O

 block/bsg-lib.c         |   3 +-
 block/bsg.c             | 141 +++++++++++++++++++++++++++++++++++++++-
 drivers/scsi/scsi_bsg.c | 118 ++++++++++++++++++++++++---------
 include/linux/bsg.h     |   6 +-
 4 files changed, 232 insertions(+), 36 deletions(-)

base-commit: 9f9e0b1e5399045982788a954d3461de894e40c7
prerequisite-patch-id: a314d6d602c04c1f965a95edc39f159adfabdc2f
prerequisite-patch-id: d2d516da234fb37cd463dfc6292a62fcdcc6673f
prerequisite-patch-id: ce66c68fb99ef813e5506ba7f3de512c1fd0c832
prerequisite-patch-id: 9de82eb99c62cae2c0a4a266744008e25a241fc2

Signature · 1 line
2.43.0
#1 lore raw

PATCH v4 1/2 block: bsg: add a per-file pool of uring_cmd completion buffers

A bsg uring_cmd driver may need to pass data that must outlive the block request to its task work: scsi_bsg has to snapshot sense data when it stops keeping completed requests until task work runs (next patch).

Let a bsg driver request a pool of fixed-size buffers for each open file, allocated at open() time and freed at release(). The driver declares the buffer size it needs when the queue is registered with bsg_register_queue(), and checks a buffer out with bsg_uring_cmd_get_completion_buf() at issue time. When the pool is exhausted, the caller can reject the command with -EAGAIN instead of allocating or sleeping in the I/O path.

On a kernel built with io_uring, bsg allocates a pool at open() for drivers that register a nonzero completion buffer size. The pool lives as long as the file, so charge it to the opener's memory cgroup with GFP_KERNEL_ACCOUNT. Without io_uring no uring_cmd can be issued, so no pool is allocated.

The pool depth is BSG_DEFAULT_CMDS (64), the value bsg reports by default through SG_GET_COMMAND_Q. For SCSI the buffers take 6 KiB per open file. A buffer stays checked out from issue until task work returns it, so the depth caps the number of commands per file that hold a buffer. The depth is deliberately not tied to the device queue
depth: end-I/O releases the request tag, so the number of commands waiting for task work is not bounded by can_queue. If the pool were sized to a device queue depth of 1, its only buffer could still be held by a completed command waiting for task work after that command's tag was freed; the next command would then fail with -EAGAIN although a tag is available.

Each buffer must be returned exactly once, using bsg_uring_cmd_put_completion_buf() before the command completes or fails. The pool warns if a buffer is returned while it is already free.

io_uring holds a reference on the file for the lifetime of a request, and a driver returns its buffer before completing the command, so bsg_release() cannot free a pool that still has buffers checked out.

The SCSI registration passes SCSI_SENSE_BUFFERSIZE; transport bsg registers no uring_cmd handler and passes 0, so it allocates no pool. No in-tree caller checks buffers out yet; scsi_bsg adopts the pool in the next patch.

---
 block/bsg-lib.c         |   3 +-
 block/bsg.c             | 141 +++++++++++++++++++++++++++++++++++++++-
 drivers/scsi/scsi_bsg.c |   2 +-
 include/linux/bsg.h     |   6 +-
 4 files changed, 147 insertions(+), 5 deletions(-)
block/bsg-lib.c +2 -1
index 536cee4430a7..6f9aedd9c8e9 100644
@@ -400,7 +400,8 @@ struct request_queue *bsg_setup_queue(struct device *dev, const char *name,
 
 	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
 
-	bset->bd = bsg_register_queue(q, dev, name, bsg_transport_sg_io_fn, NULL);
+	bset->bd = bsg_register_queue(q, dev, name, bsg_transport_sg_io_fn,
+				      NULL, 0);
 	if (IS_ERR(bset->bd)) {
 		ret = PTR_ERR(bset->bd);
 		goto out_cleanup_queue;
block/bsg.c +139 -2

179 lines not on this page · raw message

drivers/scsi/scsi_bsg.c +1 -1
index 5ffbd1843861..1c9cf77d0d6a 100644
@@ -300,5 +300,5 @@ struct bsg_device *scsi_bsg_register_queue(struct scsi_device *sdev)
 {
 	return bsg_register_queue(sdev->request_queue, &sdev->sdev_gendev,
 			dev_name(&sdev->sdev_gendev), scsi_bsg_sg_io_fn,
-			scsi_bsg_uring_cmd);
+			scsi_bsg_uring_cmd, SCSI_SENSE_BUFFERSIZE);
 }
include/linux/bsg.h +5 -1
index 162730bfc2d8..3f33ead2c3ee 100644
@@ -17,7 +17,11 @@ typedef int (bsg_uring_cmd_fn)(struct request_queue *q, struct io_uring_cmd *iou
 
 struct bsg_device *bsg_register_queue(struct request_queue *q,
 		struct device *parent, const char *name,
-		bsg_sg_io_fn *sg_io_fn, bsg_uring_cmd_fn *uring_cmd_fn);
+		bsg_sg_io_fn *sg_io_fn, bsg_uring_cmd_fn *uring_cmd_fn,
+		size_t completion_buf_size);
 void bsg_unregister_queue(struct bsg_device *bcd);
 
+void *bsg_uring_cmd_get_completion_buf(struct io_uring_cmd *ioucmd, size_t len);
+void bsg_uring_cmd_put_completion_buf(struct io_uring_cmd *ioucmd, void *buf);
+
 #endif /* _LINUX_BSG_H */
Signature · 1 line
2.43.0
#2 lore raw

PATCH v4 2/2 scsi: bsg: release io_uring requests from end-I/O

In reply to Yingjie Wang

The BSG io_uring end-I/O callback leaves completed requests for task work to release. The request keeps its tag until that work runs. If the submitting task then faults on memory backed by the same queue while its completed requests hold all the tags, it blocks waiting for a tag and never runs the task work that would free one.

Snapshot the SCSI result and bounded sense data in end-I/O, queue task work, and return RQ_END_IO_FREE for every completion. Task work then unmaps the saved BIO, copies the saved sense data and posts the CQE, without dereferencing the request or scsi_cmnd.

Store the sense snapshot in a buffer checked out from the per-file bsg completion pool, and only for commands that pass a response buffer and a nonzero response length, since only those can deliver sense bytes. This adds no allocation to the I/O path and does not grow the request.

For a bounce-mapped READ, blk_rq_unmap_user() copies the zero-filled bounce pages back to userspace even if no I/O was issued. Check out the completion buffer before mapping user data so that pool exhaustion leaves the caller's buffer untouched.

When the pool is exhausted, submission returns -EAGAIN without waiting for a buffer. An initial nonblocking attempt may be retried through io-wq; if the pool is still empty in the worker, the command completes with -EAGAIN and userspace must resubmit it to retry.

This introduces a per-file limit of 64 commands that pass both a response buffer and a nonzero response length and are in flight or waiting for task work. SG_SET_COMMAND_Q does not resize the pool.

Task work returns the buffer after the sense copy and before io_uring_cmd_done32(); io_uring keeps a reference on the file until the request is freed, so the pool cannot be released while a buffer is checked out. Clear the reported sense length on cancellation or a failed sense copy.

Fixes 7b6d3255e7f8 ("scsi: bsg: add io_uring passthrough handler")
---
 drivers/scsi/scsi_bsg.c | 116 +++++++++++++++++++++++++++++-----------
 1 file changed, 85 insertions(+), 31 deletions(-)
drivers/scsi/scsi_bsg.c +85 -31
index 1c9cf77d0d6a..8f9b2aaa3ccf 100644
@@ -16,9 +16,13 @@
  */
 struct scsi_bsg_uring_cmd_pdu {
 	struct bio *bio;		/* mapped user buffer, unmap in task work */
-	struct request *req;		/* block request, freed in task work */
+	u8 *sense_buffer;		/* sense snapshot from the per-file pool */
 	u64 response_addr;		/* user space response buffer address */
-	u32 max_response_len;		/* user response buffer size */
+	/* max_response_len is replaced by res2 after the end-I/O snapshot. */
+	union {
+		u32 max_response_len;	/* maximum sense snapshot size */
+		u64 res2;		/* saved completion result */
+	};
 };
 static_assert(sizeof(struct scsi_bsg_uring_cmd_pdu) <= sizeof_field(struct io_uring_cmd, pdu));
 
@@ -28,17 +32,22 @@ static inline struct scsi_bsg_uring_cmd_pdu *scsi_bsg_uring_cmd_pdu(
 	return io_uring_cmd_to_pdu(ioucmd, struct scsi_bsg_uring_cmd_pdu);
 }
 
-/* Task work: build res2 (layout in uapi/linux/bsg.h) and copy sense to user. */
+static u64 scsi_bsg_res2_clear_sense_len(u64 res2)
+{
+	return bsg_scsi_res2_build(bsg_scsi_res2_device_status(res2),
+				   bsg_scsi_res2_driver_status(res2),
+				   bsg_scsi_res2_host_status(res2), 0,
+				   bsg_scsi_res2_resid_len(res2));
+}
+
+/* Task work: unmap the saved BIO and copy sense to user space. */
 static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
 {
 	struct io_uring_cmd *ioucmd = io_uring_cmd_from_tw(tw_req);
 	struct scsi_bsg_uring_cmd_pdu *pdu = scsi_bsg_uring_cmd_pdu(ioucmd);
-	struct request *rq = pdu->req;
-	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(rq);
-	u64 res2;
+	u64 res2 = pdu->res2;
+	u8 sense_len = bsg_scsi_res2_sense_len(res2);
 	int ret = 0;
-	u8 driver_status = 0;
-	u8 sense_len_wr = 0;
 
 	if (pdu->bio)
 		ret = blk_rq_unmap_user(pdu->bio);
@@ -48,27 +57,22 @@ static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
 	 * has exited, this may run from a kworker or in a different task, so
 	 * don't copy the sense data to the submitter's address.
 	 */
-	if (unlikely(tw.cancel))
+	if (unlikely(tw.cancel)) {
 		ret = -ECANCELED;
-
-	if (scsi_status_is_check_condition(scmd->result)) {
-		driver_status = DRIVER_SENSE;
-		if (pdu->response_addr && !tw.cancel)
-			sense_len_wr = min_t(unsigned int, pdu->max_response_len,
-					     scmd->sense_len);
-	}
-
-	if (sense_len_wr) {
-		if (copy_to_user(uptr64(pdu->response_addr), scmd->sense_buffer,
-				 sense_len_wr))
+		sense_len = 0;
+	} else if (sense_len) {
+		if (copy_to_user(uptr64(pdu->response_addr), pdu->sense_buffer,
+				 sense_len)) {
 			ret = -EFAULT;
+			sense_len = 0;
+		}
 	}
 
-	res2 = bsg_scsi_res2_build(status_byte(scmd->result), driver_status,
-				  host_byte(scmd->result), sense_len_wr,
-				  scmd->resid_len);
+	if (!sense_len)
+		res2 = scsi_bsg_res2_clear_sense_len(res2);
 
-	blk_mq_free_request(rq);
+	if (pdu->sense_buffer)
+		bsg_uring_cmd_put_completion_buf(ioucmd, pdu->sense_buffer);
 	io_uring_cmd_done32(ioucmd, ret, res2,
 			    IO_URING_CMD_TASK_WORK_ISSUE_FLAGS);
 }
@@ -78,9 +82,26 @@ static enum rq_end_io_ret scsi_bsg_uring_cmd_done(struct request *req,
 						  const struct io_comp_batch *iocb)
 {
 	struct io_uring_cmd *ioucmd = req->end_io_data;
+	struct scsi_bsg_uring_cmd_pdu *pdu = scsi_bsg_uring_cmd_pdu(ioucmd);
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
+	u8 driver_status = 0;
+	u8 sense_len = 0;
 
+	if (scsi_status_is_check_condition(scmd->result)) {
+		driver_status = DRIVER_SENSE;
+		if (pdu->response_addr) {
+			sense_len = min_t(unsigned int, pdu->max_response_len,
+					  scmd->sense_len);
+			if (sense_len)
+				memcpy(pdu->sense_buffer, scmd->sense_buffer,
+				       sense_len);
+		}
+	}
+	pdu->res2 = bsg_scsi_res2_build(status_byte(scmd->result),
+					driver_status, host_byte(scmd->result),
+					sense_len, scmd->resid_len);
 	io_uring_cmd_do_in_task_lazy(ioucmd, scsi_bsg_uring_task_cb);
-	return RQ_END_IO_NONE;
+	return RQ_END_IO_FREE;
 }
 
 static int scsi_bsg_map_user_buffer(struct request *req,
@@ -123,8 +144,15 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 	u32 dout_xfer_len = READ_ONCE(cmd->dout_xfer_len);
 	u64 din_xferp = READ_ONCE(cmd->din_xferp);
 	u32 din_xfer_len = READ_ONCE(cmd->din_xfer_len);
+	u64 response_addr = READ_ONCE(cmd->response);
+	u32 max_response_len = min_t(u32, READ_ONCE(cmd->max_response_len),
+				     SCSI_SENSE_BUFFERSIZE);
+	u32 timeout_ms = READ_ONCE(cmd->timeout_ms);
 	int ret;
 
+	if (ioucmd->flags & IORING_URING_CMD_MULTISHOT)
+		return -EOPNOTSUPP;
+
 	if (cmd->protocol != BSG_PROTOCOL_SCSI ||
 	    cmd->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD)
 		return -EINVAL;
@@ -168,8 +196,32 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 		goto out_free_req;
 	}
 
-	pdu->response_addr = cmd->response;
-	pdu->max_response_len = cmd->max_response_len;
+	pdu->response_addr = response_addr;
+	pdu->max_response_len = max_response_len;
+
+	/*
+	 * Sense bytes can only be delivered when the caller passed a
+	 * response buffer and a nonzero response length; a command without
+	 * both never snapshots sense, so it needs no pool buffer and cannot
+	 * be rejected for pool exhaustion.
+	 *
+	 * Take the buffer before mapping the user data buffer.
+	 * blk_rq_unmap_user() is not pure cleanup: for a bounce-mapped READ
+	 * it copies the zero-filled bounce pages back into the caller's
+	 * buffer, so a command rejected for pool exhaustion after mapping
+	 * would zero the caller's data before returning -EAGAIN.
+	 */
+	if (response_addr && max_response_len) {
+		pdu->sense_buffer =
+			bsg_uring_cmd_get_completion_buf(ioucmd,
+							 SCSI_SENSE_BUFFERSIZE);
+		if (!pdu->sense_buffer) {
+			ret = -EAGAIN;
+			goto out_free_req;
+		}
+	} else {
+		pdu->sense_buffer = NULL;
+	}
 
 	if (dout_xfer_len || din_xfer_len) {
 		bool is_write = dout_xfer_len > 0;
@@ -180,22 +232,24 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 					       gfp_mask, is_write, buf_addr,
 					       buf_len);
 		if (ret)
-			goto out_free_req;
+			goto out_put_buf;
 		pdu->bio = req->bio;
 	} else {
 		pdu->bio = NULL;
 	}
 
-	req->timeout = cmd->timeout_ms ?
-		msecs_to_jiffies(cmd->timeout_ms) : BLK_DEFAULT_SG_TIMEOUT;
+	req->timeout = timeout_ms ? msecs_to_jiffies(timeout_ms) :
+				   BLK_DEFAULT_SG_TIMEOUT;
 
 	req->end_io = scsi_bsg_uring_cmd_done;
 	req->end_io_data = ioucmd;
-	pdu->req = req;
 
 	blk_execute_rq_nowait(req, false);
 	return -EIOCBQUEUED;
 
+out_put_buf:
+	if (pdu->sense_buffer)
+		bsg_uring_cmd_put_completion_buf(ioucmd, pdu->sense_buffer);
 out_free_req:
 	blk_mq_free_request(req);
 	return ret;
Signature · 1 line
2.43.0
#7 lore raw

PATCH 2/2 scsi: bsg: release io_uring requests from end-I/O

In reply to Yang Xiuwei

Holding the completed request until task work keeps its tag busy and can deadlock deferred taskrun when the submitter faults on the same queue. Snapshot result and sense in end-I/O, return RQ_END_IO_FREE, and let task work unmap and copy without touching the request. Sense snapshots use the per-device completion pool; check out before mapping user data, return -EAGAIN when empty, and put immediately in end-I/O when no sense needs delivering.

Fixes 7b6d3255e7f8 ("scsi: bsg: add io_uring passthrough handler")
---
 drivers/scsi/scsi_bsg.c | 128 +++++++++++++++++++++++++++++++---------
 1 file changed, 101 insertions(+), 27 deletions(-)
drivers/scsi/scsi_bsg.c +101 -27
index e4a65d7a2374..739778244f42 100644
@@ -16,9 +16,13 @@
  */
 struct scsi_bsg_uring_cmd_pdu {
 	struct bio *bio;		/* mapped user buffer, unmap in task work */
-	struct request *req;		/* block request, freed in task work */
+	u8 *sense_buffer;		/* sense snapshot from the per-device pool */
 	u64 response_addr;		/* user space response buffer address */
-	u32 max_response_len;		/* user response buffer size */
+	/* max_response_len is replaced by res2 after the end-I/O snapshot. */
+	union {
+		u32 max_response_len;	/* maximum sense snapshot size */
+		u64 res2;		/* saved completion result */
+	};
 };
 static_assert(sizeof(struct scsi_bsg_uring_cmd_pdu) <= sizeof_field(struct io_uring_cmd, pdu));
 
@@ -28,39 +32,48 @@ static inline struct scsi_bsg_uring_cmd_pdu *scsi_bsg_uring_cmd_pdu(
 	return io_uring_cmd_to_pdu(ioucmd, struct scsi_bsg_uring_cmd_pdu);
 }
 
-/* Task work: build res2 (layout in uapi/linux/bsg.h) and copy sense to user. */
+static u64 scsi_bsg_res2_clear_sense_len(u64 res2)
+{
+	return bsg_scsi_res2_build(bsg_scsi_res2_device_status(res2),
+				   bsg_scsi_res2_driver_status(res2),
+				   bsg_scsi_res2_host_status(res2), 0,
+				   bsg_scsi_res2_resid_len(res2));
+}
+
+/* Task work: unmap the saved BIO and copy sense to user space. */
 static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
 {
 	struct io_uring_cmd *ioucmd = io_uring_cmd_from_tw(tw_req);
 	struct scsi_bsg_uring_cmd_pdu *pdu = scsi_bsg_uring_cmd_pdu(ioucmd);
-	struct request *rq = pdu->req;
-	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(rq);
-	u64 res2;
+	u64 res2 = pdu->res2;
+	u8 sense_len = bsg_scsi_res2_sense_len(res2);
 	int ret = 0;
-	u8 driver_status = 0;
-	u8 sense_len_wr = 0;
 
 	if (pdu->bio)
 		ret = blk_rq_unmap_user(pdu->bio);
 
-	if (scsi_status_is_check_condition(scmd->result)) {
-		driver_status = DRIVER_SENSE;
-		if (pdu->response_addr)
-			sense_len_wr = min_t(unsigned int, pdu->max_response_len,
-					     scmd->sense_len);
-	}
-
-	if (sense_len_wr) {
-		if (copy_to_user(uptr64(pdu->response_addr), scmd->sense_buffer,
-				 sense_len_wr))
+	/*
+	 * If the ring is being torn down or the submitting task
+	 * has exited, this may run from a kworker or in a different task, so
+	 * don't copy the sense data to the submitter's address.
+	 */
+	if (unlikely(tw.cancel)) {
+		ret = -ECANCELED;
+		sense_len = 0;
+	} else if (sense_len) {
+		if (copy_to_user(uptr64(pdu->response_addr), pdu->sense_buffer,
+				 sense_len)) {
 			ret = -EFAULT;
+			sense_len = 0;
+		}
 	}
 
-	res2 = bsg_scsi_res2_build(status_byte(scmd->result), driver_status,
-				  host_byte(scmd->result), sense_len_wr,
-				  scmd->resid_len);
+	if (!sense_len)
+		res2 = scsi_bsg_res2_clear_sense_len(res2);
+
+	if (pdu->sense_buffer)
+		bsg_uring_cmd_put_completion_buf(ioucmd, pdu->sense_buffer);
 
-	blk_mq_free_request(rq);
 	io_uring_cmd_done32(ioucmd, ret, res2,
 			    IO_URING_CMD_TASK_WORK_ISSUE_FLAGS);
 }
@@ -70,9 +83,38 @@ static enum rq_end_io_ret scsi_bsg_uring_cmd_done(struct request *req,
 						  const struct io_comp_batch *iocb)
 {
 	struct io_uring_cmd *ioucmd = req->end_io_data;
+	struct scsi_bsg_uring_cmd_pdu *pdu = scsi_bsg_uring_cmd_pdu(ioucmd);
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
+	u8 driver_status = 0;
+	u8 sense_len = 0;
+
+	if (scsi_status_is_check_condition(scmd->result)) {
+		driver_status = DRIVER_SENSE;
+		if (pdu->response_addr) {
+			sense_len = min_t(unsigned int, pdu->max_response_len,
+					  scmd->sense_len);
+			if (sense_len)
+				memcpy(pdu->sense_buffer, scmd->sense_buffer,
+				       sense_len);
+		}
+	}
+
+	pdu->res2 = bsg_scsi_res2_build(status_byte(scmd->result),
+					driver_status, host_byte(scmd->result),
+					sense_len, scmd->resid_len);
+
+	/*
+	 * No sense to deliver: return the pool buffer now so other commands
+	 * can use it while this one waits for task work.  Task work only puts
+	 * when sense_buffer is still set.
+	 */
+	if (!sense_len && pdu->sense_buffer) {
+		bsg_uring_cmd_put_completion_buf(ioucmd, pdu->sense_buffer);
+		pdu->sense_buffer = NULL;
+	}
 
 	io_uring_cmd_do_in_task_lazy(ioucmd, scsi_bsg_uring_task_cb);
-	return RQ_END_IO_NONE;
+	return RQ_END_IO_FREE;
 }
 
 static int scsi_bsg_map_user_buffer(struct request *req,
@@ -115,8 +157,14 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 	u32 dout_xfer_len = READ_ONCE(cmd->dout_xfer_len);
 	u64 din_xferp = READ_ONCE(cmd->din_xferp);
 	u32 din_xfer_len = READ_ONCE(cmd->din_xfer_len);
+	u64 response_addr = READ_ONCE(cmd->response);
+	u32 max_response_len = min_t(u32, READ_ONCE(cmd->max_response_len),
+				     SCSI_SENSE_BUFFERSIZE);
 	int ret;
 
+	if (ioucmd->flags & IORING_URING_CMD_MULTISHOT)
+		return -EOPNOTSUPP;
+
 	if (cmd->protocol != BSG_PROTOCOL_SCSI ||
 	    cmd->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD)
 		return -EINVAL;
@@ -160,8 +208,32 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 		goto out_free_req;
 	}
 
-	pdu->response_addr = cmd->response;
-	pdu->max_response_len = cmd->max_response_len;
+	pdu->response_addr = response_addr;
+	pdu->max_response_len = max_response_len;
+
+	/*
+	 * Sense bytes can only be delivered when the caller passed a
+	 * response buffer and a nonzero response length; a command without
+	 * both never snapshots sense, so it needs no pool buffer and cannot
+	 * be rejected for pool exhaustion.
+	 *
+	 * Take the buffer before mapping the user data buffer.
+	 * blk_rq_unmap_user() is not pure cleanup: for a bounce-mapped READ
+	 * it copies the zero-filled bounce pages back into the caller's
+	 * buffer, so a command rejected for pool exhaustion after mapping
+	 * would zero the caller's data before returning -EAGAIN.
+	 */
+	if (response_addr && max_response_len) {
+		pdu->sense_buffer =
+			bsg_uring_cmd_get_completion_buf(ioucmd,
+							 SCSI_SENSE_BUFFERSIZE);
+		if (!pdu->sense_buffer) {
+			ret = -EAGAIN;
+			goto out_free_req;
+		}
+	} else {
+		pdu->sense_buffer = NULL;
+	}
 
 	if (dout_xfer_len || din_xfer_len) {
 		bool is_write = dout_xfer_len > 0;
@@ -172,7 +244,7 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 					       gfp_mask, is_write, buf_addr,
 					       buf_len);
 		if (ret)
-			goto out_free_req;
+			goto out_put_buf;
 		pdu->bio = req->bio;
 	} else {
 		pdu->bio = NULL;
@@ -183,11 +255,13 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 
 	req->end_io = scsi_bsg_uring_cmd_done;
 	req->end_io_data = ioucmd;
-	pdu->req = req;
 
 	blk_execute_rq_nowait(req, false);
 	return -EIOCBQUEUED;
 
+out_put_buf:
+	if (pdu->sense_buffer)
+		bsg_uring_cmd_put_completion_buf(ioucmd, pdu->sense_buffer);
 out_free_req:
 	blk_mq_free_request(req);
 	return ret;
Signature · 1 line
2.25.1
#4 lore raw

PATCH 1/2 block: bsg: add a per-device pool of uring_cmd completion buffers

In reply to Yang Xiuwei

scsi_bsg will need temporary storage for sense data when completed requests are freed in end-I/O instead of task work. Add a per-device buffer pool for that, allocated only when uring_cmd_fn is set.

Drivers take a buffer with bsg_uring_cmd_get_completion_buf() and return it with bsg_uring_cmd_put_completion_buf(). Exhaustion returns NULL (-EAGAIN from the caller). Depth is BSG_DEFAULT_CMDS, buffer size is SCSI_SENSE_BUFFERSIZE.

---
 block/bsg.c         | 127 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/bsg.h |   3 ++
 2 files changed, 130 insertions(+)
block/bsg.c +127 -0
index b296fa373292..96f46715ade9 100644
@@ -12,10 +12,12 @@
 #include <linux/idr.h>
 #include <linux/bsg.h>
 #include <linux/slab.h>
+#include <linux/bitmap.h>
 #include <linux/io_uring/cmd.h>
 
 #include <scsi/scsi.h>
 #include <scsi/scsi_ioctl.h>
+#include <scsi/scsi_device.h>
 #include <scsi/sg.h>
 
 #define BSG_DESCRIPTION	"Block layer SCSI generic (bsg) driver"
@@ -30,6 +32,7 @@ struct bsg_device {
 	unsigned int reserved_size;
 	bsg_sg_io_fn *sg_io_fn;
 	bsg_uring_cmd_fn *uring_cmd_fn;
+	struct bsg_completion_pool *completion_pool;
 };
 
 static inline struct bsg_device *to_bsg_device(struct inode *inode)
@@ -40,6 +43,116 @@ static inline struct bsg_device *to_bsg_device(struct inode *inode)
 #define BSG_DEFAULT_CMDS	64
 #define BSG_MAX_DEVS		(1 << MINORBITS)
 
+/*
+ * Per-device pool of fixed-size completion buffers for uring_cmd drivers that
+ * need data to outlive the block request until task work.  Allocated when the
+ * queue is registered with a uring_cmd handler; freed when the bsg device is
+ * released.  Callers check a buffer out at issue time and return it from task
+ * work.  When the pool is exhausted the caller can reject the command with
+ * -EAGAIN instead of allocating or sleeping in the I/O path.
+ *
+ * The pool has BSG_DEFAULT_CMDS buffers of SCSI_SENSE_BUFFERSIZE (the only
+ * in-tree uring_cmd user is scsi_bsg).
+ */
+struct bsg_completion_pool {
+	spinlock_t lock;		/* protects checked_out */
+	size_t buf_size;		/* size of each buffer */
+	u8 *bufs;			/* BSG_DEFAULT_CMDS buffers */
+	unsigned long *checked_out;	/* bitmap: 1 while borrowed, 0 free */
+};
+
+static struct bsg_completion_pool *bsg_completion_pool_alloc(void)
+{
+	struct bsg_completion_pool *pool;
+
+	pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+	if (!pool)
+		return NULL;
+	pool->bufs = kcalloc(BSG_DEFAULT_CMDS, SCSI_SENSE_BUFFERSIZE,
+			     GFP_KERNEL);
+	pool->checked_out = bitmap_zalloc(BSG_DEFAULT_CMDS, GFP_KERNEL);
+	if (!pool->bufs || !pool->checked_out) {
+		bitmap_free(pool->checked_out);
+		kfree(pool->bufs);
+		kfree(pool);
+		return NULL;
+	}
+	pool->buf_size = SCSI_SENSE_BUFFERSIZE;
+	spin_lock_init(&pool->lock);
+	return pool;
+}
+
+static void bsg_completion_pool_free(struct bsg_completion_pool *pool)
+{
+	if (!pool)
+		return;
+	WARN_ON(!bitmap_empty(pool->checked_out, BSG_DEFAULT_CMDS));
+	bitmap_free(pool->checked_out);
+	kfree(pool->bufs);
+	kfree(pool);
+}
+
+/**
+ * bsg_uring_cmd_get_completion_buf - check out a per-device completion buffer
+ * @ioucmd: the uring command being issued
+ * @len: required buffer size
+ *
+ * Check a buffer out of the pool of the bsg device.  The caller must return
+ * the buffer with bsg_uring_cmd_put_completion_buf() before it completes or
+ * fails the command.
+ *
+ * Return: the buffer, or NULL if the device has no pool, @len exceeds the
+ * buffer size, or the pool is exhausted.
+ */
+void *bsg_uring_cmd_get_completion_buf(struct io_uring_cmd *ioucmd, size_t len)
+{
+	struct bsg_device *bd = to_bsg_device(file_inode(ioucmd->file));
+	struct bsg_completion_pool *pool = bd->completion_pool;
+	unsigned int index;
+	void *buf;
+
+	if (!pool || WARN_ON_ONCE(len > pool->buf_size))
+		return NULL;
+
+	spin_lock(&pool->lock);
+	index = find_first_zero_bit(pool->checked_out, BSG_DEFAULT_CMDS);
+	if (index >= BSG_DEFAULT_CMDS) {
+		spin_unlock(&pool->lock);
+		return NULL;
+	}
+	__set_bit(index, pool->checked_out);
+	buf = pool->bufs + (size_t)index * pool->buf_size;
+	spin_unlock(&pool->lock);
+	return buf;
+}
+EXPORT_SYMBOL_GPL(bsg_uring_cmd_get_completion_buf);
+
+/**
+ * bsg_uring_cmd_put_completion_buf - return a per-device completion buffer
+ * @ioucmd: the uring command being completed
+ * @buf: buffer previously returned by bsg_uring_cmd_get_completion_buf()
+ *
+ * Return the buffer to the pool of the bsg device.  Each buffer must be
+ * returned exactly once.  The pool warns if a buffer is returned while it
+ * is already free.
+ */
+void bsg_uring_cmd_put_completion_buf(struct io_uring_cmd *ioucmd, void *buf)
+{
+	struct bsg_device *bd = to_bsg_device(file_inode(ioucmd->file));
+	struct bsg_completion_pool *pool = bd->completion_pool;
+	unsigned int index;
+
+	if (WARN_ON_ONCE(!buf))
+		return;
+	index = ((u8 *)buf - pool->bufs) / pool->buf_size;
+	if (WARN_ON_ONCE(index >= BSG_DEFAULT_CMDS))
+		return;
+	spin_lock(&pool->lock);
+	WARN_ON_ONCE(!__test_and_clear_bit(index, pool->checked_out));
+	spin_unlock(&pool->lock);
+}
+EXPORT_SYMBOL_GPL(bsg_uring_cmd_put_completion_buf);
+
 static DEFINE_IDA(bsg_minor_ida);
 static const struct class bsg_class;
 static int bsg_major;
@@ -203,6 +316,7 @@ static void bsg_device_release(struct device *dev)
 {
 	struct bsg_device *bd = container_of(dev, struct bsg_device, device);
 
+	bsg_completion_pool_free(bd->completion_pool);
 	ida_free(&bsg_minor_ida, MINOR(bd->device.devt));
 	kfree(bd);
 }
@@ -234,10 +348,23 @@ struct bsg_device *bsg_register_queue(struct request_queue *q,
 	bd->sg_io_fn = sg_io_fn;
 	bd->uring_cmd_fn = uring_cmd_fn;
 
+	/*
+	 * Only queues with a uring_cmd handler need completion buffers.
+	 * Transport bsg devices pass a NULL handler and allocate no pool.
+	 */
+	if (uring_cmd_fn) {
+		bd->completion_pool = bsg_completion_pool_alloc();
+		if (!bd->completion_pool) {
+			kfree(bd);
+			return ERR_PTR(-ENOMEM);
+		}
+	}
+
 	ret = ida_alloc_max(&bsg_minor_ida, BSG_MAX_DEVS - 1, GFP_KERNEL);
 	if (ret < 0) {
 		if (ret == -ENOSPC)
 			dev_err(parent, "bsg: too many bsg devices\n");
+		bsg_completion_pool_free(bd->completion_pool);
 		kfree(bd);
 		return ERR_PTR(ret);
 	}
include/linux/bsg.h +3 -0
index 162730bfc2d8..f5560be80c20 100644
@@ -20,4 +20,7 @@ struct bsg_device *bsg_register_queue(struct request_queue *q,
 		bsg_sg_io_fn *sg_io_fn, bsg_uring_cmd_fn *uring_cmd_fn);
 void bsg_unregister_queue(struct bsg_device *bcd);
 
+void *bsg_uring_cmd_get_completion_buf(struct io_uring_cmd *ioucmd, size_t len);
+void bsg_uring_cmd_put_completion_buf(struct io_uring_cmd *ioucmd, void *buf);
+
 #endif /* _LINUX_BSG_H */
Signature · 1 line
2.25.1
#3 lore raw
In reply to Yingjie Wang
Quoted 2 lines · Yingjie Wang

I chose a per-file pool over a per-queue one so that one opener cannot exhaust the buffers available to other openers ...

Thanks for the rework. I prefer a few tweaks on the pool side, see the two following patches:

1) Per-device pool at bsg_register_queue() when uring_cmd_fn is set, freed with the bsg device. Avoids open() cost for SG_IO-only openers and drops private_data + completion_buf_size. Shared -EAGAIN looks fine here; only response-bearing cmds take a slot.

2) free_top + stack and checked_out overlap. A bitmap alone is enough (find_first_zero_bit / __set_bit / __test_and_clear_bit). depth can just be BSG_DEFAULT_CMDS.

3) In end-I/O, if sense_len is 0, put the buffer immediately. No need to hold a slot until task work on the success path.

Thanks,
Yang Xiuwei

#5 lore raw

Re: [PATCH 1/2] block: bsg: add a per-device pool of uring_cmd completion buffers

In reply to Yang Xiuwei
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [High] block: bsg: IRQ-unsafe pool->lock acquisition in completion buffer helpers --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [High] block: bsg: IRQ-unsafe pool->lock acquisition in completion buffer helpers
--

commit 581e3d94856bdf0c9a6edc480ced1d507eb97743
Author: Yang Xiuwei <[email protected]>

block: bsg: add a per-device pool of uring_cmd completion buffers

This commit adds a per-device pool of fixed-size completion buffers for bsg devices registered with an io_uring command handler, along with helpers to check out and return buffers.

Quoted 4 lines
block/bsg.c +0 -0
index b296fa3732924..96f46715ade9f 100644

[ ... ]

Quoted 11 lines
block/bsg.c +7 -0
@@ -40,6 +43,116 @@ static inline struct bsg_device *to_bsg_device(struct inode *inode)
 #define BSG_DEFAULT_CMDS	64
 #define BSG_MAX_DEVS		(1 << MINORBITS)
 
+/*
+ * Per-device pool of fixed-size completion buffers for uring_cmd drivers that
+ * need data to outlive the block request until task work.  Allocated when the
+ * queue is registered with a uring_cmd handler; freed when the bsg device is
+ * released.  Callers check a buffer out at issue time and return it from task
+ * work.  When the pool is exhausted the caller can reject the command with
+ * -EAGAIN instead of allocating or sleeping in the I/O path.

[ ... ]

Quoted 37 lines
block/bsg.c +37 -0
+void *bsg_uring_cmd_get_completion_buf(struct io_uring_cmd *ioucmd, size_t len)
+{
+	struct bsg_device *bd = to_bsg_device(file_inode(ioucmd->file));
+	struct bsg_completion_pool *pool = bd->completion_pool;
+	unsigned int index;
+	void *buf;
+
+	if (!pool || WARN_ON_ONCE(len > pool->buf_size))
+		return NULL;
+
+	spin_lock(&pool->lock);
+	index = find_first_zero_bit(pool->checked_out, BSG_DEFAULT_CMDS);
+	if (index >= BSG_DEFAULT_CMDS) {
+		spin_unlock(&pool->lock);
+		return NULL;
+	}
+	__set_bit(index, pool->checked_out);
+	buf = pool->bufs + (size_t)index * pool->buf_size;
+	spin_unlock(&pool->lock);
+	return buf;
+}
+EXPORT_SYMBOL_GPL(bsg_uring_cmd_get_completion_buf);
+
+/**
+ * bsg_uring_cmd_put_completion_buf - return a per-device completion buffer
+ * @ioucmd: the uring command being completed
+ * @buf: buffer previously returned by bsg_uring_cmd_get_completion_buf()
+ *
+ * Return the buffer to the pool of the bsg device.  Each buffer must be
+ * returned exactly once.  The pool warns if a buffer is returned while it
+ * is already free.
+ */
+void bsg_uring_cmd_put_completion_buf(struct io_uring_cmd *ioucmd, void *buf)
+{
+	struct bsg_device *bd = to_bsg_device(file_inode(ioucmd->file));
+	struct bsg_completion_pool *pool = bd->completion_pool;
+	unsigned int index;

block/bsg.c static inline struct bsg_device *to_bsg_device(struct inode *inode) · line 153

Quoted 10 lines
block/bsg.c +10 -0
+
+	if (WARN_ON_ONCE(!buf))
+		return;
+	index = ((u8 *)buf - pool->bufs) / pool->buf_size;
+	if (WARN_ON_ONCE(index >= BSG_DEFAULT_CMDS))
+		return;
+	spin_lock(&pool->lock);
+	WARN_ON_ONCE(!__test_and_clear_bit(index, pool->checked_out));
+	spin_unlock(&pool->lock);
+}

[Severity: High]
Can acquiring pool->lock with plain spin_lock() deadlock when a command completes in interrupt context?

In the next patch in this series, commit "scsi: bsg: release io_uring requests from end-I/O", scsi_bsg_uring_cmd_done() runs as the req->end_io callback from hardirq or softirq context and calls bsg_uring_cmd_put_completion_buf() directly when a command completes without sense data:

bsg_uring_cmd_get_completion_buf()
spin_lock(&pool->lock)
<hardirq / softirq>

    scsi_complete()
      ...
        scsi_bsg_uring_cmd_done()
          bsg_uring_cmd_put_completion_buf()
            spin_lock(&pool->lock)  <-- spins forever

Both helpers should use spin_lock_irqsave() and spin_unlock_irqrestore(), and the comment above struct bsg_completion_pool should note that buffers may also be returned from end-I/O context.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#8 lore raw
In reply to Yang Xiuwei
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [High] scsi: bsg: IRQ-unsafe pool->lock in scsi_bsg_uring_cmd_done() --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [High] scsi: bsg: IRQ-unsafe pool->lock in scsi_bsg_uring_cmd_done() --

commit 90faa38e4aa3fb704f223fae5379944821b9aa28
Author: Yang Xiuwei <[email protected]>

scsi: bsg: release io_uring requests from end-I/O

This commit snapshots the SCSI completion status and sense data in the request end-I/O callback using the per-device BSG completion buffer pool and returns RQ_END_IO_FREE so the request tag is released before task work runs.

Quoted 4 lines
drivers/scsi/scsi_bsg.c +0 -0
index e4a65d7a23742..739778244f420 100644

[ ... ]

Quoted 23 lines
drivers/scsi/scsi_bsg.c +19 -0
@@ -70,9 +83,38 @@ static enum rq_end_io_ret scsi_bsg_uring_cmd_done(struct request *req,
 						  const struct io_comp_batch *iocb)
 {
 	struct io_uring_cmd *ioucmd = req->end_io_data;
+	struct scsi_bsg_uring_cmd_pdu *pdu = scsi_bsg_uring_cmd_pdu(ioucmd);
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
+	u8 driver_status = 0;
+	u8 sense_len = 0;
+
+	if (scsi_status_is_check_condition(scmd->result)) {
+		driver_status = DRIVER_SENSE;
+		if (pdu->response_addr) {
+			sense_len = min_t(unsigned int, pdu->max_response_len,
+					  scmd->sense_len);
+			if (sense_len)
+				memcpy(pdu->sense_buffer, scmd->sense_buffer,
+				       sense_len);
+		}
+	}
+
+	pdu->res2 = bsg_scsi_res2_build(status_byte(scmd->result),
+					driver_status, host_byte(scmd->result),
+					sense_len, scmd->resid_len);

drivers/scsi/scsi_bsg.c static enum rq_end_io_ret scsi_bsg_uring_cmd_done(struct request *req, · line 114

Quoted 10 lines
drivers/scsi/scsi_bsg.c +10 -0
+
+	/*
+	 * No sense to deliver: return the pool buffer now so other commands
+	 * can use it while this one waits for task work.  Task work only puts
+	 * when sense_buffer is still set.
+	 */
+	if (!sense_len && pdu->sense_buffer) {
+		bsg_uring_cmd_put_completion_buf(ioucmd, pdu->sense_buffer);
+		pdu->sense_buffer = NULL;
+	}

[Severity: High]
Can calling bsg_uring_cmd_put_completion_buf() from scsi_bsg_uring_cmd_done() self-deadlock on pool->lock?

Both bsg_uring_cmd_get_completion_buf() and bsg_uring_cmd_put_completion_buf() acquire pool->lock with plain spin_lock(). scsi_bsg_uring_cmd() and scsi_bsg_uring_task_cb() call them from process context with interrupts and softirqs enabled, while scsi_bsg_uring_cmd_done() is the req->end_io callback and runs in hardirq or BLOCK_SOFTIRQ context:

scsi_bsg_uring_cmd()
bsg_uring_cmd_get_completion_buf()

    spin_lock(&pool->lock)
    <softirq / hardirq>
      ...
        scsi_bsg_uring_cmd_done()
          bsg_uring_cmd_put_completion_buf()
            spin_lock(&pool->lock)  <-- spins forever

If a command that provided a response buffer completes without check condition on the same CPU while process context holds pool->lock, the CPU deadlocks.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#6 lore raw

Re: [PATCH 1/2] block: bsg: add a per-device pool of uring_cmd completion buffers

In reply to Yang Xiuwei

Hi Yingjie,

Quoted 2 lines · Yingjie Wang

I chose a per-file pool over a per-queue one so that one opener cannot exhaust the buffers available to other openers ...

One more note on that.

A per-device pool is enough here because only commands with a response buffer take a slot, and end-I/O puts it back immediately when sense_len is 0. The common success path does not pin buffers until task work, so one opener is much less likely to hold the whole pool across a DEFER_TASKRUN delay. If the pool is still empty, -EAGAIN is the same contract either way; sharing BSG_DEFAULT_CMDS across openers looks preferable to paying a pool on every open.

Also, pool->lock needs irqsave: early put from end-I/O can run in hardirq/softirq, so plain spin_lock() can self-deadlock with get.

Thanks,
Yang Xiuwei

#9 lore raw
In reply to Yang Xiuwei

Hi Yang,

Thanks for the review, and for writing the patches. That makes the changes easy to follow.

Quoted 4 lines · Yang Xiuwei

1) Per-device pool at bsg_register_queue() when uring_cmd_fn is set, freed with the bsg device. Avoids open() cost for SG_IO-only openers and drops private_data + completion_buf_size. Shared -EAGAIN looks fine here; only response-bearing cmds take a slot.

That works for me. One opener can still take all 64 slots, e.g. with CHECK CONDITION completions on a DEFER_TASKRUN ring that it never reaps, and then everyone else gets -EAGAIN. But bsg nodes are normally only accessible to privileged users, and it's -EAGAIN rather than a hang, so I'd rather accept that than allocate a pool on every open. I'll mention it in the commit message.

Quoted 3 lines

2) free_top + stack and checked_out overlap. A bitmap alone is enough (find_first_zero_bit / __set_bit / __test_and_clear_bit). depth can just be BSG_DEFAULT_CMDS.

Yes, that's simpler, and __test_and_clear_bit() still catches a double put.

Quoted 2 lines

3) In end-I/O, if sense_len is 0, put the buffer immediately. No need to hold a slot until task work on the success path.

Good idea, then a successful command doesn't hold a slot after it completes. As you said in your follow-up, the lock needs irqsave once the put can happen in end-I/O.

I'll fold all three into v5 and rerun the tests, with lockdep enabled for the end-I/O put. Since patch 1 will mostly be your code, I'll add Co-developed-by and keep your Signed-off-by from your posting.

Regards,
Yingjie

#10 lore raw

Hi Yingjie,

Quoted 3 lines · Yingjie Wang

I'll fold all three into v5 and rerun the tests, with lockdep enabled for the end-I/O put. Since patch 1 will mostly be your code, I'll add Co-developed-by and keep your Signed-off-by from your posting.

Sounds good, thanks. A couple more small things for the pool while you are at it, if you like:

1) DECLARE_BITMAP(checked_out, BSG_DEFAULT_CMDS) is enough - no need for a separate bitmap allocation.

2) Depth and slot size are fixed, so you can drop buf_size and just put u8 bufs[] at the end of the struct, with one kzalloc via struct_size().

3) The WARN_ON_ONCE on get(len) and put(!buf / bad index) feel a bit heavy for ordinary failure paths; a quiet NULL / return is fine. Keeping the double-put WARN and the !bitmap_empty() check on free looks right to me.

4) A few of the comments look a bit long and could be trimmed.

Looking forward to v5.

Thanks,
Yang Xiuwei