Skip to contentSkip to conversation

SCSI core patches

PATCH Superseded Bart Van Assche 3 participants 12 messages May 17, 2023 – May 18, 2023

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover SCSI core patches

Hi Martin,

Please consider these SCSI core patches for the next merge window.

Thanks,

Bart.

Changes compared to v2:
- Dropped patch "scsi: core: Update a source code comment".
- In patch "scsi: core: Trace SCSI sense data", changed the format for the sense key and left out a superfluous parenthese.
- In patch "scsi: core: Only kick the requeue list if necessary", moved the blk_mq_kick_requeue_list() from scsi_run_host_queues() into scsi_run_queue().

Changes compared to v1:
- Improved the SCSI tracing patch as requested by Steven Rostedt and Niklas Cassel.
- Added patch "scsi: core: Delay running the queue if the host is blocked".

Bart Van Assche (4):
scsi: core: Use min() instead of open-coding it
scsi: core: Trace SCSI sense data
scsi: core: Only kick the requeue list if necessary
scsi: core: Delay running the queue if the host is blocked

 drivers/scsi/scsi_common.c  |  3 +--
 drivers/scsi/scsi_lib.c     | 15 +++++++++------
 include/trace/events/scsi.h | 21 +++++++++++++++++++--
 3 files changed, 29 insertions(+), 10 deletions(-)
#1 lore raw

PATCH v3 1/4 scsi: core: Use min() instead of open-coding it

Use min() instead of open-coding it in scsi_normalize_sense().

---
 drivers/scsi/scsi_common.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/scsi/scsi_common.c +1 -2
index 6e50e81a8216..24dec80a6253 100644
@@ -176,8 +176,7 @@ bool scsi_normalize_sense(const u8 *sense_buffer, int sb_len,
 		if (sb_len > 2)
 			sshdr->sense_key = (sense_buffer[2] & 0xf);
 		if (sb_len > 7) {
-			sb_len = (sb_len < (sense_buffer[7] + 8)) ?
-					 sb_len : (sense_buffer[7] + 8);
+			sb_len = min(sb_len, sense_buffer[7] + 8);
 			if (sb_len > 12)
 				sshdr->asc = sense_buffer[12];
 			if (sb_len > 13)
#2 lore raw

PATCH v3 2/4 scsi: core: Trace SCSI sense data

In reply to Bart Van Assche

If a command fails, SCSI sense data is essential to determine why it failed. Hence make the sense key, ASC and ASCQ codes available in the ftrace output.

---
 include/trace/events/scsi.h | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)
include/trace/events/scsi.h +19 -2
index a2c7befd451a..512792317577 100644
@@ -269,9 +269,14 @@ DECLARE_EVENT_CLASS(scsi_cmd_done_timeout_template,
 		__field( unsigned int,	prot_sglen )
 		__field( unsigned char,	prot_op )
 		__dynamic_array(unsigned char,	cmnd, cmd->cmd_len)
+		__field( u8, sense_key )
+		__field( u8, asc )
+		__field( u8, ascq )
 	),
 
 	TP_fast_assign(
+		struct scsi_sense_hdr sshdr;
+
 		__entry->host_no	= cmd->device->host->host_no;
 		__entry->channel	= cmd->device->channel;
 		__entry->id		= cmd->device->id;
@@ -285,11 +290,22 @@ DECLARE_EVENT_CLASS(scsi_cmd_done_timeout_template,
 		__entry->prot_sglen	= scsi_prot_sg_count(cmd);
 		__entry->prot_op	= scsi_get_prot_op(cmd);
 		memcpy(__get_dynamic_array(cmnd), cmd->cmnd, cmd->cmd_len);
+		if (cmd->sense_buffer && SCSI_SENSE_VALID(cmd) &&
+		    scsi_command_normalize_sense(cmd, &sshdr)) {
+			__entry->sense_key = sshdr.sense_key;
+			__entry->asc = sshdr.asc;
+			__entry->ascq = sshdr.ascq;
+		} else {
+			__entry->sense_key = 0;
+			__entry->asc = 0;
+			__entry->ascq = 0;
+		}
 	),
 
 	TP_printk("host_no=%u channel=%u id=%u lun=%u data_sgl=%u prot_sgl=%u " \
 		  "prot_op=%s driver_tag=%d scheduler_tag=%d cmnd=(%s %s raw=%s) " \
-		  "result=(driver=%s host=%s message=%s status=%s)",
+		  "result=(driver=%s host=%s message=%s status=%s "
+		  "sense_key=%#x asc=%#x ascq=%#x)",
 		  __entry->host_no, __entry->channel, __entry->id,
 		  __entry->lun, __entry->data_sglen, __entry->prot_sglen,
 		  show_prot_op_name(__entry->prot_op), __entry->driver_tag,
@@ -299,7 +315,8 @@ DECLARE_EVENT_CLASS(scsi_cmd_done_timeout_template,
 		  "DRIVER_OK",
 		  show_hostbyte_name(((__entry->result) >> 16) & 0xff),
 		  "COMMAND_COMPLETE",
-		  show_statusbyte_name(__entry->result & 0xff))
+		  show_statusbyte_name(__entry->result & 0xff),
+		  __entry->sense_key, __entry->asc, __entry->ascq)
 );
 
 DEFINE_EVENT(scsi_cmd_done_timeout_template, scsi_dispatch_cmd_done,
#3 lore raw

Re: [PATCH v3 2/4] scsi: core: Trace SCSI sense data

Quoted 45 lines · Bart Van Assche

If a command fails, SCSI sense data is essential to determine why it failed. Hence make the sense key, ASC and ASCQ codes available in the ftrace output.

---
 include/trace/events/scsi.h | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)
include/trace/events/scsi.h +12 -0
index a2c7befd451a..512792317577 100644
@@ -269,9 +269,14 @@ DECLARE_EVENT_CLASS(scsi_cmd_done_timeout_template,
 		__field( unsigned int,	prot_sglen )
 		__field( unsigned char,	prot_op )
 		__dynamic_array(unsigned char,	cmnd, cmd->cmd_len)
+		__field( u8, sense_key )
+		__field( u8, asc )
+		__field( u8, ascq )
 	),
 
 	TP_fast_assign(
+		struct scsi_sense_hdr sshdr;
+
 		__entry->host_no	= cmd->device->host->host_no;
 		__entry->channel	= cmd->device->channel;
 		__entry->id		= cmd->device->id;
@@ -285,11 +290,22 @@ DECLARE_EVENT_CLASS(scsi_cmd_done_timeout_template,
 		__entry->prot_sglen	= scsi_prot_sg_count(cmd);
 		__entry->prot_op	= scsi_get_prot_op(cmd);
 		memcpy(__get_dynamic_array(cmnd), cmd->cmnd, cmd->cmd_len);
+		if (cmd->sense_buffer && SCSI_SENSE_VALID(cmd) &&
+		    scsi_command_normalize_sense(cmd, &sshdr)) {
+			__entry->sense_key = sshdr.sense_key;
+			__entry->asc = sshdr.asc;
+			__entry->ascq = sshdr.ascq;
+		} else {
+			__entry->sense_key = 0;

include/trace/events/scsi.h DECLARE_EVENT_CLASS(scsi_cmd_done_timeout_template, · line 308

Quoted 10 lines
include/trace/events/scsi.h +5 -1
+			__entry->asc = 0;
+			__entry->ascq = 0;
+		}
 	),
 
 	TP_printk("host_no=%u channel=%u id=%u lun=%u data_sgl=%u prot_sgl=%u " \
 		  "prot_op=%s driver_tag=%d scheduler_tag=%d cmnd=(%s %s raw=%s) " \
-		  "result=(driver=%s host=%s message=%s status=%s)",
+		  "result=(driver=%s host=%s message=%s status=%s "
+		  "sense_key=%#x asc=%#x ascq=%#x)",

This way probably breaks userspace script or utility, maybe you can just append "sense(sense_key=%#x asc=%#x ascq=%#x)" only.

thanks,
Ming

#4 lore raw

Re: [PATCH v3 2/4] scsi: core: Trace SCSI sense data

Quoted 9 lines · Ming Lei

include/trace/events/scsi.h DECLARE_EVENT_CLASS(scsi_cmd_done_timeout_template, · line 308

Quoted 5 lines · Bart Van Assche
  	TP_printk("host_no=%u channel=%u id=%u lun=%u data_sgl=%u prot_sgl=%u " \
  		  "prot_op=%s driver_tag=%d scheduler_tag=%d cmnd=(%s %s raw=%s) " \
include/trace/events/scsi.h +2 -1
-		  "result=(driver=%s host=%s message=%s status=%s)",
+		  "result=(driver=%s host=%s message=%s status=%s "
+		  "sense_key=%#x asc=%#x ascq=%#x)",

This way probably breaks userspace script or utility, maybe you can just append "sense(sense_key=%#x asc=%#x ascq=%#x)" only.

I will make this change.

Thanks,

Bart.

#5 lore raw

PATCH v3 3/4 scsi: core: Only kick the requeue list if necessary

In reply to Bart Van Assche

Instead of running the request queue of each device associated with a host every 3 ms (BLK_MQ_RESOURCE_DELAY) while host error handling is in progress, run the request queue after error handling has finished.

Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc John Garry <[email protected]>
Cc Mike Christie <[email protected]>
---
 drivers/scsi/scsi_lib.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
drivers/scsi/scsi_lib.c +8 -5
index e59eb0cbfc83..e4f34217b879 100644
@@ -122,11 +122,9 @@ static void scsi_mq_requeue_cmd(struct scsi_cmnd *cmd, unsigned long msecs)
 		WARN_ON_ONCE(true);
 	}
 
-	if (msecs) {
-		blk_mq_requeue_request(rq, false);
+	blk_mq_requeue_request(rq, false);
+	if (!scsi_host_in_recovery(cmd->device->host))
 		blk_mq_delay_kick_requeue_list(rq->q, msecs);
-	} else
-		blk_mq_requeue_request(rq, true);
 }
 
 /**
@@ -165,7 +163,8 @@ static void __scsi_queue_insert(struct scsi_cmnd *cmd, int reason, bool unbusy)
 	 */
 	cmd->result = 0;
 
-	blk_mq_requeue_request(scsi_cmd_to_rq(cmd), true);
+	blk_mq_requeue_request(scsi_cmd_to_rq(cmd),
+			       !scsi_host_in_recovery(cmd->device->host));
 }
 
 /**
@@ -449,6 +448,7 @@ static void scsi_run_queue(struct request_queue *q)
 	if (!list_empty(&sdev->host->starved_list))
 		scsi_starved_list_run(sdev->host);
 
+	blk_mq_kick_requeue_list(q);
 	blk_mq_run_hw_queues(q, false);
 }
 
@@ -499,6 +499,9 @@ static void scsi_mq_uninit_cmd(struct scsi_cmnd *cmd)
 
 static void scsi_run_queue_async(struct scsi_device *sdev)
 {
+	if (scsi_host_in_recovery(sdev->host))
+		return;
+
 	if (scsi_target(sdev)->single_lun ||
 	    !list_empty(&sdev->host->starved_list)) {
 		kblockd_schedule_work(&sdev->requeue_work);
#6 lore raw

PATCH v3 4/4 scsi: core: Delay running the queue if the host is blocked

In reply to Bart Van Assche

Tell the block layer to rerun the queue after a delay instead of immediately if the SCSI LLD decided to block the host.

Note: neither scsi_run_host_queues() nor scsi_run_queue() are called from the fast path.

---
 drivers/scsi/scsi_lib.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/scsi_lib.c +1 -1
index e4f34217b879..b37718ebbbfc 100644
@@ -1767,7 +1767,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 		break;
 	case BLK_STS_RESOURCE:
 	case BLK_STS_ZONE_RESOURCE:
-		if (scsi_device_blocked(sdev))
+		if (scsi_device_blocked(sdev) || shost->host_self_blocked)
 			ret = BLK_STS_DEV_RESOURCE;
 		break;
 	case BLK_STS_AGAIN:
#7 lore raw

Re: [PATCH v3 4/4] scsi: core: Delay running the queue if the host is blocked

Quoted 21 lines · Bart Van Assche

Tell the block layer to rerun the queue after a delay instead of immediately if the SCSI LLD decided to block the host.

Note: neither scsi_run_host_queues() nor scsi_run_queue() are called from the fast path.

---
 drivers/scsi/scsi_lib.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/scsi_lib.c +0 -0
index e4f34217b879..b37718ebbbfc 100644

drivers/scsi/scsi_lib.c static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, · line 1771

Quoted 6 lines
drivers/scsi/scsi_lib.c +1 -1
@@ -1767,7 +1767,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 		break;
 	case BLK_STS_RESOURCE:
 	case BLK_STS_ZONE_RESOURCE:
-		if (scsi_device_blocked(sdev))
+		if (scsi_device_blocked(sdev) || shost->host_self_blocked)
 			ret = BLK_STS_DEV_RESOURCE;

What if scsi_unblock_requests() is just called after the above check and before returning to block layer core? Then this request is invisible to scsi_run_host_queues()<-scsi_unblock_requests(), and io hang happens.

Thanks,
Ming

#8 lore raw

Re: [PATCH v3 4/4] scsi: core: Delay running the queue if the host is blocked

Quoted 12 lines · Ming Lei

drivers/scsi/scsi_lib.c static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, · line 1771

Quoted 7 lines · Bart Van Assche
drivers/scsi/scsi_lib.c +1 -1
@@ -1767,7 +1767,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
  		break;
  	case BLK_STS_RESOURCE:
  	case BLK_STS_ZONE_RESOURCE:
-		if (scsi_device_blocked(sdev))
+		if (scsi_device_blocked(sdev) || shost->host_self_blocked)
  			ret = BLK_STS_DEV_RESOURCE;

What if scsi_unblock_requests() is just called after the above check and before returning to block layer core? Then this request is invisible to scsi_run_host_queues()<-scsi_unblock_requests(), and io hang happens.

If returning BLK_STS_DEV_RESOURCE could cause an I/O hang, wouldn't that be a bug in the block layer core? Isn't the block layer core expected to rerun the queue after a delay if a block driver returns BLK_STS_DEV_RESOURCE? See also blk_mq_dispatch_rq_list().

Thanks,

Bart.

#9 lore raw

Re: [PATCH v3 4/4] scsi: core: Delay running the queue if the host is blocked

Quoted 14 lines · Bart Van Assche
Quoted 12 lines · Ming Lei

drivers/scsi/scsi_lib.c static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, · line 1771

Quoted 7 lines · Bart Van Assche
drivers/scsi/scsi_lib.c +1 -1
@@ -1767,7 +1767,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
  		break;
  	case BLK_STS_RESOURCE:
  	case BLK_STS_ZONE_RESOURCE:
-		if (scsi_device_blocked(sdev))
+		if (scsi_device_blocked(sdev) || shost->host_self_blocked)
  			ret = BLK_STS_DEV_RESOURCE;

What if scsi_unblock_requests() is just called after the above check and before returning to block layer core? Then this request is invisible to scsi_run_host_queues()<-scsi_unblock_requests(), and io hang happens.

Quoted 4 lines

If returning BLK_STS_DEV_RESOURCE could cause an I/O hang, wouldn't that be a bug in the block layer core? Isn't the block layer core expected to rerun the queue after a delay if a block driver returns BLK_STS_DEV_RESOURCE? See also blk_mq_dispatch_rq_list().

Please see comment for BLK_STS_DEV_RESOURCE:

/*
* BLK_STS_DEV_RESOURCE is returned from the driver to the block layer if
* device related resources are unavailable, but the driver can guarantee
* that the queue will be rerun in the future once resources become
* available again. This is typically the case for device specific
* resources that are consumed for IO. If the driver fails allocating these
* resources, we know that inflight (or pending) IO will free these
* resource upon completion.

Basically it requires driver to re-run queue.

In reality, it can be full of race, maybe we can just remove BLK_STS_DEV_RESOURCE.

Thanks,
Ming

#10 lore raw

Re: [PATCH v3 4/4] scsi: core: Delay running the queue if the host is blocked

Quoted 2 lines · Ming Lei

In reality, it can be full of race, maybe we can just remove BLK_STS_DEV_RESOURCE.

We need a better solution than changing BLK_STS_DEV_RESOURCE into BLK_STS_RESOURCE. Otherwise power usage will go up in battery-powered devices that use block drivers that return BLK_STS_DEV_RESOURCE today.

Thanks,

Bart.

#11 lore raw

Re: [PATCH v3 4/4] scsi: core: Delay running the queue if the host is blocked

In reply to Bart Van Assche
Quoted 15 lines · Bart Van Assche
Quoted 13 lines · Ming Lei
Quoted 8 lines · Bart Van Assche
(quoted diff) +0 -0
@@ -1767,7 +1767,7 @@ static blk_status_t scsi_queue_rq(struct 
 blk_mq_hw_ctx *hctx,
           break;

      case BLK_STS_RESOURCE:
      case BLK_STS_ZONE_RESOURCE:
-        if (scsi_device_blocked(sdev))
+        if (scsi_device_blocked(sdev) || shost->host_self_blocked) ret = BLK_STS_DEV_RESOURCE;

What if scsi_unblock_requests() is just called after the above check and before returning to block layer core? Then this request is invisible to scsi_run_host_queues()<-scsi_unblock_requests(), and io hang happens.

Quoted 2 lines

If returning BLK_STS_DEV_RESOURCE could cause an I/O hang, wouldn't that be a bug in the block layer core? Isn't the block layer core expected to rerun the queue after a delay if a block driver returns BLK_STS_DEV_RESOURCE? See also blk_mq_dispatch_rq_list().

DEV_RESOURCE is a tricky thing; it actually implies that the _device_ is blocked, and no further I/O is possible. And it's actually the responsibility of the device/driver to kick the queue again once the contention/block/whatever is done. So no, the block layer should not do anything here, relying on the driver to do something.

Which is why returning DEV_RESOURCE is not recommended, seeing that it's easy to get it wrong...

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke                Kernel Storage Architect
[email protected]                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew
Myers, Andrew McDonald, Martje Boudien Moerman