Skip to contentSkip to conversation

scis: don't apply per-LUN queue depth for SSD

PATCH In review Ming Lei 8 participants 34 messages Nov 18, 2019 – Dec 5, 2019

Subsystems (4)
lore
Share & tools
lore

Ming Lei

author
#0 lore raw

cover scis: don't apply per-LUN queue depth for SSD

Hi,

SCSI's per-LUN queue depth is usually for improving IO merge and balancing IO load among LUNs. blk-mq has provides fair driver tag allocation and managed IRQ balances interrupt load among queues, meantime IO merge doesn't play a big role for SSD, and NVMe doesn't apply per-namespace queue depth.

This patchset tries to don't apply per-LUN queue depth for SSD, then we can avoid the expensive atomic operation on sdev->device_busy. We do understand that this shared counter affects IOPS a lot.

The 1st two patches replaces .device_busy with two private counter for megaraid_sas and mpt3sas to track inflight per-LUN commands since the two drivers need the counter for balancing load, even though it is probably no much use.

The last two patches avoid to operate on sdev->device_busy in IO path for SSD.

Ming Lei (4):
scsi: megaraid_sas: use private counter for tracking inflight per-LUN
commands
scsi: mpt3sas: use private counter for tracking inflight per-LUN
commands
scsi: sd: register request queue after sd_revalidate_disk is done
scsi: core: don't limit per-LUN queue depth for SSD

 block/blk-sysfs.c                           | 14 +++++++++++-
 drivers/scsi/megaraid/megaraid_sas.h        |  1 +
 drivers/scsi/megaraid/megaraid_sas_base.c   | 15 +++++++++++--
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 13 +++++++----
 drivers/scsi/mpt3sas/mpt3sas_base.c         |  3 ++-
 drivers/scsi/mpt3sas/mpt3sas_base.h         |  1 +
 drivers/scsi/mpt3sas/mpt3sas_scsih.c        | 15 +++++++++++--
 drivers/scsi/scsi_lib.c                     | 24 +++++++++++++++------
 drivers/scsi/sd.c                           |  8 ++++++-
 9 files changed, 77 insertions(+), 17 deletions(-)
Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
Signature · 1 line
2.20.1
author
#1 lore raw

PATCH 1/4 scsi: megaraid_sas: use private counter for tracking inflight per-LUN commands

Prepare for bypassing sdev->device_busy for improving performance on fast SCSI storage device, so sdev->device_busy can't be relied any more.

megaraid_sas may need one such counter for balancing load among LUNs in some specific setting, so add one private counter for this purpose.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 drivers/scsi/megaraid/megaraid_sas.h        |  1 +
 drivers/scsi/megaraid/megaraid_sas_base.c   | 15 +++++++++++++--
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 13 +++++++++----
 3 files changed, 23 insertions(+), 6 deletions(-)
drivers/scsi/megaraid/megaraid_sas.h +1 -0
index a6e788c02ff4..f9562c02705b 100644
@@ -2025,6 +2025,7 @@ struct MR_PRIV_DEVICE {
 	u8 interface_type;
 	u8 task_abort_tmo;
 	u8 target_reset_tmo;
+	atomic_t active_cmds;
 };
 struct megasas_cmd;
 
drivers/scsi/megaraid/megaraid_sas_base.c +13 -2
index 42cf38c1ea99..3afead3bc96e 100644
@@ -1759,6 +1759,7 @@ megasas_queue_command(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
 {
 	struct megasas_instance *instance;
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
+	int ret;
 
 	instance = (struct megasas_instance *)
 	    scmd->device->host->hostdata;
@@ -1821,7 +1822,11 @@ megasas_queue_command(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
 		goto out_done;
 	}
 
-	return instance->instancet->build_and_issue_cmd(instance, scmd);
+	atomic_inc(&mr_device_priv_data->active_cmds);
+	ret = instance->instancet->build_and_issue_cmd(instance, scmd);
+	if (ret)
+		atomic_dec(&mr_device_priv_data->active_cmds);
+	return ret;
 
  out_done:
 	scmd->scsi_done(scmd);
@@ -2100,6 +2105,7 @@ static int megasas_slave_alloc(struct scsi_device *sdev)
 
 	atomic_set(&mr_device_priv_data->r1_ldio_hint,
 		   instance->r1_ldio_hint_default);
+	atomic_set(&mr_device_priv_data->active_cmds, 0);
 	return 0;
 }
 
@@ -3475,12 +3481,15 @@ megasas_complete_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd,
 	unsigned long flags;
 	struct fusion_context *fusion = instance->ctrl_context;
 	u32 opcode, status;
+	struct MR_PRIV_DEVICE *mr_device_priv_data = NULL;
 
 	/* flag for the retry reset */
 	cmd->retry_for_fw_reset = 0;
 
-	if (cmd->scmd)
+	if (cmd->scmd) {
 		cmd->scmd->SCp.ptr = NULL;
+		mr_device_priv_data = cmd->scmd->device->hostdata;
+	}
 
 	switch (hdr->cmd) {
 	case MFI_CMD_INVALID:
@@ -3519,6 +3528,7 @@ megasas_complete_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd,
 		if (exception) {
 
 			atomic_dec(&instance->fw_outstanding);
+			atomic_dec(&mr_device_priv_data->active_cmds);
 
 			scsi_dma_unmap(cmd->scmd);
 			cmd->scmd->scsi_done(cmd->scmd);
@@ -3567,6 +3577,7 @@ megasas_complete_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd,
 		}
 
 		atomic_dec(&instance->fw_outstanding);
+		atomic_dec(&mr_device_priv_data->active_cmds);
 
 		scsi_dma_unmap(cmd->scmd);
 		cmd->scmd->scsi_done(cmd->scmd);
drivers/scsi/megaraid/megaraid_sas_fusion.c +9 -4
index e301458bcbae..10ed3bc3b643 100644
@@ -2728,7 +2728,7 @@ megasas_build_ldio_fusion(struct megasas_instance *instance,
 	u8 *raidLUN;
 	unsigned long spinlock_flags;
 	struct MR_LD_RAID *raid = NULL;
-	struct MR_PRIV_DEVICE *mrdev_priv;
+	struct MR_PRIV_DEVICE *mrdev_priv = scp->device->hostdata;
 	struct RAID_CONTEXT *rctx;
 	struct RAID_CONTEXT_G35 *rctx_g35;
 
@@ -2826,7 +2826,7 @@ megasas_build_ldio_fusion(struct megasas_instance *instance,
 	}
 
 	if ((instance->perf_mode == MR_BALANCED_PERF_MODE) &&
-		atomic_read(&scp->device->device_busy) >
+		atomic_read(&mrdev_priv->active_cmds) >
 		(io_info.data_arms * MR_DEVICE_HIGH_IOPS_DEPTH))
 		cmd->request_desc->SCSIIO.MSIxIndex =
 			mega_mod64((atomic64_add_return(1, &instance->high_iops_outstanding) /
@@ -2849,7 +2849,6 @@ megasas_build_ldio_fusion(struct megasas_instance *instance,
 		 * with the SLD bit asserted.
 		 */
 		if (io_info.r1_alt_dev_handle != MR_DEVHANDLE_INVALID) {
-			mrdev_priv = scp->device->hostdata;
 
 			if (atomic_inc_return(&instance->fw_outstanding) >
 				(instance->host->can_queue)) {
@@ -3159,7 +3158,7 @@ megasas_build_syspd_fusion(struct megasas_instance *instance,
 	cmd->request_desc->SCSIIO.DevHandle = io_request->DevHandle;
 
 	if ((instance->perf_mode == MR_BALANCED_PERF_MODE) &&
-		atomic_read(&scmd->device->device_busy) > MR_DEVICE_HIGH_IOPS_DEPTH)
+		atomic_read(&mr_device_priv_data->active_cmds) > MR_DEVICE_HIGH_IOPS_DEPTH)
 		cmd->request_desc->SCSIIO.MSIxIndex =
 			mega_mod64((atomic64_add_return(1, &instance->high_iops_outstanding) /
 				MR_HIGH_IOPS_BATCH_COUNT), instance->low_latency_index_start);
@@ -3550,6 +3549,7 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex,
 
 	while (d_val.u.low != cpu_to_le32(UINT_MAX) &&
 	       d_val.u.high != cpu_to_le32(UINT_MAX)) {
+		struct MR_PRIV_DEVICE *mr_device_priv_data = NULL;
 
 		smid = le16_to_cpu(reply_desc->SMID);
 		cmd_fusion = fusion->cmd_list[smid - 1];
@@ -3585,6 +3585,8 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex,
 			}
 			/* Fall through - and complete IO */
 		case MEGASAS_MPI2_FUNCTION_LD_IO_REQUEST: /* LD-IO Path */
+			mr_device_priv_data = scmd_local->device->hostdata;
+			atomic_dec(&mr_device_priv_data->active_cmds);
 			atomic_dec(&instance->fw_outstanding);
 			if (cmd_fusion->r1_alt_dev_handle == MR_DEVHANDLE_INVALID) {
 				map_cmd_status(fusion, scmd_local, status,
@@ -4865,6 +4867,7 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 
 		/* Now return commands back to the OS */
 		for (i = 0 ; i < instance->max_scsi_cmds; i++) {
+			struct MR_PRIV_DEVICE *mr_device_priv_data = NULL;
 			cmd_fusion = fusion->cmd_list[i];
 			/*check for extra commands issued by driver*/
 			if (instance->adapter_type >= VENTURA_SERIES) {
@@ -4893,6 +4896,8 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 				megasas_return_cmd_fusion(instance, cmd_fusion);
 				scsi_dma_unmap(scmd_local);
 				scmd_local->scsi_done(scmd_local);
+				mr_device_priv_data = scmd_local->device->hostdata;
+				atomic_dec(&mr_device_priv_data->active_cmds);
 			}
 		}
 
Signature · 1 line
2.20.1
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] scsi: megaraid_sas: use private counter for tracking inflight per-LUN commands

Quoted 18 lines · Ming Lei

Prepare for bypassing sdev->device_busy for improving performance on fast SCSI storage device, so sdev->device_busy can't be relied any more.

megaraid_sas may need one such counter for balancing load among LUNs in some specific setting, so add one private counter for this purpose.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
Quoted 6 lines
---
 drivers/scsi/megaraid/megaraid_sas.h        |  1 +
 drivers/scsi/megaraid/megaraid_sas_base.c   | 15 +++++++++++++--
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 13 +++++++++----
 3 files changed, 23 insertions(+), 6 deletions(-)

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke		      Teamlead Storage & Networking
[email protected]			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 247165 (AG München), GF: Felix Imendörffer
maintainer
#3 lore raw

RE: [PATCH 1/4] scsi: megaraid_sas: use private counter for tracking inflight per-LUN commands

Quoted 6 lines
Quoted 5 lines
 drivers/scsi/megaraid/megaraid_sas.h        |  1 +
 drivers/scsi/megaraid/megaraid_sas_base.c   | 15 +++++++++++++--
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 13 +++++++++----
 3 files changed, 23 insertions(+), 6 deletions(-)

Ming - Sorry for delay. I will update this Patch. We prefer driver to avoid counter for per sdev if possible. We are currently testing driver using below changes.

inline unsigned long sdev_nr_inflight_request(struct request_queue *q) {
    struct blk_mq_hw_ctx *hctx = q->queue_hw_ctx[0]

    return atomic_read(&hctx->nr_active);
}
Quoted 9 lines

Cheers,

Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
[email protected] +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 247165 (AG München), GF: Felix Imendörffer

author
#4 lore raw

Re: [PATCH 1/4] scsi: megaraid_sas: use private counter for tracking inflight per-LUN commands

Quoted 11 lines · Kashyap Desai
Quoted 6 lines
Quoted 5 lines
 drivers/scsi/megaraid/megaraid_sas.h        |  1 +
 drivers/scsi/megaraid/megaraid_sas_base.c   | 15 +++++++++++++--
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 13 +++++++++----
 3 files changed, 23 insertions(+), 6 deletions(-)

Ming - Sorry for delay. I will update this Patch. We prefer driver to avoid counter for per sdev if possible. We are currently testing driver using below changes.

Quoted 5 lines
inline unsigned long sdev_nr_inflight_request(struct request_queue *q) {
    struct blk_mq_hw_ctx *hctx = q->queue_hw_ctx[0]

    return atomic_read(&hctx->nr_active);
}

OK, I am fine with this way, given it is just used for balancing irq load.

Thanks,
Ming

maintainer
#5 lore raw

RE: [PATCH 1/4] scsi: megaraid_sas: use private counter for tracking inflight per-LUN commands

Quoted 5 lines
Quoted 5 lines

Ming - Sorry for delay. I will update this Patch. We prefer driver to avoid counter for per sdev if possible. We are currently testing driver using below changes.

inline unsigned long sdev_nr_inflight_request(struct request_queue *q)

{

Quoted 5 lines
Quoted 3 lines
    struct blk_mq_hw_ctx *hctx = q->queue_hw_ctx[0]

    return atomic_read(&hctx->nr_active); }

OK, I am fine with this way, given it is just used for balancing irq

load.

We have removed sdev->device_busy check from megaraid_sas and mpt3sas drive. I am able to get required performance 3.0M IOPs on Aero controller using <hctx->nr_active>
Since this is not urgent, we will wait for some testing done by Broadcom. We will post this specific change as Driver update.

Quoted 4 lines

Thanks,
Ming

author
#6 lore raw

PATCH 2/4 scsi: mpt3sas: use private counter for tracking inflight per-LUN commands

In reply to Ming Lei

Prepare for bypassing sdev->device_busy for improving performance on fast SCSI storage device, so sdev->device_busy can't be relied any more.

mpt3sas may need one such counter for balancing load among LUNs in some specific setting, so add one private counter for this purpose.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 drivers/scsi/mpt3sas/mpt3sas_base.c  |  3 ++-
 drivers/scsi/mpt3sas/mpt3sas_base.h  |  1 +
 drivers/scsi/mpt3sas/mpt3sas_scsih.c | 15 +++++++++++++--
 3 files changed, 16 insertions(+), 3 deletions(-)
drivers/scsi/mpt3sas/mpt3sas_base.c +2 -1
index fea3cb6a090b..9c2d374b3157 100644
@@ -3480,12 +3480,13 @@ static inline u8
 _base_get_high_iops_msix_index(struct MPT3SAS_ADAPTER *ioc,
 	struct scsi_cmnd *scmd)
 {
+	struct MPT3SAS_DEVICE *sas_device_priv_data = scmd->device->hostdata;
 	/**
 	 * Round robin the IO interrupts among the high iops
 	 * reply queues in terms of batch count 16 when outstanding
 	 * IOs on the target device is >=8.
 	 */
-	if (atomic_read(&scmd->device->device_busy) >
+	if (atomic_read(&sas_device_priv_data->active_cmds) >
 	    MPT3SAS_DEVICE_HIGH_IOPS_DEPTH)
 		return base_mod64((
 		    atomic64_add_return(1, &ioc->high_iops_outstanding) /
drivers/scsi/mpt3sas/mpt3sas_base.h +1 -0
index faca0a5e71f8..9e9f319bb636 100644
@@ -467,6 +467,7 @@ struct MPT3SAS_DEVICE {
 	 * thing while a SATL command is pending.
 	 */
 	unsigned long ata_command_pending;
+	atomic_t active_cmds;
 
 };
 
drivers/scsi/mpt3sas/mpt3sas_scsih.c +13 -2
index c8e512ba6d39..194960dae1d6 100644
@@ -1770,6 +1770,7 @@ scsih_slave_alloc(struct scsi_device *sdev)
 
 	sas_device_priv_data->lun = sdev->lun;
 	sas_device_priv_data->flags = MPT_DEVICE_FLAGS_INIT;
+	atomic_set(&sas_device_priv_data->active_cmds, 0);
 
 	starget = scsi_target(sdev);
 	sas_target_priv_data = starget->hostdata;
@@ -2884,6 +2885,7 @@ scsih_abort(struct scsi_cmnd *scmd)
 	    ioc->remove_host) {
 		sdev_printk(KERN_INFO, scmd->device,
 			"device been deleted! scmd(%p)\n", scmd);
+		atomic_dec(&sas_device_priv_data->active_cmds);
 		scmd->result = DID_NO_CONNECT << 16;
 		scmd->scsi_done(scmd);
 		r = SUCCESS;
@@ -2993,7 +2995,7 @@ scsih_dev_reset(struct scsi_cmnd *scmd)
 		MPI2_SCSITASKMGMT_TASKTYPE_LOGICAL_UNIT_RESET, 0, 0,
 		tr_timeout, tr_method);
 	/* Check for busy commands after reset */
-	if (r == SUCCESS && atomic_read(&scmd->device->device_busy))
+	if (r == SUCCESS && atomic_read(&sas_device_priv_data->active_cmds))
 		r = FAILED;
  out:
 	sdev_printk(KERN_INFO, scmd->device, "device reset: %s scmd(%p)\n",
@@ -4517,9 +4519,12 @@ _scsih_flush_running_cmds(struct MPT3SAS_ADAPTER *ioc)
 	int count = 0;
 
 	for (smid = 1; smid <= ioc->scsiio_depth; smid++) {
+		struct MPT3SAS_DEVICE *sas_device_priv_data;
+
 		scmd = mpt3sas_scsih_scsi_lookup_get(ioc, smid);
 		if (!scmd)
 			continue;
+		sas_device_priv_data = scmd->device->hostdata;
 		count++;
 		_scsih_set_satl_pending(scmd, false);
 		st = scsi_cmd_priv(scmd);
@@ -4529,6 +4534,7 @@ _scsih_flush_running_cmds(struct MPT3SAS_ADAPTER *ioc)
 			scmd->result = DID_NO_CONNECT << 16;
 		else
 			scmd->result = DID_RESET << 16;
+		atomic_dec(&sas_device_priv_data->active_cmds);
 		scmd->scsi_done(scmd);
 	}
 	dtmprintk(ioc, ioc_info(ioc, "completing %d cmds\n", count));
@@ -4756,11 +4762,14 @@ scsih_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
 	    mpi_request->LUN);
 	memcpy(mpi_request->CDB.CDB32, scmd->cmnd, scmd->cmd_len);
 
+	atomic_inc(&sas_device_priv_data->active_cmds);
+
 	if (mpi_request->DataLength) {
 		pcie_device = sas_target_priv_data->pcie_dev;
 		if (ioc->build_sg_scmd(ioc, scmd, smid, pcie_device)) {
 			mpt3sas_base_free_smid(ioc, smid);
 			_scsih_set_satl_pending(scmd, false);
+			atomic_dec(&sas_device_priv_data->active_cmds);
 			goto out;
 		}
 	} else
@@ -5207,6 +5216,7 @@ _scsih_smart_predicted_fault(struct MPT3SAS_ADAPTER *ioc, u16 handle)
 static u8
 _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 {
+	struct MPT3SAS_DEVICE *sas_device_priv_data;
 	Mpi25SCSIIORequest_t *mpi_request;
 	Mpi2SCSIIOReply_t *mpi_reply;
 	struct scsi_cmnd *scmd;
@@ -5216,7 +5226,6 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 	u8 scsi_state;
 	u8 scsi_status;
 	u32 log_info;
-	struct MPT3SAS_DEVICE *sas_device_priv_data;
 	u32 response_code = 0;
 
 	mpi_reply = mpt3sas_base_get_reply_virt_addr(ioc, reply);
@@ -5225,6 +5234,7 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 	if (scmd == NULL)
 		return 1;
 
+	sas_device_priv_data = scmd->device->hostdata;
 	_scsih_set_satl_pending(scmd, false);
 
 	mpi_request = mpt3sas_base_get_msg_frame(ioc, smid);
@@ -5431,6 +5441,7 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 
 	scsi_dma_unmap(scmd);
 	mpt3sas_base_free_smid(ioc, smid);
+	atomic_dec(&sas_device_priv_data->active_cmds);
 	scmd->scsi_done(scmd);
 	return 0;
 }
Signature · 1 line
2.20.1
Reviewed-by on v1 2/4
#7 lore raw

Re: [PATCH 2/4] scsi: mpt3sas: use private counter for tracking inflight per-LUN commands

Quoted 129 lines · Ming Lei

Prepare for bypassing sdev->device_busy for improving performance on fast SCSI storage device, so sdev->device_busy can't be relied any more.

mpt3sas may need one such counter for balancing load among LUNs in some specific setting, so add one private counter for this purpose.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 drivers/scsi/mpt3sas/mpt3sas_base.c  |  3 ++-
 drivers/scsi/mpt3sas/mpt3sas_base.h  |  1 +
 drivers/scsi/mpt3sas/mpt3sas_scsih.c | 15 +++++++++++++--
 3 files changed, 16 insertions(+), 3 deletions(-)
drivers/scsi/mpt3sas/mpt3sas_base.c +2 -1
index fea3cb6a090b..9c2d374b3157 100644
@@ -3480,12 +3480,13 @@ static inline u8
 _base_get_high_iops_msix_index(struct MPT3SAS_ADAPTER *ioc,
 	struct scsi_cmnd *scmd)
 {
+	struct MPT3SAS_DEVICE *sas_device_priv_data = scmd->device->hostdata;
 	/**
 	 * Round robin the IO interrupts among the high iops
 	 * reply queues in terms of batch count 16 when outstanding
 	 * IOs on the target device is >=8.
 	 */
-	if (atomic_read(&scmd->device->device_busy) >
+	if (atomic_read(&sas_device_priv_data->active_cmds) >
 	    MPT3SAS_DEVICE_HIGH_IOPS_DEPTH)
 		return base_mod64((
 		    atomic64_add_return(1, &ioc->high_iops_outstanding) /
drivers/scsi/mpt3sas/mpt3sas_base.h +1 -0
index faca0a5e71f8..9e9f319bb636 100644
@@ -467,6 +467,7 @@ struct MPT3SAS_DEVICE {
 	 * thing while a SATL command is pending.
 	 */
 	unsigned long ata_command_pending;
+	atomic_t active_cmds;
 
 };
 
drivers/scsi/mpt3sas/mpt3sas_scsih.c +11 -1
index c8e512ba6d39..194960dae1d6 100644
@@ -1770,6 +1770,7 @@ scsih_slave_alloc(struct scsi_device *sdev)
 
 	sas_device_priv_data->lun = sdev->lun;
 	sas_device_priv_data->flags = MPT_DEVICE_FLAGS_INIT;
+	atomic_set(&sas_device_priv_data->active_cmds, 0);
 
 	starget = scsi_target(sdev);
 	sas_target_priv_data = starget->hostdata;
@@ -2884,6 +2885,7 @@ scsih_abort(struct scsi_cmnd *scmd)
 	    ioc->remove_host) {
 		sdev_printk(KERN_INFO, scmd->device,
 			"device been deleted! scmd(%p)\n", scmd);
+		atomic_dec(&sas_device_priv_data->active_cmds);
 		scmd->result = DID_NO_CONNECT << 16;
 		scmd->scsi_done(scmd);
 		r = SUCCESS;
@@ -2993,7 +2995,7 @@ scsih_dev_reset(struct scsi_cmnd *scmd)
 		MPI2_SCSITASKMGMT_TASKTYPE_LOGICAL_UNIT_RESET, 0, 0,
 		tr_timeout, tr_method);
 	/* Check for busy commands after reset */
-	if (r == SUCCESS && atomic_read(&scmd->device->device_busy))
+	if (r == SUCCESS && atomic_read(&sas_device_priv_data->active_cmds))
 		r = FAILED;
  out:
 	sdev_printk(KERN_INFO, scmd->device, "device reset: %s scmd(%p)\n",
@@ -4517,9 +4519,12 @@ _scsih_flush_running_cmds(struct MPT3SAS_ADAPTER *ioc)
 	int count = 0;
 
 	for (smid = 1; smid <= ioc->scsiio_depth; smid++) {
+		struct MPT3SAS_DEVICE *sas_device_priv_data;
+
 		scmd = mpt3sas_scsih_scsi_lookup_get(ioc, smid);
 		if (!scmd)
 			continue;
+		sas_device_priv_data = scmd->device->hostdata;
 		count++;
 		_scsih_set_satl_pending(scmd, false);
 		st = scsi_cmd_priv(scmd);
@@ -4529,6 +4534,7 @@ _scsih_flush_running_cmds(struct MPT3SAS_ADAPTER *ioc)
 			scmd->result = DID_NO_CONNECT << 16;
 		else
 			scmd->result = DID_RESET << 16;
+		atomic_dec(&sas_device_priv_data->active_cmds);
 		scmd->scsi_done(scmd);
 	}
 	dtmprintk(ioc, ioc_info(ioc, "completing %d cmds\n", count));
@@ -4756,11 +4762,14 @@ scsih_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
 	    mpi_request->LUN);
 	memcpy(mpi_request->CDB.CDB32, scmd->cmnd, scmd->cmd_len);
 
+	atomic_inc(&sas_device_priv_data->active_cmds);
+
 	if (mpi_request->DataLength) {
 		pcie_device = sas_target_priv_data->pcie_dev;
 		if (ioc->build_sg_scmd(ioc, scmd, smid, pcie_device)) {
 			mpt3sas_base_free_smid(ioc, smid);
 			_scsih_set_satl_pending(scmd, false);
+			atomic_dec(&sas_device_priv_data->active_cmds);
 			goto out;
 		}
 	} else
@@ -5207,6 +5216,7 @@ _scsih_smart_predicted_fault(struct MPT3SAS_ADAPTER *ioc, u16 handle)
 static u8
 _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 {
+	struct MPT3SAS_DEVICE *sas_device_priv_data;
 	Mpi25SCSIIORequest_t *mpi_request;
 	Mpi2SCSIIOReply_t *mpi_reply;
 	struct scsi_cmnd *scmd;

drivers/scsi/mpt3sas/mpt3sas_scsih.c _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) · line 5231

Quoted 7 lines
drivers/scsi/mpt3sas/mpt3sas_scsih.c +0 -1
@@ -5216,7 +5226,6 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 	u8 scsi_state;
 	u8 scsi_status;
 	u32 log_info;
-	struct MPT3SAS_DEVICE *sas_device_priv_data;
 	u32 response_code = 0;
 
 	mpi_reply = mpt3sas_base_get_reply_virt_addr(ioc, reply);

The above two modifications are probably not needed ...

Quoted 10 lines
drivers/scsi/mpt3sas/mpt3sas_scsih.c +1 -0
@@ -5225,6 +5234,7 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 	if (scmd == NULL)
 		return 1;
 
+	sas_device_priv_data = scmd->device->hostdata;
 	_scsih_set_satl_pending(scmd, false);
 
 	mpi_request = mpt3sas_base_get_msg_frame(ioc, smid);

drivers/scsi/mpt3sas/mpt3sas_scsih.c _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) · line 5447

Quoted 7 lines
drivers/scsi/mpt3sas/mpt3sas_scsih.c +1 -0
@@ -5431,6 +5441,7 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply)
 
 	scsi_dma_unmap(scmd);
 	mpt3sas_base_free_smid(ioc, smid);
+	atomic_dec(&sas_device_priv_data->active_cmds);
 	scmd->scsi_done(scmd);
 	return 0;
 }

Otherwise:

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke		      Teamlead Storage & Networking
[email protected]			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 247165 (AG München), GF: Felix Imendörffer
author
#8 lore raw

PATCH 3/4 scsi: sd: register request queue after sd_revalidate_disk is done

In reply to Ming Lei

Prepare for improving SSD performance in the following patch, which needs to read queue flag of QUEUE_FLAG_NONROT in IO path. So we have to freeze queue before changing this flag in sd_revalidate_disk().

However, queue freezing becomes quite slow after the queue is registered because RCU period is involved.

So delay registering queue after sd_revalidate_disk() is done for avoiding slow queue freezing which will be added to sd_revalidate_disk() in the following patch.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 drivers/scsi/sd.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/scsi/sd.c +3 -1
index 03163ac5fe95..0744c34468e1 100644
@@ -3368,12 +3368,14 @@ static int sd_probe(struct device *dev)
 	}
 
 	blk_pm_runtime_init(sdp->request_queue, dev);
-	device_add_disk(dev, gd, NULL);
+	device_add_disk_no_queue_reg(dev, gd);
 	if (sdkp->capacity)
 		sd_dif_config_host(sdkp);
 
 	sd_revalidate_disk(gd);
 
+	blk_register_queue(gd);
+
 	if (sdkp->security) {
 		sdkp->opal_dev = init_opal_dev(sdp, &sd_sec_submit);
 		if (sdkp->opal_dev)
Signature · 1 line
2.20.1
Reviewed-by on v1 3/4
#9 lore raw

Re: [PATCH 3/4] scsi: sd: register request queue after sd_revalidate_disk is done

Quoted 37 lines · Ming Lei

Prepare for improving SSD performance in the following patch, which needs to read queue flag of QUEUE_FLAG_NONROT in IO path. So we have to freeze queue before changing this flag in sd_revalidate_disk().

However, queue freezing becomes quite slow after the queue is registered because RCU period is involved.

So delay registering queue after sd_revalidate_disk() is done for avoiding slow queue freezing which will be added to sd_revalidate_disk() in the following patch.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 drivers/scsi/sd.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/scsi/sd.c +1 -1
index 03163ac5fe95..0744c34468e1 100644
@@ -3368,12 +3368,14 @@ static int sd_probe(struct device *dev)
 	}
 
 	blk_pm_runtime_init(sdp->request_queue, dev);
-	device_add_disk(dev, gd, NULL);
+	device_add_disk_no_queue_reg(dev, gd);

drivers/scsi/sd.c static int sd_probe(struct device *dev) · line 3381

Quoted 10 lines
drivers/scsi/sd.c +2 -0
 	if (sdkp->capacity)
 		sd_dif_config_host(sdkp);
 
 	sd_revalidate_disk(gd);
 
+	blk_register_queue(gd);
+
 	if (sdkp->security) {
 		sdkp->opal_dev = init_opal_dev(sdp, &sd_sec_submit);
 		if (sdkp->opal_dev)

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke		      Teamlead Storage & Networking
[email protected]			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 247165 (AG München), GF: Felix Imendörffer
author
#10 lore raw

PATCH 4/4 scsi: core: don't limit per-LUN queue depth for SSD

In reply to Ming Lei

SCSI core uses the atomic variable of sdev->device_busy to track in-flight IO requests dispatched to this scsi device. IO request may be submitted from any CPU, so the cost for maintaining the shared atomic counter can be very big on big NUMA machine with lots of CPU cores.

sdev->queue_depth is usually used for two purposes: 1) improve IO merge;
2) fair IO request scattered among all LUNs.

blk-mq already provides fair request allocation among all active shared request queues(LUNs), see hctx_may_queue().

NVMe doesn't have such per-request-queue(namespace) queue depth, so it is reasonable to ignore the limit for SCSI SSD too. Also IO merge won't play big role for reaching top SSD performance.

With this patch, big cost for tracking in-flight per-LUN requests via atomic variable can be saved.

Given QUEUE_FLAG_NONROT is read in IO path, we have to freeze queue before changing this flag.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-sysfs.c       | 14 +++++++++++++-
 drivers/scsi/scsi_lib.c | 24 ++++++++++++++++++------
 drivers/scsi/sd.c       |  4 ++++
 3 files changed, 35 insertions(+), 7 deletions(-)
block/blk-sysfs.c +13 -1
index fca9b158f4a0..9cc0dd5f88a1 100644
@@ -281,6 +281,18 @@ QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0);
 QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0);
 #undef QUEUE_SYSFS_BIT_FNS
 
+static ssize_t queue_store_nonrot_freeze(struct request_queue *q,
+		const char *page, size_t count)
+{
+	size_t ret;
+
+	blk_mq_freeze_queue(q);
+	ret = queue_store_nonrot(q, page, count);
+	blk_mq_unfreeze_queue(q);
+
+	return ret;
+}
+
 static ssize_t queue_zoned_show(struct request_queue *q, char *page)
 {
 	switch (blk_queue_zoned_model(q)) {
@@ -642,7 +654,7 @@ static struct queue_sysfs_entry queue_write_zeroes_max_entry = {
 static struct queue_sysfs_entry queue_nonrot_entry = {
 	.attr = {.name = "rotational", .mode = 0644 },
 	.show = queue_show_nonrot,
-	.store = queue_store_nonrot,
+	.store = queue_store_nonrot_freeze,
 };
 
 static struct queue_sysfs_entry queue_zoned_entry = {
drivers/scsi/scsi_lib.c +18 -6
index 62a86a82c38d..72655a049efd 100644
@@ -354,7 +354,8 @@ void scsi_device_unbusy(struct scsi_device *sdev, struct scsi_cmnd *cmd)
 	if (starget->can_queue > 0)
 		atomic_dec(&starget->target_busy);
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static void scsi_kick_queue(struct request_queue *q)
@@ -410,7 +411,8 @@ static void scsi_single_lun_run(struct scsi_device *current_sdev)
 
 static inline bool scsi_device_is_busy(struct scsi_device *sdev)
 {
-	if (atomic_read(&sdev->device_busy) >= sdev->queue_depth)
+	if (!blk_queue_nonrot(sdev->request_queue) &&
+			atomic_read(&sdev->device_busy) >= sdev->queue_depth)
 		return true;
 	if (atomic_read(&sdev->device_blocked) > 0)
 		return true;
@@ -1283,8 +1285,12 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				  struct scsi_device *sdev)
 {
 	unsigned int busy;
+	bool bypass = blk_queue_nonrot(sdev->request_queue);
 
-	busy = atomic_inc_return(&sdev->device_busy) - 1;
+	if (!bypass)
+		busy = atomic_inc_return(&sdev->device_busy) - 1;
+	else
+		busy = 0;
 	if (atomic_read(&sdev->device_blocked)) {
 		if (busy)
 			goto out_dec;
@@ -1298,12 +1304,16 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				   "unblocking device at zero depth\n"));
 	}
 
+	if (bypass)
+		return 1;
+
 	if (busy >= sdev->queue_depth)
 		goto out_dec;
 
 	return 1;
 out_dec:
-	atomic_dec(&sdev->device_busy);
+	if (!bypass)
+		atomic_dec(&sdev->device_busy);
 	return 0;
 }
 
@@ -1624,7 +1634,8 @@ static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx)
 	struct request_queue *q = hctx->queue;
 	struct scsi_device *sdev = q->queuedata;
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static bool scsi_mq_get_budget(struct blk_mq_hw_ctx *hctx)
@@ -1731,7 +1742,8 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 	case BLK_STS_OK:
 		break;
 	case BLK_STS_RESOURCE:
-		if (atomic_read(&sdev->device_busy) ||
+		if ((!blk_queue_nonrot(sdev->request_queue) &&
+		     atomic_read(&sdev->device_busy)) ||
 		    scsi_device_blocked(sdev))
 			ret = BLK_STS_DEV_RESOURCE;
 		break;
drivers/scsi/sd.c +4 -0
index 0744c34468e1..c3d47117700d 100644
@@ -2927,7 +2927,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 	rot = get_unaligned_be16(&buffer[4]);
 
 	if (rot == 1) {
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
 	}
 
@@ -3123,7 +3125,9 @@ static int sd_revalidate_disk(struct gendisk *disk)
 		 * cause this to be updated correctly and any device which
 		 * doesn't support it should be treated as rotational.
 		 */
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
 
 		if (scsi_device_supports_vpd(sdp)) {
Signature · 1 line
2.20.1
#11 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 162 lines · Ming Lei

SCSI core uses the atomic variable of sdev->device_busy to track in-flight IO requests dispatched to this scsi device. IO request may be submitted from any CPU, so the cost for maintaining the shared atomic counter can be very big on big NUMA machine with lots of CPU cores.

sdev->queue_depth is usually used for two purposes: 1) improve IO merge;
2) fair IO request scattered among all LUNs.

blk-mq already provides fair request allocation among all active shared request queues(LUNs), see hctx_may_queue().

NVMe doesn't have such per-request-queue(namespace) queue depth, so it is reasonable to ignore the limit for SCSI SSD too. Also IO merge won't play big role for reaching top SSD performance.

With this patch, big cost for tracking in-flight per-LUN requests via atomic variable can be saved.

Given QUEUE_FLAG_NONROT is read in IO path, we have to freeze queue before changing this flag.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-sysfs.c       | 14 +++++++++++++-
 drivers/scsi/scsi_lib.c | 24 ++++++++++++++++++------
 drivers/scsi/sd.c       |  4 ++++
 3 files changed, 35 insertions(+), 7 deletions(-)
block/blk-sysfs.c +13 -1
index fca9b158f4a0..9cc0dd5f88a1 100644
@@ -281,6 +281,18 @@ QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0);
 QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0);
 #undef QUEUE_SYSFS_BIT_FNS
 
+static ssize_t queue_store_nonrot_freeze(struct request_queue *q,
+		const char *page, size_t count)
+{
+	size_t ret;
+
+	blk_mq_freeze_queue(q);
+	ret = queue_store_nonrot(q, page, count);
+	blk_mq_unfreeze_queue(q);
+
+	return ret;
+}
+
 static ssize_t queue_zoned_show(struct request_queue *q, char *page)
 {
 	switch (blk_queue_zoned_model(q)) {
@@ -642,7 +654,7 @@ static struct queue_sysfs_entry queue_write_zeroes_max_entry = {
 static struct queue_sysfs_entry queue_nonrot_entry = {
 	.attr = {.name = "rotational", .mode = 0644 },
 	.show = queue_show_nonrot,
-	.store = queue_store_nonrot,
+	.store = queue_store_nonrot_freeze,
 };
 
 static struct queue_sysfs_entry queue_zoned_entry = {
drivers/scsi/scsi_lib.c +18 -6
index 62a86a82c38d..72655a049efd 100644
@@ -354,7 +354,8 @@ void scsi_device_unbusy(struct scsi_device *sdev, struct scsi_cmnd *cmd)
 	if (starget->can_queue > 0)
 		atomic_dec(&starget->target_busy);
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static void scsi_kick_queue(struct request_queue *q)
@@ -410,7 +411,8 @@ static void scsi_single_lun_run(struct scsi_device *current_sdev)
 
 static inline bool scsi_device_is_busy(struct scsi_device *sdev)
 {
-	if (atomic_read(&sdev->device_busy) >= sdev->queue_depth)
+	if (!blk_queue_nonrot(sdev->request_queue) &&
+			atomic_read(&sdev->device_busy) >= sdev->queue_depth)
 		return true;
 	if (atomic_read(&sdev->device_blocked) > 0)
 		return true;
@@ -1283,8 +1285,12 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				  struct scsi_device *sdev)
 {
 	unsigned int busy;
+	bool bypass = blk_queue_nonrot(sdev->request_queue);
 
-	busy = atomic_inc_return(&sdev->device_busy) - 1;
+	if (!bypass)
+		busy = atomic_inc_return(&sdev->device_busy) - 1;
+	else
+		busy = 0;
 	if (atomic_read(&sdev->device_blocked)) {
 		if (busy)
 			goto out_dec;
@@ -1298,12 +1304,16 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				   "unblocking device at zero depth\n"));
 	}
 
+	if (bypass)
+		return 1;
+
 	if (busy >= sdev->queue_depth)
 		goto out_dec;
 
 	return 1;
 out_dec:
-	atomic_dec(&sdev->device_busy);
+	if (!bypass)
+		atomic_dec(&sdev->device_busy);
 	return 0;
 }
 
@@ -1624,7 +1634,8 @@ static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx)
 	struct request_queue *q = hctx->queue;
 	struct scsi_device *sdev = q->queuedata;
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static bool scsi_mq_get_budget(struct blk_mq_hw_ctx *hctx)
@@ -1731,7 +1742,8 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 	case BLK_STS_OK:
 		break;
 	case BLK_STS_RESOURCE:
-		if (atomic_read(&sdev->device_busy) ||
+		if ((!blk_queue_nonrot(sdev->request_queue) &&
+		     atomic_read(&sdev->device_busy)) ||
 		    scsi_device_blocked(sdev))
 			ret = BLK_STS_DEV_RESOURCE;
 		break;
drivers/scsi/sd.c +2 -0
index 0744c34468e1..c3d47117700d 100644
@@ -2927,7 +2927,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 	rot = get_unaligned_be16(&buffer[4]);
 
 	if (rot == 1) {
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
 	}
 

drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3133

Quoted 9 lines
drivers/scsi/sd.c +2 -0
@@ -3123,7 +3125,9 @@ static int sd_revalidate_disk(struct gendisk *disk)
 		 * cause this to be updated correctly and any device which
 		 * doesn't support it should be treated as rotational.
 		 */
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
 
 		if (scsi_device_supports_vpd(sdp)) {

Hmm.

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

I would far prefer if we could delegate any queueing decision to the elevators, and completely drop the device_busy flag for all devices. But I do see that this might not be easy to do, nor necessarily something which others agree on.
So alternatively, can't we make this a per-host flag, instead of having it per device characteristic?

I'm just thinking of a storage array with a massive cache; it's anyone's guess if 'nonrot' is set there, but at the same time these things can be fast, so they might be profiting from dropping the device_busy counter there, too.

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke		      Teamlead Storage & Networking
[email protected]			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 247165 (AG München), GF: Felix Imendörffer
#12 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 7 lines · Hannes Reinecke

Hmm.

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

This concerns me as well, it seems like the SCSI ML should have it's own per-device attribute if we actually need to control this per-device instead of on a per-host or per-driver basis. And it seems like this is something that is specific to high-performance drivers, so changing the way this works for all drivers seems a bit much.

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected. (Plus Hannes just submitted submitted the patches to remove .use_cmd_list, which was another piece of ML functionality used by only a few drivers.)

Ming's patch does freeze the queue if NONROT is changed by sysfs, but the flag can be changed by other kernel code, e.g. sd_revalidate_disk() clears it and then calls sd_read_block_characteristics() which may set it again. So it's not clear to me how reliable this is.

-Ewan

#13 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 17 lines · Ewan D. Milne
Quoted 3 lines · Hannes Reinecke

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

This concerns me as well, it seems like the SCSI ML should have it's own per-device attribute if we actually need to control this per-device instead of on a per-host or per-driver basis. And it seems like this is something that is specific to high-performance drivers, so changing the way this works for all drivers seems a bit much.

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected. (Plus Hannes just submitted submitted the patches to remove .use_cmd_list, which was another piece of ML functionality used by only a few drivers.)

Quoted 4 lines

Ming's patch does freeze the queue if NONROT is changed by sysfs, but the flag can be changed by other kernel code, e.g. sd_revalidate_disk() clears it and then calls sd_read_block_characteristics() which may set it again. So it's not clear to me how reliable this is.

How about changing the default behavior into ignoring sdev->queue_depth and only honoring sdev->queue_depth if a "quirk" flag is set or if overridden by setting a sysfs attribute? My understanding is that the goal of the queue ramp-up/ramp-down mechanism is to reduce the number of times a SCSI device responds "BUSY". An alternative for queue ramp-up/ramp-down is a delayed queue re-run. I think if scsi_queue_rq() returns BLK_STS_RESOURCE that the queue is only rerun after a delay. From blk_mq_dispatch_rq_list():

	[ ... ]
	else if (needs_restart && (ret == BLK_STS_RESOURCE))
		blk_mq_delay_run_hw_queue(hctx, BLK_MQ_RESOURCE_DELAY);

Bart.

#14 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 32 lines · Bart Van Assche
Quoted 17 lines · Ewan D. Milne
Quoted 3 lines · Hannes Reinecke

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

This concerns me as well, it seems like the SCSI ML should have it's own per-device attribute if we actually need to control this per-device instead of on a per-host or per-driver basis. And it seems like this is something that is specific to high-performance drivers, so changing the way this works for all drivers seems a bit much.

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected. (Plus Hannes just submitted submitted the patches to remove .use_cmd_list, which was another piece of ML functionality used by only a few drivers.)

Quoted 4 lines

Ming's patch does freeze the queue if NONROT is changed by sysfs, but the flag can be changed by other kernel code, e.g. sd_revalidate_disk() clears it and then calls sd_read_block_characteristics() which may set it again. So it's not clear to me how reliable this is.

How about changing the default behavior into ignoring sdev->queue_depth and only honoring sdev->queue_depth if a "quirk" flag is set or if overridden by setting a sysfs attribute? My understanding is that the goal of the queue ramp-up/ramp-down mechanism is to reduce the number of times a SCSI device responds "BUSY". An alternative for queue ramp-up/ramp-down is a delayed queue re-run. I think if scsi_queue_rq() returns BLK_STS_RESOURCE that the queue is only rerun after a delay.
From blk_mq_dispatch_rq_list():

Quoted 5 lines
	[ ... ]
	else if (needs_restart && (ret == BLK_STS_RESOURCE))
		blk_mq_delay_run_hw_queue(hctx, BLK_MQ_RESOURCE_DELAY);

Bart.

In general it is better not to put in changes that affect older drivers that are not regularly tested, I think. I would prefer an opt-in for drivers desiring higher performance.

Delaying the queue re-run vs. a ramp down might negatively affect performance. I'm not sure how accurate the ramp is at discovering the optimum though.

-Ewan

#15 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Ewan,

Quoted 3 lines

Delaying the queue re-run vs. a ramp down might negatively affect performance. I'm not sure how accurate the ramp is at discovering the optimum though.

The optimum - as well as the actual limit - might change over time in a multi-initiator setup as other hosts grab resources on the device.

I do think that the only way forward here, if we want to avoid counting outstanding commands for performance reasons, is to ensure that the BUSY/QUEUE_FULL/TASK_SET_FULL handling is relatively fast path and not something deep in the bowels of error handling. Which it actually isn't. But I do think we'll need to take a closer look.

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
author
#16 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Bart Van Assche
Quoted 18 lines · Bart Van Assche
Quoted 17 lines · Ewan D. Milne
Quoted 3 lines · Hannes Reinecke

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

This concerns me as well, it seems like the SCSI ML should have it's own per-device attribute if we actually need to control this per-device instead of on a per-host or per-driver basis. And it seems like this is something that is specific to high-performance drivers, so changing the way this works for all drivers seems a bit much.

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected. (Plus Hannes just submitted submitted the patches to remove .use_cmd_list, which was another piece of ML functionality used by only a few drivers.)

Quoted 8 lines
Quoted 4 lines

Ming's patch does freeze the queue if NONROT is changed by sysfs, but the flag can be changed by other kernel code, e.g. sd_revalidate_disk() clears it and then calls sd_read_block_characteristics() which may set it again. So it's not clear to me how reliable this is.

How about changing the default behavior into ignoring sdev->queue_depth and only honoring sdev->queue_depth if a "quirk" flag is set or if overridden by setting a sysfs attribute?

Using 'quirk' was my first idea in mind when we start to discuss the issue, but problem is that it isn't flexible, for example, one HBA may connects HDD. in one setting, and SSD. in another setting.

Quoted 3 lines

My understanding is that the goal of the queue ramp-up/ramp-down mechanism is to reduce the number of times a SCSI device responds "BUSY".

I don't understand the motivation of ramp-up/ramp-down, maybe it is just for fairness among LUNs. So far, blk-mq provides fair IO submission among LUNs. One problem of ignoring it is that sequential IO performance may be dropped much compared with before.

Quoted 7 lines

An alternative for queue ramp-up/ramp-down is a delayed
queue re-run. I think if scsi_queue_rq() returns BLK_STS_RESOURCE that the queue is only rerun after a delay. From blk_mq_dispatch_rq_list():

	[ ... ]
	else if (needs_restart && (ret == BLK_STS_RESOURCE))
		blk_mq_delay_run_hw_queue(hctx, BLK_MQ_RESOURCE_DELAY);

The delay re-run can't work given we call blk_mq_get_dispatch_budget() before dequeuing request from scheduler/sw queue for improving IO merge. At that time, run queue won't be involved.

Thanks,
Ming

#17 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Ming,

Quoted 2 lines

I don't understand the motivation of ramp-up/ramp-down, maybe it is just for fairness among LUNs.

Congestion control. Devices have actual, physical limitations that are different from the tag context limitations on the HBA. You don't have that problem on NVMe because (at least for PCIe) the storage device and the controller are one and the same.

If you submit 100000 concurrent requests to a SCSI drive that does 100 IOPS, some requests will time out before they get serviced. Consequently we have the ability to raise and lower the queue depth to constrain the amount of requests in flight to a given device at any point in time.

Also, devices use BUSY/QUEUE_FULL/TASK_SET_FULL to cause the OS to back off. We frequently see issues where the host can submit burst I/O much faster than the device can de-stage from cache. In that scenario the device reports BUSY/QF/TSF and we will back off so the device gets a chance to recover. If we just let the application submit new I/O without bounds, the system would never actually recover.

Note that the actual, physical limitations for how many commands a target can handle are typically much, much lower than the number of tags the HBA can manage. SATA devices can only express 32 concurrent commands. SAS devices typically 128 concurrent commands per port. Arrays differ.

If we ignore the RAID controller use case where the controller internally queues and arbitrates commands between many devices, how is submitting 1000 concurrent requests to a device which only has 128 command slots going to work?

Some HBAs have special sauce to manage BUSY/QF/TSF, some don't. If we blindly stop restricting the number of I/Os in flight in the ML, we may exceed either the capabilities of what the transport protocol can express or internal device resources.

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
author
#18 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Hi Martin,

Quoted 11 lines · Martin K. Petersen

Ming,

Quoted 2 lines

I don't understand the motivation of ramp-up/ramp-down, maybe it is just for fairness among LUNs.

Congestion control. Devices have actual, physical limitations that are different from the tag context limitations on the HBA. You don't have that problem on NVMe because (at least for PCIe) the storage device and the controller are one and the same.

Quoted 5 lines

If you submit 100000 concurrent requests to a SCSI drive that does 100 IOPS, some requests will time out before they get serviced. Consequently we have the ability to raise and lower the queue depth to constrain the amount of requests in flight to a given device at any point in time.

blk-mq has already puts a limit on each LUN, the number is host_queue_depth / nr_active_LUNs, see hctx_may_queue().

Looks this way works for NVMe, that is why I try to bypass .device_busy for SSD which is too expensive on fast storage. Even Hannes wants to kill it completely.

Quoted 8 lines

Also, devices use BUSY/QUEUE_FULL/TASK_SET_FULL to cause the OS to back off. We frequently see issues where the host can submit burst I/O much faster than the device can de-stage from cache. In that scenario the device reports BUSY/QF/TSF and we will back off so the device gets a chance to recover. If we just let the application submit new I/O without bounds, the system would never actually recover.

Quoted 5 lines

Note that the actual, physical limitations for how many commands a target can handle are typically much, much lower than the number of tags the HBA can manage. SATA devices can only express 32 concurrent commands. SAS devices typically 128 concurrent commands per port. Arrays differ.

I understand SATA's host queue depth is set as 32.

But SAS HBA's queue depth is often big, so do we reply on .device_busy for throttling requests to SAS?

Quoted 5 lines

If we ignore the RAID controller use case where the controller internally queues and arbitrates commands between many devices, how is submitting 1000 concurrent requests to a device which only has 128 command slots going to work?

For SSD, I guess it might be fine, given NVMe sets per-hw-queue depth as 1023 usually. That means the concurrent requests can be as many as 1023 * nr_hw_queues in case of single namespace.

Quoted 5 lines

Some HBAs have special sauce to manage BUSY/QF/TSF, some don't. If we blindly stop restricting the number of I/Os in flight in the ML, we may exceed either the capabilities of what the transport protocol can express or internal device resources.

OK, one conservative approach may be just to just bypass .device_busy in case of SSD only for some high end HBA.

Or maybe we can wire up sdev->queue_depth with block layer's scheduler queue depth? One issue is that sdev->queue_depth may be updated some times.

Thanks,
Ming

#19 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Martin K. Petersen
Quoted 2 lines
Quoted 2 lines

If we ignore the RAID controller use case where the controller internally queues and arbitrates commands between many devices

These controllers should not be "ignored", but rather enabled. Many of them visualize both HDD and NVMe devices behind them, and thus forced to expose themselves as SCSI controllers.
However, they have their own queue management and IO merging capabilities. Many have capability of holding IO in queue and pull them as needed (just like NVMe), and thus does not bother if many IOs to a device or controller is sent or if there is congestion. In case of congestion, the IO will simply wait in queue, along with advanced timeout handling capabilities. Besides, as Ming pointed out, Block layer (function hctx_may_queue) already limits IO on a per controller and per LUN basis.

Overall, if the proposal does not work for all cases, then at least it should be made optional for high end controller, so that they are not disadvantaged vis-a-vis NVMe, just because they expose themselves as SCSI in order to support a wide range of devices behind them.

thanks,
Sumanesh

Quoted 38 lines · Martin K. Petersen

Ming,

Quoted 2 lines

I don't understand the motivation of ramp-up/ramp-down, maybe it is just for fairness among LUNs.

Congestion control. Devices have actual, physical limitations that are different from the tag context limitations on the HBA. You don't have that problem on NVMe because (at least for PCIe) the storage device and the controller are one and the same.

If you submit 100000 concurrent requests to a SCSI drive that does 100 IOPS, some requests will time out before they get serviced. Consequently we have the ability to raise and lower the queue depth to constrain the amount of requests in flight to a given device at any point in time.

Also, devices use BUSY/QUEUE_FULL/TASK_SET_FULL to cause the OS to back off. We frequently see issues where the host can submit burst I/O much faster than the device can de-stage from cache. In that scenario the device reports BUSY/QF/TSF and we will back off so the device gets a chance to recover. If we just let the application submit new I/O without bounds, the system would never actually recover.

Note that the actual, physical limitations for how many commands a target can handle are typically much, much lower than the number of tags the HBA can manage. SATA devices can only express 32 concurrent commands. SAS devices typically 128 concurrent commands per port. Arrays differ.

If we ignore the RAID controller use case where the controller internally queues and arbitrates commands between many devices, how is submitting 1000 concurrent requests to a device which only has 128 command slots going to work?

Some HBAs have special sauce to manage BUSY/QF/TSF, some don't. If we blindly stop restricting the number of I/Os in flight in the ML, we may exceed either the capabilities of what the transport protocol can express or internal device resources.

#20 lore raw

Re: Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Ewan D. Milne
Quoted 3 lines

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected.

I think we can have a host template attribute and discard this check altogether, that is not check device_busy for both SDD and HDDs.

As both you and Hannes mentioned, this change affects high end controllers most, and most of them have some storage IO size limit. Also, for HDD sequential IO is almost always large and
touch the controller max IO size limit. Thus, I am not sure merge matters for these kind of controllers. Database use REDO log and small sequential IO, but those are targeted to SSDs, where latency and IOPs are far more important than IO merging.
If this patch is opt-in for drivers, so any LLD that cannot take advantage of the flag need not set it, and would work as-is

Thanks,

Sumanesh

Quoted 27 lines · Ewan D. Milne
Quoted 6 lines · Hannes Reinecke

Hmm.

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

This concerns me as well, it seems like the SCSI ML should have it's own per-device attribute if we actually need to control this per-device instead of on a per-host or per-driver basis. And it seems like this is something that is specific to high-performance drivers, so changing the way this works for all drivers seems a bit much.

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected. (Plus Hannes just submitted submitted the patches to remove .use_cmd_list, which was another piece of ML functionality used by only a few drivers.)

Ming's patch does freeze the queue if NONROT is changed by sysfs, but the flag can be changed by other kernel code, e.g. sd_revalidate_disk() clears it and then calls sd_read_block_characteristics() which may set it again. So it's not clear to me how reliable this is.

-Ewan

author
#21 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Ewan D. Milne
Quoted 21 lines · Ewan D. Milne
Quoted 7 lines · Hannes Reinecke

Hmm.

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

This concerns me as well, it seems like the SCSI ML should have it's own per-device attribute if we actually need to control this per-device instead of on a per-host or per-driver basis. And it seems like this is something that is specific to high-performance drivers, so changing the way this works for all drivers seems a bit much.

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected. (Plus Hannes just submitted submitted the patches to remove .use_cmd_list, which was another piece of ML functionality used by only a few drivers.)

Quoted 4 lines

Ming's patch does freeze the queue if NONROT is changed by sysfs, but the flag can be changed by other kernel code, e.g. sd_revalidate_disk() clears it and then calls sd_read_block_characteristics() which may set it again. So it's not clear to me how reliable this is.

The queue freeze is applied in sd_revalidate_disk() too, isn't it?

Thanks,
Ming

#22 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 22 lines · Ming Lei
Quoted 21 lines · Ewan D. Milne
Quoted 7 lines · Hannes Reinecke

Hmm.

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

This concerns me as well, it seems like the SCSI ML should have it's own per-device attribute if we actually need to control this per-device instead of on a per-host or per-driver basis. And it seems like this is something that is specific to high-performance drivers, so changing the way this works for all drivers seems a bit much.

Ordinarily I'd prefer a host template attribute as Sumanesh proposed, but I dislike wrapping the examination of that and the queue flag in a macro that makes it not obvious how the behavior is affected. (Plus Hannes just submitted submitted the patches to remove .use_cmd_list, which was another piece of ML functionality used by only a few drivers.)

Quoted 7 lines
Quoted 4 lines

Ming's patch does freeze the queue if NONROT is changed by sysfs, but the flag can be changed by other kernel code, e.g. sd_revalidate_disk() clears it and then calls sd_read_block_characteristics() which may set it again. So it's not clear to me how reliable this is.

The queue freeze is applied in sd_revalidate_disk() too, isn't it?

Yes, sorry, you are right, your patch does add this. But if anything else changes the NONROT attribute for a queue associated with a SCSI device in the future it would have to freeze the queue. Because the device_busy counter mechanism would rely on it to work right. This isn't an obvious connection, it seems to me.

-Ewan

author
#23 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Hannes Reinecke
Quoted 170 lines · Hannes Reinecke
Quoted 169 lines · Ming Lei

SCSI core uses the atomic variable of sdev->device_busy to track in-flight IO requests dispatched to this scsi device. IO request may be submitted from any CPU, so the cost for maintaining the shared atomic counter can be very big on big NUMA machine with lots of CPU cores.

sdev->queue_depth is usually used for two purposes: 1) improve IO merge;
2) fair IO request scattered among all LUNs.

blk-mq already provides fair request allocation among all active shared request queues(LUNs), see hctx_may_queue().

NVMe doesn't have such per-request-queue(namespace) queue depth, so it is reasonable to ignore the limit for SCSI SSD too. Also IO merge won't play big role for reaching top SSD performance.

With this patch, big cost for tracking in-flight per-LUN requests via atomic variable can be saved.

Given QUEUE_FLAG_NONROT is read in IO path, we have to freeze queue before changing this flag.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-sysfs.c       | 14 +++++++++++++-
 drivers/scsi/scsi_lib.c | 24 ++++++++++++++++++------
 drivers/scsi/sd.c       |  4 ++++
 3 files changed, 35 insertions(+), 7 deletions(-)
block/blk-sysfs.c +13 -1
index fca9b158f4a0..9cc0dd5f88a1 100644
@@ -281,6 +281,18 @@ QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0);
 QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0);
 #undef QUEUE_SYSFS_BIT_FNS
 
+static ssize_t queue_store_nonrot_freeze(struct request_queue *q,
+		const char *page, size_t count)
+{
+	size_t ret;
+
+	blk_mq_freeze_queue(q);
+	ret = queue_store_nonrot(q, page, count);
+	blk_mq_unfreeze_queue(q);
+
+	return ret;
+}
+
 static ssize_t queue_zoned_show(struct request_queue *q, char *page)
 {
 	switch (blk_queue_zoned_model(q)) {
@@ -642,7 +654,7 @@ static struct queue_sysfs_entry queue_write_zeroes_max_entry = {
 static struct queue_sysfs_entry queue_nonrot_entry = {
 	.attr = {.name = "rotational", .mode = 0644 },
 	.show = queue_show_nonrot,
-	.store = queue_store_nonrot,
+	.store = queue_store_nonrot_freeze,
 };
 
 static struct queue_sysfs_entry queue_zoned_entry = {
drivers/scsi/scsi_lib.c +18 -6
index 62a86a82c38d..72655a049efd 100644
@@ -354,7 +354,8 @@ void scsi_device_unbusy(struct scsi_device *sdev, struct scsi_cmnd *cmd)
 	if (starget->can_queue > 0)
 		atomic_dec(&starget->target_busy);
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static void scsi_kick_queue(struct request_queue *q)
@@ -410,7 +411,8 @@ static void scsi_single_lun_run(struct scsi_device *current_sdev)
 
 static inline bool scsi_device_is_busy(struct scsi_device *sdev)
 {
-	if (atomic_read(&sdev->device_busy) >= sdev->queue_depth)
+	if (!blk_queue_nonrot(sdev->request_queue) &&
+			atomic_read(&sdev->device_busy) >= sdev->queue_depth)
 		return true;
 	if (atomic_read(&sdev->device_blocked) > 0)
 		return true;
@@ -1283,8 +1285,12 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				  struct scsi_device *sdev)
 {
 	unsigned int busy;
+	bool bypass = blk_queue_nonrot(sdev->request_queue);
 
-	busy = atomic_inc_return(&sdev->device_busy) - 1;
+	if (!bypass)
+		busy = atomic_inc_return(&sdev->device_busy) - 1;
+	else
+		busy = 0;
 	if (atomic_read(&sdev->device_blocked)) {
 		if (busy)
 			goto out_dec;
@@ -1298,12 +1304,16 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				   "unblocking device at zero depth\n"));
 	}
 
+	if (bypass)
+		return 1;
+
 	if (busy >= sdev->queue_depth)
 		goto out_dec;
 
 	return 1;
 out_dec:
-	atomic_dec(&sdev->device_busy);
+	if (!bypass)
+		atomic_dec(&sdev->device_busy);
 	return 0;
 }
 
@@ -1624,7 +1634,8 @@ static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx)
 	struct request_queue *q = hctx->queue;
 	struct scsi_device *sdev = q->queuedata;
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static bool scsi_mq_get_budget(struct blk_mq_hw_ctx *hctx)
@@ -1731,7 +1742,8 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 	case BLK_STS_OK:
 		break;
 	case BLK_STS_RESOURCE:
-		if (atomic_read(&sdev->device_busy) ||
+		if ((!blk_queue_nonrot(sdev->request_queue) &&
+		     atomic_read(&sdev->device_busy)) ||
 		    scsi_device_blocked(sdev))
 			ret = BLK_STS_DEV_RESOURCE;
 		break;
drivers/scsi/sd.c +4 -0
index 0744c34468e1..c3d47117700d 100644
@@ -2927,7 +2927,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 	rot = get_unaligned_be16(&buffer[4]);
 
 	if (rot == 1) {
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
 	}
 
@@ -3123,7 +3125,9 @@ static int sd_revalidate_disk(struct gendisk *disk)
 		 * cause this to be updated correctly and any device which
 		 * doesn't support it should be treated as rotational.
 		 */
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
 
Quoted 7 lines

drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3133

Quoted 2 lines
drivers/scsi/sd.c +0 -0
 		if (scsi_device_supports_vpd(sdp)) {

Hmm.

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

That won't be an issue given this patch freezes queue when the NONROT queue flag is changed.

Quoted 3 lines

I would far prefer if we could delegate any queueing decision to the elevators, and completely drop the device_busy flag for all devices.

If you drop it, you may create big sequential IO performance drop on HDD., that is why this patch only bypasses sdev->queue_depth on SSD. NVMe bypasses it because no one uses HDD. via NVMe.

Quoted 4 lines

But I do see that this might not be easy to do, nor necessarily something which others agree on.
So alternatively, can't we make this a per-host flag, instead of having it per device characteristic?

Are you sure each LUN shares the same NONROT flag? I think it can be true for each LUN to have different NONROT flag.

Quoted 5 lines

I'm just thinking of a storage array with a massive cache; it's anyone's guess if 'nonrot' is set there, but at the same time these things can be fast, so they might be profiting from dropping the device_busy counter there, too.

If you are worrying about slow queue freeze, that won't be an issue, given we don't switch to percpu mode until the flag is updated during sd probe, and blk_mq_freeze_queue_wait() is pretty quick in ATOMIC mode.

Thanks,
Ming

#24 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 175 lines · Ming Lei
Quoted 174 lines · Hannes Reinecke
Quoted 162 lines · Ming Lei

SCSI core uses the atomic variable of sdev->device_busy to track in-flight IO requests dispatched to this scsi device. IO request may be submitted from any CPU, so the cost for maintaining the shared atomic counter can be very big on big NUMA machine with lots of CPU cores.

sdev->queue_depth is usually used for two purposes: 1) improve IO merge;
2) fair IO request scattered among all LUNs.

blk-mq already provides fair request allocation among all active shared request queues(LUNs), see hctx_may_queue().

NVMe doesn't have such per-request-queue(namespace) queue depth, so it is reasonable to ignore the limit for SCSI SSD too. Also IO merge won't play big role for reaching top SSD performance.

With this patch, big cost for tracking in-flight per-LUN requests via atomic variable can be saved.

Given QUEUE_FLAG_NONROT is read in IO path, we have to freeze queue before changing this flag.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-sysfs.c       | 14 +++++++++++++-
 drivers/scsi/scsi_lib.c | 24 ++++++++++++++++++------
 drivers/scsi/sd.c       |  4 ++++
 3 files changed, 35 insertions(+), 7 deletions(-)
block/blk-sysfs.c +13 -1
index fca9b158f4a0..9cc0dd5f88a1 100644
@@ -281,6 +281,18 @@ QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0);
 QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0);
 #undef QUEUE_SYSFS_BIT_FNS
 
+static ssize_t queue_store_nonrot_freeze(struct request_queue *q,
+		const char *page, size_t count)
+{
+	size_t ret;
+
+	blk_mq_freeze_queue(q);
+	ret = queue_store_nonrot(q, page, count);
+	blk_mq_unfreeze_queue(q);
+
+	return ret;
+}
+
 static ssize_t queue_zoned_show(struct request_queue *q, char *page)
 {
 	switch (blk_queue_zoned_model(q)) {
@@ -642,7 +654,7 @@ static struct queue_sysfs_entry queue_write_zeroes_max_entry = {
 static struct queue_sysfs_entry queue_nonrot_entry = {
 	.attr = {.name = "rotational", .mode = 0644 },
 	.show = queue_show_nonrot,
-	.store = queue_store_nonrot,
+	.store = queue_store_nonrot_freeze,
 };
 
 static struct queue_sysfs_entry queue_zoned_entry = {
drivers/scsi/scsi_lib.c +18 -6
index 62a86a82c38d..72655a049efd 100644
@@ -354,7 +354,8 @@ void scsi_device_unbusy(struct scsi_device *sdev, struct scsi_cmnd *cmd)
 	if (starget->can_queue > 0)
 		atomic_dec(&starget->target_busy);
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static void scsi_kick_queue(struct request_queue *q)
@@ -410,7 +411,8 @@ static void scsi_single_lun_run(struct scsi_device *current_sdev)
 
 static inline bool scsi_device_is_busy(struct scsi_device *sdev)
 {
-	if (atomic_read(&sdev->device_busy) >= sdev->queue_depth)
+	if (!blk_queue_nonrot(sdev->request_queue) &&
+			atomic_read(&sdev->device_busy) >= sdev->queue_depth)
 		return true;
 	if (atomic_read(&sdev->device_blocked) > 0)
 		return true;
@@ -1283,8 +1285,12 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				  struct scsi_device *sdev)
 {
 	unsigned int busy;
+	bool bypass = blk_queue_nonrot(sdev->request_queue);
 
-	busy = atomic_inc_return(&sdev->device_busy) - 1;
+	if (!bypass)
+		busy = atomic_inc_return(&sdev->device_busy) - 1;
+	else
+		busy = 0;
 	if (atomic_read(&sdev->device_blocked)) {
 		if (busy)
 			goto out_dec;
@@ -1298,12 +1304,16 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				   "unblocking device at zero depth\n"));
 	}
 
+	if (bypass)
+		return 1;
+
 	if (busy >= sdev->queue_depth)
 		goto out_dec;
 
 	return 1;
 out_dec:
-	atomic_dec(&sdev->device_busy);
+	if (!bypass)
+		atomic_dec(&sdev->device_busy);
 	return 0;
 }
 
@@ -1624,7 +1634,8 @@ static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx)
 	struct request_queue *q = hctx->queue;
 	struct scsi_device *sdev = q->queuedata;
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static bool scsi_mq_get_budget(struct blk_mq_hw_ctx *hctx)
@@ -1731,7 +1742,8 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 	case BLK_STS_OK:
 		break;
 	case BLK_STS_RESOURCE:
-		if (atomic_read(&sdev->device_busy) ||
+		if ((!blk_queue_nonrot(sdev->request_queue) &&
+		     atomic_read(&sdev->device_busy)) ||
 		    scsi_device_blocked(sdev))
 			ret = BLK_STS_DEV_RESOURCE;
 		break;
drivers/scsi/sd.c +2 -0
index 0744c34468e1..c3d47117700d 100644
@@ -2927,7 +2927,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 	rot = get_unaligned_be16(&buffer[4]);
 
 	if (rot == 1) {
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
 	}
 

drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3133

Quoted 9 lines
drivers/scsi/sd.c +2 -0
@@ -3123,7 +3125,9 @@ static int sd_revalidate_disk(struct gendisk *disk)
 		 * cause this to be updated correctly and any device which
 		 * doesn't support it should be treated as rotational.
 		 */
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
 
 		if (scsi_device_supports_vpd(sdp)) {

Hmm.

Quoted 7 lines
Quoted 3 lines

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

That won't be an issue given this patch freezes queue when the NONROT queue flag is changed.

That's not what I'm saying.

My issue is that we're turning the device_busy counter into a conditional counter, which only must be accessed when that condition is true.
This not only puts a burden on the developer (as he has to remember to always check the condition), but also has possible issues when switching between modes.
The latter you've addressed with ensuring that the queue must be frozen when modifying that flag, but the former is a conceptual problem.

And that was what I had been referring to.

Quoted 8 lines
Quoted 3 lines

I would far prefer if we could delegate any queueing decision to the elevators, and completely drop the device_busy flag for all devices.

If you drop it, you may create big sequential IO performance drop on HDD., that is why this patch only bypasses sdev->queue_depth on SSD. NVMe bypasses it because no one uses HDD. via NVMe.

I still wonder how much performance drop we actually see; what seems to happen is that device_busy just arbitrary pushes back to the block layer, giving it more time to do merging.
I do think we can do better then that...

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke		      Teamlead Storage & Networking
[email protected]			                  +49 911 74053 688
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 247165 (AG München), GF: Felix Imendörffer
author
#25 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 183 lines · Hannes Reinecke
Quoted 175 lines · Ming Lei
Quoted 174 lines · Hannes Reinecke
Quoted 162 lines · Ming Lei

SCSI core uses the atomic variable of sdev->device_busy to track in-flight IO requests dispatched to this scsi device. IO request may be submitted from any CPU, so the cost for maintaining the shared atomic counter can be very big on big NUMA machine with lots of CPU cores.

sdev->queue_depth is usually used for two purposes: 1) improve IO merge;
2) fair IO request scattered among all LUNs.

blk-mq already provides fair request allocation among all active shared request queues(LUNs), see hctx_may_queue().

NVMe doesn't have such per-request-queue(namespace) queue depth, so it is reasonable to ignore the limit for SCSI SSD too. Also IO merge won't play big role for reaching top SSD performance.

With this patch, big cost for tracking in-flight per-LUN requests via atomic variable can be saved.

Given QUEUE_FLAG_NONROT is read in IO path, we have to freeze queue before changing this flag.

Cc Sathya Prakash <[email protected]>
Cc Chaitra P B <[email protected]>
Cc Suganath Prabu Subramani <[email protected]>
Cc Kashyap Desai <[email protected]>
Cc Sumit Saxena <[email protected]>
Cc Shivasharan S <[email protected]>
Cc Ewan D. Milne <[email protected]>
Cc Christoph Hellwig <[email protected]>,
Cc Hannes Reinecke <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-sysfs.c       | 14 +++++++++++++-
 drivers/scsi/scsi_lib.c | 24 ++++++++++++++++++------
 drivers/scsi/sd.c       |  4 ++++
 3 files changed, 35 insertions(+), 7 deletions(-)
block/blk-sysfs.c +13 -1
index fca9b158f4a0..9cc0dd5f88a1 100644
@@ -281,6 +281,18 @@ QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0);
 QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0);
 #undef QUEUE_SYSFS_BIT_FNS
 
+static ssize_t queue_store_nonrot_freeze(struct request_queue *q,
+		const char *page, size_t count)
+{
+	size_t ret;
+
+	blk_mq_freeze_queue(q);
+	ret = queue_store_nonrot(q, page, count);
+	blk_mq_unfreeze_queue(q);
+
+	return ret;
+}
+
 static ssize_t queue_zoned_show(struct request_queue *q, char *page)
 {
 	switch (blk_queue_zoned_model(q)) {
@@ -642,7 +654,7 @@ static struct queue_sysfs_entry queue_write_zeroes_max_entry = {
 static struct queue_sysfs_entry queue_nonrot_entry = {
 	.attr = {.name = "rotational", .mode = 0644 },
 	.show = queue_show_nonrot,
-	.store = queue_store_nonrot,
+	.store = queue_store_nonrot_freeze,
 };
 
 static struct queue_sysfs_entry queue_zoned_entry = {
drivers/scsi/scsi_lib.c +18 -6
index 62a86a82c38d..72655a049efd 100644
@@ -354,7 +354,8 @@ void scsi_device_unbusy(struct scsi_device *sdev, struct scsi_cmnd *cmd)
 	if (starget->can_queue > 0)
 		atomic_dec(&starget->target_busy);
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static void scsi_kick_queue(struct request_queue *q)
@@ -410,7 +411,8 @@ static void scsi_single_lun_run(struct scsi_device *current_sdev)
 
 static inline bool scsi_device_is_busy(struct scsi_device *sdev)
 {
-	if (atomic_read(&sdev->device_busy) >= sdev->queue_depth)
+	if (!blk_queue_nonrot(sdev->request_queue) &&
+			atomic_read(&sdev->device_busy) >= sdev->queue_depth)
 		return true;
 	if (atomic_read(&sdev->device_blocked) > 0)
 		return true;
@@ -1283,8 +1285,12 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				  struct scsi_device *sdev)
 {
 	unsigned int busy;
+	bool bypass = blk_queue_nonrot(sdev->request_queue);
 
-	busy = atomic_inc_return(&sdev->device_busy) - 1;
+	if (!bypass)
+		busy = atomic_inc_return(&sdev->device_busy) - 1;
+	else
+		busy = 0;
 	if (atomic_read(&sdev->device_blocked)) {
 		if (busy)
 			goto out_dec;
@@ -1298,12 +1304,16 @@ static inline int scsi_dev_queue_ready(struct request_queue *q,
 				   "unblocking device at zero depth\n"));
 	}
 
+	if (bypass)
+		return 1;
+
 	if (busy >= sdev->queue_depth)
 		goto out_dec;
 
 	return 1;
 out_dec:
-	atomic_dec(&sdev->device_busy);
+	if (!bypass)
+		atomic_dec(&sdev->device_busy);
 	return 0;
 }
 
@@ -1624,7 +1634,8 @@ static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx)
 	struct request_queue *q = hctx->queue;
 	struct scsi_device *sdev = q->queuedata;
 
-	atomic_dec(&sdev->device_busy);
+	if (!blk_queue_nonrot(sdev->request_queue))
+		atomic_dec(&sdev->device_busy);
 }
 
 static bool scsi_mq_get_budget(struct blk_mq_hw_ctx *hctx)
@@ -1731,7 +1742,8 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 	case BLK_STS_OK:
 		break;
 	case BLK_STS_RESOURCE:
-		if (atomic_read(&sdev->device_busy) ||
+		if ((!blk_queue_nonrot(sdev->request_queue) &&
+		     atomic_read(&sdev->device_busy)) ||
 		    scsi_device_blocked(sdev))
 			ret = BLK_STS_DEV_RESOURCE;
 		break;
drivers/scsi/sd.c +2 -0
index 0744c34468e1..c3d47117700d 100644
@@ -2927,7 +2927,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
 	rot = get_unaligned_be16(&buffer[4]);
 
 	if (rot == 1) {
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
 	}
 

drivers/scsi/sd.c static int sd_revalidate_disk(struct gendisk *disk) · line 3133

Quoted 9 lines
drivers/scsi/sd.c +2 -0
@@ -3123,7 +3125,9 @@ static int sd_revalidate_disk(struct gendisk *disk)
 		 * cause this to be updated correctly and any device which
 		 * doesn't support it should be treated as rotational.
 		 */
+		blk_mq_freeze_queue(q);
 		blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+		blk_mq_unfreeze_queue(q);
 		blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
 
 		if (scsi_device_supports_vpd(sdp)) {

Hmm.

Quoted 7 lines
Quoted 3 lines

I must admit I patently don't like this explicit dependency on blk_nonrot(). Having a conditional counter is just an open invitation to getting things wrong...

That won't be an issue given this patch freezes queue when the NONROT queue flag is changed.

Quoted 8 lines

That's not what I'm saying.

My issue is that we're turning the device_busy counter into a conditional counter, which only must be accessed when that condition is true.
This not only puts a burden on the developer (as he has to remember to always check the condition), but also has possible issues when switching between modes.

The conditional check can be avoided only if we can kill device_busy completely. However, as Martin commented, that isn't realistic so far.

Quoted 2 lines

The latter you've addressed with ensuring that the queue must be frozen when modifying that flag, but the former is a conceptual problem.

It can't be perfect, but it can be good by the latter.

Someone said 'Perfect is the enemy of the good', :-)

Quoted 7 lines

And that was what I had been referring to.

Quoted 4 lines
Quoted 3 lines

I would far prefer if we could delegate any queueing decision to the elevators, and completely drop the device_busy flag for all devices.

Quoted 8 lines
Quoted 4 lines

If you drop it, you may create big sequential IO performance drop on HDD., that is why this patch only bypasses sdev->queue_depth on SSD. NVMe bypasses it because no one uses HDD. via NVMe.

I still wonder how much performance drop we actually see; what seems to happen is that device_busy just arbitrary pushes back to the block layer, giving it more time to do merging.
I do think we can do better then that...

For example, running the following script[1] on 4-core VM:

------------------------------------------

                    | QD:255    | QD: 32  | 

------------------------------------------
fio read throughput | 825MB/s | 1432MB/s|
------------------------------------------

note:

QD: scsi_device's queue depth, which can be passed to test.sh. scsi debug's device queue depth is 255 at default, which is >= .can_queue

[1] test.sh

#!/bin/bash
QD=$1

modprobe scsi_debug ndelay=200000
sleep 2
DEVN=ls -d /sys/bus/pseudo/drivers/scsi_debug/adapter*/host*/target*/*/block/* | head -1 | xargs basename DEV="/dev/"$DEVN

echo $QD > /sys/block/$DEVN/device/queue_depth
SQD=cat /sys/block/$DEVN/device/queue_depth
echo "scsi device queue depth: $QD"

fio --readwrite=read --filename=$DEV --runtime=20s --time_based --group_reporting=1 \
    --ioengine=libaio --direct=1 --iodepth=64 --bs=4k --numjobs=4 --name=seq_perf

Thanks,
Ming

#26 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 17 lines · Ming Lei
Quoted 5 lines · Hannes Reinecke
Quoted 4 lines · Ming Lei
Quoted 2 lines · Hannes Reinecke

I would far prefer if we could delegate any queueing decision to the elevators, and completely drop the device_busy flag for all devices.

Quoted 8 lines
Quoted 4 lines

If you drop it, you may create big sequential IO performance drop on HDD., that is why this patch only bypasses sdev->queue_depth on SSD. NVMe bypasses it because no one uses HDD. via NVMe.

I still wonder how much performance drop we actually see; what seems to happen is that device_busy just arbitrary pushes back to the block layer, giving it more time to do merging.
I do think we can do better then that...

For example, running the following script[1] on 4-core VM:

Quoted 7 lines

------------------------------------------

                     | QD:255    | QD: 32  |

------------------------------------------
fio read throughput | 825MB/s | 1432MB/s|
------------------------------------------

[ ... ]

Hi Ming,

Thanks for having shared these numbers. I think this is very useful information. Do these results show the performance drop that happens if /sys/block/.../device/queue_depth exceeds .can_queue? What I am wondering about is how important these results are in the context of this discussion. Are there any modern SCSI devices for which a SCSI LLD sets scsi_host->can_queue and scsi_host->cmd_per_lun such that the device responds with BUSY? What surprised me is that only three SCSI LLDs call scsi_track_queue_full() (mptsas, bfa, esp_scsi). Does that mean that BUSY responses from a SCSI device or HBA are rare?

Thanks,

Bart.

#27 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 1 line · Bart Van Assche

Thanks for having shared these numbers. I think this is very useful information. Do these results show the performance drop that happens if /sys/block/.../device/queue_depth exceeds .can_queue? What I am wondering about is how important these results are in the context of this discussion. Are there any modern SCSI devices for which a SCSI LLD sets scsi_host->can_queue and scsi_host->cmd_per_lun such that the device responds with BUSY? What surprised me is that only three SCSI LLDs call scsi_track_queue_full() (mptsas, bfa, esp_scsi). Does that mean that BUSY responses from a SCSI device or HBA are rare?

That's because most of the drivers, which had queue full ramp up/ramp down in them and would have called scsi_track_queue_full() converted over to the moved-queue-full-handling-in-the-mid-layer, indicated by sht->track_queue_depth = 1.

Yes - it is still hit a lot!

-- james

#28 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 6 lines · James Smart
Quoted 1 line · Bart Van Assche

Thanks for having shared these numbers. I think this is very useful information. Do these results show the performance drop that happens if /sys/block/.../device/queue_depth exceeds .can_queue? What I am wondering about is how important these results are in the context of this discussion. Are there any modern SCSI devices for which a SCSI LLD sets scsi_host->can_queue and scsi_host->cmd_per_lun such that the device responds with BUSY? What surprised me is that only three SCSI LLDs call scsi_track_queue_full() (mptsas, bfa, esp_scsi). Does that mean that BUSY responses from a SCSI device or HBA are rare?

That's because most of the drivers, which had queue full ramp up/ramp down in them and would have called scsi_track_queue_full() converted over to the moved-queue-full-handling-in-the-mid-layer, indicated by sht->track_queue_depth = 1.

Yes - it is still hit a lot!

Hi James,

In the systems that I have been working on myself I made sure that the BUSY condition is rarely or never encountered. Anyway, since there are setups in which this condition is hit frequently we need to make sure that these setups keep performing well. I'm wondering now whether we should try to come up with an algorithm for maintaining sdev->device_busy only if it improves performance and for not maintaining sdev->device_busy for devices/HBAs that don't need it.

Bart.

author
#29 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 21 lines · Bart Van Assche
Quoted 11 lines · James Smart
Quoted 9 lines · Bart Van Assche

Thanks for having shared these numbers. I think this is very useful information. Do these results show the performance drop that happens if /sys/block/.../device/queue_depth exceeds .can_queue? What I am wondering about is how important these results are in the context of this discussion. Are there any modern SCSI devices for which a SCSI LLD sets scsi_host->can_queue and scsi_host->cmd_per_lun such that the device responds with BUSY? What surprised me is that only three SCSI LLDs call scsi_track_queue_full() (mptsas, bfa, esp_scsi). Does that mean that BUSY responses from a SCSI device or HBA are rare?

Quoted 6 lines

That's because most of the drivers, which had queue full ramp up/ramp down in them and would have called scsi_track_queue_full() converted over to the moved-queue-full-handling-in-the-mid-layer, indicated by sht->track_queue_depth = 1.

Yes - it is still hit a lot!

Hi James,

Quoted 7 lines

In the systems that I have been working on myself I made sure that the BUSY condition is rarely or never encountered. Anyway, since there are setups in which this condition is hit frequently we need to make sure that these setups keep performing well. I'm wondering now whether we should try to come up with an algorithm for maintaining sdev->device_busy only if it improves performance and for not maintaining sdev->device_busy for devices/HBAs that don't need it.

The simplest policy could be to only maintain sdev->device_busy for HDD.

Thanks,
Ming

author
#30 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Bart Van Assche
Quoted 24 lines · Bart Van Assche
Quoted 23 lines · Ming Lei
Quoted 5 lines · Hannes Reinecke
Quoted 4 lines · Ming Lei
Quoted 2 lines · Hannes Reinecke

I would far prefer if we could delegate any queueing decision to the elevators, and completely drop the device_busy flag for all devices.

Quoted 8 lines
Quoted 4 lines

If you drop it, you may create big sequential IO performance drop on HDD., that is why this patch only bypasses sdev->queue_depth on SSD. NVMe bypasses it because no one uses HDD. via NVMe.

I still wonder how much performance drop we actually see; what seems to happen is that device_busy just arbitrary pushes back to the block layer, giving it more time to do merging.
I do think we can do better then that...

For example, running the following script[1] on 4-core VM:

------------------------------------------

                     | QD:255    | QD: 32  |

------------------------------------------
fio read throughput | 825MB/s | 1432MB/s|
------------------------------------------

Quoted 7 lines
Quoted 1 line

[ ... ]

Hi Ming,

Thanks for having shared these numbers. I think this is very useful information. Do these results show the performance drop that happens if /sys/block/.../device/queue_depth exceeds .can_queue? What I am wondering

The above test just shows that IO merge plays important role here, and one important point for triggering IO merge is that .get_budget returns false.

If sdev->queue_depth is too big, .get_budget may never return false.

That is why this patch just bypasses .device_busy for SSD.

Quoted 4 lines

about is how important these results are in the context of this discussion. Are there any modern SCSI devices for which a SCSI LLD sets scsi_host->can_queue and scsi_host->cmd_per_lun such that the device responds with BUSY? What surprised me is that only three SCSI LLDs call

There are many such HBAs, for which sdev->queue_depth is smaller than .can_queue, especially in case of small number of LUNs.

Quoted 2 lines

scsi_track_queue_full() (mptsas, bfa, esp_scsi). Does that mean that BUSY responses from a SCSI device or HBA are rare?

It is only true for some HBAs.

thanks,
Ming

#31 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Bart Van Assche
Quoted 13 lines · Bart Van Assche

Hi Ming,

Thanks for having shared these numbers. I think this is very useful information. Do these results show the performance drop that happens if /sys/block/.../device/queue_depth exceeds .can_queue? What I am wondering about is how important these results are in the context of this discussion. Are there any modern SCSI devices for which a SCSI LLD sets scsi_host->can_queue and scsi_host->cmd_per_lun such that the device responds with BUSY? What surprised me is that only three SCSI LLDs call scsi_track_queue_full() (mptsas, bfa, esp_scsi). Does that mean that BUSY responses from a SCSI device or HBA are rare?

Some FC HBAs end up returning busy from ->queuecommand() but I think this is more commonly due to there being and issue with the rport rather than the device.

-Ewan

#32 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

Quoted 13 lines · Ewan D. Milne
Quoted 12 lines · Bart Van Assche

Hi Ming,

Thanks for having shared these numbers. I think this is very useful information. Do these results show the performance drop that happens if /sys/block/.../device/queue_depth exceeds .can_queue? What I am wondering about is how important these results are in the context of this discussion. Are there any modern SCSI devices for which a SCSI LLD sets scsi_host->can_queue and scsi_host->cmd_per_lun such that the device responds with BUSY? What surprised me is that only three SCSI LLDs call scsi_track_queue_full() (mptsas, bfa, esp_scsi). Does that mean that BUSY responses from a SCSI device or HBA are rare?

Quoted 6 lines

Some FC HBAs end up returning busy from ->queuecommand() but I think this is more commonly due to there being and issue with the rport rather than the device.

-Ewan

True - but I would assume busy from queuecommand() is different from BUSY/QUEUE_FULL via a SCSI response.

Adapter queuecommand busy's can be for out-of-resource limits in the driver - such as I_T io count limits enforced by the driver are reached, or if some other adapter resource limit is reached as well. Canqueue covers most of those - but we sometimes overcommit the adapter with a canqueue on physical port as well as per npiv ports, or scsi and nvme on the same port.

Going back to Bart's question - with SANS and multiple initiators sharing a target and lots of luns on that target, it's very common to hit bursty conditions where the target may reply with QUEUE_FULL.  Many arrays provide think tuning guides on how to set up values on multiple hosts, but it's mainly to help the target avoid being completely overrun as some didn't do so well.   In the end, it's very hard to predict multi-initiator load and in a lot of cases, things are usually left a bit overcommitted as the performance downside otherwise is significant.

-- james

#33 lore raw

Re: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD

In reply to Hannes Reinecke

Hannes,

Quoted 2 lines

I must admit I patently don't like this explicit dependency on blk_nonrot().

The whole idea behind using rotational/non-rotational as a trigger for this is a red herring.

Fast devices also have internal queuing limitations. And more
importantly: Fast devices also experience temporary shortages which can lead to congestion.

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering