Skip to contentSkip to conversation

ufs: Do not requeue while ungating the clock

PATCH Applied Bart Van Assche 6 participants 13 messages May 29, 2023 – Jun 8, 2023

Subsystems (2)
lore
Share & tools
lore

Bart Van Assche

author reviewer
#0 lore raw

v4 cover ufs: Do not requeue while ungating the clock

Hi Martin,

In the traces we recorded while testing zoned storage we noticed that UFS commands are requeued while the clock is being ungated. Command requeueing makes it harder than necessary to preserve the command order. Hence this patch series that modifies the SCSI core and also the UFS driver such that clock ungating does not trigger command requeueing.

Please consider this patch series for the next merge window.

Thanks,

Bart.

Changes compared to v3:
- Added a patch that removes two duplicate declarations.

Changes compared to v2:
- Only enable BLK_MQ_F_BLOCKING if clock gating is supported.
- Introduce flag queuecommand_may_block in both the SCSI host and SCSI host template data structures.

Changes compared to v1:
- Dropped patch "scsi: ufs: core: Unexport ufshcd_hold() and ufshcd_release()".
- Removed a ufshcd_scsi_block_requests() / ufshcd_scsi_unblock_requests() pair from patch "scsi: ufs: Ungate the clock synchronously".

Bart Van Assche (4):
scsi: core: Rework scsi_host_block()
scsi: core: Support setting BLK_MQ_F_BLOCKING
scsi: ufs: Conditionally enable the BLK_MQ_F_BLOCKING flag
scsi: ufs: Ungate the clock synchronously

Bart Van Assche (5):
scsi: core: Rework scsi_host_block()
scsi: core: Support setting BLK_MQ_F_BLOCKING
scsi: ufs: Conditionally enable the BLK_MQ_F_BLOCKING flag
scsi: ufs: Declare ufshcd_{hold,release}() once
scsi: ufs: Ungate the clock synchronously

 drivers/scsi/hosts.c             |  1 +
 drivers/scsi/scsi_lib.c          | 27 ++++++----
 drivers/ufs/core/ufs-sysfs.c     |  2 +-
 drivers/ufs/core/ufshcd-crypto.c |  2 +-
 drivers/ufs/core/ufshcd-priv.h   |  3 --
 drivers/ufs/core/ufshcd.c        | 87 ++++++++++----------------------
 include/scsi/scsi_host.h         |  6 +++
 include/ufs/ufshcd.h             |  2 +-
 8 files changed, 54 insertions(+), 76 deletions(-)
author reviewer
#1 lore raw

PATCH v4 1/5 scsi: core: Rework scsi_host_block()

Make scsi_host_block() easier to read by converting it to the widely used early-return style. See also commit f983622ae605 ("scsi: core: Avoid calling synchronize_rcu() for each device in scsi_host_block()").

---
 drivers/scsi/scsi_lib.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)
drivers/scsi/scsi_lib.c +13 -5
index 25489fbd94c6..5f29faa0560f 100644
@@ -2940,11 +2940,20 @@ scsi_target_unblock(struct device *dev, enum scsi_device_state new_state)
 }
 EXPORT_SYMBOL_GPL(scsi_target_unblock);
 
+/**
+ * scsi_host_block - Try to transition all logical units to the SDEV_BLOCK state
+ * @shost: device to block
+ *
+ * Pause SCSI command processing for all logical units associated with the SCSI
+ * host and wait until pending scsi_queue_rq() calls have finished.
+ *
+ * Returns zero if successful or a negative error code upon failure.
+ */
 int
 scsi_host_block(struct Scsi_Host *shost)
 {
 	struct scsi_device *sdev;
-	int ret = 0;
+	int ret;
 
 	/*
 	 * Call scsi_internal_device_block_nowait so we can avoid
@@ -2956,7 +2965,7 @@ scsi_host_block(struct Scsi_Host *shost)
 		mutex_unlock(&sdev->state_mutex);
 		if (ret) {
 			scsi_device_put(sdev);
-			break;
+			return ret;
 		}
 	}
 
@@ -2966,10 +2975,9 @@ scsi_host_block(struct Scsi_Host *shost)
 	 */
 	WARN_ON_ONCE(shost->tag_set.flags & BLK_MQ_F_BLOCKING);
 
-	if (!ret)
-		synchronize_rcu();
+	synchronize_rcu();
 
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL_GPL(scsi_host_block);
 
author reviewer
#2 lore raw

PATCH v4 2/5 scsi: core: Support setting BLK_MQ_F_BLOCKING

In reply to Bart Van Assche

Prepare for adding code in ufshcd_queuecommand() that may sleep. This patch is similar to a patch posted last year by Mike Christie. See also
https://lore.kernel.org/all/[email protected]/

---
 drivers/scsi/hosts.c     |  1 +
 drivers/scsi/scsi_lib.c  | 11 ++++-------
 include/scsi/scsi_host.h |  6 ++++++
 3 files changed, 11 insertions(+), 7 deletions(-)
drivers/scsi/hosts.c +1 -0
index f0bc8bbb3938..198edf03f929 100644
@@ -441,6 +441,7 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv
 	shost->cmd_per_lun = sht->cmd_per_lun;
 	shost->no_write_same = sht->no_write_same;
 	shost->host_tagset = sht->host_tagset;
+	shost->queuecommand_may_block = sht->queuecommand_may_block;
 
 	if (shost_eh_deadline == -1 || !sht->eh_host_reset_handler)
 		shost->eh_deadline = -1;
drivers/scsi/scsi_lib.c +4 -7
index 5f29faa0560f..e0b5d35e7949 100644
@@ -1981,6 +1981,8 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
 	tag_set->flags = BLK_MQ_F_SHOULD_MERGE;
 	tag_set->flags |=
 		BLK_ALLOC_POLICY_TO_MQ_FLAG(shost->hostt->tag_alloc_policy);
+	if (shost->queuecommand_may_block)
+		tag_set->flags |= BLK_MQ_F_BLOCKING;
 	tag_set->driver_data = shost;
 	if (shost->host_tagset)
 		tag_set->flags |= BLK_MQ_F_TAG_HCTX_SHARED;
@@ -2969,13 +2971,8 @@ scsi_host_block(struct Scsi_Host *shost)
 		}
 	}
 
-	/*
-	 * SCSI never enables blk-mq's BLK_MQ_F_BLOCKING flag so
-	 * calling synchronize_rcu() once is enough.
-	 */
-	WARN_ON_ONCE(shost->tag_set.flags & BLK_MQ_F_BLOCKING);
-
-	synchronize_rcu();
+	/* Wait for ongoing scsi_queue_rq() calls to finish. */
+	blk_mq_wait_quiesce_done(&shost->tag_set);
 
 	return 0;
 }
include/scsi/scsi_host.h +6 -0
index 0f29799efa02..70b7475dcf56 100644
@@ -458,6 +458,9 @@ struct scsi_host_template {
 	/* True if the host uses host-wide tagspace */
 	unsigned host_tagset:1;
 
+	/* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */
+	unsigned queuecommand_may_block:1;
+
 	/*
 	 * Countdown for host blocking with no commands outstanding.
 	 */
@@ -653,6 +656,9 @@ struct Scsi_Host {
 	/* True if the host uses host-wide tagspace */
 	unsigned host_tagset:1;
 
+	/* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */
+	unsigned queuecommand_may_block:1;
+
 	/* Host responded with short (<36 bytes) INQUIRY result */
 	unsigned short_inquiry:1;
 
author reviewer
#3 lore raw

PATCH v4 3/5 scsi: ufs: Conditionally enable the BLK_MQ_F_BLOCKING flag

In reply to Bart Van Assche

Prepare for adding code in ufshcd_queuecommand() that may sleep.

---
 drivers/ufs/core/ufshcd.c | 1 +
 1 file changed, 1 insertion(+)
drivers/ufs/core/ufshcd.c +1 -0
index abe9a430cc37..c2d9109102f3 100644
@@ -10187,6 +10187,7 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq)
 	host->max_channel = UFSHCD_MAX_CHANNEL;
 	host->unique_id = host->host_no;
 	host->max_cmd_len = UFS_CDB_SIZE;
+	host->queuecommand_may_block = !!(hba->caps & UFSHCD_CAP_CLK_GATING);
 
 	hba->max_pwr_info.is_valid = false;
 
author reviewer
#5 lore raw

PATCH v4 4/5 scsi: ufs: Declare ufshcd_{hold,release}() once

In reply to Bart Van Assche

ufshcd_hold() and ufshcd_release are declared twice: once in drivers/ufs/core/ufshcd-priv.h and a second time in include/ufs/ufshcd.h. Remove the declarations from ufshcd-priv.h.

Fixes dd11376b9f1b ("scsi: ufs: Split the drivers/scsi/ufs directory")
---
 drivers/ufs/core/ufshcd-priv.h | 3 ---
 1 file changed, 3 deletions(-)
drivers/ufs/core/ufshcd-priv.h +0 -3
index d53b93c21a0c..8f58c2169398 100644
@@ -84,9 +84,6 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,
 int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
 			    u8 **buf, bool ascii);
 
-int ufshcd_hold(struct ufs_hba *hba, bool async);
-void ufshcd_release(struct ufs_hba *hba);
-
 int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd);
 
 int ufshcd_exec_raw_upiu_cmd(struct ufs_hba *hba,
Reviewed-by on v4 4/5
#6 lore raw

Re: [PATCH v4 4/5] scsi: ufs: Declare ufshcd_{hold,release}() once

Quoted 6 lines · Bart Van Assche

ufshcd_hold() and ufshcd_release are declared twice: once in drivers/ufs/core/ufshcd-priv.h and a second time in include/ufs/ufshcd.h. Remove the declarations from ufshcd-priv.h.

Fixes dd11376b9f1b ("scsi: ufs: Split the drivers/scsi/ufs directory")
Quoted 18 lines
---
 drivers/ufs/core/ufshcd-priv.h | 3 ---
 1 file changed, 3 deletions(-)
drivers/ufs/core/ufshcd-priv.h +0 -3
index d53b93c21a0c..8f58c2169398 100644
@@ -84,9 +84,6 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,
 int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
 			    u8 **buf, bool ascii);
 
-int ufshcd_hold(struct ufs_hba *hba, bool async);
-void ufshcd_release(struct ufs_hba *hba);
-
 int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd);
 
 int ufshcd_exec_raw_upiu_cmd(struct ufs_hba *hba,
Reviewed-by on v4 4/5
#7 lore raw

RE: [PATCH v4 4/5] scsi: ufs: Declare ufshcd_{hold,release}() once

In reply to Bart Van Assche
Quoted 6 lines

ufshcd_hold() and ufshcd_release are declared twice: once in drivers/ufs/core/ufshcd-priv.h and a second time in include/ufs/ufshcd.h. Remove the declarations from ufshcd-priv.h.

Fixes dd11376b9f1b ("scsi: ufs: Split the drivers/scsi/ufs directory")

Best Regards,
Keoseong

Quoted 18 lines

---
drivers/ufs/core/ufshcd-priv.h | 3 ---

1 file changed, 3 deletions(-)
drivers/ufs/core/ufshcd-priv.h +0 -3
index d53b93c21a0c..8f58c2169398 100644
@@ -84,9 +84,6 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,
 int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
 			    u8 **buf, bool ascii);
 
-int ufshcd_hold(struct ufs_hba *hba, bool async);
-void ufshcd_release(struct ufs_hba *hba);
-
 int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd);
 
 int ufshcd_exec_raw_upiu_cmd(struct ufs_hba *hba,
author reviewer
#8 lore raw

PATCH v4 5/5 scsi: ufs: Ungate the clock synchronously

In reply to Bart Van Assche

Ungating the clock asynchronously causes ufshcd_queuecommand() to return SCSI_MLQUEUE_HOST_BUSY and hence causes commands to be requeued. This is suboptimal. Allow ufshcd_queuecommand() to sleep such that clock ungating does not trigger command requeuing. Remove the ufshcd_scsi_block_requests() and ufshcd_scsi_unblock_requests() calls because these are no longer needed. The flush_work(&hba->clk_gating.ungate_work) call is sufficient to make the SCSI core wait for clock ungating to complete.

---
 drivers/ufs/core/ufs-sysfs.c     |  2 +-
 drivers/ufs/core/ufshcd-crypto.c |  2 +-
 drivers/ufs/core/ufshcd.c        | 86 ++++++++++----------------------
 include/ufs/ufshcd.h             |  2 +-
 4 files changed, 30 insertions(+), 62 deletions(-)
drivers/ufs/core/ufs-sysfs.c +1 -1
index 883f0e44b54e..cdf3d5f2b77b 100644
@@ -168,7 +168,7 @@ static ssize_t auto_hibern8_show(struct device *dev,
 	}
 
 	pm_runtime_get_sync(hba->dev);
-	ufshcd_hold(hba, false);
+	ufshcd_hold(hba);
 	ahit = ufshcd_readl(hba, REG_AUTO_HIBERNATE_IDLE_TIMER);
 	ufshcd_release(hba);
 	pm_runtime_put_sync(hba->dev);
drivers/ufs/core/ufshcd-crypto.c +1 -1
index 198360fe5e8e..f2c4422cab86 100644
@@ -24,7 +24,7 @@ static int ufshcd_program_key(struct ufs_hba *hba,
 	u32 slot_offset = hba->crypto_cfg_register + slot * sizeof(*cfg);
 	int err = 0;
 
-	ufshcd_hold(hba, false);
+	ufshcd_hold(hba);
 
 	if (hba->vops && hba->vops->program_key) {
 		err = hba->vops->program_key(hba, cfg, slot);
drivers/ufs/core/ufshcd.c +27 -59

270 lines not on this page · raw message

include/ufs/ufshcd.h +1 -1
index db2e669985d5..40c537a3880b 100644
@@ -1357,7 +1357,7 @@ void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
 int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
 			    u8 **buf, bool ascii);
 
-int ufshcd_hold(struct ufs_hba *hba, bool async);
+void ufshcd_hold(struct ufs_hba *hba);
 void ufshcd_release(struct ufs_hba *hba);
 
 void ufshcd_clkgate_delay_set(struct device *dev, unsigned long value);
Reviewed-by on v4 5/5
#9 lore raw

Re: [PATCH v4 5/5] scsi: ufs: Ungate the clock synchronously

Quoted 13 lines · Bart Van Assche

Ungating the clock asynchronously causes ufshcd_queuecommand() to return
SCSI_MLQUEUE_HOST_BUSY and hence causes commands to be requeued. This is
suboptimal. Allow ufshcd_queuecommand() to sleep such that clock ungating
does not trigger command requeuing. Remove the ufshcd_scsi_block_requests()
and ufshcd_scsi_unblock_requests() calls because these are no longer needed. The flush_work(&hba->clk_gating.ungate_work) call is sufficient to
make the SCSI core wait for clock ungating to complete.

Quoted 2 lines
Reviewed-by on v4 5/5
#10 lore raw

Re: [PATCH v4 5/5] scsi: ufs: Ungate the clock synchronously

In reply to Bart Van Assche
Quoted 10 lines · Bart Van Assche

Ungating the clock asynchronously causes ufshcd_queuecommand() to return SCSI_MLQUEUE_HOST_BUSY and hence causes commands to be requeued. This is suboptimal. Allow ufshcd_queuecommand() to sleep such that clock ungating does not trigger command requeuing. Remove the ufshcd_scsi_block_requests() and ufshcd_scsi_unblock_requests() calls because these are no longer needed. The flush_work(&hba->clk_gating.ungate_work) call is sufficient to make the SCSI core wait for clock ungating to complete.

#11 lore raw
In reply to Bart Van Assche

Bart,

Quoted 6 lines

In the traces we recorded while testing zoned storage we noticed that UFS commands are requeued while the clock is being ungated. Command requeueing makes it harder than necessary to preserve the command order. Hence this patch series that modifies the SCSI core and also the UFS driver such that clock ungating does not trigger command requeueing.

Applied to 6.5/scsi-staging, thanks!

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#12 lore raw
In reply to Bart Van Assche
Quoted 9 lines · Bart Van Assche

In the traces we recorded while testing zoned storage we noticed that UFS commands are requeued while the clock is being ungated. Command requeueing makes it harder than necessary to preserve the command order. Hence this patch series that modifies the SCSI core and also the UFS driver such that clock ungating does not trigger command requeueing.

Please consider this patch series for the next merge window.

[...]

Applied to 6.5/scsi-queue, thanks!

[1/5] scsi: core: Rework scsi_host_block()

      https://git.kernel.org/mkp/scsi/c/c854bcdf5e18
[2/5] scsi: core: Support setting BLK_MQ_F_BLOCKING
      https://git.kernel.org/mkp/scsi/c/b125bb99559e
[3/5] scsi: ufs: Conditionally enable the BLK_MQ_F_BLOCKING flag
      https://git.kernel.org/mkp/scsi/c/6c03c8e9b729
[4/5] scsi: ufs: Declare ufshcd_{hold,release}() once
      https://git.kernel.org/mkp/scsi/c/4b68b7f9c46d
[5/5] scsi: ufs: Ungate the clock synchronously
      https://git.kernel.org/mkp/scsi/c/078f4f4b34d6
Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering