Skip to contentSkip to conversation

ufs: Do not requeue while ungating the clock

PATCH Superseded Bart Van Assche 4 participants 13 messages May 4, 2023 – May 17, 2023

Subsystems (2)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Bart Van Assche

author reviewer
#0 lore raw

cover ufs: Do not requeue while ungating the clock

Hi Martin,

In the traces we recorded while testing zoned storage we noticed that UFS commands are requeued while the clock is being ungated. Command requeueing makes it harder than necessary to preserve the command order. Hence this patch series that modifies the SCSI core and also the UFS driver such that clock ungating does not trigger command requeueing.

Please consider this patch series for the next merge window.

Thanks,

Bart.

Bart Van Assche (5):
scsi: core: Rework scsi_host_block()
scsi: core: Support setting BLK_MQ_F_BLOCKING
scsi: ufs: Enable the BLK_MQ_F_BLOCKING flag
scsi: ufs: core: Unexport ufshcd_hold() and ufshcd_release()
scsi: ufs: Ungate the clock synchronously

 drivers/scsi/scsi_lib.c          | 26 ++++++-----
 drivers/ufs/core/ufs-sysfs.c     |  2 +-
 drivers/ufs/core/ufshcd-crypto.c |  2 +-
 drivers/ufs/core/ufshcd-priv.h   |  2 +-
 drivers/ufs/core/ufshcd.c        | 78 ++++++++++----------------------
 include/scsi/scsi_host.h         |  3 ++
 include/ufs/ufshcd.h             |  3 --
 7 files changed, 45 insertions(+), 71 deletions(-)
author reviewer
#1 lore raw

PATCH 1/5 scsi: core: Rework scsi_host_block()

Make scsi_host_block() easier to read by converting it to the widely used early-return style. This patch reworks code introduced by commit f983622ae605 ("scsi: core: Avoid calling synchronize_rcu() for each device in scsi_host_block()").

Cc Ming Lei <[email protected]>
Cc Ye Bin <[email protected]>
Cc Hannes Reinecke <[email protected]>
---
 drivers/scsi/scsi_lib.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)
drivers/scsi/scsi_lib.c +13 -5
index b7c569a42aa4..758a57616dd3 100644
@@ -2939,11 +2939,20 @@ scsi_target_unblock(struct device *dev, enum scsi_device_state new_state)
 }
 EXPORT_SYMBOL_GPL(scsi_target_unblock);
 
+/**
+ * scsi_host_block - Try to transition all logical units to the SDEV_BLOCK state
+ * @shost: device to block
+ *
+ * Pause SCSI command processing for all logical units associated with the SCSI
+ * host and wait until pending scsi_queue_rq() calls have finished.
+ *
+ * Returns zero if successful or a negative error code upon failure.
+ */
 int
 scsi_host_block(struct Scsi_Host *shost)
 {
 	struct scsi_device *sdev;
-	int ret = 0;
+	int ret;
 
 	/*
 	 * Call scsi_internal_device_block_nowait so we can avoid
@@ -2955,7 +2964,7 @@ scsi_host_block(struct Scsi_Host *shost)
 		mutex_unlock(&sdev->state_mutex);
 		if (ret) {
 			scsi_device_put(sdev);
-			break;
+			return ret;
 		}
 	}
 
@@ -2965,10 +2974,9 @@ scsi_host_block(struct Scsi_Host *shost)
 	 */
 	WARN_ON_ONCE(shost->tag_set.flags & BLK_MQ_F_BLOCKING);
 
-	if (!ret)
-		synchronize_rcu();
+	synchronize_rcu();
 
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL_GPL(scsi_host_block);
 
Reviewed-by on v1 1/5
#2 lore raw

Re: [PATCH 1/5] scsi: core: Rework scsi_host_block()

Quoted 51 lines · Bart Van Assche

Make scsi_host_block() easier to read by converting it to the widely used early-return style. This patch reworks code introduced by commit f983622ae605 ("scsi: core: Avoid calling synchronize_rcu() for each device in scsi_host_block()").

Cc Ming Lei <[email protected]>
Cc Ye Bin <[email protected]>
Cc Hannes Reinecke <[email protected]>
---
 drivers/scsi/scsi_lib.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)
drivers/scsi/scsi_lib.c +11 -2
index b7c569a42aa4..758a57616dd3 100644
@@ -2939,11 +2939,20 @@ scsi_target_unblock(struct device *dev, enum scsi_device_state new_state)
 }
 EXPORT_SYMBOL_GPL(scsi_target_unblock);
 
+/**
+ * scsi_host_block - Try to transition all logical units to the SDEV_BLOCK state
+ * @shost: device to block
+ *
+ * Pause SCSI command processing for all logical units associated with the SCSI
+ * host and wait until pending scsi_queue_rq() calls have finished.
+ *
+ * Returns zero if successful or a negative error code upon failure.
+ */
 int
 scsi_host_block(struct Scsi_Host *shost)
 {
 	struct scsi_device *sdev;
-	int ret = 0;
+	int ret;
 
 	/*
 	 * Call scsi_internal_device_block_nowait so we can avoid
@@ -2955,7 +2964,7 @@ scsi_host_block(struct Scsi_Host *shost)
 		mutex_unlock(&sdev->state_mutex);
 		if (ret) {
 			scsi_device_put(sdev);
-			break;
+			return ret;
 		}
 	}
 
@@ -2965,10 +2974,9 @@ scsi_host_block(struct Scsi_Host *shost)
 	 */
 	WARN_ON_ONCE(shost->tag_set.flags & BLK_MQ_F_BLOCKING);

drivers/scsi/scsi_lib.c scsi_host_block(struct Scsi_Host *shost) · line 2982

Quoted 10 lines
drivers/scsi/scsi_lib.c +2 -3
 
-	if (!ret)
-		synchronize_rcu();
+	synchronize_rcu();
 
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL_GPL(scsi_host_block);
 
author reviewer
#3 lore raw

PATCH 2/5 scsi: core: Support setting BLK_MQ_F_BLOCKING

In reply to Bart Van Assche

Prepare for adding code in ufshcd_queuecommand() that may sleep. This patch is similar to a patch posted last year by Mike Christie. See also
https://lore.kernel.org/all/[email protected]/

---
 drivers/scsi/scsi_lib.c  | 10 +++-------
 include/scsi/scsi_host.h |  3 +++
 2 files changed, 6 insertions(+), 7 deletions(-)
drivers/scsi/scsi_lib.c +3 -7
index 758a57616dd3..894af68babc2 100644
@@ -1982,6 +1982,8 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
 	tag_set->flags = BLK_MQ_F_SHOULD_MERGE;
 	tag_set->flags |=
 		BLK_ALLOC_POLICY_TO_MQ_FLAG(shost->hostt->tag_alloc_policy);
+	if (shost->hostt->queuecommand_may_block)
+		tag_set->flags |= BLK_MQ_F_BLOCKING;
 	tag_set->driver_data = shost;
 	if (shost->host_tagset)
 		tag_set->flags |= BLK_MQ_F_TAG_HCTX_SHARED;
@@ -2968,13 +2970,7 @@ scsi_host_block(struct Scsi_Host *shost)
 		}
 	}
 
-	/*
-	 * SCSI never enables blk-mq's BLK_MQ_F_BLOCKING flag so
-	 * calling synchronize_rcu() once is enough.
-	 */
-	WARN_ON_ONCE(shost->tag_set.flags & BLK_MQ_F_BLOCKING);
-
-	synchronize_rcu();
+	blk_mq_wait_quiesce_done(&shost->tag_set);
 
 	return 0;
 }
include/scsi/scsi_host.h +3 -0
index 0f29799efa02..37a8a2608dc2 100644
@@ -458,6 +458,9 @@ struct scsi_host_template {
 	/* True if the host uses host-wide tagspace */
 	unsigned host_tagset:1;
 
+	/* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */
+	unsigned queuecommand_may_block:1;
+
 	/*
 	 * Countdown for host blocking with no commands outstanding.
 	 */
Reviewed-by on v1 2/5
#4 lore raw

Re: [PATCH 2/5] scsi: core: Support setting BLK_MQ_F_BLOCKING

Quoted 26 lines · Bart Van Assche

Prepare for adding code in ufshcd_queuecommand() that may sleep. This patch is similar to a patch posted last year by Mike Christie. See also
https://lore.kernel.org/all/[email protected]/

---
 drivers/scsi/scsi_lib.c  | 10 +++-------
 include/scsi/scsi_host.h |  3 +++
 2 files changed, 6 insertions(+), 7 deletions(-)
drivers/scsi/scsi_lib.c +2 -0
index 758a57616dd3..894af68babc2 100644
@@ -1982,6 +1982,8 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
 	tag_set->flags = BLK_MQ_F_SHOULD_MERGE;
 	tag_set->flags |=
 		BLK_ALLOC_POLICY_TO_MQ_FLAG(shost->hostt->tag_alloc_policy);
+	if (shost->hostt->queuecommand_may_block)
+		tag_set->flags |= BLK_MQ_F_BLOCKING;
 	tag_set->driver_data = shost;
 	if (shost->host_tagset)
 		tag_set->flags |= BLK_MQ_F_TAG_HCTX_SHARED;
@@ -2968,13 +2970,7 @@ scsi_host_block(struct Scsi_Host *shost)
 		}

drivers/scsi/scsi_lib.c scsi_host_block(struct Scsi_Host *shost) · line 2973

Quoted 10 lines
drivers/scsi/scsi_lib.c +1 -7
 	}
 
-	/*
-	 * SCSI never enables blk-mq's BLK_MQ_F_BLOCKING flag so
-	 * calling synchronize_rcu() once is enough.
-	 */
-	WARN_ON_ONCE(shost->tag_set.flags & BLK_MQ_F_BLOCKING);
-
-	synchronize_rcu();
+	blk_mq_wait_quiesce_done(&shost->tag_set);
Quoted 17 lines
drivers/scsi/scsi_lib.c +0 -0
 	return 0;
 }
include/scsi/scsi_host.h +3 -0
index 0f29799efa02..37a8a2608dc2 100644
@@ -458,6 +458,9 @@ struct scsi_host_template {
 	/* True if the host uses host-wide tagspace */
 	unsigned host_tagset:1;
 
+	/* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */
+	unsigned queuecommand_may_block:1;
+
 	/*
 	 * Countdown for host blocking with no commands outstanding.
 	 */
author reviewer
#5 lore raw

PATCH 3/5 scsi: ufs: Enable the BLK_MQ_F_BLOCKING flag

In reply to Bart Van Assche

Prepare for adding code in ufshcd_queuecommand() that may sleep.

---
 drivers/ufs/core/ufshcd.c | 1 +
 1 file changed, 1 insertion(+)
drivers/ufs/core/ufshcd.c +1 -0
index 2b8c2613f7d7..a1bce9c6aee5 100644
@@ -8755,6 +8755,7 @@ static const struct scsi_host_template ufshcd_driver_template = {
 	.max_host_blocked	= 1,
 	.track_queue_depth	= 1,
 	.skip_settle_delay	= 1,
+	.queuecommand_may_block = true,
 	.sdev_groups		= ufshcd_driver_groups,
 	.rpm_autosuspend_delay	= RPM_AUTOSUSPEND_DELAY_MS,
 };
author reviewer
#6 lore raw

PATCH 4/5 scsi: ufs: core: Unexport ufshcd_hold() and ufshcd_release()

In reply to Bart Van Assche

Unexport these functions since these are only used by the UFS core.

---
 drivers/ufs/core/ufshcd.c | 2 --
 include/ufs/ufshcd.h      | 3 ---
 2 files changed, 5 deletions(-)
drivers/ufs/core/ufshcd.c +0 -2
index a1bce9c6aee5..54f91d7d0192 100644
@@ -1847,7 +1847,6 @@ int ufshcd_hold(struct ufs_hba *hba, bool async)
 out:
 	return rc;
 }
-EXPORT_SYMBOL_GPL(ufshcd_hold);
 
 static void ufshcd_gate_work(struct work_struct *work)
 {
@@ -1950,7 +1949,6 @@ void ufshcd_release(struct ufs_hba *hba)
 	__ufshcd_release(hba);
 	spin_unlock_irqrestore(hba->host->host_lock, flags);
 }
-EXPORT_SYMBOL_GPL(ufshcd_release);
 
 static ssize_t ufshcd_clkgate_delay_show(struct device *dev,
 		struct device_attribute *attr, char *buf)
include/ufs/ufshcd.h +0 -3
index d6da1efb0212..13824462452d 100644
@@ -1358,9 +1358,6 @@ void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
 int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
 			    u8 **buf, bool ascii);
 
-int ufshcd_hold(struct ufs_hba *hba, bool async);
-void ufshcd_release(struct ufs_hba *hba);
-
 void ufshcd_clkgate_delay_set(struct device *dev, unsigned long value);
 
 u32 ufshcd_get_local_unipro_ver(struct ufs_hba *hba);
#7 lore raw

Re: [PATCH 4/5] scsi: ufs: core: Unexport ufshcd_hold() and ufshcd_release()

Quoted 1 line · Bart Van Assche

Unexport these functions since these are only used by the UFS core.

Hi Bart,
Qualcomm uses these ufshcd_hold() and ufshcd_release() functions in ufs-qcom.c. I am going to post Qualcomm's change soon.

Thanks,
Bao

Quoted 41 lines
---
  drivers/ufs/core/ufshcd.c | 2 --
  include/ufs/ufshcd.h      | 3 ---
  2 files changed, 5 deletions(-)
drivers/ufs/core/ufshcd.c +0 -2
index a1bce9c6aee5..54f91d7d0192 100644
@@ -1847,7 +1847,6 @@ int ufshcd_hold(struct ufs_hba *hba, bool async)
  out:
  	return rc;
  }
-EXPORT_SYMBOL_GPL(ufshcd_hold);
 
  static void ufshcd_gate_work(struct work_struct *work)
  {
@@ -1950,7 +1949,6 @@ void ufshcd_release(struct ufs_hba *hba)
  	__ufshcd_release(hba);
  	spin_unlock_irqrestore(hba->host->host_lock, flags);
  }
-EXPORT_SYMBOL_GPL(ufshcd_release);
 
  static ssize_t ufshcd_clkgate_delay_show(struct device *dev,
  		struct device_attribute *attr, char *buf)
include/ufs/ufshcd.h +0 -3
index d6da1efb0212..13824462452d 100644
@@ -1358,9 +1358,6 @@ void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
  int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
  			    u8 **buf, bool ascii);
 
-int ufshcd_hold(struct ufs_hba *hba, bool async);
-void ufshcd_release(struct ufs_hba *hba);
-
  void ufshcd_clkgate_delay_set(struct device *dev, unsigned long value);
 
  u32 ufshcd_get_local_unipro_ver(struct ufs_hba *hba);
author reviewer
#8 lore raw

Re: [PATCH 4/5] scsi: ufs: core: Unexport ufshcd_hold() and ufshcd_release()

Quoted 4 lines · Bao D. Nguyen
Quoted 1 line · Bart Van Assche

Unexport these functions since these are only used by the UFS core.

Qualcomm uses these ufshcd_hold() and ufshcd_release() functions in ufs-qcom.c. I am going to post Qualcomm's change soon.

Hi Bao,

I will drop this patch.

Does the code that you plan to post pass 'false' or 'true' as second argument to ufshcd_hold()?

Thanks,

Bart.

#9 lore raw

Re: [PATCH 4/5] scsi: ufs: core: Unexport ufshcd_hold() and ufshcd_release()

Quoted 4 lines · Bart Van Assche
Quoted 3 lines · Bao D. Nguyen
Quoted 1 line · Bart Van Assche

Unexport these functions since these are only used by the UFS core.

Quoted 8 lines
Quoted 1 line

Qualcomm uses these ufshcd_hold() and ufshcd_release() functions in ufs-qcom.c. I am going to post Qualcomm's change soon.

Hi Bao,

I will drop this patch.

Does the code that you plan to post pass 'false' or 'true' as second argument to ufshcd_hold()?

Thanks Bart. We are using ufshcd_hold(hba, false);

Thanks,
Bao

Quoted 4 lines

Thanks,

Bart.

author reviewer
#10 lore raw

PATCH 5/5 scsi: ufs: Ungate the clock synchronously

In reply to Bart Van Assche

Ungating the clock asynchronously causes ufshcd_queuecommand() to return SCSI_MLQUEUE_HOST_BUSY and hence causes commands to be requeued. This is suboptimal. Allow ufshcd_queuecommand() to sleep such that clock ungating does not trigger command requeuing.

---
 drivers/ufs/core/ufs-sysfs.c     |  2 +-
 drivers/ufs/core/ufshcd-crypto.c |  2 +-
 drivers/ufs/core/ufshcd-priv.h   |  2 +-
 drivers/ufs/core/ufshcd.c        | 75 ++++++++++----------------------
 4 files changed, 26 insertions(+), 55 deletions(-)
drivers/ufs/core/ufs-sysfs.c +1 -1
index 883f0e44b54e..cdf3d5f2b77b 100644
@@ -168,7 +168,7 @@ static ssize_t auto_hibern8_show(struct device *dev,
 	}
 
 	pm_runtime_get_sync(hba->dev);
-	ufshcd_hold(hba, false);
+	ufshcd_hold(hba);
 	ahit = ufshcd_readl(hba, REG_AUTO_HIBERNATE_IDLE_TIMER);
 	ufshcd_release(hba);
 	pm_runtime_put_sync(hba->dev);
drivers/ufs/core/ufshcd-crypto.c +1 -1
index 198360fe5e8e..f2c4422cab86 100644
@@ -24,7 +24,7 @@ static int ufshcd_program_key(struct ufs_hba *hba,
 	u32 slot_offset = hba->crypto_cfg_register + slot * sizeof(*cfg);
 	int err = 0;
 
-	ufshcd_hold(hba, false);
+	ufshcd_hold(hba);
 
 	if (hba->vops && hba->vops->program_key) {
 		err = hba->vops->program_key(hba, cfg, slot);
drivers/ufs/core/ufshcd-priv.h +1 -1
index d53b93c21a0c..22cac71090ae 100644
@@ -84,7 +84,7 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,
 int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index,
 			    u8 **buf, bool ascii);
 
-int ufshcd_hold(struct ufs_hba *hba, bool async);
+void ufshcd_hold(struct ufs_hba *hba);
 void ufshcd_release(struct ufs_hba *hba);
 
 int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd);
drivers/ufs/core/ufshcd.c +23 -52

241 lines not on this page · raw message

reviewer
#11 lore raw

RE: [PATCH 5/5] scsi: ufs: Ungate the clock synchronously

drivers/ufs/core/ufshcd.c static void ufshcd_ungate_work(struct work_struct *work) · line 1766

Quoted 3 lines
drivers/ufs/core/ufshcd.c +1 -1
-int ufshcd_hold(struct ufs_hba *hba, bool async)
+void ufshcd_hold(struct ufs_hba *hba)
 {

How about switching to the block quiescing API as well - blk_mq_{un}quiesce_tagset, Instead of scsi_{un}block_requests?

Thanks,
Avri

author reviewer
#12 lore raw

Re: [PATCH 5/5] scsi: ufs: Ungate the clock synchronously

Quoted 5 lines · Avri Altman

drivers/ufs/core/ufshcd.c static void ufshcd_ungate_work(struct work_struct *work) · line 1766

Quoted 3 lines
drivers/ufs/core/ufshcd.c +1 -1
-int ufshcd_hold(struct ufs_hba *hba, bool async)
+void ufshcd_hold(struct ufs_hba *hba)
  {

How about switching to the block quiescing API as well - blk_mq_{un}quiesce_tagset, Instead of scsi_{un}block_requests?

Hi Avri,

Did you perhaps want to refer to blk_mq_freeze_queue()? ufshcd_scsi_block_requests() may be called from inside ufshcd_queuecommand(). Calling blk_mq_freeze_queue() from inside ufshcd_queuecommand() would cause a deadlock.

Thanks,

Bart.