Skip to contentSkip to conversation

scsi: ufs: add some fixes

PATCH Applied Jaegeuk Kim 3 participants 13 messages Nov 17, 2020 – Nov 24, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v5 cover scsi: ufs: add some fixes

Change log from v4:
- add more fixes

Change log from v3:
- use __ufshcd_release with a fix in __ufshcd_release

Change log from v2:
- use active_req-- instead of __ufshcd_release to avoid UFS timeout

Change log from v1:
- remove clkgating_enable check in __ufshcd_release
- use __uhfshcd_release instead of active_req.

#1 lore raw

PATCH v5 1/7 scsi: ufs: avoid to call REQ_CLKS_OFF to CLKS_OFF

Once UFS was gated with CLKS_OFF, it should not call REQ_CLKS_OFF again, which caused hibern8_enter failure.

---
 drivers/scsi/ufs/ufshcd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +2 -1
index b8f573a02713..cc8d5f0c3fdc 100644
@@ -1745,7 +1745,8 @@ static void __ufshcd_release(struct ufs_hba *hba)
 	if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
 	    hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
 	    ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
-	    hba->active_uic_cmd || hba->uic_async_done)
+	    hba->active_uic_cmd || hba->uic_async_done ||
+	    hba->clk_gating.state == CLKS_OFF)
 		return;
 
 	hba->clk_gating.state = REQ_CLKS_OFF;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Reviewed-by on v5 1/7
#2 lore raw

Re: [PATCH v5 1/7] scsi: ufs: avoid to call REQ_CLKS_OFF to CLKS_OFF

Quoted 4 lines · Jaegeuk Kim

Once UFS was gated with CLKS_OFF, it should not call REQ_CLKS_OFF again, which caused hibern8_enter failure.

Quoted 18 lines
---
 drivers/scsi/ufs/ufshcd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +2 -1
index b8f573a02713..cc8d5f0c3fdc 100644
@@ -1745,7 +1745,8 @@ static void __ufshcd_release(struct ufs_hba *hba)
 	if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
 	    hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
 	    ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
-	    hba->active_uic_cmd || hba->uic_async_done)
+	    hba->active_uic_cmd || hba->uic_async_done ||
+	    hba->clk_gating.state == CLKS_OFF)
 		return;
 
 	hba->clk_gating.state = REQ_CLKS_OFF;
#3 lore raw

PATCH v5 2/7 scsi: ufs: atomic update for clkgating_enable

In reply to Jaegeuk Kim

From: Jaegeuk Kim <[email protected]>

When giving a stress test which enables/disables clkgating, we hit device timeout sometimes. This patch avoids subtle racy condition to address it.

Note that, this requires a patch to address the device stuck by REQ_CLKS_OFF in __ufshcd_release().

The fix is "scsi: ufs: avoid to call REQ_CLKS_OFF to CLKS_OFF".

---
 drivers/scsi/ufs/ufshcd.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +6 -6
index cc8d5f0c3fdc..6c9269bffcbd 100644
@@ -1808,19 +1808,19 @@ static ssize_t ufshcd_clkgate_enable_store(struct device *dev,
 		return -EINVAL;
 
 	value = !!value;
+
+	spin_lock_irqsave(hba->host->host_lock, flags);
 	if (value == hba->clk_gating.is_enabled)
 		goto out;
 
-	if (value) {
-		ufshcd_release(hba);
-	} else {
-		spin_lock_irqsave(hba->host->host_lock, flags);
+	if (value)
+		__ufshcd_release(hba);
+	else
 		hba->clk_gating.active_reqs++;
-		spin_unlock_irqrestore(hba->host->host_lock, flags);
-	}
 
 	hba->clk_gating.is_enabled = value;
 out:
+	spin_unlock_irqrestore(hba->host->host_lock, flags);
 	return count;
 }
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Reviewed-by on v5 2/7
#4 lore raw

Re: [PATCH v5 2/7] scsi: ufs: atomic update for clkgating_enable

Quoted 11 lines · Jaegeuk Kim

From: Jaegeuk Kim <[email protected]>

When giving a stress test which enables/disables clkgating, we hit device timeout sometimes. This patch avoids subtle racy condition to address it.

Note that, this requires a patch to address the device stuck by REQ_CLKS_OFF in __ufshcd_release().

The fix is "scsi: ufs: avoid to call REQ_CLKS_OFF to CLKS_OFF".

Quoted 34 lines
---
 drivers/scsi/ufs/ufshcd.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +6 -6
index cc8d5f0c3fdc..6c9269bffcbd 100644
@@ -1808,19 +1808,19 @@ static ssize_t
 ufshcd_clkgate_enable_store(struct device *dev,
 		return -EINVAL;
 
 	value = !!value;
+
+	spin_lock_irqsave(hba->host->host_lock, flags);
 	if (value == hba->clk_gating.is_enabled)
 		goto out;
 
-	if (value) {
-		ufshcd_release(hba);
-	} else {
-		spin_lock_irqsave(hba->host->host_lock, flags);
+	if (value)
+		__ufshcd_release(hba);
+	else
 		hba->clk_gating.active_reqs++;
-		spin_unlock_irqrestore(hba->host->host_lock, flags);
-	}
 
 	hba->clk_gating.is_enabled = value;
 out:
+	spin_unlock_irqrestore(hba->host->host_lock, flags);
 	return count;
 }
#5 lore raw

PATCH v5 3/7 scsi: ufs: clear UAC for FFU and RPMB LUNs

In reply to Jaegeuk Kim

From: Jaegeuk Kim <[email protected]>

In order to conduct FFU or RPMB operations, UFS needs to clear UAC. This patch clears it explicitly, so that we could get no failure given early execution.

---
 drivers/scsi/ufs/ufshcd.c | 70 +++++++++++++++++++++++++++++++++++----
 drivers/scsi/ufs/ufshcd.h |  1 +
 2 files changed, 65 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +64 -6
index 6c9269bffcbd..8e696ca79b40 100644
@@ -7058,7 +7058,6 @@ static inline void ufshcd_blk_pm_runtime_init(struct scsi_device *sdev)
 static int ufshcd_scsi_add_wlus(struct ufs_hba *hba)
 {
 	int ret = 0;
-	struct scsi_device *sdev_rpmb;
 	struct scsi_device *sdev_boot;
 
 	hba->sdev_ufs_device = __scsi_add_device(hba->host, 0, 0,
@@ -7071,14 +7070,14 @@ static int ufshcd_scsi_add_wlus(struct ufs_hba *hba)
 	ufshcd_blk_pm_runtime_init(hba->sdev_ufs_device);
 	scsi_device_put(hba->sdev_ufs_device);
 
-	sdev_rpmb = __scsi_add_device(hba->host, 0, 0,
+	hba->sdev_rpmb = __scsi_add_device(hba->host, 0, 0,
 		ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_RPMB_WLUN), NULL);
-	if (IS_ERR(sdev_rpmb)) {
-		ret = PTR_ERR(sdev_rpmb);
+	if (IS_ERR(hba->sdev_rpmb)) {
+		ret = PTR_ERR(hba->sdev_rpmb);
 		goto remove_sdev_ufs_device;
 	}
-	ufshcd_blk_pm_runtime_init(sdev_rpmb);
-	scsi_device_put(sdev_rpmb);
+	ufshcd_blk_pm_runtime_init(hba->sdev_rpmb);
+	scsi_device_put(hba->sdev_rpmb);
 
 	sdev_boot = __scsi_add_device(hba->host, 0, 0,
 		ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_BOOT_WLUN), NULL);
@@ -7602,6 +7601,63 @@ static int ufshcd_add_lus(struct ufs_hba *hba)
 	return ret;
 }
 
+static int
+ufshcd_send_request_sense(struct ufs_hba *hba, struct scsi_device *sdp);
+
+static int ufshcd_clear_ua_wlun(struct ufs_hba *hba, u8 wlun)
+{
+	struct scsi_device *sdp;
+	unsigned long flags;
+	int ret = 0;
+
+	spin_lock_irqsave(hba->host->host_lock, flags);
+	if (wlun  == UFS_UPIU_UFS_DEVICE_WLUN)
+		sdp = hba->sdev_ufs_device;
+	else if (wlun  == UFS_UPIU_RPMB_WLUN)
+		sdp = hba->sdev_rpmb;
+	else
+		BUG_ON(1);
+	if (sdp) {
+		ret = scsi_device_get(sdp);
+		if (!ret && !scsi_device_online(sdp)) {
+			ret = -ENODEV;
+			scsi_device_put(sdp);
+		}
+	} else {
+		ret = -ENODEV;
+	}
+	spin_unlock_irqrestore(hba->host->host_lock, flags);
+	if (ret)
+		goto out_err;
+
+	ret = ufshcd_send_request_sense(hba, sdp);
+	scsi_device_put(sdp);
+out_err:
+	if (ret)
+		dev_err(hba->dev, "%s: UAC clear LU=%x ret = %d\n",
+				__func__, wlun, ret);
+	return ret;
+}
+
+static int ufshcd_clear_ua_wluns(struct ufs_hba *hba)
+{
+	int ret = 0;
+
+	if (!hba->wlun_dev_clr_ua)
+		goto out;
+
+	ret = ufshcd_clear_ua_wlun(hba, UFS_UPIU_UFS_DEVICE_WLUN);
+	if (!ret)
+		ret = ufshcd_clear_ua_wlun(hba, UFS_UPIU_RPMB_WLUN);
+	if (!ret)
+		hba->wlun_dev_clr_ua = false;
+out:
+	if (ret)
+		dev_err(hba->dev, "%s: Failed to clear UAC WLUNS ret = %d\n",
+				__func__, ret);
+	return ret;
+}
+
 /**
  * ufshcd_probe_hba - probe hba to detect device and initialize
  * @hba: per-adapter instance
@@ -7721,6 +7777,8 @@ static void ufshcd_async_scan(void *data, async_cookie_t cookie)
 		pm_runtime_put_sync(hba->dev);
 		ufshcd_exit_clk_scaling(hba);
 		ufshcd_hba_exit(hba);
+	} else {
+		ufshcd_clear_ua_wluns(hba);
 	}
 }
 
drivers/scsi/ufs/ufshcd.h +1 -0
index 47eb1430274c..718881d038f5 100644
@@ -681,6 +681,7 @@ struct ufs_hba {
 	 * "UFS device" W-LU.
 	 */
 	struct scsi_device *sdev_ufs_device;
+	struct scsi_device *sdev_rpmb;
 
 	enum ufs_dev_pwr_mode curr_dev_pwr_mode;
 	enum uic_link_state uic_link_state;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#6 lore raw

PATCH v5 4/7 scsi: ufs: use WQ_HIGHPRI for gating work

In reply to Jaegeuk Kim

From: Jaegeuk Kim <[email protected]>

Must have WQ_MEM_RECLAIM
``WQ_MEM_RECLAIM``
All wq which might be used in the memory reclaim paths **MUST** have this flag set. The wq is guaranteed to have at least one execution context regardless of memory pressure.

---
 drivers/scsi/ufs/ufshcd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +1 -1
index 8e696ca79b40..c45c0cff174e 100644
@@ -1868,7 +1868,7 @@ static void ufshcd_init_clk_gating(struct ufs_hba *hba)
 	snprintf(wq_name, ARRAY_SIZE(wq_name), "ufs_clk_gating_%d",
 		 hba->host->host_no);
 	hba->clk_gating.clk_gating_workq = alloc_ordered_workqueue(wq_name,
-							   WQ_MEM_RECLAIM);
+					WQ_MEM_RECLAIM | WQ_HIGHPRI);
 
 	hba->clk_gating.is_enabled = true;
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#7 lore raw

PATCH v5 5/7 scsi: add more contexts in the ufs tracepoints

In reply to Jaegeuk Kim

From: Jaegeuk Kim <[email protected]>

This adds user-friendly tracepoints with group id.

---
 drivers/scsi/ufs/ufshcd.c  |  6 ++++--
 include/trace/events/ufs.h | 21 +++++++++++++++++----
 2 files changed, 21 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +4 -2
index c45c0cff174e..b8a54d09e750 100644
@@ -348,7 +348,7 @@ static void ufshcd_add_command_trace(struct ufs_hba *hba,
 		unsigned int tag, const char *str)
 {
 	sector_t lba = -1;
-	u8 opcode = 0;
+	u8 opcode = 0, group_id = 0;
 	u32 intr, doorbell;
 	struct ufshcd_lrb *lrbp = &hba->lrb[tag];
 	struct scsi_cmnd *cmd = lrbp->cmd;
@@ -374,13 +374,15 @@ static void ufshcd_add_command_trace(struct ufs_hba *hba,
 				lba = cmd->request->bio->bi_iter.bi_sector;
 			transfer_len = be32_to_cpu(
 				lrbp->ucd_req_ptr->sc.exp_data_transfer_len);
+			if (opcode == WRITE_10)
+				group_id = lrbp->cmd->cmnd[6];
 		}
 	}
 
 	intr = ufshcd_readl(hba, REG_INTERRUPT_STATUS);
 	doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL);
 	trace_ufshcd_command(dev_name(hba->dev), str, tag,
-				doorbell, transfer_len, intr, lba, opcode);
+			doorbell, transfer_len, intr, lba, opcode, group_id);
 }
 
 static void ufshcd_print_clk_freqs(struct ufs_hba *hba)
include/trace/events/ufs.h +17 -4
index 84841b3a7ffd..50654f352639 100644
@@ -11,6 +11,15 @@
 
 #include <linux/tracepoint.h>
 
+#define str_opcode(opcode)						\
+	__print_symbolic(opcode,					\
+		{ WRITE_16,		"WRITE_16" },			\
+		{ WRITE_10,		"WRITE_10" },			\
+		{ READ_16,		"READ_16" },			\
+		{ READ_10,		"READ_10" },			\
+		{ SYNCHRONIZE_CACHE,	"SYNC" },			\
+		{ UNMAP,		"UNMAP" })
+
 #define UFS_LINK_STATES			\
 	EM(UIC_LINK_OFF_STATE)		\
 	EM(UIC_LINK_ACTIVE_STATE)	\
@@ -215,9 +224,10 @@ DEFINE_EVENT(ufshcd_template, ufshcd_init,
 TRACE_EVENT(ufshcd_command,
 	TP_PROTO(const char *dev_name, const char *str, unsigned int tag,
 			u32 doorbell, int transfer_len, u32 intr, u64 lba,
-			u8 opcode),
+			u8 opcode, u8 group_id),
 
-	TP_ARGS(dev_name, str, tag, doorbell, transfer_len, intr, lba, opcode),
+	TP_ARGS(dev_name, str, tag, doorbell, transfer_len,
+				intr, lba, opcode, group_id),
 
 	TP_STRUCT__entry(
 		__string(dev_name, dev_name)
@@ -228,6 +238,7 @@ TRACE_EVENT(ufshcd_command,
 		__field(u32, intr)
 		__field(u64, lba)
 		__field(u8, opcode)
+		__field(u8, group_id)
 	),
 
 	TP_fast_assign(
@@ -239,13 +250,15 @@ TRACE_EVENT(ufshcd_command,
 		__entry->intr = intr;
 		__entry->lba = lba;
 		__entry->opcode = opcode;
+		__entry->group_id = group_id;
 	),
 
 	TP_printk(
-		"%s: %s: tag: %u, DB: 0x%x, size: %d, IS: %u, LBA: %llu, opcode: 0x%x",
+		"%s: %s: tag: %u, DB: 0x%x, size: %d, IS: %u, LBA: %llu, opcode: 0x%x (%s), group_id: 0x%x",
 		__get_str(str), __get_str(dev_name), __entry->tag,
 		__entry->doorbell, __entry->transfer_len,
-		__entry->intr, __entry->lba, (u32)__entry->opcode
+		__entry->intr, __entry->lba, (u32)__entry->opcode,
+		str_opcode(__entry->opcode), (u32)__entry->group_id
 	)
 );
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#8 lore raw

PATCH v5 6/7 scsi: ufs: fix clkgating on/off correctly

In reply to Jaegeuk Kim

The below call stack prevents clk_gating at every IO completion. We can remove the condition, ufshcd_any_tag_in_use(), since clkgating_work will check it again.

ufshcd_complete_requests(struct ufs_hba *hba)
ufshcd_transfer_req_compl()

    __ufshcd_transfer_req_compl()
      __ufshcd_release(hba)
        if (ufshcd_any_tag_in_use() == 1)
           return;
  ufshcd_tmc_handler(hba);
    blk_mq_tagset_busy_iter();

Note that, this still requires a work to deal with a potential racy condition when user sets clkgating.delay_ms to very small value. That can cause preventing clkgating by the check of ufshcd_any_tag_in_use() in gate_work.

Fixes 7252a3603015 ("scsi: ufs: Avoid busy-waiting by eliminating tag conflicts")
---
 drivers/scsi/ufs/ufshcd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +1 -1
index b8a54d09e750..86c8dee01ca9 100644
@@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)
 
 	if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
 	    hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
-	    ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
+	    hba->outstanding_tasks ||
 	    hba->active_uic_cmd || hba->uic_async_done ||
 	    hba->clk_gating.state == CLKS_OFF)
 		return;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#9 lore raw

PATCH v5 7/7 scsi: ufs: show lba and length for unmap commands

In reply to Jaegeuk Kim

From: Leo Liou <[email protected]>

We have lba and length for unmap commands.

---
 drivers/scsi/ufs/ufshcd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/scsi/ufs/ufshcd.c +5 -0
index 86c8dee01ca9..dba3ee307307 100644
@@ -376,6 +376,11 @@ static void ufshcd_add_command_trace(struct ufs_hba *hba,
 				lrbp->ucd_req_ptr->sc.exp_data_transfer_len);
 			if (opcode == WRITE_10)
 				group_id = lrbp->cmd->cmnd[6];
+		} else if (opcode == UNMAP) {
+			if (cmd->request) {
+				lba = scsi_get_lba(cmd);
+				transfer_len = blk_rq_bytes(cmd->request);
+			}
 		}
 	}
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Reviewed-by on v5 7/7
#10 lore raw

Re: [PATCH v5 7/7] scsi: ufs: show lba and length for unmap commands

Quoted 7 lines · Jaegeuk Kim

From: Leo Liou <[email protected]>

We have lba and length for unmap commands.

Quoted 20 lines
---
 drivers/scsi/ufs/ufshcd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/scsi/ufs/ufshcd.c +5 -0
index 86c8dee01ca9..dba3ee307307 100644
@@ -376,6 +376,11 @@ static void ufshcd_add_command_trace(struct 
 ufs_hba *hba,
 				lrbp->ucd_req_ptr->sc.exp_data_transfer_len);
 			if (opcode == WRITE_10)
 				group_id = lrbp->cmd->cmnd[6];
+		} else if (opcode == UNMAP) {
+			if (cmd->request) {
+				lba = scsi_get_lba(cmd);
+				transfer_len = blk_rq_bytes(cmd->request);
+			}
 		}
 	}
#12 lore raw
In reply to Jaegeuk Kim
Quoted 10 lines · Jaegeuk Kim

Change log from v4:
- add more fixes

Change log from v3:
- use __ufshcd_release with a fix in __ufshcd_release

Change log from v2:
- use active_req-- instead of __ufshcd_release to avoid UFS timeout

[...]

Applied to 5.11/scsi-queue, thanks!

[1/7] scsi: ufs: Avoid to call REQ_CLKS_OFF to CLKS_OFF

      https://git.kernel.org/mkp/scsi/c/fd62de114f8c
[2/7] scsi: ufs: Atomic update for clkgating_enable
      https://git.kernel.org/mkp/scsi/c/b66451129764
[3/7] scsi: ufs: Clear UAC for FFU and RPMB LUNs
      https://git.kernel.org/mkp/scsi/c/4f3e900b6282
[4/7] scsi: ufs: Use WQ_HIGHPRI for gating work
      https://git.kernel.org/mkp/scsi/c/e93e6e49fa31
[5/7] scsi: ufs: Add more contexts in the ufs tracepoints
      https://git.kernel.org/mkp/scsi/c/69a314d6a155
[6/7] scsi: ufs: Fix clkgating on/off
      https://git.kernel.org/mkp/scsi/c/8eb456be75af
[7/7] scsi: ufs: Show LBA and length for UNMAP commands
      https://git.kernel.org/mkp/scsi/c/3754cde8df91
Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering