Skip to contentSkip to conversation

scsi: ufs: core: Fix SCSI EH command ownership and remove force_compl

PATCH In review Stanley Jhu 3 participants 8 messages Sep 20 – Sep 22

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover scsi: ufs: core: Fix SCSI EH command ownership and remove force_compl

In ufshcd_host_reset_and_restore(), ufshcd_complete_requests(hba, true) couples LLD resource release (ufshcd_release_scsi_cmd()) with command completion (scsi_done()) right after ufshcd_hba_stop():

1. EH-owned SCSI commands (SCMD_STATE_COMPLETE set): skipping scsi_done() in ufshcd_mcq_force_compl_one() also skips ufshcd_release_scsi_cmd(), leaking DMA mappings and clk_gating.active_reqs whenever ufshcd_abort() fails.
2. Non-EH SCSI commands (!SCMD_STATE_COMPLETE): calling scsi_done() right after ufshcd_hba_stop() completes them before link recovery finishes.

This series enforces a strict ownership boundary between LLD hardware resources and SCSI command completion across SDB and MCQ error recovery:

- LLD resources (DMA mappings, crypto PRDT, clk_gating.active_reqs) are tied to controller execution and are released whenever the hardware stops executing a command, regardless of whether SCSI EH owns it.
- Command completion (scsi_done()) belongs exclusively to SCSI EH once SCMD_STATE_COMPLETE is set. For commands where SCMD_STATE_COMPLETE is not set, UFS cannot delegate completion to SCSI EH because UFS also performs autonomous resets (ufshcd_err_handler() on UIC/controller errors) outside of scsi_error_handler(). Since ufshcd_hba_stop() (HCE = 0) wipes all in-flight hardware transfers while SCSI EH is inactive, the driver itself requeues halted non-EH commands with DID_REQUEUE after recovery finishes so they do not stall for the 30s block timeout.

- Patch 1 tracks controller resource ownership with lrbp->in_flight and halted non-EH commands with lrbp->pending_requeue, splits host-reset cleanup into ufshcd_release_stopped_reqs() (at controller stop) and ufshcd_requeue_non_eh_cmds() (after recovery finishes), and aligns ownership across ufshcd_compl_one_cqe(), ufshcd_abort(), and ufshcd_clear_lu_cmds().
- Patch 2 removes the unused force_compl parameter from ufshcd_complete_requests() and ufshcd_mcq_compl_pending_transfer(), along with the now-unreachable ufshcd_mcq_force_compl_one() and ufshcd_mcq_compl_all_cqes_lock() helpers.

This supersedes the v2 series [3].

Tested:
Verified on QEMU ARM64 (SDB and MCQ) across probe, I/O, reset, and unbind without UAF or leaks.

Stanley Jhu (2):
scsi: ufs: core: Release command resources instead of force-completing
scsi: ufs: core: Remove unused force_compl parameter and MCQ helper

 drivers/ufs/core/ufs-mcq.c      |  26 ----
 drivers/ufs/core/ufshcd-priv.h  |   2 -
 drivers/ufs/core/ufshcd.c       | 213 +++++++++++++++++++++++---------
 include/ufs/ufshcd.h            |   4 +
 4 files changed, 157 insertions(+), 88 deletions(-)
Signature · 1 line
2.51.0
#1 lore raw

PATCH 1/2 scsi: ufs: core: Release command resources instead of force-completing

In ufshcd_host_reset_and_restore(), ufshcd_complete_requests(hba, true) couples LLD resource release (ufshcd_release_scsi_cmd()) with command completion (scsi_done()) right after ufshcd_hba_stop():

1. EH-owned SCSI commands (SCMD_STATE_COMPLETE set): skipping scsi_done() in ufshcd_mcq_force_compl_one() also skips ufshcd_release_scsi_cmd(), leaking DMA mappings and clk_gating.active_reqs whenever ufshcd_abort() fails.
2. Non-EH SCSI commands (!SCMD_STATE_COMPLETE): calling scsi_done() right after ufshcd_hba_stop() completes them before link recovery finishes.

To fix this, enforce a strict ownership boundary between LLD hardware resources and SCSI command completion:

- LLD resources (DMA mappings, crypto PRDT, clk_gating.active_reqs) are tied to controller execution and must be released whenever the hardware stops executing a command, regardless of whether SCSI EH owns the command.
- Command completion (scsi_done()) belongs exclusively to SCSI EH once SCMD_STATE_COMPLETE is set. However, for commands where SCMD_STATE_COMPLETE is not set, UFS cannot delegate completion to SCSI EH because UFS also performs autonomous resets (ufshcd_err_handler() on UIC/controller errors) outside of scsi_error_handler(). Since ufshcd_hba_stop() (HCE = 0) wipes all in-flight hardware transfers while SCSI EH is inactive, the driver itself must requeue halted non-EH commands with DID_REQUEUE after recovery finishes, or else they stall for the 30s block timeout.

Implement this in three steps:

1. Track controller resource ownership with lrbp->in_flight so ufshcd_release_scsi_cmd() is idempotent across normal completion, successful task abort, and host reset teardown.
2. At controller stop (ufshcd_release_stopped_reqs()), release LLD resources for all halted commands without calling scsi_done(), clear any uncompleted reserved dev_cmd, and mark halted non-EH commands with lrbp->pending_requeue.
3. After host and link recovery finish (ufshcd_requeue_non_eh_cmds()), requeue pending commands that remain non-EH (!SCMD_STATE_COMPLETE) with DID_REQUEUE while leaving EH-owned commands on shost->eh_cmd_q.

Fixes ab248643d3d6 ("scsi: ufs: core: Add error handling for MCQ mode")
---
 drivers/ufs/core/ufshcd.c | 183 ++++++++++++++++++++++++++++++--------
 include/ufs/ufshcd.h      |   6 ++
 2 files changed, 154 insertions(+), 35 deletions(-)
drivers/ufs/core/ufshcd.c +148 -35
index 234e18b5078f..c0772822731e 100644
@@ -2432,6 +2432,7 @@ static inline void ufshcd_send_command(struct ufs_hba *hba,
 		lrbp->compl_time_stamp = ktime_set(0, 0);
 		lrbp->compl_time_stamp_local_clock = 0;
 	}
+	lrbp->in_flight = true;
 	if (ufshcd_is_scsi_cmd(cmd)) {
 		ufshcd_add_command_trace(hba, cmd, UFS_CMD_SEND);
 		ufshcd_clk_scaling_start_busy(hba);
@@ -3116,11 +3117,15 @@ static int ufshcd_init_cmd_priv(struct Scsi_Host *host, struct scsi_cmnd *cmd)
 static enum scsi_qc_status ufshcd_queuecommand(struct Scsi_Host *host,
 					       struct scsi_cmnd *cmd)
 {
+	struct ufshcd_lrb *lrbp = scsi_cmd_priv(cmd);
 	struct ufs_hba *hba = shost_priv(host);
 	int tag = scsi_cmd_to_rq(cmd)->tag;
 	int err = 0;
 	struct ufs_hw_queue *hwq = NULL;
 
+	lrbp->in_flight = false;
+	lrbp->pending_requeue = false;
+
 	switch (hba->ufshcd_state) {
 	case UFSHCD_STATE_OPERATIONAL:
 		break;
@@ -3203,6 +3208,8 @@ static enum scsi_qc_status ufshcd_queue_reserved_command(struct Scsi_Host *host,
 	struct ufs_hw_queue *hwq =
 		hba->mcq_enabled ? ufshcd_mcq_req_to_hwq(hba, rq) : NULL;
 
+	lrbp->in_flight = false;
+	lrbp->pending_requeue = false;
 	ufshcd_add_query_upiu_trace(hba, UFS_QUERY_SEND, lrbp->ucd_req_ptr);
 	ufshcd_send_command(hba, cmd, hwq);
 	return 0;
@@ -4671,6 +4678,28 @@ int ufshcd_uic_change_pwr_mode(struct ufs_hba *hba, u8 mode)
 }
 EXPORT_SYMBOL_GPL(ufshcd_uic_change_pwr_mode);
 
+static bool ufshcd_requeue_one_non_eh_cmd(struct request *req, void *data)
+{
+	struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(req);
+	struct ufshcd_lrb *lrbp = scsi_cmd_priv(cmd);
+
+	if (blk_mq_is_reserved_rq(req) || !lrbp->pending_requeue)
+		return true;
+
+	lrbp->pending_requeue = false;
+	if (!test_bit(SCMD_STATE_COMPLETE, &cmd->state)) {
+		set_host_byte(cmd, DID_REQUEUE);
+		scsi_done(cmd);
+	}
+	return true;
+}
+
+static void ufshcd_requeue_non_eh_cmds(struct ufs_hba *hba)
+{
+	blk_mq_tagset_busy_iter(&hba->host->tag_set,
+				ufshcd_requeue_one_non_eh_cmd, hba);
+}
+
 int ufshcd_link_recovery(struct ufs_hba *hba)
 {
 	int ret;
@@ -4692,6 +4721,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
 	ufshcd_clear_eh_in_progress(hba);
 	spin_unlock_irqrestore(&hba->host->host_lock, flags);
 
+	ufshcd_requeue_non_eh_cmds(hba);
+
 	if (ret)
 		dev_err(hba->dev, "%s: link recovery failed, err %d",
 			__func__, ret);
@@ -5937,6 +5968,17 @@ static irqreturn_t ufshcd_uic_cmd_compl(struct ufs_hba *hba, u32 intr_status)
 /* Release the resources allocated for processing a SCSI command. */
 void ufshcd_release_scsi_cmd(struct ufs_hba *hba, struct scsi_cmnd *cmd)
 {
+	struct ufshcd_lrb *lrbp = scsi_cmd_priv(cmd);
+
+	lrbp->pending_requeue = false;
+	if (!lrbp->in_flight)
+		return;
+	lrbp->in_flight = false;
+
+	/* Device management commands do not own any of the below. */
+	if (!ufshcd_is_scsi_cmd(cmd))
+		return;
+
 	scsi_dma_unmap(cmd);
 	ufshcd_crypto_clear_prdt(hba, cmd);
 	ufshcd_release(hba);
@@ -5968,8 +6010,9 @@ void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag,
 		if (unlikely(ufshcd_should_inform_monitor(hba, cmd)))
 			ufshcd_update_monitor(hba, cmd);
 		ufshcd_add_command_trace(hba, cmd, UFS_CMD_COMP);
-		cmd->result = ufshcd_transfer_rsp_status(hba, cmd, cqe);
-		ufshcd_release_scsi_cmd(hba, cmd);
+		if (!test_bit(SCMD_STATE_COMPLETE, &cmd->state))
+			cmd->result =
+				ufshcd_transfer_rsp_status(hba, cmd, cqe);
 	} else {
 		if (cqe) {
 			ocs = cqe->overall_status & MASK_OCS;
@@ -5981,8 +6024,17 @@ void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag,
 			hba,
 			ocs == OCS_SUCCESS ? UFS_QUERY_COMP : UFS_QUERY_ERR,
 			(struct utp_upiu_req *)lrbp->ucd_rsp_ptr);
-		cmd->result = 0;
+		if (!test_bit(SCMD_STATE_COMPLETE, &cmd->state))
+			cmd->result = 0;
 	}
+	ufshcd_release_scsi_cmd(hba, cmd);
+	/*
+	 * Clear lrbp->pending_requeue before the tag is freed so that the
+	 * next user of this tag is not mistaken for a command halted by
+	 * ufshcd_hba_stop().
+	 */
+	lrbp->pending_requeue = false;
+
 	/* Do not touch lrbp after scsi_done() has been called. */
 	scsi_done(cmd);
 }
@@ -7912,6 +7964,8 @@ static bool ufshcd_clear_lu_cmds(struct request *req, void *priv)
 	struct ufs_hba *hba = shost_priv(shost);
 	const u64 lun = *(u64 *)priv;
 	const u32 tag = req->tag;
+	unsigned long flags;
+	bool outstanding;
 
 	if (blk_mq_is_reserved_rq(req) || sdev->lun != lun)
 		return true;
@@ -7930,7 +7984,11 @@ static bool ufshcd_clear_lu_cmds(struct request *req, void *priv)
 		return true;
 	}
 
-	ufshcd_compl_one_cqe(hba, tag, NULL);
+	spin_lock_irqsave(&hba->outstanding_lock, flags);
+	outstanding = __test_and_clear_bit(tag, &hba->outstanding_reqs);
+	spin_unlock_irqrestore(&hba->outstanding_lock, flags);
+	if (outstanding)
+		ufshcd_compl_one_cqe(hba, tag, NULL);
 	return true;
 }
 
@@ -8087,6 +8145,7 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
 	u32 reg;
 
 	ufshcd_hold(hba);
+	lrbp->pending_requeue = false;
 
 	if (!hba->mcq_enabled) {
 		reg = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL);
@@ -8153,6 +8212,16 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
 	if (hba->mcq_enabled) {
 		/* MCQ mode. Branch off to handle abort for mcq mode */
 		err = ufshcd_mcq_abort(cmd);
+		if (err == SUCCESS) {
+			struct ufs_hw_queue *hwq =
+				ufshcd_mcq_req_to_hwq(hba, rq);
+
+			if (hwq) {
+				spin_lock_irqsave(&hwq->cq_lock, flags);
+				ufshcd_release_scsi_cmd(hba, cmd);
+				spin_unlock_irqrestore(&hwq->cq_lock, flags);
+			}
+		}
 		goto release;
 	}
 
@@ -8216,6 +8285,71 @@ static void ufshcd_process_probe_result(struct ufs_hba *hba,
 			  hba->curr_dev_pwr_mode, hba->uic_link_state);
 }
 
+/*
+ * The caller must own @rq, either by having claimed its bit in
+ * hba->outstanding_reqs (legacy mode) or by holding hwq->cq_lock (MCQ mode).
+ */
+static void __ufshcd_release_stopped_req(struct ufs_hba *hba,
+					 struct request *rq,
+					 struct scsi_cmnd *cmd)
+{
+	struct ufshcd_lrb *lrbp = scsi_cmd_priv(cmd);
+
+	if (!lrbp->in_flight)
+		return;
+
+	ufshcd_release_scsi_cmd(hba, cmd);
+	if (test_bit(SCMD_STATE_COMPLETE, &cmd->state))
+		return;
+
+	if (blk_mq_is_reserved_rq(rq)) {
+		/* Device management commands are not requeued. */
+		set_host_byte(cmd, DID_TIME_OUT);
+		scsi_done(cmd);
+	} else {
+		lrbp->pending_requeue = true;
+	}
+}
+
+static bool ufshcd_release_one_stopped_req(struct request *rq, void *priv)
+{
+	struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(rq);
+	struct ufs_hba *hba = priv;
+	unsigned long flags;
+	bool owned;
+
+	if (hba->mcq_enabled) {
+		struct ufs_hw_queue *hwq = ufshcd_mcq_req_to_hwq(hba, rq);
+
+		if (hwq) {
+			spin_lock_irqsave(&hwq->cq_lock, flags);
+			__ufshcd_release_stopped_req(hba, rq, cmd);
+			spin_unlock_irqrestore(&hwq->cq_lock, flags);
+		}
+		return true;
+	}
+
+	/*
+	 * Keep hba->outstanding_lock a leaf lock, as in ufshcd_poll() and
+	 * ufshcd_abort(): clearing the bit already grants exclusive ownership.
+	 */
+	spin_lock_irqsave(&hba->outstanding_lock, flags);
+	owned = __test_and_clear_bit(rq->tag, &hba->outstanding_reqs);
+	spin_unlock_irqrestore(&hba->outstanding_lock, flags);
+	if (owned)
+		__ufshcd_release_stopped_req(hba, rq, cmd);
+
+	return true;
+}
+
+static void ufshcd_release_stopped_reqs(struct ufs_hba *hba)
+{
+	blk_mq_tagset_busy_iter(&hba->host->tag_set,
+				ufshcd_release_one_stopped_req, hba);
+	/* TMF sweep inherited from the replaced ufshcd_complete_requests(). */
+	ufshcd_tmc_handler(hba);
+}
+
 /**
  * ufshcd_host_reset_and_restore - reset and restore host controller
  * @hba: per-adapter instance
@@ -8231,13 +8365,11 @@ static int ufshcd_host_reset_and_restore(struct ufs_hba *hba)
 	int err;
 
 	/*
-	 * Stop the host controller and complete the requests
-	 * cleared by h/w
+	 * Stop the host controller and release driver resources for
+	 * requests cleared by h/w
 	 */
 	ufshcd_hba_stop(hba);
-	hba->silence_err_logs = true;
-	ufshcd_complete_requests(hba, true);
-	hba->silence_err_logs = false;
+	ufshcd_release_stopped_reqs(hba);
 
 	/* scale up clocks to max frequency before full reinitialization */
 	if (ufshcd_is_clkscaling_supported(hba))
@@ -8319,6 +8451,8 @@ static int ufshcd_reset_and_restore(struct ufs_hba *hba)
 	}
 	spin_unlock_irqrestore(&hba->host->host_lock, flags);
 
+	ufshcd_requeue_non_eh_cmds(hba);
+
 	return err;
 }
 
@@ -9597,30 +9731,6 @@ static enum scsi_timeout_action ufshcd_eh_timed_out(struct scsi_cmnd *scmd)
 	dev_info(hba->dev, "%s() finished; outstanding_tasks = %#lx.\n",
 		 __func__, hba->outstanding_tasks);
 
-	/*
-	 * ufshcd_link_recovery() may already have completed @scmd, e.g. via
-	 * the existing MCQ force-completion path.
-	 */
-	if (!test_bit(SCMD_STATE_COMPLETE, &scmd->state)) {
-		if (!hba->mcq_enabled) {
-			unsigned long flags;
-			struct request *rq = scsi_cmd_to_rq(scmd);
-
-			spin_lock_irqsave(&hba->outstanding_lock, flags);
-			__clear_bit(rq->tag, &hba->outstanding_reqs);
-			spin_unlock_irqrestore(&hba->outstanding_lock, flags);
-		}
-
-		if (ufshcd_is_scsi_cmd(scmd)) {
-			set_host_byte(scmd, DID_REQUEUE);
-			ufshcd_release_scsi_cmd(hba, scmd);
-		} else {
-			set_host_byte(scmd, DID_TIME_OUT);
-		}
-
-		scsi_done(scmd);
-	}
-
 	return SCSI_EH_DONE;
 }
 
@@ -10464,15 +10574,18 @@ static int __ufshcd_wl_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 		ufshcd_device_reset(hba);
 		WARN_ON(!ufshcd_is_link_off(hba));
 	}
-	if (ufshcd_is_link_hibern8(hba) && !ufshcd_uic_hibern8_exit(hba))
+	if (ufshcd_is_link_hibern8(hba) && !ufshcd_uic_hibern8_exit(hba)) {
 		ufshcd_set_link_active(hba);
-	else if (ufshcd_is_link_off(hba))
+	} else if (ufshcd_is_link_off(hba)) {
 		ufshcd_host_reset_and_restore(hba);
+		ufshcd_requeue_non_eh_cmds(hba);
+	}
 set_dev_active:
 	/* Can also get here needing to exit DeepSleep */
 	if (ufshcd_is_ufs_dev_deepsleep(hba)) {
 		ufshcd_device_reset(hba);
 		ufshcd_host_reset_and_restore(hba);
+		ufshcd_requeue_non_eh_cmds(hba);
 	}
 	if (!ufshcd_set_dev_pwr_mode(hba, UFS_ACTIVE_PWR_MODE))
 		ufshcd_disable_auto_bkops(hba);
include/ufs/ufshcd.h +6 -0
index dfd302f2dc7c..cd84f617f95c 100644
@@ -166,6 +166,10 @@ struct ufs_pm_lvl_states {
  * @lun: LUN of the command
  * @intr_cmd: Interrupt command (doesn't participate in interrupt aggregation)
  * @req_abort_skip: skip request abort task flag
+ * @in_flight: true after the command has been submitted to the controller and
+ *	       before its resources have been released
+ * @pending_requeue: true if command was halted by ufshcd_hba_stop() and awaits
+ *		     post-recovery requeue
  * @issue_time_stamp: time stamp for debug purposes (CLOCK_MONOTONIC)
  * @issue_time_stamp_local_clock: time stamp for debug purposes (local_clock)
  * @compl_time_stamp: time stamp for statistics (CLOCK_MONOTONIC)
@@ -190,6 +194,8 @@ struct ufshcd_lrb {
 	u8 lun; /* UPIU LUN id field is only 8-bit wide */
 	bool intr_cmd;
 	bool req_abort_skip;
+	bool in_flight;
+	bool pending_requeue;
 	ktime_t issue_time_stamp;
 	u64 issue_time_stamp_local_clock;
 	ktime_t compl_time_stamp;
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
#2 lore raw

Re: [PATCH 1/2] scsi: ufs: core: Release command resources instead of force-completing

Quoted 42 lines · Stanley Jhu

In ufshcd_host_reset_and_restore(), ufshcd_complete_requests(hba, true) couples LLD resource release (ufshcd_release_scsi_cmd()) with command completion (scsi_done()) right after ufshcd_hba_stop():

1. EH-owned SCSI commands (SCMD_STATE_COMPLETE set): skipping scsi_done() in ufshcd_mcq_force_compl_one() also skips ufshcd_release_scsi_cmd(), leaking DMA mappings and clk_gating.active_reqs whenever ufshcd_abort() fails.
2. Non-EH SCSI commands (!SCMD_STATE_COMPLETE): calling scsi_done() right after ufshcd_hba_stop() completes them before link recovery finishes.

To fix this, enforce a strict ownership boundary between LLD hardware resources and SCSI command completion:

- LLD resources (DMA mappings, crypto PRDT, clk_gating.active_reqs) are tied to controller execution and must be released whenever the hardware stops executing a command, regardless of whether SCSI EH owns the command.
- Command completion (scsi_done()) belongs exclusively to SCSI EH once SCMD_STATE_COMPLETE is set. However, for commands where SCMD_STATE_COMPLETE is not set, UFS cannot delegate completion to SCSI EH because UFS also performs autonomous resets (ufshcd_err_handler() on UIC/controller errors) outside of scsi_error_handler(). Since ufshcd_hba_stop() (HCE = 0) wipes all in-flight hardware transfers while SCSI EH is inactive, the driver itself must requeue halted non-EH commands with DID_REQUEUE after recovery finishes, or else they stall for the 30s block timeout.

Implement this in three steps:

1. Track controller resource ownership with lrbp->in_flight so ufshcd_release_scsi_cmd() is idempotent across normal completion, successful task abort, and host reset teardown.
2. At controller stop (ufshcd_release_stopped_reqs()), release LLD resources for all halted commands without calling scsi_done(), clear any uncompleted reserved dev_cmd, and mark halted non-EH commands with lrbp->pending_requeue.
3. After host and link recovery finish (ufshcd_requeue_non_eh_cmds()), requeue pending commands that remain non-EH (!SCMD_STATE_COMPLETE) with DID_REQUEUE while leaving EH-owned commands on shost->eh_cmd_q.

Quoted 2 lines
Fixes ab248643d3d6 ("scsi: ufs: core: Add error handling for MCQ mode")

No, this is not what I suggested. Additionally, I see multiple steps in the wrong direction.

The text above says "implement this in three steps". Hence, this patch should have been three patches instead of one because of the "one change per patch" rule in the Linux kernel community.

Regarding (1), why a new member variable? Why isn't rq->state sufficient?

Regarding (3), the UFS driver is a SCSI LLD (low-level driver) and hence should only do what is specific to the UFS driver. Completing commands after .eh_host_reset_handler() has been called is the responsibility of the SCSI core and should not be done by the UFS driver.

(2) seems wrong to me. The caller of ufshcd_hba_stop() should take care of pending commands instead of ufshcd_hba_stop() itself.

drivers/ufs/core/ufshcd.c int ufshcd_link_recovery(struct ufs_hba *hba) · line 4728

Quoted 12 lines
  int ufshcd_link_recovery(struct ufs_hba *hba)
  {
  	int ret;
drivers/ufs/core/ufshcd.c +2 -0
@@ -4692,6 +4721,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
  	ufshcd_clear_eh_in_progress(hba);
  	spin_unlock_irqrestore(&hba->host->host_lock, flags);
 
+	ufshcd_requeue_non_eh_cmds(hba);
+
  	if (ret)
  		dev_err(hba->dev, "%s: link recovery failed, err %d",
  			__func__, ret);

This is a behavior change that should be in a separate patch and not in a patch for releasing command resources.

Quoted 5 lines
drivers/ufs/core/ufshcd.c +1 -0
@@ -5937,6 +5968,17 @@ static irqreturn_t ufshcd_uic_cmd_compl(struct ufs_hba *hba, u32 intr_status)
  /* Release the resources allocated for processing a SCSI command. */
  void ufshcd_release_scsi_cmd(struct ufs_hba *hba, struct scsi_cmnd *cmd)
  {
+	struct ufshcd_lrb *lrbp = scsi_cmd_priv(cmd);

drivers/ufs/core/ufshcd.c static irqreturn_t ufshcd_uic_cmd_compl(struct ufs_hba *hba, u32 intr_status) · line 5981

Quoted 10 lines
drivers/ufs/core/ufshcd.c +10 -0
+
+	lrbp->pending_requeue = false;
+	if (!lrbp->in_flight)
+		return;
+	lrbp->in_flight = false;
+
+	/* Device management commands do not own any of the below. */
+	if (!ufshcd_is_scsi_cmd(cmd))
+		return;
+

Did you read this code before you posted it? As its name suggests, ufshcd_release_scsi_cmd() is only called for SCSI commands. No device management command code should be added in this function.

drivers/ufs/core/ufshcd.c void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag, · line 6015

Quoted 12 lines
drivers/ufs/core/ufshcd.c +3 -2
  	scsi_dma_unmap(cmd);
  	ufshcd_crypto_clear_prdt(hba, cmd);
  	ufshcd_release(hba);
@@ -5968,8 +6010,9 @@ void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag,
  		if (unlikely(ufshcd_should_inform_monitor(hba, cmd)))
  			ufshcd_update_monitor(hba, cmd);
  		ufshcd_add_command_trace(hba, cmd, UFS_CMD_COMP);
-		cmd->result = ufshcd_transfer_rsp_status(hba, cmd, cqe);
-		ufshcd_release_scsi_cmd(hba, cmd);
+		if (!test_bit(SCMD_STATE_COMPLETE, &cmd->state))
+			cmd->result =
+				ufshcd_transfer_rsp_status(hba, cmd, cqe);

Why has this if-condition been added? ufshcd_compl_one_cqe() should only be called if SCMD_STATE_COMPLETE has not yet been set.

drivers/ufs/core/ufshcd.c void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag, · line 6028

Quoted 7 lines
drivers/ufs/core/ufshcd.c +2 -1
@@ -5981,8 +6024,17 @@ void ufshcd_compl_one_cqe(struct ufs_hba *hba, int task_tag,
  			hba,
  			ocs == OCS_SUCCESS ? UFS_QUERY_COMP : UFS_QUERY_ERR,
  			(struct utp_upiu_req *)lrbp->ucd_rsp_ptr);
-		cmd->result = 0;
+		if (!test_bit(SCMD_STATE_COMPLETE, &cmd->state))
+			cmd->result = 0;

Same question for this new if-condition.

drivers/ufs/core/ufshcd.c static bool ufshcd_clear_lu_cmds(struct request *req, void *priv) · line 7993

Quoted 12 lines
drivers/ufs/core/ufshcd.c +5 -1
@@ -7930,7 +7984,11 @@ static bool ufshcd_clear_lu_cmds(struct request *req, void *priv)
  		return true;
  	}
 
-	ufshcd_compl_one_cqe(hba, tag, NULL);
+	spin_lock_irqsave(&hba->outstanding_lock, flags);
+	outstanding = __test_and_clear_bit(tag, &hba->outstanding_reqs);
+	spin_unlock_irqrestore(&hba->outstanding_lock, flags);
+	if (outstanding)
+		ufshcd_compl_one_cqe(hba, tag, NULL);
  	return true;
  }

Why has ufshcd_clear_lu_cmds() been modified? This function is only called for pending commands so checking &hba->outstanding_reqs is not necessary.

Quoted 12 lines
drivers/ufs/core/ufshcd.c +1 -0
@@ -8087,6 +8145,7 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
  	u32 reg;
 
  	ufshcd_hold(hba);
+	lrbp->pending_requeue = false;
 
  	if (!hba->mcq_enabled) {
  		reg = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL);
@@ -8153,6 +8212,16 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
  	if (hba->mcq_enabled) {
  		/* MCQ mode. Branch off to handle abort for mcq mode */
  		err = ufshcd_mcq_abort(cmd);

drivers/ufs/core/ufshcd.c static int ufshcd_abort(struct scsi_cmnd *cmd) · line 8224

Quoted 10 lines
drivers/ufs/core/ufshcd.c +10 -0
+		if (err == SUCCESS) {
+			struct ufs_hw_queue *hwq =
+				ufshcd_mcq_req_to_hwq(hba, rq);
+
+			if (hwq) {
+				spin_lock_irqsave(&hwq->cq_lock, flags);
+				ufshcd_release_scsi_cmd(hba, cmd);
+				spin_unlock_irqrestore(&hwq->cq_lock, flags);
+			}
+		}

This is also wrong because this change will trigger a use-after-free in scmd_eh_abort_handler() if it decides to retry or finish a SCSI command.

drivers/ufs/core/ufshcd.c static int ufshcd_host_reset_and_restore(struct ufs_hba *hba) · line 8369

Quoted 8 lines
drivers/ufs/core/ufshcd.c +2 -2
@@ -8231,13 +8365,11 @@ static int ufshcd_host_reset_and_restore(struct ufs_hba *hba)
  	int err;
 
  	/*
-	 * Stop the host controller and complete the requests
-	 * cleared by h/w
+	 * Stop the host controller and release driver resources for
+	 * requests cleared by h/w

This is wrong. The SCSI core error handler is responsible for releasing resources. The UFS driver must not do this from its host reset handler.

Bart.

#3 lore raw

Re: [PATCH 1/2] scsi: ufs: core: Release command resources instead of force-completing

Quoted 1 line · Bart Van Assche

No, this is not what I suggested.

Sorry about that - the Suggested-by: tag does not belong on this patch and I will drop it.

Quoted 3 lines

The text above says "implement this in three steps". Hence, this patch should have been three patches instead of one because of the "one change per patch" rule in the Linux kernel community.

Agreed.

Quoted 6 lines

Why has ufshcd_clear_lu_cmds() been modified? This function is only called for pending commands so checking &hba->outstanding_reqs is not necessary.

This is also wrong because this change will trigger a use-after-free in scmd_eh_abort_handler() if it decides to retry or finish a SCSI command.

Both hunks are out of scope for this patch and I will drop them.

Quoted 3 lines

Did you read this code before you posted it? As its name suggests, ufshcd_release_scsi_cmd() is only called for SCSI commands. No device management command code should be added in this function.

You are right. In ufshcd_compl_one_cqe() I moved the ufshcd_release_scsi_cmd() call out of the ufshcd_is_scsi_cmd() branch, which let device management commands reach that function, and then added an early return there to filter them back out. In the respin the call will stay inside the branch, with no device management handling in that function.

Quoted 4 lines

Regarding (3), the UFS driver is a SCSI LLD (low-level driver) and hence should only do what is specific to the UFS driver. Completing commands after .eh_host_reset_handler() has been called is the responsibility of the SCSI core and should not be done by the UFS driver.

Agreed for the path where SCSI EH drove the reset. The case I am unsure about is the other caller: ufshcd_err_handler() also runs from hba->eh_work, scheduled by ufshcd_check_errors() on UIC and controller errors. Those commands have not timed out and are not on shost->eh_cmd_q, so SCSI EH never finishes them, and the handler leaves them to the reset path on purpose:

/*
* if host reset is required then skip clearing the pending
* transfers forcefully because they will get cleared during
* host reset and restore
*/

Should those simply wait for the block layer timeout and come back through SCSI EH?

Thanks,

Stanley

#4 lore raw

Re: [PATCH 1/2] scsi: ufs: core: Release command resources instead of force-completing

Quoted 13 lines · Stanley Jhu
Quoted 4 lines · Bart Van Assche

Regarding (3), the UFS driver is a SCSI LLD (low-level driver) and hence should only do what is specific to the UFS driver. Completing commands after .eh_host_reset_handler() has been called is the responsibility of the SCSI core and should not be done by the UFS driver.

Agreed for the path where SCSI EH drove the reset. The case I am unsure about is the other caller: ufshcd_err_handler() also runs from hba->eh_work, scheduled by ufshcd_check_errors() on UIC and controller errors. Those commands have not timed out and are not on shost->eh_cmd_q, so SCSI EH never finishes them, and the handler leaves them to the reset path on purpose:

Quoted 8 lines

/*
* if host reset is required then skip clearing the pending
* transfers forcefully because they will get cleared during
* host reset and restore
*/

Should those simply wait for the block layer timeout and come back through SCSI EH?

That sounds good to me.

Thanks,

Bart.

#5 lore raw

Re: [PATCH 1/2] scsi: ufs: core: Release command resources instead of force-completing

Quoted 16 lines · Bart Van Assche
Quoted 16 lines

Agreed for the path where SCSI EH drove the reset. The case I am unsure
about is the other caller: ufshcd_err_handler() also runs from hba->eh_work, scheduled by ufshcd_check_errors() on UIC and controller
errors. Those commands have not timed out and are not on shost->eh_cmd_q, so SCSI EH never finishes them, and the handler leaves
them to the reset path on purpose:

      /*
       * if host reset is required then skip clearing the pending
       * transfers forcefully because they will get cleared during
       * host reset and restore
       */

Quoted 8 lines
Quoted 2 lines

Should those simply wait for the block layer timeout and come back through SCSI EH?

That sounds good to me.

Thanks,

Bart.

Hi Bart,

It appears unreasonable to wait for the entire 30 second timeout, as many UIC errors recover promptly. The resulting 30 second stall would manifest as a noticeable UI freeze. Maybe we shouldn’t let eh_host_reset_handler call the EH handler at all?

Thanks
Peter

#6 lore raw

Re: [PATCH 1/2] scsi: ufs: core: Release command resources instead of force-completing

Quoted 4 lines · Peter Wang

It appears unreasonable to wait for the entire 30 second timeout, as many UIC errors recover promptly. The resulting 30 second stall would manifest as a noticeable UI freeze. Maybe we shouldn’t let eh_host_reset_handler call the EH handler at all?

The default SCSI disk timeout is 30 seconds. That default timeout was chosen when hard disks were the most widely used persistent storage medium and before NAND storage became widespread. Isn't that default timeout too big for UFS devices?

As mentioned in another email, .eh_host_reset_handler() must abort all pending commands and must terminate all ongoing task management functions. This implies resetting both the UFS device and the UFS host controller. I'm not sure how to do that without calling ufshcd_host_reset_and_restore()?

Thanks,

Bart.

#7 lore raw

PATCH 2/2 scsi: ufs: core: Remove unused force_compl parameter and MCQ helper

In reply to Stanley Jhu

With ufshcd_host_reset_and_restore() converted to call ufshcd_release_stopped_reqs(), both remaining callers of ufshcd_complete_requests() (ufshcd_abort_all() and ufshcd_err_handler()) pass force_compl = false.

Remove the unused force_compl parameter from ufshcd_complete_requests() and ufshcd_mcq_compl_pending_transfer(), and delete the now-unreachable ufshcd_mcq_force_compl_one() and ufshcd_mcq_compl_all_cqes_lock() helpers.

No functional change intended.

---
 drivers/ufs/core/ufs-mcq.c     | 26 -----------------
 drivers/ufs/core/ufshcd-priv.h |  2 --
 drivers/ufs/core/ufshcd.c      | 52 +++++-----------------------------
 3 files changed, 7 insertions(+), 73 deletions(-)
drivers/ufs/core/ufs-mcq.c +0 -26
index 8106d55f4041..55ed72d15ada 100644
@@ -328,32 +328,6 @@ static void ufshcd_mcq_process_cqe(struct ufs_hba *hba,
 	}
 }
 
-/*
- * This function is called from the UFS error handler with the UFS host
- * controller disabled (HCE = 0). Reading host controller registers, e.g. the
- * CQ tail pointer (CQTPy), may not be safe with the host controller disabled.
- * Hence, iterate over all completion queue entries. This won't result in
- * double completions because ufshcd_mcq_process_cqe() clears a CQE after it
- * has been processed.
- */
-void ufshcd_mcq_compl_all_cqes_lock(struct ufs_hba *hba,
-				    struct ufs_hw_queue *hwq)
-{
-	unsigned long flags;
-	u32 entries = hwq->max_entries;
-
-	spin_lock_irqsave(&hwq->cq_lock, flags);
-	while (entries > 0) {
-		ufshcd_mcq_process_cqe(hba, hwq);
-		ufshcd_mcq_inc_cq_head_slot(hwq);
-		entries--;
-	}
-
-	ufshcd_mcq_update_cq_tail_slot(hwq);
-	hwq->cq_head_slot = hwq->cq_tail_slot;
-	spin_unlock_irqrestore(&hwq->cq_lock, flags);
-}
-
 unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,
 				       struct ufs_hw_queue *hwq)
 {
drivers/ufs/core/ufshcd-priv.h +0 -2
index e55c2a02c1f5..8ddc19143abf 100644
@@ -73,8 +73,6 @@ int ufshcd_get_hba_mac(struct ufs_hba *hba);
 int ufshcd_mcq_memory_alloc(struct ufs_hba *hba);
 struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba,
 					   struct request *req);
-void ufshcd_mcq_compl_all_cqes_lock(struct ufs_hba *hba,
-				    struct ufs_hw_queue *hwq);
 bool ufshcd_cmd_inflight(struct scsi_cmnd *cmd);
 int ufshcd_mcq_sq_cleanup(struct ufs_hba *hba, int task_tag);
 int ufshcd_mcq_abort(struct scsi_cmnd *cmd);
drivers/ufs/core/ufshcd.c +7 -45

87 lines not on this page · raw message

Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog