Skip to contentSkip to conversation

Fix up and simplify error recovery mechanism

PATCH Superseded Can Guo 2 participants 6 messages Jul 13, 2020

Subsystems (2)

v1 of 11 Superseded Latest is v11 →

lore
Share & tools
lore

Can Guo

author
#0 lore raw

cover Fix up and simplify error recovery mechanism

This series contains 4 changes, the first 3 of them are minor changes to make sure the main change "scsi: ufs: Fix up and simplify error recovery mechanism" work properly. The main change has been tested with error injections of multiple error types (and all kinds of mix of them) during runtime, e.g. hibern8 enter/ exit error, power mode change error and fatal/non-fatal error from IRQ context. During the test, error injections happen randomly across all contexts, e.g. clk scaling, clk gate/ungate, runtime suspend/resume and IRQ. There are a few more fixes to resolve other minor problems based on the main change, but they will be pushed after this series is taken, due to there are already too many lines in this change.

Can Guo (4):
scsi: ufs: Add checks before setting clk-gating states
scsi: ufs: Fix imbalanced scsi_block_reqs_cnt caused by ufshcd_hold()
ufs: ufs-qcom: Fix a few BUGs in func ufs_qcom_dump_dbg_regs()
scsi: ufs: Fix up and simplify error recovery mechanism

 drivers/scsi/ufs/ufs-qcom.c  |  17 +-
 drivers/scsi/ufs/ufs-sysfs.c |   1 +
 drivers/scsi/ufs/ufshcd.c    | 464 ++++++++++++++++++++++++++-----------------
 drivers/scsi/ufs/ufshcd.h    |  15 ++
 4 files changed, 308 insertions(+), 189 deletions(-)
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
author
#1 lore raw

PATCH 1/4 scsi: ufs: Add checks before setting clk-gating states

Clock gating features can be turned on/off selectively which means its state information is only important if it is enabled. This change makes sure that we only look at state of clk-gating if it is enabled.

---
 drivers/scsi/ufs/ufshcd.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
drivers/scsi/ufs/ufshcd.c +14 -3
index efc0a6c..ebf7a95 100644
@@ -1839,6 +1839,8 @@ static void ufshcd_init_clk_gating(struct ufs_hba *hba)
 	if (!ufshcd_is_clkgating_allowed(hba))
 		return;
 
+	hba->clk_gating.state = CLKS_ON;
+
 	hba->clk_gating.delay_ms = 150;
 	INIT_DELAYED_WORK(&hba->clk_gating.gate_work, ufshcd_gate_work);
 	INIT_WORK(&hba->clk_gating.ungate_work, ufshcd_ungate_work);
@@ -2538,7 +2540,8 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd)
 		err = SCSI_MLQUEUE_HOST_BUSY;
 		goto out;
 	}
-	WARN_ON(hba->clk_gating.state != CLKS_ON);
+	if (ufshcd_is_clkgating_allowed(hba))
+		WARN_ON(hba->clk_gating.state != CLKS_ON);
 
 	lrbp = &hba->lrb[tag];
 
@@ -8315,8 +8318,11 @@ static int ufshcd_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 		/* If link is active, device ref_clk can't be switched off */
 		__ufshcd_setup_clocks(hba, false, true);
 
-	hba->clk_gating.state = CLKS_OFF;
-	trace_ufshcd_clk_gating(dev_name(hba->dev), hba->clk_gating.state);
+	if (ufshcd_is_clkgating_allowed(hba)) {
+		hba->clk_gating.state = CLKS_OFF;
+		trace_ufshcd_clk_gating(dev_name(hba->dev),
+					hba->clk_gating.state);
+	}
 
 	/* Put the host controller in low power mode if possible */
 	ufshcd_hba_vreg_set_lpm(hba);
@@ -8456,6 +8462,11 @@ static int ufshcd_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 	if (hba->clk_scaling.is_allowed)
 		ufshcd_suspend_clkscaling(hba);
 	ufshcd_setup_clocks(hba, false);
+	if (ufshcd_is_clkgating_allowed(hba)) {
+		hba->clk_gating.state = CLKS_OFF;
+		trace_ufshcd_clk_gating(dev_name(hba->dev),
+					hba->clk_gating.state);
+	}
 out:
 	hba->pm_op_in_progress = 0;
 	if (ret)
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
author
#2 lore raw

PATCH 2/4 scsi: ufs: Fix imbalanced scsi_block_reqs_cnt caused by ufshcd_hold()

In reply to Can Guo

The scsi_block_reqs_cnt increased in ufshcd_hold() is supposed to be decreased back in ufshcd_ungate_work() in a paired way. However, if specific ufshcd_hold/release sequences are met, it is possible that scsi_block_reqs_cnt is increased twice but only one ungate work is queued. To make sure scsi_block_reqs_cnt is handled by ufshcd_hold() and ufshcd_ungate_work() in a paired way, increase it only if queue_work() returns true.

---
 drivers/scsi/ufs/ufshcd.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/scsi/ufs/ufshcd.c +3 -3
index ebf7a95..33214bb 100644
@@ -1611,12 +1611,12 @@ int ufshcd_hold(struct ufs_hba *hba, bool async)
 		 */
 		/* fallthrough */
 	case CLKS_OFF:
-		ufshcd_scsi_block_requests(hba);
 		hba->clk_gating.state = REQ_CLKS_ON;
 		trace_ufshcd_clk_gating(dev_name(hba->dev),
 					hba->clk_gating.state);
-		queue_work(hba->clk_gating.clk_gating_workq,
-			   &hba->clk_gating.ungate_work);
+		if (queue_work(hba->clk_gating.clk_gating_workq,
+			       &hba->clk_gating.ungate_work))
+			ufshcd_scsi_block_requests(hba);
 		/*
 		 * fall through to check if we should wait for this
 		 * work to be done or not.
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
author
#3 lore raw

PATCH 3/4 ufs: ufs-qcom: Fix a few BUGs in func ufs_qcom_dump_dbg_regs()

In reply to Can Guo

Dumping testbus registers needs to sleep a bit intermittently as there are too many of them. Skip them for those contexts where sleep is not allowed.

Meanwhile, if ufs_qcom_dump_dbg_regs() calls ufs_qcom_testbus_config() from ufshcd_suspend/resume and/or clk gate/ungate context, pm_runtime_get_sync() and ufshcd_hold() will cause racing problems. Fix it by removing the unnecessary calls of pm_runtime_get_sync() and ufshcd_hold().

---
 drivers/scsi/ufs/ufs-qcom.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)
drivers/scsi/ufs/ufs-qcom.c +7 -10
index 2e6ddb5..3743c17 100644
@@ -1604,9 +1604,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host *host)
 	 */
 	}
 	mask <<= offset;
-
-	pm_runtime_get_sync(host->hba->dev);
-	ufshcd_hold(host->hba, false);
 	ufshcd_rmwl(host->hba, TEST_BUS_SEL,
 		    (u32)host->testbus.select_major << 19,
 		    REG_UFS_CFG1);
@@ -1619,8 +1616,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host *host)
 	 * committed before returning.
 	 */
 	mb();
-	ufshcd_release(host->hba);
-	pm_runtime_put_sync(host->hba->dev);
 
 	return 0;
 }
@@ -1658,11 +1653,13 @@ static void ufs_qcom_dump_dbg_regs(struct ufs_hba *hba)
 
 	/* sleep a bit intermittently as we are dumping too much data */
 	ufs_qcom_print_hw_debug_reg_all(hba, NULL, ufs_qcom_dump_regs_wrapper);
-	udelay(1000);
-	ufs_qcom_testbus_read(hba);
-	udelay(1000);
-	ufs_qcom_print_unipro_testbus(hba);
-	udelay(1000);
+	if (in_task()) {
+		udelay(1000);
+		ufs_qcom_testbus_read(hba);
+		udelay(1000);
+		ufs_qcom_print_unipro_testbus(hba);
+		udelay(1000);
+	}
 }
 
 /**
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
author
#4 lore raw

PATCH 4/4 scsi: ufs: Fix up and simplify error recovery mechanism

In reply to Can Guo

Error recovery can be invoked from multiple paths, including hibern8 enter/exit, some vendor vops, ufshcd_eh_host_reset_handler(), resume and eh_work scheduled from IRQ context. Ultimately, these paths are trying to invoke ufshcd_reset_and_restore(), in either sync or async manner.

Having both sync and async manners at the same time has some problems

- If link recovery happens during clock scaling work, acquring scaling_lock in ufshcd_exec_dev_cmd() would cause dead lock, because scaling_lock is already held by scaling work before link recovery happens.

- If link recovery happens during ungate work, ufshcd_hold() would be called recursively. Although commit 53c12d0ef6fcb
("scsi: ufs: fix error recovery after the hibern8 exit failure") fixed a deadlock due to recursive calls of ufshcd_hold() by adding a check of eh_in_progress into ufshcd_hold(), this check added to ufshcd_hold() allows eh_work to run in parallel when link recovery is running.

- Similar concurrency can also happen to error recovery invoked from eh_host_reset_handler(), although it tries to protect it from happening by flushing eh_work before start invoking reset and restore, after flush work returns, eh_work can still be scheduled and running in parallel.

- Concurrency can even happen between eh_works. eh_work, currently queued on system_wq, is allowed to have multiple instances running in parallel, but we don't have proper protection for that.

To fix up and simplify error recovery mechanism, this change mainly does below things

o Queue eh_work on a single threaded workqueue to avoid concurrency between eh_works.

o According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when the link is broken. This actaully applies to any power mode change operations. In this change, if a power mode change operation (including AH8 enter/exit) fails, mark the link state as UIC_LINK_BROKEN_STATE and schedule eh_work. eh_work needs to do full reset and restore to recover the link stack to active. Before the link state is recovered back to active by eh_work, any power mode change attempts just return -ENOLINK to avoid consecutive HW error.

o To avoid concurrency between eh_work and link recovery, remove link recovery from hibern8 enter/exit func. If hibern8 enter/exit func fails, simply return error code and let eh_work run in parallel.

o Recover UFS hba runtime PM error in eh_work. If ufschd_suspend/resume fails due to UFS error, e.g. hibern8 enter/exit error and SSU cmd error, the runtime PM framework saves the error to dev.power.runtime_error. After that, hba runtime suspend/resume would not be invoked anymore until dev.power.runtime_error is cleared. The runtime PM error can be recovered in eh_work by calling pm_runtime_set_active() after reset and restore succeeds. Meanwhile, if pm_runtime_set_active() returns no error, which means dev.power.runtime_error is cleared, we also need to explicitly resume those scsi devices under hba in case any of them has failed to be resumed due to hba runtime resume error.

o Fix a racing problem between eh_work and ufshcd_suspend/resume. In the old code, it blocks scsi requests before schedules eh_work, but when eh_work calls pm_runtime_get_sync(), if ufshcd_suspend/resume is sending a scsi cmd, most likely the SSU cmd, pm_runtime_get_sync() will never return because scsi requests were blocked. To fix this racing problem,
o Don't block scsi requests before schedule eh_work, but let eh_work block scsi requests when eh_work is ready to start error recovery.
o Meanwhile, if eh_work is schueduled due to fatal error, don't requeue the scsi cmds sent from ufshcd_suspend/resume path, but simply let the scsi cmds fail. If the scsi cmds fail, hba runtime suspend/resume fails too, but it does hurt since eh_work recovers hba runtime PM error.

o Move host/regs dump in ufshcd_check_errors() to eh_work because heavy dump in IRQ context can lead to stability issues. In addition, some clean up in ufshcd_print_host_regs() and ufshcd_print_host_state().

---
 drivers/scsi/ufs/ufs-sysfs.c |   1 +
 drivers/scsi/ufs/ufshcd.c    | 441 ++++++++++++++++++++++++++-----------------
 drivers/scsi/ufs/ufshcd.h    |  15 ++
 3 files changed, 284 insertions(+), 173 deletions(-)
drivers/scsi/ufs/ufs-sysfs.c +1 -0
index 2d71d23..02d379f00 100644
@@ -16,6 +16,7 @@ static const char *ufschd_uic_link_state_to_string(
 	case UIC_LINK_OFF_STATE:	return "OFF";
 	case UIC_LINK_ACTIVE_STATE:	return "ACTIVE";
 	case UIC_LINK_HIBERN8_STATE:	return "HIBERN8";
+	case UIC_LINK_BROKEN_STATE:	return "BROKEN";
 	default:			return "UNKNOWN";
 	}
 }
drivers/scsi/ufs/ufshcd.c +268 -173

741 lines not on this page · raw message

drivers/scsi/ufs/ufshcd.h +15 -0
index 656c069..585e58b 100644
@@ -90,6 +90,7 @@ enum uic_link_state {
 	UIC_LINK_OFF_STATE	= 0, /* Link powered down or disabled */
 	UIC_LINK_ACTIVE_STATE	= 1, /* Link is in Fast/Slow/Sleep state */
 	UIC_LINK_HIBERN8_STATE	= 2, /* Link is in Hibernate state */
+	UIC_LINK_BROKEN_STATE	= 3, /* Link is in broken state */
 };
 
 #define ufshcd_is_link_off(hba) ((hba)->uic_link_state == UIC_LINK_OFF_STATE)
@@ -97,11 +98,15 @@ enum uic_link_state {
 				    UIC_LINK_ACTIVE_STATE)
 #define ufshcd_is_link_hibern8(hba) ((hba)->uic_link_state == \
 				    UIC_LINK_HIBERN8_STATE)
+#define ufshcd_is_link_broken(hba) ((hba)->uic_link_state == \
+				   UIC_LINK_BROKEN_STATE)
 #define ufshcd_set_link_off(hba) ((hba)->uic_link_state = UIC_LINK_OFF_STATE)
 #define ufshcd_set_link_active(hba) ((hba)->uic_link_state = \
 				    UIC_LINK_ACTIVE_STATE)
 #define ufshcd_set_link_hibern8(hba) ((hba)->uic_link_state = \
 				    UIC_LINK_HIBERN8_STATE)
+#define ufshcd_set_link_broken(hba) ((hba)->uic_link_state = \
+				    UIC_LINK_BROKEN_STATE)
 
 #define ufshcd_set_ufs_dev_active(h) \
 	((h)->curr_dev_pwr_mode = UFS_ACTIVE_PWR_MODE)
@@ -349,6 +354,7 @@ struct ufs_clk_gating {
 	struct device_attribute delay_attr;
 	struct device_attribute enable_attr;
 	bool is_enabled;
+	bool held_by_pm;
 	int active_reqs;
 	struct workqueue_struct *clk_gating_workq;
 };
@@ -406,6 +412,8 @@ struct ufs_err_reg_hist {
 
 /**
  * struct ufs_stats - keeps usage/err statistics
+ * @last_intr_status: record the last interrupt status.
+ * @last_intr_ts: record the last interrupt timestamp.
  * @hibern8_exit_cnt: Counter to keep track of number of exits,
  *		reset this after link-startup.
  * @last_hibern8_exit_tstamp: Set time after the hibern8 exit.
@@ -425,6 +433,9 @@ struct ufs_err_reg_hist {
  * @tsk_abort: tracks task abort events
  */
 struct ufs_stats {
+	u32 last_intr_status;
+	ktime_t last_intr_ts;
+
 	u32 hibern8_exit_cnt;
 	ktime_t last_hibern8_exit_tstamp;
 
@@ -608,12 +619,14 @@ struct ufs_hba_variant_params {
  * @intr_mask: Interrupt Mask Bits
  * @ee_ctrl_mask: Exception event control mask
  * @is_powered: flag to check if HBA is powered
+ * @eh_wq: Workqueue that eh_work works on
  * @eh_work: Worker to handle UFS errors that require s/w attention
  * @eeh_work: Worker to handle exception events
  * @errors: HBA errors
  * @uic_error: UFS interconnect layer error status
  * @saved_err: sticky error mask
  * @saved_uic_err: sticky UIC error mask
+ * @force_reset: flag to force eh_work perform a full reset
  * @silence_err_logs: flag to silence error logs
  * @dev_cmd: ufs device management command information
  * @last_dme_cmd_tstamp: time stamp of the last completed DME command
@@ -702,6 +715,7 @@ struct ufs_hba {
 	bool is_powered;
 
 	/* Work Queues */
+	struct workqueue_struct *eh_wq;
 	struct work_struct eh_work;
 	struct work_struct eeh_work;
 
@@ -711,6 +725,7 @@ struct ufs_hba {
 	u32 saved_err;
 	u32 saved_uic_err;
 	struct ufs_stats ufs_stats;
+	bool force_reset;
 	bool silence_err_logs;
 
 	/* Device management request data */
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
#5 lore raw

Re: [PATCH v1 4/4] scsi: ufs: Fix up and simplify error recovery mechanism

Thank you for the patch! Yet something to improve:

Hi Can,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on scsi/for-next]
[also build test ERROR on mkp-scsi/for-next v5.8-rc5 next-20200713] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Can-Guo/Fix-up-and-simplify-error-recovery-mechanism/20200713-130435
base:   https://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git for-next
config: alpha-allyesconfig (attached as .config)
compiler: alpha-linux-gcc (GCC) 9.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=alpha

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

drivers/scsi/ufs/ufshcd.c: In function 'ufshcd_err_handler':

>> drivers/scsi/ufs/ufshcd.c:5755:10: error: 'struct request_queue' has no member named 'dev'
    5755 |     if (q->dev && (q->rpm_status == RPM_SUSPENDED ||
         |          ^~
>> drivers/scsi/ufs/ufshcd.c:5755:21: error: 'struct request_queue' has no member named 'rpm_status'
    5755 |     if (q->dev && (q->rpm_status == RPM_SUSPENDED ||
         |                     ^~
   drivers/scsi/ufs/ufshcd.c:5756:14: error: 'struct request_queue' has no member named 'rpm_status'
    5756 |             q->rpm_status == RPM_SUSPENDING))
         |              ^~
   drivers/scsi/ufs/ufshcd.c:5757:25: error: 'struct request_queue' has no member named 'dev'
    5757 |      pm_request_resume(q->dev);
         |                         ^~

vim +5755 drivers/scsi/ufs/ufshcd.c

5579
5580 /**
5581 * ufshcd_err_handler - handle UFS errors that require s/w attention
5582 * @work: pointer to work structure
5583 */
5584 static void ufshcd_err_handler(struct work_struct *work)
5585 {
5586 struct ufs_hba *hba;
5587 struct Scsi_Host *shost;
5588 struct scsi_device *sdev;
5589 unsigned long flags;
5590 u32 err_xfer = 0;
5591 u32 err_tm = 0;
5592 int reset_err = -1;
5593 int tag;
5594 bool needs_reset = false;
5595
5596 hba = container_of(work, struct ufs_hba, eh_work);
5597 shost = hba->host;
5598
5599 spin_lock_irqsave(hba->host->host_lock, flags);
5600 if ((hba->ufshcd_state == UFSHCD_STATE_ERROR) ||
5601 (!(hba->saved_err || hba->saved_uic_err || hba->force_reset) &&
5602 !ufshcd_is_link_broken(hba))) {
5603 if (hba->ufshcd_state != UFSHCD_STATE_ERROR)
5604 hba->ufshcd_state = UFSHCD_STATE_OPERATIONAL;
5605 spin_unlock_irqrestore(hba->host->host_lock, flags);
5606 return;
5607 }
5608 ufshcd_set_eh_in_progress(hba);
5609 spin_unlock_irqrestore(hba->host->host_lock, flags);
5610 pm_runtime_get_sync(hba->dev);
5611 /*
5612 * Don't assume anything of pm_runtime_get_sync(), if resume fails,
5613 * irq and clocks can be OFF, and powers can be OFF or in LPM.
5614 */
5615 ufshcd_setup_vreg(hba, true);
5616 ufshcd_config_vreg_hpm(hba, hba->vreg_info.vccq);
5617 ufshcd_config_vreg_hpm(hba, hba->vreg_info.vccq2);
5618 ufshcd_setup_hba_vreg(hba, true);
5619 ufshcd_enable_irq(hba);
5620
5621 ufshcd_hold(hba, false);
5622 if (!ufshcd_is_clkgating_allowed(hba))
5623 ufshcd_setup_clocks(hba, true);
5624
5625 if (ufshcd_is_clkscaling_supported(hba)) {
5626 cancel_work_sync(&hba->clk_scaling.suspend_work);
5627 cancel_work_sync(&hba->clk_scaling.resume_work);
5628 ufshcd_suspend_clkscaling(hba);
5629 }
5630
5631 spin_lock_irqsave(hba->host->host_lock, flags);
5632 ufshcd_scsi_block_requests(hba);
5633 hba->ufshcd_state = UFSHCD_STATE_RESET;
5634
5635 /* Complete requests that have door-bell cleared by h/w */
5636 ufshcd_complete_requests(hba);
5637
5638 if (hba->dev_quirks & UFS_DEVICE_QUIRK_RECOVERY_FROM_DL_NAC_ERRORS) {
5639 bool ret;
5640
5641 spin_unlock_irqrestore(hba->host->host_lock, flags);
5642 /* release the lock as ufshcd_quirk_dl_nac_errors() may sleep */
5643 ret = ufshcd_quirk_dl_nac_errors(hba);
5644 spin_lock_irqsave(hba->host->host_lock, flags);
5645 if (!ret && !hba->force_reset && ufshcd_is_link_active(hba))
5646 goto skip_err_handling;
5647 }
5648
5649 if (hba->force_reset || ufshcd_is_link_broken(hba) ||
5650 ufshcd_is_saved_err_fatal(hba) ||
5651 ((hba->saved_err & UIC_ERROR) &&
5652 (hba->saved_uic_err & (UFSHCD_UIC_DL_NAC_RECEIVED_ERROR |
5653 UFSHCD_UIC_DL_TCx_REPLAY_ERROR))))
5654 needs_reset = true;
5655
5656 if (hba->saved_err & (INT_FATAL_ERRORS | UIC_ERROR |
5657 UFSHCD_UIC_HIBERN8_MASK)) {
5658 dev_err(hba->dev, "%s: saved_err 0x%x saved_uic_err 0x%x\n",
5659 __func__, hba->saved_err, hba->saved_uic_err);
5660 spin_unlock_irqrestore(hba->host->host_lock, flags);
5661 ufshcd_print_host_state(hba);
5662 ufshcd_print_pwr_info(hba);
5663 ufshcd_print_host_regs(hba);
5664 ufshcd_print_tmrs(hba, hba->outstanding_tasks);
5665 spin_lock_irqsave(hba->host->host_lock, flags);
5666 }
5667
5668 /*
5669 * if host reset is required then skip clearing the pending
5670 * transfers forcefully because they will get cleared during
5671 * host reset and restore
5672 */
5673 if (needs_reset)
5674 goto skip_pending_xfer_clear;
5675
5676 /* release lock as clear command might sleep */
5677 spin_unlock_irqrestore(hba->host->host_lock, flags);
5678 /* Clear pending transfer requests */
5679 for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) {
5680 if (ufshcd_clear_cmd(hba, tag)) {
5681 err_xfer = true;
5682 goto lock_skip_pending_xfer_clear;
5683 }
5684 }
5685
5686 /* Clear pending task management requests */
5687 for_each_set_bit(tag, &hba->outstanding_tasks, hba->nutmrs) {
5688 if (ufshcd_clear_tm_cmd(hba, tag)) {
5689 err_tm = true;
5690 goto lock_skip_pending_xfer_clear;
5691 }
5692 }
5693
5694 lock_skip_pending_xfer_clear:
5695 spin_lock_irqsave(hba->host->host_lock, flags);
5696
5697 /* Complete the requests that are cleared by s/w */
5698 ufshcd_complete_requests(hba);
5699
5700 if (err_xfer || err_tm)
5701 needs_reset = true;
5702
5703 skip_pending_xfer_clear:
5704 /* Fatal errors need reset */
5705 if (needs_reset) {
5706 unsigned long max_doorbells = (1UL << hba->nutrs) - 1;
5707
5708 /*
5709 * ufshcd_reset_and_restore() does the link reinitialization
5710 * which will need atleast one empty doorbell slot to send the
5711 * device management commands (NOP and query commands).
5712 * If there is no slot empty at this moment then free up last
5713 * slot forcefully.
5714 */
5715 if (hba->outstanding_reqs == max_doorbells)
5716 __ufshcd_transfer_req_compl(hba,
5717 (1UL << (hba->nutrs - 1)));
5718
5719 hba->force_reset = false;
5720 spin_unlock_irqrestore(hba->host->host_lock, flags);
5721 reset_err = ufshcd_reset_and_restore(hba);
5722 spin_lock_irqsave(hba->host->host_lock, flags);
5723 if (reset_err)
5724 dev_err(hba->dev, "%s: reset and restore failed\n",
5725 __func__);
5726 }
5727
5728 skip_err_handling:
5729 if (!needs_reset) {
5730 if (hba->ufshcd_state == UFSHCD_STATE_RESET)
5731 hba->ufshcd_state = UFSHCD_STATE_OPERATIONAL;
5732 if (hba->saved_err || hba->saved_uic_err)
5733 dev_err_ratelimited(hba->dev, "%s: exit: saved_err 0x%x saved_uic_err 0x%x",
5734 __func__, hba->saved_err, hba->saved_uic_err);
5735 }
5736
5737 if (!reset_err) {
5738 int ret;
5739 struct request_queue *q;
5740
5741 spin_unlock_irqrestore(hba->host->host_lock, flags);
5742 /*
5743 * Set RPM status of hba device to RPM_ACTIVE,
5744 * this also clears its runtime error.
5745 */
5746 ret = pm_runtime_set_active(hba->dev);
5747 /*
5748 * If hba device had runtime error, explicitly resume
5749 * its scsi devices so that block layer can wake up
5750 * those waiting in blk_queue_enter().
5751 */
5752 if (!ret) {
5753 list_for_each_entry(sdev, &shost->__devices, siblings) {
5754 q = sdev->request_queue;

> 5755					if (q->dev && (q->rpm_status == RPM_SUSPENDED ||

5756 q->rpm_status == RPM_SUSPENDING))
5757 pm_request_resume(q->dev);
5758 }
5759 }
5760 spin_lock_irqsave(hba->host->host_lock, flags);
5761 }
5762
5763 /* If clk_gating is held by pm ops, release it */
5764 if (pm_runtime_active(hba->dev) && hba->clk_gating.held_by_pm) {
5765 hba->clk_gating.held_by_pm = false;
5766 __ufshcd_release(hba);
5767 }
5768
5769 ufshcd_clear_eh_in_progress(hba);
5770 spin_unlock_irqrestore(hba->host->host_lock, flags);
5771 ufshcd_scsi_unblock_requests(hba);
5772 ufshcd_release(hba);
5773 if (ufshcd_is_clkscaling_supported(hba))
5774 ufshcd_resume_clkscaling(hba);
5775 pm_runtime_put_noidle(hba->dev);
5776 }
5777

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org