Fix up and simplify error recovery mechanism
Subsystems (2)
v2 cover Fix up and simplify error recovery mechanism
This series contains 4 changes, the first 3 of them are minor changes to make sure the main change "scsi: ufs: Fix up and simplify error recovery mechanism" work properly. The main change has been tested with error injections of multiple error types (and all kinds of mix of them) during runtime, e.g. hibern8 enter/ exit error, power mode change error and fatal/non-fatal error from IRQ context. During the test, error injections happen randomly across all contexts, e.g. clk scaling, clk gate/ungate, runtime suspend/resume and IRQ. There are a few more fixes to resolve other minor problems based on the main change, but they will be pushed after this series is taken, due to there are already too many lines in this change.
Change since v1:
- Fixed a compilation error in case that CONFIG_PM is N
Can Guo (4):
scsi: ufs: Add checks before setting clk-gating states
scsi: ufs: Fix imbalanced scsi_block_reqs_cnt caused by ufshcd_hold()
ufs: ufs-qcom: Fix a few BUGs in func ufs_qcom_dump_dbg_regs()
scsi: ufs: Fix up and simplify error recovery mechanism
drivers/scsi/ufs/ufs-qcom.c | 17 +- drivers/scsi/ufs/ufs-sysfs.c | 1 + drivers/scsi/ufs/ufshcd.c | 465 ++++++++++++++++++++++++++----------------- drivers/scsi/ufs/ufshcd.h | 15 ++ 4 files changed, 309 insertions(+), 189 deletions(-)
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
PATCH v2 1/4 scsi: ufs: Add checks before setting clk-gating states
Clock gating features can be turned on/off selectively which means its state information is only important if it is enabled. This change makes sure that we only look at state of clk-gating if it is enabled.
--- drivers/scsi/ufs/ufshcd.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-)
drivers/scsi/ufs/ufshcd.c +14 -3
@@ -1839,6 +1839,8 @@ static void ufshcd_init_clk_gating(struct ufs_hba *hba)if (!ufshcd_is_clkgating_allowed (hba))return ; + hba->clk_gating.state =CLKS_ON ; + hba->clk_gating.delay_ms =150 ;INIT_DELAYED_WORK (&hba->clk_gating.gate_work, ufshcd_gate_work);INIT_WORK (&hba->clk_gating.ungate_work, ufshcd_ungate_work);
@@ -2538,7 +2540,8 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) err =SCSI_MLQUEUE_HOST_BUSY ; goto out; } -WARN_ON (hba->clk_gating.state !=CLKS_ON ); +if (ufshcd_is_clkgating_allowed (hba)) +WARN_ON (hba->clk_gating.state !=CLKS_ON ); lrbp = &hba->lrb[tag];
@@ -8315,8 +8318,11 @@ static int ufshcd_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op)/* If link is active, device ref_clk can't be switched off */ __ufshcd_setup_clocks (hba, false, true); - hba->clk_gating.state =CLKS_OFF ; -trace_ufshcd_clk_gating (dev_name (hba->dev), hba->clk_gating.state); +if (ufshcd_is_clkgating_allowed (hba)) { + hba->clk_gating.state =CLKS_OFF ; +trace_ufshcd_clk_gating (dev_name (hba->dev), + hba->clk_gating.state); + }/* Put the host controller in low power mode if possible */ ufshcd_hba_vreg_set_lpm (hba);
@@ -8456,6 +8462,11 @@ static int ufshcd_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op)if (hba->clk_scaling.is_allowed)ufshcd_suspend_clkscaling (hba);ufshcd_setup_clocks (hba, false); +if (ufshcd_is_clkgating_allowed (hba)) { + hba->clk_gating.state =CLKS_OFF ; +trace_ufshcd_clk_gating (dev_name (hba->dev), + hba->clk_gating.state); + } out: hba->pm_op_in_progress =0 ;if (ret)
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [PATCH v2 1/4] scsi: ufs: Add checks before setting clk-gating states
drivers/scsi/ufs/ufshcd.c static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) · line 2544
Quoted 7 lines · Can Guo
drivers/scsi/ufs/ufshcd.c +2
-1@@ -2538,7 +2540,8 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) err =SCSI_MLQUEUE_HOST_BUSY ; goto out; } -WARN_ON (hba->clk_gating.state !=CLKS_ON ); +if (ufshcd_is_clkgating_allowed (hba)) +WARN_ON (hba->clk_gating.state !=CLKS_ON );
A minor comment: has it been considered to change this into the following?
WARN_ON(ufshcd_is_clkgating_allowed(hba) && hba->clk_gating.state != CLKS_ON);
Thanks,
Bart.
Re: [PATCH v2 1/4] scsi: ufs: Add checks before setting clk-gating states
Hi Bart,
Quoted 13 lines · Bart Van Assche
drivers/scsi/ufs/ufshcd.c static int ufshcd_queuecommand(struct Scsi_Host · line 2545
Quoted 8 lines · Can Guo
drivers/scsi/ufs/ufshcd.c +2
-1@@ -2538,7 +2540,8 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) err =SCSI_MLQUEUE_HOST_BUSY ; goto out; } -WARN_ON (hba->clk_gating.state !=CLKS_ON ); +if (ufshcd_is_clkgating_allowed (hba)) +WARN_ON (hba->clk_gating.state !=CLKS_ON );A minor comment: has it been considered to change this into the following?
Quoted 6 lines
WARN_ON(ufshcd_is_clkgating_allowed(hba) && hba->clk_gating.state != CLKS_ON);Thanks,
Bart.
Sure, will do that.
Thanks,
Can Guo.
PATCH v2 2/4 scsi: ufs: Fix imbalanced scsi_block_reqs_cnt caused by ufshcd_hold()
In reply to Can GuoThe scsi_block_reqs_cnt increased in ufshcd_hold() is supposed to be decreased back in ufshcd_ungate_work() in a paired way. However, if specific ufshcd_hold/release sequences are met, it is possible that scsi_block_reqs_cnt is increased twice but only one ungate work is queued. To make sure scsi_block_reqs_cnt is handled by ufshcd_hold() and ufshcd_ungate_work() in a paired way, increase it only if queue_work() returns true.
--- drivers/scsi/ufs/ufshcd.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/scsi/ufs/ufshcd.c +3 -3
@@ -1611,12 +1611,12 @@ int ufshcd_hold(struct ufs_hba *hba, bool async)*/ /* fallthrough */ case CLKS_OFF : -ufshcd_scsi_block_requests (hba); hba->clk_gating.state =REQ_CLKS_ON ;trace_ufshcd_clk_gating (dev_name (hba->dev), hba->clk_gating.state); -queue_work (hba->clk_gating.clk_gating_workq, - &hba->clk_gating.ungate_work); +if (queue_work (hba->clk_gating.clk_gating_workq, + &hba->clk_gating.ungate_work)) +ufshcd_scsi_block_requests (hba);/* * fall through to check if we should wait for this * work to be done or not.
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [PATCH v2 2/4] scsi: ufs: Fix imbalanced scsi_block_reqs_cnt caused by ufshcd_hold()
Quoted 23 lines · Can Guo
The
scsi_block_reqs_cntincreased inufshcd_hold()is supposed to be decreased back inufshcd_ungate_work()in a paired way. However, if specific ufshcd_hold/release sequences are met, it is possible thatscsi_block_reqs_cntis increased twice but only one ungate work is queued. To make surescsi_block_reqs_cntis handled byufshcd_hold()andufshcd_ungate_work()in a paired way, increase it only ifqueue_work()returns true.Signed-off-by Can Guo <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/scsi/ufs/ufshcd.c +0
-1@@ -1611,12 +1611,12 @@ int ufshcd_hold(struct ufs_hba *hba, bool async)*/ /* fallthrough */ case CLKS_OFF : -ufshcd_scsi_block_requests (hba); hba->clk_gating.state =REQ_CLKS_ON ;
drivers/scsi/ufs/ufshcd.c int ufshcd_hold(struct ufs_hba *hba, bool async) · line 1622
Quoted 10 lines
drivers/scsi/ufs/ufshcd.c +3
-2trace_ufshcd_clk_gating (dev_name (hba->dev), hba->clk_gating.state); -queue_work (hba->clk_gating.clk_gating_workq, - &hba->clk_gating.ungate_work); +if (queue_work (hba->clk_gating.clk_gating_workq, + &hba->clk_gating.ungate_work)) +ufshcd_scsi_block_requests (hba);/* * fall through to check if we should wait for this * work to be done or not.
Since "ungate_work" involves calling ufshcd_scsi_unblock_requests() and since this patch changes the order in which ufshcd_scsi_block_requests() and queue_work() are called, I think this patch introduces a race condition. Has it been considered to leave the ufshcd_scsi_block_requests() call where it is and to call ufshcd_scsi_unblock_requests() if queue_work() fails?
Thanks,
Bart.
Re: [PATCH v2 2/4] scsi: ufs: Fix imbalanced scsi_block_reqs_cnt caused by ufshcd_hold()
Quoted 43 lines · Bart Van Assche
Quoted 33 lines · Can Guo
The
scsi_block_reqs_cntincreased inufshcd_hold()is supposed to be decreased back inufshcd_ungate_work()in a paired way. However, if specific ufshcd_hold/release sequences are met, it is possible thatscsi_block_reqs_cntis increased twice but only one ungate work is queued. To make surescsi_block_reqs_cntis handled byufshcd_hold()andufshcd_ungate_work()in a paired way, increase it only ifqueue_work()returns true.Signed-off-by Can Guo <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)drivers/scsi/ufs/ufshcd.c +3
-3@@ -1611,12 +1611,12 @@ int ufshcd_hold(struct ufs_hba *hba, boolasync) */ /* fallthrough */ case CLKS_OFF : -ufshcd_scsi_block_requests (hba); hba->clk_gating.state =REQ_CLKS_ON ;trace_ufshcd_clk_gating (dev_name (hba->dev), hba->clk_gating.state); -queue_work (hba->clk_gating.clk_gating_workq, - &hba->clk_gating.ungate_work); +if (queue_work (hba->clk_gating.clk_gating_workq, + &hba->clk_gating.ungate_work)) +ufshcd_scsi_block_requests (hba);/* * fall through to check if we should wait for this Quoted 1 line
* work to be done or not.
Since "
ungate_work" involves callingufshcd_scsi_unblock_requests()and since this patch changes the order in whichufshcd_scsi_block_requests()andqueue_work()are called, I think this patch introduces a race condition. Has it been considered to leave theufshcd_scsi_block_requests()call where it is and to callufshcd_scsi_unblock_requests()ifqueue_work()fails?
Quoted 3 lines
Thanks,
Bart.
Hi Bart,
The racing does not exist due to we still hold the spin lock here. Before release the spin lock, the ungate_work, even it starts to run, cannot move forward as it needs to require the spin lock once in the entrance.
static void ufshcd_ungate_work(struct work_struct *work) {
...
spin_lock_irqsave(hba->host->host_lock, flags);
if (hba->clk_gating.state == CLKS_ON) {
spin_unlock_irqrestore(hba->host->host_lock, flags);
goto unblock_reqs;
}...
}
PATCH v2 3/4 ufs: ufs-qcom: Fix a few BUGs in func ufs_qcom_dump_dbg_regs()
In reply to Can GuoDumping testbus registers needs to sleep a bit intermittently as there are too many of them. Skip them for those contexts where sleep is not allowed.
Meanwhile, if ufs_qcom_dump_dbg_regs() calls ufs_qcom_testbus_config() from ufshcd_suspend/resume and/or clk gate/ungate context, pm_runtime_get_sync() and ufshcd_hold() will cause racing problems. Fix it by removing the unnecessary calls of pm_runtime_get_sync() and ufshcd_hold().
--- drivers/scsi/ufs/ufs-qcom.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)
drivers/scsi/ufs/ufs-qcom.c +7 -10
@@ -1604,9 +1604,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host *host)*/ } mask <<= offset; - -pm_runtime_get_sync (host->hba->dev); -ufshcd_hold (host->hba, false);ufshcd_rmwl (host->hba,TEST_BUS_SEL , (u32 )host->testbus.select_major <<19 ,REG_UFS_CFG1 );
@@ -1619,8 +1616,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host *host)* committed before returning. */ mb (); -ufshcd_release (host->hba); -pm_runtime_put_sync (host->hba->dev);return 0 ; }
@@ -1658,11 +1653,13 @@ static void ufs_qcom_dump_dbg_regs(struct ufs_hba *hba)/* sleep a bit intermittently as we are dumping too much data */ ufs_qcom_print_hw_debug_reg_all (hba,NULL , ufs_qcom_dump_regs_wrapper); -udelay (1000 ); -ufs_qcom_testbus_read (hba); -udelay (1000 ); -ufs_qcom_print_unipro_testbus (hba); -udelay (1000 ); +if (in_task ()) { +udelay (1000 ); +ufs_qcom_testbus_read (hba); +udelay (1000 ); +ufs_qcom_print_unipro_testbus (hba); +udelay (1000 ); + } }/**
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [PATCH v2 3/4] ufs: ufs-qcom: Fix a few BUGs in func ufs_qcom_dump_dbg_regs()
Quoted 43 lines · Can Guo
Dumping testbus registers needs to sleep a bit intermittently as there are too many of them. Skip them for those contexts where sleep is not allowed.
Meanwhile, if
ufs_qcom_dump_dbg_regs()callsufs_qcom_testbus_config()from ufshcd_suspend/resume and/or clk gate/ungate context,pm_runtime_get_sync()andufshcd_hold()will cause racing problems. Fix it by removing the unnecessary calls ofpm_runtime_get_sync()andufshcd_hold().Signed-off-by Can Guo <[email protected]>--- drivers/scsi/ufs/ufs-qcom.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)drivers/scsi/ufs/ufs-qcom.c +0
-8@@ -1604,9 +1604,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host *host)*/ } mask <<= offset; - -pm_runtime_get_sync (host->hba->dev); -ufshcd_hold (host->hba, false);ufshcd_rmwl (host->hba,TEST_BUS_SEL , (u32 )host->testbus.select_major <<19 ,REG_UFS_CFG1 );@@ -1619,8 +1616,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host *host)* committed before returning. */ mb (); -ufshcd_release (host->hba); -pm_runtime_put_sync (host->hba->dev);return 0 ; }@@ -1658,11 +1653,13 @@ static void ufs_qcom_dump_dbg_regs(struct ufs_hba *hba)/* sleep a bit intermittently as we are dumping too much data */ ufs_qcom_print_hw_debug_reg_all (hba,NULL , ufs_qcom_dump_regs_wrapper); -udelay (1000 ); -ufs_qcom_testbus_read (hba); -udelay (1000 );
drivers/scsi/ufs/ufs-qcom.c static void ufs_qcom_dump_dbg_regs(struct ufs_hba *hba) · line 1663
Quoted 10 lines
drivers/scsi/ufs/ufs-qcom.c +7
-2-ufs_qcom_print_unipro_testbus (hba); -udelay (1000 ); +if (in_task ()) { +udelay (1000 ); +ufs_qcom_testbus_read (hba); +udelay (1000 ); +ufs_qcom_print_unipro_testbus (hba); +udelay (1000 ); + } }
It is not clear to me how udelay() calls can help in code that takes long since these functions use busy-waiting? Should the udelay() calls perhaps be changed into cond_resched() calls?
Thanks,
Bart.
Re: [PATCH v2 3/4] ufs: ufs-qcom: Fix a few BUGs in func ufs_qcom_dump_dbg_regs()
Hi Bart,
Quoted 63 lines · Bart Van Assche
Quoted 47 lines · Can Guo
Dumping testbus registers needs to sleep a bit intermittently as there are too many of them. Skip them for those contexts where sleep is not allowed.
Meanwhile, if
ufs_qcom_dump_dbg_regs()callsufs_qcom_testbus_config()from ufshcd_suspend/resume and/or clk gate/ungate context,pm_runtime_get_sync()andufshcd_hold()will cause racing problems. Fix it by removing the unnecessary calls ofpm_runtime_get_sync()andufshcd_hold().Signed-off-by Can Guo <[email protected]>--- drivers/scsi/ufs/ufs-qcom.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)drivers/scsi/ufs/ufs-qcom.c +0
-3@@ -1604,9 +1604,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host*host) */ } mask <<= offset; - -pm_runtime_get_sync (host->hba->dev); -ufshcd_hold (host->hba, false); ufshcd_rmwl(host->hba, TEST_BUS_SEL, (u32)host->testbus.select_major << 19,drivers/scsi/ufs/ufs-qcom.c +0
-2REG_UFS_CFG1);@@ -1619,8 +1616,6 @@ int ufs_qcom_testbus_config(struct ufs_qcom_host*host) * committed before returning. */ mb (); -ufshcd_release (host->hba); -pm_runtime_put_sync (host->hba->dev);return 0 ;}
drivers/scsi/ufs/ufs-qcom.c +0
-3@@ -1658,11 +1653,13 @@ static void ufs_qcom_dump_dbg_regs(struct ufs_hba *hba)/* sleep a bit intermittently as we are dumping too much data */ ufs_qcom_print_hw_debug_reg_all (hba ,NULL ,ufs_qcom_dump_regs_wrapper ); -udelay (1000 ); -ufs_qcom_testbus_read (hba); -udelay (1000 );drivers/scsi/ufs/ufs-qcom.c static void ufs_qcom_dump_dbg_regs(struct · line 1665
Quoted 10 lines
drivers/scsi/ufs/ufs-qcom.c +7
-2-ufs_qcom_print_unipro_testbus (hba); -udelay (1000 ); +if (in_task ()) { +udelay (1000 ); +ufs_qcom_testbus_read (hba); +udelay (1000 ); +ufs_qcom_print_unipro_testbus (hba); +udelay (1000 ); + } }It is not clear to me how
udelay()calls can help in code that takes long since these functions use busy-waiting? Should theudelay()calls perhaps be changed intocond_resched()calls?
Quoted 3 lines
Thanks,
Bart.
Maybe you are right, but this is not the purpose of this change. I am just trying to make sure this func can be invoked from any contexts without making troubles like schedule while atomic and/or race conditions.
Thanks,
Can Guo.
PATCH v2 4/4 scsi: ufs: Fix up and simplify error recovery mechanism
In reply to Can GuoError recovery can be invoked from multiple paths, including hibern8 enter/exit, some vendor vops, ufshcd_eh_host_reset_handler(), resume and eh_work scheduled from IRQ context. Ultimately, these paths are trying to invoke ufshcd_reset_and_restore(), in either sync or async manner.
Having both sync and async manners at the same time has some problems
- If link recovery happens during clock scaling work, acquring scaling_lock in ufshcd_exec_dev_cmd() would cause dead lock, because scaling_lock is already held by scaling work before link recovery happens.
- If link recovery happens during ungate work, ufshcd_hold() would be called recursively. Although commit 53c12d0ef6fcb
("scsi: ufs: fix error recovery after the hibern8 exit failure") fixed a deadlock due to recursive calls of ufshcd_hold() by adding a check of eh_in_progress into ufshcd_hold(), this check added to ufshcd_hold() allows eh_work to run in parallel when link recovery is running.
- Similar concurrency can also happen to error recovery invoked from eh_host_reset_handler(), although it tries to protect it from happening by flushing eh_work before start invoking reset and restore, after flush work returns, eh_work can still be scheduled and running in parallel.
- Concurrency can even happen between eh_works. eh_work, currently queued on system_wq, is allowed to have multiple instances running in parallel, but we don't have proper protection for that.
To fix up and simplify error recovery mechanism, this change mainly does below things
o Queue eh_work on a single threaded workqueue to avoid concurrency between eh_works.
o According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when the link is broken. This actaully applies to any power mode change operations. In this change, if a power mode change operation (including AH8 enter/exit) fails, mark the link state as UIC_LINK_BROKEN_STATE and schedule eh_work. eh_work needs to do full reset and restore to recover the link back to active. Before the link state is recovered to active by eh_work, any power mode change attempts just return -ENOLINK to avoid consecutive HW error.
o To avoid concurrency between eh_work and link recovery, remove link recovery from hibern8 enter/exit func. If hibern8 enter/exit func fails, simply return error code and let eh_work run in parallel.
o Recover UFS hba runtime PM error in eh_work. If ufschd_suspend/resume fails due to UFS error, e.g. hibern8 enter/exit error and SSU cmd error, the runtime PM framework saves the error to dev.power.runtime_error. After that, hba runtime suspend/resume would not be invoked anymore until dev.power.runtime_error is cleared. The runtime PM error can be recovered in eh_work by calling pm_runtime_set_active() after reset and restore succeeds. Meanwhile, if pm_runtime_set_active() returns no error, which means dev.power.runtime_error is cleared, we also need to explicitly resume those scsi devices under hba in case any of them has failed to be resumed due to hba runtime resume error.
o Fix a racing problem between eh_work and ufshcd_suspend/resume. In the old code, it blocks scsi requests before schedules eh_work, but when eh_work calls pm_runtime_get_sync(), if ufshcd_suspend/resume is sending a scsi cmd, most likely the SSU cmd, pm_runtime_get_sync() will never return because scsi requests were blocked. To fix this racing problem,
o Don't block scsi requests before schedule eh_work, but let eh_work block scsi requests when eh_work is ready to start error recovery.
o Meanwhile, if eh_work is schueduled due to fatal error, don't requeue the scsi cmds sent from ufshcd_suspend/resume path, but simply let the scsi cmds fail. If the scsi cmds fail, hba runtime suspend/resume fails too, but it does hurt since eh_work recovers hba runtime PM error.
o Move host/regs dump in ufshcd_check_errors() to eh_work because heavy dump in IRQ context can lead to stability issues. In addition, some clean up in ufshcd_print_host_regs() and ufshcd_print_host_state().
--- drivers/scsi/ufs/ufs-sysfs.c | 1 + drivers/scsi/ufs/ufshcd.c | 442 ++++++++++++++++++++++++++----------------- drivers/scsi/ufs/ufshcd.h | 15 ++ 3 files changed, 285 insertions(+), 173 deletions(-)
drivers/scsi/ufs/ufs-sysfs.c +1 -0
@@ -16,6 +16,7 @@ static const char *ufschd_uic_link_state_to_string(case UIC_LINK_OFF_STATE :return "OFF" ;case UIC_LINK_ACTIVE_STATE :return "ACTIVE" ;case UIC_LINK_HIBERN8_STATE :return "HIBERN8" ; +case UIC_LINK_BROKEN_STATE :return "BROKEN" ;default :return "UNKNOWN" ; } }
drivers/scsi/ufs/ufshcd.c +269 -173
739 lines not on this page · raw message
drivers/scsi/ufs/ufshcd.h +15 -0
@@ -90,6 +90,7 @@ enum uic_link_state {UIC_LINK_OFF_STATE =0 ,/* Link powered down or disabled */ UIC_LINK_ACTIVE_STATE =1 ,/* Link is in Fast/Slow/Sleep state */ UIC_LINK_HIBERN8_STATE =2 ,/* Link is in Hibernate state */ + UIC_LINK_BROKEN_STATE = 3, /* Link is in broken state */ };#define ufshcd_is_link_off (hba) ((hba)->uic_link_state == UIC_LINK_OFF_STATE)
@@ -97,11 +98,15 @@ enum uic_link_state { UIC_LINK_ACTIVE_STATE)#define ufshcd_is_link_hibern8 (hba) ((hba)->uic_link_state == \ UIC_LINK_HIBERN8_STATE) +#define ufshcd_is_link_broken (hba) ((hba)->uic_link_state == \ + UIC_LINK_BROKEN_STATE)#define ufshcd_set_link_off (hba) ((hba)->uic_link_state = UIC_LINK_OFF_STATE)#define ufshcd_set_link_active (hba) ((hba)->uic_link_state = \ UIC_LINK_ACTIVE_STATE)#define ufshcd_set_link_hibern8 (hba) ((hba)->uic_link_state = \ UIC_LINK_HIBERN8_STATE) +#define ufshcd_set_link_broken (hba) ((hba)->uic_link_state = \ + UIC_LINK_BROKEN_STATE)#define ufshcd_set_ufs_dev_active (h) \ ((h)->curr_dev_pwr_mode = UFS_ACTIVE_PWR_MODE)
@@ -349,6 +354,7 @@ struct ufs_clk_gating {struct device_attribute delay_attr;struct device_attribute enable_attr;bool is_enabled; +bool held_by_pm;int active_reqs;struct workqueue_struct *clk_gating_workq; };
@@ -406,6 +412,8 @@ struct ufs_err_reg_hist {/** * struct ufs_stats - keeps usage/err statistics +* @last_intr_status: record the last interrupt status. +* @last_intr_ts: record the last interrupt timestamp. * @hibern8_exit_cnt: Counter to keep track of number of exits, * reset this after link-startup. * @last_hibern8_exit_tstamp: Set time after the hibern8 exit.
@@ -425,6 +433,9 @@ struct ufs_err_reg_hist {* @tsk_abort: tracks task abort events */ struct ufs_stats { +u32 last_intr_status; +ktime_t last_intr_ts; +u32 hibern8_exit_cnt;ktime_t last_hibern8_exit_tstamp;
@@ -608,12 +619,14 @@ struct ufs_hba_variant_params {* @intr_mask: Interrupt Mask Bits * @ee_ctrl_mask: Exception event control mask * @is_powered: flag to check if HBA is powered +* @eh_wq: Workqueue that eh_work works on * @eh_work: Worker to handle UFS errors that require s/w attention * @eeh_work: Worker to handle exception events * @errors: HBA errors * @uic_error: UFS interconnect layer error status * @saved_err: sticky error mask * @saved_uic_err: sticky UIC error mask +* @force_reset: flag to force eh_work perform a full reset * @silence_err_logs: flag to silence error logs * @dev_cmd: ufs device management command information * @last_dme_cmd_tstamp: time stamp of the last completed DME command
@@ -702,6 +715,7 @@ struct ufs_hba {bool is_powered;/* Work Queues */ +struct workqueue_struct *eh_wq;struct work_struct eh_work;struct work_struct eeh_work;
@@ -711,6 +725,7 @@ struct ufs_hba {u32 saved_err;u32 saved_uic_err;struct ufs_stats ufs_stats; +bool force_reset;bool silence_err_logs;/* Device management request data */
Signature · 1 line
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.
Re: [PATCH v2 4/4] scsi: ufs: Fix up and simplify error recovery mechanism
Quoted 2 lines · Can Guo
o Queue
eh_workon a single threaded workqueue to avoid concurrency betweeneh_works.
Please use another approach (mutex?) to serialize error handling. There are already way too workqueues in a running Linux system.
Quoted 36 lines
o According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when the link is broken. This actaully applies to any power mode change operations. In this change, if a power mode change operation (including AH8 enter/exit) fails, mark the link state as
UIC_LINK_BROKEN_STATEand scheduleeh_work.eh_workneeds to do full reset and restore to recover the link back to active. Before the link state is recovered to active byeh_work, any power mode change attempts just return-ENOLINKto avoid consecutive HW error.o To avoid concurrency between
eh_workand link recovery, remove link recovery from hibern8 enter/exit func. If hibern8 enter/exit func fails, simply return error code and leteh_workrun in parallel.o Recover UFS hba runtime PM error in
eh_work. If ufschd_suspend/resume fails due to UFS error, e.g. hibern8 enter/exit error and SSU cmd error, the runtime PM framework saves the error to dev.power.runtime_error. After that, hba runtime suspend/resume would not be invoked anymore until dev.power.runtime_error is cleared. The runtime PM error can be recovered ineh_workby callingpm_runtime_set_active()after reset and restore succeeds. Meanwhile, ifpm_runtime_set_active()returns no error, which means dev.power.runtime_error is cleared, we also need to explicitly resume those scsi devices under hba in case any of them has failed to be resumed due to hba runtime resume error.o Fix a racing problem between
eh_workand ufshcd_suspend/resume. In the old code, it blocks scsi requests before scheduleseh_work, but wheneh_workcallspm_runtime_get_sync(), if ufshcd_suspend/resume is sending a scsi cmd, most likely the SSU cmd,pm_runtime_get_sync()will never return because scsi requests were blocked. To fix this racing problem,
o Don't block scsi requests before scheduleeh_work, but leteh_workblock scsi requests wheneh_workis ready to start error recovery.
o Meanwhile, ifeh_workis schueduled due to fatal error, don't requeue the scsi cmds sent from ufshcd_suspend/resume path, but simply let the scsi cmds fail. If the scsi cmds fail, hba runtime suspend/resume fails too, but it does hurt sinceeh_workrecovers hba runtime PM error.
Quoted 3 lines
o Move host/regs dump in
ufshcd_check_errors()toeh_workbecause heavy dump in IRQ context can lead to stability issues. In addition, some clean up inufshcd_print_host_regs()andufshcd_print_host_state().
The above list is a long list. To me that is a sign that this patch needs to be split into multiple patches.
Thanks,
Bart.
Re: [PATCH v2 4/4] scsi: ufs: Fix up and simplify error recovery mechanism
Hi Bart,
Quoted 66 lines · Bart Van Assche
Quoted 2 lines · Can Guo
o Queue
eh_workon a single threaded workqueue to avoid concurrency betweeneh_works.Please use another approach (mutex?) to serialize error handling. There are already way too workqueues in a running Linux system.
Quoted 54 lines
o According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when the link is broken. This actaully applies to any power mode change operations. In this change, if a power mode change operation
(including
AH8 enter/exit) fails, mark the link state asUIC_LINK_BROKEN_STATE
and
scheduleeh_work.eh_workneeds to do full reset and restore to
recover
the link back to active. Before the link state is recovered to
active by
eh_work, any power mode change attempts just return-ENOLINKto
avoid
consecutive HW error.o To avoid concurrency between
eh_workand link recovery, remove link recovery from hibern8 enter/exit func. If hibern8 enter/exit func
fails,
simply return error code and leteh_workrun in parallel.o Recover UFS hba runtime PM error in
eh_work. If ufschd_suspend/resume fails due to UFS error, e.g. hibern8 enter/exit error and SSU cmd
error,
the runtime PM framework saves the error to dev.power.runtime_error. After that, hba runtime suspend/resume would not be invoked anymore
until
dev.power.runtime_error is cleared. The runtime PM error can be
recovered
ineh_workby callingpm_runtime_set_active()after reset and
restore
succeeds. Meanwhile, ifpm_runtime_set_active()returns no error,
which
means dev.power.runtime_error is cleared, we also need to explicitly resume those scsi devices under hba in case any of them has failed
to be
resumed due to hba runtime resume error.o Fix a racing problem between
eh_workand ufshcd_suspend/resume. In the old code, it blocks scsi requests before scheduleseh_work, but wheneh_workcallspm_runtime_get_sync(), if ufshcd_suspend/resume is
sending
a scsi cmd, most likely the SSU cmd,pm_runtime_get_sync()will
never
return because scsi requests were blocked. To fix this racing
problem,
o Don't block scsi requests before scheduleeh_work, but leteh_workblock scsi requests wheneh_workis ready to start error recovery.
o Meanwhile, ifeh_workis schueduled due to fatal error, don't
requeue
the scsi cmds sent from ufshcd_suspend/resume path, but simply let
the
scsi cmds fail. If the scsi cmds fail, hba runtime suspend/resume
fails
too, but it does hurt sinceeh_workrecovers hba runtime PM error.Quoted 4 lines
o Move host/regs dump in
ufshcd_check_errors()toeh_workbecause heavy dump in IRQ context can lead to stability issues. In addition, some
clean
up inufshcd_print_host_regs()andufshcd_print_host_state().
Quoted 6 lines
The above list is a long list. To me that is a sign that this patch needs to be split into multiple patches.
Thanks,
Bart.
Sure, will split it into a few patches.
Thanks,
Can Guo.
Re: [PATCH v2 4/4] scsi: ufs: Fix up and simplify error recovery mechanism
Hi Bart,
Quoted 10 lines · Can Guo
Hi Bart,
Quoted 7 lines · Bart Van Assche
Quoted 2 lines · Can Guo
o Queue
eh_workon a single threaded workqueue to avoid concurrency betweeneh_works.Please use another approach (mutex?) to serialize error handling. There are already way too workqueues in a running Linux system.
Yeah, mutex works, but in this change, we need to flush the eh_work. As per test, in real cases, flush_work can trigger warnings if the work is queued on system_wq. Please check func check_flush_dependency().
Quoted 70 lines
Quoted 70 lines
Quoted 60 lines
o According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when the link is broken. This actaully applies to any power mode change operations. In this change, if a power mode change operation
(including
AH8 enter/exit) fails, mark the link state asUIC_LINK_BROKEN_STATE
and
scheduleeh_work.eh_workneeds to do full reset and restore to
recover
the link back to active. Before the link state is recovered to
active by
eh_work, any power mode change attempts just return-ENOLINKto
avoid
consecutive HW error.o To avoid concurrency between
eh_workand link recovery, remove link recovery from hibern8 enter/exit func. If hibern8 enter/exit func
fails,
simply return error code and leteh_workrun in parallel.o Recover UFS hba runtime PM error in
eh_work. If ufschd_suspend/resume fails due to UFS error, e.g. hibern8 enter/exit error and SSU cmd
error,
the runtime PM framework saves the error to
dev.power.runtime_error.
After that, hba runtime suspend/resume would not be invoked anymore
until
dev.power.runtime_error is cleared. The runtime PM error can be
recovered
ineh_workby callingpm_runtime_set_active()after reset and
restore
succeeds. Meanwhile, ifpm_runtime_set_active()returns no error,
which
means dev.power.runtime_error is cleared, we also need to
explicitly
resume those scsi devices under hba in case any of them has failed
to be
resumed due to hba runtime resume error.o Fix a racing problem between
eh_workand ufshcd_suspend/resume. In the old code, it blocks scsi requests before scheduleseh_work, but
when
eh_workcallspm_runtime_get_sync(), if ufshcd_suspend/resume is
sending
a scsi cmd, most likely the SSU cmd,pm_runtime_get_sync()will
never
return because scsi requests were blocked. To fix this racing
problem,
o Don't block scsi requests before scheduleeh_work, but leteh_work
block scsi requests wheneh_workis ready to start error
recovery.
o Meanwhile, ifeh_workis schueduled due to fatal error, don't
requeue
the scsi cmds sent from ufshcd_suspend/resume path, but simply
let the
scsi cmds fail. If the scsi cmds fail, hba runtime suspend/resume
fails
too, but it does hurt sinceeh_workrecovers hba runtime PM
error.Quoted 4 lines
o Move host/regs dump in
ufshcd_check_errors()toeh_workbecause heavy dump in IRQ context can lead to stability issues. In addition, some
clean
up inufshcd_print_host_regs()andufshcd_print_host_state().The above list is a long list. To me that is a sign that this patch needs to be split into multiple patches.
Thanks,
Quoted 7 lines
Quoted 1 line
Bart.
Sure, will split it into a few patches.
Thanks,
Can Guo.
I tried, but I find it hard to split it as it works as a whole, it is a refactor change rather than a mixture of multiple fixes. I will try to refine the commit msg in next version. So it goes just as it is now.
Thanks,
Can Guo.