scsi: ufs: core: Add fault injection for task abort failures
Subsystems (2)
PATCH 1/1 scsi: ufs: core: Add fault injection for task abort failures
Commit 045da3077bc5 ("scsi: ufs: core: Make fault injection dynamically configurable per HBA") added fault injection support for UFS error handling and completion timeouts. However, there was no fault injection capability to simulate task abort failures.
Both legacy single-doorbell mode (in ufshcd_abort()) and MCQ mode (in ufshcd_mcq_abort()) invoke ufshcd_try_to_abort_task() to query device status and execute task aborts.
Add fault injection support for task abort:
- Add "abort" module parameter and "abort_inject" debugfs attribute.
- Hook ufs_fail_abort() at the entry of ufshcd_try_to_abort_task() to short-circuit task abort with -EIO. This universally simulates task abort failure across both single-doorbell and MCQ modes without incurring query polling delays.
--- drivers/ufs/core/ufs-fault-injection.c | 15 +++++++++++++++ drivers/ufs/core/ufs-fault-injection.h | 6 ++++++ drivers/ufs/core/ufshcd.c | 2 ++ include/ufs/ufshcd.h | 1 + 4 files changed, 24 insertions(+)
drivers/ufs/core/ufs-fault-injection.c +15 -0
@@ -34,6 +34,12 @@ MODULE_PARM_DESC(timeout, "Fault injection. timeout=<interval>,<probability>,<space>,<times>");static DECLARE_FAULT_ATTR (ufs_timeout_attr); +static char g_abort_str[FAULT_INJ_STR_SIZE ]; +module_param_cb (abort, &ufs_fault_ops, g_abort_str,0644 ); +MODULE_PARM_DESC (abort, +"Fault injection. abort=<interval>,<probability>,<space>,<times>" ); +static DECLARE_FAULT_ATTR (ufs_abort_attr); +static int ufs_fault_get (char *buffer,const struct kernel_param *kp) {const char *fault_str = kp->arg;
@@ -49,6 +55,8 @@ static int ufs_fault_set(const char *val, const struct kernel_param *kp) attr = &ufs_trigger_eh_attr; else if (kp->arg == g_timeout_str) attr = &ufs_timeout_attr; + else if (kp->arg == g_abort_str) + attr = &ufs_abort_attr;if (WARN_ON_ONCE (!attr))return -EINVAL ;
@@ -65,9 +73,11 @@ void ufs_fault_inject_hba_init(struct ufs_hba *hba) { hba->trigger_eh_attr = ufs_trigger_eh_attr; hba->timeout_attr = ufs_timeout_attr; + hba->abort_attr = ufs_abort_attr;#ifdef CONFIG_FAULT_INJECTION_DEBUG_FS fault_create_debugfs_attr ("trigger_eh_inject" , hba->debugfs_root, &hba->trigger_eh_attr);fault_create_debugfs_attr ("timeout_inject" , hba->debugfs_root, &hba->timeout_attr); +fault_create_debugfs_attr ("abort_inject" , hba->debugfs_root, &hba->abort_attr);#endif }
@@ -80,3 +90,8 @@ bool ufs_fail_completion(struct ufs_hba *hba) {return should_fail (&hba->timeout_attr,1 ); } + +bool ufs_fail_abort (struct ufs_hba *hba) +{ +return should_fail (&hba->abort_attr,1 ); +}
drivers/ufs/core/ufs-fault-injection.h +6 -0
@@ -12,6 +12,7 @@ struct ufs_hba;void ufs_fault_inject_hba_init (struct ufs_hba *hba);bool ufs_trigger_eh (struct ufs_hba *hba);bool ufs_fail_completion (struct ufs_hba *hba); +bool ufs_fail_abort (struct ufs_hba *hba);#else static inline void ufs_fault_inject_hba_init (struct ufs_hba *hba) {
@@ -26,6 +27,11 @@ static inline bool ufs_fail_completion(struct ufs_hba *hba) {return false; } + +static inline bool ufs_fail_abort (struct ufs_hba *hba) +{ +return false; +}#endif #endif /* _UFS_FAULT_INJECTION_H */
drivers/ufs/core/ufshcd.c +2 -0
@@ -7990,6 +7990,8 @@ int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag)return -EINVAL ; lrbp =scsi_cmd_priv (cmd); +if (ufs_fail_abort (hba)) +return -EIO ;for (poll_cnt =100 ; poll_cnt; poll_cnt--) { err = ufshcd_issue_tm_cmd(hba, lrbp->lun, tag, UFS_QUERY_TASK,
include/ufs/ufshcd.h +1 -0
@@ -1245,6 +1245,7 @@ struct ufs_hba {#ifdef CONFIG_SCSI_UFS_FAULT_INJECTION struct fault_attr trigger_eh_attr;struct fault_attr timeout_attr; +struct fault_attr abort_attr;#endif u32 luns_avail;unsigned int nr_hw_queues;
Signature · 1 line
2.55.0.970.g62bdec98f9-goog
Quoted 5 lines · Peter Wang (王信友)
The task-abort-fail injection bypasses
ufshcd_issue_tm_cmd. However,ufshcd_issue_tm_cmdcan return different results that involve the device and the underlying hardware. Because each result may require a different handling flow, a pure injection cannot cover all of these error paths?
Agreed. Short-circuiting ufshcd_try_to_abort_task() with -EIO bypasses ufshcd_issue_tm_cmd() entirely and does not exercise the distinct TMR response codes or hardware timeout paths.
When re-checking the call graph on an unmodified upstream kernel, there is an even more fundamental problem with hooking ufshcd_try_to_abort_task():
- ufs_fail_completion() is only called in ufshcd_transfer_req_compl() (the legacy single-doorbell completion path) and never in the MCQ completion path (ufshcd_mcq_poll_cqe_lock()), so timeout_inject cannot trigger ufshcd_mcq_abort() under MCQ.
- Under single-doorbell mode, the controller has already cleared the doorbell bit by the time the block-layer timeout fires, so ufshcd_abort() hits !(reg & (1 << tag)), completes the command, and returns before ever reaching ufshcd_try_to_abort_task().
Thanks,
Stanley