ufs: core: mcq: Add ufshcd_abort() and error handler support in MCQ mode
Subsystems (4)
v5 cover ufs: core: mcq: Add ufshcd_abort() and error handler support in MCQ mode
This patch series enable support for ufshcd_abort() and error handler in MCQ mode.
Bao D. Nguyen (7):
ufs: core: Combine 32-bit command_desc_base_addr_lo/hi
ufs: core: Update the ufshcd_clear_cmds() functionality
ufs: mcq: Add supporting functions for mcq abort
ufs: mcq: Add support for clean up mcq resources
ufs: mcq: Added ufshcd_mcq_abort()
ufs: mcq: Use ufshcd_mcq_poll_cqe_lock() in mcq mode
ufs: core: Add error handling for MCQ mode
drivers/ufs/core/ufs-mcq.c | 233 ++++++++++++++++++++++++++++++++++++++++- drivers/ufs/core/ufshcd-priv.h | 17 ++- drivers/ufs/core/ufshcd.c | 216 ++++++++++++++++++++++++++++++-------- drivers/ufs/host/ufs-qcom.c | 2 +- include/ufs/ufshcd.h | 5 +- include/ufs/ufshci.h | 23 +++- 6 files changed, 440 insertions(+), 56 deletions(-)
---v4->v5:
patch #4: fixed uninitialized variable access introduced in patch v3.
---v3->v4: Mainly addressed Bart's comments
patch #1: updated the commit message
patch #2: renamed ufshcd_clear_cmds() into ufshcd_clear_cmd() patch #3: removed result arg in ufshcd_mcq_sq_cleanup()
patch #4: removed check for "!rq" in ufshcd_cmd_inflight()
avoided access to door bell register in mcq mode
patch #5, 6: unchanged
patch #7: ufshcd_clear_cmds() to ufshcd_clear_cmd()
---v2->v3:
patch #1:
New patch per Bart's comment. Helps process utp cmd
descriptor addr easier.
patch #2:
New patch to address Bart's comment regarding potentialoverflow when mcq queue depth becomes greater than 64.
patch #3:
Address Bart's comments
. Replaced ufshcd_mcq_poll_register() with read_poll_timeout() . Replace spin_lock(sq_lock) with mutex(sq_mutex)
. Updated ufshcd_mcq_nullify_cmd() and renamed to ufshcd_mcq_nullify_sqe() . Minor cosmetic changes
patch #4:
Adress Bart's comments. Added new function ufshcd_cmd_inflight() to replace the usage of lrbp->cmd
patch #5:
Continue replacing lrbp->cmd with ufshcd_cmd_inflight()
patch #6:
No change
patch #7:
Address Stanley Chu's comment about clearing hba->dev_cmd.complete in clear ufshcd_wait_for_dev_cmd()
Address Bart's comment.
---v1->v2:
patch #1: Addressed Powen's comment. Replaced read_poll_timeout() with ufshcd_mcq_poll_register(). The function read_poll_timeout() may sleep while the caller is holding a spin_lock(). Poll the registers in a tight loop instead.
---
Signature · 1 line
2.7.4
PATCH v5 1/7 ufs: core: Combine 32-bit command_desc_base_addr_lo/hi
The UTP command descriptor base address is a 57-bit field in the UTP transfer request descriptor. Combine the two 32-bit command_desc_base_addr_lo/hi fields into a 64-bit for better handling of this field.
--- drivers/ufs/core/ufshcd.c | 6 ++---- include/ufs/ufshci.h | 6 ++---- 2 files changed, 4 insertions(+), 8 deletions(-)
drivers/ufs/core/ufshcd.c +2 -4
@@ -3875,10 +3875,8 @@ static void ufshcd_host_memory_configure(struct ufs_hba *hba)/* Configure UTRD with command descriptor base address */ cmd_desc_element_addr = (cmd_desc_dma_addr + (cmd_desc_size * i)); - utrdlp[i].command_desc_base_addr_lo = -cpu_to_le32 (lower_32_bits (cmd_desc_element_addr)); - utrdlp[i].command_desc_base_addr_hi = -cpu_to_le32 (upper_32_bits (cmd_desc_element_addr)); + utrdlp[i].command_desc_base_addr = +cpu_to_le64 (cmd_desc_element_addr);/* Response upiu and prdt offset should be in double words */ if (hba->quirks &UFSHCD_QUIRK_PRDT_BYTE_GRAN ) {
include/ufs/ufshci.h +2 -4
@@ -503,8 +503,7 @@ struct request_desc_header {/** * struct utp_transfer_req_desc - UTP Transfer Request Descriptor (UTRD) * @header: UTRD header DW-0 to DW-3 -* @command_desc_base_addr_lo: UCD base address low DW-4 -* @command_desc_base_addr_hi: UCD base address high DW-5 +* @command_desc_base_addr: UCD base address DW 4-5 * @response_upiu_length: response UPIU length DW-6 * @response_upiu_offset: response UPIU offset DW-6 * @prd_table_length: Physical region descriptor length DW-7
@@ -516,8 +515,7 @@ struct utp_transfer_req_desc {struct request_desc_header header;/* DW 4-5*/ -__le32 command_desc_base_addr_lo; -__le32 command_desc_base_addr_hi; +__le64 command_desc_base_addr;/* DW 6 */ __le16 response_upiu_length;
Signature · 1 line
2.7.4
Re: [PATCH v5 1/7] ufs: core: Combine 32-bit command_desc_base_addr_lo/hi
Quoted 4 lines · Bao D. Nguyen
The UTP command descriptor base address is a 57-bit field in the UTP transfer request descriptor. Combine the two 32-bit command_desc_base_addr_lo/hi fields into a 64-bit for better handling of this field.
PATCH v5 2/7 ufs: core: Update the ufshcd_clear_cmds() functionality
In reply to Bao D. NguyenIn the ufshcd_clear_cmds(), the 2nd pamameter would be the bit mask of the command to be cleared in the transfer request door bell register. This bit mask mechanism does not scale well in mcq mode when the queue depth becomes much greater than 64. Change the 2nd parameter to the function to be the task_tag number of the corresponding bit to be cleared in the door bell register. By doing so, mcq mode with a large queue depth can reuse this function.
Since the behavior of this function is changed from handling multiple commands into a single command, rename ufshcd_clear_cmds() into ufshcd_clear_cmd().
--- drivers/ufs/core/ufshcd.c | 37 +++++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 16 deletions(-)
drivers/ufs/core/ufshcd.c +21 -16
@@ -3006,13 +3006,15 @@ static int ufshcd_compose_dev_cmd(struct ufs_hba *hba, }/* -* Clear all the requests from the controller for which a bit has been set in -* @mask and wait until the controller confirms that these requests have been -* cleared. +* Clear the pending command in the controller and wait until +* the controller confirms that the command has been cleared. +* @hba: per adapter instance +* @task_tag: The tag number of the command to be cleared. */ -static int ufshcd_clear_cmds (struct ufs_hba *hba,u32 mask) +static int ufshcd_clear_cmd (struct ufs_hba *hba,u32 task_tag) {unsigned long flags; +u32 mask =1U << task_tag;/* clear outstanding transaction before retry */ spin_lock_irqsave (hba->host->host_lock, flags);
@@ -3113,7 +3115,7 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT ;dev_dbg (hba->dev,"%s: dev_cmd request timedout, tag %d\n" , __func__, lrbp->task_tag); -if (ufshcd_clear_cmds (hba,1U << lrbp->task_tag) ==0 ) { +if (ufshcd_clear_cmd (hba, lrbp->task_tag) ==0 ) {/* successfully cleared the command, retry if needed */ err = -EAGAIN ;/*
@@ -7286,7 +7288,7 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd)unsigned long flags, pending_reqs =0 , not_cleared =0 ;struct Scsi_Host *host;struct ufs_hba *hba; -u32 pos; +u32 pos, not_cleared_mask =0 ;int err;u8 resp =0xF , lun;
@@ -7309,17 +7311,20 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) hba->outstanding_reqs &= ~pending_reqs;spin_unlock_irqrestore (&hba->outstanding_lock, flags); -if (ufshcd_clear_cmds (hba, pending_reqs) <0 ) { -spin_lock_irqsave (&hba->outstanding_lock, flags); - not_cleared = pending_reqs & -ufshcd_readl (hba,REG_UTP_TRANSFER_REQ_DOOR_BELL ); - hba->outstanding_reqs |= not_cleared; -spin_unlock_irqrestore (&hba->outstanding_lock, flags); +for_each_set_bit (pos, &pending_reqs, hba->nutrs) { +if (ufshcd_clear_cmd (hba, pos) <0 ) { +spin_lock_irqsave (&hba->outstanding_lock, flags); + not_cleared =1U << pos & +ufshcd_readl (hba,REG_UTP_TRANSFER_REQ_DOOR_BELL ); + hba->outstanding_reqs |= not_cleared; + not_cleared_mask |= not_cleared; +spin_unlock_irqrestore (&hba->outstanding_lock, flags); -dev_err (hba->dev,"%s: failed to clear requests %#lx\n" , - __func__, not_cleared); +dev_err (hba->dev,"%s: failed to clear request %d\n" , + __func__, pos); + } } -__ufshcd_transfer_req_compl (hba, pending_reqs & ~not_cleared); +__ufshcd_transfer_req_compl (hba, pending_reqs & ~not_cleared_mask); out: hba->req_abort_count =0 ;
@@ -7416,7 +7421,7 @@ static int ufshcd_try_to_abort_task(struct ufs_hba *hba, int tag) goto out; } - err =ufshcd_clear_cmds (hba,1U << tag); + err =ufshcd_clear_cmd (hba, tag);if (err)dev_err (hba->dev,"%s: Failed clearing cmd at tag %d, err %d\n" , __func__, tag, err);
Signature · 1 line
2.7.4
Re: [PATCH v5 2/7] ufs: core: Update the ufshcd_clear_cmds() functionality
Quoted 11 lines · Bao D. Nguyen
In the
ufshcd_clear_cmds(), the 2nd pamameter would be the bit mask of the command to be cleared in the transfer request door bell register. This bit mask mechanism does not scale well in mcq mode when the queue depth becomes much greater than 64. Change the 2nd parameter to the function to be thetask_tagnumber of the corresponding bit to be cleared in the door bell register. By doing so, mcq mode with a large queue depth can reuse this function.Since the behavior of this function is changed from handling multiple commands into a single command, rename
ufshcd_clear_cmds()intoufshcd_clear_cmd().
PATCH v5 3/7 ufs: mcq: Add supporting functions for mcq abort
In reply to Bao D. NguyenAdd supporting functions to handle ufs abort in mcq mode.
--- drivers/ufs/core/ufs-mcq.c | 167 +++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 10 +++ drivers/ufs/core/ufshcd.c | 1 - include/ufs/ufshcd.h | 3 + include/ufs/ufshci.h | 17 +++++ 5 files changed, 197 insertions(+), 1 deletion(-)
drivers/ufs/core/ufs-mcq.c +167 -0
@@ -12,6 +12,10 @@#include <linux/module.h> #include <linux/platform_device.h> #include "ufshcd-priv.h" +#include <linux/delay.h> +#include <scsi/scsi_cmnd.h> +#include <linux/bitfield.h> +#include <linux/iopoll.h> #define MAX_QUEUE_SUP GENMASK(7, 0)#define UFS_MCQ_MIN_RW_QUEUES 2
@@ -27,6 +31,9 @@#define MCQ_ENTRY_SIZE_IN_DWORD 8#define CQE_UCD_BA GENMASK_ULL(63, 7) +/* Max mcq register polling time in microseconds */ +#define MCQ_POLL_US 500000 +static int rw_queue_count_set (const char *val,const struct kernel_param *kp) { return param_set_uint_minmax(val, kp, UFS_MCQ_MIN_RW_QUEUES,
@@ -419,6 +426,7 @@ int ufshcd_mcq_init(struct ufs_hba *hba) hwq->max_entries = hba->nutrs;spin_lock_init (&hwq->sq_lock);spin_lock_init (&hwq->cq_lock); +mutex_init (&hwq->sq_mutex); }/* The very first HW queue serves device commands */
@@ -429,3 +437,162 @@ int ufshcd_mcq_init(struct ufs_hba *hba) host->host_tagset =1 ;return 0 ; } + +static int ufshcd_mcq_sq_stop (struct ufs_hba *hba,struct ufs_hw_queue *hwq) +{ +void __iomem *reg; +u32 id = hwq->id, val; +int err; + +writel (SQ_STOP ,mcq_opr_base (hba,OPR_SQD , id) +REG_SQRTC ); + reg =mcq_opr_base (hba,OPR_SQD , id) +REG_SQRTS ; + err =read_poll_timeout (readl, val, val &SQ_STS ,20 , +MCQ_POLL_US , false, reg); +if (err) +dev_err (hba->dev,"%s: failed. hwq-id=%d, err=%d\n" , + __func__, id, err); +return err; +} + +static int ufshcd_mcq_sq_start (struct ufs_hba *hba,struct ufs_hw_queue *hwq) +{ +void __iomem *reg; +u32 id = hwq->id, val; +int err; + +writel (SQ_START ,mcq_opr_base (hba,OPR_SQD , id) +REG_SQRTC ); + reg =mcq_opr_base (hba,OPR_SQD , id) +REG_SQRTS ; + err =read_poll_timeout (readl, val, !(val &SQ_STS ),20 , +MCQ_POLL_US , false, reg); +if (err) +dev_err (hba->dev,"%s: failed. hwq-id=%d, err=%d\n" , + __func__, id, err); +return err; +} + +/** +* ufshcd_mcq_sq_cleanup - Clean up submission queue resources +* associated with the pending command. +* @hba - per adapter instance. +* @task_tag - The command's task tag. +* +* Returns 0 for success; error code otherwise. +*/ +int ufshcd_mcq_sq_cleanup (struct ufs_hba *hba,int task_tag) +{ +struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; +struct scsi_cmnd *cmd = lrbp->cmd; +struct ufs_hw_queue *hwq; +void __iomem *reg, *opr_sqd_base; +u32 nexus, id, val; +int err; + +if (task_tag != hba->nutrs -UFSHCD_NUM_RESERVED ) { +if (!cmd) +return -EINVAL ; + hwq =ufshcd_mcq_req_to_hwq (hba,scsi_cmd_to_rq (cmd)); + }else { + hwq = hba->dev_cmd_queue; + } + + id = hwq->id; + +mutex_lock (&hwq->sq_mutex); + +/* stop the SQ fetching before working on it */ + err =ufshcd_mcq_sq_stop (hba, hwq); +if (err) + goto unlock; + +/* SQCTI = EXT_IID, IID, LUN, Task Tag */ + nexus = lrbp->lun <<8 | task_tag; + opr_sqd_base =mcq_opr_base (hba,OPR_SQD , id); +writel (nexus, opr_sqd_base +REG_SQCTI ); + +/* SQRTCy.ICU = 1 */ +writel (SQ_ICU , opr_sqd_base +REG_SQRTC ); + +/* Poll SQRTSy.CUS = 1. Return result from SQRTSy.RTC */ + reg = opr_sqd_base +REG_SQRTS ; + err =read_poll_timeout (readl, val, val &SQ_CUS ,20 , +MCQ_POLL_US , false, reg); +if (err) +dev_err (hba->dev,"%s: failed. hwq=%d, tag=%d err=%ld\n" , + __func__, id, task_tag, +FIELD_GET (SQ_ICU_ERR_CODE_MASK ,readl (reg))); + +if (ufshcd_mcq_sq_start (hba, hwq)) + err = -ETIMEDOUT ; + +unlock: +mutex_unlock (&hwq->sq_mutex); +return err; +} + +/** +* ufshcd_mcq_nullify_sqe - Nullify the submission queue entry. +* Write the sqe's Command Type to 0xF. The host controller will not +* fetch any sqe with Command Type = 0xF. +* +* @utrd - UTP Transfer Request Descriptor to be nullified. +*/ +static void ufshcd_mcq_nullify_sqe (struct utp_transfer_req_desc *utrd) +{ +u32 dword_0; + + dword_0 =le32_to_cpu (utrd->header.dword_0); + dword_0 &= ~UPIU_COMMAND_TYPE_MASK ; + dword_0 |=FIELD_PREP (UPIU_COMMAND_TYPE_MASK ,0xF ); + utrd->header.dword_0 =cpu_to_le32 (dword_0); +} + +/** +* ufshcd_mcq_sqe_search - Search for the command in the submission queue +* If the command is in the submission queue and not issued to the device yet, +* nullify the sqe so the host controller will skip fetching the sqe. +* +* @hba - per adapter instance. +* @hwq - Hardware Queue to be searched. +* @task_tag - The command's task tag. +* +* Returns true if the SQE containing the command is present in the SQ +* (not fetched by the controller); returns false if the SQE is not in the SQ. +*/ +static bool ufshcd_mcq_sqe_search (struct ufs_hba *hba, +struct ufs_hw_queue *hwq,int task_tag) +{ +struct ufshcd_lrb *lrbp = &hba->lrb[task_tag]; +struct utp_transfer_req_desc *utrd; +u32 mask = hwq->max_entries -1 ; +__le64 cmd_desc_base_addr; +bool ret = false; +u64 addr, match; +u32 sq_head_slot; + +mutex_lock (&hwq->sq_mutex); + +ufshcd_mcq_sq_stop (hba, hwq); + sq_head_slot =ufshcd_mcq_get_sq_head_slot (hwq); +if (sq_head_slot == hwq->sq_tail_slot) + goto out; + + cmd_desc_base_addr = lrbp->utr_descriptor_ptr->command_desc_base_addr; + addr =le64_to_cpu (cmd_desc_base_addr) &CQE_UCD_BA ; + +while (sq_head_slot != hwq->sq_tail_slot) { + utrd = hwq->sqe_base_addr + + sq_head_slot *sizeof (struct utp_transfer_req_desc ); + match =le64_to_cpu (utrd->command_desc_base_addr) &CQE_UCD_BA ; +if (addr == match) { +ufshcd_mcq_nullify_sqe (utrd); + ret = true; + goto out; + } + sq_head_slot = (sq_head_slot +1 ) & mask; + } + +out: +ufshcd_mcq_sq_start (hba, hwq); +mutex_unlock (&hwq->sq_mutex); +return ret; +}
drivers/ufs/core/ufshcd-priv.h +10 -0
@@ -78,6 +78,8 @@ struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba,unsigned long ufshcd_mcq_poll_cqe_lock (struct ufs_hba *hba,struct ufs_hw_queue *hwq); +int ufshcd_mcq_sq_cleanup (struct ufs_hba *hba,int task_tag); +#define UFSHCD_MCQ_IO_QUEUE_OFFSET 1#define SD_ASCII_STD true#define SD_RAW false
@@ -404,4 +406,12 @@ static inline struct cq_entry *ufshcd_mcq_cur_cqe(struct ufs_hw_queue *q) return cqe + q->cq_head_slot; } + +static inline u32 ufshcd_mcq_get_sq_head_slot (struct ufs_hw_queue *q) +{ +u32 val =readl (q->mcq_sq_head); + +return val /sizeof (struct utp_transfer_req_desc ); +} +#endif /* _UFSHCD_PRIV_H_ */
drivers/ufs/core/ufshcd.c +0 -1
@@ -173,7 +173,6 @@ EXPORT_SYMBOL_GPL(ufshcd_dump_regs);enum {UFSHCD_MAX_CHANNEL =0 ,UFSHCD_MAX_ID =1 , -UFSHCD_NUM_RESERVED =1 ,UFSHCD_CMD_PER_LUN =32 -UFSHCD_NUM_RESERVED ,UFSHCD_CAN_QUEUE =32 -UFSHCD_NUM_RESERVED , };
include/ufs/ufshcd.h +3 -0
@@ -1088,6 +1088,7 @@ struct ufs_hba {* @cq_tail_slot: current slot to which CQ tail pointer is pointing * @cq_head_slot: current slot to which CQ head pointer is pointing * @cq_lock: Synchronize between multiple polling instances +* @sq_mutex: prevent submission queue concurrent access */ struct ufs_hw_queue {void __iomem *mcq_sq_head;
@@ -1106,6 +1107,8 @@ struct ufs_hw_queue {u32 cq_tail_slot;u32 cq_head_slot;spinlock_t cq_lock; +/* prevent concurrent access to submission queue */ +struct mutex sq_mutex; };static inline bool is_mcq_enabled (struct ufs_hba *hba)
include/ufs/ufshci.h +17 -0
@@ -99,6 +99,9 @@ enum {enum {REG_SQHP =0x0 ,REG_SQTP =0x4 , +REG_SQRTC =0x8 , +REG_SQCTI =0xC , +REG_SQRTS =0x10 , };enum {
@@ -111,12 +114,26 @@ enum { REG_CQIE = 0x4, }; +enum { +SQ_START =0x0 , +SQ_STOP =0x1 , +SQ_ICU =0x2 , +}; + +enum { +SQ_STS =0x1 , +SQ_CUS =0x2 , +}; + +#define SQ_ICU_ERR_CODE_MASK GENMASK(7, 4) +#define UPIU_COMMAND_TYPE_MASK GENMASK(31, 28)#define UFS_MASK (mask, offset) ((mask) << (offset))/* UFS Version 08h */ #define MINOR_VERSION_NUM_MASK UFS_MASK(0xFFFF, 0)#define MAJOR_VERSION_NUM_MASK UFS_MASK(0xFFFF, 16) +#define UFSHCD_NUM_RESERVED 1/* * Controller UFSHCI version * - 2.x and newer use the following scheme:
Signature · 1 line
2.7.4
Re: [PATCH v5 3/7] ufs: mcq: Add supporting functions for mcq abort
Quoted 1 line · Bao D. Nguyen
Add supporting functions to handle ufs abort in mcq mode.
PATCH v5 4/7 ufs: mcq: Add support for clean up mcq resources
In reply to Bao D. NguyenUpdate ufshcd_clear_cmd() to clean up the mcq resources similar to the function ufshcd_utrl_clear() does for sdb mode.
Update ufshcd_try_to_abort_task() to support mcq mode so that this function can be invoked in either mcq or sdb mode.
--- drivers/ufs/core/ufshcd-priv.h | 1 + drivers/ufs/core/ufshcd.c | 74 +++++++++++++++++++++++++++++++++++++----- 2 files changed, 66 insertions(+), 9 deletions(-)
drivers/ufs/core/ufshcd-priv.h +1 -0
@@ -78,6 +78,7 @@ struct ufs_hw_queue *ufshcd_mcq_req_to_hwq(struct ufs_hba *hba,unsigned long ufshcd_mcq_poll_cqe_lock (struct ufs_hba *hba,struct ufs_hw_queue *hwq); +bool ufshcd_cmd_inflight (struct scsi_cmnd *cmd);int ufshcd_mcq_sq_cleanup (struct ufs_hba *hba,int task_tag);#define UFSHCD_MCQ_IO_QUEUE_OFFSET 1
drivers/ufs/core/ufshcd.c +65 -9
111 lines not on this page · raw message
Signature · 1 line
2.7.4
Re: [PATCH v5 4/7] ufs: mcq: Add support for clean up mcq resources
Quoted 5 lines · Bao D. Nguyen
Update
ufshcd_clear_cmd()to clean up the mcq resources similar to the functionufshcd_utrl_clear()does for sdb mode.Update
ufshcd_try_to_abort_task()to support mcq mode so that this function can be invoked in either mcq or sdb mode.
PATCH v5 5/7 ufs: mcq: Added ufshcd_mcq_abort()
In reply to Bao D. NguyenAdd ufshcd_mcq_abort() to support ufs abort in mcq mode.
--- drivers/ufs/core/ufs-mcq.c | 60 ++++++++++++++++++++++++++++++++++++++++++ drivers/ufs/core/ufshcd-priv.h | 4 +++ drivers/ufs/core/ufshcd.c | 13 ++++++--- 3 files changed, 73 insertions(+), 4 deletions(-)
drivers/ufs/core/ufs-mcq.c +60 -0
@@ -596,3 +596,63 @@ static bool ufshcd_mcq_sqe_search(struct ufs_hba *hba,mutex_unlock (&hwq->sq_mutex);return ret; } + +/** +* ufshcd_mcq_abort - Abort the command in MCQ. +* @cmd - The command to be aborted. +* +* Returns SUCCESS or FAILED error codes +*/ +int ufshcd_mcq_abort (struct scsi_cmnd *cmd) +{ +struct Scsi_Host *host = cmd->device->host; +struct ufs_hba *hba =shost_priv (host); +int tag =scsi_cmd_to_rq (cmd)->tag; +struct ufshcd_lrb *lrbp = &hba->lrb[tag]; +struct ufs_hw_queue *hwq; +int err =FAILED ; + +if (!ufshcd_cmd_inflight (lrbp->cmd)) { +dev_err (hba->dev, +"%s: skip abort. cmd at tag %d already completed.\n" , + __func__, tag); + goto out; + } + +/* Skip task abort in case previous aborts failed and report failure */ +if (lrbp->req_abort_skip) { +dev_err (hba->dev,"%s: skip abort. tag %d failed earlier\n" , + __func__, tag); + goto out; + } + + hwq =ufshcd_mcq_req_to_hwq (hba,scsi_cmd_to_rq (cmd)); + +if (ufshcd_mcq_sqe_search (hba, hwq, tag)) { +/* +* Failure. The command should not be "stuck" in SQ for +* a long time which resulted in command being aborted. +*/ +dev_err (hba->dev,"%s: cmd found in sq. hwq=%d, tag=%d\n" , + __func__, hwq->id, tag); + goto out; + } + +/* +* The command is not in the submission queue, and it is not +* in the completion queue either. Query the device to see if +* the command is being processed in the device. +*/ +if (ufshcd_try_to_abort_task (hba, tag)) { +dev_err (hba->dev,"%s: device abort failed %d\n" , __func__, err); + lrbp->req_abort_skip = true; + goto out; + } + + err =SUCCESS ; +if (ufshcd_cmd_inflight (lrbp->cmd)) +ufshcd_release_scsi_cmd (hba, lrbp); + +out: +return err; +}
drivers/ufs/core/ufshcd-priv.h +4 -0
@@ -80,6 +80,10 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,bool ufshcd_cmd_inflight (struct scsi_cmnd *cmd);int ufshcd_mcq_sq_cleanup (struct ufs_hba *hba,int task_tag); +int ufshcd_mcq_abort (struct scsi_cmnd *cmd); +int ufshcd_try_to_abort_task (struct ufs_hba *hba,int tag); +void ufshcd_release_scsi_cmd (struct ufs_hba *hba, +struct ufshcd_lrb *lrbp);#define UFSHCD_MCQ_IO_QUEUE_OFFSET 1#define SD_ASCII_STD true
drivers/ufs/core/ufshcd.c +9 -4
@@ -300,7 +300,6 @@ static int ufshcd_setup_hba_vreg(struct ufs_hba *hba, bool on);static int ufshcd_setup_vreg (struct ufs_hba *hba,bool on);static inline int ufshcd_config_vreg_hpm (struct ufs_hba *hba,struct ufs_vreg *vreg); -static int ufshcd_try_to_abort_task (struct ufs_hba *hba,int tag);static void ufshcd_wb_toggle_buf_flush_during_h8 (struct ufs_hba *hba,bool enable);static void ufshcd_hba_vreg_set_lpm (struct ufs_hba *hba);
@@ -5457,8 +5456,8 @@ static irqreturn_t ufshcd_uic_cmd_compl(struct ufs_hba *hba, u32 intr_status) }/* Release the resources allocated for processing a SCSI command. */ -static void ufshcd_release_scsi_cmd (struct ufs_hba *hba, -struct ufshcd_lrb *lrbp) +void ufshcd_release_scsi_cmd (struct ufs_hba *hba, +struct ufshcd_lrb *lrbp) {struct scsi_cmnd *cmd = lrbp->cmd;
@@ -7396,7 +7395,7 @@ static void ufshcd_set_req_abort_skip(struct ufs_hba *hba, unsigned long bitmap)* * Returns zero on success, non-zero on failure */ -static int ufshcd_try_to_abort_task (struct ufs_hba *hba,int tag) +int ufshcd_try_to_abort_task (struct ufs_hba *hba,int tag) {struct ufshcd_lrb *lrbp = &hba->lrb[tag];int err =0 ;
@@ -7562,6 +7561,12 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)if (is_mcq_enabled (hba)) goto release; +if (is_mcq_enabled (hba)) { +/* MCQ mode. Branch off to handle abort for mcq mode */ + err =ufshcd_mcq_abort (cmd); + goto release; + } +/* Skip task abort in case previous aborts failed and report failure */ if (lrbp->req_abort_skip) {dev_err (hba->dev,"%s: skipping abort\n" , __func__);
Signature · 1 line
2.7.4
Re: [PATCH v5 5/7] ufs: mcq: Added ufshcd_mcq_abort()
Hi Bao D.
drivers/ufs/core/ufshcd.c static int ufshcd_abort(struct scsi_cmnd *cmd) · line 7562
Quoted 3 lines
drivers/ufs/core/ufshcd.c +0
-0@@ -7562,6 +7561,12 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)if (is_mcq_enabled (hba)) goto release;
Should we remove the above lines because the code below will not be executed in the MCQ path?
Quoted 13 lines
drivers/ufs/core/ufshcd.c +6
-1+if (is_mcq_enabled (hba)) { +/* MCQ mode. Branch off to handle abort for mcq mode */ + err =ufshcd_mcq_abort (cmd); + goto release; + } +/* Skip task abort in case previous aborts failed and report failure */ if (lrbp->req_abort_skip) {dev_err (hba->dev,"%s: skipping abort\n" , __func__); --2.7.4
Re: [PATCH v5 5/7] ufs: mcq: Added ufshcd_mcq_abort()
Quoted 8 lines · Stanley Chu
Hi Bao D.
drivers/ufs/core/ufshcd.c static int ufshcd_abort(struct scsi_cmnd *cmd) · line 7562
Quoted 3 lines
drivers/ufs/core/ufshcd.c +0
-0@@ -7562,6 +7561,12 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)if (is_mcq_enabled (hba)) goto release;Should we remove the above lines because the code below will not be executed in the MCQ path?
Thanks Stanley. It was my silly rebase mistake :-(. I will fix it.
Thanks,
Bao
Quoted 14 lines
Quoted 14 lines
drivers/ufs/core/ufshcd.c +6
-1+if (is_mcq_enabled (hba)) { +/* MCQ mode. Branch off to handle abort for mcq mode */ + err =ufshcd_mcq_abort (cmd); + goto release; + } +/* Skip task abort in case previous aborts failed and report failure */ if (lrbp->req_abort_skip) {dev_err (hba->dev,"%s: skipping abort\n" , __func__); --2.7.4
PATCH v5 6/7 ufs: mcq: Use ufshcd_mcq_poll_cqe_lock() in mcq mode
In reply to Bao D. NguyenIn preparation for adding mcq error handler support, update the mcq code to use the ufshcd_mcq_poll_cqe_lock() in interrupt context instead of using ufshcd_mcq_poll_cqe_nolock(). This is to keep synchronization between mcq interrupt and error handler contexts because both need to access the mcq hardware in separate contexts.
--- drivers/ufs/core/ufs-mcq.c | 6 +++--- drivers/ufs/core/ufshcd-priv.h | 2 -- drivers/ufs/core/ufshcd.c | 2 +- drivers/ufs/host/ufs-qcom.c | 2 +- include/ufs/ufshcd.h | 2 +- 5 files changed, 6 insertions(+), 8 deletions(-)
drivers/ufs/core/ufs-mcq.c +3 -3
@@ -284,8 +284,8 @@ static void ufshcd_mcq_process_cqe(struct ufs_hba *hba,ufshcd_compl_one_cqe (hba, tag, cqe); } -unsigned long ufshcd_mcq_poll_cqe_nolock (struct ufs_hba *hba, -struct ufs_hw_queue *hwq) +static unsigned long ufshcd_mcq_poll_cqe_nolock (struct ufs_hba *hba, +struct ufs_hw_queue *hwq) {unsigned long completed_reqs =0 ;
@@ -301,7 +301,6 @@ unsigned long ufshcd_mcq_poll_cqe_nolock(struct ufs_hba *hba,return completed_reqs; } -EXPORT_SYMBOL_GPL (ufshcd_mcq_poll_cqe_nolock);unsigned long ufshcd_mcq_poll_cqe_lock (struct ufs_hba *hba,struct ufs_hw_queue *hwq)
@@ -314,6 +313,7 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,return completed_reqs; } +EXPORT_SYMBOL_GPL (ufshcd_mcq_poll_cqe_lock);void ufshcd_mcq_make_queues_operational (struct ufs_hba *hba) {
drivers/ufs/core/ufshcd-priv.h +0 -2
@@ -71,8 +71,6 @@ void ufshcd_mcq_config_mac(struct ufs_hba *hba, u32 max_active_cmds);void ufshcd_mcq_select_mcq_mode (struct ufs_hba *hba);u32 ufshcd_mcq_read_cqis (struct ufs_hba *hba,int i);void ufshcd_mcq_write_cqis (struct ufs_hba *hba,u32 val,int i); -unsigned long ufshcd_mcq_poll_cqe_nolock (struct ufs_hba *hba, -struct ufs_hw_queue *hwq);struct ufs_hw_queue *ufshcd_mcq_req_to_hwq (struct ufs_hba *hba,struct request *req); unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba,
drivers/ufs/core/ufshcd.c +1 -1
@@ -6811,7 +6811,7 @@ static irqreturn_t ufshcd_handle_mcq_cq_events(struct ufs_hba *hba)ufshcd_mcq_write_cqis (hba, events, i);if (events &UFSHCD_MCQ_CQIS_TAIL_ENT_PUSH_STS ) -ufshcd_mcq_poll_cqe_nolock (hba, hwq); +ufshcd_mcq_poll_cqe_lock (hba, hwq); }return IRQ_HANDLED ;
drivers/ufs/host/ufs-qcom.c +1 -1
@@ -1556,7 +1556,7 @@ static irqreturn_t ufs_qcom_mcq_esi_handler(int irq, void *__hba)struct ufs_hw_queue *hwq = &hba->uhq[id];ufshcd_mcq_write_cqis (hba,0x1 , id); -ufshcd_mcq_poll_cqe_nolock (hba, hwq); +ufshcd_mcq_poll_cqe_lock (hba, hwq);return IRQ_HANDLED ; }
include/ufs/ufshcd.h +1 -1
@@ -1244,7 +1244,7 @@ void ufshcd_update_evt_hist(struct ufs_hba *hba, u32 id, u32 val);void ufshcd_hba_stop (struct ufs_hba *hba);void ufshcd_schedule_eh_work (struct ufs_hba *hba);void ufshcd_mcq_write_cqis (struct ufs_hba *hba,u32 val,int i); -unsigned long ufshcd_mcq_poll_cqe_nolock (struct ufs_hba *hba, +unsigned long ufshcd_mcq_poll_cqe_lock (struct ufs_hba *hba,struct ufs_hw_queue *hwq);void ufshcd_mcq_enable_esi (struct ufs_hba *hba);void ufshcd_mcq_config_esi (struct ufs_hba *hba,struct msi_msg *msg);
Signature · 1 line
2.7.4
Re: [PATCH v5 6/7] ufs: mcq: Use ufshcd_mcq_poll_cqe_lock() in mcq mode
Quoted 5 lines · Bao D. Nguyen
In preparation for adding mcq error handler support, update the mcq code to use the
ufshcd_mcq_poll_cqe_lock()in interrupt context instead of usingufshcd_mcq_poll_cqe_nolock(). This is to keep synchronization between mcq interrupt and error handler contexts because both need to access the mcq hardware in separate contexts.
PATCH v5 7/7 ufs: core: Add error handling for MCQ mode
In reply to Bao D. NguyenAdd support for error handling for MCQ mode.
--- drivers/ufs/core/ufshcd.c | 85 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 74 insertions(+), 11 deletions(-)
drivers/ufs/core/ufshcd.c +74 -11
@@ -3148,6 +3148,16 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT ;dev_dbg (hba->dev,"%s: dev_cmd request timedout, tag %d\n" , __func__, lrbp->task_tag); + +/* MCQ mode */ +if (is_mcq_enabled (hba)) { + err =ufshcd_clear_cmd (hba, lrbp->task_tag); +if (!err) + hba->dev_cmd.complete =NULL ; +return err; + } + +/* SDB mode */ if (ufshcd_clear_cmd (hba, lrbp->task_tag) ==0 ) {/* successfully cleared the command, retry if needed */ err = -EAGAIN ;
@@ -5581,6 +5591,10 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)*/ static irqreturn_t ufshcd_transfer_req_compl (struct ufs_hba *hba) { +struct ufshcd_lrb *lrbp; +u32 hwq_num, utag; +int tag; +/* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. * In order to prevent other interrupts starvation the DB is read once
@@ -5599,7 +5613,22 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba)* Ignore the ufshcd_poll() return value and return IRQ_HANDLED since we * do not want polling to trigger spurious interrupt complaints. */ -ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); +if (!is_mcq_enabled (hba)) { +ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); + goto out; + } + +/* MCQ mode */ +for (tag =0 ; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; +if (ufshcd_cmd_inflight (lrbp->cmd)) { + utag =blk_mq_unique_tag (scsi_cmd_to_rq (lrbp->cmd)); + hwq_num =blk_mq_unique_tag_to_hwq (utag); +ufshcd_poll (hba->host, hwq_num); + } + } + +out:return IRQ_HANDLED ; }
@@ -6378,18 +6407,36 @@ static bool ufshcd_abort_all(struct ufs_hba *hba)bool needs_reset = false;int tag, ret; -/* Clear pending transfer requests */ -for_each_set_bit (tag, &hba->outstanding_reqs, hba->nutrs) { - ret =ufshcd_try_to_abort_task (hba, tag); -dev_err (hba->dev,"Aborting tag %d / CDB %#02x %s\n" , tag, - hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0 ] :-1 , - ret ?"failed" :"succeeded" ); -if (ret) { - needs_reset = true; - goto out; +if (is_mcq_enabled (hba)) { +struct ufshcd_lrb *lrbp; +int tag; + +for (tag =0 ; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; +if (!ufshcd_cmd_inflight (lrbp->cmd)) +continue ; + ret =ufshcd_try_to_abort_task (hba, tag); +dev_err (hba->dev,"Aborting tag %d / CDB %#02x %s\n" , tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0 ] :-1 , + ret ?"failed" :"succeeded" ); +if (ret) { + needs_reset = true; + goto out; + } + } + }else { +/* Clear pending transfer requests */ +for_each_set_bit (tag, &hba->outstanding_reqs, hba->nutrs) { + ret =ufshcd_try_to_abort_task (hba, tag); +dev_err (hba->dev,"Aborting tag %d / CDB %#02x %s\n" , tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0 ] :-1 , + ret ?"failed" :"succeeded" ); +if (ret) { + needs_reset = true; + goto out; + } } } -/* Clear pending task management requests */ for_each_set_bit (tag, &hba->outstanding_tasks, hba->nutmrs) {if (ufshcd_clear_tm_cmd (hba, tag)) {
@@ -7321,6 +7368,8 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd)unsigned long flags, pending_reqs =0 , not_cleared =0 ;struct Scsi_Host *host;struct ufs_hba *hba; +struct ufs_hw_queue *hwq; +struct ufshcd_lrb *lrbp;u32 pos, not_cleared_mask =0 ;int err;u8 resp =0xF , lun;
@@ -7336,6 +7385,20 @@ static int ufshcd_eh_device_reset_handler(struct scsi_cmnd *cmd) goto out; } +if (is_mcq_enabled (hba)) { +for (pos =0 ; pos < hba->nutrs; pos++) { + lrbp = &hba->lrb[pos]; +if (ufshcd_cmd_inflight (lrbp->cmd) && + lrbp->lun == lun) { +ufshcd_clear_cmd (hba, pos); + hwq =ufshcd_mcq_req_to_hwq (hba,scsi_cmd_to_rq (lrbp->cmd)); +ufshcd_mcq_poll_cqe_lock (hba, hwq); + } + } + err =0 ; + goto out; + } +/* clear the commands that were pending for corresponding LUN */ spin_lock_irqsave (&hba->outstanding_lock, flags);for_each_set_bit (pos, &hba->outstanding_reqs, hba->nutrs)
Signature · 1 line
2.7.4
Bart Van Assche
reviewerRe: [PATCH v5 7/7] ufs: core: Add error handling for MCQ mode
Quoted 33 lines · Bao D. Nguyen
drivers/ufs/core/ufshcd.c +20
-9@@ -6378,18 +6407,36 @@ static bool ufshcd_abort_all(struct ufs_hba *hba)bool needs_reset = false;int tag, ret; -/* Clear pending transfer requests */ -for_each_set_bit (tag, &hba->outstanding_reqs, hba->nutrs) { - ret =ufshcd_try_to_abort_task (hba, tag); -dev_err (hba->dev,"Aborting tag %d / CDB %#02x %s\n" , tag, - hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0 ] :-1 , - ret ?"failed" :"succeeded" ); -if (ret) { - needs_reset = true; - goto out; +if (is_mcq_enabled (hba)) { +struct ufshcd_lrb *lrbp; +int tag; + +for (tag =0 ; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; +if (!ufshcd_cmd_inflight (lrbp->cmd)) +continue ; + ret =ufshcd_try_to_abort_task (hba, tag); +dev_err (hba->dev,"Aborting tag %d / CDB %#02x %s\n" , tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0 ] :-1 , + ret ?"failed" :"succeeded" ); +if (ret) { + needs_reset = true; + goto out; + } + } + }else { +/* Clear pending transfer requests */ +for_each_set_bit (tag, &hba->outstanding_reqs, hba->nutrs) {
drivers/ufs/core/ufshcd.c static bool ufshcd_abort_all(struct ufs_hba *hba) · line 6439
Quoted 10 lines
drivers/ufs/core/ufshcd.c +8
-0+ ret =ufshcd_try_to_abort_task (hba, tag); +dev_err (hba->dev,"Aborting tag %d / CDB %#02x %s\n" , tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0 ] :-1 , + ret ?"failed" :"succeeded" ); +if (ret) { + needs_reset = true; + goto out; + } } }
Please move the ufshcd_cmd_inflight() check into ufshcd_try_to_abort_task() such that the same code path can be used for MCQ and legacy mode.
Thanks,
Bart.
Re: [PATCH v5 7/7] ufs: core: Add error handling for MCQ mode
Quoted 38 lines · Bart Van Assche
Quoted 37 lines · Bao D. Nguyen
drivers/ufs/core/ufshcd.c +25
-9@@ -6378,18 +6407,36 @@ static bool ufshcd_abort_all(struct ufs_hba *hba) bool needs_reset = false; int tag, ret; - /* Clear pending transfer requests */ - for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { - ret = ufshcd_try_to_abort_task(hba, tag); - dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, - hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, - ret ? "failed" : "succeeded"); - if (ret) { - needs_reset = true; - goto out; + if (is_mcq_enabled(hba)) { + struct ufshcd_lrb *lrbp; + int tag; + + for (tag = 0; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; + if (!ufshcd_cmd_inflight(lrbp->cmd)) + continue; + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + if (ret) { + needs_reset = true; + goto out; + } + } + } else { + /* Clear pending transfer requests */ + for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + if (ret) {
Quoted 8 lines
drivers/ufs/core/ufshcd.c static bool ufshcd_abort_all(struct ufs_hba *hba) · line 6439
Quoted 5 lines
drivers/ufs/core/ufshcd.c +3
-0+ needs_reset = true; + goto out; + } } }Please move the
ufshcd_cmd_inflight()check intoufshcd_try_to_abort_task()such that the same code path can be used for MCQ and legacy mode.
Thank you, Bart.
I will make the change.
Thanks
Bao
Quoted 4 lines
Thanks,
Bart.
Re: [PATCH v5 7/7] ufs: core: Add error handling for MCQ mode
In reply to Bart Van AsscheQuoted 38 lines · Bart Van Assche
Quoted 37 lines · Bao D. Nguyen
drivers/ufs/core/ufshcd.c +25
-9@@ -6378,18 +6407,36 @@ static bool ufshcd_abort_all(struct ufs_hba *hba) bool needs_reset = false; int tag, ret; - /* Clear pending transfer requests */ - for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { - ret = ufshcd_try_to_abort_task(hba, tag); - dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, - hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, - ret ? "failed" : "succeeded"); - if (ret) { - needs_reset = true; - goto out; + if (is_mcq_enabled(hba)) { + struct ufshcd_lrb *lrbp; + int tag; + + for (tag = 0; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; + if (!ufshcd_cmd_inflight(lrbp->cmd)) + continue; + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + if (ret) { + needs_reset = true; + goto out; + } + } + } else { + /* Clear pending transfer requests */ + for_each_set_bit(tag, &hba->outstanding_reqs, hba->nutrs) { + ret = ufshcd_try_to_abort_task(hba, tag); + dev_err(hba->dev, "Aborting tag %d / CDB %#02x %s\n", tag, + hba->lrb[tag].cmd ? hba->lrb[tag].cmd->cmnd[0] : -1, + ret ? "failed" : "succeeded"); + if (ret) {
Quoted 8 lines
drivers/ufs/core/ufshcd.c static bool ufshcd_abort_all(struct ufs_hba *hba) · line 6439
Quoted 5 lines
drivers/ufs/core/ufshcd.c +3
-0+ needs_reset = true; + goto out; + } } }Please move the
ufshcd_cmd_inflight()check intoufshcd_try_to_abort_task()such that the same code path can be used for MCQ and legacy mode.
Hi Bart,
Because the ufshcd_try_to_abort_task() is shared by sdb and mcq modes, I feel a bit uncomfortable using the new function ufshcd_cmd_inflight() in ufshcd_try_to_abort_task(). In this patch series, I am trying to avoid changing the sdb error handling logic as much as possible; only add error handling support for mcq mode. If you feel there is a very good benefit in making the change, I would give it a try. Otherwise, I would prefer not touching sdb error handling code that has been working well. Please let me know.
Thanks,
Bao
Quoted 4 lines
Thanks,
Bart.
Re: [PATCH v5 7/7] ufs: core: Add error handling for MCQ mode
In reply to Bao D. NguyenHi Bao,
Bao D. Nguyen <[email protected]> 於 2023年5月12日 週五 下午2:34寫道:
Quoted 13 lines
Add support for error handling for MCQ mode.
Signed-off-by Bao D. Nguyen <[email protected]>--- drivers/ufs/core/ufshcd.c | 85 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 74 insertions(+), 11 deletions(-)drivers/ufs/core/ufshcd.c +0
-0
drivers/ufs/core/ufshcd.c static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, · line 3156
Quoted 9 lines
drivers/ufs/core/ufshcd.c +6
-0@@ -3148,6 +3148,16 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT ;dev_dbg (hba->dev,"%s: dev_cmd request timedout, tag %d\n" , __func__, lrbp->task_tag); + +/* MCQ mode */ +if (is_mcq_enabled (hba)) { + err =ufshcd_clear_cmd (hba, lrbp->task_tag); +if (!err) + hba->dev_cmd.complete =NULL ;
How about always clearing hba->dev_cmd.complete? If ufshcd_clear_cmd() fails (for example, times out), "complete" should be cleared, similar to the "pending" case in the SDB path.
Quoted 27 lines
drivers/ufs/core/ufshcd.c +12
-1+return err; + } + +/* SDB mode */ if (ufshcd_clear_cmd (hba, lrbp->task_tag) ==0 ) {/* successfully cleared the command, retry if needed */ err = -EAGAIN ;@@ -5581,6 +5591,10 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)*/ static irqreturn_t ufshcd_transfer_req_compl (struct ufs_hba *hba) { +struct ufshcd_lrb *lrbp; +u32 hwq_num, utag; +int tag; +/* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. * In order to prevent other interrupts starvation the DB is read once @@ -5599,7 +5613,22 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba)* Ignore the ufshcd_poll() return value and return IRQ_HANDLED since we * do not want polling to trigger spurious interrupt complaints. */ -ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); +if (!is_mcq_enabled (hba)) { +ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); + goto out; + }
drivers/ufs/core/ufshcd.c static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) · line 5629
Quoted 10 lines
drivers/ufs/core/ufshcd.c +10
-0+ +/* MCQ mode */ +for (tag =0 ; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; +if (ufshcd_cmd_inflight (lrbp->cmd)) { + utag =blk_mq_unique_tag (scsi_cmd_to_rq (lrbp->cmd)); + hwq_num =blk_mq_unique_tag_to_hwq (utag); +ufshcd_poll (hba->host, hwq_num); + } + }
In SDB mode, the DOOR_BELL is reset by ufshcd_hba_stop(). All bits that were previously set in DOOR_BELL are also set in "completed_reqs" in ufshcd_poll(). This allows ufshcd_poll() to handle all outstanding requests properly.
However, in MCQ mode, the CQ tail registers cannot provide the same information after they are reset. Hence, they cannot be properly referenced by ufshcd_poll().
Thanks,
Stanley Chu
Re: [PATCH v5 7/7] ufs: core: Add error handling for MCQ mode
Quoted 74 lines · Stanley Chu
Hi Bao,
Bao D. Nguyen <[email protected]> 於 2023年5月12日 週五 下午2:34寫道:
Quoted 13 lines
Add support for error handling for MCQ mode.
Signed-off-by Bao D. Nguyen <[email protected]>--- drivers/ufs/core/ufshcd.c | 85 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 74 insertions(+), 11 deletions(-)drivers/ufs/core/ufshcd.c +0
-0drivers/ufs/core/ufshcd.c static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, · line 3156
Quoted 9 lines
drivers/ufs/core/ufshcd.c +6
-0@@ -3148,6 +3148,16 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT ;dev_dbg (hba->dev,"%s: dev_cmd request timedout, tag %d\n" , __func__, lrbp->task_tag); + +/* MCQ mode */ +if (is_mcq_enabled (hba)) { + err =ufshcd_clear_cmd (hba, lrbp->task_tag); +if (!err) + hba->dev_cmd.complete =NULL ;How about always clearing
hba->dev_cmd.complete? Ifufshcd_clear_cmd()fails (for example, times out), "complete" should be cleared, similar to the "pending" case in the SDB path.Quoted 27 lines
drivers/ufs/core/ufshcd.c +12
-1+return err; + } + +/* SDB mode */ if (ufshcd_clear_cmd (hba, lrbp->task_tag) ==0 ) {/* successfully cleared the command, retry if needed */ err = -EAGAIN ;@@ -5581,6 +5591,10 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)*/ static irqreturn_t ufshcd_transfer_req_compl (struct ufs_hba *hba) { +struct ufshcd_lrb *lrbp; +u32 hwq_num, utag; +int tag; +/* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. * In order to prevent other interrupts starvation the DB is read once @@ -5599,7 +5613,22 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba)* Ignore the ufshcd_poll() return value and return IRQ_HANDLED since we * do not want polling to trigger spurious interrupt complaints. */ -ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); +if (!is_mcq_enabled (hba)) { +ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); + goto out; + }drivers/ufs/core/ufshcd.c static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) · line 5629
Quoted 10 lines
drivers/ufs/core/ufshcd.c +10
-0+ +/* MCQ mode */ +for (tag =0 ; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; +if (ufshcd_cmd_inflight (lrbp->cmd)) { + utag =blk_mq_unique_tag (scsi_cmd_to_rq (lrbp->cmd)); + hwq_num =blk_mq_unique_tag_to_hwq (utag); +ufshcd_poll (hba->host, hwq_num); + } + }In SDB mode, the
DOOR_BELLis reset byufshcd_hba_stop(). All bits that were previously set inDOOR_BELLare also set in "completed_reqs" inufshcd_poll(). This allowsufshcd_poll()to handle all outstanding requests properly.
Quoted 3 lines
However, in MCQ mode, the CQ tail registers cannot provide the same information after they are reset. Hence, they cannot be properly referenced by
ufshcd_poll().
A fixed version sample is as follows and has been tested on our end.
struct scsi_cmnd *cmd;
for (tag = 0; tag < hba->nutrs; tag++) {
lrbp = &hba->lrb[tag];
cmd = lrbp->cmd;
if (ufshcd_cmd_inflight(cmd)) {
set_host_byte(cmd, DID_ERROR);
ufshcd_release_scsi_cmd(hba, lrbp);
scsi_done(cmd);
}
}Thanks,
Stanley Chu
Re: [PATCH v5 7/7] ufs: core: Add error handling for MCQ mode
Quoted 75 lines · Stanley Chu
Quoted 74 lines · Stanley Chu
Hi Bao,
Bao D. Nguyen <[email protected]> 於 2023年5月12日 週五 下午2:34寫道:
Quoted 13 lines
Add support for error handling for MCQ mode.
Signed-off-by Bao D. Nguyen <[email protected]>--- drivers/ufs/core/ufshcd.c | 85 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 74 insertions(+), 11 deletions(-)drivers/ufs/core/ufshcd.c +0
-0drivers/ufs/core/ufshcd.c static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, · line 3156
Quoted 9 lines
drivers/ufs/core/ufshcd.c +6
-0@@ -3148,6 +3148,16 @@ static int ufshcd_wait_for_dev_cmd(struct ufs_hba *hba, err = -ETIMEDOUT ;dev_dbg (hba->dev,"%s: dev_cmd request timedout, tag %d\n" , __func__, lrbp->task_tag); + +/* MCQ mode */ +if (is_mcq_enabled (hba)) { + err =ufshcd_clear_cmd (hba, lrbp->task_tag); +if (!err) + hba->dev_cmd.complete =NULL ;How about always clearing
hba->dev_cmd.complete? Ifufshcd_clear_cmd()fails (for example, times out), "complete" should be cleared, similar to the "pending" case in the SDB path.Quoted 27 lines
drivers/ufs/core/ufshcd.c +12
-1+return err; + } + +/* SDB mode */ if (ufshcd_clear_cmd (hba, lrbp->task_tag) ==0 ) {/* successfully cleared the command, retry if needed */ err = -EAGAIN ;@@ -5581,6 +5591,10 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)*/ static irqreturn_t ufshcd_transfer_req_compl (struct ufs_hba *hba) { +struct ufshcd_lrb *lrbp; +u32 hwq_num, utag; +int tag; +/* Resetting interrupt aggregation counters first and reading the * DOOR_BELL afterward allows us to handle all the completed requests. * In order to prevent other interrupts starvation the DB is read once @@ -5599,7 +5613,22 @@ static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba)* Ignore the ufshcd_poll() return value and return IRQ_HANDLED since we * do not want polling to trigger spurious interrupt complaints. */ -ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); +if (!is_mcq_enabled (hba)) { +ufshcd_poll (hba->host,UFSHCD_POLL_FROM_INTERRUPT_CONTEXT ); + goto out; + }drivers/ufs/core/ufshcd.c static irqreturn_t ufshcd_transfer_req_compl(struct ufs_hba *hba) · line 5629
Quoted 10 lines
drivers/ufs/core/ufshcd.c +10
-0+ +/* MCQ mode */ +for (tag =0 ; tag < hba->nutrs; tag++) { + lrbp = &hba->lrb[tag]; +if (ufshcd_cmd_inflight (lrbp->cmd)) { + utag =blk_mq_unique_tag (scsi_cmd_to_rq (lrbp->cmd)); + hwq_num =blk_mq_unique_tag_to_hwq (utag); +ufshcd_poll (hba->host, hwq_num); + } + }In SDB mode, the
DOOR_BELLis reset byufshcd_hba_stop(). All bits that were previously set inDOOR_BELLare also set in "completed_reqs" inufshcd_poll(). This allowsufshcd_poll()to handle all outstanding requests properly.
Quoted 5 lines
Quoted 3 lines
However, in MCQ mode, the CQ tail registers cannot provide the same information after they are reset. Hence, they cannot be properly referenced by
ufshcd_poll().A fixed version sample is as follows and has been tested on our end.
Thank you Stanley. I will make the change.
Quoted 15 lines
struct
scsi_cmnd*cmd;for (tag = 0; tag < hba->nutrs; tag++) { lrbp = &hba->lrb[tag]; cmd = lrbp->cmd; if (ufshcd_cmd_inflight(cmd)) { set_host_byte(cmd, DID_ERROR); ufshcd_release_scsi_cmd(hba, lrbp); scsi_done(cmd); } }Thanks,
Stanley Chu