Change log from v3:
- use __ufshcd_release with a fix in __ufshcd_release
Change log from v2:
- use active_req-- instead of __ufshcd_release to avoid UFS timeout
Change log from v1:
- remove clkgating_enable check in __ufshcd_release
- use __uhfshcd_release instead of active_req.
PATCH v4 1/5 scsi: ufs: atomic update for clkgating_enable
From: Jaegeuk Kim <[email protected]>
When giving a stress test which enables/disables clkgating, we hit device timeout sometimes. This patch avoids subtle racy condition to address it.
Note that, this requires a patch to address the device stuck by REQ_CLKS_OFF in __ufshcd_release().
The fix is "scsi: ufs: avoid to call REQ_CLKS_OFF to CLKS_OFF".
--- drivers/scsi/ufs/ufshcd.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +6 -6
@@ -1808,19 +1808,19 @@ static ssize_t ufshcd_clkgate_enable_store(struct device *dev,return -EINVAL ; value = !!value; + +spin_lock_irqsave (hba->host->host_lock, flags);if (value == hba->clk_gating.is_enabled) goto out; -if (value) { -ufshcd_release (hba); - }else { -spin_lock_irqsave (hba->host->host_lock, flags); +if (value) +__ufshcd_release (hba); +else hba->clk_gating.active_reqs++; -spin_unlock_irqrestore (hba->host->host_lock, flags); - } hba->clk_gating.is_enabled = value; out: +spin_unlock_irqrestore (hba->host->host_lock, flags);return count; }
Signature · 1 line
2.29.0.rc1.297.gfa9743e501-goog
Re: [PATCH v4 1/5] scsi: ufs: atomic update for clkgating_enable
Quoted 9 lines · Jaegeuk Kim
From: Jaegeuk Kim <[email protected]>
When giving a stress test which enables/disables clkgating, we hit device timeout sometimes. This patch avoids subtle racy condition to address it.
Note that, this requires a patch to address the device stuck by
REQ_CLKS_OFFin__ufshcd_release().The fix is "scsi: ufs: avoid to call
REQ_CLKS_OFFtoCLKS_OFF".
Why don't you just squash the fix into this one?
Thanks,
Can Guo.
Quoted 36 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/scsi/ufs/ufshcd.c +6
-6@@ -1808,19 +1808,19 @@ static ssize_t ufshcd_clkgate_enable_store(struct device *dev, return -EINVAL; value = !!value; + +spin_lock_irqsave (hba->host->host_lock, flags);if (value == hba->clk_gating.is_enabled) goto out; -if (value) { -ufshcd_release (hba); - }else { -spin_lock_irqsave (hba->host->host_lock, flags); +if (value) +__ufshcd_release (hba); +else hba->clk_gating.active_reqs++; -spin_unlock_irqrestore (hba->host->host_lock, flags); - } hba->clk_gating.is_enabled = value; out: +spin_unlock_irqrestore (hba->host->host_lock, flags);return count; }
Re: [PATCH v4 1/5] scsi: ufs: atomic update for clkgating_enable
Quoted 8 lines · Can Guo
Quoted 7 lines · Jaegeuk Kim
From: Jaegeuk Kim <[email protected]>
When giving a stress test which enables/disables clkgating, we hit device
timeout sometimes. This patch avoids subtle racy condition to address it.
Quoted 7 lines
Quoted 5 lines
Note that, this requires a patch to address the device stuck by
REQ_CLKS_OFFin__ufshcd_release().The fix is "scsi: ufs: avoid to call
REQ_CLKS_OFFtoCLKS_OFF".Why don't you just squash the fix into this one?
I'm seeing this patch just revealed that problem.
Quoted 41 lines
Thanks,
Can Guo.
Quoted 36 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/scsi/ufs/ufshcd.c +6
-6@@ -1808,19 +1808,19 @@ static ssize_t ufshcd_clkgate_enable_store(struct device *dev, return -EINVAL; value = !!value; + +spin_lock_irqsave (hba->host->host_lock, flags);if (value == hba->clk_gating.is_enabled) goto out; -if (value) { -ufshcd_release (hba); - }else { -spin_lock_irqsave (hba->host->host_lock, flags); +if (value) +__ufshcd_release (hba); +else hba->clk_gating.active_reqs++; -spin_unlock_irqrestore (hba->host->host_lock, flags); - } hba->clk_gating.is_enabled = value; out: +spin_unlock_irqrestore (hba->host->host_lock, flags);return count; }
Re: [PATCH v4 1/5] scsi: ufs: atomic update for clkgating_enable
Quoted 13 lines · Jaegeuk Kim
Quoted 12 lines · Can Guo
Quoted 11 lines · Jaegeuk Kim
From: Jaegeuk Kim <[email protected]>
When giving a stress test which enables/disables clkgating, we hit device
timeout sometimes. This patch avoids subtle racy condition to address it.Note that, this requires a patch to address the device stuck by
REQ_CLKS_OFFin__ufshcd_release().
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
The fix is "scsi: ufs: avoid to call
REQ_CLKS_OFFtoCLKS_OFF".Why don't you just squash the fix into this one?
I'm seeing this patch just revealed that problem.
That scenario (back to back calling of ufshcd_release()) only happens when you stress the clkgate_enable sysfs node, so let's keep the fix as one to make things simple. What do you think?
Thanks,
Can Guo.
Quoted 42 lines
Quoted 42 lines
Thanks,
Can Guo.
Quoted 36 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/scsi/ufs/ufshcd.c +6
-6@@ -1808,19 +1808,19 @@ static ssize_t ufshcd_clkgate_enable_store(struct device *dev, return -EINVAL; value = !!value; + +spin_lock_irqsave (hba->host->host_lock, flags);if (value == hba->clk_gating.is_enabled) goto out; -if (value) { -ufshcd_release (hba); - }else { -spin_lock_irqsave (hba->host->host_lock, flags); +if (value) +__ufshcd_release (hba); +else hba->clk_gating.active_reqs++; -spin_unlock_irqrestore (hba->host->host_lock, flags); - } hba->clk_gating.is_enabled = value; out: +spin_unlock_irqrestore (hba->host->host_lock, flags);return count; }
Re: [PATCH v4 1/5] scsi: ufs: atomic update for clkgating_enable
Quoted 16 lines · Can Guo
Quoted 15 lines · Jaegeuk Kim
Quoted 14 lines · Can Guo
Quoted 12 lines · Jaegeuk Kim
From: Jaegeuk Kim <[email protected]>
When giving a stress test which enables/disables clkgating, we hit device
timeout sometimes. This patch avoids subtle racy condition to address it.Note that, this requires a patch to address the device stuck by
REQ_CLKS_OFFin__ufshcd_release().The fix is "scsi: ufs: avoid to call
REQ_CLKS_OFFtoCLKS_OFF".
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Why don't you just squash the fix into this one?
I'm seeing this patch just revealed that problem.
That scenario (back to back calling of
ufshcd_release()) only happens when you stress theclkgate_enablesysfs node, so let's keep the fix as one to make things simple. What do you think?
If we don't have this patch, do you think this issue won't happen at all?
Quoted 47 lines
Thanks,
Can Guo.
Quoted 42 lines
Quoted 42 lines
Thanks,
Can Guo.
Quoted 36 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/scsi/ufs/ufshcd.c +6
-6@@ -1808,19 +1808,19 @@ static ssize_t ufshcd_clkgate_enable_store(struct device *dev, return -EINVAL; value = !!value; + +spin_lock_irqsave (hba->host->host_lock, flags);if (value == hba->clk_gating.is_enabled) goto out; -if (value) { -ufshcd_release (hba); - }else { -spin_lock_irqsave (hba->host->host_lock, flags); +if (value) +__ufshcd_release (hba); +else hba->clk_gating.active_reqs++; -spin_unlock_irqrestore (hba->host->host_lock, flags); - } hba->clk_gating.is_enabled = value; out: +spin_unlock_irqrestore (hba->host->host_lock, flags);return count; }
Re: [PATCH v4 1/5] scsi: ufs: atomic update for clkgating_enable
Quoted 19 lines · Jaegeuk Kim
Quoted 18 lines · Can Guo
Quoted 17 lines · Jaegeuk Kim
Quoted 15 lines · Can Guo
Quoted 7 lines · Jaegeuk Kim
From: Jaegeuk Kim <[email protected]>
When giving a stress test which enables/disables clkgating, we hit device
timeout sometimes. This patch avoids subtle racy condition to address it.Quoted 5 lines
Note that, this requires a patch to address the device stuck by
REQ_CLKS_OFFin__ufshcd_release().The fix is "scsi: ufs: avoid to call
REQ_CLKS_OFFtoCLKS_OFF".Why don't you just squash the fix into this one?
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
I'm seeing this patch just revealed that problem.
That scenario (back to back calling of
ufshcd_release()) only happens when you stress theclkgate_enablesysfs node, so let's keep the fix as one to make things simple. What do you think?If we don't have this patch, do you think this issue won't happen at all?
At least I've never seen this scenario these years, otherwise I would have put up a fix.
Thanks,
Can Guo.
Quoted 47 lines
Quoted 47 lines
Thanks,
Can Guo.
Quoted 42 lines
Quoted 42 lines
Thanks,
Can Guo.
Quoted 36 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- drivers/scsi/ufs/ufshcd.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)drivers/scsi/ufs/ufshcd.c +6
-6@@ -1808,19 +1808,19 @@ static ssize_t ufshcd_clkgate_enable_store(struct device *dev, return -EINVAL; value = !!value; + +spin_lock_irqsave (hba->host->host_lock, flags);if (value == hba->clk_gating.is_enabled) goto out; -if (value) { -ufshcd_release (hba); - }else { -spin_lock_irqsave (hba->host->host_lock, flags); +if (value) +__ufshcd_release (hba); +else hba->clk_gating.active_reqs++; -spin_unlock_irqrestore (hba->host->host_lock, flags); - } hba->clk_gating.is_enabled = value; out: +spin_unlock_irqrestore (hba->host->host_lock, flags);return count; }
PATCH v4 2/5 scsi: ufs: clear UAC for FFU and RPMB LUNs
In reply to Jaegeuk KimFrom: Jaegeuk Kim <[email protected]>
In order to conduct FFU or RPMB operations, UFS needs to clear UAC. This patch clears it explicitly, so that we could get no failure given early execution.
--- drivers/scsi/ufs/ufshcd.c | 70 +++++++++++++++++++++++++++++++++++---- drivers/scsi/ufs/ufshcd.h | 1 + 2 files changed, 65 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +64 -6
@@ -7058,7 +7058,6 @@ static inline void ufshcd_blk_pm_runtime_init(struct scsi_device *sdev)static int ufshcd_scsi_add_wlus (struct ufs_hba *hba) {int ret =0 ; -struct scsi_device *sdev_rpmb;struct scsi_device *sdev_boot; hba->sdev_ufs_device = __scsi_add_device(hba->host, 0, 0,
@@ -7071,14 +7070,14 @@ static int ufshcd_scsi_add_wlus(struct ufs_hba *hba)ufshcd_blk_pm_runtime_init (hba->sdev_ufs_device);scsi_device_put (hba->sdev_ufs_device); - sdev_rpmb =__scsi_add_device (hba->host,0 ,0 , + hba->sdev_rpmb =__scsi_add_device (hba->host,0 ,0 ,ufshcd_upiu_wlun_to_scsi_wlun (UFS_UPIU_RPMB_WLUN ),NULL ); -if (IS_ERR (sdev_rpmb)) { - ret =PTR_ERR (sdev_rpmb); +if (IS_ERR (hba->sdev_rpmb)) { + ret =PTR_ERR (hba->sdev_rpmb); goto remove_sdev_ufs_device; } -ufshcd_blk_pm_runtime_init (sdev_rpmb); -scsi_device_put (sdev_rpmb); +ufshcd_blk_pm_runtime_init (hba->sdev_rpmb); +scsi_device_put (hba->sdev_rpmb); sdev_boot =__scsi_add_device (hba->host,0 ,0 ,ufshcd_upiu_wlun_to_scsi_wlun (UFS_UPIU_BOOT_WLUN ),NULL );
@@ -7602,6 +7601,63 @@ static int ufshcd_add_lus(struct ufs_hba *hba)return ret; } +static int +ufshcd_send_request_sense (struct ufs_hba *hba,struct scsi_device *sdp); + +static int ufshcd_clear_ua_wlun (struct ufs_hba *hba,u8 wlun) +{ +struct scsi_device *sdp; +unsigned long flags; +int ret =0 ; + +spin_lock_irqsave (hba->host->host_lock, flags); +if (wlun ==UFS_UPIU_UFS_DEVICE_WLUN ) + sdp = hba->sdev_ufs_device; +else if (wlun ==UFS_UPIU_RPMB_WLUN ) + sdp = hba->sdev_rpmb; +else +BUG_ON (1 ); +if (sdp) { + ret =scsi_device_get (sdp); +if (!ret && !scsi_device_online (sdp)) { + ret = -ENODEV ; +scsi_device_put (sdp); + } + }else { + ret = -ENODEV ; + } +spin_unlock_irqrestore (hba->host->host_lock, flags); +if (ret) + goto out_err; + + ret =ufshcd_send_request_sense (hba, sdp); +scsi_device_put (sdp); +out_err: +if (ret) +dev_err (hba->dev,"%s: UAC clear LU=%x ret = %d\n" , + __func__, wlun, ret); +return ret; +} + +static int ufshcd_clear_ua_wluns (struct ufs_hba *hba) +{ +int ret =0 ; + +if (!hba->wlun_dev_clr_ua) + goto out; + + ret =ufshcd_clear_ua_wlun (hba,UFS_UPIU_UFS_DEVICE_WLUN ); +if (!ret) + ret =ufshcd_clear_ua_wlun (hba,UFS_UPIU_RPMB_WLUN ); +if (!ret) + hba->wlun_dev_clr_ua = false; +out: +if (ret) +dev_err (hba->dev,"%s: Failed to clear UAC WLUNS ret = %d\n" , + __func__, ret); +return ret; +} +/** * ufshcd_probe_hba - probe hba to detect device and initialize * @hba: per-adapter instance
@@ -7721,6 +7777,8 @@ static void ufshcd_async_scan(void *data, async_cookie_t cookie)pm_runtime_put_sync (hba->dev);ufshcd_exit_clk_scaling (hba);ufshcd_hba_exit (hba); + }else { +ufshcd_clear_ua_wluns (hba); } }
drivers/scsi/ufs/ufshcd.h +1 -0
@@ -681,6 +681,7 @@ struct ufs_hba {* "UFS device" W-LU. */ struct scsi_device *sdev_ufs_device; +struct scsi_device *sdev_rpmb;enum ufs_dev_pwr_mode curr_dev_pwr_mode;enum uic_link_state uic_link_state;
Signature · 1 line
2.29.0.rc1.297.gfa9743e501-goog
PATCH v4 3/5 scsi: ufs: use WQ_HIGHPRI for gating work
In reply to Jaegeuk KimFrom: Jaegeuk Kim <[email protected]>
Must have WQ_MEM_RECLAIM
``WQ_MEM_RECLAIM``
All wq which might be used in the memory reclaim paths **MUST** have this flag set. The wq is guaranteed to have at least one execution context regardless of memory pressure.
--- drivers/scsi/ufs/ufshcd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +1 -1
@@ -1868,7 +1868,7 @@ static void ufshcd_init_clk_gating(struct ufs_hba *hba)snprintf (wq_name,ARRAY_SIZE (wq_name),"ufs_clk_gating_%d" , hba->host->host_no); hba->clk_gating.clk_gating_workq =alloc_ordered_workqueue (wq_name, -WQ_MEM_RECLAIM ); +WQ_MEM_RECLAIM |WQ_HIGHPRI ); hba->clk_gating.is_enabled = true;
Signature · 1 line
2.29.0.rc1.297.gfa9743e501-goog
PATCH v4 4/5 scsi: add more contexts in the ufs tracepoints
In reply to Jaegeuk KimFrom: Jaegeuk Kim <[email protected]>
This adds user-friendly tracepoints with group id.
--- drivers/scsi/ufs/ufshcd.c | 6 ++++-- include/trace/events/ufs.h | 21 +++++++++++++++++---- 2 files changed, 21 insertions(+), 6 deletions(-)
drivers/scsi/ufs/ufshcd.c +4 -2
@@ -348,7 +348,7 @@ static void ufshcd_add_command_trace(struct ufs_hba *hba, unsigned int tag, const char *str) { sector_t lba = -1; -u8 opcode =0 ; +u8 opcode =0 , group_id =0 ;u32 intr, doorbell;struct ufshcd_lrb *lrbp = &hba->lrb[tag];struct scsi_cmnd *cmd = lrbp->cmd;
@@ -374,13 +374,15 @@ static void ufshcd_add_command_trace(struct ufs_hba *hba, lba = cmd->request->bio->bi_iter.bi_sector; transfer_len =be32_to_cpu ( lrbp->ucd_req_ptr->sc.exp_data_transfer_len); +if (opcode ==WRITE_10 ) + group_id = lrbp->cmd->cmnd[6 ]; } } intr =ufshcd_readl (hba,REG_INTERRUPT_STATUS ); doorbell =ufshcd_readl (hba,REG_UTP_TRANSFER_REQ_DOOR_BELL );trace_ufshcd_command (dev_name (hba->dev), str, tag, - doorbell, transfer_len, intr, lba, opcode); + doorbell, transfer_len, intr, lba, opcode, group_id); }static void ufshcd_print_clk_freqs (struct ufs_hba *hba)
include/trace/events/ufs.h +17 -4
@@ -11,6 +11,15 @@#include <linux/tracepoint.h> +#define str_opcode (opcode) \ + __print_symbolic(opcode, \ + { WRITE_16, "WRITE_16" }, \ + { WRITE_10, "WRITE_10" }, \ + { READ_16, "READ_16" }, \ + { READ_10, "READ_10" }, \ + { SYNCHRONIZE_CACHE, "SYNC" }, \ + { UNMAP, "UNMAP" }) +#define UFS_LINK_STATES \ EM(UIC_LINK_OFF_STATE) \ EM(UIC_LINK_ACTIVE_STATE) \
@@ -215,9 +224,10 @@ DEFINE_EVENT(ufshcd_template, ufshcd_init, TRACE_EVENT(ufshcd_command, TP_PROTO(const char *dev_name, const char *str, unsigned int tag, u32 doorbell, int transfer_len, u32 intr, u64 lba, - u8 opcode), + u8 opcode, u8 group_id), - TP_ARGS(dev_name, str, tag, doorbell, transfer_len, intr, lba, opcode), + TP_ARGS(dev_name, str, tag, doorbell, transfer_len, + intr, lba, opcode, group_id), TP_STRUCT__entry( __string(dev_name, dev_name)
@@ -228,6 +238,7 @@ TRACE_EVENT(ufshcd_command, __field(u32, intr) __field(u64, lba) __field(u8, opcode) +__field (u8, group_id) ), TP_fast_assign(
@@ -239,13 +250,15 @@ TRACE_EVENT(ufshcd_command, __entry->intr = intr; __entry->lba = lba; __entry->opcode = opcode; + __entry->group_id = group_id; ),TP_printk ( -"%s: %s: tag: %u, DB: 0x%x, size: %d, IS: %u, LBA: %llu, opcode: 0x%x" , +"%s: %s: tag: %u, DB: 0x%x, size: %d, IS: %u, LBA: %llu, opcode: 0x%x (%s), group_id: 0x%x" ,__get_str (str),__get_str (dev_name), __entry->tag, __entry->doorbell, __entry->transfer_len, - __entry->intr, __entry->lba, (u32 )__entry->opcode + __entry->intr, __entry->lba, (u32 )__entry->opcode, +str_opcode (__entry->opcode), (u32 )__entry->group_id ) );
Signature · 1 line
2.29.0.rc1.297.gfa9743e501-goog
PATCH v4 5/5 scsi: ufs: fix clkgating on/off correctly
In reply to Jaegeuk KimThe below call stack prevents clk_gating at every IO completion. We can remove the condition, ufshcd_any_tag_in_use(), since clkgating_work will check it again.
ufshcd_complete_requests(struct ufs_hba *hba)
ufshcd_transfer_req_compl()
__ufshcd_transfer_req_compl()
__ufshcd_release(hba)
if (ufshcd_any_tag_in_use() == 1)
return;
ufshcd_tmc_handler(hba);
blk_mq_tagset_busy_iter();Note that, this still requires a work to deal with a potential racy condition when user sets clkgating.delay_ms to very small value. That can cause preventing clkgating by the check of ufshcd_any_tag_in_use() in gate_work.
--- drivers/scsi/ufs/ufshcd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +1 -1
@@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended || hba->ufshcd_state !=UFSHCD_STATE_OPERATIONAL || -ufshcd_any_tag_in_use (hba) || hba->outstanding_tasks || + hba->outstanding_tasks || hba->active_uic_cmd || hba->uic_async_done || hba->clk_gating.state ==CLKS_OFF )return ;
Signature · 1 line
2.29.0.rc1.297.gfa9743e501-goog
Re: [PATCH v4 5/5] scsi: ufs: fix clkgating on/off correctly
Quoted 13 lines · Jaegeuk Kim
The below call stack prevents
clk_gatingat every IO completion. We can remove the condition,ufshcd_any_tag_in_use(), sinceclkgating_workwill check it again.
ufshcd_complete_requests(struct ufs_hba *hba)
ufshcd_transfer_req_compl()__ufshcd_transfer_req_compl() __ufshcd_release(hba) if (ufshcd_any_tag_in_use() == 1) return; ufshcd_tmc_handler(hba); blk_mq_tagset_busy_iter();
Quoted 8 lines
Note that, this still requires a work to deal with a potential racy condition when user sets clkgating.delay_ms to very small value. That can cause preventing clkgating by the check of
ufshcd_any_tag_in_use()ingate_work.Fixes 7252a3603015 ("scsi: ufs: Avoid busy-waiting by eliminating tagconflicts")
Signed-off-by Jaegeuk Kim <[email protected]>Reviewed-by Asutosh Das <[email protected]>
Quoted 17 lines
--- drivers/scsi/ufs/ufshcd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/scsi/ufs/ufshcd.c +1
-1@@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended || hba->ufshcd_state !=UFSHCD_STATE_OPERATIONAL || -ufshcd_any_tag_in_use (hba) || hba->outstanding_tasks || + hba->outstanding_tasks || hba->active_uic_cmd || hba->uic_async_done || hba->clk_gating.state ==CLKS_OFF )return ;