Skip to contentSkip to conversation

scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

PATCH In review Jaegeuk Kim 4 participants 13 messages Dec 18, 2020 – Jan 6, 2021 SCSI

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows such as suspend/resume, link_recovery, and error_handler.

---
 drivers/scsi/ufs/ufshcd.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd.c +9 -5
index e221add25a7e..e711def829cd 100644
@@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
 	if (ret)
 		dev_err(hba->dev, "%s: link recovery failed, err %d",
 			__func__, ret);
+	else
+		ufshcd_clear_ua_wluns(hba);
 
 	return ret;
 }
@@ -5968,6 +5970,8 @@ static void ufshcd_err_handler(struct work_struct *work)
 	ufshcd_scsi_unblock_requests(hba);
 	ufshcd_err_handling_unprepare(hba);
 	up(&hba->eh_sem);
+
+	ufshcd_clear_ua_wluns(hba);
 }
 
 /**
@@ -6908,14 +6912,11 @@ static int ufshcd_host_reset_and_restore(struct ufs_hba *hba)
 	ufshcd_set_clk_freq(hba, true);
 
 	err = ufshcd_hba_enable(hba);
-	if (err)
-		goto out;
 
 	/* Establish the link again and restore the device */
-	err = ufshcd_probe_hba(hba, false);
 	if (!err)
-		ufshcd_clear_ua_wluns(hba);
-out:
+		err = ufshcd_probe_hba(hba, false);
+
 	if (err)
 		dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
 	ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
@@ -8745,6 +8746,7 @@ static int ufshcd_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 		ufshcd_resume_clkscaling(hba);
 	hba->clk_gating.is_suspended = false;
 	hba->dev_info.b_rpm_dev_flush_capable = false;
+	ufshcd_clear_ua_wluns(hba);
 	ufshcd_release(hba);
 out:
 	if (hba->dev_info.b_rpm_dev_flush_capable) {
@@ -8855,6 +8857,8 @@ static int ufshcd_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 		cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
 	}
 
+	ufshcd_clear_ua_wluns(hba);
+
 	/* Schedule clock gating in case of no access to UFS device yet */
 	ufshcd_release(hba);
 
Signature · 1 line
2.29.2.729.g45daf8777d-goog
#1 lore raw

Hi J,

Quoted 11 lines

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
such as suspend/resume, link_recovery, and error_handler.

Not sure that suspend/resume are UAC events?

Also the 'fixes' tag is missing.

Thanks,
Avri

#8 lore raw
In reply to Jaegeuk Kim
Quoted 29 lines · Jaegeuk Kim

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows such as suspend/resume, link_recovery, and error_handler.

---
 drivers/scsi/ufs/ufshcd.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd.c +2 -0
index e221add25a7e..e711def829cd 100644
@@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
 	if (ret)
 		dev_err(hba->dev, "%s: link recovery failed, err %d",
 			__func__, ret);
+	else
+		ufshcd_clear_ua_wluns(hba);
 
 	return ret;
 }

drivers/scsi/ufs/ufshcd.c static void ufshcd_err_handler(struct work_struct · line 5974

Quoted 5 lines
drivers/scsi/ufs/ufshcd.c +1 -0
@@ -5968,6 +5970,8 @@ static void ufshcd_err_handler(struct work_struct 
 *work)
 	ufshcd_scsi_unblock_requests(hba);
 	ufshcd_err_handling_unprepare(hba);
 	up(&hba->eh_sem);
+

Maybe add a check like if (!err && needs_reset) as error handler also handles non-fatal errors which do not require a full reset and restore?

Quoted 40 lines
drivers/scsi/ufs/ufshcd.c +1 -0
+	ufshcd_clear_ua_wluns(hba);
 }
 

/**

drivers/scsi/ufs/ufshcd.c +2 -5
@@ -6908,14 +6912,11 @@ static int
 ufshcd_host_reset_and_restore(struct ufs_hba *hba)
 	ufshcd_set_clk_freq(hba, true);
 
 	err = ufshcd_hba_enable(hba);
-	if (err)
-		goto out;
 
 	/* Establish the link again and restore the device */
-	err = ufshcd_probe_hba(hba, false);
 	if (!err)
-		ufshcd_clear_ua_wluns(hba);
-out:
+		err = ufshcd_probe_hba(hba, false);
+
 	if (err)
 		dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
drivers/scsi/ufs/ufshcd.c +1 -0
 	ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
@@ -8745,6 +8746,7 @@ static int ufshcd_suspend(struct ufs_hba *hba,
 enum ufs_pm_op pm_op)
 		ufshcd_resume_clkscaling(hba);
 	hba->clk_gating.is_suspended = false;
 	hba->dev_info.b_rpm_dev_flush_capable = false;
+	ufshcd_clear_ua_wluns(hba);
 	ufshcd_release(hba);
 out:
drivers/scsi/ufs/ufshcd.c +2 -0
 	if (hba->dev_info.b_rpm_dev_flush_capable) {
@@ -8855,6 +8857,8 @@ static int ufshcd_resume(struct ufs_hba *hba,
 enum ufs_pm_op pm_op)
 		cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
 	}
 
+	ufshcd_clear_ua_wluns(hba);
+
 	/* Schedule clock gating in case of no access to UFS device yet */
 	ufshcd_release(hba);
#9 lore raw
Quoted 28 lines · Can Guo
Quoted 27 lines · Jaegeuk Kim

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
such as suspend/resume, link_recovery, and error_handler.

---
 drivers/scsi/ufs/ufshcd.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd.c +2 -0
index e221add25a7e..e711def829cd 100644
@@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
 	if (ret)
 		dev_err(hba->dev, "%s: link recovery failed, err %d",
 			__func__, ret);
+	else
+		ufshcd_clear_ua_wluns(hba);
 
Quoted 12 lines

drivers/scsi/ufs/ufshcd.c static void ufshcd_err_handler(struct work_struct · line 5974

Quoted 8 lines
drivers/scsi/ufs/ufshcd.c +1 -0
 	return ret;
 }
@@ -5968,6 +5970,8 @@ static void ufshcd_err_handler(struct work_struct
 *work)
 	ufshcd_scsi_unblock_requests(hba);
 	ufshcd_err_handling_unprepare(hba);
 	up(&hba->eh_sem);
+

Maybe add a check like if (!err && needs_reset) as error handler also handles non-fatal errors which do not require a full reset and restore?

I see. Let me add it in v2.

Quoted 41 lines
Quoted 41 lines
drivers/scsi/ufs/ufshcd.c +1 -0
+	ufshcd_clear_ua_wluns(hba);
 }
 

/**

drivers/scsi/ufs/ufshcd.c +2 -5
@@ -6908,14 +6912,11 @@ static int
 ufshcd_host_reset_and_restore(struct ufs_hba *hba)
 	ufshcd_set_clk_freq(hba, true);
 
 	err = ufshcd_hba_enable(hba);
-	if (err)
-		goto out;
 
 	/* Establish the link again and restore the device */
-	err = ufshcd_probe_hba(hba, false);
 	if (!err)
-		ufshcd_clear_ua_wluns(hba);
-out:
+		err = ufshcd_probe_hba(hba, false);
+
 	if (err)
 		dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
drivers/scsi/ufs/ufshcd.c +1 -0
 	ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
@@ -8745,6 +8746,7 @@ static int ufshcd_suspend(struct ufs_hba *hba,
 enum ufs_pm_op pm_op)
 		ufshcd_resume_clkscaling(hba);
 	hba->clk_gating.is_suspended = false;
 	hba->dev_info.b_rpm_dev_flush_capable = false;
+	ufshcd_clear_ua_wluns(hba);
 	ufshcd_release(hba);
 out:
drivers/scsi/ufs/ufshcd.c +2 -0
 	if (hba->dev_info.b_rpm_dev_flush_capable) {
@@ -8855,6 +8857,8 @@ static int ufshcd_resume(struct ufs_hba *hba,
 enum ufs_pm_op pm_op)
 		cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
 	}
 
+	ufshcd_clear_ua_wluns(hba);
+
 	/* Schedule clock gating in case of no access to UFS device yet */
 	ufshcd_release(hba);
#2 lore raw
In reply to Avri Altman

Hi,

Quoted 10 lines · Avri Altman

Hi J,

Quoted 8 lines

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

Quoted 4 lines
Quoted 3 lines

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
such as suspend/resume, link_recovery, and error_handler.

Not sure that suspend/resume are UAC events?

Could you elaborate a bit? The goal is to clear UAC after UFS reset happens.

Quoted 2 lines

Also the 'fixes' tag is missing.

Added. Thanks,

Quoted 3 lines

Thanks,
Avri

#10 lore raw
In reply to Jaegeuk Kim

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows such as suspend/resume, link_recovery, and error_handler.

Fixes 1918651f2d7e ("scsi: ufs: Clear UAC for RPMB after ufshcd resets")

---

Change log from v1:
- add condition check to call
- add Fixes tag

 drivers/scsi/ufs/ufshcd.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
drivers/scsi/ufs/ufshcd.c +10 -5
index e221add25a7e..29a62552f6f1 100644
@@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
 	if (ret)
 		dev_err(hba->dev, "%s: link recovery failed, err %d",
 			__func__, ret);
+	else
+		ufshcd_clear_ua_wluns(hba);
 
 	return ret;
 }
@@ -5968,6 +5970,9 @@ static void ufshcd_err_handler(struct work_struct *work)
 	ufshcd_scsi_unblock_requests(hba);
 	ufshcd_err_handling_unprepare(hba);
 	up(&hba->eh_sem);
+
+	if (!err && needs_reset)
+		ufshcd_clear_ua_wluns(hba);
 }
 
 /**
@@ -6908,14 +6913,11 @@ static int ufshcd_host_reset_and_restore(struct ufs_hba *hba)
 	ufshcd_set_clk_freq(hba, true);
 
 	err = ufshcd_hba_enable(hba);
-	if (err)
-		goto out;
 
 	/* Establish the link again and restore the device */
-	err = ufshcd_probe_hba(hba, false);
 	if (!err)
-		ufshcd_clear_ua_wluns(hba);
-out:
+		err = ufshcd_probe_hba(hba, false);
+
 	if (err)
 		dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
 	ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
@@ -8745,6 +8747,7 @@ static int ufshcd_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 		ufshcd_resume_clkscaling(hba);
 	hba->clk_gating.is_suspended = false;
 	hba->dev_info.b_rpm_dev_flush_capable = false;
+	ufshcd_clear_ua_wluns(hba);
 	ufshcd_release(hba);
 out:
 	if (hba->dev_info.b_rpm_dev_flush_capable) {
@@ -8855,6 +8858,8 @@ static int ufshcd_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op)
 		cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
 	}
 
+	ufshcd_clear_ua_wluns(hba);
+
 	/* Schedule clock gating in case of no access to UFS device yet */
 	ufshcd_release(hba);
 
Signature · 1 line
2.29.2.729.g45daf8777d-goog
#3 lore raw
In reply to Jaegeuk Kim
Quoted 8 lines
Quoted 1 line
Quoted 1 line

When gate_work/ungate_work gets an error during hibern8_enter or

exit,

Quoted 6 lines
Quoted 6 lines

ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

Quoted 6 lines
Quoted 4 lines
Quoted 3 lines

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
such as suspend/resume, link_recovery, and error_handler.

Not sure that suspend/resume are UAC events?

Could you elaborate a bit? The goal is to clear UAC after UFS reset happens.

So why calling it on every suspend and resume?

Quoted 9 lines
Quoted 3 lines

Also the 'fixes' tag is missing.

Added. Thanks,

Quoted 3 lines

Thanks,
Avri

#4 lore raw
Quoted 8 lines · Avri Altman
Quoted 8 lines
Quoted 1 line
Quoted 1 line

When gate_work/ungate_work gets an error during hibern8_enter or

exit,

Quoted 6 lines
Quoted 6 lines

ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 3 lines

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
such as suspend/resume, link_recovery, and error_handler.

Not sure that suspend/resume are UAC events?

Could you elaborate a bit? The goal is to clear UAC after UFS reset happens.

So why calling it on every suspend and resume?

1. If UAC was cleared, there's no impact.
2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.

Quoted 10 lines
Quoted 10 lines
Quoted 4 lines

Also the 'fixes' tag is missing.

Added. Thanks,

Quoted 3 lines

Thanks,
Avri

#5 lore raw
Quoted 15 lines · Avri Altman
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line

When gate_work/ungate_work gets an error during hibern8_enter or

exit,

Quoted 7 lines
Quoted 7 lines

ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per

recovery

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

flows
such as suspend/resume, link_recovery, and error_handler.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Not sure that suspend/resume are UAC events?

Could you elaborate a bit? The goal is to clear UAC after UFS reset

happens.

Quoted 1 line

So why calling it on every suspend and resume?

1. If UAC was cleared, there's no impact.

But the command is still sent.

Quoted 2 lines

2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.

Seems excessive IMO.
Why not selectively send when indeed required, e.g. on reset?

#6 lore raw
Quoted 15 lines · Avri Altman
Quoted 15 lines · Avri Altman
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line

When gate_work/ungate_work gets an error during hibern8_enter or

exit,

Quoted 7 lines
Quoted 7 lines

ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per

recovery

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

flows
such as suspend/resume, link_recovery, and error_handler.

Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Not sure that suspend/resume are UAC events?

Could you elaborate a bit? The goal is to clear UAC after UFS reset

happens.

Quoted 1 line

So why calling it on every suspend and resume?

1. If UAC was cleared, there's no impact.

But the command is still sent.

No, ufshcd_clear_ua_wluns() will return by hba->wlun_dev_clr_ua.

Quoted 5 lines
Quoted 3 lines

2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.

Seems excessive IMO.
Why not selectively send when indeed required, e.g. on reset?

I think hba->wlun_dev_clr_ua is the indicator whether there was a reset or not.

#7 lore raw
Quoted 16 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

In order to avoid it, ufshcd_clear_ua_wluns() can be called per

recovery

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

flows
such as suspend/resume, link_recovery, and error_handler.

Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Not sure that suspend/resume are UAC events?

Could you elaborate a bit? The goal is to clear UAC after UFS reset

happens.

Quoted 1 line

So why calling it on every suspend and resume?

1. If UAC was cleared, there's no impact.

But the command is still sent.

No, ufshcd_clear_ua_wluns() will return by hba->wlun_dev_clr_ua.

Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.

Seems excessive IMO.
Why not selectively send when indeed required, e.g. on reset?

I think hba->wlun_dev_clr_ua is the indicator whether there was a reset or not.

Ahha - I missed that. Thanks for clarifying it.
OK Then.

Thanks,
Avri

#11 lore raw
In reply to Jaegeuk Kim

Jaegeuk,

Quoted 4 lines

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

Quoted 8 lines
     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows such as suspend/resume, link_recovery, and error_handler.

Fixes 1918651f2d7e ("scsi: ufs: Clear UAC for RPMB after ufshcd resets")

Please resubmit instead of replying to an existing patch. Both b4 and patchwork get confused.

Thanks!

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#12 lore raw
Quoted 10 lines · Martin K. Petersen

Jaegeuk,

Quoted 7 lines

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()

     ufshcd_clear_ua_wluns() -> stuck

ufshcd_scsi_unblock_requests()

Quoted 8 lines
Quoted 5 lines

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows such as suspend/resume, link_recovery, and error_handler.

Fixes 1918651f2d7e ("scsi: ufs: Clear UAC for RPMB after ufshcd resets")

Please resubmit instead of replying to an existing patch. Both b4 and patchwork get confused.

Ok, I posted a new one. Thanks,

Quoted 5 lines

Thanks!

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering