Skip to contentSkip to conversation

scsi: ufs: Fix deadlock issue in ufshcd_wait_for_doorbell_clr()

PATCH Applied Bean Huo 3 participants 4 messages Dec 14, 2021 – Dec 23, 2021 SCSI

lore
Share & tools
lore

Bean Huo

author
#0 lore raw

PATCH v2 1/1 scsi: ufs: Fix deadlock issue in ufshcd_wait_for_doorbell_clr()

From: Bean Huo <[email protected]>

Call shost_for_each_device() with holding host->host_lock will cause a deadlock situation, which will cause the system to stall (the log as follow). Fix this issue by using __shost_for_each_device() in ufshcd_pending_cmds().

stalls on CPUs/tasks:
all trace:
__switch_to+0x120/0x170
0xffff800011643998
ask dump for CPU 5:
ask:kworker/u16:2 state:R running task stack: 0 pid: 80 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:
__switch_to+0x120/0x170
0x0
ask dump for CPU 6:
ask:kworker/u16:6 state:R running task stack: 0 pid: 164 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:
__switch_to+0x120/0x170
0xffff54e7c4429f80
ask dump for CPU 7:
ask:kworker/u16:4 state:R running task stack: 0 pid: 153 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:

__switch_to+0x120/0x170
blk_mq_run_hw_queue+0x34/0x110
blk_mq_sched_insert_request+0xb0/0x120
blk_execute_rq_nowait+0x68/0x88
blk_execute_rq+0x4c/0xd8
__scsi_execute+0xec/0x1d0
scsi_vpd_inquiry+0x84/0xf0
scsi_get_vpd_buf+0x34/0xb8
scsi_attach_vpd+0x34/0x140
scsi_probe_and_add_lun+0xa6c/0xab8
__scsi_scan_target+0x438/0x4f8
scsi_scan_channel+0x6c/0xa8
scsi_scan_host_selected+0xf0/0x150
do_scsi_scan_host+0x88/0x90
scsi_scan_host+0x1b4/0x1d0
ufshcd_async_scan+0x248/0x310
async_run_entry_fn+0x30/0x178
process_one_work+0x1e8/0x368
worker_thread+0x40/0x478
kthread+0x174/0x180
ret_from_fork+0x10/0x20
Fixes 8d077ede48c1 ("scsi: ufs: Optimize the command queueing code")
---
 drivers/scsi/ufs/ufshcd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +2 -1
index 6dd517267f1b..fbebb8309ef7 100644
@@ -1082,7 +1082,8 @@ static u32 ufshcd_pending_cmds(struct ufs_hba *hba)
 	struct scsi_device *sdev;
 	u32 pending = 0;
 
-	shost_for_each_device(sdev, hba->host)
+	lockdep_assert_held(hba->host->host_lock);
+	__shost_for_each_device(sdev, hba->host)
 		pending += sbitmap_weight(&sdev->budget_map);
 
 	return pending;
Signature · 1 line
2.25.1
Reviewed-by Tested-by on v2 1/1
#1 lore raw
Quoted 49 lines · Bean Huo

From: Bean Huo <[email protected]>

Call shost_for_each_device() with holding host->host_lock will cause a deadlock situation, which will cause the system to stall (the log as follow). Fix this issue by using __shost_for_each_device() in ufshcd_pending_cmds().

stalls on CPUs/tasks:
all trace:
__switch_to+0x120/0x170
0xffff800011643998
ask dump for CPU 5:
ask:kworker/u16:2 state:R running task stack: 0 pid: 80 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:
__switch_to+0x120/0x170
0x0
ask dump for CPU 6:
ask:kworker/u16:6 state:R running task stack: 0 pid: 164 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:
__switch_to+0x120/0x170
0xffff54e7c4429f80
ask dump for CPU 7:
ask:kworker/u16:4 state:R running task stack: 0 pid: 153 ppid: 2 flags:0x0000000a
orkqueue: events_unbound async_run_entry_fn
all trace:

__switch_to+0x120/0x170
blk_mq_run_hw_queue+0x34/0x110
blk_mq_sched_insert_request+0xb0/0x120
blk_execute_rq_nowait+0x68/0x88
blk_execute_rq+0x4c/0xd8
__scsi_execute+0xec/0x1d0
scsi_vpd_inquiry+0x84/0xf0
scsi_get_vpd_buf+0x34/0xb8
scsi_attach_vpd+0x34/0x140
scsi_probe_and_add_lun+0xa6c/0xab8
__scsi_scan_target+0x438/0x4f8
scsi_scan_channel+0x6c/0xa8
scsi_scan_host_selected+0xf0/0x150
do_scsi_scan_host+0x88/0x90
scsi_scan_host+0x1b4/0x1d0
ufshcd_async_scan+0x248/0x310
async_run_entry_fn+0x30/0x178
process_one_work+0x1e8/0x368
worker_thread+0x40/0x478
kthread+0x174/0x180
ret_from_fork+0x10/0x20
Quoted 6 lines
Fixes 8d077ede48c1 ("scsi: ufs: Optimize the command queueing code")

Regards,
Bjorn

Quoted 22 lines
---
 drivers/scsi/ufs/ufshcd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd.c +2 -1
index 6dd517267f1b..fbebb8309ef7 100644
@@ -1082,7 +1082,8 @@ static u32 ufshcd_pending_cmds(struct ufs_hba *hba)
 	struct scsi_device *sdev;
 	u32 pending = 0;
 
-	shost_for_each_device(sdev, hba->host)
+	lockdep_assert_held(hba->host->host_lock);
+	__shost_for_each_device(sdev, hba->host)
 		pending += sbitmap_weight(&sdev->budget_map);
 
 	return pending;
Signature · 1 line
2.25.1
#2 lore raw
In reply to Bean Huo

Bean,

Quoted 4 lines

Call shost_for_each_device() with holding host->host_lock will cause a deadlock situation, which will cause the system to stall (the log as follow). Fix this issue by using __shost_for_each_device() in ufshcd_pending_cmds().

Applied to 5.17/scsi-staging, thanks!

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#3 lore raw
In reply to Bean Huo
Quoted 8 lines · Bean Huo

From: Bean Huo <[email protected]>

Call shost_for_each_device() with holding host->host_lock will cause a deadlock situation, which will cause the system to stall (the log as follow). Fix this issue by using __shost_for_each_device() in ufshcd_pending_cmds().

[...]

Applied to 5.17/scsi-queue, thanks!

[1/1] scsi: ufs: Fix deadlock issue in ufshcd_wait_for_doorbell_clr()

      https://git.kernel.org/mkp/scsi/c/99c66a8868e3
Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering