Skip to contentSkip to conversation

scsi: qla4xxx: fix potential deadlock on frwd/back lock

PATCH In review Chengfeng Ye 2 participants 3 messages Sep 28, 2023 – Sep 29, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 scsi: qla4xxx: fix potential deadlock on frwd/back lock

iscsi_complete_pdu() is called by several rx callback, under softirq context. Thus the callsite of it inside qla4xxx_task_work() should better disable bottom half, as work queue is executed under process context, it needs to protect race with softirq context locking to avoid deadlock.

<potential deadlock #1>
qla4xxx_task_work()
--> iscsi_complete_pdu()
--> spin_lock(&conn->session->back_lock); <interrupt>
--> iscsi_tcp_data_recv_done()
--> iscsi_complete_pdu()
--> spin_lock(&conn->session->back_lock) (deadlock)

<potential deadlock #2>
qla4xxx_task_work()
--> iscsi_complete_pdu()
--> __iscsi_complete_pdu()
--> spin_lock(&conn->session->frwd_lock) <interrupt>
--> iscsi_tcp_data_recv_done()
--> iscsi_complete_pdu()
--> __iscsi_complete_pdu()
--> spin_lock(&conn->session->frwd_lock) (deadlock)

This flaw was found by an experimental static analysis tool I am developing for irq-related deadlock.

To avoid the problem, disable bh inside qla4xxx_task_work() before calling iscsi_complete_pdu().

---
 drivers/scsi/qla4xxx/ql4_os.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/scsi/qla4xxx/ql4_os.c +2 -0
index 675332e49a7b..c60781148e6c 100644
@@ -3382,7 +3382,9 @@ static void qla4xxx_task_work(struct work_struct *wdata)
 		hdr->itt = itt;
 		data = task_data->resp_buffer + hdr_len;
 		data_len = task_data->resp_len - hdr_len;
+		local_bh_disable();
 		iscsi_complete_pdu(conn, hdr, data, data_len);
+		local_bh_enable();
 		break;
 	default:
 		ql4_printk(KERN_ERR, ha, "Passthru failed status = 0x%x\n",
Signature · 1 line
2.17.1
#1 lore raw
Quoted 6 lines · Chengfeng Ye

iscsi_complete_pdu() is called by several rx callback, under softirq context. Thus the callsite of it inside qla4xxx_task_work() should better disable bottom half, as work queue is executed under process context, it needs to protect race with softirq context locking to avoid deadlock.

Quoted 9 lines

<potential deadlock #1>
qla4xxx_task_work()
--> iscsi_complete_pdu()
--> spin_lock(&conn->session->back_lock); <interrupt>
--> iscsi_tcp_data_recv_done()
--> iscsi_complete_pdu()
--> spin_lock(&conn->session->back_lock) (deadlock)

This will not happen, because qla4xxx does not use the libiscsi_tcp code. If you had a iscsi session with qla4xxx and another iscsi session with iscsi_tcp, then you could get an interrupt like this, but the "conn->session->back_lock" that's taken above would be for the iscsi_tcp session.

So you would need to make sure that the above can't happen for iscsi_tcp an iscsi_tcp session. Same with the below analysis.

Quoted 36 lines

<potential deadlock #2>
qla4xxx_task_work()
--> iscsi_complete_pdu()
--> __iscsi_complete_pdu()
--> spin_lock(&conn->session->frwd_lock) <interrupt>
--> iscsi_tcp_data_recv_done()
--> iscsi_complete_pdu()
--> __iscsi_complete_pdu()
--> spin_lock(&conn->session->frwd_lock) (deadlock)

This flaw was found by an experimental static analysis tool I am developing for irq-related deadlock.

To avoid the problem, disable bh inside qla4xxx_task_work() before calling iscsi_complete_pdu().

---
 drivers/scsi/qla4xxx/ql4_os.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/scsi/qla4xxx/ql4_os.c +2 -0
index 675332e49a7b..c60781148e6c 100644
@@ -3382,7 +3382,9 @@ static void qla4xxx_task_work(struct work_struct *wdata)
 		hdr->itt = itt;
 		data = task_data->resp_buffer + hdr_len;
 		data_len = task_data->resp_len - hdr_len;
+		local_bh_disable();
 		iscsi_complete_pdu(conn, hdr, data, data_len);
+		local_bh_enable();
 		break;
 	default:
 		ql4_printk(KERN_ERR, ha, "Passthru failed status = 0x%x\n",
#2 lore raw

Thanks for the explanation!

I thought the two locks are the same so made this bogus patch, sorry for the confusion and thanks much for your time.

Regards,
Chengfeng