Skip to contentSkip to conversation

scsi: qla2xxx: fixes for driver unloading

PATCH Applied [email protected] 4 participants 7 messages Apr 21, 2020 – Apr 22, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v4 cover scsi: qla2xxx: fixes for driver unloading

From: Martin Wilck <[email protected]>

Hello Martin, Arun, Himanshu, all,

here is v4 of the little series I first submitted on Nov 29, 2019. I dropped the more controversial part, hoping to get the actual fix for the issue merged.

Reviews welcome.
Martin

Changes since v3:
- In patch 2, moved check from qla2x00_post_async_XYZ_work() to qla2x00_alloc_work() as suggested by Arun
- Dropped patch 3-5. With 1 and 2 applied, I haven't been able to reproduce shutdown hangs any more.

Changes since v2:
- Removed "scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped", because the first hunk is obsoleted by the (new) 1/3, and Arun suggested to use a different approach (which is now in 4/3) for the second hunk.
- Removed "scsi: qla2xxx: don't shut down firmware before closing sessions" (nak'd by Arun).
- Former 3/3 is now 1/5
- Added "scsi: qla2xxx: check UNLOADING before posting async work". This one is key for avoiding lags when qla2xxx is unloaded.
- Added revert of "scsi: qla2xxx: Fix unbound sleep in fcport delete path.", as I believe it's now obsolete.
If we ever encounter unbound sleep there again, we should rather figure out the reason than simply abort waiting.
- Added patch 4 and 5, a new attempt at avoiding mailbox and HW request queue access at low level. 4/5 was motivated by Arun's comments on my v2 series. 5/5 is obviously similar in spirit to 77ddb94a4853 ("scsi: qla2xxx: Only allow operational MBX to proceed during RESET."), but I found that the rom_cmds list contains commands that would hang when the FW is stopped, so I created a new list. Perhaps some day the two can be consolidated.

Changes since v1:
- Added patch 3 to set the UNLOADING flag before waiting for sessions to end (Roman)
- Use test_and_set_bit() for UNLOADING (Hannes)

Martin Wilck (2):
scsi: qla2xxx: set UNLOADING before waiting for session deletion
scsi: qla2xxx: check UNLOADING before posting async work

 drivers/scsi/qla2xxx/qla_os.c | 35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)
Signature · 1 line
2.26.0
#1 lore raw

PATCH v4 1/2 scsi: qla2xxx: set UNLOADING before waiting for session deletion

From: Martin Wilck <[email protected]>

The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown.

Furthermore, use atomic test_and_set_bit() to avoid the shutdown being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
 drivers/scsi/qla2xxx/qla_os.c | 32 ++++++++++++++------------------
 1 file changed, 14 insertions(+), 18 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +14 -18
index d9072ea..ce0dabb 100644
@@ -3732,6 +3732,13 @@ qla2x00_remove_one(struct pci_dev *pdev)
 	}
 	qla2x00_wait_for_hba_ready(base_vha);
 
+	/*
+	 * if UNLOADING flag is already set, then continue unload,
+	 * where it was set first.
+	 */
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
+
 	if (IS_QLA25XX(ha) || IS_QLA2031(ha) || IS_QLA27XX(ha) ||
 	    IS_QLA28XX(ha)) {
 		if (ha->flags.fw_started)
@@ -3750,15 +3757,6 @@ qla2x00_remove_one(struct pci_dev *pdev)
 
 	qla2x00_wait_for_sess_deletion(base_vha);
 
-	/*
-	 * if UNLOAD flag is already set, then continue unload,
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
-	set_bit(UNLOADING, &base_vha->dpc_flags);
-
 	qla_nvme_delete(base_vha);
 
 	dma_free_coherent(&ha->pdev->dev,
@@ -6628,13 +6626,6 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
 	struct pci_dev *pdev = ha->pdev;
 	scsi_qla_host_t *base_vha = pci_get_drvdata(ha->pdev);
 
-	/*
-	 * if UNLOAD flag is already set, then continue unload,
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
 	ql_log(ql_log_warn, base_vha, 0x015b,
 	    "Disabling adapter.\n");
 
@@ -6645,9 +6636,14 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
 		return;
 	}
 
-	qla2x00_wait_for_sess_deletion(base_vha);
+	/*
+	 * if UNLOADING flag is already set, then continue unload,
+	 * where it was set first.
+	 */
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
 
-	set_bit(UNLOADING, &base_vha->dpc_flags);
+	qla2x00_wait_for_sess_deletion(base_vha);
 
 	qla2x00_delete_all_vps(ha, base_vha);
 
Signature · 1 line
2.26.0
Reviewed-by on v4 1/2
#2 lore raw

Re: [PATCH v4 1/2] scsi: qla2xxx: set UNLOADING before waiting for session deletion

Quoted 76 lines · On 4/21/20 3:46 PM, [email protected] wrote:

From: Martin Wilck <[email protected]>

The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown.

Furthermore, use atomic test_and_set_bit() to avoid the shutdown being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
  drivers/scsi/qla2xxx/qla_os.c | 32 ++++++++++++++------------------
  1 file changed, 14 insertions(+), 18 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +9 -17
index d9072ea..ce0dabb 100644
@@ -3732,6 +3732,13 @@ qla2x00_remove_one(struct pci_dev *pdev)
  	}
  	qla2x00_wait_for_hba_ready(base_vha);
 
+	/*
+	 * if UNLOADING flag is already set, then continue unload,
+	 * where it was set first.
+	 */
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
+
  	if (IS_QLA25XX(ha) || IS_QLA2031(ha) || IS_QLA27XX(ha) ||
  	    IS_QLA28XX(ha)) {
  		if (ha->flags.fw_started)
@@ -3750,15 +3757,6 @@ qla2x00_remove_one(struct pci_dev *pdev)
 
  	qla2x00_wait_for_sess_deletion(base_vha);
 
-	/*
-	 * if UNLOAD flag is already set, then continue unload,
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
-	set_bit(UNLOADING, &base_vha->dpc_flags);
-
  	qla_nvme_delete(base_vha);
 
  	dma_free_coherent(&ha->pdev->dev,
@@ -6628,13 +6626,6 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
  	struct pci_dev *pdev = ha->pdev;
  	scsi_qla_host_t *base_vha = pci_get_drvdata(ha->pdev);
 
-	/*
-	 * if UNLOAD flag is already set, then continue unload,
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
  	ql_log(ql_log_warn, base_vha, 0x015b,
  	    "Disabling adapter.\n");
 
@@ -6645,9 +6636,14 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
  		return;
  	}
 
-	qla2x00_wait_for_sess_deletion(base_vha);
+	/*
+	 * if UNLOADING flag is already set, then continue unload,

drivers/scsi/qla2xxx/qla_os.c qla2x00_disable_board_on_pci_error(struct work_struct *work) · line 6649

Quoted 10 lines
drivers/scsi/qla2xxx/qla_os.c +5 -1
+	 * where it was set first.
+	 */
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
 
-	set_bit(UNLOADING, &base_vha->dpc_flags);
+	qla2x00_wait_for_sess_deletion(base_vha);
 
  	qla2x00_delete_all_vps(ha, base_vha);
 
Signature · 2 lines
Himanshu Madhani
Oracle Linux Engineering
#3 lore raw

PATCH v4 2/2 scsi: qla2xxx: check UNLOADING before posting async work

In reply to [email protected]

From: Martin Wilck <[email protected]>

qlt_free_session_done() tries to post async PRLO / LOGO, and waits for the completion of these async commands. If UNLOADING is set, this is doomed to timeout, because the async logout command will never complete.

The only way to avoid waiting pointlessly is to fail posting these commands in the first place if the driver is in UNLOADING state. In general, posting any command should be avoided when the driver is UNLOADING.

With this patch, "rmmod qla2xxx" completes without noticeable delay.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
 drivers/scsi/qla2xxx/qla_os.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/scsi/qla2xxx/qla_os.c +3 -0
index ce0dabb..8cce3e2 100644
@@ -4862,6 +4862,9 @@ qla2x00_alloc_work(struct scsi_qla_host *vha, enum qla_work_type type)
 	struct qla_work_evt *e;
 	uint8_t bail;
 
+	if (test_bit(UNLOADING, &vha->dpc_flags))
+		return NULL;
+
 	QLA_VHA_MARK_BUSY(vha, bail);
 	if (bail)
 		return NULL;
Signature · 1 line
2.26.0
Reviewed-by on v4 2/2
#4 lore raw

Re: [PATCH v4 2/2] scsi: qla2xxx: check UNLOADING before posting async work

Quoted 26 lines · On 4/21/20 3:46 PM, [email protected] wrote:

From: Martin Wilck <[email protected]>

qlt_free_session_done() tries to post async PRLO / LOGO, and waits for the completion of these async commands. If UNLOADING is set, this is doomed to timeout, because the async logout command will never complete.

The only way to avoid waiting pointlessly is to fail posting these commands in the first place if the driver is in UNLOADING state. In general, posting any command should be avoided when the driver is UNLOADING.

With this patch, "rmmod qla2xxx" completes without noticeable delay.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
  drivers/scsi/qla2xxx/qla_os.c | 3 +++
  1 file changed, 3 insertions(+)
drivers/scsi/qla2xxx/qla_os.c +0 -0
index ce0dabb..8cce3e2 100644

drivers/scsi/qla2xxx/qla_os.c qla2x00_alloc_work(struct scsi_qla_host *vha, enum qla_work_type type) · line 4870

Quoted 9 lines
drivers/scsi/qla2xxx/qla_os.c +3 -0
@@ -4862,6 +4862,9 @@ qla2x00_alloc_work(struct scsi_qla_host *vha, enum qla_work_type type)
  	struct qla_work_evt *e;
  	uint8_t bail;
 
+	if (test_bit(UNLOADING, &vha->dpc_flags))
+		return NULL;
+
  	QLA_VHA_MARK_BUSY(vha, bail);
  	if (bail)
  		return NULL;
Signature · 2 lines
Himanshu Madhani
Oracle Linux Engineering
#5 lore raw
In reply to [email protected]

Series look good.

Thank you for the patches and testing, Martin.

Regards,
-Arun

Quoted 54 lines · On Tue, 21 Apr 2020, 1:46pm, [email protected] wrote:

External Email

----------------------------------------------------------------------
From: Martin Wilck <[email protected]>

Hello Martin, Arun, Himanshu, all,

here is v4 of the little series I first submitted on Nov 29, 2019. I dropped the more controversial part, hoping to get the actual fix for the issue merged.

Reviews welcome.
Martin

Changes since v3:
- In patch 2, moved check from qla2x00_post_async_XYZ_work() to qla2x00_alloc_work() as suggested by Arun
- Dropped patch 3-5. With 1 and 2 applied, I haven't been able to reproduce shutdown hangs any more.

Changes since v2:
- Removed "scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped", because the first hunk is obsoleted by the (new) 1/3, and Arun suggested to use a different approach (which is now in 4/3) for the second hunk.
- Removed "scsi: qla2xxx: don't shut down firmware before closing sessions" (nak'd by Arun).
- Former 3/3 is now 1/5
- Added "scsi: qla2xxx: check UNLOADING before posting async work". This one is key for avoiding lags when qla2xxx is unloaded.
- Added revert of "scsi: qla2xxx: Fix unbound sleep in fcport delete path.", as I believe it's now obsolete.
If we ever encounter unbound sleep there again, we should rather figure out the reason than simply abort waiting.
- Added patch 4 and 5, a new attempt at avoiding mailbox and HW request queue access at low level. 4/5 was motivated by Arun's comments on my v2 series. 5/5 is obviously similar in spirit to 77ddb94a4853 ("scsi: qla2xxx: Only allow operational MBX to proceed during RESET."), but I found that the rom_cmds list contains commands that would hang when the FW is stopped, so I created a new list. Perhaps some day the two can be consolidated.

Changes since v1:
- Added patch 3 to set the UNLOADING flag before waiting for sessions to end (Roman)
- Use test_and_set_bit() for UNLOADING (Hannes)

Martin Wilck (2):
scsi: qla2xxx: set UNLOADING before waiting for session deletion
scsi: qla2xxx: check UNLOADING before posting async work

 drivers/scsi/qla2xxx/qla_os.c | 35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)
#6 lore raw
In reply to [email protected]

Martin,

Quoted 3 lines

here is v4 of the little series I first submitted on Nov 29, 2019. I dropped the more controversial part, hoping to get the actual fix for the issue merged.

Applied to 5.7/scsi-fixes, thanks!

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering