Skip to contentSkip to conversation

scsi: qla2xxx: fixes for driver unloading

PATCH Superseded [email protected] 5 participants 24 messages Feb 5, 2020 – Mar 25, 2020

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover scsi: qla2xxx: fixes for driver unloading

From: Martin Wilck <[email protected]>

Hello Martin, Himanshu, all,

here is v2 of the little series I first submitted on Nov 29, 2019.

Changes wrt v2:
- Added patch 3 to set the UNLOADING flag before waiting for sessions to end (Roman)
- Use test_and_set_bit() for UNLOADING (Hannes)

Martin Wilck (3):
scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped
scsi: qla2xxx: don't shut down firmware before closing sessions
scsi: qla2xxx: set UNLOADING before waiting for session deletion

 drivers/scsi/qla2xxx/qla_mbx.c |  3 +++
 drivers/scsi/qla2xxx/qla_os.c  | 39 +++++++++++++++++-----------------
 2 files changed, 22 insertions(+), 20 deletions(-)
Signature · 1 line
2.25.0
#1 lore raw

PATCH v2 1/3 scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

From: Martin Wilck <[email protected]>

Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"), it is possible that FC commands are scheduled after the adapter firmware has been shut down. IO sent to the firmware in this situation hangs indefinitely. Avoid this for the LOGO code path that is typically taken when adapters are shut down.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
 drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
 drivers/scsi/qla2xxx/qla_os.c  | 3 +++
 2 files changed, 6 insertions(+)
drivers/scsi/qla2xxx/qla_mbx.c +3 -0
index 9e09964..53129f2 100644
@@ -2644,6 +2644,9 @@ qla24xx_fabric_logout(scsi_qla_host_t *vha, uint16_t loop_id, uint8_t domain,
 	ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x106d,
 	    "Entered %s.\n", __func__);
 
+	if (!ha->flags.fw_started)
+		return QLA_FUNCTION_FAILED;
+
 	lg = dma_pool_zalloc(ha->s_dma_pool, GFP_KERNEL, &lg_dma);
 	if (lg == NULL) {
 		ql_log(ql_log_warn, vha, 0x106e,
drivers/scsi/qla2xxx/qla_os.c +3 -0
index b520a98..2dafb46 100644
@@ -4878,6 +4878,9 @@ qla2x00_post_work(struct scsi_qla_host *vha, struct qla_work_evt *e)
 	unsigned long flags;
 	bool q = false;
 
+	if (!vha->hw->flags.fw_started)
+		return QLA_FUNCTION_FAILED;
+
 	spin_lock_irqsave(&vha->work_lock, flags);
 	list_add_tail(&e->list, &vha->work_list);
 
Signature · 1 line
2.25.0
#2 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Hi Martin,

Quoted 11 lines · On Wed, Feb 05, 2020 at 10:44:20PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"), it is possible that FC commands are scheduled after the adapter firmware has been shut down. IO sent to the firmware in this situation hangs indefinitely. Avoid this for the LOGO code path that is typically taken when adapters are shut down.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")

Just to understand it correctly: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip") is saying FW is not able to shutdown propably and therefore we kill it first and still try to do some cleanup.

Are you sure you got all the necessary places fixed up?

I tend to say we should add

	if (!ha->flags.fw_started)
		return QLA_FUNCTION_FAILED;

do qla2x00_mailbox_command() and handle the errors one by one. Just an idea.

Thanks,
Daniel

#3 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 19 lines · Daniel Wagner

Hi Martin,

Quoted 11 lines · On Wed, Feb 05, 2020 at 10:44:20PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"),
it is possible that FC commands are scheduled after the adapter firmware
has been shut down. IO sent to the firmware in this situation hangs indefinitely. Avoid this for the LOGO code path that is typically taken
when adapters are shut down.

Quoted 4 lines
Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting

down chip")

Quoted 4 lines

Just to understand it correctly: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip") is saying FW is not able to shutdown propably and therefore we kill it first and still try to do some cleanup.

Yes, that's what I observed. Mailbox access hangs in this case, as the mailbox simply won't execute the command and the expected status change will not happen.

Quoted 10 lines

Are you sure you got all the necessary places fixed up?

I tend to say we should add

	if (!ha->flags.fw_started)
		return QLA_FUNCTION_FAILED;

do qla2x00_mailbox_command() and handle the errors one by one. Just an
idea.

I had that idea too. I even tried it out. IIRC it's not that easy. Some commands need to be sent to the mailbox even in shut-down state (MBC_EXECUTE_FIRMWARE, for example?). I admit I didn't dare going down the path you're suggesting, I thought it had quite a potential to cause regressions.

Did you mean this as a negative review, or rather an additional suggestion?

Thanks
Martin

Reviewed-by
#4 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 12 lines · Martin Wilck
Quoted 10 lines · Daniel Wagner

Are you sure you got all the necessary places fixed up?

I tend to say we should add

	if (!ha->flags.fw_started)
		return QLA_FUNCTION_FAILED;

do qla2x00_mailbox_command() and handle the errors one by one. Just an
idea.

Quoted 5 lines

I had that idea too. I even tried it out. IIRC it's not that easy. Some commands need to be sent to the mailbox even in shut-down state (MBC_EXECUTE_FIRMWARE, for example?). I admit I didn't dare going down the path you're suggesting, I thought it had quite a potential to cause regressions.

Yes, this certainly causing regressions. This patch is using the blacklist approach which is probably introducing less regression.

But I think for finding all the right spots the white listing approach is better. Maybe only during development phase. Just an idea.

Quoted 2 lines

Did you mean this as a negative review, or rather an additional suggestion?

This patch clearly improves things. So yes, I approve.

Thanks,
Daniel

#5 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

In reply to [email protected]
Quoted 21 lines · On Wed, 5 Feb 2020, 1:44pm, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"), it is possible that FC commands are scheduled after the adapter firmware has been shut down. IO sent to the firmware in this situation hangs indefinitely. Avoid this for the LOGO code path that is typically taken when adapters are shut down.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
 drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
 drivers/scsi/qla2xxx/qla_os.c  | 3 +++
 2 files changed, 6 insertions(+)
drivers/scsi/qla2xxx/qla_mbx.c +0 -0
index 9e09964..53129f2 100644

drivers/scsi/qla2xxx/qla_mbx.c qla24xx_fabric_logout(scsi_qla_host_t *vha, uint16_t loop_id, uint8_t domain, · line 2649

Quoted 6 lines
drivers/scsi/qla2xxx/qla_mbx.c +3 -0
@@ -2644,6 +2644,9 @@ qla24xx_fabric_logout(scsi_qla_host_t *vha, uint16_t loop_id, uint8_t domain,
 	ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x106d,
 	    "Entered %s.\n", __func__);
 
+	if (!ha->flags.fw_started)
+		return QLA_FUNCTION_FAILED;
+

Ok.

Quoted 8 lines
drivers/scsi/qla2xxx/qla_mbx.c +0 -0
 	lg = dma_pool_zalloc(ha->s_dma_pool, GFP_KERNEL, &lg_dma);
 	if (lg == NULL) {
 		ql_log(ql_log_warn, vha, 0x106e,
drivers/scsi/qla2xxx/qla_os.c +0 -0
index b520a98..2dafb46 100644

drivers/scsi/qla2xxx/qla_os.c qla2x00_post_work(struct scsi_qla_host *vha, struct qla_work_evt *e) · line 4883

Quoted 6 lines
drivers/scsi/qla2xxx/qla_os.c +3 -0
@@ -4878,6 +4878,9 @@ qla2x00_post_work(struct scsi_qla_host *vha, struct qla_work_evt *e)
 	unsigned long flags;
 	bool q = false;
 
+	if (!vha->hw->flags.fw_started)
+		return QLA_FUNCTION_FAILED;
+

I'd probably not do it here; rather in qla2x00_get_sp() /qla2x00_start_sp()/ qla2xxx_get_qpair_sp() time. Not all works are for posting to firmware (QLA_EVT_IDC_ACK, QLA_EVT_UNMAP etc.).

Regards,
-Arun

Quoted 4 lines
drivers/scsi/qla2xxx/qla_os.c +0 -0
 	spin_lock_irqsave(&vha->work_lock, flags);
 	list_add_tail(&e->list, &vha->work_list);
 
#6 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 30 lines · Arun Easi
Quoted 28 lines · On Wed, 5 Feb 2020, 1:44pm, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"),
it is possible that FC commands are scheduled after the adapter firmware
has been shut down. IO sent to the firmware in this situation hangs indefinitely. Avoid this for the LOGO code path that is typically taken
when adapters are shut down.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting

down chip")

---
 drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
 drivers/scsi/qla2xxx/qla_os.c  | 3 +++
 2 files changed, 6 insertions(+)
drivers/scsi/qla2xxx/qla_os.c +0 -0
@@ -4878,6 +4878,9 @@ qla2x00_post_work(struct scsi_qla_host *vha,
 struct qla_work_evt *e)
 	unsigned long flags;
 	bool q = false;
 
Quoted 8 lines

drivers/scsi/qla2xxx/qla_os.c qla2x00_post_work(struct scsi_qla_host *vha, struct qla_work_evt *e) · line 4883

Quoted 3 lines
drivers/scsi/qla2xxx/qla_os.c +3 -0
+	if (!vha->hw->flags.fw_started)
+		return QLA_FUNCTION_FAILED;
+

I'd probably not do it here; rather in qla2x00_get_sp() /qla2x00_start_sp()/ qla2xxx_get_qpair_sp() time. Not all works are for
posting to firmware (QLA_EVT_IDC_ACK, QLA_EVT_UNMAP etc.).

qla81xx_idc_ack() calls qla2x00_mailbox_command(), which should be avoided IMO. But I'll review the various cases and re-post the patch.

Thanks,
Martin

Quoted 8 lines

Regards,
-Arun

Quoted 4 lines
drivers/scsi/qla2xxx/qla_os.c +0 -0
 	spin_lock_irqsave(&vha->work_lock, flags);
 	list_add_tail(&e->list, &vha->work_list);
#7 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 43 lines · Martin Wilck
Quoted 33 lines · Arun Easi
Quoted 31 lines · On Wed, 5 Feb 2020, 1:44pm, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"),
it is possible that FC commands are scheduled after the adapter firmware
has been shut down. IO sent to the firmware in this situation hangs
indefinitely. Avoid this for the LOGO code path that is typically taken
when adapters are shut down.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by

shutting
down chip")

---
 drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
 drivers/scsi/qla2xxx/qla_os.c  | 3 +++
 2 files changed, 6 insertions(+)

[...]

drivers/scsi/qla2xxx/qla_os.c +0 -0
@@ -4878,6 +4878,9 @@ qla2x00_post_work(struct scsi_qla_host
 *vha,
 struct qla_work_evt *e)
 	unsigned long flags;
 	bool q = false;
 
Quoted 8 lines

drivers/scsi/qla2xxx/qla_os.c qla2x00_post_work(struct scsi_qla_host *vha, struct qla_work_evt *e) · line 4883

Quoted 3 lines
drivers/scsi/qla2xxx/qla_os.c +3 -0
+	if (!vha->hw->flags.fw_started)
+		return QLA_FUNCTION_FAILED;
+

I'd probably not do it here; rather in qla2x00_get_sp() /qla2x00_start_sp()/ qla2xxx_get_qpair_sp() time. Not all works are for
posting to firmware (QLA_EVT_IDC_ACK, QLA_EVT_UNMAP etc.).

Quoted 2 lines

qla81xx_idc_ack() calls qla2x00_mailbox_command(), which should be avoided IMO. But I'll review the various cases and re-post the patch.

Thinking about it once more, the approach is racy.

qla2x00_try_stop_firmware() can be called any time, and it sets fw_started = 0 *after* actually stopping the firmware. Even if we check fw_started, the firwmare might be stopped between our check and our actual mailbox / FW register access, and we'd end up hanging.

At least the check should be as close as possible to the actual FW access, e.g. in qla2x00_mailbox_command(), or before writing to the request queue registers in qla2x00_start_sp() etc.

Perhaps the (!fw_started) condition should be treated like ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if is_rom_cmd() returns true?

I can re-post, but I feel this should really be done by someone who knows exactly how the firmware operates, IOW Marvell staff.

Regards
Martin

#8 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 3 lines · Martin Wilck

Perhaps the (!fw_started) condition should be treated like ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if is_rom_cmd() returns true?

No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in rom_cmds[], but has been found to hang (this is why I had the hunk in qla24xx_fabric_logout()). The list of mailbox commands that must be passed on when the FW is stopped has to be shorter than rom_cmds[].

Better suggestions welcome.

Martin

#9 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 11 lines · Martin Wilck
Quoted 3 lines · Martin Wilck

Perhaps the (!fw_started) condition should be treated like ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if is_rom_cmd() returns true?

No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in rom_cmds[], but has been found to hang (this is why I had the hunk in qla24xx_fabric_logout()). The list of mailbox commands that must be passed on when the FW is stopped has to be shorter than rom_cmds[].

With your patch 2/3, where UNLOADING is set prior to the reset of chip, in place this issue should be largely addressed. Basically, paths that send out request to wire check UNLOADING bit (if any path is missing that, should add the check) before sending it out.

Now with UNLOADING set (with your patch 2/3), chip reset, and all outstanding command's completion called (qla2x00_abort_isp_cleanup) I see less chance of anything being sent out. If you see any issue with your patches 1 & 2 (addressing my comments) applied, let me know and we can tackle then. How about that?

Regards,
-Arun

#10 lore raw

Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 24 lines · Arun Easi
Quoted 5 lines · Martin Wilck
Quoted 3 lines · Martin Wilck

Perhaps the (!fw_started) condition should be treated like ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if is_rom_cmd() returns true?

Quoted 8 lines

No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in rom_cmds[], but has been found to hang (this is why I had the hunk in
qla24xx_fabric_logout()). The list of mailbox commands that must be passed on when the FW is stopped has to be shorter than
rom_cmds[].

With your patch 2/3, where UNLOADING is set prior to the reset of chip, in
place this issue should be largely addressed. Basically, paths that send
out request to wire check UNLOADING bit (if any path is missing that,
should add the check) before sending it out.

Quoted 8 lines

Now with UNLOADING set (with your patch 2/3), chip reset, and all outstanding command's completion called (qla2x00_abort_isp_cleanup) I see
less chance of anything being sent out. If you see any issue with your
patches 1 & 2 (addressing my comments) applied, let me know and we can
tackle then. How about that?

It sounds like a plan. Although it means that I just wasted time trying to figure out which mailbox commands need to be processed even if the firmware is down :-)

Thanks,
Martin

#11 lore raw

Re: [EXT] Re: [PATCH v2 1/3] scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped

Quoted 30 lines · Martin Wilck

----------------------------------------------------------------------

Quoted 21 lines · Arun Easi
Quoted 5 lines · Martin Wilck
Quoted 3 lines · Martin Wilck

Perhaps the (!fw_started) condition should be treated like ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if is_rom_cmd() returns true?

Quoted 8 lines

No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in rom_cmds[], but has been found to hang (this is why I had the hunk in
qla24xx_fabric_logout()). The list of mailbox commands that must be passed on when the FW is stopped has to be shorter than
rom_cmds[].

With your patch 2/3, where UNLOADING is set prior to the reset of chip, in place this issue should be largely addressed. Basically, paths that send out request to wire check UNLOADING bit (if any path is missing that, should add the check) before sending it out.

Quoted 5 lines

Now with UNLOADING set (with your patch 2/3), chip reset, and all outstanding command's completion called (qla2x00_abort_isp_cleanup) I see less chance of anything being sent out. If you see any issue with your patches 1 & 2 (addressing my comments) applied, let me know and we can tackle then. How about that?

Quoted 4 lines

It sounds like a plan. Although it means that I just wasted time trying to figure out which mailbox commands need to be processed even if the firmware is down :-)

I hope it was not too much time. On the plus side, you know the mailbox path very well now. :)

Regards,
-Arun

#12 lore raw

PATCH v2 2/3 scsi: qla2xxx: don't shut down firmware before closing sessions

In reply to [email protected]

From: Martin Wilck <[email protected]>

Since 45235022da99, the firmware is shut down early in the controller shutdown process. This causes commands sent to the firmware (such as LOGO) to hang forever. Eventually one or more timeouts will be triggered. Move the stopping of the firmware until after sessions have terminated.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
 drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +10 -11
index 2dafb46..e81080d 100644
@@ -3720,6 +3720,16 @@ qla2x00_remove_one(struct pci_dev *pdev)
 	}
 	qla2x00_wait_for_hba_ready(base_vha);
 
+	qla2x00_wait_for_sess_deletion(base_vha);
+
+	/*
+	 * if UNLOAD flag is already set, then continue unload,
+	 * where it was set first.
+	 */
+	if (test_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
+
+	set_bit(UNLOADING, &base_vha->dpc_flags);
 	if (IS_QLA25XX(ha) || IS_QLA2031(ha) || IS_QLA27XX(ha) ||
 	    IS_QLA28XX(ha)) {
 		if (ha->flags.fw_started)
@@ -3736,17 +3746,6 @@ qla2x00_remove_one(struct pci_dev *pdev)
 		qla2x00_try_to_stop_firmware(base_vha);
 	}
 
-	qla2x00_wait_for_sess_deletion(base_vha);
-
-	/*
-	 * if UNLOAD flag is already set, then continue unload,
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
-	set_bit(UNLOADING, &base_vha->dpc_flags);
-
 	qla_nvme_delete(base_vha);
 
 	dma_free_coherent(&ha->pdev->dev,
Signature · 1 line
2.25.0
Reviewed-by on v2 2/3
#13 lore raw

Re: [PATCH v2 2/3] scsi: qla2xxx: don't shut down firmware before closing sessions

Quoted 9 lines · On Wed, Feb 05, 2020 at 10:44:21PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since 45235022da99, the firmware is shut down early in the controller shutdown process. This causes commands sent to the firmware (such as LOGO) to hang forever. Eventually one or more timeouts will be triggered. Move the stopping of the firmware until after sessions have terminated.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Reviewed-by on v2 2/3
#14 lore raw

Re: [PATCH v2 2/3] scsi: qla2xxx: don't shut down firmware before closing sessions

In reply to [email protected]
Quoted 7 lines · On Wed, Feb 05, 2020 at 10:44:21PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since 45235022da99, the firmware is shut down early in the controller shutdown process. This causes commands sent to the firmware (such as LOGO) to hang forever. Eventually one or more timeouts will be triggered. Move the stopping of the firmware until after sessions have terminated.

Quoted 6 lines
Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
 drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)

Hi Martin,

BTW, I think qla_nvme_delete() should be moved up to split the function into two parts:
* session/port cleanup
* chip shutdown

Although invocation of the function after chip shutdown has no functional implications at the moment.

Best regards,
Roman

#15 lore raw

Re: [PATCH v2 2/3] scsi: qla2xxx: don't shut down firmware before closing sessions

In reply to [email protected]
Quoted 43 lines · On Wed, 5 Feb 2020, 1:44pm, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since 45235022da99, the firmware is shut down early in the controller shutdown process. This causes commands sent to the firmware (such as LOGO) to hang forever. Eventually one or more timeouts will be triggered. Move the stopping of the firmware until after sessions have terminated.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
---
 drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +10 -4
index 2dafb46..e81080d 100644
@@ -3720,6 +3720,16 @@ qla2x00_remove_one(struct pci_dev *pdev)
 	}
 	qla2x00_wait_for_hba_ready(base_vha);
 
+	qla2x00_wait_for_sess_deletion(base_vha);
+
+	/*
+	 * if UNLOAD flag is already set, then continue unload,
+	 * where it was set first.
+	 */
+	if (test_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
+
+	set_bit(UNLOADING, &base_vha->dpc_flags);
 	if (IS_QLA25XX(ha) || IS_QLA2031(ha) || IS_QLA27XX(ha) ||
 	    IS_QLA28XX(ha)) {
 		if (ha->flags.fw_started)
@@ -3736,17 +3746,6 @@ qla2x00_remove_one(struct pci_dev *pdev)
 		qla2x00_try_to_stop_firmware(base_vha);
 	}
 
-	qla2x00_wait_for_sess_deletion(base_vha);
-
-	/*
-	 * if UNLOAD flag is already set, then continue unload,

drivers/scsi/qla2xxx/qla_os.c qla2x00_remove_one(struct pci_dev *pdev) · line 3751

Quoted 10 lines
drivers/scsi/qla2xxx/qla_os.c +0 -7
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
-	set_bit(UNLOADING, &base_vha->dpc_flags);
-
 	qla_nvme_delete(base_vha);
 
 	dma_free_coherent(&ha->pdev->dev,

NAK.

The fcport deletion was done after chip reset to minimize interference and further action on fcports. We should not be sending out logouts during unload (driver just implicitly logs out). If you experience any hangs, please let us know.

Regards,
-Arun

#16 lore raw

Re: [PATCH v2 2/3] scsi: qla2xxx: don't shut down firmware before closing sessions

Quoted 20 lines · Arun Easi
Quoted 18 lines · On Wed, 5 Feb 2020, 1:44pm, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since 45235022da99, the firmware is shut down early in the controller
shutdown process. This causes commands sent to the firmware (such as LOGO)
to hang forever. Eventually one or more timeouts will be triggered. Move the stopping of the firmware until after sessions have terminated.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting

down chip")

---
 drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)

...

Quoted 9 lines

NAK.

The fcport deletion was done after chip reset to minimize interference and
further action on fcports. We should not be sending out logouts during
unload (driver just implicitly logs out). If you experience any hangs,
please let us know.

What about target mode? AFAIK target ports need to send explicit LOGO.

Regards
Martin

#17 lore raw

Re: [PATCH v2 2/3] scsi: qla2xxx: don't shut down firmware before closing sessions

Quoted 24 lines · Martin Wilck
Quoted 20 lines · Arun Easi
Quoted 18 lines · On Wed, 5 Feb 2020, 1:44pm, [email protected] wrote:

From: Martin Wilck <[email protected]>

Since 45235022da99, the firmware is shut down early in the controller
shutdown process. This causes commands sent to the firmware (such as LOGO)
to hang forever. Eventually one or more timeouts will be triggered. Move the stopping of the firmware until after sessions have terminated.

Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting

down chip")

---
 drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)

...

Quoted 9 lines
Quoted 6 lines

NAK.

The fcport deletion was done after chip reset to minimize interference and further action on fcports. We should not be sending out logouts during unload (driver just implicitly logs out). If you experience any hangs, please let us know.

What about target mode? AFAIK target ports need to send explicit LOGO.

I do not see a hard requirement cited in spec, correct me if you see otherwise. Other initiators should see a RSCN and see this device has gone away.

Regards,
-Arun

#18 lore raw

PATCH v2 3/3 scsi: qla2xxx: set UNLOADING before waiting for session deletion

In reply to [email protected]

From: Martin Wilck <[email protected]>

The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown. The only operations that must be able to continue are LOGO, PRLO, and the like.

Furthermore, use atomic test_and_set_bit() to avoid the shutdown being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.

---
 drivers/scsi/qla2xxx/qla_os.c | 25 +++++++++++--------------
 1 file changed, 11 insertions(+), 14 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +11 -14
index e81080d..8329f95 100644
@@ -3720,16 +3720,15 @@ qla2x00_remove_one(struct pci_dev *pdev)
 	}
 	qla2x00_wait_for_hba_ready(base_vha);
 
-	qla2x00_wait_for_sess_deletion(base_vha);
-
 	/*
-	 * if UNLOAD flag is already set, then continue unload,
+	 * if UNLOADING flag is already set, then continue unload,
 	 * where it was set first.
 	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
 		return;
 
-	set_bit(UNLOADING, &base_vha->dpc_flags);
+	qla2x00_wait_for_sess_deletion(base_vha);
+
 	if (IS_QLA25XX(ha) || IS_QLA2031(ha) || IS_QLA27XX(ha) ||
 	    IS_QLA28XX(ha)) {
 		if (ha->flags.fw_started)
@@ -6046,13 +6045,6 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
 	struct pci_dev *pdev = ha->pdev;
 	scsi_qla_host_t *base_vha = pci_get_drvdata(ha->pdev);
 
-	/*
-	 * if UNLOAD flag is already set, then continue unload,
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
 	ql_log(ql_log_warn, base_vha, 0x015b,
 	    "Disabling adapter.\n");
 
@@ -6063,9 +6055,14 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
 		return;
 	}
 
-	qla2x00_wait_for_sess_deletion(base_vha);
+	/*
+	 * if UNLOADING flag is already set, then continue unload,
+	 * where it was set first.
+	 */
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
 
-	set_bit(UNLOADING, &base_vha->dpc_flags);
+	qla2x00_wait_for_sess_deletion(base_vha);
 
 	qla2x00_delete_all_vps(ha, base_vha);
 
Signature · 1 line
2.25.0
Reviewed-by on v2 3/3
#19 lore raw

Re: [PATCH v2 3/3] scsi: qla2xxx: set UNLOADING before waiting for session deletion

Quoted 8 lines · On Wed, Feb 05, 2020 at 10:44:22PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown. The only operations that must be able to continue are LOGO, PRLO, and the like.

Quoted 6 lines

Furthermore, use atomic test_and_set_bit() to avoid the shutdown being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.

Reviewed-by on v2 3/3
#20 lore raw

Re: [PATCH v2 3/3] scsi: qla2xxx: set UNLOADING before waiting for session deletion

In reply to [email protected]
Quoted 13 lines · On Wed, Feb 05, 2020 at 10:44:22PM +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown. The only operations that must be able to continue are LOGO, PRLO, and the like.

Furthermore, use atomic test_and_set_bit() to avoid the shutdown being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.

Quoted 5 lines
---
 drivers/scsi/qla2xxx/qla_os.c | 25 +++++++++++--------------
 1 file changed, 11 insertions(+), 14 deletions(-)

Hi Martin,

Thanks,
Roman

#21 lore raw

Re: [PATCH v2 3/3] scsi: qla2xxx: set UNLOADING before waiting for session deletion

In reply to [email protected]
Quoted 65 lines · On Wed, 5 Feb 2020, 1:44pm, [email protected] wrote:

From: Martin Wilck <[email protected]>

The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown. The only operations that must be able to continue are LOGO, PRLO, and the like.

Furthermore, use atomic test_and_set_bit() to avoid the shutdown being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.

---
 drivers/scsi/qla2xxx/qla_os.c | 25 +++++++++++--------------
 1 file changed, 11 insertions(+), 14 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +6 -13
index e81080d..8329f95 100644
@@ -3720,16 +3720,15 @@ qla2x00_remove_one(struct pci_dev *pdev)
 	}
 	qla2x00_wait_for_hba_ready(base_vha);
 
-	qla2x00_wait_for_sess_deletion(base_vha);
-
 	/*
-	 * if UNLOAD flag is already set, then continue unload,
+	 * if UNLOADING flag is already set, then continue unload,
 	 * where it was set first.
 	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
 		return;
 
-	set_bit(UNLOADING, &base_vha->dpc_flags);
+	qla2x00_wait_for_sess_deletion(base_vha);
+
 	if (IS_QLA25XX(ha) || IS_QLA2031(ha) || IS_QLA27XX(ha) ||
 	    IS_QLA28XX(ha)) {
 		if (ha->flags.fw_started)
@@ -6046,13 +6045,6 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
 	struct pci_dev *pdev = ha->pdev;
 	scsi_qla_host_t *base_vha = pci_get_drvdata(ha->pdev);
 
-	/*
-	 * if UNLOAD flag is already set, then continue unload,
-	 * where it was set first.
-	 */
-	if (test_bit(UNLOADING, &base_vha->dpc_flags))
-		return;
-
 	ql_log(ql_log_warn, base_vha, 0x015b,
 	    "Disabling adapter.\n");
 
@@ -6063,9 +6055,14 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
 		return;
 	}
 
-	qla2x00_wait_for_sess_deletion(base_vha);
+	/*
+	 * if UNLOADING flag is already set, then continue unload,

drivers/scsi/qla2xxx/qla_os.c qla2x00_disable_board_on_pci_error(struct work_struct *work) · line 6068

Quoted 10 lines
drivers/scsi/qla2xxx/qla_os.c +5 -1
+	 * where it was set first.
+	 */
+	if (test_and_set_bit(UNLOADING, &base_vha->dpc_flags))
+		return;
 
-	set_bit(UNLOADING, &base_vha->dpc_flags);
+	qla2x00_wait_for_sess_deletion(base_vha);
 
 	qla2x00_delete_all_vps(ha, base_vha);
 

This was done on top of PATCH 2/3, please resend dropping 2. The test_and_set_bit() part looks ok.

Regards,
-Arun

#22 lore raw
In reply to [email protected]

Hello Martin, Himanshu,

Quoted 18 lines · On Wed, 2020-02-05 at 22:44 +0100, [email protected] wrote:

From: Martin Wilck <[email protected]>

Hello Martin, Himanshu, all,

here is v2 of the little series I first submitted on Nov 29, 2019.

Changes wrt v2:
- Added patch 3 to set the UNLOADING flag before waiting for
sessions
to end (Roman)
- Use test_and_set_bit() for UNLOADING (Hannes)

Martin Wilck (3):
scsi: qla2xxx: avoid sending mailbox commands if firmware is
stopped
scsi: qla2xxx: don't shut down firmware before closing sessions
scsi: qla2xxx: set UNLOADING before waiting for session deletion

Quoted 5 lines
 drivers/scsi/qla2xxx/qla_mbx.c |  3 +++
 drivers/scsi/qla2xxx/qla_os.c  | 39 +++++++++++++++++---------------
--
 2 files changed, 22 insertions(+), 20 deletions(-)

what about this series? Will it be merged for 5.7? It got positive reviews from Daniel, and no negative ones so far, and it still applies cleanly to 5.7/scsi-queue.

Thanks,
Martin W.

#23 lore raw

Martin,

Quoted 3 lines

what about this series? Will it be merged for 5.7? It got positive reviews from Daniel, and no negative ones so far, and it still applies cleanly to 5.7/scsi-queue.

Still waiting for feedback from Marvell on these changes (and your other qla2xxx patch).

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering