scsi: qla2xxx: fixes for driver unloading
Subsystems (2)
v4 cover scsi: qla2xxx: fixes for driver unloading
From: Martin Wilck <[email protected]>
Hello Martin, Arun, Himanshu, all,
here is v4 of the little series I first submitted on Nov 29, 2019. I dropped the more controversial part, hoping to get the actual fix for the issue merged.
Reviews welcome.
Martin
Changes since v3:
- In patch 2, moved check from qla2x00_post_async_XYZ_work() to qla2x00_alloc_work() as suggested by Arun
- Dropped patch 3-5. With 1 and 2 applied, I haven't been able to reproduce shutdown hangs any more.
Changes since v2:
- Removed "scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped", because the first hunk is obsoleted by the (new) 1/3, and Arun suggested to use a different approach (which is now in 4/3) for the second hunk.
- Removed "scsi: qla2xxx: don't shut down firmware before closing sessions" (nak'd by Arun).
- Former 3/3 is now 1/5
- Added "scsi: qla2xxx: check UNLOADING before posting async work". This one is key for avoiding lags when qla2xxx is unloaded.
- Added revert of "scsi: qla2xxx: Fix unbound sleep in fcport delete path.", as I believe it's now obsolete.
If we ever encounter unbound sleep there again, we should rather figure out the reason than simply abort waiting.
- Added patch 4 and 5, a new attempt at avoiding mailbox and HW request queue access at low level. 4/5 was motivated by Arun's comments on my v2 series. 5/5 is obviously similar in spirit to 77ddb94a4853 ("scsi: qla2xxx: Only allow operational MBX to proceed during RESET."), but I found that the rom_cmds list contains commands that would hang when the FW is stopped, so I created a new list. Perhaps some day the two can be consolidated.
Changes since v1:
- Added patch 3 to set the UNLOADING flag before waiting for sessions to end (Roman)
- Use test_and_set_bit() for UNLOADING (Hannes)
Martin Wilck (2):
scsi: qla2xxx: set UNLOADING before waiting for session deletion
scsi: qla2xxx: check UNLOADING before posting async work
drivers/scsi/qla2xxx/qla_os.c | 35 +++++++++++++++++------------------ 1 file changed, 17 insertions(+), 18 deletions(-)
Signature · 1 line
2.26.0
PATCH v4 1/2 scsi: qla2xxx: set UNLOADING before waiting for session deletion
From: Martin Wilck <[email protected]>
The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown.
Furthermore, use atomic test_and_set_bit() to avoid the shutdown being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.
--- drivers/scsi/qla2xxx/qla_os.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +14 -18
@@ -3732,6 +3732,13 @@ qla2x00_remove_one(struct pci_dev *pdev) }qla2x00_wait_for_hba_ready (base_vha); +/* +* if UNLOADING flag is already set, then continue unload, +* where it was set first. +*/ +if (test_and_set_bit (UNLOADING , &base_vha->dpc_flags)) +return ; +if (IS_QLA25XX (ha) ||IS_QLA2031 (ha) ||IS_QLA27XX (ha) ||IS_QLA28XX (ha)) {if (ha->flags.fw_started)
@@ -3750,15 +3757,6 @@ qla2x00_remove_one(struct pci_dev *pdev)qla2x00_wait_for_sess_deletion (base_vha); -/* -* if UNLOAD flag is already set, then continue unload, -* where it was set first. -*/ -if (test_bit (UNLOADING , &base_vha->dpc_flags)) -return ; - -set_bit (UNLOADING , &base_vha->dpc_flags); -qla_nvme_delete (base_vha); dma_free_coherent(&ha->pdev->dev,
@@ -6628,13 +6626,6 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)struct pci_dev *pdev = ha->pdev;scsi_qla_host_t *base_vha =pci_get_drvdata (ha->pdev); -/* -* if UNLOAD flag is already set, then continue unload, -* where it was set first. -*/ -if (test_bit (UNLOADING , &base_vha->dpc_flags)) -return ; -ql_log (ql_log_warn, base_vha,0x015b ,"Disabling adapter.\n" );
@@ -6645,9 +6636,14 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)return ; } -qla2x00_wait_for_sess_deletion (base_vha); +/* +* if UNLOADING flag is already set, then continue unload, +* where it was set first. +*/ +if (test_and_set_bit (UNLOADING , &base_vha->dpc_flags)) +return ; -set_bit (UNLOADING , &base_vha->dpc_flags); +qla2x00_wait_for_sess_deletion (base_vha);qla2x00_delete_all_vps (ha, base_vha);
Signature · 1 line
2.26.0
Re: [PATCH v4 1/2] scsi: qla2xxx: set UNLOADING before waiting for session deletion
Quoted 76 lines · On 4/21/20 3:46 PM, [email protected] wrote:
From: Martin Wilck <[email protected]>
The purpose of the UNLOADING flag is to avoid port login procedures to continue when a controller is in the process of shutting down. It makes sense to set this flag before starting session teardown.
Furthermore, use atomic
test_and_set_bit()to avoid the shutdown being run multiple times in parallel. Inqla2x00_disable_board_on_pci_error(), the test for UNLOADING is postponed until after the check for an already disabled PCI board.Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")Reviewed-by Arun Easi <[email protected]>Reviewed-by Daniel Wagner <[email protected]>Reviewed-by Roman Bolshakov <[email protected]>Signed-off-by Martin Wilck <[email protected]>--- drivers/scsi/qla2xxx/qla_os.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-)drivers/scsi/qla2xxx/qla_os.c +9
-17@@ -3732,6 +3732,13 @@ qla2x00_remove_one(struct pci_dev *pdev) }qla2x00_wait_for_hba_ready (base_vha); +/* +* if UNLOADING flag is already set, then continue unload, +* where it was set first. +*/ +if (test_and_set_bit (UNLOADING , &base_vha->dpc_flags)) +return ; +if (IS_QLA25XX (ha) ||IS_QLA2031 (ha) ||IS_QLA27XX (ha) ||IS_QLA28XX (ha)) {if (ha->flags.fw_started)@@ -3750,15 +3757,6 @@ qla2x00_remove_one(struct pci_dev *pdev)qla2x00_wait_for_sess_deletion (base_vha); -/* -* if UNLOAD flag is already set, then continue unload, -* where it was set first. -*/ -if (test_bit (UNLOADING , &base_vha->dpc_flags)) -return ; - -set_bit (UNLOADING , &base_vha->dpc_flags); -qla_nvme_delete (base_vha); dma_free_coherent(&ha->pdev->dev,@@ -6628,13 +6626,6 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)struct pci_dev *pdev = ha->pdev;scsi_qla_host_t *base_vha =pci_get_drvdata (ha->pdev); -/* -* if UNLOAD flag is already set, then continue unload, -* where it was set first. -*/ -if (test_bit (UNLOADING , &base_vha->dpc_flags)) -return ; -ql_log (ql_log_warn, base_vha,0x015b ,"Disabling adapter.\n" );@@ -6645,9 +6636,14 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)return ; } -qla2x00_wait_for_sess_deletion (base_vha); +/* +* if UNLOADING flag is already set, then continue unload,
drivers/scsi/qla2xxx/qla_os.c qla2x00_disable_board_on_pci_error(struct work_struct *work) · line 6649
Quoted 10 lines
drivers/scsi/qla2xxx/qla_os.c +5
-1+* where it was set first. +*/ +if (test_and_set_bit (UNLOADING , &base_vha->dpc_flags)) +return ; -set_bit (UNLOADING , &base_vha->dpc_flags); +qla2x00_wait_for_sess_deletion (base_vha);qla2x00_delete_all_vps (ha, base_vha);
Signature · 2 lines
Himanshu Madhani Oracle Linux Engineering
PATCH v4 2/2 scsi: qla2xxx: check UNLOADING before posting async work
In reply to [email protected]From: Martin Wilck <[email protected]>
qlt_free_session_done() tries to post async PRLO / LOGO, and waits for the completion of these async commands. If UNLOADING is set, this is doomed to timeout, because the async logout command will never complete.
The only way to avoid waiting pointlessly is to fail posting these commands in the first place if the driver is in UNLOADING state. In general, posting any command should be avoided when the driver is UNLOADING.
With this patch, "rmmod qla2xxx" completes without noticeable delay.
--- drivers/scsi/qla2xxx/qla_os.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/scsi/qla2xxx/qla_os.c +3 -0
@@ -4862,6 +4862,9 @@ qla2x00_alloc_work(struct scsi_qla_host *vha, enum qla_work_type type)struct qla_work_evt *e;uint8_t bail; +if (test_bit (UNLOADING , &vha->dpc_flags)) +return NULL ; +QLA_VHA_MARK_BUSY (vha, bail);if (bail)return NULL ;
Signature · 1 line
2.26.0
Re: [PATCH v4 2/2] scsi: qla2xxx: check UNLOADING before posting async work
Quoted 26 lines · On 4/21/20 3:46 PM, [email protected] wrote:
From: Martin Wilck <[email protected]>
qlt_free_session_done()tries to post async PRLO / LOGO, and waits for the completion of these async commands. If UNLOADING is set, this is doomed to timeout, because the async logout command will never complete.The only way to avoid waiting pointlessly is to fail posting these commands in the first place if the driver is in UNLOADING state. In general, posting any command should be avoided when the driver is UNLOADING.
With this patch, "rmmod qla2xxx" completes without noticeable delay.
Fixes 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")Signed-off-by Martin Wilck <[email protected]>--- drivers/scsi/qla2xxx/qla_os.c | 3 +++ 1 file changed, 3 insertions(+)drivers/scsi/qla2xxx/qla_os.c +0
-0
drivers/scsi/qla2xxx/qla_os.c qla2x00_alloc_work(struct scsi_qla_host *vha, enum qla_work_type type) · line 4870
Quoted 9 lines
drivers/scsi/qla2xxx/qla_os.c +3
-0@@ -4862,6 +4862,9 @@ qla2x00_alloc_work(struct scsi_qla_host *vha, enum qla_work_type type)struct qla_work_evt *e;uint8_t bail; +if (test_bit (UNLOADING , &vha->dpc_flags)) +return NULL ; +QLA_VHA_MARK_BUSY (vha, bail);if (bail)return NULL ;
Signature · 2 lines
Himanshu Madhani Oracle Linux Engineering
Series look good.
Thank you for the patches and testing, Martin.
Regards,
-Arun
Quoted 54 lines · On Tue, 21 Apr 2020, 1:46pm, [email protected] wrote:
External Email
----------------------------------------------------------------------
From: Martin Wilck <[email protected]>Hello Martin, Arun, Himanshu, all,
here is v4 of the little series I first submitted on Nov 29, 2019. I dropped the more controversial part, hoping to get the actual fix for the issue merged.
Reviews welcome.
MartinChanges since v3:
- In patch 2, moved check fromqla2x00_post_async_XYZ_work()toqla2x00_alloc_work()as suggested by Arun
- Dropped patch 3-5. With 1 and 2 applied, I haven't been able to reproduce shutdown hangs any more.Changes since v2:
- Removed "scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped", because the first hunk is obsoleted by the (new) 1/3, and Arun suggested to use a different approach (which is now in 4/3) for the second hunk.
- Removed "scsi: qla2xxx: don't shut down firmware before closing sessions" (nak'd by Arun).
- Former 3/3 is now 1/5
- Added "scsi: qla2xxx: check UNLOADING before posting async work". This one is key for avoiding lags when qla2xxx is unloaded.
- Added revert of "scsi: qla2xxx: Fix unbound sleep in fcport delete path.", as I believe it's now obsolete.
If we ever encounter unbound sleep there again, we should rather figure out the reason than simply abort waiting.
- Added patch 4 and 5, a new attempt at avoiding mailbox and HW request queue access at low level. 4/5 was motivated by Arun's comments on my v2 series. 5/5 is obviously similar in spirit to77ddb94a4853("scsi: qla2xxx: Only allow operational MBX to proceed during RESET."), but I found that therom_cmdslist contains commands that would hang when the FW is stopped, so I created a new list. Perhaps some day the two can be consolidated.Changes since v1:
- Added patch 3 to set the UNLOADING flag before waiting for sessions to end (Roman)
- Usetest_and_set_bit()for UNLOADING (Hannes)Martin Wilck (2):
scsi: qla2xxx: set UNLOADING before waiting for session deletion
scsi: qla2xxx: check UNLOADING before posting async workdrivers/scsi/qla2xxx/qla_os.c | 35 +++++++++++++++++------------------ 1 file changed, 17 insertions(+), 18 deletions(-)
Martin K. Petersen
AppliedMartin,
Quoted 3 lines
here is v4 of the little series I first submitted on Nov 29, 2019. I dropped the more controversial part, hoping to get the actual fix for the issue merged.
Applied to 5.7/scsi-fixes, thanks!
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering