ibmvfc: Move login error handling
Subsystems (3)
cover ibmvfc: Move login error handling
This set of patches makes the move login handling in the ibmvfc driver more robust and resolves some issues seen in our SAN interop testing.
Brian King (3):
ibmvfc: Handle move login failure
ibmvfc: Avoid move login if fast fail is enabled
ibmvfc: Reinit target retries
drivers/scsi/ibmvscsi/ibmvfc.c | 60 +++++++++++++++++++++++++++++------------- drivers/scsi/ibmvscsi/ibmvfc.h | 3 ++- 2 files changed, 43 insertions(+), 20 deletions(-)
Signature · 1 line
1.8.3.1
PATCH 1/3 ibmvfc: Handle move login failure
When service is being performed on an SVC with NPIV enabled, the WWPN of the canister / node being serviced fails over to the another canister / node. This looks to the ibmvfc driver as a WWPN moving from one SCSI ID to another. The driver will first attempt to do an implicit logout of the old SCSI ID. If this works, we simply delete the rport at the old location and add an rport at the new location and the FC transport class handles everything. However, if there is I/O outstanding, this implicit logout will fail, in which case we will send a "move login" request to the VIOS. This will cancel any outstanding I/O to that port, logout the port, and PLOGI the new port. Recently we've encountered a scenario where the move login fails. This was resulting in an attempted plogi to the new scsi id, without the old scsi id getting logged out, which is a VIOS protocol violation. To solve this, we want to keep tracking the old scsi id as the current scsi id. That way, once terminate_rport_io cancels the outstanding i/o, it will send us back through to do an implicit logout of the old scsi id, rather than the new scsi id, and then we can plogi the new scsi id.
--- drivers/scsi/ibmvscsi/ibmvfc.c | 16 ++++++++-------- drivers/scsi/ibmvscsi/ibmvfc.h | 2 +- 2 files changed, 9 insertions(+), 9 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc.c +8 -8
@@ -4299,9 +4299,10 @@ static void ibmvfc_tgt_move_login_done(struct ibmvfc_event *evt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );switch (status) {case IBMVFC_MAD_SUCCESS : -tgt_dbg (tgt,"Move Login succeeded for old scsi_id: %llX\n" , tgt->old_scsi_id); +tgt_dbg (tgt,"Move Login succeeded for new scsi_id: %llX\n" , tgt->new_scsi_id); tgt->ids.node_name =wwn_to_u64 (rsp->service_parms.node_name); tgt->ids.port_name =wwn_to_u64 (rsp->service_parms.port_name); + tgt->scsi_id = tgt->new_scsi_id; tgt->ids.port_id = tgt->scsi_id;memcpy (&tgt->service_parms, &rsp->service_parms,sizeof (tgt->service_parms));
@@ -4319,8 +4320,8 @@ static void ibmvfc_tgt_move_login_done(struct ibmvfc_event *evt) level +=ibmvfc_retry_tgt_init (tgt, ibmvfc_tgt_move_login);tgt_log (tgt, level, -"Move Login failed: old scsi_id: %llX, flags:%x, vios_flags:%x, rc=0x%02X\n" , - tgt->old_scsi_id,be32_to_cpu (rsp->flags),be16_to_cpu (rsp->vios_flags), +"Move Login failed: new scsi_id: %llX, flags:%x, vios_flags:%x, rc=0x%02X\n" , + tgt->new_scsi_id,be32_to_cpu (rsp->flags),be16_to_cpu (rsp->vios_flags), status);break ; }
@@ -4357,8 +4358,8 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *tgt) move->common.opcode =cpu_to_be32 (IBMVFC_MOVE_LOGIN ); move->common.length =cpu_to_be16 (sizeof (*move)); - move->old_scsi_id =cpu_to_be64 (tgt->old_scsi_id); - move->new_scsi_id =cpu_to_be64 (tgt->scsi_id); + move->old_scsi_id =cpu_to_be64 (tgt->scsi_id); + move->new_scsi_id =cpu_to_be64 (tgt->new_scsi_id); move->wwpn =cpu_to_be64 (tgt->wwpn); move->node_name =cpu_to_be64 (tgt->ids.node_name);
@@ -4367,7 +4368,7 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_DEL_RPORT );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent Move Login for old scsi_id: %llX\n" , tgt->old_scsi_id); +tgt_dbg (tgt,"Sent Move Login for new scsi_id: %llX\n" , tgt->new_scsi_id); }/**
@@ -4737,8 +4738,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,* normal ibmvfc_set_tgt_action to set this, as we * don't normally want to allow this state change. */ - wtgt->old_scsi_id = wtgt->scsi_id; - wtgt->scsi_id = scsi_id; + wtgt->new_scsi_id = scsi_id; wtgt->action =IBMVFC_TGT_ACTION_INIT ;ibmvfc_init_tgt (wtgt, ibmvfc_tgt_move_login); goto unlock_out;
drivers/scsi/ibmvscsi/ibmvfc.h +1 -1
@@ -718,7 +718,7 @@ struct ibmvfc_target {struct ibmvfc_host *vhost;u64 scsi_id;u64 wwpn; -u64 old_scsi_id; +u64 new_scsi_id;struct fc_rport *rport;int target_id;enum ibmvfc_target_action action;
Signature · 1 line
1.8.3.1
PATCH 2/3 ibmvfc: Avoid move login if fast fail is enabled
In reply to Brian KingIf fast fail is enabled and we encounter a WWPN moving from one port id to another port id with I/O outstanding, if we use the move login MAD, although it will work, it will leave any outstanding I/O still outstanding to the old port id. Eventually, the SCSI command timers will fire and we will abort these commands, however, this is generally much longer than the fast fail timeout, which can lead to I/O operations being outstanding for a long time. This patch changes the behavior to avoid the move login if fast fail is enabled. Once terminate_rport_io cleans up the rport, then we force the target back through the delete process, which re-drives the implicit logout, then kicks us back into discovery where we will discover the WWPN at the new location and do a PLOGI to it.
--- drivers/scsi/ibmvscsi/ibmvfc.c | 39 ++++++++++++++++++++++++++++----------- drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 29 insertions(+), 11 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc.c +28 -11
@@ -4728,19 +4728,24 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,* and it failed for some reason, such as there being I/O * pending to the target. In this case, we will have already * deleted the rport from the FC transport so we do a move -* login, which works even with I/O pending, as it will cancel -* any active commands. +* login, which works even with I/O pending, however, if +* there is still I/O pending, it will stay outstanding, so +* we only do this if fast fail is disabled for the rport, +* otherwise we let terminate_rport_io clean up the port +* before we login at the new location. */ if (wtgt->action ==IBMVFC_TGT_ACTION_LOGOUT_DELETED_RPORT ) { -/* -* Do a move login here. The old target is no longer -* known to the transport layer We don't use the -* normal ibmvfc_set_tgt_action to set this, as we -* don't normally want to allow this state change. -*/ - wtgt->new_scsi_id = scsi_id; - wtgt->action =IBMVFC_TGT_ACTION_INIT ; -ibmvfc_init_tgt (wtgt, ibmvfc_tgt_move_login); +if (wtgt->move_login) { +/* +* Do a move login here. The old target is no longer +* known to the transport layer We don't use the +* normal ibmvfc_set_tgt_action to set this, as we +* don't normally want to allow this state change. +*/ + wtgt->new_scsi_id = scsi_id; + wtgt->action =IBMVFC_TGT_ACTION_INIT ; +ibmvfc_init_tgt (wtgt, ibmvfc_tgt_move_login); + } goto unlock_out; }else { tgt_err(wtgt, "Unexpected target state: %d, %p\n",
@@ -5486,6 +5491,18 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost) rport = tgt->rport; tgt->rport =NULL ;ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_LOGOUT_DELETED_RPORT ); + +/* +* If fast fail is enabled, we wait for it to fire and then clean up +* the old port, since we expect the fast fail timer to clean up the +* outstanding I/O faster than waiting for normal command timeouts. +* However, if fast fail is disabled, any I/O outstanding to the +* rport LUNs will stay outstanding indefinitely, since the EH handlers +* won't get invoked for I/O's timing out. If this is a NPIV failover +* scenario, the better alternative is to use the move login. +*/ +if (rport && rport->fast_io_fail_tmo ==-1 ) + tgt->move_login =1 ;spin_unlock_irqrestore (vhost->host->host_lock, flags);if (rport)fc_remote_port_delete (rport);
drivers/scsi/ibmvscsi/ibmvfc.h +1 -0
@@ -726,6 +726,7 @@ struct ibmvfc_target {int add_rport;int init_retries;int logo_rcvd; +int move_login;u32 cancel_key;struct ibmvfc_service_parms service_parms;struct ibmvfc_service_parms service_parms_change;
Signature · 1 line
1.8.3.1
PATCH 3/3 ibmvfc: Reinit target retries
In reply to Brian KingIf rport target discovery commands fail for some reason, they get retried up to a set number of retries. Once the retry limit is exceeded, the target is deleted. In order to delete the target, we either need to do an implicit logout or a move login. In the move login case, if the move login fails, we want to retry it. This ensures the retry counter gets reinitialized so the move login will get retried.
--- drivers/scsi/ibmvscsi/ibmvfc.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/scsi/ibmvscsi/ibmvfc.c +6 -1
@@ -654,8 +654,10 @@ static void ibmvfc_reinit_host(struct ibmvfc_host *vhost)**/ static void ibmvfc_del_tgt (struct ibmvfc_target *tgt) { -if (!ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_LOGOUT_RPORT )) +if (!ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_LOGOUT_RPORT )) { tgt->job_step = ibmvfc_tgt_implicit_logout_and_del; + tgt->init_retries =0 ; + }wake_up (&tgt->vhost->work_wait_q); }
@@ -4744,6 +4746,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,*/ wtgt->new_scsi_id = scsi_id; wtgt->action =IBMVFC_TGT_ACTION_INIT ; + wtgt->init_retries =0 ;ibmvfc_init_tgt (wtgt, ibmvfc_tgt_move_login); } goto unlock_out;
@@ -5336,6 +5339,7 @@ static void ibmvfc_tgt_add_rport(struct ibmvfc_target *tgt)tgt_dbg (tgt,"Deleting rport with outstanding I/O\n" );ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_LOGOUT_DELETED_RPORT ); tgt->rport =NULL ; + tgt->init_retries =0 ;spin_unlock_irqrestore (vhost->host->host_lock, flags);fc_remote_port_delete (rport);return ;
@@ -5490,6 +5494,7 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)tgt_dbg (tgt,"Deleting rport with I/O outstanding\n" ); rport = tgt->rport; tgt->rport =NULL ; + tgt->init_retries =0 ;ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_LOGOUT_DELETED_RPORT );/*
Signature · 1 line
1.8.3.1
Martin K. Petersen
AppliedBrian,
Quoted 2 lines
This set of patches makes the move login handling in the ibmvfc driver more robust and resolves some issues seen in our SAN interop testing.
Applied to 5.14/scsi-staging, thanks!
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering