Skip to contentSkip to conversation

zfcp FC BSG timeout fixes for 2.6.33-rc4

PATCH New Christof Schmitt 1 participant 4 messages Jan 14, 2010

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover zfcp FC BSG timeout fixes for 2.6.33-rc4

Here is the patch series for the FC BSG timeout problem. It is based on the initial proposal here:
http://marc.info/?l=linux-scsi&m=126043474110957&w=2

To summarize the problem and the approach taken: The block layer runs a timer for each FC BSG request. When the timer expires, the LLD's bsg_timeout function is called to abort the request. With zfcp, the hardware does not support aborting the CT or ELS request. Instead the hardware runs an internal timer that is setup from Linux for each CT and BSG request. When the request times out in the block layer, the request data structures are destroyed. When the still pending hardware request returns, it will access already freed memory.

The solution taken here is to add a special return code to the bsg_timeout callback to indicate that the block layer timer should be reset. zfcp uses this to prevent the block layer timeout. To return the FC BSG request in the time requested by the user, the hardware timer is setup with the timeout value from the FC BSG request.

The patches apply cleanly on top of 2.6.33-rc4 and the zfcp fixes posted yesterday.

--
Christof Schmitt

#1 lore raw

PATCH 1/3 scsi_transport_fc: Allow LLD to reset FC BSG timeout

From: Swen Schillig <[email protected]>

The hardware used with zfcp cannot abort a currently pending CT or ELS request. Therefore we need the option to postpone the timeout triggered request abort within the fc layer, since there is nothing zfcp can do to stop the request at this point.

---
 drivers/scsi/scsi_transport_fc.c |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/scsi/scsi_transport_fc.c +4 -1
@@ -3527,7 +3527,10 @@ fc_bsg_job_timeout(struct request *req)
 	if (!done && i->f->bsg_timeout) {
 		/* call LLDD to abort the i/o as it has timed out */
 		err = i->f->bsg_timeout(job);
-		if (err)
+		if (err == -EAGAIN) {
+			job->ref_cnt--;
+			return BLK_EH_RESET_TIMER;
+		} else if (err)
 			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
 				"abort failed with status %d\n", err);
 	}
#2 lore raw

PATCH 2/3 zfcp: Introduce bsg_timeout callback.

In reply to Christof Schmitt

From: Swen Schillig <[email protected]>

Introduce a zfcp callback for timeouts triggered from FC BSG. With zfcp, the underlying hardware cannot abort CT or ELS requests, so there is nothing to do when the block layer timeout expires. To avoid interference with the block layer timeout, simply indicate that the block layer timer should be reset. The timer running in the hardware for the pending CT or ELS request will return the request when it expires.

---
 drivers/s390/scsi/zfcp_ext.h  |    1 +
 drivers/s390/scsi/zfcp_fc.c   |    6 ++++++
 drivers/s390/scsi/zfcp_scsi.c |    1 +
 3 files changed, 8 insertions(+)
drivers/s390/scsi/zfcp_ext.h +1 -0
@@ -108,6 +108,7 @@ extern void zfcp_fc_wka_ports_force_offl
 extern int zfcp_fc_gs_setup(struct zfcp_adapter *);
 extern void zfcp_fc_gs_destroy(struct zfcp_adapter *);
 extern int zfcp_fc_exec_bsg_job(struct fc_bsg_job *);
+extern int zfcp_fc_timeout_bsg_job(struct fc_bsg_job *);
 
 /* zfcp_fsf.c */
 extern int zfcp_fsf_open_port(struct zfcp_erp_action *);
drivers/s390/scsi/zfcp_fc.c +6 -0
@@ -788,6 +788,12 @@ int zfcp_fc_exec_bsg_job(struct fc_bsg_j
 	}
 }
 
+int zfcp_fc_timeout_bsg_job(struct fc_bsg_job *job)
+{
+	/* hardware tracks timeout, reset bsg timeout to not interfere */
+	return -EAGAIN;
+}
+
 int zfcp_fc_gs_setup(struct zfcp_adapter *adapter)
 {
 	struct zfcp_fc_wka_ports *wka_ports;
drivers/s390/scsi/zfcp_scsi.c +1 -0
@@ -652,6 +652,7 @@ struct fc_function_template zfcp_transpo
 	.show_host_port_state = 1,
 	.show_host_active_fc4s = 1,
 	.bsg_request = zfcp_fc_exec_bsg_job,
+	.bsg_timeout = zfcp_fc_timeout_bsg_job,
 	/* no functions registered for following dynamic attributes but
 	   directly set by LLDD */
 	.show_host_port_type = 1,
#3 lore raw

PATCH 3/3 zfcp: Set hardware timeout as requested by BSG request.

In reply to Christof Schmitt

From: Swen Schillig <[email protected]>

The hardware used with zfcp provides a timer for CT and ELS requests instead of an abort capability for these commands. To correctly handle the FC BSG timeouts, pass the timeout from the BSG requests to the hardware.

---
 drivers/s390/scsi/zfcp_ext.h |    4 ++--
 drivers/s390/scsi/zfcp_fc.c  |   13 ++++++++-----
 drivers/s390/scsi/zfcp_fc.h  |    2 ++
 drivers/s390/scsi/zfcp_fsf.c |   19 ++++++++++---------
 4 files changed, 22 insertions(+), 16 deletions(-)
drivers/s390/scsi/zfcp_ext.h +2 -2
@@ -130,9 +130,9 @@ extern void zfcp_fsf_req_dismiss_all(str
 extern int zfcp_fsf_status_read(struct zfcp_qdio *);
 extern int zfcp_status_read_refill(struct zfcp_adapter *adapter);
 extern int zfcp_fsf_send_ct(struct zfcp_fc_wka_port *, struct zfcp_fsf_ct_els *,
-			    mempool_t *);
+			    mempool_t *, unsigned int);
 extern int zfcp_fsf_send_els(struct zfcp_adapter *, u32,
-			     struct zfcp_fsf_ct_els *);
+			     struct zfcp_fsf_ct_els *, unsigned int);
 extern int zfcp_fsf_send_fcp_command_task(struct zfcp_unit *,
 					  struct scsi_cmnd *);
 extern void zfcp_fsf_req_free(struct zfcp_fsf_req *);
drivers/s390/scsi/zfcp_fc.c +8 -5
@@ -258,7 +258,8 @@ static int zfcp_fc_ns_gid_pn_request(str
 	gid_pn->gid_pn_req.gid_pn.fn_wwpn = port->wwpn;
 
 	ret = zfcp_fsf_send_ct(&adapter->gs->ds, &gid_pn->ct,
-			       adapter->pool.gid_pn_req);
+			       adapter->pool.gid_pn_req,
+			       ZFCP_FC_CTELS_TMO);
 	if (!ret) {
 		wait_for_completion(&completion);
 		zfcp_fc_ns_gid_pn_eval(gid_pn);
@@ -421,7 +422,8 @@ static int zfcp_fc_adisc(struct zfcp_por
 	hton24(adisc->adisc_req.adisc_port_id,
 	       fc_host_port_id(adapter->scsi_host));
 
-	ret = zfcp_fsf_send_els(adapter, port->d_id, &adisc->els);
+	ret = zfcp_fsf_send_els(adapter, port->d_id, &adisc->els,
+				ZFCP_FC_CTELS_TMO);
 	if (ret)
 		kmem_cache_free(zfcp_data.adisc_cache, adisc);
 
@@ -532,7 +534,8 @@ static int zfcp_fc_send_gpn_ft(struct zf
 	ct->req = &gpn_ft->sg_req;
 	ct->resp = gpn_ft->sg_resp;
 
-	ret = zfcp_fsf_send_ct(&adapter->gs->ds, ct, NULL);
+	ret = zfcp_fsf_send_ct(&adapter->gs->ds, ct, NULL,
+			       ZFCP_FC_CTELS_TMO);
 	if (!ret)
 		wait_for_completion(&completion);
 	return ret;
@@ -734,7 +737,7 @@ static int zfcp_fc_exec_els_job(struct f
 		d_id = ntoh24(job->request->rqst_data.h_els.port_id);
 
 	els->handler = zfcp_fc_ct_els_job_handler;
-	return zfcp_fsf_send_els(adapter, d_id, els);
+	return zfcp_fsf_send_els(adapter, d_id, els, job->req->timeout / HZ);
 }
 
 static int zfcp_fc_exec_ct_job(struct fc_bsg_job *job,
@@ -753,7 +756,7 @@ static int zfcp_fc_exec_ct_job(struct fc
 		return ret;
 
 	ct->handler = zfcp_fc_ct_job_handler;
-	ret = zfcp_fsf_send_ct(wka_port, ct, NULL);
+	ret = zfcp_fsf_send_ct(wka_port, ct, NULL, job->req->timeout / HZ);
 	if (ret)
 		zfcp_fc_wka_port_put(wka_port);
 
drivers/s390/scsi/zfcp_fc.h +2 -0
@@ -27,6 +27,8 @@
 #define ZFCP_FC_GPN_FT_MAX_ENT	  (ZFCP_FC_GPN_FT_NUM_BUFS * \
 					(ZFCP_FC_GPN_FT_ENT_PAGE + 1))
 
+#define ZFCP_FC_CTELS_TMO	(2 * FC_DEF_R_A_TOV / 1000)
+
 /**
  * struct zfcp_fc_gid_pn_req - container for ct header plus gid_pn request
  * @ct_hdr: FC GS common transport header
drivers/s390/scsi/zfcp_fsf.c +10 -9
@@ -1068,20 +1068,20 @@ static int zfcp_fsf_setup_ct_els_sbals(s
 static int zfcp_fsf_setup_ct_els(struct zfcp_fsf_req *req,
 				 struct scatterlist *sg_req,
 				 struct scatterlist *sg_resp,
-				 int max_sbals)
+				 int max_sbals, unsigned int timeout)
 {
 	int ret;
-	unsigned int fcp_chan_timeout;
 
 	ret = zfcp_fsf_setup_ct_els_sbals(req, sg_req, sg_resp, max_sbals);
 	if (ret)
 		return ret;
 
 	/* common settings for ct/gs and els requests */
-	fcp_chan_timeout = 2 * FC_DEF_R_A_TOV / 1000;
+	if (timeout > 255)
+		timeout = 255; /* max value accepted by hardware */
 	req->qtcb->bottom.support.service_class = FSF_CLASS_3;
-	req->qtcb->bottom.support.timeout = fcp_chan_timeout;
-	zfcp_fsf_start_timer(req, (fcp_chan_timeout + 10) * HZ);
+	req->qtcb->bottom.support.timeout = timeout;
+	zfcp_fsf_start_timer(req, (timeout + 10) * HZ);
 
 	return 0;
 }
@@ -1092,7 +1092,8 @@ static int zfcp_fsf_setup_ct_els(struct 
  * @pool: if non-null this mempool is used to allocate struct zfcp_fsf_req
  */
 int zfcp_fsf_send_ct(struct zfcp_fc_wka_port *wka_port,
-		     struct zfcp_fsf_ct_els *ct, mempool_t *pool)
+		     struct zfcp_fsf_ct_els *ct, mempool_t *pool,
+		     unsigned int timeout)
 {
 	struct zfcp_qdio *qdio = wka_port->adapter->qdio;
 	struct zfcp_fsf_req *req;
@@ -1111,7 +1112,7 @@ int zfcp_fsf_send_ct(struct zfcp_fc_wka_
 
 	req->status |= ZFCP_STATUS_FSFREQ_CLEANUP;
 	ret = zfcp_fsf_setup_ct_els(req, ct->req, ct->resp,
-				    FSF_MAX_SBALS_PER_REQ);
+				    FSF_MAX_SBALS_PER_REQ, timeout);
 	if (ret)
 		goto failed_send;
 
@@ -1188,7 +1189,7 @@ skip_fsfstatus:
  * @els: pointer to struct zfcp_send_els with data for the command
  */
 int zfcp_fsf_send_els(struct zfcp_adapter *adapter, u32 d_id,
-		      struct zfcp_fsf_ct_els *els)
+		      struct zfcp_fsf_ct_els *els, unsigned int timeout)
 {
 	struct zfcp_fsf_req *req;
 	struct zfcp_qdio *qdio = adapter->qdio;
@@ -1206,7 +1207,7 @@ int zfcp_fsf_send_els(struct zfcp_adapte
 	}
 
 	req->status |= ZFCP_STATUS_FSFREQ_CLEANUP;
-	ret = zfcp_fsf_setup_ct_els(req, els->req, els->resp, 2);
+	ret = zfcp_fsf_setup_ct_els(req, els->req, els->resp, 2, timeout);
 
 	if (ret)
 		goto failed_send;