Skip to contentSkip to conversation

FC pass thru - Rev IV

RFC In review James Smart 6 participants 54 messages Nov 18, 2008 – Mar 25, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC 1/1 FC pass thru - Rev IV

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   53 ++
 6 files changed, 946 insertions(+), 4 deletions(-)
Documentation/scsi/scsi_fc_transport.txt +10 -1
diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt
@@ -1,10 +1,11 @@
                              SCSI FC Tansport
                  =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008
 Kernel Revisions for features:
   rports : <<TBS>>
   vports : 2.6.22 (? TBD)
+  bsg support : 2.6.29 (?TBD)
 
 
 Introduction
@@ -15,6 +16,7 @@ The FC transport can be found at:
   drivers/scsi/scsi_transport_fc.c
   include/scsi/scsi_transport_fc.h
   include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 
 This file is found at Documentation/scsi/scsi_fc_transport.txt
 
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+========================================================================
+<< To Be Supplied >>
+
+
+
+
 Credits
 =======
 The following people have contributed to this document:
Documentation/scsi/scsi_mid_low_api.txt +5 -0
diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
@@ -1271,6 +1271,11 @@ of interest:
     hostdata[0]  - area reserved for LLD at end of struct Scsi_Host. Size
                    is set by the second argument (named 'xtr_bytes') to
                    scsi_host_alloc() or scsi_register().
+    vendor_id    - a unique value that identifies the vendor supplying
+                   the LLD for the Scsi_Host.  Used most often in validating
+                   vendor-specific message requests.  Value consists of an
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid formats.
 
 The scsi_host structure is defined in include/scsi/scsi_host.h
 
drivers/scsi/scsi_transport_fc.c +580 -1

635 lines not on this page · raw message

include/scsi/scsi_bsg_fc.h +291 -0

291 lines not on this page · raw message

include/scsi/scsi_host.h +9 -0
diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
@@ -478,6 +478,15 @@ struct scsi_host_template {
 	 * module_init/module_exit.
 	 */
 	struct list_head legacy_hosts;
+
+	/*
+	 * Vendor Identifier associated with the host
+	 *
+	 * Note: When specifying vendor_id, be sure to read the
+	 *   Vendor Type and ID formatting requirements specified in
+	 *   scsi_netlink.h
+	 */
+	u64 vendor_id;
 };
 
 /*
include/scsi/scsi_transport_fc.h +51 -2

92 lines not on this page · raw message

#1 lore raw

Hi James,

thanks for the rework, looks pretty good two me :) One first question below.

James Smart wrote:
<snip>

include/scsi/scsi_bsg_fc.h line 54

Quoted 6 lines
include/scsi/scsi_bsg_fc.h +6 -0
+	/* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
+#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
+#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
+

Shouldn't we add FC_BSG_HST_CT_NOLOGIN as well?
As far as I can see from looking over the patch (correct me when I am wrong), the way to issue a CT request now is to send an FC_BSG_HST_ADD_RPORT for the wka port in question (if not enumerated by the driver, which zfcp does not do), then send the FC_BSG_RPT_CT followed by a FC_BSG_HST_DEL_RPORT. I was under the impression the CT stuff was one of the main reasons to have devices for the fc_host. What do you think?

include/scsi/scsi_bsg_fc.h line 58

Quoted 4 lines
include/scsi/scsi_bsg_fc.h +4 -0
+	/* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
+#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
+

</snip>

Sven

#2 lore raw
Quoted 1 line · Sven Schuetz

Shouldn't we add FC_BSG_HST_CT_NOLOGIN as well?

I didn't add this as FC-GS-6 r9.21, which defines the CT protocol, says:
4.5.2.1 Fabric login and N_Port login
An Nx_Port shall perform Fabric Login, and shall perform N_Port Login with the WKA or N_Port Identifier where the Service is offered, in the manner that is specified in FC-FS, before making any requests of a Server provided by the Service. An Nx_Port that has completed its registration with a Server shall perform explicit N_Port Logout with the Service. An Nx_Port that has completed any other requests with a Server should also perform explicit N_Port Logout with the Service.

So - a login for CT requests is a must (at least as I read it).

That's not the case for ELS's, which can usually survive on an implicit login - especially things like ADISC/PDISC. Login is also the
distinction in my opinion for whether something should or should not be exported via an rport in the transport.

Quoted 6 lines

As far as I can see from looking over the patch (correct me when I am wrong), the way to issue a CT request now is to send an FC_BSG_HST_ADD_RPORT for the wka port in question (if not enumerated by the driver, which zfcp does not do), then send the FC_BSG_RPT_CT followed by a FC_BSG_HST_DEL_RPORT. I was under the impression the CT stuff was one of the main reasons to have devices for the fc_host. What do you think?

Your flow is exactly what I intended...

I also did not involve all the rport build-up, etc in the transport for the adding of the rport. Just as today, all of this occurs within the LLDD for normal discovery (deciding when to send the ELS's, when to make the transport calls, etc) - I'm assuming the same holds true for the add rport.

Also not addressed - how do we enable CT-receive and ELS-recieve. This, due to the buffering and routing definitions (what process gets what) is a fun problem.

Thanks Sven.

-- james s

#3 lore raw
Quoted 13 lines · James Smart
Quoted 1 line · Sven Schuetz

Shouldn't we add FC_BSG_HST_CT_NOLOGIN as well?

I didn't add this as FC-GS-6 r9.21, which defines the CT protocol, says:
4.5.2.1 Fabric login and N_Port login
An Nx_Port shall perform Fabric Login, and shall perform N_Port Login with the WKA or N_Port Identifier where the Service is offered, in the manner that is specified in FC-FS, before making any requests of a Server provided by the Service. An Nx_Port that has completed its registration with a Server shall perform explicit N_Port Logout with the Service. An Nx_Port that has completed any other requests with a Server should also perform explicit N_Port Logout with the Service.

Quoted 6 lines

So - a login for CT requests is a must (at least as I read it).

That's not the case for ELS's, which can usually survive on an implicit
login - especially things like ADISC/PDISC. Login is also the
distinction in my opinion for whether something should or should not be exported via an rport in the transport.

Hi James,

I agree.

Quoted 12 lines
Quoted 11 lines

As far as I can see from looking over the patch (correct me when I am wrong),
the way to issue a CT request now is to send an FC_BSG_HST_ADD_RPORT for the wka
port in question (if not enumerated by the driver, which zfcp does not do), then
send the FC_BSG_RPT_CT followed by a FC_BSG_HST_DEL_RPORT. I was under the
impression the CT stuff was one of the main reasons to have devices for the
fc_host. What do you think?

Quoted 1 line

Your flow is exactly what I intended...

OK, let me be a bit more precise. Maybe I shouldn't have called it FC_BSG_HST_CT_NOLOGIN but something like FC_BSG_HST_CT_NOT_ENUMERATE.

As the FS-GS dictates, we do a port login and logout for CT requests, but completely in the driver. We do that for some internal CT stuff and I have implemented it for userspace CT requests with Seokmanns patch as well. We just do not advertise the ports to fc_transport (we are only logged in for the duration of the request). Having to react to FC_BSG_HST_ADD_RPORT would mean to rewrite our wka port handling, plus some of it is done in our firmware and not in zfcp.

So my suggestion would be to add something like FC_BSG_HST_CT_NOT_ENUMERATE for drivers that do the port logins on their own. Other drivers can respond with an error code, making it necessary for the app to do the port login on their own (FC_BSG_HST_ADD_RPORT), that means following the procedure I outlined in my previous mail.

In addition, userspace apps that deal with complete CT frames (like implementations of HBA API) would not have to rip the frame apapart to find out the generic service/wka port in question and then do the port login. They can just send the frame to the driver and rely on the driver to do the port login/logout.

Sven

#4 lore raw

You are making many of the same arguments I tried to enumerate with James B when we first started discussing whether we had to enumerate everything before talking to it. Your points are a bit more eloquent than mine. I put the ELS_NOLOGIN on the host in to test the argument.

I'm not married to requiring everything to be enumerated to talk to it via CT/ELS and figured in many cases, simply making the request to the host really was the right thing to do. Being able to do so via the host, as well as via anything enumerated, plus the request to enumerate something, are enough primitives to cover everything.

So, following your proposal, let's add a FC_BSG_HST_CT request, that looks similar in form to the FC_BSG_HST_ELS_NOLOGIN request and with response semantics similar to the rport version. We'll let the LLDD determine whether it needs to do logins/logouts, and whether it wants to enumerate anything in the process.

-- james s

Quoted 25 lines · Sven Schuetz

OK, let me be a bit more precise. Maybe I shouldn't have called it FC_BSG_HST_CT_NOLOGIN but something like FC_BSG_HST_CT_NOT_ENUMERATE.

As the FS-GS dictates, we do a port login and logout for CT requests, but completely in the driver. We do that for some internal CT stuff and I have implemented it for userspace CT requests with Seokmanns patch as well. We just do not advertise the ports to fc_transport (we are only logged in for the duration of the request). Having to react to FC_BSG_HST_ADD_RPORT would mean to rewrite our wka port handling, plus some of it is done in our firmware and not in zfcp.

So my suggestion would be to add something like FC_BSG_HST_CT_NOT_ENUMERATE for drivers that do the port logins on their own. Other drivers can respond with an error code, making it necessary for the app to do the port login on their own (FC_BSG_HST_ADD_RPORT), that means following the procedure I outlined in my previous mail.

In addition, userspace apps that deal with complete CT frames (like implementations of HBA API) would not have to rip the frame apapart to find out the generic service/wka port in question and then do the port login. They can just send the frame to the driver and rely on the driver to do the port login/logout.

Sven

#5 lore raw
In reply to James Smart
Quoted 3 lines · James Smart

All,

I've reworked Seokmann's patch for the following items:

Thanks for the update. Overall, it looks good to me. It looks much clear on what the feature support should looks like, esp., for the "...some request types to the fc_host.." as you mentioned in the earlier mails.
A couple of questions.
1. Should 'loopback' considered at this time?
2. Would it be valuable to consider sending requests with WWNN:WWPN?

I'm looking through the patch and some questions are inlined.

Thank you,
Seokmann

Quoted 25 lines

- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Quoted 6 lines

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

In my thought, the 'job' is preferred over the 'blk' as contents the 'request' is nothing but FC specific service requests.

Quoted 286 lines

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

Documentation/scsi/scsi_fc_transport.txt |   11
Documentation/scsi/scsi_mid_low_api.txt  |    5
drivers/scsi/scsi_transport_fc.c         |  581 +++++++++++++++++++++ 
+++++++++-
include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
include/scsi/scsi_host.h                 |    9
include/scsi/scsi_transport_fc.h         |   53 ++
6 files changed, 946 insertions(+), 4 deletions(-)

diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt
--- a/Documentation/scsi/scsi_fc_transport.txt 2008-09-23
15:11:57.000000000 -0400
+++ b/Documentation/scsi/scsi_fc_transport.txt 2008-11-18
15:54:25.000000000 -0500

Documentation/scsi/scsi_fc_transport.txt +1 -1
@@ -1,10 +1,11 @@
                             SCSI FC Tansport
                 =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008

Kernel Revisions for features:
rports : <<TBS>>
vports : 2.6.22 (? TBD)
+ bsg support : 2.6.29 (?TBD)

Introduction

Documentation/scsi/scsi_fc_transport.txt +1 -0
@@ -15,6 +16,7 @@ The FC transport can be found at:
  drivers/scsi/scsi_transport_fc.c
  include/scsi/scsi_transport_fc.h
  include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 

This file is found at Documentation/scsi/scsi_fc_transport.txt

Documentation/scsi/scsi_fc_transport.txt +2 -0
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+ 

========================================================================
+<< To Be Supplied >>

Documentation/scsi/scsi_fc_transport.txt +4 -0
+
+
+
+

Credits
=======
The following people have contributed to this document:
diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
--- a/Documentation/scsi/scsi_mid_low_api.txt 2008-09-23
15:11:57.000000000 -0400
+++ b/Documentation/scsi/scsi_mid_low_api.txt 2008-11-18
15:54:25.000000000 -0500

(quoted diff) +0 -0
@@ -1271,6 +1271,11 @@ of interest:
    hostdata[0]  - area reserved for LLD at end of struct Scsi_Host.  

Size
is set by the second argument (named 'xtr_bytes')
to
scsi_host_alloc() or scsi_register().
+ vendor_id - a unique value that identifies the vendor
supplying
+ the LLD for the Scsi_Host. Used most often in
validating
+ vendor-specific message requests. Value
consists of an

(quoted diff) +2 -0
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid  

formats.

The scsi_host structure is defined in include/scsi/scsi_host.h

diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c
--- a/drivers/scsi/scsi_transport_fc.c 2008-10-18 10:32:52.000000000
-0400
+++ b/drivers/scsi/scsi_transport_fc.c 2008-11-18 15:58:38.000000000
-0500

drivers/scsi/scsi_transport_fc.c +5 -0
@@ -35,6 +35,7 @@
 #include <linux/netlink.h>
 #include <net/netlink.h>
 #include <scsi/scsi_netlink_fc.h>
+#include <scsi/scsi_bsg_fc.h>
 #include "scsi_priv.h"
 #include "scsi_transport_fc_internal.h"
 
@@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
 static int fc_vport_setup(struct Scsi_Host *shost, int channel,
 	struct device *pdev, struct fc_vport_identifiers  *ids,
 	struct fc_vport **vport);
+static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs  
 *);
+static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
+static void fc_bsg_remove(struct request_queue *);
+static void fc_bsg_goose_queue(struct fc_rport *);
 
 /*

* Redefine so that we can have same named attributes in the

drivers/scsi/scsi_transport_fc.c +14 -1
@@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
 		return -ENOMEM;
 	}
 
+	fc_bsg_hostadd(shost, fc_host);
+	/* ignore any bsg add error - we just can't do sgio */
+
+	return 0;
+}
+
+static int fc_host_remove(struct transport_container *tc, struct  
 device *dev,
+			 struct device *cdev)
+{
+	struct Scsi_Host *shost = dev_to_shost(dev);
+	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
+
+	fc_bsg_remove(fc_host->rqst_q);
 	return 0;
 }
 
 static DECLARE_TRANSPORT_CLASS(fc_host_class,
 			       "fc_host",
 			       fc_host_setup,
-			       NULL,
+			       fc_host_remove,
 			       NULL);
 

/*

drivers/scsi/scsi_transport_fc.c +39 -0
@@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
 		scsi_flush_work(shost);
 
 	fc_terminate_rport_io(rport);
+
 	/*
 	 * Cancel any outstanding timers. These should really exist
 	 * only when rmmod'ing the LLDD and we're asking for
@@ -2411,6 +2430,8 @@ fc_rport_final_delete(struct work_struct
 	if (i->f->dev_loss_tmo_callbk)
 		i->f->dev_loss_tmo_callbk(rport);
 
+	fc_bsg_remove(rport->rqst_q);
+
 	transport_remove_device(dev);
 	device_del(dev);
 	transport_destroy_device(dev);
@@ -2498,6 +2519,9 @@ fc_rport_create(struct Scsi_Host *shost,
 	transport_add_device(dev);
 	transport_configure_device(dev);
 
+	fc_bsg_rportadd(shost, rport);
+	/* ignore any bsg add error - we just can't do sgio */
+
 	if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
 		/* initiate a scan of the target */
 		rport->flags |= FC_RPORT_SCAN_PENDING;
@@ -2661,6 +2685,8 @@ fc_remote_port_add(struct Scsi_Host *sho
 					spin_unlock_irqrestore(shost->host_lock,
 							flags);
 
+				fc_bsg_goose_queue(rport);
+
 				return rport;
 			}
 		}
@@ -3327,6 +3353,559 @@ fc_vport_sched_delete(struct work_struct
 }
 
 
+/*
+ * BSG support
+ */
+
+
+/**
+ * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
+ * @job:	fc_bsg_job that is to be torn down
+ */
+static void
+fc_destroy_bsgjob(struct fc_bsg_job *job)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->ref_cnt) {
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		return;
+	}
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	put_device(job->dev);	/* release reference for the request */
+
+	kfree(job->request_payload.sg_list);
+	kfree(job->reply_payload.sg_list);
+	kfree(job);
+}
+
+
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the  

LLD has
+ * completed
+ * @job: fc_bsg_job that is complete
+ */

drivers/scsi/scsi_transport_fc.c +45 -0
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req->next_rq;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));
+
+	fc_destroy_bsgjob(job);
+}
+
+
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:	request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+	struct fc_bsg_job *job = (void *) req->special;
+	struct Scsi_Host *shost = job->shost;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	unsigned long flags;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3444

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+	int err = 0, done = 0;
+
+	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
+		return BLK_EH_RESET_TIMER;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->state_flags & FC_RQST_STATE_DONE)
+		done = 1;
+	else
+		job->ref_cnt++;

Any reason to increase ref_cnt here?
I think that the ref_cnt is already 1 when it came here so that we don't want to change it to destroy the job successfully.

Quoted 276 lines
drivers/scsi/scsi_transport_fc.c +39 -0
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	if (!done && i->f->bsg_timeout) {
+		/* call LLDD to abort the i/o as it has timed out */
+		err = i->f->bsg_timeout(job);
+		if (err)
+			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
+				"abort failed with status %d\n", err);
+	}
+
+	if (!done) {
+		spin_lock_irqsave(&job->job_lock, flags);
+		job->ref_cnt--;
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		fc_destroy_bsgjob(job);
+	}
+
+	/* the blk_end_sync_io() doesn't check the error */
+	return BLK_EH_HANDLED;
+}
+
+
+
+static void
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+}
+
+
+/**
+ * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for  

the
+ * bsg request
+ * @shost: SCSI Host corresponding to the bsg object
+ * @rport: (optional) FC Remote Port corresponding to the bsg object
+ * @req: BSG request that needs a job structure
+ */

drivers/scsi/scsi_transport_fc.c +62 -0
+static int
+fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
+	struct request *req)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request *rsp = req->next_rq;
+	struct fc_bsg_job *job;
+
+	BUG_ON(req->special);
+
+	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
+			GFP_KERNEL);
+	if (!job)
+		return -ENOMEM;
+
+	/*
+	 * Note: this is a bit silly.
+	 * The request gets formatted as a SGIO v4 ioctl request, which
+	 * then gets reformatted as a blk request, which then gets
+	 * reformatted as a fc bsg request. And on completion, we have
+	 * to wrap return results such that SGIO v4 thinks it was a scsi
+	 * status.  I hope this was all worth it.
+	 */
+
+	req->special = job;
+	job->shost = shost;
+	job->rport = rport;
+	job->req = req;
+	if (i->f->dd_bsg_size)
+		job->dd_data = (void *)&job[1];
+	spin_lock_init(&job->job_lock);
+	job->request = (struct fc_bsg_request *)req->cmd;
+	job->request_len = req->cmd_len;
+	job->reply = req->sense;
+	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
+						 * allocated */
+	if (req->bio)
+		fc_bsg_map_buffer(&job->request_payload, req);
+	if (rsp && rsp->bio)
+		fc_bsg_map_buffer(&job->reply_payload, rsp);
+	job->job_done = fc_bsg_jobdone;
+	if (rport)
+		job->dev = &rport->dev;
+	else
+		job->dev = &shost->shost_gendev;
+	get_device(job->dev);		/* take a reference for the request */
+
+	job->ref_cnt = 1;
+
+	return 0;
+}
+
+
+enum fc_dispatch_result {
+	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
+	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
+	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
+};
+
+
+/**
+ * fc_bsg_host_dispatch - process fc host bsg requests and dispatch  

to LLDD
+ * @shost: scsi host rport attached to
+ * @job: bsg job to be processed
+ */

(quoted diff) +2 -0
+static enum fc_dispatch_result
+fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host  

*shost,

drivers/scsi/scsi_transport_fc.c +78 -0
+			 struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the host command */
+	switch (job->request->msgcode) {
+	case FC_BSG_HST_ADD_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_add_rport);
+		break;
+
+	case FC_BSG_HST_DEL_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_del_rport);
+		break;
+
+	case FC_BSG_HST_ELS_NOLOGIN:
+		cmdlen += sizeof(struct fc_bsg_host_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_host_msg;
+		}
+		break;
+
+	case FC_BSG_HST_VENDOR:
+		cmdlen += sizeof(struct fc_bsg_host_vendor);
+		if ((shost->hostt->vendor_id == 0L) ||
+		    (job->request->rqst_data.h_vendor.vendor_id !=
+			shost->hostt->vendor_id)) {
+			ret = -ESRCH;
+			goto fail_host_msg;
+		}
+		break;
+
+	default:
+		ret = -EBADR;
+		goto fail_host_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_host_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_host_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/*
+ * fc_bsg_goose_queue - restart rport queue in case it was stopped
+ * @rport:	rport to be restarted
+ */
+static void
+fc_bsg_goose_queue(struct fc_rport *rport)
+{
+	int flagset;
+
+	if (!rport->rqst_q)
+		return;
+
+	get_device(&rport->dev);
+	
+	spin_lock(rport->rqst_q->queue_lock);
+	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
Quoted 1 line

queue_flags) &&

drivers/scsi/scsi_transport_fc.c +14 -0
+		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
+	if (flagset)
+		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
+	__blk_run_queue(rport->rqst_q);
+	if (flagset)
+		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
+	spin_unlock(rport->rqst_q->queue_lock);
+
+	put_device(&rport->dev);
+}
+
+
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch  

to LLDD
+ * @shost: scsi host rport attached to
+ * @rport: rport request destined to
+ * @job: bsg job to be processed
+ */

(quoted diff) +2 -0
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host  

*shost,

drivers/scsi/scsi_transport_fc.c +53 -0
+			 struct fc_rport *rport, struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the rport command */
+	switch (job->request->msgcode) {
+	case FC_BSG_RPT_ELS:
+	case FC_BSG_RPT_CT:
+		cmdlen += sizeof(struct fc_bsg_rport_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_rport_msg;
+		}
+		break;
+	default:
+		ret = -EBADR;
+		goto fail_rport_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_rport_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_rport_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:		request queue to manage
+ * @shost:	Scsi_Host related to the bsg object
+ * @rport:	FC remote port related to the bsg object (optional)
+ * @dev:	device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host  

*shost,

drivers/scsi/scsi_transport_fc.c +6 -0
+		       struct fc_rport *rport, struct device *dev)
+{
+	struct request *req;
+	struct fc_bsg_job *job;
+	enum fc_dispatch_result ret;
+

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3722

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+	if (!get_device(dev))
+		return;
+
+	while (!blk_queue_plugged(q)) {
+		req = elv_next_request(q);
+		if (!req)
+			break;
+
+		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
+				break;

Shouldn't this be moved to into fc_bsg_rport_dispatch()?

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3730

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +8 -0
+
+		blkdev_dequeue_request(req);
+
+		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
+			req->errors = -ENXIO;
+			blk_complete_request(req);
+			continue;
+		}

This one, too?

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3738

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +8 -0
+
+		spin_unlock_irq(q->queue_lock);
+
+		ret = fc_req_to_bsgjob(shost, rport, req);
+		if (ret) {
+			req->errors = ret;
+			spin_lock_irq(q->queue_lock);
+			blk_complete_request(req);

Shouldn't this be blk_end_bidi_request()/blk_end_request()?

Quoted 604 lines
drivers/scsi/scsi_transport_fc.c +65 -0
+			continue;
+		}
+
+		job = req->special;
+
+		/* check if we have the msgcode value at least */
+		if (job->request_len < sizeof(uint32_t)) {
+			BUG_ON(job->reply_len < sizeof(uint32_t));
+			job->reply->result = -ENOMSG;
+			job->reply_len = sizeof(uint32_t);
+			fc_bsg_jobdone(job);
+			spin_lock_irq(q->queue_lock);
+			continue;
+		}
+		
+		/* the dispatch routines will unlock the queue_lock */
+		if (rport)
+			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
+		else
+			ret = fc_bsg_host_dispatch(q, shost, job);
+
+		/* did dispatcher hit state that can't process any more */
+		if (ret == FC_DISPATCH_BREAK)
+			break;
+
+		/* did dispatcher had released the lock */
+		if (ret == FC_DISPATCH_UNLOCKED)
+			spin_lock_irq(q->queue_lock);
+	}
+
+	spin_unlock_irq(q->queue_lock);
+	put_device(dev);
+	spin_lock_irq(q->queue_lock);
+}
+
+
+/**
+ * fc_bsg_host_handler - handler for bsg requests for a fc host
+ * @q:		fc host request queue
+ */
+static void
+fc_bsg_host_handler(struct request_queue *q)
+{
+	struct Scsi_Host *shost = q->queuedata;
+
+	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
+}
+
+
+/**
+ * fc_bsg_rport_handler - handler for bsg requests for a fc rport
+ * @q:		rport request queue
+ */
+static void
+fc_bsg_rport_handler(struct request_queue *q)
+{
+	struct fc_rport *rport = q->queuedata;
+	struct Scsi_Host *shost = rport_to_shost(rport);
+
+	fc_bsg_request_handler(q, shost, rport, &rport->dev);
+}
+
+
+/**
+ * fc_bsg_hostadd - Create and add the bsg hooks so we can receive  

requests
+ * @shost: shost for fc_host
+ * @fc_host: fc_host adding the structures to
+ */

(quoted diff) +2 -0
+static int
+fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs  

*fc_host)

drivers/scsi/scsi_transport_fc.c +43 -0
+{
+	struct device *dev = &shost->shost_gendev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+
+	fc_host->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	snprintf(fc_host->bsg_name, sizeof(fc_host->bsg_name),
+		 "fc_host%d", shost->host_no);
+
+	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
+	if (!q) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - no request queue\n",
+				 shost->host_no);
+		return -ENOMEM;
+	}
+
+	q->queuedata = shost;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, fc_host->bsg_name, NULL);
+	if (err) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - register queue\n",
+			 	shost->host_no);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	fc_host->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_rportadd - Create and add the bsg hooks so we can receive  

requests
+ * @shost: shost that rport is attached to
+ * @rport: rport that the bsg hooks are being attached to
+ */

drivers/scsi/scsi_transport_fc.c +54 -0
+static int
+fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
+{
+	struct device *dev = &rport->dev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+
+	rport->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
+	if (!q) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - no request queue\n",
+				 dev->bus_id);
+		return -ENOMEM;
+	}
+
+	q->queuedata = rport;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, dev->bus_id, NULL);
+	if (err) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - register queue\n",
+				 dev->bus_id);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	rport->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
+ * @q:	the request_queue that is to be torn down.
+ */
+static void
+fc_bsg_remove(struct request_queue *q)
+{
+	if (q) {
+		bsg_unregister_queue(q);
+		blk_cleanup_queue(q);
+	}
+}
+
+

/* Original Author: Martin Hicks */
MODULE_AUTHOR("James Smart");
MODULE_DESCRIPTION("FC Transport Attributes");
diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h

include/scsi/scsi_bsg_fc.h +6 -0
@@ -0,0 +1,291 @@
+/*
+ *  FC Transport BSG Interface
+ *
+ *  Copyright (C) 2008   James Smart, Emulex Corporation
+ *
+ *  This program is free software; you can redistribute it and/or  

modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *

+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA   

02111-1307 USA
+ *
+ */

include/scsi/scsi_bsg_fc.h +72 -0
+
+#ifndef SCSI_BSG_FC_H
+#define SCSI_BSG_FC_H
+
+/*
+ * This file intended to be included by both kernel and user space
+ */
+
+#include <scsi/scsi.h>
+#include <scsi/sg.h>
+
+/*
+ * FC Transport SGIO v4 BSG Message Support
+ */
+
+/* Default BSG request timeout (in seconds) */
+#define FC_DEFAULT_BSG_TIMEOUT		10
+
+
+/*
+ * Request Message Codes supported by the FC Transport
+ */
+
+/* define the class masks for the message codes */
+#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
+#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
+#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
+
+	/* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
+#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
+#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
+
+	/* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
+#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
+
+
+
+/*
+ * FC Address Identifiers in Message Structures :
+ *
+ *   Whenever a command payload contains a FC Address Identifier
+ *   (aka port_id), the value is effectively in big-endian
+ *   order, thus the array elements are decoded as follows:
+ *     element [0] is bits 23:16 of the FC Address Identifier
+ *     element [1] is bits 15:8 of the FC Address Identifier
+ *     element [2] is bits 7:0 of the FC Address Identifier
+ */
+
+
+/*
+ * FC Host Messages
+ */
+
+/* FC_BSG_HST_ADDR_PORT : */
+
+/* Request:
+ * This message requests the FC host to login to the remote port
+ * at the specified N_Port_Id.  The remote port is to be enumerated
+ * with the transport upon completion of the login.
+ */
+struct fc_bsg_host_add_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to login to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is  

sufficient
+ */

include/scsi/scsi_bsg_fc.h +20 -0
+
+
+/* FC_BSG_HST_DEL_RPORT : */
+
+/* Request:
+ * This message requests the FC host to remove an enumerated
+ * remote port and to terminate the login to it.
+ *
+ * Note: The driver is free to reject this request if it desires to
+ * remain logged in with the remote port.
+ */
+struct fc_bsg_host_del_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to logout of */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is  

sufficient
+ */

include/scsi/scsi_bsg_fc.h +71 -0
+
+
+/* FC_BSG_HST_ELS_NOLOGIN : */
+
+/* Request:
+ * This message requests the FC_Host to send an ELS to a specific
+ * N_Port_ID. The host does not need to log into the remote port,
+ * nor does it need to enumerate the rport for further traffic
+ * (although, the FC host is free to do so if it desires).
+ */
+struct fc_bsg_host_els {
+	/*
+	 * ELS Command Code being sent (must be the same as byte 0
+	 * of the payload)
+	 */
+	uint8_t 	command_code;
+
+	/* FC Address Identier of the remote port to send the ELS to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ */
+/* fc_bsg_ctels_reply->status values */
+#define FC_CTELS_STATUS_OK	0x00000000
+#define FC_CTELS_STATUS_REJECT	0x00000001
+#define FC_CTELS_STATUS_P_RJT	0x00000002
+#define FC_CTELS_STATUS_F_RJT	0x00000003
+#define FC_CTELS_STATUS_P_BSY	0x00000004
+#define FC_CTELS_STATUS_F_BSY	0x00000006
+struct fc_bsg_ctels_reply {
+	/*
+	 * Note: An ELS LS_RJT may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the ELS receive payload to determine
+	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
+	 *     data will be in word 1.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
+	 *   the receive payload word 0 indicates LS_ACC
+	 *   (e.g. value is 0x02xxxxxx).
+	 *
+	 * Note: Similarly, a CT Reject may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the CT receive payload to determine
+	 *     Accept or Reject (by contents of word 2). The reject
+	 *     data will be in word 3.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: x_RJT/BSY status will indicae that the rjt_data field
+	 *   is valid and contains the reason/explanation values.
+	 */
+	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
+
+	/* valid if status is not FC_CTELS_STATUS_OK */
+	struct	{
+		uint8_t	action;		/* fragment_id for CT REJECT */
+		uint8_t	reason_code;
+		uint8_t	reason_explanation;
+		uint8_t	vendor_unique;
+	} rjt_data;
+};
+
+
+/* FC_BSG_HST_VENDOR : */
+
+/* Request:
+ * Note: When specifying vendor_id, be sure to read the Vendor Type  

and ID
+ * formatting requirements specified in scsi_netlink.h
+ */

include/scsi/scsi_bsg_fc.h +47 -0
+struct fc_bsg_host_vendor {
+	/*
+	 * Identifies the vendor that the message is formatted for. This
+	 * should be the recipient of the message.
+	 */
+	uint64_t vendor_id;
+
+	/* start of vendor command area */
+	uint32_t vendor_cmd[0];
+};
+
+/* Response:
+ */
+struct fc_bsg_host_vendor_reply {
+	/* start of vendor response area */
+	uint32_t vendor_rsp[0];
+};
+
+
+
+/*
+ * FC Remote Port Messages
+ */
+
+/* FC_BSG_RPT_ELS : */
+
+/* Request:
+ * This message requests that an ELS be performed with the rport.
+ */
+struct fc_bsg_rport_els {
+	/*
+	 * ELS Command Code being sent (must be the same as
+	 * byte 0 of the payload)
+	 */
+	uint8_t els_code;
+};
+
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_RPT_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the  

rport.
+ */

include/scsi/scsi_bsg_fc.h +56 -0
+struct fc_bsg_rport_ct {
+	/*
+	 * We need words 0-2 of the generic preamble for the LLD's
+	 */
+	uint32_t	preamble_word0;	/* revision & IN_ID */
+	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
+	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+
+
+/* request (CDB) structure of the sg_io_v4 */
+struct fc_bsg_request {
+	uint32_t msgcode;
+	union {
+		struct fc_bsg_host_add_rport	h_addrport;
+		struct fc_bsg_host_del_rport	h_delrport;
+		struct fc_bsg_host_els	h_els;
+		struct fc_bsg_host_vendor	h_vendor;
+
+		struct fc_bsg_rport_els		r_els;
+		struct fc_bsg_rport_ct		r_ct;
+	} rqst_data;
+};
+
+
+/* response (request sense data) structure of the sg_io_v4 */
+struct fc_bsg_reply {
+	/*
+	 * The completion result. Result exists in two forms:
+	 *  if negative, it is an -Exxx system errno value. There will
+	 *    be no further reply information supplied.
+	 *  else, it's the 4-byte scsi error result, with driver, host,
+	 *    msg and status fields. The per-msgcode reply structure
+	 *    will contain valid data.
+	 */
+	uint32_t result;
+
+	/* If there was reply_payload, how much was recevied ? */
+	uint32_t reply_payload_rcv_len;
+
+	union {
+		struct fc_bsg_host_vendor_reply		vendor_reply;
+
+		struct fc_bsg_ctels_reply		ctels_reply;
+	} reply_data;
+};
+
+
+#endif /* SCSI_BSG_FC_H */
+

diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h

include/scsi/scsi_host.h +9 -0
@@ -478,6 +478,15 @@ struct scsi_host_template {
 	 * module_init/module_exit.
 	 */
 	struct list_head legacy_hosts;
+
+	/*
+	 * Vendor Identifier associated with the host
+	 *
+	 * Note: When specifying vendor_id, be sure to read the
+	 *   Vendor Type and ID formatting requirements specified in
+	 *   scsi_netlink.h
+	 */
+	u64 vendor_id;
 };
 
 /*

diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h
--- a/include/scsi/scsi_transport_fc.h 2008-10-18 10:32:54.000000000
-0400
+++ b/include/scsi/scsi_transport_fc.h 2008-11-18 15:54:25.000000000
-0500

include/scsi/scsi_host.h +51 -2
@@ -33,7 +33,6 @@
 
 struct scsi_transport_template;
 
-
 /*
 * FC Port definitions - Following FC HBAAPI guidelines
 *
@@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
 	struct delayed_work fail_io_work;
 	struct work_struct stgt_delete_work;
 	struct work_struct rport_delete_work;
+	struct request_queue *rqst_q;	/* bsg support */
 } __attribute__((aligned(sizeof(unsigned long))));
 
 /* bit field values for struct fc_rport "flags" field: */
@@ -513,6 +513,10 @@ struct fc_host_attrs {
 	struct workqueue_struct *work_q;
 	char devloss_work_q_name[20];
 	struct workqueue_struct *devloss_work_q;
+
+	/* bsg support */
+	char bsg_name[20];
+	struct request_queue *rqst_q;
 };
 
 #define shost_to_fc_host(x) \
@@ -578,6 +582,47 @@ struct fc_host_attrs {
 	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
 
 
+struct fc_bsg_buffer {
+	unsigned int payload_len;
+	int sg_cnt;
+	struct scatterlist *sg_list;
+};
+
+/* Values for fc_bsg_job->state_flags (bitflags) */
+#define FC_RQST_STATE_INPROGRESS	0
+#define FC_RQST_STATE_DONE		1
+
+struct fc_bsg_job {
+	struct Scsi_Host *shost;
+	struct fc_rport *rport;
+	struct device *dev;
+	struct request *req;
+	spinlock_t job_lock;
+	unsigned int state_flags;
+	unsigned int ref_cnt;
+	void (*job_done)(struct fc_bsg_job *);
+
+	struct fc_bsg_request *request;
+	struct fc_bsg_reply *reply;
+	unsigned int request_len;
+	unsigned int reply_len;
+	/*
+	 * On entry : reply_len indicates the buffer size allocated for
+	 * the reply.
+	 *
+	 * Upon completion : the message handler must set reply_len
+	 *  to indicates the size of the reply to be returned to the
+	 *  caller.
+	 */
+
+	/* DMA payloads for the request/response */
+	struct fc_bsg_buffer request_payload;
+	struct fc_bsg_buffer reply_payload;
+
+	void *dd_data;			/* Used for driver-specific storage */
+};
+
+
 /* The functions by which the transport class and the driver communicate */
 struct fc_function_template {
 	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
@@ -613,9 +658,14 @@ struct fc_function_template {
 	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
 	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
 
+	/* bsg support */
+	int	(* bsg_request)(struct fc_bsg_job *);
+	int	(* bsg_timeout)(struct fc_bsg_job *);
+
 	/* allocation lengths for host-specific data */
 	u32	 			dd_fcrport_size;
 	u32	 			dd_fcvport_size;
+	u32				dd_bsg_size;
 
 	/*
 	 * The driver sets these to tell the transport class it
@@ -736,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
 	vport->vport_state = new_state;
 }
 
-

struct scsi_transport_template *fc_attach_transport(

			struct fc_function_template *);

void fc_release_transport(struct scsi_transport_template *);

#6 lore raw
Quoted 2 lines · Seokmann Ju

A couple of questions.
1. Should 'loopback' considered at this time?

:)

This is a topic for the transport generically, independent of any pass-thru thing.

Quoted 2 lines

2. Would it be valuable to consider sending requests with WWNN:WWPN?

I considered this, especially for the fchost ELS request. But it means the LLDD has to do a nameserver query to find the n_port_id corresponding to the names before it sent the ELS. It started to feel
like too much ancillary stuff for the LLDD to do to support the
request - so I punted. However, the app, where we typically don't
care if it has to do more work, is free to do this and resolve it to
the basic primitives. :)

Quoted 16 lines
Quoted 16 lines
drivers/scsi/scsi_transport_fc.c +16 -0
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:     request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+     struct fc_bsg_job *job = (void *) req->special;
+     struct Scsi_Host *shost = job->shost;
+     struct fc_internal *i = to_fc_internal(shost->transportt);
+     unsigned long flags;
+     int err = 0, done = 0;
+
+     if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
+             return BLK_EH_RESET_TIMER;
+
Quoted 8 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3444

Quoted 5 lines
drivers/scsi/scsi_transport_fc.c +5 -0
+     spin_lock_irqsave(&job->job_lock, flags);
+     if (job->state_flags & FC_RQST_STATE_DONE)
+             done = 1;
+     else
+             job->ref_cnt++;

Any reason to increase ref_cnt here?
I think that the ref_cnt is already 1 when it came here so that we don't want to change it to destroy the job successfully.

Yes - I'm taking an additional reference on the job, so that if the completion routine is completing in parallel to the timeout code path, it won't free the job out from under the timeout call that calls back into the LLDD.

Quoted 19 lines
Quoted 19 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:               request queue to manage
+ * @shost:   Scsi_Host related to the bsg object
+ * @rport:   FC remote port related to the bsg object (optional)
+ * @dev:     device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host

*shost,

drivers/scsi/scsi_transport_fc.c +9 -0
+                    struct fc_rport *rport, struct device *dev)
+{
+     struct request *req;
+     struct fc_bsg_job *job;
+     enum fc_dispatch_result ret;
+
+     if (!get_device(dev))
+             return;
+
Quoted 8 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3722

Quoted 7 lines
drivers/scsi/scsi_transport_fc.c +7 -0
+     while (!blk_queue_plugged(q)) {
+             req = elv_next_request(q);
+             if (!req)
+                     break;
+
+             if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
+                             break;

Shouldn't this be moved to into fc_bsg_rport_dispatch()?

I had originally done this, but moved it back to this location.

Why ? if I moved this out, I also had to move out the code between this and the call to fc_bsg_rport_dispatch() too. This was just a code organization choice so I could keep the calls to allocate the job, etc within this one routine. It also created a nice division on which code-path-on-error had to be dealt with when it fails. Now, this generic routine is the only one that would call blk_complete_request() or leave the request queued, while the rport/fchost dispatch functions *always* call the fc_bsg_jobdone() function - on error or completion.

Quoted 9 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3738

Quoted 8 lines
drivers/scsi/scsi_transport_fc.c +8 -0
+
+             spin_unlock_irq(q->queue_lock);
+
+             ret = fc_req_to_bsgjob(shost, rport, req);
+             if (ret) {
+                     req->errors = ret;
+                     spin_lock_irq(q->queue_lock);
+                     blk_complete_request(req);

Shouldn't this be blk_end_bidi_request()/blk_end_request()?

A good question, that I was hoping to have cleared up. If this was true, there were a lot of errors in the last passthru code.

If I remember right - I looked at what was also happening within scsi, and it sure appeared as if blk_complete_request() will handle the dual condition fine.

-- james s

#7 lore raw
Quoted 13 lines · James Smart
Quoted 2 lines · Seokmann Ju

A couple of questions.
1. Should 'loopback' considered at this time?

:)

This is a topic for the transport generically, independent of any pass-thru thing.

Quoted 2 lines

2. Would it be valuable to consider sending requests with WWNN:WWPN?

Quoted 6 lines

I considered this, especially for the fchost ELS request. But it means the LLDD has to do a nameserver query to find the n_port_id corresponding to the names before it sent the ELS. It started to feel
like too much ancillary stuff for the LLDD to do to support the
request - so I punted. However, the app, where we typically don't
care if it has to do more work, is free to do this and resolve it to
the basic primitives. :)

Thanks, I've got it.

Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
drivers/scsi/scsi_transport_fc.c +14 -0
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:     request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+     struct fc_bsg_job *job = (void *) req->special;
+     struct Scsi_Host *shost = job->shost;
+     struct fc_internal *i = to_fc_internal(shost->transportt);
+     unsigned long flags;
+     int err = 0, done = 0;
+
+     if (job->rport && job->rport->port_state ==  

FC_PORTSTATE_BLOCKED)

(quoted diff) +2 -0
+             return BLK_EH_RESET_TIMER;
+
Quoted 8 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3444

Quoted 5 lines
drivers/scsi/scsi_transport_fc.c +5 -0
+     spin_lock_irqsave(&job->job_lock, flags);
+     if (job->state_flags & FC_RQST_STATE_DONE)
+             done = 1;
+     else
+             job->ref_cnt++;

Any reason to increase ref_cnt here?
I think that the ref_cnt is already 1 when it came here so that we don't want to change it to destroy the job successfully.

Quoted 4 lines

Yes - I'm taking an additional reference on the job, so that if the completion routine is completing in parallel to the timeout code path, it won't free the job out from under the timeout call that calls back into the LLDD.

I see.. I've got it. Thanks.

Quoted 34 lines
Quoted 23 lines
Quoted 23 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:               request queue to manage
+ * @shost:   Scsi_Host related to the bsg object
+ * @rport:   FC remote port related to the bsg object (optional)
+ * @dev:     device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host

*shost,

drivers/scsi/scsi_transport_fc.c +10 -0
+                    struct fc_rport *rport, struct device *dev)
+{
+     struct request *req;
+     struct fc_bsg_job *job;
+     enum fc_dispatch_result ret;
+
+     if (!get_device(dev))
+             return;
+
+     while (!blk_queue_plugged(q)) {
Quoted 8 lines
Quoted 7 lines
drivers/scsi/scsi_transport_fc.c +5 -0
+             req = elv_next_request(q);
+             if (!req)
+                     break;
+
+             if (rport && (rport->port_state ==  

FC_PORTSTATE_BLOCKED))
+ break;

Shouldn't this be moved to into fc_bsg_rport_dispatch()?

I had originally done this, but moved it back to this location.

Quoted 8 lines

Why ? if I moved this out, I also had to move out the code between this and the call to fc_bsg_rport_dispatch() too. This was just a code organization choice so I could keep the calls to allocate the job, etc within this one routine. It also created a nice division on which code-path-on-error had to be dealt with when it fails. Now, this generic routine is the only one that would call blk_complete_request() or leave the request queued, while the rport/fchost dispatch functions *always* call the fc_bsg_jobdone() function - on error or completion.

OK. I see your point and I agree on that "...error had to be dealt with when it fails.".

Quoted 12 lines
Quoted 11 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3738

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +8 -0
+
+             spin_unlock_irq(q->queue_lock);
+
+             ret = fc_req_to_bsgjob(shost, rport, req);
+             if (ret) {
+                     req->errors = ret;
+                     spin_lock_irq(q->queue_lock);
+                     blk_complete_request(req);

Shouldn't this be blk_end_bidi_request()/blk_end_request()?

Quoted 1 line

A good question, that I was hoping to have cleared up. If this was true, there were a lot of errors in the last passthru code.

I'm not sure whether the previous code had the 'blk_complete_request()' somewhere in the code, but my main concern was to make sure that the code completes the requests in such a way so that it remove the requests from the timeout_list by hitting blk_delete_timer().

Seokmann

#8 lore raw
Quoted 9 lines · Seokmann Ju
Quoted 9 lines
Quoted 9 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3738

Quoted 8 lines
drivers/scsi/scsi_transport_fc.c +8 -0
+
+             spin_unlock_irq(q->queue_lock);
+
+             ret = fc_req_to_bsgjob(shost, rport, req);
+             if (ret) {
+                     req->errors = ret;
+                     spin_lock_irq(q->queue_lock);
+                     blk_complete_request(req);

Shouldn't this be blk_end_bidi_request()/blk_end_request()?

Quoted 7 lines
Quoted 2 lines

A good question, that I was hoping to have cleared up. If this was true, there were a lot of errors in the last passthru code.

I'm not sure whether the previous code had the 'blk_complete_request()' somewhere in the code, but my main concern was to make sure that the code completes the requests in such a way so that it remove the requests from the timeout_list by hitting blk_delete_timer().

Hopefully, Jens, Fujita, or Boaz can give us some guidance...

I'd really like to ensure I'm calling the right callback :
a) if it differs based on being a bidi request or not
b) if it differs based on based on whether I've called blk_rq_map_sg()
c) if I don't dequeue it, whether my "goose queue" is handled right.

I found the blk api to be rather confusing on what when.

-- james

#9 lore raw
In reply to Seokmann Ju

Hi James,

Quoted 2 lines · Seokmann Ju

On Nov 24, 2008, at 1:03 PM, James Smart wrote:

[snip]

Quoted 11 lines
Quoted 10 lines
Quoted 3 lines

2. Would it be valuable to consider sending requests with WWNN:WWPN?

I considered this, especially for the fchost ELS request. But it means the LLDD has to do a nameserver query to find the n_port_id corresponding to the names before it sent the ELS. It started to feel
like too much ancillary stuff for the LLDD to do to support the
request - so I punted. However, the app, where we typically don't
care if it has to do more work, is free to do this and resolve it to
the basic primitives. :)

Thanks, I've got it.

One minor questions/clarification.
If I understood correctly, ELS only supports single sg. If this is true, shouldn't it checked somewhere before the transport layer calls LLD?

Thank you,
Seokmann

#10 lore raw
Quoted 4 lines · Seokmann Ju

One minor questions/clarification.
If I understood correctly, ELS only supports single sg. If this is true, shouldn't it checked somewhere before the transport layer calls LLD?

I didn't think we needed to mandate this. So, I've left it in the position where the transport allows it to be more than one, but the driver is free to reject it if it desires. That said, I wouldn't expect an ELS to be more than 1 buffer. If everyone feels strongly enough about it - we can make this restriction.

However, we can't do this on CT operations.

-- james

#11 lore raw
In reply to James Smart
Quoted 3 lines · James Smart

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

So we need to add this to include/scsi/Kbuild, right?

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3393

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req->next_rq;

Shouldn't this be just job->req?

Quoted 13 lines
drivers/scsi/scsi_transport_fc.c +13 -0
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3416

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+	else
+		job->req->sense_len = job->reply_len;
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);

Here is a problem when a LLD does not set job->reply->reply_payload_rcv_len, but has created a response.
blk_end_bidi_request will be called with a length of 0 for the last parameter and will thus return "1: still buffers pending for this request", which we do not evaluate. A few lines later the job is destroyed. But a few seconds later the timeout kicks in, leading to a crash (I suppose because the job has been destroyed).
I see two options:
1. check if we have a response and see if a length has been set - but what do for cases which have a response and no length set? throw the request away?
2. always use the blk_rq_bytes(job->req->next_rq) from Seokmanns initial patch (which I prefer)

Quoted 6 lines
drivers/scsi/scsi_transport_fc.c +6 -0
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));
+
+	fc_destroy_bsgjob(job);
+}
+

<snip>

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @rport:	rport request destined to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_rport *rport, struct fc_bsg_job *job)

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3664

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the rport command */
+	switch (job->request->msgcode) {
+	case FC_BSG_RPT_ELS:
+	case FC_BSG_RPT_CT:
+		cmdlen += sizeof(struct fc_bsg_rport_els);

Why do we do the same here for ELS and CT? The struct fc_bsg_rport_ct is some bytes larger than for the ELS and should get its own case.

Quoted 33 lines
drivers/scsi/scsi_transport_fc.c +33 -0
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_rport_msg;
+		}
+		break;
+	default:
+		ret = -EBADR;
+		goto fail_rport_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_rport_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_rport_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+

<snip>

Quoted 12 lines
include/scsi/scsi_transport_fc.h +12 -0
+struct fc_bsg_job {
+	struct Scsi_Host *shost;
+	struct fc_rport *rport;
+	struct device *dev;
+	struct request *req;
+	spinlock_t job_lock;
+	unsigned int state_flags;
+	unsigned int ref_cnt;
+	void (*job_done)(struct fc_bsg_job *);
+
+	struct fc_bsg_request *request;
+	struct fc_bsg_reply *reply;

include/scsi/scsi_transport_fc.h struct fc_host_attrs { · line 616

Quoted 10 lines
include/scsi/scsi_transport_fc.h +10 -0
+	unsigned int request_len;
+	unsigned int reply_len;
+	/*
+	 * On entry : reply_len indicates the buffer size allocated for
+	 * the reply.
+	 *
+	 * Upon completion : the message handler must set reply_len
+	 *  to indicates the size of the reply to be returned to the
+	 *  caller.
+	 */

Do we need this field (reply_len)? Both the fc_bsg_buffer struct (for the vector i/o) as well as the fc_bsg_reply struct (for the sense data) contain fields for the payload length.

include/scsi/scsi_transport_fc.h struct fc_host_attrs { · line 623

Quoted 7 lines
include/scsi/scsi_transport_fc.h +7 -0
+
+	/* DMA payloads for the request/response */
+	struct fc_bsg_buffer request_payload;
+	struct fc_bsg_buffer reply_payload;
+
+	void *dd_data;			/* Used for driver-specific storage */
+};

Sven

#12 lore raw
Quoted 7 lines · Sven Schuetz
Quoted 3 lines · James Smart

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

So we need to add this to include/scsi/Kbuild, right?

yes, whatever file(s) control the inventory

Quoted 11 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3393

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:     fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+     struct request *req = job->req->next_rq;

Shouldn't this be just job->req?

yes

Quoted 28 lines
Quoted 13 lines
drivers/scsi/scsi_transport_fc.c +13 -0
+     struct request *rsp = req->next_rq;
+     unsigned long flags;
+     int err;
+
+     spin_lock_irqsave(&job->job_lock, flags);
+     job->state_flags |= FC_RQST_STATE_DONE;
+     job->ref_cnt--;
+     spin_unlock_irqrestore(&job->job_lock, flags);
+
+     err = job->req->errors = job->reply->result;
+     if (err < 0)
+             /* we're only returning the result field in the reply */
+             job->req->sense_len = sizeof(uint32_t);

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3416

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+     else
+             job->req->sense_len = job->reply_len;
+
+     /*
+      * we'll cheat: tell blk layer all of the xmt data was sent.
+      * but try to be honest about the amount of rcv data received
+      */
+     if (rsp)
+             blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+                                  job->reply->reply_payload_rcv_len);

Here is a problem when a LLD does not set job->reply->reply_payload_rcv_len, but has created a response.
blk_end_bidi_request will be called with a length of 0 for the last parameter and will thus return "1: still buffers pending for this request", which we do

Quoted 8 lines

not evaluate. A few lines later the job is destroyed. But a few seconds later the timeout kicks in, leading to a crash (I suppose because the job has been destroyed).
I see two options:
1. check if we have a response and see if a length has been set - but what do for cases which have a response and no length set? throw the request away?
2. always use the blk_rq_bytes(job->req->next_rq) from Seokmanns initial patch (which I prefer)

Good Questions..
- If the driver isn't setting reply_payload_rcv_len, isn't it a driver bug ? (which we don't work around).

- But, I can see a 0 length receive being a valid value as well, which implies the blk infrastructure can't deal with it - this should be fixed. How is this working with normal BIDI commands ?

- I tried to use the real length for the response, so that it tracked back through the calling sequences to bsg.c, so that it could return the length to the app. If we don't have the correct value being passed back - how does the app know how much data it actually received ? This is the flaw I see in choosing #2.

Quoted 14 lines
Quoted 14 lines
drivers/scsi/scsi_transport_fc.c +14 -0
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
+ * @shost:   scsi host rport attached to
+ * @rport:   rport request destined to
+ * @job:     bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+                      struct fc_rport *rport, struct fc_bsg_job *job)
+{
+     struct fc_internal *i = to_fc_internal(shost->transportt);
+     int cmdlen = sizeof(uint32_t);  /* start with length of msgcode */
+     int ret;
+
Quoted 8 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3664

Quoted 5 lines
drivers/scsi/scsi_transport_fc.c +5 -0
+     /* Validate the rport command */
+     switch (job->request->msgcode) {
+     case FC_BSG_RPT_ELS:
+     case FC_BSG_RPT_CT:
+             cmdlen += sizeof(struct fc_bsg_rport_els);

Why do we do the same here for ELS and CT? The struct fc_bsg_rport_ct is some bytes larger than for the ELS and should get its own case.

A bug...

Quoted 18 lines
Quoted 18 lines
include/scsi/scsi_transport_fc.h +18 -0
+struct fc_bsg_job {
+     struct Scsi_Host *shost;
+     struct fc_rport *rport;
+     struct device *dev;
+     struct request *req;
+     spinlock_t job_lock;
+     unsigned int state_flags;
+     unsigned int ref_cnt;
+     void (*job_done)(struct fc_bsg_job *);
+
+     struct fc_bsg_request *request;
+     struct fc_bsg_reply *reply;
+     unsigned int request_len;
+     unsigned int reply_len;
+     /*
+      * On entry : reply_len indicates the buffer size allocated for
+      * the reply.
+      *
Quoted 8 lines

include/scsi/scsi_transport_fc.h struct fc_host_attrs { · line 616

Quoted 4 lines
include/scsi/scsi_transport_fc.h +4 -0
+      * Upon completion : the message handler must set reply_len
+      *  to indicates the size of the reply to be returned to the
+      *  caller.
+      */

Do we need this field (reply_len)? Both the fc_bsg_buffer struct (for the vector i/o) as well as the fc_bsg_reply struct (for the sense data) contain fields for the payload length.

Yes. This isn't the "dma reply" length, but rather the amount of the sense buffer that is returned with "reply" or status information for the the actual request. E.g. upon finishing a job, the driver should always set this to sizeof(struct fc_bsg_ctels_reply), or whatever the reply structure is to be.

-- james

#13 lore raw
In reply to James Smart
Quoted 77 lines · On Tue, 18 Nov 2008 16:24:58 -0500 James Smart <[email protected]> wrote:

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   53 ++
 6 files changed, 946 insertions(+), 4 deletions(-)

(snip)

Quoted 24 lines
drivers/scsi/scsi_transport_fc.c +24 -0
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req->next_rq;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3418

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));

I think that you can use blk_end_bidi_request() for non-bidi requests:

	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
			rsp ?
			 job->reply->reply_payload_rcv_len : 0);

I guess that it would be better to have one function to complete a request, instead of blk_end_bidi_request and blk_end_request.

Quoted 46 lines
drivers/scsi/scsi_transport_fc.c +46 -0
+	fc_destroy_bsgjob(job);
+}
+
+
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:	request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+	struct fc_bsg_job *job = (void *) req->special;
+	struct Scsi_Host *shost = job->shost;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	unsigned long flags;
+	int err = 0, done = 0;
+
+	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
+		return BLK_EH_RESET_TIMER;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->state_flags & FC_RQST_STATE_DONE)
+		done = 1;
+	else
+		job->ref_cnt++;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	if (!done && i->f->bsg_timeout) {
+		/* call LLDD to abort the i/o as it has timed out */
+		err = i->f->bsg_timeout(job);
+		if (err)
+			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
+				"abort failed with status %d\n", err);
+	}
+
+	if (!done) {
+		spin_lock_irqsave(&job->job_lock, flags);
+		job->ref_cnt--;
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		fc_destroy_bsgjob(job);
+	}
+
+	/* the blk_end_sync_io() doesn't check the error */
+	return BLK_EH_HANDLED;
+}
+

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3475

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+
+static void
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);

Needs to handle ENOMEM?

Quoted 243 lines
drivers/scsi/scsi_transport_fc.c +243 -0
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+}
+
+
+/**
+ * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for the
+ *                   bsg request
+ * @shost:	SCSI Host corresponding to the bsg object
+ * @rport:	(optional) FC Remote Port corresponding to the bsg object
+ * @req:	BSG request that needs a job structure
+ */
+static int
+fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
+	struct request *req)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request *rsp = req->next_rq;
+	struct fc_bsg_job *job;
+
+	BUG_ON(req->special);
+
+	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
+			GFP_KERNEL);
+	if (!job)
+		return -ENOMEM;
+
+	/*
+	 * Note: this is a bit silly.
+	 * The request gets formatted as a SGIO v4 ioctl request, which
+	 * then gets reformatted as a blk request, which then gets
+	 * reformatted as a fc bsg request. And on completion, we have
+	 * to wrap return results such that SGIO v4 thinks it was a scsi
+	 * status.  I hope this was all worth it.
+	 */
+
+	req->special = job;
+	job->shost = shost;
+	job->rport = rport;
+	job->req = req;
+	if (i->f->dd_bsg_size)
+		job->dd_data = (void *)&job[1];
+	spin_lock_init(&job->job_lock);
+	job->request = (struct fc_bsg_request *)req->cmd;
+	job->request_len = req->cmd_len;
+	job->reply = req->sense;
+	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
+						 * allocated */
+	if (req->bio)
+		fc_bsg_map_buffer(&job->request_payload, req);
+	if (rsp && rsp->bio)
+		fc_bsg_map_buffer(&job->reply_payload, rsp);
+	job->job_done = fc_bsg_jobdone;
+	if (rport)
+		job->dev = &rport->dev;
+	else
+		job->dev = &shost->shost_gendev;
+	get_device(job->dev);		/* take a reference for the request */
+
+	job->ref_cnt = 1;
+
+	return 0;
+}
+
+
+enum fc_dispatch_result {
+	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
+	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
+	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
+};
+
+
+/**
+ * fc_bsg_host_dispatch - process fc host bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the host command */
+	switch (job->request->msgcode) {
+	case FC_BSG_HST_ADD_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_add_rport);
+		break;
+
+	case FC_BSG_HST_DEL_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_del_rport);
+		break;
+
+	case FC_BSG_HST_ELS_NOLOGIN:
+		cmdlen += sizeof(struct fc_bsg_host_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_host_msg;
+		}
+		break;
+
+	case FC_BSG_HST_VENDOR:
+		cmdlen += sizeof(struct fc_bsg_host_vendor);
+		if ((shost->hostt->vendor_id == 0L) || 
+		    (job->request->rqst_data.h_vendor.vendor_id !=
+			shost->hostt->vendor_id)) {
+			ret = -ESRCH;
+			goto fail_host_msg;
+		}
+		break;
+
+	default:
+		ret = -EBADR;
+		goto fail_host_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_host_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_host_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/*
+ * fc_bsg_goose_queue - restart rport queue in case it was stopped
+ * @rport:	rport to be restarted
+ */
+static void
+fc_bsg_goose_queue(struct fc_rport *rport)
+{
+	int flagset;
+
+	if (!rport->rqst_q)
+		return;
+
+	get_device(&rport->dev);
+	
+	spin_lock(rport->rqst_q->queue_lock);
+	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags) &&
+		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
+	if (flagset)
+		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
+	__blk_run_queue(rport->rqst_q);
+	if (flagset)
+		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
+	spin_unlock(rport->rqst_q->queue_lock);
+
+	put_device(&rport->dev);
+}
+
+
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @rport:	rport request destined to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_rport *rport, struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the rport command */
+	switch (job->request->msgcode) {
+	case FC_BSG_RPT_ELS:
+	case FC_BSG_RPT_CT:
+		cmdlen += sizeof(struct fc_bsg_rport_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_rport_msg;
+		}
+		break;
+	default:
+		ret = -EBADR;
+		goto fail_rport_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_rport_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_rport_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:		request queue to manage
+ * @shost:	Scsi_Host related to the bsg object
+ * @rport:	FC remote port related to the bsg object (optional)
+ * @dev:	device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host *shost,
+		       struct fc_rport *rport, struct device *dev)
+{
+	struct request *req;
+	struct fc_bsg_job *job;
+	enum fc_dispatch_result ret;
+
+	if (!get_device(dev))
+		return;
+
+	while (!blk_queue_plugged(q)) {
+		req = elv_next_request(q);
+		if (!req)

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3728

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+			break;
+
+		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
+				break;
+
+		blkdev_dequeue_request(req);
+
+		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
+			req->errors = -ENXIO;
+			blk_complete_request(req);

I think that calling blk_complete_request() here crashes your kernel since you don't set q->softirq_done_fn.

Calling blk_end_bidi_request should work here. Note that you need to call spin_unlock_irq(q->queue_lock) before it.

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3738

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+			continue;
+		}
+
+		spin_unlock_irq(q->queue_lock);
+
+		ret = fc_req_to_bsgjob(shost, rport, req);
+		if (ret) {
+			req->errors = ret;
+			spin_lock_irq(q->queue_lock);
+			blk_complete_request(req);

Ditto.

Quoted 73 lines
drivers/scsi/scsi_transport_fc.c +73 -0
+			continue;
+		}
+
+		job = req->special;
+
+		/* check if we have the msgcode value at least */
+		if (job->request_len < sizeof(uint32_t)) {
+			BUG_ON(job->reply_len < sizeof(uint32_t));
+			job->reply->result = -ENOMSG;
+			job->reply_len = sizeof(uint32_t);
+			fc_bsg_jobdone(job);
+			spin_lock_irq(q->queue_lock);
+			continue;
+		}
+		
+		/* the dispatch routines will unlock the queue_lock */
+		if (rport)
+			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
+		else
+			ret = fc_bsg_host_dispatch(q, shost, job);
+
+		/* did dispatcher hit state that can't process any more */
+		if (ret == FC_DISPATCH_BREAK)
+			break;
+
+		/* did dispatcher had released the lock */
+		if (ret == FC_DISPATCH_UNLOCKED)
+			spin_lock_irq(q->queue_lock);
+	}
+
+	spin_unlock_irq(q->queue_lock);
+	put_device(dev);
+	spin_lock_irq(q->queue_lock);
+}
+
+
+/**
+ * fc_bsg_host_handler - handler for bsg requests for a fc host
+ * @q:		fc host request queue 
+ */
+static void
+fc_bsg_host_handler(struct request_queue *q)
+{
+	struct Scsi_Host *shost = q->queuedata;
+
+	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
+}
+
+
+/**
+ * fc_bsg_rport_handler - handler for bsg requests for a fc rport
+ * @q:		rport request queue 
+ */
+static void
+fc_bsg_rport_handler(struct request_queue *q)
+{
+	struct fc_rport *rport = q->queuedata;
+	struct Scsi_Host *shost = rport_to_shost(rport);
+
+	fc_bsg_request_handler(q, shost, rport, &rport->dev);
+}
+
+
+/**
+ * fc_bsg_hostadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost for fc_host
+ * @fc_host:	fc_host adding the structures to
+ */
+static int
+fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs *fc_host)
+{
+	struct device *dev = &shost->shost_gendev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3821

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+	struct request_queue *q;
+	int err;
+
+	fc_host->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	snprintf(fc_host->bsg_name, sizeof(fc_host->bsg_name),
+		 "fc_host%d", shost->host_no);

You don't need fc_host->bsg_name. bsg_register_queue() allocates memory for the name.

Quoted 49 lines
drivers/scsi/scsi_transport_fc.c +49 -0
+	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
+	if (!q) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - no request queue\n",
+				 shost->host_no);
+		return -ENOMEM;
+	} 
+
+	q->queuedata = shost;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, fc_host->bsg_name, NULL);
+	if (err) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - register queue\n",
+			 	shost->host_no);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	fc_host->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_rportadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost that rport is attached to
+ * @rport:	rport that the bsg hooks are being attached to
+ */
+static int
+fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
+{
+	struct device *dev = &rport->dev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+
+	rport->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
+	if (!q) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - no request queue\n",

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3881

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+				 dev->bus_id);
+		return -ENOMEM;
+	} 
+
+	q->queuedata = rport;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, dev->bus_id, NULL);

You don't need to pass dev->bus_id. It is used if you pass NULL.

bsg_register_queue(q, dev, NULL, NULL);

Quoted 54 lines
drivers/scsi/scsi_transport_fc.c +27 -0
+	if (err) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - register queue\n",
+				 dev->bus_id);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	rport->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
+ * @q:	the request_queue that is to be torn down.
+ */
+static void
+fc_bsg_remove(struct request_queue *q)
+{
+	if (q) {
+		bsg_unregister_queue(q);
+		blk_cleanup_queue(q);
+	}
+}
+
+
 /* Original Author:  Martin Hicks */
 MODULE_AUTHOR("James Smart");
 MODULE_DESCRIPTION("FC Transport Attributes");

diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h

include/scsi/scsi_bsg_fc.h +20 -0
@@ -0,0 +1,291 @@
+/*
+ *  FC Transport BSG Interface
+ *
+ *  Copyright (C) 2008   James Smart, Emulex Corporation
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ *
+ */

include/scsi/scsi_bsg_fc.h line 30

Quoted 10 lines
include/scsi/scsi_bsg_fc.h +10 -0
+
+#ifndef SCSI_BSG_FC_H
+#define SCSI_BSG_FC_H
+
+/*
+ * This file intended to be included by both kernel and user space
+ */
+
+#include <scsi/scsi.h>
+#include <scsi/sg.h>

<scsi/sg.h> is necessary?

include/scsi/scsi_bsg_fc.h line 37

Quoted 6 lines
include/scsi/scsi_bsg_fc.h +6 -0
+/*
+ * FC Transport SGIO v4 BSG Message Support
+ */
+
+/* Default BSG request timeout (in seconds) */
+#define FC_DEFAULT_BSG_TIMEOUT		10

Should be (10 * HZ)?

Quoted 380 lines
include/scsi/scsi_bsg_fc.h +253 -0
+
+/*
+ * Request Message Codes supported by the FC Transport
+ */
+
+/* define the class masks for the message codes */
+#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
+#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
+#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
+
+	/* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
+#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
+#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
+
+	/* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
+#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
+
+
+
+/*
+ * FC Address Identifiers in Message Structures :
+ *
+ *   Whenever a command payload contains a FC Address Identifier
+ *   (aka port_id), the value is effectively in big-endian
+ *   order, thus the array elements are decoded as follows:
+ *     element [0] is bits 23:16 of the FC Address Identifier
+ *     element [1] is bits 15:8 of the FC Address Identifier
+ *     element [2] is bits 7:0 of the FC Address Identifier
+ */
+
+
+/*
+ * FC Host Messages
+ */
+
+/* FC_BSG_HST_ADDR_PORT : */
+
+/* Request:
+ * This message requests the FC host to login to the remote port
+ * at the specified N_Port_Id.  The remote port is to be enumerated
+ * with the transport upon completion of the login.
+ */
+struct fc_bsg_host_add_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to login to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient
+ */
+
+
+/* FC_BSG_HST_DEL_RPORT : */
+
+/* Request:
+ * This message requests the FC host to remove an enumerated
+ * remote port and to terminate the login to it.
+ *
+ * Note: The driver is free to reject this request if it desires to
+ * remain logged in with the remote port.
+ */
+struct fc_bsg_host_del_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to logout of */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient 
+ */
+
+
+/* FC_BSG_HST_ELS_NOLOGIN : */
+
+/* Request:
+ * This message requests the FC_Host to send an ELS to a specific
+ * N_Port_ID. The host does not need to log into the remote port,
+ * nor does it need to enumerate the rport for further traffic
+ * (although, the FC host is free to do so if it desires).
+ */
+struct fc_bsg_host_els {
+	/* 
+	 * ELS Command Code being sent (must be the same as byte 0
+	 * of the payload)
+	 */
+	uint8_t 	command_code;
+
+	/* FC Address Identier of the remote port to send the ELS to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ */
+/* fc_bsg_ctels_reply->status values */
+#define FC_CTELS_STATUS_OK	0x00000000
+#define FC_CTELS_STATUS_REJECT	0x00000001
+#define FC_CTELS_STATUS_P_RJT	0x00000002
+#define FC_CTELS_STATUS_F_RJT	0x00000003
+#define FC_CTELS_STATUS_P_BSY	0x00000004
+#define FC_CTELS_STATUS_F_BSY	0x00000006
+struct fc_bsg_ctels_reply {
+	/*
+	 * Note: An ELS LS_RJT may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the ELS receive payload to determine
+	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
+	 *     data will be in word 1.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
+	 *   the receive payload word 0 indicates LS_ACC
+	 *   (e.g. value is 0x02xxxxxx).
+	 *
+	 * Note: Similarly, a CT Reject may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the CT receive payload to determine
+	 *     Accept or Reject (by contents of word 2). The reject
+	 *     data will be in word 3.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: x_RJT/BSY status will indicae that the rjt_data field
+	 *   is valid and contains the reason/explanation values.
+	 */
+	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
+
+	/* valid if status is not FC_CTELS_STATUS_OK */
+	struct	{
+		uint8_t	action;		/* fragment_id for CT REJECT */
+		uint8_t	reason_code;
+		uint8_t	reason_explanation;
+		uint8_t	vendor_unique;
+	} rjt_data;
+};
+
+
+/* FC_BSG_HST_VENDOR : */
+
+/* Request:
+ * Note: When specifying vendor_id, be sure to read the Vendor Type and ID
+ *   formatting requirements specified in scsi_netlink.h
+ */
+struct fc_bsg_host_vendor {
+	/*
+	 * Identifies the vendor that the message is formatted for. This
+	 * should be the recipient of the message.
+	 */
+	uint64_t vendor_id;
+
+	/* start of vendor command area */
+	uint32_t vendor_cmd[0];
+};
+
+/* Response:
+ */
+struct fc_bsg_host_vendor_reply {
+	/* start of vendor response area */
+	uint32_t vendor_rsp[0];
+};
+
+
+
+/*
+ * FC Remote Port Messages
+ */
+
+/* FC_BSG_RPT_ELS : */
+
+/* Request:
+ * This message requests that an ELS be performed with the rport.
+ */
+struct fc_bsg_rport_els {
+	/* 
+	 * ELS Command Code being sent (must be the same as
+	 * byte 0 of the payload)
+	 */
+	uint8_t els_code;
+};
+
+/* Response:
+ * 
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_RPT_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the rport.
+ */
+struct fc_bsg_rport_ct {
+	/* 
+	 * We need words 0-2 of the generic preamble for the LLD's
+	 */
+	uint32_t	preamble_word0;	/* revision & IN_ID */
+	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
+	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+
+
+/* request (CDB) structure of the sg_io_v4 */
+struct fc_bsg_request {
+	uint32_t msgcode;
+	union {
+		struct fc_bsg_host_add_rport	h_addrport;
+		struct fc_bsg_host_del_rport	h_delrport;
+		struct fc_bsg_host_els	h_els;
+		struct fc_bsg_host_vendor	h_vendor;
+
+		struct fc_bsg_rport_els		r_els;
+		struct fc_bsg_rport_ct		r_ct;
+	} rqst_data;
+};
+
+
+/* response (request sense data) structure of the sg_io_v4 */
+struct fc_bsg_reply {
+	/*
+	 * The completion result. Result exists in two forms:
+	 *  if negative, it is an -Exxx system errno value. There will
+	 *    be no further reply information supplied.
+	 *  else, it's the 4-byte scsi error result, with driver, host,
+	 *    msg and status fields. The per-msgcode reply structure
+	 *    will contain valid data.
+	 */
+	uint32_t result;
+
+	/* If there was reply_payload, how much was recevied ? */
+	uint32_t reply_payload_rcv_len;
+
+	union {
+		struct fc_bsg_host_vendor_reply		vendor_reply;
+
+		struct fc_bsg_ctels_reply		ctels_reply;
+	} reply_data;
+};
+
+
+#endif /* SCSI_BSG_FC_H */
+

diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h

include/scsi/scsi_host.h +9 -0
@@ -478,6 +478,15 @@ struct scsi_host_template {
 	 * module_init/module_exit.
 	 */
 	struct list_head legacy_hosts;
+
+	/*
+	 * Vendor Identifier associated with the host
+	 *
+	 * Note: When specifying vendor_id, be sure to read the
+	 *   Vendor Type and ID formatting requirements specified in
+	 *   scsi_netlink.h
+	 */
+	u64 vendor_id;
 };
 
 /*

diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h

include/scsi/scsi_transport_fc.h +51 -3
@@ -33,7 +33,6 @@
 
 struct scsi_transport_template;
 
-
 /*
  * FC Port definitions - Following FC HBAAPI guidelines
  *
@@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
  	struct delayed_work fail_io_work;
  	struct work_struct stgt_delete_work;
 	struct work_struct rport_delete_work;
+	struct request_queue *rqst_q;	/* bsg support */
 } __attribute__((aligned(sizeof(unsigned long))));
 
 /* bit field values for struct fc_rport "flags" field: */
@@ -513,6 +513,10 @@ struct fc_host_attrs {
 	struct workqueue_struct *work_q;
 	char devloss_work_q_name[20];
 	struct workqueue_struct *devloss_work_q;
+
+	/* bsg support */
+	char bsg_name[20];
+	struct request_queue *rqst_q;
 };
 
 #define shost_to_fc_host(x) \
@@ -578,6 +582,47 @@ struct fc_host_attrs {
 	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
 
 
+struct fc_bsg_buffer {
+	unsigned int payload_len;
+	int sg_cnt;
+	struct scatterlist *sg_list;
+};
+
+/* Values for fc_bsg_job->state_flags (bitflags) */
+#define FC_RQST_STATE_INPROGRESS	0
+#define FC_RQST_STATE_DONE		1
+
+struct fc_bsg_job {
+	struct Scsi_Host *shost;
+	struct fc_rport *rport;
+	struct device *dev;
+	struct request *req;
+	spinlock_t job_lock;
+	unsigned int state_flags;
+	unsigned int ref_cnt;
+	void (*job_done)(struct fc_bsg_job *);
+
+	struct fc_bsg_request *request;
+	struct fc_bsg_reply *reply;
+	unsigned int request_len;
+	unsigned int reply_len;
+	/*
+	 * On entry : reply_len indicates the buffer size allocated for
+	 * the reply.
+	 *
+	 * Upon completion : the message handler must set reply_len
+	 *  to indicates the size of the reply to be returned to the
+	 *  caller.
+	 */
+
+	/* DMA payloads for the request/response */
+	struct fc_bsg_buffer request_payload;
+	struct fc_bsg_buffer reply_payload;
+
+	void *dd_data;			/* Used for driver-specific storage */
+};
+
+
 /* The functions by which the transport class and the driver communicate */
 struct fc_function_template {
 	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
@@ -613,9 +658,14 @@ struct fc_function_template {
 	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
 	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
 
+	/* bsg support */
+	int	(* bsg_request)(struct fc_bsg_job *);
+	int	(* bsg_timeout)(struct fc_bsg_job *);
+
 	/* allocation lengths for host-specific data */
 	u32	 			dd_fcrport_size;
 	u32	 			dd_fcvport_size;
+	u32				dd_bsg_size;
 
 	/*
 	 * The driver sets these to tell the transport class it
@@ -736,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
 	vport->vport_state = new_state;
 }
 
-
 struct scsi_transport_template *fc_attach_transport(
 			struct fc_function_template *);
 void fc_release_transport(struct scsi_transport_template *);
 
 
 
--

To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#14 lore raw
Quoted 83 lines · FUJITA Tomonori
Quoted 77 lines · On Tue, 18 Nov 2008 16:24:58 -0500 James Smart <[email protected]> wrote:

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   53 ++
 6 files changed, 946 insertions(+), 4 deletions(-)

(snip)

Quoted 12 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3396

Quoted 12 lines
drivers/scsi/scsi_transport_fc.c +12 -0
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req->next_rq;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	int err;

+ unsigned bytes_requested = 0;

Quoted 25 lines
Quoted 12 lines
drivers/scsi/scsi_transport_fc.c +12 -0
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3418

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));

I think that you can use blk_end_bidi_request() for non-bidi requests:

Quoted 9 lines
	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
			rsp ?
			 job->reply->reply_payload_rcv_len : 0);

I guess that it would be better to have one function to complete a request, instead of blk_end_bidi_request and blk_end_request.

+ /*
+ * tell blk layer all of the xmt data was sent.
+ * but set residual count to: requested - received
+ */
+
+ if (rsp) {
+ bytes_requested = blk_rq_bytes(rsp);
+ rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
+ }
+
+ blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);

The residual count is left in req->data_len. Does bsg have a way to return the residual to user-mode? It must, since Pete was using that for sure. Note that you are looking for the bidi_read residual count.

As was said by people. You must complete ALL bytes on both sides. Residual information is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

Quoted 321 lines
Quoted 46 lines
drivers/scsi/scsi_transport_fc.c +46 -0
+	fc_destroy_bsgjob(job);
+}
+
+
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:	request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+	struct fc_bsg_job *job = (void *) req->special;
+	struct Scsi_Host *shost = job->shost;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	unsigned long flags;
+	int err = 0, done = 0;
+
+	if (job->rport && job->rport->port_state == FC_PORTSTATE_BLOCKED)
+		return BLK_EH_RESET_TIMER;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->state_flags & FC_RQST_STATE_DONE)
+		done = 1;
+	else
+		job->ref_cnt++;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	if (!done && i->f->bsg_timeout) {
+		/* call LLDD to abort the i/o as it has timed out */
+		err = i->f->bsg_timeout(job);
+		if (err)
+			printk(KERN_ERR "ERROR: FC BSG request timeout - LLD "
+				"abort failed with status %d\n", err);
+	}
+
+	if (!done) {
+		spin_lock_irqsave(&job->job_lock, flags);
+		job->ref_cnt--;
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		fc_destroy_bsgjob(job);
+	}
+
+	/* the blk_end_sync_io() doesn't check the error */
+	return BLK_EH_HANDLED;
+}
+

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3475

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+
+static void
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);

Needs to handle ENOMEM?

Quoted 243 lines
drivers/scsi/scsi_transport_fc.c +243 -0
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+}
+
+
+/**
+ * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for the
+ *                   bsg request
+ * @shost:	SCSI Host corresponding to the bsg object
+ * @rport:	(optional) FC Remote Port corresponding to the bsg object
+ * @req:	BSG request that needs a job structure
+ */
+static int
+fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
+	struct request *req)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request *rsp = req->next_rq;
+	struct fc_bsg_job *job;
+
+	BUG_ON(req->special);
+
+	job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
+			GFP_KERNEL);
+	if (!job)
+		return -ENOMEM;
+
+	/*
+	 * Note: this is a bit silly.
+	 * The request gets formatted as a SGIO v4 ioctl request, which
+	 * then gets reformatted as a blk request, which then gets
+	 * reformatted as a fc bsg request. And on completion, we have
+	 * to wrap return results such that SGIO v4 thinks it was a scsi
+	 * status.  I hope this was all worth it.
+	 */
+
+	req->special = job;
+	job->shost = shost;
+	job->rport = rport;
+	job->req = req;
+	if (i->f->dd_bsg_size)
+		job->dd_data = (void *)&job[1];
+	spin_lock_init(&job->job_lock);
+	job->request = (struct fc_bsg_request *)req->cmd;
+	job->request_len = req->cmd_len;
+	job->reply = req->sense;
+	job->reply_len = SCSI_SENSE_BUFFERSIZE;	/* Size of sense buffer
+						 * allocated */
+	if (req->bio)
+		fc_bsg_map_buffer(&job->request_payload, req);
+	if (rsp && rsp->bio)
+		fc_bsg_map_buffer(&job->reply_payload, rsp);
+	job->job_done = fc_bsg_jobdone;
+	if (rport)
+		job->dev = &rport->dev;
+	else
+		job->dev = &shost->shost_gendev;
+	get_device(job->dev);		/* take a reference for the request */
+
+	job->ref_cnt = 1;
+
+	return 0;
+}
+
+
+enum fc_dispatch_result {
+	FC_DISPATCH_BREAK,	/* on return, q is locked, break from q loop */
+	FC_DISPATCH_LOCKED,	/* on return, q is locked, continue on */
+	FC_DISPATCH_UNLOCKED,	/* on return, q is unlocked, continue on */
+};
+
+
+/**
+ * fc_bsg_host_dispatch - process fc host bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the host command */
+	switch (job->request->msgcode) {
+	case FC_BSG_HST_ADD_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_add_rport);
+		break;
+
+	case FC_BSG_HST_DEL_RPORT:
+		cmdlen += sizeof(struct fc_bsg_host_del_rport);
+		break;
+
+	case FC_BSG_HST_ELS_NOLOGIN:
+		cmdlen += sizeof(struct fc_bsg_host_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_host_msg;
+		}
+		break;
+
+	case FC_BSG_HST_VENDOR:
+		cmdlen += sizeof(struct fc_bsg_host_vendor);
+		if ((shost->hostt->vendor_id == 0L) || 
+		    (job->request->rqst_data.h_vendor.vendor_id !=
+			shost->hostt->vendor_id)) {
+			ret = -ESRCH;
+			goto fail_host_msg;
+		}
+		break;
+
+	default:
+		ret = -EBADR;
+		goto fail_host_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_host_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_host_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/*
+ * fc_bsg_goose_queue - restart rport queue in case it was stopped
+ * @rport:	rport to be restarted
+ */
+static void
+fc_bsg_goose_queue(struct fc_rport *rport)
+{
+	int flagset;
+
+	if (!rport->rqst_q)
+		return;
+
+	get_device(&rport->dev);
+	
+	spin_lock(rport->rqst_q->queue_lock);
+	flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags) &&
+		  !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q->queue_flags);
+	if (flagset)
+		queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
+	__blk_run_queue(rport->rqst_q);
+	if (flagset)
+		queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
+	spin_unlock(rport->rqst_q->queue_lock);
+
+	put_device(&rport->dev);
+}
+
+
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch to LLDD
+ * @shost:	scsi host rport attached to
+ * @rport:	rport request destined to
+ * @job:	bsg job to be processed
+ */
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host *shost,
+			 struct fc_rport *rport, struct fc_bsg_job *job)
+{
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	int cmdlen = sizeof(uint32_t);	/* start with length of msgcode */
+	int ret;
+
+	/* Validate the rport command */
+	switch (job->request->msgcode) {
+	case FC_BSG_RPT_ELS:
+	case FC_BSG_RPT_CT:
+		cmdlen += sizeof(struct fc_bsg_rport_els);
+		/* there better be a xmt and rcv payloads */
+		if ((!job->request_payload.payload_len) ||
+		    (!job->reply_payload.payload_len)) {
+			ret = -EINVAL;
+			goto fail_rport_msg;
+		}
+		break;
+	default:
+		ret = -EBADR;
+		goto fail_rport_msg;
+	}
+
+	/* check if we really have all the request data needed */
+	if (job->request_len < cmdlen) {
+		ret = -ENOMSG;
+		goto fail_rport_msg;
+	}
+
+	ret = i->f->bsg_request(job);
+	if (ret) {
+fail_rport_msg:
+		/* return the errno failure code as the only status */
+		BUG_ON(job->reply_len < sizeof(uint32_t));
+		job->reply->result = ret;
+		job->reply_len = sizeof(uint32_t);
+		fc_bsg_jobdone(job);
+		/* fall thru */
+	}
+
+	return FC_DISPATCH_UNLOCKED;
+}
+
+
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:		request queue to manage
+ * @shost:	Scsi_Host related to the bsg object
+ * @rport:	FC remote port related to the bsg object (optional)
+ * @dev:	device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host *shost,
+		       struct fc_rport *rport, struct device *dev)
+{
+	struct request *req;
+	struct fc_bsg_job *job;
+	enum fc_dispatch_result ret;
+
+	if (!get_device(dev))
+		return;
+
+	while (!blk_queue_plugged(q)) {
+		req = elv_next_request(q);
+		if (!req)

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3728

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+			break;
+
+		if (rport && (rport->port_state == FC_PORTSTATE_BLOCKED))
+				break;
+
+		blkdev_dequeue_request(req);
+
+		if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
+			req->errors = -ENXIO;
+			blk_complete_request(req);

I think that calling blk_complete_request() here crashes your kernel since you don't set q->softirq_done_fn.

Calling blk_end_bidi_request should work here. Note that you need to call spin_unlock_irq(q->queue_lock) before it.

Quoted 13 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3738

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+			continue;
+		}
+
+		spin_unlock_irq(q->queue_lock);
+
+		ret = fc_req_to_bsgjob(shost, rport, req);
+		if (ret) {
+			req->errors = ret;
+			spin_lock_irq(q->queue_lock);
+			blk_complete_request(req);

Ditto.

I would suggest an fc_complete_request() defined that would do that.

Quoted 615 lines
Quoted 73 lines
drivers/scsi/scsi_transport_fc.c +73 -0
+			continue;
+		}
+
+		job = req->special;
+
+		/* check if we have the msgcode value at least */
+		if (job->request_len < sizeof(uint32_t)) {
+			BUG_ON(job->reply_len < sizeof(uint32_t));
+			job->reply->result = -ENOMSG;
+			job->reply_len = sizeof(uint32_t);
+			fc_bsg_jobdone(job);
+			spin_lock_irq(q->queue_lock);
+			continue;
+		}
+		
+		/* the dispatch routines will unlock the queue_lock */
+		if (rport)
+			ret = fc_bsg_rport_dispatch(q, shost, rport, job);
+		else
+			ret = fc_bsg_host_dispatch(q, shost, job);
+
+		/* did dispatcher hit state that can't process any more */
+		if (ret == FC_DISPATCH_BREAK)
+			break;
+
+		/* did dispatcher had released the lock */
+		if (ret == FC_DISPATCH_UNLOCKED)
+			spin_lock_irq(q->queue_lock);
+	}
+
+	spin_unlock_irq(q->queue_lock);
+	put_device(dev);
+	spin_lock_irq(q->queue_lock);
+}
+
+
+/**
+ * fc_bsg_host_handler - handler for bsg requests for a fc host
+ * @q:		fc host request queue 
+ */
+static void
+fc_bsg_host_handler(struct request_queue *q)
+{
+	struct Scsi_Host *shost = q->queuedata;
+
+	fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
+}
+
+
+/**
+ * fc_bsg_rport_handler - handler for bsg requests for a fc rport
+ * @q:		rport request queue 
+ */
+static void
+fc_bsg_rport_handler(struct request_queue *q)
+{
+	struct fc_rport *rport = q->queuedata;
+	struct Scsi_Host *shost = rport_to_shost(rport);
+
+	fc_bsg_request_handler(q, shost, rport, &rport->dev);
+}
+
+
+/**
+ * fc_bsg_hostadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost for fc_host
+ * @fc_host:	fc_host adding the structures to
+ */
+static int
+fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs *fc_host)
+{
+	struct device *dev = &shost->shost_gendev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3821

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+	struct request_queue *q;
+	int err;
+
+	fc_host->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	snprintf(fc_host->bsg_name, sizeof(fc_host->bsg_name),
+		 "fc_host%d", shost->host_no);

You don't need fc_host->bsg_name. bsg_register_queue() allocates memory for the name.

Quoted 49 lines
drivers/scsi/scsi_transport_fc.c +49 -0
+	q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
+	if (!q) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - no request queue\n",
+				 shost->host_no);
+		return -ENOMEM;
+	} 
+
+	q->queuedata = shost;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, fc_host->bsg_name, NULL);
+	if (err) {
+		printk(KERN_ERR "fc_host%d: bsg interface failed to "
+				"initialize - register queue\n",
+			 	shost->host_no);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	fc_host->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_rportadd - Create and add the bsg hooks so we can receive requests
+ * @shost:	shost that rport is attached to
+ * @rport:	rport that the bsg hooks are being attached to
+ */
+static int
+fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
+{
+	struct device *dev = &rport->dev;
+	struct fc_internal *i = to_fc_internal(shost->transportt);
+	struct request_queue *q;
+	int err;
+
+	rport->rqst_q = NULL;
+
+	if (!i->f->bsg_request)
+		return -ENOTSUPP;
+
+	q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
+	if (!q) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - no request queue\n",

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3881

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+				 dev->bus_id);
+		return -ENOMEM;
+	} 
+
+	q->queuedata = rport;
+	queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+	blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+	blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+	err = bsg_register_queue(q, dev, dev->bus_id, NULL);

You don't need to pass dev->bus_id. It is used if you pass NULL.

bsg_register_queue(q, dev, NULL, NULL);

Quoted 54 lines
drivers/scsi/scsi_transport_fc.c +27 -0
+	if (err) {
+		printk(KERN_ERR "%s: bsg interface failed to "
+				"initialize - register queue\n",
+				 dev->bus_id);
+		blk_cleanup_queue(q);
+		return err;
+	}
+
+	rport->rqst_q = q;
+	return 0;
+}
+
+
+/**
+ * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
+ * @q:	the request_queue that is to be torn down.
+ */
+static void
+fc_bsg_remove(struct request_queue *q)
+{
+	if (q) {
+		bsg_unregister_queue(q);
+		blk_cleanup_queue(q);
+	}
+}
+
+
 /* Original Author:  Martin Hicks */
 MODULE_AUTHOR("James Smart");
 MODULE_DESCRIPTION("FC Transport Attributes");

diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h

include/scsi/scsi_bsg_fc.h +20 -0
@@ -0,0 +1,291 @@
+/*
+ *  FC Transport BSG Interface
+ *
+ *  Copyright (C) 2008   James Smart, Emulex Corporation
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ *
+ */

include/scsi/scsi_bsg_fc.h line 30

Quoted 10 lines
include/scsi/scsi_bsg_fc.h +10 -0
+
+#ifndef SCSI_BSG_FC_H
+#define SCSI_BSG_FC_H
+
+/*
+ * This file intended to be included by both kernel and user space
+ */
+
+#include <scsi/scsi.h>
+#include <scsi/sg.h>

<scsi/sg.h> is necessary?

include/scsi/scsi_bsg_fc.h line 37

Quoted 6 lines
include/scsi/scsi_bsg_fc.h +6 -0
+/*
+ * FC Transport SGIO v4 BSG Message Support
+ */
+
+/* Default BSG request timeout (in seconds) */
+#define FC_DEFAULT_BSG_TIMEOUT		10

Should be (10 * HZ)?

Quoted 377 lines
include/scsi/scsi_bsg_fc.h +253 -0
+
+/*
+ * Request Message Codes supported by the FC Transport
+ */
+
+/* define the class masks for the message codes */
+#define FC_BSG_CLS_MASK		0xF0000000	/* find object class */
+#define FC_BSG_HST_MASK		0x80000000	/* fc host class */
+#define FC_BSG_RPT_MASK		0x40000000	/* fc rport class */
+
+	/* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT		(FC_BSG_HST_MASK | 0x00000001)
+#define FC_BSG_HST_DEL_RPORT		(FC_BSG_HST_MASK | 0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN		(FC_BSG_HST_MASK | 0x00000003)
+#define FC_BSG_HST_VENDOR		(FC_BSG_HST_MASK | 0x000000FF)
+
+	/* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS			(FC_BSG_RPT_MASK | 0x00000001)
+#define FC_BSG_RPT_CT			(FC_BSG_RPT_MASK | 0x00000002)
+
+
+
+/*
+ * FC Address Identifiers in Message Structures :
+ *
+ *   Whenever a command payload contains a FC Address Identifier
+ *   (aka port_id), the value is effectively in big-endian
+ *   order, thus the array elements are decoded as follows:
+ *     element [0] is bits 23:16 of the FC Address Identifier
+ *     element [1] is bits 15:8 of the FC Address Identifier
+ *     element [2] is bits 7:0 of the FC Address Identifier
+ */
+
+
+/*
+ * FC Host Messages
+ */
+
+/* FC_BSG_HST_ADDR_PORT : */
+
+/* Request:
+ * This message requests the FC host to login to the remote port
+ * at the specified N_Port_Id.  The remote port is to be enumerated
+ * with the transport upon completion of the login.
+ */
+struct fc_bsg_host_add_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to login to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient
+ */
+
+
+/* FC_BSG_HST_DEL_RPORT : */
+
+/* Request:
+ * This message requests the FC host to remove an enumerated
+ * remote port and to terminate the login to it.
+ *
+ * Note: The driver is free to reject this request if it desires to
+ * remain logged in with the remote port.
+ */
+struct fc_bsg_host_del_rport {
+	uint8_t		reserved;
+
+	/* FC Address Identier of the remote port to logout of */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is sufficient 
+ */
+
+
+/* FC_BSG_HST_ELS_NOLOGIN : */
+
+/* Request:
+ * This message requests the FC_Host to send an ELS to a specific
+ * N_Port_ID. The host does not need to log into the remote port,
+ * nor does it need to enumerate the rport for further traffic
+ * (although, the FC host is free to do so if it desires).
+ */
+struct fc_bsg_host_els {
+	/* 
+	 * ELS Command Code being sent (must be the same as byte 0
+	 * of the payload)
+	 */
+	uint8_t 	command_code;
+
+	/* FC Address Identier of the remote port to send the ELS to */
+	uint8_t		port_id[3];
+};
+
+/* Response:
+ */
+/* fc_bsg_ctels_reply->status values */
+#define FC_CTELS_STATUS_OK	0x00000000
+#define FC_CTELS_STATUS_REJECT	0x00000001
+#define FC_CTELS_STATUS_P_RJT	0x00000002
+#define FC_CTELS_STATUS_F_RJT	0x00000003
+#define FC_CTELS_STATUS_P_BSY	0x00000004
+#define FC_CTELS_STATUS_F_BSY	0x00000006
+struct fc_bsg_ctels_reply {
+	/*
+	 * Note: An ELS LS_RJT may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the ELS receive payload to determine
+	 *     LS_ACC or LS_RJT (by contents of word 0). The reject
+	 *     data will be in word 1.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK, and
+	 *   the receive payload word 0 indicates LS_ACC
+	 *   (e.g. value is 0x02xxxxxx).
+	 *
+	 * Note: Similarly, a CT Reject may be reported in 2 ways:
+	 *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+	 *     is to look into the CT receive payload to determine
+	 *     Accept or Reject (by contents of word 2). The reject
+	 *     data will be in word 3.
+	 *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+	 *     rjt_data field will contain valid data.
+	 *
+	 * Note: x_RJT/BSY status will indicae that the rjt_data field
+	 *   is valid and contains the reason/explanation values.
+	 */
+	uint32_t	status;		/* See FC_CTELS_STATUS_xxx */
+
+	/* valid if status is not FC_CTELS_STATUS_OK */
+	struct	{
+		uint8_t	action;		/* fragment_id for CT REJECT */
+		uint8_t	reason_code;
+		uint8_t	reason_explanation;
+		uint8_t	vendor_unique;
+	} rjt_data;
+};
+
+
+/* FC_BSG_HST_VENDOR : */
+
+/* Request:
+ * Note: When specifying vendor_id, be sure to read the Vendor Type and ID
+ *   formatting requirements specified in scsi_netlink.h
+ */
+struct fc_bsg_host_vendor {
+	/*
+	 * Identifies the vendor that the message is formatted for. This
+	 * should be the recipient of the message.
+	 */
+	uint64_t vendor_id;
+
+	/* start of vendor command area */
+	uint32_t vendor_cmd[0];
+};
+
+/* Response:
+ */
+struct fc_bsg_host_vendor_reply {
+	/* start of vendor response area */
+	uint32_t vendor_rsp[0];
+};
+
+
+
+/*
+ * FC Remote Port Messages
+ */
+
+/* FC_BSG_RPT_ELS : */
+
+/* Request:
+ * This message requests that an ELS be performed with the rport.
+ */
+struct fc_bsg_rport_els {
+	/* 
+	 * ELS Command Code being sent (must be the same as
+	 * byte 0 of the payload)
+	 */
+	uint8_t els_code;
+};
+
+/* Response:
+ * 
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_RPT_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the rport.
+ */
+struct fc_bsg_rport_ct {
+	/* 
+	 * We need words 0-2 of the generic preamble for the LLD's
+	 */
+	uint32_t	preamble_word0;	/* revision & IN_ID */
+	uint32_t	preamble_word1;	/* GS_Type, GS_SubType, Options, Rsvd */
+	uint32_t	preamble_word2;	/* Cmd Code, Max Size */
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+
+
+/* request (CDB) structure of the sg_io_v4 */
+struct fc_bsg_request {
+	uint32_t msgcode;
+	union {
+		struct fc_bsg_host_add_rport	h_addrport;
+		struct fc_bsg_host_del_rport	h_delrport;
+		struct fc_bsg_host_els	h_els;
+		struct fc_bsg_host_vendor	h_vendor;
+
+		struct fc_bsg_rport_els		r_els;
+		struct fc_bsg_rport_ct		r_ct;
+	} rqst_data;
+};
+
+
+/* response (request sense data) structure of the sg_io_v4 */
+struct fc_bsg_reply {
+	/*
+	 * The completion result. Result exists in two forms:
+	 *  if negative, it is an -Exxx system errno value. There will
+	 *    be no further reply information supplied.
+	 *  else, it's the 4-byte scsi error result, with driver, host,
+	 *    msg and status fields. The per-msgcode reply structure
+	 *    will contain valid data.
+	 */
+	uint32_t result;
+
+	/* If there was reply_payload, how much was recevied ? */
+	uint32_t reply_payload_rcv_len;
+
+	union {
+		struct fc_bsg_host_vendor_reply		vendor_reply;
+
+		struct fc_bsg_ctels_reply		ctels_reply;
+	} reply_data;
+};
+
+
+#endif /* SCSI_BSG_FC_H */
+

diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h

include/scsi/scsi_host.h +9 -0
@@ -478,6 +478,15 @@ struct scsi_host_template {
 	 * module_init/module_exit.
 	 */
 	struct list_head legacy_hosts;
+
+	/*
+	 * Vendor Identifier associated with the host
+	 *
+	 * Note: When specifying vendor_id, be sure to read the
+	 *   Vendor Type and ID formatting requirements specified in
+	 *   scsi_netlink.h
+	 */
+	u64 vendor_id;
 };
 
 /*

diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h

include/scsi/scsi_transport_fc.h +51 -3
@@ -33,7 +33,6 @@
 
 struct scsi_transport_template;
 
-
 /*
  * FC Port definitions - Following FC HBAAPI guidelines
  *
@@ -352,6 +351,7 @@ struct fc_rport {	/* aka fc_starget_attr
  	struct delayed_work fail_io_work;
  	struct work_struct stgt_delete_work;
 	struct work_struct rport_delete_work;
+	struct request_queue *rqst_q;	/* bsg support */
 } __attribute__((aligned(sizeof(unsigned long))));
 
 /* bit field values for struct fc_rport "flags" field: */
@@ -513,6 +513,10 @@ struct fc_host_attrs {
 	struct workqueue_struct *work_q;
 	char devloss_work_q_name[20];
 	struct workqueue_struct *devloss_work_q;
+
+	/* bsg support */
+	char bsg_name[20];
+	struct request_queue *rqst_q;
 };
 
 #define shost_to_fc_host(x) \
@@ -578,6 +582,47 @@ struct fc_host_attrs {
 	(((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
 
 
+struct fc_bsg_buffer {
+	unsigned int payload_len;
+	int sg_cnt;
+	struct scatterlist *sg_list;
+};
+
+/* Values for fc_bsg_job->state_flags (bitflags) */
+#define FC_RQST_STATE_INPROGRESS	0
+#define FC_RQST_STATE_DONE		1
+
+struct fc_bsg_job {
+	struct Scsi_Host *shost;
+	struct fc_rport *rport;
+	struct device *dev;
+	struct request *req;
+	spinlock_t job_lock;
+	unsigned int state_flags;
+	unsigned int ref_cnt;
+	void (*job_done)(struct fc_bsg_job *);
+
+	struct fc_bsg_request *request;
+	struct fc_bsg_reply *reply;
+	unsigned int request_len;
+	unsigned int reply_len;
+	/*
+	 * On entry : reply_len indicates the buffer size allocated for
+	 * the reply.
+	 *
+	 * Upon completion : the message handler must set reply_len
+	 *  to indicates the size of the reply to be returned to the
+	 *  caller.
+	 */
+
+	/* DMA payloads for the request/response */
+	struct fc_bsg_buffer request_payload;
+	struct fc_bsg_buffer reply_payload;
+
+	void *dd_data;			/* Used for driver-specific storage */
+};
+
+
 /* The functions by which the transport class and the driver communicate */
 struct fc_function_template {
 	void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
@@ -613,9 +658,14 @@ struct fc_function_template {
 	int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64, int);
 	int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
 
+	/* bsg support */
+	int	(* bsg_request)(struct fc_bsg_job *);
+	int	(* bsg_timeout)(struct fc_bsg_job *);
+
 	/* allocation lengths for host-specific data */
 	u32	 			dd_fcrport_size;
 	u32	 			dd_fcvport_size;
+	u32				dd_bsg_size;
 
 	/*
 	 * The driver sets these to tell the transport class it
@@ -736,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
 	vport->vport_state = new_state;
 }
 
-
 struct scsi_transport_template *fc_attach_transport(
 			struct fc_function_template *);
 void fc_release_transport(struct scsi_transport_template *);
 
 
 
--
Quoted 3 lines

To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#15 lore raw
Quoted 146 lines · On Thu, 27 Nov 2008 10:58:43 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 95 lines · FUJITA Tomonori
Quoted 77 lines · On Tue, 18 Nov 2008 16:24:58 -0500 James Smart <[email protected]> wrote:

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   53 ++
 6 files changed, 946 insertions(+), 4 deletions(-)

(snip)

Quoted 12 lines
drivers/scsi/scsi_transport_fc.c +12 -0
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req->next_rq;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	int err;
(quoted diff) +1 -0
+	unsigned bytes_requested = 0;
Quoted 35 lines
Quoted 13 lines
drivers/scsi/scsi_transport_fc.c +12 -0
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3418

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));

I think that you can use blk_end_bidi_request() for non-bidi requests:

	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
			rsp ?
			 job->reply->reply_payload_rcv_len : 0);

I guess that it would be better to have one function to complete a request, instead of blk_end_bidi_request and blk_end_request.

(quoted diff) +11 -0
+	/*
+	 * tell blk layer all of the xmt data was sent.
+	 * but set residual count to: requested - received
+	 */
+
+	if (rsp) {
+		bytes_requested = blk_rq_bytes(rsp);
+		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
+	}
+
+	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
Quoted 3 lines

The residual count is left in req->data_len. Does bsg have a way to return the residual to user-mode? It must, since Pete was using that for sure. Note that you are looking for the bidi_read residual count.

Yeah, bsg has. struct sg_io_v4 has:

__s32 din_resid; /* [o] din_xfer_len - actual_din_xfer_len */
__s32 dout_resid; /* [o] dout_xfer_len - actual_dout_xfer_len */

Quoted 4 lines

As was said by people. You must complete ALL bytes on both sides. Residual information is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

We always complete all bytes on both sides. So why we do something like:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)

{
	unsigned int bidi_bytes	= 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = req->next_rq->data_len;

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}

The callers can do something like:

blk_end_request(rq, err, rq->data_len);
rq-->next_rq->data_len = resid;

#16 lore raw
Quoted 173 lines · FUJITA Tomonori
Quoted 142 lines · On Thu, 27 Nov 2008 10:58:43 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 94 lines · FUJITA Tomonori
Quoted 77 lines · On Tue, 18 Nov 2008 16:24:58 -0500 James Smart <[email protected]> wrote:

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  581 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  291 +++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   53 ++
 6 files changed, 946 insertions(+), 4 deletions(-)

(snip)

Quoted 12 lines
drivers/scsi/scsi_transport_fc.c +12 -0
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req->next_rq;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	int err;
(quoted diff) +1 -0
+	unsigned bytes_requested = 0;
Quoted 34 lines
Quoted 13 lines
drivers/scsi/scsi_transport_fc.c +12 -0
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3418

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));

I think that you can use blk_end_bidi_request() for non-bidi requests:

	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
			rsp ?
			 job->reply->reply_payload_rcv_len : 0);

I guess that it would be better to have one function to complete a request, instead of blk_end_bidi_request and blk_end_request.

(quoted diff) +11 -0
+	/*
+	 * tell blk layer all of the xmt data was sent.
+	 * but set residual count to: requested - received
+	 */
+
+	if (rsp) {
+		bytes_requested = blk_rq_bytes(rsp);
+		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
+	}
+
+	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
Quoted 3 lines

The residual count is left in req->data_len. Does bsg have a way to return the residual to user-mode? It must, since Pete was using that for sure. Note that you are looking for the bidi_read residual count.

Yeah, bsg has. struct sg_io_v4 has:

__s32 din_resid; /* [o] din_xfer_len - actual_din_xfer_len */
__s32 dout_resid; /* [o] dout_xfer_len - actual_dout_xfer_len */

Quoted 4 lines

As was said by people. You must complete ALL bytes on both sides. Residual information is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

We always complete all bytes on both sides. So why we do something like:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)

{
	unsigned int bidi_bytes	= 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = req->next_rq->data_len;

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}
Quoted 4 lines

The callers can do something like:

blk_end_request(rq, err, rq->data_len);
rq-->next_rq->data_len = resid;

Sorry TOMO, I do not understand what you mean. Do you say that we should change blk_end_request() in blk-core.c ?

In anyway, the code you suggest has a bug you can not use rq-> after call to blk_end_io() because it might not exist at this point. You must set residual before. And also you should use blk_rq_bytes(rq). To see how a request is fully competed see scsi_end_bidi_request().

Boaz

#17 lore raw
Quoted 72 lines · On Thu, 27 Nov 2008 13:51:51 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 72 lines
Quoted 44 lines
Quoted 32 lines
Quoted 11 lines
drivers/scsi/scsi_transport_fc.c +11 -0
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3418

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));

I think that you can use blk_end_bidi_request() for non-bidi requests:

	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
			rsp ?
			 job->reply->reply_payload_rcv_len : 0);

I guess that it would be better to have one function to complete a request, instead of blk_end_bidi_request and blk_end_request.

(quoted diff) +11 -0
+	/*
+	 * tell blk layer all of the xmt data was sent.
+	 * but set residual count to: requested - received
+	 */
+
+	if (rsp) {
+		bytes_requested = blk_rq_bytes(rsp);
+		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
+	}
+
+	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
Quoted 3 lines

The residual count is left in req->data_len. Does bsg have a way to return the residual to user-mode? It must, since Pete was using that for sure. Note that you are looking for the bidi_read residual count.

Yeah, bsg has. struct sg_io_v4 has:

__s32 din_resid; /* [o] din_xfer_len - actual_din_xfer_len */
__s32 dout_resid; /* [o] dout_xfer_len - actual_dout_xfer_len */

Quoted 4 lines

As was said by people. You must complete ALL bytes on both sides. Residual information is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

We always complete all bytes on both sides. So why we do something like:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)

{
	unsigned int bidi_bytes	= 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = req->next_rq->data_len;

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}
Quoted 7 lines
Quoted 4 lines

The callers can do something like:

blk_end_request(rq, err, rq->data_len);
rq-->next_rq->data_len = resid;

Sorry TOMO, I do not understand what you mean. Do you say that we should change blk_end_request() in blk-core.c ?

Having two kinds of functions (blk_end_request and blk_end_bidi_request) to complete requests confuse people. As we saw, developers tend to do something like this:

+ if (rsp)
+ blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+ job->reply->reply_payload_rcv_len);
+ else
+ blk_end_request(job->req, err, blk_rq_bytes(job->req));

The callers don't care about whether a request is bidi or not. It's be simpler to have a single function to complete a request (whether a request is bidi or not) rather than having two different functions.

We must complete all bytes on both sides with a bidi request. So why can't we modify blk_end_request to handle both bidi and non-bidi requests:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes) {
unsigned int bidi_bytes = 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = blk_rq_bytes(rq->next_rq);

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}
Quoted 2 lines

In anyway, the code you suggest has a bug you can not use rq-> after call to blk_end_io() because it might not exist at this point. You must set residual before. And also

What is 'rq->' exactly?

We must set residual before calling blk_end_request? Really?

Note that scsi-ml and bsg (blk_execute_rq) work differently. For scsi-ml, blk_end_io frees request structure (end_that_request_last) but for blk_execute_rq, it doesn't.

Anyway, it's fine to set bidi_resid before blk_end_request, I guess. FC pass thru code could do something like this if we modify blk_end_request in the above way:

/* we calculate bidi_resid here */

if (blk_bidi_rq(req))

	req->next_rq->data_len = bidi_resid;

blk_end_request(req, 0, blk_rq_bytes(req));

#18 lore raw
Quoted 113 lines · FUJITA Tomonori
Quoted 70 lines · On Thu, 27 Nov 2008 13:51:51 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 70 lines
Quoted 44 lines
Quoted 32 lines
Quoted 11 lines
drivers/scsi/scsi_transport_fc.c +11 -0
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3418

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+
+	/*
+	 * we'll cheat: tell blk layer all of the xmt data was sent.
+	 * but try to be honest about the amount of rcv data received
+	 */
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));

I think that you can use blk_end_bidi_request() for non-bidi requests:

	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
			rsp ?
			 job->reply->reply_payload_rcv_len : 0);

I guess that it would be better to have one function to complete a request, instead of blk_end_bidi_request and blk_end_request.

(quoted diff) +11 -0
+	/*
+	 * tell blk layer all of the xmt data was sent.
+	 * but set residual count to: requested - received
+	 */
+
+	if (rsp) {
+		bytes_requested = blk_rq_bytes(rsp);
+		rsp->data_len = bytes_requested - job->reply->reply_payload_rcv_len;
+	}
+
+	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);
Quoted 3 lines

The residual count is left in req->data_len. Does bsg have a way to return the residual to user-mode? It must, since Pete was using that for sure. Note that you are looking for the bidi_read residual count.

Yeah, bsg has. struct sg_io_v4 has:

__s32 din_resid; /* [o] din_xfer_len - actual_din_xfer_len */
__s32 dout_resid; /* [o] dout_xfer_len - actual_dout_xfer_len */

Quoted 4 lines

As was said by people. You must complete ALL bytes on both sides. Residual information is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

We always complete all bytes on both sides. So why we do something like:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)

{
	unsigned int bidi_bytes	= 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = req->next_rq->data_len;

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}
Quoted 6 lines
Quoted 4 lines

The callers can do something like:

blk_end_request(rq, err, rq->data_len);
rq-->next_rq->data_len = resid;

Sorry TOMO, I do not understand what you mean. Do you say that we should change blk_end_request() in blk-core.c ?

Having two kinds of functions (blk_end_request and blk_end_bidi_request) to complete requests confuse people. As we saw, developers tend to do something like this:

drivers/scsi/scsi_transport_fc.c +5 -0
+	if (rsp)
+		blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req),
+	    			     job->reply->reply_payload_rcv_len);
+	else
+		blk_end_request(job->req, err, blk_rq_bytes(job->req));

The callers don't care about whether a request is bidi or not. It's be simpler to have a single function to complete a request (whether a request is bidi or not) rather than having two different functions.

We must complete all bytes on both sides with a bidi request. So why can't we modify blk_end_request to handle both bidi and non-bidi requests:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes) {
unsigned int bidi_bytes = 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = blk_rq_bytes(rq->next_rq);

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}
Quoted 2 lines

In anyway, the code you suggest has a bug you can not use rq-> after call to blk_end_io() because it might not exist at this point. You must set residual before. And also

Quoted 8 lines

What is 'rq->' exactly?

We must set residual before calling blk_end_request? Really?

Note that scsi-ml and bsg (blk_execute_rq) work differently. For scsi-ml, blk_end_io frees request structure (end_that_request_last) but for blk_execute_rq, it doesn't.

It does not matter if bsg or any other user has an extra reference on the request (so end_that_request_last does not deallocate the request). The end_io function is called from within the end_that_request_last so setting the residual into req->data_len will be to late.

Quoted 10 lines

Anyway, it's fine to set bidi_resid before blk_end_request, I guess. FC pass thru code could do something like this if we modify blk_end_request in the above way:

/* we calculate bidi_resid here */

if (blk_bidi_rq(req))

	req->next_rq->data_len = bidi_resid;

No that will not work. You lost the req->next_rq->data_len byte-count and blk_end_request() will not be able to complete all bytes of req->next_rq.

Please see scsi_end_bidi_request() for the only way to complete a bidi request with returned residual count on both sides.

Quoted 1 line

blk_end_request(req, 0, blk_rq_bytes(req));

The way blk_end_request() is now it cannot complete bidi requests. because of residual count missing. I have shown above the only way that you can complete both bidi or uni request with a single call to blk_end_bidi_request(). If you want people not to get confused it is blk_end_request() that should be dropped.

Boaz

maintainer
#19 lore raw
In reply to FUJITA Tomonori
Quoted 21 lines · FUJITA Tomonori
Quoted 5 lines
(quoted diff) +1 -0
+	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);

The residual count is left in req->data_len. Does bsg have a way to return the residual to user-mode? It must, since Pete was using that for sure. Note that you are looking for the bidi_read residual count.

Yeah, bsg has. struct sg_io_v4 has:

__s32 din_resid; /* [o] din_xfer_len - actual_din_xfer_len */
__s32 dout_resid; /* [o] dout_xfer_len - actual_dout_xfer_len */

Quoted 4 lines

As was said by people. You must complete ALL bytes on both sides. Residual information is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

We always complete all bytes on both sides. So why we do something like:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)

Quoted 8 lines
{
	unsigned int bidi_bytes	= 0;

	if (blk_bidi_rq(rq))
		bidi_bytes = req->next_rq->data_len;

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}

That looks good ... care to patch it up (including conversion of all the users) and send it to Jens for consideration?

Quoted 4 lines

The callers can do something like:

blk_end_request(rq, err, rq->data_len);
rq-->next_rq->data_len = resid;

Um ... I see what you're trying to do, but I don't think it works like this. The residual could be collected in the end_io call back which is inline in the blk_end_request().

James

#20 lore raw
Quoted 25 lines · On Thu, 27 Nov 2008 20:01:41 -0600 James Bottomley…
Quoted 24 lines · FUJITA Tomonori
Quoted 5 lines
(quoted diff) +1 -0
+	blk_end_bidi_request(job->req, err, blk_rq_bytes(job->req), bytes_requested);

The residual count is left in req->data_len. Does bsg have a way to return the residual to user-mode? It must, since Pete was using that for sure. Note that you are looking for the bidi_read residual count.

Yeah, bsg has. struct sg_io_v4 has:

__s32 din_resid; /* [o] din_xfer_len - actual_din_xfer_len */
__s32 dout_resid; /* [o] dout_xfer_len - actual_dout_xfer_len */

Quoted 4 lines

As was said by people. You must complete ALL bytes on both sides. Residual information is passed through req->data_len. Other wise the request is still active.

(And yes blk_end_request uses blk_end_bidi_request internally)

We always complete all bytes on both sides. So why we do something like:

int blk_end_request(struct request *rq, int error, unsigned int nr_bytes)

{
	unsigned int bidi_bytes	= 0;
Quoted 8 lines
Quoted 5 lines
	if (blk_bidi_rq(rq))
		bidi_bytes = req->next_rq->data_len;

	return blk_end_io(rq, error, nr_bytes, bidi_bytes, NULL);
}

That looks good ... care to patch it up (including conversion of all the users) and send it to Jens for consideration?

Sure, I'll do.

Quoted 8 lines
Quoted 4 lines

The callers can do something like:

blk_end_request(rq, err, rq->data_len);
rq-->next_rq->data_len = resid;

Um ... I see what you're trying to do, but I don't think it works like this. The residual could be collected in the end_io call back which is inline in the blk_end_request().

Yeah, you are right. bsg doesn't though. But after calling blk_end_request, bsg can touch (or free) rq any time. So the callers can't touch rq in the above way after calling blk_end_request (it's racy).

As Boaz said, we must (and can) set resid before blk_end_request.

#21 lore raw

RFC 1/1 FC pass thru - Rev V

In reply to James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

It currently expects that blk_end_request() has been updated by Fujita's patch to incorporate blk_end_bidi_request() functionality :
http://marc.info/?l-linux-scsi&m=122785157116659&w=2

-- james s

======

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  614 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   52 ++
 6 files changed, 1009 insertions(+), 4 deletions(-)
Documentation/scsi/scsi_fc_transport.txt +10 -1
diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt
@@ -1,10 +1,11 @@
                              SCSI FC Tansport
                  =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008
 Kernel Revisions for features:
   rports : <<TBS>>
   vports : 2.6.22 (? TBD)
+  bsg support : 2.6.29 (?TBD)
 
 
 Introduction
@@ -15,6 +16,7 @@ The FC transport can be found at:
   drivers/scsi/scsi_transport_fc.c
   include/scsi/scsi_transport_fc.h
   include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 
 This file is found at Documentation/scsi/scsi_fc_transport.txt
 
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+========================================================================
+<< To Be Supplied >>
+
+
+
+
 Credits
 =======
 The following people have contributed to this document:
Documentation/scsi/scsi_mid_low_api.txt +5 -0
diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
@@ -1271,6 +1271,11 @@ of interest:
     hostdata[0]  - area reserved for LLD at end of struct Scsi_Host. Size
                    is set by the second argument (named 'xtr_bytes') to
                    scsi_host_alloc() or scsi_register().
+    vendor_id    - a unique value that identifies the vendor supplying
+                   the LLD for the Scsi_Host.  Used most often in validating
+                   vendor-specific message requests.  Value consists of an
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid formats.
 
 The scsi_host structure is defined in include/scsi/scsi_host.h
 
drivers/scsi/scsi_transport_fc.c +613 -1

668 lines not on this page · raw message

include/scsi/scsi_bsg_fc.h +322 -0

322 lines not on this page · raw message

include/scsi/scsi_host.h +9 -0
diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
@@ -478,6 +478,15 @@ struct scsi_host_template {
 	 * module_init/module_exit.
 	 */
 	struct list_head legacy_hosts;
+
+	/*
+	 * Vendor Identifier associated with the host
+	 *
+	 * Note: When specifying vendor_id, be sure to read the
+	 *   Vendor Type and ID formatting requirements specified in
+	 *   scsi_netlink.h
+	 */
+	u64 vendor_id;
 };
 
 /*
include/scsi/scsi_transport_fc.h +50 -2

91 lines not on this page · raw message

#22 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer. The driver with the changes should be available within a couple of weeks.

Thank you,
Seokmann

Quoted 1,351 lines

It currently expects that blk_end_request() has been updated by Fujita's patch to incorporate blk_end_bidi_request() functionality :
http://marc.info/?l-linux-scsi&m=122785157116659&w=2

-- james s

======

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

Documentation/scsi/scsi_fc_transport.txt |   11
Documentation/scsi/scsi_mid_low_api.txt  |    5
drivers/scsi/scsi_transport_fc.c         |  614 +++++++++++++++++++++ 
+++++++++-
include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
include/scsi/scsi_host.h                 |    9
include/scsi/scsi_transport_fc.h         |   52 ++
6 files changed, 1009 insertions(+), 4 deletions(-)

diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt --- a/Documentation/scsi/scsi_fc_transport.txt 2009-01-27
09:44:22.000000000 -0500
+++ b/Documentation/scsi/scsi_fc_transport.txt 2009-02-10
10:34:42.000000000 -0500

Documentation/scsi/scsi_fc_transport.txt +1 -1
@@ -1,10 +1,11 @@
                             SCSI FC Tansport
                 =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008

Kernel Revisions for features:
rports : <<TBS>>
vports : 2.6.22 (? TBD)
+ bsg support : 2.6.29 (?TBD)

Introduction

Documentation/scsi/scsi_fc_transport.txt +1 -0
@@ -15,6 +16,7 @@ The FC transport can be found at:
  drivers/scsi/scsi_transport_fc.c
  include/scsi/scsi_transport_fc.h
  include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 

This file is found at Documentation/scsi/scsi_fc_transport.txt

Documentation/scsi/scsi_fc_transport.txt +2 -0
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+ 

========================================================================
+<< To Be Supplied >>

Documentation/scsi/scsi_fc_transport.txt +4 -0
+
+
+
+

Credits
=======
The following people have contributed to this document:
diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
--- a/Documentation/scsi/scsi_mid_low_api.txt 2008-09-23
15:11:57.000000000 -0400
+++ b/Documentation/scsi/scsi_mid_low_api.txt 2009-02-10
10:34:42.000000000 -0500

(quoted diff) +0 -0
@@ -1271,6 +1271,11 @@ of interest:
    hostdata[0]  - area reserved for LLD at end of struct Scsi_Host.  

Size
is set by the second argument (named 'xtr_bytes')
to
scsi_host_alloc() or scsi_register().
+ vendor_id - a unique value that identifies the vendor
supplying
+ the LLD for the Scsi_Host. Used most often in
validating
+ vendor-specific message requests. Value
consists of an

(quoted diff) +2 -0
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid  

formats.

The scsi_host structure is defined in include/scsi/scsi_host.h

diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c --- a/drivers/scsi/scsi_transport_fc.c 2009-01-27 09:44:31.000000000 -0500
+++ b/drivers/scsi/scsi_transport_fc.c 2009-02-10 14:58:39.000000000 -0500

drivers/scsi/scsi_transport_fc.c +5 -0
@@ -35,6 +35,7 @@
 #include <linux/netlink.h>
 #include <net/netlink.h>
 #include <scsi/scsi_netlink_fc.h>
+#include <scsi/scsi_bsg_fc.h>
 #include "scsi_priv.h"
 #include "scsi_transport_fc_internal.h"
 
@@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
 static int fc_vport_setup(struct Scsi_Host *shost, int channel,
       struct device *pdev, struct fc_vport_identifiers  *ids,
       struct fc_vport **vport);
+static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs  
 *);
+static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
+static void fc_bsg_remove(struct request_queue *);
+static void fc_bsg_goose_queue(struct fc_rport *);
 
 /*

* Redefine so that we can have same named attributes in the

drivers/scsi/scsi_transport_fc.c +14 -1
@@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
               return -ENOMEM;
       }
 
+       fc_bsg_hostadd(shost, fc_host);
+       /* ignore any bsg add error - we just can't do sgio */
+
+       return 0;
+}
+
+static int fc_host_remove(struct transport_container *tc, struct  
 device *dev,
+                        struct device *cdev)
+{
+       struct Scsi_Host *shost = dev_to_shost(dev);
+       struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
+
+       fc_bsg_remove(fc_host->rqst_q);
       return 0;
 }
 
 static DECLARE_TRANSPORT_CLASS(fc_host_class,
                              "fc_host",
                              fc_host_setup,
-                              NULL,
+                              fc_host_remove,
                              NULL);
 

/*

drivers/scsi/scsi_transport_fc.c +8 -0
@@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
               scsi_flush_work(shost);
 
       fc_terminate_rport_io(rport);
+
       /*
        * Cancel any outstanding timers. These should really exist
        * only when rmmod'ing the LLDD and we're asking for
@@ -2415,6 +2434,8 @@ fc_rport_final_delete(struct work_struct
           (i->f->dev_loss_tmo_callbk))
               i->f->dev_loss_tmo_callbk(rport);
 
+       fc_bsg_remove(rport->rqst_q);
+
       transport_remove_device(dev);
       device_del(dev);
       transport_destroy_device(dev);
@@ -2502,6 +2523,9 @@ fc_rport_create(struct Scsi_Host *shost,
       transport_add_device(dev);
       transport_configure_device(dev);
 
+       fc_bsg_rportadd(shost, rport);
+       /* ignore any bsg add error - we just can't do sgio */
+
       if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
               /* initiate a scan of the target */
               rport->flags |= FC_RPORT_SCAN_PENDING;
@@ -2666,6 +2690,8 @@ fc_remote_port_add(struct Scsi_Host *sho
                                       spin_unlock_irqrestore(shost- 
 >host_lock,
                                                       flags);
 
+                               fc_bsg_goose_queue(rport);
+
                               return rport;
                       }
               }
drivers/scsi/scsi_transport_fc.c +31 -0
@@ -3351,6 +3377,592 @@ fc_vport_sched_delete(struct work_struct
 }
 
 
+/*
+ * BSG support
+ */
+
+
+/**
+ * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
+ * @job:       fc_bsg_job that is to be torn down
+ */
+static void
+fc_destroy_bsgjob(struct fc_bsg_job *job)
+{
+       unsigned long flags;
+
+       spin_lock_irqsave(&job->job_lock, flags);
+       if (job->ref_cnt) {
+               spin_unlock_irqrestore(&job->job_lock, flags);
+               return;
+       }
+       spin_unlock_irqrestore(&job->job_lock, flags);
+
+       put_device(job->dev);   /* release reference for the request  
 */
+
+       kfree(job->request_payload.sg_list);
+       kfree(job->reply_payload.sg_list);
+       kfree(job);
+}
+
+
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the  

LLD has
+ * completed
+ * @job: fc_bsg_job that is complete
+ */

drivers/scsi/scsi_transport_fc.c +17 -0
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+       struct request *req = job->req;
+       struct request *rsp = req->next_rq;
+       unsigned long flags;
+       unsigned rsp_len;
+       int err;
+
+       spin_lock_irqsave(&job->job_lock, flags);
+       job->state_flags |= FC_RQST_STATE_DONE;
+       job->ref_cnt--;
+       spin_unlock_irqrestore(&job->job_lock, flags);
+
+       err = job->req->errors = job->reply->result;
+       if (err < 0)
+               /* we're only returning the result field in the  

reply */

drivers/scsi/scsi_transport_fc.c +9 -0
+               job->req->sense_len = sizeof(uint32_t);
+       else
+               job->req->sense_len = job->reply_len;
+
+       if (rsp) {
+               rsp_len = blk_rq_bytes(rsp);
+               BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+               /* set reply (bidi) residual */
+               rsp->data_len = (rsp_len - job->reply- 
Quoted 1 line

reply_payload_rcv_len);

drivers/scsi/scsi_transport_fc.c +23 -0
+       }
+
+       /* we assume all request payload was transferred */
+       blk_end_request(req, err, blk_rq_bytes(req));
+
+       fc_destroy_bsgjob(job);
+}
+
+
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:       request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+       struct fc_bsg_job *job = (void *) req->special;
+       struct Scsi_Host *shost = job->shost;
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       unsigned long flags;
+       int err = 0, done = 0;
+
+       if (job->rport && job->rport->port_state ==  

FC_PORTSTATE_BLOCKED)

drivers/scsi/scsi_transport_fc.c +14 -0
+               return BLK_EH_RESET_TIMER;
+
+       spin_lock_irqsave(&job->job_lock, flags);
+       if (job->state_flags & FC_RQST_STATE_DONE)
+               done = 1;
+       else
+               job->ref_cnt++;
+       spin_unlock_irqrestore(&job->job_lock, flags);
+
+       if (!done && i->f->bsg_timeout) {
+               /* call LLDD to abort the i/o as it has timed out */
+               err = i->f->bsg_timeout(job);
+               if (err)
+                       printk(KERN_ERR "ERROR: FC BSG request  

timeout - LLD "

drivers/scsi/scsi_transport_fc.c +20 -0
+                               "abort failed with status %d\n", err);
+       }
+
+       if (!done) {
+               spin_lock_irqsave(&job->job_lock, flags);
+               job->ref_cnt--;
+               spin_unlock_irqrestore(&job->job_lock, flags);
+               fc_destroy_bsgjob(job);
+       }
+
+       /* the blk_end_sync_io() doesn't check the error */
+       return BLK_EH_HANDLED;
+}
+
+
+
+static int
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+       size_t sz = (sizeof(struct scatterlist) * req- 
Quoted 1 line

nr_phys_segments);

drivers/scsi/scsi_transport_fc.c +15 -0
+
+       BUG_ON(!req->nr_phys_segments);
+
+       buf->sg_list = kzalloc(sz, GFP_KERNEL);
+       if (!buf->sg_list)
+               return -ENOMEM;
+       sg_init_table(buf->sg_list, req->nr_phys_segments);
+       buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+       buf->payload_len = req->data_len;
+       return 0;
+}
+
+
+/**
+ * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for  

the
+ * bsg request
+ * @shost: SCSI Host corresponding to the bsg object
+ * @rport: (optional) FC Remote Port corresponding to the bsg
object
+ * @req: BSG request that needs a job structure
+ */

drivers/scsi/scsi_transport_fc.c +19 -0
+static int
+fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
+       struct request *req)
+{
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       struct request *rsp = req->next_rq;
+       struct fc_bsg_job *job;
+       int ret;
+
+       BUG_ON(req->special);
+
+       job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
+                       GFP_KERNEL);
+       if (!job)
+               return -ENOMEM;
+
+       /*
+        * Note: this is a bit silly.
+        * The request gets formatted as a SGIO v4 ioctl request,  

which

drivers/scsi/scsi_transport_fc.c +3 -0
+        * then gets reformatted as a blk request, which then gets
+        * reformatted as a fc bsg request. And on completion, we have
+        * to wrap return results such that SGIO v4 thinks it was a  

scsi

drivers/scsi/scsi_transport_fc.c +14 -0
+        * status.  I hope this was all worth it.
+        */
+
+       req->special = job;
+       job->shost = shost;
+       job->rport = rport;
+       job->req = req;
+       if (i->f->dd_bsg_size)
+               job->dd_data = (void *)&job[1];
+       spin_lock_init(&job->job_lock);
+       job->request = (struct fc_bsg_request *)req->cmd;
+       job->request_len = req->cmd_len;
+       job->reply = req->sense;
+       job->reply_len = SCSI_SENSE_BUFFERSIZE; /* Size of sense  

buffer

drivers/scsi/scsi_transport_fc.c +17 -0
+                                                * allocated */
+       if (req->bio) {
+               ret = fc_bsg_map_buffer(&job->request_payload, req);
+               if (ret)
+                       goto failjob_rls_job;
+       }
+       if (rsp && rsp->bio) {
+               ret = fc_bsg_map_buffer(&job->reply_payload, rsp);
+               if (ret)
+                       goto failjob_rls_rqst_payload;
+       }
+       job->job_done = fc_bsg_jobdone;
+       if (rport)
+               job->dev = &rport->dev;
+       else
+               job->dev = &shost->shost_gendev;
+       get_device(job->dev);           /* take a reference for the  

request */

drivers/scsi/scsi_transport_fc.c +16 -0
+
+       job->ref_cnt = 1;
+
+       return 0;
+
+
+failjob_rls_rqst_payload:
+       kfree(job->request_payload.sg_list);
+failjob_rls_job:
+       kfree(job);
+       return -ENOMEM;
+}
+
+
+enum fc_dispatch_result {
+       FC_DISPATCH_BREAK,      /* on return, q is locked, break  

from q loop */
+ FC_DISPATCH_LOCKED, /* on return, q is locked, continue
on */
+ FC_DISPATCH_UNLOCKED, /* on return, q is unlocked,
continue on */

drivers/scsi/scsi_transport_fc.c +5 -0
+};
+
+
+/**
+ * fc_bsg_host_dispatch - process fc host bsg requests and dispatch  

to LLDD
+ * @shost: scsi host rport attached to
+ * @job: bsg job to be processed
+ */

(quoted diff) +2 -0
+static enum fc_dispatch_result
+fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host  

*shost,

drivers/scsi/scsi_transport_fc.c +4 -0
+                        struct fc_bsg_job *job)
+{
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       int cmdlen = sizeof(uint32_t);  /* start with length of  

msgcode */

drivers/scsi/scsi_transport_fc.c +83 -0
+       int ret;
+
+       /* Validate the host command */
+       switch (job->request->msgcode) {
+       case FC_BSG_HST_ADD_RPORT:
+               cmdlen += sizeof(struct fc_bsg_host_add_rport);
+               break;
+
+       case FC_BSG_HST_DEL_RPORT:
+               cmdlen += sizeof(struct fc_bsg_host_del_rport);
+               break;
+
+       case FC_BSG_HST_ELS_NOLOGIN:
+               cmdlen += sizeof(struct fc_bsg_host_els);
+               /* there better be a xmt and rcv payloads */
+               if ((!job->request_payload.payload_len) ||
+                   (!job->reply_payload.payload_len)) {
+                       ret = -EINVAL;
+                       goto fail_host_msg;
+               }
+               break;
+
+       case FC_BSG_HST_CT:
+               cmdlen += sizeof(struct fc_bsg_host_ct);
+               /* there better be xmt and rcv payloads */
+               if ((!job->request_payload.payload_len) ||
+                   (!job->reply_payload.payload_len)) {
+                       ret = -EINVAL;
+                       goto fail_host_msg;
+               }
+               break;
+
+       case FC_BSG_HST_VENDOR:
+               cmdlen += sizeof(struct fc_bsg_host_vendor);
+               if ((shost->hostt->vendor_id == 0L) ||
+                   (job->request->rqst_data.h_vendor.vendor_id !=
+                       shost->hostt->vendor_id)) {
+                       ret = -ESRCH;
+                       goto fail_host_msg;
+               }
+               break;
+
+       default:
+               ret = -EBADR;
+               goto fail_host_msg;
+       }
+
+       /* check if we really have all the request data needed */
+       if (job->request_len < cmdlen) {
+               ret = -ENOMSG;
+               goto fail_host_msg;
+       }
+
+       ret = i->f->bsg_request(job);
+       if (!ret)
+               return FC_DISPATCH_UNLOCKED;
+
+fail_host_msg:
+       /* return the errno failure code as the only status */
+       BUG_ON(job->reply_len < sizeof(uint32_t));
+       job->reply->result = ret;
+       job->reply_len = sizeof(uint32_t);
+       fc_bsg_jobdone(job);
+       return FC_DISPATCH_UNLOCKED;
+}
+
+
+/*
+ * fc_bsg_goose_queue - restart rport queue in case it was stopped
+ * @rport:     rport to be restarted
+ */
+static void
+fc_bsg_goose_queue(struct fc_rport *rport)
+{
+       int flagset;
+
+       if (!rport->rqst_q)
+               return;
+
+       get_device(&rport->dev);
+
+       spin_lock(rport->rqst_q->queue_lock);
+       flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
Quoted 1 line

queue_flags) &&

+ !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q-

Quoted 1 line

queue_flags);

drivers/scsi/scsi_transport_fc.c +13 -0
+       if (flagset)
+               queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
+       __blk_run_queue(rport->rqst_q);
+       if (flagset)
+               queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
+       spin_unlock(rport->rqst_q->queue_lock);
+
+       put_device(&rport->dev);
+}
+
+
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch  

to LLDD

+ * @shost:     scsi host rport attached to
+ * @rport:     rport request destined to
+ * @job:       bsg job to be processed

+ */

(quoted diff) +2 -0
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host  

*shost,
+ struct fc_rport *rport, struct fc_bsg_job
*job)

drivers/scsi/scsi_transport_fc.c +3 -0
+{
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       int cmdlen = sizeof(uint32_t);  /* start with length of  

msgcode */

drivers/scsi/scsi_transport_fc.c +52 -0
+       int ret;
+
+       /* Validate the rport command */
+       switch (job->request->msgcode) {
+       case FC_BSG_RPT_ELS:
+               cmdlen += sizeof(struct fc_bsg_rport_els);
+               goto check_bidi;
+
+       case FC_BSG_RPT_CT:
+               cmdlen += sizeof(struct fc_bsg_rport_ct);
+check_bidi:
+               /* there better be xmt and rcv payloads */
+               if ((!job->request_payload.payload_len) ||
+                   (!job->reply_payload.payload_len)) {
+                       ret = -EINVAL;
+                       goto fail_rport_msg;
+               }
+               break;
+       default:
+               ret = -EBADR;
+               goto fail_rport_msg;
+       }
+
+       /* check if we really have all the request data needed */
+       if (job->request_len < cmdlen) {
+               ret = -ENOMSG;
+               goto fail_rport_msg;
+       }
+
+       ret = i->f->bsg_request(job);
+       if (!ret)
+               return FC_DISPATCH_UNLOCKED;
+
+fail_rport_msg:
+       /* return the errno failure code as the only status */
+       BUG_ON(job->reply_len < sizeof(uint32_t));
+       job->reply->result = ret;
+       job->reply_len = sizeof(uint32_t);
+       fc_bsg_jobdone(job);
+       return FC_DISPATCH_UNLOCKED;
+}
+
+
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:         request queue to manage
+ * @shost:     Scsi_Host related to the bsg object
+ * @rport:     FC remote port related to the bsg object (optional)
+ * @dev:       device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host  

*shost,

drivers/scsi/scsi_transport_fc.c +15 -0
+                      struct fc_rport *rport, struct device *dev)
+{
+       struct request *req;
+       struct fc_bsg_job *job;
+       enum fc_dispatch_result ret;
+
+       if (!get_device(dev))
+               return;
+
+       while (!blk_queue_plugged(q)) {
+               req = elv_next_request(q);
+               if (!req)
+                       break;
+
+               if (rport && (rport->port_state ==  

FC_PORTSTATE_BLOCKED))

drivers/scsi/scsi_transport_fc.c +5 -0
+                               break;
+
+               blkdev_dequeue_request(req);
+
+               if (rport && (rport->port_state !=  

FC_PORTSTATE_ONLINE)) {

drivers/scsi/scsi_transport_fc.c +3 -0
+                       req->errors = -ENXIO;
+                       spin_unlock_irq(q->queue_lock);
+                       blk_end_request(req, -ENXIO,  

blk_rq_bytes(req));

drivers/scsi/scsi_transport_fc.c +29 -0
+                       spin_lock_irq(q->queue_lock);
+                       continue;
+               }
+
+               spin_unlock_irq(q->queue_lock);
+
+               ret = fc_req_to_bsgjob(shost, rport, req);
+               if (ret) {
+                       req->errors = ret;
+                       blk_end_request(req, ret, blk_rq_bytes(req));
+                       spin_lock_irq(q->queue_lock);
+                       continue;
+               }
+
+               job = req->special;
+
+               /* check if we have the msgcode value at least */
+               if (job->request_len < sizeof(uint32_t)) {
+                       BUG_ON(job->reply_len < sizeof(uint32_t));
+                       job->reply->result = -ENOMSG;
+                       job->reply_len = sizeof(uint32_t);
+                       fc_bsg_jobdone(job);
+                       spin_lock_irq(q->queue_lock);
+                       continue;
+               }
+
+               /* the dispatch routines will unlock the queue_lock */
+               if (rport)
+                       ret = fc_bsg_rport_dispatch(q, shost, rport,  

job);

drivers/scsi/scsi_transport_fc.c +4 -0
+               else
+                       ret = fc_bsg_host_dispatch(q, shost, job);
+
+               /* did dispatcher hit state that can't process any  

more */

drivers/scsi/scsi_transport_fc.c +43 -0
+               if (ret == FC_DISPATCH_BREAK)
+                       break;
+
+               /* did dispatcher had released the lock */
+               if (ret == FC_DISPATCH_UNLOCKED)
+                       spin_lock_irq(q->queue_lock);
+       }
+
+       spin_unlock_irq(q->queue_lock);
+       put_device(dev);
+       spin_lock_irq(q->queue_lock);
+}
+
+
+/**
+ * fc_bsg_host_handler - handler for bsg requests for a fc host
+ * @q:         fc host request queue
+ */
+static void
+fc_bsg_host_handler(struct request_queue *q)
+{
+       struct Scsi_Host *shost = q->queuedata;
+
+       fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
+}
+
+
+/**
+ * fc_bsg_rport_handler - handler for bsg requests for a fc rport
+ * @q:         rport request queue
+ */
+static void
+fc_bsg_rport_handler(struct request_queue *q)
+{
+       struct fc_rport *rport = q->queuedata;
+       struct Scsi_Host *shost = rport_to_shost(rport);
+
+       fc_bsg_request_handler(q, shost, rport, &rport->dev);
+}
+
+
+/**
+ * fc_bsg_hostadd - Create and add the bsg hooks so we can receive  

requests
+ * @shost: shost for fc_host
+ * @fc_host: fc_host adding the structures to
+ */

(quoted diff) +2 -0
+static int
+fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs  

*fc_host)

drivers/scsi/scsi_transport_fc.c +44 -0
+{
+       struct device *dev = &shost->shost_gendev;
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       struct request_queue *q;
+       int err;
+       char bsg_name[BUS_ID_SIZE]; /*20*/
+
+       fc_host->rqst_q = NULL;
+
+       if (!i->f->bsg_request)
+               return -ENOTSUPP;
+
+       snprintf(bsg_name, sizeof(bsg_name),
+                "fc_host%d", shost->host_no);
+
+       q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
+       if (!q) {
+               printk(KERN_ERR "fc_host%d: bsg interface failed to "
+                               "initialize - no request queue\n",
+                                shost->host_no);
+               return -ENOMEM;
+       }
+
+       q->queuedata = shost;
+       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+       blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
+
+       err = bsg_register_queue(q, dev, bsg_name, NULL);
+       if (err) {
+               printk(KERN_ERR "fc_host%d: bsg interface failed to "
+                               "initialize - register queue\n",
+                               shost->host_no);
+               blk_cleanup_queue(q);
+               return err;
+       }
+
+       fc_host->rqst_q = q;
+       return 0;
+}
+
+
+/**
+ * fc_bsg_rportadd - Create and add the bsg hooks so we can receive  

requests
+ * @shost: shost that rport is attached to
+ * @rport: rport that the bsg hooks are being attached to
+ */

drivers/scsi/scsi_transport_fc.c +54 -0
+static int
+fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
+{
+       struct device *dev = &rport->dev;
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       struct request_queue *q;
+       int err;
+
+       rport->rqst_q = NULL;
+
+       if (!i->f->bsg_request)
+               return -ENOTSUPP;
+
+       q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
+       if (!q) {
+               printk(KERN_ERR "%s: bsg interface failed to "
+                               "initialize - no request queue\n",
+                                dev->bus_id);
+               return -ENOMEM;
+       }
+
+       q->queuedata = rport;
+       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+       blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+       err = bsg_register_queue(q, dev, NULL, NULL);
+       if (err) {
+               printk(KERN_ERR "%s: bsg interface failed to "
+                               "initialize - register queue\n",
+                                dev->bus_id);
+               blk_cleanup_queue(q);
+               return err;
+       }
+
+       rport->rqst_q = q;
+       return 0;
+}
+
+
+/**
+ * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
+ * @q: the request_queue that is to be torn down.
+ */
+static void
+fc_bsg_remove(struct request_queue *q)
+{
+       if (q) {
+               bsg_unregister_queue(q);
+               blk_cleanup_queue(q);
+       }
+}
+
+

/* Original Author: Martin Hicks */
MODULE_AUTHOR("James Smart");
MODULE_DESCRIPTION("FC Transport Attributes");
diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
--- a/include/scsi/scsi_bsg_fc.h 1969-12-31
19:00:00.000000000 -0500
+++ b/include/scsi/scsi_bsg_fc.h 2009-02-10
13:26:17.000000000 -0500

include/scsi/scsi_bsg_fc.h +6 -0
@@ -0,0 +1,322 @@
+/*
+ *  FC Transport BSG Interface
+ *
+ *  Copyright (C) 2008   James Smart, Emulex Corporation
+ *
+ *  This program is free software; you can redistribute it and/or  

modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *

+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA   

02111-1307 USA
+ *
+ */

include/scsi/scsi_bsg_fc.h +24 -0
+
+#ifndef SCSI_BSG_FC_H
+#define SCSI_BSG_FC_H
+
+/*
+ * This file intended to be included by both kernel and user space
+ */
+
+#include <scsi/scsi.h>
+
+/*
+ * FC Transport SGIO v4 BSG Message Support
+ */
+
+/* Default BSG request timeout (in seconds) */
+#define FC_DEFAULT_BSG_TIMEOUT         (10 * HZ)
+
+
+/*
+ * Request Message Codes supported by the FC Transport
+ */
+
+/* define the class masks for the message codes */
+#define FC_BSG_CLS_MASK                0xF0000000      /* find  

object class */
+#define FC_BSG_HST_MASK 0x80000000 /* fc host
class */
+#define FC_BSG_RPT_MASK 0x40000000 /* fc rport
class */

include/scsi/scsi_bsg_fc.h +46 -0
+
+       /* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT           (FC_BSG_HST_MASK | 0x00000001)
+#define FC_BSG_HST_DEL_RPORT           (FC_BSG_HST_MASK | 0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN         (FC_BSG_HST_MASK | 0x00000003)
+#define FC_BSG_HST_CT                  (FC_BSG_HST_MASK | 0x00000004)
+#define FC_BSG_HST_VENDOR              (FC_BSG_HST_MASK | 0x000000FF)
+
+       /* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS                 (FC_BSG_RPT_MASK | 0x00000001)
+#define FC_BSG_RPT_CT                  (FC_BSG_RPT_MASK | 0x00000002)
+
+
+
+/*
+ * FC Address Identifiers in Message Structures :
+ *
+ *   Whenever a command payload contains a FC Address Identifier
+ *   (aka port_id), the value is effectively in big-endian
+ *   order, thus the array elements are decoded as follows:
+ *     element [0] is bits 23:16 of the FC Address Identifier
+ *     element [1] is bits 15:8 of the FC Address Identifier
+ *     element [2] is bits 7:0 of the FC Address Identifier
+ */
+
+
+/*
+ * FC Host Messages
+ */
+
+/* FC_BSG_HST_ADDR_PORT : */
+
+/* Request:
+ * This message requests the FC host to login to the remote port
+ * at the specified N_Port_Id.  The remote port is to be enumerated
+ * with the transport upon completion of the login.
+ */
+struct fc_bsg_host_add_rport {
+       uint8_t         reserved;
+
+       /* FC Address Identier of the remote port to login to */
+       uint8_t         port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is  

sufficient
+ */

include/scsi/scsi_bsg_fc.h +20 -0
+
+
+/* FC_BSG_HST_DEL_RPORT : */
+
+/* Request:
+ * This message requests the FC host to remove an enumerated
+ * remote port and to terminate the login to it.
+ *
+ * Note: The driver is free to reject this request if it desires to
+ * remain logged in with the remote port.
+ */
+struct fc_bsg_host_del_rport {
+       uint8_t         reserved;
+
+       /* FC Address Identier of the remote port to logout of */
+       uint8_t         port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is  

sufficient
+ */

include/scsi/scsi_bsg_fc.h +18 -0
+
+
+/* FC_BSG_HST_ELS_NOLOGIN : */
+
+/* Request:
+ * This message requests the FC_Host to send an ELS to a specific
+ * N_Port_ID. The host does not need to log into the remote port,
+ * nor does it need to enumerate the rport for further traffic
+ * (although, the FC host is free to do so if it desires).
+ */
+struct fc_bsg_host_els {
+       /*
+        * ELS Command Code being sent (must be the same as byte 0
+        * of the payload)
+        */
+       uint8_t         command_code;
+
+       /* FC Address Identier of the remote port to send the ELS to  

*/

include/scsi/scsi_bsg_fc.h +23 -0
+       uint8_t         port_id[3];
+};
+
+/* Response:
+ */
+/* fc_bsg_ctels_reply->status values */
+#define FC_CTELS_STATUS_OK     0x00000000
+#define FC_CTELS_STATUS_REJECT 0x00000001
+#define FC_CTELS_STATUS_P_RJT  0x00000002
+#define FC_CTELS_STATUS_F_RJT  0x00000003
+#define FC_CTELS_STATUS_P_BSY  0x00000004
+#define FC_CTELS_STATUS_F_BSY  0x00000006
+struct fc_bsg_ctels_reply {
+       /*
+        * Note: An ELS LS_RJT may be reported in 2 ways:
+        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+        *     is to look into the ELS receive payload to determine
+        *     LS_ACC or LS_RJT (by contents of word 0). The reject
+        *     data will be in word 1.
+        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+        *     rjt_data field will contain valid data.
+        *
+        * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK,  

and

include/scsi/scsi_bsg_fc.h +19 -0
+        *   the receive payload word 0 indicates LS_ACC
+        *   (e.g. value is 0x02xxxxxx).
+        *
+        * Note: Similarly, a CT Reject may be reported in 2 ways:
+        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+        *     is to look into the CT receive payload to determine
+        *     Accept or Reject (by contents of word 2). The reject
+        *     data will be in word 3.
+        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+        *     rjt_data field will contain valid data.
+        *
+        * Note: x_RJT/BSY status will indicae that the rjt_data field
+        *   is valid and contains the reason/explanation values.
+        */
+       uint32_t        status;         /* See FC_CTELS_STATUS_xxx */
+
+       /* valid if status is not FC_CTELS_STATUS_OK */
+       struct  {
+               uint8_t action;         /* fragment_id for CT REJECT  

*/

include/scsi/scsi_bsg_fc.h +13 -0
+               uint8_t reason_code;
+               uint8_t reason_explanation;
+               uint8_t vendor_unique;
+       } rjt_data;
+};
+
+
+/* FC_BSG_HST_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the
+ * indicated N_Port_ID. The driver is responsible for logging in with
+ * the fabric and/or N_Port_ID, etc as per FC rules. This request  

does
+ * not mandate that the driver must enumerate the destination in the
+ * transport. The driver is allowed to decide whether to enumerate it,
+ * and whether to tear it down after the request.
+ */

include/scsi/scsi_bsg_fc.h +4 -0
+struct fc_bsg_host_ct {
+       uint8_t         reserved;
+
+       /* FC Address Identier of the remote port to send the ELS to  

*/

include/scsi/scsi_bsg_fc.h +7 -0
+       uint8_t         port_id[3];
+
+       /*
+        * We need words 0-2 of the generic preamble for the LLD's
+        */
+       uint32_t        preamble_word0; /* revision & IN_ID */
+       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  

Options, Rsvd */

include/scsi/scsi_bsg_fc.h +13 -0
+       uint32_t        preamble_word2; /* Cmd Code, Max Size */
+
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_HST_VENDOR : */
+
+/* Request:
+ * Note: When specifying vendor_id, be sure to read the Vendor Type  

and ID
+ * formatting requirements specified in scsi_netlink.h
+ */

include/scsi/scsi_bsg_fc.h +3 -0
+struct fc_bsg_host_vendor {
+       /*
+        * Identifies the vendor that the message is formatted for.  

This

include/scsi/scsi_bsg_fc.h +44 -0
+        * should be the recipient of the message.
+        */
+       uint64_t vendor_id;
+
+       /* start of vendor command area */
+       uint32_t vendor_cmd[0];
+};
+
+/* Response:
+ */
+struct fc_bsg_host_vendor_reply {
+       /* start of vendor response area */
+       uint32_t vendor_rsp[0];
+};
+
+
+
+/*
+ * FC Remote Port Messages
+ */
+
+/* FC_BSG_RPT_ELS : */
+
+/* Request:
+ * This message requests that an ELS be performed with the rport.
+ */
+struct fc_bsg_rport_els {
+       /*
+        * ELS Command Code being sent (must be the same as
+        * byte 0 of the payload)
+        */
+       uint8_t els_code;
+};
+
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_RPT_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the  

rport.
+ */

include/scsi/scsi_bsg_fc.h +6 -0
+struct fc_bsg_rport_ct {
+       /*
+        * We need words 0-2 of the generic preamble for the LLD's
+        */
+       uint32_t        preamble_word0; /* revision & IN_ID */
+       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  

Options, Rsvd */

include/scsi/scsi_bsg_fc.h +33 -0
+       uint32_t        preamble_word2; /* Cmd Code, Max Size */
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+
+
+/* request (CDB) structure of the sg_io_v4 */
+struct fc_bsg_request {
+       uint32_t msgcode;
+       union {
+               struct fc_bsg_host_add_rport    h_addrport;
+               struct fc_bsg_host_del_rport    h_delrport;
+               struct fc_bsg_host_els          h_els;
+               struct fc_bsg_host_ct           h_ct;
+               struct fc_bsg_host_vendor       h_vendor;
+
+               struct fc_bsg_rport_els         r_els;
+               struct fc_bsg_rport_ct          r_ct;
+       } rqst_data;
+};
+
+
+/* response (request sense data) structure of the sg_io_v4 */
+struct fc_bsg_reply {
+       /*
+        * The completion result. Result exists in two forms:
+        *  if negative, it is an -Exxx system errno value. There will
+        *    be no further reply information supplied.
+        *  else, it's the 4-byte scsi error result, with driver,  

host,

include/scsi/scsi_bsg_fc.h +18 -0
+        *    msg and status fields. The per-msgcode reply structure
+        *    will contain valid data.
+        */
+       uint32_t result;
+
+       /* If there was reply_payload, how much was recevied ? */
+       uint32_t reply_payload_rcv_len;
+
+       union {
+               struct fc_bsg_host_vendor_reply         vendor_reply;
+
+               struct fc_bsg_ctels_reply               ctels_reply;
+       } reply_data;
+};
+
+
+#endif /* SCSI_BSG_FC_H */
+

diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +9 -0
@@ -478,6 +478,15 @@ struct scsi_host_template {
        * module_init/module_exit.
        */
       struct list_head legacy_hosts;
+
+       /*
+        * Vendor Identifier associated with the host
+        *
+        * Note: When specifying vendor_id, be sure to read the
+        *   Vendor Type and ID formatting requirements specified in
+        *   scsi_netlink.h
+        */
+       u64 vendor_id;
 };
 
 /*

diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h --- a/include/scsi/scsi_transport_fc.h 2009-01-27 09:44:40.000000000 -0500
+++ b/include/scsi/scsi_transport_fc.h 2009-02-10 13:12:20.000000000 -0500

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +30 -1
@@ -33,7 +33,6 @@
 
 struct scsi_transport_template;
 
-
 /*
 * FC Port definitions - Following FC HBAAPI guidelines
 *
@@ -352,6 +351,7 @@ struct fc_rport {   /* aka fc_starget_attr
       struct delayed_work fail_io_work;
       struct work_struct stgt_delete_work;
       struct work_struct rport_delete_work;
+       struct request_queue *rqst_q;   /* bsg support */
 } __attribute__((aligned(sizeof(unsigned long))));
 
 /* bit field values for struct fc_rport "flags" field: */
@@ -514,6 +514,9 @@ struct fc_host_attrs {
       struct workqueue_struct *work_q;
       char devloss_work_q_name[20];
       struct workqueue_struct *devloss_work_q;
+
+       /* bsg support */
+       struct request_queue *rqst_q;
 };
 
 #define shost_to_fc_host(x) \
@@ -579,6 +582,47 @@ struct fc_host_attrs {
       (((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
 
 
+struct fc_bsg_buffer {
+       unsigned int payload_len;
+       int sg_cnt;
+       struct scatterlist *sg_list;
+};
+
+/* Values for fc_bsg_job->state_flags (bitflags) */
+#define FC_RQST_STATE_INPROGRESS       0
+#define FC_RQST_STATE_DONE             1
+
+struct fc_bsg_job {
+       struct Scsi_Host *shost;
+       struct fc_rport *rport;
+       struct device *dev;
+       struct request *req;
+       spinlock_t job_lock;
+       unsigned int state_flags;
+       unsigned int ref_cnt;
+       void (*job_done)(struct fc_bsg_job *);
+
+       struct fc_bsg_request *request;
+       struct fc_bsg_reply *reply;
+       unsigned int request_len;
+       unsigned int reply_len;
+       /*
+        * On entry : reply_len indicates the buffer size allocated  

for

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +12 -0
+        * the reply.
+        *
+        * Upon completion : the message handler must set reply_len
+        *  to indicates the size of the reply to be returned to the
+        *  caller.
+        */
+
+       /* DMA payloads for the request/response */
+       struct fc_bsg_buffer request_payload;
+       struct fc_bsg_buffer reply_payload;
+
+       void *dd_data;                  /* Used for driver-specific  

storage */

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +3 -0
+};
+
+

/* The functions by which the transport class and the driver communicate */

struct fc_function_template {
       void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +5 -0
@@ -614,9 +658,14 @@ struct fc_function_template {
       int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64,  
 int);
       int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
 
+       /* bsg support */
+       int     (*bsg_request)(struct fc_bsg_job *);
+       int     (*bsg_timeout)(struct fc_bsg_job *);
+
       /* allocation lengths for host-specific data */
       u32                             dd_fcrport_size;
       u32                             dd_fcvport_size;
+       u32                             dd_bsg_size;
 
       /*

* The driver sets these to tell the transport class it

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +0 -1
@@ -737,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
       vport->vport_state = new_state;
 }
 
-

struct scsi_transport_template *fc_attach_transport(

                       struct fc_function_template *);

void fc_release_transport(struct scsi_transport_template *);

#23 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 9 lines · Seokmann Ju
Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer. The driver with the changes should be available within a couple of weeks.

When the FC transport layer calls fc_bsg_map_buffer(), it eventually calls blk_rq_map_sg() to build sg_list for mapped bio in the request structure. I'm having trouble, however, in getting the sg_list for both dout_xferp and din_xferp, which is mapped into the bio in the request structure. With the pass-thru - Rev V, the sg_list returned from the fc_bsg_map_buffer() seems not right, particularly, the dma_address and dma_length fields are NULL. The sg_cnt and payload_len of the request_payload/reply_payload are correctly set by the function, though.

I've been trying to figure out what causes those fields to be NULL but, no luck.

Is there something that I need to be cautious in traveling these code path?

Any comments would be helpful.

Thank you,
Seokmann

Quoted 1,371 lines

Thank you,
Seokmann

Quoted 1,366 lines

It currently expects that blk_end_request() has been updated by Fujita's patch to incorporate blk_end_bidi_request() functionality :
http://marc.info/?l-linux-scsi&m=122785157116659&w=2

-- james s

======

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

Documentation/scsi/scsi_fc_transport.txt |   11
Documentation/scsi/scsi_mid_low_api.txt  |    5
drivers/scsi/scsi_transport_fc.c         |  614 ++++++++++++++++++++ 
++++++++++-
include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
include/scsi/scsi_host.h                 |    9
include/scsi/scsi_transport_fc.h         |   52 ++
6 files changed, 1009 insertions(+), 4 deletions(-)

diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt --- a/Documentation/scsi/scsi_fc_transport.txt 2009-01-27
09:44:22.000000000 -0500
+++ b/Documentation/scsi/scsi_fc_transport.txt 2009-02-10
10:34:42.000000000 -0500

Documentation/scsi/scsi_fc_transport.txt +1 -1
@@ -1,10 +1,11 @@
                            SCSI FC Tansport
                =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008

Kernel Revisions for features:
rports : <<TBS>>
vports : 2.6.22 (? TBD)
+ bsg support : 2.6.29 (?TBD)

Introduction

Documentation/scsi/scsi_fc_transport.txt +1 -0
@@ -15,6 +16,7 @@ The FC transport can be found at:
 drivers/scsi/scsi_transport_fc.c
 include/scsi/scsi_transport_fc.h
 include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 

This file is found at Documentation/scsi/scsi_fc_transport.txt

Documentation/scsi/scsi_fc_transport.txt +2 -0
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+ 

========================================================================
+<< To Be Supplied >>

Documentation/scsi/scsi_fc_transport.txt +4 -0
+
+
+
+

Credits
=======
The following people have contributed to this document:
diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt
--- a/Documentation/scsi/scsi_mid_low_api.txt 2008-09-23
15:11:57.000000000 -0400
+++ b/Documentation/scsi/scsi_mid_low_api.txt 2009-02-10
10:34:42.000000000 -0500

(quoted diff) +0 -0
@@ -1271,6 +1271,11 @@ of interest:
   hostdata[0]  - area reserved for LLD at end of struct Scsi_Host.  

Size
is set by the second argument (named 'xtr_bytes')
to
scsi_host_alloc() or scsi_register().
+ vendor_id - a unique value that identifies the vendor
supplying
+ the LLD for the Scsi_Host. Used most often in
validating
+ vendor-specific message requests. Value
consists of an

(quoted diff) +2 -0
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid  

formats.

The scsi_host structure is defined in include/scsi/scsi_host.h

diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c --- a/drivers/scsi/scsi_transport_fc.c 2009-01-27 09:44:31.000000000 -0500
+++ b/drivers/scsi/scsi_transport_fc.c 2009-02-10 14:58:39.000000000 -0500

drivers/scsi/scsi_transport_fc.c +5 -0
@@ -35,6 +35,7 @@
 #include <linux/netlink.h>
 #include <net/netlink.h>
 #include <scsi/scsi_netlink_fc.h>
+#include <scsi/scsi_bsg_fc.h>
 #include "scsi_priv.h"
 #include "scsi_transport_fc_internal.h"
 
@@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
 static int fc_vport_setup(struct Scsi_Host *shost, int channel,
      struct device *pdev, struct fc_vport_identifiers  *ids,
      struct fc_vport **vport);
+static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs  
 *);
+static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
+static void fc_bsg_remove(struct request_queue *);
+static void fc_bsg_goose_queue(struct fc_rport *);
 
 /*

* Redefine so that we can have same named attributes in the

drivers/scsi/scsi_transport_fc.c +14 -1
@@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
              return -ENOMEM;
      }
 
+       fc_bsg_hostadd(shost, fc_host);
+       /* ignore any bsg add error - we just can't do sgio */
+
+       return 0;
+}
+
+static int fc_host_remove(struct transport_container *tc, struct  
 device *dev,
+                        struct device *cdev)
+{
+       struct Scsi_Host *shost = dev_to_shost(dev);
+       struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
+
+       fc_bsg_remove(fc_host->rqst_q);
      return 0;
 }
 
 static DECLARE_TRANSPORT_CLASS(fc_host_class,
                             "fc_host",
                             fc_host_setup,
-                              NULL,
+                              fc_host_remove,
                             NULL);
 

/*

drivers/scsi/scsi_transport_fc.c +8 -0
@@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
              scsi_flush_work(shost);
 
      fc_terminate_rport_io(rport);
+
      /*
       * Cancel any outstanding timers. These should really exist
       * only when rmmod'ing the LLDD and we're asking for
@@ -2415,6 +2434,8 @@ fc_rport_final_delete(struct work_struct
          (i->f->dev_loss_tmo_callbk))
              i->f->dev_loss_tmo_callbk(rport);
 
+       fc_bsg_remove(rport->rqst_q);
+
      transport_remove_device(dev);
      device_del(dev);
      transport_destroy_device(dev);
@@ -2502,6 +2523,9 @@ fc_rport_create(struct Scsi_Host *shost,
      transport_add_device(dev);
      transport_configure_device(dev);
 
+       fc_bsg_rportadd(shost, rport);
+       /* ignore any bsg add error - we just can't do sgio */
+
      if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
              /* initiate a scan of the target */
              rport->flags |= FC_RPORT_SCAN_PENDING;
@@ -2666,6 +2690,8 @@ fc_remote_port_add(struct Scsi_Host *sho
                                      spin_unlock_irqrestore(shost- 
 >host_lock,
                                                      flags);
 
+                               fc_bsg_goose_queue(rport);
+
                              return rport;
                      }
              }
drivers/scsi/scsi_transport_fc.c +22 -0
@@ -3351,6 +3377,592 @@ fc_vport_sched_delete(struct work_struct
 }
 
 
+/*
+ * BSG support
+ */
+
+
+/**
+ * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
+ * @job:       fc_bsg_job that is to be torn down
+ */
+static void
+fc_destroy_bsgjob(struct fc_bsg_job *job)
+{
+       unsigned long flags;
+
+       spin_lock_irqsave(&job->job_lock, flags);
+       if (job->ref_cnt) {
+               spin_unlock_irqrestore(&job->job_lock, flags);
+               return;
+       }
+       spin_unlock_irqrestore(&job->job_lock, flags);
+
+       put_device(job->dev);   /* release reference for the  

request */

drivers/scsi/scsi_transport_fc.c +9 -0
+
+       kfree(job->request_payload.sg_list);
+       kfree(job->reply_payload.sg_list);
+       kfree(job);
+}
+
+
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the  

LLD has
+ * completed
+ * @job: fc_bsg_job that is complete
+ */

drivers/scsi/scsi_transport_fc.c +17 -0
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+       struct request *req = job->req;
+       struct request *rsp = req->next_rq;
+       unsigned long flags;
+       unsigned rsp_len;
+       int err;
+
+       spin_lock_irqsave(&job->job_lock, flags);
+       job->state_flags |= FC_RQST_STATE_DONE;
+       job->ref_cnt--;
+       spin_unlock_irqrestore(&job->job_lock, flags);
+
+       err = job->req->errors = job->reply->result;
+       if (err < 0)
+               /* we're only returning the result field in the  

reply */

drivers/scsi/scsi_transport_fc.c +9 -0
+               job->req->sense_len = sizeof(uint32_t);
+       else
+               job->req->sense_len = job->reply_len;
+
+       if (rsp) {
+               rsp_len = blk_rq_bytes(rsp);
+               BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+               /* set reply (bidi) residual */
+               rsp->data_len = (rsp_len - job->reply- 
Quoted 1 line

reply_payload_rcv_len);

drivers/scsi/scsi_transport_fc.c +23 -0
+       }
+
+       /* we assume all request payload was transferred */
+       blk_end_request(req, err, blk_rq_bytes(req));
+
+       fc_destroy_bsgjob(job);
+}
+
+
+/**
+ * fc_bsg_job_timeout - handler for when a bsg request timesout
+ * @req:       request that timed out
+ */
+static enum blk_eh_timer_return
+fc_bsg_job_timeout(struct request *req)
+{
+       struct fc_bsg_job *job = (void *) req->special;
+       struct Scsi_Host *shost = job->shost;
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       unsigned long flags;
+       int err = 0, done = 0;
+
+       if (job->rport && job->rport->port_state ==  

FC_PORTSTATE_BLOCKED)

drivers/scsi/scsi_transport_fc.c +14 -0
+               return BLK_EH_RESET_TIMER;
+
+       spin_lock_irqsave(&job->job_lock, flags);
+       if (job->state_flags & FC_RQST_STATE_DONE)
+               done = 1;
+       else
+               job->ref_cnt++;
+       spin_unlock_irqrestore(&job->job_lock, flags);
+
+       if (!done && i->f->bsg_timeout) {
+               /* call LLDD to abort the i/o as it has timed out */
+               err = i->f->bsg_timeout(job);
+               if (err)
+                       printk(KERN_ERR "ERROR: FC BSG request  

timeout - LLD "
+ "abort failed with status %d\n",
err);

drivers/scsi/scsi_transport_fc.c +19 -0
+       }
+
+       if (!done) {
+               spin_lock_irqsave(&job->job_lock, flags);
+               job->ref_cnt--;
+               spin_unlock_irqrestore(&job->job_lock, flags);
+               fc_destroy_bsgjob(job);
+       }
+
+       /* the blk_end_sync_io() doesn't check the error */
+       return BLK_EH_HANDLED;
+}
+
+
+
+static int
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+       size_t sz = (sizeof(struct scatterlist) * req- 
Quoted 1 line

nr_phys_segments);

drivers/scsi/scsi_transport_fc.c +15 -0
+
+       BUG_ON(!req->nr_phys_segments);
+
+       buf->sg_list = kzalloc(sz, GFP_KERNEL);
+       if (!buf->sg_list)
+               return -ENOMEM;
+       sg_init_table(buf->sg_list, req->nr_phys_segments);
+       buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+       buf->payload_len = req->data_len;
+       return 0;
+}
+
+
+/**
+ * fc_req_to_bsgjob - Allocate/create the fc_bsg_job structure for  

the
+ * bsg request
+ * @shost: SCSI Host corresponding to the bsg object
+ * @rport: (optional) FC Remote Port corresponding to the bsg
object
+ * @req: BSG request that needs a job structure
+ */

drivers/scsi/scsi_transport_fc.c +19 -0
+static int
+fc_req_to_bsgjob(struct Scsi_Host *shost, struct fc_rport *rport,
+       struct request *req)
+{
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       struct request *rsp = req->next_rq;
+       struct fc_bsg_job *job;
+       int ret;
+
+       BUG_ON(req->special);
+
+       job = kzalloc(sizeof(struct fc_bsg_job) + i->f->dd_bsg_size,
+                       GFP_KERNEL);
+       if (!job)
+               return -ENOMEM;
+
+       /*
+        * Note: this is a bit silly.
+        * The request gets formatted as a SGIO v4 ioctl request,  

which

(quoted diff) +2 -0
+        * then gets reformatted as a blk request, which then gets
+        * reformatted as a fc bsg request. And on completion, we  

have
+ * to wrap return results such that SGIO v4 thinks it was a
scsi

drivers/scsi/scsi_transport_fc.c +14 -0
+        * status.  I hope this was all worth it.
+        */
+
+       req->special = job;
+       job->shost = shost;
+       job->rport = rport;
+       job->req = req;
+       if (i->f->dd_bsg_size)
+               job->dd_data = (void *)&job[1];
+       spin_lock_init(&job->job_lock);
+       job->request = (struct fc_bsg_request *)req->cmd;
+       job->request_len = req->cmd_len;
+       job->reply = req->sense;
+       job->reply_len = SCSI_SENSE_BUFFERSIZE; /* Size of sense  

buffer

drivers/scsi/scsi_transport_fc.c +17 -0
+                                                * allocated */
+       if (req->bio) {
+               ret = fc_bsg_map_buffer(&job->request_payload, req);
+               if (ret)
+                       goto failjob_rls_job;
+       }
+       if (rsp && rsp->bio) {
+               ret = fc_bsg_map_buffer(&job->reply_payload, rsp);
+               if (ret)
+                       goto failjob_rls_rqst_payload;
+       }
+       job->job_done = fc_bsg_jobdone;
+       if (rport)
+               job->dev = &rport->dev;
+       else
+               job->dev = &shost->shost_gendev;
+       get_device(job->dev);           /* take a reference for the  

request */

drivers/scsi/scsi_transport_fc.c +16 -0
+
+       job->ref_cnt = 1;
+
+       return 0;
+
+
+failjob_rls_rqst_payload:
+       kfree(job->request_payload.sg_list);
+failjob_rls_job:
+       kfree(job);
+       return -ENOMEM;
+}
+
+
+enum fc_dispatch_result {
+       FC_DISPATCH_BREAK,      /* on return, q is locked, break  

from q loop */
+ FC_DISPATCH_LOCKED, /* on return, q is locked, continue
on */
+ FC_DISPATCH_UNLOCKED, /* on return, q is unlocked,
continue on */

drivers/scsi/scsi_transport_fc.c +5 -0
+};
+
+
+/**
+ * fc_bsg_host_dispatch - process fc host bsg requests and  

dispatch to LLDD
+ * @shost: scsi host rport attached to
+ * @job: bsg job to be processed
+ */

(quoted diff) +2 -0
+static enum fc_dispatch_result
+fc_bsg_host_dispatch(struct request_queue *q, struct Scsi_Host  

*shost,

drivers/scsi/scsi_transport_fc.c +4 -0
+                        struct fc_bsg_job *job)
+{
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       int cmdlen = sizeof(uint32_t);  /* start with length of  

msgcode */

drivers/scsi/scsi_transport_fc.c +83 -0
+       int ret;
+
+       /* Validate the host command */
+       switch (job->request->msgcode) {
+       case FC_BSG_HST_ADD_RPORT:
+               cmdlen += sizeof(struct fc_bsg_host_add_rport);
+               break;
+
+       case FC_BSG_HST_DEL_RPORT:
+               cmdlen += sizeof(struct fc_bsg_host_del_rport);
+               break;
+
+       case FC_BSG_HST_ELS_NOLOGIN:
+               cmdlen += sizeof(struct fc_bsg_host_els);
+               /* there better be a xmt and rcv payloads */
+               if ((!job->request_payload.payload_len) ||
+                   (!job->reply_payload.payload_len)) {
+                       ret = -EINVAL;
+                       goto fail_host_msg;
+               }
+               break;
+
+       case FC_BSG_HST_CT:
+               cmdlen += sizeof(struct fc_bsg_host_ct);
+               /* there better be xmt and rcv payloads */
+               if ((!job->request_payload.payload_len) ||
+                   (!job->reply_payload.payload_len)) {
+                       ret = -EINVAL;
+                       goto fail_host_msg;
+               }
+               break;
+
+       case FC_BSG_HST_VENDOR:
+               cmdlen += sizeof(struct fc_bsg_host_vendor);
+               if ((shost->hostt->vendor_id == 0L) ||
+                   (job->request->rqst_data.h_vendor.vendor_id !=
+                       shost->hostt->vendor_id)) {
+                       ret = -ESRCH;
+                       goto fail_host_msg;
+               }
+               break;
+
+       default:
+               ret = -EBADR;
+               goto fail_host_msg;
+       }
+
+       /* check if we really have all the request data needed */
+       if (job->request_len < cmdlen) {
+               ret = -ENOMSG;
+               goto fail_host_msg;
+       }
+
+       ret = i->f->bsg_request(job);
+       if (!ret)
+               return FC_DISPATCH_UNLOCKED;
+
+fail_host_msg:
+       /* return the errno failure code as the only status */
+       BUG_ON(job->reply_len < sizeof(uint32_t));
+       job->reply->result = ret;
+       job->reply_len = sizeof(uint32_t);
+       fc_bsg_jobdone(job);
+       return FC_DISPATCH_UNLOCKED;
+}
+
+
+/*
+ * fc_bsg_goose_queue - restart rport queue in case it was stopped
+ * @rport:     rport to be restarted
+ */
+static void
+fc_bsg_goose_queue(struct fc_rport *rport)
+{
+       int flagset;
+
+       if (!rport->rqst_q)
+               return;
+
+       get_device(&rport->dev);
+
+       spin_lock(rport->rqst_q->queue_lock);
+       flagset = test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q- 
Quoted 1 line

queue_flags) &&

+ !test_bit(QUEUE_FLAG_REENTER, &rport->rqst_q-

Quoted 1 line

queue_flags);

drivers/scsi/scsi_transport_fc.c +13 -0
+       if (flagset)
+               queue_flag_set(QUEUE_FLAG_REENTER, rport->rqst_q);
+       __blk_run_queue(rport->rqst_q);
+       if (flagset)
+               queue_flag_clear(QUEUE_FLAG_REENTER, rport->rqst_q);
+       spin_unlock(rport->rqst_q->queue_lock);
+
+       put_device(&rport->dev);
+}
+
+
+/**
+ * fc_bsg_rport_dispatch - process rport bsg requests and dispatch  

to LLDD

+ * @shost:     scsi host rport attached to
+ * @rport:     rport request destined to
+ * @job:       bsg job to be processed

+ */

(quoted diff) +2 -0
+static enum fc_dispatch_result
+fc_bsg_rport_dispatch(struct request_queue *q, struct Scsi_Host  

*shost,
+ struct fc_rport *rport, struct fc_bsg_job
*job)

drivers/scsi/scsi_transport_fc.c +3 -0
+{
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       int cmdlen = sizeof(uint32_t);  /* start with length of  

msgcode */

drivers/scsi/scsi_transport_fc.c +52 -0
+       int ret;
+
+       /* Validate the rport command */
+       switch (job->request->msgcode) {
+       case FC_BSG_RPT_ELS:
+               cmdlen += sizeof(struct fc_bsg_rport_els);
+               goto check_bidi;
+
+       case FC_BSG_RPT_CT:
+               cmdlen += sizeof(struct fc_bsg_rport_ct);
+check_bidi:
+               /* there better be xmt and rcv payloads */
+               if ((!job->request_payload.payload_len) ||
+                   (!job->reply_payload.payload_len)) {
+                       ret = -EINVAL;
+                       goto fail_rport_msg;
+               }
+               break;
+       default:
+               ret = -EBADR;
+               goto fail_rport_msg;
+       }
+
+       /* check if we really have all the request data needed */
+       if (job->request_len < cmdlen) {
+               ret = -ENOMSG;
+               goto fail_rport_msg;
+       }
+
+       ret = i->f->bsg_request(job);
+       if (!ret)
+               return FC_DISPATCH_UNLOCKED;
+
+fail_rport_msg:
+       /* return the errno failure code as the only status */
+       BUG_ON(job->reply_len < sizeof(uint32_t));
+       job->reply->result = ret;
+       job->reply_len = sizeof(uint32_t);
+       fc_bsg_jobdone(job);
+       return FC_DISPATCH_UNLOCKED;
+}
+
+
+/**
+ * fc_bsg_request_handler - generic handler for bsg requests
+ * @q:         request queue to manage
+ * @shost:     Scsi_Host related to the bsg object
+ * @rport:     FC remote port related to the bsg object (optional)
+ * @dev:       device structure for bsg object
+ */
+static void
+fc_bsg_request_handler(struct request_queue *q, struct Scsi_Host  

*shost,

drivers/scsi/scsi_transport_fc.c +15 -0
+                      struct fc_rport *rport, struct device *dev)
+{
+       struct request *req;
+       struct fc_bsg_job *job;
+       enum fc_dispatch_result ret;
+
+       if (!get_device(dev))
+               return;
+
+       while (!blk_queue_plugged(q)) {
+               req = elv_next_request(q);
+               if (!req)
+                       break;
+
+               if (rport && (rport->port_state ==  

FC_PORTSTATE_BLOCKED))

drivers/scsi/scsi_transport_fc.c +5 -0
+                               break;
+
+               blkdev_dequeue_request(req);
+
+               if (rport && (rport->port_state !=  

FC_PORTSTATE_ONLINE)) {

drivers/scsi/scsi_transport_fc.c +3 -0
+                       req->errors = -ENXIO;
+                       spin_unlock_irq(q->queue_lock);
+                       blk_end_request(req, -ENXIO,  

blk_rq_bytes(req));

drivers/scsi/scsi_transport_fc.c +27 -0
+                       spin_lock_irq(q->queue_lock);
+                       continue;
+               }
+
+               spin_unlock_irq(q->queue_lock);
+
+               ret = fc_req_to_bsgjob(shost, rport, req);
+               if (ret) {
+                       req->errors = ret;
+                       blk_end_request(req, ret, blk_rq_bytes(req));
+                       spin_lock_irq(q->queue_lock);
+                       continue;
+               }
+
+               job = req->special;
+
+               /* check if we have the msgcode value at least */
+               if (job->request_len < sizeof(uint32_t)) {
+                       BUG_ON(job->reply_len < sizeof(uint32_t));
+                       job->reply->result = -ENOMSG;
+                       job->reply_len = sizeof(uint32_t);
+                       fc_bsg_jobdone(job);
+                       spin_lock_irq(q->queue_lock);
+                       continue;
+               }
+
+               /* the dispatch routines will unlock the queue_lock  

*/

(quoted diff) +2 -0
+               if (rport)
+                       ret = fc_bsg_rport_dispatch(q, shost,  

rport, job);

drivers/scsi/scsi_transport_fc.c +4 -0
+               else
+                       ret = fc_bsg_host_dispatch(q, shost, job);
+
+               /* did dispatcher hit state that can't process any  

more */

drivers/scsi/scsi_transport_fc.c +43 -0
+               if (ret == FC_DISPATCH_BREAK)
+                       break;
+
+               /* did dispatcher had released the lock */
+               if (ret == FC_DISPATCH_UNLOCKED)
+                       spin_lock_irq(q->queue_lock);
+       }
+
+       spin_unlock_irq(q->queue_lock);
+       put_device(dev);
+       spin_lock_irq(q->queue_lock);
+}
+
+
+/**
+ * fc_bsg_host_handler - handler for bsg requests for a fc host
+ * @q:         fc host request queue
+ */
+static void
+fc_bsg_host_handler(struct request_queue *q)
+{
+       struct Scsi_Host *shost = q->queuedata;
+
+       fc_bsg_request_handler(q, shost, NULL, &shost->shost_gendev);
+}
+
+
+/**
+ * fc_bsg_rport_handler - handler for bsg requests for a fc rport
+ * @q:         rport request queue
+ */
+static void
+fc_bsg_rport_handler(struct request_queue *q)
+{
+       struct fc_rport *rport = q->queuedata;
+       struct Scsi_Host *shost = rport_to_shost(rport);
+
+       fc_bsg_request_handler(q, shost, rport, &rport->dev);
+}
+
+
+/**
+ * fc_bsg_hostadd - Create and add the bsg hooks so we can receive  

requests
+ * @shost: shost for fc_host
+ * @fc_host: fc_host adding the structures to
+ */

(quoted diff) +2 -0
+static int
+fc_bsg_hostadd(struct Scsi_Host *shost, struct fc_host_attrs  

*fc_host)

drivers/scsi/scsi_transport_fc.c +44 -0
+{
+       struct device *dev = &shost->shost_gendev;
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       struct request_queue *q;
+       int err;
+       char bsg_name[BUS_ID_SIZE]; /*20*/
+
+       fc_host->rqst_q = NULL;
+
+       if (!i->f->bsg_request)
+               return -ENOTSUPP;
+
+       snprintf(bsg_name, sizeof(bsg_name),
+                "fc_host%d", shost->host_no);
+
+       q = __scsi_alloc_queue(shost, fc_bsg_host_handler);
+       if (!q) {
+               printk(KERN_ERR "fc_host%d: bsg interface failed to "
+                               "initialize - no request queue\n",
+                                shost->host_no);
+               return -ENOMEM;
+       }
+
+       q->queuedata = shost;
+       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+       blk_queue_rq_timeout(q, FC_DEFAULT_BSG_TIMEOUT);
+
+       err = bsg_register_queue(q, dev, bsg_name, NULL);
+       if (err) {
+               printk(KERN_ERR "fc_host%d: bsg interface failed to "
+                               "initialize - register queue\n",
+                               shost->host_no);
+               blk_cleanup_queue(q);
+               return err;
+       }
+
+       fc_host->rqst_q = q;
+       return 0;
+}
+
+
+/**
+ * fc_bsg_rportadd - Create and add the bsg hooks so we can  

receive requests
+ * @shost: shost that rport is attached to
+ * @rport: rport that the bsg hooks are being attached to
+ */

drivers/scsi/scsi_transport_fc.c +54 -0
+static int
+fc_bsg_rportadd(struct Scsi_Host *shost, struct fc_rport *rport)
+{
+       struct device *dev = &rport->dev;
+       struct fc_internal *i = to_fc_internal(shost->transportt);
+       struct request_queue *q;
+       int err;
+
+       rport->rqst_q = NULL;
+
+       if (!i->f->bsg_request)
+               return -ENOTSUPP;
+
+       q = __scsi_alloc_queue(shost, fc_bsg_rport_handler);
+       if (!q) {
+               printk(KERN_ERR "%s: bsg interface failed to "
+                               "initialize - no request queue\n",
+                                dev->bus_id);
+               return -ENOMEM;
+       }
+
+       q->queuedata = rport;
+       queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+       blk_queue_rq_timed_out(q, fc_bsg_job_timeout);
+       blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+       err = bsg_register_queue(q, dev, NULL, NULL);
+       if (err) {
+               printk(KERN_ERR "%s: bsg interface failed to "
+                               "initialize - register queue\n",
+                                dev->bus_id);
+               blk_cleanup_queue(q);
+               return err;
+       }
+
+       rport->rqst_q = q;
+       return 0;
+}
+
+
+/**
+ * fc_bsg_remove - Deletes the bsg hooks on fchosts/rports
+ * @q: the request_queue that is to be torn down.
+ */
+static void
+fc_bsg_remove(struct request_queue *q)
+{
+       if (q) {
+               bsg_unregister_queue(q);
+               blk_cleanup_queue(q);
+       }
+}
+
+

/* Original Author: Martin Hicks */
MODULE_AUTHOR("James Smart");
MODULE_DESCRIPTION("FC Transport Attributes");
diff -upNr a/include/scsi/scsi_bsg_fc.h b/include/scsi/scsi_bsg_fc.h
--- a/include/scsi/scsi_bsg_fc.h 1969-12-31
19:00:00.000000000 -0500
+++ b/include/scsi/scsi_bsg_fc.h 2009-02-10
13:26:17.000000000 -0500

include/scsi/scsi_bsg_fc.h +6 -0
@@ -0,0 +1,322 @@
+/*
+ *  FC Transport BSG Interface
+ *
+ *  Copyright (C) 2008   James Smart, Emulex Corporation
+ *
+ *  This program is free software; you can redistribute it and/or  

modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *

+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.

+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ */

include/scsi/scsi_bsg_fc.h +24 -0
+
+#ifndef SCSI_BSG_FC_H
+#define SCSI_BSG_FC_H
+
+/*
+ * This file intended to be included by both kernel and user space
+ */
+
+#include <scsi/scsi.h>
+
+/*
+ * FC Transport SGIO v4 BSG Message Support
+ */
+
+/* Default BSG request timeout (in seconds) */
+#define FC_DEFAULT_BSG_TIMEOUT         (10 * HZ)
+
+
+/*
+ * Request Message Codes supported by the FC Transport
+ */
+
+/* define the class masks for the message codes */
+#define FC_BSG_CLS_MASK                0xF0000000      /* find  

object class */
+#define FC_BSG_HST_MASK 0x80000000 /* fc host
class */
+#define FC_BSG_RPT_MASK 0x40000000 /* fc rport
class */

include/scsi/scsi_bsg_fc.h +3 -0
+
+       /* fc_host Message Codes */
+#define FC_BSG_HST_ADD_RPORT           (FC_BSG_HST_MASK |  

0x00000001)
+#define FC_BSG_HST_DEL_RPORT (FC_BSG_HST_MASK |
0x00000002)
+#define FC_BSG_HST_ELS_NOLOGIN (FC_BSG_HST_MASK |
0x00000003)
+#define FC_BSG_HST_CT (FC_BSG_HST_MASK |
0x00000004)
+#define FC_BSG_HST_VENDOR (FC_BSG_HST_MASK |
0x000000FF)

include/scsi/scsi_bsg_fc.h +3 -0
+
+       /* fc_rport Message Codes */
+#define FC_BSG_RPT_ELS                 (FC_BSG_RPT_MASK |  

0x00000001)
+#define FC_BSG_RPT_CT (FC_BSG_RPT_MASK |
0x00000002)

include/scsi/scsi_bsg_fc.h +35 -0
+
+
+
+/*
+ * FC Address Identifiers in Message Structures :
+ *
+ *   Whenever a command payload contains a FC Address Identifier
+ *   (aka port_id), the value is effectively in big-endian
+ *   order, thus the array elements are decoded as follows:
+ *     element [0] is bits 23:16 of the FC Address Identifier
+ *     element [1] is bits 15:8 of the FC Address Identifier
+ *     element [2] is bits 7:0 of the FC Address Identifier
+ */
+
+
+/*
+ * FC Host Messages
+ */
+
+/* FC_BSG_HST_ADDR_PORT : */
+
+/* Request:
+ * This message requests the FC host to login to the remote port
+ * at the specified N_Port_Id.  The remote port is to be enumerated
+ * with the transport upon completion of the login.
+ */
+struct fc_bsg_host_add_rport {
+       uint8_t         reserved;
+
+       /* FC Address Identier of the remote port to login to */
+       uint8_t         port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is  

sufficient
+ */

include/scsi/scsi_bsg_fc.h +20 -0
+
+
+/* FC_BSG_HST_DEL_RPORT : */
+
+/* Request:
+ * This message requests the FC host to remove an enumerated
+ * remote port and to terminate the login to it.
+ *
+ * Note: The driver is free to reject this request if it desires to
+ * remain logged in with the remote port.
+ */
+struct fc_bsg_host_del_rport {
+       uint8_t         reserved;
+
+       /* FC Address Identier of the remote port to logout of */
+       uint8_t         port_id[3];
+};
+
+/* Response:
+ * There is no additional response data - fc_bsg_reply->result is  

sufficient
+ */

include/scsi/scsi_bsg_fc.h +18 -0
+
+
+/* FC_BSG_HST_ELS_NOLOGIN : */
+
+/* Request:
+ * This message requests the FC_Host to send an ELS to a specific
+ * N_Port_ID. The host does not need to log into the remote port,
+ * nor does it need to enumerate the rport for further traffic
+ * (although, the FC host is free to do so if it desires).
+ */
+struct fc_bsg_host_els {
+       /*
+        * ELS Command Code being sent (must be the same as byte 0
+        * of the payload)
+        */
+       uint8_t         command_code;
+
+       /* FC Address Identier of the remote port to send the ELS  

to */

include/scsi/scsi_bsg_fc.h +23 -0
+       uint8_t         port_id[3];
+};
+
+/* Response:
+ */
+/* fc_bsg_ctels_reply->status values */
+#define FC_CTELS_STATUS_OK     0x00000000
+#define FC_CTELS_STATUS_REJECT 0x00000001
+#define FC_CTELS_STATUS_P_RJT  0x00000002
+#define FC_CTELS_STATUS_F_RJT  0x00000003
+#define FC_CTELS_STATUS_P_BSY  0x00000004
+#define FC_CTELS_STATUS_F_BSY  0x00000006
+struct fc_bsg_ctels_reply {
+       /*
+        * Note: An ELS LS_RJT may be reported in 2 ways:
+        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+        *     is to look into the ELS receive payload to determine
+        *     LS_ACC or LS_RJT (by contents of word 0). The reject
+        *     data will be in word 1.
+        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+        *     rjt_data field will contain valid data.
+        *
+        * Note: ELS LS_ACC is determined by an FC_CTELS_STATUS_OK,  

and

include/scsi/scsi_bsg_fc.h +12 -0
+        *   the receive payload word 0 indicates LS_ACC
+        *   (e.g. value is 0x02xxxxxx).
+        *
+        * Note: Similarly, a CT Reject may be reported in 2 ways:
+        *  a) A status of FC_CTELS_STATUS_OK is returned. The caller
+        *     is to look into the CT receive payload to determine
+        *     Accept or Reject (by contents of word 2). The reject
+        *     data will be in word 3.
+        *  b) A status of FC_CTELS_STATUS_REJECT is returned, The
+        *     rjt_data field will contain valid data.
+        *
+        * Note: x_RJT/BSY status will indicae that the rjt_data  

field

include/scsi/scsi_bsg_fc.h +7 -0
+        *   is valid and contains the reason/explanation values.
+        */
+       uint32_t        status;         /* See FC_CTELS_STATUS_xxx */
+
+       /* valid if status is not FC_CTELS_STATUS_OK */
+       struct  {
+               uint8_t action;         /* fragment_id for CT  

REJECT */

include/scsi/scsi_bsg_fc.h +12 -0
+               uint8_t reason_code;
+               uint8_t reason_explanation;
+               uint8_t vendor_unique;
+       } rjt_data;
+};
+
+
+/* FC_BSG_HST_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the
+ * indicated N_Port_ID. The driver is responsible for logging in  

with
+ * the fabric and/or N_Port_ID, etc as per FC rules. This request does
+ * not mandate that the driver must enumerate the destination in the
+ * transport. The driver is allowed to decide whether to enumerate it,
+ * and whether to tear it down after the request.
+ */

include/scsi/scsi_bsg_fc.h +4 -0
+struct fc_bsg_host_ct {
+       uint8_t         reserved;
+
+       /* FC Address Identier of the remote port to send the ELS  

to */

include/scsi/scsi_bsg_fc.h +7 -0
+       uint8_t         port_id[3];
+
+       /*
+        * We need words 0-2 of the generic preamble for the LLD's
+        */
+       uint32_t        preamble_word0; /* revision & IN_ID */
+       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  

Options, Rsvd */

include/scsi/scsi_bsg_fc.h +13 -0
+       uint32_t        preamble_word2; /* Cmd Code, Max Size */
+
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_HST_VENDOR : */
+
+/* Request:
+ * Note: When specifying vendor_id, be sure to read the Vendor  

Type and ID
+ * formatting requirements specified in scsi_netlink.h
+ */

include/scsi/scsi_bsg_fc.h +3 -0
+struct fc_bsg_host_vendor {
+       /*
+        * Identifies the vendor that the message is formatted for.  

This

include/scsi/scsi_bsg_fc.h +44 -0
+        * should be the recipient of the message.
+        */
+       uint64_t vendor_id;
+
+       /* start of vendor command area */
+       uint32_t vendor_cmd[0];
+};
+
+/* Response:
+ */
+struct fc_bsg_host_vendor_reply {
+       /* start of vendor response area */
+       uint32_t vendor_rsp[0];
+};
+
+
+
+/*
+ * FC Remote Port Messages
+ */
+
+/* FC_BSG_RPT_ELS : */
+
+/* Request:
+ * This message requests that an ELS be performed with the rport.
+ */
+struct fc_bsg_rport_els {
+       /*
+        * ELS Command Code being sent (must be the same as
+        * byte 0 of the payload)
+        */
+       uint8_t els_code;
+};
+
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+/* FC_BSG_RPT_CT : */
+
+/* Request:
+ * This message requests that a CT Request be performed with the  

rport.
+ */

include/scsi/scsi_bsg_fc.h +6 -0
+struct fc_bsg_rport_ct {
+       /*
+        * We need words 0-2 of the generic preamble for the LLD's
+        */
+       uint32_t        preamble_word0; /* revision & IN_ID */
+       uint32_t        preamble_word1; /* GS_Type, GS_SubType,  

Options, Rsvd */

include/scsi/scsi_bsg_fc.h +31 -0
+       uint32_t        preamble_word2; /* Cmd Code, Max Size */
+};
+/* Response:
+ *
+ * The reply structure is an fc_bsg_ctels_reply structure
+ */
+
+
+
+
+/* request (CDB) structure of the sg_io_v4 */
+struct fc_bsg_request {
+       uint32_t msgcode;
+       union {
+               struct fc_bsg_host_add_rport    h_addrport;
+               struct fc_bsg_host_del_rport    h_delrport;
+               struct fc_bsg_host_els          h_els;
+               struct fc_bsg_host_ct           h_ct;
+               struct fc_bsg_host_vendor       h_vendor;
+
+               struct fc_bsg_rport_els         r_els;
+               struct fc_bsg_rport_ct          r_ct;
+       } rqst_data;
+};
+
+
+/* response (request sense data) structure of the sg_io_v4 */
+struct fc_bsg_reply {
+       /*
+        * The completion result. Result exists in two forms:
+        *  if negative, it is an -Exxx system errno value. There  

will

(quoted diff) +2 -0
+        *    be no further reply information supplied.
+        *  else, it's the 4-byte scsi error result, with driver,  

host,

include/scsi/scsi_bsg_fc.h +18 -0
+        *    msg and status fields. The per-msgcode reply structure
+        *    will contain valid data.
+        */
+       uint32_t result;
+
+       /* If there was reply_payload, how much was recevied ? */
+       uint32_t reply_payload_rcv_len;
+
+       union {
+               struct fc_bsg_host_vendor_reply         vendor_reply;
+
+               struct fc_bsg_ctels_reply               ctels_reply;
+       } reply_data;
+};
+
+
+#endif /* SCSI_BSG_FC_H */
+

diff -upNr a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +9 -0
@@ -478,6 +478,15 @@ struct scsi_host_template {
       * module_init/module_exit.
       */
      struct list_head legacy_hosts;
+
+       /*
+        * Vendor Identifier associated with the host
+        *
+        * Note: When specifying vendor_id, be sure to read the
+        *   Vendor Type and ID formatting requirements specified in
+        *   scsi_netlink.h
+        */
+       u64 vendor_id;
 };
 
 /*

diff -upNr a/include/scsi/scsi_transport_fc.h b/include/scsi/scsi_transport_fc.h --- a/include/scsi/scsi_transport_fc.h 2009-01-27 09:44:40.000000000 -0500
+++ b/include/scsi/scsi_transport_fc.h 2009-02-10 13:12:20.000000000 -0500

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +30 -1
@@ -33,7 +33,6 @@
 
 struct scsi_transport_template;
 
-
 /*
 * FC Port definitions - Following FC HBAAPI guidelines
 *
@@ -352,6 +351,7 @@ struct fc_rport {   /* aka fc_starget_attr
      struct delayed_work fail_io_work;
      struct work_struct stgt_delete_work;
      struct work_struct rport_delete_work;
+       struct request_queue *rqst_q;   /* bsg support */
 } __attribute__((aligned(sizeof(unsigned long))));
 
 /* bit field values for struct fc_rport "flags" field: */
@@ -514,6 +514,9 @@ struct fc_host_attrs {
      struct workqueue_struct *work_q;
      char devloss_work_q_name[20];
      struct workqueue_struct *devloss_work_q;
+
+       /* bsg support */
+       struct request_queue *rqst_q;
 };
 
 #define shost_to_fc_host(x) \
@@ -579,6 +582,47 @@ struct fc_host_attrs {
      (((struct fc_host_attrs *)(x)->shost_data)->devloss_work_q)
 
 
+struct fc_bsg_buffer {
+       unsigned int payload_len;
+       int sg_cnt;
+       struct scatterlist *sg_list;
+};
+
+/* Values for fc_bsg_job->state_flags (bitflags) */
+#define FC_RQST_STATE_INPROGRESS       0
+#define FC_RQST_STATE_DONE             1
+
+struct fc_bsg_job {
+       struct Scsi_Host *shost;
+       struct fc_rport *rport;
+       struct device *dev;
+       struct request *req;
+       spinlock_t job_lock;
+       unsigned int state_flags;
+       unsigned int ref_cnt;
+       void (*job_done)(struct fc_bsg_job *);
+
+       struct fc_bsg_request *request;
+       struct fc_bsg_reply *reply;
+       unsigned int request_len;
+       unsigned int reply_len;
+       /*
+        * On entry : reply_len indicates the buffer size allocated  

for

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +12 -0
+        * the reply.
+        *
+        * Upon completion : the message handler must set reply_len
+        *  to indicates the size of the reply to be returned to the
+        *  caller.
+        */
+
+       /* DMA payloads for the request/response */
+       struct fc_bsg_buffer request_payload;
+       struct fc_bsg_buffer reply_payload;
+
+       void *dd_data;                  /* Used for driver-specific  

storage */

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +3 -0
+};
+
+

/* The functions by which the transport class and the driver communicate */

struct fc_function_template {
      void    (*get_rport_dev_loss_tmo)(struct fc_rport *);
include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +5 -0
@@ -614,9 +658,14 @@ struct fc_function_template {
      int     (* tsk_mgmt_response)(struct Scsi_Host *, u64, u64,  
 int);
      int     (* it_nexus_response)(struct Scsi_Host *, u64, int);
 
+       /* bsg support */
+       int     (*bsg_request)(struct fc_bsg_job *);
+       int     (*bsg_timeout)(struct fc_bsg_job *);
+
      /* allocation lengths for host-specific data */
      u32                             dd_fcrport_size;
      u32                             dd_fcvport_size;
+       u32                             dd_bsg_size;
 
      /*

* The driver sets these to tell the transport class it

include/scsi/scsi_host.h 2009-02-10 10:34:42.000000000 -0500 renamed from include/scsi/scsi_host.h 2008-10-18 10:32:54.000000000 -0400 +0 -1
@@ -737,7 +786,6 @@ fc_vport_set_state(struct fc_vport *vpor
      vport->vport_state = new_state;
 }
 
-

struct scsi_transport_template *fc_attach_transport(

                      struct fc_function_template *);

void fc_release_transport(struct scsi_transport_template *);

#24 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 2 lines · Seokmann Ju

Is there something that I need to be cautious in traveling these code path?

as mentioned in the original post... it has yet to be fully tested, so expect issues...

-- james s

#25 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to Seokmann Ju
Quoted 14 lines · On Thu, 19 Feb 2009 18:33:43 -0800 Seokmann Ju <[email protected]> wrote:
Quoted 11 lines · Seokmann Ju
Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer.
The driver with the changes should be available within a couple of weeks.

Quoted 13 lines

When the FC transport layer calls fc_bsg_map_buffer(), it eventually calls
blk_rq_map_sg() to build sg_list for mapped bio in the request structure.
I'm having trouble, however, in getting the sg_list for both dout_xferp and din_xferp, which is mapped into the bio in the request structure. With the pass-thru - Rev V, the sg_list returned from the fc_bsg_map_buffer()
seems not right, particularly, the dma_address and dma_length fields are NULL.
The sg_cnt and payload_len of the request_payload/reply_payload are correctly
set by the function, though.

I've not read the code yet but the above behavior doesn't sound a bug.

dma_map_sg is a function that sets up the dma_address and dma_length fields in a scatter list. LLDs gets a request from the transport layer and need to call dma_map_sg.

#26 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 31 lines · FUJITA Tomonori
Quoted 14 lines · On Thu, 19 Feb 2009 18:33:43 -0800 Seokmann Ju <[email protected]> wrote:
Quoted 11 lines · Seokmann Ju
Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer.
The driver with the changes should be available within a couple of weeks.

Quoted 13 lines

When the FC transport layer calls fc_bsg_map_buffer(), it eventually calls
blk_rq_map_sg() to build sg_list for mapped bio in the request structure.
I'm having trouble, however, in getting the sg_list for both dout_xferp and din_xferp, which is mapped into the bio in the request structure. With the pass-thru - Rev V, the sg_list returned from the fc_bsg_map_buffer()
seems not right, particularly, the dma_address and dma_length fields are NULL.
The sg_cnt and payload_len of the request_payload/reply_payload are correctly
set by the function, though.

Quoted 5 lines

I've not read the code yet but the above behavior doesn't sound a bug.

dma_map_sg is a function that sets up the dma_address and dma_length fields in a scatter list. LLDs gets a request from the transport layer and need to call dma_map_sg.

Thanks for the point.
Actually, we talked about this before as below, somehow, I've forgotten.
http://markmail.org/message/vfxzsupax4xhozfy

#27 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to Seokmann Ju
Quoted 9 lines · Seokmann Ju
Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer. The driver with the changes should be available within a couple of weeks.

Here is a change that I've made for transport layer while I'm testing the feature.
---

drivers/scsi/scsi_transport_fc.c +0 -0
index 63ab7bc..4bca164 100644
@@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
                BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
                /* set reply (bidi) residual */
                rsp->data_len = (rsp_len - job->reply- 
Quoted 1 line

reply_payload_rcv_len);

- }
+ blk_end_bidi_request(req, err, blk_rq_bytes(req),
+ job->reply->reply_payload_rcv_len);

-       /* we assume all request payload was transferred */
-       blk_end_request(req, err, blk_rq_bytes(req));
+       } else

+ blk_end_request(req, err, blk_rq_bytes(req));

        fc_destroy_bsgjob(job);

}
---

With this change, I'm able to issue several ELS/CT services to the qla2xxx module for those registered to the transport layer and have rport associated with them.

And, below is the driver changes for just review purpose.

Do we have time table for this support to make it available in the upstream? I think it would be good to have the feature in 2.6.30 kernel, if possible. I will continue to work on and add rest of helper function in the driver.

Thank you,
Seokmann

---

/qla_attr.c +457 -0

475 lines not on this page · raw message

/qla_def.h +1 -0
diff -Naurp qla2xxx-org//qla_def.h qla2xxx//qla_def.h
@@ -31,6 +31,7 @@
 #include <scsi/scsi_device.h>
 #include <scsi/scsi_cmnd.h>
 #include <scsi/scsi_transport_fc.h>
+#include <scsi/scsi_bsg_fc.h>
 
 #define QLA2XXX_DRIVER_NAME  "qla2xxx"
 
/qla_isr.c +64 -0

74 lines not on this page · raw message

---

#28 lore raw

Re: [RFC] FC pass thru - Rev V

Hi Seokmann,

Quoted 9 lines

Do we have time table for this support to make it available in the upstream? I think it would be good to have the feature in 2.6.30 kernel, if possible. I will continue to work on and add rest of helper function in the driver.

Thank you,
Seokmann

---

I am preparing a patch for our zfcp driver and would love to see both James&your fc_transport patch as well as the patches for our drivers in 2.6.30 too :)

Tomo, Boaz - since the current implemantation is based on that patch:
http://marc.info/?l-linux-scsi&m=122785157116659&w=2
which is not agreed upon - can we start the discussion again?
My understanding from the discussion back then was that Boaz proposed a change to the patch which Tomo agreed to. See here: http://marc.info/?l=linux-scsi&m=122812818916080&w=2
However, a new version of the patch was never posted. So would a patch with the proposed change be ok for the two of you which could be posted to Jens? I really appreciate all of your efforts, thanks a lot for that :-)
Have a nice weekend,
Sven

#29 lore raw

Re: [RFC] FC pass thru - Rev V

Hi Sven,

Quoted 8 lines · Sven Schuetz

Hi Seokmann,

Quoted 7 lines

Do we have time table for this support to make it available in the upstream?
I think it would be good to have the feature in 2.6.30 kernel, if possible.
I will continue to work on and add rest of helper function in the driver.

Quoted 8 lines
Quoted 4 lines

Thank you,
Seokmann

---

I am preparing a patch for our zfcp driver and would love to see both James&your fc_transport patch as well as the patches for our drivers in 2.6.30 too :)

Yeah, that is also my hope, too.
As I get move forward further, I will post updates for rest of work.

Thank you and have a great weekend.

Seokmann

Quoted 12 lines

Tomo, Boaz - since the current implemantation is based on that patch:
http://marc.info/?l-linux-scsi&m=122785157116659&w=2
which is not agreed upon - can we start the discussion again? My understanding from the discussion back then was that Boaz proposed a change to the patch which Tomo agreed to. See here:
http://marc.info/?l=linux-scsi&m=122812818916080&w=2
However, a new version of the patch was never posted. So would a patch with the proposed change be ok for the two of you which could be posted to Jens?
I really appreciate all of your efforts, thanks a lot for that :-) Have a nice weekend,
Sven

#30 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to Sven Schuetz
Quoted 16 lines · Sven Schuetz

Hi Seokmann,

Quoted 10 lines

Do we have time table for this support to make it available in the upstream?
I think it would be good to have the feature in 2.6.30 kernel, if possible.
I will continue to work on and add rest of helper function in the driver.

Thank you,
Seokmann

---

I am preparing a patch for our zfcp driver and would love to see both James&your fc_transport patch as well as the patches for our drivers in 2.6.30 too :)

Quoted 12 lines

Tomo, Boaz - since the current implemantation is based on that patch:
http://marc.info/?l-linux-scsi&m=122785157116659&w=2
which is not agreed upon - can we start the discussion again? My understanding from the discussion back then was that Boaz proposed a change to the patch which Tomo agreed to. See here:
http://marc.info/?l=linux-scsi&m=122812818916080&w=2
However, a new version of the patch was never posted. So would a patch with the proposed change be ok for the two of you which could be posted to Jens?
I really appreciate all of your efforts, thanks a lot for that :-) Have a nice weekend,
Sven

My humble opinion is: That none of the patches should go in. It is easy enough to use the API as it is, and it is more correct then any of the suggested alternatives.

Please point me to the latest code-base and I'll fix you a patch that I think looks very simple and will work well.

Cheers, have a good day
Boaz

#31 lore raw

Re: [RFC] FC pass thru - Rev V

Current patch is the RFC, at :
http://marc.info/?l=linux-scsi&m=123436574018579&w=2

-- james s

Quoted 10 lines · Boaz Harrosh

My humble opinion is: That none of the patches should go in. It is easy enough to use the API as it is, and it is more correct then any of the suggested alternatives.

Please point me to the latest code-base and I'll fix you a patch that I think looks very simple and will work well.

Cheers, have a good day
Boaz

#32 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 16 lines · James Smart

Current patch is the RFC, at :
http://marc.info/?l=linux-scsi&m=123436574018579&w=2

-- james s

Quoted 10 lines · Boaz Harrosh

My humble opinion is: That none of the patches should go in. It is easy enough to use the API as it is, and it is more correct then any of the suggested alternatives.

Please point me to the latest code-base and I'll fix you a patch that I think looks very simple and will work well.

Cheers, have a good day
Boaz

My changes below

Review

Quoted 7 lines
(quoted diff) +7 -0
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	unsigned rsp_len;

- unsigned rsp_len;
+ unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

Quoted 12 lines
(quoted diff) +12 -0
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else

Review

Quoted 10 lines
(quoted diff) +10 -0
+		job->req->sense_len = job->reply_len;
+
+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	/* we assume all request payload was transferred */

- /* we assume all request payload was transferred */
+ /* we assume all request payload was transferred, residual == 0 */
+ req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer residual of all payload. On the other hand you might have more precise residual information here, as received from the scsi transport? (Specially in the error case, perhaps in the future).

Review

Quoted 1 line
(quoted diff) +1 -0
+	blk_end_request(req, err, blk_rq_bytes(req));

- blk_end_request(req, err, blk_rq_bytes(req));
+ blk_end_bidi_request(req, err, req_len, rsp_len);

/qla_attr.c qla2x00_terminate_rport_io(struct fc_rpo · line 1088

Quoted 3 lines
/qla_attr.c +3 -0
+
+	fc_destroy_bsgjob(job);
+}

This should take care of all possible cases, (bidi or otherwise), and will return the proper transfer residual count of 0 to user-mode.

Thanks
Boaz

#33 lore raw

Re: [RFC] FC pass thru - Rev V

Boaz Harrosh wrote:
<snip..>

Quoted 23 lines
Quoted 8 lines
(quoted diff) +8 -0
+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	/* we assume all request payload was transferred */
(quoted diff) +2 -1
-	/* we assume all request payload was transferred */
+	/* we assume all request payload was transferred, residual == 0 */
+	req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer residual of all payload. On the other hand you might have more precise residual information here, as received from the scsi transport? (Specially in the error case, perhaps in the future).

Quoted 1 line
(quoted diff) +1 -0
+	blk_end_request(req, err, blk_rq_bytes(req));
(quoted diff) +1 -1
-	blk_end_request(req, err, blk_rq_bytes(req));
+	blk_end_bidi_request(req, err, req_len, rsp_len);
Quoted 6 lines

/qla_attr.c qla2x00_terminate_rport_io(struct fc_rpo · line 1088

Quoted 3 lines
/qla_attr.c +3 -0
+
+	fc_destroy_bsgjob(job);
+}

This should take care of all possible cases, (bidi or otherwise), and will return the proper transfer residual count of 0 to user-mode.

We should be calling blk_end_bidi_request() even in cases where there wasn't a req->next_rq ?

Here's what I would have done. Has your changes for setting req_len and req->data_len, and Seokman's mods for calling either blk_end_bidi_request() or blk_end_request() depending on whether there is a rsp request.

-- james s

/**
* fc_bsg_jobdone - completion routine for bsg requests that the LLD has
* completed
* @job: fc_bsg_job that is complete
*/
static void
fc_bsg_jobdone(struct fc_bsg_job *job)

{
	struct request *req = job->req;
	struct request *rsp = req->next_rq;
	unsigned long flags;
	unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

	int err;

	spin_lock_irqsave(&job->job_lock, flags);
	job->state_flags |= FC_RQST_STATE_DONE;
	job->ref_cnt--;
	spin_unlock_irqrestore(&job->job_lock, flags);

	err = job->req->errors = job->reply->result;
	if (err < 0)
		/* we're only returning the result field in the reply */
		job->req->sense_len = sizeof(uint32_t);
	else
		job->req->sense_len = job->reply_len;

	/* we assume all request payload was transferred, residual == 0 */
	req->data_len = 0;

	if (rsp) {
		rsp_len = blk_rq_bytes(rsp);
		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
		/* set reply (bidi) residual */
		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
		blk_end_bidi_request(req, err, req_len, rsp_len);
	} else
		blk_end_request(req, err, req_len);

	fc_destroy_bsgjob(job);
}
#34 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 73 lines · James Smart

Boaz Harrosh wrote:
<snip..>

Quoted 21 lines
Quoted 8 lines
(quoted diff) +8 -0
+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	/* we assume all request payload was transferred */
(quoted diff) +2 -1
-	/* we assume all request payload was transferred */
+	/* we assume all request payload was transferred, residual == 0 */
+	req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer residual of all payload. On the other hand you might have more precise residual information here, as received from the scsi transport? (Specially in the error case, perhaps in the future).

Quoted 1 line
(quoted diff) +1 -0
+	blk_end_request(req, err, blk_rq_bytes(req));
(quoted diff) +1 -1
-	blk_end_request(req, err, blk_rq_bytes(req));
+	blk_end_bidi_request(req, err, req_len, rsp_len);
Quoted 5 lines

/qla_attr.c qla2x00_terminate_rport_io(struct fc_rpo · line 1088

Quoted 3 lines
/qla_attr.c +3 -0
+
+	fc_destroy_bsgjob(job);
+}

This should take care of all possible cases, (bidi or otherwise), and will return the proper transfer residual count of 0 to user-mode.

We should be calling blk_end_bidi_request() even in cases where there wasn't a req->next_rq ?

Here's what I would have done. Has your changes for setting req_len and req->data_len, and Seokman's mods for calling either blk_end_bidi_request() or blk_end_request() depending on whether there is a rsp request.

-- james s

/**
* fc_bsg_jobdone - completion routine for bsg requests that the LLD has
* completed
* @job: fc_bsg_job that is complete
*/
static void
fc_bsg_jobdone(struct fc_bsg_job *job)

{
	struct request *req = job->req;
	struct request *rsp = req->next_rq;
	unsigned long flags;
	unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

	int err;

	spin_lock_irqsave(&job->job_lock, flags);
	job->state_flags |= FC_RQST_STATE_DONE;
	job->ref_cnt--;
	spin_unlock_irqrestore(&job->job_lock, flags);

	err = job->req->errors = job->reply->result;
	if (err < 0)
		/* we're only returning the result field in the reply */
		job->req->sense_len = sizeof(uint32_t);
	else
		job->req->sense_len = job->reply_len;

	/* we assume all request payload was transferred, residual == 0 */
	req->data_len = 0;

	if (rsp) {
		rsp_len = blk_rq_bytes(rsp);
		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
Quoted 8 lines
		/* set reply (bidi) residual */
		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
		blk_end_bidi_request(req, err, req_len, rsp_len);
	} else
		blk_end_request(req, err, req_len);

	fc_destroy_bsgjob(job);
}

No! Please most certainly don't. this is the all issue here. blk_end_request is just a wrapper for an internal blk_end_bidi_request(). They are essentially the same exact call when the first just puts a zero at bidi_bytes.

My code is the preferred and easiest and most correct way. Again calling blk_end_bidi_request() is absolutely always safe, only thing missing is a proper comment for blk_end_bidi_request() that says so.

Cheers
Boaz

#35 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to James Smart
Quoted 16 lines · James Smart

Current patch is the RFC, at :
http://marc.info/?l=linux-scsi&m=123436574018579&w=2

-- james s

Quoted 10 lines · Boaz Harrosh

My humble opinion is: That none of the patches should go in. It is easy enough to use the API as it is, and it is more correct then any of the suggested alternatives.

Please point me to the latest code-base and I'll fix you a patch that I think looks very simple and will work well.

Cheers, have a good day
Boaz

My changes below

Review

Quoted 7 lines
(quoted diff) +7 -0
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	unsigned rsp_len;

- unsigned rsp_len;
+ unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

Quoted 12 lines
(quoted diff) +12 -0
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else

Review

Quoted 10 lines
(quoted diff) +10 -0
+		job->req->sense_len = job->reply_len;
+
+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	/* we assume all request payload was transferred */

- /* we assume all request payload was transferred */
+ /* we assume all request payload was transferred, residual == 0 */
+ req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer residual of all payload. On the other hand you might have more precise residual information here, as received from the scsi transport? (Specially in the error case, perhaps in the future).

Review

Quoted 1 line
(quoted diff) +1 -0
+	blk_end_request(req, err, blk_rq_bytes(req));

- blk_end_request(req, err, blk_rq_bytes(req));
+ blk_end_bidi_request(req, err, req_len, rsp_len);

/qla_attr.c qla2x00_terminate_rport_io(struct fc_rpo · line 1088

Quoted 3 lines
/qla_attr.c +3 -0
+
+	fc_destroy_bsgjob(job);
+}

This should take care of all possible cases, (bidi or otherwise), and will return the proper transfer residual count of 0 to user-mode.

Thanks
Boaz

(Sorry if it's a repost, we had a small email outage)

#36 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to Boaz Harrosh
Quoted 30 lines · On Sun, 15 Mar 2009 11:34:59 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 16 lines · Sven Schuetz

Hi Seokmann,

Quoted 10 lines

Do we have time table for this support to make it available in the upstream?
I think it would be good to have the feature in 2.6.30 kernel, if possible.
I will continue to work on and add rest of helper function in the driver.

Thank you,
Seokmann

---

I am preparing a patch for our zfcp driver and would love to see both James&your fc_transport patch as well as the patches for our drivers in 2.6.30 too :)

Quoted 12 lines

Tomo, Boaz - since the current implemantation is based on that patch:
http://marc.info/?l-linux-scsi&m=122785157116659&w=2
which is not agreed upon - can we start the discussion again? My understanding from the discussion back then was that Boaz proposed a change to the patch which Tomo agreed to. See here:
http://marc.info/?l=linux-scsi&m=122812818916080&w=2
However, a new version of the patch was never posted. So would a patch with the proposed change be ok for the two of you which could be posted to Jens?
I really appreciate all of your efforts, thanks a lot for that :-) Have a nice weekend,
Sven

Quoted 3 lines

My humble opinion is: That none of the patches should go in. It is easy enough to use the API as it is, and it is more correct then any of the suggested alternatives.

It's pretty clear that the current API is pretty confusing. If not, why do you need to tell SCSI developers how to use it again and again?

Please don't say 'add comments along with the API'. Commenting doesn't fix the confusing API. People can use good API without any explanation.

But it's better to live with the current API for now. We can fix the API latter. I'll read Tejun's block cleanup patchset and figure out what to do.

#37 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to Seokmann Ju

Thank you Seokmann. I'm glad the RFC worked well for you. Your patch is appreciated, it was the portion that needed to be reverted as the collapsed blk_end_request() patch wasn't taken. To double check - there are no patches required with the RFC now - right ?

I'll repost as a non-RFC, and we'll see where James takes it.

-- james s

Quoted 9 lines · Seokmann Ju

On Feb 11, 2009, at 7:43 AM, Seokmann Ju wrote:

Here is a change that I've made for transport layer while I'm testing the feature.
---

drivers/scsi/ renamed from drivers/scsi/scsi_transport_fc.c +0 -0

scsi_transport_fc.c
index 63ab7bc..4bca164 100644

<snip...>

Quoted 16 lines

With this change, I'm able to issue several ELS/CT services to the qla2xxx module for those registered to the transport layer and have rport associated with them.

And, below is the driver changes for just review purpose.

Do we have time table for this support to make it available in the upstream?
I think it would be good to have the feature in 2.6.30 kernel, if possible.
I will continue to work on and add rest of helper function in the driver.

Thank you,
Seokmann

#38 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 4 lines · James Smart

Thank you Seokmann. I'm glad the RFC worked well for you. Your patch is appreciated, it was the portion that needed to be reverted as the collapsed blk_end_request() patch wasn't taken. To double check - there are no patches required with the RFC now - right ?

Yes, for those ELS/CT service handling through rports, it works for me.

Thank you,
Seokmann

#39 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to James Smart
Quoted 6 lines · James Smart

Thank you Seokmann. I'm glad the RFC worked well for you. Your patch is appreciated, it was the portion that needed to be reverted as the collapsed blk_end_request() patch wasn't taken. To double check - there are no patches required with the RFC now - right ?

I'll repost as a non-RFC, and we'll see where James takes it.

Could you share your plan on this?
Just trying to figure out when we will have it in the upstream tree, hoping that 2.6.30 is viable target.

Thank you,
Seokmann

#40 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to Seokmann Ju
Quoted 21 lines · Seokmann Ju
Quoted 10 lines · Seokmann Ju
Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer.
The driver with the changes should be available within a couple of weeks.

Here is a change that I've made for transport layer while I'm testing the feature.
---

drivers/scsi/ renamed from drivers/scsi/scsi_transport_fc.c +0 -0

scsi_transport_fc.c
index 63ab7bc..4bca164 100644

drivers/scsi/scsi_transport_fc.c fc_bsg_jobdone(struct fc_bsg_job *job) · line 3438

Quoted 4 lines
drivers/scsi/scsi_transport_fc.c +0 -0
@@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
                 BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
                 /* set reply (bidi) residual */
                 rsp->data_len = (rsp_len - job->reply- 
 >reply_payload_rcv_len);

This part looks a bit white-space broken.

drivers/scsi/scsi_transport_fc.c fc_bsg_jobdone(struct fc_bsg_job *job) · line 3443

Quoted 8 lines
drivers/scsi/scsi_transport_fc.c +4 -3
-       }
+               blk_end_bidi_request(req, err, blk_rq_bytes(req),
+                   job->reply->reply_payload_rcv_len);
 
-       /* we assume all request payload was transferred */
-       blk_end_request(req, err, blk_rq_bytes(req));
+       } else
+               blk_end_request(req, err, blk_rq_bytes(req));

I still do not like this version for more then one reason. There is a more proper simpler way.

Where is the code-base this is patching? I'll make a proper patch for you.

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +0 -1
         fc_destroy_bsgjob(job);
  }
---

With this change, I'm able to issue several ELS/CT services to the qla2xxx module for those registered to the transport layer and have rport associated with them.

<snip>

Boaz

#41 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 38 lines · Boaz Harrosh
Quoted 21 lines · Seokmann Ju
Quoted 10 lines · Seokmann Ju
Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer.
The driver with the changes should be available within a couple of weeks.

Here is a change that I've made for transport layer while I'm testing the feature.
---

drivers/scsi/ renamed from drivers/scsi/scsi_transport_fc.c +0 -0

scsi_transport_fc.c
index 63ab7bc..4bca164 100644

drivers/scsi/scsi_transport_fc.c fc_bsg_jobdone(struct fc_bsg_job *job) · line 3438

Quoted 4 lines
drivers/scsi/scsi_transport_fc.c +0 -0
@@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
                BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
                /* set reply (bidi) residual */
                rsp->data_len = (rsp_len - job->reply-
 > reply_payload_rcv_len);

This part looks a bit white-space broken.

drivers/scsi/scsi_transport_fc.c fc_bsg_jobdone(struct fc_bsg_job *job) · line 3443

Quoted 8 lines
drivers/scsi/scsi_transport_fc.c +4 -3
-       }
+               blk_end_bidi_request(req, err, blk_rq_bytes(req),
+                   job->reply->reply_payload_rcv_len);
 
-       /* we assume all request payload was transferred */
-       blk_end_request(req, err, blk_rq_bytes(req));
+       } else
+               blk_end_request(req, err, blk_rq_bytes(req));
Quoted 4 lines

I still do not like this version for more then one reason. There is a more proper simpler way.

Where is the code-base this is patching? I'll make a proper patch for you.

I took James S's patch from here.
http://lwn.net/Articles/318731/

Thank you,
Seokmann

#42 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 38 lines · Seokmann Ju
Quoted 36 lines · Boaz Harrosh
Quoted 21 lines · Seokmann Ju
Quoted 10 lines · Seokmann Ju
Quoted 4 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

I'm looking the driver changes to match it with latest updates in the transport layer.
The driver with the changes should be available within a couple of weeks.

Here is a change that I've made for transport layer while I'm testing the feature.
---

drivers/scsi/ renamed from drivers/scsi/scsi_transport_fc.c +0 -0

scsi_transport_fc.c
index 63ab7bc..4bca164 100644

drivers/scsi/scsi_transport_fc.c fc_bsg_jobdone(struct fc_bsg_job *job) · line 3438

Quoted 4 lines
drivers/scsi/scsi_transport_fc.c +0 -0
@@ -3435,10 +3435,11 @@ fc_bsg_jobdone(struct fc_bsg_job *job)
                BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
                /* set reply (bidi) residual */
                rsp->data_len = (rsp_len - job->reply-
 > reply_payload_rcv_len);

This part looks a bit white-space broken.

Quoted 8 lines
drivers/scsi/scsi_transport_fc.c +4 -3
-       }
+               blk_end_bidi_request(req, err, blk_rq_bytes(req),
+                   job->reply->reply_payload_rcv_len);
 
-       /* we assume all request payload was transferred */
-       blk_end_request(req, err, blk_rq_bytes(req));
+       } else
+               blk_end_request(req, err, blk_rq_bytes(req));
Quoted 8 lines
Quoted 5 lines

I still do not like this version for more then one reason. There is a more proper simpler way.

Where is the code-base this is patching? I'll make a proper patch for you.

I took James S's patch from here.
http://lwn.net/Articles/318731/

Not good, sorry. Quoted the original patch in link above please change lines below

Review

Quoted 7 lines · James Smart
(quoted diff) +7 -0
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	unsigned rsp_len;

- unsigned rsp_len;
+ unsigned rsp_len = 0, req_len = blk_rq_bytes(req);

Quoted 12 lines
(quoted diff) +12 -0
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else

Review

Quoted 10 lines
(quoted diff) +10 -0
+		job->req->sense_len = job->reply_len;
+
+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	/* we assume all request payload was transferred */

- /* we assume all request payload was transferred */
+ /* we assume all request payload was transferred, residual == 0 */
+ req->data_len = 0;

Note: this is the proper procedure. Otherwise bsg user will receive transfer residual of all payload. On the other hand you might have more precise residual information here, as received from the scsi transport? (Specially in the error case, perhaps in the future).

Review

Quoted 1 line
(quoted diff) +1 -0
+	blk_end_request(req, err, blk_rq_bytes(req));

- blk_end_request(req, err, blk_rq_bytes(req));
+ blk_end_bidi_request(req, err, req_len, rsp_len);

Quoted 6 lines
/qla_attr.c +3 -0
+
+	fc_destroy_bsgjob(job);
+}

Thank you,
Seokmann

This should take care of all possible cases, (bidi or otherwise), and will return the proper transfer residual count of 0 to user-mode.

Thanks
Boaz

#43 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 1 line · Boaz Harrosh

Seokmann Ju wrote:

<snip>

Quoted 6 lines
Quoted 3 lines

I took James S's patch from here.
http://lwn.net/Articles/318731/

Not good, sorry. Quoted the original patch in link above please change lines below

Yes, Now I've got all relevant emails that you've been posted over the weekend. My email seems a bit slow today for some reason.

Thank you,
Seokmann

#44 lore raw

Re: [RFC] FC pass thru - Rev V

In reply to James Smart
Quoted 10 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

It currently expects that blk_end_request() has been updated by Fujita's patch to incorporate blk_end_bidi_request() functionality :
http://marc.info/?l-linux-scsi&m=122785157116659&w=2

I did not accept this patch and it did not go in right? I still don't like it, it's a performance regression.

Quoted 259 lines

-- james s

======

All,

I've reworked Seokmann's patch for the following items:
- Add an fchost interface for bsg requests

- Formalized the request/response structures that I expect to have us stuff into the bsg cmd/sense data areas. These are now genericized so we can essentially pass any kind of transaction. It can be a request that has no transmit or receive payload, and simply returns a response.

- A new file was created, scsi_bsg_fc.h, which contains the request/response data structures that should be shared between the application and the kernel entities.

- I stripped out some things that were in the request structure that were actually LLD fields. Instead, I added a dd_bsgsize structure to the template, so the transport will allocate LLD work space along with the job structure. I expect the missing fields to move to this area.

- I've made a strong attempt at ensuring that the request has all the information necessary for the LLD, so that there is no need to have the LLD remap the transmit payload to figure things out. Granted, this comes at the cost of replicating some data items.

Sven, I've added the CT information you needed as part of this.

- I've renamed things quite a bit, hoping to make it clarity better. The "service" struct is now a job. I still have headaches with "request" (is it the blk request, or the job request, or what..)

- The CT/ELS response is a bit funky. I've noted that the way Emulex returns a response, vs Qlogic is a bit different, thus the 2 ways to indicate "reject".

- fixed a couple of bugs in Seokmann's code, in the teardown, error flows, request que dma settings, etc.

- I added a "vendor_id" field to the scsi_host_template to use when verifying that the recipient knows how to decode vendor-specific message. I didn't do this with the netlink things as I was prepping it to not break kabi in existing and older kernels. But, I believe this is a good time to add it.

- I've started the Documentation/scsi/scsi_transport_fc.txt documentation, but punted finishing it in lieu of sending this RFC. I'm starting from Seokman's original emails and will be updating for this reformat.

I'm only starting to debug this, so user beware.

I could really use some code review from Fujita or Boaz, to make sure I'm calling the right blk_xx completion functions relative to the setup flow, and to ensure that the "goose" when I jump out while the rport is blocked is correct.

Comments welcome

-- james s

---

 Documentation/scsi/scsi_fc_transport.txt |   11 
 Documentation/scsi/scsi_mid_low_api.txt  |    5 
 drivers/scsi/scsi_transport_fc.c         |  614 ++++++++++++++++++++++++++++++-
 include/scsi/scsi_bsg_fc.h               |  322 ++++++++++++++++
 include/scsi/scsi_host.h                 |    9 
 include/scsi/scsi_transport_fc.h         |   52 ++
 6 files changed, 1009 insertions(+), 4 deletions(-)

diff -upNr a/Documentation/scsi/scsi_fc_transport.txt b/Documentation/scsi/scsi_fc_transport.txt

Documentation/scsi/scsi_fc_transport.txt +10 -1
@@ -1,10 +1,11 @@
                              SCSI FC Tansport
                  =============================================
 
-Date:  4/12/2007
+Date:  11/18/2008
 Kernel Revisions for features:
   rports : <<TBS>>
   vports : 2.6.22 (? TBD)
+  bsg support : 2.6.29 (?TBD)
 
 
 Introduction
@@ -15,6 +16,7 @@ The FC transport can be found at:
   drivers/scsi/scsi_transport_fc.c
   include/scsi/scsi_transport_fc.h
   include/scsi/scsi_netlink_fc.h
+  include/scsi/scsi_bsg_fc.h
 
 This file is found at Documentation/scsi/scsi_fc_transport.txt
 
@@ -472,6 +474,13 @@ int
 fc_vport_terminate(struct fc_vport *vport)
 
 
+FC BSG support (CT & ELS passthru, and more)
+========================================================================
+<< To Be Supplied >>
+
+
+
+
 Credits
 =======
 The following people have contributed to this document:

diff -upNr a/Documentation/scsi/scsi_mid_low_api.txt b/Documentation/scsi/scsi_mid_low_api.txt

Documentation/scsi/scsi_mid_low_api.txt +5 -0
@@ -1271,6 +1271,11 @@ of interest:
     hostdata[0]  - area reserved for LLD at end of struct Scsi_Host. Size
                    is set by the second argument (named 'xtr_bytes') to
                    scsi_host_alloc() or scsi_register().
+    vendor_id    - a unique value that identifies the vendor supplying
+                   the LLD for the Scsi_Host.  Used most often in validating
+                   vendor-specific message requests.  Value consists of an
+                   identifier type and a vendor-specific value.
+                   See scsi_netlink.h for a description of valid formats.
 
 The scsi_host structure is defined in include/scsi/scsi_host.h
 

diff -upNr a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c

drivers/scsi/scsi_transport_fc.c +58 -1
@@ -35,6 +35,7 @@
 #include <linux/netlink.h>
 #include <net/netlink.h>
 #include <scsi/scsi_netlink_fc.h>
+#include <scsi/scsi_bsg_fc.h>
 #include "scsi_priv.h"
 #include "scsi_transport_fc_internal.h"
 
@@ -43,6 +44,10 @@ static void fc_vport_sched_delete(struct
 static int fc_vport_setup(struct Scsi_Host *shost, int channel,
 	struct device *pdev, struct fc_vport_identifiers  *ids,
 	struct fc_vport **vport);
+static int fc_bsg_hostadd(struct Scsi_Host *, struct fc_host_attrs *);
+static int fc_bsg_rportadd(struct Scsi_Host *, struct fc_rport *);
+static void fc_bsg_remove(struct request_queue *);
+static void fc_bsg_goose_queue(struct fc_rport *);
 
 /*
  * Redefine so that we can have same named attributes in the
@@ -411,13 +416,26 @@ static int fc_host_setup(struct transpor
 		return -ENOMEM;
 	}
 
+	fc_bsg_hostadd(shost, fc_host);
+	/* ignore any bsg add error - we just can't do sgio */
+
+	return 0;
+}
+
+static int fc_host_remove(struct transport_container *tc, struct device *dev,
+			 struct device *cdev)
+{
+	struct Scsi_Host *shost = dev_to_shost(dev);
+	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
+
+	fc_bsg_remove(fc_host->rqst_q);
 	return 0;
 }
 
 static DECLARE_TRANSPORT_CLASS(fc_host_class,
 			       "fc_host",
 			       fc_host_setup,
-			       NULL,
+			       fc_host_remove,
 			       NULL);
 
 /*
@@ -2383,6 +2401,7 @@ fc_rport_final_delete(struct work_struct
 		scsi_flush_work(shost);
 
 	fc_terminate_rport_io(rport);
+
 	/*
 	 * Cancel any outstanding timers. These should really exist
 	 * only when rmmod'ing the LLDD and we're asking for
@@ -2415,6 +2434,8 @@ fc_rport_final_delete(struct work_struct
 	    (i->f->dev_loss_tmo_callbk))
 		i->f->dev_loss_tmo_callbk(rport);
 
+	fc_bsg_remove(rport->rqst_q);
+
 	transport_remove_device(dev);
 	device_del(dev);
 	transport_destroy_device(dev);
@@ -2502,6 +2523,9 @@ fc_rport_create(struct Scsi_Host *shost,
 	transport_add_device(dev);
 	transport_configure_device(dev);
 
+	fc_bsg_rportadd(shost, rport);
+	/* ignore any bsg add error - we just can't do sgio */
+
 	if (rport->roles & FC_PORT_ROLE_FCP_TARGET) {
 		/* initiate a scan of the target */
 		rport->flags |= FC_RPORT_SCAN_PENDING;
@@ -2666,6 +2690,8 @@ fc_remote_port_add(struct Scsi_Host *sho
 					spin_unlock_irqrestore(shost->host_lock,
 							flags);
 
+				fc_bsg_goose_queue(rport);
+
 				return rport;
 			}
 		}
@@ -3351,6 +3377,592 @@ fc_vport_sched_delete(struct work_struct
 }
 
 
+/*
+ * BSG support
+ */
+
+
+/**
+ * fc_destroy_bsgjob - routine to teardown/delete a fc bsg job
+ * @job:	fc_bsg_job that is to be torn down
+ */
+static void
+fc_destroy_bsgjob(struct fc_bsg_job *job)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	if (job->ref_cnt) {
+		spin_unlock_irqrestore(&job->job_lock, flags);
+		return;
+	}
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	put_device(job->dev);	/* release reference for the request */
+
+	kfree(job->request_payload.sg_list);
+	kfree(job->reply_payload.sg_list);
+	kfree(job);
+}
+
+
+/**
+ * fc_bsg_jobdone - completion routine for bsg requests that the LLD has

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3420

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+ *                  completed
+ * @job:	fc_bsg_job that is complete
+ */
+static void
+fc_bsg_jobdone(struct fc_bsg_job *job)
+{
+	struct request *req = job->req;
+	struct request *rsp = req->next_rq;
+	unsigned long flags;
+	unsigned rsp_len;

- unsigned rsp_len;
+ unsigned rsp_len = 0;
+ unsigned req_len = 0;

See below

Quoted 4 lines
drivers/scsi/scsi_transport_fc.c +4 -0
+	int err;
+
+	spin_lock_irqsave(&job->job_lock, flags);
+	job->state_flags |= FC_RQST_STATE_DONE;

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3434

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+	job->ref_cnt--;
+	spin_unlock_irqrestore(&job->job_lock, flags);
+
+	err = job->req->errors = job->reply->result;
+	if (err < 0)
+		/* we're only returning the result field in the reply */
+		job->req->sense_len = sizeof(uint32_t);
+	else
+		job->req->sense_len = job->reply_len;
+

Why use of job->req when you have req = job->req above, it's confusing.

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3443

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+	if (rsp) {
+		rsp_len = blk_rq_bytes(rsp);
+		BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+		/* set reply (bidi) residual */
+		rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+	}
+
+	/* we assume all request payload was transferred */
+	blk_end_request(req, err, blk_rq_bytes(req));

Don't you need residual count in bsg caller?

This code will always return full request->data_len as residual count of out/uni_in to caller of bsg SG_IO caller.

and it will only work with the un-accepted patch above just do:

+ if (rsp) {
+ rsp_len = blk_rq_bytes(rsp);
+ BUG_ON(job->reply->reply_payload_rcv_len > rsp_len);
+ /* set reply (bidi) residual */
+ rsp->data_len = (rsp_len - job->reply->reply_payload_rcv_len);
+ }
+
+ req_len = blk_rq_bytes(req);
+ /* we assume all request payload was transferred */
+ req->data_len = 0;
+ blk_end_bidi_request(req, err, req_len, rsp_len);

This will handle all cases today without need of above patch.

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3448

Quoted 5 lines
drivers/scsi/scsi_transport_fc.c +5 -0
+
+	fc_destroy_bsgjob(job);
+}
+
+

<snip>

Thanks
Boaz

#45 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 6 lines · On Wed, 11 Feb 2009 18:15:07 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 5 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

Quoted 7 lines
Quoted 5 lines

It currently expects that blk_end_request() has been updated by Fujita's patch to incorporate blk_end_bidi_request() functionality :
http://marc.info/?l-linux-scsi&m=122785157116659&w=2

I did not accept this patch and it did not go in right?

I think that Jens has not merged mine or yours. I don't care about either but I still think that it's better to kill blk_end_bidi_request(). It's really confusing API.

Quoted 1 line

I still don't like it, it's a performance regression.

Hmm, I've not seen the figures. Please show the figures if you insist a performance regression.

It's about a bidi request. We already have tons of loops, memory allocations, etc in the path. Do you think that adding one more loop leads to a notable performance regression?

Well, if you say that it's hacky then I would agree. But your patch using ~0 is hacky too.

#46 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 21 lines · FUJITA Tomonori
Quoted 6 lines · On Wed, 11 Feb 2009 18:15:07 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 5 lines · James Smart

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

Quoted 6 lines
Quoted 5 lines

It currently expects that blk_end_request() has been updated by Fujita's patch to incorporate blk_end_bidi_request() functionality :
http://marc.info/?l-linux-scsi&m=122785157116659&w=2

I did not accept this patch and it did not go in right?

I think that Jens has not merged mine or yours. I don't care about either but I still think that it's better to kill blk_end_bidi_request(). It's really confusing API.

Quoted 5 lines
Quoted 1 line

I still don't like it, it's a performance regression.

Hmm, I've not seen the figures. Please show the figures if you insist a performance regression.

Come on man. this discussion all over again. You do a loop to find information that was in a CPU register 10 cycles before. That is plain bad programing, and just a cover up of bad API.

In my book it is: Someone got lazy.

Quoted 6 lines

It's about a bidi request. We already have tons of loops, memory allocations, etc in the path. Do you think that adding one more loop leads to a notable performance regression?

Well, if you say that it's hacky then I would agree. But your patch using ~0 is hacky too.

It is an hack if used by an outside user, because it assumes knowledge of block-internals. It is much less of an hack if done by block-internals which knows for sure that this has no side effects.

But I agree that this is not clean. The clean solution is to add an extra parameter to blk_end_request() and change all callers.

Or even cleaner is to add a new request->residual member and leave request->data_len be in peace. Then change the few users that care about residual, and one caller that sets it. I'll prepare a patch.

Boaz

#47 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 7 lines · On Wed, 11 Feb 2009 18:55:00 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 6 lines

It's about a bidi request. We already have tons of loops, memory allocations, etc in the path. Do you think that adding one more loop leads to a notable performance regression?

Well, if you say that it's hacky then I would agree. But your patch using ~0 is hacky too.

Quoted 6 lines

It is an hack if used by an outside user, because it assumes knowledge of block-internals. It is much less of an hack if done by block-internals which knows for sure that this has no side effects.

But I agree that this is not clean. The clean solution is to add an extra parameter to blk_end_request() and change all callers.

I don't agree.

Quoted 3 lines

Or even cleaner is to add a new request->residual member and leave request->data_len be in peace. Then change the few users that care about residual, and one caller that sets it. I'll prepare a patch.

Yeah, it's clean but I'm not sure Jens would accept such patch since fattening request struct leads to a notable performance regression.

#48 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 19 lines · FUJITA Tomonori
Quoted 4 lines · On Wed, 11 Feb 2009 18:55:00 +0200 Boaz Harrosh <[email protected]> wrote:
Quoted 4 lines

It's about a bidi request. We already have tons of loops, memory allocations, etc in the path. Do you think that adding one more loop leads to a notable performance regression?

Quoted 8 lines
Quoted 2 lines

Well, if you say that it's hacky then I would agree. But your patch using ~0 is hacky too.

It is an hack if used by an outside user, because it assumes knowledge of block-internals. It is much less of an hack if done by block-internals which knows for sure that this has no side effects.

But I agree that this is not clean. The clean solution is to add an extra parameter to blk_end_request() and change all callers.

I don't agree.

Quoted 6 lines
Quoted 3 lines

Or even cleaner is to add a new request->residual member and leave request->data_len be in peace. Then change the few users that care about residual, and one caller that sets it. I'll prepare a patch.

Yeah, it's clean but I'm not sure Jens would accept such patch since fattening request struct leads to a notable performance regression.

OK, then give up, a proper comment next to blk_end_bidi_request() that says:
"... can be used with any requests, in case of uni-requests the second length will be ignored"

Actual FC code changes is:
- blk_end_request(req, err, req_len);
+ blk_end_bidi_request(req, err, req_len, rsp_len);

That is not so hard to understand. There are much more complicated things in kernel then that simple thing.

Boaz

#49 lore raw

RE: [RFC] FC pass thru - Rev V

In reply to James Smart
Quoted 7 lines

From: James Smart [mailto:[email protected]]
Sent: Wednesday, February 11, 2009 7:14 AM

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

<snip>

Quoted 5 lines
drivers/scsi/scsi_transport_fc.c +5 -0
+static int
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3507

Quoted 10 lines
drivers/scsi/scsi_transport_fc.c +10 -0
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);
+	if (!buf->sg_list)
+		return -ENOMEM;
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+	return 0;
+}

It looks like that there is possibility that the dma buffer with more than one sg element for ELS services - which is not allowed per FC spec. I've seen a couple of times from the driver testing.
I'm looking for some ways to making sure the ELS services
to get single sg for both request/reply_payload.
Any comments/suggestion?

Thank you,
Seokmann

#50 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 17 lines · Seokmann Ju
Quoted 7 lines

From: James Smart [mailto:[email protected]]
Sent: Wednesday, February 11, 2009 7:14 AM

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

<snip>

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+static int
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);
+	if (!buf->sg_list)
Quoted 8 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3507

Quoted 6 lines
drivers/scsi/scsi_transport_fc.c +6 -0
+		return -ENOMEM;
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+	return 0;
+}

It looks like that there is possibility that the dma buffer with more than one sg element for ELS services - which is not allowed per FC spec.

How can the T11 FC spec ever reference anything about sg elements ? They are completely independent.

Quoted 4 lines

I've seen a couple of times from the driver testing. I'm looking for some ways to making sure the ELS services to get single sg for both request/reply_payload. Any comments/suggestion?

For ELS's, the payload should be small - usually <= 256bytes so it fits within default login parameters. It should be 1 buffer, thus 1 sge. (Note: the routine above is for any request, which covers more than just ELS's).

If you're getting multiple sg's, then either the buffer spans some condition that crosses a boundary that blk_rq_map_sg() enforces (like a 4G boundary, but I doubt that's it), or - and more likely - your application has a buffer that spans 2 pages which look contiguous to the app, but are actually virtual-memory mapped to 2 independent physical pages (thus the need for 2 sg's). If your app knows the system page size, and was careful about buffer placement and length within page boundaries, it likely would never encounter this. But, we would never want to force apps to be this smart, and this would be a good justification for supporting more than 1 sg.

-- james s

#51 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 29 lines · James Smart
Quoted 17 lines · Seokmann Ju
Quoted 7 lines

From: James Smart [mailto:[email protected]]
Sent: Wednesday, February 11, 2009 7:14 AM

Trying to kick-start this again...
I've updated the prior RFC with the comments from Seokmann, SvenFujita, and Boaz. I would still like review on the blk_xxx completion calls in the std and error paths.

<snip>

Quoted 9 lines
drivers/scsi/scsi_transport_fc.c +9 -0
+static int
+fc_bsg_map_buffer(struct fc_bsg_buffer *buf, struct request *req)
+{
+	size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+	BUG_ON(!req->nr_phys_segments);
+
+	buf->sg_list = kzalloc(sz, GFP_KERNEL);
+	if (!buf->sg_list)
Quoted 8 lines

drivers/scsi/scsi_transport_fc.c fc_vport_sched_delete(struct work_struct · line 3507

Quoted 6 lines
drivers/scsi/scsi_transport_fc.c +6 -0
+		return -ENOMEM;
+	sg_init_table(buf->sg_list, req->nr_phys_segments);
+	buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+	buf->payload_len = req->data_len;
+	return 0;
+}

It looks like that there is possibility that the dma buffer with more than one sg element for ELS services - which is not allowed per FC spec.

Quoted 2 lines

How can the T11 FC spec ever reference anything about sg elements ? They are completely independent.

OK. Thanks for the clarification.

Quoted 11 lines
Quoted 6 lines

I've seen a couple of times from the driver testing. I'm looking for some ways to making sure the ELS services to get single sg for both request/reply_payload. Any comments/suggestion?

For ELS's, the payload should be small - usually <= 256bytes so it fits within default login parameters. It should be 1 buffer, thus 1 sge. (Note: the routine above is for any request, which covers more than just ELS's).

Yes, understood it.

Quoted 12 lines

If you're getting multiple sg's, then either the buffer spans some condition that crosses a boundary that blk_rq_map_sg() enforces (like a 4G boundary, but I doubt that's it), or - and more likely - your application has a buffer that spans 2 pages which look contiguous to the app, but are actually virtual-memory mapped to 2 independent physical pages (thus the need for 2 sg's). If your app knows the system page size, and was careful about buffer placement and length within page boundaries, it likely would never encounter this. But, we would never want to force apps to be this smart, and this would be a good justification for supporting more than 1 sg.

OK. So the whole point that I brought up is implementation specific. Thank you for the comment.

Seokmann

#52 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 10 lines
Quoted 10 lines

If you're getting multiple sg's, then either the buffer spans some condition that crosses a boundary that blk_rq_map_sg() enforces (like a 4G boundary, but I doubt that's it), or - and more likely - your application has a buffer that spans 2 pages which look contiguous to the app, but are actually virtual-memory mapped to 2 independent physical pages (thus the need for 2 sg's). If your app knows the system page size, and was careful about buffer placement and length within page boundaries, it likely would never encounter this. But, we would never want to force apps to be this smart, and this would be a good justification for supporting more than 1 sg.

Quoted 2 lines

OK. So the whole point that I brought up is implementation specific. Thank you for the comment.

Yes, but I wouldn't be surprised to see it occur often enough to be a pain. What I would recommend is : if your hardware wants only one buffer descriptor for ELSs, and there's 2 BGs, that you double-buffer it. Allocate a local buffer, transfer the users data into the buffer, and then use the local buffer for the hardware. Same on the receive path. A pain, yes, but this isn't time critical stuff, and it's much better than just returning an error to the app based on it's buffer alignment (which it probably doesn't even realize).

The above works for ELS's (as its small), but I fully expect other things, like CT requests, to not be small and require multiple buffers. On those things, I hope the hardware supports multiple sg's as double-buffering it is ugly.

-- james s

#53 lore raw

Re: [RFC] FC pass thru - Rev V

Quoted 15 lines · James Smart
Quoted 12 lines
Quoted 4 lines

If you're getting multiple sg's, then either the buffer spans some condition that crosses a boundary that blk_rq_map_sg() enforces (like a
4G boundary, but I doubt that's it), or - and more likely - your

Quoted 8 lines

application has a buffer that spans 2 pages which look contiguous to the
app, but are actually virtual-memory mapped to 2 independent physical pages (thus the need for 2 sg's). If your app knows the system page size, and was careful about buffer placement and length within page boundaries, it likely would never encounter this. But, we would never want to force apps to be this smart, and this would be a good justification for supporting more than 1 sg.

Quoted 2 lines

OK. So the whole point that I brought up is implementation specific. Thank you for the comment.

Quoted 9 lines

Yes, but I wouldn't be surprised to see it occur often enough to be a pain. What I would recommend is : if your hardware wants only one buffer descriptor for ELSs, and there's 2 BGs, that you double-buffer it. Allocate a local buffer, transfer the users data into the buffer, and then use the local buffer for the hardware. Same on the receive path. A pain, yes, but this isn't time critical stuff, and it's much better than just returning an error to the app based on it's buffer alignment (which it probably doesn't even realize).

Or force a block bounce buffers by setting the queue alignment requirements (Can only work for max transfer of PAGE_SIZE/2 )

Quoted 7 lines

The above works for ELS's (as its small), but I fully expect other things, like CT requests, to not be small and require multiple buffers. On those things, I hope the hardware supports multiple sg's as double-buffering it is ugly.

-- james s

Boaz