Skip to contentSkip to conversation

FCoE HBA Driver (fnic) for 2.6.30 feature window

PATCH In review Abhijeet Joglekar 2 participants 13 messages Feb 28, 2009 – Mar 17, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover FCoE HBA Driver (fnic) for 2.6.30 feature window

The following series implements a driver for a Fibre Channel over Ethernet HBA. The driver requires libFC module.

This patch series applies on top of:
(1) Linus' kernel tree (2.6.29-rc6) or scsi-misc or scsi-rc-fixes WITH
(2) LibFC patches re-submitted today by Robert Love
(http://marc.info/?l=linux-scsi&m=123576089125544&w=2, libfc and fcoe fixes for 2.6.29-rc)

The driver interfaces with scsi and scsi-transport-fc for SCSI FCP processing, and with libFC for Fibre Channel Services (Fabric Login, Discovery, Remote Port Logins)

Patch 1) This adds main module infrastructure files and interfaces to scsi, scsi-transport-fc, and libFC

Patch 2) This adds SCSI FCP processing

Patch 3) This adds interface to libFC to handle Fibre Channel Services

Patch 4) Resource allocations for the driver

Patch 5) Interfaces to hardware, fw, queues, descriptors, buffers

Patch 6) This adds to scsi/Kconfig, scsi/Makefile, and MAINTAINERS

We would like to be considered for inclusion in the 2.6.30 feature window, and would appreciate review comments.

Thanks,
Abhijeet Joglekar
Joe Eykholt

---

Abhijeet Joglekar (6):
fnic: Patch MAINTAINERS, scsi Makefile, scsi Kconfig
fnic: add descriptor right, buffers, device interface
fnic: adds resource allocation, interupt interfaces
fnic: Add support for Fibre Channel Services through libFC
fnic: add SCSI FCP handling
fnic: add main file with module infrastructure, fnic structure, Makefile

 MAINTAINERS                       |    8 
 drivers/scsi/Kconfig              |   11 
 drivers/scsi/Makefile             |    1 
 drivers/scsi/fnic/Makefile        |   15 
 drivers/scsi/fnic/cq_desc.h       |   78 ++
 drivers/scsi/fnic/cq_enet_desc.h  |  167 +++
 drivers/scsi/fnic/cq_exch_desc.h  |  182 ++++
 drivers/scsi/fnic/fcpio.h         |  780 +++++++++++++++
 drivers/scsi/fnic/fnic.h          |  283 ++++++
 drivers/scsi/fnic/fnic_attrs.c    |   71 +
 drivers/scsi/fnic/fnic_fcs.c      |  719 ++++++++++++++
 drivers/scsi/fnic/fnic_io.h       |   67 +
 drivers/scsi/fnic/fnic_isr.c      |  334 +++++++
 drivers/scsi/fnic/fnic_main.c     |  998 ++++++++++++++++++++
 drivers/scsi/fnic/fnic_res.c      |  431 +++++++++
 drivers/scsi/fnic/fnic_res.h      |  197 ++++
 drivers/scsi/fnic/fnic_scsi.c     | 1869 +++++++++++++++++++++++++++++++++++++
 drivers/scsi/fnic/rq_enet_desc.h  |   58 +
 drivers/scsi/fnic/vnic_cq.c       |   86 ++
 drivers/scsi/fnic/vnic_cq.h       |  121 ++
 drivers/scsi/fnic/vnic_cq_copy.h  |   62 +
 drivers/scsi/fnic/vnic_dev.c      |  690 ++++++++++++++
 drivers/scsi/fnic/vnic_dev.h      |  161 +++
 drivers/scsi/fnic/vnic_devcmd.h   |  281 ++++++
 drivers/scsi/fnic/vnic_intr.c     |   60 +
 drivers/scsi/fnic/vnic_intr.h     |  118 ++
 drivers/scsi/fnic/vnic_nic.h      |   69 +
 drivers/scsi/fnic/vnic_resource.h |   61 +
 drivers/scsi/fnic/vnic_rq.c       |  197 ++++
 drivers/scsi/fnic/vnic_rq.h       |  235 +++++
 drivers/scsi/fnic/vnic_scsi.h     |   99 ++
 drivers/scsi/fnic/vnic_stats.h    |   68 +
 drivers/scsi/fnic/vnic_wq.c       |  183 ++++
 drivers/scsi/fnic/vnic_wq.h       |  175 +++
 drivers/scsi/fnic/vnic_wq_copy.c  |  118 ++
 drivers/scsi/fnic/vnic_wq_copy.h  |  129 +++
 drivers/scsi/fnic/wq_enet_desc.h  |   96 ++
 37 files changed, 9278 insertions(+), 0 deletions(-)
 create mode 100644 drivers/scsi/fnic/Makefile
 create mode 100644 drivers/scsi/fnic/cq_desc.h
 create mode 100644 drivers/scsi/fnic/cq_enet_desc.h
 create mode 100644 drivers/scsi/fnic/cq_exch_desc.h
 create mode 100644 drivers/scsi/fnic/fcpio.h
 create mode 100644 drivers/scsi/fnic/fnic.h
 create mode 100644 drivers/scsi/fnic/fnic_attrs.c
 create mode 100644 drivers/scsi/fnic/fnic_fcs.c
 create mode 100644 drivers/scsi/fnic/fnic_io.h
 create mode 100644 drivers/scsi/fnic/fnic_isr.c
 create mode 100644 drivers/scsi/fnic/fnic_main.c
 create mode 100644 drivers/scsi/fnic/fnic_res.c
 create mode 100644 drivers/scsi/fnic/fnic_res.h
 create mode 100644 drivers/scsi/fnic/fnic_scsi.c
 create mode 100644 drivers/scsi/fnic/rq_enet_desc.h
 create mode 100644 drivers/scsi/fnic/vnic_cq.c
 create mode 100644 drivers/scsi/fnic/vnic_cq.h
 create mode 100644 drivers/scsi/fnic/vnic_cq_copy.h
 create mode 100644 drivers/scsi/fnic/vnic_dev.c
 create mode 100644 drivers/scsi/fnic/vnic_dev.h
 create mode 100644 drivers/scsi/fnic/vnic_devcmd.h
 create mode 100644 drivers/scsi/fnic/vnic_intr.c
 create mode 100644 drivers/scsi/fnic/vnic_intr.h
 create mode 100644 drivers/scsi/fnic/vnic_nic.h
 create mode 100644 drivers/scsi/fnic/vnic_resource.h
 create mode 100644 drivers/scsi/fnic/vnic_rq.c
 create mode 100644 drivers/scsi/fnic/vnic_rq.h
 create mode 100644 drivers/scsi/fnic/vnic_scsi.h
 create mode 100644 drivers/scsi/fnic/vnic_stats.h
 create mode 100644 drivers/scsi/fnic/vnic_wq.c
 create mode 100644 drivers/scsi/fnic/vnic_wq.h
 create mode 100644 drivers/scsi/fnic/vnic_wq_copy.c
 create mode 100644 drivers/scsi/fnic/vnic_wq_copy.h
 create mode 100644 drivers/scsi/fnic/wq_enet_desc.h
Signature · 1 line
Signature
#1 lore raw

RFC PATCH 1/6 fnic: add main file with module infrastructure, fnic structure, Makefile

fnic: add main file with module infrastructure, fnic structure, Makefile

fnic_main.c: includes module load and unload, PCI device probe, scsi-ml, libFC, and scsi-transport-fc registration and interfaces

fnic.h: has the fnic definition and other related data types

Makefile:

---
 drivers/scsi/fnic/Makefile    |   15 +
 drivers/scsi/fnic/fnic.h      |  283 ++++++++++++
 drivers/scsi/fnic/fnic_main.c |  998 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 1296 insertions(+), 0 deletions(-)
 create mode 100644 drivers/scsi/fnic/Makefile
 create mode 100644 drivers/scsi/fnic/fnic.h
 create mode 100644 drivers/scsi/fnic/fnic_main.c
drivers/scsi/fnic/Makefile new file +15 -0
new file mode 100644
index 0000000..37c3440
@@ -0,0 +1,15 @@
+obj-$(CONFIG_FCOE_FNIC) += fnic.o
+
+fnic-y	:= \
+	fnic_attrs.o \
+	fnic_isr.o \
+	fnic_main.o \
+	fnic_res.o \
+	fnic_fcs.o \
+	fnic_scsi.o \
+	vnic_cq.o \
+	vnic_dev.o \
+	vnic_intr.o \
+	vnic_rq.o \
+	vnic_wq_copy.o \
+	vnic_wq.o
drivers/scsi/fnic/fnic.h new file +283 -0

283 lines not on this page · raw message

drivers/scsi/fnic/fnic_main.c new file +998 -0

998 lines not on this page · raw message

#2 lore raw

Re: [RFC][PATCH 1/6] fnic: add main file with module infrastructure, fnic structure, Makefile

Quoted 19 lines · Abhijeet Joglekar
drivers/scsi/fnic/fnic_main.c new file +19 -0
+
+void fnic_handle_link_event(struct fnic *fnic)
+{
+	int link_status = vnic_dev_link_status(fnic->vdev);
+	struct fnic_event *event;
+	unsigned long flags;
+
+	spin_lock_irqsave(&fnic->fnic_lock, flags);
+	if (fnic->stop_rx_link_events) {
+		spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+		return;
+	}
+
+	FNIC_MAIN_DBG(KERN_DEBUG, DFX "link %s\n", fnic->fnic_no,
+		      (link_status ? "up" : "down"));
+
+	event = kmem_cache_alloc(fnic_ev_cache, GFP_ATOMIC);
+	if (!event) {
+		FNIC_MAIN_DBG(KERN_DEBUG, DFX "Cannot allocate a event, "

drivers/scsi/fnic/fnic_main.c line 265

Quoted 10 lines
drivers/scsi/fnic/fnic_main.c new file +10 -0
+			      "cannot indicate link event to FCS\n",
+			      fnic->fnic_no);
+		spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+		return;
+	}
+
+	/* Queue the link event in fnic workQ */
+	memset(event, 0, sizeof(struct fnic_event));
+	event->fnic = fnic;
+	event->ev_type = EV_TYPE_LINK_UP;

How may events types are there going to be?

If the link is down and you need this to get the link up again then you do not want to rely on just a GFP_ATOMIC allocation.

If there is just a couple events then maybe allocating a work_struct for each one in the fnic struct would be ok. You could then just queue that up here and add some code to handle multiple ups/downs.

Or at least I think you want a mempool of events. That does not fully over you but at least you know you can handle some events and then code it so you can recover.

Quoted 45 lines
drivers/scsi/fnic/fnic_main.c new file +45 -0
+	if (!link_status) {
+		event->ev_type = EV_TYPE_LINK_DOWN;
+		fnic->lport->host_stats.link_failure_count++;
+	}
+	fnic->event_count++;
+	spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+
+	INIT_WORK(&event->event_work, fnic_event_work);
+	queue_work(fnic_event_queue, &event->event_work);
+
+}
+
+static int fnic_notify_set(struct fnic *fnic)
+{
+	int err;
+
+	switch (vnic_dev_get_intr_mode(fnic->vdev)) {
+	case VNIC_DEV_INTR_MODE_INTX:
+		err = vnic_dev_notify_set(fnic->vdev, FNIC_INTX_NOTIFY);
+		break;
+	case VNIC_DEV_INTR_MODE_MSI:
+		err = vnic_dev_notify_set(fnic->vdev, -1);
+		break;
+	case VNIC_DEV_INTR_MODE_MSIX:
+		err = vnic_dev_notify_set(fnic->vdev, FNIC_MSIX_ERR_NOTIFY);
+		break;
+	default:
+		printk(KERN_ERR DFX "Interrupt mode should be set up"
+		       " before devcmd notify set %d\n", fnic->fnic_no,
+		       vnic_dev_get_intr_mode(fnic->vdev));
+		err = -1;
+		break;
+	}
+
+	return err;
+}
+
+static void fnic_notify_timer(unsigned long data)
+{
+	struct fnic *fnic = (struct fnic *)data;
+
+	fnic_handle_link_event(fnic);
+	mod_timer(&fnic->notify_timer,
+		  round_jiffies(jiffies + FNIC_NOTIFY_TIMER_PERIOD));
+}

drivers/scsi/fnic/fnic_main.c line 320

Quoted 10 lines
drivers/scsi/fnic/fnic_main.c new file +10 -0
+
+static void fnic_notify_timer_start(struct fnic *fnic)
+{
+	switch (vnic_dev_get_intr_mode(fnic->vdev)) {
+	case VNIC_DEV_INTR_MODE_MSI:
+		/*
+		 * Schedule first timeout immediately. The driver is
+		 * initiatialized and ready to look for link up notification
+		 */
+		mod_timer(&fnic->notify_timer, jiffies);

Do you just want to call fnic_notify_timer() here instead of going through the timer?

Quoted 21 lines
drivers/scsi/fnic/fnic_main.c new file +21 -0
+}
+
+static int __devinit fnic_probe(struct pci_dev *pdev,
+				const struct pci_device_id *ent)
+{
+	struct Scsi_Host *host;
+	struct fc_lport *lp;
+	struct fnic *fnic;
+	mempool_t *pool;
+	int err;
+	int i;
+	unsigned long flags;
+
+	/*
+	 * Allocate SCSI Host and set up association between host,
+	 * local port, and fnic
+	 */
+	host = scsi_host_alloc(&fnic_host_template,
+			       sizeof(struct fc_lport) + sizeof(struct fnic));
+	if (!host) {
+		printk(KERN_ERR PFX "Unable to alloc SCSI host\n");

drivers/scsi/fnic/fnic_main.c line 474

Quoted 10 lines
drivers/scsi/fnic/fnic_main.c new file +10 -0
+		err = -ENOMEM;
+		goto err_out;
+	}
+	lp = shost_priv(host);
+	lp->host = host;
+	fnic = lport_priv(lp);
+	fnic->lport = lp;
+
+	/* fnic number starts from 0 onwards */
+	fnic->fnic_no = atomic_add_return(1, &fnic_no);

This is basically the shost->host_no. I think it is best if you just use that so when you print it out users do not have look through the logs to match up the fnic host no with the scsi host no.

I think something built on shost_printk or dev_printk is nice to use for driver logging too. I think this is optional and if needed can probably be fixed after this is merged though.

#3 lore raw

RFC PATCH 2/6 fnic: add SCSI FCP handling

In reply to Abhijeet Joglekar

fnic: add SCSI FCP handling

fnic_scsi.c: contains SCSI FCP handling, fw reset, flogi addr registration

fnic_io.h: defines per IO request state structure

fcpio.h: defines data structures used for driver-fw communication

---
 drivers/scsi/fnic/fcpio.h     |  780 +++++++++++++++++
 drivers/scsi/fnic/fnic_io.h   |   67 +
 drivers/scsi/fnic/fnic_scsi.c | 1869 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 2716 insertions(+), 0 deletions(-)
 create mode 100644 drivers/scsi/fnic/fcpio.h
 create mode 100644 drivers/scsi/fnic/fnic_io.h
 create mode 100644 drivers/scsi/fnic/fnic_scsi.c
drivers/scsi/fnic/fcpio.h new file +780 -0
new file mode 100644
index 0000000..12d770d
@@ -0,0 +1,780 @@
+/*
+ * Copyright 2008 Cisco Systems, Inc.  All rights reserved.
+ * Copyright 2007 Nuova Systems, Inc.  All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#ifndef _FCPIO_H_
+#define _FCPIO_H_
+
+#include <linux/if_ether.h>
+
+/*
+ * This header file includes all of the data structures used for
+ * communication by the host driver to the fcp firmware.
+ */
+
+/*
+ * Exchange and sequence id space allocated to the host driver
+ */
+#define FCPIO_HOST_EXCH_RANGE_START         0x1000
+#define FCPIO_HOST_EXCH_RANGE_END           0x1fff
+#define FCPIO_HOST_SEQ_ID_RANGE_START       0x80
+#define FCPIO_HOST_SEQ_ID_RANGE_END         0xff
+
+/*
+ * Command entry type
+ */
+enum fcpio_type {
+	/*
+	 * Initiator request types
+	 */
+	FCPIO_ICMND_16 = 0x1,
+	FCPIO_ICMND_32,
+	FCPIO_ICMND_CMPL,
+	FCPIO_ITMF,
+	FCPIO_ITMF_CMPL,
+
+	/*
+	 * Target request types
+	 */
+	FCPIO_TCMND_16 = 0x11,
+	FCPIO_TCMND_32,
+	FCPIO_TDATA,
+	FCPIO_TXRDY,
+	FCPIO_TRSP,
+	FCPIO_TDRSP_CMPL,
+	FCPIO_TTMF,
+	FCPIO_TTMF_ACK,
+	FCPIO_TABORT,
+	FCPIO_TABORT_CMPL,
+
+	/*
+	 * Misc request types
+	 */
+	FCPIO_ACK = 0x20,
+	FCPIO_RESET,
+	FCPIO_RESET_CMPL,
+	FCPIO_FLOGI_REG,
+	FCPIO_FLOGI_REG_CMPL,
+	FCPIO_ECHO,
+	FCPIO_ECHO_CMPL,
+	FCPIO_LUNMAP_CHNG,
+	FCPIO_LUNMAP_REQ,
+	FCPIO_LUNMAP_REQ_CMPL,
+	FCPIO_FLOGI_FIP_REG,
+	FCPIO_FLOGI_FIP_REG_CMPL,
+};
+
+/*
+ * Header status codes from the firmware
+ */
+enum fcpio_status {
+	FCPIO_SUCCESS = 0,              /* request was successful */
+
+	/*
+	 * If a request to the firmware is rejected, the original request
+	 * header will be returned with the status set to one of the following:
+	 */
+	FCPIO_INVALID_HEADER,    /* header contains invalid data */
+	FCPIO_OUT_OF_RESOURCE,   /* out of resources to complete request */
+	FCPIO_INVALID_PARAM,     /* some parameter in request is invalid */
+	FCPIO_REQ_NOT_SUPPORTED, /* request type is not supported */
+	FCPIO_IO_NOT_FOUND,      /* requested I/O was not found */
+
+	/*
+	 * Once a request is processed, the firmware will usually return
+	 * a cmpl message type.  In cases where errors occurred,
+	 * the header status field would be filled in with one of the following:
+	 */
+	FCPIO_ABORTED = 0x41,     /* request was aborted */
+	FCPIO_TIMEOUT,            /* request was timed out */
+	FCPIO_SGL_INVALID,        /* request was aborted due to sgl error */
+	FCPIO_MSS_INVALID,        /* request was aborted due to mss error */
+	FCPIO_DATA_CNT_MISMATCH,  /* recv/sent more/less data than exp. */
+	FCPIO_FW_ERR,             /* request was terminated due to fw error */
+	FCPIO_ITMF_REJECTED,      /* itmf req was rejected by remote node */
+	FCPIO_ITMF_FAILED,        /* itmf req was failed by remote node */
+	FCPIO_ITMF_INCORRECT_LUN, /* itmf req targeted incorrect LUN */
+	FCPIO_CMND_REJECTED,      /* request was invalid and rejected */
+	FCPIO_NO_PATH_AVAIL,      /* no paths to the lun was available */
+	FCPIO_PATH_FAILED,        /* i/o sent to current path failed */
+	FCPIO_LUNMAP_CHNG_PEND,   /* i/o rejected due to lunmap change */
+};
+
+/*
+ * The header command tag.  All host requests will use the "tag" field
+ * to mark commands with a unique tag.  When the firmware responds to
+ * a host request, it will copy the tag field into the response.
+ *
+ * The only firmware requests that will use the rx_id/ox_id fields instead
+ * of the tag field will be the target command and target task management
+ * requests.  These two requests do not have corresponding host requests
+ * since they come directly from the FC initiator on the network.
+ */
+struct fcpio_tag {
+	union {
+		u32 req_id;
+		struct {
+			u16 rx_id;
+			u16 ox_id;
+		} ex_id;
+	} u;
+};
+
+static inline void
+fcpio_tag_id_enc(struct fcpio_tag *tag, u32 id)
+{
+	tag->u.req_id = id;
+}
+
+static inline void
+fcpio_tag_id_dec(struct fcpio_tag *tag, u32 *id)
+{
+	*id = tag->u.req_id;
+}
+
+static inline void
+fcpio_tag_exid_enc(struct fcpio_tag *tag, u16 ox_id, u16 rx_id)
+{
+	tag->u.ex_id.rx_id = rx_id;
+	tag->u.ex_id.ox_id = ox_id;
+}
+
+static inline void
+fcpio_tag_exid_dec(struct fcpio_tag *tag, u16 *ox_id, u16 *rx_id)
+{
+	*rx_id = tag->u.ex_id.rx_id;
+	*ox_id = tag->u.ex_id.ox_id;
+}
+
+/*
+ * The header for an fcpio request, whether from the firmware or from the
+ * host driver
+ */
+struct fcpio_header {
+	u8            type;           /* enum fcpio_type */
+	u8            status;         /* header status entry */
+	u16           _resvd;         /* reserved */
+	struct fcpio_tag    tag;      /* header tag */
+};
+
+static inline void
+fcpio_header_enc(struct fcpio_header *hdr,
+		 u8 type, u8 status,
+		 struct fcpio_tag tag)
+{
+	hdr->type = type;
+	hdr->status = status;
+	hdr->_resvd = 0;
+	hdr->tag = tag;
+}
+
+static inline void
+fcpio_header_dec(struct fcpio_header *hdr,
+		 u8 *type, u8 *status,
+		 struct fcpio_tag *tag)
+{
+	*type = hdr->type;
+	*status = hdr->status;
+	*tag = hdr->tag;
+}
+
+#define CDB_16      16
+#define CDB_32      32
+#define LUN_ADDRESS 8
+
+/*
+ * fcpio_icmnd_16: host -> firmware request
+ *
+ * used for sending out an initiator SCSI 16-byte command
+ */
+struct fcpio_icmnd_16 {
+	u32	  lunmap_id;		/* index into lunmap table */
+	u8	  special_req_flags;	/* special exchange request flags */
+	u8	  _resvd0[3];	        /* reserved */
+	u32	  sgl_cnt;		/* scatter-gather list count */
+	u32	  sense_len;		/* sense buffer length */
+	u64	  sgl_addr;		/* scatter-gather list addr */
+	u64	  sense_addr;		/* sense buffer address */
+	u8	  crn;			/* SCSI Command Reference No. */
+	u8	  pri_ta;		/* SCSI Priority and Task attribute */
+	u8	  _resvd1;		/* reserved: should be 0 */
+	u8	  flags;		/* command flags */
+	u8	  scsi_cdb[CDB_16];	/* SCSI Cmnd Descriptor Block */
+	u32	  data_len;		/* length of data expected */
+	u8	  lun[LUN_ADDRESS];	/* FC vNIC only: LUN address */
+	u8	  _resvd2;		/* reserved */
+	u8	  d_id[3];		/* FC vNIC only: Target D_ID */
+	u16	  mss;			/* FC vNIC only: max burst */
+	u16	  _resvd3;		/* reserved */
+	u32	  r_a_tov;		/* FC vNIC only: Res. Alloc Timeout */
+	u32	  e_d_tov;	        /* FC vNIC only: Err Detect Timeout */
+};
+
+/*
+ * Special request flags
+ */
+#define FCPIO_ICMND_SRFLAG_RETRY 0x01   /* Enable Retry handling on exchange */
+
+/*
+ * Priority/Task Attribute settings
+ */
+#define FCPIO_ICMND_PTA_SIMPLE      0   /* simple task attribute */
+#define FCPIO_ICMND_PTA_HEADQ       1   /* head of queue task attribute */
+#define FCPIO_ICMND_PTA_ORDERED     2   /* ordered task attribute */
+#define FCPIO_ICMND_PTA_ACA         4   /* auto contingent allegiance */
+#define FCPIO_ICMND_PRI_SHIFT       3   /* priority field starts in bit 3 */
+
+/*
+ * Command flags
+ */
+#define FCPIO_ICMND_RDDATA      0x02    /* read data */
+#define FCPIO_ICMND_WRDATA      0x01    /* write data */
+
+/*
+ * fcpio_icmnd_32: host -> firmware request
+ *
+ * used for sending out an initiator SCSI 32-byte command
+ */
+struct fcpio_icmnd_32 {
+	u32   lunmap_id;              /* index into lunmap table */
+	u8    special_req_flags;      /* special exchange request flags */
+	u8    _resvd0[3];             /* reserved */
+	u32   sgl_cnt;                /* scatter-gather list count */
+	u32   sense_len;              /* sense buffer length */
+	u64   sgl_addr;               /* scatter-gather list addr */
+	u64   sense_addr;             /* sense buffer address */
+	u8    crn;                    /* SCSI Command Reference No. */
+	u8    pri_ta;                 /* SCSI Priority and Task attribute */
+	u8    _resvd1;                /* reserved: should be 0 */
+	u8    flags;                  /* command flags */
+	u8    scsi_cdb[CDB_32];       /* SCSI Cmnd Descriptor Block */
+	u32   data_len;               /* length of data expected */
+	u8    lun[LUN_ADDRESS];       /* FC vNIC only: LUN address */
+	u8    _resvd2;                /* reserved */
+	u8    d_id[3];		      /* FC vNIC only: Target D_ID */
+	u16   mss;                    /* FC vNIC only: max burst */
+	u16   _resvd3;                /* reserved */
+	u32   r_a_tov;                /* FC vNIC only: Res. Alloc Timeout */
+	u32   e_d_tov;                /* FC vNIC only: Error Detect Timeout */
+};
+
+/*
+ * fcpio_itmf: host -> firmware request
+ *
+ * used for requesting the firmware to abort a request and/or send out
+ * a task management function
+ *
+ * The t_tag field is only needed when the request type is ABT_TASK.
+ */
+struct fcpio_itmf {
+	u32   lunmap_id;              /* index into lunmap table */
+	u32   tm_req;                 /* SCSI Task Management request */
+	u32   t_tag;                  /* header tag of fcpio to be aborted */
+	u32   _resvd;                 /* _reserved */
+	u8    lun[LUN_ADDRESS];       /* FC vNIC only: LUN address */
+	u8    _resvd1;                /* reserved */
+	u8    d_id[3];		      /* FC vNIC only: Target D_ID */
+	u32   r_a_tov;                /* FC vNIC only: R_A_TOV in msec */
+	u32   e_d_tov;                /* FC vNIC only: E_D_TOV in msec */
+};
+
+/*
+ * Task Management request
+ */
+enum fcpio_itmf_tm_req_type {
+	FCPIO_ITMF_ABT_TASK_TERM = 0x01,    /* abort task and terminate */
+	FCPIO_ITMF_ABT_TASK,                /* abort task and issue abts */
+	FCPIO_ITMF_ABT_TASK_SET,            /* abort task set */
+	FCPIO_ITMF_CLR_TASK_SET,            /* clear task set */
+	FCPIO_ITMF_LUN_RESET,               /* logical unit reset task mgmt */
+	FCPIO_ITMF_CLR_ACA,                 /* Clear ACA condition */
+};
+
+/*
+ * fcpio_tdata: host -> firmware request
+ *
+ * used for requesting the firmware to send out a read data transfer for a
+ * target command
+ */
+struct fcpio_tdata {
+	u16   rx_id;                  /* FC rx_id of target command */
+	u16   flags;                  /* command flags */
+	u32   rel_offset;             /* data sequence relative offset */
+	u32   sgl_cnt;                /* scatter-gather list count */
+	u32   data_len;               /* length of data expected to send */
+	u64   sgl_addr;               /* scatter-gather list address */
+};
+
+/*
+ * Command flags
+ */
+#define FCPIO_TDATA_SCSI_RSP    0x01    /* send a scsi resp. after last frame */
+
+/*
+ * fcpio_txrdy: host -> firmware request
+ *
+ * used for requesting the firmware to send out a write data transfer for a
+ * target command
+ */
+struct fcpio_txrdy {
+	u16   rx_id;                  /* FC rx_id of target command */
+	u16   _resvd0;                /* reserved */
+	u32   rel_offset;             /* data sequence relative offset */
+	u32   sgl_cnt;                /* scatter-gather list count */
+	u32   data_len;               /* length of data expected to send */
+	u64   sgl_addr;               /* scatter-gather list address */
+};
+
+/*
+ * fcpio_trsp: host -> firmware request
+ *
+ * used for requesting the firmware to send out a response for a target
+ * command
+ */
+struct fcpio_trsp {
+	u16   rx_id;                  /* FC rx_id of target command */
+	u16   _resvd0;                /* reserved */
+	u32   sense_len;              /* sense data buffer length */
+	u64   sense_addr;             /* sense data buffer address */
+	u16   _resvd1;                /* reserved */
+	u8    flags;                  /* response request flags */
+	u8    scsi_status;            /* SCSI status */
+	u32   residual;               /* SCSI data residual value of I/O */
+};
+
+/*
+ * resposnse request flags
+ */
+#define FCPIO_TRSP_RESID_UNDER  0x08   /* residual is valid and is underflow */
+#define FCPIO_TRSP_RESID_OVER   0x04   /* residual is valid and is overflow */
+
+/*
+ * fcpio_ttmf_ack: host -> firmware response
+ *
+ * used by the host to indicate to the firmware it has received and processed
+ * the target tmf request
+ */
+struct fcpio_ttmf_ack {
+	u16   rx_id;                  /* FC rx_id of target command */
+	u16   _resvd0;                /* reserved */
+	u32   tmf_status;             /* SCSI task management status */
+};
+
+/*
+ * fcpio_tabort: host -> firmware request
+ *
+ * used by the host to request the firmware to abort a target request that was
+ * received by the firmware
+ */
+struct fcpio_tabort {
+	u16   rx_id;                  /* rx_id of the target request */
+};
+
+/*
+ * fcpio_reset: host -> firmware request
+ *
+ * used by the host to signal a reset of the driver to the firmware
+ * and to request firmware to clean up all outstanding I/O
+ */
+struct fcpio_reset {
+	u32   _resvd;
+};
+
+enum fcpio_flogi_reg_format_type {
+	FCPIO_FLOGI_REG_DEF_DEST = 0,    /* Use the oui | s_id mac format */
+	FCPIO_FLOGI_REG_GW_DEST,         /* Use the fixed gateway mac */
+};
+
+/*
+ * fcpio_flogi_reg: host -> firmware request
+ *
+ * fc vnic only
+ * used by the host to notify the firmware of the lif's s_id
+ * and destination mac address format
+ */
+struct fcpio_flogi_reg {
+	u8 format;
+	u8 s_id[3];			/* FC vNIC only: Source S_ID */
+	u8 gateway_mac[ETH_ALEN];	/* Destination gateway mac */
+	u16 _resvd;
+	u32 r_a_tov;			/* R_A_TOV in msec */
+	u32 e_d_tov;			/* E_D_TOV in msec */
+};
+
+/*
+ * fcpio_echo: host -> firmware request
+ *
+ * sends a heartbeat echo request to the firmware
+ */
+struct fcpio_echo {
+	u32 _resvd;
+};
+
+/*
+ * fcpio_lunmap_req: host -> firmware request
+ *
+ * scsi vnic only
+ * sends a request to retrieve the lunmap table for scsi vnics
+ */
+struct fcpio_lunmap_req {
+	u64 addr;                     /* address of the buffer */
+	u32 len;                      /* len of the buffer */
+};
+
+/*
+ * fcpio_flogi_fip_reg: host -> firmware request
+ *
+ * fc vnic only
+ * used by the host to notify the firmware of the lif's s_id
+ * and destination mac address format
+ */
+struct fcpio_flogi_fip_reg {
+	u8    _resvd0;
+	u8     s_id[3];               /* FC vNIC only: Source S_ID */
+	u8     fcf_mac[ETH_ALEN];     /* FCF Target destination mac */
+	u16   _resvd1;
+	u32   r_a_tov;                /* R_A_TOV in msec */
+	u32   e_d_tov;                /* E_D_TOV in msec */
+	u8    ha_mac[ETH_ALEN];       /* Host adapter source mac */
+	u16   _resvd2;
+};
+
+/*
+ * Basic structure for all fcpio structures that are sent from the host to the
+ * firmware.  They are 128 bytes per structure.
+ */
+#define FCPIO_HOST_REQ_LEN      128     /* expected length of host requests */
+
+struct fcpio_host_req {
+	struct fcpio_header hdr;
+
+	union {
+		/*
+		 * Defines space needed for request
+		 */
+		u8 buf[FCPIO_HOST_REQ_LEN - sizeof(struct fcpio_header)];
+
+		/*
+		 * Initiator host requests
+		 */
+		struct fcpio_icmnd_16               icmnd_16;
+		struct fcpio_icmnd_32               icmnd_32;
+		struct fcpio_itmf                   itmf;
+
+		/*
+		 * Target host requests
+		 */
+		struct fcpio_tdata                  tdata;
+		struct fcpio_txrdy                  txrdy;
+		struct fcpio_trsp                   trsp;
+		struct fcpio_ttmf_ack               ttmf_ack;
+		struct fcpio_tabort                 tabort;
+
+		/*
+		 * Misc requests
+		 */
+		struct fcpio_reset                  reset;
+		struct fcpio_flogi_reg              flogi_reg;
+		struct fcpio_echo                   echo;
+		struct fcpio_lunmap_req             lunmap_req;
+		struct fcpio_flogi_fip_reg          flogi_fip_reg;
+	} u;
+};
+
+/*
+ * fcpio_icmnd_cmpl: firmware -> host response
+ *
+ * used for sending the host a response to an initiator command
+ */
+struct fcpio_icmnd_cmpl {
+	u8    _resvd0[6];             /* reserved */
+	u8    flags;                  /* response flags */
+	u8    scsi_status;            /* SCSI status */
+	u32   residual;               /* SCSI data residual length */
+	u32   sense_len;              /* SCSI sense length */
+};
+
+/*
+ * response flags
+ */
+#define FCPIO_ICMND_CMPL_RESID_UNDER    0x08    /* resid under and valid */
+#define FCPIO_ICMND_CMPL_RESID_OVER     0x04    /* resid over and valid */
+
+/*
+ * fcpio_itmf_cmpl: firmware -> host response
+ *
+ * used for sending the host a response for a itmf request
+ */
+struct fcpio_itmf_cmpl {
+	u32    _resvd;                /* reserved */
+};
+
+/*
+ * fcpio_tcmnd_16: firmware -> host request
+ *
+ * used by the firmware to notify the host of an incoming target SCSI 16-Byte
+ * request
+ */
+struct fcpio_tcmnd_16 {
+	u8    lun[LUN_ADDRESS];       /* FC vNIC only: LUN address */
+	u8    crn;                    /* SCSI Command Reference No. */
+	u8    pri_ta;                 /* SCSI Priority and Task attribute */
+	u8    _resvd2;                /* reserved: should be 0 */
+	u8    flags;                  /* command flags */
+	u8    scsi_cdb[CDB_16];       /* SCSI Cmnd Descriptor Block */
+	u32   data_len;               /* length of data expected */
+	u8    _resvd1;                /* reserved */
+	u8    s_id[3];		      /* FC vNIC only: Source S_ID */
+};
+
+/*
+ * Priority/Task Attribute settings
+ */
+#define FCPIO_TCMND_PTA_SIMPLE      0   /* simple task attribute */
+#define FCPIO_TCMND_PTA_HEADQ       1   /* head of queue task attribute */
+#define FCPIO_TCMND_PTA_ORDERED     2   /* ordered task attribute */
+#define FCPIO_TCMND_PTA_ACA         4   /* auto contingent allegiance */
+#define FCPIO_TCMND_PRI_SHIFT       3   /* priority field starts in bit 3 */
+
+/*
+ * Command flags
+ */
+#define FCPIO_TCMND_RDDATA      0x02    /* read data */
+#define FCPIO_TCMND_WRDATA      0x01    /* write data */
+
+/*
+ * fcpio_tcmnd_32: firmware -> host request
+ *
+ * used by the firmware to notify the host of an incoming target SCSI 32-Byte
+ * request
+ */
+struct fcpio_tcmnd_32 {
+	u8    lun[LUN_ADDRESS];       /* FC vNIC only: LUN address */
+	u8    crn;                    /* SCSI Command Reference No. */
+	u8    pri_ta;                 /* SCSI Priority and Task attribute */
+	u8    _resvd2;                /* reserved: should be 0 */
+	u8    flags;                  /* command flags */
+	u8    scsi_cdb[CDB_32];       /* SCSI Cmnd Descriptor Block */
+	u32   data_len;               /* length of data expected */
+	u8    _resvd0;                /* reserved */
+	u8    s_id[3];		      /* FC vNIC only: Source S_ID */
+};
+
+/*
+ * fcpio_tdrsp_cmpl: firmware -> host response
+ *
+ * used by the firmware to notify the host of a response to a host target
+ * command
+ */
+struct fcpio_tdrsp_cmpl {
+	u16   rx_id;                  /* rx_id of the target request */
+	u16   _resvd0;                /* reserved */
+};
+
+/*
+ * fcpio_ttmf: firmware -> host request
+ *
+ * used by the firmware to notify the host of an incoming task management
+ * function request
+ */
+struct fcpio_ttmf {
+	u8    _resvd0;                /* reserved */
+	u8    s_id[3];		      /* FC vNIC only: Source S_ID */
+	u8    lun[LUN_ADDRESS];       /* FC vNIC only: LUN address */
+	u8    crn;                    /* SCSI Command Reference No. */
+	u8    _resvd2[3];             /* reserved */
+	u32   tmf_type;               /* task management request type */
+};
+
+/*
+ * Task Management request
+ */
+#define FCPIO_TTMF_CLR_ACA      0x40    /* Clear ACA condition */
+#define FCPIO_TTMF_LUN_RESET    0x10    /* logical unit reset task mgmt */
+#define FCPIO_TTMF_CLR_TASK_SET 0x04    /* clear task set */
+#define FCPIO_TTMF_ABT_TASK_SET 0x02    /* abort task set */
+#define FCPIO_TTMF_ABT_TASK     0x01    /* abort task */
+
+/*
+ * fcpio_tabort_cmpl: firmware -> host response
+ *
+ * used by the firmware to respond to a host's tabort request
+ */
+struct fcpio_tabort_cmpl {
+	u16   rx_id;                  /* rx_id of the target request */
+	u16   _resvd0;                /* reserved */
+};
+
+/*
+ * fcpio_ack: firmware -> host response
+ *
+ * used by firmware to notify the host of the last work request received
+ */
+struct fcpio_ack {
+	u16  request_out;             /* last host entry received */
+	u16  _resvd;
+};
+
+/*
+ * fcpio_reset_cmpl: firmware -> host response
+ *
+ * use by firmware to respond to the host's reset request
+ */
+struct fcpio_reset_cmpl {
+	u16   vnic_id;
+};
+
+/*
+ * fcpio_flogi_reg_cmpl: firmware -> host response
+ *
+ * fc vnic only
+ * response to the fcpio_flogi_reg request
+ */
+struct fcpio_flogi_reg_cmpl {
+	u32 _resvd;
+};
+
+/*
+ * fcpio_echo_cmpl: firmware -> host response
+ *
+ * response to the fcpio_echo request
+ */
+struct fcpio_echo_cmpl {
+	u32 _resvd;
+};
+
+/*
+ * fcpio_lunmap_chng: firmware -> host notification
+ *
+ * scsi vnic only
+ * notifies the host that the lunmap tables have changed
+ */
+struct fcpio_lunmap_chng {
+	u32 _resvd;
+};
+
+/*
+ * fcpio_lunmap_req_cmpl: firmware -> host response
+ *
+ * scsi vnic only
+ * response for lunmap table request from the host
+ */
+struct fcpio_lunmap_req_cmpl {
+	u32 _resvd;
+};
+
+/*
+ * Basic structure for all fcpio structures that are sent from the firmware to
+ * the host.  They are 64 bytes per structure.
+ */
+#define FCPIO_FW_REQ_LEN        64      /* expected length of fw requests */
+struct fcpio_fw_req {
+	struct fcpio_header hdr;
+
+	union {
+		/*
+		 * Defines space needed for request
+		 */
+		u8 buf[FCPIO_FW_REQ_LEN - sizeof(struct fcpio_header)];
+
+		/*
+		 * Initiator firmware responses
+		 */
+		struct fcpio_icmnd_cmpl         icmnd_cmpl;
+		struct fcpio_itmf_cmpl          itmf_cmpl;
+
+		/*
+		 * Target firmware new requests
+		 */
+		struct fcpio_tcmnd_16           tcmnd_16;
+		struct fcpio_tcmnd_32           tcmnd_32;
+
+		/*
+		 * Target firmware responses
+		 */
+		struct fcpio_tdrsp_cmpl         tdrsp_cmpl;
+		struct fcpio_ttmf               ttmf;
+		struct fcpio_tabort_cmpl        tabort_cmpl;
+
+		/*
+		 * Firmware response to work received
+		 */
+		struct fcpio_ack                ack;
+
+		/*
+		 * Misc requests
+		 */
+		struct fcpio_reset_cmpl         reset_cmpl;
+		struct fcpio_flogi_reg_cmpl     flogi_reg_cmpl;
+		struct fcpio_echo_cmpl          echo_cmpl;
+		struct fcpio_lunmap_chng        lunmap_chng;
+		struct fcpio_lunmap_req_cmpl    lunmap_req_cmpl;
+	} u;
+};
+
+/*
+ * Access routines to encode and decode the color bit, which is the most
+ * significant bit of the MSB of the structure
+ */
+static inline void fcpio_color_enc(struct fcpio_fw_req *fw_req, u8 color)
+{
+	u8 *c = ((u8 *) fw_req) + sizeof(struct fcpio_fw_req) - 1;
+
+	if (color)
+		*c |= 0x80;
+	else
+		*c &= ~0x80;
+}
+
+static inline void fcpio_color_dec(struct fcpio_fw_req *fw_req, u8 *color)
+{
+	u8 *c = ((u8 *) fw_req) + sizeof(struct fcpio_fw_req) - 1;
+
+	*color = *c >> 7;
+
+	/*
+	 * Make sure color bit is read from desc *before* other fields
+	 * are read from desc.  Hardware guarantees color bit is last
+	 * bit (byte) written.  Adding the rmb() prevents the compiler
+	 * and/or CPU from reordering the reads which would potentially
+	 * result in reading stale values.
+	 */
+
+	rmb();
+
+}
+
+/*
+ * Lunmap table entry for scsi vnics
+ */
+#define FCPIO_LUNMAP_TABLE_SIZE     256
+#define FCPIO_FLAGS_LUNMAP_VALID    0x80
+#define FCPIO_FLAGS_BOOT            0x01
+struct fcpio_lunmap_entry {
+	u8    bus;
+	u8    target;
+	u8    lun;
+	u8    path_cnt;
+	u16   flags;
+	u16   update_cnt;
+};
+
+struct fcpio_lunmap_tbl {
+	u32                   update_cnt;
+	struct fcpio_lunmap_entry   lunmaps[FCPIO_LUNMAP_TABLE_SIZE];
+};
+
+#endif /* _FCPIO_H_ */
drivers/scsi/fnic/fnic_io.h new file +67 -0

67 lines not on this page · raw message

drivers/scsi/fnic/fnic_scsi.c new file +1869 -0

1,869 lines not on this page · raw message

#4 lore raw

Re: [RFC][PATCH 2/6] fnic: add SCSI FCP handling

drivers/scsi/fnic/fcpio.h line 778

Quoted 5 lines · Abhijeet Joglekar
drivers/scsi/fnic/fcpio.h new file +5 -0
+
+struct fcpio_lunmap_tbl {
+	u32                   update_cnt;
+	struct fcpio_lunmap_entry   lunmaps[FCPIO_LUNMAP_TABLE_SIZE];
+};

I think this is not used now. You can delete the code related to it.

Quoted 76 lines
drivers/scsi/fnic/fnic_scsi.c new file +76 -0
+
+/*
+ * fnic_fcpio_flogi_reg_cmpl_handler
+ * Routine to handle flogi register completion
+ */
+static int fnic_fcpio_flogi_reg_cmpl_handler(struct fnic *fnic,
+					     struct fcpio_fw_req *desc)
+{
+	u8 type;
+	u8 hdr_status;
+	struct fcpio_tag tag;
+	struct fnic_event *event;
+	int ret = 0;
+	struct fc_frame *flogi_resp = NULL;
+	unsigned long flags;
+
+	fcpio_header_dec(&desc->hdr, &type, &hdr_status, &tag);
+
+	/* Update fnic state based on status of flogi reg completion */
+	spin_lock_irqsave(&fnic->fnic_lock, flags);
+
+	flogi_resp = fnic->flogi_resp;
+	fnic->flogi_resp = NULL;
+
+	if (fnic->state == FNIC_IN_ETH_TRANS_FC_MODE) {
+
+		/* Check flogi registration completion status */
+		if (!hdr_status) {
+			FNIC_SCSI_DBG(KERN_DEBUG, DFX "flog reg succeeded\n",
+				      fnic->fnic_no);
+			fnic->state = FNIC_IN_FC_MODE;
+		} else {
+			FNIC_SCSI_DBG(KERN_DEBUG,
+				      DFX "fnic flogi reg :failed %s\n",
+				      fnic->fnic_no,
+				      fnic_fcpio_status_to_str(hdr_status));
+			fnic->state = FNIC_IN_ETH_MODE;
+			ret = -1;
+		}
+	} else {
+		FNIC_SCSI_DBG(KERN_DEBUG, DFX "Unexpected fnic state %s while"
+			      " processing flogi reg completion\n",
+			      fnic->fnic_no, fnic_state_to_str(fnic->state));
+		ret = -1;
+	}
+
+	/* Successful flogi reg cmpl, pass frame to LibFC */
+	if (!ret && flogi_resp) {
+		if (fnic->stop_rx_link_events) {
+			spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+			goto reg_cmpl_handler_end;
+		}
+
+		event = kmem_cache_alloc(fnic_ev_cache, GFP_ATOMIC);
+
+		if (!event) {
+			spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+			dev_kfree_skb_irq(fp_skb(flogi_resp));
+			ret = -1;
+			goto reg_cmpl_handler_end;
+		}
+
+		memset(event, 0, sizeof(struct fnic_event));
+
+		/* initialize the event structure */
+		event->fp = flogi_resp;
+		fr_dev(flogi_resp) = fnic->lport;
+		event->fnic = fnic;
+		event->ev_type = EV_TYPE_FRAME;
+		event->is_flogi_resp_frame = 1;
+		fnic->event_count++;
+		spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+
+		/* insert it into event queue */
+		INIT_WORK(&event->event_work, fnic_event_work);
+		queue_work(fnic_event_queue, &event->event_work);

drivers/scsi/fnic/fnic_scsi.c line 620

Quoted 10 lines
drivers/scsi/fnic/fnic_scsi.c new file +10 -0
+
+	} else {
+		spin_unlock_irqrestore(&fnic->fnic_lock, flags);
+		if (flogi_resp)
+			dev_kfree_skb_irq(fp_skb(flogi_resp));
+	}
+
+reg_cmpl_handler_end:
+	return ret;
+}

What happens if this fails? Is this run when we relogin? It seems dangerous if there is memory pressure, pages were getting written out to this driver to free up mem, and if something bad happened and we had to relogin. If it fails are we stuck? Will something figure this out and retry? What about preallacting one event for this? Would that work? Or what about allocating the event in the fnic then just queueing the work here.

Block/scsi driver/code preallocate pretty much everything they need to make forward progress on at least one request. For relogin like this though, it seems like some drivers/code does not care, or we are getting careless or it is a lot of work and ugly code to make it work. So I am not sure if you have to handle the allocation failure here or not. Maybe we just have to do this in the main IO path.

drivers/scsi/fnic/fnic_scsi.c line 1842

Quoted 5 lines
drivers/scsi/fnic/fnic_scsi.c new file +5 -0
+
+void fnic_empty_scsi_cleanup(struct fc_lport *lp)
+{
+}
+

This is kind of odd. I guess I will leave it to you and Rob to discuss if that is how libfc is supposed to be used or if libfc should be modified to handle where you do not want the callout and do not want the default libfc gives you.

#5 lore raw

RFC PATCH 3/6 fnic: Add support for Fibre Channel Services through libFC

In reply to Abhijeet Joglekar

fnic: Add support for Fibre Channel Services through libFC

fnic_fcs.c: adds support for sending/receiving FCS frames in Ethernet and FC mode. Interacfaces with libFC, passes link events and frames to/from libFC.

fnic_attr.c: exposes host state, link state, driver version in SysFS

---
 drivers/scsi/fnic/fnic_attrs.c |   71 ++++
 drivers/scsi/fnic/fnic_fcs.c   |  719 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 790 insertions(+), 0 deletions(-)
 create mode 100644 drivers/scsi/fnic/fnic_attrs.c
 create mode 100644 drivers/scsi/fnic/fnic_fcs.c
drivers/scsi/fnic/fnic_attrs.c new file +71 -0
new file mode 100644
index 0000000..8144018
@@ -0,0 +1,71 @@
+/*
+ * Copyright 2008 Cisco Systems, Inc.  All rights reserved.
+ * Copyright 2007 Nuova Systems, Inc.  All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+#include <linux/init.h>
+#include <linux/pci.h>
+#include <linux/skbuff.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/kthread.h>
+#include <linux/device.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_host.h>
+#include "vnic_dev.h"
+#include "vnic_intr.h"
+#include "vnic_stats.h"
+#include "fnic_io.h"
+#include "fnic.h"
+
+static ssize_t fnic_show_state(struct device *dev,
+			       struct device_attribute *attr, char *buf)
+{
+	struct fc_lport *lp = shost_priv(class_to_shost(dev));
+	struct fnic *fnic = lport_priv(lp);
+
+	return snprintf(buf, PAGE_SIZE, "%s\n", fnic_state_str[fnic->state]);
+}
+
+static ssize_t fnic_show_drv_version(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", DRV_VERSION);
+}
+
+static ssize_t fnic_show_link_state(struct device *dev,
+				    struct device_attribute *attr, char *buf)
+{
+	struct fc_lport *lp = shost_priv(class_to_shost(dev));
+
+	return snprintf(buf, PAGE_SIZE, "%s\n", (lp->link_up)
+			? "Link Up" : "Link Down");
+}
+
+static DEVICE_ATTR(fnic_state, S_IRUGO, fnic_show_state, NULL);
+static DEVICE_ATTR(drv_version, S_IRUGO, fnic_show_drv_version, NULL);
+static DEVICE_ATTR(link_state, S_IRUGO, fnic_show_link_state, NULL);
+
+struct device_attribute *fnic_attrs[] = {
+	&dev_attr_fnic_state,
+	&dev_attr_drv_version,
+	&dev_attr_link_state,
+	NULL,
+};
drivers/scsi/fnic/fnic_fcs.c new file +719 -0

719 lines not on this page · raw message

#6 lore raw

Re: [RFC][PATCH 3/6] fnic: Add support for Fibre Channel Services through libFC

drivers/scsi/fnic/fnic_attrs.c line 51

Quoted 6 lines · Abhijeet Joglekar
drivers/scsi/fnic/fnic_attrs.c new file +6 -0
+
+static ssize_t fnic_show_drv_version(struct device *dev,
+				     struct device_attribute *attr, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", DRV_VERSION);
+}

I think in the past we were supposed to just export this with the MODULE_VERSION() function (you can then see it /sys/module/fnic/version).

I saw other FC drivers put it on the host now though, so maybe it is ok now.

#7 lore raw

RFC PATCH 4/6 fnic: adds resource allocation, interupt interfaces

In reply to Abhijeet Joglekar

fnic: adds resource allocation, interupt interfaces

fnic_res.[h,c]: gets device config, maps PCI resources, allocates descriptor rings, buffers

fnic_isr.c: interrupt vectors

---
 drivers/scsi/fnic/fnic_isr.c |  334 +++++++++++++++++++++++++++++++++
 drivers/scsi/fnic/fnic_res.c |  431 ++++++++++++++++++++++++++++++++++++++++++
 drivers/scsi/fnic/fnic_res.h |  197 +++++++++++++++++++
 3 files changed, 962 insertions(+), 0 deletions(-)
 create mode 100644 drivers/scsi/fnic/fnic_isr.c
 create mode 100644 drivers/scsi/fnic/fnic_res.c
 create mode 100644 drivers/scsi/fnic/fnic_res.h
drivers/scsi/fnic/fnic_isr.c new file +334 -0
new file mode 100644
index 0000000..b4f4771
@@ -0,0 +1,334 @@
+/*
+ * Copyright 2008 Cisco Systems, Inc.  All rights reserved.
+ * Copyright 2007 Nuova Systems, Inc.  All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+#include <linux/init.h>
+#include <linux/pci.h>
+#include <linux/skbuff.h>
+#include <linux/mempool.h>
+#include <linux/interrupt.h>
+#include <scsi/libfc.h>
+#include <scsi/fc_frame.h>
+#include "vnic_dev.h"
+#include "vnic_intr.h"
+#include "vnic_stats.h"
+#include "fnic_io.h"
+#include "fnic.h"
+
+static irqreturn_t fnic_isr_legacy(int irq, void *data)
+{
+	struct fnic *fnic = (struct fnic *)data;
+	u32 pba;
+	unsigned long work_done = 0;
+
+	pba = vnic_intr_legacy_pba(fnic->legacy_pba);
+	if (!pba)
+		return IRQ_NONE;
+
+	if (pba & (1 << FNIC_INTX_NOTIFY)) {
+		vnic_intr_return_all_credits(&fnic->intr[FNIC_INTX_NOTIFY]);
+		fnic_handle_link_event(fnic);
+	}
+
+	if (pba & (1 << FNIC_INTX_ERR)) {
+		vnic_intr_return_all_credits(&fnic->intr[FNIC_INTX_ERR]);
+		fnic_log_q_error(fnic);
+	}
+
+	if (pba & (1 << FNIC_INTX_WQ_RQ_COPYWQ)) {
+		work_done += fnic_wq_copy_cmpl_handler(fnic, 8);
+		work_done += fnic_wq_cmpl_handler(fnic, 4);
+		work_done += fnic_rq_cmpl_handler(fnic, 4);
+
+		vnic_intr_return_credits(&fnic->intr[FNIC_INTX_WQ_RQ_COPYWQ],
+					 work_done,
+					 1 /* unmask intr */,
+					 1 /* reset intr timer */);
+	}
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t fnic_isr_msi(int irq, void *data)
+{
+	struct fnic *fnic = (struct fnic *)data;
+	unsigned long work_done = 0;
+
+	work_done += fnic_wq_copy_cmpl_handler(fnic, 8);
+	work_done += fnic_wq_cmpl_handler(fnic, 4);
+	work_done += fnic_rq_cmpl_handler(fnic, 4);
+
+	vnic_intr_return_credits(&fnic->intr[0],
+				 work_done,
+				 1 /* unmask intr */,
+				 1 /* reset intr timer */);
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t fnic_isr_msix_rq(int irq, void *data)
+{
+	struct fnic *fnic = (struct fnic *)data;
+	unsigned long rq_work_done = 0;
+
+	rq_work_done = fnic_rq_cmpl_handler(fnic, 4);
+	vnic_intr_return_credits(&fnic->intr[FNIC_MSIX_RQ],
+				 rq_work_done,
+				 1 /* unmask intr */,
+				 1 /* reset intr timer */);
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t fnic_isr_msix_wq(int irq, void *data)
+{
+	struct fnic *fnic = (struct fnic *)data;
+	unsigned long wq_work_done = 0;
+
+	wq_work_done = fnic_wq_cmpl_handler(fnic, 4);
+	vnic_intr_return_credits(&fnic->intr[FNIC_MSIX_WQ],
+				 wq_work_done,
+				 1 /* unmask intr */,
+				 1 /* reset intr timer */);
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t fnic_isr_msix_wq_copy(int irq, void *data)
+{
+	struct fnic *fnic = (struct fnic *)data;
+	unsigned long wq_copy_work_done = 0;
+
+	wq_copy_work_done = fnic_wq_copy_cmpl_handler(fnic, 8);
+	vnic_intr_return_credits(&fnic->intr[FNIC_MSIX_WQ_COPY],
+				 wq_copy_work_done,
+				 1 /* unmask intr */,
+				 1 /* reset intr timer */);
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t fnic_isr_msix_err_notify(int irq, void *data)
+{
+	struct fnic *fnic = (struct fnic *)data;
+
+	vnic_intr_return_all_credits(&fnic->intr[FNIC_MSIX_ERR_NOTIFY]);
+	fnic_log_q_error(fnic);
+	fnic_handle_link_event(fnic);
+
+	return IRQ_HANDLED;
+}
+
+void fnic_free_intr(struct fnic *fnic)
+{
+	int i;
+
+	switch (vnic_dev_get_intr_mode(fnic->vdev)) {
+	case VNIC_DEV_INTR_MODE_INTX:
+	case VNIC_DEV_INTR_MODE_MSI:
+		free_irq(fnic->pdev->irq, fnic);
+		break;
+
+	case VNIC_DEV_INTR_MODE_MSIX:
+		for (i = 0; i < ARRAY_SIZE(fnic->msix); i++)
+			if (fnic->msix[i].requested)
+				free_irq(fnic->msix_entry[i].vector,
+					 fnic->msix[i].devid);
+		break;
+
+	default:
+		break;
+	}
+}
+
+int fnic_request_intr(struct fnic *fnic)
+{
+	int err = 0;
+	int i;
+
+	switch (vnic_dev_get_intr_mode(fnic->vdev)) {
+
+	case VNIC_DEV_INTR_MODE_INTX:
+		err = request_irq(fnic->pdev->irq, &fnic_isr_legacy,
+				  IRQF_SHARED, DRV_NAME, fnic);
+		break;
+
+	case VNIC_DEV_INTR_MODE_MSI:
+		err = request_irq(fnic->pdev->irq, &fnic_isr_msi,
+				  0, fnic->name, fnic);
+		break;
+
+	case VNIC_DEV_INTR_MODE_MSIX:
+
+		sprintf(fnic->msix[FNIC_MSIX_RQ].devname,
+			"%.11s-fcs-rq", fnic->name);
+		fnic->msix[FNIC_MSIX_RQ].isr = fnic_isr_msix_rq;
+		fnic->msix[FNIC_MSIX_RQ].devid = fnic;
+
+		sprintf(fnic->msix[FNIC_MSIX_WQ].devname,
+			"%.11s-fcs-wq", fnic->name);
+		fnic->msix[FNIC_MSIX_WQ].isr = fnic_isr_msix_wq;
+		fnic->msix[FNIC_MSIX_WQ].devid = fnic;
+
+		sprintf(fnic->msix[FNIC_MSIX_WQ_COPY].devname,
+			"%.11s-scsi-wq", fnic->name);
+		fnic->msix[FNIC_MSIX_WQ_COPY].isr = fnic_isr_msix_wq_copy;
+		fnic->msix[FNIC_MSIX_WQ_COPY].devid = fnic;
+
+		sprintf(fnic->msix[FNIC_MSIX_ERR_NOTIFY].devname,
+			"%.11s-err-notify", fnic->name);
+		fnic->msix[FNIC_MSIX_ERR_NOTIFY].isr =
+			fnic_isr_msix_err_notify;
+		fnic->msix[FNIC_MSIX_ERR_NOTIFY].devid = fnic;
+
+		for (i = 0; i < ARRAY_SIZE(fnic->msix); i++) {
+			err = request_irq(fnic->msix_entry[i].vector,
+					  fnic->msix[i].isr, 0,
+					  fnic->msix[i].devname,
+					  fnic->msix[i].devid);
+			if (err) {
+				printk(KERN_ERR PFX "MSIX: request_irq"
+				       " failed %d\n", err);
+				fnic_free_intr(fnic);
+				break;
+			}
+			fnic->msix[i].requested = 1;
+		}
+		break;
+
+	default:
+		break;
+	}
+
+	return err;
+}
+
+int fnic_set_intr_mode(struct fnic *fnic)
+{
+	unsigned int n = ARRAY_SIZE(fnic->rq);
+	unsigned int m = ARRAY_SIZE(fnic->wq);
+	unsigned int o = ARRAY_SIZE(fnic->wq_copy);
+	unsigned int i;
+
+	/*
+	 * Set interrupt mode (INTx, MSI, MSI-X) depending
+	 * system capabilities.
+	 *
+	 * Try MSI-X first
+	 *
+	 * We need n RQs, m WQs, o Copy WQs, n+m+o CQs, and n+m+o+1 INTRs
+	 * (last INTR is used for WQ/RQ errors and notification area)
+	 */
+
+	BUG_ON(ARRAY_SIZE(fnic->msix_entry) < n + m + o + 1);
+	for (i = 0; i < n + m + o + 1; i++)
+		fnic->msix_entry[i].entry = i;
+
+	if (fnic->rq_count >= n &&
+	    fnic->raw_wq_count >= m &&
+	    fnic->wq_copy_count >= o &&
+	    fnic->cq_count >= n + m + o) {
+		if (!pci_enable_msix(fnic->pdev, fnic->msix_entry,
+				    n + m + o + 1)) {
+			fnic->rq_count = n;
+			fnic->raw_wq_count = m;
+			fnic->wq_copy_count = o;
+			fnic->wq_count = m + o;
+			fnic->cq_count = n + m + o;
+			fnic->intr_count = n + m + o + 1;
+			fnic->err_intr_offset = FNIC_MSIX_ERR_NOTIFY;
+
+			FNIC_ISR_DBG(KERN_DEBUG,
+				     PFX "Using MSI-X Interrupts\n");
+			vnic_dev_set_intr_mode(fnic->vdev,
+					       VNIC_DEV_INTR_MODE_MSIX);
+			return 0;
+		}
+	}
+
+	/*
+	 * Next try MSI
+	 * We need 1 RQ, 1 WQ, 1 WQ_COPY, 3 CQs, and 1 INTR
+	 */
+	if (fnic->rq_count >= 1 &&
+	    fnic->raw_wq_count >= 1 &&
+	    fnic->wq_copy_count >= 1 &&
+	    fnic->cq_count >= 3 &&
+	    fnic->intr_count >= 1 &&
+	    !pci_enable_msi(fnic->pdev)) {
+
+		fnic->rq_count = 1;
+		fnic->raw_wq_count = 1;
+		fnic->wq_copy_count = 1;
+		fnic->wq_count = 2;
+		fnic->cq_count = 3;
+		fnic->intr_count = 1;
+		fnic->err_intr_offset = 0;
+
+		FNIC_ISR_DBG(KERN_DEBUG, PFX "Using MSI Interrupts\n");
+		vnic_dev_set_intr_mode(fnic->vdev, VNIC_DEV_INTR_MODE_MSI);
+
+		return 0;
+	}
+
+	/*
+	 * Next try INTx
+	 * We need 1 RQ, 1 WQ, 1 WQ_COPY, 3 CQs, and 3 INTRs
+	 * 1 INTR is used for all 3 queues, 1 INTR for queue errors
+	 * 1 INTR for notification area
+	 */
+
+	if (fnic->rq_count >= 1 &&
+	    fnic->raw_wq_count >= 1 &&
+	    fnic->wq_copy_count >= 1 &&
+	    fnic->cq_count >= 3 &&
+	    fnic->intr_count >= 3) {
+
+		fnic->rq_count = 1;
+		fnic->raw_wq_count = 1;
+		fnic->wq_copy_count = 1;
+		fnic->cq_count = 3;
+		fnic->intr_count = 3;
+
+		FNIC_ISR_DBG(KERN_DEBUG, PFX "Using Legacy Interrupts\n");
+		vnic_dev_set_intr_mode(fnic->vdev, VNIC_DEV_INTR_MODE_INTX);
+
+		return 0;
+	}
+
+	vnic_dev_set_intr_mode(fnic->vdev, VNIC_DEV_INTR_MODE_UNKNOWN);
+
+	return -EINVAL;
+}
+
+void fnic_clear_intr_mode(struct fnic *fnic)
+{
+	switch (vnic_dev_get_intr_mode(fnic->vdev)) {
+	case VNIC_DEV_INTR_MODE_MSIX:
+		pci_disable_msix(fnic->pdev);
+		break;
+	case VNIC_DEV_INTR_MODE_MSI:
+		pci_disable_msi(fnic->pdev);
+		break;
+	default:
+		break;
+	}
+
+	vnic_dev_set_intr_mode(fnic->vdev, VNIC_DEV_INTR_MODE_INTX);
+}
+
drivers/scsi/fnic/fnic_res.c new file +431 -0

431 lines not on this page · raw message

drivers/scsi/fnic/fnic_res.h new file +197 -0

197 lines not on this page · raw message

#8 lore raw

RFC PATCH 5/6 fnic: add descriptor right, buffers, device interface

In reply to Abhijeet Joglekar

fnic: add descriptor right, buffers, device interface

These files describe hardware descriptor, queue, buffer interface and also control interfaces

---
 drivers/scsi/fnic/cq_desc.h       |   78 ++++
 drivers/scsi/fnic/cq_enet_desc.h  |  167 +++++++++
 drivers/scsi/fnic/cq_exch_desc.h  |  182 ++++++++++
 drivers/scsi/fnic/rq_enet_desc.h  |   58 +++
 drivers/scsi/fnic/vnic_cq.c       |   86 +++++
 drivers/scsi/fnic/vnic_cq.h       |  121 ++++++
 drivers/scsi/fnic/vnic_cq_copy.h  |   62 +++
 drivers/scsi/fnic/vnic_dev.c      |  690 +++++++++++++++++++++++++++++++++++++
 drivers/scsi/fnic/vnic_dev.h      |  161 +++++++++
 drivers/scsi/fnic/vnic_devcmd.h   |  281 +++++++++++++++
 drivers/scsi/fnic/vnic_intr.c     |   60 +++
 drivers/scsi/fnic/vnic_intr.h     |  118 ++++++
 drivers/scsi/fnic/vnic_nic.h      |   69 ++++
 drivers/scsi/fnic/vnic_resource.h |   61 +++
 drivers/scsi/fnic/vnic_rq.c       |  197 +++++++++++
 drivers/scsi/fnic/vnic_rq.h       |  235 +++++++++++++
 drivers/scsi/fnic/vnic_scsi.h     |   99 +++++
 drivers/scsi/fnic/vnic_stats.h    |   68 ++++
 drivers/scsi/fnic/vnic_wq.c       |  183 ++++++++++
 drivers/scsi/fnic/vnic_wq.h       |  175 +++++++++
 drivers/scsi/fnic/vnic_wq_copy.c  |  118 ++++++
 drivers/scsi/fnic/vnic_wq_copy.h  |  129 +++++++
 drivers/scsi/fnic/wq_enet_desc.h  |   96 +++++
 23 files changed, 3494 insertions(+), 0 deletions(-)
 create mode 100644 drivers/scsi/fnic/cq_desc.h
 create mode 100644 drivers/scsi/fnic/cq_enet_desc.h
 create mode 100644 drivers/scsi/fnic/cq_exch_desc.h
 create mode 100644 drivers/scsi/fnic/rq_enet_desc.h
 create mode 100644 drivers/scsi/fnic/vnic_cq.c
 create mode 100644 drivers/scsi/fnic/vnic_cq.h
 create mode 100644 drivers/scsi/fnic/vnic_cq_copy.h
 create mode 100644 drivers/scsi/fnic/vnic_dev.c
 create mode 100644 drivers/scsi/fnic/vnic_dev.h
 create mode 100644 drivers/scsi/fnic/vnic_devcmd.h
 create mode 100644 drivers/scsi/fnic/vnic_intr.c
 create mode 100644 drivers/scsi/fnic/vnic_intr.h
 create mode 100644 drivers/scsi/fnic/vnic_nic.h
 create mode 100644 drivers/scsi/fnic/vnic_resource.h
 create mode 100644 drivers/scsi/fnic/vnic_rq.c
 create mode 100644 drivers/scsi/fnic/vnic_rq.h
 create mode 100644 drivers/scsi/fnic/vnic_scsi.h
 create mode 100644 drivers/scsi/fnic/vnic_stats.h
 create mode 100644 drivers/scsi/fnic/vnic_wq.c
 create mode 100644 drivers/scsi/fnic/vnic_wq.h
 create mode 100644 drivers/scsi/fnic/vnic_wq_copy.c
 create mode 100644 drivers/scsi/fnic/vnic_wq_copy.h
 create mode 100644 drivers/scsi/fnic/wq_enet_desc.h
drivers/scsi/fnic/cq_desc.h new file +78 -0
new file mode 100644
index 0000000..d1225cf
@@ -0,0 +1,78 @@
+/*
+ * Copyright 2008 Cisco Systems, Inc.  All rights reserved.
+ * Copyright 2007 Nuova Systems, Inc.  All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#ifndef _CQ_DESC_H_
+#define _CQ_DESC_H_
+
+/*
+ * Completion queue descriptor types
+ */
+enum cq_desc_types {
+	CQ_DESC_TYPE_WQ_ENET = 0,
+	CQ_DESC_TYPE_DESC_COPY = 1,
+	CQ_DESC_TYPE_WQ_EXCH = 2,
+	CQ_DESC_TYPE_RQ_ENET = 3,
+	CQ_DESC_TYPE_RQ_FCP = 4,
+};
+
+/* Completion queue descriptor: 16B
+ *
+ * All completion queues have this basic layout.  The
+ * type_specfic area is unique for each completion
+ * queue type.
+ */
+struct cq_desc {
+	__le16 completed_index;
+	__le16 q_number;
+	u8 type_specfic[11];
+	u8 type_color;
+};
+
+#define CQ_DESC_TYPE_BITS        4
+#define CQ_DESC_TYPE_MASK        ((1 << CQ_DESC_TYPE_BITS) - 1)
+#define CQ_DESC_COLOR_MASK       1
+#define CQ_DESC_COLOR_SHIFT      7
+#define CQ_DESC_Q_NUM_BITS       10
+#define CQ_DESC_Q_NUM_MASK       ((1 << CQ_DESC_Q_NUM_BITS) - 1)
+#define CQ_DESC_COMP_NDX_BITS    12
+#define CQ_DESC_COMP_NDX_MASK    ((1 << CQ_DESC_COMP_NDX_BITS) - 1)
+
+static inline void cq_desc_dec(const struct cq_desc *desc_arg,
+	u8 *type, u8 *color, u16 *q_number, u16 *completed_index)
+{
+	const struct cq_desc *desc = desc_arg;
+	const u8 type_color = desc->type_color;
+
+	*color = (type_color >> CQ_DESC_COLOR_SHIFT) & CQ_DESC_COLOR_MASK;
+
+	/*
+	 * Make sure color bit is read from desc *before* other fields
+	 * are read from desc.  Hardware guarantees color bit is last
+	 * bit (byte) written.  Adding the rmb() prevents the compiler
+	 * and/or CPU from reordering the reads which would potentially
+	 * result in reading stale values.
+	 */
+
+	rmb();
+
+	*type = type_color & CQ_DESC_TYPE_MASK;
+	*q_number = le16_to_cpu(desc->q_number) & CQ_DESC_Q_NUM_MASK;
+	*completed_index = le16_to_cpu(desc->completed_index) &
+		CQ_DESC_COMP_NDX_MASK;
+}
+
+#endif /* _CQ_DESC_H_ */
drivers/scsi/fnic/cq_enet_desc.h new file +167 -0

167 lines not on this page · raw message

drivers/scsi/fnic/cq_exch_desc.h new file +182 -0

182 lines not on this page · raw message

drivers/scsi/fnic/rq_enet_desc.h new file +58 -0

58 lines not on this page · raw message

drivers/scsi/fnic/vnic_cq.c new file +86 -0

86 lines not on this page · raw message

drivers/scsi/fnic/vnic_cq.h new file +121 -0

121 lines not on this page · raw message

drivers/scsi/fnic/vnic_cq_copy.h new file +62 -0

62 lines not on this page · raw message

drivers/scsi/fnic/vnic_dev.c new file +690 -0

690 lines not on this page · raw message

drivers/scsi/fnic/vnic_dev.h new file +161 -0

161 lines not on this page · raw message

drivers/scsi/fnic/vnic_devcmd.h new file +281 -0

281 lines not on this page · raw message

drivers/scsi/fnic/vnic_intr.c new file +60 -0

60 lines not on this page · raw message

drivers/scsi/fnic/vnic_intr.h new file +118 -0

118 lines not on this page · raw message

drivers/scsi/fnic/vnic_nic.h new file +69 -0

69 lines not on this page · raw message

drivers/scsi/fnic/vnic_resource.h new file +61 -0

61 lines not on this page · raw message

drivers/scsi/fnic/vnic_rq.c new file +197 -0

197 lines not on this page · raw message

drivers/scsi/fnic/vnic_rq.h new file +235 -0

235 lines not on this page · raw message

drivers/scsi/fnic/vnic_scsi.h new file +99 -0

99 lines not on this page · raw message

drivers/scsi/fnic/vnic_stats.h new file +68 -0

68 lines not on this page · raw message

drivers/scsi/fnic/vnic_wq.c new file +183 -0

183 lines not on this page · raw message

drivers/scsi/fnic/vnic_wq.h new file +175 -0

24 lines not on this page · raw message

drivers/scsi/fnic/vnic_wq_copy.c new file +118 -0

118 lines not on this page · raw message

drivers/scsi/fnic/vnic_wq_copy.h new file +129 -0

129 lines not on this page · raw message

drivers/scsi/fnic/wq_enet_desc.h new file +96 -0

96 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.

#9 lore raw

Re: [RFC][PATCH 5/6] fnic: add descriptor right, buffers, device interface

Quoted 8 lines · Abhijeet Joglekar
drivers/scsi/fnic/vnic_cq.h new file +1 -0
new file mode 100644
index 0000000..4ede680
 
 
+

drivers/scsi/fnic/vnic_cq.h line 33

Quoted 10 lines
drivers/scsi/fnic/vnic_cq.h new file +10 -0
+/*
+ * These defines avoid symbol clash between fnic and enic (Cisco 10G Eth
+ * Driver) when both are built with CONFIG options =y
+ */
+#define vnic_cq_service fnic_cq_service
+#define vnic_cq_free fnic_cq_free
+#define vnic_cq_alloc fnic_cq_alloc
+#define vnic_cq_init fnic_cq_init
+#define vnic_cq_clean fnic_cq_clean
+

Are these functions implementations the same as the net ones? I only looked at the first one and it looked the same.

If you are going to share code then you probably want to put them in a new module that both enic and fnic can use.

Are you going to ask where the module should then live (drivers/net vs driver/scsi)? I do not know the answer.

#10 lore raw

RE: [RFC][PATCH 5/6] fnic: add descriptor right, buffers, device interface

Quoted 22 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Mike Christie
Sent: Thursday, March 12, 2009 1:44 PM
To: Abhijeet Arvind Joglekar (abjoglek)
Cc: [email protected]; [email protected]; Joe Eykholt (jeykholt)
Subject: Re: [RFC][PATCH 5/6] fnic: add descriptor right, buffers, device interface

Review

Quoted 1 line · Abhijeet Joglekar
(quoted diff) +0 -0

b/drivers/scsi/fnic/vnic_cq.h

drivers/scsi/fnic/vnic_cq.h

Quoted 8 lines

new file mode 100644 index 0000000..4ede680

drivers/scsi/fnic/vnic_cq.h new file +3 -0
 
 
+
+/*
+ * These defines avoid symbol clash between fnic and enic 

(Cisco 10G Eth

Quoted 11 lines

drivers/scsi/fnic/vnic_cq.h line 33

Quoted 8 lines

+ * Driver) when both are built with CONFIG options =y
+ */

drivers/scsi/fnic/vnic_cq.h new file +6 -0
+#define vnic_cq_service fnic_cq_service
+#define vnic_cq_free fnic_cq_free
+#define vnic_cq_alloc fnic_cq_alloc
+#define vnic_cq_init fnic_cq_init
+#define vnic_cq_clean fnic_cq_clean
+

Are these functions implementations the same as the net ones? I only looked at the first one and it looked the same.

Yes, they are the same, it is the low-level queue/buffer code that we wanted to share across enic and fnic. We did consider making a new module out of it, but its not a lot of code, and there was the problem of where the common files will reside, and how we share it across the net and scsi subsystems.

So, the plan was to re-define them for fnic (since enic is already integrated) and then go with what the kernel community thinks the best way is.

Thanks for the review
-- abhijeet

Quoted 14 lines

If you are going to share code then you probably want to put them in a
new module that both enic and fnic can use.

Are you going to ask where the module should then live (drivers/net vs
driver/scsi)? I do not know the answer.

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#11 lore raw

Re: [RFC][PATCH 5/6] fnic: add descriptor right, buffers, device interface

Quoted 28 lines · Abhijeet Arvind Joglekar (abjoglek)
Quoted 18 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Mike Christie
Sent: Thursday, March 12, 2009 1:44 PM
To: Abhijeet Arvind Joglekar (abjoglek)
Cc: [email protected]; [email protected]; Joe Eykholt (jeykholt)
Subject: Re: [RFC][PATCH 5/6] fnic: add descriptor right, buffers, device interface

Review

Quoted 1 line · Abhijeet Joglekar
(quoted diff) +0 -0

b/drivers/scsi/fnic/vnic_cq.h

drivers/scsi/fnic/vnic_cq.h

Quoted 7 lines

new file mode 100644 index 0000000..4ede680

drivers/scsi/fnic/vnic_cq.h new file +3 -0
 
+
+/*
+ * These defines avoid symbol clash between fnic and enic 

(Cisco 10G Eth

Quoted 9 lines

drivers/scsi/fnic/vnic_cq.h line 33

Quoted 8 lines

+ * Driver) when both are built with CONFIG options =y
+ */

drivers/scsi/fnic/vnic_cq.h new file +6 -0
+#define vnic_cq_service fnic_cq_service
+#define vnic_cq_free fnic_cq_free
+#define vnic_cq_alloc fnic_cq_alloc
+#define vnic_cq_init fnic_cq_init
+#define vnic_cq_clean fnic_cq_clean
+

Are these functions implementations the same as the net ones? I only looked at the first one and it looked the same.

Quoted 3 lines

Yes, they are the same, it is the low-level queue/buffer code that we wanted to share across enic and fnic. We did consider making a new module out of it, but its not a lot of code, and there was the problem

Normally I would just make it a new module, but I looked it over again and it is less than 100 lines so I am not sure if there is some point we say it is ok to duplicate code. Personally I would not like having to update multiple modules through two different maintainers (netdev and scsi). OTOH, updating the one module then making sure that two maintainers have the right code so their tress can compile can be tricky.

Quoted 3 lines

of where the common files will reside, and how we share it across the net and scsi subsystems.

I think it can live anywhere. The Kconfig and userspace module deps will make sure it gets built and loaded correctly.

Quoted 4 lines

So, the plan was to re-define them for fnic (since enic is already integrated) and then go with what the kernel community thinks the best way is.

#12 lore raw

RFC PATCH 6/6 fnic: Patch MAINTAINERS, scsi Makefile, scsi Kconfig

In reply to Abhijeet Joglekar

fnic: Patch MAINTAINERS, scsi Makefile, scsi Kconfig

---
 MAINTAINERS           |    8 ++++++++
 drivers/scsi/Kconfig  |   11 +++++++++++
 drivers/scsi/Makefile |    1 +
 3 files changed, 20 insertions(+), 0 deletions(-)
MAINTAINERS +8 -0
index 59fd2d1..5617798 100644
@@ -1144,6 +1144,14 @@ P:	Nils Faerber (port to kernel 2.4)
 M:	Nils Faerber <[email protected]>
 S:	Maintained
 
+CISCO FCoE HBA DRIVER (drivers/scsi/fnic)
+P:	Abhijeet Joglekar
+M:	[email protected]
+P:	Joe Eykholt
+M:	[email protected]
+L:	[email protected]
+S:	Supported
+
 CODA FILE SYSTEM
 P:	Jan Harkes
 M:	[email protected]
drivers/scsi/Kconfig +11 -0
index 256c7be..236bb39 100644
@@ -618,6 +618,17 @@ config FCOE
 	---help---
 	  Fibre Channel over Ethernet module
 
+config FCOE_FNIC
+	tristate "Cisco FNIC Driver"
+	depends on PCI && X86
+	select LIBFC
+	help
+	  This is support for the Cisco PCI-Express FCoE HBA.
+
+	  To compile this driver as a module, choose M here and read
+	  <file:Documentation/scsi/scsi.txt>.
+	  The module will be called fnic.
+
 config SCSI_DMX3191D
 	tristate "DMX3191D SCSI support"
 	depends on PCI && SCSI
drivers/scsi/Makefile +1 -0
index 7461eb0..e0aed2f 100644
@@ -38,6 +38,7 @@ obj-$(CONFIG_SCSI_DH)		+= device_handler/
 
 obj-$(CONFIG_LIBFC)		+= libfc/
 obj-$(CONFIG_FCOE)		+= fcoe/
+obj-$(CONFIG_FCOE_FNIC)		+= fnic/
 obj-$(CONFIG_ISCSI_TCP) 	+= libiscsi.o	libiscsi_tcp.o iscsi_tcp.o
 obj-$(CONFIG_INFINIBAND_ISER) 	+= libiscsi.o
 obj-$(CONFIG_SCSI_A4000T)	+= 53c700.o	a4000t.o