lpfc new hardware patches for 12.0.0.0
Subsystems (2)
v2 cover lpfc new hardware patches for 12.0.0.0
This patch set adds support for Broadcom's new G7 product that supports G4G FC.
The patches were cut against the Martin's 4.16/scsi-queue tree.
The patches are dependent (layered on top of) the lpfc 11.4.0.7 patchset that was recently posted. See:
https://www.spinics.net/lists/linux-scsi/msg116956.html
v2:
Address comments:
patch 1, 2, 4, 7 - patch description typos
patch 6 - typo on 256GBit speed define
James Smart (13):
lpfc: Rework lpfc to allow different sli4 cq and eq handlers
lpfc: Rework sli4 doorbell infrastructure
lpfc: Add SLI-4 if_type=6 support to the code base
lpfc: Add push-to-adapter support to sli4
lpfc: Add PCI Ids for if_type=6 hardware
lpfc: Add 64G link speed support
lpfc: Add if_type=6 support for cycling valid bits
lpfc: Enable fw download on if_type=6 devices
lpfc: Add embedded data pointers for enhanced performance
lpfc: Fix nvme embedded io length on new hardware
lpfc: Work around NVME cmd iu SGL type
lpfc: update driver version to 12.0.0.0
lpfc: Update 12.0.0.0 modified files for 2018 Copyright
drivers/scsi/lpfc/lpfc.h | 20 +- drivers/scsi/lpfc/lpfc_attr.c | 85 +++++-- drivers/scsi/lpfc/lpfc_bsg.c | 6 +- drivers/scsi/lpfc/lpfc_ct.c | 7 +- drivers/scsi/lpfc/lpfc_debugfs.c | 22 +- drivers/scsi/lpfc/lpfc_debugfs.h | 13 +- drivers/scsi/lpfc/lpfc_els.c | 5 + drivers/scsi/lpfc/lpfc_hbadisc.c | 1 + drivers/scsi/lpfc/lpfc_hw.h | 15 +- drivers/scsi/lpfc/lpfc_hw4.h | 115 ++++++++- drivers/scsi/lpfc/lpfc_ids.h | 4 +- drivers/scsi/lpfc/lpfc_init.c | 245 +++++++++++++++--- drivers/scsi/lpfc/lpfc_mbox.c | 4 + drivers/scsi/lpfc/lpfc_nvme.c | 69 +++-- drivers/scsi/lpfc/lpfc_nvmet.c | 24 ++ drivers/scsi/lpfc/lpfc_scsi.c | 8 +- drivers/scsi/lpfc/lpfc_sli.c | 535 +++++++++++++++++++++++++++++---------- drivers/scsi/lpfc/lpfc_sli4.h | 30 ++- drivers/scsi/lpfc/lpfc_version.h | 2 +- 19 files changed, 960 insertions(+), 250 deletions(-)
Signature · 1 line
2.13.1
PATCH v2 1/13 lpfc: Rework lpfc to allow different sli4 cq and eq handlers
Up until now, an SLI-4 device had no variance in the way it handled its EQs and CQs. With newer hardware, there are now differences in doorbells and some differences in how entries are valid.
Prepare the code for new hardware by creating a sli4-based callout table that can be set based on if_type.
--- drivers/scsi/lpfc/lpfc_init.c | 7 +++++ drivers/scsi/lpfc/lpfc_sli.c | 63 ++++++++++++++++++++++--------------------- drivers/scsi/lpfc/lpfc_sli4.h | 5 ++++ 3 files changed, 44 insertions(+), 31 deletions(-)
drivers/scsi/lpfc/lpfc_init.c +7 -0
@@ -9540,6 +9540,13 @@ lpfc_sli4_pci_mem_setup(struct lpfc_hba *phba) } } +/* Set up the EQ/CQ register handeling functions now */ +if (if_type <=LPFC_SLI_INTF_IF_TYPE_2 ) { + phba->sli4_hba.sli4_eq_clr_intr = lpfc_sli4_eq_clr_intr; + phba->sli4_hba.sli4_eq_release = lpfc_sli4_eq_release; + phba->sli4_hba.sli4_cq_release = lpfc_sli4_cq_release; + } +return 0 ; out_iounmap_all:
drivers/scsi/lpfc/lpfc_sli.c +32 -31
168 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_sli4.h +5 -0
@@ -582,6 +582,10 @@ struct lpfc_sli4_hba {struct lpfc_bbscn_params bbscn_params;struct lpfc_hba_eq_hdl *hba_eq_hdl;/* HBA per-WQ handle */ +void (*sli4_eq_clr_intr)(struct lpfc_queue *q); +uint32_t (*sli4_eq_release)(struct lpfc_queue *q,bool arm); +uint32_t (*sli4_cq_release)(struct lpfc_queue *q,bool arm); +/* Pointers to the constructed SLI4 queues */ struct lpfc_queue **hba_eq;/* Event queues for HBA */ struct lpfc_queue **fcp_cq;/* Fast-path FCP compl queue */
@@ -848,6 +852,7 @@ void lpfc_sli_remove_dflt_fcf(struct lpfc_hba *);int lpfc_sli4_get_els_iocb_cnt (struct lpfc_hba *);int lpfc_sli4_get_iocb_cnt (struct lpfc_hba *phba);int lpfc_sli4_init_vpi (struct lpfc_vport *); +inline void lpfc_sli4_eq_clr_intr (struct lpfc_queue *);uint32_t lpfc_sli4_cq_release (struct lpfc_queue *,bool );uint32_t lpfc_sli4_eq_release (struct lpfc_queue *,bool );void lpfc_sli4_fcfi_unreg (struct lpfc_hba *,uint16_t );
Signature · 1 line
2.13.1
Re: [PATCH v2 01/13] lpfc: Rework lpfc to allow different sli4 cq and eq handlers
I first though of renameing:lpfc_sli4_eq_clr_intr -> lpfc_sli4_t2_eq_clr_intrlpfc_sli4_eq_release -> lpfc_sli4_t2_eq_releaselpfc_sli4_cq_release -> lpfc_sli4_t2_cq_release
and then make lpfc_sli4_eq_clr_intr, etc... wrappers over the new function pointers so and you could keep the callsites, but then you'd have to change the callsites anyways as the function prototype will need to change.
So looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 2/13 lpfc: Rework sli4 doorbell infrastructure
In reply to James SmartUp until now, all SLI-4 devices had the same doorbells at the same bar locations. With newer hardware, there are now independent EQ and CQ doorbells and the bar locations differ.
Prepare the code for new hardware by separating the eq/cq doorbell into separate components. The components can be set based on if_type.
--- drivers/scsi/lpfc/lpfc_debugfs.c | 20 ++++++++++++++------ drivers/scsi/lpfc/lpfc_debugfs.h | 11 ++++++----- drivers/scsi/lpfc/lpfc_init.c | 9 ++++++--- drivers/scsi/lpfc/lpfc_sli.c | 8 ++++---- drivers/scsi/lpfc/lpfc_sli4.h | 3 ++- 5 files changed, 32 insertions(+), 19 deletions(-)
drivers/scsi/lpfc/lpfc_debugfs.c +14 -6
@@ -3944,10 +3944,15 @@ lpfc_idiag_drbacc_read_reg(struct lpfc_hba *phba, char *pbuffer,return 0 ;switch (drbregid) { -case LPFC_DRB_EQCQ : - len +=snprintf (pbuffer+len,LPFC_DRB_ACC_BUF_SIZE -len, -"EQCQ-DRB-REG: 0x%08x\n" , -readl (phba->sli4_hba.EQCQDBregaddr)); +case LPFC_DRB_EQ : + len +=snprintf (pbuffer + len,LPFC_DRB_ACC_BUF_SIZE -len, +"EQ-DRB-REG: 0x%08x\n" , +readl (phba->sli4_hba.EQDBregaddr)); +break ; +case LPFC_DRB_CQ : + len +=snprintf (pbuffer + len,LPFC_DRB_ACC_BUF_SIZE - len, +"CQ-DRB-REG: 0x%08x\n" , +readl (phba->sli4_hba.CQDBregaddr));break ;case LPFC_DRB_MQ : len += snprintf(pbuffer+len, LPFC_DRB_ACC_BUF_SIZE-len,
@@ -4086,8 +4091,11 @@ lpfc_idiag_drbacc_write(struct file *file, const char __user *buf, idiag.cmd.opcode == LPFC_IDIAG_CMD_DRBACC_ST || idiag.cmd.opcode == LPFC_IDIAG_CMD_DRBACC_CL) {switch (drb_reg_id) { -case LPFC_DRB_EQCQ : - drb_reg = phba->sli4_hba.EQCQDBregaddr; +case LPFC_DRB_EQ : + drb_reg = phba->sli4_hba.EQDBregaddr; +break ; +case LPFC_DRB_CQ : + drb_reg = phba->sli4_hba.CQDBregaddr;break ;case LPFC_DRB_MQ : drb_reg = phba->sli4_hba.MQDBregaddr;
drivers/scsi/lpfc/lpfc_debugfs.h +6 -5
@@ -126,12 +126,13 @@#define LPFC_DRB_ACC_WR_CMD_ARG 2#define LPFC_DRB_ACC_BUF_SIZE 256 -#define LPFC_DRB_EQCQ 1 -#define LPFC_DRB_MQ 2 -#define LPFC_DRB_WQ 3 -#define LPFC_DRB_RQ 4 +#define LPFC_DRB_EQ 1 +#define LPFC_DRB_CQ 2 +#define LPFC_DRB_MQ 3 +#define LPFC_DRB_WQ 4 +#define LPFC_DRB_RQ 5 -#define LPFC_DRB_MAX 4 +#define LPFC_DRB_MAX 5#define IDIAG_DRBACC_REGID_INDX 0#define IDIAG_DRBACC_VALUE_INDX 1
drivers/scsi/lpfc/lpfc_init.c +6 -3
@@ -7430,8 +7430,9 @@ lpfc_sli4_bar0_register_memmap(struct lpfc_hba *phba, uint32_t if_type) phba->sli4_hba.WQDBregaddr = phba->sli4_hba.conf_regs_memmap_p +LPFC_ULP0_WQ_DOORBELL ; - phba->sli4_hba.EQCQDBregaddr = + phba->sli4_hba.CQDBregaddr = phba->sli4_hba.conf_regs_memmap_p +LPFC_EQCQ_DOORBELL ; + phba->sli4_hba.EQDBregaddr = phba->sli4_hba.CQDBregaddr; phba->sli4_hba.MQDBregaddr = phba->sli4_hba.conf_regs_memmap_p +LPFC_MQ_DOORBELL ; phba->sli4_hba.BMBXregaddr =
@@ -7488,8 +7489,10 @@ lpfc_sli4_bar2_register_memmap(struct lpfc_hba *phba, uint32_t vf) phba->sli4_hba.WQDBregaddr = (phba->sli4_hba.drbl_regs_memmap_p + vf *LPFC_VFR_PAGE_SIZE +LPFC_ULP0_WQ_DOORBELL ); - phba->sli4_hba.EQCQDBregaddr = (phba->sli4_hba.drbl_regs_memmap_p + - vf *LPFC_VFR_PAGE_SIZE +LPFC_EQCQ_DOORBELL ); + phba->sli4_hba.CQDBregaddr = (phba->sli4_hba.drbl_regs_memmap_p + + vf *LPFC_VFR_PAGE_SIZE + +LPFC_EQCQ_DOORBELL ); + phba->sli4_hba.EQDBregaddr = phba->sli4_hba.CQDBregaddr; phba->sli4_hba.MQDBregaddr = (phba->sli4_hba.drbl_regs_memmap_p + vf *LPFC_VFR_PAGE_SIZE +LPFC_MQ_DOORBELL ); phba->sli4_hba.BMBXregaddr = (phba->sli4_hba.drbl_regs_memmap_p +
drivers/scsi/lpfc/lpfc_sli.c +4 -4
@@ -310,7 +310,7 @@ lpfc_sli4_eq_clr_intr(struct lpfc_queue *q)bf_set (lpfc_eqcq_doorbell_eqid_hi, &doorbell, (q->queue_id >>LPFC_EQID_HI_FIELD_SHIFT ));bf_set (lpfc_eqcq_doorbell_eqid_lo, &doorbell, q->queue_id); -writel (doorbell.word0, q->phba->sli4_hba.EQCQDBregaddr); +writel (doorbell.word0, q->phba->sli4_hba.EQDBregaddr); }/**
@@ -360,10 +360,10 @@ lpfc_sli4_eq_release(struct lpfc_queue *q, bool arm)bf_set (lpfc_eqcq_doorbell_eqid_hi, &doorbell, (q->queue_id >>LPFC_EQID_HI_FIELD_SHIFT ));bf_set (lpfc_eqcq_doorbell_eqid_lo, &doorbell, q->queue_id); -writel (doorbell.word0, q->phba->sli4_hba.EQCQDBregaddr); +writel (doorbell.word0, q->phba->sli4_hba.EQDBregaddr);/* PCI read to flush PCI pipeline on re-arming for INTx mode */ if ((q->phba->intr_type == INTx) && (arm ==LPFC_QUEUE_REARM )) -readl (q->phba->sli4_hba.EQCQDBregaddr); +readl (q->phba->sli4_hba.EQDBregaddr);return released; }
@@ -453,7 +453,7 @@ lpfc_sli4_cq_release(struct lpfc_queue *q, bool arm)bf_set (lpfc_eqcq_doorbell_cqid_hi, &doorbell, (q->queue_id >>LPFC_CQID_HI_FIELD_SHIFT ));bf_set (lpfc_eqcq_doorbell_cqid_lo, &doorbell, q->queue_id); -writel (doorbell.word0, q->phba->sli4_hba.EQCQDBregaddr); +writel (doorbell.word0, q->phba->sli4_hba.CQDBregaddr);return released; }
drivers/scsi/lpfc/lpfc_sli4.h +2 -1
@@ -569,7 +569,8 @@ struct lpfc_sli4_hba {/* IF type 0, BAR 0 and if type 2, BAR 0 doorbell register memory map */ void __iomem *RQDBregaddr;/* RQ_DOORBELL register */ void __iomem *WQDBregaddr;/* WQ_DOORBELL register */ -void __iomem *EQCQDBregaddr;/* EQCQ_DOORBELL register */ +void __iomem *CQDBregaddr;/* CQ_DOORBELL register */ +void __iomem *EQDBregaddr;/* EQ_DOORBELL register */ void __iomem *MQDBregaddr;/* MQ_DOORBELL register */ void __iomem *BMBXregaddr;/* BootStrap MBX register */
Signature · 1 line
2.13.1
Re: [PATCH v2 02/13] lpfc: Rework sli4 doorbell infrastructure
drivers/scsi/lpfc/lpfc_sli4.h struct lpfc_sli4_hba { · line 575
Quoted 13 lines
drivers/scsi/lpfc/lpfc_sli4.h +2
-1@@ -569,7 +569,8 @@ struct lpfc_sli4_hba {/* IF type 0, BAR 0 and if type 2, BAR 0 doorbell register memory map */ void __iomem *RQDBregaddr;/* RQ_DOORBELL register */ void __iomem *WQDBregaddr;/* WQ_DOORBELL register */ -void __iomem *EQCQDBregaddr;/* EQCQ_DOORBELL register */ +void __iomem *CQDBregaddr;/* CQ_DOORBELL register */ +void __iomem *EQDBregaddr;/* EQ_DOORBELL register */ void __iomem *MQDBregaddr;/* MQ_DOORBELL register */ void __iomem *BMBXregaddr;/* BootStrap MBX register */
Although CQDBregaddr, EQDBregaddr, BMBXregaddr, etc... really look ugly and this would've been the perfect chance to change it ;-)
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 3/13 lpfc: Add SLI-4 if_type=6 support to the code base
In reply to James SmartNew hardware supports a SLI-4 interface, but with a new if_type variant of 6.
If_type=6 has a different PCI BAR map, separate EQ/CQ doorbells, and some changes in doorbell formats.
Add the changes for the if_type into headers, adapter initialization and control flows. Add new eq and cq handlers.
--- drivers/scsi/lpfc/lpfc_bsg.c | 4 +- drivers/scsi/lpfc/lpfc_hw4.h | 54 ++++++++++++++++++- drivers/scsi/lpfc/lpfc_init.c | 120 +++++++++++++++++++++++++++++++++++------- drivers/scsi/lpfc/lpfc_sli.c | 120 ++++++++++++++++++++++++++++++++++++++++-- drivers/scsi/lpfc/lpfc_sli4.h | 3 ++ 5 files changed, 275 insertions(+), 26 deletions(-)
drivers/scsi/lpfc/lpfc_bsg.c +2 -2
@@ -3867,7 +3867,7 @@ lpfc_bsg_sli_cfg_read_cmd_ext(struct lpfc_hba *phba, struct bsg_job *job, "ext_buf_cnt:%d\n", ext_buf_cnt); }else {/* sanity check on interface type for support */ -if (bf_get (lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) != +if (bf_get (lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) <LPFC_SLI_INTF_IF_TYPE_2 ) { rc = -ENODEV ; goto job_error;
@@ -4053,7 +4053,7 @@ lpfc_bsg_sli_cfg_write_cmd_ext(struct lpfc_hba *phba, struct bsg_job *job, "ext_buf_cnt:%d\n", ext_buf_cnt); }else {/* sanity check on interface type for support */ -if (bf_get (lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) != +if (bf_get (lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) <LPFC_SLI_INTF_IF_TYPE_2 )return -ENODEV ;/* nemb_tp == nemb_hbd */
drivers/scsi/lpfc/lpfc_hw4.h +52 -2
87 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_init.c +101 -19
222 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_sli.c +117 -3
162 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_sli4.h +3 -0
@@ -856,6 +856,9 @@ int lpfc_sli4_init_vpi(struct lpfc_vport *);inline void lpfc_sli4_eq_clr_intr (struct lpfc_queue *);uint32_t lpfc_sli4_cq_release (struct lpfc_queue *,bool );uint32_t lpfc_sli4_eq_release (struct lpfc_queue *,bool ); +inline void lpfc_sli4_if6_eq_clr_intr (struct lpfc_queue *q); +uint32_t lpfc_sli4_if6_cq_release (struct lpfc_queue *q,bool arm); +uint32_t lpfc_sli4_if6_eq_release (struct lpfc_queue *q,bool arm);void lpfc_sli4_fcfi_unreg (struct lpfc_hba *,uint16_t );int lpfc_sli4_fcf_scan_read_fcf_rec (struct lpfc_hba *,uint16_t );int lpfc_sli4_fcf_rr_read_fcf_rec (struct lpfc_hba *,uint16_t );
Signature · 1 line
2.13.1
Re: [PATCH v2 03/13] lpfc: Add SLI-4 if_type=6 support to the code base
Looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 4/13 lpfc: Add push-to-adapter support to sli4
In reply to James SmartNew if_type=6 adapters support an additional BAR that provides apertures to allow direct WQE to adapter push support - termed Direct Packet Push (DPP). WQ creation differs slightly to ask for a WQ to be DPP-ized. When submitting a WQE to a DPP WQ, it is submitted to the host memory for the WQ normally, but is also written by the host cpu directly to a BAR aperture. Write buffer coalescing in hardware is (hopefully) turned on, enabling single pci write operation support. The doorbell is thing rung to indicate the WQE is available and was pushed to the aperture.
This patch:
- Updates the WQ Create commands for the DPP options
- Adds the bar mapping for if_type=6 DPP bar
- Adds the WQE pushing to the DDP aperture received from WQ create
- Adds a new module parameter to disable DPP operation if desired. Default is enabled.
--- drivers/scsi/lpfc/lpfc.h | 3 +- drivers/scsi/lpfc/lpfc_attr.c | 10 ++ drivers/scsi/lpfc/lpfc_hw4.h | 31 ++++++ drivers/scsi/lpfc/lpfc_init.c | 18 ++++ drivers/scsi/lpfc/lpfc_sli.c | 234 +++++++++++++++++++++++++++--------------- drivers/scsi/lpfc/lpfc_sli4.h | 16 ++- 6 files changed, 225 insertions(+), 87 deletions(-)
drivers/scsi/lpfc/lpfc.h +2 -1
@@ -840,7 +840,8 @@ struct lpfc_hba {uint32_t cfg_enable_SmartSAN;uint32_t cfg_enable_mds_diags;uint32_t cfg_enable_fc4_type; -uint32_t cfg_enable_bbcr;/*Enable BB Credit Recovery*/ +uint32_t cfg_enable_bbcr;/* Enable BB Credit Recovery */ +uint32_t cfg_enable_dpp;/* Enable Direct Packet Push */ uint32_t cfg_xri_split;#define LPFC_ENABLE_FCP 1#define LPFC_ENABLE_NVME 2
drivers/scsi/lpfc/lpfc_attr.c +10 -0
@@ -5204,6 +5204,14 @@ LPFC_ATTR_R(enable_mds_diags, 0, 0, 1, "Enable MDS Diagnostics");*/ LPFC_BBCR_ATTR_RW (enable_bbcr,1 ,0 ,1 ,"Enable BBC Recovery" ); +/* +* lpfc_enable_dpp: Enable DPP on G7 +* 0 = DPP on G7 disabled +* 1 = DPP on G7 enabled (default) +* Value range is [0,1]. Default value is 1. +*/ +LPFC_ATTR_RW (enable_dpp,1 ,0 ,1 ,"Enable Direct Packet Push" ); +struct device_attribute *lpfc_hba_attrs[] = { &dev_attr_nvme_info, &dev_attr_bg_info,
@@ -5312,6 +5320,7 @@ struct device_attribute *lpfc_hba_attrs[] = { &dev_attr_lpfc_xlane_supported, &dev_attr_lpfc_enable_mds_diags, &dev_attr_lpfc_enable_bbcr, + &dev_attr_lpfc_enable_dpp, NULL, };
@@ -6324,6 +6333,7 @@ lpfc_get_cfgparam(struct lpfc_hba *phba)lpfc_fcp_io_channel_init (phba, lpfc_fcp_io_channel);lpfc_nvme_io_channel_init (phba, lpfc_nvme_io_channel);lpfc_enable_bbcr_init (phba, lpfc_enable_bbcr); +lpfc_enable_dpp_init (phba, lpfc_enable_dpp);if (phba->sli_rev !=LPFC_SLI_REV4 ) {/* NVME only supported on SLI4 */
drivers/scsi/lpfc/lpfc_hw4.h +31 -0
@@ -1372,6 +1372,15 @@ struct lpfc_mbx_wq_create {#define lpfc_mbx_wq_create_page_size_MASK 0x000000FF#define lpfc_mbx_wq_create_page_size_WORD word1#define LPFC_WQ_PAGE_SIZE_4096 0x1 +#define lpfc_mbx_wq_create_dpp_req_SHIFT 15 +#define lpfc_mbx_wq_create_dpp_req_MASK 0x00000001 +#define lpfc_mbx_wq_create_dpp_req_WORD word1 +#define lpfc_mbx_wq_create_doe_SHIFT 14 +#define lpfc_mbx_wq_create_doe_MASK 0x00000001 +#define lpfc_mbx_wq_create_doe_WORD word1 +#define lpfc_mbx_wq_create_toe_SHIFT 13 +#define lpfc_mbx_wq_create_toe_MASK 0x00000001 +#define lpfc_mbx_wq_create_toe_WORD word1#define lpfc_mbx_wq_create_wqe_size_SHIFT 8#define lpfc_mbx_wq_create_wqe_size_MASK 0x0000000F#define lpfc_mbx_wq_create_wqe_size_WORD word1
@@ -1400,6 +1409,28 @@ struct lpfc_mbx_wq_create {#define lpfc_mbx_wq_create_db_format_MASK 0x0000FFFF#define lpfc_mbx_wq_create_db_format_WORD word2 } response; +struct { +uint32_t word0; +#define lpfc_mbx_wq_create_dpp_rsp_SHIFT 31 +#define lpfc_mbx_wq_create_dpp_rsp_MASK 0x00000001 +#define lpfc_mbx_wq_create_dpp_rsp_WORD word0 +#define lpfc_mbx_wq_create_v1_q_id_SHIFT 0 +#define lpfc_mbx_wq_create_v1_q_id_MASK 0x0000FFFF +#define lpfc_mbx_wq_create_v1_q_id_WORD word0 +uint32_t word1; +#define lpfc_mbx_wq_create_v1_bar_set_SHIFT 0 +#define lpfc_mbx_wq_create_v1_bar_set_MASK 0x0000000F +#define lpfc_mbx_wq_create_v1_bar_set_WORD word1 +uint32_t doorbell_offset; +uint32_t word3; +#define lpfc_mbx_wq_create_dpp_id_SHIFT 16 +#define lpfc_mbx_wq_create_dpp_id_MASK 0x0000001F +#define lpfc_mbx_wq_create_dpp_id_WORD word3 +#define lpfc_mbx_wq_create_dpp_bar_SHIFT 0 +#define lpfc_mbx_wq_create_dpp_bar_MASK 0x0000000F +#define lpfc_mbx_wq_create_dpp_bar_WORD word3 +uint32_t dpp_offset; + } response_1; } u; };
drivers/scsi/lpfc/lpfc_init.c +18 -0
@@ -9611,6 +9611,24 @@ lpfc_sli4_pci_mem_setup(struct lpfc_hba *phba) } } +if ((if_type ==LPFC_SLI_INTF_IF_TYPE_6 ) && + (pci_resource_start (pdev,PCI_64BIT_BAR4 ))) { +/* +* Map SLI4 if type 6 HBA DPP Register base to a kernel +* virtual address and setup the registers. +*/ + phba->pci_bar2_map =pci_resource_start (pdev,PCI_64BIT_BAR4 ); + bar2map_len =pci_resource_len (pdev,PCI_64BIT_BAR4 ); + phba->sli4_hba.dpp_regs_memmap_p = +ioremap (phba->pci_bar2_map, bar2map_len); +if (!phba->sli4_hba.dpp_regs_memmap_p) { +dev_err (&pdev->dev, +"ioremap failed for SLI4 HBA dpp registers.\n" ); + goto out_iounmap_ctrl; + } + phba->pci_bar4_memmap_p = phba->sli4_hba.dpp_regs_memmap_p; + } +/* Set up the EQ/CQ register handeling functions now */ switch (if_type) {case LPFC_SLI_INTF_IF_TYPE_0 :
drivers/scsi/lpfc/lpfc_sli.c +151 -83
299 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_sli4.h +13 -3
@@ -180,6 +180,10 @@ struct lpfc_queue {uint8_t q_flag;#define HBA_NVMET_WQFULL 0x1/* We hit WQ Full condition for NVMET */ void __iomem *db_regaddr; +uint16_t dpp_enable; +uint16_t dpp_id; +void __iomem *dpp_regaddr; +/* For q stats */ uint32_t q_cnt_1;uint32_t q_cnt_2;
@@ -524,11 +528,17 @@ struct lpfc_vector_map_info {/* SLI4 HBA data structure entries */ struct lpfc_sli4_hba { void __iomem *conf_regs_memmap_p; /* Kernel memory mapped address for - PCI BAR0, config space registers */ + * config space registers + */ void __iomem *ctrl_regs_memmap_p; /* Kernel memory mapped address for - PCI BAR1, control registers */ + * control registers + */ void __iomem *drbl_regs_memmap_p; /* Kernel memory mapped address for - PCI BAR2, doorbell registers */ + * doorbell registers + */ + void __iomem *dpp_regs_memmap_p; /* Kernel memory mapped address for + * dpp registers + */ union { struct { /* IF Type 0, BAR 0 PCI cfg space reg mem map */
Signature · 1 line
2.13.1
Re: [PATCH v2 04/13] lpfc: Add push-to-adapter support to sli4
drivers/scsi/lpfc/lpfc_init.c lpfc_sli4_pci_mem_setup(struct lpfc_hba *phba) · line 9615
Quoted 2 lines
drivers/scsi/lpfc/lpfc_init.c +2
-0+if ((if_type ==LPFC_SLI_INTF_IF_TYPE_6 ) && + (pci_resource_start (pdev,PCI_64BIT_BAR4 ))) {
The above contains a lot of unneeded parenthesis.
[...]
Quoted 6 lines
drivers/scsi/lpfc/lpfc_sli.c +6
-0+/* Enable combined writes for DPP aperture */ + pg_addr = (unsigned long )(wq->dpp_regaddr) &PAGE_MASK ; +#ifdef CONFIG_X86 + rc =set_memory_wc (pg_addr,1 ); +if (rc) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
drivers/scsi/lpfc/lpfc_sli.c lpfc_wq_create(struct lpfc_hba *phba, struct lpfc_queue *wq, · line 15259
Quoted 10 lines
drivers/scsi/lpfc/lpfc_sli.c +10
-0+"3272 Cannot setup Combined " +"Write on WQ[%d] - disable DPP\n" , + wq->queue_id); + phba->cfg_enable_dpp = 0; + } +#else + phba->cfg_enable_dpp =0 ; +#endif + }else + wq->db_regaddr = phba->sli4_hba.WQDBregaddr;
I don't really like the set_memory_wc() call here. Neither do I like the ifdef CONFIG_X86 special casing.
If you really need write combining, can't you at least use ioremap_wc()?
Thanks,
Johannes
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
Re: [PATCH v2 04/13] lpfc: Add push-to-adapter support to sli4
Quoted 20 lines · Johannes Thumshirn
Quoted 6 lines
drivers/scsi/lpfc/lpfc_sli.c +6
-0+/* Enable combined writes for DPP aperture */ + pg_addr = (unsigned long )(wq->dpp_regaddr) &PAGE_MASK ; +#ifdef CONFIG_X86 + rc =set_memory_wc (pg_addr,1 ); +if (rc) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT,drivers/scsi/lpfc/lpfc_sli.c lpfc_wq_create(struct lpfc_hba *phba, struct lpfc_queue *wq, · line 15259
Quoted 10 lines
drivers/scsi/lpfc/lpfc_sli.c +10
-0+"3272 Cannot setup Combined " +"Write on WQ[%d] - disable DPP\n" , + wq->queue_id); + phba->cfg_enable_dpp = 0; + } +#else + phba->cfg_enable_dpp =0 ; +#endif + }else + wq->db_regaddr = phba->sli4_hba.WQDBregaddr;I don't really like the
set_memory_wc()call here. Neither do I like the ifdefCONFIG_X86special casing.
Quoted 1 line
If you really need write combining, can't you at least use
ioremap_wc()?
Coming back to this again (after talking to our ARM/POWER folks internally). Is this really x86 specific here? I know there are servers with other architectures using lpfcs out there.
I _think_ write combining should be possible on other architectures (that have PCIe and aren't dead) as well.
The ioremap_wc() I suggested is probably wrong.
So can you please revisit this? I CCed Mark and Michael, maybe they can help here.
Byte,
Johannes
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
Re: [PATCH v2 04/13] lpfc: Add push-to-adapter support to sli4
Quoted 27 lines · Johannes Thumshirn
Quoted 20 lines · Johannes Thumshirn
Quoted 6 lines
drivers/scsi/lpfc/lpfc_sli.c +6
-0+/* Enable combined writes for DPP aperture */ + pg_addr = (unsigned long )(wq->dpp_regaddr) &PAGE_MASK ; +#ifdef CONFIG_X86 + rc =set_memory_wc (pg_addr,1 ); +if (rc) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT,drivers/scsi/lpfc/lpfc_sli.c lpfc_wq_create(struct lpfc_hba *phba, struct lpfc_queue *wq, · line 15259
Quoted 10 lines
drivers/scsi/lpfc/lpfc_sli.c +10
-0+"3272 Cannot setup Combined " +"Write on WQ[%d] - disable DPP\n" , + wq->queue_id); + phba->cfg_enable_dpp = 0; + } +#else + phba->cfg_enable_dpp =0 ; +#endif + }else + wq->db_regaddr = phba->sli4_hba.WQDBregaddr;I don't really like the
set_memory_wc()call here. Neither do I like the ifdefCONFIG_X86special casing.Quoted 1 line
If you really need write combining, can't you at least use
ioremap_wc()?Coming back to this again (after talking to our ARM/POWER folks internally). Is this really x86 specific here? I know there are servers with other architectures using lpfcs out there.
Quoted 7 lines
I _think_ write combining should be possible on other architectures (that have PCIe and aren't dead) as well.
The
ioremap_wc()I suggested is probably wrong.So can you please revisit this? I CCed Mark and Michael, maybe they can help here.
yes - we really are looking for write combining. We were following the lead of a couple of other entities in the kernel and hadn't found the right combination for something other than X86. We figured we would come back and add the non-86 enablements later when we found the right options.
-- james
Re: [PATCH v2 04/13] lpfc: Add push-to-adapter support to sli4
In reply to Johannes ThumshirnQuoted 27 lines · Johannes Thumshirn
Quoted 20 lines · Johannes Thumshirn
Quoted 6 lines
drivers/scsi/lpfc/lpfc_sli.c +6
-0+/* Enable combined writes for DPP aperture */ + pg_addr = (unsigned long )(wq->dpp_regaddr) &PAGE_MASK ; +#ifdef CONFIG_X86 + rc =set_memory_wc (pg_addr,1 ); +if (rc) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT,drivers/scsi/lpfc/lpfc_sli.c lpfc_wq_create(struct lpfc_hba *phba, struct lpfc_queue *wq, · line 15259
Quoted 10 lines
drivers/scsi/lpfc/lpfc_sli.c +10
-0+"3272 Cannot setup Combined " +"Write on WQ[%d] - disable DPP\n" , + wq->queue_id); + phba->cfg_enable_dpp = 0; + } +#else + phba->cfg_enable_dpp =0 ; +#endif + }else + wq->db_regaddr = phba->sli4_hba.WQDBregaddr;I don't really like the
set_memory_wc()call here. Neither do I like the ifdefCONFIG_X86special casing.Quoted 1 line
If you really need write combining, can't you at least use
ioremap_wc()?Coming back to this again (after talking to our ARM/POWER folks internally). Is this really x86 specific here? I know there are servers with other architectures using lpfcs out there.
Quoted 7 lines
I _think_ write combining should be possible on other architectures (that have PCIe and aren't dead) as well.
The
ioremap_wc()I suggested is probably wrong.So can you please revisit this? I CCed Mark and Michael, maybe they can help here.
I'm not much of an I/O guy, but I do know that on powerpc we don't implement set_memory_wc(). So if you're using that then you do need the ifdef.
I couldn't easily find the rest of this thread, so I'm not sure if ioremap_wc() is an option. We do implement that and on modern CPUs at least it will give you something that's not just a plain uncached mapping.
cheers
Re: [PATCH v2 04/13] lpfc: Add push-to-adapter support to sli4
Quoted 37 lines · Michael Ellerman
Quoted 27 lines · Johannes Thumshirn
Quoted 20 lines · Johannes Thumshirn
Quoted 6 lines
drivers/scsi/lpfc/lpfc_sli.c +6
-0+/* Enable combined writes for DPP aperture */ + pg_addr = (unsigned long )(wq->dpp_regaddr) &PAGE_MASK ; +#ifdef CONFIG_X86 + rc =set_memory_wc (pg_addr,1 ); +if (rc) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT,drivers/scsi/lpfc/lpfc_sli.c lpfc_wq_create(struct lpfc_hba *phba, struct lpfc_queue *wq, · line 15259
Quoted 10 lines
drivers/scsi/lpfc/lpfc_sli.c +10
-0+"3272 Cannot setup Combined " +"Write on WQ[%d] - disable DPP\n" , + wq->queue_id); + phba->cfg_enable_dpp = 0; + } +#else + phba->cfg_enable_dpp =0 ; +#endif + }else + wq->db_regaddr = phba->sli4_hba.WQDBregaddr;I don't really like the
set_memory_wc()call here. Neither do I like the ifdefCONFIG_X86special casing.Quoted 1 line
If you really need write combining, can't you at least use
ioremap_wc()?Coming back to this again (after talking to our ARM/POWER folks internally). Is this really x86 specific here? I know there are servers with other architectures using lpfcs out there.
Quoted 7 lines
I _think_ write combining should be possible on other architectures (that have PCIe and aren't dead) as well.
The
ioremap_wc()I suggested is probably wrong.So can you please revisit this? I CCed Mark and Michael, maybe they can help here.
Quoted 8 lines
I'm not much of an I/O guy, but I do know that on powerpc we don't implement
set_memory_wc(). So if you're using that then you do need the ifdef.I couldn't easily find the rest of this thread, so I'm not sure if
ioremap_wc()is an option. We do implement that and on modern CPUs at least it will give you something that's not just a plain uncached mapping.
I went back and looked at things. It does appear that we should be using ioremap_wc(). There's a pci routine that wrappers it, but as we're already are using the other routines in the wrapper, it's not very interesting. Ioremap_wc seems to be supported pretty much anywhere, with platforms managing what it resolves to. Granted, some platforms may not do write combining but will relax the caching aspects (as Michael indicates).
The interesting thing is - when wc is truly on, we see a substantial difference. But in cases where wc isn't on and we perform the individual writes plus the flush before the doorbell write to synchronize things, it turns out it takes longer than if we don't use the feature. So, in cases where we don't have real wc, I'm going to turn it off. Based on what we've tested so far (includes ppc p8), we'll be leaving it enabled on X86 only.
-- james
PATCH v2 5/13 lpfc: Add PCI Ids for if_type=6 hardware
In reply to James SmartAdd PCI ids for the new G7 adapter
--- drivers/scsi/lpfc/lpfc_hw.h | 1 + drivers/scsi/lpfc/lpfc_ids.h | 2 ++ 2 files changed, 3 insertions(+)
drivers/scsi/lpfc/lpfc_hw.h +1 -0
@@ -1580,6 +1580,7 @@ struct lpfc_fdmi_reg_portattr {#define PCI_DEVICE_ID_LANCER_FCOE 0xe260#define PCI_DEVICE_ID_LANCER_FCOE_VF 0xe268#define PCI_DEVICE_ID_LANCER_G6_FC 0xe300 +#define PCI_DEVICE_ID_LANCER_G7_FC 0xf400#define PCI_DEVICE_ID_SAT_SMB 0xf011#define PCI_DEVICE_ID_SAT_MID 0xf015#define PCI_DEVICE_ID_RFLY 0xf095
drivers/scsi/lpfc/lpfc_ids.h +2 -0
@@ -116,6 +116,8 @@ const struct pci_device_id lpfc_id_table[] = { PCI_ANY_ID, PCI_ANY_ID, }, {PCI_VENDOR_ID_EMULEX, PCI_DEVICE_ID_LANCER_G6_FC, PCI_ANY_ID, PCI_ANY_ID, }, + {PCI_VENDOR_ID_EMULEX, PCI_DEVICE_ID_LANCER_G7_FC, + PCI_ANY_ID, PCI_ANY_ID, }, {PCI_VENDOR_ID_EMULEX, PCI_DEVICE_ID_SKYHAWK, PCI_ANY_ID, PCI_ANY_ID, }, {PCI_VENDOR_ID_EMULEX, PCI_DEVICE_ID_SKYHAWK_VF,
Signature · 1 line
2.13.1
Re: [PATCH v2 05/13] lpfc: Add PCI Ids for if_type=6 hardware
Looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 6/13 lpfc: Add 64G link speed support
In reply to James SmartThe G7 adapter supports 64G link speeds. Add support to the driver.
In addition, a small cleanup to replace the odd bitmap logic with a switch case.
---
v2:
address review comment of typo in define name. Define wasn't used anywhere yet.
--- drivers/scsi/lpfc/lpfc.h | 14 +++------ drivers/scsi/lpfc/lpfc_attr.c | 61 ++++++++++++++++++++++++++++------------ drivers/scsi/lpfc/lpfc_ct.c | 5 ++++ drivers/scsi/lpfc/lpfc_els.c | 5 ++++ drivers/scsi/lpfc/lpfc_hbadisc.c | 1 + drivers/scsi/lpfc/lpfc_hw.h | 12 ++++++++ drivers/scsi/lpfc/lpfc_hw4.h | 3 ++ drivers/scsi/lpfc/lpfc_init.c | 17 +++++++++-- drivers/scsi/lpfc/lpfc_mbox.c | 4 +++ 9 files changed, 92 insertions(+), 30 deletions(-)
drivers/scsi/lpfc/lpfc.h +4 -10
@@ -544,16 +544,10 @@ struct unsol_rcv_ct_ctx {#define LPFC_USER_LINK_SPEED_10G 10/* 10 Gigabaud */ #define LPFC_USER_LINK_SPEED_16G 16/* 16 Gigabaud */ #define LPFC_USER_LINK_SPEED_32G 32/* 32 Gigabaud */ -#define LPFC_USER_LINK_SPEED_MAX LPFC_USER_LINK_SPEED_32G -#define LPFC_USER_LINK_SPEED_BITMAP ((1ULL << LPFC_USER_LINK_SPEED_32G) | \ - (1 << LPFC_USER_LINK_SPEED_16G) | \ - (1 << LPFC_USER_LINK_SPEED_10G) | \ - (1 << LPFC_USER_LINK_SPEED_8G) | \ - (1 << LPFC_USER_LINK_SPEED_4G) | \ - (1 << LPFC_USER_LINK_SPEED_2G) | \ - (1 << LPFC_USER_LINK_SPEED_1G) | \ - (1 << LPFC_USER_LINK_SPEED_AUTO)) -#define LPFC_LINK_SPEED_STRING "0, 1, 2, 4, 8, 10, 16, 32" +#define LPFC_USER_LINK_SPEED_64G 64/* 64 Gigabaud */ +#define LPFC_USER_LINK_SPEED_MAX LPFC_USER_LINK_SPEED_64G + +#define LPFC_LINK_SPEED_STRING "0, 1, 2, 4, 8, 10, 16, 32, 64"enum nemb_type { nemb_mse =1 ,
drivers/scsi/lpfc/lpfc_attr.c +43 -18
109 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_ct.c +5 -0
@@ -2130,6 +2130,8 @@ lpfc_fdmi_port_attr_support_speed(struct lpfc_vport *vport, ae->un.AttrInt =0 ;if (!(phba->hba_flag &HBA_FCOE_MODE )) { +if (phba->lmt & LMT_64Gb) + ae->un.AttrInt |=HBA_PORTSPEED_64GFC ;if (phba->lmt & LMT_32Gb) ae->un.AttrInt |=HBA_PORTSPEED_32GFC ;if (phba->lmt & LMT_16Gb)
@@ -2201,6 +2203,9 @@ lpfc_fdmi_port_attr_speed(struct lpfc_vport *vport,case LPFC_LINK_SPEED_32GHZ : ae->un.AttrInt =HBA_PORTSPEED_32GFC ;break ; +case LPFC_LINK_SPEED_64GHZ : + ae->un.AttrInt =HBA_PORTSPEED_64GFC ; +break ;default : ae->un.AttrInt =HBA_PORTSPEED_UNKNOWN ;break ;
drivers/scsi/lpfc/lpfc_els.c +5 -0
@@ -5270,6 +5270,9 @@ lpfc_rdp_res_speed(struct fc_rdp_port_speed_desc *desc, struct lpfc_hba *phba)case LPFC_LINK_SPEED_32GHZ : rdp_speed =RDP_PS_32GB ;break ; +case LPFC_LINK_SPEED_64GHZ : + rdp_speed =RDP_PS_64GB ; +break ;default : rdp_speed =RDP_PS_UNKNOWN ;break ;
@@ -5277,6 +5280,8 @@ lpfc_rdp_res_speed(struct fc_rdp_port_speed_desc *desc, struct lpfc_hba *phba) desc->info.port_speed.speed =cpu_to_be16 (rdp_speed); +if (phba->lmt & LMT_64Gb) + rdp_cap |=RDP_PS_64GB ;if (phba->lmt & LMT_32Gb) rdp_cap |=RDP_PS_32GB ;if (phba->lmt & LMT_16Gb)
drivers/scsi/lpfc/lpfc_hbadisc.c +1 -0
@@ -3084,6 +3084,7 @@ lpfc_mbx_process_link_up(struct lpfc_hba *phba, struct lpfc_mbx_read_top *la)case LPFC_LINK_SPEED_10GHZ :case LPFC_LINK_SPEED_16GHZ :case LPFC_LINK_SPEED_32GHZ : +case LPFC_LINK_SPEED_64GHZ :break ;default : phba->fc_linkspeed =LPFC_LINK_SPEED_UNKNOWN ;
drivers/scsi/lpfc/lpfc_hw.h +12 -0
@@ -1177,6 +1177,9 @@ struct fc_rdp_link_error_status_desc {#define RDP_PS_8GB 0x0800#define RDP_PS_16GB 0x0400#define RDP_PS_32GB 0x0200 +#define RDP_PS_64GB 0x0100 +#define RDP_PS_128GB 0x0080 +#define RDP_PS_256GB 0x0040#define RDP_CAP_USER_CONFIGURED 0x0002#define RDP_CAP_UNKNOWN 0x0001
@@ -2258,6 +2261,9 @@ typedef struct {#define LINK_SPEED_10G 0x10/* 10 Gigabaud */ #define LINK_SPEED_16G 0x11/* 16 Gigabaud */ #define LINK_SPEED_32G 0x14/* 32 Gigabaud */ +#define LINK_SPEED_64G 0x17/* 64 Gigabaud */ +#define LINK_SPEED_128G 0x1A/* 128 Gigabaud */ +#define LINK_SPEED_256G 0x1D/* 256 Gigabaud */ }INIT_LINK_VAR ;
@@ -2442,6 +2448,9 @@ typedef struct {#define LMT_10Gb 0x100#define LMT_16Gb 0x200#define LMT_32Gb 0x400 +#define LMT_64Gb 0x800 +#define LMT_128Gb 0x1000 +#define LMT_256Gb 0x2000uint32_t rsvd2;uint32_t rsvd3;uint32_t max_xri;
@@ -2966,6 +2975,9 @@ struct lpfc_mbx_read_top {#define LPFC_LINK_SPEED_10GHZ 0x40#define LPFC_LINK_SPEED_16GHZ 0x80#define LPFC_LINK_SPEED_32GHZ 0x90 +#define LPFC_LINK_SPEED_64GHZ 0xA0 +#define LPFC_LINK_SPEED_128GHZ 0xB0 +#define LPFC_LINK_SPEED_256GHZ 0xC0 };/* Structure for MB Command CLEAR_LA (22) */
drivers/scsi/lpfc/lpfc_hw4.h +3 -0
@@ -3961,6 +3961,9 @@ struct lpfc_acqe_fc_la {#define LPFC_FC_LA_SPEED_10G 0xA#define LPFC_FC_LA_SPEED_16G 0x10#define LPFC_FC_LA_SPEED_32G 0x20 +#define LPFC_FC_LA_SPEED_64G 0x21 +#define LPFC_FC_LA_SPEED_128G 0x22 +#define LPFC_FC_LA_SPEED_256G 0x23#define lpfc_acqe_fc_la_topology_SHIFT 16#define lpfc_acqe_fc_la_topology_MASK 0x000000FF#define lpfc_acqe_fc_la_topology_WORD word0
drivers/scsi/lpfc/lpfc_init.c +15 -2
@@ -731,7 +731,9 @@ lpfc_hba_init_link_fc_topology(struct lpfc_hba *phba, uint32_t fc_topology, ((phba->cfg_link_speed == LPFC_USER_LINK_SPEED_16G) && !(phba->lmt & LMT_16Gb)) || ((phba->cfg_link_speed == LPFC_USER_LINK_SPEED_32G) && - !(phba->lmt & LMT_32Gb))) { + !(phba->lmt & LMT_32Gb)) || + ((phba->cfg_link_speed == LPFC_USER_LINK_SPEED_64G) && + !(phba->lmt & LMT_64Gb))) {/* Reset link speed to auto */ lpfc_printf_log (phba,KERN_ERR ,LOG_LINK_EVENT , "1302 Invalid speed for this board:%d "
@@ -2274,7 +2276,9 @@ lpfc_get_hba_model_desc(struct lpfc_hba *phba, uint8_t *mdp, uint8_t *descp) && descp && descp[0] != '\0')return ; -if (phba->lmt & LMT_32Gb) +if (phba->lmt & LMT_64Gb) + max_speed =64 ; +else if (phba->lmt & LMT_32Gb) max_speed =32 ;else if (phba->lmt & LMT_16Gb) max_speed =16 ;
@@ -4109,6 +4113,8 @@ void lpfc_host_attrib_init(struct Scsi_Host *shost) sizeof fc_host_symbolic_name(shost));fc_host_supported_speeds (shost) =0 ; +if (phba->lmt & LMT_64Gb) +fc_host_supported_speeds (shost) |=FC_PORTSPEED_64GBIT ;if (phba->lmt & LMT_32Gb)fc_host_supported_speeds (shost) |=FC_PORTSPEED_32GBIT ;if (phba->lmt & LMT_16Gb)
@@ -4445,6 +4451,9 @@ lpfc_sli4_port_speed_parse(struct lpfc_hba *phba, uint32_t evt_code,case LPFC_FC_LA_SPEED_32G : port_speed =32000 ;break ; +case LPFC_FC_LA_SPEED_64G : + port_speed =64000 ; +break ;default : port_speed =0 ; }
@@ -7811,6 +7820,10 @@ lpfc_sli4_read_config(struct lpfc_hba *phba) phba->cfg_link_speed =LPFC_USER_LINK_SPEED_32G ;break ; +case LINK_SPEED_64G : + phba->cfg_link_speed = +LPFC_USER_LINK_SPEED_64G ; +break ;case 0xffff : phba->cfg_link_speed =LPFC_USER_LINK_SPEED_AUTO ;
drivers/scsi/lpfc/lpfc_mbox.c +4 -0
@@ -557,6 +557,10 @@ lpfc_init_link(struct lpfc_hba * phba, mb->un.varInitLnk.link_flags |=FLAGS_LINK_SPEED ; mb->un.varInitLnk.link_speed =LINK_SPEED_32G ;break ; +case LPFC_USER_LINK_SPEED_64G : + mb->un.varInitLnk.link_flags |=FLAGS_LINK_SPEED ; + mb->un.varInitLnk.link_speed =LINK_SPEED_64G ; +break ;case LPFC_USER_LINK_SPEED_AUTO :default : mb->un.varInitLnk.link_speed =LINK_SPEED_AUTO ;
Signature · 1 line
2.13.1
Re: [PATCH v2 06/13] lpfc: Add 64G link speed support
drivers/scsi/lpfc/lpfc_attr.c lpfc_link_speed_store(struct device *dev, struct device_attribute *attr, · line 4172
Quoted 5 lines · James Smart
drivers/scsi/lpfc/lpfc_attr.c +2
-2lpfc_printf_log (phba,KERN_ERR ,LOG_INIT , -"0469 lpfc_link_speed attribute cannot be set to %d, " -"allowed values are [" LPFC_LINK_SPEED_STRING "]\n" , val); +"0469 lpfc_link_speed attribute cannot be set to %d, " +"allowed values are [" LPFC_LINK_SPEED_STRING "]\n" , val);
lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
"0469 lpfc_link_speed attribute cannot be set to %d, " "allowed values are [%s]\n",
val, LPFC_LINK_SPEED_STRING);
And possible have the whole quoted string on one line for easier grepping. checkplatch.pl should've told you that.
Thanks,
Johannes
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
Re: [PATCH v2 06/13] lpfc: Add 64G link speed support
Quoted 12 lines · Johannes Thumshirn
drivers/scsi/lpfc/lpfc_attr.c lpfc_link_speed_store(struct device *dev, struct device_attribute *attr, · line 4172
Quoted 5 lines · James Smart
lpfc_printf_log(phba,
KERN_ERR,LOG_INIT,drivers/scsi/lpfc/lpfc_attr.c +2
-2-"0469 lpfc_link_speed attribute cannot be set to %d, " - "allowed values are ["LPFC_LINK_SPEED_STRING"]\n", val); +"0469 lpfc_link_speed attribute cannot be set to %d, " + "allowed values are ["LPFC_LINK_SPEED_STRING"]\n", val);lpfc_printf_log(phba,
KERN_ERR,LOG_INIT,
"0469lpfc_link_speedattribute cannot be set to %d, " "allowed values are [%s]\n",
val,LPFC_LINK_SPEED_STRING);
Quoted 3 lines
And possible have the whole quoted string on one line for easier grepping. checkplatch.pl should've told you that.
Checkpatch didn't say anything interesting. We consistently have line spanning as the lines are fairly verbose. Not an easy way to avoid that.
Regardless, no big deal.
-- james
Re: [PATCH v2 06/13] lpfc: Add 64G link speed support
Quoted 13 lines · James Smart
Quoted 12 lines · Johannes Thumshirn
drivers/scsi/lpfc/lpfc_attr.c lpfc_link_speed_store(struct device *dev, struct device_attribute *attr, · line 4172
Quoted 5 lines · James Smart
lpfc_printf_log(phba,
KERN_ERR,LOG_INIT,drivers/scsi/lpfc/lpfc_attr.c +2
-2-"0469 lpfc_link_speed attribute cannot be set to %d, " - "allowed values are ["LPFC_LINK_SPEED_STRING"]\n", val); +"0469 lpfc_link_speed attribute cannot be set to %d, " + "allowed values are ["LPFC_LINK_SPEED_STRING"]\n", val);lpfc_printf_log(phba,
KERN_ERR,LOG_INIT,
"0469lpfc_link_speedattribute cannot be set to %d, " "allowed values are [%s]\n",
val,LPFC_LINK_SPEED_STRING);
Quoted 8 lines
Quoted 3 lines
And possible have the whole quoted string on one line for easier grepping. checkplatch.pl should've told you that.
Checkpatch didn't say anything interesting. We consistently have line spanning as the lines are fairly verbose. Not an easy way to avoid that.
Regardless, no big deal.
The line-breaks make it non-trivial to grep for an error message. No big deal and with lpfc having the numeric prefixes for prints this is actually not needed.
What I mainly didn't like was the implicit pasting of the define instead of %s. But no big deal as well.
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 7/13 lpfc: Add if_type=6 support for cycling valid bits
In reply to James SmartTraditional SLI4 required the driver to clear Valid bits on EQEs and CQEs after consuming them.
The new if_type=6 hardware will cycle the value for what is valid on each queue itteration. The driver no longer has to touch the valid bits. This also means all the cpu cache dirtying and perhaps flush/refill's done by the hardware in accessing the EQ/CQ elements is eliminated.
--- drivers/scsi/lpfc/lpfc_hw4.h | 18 ++++++++-- drivers/scsi/lpfc/lpfc_init.c | 11 +++++++ drivers/scsi/lpfc/lpfc_sli.c | 77 +++++++++++++++++++++++++++++++++++-------- drivers/scsi/lpfc/lpfc_sli4.h | 3 ++ 4 files changed, 92 insertions(+), 17 deletions(-)
drivers/scsi/lpfc/lpfc_hw4.h +15 -3
@@ -1040,6 +1040,9 @@ struct eq_context {#define lpfc_eq_context_valid_SHIFT 29#define lpfc_eq_context_valid_MASK 0x00000001#define lpfc_eq_context_valid_WORD word0 +#define lpfc_eq_context_autovalid_SHIFT 28 +#define lpfc_eq_context_autovalid_MASK 0x00000001 +#define lpfc_eq_context_autovalid_WORD word0uint32_t word1;#define lpfc_eq_context_count_SHIFT 26#define lpfc_eq_context_count_MASK 0x00000003
@@ -1173,6 +1176,9 @@ struct cq_context {#define LPFC_CQ_CNT_512 0x1#define LPFC_CQ_CNT_1024 0x2#define LPFC_CQ_CNT_WORD7 0x3 +#define lpfc_cq_context_autovalid_SHIFT 15 +#define lpfc_cq_context_autovalid_MASK 0x00000001 +#define lpfc_cq_context_autovalid_WORD word0uint32_t word1;#define lpfc_cq_eq_id_SHIFT 22/* Version 0 Only */ #define lpfc_cq_eq_id_MASK 0x000000FF
@@ -1231,9 +1237,9 @@ struct lpfc_mbx_cq_create_set {#define lpfc_mbx_cq_create_set_cqe_size_SHIFT 25#define lpfc_mbx_cq_create_set_cqe_size_MASK 0x00000003#define lpfc_mbx_cq_create_set_cqe_size_WORD word1 -#define lpfc_mbx_cq_create_set_auto_SHIFT 15 -#define lpfc_mbx_cq_create_set_auto_MASK 0x0000001 -#define lpfc_mbx_cq_create_set_auto_WORD word1 +#define lpfc_mbx_cq_create_set_autovalid_SHIFT 15 +#define lpfc_mbx_cq_create_set_autovalid_MASK 0x0000001 +#define lpfc_mbx_cq_create_set_autovalid_WORD word1#define lpfc_mbx_cq_create_set_nodelay_SHIFT 14#define lpfc_mbx_cq_create_set_nodelay_MASK 0x00000001#define lpfc_mbx_cq_create_set_nodelay_WORD word1
@@ -3288,6 +3294,9 @@ struct lpfc_sli4_parameters {#define cfg_sli_hint_2_MASK 0x0000001f#define cfg_sli_hint_2_WORD word1uint32_t word2; +#define cfg_eqav_SHIFT 31 +#define cfg_eqav_MASK 0x00000001 +#define cfg_eqav_WORD word2uint32_t word3;uint32_t word4;#define cfg_cqv_SHIFT 14
@@ -3296,6 +3305,9 @@ struct lpfc_sli4_parameters {#define cfg_cqpsize_SHIFT 16#define cfg_cqpsize_MASK 0x000000ff#define cfg_cqpsize_WORD word4 +#define cfg_cqav_SHIFT 31 +#define cfg_cqav_MASK 0x00000001 +#define cfg_cqav_WORD word4uint32_t word5;uint32_t word6;#define cfg_mqv_SHIFT 14
drivers/scsi/lpfc/lpfc_init.c +11 -0
@@ -8060,6 +8060,7 @@ lpfc_alloc_nvme_wq_cq(struct lpfc_hba *phba, int wqidx) wqidx);return 1 ; } + qdesc->qe_valid =1 ; phba->sli4_hba.nvme_cq[wqidx] = qdesc; qdesc = lpfc_sli4_queue_alloc(phba, LPFC_EXPANDED_PAGE_SIZE,
@@ -8097,6 +8098,7 @@ lpfc_alloc_fcp_wq_cq(struct lpfc_hba *phba, int wqidx) "0499 Failed allocate fast-path FCP CQ (%d)\n", wqidx);return 1 ; } + qdesc->qe_valid =1 ; phba->sli4_hba.fcp_cq[wqidx] = qdesc;/* Create Fast Path FCP WQs */
@@ -8290,6 +8292,7 @@ lpfc_sli4_queue_create(struct lpfc_hba *phba) "0497 Failed allocate EQ (%d)\n", idx); goto out_error; } + qdesc->qe_valid =1 ; phba->sli4_hba.hba_eq[idx] = qdesc; }
@@ -8315,6 +8318,7 @@ lpfc_sli4_queue_create(struct lpfc_hba *phba) "CQ Set (%d)\n", idx); goto out_error; } + qdesc->qe_valid =1 ; phba->sli4_hba.nvmet_cqset[idx] = qdesc; } }
@@ -8332,6 +8336,7 @@ lpfc_sli4_queue_create(struct lpfc_hba *phba) "0500 Failed allocate slow-path mailbox CQ\n"); goto out_error; } + qdesc->qe_valid =1 ; phba->sli4_hba.mbx_cq = qdesc;/* Create slow-path ELS Complete Queue */
@@ -8343,6 +8348,7 @@ lpfc_sli4_queue_create(struct lpfc_hba *phba) "0501 Failed allocate slow-path ELS CQ\n"); goto out_error; } + qdesc->qe_valid =1 ; phba->sli4_hba.els_cq = qdesc;
@@ -8388,6 +8394,7 @@ lpfc_sli4_queue_create(struct lpfc_hba *phba) "6079 Failed allocate NVME LS CQ\n"); goto out_error; } + qdesc->qe_valid =1 ; phba->sli4_hba.nvmels_cq = qdesc;/* Create NVME LS Work Queue */
@@ -10566,6 +10573,8 @@ lpfc_get_sli4_parameters(struct lpfc_hba *phba, LPFC_MBOXQ_t *mboxq) sli4_params->mqv =bf_get (cfg_mqv, mbx_sli4_parameters); sli4_params->wqv =bf_get (cfg_wqv, mbx_sli4_parameters); sli4_params->rqv =bf_get (cfg_rqv, mbx_sli4_parameters); + sli4_params->eqav =bf_get (cfg_eqav, mbx_sli4_parameters); + sli4_params->cqav =bf_get (cfg_cqav, mbx_sli4_parameters); sli4_params->wqsize =bf_get (cfg_wqsize, mbx_sli4_parameters); sli4_params->sgl_pages_max =bf_get (cfg_sgl_page_cnt, mbx_sli4_parameters);
@@ -12384,6 +12393,7 @@ lpfc_fof_queue_create(struct lpfc_hba *phba)if (!qdesc) goto out_error; + qdesc->qe_valid =1 ; phba->sli4_hba.fof_eq = qdesc;if (phba->cfg_fof) {
@@ -12402,6 +12412,7 @@ lpfc_fof_queue_create(struct lpfc_hba *phba)if (!qdesc) goto out_error; + qdesc->qe_valid =1 ; phba->sli4_hba.oas_cq = qdesc;/* Create OAS WQ */
drivers/scsi/lpfc/lpfc_sli.c +63 -14
230 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_sli4.h +3 -0
@@ -216,6 +216,7 @@ struct lpfc_queue {struct work_struct spwork;uint64_t isr_timestamp; +uint8_t qe_valid;struct lpfc_queue *assoc_qp;union sli4_qe qe[1 ];/* array to index entries (must be last) */ };
@@ -486,6 +487,8 @@ struct lpfc_pc_sli4_params {uint8_t mqv;uint8_t wqv;uint8_t rqv; +uint8_t eqav; +uint8_t cqav;uint8_t wqsize;#define LPFC_WQ_SZ64_SUPPORT 1#define LPFC_WQ_SZ128_SUPPORT 2
Signature · 1 line
2.13.1
Re: [PATCH v2 07/13] lpfc: Add if_type=6 support for cycling valid bits
Looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 8/13 lpfc: Enable fw download on if_type=6 devices
In reply to James SmartCurrent code is very explicit in what it allows to be downloaded. The driver checking prevented G7 firmware download. The driver checking is unnecessary as the device will validate what it receives.
Revise the firmware download interface checking. Added a little debug support in case there is still a failure.
--- drivers/scsi/lpfc/lpfc_hw4.h | 5 +---- drivers/scsi/lpfc/lpfc_init.c | 44 ++++++++++++++++++++++++++++++------------- drivers/scsi/lpfc/lpfc_sli.c | 1 + 3 files changed, 33 insertions(+), 17 deletions(-)
drivers/scsi/lpfc/lpfc_hw4.h +1 -4
@@ -2241,6 +2241,7 @@ struct lpfc_mbx_redisc_fcf_tbl {* command. */ #define ADD_STATUS_OPERATION_ALREADY_ACTIVE 0x67 +#define ADD_STATUS_FW_NOT_SUPPORTED 0xEBstruct lpfc_mbx_sli4_config {struct mbox_header header;
@@ -4603,10 +4604,6 @@ union lpfc_wqe128 {struct gen_req64_wqe gen_req; }; -#define LPFC_GROUP_OJECT_MAGIC_G5 0xfeaa0001 -#define LPFC_GROUP_OJECT_MAGIC_G6 0xfeaa0003 -#define LPFC_FILE_TYPE_GROUP 0xf7 -#define LPFC_FILE_ID_GROUP 0xa2struct lpfc_grp_hdr {uint32_t size;uint32_t magic_number;
drivers/scsi/lpfc/lpfc_init.c +31 -13
68 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_sli.c +1 -0
@@ -18876,6 +18876,7 @@ lpfc_wr_object(struct lpfc_hba *phba, struct list_head *dmabuf_list,"status x%x add_status x%x, mbx status x%x\n" , shdr_status, shdr_add_status, rc); rc = -ENXIO ; + *offset = shdr_add_status; }else *offset += wr_object->u.response.actual_write_length;return rc;
Signature · 1 line
2.13.1
Re: [PATCH v2 08/13] lpfc: Enable fw download on if_type=6 devices
Looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 9/13 lpfc: Add embedded data pointers for enhanced performance
In reply to James SmartThe current driver isn't taking advantage of a performance hint whereby the initial data buffer descriptor can be placed in the WQE as well as the SGL.
Add the logic to detect support for the feature and to use it when supported.
--- drivers/scsi/lpfc/lpfc.h | 2 ++ drivers/scsi/lpfc/lpfc_hw4.h | 3 +++ drivers/scsi/lpfc/lpfc_init.c | 21 +++++++++++++++++++++ drivers/scsi/lpfc/lpfc_nvme.c | 18 ++++++++++++++++++ drivers/scsi/lpfc/lpfc_nvmet.c | 24 ++++++++++++++++++++++++ drivers/scsi/lpfc/lpfc_scsi.c | 8 ++++++-- drivers/scsi/lpfc/lpfc_sli.c | 25 +++++++++++++++++++++---- 7 files changed, 95 insertions(+), 6 deletions(-)
drivers/scsi/lpfc/lpfc.h +2 -0
@@ -840,6 +840,8 @@ struct lpfc_hba {#define LPFC_ENABLE_FCP 1#define LPFC_ENABLE_NVME 2#define LPFC_ENABLE_BOTH 3 +uint32_t nvme_embed_pbde; +uint32_t fcp_embed_pbde;uint32_t io_channel_irqs;/* number of irqs for io channels */ struct nvmet_fc_target_port *targetport;lpfc_vpd_t vpd;/* vital product data */
drivers/scsi/lpfc/lpfc_hw4.h +3 -0
@@ -4226,6 +4226,9 @@ struct wqe_common {#define wqe_irsp_SHIFT 4#define wqe_irsp_MASK 0x00000001#define wqe_irsp_WORD word11 +#define wqe_pbde_SHIFT 5 +#define wqe_pbde_MASK 0x00000001 +#define wqe_pbde_WORD word11#define wqe_sup_SHIFT 6#define wqe_sup_MASK 0x00000001#define wqe_sup_WORD word11
drivers/scsi/lpfc/lpfc_init.c +21 -0
@@ -10605,6 +10605,19 @@ lpfc_get_sli4_parameters(struct lpfc_hba *phba, LPFC_MBOXQ_t *mboxq) phba->cfg_enable_fc4_type =LPFC_ENABLE_FCP ; } +/* Only embed PBDE for if_type 6 */ +if (bf_get (lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) == +LPFC_SLI_INTF_IF_TYPE_6 ) { + phba->fcp_embed_pbde =1 ; + phba->nvme_embed_pbde =1 ; + } + +/* PBDE support requires xib be set */ +if (!bf_get (cfg_xib, mbx_sli4_parameters)) { + phba->fcp_embed_pbde =0 ; + phba->nvme_embed_pbde =0 ; + } +/* * To support Suppress Response feature we must satisfy 3 conditions. * lpfc_suppress_rsp module parameter must be set (default).
@@ -10636,6 +10649,14 @@ lpfc_get_sli4_parameters(struct lpfc_hba *phba, LPFC_MBOXQ_t *mboxq)else phba->fcp_embed_io =0 ; +lpfc_printf_log (phba,KERN_INFO ,LOG_INIT |LOG_NVME , +"6422 XIB %d: FCP %d %d " +"NVME %d %d %d\n" , +bf_get (cfg_xib, mbx_sli4_parameters), + phba->fcp_embed_pbde, phba->fcp_embed_io, + phba->nvme_support, phba->nvme_embed_pbde, + phba->cfg_suppress_rsp); +if ((bf_get (cfg_cqpsize, mbx_sli4_parameters) &LPFC_CQ_16K_PAGE_SZ ) && (bf_get (cfg_wqpsize, mbx_sli4_parameters) &LPFC_WQ_16K_PAGE_SZ ) && (sli4_params->wqsize &LPFC_WQ_SZ128_SUPPORT ))
drivers/scsi/lpfc/lpfc_nvme.c +18 -0
@@ -1170,6 +1170,7 @@ lpfc_nvme_prep_io_dma(struct lpfc_vport *vport,struct sli4_sge *sgl = lpfc_ncmd->nvme_sgl;struct scatterlist *data_sg;struct sli4_sge *first_data_sgl; +struct ulp_bde64 *bde;dma_addr_t physaddr;uint32_t num_bde =0 ;uint32_t dma_len;
@@ -1237,7 +1238,24 @@ lpfc_nvme_prep_io_dma(struct lpfc_vport *vport, data_sg =sg_next (data_sg); sgl++; } +if (phba->nvme_embed_pbde) { +/* Use PBDE support for first SGL only, offset == 0 */ +/* Words 13-15 */ + bde = (struct ulp_bde64 *) + &wqe->words[13 ]; + bde->addrLow = first_data_sgl->addr_lo; + bde->addrHigh = first_data_sgl->addr_hi; + bde->tus.f.bdeSize = +le32_to_cpu (first_data_sgl->sge_len); + bde->tus.f.bdeFlags =BUFF_TYPE_BDE_64 ; + bde->tus.w =cpu_to_le32 (bde->tus.w); +bf_set (wqe_pbde, &wqe->generic.wqe_com,1 ); + }else +bf_set (wqe_pbde, &wqe->generic.wqe_com,0 ); + }else { +bf_set (wqe_pbde, &wqe->generic.wqe_com,0 ); +/* For this clause to be valid, the payload_length * and sg_cnt must zero. */
drivers/scsi/lpfc/lpfc_nvmet.c +24 -0
@@ -2150,9 +2150,11 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,struct lpfc_iocbq *nvmewqe;struct scatterlist *sgel;union lpfc_wqe128 *wqe; +struct ulp_bde64 *bde;uint32_t *txrdy;dma_addr_t physaddr;int i, cnt; +int do_pbde;int xc =1 ;if (!lpfc_is_link_up (phba)) {
@@ -2243,6 +2245,7 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,/* Word 7 */ bf_set (wqe_pu, &wqe->fcp_tsend.wqe_com,1 );bf_set (wqe_cmnd, &wqe->fcp_tsend.wqe_com,CMD_FCP_TSEND64_WQE ); + do_pbde =0 ;/* Word 8 */ wqe->fcp_tsend.wqe_com.abort_tag = nvmewqe->iotag;
@@ -2355,6 +2358,10 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,bf_set (wqe_ar, &wqe->fcp_treceive.wqe_com,0 );bf_set (wqe_cmnd, &wqe->fcp_treceive.wqe_com,CMD_FCP_TRECEIVE64_WQE ); +if (phba->nvme_embed_pbde) + do_pbde =1 ; +else + do_pbde =0 ;/* Word 8 */ wqe->fcp_treceive.wqe_com.abort_tag = nvmewqe->iotag;
@@ -2438,6 +2445,7 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,bf_set (wqe_pu, &wqe->fcp_trsp.wqe_com,0 );bf_set (wqe_ag, &wqe->fcp_trsp.wqe_com,1 );bf_set (wqe_cmnd, &wqe->fcp_trsp.wqe_com,CMD_FCP_TRSP64_WQE ); + do_pbde =0 ;/* Word 8 */ wqe->fcp_trsp.wqe_com.abort_tag = nvmewqe->iotag;
@@ -2508,9 +2516,25 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,bf_set (lpfc_sli4_sge_last, sgl,1 ); sgl->word2 =cpu_to_le32 (sgl->word2); sgl->sge_len =cpu_to_le32 (cnt); +if (do_pbde && (i ==0 )) { + bde = (struct ulp_bde64 *)&wqe->words[13 ]; +memset (bde,0 ,sizeof (struct ulp_bde64 )); +/* Words 13-15 (PBDE)*/ + bde->addrLow = sgl->addr_lo; + bde->addrHigh = sgl->addr_hi; + bde->tus.f.bdeSize = +le32_to_cpu (sgl->sge_len); + bde->tus.f.bdeFlags =BUFF_TYPE_BDE_64 ; + bde->tus.w =cpu_to_le32 (bde->tus.w); + } sgl++; ctxp->offset += cnt; } + +if (do_pbde) +bf_set (wqe_pbde, &wqe->generic.wqe_com,1 ); +else +bf_set (wqe_pbde, &wqe->generic.wqe_com,0 ); ctxp->state =LPFC_NVMET_STE_DATA ; ctxp->entry_cnt++;return nvmewqe;
drivers/scsi/lpfc/lpfc_scsi.c +6 -2
@@ -3304,8 +3304,12 @@ lpfc_scsi_prep_dma_buf_s4(struct lpfc_hba *phba, struct lpfc_scsi_buf *lpfc_cmd) dma_offset += dma_len; sgl++; } -/* setup the performance hint (first data BDE) if enabled */ -if (phba->sli3_options &LPFC_SLI4_PERFH_ENABLED ) { +/* +* Setup the first Payload BDE. For FCoE we just key off +* Performance Hints, for FC we utilize fcp_embed_pbde. +*/ +if ((phba->sli3_options &LPFC_SLI4_PERFH_ENABLED ) || + phba->fcp_embed_pbde) { bde = (struct ulp_bde64 *) &(iocb_cmd->unsli3.sli3Words[5 ]); bde->addrLow = first_data_sgl->addr_lo;
drivers/scsi/lpfc/lpfc_sli.c +21 -4
@@ -6963,10 +6963,15 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba) "0378 No support for fcpi mode.\n"); ftr_rsp++; } -if (bf_get (lpfc_mbx_rq_ftr_rsp_perfh, &mqe->un.req_ftrs)) - phba->sli3_options |=LPFC_SLI4_PERFH_ENABLED ; -else - phba->sli3_options &= ~LPFC_SLI4_PERFH_ENABLED ; + +/* Performance Hints are ONLY for FCoE */ +if (phba->hba_flag &HBA_FCOE_MODE ) { +if (bf_get (lpfc_mbx_rq_ftr_rsp_perfh, &mqe->un.req_ftrs)) + phba->sli3_options |=LPFC_SLI4_PERFH_ENABLED ; +else + phba->sli3_options &= ~LPFC_SLI4_PERFH_ENABLED ; + } +/* * If the port cannot support the host's requested features * then turn off the global config parameters to disable the
@@ -9068,6 +9073,12 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq, }/* Note, word 10 is already initialized to 0 */ +/* Don't set PBDE for Perf hints, just fcp_embed_pbde */ +if (phba->fcp_embed_pbde) +bf_set (wqe_pbde, &wqe->fcp_iwrite.wqe_com,1 ); +else +bf_set (wqe_pbde, &wqe->fcp_iwrite.wqe_com,0 ); +if (phba->fcp_embed_io) {struct lpfc_scsi_buf *lpfc_cmd;struct sli4_sge *sgl;
@@ -9127,6 +9138,12 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq, }/* Note, word 10 is already initialized to 0 */ +/* Don't set PBDE for Perf hints, just fcp_embed_pbde */ +if (phba->fcp_embed_pbde) +bf_set (wqe_pbde, &wqe->fcp_iread.wqe_com,1 ); +else +bf_set (wqe_pbde, &wqe->fcp_iread.wqe_com,0 ); +if (phba->fcp_embed_io) {struct lpfc_scsi_buf *lpfc_cmd;struct sli4_sge *sgl;
Signature · 1 line
2.13.1
Re: [PATCH v2 09/13] lpfc: Add embedded data pointers for enhanced performance
drivers/scsi/lpfc/lpfc_nvmet.c lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, · line 2519
Quoted 1 line · James Smart
drivers/scsi/lpfc/lpfc_nvmet.c +1
-0+if (do_pbde && (i ==0 )) {
Nit:
if (do_pde && i == 0) {Else:
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 10/13 lpfc: Fix nvme embedded io length on new hardware
In reply to James SmartNewer hardware more strictly enforces buffer lenghts, causing an mis-set value to be identified. Older hardware won't catch it. The difference is benign on old hardware.
Set the right embedded buffer length for nvme ios.
--- drivers/scsi/lpfc/lpfc_nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/lpfc/lpfc_nvme.c +1 -1
@@ -655,7 +655,7 @@ lpfc_nvme_adj_fcp_sgls(struct lpfc_vport *vport,/* Word 0-2 - NVME CMND IU (embedded payload) */ wqe->generic.bde.tus.f.bdeFlags =BUFF_TYPE_BDE_IMMED ; - wqe->generic.bde.tus.f.bdeSize =60 ; + wqe->generic.bde.tus.f.bdeSize =56 ; wqe->generic.bde.addrHigh =0 ; wqe->generic.bde.addrLow =64 ;/* Word 16 */
Signature · 1 line
2.13.1
Re: [PATCH v2 10/13] lpfc: Fix nvme embedded io length on new hardware
Looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 11/13 lpfc: Work around NVME cmd iu SGL type
In reply to James SmartThe hardware offload for NVME commands was created when the FC-NVME standard was setting SGL Descriptor Type to SGL Data Block Descriptor (0h) and SGL Descriptor Sub Type to Address (0h).
A late change in NVMe-over-Fabrics obsoleted these values, creating a transport SGL descriptor type with new values to go into these fields.
For initial hardware support, in order to be compliant to the spec, use host-supplied cmd IU buffers instead of the adapter generated values. Later hardware will correct this.
Add a module parameter to override this offload disablement if looking for lowest latency. This is reasonable as nothing in FC-NVME uses the SQE SGL values.
--- drivers/scsi/lpfc/lpfc.h | 1 + drivers/scsi/lpfc/lpfc_attr.c | 14 ++++++++++++ drivers/scsi/lpfc/lpfc_hw4.h | 1 + drivers/scsi/lpfc/lpfc_init.c | 4 ++-- drivers/scsi/lpfc/lpfc_nvme.c | 51 ++++++++++++++++++++++++++++++------------- drivers/scsi/lpfc/lpfc_sli.c | 15 +++++++++++++ 6 files changed, 69 insertions(+), 17 deletions(-)
drivers/scsi/lpfc/lpfc.h +1 -0
@@ -782,6 +782,7 @@ struct lpfc_hba {uint32_t cfg_fcp_io_channel;uint32_t cfg_suppress_rsp;uint32_t cfg_nvme_oas; +uint32_t cfg_nvme_embed_cmd;uint32_t cfg_nvme_io_channel;uint32_t cfg_nvmet_mrq;uint32_t cfg_enable_nvmet;
drivers/scsi/lpfc/lpfc_attr.c +14 -0
@@ -5059,6 +5059,18 @@ LPFC_ATTR_RW(nvme_oas, 0, 0, 1, "Use OAS bit on NVME IOs");/* +* lpfc_nvme_embed_cmd: Use the oas bit when sending NVME/NVMET IOs +* +* 0 = Put NVME Command in SGL +* 1 = Embed NVME Command in WQE (unless G7) +* 2 = Embed NVME Command in WQE (force) +* +* Value range is [0,2]. Default value is 1. +*/ +LPFC_ATTR_RW (nvme_embed_cmd,1 ,0 ,2 , +"Embed NVME Command in WQE" ); + +/* * lpfc_fcp_io_channel: Set the number of FCP IO channels the driver * will advertise it supports to the SCSI layer. This also will map to * the number of WQs the driver will create.
@@ -5299,6 +5311,7 @@ struct device_attribute *lpfc_hba_attrs[] = { &dev_attr_lpfc_task_mgmt_tmo, &dev_attr_lpfc_use_msi, &dev_attr_lpfc_nvme_oas, + &dev_attr_lpfc_nvme_embed_cmd, &dev_attr_lpfc_auto_imax, &dev_attr_lpfc_fcp_imax, &dev_attr_lpfc_fcp_cpu_map,
@@ -6323,6 +6336,7 @@ lpfc_get_cfgparam(struct lpfc_hba *phba)lpfc_enable_SmartSAN_init (phba, lpfc_enable_SmartSAN);lpfc_use_msi_init (phba, lpfc_use_msi);lpfc_nvme_oas_init (phba, lpfc_nvme_oas); +lpfc_nvme_embed_cmd_init (phba, lpfc_nvme_embed_cmd);lpfc_auto_imax_init (phba, lpfc_auto_imax);lpfc_fcp_imax_init (phba, lpfc_fcp_imax);lpfc_fcp_cpu_map_init (phba, lpfc_fcp_cpu_map);
drivers/scsi/lpfc/lpfc_hw4.h +1 -0
@@ -2678,6 +2678,7 @@ struct lpfc_mbx_read_rev {#define lpfc_mbx_rd_rev_vpd_MASK 0x00000001#define lpfc_mbx_rd_rev_vpd_WORD word1uint32_t first_hw_rev; +#define LPFC_G7_ASIC_1 0xduint32_t second_hw_rev;uint32_t word4_rsvd;uint32_t third_hw_rev;
drivers/scsi/lpfc/lpfc_init.c +2 -2
@@ -10651,11 +10651,11 @@ lpfc_get_sli4_parameters(struct lpfc_hba *phba, LPFC_MBOXQ_t *mboxq)lpfc_printf_log (phba,KERN_INFO ,LOG_INIT |LOG_NVME ,"6422 XIB %d: FCP %d %d " -"NVME %d %d %d\n" , +"NVME %d %d %d %d\n" ,bf_get (cfg_xib, mbx_sli4_parameters), phba->fcp_embed_pbde, phba->fcp_embed_io, phba->nvme_support, phba->nvme_embed_pbde, - phba->cfg_suppress_rsp); + phba->cfg_nvme_embed_cmd, phba->cfg_suppress_rsp); if ((bf_get(cfg_cqpsize, mbx_sli4_parameters) & LPFC_CQ_16K_PAGE_SZ) && (bf_get(cfg_wqpsize, mbx_sli4_parameters) & LPFC_WQ_16K_PAGE_SZ) &&
drivers/scsi/lpfc/lpfc_nvme.c +36 -15
80 lines not on this page · raw message
drivers/scsi/lpfc/lpfc_sli.c +15 -0
@@ -6885,6 +6885,18 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba)/* Save information as VPD data */ phba->vpd.rev.biuRev = mqe->un.read_rev.first_hw_rev; phba->vpd.rev.smRev = mqe->un.read_rev.second_hw_rev; + +/* +* This is because first G7 ASIC doesn't support the standard +* 0x5a NVME cmd descriptor type/subtype +*/ +if ((bf_get (lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) == +LPFC_SLI_INTF_IF_TYPE_6 ) && + (phba->vpd.rev.biuRev ==LPFC_G7_ASIC_1 ) && + (phba->vpd.rev.smRev ==0 ) && + (phba->cfg_nvme_embed_cmd ==1 )) + phba->cfg_nvme_embed_cmd =0 ; + phba->vpd.rev.endecRev = mqe->un.read_rev.third_hw_rev; phba->vpd.rev.fcphHigh =bf_get (lpfc_mbx_rd_rev_fcph_high, &mqe->un.read_rev);
@@ -9101,6 +9113,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq, wqe128->generic.bde.addrLow =88 ;/* Word 22 */ bf_set (wqe_wqes, &wqe128->fcp_iwrite.wqe_com,1 ); +bf_set (wqe_dbde, &wqe128->fcp_iwrite.wqe_com,0 );/* Word 22-29 FCP CMND Payload */ ptr = &wqe128->words[22 ];
@@ -9166,6 +9179,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq, wqe128->generic.bde.addrLow =88 ;/* Word 22 */ bf_set (wqe_wqes, &wqe128->fcp_iread.wqe_com,1 ); +bf_set (wqe_dbde, &wqe128->fcp_iread.wqe_com,0 );/* Word 22-29 FCP CMND Payload */ ptr = &wqe128->words[22 ];
@@ -9224,6 +9238,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq, wqe128->generic.bde.addrLow =88 ;/* Word 22 */ bf_set (wqe_wqes, &wqe128->fcp_icmd.wqe_com,1 ); +bf_set (wqe_dbde, &wqe128->fcp_icmd.wqe_com,0 );/* Word 22-29 FCP CMND Payload */ ptr = &wqe128->words[22 ];
Signature · 1 line
2.13.1
Re: [PATCH v2 11/13] lpfc: Work around NVME cmd iu SGL type
Looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 12/13 lpfc: update driver version to 12.0.0.0
In reply to James SmartUpdate the driver version to 12.0.0.0
--- drivers/scsi/lpfc/lpfc_version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/lpfc/lpfc_version.h +1 -1
@@ -20,7 +20,7 @@* included with this package. * *******************************************************************/ -#define LPFC_DRIVER_VERSION "11.4.0.7" +#define LPFC_DRIVER_VERSION "12.0.0.0"#define LPFC_DRIVER_NAME "lpfc"/* Used for SLI 2/3 */
Signature · 1 line
2.13.1
Re: [PATCH v2 12/13] lpfc: update driver version to 12.0.0.0
Looks good,
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
PATCH v2 13/13 lpfc: Update 12.0.0.0 modified files for 2018 Copyright
In reply to James SmartUpdated Copyright in files updated 12.0.0.0
--- drivers/scsi/lpfc/lpfc_bsg.c | 2 +- drivers/scsi/lpfc/lpfc_ct.c | 2 +- drivers/scsi/lpfc/lpfc_debugfs.c | 2 +- drivers/scsi/lpfc/lpfc_debugfs.h | 2 +- drivers/scsi/lpfc/lpfc_hw.h | 2 +- drivers/scsi/lpfc/lpfc_ids.h | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-)
drivers/scsi/lpfc/lpfc_bsg.c +1 -1
@@ -1,7 +1,7 @@/******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * -* Copyright (C) 2017 Broadcom. All Rights Reserved. The term * +* Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Limited and/or its subsidiaries. * * Copyright (C) 2009-2015 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. *
drivers/scsi/lpfc/lpfc_ct.c +1 -1
@@ -1,7 +1,7 @@/******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * -* Copyright (C) 2017 Broadcom. All Rights Reserved. The term * +* Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Limited and/or its subsidiaries. * * Copyright (C) 2004-2016 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. *
drivers/scsi/lpfc/lpfc_debugfs.c +1 -1
@@ -1,7 +1,7 @@/******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * -* Copyright (C) 2017 Broadcom. All Rights Reserved. The term * +* Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Limited and/or its subsidiaries. * * Copyright (C) 2007-2015 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. *
drivers/scsi/lpfc/lpfc_debugfs.h +1 -1
@@ -1,7 +1,7 @@/******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * -* Copyright (C) 2017 Broadcom. All Rights Reserved. The term * +* Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Limited and/or its subsidiaries. * * Copyright (C) 2007-2011 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. *
drivers/scsi/lpfc/lpfc_hw.h +1 -1
@@ -1,7 +1,7 @@/******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * -* Copyright (C) 2017 Broadcom. All Rights Reserved. The term * +* Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Limited and/or its subsidiaries. * * Copyright (C) 2004-2016 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. *
drivers/scsi/lpfc/lpfc_ids.h +1 -1
@@ -1,7 +1,7 @@/******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * -* Copyright (C) 2017 Broadcom. All Rights Reserved. The term * +* Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Limited and/or its subsidiaries. * * Copyright (C) 2004-2016 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. *
Signature · 1 line
2.13.1
Re: [PATCH v2 13/13] lpfc: Update 12.0.0.0 modified files for 2018 Copyright
Looks good,
Although I'd change the subject to:
"lpfc: Change Copyright of Update 12.0.0.0 modified files to 2018"
Reads easier for the non-native english speaker (like me).
Signature · 6 lines
Johannes Thumshirn Storage [email protected] +49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850