Skip to contentSkip to conversation

lpfc: Update lpfc to revision 12.8.0.2

PATCH Applied James Smart 2 participants 18 messages Jun 30, 2020 – Jul 3, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover lpfc: Update lpfc to revision 12.8.0.2

Update lpfc to revision 12.8.0.2

Patch set contains fixes as a couple of small additions, and two larger additions - blk_io_poll support and a different log message facility.

The patches were cut against Martin's 5.9/scsi-queue tree

James Smart (14):
lpfc: Fix unused assignment in lpfc_sli4_bsg_link_diag_test
lpfc: Fix missing MDS functionality
lpfc: Fix first-burst driver implementation.
lpfc: Fix NVME rport deregister and registration during ADISC
lpfc: Fix oops due to overrun when reading SLI3 data
lpfc: Fix stack trace seen while setting rrq active
lpfc: Fix shost refcount mismatch when deleting vport
lpfc: Fix kdump hang on PPC
lpfc: Fix language in 0373 message to reflect non-error message
lpfc: Allow applications to issue Common Set Features mailbox command
lpfc: Add support to display if adapter dumps are available
lpfc: Add blk_io_poll support for latency improvment
lpfc: Add an internal trace log buffer
lpfc: Update lpfc version to 12.8.0.2

 drivers/scsi/lpfc/lpfc.h           |  15 +
 drivers/scsi/lpfc/lpfc_bsg.c       |  35 +-
 drivers/scsi/lpfc/lpfc_bsg.h       |  14 +
 drivers/scsi/lpfc/lpfc_crtn.h      |   6 +-
 drivers/scsi/lpfc/lpfc_ct.c        |  16 +-
 drivers/scsi/lpfc/lpfc_disc.h      |   1 +
 drivers/scsi/lpfc/lpfc_els.c       | 135 +++---
 drivers/scsi/lpfc/lpfc_hbadisc.c   | 220 +++++-----
 drivers/scsi/lpfc/lpfc_hw4.h       |   5 +-
 drivers/scsi/lpfc/lpfc_init.c      | 667 ++++++++++++++++++-----------
 drivers/scsi/lpfc/lpfc_logmsg.h    |  24 +-
 drivers/scsi/lpfc/lpfc_mem.c       |   3 +-
 drivers/scsi/lpfc/lpfc_nportdisc.c |  83 ++--
 drivers/scsi/lpfc/lpfc_nvme.c      |  67 ++-
 drivers/scsi/lpfc/lpfc_nvmet.c     |  96 ++---
 drivers/scsi/lpfc/lpfc_scsi.c      | 421 +++++++++++++++---
 drivers/scsi/lpfc/lpfc_scsi.h      |  24 ++
 drivers/scsi/lpfc/lpfc_sli.c       | 587 ++++++++++++++++---------
 drivers/scsi/lpfc/lpfc_sli.h       |   1 +
 drivers/scsi/lpfc/lpfc_sli4.h      |  18 +
 drivers/scsi/lpfc/lpfc_version.h   |   2 +-
 drivers/scsi/lpfc/lpfc_vport.c     |  86 ++--
 22 files changed, 1636 insertions(+), 890 deletions(-)
Signature · 1 line
2.25.0
#1 lore raw

PATCH 1/14 lpfc: Fix unused assignment in lpfc_sli4_bsg_link_diag_test

Coverity reported the following error:
Assigned value that is never used may represent unnecessary computation.

The rc variable was initially assigned a value but in several cases, when an error case is detected, it is reassigned a new value. The initial value had little use.

In code-reviewing this routine, it could use some cleanup:
- Setting the initialization value to -ENODEV is a much better choice and lessens code in the routine.
- The wasn't tracking logic errors vs no error and mailbox failure. Better to resolve by adding a status to track the mailbox failure and merge it with the logic error when the routine returns.

---
 drivers/scsi/lpfc/lpfc_bsg.c | 34 ++++++++++++++++------------------
 1 file changed, 16 insertions(+), 18 deletions(-)
drivers/scsi/lpfc/lpfc_bsg.c +16 -18
index 0ea03ae93d91..e91466aa1673 100644
@@ -2404,33 +2404,27 @@ lpfc_sli4_bsg_link_diag_test(struct bsg_job *job)
 	union lpfc_sli4_cfg_shdr *shdr;
 	uint32_t shdr_status, shdr_add_status;
 	struct diag_status *diag_status_reply;
-	int mbxstatus, rc = 0;
+	int mbxstatus, rc = -ENODEV, rc1 = 0;
 
 	shost = fc_bsg_to_shost(job);
-	if (!shost) {
-		rc = -ENODEV;
+	if (!shost)
 		goto job_error;
-	}
+
 	vport = shost_priv(shost);
-	if (!vport) {
-		rc = -ENODEV;
+	if (!vport)
 		goto job_error;
-	}
+
 	phba = vport->phba;
-	if (!phba) {
-		rc = -ENODEV;
+	if (!phba)
 		goto job_error;
-	}
 
-	if (phba->sli_rev < LPFC_SLI_REV4) {
-		rc = -ENODEV;
+
+	if (phba->sli_rev < LPFC_SLI_REV4)
 		goto job_error;
-	}
+
 	if (bf_get(lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf) <
-	    LPFC_SLI_INTF_IF_TYPE_2) {
-		rc = -ENODEV;
+	    LPFC_SLI_INTF_IF_TYPE_2)
 		goto job_error;
-	}
 
 	if (job->request_len < sizeof(struct fc_bsg_request) +
 	    sizeof(struct sli4_link_diag)) {
@@ -2465,8 +2459,10 @@ lpfc_sli4_bsg_link_diag_test(struct bsg_job *job)
 	alloc_len = lpfc_sli4_config(phba, pmboxq, LPFC_MBOX_SUBSYSTEM_FCOE,
 				     LPFC_MBOX_OPCODE_FCOE_LINK_DIAG_STATE,
 				     req_len, LPFC_SLI4_MBX_EMBED);
-	if (alloc_len != req_len)
+	if (alloc_len != req_len) {
+		rc = -ENOMEM;
 		goto link_diag_test_exit;
+	}
 
 	run_link_diag_test = &pmboxq->u.mqe.un.link_diag_test;
 	bf_set(lpfc_mbx_run_diag_test_link_num, &run_link_diag_test->u.req,
@@ -2515,7 +2511,7 @@ lpfc_sli4_bsg_link_diag_test(struct bsg_job *job)
 	diag_status_reply->shdr_add_status = shdr_add_status;
 
 link_diag_test_exit:
-	rc = lpfc_sli4_bsg_set_link_diag_state(phba, 0);
+	rc1 = lpfc_sli4_bsg_set_link_diag_state(phba, 0);
 
 	if (pmboxq)
 		mempool_free(pmboxq, phba->mbox_mem_pool);
@@ -2524,6 +2520,8 @@ lpfc_sli4_bsg_link_diag_test(struct bsg_job *job)
 
 job_error:
 	/* make error code available to userspace */
+	if (rc1 && !rc)
+		rc = rc1;
 	bsg_reply->result = rc;
 	/* complete the job back to userspace if no error */
 	if (rc == 0)
Signature · 1 line
2.25.0
#2 lore raw

PATCH 2/14 lpfc: Fix missing MDS functionality

In reply to James Smart

Visual code inspection of the MDS implementation revealed two errors in the driver:
- The set features Feature Code had an incorrect value
- The routine that classifies command type for cmd completions was missing the Send Frame definition. Send Frame is used for MDS driver loopback.

---
 drivers/scsi/lpfc/lpfc_hw4.h | 2 +-
 drivers/scsi/lpfc/lpfc_sli.c | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/lpfc/lpfc_hw4.h +1 -1
index 6dfff0376547..b8d3144a452d 100644
@@ -3531,7 +3531,7 @@ struct lpfc_sli4_parameters {
 };
 
 #define LPFC_SET_UE_RECOVERY		0x10
-#define LPFC_SET_MDS_DIAGS		0x11
+#define LPFC_SET_MDS_DIAGS		0x12
 #define LPFC_SET_DUAL_DUMP		0x1e
 struct lpfc_mbx_set_feature {
 	struct mbox_header header;
drivers/scsi/lpfc/lpfc_sli.c +1 -0
index 25653baba367..1575fccc8b0c 100644
@@ -1491,6 +1491,7 @@ lpfc_sli_iocb_cmd_type(uint8_t iocb_cmnd)
 	case DSSCMD_IWRITE64_CX:
 	case DSSCMD_IREAD64_CR:
 	case DSSCMD_IREAD64_CX:
+	case CMD_SEND_FRAME:
 		type = LPFC_SOL_IOCB;
 		break;
 	case CMD_ABORT_XRI_CN:
Signature · 1 line
2.25.0
#3 lore raw

PATCH 3/14 lpfc: Fix first-burst driver implementation.

In reply to James Smart

The current implementation of the driver's first-burst behavior had a module parameter setting a first burst size that was then used unconditionally on all targets and was ignorant of any limits that the targets have. The driver should have been determining the capability of each target and enforced target-specific limits.

The following changes were made:
- lpfc_first_burst_size parameter is now an enable/disable value.
- Upon PRLI completion, if first-burst is enabled and the target supports first burst, the driver will issue a modesense6 scsi command to obtain the disconnect-reconnect page that has transport specific limits. This page reports the max first-burst size supported on the target. The size supported is saved in the target node structure.
- The saved first burst size was increased to 32bits so that 64k could be supported.
- Fix:Ensure that first burst size requested isn't larger than the io itself.

---
 drivers/scsi/lpfc/lpfc_crtn.h      |   4 +
 drivers/scsi/lpfc/lpfc_disc.h      |   1 +
 drivers/scsi/lpfc/lpfc_els.c       |   4 +-
 drivers/scsi/lpfc/lpfc_hbadisc.c   |   4 +
 drivers/scsi/lpfc/lpfc_nportdisc.c |  16 +-
 drivers/scsi/lpfc/lpfc_scsi.c      | 296 ++++++++++++++++++++++++++++-
 drivers/scsi/lpfc/lpfc_scsi.h      |  24 +++
 drivers/scsi/lpfc/lpfc_sli.c       |  31 ++-
 drivers/scsi/lpfc/lpfc_sli.h       |   1 +
 9 files changed, 365 insertions(+), 16 deletions(-)
drivers/scsi/lpfc/lpfc_crtn.h +4 -0
index 9ee6b930a655..c832a7d26e86 100644
@@ -591,6 +591,10 @@ struct lpfc_io_buf *lpfc_get_io_buf(struct lpfc_hba *phba,
 				int);
 void lpfc_release_io_buf(struct lpfc_hba *phba, struct lpfc_io_buf *ncmd,
 			 struct lpfc_sli4_hdw_queue *qp);
+void lpfc_mode_sense_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *p_io_in,
+			  struct lpfc_iocbq *p_io_out);
+int lpfc_issue_scsi_cmd(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
+			struct lpfc_scsi_io_req *cmd_req);
 void lpfc_io_ktime(struct lpfc_hba *phba, struct lpfc_io_buf *ncmd);
 void lpfc_nvme_cmd_template(void);
 void lpfc_nvmet_cmd_template(void);
drivers/scsi/lpfc/lpfc_disc.h +1 -0
index 482e4a888dae..42b6327cc3a4 100644
@@ -115,6 +115,7 @@ struct lpfc_nodelist {
 	u8		nlp_nvme_info;	        /* NVME NSLER Support */
 #define NLP_NVME_NSLER     0x1			/* NVME NSLER device */
 
+	uint32_t        first_burst;	/* First Burst size for ndlp */
 	uint16_t        nlp_usg_map;	/* ndlp management usage bitmap */
 #define NLP_USG_NODE_ACT_BIT	0x1	/* Indicate ndlp is actively used */
 #define NLP_USG_IACT_REQ_BIT	0x2	/* Request to inactivate ndlp */
drivers/scsi/lpfc/lpfc_els.c +3 -1
index 3d670568a276..a7bbe628bc52 100644
@@ -2425,7 +2425,9 @@ lpfc_issue_els_prli(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
 		}
 		npr->estabImagePair = 1;
 		npr->readXferRdyDis = 1;
-		if (vport->cfg_first_burst_size)
+		if (phba->sli_rev == LPFC_SLI_REV4 &&
+		    !(phba->hba_flag & HBA_FCOE_MODE) &&
+		    vport->cfg_first_burst_size)
 			npr->writeXferRdyDis = 1;
 
 		/* For FCP support */
drivers/scsi/lpfc/lpfc_hbadisc.c +4 -0
index 4084f7f2b821..47259970907d 100644
@@ -4279,6 +4279,10 @@ lpfc_nlp_state_cleanup(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
 	/* FCP and NVME Transport interface */
 	if ((old_state == NLP_STE_MAPPED_NODE ||
 	     old_state == NLP_STE_UNMAPPED_NODE)) {
+		/* SCSI-FCP first burst needs to be renegotiated on the
+		 * next successful PRLI completion.
+		 */
+		ndlp->first_burst = 0;
 		if (ndlp->rport) {
 			vport->phba->nport_event_cnt++;
 			lpfc_unregister_remote_port(ndlp);
drivers/scsi/lpfc/lpfc_nportdisc.c +14 -2
index d8501bd959e7..2871c9b0cbfa 100644
@@ -2160,7 +2160,9 @@ lpfc_cmpl_prli_prli_issue(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
 	IOCB_t *irsp;
 	PRLI *npr;
 	struct lpfc_nvme_prli *nvpr;
+	struct lpfc_scsi_io_req cmd_req;
 	void *temp_ptr;
+	int ret = 0;
 
 	cmdiocb = (struct lpfc_iocbq *) arg;
 	rspiocb = cmdiocb->context_un.rsp_iocb;
@@ -2200,16 +2202,26 @@ lpfc_cmpl_prli_prli_issue(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
 				 "6028 FCP NPR PRLI Cmpl Init %d Target %d\n",
 				 npr->initiatorFunc,
 				 npr->targetFunc);
+
 		if (npr->initiatorFunc)
 			ndlp->nlp_type |= NLP_FCP_INITIATOR;
 		if (npr->targetFunc) {
 			ndlp->nlp_type |= NLP_FCP_TARGET;
-			if (npr->writeXferRdyDis)
+			if (npr->writeXferRdyDis) {
 				ndlp->nlp_flag |= NLP_FIRSTBURST;
+				cmd_req.cmd_id = MODE_SENSE;
+				cmd_req.dev_id = CTRL_ID;
+				cmd_req.cmd_cmpl = lpfc_mode_sense_cmpl;
+				ret = lpfc_issue_scsi_cmd(vport, ndlp,
+							  &cmd_req);
+				lpfc_printf_vlog(vport, KERN_INFO,
+						 LOG_FCP | LOG_FCP_ERROR,
+						 "6334 FB issue returns %d\n",
+						 ret);
+			}
 		}
 		if (npr->Retry)
 			ndlp->nlp_fcp_info |= NLP_FCP_2_DEVICE;
-
 	} else if (nvpr &&
 		   (bf_get_be32(prli_acc_rsp_code, nvpr) ==
 		    PRLI_REQ_EXECUTED) &&
drivers/scsi/lpfc/lpfc_scsi.c +286 -10

334 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_scsi.h +24 -0
index f76667b7da7b..964d0081fbb2 100644
@@ -24,6 +24,9 @@
 
 struct lpfc_hba;
 #define LPFC_FCP_CDB_LEN 16
+#define CTRL_ID 0
+#define LPFC_DATA_SIZE 255
+#define LPFC_FCP_FB_MAX 65536
 
 #define list_remove_head(list, entry, type, member)		\
 	do {							\
@@ -130,6 +133,27 @@ struct lpfc_scsicmd_bkt {
 	uint32_t cmd_count;
 };
 
+struct lpfc_scsi_io_req {
+	u32 dev_id;
+	u32 cmd_id;
+	void (*cmd_cmpl)(struct lpfc_hba *phba, struct lpfc_iocbq *p_io_in,
+			 struct lpfc_iocbq *p_io_out);
+};
+
+struct lpfc_first_burst_page {
+	u8 page_code;
+	u8 page_len;
+	u8 buf_full_r;
+	u8 buf_mt_r;
+	__be16 bus_inact_lmt;
+	__be16 disc_tmo;
+	__be16 conn_tmo;
+	__be16 max_burst;
+	u8 byte12;
+	u8 rsvd;
+	__be16 first_burst;
+};
+
 #define LPFC_SCSI_DMA_EXT_SIZE	264
 #define LPFC_BPL_SIZE		1024
 #define MDAC_DIRECT_CMD		0x22
drivers/scsi/lpfc/lpfc_sli.c +28 -3
index 1575fccc8b0c..3c61a6d72a1a 100644
@@ -9467,6 +9467,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 	struct lpfc_nodelist *ndlp;
 	uint32_t *pcmd;
 	uint32_t if_type;
+	struct lpfc_io_buf *lpfc_cmd;
 
 	fip = phba->hba_flag & HBA_FIP_SUPPORT;
 	/* The fcp commands will set command type */
@@ -9481,6 +9482,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 		memset(wqe, 0, sizeof(union lpfc_wqe128));
 	/* Some of the fields are in the right position already */
 	memcpy(wqe, &iocbq->iocb, sizeof(union lpfc_wqe));
+
 	/* The ct field has moved so reset */
 	wqe->generic.wqe_com.word7 = 0;
 	wqe->generic.wqe_com.word10 = 0;
@@ -9625,6 +9627,14 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 		bf_set(wqe_ebde_cnt, &wqe->xmit_bcast64.wqe_com, 0);
 		break;
 	case CMD_FCP_IWRITE64_CR:
+		lpfc_cmd = iocbq->context1;
+
+		/* The ndlp indicates if first burst support is supported. */
+		if (iocbq->iocb_flag & LPFC_IO_LIBDFC)
+			ndlp = iocbq->context_un.ndlp;
+		else
+			ndlp = lpfc_cmd->ndlp;
+
 		command_type = FCP_COMMAND_DATA_OUT;
 		/* word3 iocb=iotag wqe=payload_offset_len */
 		/* Add the FCP_CMD and FCP_RSP sizes to get the offset */
@@ -9632,8 +9642,19 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 		       xmit_len + sizeof(struct fcp_rsp));
 		bf_set(cmd_buff_len, &wqe->fcp_iwrite,
 		       0);
-		/* word4 iocb=parameter wqe=total_xfer_length memcpy */
-		/* word5 iocb=initial_xfer_len wqe=initial_xfer_len memcpy */
+
+		/* Set first-burst provided it was successfully negotiated */
+		if (phba->sli_rev == LPFC_SLI_REV4 &&
+		    iocbq->iocb_flag & LPFC_IO_FCP &&
+		    !(phba->hba_flag & HBA_FCOE_MODE) && ndlp->first_burst) {
+			u32 init_len;
+
+			total_len = be32_to_cpu(lpfc_cmd->fcp_cmnd->fcpDl);
+			init_len = min(total_len, ndlp->first_burst);
+			wqe->fcp_iwrite.initial_xfer_len = init_len;
+			wqe->fcp_iwrite.total_xfer_len = total_len;
+		}
+
 		bf_set(wqe_erp, &wqe->fcp_iwrite.wqe_com,
 		       iocbq->iocb.ulpFCP2Rcvy);
 		bf_set(wqe_lnk, &wqe->fcp_iwrite.wqe_com, iocbq->iocb.ulpXS);
@@ -9663,7 +9684,6 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 			bf_set(wqe_pbde, &wqe->fcp_iwrite.wqe_com, 0);
 
 		if (phba->fcp_embed_io) {
-			struct lpfc_io_buf *lpfc_cmd;
 			struct sli4_sge *sgl;
 			struct fcp_cmnd *fcp_cmnd;
 			uint32_t *ptr;
@@ -10031,6 +10051,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 	bf_set(wqe_cmnd, &wqe->generic.wqe_com, cmnd);
 	bf_set(wqe_class, &wqe->generic.wqe_com, iocbq->iocb.ulpClass);
 	bf_set(wqe_cqid, &wqe->generic.wqe_com, LPFC_WQE_CQ_ID_DEFAULT);
+
 	return 0;
 }
 
@@ -20452,6 +20473,10 @@ void lpfc_release_io_buf(struct lpfc_hba *phba, struct lpfc_io_buf *lpfc_ncmd,
 	lpfc_ncmd->cur_iocbq.wqe_cmpl = NULL;
 	lpfc_ncmd->cur_iocbq.iocb_cmpl = NULL;
 
+	/* First Burst initial and total transfer length must be zeroed. */
+	lpfc_ncmd->cur_iocbq.wqe.words[4] = 0;
+	lpfc_ncmd->cur_iocbq.wqe.words[5] = 0;
+
 	if (phba->cfg_xpsgl && !phba->nvmet_support &&
 	    !list_empty(&lpfc_ncmd->dma_sgl_xtra_list))
 		lpfc_put_sgl_per_hdwq(phba, lpfc_ncmd);
drivers/scsi/lpfc/lpfc_sli.h +1 -0
index 93d976ea8c5d..d6207974abf4 100644
@@ -100,6 +100,7 @@ struct lpfc_iocbq {
 #define LPFC_IO_NVME	        0x200000 /* NVME FCP command */
 #define LPFC_IO_NVME_LS		0x400000 /* NVME LS command */
 #define LPFC_IO_NVMET		0x800000 /* NVMET command */
+#define LPFC_IO_DRVR_INIT       0x80000000
 
 	uint32_t drvrTimeout;	/* driver timeout in seconds */
 	struct lpfc_vport *vport;/* virtual port pointer */
Signature · 1 line
2.25.0
#4 lore raw

Re: [PATCH 03/14] lpfc: Fix first-burst driver implementation.

James,

Quoted 5 lines

- Upon PRLI completion, if first-burst is enabled and the target supports first burst, the driver will issue a modesense6 scsi command to obtain the disconnect-reconnect page that has transport specific limits. This page reports the max first-burst size supported on the target. The size supported is saved in the target node structure.

I didn't make it beyond this patch :(

Why do this in the driver? If you need the Disconnect-Reconnect page, then let's ask for it in the core code. Maybe in the fc transport so we don't risk upsetting USB devices, etc. See sas_read_port_mode_page().

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#5 lore raw

Re: [PATCH 03/14] lpfc: Fix first-burst driver implementation.

Quoted 9 lines · Martin K. Petersen

James,

Quoted 5 lines

- Upon PRLI completion, if first-burst is enabled and the target supports first burst, the driver will issue a modesense6 scsi command to obtain the disconnect-reconnect page that has transport specific limits. This page reports the max first-burst size supported on the target. The size supported is saved in the target node structure.

Quoted 6 lines

I didn't make it beyond this patch :(

Why do this in the driver? If you need the Disconnect-Reconnect page, then let's ask for it in the core code. Maybe in the fc transport so we don't risk upsetting USB devices, etc. See sas_read_port_mode_page().

Because it was rather awkward to coordinate, snoop, sample at the right time, and a lot more work when implemented in the midlayer. I really don't like that scsi put transport controls in a scsi mode page.

I will take another look at it. It should be queried about the same time as when luns are first probed - lun 0 or report luns, or on the first lun probed as it's an I_T set of parameters. Not sure if it should be saved in the starget (fields are defined generically) or in the transport.

Please review the rest and I can repost for any comments on the other patches and I'll remove this patch.

-- james

#6 lore raw

PATCH 4/14 lpfc: Fix NVME rport deregister and registration during ADISC

In reply to James Smart

During driver unload/reload testing, the nvme initiator would not re-establish connectivity to nvme controllers on reload.

The failing nvme array supports concurrent FCP and NVME operation via different nport_id's. The array was repeatedly sending an ADISC every 2 seconds after PLOGI completed and while nvme subsytems were executing discovery. The target would continue this state for roughly 45 seconds.

The driver's current behavior on ADISC receipt is to validate a the ADISC vs the device and issue a RESUME_RPI to restore transmission. The receipt of the ADISC effectively caused a driver to take actions similar to a logout and login for the remote port, causing the deregistration of the nvme rport and a subsequent re-registration. This caused a constant reset and re-connect of the nvme controller while this 45s window occurred. There was no need for the state changes as ADISC does not change login state.

This patch corrects this behavior by validating if the remoteport is already logged in (MAPPED) and when true, avoids the call to set the ndlp state to MAPPED, which triggers the unreg/re-reg. Thus ADISC does not change the login state of the node.

---
 drivers/scsi/lpfc/lpfc_nportdisc.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
drivers/scsi/lpfc/lpfc_nportdisc.c +11 -5
index 2871c9b0cbfa..057af32f498d 100644
@@ -797,11 +797,17 @@ lpfc_rcv_padisc(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
 				ndlp, NULL);
 		}
 out:
-		/* If we are authenticated, move to the proper state */
-		if (ndlp->nlp_type & (NLP_FCP_TARGET | NLP_NVME_TARGET))
-			lpfc_nlp_set_state(vport, ndlp, NLP_STE_MAPPED_NODE);
-		else
-			lpfc_nlp_set_state(vport, ndlp, NLP_STE_UNMAPPED_NODE);
+		/* If we are authenticated, move to the proper state.
+		 * It is possible an ADISC arrived and the remote nport
+		 * is already in MAPPED or UNMAPPED state.  Catch this
+		 * condition and don't set the nlp_state again because
+		 * it causes an unnecessary transport unregister/register.
+		 */
+		if (ndlp->nlp_type & (NLP_FCP_TARGET | NLP_NVME_TARGET)) {
+			if (ndlp->nlp_state != NLP_STE_MAPPED_NODE)
+				lpfc_nlp_set_state(vport, ndlp,
+						   NLP_STE_MAPPED_NODE);
+		}
 
 		return 1;
 	}
Signature · 1 line
2.25.0
#7 lore raw

PATCH 5/14 lpfc: Fix oops due to overrun when reading SLI3 data

In reply to James Smart

When using DUMP on SLI3 to read VPD and Port status data (config region 23), the adapter is overruning the kmalloc'd buffer causing havoc on other consumers of the allocation pools.

Rework the loops processing the dump data and validate/size memory lengths before performing bcopy's.

---
 drivers/scsi/lpfc/lpfc_init.c | 14 ++++++++------
 drivers/scsi/lpfc/lpfc_sli.c  | 14 +++++++-------
 2 files changed, 15 insertions(+), 13 deletions(-)
drivers/scsi/lpfc/lpfc_init.c +8 -6
index 69a5249e007a..287a78185dc7 100644
@@ -253,13 +253,15 @@ lpfc_config_port_prep(struct lpfc_hba *phba)
 		 */
 		if (mb->un.varDmp.word_cnt == 0)
 			break;
-		if (mb->un.varDmp.word_cnt > DMP_VPD_SIZE - offset)
-			mb->un.varDmp.word_cnt = DMP_VPD_SIZE - offset;
+
+		i =  mb->un.varDmp.word_cnt * sizeof(uint32_t);
+		if (offset + i >  DMP_VPD_SIZE)
+			i =  DMP_VPD_SIZE - offset;
 		lpfc_sli_pcimem_bcopy(((uint8_t *)mb) + DMP_RSP_OFFSET,
-				      lpfc_vpd_data + offset,
-				      mb->un.varDmp.word_cnt);
-		offset += mb->un.varDmp.word_cnt;
-	} while (mb->un.varDmp.word_cnt && offset < DMP_VPD_SIZE);
+				      lpfc_vpd_data  + offset, i);
+		offset += i;
+	} while (offset < DMP_VPD_SIZE);
+
 	lpfc_parse_vpd(phba, lpfc_vpd_data, offset);
 
 	kfree(lpfc_vpd_data);
drivers/scsi/lpfc/lpfc_sli.c +7 -7
index 3c61a6d72a1a..c598bef5cad4 100644
@@ -19347,7 +19347,7 @@ lpfc_sli_get_config_region23(struct lpfc_hba *phba, char *rgn23_data)
 	LPFC_MBOXQ_t *pmb = NULL;
 	MAILBOX_t *mb;
 	uint32_t offset = 0;
-	int rc;
+	int i, rc;
 
 	if (!rgn23_data)
 		return 0;
@@ -19377,14 +19377,14 @@ lpfc_sli_get_config_region23(struct lpfc_hba *phba, char *rgn23_data)
 		 */
 		if (mb->un.varDmp.word_cnt == 0)
 			break;
-		if (mb->un.varDmp.word_cnt > DMP_RGN23_SIZE - offset)
-			mb->un.varDmp.word_cnt = DMP_RGN23_SIZE - offset;
 
+		i =  mb->un.varDmp.word_cnt * sizeof(uint32_t);
+		if (offset + i >  DMP_RGN23_SIZE)
+			i =  DMP_RGN23_SIZE - offset;
 		lpfc_sli_pcimem_bcopy(((uint8_t *)mb) + DMP_RSP_OFFSET,
-				       rgn23_data + offset,
-				       mb->un.varDmp.word_cnt);
-		offset += mb->un.varDmp.word_cnt;
-	} while (mb->un.varDmp.word_cnt && offset < DMP_RGN23_SIZE);
+				      rgn23_data  + offset, i);
+		offset += i;
+	} while (offset < DMP_RGN23_SIZE);
 
 	mempool_free(pmb, phba->mbox_mem_pool);
 	return offset;
Signature · 1 line
2.25.0
#8 lore raw

PATCH 6/14 lpfc: Fix stack trace seen while setting rrq active

In reply to James Smart

Call traces have been observed running different tests that involve aborts and setting the rrq active flag. The lpfc_set_rrq_active routine is doing a mempool_alloc under the soft_irq processing level. When the mempool needs to get a new buffer from the free pool and has to wait for memory to become free it will check the flags passed in on the alloc and dump the stack if the thread is running in interrupt context.

Replace the GFP_KERNEL flag with GFP_ATOMIC so that the memory allocation will not attempt to sleep if there is no mem available.

---
 drivers/scsi/lpfc/lpfc_mem.c | 3 ++-
 drivers/scsi/lpfc/lpfc_sli.c | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
drivers/scsi/lpfc/lpfc_mem.c +2 -1
index 726f6619230f..e8c0066eb4aa 100644
@@ -45,6 +45,7 @@
 #define LPFC_MBUF_POOL_SIZE     64      /* max elements in MBUF safety pool */
 #define LPFC_MEM_POOL_SIZE      64      /* max elem in non-DMA safety pool */
 #define LPFC_DEVICE_DATA_POOL_SIZE 64   /* max elements in device data pool */
+#define LPFC_RRQ_POOL_SIZE	256	/* max elements in non-DMA  pool */
 
 int
 lpfc_mem_alloc_active_rrq_pool_s4(struct lpfc_hba *phba) {
@@ -121,7 +122,7 @@ lpfc_mem_alloc(struct lpfc_hba *phba, int align)
 
 	if (phba->sli_rev == LPFC_SLI_REV4) {
 		phba->rrq_pool =
-			mempool_create_kmalloc_pool(LPFC_MEM_POOL_SIZE,
+			mempool_create_kmalloc_pool(LPFC_RRQ_POOL_SIZE,
 						sizeof(struct lpfc_node_rrq));
 		if (!phba->rrq_pool)
 			goto fail_free_nlp_mem_pool;
drivers/scsi/lpfc/lpfc_sli.c +1 -1
index c598bef5cad4..c439cf9a82c7 100644
@@ -1079,7 +1079,7 @@ lpfc_set_rrq_active(struct lpfc_hba *phba, struct lpfc_nodelist *ndlp,
 		goto out;
 
 	spin_unlock_irqrestore(&phba->hbalock, iflags);
-	rrq = mempool_alloc(phba->rrq_pool, GFP_KERNEL);
+	rrq = mempool_alloc(phba->rrq_pool, GFP_ATOMIC);
 	if (!rrq) {
 		lpfc_printf_log(phba, KERN_INFO, LOG_SLI,
 				"3155 Unable to allocate RRQ xri:0x%x rxid:0x%x"
Signature · 1 line
2.25.0
#9 lore raw

PATCH 7/14 lpfc: Fix shost refcount mismatch when deleting vport

In reply to James Smart

When vports are deleted, it is observed that there is memory/kthread leakage as the vport isn't fully being released.

There is a shost reference taken in scsi_add_host_dma that is not released during scsi_remove_host. It was noticed that other drivers resolve this by doing a scsi_host_put after calling scsi_remove_host.

The vport_delete routine is taking two references one that corresponds to an access to the scsi_host in the vport_delete routine and another that is released after the adapter mailbox command completes that destroys the VPI that corresponds to the vport.

Remove one of the references taken such that the second reference that is put will complete the missing scsi_add_host_dma reference and the shost will be terminated.

---
 drivers/scsi/lpfc/lpfc_vport.c | 26 ++++++++------------------
 1 file changed, 8 insertions(+), 18 deletions(-)
drivers/scsi/lpfc/lpfc_vport.c +8 -18
index b76646357980..d0296f7cf45f 100644
@@ -642,27 +642,16 @@ lpfc_vport_delete(struct fc_vport *fc_vport)
 		    vport->port_state < LPFC_VPORT_READY)
 			return -EAGAIN;
 	}
+
 	/*
-	 * This is a bit of a mess.  We want to ensure the shost doesn't get
-	 * torn down until we're done with the embedded lpfc_vport structure.
-	 *
-	 * Beyond holding a reference for this function, we also need a
-	 * reference for outstanding I/O requests we schedule during delete
-	 * processing.  But once we scsi_remove_host() we can no longer obtain
-	 * a reference through scsi_host_get().
-	 *
-	 * So we take two references here.  We release one reference at the
-	 * bottom of the function -- after delinking the vport.  And we
-	 * release the other at the completion of the unreg_vpi that get's
-	 * initiated after we've disposed of all other resources associated
-	 * with the port.
+	 * Take early refcount for outstanding I/O requests we schedule during
+	 * delete processing for unreg_vpi.  Always keep this before
+	 * scsi_remove_host() as we can no longer obtain a reference through
+	 * scsi_host_get() after scsi_host_remove as shost is set to SHOST_DEL.
 	 */
 	if (!scsi_host_get(shost))
 		return VPORT_INVAL;
-	if (!scsi_host_get(shost)) {
-		scsi_host_put(shost);
-		return VPORT_INVAL;
-	}
+
 	lpfc_free_sysfs_attr(vport);
 
 	lpfc_debugfs_terminate(vport);
@@ -809,8 +798,9 @@ lpfc_vport_delete(struct fc_vport *fc_vport)
 		if (!(vport->vpi_state & LPFC_VPI_REGISTERED) ||
 				lpfc_mbx_unreg_vpi(vport))
 			scsi_host_put(shost);
-	} else
+	} else {
 		scsi_host_put(shost);
+	}
 
 	lpfc_free_vpi(phba, vport->vpi);
 	vport->work_port_events = 0;
Signature · 1 line
2.25.0
#10 lore raw

PATCH 8/14 lpfc: Fix kdump hang on PPC

In reply to James Smart

When the kdump kernel shuts down lpfc calls flush_work_queue on an interrupt to schedule the cq handler. When there is only one cpu active on the kdump kernel, it is possible for the work_on to get scheduled on a non-active cpu causing it to never be scheduled.

When in the kdump environment, per-cpu affinity of cq's to cpus is not necessary. In those cases, use a general queue_work rather than a queue_work_on().

---
 drivers/scsi/lpfc/lpfc_sli.c | 36 ++++++++++++++++++++++++++++++------
 1 file changed, 30 insertions(+), 6 deletions(-)
drivers/scsi/lpfc/lpfc_sli.c +30 -6
index c439cf9a82c7..2fc8a1db81a4 100644
@@ -35,6 +35,7 @@
 #include <scsi/scsi_transport_fc.h>
 #include <scsi/fc/fc_fs.h>
 #include <linux/aer.h>
+#include <linux/crash_dump.h>
 #ifdef CONFIG_X86
 #include <asm/set_memory.h>
 #endif
@@ -13723,6 +13724,7 @@ lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
 {
 	struct lpfc_queue *cq = NULL, *childq;
 	uint16_t cqid;
+	int ret = 0;
 
 	/* Get the reference to the corresponding CQ */
 	cqid = bf_get_le32(lpfc_eqe_resource_id, eqe);
@@ -13744,7 +13746,12 @@ lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
 	/* Save EQ associated with this CQ */
 	cq->assoc_qp = speq;
 
-	if (!queue_work_on(cq->chann, phba->wq, &cq->spwork))
+	if (is_kdump_kernel())
+		ret = queue_work(phba->wq, &cq->spwork);
+	else
+		ret = queue_work_on(cq->chann, phba->wq, &cq->spwork);
+
+	if (!ret)
 		lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
 				"0390 Cannot schedule soft IRQ "
 				"for CQ eqcqid=%d, cqid=%d on CPU %d\n",
@@ -13857,6 +13864,7 @@ __lpfc_sli4_sp_process_cq(struct lpfc_queue *cq)
 	struct lpfc_hba *phba = cq->phba;
 	unsigned long delay;
 	bool workposted = false;
+	int ret = 0;
 
 	/* Process and rearm the CQ */
 	switch (cq->type) {
@@ -13883,8 +13891,13 @@ __lpfc_sli4_sp_process_cq(struct lpfc_queue *cq)
 	}
 
 	if (delay) {
-		if (!queue_delayed_work_on(cq->chann, phba->wq,
-					   &cq->sched_spwork, delay))
+		if (is_kdump_kernel())
+			ret = queue_delayed_work(phba->wq, &cq->sched_spwork,
+						delay);
+		else
+			ret = queue_delayed_work_on(cq->chann, phba->wq,
+						&cq->sched_spwork, delay);
+		if (!ret)
 			lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
 				"0394 Cannot schedule soft IRQ "
 				"for cqid=%d on CPU %d\n",
@@ -14236,6 +14249,7 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_queue *eq,
 	struct lpfc_queue *cq = NULL;
 	uint32_t qidx = eq->hdwq;
 	uint16_t cqid, id;
+	int ret = 0;
 
 	if (unlikely(bf_get_le32(lpfc_eqe_major_code, eqe) != 0)) {
 		lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
@@ -14295,7 +14309,11 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_queue *eq,
 	else
 		cq->isr_timestamp = 0;
 #endif
-	if (!queue_work_on(cq->chann, phba->wq, &cq->irqwork))
+	if (is_kdump_kernel())
+		ret = queue_work(phba->wq, &cq->irqwork);
+	else
+		ret = queue_work_on(cq->chann, phba->wq, &cq->irqwork);
+	if (!ret)
 		lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
 				"0363 Cannot schedule soft IRQ "
 				"for CQ eqcqid=%d, cqid=%d on CPU %d\n",
@@ -14323,14 +14341,20 @@ __lpfc_sli4_hba_process_cq(struct lpfc_queue *cq)
 	struct lpfc_hba *phba = cq->phba;
 	unsigned long delay;
 	bool workposted = false;
+	int ret = 0;
 
 	/* process and rearm the CQ */
 	workposted |= __lpfc_sli4_process_cq(phba, cq, lpfc_sli4_fp_handle_cqe,
 					     &delay);
 
 	if (delay) {
-		if (!queue_delayed_work_on(cq->chann, phba->wq,
-					   &cq->sched_irqwork, delay))
+		if (is_kdump_kernel())
+			ret = queue_delayed_work(phba->wq, &cq->sched_irqwork,
+						delay);
+		else
+			ret = queue_delayed_work_on(cq->chann, phba->wq,
+						&cq->sched_irqwork, delay);
+		if (!ret)
 			lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
 				"0367 Cannot schedule soft IRQ "
 				"for cqid=%d on CPU %d\n",
Signature · 1 line
2.25.0
#11 lore raw

PATCH 9/14 lpfc: Fix language in 0373 message to reflect non-error message

In reply to James Smart

Change vocabulary of 0373 log msg from "error" to "cmpl"
The current language of the 0373 message contains the word "error" which caused a number of customers to inquire about the "error" and if it should be a concern. It isn't an error, it's simply an io completion status.

Revise the message to replace the word "error" with "compl" for completion.

---
 drivers/scsi/lpfc/lpfc_sli.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/scsi/lpfc/lpfc_sli.c +2 -2
index 2fc8a1db81a4..271ba82f5d85 100644
@@ -13968,9 +13968,9 @@ lpfc_sli4_fp_handle_fcp_wcqe(struct lpfc_hba *phba, struct lpfc_queue *cq,
 		     IOERR_NO_RESOURCES))
 			phba->lpfc_rampdown_queue_depth(phba);
 
-		/* Log the error status */
+		/* Log the cmpl status */
 		lpfc_printf_log(phba, KERN_INFO, LOG_SLI,
-				"0373 FCP CQE error: status=x%x: "
+				"0373 FCP CQE cmpl: status=x%x: "
 				"CQE: %08x %08x %08x %08x\n",
 				bf_get(lpfc_wcqe_c_status, wcqe),
 				wcqe->word0, wcqe->total_data_placed,
Signature · 1 line
2.25.0
#12 lore raw

PATCH 10/14 lpfc: Allow applications to issue Common Set Features mailbox command

In reply to James Smart

Currently the driver validates command codes received from the application. COMMON_SET_FEATURES is not currently being approved.

Add definition of the missing command and allow it to be issued by applications.

---
 drivers/scsi/lpfc/lpfc_bsg.c |  1 +
 drivers/scsi/lpfc/lpfc_bsg.h | 14 ++++++++++++++
 2 files changed, 15 insertions(+)
drivers/scsi/lpfc/lpfc_bsg.c +1 -0
index e91466aa1673..1d88fedaf3f0 100644
@@ -4304,6 +4304,7 @@ lpfc_bsg_handle_sli_cfg_mbox(struct lpfc_hba *phba, struct bsg_job *job,
 			case COMN_OPCODE_GET_CNTL_ADDL_ATTRIBUTES:
 			case COMN_OPCODE_GET_CNTL_ATTRIBUTES:
 			case COMN_OPCODE_GET_PROFILE_CONFIG:
+			case COMN_OPCODE_SET_FEATURES:
 				lpfc_printf_log(phba, KERN_INFO, LOG_LIBDFC,
 						"3106 Handled SLI_CONFIG "
 						"subsys_comn, opcode:x%x\n",
drivers/scsi/lpfc/lpfc_bsg.h +14 -0
index d1708133fd54..2dc71243775d 100644
@@ -225,6 +225,10 @@ struct lpfc_sli_config_hdr {
 	uint32_t reserved5;
 };
 
+#define LPFC_CSF_BOOT_DEV		0x1D
+#define LPFC_CSF_QUERY			0
+#define LPFC_CSF_SAVE			1
+
 struct lpfc_sli_config_emb0_subsys {
 	struct lpfc_sli_config_hdr	sli_config_hdr;
 #define LPFC_MBX_SLI_CONFIG_MAX_MSE     19
@@ -243,6 +247,15 @@ struct lpfc_sli_config_emb0_subsys {
 #define FCOE_OPCODE_ADD_FCF		0x09
 #define FCOE_OPCODE_SET_DPORT_MODE	0x27
 #define FCOE_OPCODE_GET_DPORT_RESULTS	0x28
+	uint32_t timeout;		/* comn_set_feature timeout */
+	uint32_t request_length;	/* comn_set_feature request len */
+	uint32_t version;		/* comn_set_feature version */
+	uint32_t csf_feature;		/* comn_set_feature feature */
+	uint32_t word69;		/* comn_set_feature parameter len */
+	uint32_t word70;		/* comn_set_feature parameter val0 */
+#define lpfc_emb0_subcmnd_csf_p0_SHIFT	0
+#define lpfc_emb0_subcmnd_csf_p0_MASK	0x3
+#define lpfc_emb0_subcmnd_csf_p0_WORD	word70
 };
 
 struct lpfc_sli_config_emb1_subsys {
@@ -261,6 +274,7 @@ struct lpfc_sli_config_emb1_subsys {
 #define COMN_OPCODE_WRITE_OBJECT	0xAC
 #define COMN_OPCODE_READ_OBJECT_LIST	0xAD
 #define COMN_OPCODE_DELETE_OBJECT	0xAE
+#define COMN_OPCODE_SET_FEATURES	0xBF
 #define COMN_OPCODE_GET_CNTL_ADDL_ATTRIBUTES	0x79
 #define COMN_OPCODE_GET_CNTL_ATTRIBUTES	0x20
 	uint32_t timeout;
Signature · 1 line
2.25.0
#13 lore raw

PATCH 11/14 lpfc: Add support to display if adapter dumps are available

In reply to James Smart

Currently, if there has been an issue whereby an adapter dump was taken, there is nothing displayed to hint that it is present. Utilities must be run and they must query for the status in order to then download the dump.

Add a message to the driver to query dump image presence when initializing the SLI Port.

---
 drivers/scsi/lpfc/lpfc_hw4.h |  3 +++
 drivers/scsi/lpfc/lpfc_sli.c | 22 ++++++++++++++++++++++
 2 files changed, 25 insertions(+)
drivers/scsi/lpfc/lpfc_hw4.h +3 -0
index b8d3144a452d..c4ba8273a63f 100644
@@ -650,6 +650,9 @@ struct lpfc_register {
 #define lpfc_sliport_status_oti_SHIFT	29
 #define lpfc_sliport_status_oti_MASK	0x1
 #define lpfc_sliport_status_oti_WORD	word0
+#define lpfc_sliport_status_dip_SHIFT	25
+#define lpfc_sliport_status_dip_MASK	0x1
+#define lpfc_sliport_status_dip_WORD	word0
 #define lpfc_sliport_status_rn_SHIFT	24
 #define lpfc_sliport_status_rn_MASK	0x1
 #define lpfc_sliport_status_rn_WORD	word0
drivers/scsi/lpfc/lpfc_sli.c +22 -0
index 271ba82f5d85..1ea710f9e842 100644
@@ -7300,6 +7300,26 @@ lpfc_post_rq_buffer(struct lpfc_hba *phba, struct lpfc_queue *hrq,
 	return 1;
 }
 
+static void lpfc_sli4_dip(struct lpfc_hba *phba)
+{
+	uint32_t if_type;
+
+	if_type = bf_get(lpfc_sli_intf_if_type, &phba->sli4_hba.sli_intf);
+	if (if_type == LPFC_SLI_INTF_IF_TYPE_2 ||
+	    if_type == LPFC_SLI_INTF_IF_TYPE_6) {
+		struct lpfc_register reg_data;
+
+		if (lpfc_readl(phba->sli4_hba.u.if_type2.STATUSregaddr,
+			       &reg_data.word0))
+			return;
+
+		if (bf_get(lpfc_sliport_status_dip, &reg_data))
+			lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
+					"2904 Firmware Dump Image Present"
+					" on Adapter");
+	}
+}
+
 /**
  * lpfc_sli4_hba_setup - SLI4 device initialization PCI function
  * @phba: Pointer to HBA context object.
@@ -7338,6 +7358,8 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba)
 		spin_unlock_irq(&phba->hbalock);
 	}
 
+	lpfc_sli4_dip(phba);
+
 	/*
 	 * Allocate a single mailbox container for initializing the
 	 * port.
Signature · 1 line
2.25.0
#14 lore raw

PATCH 12/14 lpfc: Add blk_io_poll support for latency improvment

In reply to James Smart

Although the existing implementation is very good at high i/o load, on tests involving light load, especially on only a few hardware queues, latency was a little higher than it can be due to using workqueue scheduling. Other tasks in the system can delay handling.

Change the lower level to use irq_poll by default which uses a softirq for io completion. This gives better latency as variance in when the cq is processed is reduced over the workqueue interface. However, as high load is better served by not being in softirq when the cpu is loaded, work queues are still used under high io load.

---
 drivers/scsi/lpfc/lpfc.h      |   3 +
 drivers/scsi/lpfc/lpfc_init.c |  90 +++++++++++++++++++++-
 drivers/scsi/lpfc/lpfc_sli.c  | 138 ++++++++++++++++++++++++++++------
 drivers/scsi/lpfc/lpfc_sli4.h |  18 +++++
 4 files changed, 224 insertions(+), 25 deletions(-)
drivers/scsi/lpfc/lpfc.h +3 -0
index c3ceb6e5b061..2ddcdedfdb8c 100644
@@ -709,6 +709,9 @@ struct lpfc_hba {
 	struct workqueue_struct *wq;
 	struct delayed_work     eq_delay_work;
 
+#define LPFC_IDLE_STAT_DELAY 1000
+	struct delayed_work	idle_stat_delay_work;
+
 	struct lpfc_sli sli;
 	uint8_t pci_dev_grp;	/* lpfc PCI dev group: 0x0, 0x1, 0x2,... */
 	uint32_t sli_rev;		/* SLI2, SLI3, or SLI4 */
drivers/scsi/lpfc/lpfc_init.c +88 -2

134 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_sli.c +115 -23

259 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_sli4.h +18 -0
index 4decb53d81c3..a966cdeb52ee 100644
@@ -20,6 +20,9 @@
  * included with this package.                                     *
  *******************************************************************/
 
+#include <linux/irq_poll.h>
+#include <linux/cpufreq.h>
+
 #if defined(CONFIG_DEBUG_FS) && !defined(CONFIG_SCSI_LPFC_DEBUG_FS)
 #define CONFIG_SCSI_LPFC_DEBUG_FS
 #endif
@@ -135,6 +138,16 @@ struct lpfc_rqb {
 					       struct rqb_dmabuf *);
 };
 
+enum lpfc_poll_mode {
+	LPFC_QUEUE_WORK,
+	LPFC_IRQ_POLL
+};
+
+struct lpfc_idle_stat {
+	u64 prev_idle;
+	u64 prev_wall;
+};
+
 struct lpfc_queue {
 	struct list_head list;
 	struct list_head wq_list;
@@ -265,6 +278,10 @@ struct lpfc_queue {
 	struct lpfc_queue *assoc_qp;
 	struct list_head _poll_list;
 	void **q_pgs;	/* array to index entries per page */
+
+#define LPFC_IRQ_POLL_WEIGHT 256
+	struct irq_poll iop;
+	enum lpfc_poll_mode poll_mode;
 };
 
 struct lpfc_sli4_link {
@@ -926,6 +943,7 @@ struct lpfc_sli4_hba {
 #ifdef CONFIG_SCSI_LPFC_DEBUG_FS
 	struct lpfc_hdwq_stat __percpu *c_stat;
 #endif
+	struct lpfc_idle_stat *idle_stat;
 	uint32_t conf_trunk;
 #define lpfc_conf_trunk_port0_WORD	conf_trunk
 #define lpfc_conf_trunk_port0_SHIFT	0
Signature · 1 line
2.25.0
#15 lore raw

PATCH 13/14 lpfc: Add an internal trace log buffer

In reply to James Smart

The current logging methods typically end up requesting a reproduction with a different logging level set to figure out what happened. This was mainly by design to not close the kernel log messages with things that were typically no interesting and the messages themselves could cause other issues.

When looking to make a better system, it was seen that in many cases when more data was wanted was when another message, usually at KERN_ERR level, was logged. And in most cases, what the additional logging that was then enabled was typically. Most of these areas fell into the discovery machine.

Based on this summary, the following design has been put in place:
The driver will maintain an internal log (256 elements of 256 bytes). The "additional logging" messages that are usually enabled in a reproduction will be changed to now log all the time to the internal log. A new logging level is defined - LOG_TRACE_EVENT. When this level is set (it is not by default) and a message marked as KERN_ERR is logged, all the messages in the internal log will be dumped to the kernel log before the KERN_ERR message is logged.

There is a timestamp on each message added to the internal log. However, this timestamp is not converted to wall time when logged. The value of the timestamp is solely to give a crude time reference for the messages.

---
 drivers/scsi/lpfc/lpfc.h           |  12 +
 drivers/scsi/lpfc/lpfc_crtn.h      |   2 +-
 drivers/scsi/lpfc/lpfc_ct.c        |  16 +-
 drivers/scsi/lpfc/lpfc_els.c       | 131 +++----
 drivers/scsi/lpfc/lpfc_hbadisc.c   | 216 +++++------
 drivers/scsi/lpfc/lpfc_init.c      | 565 +++++++++++++++++------------
 drivers/scsi/lpfc/lpfc_logmsg.h    |  24 +-
 drivers/scsi/lpfc/lpfc_nportdisc.c |  51 +--
 drivers/scsi/lpfc/lpfc_nvme.c      |  67 ++--
 drivers/scsi/lpfc/lpfc_nvmet.c     |  96 +++--
 drivers/scsi/lpfc/lpfc_scsi.c      | 125 +++----
 drivers/scsi/lpfc/lpfc_sli.c       | 355 +++++++++---------
 drivers/scsi/lpfc/lpfc_vport.c     |  60 +--
 13 files changed, 928 insertions(+), 792 deletions(-)
drivers/scsi/lpfc/lpfc.h +12 -0
index 2ddcdedfdb8c..549adfaa97ce 100644
@@ -627,6 +627,14 @@ struct lpfc_ras_fwlog {
 	enum ras_state state;    /* RAS logging running state */
 };
 
+#define DBG_LOG_STR_SZ 256
+#define DBG_LOG_SZ 256
+
+struct dbg_log_ent {
+	char log[DBG_LOG_STR_SZ];
+	u64     t_ns;
+};
+
 enum lpfc_irq_chann_mode {
 	/* Assign IRQs to all possible cpus that have hardware queues */
 	NORMAL_MODE,
@@ -1240,6 +1248,10 @@ struct lpfc_hba {
 	struct scsi_host_template port_template;
 	/* SCSI host template information - for all vports */
 	struct scsi_host_template vport_template;
+	atomic_t dbg_log_idx;
+	atomic_t dbg_log_cnt;
+	atomic_t dbg_log_dmping;
+	struct dbg_log_ent dbg_log[DBG_LOG_SZ];
 };
 
 static inline struct Scsi_Host *
drivers/scsi/lpfc/lpfc_crtn.h +1 -1
index c832a7d26e86..1511300c837f 100644
@@ -386,7 +386,7 @@ void lpfc_rq_buf_free(struct lpfc_hba *phba, struct lpfc_dmabuf *mp);
 int lpfc_link_reset(struct lpfc_vport *vport);
 
 /* Function prototypes. */
-int lpfc_check_pci_resettable(const struct lpfc_hba *phba);
+int lpfc_check_pci_resettable(struct lpfc_hba *phba);
 const char* lpfc_info(struct Scsi_Host *);
 int lpfc_scan_finished(struct Scsi_Host *, unsigned long);
 
drivers/scsi/lpfc/lpfc_ct.c +8 -8
index 69d4710d95a0..701f2405cf02 100644
@@ -750,7 +750,7 @@ lpfc_cmpl_ct_cmd_gid_ft(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 		if (vport->fc_flag & FC_RSCN_MODE)
 			lpfc_els_flush_rscn(vport);
 		lpfc_vport_set_state(vport, FC_VPORT_FAILED);
-		lpfc_printf_vlog(vport, KERN_ERR, LOG_ELS,
+		lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 				 "0257 GID_FT Query error: 0x%x 0x%x\n",
 				 irsp->ulpStatus, vport->fc_ns_retry);
 	} else {
@@ -811,7 +811,7 @@ lpfc_cmpl_ct_cmd_gid_ft(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 
 		} else {
 			/* NameServer Rsp Error */
-			lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+			lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 					"0241 NameServer Rsp Error "
 					"Data: x%x x%x x%x x%x\n",
 					CTrsp->CommandResponse.bits.CmdRsp,
@@ -951,7 +951,7 @@ lpfc_cmpl_ct_cmd_gid_pt(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 		if (vport->fc_flag & FC_RSCN_MODE)
 			lpfc_els_flush_rscn(vport);
 		lpfc_vport_set_state(vport, FC_VPORT_FAILED);
-		lpfc_printf_vlog(vport, KERN_ERR, LOG_ELS,
+		lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 				 "4103 GID_FT Query error: 0x%x 0x%x\n",
 				 irsp->ulpStatus, vport->fc_ns_retry);
 	} else {
@@ -1012,7 +1012,7 @@ lpfc_cmpl_ct_cmd_gid_pt(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 			}
 		} else {
 			/* NameServer Rsp Error */
-			lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+			lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 					 "4109 NameServer Rsp Error "
 					 "Data: x%x x%x x%x x%x\n",
 					 CTrsp->CommandResponse.bits.CmdRsp,
@@ -1143,7 +1143,7 @@ lpfc_cmpl_ct_cmd_gff_id(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 				}
 			}
 		}
-		lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+		lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 				 "0267 NameServer GFF Rsp "
 				 "x%x Error (%d %d) Data: x%x x%x\n",
 				 did, irsp->ulpStatus, irsp->un.ulpWord[4],
@@ -1271,7 +1271,7 @@ lpfc_cmpl_ct_cmd_gft_id(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 			}
 		}
 	} else
-		lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+		lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 				 "3065 GFT_ID failed x%08x\n", irsp->ulpStatus);
 
 	lpfc_ct_free_iocb(phba, cmdiocb);
@@ -1320,7 +1320,7 @@ lpfc_cmpl_ct(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 		irsp->ulpStatus, irsp->un.ulpWord[4], cmdcode);
 
 	if (irsp->ulpStatus) {
-		lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+		lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 				 "0268 NS cmd x%x Error (x%x x%x)\n",
 				 cmdcode, irsp->ulpStatus, irsp->un.ulpWord[4]);
 
@@ -1843,7 +1843,7 @@ lpfc_ns_cmd(struct lpfc_vport *vport, int cmdcode,
 ns_cmd_free_mp:
 	kfree(mp);
 ns_cmd_exit:
-	lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+	lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT,
 			 "0266 Issue NameServer Req x%x err %d Data: x%x x%x\n",
 			 cmdcode, rc, vport->fc_flag, vport->fc_rscn_id_cnt);
 	return 1;
drivers/scsi/lpfc/lpfc_els.c +69 -62

414 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_hbadisc.c +114 -102

595 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_init.c +331 -234

1,657 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_logmsg.h +19 -5
index 148d02a27b58..5660a8729462 100644
@@ -44,7 +44,11 @@
 #define LOG_NVME_DISC   0x00200000      /* NVME Discovery/Connect events. */
 #define LOG_NVME_ABTS   0x00400000      /* NVME ABTS events. */
 #define LOG_NVME_IOERR  0x00800000      /* NVME IO Error events. */
-#define LOG_ALL_MSG	0xffffffff	/* LOG all messages */
+#define LOG_TRACE_EVENT 0x80000000	/* Dmp the DBG log on this err */
+#define LOG_ALL_MSG	0x7fffffff	/* LOG all messages */
+
+void lpfc_dmp_dbg(struct lpfc_hba *phba);
+void lpfc_dbg_print(struct lpfc_hba *phba, const char *fmt, ...);
 
 /* generate message by verbose log setting or severity */
 #define lpfc_vlog_msg(vport, level, mask, fmt, arg...) \
@@ -65,9 +69,15 @@ do { \
 
 #define lpfc_printf_vlog(vport, level, mask, fmt, arg...) \
 do { \
-	{ if (((mask) & (vport)->cfg_log_verbose) || (level[1] <= '3')) \
+	{ if (((mask) & (vport)->cfg_log_verbose) || (level[1] <= '3')) { \
+		if ((mask) & LOG_TRACE_EVENT) \
+			lpfc_dmp_dbg((vport)->phba); \
 		dev_printk(level, &((vport)->phba->pcidev)->dev, "%d:(%d):" \
-			   fmt, (vport)->phba->brd_no, vport->vpi, ##arg); } \
+			   fmt, (vport)->phba->brd_no, vport->vpi, ##arg);  \
+		} else if (!(vport)->cfg_log_verbose) \
+			lpfc_dbg_print((vport)->phba, "%d:(%d):" fmt, \
+				(vport)->phba->brd_no, (vport)->vpi, ##arg); \
+	} \
 } while (0)
 
 #define lpfc_printf_log(phba, level, mask, fmt, arg...) \
@@ -75,8 +85,12 @@ do { \
 	{ uint32_t log_verbose = (phba)->pport ? \
 				 (phba)->pport->cfg_log_verbose : \
 				 (phba)->cfg_log_verbose; \
-	  if (((mask) & log_verbose) || (level[1] <= '3')) \
+	if (((mask) & log_verbose) || (level[1] <= '3')) { \
+		if ((mask) & LOG_TRACE_EVENT) \
+			lpfc_dmp_dbg(phba); \
 		dev_printk(level, &((phba)->pcidev)->dev, "%d:" \
-			   fmt, phba->brd_no, ##arg); \
+			fmt, phba->brd_no, ##arg); \
+	} else  if (!(phba)->cfg_log_verbose)\
+		lpfc_dbg_print(phba, "%d:" fmt, phba->brd_no, ##arg); \
 	} \
 } while (0)
drivers/scsi/lpfc/lpfc_nportdisc.c +26 -25

165 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_nvme.c +29 -38

30 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_nvmet.c +47 -49

96 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_scsi.c +64 -61

125 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_sli.c +177 -178

355 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_vport.c +31 -29

60 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.25.0

Part of the diff is not shown here. View the raw message.

#16 lore raw

PATCH 14/14 lpfc: Update lpfc version to 12.8.0.2

In reply to James Smart

Update lpfc version to 12.8.0.2

---
 drivers/scsi/lpfc/lpfc_version.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/scsi/lpfc/lpfc_version.h +1 -1
index ab0bc26c098d..1987c6666279 100644
@@ -20,7 +20,7 @@
  * included with this package.                                     *
  *******************************************************************/
 
-#define LPFC_DRIVER_VERSION "12.8.0.1"
+#define LPFC_DRIVER_VERSION "12.8.0.2"
 #define LPFC_DRIVER_NAME		"lpfc"
 
 /* Used for SLI 2/3 */
Signature · 1 line
2.25.0
#17 lore raw
In reply to James Smart
Quoted 8 lines · James Smart

Update lpfc to revision 12.8.0.2

Patch set contains fixes as a couple of small additions, and two larger additions - blk_io_poll support and a different log message facility.

The patches were cut against Martin's 5.9/scsi-queue tree

[...]

Applied to 5.9/scsi-queue, thanks!

[01/14] scsi: lpfc: Fix unused assignment in lpfc_sli4_bsg_link_diag_test

        https://git.kernel.org/mkp/scsi/c/e5fcb81d40d4
[02/14] scsi: lpfc: Fix missing MDS functionality
        https://git.kernel.org/mkp/scsi/c/c93764a65b4b
[04/14] scsi: lpfc: Fix NVMe rport deregister and registration during ADISC
        https://git.kernel.org/mkp/scsi/c/9806c984d43a
[05/14] scsi: lpfc: Fix oops due to overrun when reading SLI3 data
        https://git.kernel.org/mkp/scsi/c/d91e3abb682b
[06/14] scsi: lpfc: Fix stack trace seen while setting rrq active
        https://git.kernel.org/mkp/scsi/c/9dace1fa91ca
[07/14] scsi: lpfc: Fix shost refcount mismatch when deleting vport
        https://git.kernel.org/mkp/scsi/c/03dbfe0668e6
[08/14] scsi: lpfc: Fix kdump hang on PPC
        https://git.kernel.org/mkp/scsi/c/86ee57a97a17
[09/14] scsi: lpfc: Fix language in 0373 message to reflect non-error message
        https://git.kernel.org/mkp/scsi/c/28ed7374401b
[10/14] scsi: lpfc: Allow applications to issue Common Set Features mailbox command
        https://git.kernel.org/mkp/scsi/c/45bc44270f0c
[11/14] scsi: lpfc: Add support to display if adapter dumps are available
        https://git.kernel.org/mkp/scsi/c/f0020e428af7
[12/14] scsi: lpfc: Add blk_io_poll support for latency improvment
        https://git.kernel.org/mkp/scsi/c/317aeb83c92b
[13/14] scsi: lpfc: Add an internal trace log buffer
        https://git.kernel.org/mkp/scsi/c/372c187b8a70
[14/14] scsi: lpfc: Update lpfc version to 12.8.0.2
        https://git.kernel.org/mkp/scsi/c/3fed58b94e3d
Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering