Skip to contentSkip to conversation

nvme-fc: FPIN link integrity handling

PATCH Applied Bryan Gurney 5 participants 29 messages Aug 13, 2025 – Sep 30, 2025

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

v9 cover nvme-fc: FPIN link integrity handling

FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used if the 'optimized' paths are unavailable.
Upon port reset the paths should be put back in service as the affected hardware might have been replaced.
This patch adds a new controller flag 'NVME_CTRL_MARGINAL' which will be checked during multipath path selection, causing the path to be skipped when checking for 'optimized' paths. If no optimized paths are available the 'marginal' paths are considered for path selection alongside the 'non-optimized' paths. It also introduces a new nvme-fc callback 'nvme_fc_fpin_rcv()' to evaluate the FPIN LI TLV payload and set the 'marginal' state on all affected rports.

The testing for this patch set was performed by Bryan Gurney, using the process outlined by John Meneghini's presentation at LSFMM 2024, where the fibre channel switch sends an FPIN notification on a specific switch port, and the following is checked on the initiator:

1. The controllers corresponding to the paths on the port that has received the notification are showing a set NVME_CTRL_MARGINAL flag.

   \
    +- nvme4 fc traddr=c,host_traddr=e live optimized
    +- nvme5 fc traddr=8,host_traddr=e live non-optimized
    +- nvme8 fc traddr=e,host_traddr=f marginal optimized
    +- nvme9 fc traddr=a,host_traddr=f marginal non-optimized

2. The I/O statistics of the test namespace show no I/O activity on the controllers with NVME_CTRL_MARGINAL set.

Device tps MB_read/s MB_wrtn/s MB_dscd/s

   nvme4c4n1         0.00         0.00         0.00         0.00
   nvme4c5n1     25001.00         0.00        97.66         0.00
   nvme4c9n1     25000.00         0.00        97.66         0.00
   nvme4n1       50011.00         0.00       195.36         0.00

Device tps MB_read/s MB_wrtn/s MB_dscd/s

   nvme4c4n1         0.00         0.00         0.00         0.00
   nvme4c5n1     48360.00         0.00       188.91         0.00
   nvme4c9n1      1642.00         0.00         6.41         0.00
   nvme4n1       49981.00         0.00       195.24         0.00

Device tps MB_read/s MB_wrtn/s MB_dscd/s

   nvme4c4n1         0.00         0.00         0.00         0.00
   nvme4c5n1     50001.00         0.00       195.32         0.00
   nvme4c9n1         0.00         0.00         0.00         0.00
   nvme4n1       50016.00         0.00       195.38         0.00

Testing has been performed by sending all FPIN LI ELS messages from the switch to the Host and verifying the proper nvme multi-pathing behavior is effected with each of the eight different FPIN link integrity events. Results were verified with iostat and with the nvme list-subsys command.

These tests were run with all scenarios including where there were only non-optimized paths available, and where all paths were marginal/degraded. All multi-path io-policies were tested including:
numa, round-robin and queue-depth. When all paths on the host are marginal/degraded, I/O continues on the optimized path that was most recently non-marginal. If both of the optimized paths are down, I/O properly continues on one of the marginal/degraded non-optimized paths.

Testing has been complete with both Broadcom (lpfc) and Marvell (qla2xx) 32GB HBAs. Both HBAs successfully complete all tests.

For a complete description of the tests that were run, please see bugzilla 20329.

Changes to the original submission:
- Changed flag name to 'marginal'
- Do not block marginal path; influence path selection instead to de-prioritize marginal paths

Changes to v2:
- Split off driver-specific modifications
- Introduce 'union fc_tlv_desc' to avoid casts

Changes to v3:
- Include reviews from Justin Tee
- Split marginal path handling patch

Changes to v4:
- Change 'u8' to '__u8' on fc_tlv_desc to fix a failure to build
- Print 'marginal' instead of 'live' in the state of controllers when they are marginal

Changes to v5:
- Minor spelling corrections to patch descriptions

Changes to v6:
- No code changes; added note about additional testing

Changes to v7:
- Split nvme core marginal flag addition into its own patch
- Add patch for queue_depth marginal path support

Changes to v8:
- Rebased patch series to nvme-6.17.
- Added patch from Gustavo Silva, "scsi: qla2xxx: Fix memcpy field-spanning write issue", which resolves the field-spanning write issue
- We decided to leave the "marginal" state as is, because the transport driver uses the term "marginal".

This patch series is based upon nvme-6.17.

Bryan Gurney (2):
nvme: add NVME_CTRL_MARGINAL flag
nvme: sysfs: emit the marginal path state in show_state()

Gustavo A. R. Silva (1):
scsi: qla2xxx: Fix memcpy field-spanning write issue

Hannes Reinecke (5):
fc_els: use 'union fc_tlv_desc'
nvme-fc: marginal path handling
nvme-fc: nvme_fc_fpin_rcv() callback
lpfc: enable FPIN notification for NVMe
qla2xxx: enable FPIN notification for NVMe

John Meneghini (1):
nvme-multipath: queue-depth support for marginal paths

 drivers/nvme/host/core.c         |   1 +
 drivers/nvme/host/fc.c           |  99 +++++++++++++++++++
 drivers/nvme/host/multipath.c    |  24 +++--
 drivers/nvme/host/nvme.h         |   6 ++
 drivers/nvme/host/sysfs.c        |   4 +-
 drivers/scsi/lpfc/lpfc_els.c     |  84 ++++++++--------
 drivers/scsi/qla2xxx/qla_def.h   |  10 +-
 drivers/scsi/qla2xxx/qla_isr.c   |  20 ++--
 drivers/scsi/qla2xxx/qla_nvme.c  |   2 +-
 drivers/scsi/qla2xxx/qla_os.c    |   5 +-
 drivers/scsi/scsi_transport_fc.c |  27 +++--
 include/linux/nvme-fc-driver.h   |   3 +
 include/uapi/scsi/fc/fc_els.h    | 165 +++++++++++++++++--------------
 13 files changed, 293 insertions(+), 157 deletions(-)
Signature · 1 line
2.50.1
#1 lore raw

PATCH v9 1/9 fc_els: use 'union fc_tlv_desc'

From: Hannes Reinecke <[email protected]>

Introduce 'union fc_tlv_desc' to have a common structure for all FC ELS TLV structures and avoid type casts.

[bgurney: The cast inside the union fc_tlv_next_desc() has "u8", which causes a failure to build. Use "__u8" instead.]

---
 drivers/scsi/lpfc/lpfc_els.c     |  75 +++++++-------
 drivers/scsi/scsi_transport_fc.c |  27 +++--
 include/uapi/scsi/fc/fc_els.h    | 165 +++++++++++++++++--------------
 3 files changed, 135 insertions(+), 132 deletions(-)
drivers/scsi/lpfc/lpfc_els.c +34 -41
index b1a61eca8295..c7cbc5b50dfe 100644
@@ -3937,7 +3937,7 @@ lpfc_cmpl_els_edc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 {
 	IOCB_t *irsp_iocb;
 	struct fc_els_edc_resp *edc_rsp;
-	struct fc_tlv_desc *tlv;
+	union fc_tlv_desc *tlv;
 	struct fc_diag_cg_sig_desc *pcgd;
 	struct fc_diag_lnkflt_desc *plnkflt;
 	struct lpfc_dmabuf *pcmd, *prsp;
@@ -4028,7 +4028,7 @@ lpfc_cmpl_els_edc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 			goto out;
 		}
 
-		dtag = be32_to_cpu(tlv->desc_tag);
+		dtag = be32_to_cpu(tlv->hdr.desc_tag);
 		switch (dtag) {
 		case ELS_DTAG_LNK_FAULT_CAP:
 			if (bytes_remain < FC_TLV_DESC_SZ_FROM_LENGTH(tlv) ||
@@ -4043,7 +4043,7 @@ lpfc_cmpl_els_edc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 					sizeof(struct fc_diag_lnkflt_desc));
 				goto out;
 			}
-			plnkflt = (struct fc_diag_lnkflt_desc *)tlv;
+			plnkflt = &tlv->lnkflt;
 			lpfc_printf_log(phba, KERN_INFO,
 				LOG_ELS | LOG_LDS_EVENT,
 				"4617 Link Fault Desc Data: 0x%08x 0x%08x "
@@ -4070,7 +4070,7 @@ lpfc_cmpl_els_edc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 				goto out;
 			}
 
-			pcgd = (struct fc_diag_cg_sig_desc *)tlv;
+			pcgd = &tlv->cg_sig;
 			lpfc_printf_log(
 				phba, KERN_INFO, LOG_ELS | LOG_CGN_MGMT,
 				"4616 CGN Desc Data: 0x%08x 0x%08x "
@@ -4125,10 +4125,8 @@ lpfc_cmpl_els_edc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
 }
 
 static void
-lpfc_format_edc_lft_desc(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
+lpfc_format_edc_lft_desc(struct lpfc_hba *phba, struct fc_diag_lnkflt_desc *lft)
 {
-	struct fc_diag_lnkflt_desc *lft = (struct fc_diag_lnkflt_desc *)tlv;
-
 	lft->desc_tag = cpu_to_be32(ELS_DTAG_LNK_FAULT_CAP);
 	lft->desc_len = cpu_to_be32(
 		FC_TLV_DESC_LENGTH_FROM_SZ(struct fc_diag_lnkflt_desc));
@@ -4141,10 +4139,8 @@ lpfc_format_edc_lft_desc(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
 }
 
 static void
-lpfc_format_edc_cgn_desc(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
+lpfc_format_edc_cgn_desc(struct lpfc_hba *phba, struct fc_diag_cg_sig_desc *cgd)
 {
-	struct fc_diag_cg_sig_desc *cgd = (struct fc_diag_cg_sig_desc *)tlv;
-
 	/* We are assuming cgd was zero'ed before calling this routine */
 
 	/* Configure the congestion detection capability */
@@ -4233,7 +4229,7 @@ lpfc_issue_els_edc(struct lpfc_vport *vport, uint8_t retry)
 	struct lpfc_hba  *phba = vport->phba;
 	struct lpfc_iocbq *elsiocb;
 	struct fc_els_edc *edc_req;
-	struct fc_tlv_desc *tlv;
+	union fc_tlv_desc *tlv;
 	u16 cmdsize;
 	struct lpfc_nodelist *ndlp;
 	u8 *pcmd = NULL;
@@ -4272,13 +4268,13 @@ lpfc_issue_els_edc(struct lpfc_vport *vport, uint8_t retry)
 	tlv = edc_req->desc;
 
 	if (cgn_desc_size) {
-		lpfc_format_edc_cgn_desc(phba, tlv);
+		lpfc_format_edc_cgn_desc(phba, &tlv->cg_sig);
 		phba->cgn_sig_freq = lpfc_fabric_cgn_frequency;
 		tlv = fc_tlv_next_desc(tlv);
 	}
 
 	if (lft_desc_size)
-		lpfc_format_edc_lft_desc(phba, tlv);
+		lpfc_format_edc_lft_desc(phba, &tlv->lnkflt);
 
 	lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT,
 			 "4623 Xmit EDC to remote "
@@ -5824,7 +5820,7 @@ lpfc_issue_els_edc_rsp(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb,
 {
 	struct lpfc_hba  *phba = vport->phba;
 	struct fc_els_edc_resp *edc_rsp;
-	struct fc_tlv_desc *tlv;
+	union fc_tlv_desc *tlv;
 	struct lpfc_iocbq *elsiocb;
 	IOCB_t *icmd, *cmd;
 	union lpfc_wqe128 *wqe;
@@ -5868,10 +5864,10 @@ lpfc_issue_els_edc_rsp(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb,
 		FC_TLV_DESC_LENGTH_FROM_SZ(struct fc_els_lsri_desc));
 	edc_rsp->lsri.rqst_w0.cmd = ELS_EDC;
 	tlv = edc_rsp->desc;
-	lpfc_format_edc_cgn_desc(phba, tlv);
+	lpfc_format_edc_cgn_desc(phba, &tlv->cg_sig);
 	tlv = fc_tlv_next_desc(tlv);
 	if (lft_desc_size)
-		lpfc_format_edc_lft_desc(phba, tlv);
+		lpfc_format_edc_lft_desc(phba, &tlv->lnkflt);
 
 	lpfc_debugfs_disc_trc(vport, LPFC_DISC_TRC_ELS_RSP,
 			      "Issue EDC ACC:      did:x%x flg:x%lx refcnt %d",
@@ -9256,7 +9252,7 @@ lpfc_els_rcv_edc(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb,
 {
 	struct lpfc_hba  *phba = vport->phba;
 	struct fc_els_edc *edc_req;
-	struct fc_tlv_desc *tlv;
+	union fc_tlv_desc *tlv;
 	uint8_t *payload;
 	uint32_t *ptr, dtag;
 	const char *dtag_nm;
@@ -9299,7 +9295,7 @@ lpfc_els_rcv_edc(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb,
 			goto out;
 		}
 
-		dtag = be32_to_cpu(tlv->desc_tag);
+		dtag = be32_to_cpu(tlv->hdr.desc_tag);
 		switch (dtag) {
 		case ELS_DTAG_LNK_FAULT_CAP:
 			if (bytes_remain < FC_TLV_DESC_SZ_FROM_LENGTH(tlv) ||
@@ -9314,7 +9310,7 @@ lpfc_els_rcv_edc(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb,
 					sizeof(struct fc_diag_lnkflt_desc));
 				goto out;
 			}
-			plnkflt = (struct fc_diag_lnkflt_desc *)tlv;
+			plnkflt = &tlv->lnkflt;
 			lpfc_printf_log(phba, KERN_INFO,
 				LOG_ELS | LOG_LDS_EVENT,
 				"4626 Link Fault Desc Data: x%08x len x%x "
@@ -9351,7 +9347,7 @@ lpfc_els_rcv_edc(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb,
 			phba->cgn_sig_freq = lpfc_fabric_cgn_frequency;
 
 			lpfc_least_capable_settings(
-				phba, (struct fc_diag_cg_sig_desc *)tlv);
+				phba, &tlv->cg_sig);
 			break;
 		default:
 			dtag_nm = lpfc_get_tlv_dtag_nm(dtag);
@@ -9942,14 +9938,13 @@ lpfc_display_fpin_wwpn(struct lpfc_hba *phba, __be64 *wwnlist, u32 cnt)
 /**
  * lpfc_els_rcv_fpin_li - Process an FPIN Link Integrity Event.
  * @phba: Pointer to phba object.
- * @tlv:  Pointer to the Link Integrity Notification Descriptor.
+ * @li:  Pointer to the Link Integrity Notification Descriptor.
  *
  * This function processes a Link Integrity FPIN event by logging a message.
  **/
 static void
-lpfc_els_rcv_fpin_li(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
+lpfc_els_rcv_fpin_li(struct lpfc_hba *phba, struct fc_fn_li_desc *li)
 {
-	struct fc_fn_li_desc *li = (struct fc_fn_li_desc *)tlv;
 	const char *li_evt_str;
 	u32 li_evt, cnt;
 
@@ -9973,14 +9968,13 @@ lpfc_els_rcv_fpin_li(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
 /**
  * lpfc_els_rcv_fpin_del - Process an FPIN Delivery Event.
  * @phba: Pointer to hba object.
- * @tlv:  Pointer to the Delivery Notification Descriptor TLV
+ * @del:  Pointer to the Delivery Notification Descriptor TLV
  *
  * This function processes a Delivery FPIN event by logging a message.
  **/
 static void
-lpfc_els_rcv_fpin_del(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
+lpfc_els_rcv_fpin_del(struct lpfc_hba *phba, struct fc_fn_deli_desc *del)
 {
-	struct fc_fn_deli_desc *del = (struct fc_fn_deli_desc *)tlv;
 	const char *del_rsn_str;
 	u32 del_rsn;
 	__be32 *frame;
@@ -10011,14 +10005,14 @@ lpfc_els_rcv_fpin_del(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
 /**
  * lpfc_els_rcv_fpin_peer_cgn - Process a FPIN Peer Congestion Event.
  * @phba: Pointer to hba object.
- * @tlv:  Pointer to the Peer Congestion Notification Descriptor TLV
+ * @pc:  Pointer to the Peer Congestion Notification Descriptor TLV
  *
  * This function processes a Peer Congestion FPIN event by logging a message.
  **/
 static void
-lpfc_els_rcv_fpin_peer_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
+lpfc_els_rcv_fpin_peer_cgn(struct lpfc_hba *phba,
+			   struct fc_fn_peer_congn_desc *pc)
 {
-	struct fc_fn_peer_congn_desc *pc = (struct fc_fn_peer_congn_desc *)tlv;
 	const char *pc_evt_str;
 	u32 pc_evt, cnt;
 
@@ -10046,7 +10040,7 @@ lpfc_els_rcv_fpin_peer_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
 /**
  * lpfc_els_rcv_fpin_cgn - Process an FPIN Congestion notification
  * @phba: Pointer to hba object.
- * @tlv:  Pointer to the Congestion Notification Descriptor TLV
+ * @cgn:  Pointer to the Congestion Notification Descriptor TLV
  *
  * This function processes an FPIN Congestion Notifiction.  The notification
  * could be an Alarm or Warning.  This routine feeds that data into driver's
@@ -10055,10 +10049,9 @@ lpfc_els_rcv_fpin_peer_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
  * to the upper layer or 0 to indicate don't deliver it.
  **/
 static int
-lpfc_els_rcv_fpin_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv)
+lpfc_els_rcv_fpin_cgn(struct lpfc_hba *phba, struct fc_fn_congn_desc *cgn)
 {
 	struct lpfc_cgn_info *cp;
-	struct fc_fn_congn_desc *cgn = (struct fc_fn_congn_desc *)tlv;
 	const char *cgn_evt_str;
 	u32 cgn_evt;
 	const char *cgn_sev_str;
@@ -10161,7 +10154,7 @@ lpfc_els_rcv_fpin(struct lpfc_vport *vport, void *p, u32 fpin_length)
 {
 	struct lpfc_hba *phba = vport->phba;
 	struct fc_els_fpin *fpin = (struct fc_els_fpin *)p;
-	struct fc_tlv_desc *tlv, *first_tlv, *current_tlv;
+	union fc_tlv_desc *tlv, *first_tlv, *current_tlv;
 	const char *dtag_nm;
 	int desc_cnt = 0, bytes_remain, cnt;
 	u32 dtag, deliver = 0;
@@ -10186,7 +10179,7 @@ lpfc_els_rcv_fpin(struct lpfc_vport *vport, void *p, u32 fpin_length)
 		return;
 	}
 
-	tlv = (struct fc_tlv_desc *)&fpin->fpin_desc[0];
+	tlv = &fpin->fpin_desc[0];
 	first_tlv = tlv;
 	bytes_remain = fpin_length - offsetof(struct fc_els_fpin, fpin_desc);
 	bytes_remain = min_t(u32, bytes_remain, be32_to_cpu(fpin->desc_len));
@@ -10194,22 +10187,22 @@ lpfc_els_rcv_fpin(struct lpfc_vport *vport, void *p, u32 fpin_length)
 	/* process each descriptor separately */
 	while (bytes_remain >= FC_TLV_DESC_HDR_SZ &&
 	       bytes_remain >= FC_TLV_DESC_SZ_FROM_LENGTH(tlv)) {
-		dtag = be32_to_cpu(tlv->desc_tag);
+		dtag = be32_to_cpu(tlv->hdr.desc_tag);
 		switch (dtag) {
 		case ELS_DTAG_LNK_INTEGRITY:
-			lpfc_els_rcv_fpin_li(phba, tlv);
+			lpfc_els_rcv_fpin_li(phba, &tlv->li);
 			deliver = 1;
 			break;
 		case ELS_DTAG_DELIVERY:
-			lpfc_els_rcv_fpin_del(phba, tlv);
+			lpfc_els_rcv_fpin_del(phba, &tlv->deli);
 			deliver = 1;
 			break;
 		case ELS_DTAG_PEER_CONGEST:
-			lpfc_els_rcv_fpin_peer_cgn(phba, tlv);
+			lpfc_els_rcv_fpin_peer_cgn(phba, &tlv->peer_congn);
 			deliver = 1;
 			break;
 		case ELS_DTAG_CONGESTION:
-			deliver = lpfc_els_rcv_fpin_cgn(phba, tlv);
+			deliver = lpfc_els_rcv_fpin_cgn(phba, &tlv->congn);
 			break;
 		default:
 			dtag_nm = lpfc_get_tlv_dtag_nm(dtag);
@@ -10222,12 +10215,12 @@ lpfc_els_rcv_fpin(struct lpfc_vport *vport, void *p, u32 fpin_length)
 			return;
 		}
 		lpfc_cgn_update_stat(phba, dtag);
-		cnt = be32_to_cpu(tlv->desc_len);
+		cnt = be32_to_cpu(tlv->hdr.desc_len);
 
 		/* Sanity check descriptor length. The desc_len value does not
 		 * include space for the desc_tag and the desc_len fields.
 		 */
-		len -= (cnt + sizeof(struct fc_tlv_desc));
+		len -= (cnt + sizeof(struct fc_tlv_desc_hdr));
 		if (len < 0) {
 			dtag_nm = lpfc_get_tlv_dtag_nm(dtag);
 			lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT,
drivers/scsi/scsi_transport_fc.c +11 -16
index 6b165a3ec6de..4462f2f7b102 100644
@@ -750,13 +750,12 @@ fc_cn_stats_update(u16 event_type, struct fc_fpin_stats *stats)
  *
  */
 static void
-fc_fpin_li_stats_update(struct Scsi_Host *shost, struct fc_tlv_desc *tlv)
+fc_fpin_li_stats_update(struct Scsi_Host *shost, struct fc_fn_li_desc *li_desc)
 {
 	u8 i;
 	struct fc_rport *rport = NULL;
 	struct fc_rport *attach_rport = NULL;
 	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
-	struct fc_fn_li_desc *li_desc = (struct fc_fn_li_desc *)tlv;
 	u16 event_type = be16_to_cpu(li_desc->event_type);
 	u64 wwpn;
 
@@ -799,12 +798,11 @@ fc_fpin_li_stats_update(struct Scsi_Host *shost, struct fc_tlv_desc *tlv)
  */
 static void
 fc_fpin_delivery_stats_update(struct Scsi_Host *shost,
-			      struct fc_tlv_desc *tlv)
+			      struct fc_fn_deli_desc *dn_desc)
 {
 	struct fc_rport *rport = NULL;
 	struct fc_rport *attach_rport = NULL;
 	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
-	struct fc_fn_deli_desc *dn_desc = (struct fc_fn_deli_desc *)tlv;
 	u32 reason_code = be32_to_cpu(dn_desc->deli_reason_code);
 
 	rport = fc_find_rport_by_wwpn(shost,
@@ -830,13 +828,11 @@ fc_fpin_delivery_stats_update(struct Scsi_Host *shost,
  */
 static void
 fc_fpin_peer_congn_stats_update(struct Scsi_Host *shost,
-				struct fc_tlv_desc *tlv)
+				struct fc_fn_peer_congn_desc *pc_desc)
 {
 	u8 i;
 	struct fc_rport *rport = NULL;
 	struct fc_rport *attach_rport = NULL;
-	struct fc_fn_peer_congn_desc *pc_desc =
-	    (struct fc_fn_peer_congn_desc *)tlv;
 	u16 event_type = be16_to_cpu(pc_desc->event_type);
 	u64 wwpn;
 
@@ -876,10 +872,9 @@ fc_fpin_peer_congn_stats_update(struct Scsi_Host *shost,
  */
 static void
 fc_fpin_congn_stats_update(struct Scsi_Host *shost,
-			   struct fc_tlv_desc *tlv)
+			   struct fc_fn_congn_desc *congn)
 {
 	struct fc_host_attrs *fc_host = shost_to_fc_host(shost);
-	struct fc_fn_congn_desc *congn = (struct fc_fn_congn_desc *)tlv;
 
 	fc_cn_stats_update(be16_to_cpu(congn->event_type),
 			   &fc_host->fpin_stats);
@@ -899,32 +894,32 @@ fc_host_fpin_rcv(struct Scsi_Host *shost, u32 fpin_len, char *fpin_buf,
 		u8 event_acknowledge)
 {
 	struct fc_els_fpin *fpin = (struct fc_els_fpin *)fpin_buf;
-	struct fc_tlv_desc *tlv;
+	union fc_tlv_desc *tlv;
 	u32 bytes_remain;
 	u32 dtag;
 	enum fc_host_event_code event_code =
 		event_acknowledge ? FCH_EVT_LINK_FPIN_ACK : FCH_EVT_LINK_FPIN;
 
 	/* Update Statistics */
-	tlv = (struct fc_tlv_desc *)&fpin->fpin_desc[0];
+	tlv = &fpin->fpin_desc[0];
 	bytes_remain = fpin_len - offsetof(struct fc_els_fpin, fpin_desc);
 	bytes_remain = min_t(u32, bytes_remain, be32_to_cpu(fpin->desc_len));
 
 	while (bytes_remain >= FC_TLV_DESC_HDR_SZ &&
 	       bytes_remain >= FC_TLV_DESC_SZ_FROM_LENGTH(tlv)) {
-		dtag = be32_to_cpu(tlv->desc_tag);
+		dtag = be32_to_cpu(tlv->hdr.desc_tag);
 		switch (dtag) {
 		case ELS_DTAG_LNK_INTEGRITY:
-			fc_fpin_li_stats_update(shost, tlv);
+			fc_fpin_li_stats_update(shost, &tlv->li);
 			break;
 		case ELS_DTAG_DELIVERY:
-			fc_fpin_delivery_stats_update(shost, tlv);
+			fc_fpin_delivery_stats_update(shost, &tlv->deli);
 			break;
 		case ELS_DTAG_PEER_CONGEST:
-			fc_fpin_peer_congn_stats_update(shost, tlv);
+			fc_fpin_peer_congn_stats_update(shost, &tlv->peer_congn);
 			break;
 		case ELS_DTAG_CONGESTION:
-			fc_fpin_congn_stats_update(shost, tlv);
+			fc_fpin_congn_stats_update(shost, &tlv->congn);
 		}
 
 		bytes_remain -= FC_TLV_DESC_SZ_FROM_LENGTH(tlv);
include/uapi/scsi/fc/fc_els.h +90 -75

215 lines not on this page · raw message

Signature · 1 line
2.50.1
#2 lore raw

PATCH v9 2/9 nvme: add NVME_CTRL_MARGINAL flag

In reply to Bryan Gurney

Add a new controller flag, NVME_CTRL_MARGINAL, to help multipath I/O policies to react to a path that is set to a "marginal" state.

The flag is cleared on controller reset, which is often the case when faulty cabling or transceiver hardware is replaced.

---
 drivers/nvme/host/core.c | 1 +
 drivers/nvme/host/nvme.h | 6 ++++++
 2 files changed, 7 insertions(+)
drivers/nvme/host/core.c +1 -0
index 812c1565114f..fafbd5c9c53d 100644
@@ -5069,6 +5069,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	WRITE_ONCE(ctrl->state, NVME_CTRL_NEW);
 	ctrl->passthru_err_log_enabled = false;
 	clear_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags);
+	clear_bit(NVME_CTRL_MARGINAL, &ctrl->flags);
 	spin_lock_init(&ctrl->lock);
 	mutex_init(&ctrl->namespaces_lock);
 
drivers/nvme/host/nvme.h +6 -0
index cfd2b5b90b91..d71e6668f11c 100644
@@ -275,6 +275,7 @@ enum nvme_ctrl_flags {
 	NVME_CTRL_SKIP_ID_CNS_CS	= 4,
 	NVME_CTRL_DIRTY_CAPABILITY	= 5,
 	NVME_CTRL_FROZEN		= 6,
+	NVME_CTRL_MARGINAL		= 7,
 };
 
 struct nvme_ctrl {
@@ -417,6 +418,11 @@ static inline enum nvme_ctrl_state nvme_ctrl_state(struct nvme_ctrl *ctrl)
 	return READ_ONCE(ctrl->state);
 }
 
+static inline bool nvme_ctrl_is_marginal(struct nvme_ctrl *ctrl)
+{
+	return test_bit(NVME_CTRL_MARGINAL, &ctrl->flags);
+}
+
 enum nvme_iopolicy {
 	NVME_IOPOLICY_NUMA,
 	NVME_IOPOLICY_RR,
Signature · 1 line
2.50.1
maintainer Reviewed-by on v9 2/9
#3 lore raw

Re: [PATCH v9 2/9] nvme: add NVME_CTRL_MARGINAL flag

Quoted 12 lines · Bryan Gurney

Add a new controller flag, NVME_CTRL_MARGINAL, to help multipath I/O policies to react to a path that is set to a "marginal" state.

The flag is cleared on controller reset, which is often the case when faulty cabling or transceiver hardware is replaced.

---
  drivers/nvme/host/core.c | 1 +
  drivers/nvme/host/nvme.h | 6 ++++++
  2 files changed, 7 insertions(+)

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
#4 lore raw

PATCH v9 3/9 nvme-fc: marginal path handling

In reply to Bryan Gurney

From: Hannes Reinecke <[email protected]>

FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used if the 'optimized' paths are unavailable.
To handle this a new controller flag 'NVME_CTRL_MARGINAL' is added which will cause the multipath scheduler to skip these paths when checking for 'optimized' paths. They are, however, still eligible for non-optimized path selected. The flag is cleared upon reset as then the faulty hardware might be replaced.

---
 drivers/nvme/host/fc.c        |  4 ++++
 drivers/nvme/host/multipath.c | 17 +++++++++++------
 2 files changed, 15 insertions(+), 6 deletions(-)
drivers/nvme/host/fc.c +4 -0
index 3e12d4683ac7..bf6188b554ce 100644
@@ -786,6 +786,10 @@ nvme_fc_ctrl_connectivity_loss(struct nvme_fc_ctrl *ctrl)
 		"Reconnect", ctrl->cnum);
 
 	set_bit(ASSOC_FAILED, &ctrl->flags);
+
+	/* clear 'marginal' flag as controller will be reset */
+	clear_bit(NVME_CTRL_MARGINAL, &ctrl->flags);
+
 	nvme_reset_ctrl(&ctrl->ctrl);
 }
 
drivers/nvme/host/multipath.c +11 -6
index 3da980dc60d9..c042a9a11ce3 100644
@@ -324,11 +324,14 @@ static struct nvme_ns *__nvme_find_path(struct nvme_ns_head *head, int node)
 
 		switch (ns->ana_state) {
 		case NVME_ANA_OPTIMIZED:
-			if (distance < found_distance) {
-				found_distance = distance;
-				found = ns;
+			if (!nvme_ctrl_is_marginal(ns->ctrl)) {
+				if (distance < found_distance) {
+					found_distance = distance;
+					found = ns;
+				}
+				break;
 			}
-			break;
+			fallthrough;
 		case NVME_ANA_NONOPTIMIZED:
 			if (distance < fallback_distance) {
 				fallback_distance = distance;
@@ -381,7 +384,8 @@ static struct nvme_ns *nvme_round_robin_path(struct nvme_ns_head *head)
 
 		if (ns->ana_state == NVME_ANA_OPTIMIZED) {
 			found = ns;
-			goto out;
+			if (!nvme_ctrl_is_marginal(ns->ctrl))
+				goto out;
 		}
 		if (ns->ana_state == NVME_ANA_NONOPTIMIZED)
 			found = ns;
@@ -445,7 +449,8 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head)
 static inline bool nvme_path_is_optimized(struct nvme_ns *ns)
 {
 	return nvme_ctrl_state(ns->ctrl) == NVME_CTRL_LIVE &&
-		ns->ana_state == NVME_ANA_OPTIMIZED;
+		ns->ana_state == NVME_ANA_OPTIMIZED &&
+		!nvme_ctrl_is_marginal(ns->ctrl);
 }
 
 static struct nvme_ns *nvme_numa_path(struct nvme_ns_head *head)
Signature · 1 line
2.50.1
#5 lore raw

PATCH v9 4/9 nvme-fc: nvme_fc_fpin_rcv() callback

In reply to Bryan Gurney

From: Hannes Reinecke <[email protected]>

Add a callback nvme_fc_fpin_rcv() to evaluate the FPIN LI TLV information and set the 'marginal' path status for all affected rports.

---
 drivers/nvme/host/fc.c         | 95 ++++++++++++++++++++++++++++++++++
 include/linux/nvme-fc-driver.h |  3 ++
 2 files changed, 98 insertions(+)
drivers/nvme/host/fc.c +95 -0
index bf6188b554ce..a640310f1424 100644
@@ -3724,6 +3724,101 @@ static struct nvmf_transport_ops nvme_fc_transport = {
 	.create_ctrl	= nvme_fc_create_ctrl,
 };
 
+static struct nvme_fc_rport *nvme_fc_rport_from_wwpn(struct nvme_fc_lport *lport,
+		u64 rport_wwpn)
+{
+	struct nvme_fc_rport *rport;
+
+	list_for_each_entry(rport, &lport->endp_list, endp_list) {
+		if (!nvme_fc_rport_get(rport))
+			continue;
+		if (rport->remoteport.port_name == rport_wwpn &&
+		    rport->remoteport.port_role & FC_PORT_ROLE_NVME_TARGET)
+			return rport;
+		nvme_fc_rport_put(rport);
+	}
+	return NULL;
+}
+
+static void
+nvme_fc_fpin_li_lport_update(struct nvme_fc_lport *lport, struct fc_fn_li_desc *li)
+{
+	unsigned int i, pname_count = be32_to_cpu(li->pname_count);
+	u64 attached_wwpn = be64_to_cpu(li->attached_wwpn);
+	struct nvme_fc_rport *attached_rport;
+
+	for (i = 0; i < pname_count; i++) {
+		struct nvme_fc_rport *rport;
+		u64 wwpn = be64_to_cpu(li->pname_list[i]);
+
+		rport = nvme_fc_rport_from_wwpn(lport, wwpn);
+		if (!rport)
+			continue;
+		if (wwpn != attached_wwpn) {
+			struct nvme_fc_ctrl *ctrl;
+
+			spin_lock_irq(&rport->lock);
+			list_for_each_entry(ctrl, &rport->ctrl_list, ctrl_list)
+				set_bit(NVME_CTRL_MARGINAL, &ctrl->ctrl.flags);
+			spin_unlock_irq(&rport->lock);
+		}
+		nvme_fc_rport_put(rport);
+	}
+
+	attached_rport = nvme_fc_rport_from_wwpn(lport, attached_wwpn);
+	if (attached_rport) {
+		struct nvme_fc_ctrl *ctrl;
+
+		spin_lock_irq(&attached_rport->lock);
+		list_for_each_entry(ctrl, &attached_rport->ctrl_list, ctrl_list)
+			set_bit(NVME_CTRL_MARGINAL, &ctrl->ctrl.flags);
+		spin_unlock_irq(&attached_rport->lock);
+		nvme_fc_rport_put(attached_rport);
+	}
+}
+
+/**
+ * fc_host_fpin_rcv() - Process a received FPIN.
+ * @localport:		local port the FPIN was received on
+ * @fpin_len:		length of FPIN payload, in bytes
+ * @fpin_buf:		pointer to FPIN payload
+ * Notes:
+ *	This routine assumes no locks are held on entry.
+ */
+void
+nvme_fc_fpin_rcv(struct nvme_fc_local_port *localport,
+		 u32 fpin_len, char *fpin_buf)
+{
+	struct nvme_fc_lport *lport;
+	struct fc_els_fpin *fpin = (struct fc_els_fpin *)fpin_buf;
+	union fc_tlv_desc *tlv;
+	u32 bytes_remain;
+	u32 dtag;
+
+	if (!localport)
+		return;
+	lport = localport_to_lport(localport);
+	tlv = &fpin->fpin_desc[0];
+	bytes_remain = fpin_len - offsetof(struct fc_els_fpin, fpin_desc);
+	bytes_remain = min_t(u32, bytes_remain, be32_to_cpu(fpin->desc_len));
+
+	while (bytes_remain >= FC_TLV_DESC_HDR_SZ &&
+	       bytes_remain >= FC_TLV_DESC_SZ_FROM_LENGTH(tlv)) {
+		dtag = be32_to_cpu(tlv->hdr.desc_tag);
+		switch (dtag) {
+		case ELS_DTAG_LNK_INTEGRITY:
+			nvme_fc_fpin_li_lport_update(lport, &tlv->li);
+			break;
+		default:
+			break;
+		}
+
+		bytes_remain -= FC_TLV_DESC_SZ_FROM_LENGTH(tlv);
+		tlv = fc_tlv_next_desc(tlv);
+	}
+}
+EXPORT_SYMBOL(nvme_fc_fpin_rcv);
+
 /* Arbitrary successive failures max. With lots of subsystems could be high */
 #define DISCOVERY_MAX_FAIL	20
 
include/linux/nvme-fc-driver.h +3 -0
index 9f6acadfe0c8..bcd3b1e5a256 100644
@@ -536,6 +536,9 @@ void nvme_fc_rescan_remoteport(struct nvme_fc_remote_port *remoteport);
 int nvme_fc_set_remoteport_devloss(struct nvme_fc_remote_port *remoteport,
 			u32 dev_loss_tmo);
 
+void nvme_fc_fpin_rcv(struct nvme_fc_local_port *localport,
+		      u32 fpin_len, char *fpin_buf);
+
 /*
  * Routine called to pass a NVME-FC LS request, received by the lldd,
  * to the nvme-fc transport.
Signature · 1 line
2.50.1
#6 lore raw

PATCH v9 5/9 lpfc: enable FPIN notification for NVMe

In reply to Bryan Gurney

From: Hannes Reinecke <[email protected]>

Call 'nvme_fc_fpin_rcv()' to enable FPIN notifications for NVMe.

---
 drivers/scsi/lpfc/lpfc_els.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/scsi/lpfc/lpfc_els.c +8 -1
index c7cbc5b50dfe..146ed6fd41af 100644
@@ -33,6 +33,7 @@
 #include <scsi/scsi_transport_fc.h>
 #include <uapi/scsi/fc/fc_fs.h>
 #include <uapi/scsi/fc/fc_els.h>
+#include <linux/nvme-fc-driver.h>
 
 #include "lpfc_hw4.h"
 #include "lpfc_hw.h"
@@ -10249,9 +10250,15 @@ lpfc_els_rcv_fpin(struct lpfc_vport *vport, void *p, u32 fpin_length)
 		fpin_length += sizeof(struct fc_els_fpin); /* the entire FPIN */
 
 		/* Send every descriptor individually to the upper layer */
-		if (deliver)
+		if (deliver) {
 			fc_host_fpin_rcv(lpfc_shost_from_vport(vport),
 					 fpin_length, (char *)fpin, 0);
+#if (IS_ENABLED(CONFIG_NVME_FC))
+			if (vport->cfg_enable_fc4_type & LPFC_ENABLE_NVME)
+				nvme_fc_fpin_rcv(vport->localport,
+						 fpin_length, (char *)fpin);
+#endif
+		}
 		desc_cnt++;
 	}
 }
Signature · 1 line
2.50.1
#7 lore raw

PATCH v9 6/9 qla2xxx: enable FPIN notification for NVMe

In reply to Bryan Gurney

From: Hannes Reinecke <[email protected]>

Call 'nvme_fc_fpin_rcv()' to enable FPIN notifications for NVMe.

---
 drivers/scsi/qla2xxx/qla_isr.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/scsi/qla2xxx/qla_isr.c +3 -0
index c4c6b5c6658c..f5e40e22ad7d 100644
@@ -46,6 +46,9 @@ qla27xx_process_purex_fpin(struct scsi_qla_host *vha, struct purex_item *item)
 		       pkt, pkt_size);
 
 	fc_host_fpin_rcv(vha->host, pkt_size, (char *)pkt, 0);
+#if (IS_ENABLED(CONFIG_NVME_FC))
+	nvme_fc_fpin_rcv(vha->nvme_local_port, pkt_size, (char *)pkt);
+#endif
 }
 
 const char *const port_state_str[] = {
Signature · 1 line
2.50.1
#8 lore raw

PATCH v9 7/9 nvme: sysfs: emit the marginal path state in show_state()

In reply to Bryan Gurney

If a controller has received a link integrity or congestion event, and has the NVME_CTRL_MARGINAL flag set, emit "marginal" in the state instead of "live", to identify the marginal paths.

---
 drivers/nvme/host/sysfs.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/nvme/host/sysfs.c +3 -1
index 29430949ce2f..4a6135c2f9cb 100644
@@ -430,7 +430,9 @@ static ssize_t nvme_sysfs_show_state(struct device *dev,
 	};
 
 	if (state < ARRAY_SIZE(state_name) && state_name[state])
-		return sysfs_emit(buf, "%s\n", state_name[state]);
+		return sysfs_emit(buf, "%s\n",
+			(nvme_ctrl_is_marginal(ctrl)) ? "marginal" :
+			state_name[state]);
 
 	return sysfs_emit(buf, "unknown state\n");
 }
Signature · 1 line
2.50.1
#9 lore raw

PATCH v9 8/9 nvme-multipath: queue-depth support for marginal paths

In reply to Bryan Gurney

From: John Meneghini <[email protected]>

Exclude marginal paths from queue-depth io policy. In the case where all paths are marginal and no optimized or non-optimized path is found, we fall back to __nvme_find_path which selects the best marginal path.

---
 drivers/nvme/host/multipath.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/nvme/host/multipath.c +6 -1
index c042a9a11ce3..38e40dd88e52 100644
@@ -420,6 +420,9 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head)
 		if (nvme_path_is_disabled(ns))
 			continue;
 
+		if (nvme_ctrl_is_marginal(ns->ctrl))
+			continue;
+
 		depth = atomic_read(&ns->ctrl->nr_active);
 
 		switch (ns->ana_state) {
@@ -443,7 +446,9 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head)
 			return best_opt;
 	}
 
-	return best_opt ? best_opt : best_nonopt;
+	best_opt = (best_opt) ? best_opt : best_nonopt;
+
+	return best_opt ? best_opt : __nvme_find_path(head, numa_node_id());
 }
 
 static inline bool nvme_path_is_optimized(struct nvme_ns *ns)
Signature · 1 line
2.50.1
maintainer Reviewed-by on v9 8/9
#10 lore raw

Re: [PATCH v9 8/9] nvme-multipath: queue-depth support for marginal paths

Quoted 6 lines · Bryan Gurney

From: John Meneghini <[email protected]>

Exclude marginal paths from queue-depth io policy. In the case where all paths are marginal and no optimized or non-optimized path is found, we fall back to __nvme_find_path which selects the best marginal path.

Quoted 6 lines
---
  drivers/nvme/host/multipath.c | 7 ++++++-
  1 file changed, 6 insertions(+), 1 deletion(-)

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
#11 lore raw

PATCH v9 9/9 scsi: qla2xxx: Fix memcpy field-spanning write issue

In reply to Bryan Gurney

From: "Gustavo A. R. Silva" <[email protected]>

purex_item.iocb is defined as a 64-element u8 array, but 64 is the minimum size and it can be allocated larger. This makes it a standard empty flex array.

This was motivated by field-spanning write warnings during FPIN testing.
https://lore.kernel.org/linux-nvme/[email protected]/

Quoted 3 lines

kernel: memcpy: detected field-spanning write (size 60) of single field "((uint8_t *)fpin_pkt + buffer_copy_offset)"
at drivers/scsi/qla2xxx/qla_isr.c:1221 (size 44)

I removed the outer wrapper from the iocb flex array, so that it can be linked to purex_item.size with __counted_by.

These changes remove the default minimum 64-byte allocation, requiring further changes.

In struct scsi_qla_host the embedded default_item is now followed by __default_item_iocb[QLA_DEFAULT_PAYLOAD_SIZE] to reserve space that will be used as default_item.iocb. This is wrapped using the TRAILING_OVERLAP() macro helper, which effectively creates a union between flexible-array member default_item.iocb and __default_item_iocb.

Since struct pure_item now contains a flexible-array member, the helper must be placed at the end of struct scsi_qla_host to prevent a -Wflex-array-member-not-at-end warning.

qla24xx_alloc_purex_item() is adjusted to no longer expect the default minimum size to be part of sizeof(struct purex_item), the entire flexible array size is added to the structure size for allocation.

This also slightly changes the layout of the purex_item struct, as 2-bytes of padding are added between size and iocb. The resulting size is the same, but iocb is shifted 2-bytes (the original purex_item structure was padded at the end, after the 64-byte defined array size). I don't think this is a problem.

---
 drivers/scsi/qla2xxx/qla_def.h  | 10 ++++++----
 drivers/scsi/qla2xxx/qla_isr.c  | 17 ++++++++---------
 drivers/scsi/qla2xxx/qla_nvme.c |  2 +-
 drivers/scsi/qla2xxx/qla_os.c   |  5 +++--
 4 files changed, 18 insertions(+), 16 deletions(-)
drivers/scsi/qla2xxx/qla_def.h +6 -4
index cb95b7b12051..604e66bead1e 100644
@@ -4890,9 +4890,7 @@ struct purex_item {
 			     struct purex_item *pkt);
 	atomic_t in_use;
 	uint16_t size;
-	struct {
-		uint8_t iocb[64];
-	} iocb;
+	uint8_t iocb[] __counted_by(size);
 };
 
 #include "qla_edif.h"
@@ -5101,7 +5099,6 @@ typedef struct scsi_qla_host {
 		struct list_head head;
 		spinlock_t lock;
 	} purex_list;
-	struct purex_item default_item;
 
 	struct name_list_extended gnl;
 	/* Count of active session/fcport */
@@ -5130,6 +5127,11 @@ typedef struct scsi_qla_host {
 #define DPORT_DIAG_IN_PROGRESS                 BIT_0
 #define DPORT_DIAG_CHIP_RESET_IN_PROGRESS      BIT_1
 	uint16_t dport_status;
+
+	/* Must be last --ends in a flexible-array member. */
+	TRAILING_OVERLAP(struct purex_item, default_item, iocb,
+		uint8_t __default_item_iocb[QLA_DEFAULT_PAYLOAD_SIZE];
+	);
 } scsi_qla_host_t;
 
 struct qla27xx_image_status {
drivers/scsi/qla2xxx/qla_isr.c +8 -9
index f5e40e22ad7d..b90879e844aa 100644
@@ -1080,17 +1080,17 @@ static struct purex_item *
 qla24xx_alloc_purex_item(scsi_qla_host_t *vha, uint16_t size)
 {
 	struct purex_item *item = NULL;
-	uint8_t item_hdr_size = sizeof(*item);
 
 	if (size > QLA_DEFAULT_PAYLOAD_SIZE) {
-		item = kzalloc(item_hdr_size +
-		    (size - QLA_DEFAULT_PAYLOAD_SIZE), GFP_ATOMIC);
+		item = kzalloc(struct_size(item, iocb, size), GFP_ATOMIC);
 	} else {
 		if (atomic_inc_return(&vha->default_item.in_use) == 1) {
 			item = &vha->default_item;
 			goto initialize_purex_header;
 		} else {
-			item = kzalloc(item_hdr_size, GFP_ATOMIC);
+			item = kzalloc(
+				struct_size(item, iocb, QLA_DEFAULT_PAYLOAD_SIZE),
+				GFP_ATOMIC);
 		}
 	}
 	if (!item) {
@@ -1130,17 +1130,16 @@ qla24xx_queue_purex_item(scsi_qla_host_t *vha, struct purex_item *pkt,
  * @vha: SCSI driver HA context
  * @pkt: ELS packet
  */
-static struct purex_item
-*qla24xx_copy_std_pkt(struct scsi_qla_host *vha, void *pkt)
+static struct purex_item *
+qla24xx_copy_std_pkt(struct scsi_qla_host *vha, void *pkt)
 {
 	struct purex_item *item;
 
-	item = qla24xx_alloc_purex_item(vha,
-					QLA_DEFAULT_PAYLOAD_SIZE);
+	item = qla24xx_alloc_purex_item(vha, QLA_DEFAULT_PAYLOAD_SIZE);
 	if (!item)
 		return item;
 
-	memcpy(&item->iocb, pkt, sizeof(item->iocb));
+	memcpy(&item->iocb, pkt, QLA_DEFAULT_PAYLOAD_SIZE);
 	return item;
 }
 
drivers/scsi/qla2xxx/qla_nvme.c +1 -1
index 8ee2e337c9e1..92488890bc04 100644
@@ -1308,7 +1308,7 @@ void qla2xxx_process_purls_iocb(void **pkt, struct rsp_que **rsp)
 
 	ql_dbg(ql_dbg_unsol, vha, 0x2121,
 	       "PURLS OP[%01x] size %d xchg addr 0x%x portid %06x\n",
-	       item->iocb.iocb[3], item->size, uctx->exchange_address,
+	       item->iocb[3], item->size, uctx->exchange_address,
 	       fcport->d_id.b24);
 	/* +48    0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F
 	 * ----- -----------------------------------------------
drivers/scsi/qla2xxx/qla_os.c +3 -2
index d4b484c0fd9d..253f802605d6 100644
@@ -6459,9 +6459,10 @@ void qla24xx_process_purex_rdp(struct scsi_qla_host *vha,
 void
 qla24xx_free_purex_item(struct purex_item *item)
 {
-	if (item == &item->vha->default_item)
+	if (item == &item->vha->default_item) {
 		memset(&item->vha->default_item, 0, sizeof(struct purex_item));
-	else
+		memset(&item->vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+	} else
 		kfree(item);
 }
 
Signature · 1 line
2.50.1
maintainer Reviewed-by on v9 9/9
#12 lore raw

Re: [PATCH v9 9/9] scsi: qla2xxx: Fix memcpy field-spanning write issue

Quoted 44 lines · Bryan Gurney

From: "Gustavo A. R. Silva" <[email protected]>

purex_item.iocb is defined as a 64-element u8 array, but 64 is the minimum size and it can be allocated larger. This makes it a standard empty flex array.

This was motivated by field-spanning write warnings during FPIN testing.
https://lore.kernel.org/linux-nvme/[email protected]/

Quoted 3 lines

kernel: memcpy: detected field-spanning write (size 60) of single field "((uint8_t *)fpin_pkt + buffer_copy_offset)"
at drivers/scsi/qla2xxx/qla_isr.c:1221 (size 44)

I removed the outer wrapper from the iocb flex array, so that it can be linked to purex_item.size with __counted_by.

These changes remove the default minimum 64-byte allocation, requiring further changes.

In struct scsi_qla_host the embedded default_item is now followed by __default_item_iocb[QLA_DEFAULT_PAYLOAD_SIZE] to reserve space that will be used as default_item.iocb. This is wrapped using the TRAILING_OVERLAP() macro helper, which effectively creates a union between flexible-array member default_item.iocb and __default_item_iocb.

Since struct pure_item now contains a flexible-array member, the helper must be placed at the end of struct scsi_qla_host to prevent a -Wflex-array-member-not-at-end warning.

qla24xx_alloc_purex_item() is adjusted to no longer expect the default minimum size to be part of sizeof(struct purex_item), the entire flexible array size is added to the structure size for allocation.

This also slightly changes the layout of the purex_item struct, as 2-bytes of padding are added between size and iocb. The resulting size is the same, but iocb is shifted 2-bytes (the original purex_item structure was padded at the end, after the 64-byte defined array size). I don't think this is a problem.

Quoted 7 lines
---
  drivers/scsi/qla2xxx/qla_def.h  | 10 ++++++----
  drivers/scsi/qla2xxx/qla_isr.c  | 17 ++++++++---------
  drivers/scsi/qla2xxx/qla_nvme.c |  2 +-
  drivers/scsi/qla2xxx/qla_os.c   |  5 +++--
  4 files changed, 18 insertions(+), 16 deletions(-)

Not sure if this shouldn't be a stand-alone patch, but anyway:

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
#13 lore raw

Re: [PATCH v9 9/9] scsi: qla2xxx: Fix memcpy field-spanning write issue

In reply to Bryan Gurney

Bryan,

Quoted 3 lines

purex_item.iocb is defined as a 64-element u8 array, but 64 is the minimum size and it can be allocated larger. This makes it a standard empty flex array.

Applied to 6.18/scsi-staging, thanks!

Signature · 1 line
Martin K. Petersen
Reviewed-by on the v9 series
#14 lore raw
In reply to Bryan Gurney

Bryan,

Quoted 4 lines

FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used if the 'optimized' paths are unavailable.

SCSI bits look OK to me. I am fine with this series going through the NVMe tree.

Signature · 1 line
Martin K. Petersen
#15 lore raw

Re: (subset) [PATCH v9 0/8] nvme-fc: FPIN link integrity handling

In reply to Bryan Gurney
Quoted 15 lines · Bryan Gurney

FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used if the 'optimized' paths are unavailable.
Upon port reset the paths should be put back in service as the affected hardware might have been replaced.
This patch adds a new controller flag 'NVME_CTRL_MARGINAL' which will be checked during multipath path selection, causing the path to be skipped when checking for 'optimized' paths. If no optimized paths are available the 'marginal' paths are considered for path selection alongside the 'non-optimized' paths. It also introduces a new nvme-fc callback 'nvme_fc_fpin_rcv()' to evaluate the FPIN LI TLV payload and set the 'marginal' state on all affected rports.

Quoted 1 line

[...]

Applied to 6.18/scsi-queue, thanks!

[9/9] scsi: qla2xxx: Fix memcpy field-spanning write issue

      https://git.kernel.org/mkp/scsi/c/6f4b10226b6b
Signature · 1 line
Martin K. Petersen
#16 lore raw

Re: (subset) [PATCH v9 0/8] nvme-fc: FPIN link integrity handling

Hey Martin.

We've been testing this patch and it turns out there is a significant bug here.

This bug causes a call trace and a driver hang.

I suggest you remove this from scsi-queue and let us fix it.

/John

drivers/scsi/qla2xxx/qla_os.c +4 -3
index 253f802605d6..44e9f7df8db2 100644
@@ -6459,9 +6459,10 @@ void qla24xx_process_purex_rdp(struct scsi_qla_host *vha,
 void
 qla24xx_free_purex_item(struct purex_item *item)
 {
-       if (item == &item->vha->default_item) {
-               memset(&item->vha->default_item, 0, sizeof(struct purex_item));
-               memset(&item->vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+       scsi_qla_host_t *base_vha = item->vha;
+       if (item == &base_vha->default_item) {
+               memset(&base_vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+               memset(&base_vha->default_item, 0, sizeof(struct purex_item));
        } else
                kfree(item);
 }
Quoted 24 lines · Martin K. Petersen
Quoted 15 lines · Bryan Gurney

FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used if the 'optimized' paths are unavailable.
Upon port reset the paths should be put back in service as the affected hardware might have been replaced.
This patch adds a new controller flag 'NVME_CTRL_MARGINAL' which will be checked during multipath path selection, causing the path to be skipped when checking for 'optimized' paths. If no optimized paths are available the 'marginal' paths are considered for path selection alongside the 'non-optimized' paths. It also introduces a new nvme-fc callback 'nvme_fc_fpin_rcv()' to evaluate the FPIN LI TLV payload and set the 'marginal' state on all affected rports.

Quoted 1 line

[...]

Applied to 6.18/scsi-queue, thanks!

[9/9] scsi: qla2xxx: Fix memcpy field-spanning write issue

       https://git.kernel.org/mkp/scsi/c/6f4b10226b6b
#17 lore raw

Re: (subset) [PATCH v9 0/8] nvme-fc: FPIN link integrity handling

John,

Quoted 6 lines

We've been testing this patch and it turns out there is a significant bug here.

This bug causes a call trace and a driver hang.

I suggest you remove this from scsi-queue and let us fix it.

I am not rebasing the tree at this point. Please submit a revert with a problem description.

Thanks!

Signature · 1 line
Martin K. Petersen
#18 lore raw

PATCH 1/1 Revert "scsi: qla2xxx: Fix memcpy() field-spanning write issue"

This reverts commit 6f4b10226b6b1e7d1ff3cdb006cf0f6da6eed71e.

We've been testing this patch and it turns out there is a significant bug here. This leaks memory and causes a driver hang.

Link:
https://lore.kernel.org/linux-scsi/[email protected]/

---
 drivers/scsi/qla2xxx/qla_def.h  | 10 ++++------
 drivers/scsi/qla2xxx/qla_isr.c  | 17 +++++++++--------
 drivers/scsi/qla2xxx/qla_nvme.c |  2 +-
 drivers/scsi/qla2xxx/qla_os.c   |  5 ++---
 4 files changed, 16 insertions(+), 18 deletions(-)
drivers/scsi/qla2xxx/qla_def.h +4 -6
index 604e66bead1e..cb95b7b12051 100644
@@ -4890,7 +4890,9 @@ struct purex_item {
 			     struct purex_item *pkt);
 	atomic_t in_use;
 	uint16_t size;
-	uint8_t iocb[] __counted_by(size);
+	struct {
+		uint8_t iocb[64];
+	} iocb;
 };
 
 #include "qla_edif.h"
@@ -5099,6 +5101,7 @@ typedef struct scsi_qla_host {
 		struct list_head head;
 		spinlock_t lock;
 	} purex_list;
+	struct purex_item default_item;
 
 	struct name_list_extended gnl;
 	/* Count of active session/fcport */
@@ -5127,11 +5130,6 @@ typedef struct scsi_qla_host {
 #define DPORT_DIAG_IN_PROGRESS                 BIT_0
 #define DPORT_DIAG_CHIP_RESET_IN_PROGRESS      BIT_1
 	uint16_t dport_status;
-
-	/* Must be last --ends in a flexible-array member. */
-	TRAILING_OVERLAP(struct purex_item, default_item, iocb,
-		uint8_t __default_item_iocb[QLA_DEFAULT_PAYLOAD_SIZE];
-	);
 } scsi_qla_host_t;
 
 struct qla27xx_image_status {
drivers/scsi/qla2xxx/qla_isr.c +9 -8
index 4559b490614d..c4c6b5c6658c 100644
@@ -1077,17 +1077,17 @@ static struct purex_item *
 qla24xx_alloc_purex_item(scsi_qla_host_t *vha, uint16_t size)
 {
 	struct purex_item *item = NULL;
+	uint8_t item_hdr_size = sizeof(*item);
 
 	if (size > QLA_DEFAULT_PAYLOAD_SIZE) {
-		item = kzalloc(struct_size(item, iocb, size), GFP_ATOMIC);
+		item = kzalloc(item_hdr_size +
+		    (size - QLA_DEFAULT_PAYLOAD_SIZE), GFP_ATOMIC);
 	} else {
 		if (atomic_inc_return(&vha->default_item.in_use) == 1) {
 			item = &vha->default_item;
 			goto initialize_purex_header;
 		} else {
-			item = kzalloc(
-				struct_size(item, iocb, QLA_DEFAULT_PAYLOAD_SIZE),
-				GFP_ATOMIC);
+			item = kzalloc(item_hdr_size, GFP_ATOMIC);
 		}
 	}
 	if (!item) {
@@ -1127,16 +1127,17 @@ qla24xx_queue_purex_item(scsi_qla_host_t *vha, struct purex_item *pkt,
  * @vha: SCSI driver HA context
  * @pkt: ELS packet
  */
-static struct purex_item *
-qla24xx_copy_std_pkt(struct scsi_qla_host *vha, void *pkt)
+static struct purex_item
+*qla24xx_copy_std_pkt(struct scsi_qla_host *vha, void *pkt)
 {
 	struct purex_item *item;
 
-	item = qla24xx_alloc_purex_item(vha, QLA_DEFAULT_PAYLOAD_SIZE);
+	item = qla24xx_alloc_purex_item(vha,
+					QLA_DEFAULT_PAYLOAD_SIZE);
 	if (!item)
 		return item;
 
-	memcpy(&item->iocb, pkt, QLA_DEFAULT_PAYLOAD_SIZE);
+	memcpy(&item->iocb, pkt, sizeof(item->iocb));
 	return item;
 }
 
drivers/scsi/qla2xxx/qla_nvme.c +1 -1
index 065f9bcca26f..316594aa40cc 100644
@@ -1308,7 +1308,7 @@ void qla2xxx_process_purls_iocb(void **pkt, struct rsp_que **rsp)
 
 	ql_dbg(ql_dbg_unsol, vha, 0x2121,
 	       "PURLS OP[%01x] size %d xchg addr 0x%x portid %06x\n",
-	       item->iocb[3], item->size, uctx->exchange_address,
+	       item->iocb.iocb[3], item->size, uctx->exchange_address,
 	       fcport->d_id.b24);
 	/* +48    0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F
 	 * ----- -----------------------------------------------
drivers/scsi/qla2xxx/qla_os.c +2 -3
index 98a5c105fdfd..9a2f328200ab 100644
@@ -6459,10 +6459,9 @@ void qla24xx_process_purex_rdp(struct scsi_qla_host *vha,
 void
 qla24xx_free_purex_item(struct purex_item *item)
 {
-	if (item == &item->vha->default_item) {
+	if (item == &item->vha->default_item)
 		memset(&item->vha->default_item, 0, sizeof(struct purex_item));
-		memset(&item->vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
-	} else
+	else
 		kfree(item);
 }
 
Signature · 1 line
2.51.0
#19 lore raw

Re: [PATCH] Revert "scsi: qla2xxx: Fix memcpy() field-spanning write issue"

Quoted 7 lines · John Meneghini

This reverts commit 6f4b10226b6b1e7d1ff3cdb006cf0f6da6eed71e.

We've been testing this patch and it turns out there is a significant bug here. This leaks memory and causes a driver hang.

Link:
https://lore.kernel.org/linux-scsi/[email protected]/

Thanks for the report. I wonder if you have any logs or something I could look at to figure out what's going on.

Bryan,

Could you please share how this patch[1] was tested?

Thanks
-Gustavo

[1] https://lore.kernel.org/linux-scsi/[email protected]/

#20 lore raw

Re: [PATCH] Revert "scsi: qla2xxx: Fix memcpy() field-spanning write issue"

Quoted 11 lines · Gustavo A. R. Silva
Quoted 7 lines · John Meneghini

This reverts commit 6f4b10226b6b1e7d1ff3cdb006cf0f6da6eed71e.

We've been testing this patch and it turns out there is a significant bug here. This leaks memory and causes a driver hang.

Link:
https://lore.kernel.org/linux-scsi/[email protected]/

Thanks for the report. I wonder if you have any logs or something I could look at to figure out what's going on.

We have a fix already. Chris and Bryan figured it out.

Quoted 3 lines

Bryan,

Could you please share how this patch[1] was tested?

Bryan, please reply with bug fix patch you emailed me yesterday as an RFC patch.

Gustavo, this patch is being tested as a part of our FPIN LI changes. To run this code you need a Brocade switch and a whole lot of hardware.

You can see a example test plan here: https://bugzilla.kernel.org/attachment.cgi?id=308368&action=view

I am about to submit a version 10 patch series for these changes and I will include a new/fixed version of your patch in that series.

/John

Quoted 5 lines

Thanks
-Gustavo

[1] https://lore.kernel.org/linux-scsi/[email protected]/

#21 lore raw

Re: [PATCH] Revert "scsi: qla2xxx: Fix memcpy() field-spanning write issue"

Quoted 20 lines · John Meneghini
Quoted 11 lines · Gustavo A. R. Silva
Quoted 7 lines · John Meneghini

This reverts commit 6f4b10226b6b1e7d1ff3cdb006cf0f6da6eed71e.

We've been testing this patch and it turns out there is a significant bug here. This leaks memory and causes a driver hang.

Link:
https://lore.kernel.org/linux-scsi/[email protected]/

Thanks for the report. I wonder if you have any logs or something I could look at to figure out what's going on.

We have a fix already.  Chris and Bryan figured it out.

Quoted 3 lines

Bryan,

Could you please share how this patch[1] was tested?

Quoted 7 lines

Bryan, please reply with bug fix patch you emailed me yesterday as an RFC patch.

Gustavo, this patch is being tested as a part of our FPIN LI changes. To run this code you need a Brocade switch and a whole lot of hardware.

You can see a example test plan here: https://bugzilla.kernel.org/attachment.cgi?id=308368&action=view

I am about to submit a version 10 patch series for these changes and I will include a new/fixed version of your patch in that series.

Awesome, thank you!

I was in the process of writing the following (draft) patch, which is much less intrusive than the other one:

drivers/scsi/qla2xxx/qla_def.h +7 -4
index cb95b7b12051..1b000709ccd8 100644
@@ -4890,9 +4890,10 @@ struct purex_item {
                             struct purex_item *pkt);
        atomic_t in_use;
        uint16_t size;
-       struct {
-               uint8_t iocb[64];
-       } iocb;
+       union {
+               uint8_t min_iocb[QLA_DEFAULT_PAYLOAD_SIZE];
+               DECLARE_FLEX_ARRAY(uint8_t, iocb);
+       };
 };
 
 #include "qla_edif.h"
@@ -5101,7 +5102,6 @@ typedef struct scsi_qla_host {
                struct list_head head;
                spinlock_t lock;
        } purex_list;
-       struct purex_item default_item;
 
        struct name_list_extended gnl;
        /* Count of active session/fcport */
@@ -5130,6 +5130,9 @@ typedef struct scsi_qla_host {
 #define DPORT_DIAG_IN_PROGRESS                 BIT_0
 #define DPORT_DIAG_CHIP_RESET_IN_PROGRESS      BIT_1
        uint16_t dport_status;
+
+       /* Must be last --ends in a flexible-array member. */
+       struct purex_item default_item;
 } scsi_qla_host_t;
 
 struct qla27xx_image_status {
drivers/scsi/qla2xxx/qla_isr.c +1 -1
index c4c6b5c6658c..a342e137a53a 100644
@@ -1137,7 +1137,7 @@ static struct purex_item
        if (!item)
                return item;
 
-       memcpy(&item->iocb, pkt, sizeof(item->iocb));
+       memcpy(&item->iocb, pkt, QLA_DEFAULT_PAYLOAD_SIZE);
        return item;
 }
 
drivers/scsi/qla2xxx/qla_nvme.c +1 -1
index 316594aa40cc..065f9bcca26f 100644
@@ -1308,7 +1308,7 @@ void qla2xxx_process_purls_iocb(void **pkt, struct rsp_que **rsp)
 
        ql_dbg(ql_dbg_unsol, vha, 0x2121,
               "PURLS OP[%01x] size %d xchg addr 0x%x portid %06x\n",
-              item->iocb.iocb[3], item->size, uctx->exchange_address,
+              item->iocb[3], item->size, uctx->exchange_address,
               fcport->d_id.b24);
        /* +48    0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F
         * ----- -----------------------------------------------

But if you already figured it out, that's great. :)

Thanks
-Gustavo

Quoted 8 lines

/John

Quoted 5 lines

Thanks
-Gustavo

[1] https://lore.kernel.org/linux-scsi/[email protected]/

#22 lore raw

RFC PATCH 1/1 scsi: qla2xxx: zero default_item last in qla24xx_free_purex_item

In reply to John Meneghini

In order to avoid a null pointer dereference, the vha->default_item should be set to 0 last if the item pointer passed to the function matches.

BUG: kernel NULL pointer dereference, address: 0000000000000936
...
RIP: 0010:qla24xx_free_purex_item+0x5e/0x90 [qla2xxx]
...
Call Trace:
 <TASK>
 qla24xx_process_purex_list+0xda/0x110 [qla2xxx]
 qla2x00_do_dpc+0x8ac/0xab0 [qla2xxx]
 ? __pfx_qla2x00_do_dpc+0x10/0x10 [qla2xxx]
 kthread+0xf9/0x240
 ? __pfx_kthread+0x10/0x10
 ret_from_fork+0xf1/0x110
 ? __pfx_kthread+0x10/0x10

Also use a local variable to avoid multiple de-referencing of the item.

Fixes 6f4b10226b6b ("scsi: qla2xxx: Fix memcpy() field-spanning write issue")
---
 drivers/scsi/qla2xxx/qla_os.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +5 -3
index 98a5c105fdfd..7e28c7e9aa60 100644
@@ -6459,9 +6459,11 @@ void qla24xx_process_purex_rdp(struct scsi_qla_host *vha,
 void
 qla24xx_free_purex_item(struct purex_item *item)
 {
-	if (item == &item->vha->default_item) {
-		memset(&item->vha->default_item, 0, sizeof(struct purex_item));
-		memset(&item->vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+	scsi_qla_host_t *base_vha = item->vha;
+
+	if (item == &base_vha->default_item) {
+		memset(&base_vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+		memset(&base_vha->default_item, 0, sizeof(struct purex_item));
 	} else
 		kfree(item);
 }
Signature · 1 line
2.51.0
#23 lore raw

Re: [PATCH RFC] scsi: qla2xxx: zero default_item last in qla24xx_free_purex_item

Quoted 35 lines · Bryan Gurney

In order to avoid a null pointer dereference, the vha->default_item should be set to 0 last if the item pointer passed to the function matches.

BUG: kernel NULL pointer dereference, address: 0000000000000936
...
RIP: 0010:qla24xx_free_purex_item+0x5e/0x90 [qla2xxx]
...
Call Trace:
  <TASK>
  qla24xx_process_purex_list+0xda/0x110 [qla2xxx]
  qla2x00_do_dpc+0x8ac/0xab0 [qla2xxx]
  ? __pfx_qla2x00_do_dpc+0x10/0x10 [qla2xxx]
  kthread+0xf9/0x240
  ? __pfx_kthread+0x10/0x10
  ret_from_fork+0xf1/0x110
  ? __pfx_kthread+0x10/0x10

Also use a local variable to avoid multiple de-referencing of the item.

Fixes 6f4b10226b6b ("scsi: qla2xxx: Fix memcpy() field-spanning write issue")
---
  drivers/scsi/qla2xxx/qla_os.c | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +0 -1
index 98a5c105fdfd..7e28c7e9aa60 100644
@@ -6459,9 +6459,11 @@ void qla24xx_process_purex_rdp(struct scsi_qla_host *vha,
  void
  qla24xx_free_purex_item(struct purex_item *item)
  {
-	if (item == &item->vha->default_item) {

drivers/scsi/qla2xxx/qla_os.c void qla24xx_process_purex_rdp(struct scsi_qla_host *vha, · line 6469

Quoted 10 lines
drivers/scsi/qla2xxx/qla_os.c +5 -2
-		memset(&item->vha->default_item, 0, sizeof(struct purex_item));
-		memset(&item->vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+	scsi_qla_host_t *base_vha = item->vha;
+
+	if (item == &base_vha->default_item) {
+		memset(&base_vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+		memset(&base_vha->default_item, 0, sizeof(struct purex_item));
  	} else
  		kfree(item);
  }

I see. I think it's probably better to go ahead with the revert, and then apply the patch I proposed in my previous e-mail (it's more straightforward and introduces fewer changes).

If you agree with that, I can submit both the revert and the patch.

Thanks
-Gustavo

#24 lore raw

Re: [PATCH RFC] scsi: qla2xxx: zero default_item last in qla24xx_free_purex_item

Quoted 54 lines · Gustavo A. R. Silva
Quoted 35 lines · Bryan Gurney

In order to avoid a null pointer dereference, the vha->default_item should be set to 0 last if the item pointer passed to the function matches.

BUG: kernel NULL pointer dereference, address: 0000000000000936
...
RIP: 0010:qla24xx_free_purex_item+0x5e/0x90 [qla2xxx]
...
Call Trace:
  <TASK>
  qla24xx_process_purex_list+0xda/0x110 [qla2xxx]
  qla2x00_do_dpc+0x8ac/0xab0 [qla2xxx]
  ? __pfx_qla2x00_do_dpc+0x10/0x10 [qla2xxx]
  kthread+0xf9/0x240
  ? __pfx_kthread+0x10/0x10
  ret_from_fork+0xf1/0x110
  ? __pfx_kthread+0x10/0x10

Also use a local variable to avoid multiple de-referencing of the item.

Fixes 6f4b10226b6b ("scsi: qla2xxx: Fix memcpy() field-spanning write issue")
---
  drivers/scsi/qla2xxx/qla_os.c | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)
drivers/scsi/qla2xxx/qla_os.c +0 -1
index 98a5c105fdfd..7e28c7e9aa60 100644
@@ -6459,9 +6459,11 @@ void qla24xx_process_purex_rdp(struct scsi_qla_host *vha,
  void
  qla24xx_free_purex_item(struct purex_item *item)
  {
-     if (item == &item->vha->default_item) {

drivers/scsi/qla2xxx/qla_os.c void qla24xx_process_purex_rdp(struct scsi_qla_host *vha, · line 6469

Quoted 10 lines
drivers/scsi/qla2xxx/qla_os.c +5 -2
-             memset(&item->vha->default_item, 0, sizeof(struct purex_item));
-             memset(&item->vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+     scsi_qla_host_t *base_vha = item->vha;
+
+     if (item == &base_vha->default_item) {
+             memset(&base_vha->__default_item_iocb, 0, QLA_DEFAULT_PAYLOAD_SIZE);
+             memset(&base_vha->default_item, 0, sizeof(struct purex_item));
      } else
              kfree(item);
  }

I see. I think it's probably better to go ahead with the revert, and then apply the patch I proposed in my previous e-mail (it's more straightforward and introduces fewer changes).

Quoted 5 lines

If you agree with that, I can submit both the revert and the patch.

Thanks
-Gustavo

Hi Gustavo,

I just built a kernel with your patch, on top of the NVMe FPIN link integrity v9 patch set, and a test run on qla2xxx passes without any field-spanning write warnings, nor with any null pointer dereference errors.

Thanks,

Bryan

#25 lore raw

Re: [PATCH RFC] scsi: qla2xxx: zero default_item last in qla24xx_free_purex_item

Quoted 11 lines
Quoted 10 lines

I see. I think it's probably better to go ahead with the revert, and then apply the patch I proposed in my previous e-mail (it's more straightforward and introduces fewer changes).

If you agree with that, I can submit both the revert and the patch.

Thanks
-Gustavo

Quoted 6 lines

Hi Gustavo,

I just built a kernel with your patch, on top of the NVMe FPIN link integrity v9 patch set, and a test run on qla2xxx passes without any field-spanning write warnings, nor with any null pointer dereference errors.

Awesome. Thanks so much for testing it, Bryan. :)

Okay, so I'll go ahead and ACK the revert submitted by John (so as not to duplicate efforts), and once it's applied, I'll submit the proposed patch.

Thank you, folks.
-Gustavo

Acked-by on v1 1/1
#26 lore raw

Re: [PATCH] Revert "scsi: qla2xxx: Fix memcpy() field-spanning write issue"

In reply to John Meneghini
Quoted 9 lines · John Meneghini

This reverts commit 6f4b10226b6b1e7d1ff3cdb006cf0f6da6eed71e.

We've been testing this patch and it turns out there is a significant bug here. This leaks memory and causes a driver hang.

Link:
https://lore.kernel.org/linux-scsi/[email protected]/

Thanks!
-Gustavo

#27 lore raw

Re: [PATCH] Revert "scsi: qla2xxx: Fix memcpy() field-spanning write issue"

Martin,

I don't see this patch in scsi/6.18/scsi-queue yet.

Are we going to revert this ?

/John

Quoted 18 lines · Gustavo A. R. Silva
Quoted 9 lines · John Meneghini

This reverts commit 6f4b10226b6b1e7d1ff3cdb006cf0f6da6eed71e.

We've been testing this patch and it turns out there is a significant bug here. This leaks memory and causes a driver hang.

Link:
https://lore.kernel.org/linux-scsi/[email protected]/

Thanks!
-Gustavo

#28 lore raw

Re: [PATCH] Revert "scsi: qla2xxx: Fix memcpy() field-spanning write issue"

John,

Quoted 3 lines

I don't see this patch in scsi/6.18/scsi-queue yet.

Are we going to revert this ?

It's in 6.18/scsi-staging. Stuff usually sits in staging for a week or so before getting shuffled over to queue. This lets the static checkers do their thing...

Signature · 1 line
Martin K. Petersen