Skip to contentSkip to conversation

nvmet/nvmet_fc: add events for discovery controller rescan

PATCH New James Smart 3 participants 15 messages May 14, 2019 – May 30, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover nvmet/nvmet_fc: add events for discovery controller rescan

A transport may have a transport-specific mechanism that can signal when discovery controller content has changed and request a host to rescan to the discovery controller.

FC is such a transport. RSCNs may be generated by the subsystem's FC port and sent to the initiator or fabric controller. If a fabric, the fabric controller will broadcast the RSCN to registered hosts. A host, upon receiving the RSCN, would validate connectivity then initiate a discovery controller rescan.

These patches:
- Modify the nvmet core layer to call a transport callback whenever a port discovery change occurs. To facilitate the callback and avoid new routines between core. and discovery.c the port structure now has a copy of the transport ops structure.
- Modify the nvmet-fc transport to support the nvmet callback, and add its own internal lldd api to request the lldd to generate an RSCN.
- Update nvme-fcloop test harness to support the lldd api and invoke the rescan on the host.
- Modify the lpfc driver to support the new interfaces:
- Adds a new routine to transmit an RSCN to the other port (pt2pt) or fabric controller. Add recognition for receipt of an RSCN.
- Add support for the nvmet lldd api for discovery event, which invokes the RSCN transmit.
- Ties into the RSCN receipt and requests the nvme_fc transport to rescan the remote port (discovery event will be posted).
- Adds a sysfs routine to enable manual generation of an RSCN.

v2:
Revise lpfc patch 6 to check for port role nvme discovery not just
nvme target

James Smart (7):
nvmet: add transport discovery change op
nvmet_fc: add transport discovery change event callback support
nvme-fcloop: Add support for nvmet discovery_event op
lpfc: Add support to generate RSCN events for nport
lpfc: add nvmet discovery_event op support
lpfc: Add support for translating an RSCN rcv into a discovery rescan
lpfc: Add sysfs interface to post NVME RSCN

 drivers/nvme/target/core.c       |   2 +
 drivers/nvme/target/discovery.c  |   4 ++
 drivers/nvme/target/fc.c         |  11 ++++
 drivers/nvme/target/fcloop.c     |  37 ++++++++++++
 drivers/nvme/target/nvmet.h      |   2 +
 drivers/scsi/lpfc/lpfc.h         |   2 +
 drivers/scsi/lpfc/lpfc_attr.c    |  60 ++++++++++++++++++
 drivers/scsi/lpfc/lpfc_crtn.h    |   4 ++
 drivers/scsi/lpfc/lpfc_els.c     | 127 +++++++++++++++++++++++++++++++++++++++
 drivers/scsi/lpfc/lpfc_hbadisc.c |  35 +++++++++++
 drivers/scsi/lpfc/lpfc_hw.h      |   2 +
 drivers/scsi/lpfc/lpfc_nvme.c    |  44 ++++++++++++++
 drivers/scsi/lpfc/lpfc_nvmet.c   |  17 ++++++
 drivers/scsi/lpfc/lpfc_sli.c     |   1 +
 include/linux/nvme-fc-driver.h   |   6 ++
 15 files changed, 354 insertions(+)
Signature · 1 line
2.13.7
#1 lore raw

PATCH v2 1/7 nvmet: add transport discovery change op

Some transports, such as FC-NVME, support discovery controller change events without the use of a persistent discovery controller. FC receives events via RSCN from the FC Fabric Controller or subsystem FC port.

This patch adds a nvmet transport op that is called whenever a discovery change event occurs in the nvmet layer.

To facilitate the callback without adding another layer to cross into core.c to reference the transport ops, the port structure snapshots the transport ops when the port is enabled and clears them when disabled.

Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
---
 drivers/nvme/target/core.c      | 2 ++
 drivers/nvme/target/discovery.c | 4 ++++
 drivers/nvme/target/nvmet.h     | 2 ++
 3 files changed, 8 insertions(+)
drivers/nvme/target/core.c +2 -0
index 7734a6acff85..43e8c4adc1f4 100644
@@ -311,6 +311,7 @@ int nvmet_enable_port(struct nvmet_port *port)
 		port->inline_data_size = 0;
 
 	port->enabled = true;
+	port->tr_ops = ops;
 	return 0;
 }
 
@@ -321,6 +322,7 @@ void nvmet_disable_port(struct nvmet_port *port)
 	lockdep_assert_held(&nvmet_config_sem);
 
 	port->enabled = false;
+	port->tr_ops = NULL;
 
 	ops = nvmet_transports[port->disc_addr.trtype];
 	ops->remove_port(port);
drivers/nvme/target/discovery.c +4 -0
index 5baf269f3f8a..8efca26b4776 100644
@@ -41,6 +41,10 @@ void nvmet_port_disc_changed(struct nvmet_port *port,
 		__nvmet_disc_changed(port, ctrl);
 	}
 	mutex_unlock(&nvmet_disc_subsys->lock);
+
+	/* If transport can signal change, notify transport */
+	if (port->tr_ops && port->tr_ops->discovery_chg)
+		port->tr_ops->discovery_chg(port);
 }
 
 static void __nvmet_subsys_disc_changed(struct nvmet_port *port,
drivers/nvme/target/nvmet.h +2 -0
index c25d88fc9dec..dc270944bb25 100644
@@ -140,6 +140,7 @@ struct nvmet_port {
 	void				*priv;
 	bool				enabled;
 	int				inline_data_size;
+	const struct nvmet_fabrics_ops	*tr_ops;
 };
 
 static inline struct nvmet_port *to_nvmet_port(struct config_item *item)
@@ -277,6 +278,7 @@ struct nvmet_fabrics_ops {
 	void (*disc_traddr)(struct nvmet_req *req,
 			struct nvmet_port *port, char *traddr);
 	u16 (*install_queue)(struct nvmet_sq *nvme_sq);
+	void (*discovery_chg)(struct nvmet_port *port);
 };
 
 #define NVMET_MAX_INLINE_BIOVEC	8
Signature · 1 line
2.13.7
maintainer
#2 lore raw

PATCH v2 1/7 nvmet: add transport discovery change op

drivers/nvme/target/discovery.c void nvmet_port_disc_changed(struct nvmet_port *port, · line 47

Quoted 13 lines
drivers/nvme/target/discovery.c +4 -0
index 5baf269f3f8a..8efca26b4776 100644
@@ -41,6 +41,10 @@ void nvmet_port_disc_changed(struct nvmet_port *port,
  		__nvmet_disc_changed(port, ctrl);
  	}
  	mutex_unlock(&nvmet_disc_subsys->lock);
+
+	/* If transport can signal change, notify transport */
+	if (port->tr_ops && port->tr_ops->discovery_chg)
+		port->tr_ops->discovery_chg(port);

So you are shooting for both transport and disc aen to happen at the same time?

#3 lore raw

PATCH v2 1/7 nvmet: add transport discovery change op

Quoted 17 lines · Sagi Grimberg
Quoted 7 lines
drivers/nvme/target/discovery.c +0 -0
index 5baf269f3f8a..8efca26b4776 100644

drivers/nvme/target/discovery.c void nvmet_port_disc_changed(struct nvmet_port *port, · line 47

Quoted 7 lines
drivers/nvme/target/discovery.c +4 -0
@@ -41,6 +41,10 @@ void nvmet_port_disc_changed(struct nvmet_port *port,
 ????????? __nvmet_disc_changed(port, ctrl);
 ????? }
 ????? mutex_unlock(&nvmet_disc_subsys->lock);
+
+??? /* If transport can signal change, notify transport */
+??? if (port->tr_ops && port->tr_ops->discovery_chg)
+??????? port->tr_ops->discovery_chg(port);

So you are shooting for both transport and disc aen to happen at the same time?

they could if the discovery controller is a persistent one.

-- james

maintainer
#4 lore raw

PATCH v2 1/7 nvmet: add transport discovery change op

Quoted 18 lines
Quoted 17 lines
Quoted 7 lines
drivers/nvme/target/discovery.c +0 -0
index 5baf269f3f8a..8efca26b4776 100644

drivers/nvme/target/discovery.c void nvmet_port_disc_changed(struct nvmet_port *port, · line 47

Quoted 7 lines
drivers/nvme/target/discovery.c +4 -0
@@ -41,6 +41,10 @@ void nvmet_port_disc_changed(struct nvmet_port *port,
 ????????? __nvmet_disc_changed(port, ctrl);
 ????? }
 ????? mutex_unlock(&nvmet_disc_subsys->lock);
+
+??? /* If transport can signal change, notify transport */
+??? if (port->tr_ops && port->tr_ops->discovery_chg)
+??????? port->tr_ops->discovery_chg(port);

So you are shooting for both transport and disc aen to happen at the same time?

Quoted 1 line

they could if the discovery controller is a persistent one.

That's fine I suppose.

So the series looks pretty reasonable to me. Any feedback from the FC folks?

#5 lore raw

PATCH v2 1/7 nvmet: add transport discovery change op

Quoted 19 lines · Sagi Grimberg
Quoted 19 lines
Quoted 18 lines
Quoted 7 lines
drivers/nvme/target/discovery.c +0 -0
index 5baf269f3f8a..8efca26b4776 100644

drivers/nvme/target/discovery.c void nvmet_port_disc_changed(struct nvmet_port · line 48

Quoted 8 lines
drivers/nvme/target/discovery.c +4 -0
@@ -41,6 +41,10 @@ void nvmet_port_disc_changed(struct nvmet_port 
 *port,
 ????????? __nvmet_disc_changed(port, ctrl);
 ????? }
 ????? mutex_unlock(&nvmet_disc_subsys->lock);
+
+??? /* If transport can signal change, notify transport */
+??? if (port->tr_ops && port->tr_ops->discovery_chg)
+??????? port->tr_ops->discovery_chg(port);

So you are shooting for both transport and disc aen to happen at the same time?

Quoted 6 lines
Quoted 1 line

they could if the discovery controller is a persistent one.

That's fine I suppose.

So the series looks pretty reasonable to me. Any feedback from the FC folks?

in what way ?

no disagreements here on the list. Already present in the FC-NVME spec.

-- james

maintainer
#6 lore raw

PATCH v2 1/7 nvmet: add transport discovery change op

Quoted 8 lines
Quoted 4 lines

That's fine I suppose.

So the series looks pretty reasonable to me. Any feedback from the FC folks?

in what way ?

no disagreements here on the list. Already present in the FC-NVME spec.

Just wanted to understand if no disagreements because no one looked, or because it makes sense to everyone..

#7 lore raw

PATCH v2 2/7 nvmet_fc: add transport discovery change event callback support

In reply to James Smart

This patch adds support for the nvmet discovery_change transport op. In turn, the transport adds it's own LLDD api callback discovery_event op to request the LLDD to generate an RSCN for the discovery change.

Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
---
 drivers/nvme/target/fc.c       | 11 +++++++++++
 include/linux/nvme-fc-driver.h |  6 ++++++
 2 files changed, 17 insertions(+)
drivers/nvme/target/fc.c +11 -0
index 508661af0f50..1f252c9a953a 100644
@@ -2549,6 +2549,16 @@ nvmet_fc_remove_port(struct nvmet_port *port)
 	kfree(pe);
 }
 
+static void
+nvmet_fc_discovery_chg(struct nvmet_port *port)
+{
+	struct nvmet_fc_port_entry *pe = port->priv;
+	struct nvmet_fc_tgtport *tgtport = pe->tgtport;
+
+	if (tgtport && tgtport->ops->discovery_event)
+		tgtport->ops->discovery_event(&tgtport->fc_target_port);
+}
+
 static const struct nvmet_fabrics_ops nvmet_fc_tgt_fcp_ops = {
 	.owner			= THIS_MODULE,
 	.type			= NVMF_TRTYPE_FC,
@@ -2557,6 +2567,7 @@ static const struct nvmet_fabrics_ops nvmet_fc_tgt_fcp_ops = {
 	.remove_port		= nvmet_fc_remove_port,
 	.queue_response		= nvmet_fc_fcp_nvme_cmd_done,
 	.delete_ctrl		= nvmet_fc_delete_ctrl,
+	.discovery_chg		= nvmet_fc_discovery_chg,
 };
 
 static int __init nvmet_fc_init_module(void)
include/linux/nvme-fc-driver.h +6 -0
index 2bb349035431..5bbcb7afef39 100644
@@ -797,6 +797,11 @@ struct nvmet_fc_target_port {
  *       nvmefc_tgt_fcp_req.
  *       Entrypoint is Optional.
  *
+ * @discovery_event:  Called by the transport to generate an RSCN
+ *       change notifications to NVME initiators. The RSCN notifications
+ *       should cause the initiator to rescan the discovery controller
+ *       on the targetport.
+ *
  * @max_hw_queues:  indicates the maximum number of hw queues the LLDD
  *       supports for cpu affinitization.
  *       Value is Mandatory. Must be at least 1.
@@ -838,6 +843,7 @@ struct nvmet_fc_target_template {
 				struct nvmefc_tgt_fcp_req *fcpreq);
 	void (*defer_rcv)(struct nvmet_fc_target_port *tgtport,
 				struct nvmefc_tgt_fcp_req *fcpreq);
+	void (*discovery_event)(struct nvmet_fc_target_port *tgtport);
 
 	u32	max_hw_queues;
 	u16	max_sgl_segments;
Signature · 1 line
2.13.7
#8 lore raw

PATCH v2 3/7 nvme-fcloop: Add support for nvmet discovery_event op

In reply to James Smart

Update fcloop to support the discovery_event operation and invoke a nvme rescan. In a real fc adapter, this would generate an RSCN, which the host would receive and convert into a nvme rescan on the remote port specified in the rscn payload.

Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
---
 drivers/nvme/target/fcloop.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
drivers/nvme/target/fcloop.c +37 -0
index 381b5a90c48b..6ddcd090ec07 100644
@@ -231,6 +231,11 @@ struct fcloop_lsreq {
 	int				status;
 };
 
+struct fcloop_rscn {
+	struct fcloop_tport		*tport;
+	struct work_struct		work;
+};
+
 enum {
 	INI_IO_START		= 0,
 	INI_IO_ACTIVE		= 1,
@@ -348,6 +353,37 @@ fcloop_xmt_ls_rsp(struct nvmet_fc_target_port *tport,
 	return 0;
 }
 
+/*
+ * Simulate reception of RSCN and converting it to a initiator transport
+ * call to rescan a remote port.
+ */
+static void
+fcloop_tgt_rscn_work(struct work_struct *work)
+{
+	struct fcloop_rscn *tgt_rscn =
+		container_of(work, struct fcloop_rscn, work);
+	struct fcloop_tport *tport = tgt_rscn->tport;
+
+	if (tport->remoteport)
+		nvme_fc_rescan_remoteport(tport->remoteport);
+	kfree(tgt_rscn);
+}
+
+void
+fcloop_tgt_discovery_evt(struct nvmet_fc_target_port *tgtport)
+{
+	struct fcloop_rscn *tgt_rscn;
+
+	tgt_rscn = kzalloc(sizeof(*tgt_rscn), GFP_KERNEL);
+	if (!tgt_rscn)
+		return;
+
+	tgt_rscn->tport = tgtport->private;
+	INIT_WORK(&tgt_rscn->work, fcloop_tgt_rscn_work);
+
+	schedule_work(&tgt_rscn->work);
+}
+
 static void
 fcloop_tfcp_req_free(struct kref *ref)
 {
@@ -839,6 +875,7 @@ static struct nvmet_fc_target_template tgttemplate = {
 	.fcp_op			= fcloop_fcp_op,
 	.fcp_abort		= fcloop_tgt_fcp_abort,
 	.fcp_req_release	= fcloop_fcp_req_release,
+	.discovery_event	= fcloop_tgt_discovery_evt,
 	.max_hw_queues		= FCLOOP_HW_QUEUES,
 	.max_sgl_segments	= FCLOOP_SGL_SEGS,
 	.max_dif_sgl_segments	= FCLOOP_SGL_SEGS,
Signature · 1 line
2.13.7
#9 lore raw

PATCH v2 4/7 lpfc: Add support to generate RSCN events for nport

In reply to James Smart

This patch adds general RSCN support:
- The ability to transmit an RSCN to the port on the other end of the link (regular port if pt2pt, or fabric controller if fabric).
- And general recognition of an RSCN ELS when an ELS is received.

Signed-off-by Dick Kennedy <dick.kennedy at broadcom.com>
Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
---
 drivers/scsi/lpfc/lpfc.h         |   1 +
 drivers/scsi/lpfc/lpfc_crtn.h    |   2 +
 drivers/scsi/lpfc/lpfc_els.c     | 122 +++++++++++++++++++++++++++++++++++++++
 drivers/scsi/lpfc/lpfc_hbadisc.c |  35 +++++++++++
 drivers/scsi/lpfc/lpfc_hw.h      |   2 +
 drivers/scsi/lpfc/lpfc_sli.c     |   1 +
 6 files changed, 163 insertions(+)
drivers/scsi/lpfc/lpfc.h +1 -0
index 41d849f283f6..9246e0212a86 100644
@@ -274,6 +274,7 @@ struct lpfc_stats {
 	uint32_t elsXmitADISC;
 	uint32_t elsXmitLOGO;
 	uint32_t elsXmitSCR;
+	uint32_t elsXmitRSCN;
 	uint32_t elsXmitRNID;
 	uint32_t elsXmitFARP;
 	uint32_t elsXmitFARPR;
drivers/scsi/lpfc/lpfc_crtn.h +2 -0
index e0b14d791b8c..4b8eb9107b85 100644
@@ -141,6 +141,7 @@ int lpfc_issue_els_adisc(struct lpfc_vport *, struct lpfc_nodelist *, uint8_t);
 int lpfc_issue_els_logo(struct lpfc_vport *, struct lpfc_nodelist *, uint8_t);
 int lpfc_issue_els_npiv_logo(struct lpfc_vport *, struct lpfc_nodelist *);
 int lpfc_issue_els_scr(struct lpfc_vport *, uint32_t, uint8_t);
+int lpfc_issue_els_rscn(struct lpfc_vport *vport, uint8_t retry);
 int lpfc_issue_fabric_reglogin(struct lpfc_vport *);
 int lpfc_els_free_iocb(struct lpfc_hba *, struct lpfc_iocbq *);
 int lpfc_ct_free_iocb(struct lpfc_hba *, struct lpfc_iocbq *);
@@ -355,6 +356,7 @@ void lpfc_mbox_timeout_handler(struct lpfc_hba *);
 struct lpfc_nodelist *lpfc_findnode_did(struct lpfc_vport *, uint32_t);
 struct lpfc_nodelist *lpfc_findnode_wwpn(struct lpfc_vport *,
 					 struct lpfc_name *);
+struct lpfc_nodelist *lpfc_findnode_mapped(struct lpfc_vport *vport);
 
 int lpfc_sli_issue_mbox_wait(struct lpfc_hba *, LPFC_MBOXQ_t *, uint32_t);
 
drivers/scsi/lpfc/lpfc_els.c +122 -0

140 lines not on this page · raw message

drivers/scsi/lpfc/lpfc_hbadisc.c +35 -0
index aa4961a2caf8..9b016666601f 100644
@@ -5261,6 +5261,41 @@ lpfc_findnode_did(struct lpfc_vport *vport, uint32_t did)
 }
 
 struct lpfc_nodelist *
+lpfc_findnode_mapped(struct lpfc_vport *vport)
+{
+	struct Scsi_Host *shost = lpfc_shost_from_vport(vport);
+	struct lpfc_nodelist *ndlp;
+	uint32_t data1;
+	unsigned long iflags;
+
+	spin_lock_irqsave(shost->host_lock, iflags);
+
+	list_for_each_entry(ndlp, &vport->fc_nodes, nlp_listp) {
+		if (ndlp->nlp_state == NLP_STE_UNMAPPED_NODE ||
+		    ndlp->nlp_state == NLP_STE_MAPPED_NODE) {
+			data1 = (((uint32_t)ndlp->nlp_state << 24) |
+				 ((uint32_t)ndlp->nlp_xri << 16) |
+				 ((uint32_t)ndlp->nlp_type << 8) |
+				 ((uint32_t)ndlp->nlp_rpi & 0xff));
+			spin_unlock_irqrestore(shost->host_lock, iflags);
+			lpfc_printf_vlog(vport, KERN_INFO, LOG_NODE,
+					 "2025 FIND node DID "
+					 "Data: x%p x%x x%x x%x %p\n",
+					 ndlp, ndlp->nlp_DID,
+					 ndlp->nlp_flag, data1,
+					 ndlp->active_rrqs_xri_bitmap);
+			return ndlp;
+		}
+	}
+	spin_unlock_irqrestore(shost->host_lock, iflags);
+
+	/* FIND node did <did> NOT FOUND */
+	lpfc_printf_vlog(vport, KERN_INFO, LOG_NODE,
+			 "2026 FIND mapped did NOT FOUND.\n");
+	return NULL;
+}
+
+struct lpfc_nodelist *
 lpfc_setup_disc_node(struct lpfc_vport *vport, uint32_t did)
 {
 	struct Scsi_Host *shost = lpfc_shost_from_vport(vport);
drivers/scsi/lpfc/lpfc_hw.h +2 -0
index ec1227018913..5e5b3318042e 100644
@@ -599,6 +599,7 @@ struct fc_vft_header {
 #define ELS_CMD_RPL       0x57000000
 #define ELS_CMD_FAN       0x60000000
 #define ELS_CMD_RSCN      0x61040000
+#define ELS_CMD_RSCN_XMT  0x61040008
 #define ELS_CMD_SCR       0x62000000
 #define ELS_CMD_RNID      0x78000000
 #define ELS_CMD_LIRR      0x7A000000
@@ -639,6 +640,7 @@ struct fc_vft_header {
 #define ELS_CMD_RPL       0x57
 #define ELS_CMD_FAN       0x60
 #define ELS_CMD_RSCN      0x0461
+#define ELS_CMD_RSCN_XMT  0x08000461
 #define ELS_CMD_SCR       0x62
 #define ELS_CMD_RNID      0x78
 #define ELS_CMD_LIRR      0x7A
drivers/scsi/lpfc/lpfc_sli.c +1 -0
index 57b4a463b589..cbe7a0f36f4b 100644
@@ -9373,6 +9373,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq,
 		if (if_type >= LPFC_SLI_INTF_IF_TYPE_2) {
 			if (pcmd && (*pcmd == ELS_CMD_FLOGI ||
 				*pcmd == ELS_CMD_SCR ||
+				*pcmd == ELS_CMD_RSCN_XMT ||
 				*pcmd == ELS_CMD_FDISC ||
 				*pcmd == ELS_CMD_LOGO ||
 				*pcmd == ELS_CMD_PLOGI)) {
Signature · 1 line
2.13.7
#10 lore raw

PATCH v2 5/7 lpfc: add nvmet discovery_event op support

In reply to James Smart

This patch adds support for the nvmet discovery op. When the callback routine is called, the driver will call the routine to generate an RSCN to the port on the other end of the link.

Signed-off-by Dick Kennedy <dick.kennedy at broadcom.com>
Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
---
 drivers/scsi/lpfc/lpfc_nvmet.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
drivers/scsi/lpfc/lpfc_nvmet.c +17 -0
index 361e2b103648..1eb02a0b8db6 100644
@@ -1141,6 +1141,22 @@ lpfc_nvmet_defer_rcv(struct nvmet_fc_target_port *tgtport,
 	spin_unlock_irqrestore(&ctxp->ctxlock, iflag);
 }
 
+static void
+lpfc_nvmet_discovery_event(struct nvmet_fc_target_port *tgtport)
+{
+	struct lpfc_nvmet_tgtport *tgtp;
+	struct lpfc_hba *phba;
+	uint32_t rc;
+
+	tgtp = tgtport->private;
+	phba = tgtp->phba;
+
+	rc = lpfc_issue_els_rscn(phba->pport, 0);
+	lpfc_printf_log(phba, KERN_ERR, LOG_NVME,
+			"6420 NVMET subsystem change: Notification %s\n",
+			(rc) ? "Failed" : "Sent");
+}
+
 static struct nvmet_fc_target_template lpfc_tgttemplate = {
 	.targetport_delete = lpfc_nvmet_targetport_delete,
 	.xmt_ls_rsp     = lpfc_nvmet_xmt_ls_rsp,
@@ -1148,6 +1164,7 @@ static struct nvmet_fc_target_template lpfc_tgttemplate = {
 	.fcp_abort      = lpfc_nvmet_xmt_fcp_abort,
 	.fcp_req_release = lpfc_nvmet_xmt_fcp_release,
 	.defer_rcv	= lpfc_nvmet_defer_rcv,
+	.discovery_event = lpfc_nvmet_discovery_event,
 
 	.max_hw_queues  = 1,
 	.max_sgl_segments = LPFC_NVMET_DEFAULT_SEGS,
Signature · 1 line
2.13.7
#11 lore raw

PATCH v2 6/7 lpfc: Add support for translating an RSCN rcv into a discovery rescan

In reply to James Smart

This patch updates RSCN receive processing to check for the remote port being an NVME port, and if so, invoke the nvme_fc callback to rescan the remote port. The rescan will generate a discovery udev event.

Signed-off-by Dick Kennedy <dick.kennedy at broadcom.com>
Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>

---
v2: correct rport role to check for nvme discovery not just nvme target

---
 drivers/scsi/lpfc/lpfc_crtn.h |  2 ++
 drivers/scsi/lpfc/lpfc_els.c  |  5 +++++
 drivers/scsi/lpfc/lpfc_nvme.c | 44 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+)
drivers/scsi/lpfc/lpfc_crtn.h +2 -0
index 4b8eb9107b85..866374801140 100644
@@ -557,6 +557,8 @@ void lpfc_ras_stop_fwlog(struct lpfc_hba *phba);
 int lpfc_check_fwlog_support(struct lpfc_hba *phba);
 
 /* NVME interfaces. */
+void lpfc_nvme_rescan_port(struct lpfc_vport *vport,
+			   struct lpfc_nodelist *ndlp);
 void lpfc_nvme_unregister_port(struct lpfc_vport *vport,
 			struct lpfc_nodelist *ndlp);
 int lpfc_nvme_register_port(struct lpfc_vport *vport,
drivers/scsi/lpfc/lpfc_els.c +5 -0
index c9a40e05edcc..c8b305c1aafb 100644
@@ -6329,6 +6329,8 @@ lpfc_rscn_recovery_check(struct lpfc_vport *vport)
 			continue;
 		}
 
+		if (ndlp->nlp_fc4_type & NLP_FC4_NVME)
+			lpfc_nvme_rescan_port(vport, ndlp);
 
 		lpfc_disc_state_machine(vport, ndlp, NULL,
 					NLP_EVT_DEVICE_RECOVERY);
@@ -6440,6 +6442,9 @@ lpfc_els_rcv_rscn(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb,
 				 "2024 pt2pt RSCN %08x Data: x%x x%x\n",
 				 *lp, vport->fc_flag, payload_len);
 		lpfc_els_rsp_acc(vport, ELS_CMD_ACC, cmdiocb, ndlp, NULL);
+
+		if (ndlp->nlp_fc4_type & NLP_FC4_NVME)
+			lpfc_nvme_rescan_port(vport, ndlp);
 		return 0;
 	}
 
drivers/scsi/lpfc/lpfc_nvme.c +44 -0

50 lines not on this page · raw message

Signature · 1 line
2.13.7
#12 lore raw

PATCH v2 7/7 lpfc: Add sysfs interface to post NVME RSCN

In reply to James Smart

To support scenarios which aren't bound to nvmetcli add port scenarios, which is currently where the nvmet_fc transport invokes the discovery event callbacks, a syfs attribute is added to lpfc which can be written to cause an RSCN to be generated for the nport.

Signed-off-by Dick Kennedy <dick.kennedy at broadcom.com>
Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
---
 drivers/scsi/lpfc/lpfc.h      |  1 +
 drivers/scsi/lpfc/lpfc_attr.c | 60 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)
drivers/scsi/lpfc/lpfc.h +1 -0
index 9246e0212a86..5ef0efb830d2 100644
@@ -820,6 +820,7 @@ struct lpfc_hba {
 	uint32_t cfg_use_msi;
 	uint32_t cfg_auto_imax;
 	uint32_t cfg_fcp_imax;
+	uint32_t cfg_force_rscn;
 	uint32_t cfg_cq_poll_threshold;
 	uint32_t cfg_cq_max_proc_limit;
 	uint32_t cfg_fcp_cpu_map;
drivers/scsi/lpfc/lpfc_attr.c +60 -0

78 lines not on this page · raw message

Signature · 1 line
2.13.7
#13 lore raw

v2 cover nvmet/nvmet_fc: add events for discovery controller rescan

In reply to James Smart

Apologies for the delay, wanted to test out some of these changes, but it looks it would be a bit longer.

Anyway, the series look good to me.

-- arun

Quoted 60 lines · James Smart

A transport may have a transport-specific mechanism that can signal when discovery controller content has changed and request a host to rescan to the discovery controller.

FC is such a transport. RSCNs may be generated by the subsystem's FC port and sent to the initiator or fabric controller. If a fabric, the fabric controller will broadcast the RSCN to registered hosts. A host, upon receiving the RSCN, would validate connectivity then initiate a discovery controller rescan.

These patches:
- Modify the nvmet core layer to call a transport callback whenever a port discovery change occurs. To facilitate the callback and avoid new routines between core. and discovery.c the port structure now has a copy of the transport ops structure.
- Modify the nvmet-fc transport to support the nvmet callback, and add its own internal lldd api to request the lldd to generate an RSCN.
- Update nvme-fcloop test harness to support the lldd api and invoke the rescan on the host.
- Modify the lpfc driver to support the new interfaces:
- Adds a new routine to transmit an RSCN to the other port (pt2pt) or fabric controller. Add recognition for receipt of an RSCN.
- Add support for the nvmet lldd api for discovery event, which invokes the RSCN transmit.
- Ties into the RSCN receipt and requests the nvme_fc transport to rescan the remote port (discovery event will be posted).
- Adds a sysfs routine to enable manual generation of an RSCN.

v2:
Revise lpfc patch 6 to check for port role nvme discovery not just
nvme target

James Smart (7):
nvmet: add transport discovery change op
nvmet_fc: add transport discovery change event callback support
nvme-fcloop: Add support for nvmet discovery_event op
lpfc: Add support to generate RSCN events for nport
lpfc: add nvmet discovery_event op support
lpfc: Add support for translating an RSCN rcv into a discovery rescan
lpfc: Add sysfs interface to post NVME RSCN

 drivers/nvme/target/core.c       |   2 +
 drivers/nvme/target/discovery.c  |   4 ++
 drivers/nvme/target/fc.c         |  11 ++++
 drivers/nvme/target/fcloop.c     |  37 ++++++++++++
 drivers/nvme/target/nvmet.h      |   2 +
 drivers/scsi/lpfc/lpfc.h         |   2 +
 drivers/scsi/lpfc/lpfc_attr.c    |  60 ++++++++++++++++++
 drivers/scsi/lpfc/lpfc_crtn.h    |   4 ++
 drivers/scsi/lpfc/lpfc_els.c     | 127 +++++++++++++++++++++++++++++++++++++++
 drivers/scsi/lpfc/lpfc_hbadisc.c |  35 +++++++++++
 drivers/scsi/lpfc/lpfc_hw.h      |   2 +
 drivers/scsi/lpfc/lpfc_nvme.c    |  44 ++++++++++++++
 drivers/scsi/lpfc/lpfc_nvmet.c   |  17 ++++++
 drivers/scsi/lpfc/lpfc_sli.c     |   1 +
 include/linux/nvme-fc-driver.h   |   6 ++
 15 files changed, 354 insertions(+)
maintainer
#14 lore raw

v2 cover nvmet/nvmet_fc: add events for discovery controller rescan

Quoted 5 lines

Apologies for the delay, wanted to test out some of these changes, but it looks it would be a bit longer.

Anyway, the series look good to me.

OK, so unless anyone has any objections we'll pick it up for 5.3

Thanks James and Arun