Skip to contentSkip to conversation

nvme-cli: nvmf auto-connect scripts

RFC PATCH Superseded James Smart 5 participants 39 messages Jul 12, 2019 – Jul 16, 2019

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover nvme-cli: nvmf auto-connect scripts

This posting is a combination of the nvme-fc auto-connect scripts posted by Hannes, the RFC from Sagi to allow auto-connect for persistent discovery controllers that send AENs, and the RFC that I posted addressing comments. It does not contain the code that handles the persistent discovery controller AEN and generates the event.

The auto-connect scripts are now commonized with the exception of the nvme-fc boot connections service. It's expected that both the persistent discovery controllers and nvme-fc can post the same udev event. Whenever the discovery aen generates an event, the new --device=<name> argument will be used to identify the persistent discovery controller. When --device is used, the other connect attributes will be specified, set to the values of the discovery controller, and will be used by the cli to validate the device is who it should be or a new discovery controller will be created. When nvme-fc generates it's events, it will specify --device=none and specify the discovery controller addressing arguments. If a persistent discovery controller exists on nvme-fc, will do nothing special and AEN's from it will follow the --device=<name> syntax.

The udev event handler supports both the new event to be used by discovery controllers as well as the existing nvme-fc transport event. The nvme-fc transport will be migrated to issue the new event syntax. The udev script will continue to support the older style nvme-fc event info in case the cli is running against an older kernel.

v2:
Addressed a couple of comments from Minwoo Im
Break device arg extension into multiple patches
Slight tweaks to parse routine
Add reviewed-by's

James Smart (8):
nvme-cli: ignore arguments that pass in "none"
nvme-cli: allow discover to address discovery controller by persistent
name
nvme-cli: Refactor to create a get_nvme_ctrl_info routine
nvme-cli: extend ctrl_list_item for connect attributes
nvme-cli: Add routine to compare ctrl_list_item to connect args
nvme-cli: Add routine to search for controller with specific
attributes
nvme-cli: Expand --device argument processing
nvme-cli: nvmf auto-connect scripts

Sagi Grimberg (2):
nvme-cli: support persistent connections to a discovery controller
nvme-cli: add --quiet option

 Makefile                                         |  16 +-
 fabrics.c                                        | 117 +++++++++---
 fabrics.h                                        |   2 +
 nvme.c                                           | 228 +++++++++++++++++++----
 nvme.h                                           |  22 +++
 nvme.spec.in                                     |   9 +
 nvmf-autoconnect/70-nvmf-autoconnect.conf        |   1 +
 nvmf-autoconnect/70-nvmf-autoconnect.rules       |  18 ++
 nvmf-autoconnect/nvmefc-boot-connections.service |   9 +
 nvmf-autoconnect/nvmf-connect.target             |   2 +
 nvmf-autoconnect/nvmf-connect at .service           |  14 ++
 11 files changed, 381 insertions(+), 57 deletions(-)
 create mode 100644 nvmf-autoconnect/70-nvmf-autoconnect.conf
 create mode 100644 nvmf-autoconnect/70-nvmf-autoconnect.rules
 create mode 100644 nvmf-autoconnect/nvmefc-boot-connections.service
 create mode 100644 nvmf-autoconnect/nvmf-connect.target
 create mode 100644 nvmf-autoconnect/nvmf-connect at .service
Signature · 1 line
2.13.7
#1 lore raw

RFC PATCH v2 1/10 nvme-cli: ignore arguments that pass in "none"

As we want to support discovery uevents over different transports, we want to allow the kernel to provide missing information in the form of none and have nvme-cli properly ignore it.

One example is the host_traddr. If it is not set (which means that the default route determined the host address) we will want to do the same for newly discovered controllers.

So allow users to pass 'none' arguments as well.

Example:
nvme connect-all ... --hostnqn=none --hostid=none --host_traddr=none

Signed-off-by Sagi Grimberg <sagi at grimberg.me>
Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
Reviewed-by Minwoo Im <minwoo.im.dev at gmail.com>
---
 fabrics.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
fabrics.c +4 -4
index 32c3a9c..5757aaf 100644
@@ -578,7 +578,7 @@ add_argument(char **argstr, int *max_len, char *arg_str, char *arg)
 {
 	int len;
 
-	if (arg) {
+	if (arg && strcmp(arg, "none")) {
 		len = snprintf(*argstr, *max_len, ",%s=%s", arg_str, arg);
 		if (len < 0)
 			return -EINVAL;
@@ -671,14 +671,14 @@ retry:
 		return -EINVAL;
 	p += len;
 
-	if (cfg.hostnqn) {
+	if (cfg.hostnqn && strcmp(cfg.hostnqn, "none")) {
 		len = sprintf(p, ",hostnqn=%s", cfg.hostnqn);
 		if (len < 0)
 			return -EINVAL;
 		p += len;
 	}
 
-	if (cfg.hostid) {
+	if (cfg.hostid && strcmp(cfg.hostid, "none")) {
 		len = sprintf(p, ",hostid=%s", cfg.hostid);
 		if (len < 0)
 			return -EINVAL;
@@ -713,7 +713,7 @@ retry:
 		p += len;
 	}
 
-	if (cfg.host_traddr) {
+	if (cfg.host_traddr && strcmp(cfg.host_traddr, "none")) {
 		len = sprintf(p, ",host_traddr=%s", cfg.host_traddr);
 		if (len < 0)
 			return -EINVAL;
Signature · 1 line
2.13.7
#2 lore raw

RFC PATCH v2 2/10 nvme-cli: support persistent connections to a discovery controller

In reply to James Smart

From: Sagi Grimberg <[email protected]>

Simply don't destroy the discovery controller after getting the log pages. Note that persistent connection to a discovery subsystem require to pass in a non-zero kato value, so if not provided we simply use a default of 30 seconds kato.

Example:
nvme connect-all ... --persistent

Signed-off-by Sagi Grimberg <sagi at grimberg.me>
Reviewed-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
Reviewed-by Max Gurtovoy <maxg at mellanox.com>
Reviewed-by Minwoo Im <minwoo.im.dev at gmail.com>
---
 fabrics.c | 15 ++++++++++++---
 fabrics.h |  2 ++
 2 files changed, 14 insertions(+), 3 deletions(-)
fabrics.c +12 -3
index 5757aaf..75dedf8 100644
@@ -66,6 +66,7 @@ static struct config {
 	int  disable_sqflow;
 	int  hdr_digest;
 	int  data_digest;
+	bool persistent;
 } cfg = { NULL };
 
 #define BUF_SIZE		4096
@@ -872,9 +873,11 @@ static int do_discover(char *argstr, bool connect)
 		return -errno;
 	ret = nvmf_get_log_page_discovery(dev_name, &log, &numrec, &status);
 	free(dev_name);
-	err = remove_ctrl(instance);
-	if (err)
-		return err;
+	if (!cfg.persistent) {
+		err = remove_ctrl(instance);
+		if (err)
+			return err;
+	}
 
 	switch (ret) {
 	case DISC_OK:
@@ -957,6 +960,9 @@ static int discover_from_conf_file(const char *desc, char *argstr,
 		if (err)
 			continue;
 
+		if (cfg.persistent && !cfg.keep_alive_tmo)
+			cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
+
 		err = build_options(argstr, BUF_SIZE);
 		if (err) {
 			ret = err;
@@ -999,6 +1005,7 @@ int discover(const char *desc, int argc, char **argv, bool connect)
 		{"nr-write-queues", 'W', "LIST", CFG_INT, &cfg.nr_write_queues,    required_argument, "number of write queues to use (default 0)" },
 		{"nr-poll-queues",  'P', "LIST", CFG_INT, &cfg.nr_poll_queues,    required_argument, "number of poll queues to use (default 0)" },
 		{"queue-size",      'Q', "LIST", CFG_INT, &cfg.queue_size,      required_argument, "number of io queue elements to use (default 128)" },
+		{"persistent",  'p', "LIST", CFG_NONE, &cfg.persistent,  no_argument, "persistent discovery connection" },
 		{NULL},
 	};
 
@@ -1013,6 +1020,8 @@ int discover(const char *desc, int argc, char **argv, bool connect)
 		ret = discover_from_conf_file(desc, argstr,
 				command_line_options, connect);
 	} else {
+		if (cfg.persistent && !cfg.keep_alive_tmo)
+			cfg.keep_alive_tmo = NVMF_DEF_DISC_TMO;
 		ret = build_options(argstr, BUF_SIZE);
 		if (ret)
 			goto out;
fabrics.h +2 -0
index 988f3ef..7c1664b 100644
@@ -1,6 +1,8 @@
 #ifndef _DISCOVER_H
 #define _DISCOVER_H
 
+#define NVMF_DEF_DISC_TMO	30
+
 extern int discover(const char *desc, int argc, char **argv, bool connect);
 extern int connect(const char *desc, int argc, char **argv);
 extern int disconnect(const char *desc, int argc, char **argv);
Signature · 1 line
2.13.7
#3 lore raw

RFC PATCH v2 3/10 nvme-cli: allow discover to address discovery controller by persistent name

In reply to James Smart

To support discovery (connect/connect-all) to operate against a persistent discovery controller, let the discovery controller to be specified by its device node name rather than new connection attributes.

Example:
nvme connect-all ... --device=nvme5

Also centralize extraction of controller instance from the controller name to a common helper.

Signed-off-by Sagi Grimberg <sagi at grimberg.me>
Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Max Gurtovoy <maxg at mellanox.com>

---
v2:
Rework ctrl_instance for return value on error.
Validate device name is a controller name.

---
 fabrics.c | 37 +++++++++++++++++++++++++++----------
 1 file changed, 27 insertions(+), 10 deletions(-)
fabrics.c +27 -10
index 75dedf8..d92c2ff 100644
@@ -190,6 +190,21 @@ static const char *cms_str(__u8 cm)
 
 static int do_discover(char *argstr, bool connect);
 
+static int ctrl_instance(char *device)
+{
+	char d[64];
+	int ret, instance;
+
+	device = basename(device);
+	ret = sscanf(device, "nvme%d", &instance);
+	if (ret <= 0)
+		return -EINVAL;
+	if (snprintf(d, sizeof(d), "nvme%d", instance) <= 0 ||
+	    strcmp(device, d))
+		return -EINVAL;
+	return instance;
+}
+
 static int add_ctrl(const char *argstr)
 {
 	substring_t args[MAX_OPT_ARGS];
@@ -865,7 +880,10 @@ static int do_discover(char *argstr, bool connect)
 	int instance, numrec = 0, ret, err;
 	int status = 0;
 
-	instance = add_ctrl(argstr);
+	if (!cfg.device)
+		instance = add_ctrl(argstr);
+	else
+		instance = ctrl_instance(cfg.device);
 	if (instance < 0)
 		return instance;
 
@@ -873,7 +891,7 @@ static int do_discover(char *argstr, bool connect)
 		return -errno;
 	ret = nvmf_get_log_page_discovery(dev_name, &log, &numrec, &status);
 	free(dev_name);
-	if (!cfg.persistent) {
+	if (!cfg.device && !cfg.persistent) {
 		err = remove_ctrl(instance);
 		if (err)
 			return err;
@@ -996,6 +1014,7 @@ int discover(const char *desc, int argc, char **argv, bool connect)
 		{"hostnqn",     'q', "LIST", CFG_STRING, &cfg.hostnqn,     required_argument, "user-defined hostnqn (if default not used)" },
 		{"hostid",      'I', "LIST", CFG_STRING, &cfg.hostid,      required_argument, "user-defined hostid (if default not used)"},
 		{"raw",         'r', "LIST", CFG_STRING, &cfg.raw,         required_argument, "raw output file" },
+		{"device",      'd', "LIST", CFG_STRING, &cfg.device, required_argument, "use existing discovery controller device" },
 		{"keep-alive-tmo",  'k', "LIST", CFG_INT, &cfg.keep_alive_tmo,  required_argument, "keep alive timeout period in seconds" },
 		{"reconnect-delay", 'c', "LIST", CFG_INT, &cfg.reconnect_delay, required_argument, "reconnect timeout period in seconds" },
 		{"ctrl-loss-tmo",   'l', "LIST", CFG_INT, &cfg.ctrl_loss_tmo,   required_argument, "controller loss timeout period in seconds" },
@@ -1014,6 +1033,9 @@ int discover(const char *desc, int argc, char **argv, bool connect)
 	if (ret)
 		goto out;
 
+	if (cfg.device && !strcmp(cfg.device, "none"))
+		cfg.device = NULL;
+
 	cfg.nqn = NVME_DISC_SUBSYS_NAME;
 
 	if (!cfg.transport && !cfg.traddr) {
@@ -1157,15 +1179,10 @@ static int disconnect_by_nqn(char *nqn)
 static int disconnect_by_device(char *device)
 {
 	int instance;
-	int ret;
-
-	device = basename(device);
-	ret = sscanf(device, "nvme%d", &instance);
-	if (ret < 0)
-		return ret;
-	if (!ret)
-		return -1;
 
+	instance = ctrl_instance(device);
+	if (instance < 0)
+		return instance;
 	return remove_ctrl(instance);
 }
 
Signature · 1 line
2.13.7
#4 lore raw

RFC PATCH v2 3/10 nvme-cli: allow discover to address discovery controller by persistent name

Quoted 10 lines
fabrics.c +2 -0
index 75dedf8..d92c2ff 100644
@@ -190,6 +190,21 @@ static const char *cms_str(__u8 cm)
 
 static int do_discover(char *argstr, bool connect);
 
+static int ctrl_instance(char *device)
+{

fabrics.c static const char *cms_str(__u8 cm) · line 204

Quoted 10 lines
fabrics.c +10 -0
+	char d[64];
+	int ret, instance;
+
+	device = basename(device);
+	ret = sscanf(device, "nvme%d", &instance);
+	if (ret <= 0)
+		return -EINVAL;
+	if (snprintf(d, sizeof(d), "nvme%d", instance) <= 0 ||
+	    strcmp(device, d))
+		return -EINVAL;

Okay, I think it looks better :). Also it would be better if it Can provide error messages here to say to give controller instance Instead of blkdev, But it can be made later time.

Reviewed-by Minwoo Im <minwoo.im.dev at gmail.com>
#5 lore raw

RFC PATCH v2 3/10 nvme-cli: allow discover to address discovery controller by persistent name

In reply to James Smart
Quoted 15 lines · James Smart

To support discovery (connect/connect-all) to operate against a persistent discovery controller, let the discovery controller to be specified by its device node name rather than new connection attributes.

Example:
nvme connect-all ... --device=nvme5

Also centralize extraction of controller instance from the controller name to a common helper.

Signed-off-by Sagi Grimberg <sagi at grimberg.me>
Signed-off-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Max Gurtovoy <maxg at mellanox.com>
Quoted 8 lines

---
v2:
Rework ctrl_instance for return value on error.
Validate device name is a controller name.

---
 fabrics.c | 37 +++++++++++++++++++++++++++----------
 1 file changed, 27 insertions(+), 10 deletions(-)
Reviewed-by Hannes Reinecke <hare at suse.com>

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
maintainer
#6 lore raw

RFC PATCH v2 3/10 nvme-cli: allow discover to address discovery controller by persistent name

In reply to James Smart
Reviewed-by Sagi Grimberg <sagi at grimberg.me>
#7 lore raw

RFC PATCH v2 4/10 nvme-cli: Refactor to create a get_nvme_ctrl_info routine

In reply to James Smart

In preparation for searching controllers to match with connect args:

Refactor the get_nvme_subsystem_info() routine to take the portion that creates a ctrl_list_item and sets its values and put it in a separate get_nvme_ctrl_info() routine.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 77 ++++++++++++++++++++++++++++++++++++++----------------------------
 1 file changed, 45 insertions(+), 32 deletions(-)
nvme.c +45 -32
index 6fe99eb..dabbbe0 100644
@@ -1544,10 +1544,50 @@ static void free_ctrl_list_item(struct ctrl_list_item *ctrls)
 	free(ctrls->ana_state);
 }
 
+static int get_nvme_ctrl_info(char *name, char *path,
+			struct ctrl_list_item *ctrl, __u32 nsid)
+{
+	char ctrl_path[512];
+
+	ctrl->name = strdup(name);
+
+	snprintf(ctrl_path, sizeof(ctrl_path), "%s/%s", path, ctrl->name);
+
+	ctrl->address = get_nvme_ctrl_attr(ctrl_path, "address");
+	if (!ctrl->address) {
+		fprintf(stderr, "%s: failed to get controller address.\n",
+			ctrl->name);
+		goto free_ctrl_items;
+	}
+
+	ctrl->transport = get_nvme_ctrl_attr(ctrl_path, "transport");
+	if (!ctrl->transport) {
+		fprintf(stderr, "%s: failed to get controller transport.\n",
+			ctrl->name);
+		goto free_ctrl_items;
+	}
+
+	ctrl->state = get_nvme_ctrl_attr(ctrl_path, "state");
+	if (!ctrl->state) {
+		fprintf(stderr, "%s: failed to get controller state.\n",
+			ctrl->name);
+		goto free_ctrl_items;
+	}
+
+	if (nsid != NVME_NSID_ALL)
+		ctrl->ana_state = get_nvme_ctrl_path_ana_state(ctrl_path, nsid);
+
+	return 0;	/* success */
+
+free_ctrl_items:
+	free_ctrl_list_item(ctrl);
+
+	return 1;	/* failure */
+}
+
 static int get_nvme_subsystem_info(char *name, char *path,
 				struct subsys_list_item *item, __u32 nsid)
 {
-	char ctrl_path[512];
 	struct dirent **ctrls;
 	int n, i, ret = 1, ccnt = 0;
 
@@ -1574,38 +1614,11 @@ static int get_nvme_subsystem_info(char *name, char *path,
 	item->nctrls = n;
 
 	for (i = 0; i < n; i++) {
-		item->ctrls[ccnt].name = strdup(ctrls[i]->d_name);
-
-		snprintf(ctrl_path, sizeof(ctrl_path), "%s/%s", path,
-			 item->ctrls[ccnt].name);
-
-		item->ctrls[ccnt].address =
-				get_nvme_ctrl_attr(ctrl_path, "address");
-		if (!item->ctrls[ccnt].address) {
-			fprintf(stderr, "failed to get controller[%d] address.\n", i);
-			free_ctrl_list_item(&item->ctrls[ccnt]);
-			continue;
+		if (get_nvme_ctrl_info(ctrls[i]->d_name, path,
+				&item->ctrls[ccnt], nsid)) {
+			fprintf(stderr, "failed to get controller[%d] info.\n",
+					i);
 		}
-
-		item->ctrls[ccnt].transport =
-				get_nvme_ctrl_attr(ctrl_path, "transport");
-		if (!item->ctrls[ccnt].transport) {
-			fprintf(stderr, "failed to get controller[%d] transport.\n", i);
-			free_ctrl_list_item(&item->ctrls[ccnt]);
-			continue;
-		}
-
-		item->ctrls[ccnt].state =
-				get_nvme_ctrl_attr(ctrl_path, "state");
-		if (!item->ctrls[ccnt].state) {
-			fprintf(stderr, "failed to get controller[%d] state.\n", i);
-			free_ctrl_list_item(&item->ctrls[ccnt]);
-			continue;
-		}
-
-		if (nsid != NVME_NSID_ALL)
-			item->ctrls[ccnt].ana_state =
-				get_nvme_ctrl_path_ana_state(ctrl_path, nsid);
 		ccnt++;
 	}
 
Signature · 1 line
2.13.7
maintainer
#8 lore raw

RFC PATCH v2 4/10 nvme-cli: Refactor to create a get_nvme_ctrl_info routine

Reviewed-by Sagi Grimberg <sagi at grimberg.me>
#9 lore raw

RFC PATCH v2 4/10 nvme-cli: Refactor to create a get_nvme_ctrl_info routine

In reply to James Smart
Quoted 6 lines · James Smart

In preparation for searching controllers to match with connect args:

Refactor the get_nvme_subsystem_info() routine to take the portion that creates a ctrl_list_item and sets its values and put it in a separate get_nvme_ctrl_info() routine.

Quoted 7 lines
Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 77 ++++++++++++++++++++++++++++++++++++++----------------------------
 1 file changed, 45 insertions(+), 32 deletions(-)
Reviewed-by Hannes Reinecke <hare at suse.com>

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#10 lore raw

RFC PATCH v2 5/10 nvme-cli: extend ctrl_list_item for connect attributes

In reply to James Smart

In preparation for searching controllers to match with connect args:

Extend the ctrl_list_item with elements that can be compared against connect arguments. Extend the get_nvme_ctrl_info() routine to set the fields.

subsysnqn was added as the ctrl_list_item may now be used outside of a subsystem listing.

traddr, trsvid, and host_traddr are added. Their values come from the address attribute. A new parsing routine was added that extracts the values from the address string.

The new parsing routine and supporting field names strings are declared as a global interface as a subsequent patch will call it from the fabrics routines.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 nvme.h | 10 ++++++++++
 2 files changed, 82 insertions(+)
nvme.c +72 -0
index dabbbe0..66ba2fc 100644
@@ -1542,6 +1542,64 @@ static void free_ctrl_list_item(struct ctrl_list_item *ctrls)
 	free(ctrls->address);
 	free(ctrls->state);
 	free(ctrls->ana_state);
+	free(ctrls->subsysnqn);
+	free(ctrls->traddr);
+	free(ctrls->trsvcid);
+	free(ctrls->host_traddr);
+}
+
+static const char delim_space  = ' ';
+const char *conarg_traddr = "traddr";
+const char *conarg_trsvcid = "trsvcid";
+const char *conarg_host_traddr = "host_traddr";
+
+/*
+ * parse strings with connect arguments to find a particular field.
+ * If field found, return string containing field value. If field
+ * not found, return an empty string.
+ */
+char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm)
+{
+	char *s, *e;
+	size_t cnt;
+
+	/*
+	 * There are field name overlaps: traddr and host_traddr.
+	 * By chance, both connect arg strings are set up to
+	 * have traddr field followed by host_traddr field. Thus field
+	 * name matching doesn't overlap in the searches. Technically,
+	 * as is, the loop and delimiter checking isn't necessary.
+	 * However, better to be prepared.
+	 */
+	do {
+		s = strstr(conargs, fieldnm);
+		if (!s)
+			goto empty_field;
+		/* validate prior character is delimiter */
+		if (s == conargs || *(s - 1) == delim) {
+			/* match requires next character to be assignment */
+			s += strlen(fieldnm);
+			if (*s == '=')
+				/* match */
+				break;
+		}
+		/* field overlap: seek to delimiter and keep looking */
+		conargs = strchr(s, delim);
+		if (!conargs)
+			goto empty_field;
+		conargs++;	/* skip delimiter */
+	} while (1);
+	s++;		/* skip assignment character */
+	e = strchr(s, delim);
+	if (e)
+		cnt = e - s;
+	else
+		cnt = strlen(s);
+
+	return strndup(s, cnt);
+
+empty_field:
+	return strdup("\0");
 }
 
 static int get_nvme_ctrl_info(char *name, char *path,
@@ -1577,6 +1635,20 @@ static int get_nvme_ctrl_info(char *name, char *path,
 	if (nsid != NVME_NSID_ALL)
 		ctrl->ana_state = get_nvme_ctrl_path_ana_state(ctrl_path, nsid);
 
+	ctrl->subsysnqn = get_nvme_ctrl_attr(ctrl_path, "subsysnqn");
+	if (!ctrl->subsysnqn) {
+		fprintf(stderr, "%s: failed to get controller subsysnqn.\n",
+			ctrl->name);
+		goto free_ctrl_items;
+	}
+
+	ctrl->traddr = __parse_connect_arg(ctrl->address, delim_space,
+					conarg_traddr);
+	ctrl->trsvcid = __parse_connect_arg(ctrl->address, delim_space,
+					conarg_trsvcid);
+	ctrl->host_traddr = __parse_connect_arg(ctrl->address, delim_space,
+					conarg_host_traddr);
+
 	return 0;	/* success */
 
 free_ctrl_items:
nvme.h +10 -0
index 3409515..470f702 100644
@@ -160,6 +160,10 @@ struct ctrl_list_item {
 	char *transport;
 	char *state;
 	char *ana_state;
+	char *subsysnqn;
+	char *traddr;
+	char *trsvcid;
+	char *host_traddr;
 };
 
 struct subsys_list_item {
@@ -175,6 +179,12 @@ enum {
 	BINARY,
 };
 
+char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm);
+
+extern const char *conarg_traddr;
+extern const char *conarg_trsvcid;
+extern const char *conarg_host_traddr;
+
 void register_extension(struct plugin *plugin);
 
 #include "argconfig.h"
Signature · 1 line
2.13.7
maintainer
#11 lore raw

RFC PATCH v2 5/10 nvme-cli: extend ctrl_list_item for connect attributes

Quoted 6 lines
nvme.c +5 -0
+char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm)
+{
+	char *s, *e;
+	size_t cnt;
+

nvme.c static void free_ctrl_list_item(struct ctrl_list_item *ctrls) · line 1575

Quoted 10 lines
nvme.c +10 -0
+	/*
+	 * There are field name overlaps: traddr and host_traddr.
+	 * By chance, both connect arg strings are set up to
+	 * have traddr field followed by host_traddr field. Thus field
+	 * name matching doesn't overlap in the searches. Technically,
+	 * as is, the loop and delimiter checking isn't necessary.
+	 * However, better to be prepared.
+	 */
+	do {
+		s = strstr(conargs, fieldnm);

Any reason not to use the normal strsep for loop?

#12 lore raw

RFC PATCH v2 5/10 nvme-cli: extend ctrl_list_item for connect attributes

Quoted 11 lines · Sagi Grimberg
Quoted 11 lines

+char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm)

(quoted diff) +8 -0
+{
+??? char *s, *e;
+??? size_t cnt;
+
+??? /*
+???? * There are field name overlaps: traddr and host_traddr.
+???? * By chance, both connect arg strings are set up to
+???? * have traddr field followed by host_traddr field. Thus field
Quoted 8 lines
Quoted 6 lines

+???? * name matching doesn't overlap in the searches. Technically, +???? * as is, the loop and delimiter checking isn't necessary. +???? * However, better to be prepared.
+???? */
+??? do {
+??????? s = strstr(conargs, fieldnm);

Any reason not to use the normal strsep for loop?

strsep wasn't used as I didn't want to modify the args string.

I could convert, but it means I need to duplicate the args string both in do_discover and in the nvme_ctrl_listitem routine or have the parse routine duplicate it then throw it away. Using strsep would also mean I have to continue to have the check for '=' token and extracting the value. Not sure I gain much code reduction.

Any preference over leaving as is or having the parse routine duplicate it ?

-- james

maintainer
#13 lore raw

RFC PATCH v2 5/10 nvme-cli: extend ctrl_list_item for connect attributes

Quoted 22 lines
Quoted 11 lines
Quoted 11 lines

+char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm)

(quoted diff) +8 -0
+{
+??? char *s, *e;
+??? size_t cnt;
+
+??? /*
+???? * There are field name overlaps: traddr and host_traddr.
+???? * By chance, both connect arg strings are set up to
+???? * have traddr field followed by host_traddr field. Thus field
Quoted 8 lines
Quoted 6 lines

+???? * name matching doesn't overlap in the searches. Technically, +???? * as is, the loop and delimiter checking isn't necessary. +???? * However, better to be prepared.
+???? */
+??? do {
+??????? s = strstr(conargs, fieldnm);

Any reason not to use the normal strsep for loop?

strsep wasn't used as I didn't want to modify the args string.

Quoted 8 lines

I could convert, but it means I need to duplicate the args string both in do_discover and in the nvme_ctrl_listitem routine or have the parse routine duplicate it then throw it away. Using strsep would also mean I have to continue to have the check for '=' token and extracting the value. Not sure I gain much code reduction.

Any preference over leaving as is or having the parse routine duplicate it ?

No hard emotions on this, just thought it would be less convoluted and more common with other parsers, but if its create a more convoluted code we don't have to go there...

#14 lore raw

RFC PATCH v2 5/10 nvme-cli: extend ctrl_list_item for connect attributes

In reply to James Smart
Quoted 17 lines · James Smart

In preparation for searching controllers to match with connect args:

Extend the ctrl_list_item with elements that can be compared against connect arguments. Extend the get_nvme_ctrl_info() routine to set the fields.

subsysnqn was added as the ctrl_list_item may now be used outside of a subsystem listing.

traddr, trsvid, and host_traddr are added. Their values come from the address attribute. A new parsing routine was added that extracts the values from the address string.

The new parsing routine and supporting field names strings are declared as a global interface as a subsequent patch will call it from the fabrics routines.

Quoted 8 lines
Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 nvme.h | 10 ++++++++++
 2 files changed, 82 insertions(+)
Reviewed-by Hannes Reinecke <hare at suse.com

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#15 lore raw

RFC PATCH v2 6/10 nvme-cli: Add routine to compare ctrl_list_item to connect args

In reply to James Smart

In preparation for searching controllers to match with connect args:

Create a new routine that receives a structure containing connect arguments and a device name. The routine will build and fill in a ctrl_list_item for the device and compare its attributes with the connect arguments. The routine returns true/false on whether they match.

Routine is defined as a global as a subsequent patch will use it from the fabrics routines.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 40 ++++++++++++++++++++++++++++++++++++++++
 nvme.h |  9 +++++++++
 2 files changed, 49 insertions(+)
nvme.c +40 -0
index 66ba2fc..ef3c61c 100644
@@ -2010,6 +2010,46 @@ static int list(int argc, char **argv, struct command *cmd, struct plugin *plugi
 	return nvme_status_to_errno(ret, false);
 }
 
+static char *nvme_ctrl_dir = "/sys/class/nvme/";
+
+/*
+ * Given a controller name, create a ctrl_list_item with its
+ * attributes and compare the attributes against the connect args
+ * given.
+ * Return true/false based on whether it matches
+ */
+bool ctrl_matches_connectargs(char *name, struct connect_args *args)
+{
+	struct ctrl_list_item *ctrl;
+	bool found = false;
+
+	ctrl = malloc(sizeof(*ctrl));
+	if (!ctrl) {
+		fprintf(stderr, "Failed to allocate controller list element\n");
+		return false;
+	}
+	memset(ctrl, 0, sizeof(*ctrl));
+
+	if (get_nvme_ctrl_info(name, nvme_ctrl_dir, ctrl, NVME_NSID_ALL))
+		goto cleanup_exit;
+
+	if (!strcmp(ctrl->subsysnqn, args->subsysnqn) &&
+	    !strcmp(ctrl->transport, args->transport) &&
+	    (!strcmp(ctrl->traddr, args->traddr) ||
+	     !strcmp(args->traddr, "none")) &&
+	    (!strcmp(ctrl->trsvcid, args->trsvcid) ||
+	     !strcmp(args->trsvcid, "none")) &&
+	    (!strcmp(ctrl->host_traddr, args->host_traddr) ||
+	     !strcmp(args->host_traddr, "none")))
+		found = true;
+
+cleanup_exit:
+	free_ctrl_list_item(ctrl);
+	free(ctrl);
+
+	return found;
+}
+
 int __id_ctrl(int argc, char **argv, struct command *cmd, struct plugin *plugin, void (*vs)(__u8 *vs, struct json_object *root))
 {
 	const char *desc = "Send an Identify Controller command to "\
nvme.h +9 -0
index 470f702..537b8dd 100644
@@ -179,6 +179,15 @@ enum {
 	BINARY,
 };
 
+struct connect_args {
+	char *subsysnqn;
+	char *transport;
+	char *traddr;
+	char *trsvcid;
+	char *host_traddr;
+};
+
+bool ctrl_matches_connectargs(char *name, struct connect_args *args);
 char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm);
 
 extern const char *conarg_traddr;
Signature · 1 line
2.13.7
maintainer
#16 lore raw

RFC PATCH v2 6/10 nvme-cli: Add routine to compare ctrl_list_item to connect args

Quoted 25 lines

In preparation for searching controllers to match with connect args:

Create a new routine that receives a structure containing connect arguments and a device name. The routine will build and fill in a ctrl_list_item for the device and compare its attributes with the connect arguments. The routine returns true/false on whether they match.

Routine is defined as a global as a subsequent patch will use it from the fabrics routines.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
  nvme.c | 40 ++++++++++++++++++++++++++++++++++++++++
  nvme.h |  9 +++++++++
  2 files changed, 49 insertions(+)
nvme.c +0 -0
index 66ba2fc..ef3c61c 100644

nvme.c static int list(int argc, char **argv, struct command *cmd, struct plugin *plugi · line 2013

Quoted 4 lines
nvme.c +1 -0
@@ -2010,6 +2010,46 @@ static int list(int argc, char **argv, struct command *cmd, struct plugin *plugi
  	return nvme_status_to_errno(ret, false);
  }
 
+static char *nvme_ctrl_dir = "/sys/class/nvme/";

SYS_NVME?

Other than that, looks good

Reviewed-by Sagi Grimberg <sagi at grimberg.me>
#17 lore raw

RFC PATCH v2 6/10 nvme-cli: Add routine to compare ctrl_list_item to connect args

In reply to James Smart
Quoted 11 lines · James Smart

In preparation for searching controllers to match with connect args:

Create a new routine that receives a structure containing connect arguments and a device name. The routine will build and fill in a ctrl_list_item for the device and compare its attributes with the connect arguments. The routine returns true/false on whether they match.

Routine is defined as a global as a subsequent patch will use it from the fabrics routines.

Quoted 8 lines
Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 40 ++++++++++++++++++++++++++++++++++++++++
 nvme.h |  9 +++++++++
 2 files changed, 49 insertions(+)
Reviewed-by Hannes Reinecke <hare at suse.com>

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#18 lore raw

RFC PATCH v2 7/10 nvme-cli: Add routine to search for controller with specific attributes

In reply to James Smart

In preparation for searching controllers to match with connect args:

Create a new routine find_ctrl_with_connectargs() that will search the controllers that exist in the system to find one that has attributes that match the connect arguments specified. If found, the routine returns the controller name ("nvme?"). If not found, a NULL is returned.

Routine is defined as a global as a subsequent patch will use it from the fabrics routines.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 37 +++++++++++++++++++++++++++++++++++++
 nvme.h |  1 +
 2 files changed, 38 insertions(+)
nvme.c +37 -0
index ef3c61c..314a04f 100644
@@ -2050,6 +2050,43 @@ cleanup_exit:
 	return found;
 }
 
+/*
+ * Look through the system to find an existing controller whose
+ * attributes match the connect arguments specified
+ * If found, a string containing the controller name (ex: "nvme?")
+ * is returned.
+ * If not found, a NULL is returned.
+ */
+char *find_ctrl_with_connectargs(struct connect_args *args)
+{
+	struct dirent **devices;
+	char *devname = NULL;
+	int i, n;
+
+	n = scandir(nvme_ctrl_dir, &devices, scan_ctrls_filter, alphasort);
+	if (n < 0) {
+		fprintf(stderr, "no NVMe controller(s) detected.\n");
+		return NULL;
+	}
+
+	for (i = 0; i < n; i++) {
+		if (ctrl_matches_connectargs(devices[i]->d_name, args)) {
+			devname = strdup(devices[i]->d_name);
+			if (devname == NULL)
+				fprintf(stderr, "no memory for ctlr name %s\n",
+						devices[i]->d_name);
+			goto cleanup_devices;
+		}
+	}
+
+cleanup_devices:
+	for (i = 0; i < n; i++)
+		free(devices[i]);
+	free(devices);
+
+	return devname;
+}
+
 int __id_ctrl(int argc, char **argv, struct command *cmd, struct plugin *plugin, void (*vs)(__u8 *vs, struct json_object *root))
 {
 	const char *desc = "Send an Identify Controller command to "\
nvme.h +1 -0
index 537b8dd..04f8d46 100644
@@ -188,6 +188,7 @@ struct connect_args {
 };
 
 bool ctrl_matches_connectargs(char *name, struct connect_args *args);
+char *find_ctrl_with_connectargs(struct connect_args *args);
 char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm);
 
 extern const char *conarg_traddr;
Signature · 1 line
2.13.7
maintainer
#19 lore raw

RFC PATCH v2 7/10 nvme-cli: Add routine to search for controller with specific attributes

Reviewed-by Sagi Grimberg <sagi at grimberg.me>
#20 lore raw

RFC PATCH v2 7/10 nvme-cli: Add routine to search for controller with specific attributes

In reply to James Smart
Quoted 10 lines · James Smart

In preparation for searching controllers to match with connect args:

Create a new routine find_ctrl_with_connectargs() that will search the controllers that exist in the system to find one that has attributes that match the connect arguments specified. If found, the routine returns the controller name ("nvme?"). If not found, a NULL is returned.

Routine is defined as a global as a subsequent patch will use it from the fabrics routines.

Quoted 8 lines
Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 nvme.c | 37 +++++++++++++++++++++++++++++++++++++
 nvme.h |  1 +
 2 files changed, 38 insertions(+)
Reviewed-by Hannes Reinecke <hare at suse.com>

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#21 lore raw

RFC PATCH v2 8/10 nvme-cli: Expand --device argument processing

In reply to James Smart

The connect-all --device argument was used to specify a specific device to be used as the discovery controller. The device is typically a long-lived discovery controller that posted a discovery event. No attempt was made to ensure the device is who it is supposed to be before using it.

Revised the code to use the other arguments in the connect-all request to validate that the device is the entity that was expected. If the device doesn't match, the cli will look for an existing matching device in the system (should be a discovery controller due to nqn) with the same connect parameters and use it. If one is not found on the system, a new discovery controller will be created for the connect-all request.

The revision uses new routines to parse the connect arguments given in the argstr parameter. As a couple of new fieldnames are needed, the parse routine constants were expanded for them.

The revision uses the new routines to match the specified device vs it's attributes as well as the search routine that looks for a device with the connect arguments.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 fabrics.c | 38 ++++++++++++++++++++++++++++++++++++++
 nvme.c    |  2 ++
 nvme.h    |  2 ++
 3 files changed, 42 insertions(+)
fabrics.c +38 -0
index d92c2ff..e101ead 100644
@@ -873,6 +873,8 @@ static int connect_ctrls(struct nvmf_disc_rsp_page_hdr *log, int numrec)
 	return ret;
 }
 
+static const char delim_comma  = ',';
+
 static int do_discover(char *argstr, bool connect)
 {
 	struct nvmf_disc_rsp_page_hdr *log = NULL;
@@ -880,6 +882,42 @@ static int do_discover(char *argstr, bool connect)
 	int instance, numrec = 0, ret, err;
 	int status = 0;
 
+	if (cfg.device) {
+		struct connect_args cargs;
+
+		memset(&cargs, 0, sizeof(cargs));
+		cargs.subsysnqn = __parse_connect_arg(argstr, delim_comma,
+						conarg_nqn);
+		cargs.transport = __parse_connect_arg(argstr, delim_comma,
+						conarg_transport);
+		cargs.traddr = __parse_connect_arg(argstr, delim_comma,
+						conarg_traddr);
+		cargs.trsvcid = __parse_connect_arg(argstr, delim_comma,
+						conarg_trsvcid);
+		cargs.host_traddr = __parse_connect_arg(argstr, delim_comma,
+						conarg_host_traddr);
+
+		/*
+		 * if the cfg.device passed in matches the connect args
+		 *    cfg.device is left as-is
+		 * else if there exists a controller that matches the
+		 *         connect args
+		 *    cfg.device is the matching ctrl name
+		 * else if no ctrl matches the connect args
+		 *    cfg.device is set to null. This will attempt to
+		 *    create a new ctrl.
+		 * endif
+		 */
+		if (!ctrl_matches_connectargs(cfg.device, &cargs))
+			cfg.device = find_ctrl_with_connectargs(&cargs);
+
+		free(cargs.subsysnqn);
+		free(cargs.transport);
+		free(cargs.traddr);
+		free(cargs.trsvcid);
+		free(cargs.host_traddr);
+	}
+
 	if (!cfg.device)
 		instance = add_ctrl(argstr);
 	else
nvme.c +2 -0
index 314a04f..74d7c21 100644
@@ -1549,6 +1549,8 @@ static void free_ctrl_list_item(struct ctrl_list_item *ctrls)
 }
 
 static const char delim_space  = ' ';
+const char *conarg_nqn = "nqn";
+const char *conarg_transport = "transport";
 const char *conarg_traddr = "traddr";
 const char *conarg_trsvcid = "trsvcid";
 const char *conarg_host_traddr = "host_traddr";
nvme.h +2 -0
index 04f8d46..eedc234 100644
@@ -191,6 +191,8 @@ bool ctrl_matches_connectargs(char *name, struct connect_args *args);
 char *find_ctrl_with_connectargs(struct connect_args *args);
 char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm);
 
+extern const char *conarg_nqn;
+extern const char *conarg_transport;
 extern const char *conarg_traddr;
 extern const char *conarg_trsvcid;
 extern const char *conarg_host_traddr;
Signature · 1 line
2.13.7
maintainer
#22 lore raw

RFC PATCH v2 8/10 nvme-cli: Expand --device argument processing

Reviewed-by Sagi Grimberg <sagi at grimberg.me>
#23 lore raw

RFC PATCH v2 8/10 nvme-cli: Expand --device argument processing

In reply to James Smart
Quoted 21 lines · James Smart

The connect-all --device argument was used to specify a specific device to be used as the discovery controller. The device is typically a long-lived discovery controller that posted a discovery event. No attempt was made to ensure the device is who it is supposed to be before using it.

Revised the code to use the other arguments in the connect-all request to validate that the device is the entity that was expected. If the device doesn't match, the cli will look for an existing matching device in the system (should be a discovery controller due to nqn) with the same connect parameters and use it. If one is not found on the system, a new discovery controller will be created for the connect-all request.

The revision uses new routines to parse the connect arguments given in the argstr parameter. As a couple of new fieldnames are needed, the parse routine constants were expanded for them.

The revision uses the new routines to match the specified device vs it's attributes as well as the search routine that looks for a device with the connect arguments.

Quoted 9 lines
Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Sagi Grimberg <sagi at grimberg.me>
CC Hannes Reinecke <hare at suse.com>
---
 fabrics.c | 38 ++++++++++++++++++++++++++++++++++++++
 nvme.c    |  2 ++
 nvme.h    |  2 ++
 3 files changed, 42 insertions(+)
Reviewed-by Hannes Reinecke <hare at suse.com>

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare at suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG N?rnberg)
#24 lore raw

RFC PATCH v2 9/10 nvme-cli: add --quiet option

In reply to James Smart

From: Sagi Grimberg <[email protected]>

Now we are going to have discovery log change events, so instead of trying to figure out which NVM subsystem ports are already connected, we let the driver do that and only suppress the failure messages.

Example:
nvme connect-all ... --quiet

This option will be used by the discovery log change corresponding udev rule.

Signed-off-by Sagi Grimberg <sagi at grimberg.me>
Reviewed-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
Reviewed-by Minwoo Im <minwoo.im.dev at gmail.com>
---
 fabrics.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)
fabrics.c +14 -7
index e101ead..13e2cf8 100644
@@ -67,6 +67,7 @@ static struct config {
 	int  hdr_digest;
 	int  data_digest;
 	bool persistent;
+	bool quiet;
 } cfg = { NULL };
 
 #define BUF_SIZE		4096
@@ -219,9 +220,11 @@ static int add_ctrl(const char *argstr)
 		goto out;
 	}
 
-	if (write(fd, argstr, len) != len) {
-		fprintf(stderr, "Failed to write to %s: %s\n",
-			 PATH_NVME_FABRICS, strerror(errno));
+	ret = write(fd, argstr, len);
+	if (ret != len) {
+		if (errno != EALREADY || !cfg.quiet)
+			fprintf(stderr, "Failed to write to %s: %s\n",
+				 PATH_NVME_FABRICS, strerror(errno));
 		ret = -errno;
 		goto out_close;
 	}
@@ -855,10 +858,13 @@ static int connect_ctrls(struct nvmf_disc_rsp_page_hdr *log, int numrec)
 		/* already connected print message	*/
 		if (instance == -EALREADY) {
 			const char *traddr = log->entries[i].traddr;
-			fprintf(stderr,
-				"traddr=%.*s is already connected\n",
-				space_strip_len(NVMF_TRADDR_SIZE, traddr),
-				traddr);
+
+			if (!cfg.quiet)
+				fprintf(stderr,
+					"traddr=%.*s is already connected\n",
+					space_strip_len(NVMF_TRADDR_SIZE,
+							traddr),
+					traddr);
 			continue;
 		}
 
@@ -1063,6 +1069,7 @@ int discover(const char *desc, int argc, char **argv, bool connect)
 		{"nr-poll-queues",  'P', "LIST", CFG_INT, &cfg.nr_poll_queues,    required_argument, "number of poll queues to use (default 0)" },
 		{"queue-size",      'Q', "LIST", CFG_INT, &cfg.queue_size,      required_argument, "number of io queue elements to use (default 128)" },
 		{"persistent",  'p', "LIST", CFG_NONE, &cfg.persistent,  no_argument, "persistent discovery connection" },
+		{"quiet",       'Q', "LIST", CFG_NONE, &cfg.quiet,  no_argument, "suppress already connected errors" },
 		{NULL},
 	};
 
Signature · 1 line
2.13.7
maintainer
#25 lore raw

RFC PATCH v2 9/10 nvme-cli: add --quiet option

Quoted 68 lines

From: Sagi Grimberg <sagi at grimberg.me>

Now we are going to have discovery log change events, so instead of trying to figure out which NVM subsystem ports are already connected, we let the driver do that and only suppress the failure messages.

Example:

   nvme connect-all ... --quiet

This option will be used by the discovery log change corresponding udev rule.

Signed-off-by Sagi Grimberg <sagi at grimberg.me>
Reviewed-by James Smart <jsmart2021 at gmail.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
Reviewed-by Minwoo Im <minwoo.im.dev at gmail.com>
---
  fabrics.c | 21 ++++++++++++++-------
  1 file changed, 14 insertions(+), 7 deletions(-)
fabrics.c +13 -7
index e101ead..13e2cf8 100644
@@ -67,6 +67,7 @@ static struct config {
  	int  hdr_digest;
  	int  data_digest;
  	bool persistent;
+	bool quiet;
  } cfg = { NULL };
 
  #define BUF_SIZE		4096
@@ -219,9 +220,11 @@ static int add_ctrl(const char *argstr)
  		goto out;
  	}
 
-	if (write(fd, argstr, len) != len) {
-		fprintf(stderr, "Failed to write to %s: %s\n",
-			 PATH_NVME_FABRICS, strerror(errno));
+	ret = write(fd, argstr, len);
+	if (ret != len) {
+		if (errno != EALREADY || !cfg.quiet)
+			fprintf(stderr, "Failed to write to %s: %s\n",
+				 PATH_NVME_FABRICS, strerror(errno));
  		ret = -errno;
  		goto out_close;
  	}
@@ -855,10 +858,13 @@ static int connect_ctrls(struct nvmf_disc_rsp_page_hdr *log, int numrec)
  		/* already connected print message	*/
  		if (instance == -EALREADY) {
  			const char *traddr = log->entries[i].traddr;
-			fprintf(stderr,
-				"traddr=%.*s is already connected\n",
-				space_strip_len(NVMF_TRADDR_SIZE, traddr),
-				traddr);
+
+			if (!cfg.quiet)
+				fprintf(stderr,
+					"traddr=%.*s is already connected\n",
+					space_strip_len(NVMF_TRADDR_SIZE,
+							traddr),
+					traddr);
  			continue;
  		}
 

fabrics.c int discover(const char *desc, int argc, char **argv, bool connect) · line 1072

Quoted 4 lines
fabrics.c +1 -0
@@ -1063,6 +1069,7 @@ int discover(const char *desc, int argc, char **argv, bool connect)
  		{"nr-poll-queues",  'P', "LIST", CFG_INT, &cfg.nr_poll_queues,    required_argument, "number of poll queues to use (default 0)" },
  		{"queue-size",      'Q', "LIST", CFG_INT, &cfg.queue_size,      required_argument, "number of io queue elements to use (default 128)" },
  		{"persistent",  'p', "LIST", CFG_NONE, &cfg.persistent,  no_argument, "persistent discovery connection" },
+		{"quiet",       'Q', "LIST", CFG_NONE, &cfg.quiet,  no_argument, "suppress already connected errors" },

Q is already a short-arg (queue-size). Lets have it 'S' (stand for Suppress). Also, when I tested it locally for some reason it did not take:
--

Jul 12 17:30:00 nvmc systemd[1]: Started NVMf auto-connect scan upon 
nvme discovery controller Events.
Jul 12 17:30:00 nvmc sh[2767]: Failed to write to /dev/nvme-fabrics: 
Operation already in progress
Jul 12 17:30:00 nvmc sh[2767]: traddr=192.168.123.1 is already connected
Jul 12 17:30:00 nvmc kernel: nvme nvme2: creating 4 I/O queues.
Jul 12 17:30:00 nvmc kernel: nvme nvme2: mapped 4/0/0 default/read/poll 
queues.
Jul 12 17:30:00 nvmc kernel: nvme nvme2: new ctrl: NQN "testnqn2", addr 

192.168.123.1:8009
--

Adding print to add_ctrl:

Jul 12 17:30:00 nvmc sh[2767]: errno 114 quiet 0

When I pass it manually it works, but not from udev... Not sure what is going on...

#26 lore raw

RFC PATCH v2 9/10 nvme-cli: add --quiet option

Quoted 3 lines · Sagi Grimberg

Q is already a short-arg (queue-size). Lets have it 'S' (stand for Suppress).

ok...

Quoted 20 lines

Also, when I tested it locally for some reason it did not take:

Signature · 17 lines
Jul 12 17:30:00 nvmc systemd[1]: Started NVMf auto-connect scan upon 
nvme discovery controller Events.
Jul 12 17:30:00 nvmc sh[2767]: Failed to write to /dev/nvme-fabrics: 
Operation already in progress
Jul 12 17:30:00 nvmc sh[2767]: traddr=192.168.123.1 is already connected
Jul 12 17:30:00 nvmc kernel: nvme nvme2: creating 4 I/O queues.
Jul 12 17:30:00 nvmc kernel: nvme nvme2: mapped 4/0/0 
default/read/poll queues.
Jul 12 17:30:00 nvmc kernel: nvme nvme2: new ctrl: NQN "testnqn2", 
addr 192.168.123.1:8009
-- 

Adding print to add_ctrl:
Jul 12 17:30:00 nvmc sh[2767]: errno 114 quiet 0

When I pass it manually it works, but not from udev... Not sure what
is going on...

?? I'll check it on my side

#27 lore raw

RFC PATCH v2 9/10 nvme-cli: add --quiet option

Quoted 7 lines · James Smart
Quoted 2 lines · Sagi Grimberg

When I pass it manually it works, but not from udev... Not sure what is going on...

?? I'll check it on my side

yeah - the --quiet option wasn't passed right. It became part of the service name, not in the command invoked.

fixing it

-- james

#28 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

In reply to James Smart

This set of scripts is a combination of those sent by Hannes, Sagi, and I in earlier patches and RFC's.

Auto-connect operates by the nvme core layer or nvme-fc transport generating a udev event with directory-controller addressing information. The nvme core layer generates an event when a persistent discovery controller generates a Discovery Log Change Notification AEN. The nvme-fc transport generates an event when an FC rport that has a NVME Discovery controller is detected or when a FC state change event occurs for for an FC rport that has a NVME Discovery controller

The udev event is handled by a script that extracts the Discovery controller addressing information and initiates a systemd service to perform a "nvme connect-all" to the Discovery controller. The "nvme connect-all" request is not called directly from the udev handler itself as the request may take some time or stall altogether, which would block other udev event handling. By transitioning to a sytemd service, the call can take as much time as needed to complete.

The scripts consist of:
- A udev script that handles nvme core and nvme-fc udev events. The udev handler starts a nvmf-connect systemd service.
- A nvmf-connect systemd service. The service, in its instance name, is passed the connect arguments for the discovery controller. The service performs a "nvme connect-all" to the discovery controller.
- A nvmefc-boot-connections systemd service. This is a run-once service run after udev is enabled, which will replay events generated by NVME-FC devices detected during boot while udev is not yet running.
- To stop autoconnect an additional nvmefc-connect.target has been added, which will instruct systemd to cancel all outstanding autoconnect services.

Note: Although the nvme-fc subsystem is converting to use the
same nvme core layer event mechanism, the nvme-fc-specific udev event that has been in existence for a while is contained in in the script so that the utilities may run against older kernels.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Simon Schricker <sschricker at suse.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
CC Sagi Grimberg <sagi at grimberg.me>

---
Added install lines for 70-nvmf-autoconnect.conf and nvmf-connect.target Added LIBDIR to Makefile, hardset to /usr/lib to match nvme.spec.in
(thus why it is not using PREFIX)
Revised Makefile to pass LIBDIR to rpmbuild.
Revised nvme.spec.in to use %{_libdir}
Removed bogus fi in nvme.spec.in
Added --quiet option udev scripts that specify nvme connect-all args

---
 Makefile                                         | 16 ++++++++++++++--
 nvme.spec.in                                     |  9 +++++++++
 nvmf-autoconnect/70-nvmf-autoconnect.conf        |  1 +
 nvmf-autoconnect/70-nvmf-autoconnect.rules       | 18 ++++++++++++++++++
 nvmf-autoconnect/nvmefc-boot-connections.service |  9 +++++++++
 nvmf-autoconnect/nvmf-connect.target             |  2 ++
 nvmf-autoconnect/nvmf-connect at .service           | 14 ++++++++++++++
 7 files changed, 67 insertions(+), 2 deletions(-)
 create mode 100644 nvmf-autoconnect/70-nvmf-autoconnect.conf
 create mode 100644 nvmf-autoconnect/70-nvmf-autoconnect.rules
 create mode 100644 nvmf-autoconnect/nvmefc-boot-connections.service
 create mode 100644 nvmf-autoconnect/nvmf-connect.target
 create mode 100644 nvmf-autoconnect/nvmf-connect at .service
Makefile +14 -2
index db594a3..4ea8b41 100644
@@ -8,6 +8,10 @@ DESTDIR =
 PREFIX ?= /usr/local
 SYSCONFDIR = /etc
 SBINDIR = $(PREFIX)/sbin
+LIBDIR ?= /usr/lib
+SYSTEMDDIR ?= $(LIBDIR)/systemd
+UDEVDIR ?= $(LIBDIR)/udev
+DRACUTDIR ?= $(LIBDIR)/dracut
 LIB_DEPENDS =
 
 ifeq ($(LIBUUID),0)
@@ -89,6 +93,14 @@ install-bash-completion:
 	$(INSTALL) -d $(DESTDIR)$(PREFIX)/share/bash-completion/completions
 	$(INSTALL) -m 644 -T ./completions/bash-nvme-completion.sh $(DESTDIR)$(PREFIX)/share/bash-completion/completions/nvme
 
+install-udev:
+	$(INSTALL) -d $(DESTDIR)$(SYSTEMDDIR)/system
+	$(INSTALL) -m 644 ./nvmf-autoconnect/*.service ./nvmf-autoconnect/*.target $(DESTDIR)$(SYSTEMDDIR)/system
+	$(INSTALL) -d $(DESTDIR)$(UDEVDIR)/rules.d
+	$(INSTALL) -m 644 ./nvmf-autoconnect/*.rules $(DESTDIR)$(UDEVDIR)/rules.d
+	$(INSTALL) -d $(DESTDIR)$(DRACUTDIR)/dracut.conf.d
+	$(INSTALL) -m 644 ./nvmf-autoconnect/*.conf $(DESTDIR)$(DRACUTDIR)/dracut.conf.d
+
 install-zsh-completion:
 	$(INSTALL) -d $(DESTDIR)$(PREFIX)/share/zsh/site-functions
 	$(INSTALL) -m 644 -T ./completions/_nvme $(DESTDIR)$(PREFIX)/share/zsh/site-functions/_nvme
@@ -109,7 +121,7 @@ install-etc:
 		$(INSTALL) -m 644 -T ./etc/discovery.conf.in $(DESTDIR)$(SYSCONFDIR)/nvme/discovery.conf; \
 	fi
 
-install-spec: install-bin install-man install-bash-completion install-zsh-completion install-etc
+install-spec: install-bin install-man install-bash-completion install-zsh-completion install-etc install-udev
 install: install-spec install-hostparams
 
 nvme.spec: nvme.spec.in NVME-VERSION-FILE
@@ -177,7 +189,7 @@ deb-light: $(NVME) pkg nvme.control.in
 	dpkg-deb --build nvme-$(NVME_VERSION)
 
 rpm: dist
-	$(RPMBUILD) -ta nvme-$(NVME_VERSION).tar.gz
+	$(RPMBUILD) --define '_libdir ${LIBDIR}' -ta nvme-$(NVME_VERSION).tar.gz
 
 .PHONY: default doc all clean clobber install-man install-bin install
 .PHONY: dist pkg dist-orig deb deb-light rpm FORCE test
nvme.spec.in +9 -0
index 6934f8f..1286e58 100644
@@ -35,6 +35,11 @@ make install-spec DESTDIR=%{buildroot} PREFIX=/usr
 %{_sysconfdir}/nvme/hostnqn
 %{_sysconfdir}/nvme/hostid
 %{_sysconfdir}/nvme/discovery.conf
+%{_libdir}/udev/rules.d/70-nvmf-autoconnect.rules
+%{_libdir}/dracut/dracut.conf.d/70-nvmf-autoconnect.conf
+%{_libdir}/systemd/system/nvmf-connect at .service
+%{_libdir}/systemd/system/nvmefc-boot-connections.service
+%{_libdir}/systemd/system/nvmf-connect.target
 
 %clean
 rm -rf $RPM_BUILD_ROOT
@@ -47,6 +52,10 @@ if [ $1 -eq 1 ]; then # 1 : This package is being installed for the first time
         if [ ! -s %{_sysconfdir}/nvme/hostid ]; then
                 uuidgen > %{_sysconfdir}/nvme/hostid
         fi
+
+	# apply udev and systemd changes that we did
+	udevadm control --reload-rules && udevadm trigger
+	systemctl daemon-reload
 fi
 
 %changelog
nvmf-autoconnect/70-nvmf-autoconnect.conf new file +1 -0
new file mode 100644
index 0000000..844f3d9
@@ -0,0 +1 @@
+install_items+="/usr/lib/udev/rules.d/70-nvmf-autoconnect.rules"
nvmf-autoconnect/70-nvmf-autoconnect.rules new file +18 -0
new file mode 100644
index 0000000..b241b12
@@ -0,0 +1,18 @@
+#
+# nvmf-autoconnect.rules:
+#   Handles udev events which invoke automatically scan via discovery
+#   controller and connect to elements in the discovery log.
+#
+#
+
+# Events from persistent discovery controllers or nvme-fc transport events
+ACTION=="change", SUBSYSTEM=="nvme", ENV{NVME_EVENT}=="discovery",\
+  ENV{NVME_CTLR_NAME}=="*", ENV{NVME_TRTYPE}=="*", ENV{NVME_TRADDR}=="*", \
+  ENV{NVME_TRSVCID}=="*", ENV{NVME_HOST_TRADDR}=="*", \
+  RUN+="/usr/bin/systemctl --no-block start nvmf-connect at --quiet\t--device=$env{NVME_CTLR_NAME}\t--transport=$env{NVME_TRTYPE}\t--traddr=$env{NVME_TRADDR}\t--trsvcid=$env{NVME_TRSVCID}\t--host-traddr=$env{NVME_HOST_TRADDR}.service"
+
+# nvme-fc transport generated events (old-style for compatibility)
+ACTION=="change", SUBSYSTEM=="fc", ENV{FC_EVENT}=="nvmediscovery", \
+  ENV{NVMEFC_HOST_TRADDR}=="*",  ENV{NVMEFC_TRADDR}=="*", \
+  RUN+="/usr/bin/systemctl --no-block start nvmf-connect at --quiet\t--device=none\t--transport=fc\t--traddr=$env{NVMEFC_TRADDR}\t--trsvcid=none\t--host-traddr=$env{NVMEFC_HOST_TRADDR}.service"
+
nvmf-autoconnect/nvmefc-boot-connections.service new file +9 -0
new file mode 100644
index 0000000..84f6486
@@ -0,0 +1,9 @@
+[Unit]
+Description=Auto-connect to subsystems on FC-NVME devices found during boot
+
+[Service]
+Type=oneshot
+ExecStart=/bin/sh -c "echo add > /sys/class/fc/fc_udev_device/nvme_discovery"
+
+[Install]
+WantedBy=default.target
nvmf-autoconnect/nvmf-connect.target new file +2 -0
new file mode 100644
index 0000000..f64a37c
@@ -0,0 +1,2 @@
+[Unit]
+Description=All instances of nvmf-autoconnect daemon
nvmf-autoconnect/nvmf-connect at .service new file +14 -0
new file mode 100644
index 0000000..7d9f856
@@ -0,0 +1,14 @@
+#
+# Unit file used by 70-nvmf-autoconnect.rules.
+#
+
+[Unit]
+Description=NVMf auto-connect scan upon nvme discovery controller Events
+After=syslog.target
+PartOf=nvmf-connect.target
+Requires=nvmf-connect.target
+
+[Service]
+Type=simple
+Environment="CONNECT_ARGS=%i"
+ExecStart=/bin/sh -c "/usr/sbin/nvme connect-all `/usr/bin/echo -e $CONNECT_ARGS`"
Signature · 1 line
2.13.7
maintainer
#29 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

Quoted 87 lines · James Smart

This set of scripts is a combination of those sent by Hannes, Sagi, and I in earlier patches and RFC's.

Auto-connect operates by the nvme core layer or nvme-fc transport generating a udev event with directory-controller addressing information. The nvme core layer generates an event when a persistent discovery controller generates a Discovery Log Change Notification AEN. The nvme-fc transport generates an event when an FC rport that has a NVME Discovery controller is detected or when a FC state change event occurs for for an FC rport that has a NVME Discovery controller

The udev event is handled by a script that extracts the Discovery controller addressing information and initiates a systemd service to perform a "nvme connect-all" to the Discovery controller. The "nvme connect-all" request is not called directly from the udev handler itself as the request may take some time or stall altogether, which would block other udev event handling. By transitioning to a sytemd service, the call can take as much time as needed to complete.

The scripts consist of:
- A udev script that handles nvme core and nvme-fc udev events. The udev handler starts a nvmf-connect systemd service.
- A nvmf-connect systemd service. The service, in its instance name, is passed the connect arguments for the discovery controller. The service performs a "nvme connect-all" to the discovery controller.
- A nvmefc-boot-connections systemd service. This is a run-once service run after udev is enabled, which will replay events generated by NVME-FC devices detected during boot while udev is not yet running.
- To stop autoconnect an additional nvmefc-connect.target has been added, which will instruct systemd to cancel all outstanding autoconnect services.

Note: Although the nvme-fc subsystem is converting to use the
same nvme core layer event mechanism, the nvme-fc-specific udev event that has been in existence for a while is contained in in the script so that the utilities may run against older kernels.

Signed-off-by James Smart <jsmart2021 at gmail.com>
CC Simon Schricker <sschricker at suse.com>
Reviewed-by Hannes Reinecke <hare at suse.com>
CC Sagi Grimberg <sagi at grimberg.me>

---
Added install lines for 70-nvmf-autoconnect.conf and nvmf-connect.target Added LIBDIR to Makefile, hardset to /usr/lib to match nvme.spec.in
(thus why it is not using PREFIX)
Revised Makefile to pass LIBDIR to rpmbuild.
Revised nvme.spec.in to use %{_libdir}
Removed bogus fi in nvme.spec.in
Added --quiet option udev scripts that specify nvme connect-all args

---
  Makefile                                         | 16 ++++++++++++++--
  nvme.spec.in                                     |  9 +++++++++
  nvmf-autoconnect/70-nvmf-autoconnect.conf        |  1 +
  nvmf-autoconnect/70-nvmf-autoconnect.rules       | 18 ++++++++++++++++++
  nvmf-autoconnect/nvmefc-boot-connections.service |  9 +++++++++
  nvmf-autoconnect/nvmf-connect.target             |  2 ++
  nvmf-autoconnect/nvmf-connect at .service           | 14 ++++++++++++++
  7 files changed, 67 insertions(+), 2 deletions(-)

create mode 100644 nvmf-autoconnect/70-nvmf-autoconnect.conf create mode 100644 nvmf-autoconnect/70-nvmf-autoconnect.rules create mode 100644 nvmf-autoconnect/nvmefc-boot-connections.service create mode 100644 nvmf-autoconnect/nvmf-connect.target create mode 100644 nvmf-autoconnect/nvmf-connect at .service

Makefile +4 -0
index db594a3..4ea8b41 100644
@@ -8,6 +8,10 @@ DESTDIR =
  PREFIX ?= /usr/local
  SYSCONFDIR = /etc
  SBINDIR = $(PREFIX)/sbin
+LIBDIR ?= /usr/lib
+SYSTEMDDIR ?= $(LIBDIR)/systemd
+UDEVDIR ?= $(LIBDIR)/udev
+DRACUTDIR ?= $(LIBDIR)/dracut
  LIB_DEPENDS =
 
  ifeq ($(LIBUUID),0)
@@ -89,6 +93,14 @@ install-bash-completion:
  	$(INSTALL) -d $(DESTDIR)$(PREFIX)/share/bash-completion/completions

Makefile install-bash-completion: · line 103

Quoted 10 lines
Makefile +8 -0
  	$(INSTALL) -m 644 -T ./completions/bash-nvme-completion.sh $(DESTDIR)$(PREFIX)/share/bash-completion/completions/nvme
 
+install-udev:
+	$(INSTALL) -d $(DESTDIR)$(SYSTEMDDIR)/system
+	$(INSTALL) -m 644 ./nvmf-autoconnect/*.service ./nvmf-autoconnect/*.target $(DESTDIR)$(SYSTEMDDIR)/system
+	$(INSTALL) -d $(DESTDIR)$(UDEVDIR)/rules.d
+	$(INSTALL) -m 644 ./nvmf-autoconnect/*.rules $(DESTDIR)$(UDEVDIR)/rules.d
+	$(INSTALL) -d $(DESTDIR)$(DRACUTDIR)/dracut.conf.d
+	$(INSTALL) -m 644 ./nvmf-autoconnect/*.conf $(DESTDIR)$(DRACUTDIR)/dracut.conf.d
+

I think we want to have service files in systemd/ dir, rules files in udev/ dir, and dracut files in dracut/ dir. This way we can easily add more things that are not related to autoreconnect.

Makefile install-etc: · line 124

Quoted 9 lines
Makefile +1 -1
  install-zsh-completion:
  	$(INSTALL) -d $(DESTDIR)$(PREFIX)/share/zsh/site-functions
  	$(INSTALL) -m 644 -T ./completions/_nvme $(DESTDIR)$(PREFIX)/share/zsh/site-functions/_nvme
@@ -109,7 +121,7 @@ install-etc:
  		$(INSTALL) -m 644 -T ./etc/discovery.conf.in $(DESTDIR)$(SYSCONFDIR)/nvme/discovery.conf; \
  	fi
 
-install-spec: install-bin install-man install-bash-completion install-zsh-completion install-etc
+install-spec: install-bin install-man install-bash-completion install-zsh-completion install-etc install-udev

Lets separate to targets:
install-udev
install-systemd
install-dracut

maintainer
#30 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

In reply to James Smart
Quoted 11 lines
nvmf-autoconnect/nvmf-connect at .service new file +4 -0
new file mode 100644
index 0000000..7d9f856
@@ -0,0 +1,14 @@
+#
+# Unit file used by 70-nvmf-autoconnect.rules.
+#
+

nvmf-autoconnect/nvmf-connect at .service line 14

Quoted 10 lines
nvmf-autoconnect/nvmf-connect at .service new file +10 -0
+[Unit]
+Description=NVMf auto-connect scan upon nvme discovery controller Events
+After=syslog.target
+PartOf=nvmf-connect.target
+Requires=nvmf-connect.target
+
+[Service]
+Type=simple
+Environment="CONNECT_ARGS=%i"
+ExecStart=/bin/sh -c "/usr/sbin/nvme connect-all `/usr/bin/echo -e $CONNECT_ARGS`"

Lets lose the nvme full path, sh allows us to have that (can be in /usr/local/sbin or somewhere else). and not /bin/echo?

#31 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

Quoted 6 lines · Sagi Grimberg
Quoted 3 lines

+ExecStart=/bin/sh -c "/usr/sbin/nvme connect-all /usr/bin/echo -e $CONNECT_ARGS"

Lets lose the nvme full path, sh allows us to have that (can be in /usr/local/sbin or somewhere else). and not /bin/echo?

For nvme, echo or both ?

maintainer
#32 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

Quoted 10 lines · Sagi Grimberg
Quoted 6 lines
Quoted 3 lines

+ExecStart=/bin/sh -c "/usr/sbin/nvme connect-all /usr/bin/echo -e $CONNECT_ARGS"

Lets lose the nvme full path, sh allows us to have that (can be in /usr/local/sbin or somewhere else). and not /bin/echo?

For nvme, echo or both ?

For nvme for sure, for echo, it should always be in /bin/echo so whatever you prefer.

maintainer
#33 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

In reply to James Smart
Quoted 9 lines
nvmf-autoconnect/70-nvmf-autoconnect.rules new file +2 -0
new file mode 100644
index 0000000..b241b12
@@ -0,0 +1,18 @@
+#
+# nvmf-autoconnect.rules:

nvmf-autoconnect/70-nvmf-autoconnect.rules line 12

Quoted 10 lines
nvmf-autoconnect/70-nvmf-autoconnect.rules new file +10 -0
+#   Handles udev events which invoke automatically scan via discovery
+#   controller and connect to elements in the discovery log.
+#
+#
+
+# Events from persistent discovery controllers or nvme-fc transport events
+ACTION=="change", SUBSYSTEM=="nvme", ENV{NVME_EVENT}=="discovery",\
+  ENV{NVME_CTLR_NAME}=="*", ENV{NVME_TRTYPE}=="*", ENV{NVME_TRADDR}=="*", \
+  ENV{NVME_TRSVCID}=="*", ENV{NVME_HOST_TRADDR}=="*", \
+  RUN+="/usr/bin/systemctl --no-block start nvmf-connect at --quiet\t--device=$env{NVME_CTLR_NAME}\t--transport=$env{NVME_TRTYPE}\t--traddr=$env{NVME_TRADDR}\t--trsvcid=$env{NVME_TRSVCID}\t--host-traddr=$env{NVME_HOST_TRADDR}.service"

NVME_CTRL_NAME

#34 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

Quoted 17 lines · Sagi Grimberg
Quoted 17 lines
nvmf-autoconnect/70-nvmf-autoconnect.rules new file +8 -0
new file mode 100644
index 0000000..b241b12
@@ -0,0 +1,18 @@
+#
+# nvmf-autoconnect.rules:
+#?? Handles udev events which invoke automatically scan via discovery
+#?? controller and connect to elements in the discovery log.
+#
+#
+
+# Events from persistent discovery controllers or nvme-fc transport 

events

Quoted 9 lines
Quoted 6 lines

+ACTION=="change", SUBSYSTEM=="nvme", ENV{NVME_EVENT}=="discovery",\
+? ENV{NVME_CTLR_NAME}=="*", ENV{NVME_TRTYPE}=="*",
ENV{NVME_TRADDR}=="*", \
+? ENV{NVME_TRSVCID}=="*", ENV{NVME_HOST_TRADDR}=="*", \
+? RUN+="/usr/bin/systemctl --no-block start
nvmf-connect at --quiet\t--device=$env{NVME_CTLR_NAME}\t--transport=$env{NVME_TRTYPE}\t--traddr=$env{NVME_TRADDR}\t--trsvcid=$env{NVME_TRSVCID}\t--host-traddr=$env{NVME_HOST_TRADDR}.service"

NVME_CTRL_NAME

ok

maintainer
#35 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

In reply to James Smart

Makefile DESTDIR = · line 13

Quoted 12 lines
Makefile +3 -0
index db594a3..4ea8b41 100644
@@ -8,6 +8,10 @@ DESTDIR =
  PREFIX ?= /usr/local
  SYSCONFDIR = /etc
  SBINDIR = $(PREFIX)/sbin
+LIBDIR ?= /usr/lib
+SYSTEMDDIR ?= $(LIBDIR)/systemd
+UDEVDIR ?= $(LIBDIR)/udev

UDEVDIR needs to be built from $(SYSCONFDIR)/udev as LIBDIR might be /usr/local/lib and udev doesn't look there.

#36 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

Quoted 15 lines · Sagi Grimberg

Makefile DESTDIR = · line 13

Quoted 12 lines
Makefile +0 -0
index db594a3..4ea8b41 100644
@@ -8,6 +8,10 @@ DESTDIR =

? PREFIX ?= /usr/local
? SYSCONFDIR = /etc
? SBINDIR = $(PREFIX)/sbin

Makefile +3 -0
+LIBDIR ?= /usr/lib
+SYSTEMDDIR ?= $(LIBDIR)/systemd
+UDEVDIR ?= $(LIBDIR)/udev

UDEVDIR needs to be built from $(SYSCONFDIR)/udev as LIBDIR might be /usr/local/lib and udev doesn't look there.

I really would like to change prefix to /usr, so that it matches what is in nvme.spec.in where the %install paragraph sets PREFIX to /usr.?? We have two separate places setting a value and they are in conflict.

Based on the spec file setting - the only make targets that match PREFIX outside of the install targets is the pkg target which is mainly man files and a copy of nvme.? The man files perhaps would be under /usr/local, but everything else we're talking about would be very odd under /usr/local.?? Perhaps a better answer is PREFIX=/usr (which also set SBINDIR for the nvme copy), LIBDIR=$(PREFIX)/lib, other variables the same, and add a MANPREFIX=/usr/local for the man files.

-- james

maintainer
#37 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

Quoted 22 lines · Sagi Grimberg
Quoted 15 lines

Makefile DESTDIR = · line 13

Quoted 12 lines
Makefile +0 -0
index db594a3..4ea8b41 100644
@@ -8,6 +8,10 @@ DESTDIR =

? PREFIX ?= /usr/local
? SYSCONFDIR = /etc
? SBINDIR = $(PREFIX)/sbin

Makefile +3 -0
+LIBDIR ?= /usr/lib
+SYSTEMDDIR ?= $(LIBDIR)/systemd
+UDEVDIR ?= $(LIBDIR)/udev

UDEVDIR needs to be built from $(SYSCONFDIR)/udev as LIBDIR might be /usr/local/lib and udev doesn't look there.

I really would like to change prefix to /usr, so that it matches what is in nvme.spec.in where the %install paragraph sets PREFIX to /usr.?? We have two separate places setting a value and they are in conflict.

Quoted 7 lines

Based on the spec file setting - the only make targets that match PREFIX outside of the install targets is the pkg target which is mainly man files and a copy of nvme.? The man files perhaps would be under /usr/local, but everything else we're talking about would be very odd under /usr/local.?? Perhaps a better answer is PREFIX=/usr (which also set SBINDIR for the nvme copy), LIBDIR=$(PREFIX)/lib, other variables the same, and add a MANPREFIX=/usr/local for the man files.

Why not just place it in /etc? its equivalent and works regardless.

#38 lore raw

RFC PATCH v2 10/10 nvme-cli: nvmf auto-connect scripts

Quoted 32 lines · Sagi Grimberg
Quoted 23 lines · Sagi Grimberg
Quoted 15 lines

Makefile DESTDIR = · line 13

Quoted 12 lines
Makefile +0 -0
index db594a3..4ea8b41 100644
@@ -8,6 +8,10 @@ DESTDIR =

? PREFIX ?= /usr/local
? SYSCONFDIR = /etc
? SBINDIR = $(PREFIX)/sbin

Makefile +3 -0
+LIBDIR ?= /usr/lib
+SYSTEMDDIR ?= $(LIBDIR)/systemd
+UDEVDIR ?= $(LIBDIR)/udev

UDEVDIR needs to be built from $(SYSCONFDIR)/udev as LIBDIR might be /usr/local/lib and udev doesn't look there.

I really would like to change prefix to /usr, so that it matches what is in nvme.spec.in where the %install paragraph sets PREFIX to /usr.?? We have two separate places setting a value and they are in conflict.

Quoted 8 lines

Based on the spec file setting - the only make targets that match PREFIX outside of the install targets is the pkg target which is mainly man files and a copy of nvme.? The man files perhaps would be under /usr/local, but everything else we're talking about would be very odd under /usr/local.?? Perhaps a better answer is PREFIX=/usr (which also set SBINDIR for the nvme copy), LIBDIR=$(PREFIX)/lib, other variables the same, and add a MANPREFIX=/usr/local for the man files.

Quoted 1 line

Why not just place it in /etc? its equivalent and works regardless.

you lost me.? "/etc"?

I know of no reason not to set PREFIX to /usr. As this package becomes standard, it is / will be there. Today's nvme utility is installed in /usr.? /usr/local does not seem correct.

-- james