Skip to contentSkip to conversation

SUSE SLES resync

PATCH In review Hannes Reinecke 3 participants 98 messages Mar 16, 2015 – Mar 30, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover SUSE SLES resync

Hi Christophe,

here are the patches which I've queued up during SLES12 development. Most of them are (more or less) obvious bugfixes, but there are some things which would warrant a deeper review:
- device-mapper cookie handling: This _really_ is an arcane topic, and I would love to have someone from the device-mapper team to review it. I'm still facing some issues where udev emits a warning 'conflicting device node /dev/mapper/XXX ..', so there might be some issue with it.
- Asynchronous configuration: On large systems configure() as being called during multipathd startup might exceed the systemd job timeout, resulting in multipathd being killed by systemd. For that I've been implementing an udev retrigger method, which just issues an 'add' event for all available block devices.
- Related to the above: I've implemented an asynchronous 'reconfigure' CLI call, as for the same reason systemd might kill the multipathd during a 'restart' operation
- systemd/udev integration: This is _a pain_. It took me months to come up with a working solution. _Especially_ as apparently RH is shipping an '11-dm-multipath.rules' udev ruleset for multipathd, which supposedly should be handling this.
It didn't do it for me, though, so I've attached my own version here. The udev integration I've been settling for works like this:
- 11-dm-mpath.rules checks if the device is eligible for multipathing For that I've implemented a new option '-u' which looks at the uevent variables _only_.
- if a device should be handled by multipathing the 'by-id' and 'by-uuid' persistent symlinks are _not_ generated, as this would trigger systemd to start scanning/mounting/lvm activation/whatnot; the 'by-id' and 'by-uuid' symlinks are only generated for the multipath devices themselves.
- I've added a new multipath.rules file which should be inserted between 55-scsi-sg3_id.rules and 58-scsi_sg3_symlink.rules. This will intercept the SCSI IDENT variables and blank them out if the (block) device is handled by multipathing.
- sysfs VPD page handling. I've planned to implement this as a direct replacement of the original scsi_id program, but then it turned out that our WWID handling is actually too generic for that: Both getuid and uid_attribute can basically take _any_ value, so it's impossible to tell if the system uses the original (scsi_id compatible) IDs or not. And as the VPD parsing code can only generate scsi_id compatible IDs we might run into compability issues.
So it's disabled per default. I'd really would like to clean it up, as I suspect that about 95% of the installations are using scsi_id compatible IDs. But for this to happen we need to clarify/restrict the ID generation mechanism.

As usual, comments and reviews are welcome.

The patchset is based upon the 'find_multipaths' patchset from Ben Marzinski; it can be retrieved from

github.com:/hreinecke/multipath-tools.git branch suse-resync

Cheers,

Hannes

Hannes Reinecke (78):
libmultipath: remove compilation warning in devmapper.c
mpath_persist: Do not call exit() from a shared library
libmultipath: filter for missing property in get_refwwid() Double uevent stacksize yet again
discovery: do not fail discovery on individual devices
libmultipath: Prefer deprecated 'getuid' callout
libmultipath: Skip paths with empty wwid
Make systemd installation path configurable
Add multipath rules for systemd support
Fixup multipathd.socket to resolve ordering dependeny Fixup dependencies in multipathd.service
multipathd: set correct PID when running in debug mode Do not print empty device strings during discovery kpartx.rules: do not call blkid
Use 'SCSI_IDENT_.*' as the default property whitelist Fixup wwid blacklist printing
Allow for empty path argument when printing information Disable reassign maps per default
multipathd: implement 'list path <path>' cli command Make checker_put() and prio_put() idempotent Remove trailing linefeed from sysfs attributes
multipath: implement option '-u' for uevents Install multipath rule under '56-multipath.rules' multipath.rules: Whitelist devices multipath.rules: fixup race condition with systemd 11-dm-mpath.rules: Import blkid values if all paths are down kpartx.rules: Skip kpartx for multipath events
multipathd: handle DOMAP_RETRY
multipathd: cleanup foreground operation
Update hwtable for EMC XtremIO
multipath: check for running daemon when called with '-u' Revert 'return PATH_DOWN for quiesced paths'
Do not treat 'transport-offline' paths as 'offline' Check for valid DM_DEVICE_INFO before proceeding Separate out uevent parsing functions
Use poll() when receiving uevents
mpath_persist: cleanup
kpartx: use standard 'major' and 'minor' macros
multipath: Use standard 'major' macro
Remove sysfs_get_dev
Add paths with a size of '0' as 'ghost' paths Remove last argument from verify_paths()
Fixup device-mapper 'cookie' handling
multipath: do not print state 'orphan' for option '-l' Return error when receiving CLI packet
Implement 'uxsock_timeout' keyword
Do not print empty multipaths section
multipathd: reload map if reinstate failed
multipathd: do not remove paths without uevents Allow zero-sized devices during configuration Rework uev_add_path()
multipathd: Issue warning on CLI command timeout Use strlen() when checking for valid wwid
multipathd: Use standard lists for CLI handling
uxlsnr: use typedef for trigger function
multipathd: lock cli client list
multipath: enable sync support
Remove dm_udev_XXX wrapper functions
Revert to ACT_RELOAD in domap if the map exists
multipathd: use local variable for watchdog configuration Ignore devices when sysfs_get_tgt_nodename fails Skip USB devices during discovery
Read wwid from sysfs vpg_pg83 attribute
Assign local priority for NAA VPD descriptor
Use sysfs attribute vpd_pg80 to read serial number Update multipath.conf.5 to clarify wwid generation
libmultipath: Fall back to SG_IO if no UID could be assigned
libmultipath: unset 'uid_attribute' on failure Separate out vpd parsing functions
multipathd: use SG_IO as fallback to generate uid Do not automatically fall back to vpd uid generation
libmultipath: make vpd page 0x80 optional
multipathd: push down lock in checkerloop()
Allow specific CLI commands to run unlocked
Push down vector lock during uevent processing
multipathd: timeout CLI commands when waiting for lock
multipathd: asynchronous configuration
multipathd: trigger all devices on startup

 Makefile.inc                     |   8 +-
 kpartx/devmapper.c               |  53 +++-
 kpartx/devmapper.h               |  11 +-
 kpartx/kpartx.c                  |  35 +--
 kpartx/kpartx.rules              |  10 +-
 libmpathpersist/mpath_persist.c  | 172 ++++++-----
 libmpathpersist/mpath_updatepr.c |  20 +-
 libmpathpersist/mpathpr.h        |   3 +-
 libmultipath/blacklist.c         |   8 +-
 libmultipath/blacklist.h         |   2 +-
 libmultipath/checkers.c          |   2 +-
 libmultipath/config.c            |   2 +
 libmultipath/config.h            |   8 +-
 libmultipath/configure.c         |  82 ++++--
 libmultipath/defaults.h          |   4 +-
 libmultipath/devmapper.c         |  77 +++--
 libmultipath/devmapper.h         |   4 +-
 libmultipath/dict.c              |  33 +++
 libmultipath/discovery.c         | 613 ++++++++++++++++++++++++++++++++++-----
 libmultipath/discovery.h         |   8 +-
 libmultipath/hwtable.c           |  13 +
 libmultipath/list.h              |  49 +++-
 libmultipath/lock.h              |   4 +-
 libmultipath/log_pthread.c       |   2 +-
 libmultipath/print.c             |  20 +-
 libmultipath/prio.c              |   4 +-
 libmultipath/propsel.c           |   6 +-
 libmultipath/structs_vec.c       |  29 +-
 libmultipath/structs_vec.h       |   2 +-
 libmultipath/sysfs.c             |  53 ++++
 libmultipath/sysfs.h             |   2 +
 libmultipath/uevent.c            | 316 ++++++++++++--------
 libmultipath/uxsock.c            |  32 +-
 libmultipath/uxsock.h            |   4 +-
 multipath.conf.defaults          |   2 +-
 multipath/11-dm-mpath.rules      |   8 +-
 multipath/Makefile               |   2 +
 multipath/dev_t.h                |   3 -
 multipath/main.c                 |  53 +++-
 multipath/multipath.8            |   6 +-
 multipath/multipath.conf.5       |  42 ++-
 multipath/multipath.rules        |  14 +
 multipathd/cli.c                 |  47 ++-
 multipathd/cli.h                 |   4 +-
 multipathd/cli_handlers.c        |  64 +++-
 multipathd/cli_handlers.h        |   1 +
 multipathd/main.c                | 400 ++++++++++++++++---------
 multipathd/main.h                |   2 +
 multipathd/multipathd.service    |   4 +-
 multipathd/multipathd.socket     |   8 +-
 multipathd/uxclnt.c              |  26 +-
 multipathd/uxclnt.h              |   2 +-
 multipathd/uxlsnr.c              | 102 +++++--
 multipathd/uxlsnr.h              |   8 +-
 54 files changed, 1834 insertions(+), 655 deletions(-)
 delete mode 100644 multipath/dev_t.h
 create mode 100644 multipath/multipath.rules
Signature · 1 line
1.8.4.5
#1 lore raw

PATCH 1/78 libmultipath: remove compilation warning in devmapper.c

When LIBDM_API_DEFERRED is not defined the compiler would warn about 'dm_cancel_remove_partmaps' being defined but never used.

---
 libmultipath/devmapper.c | 2 ++
 1 file changed, 2 insertions(+)
libmultipath/devmapper.c +2 -0
index 69491a3..5e68aeb 100644
@@ -32,7 +32,9 @@
 #define UUID_PREFIX "mpath-"
 #define UUID_PREFIX_LEN 6
 
+#ifdef LIBDM_API_DEFERRED
 static int dm_cancel_remove_partmaps(const char * mapname);
+#endif
 
 #ifndef LIBDM_API_COOKIE
 static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
Signature · 1 line
1.8.4.5
#2 lore raw

PATCH 2/78 mpath_persist: Do not call exit() from a shared library

In reply to Hannes Reinecke

mpath_persist is a shared library, and shouldn't call exit(). Just return an error code here.

And cleanup formatting while we're at it.

---
 libmpathpersist/mpath_persist.c | 52 ++++++++++++++++++++++++-----------------
 1 file changed, 31 insertions(+), 21 deletions(-)
libmpathpersist/mpath_persist.c +31 -21
index bd30125..9f65ca6 100644
@@ -386,18 +386,20 @@ void * mpath_prin_pthread_fn (void *p)
 	int ret;
 	struct prin_param * pparam = (struct prin_param *)p;
 
-	ret = prin_do_scsi_ioctl(pparam->dev, pparam->rq_servact, pparam->resp,  pparam->noisy);
-	pparam->status = ret;	
-	pthread_exit(NULL);	
+	ret = prin_do_scsi_ioctl(pparam->dev, pparam->rq_servact,
+				 pparam->resp,  pparam->noisy);
+	pparam->status = ret;
+	pthread_exit(NULL);
 }
 
-int mpath_send_prin_activepath (char * dev, int rq_servact, struct prin_resp * resp, int noisy)
+int mpath_send_prin_activepath (char * dev, int rq_servact,
+				struct prin_resp * resp, int noisy)
 {
 
 	int rc;
 
 	rc = prin_do_scsi_ioctl(dev, rq_servact, resp,  noisy);
-	
+
 	return (rc);
 }
 
@@ -409,14 +411,14 @@ int mpath_prout_reg(struct multipath *mpp,int rq_servact, int rq_scope,
 	struct pathgroup *pgp = NULL;
 	struct path *pp = NULL;
 	int rollback = 0;
-	int active_pathcount=0;	
+	int active_pathcount=0;
 	int rc;
 	int count=0;
 	int status = MPATH_PR_SUCCESS;
 	uint64_t sa_key = 0;
 
 	if (!mpp)
-		return MPATH_PR_DMMP_ERROR; 
+		return MPATH_PR_DMMP_ERROR;
 
 	active_pathcount = pathcount(mpp, PATH_UP) + pathcount(mpp, PATH_GHOST);
 
@@ -444,12 +446,13 @@ int mpath_prout_reg(struct multipath *mpp,int rq_servact, int rq_scope,
 
 		condlog (3, "THRED ID [%d] INFO]", i);
 		condlog (3, "rq_servact=%d ", thread[i].param.rq_servact);
-		condlog (3, "rq_scope=%d ", thread[i].param.rq_scope); 
-		condlog (3, "rq_type=%d ", thread[i].param.rq_type);  
-		condlog (3, "rkey="); 
-		condlog (3, "paramp->sa_flags =%02x ", thread[i].param.paramp->sa_flags); 
-		condlog (3, "noisy=%d ", thread[i].param.noisy); 
-		condlog (3, "status=%d ", thread[i].param.status); 
+		condlog (3, "rq_scope=%d ", thread[i].param.rq_scope);
+		condlog (3, "rq_type=%d ", thread[i].param.rq_type);
+		condlog (3, "rkey=");
+		condlog (3, "paramp->sa_flags =%02x ",
+			 thread[i].param.paramp->sa_flags);
+		condlog (3, "noisy=%d ", thread[i].param.noisy);
+		condlog (3, "status=%d ", thread[i].param.status);
 	}
 
 	pthread_attr_t attr;
@@ -548,13 +551,15 @@ int mpath_prout_common(struct multipath *mpp,int rq_servact, int rq_scope,
 	vector_foreach_slot (mpp->pg, pgp, j){
 		vector_foreach_slot (pgp->paths, pp, i){
 			if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){
-				condlog (1, "%s: %s path not up. Skip", mpp->wwid, pp->dev); 
+				condlog (1, "%s: %s path not up. Skip",
+					 mpp->wwid, pp->dev);
 				continue;
 			}
 
 			condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev);
-			ret = send_prout_activepath(pp->dev, rq_servact, rq_scope, rq_type, 
-					paramp, noisy); 
+			ret = send_prout_activepath(pp->dev, rq_servact,
+						    rq_scope, rq_type,
+						    paramp, noisy);
 			return ret ;
 		}
 	}
@@ -585,7 +590,7 @@ int send_prout_activepath(char * dev, int rq_servact, int rq_scope,
 	rc = pthread_create(&thread, &attr, mpath_prout_pthread_fn, (void *)(&param));
 	if (rc){
 		condlog (3, "%s: failed to create thread %d", dev, rc);
-		exit(-1);
+		return MPATH_PR_OTHER;
 	}
 	/* Free attribute and wait for the other threads */
 	pthread_attr_destroy(&attr);
@@ -723,16 +728,21 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 		condlog (3, "%s: reservation key set.", mpp->wwid);
 	}
 
-	mpath_prout_common (mpp, MPATH_PROUT_CLEAR_SA, rq_scope, rq_type, pamp,
-			noisy);
+	status = mpath_prout_common (mpp, MPATH_PROUT_CLEAR_SA,
+				     rq_scope, rq_type, pamp, noisy);
+
+	if (status) {
+		condlog(0, "%s: failed to send CLEAR_SA", mpp->wwid);
+		goto out1;
+	}
 
 	pamp->num_transportid = 1;
 	pptr=pamp->trnptid_list[0];
 
 	for (i = 0; i < num; i++){
-		if (mpp->reservation_key && 
+		if (mpp->reservation_key &&
 			memcmp(pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key,
-			mpp->reservation_key, 8)){	
+			       mpp->reservation_key, 8)){
 			/*register with tarnsport id*/
 			memset(pamp, 0, length);
 			pamp->trnptid_list[0] = pptr;
Signature · 1 line
1.8.4.5
#3 lore raw

PATCH 3/78 libmultipath: filter for missing property in get_refwwid()

In reply to Hannes Reinecke

get_refwwid() needs to filter for missing udev properties, otherwise it'll accept just about any SCSI device.

---
 libmultipath/configure.c | 6 ++++++
 1 file changed, 6 insertions(+)
libmultipath/configure.c +6 -0
index 1e6a0c5..9012258 100644
@@ -962,6 +962,9 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 				return ret;
 			}
 		}
+		if (pp->udev && filter_property(conf, pp->udev) > 0)
+			return 2;
+
 		refwwid = pp->wwid;
 		goto out;
 	}
@@ -986,6 +989,9 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 				return ret;
 			}
 		}
+		if (pp->udev && filter_property(conf, pp->udev) > 0)
+			return 2;
+
 		refwwid = pp->wwid;
 		goto out;
 	}
Signature · 1 line
1.8.4.5
#4 lore raw

PATCH 4/78 Double uevent stacksize yet again

In reply to Hannes Reinecke

Another report indicates that the uevent stacksize might still be too small.

---
 libmultipath/defaults.h | 1 +
 multipathd/main.c       | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
libmultipath/defaults.h +1 -0
index fd64a12..7831992 100644
@@ -19,6 +19,7 @@
 #define DEFAULT_DETECT_PRIO DETECT_PRIO_OFF
 #define DEFAULT_DEFERRED_REMOVE DEFERRED_REMOVE_OFF
 #define DEFAULT_DELAY_CHECKS DELAY_CHECKS_OFF
+#define DEFAULT_UEVENT_STACKSIZE 256
 
 #define DEFAULT_CHECKINT	5
 #define MAX_CHECKINT(a)		(a << 2)
multipathd/main.c +1 -1
index a160d75..9667ba1 100644
@@ -1710,7 +1710,7 @@ child (void * param)
 	udev = udev_new();
 
 	setup_thread_attr(&misc_attr, 64 * 1024, 1);
-	setup_thread_attr(&uevent_attr, 128 * 1024, 1);
+	setup_thread_attr(&uevent_attr, DEFAULT_UEVENT_STACKSIZE * 1024, 1);
 	setup_thread_attr(&waiter_attr, 32 * 1024, 1);
 
 	if (logsink == 1) {
Signature · 1 line
1.8.4.5
#5 lore raw

PATCH 5/78 discovery: do not fail discovery on individual devices

In reply to Hannes Reinecke

Any errors returned from individual devices during discovery do not invalidate the entire discovery. So instead of returning the pointless accumulated error count we should be returning the number of devices which which could not be discovered or the negative error number if the discovery itself failed.

---
 libmpathpersist/mpath_persist.c | 14 ++++++-------
 libmultipath/discovery.c        | 46 ++++++++++++++++++++---------------------
 libmultipath/discovery.h        |  4 ++++
 multipath/main.c                |  2 +-
 multipathd/main.c               |  6 ++++--
 5 files changed, 38 insertions(+), 34 deletions(-)
libmpathpersist/mpath_persist.c +7 -7
index 9f65ca6..d427c5b 100644
@@ -178,7 +178,7 @@ int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp,
 		goto out;
 	}
 
-	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER)) {
+	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER) < 0) {
 		ret = MPATH_PR_DMMP_ERROR;
 		goto out1;
 	}
@@ -262,13 +262,13 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 	curmp = vector_alloc ();
 	pathvec = vector_alloc ();
 
-	 if (!curmp || !pathvec){
-                condlog (0, "%s: vector allocation failed.", alias);
-                ret = MPATH_PR_DMMP_ERROR;
-                goto out;
-        }
+	if (!curmp || !pathvec){
+		condlog (0, "%s: vector allocation failed.", alias);
+		ret = MPATH_PR_DMMP_ERROR;
+		goto out;
+	}
 
-	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER)) {
+	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER) < 0) {
 		ret = MPATH_PR_DMMP_ERROR;
 		goto out1;
 	}
libmultipath/discovery.c +22 -24

133 lines not on this page · raw message

libmultipath/discovery.h +4 -0
index 6f78918..f14fcee 100644
@@ -24,6 +24,10 @@
 #define SCSI_COMMAND_TERMINATED 0x22
 #define SG_ERR_DRIVER_SENSE     0x08
 
+#define PATHINFO_OK 0
+#define PATHINFO_FAILED 1
+#define PATHINFO_SKIPPED 2
+
 struct config;
 
 ssize_t sysfs_get_dev (struct udev_device *udev, char * buff, size_t len);
multipath/main.c +1 -1
index d085f64..dadbb2a 100644
@@ -343,7 +343,7 @@ configure (void)
 		/* maximum info */
 		di_flag = DI_ALL;
 
-	if (path_discovery(pathvec, conf, di_flag))
+	if (path_discovery(pathvec, conf, di_flag) < 0)
 		goto out;
 
 	if (conf->verbosity > 2)
multipathd/main.c +4 -2
index 9667ba1..4f02211 100644
@@ -1409,7 +1409,7 @@ configure (struct vectors * vecs, int start_waiters)
 	struct multipath * mpp;
 	struct path * pp;
 	vector mpvec;
-	int i;
+	int i, ret;
 
 	if (!vecs->pathvec && !(vecs->pathvec = vector_alloc()))
 		return 1;
@@ -1423,7 +1423,9 @@ configure (struct vectors * vecs, int start_waiters)
 	/*
 	 * probe for current path (from sysfs) and map (from dm) sets
 	 */
-	path_discovery(vecs->pathvec, conf, DI_ALL);
+	ret = path_discovery(vecs->pathvec, conf, DI_ALL);
+	if (ret < 0)
+		return 1;
 
 	vector_foreach_slot (vecs->pathvec, pp, i){
 		if (filter_path(conf, pp) > 0){
Signature · 1 line
1.8.4.5
#6 lore raw

PATCH 6/78 libmultipath: Prefer deprecated 'getuid' callout

In reply to Hannes Reinecke

We need first to check if the deprecated 'getuid' callout should be used, otherwise it might never be called.

---
 libmultipath/propsel.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
libmultipath/propsel.c +3 -3
index 46f8f63..f64d5e4 100644
@@ -351,12 +351,12 @@ select_getuid (struct path * pp)
 {
 	char *origin;
 
-	pp_set_ovr(uid_attribute);
 	pp_set_ovr(getuid);
-	pp_set_hwe(uid_attribute);
+	pp_set_ovr(uid_attribute);
 	pp_set_hwe(getuid);
-	pp_set_conf(uid_attribute);
+	pp_set_hwe(uid_attribute);
 	pp_set_conf(getuid);
+	pp_set_conf(uid_attribute);
 	pp_set_default(uid_attribute, DEFAULT_UID_ATTRIBUTE);
 out:
 	if (pp->uid_attribute)
Signature · 1 line
1.8.4.5
#7 lore raw

PATCH 7/78 libmultipath: Skip paths with empty wwid

In reply to Hannes Reinecke

If the WWID is empty this is pretty much a terminal error, as we don't have anything to identify the multipath device. So skip those devices during discovery.

---
 libmultipath/discovery.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
libmultipath/discovery.c +4 -1
index 3e0afc4..948fa49 100644
@@ -1230,7 +1230,8 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	if ((mask & DI_WWID) && !strlen(pp->wwid))
 		get_uid(pp);
 	if (mask & DI_BLACKLIST && mask & DI_WWID) {
-		if (filter_wwid(conf->blist_wwid, conf->elist_wwid,
+		if (!strlen(pp->wwid) ||
+		    filter_wwid(conf->blist_wwid, conf->elist_wwid,
 				pp->wwid) > 0) {
 			return PATHINFO_SKIPPED;
 		}
@@ -1244,6 +1245,8 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 		if (pp->state != PATH_DOWN || pp->priority == PRIO_UNDEF) {
 			if (!strlen(pp->wwid))
 				get_uid(pp);
+			if (!strlen(pp->wwid))
+				return PATHINFO_SKIPPED;
 			get_prio(pp);
 		}
 	}
Signature · 1 line
1.8.4.5
#8 lore raw

PATCH 8/78 Make systemd installation path configurable

In reply to Hannes Reinecke

Older systems might have systemd installed in /lib, so make the path for systemd and udev configurable.

---
 Makefile.inc | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Makefile.inc +6 -2
index 20ae23e..c3ed73f 100644
@@ -27,10 +27,14 @@ ifndef SYSTEMD
 	endif
 endif
 
+ifndef SYSTEMDPATH
+	SYSTEMDPATH=usr/lib
+endif
+
 prefix      = 
 exec_prefix = $(prefix)
 bindir      = $(exec_prefix)/sbin
-libudevdir  = ${prefix}/lib/udev
+libudevdir  = $(prefix)/$(SYSTEMDPATH)/udev
 udevrulesdir = $(libudevdir)/rules.d
 multipathdir = $(TOPDIR)/libmultipath
 mandir      = $(prefix)/usr/share/man/man8
@@ -39,7 +43,7 @@ man3dir      = $(prefix)/usr/share/man/man3
 rcdir	    = $(prefix)/etc/init.d
 syslibdir   = $(prefix)/$(LIB)
 libdir	    = $(prefix)/$(LIB)/multipath
-unitdir     = $(prefix)/usr/lib/systemd/system
+unitdir     = $(prefix)/$(SYSTEMDPATH)/systemd/system
 mpathpersistdir = $(TOPDIR)/libmpathpersist
 
 GZIP        = gzip -9 -c
Signature · 1 line
1.8.4.5
#9 lore raw

PATCH 9/78 Add multipath rules for systemd support

In reply to Hannes Reinecke

Systemd and dracut rely on the DM_MULTIPATH_DEVICE_PATH setting to mark the devices as used by multipathing. So add a ruleset providing this setting.

---
 multipath/Makefile        |  2 ++
 multipath/multipath.rules | 13 +++++++++++++
 2 files changed, 15 insertions(+)
 create mode 100644 multipath/multipath.rules
multipath/Makefile +2 -0
index e295676..9f2e963 100644
@@ -23,6 +23,7 @@ install:
 	$(INSTALL_PROGRAM) -m 755 $(EXEC) $(DESTDIR)$(bindir)/
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(udevrulesdir)
 	$(INSTALL_PROGRAM) -m 644 11-dm-mpath.rules $(DESTDIR)$(udevrulesdir)
+	$(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir)
@@ -31,6 +32,7 @@ install:
 uninstall:
 	rm $(DESTDIR)$(bindir)/$(EXEC)
 	rm $(DESTDIR)$(udevrulesdir)/11-dm-mpath.rules
+	rm $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
 	rm $(DESTDIR)$(mandir)/$(EXEC).8.gz
 	rm $(DESTDIR)$(man5dir)/$(EXEC).conf.5.gz
 
multipath/multipath.rules new file +13 -0
new file mode 100644
index 0000000..799fbb0
@@ -0,0 +1,13 @@
+# Set DM_MULTIPATH_DEVICE_PATH if the device should be handled by multipath
+SUBSYSTEM!="block", GOTO="end_mpath"
+
+ENV{MPATH_SBIN_PATH}="/sbin"
+TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
+
+SUBSYSTEM=="block", ACTION=="add|change", KERNEL!="dm-*", \
+	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
+	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -v 0 -c $tempnode", \
+	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
+	ENV{SYSTEMD_READY}="0"
+
+LABEL="end_mpath"
Signature · 1 line
1.8.4.5
#10 lore raw

PATCH 10/78 Fixup multipathd.socket to resolve ordering dependeny

In reply to Hannes Reinecke

Per default any socket file is started after /run is available, as the socket might be placed there.
This results in a dependency cycle as multipathd requires the socket, but the daemon should be started as early as possible, and certainly before /run is mounted.

---
 multipathd/multipathd.service | 3 ++-
 multipathd/multipathd.socket  | 8 +++++---
 2 files changed, 7 insertions(+), 4 deletions(-)
multipathd/multipathd.service +2 -1
index be3ba3f..1ecc151 100644
@@ -1,8 +1,9 @@
 [Unit]
 Description=Device-Mapper Multipath Device Controller
 Before=iscsi.service iscsid.service lvm2-activation-early.service
-After=syslog.target
+Before=local-fs-pre.target
 DefaultDependencies=no
+Wants=multipathd.socket
 Conflicts=shutdown.target
 
 [Service]
multipathd/multipathd.socket +5 -3
index 3d4b6da..921706d 100644
@@ -1,5 +1,7 @@
+[Unit]
+Description=multipathd control socket
+DefaultDependencies=no
+Before=sockets.target
+
 [Socket]
 ListenStream=@/org/kernel/linux/storage/multipathd
-
-[Install]
-WantedBy=sockets.target
Signature · 1 line
1.8.4.5
#11 lore raw

PATCH 11/78 Fixup dependencies in multipathd.service

In reply to Hannes Reinecke

The multipathd.service should be started after the .socket, and it requires local-fs-pre to be reached. Otherwise it won't be able to access /sys etc.

References: bnc#872712

---
 multipathd/multipathd.service | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
multipathd/multipathd.service +2 -1
index 1ecc151..b5b755b 100644
@@ -2,8 +2,9 @@
 Description=Device-Mapper Multipath Device Controller
 Before=iscsi.service iscsid.service lvm2-activation-early.service
 Before=local-fs-pre.target
+After=multipathd.socket
 DefaultDependencies=no
-Wants=multipathd.socket
+Wants=local-fs-pre.target multipathd.socket
 Conflicts=shutdown.target
 
 [Service]
Signature · 1 line
1.8.4.5
#12 lore raw

PATCH 12/78 multipathd: set correct PID when running in debug mode

In reply to Hannes Reinecke

When running in debug mode the PID is not set, causing the PID in the pidfile to always be set to '0'.

---
 multipathd/main.c | 1 +
 1 file changed, 1 insertion(+)
multipathd/main.c +1 -0
index 4f02211..3e22ad1 100644
@@ -2056,6 +2056,7 @@ main (int argc, char *argv[])
 		if (!isatty(fileno(stdout)))
 			setbuf(stdout, NULL);
 		err = 0;
+		daemon_pid = getpid();
 	} else
 		err = daemonize();
 
Signature · 1 line
1.8.4.5
#13 lore raw

PATCH 13/78 Do not print empty device strings during discovery

In reply to Hannes Reinecke

If the path provides no information we should not print out empty strings, but rather an instructive message telling us so.

References: bnc#872712

---
 libmultipath/structs_vec.c | 6 ++++++
 1 file changed, 6 insertions(+)
libmultipath/structs_vec.c +6 -0
index a15e3c0..a1f2212 100644
@@ -226,6 +226,12 @@ extract_hwe_from_path(struct multipath * mpp)
 	}
 
 	if (pp) {
+		if (!strlen(pp->vendor_id) ||
+		    !strlen(pp->product_id) ||
+		    !strlen(pp->rev)) {
+			condlog(3, "%s: no device details available", pp->dev);
+			return NULL;
+		}
 		condlog(3, "%s: vendor = %s", pp->dev, pp->vendor_id);
 		condlog(3, "%s: product = %s", pp->dev, pp->product_id);
 		condlog(3, "%s: rev = %s", pp->dev, pp->rev);
Signature · 1 line
1.8.4.5
#14 lore raw

PATCH 14/78 kpartx.rules: do not call blkid

In reply to Hannes Reinecke

blkid is already called by the device-mapper rules, no need to do it ourselves.

---
 kpartx/kpartx.rules | 6 ------
 1 file changed, 6 deletions(-)
kpartx/kpartx.rules +0 -6
index 5c0d0ff..226e44a 100644
@@ -12,12 +12,6 @@ ENV{DM_DEPS}=="0", GOTO="kpartx_end"
 
 ENV{DM_UUID}=="?*", IMPORT{program}=="kpartx_id %M %m $env{DM_UUID}"
 
-ENV{DM_ACTION}=="PATH_FAILED", ENV{DM_NR_VALID_PATHS}=="0", \
-	GOTO="blkid_end"
-ENV{DM_UUID}=="mpath-*", IMPORT{program}="/sbin/blkid -o udev -p $tempnode"
-ENV{DM_PART}=="?*", IMPORT{program}="/sbin/blkid -o udev -p $tempnode"
-LABEL="blkid_end"
-
 OPTIONS="link_priority=50"
 
 # Create persistent links for multipath tables
Signature · 1 line
1.8.4.5
#15 lore raw

PATCH 15/78 Use 'SCSI_IDENT_.*' as the default property whitelist

In reply to Hannes Reinecke

59-scsi-sg_utils.rules export the VPD pages as SCSI_IDENT_<association>_<type>.
So whenever we have a SCSI_IDENT_* property we know it has come from VPD pages and we have a legit device.

---
 libmultipath/blacklist.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
libmultipath/blacklist.c +1 -1
index cea128c..30c5031 100644
@@ -196,7 +196,7 @@ setup_default_blist (struct config * conf)
 	if (store_ble(conf->blist_devnode, str, ORIGIN_DEFAULT))
 		return 1;
 
-	str = STRDUP("(ID_SCSI_VPD|ID_WWN)");
+	str = STRDUP("(SCSI_IDENT_.*|ID_WWN)");
 	if (!str)
 		return 1;
 	if (store_ble(conf->elist_property, str, ORIGIN_DEFAULT))
Signature · 1 line
1.8.4.5
#16 lore raw

PATCH 16/78 Fixup wwid blacklist printing

In reply to Hannes Reinecke

When a wwid is blacklisted the log message looks like '(null): wwid XXX blacklisted'.
Fix it up by printing either the corresponding device or avoid printing the '(null)' entry if no corresponding device is given.

---
 libmultipath/blacklist.c  | 6 ++++--
 libmultipath/blacklist.h  | 2 +-
 libmultipath/configure.c  | 2 +-
 libmultipath/discovery.c  | 2 +-
 multipathd/cli_handlers.c | 2 +-
 5 files changed, 8 insertions(+), 6 deletions(-)
libmultipath/blacklist.c +4 -2
index 30c5031..2400eda 100644
@@ -228,6 +228,8 @@ setup_default_blist (struct config * conf)
 	if (vendor && product)						\
 		condlog(3, "%s: (%s:%s) %s %s",				\
 			dev, vendor, product, (M), (S));		\
+	else if (wwid && !dev)						\
+		condlog(3, "%s: %s %s", wwid, (M), (S));		\
 	else if (wwid)							\
 		condlog(3, "%s: %s %s %s", dev, (M), wwid, (S));	\
 	else if (env)							\
@@ -328,10 +330,10 @@ _filter_wwid (vector blist, vector elist, char * wwid)
 }
 
 int
-filter_wwid (vector blist, vector elist, char * wwid)
+filter_wwid (vector blist, vector elist, char * wwid, char * dev)
 {
 	int r = _filter_wwid(blist, elist, wwid);
-	log_filter(NULL, NULL, NULL, wwid, NULL, r);
+	log_filter(dev, NULL, NULL, wwid, NULL, r);
 	return r;
 }
 
libmultipath/blacklist.h +1 -1
index 0e90e9a..24a5fa5 100644
@@ -32,7 +32,7 @@ struct blentry_device {
 int setup_default_blist (struct config *);
 int alloc_ble_device (vector);
 int filter_devnode (vector, vector, char *);
-int filter_wwid (vector, vector, char *);
+int filter_wwid (vector, vector, char *, char *);
 int filter_device (vector, vector, char *, char *);
 int filter_path (struct config *, struct path *);
 int filter_property(struct config *, struct udev_device *);
libmultipath/configure.c +1 -1
index 9012258..c9f16c7 100644
@@ -1025,7 +1025,7 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 check:
 		if (refwwid && strlen(refwwid)) {
 			if (filter_wwid(conf->blist_wwid, conf->elist_wwid,
-					refwwid) > 0)
+					refwwid, NULL) > 0)
 			return 2;
 		}
 	}
libmultipath/discovery.c +1 -1
index 948fa49..6d6a508 100644
@@ -1232,7 +1232,7 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	if (mask & DI_BLACKLIST && mask & DI_WWID) {
 		if (!strlen(pp->wwid) ||
 		    filter_wwid(conf->blist_wwid, conf->elist_wwid,
-				pp->wwid) > 0) {
+				pp->wwid, pp->dev) > 0) {
 			return PATHINFO_SKIPPED;
 		}
 	}
multipathd/cli_handlers.c +1 -1
index 0ce1408..43372d3 100644
@@ -494,7 +494,7 @@ cli_add_map (void * v, char ** reply, int * len, void * data)
 	param = convert_dev(param, 0);
 	condlog(2, "%s: add map (operator)", param);
 
-	if (filter_wwid(conf->blist_wwid, conf->elist_wwid, param) > 0) {
+	if (filter_wwid(conf->blist_wwid, conf->elist_wwid, param, NULL) > 0) {
 		*reply = strdup("blacklisted\n");
 		*len = strlen(*reply) + 1;
 		condlog(2, "%s: map blacklisted", param);
Signature · 1 line
1.8.4.5
#17 lore raw

PATCH 17/78 Allow for empty path argument when printing information

In reply to Hannes Reinecke

When calling the various print functions we should be allowing for empty path argument.

---
 libmultipath/print.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)
libmultipath/print.c +16 -6
index 130a9af..39b4f98 100644
@@ -287,7 +287,7 @@ snprint_path_uuid (char * buff, size_t len, struct path * pp)
 static int
 snprint_hcil (char * buff, size_t len, struct path * pp)
 {
-	if (pp->sg_id.host_no < 0)
+	if (!pp || pp->sg_id.host_no < 0)
 		return snprintf(buff, len, "#:#:#:#");
 
 	return snprintf(buff, len, "%i:%i:%i:%i",
@@ -300,7 +300,7 @@ snprint_hcil (char * buff, size_t len, struct path * pp)
 static int
 snprint_dev (char * buff, size_t len, struct path * pp)
 {
-	if (!strlen(pp->dev))
+	if (!pp || !strlen(pp->dev))
 		return snprintf(buff, len, "-");
 	else
 		return snprint_str(buff, len, pp->dev);
@@ -309,7 +309,7 @@ snprint_dev (char * buff, size_t len, struct path * pp)
 static int
 snprint_dev_t (char * buff, size_t len, struct path * pp)
 {
-	if (!strlen(pp->dev))
+	if (!pp || !strlen(pp->dev))
 		return snprintf(buff, len, "#:#");
 	else
 		return snprint_str(buff, len, pp->dev_t);
@@ -318,8 +318,12 @@ snprint_dev_t (char * buff, size_t len, struct path * pp)
 static int
 snprint_offline (char * buff, size_t len, struct path * pp)
 {
-	if (pp->offline)
+	if (!pp)
+		return snprintf(buff, len, "unknown");
+	else if (pp->offline)
 		return snprintf(buff, len, "offline");
+	else if (!pp->mpp)
+		return snprintf(buff, len, "orphan");
 	else
 		return snprintf(buff, len, "running");
 }
@@ -327,6 +331,9 @@ snprint_offline (char * buff, size_t len, struct path * pp)
 static int
 snprint_chk_state (char * buff, size_t len, struct path * pp)
 {
+	if (!pp)
+		return snprintf(buff, len, "undef");
+
 	switch (pp->state) {
 	case PATH_UP:
 		return snprintf(buff, len, "ready");
@@ -350,6 +357,9 @@ snprint_chk_state (char * buff, size_t len, struct path * pp)
 static int
 snprint_dm_path_state (char * buff, size_t len, struct path * pp)
 {
+	if (!pp)
+		return snprintf(buff, len, "undef");
+
 	switch (pp->dmstate) {
 	case PSTATE_ACTIVE:
 		return snprintf(buff, len, "active");
@@ -370,7 +380,7 @@ snprint_vpr (char * buff, size_t len, struct path * pp)
 static int
 snprint_next_check (char * buff, size_t len, struct path * pp)
 {
-	if (!pp->mpp)
+	if (!pp || !pp->mpp)
 		return snprintf(buff, len, "orphan");
 
 	return snprint_progress(buff, len, pp->tick, pp->checkint);
@@ -379,7 +389,7 @@ snprint_next_check (char * buff, size_t len, struct path * pp)
 static int
 snprint_pri (char * buff, size_t len, struct path * pp)
 {
-	return snprint_int(buff, len, pp->priority);
+	return snprint_int(buff, len, pp ? pp->priority : -1);
 }
 
 static int
Signature · 1 line
1.8.4.5
#18 lore raw

PATCH 18/78 Disable reassign maps per default

In reply to Hannes Reinecke

lvm has undergone various updates, and should now check for correct device assignment. So disable the 'reassign_maps' feature per default.

---
 libmultipath/defaults.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
libmultipath/defaults.h +1 -1
index 7831992..878da14 100644
@@ -12,7 +12,7 @@
 #define DEFAULT_RR_WEIGHT      RR_WEIGHT_NONE
 #define DEFAULT_NO_PATH_RETRY  NO_PATH_RETRY_UNDEF
 #define DEFAULT_VERBOSITY	2
-#define DEFAULT_REASSIGN_MAPS	1
+#define DEFAULT_REASSIGN_MAPS	0
 #define DEFAULT_FIND_MULTIPATHS	0
 #define DEFAULT_FAST_IO_FAIL	5
 #define DEFAULT_RETAIN_HWHANDLER RETAIN_HWHANDLER_OFF
Signature · 1 line
1.8.4.5
#19 lore raw

PATCH 19/78 multipathd: implement 'list path <path>' cli command

In reply to Hannes Reinecke

Implement CLI command for printing the status of a single path.

---
 multipathd/cli.c          |  1 +
 multipathd/cli_handlers.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 multipathd/cli_handlers.h |  1 +
 multipathd/main.c         |  1 +
 4 files changed, 47 insertions(+)
multipathd/cli.c +1 -0
index 2a5edfa..acc4249 100644
@@ -425,6 +425,7 @@ cli_init (void) {
 
 	add_handler(LIST+PATHS, NULL);
 	add_handler(LIST+PATHS+FMT, NULL);
+	add_handler(LIST+PATH, NULL);
 	add_handler(LIST+STATUS, NULL);
 	add_handler(LIST+DAEMON, NULL);
 	add_handler(LIST+MAPS, NULL);
multipathd/cli_handlers.c +44 -0

56 lines not on this page · raw message

multipathd/cli_handlers.h +1 -0
index de51961..c4636d2 100644
@@ -1,5 +1,6 @@
 int cli_list_paths (void * v, char ** reply, int * len, void * data);
 int cli_list_paths_fmt (void * v, char ** reply, int * len, void * data);
+int cli_list_path (void * v, char ** reply, int * len, void * data);
 int cli_list_status (void * v, char ** reply, int * len, void * data);
 int cli_list_daemon (void * v, char ** reply, int * len, void * data);
 int cli_list_maps (void * v, char ** reply, int * len, void * data);
multipathd/main.c +1 -0
index 3e22ad1..2107113 100644
@@ -886,6 +886,7 @@ uxlsnrloop (void * ap)
 
 	set_handler_callback(LIST+PATHS, cli_list_paths);
 	set_handler_callback(LIST+PATHS+FMT, cli_list_paths_fmt);
+	set_handler_callback(LIST+PATH, cli_list_path);
 	set_handler_callback(LIST+MAPS, cli_list_maps);
 	set_handler_callback(LIST+STATUS, cli_list_status);
 	set_handler_callback(LIST+DAEMON, cli_list_daemon);
Signature · 1 line
1.8.4.5
#20 lore raw

PATCH 20/78 Make checker_put() and prio_put() idempotent

In reply to Hannes Reinecke

checker_put() and prio_put() should check whether an instance was selected prior to dropping a reference to it. Otherwise they'll be dropping a reference which they haven't increased in the first place.

---
 libmultipath/checkers.c | 2 +-
 libmultipath/prio.c     | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
libmultipath/checkers.c +1 -1
index 1dd5525..ef1d099 100644
@@ -195,7 +195,7 @@ void checker_put (struct checker * dst)
 {
 	struct checker * src;
 
-	if (!dst)
+	if (!dst || !dst->check)
 		return;
 	src = checker_lookup(dst->name);
 	if (dst->free)
libmultipath/prio.c +2 -2
index 6ee0b9c..ab8eca9 100644
@@ -132,7 +132,7 @@ int prio_getprio (struct prio * p, struct path * pp)
 
 int prio_selected (struct prio * p)
 {
-	if (!p || !p->getprio)
+	if (!p)
 		return 0;
 	return (p->getprio) ? 1 : 0;
 }
@@ -169,7 +169,7 @@ void prio_put (struct prio * dst)
 {
 	struct prio * src;
 
-	if (!dst)
+	if (!dst || !dst->getprio)
 		return;
 
 	src = prio_lookup(dst->name);
Signature · 1 line
1.8.4.5
#21 lore raw

PATCH 21/78 Remove trailing linefeed from sysfs attributes

In reply to Hannes Reinecke

When reading from sysfs attributes might have a trailing linefeed, which will then show up in the log messages as an additional linefeed.

---
 libmultipath/sysfs.c | 2 ++
 1 file changed, 2 insertions(+)
libmultipath/sysfs.c +2 -0
index 0e05316..102135a 100644
@@ -93,6 +93,8 @@ ssize_t sysfs_attr_get_value(struct udev_device *dev, const char *attr_name,
 	}
 
 	close(fd);
+	if (size > 0)
+		size = strchop(value);
 	return size;
 }
 
Signature · 1 line
1.8.4.5
#22 lore raw

PATCH 22/78 multipath: implement option '-u' for uevents

In reply to Hannes Reinecke

When calling 'multipath' from an uevent udev will not have all information for that device, as it's being written into the database _after_ the event has been processed. This patch implements an option '-u' which uses the information from the program environment when checking the device.

---
 libmultipath/config.c      |  1 +
 libmultipath/config.h      |  3 ++-
 libmultipath/configure.c   | 28 ++++++++++++++++++++++++++--
 libmultipath/log_pthread.c |  2 +-
 multipath/main.c           | 21 ++++++++++++++++++---
 multipath/multipath.8      |  6 +++++-
 multipath/multipath.rules  |  2 +-
 7 files changed, 54 insertions(+), 9 deletions(-)
libmultipath/config.c +1 -0
index 79c6bc1..1007f32 100644
@@ -617,6 +617,7 @@ load_config (char * file, struct udev *udev)
 	conf->partition_delim = NULL;
 	conf->processed_main_config = 0;
 	conf->find_multipaths = DEFAULT_FIND_MULTIPATHS;
+	conf->uid_attribute = set_default(DEFAULT_UID_ATTRIBUTE);
 
 	/*
 	 * preload default hwtable
libmultipath/config.h +2 -1
index a680e2b..d304a6c 100644
@@ -19,7 +19,8 @@ enum devtypes {
 	DEV_NONE,
 	DEV_DEVT,
 	DEV_DEVNODE,
-	DEV_DEVMAP
+	DEV_DEVMAP,
+	DEV_UEVENT
 };
 
 enum mpath_cmds {
libmultipath/configure.c +26 -2
index c9f16c7..a30ca59 100644
@@ -957,8 +957,8 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 			udev_device_unref(udevice);
 			if (!pp) {
 				if (ret == 1)
-					condlog(0, "%s can't store path info",
-						buff);
+					condlog(0, "%s: can't store path info",
+						dev);
 				return ret;
 			}
 		}
@@ -995,6 +995,30 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 		refwwid = pp->wwid;
 		goto out;
 	}
+
+	if (dev_type == DEV_UEVENT) {
+		struct udev_device *udevice = udev_device_new_from_environment(conf->udev);
+
+		if (!udevice) {
+			condlog(2, "%s: can't get udev device", dev);
+			return 1;
+		}
+		ret = store_pathinfo(pathvec, conf->hwtable, udevice,
+				     DI_SYSFS | DI_WWID, &pp);
+		udev_device_unref(udevice);
+		if (!pp) {
+			if (ret == 1)
+				condlog(0, "%s: can't store path info",
+					dev);
+			return ret;
+		}
+		if (pp->udev && filter_property(conf, pp->udev) > 0)
+			return 2;
+
+		refwwid = pp->wwid;
+		goto out;
+	}
+
 	if (dev_type == DEV_DEVMAP) {
 
 		if (((dm_get_uuid(dev, tmpwwid)) == 0) && (strlen(tmpwwid))) {
libmultipath/log_pthread.c +1 -1
index 47d75a1..e6f4b5c 100644
@@ -25,7 +25,7 @@ int logq_running;
 void log_safe (int prio, const char * fmt, va_list ap)
 {
 	if (log_thr == (pthread_t)0) {
-		syslog(prio, fmt, ap);
+		vsyslog(prio, fmt, ap);
 		return;
 	}
 
multipath/main.c +18 -3
index dadbb2a..08ba66c 100644
@@ -28,6 +28,7 @@
 #include <unistd.h>
 #include <ctype.h>
 #include <libudev.h>
+#include <syslog.h>
 
 #include <checkers.h>
 #include <prio.h>
@@ -283,6 +284,7 @@ configure (void)
 		int failed = get_refwwid(conf->dev, conf->dev_type, pathvec,
 					 &refwwid);
 		if (!refwwid) {
+			condlog(3, "%s: failed to get wwid", conf->dev);
 			if (failed == 2 && conf->cmd == CMD_VALID_PATH)
 				printf("%s is not a valid multipath device path\n", conf->dev);
 			else
@@ -471,11 +473,11 @@ main (int argc, char *argv[])
 	int r = 1;
 
 	udev = udev_new();
-
+	logsink = 0;
 	if (load_config(DEFAULT_CONFIGFILE, udev))
 		exit(1);
 
-	while ((arg = getopt(argc, argv, ":adchl::FfM:v:p:b:BritqwW")) != EOF ) {
+	while ((arg = getopt(argc, argv, ":adchl::FfM:v:p:b:BritquwW")) != EOF ) {
 		switch(arg) {
 		case 1: printf("optarg : %s\n",optarg);
 			break;
@@ -542,6 +544,10 @@ main (int argc, char *argv[])
 		case 'h':
 			usage(argv[0]);
 			exit(0);
+		case 'u':
+			conf->cmd = CMD_VALID_PATH;
+			conf->dev_type = DEV_UEVENT;
+			break;
 		case 'w':
 			conf->cmd = CMD_REMOVE_WWID;
 			break;
@@ -581,9 +587,15 @@ main (int argc, char *argv[])
 			goto out;
 
 		strncpy(conf->dev, argv[optind], FILE_NAME_SIZE);
-		conf->dev_type = get_dev_type(conf->dev);
+		if (conf->dev_type != DEV_UEVENT)
+			conf->dev_type = get_dev_type(conf->dev);
 	}
 	conf->daemon = 0;
+	if (conf->dev_type == DEV_UEVENT) {
+		openlog("multipath", 0, LOG_DAEMON);
+		setlogmask(LOG_UPTO(conf->verbosity + 3));
+		logsink = 1;
+	}
 
 	if (conf->max_fds) {
 		struct rlimit fd_limit;
@@ -659,6 +671,9 @@ out:
 	cleanup_prio();
 	cleanup_checkers();
 
+	if (conf->dev_type == DEV_UEVENT)
+		closelog();
+
 out_free_config:
 	/*
 	 * Freeing config must be done after dm_lib_exit(), because
multipath/multipath.8 +5 -1
index 13e2e89..966139e 100644
@@ -8,7 +8,7 @@ multipath \- Device mapper target autoconfig
 .RB [\| \-b\ \c
 .IR bindings_file \|]
 .RB [\| \-d \|]
-.RB [\| \-h | \-l | \-ll | \-f | \-t | \-F | \-B | \-c | \-q | \|-r | \|-i | \-a | \-w | \-W \|]
+.RB [\| \-h | \-l | \-ll | \-f | \-t | \-F | \-B | \-c | \-q | \|-r | \|-i | \-a | \|-u | \-w | \-W \|]
 .RB [\| \-p\ \c
 .BR failover | multibus | group_by_serial | group_by_prio | group_by_node_name \|]
 .RB [\| device \|]
@@ -74,6 +74,10 @@ allow device tables with queue_if_no_path when multipathd is not running
 .B \-a
 add the wwid for the specified device to the wwids file
 .TP
+.B \-u
+check if the device specified in the program environment should be
+a path in a multipath device.
+.TP
 .B \-w
 remove the wwid for the specified device from the wwids file
 .TP
multipath/multipath.rules +1 -1
index 799fbb0..5bc5068 100644
@@ -6,7 +6,7 @@ TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
 
 SUBSYSTEM=="block", ACTION=="add|change", KERNEL!="dm-*", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
-	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -v 0 -c $tempnode", \
+	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -u %k", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
 	ENV{SYSTEMD_READY}="0"
 
Signature · 1 line
1.8.4.5
#23 lore raw

PATCH 23/78 Install multipath rule under '56-multipath.rules'

In reply to Hannes Reinecke

The multipath rule needs to be inserted between the sg3_utils rules for it to be able to intercept symlink generation.

References: bnc#873151

---
 multipath/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
multipath/Makefile +2 -2
index 9f2e963..7f18e9a 100644
@@ -23,7 +23,7 @@ install:
 	$(INSTALL_PROGRAM) -m 755 $(EXEC) $(DESTDIR)$(bindir)/
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(udevrulesdir)
 	$(INSTALL_PROGRAM) -m 644 11-dm-mpath.rules $(DESTDIR)$(udevrulesdir)
-	$(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
+	$(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir)
@@ -32,7 +32,7 @@ install:
 uninstall:
 	rm $(DESTDIR)$(bindir)/$(EXEC)
 	rm $(DESTDIR)$(udevrulesdir)/11-dm-mpath.rules
-	rm $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
+	rm $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules
 	rm $(DESTDIR)$(mandir)/$(EXEC).8.gz
 	rm $(DESTDIR)$(man5dir)/$(EXEC).conf.5.gz
 
Signature · 1 line
1.8.4.5
#24 lore raw

PATCH 24/78 multipath.rules: Whitelist devices

In reply to Hannes Reinecke

Currently multipath runs only on SCSI and DASD devices (and cciss, but they are no longer supported). So we can as well whitelist them and avoid the curious warning about 'multipath -u /dev/loop failed'.

---
 multipath/multipath.rules | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
multipath/multipath.rules +1 -1
index 5bc5068..56056bf 100644
@@ -4,7 +4,7 @@ SUBSYSTEM!="block", GOTO="end_mpath"
 ENV{MPATH_SBIN_PATH}="/sbin"
 TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
 
-SUBSYSTEM=="block", ACTION=="add|change", KERNEL!="dm-*", \
+SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd*[!0-9]|dasd*[!0-9]", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
 	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -u %k", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
Signature · 1 line
1.8.4.5
#25 lore raw

PATCH 25/78 multipath.rules: fixup race condition with systemd

In reply to Hannes Reinecke

systemd will call 'fsck' on any device, even if no device type is found. This might cause a race between the mount call and any fsck call on the device-mapper device.
So for multipath we should arbitrary set the device type to 'none', as then systemd with detect a missing 'fsck.none' command and skip fsck here.

---
 multipath/multipath.rules | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
multipath/multipath.rules +2 -1
index 56056bf..c76e6b8 100644
@@ -4,10 +4,11 @@ SUBSYSTEM!="block", GOTO="end_mpath"
 ENV{MPATH_SBIN_PATH}="/sbin"
 TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
 
-SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd*[!0-9]|dasd*[!0-9]", \
+SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd*|dasd*", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
 	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -u %k", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
+	ENV{ID_FS_TYPE}="none" \
 	ENV{SYSTEMD_READY}="0"
 
 LABEL="end_mpath"
Signature · 1 line
1.8.4.5
#26 lore raw

PATCH 26/78 11-dm-mpath.rules: Import blkid values if all paths are down

In reply to Hannes Reinecke

When all paths are failed we should not try to run any programs requiring disk access. However, we still need to create the symlinks so as not to confuse systemd.
So import the blkid values from the database in these cases.

---
 multipath/11-dm-mpath.rules | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
multipath/11-dm-mpath.rules +7 -1
index 69c24b5..2e7076d 100644
@@ -6,7 +6,13 @@ ENV{DM_UUID}!="mpath-?*", GOTO="mpath_end"
 # otherwise there would be a hang or IO error on access.
 # We'd like to avoid this, especially within udev processing.
 ENV{DM_NR_VALID_PATHS}!="?*", IMPORT{db}="DM_NR_VALID_PATHS"
-ENV{DM_NR_VALID_PATHS}=="0", ENV{DM_NOSCAN}="1"
+ENV{DM_NR_VALID_PATHS}!="0", GOTO="mpath_blkid_end"
+IMPORT{db}="ID_FS_TYPE"
+IMPORT{db}="ID_FS_USAGE"
+IMPORT{db}="ID_FS_UUID"
+IMPORT{db}="ID_FS_UUID_ENC"
+IMPORT{db}="ID_FS_VERSION"
+LABEL="mpath_blkid_end"
 
 # Also skip all foreign rules if no path is available.
 # Remember the original value of DM_DISABLE_OTHER_RULES_FLAG
Signature · 1 line
1.8.4.5
#27 lore raw

Re: [PATCH 26/78] 11-dm-mpath.rules: Import blkid values if all paths are down

Quoted 4 lines · Hannes Reinecke

When all paths are failed we should not try to run any programs requiring disk access. However, we still need to create the symlinks so as not to confuse systemd.
So import the blkid values from the database in these cases.

Don't we still need to set DM_NOSCAN here?

-Ben

Quoted 27 lines
---
 multipath/11-dm-mpath.rules | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
multipath/11-dm-mpath.rules +7 -1
index 69c24b5..2e7076d 100644
@@ -6,7 +6,13 @@ ENV{DM_UUID}!="mpath-?*", GOTO="mpath_end"
 # otherwise there would be a hang or IO error on access.
 # We'd like to avoid this, especially within udev processing.
 ENV{DM_NR_VALID_PATHS}!="?*", IMPORT{db}="DM_NR_VALID_PATHS"
-ENV{DM_NR_VALID_PATHS}=="0", ENV{DM_NOSCAN}="1"
+ENV{DM_NR_VALID_PATHS}!="0", GOTO="mpath_blkid_end"
+IMPORT{db}="ID_FS_TYPE"
+IMPORT{db}="ID_FS_USAGE"
+IMPORT{db}="ID_FS_UUID"
+IMPORT{db}="ID_FS_UUID_ENC"
+IMPORT{db}="ID_FS_VERSION"
+LABEL="mpath_blkid_end"
 
 # Also skip all foreign rules if no path is available.
 # Remember the original value of DM_DISABLE_OTHER_RULES_FLAG
Signature · 1 line
1.8.4.5
#28 lore raw

Re: [PATCH 26/78] 11-dm-mpath.rules: Import blkid values if all paths are down

Quoted 8 lines · Benjamin Marzinski
Quoted 4 lines · Hannes Reinecke

When all paths are failed we should not try to run any programs requiring disk access. However, we still need to create the symlinks so as not to confuse systemd.
So import the blkid values from the database in these cases.

Don't we still need to set DM_NOSCAN here?

I have no idea at all. I just found the setting somewhere and thought it'd be a good idea.
But don't ask me what's for; device-mapper/LVM udev variables are an arcane subject which I've yet to grasp.

Incidentally, a similar fix is needed for 10-dm.rules. But I'll prepare a separate patch for this.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#29 lore raw

Re: [PATCH 26/78] 11-dm-mpath.rules: Import blkid values if all paths are down

Quoted 7 lines · Hannes Reinecke
Quoted 6 lines · Benjamin Marzinski
Quoted 4 lines · Hannes Reinecke

When all paths are failed we should not try to run any programs requiring disk access. However, we still need to create the symlinks so as not to confuse systemd.
So import the blkid values from the database in these cases.

Quoted 6 lines
Quoted 2 lines

Don't we still need to set DM_NOSCAN here?

I have no idea at all. I just found the setting somewhere and thought it'd be a good idea.
But don't ask me what's for; device-mapper/LVM udev variables are an arcane subject which I've yet to grasp.

But what I mean is that your patch removes DM_NOSCAN

-ENV{DM_NR_VALID_PATHS}=="0", ENV{DM_NOSCAN}="1"

I assume that this is an oversight, since as you mentioned, we don't want to be running blkid on a device with no valid paths.

-Ben

Quoted 13 lines

Incidentally, a similar fix is needed for 10-dm.rules. But I'll prepare a separate patch for this.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#30 lore raw

PATCH 27/78 kpartx.rules: Skip kpartx for multipath events

In reply to Hannes Reinecke

Whenever multipath signals a PATH_FAILED or PATH_REINSTATED event we don't have to call 'kpartx'; the contents on the disk haven't changed.
And we should never call kpartx if all paths are down, irrespective of the uevent.

---
 kpartx/kpartx.rules | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kpartx/kpartx.rules +2 -2
index 226e44a..022361f 100644
@@ -40,8 +40,8 @@ ENV{ID_FS_USAGE}=="filesystem|other", ENV{ID_FS_LABEL_ENC}=="?*", \
 	SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"
 
 # Create dm tables for partitions
-ENV{DM_ACTION}=="PATH_FAILED", ENV{DM_NR_VALID_PATHS}=="0", \
-	GOTO="kpartx_end"
+ENV{DM_ACTION}=="PATH_FAILED|PATH_REINSTATED", GOTO="kpartx_end"
+ENV{DM_NR_VALID_PATHS}=="0", GOTO="kpartx_end"
 ENV{DM_STATE}!="SUSPENDED", ENV{DM_UUID}=="mpath-*", \
         RUN+="/sbin/kpartx -u -p -part /dev/$name"
 
Signature · 1 line
1.8.4.5
#31 lore raw

PATCH 28/78 multipathd: handle DOMAP_RETRY

In reply to Hannes Reinecke

Calling domap() will first check if all paths are available by using 'flock' on each path. This might fail if other processes are holding the lock already. In these cases we should retry domap() as the processes should terminate eventually.

---
 multipathd/main.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
multipathd/main.c +10 -1
index 2107113..494f4bb 100644
@@ -467,6 +467,7 @@ ev_add_path (struct path * pp, struct vectors * vecs)
 	char params[PARAMS_SIZE] = {0};
 	int retries = 3;
 	int start_waiter = 0;
+	int ret;
 
 	/*
 	 * need path UID to go any further
@@ -546,7 +547,15 @@ rescan:
 	/*
 	 * reload the map for the multipath mapped device
 	 */
-	if (domap(mpp, params) <= 0) {
+retry:
+	ret = domap(mpp, params);
+	if (ret <= 0) {
+		if (ret < 0 && retries-- > 0) {
+			condlog(0, "%s: retry domap for addition of new "
+				"path %s", mpp->alias, pp->dev);
+			sleep(1);
+			goto retry;
+		}
 		condlog(0, "%s: failed in domap for addition of new "
 			"path %s", mpp->alias, pp->dev);
 		/*
Signature · 1 line
1.8.4.5
#32 lore raw

PATCH 29/78 multipathd: cleanup foreground operation

In reply to Hannes Reinecke

When specifying '-d' it actually does two things, namely do not daemonize the program and prints logging messages to stdout instead of syslog.
This partially clashes with '-s' which suppresses the timestamp printing. So separate both use cases and ensure that '-d' doesn't overwrite '-s' if specified later.

---
 multipathd/main.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
multipathd/main.c +5 -2
index 494f4bb..ac1e1a2 100644
@@ -1998,6 +1998,7 @@ main (int argc, char *argv[])
 	extern int optind;
 	int arg;
 	int err;
+	int foreground = 0;
 
 	logsink = 1;
 	running_state = DAEMON_INIT;
@@ -2022,7 +2023,9 @@ main (int argc, char *argv[])
 	while ((arg = getopt(argc, argv, ":dsv:k::B")) != EOF ) {
 	switch(arg) {
 		case 'd':
-			logsink = 0;
+			foreground = 1;
+			if (logsink > 0)
+				logsink = 0;
 			//debug=1; /* ### comment me out ### */
 			break;
 		case 'v':
@@ -2062,7 +2065,7 @@ main (int argc, char *argv[])
 		exit(0);
 	}
 
-	if (logsink < 1) {
+	if (foreground) {
 		if (!isatty(fileno(stdout)))
 			setbuf(stdout, NULL);
 		err = 0;
Signature · 1 line
1.8.4.5
#33 lore raw

PATCH 30/78 Update hwtable for EMC XtremIO

In reply to Hannes Reinecke
---
 libmultipath/hwtable.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
libmultipath/hwtable.c +13 -0
index ead4d48..13611fe 100644
@@ -289,6 +289,19 @@ static struct hwentry default_hw[] = {
 		.prio_name     = DEFAULT_PRIO,
 		.prio_args     = NULL,
 	},
+	{
+		.vendor        = "XtremIO",
+		.product       = "XtremApp",
+		.features      = DEFAULT_FEATURES,
+		.hwhandler     = DEFAULT_HWHANDLER,
+		.pgpolicy      = MULTIBUS,
+		.pgfailback    = FAILBACK_UNDEF,
+		.rr_weight     = RR_WEIGHT_NONE,
+		.no_path_retry = 5,
+		.checker_name  = TUR,
+		.prio_name     = DEFAULT_PRIO,
+		.prio_args     = NULL,
+	},
 	/*
 	 * Fujitsu controller family
 	 *
Signature · 1 line
1.8.4.5
#34 lore raw

PATCH 31/78 multipath: check for running daemon when called with '-u'

In reply to Hannes Reinecke

When multipath is called to check for a valid device it really should check if the daemon is running. Otherwise it'll blindly assume the device to be eligible for multipathing even though the daemon isn't running.

---
 multipath/main.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
multipath/main.c +13 -0
index 08ba66c..ce8fb5b 100644
@@ -56,6 +56,7 @@
 #include <sys/time.h>
 #include <sys/resource.h>
 #include <wwids.h>
+#include <uxsock.h>
 #include "dev_t.h"
 
 int logsink;
@@ -622,6 +623,18 @@ main (int argc, char *argv[])
 		condlog(0, "the -c option requires a path to check");
 		goto out;
 	}
+	if (conf->cmd == CMD_VALID_PATH &&
+	    conf->dev_type == DEV_UEVENT) {
+		int fd;
+
+		fd = ux_socket_connect(DEFAULT_SOCKET);
+		if (fd == -1) {
+			printf("%s is not a valid multipath device path\n",
+				conf->dev);
+			goto out;
+		}
+		close(fd);
+	}
 	if (conf->cmd == CMD_REMOVE_WWID && !conf->dev) {
 		condlog(0, "the -w option requires a device");
 		goto out;
Signature · 1 line
1.8.4.5
#35 lore raw

PATCH 32/78 Revert 'return PATH_DOWN for quiesced paths'

In reply to Hannes Reinecke

Has been superseded by commit 00a2549a42707e5e1ef51ef6f638dc5ceaa15740

---
 libmultipath/discovery.c | 1 -
 1 file changed, 1 deletion(-)
libmultipath/discovery.c +0 -1
index 6d6a508..cf37813 100644
@@ -958,7 +958,6 @@ path_offline (struct path * pp)
 	condlog(3, "%s: path state = %s", pp->dev, buff);
 
 	if (!strncmp(buff, "offline", 7) ||
-	    !strncmp(buff, "quiesce", 7) ||
 	    !strncmp(buff, "transport-offline", 17)) {
 		pp->offline = 1;
 		return PATH_DOWN;
Signature · 1 line
1.8.4.5
#36 lore raw

PATCH 33/78 Do not treat 'transport-offline' paths as 'offline'

In reply to Hannes Reinecke

'transport-offline' status is entered whenever the transport has detected an offline device, but the SCSI midlayer still has references to it.
Once the last reference drops the SCSI midlayer will properly deregister the device and multipath will be updating the status. But until then we should treat this device as simply 'path down', and not marking it offline.
After all, some transports like FC will be entering the 'transport-offline' state after fast_io_fail triggered, and might revert back to 'running' after reconfiguration.

References: bnc#888378

---
 libmultipath/discovery.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
libmultipath/discovery.c +1 -2
index cf37813..899cbd5 100644
@@ -957,8 +957,7 @@ path_offline (struct path * pp)
 
 	condlog(3, "%s: path state = %s", pp->dev, buff);
 
-	if (!strncmp(buff, "offline", 7) ||
-	    !strncmp(buff, "transport-offline", 17)) {
+	if (!strncmp(buff, "offline", 7)) {
 		pp->offline = 1;
 		return PATH_DOWN;
 	}
Signature · 1 line
1.8.4.5
#37 lore raw

PATCH 34/78 Check for valid DM_DEVICE_INFO before proceeding

In reply to Hannes Reinecke

Calling 'DM_DEVICE_INFO' might succeed but the returned context might refer to an invalid device. So one needs to check the 'exists' field to avoid this.

---
 libmultipath/devmapper.c | 9 +++++++++
 1 file changed, 9 insertions(+)
libmultipath/devmapper.c +9 -0
index 5e68aeb..1901052 100644
@@ -615,6 +615,9 @@ dm_get_opencount (const char * mapname)
 	if (!dm_task_get_info(dmt, &info))
 		goto out;
 
+	if (!info.exists)
+		goto out;
+
 	r = info.open_count;
 out:
 	dm_task_destroy(dmt);
@@ -640,6 +643,9 @@ dm_get_major (char * mapname)
 	if (!dm_task_get_info(dmt, &info))
 		goto out;
 
+	if (!info.exists)
+		goto out;
+
 	r = info.major;
 out:
 	dm_task_destroy(dmt);
@@ -665,6 +671,9 @@ dm_get_minor (char * mapname)
 	if (!dm_task_get_info(dmt, &info))
 		goto out;
 
+	if (!info.exists)
+		goto out;
+
 	r = info.minor;
 out:
 	dm_task_destroy(dmt);
Signature · 1 line
1.8.4.5
#38 lore raw

PATCH 35/78 Separate out uevent parsing functions

In reply to Hannes Reinecke

Move the uevent parsing into separate functions.
No functional change.

---
 libmultipath/uevent.c | 254 +++++++++++++++++++++++++++-----------------------
 1 file changed, 136 insertions(+), 118 deletions(-)
libmultipath/uevent.c +136 -118
index 9ee3ade..7e41c6f 100644
@@ -182,6 +182,80 @@ int uevent_dispatch(int (*uev_trigger)(struct uevent *, void * trigger_data),
 	return 0;
 }
 
+struct uevent *uevent_from_buffer(char *buf, ssize_t buflen)
+{
+	struct uevent *uev;
+	char *buffer;
+	size_t bufpos;
+	int i;
+	char *pos;
+
+	uev = alloc_uevent();
+	if (!uev) {
+		condlog(1, "lost uevent, oom");
+		return NULL;
+	}
+
+	if ((size_t)buflen > sizeof(buf)-1)
+		buflen = sizeof(buf)-1;
+
+	/*
+	 * Copy the shared receive buffer contents to buffer private
+	 * to this uevent so we can immediately reuse the shared buffer.
+	 */
+	memcpy(uev->buffer, buf, HOTPLUG_BUFFER_SIZE + OBJECT_SIZE);
+	buffer = uev->buffer;
+	buffer[buflen] = '\0';
+
+	/* save start of payload */
+	bufpos = strlen(buffer) + 1;
+
+	/* action string */
+	uev->action = buffer;
+	pos = strchr(buffer, '@');
+	if (!pos) {
+		condlog(3, "bad action string '%s'", buffer);
+		FREE(uev);
+		return NULL;
+	}
+	pos[0] = '\0';
+
+	/* sysfs path */
+	uev->devpath = &pos[1];
+
+	/* hotplug events have the environment attached - reconstruct envp[] */
+	for (i = 0; (bufpos < (size_t)buflen) && (i < HOTPLUG_NUM_ENVP-1); i++) {
+		int keylen;
+		char *key;
+
+		key = &buffer[bufpos];
+		keylen = strlen(key);
+		uev->envp[i] = key;
+		/* Filter out sequence number */
+		if (strncmp(key, "SEQNUM=", 7) == 0) {
+			char *eptr;
+
+			uev->seqnum = strtoul(key + 7, &eptr, 10);
+			if (eptr == key + 7)
+				uev->seqnum = -1;
+		}
+		bufpos += keylen + 1;
+	}
+	uev->envp[i] = NULL;
+
+	condlog(3, "uevent %ld '%s' from '%s'", uev->seqnum,
+		uev->action, uev->devpath);
+	uev->kernel = strrchr(uev->devpath, '/');
+	if (uev->kernel)
+		uev->kernel++;
+
+	/* print payload environment */
+	for (i = 0; uev->envp[i] != NULL; i++)
+		condlog(5, "%s", uev->envp[i]);
+
+	return uev;
+}
+
 int failback_listen(void)
 {
 	int sock;
@@ -266,12 +340,9 @@ int failback_listen(void)
 	}
 
 	while (1) {
-		int i;
-		char *pos;
 		size_t bufpos;
 		ssize_t buflen;
 		struct uevent *uev;
-		char *buffer;
 		struct msghdr smsg;
 		struct iovec iov;
 		char cred_msg[CMSG_SPACE(sizeof(struct ucred))];
@@ -324,69 +395,9 @@ int failback_listen(void)
 			buflen = sizeof(buf)-1;
 		}
 
-		uev = alloc_uevent();
-
-		if (!uev) {
-			condlog(1, "lost uevent, oom");
-			continue;
-		}
-
-		if ((size_t)buflen > sizeof(buf)-1)
-			buflen = sizeof(buf)-1;
-
-		/*
-		 * Copy the shared receive buffer contents to buffer private
-		 * to this uevent so we can immediately reuse the shared buffer.
-		 */
-		memcpy(uev->buffer, buf, HOTPLUG_BUFFER_SIZE + OBJECT_SIZE);
-		buffer = uev->buffer;
-		buffer[buflen] = '\0';
-
-		/* save start of payload */
-		bufpos = strlen(buffer) + 1;
-
-		/* action string */
-		uev->action = buffer;
-		pos = strchr(buffer, '@');
-		if (!pos) {
-			condlog(3, "bad action string '%s'", buffer);
+		uev = uevent_from_buffer(buf, buflen);
+		if (!uev)
 			continue;
-		}
-		pos[0] = '\0';
-
-		/* sysfs path */
-		uev->devpath = &pos[1];
-
-		/* hotplug events have the environment attached - reconstruct envp[] */
-		for (i = 0; (bufpos < (size_t)buflen) && (i < HOTPLUG_NUM_ENVP-1); i++) {
-			int keylen;
-			char *key;
-
-			key = &buffer[bufpos];
-			keylen = strlen(key);
-			uev->envp[i] = key;
-			/* Filter out sequence number */
-			if (strncmp(key, "SEQNUM=", 7) == 0) {
-				char *eptr;
-
-				uev->seqnum = strtoul(key + 7, &eptr, 10);
-				if (eptr == key + 7)
-					uev->seqnum = -1;
-			}
-			bufpos += keylen + 1;
-		}
-		uev->envp[i] = NULL;
-
-		condlog(3, "uevent %ld '%s' from '%s'", uev->seqnum,
-			uev->action, uev->devpath);
-		uev->kernel = strrchr(uev->devpath, '/');
-		if (uev->kernel)
-			uev->kernel++;
-
-		/* print payload environment */
-		for (i = 0; uev->envp[i] != NULL; i++)
-			condlog(5, "%s", uev->envp[i]);
-
 		/*
 		 * Queue uevent and poke service pthread.
 		 */
@@ -401,6 +412,62 @@ exit:
 	return 1;
 }
 
+struct uevent *uevent_from_udev_device(struct udev_device *dev)
+{
+	struct uevent *uev;
+	int i = 0;
+	char *pos, *end;
+	struct udev_list_entry *list_entry;
+
+	uev = alloc_uevent();
+	if (!uev) {
+		udev_device_unref(dev);
+		condlog(1, "lost uevent, oom");
+		return NULL;
+	}
+	pos = uev->buffer;
+	end = pos + HOTPLUG_BUFFER_SIZE + OBJECT_SIZE - 1;
+	udev_list_entry_foreach(list_entry, udev_device_get_properties_list_entry(dev)) {
+		const char *name, *value;
+		int bytes;
+
+		name = udev_list_entry_get_name(list_entry);
+		if (!name)
+			name = "(null)";
+		value = udev_list_entry_get_value(list_entry);
+		if (!value)
+			value = "(null)";
+		bytes = snprintf(pos, end - pos, "%s=%s", name, value);
+		if (pos + bytes >= end) {
+			condlog(2, "buffer overflow for uevent");
+			break;
+		}
+		uev->envp[i] = pos;
+		pos += bytes;
+		*pos = '\0';
+		pos++;
+		if (strcmp(name, "DEVPATH") == 0)
+			uev->devpath = uev->envp[i] + 8;
+		if (strcmp(name, "ACTION") == 0)
+			uev->action = uev->envp[i] + 7;
+		i++;
+		if (i == HOTPLUG_NUM_ENVP - 1)
+			break;
+	}
+	uev->udev = dev;
+	uev->envp[i] = NULL;
+
+	condlog(3, "uevent '%s' from '%s'", uev->action, uev->devpath);
+	uev->kernel = strrchr(uev->devpath, '/');
+	if (uev->kernel)
+		uev->kernel++;
+
+	/* print payload environment */
+	for (i = 0; uev->envp[i] != NULL; i++)
+		condlog(5, "%s", uev->envp[i]);
+	return uev;
+}
+
 int uevent_listen(struct udev *udev)
 {
 	int err;
@@ -456,69 +523,20 @@ int uevent_listen(struct udev *udev)
 		goto out;
 	}
 	while (1) {
-		int i = 0;
-		char *pos, *end;
 		struct uevent *uev;
 		struct udev_device *dev;
-                struct udev_list_entry *list_entry;
 
 		dev = udev_monitor_receive_device(monitor);
 		if (!dev) {
 			condlog(0, "failed getting udev device");
 			continue;
 		}
-
-		uev = alloc_uevent();
-		if (!uev) {
-			udev_device_unref(dev);
-			condlog(1, "lost uevent, oom");
+		uev = uevent_from_udev_device(dev);
+		if (!uev)
 			continue;
-		}
-		pos = uev->buffer;
-		end = pos + HOTPLUG_BUFFER_SIZE + OBJECT_SIZE - 1;
-		udev_list_entry_foreach(list_entry, udev_device_get_properties_list_entry(dev)) {
-			const char *name, *value;
-			int bytes;
-
-			name = udev_list_entry_get_name(list_entry);
-			if (!name)
-				name = "(null)";
-			value = udev_list_entry_get_value(list_entry);
-			if (!value)
-				value = "(null)";
-			bytes = snprintf(pos, end - pos, "%s=%s", name,
-					value);
-			if (pos + bytes >= end) {
-				condlog(2, "buffer overflow for uevent");
-				break;
-			}
-			uev->envp[i] = pos;
-			pos += bytes;
-			*pos = '\0';
-			pos++;
-			if (strcmp(name, "DEVPATH") == 0)
-				uev->devpath = uev->envp[i] + 8;
-			if (strcmp(name, "ACTION") == 0)
-				uev->action = uev->envp[i] + 7;
-			i++;
-			if (i == HOTPLUG_NUM_ENVP - 1)
-				break;
-		}
-		uev->udev = dev;
-		uev->envp[i] = NULL;
-
-		condlog(3, "uevent '%s' from '%s'", uev->action, uev->devpath);
-		uev->kernel = strrchr(uev->devpath, '/');
-		if (uev->kernel)
-			uev->kernel++;
-
-		/* print payload environment */
-		for (i = 0; uev->envp[i] != NULL; i++)
-			condlog(5, "%s", uev->envp[i]);
-
 		/*
- 		 * Queue uevent and poke service pthread.
- 		 */
+		 * Queue uevent and poke service pthread.
+		 */
 		pthread_mutex_lock(uevq_lockp);
 		list_add_tail(&uev->node, &uevq);
 		pthread_cond_signal(uev_condp);
Signature · 1 line
1.8.4.5
#39 lore raw

PATCH 36/78 Use poll() when receiving uevents

In reply to Hannes Reinecke

Currently uevent_listen() will be adding each event individually to the internal queue. This leads to a lock contention on high load, as uevent_listen() has to grab the shared lock before doing so.
This patch batches the uevent reception so that uevents will only ever be added to the internal queue if there are no more events pending.

---
 libmultipath/list.h   | 47 +++++++++++++++++++++++++-------
 libmultipath/uevent.c | 74 +++++++++++++++++++++++++++++++++++++++------------
 2 files changed, 95 insertions(+), 26 deletions(-)
libmultipath/list.h +38 -9
index 8626630..7a3cf16 100644
@@ -161,18 +161,18 @@ static inline int list_empty(struct list_head *head)
 	return head->next == head;
 }
 
-static inline void __list_splice(struct list_head *list,
-				 struct list_head *head)
+static inline void __list_splice(const struct list_head *list,
+				 struct list_head *prev,
+				 struct list_head *next)
 {
 	struct list_head *first = list->next;
 	struct list_head *last = list->prev;
-	struct list_head *at = head->next;
 
-	first->prev = head;
-	head->next = first;
+	first->prev = prev;
+	prev->next = first;
 
-	last->next = at;
-	at->prev = last;
+	last->next = next;
+	next->prev = last;
 }
 
 /**
@@ -183,7 +183,19 @@ static inline void __list_splice(struct list_head *list,
 static inline void list_splice(struct list_head *list, struct list_head *head)
 {
 	if (!list_empty(list))
-		__list_splice(list, head);
+		__list_splice(list, head, head->next);
+}
+
+/**
+ * list_splice_tail - join two lists, each list being a queue
+ * @list: the new list to add.
+ * @head: the place to add it in the first list.
+ */
+static inline void list_splice_tail(struct list_head *list,
+				    struct list_head *head)
+{
+	if (!list_empty(list))
+		__list_splice(list, head->prev, head);
 }
 
 /**
@@ -197,7 +209,24 @@ static inline void list_splice_init(struct list_head *list,
 				    struct list_head *head)
 {
 	if (!list_empty(list)) {
-		__list_splice(list, head);
+		__list_splice(list, head, head->next);
+		INIT_LIST_HEAD(list);
+	}
+}
+
+/**
+ * list_splice_tail_init - join two lists and reinitialise the emptied list
+ * @list: the new list to add.
+ * @head: the place to add it in the first list.
+ *
+ * Each of the lists is a queue.
+ * The list at @list is reinitialised
+ */
+static inline void list_splice_tail_init(struct list_head *list,
+					 struct list_head *head)
+{
+	if (!list_empty(list)) {
+		__list_splice(list, head->prev, head);
 		INIT_LIST_HEAD(list);
 	}
 }
libmultipath/uevent.c +57 -17

111 lines not on this page · raw message

Signature · 1 line
1.8.4.5
#40 lore raw

PATCH 37/78 mpath_persist: cleanup

In reply to Hannes Reinecke

Cleanup whitespace issues and use the standard 'major' and 'minor' macros.

---
 libmpathpersist/mpath_persist.c | 106 +++++++++++++++++++++-------------------
 libmpathpersist/mpathpr.h       |   3 +-
 2 files changed, 58 insertions(+), 51 deletions(-)
libmpathpersist/mpath_persist.c +57 -49
index d427c5b..eba26a2 100644
@@ -1,7 +1,7 @@
 #include <libdevmapper.h>
 #include <defaults.h>
 #include <sys/stat.h>
-#include <linux/kdev_t.h>
+#include <sys/types.h>
 #include <fcntl.h>
 #include <vector.h>
 #include <checkers.h>
@@ -19,6 +19,7 @@
 #include <dmparser.h>
 #include <ctype.h>
 #include <propsel.h>
+#include <util.h>
 
 #include "mpath_persist.h"
 #include "mpathpr.h"
@@ -71,7 +72,8 @@ updatepaths (struct multipath * mpp)
 
 		vector_foreach_slot (pgp->paths, pp, j){
 			if (!strlen(pp->dev)){
-				if (devt2devname(pp->dev, pp->dev_t)){
+				if (devt2devname(pp->dev, PATH_SIZE,
+						 pp->dev_t)){
 					/*
 					 * path is not in sysfs anymore
 					 */
@@ -94,7 +96,7 @@ updatepaths (struct multipath * mpp)
 	return 0;
 }
 
-int 
+int
 mpath_prin_activepath (struct multipath *mpp, int rq_servact,
 	struct prin_resp * resp, int noisy)
 {
@@ -104,14 +106,19 @@ mpath_prin_activepath (struct multipath *mpp, int rq_servact,
 
 	vector_foreach_slot (mpp->pg, pgp, j){
 		vector_foreach_slot (pgp->paths, pp, i){
-			if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){
-				condlog(2, "%s: %s not available. Skip.", mpp->wwid, pp->dev);
-				condlog(3, "%s: status = %d.", mpp->wwid, pp->state);
+			if (!((pp->state == PATH_UP) ||
+			      (pp->state == PATH_GHOST))){
+				condlog(2, "%s: %s not available. Skip.",
+					mpp->wwid, pp->dev);
+				condlog(3, "%s: status = %d.",
+					mpp->wwid, pp->state);
 				continue;
 			}
 
-			condlog(3, "%s: sending pr in command to %s ", mpp->wwid, pp->dev);
-			ret = mpath_send_prin_activepath(pp->dev, rq_servact, resp, noisy);
+			condlog(3, "%s: sending pr in command to %s ",
+				mpp->wwid, pp->dev);
+			ret = mpath_send_prin_activepath(pp->dev, rq_servact,
+							 resp, noisy);
 			switch(ret)
 			{
 				case MPATH_PR_SUCCESS:
@@ -122,10 +129,11 @@ mpath_prin_activepath (struct multipath *mpp, int rq_servact,
 			}
 		}
 	}
-	return ret;	
+	return ret;
 }
 
-int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp, int noisy, int verbose)
+int mpath_persistent_reserve_in (int fd, int rq_servact,
+	struct prin_resp *resp, int noisy, int verbose)
 {
 	struct stat info;
 	vector curmp = NULL;
@@ -141,14 +149,14 @@ int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp,
 	if (fstat( fd, &info) != 0){
 		condlog(0, "stat error %d", fd);
 		return MPATH_PR_FILE_ERROR;
-	} 
+	}
 	if(!S_ISBLK(info.st_mode)){
 		condlog(0, "Failed to get major:minor. fd = %d", fd);
 		return MPATH_PR_FILE_ERROR;
 	}
 
-	major = (int)MAJOR(info.st_rdev);
-	minor = (int)MINOR(info.st_rdev);	
+	major = major(info.st_rdev);
+	minor = minor(info.st_rdev);
 	condlog(4, "Device %d:%d:  ", major, minor);
 
 	/* get alias from major:minor*/
@@ -201,14 +209,14 @@ int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp,
 
 out1:
 	free_multipathvec(curmp, KEEP_PATHS);
-	free_pathvec(pathvec, FREE_PATHS);	
+	free_pathvec(pathvec, FREE_PATHS);
 out:
 	FREE(alias);
-	return ret; 						
+	return ret;
 }
 
 int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
-		unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy, int verbose)
+	unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy, int verbose)
 {
 
 	struct stat info;
@@ -223,7 +231,7 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 	int ret;
 	int j;
 	unsigned char *keyp;
-	uint64_t prkey;		
+	uint64_t prkey;
 
 	conf->verbosity = verbose;
 
@@ -234,11 +242,11 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 
 	if(!S_ISBLK(info.st_mode)){
 		condlog(3, "Failed to get major:minor. fd=%d", fd);
-		return MPATH_PR_FILE_ERROR;	
-	}	
+		return MPATH_PR_FILE_ERROR;
+	}
 
-	major = (int)MAJOR(info.st_rdev);
-	minor = (int)MINOR(info.st_rdev);
+	major = major(info.st_rdev);
+	minor = minor(info.st_rdev);
 	condlog(4, "Device  %d:%d", major, minor);
 
 	/* get WWN of the device from major:minor*/
@@ -292,22 +300,22 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 
 	switch(rq_servact)
 	{
-		case MPATH_PROUT_REG_SA: 
-		case MPATH_PROUT_REG_IGN_SA:  
-			ret= mpath_prout_reg(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
-			break;
-		case MPATH_PROUT_RES_SA :  
-		case MPATH_PROUT_PREE_SA :  
-		case MPATH_PROUT_PREE_AB_SA :  
-		case MPATH_PROUT_CLEAR_SA:  
-			ret = mpath_prout_common(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
-			break;
-		case MPATH_PROUT_REL_SA:
-			ret = mpath_prout_rel(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
-			break;
-		default:
-			ret = MPATH_PR_OTHER;
-			goto out1;
+	case MPATH_PROUT_REG_SA:
+	case MPATH_PROUT_REG_IGN_SA:
+		ret= mpath_prout_reg(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
+		break;
+	case MPATH_PROUT_RES_SA :
+	case MPATH_PROUT_PREE_SA :
+	case MPATH_PROUT_PREE_AB_SA :
+	case MPATH_PROUT_CLEAR_SA:
+		ret = mpath_prout_common(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
+		break;
+	case MPATH_PROUT_REL_SA:
+		ret = mpath_prout_rel(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
+		break;
+	default:
+		ret = MPATH_PR_OTHER;
+		goto out1;
 	}
 
 	if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_REG_SA) ||
@@ -326,7 +334,7 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 		else
 			update_prflag(alias, "set", noisy);
 	} else {
-		if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_CLEAR_SA) || 
+		if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_CLEAR_SA) ||
 					(rq_servact == MPATH_PROUT_PREE_AB_SA ))){
 			update_prflag(alias, "unset", noisy);
 		}
@@ -337,7 +345,7 @@ out1:
 
 out:
 	FREE(alias);
-	return ret; 
+	return ret;
 }
 
 int
@@ -365,9 +373,9 @@ get_mpvec (vector curmp, vector pathvec, char * refwwid)
 		dm_get_map(mpp->alias, &mpp->size, params);
 		condlog(3, "params = %s", params);
 		dm_get_status(mpp->alias, status);
-                condlog(3, "status = %s", status);
+		condlog(3, "status = %s", status);
 		disassemble_map (pathvec, params, mpp);
-		
+
 		/*
 		 * disassemble_map() can add new paths to pathvec.
 		 * If not in "fast list mode", we need to fetch information
@@ -542,7 +550,7 @@ void * mpath_prout_pthread_fn(void *p)
 }
 
 int mpath_prout_common(struct multipath *mpp,int rq_servact, int rq_scope,
-        unsigned int rq_type, struct prout_param_descriptor* paramp, int noisy)
+	unsigned int rq_type, struct prout_param_descriptor* paramp, int noisy)
 {
 	int i,j, ret;
 	struct pathgroup *pgp = NULL;
@@ -600,7 +608,7 @@ int send_prout_activepath(char * dev, int rq_servact, int rq_scope,
 }
 
 int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
-        unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy)
+	unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy)
 {
 	int i, j;
 	int num = 0;
@@ -615,7 +623,7 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 	struct prout_param_descriptor *pamp;
 	struct prin_resp *pr_buff;
 	int length;
-	struct transportid *pptr;	
+	struct transportid *pptr;
 
 	if (!mpp)
 		return MPATH_PR_DMMP_ERROR;
@@ -649,7 +657,7 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 				condlog (1, "%s: %s path not up.", mpp->wwid, pp->dev);
 				continue;
 			}
-			
+
 			strncpy(thread[count].param.dev, pp->dev, FILE_NAME_SIZE);
 			condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev);
 			rc = pthread_create (&thread[count].id, &attr, mpath_prout_pthread_fn,
@@ -686,13 +694,13 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 	num = resp.prin_descriptor.prin_readresv.additional_length / 8;
 	if (num == 0){
 		condlog (2, "%s: Path holding reservation is released.", mpp->wwid);
-		return MPATH_PR_SUCCESS;	
+		return MPATH_PR_SUCCESS;
 	}
 	condlog (2, "%s: Path holding reservation is not avialable.", mpp->wwid);
 
 	pr_buff =  mpath_alloc_prin_response(MPATH_PRIN_RFSTAT_SA);
 	if (!pr_buff){
-		condlog (0, "%s: failed to  alloc pr in response buffer.", mpp->wwid);	
+		condlog (0, "%s: failed to  alloc pr in response buffer.", mpp->wwid);
 		return MPATH_PR_OTHER;
 	}
 
@@ -778,7 +786,7 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 		memset (pamp, 0, length);
 		memcpy (pamp->sa_key, mpp->reservation_key, 8);
 		memset (pamp->key, 0, 8);
-		status = mpath_prout_reg(mpp, MPATH_PROUT_REG_SA, rq_scope, rq_type, pamp, noisy);	
+		status = mpath_prout_reg(mpp, MPATH_PROUT_REG_SA, rq_scope, rq_type, pamp, noisy);
 	}
 
 
@@ -812,7 +820,7 @@ void * mpath_alloc_prin_response(int prin_sa)
 			memset(ptr, 0, size);
 			break;
 		case MPATH_PRIN_RFSTAT_SA:
-			size = sizeof(struct print_fulldescr_list) + 
+			size = sizeof(struct print_fulldescr_list) +
 				sizeof(struct prin_fulldescr *)*MPATH_MX_TIDS;
 			ptr = malloc(size);
 			memset(ptr, 0, size);
libmpathpersist/mpathpr.h +1 -2
index 54dfb3e..d69a732 100644
@@ -50,6 +50,5 @@ int send_prout_activepath(char * dev, int rq_servact, int rq_scope,
 int update_prflag(char * arg1, char * arg2, int noisy);
 void * mpath_alloc_prin_response(int prin_sa);
 int update_map_pr(struct multipath *mpp);
-int devt2devname (char *devname, char *devt);
 
-#endif  
+#endif
Signature · 1 line
1.8.4.5
#41 lore raw

PATCH 38/78 kpartx: use standard 'major' and 'minor' macros

In reply to Hannes Reinecke

Instead of hand-coding our own versions we should be using the standard 'major' and 'minor' macros.

---
 kpartx/devmapper.h |  7 ++++---
 kpartx/kpartx.c    | 13 +++++--------
 2 files changed, 9 insertions(+), 11 deletions(-)
kpartx/devmapper.h +4 -3
index d962e43..4b867df 100644
@@ -1,6 +1,5 @@
-#define MAJOR(dev)      ((dev & 0xfff00) >> 8)
-#define MINOR(dev)      ((dev & 0xff) | ((dev >> 12) & 0xfff00))
-#define MKDEV(ma,mi)    ((mi & 0xff) | (ma << 8) | ((mi & ~0xff) << 12))
+#ifndef _KPARTX_DEVMAPPER_H
+#define _KPARTX_DEVMAPPER_H
 
 #ifdef DM_SUBSYSTEM_UDEV_FLAG0
 #define MPATH_UDEV_RELOAD_FLAG DM_SUBSYSTEM_UDEV_FLAG0
@@ -20,3 +19,5 @@ dev_t dm_get_first_dep(char *devname);
 char * dm_mapuuid(int major, int minor);
 int dm_devn (char * mapname, int *major, int *minor);
 int dm_no_partitions(int major, int minor);
+
+#endif /* _KPARTX_DEVMAPPER_H */
kpartx/kpartx.c +5 -8
index fac98dc..18c1d23 100644
@@ -168,8 +168,8 @@ get_hotplug_device(void)
 	if (stat(devname, &buf))
 		return NULL;
 
-	major = (unsigned int)MAJOR(buf.st_rdev);
-	minor = (unsigned int)MINOR(buf.st_rdev);
+	major = major(buf.st_rdev);
+	minor = minor(buf.st_rdev);
 
 	if (!(mapname = dm_mapname(major, minor))) /* Not dm device. */
 		return NULL;
@@ -327,10 +327,8 @@ main(int argc, char **argv){
 	off = find_devname_offset(device);
 
 	if (!loopdev) {
-		uuid = dm_mapuuid((unsigned int)MAJOR(buf.st_rdev),
-				  (unsigned int)MINOR(buf.st_rdev));
-		mapname = dm_mapname((unsigned int)MAJOR(buf.st_rdev),
-				     (unsigned int)MINOR(buf.st_rdev));
+		uuid = dm_mapuuid(major(buf.st_rdev), minor(buf.st_rdev));
+		mapname = dm_mapname(major(buf.st_rdev), minor(buf.st_rdev));
 	}
 
 	if (!uuid)
@@ -339,8 +337,7 @@ main(int argc, char **argv){
 	if (!mapname)
 		mapname = device + off;
 	else if (!force_devmap &&
-		 dm_no_partitions((unsigned int)MAJOR(buf.st_rdev),
-				  (unsigned int)MINOR(buf.st_rdev))) {
+		 dm_no_partitions(major(buf.st_rdev), minor(buf.st_rdev))) {
 		/* Feature 'no_partitions' is set, return */
 		return 0;
 	}
Signature · 1 line
1.8.4.5
#42 lore raw

PATCH 39/78 multipath: Use standard 'major' macro

In reply to Hannes Reinecke

Instead of carrying our own hand-crafted macros we should be using the system-provided 'major' macro.

---
 multipath/dev_t.h | 3 ---
 multipath/main.c  | 3 +--
 2 files changed, 1 insertion(+), 5 deletions(-)
 delete mode 100644 multipath/dev_t.h
multipath/dev_t.h deleted +0 -3
deleted file mode 100644
index aa80d5e..0000000
@@ -1,3 +0,0 @@
-#define MAJOR(dev)      ((dev & 0xfff00) >> 8)
-#define MINOR(dev)      ((dev & 0xff) | ((dev >> 12) & 0xfff00))
-#define MKDEV(ma,mi)    ((mi & 0xff) | (ma << 8) | ((mi & ~0xff) << 12))
multipath/main.c +1 -2
index ce8fb5b..1c1191a 100644
@@ -57,7 +57,6 @@
 #include <sys/resource.h>
 #include <wwids.h>
 #include <uxsock.h>
-#include "dev_t.h"
 
 int logsink;
 
@@ -454,7 +453,7 @@ get_dev_type(char *dev) {
 	int i;
 
 	if (stat(dev, &buf) == 0 && S_ISBLK(buf.st_mode)) {
-		if (dm_is_dm_major(MAJOR(buf.st_rdev)))
+		if (dm_is_dm_major(major(buf.st_rdev)))
 			return DEV_DEVMAP;
 		return DEV_DEVNODE;
 	}
Signature · 1 line
1.8.4.5
#43 lore raw

PATCH 40/78 Remove sysfs_get_dev

In reply to Hannes Reinecke

Pointless, and can be replaced by udev_device_get_devnum().

---
 libmultipath/discovery.c   | 9 ++++-----
 libmultipath/discovery.h   | 1 -
 libmultipath/structs_vec.c | 4 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)
libmultipath/discovery.c +4 -5
index 899cbd5..7d3ff49 100644
@@ -172,7 +172,6 @@ declare_sysfs_get_str(devtype);
 declare_sysfs_get_str(vendor);
 declare_sysfs_get_str(model);
 declare_sysfs_get_str(rev);
-declare_sysfs_get_str(dev);
 
 int
 sysfs_get_timeout(struct path *pp, unsigned int *timeout)
@@ -900,6 +899,8 @@ cciss_sysfs_pathinfo (struct path * pp)
 static int
 common_sysfs_pathinfo (struct path * pp)
 {
+	dev_t devt;
+
 	if (!pp)
 		return 1;
 
@@ -907,10 +908,8 @@ common_sysfs_pathinfo (struct path * pp)
 		condlog(4, "%s: udev not initialised", pp->dev);
 		return 1;
 	}
-	if (sysfs_get_dev(pp->udev, pp->dev_t, BLK_DEV_SIZE) <= 0) {
-		condlog(3, "%s: no 'dev' attribute in sysfs", pp->dev);
-		return 1;
-	}
+	devt = udev_device_get_devnum(pp->udev);
+	snprintf(pp->dev_t, BLK_DEV_SIZE, "%d:%d", major(devt), minor(devt));
 
 	condlog(3, "%s: dev_t = %s", pp->dev, pp->dev_t);
 
libmultipath/discovery.h +0 -1
index f14fcee..7e5680e 100644
@@ -30,7 +30,6 @@
 
 struct config;
 
-ssize_t sysfs_get_dev (struct udev_device *udev, char * buff, size_t len);
 int path_discovery (vector pathvec, struct config * conf, int flag);
 
 int do_tur (char *);
libmultipath/structs_vec.c +3 -1
index a1f2212..79f31b1 100644
@@ -8,6 +8,7 @@
 #include "debug.h"
 #include "structs.h"
 #include "structs_vec.h"
+#include "sysfs.h"
 #include "waiter.h"
 #include "devmapper.h"
 #include "dmparser.h"
@@ -501,7 +502,8 @@ verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec)
 		/*
 		 * see if path is in sysfs
 		 */
-		if (sysfs_get_dev(pp->udev, pp->dev_t, BLK_DEV_SIZE) <= 0) {
+		if (sysfs_attr_get_value(pp->udev, "dev",
+					 pp->dev_t, BLK_DEV_SIZE) < 0) {
 			if (pp->state != PATH_DOWN) {
 				condlog(1, "%s: removing valid path %s in state %d",
 					mpp->alias, pp->dev, pp->state);
Signature · 1 line
1.8.4.5
#44 lore raw

PATCH 41/78 Add paths with a size of '0' as 'ghost' paths

In reply to Hannes Reinecke

SPC-3 does not require that a port in 'standby' has to support the 'READ CAPACITY' command. As such we should not reject those paths, but rather add them as 'ghost' paths.

---
 libmultipath/discovery.c   |  5 +++++
 libmultipath/structs_vec.c |  2 +-
 multipathd/main.c          | 26 +++++---------------------
 3 files changed, 11 insertions(+), 22 deletions(-)
libmultipath/discovery.c +5 -0
index 7d3ff49..6ba14ac 100644
@@ -1215,6 +1215,11 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 				goto blank;
 			if (pp->state == PATH_TIMEOUT)
 				pp->state = PATH_DOWN;
+			if (pp->state == PATH_UP && !pp->size) {
+				condlog(3, "%s: device size is 0, "
+					"path unuseable", pp->dev);
+				pp->state = PATH_GHOST;
+			}
 		} else {
 			condlog(3, "%s: path inaccessible", pp->dev);
 			pp->chkrstate = pp->state = path_state;
libmultipath/structs_vec.c +1 -1
index 79f31b1..6be8c51 100644
@@ -558,7 +558,7 @@ int update_multipath (struct vectors *vecs, char *mapname, int reset)
 
 			if (pp->state != PATH_DOWN) {
 				int oldstate = pp->state;
-				condlog(2, "%s: mark as failed", pp->dev_t);
+				condlog(2, "%s: mark as failed", pp->dev);
 				mpp->stat_path_failures++;
 				pp->state = PATH_DOWN;
 				if (oldstate == PATH_UP ||
multipathd/main.c +5 -21
index ac1e1a2..6a3fd5e 100644
@@ -479,15 +479,10 @@ ev_add_path (struct path * pp, struct vectors * vecs)
 	mpp = pp->mpp = find_mp_by_wwid(vecs->mpvec, pp->wwid);
 rescan:
 	if (mpp) {
-		if ((!pp->size) || (mpp->size != pp->size)) {
-			if (!pp->size)
-				condlog(0, "%s: failed to add new path %s, "
-					"device size is 0",
-					mpp->alias, pp->dev);
-			else
-				condlog(0, "%s: failed to add new path %s, "
-					"device size mismatch",
-					mpp->alias, pp->dev);
+		if (mpp->size != pp->size) {
+			condlog(0, "%s: failed to add new path %s, "
+				"device size mismatch",
+				mpp->alias, pp->dev);
 			int i = find_slot(vecs->pathvec, (void *)pp);
 			if (i != -1)
 				vector_del_slot(vecs->pathvec, i);
@@ -503,18 +498,7 @@ rescan:
 		verify_paths(mpp, vecs, NULL);
 		mpp->flush_on_last_del = FLUSH_UNDEF;
 		mpp->action = ACT_RELOAD;
-	}
-	else {
-		if (!pp->size) {
-			condlog(0, "%s: failed to create new map,"
-				" device size is 0 ", pp->dev);
-			int i = find_slot(vecs->pathvec, (void *)pp);
-			if (i != -1)
-				vector_del_slot(vecs->pathvec, i);
-			free_path(pp);
-			return 1;
-		}
-
+	} else {
 		if (conf->find_multipaths &&
 		    !should_multipath(pp, vecs->pathvec)) {
 			orphan_path(pp, "only one path");
Signature · 1 line
1.8.4.5
#45 lore raw

PATCH 42/78 Remove last argument from verify_paths()

In reply to Hannes Reinecke

Argument is always NULL, so remove it.

---
 libmultipath/configure.c   |  2 +-
 libmultipath/structs_vec.c | 14 +++++---------
 libmultipath/structs_vec.h |  2 +-
 multipathd/main.c          |  2 +-
 4 files changed, 8 insertions(+), 12 deletions(-)
libmultipath/configure.c +1 -1
index a30ca59..2465563 100644
@@ -820,7 +820,7 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			if (pp2->priority == PRIO_UNDEF)
 				mpp->action = ACT_REJECT;
 		}
-		verify_paths(mpp, vecs, NULL);
+		verify_paths(mpp, vecs);
 
 		params[0] = '\0';
 		if (setup_map(mpp, params, PARAMS_SIZE)) {
libmultipath/structs_vec.c +5 -9
index 6be8c51..6d2d45e 100644
@@ -489,7 +489,7 @@ out:
 }
 
 extern int
-verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec)
+verify_paths(struct multipath * mpp, struct vectors * vecs)
 {
 	struct path * pp;
 	int count = 0;
@@ -515,14 +515,10 @@ verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec)
 			vector_del_slot(mpp->paths, i);
 			i--;
 
-			if (rpvec)
-				store_path(rpvec, pp);
-			else {
-				if ((j = find_slot(vecs->pathvec,
-						   (void *)pp)) != -1)
-					vector_del_slot(vecs->pathvec, j);
-				free_path(pp);
-			}
+			if ((j = find_slot(vecs->pathvec,
+					   (void *)pp)) != -1)
+				vector_del_slot(vecs->pathvec, j);
+			free_path(pp);
 		} else {
 			condlog(4, "%s: verified path %s dev_t %s",
 				mpp->alias, pp->dev, pp->dev_t);
libmultipath/structs_vec.h +1 -1
index c6278ac..eb8e672 100644
@@ -19,7 +19,7 @@ int adopt_paths (vector pathvec, struct multipath * mpp, int get_info);
 void orphan_paths (vector pathvec, struct multipath * mpp);
 void orphan_path (struct path * pp, const char *reason);
 
-int verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec);
+int verify_paths(struct multipath * mpp, struct vectors * vecs);
 int update_mpp_paths(struct multipath * mpp, vector pathvec);
 int __setup_multipath (struct vectors * vecs, struct multipath * mpp,
 		       int reset);
multipathd/main.c +1 -1
index 6a3fd5e..ffe4326 100644
@@ -495,7 +495,7 @@ rescan:
 		if (adopt_paths(vecs->pathvec, mpp, 1))
 			goto fail; /* leave path added to pathvec */
 
-		verify_paths(mpp, vecs, NULL);
+		verify_paths(mpp, vecs);
 		mpp->flush_on_last_del = FLUSH_UNDEF;
 		mpp->action = ACT_RELOAD;
 	} else {
Signature · 1 line
1.8.4.5
#46 lore raw

PATCH 43/78 Fixup device-mapper 'cookie' handling

In reply to Hannes Reinecke

device-mapper has a 'cookie', which is inserted with the ioctl for modifying device-mapper devices.
It is used as a synchronization point between udev and any other applications to notify the latter when udev has finished processing the event.
Originally multipath would only use a single cookie for every transaction, and wait for that cookie at the end of the program. Which works well if you only have one transaction, but for several (like calling 'multipath') it will actually overwrite the cookie and fail to wait for earlier events.
This causes libdevmapper to create the device nodes on its own, and the device nodes not being handled by udev.

---
 kpartx/devmapper.c        | 53 +++++++++++++++++++++++++++++++++++------------
 kpartx/devmapper.h        |  4 ++--
 kpartx/kpartx.c           | 22 +++++++++-----------
 libmultipath/config.h     |  1 -
 libmultipath/configure.c  |  5 +++--
 libmultipath/devmapper.c  | 48 +++++++++++++++++++++++++++++++-----------
 libmultipath/devmapper.h  |  2 +-
 multipath/main.c          |  2 --
 multipathd/cli_handlers.c |  4 ++--
 9 files changed, 94 insertions(+), 47 deletions(-)
kpartx/devmapper.c +40 -13
index a3272d4..82be990 100644
@@ -14,13 +14,6 @@
 #define MAX_PREFIX_LEN 8
 #define PARAMS_SIZE 1024
 
-#ifndef LIBDM_API_COOKIE
-static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
-{
-	return 1;
-}
-#endif
-
 extern int
 dm_prereq (char * str, int x, int y, int z)
 {
@@ -60,10 +53,13 @@ dm_prereq (char * str, int x, int y, int z)
 }
 
 extern int
-dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16_t udev_flags) {
+dm_simplecmd (int task, const char *name, int no_flush, uint16_t udev_flags) {
 	int r = 0;
 	int udev_wait_flag = (task == DM_DEVICE_RESUME ||
 			      task == DM_DEVICE_REMOVE);
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+#endif
 	struct dm_task *dmt;
 
 	if (!(dmt = dm_task_create(task)))
@@ -78,10 +74,23 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 	if (no_flush)
 		dm_task_no_flush(dmt);
 
-	if (udev_wait_flag && !dm_task_set_cookie(dmt, cookie, ((udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK) | udev_flags))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags |= DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto out;
+	}
+#endif
 	r = dm_task_run(dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (udev_wait_flag) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 	out:
 	dm_task_destroy(dmt);
 	return r;
@@ -90,10 +99,14 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 extern int
 dm_addmap (int task, const char *name, const char *target,
 	   const char *params, uint64_t size, int ro, const char *uuid, int part,
-	   mode_t mode, uid_t uid, gid_t gid, uint32_t *cookie) {
+	   mode_t mode, uid_t uid, gid_t gid) {
 	int r = 0;
 	struct dm_task *dmt;
 	char *prefixed_uuid = NULL;
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+	uint16_t udev_flags = 0;
+#endif
 
 	if (!(dmt = dm_task_create (task)))
 		return 0;
@@ -128,10 +141,24 @@ dm_addmap (int task, const char *name, const char *target,
 
 	dm_task_no_open_count(dmt);
 
-	if (task == DM_DEVICE_CREATE && !dm_task_set_cookie(dmt, cookie, (udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (task == DM_DEVICE_CREATE &&
+	    !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto addout;
+	}
+#endif
 	r = dm_task_run (dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (task == DM_DEVICE_CREATE) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 addout:
 	dm_task_destroy (dmt);
 	free(prefixed_uuid);
kpartx/devmapper.h +2 -2
index 4b867df..ac1d5d9 100644
@@ -10,9 +10,9 @@
 extern int udev_sync;
 
 int dm_prereq (char *, int, int, int);
-int dm_simplecmd (int, const char *, int, uint32_t *, uint16_t);
+int dm_simplecmd (int, const char *, int, uint16_t);
 int dm_addmap (int, const char *, const char *, const char *, uint64_t,
-	       int, const char *, int, mode_t, uid_t, gid_t, uint32_t *);
+	       int, const char *, int, mode_t, uid_t, gid_t);
 int dm_map_present (char *);
 char * dm_mapname(int major, int minor);
 dev_t dm_get_first_dep(char *devname);
kpartx/kpartx.c +10 -12
index 18c1d23..d69f9af 100644
@@ -208,7 +208,6 @@ main(int argc, char **argv){
 	int hotplug = 0;
 	int loopcreated = 0;
 	struct stat buf;
-	uint32_t cookie = 0;
 
 	initpts();
 	init_crc32();
@@ -281,6 +280,8 @@ main(int argc, char **argv){
 #ifdef LIBDM_API_COOKIE
 	if (!udev_sync)
 		dm_udev_set_sync_support(0);
+	else
+		dm_udev_set_sync_support(1);
 #endif
 
 	if (dm_prereq(DM_TARGET, 0, 0, 0) && (what == ADD || what == DELETE || what == UPDATE)) {
@@ -437,7 +438,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE, partname,
-						  0, &cookie, 0)) {
+						  0, 0)) {
 					r++;
 					continue;
 				}
@@ -488,18 +489,19 @@ main(int argc, char **argv){
 				if (!dm_addmap(op, partname, DM_TARGET, params,
 					       slices[j].size, ro, uuid, j+1,
 					       buf.st_mode & 0777, buf.st_uid,
-					       buf.st_gid, &cookie)) {
+					       buf.st_gid)) {
 					fprintf(stderr, "create/reload failed on %s\n",
 						partname);
 					r++;
 				}
 				if (op == DM_DEVICE_RELOAD &&
 				    !dm_simplecmd(DM_DEVICE_RESUME, partname,
-						  1, &cookie, MPATH_UDEV_RELOAD_FLAG)) {
+						  1, MPATH_UDEV_RELOAD_FLAG)) {
 					fprintf(stderr, "resume failed on %s\n",
 						partname);
 					r++;
 				}
+
 				dm_devn(partname, &slices[j].major,
 					&slices[j].minor);
 
@@ -551,14 +553,12 @@ main(int argc, char **argv){
 					dm_addmap(op, partname, DM_TARGET, params,
 						  slices[j].size, ro, uuid, j+1,
 						  buf.st_mode & 0777,
-						  buf.st_uid, buf.st_gid,
-						  &cookie);
+						  buf.st_uid, buf.st_gid);
 
 					if (op == DM_DEVICE_RELOAD)
 						dm_simplecmd(DM_DEVICE_RESUME,
 							     partname, 1,
-							     &cookie, MPATH_UDEV_RELOAD_FLAG);
-
+							     MPATH_UDEV_RELOAD_FLAG);
 					dm_devn(partname, &slices[j].major,
 						&slices[j].minor);
 
@@ -590,7 +590,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE,
-						  partname, 1, &cookie, 0)) {
+						  partname, 1, 0)) {
 					r++;
 					continue;
 				}
@@ -616,9 +616,7 @@ main(int argc, char **argv){
 		}
 		printf("loop deleted : %s\n", device);
 	}
-#ifdef LIBDM_API_COOKIE
-	dm_udev_wait(cookie);
-#endif
+
 	dm_lib_release();
 	dm_lib_exit();
 
libmultipath/config.h +0 -1
index d304a6c..eff127e 100644
@@ -127,7 +127,6 @@ struct config {
 	uid_t uid;
 	gid_t gid;
 	mode_t mode;
-	uint32_t cookie;
 	int reassign_maps;
 	int retain_hwhandler;
 	int detect_prio;
libmultipath/configure.c +3 -2
index 2465563..3c230a1 100644
@@ -623,7 +623,8 @@ domap (struct multipath * mpp, char * params)
 	case ACT_RELOAD:
 		r = dm_addmap_reload(mpp, params);
 		if (r)
-			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias,
+						 0, MPATH_UDEV_RELOAD_FLAG);
 		break;
 
 	case ACT_RESIZE:
@@ -641,7 +642,7 @@ domap (struct multipath * mpp, char * params)
 		if (r) {
 			r = dm_addmap_reload(mpp, params);
 			if (r)
-				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG);
 		}
 		break;
 
libmultipath/devmapper.c +36 -12

124 lines not on this page · raw message

libmultipath/devmapper.h +1 -1
index 5c8c50d..8188f48 100644
@@ -16,7 +16,7 @@ void dm_init(void);
 int dm_prereq (void);
 int dm_drv_version (unsigned int * version, char * str);
 int dm_simplecmd_flush (int, const char *, int, uint16_t);
-int dm_simplecmd_noflush (int, const char *, uint16_t);
+int dm_simplecmd_noflush (int, const char *, int, uint16_t);
 int dm_addmap_create (struct multipath *mpp, char *params);
 int dm_addmap_reload (struct multipath *mpp, char *params);
 int dm_map_present (const char *);
multipath/main.c +0 -2
index 1c1191a..c46a9f6 100644
@@ -675,8 +675,6 @@ main (int argc, char *argv[])
 		condlog(3, "restart multipath configuration process");
 
 out:
-	udev_wait(conf->cookie);
-
 	dm_lib_release();
 	dm_lib_exit();
 
multipathd/cli_handlers.c +2 -2
index 6abe72a..772531e 100644
@@ -839,7 +839,7 @@ cli_suspend(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: suspend (operator)", param);
@@ -861,7 +861,7 @@ cli_resume(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: resume (operator)", param);
Signature · 1 line
1.8.4.5
#47 lore raw

Re: [PATCH 43/78] Fixup device-mapper 'cookie' handling

Quoted 10 lines · Hannes Reinecke

device-mapper has a 'cookie', which is inserted with the ioctl for modifying device-mapper devices.
It is used as a synchronization point between udev and any other applications to notify the latter when udev has finished processing the event.
Originally multipath would only use a single cookie for every transaction, and wait for that cookie at the end of the program. Which works well if you only have one transaction, but for several (like calling 'multipath') it will actually overwrite the cookie and fail to wait for earlier events.

That shouldn't be happening. Looking at the dm_task_set_cookie code

        if (*cookie) {
                if (!_get_cookie_sem(*cookie, &semid))
                        goto_bad;
        } else if (!_udev_notify_sem_create(cookie, &semid))
                goto_bad;

The first time we use that cookie, it should be zero, so a new semaphore will be created, and the id will be returned to us. Subsequent calls to dm_task_set_cookie with the same cookie (which is now non-zero) should just be using the same semaphore, allowing you you wait just one time on all the actions linked to that cookie. This should be more efficient than having to wait on each action (since udev can complete them in the background as we go on).

If there really are cookies that are not waited for, you should be able to see that by running

# dmsetup udevcookies

After running the multipath command. Cookies won't go away until someone waits for them. If there are cookies listed there, then my guess would be that something is resetting conf->cookie to 0 after our first call to dm_task_set_cookie. If there aren't, then it would appear that something else, instead of not waiting for the cookies, is causing device mapper to fail back to manual device node creation.

If you can verify that you are passing the cookie value that you get back from the first call to dm_task_set_cookie() into sebsequent calls, and you still are seeing non-watited for cookies with "dmsetup udevcookies" then that sounds to me like a problem in device-mapper, and we should check that out.

-Ben

Quoted 446 lines

This causes libdevmapper to create the device nodes on its own, and the device nodes not being handled by udev.

---
 kpartx/devmapper.c        | 53 +++++++++++++++++++++++++++++++++++------------
 kpartx/devmapper.h        |  4 ++--
 kpartx/kpartx.c           | 22 +++++++++-----------
 libmultipath/config.h     |  1 -
 libmultipath/configure.c  |  5 +++--
 libmultipath/devmapper.c  | 48 +++++++++++++++++++++++++++++++-----------
 libmultipath/devmapper.h  |  2 +-
 multipath/main.c          |  2 --
 multipathd/cli_handlers.c |  4 ++--
 9 files changed, 94 insertions(+), 47 deletions(-)
kpartx/devmapper.c +40 -13
index a3272d4..82be990 100644
@@ -14,13 +14,6 @@
 #define MAX_PREFIX_LEN 8
 #define PARAMS_SIZE 1024
 
-#ifndef LIBDM_API_COOKIE
-static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
-{
-	return 1;
-}
-#endif
-
 extern int
 dm_prereq (char * str, int x, int y, int z)
 {
@@ -60,10 +53,13 @@ dm_prereq (char * str, int x, int y, int z)
 }
 
 extern int
-dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16_t udev_flags) {
+dm_simplecmd (int task, const char *name, int no_flush, uint16_t udev_flags) {
 	int r = 0;
 	int udev_wait_flag = (task == DM_DEVICE_RESUME ||
 			      task == DM_DEVICE_REMOVE);
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+#endif
 	struct dm_task *dmt;
 
 	if (!(dmt = dm_task_create(task)))
@@ -78,10 +74,23 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 	if (no_flush)
 		dm_task_no_flush(dmt);
 
-	if (udev_wait_flag && !dm_task_set_cookie(dmt, cookie, ((udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK) | udev_flags))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags |= DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto out;
+	}
+#endif
 	r = dm_task_run(dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (udev_wait_flag) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 	out:
 	dm_task_destroy(dmt);
 	return r;
@@ -90,10 +99,14 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 extern int
 dm_addmap (int task, const char *name, const char *target,
 	   const char *params, uint64_t size, int ro, const char *uuid, int part,
-	   mode_t mode, uid_t uid, gid_t gid, uint32_t *cookie) {
+	   mode_t mode, uid_t uid, gid_t gid) {
 	int r = 0;
 	struct dm_task *dmt;
 	char *prefixed_uuid = NULL;
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+	uint16_t udev_flags = 0;
+#endif
 
 	if (!(dmt = dm_task_create (task)))
 		return 0;
@@ -128,10 +141,24 @@ dm_addmap (int task, const char *name, const char *target,
 
 	dm_task_no_open_count(dmt);
 
-	if (task == DM_DEVICE_CREATE && !dm_task_set_cookie(dmt, cookie, (udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (task == DM_DEVICE_CREATE &&
+	    !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto addout;
+	}
+#endif
 	r = dm_task_run (dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (task == DM_DEVICE_CREATE) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 addout:
 	dm_task_destroy (dmt);
 	free(prefixed_uuid);
kpartx/devmapper.h +2 -2
index 4b867df..ac1d5d9 100644
@@ -10,9 +10,9 @@
 extern int udev_sync;
 
 int dm_prereq (char *, int, int, int);
-int dm_simplecmd (int, const char *, int, uint32_t *, uint16_t);
+int dm_simplecmd (int, const char *, int, uint16_t);
 int dm_addmap (int, const char *, const char *, const char *, uint64_t,
-	       int, const char *, int, mode_t, uid_t, gid_t, uint32_t *);
+	       int, const char *, int, mode_t, uid_t, gid_t);
 int dm_map_present (char *);
 char * dm_mapname(int major, int minor);
 dev_t dm_get_first_dep(char *devname);
kpartx/kpartx.c +10 -12
index 18c1d23..d69f9af 100644
@@ -208,7 +208,6 @@ main(int argc, char **argv){
 	int hotplug = 0;
 	int loopcreated = 0;
 	struct stat buf;
-	uint32_t cookie = 0;
 
 	initpts();
 	init_crc32();
@@ -281,6 +280,8 @@ main(int argc, char **argv){
 #ifdef LIBDM_API_COOKIE
 	if (!udev_sync)
 		dm_udev_set_sync_support(0);
+	else
+		dm_udev_set_sync_support(1);
 #endif
 
 	if (dm_prereq(DM_TARGET, 0, 0, 0) && (what == ADD || what == DELETE || what == UPDATE)) {
@@ -437,7 +438,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE, partname,
-						  0, &cookie, 0)) {
+						  0, 0)) {
 					r++;
 					continue;
 				}
@@ -488,18 +489,19 @@ main(int argc, char **argv){
 				if (!dm_addmap(op, partname, DM_TARGET, params,
 					       slices[j].size, ro, uuid, j+1,
 					       buf.st_mode & 0777, buf.st_uid,
-					       buf.st_gid, &cookie)) {
+					       buf.st_gid)) {
 					fprintf(stderr, "create/reload failed on %s\n",
 						partname);
 					r++;
 				}
 				if (op == DM_DEVICE_RELOAD &&
 				    !dm_simplecmd(DM_DEVICE_RESUME, partname,
-						  1, &cookie, MPATH_UDEV_RELOAD_FLAG)) {
+						  1, MPATH_UDEV_RELOAD_FLAG)) {
 					fprintf(stderr, "resume failed on %s\n",
 						partname);
 					r++;
 				}
+
 				dm_devn(partname, &slices[j].major,
 					&slices[j].minor);
 
@@ -551,14 +553,12 @@ main(int argc, char **argv){
 					dm_addmap(op, partname, DM_TARGET, params,
 						  slices[j].size, ro, uuid, j+1,
 						  buf.st_mode & 0777,
-						  buf.st_uid, buf.st_gid,
-						  &cookie);
+						  buf.st_uid, buf.st_gid);
 
 					if (op == DM_DEVICE_RELOAD)
 						dm_simplecmd(DM_DEVICE_RESUME,
 							     partname, 1,
-							     &cookie, MPATH_UDEV_RELOAD_FLAG);
-
+							     MPATH_UDEV_RELOAD_FLAG);
 					dm_devn(partname, &slices[j].major,
 						&slices[j].minor);
 
@@ -590,7 +590,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE,
-						  partname, 1, &cookie, 0)) {
+						  partname, 1, 0)) {
 					r++;
 					continue;
 				}
@@ -616,9 +616,7 @@ main(int argc, char **argv){
 		}
 		printf("loop deleted : %s\n", device);
 	}
-#ifdef LIBDM_API_COOKIE
-	dm_udev_wait(cookie);
-#endif
+
 	dm_lib_release();
 	dm_lib_exit();
 
libmultipath/config.h +0 -1
index d304a6c..eff127e 100644
@@ -127,7 +127,6 @@ struct config {
 	uid_t uid;
 	gid_t gid;
 	mode_t mode;
-	uint32_t cookie;
 	int reassign_maps;
 	int retain_hwhandler;
 	int detect_prio;
libmultipath/configure.c +3 -2
index 2465563..3c230a1 100644
@@ -623,7 +623,8 @@ domap (struct multipath * mpp, char * params)
 	case ACT_RELOAD:
 		r = dm_addmap_reload(mpp, params);
 		if (r)
-			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias,
+						 0, MPATH_UDEV_RELOAD_FLAG);
 		break;
 
 	case ACT_RESIZE:
@@ -641,7 +642,7 @@ domap (struct multipath * mpp, char * params)
 		if (r) {
 			r = dm_addmap_reload(mpp, params);
 			if (r)
-				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG);
 		}
 		break;
 
libmultipath/devmapper.c +36 -12
index 1901052..f0b0da1 100644
@@ -216,6 +216,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 	int r = 0;
 	int udev_wait_flag = (need_sync && (task == DM_DEVICE_RESUME ||
 					    task == DM_DEVICE_REMOVE));
+	uint32_t cookie = 0;
 	struct dm_task *dmt;
 
 	if (!(dmt = dm_task_create (task)))
@@ -234,10 +235,18 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 	if (do_deferred(deferred_remove))
 		dm_task_deferred_remove(dmt);
 #endif
-	if (udev_wait_flag && !dm_task_set_cookie(dmt, &conf->cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags))
+	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags)) {
+		dm_udev_complete(cookie);
 		goto out;
+	}
 	r = dm_task_run (dmt);
 
+	if (udev_wait_flag) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			udev_wait(cookie);
+	}
 	out:
 	dm_task_destroy (dmt);
 	return r;
@@ -249,8 +258,8 @@ dm_simplecmd_flush (int task, const char *name, int needsync, uint16_t udev_flag
 }
 
 extern int
-dm_simplecmd_noflush (int task, const char *name, uint16_t udev_flags) {
-	return dm_simplecmd(task, name, 1, 1, udev_flags, 0);
+dm_simplecmd_noflush (int task, const char *name, int needsync, uint16_t udev_flags) {
+	return dm_simplecmd(task, name, 1, needsync, udev_flags, 0);
 }
 
 static int
@@ -265,6 +274,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 	int r = 0;
 	struct dm_task *dmt;
 	char *prefixed_uuid = NULL;
+	uint32_t cookie = 0;
 
 	if (!(dmt = dm_task_create (task)))
 		return 0;
@@ -305,10 +315,18 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 	dm_task_no_open_count(dmt);
 
 	if (task == DM_DEVICE_CREATE &&
-	    !dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
+	    !dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0)) {
+		dm_udev_complete(cookie);
 		goto freeout;
+	}
 	r = dm_task_run (dmt);
 
+	if (task == DM_DEVICE_CREATE) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			udev_wait(cookie);
+	}
 	freeout:
 	if (prefixed_uuid)
 		FREE(prefixed_uuid);
@@ -326,7 +344,8 @@ dm_addmap_create (struct multipath *mpp, char * params) {
 	for (ro = 0; ro <= 1; ro++) {
 		int err;
 
-		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH, mpp, params, 1, ro))
+		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH,
+			      mpp, params, 1, ro))
 			return 1;
 		/*
 		 * DM_DEVICE_CREATE is actually DM_DEV_CREATE + DM_TABLE_LOAD.
@@ -755,14 +774,14 @@ dm_suspend_and_flush_map (const char * mapname)
 	if (s)
 		queue_if_no_path = 0;
 	else
-		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 0, 0);
+		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 1, 0);
 
 	if (!dm_flush_map(mapname)) {
 		condlog(4, "multipath map %s removed", mapname);
 		return 0;
 	}
 	condlog(2, "failed to remove multipath map %s", mapname);
-	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 0);
+	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 1, 0);
 	if (queue_if_no_path)
 		s = dm_queue_if_no_path((char *)mapname, 1);
 	return 1;
@@ -1312,6 +1331,7 @@ dm_rename (char * old, char * new)
 {
 	int r = 0;
 	struct dm_task *dmt;
+	uint32_t cookie;
 
 	if (dm_rename_partmaps(old, new))
 		return r;
@@ -1327,14 +1347,18 @@ dm_rename (char * old, char * new)
 
 	dm_task_no_open_count(dmt);
 
-	if (!dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
-		goto out;
-	if (!dm_task_run(dmt))
+	if (!dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
 		goto out;
+	r = dm_task_run(dmt);
+
+	if (!r)
+		dm_udev_complete(cookie);
+	else
+		udev_wait(cookie);
 
-	r = 1;
 out:
 	dm_task_destroy(dmt);
+
 	return r;
 }
 
@@ -1399,7 +1423,7 @@ int dm_reassign_table(const char *name, char *old, char *new)
 			condlog(3, "%s: failed to reassign targets", name);
 			goto out_reload;
 		}
-		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, MPATH_UDEV_RELOAD_FLAG);
+		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, 1, MPATH_UDEV_RELOAD_FLAG);
 	}
 	r = 1;
 
libmultipath/devmapper.h +1 -1
index 5c8c50d..8188f48 100644
@@ -16,7 +16,7 @@ void dm_init(void);
 int dm_prereq (void);
 int dm_drv_version (unsigned int * version, char * str);
 int dm_simplecmd_flush (int, const char *, int, uint16_t);
-int dm_simplecmd_noflush (int, const char *, uint16_t);
+int dm_simplecmd_noflush (int, const char *, int, uint16_t);
 int dm_addmap_create (struct multipath *mpp, char *params);
 int dm_addmap_reload (struct multipath *mpp, char *params);
 int dm_map_present (const char *);
multipath/main.c +0 -2
index 1c1191a..c46a9f6 100644
@@ -675,8 +675,6 @@ main (int argc, char *argv[])
 		condlog(3, "restart multipath configuration process");
 
 out:
-	udev_wait(conf->cookie);
-
 	dm_lib_release();
 	dm_lib_exit();
 
multipathd/cli_handlers.c +2 -2
index 6abe72a..772531e 100644
@@ -839,7 +839,7 @@ cli_suspend(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: suspend (operator)", param);
@@ -861,7 +861,7 @@ cli_resume(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: resume (operator)", param);
Signature · 1 line
1.8.4.5
#48 lore raw

Re: [PATCH 43/78] Fixup device-mapper 'cookie' handling

Quoted 11 lines · Benjamin Marzinski
Quoted 10 lines · Hannes Reinecke

device-mapper has a 'cookie', which is inserted with the ioctl for modifying device-mapper devices.
It is used as a synchronization point between udev and any other applications to notify the latter when udev has finished processing the event.
Originally multipath would only use a single cookie for every transaction, and wait for that cookie at the end of the program. Which works well if you only have one transaction, but for several (like calling 'multipath') it will actually overwrite the cookie and fail to wait for earlier events.

Another thing that might be interesting to try for debugging purposes:

If you enable DM_UDEV_DISABLE_LIBRARY_FALLBACK for multipath as well as multipathd, then multipath shouldn't failback to manual device node creation. Even if multipath isn't waiting on it, udev should create the device node eventually. If you are always getting all of your device nodes when you do this, then it seems that there is a race going on. If nodes occasionally aren't getting created at all when DM_UDEV_DISABLE_LIBRARY_FALLBACK is enabled, then it would seem to point to an issue in udev.

just a thought.

Quoted 488 lines

That shouldn't be happening. Looking at the dm_task_set_cookie code

        if (*cookie) {
                if (!_get_cookie_sem(*cookie, &semid))
                        goto_bad;
        } else if (!_udev_notify_sem_create(cookie, &semid))
                goto_bad;

The first time we use that cookie, it should be zero, so a new semaphore will be created, and the id will be returned to us. Subsequent calls to dm_task_set_cookie with the same cookie (which is now non-zero) should just be using the same semaphore, allowing you you wait just one time on all the actions linked to that cookie. This should be more efficient than having to wait on each action (since udev can complete them in the background as we go on).

If there really are cookies that are not waited for, you should be able to see that by running

# dmsetup udevcookies

After running the multipath command. Cookies won't go away until someone waits for them. If there are cookies listed there, then my guess would be that something is resetting conf->cookie to 0 after our first call to dm_task_set_cookie. If there aren't, then it would appear that something else, instead of not waiting for the cookies, is causing device mapper to fail back to manual device node creation.

If you can verify that you are passing the cookie value that you get back from the first call to dm_task_set_cookie() into sebsequent calls, and you still are seeing non-watited for cookies with "dmsetup udevcookies" then that sounds to me like a problem in device-mapper, and we should check that out.

-Ben

Quoted 446 lines

This causes libdevmapper to create the device nodes on its own, and the device nodes not being handled by udev.

---
 kpartx/devmapper.c        | 53 +++++++++++++++++++++++++++++++++++------------
 kpartx/devmapper.h        |  4 ++--
 kpartx/kpartx.c           | 22 +++++++++-----------
 libmultipath/config.h     |  1 -
 libmultipath/configure.c  |  5 +++--
 libmultipath/devmapper.c  | 48 +++++++++++++++++++++++++++++++-----------
 libmultipath/devmapper.h  |  2 +-
 multipath/main.c          |  2 --
 multipathd/cli_handlers.c |  4 ++--
 9 files changed, 94 insertions(+), 47 deletions(-)
kpartx/devmapper.c +40 -13
index a3272d4..82be990 100644
@@ -14,13 +14,6 @@
 #define MAX_PREFIX_LEN 8
 #define PARAMS_SIZE 1024
 
-#ifndef LIBDM_API_COOKIE
-static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
-{
-	return 1;
-}
-#endif
-
 extern int
 dm_prereq (char * str, int x, int y, int z)
 {
@@ -60,10 +53,13 @@ dm_prereq (char * str, int x, int y, int z)
 }
 
 extern int
-dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16_t udev_flags) {
+dm_simplecmd (int task, const char *name, int no_flush, uint16_t udev_flags) {
 	int r = 0;
 	int udev_wait_flag = (task == DM_DEVICE_RESUME ||
 			      task == DM_DEVICE_REMOVE);
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+#endif
 	struct dm_task *dmt;
 
 	if (!(dmt = dm_task_create(task)))
@@ -78,10 +74,23 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 	if (no_flush)
 		dm_task_no_flush(dmt);
 
-	if (udev_wait_flag && !dm_task_set_cookie(dmt, cookie, ((udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK) | udev_flags))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags |= DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto out;
+	}
+#endif
 	r = dm_task_run(dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (udev_wait_flag) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 	out:
 	dm_task_destroy(dmt);
 	return r;
@@ -90,10 +99,14 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 extern int
 dm_addmap (int task, const char *name, const char *target,
 	   const char *params, uint64_t size, int ro, const char *uuid, int part,
-	   mode_t mode, uid_t uid, gid_t gid, uint32_t *cookie) {
+	   mode_t mode, uid_t uid, gid_t gid) {
 	int r = 0;
 	struct dm_task *dmt;
 	char *prefixed_uuid = NULL;
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+	uint16_t udev_flags = 0;
+#endif
 
 	if (!(dmt = dm_task_create (task)))
 		return 0;
@@ -128,10 +141,24 @@ dm_addmap (int task, const char *name, const char *target,
 
 	dm_task_no_open_count(dmt);
 
-	if (task == DM_DEVICE_CREATE && !dm_task_set_cookie(dmt, cookie, (udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (task == DM_DEVICE_CREATE &&
+	    !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto addout;
+	}
+#endif
 	r = dm_task_run (dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (task == DM_DEVICE_CREATE) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 addout:
 	dm_task_destroy (dmt);
 	free(prefixed_uuid);
kpartx/devmapper.h +2 -2
index 4b867df..ac1d5d9 100644
@@ -10,9 +10,9 @@
 extern int udev_sync;
 
 int dm_prereq (char *, int, int, int);
-int dm_simplecmd (int, const char *, int, uint32_t *, uint16_t);
+int dm_simplecmd (int, const char *, int, uint16_t);
 int dm_addmap (int, const char *, const char *, const char *, uint64_t,
-	       int, const char *, int, mode_t, uid_t, gid_t, uint32_t *);
+	       int, const char *, int, mode_t, uid_t, gid_t);
 int dm_map_present (char *);
 char * dm_mapname(int major, int minor);
 dev_t dm_get_first_dep(char *devname);
kpartx/kpartx.c +10 -12
index 18c1d23..d69f9af 100644
@@ -208,7 +208,6 @@ main(int argc, char **argv){
 	int hotplug = 0;
 	int loopcreated = 0;
 	struct stat buf;
-	uint32_t cookie = 0;
 
 	initpts();
 	init_crc32();
@@ -281,6 +280,8 @@ main(int argc, char **argv){
 #ifdef LIBDM_API_COOKIE
 	if (!udev_sync)
 		dm_udev_set_sync_support(0);
+	else
+		dm_udev_set_sync_support(1);
 #endif
 
 	if (dm_prereq(DM_TARGET, 0, 0, 0) && (what == ADD || what == DELETE || what == UPDATE)) {
@@ -437,7 +438,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE, partname,
-						  0, &cookie, 0)) {
+						  0, 0)) {
 					r++;
 					continue;
 				}
@@ -488,18 +489,19 @@ main(int argc, char **argv){
 				if (!dm_addmap(op, partname, DM_TARGET, params,
 					       slices[j].size, ro, uuid, j+1,
 					       buf.st_mode & 0777, buf.st_uid,
-					       buf.st_gid, &cookie)) {
+					       buf.st_gid)) {
 					fprintf(stderr, "create/reload failed on %s\n",
 						partname);
 					r++;
 				}
 				if (op == DM_DEVICE_RELOAD &&
 				    !dm_simplecmd(DM_DEVICE_RESUME, partname,
-						  1, &cookie, MPATH_UDEV_RELOAD_FLAG)) {
+						  1, MPATH_UDEV_RELOAD_FLAG)) {
 					fprintf(stderr, "resume failed on %s\n",
 						partname);
 					r++;
 				}
+
 				dm_devn(partname, &slices[j].major,
 					&slices[j].minor);
 
@@ -551,14 +553,12 @@ main(int argc, char **argv){
 					dm_addmap(op, partname, DM_TARGET, params,
 						  slices[j].size, ro, uuid, j+1,
 						  buf.st_mode & 0777,
-						  buf.st_uid, buf.st_gid,
-						  &cookie);
+						  buf.st_uid, buf.st_gid);
 
 					if (op == DM_DEVICE_RELOAD)
 						dm_simplecmd(DM_DEVICE_RESUME,
 							     partname, 1,
-							     &cookie, MPATH_UDEV_RELOAD_FLAG);
-
+							     MPATH_UDEV_RELOAD_FLAG);
 					dm_devn(partname, &slices[j].major,
 						&slices[j].minor);
 
@@ -590,7 +590,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE,
-						  partname, 1, &cookie, 0)) {
+						  partname, 1, 0)) {
 					r++;
 					continue;
 				}
@@ -616,9 +616,7 @@ main(int argc, char **argv){
 		}
 		printf("loop deleted : %s\n", device);
 	}
-#ifdef LIBDM_API_COOKIE
-	dm_udev_wait(cookie);
-#endif
+
 	dm_lib_release();
 	dm_lib_exit();
 
libmultipath/config.h +0 -1
index d304a6c..eff127e 100644
@@ -127,7 +127,6 @@ struct config {
 	uid_t uid;
 	gid_t gid;
 	mode_t mode;
-	uint32_t cookie;
 	int reassign_maps;
 	int retain_hwhandler;
 	int detect_prio;
libmultipath/configure.c +3 -2
index 2465563..3c230a1 100644
@@ -623,7 +623,8 @@ domap (struct multipath * mpp, char * params)
 	case ACT_RELOAD:
 		r = dm_addmap_reload(mpp, params);
 		if (r)
-			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias,
+						 0, MPATH_UDEV_RELOAD_FLAG);
 		break;
 
 	case ACT_RESIZE:
@@ -641,7 +642,7 @@ domap (struct multipath * mpp, char * params)
 		if (r) {
 			r = dm_addmap_reload(mpp, params);
 			if (r)
-				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG);
 		}
 		break;
 
libmultipath/devmapper.c +36 -12
index 1901052..f0b0da1 100644
@@ -216,6 +216,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 	int r = 0;
 	int udev_wait_flag = (need_sync && (task == DM_DEVICE_RESUME ||
 					    task == DM_DEVICE_REMOVE));
+	uint32_t cookie = 0;
 	struct dm_task *dmt;
 
 	if (!(dmt = dm_task_create (task)))
@@ -234,10 +235,18 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 	if (do_deferred(deferred_remove))
 		dm_task_deferred_remove(dmt);
 #endif
-	if (udev_wait_flag && !dm_task_set_cookie(dmt, &conf->cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags))
+	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags)) {
+		dm_udev_complete(cookie);
 		goto out;
+	}
 	r = dm_task_run (dmt);
 
+	if (udev_wait_flag) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			udev_wait(cookie);
+	}
 	out:
 	dm_task_destroy (dmt);
 	return r;
@@ -249,8 +258,8 @@ dm_simplecmd_flush (int task, const char *name, int needsync, uint16_t udev_flag
 }
 
 extern int
-dm_simplecmd_noflush (int task, const char *name, uint16_t udev_flags) {
-	return dm_simplecmd(task, name, 1, 1, udev_flags, 0);
+dm_simplecmd_noflush (int task, const char *name, int needsync, uint16_t udev_flags) {
+	return dm_simplecmd(task, name, 1, needsync, udev_flags, 0);
 }
 
 static int
@@ -265,6 +274,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 	int r = 0;
 	struct dm_task *dmt;
 	char *prefixed_uuid = NULL;
+	uint32_t cookie = 0;
 
 	if (!(dmt = dm_task_create (task)))
 		return 0;
@@ -305,10 +315,18 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 	dm_task_no_open_count(dmt);
 
 	if (task == DM_DEVICE_CREATE &&
-	    !dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
+	    !dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0)) {
+		dm_udev_complete(cookie);
 		goto freeout;
+	}
 	r = dm_task_run (dmt);
 
+	if (task == DM_DEVICE_CREATE) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			udev_wait(cookie);
+	}
 	freeout:
 	if (prefixed_uuid)
 		FREE(prefixed_uuid);
@@ -326,7 +344,8 @@ dm_addmap_create (struct multipath *mpp, char * params) {
 	for (ro = 0; ro <= 1; ro++) {
 		int err;
 
-		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH, mpp, params, 1, ro))
+		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH,
+			      mpp, params, 1, ro))
 			return 1;
 		/*
 		 * DM_DEVICE_CREATE is actually DM_DEV_CREATE + DM_TABLE_LOAD.
@@ -755,14 +774,14 @@ dm_suspend_and_flush_map (const char * mapname)
 	if (s)
 		queue_if_no_path = 0;
 	else
-		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 0, 0);
+		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 1, 0);
 
 	if (!dm_flush_map(mapname)) {
 		condlog(4, "multipath map %s removed", mapname);
 		return 0;
 	}
 	condlog(2, "failed to remove multipath map %s", mapname);
-	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 0);
+	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 1, 0);
 	if (queue_if_no_path)
 		s = dm_queue_if_no_path((char *)mapname, 1);
 	return 1;
@@ -1312,6 +1331,7 @@ dm_rename (char * old, char * new)
 {
 	int r = 0;
 	struct dm_task *dmt;
+	uint32_t cookie;
 
 	if (dm_rename_partmaps(old, new))
 		return r;
@@ -1327,14 +1347,18 @@ dm_rename (char * old, char * new)
 
 	dm_task_no_open_count(dmt);
 
-	if (!dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
-		goto out;
-	if (!dm_task_run(dmt))
+	if (!dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
 		goto out;
+	r = dm_task_run(dmt);
+
+	if (!r)
+		dm_udev_complete(cookie);
+	else
+		udev_wait(cookie);
 
-	r = 1;
 out:
 	dm_task_destroy(dmt);
+
 	return r;
 }
 
@@ -1399,7 +1423,7 @@ int dm_reassign_table(const char *name, char *old, char *new)
 			condlog(3, "%s: failed to reassign targets", name);
 			goto out_reload;
 		}
-		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, MPATH_UDEV_RELOAD_FLAG);
+		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, 1, MPATH_UDEV_RELOAD_FLAG);
 	}
 	r = 1;
 
libmultipath/devmapper.h +1 -1
index 5c8c50d..8188f48 100644
@@ -16,7 +16,7 @@ void dm_init(void);
 int dm_prereq (void);
 int dm_drv_version (unsigned int * version, char * str);
 int dm_simplecmd_flush (int, const char *, int, uint16_t);
-int dm_simplecmd_noflush (int, const char *, uint16_t);
+int dm_simplecmd_noflush (int, const char *, int, uint16_t);
 int dm_addmap_create (struct multipath *mpp, char *params);
 int dm_addmap_reload (struct multipath *mpp, char *params);
 int dm_map_present (const char *);
multipath/main.c +0 -2
index 1c1191a..c46a9f6 100644
@@ -675,8 +675,6 @@ main (int argc, char *argv[])
 		condlog(3, "restart multipath configuration process");
 
 out:
-	udev_wait(conf->cookie);
-
 	dm_lib_release();
 	dm_lib_exit();
 
multipathd/cli_handlers.c +2 -2
index 6abe72a..772531e 100644
@@ -839,7 +839,7 @@ cli_suspend(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: suspend (operator)", param);
@@ -861,7 +861,7 @@ cli_resume(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: resume (operator)", param);
Signature · 1 line
1.8.4.5

--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

#49 lore raw

Re: [PATCH 43/78] Fixup device-mapper 'cookie' handling

In reply to Benjamin Marzinski
Quoted 40 lines · Benjamin Marzinski
Quoted 10 lines · Hannes Reinecke

device-mapper has a 'cookie', which is inserted with the ioctl for modifying device-mapper devices.
It is used as a synchronization point between udev and any other applications to notify the latter when udev has finished processing the event.
Originally multipath would only use a single cookie for every transaction, and wait for that cookie at the end of the program. Which works well if you only have one transaction, but for several (like calling 'multipath') it will actually overwrite the cookie and fail to wait for earlier events.

That shouldn't be happening. Looking at the dm_task_set_cookie code

        if (*cookie) {
                if (!_get_cookie_sem(*cookie, &semid))
                        goto_bad;
        } else if (!_udev_notify_sem_create(cookie, &semid))
                goto_bad;

The first time we use that cookie, it should be zero, so a new semaphore will be created, and the id will be returned to us. Subsequent calls to dm_task_set_cookie with the same cookie (which is now non-zero) should just be using the same semaphore, allowing you you wait just one time on all the actions linked to that cookie. This should be more efficient than having to wait on each action (since udev can complete them in the background as we go on).

If there really are cookies that are not waited for, you should be able to see that by running

# dmsetup udevcookies

After running the multipath command. Cookies won't go away until someone waits for them. If there are cookies listed there, then my guess would be that something is resetting conf->cookie to 0 after our first call to dm_task_set_cookie. If there aren't, then it would appear that something else, instead of not waiting for the cookies, is causing device mapper to fail back to manual device node creation.

Quoted 6 lines

If you can verify that you are passing the cookie value that you get back from the first call to dm_task_set_cookie() into sebsequent calls, and you still are seeing non-watited for cookies with "dmsetup udevcookies" then that sounds to me like a problem in device-mapper, and we should check that out.

Hmm. It _might_ have been caused by multipathd not properly synchronized with udev (the original service files didn't have a dependency on udev), hence the call to dm_udev_set_sync_support() would cause device-mapper to switch off the udev fallback.

Speaking of nasty side-effects ...
Can't we have a proper warning in dm_udev_set_sync_support(), alerting us that the system will not behave as expected?

But with that resolved it might be that indeed the patch is not required. I'll check.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#50 lore raw

PATCH 44/78 multipath: do not print state 'orphan' for option '-l'

In reply to Hannes Reinecke

When called with '-l' multipath would mistakenly print a state of 'orphan' instead off 'running' of 'offline'.
This is a regression introduced by commit 7023d320.

---
 libmultipath/print.c | 2 --
 1 file changed, 2 deletions(-)
libmultipath/print.c +0 -2
index 39b4f98..5d63ed3 100644
@@ -322,8 +322,6 @@ snprint_offline (char * buff, size_t len, struct path * pp)
 		return snprintf(buff, len, "unknown");
 	else if (pp->offline)
 		return snprintf(buff, len, "offline");
-	else if (!pp->mpp)
-		return snprintf(buff, len, "orphan");
 	else
 		return snprintf(buff, len, "running");
 }
Signature · 1 line
1.8.4.5
#51 lore raw

PATCH 45/78 Return error when receiving CLI packet

In reply to Hannes Reinecke

When receiving a packet from the CLI we should be returning an error code.

---
 libmpathpersist/mpath_updatepr.c | 20 ++++++++++++--------
 libmultipath/configure.c         |  2 +-
 libmultipath/defaults.h          |  1 +
 libmultipath/uxsock.c            | 30 +++++++++++++++++++-----------
 libmultipath/uxsock.h            |  4 ++--
 multipathd/main.c                |  4 ++--
 multipathd/uxclnt.c              | 26 +++++++++++++++++---------
 multipathd/uxclnt.h              |  2 +-
 multipathd/uxlsnr.c              |  3 ++-
 9 files changed, 57 insertions(+), 35 deletions(-)
libmpathpersist/mpath_updatepr.c +12 -8
index 8597d40..dce580f 100644
@@ -35,15 +35,19 @@ int update_prflag(char * arg1, char * arg2, int noisy)
 	snprintf(str,sizeof(str),"map %s %s", arg1, arg2);
 	condlog (2, "%s: pr flag message=%s", arg1, str);
 	send_packet(fd, str, strlen(str) + 1);
-	recv_packet(fd, &reply, &len);
-
-	condlog (2, "%s: message=%s reply=%s", arg1, str, reply);
-	if (!reply || strncmp(reply,"ok", 2) == 0)
-		ret = -1;
-	else if (strncmp(reply, "fail", 4) == 0)
+	ret = recv_packet(fd, &reply, &len, DEFAULT_UXSOCK_TIMEOUT);
+	if (ret < 0) {
+		condlog(2, "%s: message=%s error=%d", arg1, str, -ret);
 		ret = -2;
-	else{
-		ret = atoi(reply);
+	} else {
+		condlog (2, "%s: message=%s reply=%s", arg1, str, reply);
+		if (!reply || strncmp(reply,"ok", 2) == 0)
+			ret = -1;
+		else if (strncmp(reply, "fail", 4) == 0)
+			ret = -2;
+		else{
+			ret = atoi(reply);
+		}
 	}
 
 	free(reply);
libmultipath/configure.c +1 -1
index 3c230a1..720d074 100644
@@ -714,7 +714,7 @@ int check_daemon(void)
 
 	if (send_packet(fd, "show daemon", 12) != 0)
 		goto out;
-	if (recv_packet(fd, &reply, &len) != 0)
+	if (recv_packet(fd, &reply, &len, DEFAULT_UXSOCK_TIMEOUT) != 0)
 		goto out;
 
 	if (strstr(reply, "shutdown"))
libmultipath/defaults.h +1 -0
index 878da14..8902f40 100644
@@ -20,6 +20,7 @@
 #define DEFAULT_DEFERRED_REMOVE DEFERRED_REMOVE_OFF
 #define DEFAULT_DELAY_CHECKS DELAY_CHECKS_OFF
 #define DEFAULT_UEVENT_STACKSIZE 256
+#define DEFAULT_UXSOCK_TIMEOUT  1000
 
 #define DEFAULT_CHECKINT	5
 #define MAX_CHECKINT(a)		(a << 2)
libmultipath/uxsock.c +19 -11
index aff7a62..af0798c 100644
@@ -128,7 +128,7 @@ size_t write_all(int fd, const void *buf, size_t len)
 /*
  * keep reading until its all read
  */
-size_t read_all(int fd, void *buf, size_t len)
+ssize_t read_all(int fd, void *buf, size_t len, unsigned int timeout)
 {
 	size_t total = 0;
 	ssize_t n;
@@ -138,21 +138,20 @@ size_t read_all(int fd, void *buf, size_t len)
 	while (len) {
 		pfd.fd = fd;
 		pfd.events = POLLIN;
-		ret = poll(&pfd, 1, 1000);
+		ret = poll(&pfd, 1, timeout);
 		if (!ret) {
-			errno = ETIMEDOUT;
-			return total;
+			return -ETIMEDOUT;
 		} else if (ret < 0) {
 			if (errno == EINTR)
 				continue;
-			return total;
+			return -errno;
 		} else if (!pfd.revents & POLLIN)
 			continue;
 		n = read(fd, buf, len);
 		if (n < 0) {
 			if ((errno == EINTR) || (errno == EAGAIN))
 				continue;
-			return total;
+			return -errno;
 		}
 		if (!n)
 			return total;
@@ -190,12 +189,20 @@ int send_packet(int fd, const char *buf, size_t len)
 /*
  * receive a packet in length prefix format
  */
-int recv_packet(int fd, char **buf, size_t *len)
+int recv_packet(int fd, char **buf, size_t *len, unsigned int timeout)
 {
-	if (read_all(fd, len, sizeof(*len)) != sizeof(*len)) {
+	ssize_t ret;
+
+	ret = read_all(fd, len, sizeof(*len), timeout);
+	if (ret < 0) {
 		(*buf) = NULL;
 		*len = 0;
-		return -1;
+		return ret;
+	}
+	if (ret < sizeof(*len)) {
+		(*buf) = NULL;
+		*len = 0;
+		return -EIO;
 	}
 	if (len == 0) {
 		(*buf) = NULL;
@@ -204,11 +211,12 @@ int recv_packet(int fd, char **buf, size_t *len)
 	(*buf) = MALLOC(*len);
 	if (!*buf)
 		return -1;
-	if (read_all(fd, *buf, *len) != *len) {
+	ret = read_all(fd, *buf, *len, timeout);
+	if (ret != *len) {
 		FREE(*buf);
 		(*buf) = NULL;
 		*len = 0;
-		return -1;
+		return ret < 0 ? ret : -EIO;
 	}
 	return 0;
 }
libmultipath/uxsock.h +2 -2
index fd82552..94af8d8 100644
@@ -2,6 +2,6 @@
 int ux_socket_connect(const char *name);
 int ux_socket_listen(const char *name);
 int send_packet(int fd, const char *buf, size_t len);
-int recv_packet(int fd, char **buf, size_t *len);
+int recv_packet(int fd, char **buf, size_t *len, unsigned int timeout);
 size_t write_all(int fd, const void *buf, size_t len);
-size_t read_all(int fd, void *buf, size_t len);
+ssize_t read_all(int fd, void *buf, size_t len, unsigned int timeout);
multipathd/main.c +2 -2
index ffe4326..9db2e55 100644
@@ -2023,7 +2023,7 @@ main (int argc, char *argv[])
 			logsink = -1;
 			break;
 		case 'k':
-			uxclnt(optarg);
+			uxclnt(optarg, DEFAULT_UXSOCK_TIMEOUT);
 			exit(0);
 		case 'B':
 			conf->bindings_read_only = 1;
@@ -2045,7 +2045,7 @@ main (int argc, char *argv[])
 			optind++;
 		}
 		c += snprintf(c, s + CMDSIZE - c, "\n");
-		uxclnt(s);
+		uxclnt(s, DEFAULT_UXSOCK_TIMEOUT);
 		exit(0);
 	}
 
multipathd/uxclnt.c +17 -9
index e86be21..536579f 100644
@@ -9,6 +9,7 @@
 #include <unistd.h>
 #include <stdarg.h>
 #include <fcntl.h>
+#include <errno.h>
 #include <sys/ioctl.h>
 #include <sys/types.h>
 #include <sys/socket.h>
@@ -63,10 +64,11 @@ static int need_quit(char *str, size_t len)
 /*
  * process the client
  */
-static void process(int fd)
+static void process(int fd, unsigned int timeout)
 {
 	char *line;
 	char *reply;
+	int ret;
 
 	cli_init();
 	rl_readline_name = "multipathd";
@@ -84,7 +86,8 @@ static void process(int fd)
 			break;
 
 		if (send_packet(fd, line, llen + 1) != 0) break;
-		if (recv_packet(fd, &reply, &len) != 0) break;
+		ret = recv_packet(fd, &reply, &len, timeout);
+		if (ret != 0) break;
 
 		print_reply(reply);
 
@@ -96,18 +99,23 @@ static void process(int fd)
 	}
 }
 
-static void process_req(int fd, char * inbuf)
+static void process_req(int fd, char * inbuf, unsigned int timeout)
 {
 	char *reply;
 	size_t len;
+	int ret;
 
 	if (send_packet(fd, inbuf, strlen(inbuf) + 1) != 0) {
 		printf("cannot send packet\n");
 		return;
 	}
-	if (recv_packet(fd, &reply, &len) != 0)
-		printf("error receiving packet\n");
-	else {
+	ret = recv_packet(fd, &reply, &len, timeout);
+	if (ret < 0) {
+		if (ret == -ETIMEDOUT)
+			printf("timeout receiving packet\n");
+		else
+			printf("error %d receiving packet\n", ret);
+	} else {
 		printf("%s", reply);
 		FREE(reply);
 	}
@@ -116,7 +124,7 @@ static void process_req(int fd, char * inbuf)
 /*
  * entry point
  */
-int uxclnt(char * inbuf)
+int uxclnt(char * inbuf, unsigned int timeout)
 {
 	int fd;
 
@@ -125,9 +133,9 @@ int uxclnt(char * inbuf)
 		exit(1);
 
 	if (inbuf)
-		process_req(fd, inbuf);
+		process_req(fd, inbuf, timeout);
 	else
-		process(fd);
+		process(fd, timeout);
 
 	return 0;
 }
multipathd/uxclnt.h +1 -1
index 0667a24..8e2cdce 100644
@@ -1 +1 @@
-int uxclnt(char * inbuf);
+int uxclnt(char * inbuf, unsigned int timeout);
multipathd/uxlsnr.c +2 -1
index ed8e012..624d4eb 100644
@@ -158,7 +158,8 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 			struct client *next = c->next;
 
 			if (polls[i].revents & POLLIN) {
-				if (recv_packet(c->fd, &inbuf, &len) != 0) {
+				if (recv_packet(c->fd, &inbuf, &len,
+						DEFAULT_UXSOCK_TIMEOUT) != 0) {
 					dead_client(c);
 				} else {
 					inbuf[len - 1] = 0;
Signature · 1 line
1.8.4.5
#52 lore raw

PATCH 46/78 Implement 'uxsock_timeout' keyword

In reply to Hannes Reinecke

On larger systems the default CLI receive timeout of 1 second might be too small, and the CLI commands might fail before they can be processed. For these cases I've implemented a new keywork 'uxsock_timeout' which can be increased to avoid these issues.

---
 libmultipath/config.c      |  1 +
 libmultipath/config.h      |  1 +
 libmultipath/configure.c   |  2 +-
 libmultipath/dict.c        | 30 ++++++++++++++++++++++++++++++
 multipath/multipath.conf.5 |  9 +++++++++
 multipathd/main.c          |  8 ++++++--
 6 files changed, 48 insertions(+), 3 deletions(-)
libmultipath/config.c +1 -0
index 1007f32..c788cf6 100644
@@ -617,6 +617,7 @@ load_config (char * file, struct udev *udev)
 	conf->partition_delim = NULL;
 	conf->processed_main_config = 0;
 	conf->find_multipaths = DEFAULT_FIND_MULTIPATHS;
+	conf->uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
 	conf->uid_attribute = set_default(DEFAULT_UID_ATTRIBUTE);
 
 	/*
libmultipath/config.h +1 -0
index eff127e..0149ad3 100644
@@ -135,6 +135,7 @@ struct config {
 	int processed_main_config;
 	int delay_watch_checks;
 	int delay_wait_checks;
+	int uxsock_timeout;
 	unsigned int version[3];
 
 	char * dev;
libmultipath/configure.c +1 -1
index 720d074..2c10c22 100644
@@ -714,7 +714,7 @@ int check_daemon(void)
 
 	if (send_packet(fd, "show daemon", 12) != 0)
 		goto out;
-	if (recv_packet(fd, &reply, &len, DEFAULT_UXSOCK_TIMEOUT) != 0)
+	if (recv_packet(fd, &reply, &len, conf->uxsock_timeout) != 0)
 		goto out;
 
 	if (strstr(reply, "shutdown"))
libmultipath/dict.c +30 -0
index 35a1cf2..5c2da43 100644
@@ -1034,6 +1034,26 @@ declare_hw_snprint(delay_wait_checks, print_delay_checks)
 declare_mp_handler(delay_wait_checks, set_delay_checks)
 declare_mp_snprint(delay_wait_checks, print_delay_checks)
 
+static int
+def_uxsock_timeout_handler(vector strvec)
+{
+	unsigned int uxsock_timeout;
+	char *buff;
+
+	buff = set_value(strvec);
+	if (!buff)
+		return 1;
+
+	if (sscanf(buff, "%u", &uxsock_timeout) == 1 &&
+	    uxsock_timeout > DEFAULT_UXSOCK_TIMEOUT)
+		conf->uxsock_timeout = uxsock_timeout;
+	else
+		conf->uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
+
+	free(buff);
+	return 0;
+}
+
 /*
  * blacklist block handlers
  */
@@ -1115,6 +1135,15 @@ declare_ble_handler(blist_property)
 declare_ble_handler(elist_property)
 
 static int
+snprint_def_uxsock_timeout(char * buff, int len, void * data)
+{
+	if (conf->uxsock_timeout == DEFAULT_UXSOCK_TIMEOUT)
+		return 0;
+
+	return snprintf(buff, len, "%u", conf->uxsock_timeout);
+}
+
+static int
 snprint_ble_simple (char * buff, int len, void * data)
 {
 	struct blentry * ble = (struct blentry *)data;
@@ -1335,6 +1364,7 @@ init_keywords(void)
 	install_keyword("delay_watch_checks", &def_delay_watch_checks_handler, &snprint_def_delay_watch_checks);
 	install_keyword("delay_wait_checks", &def_delay_wait_checks_handler, &snprint_def_delay_wait_checks);
 	install_keyword("find_multipaths", &def_find_multipaths_handler, &snprint_def_find_multipaths);
+	install_keyword("uxsock_timeout", &def_uxsock_timeout_handler, &snprint_def_uxsock_timeout);
 	__deprecated install_keyword("default_selector", &def_selector_handler, NULL);
 	__deprecated install_keyword("default_path_grouping_policy", &def_pgpolicy_handler, NULL);
 	__deprecated install_keyword("default_uid_attribute", &def_uid_attribute_handler, NULL);
multipath/multipath.conf.5 +9 -0
index d60dc2e..7e17f62 100644
@@ -480,6 +480,15 @@ used until it has passed
 .I delay_wait_checks
 checks. Default is
 .I no
+.TP
+.B uxsock_timeout
+CLI receive timeout in milliseconds. For larger systems CLI commands
+might timeout before the multipathd lock is released and the CLI command
+can be processed. This will result in errors like
+'timeout receiving packet' to be returned from CLI commands.
+In these cases it is recommended to increase the CLI timeout to avoid
+those issues. The default is
+.I 1000
 .
 .SH "blacklist section"
 The
multipathd/main.c +6 -2
index 9db2e55..e197d59 100644
@@ -2023,7 +2023,9 @@ main (int argc, char *argv[])
 			logsink = -1;
 			break;
 		case 'k':
-			uxclnt(optarg, DEFAULT_UXSOCK_TIMEOUT);
+			if (load_config(DEFAULT_CONFIGFILE, udev_new()))
+				exit(1);
+			uxclnt(optarg, conf->uxsock_timeout);
 			exit(0);
 		case 'B':
 			conf->bindings_read_only = 1;
@@ -2037,6 +2039,8 @@ main (int argc, char *argv[])
 		char * s = cmd;
 		char * c = s;
 
+		if (load_config(DEFAULT_CONFIGFILE, udev_new()))
+			exit(1);
 		while (optind < argc) {
 			if (strchr(argv[optind], ' '))
 				c += snprintf(c, s + CMDSIZE - c, "\"%s\" ", argv[optind]);
@@ -2045,7 +2049,7 @@ main (int argc, char *argv[])
 			optind++;
 		}
 		c += snprintf(c, s + CMDSIZE - c, "\n");
-		uxclnt(s, DEFAULT_UXSOCK_TIMEOUT);
+		uxclnt(s, conf->uxsock_timeout);
 		exit(0);
 	}
 
Signature · 1 line
1.8.4.5
#53 lore raw

PATCH 47/78 Do not print empty multipaths section

In reply to Hannes Reinecke

The CLI command 'show config' or the 'multipath -t' command should not print an empty 'multipaths' section.

---
 multipath/main.c          | 11 +++++++----
 multipathd/cli_handlers.c |  9 ++++++---
 2 files changed, 13 insertions(+), 7 deletions(-)
multipath/main.c +7 -4
index c46a9f6..ebca4dc 100644
@@ -436,10 +436,13 @@ dump_config (void)
 			reply = REALLOC(reply, maxlen *= 2);
 			continue;
 		}
-		c += snprint_mptable(c, reply + maxlen - c, conf->mptable);
-		again = ((c - reply) == maxlen);
-		if (again)
-			reply = REALLOC(reply, maxlen *= 2);
+		if (VECTOR_SIZE(conf->mptable) > 0) {
+			c += snprint_mptable(c, reply + maxlen - c,
+					     conf->mptable);
+			again = ((c - reply) == maxlen);
+			if (again)
+				reply = REALLOC(reply, maxlen *= 2);
+		}
 	}
 
 	printf("%s", reply);
multipathd/cli_handlers.c +6 -3
index 772531e..dc96c45 100644
@@ -229,9 +229,12 @@ show_config (char ** r, int * len)
 			maxlen *= 2;
 			continue;
 		}
-		c += snprint_mptable(c, reply + maxlen - c, conf->mptable);
-		again = ((c - reply) == maxlen);
-		REALLOC_REPLY(reply, again, maxlen);
+		if (VECTOR_SIZE(conf->mptable) > 0) {
+			c += snprint_mptable(c, reply + maxlen - c,
+					     conf->mptable);
+			again = ((c - reply) == maxlen);
+			REALLOC_REPLY(reply, again, maxlen);
+		}
 	}
 	*r = reply;
 	*len = (int)(c - reply + 1);
Signature · 1 line
1.8.4.5
#54 lore raw

PATCH 48/78 multipathd: reload map if reinstate failed

In reply to Hannes Reinecke

The kernel might fail the 'reinstate' device-mapper message if the path is disabled. In these cases we need to reload the map to give device-mapper a chance to add the correct devices to the table.

---
 multipathd/main.c | 30 +++++++++++++++++++++++-------
 1 file changed, 23 insertions(+), 7 deletions(-)
multipathd/main.c +23 -7
index e197d59..2eade36 100644
@@ -965,19 +965,23 @@ fail_path (struct path * pp, int del_active)
 /*
  * caller must have locked the path list before calling that function
  */
-static void
+static int
 reinstate_path (struct path * pp, int add_active)
 {
+	int ret = 0;
+
 	if (!pp->mpp)
-		return;
+		return 0;
 
-	if (dm_reinstate_path(pp->mpp->alias, pp->dev_t))
+	if (dm_reinstate_path(pp->mpp->alias, pp->dev_t)) {
 		condlog(0, "%s: reinstate failed", pp->dev_t);
-	else {
+		ret = 1;
+	} else {
 		condlog(2, "%s: reinstated", pp->dev_t);
 		if (add_active)
 			update_queue_mode_add_path(pp->mpp);
 	}
+	return ret;
 }
 
 static void
@@ -1132,6 +1136,7 @@ check_path (struct vectors * vecs, struct path * pp)
 	int newstate;
 	int new_path_up = 0;
 	int chkr_new_path_up = 0;
+	int add_active;
 	int oldchkrstate = pp->chkrstate;
 
 	if (pp->initialized && !pp->mpp)
@@ -1259,11 +1264,17 @@ check_path (struct vectors * vecs, struct path * pp)
 		    oldstate != PATH_GHOST) {
 			if (pp->mpp->delay_watch_checks > 0)
 				pp->watch_checks = pp->mpp->delay_watch_checks;
-			reinstate_path(pp, 1);
+			add_active = 1;
 		} else {
 			if (pp->watch_checks > 0)
 				pp->watch_checks--;
-			reinstate_path(pp, 0);
+			add_active = 0;
+		}
+		if (reinstate_path(pp, add_active)) {
+			condlog(3, "%s: reload map", pp->dev);
+			ev_add_path(pp, vecs);
+			pp->tick = 1;
+			return 0;
 		}
 		new_path_up = 1;
 
@@ -1281,7 +1292,12 @@ check_path (struct vectors * vecs, struct path * pp)
 		if (pp->dmstate == PSTATE_FAILED ||
 		    pp->dmstate == PSTATE_UNDEF) {
 			/* Clear IO errors */
-			reinstate_path(pp, 0);
+			if (reinstate_path(pp, 0)) {
+				condlog(3, "%s: reload map", pp->dev);
+				ev_add_path(pp, vecs);
+				pp->tick = 1;
+				return 0;
+			}
 		} else {
 			LOG_MSG(4, checker_message(&pp->checker));
 			if (pp->checkint != conf->max_checkint) {
Signature · 1 line
1.8.4.5
#55 lore raw

PATCH 49/78 multipathd: do not remove paths without uevents

In reply to Hannes Reinecke

multipathd should not remove any paths without the corresponding uevent. Otherwise the daemon will never be able to reinstate that path as it wouldn't get any uevents.

---
 multipathd/main.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
multipathd/main.c +8 -5
index 2eade36..0608f06 100644
@@ -1152,11 +1152,14 @@ check_path (struct vectors * vecs, struct path * pp)
 	pp->tick = conf->checkint;
 
 	newstate = path_offline(pp);
-	if (newstate == PATH_REMOVED) {
-		condlog(2, "%s: remove path (checker)", pp->dev);
-		ev_remove_path(pp, vecs);
-		return 0;
-	}
+	/*
+	 * Wait for uevent for removed paths;
+	 * some LLDDs like zfcp keep paths unavailable
+	 * without sending uevents.
+	 */
+	if (newstate == PATH_REMOVED)
+		newstate = PATH_DOWN;
+
 	if (newstate == PATH_UP)
 		newstate = get_state(pp, 1);
 	else
Signature · 1 line
1.8.4.5
#56 lore raw

PATCH 50/78 Allow zero-sized devices during configuration

In reply to Hannes Reinecke

A size of '0' doesn't indicate an invalid device; other paths might end up with a correct size.

---
 libmultipath/configure.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
libmultipath/configure.c +6 -5
index 2c10c22..ddbd3ed 100644
@@ -768,8 +768,8 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			continue;
 
 		/* 3. if path has disappeared */
-		if (!pp1->size) {
-			orphan_path(pp1, "invalid size");
+		if (pp1->state == PATH_REMOVED) {
+			orphan_path(pp1, "path removed");
 			continue;
 		}
 
@@ -806,10 +806,11 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			if (strcmp(pp1->wwid, pp2->wwid))
 				continue;
 
-			if (!pp2->size)
-				continue;
+			if (!mpp->size && pp2->size)
+				mpp->size = pp2->size;
 
-			if (pp2->size != mpp->size) {
+			if (mpp->size && pp2->size &&
+			    pp2->size != mpp->size) {
 				/*
 				 * ouch, avoid feeding that to the DM
 				 */
Signature · 1 line
1.8.4.5
#57 lore raw

Re: [PATCH 50/78] Allow zero-sized devices during configuration

Quoted 2 lines · Hannes Reinecke

A size of '0' doesn't indicate an invalid device; other paths might end up with a correct size.

don't we also need to change this is ev_add_path, then?

-Ben

Quoted 38 lines
---
 libmultipath/configure.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
libmultipath/configure.c +6 -5
index 2c10c22..ddbd3ed 100644
@@ -768,8 +768,8 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			continue;
 
 		/* 3. if path has disappeared */
-		if (!pp1->size) {
-			orphan_path(pp1, "invalid size");
+		if (pp1->state == PATH_REMOVED) {
+			orphan_path(pp1, "path removed");
 			continue;
 		}
 
@@ -806,10 +806,11 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			if (strcmp(pp1->wwid, pp2->wwid))
 				continue;
 
-			if (!pp2->size)
-				continue;
+			if (!mpp->size && pp2->size)
+				mpp->size = pp2->size;
 
-			if (pp2->size != mpp->size) {
+			if (mpp->size && pp2->size &&
+			    pp2->size != mpp->size) {
 				/*
 				 * ouch, avoid feeding that to the DM
 				 */
Signature · 1 line
1.8.4.5
#58 lore raw

Re: [PATCH 50/78] Allow zero-sized devices during configuration

Quoted 6 lines · Benjamin Marzinski
Quoted 2 lines · Hannes Reinecke

A size of '0' doesn't indicate an invalid device; other paths might end up with a correct size.

don't we also need to change this is ev_add_path, then?

Probably. I'll check.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#59 lore raw

PATCH 51/78 Rework uev_add_path()

In reply to Hannes Reinecke

Rework uev_add_path() to handle failed and blacklisted paths properly.

---
 libmultipath/discovery.c   | 34 +++++++++++++++++++++++
 libmultipath/discovery.h   |  2 ++
 libmultipath/structs_vec.c |  3 +++
 multipathd/main.c          | 67 +++++++++++++++++++++++++++++-----------------
 4 files changed, 82 insertions(+), 24 deletions(-)
libmultipath/discovery.c +34 -0
index 6ba14ac..b1db00f 100644
@@ -31,6 +31,40 @@
 #include "defaults.h"
 
 int
+alloc_path_with_pathinfo (vector hwtable, struct udev_device *udevice,
+			  int flag, struct path **pp_ptr)
+{
+	int err = PATHINFO_FAILED;
+	struct path * pp;
+	const char * devname;
+
+	if (pp_ptr)
+		*pp_ptr = NULL;
+
+	devname = udev_device_get_sysname(udevice);
+	if (!devname)
+		return PATHINFO_FAILED;
+
+	pp = alloc_path();
+
+	if (!pp)
+		return PATHINFO_FAILED;
+
+	if (safe_sprintf(pp->dev, "%s", devname)) {
+		condlog(0, "pp->dev too small");
+	} else {
+		pp->udev = udev_device_ref(udevice);
+		err = pathinfo(pp, hwtable, flag | DI_BLACKLIST);
+	}
+
+	if (err)
+		free_path(pp);
+	else if (pp_ptr)
+		*pp_ptr = pp;
+	return err;
+}
+
+int
 store_pathinfo (vector pathvec, vector hwtable, struct udev_device *udevice,
 		int flag, struct path **pp_ptr)
 {
libmultipath/discovery.h +2 -0
index 7e5680e..da7652c 100644
@@ -36,6 +36,8 @@ int do_tur (char *);
 int path_offline (struct path *);
 int get_state (struct path * pp, int daemon);
 int pathinfo (struct path *, vector hwtable, int mask);
+int alloc_path_with_pathinfo (vector hwtable, struct udev_device *udevice,
+			      int flag, struct path **pp_ptr);
 int store_pathinfo (vector pathvec, vector hwtable,
 		    struct udev_device *udevice, int flag,
 		    struct path **pp_ptr);
libmultipath/structs_vec.c +3 -0
index 6d2d45e..d9a731a 100644
@@ -459,6 +459,9 @@ add_map_with_path (struct vectors * vecs,
 {
 	struct multipath * mpp;
 
+	if (!strlen(pp->wwid))
+		return NULL;
+
 	if (!(mpp = alloc_multipath()))
 		return NULL;
 
multipathd/main.c +43 -24

94 lines not on this page · raw message

Signature · 1 line
1.8.4.5
#60 lore raw

PATCH 52/78 multipathd: Issue warning on CLI command timeout

In reply to Hannes Reinecke

Some CLI commands like 'reconfigure' could take longer than the socket timeout, causing the CLI to report 'timeout receiving packet'. In these cases the multipath daemon should issue a warning with the actual time spent in the CLI call to give the administator some hints on the best socket timeout.

---
 multipathd/main.c   |  4 ++++
 multipathd/uxlsnr.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 52 insertions(+), 4 deletions(-)
multipathd/main.c +4 -0
index 394bec4..2d527df 100644
@@ -1522,6 +1522,8 @@ reconfigure (struct vectors * vecs)
 	struct config * old = conf;
 	int retval = 1;
 
+	running_state = DAEMON_CONFIGURE;
+
 	/*
 	 * free old map and path vectors ... they use old conf state
 	 */
@@ -1546,6 +1548,8 @@ reconfigure (struct vectors * vecs)
 		retval = 0;
 	}
 
+	running_state = DAEMON_RUNNING;
+
 	return retval;
 }
 
multipathd/uxlsnr.c +48 -4

101 lines not on this page · raw message

Signature · 1 line
1.8.4.5
#61 lore raw

PATCH 53/78 Use strlen() when checking for valid wwid

In reply to Hannes Reinecke

The wwid is required to be a string, so we should be using strlen() when checking for a valid WWID.

---
 libmultipath/configure.c | 5 +----
 multipathd/main.c        | 3 +--
 2 files changed, 2 insertions(+), 6 deletions(-)
libmultipath/configure.c +1 -4
index ddbd3ed..ef217de 100644
@@ -734,7 +734,6 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 {
 	int r = 1;
 	int k, i;
-	char empty_buff[WWID_SIZE];
 	char params[PARAMS_SIZE];
 	struct multipath * mpp;
 	struct path * pp1;
@@ -742,8 +741,6 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 	vector curmp = vecs->mpvec;
 	vector pathvec = vecs->pathvec;
 
-	memset(empty_buff, 0, WWID_SIZE);
-
 	/* ignore refwwid if it's empty */
 	if (refwwid && !strlen(refwwid))
 		refwwid = NULL;
@@ -757,7 +754,7 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 		/* skip this path for some reason */
 
 		/* 1. if path has no unique id or wwid blacklisted */
-		if (memcmp(empty_buff, pp1->wwid, WWID_SIZE) == 0 ||
+		if (strlen(pp1->wwid) == 0 ||
 		    filter_path(conf, pp1) > 0) {
 			orphan_path(pp1, "wwid blacklisted");
 			continue;
multipathd/main.c +1 -2
index 2d527df..b86e3b6 100644
@@ -482,7 +482,6 @@ int
 ev_add_path (struct path * pp, struct vectors * vecs)
 {
 	struct multipath * mpp;
-	char empty_buff[WWID_SIZE] = {0};
 	char params[PARAMS_SIZE] = {0};
 	int retries = 3;
 	int start_waiter = 0;
@@ -491,7 +490,7 @@ ev_add_path (struct path * pp, struct vectors * vecs)
 	/*
 	 * need path UID to go any further
 	 */
-	if (memcmp(empty_buff, pp->wwid, WWID_SIZE) == 0) {
+	if (strlen(pp->wwid) == 0) {
 		condlog(0, "%s: failed to get path uid", pp->dev);
 		goto fail; /* leave path added to pathvec */
 	}
Signature · 1 line
1.8.4.5
#62 lore raw

PATCH 54/78 multipathd: Use standard lists for CLI handling

In reply to Hannes Reinecke

We already have standard list handling macros in list.h, so we should be using them for CLI, too.

---
 libmultipath/list.h   |  2 ++
 libmultipath/uxsock.c |  2 +-
 multipathd/uxlsnr.c   | 40 ++++++++++++++++++++++++++--------------
 3 files changed, 29 insertions(+), 15 deletions(-)
libmultipath/list.h +2 -0
index 7a3cf16..a0d8184 100644
@@ -8,6 +8,8 @@
 #ifndef _LIST_H
 #define _LIST_H
 
+#include <stddef.h>
+
 /**
  * container_of - cast a member of a structure out to the containing structure
  *
libmultipath/uxsock.c +1 -1
index af0798c..300d657 100644
@@ -210,7 +210,7 @@ int recv_packet(int fd, char **buf, size_t *len, unsigned int timeout)
 	}
 	(*buf) = MALLOC(*len);
 	if (!*buf)
-		return -1;
+		return -ENOMEM;
 	ret = read_all(fd, *buf, *len, timeout);
 	if (ret != *len) {
 		FREE(*buf);
multipathd/uxlsnr.c +26 -14
index f20982f..7b13190 100644
@@ -39,11 +39,11 @@
 struct timespec sleep_time = {5, 0};
 
 struct client {
+	struct list_head node;
 	int fd;
-	struct client *next, *prev;
 };
 
-static struct client *clients;
+LIST_HEAD(clients);
 static unsigned num_clients;
 struct pollfd *polls;
 volatile sig_atomic_t reconfig_sig = 0;
@@ -67,10 +67,10 @@ static void new_client(int ux_sock)
 	/* put it in our linked list */
 	c = (struct client *)MALLOC(sizeof(*c));
 	memset(c, 0, sizeof(*c));
+	INIT_LIST_HEAD(&c->node);
 	c->fd = fd;
-	c->next = clients;
-	if (c->next) c->next->prev = c;
-	clients = c;
+
+	list_add_tail(&c->node, &clients);
 	num_clients++;
 }
 
@@ -80,9 +80,8 @@ static void new_client(int ux_sock)
 static void dead_client(struct client *c)
 {
 	close(c->fd);
-	if (c->prev) c->prev->next = c->next;
-	if (c->next) c->next->prev = c->prev;
-	if (c == clients) clients = c->next;
+	c->fd = -1;
+	list_del_init(&c->node);
 	FREE(c);
 	num_clients--;
 }
@@ -151,7 +150,7 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 	sigdelset(&mask, SIGHUP);
 	sigdelset(&mask, SIGUSR1);
 	while (1) {
-		struct client *c;
+		struct client *c, *tmp;
 		int i, poll_count;
 
 		/*
@@ -168,9 +167,13 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 		polls[0].events = POLLIN;
 
 		/* setup the clients */
-		for (i=1, c = clients; c; i++, c = c->next) {
+		i = 1;
+		list_for_each_entry(c, &clients, node) {
 			polls[i].fd = c->fd;
 			polls[i].events = POLLIN;
+			if (i > num_clients)
+				break;
+			i++;
 		}
 
 		/* most of our life is spent in this call */
@@ -191,12 +194,22 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 			continue;
 
 		/* see if a client wants to speak to us */
-		for (i=1, c = clients; c; i++) {
-			struct client *next = c->next;
-
+		for (i = 1; i < num_clients + 1; i++) {
 			if (polls[i].revents & POLLIN) {
 				struct timeval start_time;
 
+				c = NULL;
+				list_for_each_entry(tmp, &clients, node) {
+					if (tmp->fd == polls[i].fd) {
+						c = tmp;
+						break;
+					}
+				}
+				if (!c) {
+					condlog(3, "cli%d: invalid fd %d",
+						i, polls[i].fd);
+					continue;
+				}
 				if (gettimeofday(&start_time, NULL) != 0)
 					start_time.tv_sec = 0;
 
@@ -223,7 +236,6 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 					FREE(inbuf);
 				}
 			}
-			c = next;
 		}
 
 		/* see if we got a new client */
Signature · 1 line
1.8.4.5
#63 lore raw

PATCH 55/78 uxlsnr: use typedef for trigger function

In reply to Hannes Reinecke
---
 multipathd/uxlsnr.c | 3 +--
 multipathd/uxlsnr.h | 8 +++++---
 2 files changed, 6 insertions(+), 5 deletions(-)
multipathd/uxlsnr.c +1 -2
index 7b13190..550dcc4 100644
@@ -119,8 +119,7 @@ void uxsock_cleanup(void *arg)
 /*
  * entry point
  */
-void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
-			void * trigger_data)
+void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 {
 	int ux_sock;
 	size_t len;
multipathd/uxlsnr.h +5 -3
index 8b5a525..d274b04 100644
@@ -1,11 +1,13 @@
 #ifndef _UXLSNR_H
 #define _UXLSNR_H
 
-void * uxsock_listen(int (*uxsock_trigger)
-			(char *, char **, int *, void *),
-			void * trigger_data);
+typedef int (uxsock_trigger_fn)(char *, char **, int *, void *);
+
+void * uxsock_listen(uxsock_trigger_fn uxsock_trigger,
+		     void * trigger_data);
 
 extern volatile sig_atomic_t reconfig_sig;
 extern volatile sig_atomic_t log_reset_sig;
+
 #endif
 
Signature · 1 line
1.8.4.5
#64 lore raw

PATCH 56/78 multipathd: lock cli client list

In reply to Hannes Reinecke
---
 multipathd/uxlsnr.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)
multipathd/uxlsnr.c +16 -8
index 550dcc4..61ba49a 100644
@@ -44,7 +44,7 @@ struct client {
 };
 
 LIST_HEAD(clients);
-static unsigned num_clients;
+pthread_mutex_t client_lock = PTHREAD_MUTEX_INITIALIZER;
 struct pollfd *polls;
 volatile sig_atomic_t reconfig_sig = 0;
 volatile sig_atomic_t log_reset_sig = 0;
@@ -64,14 +64,15 @@ static void new_client(int ux_sock)
 	if (fd == -1)
 		return;
 
-	/* put it in our linked list */
 	c = (struct client *)MALLOC(sizeof(*c));
 	memset(c, 0, sizeof(*c));
 	INIT_LIST_HEAD(&c->node);
 	c->fd = fd;
 
+	/* put it in our linked list */
+	pthread_mutex_lock(&client_lock);
 	list_add_tail(&c->node, &clients);
-	num_clients++;
+	pthread_mutex_unlock(&client_lock);
 }
 
 /*
@@ -79,11 +80,12 @@ static void new_client(int ux_sock)
  */
 static void dead_client(struct client *c)
 {
+	pthread_mutex_lock(&client_lock);
+	list_del_init(&c->node);
+	pthread_mutex_unlock(&client_lock);
 	close(c->fd);
 	c->fd = -1;
-	list_del_init(&c->node);
 	FREE(c);
-	num_clients--;
 }
 
 void free_polls (void)
@@ -150,7 +152,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 	sigdelset(&mask, SIGUSR1);
 	while (1) {
 		struct client *c, *tmp;
-		int i, poll_count;
+		int i, poll_count, num_clients;
 
 		/*
 		 * Store configuration timeout;
@@ -161,6 +163,11 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 			timeout = conf->uxsock_timeout;
 
 		/* setup for a poll */
+		pthread_mutex_lock(&client_lock);
+		num_clients = 0;
+		list_for_each_entry(c, &clients, node) {
+			num_clients++;
+		}
 		polls = REALLOC(polls, (1+num_clients) * sizeof(*polls));
 		polls[0].fd = ux_sock;
 		polls[0].events = POLLIN;
@@ -170,10 +177,9 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 		list_for_each_entry(c, &clients, node) {
 			polls[i].fd = c->fd;
 			polls[i].events = POLLIN;
-			if (i > num_clients)
-				break;
 			i++;
 		}
+		pthread_mutex_unlock(&client_lock);
 
 		/* most of our life is spent in this call */
 		poll_count = ppoll(polls, i, &sleep_time, &mask);
@@ -198,12 +204,14 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 				struct timeval start_time;
 
 				c = NULL;
+				pthread_mutex_lock(&client_lock);
 				list_for_each_entry(tmp, &clients, node) {
 					if (tmp->fd == polls[i].fd) {
 						c = tmp;
 						break;
 					}
 				}
+				pthread_mutex_unlock(&client_lock);
 				if (!c) {
 					condlog(3, "cli%d: invalid fd %d",
 						i, polls[i].fd);
Signature · 1 line
1.8.4.5
#65 lore raw

PATCH 57/78 multipath: enable sync support

In reply to Hannes Reinecke

multipath didn't enable 'sync support' for device-mapper, causing device-mapper to create the device nodes internally rather than relying on udev here.

---
 multipath/main.c | 1 +
 1 file changed, 1 insertion(+)
multipath/main.c +1 -0
index ebca4dc..bee9a65 100644
@@ -582,6 +582,7 @@ main (int argc, char *argv[])
 	if (dm_prereq())
 		exit(1);
 	dm_drv_version(conf->version, TGT_MPATH);
+	udev_set_sync_support(1);
 
 	if (optind < argc) {
 		conf->dev = MALLOC(FILE_NAME_SIZE);
Signature · 1 line
1.8.4.5
#66 lore raw

PATCH 58/78 Remove dm_udev_XXX wrapper functions

In reply to Hannes Reinecke

As the definitions are encapsulated with the correct definitions anyway there is no need to have private wrappers here.

---
 libmultipath/devmapper.c | 19 +++++--------------
 libmultipath/devmapper.h |  2 --
 multipath/main.c         |  2 +-
 multipathd/main.c        |  2 +-
 4 files changed, 7 insertions(+), 18 deletions(-)
libmultipath/devmapper.c +5 -14
index f0b0da1..1712c56 100644
@@ -42,23 +42,14 @@ static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
 	return 1;
 }
 
-void udev_wait(unsigned int c)
+void dm_udev_wait(unsigned int c)
 {
 }
 
-void udev_set_sync_support(int c)
+void dm_udev_set_sync_support(int c)
 {
 }
-#else
-void udev_wait(unsigned int c)
-{
-	dm_udev_wait(c);
-}
 
-void udev_set_sync_support(int c)
-{
-	dm_udev_set_sync_support(c);
-}
 #endif
 
 static void
@@ -245,7 +236,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 		if (!r)
 			dm_udev_complete(cookie);
 		else
-			udev_wait(cookie);
+			dm_udev_wait(cookie);
 	}
 	out:
 	dm_task_destroy (dmt);
@@ -325,7 +316,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 		if (!r)
 			dm_udev_complete(cookie);
 		else
-			udev_wait(cookie);
+			dm_udev_wait(cookie);
 	}
 	freeout:
 	if (prefixed_uuid)
@@ -1354,7 +1345,7 @@ dm_rename (char * old, char * new)
 	if (!r)
 		dm_udev_complete(cookie);
 	else
-		udev_wait(cookie);
+		dm_udev_wait(cookie);
 
 out:
 	dm_task_destroy(dmt);
libmultipath/devmapper.h +0 -2
index 8188f48..2613503 100644
@@ -49,8 +49,6 @@ int dm_rename (char * old, char * new);
 int dm_reassign(const char * mapname);
 int dm_reassign_table(const char *name, char *old, char *new);
 int dm_setgeometry(struct multipath *mpp);
-void udev_wait(unsigned int c);
-void udev_set_sync_support(int c);
 
 #define VERSION_GE(v, minv) ( \
  (v[0] > minv[0]) || \
multipath/main.c +1 -1
index bee9a65..f62dcb1 100644
@@ -582,7 +582,7 @@ main (int argc, char *argv[])
 	if (dm_prereq())
 		exit(1);
 	dm_drv_version(conf->version, TGT_MPATH);
-	udev_set_sync_support(1);
+	dm_udev_set_sync_support(1);
 
 	if (optind < argc) {
 		conf->dev = MALLOC(FILE_NAME_SIZE);
multipathd/main.c +1 -1
index b86e3b6..fd3514f 100644
@@ -1817,7 +1817,7 @@ child (void * param)
 	set_oom_adj();
 
 	conf->daemon = 1;
-	udev_set_sync_support(0);
+	dm_udev_set_sync_support(0);
 #ifdef USE_SYSTEMD
 	envp = getenv("WATCHDOG_USEC");
 	if (envp && sscanf(envp, "%lu", &checkint) == 1) {
Signature · 1 line
1.8.4.5
#67 lore raw

PATCH 59/78 Revert to ACT_RELOAD in domap if the map exists

In reply to Hannes Reinecke

ACT_CREATE checks for an existing map already, so we might as well revert to ACT_RELOAD in these cases and avoid an error during configuration.

---
 libmultipath/configure.c | 16 ++++++++--------
 libmultipath/devmapper.c |  5 ++++-
 2 files changed, 12 insertions(+), 9 deletions(-)
libmultipath/configure.c +8 -8
index ef217de..24ad948 100644
@@ -577,7 +577,7 @@ fail:
 extern int
 domap (struct multipath * mpp, char * params)
 {
-	int r = 0;
+	int r = DOMAP_FAIL;
 
 	/*
 	 * last chance to quit before touching the devmaps
@@ -587,6 +587,12 @@ domap (struct multipath * mpp, char * params)
 		return DOMAP_DRY;
 	}
 
+	if (mpp->action == ACT_CREATE &&
+	    dm_map_present(mpp->alias)) {
+		condlog(3, "%s: map already present", mpp->alias);
+		mpp->action = ACT_RELOAD;
+	}
+
 	switch (mpp->action) {
 	case ACT_REJECT:
 	case ACT_NOTHING:
@@ -609,12 +615,6 @@ domap (struct multipath * mpp, char * params)
 			return DOMAP_RETRY;
 		}
 
-		if (dm_map_present(mpp->alias)) {
-			condlog(3, "%s: map already present", mpp->alias);
-			lock_multipath(mpp, 0);
-			break;
-		}
-
 		r = dm_addmap_create(mpp, params);
 
 		lock_multipath(mpp, 0);
@@ -650,7 +650,7 @@ domap (struct multipath * mpp, char * params)
 		break;
 	}
 
-	if (r) {
+	if (r == DOMAP_OK) {
 		/*
 		 * DM_DEVICE_CREATE, DM_DEVICE_RENAME, or DM_DEVICE_RELOAD
 		 * succeeded
libmultipath/devmapper.c +4 -1
index 1712c56..560d418 100644
@@ -347,8 +347,11 @@ dm_addmap_create (struct multipath *mpp, char * params) {
 			condlog(3, "%s: failed to load map (a path might be in use)", mpp->alias);
 			dm_flush_map_nosync(mpp->alias);
 		}
-		if (err != EROFS)
+		if (err != EROFS) {
+			condlog(3, "%s: failed to load map, error %d",
+				mpp->alias, err);
 			break;
+		}
 	}
 	return 0;
 }
Signature · 1 line
1.8.4.5
#68 lore raw

PATCH 60/78 multipathd: use local variable for watchdog configuration

In reply to Hannes Reinecke

The 'watchdog' setting in struct config needs to be a local variable in multipathd/main.c, as the config will be freed during reconfiguration and the checkerloop might then access an invalid structure.

---
 libmultipath/config.h | 1 -
 multipathd/main.c     | 8 ++++++--
 2 files changed, 6 insertions(+), 3 deletions(-)
libmultipath/config.h +0 -1
index 0149ad3..0183969 100644
@@ -116,7 +116,6 @@ struct config {
 	int ignore_wwids;
 	int checker_timeout;
 	int daemon;
-	int watchdog;
 	int flush_on_last_del;
 	int attribute_flags;
 	int fast_io_fail;
multipathd/main.c +6 -2
index fd3514f..f876258 100644
@@ -29,6 +29,10 @@
  */
 #include <checkers.h>
 
+#ifdef USE_SYSTEMD
+static int use_watchdog;
+#endif
+
 /*
  * libmultipath
  */
@@ -1400,7 +1404,7 @@ checkerloop (void *ap)
 		pthread_testcancel();
 		condlog(4, "tick");
 #ifdef USE_SYSTEMD
-		if (conf->watchdog)
+		if (use_watchdog)
 			sd_notify(0, "WATCHDOG=1");
 #endif
 		if (vecs->pathvec) {
@@ -1830,7 +1834,7 @@ child (void * param)
 			conf->checkint = conf->max_checkint / 4;
 		condlog(3, "enabling watchdog, interval %d max %d",
 			conf->checkint, conf->max_checkint);
-		conf->watchdog = conf->checkint;
+		use_watchdog = conf->checkint;
 	}
 #endif
 	/*
Signature · 1 line
1.8.4.5
#69 lore raw

PATCH 61/78 Ignore devices when sysfs_get_tgt_nodename fails

In reply to Hannes Reinecke

If sysfs_get_tgt_nodename fails we should not try to use this device.

---
 libmultipath/discovery.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
libmultipath/discovery.c +7 -5
index b1db00f..a9962d1 100644
@@ -349,8 +349,9 @@ sysfs_get_tgt_nodename (struct path *pp, char * node)
 		snprintf(node, NODE_NAME_SIZE, "ata-%d.00", tgtid);
 		return 0;
 	}
+	/* Unknown SCSI transport. Keep fingers crossed */
 	pp->sg_id.proto_id = SCSI_PROTOCOL_UNSPEC;
-	return 1;
+	return 0;
 }
 
 int sysfs_get_host_adapter_name(struct path *pp, char *adapter_name)
@@ -807,10 +808,11 @@ scsi_sysfs_pathinfo (struct path * pp)
 	/*
 	 * target node name
 	 */
-	if(!sysfs_get_tgt_nodename(pp, pp->tgt_node_name)) {
-		condlog(3, "%s: tgt_node_name = %s",
-			pp->dev, pp->tgt_node_name);
-	}
+	if(sysfs_get_tgt_nodename(pp, pp->tgt_node_name))
+		return 1;
+
+	condlog(3, "%s: tgt_node_name = %s",
+		pp->dev, pp->tgt_node_name);
 
 	return 0;
 }
Signature · 1 line
1.8.4.5
#70 lore raw

PATCH 62/78 Skip USB devices during discovery

In reply to Hannes Reinecke

Some USB devices even present VPD pages, but they still remain USB devices, and we shouldn't run multipath on those. So detect USB devices properly and skip them during discovery.

---
 libmultipath/discovery.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
libmultipath/discovery.c +13 -0
index a9962d1..c4aee1c 100644
@@ -275,6 +275,19 @@ sysfs_get_tgt_nodename (struct path *pp, char * node)
 		}
 	}
 
+	/* Check for USB */
+	tgtdev = udev_device_get_parent(parent);
+	while (tgtdev) {
+		value = udev_device_get_subsystem(tgtdev);
+		if (value && !strcmp(value, "usb")) {
+			pp->sg_id.proto_id = SCSI_PROTOCOL_UNSPEC;
+			tgtname = udev_device_get_sysname(tgtdev);
+			strncpy(node, tgtname, strlen(tgtname));
+			condlog(3, "%s: skip USB device %s", pp->dev, node);
+			return 1;
+		}
+		tgtdev = udev_device_get_parent(tgtdev);
+	}
 	parent = udev_device_get_parent_with_subsystem_devtype(pp->udev, "scsi", "scsi_target");
 	if (!parent)
 		return 1;
Signature · 1 line
1.8.4.5
#71 lore raw

PATCH 63/78 Read wwid from sysfs vpg_pg83 attribute

In reply to Hannes Reinecke

Using 'uid_attribute' per default has the problem that udev might not be able to retrieve the device ID in time as the device might be (temporarily) blocked.
It also has the problem that the 'ID_SERIAL' attribute is not well defined and might have been overridden by other udev rules.

As recent kernels have a 'vpd_pg83' sysfs attribute multipath should be reading this one directly and extract the uid from there. With that multipath does not need to do any I/O to generate the device wwid, eliminating one common error cause during failover.

---
 libmultipath/discovery.c | 305 ++++++++++++++++++++++++++++++++++++++++++-----
 libmultipath/sysfs.c     |  51 ++++++++
 libmultipath/sysfs.h     |   2 +
 3 files changed, 331 insertions(+), 27 deletions(-)
libmultipath/discovery.c +278 -27
index c4aee1c..681ee25 100644
@@ -207,6 +207,30 @@ declare_sysfs_get_str(vendor);
 declare_sysfs_get_str(model);
 declare_sysfs_get_str(rev);
 
+ssize_t
+sysfs_get_vpd (struct udev_device * udev, int pg,
+	       unsigned char * buff, size_t len)
+{
+	ssize_t attr_len;
+	char attrname[9];
+	const char * devname;
+
+	if (!udev) {
+		condlog(3, "No udev device given\n");
+		return -ENOSYS;
+	}
+
+	devname = udev_device_get_sysname(udev);
+	sprintf(attrname, "vpd_pg%02x", pg);
+	attr_len = sysfs_bin_attr_get_value(udev, attrname, buff, len);
+	if (attr_len < 0) {
+		condlog(3, "%s: attribute %s not found in sysfs",
+			devname, attrname);
+		return attr_len;
+	}
+	return attr_len;
+}
+
 int
 sysfs_get_timeout(struct path *pp, unsigned int *timeout)
 {
@@ -764,6 +788,183 @@ get_geometry(struct path *pp)
 }
 
 static int
+get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
+{
+	int len = -ENODATA, buff_len;
+	unsigned char buff[4096];
+
+	memset(buff, 0x0, 4096);
+	if (sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
+		condlog(3, "failed to get vpd pg%02x", pg);
+		return -EIO;
+	}
+
+	if (buff[1] != pg) {
+		condlog(3, "vpd pg%02x error, invalid vpd page %02x",
+			pg, buff[1]);
+		return -ENODATA;
+	}
+	buff_len = (buff[2] << 8) + buff[3] + 4;
+	if (buff_len > 4096)
+		condlog(3, "vpd pg%02x page truncated", pg);
+
+	if (pg == 0x80) {
+		char *p = NULL;
+		len = buff[3] + (buff[2] << 8);
+		if (len >= maxlen) {
+			condlog(3, "vpd pg%02x overflow, %d/%d bytes required",
+				pg, len, maxlen);
+			return -EINVAL;
+		}
+		if (len > 0) {
+			memcpy(str, buff + 4, len);
+			str[len] = '\0';
+		}
+		p = str + len - 1;
+		while (p > str && *p == ' ') {
+			*p = '\0';
+			p--;
+			len --;
+		}
+	} else if (pg == 0x83) {
+		unsigned char *d;
+		unsigned char *vpd = NULL;
+		int vpd_type, vpd_len, prio = -1, i;
+
+		d = (unsigned char *)buff + 4;
+		while (d < (unsigned char *)buff + buff_len) {
+			/* Select 'association: LUN' */
+			if ((d[1] & 0x30) != 0) {
+				d += d[3] + 4;
+				continue;
+			}
+			switch (d[1] & 0xf) {
+			case 0x3:
+				/* NAA: Prio 5 */
+				if (prio < 5) {
+					prio = 5;
+					vpd = d;
+				}
+				break;
+			case 0x8:
+				/* SCSI Name: Prio 4 */
+				if (memcmp(d + 4, "eui.", 4) &&
+				    memcmp(d + 4, "naa.", 4) &&
+				    memcmp(d + 4, "iqn.", 4))
+					continue;
+				if (prio < 4) {
+					prio = 4;
+					vpd = d;
+				}
+				break;
+			case 0x2:
+				/* EUI-64: Prio 3 */
+				if (prio < 3) {
+					prio = 3;
+					vpd = d;
+				}
+				break;
+			case 0x1:
+				/* T-10 Vendor ID: Prio 2 */
+				if (prio < 2) {
+					prio = 2;
+					vpd = d;
+				}
+				break;
+			}
+			d += d[3] + 4;
+		}
+		if (prio > 0) {
+			vpd_type = vpd[1] & 0xf;
+			vpd_len = vpd[3];
+			vpd += 4;
+			if (vpd_type == 0x2 || vpd_type == 0x3) {
+				int i;
+
+				len = sprintf(str, "%d", vpd_type);
+				for (i = 0; i < vpd_len; i++) {
+					len += sprintf(str + len,
+						       "%02x", vpd[i]);
+					if (len >= maxlen)
+						break;
+				}
+			} else if (vpd_type == 0x8) {
+				if (!memcmp("eui.", vpd, 4)) {
+					str[0] =  '2';
+					len = 1;
+					vpd += 4;
+					vpd_len -= 4;
+					for (i = 0; i < vpd_len; i++) {
+						len += sprintf(str + len, "%c",
+							       tolower(vpd[i]));
+						if (len >= maxlen)
+							break;
+					}
+					len = vpd_len + 1;
+					str[len] = '\0';
+				} else if (!memcmp("naa.", vpd, 4)) {
+					str[0] = '3';
+					len = 1;
+					vpd += 4;
+					vpd_len -= 4;
+					for (i = 0; i < vpd_len; i++) {
+						len += sprintf(str + len, "%c",
+							       tolower(vpd[i]));
+						if (len >= maxlen)
+							break;
+					}
+					len = vpd_len + 1;
+					str[len] = '\0';
+				} else {
+					str[0] = '8';
+					len = 1;
+					vpd += 4;
+					vpd_len -= 4;
+					if (vpd_len > maxlen + 2)
+						vpd_len = maxlen - 2;
+					memcpy(str, vpd, vpd_len);
+					len = vpd_len + 1;
+					str[len] = '\0';
+				}
+			} else if (vpd_type == 0x1) {
+				unsigned char *p;
+				int p_len;
+
+				str[0] = '1';
+				len = 1;
+				p = vpd;
+				while ((p = memchr(vpd, ' ', vpd_len))) {
+					p_len = p - vpd;
+					if (len + p_len > maxlen - 1)
+						p_len = maxlen - len - 2;
+					memcpy(str + len, vpd, p_len);
+					len += p_len;
+					if (len >= maxlen - 1) {
+						str[len] = '\0';
+						break;
+					}
+					str[len] = '_';
+					len ++;
+					vpd = p;
+					vpd_len -= p_len;
+					while (vpd && *vpd == ' ') {
+						vpd++;
+						vpd_len --;
+					}
+				}
+				if (len > 1 && str[len - 1] == '_') {
+					str[len - 1] = '\0';
+					len--;
+				}
+			}
+		}
+	} else
+		len = -ENOSYS;
+
+	return len;
+}
+
+static int
 scsi_sysfs_pathinfo (struct path * pp)
 {
 	struct udev_device *parent;
@@ -803,6 +1004,9 @@ scsi_sysfs_pathinfo (struct path * pp)
 
 	condlog(3, "%s: rev = %s", pp->dev, pp->rev);
 
+	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) >= 0)
+		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
+
 	/*
 	 * set the hwe configlet pointer
 	 */
@@ -1051,7 +1255,8 @@ static int
 scsi_ioctl_pathinfo (struct path * pp, int mask)
 {
 	if (mask & DI_SERIAL) {
-		get_serial(pp->serial, SERIAL_SIZE, pp->fd);
+		if (strlen(pp->serial) == 0)
+			get_serial(pp->serial, SERIAL_SIZE, pp->fd);
 		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
 	}
 
@@ -1144,10 +1349,56 @@ get_prio (struct path * pp)
 }
 
 static int
+get_udev_uid(struct path * pp, char *uid_attribute)
+{
+	ssize_t len;
+	const char *value;
+
+	value = udev_device_get_property_value(pp->udev,
+					       uid_attribute);
+	if ((!value || strlen(value) == 0) && conf->cmd == CMD_VALID_PATH)
+		value = getenv(uid_attribute);
+	if (value && strlen(value)) {
+		if (strlen(value) + 1 > WWID_SIZE) {
+			condlog(0, "%s: wwid overflow", pp->dev);
+			len = WWID_SIZE;
+		} else {
+			len = strlen(value);
+		}
+		strncpy(pp->wwid, value, len);
+	} else {
+		condlog(3, "%s: no %s attribute", pp->dev,
+			uid_attribute);
+		len = -EINVAL;
+	}
+	return len;
+}
+
+static int
+get_vpd_uid(struct path * pp)
+{
+	struct udev_device *parent = pp->udev;
+
+	while (parent) {
+		const char *subsys = udev_device_get_subsystem(parent);
+		if (subsys && !strncmp(subsys, "scsi", 4))
+			break;
+		parent = udev_device_get_parent(parent);
+	}
+
+	if (!parent) {
+		condlog(3, "%s: no scsi device found in sysfs", pp->dev);
+		return -ENXIO;
+	}
+	return get_vpd(parent, 0x83, pp->wwid, WWID_SIZE);
+}
+
+static int
 get_uid (struct path * pp)
 {
 	char *c;
-	const char *origin;
+	const char *origin = "none";
+	ssize_t len = 0;
 
 	if (!pp->uid_attribute && !pp->getuid)
 		select_getuid(pp);
@@ -1166,40 +1417,40 @@ get_uid (struct path * pp)
 		if (apply_format(pp->getuid, &buff[0], pp)) {
 			condlog(0, "error formatting uid callout command");
 			memset(pp->wwid, 0, WWID_SIZE);
+			len = -EINVAL;
 		} else if (execute_program(buff, pp->wwid, WWID_SIZE)) {
 			condlog(3, "error calling out %s", buff);
 			memset(pp->wwid, 0, WWID_SIZE);
-		}
+			len = -EIO;
+		} else
+			len = strlen(pp->wwid);
 		origin = "callout";
 	} else {
-		const char *value;
-
-		value = udev_device_get_property_value(pp->udev,
-						       pp->uid_attribute);
-		if ((!value || strlen(value) == 0) &&
-		    conf->cmd == CMD_VALID_PATH)
-			value = getenv(pp->uid_attribute);
-		if (value && strlen(value)) {
-			size_t len = WWID_SIZE;
-
-			if (strlen(value) + 1 > WWID_SIZE) {
-				condlog(0, "%s: wwid overflow", pp->dev);
-			} else {
-				len = strlen(value);
-			}
-			strncpy(pp->wwid, value, len);
+		if (pp->uid_attribute) {
+			len = get_udev_uid(pp, pp->uid_attribute);
+			origin = "udev";
 		} else {
-			condlog(3, "%s: no %s attribute", pp->dev,
-				pp->uid_attribute);
+			len = get_vpd_uid(pp);
+			if (len > 0)
+				origin = "sysfs";
+			else {
+				len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+				origin = "udev";
+			}
 		}
-		origin = "udev";
 	}
-	/* Strip any trailing blanks */
-	c = strchr(pp->wwid, '\0');
-	c--;
-	while (c && c >= pp->wwid && *c == ' ') {
-		*c = '\0';
+	if ( len < 0 ) {
+		condlog(1, "%s: failed to get uid: %s",
+			pp->dev, strerror(-len));
+		memset(pp->wwid, 0x0, WWID_SIZE);
+	} else {
+		/* Strip any trailing blanks */
+		c = strchr(pp->wwid, '\0');
 		c--;
+		while (c && c >= pp->wwid && *c == ' ') {
+			*c = '\0';
+			c--;
+		}
 	}
 	condlog(3, "%s: uid = %s (%s)", pp->dev,
 		*pp->wwid == '\0' ? "<empty>" : pp->wwid, origin);
libmultipath/sysfs.c +51 -0

57 lines not on this page · raw message

libmultipath/sysfs.h +2 -0
index 34f3e00..2588c24 100644
@@ -9,6 +9,8 @@ ssize_t sysfs_attr_set_value(struct udev_device *dev, const char *attr_name,
 			     char * value, size_t value_len);
 ssize_t sysfs_attr_get_value(struct udev_device *dev, const char *attr_name,
 			     char * value, size_t value_len);
+ssize_t sysfs_bin_attr_get_value(struct udev_device *dev, const char *attr_name,
+				 unsigned char * value, size_t value_len);
 int sysfs_get_size (struct path *pp, unsigned long long * size);
 int sysfs_check_holders(char * check_devt, char * new_devt);
 #endif
Signature · 1 line
1.8.4.5
#72 lore raw

PATCH 64/78 Assign local priority for NAA VPD descriptor

In reply to Hannes Reinecke

Some devices assign several NAA VPD descriptors. So implement an internal priority to select the 'best' NAA descriptor.

---
 libmultipath/discovery.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)
libmultipath/discovery.c +25 -3
index 681ee25..b1cd6a0 100644
@@ -829,7 +829,7 @@ get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
 	} else if (pg == 0x83) {
 		unsigned char *d;
 		unsigned char *vpd = NULL;
-		int vpd_type, vpd_len, prio = -1, i;
+		int vpd_type, vpd_len, prio = -1, i, naa_prio;
 
 		d = (unsigned char *)buff + 4;
 		while (d < (unsigned char *)buff + buff_len) {
@@ -841,8 +841,30 @@ get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
 			switch (d[1] & 0xf) {
 			case 0x3:
 				/* NAA: Prio 5 */
-				if (prio < 5) {
-					prio = 5;
+				switch (d[4] >> 4) {
+				case 6:
+					/* IEEE Registered Extended: Prio 8 */
+					naa_prio = 8;
+					break;
+				case 5:
+					/* IEEE Registered: Prio 7 */
+					naa_prio = 7;
+					break;
+				case 2:
+					/* IEEE Extended: Prio 6 */
+					naa_prio = 6;
+					break;
+				case 3:
+					/* IEEE Locally assigned: Prio 1 */
+					naa_prio = 1;
+					break;
+				default:
+					/* Default: no priority */
+					naa_prio = -1;
+					break;
+				}
+				if (prio < naa_prio) {
+					prio = naa_prio;
 					vpd = d;
 				}
 				break;
Signature · 1 line
1.8.4.5
#73 lore raw

PATCH 65/78 Use sysfs attribute vpd_pg80 to read serial number

In reply to Hannes Reinecke

There is no need to issue an ioctl to read the serial number as it's already present in sysfs. And we should not print out the serial number unconditionally but rather check the discovery mask.

---
 libmultipath/discovery.c | 36 ++++++++++++++++++++++++++++--------
 1 file changed, 28 insertions(+), 8 deletions(-)
libmultipath/discovery.c +28 -8
index b1cd6a0..f8a0503 100644
@@ -1026,9 +1026,6 @@ scsi_sysfs_pathinfo (struct path * pp)
 
 	condlog(3, "%s: rev = %s", pp->dev, pp->rev);
 
-	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) >= 0)
-		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
-
 	/*
 	 * set the hwe configlet pointer
 	 */
@@ -1276,13 +1273,36 @@ sysfs_pathinfo(struct path * pp)
 static int
 scsi_ioctl_pathinfo (struct path * pp, int mask)
 {
-	if (mask & DI_SERIAL) {
-		if (strlen(pp->serial) == 0)
-			get_serial(pp->serial, SERIAL_SIZE, pp->fd);
-		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
+	struct udev_device *parent;
+	const char *attr_path = NULL;
+
+	if (!(mask & DI_SERIAL))
+		return 0;
+
+	parent = pp->udev;
+	while (parent) {
+		const char *subsys = udev_device_get_subsystem(parent);
+		if (subsys && !strncmp(subsys, "scsi", 4)) {
+			attr_path = udev_device_get_sysname(parent);
+			if (!attr_path)
+				break;
+			if (sscanf(attr_path, "%i:%i:%i:%i",
+				   &pp->sg_id.host_no,
+				   &pp->sg_id.channel,
+				   &pp->sg_id.scsi_id,
+				   &pp->sg_id.lun) == 4)
+				break;
+		}
+		parent = udev_device_get_parent(parent);
 	}
+	if (!attr_path || pp->sg_id.host_no == -1)
+		return -ENODEV;
 
-	return 0;
+	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
+		condlog(3, "%s: serial = %s",
+			pp->dev, pp->serial);
+
+	return strlen(pp->serial) ? 0 : -EIO;
 }
 
 static int
Signature · 1 line
1.8.4.5
#74 lore raw

PATCH 66/78 Update multipath.conf.5 to clarify wwid generation

In reply to Hannes Reinecke

Add a new section in multipath.conf.5 to clarify wwid generation and update wrong defaults for the property whitelist.

---
 multipath.conf.defaults    |  2 +-
 multipath/multipath.conf.5 | 33 ++++++++++++++++++++++++++++-----
 2 files changed, 29 insertions(+), 6 deletions(-)
multipath.conf.defaults +1 -1
index a12a4f4..a4e8f75 100644
@@ -73,7 +73,7 @@
 #	}
 #}
 #blacklist_exceptions {
-#	property "(ID_SCSI_VPD|ID_WWN)"
+#	property "(SCSI_IDENT_.*|ID_WWN)"
 #}
 #devices {
 #	device {
multipath/multipath.conf.5 +28 -5
index 7e17f62..6980010 100644
@@ -55,9 +55,8 @@ section.
 .TP
 .B multipaths
 This section defines the multipath topologies. They are indexed by a
-\fIWorld Wide Identifier\fR(wwid), which is taken to be the value of
-the udev attribute given by the
-\fIuid_attribute\fR keyword.
+\fIWorld Wide Identifier\fR(wwid). For details on the wwid generation
+see section \fBWWID generation\fR below.
 .TP
 .B devices
 This section defines the device-specific settings.
@@ -176,7 +175,7 @@ identifier. Default value is
 .B getuid_callout
 The default program and args to callout to obtain a unique path
 identifier. Should be specified with an absolute path.
-This parameter is deprecated; \fIuid_attribute\fR should be used instead.
+This parameter is deprecated.
 .TP
 .B prio
 The name of the path priority routine. The specified routine
@@ -534,7 +533,7 @@ The \fIWorld Wide Identification\fR of a device.
 .TP
 .B property
 Regular expression of the udev property to be whitelisted. Defaults to
-.I (ID_WWN|ID_SCSI_VPD)
+.I (ID_WWN|SCSI_IDENT_.*)
 .TP
 .B devnode
 Regular expression of the device nodes to be whitelisted.
@@ -771,6 +770,30 @@ sections:
 .RE
 .PD
 .LP
+.SH "WWID generation"
+Multipath uses a \fIWorld Wide Identification\fR (wwid) to determine
+which paths belong to the same device. Each path presenting the same
+wwid is assumed to point to the same device.
+.LP
+The wwid is generated by three methods (in the order of preference):
+.TP 17
+.B getuid_callout
+Use the specified external program; cf \fIgetuid_callout\fR above.
+Care should be taken when using this method; the external program
+needs to be loaded from disk for execution, which might lead to
+deadlock situations in an all-paths-down scenario.
+.TP
+.B uid_attribute
+Use the value of the specified udev attribute; cf \fIuid_attribute\fR
+above. This method is preferred to \fIgetuid_callout\fR as multipath
+does not need to call any external programs here. However, under
+certain circumstances udev might not be able to generate the requested
+variable.
+.TP
+.B vpd_pg83
+If none of the \fIgetuid_callout\fR or \fIuid_attribute\fR parameters
+are present multipath will try to use the sysfs attribute
+\fIvpd_pg83\fR to generate the wwid.
 .SH "KNOWN ISSUES"
 The usage of
 .B queue_if_no_path
Signature · 1 line
1.8.4.5
#75 lore raw

PATCH 67/78 libmultipath: Fall back to SG_IO if no UID could be assigned

In reply to Hannes Reinecke

Relying on udev attributes or sysfs vpd pages might fail, in which case we're unable to assign an UID and multipath will fail to start up.
Implement a fallback to read VPD page 0x83 directly in these cases.

---
 libmultipath/discovery.c | 53 +++++++++++++++++++++++++++++++++++-------------
 1 file changed, 39 insertions(+), 14 deletions(-)
libmultipath/discovery.c +39 -14
index f8a0503..d5dda2c 100644
@@ -770,6 +770,29 @@ get_serial (char * str, int maxlen, int fd)
 	return 1;
 }
 
+#define DEFAULT_SGIO_LEN 254
+
+static int
+sgio_get_vpd (unsigned char * buff, int maxlen, int fd)
+{
+	int len = DEFAULT_SGIO_LEN;
+
+	if (fd < 0) {
+		errno = EBADF;
+		return -1;
+	}
+retry:
+	if (0 == do_inq(fd, 0, 1, 0x83, buff, len)) {
+		len = buff[3] + (buff[2] << 8);
+		if (len >= maxlen)
+			return len;
+		if (len > DEFAULT_SGIO_LEN)
+			goto retry;
+		return 0;
+	}
+	return -1;
+}
+
 static int
 get_geometry(struct path *pp)
 {
@@ -788,15 +811,19 @@ get_geometry(struct path *pp)
 }
 
 static int
-get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
+get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 {
 	int len = -ENODATA, buff_len;
 	unsigned char buff[4096];
 
 	memset(buff, 0x0, 4096);
-	if (sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
-		condlog(3, "failed to get vpd pg%02x", pg);
-		return -EIO;
+	if (!parent || sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
+		condlog(3, "failed to read sysfs vpd pg%02x", pg);
+		if (sgio_get_vpd(buff, 4096, fd) <= 0) {
+			condlog(3, "failed to issue vpd inquiry for pg%02x",
+				pg);
+			return -errno;
+		}
 	}
 
 	if (buff[1] != pg) {
@@ -1298,7 +1325,7 @@ scsi_ioctl_pathinfo (struct path * pp, int mask)
 	if (!attr_path || pp->sg_id.host_no == -1)
 		return -ENODEV;
 
-	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
+	if (get_vpd(parent, pp->fd, 0x80, pp->serial, SERIAL_SIZE) > 0)
 		condlog(3, "%s: serial = %s",
 			pp->dev, pp->serial);
 
@@ -1428,11 +1455,7 @@ get_vpd_uid(struct path * pp)
 		parent = udev_device_get_parent(parent);
 	}
 
-	if (!parent) {
-		condlog(3, "%s: no scsi device found in sysfs", pp->dev);
-		return -ENXIO;
-	}
-	return get_vpd(parent, 0x83, pp->wwid, WWID_SIZE);
+	return get_vpd(parent, pp->fd, 0x83, pp->wwid, WWID_SIZE);
 }
 
 static int
@@ -1471,14 +1494,16 @@ get_uid (struct path * pp)
 		if (pp->uid_attribute) {
 			len = get_udev_uid(pp, pp->uid_attribute);
 			origin = "udev";
-		} else {
+		}
+		if (len <= 0) {
 			len = get_vpd_uid(pp);
 			if (len > 0)
 				origin = "sysfs";
-			else {
-				len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+		}
+		if (len <= 0) {
+			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+			if (len > 0)
 				origin = "udev";
-			}
 		}
 	}
 	if ( len < 0 ) {
Signature · 1 line
1.8.4.5
#76 lore raw

PATCH 68/78 libmultipath: unset 'uid_attribute' on failure

In reply to Hannes Reinecke

Due to a race condition within udev the 'uid_attribute' might not always be set. So we should be zeroing the 'uid_attribute' when retrieving the uid by other means, otherwise the discovery process will blacklist the device.

---
 libmultipath/discovery.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
libmultipath/discovery.c +7 -3
index d5dda2c..8e6b228 100644
@@ -1497,13 +1497,17 @@ get_uid (struct path * pp)
 		}
 		if (len <= 0) {
 			len = get_vpd_uid(pp);
-			if (len > 0)
+			if (len > 0) {
 				origin = "sysfs";
+				pp->uid_attribute = NULL;
+			}
 		}
 		if (len <= 0) {
 			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
-			if (len > 0)
-				origin = "udev";
+			if (len > 0) {
+				origin = "default";
+				pp->uid_attribute = DEFAULT_UID_ATTRIBUTE;
+			}
 		}
 	}
 	if ( len < 0 ) {
Signature · 1 line
1.8.4.5
#77 lore raw

Re: [PATCH 68/78] libmultipath: unset 'uid_attribute' on failure

Quoted 4 lines · Hannes Reinecke

Due to a race condition within udev the 'uid_attribute' might not always be set. So we should be zeroing the 'uid_attribute' when retrieving the uid by other means, otherwise the discovery process will blacklist the device.

Possibly I'm just missing obvious here, but I don't get the point of zeroing out the uid_attribute. Won't it just get reset on the next call to get_uid?

-Ben

Quoted 33 lines
---
 libmultipath/discovery.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
libmultipath/discovery.c +7 -3
index d5dda2c..8e6b228 100644
@@ -1497,13 +1497,17 @@ get_uid (struct path * pp)
 		}
 		if (len <= 0) {
 			len = get_vpd_uid(pp);
-			if (len > 0)
+			if (len > 0) {
 				origin = "sysfs";
+				pp->uid_attribute = NULL;
+			}
 		}
 		if (len <= 0) {
 			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
-			if (len > 0)
-				origin = "udev";
+			if (len > 0) {
+				origin = "default";
+				pp->uid_attribute = DEFAULT_UID_ATTRIBUTE;
+			}
 		}
 	}
 	if ( len < 0 ) {
Signature · 1 line
1.8.4.5
#78 lore raw

Re: [PATCH 68/78] libmultipath: unset 'uid_attribute' on failure

Quoted 10 lines · Benjamin Marzinski
Quoted 4 lines · Hannes Reinecke

Due to a race condition within udev the 'uid_attribute' might not always be set. So we should be zeroing the 'uid_attribute' when retrieving the uid by other means, otherwise the discovery process will blacklist the device.

Possibly I'm just missing obvious here, but I don't get the point of zeroing out the uid_attribute. Won't it just get reset on the next call to get_uid?

The uid_attribute setting is evaluated later on, so when we're not zeroing it the code assumes we've got the UID from the udev attribute, which is untrue.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#79 lore raw

PATCH 69/78 Separate out vpd parsing functions

In reply to Hannes Reinecke
---
 libmultipath/discovery.c | 367 +++++++++++++++++++++++++----------------------
 1 file changed, 193 insertions(+), 174 deletions(-)
libmultipath/discovery.c +193 -174
index 8e6b228..50444db 100644
@@ -811,9 +811,196 @@ get_geometry(struct path *pp)
 }
 
 static int
+parse_vpd_pg80(const unsigned char *in, char *out, size_t out_len)
+{
+	char *p = NULL;
+	int len = in[3] + (in[2] << 8);
+
+	if (len >= out_len) {
+		condlog(2, "vpd pg80 overflow, %d/%d bytes required",
+			len, (int)out_len);
+		len = out_len;
+	}
+	if (len > 0) {
+		memcpy(out, in + 4, len);
+		out[len] = '\0';
+	}
+	/*
+	 * Strip trailing whitspaces
+	 */
+	p = out + len - 1;
+	while (p > out && *p == ' ') {
+		*p = '\0';
+		p--;
+		len --;
+	}
+	return len;
+}
+
+static int
+parse_vpd_pg83(const unsigned char *in, size_t in_len,
+	       char *out, size_t out_len)
+{
+	unsigned char *d;
+	unsigned char *vpd = NULL;
+	int len = -ENODATA, vpd_type, vpd_len, prio = -1, i, naa_prio;
+
+	d = (unsigned char *)in + 4;
+	while (d < (unsigned char *)in + in_len) {
+		/* Select 'association: LUN' */
+		if ((d[1] & 0x30) != 0) {
+			d += d[3] + 4;
+			continue;
+		}
+		switch (d[1] & 0xf) {
+		case 0x3:
+			/* NAA: Prio 5 */
+			switch (d[4] >> 4) {
+			case 6:
+				/* IEEE Registered Extended: Prio 8 */
+				naa_prio = 8;
+				break;
+			case 5:
+				/* IEEE Registered: Prio 7 */
+				naa_prio = 7;
+				break;
+			case 2:
+				/* IEEE Extended: Prio 6 */
+				naa_prio = 6;
+				break;
+			case 3:
+				/* IEEE Locally assigned: Prio 1 */
+				naa_prio = 1;
+				break;
+			default:
+				/* Default: no priority */
+				naa_prio = -1;
+				break;
+			}
+			if (prio < naa_prio) {
+				prio = naa_prio;
+				vpd = d;
+			}
+			break;
+		case 0x8:
+			/* SCSI Name: Prio 4 */
+			if (memcmp(d + 4, "eui.", 4) &&
+			    memcmp(d + 4, "naa.", 4) &&
+			    memcmp(d + 4, "iqn.", 4))
+				continue;
+			if (prio < 4) {
+				prio = 4;
+				vpd = d;
+			}
+			break;
+		case 0x2:
+			/* EUI-64: Prio 3 */
+			if (prio < 3) {
+				prio = 3;
+				vpd = d;
+			}
+			break;
+		case 0x1:
+			/* T-10 Vendor ID: Prio 2 */
+			if (prio < 2) {
+				prio = 2;
+				vpd = d;
+			}
+			break;
+		}
+		d += d[3] + 4;
+	}
+	if (prio > 0) {
+		vpd_type = vpd[1] & 0xf;
+		vpd_len = vpd[3];
+		vpd += 4;
+		if (vpd_type == 0x2 || vpd_type == 0x3) {
+			int i;
+
+			len = sprintf(out, "%d", vpd_type);
+			for (i = 0; i < vpd_len; i++) {
+				len += sprintf(out + len,
+					       "%02x", vpd[i]);
+				if (len >= out_len)
+					break;
+			}
+		} else if (vpd_type == 0x8) {
+			if (!memcmp("eui.", vpd, 4)) {
+				out[0] =  '2';
+				len = 1;
+				vpd += 4;
+				vpd_len -= 4;
+				for (i = 0; i < vpd_len; i++) {
+					len += sprintf(out + len, "%c",
+						       tolower(vpd[i]));
+					if (len >= out_len)
+						break;
+				}
+				len = vpd_len + 1;
+				out[len] = '\0';
+			} else if (!memcmp("naa.", vpd, 4)) {
+				out[0] = '3';
+				len = 1;
+				vpd += 4;
+				vpd_len -= 4;
+				for (i = 0; i < vpd_len; i++) {
+					len += sprintf(out + len, "%c",
+						       tolower(vpd[i]));
+					if (len >= out_len)
+						break;
+				}
+				len = vpd_len + 1;
+				out[len] = '\0';
+			} else {
+				out[0] = '8';
+				len = 1;
+				vpd += 4;
+				vpd_len -= 4;
+				if (vpd_len > out_len + 2)
+					vpd_len = out_len - 2;
+				memcpy(out, vpd, vpd_len);
+				len = vpd_len + 1;
+				out[len] = '\0';
+			}
+		} else if (vpd_type == 0x1) {
+			unsigned char *p;
+			int p_len;
+
+			out[0] = '1';
+			len = 1;
+			p = vpd;
+			while ((p = memchr(vpd, ' ', vpd_len))) {
+				p_len = p - vpd;
+				if (len + p_len > out_len - 1)
+					p_len = out_len - len - 2;
+				memcpy(out + len, vpd, p_len);
+				len += p_len;
+				if (len >= out_len - 1) {
+					out[len] = '\0';
+					break;
+				}
+				out[len] = '_';
+				len ++;
+				vpd = p;
+				vpd_len -= p_len;
+				while (vpd && *vpd == ' ') {
+					vpd++;
+					vpd_len --;
+				}
+			}
+			if (len > 1 && out[len - 1] == '_') {
+				out[len - 1] = '\0';
+				len--;
+			}
+		}
+	}
+	return len;
+}
+
+static int
 get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 {
-	int len = -ENODATA, buff_len;
+	int len, buff_len;
 	unsigned char buff[4096];
 
 	memset(buff, 0x0, 4096);
@@ -835,179 +1022,11 @@ get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 	if (buff_len > 4096)
 		condlog(3, "vpd pg%02x page truncated", pg);
 
-	if (pg == 0x80) {
-		char *p = NULL;
-		len = buff[3] + (buff[2] << 8);
-		if (len >= maxlen) {
-			condlog(3, "vpd pg%02x overflow, %d/%d bytes required",
-				pg, len, maxlen);
-			return -EINVAL;
-		}
-		if (len > 0) {
-			memcpy(str, buff + 4, len);
-			str[len] = '\0';
-		}
-		p = str + len - 1;
-		while (p > str && *p == ' ') {
-			*p = '\0';
-			p--;
-			len --;
-		}
-	} else if (pg == 0x83) {
-		unsigned char *d;
-		unsigned char *vpd = NULL;
-		int vpd_type, vpd_len, prio = -1, i, naa_prio;
-
-		d = (unsigned char *)buff + 4;
-		while (d < (unsigned char *)buff + buff_len) {
-			/* Select 'association: LUN' */
-			if ((d[1] & 0x30) != 0) {
-				d += d[3] + 4;
-				continue;
-			}
-			switch (d[1] & 0xf) {
-			case 0x3:
-				/* NAA: Prio 5 */
-				switch (d[4] >> 4) {
-				case 6:
-					/* IEEE Registered Extended: Prio 8 */
-					naa_prio = 8;
-					break;
-				case 5:
-					/* IEEE Registered: Prio 7 */
-					naa_prio = 7;
-					break;
-				case 2:
-					/* IEEE Extended: Prio 6 */
-					naa_prio = 6;
-					break;
-				case 3:
-					/* IEEE Locally assigned: Prio 1 */
-					naa_prio = 1;
-					break;
-				default:
-					/* Default: no priority */
-					naa_prio = -1;
-					break;
-				}
-				if (prio < naa_prio) {
-					prio = naa_prio;
-					vpd = d;
-				}
-				break;
-			case 0x8:
-				/* SCSI Name: Prio 4 */
-				if (memcmp(d + 4, "eui.", 4) &&
-				    memcmp(d + 4, "naa.", 4) &&
-				    memcmp(d + 4, "iqn.", 4))
-					continue;
-				if (prio < 4) {
-					prio = 4;
-					vpd = d;
-				}
-				break;
-			case 0x2:
-				/* EUI-64: Prio 3 */
-				if (prio < 3) {
-					prio = 3;
-					vpd = d;
-				}
-				break;
-			case 0x1:
-				/* T-10 Vendor ID: Prio 2 */
-				if (prio < 2) {
-					prio = 2;
-					vpd = d;
-				}
-				break;
-			}
-			d += d[3] + 4;
-		}
-		if (prio > 0) {
-			vpd_type = vpd[1] & 0xf;
-			vpd_len = vpd[3];
-			vpd += 4;
-			if (vpd_type == 0x2 || vpd_type == 0x3) {
-				int i;
-
-				len = sprintf(str, "%d", vpd_type);
-				for (i = 0; i < vpd_len; i++) {
-					len += sprintf(str + len,
-						       "%02x", vpd[i]);
-					if (len >= maxlen)
-						break;
-				}
-			} else if (vpd_type == 0x8) {
-				if (!memcmp("eui.", vpd, 4)) {
-					str[0] =  '2';
-					len = 1;
-					vpd += 4;
-					vpd_len -= 4;
-					for (i = 0; i < vpd_len; i++) {
-						len += sprintf(str + len, "%c",
-							       tolower(vpd[i]));
-						if (len >= maxlen)
-							break;
-					}
-					len = vpd_len + 1;
-					str[len] = '\0';
-				} else if (!memcmp("naa.", vpd, 4)) {
-					str[0] = '3';
-					len = 1;
-					vpd += 4;
-					vpd_len -= 4;
-					for (i = 0; i < vpd_len; i++) {
-						len += sprintf(str + len, "%c",
-							       tolower(vpd[i]));
-						if (len >= maxlen)
-							break;
-					}
-					len = vpd_len + 1;
-					str[len] = '\0';
-				} else {
-					str[0] = '8';
-					len = 1;
-					vpd += 4;
-					vpd_len -= 4;
-					if (vpd_len > maxlen + 2)
-						vpd_len = maxlen - 2;
-					memcpy(str, vpd, vpd_len);
-					len = vpd_len + 1;
-					str[len] = '\0';
-				}
-			} else if (vpd_type == 0x1) {
-				unsigned char *p;
-				int p_len;
-
-				str[0] = '1';
-				len = 1;
-				p = vpd;
-				while ((p = memchr(vpd, ' ', vpd_len))) {
-					p_len = p - vpd;
-					if (len + p_len > maxlen - 1)
-						p_len = maxlen - len - 2;
-					memcpy(str + len, vpd, p_len);
-					len += p_len;
-					if (len >= maxlen - 1) {
-						str[len] = '\0';
-						break;
-					}
-					str[len] = '_';
-					len ++;
-					vpd = p;
-					vpd_len -= p_len;
-					while (vpd && *vpd == ' ') {
-						vpd++;
-						vpd_len --;
-					}
-				}
-				if (len > 1 && str[len - 1] == '_') {
-					str[len - 1] = '\0';
-					len--;
-				}
-			}
-		}
-	} else
+	if (pg == 0x80)
+		len = parse_vpd_pg80(buff, str, maxlen);
+	else if (pg == 0x83)
+		len = parse_vpd_pg83(buff, buff_len, str, maxlen);
+	else
 		len = -ENOSYS;
 
 	return len;
Signature · 1 line
1.8.4.5
#80 lore raw

PATCH 70/78 multipathd: use SG_IO as fallback to generate uid

In reply to Hannes Reinecke

If both udev attribute and sysfs vpd fails to generate a valid WWID fallback to use SG_IO directly.

---
 libmultipath/discovery.c | 61 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 50 insertions(+), 11 deletions(-)
libmultipath/discovery.c +50 -11
index 50444db..009ef0c 100644
@@ -998,7 +998,7 @@ parse_vpd_pg83(const unsigned char *in, size_t in_len,
 }
 
 static int
-get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
+get_vpd_sysfs (struct udev_device *parent, int pg, char * str, int maxlen)
 {
 	int len, buff_len;
 	unsigned char buff[4096];
@@ -1006,11 +1006,39 @@ get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 	memset(buff, 0x0, 4096);
 	if (!parent || sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
 		condlog(3, "failed to read sysfs vpd pg%02x", pg);
-		if (sgio_get_vpd(buff, 4096, fd) <= 0) {
-			condlog(3, "failed to issue vpd inquiry for pg%02x",
-				pg);
-			return -errno;
-		}
+		return -EINVAL;
+	}
+
+	if (buff[1] != pg) {
+		condlog(3, "vpd pg%02x error, invalid vpd page %02x",
+			pg, buff[1]);
+		return -ENODATA;
+	}
+	buff_len = (buff[2] << 8) + buff[3] + 4;
+	if (buff_len > 4096)
+		condlog(3, "vpd pg%02x page truncated", pg);
+
+	if (pg == 0x80)
+		len = parse_vpd_pg80(buff, str, maxlen);
+	else if (pg == 0x83)
+		len = parse_vpd_pg83(buff, buff_len, str, maxlen);
+	else
+		len = -ENOSYS;
+
+	return len;
+}
+
+static int
+get_vpd_sgio (int fd, int pg, char * str, int maxlen)
+{
+	int len, buff_len;
+	unsigned char buff[4096];
+
+	memset(buff, 0x0, 4096);
+	if (sgio_get_vpd(buff, 4096, fd) <= 0) {
+		condlog(3, "failed to issue vpd inquiry for pg%02x",
+			pg);
+		return -errno;
 	}
 
 	if (buff[1] != pg) {
@@ -1344,7 +1372,7 @@ scsi_ioctl_pathinfo (struct path * pp, int mask)
 	if (!attr_path || pp->sg_id.host_no == -1)
 		return -ENODEV;
 
-	if (get_vpd(parent, pp->fd, 0x80, pp->serial, SERIAL_SIZE) > 0)
+	if (get_vpd_sysfs(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
 		condlog(3, "%s: serial = %s",
 			pp->dev, pp->serial);
 
@@ -1474,7 +1502,7 @@ get_vpd_uid(struct path * pp)
 		parent = udev_device_get_parent(parent);
 	}
 
-	return get_vpd(parent, pp->fd, 0x83, pp->wwid, WWID_SIZE);
+	return get_vpd_sysfs(parent, 0x83, pp->wwid, WWID_SIZE);
 }
 
 static int
@@ -1513,19 +1541,30 @@ get_uid (struct path * pp)
 		if (pp->uid_attribute) {
 			len = get_udev_uid(pp, pp->uid_attribute);
 			origin = "udev";
+			if (len <= 0)
+				condlog(2,
+					"%s: failed to get UID attribute '%s'",
+					pp->dev, pp->uid_attribute);
 		}
 		if (len <= 0) {
 			len = get_vpd_uid(pp);
 			if (len > 0) {
 				origin = "sysfs";
 				pp->uid_attribute = NULL;
+			} else {
+				condlog(2, "%s: failed to get sysfs vpd pg83",
+					pp->dev);
 			}
 		}
 		if (len <= 0) {
-			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+			len = get_vpd_sgio(pp->fd, 0x83, pp->wwid,
+					   WWID_SIZE);
 			if (len > 0) {
-				origin = "default";
-				pp->uid_attribute = DEFAULT_UID_ATTRIBUTE;
+				origin = "sgio";
+				pp->uid_attribute = NULL;
+			} else {
+				condlog(2, "%s: failed to get sgio vpd pg83",
+					pp->dev);
 			}
 		}
 	}
Signature · 1 line
1.8.4.5
#81 lore raw

PATCH 71/78 Do not automatically fall back to vpd uid generation

In reply to Hannes Reinecke

When a wwid via uid_attribute cannot be generated we should not fall back to using sysfs vpd attributes automatically. The uid_attribute can be set to virtually anything and an automatic fallback will then generated a different wwid, leading to an incorrect setup.
A fallback is only valid if the uid_attribute value is the default setting of 'ID_SERIAL', for which the format is fixed.

---
 libmultipath/discovery.c | 39 ++++++++++++++++-----------------------
 1 file changed, 16 insertions(+), 23 deletions(-)
libmultipath/discovery.c +16 -23
index 009ef0c..d1a16e6 100644
@@ -1509,7 +1509,7 @@ static int
 get_uid (struct path * pp)
 {
 	char *c;
-	const char *origin = "none";
+	const char *origin = "unknown";
 	ssize_t len = 0;
 
 	if (!pp->uid_attribute && !pp->getuid)
@@ -1542,35 +1542,28 @@ get_uid (struct path * pp)
 			len = get_udev_uid(pp, pp->uid_attribute);
 			origin = "udev";
 			if (len <= 0)
-				condlog(2,
-					"%s: failed to get UID attribute '%s'",
-					pp->dev, pp->uid_attribute);
+				condlog(1,
+					"%s: failed to get udev uid: %s",
+					pp->dev, strerror(-len));
+
 		}
-		if (len <= 0) {
+		if (len <= 0 &&
+		    !strcmp(pp->uid_attribute, DEFAULT_UID_ATTRIBUTE)) {
 			len = get_vpd_uid(pp);
-			if (len > 0) {
-				origin = "sysfs";
-				pp->uid_attribute = NULL;
-			} else {
-				condlog(2, "%s: failed to get sysfs vpd pg83",
-					pp->dev);
-			}
-		}
-		if (len <= 0) {
-			len = get_vpd_sgio(pp->fd, 0x83, pp->wwid,
-					   WWID_SIZE);
-			if (len > 0) {
+			origin = "sysfs";
+			pp->uid_attribute = NULL;
+			if (len < 0) {
+				condlog(1, "%s: failed to get sysfs uid: %s",
+					pp->dev, strerror(-len));
+				len = get_vpd_sgio(pp->fd, 0x83, pp->wwid,
+						   WWID_SIZE);
 				origin = "sgio";
-				pp->uid_attribute = NULL;
-			} else {
-				condlog(2, "%s: failed to get sgio vpd pg83",
-					pp->dev);
 			}
 		}
 	}
 	if ( len < 0 ) {
-		condlog(1, "%s: failed to get uid: %s",
-			pp->dev, strerror(-len));
+		condlog(1, "%s: failed to get %s uid: %s",
+			pp->dev, origin, strerror(-len));
 		memset(pp->wwid, 0x0, WWID_SIZE);
 	} else {
 		/* Strip any trailing blanks */
Signature · 1 line
1.8.4.5
#82 lore raw

PATCH 72/78 libmultipath: make vpd page 0x80 optional

In reply to Hannes Reinecke

VPD page 0x80 is optional, so we shouldn't return an error if the page is not present.

---
 libmultipath/discovery.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
libmultipath/discovery.c +2 -2
index d1a16e6..4582a20 100644
@@ -1370,13 +1370,13 @@ scsi_ioctl_pathinfo (struct path * pp, int mask)
 		parent = udev_device_get_parent(parent);
 	}
 	if (!attr_path || pp->sg_id.host_no == -1)
-		return -ENODEV;
+		return 0;
 
 	if (get_vpd_sysfs(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
 		condlog(3, "%s: serial = %s",
 			pp->dev, pp->serial);
 
-	return strlen(pp->serial) ? 0 : -EIO;
+	return 0;
 }
 
 static int
Signature · 1 line
1.8.4.5
#83 lore raw

PATCH 73/78 multipathd: push down lock in checkerloop()

In reply to Hannes Reinecke

Instead of grabbing the lock at the start of the checkerloop and releasing it at the end we should be holding it only during the time when we actually need it.

---
 multipathd/main.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
multipathd/main.c +12 -4
index f876258..9e7bf4f 100644
@@ -1399,32 +1399,40 @@ checkerloop (void *ap)
 
 		if (gettimeofday(&start_time, NULL) != 0)
 			start_time.tv_sec = 0;
-		pthread_cleanup_push(cleanup_lock, &vecs->lock);
-		lock(vecs->lock);
-		pthread_testcancel();
 		condlog(4, "tick");
 #ifdef USE_SYSTEMD
 		if (use_watchdog)
 			sd_notify(0, "WATCHDOG=1");
 #endif
 		if (vecs->pathvec) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			vector_foreach_slot (vecs->pathvec, pp, i) {
 				num_paths += check_path(vecs, pp);
 			}
+			lock_cleanup_pop(vecs->lock);
 		}
 		if (vecs->mpvec) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			defered_failback_tick(vecs->mpvec);
 			retry_count_tick(vecs->mpvec);
+			lock_cleanup_pop(vecs->lock);
 		}
 		if (count)
 			count--;
 		else {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			condlog(4, "map garbage collection");
 			mpvec_garbage_collector(vecs);
 			count = MAPGCINT;
+			lock_cleanup_pop(vecs->lock);
 		}
 
-		lock_cleanup_pop(vecs->lock);
 		if (start_time.tv_sec &&
 		    gettimeofday(&end_time, NULL) == 0 &&
 		    num_paths) {
Signature · 1 line
1.8.4.5
#84 lore raw

Re: [PATCH 73/78] multipathd: push down lock in checkerloop()

Quoted 3 lines · Hannes Reinecke

Instead of grabbing the lock at the start of the checkerloop and releasing it at the end we should be holding it only during the time when we actually need it.

the vecs lock is designed to protect the vecs vectors (and conf), so checking vecs->pathvec and vecs->mpvec outside of the vecs lock doesn't guarantee us that they will still be there once we've grabbed the lock. I don't think this could actually cause a memory issue, since all the vecs code already checks if the vector is NULL, and the check before locking will probably be right most of the time. But perhaps we should check again after we do the locking, or at least add a comment saying that we know that the pre-lock check isn't guaranteed to be correct, so as not to confuse people.

-Ben

Quoted 57 lines
---
 multipathd/main.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
multipathd/main.c +12 -4
index f876258..9e7bf4f 100644
@@ -1399,32 +1399,40 @@ checkerloop (void *ap)
 
 		if (gettimeofday(&start_time, NULL) != 0)
 			start_time.tv_sec = 0;
-		pthread_cleanup_push(cleanup_lock, &vecs->lock);
-		lock(vecs->lock);
-		pthread_testcancel();
 		condlog(4, "tick");
 #ifdef USE_SYSTEMD
 		if (use_watchdog)
 			sd_notify(0, "WATCHDOG=1");
 #endif
 		if (vecs->pathvec) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			vector_foreach_slot (vecs->pathvec, pp, i) {
 				num_paths += check_path(vecs, pp);
 			}
+			lock_cleanup_pop(vecs->lock);
 		}
 		if (vecs->mpvec) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			defered_failback_tick(vecs->mpvec);
 			retry_count_tick(vecs->mpvec);
+			lock_cleanup_pop(vecs->lock);
 		}
 		if (count)
 			count--;
 		else {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			condlog(4, "map garbage collection");
 			mpvec_garbage_collector(vecs);
 			count = MAPGCINT;
+			lock_cleanup_pop(vecs->lock);
 		}
 
-		lock_cleanup_pop(vecs->lock);
 		if (start_time.tv_sec &&
 		    gettimeofday(&end_time, NULL) == 0 &&
 		    num_paths) {
Signature · 1 line
1.8.4.5
#85 lore raw

PATCH 74/78 Allow specific CLI commands to run unlocked

In reply to Hannes Reinecke

When multipath is busy with checking paths or processing udev events it'll take the vector lock, causing the CLI
to become unresponsive.
This patch allows certain CLI commands to not wait for the vector lock, so that those commands will always succeed.

---
 multipathd/cli.c  | 26 +++++++++++++++++++++++++-
 multipathd/cli.h  |  2 ++
 multipathd/main.c | 17 ++++++-----------
 3 files changed, 33 insertions(+), 12 deletions(-)
multipathd/cli.c +25 -1
index acc4249..2d3d02d 100644
@@ -1,8 +1,11 @@
 /*
  * Copyright (c) 2005 Christophe Varoqui
  */
+#include <pthread.h>
 #include <memory.h>
 #include <vector.h>
+#include <structs.h>
+#include <structs_vec.h>
 #include <parser.h>
 #include <util.h>
 #include <version.h>
@@ -99,6 +102,19 @@ set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *
 	if (!h)
 		return 1;
 	h->fn = fn;
+	h->locked = 1;
+	return 0;
+}
+
+int
+set_unlocked_handler_callback (unsigned long fp,int (*fn)(void *, char **, int *, void *))
+{
+	struct handler * h = find_handler(fp);
+
+	if (!h)
+		return 1;
+	h->fn = fn;
+	h->locked = 0;
 	return 0;
 }
 
@@ -396,7 +412,15 @@ parse_cmd (char * cmd, char ** reply, int * len, void * data)
 	/*
 	 * execute handler
 	 */
-	r = h->fn(cmdvec, reply, len, data);
+	if (h->locked) {
+		struct vectors * vecs = (struct vectors *)data;
+		pthread_cleanup_push(cleanup_lock, &vecs->lock);
+		lock(vecs->lock);
+		pthread_testcancel();
+		r = h->fn(cmdvec, reply, len, data);
+		lock_cleanup_pop(vecs->lock);
+	} else
+		r = h->fn(cmdvec, reply, len, data);
 	free_keys(cmdvec);
 
 	return r;
multipathd/cli.h +2 -0
index 09fdc68..b35a315 100644
@@ -82,12 +82,14 @@ struct key {
 
 struct handler {
 	unsigned long fingerprint;
+	int locked;
 	int (*fn)(void *, char **, int *, void *);
 };
 
 int alloc_handlers (void);
 int add_handler (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
+int set_unlocked_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int parse_cmd (char * cmd, char ** reply, int * len, void *);
 int load_keys (void);
 char * get_keyparam (vector v, unsigned long code);
multipathd/main.c +6 -11
index 9e7bf4f..ab2a3a7 100644
@@ -773,10 +773,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	*len = 0;
 	vecs = (struct vectors *)trigger_data;
 
-	pthread_cleanup_push(cleanup_lock, &vecs->lock);
-	lock(vecs->lock);
-	pthread_testcancel();
-
 	r = parse_cmd(str, reply, len, vecs);
 
 	if (r > 0) {
@@ -791,7 +787,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	}
 	/* else if (r < 0) leave *reply alone */
 
-	lock_cleanup_pop(vecs->lock);
 	return r;
 }
 
@@ -903,16 +898,16 @@ uxlsnrloop (void * ap)
 	set_handler_callback(LIST+PATHS+FMT, cli_list_paths_fmt);
 	set_handler_callback(LIST+PATH, cli_list_path);
 	set_handler_callback(LIST+MAPS, cli_list_maps);
-	set_handler_callback(LIST+STATUS, cli_list_status);
-	set_handler_callback(LIST+DAEMON, cli_list_daemon);
+	set_unlocked_handler_callback(LIST+STATUS, cli_list_status);
+	set_unlocked_handler_callback(LIST+DAEMON, cli_list_daemon);
 	set_handler_callback(LIST+MAPS+STATUS, cli_list_maps_status);
 	set_handler_callback(LIST+MAPS+STATS, cli_list_maps_stats);
 	set_handler_callback(LIST+MAPS+FMT, cli_list_maps_fmt);
 	set_handler_callback(LIST+MAPS+TOPOLOGY, cli_list_maps_topology);
 	set_handler_callback(LIST+TOPOLOGY, cli_list_maps_topology);
 	set_handler_callback(LIST+MAP+TOPOLOGY, cli_list_map_topology);
-	set_handler_callback(LIST+CONFIG, cli_list_config);
-	set_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
+	set_unlocked_handler_callback(LIST+CONFIG, cli_list_config);
+	set_unlocked_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
 	set_handler_callback(LIST+DEVICES, cli_list_devices);
 	set_handler_callback(LIST+WILDCARDS, cli_list_wildcards);
 	set_handler_callback(ADD+PATH, cli_add_path);
@@ -932,8 +927,8 @@ uxlsnrloop (void * ap)
 	set_handler_callback(RESTOREQ+MAP, cli_restore_queueing);
 	set_handler_callback(DISABLEQ+MAPS, cli_disable_all_queueing);
 	set_handler_callback(RESTOREQ+MAPS, cli_restore_all_queueing);
-	set_handler_callback(QUIT, cli_quit);
-	set_handler_callback(SHUTDOWN, cli_shutdown);
+	set_unlocked_handler_callback(QUIT, cli_quit);
+	set_unlocked_handler_callback(SHUTDOWN, cli_shutdown);
 	set_handler_callback(GETPRSTATUS+MAP, cli_getprstatus);
 	set_handler_callback(SETPRSTATUS+MAP, cli_setprstatus);
 	set_handler_callback(UNSETPRSTATUS+MAP, cli_unsetprstatus);
Signature · 1 line
1.8.4.5
#86 lore raw

Re: [PATCH 74/78] Allow specific CLI commands to run unlocked

Quoted 5 lines · Hannes Reinecke

When multipath is busy with checking paths or processing udev events it'll take the vector lock, causing the CLI
to become unresponsive.
This patch allows certain CLI commands to not wait for the vector lock, so that those commands will always succeed.

This one looks like it can actually cause memory corruption.

Holding the vecs lock is necessary to protect the vecs vectors and conf, and the only unlocked handler that doesn't touch anything that vecs protects is cli_quit.

All the others at least call condlog, which calls dlog, which includes this line

thres = (conf) ? conf->verbosity : 0;

During a reconfigure, conf will temporarily be null, and it can happen
after the check. Now as the code stands at this patch, the cli
handlers are protected from concurrent reconfigure calls because reconfigure() is also called by uxsock_listen, so it can't run at the same time as a cli handler. But your later patch "multipathd:
asynchronous configuration" moves this handling to child(), and I don't see anything that will keep it from running reconfigure() while a cli_handler is running.

cli_reconfigure and cli_list_blacklist make use of conf for much more than the initial condlog, so they are even more at risk.

But the biggest problem is cli_list_status, which traverses vecs->pathvec without the vecs lock held. I don't see anything to stop, for instance, ev_add_path from running at the same time. This means that it's possible that vecs->pathvec will get realloced out from under cli_list_status.

-Ben

Quoted 137 lines
---
 multipathd/cli.c  | 26 +++++++++++++++++++++++++-
 multipathd/cli.h  |  2 ++
 multipathd/main.c | 17 ++++++-----------
 3 files changed, 33 insertions(+), 12 deletions(-)
multipathd/cli.c +25 -1
index acc4249..2d3d02d 100644
@@ -1,8 +1,11 @@
 /*
  * Copyright (c) 2005 Christophe Varoqui
  */
+#include <pthread.h>
 #include <memory.h>
 #include <vector.h>
+#include <structs.h>
+#include <structs_vec.h>
 #include <parser.h>
 #include <util.h>
 #include <version.h>
@@ -99,6 +102,19 @@ set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *
 	if (!h)
 		return 1;
 	h->fn = fn;
+	h->locked = 1;
+	return 0;
+}
+
+int
+set_unlocked_handler_callback (unsigned long fp,int (*fn)(void *, char **, int *, void *))
+{
+	struct handler * h = find_handler(fp);
+
+	if (!h)
+		return 1;
+	h->fn = fn;
+	h->locked = 0;
 	return 0;
 }
 
@@ -396,7 +412,15 @@ parse_cmd (char * cmd, char ** reply, int * len, void * data)
 	/*
 	 * execute handler
 	 */
-	r = h->fn(cmdvec, reply, len, data);
+	if (h->locked) {
+		struct vectors * vecs = (struct vectors *)data;
+		pthread_cleanup_push(cleanup_lock, &vecs->lock);
+		lock(vecs->lock);
+		pthread_testcancel();
+		r = h->fn(cmdvec, reply, len, data);
+		lock_cleanup_pop(vecs->lock);
+	} else
+		r = h->fn(cmdvec, reply, len, data);
 	free_keys(cmdvec);
 
 	return r;
multipathd/cli.h +2 -0
index 09fdc68..b35a315 100644
@@ -82,12 +82,14 @@ struct key {
 
 struct handler {
 	unsigned long fingerprint;
+	int locked;
 	int (*fn)(void *, char **, int *, void *);
 };
 
 int alloc_handlers (void);
 int add_handler (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
+int set_unlocked_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int parse_cmd (char * cmd, char ** reply, int * len, void *);
 int load_keys (void);
 char * get_keyparam (vector v, unsigned long code);
multipathd/main.c +6 -11
index 9e7bf4f..ab2a3a7 100644
@@ -773,10 +773,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	*len = 0;
 	vecs = (struct vectors *)trigger_data;
 
-	pthread_cleanup_push(cleanup_lock, &vecs->lock);
-	lock(vecs->lock);
-	pthread_testcancel();
-
 	r = parse_cmd(str, reply, len, vecs);
 
 	if (r > 0) {
@@ -791,7 +787,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	}
 	/* else if (r < 0) leave *reply alone */
 
-	lock_cleanup_pop(vecs->lock);
 	return r;
 }
 
@@ -903,16 +898,16 @@ uxlsnrloop (void * ap)
 	set_handler_callback(LIST+PATHS+FMT, cli_list_paths_fmt);
 	set_handler_callback(LIST+PATH, cli_list_path);
 	set_handler_callback(LIST+MAPS, cli_list_maps);
-	set_handler_callback(LIST+STATUS, cli_list_status);
-	set_handler_callback(LIST+DAEMON, cli_list_daemon);
+	set_unlocked_handler_callback(LIST+STATUS, cli_list_status);
+	set_unlocked_handler_callback(LIST+DAEMON, cli_list_daemon);
 	set_handler_callback(LIST+MAPS+STATUS, cli_list_maps_status);
 	set_handler_callback(LIST+MAPS+STATS, cli_list_maps_stats);
 	set_handler_callback(LIST+MAPS+FMT, cli_list_maps_fmt);
 	set_handler_callback(LIST+MAPS+TOPOLOGY, cli_list_maps_topology);
 	set_handler_callback(LIST+TOPOLOGY, cli_list_maps_topology);
 	set_handler_callback(LIST+MAP+TOPOLOGY, cli_list_map_topology);
-	set_handler_callback(LIST+CONFIG, cli_list_config);
-	set_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
+	set_unlocked_handler_callback(LIST+CONFIG, cli_list_config);
+	set_unlocked_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
 	set_handler_callback(LIST+DEVICES, cli_list_devices);
 	set_handler_callback(LIST+WILDCARDS, cli_list_wildcards);
 	set_handler_callback(ADD+PATH, cli_add_path);
@@ -932,8 +927,8 @@ uxlsnrloop (void * ap)
 	set_handler_callback(RESTOREQ+MAP, cli_restore_queueing);
 	set_handler_callback(DISABLEQ+MAPS, cli_disable_all_queueing);
 	set_handler_callback(RESTOREQ+MAPS, cli_restore_all_queueing);
-	set_handler_callback(QUIT, cli_quit);
-	set_handler_callback(SHUTDOWN, cli_shutdown);
+	set_unlocked_handler_callback(QUIT, cli_quit);
+	set_unlocked_handler_callback(SHUTDOWN, cli_shutdown);
 	set_handler_callback(GETPRSTATUS+MAP, cli_getprstatus);
 	set_handler_callback(SETPRSTATUS+MAP, cli_setprstatus);
 	set_handler_callback(UNSETPRSTATUS+MAP, cli_unsetprstatus);
Signature · 1 line
1.8.4.5
#87 lore raw

PATCH 75/78 Push down vector lock during uevent processing

In reply to Hannes Reinecke

When adding lots of paths the vector lock which is taken at the start of the uevent handler will become a bottleneck as it'll compete with the checkerloop.
So move the vector handling down into the individual event handler.

---
 multipathd/main.c | 67 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 48 insertions(+), 19 deletions(-)
multipathd/main.c +48 -19
index ab2a3a7..77a1241 100644
@@ -279,7 +279,11 @@ uev_add_map (struct uevent * uev, struct vectors * vecs)
 			return 1;
 		}
 	}
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	rc = ev_add_map(uev->kernel, alias, vecs);
+	lock_cleanup_pop(vecs->lock);
 	FREE(alias);
 	return rc;
 }
@@ -361,6 +365,10 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
 		return 0;
 	}
 	minor = uevent_get_minor(uev);
+
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	mpp = find_mp_by_minor(vecs->mpvec, minor);
 
 	if (!mpp) {
@@ -377,10 +385,12 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
 	orphan_paths(vecs->pathvec, mpp);
 	remove_map_and_stop_waiter(mpp, vecs, 1);
 out:
+	lock_cleanup_pop(vecs->lock);
 	FREE(alias);
 	return 0;
 }
 
+/* Called from CLI handler */
 int
 ev_remove_map (char * devname, char * alias, int minor, struct vectors * vecs)
 {
@@ -416,6 +426,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		return 1;
 	}
 
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
 	if (pp) {
 		int r;
@@ -443,8 +456,15 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 				ret = 1;
 			}
 		}
-		return ret;
 	}
+	/*
+	 * The linux implementation of pthread_lock() and pthread_unlock()
+	 * requires that both must be at the same indentation level,
+	 * hence the slightly odd coding.
+	 */
+	lock_cleanup_pop(vecs->lock);
+	if (pp)
+		return ret;
 
 	/*
 	 * get path vital state
@@ -462,6 +482,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		free_path(pp);
 		return 1;
 	}
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	ret = store_path(vecs->pathvec, pp);
 	if (!ret) {
 		pp->checkint = conf->checkint;
@@ -473,7 +496,7 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		free_path(pp);
 		ret = 1;
 	}
-
+	lock_cleanup_pop(vecs->lock);
 	return ret;
 }
 
@@ -535,12 +558,12 @@ rescan:
 			 */
 			start_waiter = 1;
 		}
-		else
+		if (!start_waiter)
 			goto fail; /* leave path added to pathvec */
 	}
 
-	/* persistent reseravtion check*/
-	mpath_pr_event_handle(pp);	
+	/* persistent reservation check*/
+	mpath_pr_event_handle(pp);
 
 	/*
 	 * push the map to the device-mapper
@@ -568,7 +591,7 @@ retry:
 		 * deal with asynchronous uevents :((
 		 */
 		if (mpp->action == ACT_RELOAD && retries-- > 0) {
-			condlog(0, "%s: uev_add_path sleep", mpp->alias);
+			condlog(0, "%s: ev_add_path sleep", mpp->alias);
 			sleep(1);
 			update_mpp_paths(mpp, vecs->pathvec);
 			goto rescan;
@@ -597,8 +620,7 @@ retry:
 		condlog(2, "%s [%s]: path added to devmap %s",
 			pp->dev, pp->dev_t, mpp->alias);
 		return 0;
-	}
-	else
+	} else
 		goto fail;
 
 fail_map:
@@ -612,17 +634,22 @@ static int
 uev_remove_path (struct uevent *uev, struct vectors * vecs)
 {
 	struct path *pp;
+	int ret;
 
 	condlog(2, "%s: remove path (uevent)", uev->kernel);
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
-
+	if (pp)
+		ret = ev_remove_path(pp, vecs);
+	lock_cleanup_pop(vecs->lock);
 	if (!pp) {
 		/* Not an error; path might have been purged earlier */
 		condlog(0, "%s: path already removed", uev->kernel);
 		return 0;
 	}
-
-	return ev_remove_path(pp, vecs);
+	return ret;
 }
 
 int
@@ -726,20 +753,27 @@ uev_update_path (struct uevent *uev, struct vectors * vecs)
 
 	if (ro >= 0) {
 		struct path * pp;
+		struct multipath *mpp = NULL;
 
 		condlog(2, "%s: update path write_protect to '%d' (uevent)",
 			uev->kernel, ro);
+		pthread_cleanup_push(cleanup_lock, &vecs->lock);
+		lock(vecs->lock);
+		pthread_testcancel();
 		pp = find_path_by_dev(vecs->pathvec, uev->kernel);
+		if (pp)
+			mpp = pp->mpp;
+		lock_cleanup_pop(vecs->lock);
 		if (!pp) {
 			condlog(0, "%s: spurious uevent, path not found",
 				uev->kernel);
 			return 1;
 		}
-		if (pp->mpp) {
-			retval = reload_map(vecs, pp->mpp, 0);
+		if (mpp) {
+			retval = reload_map(vecs, mpp, 0);
 
 			condlog(2, "%s: map %s reloaded (retval %d)",
-				uev->kernel, pp->mpp->alias, retval);
+				uev->kernel, mpp->alias, retval);
 		}
 
 	}
@@ -823,10 +857,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	if (uev_discard(uev->devpath))
 		return 0;
 
-	pthread_cleanup_push(cleanup_lock, &vecs->lock);
-	lock(vecs->lock);
-	pthread_testcancel();
-
 	/*
 	 * device map event
 	 * Add events are ignored here as the tables
@@ -865,7 +895,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	}
 
 out:
-	lock_cleanup_pop(vecs->lock);
 	return r;
 }
 
Signature · 1 line
1.8.4.5
#88 lore raw

Re: [PATCH 75/78] Push down vector lock during uevent processing

Quoted 5 lines · Hannes Reinecke

When adding lots of paths the vector lock which is taken at the start of the uevent handler will become a bottleneck as it'll compete with the checkerloop.
So move the vector handling down into the individual event handler.

Again, with these changes, there are places in these code paths where conf is dereferenced without vecs being held, and there's nothing to keep reconfigure from running at the same time as these functions.

-Ben

Quoted 205 lines
---
 multipathd/main.c | 67 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 48 insertions(+), 19 deletions(-)
multipathd/main.c +48 -19
index ab2a3a7..77a1241 100644
@@ -279,7 +279,11 @@ uev_add_map (struct uevent * uev, struct vectors * vecs)
 			return 1;
 		}
 	}
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	rc = ev_add_map(uev->kernel, alias, vecs);
+	lock_cleanup_pop(vecs->lock);
 	FREE(alias);
 	return rc;
 }
@@ -361,6 +365,10 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
 		return 0;
 	}
 	minor = uevent_get_minor(uev);
+
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	mpp = find_mp_by_minor(vecs->mpvec, minor);
 
 	if (!mpp) {
@@ -377,10 +385,12 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
 	orphan_paths(vecs->pathvec, mpp);
 	remove_map_and_stop_waiter(mpp, vecs, 1);
 out:
+	lock_cleanup_pop(vecs->lock);
 	FREE(alias);
 	return 0;
 }
 
+/* Called from CLI handler */
 int
 ev_remove_map (char * devname, char * alias, int minor, struct vectors * vecs)
 {
@@ -416,6 +426,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		return 1;
 	}
 
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
 	if (pp) {
 		int r;
@@ -443,8 +456,15 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 				ret = 1;
 			}
 		}
-		return ret;
 	}
+	/*
+	 * The linux implementation of pthread_lock() and pthread_unlock()
+	 * requires that both must be at the same indentation level,
+	 * hence the slightly odd coding.
+	 */
+	lock_cleanup_pop(vecs->lock);
+	if (pp)
+		return ret;
 
 	/*
 	 * get path vital state
@@ -462,6 +482,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		free_path(pp);
 		return 1;
 	}
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	ret = store_path(vecs->pathvec, pp);
 	if (!ret) {
 		pp->checkint = conf->checkint;
@@ -473,7 +496,7 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		free_path(pp);
 		ret = 1;
 	}
-
+	lock_cleanup_pop(vecs->lock);
 	return ret;
 }
 
@@ -535,12 +558,12 @@ rescan:
 			 */
 			start_waiter = 1;
 		}
-		else
+		if (!start_waiter)
 			goto fail; /* leave path added to pathvec */
 	}
 
-	/* persistent reseravtion check*/
-	mpath_pr_event_handle(pp);	
+	/* persistent reservation check*/
+	mpath_pr_event_handle(pp);
 
 	/*
 	 * push the map to the device-mapper
@@ -568,7 +591,7 @@ retry:
 		 * deal with asynchronous uevents :((
 		 */
 		if (mpp->action == ACT_RELOAD && retries-- > 0) {
-			condlog(0, "%s: uev_add_path sleep", mpp->alias);
+			condlog(0, "%s: ev_add_path sleep", mpp->alias);
 			sleep(1);
 			update_mpp_paths(mpp, vecs->pathvec);
 			goto rescan;
@@ -597,8 +620,7 @@ retry:
 		condlog(2, "%s [%s]: path added to devmap %s",
 			pp->dev, pp->dev_t, mpp->alias);
 		return 0;
-	}
-	else
+	} else
 		goto fail;
 
 fail_map:
@@ -612,17 +634,22 @@ static int
 uev_remove_path (struct uevent *uev, struct vectors * vecs)
 {
 	struct path *pp;
+	int ret;
 
 	condlog(2, "%s: remove path (uevent)", uev->kernel);
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
-
+	if (pp)
+		ret = ev_remove_path(pp, vecs);
+	lock_cleanup_pop(vecs->lock);
 	if (!pp) {
 		/* Not an error; path might have been purged earlier */
 		condlog(0, "%s: path already removed", uev->kernel);
 		return 0;
 	}
-
-	return ev_remove_path(pp, vecs);
+	return ret;
 }
 
 int
@@ -726,20 +753,27 @@ uev_update_path (struct uevent *uev, struct vectors * vecs)
 
 	if (ro >= 0) {
 		struct path * pp;
+		struct multipath *mpp = NULL;
 
 		condlog(2, "%s: update path write_protect to '%d' (uevent)",
 			uev->kernel, ro);
+		pthread_cleanup_push(cleanup_lock, &vecs->lock);
+		lock(vecs->lock);
+		pthread_testcancel();
 		pp = find_path_by_dev(vecs->pathvec, uev->kernel);
+		if (pp)
+			mpp = pp->mpp;
+		lock_cleanup_pop(vecs->lock);
 		if (!pp) {
 			condlog(0, "%s: spurious uevent, path not found",
 				uev->kernel);
 			return 1;
 		}
-		if (pp->mpp) {
-			retval = reload_map(vecs, pp->mpp, 0);
+		if (mpp) {
+			retval = reload_map(vecs, mpp, 0);
 
 			condlog(2, "%s: map %s reloaded (retval %d)",
-				uev->kernel, pp->mpp->alias, retval);
+				uev->kernel, mpp->alias, retval);
 		}
 
 	}
@@ -823,10 +857,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	if (uev_discard(uev->devpath))
 		return 0;
 
-	pthread_cleanup_push(cleanup_lock, &vecs->lock);
-	lock(vecs->lock);
-	pthread_testcancel();
-
 	/*
 	 * device map event
 	 * Add events are ignored here as the tables
@@ -865,7 +895,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	}
 
 out:
-	lock_cleanup_pop(vecs->lock);
 	return r;
 }
 
Signature · 1 line
1.8.4.5
#89 lore raw

PATCH 76/78 multipathd: timeout CLI commands when waiting for lock

In reply to Hannes Reinecke

When a CLI command is waiting for the vector lock it might timeout, but the command itself remains queued. This causes some irritation as no other commands can be send until the original command is processed.

---
 libmultipath/config.h    |  2 ++
 libmultipath/configure.c |  7 ++++---
 libmultipath/dict.c      | 11 +++++++----
 libmultipath/lock.h      |  4 ++--
 multipathd/cli.c         | 26 ++++++++++++++++++++++----
 multipathd/cli.h         |  2 +-
 multipathd/main.c        | 12 ++++++++----
 multipathd/main.h        |  1 +
 multipathd/uxlsnr.c      | 16 +++-------------
 9 files changed, 50 insertions(+), 31 deletions(-)
libmultipath/config.h +2 -0
index 0183969..e512321 100644
@@ -170,6 +170,8 @@ struct config {
 	vector elist_property;
 };
 
+extern int uxsock_timeout;
+
 struct config * conf;
 
 struct hwentry * find_hwe (vector hwtable, char * vendor, char * product, char *revision);
libmultipath/configure.c +4 -3
index 24ad948..3a90950 100644
@@ -701,7 +701,7 @@ deadmap (struct multipath * mpp)
 	return 1; /* dead */
 }
 
-int check_daemon(void)
+int check_daemon(int timeout)
 {
 	int fd;
 	char *reply;
@@ -714,7 +714,7 @@ int check_daemon(void)
 
 	if (send_packet(fd, "show daemon", 12) != 0)
 		goto out;
-	if (recv_packet(fd, &reply, &len, conf->uxsock_timeout) != 0)
+	if (recv_packet(fd, &reply, &len, timeout) != 0)
 		goto out;
 
 	if (strstr(reply, "shutdown"))
@@ -848,7 +848,8 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 		if (r == DOMAP_DRY)
 			continue;
 
-		if (!conf->daemon && !conf->allow_queueing && !check_daemon()) {
+		if (!conf->daemon && !conf->allow_queueing &&
+		    !check_daemon(uxsock_timeout)) {
 			if (mpp->no_path_retry != NO_PATH_RETRY_UNDEF &&
 			    mpp->no_path_retry != NO_PATH_RETRY_FAIL)
 				condlog(3, "%s: multipathd not running, unset "
libmultipath/dict.c +7 -4
index 5c2da43..ff06a93 100644
@@ -1037,23 +1037,26 @@ declare_mp_snprint(delay_wait_checks, print_delay_checks)
 static int
 def_uxsock_timeout_handler(vector strvec)
 {
-	unsigned int uxsock_timeout;
+	unsigned int timeout;
 	char *buff;
 
 	buff = set_value(strvec);
 	if (!buff)
 		return 1;
 
-	if (sscanf(buff, "%u", &uxsock_timeout) == 1 &&
-	    uxsock_timeout > DEFAULT_UXSOCK_TIMEOUT)
-		conf->uxsock_timeout = uxsock_timeout;
+	if (sscanf(buff, "%u", &timeout) == 1 &&
+	    timeout > DEFAULT_UXSOCK_TIMEOUT)
+		conf->uxsock_timeout = timeout;
 	else
 		conf->uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
 
 	free(buff);
+	uxsock_timeout = conf->uxsock_timeout;
 	return 0;
 }
 
+int uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
+
 /*
  * blacklist block handlers
  */
libmultipath/lock.h +2 -2
index 04ef78d..97af0f4 100644
@@ -21,11 +21,11 @@ struct mutex_lock {
 	a.depth--; pthread_mutex_unlock(a.mutex)
 #define lock_cleanup_pop(a) \
 		fprintf(stderr, "%s:%s(%i) unlock %p depth: %d (%ld)\n", __FILE__, __FUNCTION__, __LINE__, a.mutex, a.depth, pthread_self()); \
-	pthread_cleanup_pop(1);
+	pthread_cleanup_pop(1)
 #else
 #define lock(a) a.depth++; pthread_mutex_lock(a.mutex)
 #define unlock(a) a.depth--; pthread_mutex_unlock(a.mutex)
-#define lock_cleanup_pop(a) pthread_cleanup_pop(1);
+#define lock_cleanup_pop(a) pthread_cleanup_pop(1)
 #endif
 
 void cleanup_lock (void * data);
multipathd/cli.c +22 -4
index 2d3d02d..2c78ee4 100644
@@ -1,6 +1,7 @@
 /*
  * Copyright (c) 2005 Christophe Varoqui
  */
+#include <sys/time.h>
 #include <pthread.h>
 #include <memory.h>
 #include <vector.h>
@@ -386,11 +387,13 @@ genhelp_handler (void)
 }
 
 int
-parse_cmd (char * cmd, char ** reply, int * len, void * data)
+parse_cmd (char * cmd, char ** reply, int * len, void * data, int timeout )
 {
 	int r;
 	struct handler * h;
 	vector cmdvec = NULL;
+	struct timespec tmo;
+	struct timeval now;
 
 	r = get_cmdvec(cmd, &cmdvec);
 
@@ -412,12 +415,27 @@ parse_cmd (char * cmd, char ** reply, int * len, void * data)
 	/*
 	 * execute handler
 	 */
+	if (gettimeofday(&now, NULL) == 0) {
+		tmo.tv_sec = now.tv_sec + timeout;
+		tmo.tv_nsec = now.tv_usec * 1000;
+	} else {
+		tmo.tv_sec = 0;
+	}
 	if (h->locked) {
 		struct vectors * vecs = (struct vectors *)data;
+
 		pthread_cleanup_push(cleanup_lock, &vecs->lock);
-		lock(vecs->lock);
-		pthread_testcancel();
-		r = h->fn(cmdvec, reply, len, data);
+		if (tmo.tv_sec) {
+			vecs->lock.depth++;
+			r = pthread_mutex_timedlock(vecs->lock.mutex, &tmo);
+		} else {
+			lock(vecs->lock);
+			r = 0;
+		}
+		if (r == 0) {
+			pthread_testcancel();
+			r = h->fn(cmdvec, reply, len, data);
+		}
 		lock_cleanup_pop(vecs->lock);
 	} else
 		r = h->fn(cmdvec, reply, len, data);
multipathd/cli.h +1 -1
index b35a315..de62278 100644
@@ -90,7 +90,7 @@ int alloc_handlers (void);
 int add_handler (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int set_unlocked_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
-int parse_cmd (char * cmd, char ** reply, int * len, void *);
+int parse_cmd (char * cmd, char ** reply, int * len, void *, int);
 int load_keys (void);
 char * get_keyparam (vector v, unsigned long code);
 void free_keys (vector vec);
multipathd/main.c +8 -4
index 77a1241..6c98686 100644
@@ -807,10 +807,13 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	*len = 0;
 	vecs = (struct vectors *)trigger_data;
 
-	r = parse_cmd(str, reply, len, vecs);
+	r = parse_cmd(str, reply, len, vecs, uxsock_timeout / 1000);
 
 	if (r > 0) {
-		*reply = STRDUP("fail\n");
+		if (r == ETIMEDOUT)
+			*reply = STRDUP("timeout\n");
+		else
+			*reply = STRDUP("fail\n");
 		*len = strlen(*reply) + 1;
 		r = 1;
 	}
@@ -917,6 +920,7 @@ uevqloop (void * ap)
 
 	return NULL;
 }
+
 static void *
 uxlsnrloop (void * ap)
 {
@@ -2102,7 +2106,7 @@ main (int argc, char *argv[])
 		case 'k':
 			if (load_config(DEFAULT_CONFIGFILE, udev_new()))
 				exit(1);
-			uxclnt(optarg, conf->uxsock_timeout);
+			uxclnt(optarg, uxsock_timeout + 100);
 			exit(0);
 		case 'B':
 			conf->bindings_read_only = 1;
@@ -2126,7 +2130,7 @@ main (int argc, char *argv[])
 			optind++;
 		}
 		c += snprintf(c, s + CMDSIZE - c, "\n");
-		uxclnt(s, conf->uxsock_timeout);
+		uxclnt(s, uxsock_timeout + 100);
 		exit(0);
 	}
 
multipathd/main.h +1 -0
index 10378ef..1813633 100644
@@ -15,6 +15,7 @@ struct prout_param_descriptor;
 struct prin_resp;
 
 extern pid_t daemon_pid;
+extern int uxsock_timeout;
 
 void exit_daemon(void);
 const char * daemon_status(void);
multipathd/uxlsnr.c +3 -13
index 61ba49a..698641b 100644
@@ -125,7 +125,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 {
 	int ux_sock;
 	size_t len;
-	int rlen, timeout;
+	int rlen;
 	char *inbuf;
 	char *reply;
 	sigset_t mask;
@@ -142,8 +142,6 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 		return NULL;
 	}
 
-	timeout = conf->uxsock_timeout;
-
 	pthread_cleanup_push(uxsock_cleanup, NULL);
 
 	polls = (struct pollfd *)MALLOC(0);
@@ -154,14 +152,6 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 		struct client *c, *tmp;
 		int i, poll_count, num_clients;
 
-		/*
-		 * Store configuration timeout;
-		 * configuration might change during
-		 * the call to 'reconfigure'.
-		 */
-		if (conf)
-			timeout = conf->uxsock_timeout;
-
 		/* setup for a poll */
 		pthread_mutex_lock(&client_lock);
 		num_clients = 0;
@@ -221,7 +211,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 					start_time.tv_sec = 0;
 
 				if (recv_packet(c->fd, &inbuf, &len,
-						timeout) != 0) {
+						uxsock_timeout) != 0) {
 					dead_client(c);
 				} else {
 					inbuf[len - 1] = 0;
@@ -239,7 +229,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 						reply = NULL;
 					}
 					check_timeout(start_time, inbuf,
-						      timeout);
+						      uxsock_timeout);
 					FREE(inbuf);
 				}
 			}
Signature · 1 line
1.8.4.5
#90 lore raw

PATCH 77/78 multipathd: asynchronous configuration

In reply to Hannes Reinecke

For initial configuration multipathd waits it has synchronized with the existing setup. On larger systems this takes up quite some time (I've measured 80 seconds on a system with 1024 paths) causing systemd to stall and the system to fail booting. This patch makes the initial configuration asynchronous, and using the same codepath as the existing 'reconfigure' CLI command.

---
 multipathd/cli_handlers.c |  5 ++-
 multipathd/main.c         | 79 ++++++++++++++++++++++++++++-------------------
 multipathd/main.h         |  1 +
 3 files changed, 51 insertions(+), 34 deletions(-)
multipathd/cli_handlers.c +2 -3
index dc96c45..e51537e 100644
@@ -830,11 +830,10 @@ cli_switch_group(void * v, char ** reply, int * len, void * data)
 int
 cli_reconfigure(void * v, char ** reply, int * len, void * data)
 {
-	struct vectors * vecs = (struct vectors *)data;
-
 	condlog(2, "reconfigure (operator)");
 
-	return reconfigure(vecs);
+	post_config_state(DAEMON_CONFIGURE);
+	return 0;
 }
 
 int
multipathd/main.c +48 -31

167 lines not on this page · raw message

multipathd/main.h +1 -0
index 1813633..6edf4ab 100644
@@ -25,6 +25,7 @@ int ev_remove_path (struct path *, struct vectors *);
 int ev_add_map (char *, char *, struct vectors *);
 int ev_remove_map (char *, char *, int, struct vectors *);
 void sync_map_state (struct multipath *);
+void post_config_state(enum daemon_status);
 void * mpath_alloc_prin_response(int prin_sa);
 int prin_do_scsi_ioctl(char *, int rq_servact, struct prin_resp * resp,
        int noisy);
Signature · 1 line
1.8.4.5
#91 lore raw

Re: [PATCH 77/78] multipathd: asynchronous configuration

Quoted 7 lines · Hannes Reinecke

For initial configuration multipathd waits it has synchronized with the existing setup. On larger systems this takes up quite some time (I've measured 80 seconds on a system with 1024 paths) causing systemd to stall and the system to fail booting. This patch makes the initial configuration asynchronous, and using the same codepath as the existing 'reconfigure' CLI command.

If the issue is that configure takes too long, and is keeping multipath from resetting the watchdog timer, can't this problem still happen?

reconfigure still is called under the vecs lock, and checkerloop still locks the vecs lock, so a call to reconfigure can still stall the checker loop for just as long.

Or am I missing something?
-Ben

Quoted 223 lines
---
 multipathd/cli_handlers.c |  5 ++-
 multipathd/main.c         | 79 ++++++++++++++++++++++++++++-------------------
 multipathd/main.h         |  1 +
 3 files changed, 51 insertions(+), 34 deletions(-)
multipathd/cli_handlers.c +2 -3
index dc96c45..e51537e 100644
@@ -830,11 +830,10 @@ cli_switch_group(void * v, char ** reply, int * len, void * data)
 int
 cli_reconfigure(void * v, char ** reply, int * len, void * data)
 {
-	struct vectors * vecs = (struct vectors *)data;
-
 	condlog(2, "reconfigure (operator)");
 
-	return reconfigure(vecs);
+	post_config_state(DAEMON_CONFIGURE);
+	return 0;
 }
 
 int
multipathd/main.c +48 -31
index 6c98686..d9f2435 100644
@@ -93,10 +93,11 @@ struct mpath_event_param
 unsigned int mpath_mx_alloc_len;
 
 int logsink;
-enum daemon_status running_state;
+enum daemon_status running_state = DAEMON_INIT;
 pid_t daemon_pid;
+pthread_mutex_t config_lock = PTHREAD_MUTEX_INITIALIZER;
+pthread_cond_t config_cond = PTHREAD_COND_INITIALIZER;
 
-static sem_t exit_sem;
 /*
  * global copy of vecs for use in sig handlers
  */
@@ -104,6 +105,21 @@ struct vectors * gvecs;
 
 struct udev * udev;
 
+static void config_cleanup(void *arg)
+{
+	pthread_mutex_unlock(&config_lock);
+}
+
+void post_config_state(enum daemon_status state)
+{
+	pthread_mutex_lock(&config_lock);
+	if (state != running_state) {
+		running_state = state;
+		pthread_cond_broadcast(&config_cond);
+	}
+	pthread_mutex_unlock(&config_lock);
+}
+
 static int
 need_switch_pathgroup (struct multipath * mpp, int refresh)
 {
@@ -860,6 +876,15 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	if (uev_discard(uev->devpath))
 		return 0;
 
+	pthread_cleanup_push(config_cleanup, NULL);
+	pthread_mutex_lock(&config_lock);
+	if (running_state != DAEMON_RUNNING)
+		pthread_cond_wait(&config_cond, &config_lock);
+	pthread_cleanup_pop(1);
+
+	if (running_state == DAEMON_SHUTDOWN)
+		return 0;
+
 	/*
 	 * device map event
 	 * Add events are ignored here as the tables
@@ -948,7 +973,7 @@ uxlsnrloop (void * ap)
 	set_handler_callback(ADD+MAP, cli_add_map);
 	set_handler_callback(DEL+MAP, cli_del_map);
 	set_handler_callback(SWITCH+MAP+GROUP, cli_switch_group);
-	set_handler_callback(RECONFIGURE, cli_reconfigure);
+	set_unlocked_handler_callback(RECONFIGURE, cli_reconfigure);
 	set_handler_callback(SUSPEND+MAP, cli_suspend);
 	set_handler_callback(RESUME+MAP, cli_resume);
 	set_handler_callback(RESIZE+MAP, cli_resize);
@@ -977,7 +1002,7 @@ uxlsnrloop (void * ap)
 void
 exit_daemon (void)
 {
-	sem_post(&exit_sem);
+	post_config_state(DAEMON_SHUTDOWN);
 }
 
 const char *
@@ -1561,8 +1586,6 @@ reconfigure (struct vectors * vecs)
 	struct config * old = conf;
 	int retval = 1;
 
-	running_state = DAEMON_CONFIGURE;
-
 	/*
 	 * free old map and path vectors ... they use old conf state
 	 */
@@ -1587,7 +1610,7 @@ reconfigure (struct vectors * vecs)
 		retval = 0;
 	}
 
-	running_state = DAEMON_RUNNING;
+	post_config_state(DAEMON_RUNNING);
 
 	return retval;
 }
@@ -1643,12 +1666,7 @@ handle_signals(void)
 {
 	if (reconfig_sig && running_state == DAEMON_RUNNING) {
 		condlog(2, "reconfigure (signal)");
-		pthread_cleanup_push(cleanup_lock,
-				&gvecs->lock);
-		lock(gvecs->lock);
-		pthread_testcancel();
-		reconfigure(gvecs);
-		lock_cleanup_pop(gvecs->lock);
+		post_config_state(DAEMON_CONFIGURE);
 	}
 	if (log_reset_sig) {
 		condlog(2, "reset log (signal)");
@@ -1781,7 +1799,6 @@ child (void * param)
 	char *envp;
 
 	mlockall(MCL_CURRENT | MCL_FUTURE);
-	sem_init(&exit_sem, 0, 0);
 	signal_init();
 
 	udev = udev_new();
@@ -1796,7 +1813,7 @@ child (void * param)
 		pthread_attr_destroy(&log_attr);
 	}
 
-	running_state = DAEMON_START;
+	post_config_state(DAEMON_START);
 
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=startup");
@@ -1891,15 +1908,7 @@ child (void * param)
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=configure");
 #endif
-	running_state = DAEMON_CONFIGURE;
-
-	lock(vecs->lock);
-	if (configure(vecs, 1)) {
-		unlock(vecs->lock);
-		condlog(0, "failure during configuration");
-		goto failed;
-	}
-	unlock(vecs->lock);
+	post_config_state(DAEMON_CONFIGURE);
 
 	/*
 	 * start threads
@@ -1918,20 +1927,29 @@ child (void * param)
 	pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid);
 	/* Ignore errors, we can live without */
 
-	running_state = DAEMON_RUNNING;
 #ifdef USE_SYSTEMD
 	sd_notify(0, "READY=1\nSTATUS=running");
 #endif
 
-	/*
-	 * exit path
-	 */
-	while(sem_wait(&exit_sem) != 0); /* Do nothing */
+	while (running_state != DAEMON_SHUTDOWN) {
+		pthread_cleanup_push(config_cleanup, NULL);
+		pthread_mutex_lock(&config_lock);
+		if (running_state == DAEMON_RUNNING) {
+			pthread_cond_wait(&config_cond, &config_lock);
+		}
+		pthread_cleanup_pop(1);
+		if (running_state == DAEMON_CONFIGURE) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
+			reconfigure(vecs);
+			lock_cleanup_pop(vecs->lock);
+		}
+	}
 
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=shutdown");
 #endif
-	running_state = DAEMON_SHUTDOWN;
 	lock(vecs->lock);
 	if (conf->queue_without_daemon == QUE_NO_DAEMON_OFF)
 		vector_foreach_slot(vecs->mpvec, mpp, i)
@@ -2066,7 +2084,6 @@ main (int argc, char *argv[])
 	int foreground = 0;
 
 	logsink = 1;
-	running_state = DAEMON_INIT;
 	dm_init();
 
 	if (getuid() != 0) {
multipathd/main.h +1 -0
index 1813633..6edf4ab 100644
@@ -25,6 +25,7 @@ int ev_remove_path (struct path *, struct vectors *);
 int ev_add_map (char *, char *, struct vectors *);
 int ev_remove_map (char *, char *, int, struct vectors *);
 void sync_map_state (struct multipath *);
+void post_config_state(enum daemon_status);
 void * mpath_alloc_prin_response(int prin_sa);
 int prin_do_scsi_ioctl(char *, int rq_servact, struct prin_resp * resp,
        int noisy);
Signature · 1 line
1.8.4.5
#92 lore raw

Re: [PATCH 77/78] multipathd: asynchronous configuration

Quoted 9 lines · Benjamin Marzinski
Quoted 7 lines · Hannes Reinecke

For initial configuration multipathd waits it has synchronized with the existing setup. On larger systems this takes up quite some time (I've measured 80 seconds on a system with 1024 paths) causing systemd to stall and the system to fail booting. This patch makes the initial configuration asynchronous, and using the same codepath as the existing 'reconfigure' CLI command.

Quoted 8 lines

If the issue is that configure takes too long, and is keeping multipath from resetting the watchdog timer, can't this problem still happen?

reconfigure still is called under the vecs lock, and checkerloop still locks the vecs lock, so a call to reconfigure can still stall the checker loop for just as long.

Or am I missing something?

The problem here is that on large configurations multipathd might spend too much time in configure(), so the sd_notify() call with 'RUNNING' is would be later than the default systemd service timeout. Hence multipathd will be killed by systemd with a job timeout, with no chance to get it to run, ever.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#93 lore raw

PATCH 78/78 multipathd: trigger all devices on startup

In reply to Hannes Reinecke

During startup multipathd might race with udev and device discovery. During that time any information from libudev might not be fully available, leading to spurious multipathd failures during startup. So instead of scanning all devices on our own we should just re-trigger the existing devices; with that we'll read _all_ devices via uevents during startup and avoid the race condition.

---
 libmultipath/discovery.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++
 libmultipath/discovery.h |  1 +
 multipathd/main.c        | 36 +++++++++++++++++++++++++++++----
 3 files changed, 85 insertions(+), 4 deletions(-)
libmultipath/discovery.c +52 -0
index 4582a20..8762819 100644
@@ -134,6 +134,58 @@ path_discover (vector pathvec, struct config * conf,
 }
 
 int
+path_trigger (struct config * conf, int flag)
+{
+	struct udev_enumerate *udev_iter;
+	struct udev_list_entry *entry;
+	int num_paths = 0;
+
+	udev_iter = udev_enumerate_new(conf->udev);
+	if (!udev_iter)
+		return -ENOMEM;
+
+	udev_enumerate_add_match_subsystem(udev_iter, "block");
+	udev_enumerate_scan_devices(udev_iter);
+
+	udev_list_entry_foreach(entry,
+				udev_enumerate_get_list_entry(udev_iter)) {
+		const char *devpath;
+		char *devname;
+		char filename[PATH_MAX];
+		int fd;
+
+		devpath = udev_list_entry_get_name(entry);
+		condlog(3, "Trigger device %s", devpath);
+		devname = strrchr(devpath, '/');
+		if (!devname) {
+			condlog(3, "%s: invalid devpath", devpath);
+			continue;
+		}
+		devname++;
+		if (filter_devnode(conf->blist_devnode,
+				   conf->elist_devnode, devname) > 0) {
+			condlog(3, "%s: blacklisted", devname);
+			continue;
+		}
+		strncpy(filename, devpath, strlen(devpath) + 1);
+		strncat(filename, "/uevent", 8);
+		fd = open(filename, O_WRONLY | O_CLOEXEC);
+		if (fd < 0)
+			continue;
+		if (write(fd, "add", 3) < 0) {
+			condlog(3, "%s: Failed to trigger 'add' uevent: %m",
+				devpath);
+		} else {
+			num_paths++;
+		}
+		close(fd);
+	}
+	udev_enumerate_unref(udev_iter);
+	condlog(4, "Triggered %d paths", num_paths);
+	return num_paths;
+}
+
+int
 path_discovery (vector pathvec, struct config * conf, int flag)
 {
 	struct udev_enumerate *udev_iter;
libmultipath/discovery.h +1 -0
index da7652c..9ae3d23 100644
@@ -31,6 +31,7 @@
 struct config;
 
 int path_discovery (vector pathvec, struct config * conf, int flag);
+int path_trigger (struct config * conf, int flag);
 
 int do_tur (char *);
 int path_offline (struct path *);
multipathd/main.c +32 -4
index d9f2435..480b10d 100644
@@ -1581,6 +1581,24 @@ configure (struct vectors * vecs, int start_waiters)
 }
 
 int
+trigger_devices (struct vectors * vecs)
+{
+
+	if (!vecs->pathvec && !(vecs->pathvec = vector_alloc()))
+		return 1;
+
+	if (!vecs->mpvec && !(vecs->mpvec = vector_alloc()))
+		return 1;
+
+	/*
+	 * Trigger all non-blacklisted block devices
+	 */
+	path_trigger(conf, DI_ALL);
+
+	return 0;
+}
+
+int
 reconfigure (struct vectors * vecs)
 {
 	struct config * old = conf;
@@ -1902,15 +1920,21 @@ child (void * param)
 		condlog(0, "failed to create cli listener: %d", rc);
 		goto failed;
 	}
-	/*
-	 * fetch and configure both paths and multipaths
-	 */
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=configure");
 #endif
 	post_config_state(DAEMON_CONFIGURE);
 
 	/*
+	 * Trigger all paths to force reconfiguration
+	 */
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
+	trigger_devices(vecs);
+	lock_cleanup_pop(vecs->lock);
+
+	/*
 	 * start threads
 	 */
 	if ((rc = pthread_create(&check_thr, &misc_attr, checkerloop, vecs))) {
@@ -1927,6 +1951,8 @@ child (void * param)
 	pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid);
 	/* Ignore errors, we can live without */
 
+	post_config_state(DAEMON_RUNNING);
+
 #ifdef USE_SYSTEMD
 	sd_notify(0, "READY=1\nSTATUS=running");
 #endif
@@ -2129,7 +2155,9 @@ main (int argc, char *argv[])
 			conf->bindings_read_only = 1;
 			break;
 		default:
-			;
+			fprintf(stderr, "Invalid argument '-%c'\n",
+				optopt);
+			exit(1);
 		}
 	}
 	if (optind < argc) {
Signature · 1 line
1.8.4.5
#94 lore raw

Re: [PATCH 78/78] multipathd: trigger all devices on startup

Quoted 6 lines · Hannes Reinecke

During startup multipathd might race with udev and device discovery. During that time any information from libudev might not be fully available, leading to spurious multipathd failures during startup. So instead of scanning all devices on our own we should just re-trigger the existing devices; with that we'll read _all_ devices via uevents during startup and avoid the race condition.

Should we also be sending change uevents for the dm devices?

-Ben

Quoted 159 lines
---
 libmultipath/discovery.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++
 libmultipath/discovery.h |  1 +
 multipathd/main.c        | 36 +++++++++++++++++++++++++++++----
 3 files changed, 85 insertions(+), 4 deletions(-)
libmultipath/discovery.c +52 -0
index 4582a20..8762819 100644
@@ -134,6 +134,58 @@ path_discover (vector pathvec, struct config * conf,
 }
 
 int
+path_trigger (struct config * conf, int flag)
+{
+	struct udev_enumerate *udev_iter;
+	struct udev_list_entry *entry;
+	int num_paths = 0;
+
+	udev_iter = udev_enumerate_new(conf->udev);
+	if (!udev_iter)
+		return -ENOMEM;
+
+	udev_enumerate_add_match_subsystem(udev_iter, "block");
+	udev_enumerate_scan_devices(udev_iter);
+
+	udev_list_entry_foreach(entry,
+				udev_enumerate_get_list_entry(udev_iter)) {
+		const char *devpath;
+		char *devname;
+		char filename[PATH_MAX];
+		int fd;
+
+		devpath = udev_list_entry_get_name(entry);
+		condlog(3, "Trigger device %s", devpath);
+		devname = strrchr(devpath, '/');
+		if (!devname) {
+			condlog(3, "%s: invalid devpath", devpath);
+			continue;
+		}
+		devname++;
+		if (filter_devnode(conf->blist_devnode,
+				   conf->elist_devnode, devname) > 0) {
+			condlog(3, "%s: blacklisted", devname);
+			continue;
+		}
+		strncpy(filename, devpath, strlen(devpath) + 1);
+		strncat(filename, "/uevent", 8);
+		fd = open(filename, O_WRONLY | O_CLOEXEC);
+		if (fd < 0)
+			continue;
+		if (write(fd, "add", 3) < 0) {
+			condlog(3, "%s: Failed to trigger 'add' uevent: %m",
+				devpath);
+		} else {
+			num_paths++;
+		}
+		close(fd);
+	}
+	udev_enumerate_unref(udev_iter);
+	condlog(4, "Triggered %d paths", num_paths);
+	return num_paths;
+}
+
+int
 path_discovery (vector pathvec, struct config * conf, int flag)
 {
 	struct udev_enumerate *udev_iter;
libmultipath/discovery.h +1 -0
index da7652c..9ae3d23 100644
@@ -31,6 +31,7 @@
 struct config;
 
 int path_discovery (vector pathvec, struct config * conf, int flag);
+int path_trigger (struct config * conf, int flag);
 
 int do_tur (char *);
 int path_offline (struct path *);
multipathd/main.c +32 -4
index d9f2435..480b10d 100644
@@ -1581,6 +1581,24 @@ configure (struct vectors * vecs, int start_waiters)
 }
 
 int
+trigger_devices (struct vectors * vecs)
+{
+
+	if (!vecs->pathvec && !(vecs->pathvec = vector_alloc()))
+		return 1;
+
+	if (!vecs->mpvec && !(vecs->mpvec = vector_alloc()))
+		return 1;
+
+	/*
+	 * Trigger all non-blacklisted block devices
+	 */
+	path_trigger(conf, DI_ALL);
+
+	return 0;
+}
+
+int
 reconfigure (struct vectors * vecs)
 {
 	struct config * old = conf;
@@ -1902,15 +1920,21 @@ child (void * param)
 		condlog(0, "failed to create cli listener: %d", rc);
 		goto failed;
 	}
-	/*
-	 * fetch and configure both paths and multipaths
-	 */
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=configure");
 #endif
 	post_config_state(DAEMON_CONFIGURE);
 
 	/*
+	 * Trigger all paths to force reconfiguration
+	 */
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
+	trigger_devices(vecs);
+	lock_cleanup_pop(vecs->lock);
+
+	/*
 	 * start threads
 	 */
 	if ((rc = pthread_create(&check_thr, &misc_attr, checkerloop, vecs))) {
@@ -1927,6 +1951,8 @@ child (void * param)
 	pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid);
 	/* Ignore errors, we can live without */
 
+	post_config_state(DAEMON_RUNNING);
+
 #ifdef USE_SYSTEMD
 	sd_notify(0, "READY=1\nSTATUS=running");
 #endif
@@ -2129,7 +2155,9 @@ main (int argc, char *argv[])
 			conf->bindings_read_only = 1;
 			break;
 		default:
-			;
+			fprintf(stderr, "Invalid argument '-%c'\n",
+				optopt);
+			exit(1);
 		}
 	}
 	if (optind < argc) {
Signature · 1 line
1.8.4.5
#95 lore raw

Re: [PATCH 78/78] multipathd: trigger all devices on startup

Quoted 10 lines · Benjamin Marzinski
Quoted 6 lines · Hannes Reinecke

During startup multipathd might race with udev and device discovery. During that time any information from libudev might not be fully available, leading to spurious multipathd failures during startup. So instead of scanning all devices on our own we should just re-trigger the existing devices; with that we'll read _all_ devices via uevents during startup and avoid the race condition.

Should we also be sending change uevents for the dm devices?

No. We just need to retrigger the 'sd' devices, as with them we'll be reconfiguring the dm devices anyway.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#96 lore raw
In reply to Hannes Reinecke

Hi,

This patchset is merged up to 75734af37267100778468319813cecc050befb1a : multipathd: push down lock in checkerloop().

Ben,
incidentally the find_multipath patches are in.

Best regards,
Christophe Varoqui

Quoted 202 lines · Hannes Reinecke

Hi Christophe,

here are the patches which I've queued up during SLES12 development. Most of them are (more or less) obvious bugfixes, but there are some things which would warrant a deeper review:
- device-mapper cookie handling: This _really_ is an arcane topic, and I would love to have someone from the device-mapper team to review it. I'm still facing some issues where udev emits a warning 'conflicting device node /dev/mapper/XXX ..', so there might be some issue with it.
- Asynchronous configuration: On large systems configure() as being called during multipathd startup might exceed the systemd job timeout, resulting in multipathd being killed by systemd. For that I've been implementing an udev retrigger method, which just issues an 'add' event for all available block devices.
- Related to the above: I've implemented an asynchronous 'reconfigure' CLI call, as for the same reason systemd might kill the multipathd during a 'restart' operation
- systemd/udev integration: This is _a pain_. It took me months to come up with a working solution. _Especially_ as apparently RH is shipping an '11-dm-multipath.rules' udev ruleset for multipathd, which supposedly should be handling this.
It didn't do it for me, though, so I've attached my own version here. The udev integration I've been settling for works like this:
- 11-dm-mpath.rules checks if the device is eligible for multipathing For that I've implemented a new option '-u' which looks at the uevent variables _only_.
- if a device should be handled by multipathing the 'by-id' and 'by-uuid' persistent symlinks are _not_ generated, as this would trigger systemd to start scanning/mounting/lvm activation/whatnot; the 'by-id' and 'by-uuid' symlinks are only generated for the multipath devices themselves.
- I've added a new multipath.rules file which should be inserted between 55-scsi-sg3_id.rules and 58-scsi_sg3_symlink.rules. This will intercept the SCSI IDENT variables and blank them out if the (block) device is handled by multipathing.
- sysfs VPD page handling. I've planned to implement this as a direct replacement of the original scsi_id program, but then it turned out that our WWID handling is actually too generic for that: Both getuid and uid_attribute can basically take _any_ value, so it's impossible to tell if the system uses the original (scsi_id compatible) IDs or not. And as the VPD parsing code can only generate scsi_id compatible IDs we might run into compability issues.
So it's disabled per default. I'd really would like to clean it up, as I suspect that about 95% of the installations are using scsi_id compatible IDs. But for this to happen we need to clarify/restrict the ID generation mechanism.

As usual, comments and reviews are welcome.

The patchset is based upon the 'find_multipaths' patchset from Ben Marzinski; it can be retrieved from

github.com:/hreinecke/multipath-tools.git branch suse-resync

Cheers,

Hannes

Hannes Reinecke (78):
libmultipath: remove compilation warning in devmapper.c
mpath_persist: Do not call exit() from a shared library
libmultipath: filter for missing property in get_refwwid() Double uevent stacksize yet again
discovery: do not fail discovery on individual devices
libmultipath: Prefer deprecated 'getuid' callout
libmultipath: Skip paths with empty wwid
Make systemd installation path configurable
Add multipath rules for systemd support
Fixup multipathd.socket to resolve ordering dependeny Fixup dependencies in multipathd.service
multipathd: set correct PID when running in debug mode Do not print empty device strings during discovery kpartx.rules: do not call blkid
Use 'SCSI_IDENT_.*' as the default property whitelist Fixup wwid blacklist printing
Allow for empty path argument when printing information Disable reassign maps per default
multipathd: implement 'list path <path>' cli command Make checker_put() and prio_put() idempotent Remove trailing linefeed from sysfs attributes
multipath: implement option '-u' for uevents Install multipath rule under '56-multipath.rules' multipath.rules: Whitelist devices multipath.rules: fixup race condition with systemd 11-dm-mpath.rules: Import blkid values if all paths are down kpartx.rules: Skip kpartx for multipath events
multipathd: handle DOMAP_RETRY
multipathd: cleanup foreground operation
Update hwtable for EMC XtremIO
multipath: check for running daemon when called with '-u' Revert 'return PATH_DOWN for quiesced paths'
Do not treat 'transport-offline' paths as 'offline' Check for valid DM_DEVICE_INFO before proceeding Separate out uevent parsing functions
Use poll() when receiving uevents
mpath_persist: cleanup
kpartx: use standard 'major' and 'minor' macros
multipath: Use standard 'major' macro
Remove sysfs_get_dev
Add paths with a size of '0' as 'ghost' paths Remove last argument from verify_paths()
Fixup device-mapper 'cookie' handling
multipath: do not print state 'orphan' for option '-l' Return error when receiving CLI packet
Implement 'uxsock_timeout' keyword
Do not print empty multipaths section
multipathd: reload map if reinstate failed
multipathd: do not remove paths without uevents Allow zero-sized devices during configuration Rework uev_add_path()
multipathd: Issue warning on CLI command timeout Use strlen() when checking for valid wwid
multipathd: Use standard lists for CLI handling
uxlsnr: use typedef for trigger function
multipathd: lock cli client list
multipath: enable sync support
Remove dm_udev_XXX wrapper functions
Revert to ACT_RELOAD in domap if the map exists
multipathd: use local variable for watchdog configuration Ignore devices when sysfs_get_tgt_nodename fails Skip USB devices during discovery
Read wwid from sysfs vpg_pg83 attribute
Assign local priority for NAA VPD descriptor
Use sysfs attribute vpd_pg80 to read serial number Update multipath.conf.5 to clarify wwid generation
libmultipath: Fall back to SG_IO if no UID could be assigned
libmultipath: unset 'uid_attribute' on failure Separate out vpd parsing functions
multipathd: use SG_IO as fallback to generate uid Do not automatically fall back to vpd uid generation
libmultipath: make vpd page 0x80 optional
multipathd: push down lock in checkerloop()
Allow specific CLI commands to run unlocked
Push down vector lock during uevent processing
multipathd: timeout CLI commands when waiting for lock
multipathd: asynchronous configuration
multipathd: trigger all devices on startup

 Makefile.inc                     |   8 +-
 kpartx/devmapper.c               |  53 +++-
 kpartx/devmapper.h               |  11 +-
 kpartx/kpartx.c                  |  35 +--
 kpartx/kpartx.rules              |  10 +-
 libmpathpersist/mpath_persist.c  | 172 ++++++-----
 libmpathpersist/mpath_updatepr.c |  20 +-
 libmpathpersist/mpathpr.h        |   3 +-
 libmultipath/blacklist.c         |   8 +-
 libmultipath/blacklist.h         |   2 +-
 libmultipath/checkers.c          |   2 +-
 libmultipath/config.c            |   2 +
 libmultipath/config.h            |   8 +-
 libmultipath/configure.c         |  82 ++++--
 libmultipath/defaults.h          |   4 +-
 libmultipath/devmapper.c         |  77 +++--
 libmultipath/devmapper.h         |   4 +-
 libmultipath/dict.c              |  33 +++
 libmultipath/discovery.c         | 613
++++++++++++++++++++++++++++++++++-----
 libmultipath/discovery.h         |   8 +-
 libmultipath/hwtable.c           |  13 +
 libmultipath/list.h              |  49 +++-
 libmultipath/lock.h              |   4 +-
 libmultipath/log_pthread.c       |   2 +-
 libmultipath/print.c             |  20 +-
 libmultipath/prio.c              |   4 +-
 libmultipath/propsel.c           |   6 +-
 libmultipath/structs_vec.c       |  29 +-
 libmultipath/structs_vec.h       |   2 +-
 libmultipath/sysfs.c             |  53 ++++
 libmultipath/sysfs.h             |   2 +
 libmultipath/uevent.c            | 316 ++++++++++++--------
 libmultipath/uxsock.c            |  32 +-
 libmultipath/uxsock.h            |   4 +-
 multipath.conf.defaults          |   2 +-
 multipath/11-dm-mpath.rules      |   8 +-
 multipath/Makefile               |   2 +
 multipath/dev_t.h                |   3 -
 multipath/main.c                 |  53 +++-
 multipath/multipath.8            |   6 +-
 multipath/multipath.conf.5       |  42 ++-
 multipath/multipath.rules        |  14 +
 multipathd/cli.c                 |  47 ++-
 multipathd/cli.h                 |   4 +-
 multipathd/cli_handlers.c        |  64 +++-
 multipathd/cli_handlers.h        |   1 +
 multipathd/main.c                | 400 ++++++++++++++++---------
 multipathd/main.h                |   2 +
 multipathd/multipathd.service    |   4 +-
 multipathd/multipathd.socket     |   8 +-
 multipathd/uxclnt.c              |  26 +-
 multipathd/uxclnt.h              |   2 +-
 multipathd/uxlsnr.c              | 102 +++++--
 multipathd/uxlsnr.h              |   8 +-
 54 files changed, 1834 insertions(+), 655 deletions(-)
 delete mode 100644 multipath/dev_t.h
 create mode 100644 multipath/multipath.rules

--
1.8.4.5

#97 lore raw
Quoted 9 lines · Christophe Varoqui

Hi,

This patchset is merged up
to 75734af37267100778468319813cecc050befb1a : multipathd: push down lock in checkerloop().

Ben,
incidentally the find_multipath patches are in.

Thanks. I'll be working on the remaining ones to address the objections Ben raised.

Cheers,

Hannes

Signature · 10 lines
Dr. Hannes Reinecke		               zSeries & Storage
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel