multipath-tools series part V: removed path handling
cover multipath-tools series part V: removed path handling
From: Martin Wilck <[email protected]>
Hi Christophe, hi Ben,
This is part V of a larger patch series for multipath-tools I've been preparing. It's based on the previously submitted part IV.
The full series will also be available here:
https://github.com/mwilck/multipath-tools/tree/ups/submit-2007
There are tags in that repo for each part of the series. This part is tagged "submit-200709-5".
The series addresses several issues with missing, unintialized, and removed paths. The motivation was that I strongly dislike the side effects of disassemble_map(), adding paths and setting WWIDs while parsing map parameters. IMO this has always been a layering violation. This patch set adds a new function dedicated to handling paths which have not been detected via udev but are present as members of dm maps. That makes it much easier to track and improve the logic applied when such devices are detected.
I believe that the new logic will also fix the issue recently reported by Chongyun ("libmultipath/dmparser: add missing path with good status when sync state with dm kernel"). At least, the approach is very similar, although the call to pathinfo() for new devices now happens in update_pathvec_from_dm() rather than in disassemble_map().
The patch set also gets rid of the "is_daemon" argument for disassemble_map(), causing semantics in daemon and non-daemon mode. The reason for this parameter goes way back into history; it was avoiding to re-add manually removed paths because they were still showing up in maps. But OTOH we must add missing devices which we've failed to detect. The patch set handles this by tracking the state of "being removed" as a new init state, and only actually removing the paths from pathvec when they don't show up in maps any more.
Regards
Martin
Martin Wilck (21):
libmultipath: protect use of pp->udev
libmultipath: add uninitialize_path()
multipath-tools: introduce INIT_REMOVED state
libmultipath: update_mpp_paths(): handle INIT_REMOVED
libmultipath: verify_paths(): don't delete paths from pathvec
libmultipath: sync_paths(): handle INIT_REMOVED
libmultipath: orphan_paths(): delete paths in INIT_REMOVED state
libmultipath: adopt_paths(): skip removed paths
multipathd: ev_remove_path(): use INIT_REMOVED
multipathd: deal with INIT_REMOVED during path addition
multipathd: check_path(): set retrigger_delay if necessary
libmultipath: add update_pathvec_from_dm()
libmultipath: update_pathvec_from_dm: handle pp->mpp mismatch
libmultipath: disassemble_map(): always search paths by dev_t
libmultipath: disassemble_map(): require non-NULL pathvec
libmultipath: disassemble_map(): get rid of "is_daemon" argument
libmultipath: disassemble_map(): do not change pathvec and WWIDs
multipath: use update_pathvec_from_dm()
libmpathpersist: use update_pathvec_from_dm()
libmultipath: decrease loglevel in store_path()
libmultipath: dmparser: constify function arguments
libmpathpersist/mpath_persist.c | 56 +----- libmultipath/configure.c | 2 +- libmultipath/discovery.c | 15 +- libmultipath/dmparser.c | 70 ++----- libmultipath/dmparser.h | 4 +- libmultipath/prioritizers/alua_rtpg.c | 6 +- libmultipath/structs.c | 21 +- libmultipath/structs.h | 6 + libmultipath/structs_vec.c | 263 +++++++++++++++++++++++--- libmultipath/structs_vec.h | 11 +- multipath/main.c | 71 +------ multipathd/cli_handlers.c | 49 ++++- multipathd/main.c | 113 ++++++++--- 13 files changed, 445 insertions(+), 242 deletions(-)
Signature · 1 line
2.26.2
PATCH 54/74 libmultipath: protect use of pp->udev
From: Martin Wilck <[email protected]>
We could never be 100% certain that pp->udev was always set. With the upcoming change, we can be even less certain. Always check pp->udev before using it.
--- libmultipath/discovery.c | 11 +++++++++-- libmultipath/prioritizers/alua_rtpg.c | 6 ++++-- libmultipath/structs_vec.c | 2 +- 3 files changed, 14 insertions(+), 5 deletions(-)
libmultipath/discovery.c +9 -2
@@ -339,7 +339,10 @@ sysfs_get_tgt_nodename(struct path *pp, char *node)struct udev_device *parent, *tgtdev;int host, channel, tgtid =-1 ; - parent =udev_device_get_parent_with_subsystem_devtype (pp->udev,"scsi" ,"scsi_device" ); +if (!pp->udev) +return 1 ; + parent =udev_device_get_parent_with_subsystem_devtype (pp->udev, +"scsi" ,"scsi_device" );if (!parent)return 1 ;/* Check for SAS */
@@ -1378,7 +1381,8 @@ nvme_sysfs_pathinfo (struct path *pp, const struct _vector *hwtable)const char *attr_path =NULL ;const char *attr; - attr_path =udev_device_get_sysname (pp->udev); +if (pp->udev) + attr_path =udev_device_get_sysname (pp->udev);if (!attr_path)return PATHINFO_FAILED ;
@@ -1958,6 +1962,9 @@ static ssize_t uid_fallback(struct path *pp, int path_state, } }else if (pp->bus ==SYSFS_BUS_NVME ) {char value[256 ]; + +if (!pp->udev) +return -1 ; len =sysfs_attr_get_value (pp->udev,"wwid" , value,sizeof (value));if (len <=0 )
libmultipath/prioritizers/alua_rtpg.c +4 -2
@@ -188,9 +188,11 @@ retry:int do_inquiry (const struct path *pp,int evpd,unsigned int codepage,void *resp,int resplen,unsigned int timeout) { -struct udev_device *ud; +struct udev_device *ud =NULL ; - ud =udev_device_get_parent_with_subsystem_devtype (pp->udev,"scsi" , +if (pp->udev) + ud =udev_device_get_parent_with_subsystem_devtype (pp->udev, +"scsi" ,"scsi_device" );if (ud !=NULL ) {int rc;
libmultipath/structs_vec.c +1 -1
@@ -494,7 +494,7 @@ int verify_paths(struct multipath *mpp, struct vectors *vecs)/* * see if path is in sysfs */ -if (sysfs_attr_get_value (pp->udev,"dev" , +if (!pp->udev ||sysfs_attr_get_value (pp->udev,"dev" , pp->dev_t,BLK_DEV_SIZE ) <0 ) {if (pp->state !=PATH_DOWN ) { condlog(1, "%s: removing valid path %s in state %d",
Signature · 1 line
2.26.2
PATCH 55/74 libmultipath: add uninitialize_path()
In reply to [email protected]From: Martin Wilck <[email protected]>
This helper clears all fields of struct path (except pp->udev) that must be re-ininitialized if the path ever is to be used again.
--- libmultipath/structs.c | 19 +++++++++++++++++-- libmultipath/structs.h | 1 + libmultipath/structs_vec.c | 9 +-------- 3 files changed, 19 insertions(+), 10 deletions(-)
libmultipath/structs.c +17 -2
@@ -114,19 +114,34 @@ alloc_path (void) }void -free_path (struct path * pp) +uninitialize_path (struct path *pp) {if (!pp)return ; + pp->dmstate =PSTATE_UNDEF ; + pp->uid_attribute =NULL ; + pp->getuid =NULL ; +if (checker_selected (&pp->checker))checker_put (&pp->checker);if (prio_selected (&pp->prio))prio_put (&pp->prio); -if (pp->fd >=0 ) +if (pp->fd >=0 ) {close (pp->fd); + pp->fd =-1 ; + } +} + +void +free_path (struct path * pp) +{ +if (!pp) +return ; + +uninitialize_path (pp);if (pp->udev) {udev_device_unref (pp->udev);
libmultipath/structs.h +1 -0
@@ -416,6 +416,7 @@ struct host_group {struct path *alloc_path (void );struct pathgroup *alloc_pathgroup (void );struct multipath *alloc_multipath (void ); +void uninitialize_path (struct path *pp);void free_path (struct path *);void free_pathvec (vector vec,enum free_path_mode free_paths);void free_pathgroup (struct pathgroup * pgp,enum free_path_mode free_paths);
libmultipath/structs_vec.c +1 -8
@@ -98,14 +98,7 @@ void orphan_path(struct path *pp, const char *reason) {condlog (3 ,"%s: orphan path, %s" , pp->dev, reason); pp->mpp =NULL ; - pp->dmstate =PSTATE_UNDEF ; - pp->uid_attribute =NULL ; - pp->getuid =NULL ; -prio_put (&pp->prio); -checker_put (&pp->checker); -if (pp->fd >=0 ) -close (pp->fd); - pp->fd =-1 ; +uninitialize_path (pp); }void orphan_paths (vector pathvec,struct multipath *mpp,const char *reason)
Signature · 1 line
2.26.2
PATCH 56/74 multipath-tools: introduce INIT_REMOVED state
In reply to [email protected]From: Martin Wilck <[email protected]>
Introduce a new state for pp->initialized, INIT_REMOVED. This state means that the path is about to be removed, either by a remove uevent or by the operator. It will normally be a very short-lived state, because the path will be deleted from pathvec quickly after setting this state. Only if the path is member of a multipath map, and reloading the map fails, this state will persist until a later map reload or flush eventually cancels the membership in the map.
Paths in INIT_REMOVED state are treated as if they didn't exist.
--- libmultipath/discovery.c | 4 ++++ libmultipath/structs.h | 5 +++++ libmultipath/structs_vec.c | 17 +++++++++++++++++ libmultipath/structs_vec.h | 1 + multipathd/main.c | 5 +++-- 5 files changed, 30 insertions(+), 2 deletions(-)
libmultipath/discovery.c +4 -0
@@ -2078,6 +2078,10 @@ int pathinfo(struct path *pp, struct config *conf, int mask)if (!pp || !conf)return PATHINFO_FAILED ; +/* Treat removed paths as if they didn't exist */ +if (pp->initialized ==INIT_REMOVED ) +return PATHINFO_FAILED ; +/* * For behavior backward-compatibility with multipathd, * the blacklisting by filter_property|devnode() is not
libmultipath/structs.h +5 -0
@@ -193,6 +193,11 @@ enum initialized_states {INIT_MISSING_UDEV ,INIT_REQUESTED_UDEV ,INIT_OK , +/* +* INIT_REMOVED: supposed to be removed from pathvec, but still +* mapped by some multipath map because of map reload failure. +*/ + INIT_REMOVED, }; enum prkey_sources {
libmultipath/structs_vec.c +17 -0
@@ -113,6 +113,23 @@ void orphan_paths(vector pathvec, struct multipath *mpp, const char *reason) } } +void set_path_removed (struct path *pp) +{ +struct multipath *mpp = pp->mpp; + +orphan_path (pp,"removed" ); +/* +* Keep link to mpp. It will be removed when the path +* is successfully removed from the map. +*/ +if (!mpp) { +condlog (0 ,"%s: internal error: mpp == NULL" , pp->dev); +return ; + } + pp->mpp = mpp; + pp->initialized =INIT_REMOVED ; +} +void remove_map (struct multipath * mpp,struct vectors * vecs,int purge_vec) {
libmultipath/structs_vec.h +1 -0
@@ -18,6 +18,7 @@ int adopt_paths (vector pathvec, struct multipath * mpp);void orphan_paths (vector pathvec,struct multipath *mpp,const char *reason);void orphan_path (struct path * pp,const char *reason); +void set_path_removed (struct path *pp);int verify_paths (struct multipath * mpp,struct vectors * vecs);int update_mpp_paths (struct multipath * mpp,vector pathvec);
multipathd/main.c +3 -2
@@ -1962,8 +1962,9 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks)int marginal_pathgroups, marginal_changed =0 ;int ret; -if ((pp->initialized ==INIT_OK || - pp->initialized ==INIT_REQUESTED_UDEV ) && !pp->mpp) +if (((pp->initialized ==INIT_OK || + pp->initialized ==INIT_REQUESTED_UDEV ) && !pp->mpp) || + pp->initialized ==INIT_REMOVED )return 0 ;if (pp->tick)
Signature · 1 line
2.26.2
PATCH 57/74 libmultipath: update_mpp_paths(): handle INIT_REMOVED
In reply to [email protected]From: Martin Wilck <[email protected]>
Since the ancient commit b96dead ("[multipathd] remove the retry login in uev_remove_path()"), update_mpp_paths() was used to check whether devices found in disassembled maps were still in the pathvec, and to skip them while re-assembling the new params string on reload. The reason was to deal with failed reloads. With the introduction of INIT_REMOVED, we need to skip paths in that state, too. Moreover, past reloads may have succeeded in removing REMOVED paths from the map, so check if any INIT_REMOVED paths can now be deleted for good. This is the right place to do this check, because update_mpp_paths() is called after obtaining the current kernel state, and before reloading.
--- libmultipath/structs_vec.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
libmultipath/structs_vec.c +16 -5
@@ -29,6 +29,7 @@ int update_mpp_paths(struct multipath *mpp, vector pathvec)struct pathgroup * pgp;struct path * pp;int i,j; +bool store_failure = false;if (!mpp || !mpp->pg)return 0 ;
@@ -39,13 +40,23 @@ int update_mpp_paths(struct multipath *mpp, vector pathvec) vector_foreach_slot (mpp->pg, pgp, i) { vector_foreach_slot (pgp->paths, pp, j) { -if (!find_path_by_devt (mpp->paths, pp->dev_t) && - (find_path_by_devt (pathvec, pp->dev_t)) && -store_path (mpp->paths, pp)) -return 1 ; +if (!find_path_by_devt (mpp->paths, pp->dev_t)) { +struct path *pp1; + +/* +* Avoid adding removed paths to the map again +* when we reload it. Such paths may exist if +* domap fails in ev_remove_path(). +*/ + pp1 =find_path_by_devt (pathvec, pp->dev_t); +if (pp1 && pp->initialized !=INIT_REMOVED && +store_path (mpp->paths, pp)) + store_failure = true; + } } } -return 0 ; + +return store_failure; }int adopt_paths (vector pathvec,struct multipath *mpp)
Signature · 1 line
2.26.2
PATCH 58/74 libmultipath: verify_paths(): don't delete paths from pathvec
In reply to [email protected]From: Martin Wilck <[email protected]>
If we encounter a non-existing device verify_paths(), just set it to INIT_REMOVED state. Actual path deletion is postponed until we don't see that path in the kernel map any more.
This allows us to get rid of the "pathvec" argument to this function.
--- libmultipath/configure.c | 2 +- libmultipath/structs_vec.c | 16 +++++++++------- libmultipath/structs_vec.h | 2 +- multipathd/main.c | 4 ++-- 4 files changed, 13 insertions(+), 11 deletions(-)
libmultipath/configure.c +1 -1
@@ -1196,7 +1196,7 @@ int coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid,set_bit_in_bitfield (i, size_mismatch_seen); } } -verify_paths (mpp, vecs); +verify_paths (mpp);if (does_alias_exist (newmp, mpp)) {remove_map (mpp, vecs,PURGE_VEC );
libmultipath/structs_vec.c +9 -7
@@ -502,11 +502,11 @@ out:return NULL ; } -int verify_paths (struct multipath *mpp,struct vectors *vecs) +int verify_paths (struct multipath *mpp) {struct path * pp;int count =0 ; -int i, j; +int i;if (!mpp)return 0 ;
@@ -521,7 +521,7 @@ int verify_paths(struct multipath *mpp, struct vectors *vecs)condlog (1 ,"%s: removing valid path %s in state %d" , mpp->alias, pp->dev, pp->state); }else { -condlog (3 ,"%s: failed to access path %s" , +condlog (2 ,"%s: failed to access path %s" , mpp->alias, pp->dev); } count++;
@@ -534,10 +534,12 @@ int verify_paths(struct multipath *mpp, struct vectors *vecs)*/ if (mpp->hwe == pp->hwe) mpp->hwe =NULL ; -if ((j =find_slot (vecs->pathvec, - (void *)pp)) !=-1 ) -vector_del_slot (vecs->pathvec, j); -free_path (pp); +/* +* Don't delete path from pathvec yet. We'll do this +* after the path has been removed from the map, in +* sync_paths(). +*/ +set_path_removed (pp); }else {condlog (4 ,"%s: verified path %s dev_t %s" , mpp->alias, pp->dev, pp->dev_t);
libmultipath/structs_vec.h +1 -1
@@ -20,7 +20,7 @@ void orphan_paths(vector pathvec, struct multipath *mpp,void orphan_path (struct path * pp,const char *reason);void set_path_removed (struct path *pp); -int verify_paths (struct multipath * mpp,struct vectors * vecs); +int verify_paths (struct multipath *mpp);int update_mpp_paths (struct multipath * mpp,vector pathvec);int update_multipath_strings (struct multipath *mpp,vector pathvec,int is_daemon);
multipathd/main.c +2 -2
@@ -492,7 +492,7 @@ retry: retries =-1 ; goto fail; } -verify_paths (mpp, vecs); +verify_paths (mpp); mpp->action =ACT_RELOAD ;if (setup_map (mpp, params,PARAMS_SIZE , vecs)) {
@@ -959,7 +959,7 @@ rescan:if (adopt_paths (vecs->pathvec, mpp)) goto fail;/* leave path added to pathvec */ -verify_paths (mpp, vecs); +verify_paths (mpp); mpp->action =ACT_RELOAD ; }else {if (!should_multipath (pp, vecs->pathvec, vecs->mpvec)) {
Signature · 1 line
2.26.2
PATCH 59/74 libmultipath: sync_paths(): handle INIT_REMOVED
In reply to [email protected]From: Martin Wilck <[email protected]>
sync_paths() is the function which is called after getting kernel state with disassemble_map(). This is the place where we should check if paths that can eventually be deleted.
--- libmultipath/structs_vec.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
libmultipath/structs_vec.c +33 -0
@@ -264,6 +264,38 @@ update_multipath_status (struct multipath *mpp)return DMP_OK ; } +static struct path *find_devt_in_pathgroups (const struct multipath *mpp, +const char *dev_t) +{ +struct pathgroup *pgp; +struct path *pp; +int j; + + vector_foreach_slot(mpp->pg, pgp, j) { + pp =find_path_by_devt (pgp->paths, dev_t); +if (pp) +return pp; + } +return NULL ; +} + +static void check_removed_paths (const struct multipath *mpp,vector pathvec) +{ +struct path *pp; +int i; + + vector_foreach_slot(pathvec, pp, i) { +if (pp->initialized !=INIT_REMOVED || pp->mpp != mpp) +continue ; +if (!find_devt_in_pathgroups (mpp, pp->dev_t)) { +condlog (2 ,"%s: %s: freeing path in removed state" , + __func__, pp->dev); +vector_del_slot (pathvec, i--); +free_path (pp); + } + } +} +void sync_paths (struct multipath *mpp,vector pathvec) {struct path *pp;
@@ -284,6 +316,7 @@ void sync_paths(struct multipath *mpp, vector pathvec)orphan_path (pp,"path removed externally" ); } } +check_removed_paths (mpp, pathvec);update_mpp_paths (mpp, pathvec);vector_foreach_slot (mpp->paths, pp, i) pp->mpp = mpp;
Signature · 1 line
2.26.2
PATCH 60/74 libmultipath: orphan_paths(): delete paths in INIT_REMOVED state
In reply to [email protected]From: Martin Wilck <[email protected]>
A path in INIT_REMOVED state is only waiting for its last association with a multipath map to be dropped. When orphan_paths() encounters such a path, rather than orphaning it, free it.
--- libmultipath/structs_vec.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
libmultipath/structs_vec.c +7 -1
@@ -119,7 +119,13 @@ void orphan_paths(vector pathvec, struct multipath *mpp, const char *reason) vector_foreach_slot (pathvec, pp, i) {if (pp->mpp == mpp) { -orphan_path (pp, reason); +if (pp->initialized ==INIT_REMOVED ) { +condlog (3 ,"%s: freeing path in removed state" , + pp->dev); +vector_del_slot (pathvec, i--); +free_path (pp); + }else +orphan_path (pp, reason); } } }
Signature · 1 line
2.26.2
PATCH 61/74 libmultipath: adopt_paths(): skip removed paths
In reply to [email protected]From: Martin Wilck <[email protected]>
If we don't do this, pathinfo() will fail on these paths, causing adopt_paths() to fail.
--- libmultipath/structs_vec.c | 2 ++ 1 file changed, 2 insertions(+)
libmultipath/structs_vec.c +2 -0
@@ -79,6 +79,8 @@ int adopt_paths(vector pathvec, struct multipath *mpp) pp->dev, mpp->alias);continue ; } +if (pp->initialized ==INIT_REMOVED ) +continue ;condlog (3 ,"%s: ownership set to %s" , pp->dev, mpp->alias); pp->mpp = mpp;
Signature · 1 line
2.26.2
Benjamin Marzinski
maintainerRe: [PATCH 61/74] libmultipath: adopt_paths(): skip removed paths
Quoted 5 lines · On Thu, Jul 09, 2020 at 12:51:32PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
If we don't do this,
pathinfo()will fail on these paths, causingadopt_paths()to fail.
This is probably unnecessary, but it seems safer to make sure that pp->mpp is set to mpp, before bailing out on removed paths.
-Ben
Quoted 20 lines
Signed-off-by Martin Wilck <[email protected]>--- libmultipath/structs_vec.c | 2 ++ 1 file changed, 2 insertions(+)libmultipath/structs_vec.c +2
-0@@ -79,6 +79,8 @@ int adopt_paths(vector pathvec, struct multipath *mpp) pp->dev, mpp->alias);continue ; } +if (pp->initialized ==INIT_REMOVED ) +continue ;condlog (3 ,"%s: ownership set to %s" , pp->dev, mpp->alias); pp->mpp = mpp;Signature · 1 line
2.26.2
PATCH 62/74 multipathd: ev_remove_path(): use INIT_REMOVED
In reply to [email protected]From: Martin Wilck <[email protected]>
Set paths belonging to a map to INIT_REMOVED state before attempting to reload or flush the map. If the map association is successfully removed, the path will be actually deleted, either via flush_map() -> orphan_paths(), or in the update_multipath_strings()->sync_paths() code path.
--- multipathd/main.c | 38 +++++++++++++++++++++++++++----------- 1 file changed, 27 insertions(+), 11 deletions(-)
multipathd/main.c +27 -11
@@ -1102,6 +1102,18 @@ ev_remove_path (struct path *pp, struct vectors * vecs, int need_do_map) goto fail; } +/* +* Mark the path as removed. In case of success, we +* will delete it for good. Otherwise, it will be deleted +* later, unless all attempts to reload this map fail. +* Note: we have to explicitly remove pp from mpp->paths, +* update_mpp_paths() doesn't do that. +*/ +set_path_removed (pp); + i =find_slot (mpp->paths, pp); +if (i !=-1 ) +vector_del_slot (mpp->paths, i); +/* * Make sure mpp->hwe doesn't point to freed memory * We call extract_hwe_from_path() below to restore mpp->hwe
@@ -1109,9 +1121,6 @@ ev_remove_path (struct path *pp, struct vectors * vecs, int need_do_map)if (mpp->hwe == pp->hwe) mpp->hwe =NULL ; -if ((i =find_slot (mpp->paths, (void *)pp)) !=-1 ) -vector_del_slot (mpp->paths, i); -/* * remove the map IF removing the last path */
@@ -1135,6 +1144,7 @@ ev_remove_path (struct path *pp, struct vectors * vecs, int need_do_map)" removing all paths" , alias); retval =0 ; +/* flush_map() has freed the path */ goto out; }/*
@@ -1171,21 +1181,27 @@ ev_remove_path (struct path *pp, struct vectors * vecs, int need_do_map)/* * update our state from kernel */ +char devt[BLK_DEV_SIZE ]; + +strlcpy (devt, pp->dev_t,sizeof (devt));if (setup_multipath (vecs, mpp))return 1 ; +/* +* Successful map reload without this path: +* sync_map_state() will free it. +*/ sync_map_state (mpp); -condlog (2 ,"%s [%s]: path removed from map %s" , - pp->dev, pp->dev_t, mpp->alias); +condlog (2 ,"%s: path removed from map %s" , + devt, mpp->alias); } + }else { +/* mpp == NULL */ +if ((i =find_slot (vecs->pathvec, (void *)pp)) !=-1 ) +vector_del_slot (vecs->pathvec, i); +free_path (pp); } - out: -if ((i =find_slot (vecs->pathvec, (void *)pp)) !=-1 ) -vector_del_slot (vecs->pathvec, i); - -free_path (pp); -return retval; fail:
Signature · 1 line
2.26.2
PATCH 63/74 multipathd: deal with INIT_REMOVED during path addition
In reply to [email protected]From: Martin Wilck <[email protected]>
With the introduction of INIT_REMOVED, we have to deal with the situation when a path is re-added in this state. This enables us to detect the situation where a path is added while still part of a map after a failed removal, which we couldn't before. Dealing with this correctly requires some additional logic. There's a good case (re-added path is still mapped by a map with matching WWID) and a bad case (non-matching WWID).
The logic is very similar in uev_add_path() and cli_add_path().
--- multipathd/cli_handlers.c | 49 +++++++++++++++++++++++++++++++++-- multipathd/main.c | 54 ++++++++++++++++++++++++++++++++++++--- 2 files changed, 97 insertions(+), 6 deletions(-)
multipathd/cli_handlers.c +47 -2
@@ -713,11 +713,56 @@ cli_add_path (void * v, char ** reply, int * len, void * data) goto blacklisted; pp =find_path_by_dev (vecs->pathvec, param); -if (pp) { +if (pp && pp->initialized !=INIT_REMOVED ) {condlog (2 ,"%s: path already in pathvec" , param);if (pp->mpp)return 0 ; - }else { + }else if (pp) { +/* Trying to add a path in INIT_REMOVED state */ +struct multipath *prev_mpp; + + prev_mpp = pp->mpp; + pp->mpp =NULL ; + pp->initialized =INIT_NEW ; + pp->wwid[0 ] ='\0' ; + conf =get_multipath_config (); +pthread_cleanup_push (put_multipath_config, conf); + r =pathinfo (pp, conf,DI_ALL |DI_BLACKLIST ); +pthread_cleanup_pop (1 ); + +if (prev_mpp) { +/* Similar logic as in uev_add_path() */ + pp->mpp = prev_mpp; +if (r ==PATHINFO_OK && + !strncmp (prev_mpp->wwid, pp->wwid,WWID_SIZE )) { +condlog (2 ,"%s: path re-added to %s" , pp->dev, + pp->mpp->alias); +return 0 ; + }else if (!ev_remove_path (pp, vecs, true)) +/* Path removed in ev_remove_path() */ + pp =NULL ; +else { +/* Init state is now INIT_REMOVED again */ + pp->dmstate =PSTATE_FAILED ; +dm_fail_path (pp->mpp->alias, pp->dev_t); +condlog (1 ,"%s: failed to re-add path still mapped in %s" , + pp->dev, pp->mpp->alias); +return 1 ; + } + }else { +switch (r) { +case PATHINFO_SKIPPED : + goto blacklisted; +case PATHINFO_OK : +break ; +default : +condlog (0 ,"%s: failed to get pathinfo" , param); +return 1 ; + } + } + } + +if (!pp) {struct udev_device *udevice; udevice = udev_device_new_from_subsystem_sysname(udev,
multipathd/main.c +50 -4
68 lines not on this page · raw message
Signature · 1 line
2.26.2
Benjamin Marzinski
maintainerRe: [PATCH 63/74] multipathd: deal with INIT_REMOVED during path addition
Quoted 26 lines · On Thu, Jul 09, 2020 at 12:51:34PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
With the introduction of
INIT_REMOVED, we have to deal with the situation when a path is re-added in this state. This enables us to detect the situation where a path is added while still part of a map after a failed removal, which we couldn't before. Dealing with this correctly requires some additional logic. There's a good case (re-added path is still mapped by a map with matching WWID) and a bad case (non-matching WWID).The logic is very similar in
uev_add_path()andcli_add_path().Signed-off-by Martin Wilck <[email protected]>--- multipathd/cli_handlers.c | 49 +++++++++++++++++++++++++++++++++-- multipathd/main.c | 54 ++++++++++++++++++++++++++++++++++++--- 2 files changed, 97 insertions(+), 6 deletions(-)multipathd/cli_handlers.c +0
-1@@ -713,11 +713,56 @@ cli_add_path (void * v, char ** reply, int * len, void * data) goto blacklisted; pp =find_path_by_dev (vecs->pathvec, param); -if (pp) {
multipathd/cli_handlers.c cli_add_path (void * v, char ** reply, int * len, void * data) · line 724
Quoted 10 lines
multipathd/cli_handlers.c +6
-1+if (pp && pp->initialized !=INIT_REMOVED ) {condlog (2 ,"%s: path already in pathvec" , param);if (pp->mpp)return 0 ; - }else { + }else if (pp) { +/* Trying to add a path in INIT_REMOVED state */ +struct multipath *prev_mpp; + + prev_mpp = pp->mpp;
O.k. I might be missing something here. How is it possible for a path to be in the INIT_REMOVED state, and not have pp->mpp set? Is this just a safety check? Is so, should we print a message, since this is an unexpected state.
Quoted 5 lines
multipathd/cli_handlers.c +5
-0+ pp->mpp =NULL ; + pp->initialized =INIT_NEW ; + pp->wwid[0 ] ='\0' ; + conf =get_multipath_config (); +pthread_cleanup_push (put_multipath_config, conf);
multipathd/cli_handlers.c cli_add_path (void * v, char ** reply, int * len, void * data) · line 739
Quoted 10 lines
multipathd/cli_handlers.c +10
-0+ r =pathinfo (pp, conf,DI_ALL |DI_BLACKLIST ); +pthread_cleanup_pop (1 ); + +if (prev_mpp) { +/* Similar logic as in uev_add_path() */ + pp->mpp = prev_mpp; +if (r ==PATHINFO_OK && + !strncmp (prev_mpp->wwid, pp->wwid,WWID_SIZE )) { +condlog (2 ,"%s: path re-added to %s" , pp->dev, + pp->mpp->alias);
Should multipath be reinstating the path with dm here? I realize that it will get resinstated when it's checked next. At the very least, multipathd should be making sure it get checked the next time it's examined in checkerloop.
Quoted 34 lines
multipathd/cli_handlers.c +26
-0+return 0 ; + }else if (!ev_remove_path (pp, vecs, true)) +/* Path removed in ev_remove_path() */ + pp =NULL ; +else { +/* Init state is now INIT_REMOVED again */ + pp->dmstate =PSTATE_FAILED ; +dm_fail_path (pp->mpp->alias, pp->dev_t); +condlog (1 ,"%s: failed to re-add path still mapped in %s" , + pp->dev, pp->mpp->alias); +return 1 ; + } + }else { +switch (r) { +case PATHINFO_SKIPPED : + goto blacklisted; +case PATHINFO_OK : +break ; +default : +condlog (0 ,"%s: failed to get pathinfo" , param); +return 1 ; + } + } + } + +if (!pp) {struct udev_device *udevice; udevice = udev_device_new_from_subsystem_sysname(udev,multipathd/main.c +0
-0
multipathd/main.c uev_add_path (struct uevent *uev, struct vectors * vecs, int need_do_map) · line 846
Quoted 5 lines
multipathd/main.c +2
-0@@ -842,9 +842,21 @@ uev_add_path (struct uevent *uev, struct vectors * vecs, int need_do_map) pp =find_path_by_dev (vecs->pathvec, uev->kernel);if (pp) {int r; +struct multipath *prev_mpp =NULL ; +bool was_removed = pp->initialized ==INIT_REMOVED ;
was_removed is only used once, so it's not necessary.
Quoted 19 lines
multipathd/main.c +12
-2+ +if (was_removed) { +condlog (3 ,"%s: re-adding removed path" , pp->dev); + pp->initialized =INIT_NEW ; + prev_mpp = pp->mpp; + pp->mpp =NULL ; +/* make sure get_uid() is called */ + pp->wwid[0 ] ='\0' ; + }else +condlog (3 , +"%s: spurious uevent, path already in pathvec" , + uev->kernel); -condlog (3 ,"%s: spurious uevent, path already in pathvec" , - uev->kernel);if (!pp->mpp && !strlen (pp->wwid)) {condlog (3 ,"%s: reinitialize path" , uev->kernel);udev_device_unref (pp->udev);
multipathd/main.c uev_add_path (struct uevent *uev, struct vectors * vecs, int need_do_map) · line 871
Quoted 8 lines
multipathd/main.c +2
-2@@ -854,9 +866,43 @@ uev_add_path (struct uevent *uev, struct vectors * vecs, int need_do_map) r =pathinfo (pp, conf,DI_ALL |DI_BLACKLIST );pthread_cleanup_pop (1 ); -if (r ==PATHINFO_OK ) +if (r ==PATHINFO_OK && !prev_mpp) ret =ev_add_path (pp, vecs, need_do_map); -else if (r ==PATHINFO_SKIPPED ) { + else if (r == PATHINFO_OK &&
again, should we reinstate this path?
Quoted 39 lines
multipathd/main.c +34
-0+ !strncmp(pp->wwid, prev_mpp->wwid, WWID_SIZE)) { +/* +* Path was unsuccessfully removed, but now +* re-added, and still belongs to the right map +* - all fine. +*/ + pp->mpp = prev_mpp; + ret =0 ; + } else if (prev_mpp) { +/* +* Bad: re-added path still hangs in wrong map +* Make another attempt to remove the path +*/ + pp->mpp = prev_mpp; + ret =ev_remove_path (pp, vecs, true); +if (r ==PATHINFO_OK && !ret) +/* +* Path successfully freed, move on to +* "new path" code path below +*/ + pp =NULL ; +else { +/* +* Failure in ev_remove_path will keep +* path in pathvec in INIT_REMOVED state +* Fail the path to make sure it isn't +* used any more. +*/ + pp->dmstate =PSTATE_FAILED ; +dm_fail_path (pp->mpp->alias, pp->dev_t); +condlog (1 ,"%s: failed to re-add path still mapped in %s" , + pp->dev, pp->mpp->alias); + } + } else if (r == PATHINFO_SKIPPED) {condlog (3 ,"%s: remove blacklisted path" , uev->kernel); i =find_slot (vecs->pathvec, (void *)pp);Signature · 1 line
2.26.2
PATCH 64/74 multipathd: check_path(): set retrigger_delay if necessary
In reply to [email protected]From: Martin Wilck <[email protected]>
In a follow up patch, we will set INIT_MISSING_UDEV and set tick=1 (minimal) at the same time. In this case, which is new, check_path() must reset the delay when it first triggers an uevent.
--- multipathd/main.c | 4 ++++ 1 file changed, 4 insertions(+)
multipathd/main.c +4 -0
@@ -2019,6 +2019,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks)int disable_reinstate =0 ;int oldchkrstate = pp->chkrstate;int retrigger_tries, verbosity; +unsigned int retrigger_delay;unsigned int checkint, max_checkint;struct config *conf;int marginal_pathgroups, marginal_changed =0 ;
@@ -2036,6 +2037,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks) conf =get_multipath_config (); retrigger_tries = conf->retrigger_tries; + retrigger_delay = conf->retrigger_delay; checkint = conf->checkint; max_checkint = conf->max_checkint; verbosity = conf->verbosity;
@@ -2048,6 +2050,8 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks) };if (!pp->mpp && pp->initialized ==INIT_MISSING_UDEV ) { +if (pp->tick != retrigger_delay) + pp->tick = conf->retrigger_delay;if (pp->retriggers < retrigger_tries) {condlog (2 ,"%s: triggering change event to reinitialize" , pp->dev);
Signature · 1 line
2.26.2
Benjamin Marzinski
maintainerRe: [PATCH 64/74] multipathd: check_path(): set retrigger_delay if necessary
Quoted 5 lines · On Thu, Jul 09, 2020 at 12:51:35PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
In a follow up patch, we will set
INIT_MISSING_UDEVand set tick=1 (minimal) at the same time. In this case, which is new,check_path()must reset the delay when it first triggers an uevent.
Maybe I'm just being obtuse here, but I don't get what this does. pp->tick will always be 0 for any path that makes it to the check
if (!pp->mpp && pp->initialized == INIT_MISSING_UDEV) {
And then if it's not out of retries, the path will get set to INIT_REQUESTED_UDEV, and pathinfo() will take care of resetting pp->tick when it gets called by the new uevent.
If it is out of retries, the path won't get pp->tick reset, which seems wrong, but it that case it should probably be set to max_checkint, like happens when the "add missing paths" code fails.
Or like I said, maybe I'm just missing something.
-Ben
Quoted 37 lines
Signed-off-by Martin Wilck <[email protected]>--- multipathd/main.c | 4 ++++ 1 file changed, 4 insertions(+)multipathd/main.c +4
-0@@ -2019,6 +2019,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks)int disable_reinstate =0 ;int oldchkrstate = pp->chkrstate;int retrigger_tries, verbosity; +unsigned int retrigger_delay;unsigned int checkint, max_checkint;struct config *conf;int marginal_pathgroups, marginal_changed =0 ;@@ -2036,6 +2037,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks) conf =get_multipath_config (); retrigger_tries = conf->retrigger_tries; + retrigger_delay = conf->retrigger_delay; checkint = conf->checkint; max_checkint = conf->max_checkint; verbosity = conf->verbosity;@@ -2048,6 +2050,8 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks) };if (!pp->mpp && pp->initialized ==INIT_MISSING_UDEV ) { +if (pp->tick != retrigger_delay) + pp->tick = conf->retrigger_delay;if (pp->retriggers < retrigger_tries) {condlog (2 ,"%s: triggering change event to reinitialize" , pp->dev);Signature · 1 line
2.26.2
Re: [PATCH 64/74] multipathd: check_path(): set retrigger_delay if necessary
Quoted 18 lines · Benjamin Marzinski
Quoted 6 lines · On Thu, Jul 09, 2020 at 12:51:35PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
In a follow up patch, we will set
INIT_MISSING_UDEVand set tick=1 (minimal) at the same time. In this case, which is new,check_path()
must reset the delay when it first triggers an uevent.Maybe I'm just being obtuse here, but I don't get what this does.
pp->tickwill always be 0 for any path that makes it to the checkif (!pp->mpp &&
pp->initialized==INIT_MISSING_UDEV) {And then if it's not out of retries, the path will get set to
INIT_REQUESTED_UDEV, andpathinfo()will take care of resetting pp-Quoted 1 line
tick
when it gets called by the new uevent.
Quoted 7 lines
If it is out of retries, the path won't get
pp->tickreset, which seems
wrong, but it that case it should probably be set tomax_checkint, like
happens when the "add missing paths" code fails.Or like I said, maybe I'm just missing something.
You're not. This was just plain stupid.
Thanks
Martin
PATCH 65/74 libmultipath: add update_pathvec_from_dm()
In reply to [email protected]From: Martin Wilck <[email protected]>
It can happen in particular during boot or startup that we encounter paths as map members which haven't been discovered or fully initialized yet, and are thus not in the pathvec. These paths need special treatment in various ways. Currently this is dealt with in disassemble_map(). That's a layer violation, and the way it is done is suboptimal in various ways.
As a preparation to change that, this patch introduces a new function, update_pathvec_from_dm(), that is supposed to deal with newly discovered paths from disassemble_map(). It has to be called after disassemble_map() has finished.
The logic of the new function is similar but not identical to what disassemble_map() was doing before. Firstly, the function will simply remove path devices that don't exist - there's no point in carrying these around. Map reloads can't be called from this code for reasons of the overall program logic. But it's prepared to signal to the caller that a map reload is in order. Using this return value will be future work.
Second, pathinfo() is now called on new paths rather than just setting pp->dev. The pathinfo flags can be passed as argument to make the function flexible for different callers.
Finally, treatment of WWIDs is different now. There'll be only one attempt at guessing the map WWID if it's not set yet. If a non-matching path WWID is found, the path is removed, and a new uevent triggered (this is the most important change wrt the dangerous previous code that would simply overwrite the path WWID). If the path WWID is empty, it will still be set from the map WWID, which I consider not perfect, but no worse than what we did before.
--- libmultipath/structs_vec.c | 134 +++++++++++++++++++++++++++++++++++++ libmultipath/structs_vec.h | 2 + 2 files changed, 136 insertions(+)
libmultipath/structs_vec.c +134 -0
@@ -59,6 +59,140 @@ int update_mpp_paths(struct multipath *mpp, vector pathvec)return store_failure; } +static bool guess_mpp_wwid (struct multipath *mpp) +{ +int i, j; +struct pathgroup *pgp; +struct path *pp; + +if (strlen (mpp->wwid) || !mpp->pg) +return true; + + vector_foreach_slot(mpp->pg, pgp, i) { +if (!pgp->paths) +continue ; + vector_foreach_slot(pgp->paths, pp, j) { +if (pp->initialized ==INIT_OK &&strlen (pp->wwid)) { +strlcpy (mpp->wwid, pp->wwid,sizeof (mpp->wwid)); +condlog (2 ,"%s: guessed WWID %s from path %s" , + mpp->alias, mpp->wwid, pp->dev); +return true; + } + } + } +condlog (1 ,"%s: unable to guess WWID" , mpp->alias); +return false; +} + +/* +* update_pathvec_from_dm() - update pathvec after disassemble_map() +* +* disassemble_map() may return block devices that are members in +* multipath maps but haven't been discovered. Check whether they +* need to be added to pathvec or discarded. +* +* Returns: true if immediate map reload is desirable +* +* Side effects: +* - may delete non-existing paths and empty pathgroups from mpp +* - may set pp->wwid and / or mpp->wwid +* - calls pathinfo() on existing paths is pathinfo_flags is not 0 +*/ +bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp, +int pathinfo_flags) +{ +int i, j; +struct pathgroup *pgp; +struct path *pp; +struct config *conf; +bool mpp_has_wwid; +bool must_reload = false; + +if (!mpp->pg) +return false; + + mpp_has_wwid =guess_mpp_wwid (mpp); + + vector_foreach_slot(mpp->pg, pgp, i) { +if (!pgp->paths) + goto delete_pg; + + vector_foreach_slot(pgp->paths, pp, j) { + pp->mpp = mpp; + +if (pp->udev) { +if (pathinfo_flags) { + conf =get_multipath_config (); +pthread_cleanup_push (put_multipath_config, + conf); +pathinfo (pp, conf, pathinfo_flags); +pthread_cleanup_pop (1 ); + } +continue ; + } + +/* If this fails, the device is not in sysfs */ + pp->udev =get_udev_device (pp->dev_t,DEV_DEVT ); +if (!pp->udev) { +condlog (2 ,"%s: discarding non-existing path %s" , + mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); +free_path (pp); + must_reload = true; + }else { + +devt2devname (pp->dev,sizeof (pp->dev), + pp->dev_t); + conf =get_multipath_config (); +pthread_cleanup_push (put_multipath_config, + conf); + pp->checkint = conf->checkint; +if (pathinfo (pp, conf, +DI_SYSFS |DI_WWID |pathinfo_flags) + !=PATHINFO_OK ) +condlog (1 ,"%s: error in pathinfo" , + pp->dev); +pthread_cleanup_pop (1 ); +if (mpp_has_wwid && !strlen (pp->wwid)) { +condlog (3 ,"%s: setting wwid from map: %s" , + pp->dev, mpp->wwid); +strlcpy (pp->wwid, mpp->wwid, +sizeof (pp->wwid)); + }else if (mpp_has_wwid && +strcmp (mpp->wwid, pp->wwid)) { + +condlog (0 ,"%s: path %s WWID %s doesn't match, removing from map" , + mpp->wwid, pp->dev_t, pp->wwid); +/* +* This path exists, but in the wong map. +* We can't reload the map from here. +* Instead, treat this path like "missing udev", +* which it probably is. +* check_path() will trigger an uevent +* and reset pp->tick. +*/ + must_reload = true; + pp->mpp =NULL ; +dm_fail_path (mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); + pp->initialized =INIT_MISSING_UDEV ; + pp->tick =1 ; + } +condlog (2 ,"%s: adding new path %s" , + mpp->alias, pp->dev); +store_path (pathvec, pp); + } + } +if (VECTOR_SIZE (pgp->paths) !=0 ) +continue ; + delete_pg: +condlog (2 ,"%s: removing empty pathgroup %d" , mpp->alias, i); +vector_del_slot (mpp->pg, i--); +free_pathgroup (pgp,KEEP_PATHS ); + } +return must_reload; +} +int adopt_paths (vector pathvec,struct multipath *mpp) {int i, ret;
libmultipath/structs_vec.h +2 -0
@@ -21,6 +21,8 @@ void orphan_path (struct path * pp, const char *reason);void set_path_removed (struct path *pp);int verify_paths (struct multipath *mpp); +bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp, +int pathinfo_flags);int update_mpp_paths (struct multipath * mpp,vector pathvec);int update_multipath_strings (struct multipath *mpp,vector pathvec,int is_daemon);
Signature · 1 line
2.26.2
Benjamin Marzinski
maintainerRe: [PATCH 65/74] libmultipath: add update_pathvec_from_dm()
Quoted 8 lines · On Thu, Jul 09, 2020 at 12:51:36PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
It can happen in particular during boot or startup that we encounter paths as map members which haven't been discovered or fully initialized yet, and are thus not in the pathvec. These paths need special treatment in various ways. Currently this is dealt with in
disassemble_map(). That's a layer violation, and the way it is done is suboptimal in various ways.
Quoted 4 lines
As a preparation to change that, this patch introduces a new function,
update_pathvec_from_dm(), that is supposed to deal with newly discovered paths fromdisassemble_map(). It has to be called afterdisassemble_map()has finished.
I don't really understand the reason for adding paths to pathvec, just because they appear in disassemble_map(). If multipathd sees a map with a path that it doesn't have in pathvec, then that path should either sortly appear, or multipathd likely doesn't have it for a reason. I agree that it's possible that uevent got missed, but it seems more likely that the user updated multipath.conf and hasn't reconfigured multipathd.
Maybe when I read future patches this will make more sense, but can't multipathd add paths to the pathvec that it is configured to blacklist. I would like to think that multipathd is definitive for the multipath devices that it controls. If multipathd doesn't think a path belongs in a multipath device, then the path doesn't belong in the multipath device. There are corner cases where a path can appear in a multipath device before multipathd gets notified about the path, but they all seem pretty rare. Someone running "multipath" before multipathd has seen the path addition uevent, for instance. Perhaps if pathinfo() should always set DI_BLACKLIST, and if the path is skipped, flag the device for reloading.
Quoted 93 lines
The logic of the new function is similar but not identical to what
disassemble_map()was doing before. Firstly, the function will simply remove path devices that don't exist - there's no point in carrying these around. Map reloads can't be called from this code for reasons of the overall program logic. But it's prepared to signal to the caller that a map reload is in order. Using this return value will be future work.Second,
pathinfo()is now called on new paths rather than just settingpp->dev. The pathinfo flags can be passed as argument to make the function flexible for different callers.Finally, treatment of WWIDs is different now. There'll be only one attempt at guessing the map WWID if it's not set yet. If a non-matching path WWID is found, the path is removed, and a new uevent triggered (this is the most important change wrt the dangerous previous code that would simply overwrite the path WWID). If the path WWID is empty, it will still be set from the map WWID, which I consider not perfect, but no worse than what we did before.
Signed-off-by Martin Wilck <[email protected]>--- libmultipath/structs_vec.c | 134 +++++++++++++++++++++++++++++++++++++ libmultipath/structs_vec.h | 2 + 2 files changed, 136 insertions(+)libmultipath/structs_vec.c +60
-0@@ -59,6 +59,140 @@ int update_mpp_paths(struct multipath *mpp, vector pathvec)return store_failure; } +static bool guess_mpp_wwid (struct multipath *mpp) +{ +int i, j; +struct pathgroup *pgp; +struct path *pp; + +if (strlen (mpp->wwid) || !mpp->pg) +return true; + + vector_foreach_slot(mpp->pg, pgp, i) { +if (!pgp->paths) +continue ; + vector_foreach_slot(pgp->paths, pp, j) { +if (pp->initialized ==INIT_OK &&strlen (pp->wwid)) { +strlcpy (mpp->wwid, pp->wwid,sizeof (mpp->wwid)); +condlog (2 ,"%s: guessed WWID %s from path %s" , + mpp->alias, mpp->wwid, pp->dev); +return true; + } + } + } +condlog (1 ,"%s: unable to guess WWID" , mpp->alias); +return false; +} + +/* +* update_pathvec_from_dm() - update pathvec after disassemble_map() +* +* disassemble_map() may return block devices that are members in +* multipath maps but haven't been discovered. Check whether they +* need to be added to pathvec or discarded. +* +* Returns: true if immediate map reload is desirable +* +* Side effects: +* - may delete non-existing paths and empty pathgroups from mpp +* - may set pp->wwid and / or mpp->wwid +* - calls pathinfo() on existing paths is pathinfo_flags is not 0 +*/ +bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp, +int pathinfo_flags) +{ +int i, j; +struct pathgroup *pgp; +struct path *pp; +struct config *conf; +bool mpp_has_wwid; +bool must_reload = false; + +if (!mpp->pg) +return false; + + mpp_has_wwid =guess_mpp_wwid (mpp); + + vector_foreach_slot(mpp->pg, pgp, i) { +if (!pgp->paths) + goto delete_pg; + + vector_foreach_slot(pgp->paths, pp, j) { + pp->mpp = mpp;
libmultipath/structs_vec.c int update_mpp_paths(struct multipath *mpp, vector pathvec) · line 131
Quoted 10 lines
libmultipath/structs_vec.c +10
-0+ +if (pp->udev) { +if (pathinfo_flags) { + conf =get_multipath_config (); +pthread_cleanup_push (put_multipath_config, + conf); +pathinfo (pp, conf, pathinfo_flags); +pthread_cleanup_pop (1 ); + } +continue ;
I suppose that if pp->udev is set then the path is in pathvec, but the lack of check seemed odd to me.
Quoted 13 lines
libmultipath/structs_vec.c +13
-0+ } + +/* If this fails, the device is not in sysfs */ + pp->udev =get_udev_device (pp->dev_t,DEV_DEVT ); +if (!pp->udev) { +condlog (2 ,"%s: discarding non-existing path %s" , + mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); +free_path (pp); + must_reload = true; + }else { + + devt2devname(pp->dev, sizeof(pp->dev),
libmultipath/structs_vec.c int update_mpp_paths(struct multipath *mpp, vector pathvec) · line 154
Quoted 10 lines
libmultipath/structs_vec.c +10
-0+ pp->dev_t); + conf =get_multipath_config (); +pthread_cleanup_push (put_multipath_config, + conf); + pp->checkint = conf->checkint; +if (pathinfo (pp, conf, +DI_SYSFS |DI_WWID |pathinfo_flags) + !=PATHINFO_OK ) +condlog (1 ,"%s: error in pathinfo" , + pp->dev);
This seems wrong to me. A blacklisted path shouldn't be added to pathvec.
Quoted 27 lines
libmultipath/structs_vec.c +27
-0+pthread_cleanup_pop (1 ); +if (mpp_has_wwid && !strlen (pp->wwid)) { +condlog (3 ,"%s: setting wwid from map: %s" , + pp->dev, mpp->wwid); +strlcpy (pp->wwid, mpp->wwid, +sizeof (pp->wwid)); + }else if (mpp_has_wwid && +strcmp (mpp->wwid, pp->wwid)) { + +condlog (0 ,"%s: path %s WWID %s doesn't match, removing from map" , + mpp->wwid, pp->dev_t, pp->wwid); +/* +* This path exists, but in the wong map. +* We can't reload the map from here. +* Instead, treat this path like "missing udev", +* which it probably is. +* check_path() will trigger an uevent +* and reset pp->tick. +*/ + must_reload = true; + pp->mpp =NULL ; +dm_fail_path (mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); + pp->initialized =INIT_MISSING_UDEV ; + pp->tick =1 ; + } + condlog(2, "%s: adding new path %s",
libmultipath/structs_vec.c int update_mpp_paths(struct multipath *mpp, vector pathvec) · line 191
Quoted 10 lines
libmultipath/structs_vec.c +10
-0+ mpp->alias, pp->dev); +store_path (pathvec, pp); + } + } +if (VECTOR_SIZE (pgp->paths) !=0 ) +continue ; + delete_pg: +condlog (2 ,"%s: removing empty pathgroup %d" , mpp->alias, i); +vector_del_slot (mpp->pg, i--); +free_pathgroup (pgp,KEEP_PATHS );
Should this flag the device for reloading, to remove the useless path group?
Quoted 22 lines
libmultipath/structs_vec.c +4
-0+ } +return must_reload; +} +int adopt_paths (vector pathvec,struct multipath *mpp) {int i, ret;libmultipath/structs_vec.h +2
-0@@ -21,6 +21,8 @@ void orphan_path (struct path * pp, const char *reason);void set_path_removed (struct path *pp);int verify_paths (struct multipath *mpp); +bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp, +int pathinfo_flags);int update_mpp_paths (struct multipath * mpp,vector pathvec);int update_multipath_strings (struct multipath *mpp,vector pathvec,int is_daemon);Signature · 1 line
2.26.2
Re: [PATCH 65/74] libmultipath: add update_pathvec_from_dm()
Quoted 32 lines · Benjamin Marzinski
Quoted 13 lines · On Thu, Jul 09, 2020 at 12:51:36PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
It can happen in particular during boot or startup that we encounter
paths as map members which haven't been discovered or fully initialized
yet, and are thus not in the pathvec. These paths need special treatment
in various ways. Currently this is dealt with indisassemble_map(). That's
a layer violation, and the way it is done is suboptimal in various ways.Quoted 7 lines
As a preparation to change that, this patch introduces a new function,
update_pathvec_from_dm(), that is supposed to deal with newly discovered
paths fromdisassemble_map(). It has to be called afterdisassemble_map()
has finished.I don't really understand the reason for adding paths to pathvec, just
because they appear indisassemble_map(). If multipathd sees a map with
a path that it doesn't have in pathvec, then that path should either sortly appear, or multipathd likely doesn't have it for a reason. I agree that it's possible that uevent got missed, but it seems more likely that the user updated multipath.conf and hasn't reconfigured multipathd.
Quoted 16 lines
Maybe when I read future patches this will make more sense, but can't multipathd add paths to the pathvec that it is configured to blacklist.
I would like to think that multipathd is definitive for the multipath devices that it controls. If multipathd doesn't think a path belongs in
a multipath device, then the path doesn't belong in the multipath device. There are corner cases where a path can appear in a multipath device before multipathd gets notified about the path, but they all seem
pretty rare. Someone running "multipath" before multipathd has seen the
path addition uevent, for instance. Perhaps ifpathinfo()should always
setDI_BLACKLIST, and if the path is skipped, flag the device for reloading.
I agree wrt blacklisting. But for non-blacklisted paths that look like valid map members, I think adding them to the pathvec makes sense.
The current upstream code doesn't add the code to pathvec; instead it keeps paths in multipath maps that are not in pathvec. I tend to find that inconsistent. I believe that pathvec should contain all paths that multipathd is supposed to deal with in some manner (at least that's one possible definition of it's semantics, see my reply to 69/74).
Note that multipathd used to add these paths to pathvec in the past, but later stopped doing that to avoid removed paths being re-added (see also my commit message for 'libmultipath: disassemble_map(): get rid of "is_daemon" argument'). I've introduced INIT_REMOVED to deal with that latter issue; thus we can now add all paths to pathvec which appear to be valid multipath map members, even if they're not fully initialized (yet).
update_pathvec_from_dm() is introduced as a single function to incorporate all the related logic in one place, making it easy to track.
To my understanding, the two main reasons why paths can be found in maps that are not in pathvec (besides what you mentioned already) are:
1) paths having been multipathed during initrd processing, but being not fully initalized after switching root yet, because udev coldplug processing is delayed for some reason,
2) paths that are removed but couldn't be deleted from maps because the respective device-mapper RELOAD calls failed.
2) should be mostly dealt with by the earlier INIT_REMOVED patches. In general, the issues with path removal are mitigated a lot by your kernel patch 86f1152b ("dm: allow active and inactive tables to share dm_devs"), which eliminated the main reason for RELOAD ioctls to fail. But multipath-tools supports older kernels that don't have this patch yet (at least we never made a statement to the contrary).
Anyway, issue 1) remains, and cannot easily be excluded. Simply removing such partially-initialized paths from maps is highly dangerous IMO; maps might degrade and fail for no good reason, in particular during startup / boot.
Quoted 121 lines
Quoted 121 lines
The logic of the new function is similar but not identical to what
disassemble_map()was doing before. Firstly, the function will simply
remove path devices that don't exist - there's no point in carrying these
around. Map reloads can't be called from this code for reasons of the
overall program logic. But it's prepared to signal to the caller that
a map reload is in order. Using this return value will be future work.Second,
pathinfo()is now called on new paths rather than just settingpp->dev. The pathinfo flags can be passed as argument to make the function
flexible for different callers.Finally, treatment of WWIDs is different now. There'll be only one attempt
at guessing the map WWID if it's not set yet. If a non-matching path WWID
is found, the path is removed, and a new uevent triggered (this is the
most important change wrt the dangerous previous code that would simply
overwrite the path WWID). If the path WWID is empty, it will still be
set from the map WWID, which I consider not perfect, but no worse than what we did before.Signed-off-by Martin Wilck <[email protected]>--- libmultipath/structs_vec.c | 134 +++++++++++++++++++++++++++++++++++++ libmultipath/structs_vec.h | 2 + 2 files changed, 136 insertions(+)libmultipath/structs_vec.c +16
-0@@ -59,6 +59,140 @@ int update_mpp_paths(struct multipath *mpp, vector pathvec)return store_failure; } +static bool guess_mpp_wwid (struct multipath *mpp) +{ +int i, j; +struct pathgroup *pgp; +struct path *pp; + +if (strlen (mpp->wwid) || !mpp->pg) +return true; + + vector_foreach_slot(mpp->pg, pgp, i) { +if (!pgp->paths) +continue ; + vector_foreach_slot(pgp->paths, pp, j) { +if (pp->initialized ==INIT_OK &&strlen (pp- >wwid)) { +strlcpy (mpp->wwid, pp->wwid,sizeof (mpp->wwid)); + condlog(2, "%s: guessed WWID %s frompath %s",
libmultipath/structs_vec.c +1
-0+ mpp->alias, mpp->wwid, pp-Quoted 1 line
dev);
libmultipath/structs_vec.c +10
-0+return true; + } + } + } +condlog (1 ,"%s: unable to guess WWID" , mpp->alias); +return false; +} + +/* +* update_pathvec_from_dm() - update pathvec after
disassemble_map()
+ *
+ *disassemble_map()may return block devices that are members in
+ * multipath maps but haven't been discovered. Check whether they
+ * need to be added to pathvec or discarded.
+ *
+ * Returns: true if immediate map reload is desirable
+ *
+ * Side effects:
+ * - may delete non-existing paths and empty pathgroups from mpp
+ * - may setpp->wwidand / ormpp->wwid
+ * - callspathinfo()on existing paths ispathinfo_flagsis not 0
+ */libmultipath/structs_vec.c +26
-0+bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp, +int pathinfo_flags) +{ +int i, j; +struct pathgroup *pgp; +struct path *pp; +struct config *conf; +bool mpp_has_wwid; +bool must_reload = false; + +if (!mpp->pg) +return false; + + mpp_has_wwid =guess_mpp_wwid (mpp); + + vector_foreach_slot(mpp->pg, pgp, i) { +if (!pgp->paths) + goto delete_pg; + + vector_foreach_slot(pgp->paths, pp, j) { + pp->mpp = mpp; + +if (pp->udev) { +if (pathinfo_flags) { + conf =get_multipath_config (); + pthread_cleanup_push(put_multip
ath_config,libmultipath/structs_vec.c +2
-0+ conf); + pathinfo(pp, conf,
Quoted 8 lines
libmultipath/structs_vec.c int update_mpp_paths(struct multipath *mpp, vector pathvec) · line 131
Quoted 4 lines
pathinfo_flags);libmultipath/structs_vec.c +3
-0+pthread_cleanup_pop (1 ); + } +continue ;I suppose that if
pp->udevis set then the path is in pathvec, but the
lack of check seemed odd to me.
disassemble_map() looks up paths in pathvec before adding them to the pg. If it does not find them in pathvec, it stores an empty path with only pp->dev_t initialized in pgp->paths. Thus pp->udev can only be set if the path had been found in pathvec.
I'll add a comment explaining that.
Quoted 24 lines
Quoted 24 lines
libmultipath/structs_vec.c +4
-0+ } + +/* If this fails, the device is not in sysfs */ + pp->udev = get_udev_device(pp->dev_t,
DEV_DEVT);libmultipath/structs_vec.c +2
-0+if (!pp->udev) { + condlog(2, "%s: discarding non-existingpath %s",
libmultipath/structs_vec.c +10
-0+ mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); +free_path (pp); + must_reload = true; + }else { + +devt2devname (pp->dev,sizeof (pp->dev), + pp->dev_t); + conf =get_multipath_config (); + pthread_cleanup_push(put_multipath_config,
libmultipath/structs_vec.c +4
-0+ conf); + pp->checkint = conf->checkint; + if (pathinfo(pp, conf, + DI_SYSFS|DI_WWID|pathinfo_
Quoted 8 lines
libmultipath/structs_vec.c int update_mpp_paths(struct multipath *mpp, vector pathvec) · line 154
Quoted 5 lines
flags)
libmultipath/structs_vec.c +2
-0+ != PATHINFO_OK) + condlog(1, "%s: error inpathinfo",
libmultipath/structs_vec.c +1
-0+ pp->dev);This seems wrong to me. A blacklisted path shouldn't be added to pathvec.
Right, I should handle the error condition differently here, and I should be using DI_BLACKLIST, too.
Quoted 47 lines
Quoted 47 lines
libmultipath/structs_vec.c +2
-0+pthread_cleanup_pop (1 ); +if (mpp_has_wwid && !strlen (pp->wwid)){
libmultipath/structs_vec.c +1
-0+ condlog(3, "%s: setting wwidfrom map: %s",
libmultipath/structs_vec.c +5
-0+ pp->dev, mpp->wwid); +strlcpy (pp->wwid, mpp->wwid, +sizeof (pp->wwid)); + }else if (mpp_has_wwid && +strcmp (mpp->wwid, pp->wwid)){
libmultipath/structs_vec.c +2
-0+ + condlog(0, "%s: path %s WWID %sdoesn't match, removing from map",
libmultipath/structs_vec.c +1
-0+ mpp->wwid, pp->dev_t,
pp->wwid);libmultipath/structs_vec.c +2
-0+/* +* This path exists, but in the wong map.
libmultipath/structs_vec.c +1
-0+* We can't reload the map from here.
libmultipath/structs_vec.c +1
-0+* Instead, treat this path like "missing udev",
libmultipath/structs_vec.c +2
-0+* which it probably is. +* check_path() will trigger an uevent
libmultipath/structs_vec.c +5
-0+* and reset pp->tick. +*/ + must_reload = true; + pp->mpp =NULL ; + dm_fail_path(mpp->alias, pp-Quoted 1 line
dev_t);libmultipath/structs_vec.c +2
-1+ vector_del_slot(pgp->paths, j --); + pp->initialized =
INIT_MISSING_UDEV;libmultipath/structs_vec.c +11
-0+ pp->tick =1 ; + } +condlog (2 ,"%s: adding new path %s" , + mpp->alias, pp->dev); +store_path (pathvec, pp); + } + } +if (VECTOR_SIZE (pgp->paths) !=0 ) +continue ; + delete_pg: + condlog(2, "%s: removing empty pathgroup %d", mpp-
Quoted 6 lines
libmultipath/structs_vec.c int update_mpp_paths(struct multipath *mpp, vector pathvec) · line 191
Quoted 3 lines
Quoted 1 line
alias, i);
libmultipath/structs_vec.c +2
-0+vector_del_slot (mpp->pg, i--); +free_pathgroup (pgp,KEEP_PATHS );Should this flag the device for reloading, to remove the useless path group?
Good point, thanks.
Quoted 26 lines
Quoted 26 lines
libmultipath/structs_vec.c +4
-0+ } +return must_reload; +} +int adopt_paths (vector pathvec,struct multipath *mpp) {int i, ret;libmultipath/structs_vec.h +2
-0@@ -21,6 +21,8 @@ void orphan_path (struct path * pp, const char *reason);void set_path_removed (struct path *pp);int verify_paths (struct multipath *mpp); +bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp, +int pathinfo_flags);int update_mpp_paths (struct multipath * mpp,vector pathvec); int update_multipath_strings (struct multipath *mpp, vectorpathvec,
intis_daemon);Signature · 1 line
2.26.2
PATCH 66/74 libmultipath: update_pathvec_from_dm: handle pp->mpp mismatch
In reply to [email protected]From: Martin Wilck <[email protected]>
Treat this like a WWID mismatch.
--- libmultipath/structs_vec.c | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-)
libmultipath/structs_vec.c +23 -14
@@ -118,6 +118,12 @@ bool update_pathvec_from_dm(vector pathvec, struct multipath *mpp, goto delete_pg; vector_foreach_slot(pgp->paths, pp, j) { + +if (pp->mpp && pp->mpp != mpp) { +condlog (0 ,"BUG: %s: found path %s which is already in %s" , + mpp->alias, pp->dev, pp->mpp->alias); + goto bad_path; + } pp->mpp = mpp;if (pp->udev) {
@@ -163,25 +169,28 @@ bool update_pathvec_from_dm(vector pathvec, struct multipath *mpp,condlog (0 ,"%s: path %s WWID %s doesn't match, removing from map" , mpp->wwid, pp->dev_t, pp->wwid); -/* -* This path exists, but in the wong map. -* We can't reload the map from here. -* Instead, treat this path like "missing udev", -* which it probably is. -* check_path() will trigger an uevent -* and reset pp->tick. -*/ - must_reload = true; - pp->mpp =NULL ; -dm_fail_path (mpp->alias, pp->dev_t); -vector_del_slot (pgp->paths, j--); - pp->initialized =INIT_MISSING_UDEV ; - pp->tick =1 ; + goto bad_path; }condlog (2 ,"%s: adding new path %s" , mpp->alias, pp->dev);store_path (pathvec, pp); + } +continue ; + + bad_path: +/* +* This path exists, but in the wrong map. +* We can't reload the map from here. +* Instead, treat this path like "missing udev". +* check_path() will trigger an uevent and reset pp->tick. +*/ + must_reload = true; + pp->mpp =NULL ; +dm_fail_path (mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); + pp->initialized =INIT_MISSING_UDEV ; + pp->tick =1 ; }if (VECTOR_SIZE (pgp->paths) !=0 )continue ;
Signature · 1 line
2.26.2
Benjamin Marzinski
maintainerRe: [PATCH 66/74] libmultipath: update_pathvec_from_dm: handle pp->mpp mismatch
Quoted 56 lines · On Thu, Jul 09, 2020 at 12:51:37PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
Treat this like a WWID mismatch.
Signed-off-by Martin Wilck <[email protected]>--- libmultipath/structs_vec.c | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-)libmultipath/structs_vec.c +13
-14@@ -118,6 +118,12 @@ bool update_pathvec_from_dm(vector pathvec, struct multipath *mpp, goto delete_pg; vector_foreach_slot(pgp->paths, pp, j) { + +if (pp->mpp && pp->mpp != mpp) { +condlog (0 ,"BUG: %s: found path %s which is already in %s" , + mpp->alias, pp->dev, pp->mpp->alias); + goto bad_path; + } pp->mpp = mpp;if (pp->udev) {@@ -163,25 +169,28 @@ bool update_pathvec_from_dm(vector pathvec, struct multipath *mpp,condlog (0 ,"%s: path %s WWID %s doesn't match, removing from map" , mpp->wwid, pp->dev_t, pp->wwid); -/* -* This path exists, but in the wong map. -* We can't reload the map from here. -* Instead, treat this path like "missing udev", -* which it probably is. -* check_path() will trigger an uevent -* and reset pp->tick. -*/ - must_reload = true; - pp->mpp =NULL ; -dm_fail_path (mpp->alias, pp->dev_t); -vector_del_slot (pgp->paths, j--); - pp->initialized =INIT_MISSING_UDEV ; - pp->tick =1 ; + goto bad_path; }condlog (2 ,"%s: adding new path %s" , mpp->alias, pp->dev);store_path (pathvec, pp); + } +continue ; + + bad_path: +/* +* This path exists, but in the wrong map.
libmultipath/structs_vec.c bool update_pathvec_from_dm(vector pathvec, struct multipath *mpp, · line 193
Quoted 10 lines
libmultipath/structs_vec.c +10
-0+* We can't reload the map from here. +* Instead, treat this path like "missing udev". +* check_path() will trigger an uevent and reset pp->tick. +*/ + must_reload = true; + pp->mpp =NULL ; +dm_fail_path (mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); + pp->initialized =INIT_MISSING_UDEV ; + pp->tick =1 ;
Is there a reason not to call orphan_path() to clean up things like any open fd, until we figure out what to do with the path.
Quoted 5 lines
libmultipath/structs_vec.c +0
-0}if (VECTOR_SIZE (pgp->paths) !=0 )continue ;Signature · 1 line
2.26.2
Re: [PATCH 66/74] libmultipath: update_pathvec_from_dm: handle pp->mpp mismatch
Quoted 29 lines · Benjamin Marzinski
Quoted 28 lines · On Thu, Jul 09, 2020 at 12:51:37PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
Treat this like a WWID mismatch.
Signed-off-by Martin Wilck <[email protected]>--- libmultipath/structs_vec.c | 37 +++++++++++++++++++++++----------- --- 1 file changed, 23 insertions(+), 14 deletions(-)libmultipath/structs_vec.c +5
-0+ bad_path: +/* +* This path exists, but in the wrong map. +* We can't reload the map from here. +* Instead, treat this path like "missing udev".
libmultipath/structs_vec.c +1
-0+* check_path() will trigger an uevent and reset
pp->tick.libmultipath/structs_vec.c +3
-0+*/ + must_reload = true; + pp->mpp =NULL ;
Quoted 8 lines
libmultipath/structs_vec.c bool update_pathvec_from_dm(vector pathvec, struct multipath *mpp, · line 193
Quoted 4 lines
libmultipath/structs_vec.c +4
-0+dm_fail_path (mpp->alias, pp->dev_t); +vector_del_slot (pgp->paths, j--); + pp->initialized =INIT_MISSING_UDEV ; + pp->tick =1 ;Is there a reason not to call
orphan_path()to clean up things like any
open fd, until we figure out what to do with the path.
Thanks for the suggestion. It makes sense for the 2nd "bad_path" condition but not for the first. I'll treat the two cases differently.
Regards
Martin
PATCH 67/74 libmultipath: disassemble_map(): always search paths by dev_t
In reply to [email protected]From: Martin Wilck <[email protected]>
There's no point in searching for the devname first. dev_t is the primary device property for both device mapper and udev.
--- libmultipath/dmparser.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
libmultipath/dmparser.c +2 -6
@@ -295,12 +295,8 @@ int disassemble_map(vector pathvec, char *params, struct multipath *mpp, devname[0 ] ='\0' ; } -if (pathvec) { -if (strlen (devname)) - pp =find_path_by_dev (pathvec, devname); -else - pp =find_path_by_devt (pathvec, word); - } +if (pathvec) + pp =find_path_by_devt (pathvec, word);if (!pp) { pp =alloc_path ();
Signature · 1 line
2.26.2
PATCH 68/74 libmultipath: disassemble_map(): require non-NULL pathvec
In reply to [email protected]From: Martin Wilck <[email protected]>
We would fail in store_path() at the latest if pathvec was NULL. And all callers set pathvec anyway.
--- libmultipath/dmparser.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
libmultipath/dmparser.c +3 -2
@@ -6,6 +6,7 @@#include <stdio.h> #include <stdlib.h> #include <string.h> +#include <assert.h> #include "checkers.h" #include "vector.h"
@@ -137,6 +138,7 @@ int disassemble_map(vector pathvec, char *params, struct multipath *mpp,struct path * pp;struct pathgroup * pgp; +assert (pathvec !=NULL ); p = params;condlog (4 ,"%s: disassemble map [%s]" , mpp->alias, params);
@@ -295,8 +297,7 @@ int disassemble_map(vector pathvec, char *params, struct multipath *mpp, devname[0 ] ='\0' ; } -if (pathvec) - pp =find_path_by_devt (pathvec, word); + pp =find_path_by_devt (pathvec, word);if (!pp) { pp =alloc_path ();
Signature · 1 line
2.26.2
PATCH 69/74 libmultipath: disassemble_map(): get rid of "is_daemon" argument
In reply to [email protected]From: Martin Wilck <[email protected]>
The reason for the is_daemon parameter in disassemble_map() lies deep in multipath-tools' past, in b96dead ("[multipathd] remove the retry login in uev_remove_path()"): By not adding paths from disassembled maps to the pathvec, we avoided to re-add removed paths on future map reloads (logic in update_mpp_paths()). As we can handle this with INIT_REMOVED now, we don't need to distinguish daemon and non-daemon mode any more. This fixes a memory leak, because only paths which are in pathvec will be freed on program exit.
--- libmpathpersist/mpath_persist.c | 2 +- libmultipath/dmparser.c | 6 ++---- libmultipath/dmparser.h | 2 +- libmultipath/structs_vec.c | 9 ++++----- libmultipath/structs_vec.h | 6 ++---- multipath/main.c | 4 ++-- multipathd/main.c | 8 ++++---- 7 files changed, 16 insertions(+), 21 deletions(-)
libmpathpersist/mpath_persist.c +1 -1
@@ -391,7 +391,7 @@ get_mpvec (vector curmp, vector pathvec, char * refwwid)condlog (3 ,"params = %s" , params);dm_get_status (mpp->alias, status);condlog (3 ,"status = %s" , status); -disassemble_map (pathvec, params, mpp,0 ); +disassemble_map (pathvec, params, mpp);/* * disassemble_map() can add new paths to pathvec.
libmultipath/dmparser.c +2 -4
@@ -122,8 +122,7 @@ err:#undef APPEND -int disassemble_map (vector pathvec,char *params,struct multipath *mpp, -int is_daemon) +int disassemble_map (vector pathvec,char *params,struct multipath *mpp) {char * word;char * p;
@@ -311,8 +310,7 @@ int disassemble_map(vector pathvec, char *params, struct multipath *mpp,strlcpy (pp->wwid, mpp->wwid,WWID_SIZE ); } -/* Only call this in multipath client mode */ -if (!is_daemon &&store_path (pathvec, pp)) +if (store_path (pathvec, pp)) goto out1; }else { if (!strlen(pp->wwid) &&
libmultipath/dmparser.h +1 -1
@@ -1,3 +1,3 @@int assemble_map (struct multipath *,char *,int ); -int disassemble_map (vector ,char *,struct multipath *,int ); +int disassemble_map (vector ,char *,struct multipath *);int disassemble_status (char *,struct multipath *);
libmultipath/structs_vec.c +4 -5
@@ -370,7 +370,7 @@ extract_hwe_from_path(struct multipath * mpp) }int -update_multipath_table (struct multipath *mpp,vector pathvec,int is_daemon) +update_multipath_table (struct multipath *mpp,vector pathvec) {int r =DMP_ERR ;char params[PARAMS_SIZE ] = {0 };
@@ -384,7 +384,7 @@ update_multipath_table (struct multipath *mpp, vector pathvec, int is_daemon)return r; } -if (disassemble_map (pathvec, params, mpp, is_daemon)) { +if (disassemble_map (pathvec, params, mpp)) {condlog (3 ,"%s: cannot disassemble map" , mpp->alias);return DMP_ERR ; }
@@ -474,7 +474,7 @@ void sync_paths(struct multipath *mpp, vector pathvec) }int -update_multipath_strings (struct multipath *mpp,vector pathvec,int is_daemon) +update_multipath_strings (struct multipath *mpp,vector pathvec) {struct pathgroup *pgp;int i, r =DMP_ERR ;
@@ -489,10 +489,9 @@ update_multipath_strings(struct multipath *mpp, vector pathvec, int is_daemon)free_pgvec (mpp->pg,KEEP_PATHS ); mpp->pg =NULL ; - r =update_multipath_table (mpp, pathvec, is_daemon); + r =update_multipath_table (mpp, pathvec);if (r !=DMP_OK )return r; -sync_paths (mpp, pathvec); r =update_multipath_status (mpp);
libmultipath/structs_vec.h +2 -4
@@ -24,8 +24,7 @@ int verify_paths(struct multipath *mpp);bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp,int pathinfo_flags);int update_mpp_paths (struct multipath * mpp,vector pathvec); -int update_multipath_strings (struct multipath *mpp,vector pathvec, -int is_daemon); +int update_multipath_strings (struct multipath *mpp,vector pathvec);void extract_hwe_from_path (struct multipath * mpp);#define PURGE_VEC 1
@@ -41,8 +40,7 @@ struct multipath * add_map_with_path (struct vectors * vecs, struct path * pp, int add_vec);void update_queue_mode_del_path (struct multipath *mpp);void update_queue_mode_add_path (struct multipath *mpp); -int update_multipath_table (struct multipath *mpp,vector pathvec, -int is_daemon); +int update_multipath_table (struct multipath *mpp,vector pathvec);int update_multipath_status (struct multipath *mpp);vector get_used_hwes (const struct _vector *pathvec);
multipath/main.c +2 -2
@@ -272,7 +272,7 @@ get_dm_mpvec (enum mpath_cmds cmd, vector curmp, vector pathvec, char * refwwid)dm_get_status (mpp->alias, status);condlog (3 ,"status = %s" , status); -disassemble_map (pathvec, params, mpp,0 ); +disassemble_map (pathvec, params, mpp);/* * disassemble_map() can add new paths to pathvec.
@@ -352,7 +352,7 @@ static int check_usable_paths(struct config *conf,dm_get_map (mpp->alias, &mpp->size, params);dm_get_status (mpp->alias, status); -disassemble_map (pathvec, params, mpp,0 ); +disassemble_map (pathvec, params, mpp);disassemble_status (status, mpp); vector_foreach_slot (mpp->pg, pg, i) {
multipathd/main.c +4 -4
@@ -409,7 +409,7 @@ int __setup_multipath(struct vectors *vecs, struct multipath *mpp, goto out; } -if (update_multipath_strings (mpp, vecs->pathvec,1 ) !=DMP_OK ) { +if (update_multipath_strings (mpp, vecs->pathvec) !=DMP_OK ) {condlog (0 ,"%s: failed to setup multipath" , mpp->alias); goto out; }
@@ -551,7 +551,7 @@ add_map_without_path (struct vectors *vecs, const char *alias) mpp->mpe =find_mpe (conf->mptable, mpp->wwid);put_multipath_config (conf); -if (update_multipath_table (mpp, vecs->pathvec,1 ) !=DMP_OK ) +if (update_multipath_table (mpp, vecs->pathvec) !=DMP_OK ) goto out;if (update_multipath_status (mpp) !=DMP_OK ) goto out;
@@ -1410,7 +1410,7 @@ map_discovery (struct vectors * vecs)return 1 ; vector_foreach_slot (vecs->mpvec, mpp, i) -if (update_multipath_table (mpp, vecs->pathvec,1 ) !=DMP_OK || +if (update_multipath_table (mpp, vecs->pathvec) !=DMP_OK ||update_multipath_status (mpp) !=DMP_OK ) {remove_map (mpp, vecs,1 ); i--;
@@ -2153,7 +2153,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks)/* * Synchronize with kernel state */ - ret =update_multipath_strings (pp->mpp, vecs->pathvec,1 ); + ret =update_multipath_strings (pp->mpp, vecs->pathvec);if (ret !=DMP_OK ) {if (ret ==DMP_NOT_FOUND ) {/* multipath device missing. Likely removed */
Signature · 1 line
2.26.2
Benjamin Marzinski
maintainerRe: [PATCH 69/74] libmultipath: disassemble_map(): get rid of "is_daemon" argument
Quoted 10 lines · On Thu, Jul 09, 2020 at 12:51:40PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
The reason for the
is_daemonparameter indisassemble_map()lies deep in multipath-tools' past, in b96dead ("[multipathd] remove the retry login inuev_remove_path()"): By not adding paths from disassembled maps to the pathvec, we avoided to re-add removed paths on future map reloads (logic inupdate_mpp_paths()). As we can handle this withINIT_REMOVEDnow, we don't need to distinguish daemon and non-daemon mode any more. This fixes a memory leak, because only paths which are in pathvec will be freed on program exit.
I don't have any problems with the code in this patch. I just want to reiterate that I don't think that multipathd should automatically be adding paths, just because they were in a multipath device. In my opinion, multipathd should have the final decision as to what a multipath device should look like. If it sees an unexpected path, and runs pathinfo on it, and finds that that path does belong, it's fine to add it. But if pathinfo says that the path doesn't belong, multipathd shouldn't add it to the pathvec. It should instead trigger a reload of the device to remove path.
-Ben
Quoted 188 lines
Signed-off-by Martin Wilck <[email protected]>--- libmpathpersist/mpath_persist.c | 2 +- libmultipath/dmparser.c | 6 ++---- libmultipath/dmparser.h | 2 +- libmultipath/structs_vec.c | 9 ++++----- libmultipath/structs_vec.h | 6 ++---- multipath/main.c | 4 ++-- multipathd/main.c | 8 ++++---- 7 files changed, 16 insertions(+), 21 deletions(-)libmpathpersist/mpath_persist.c +1
-1@@ -391,7 +391,7 @@ get_mpvec (vector curmp, vector pathvec, char * refwwid)condlog (3 ,"params = %s" , params);dm_get_status (mpp->alias, status);condlog (3 ,"status = %s" , status); -disassemble_map (pathvec, params, mpp,0 ); +disassemble_map (pathvec, params, mpp);/* * disassemble_map() can add new paths to pathvec. libmultipath/dmparser.c +2
-4@@ -122,8 +122,7 @@ err:#undef APPEND -int disassemble_map (vector pathvec,char *params,struct multipath *mpp, -int is_daemon) +int disassemble_map (vector pathvec,char *params,struct multipath *mpp) {char * word;char * p;@@ -311,8 +310,7 @@ int disassemble_map(vector pathvec, char *params, struct multipath *mpp,strlcpy (pp->wwid, mpp->wwid,WWID_SIZE ); } -/* Only call this in multipath client mode */ -if (!is_daemon &&store_path (pathvec, pp)) +if (store_path (pathvec, pp)) goto out1; }else { if (!strlen(pp->wwid) &&libmultipath/dmparser.h +1
-1@@ -1,3 +1,3 @@int assemble_map (struct multipath *,char *,int ); -int disassemble_map (vector ,char *,struct multipath *,int ); +int disassemble_map (vector ,char *,struct multipath *);int disassemble_status (char *,struct multipath *);libmultipath/structs_vec.c +4
-5@@ -370,7 +370,7 @@ extract_hwe_from_path(struct multipath * mpp) }int -update_multipath_table (struct multipath *mpp,vector pathvec,int is_daemon) +update_multipath_table (struct multipath *mpp,vector pathvec) {int r =DMP_ERR ;char params[PARAMS_SIZE ] = {0 };@@ -384,7 +384,7 @@ update_multipath_table (struct multipath *mpp, vector pathvec, int is_daemon)return r; } -if (disassemble_map (pathvec, params, mpp, is_daemon)) { +if (disassemble_map (pathvec, params, mpp)) {condlog (3 ,"%s: cannot disassemble map" , mpp->alias);return DMP_ERR ; }@@ -474,7 +474,7 @@ void sync_paths(struct multipath *mpp, vector pathvec) }int -update_multipath_strings (struct multipath *mpp,vector pathvec,int is_daemon) +update_multipath_strings (struct multipath *mpp,vector pathvec) {struct pathgroup *pgp;int i, r =DMP_ERR ;@@ -489,10 +489,9 @@ update_multipath_strings(struct multipath *mpp, vector pathvec, int is_daemon)free_pgvec (mpp->pg,KEEP_PATHS ); mpp->pg =NULL ; - r =update_multipath_table (mpp, pathvec, is_daemon); + r =update_multipath_table (mpp, pathvec);if (r !=DMP_OK )return r; -sync_paths (mpp, pathvec); r =update_multipath_status (mpp);libmultipath/structs_vec.h +2
-4@@ -24,8 +24,7 @@ int verify_paths(struct multipath *mpp);bool update_pathvec_from_dm (vector pathvec,struct multipath *mpp,int pathinfo_flags);int update_mpp_paths (struct multipath * mpp,vector pathvec); -int update_multipath_strings (struct multipath *mpp,vector pathvec, -int is_daemon); +int update_multipath_strings (struct multipath *mpp,vector pathvec);void extract_hwe_from_path (struct multipath * mpp);#define PURGE_VEC 1@@ -41,8 +40,7 @@ struct multipath * add_map_with_path (struct vectors * vecs, struct path * pp, int add_vec);void update_queue_mode_del_path (struct multipath *mpp);void update_queue_mode_add_path (struct multipath *mpp); -int update_multipath_table (struct multipath *mpp,vector pathvec, -int is_daemon); +int update_multipath_table (struct multipath *mpp,vector pathvec);int update_multipath_status (struct multipath *mpp);vector get_used_hwes (const struct _vector *pathvec);multipath/main.c +2
-2@@ -272,7 +272,7 @@ get_dm_mpvec (enum mpath_cmds cmd, vector curmp, vector pathvec, char * refwwid)dm_get_status (mpp->alias, status);condlog (3 ,"status = %s" , status); -disassemble_map (pathvec, params, mpp,0 ); +disassemble_map (pathvec, params, mpp);/* * disassemble_map() can add new paths to pathvec. @@ -352,7 +352,7 @@ static int check_usable_paths(struct config *conf,dm_get_map (mpp->alias, &mpp->size, params);dm_get_status (mpp->alias, status); -disassemble_map (pathvec, params, mpp,0 ); +disassemble_map (pathvec, params, mpp);disassemble_status (status, mpp); vector_foreach_slot (mpp->pg, pg, i) {multipathd/main.c +4
-4@@ -409,7 +409,7 @@ int __setup_multipath(struct vectors *vecs, struct multipath *mpp, goto out; } -if (update_multipath_strings (mpp, vecs->pathvec,1 ) !=DMP_OK ) { +if (update_multipath_strings (mpp, vecs->pathvec) !=DMP_OK ) {condlog (0 ,"%s: failed to setup multipath" , mpp->alias); goto out; }@@ -551,7 +551,7 @@ add_map_without_path (struct vectors *vecs, const char *alias) mpp->mpe =find_mpe (conf->mptable, mpp->wwid);put_multipath_config (conf); -if (update_multipath_table (mpp, vecs->pathvec,1 ) !=DMP_OK ) +if (update_multipath_table (mpp, vecs->pathvec) !=DMP_OK ) goto out;if (update_multipath_status (mpp) !=DMP_OK ) goto out;@@ -1410,7 +1410,7 @@ map_discovery (struct vectors * vecs)return 1 ; vector_foreach_slot (vecs->mpvec, mpp, i) -if (update_multipath_table (mpp, vecs->pathvec,1 ) !=DMP_OK || +if (update_multipath_table (mpp, vecs->pathvec) !=DMP_OK ||update_multipath_status (mpp) !=DMP_OK ) {remove_map (mpp, vecs,1 ); i--;@@ -2153,7 +2153,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks)/* * Synchronize with kernel state */ - ret =update_multipath_strings (pp->mpp, vecs->pathvec,1 ); + ret =update_multipath_strings (pp->mpp, vecs->pathvec);if (ret !=DMP_OK ) {if (ret ==DMP_NOT_FOUND ) {/* multipath device missing. Likely removed */ Signature · 1 line
2.26.2
Re: [PATCH 69/74] libmultipath: disassemble_map(): get rid of "is_daemon" argument
Quoted 16 lines · Benjamin Marzinski
Quoted 14 lines · On Thu, Jul 09, 2020 at 12:51:40PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
The reason for the
is_daemonparameter indisassemble_map()lies deep in multipath-tools' past, in b96dead ("[multipathd] remove the retry login inuev_remove_path()"): By not adding paths from disassembled maps to the pathvec, we avoided to re-add removed paths on
future map reloads (logic inupdate_mpp_paths()). As we can handle this withINIT_REMOVEDnow, we don't need to distinguish daemon and non- daemon
mode any more. This fixes a memory leak, because only paths which are in
pathvec will be freed on program exit.
Quoted 12 lines
I don't have any problems with the code in this patch. I just want to reiterate that I don't think that multipathd should automatically be adding paths, just because they were in a multipath device. In my opinion, multipathd should have the final decision as to what a multipath device should look like. If it sees an unexpected path, and
runs pathinfo on it, and finds that that path does belong, it's fine to
add it. But if pathinfo says that the path doesn't belong, multipathd shouldn't add it to the pathvec. It should instead trigger a reload of
the device to remove path.
Got it. I commented in my reply to 65/74. I'll repost this part with the minor issues you raised fixed (hopefully). Then let's review / discuss this again. If we agree on your PoV, we can probably ditch the whole INIT_REMOVED part of my series.
I hope you agree that "if (!is_daemon)" so deep in libmultipath is ugly and should be replaced by something cleaner.
We should take the opportunity to agree on a definition on the exact semantics of pathvec, i.e. which devices should be members of pathvec, and which ones shouldn't. I don't see a clear, consequent definition currently.
Martin
Benjamin Marzinski
maintainerRe: [PATCH 69/74] libmultipath: disassemble_map(): get rid of "is_daemon" argument
Quoted 30 lines · Martin Wilck
Quoted 16 lines · Benjamin Marzinski
Quoted 14 lines · On Thu, Jul 09, 2020 at 12:51:40PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
The reason for the
is_daemonparameter indisassemble_map()lies deep in multipath-tools' past, in b96dead ("[multipathd] remove the retry login inuev_remove_path()"): By not adding paths from disassembled maps to the pathvec, we avoided to re-add removed paths on
future map reloads (logic inupdate_mpp_paths()). As we can handle this withINIT_REMOVEDnow, we don't need to distinguish daemon and non- daemon
mode any more. This fixes a memory leak, because only paths which are in
pathvec will be freed on program exit.Quoted 12 lines
I don't have any problems with the code in this patch. I just want to reiterate that I don't think that multipathd should automatically be adding paths, just because they were in a multipath device. In my opinion, multipathd should have the final decision as to what a multipath device should look like. If it sees an unexpected path, and
runs pathinfo on it, and finds that that path does belong, it's fine to
add it. But if pathinfo says that the path doesn't belong, multipathd shouldn't add it to the pathvec. It should instead trigger a reload of
the device to remove path.
Quoted 4 lines
Got it. I commented in my reply to 65/74. I'll repost this part with the minor issues you raised fixed (hopefully). Then let's review / discuss this again. If we agree on your PoV, we can probably ditch the whole
INIT_REMOVEDpart of my series.
Sure.
Quoted 2 lines
I hope you agree that "if (!is_daemon)" so deep in libmultipath is ugly and should be replaced by something cleaner.
That really never bothered me, but I see your point.
Quoted 4 lines
We should take the opportunity to agree on a definition on the exact semantics of pathvec, i.e. which devices should be members of pathvec, and which ones shouldn't. I don't see a clear, consequent definition currently.
I think that you're correct that all paths that multipathd is dealing with should be on the pathvec. Obviously paths that are only accessable via a local variable are fine, but code shouldn't be dropping the vecs lock with a path that is accessable globally (for instance from mpp->paths) but not on the pathvec. What disassemble_map() has been doing is definitely wrong.
-Ben
Quoted 2 lines
Martin
PATCH 70/74 libmultipath: disassemble_map(): do not change pathvec and WWIDs
In reply to [email protected]From: Martin Wilck <[email protected]>
After introducing update_pathvec_from_dm() in a predecessor patch, the "layer violations" in disassemble_map() can now be removed. I hope this clarifies program logic a little bit.
Callers need to call update_pathvec_from_dm() after disassemble_map().
--- libmultipath/dmparser.c | 48 +++++--------------------------------- libmultipath/structs_vec.c | 3 +++ 2 files changed, 9 insertions(+), 42 deletions(-)
libmultipath/dmparser.c +6 -42
@@ -122,6 +122,12 @@ err:#undef APPEND +/* +* Caution callers: If this function encounters yet unkown path devices, it +* adds them uninitialized to the mpp. +* Call update_pathvec_from_dm() after this function to make sure +* all data structures are in a sane state. +*/ int disassemble_map (vector pathvec,char *params,struct multipath *mpp) {char * word;
@@ -282,20 +288,12 @@ int disassemble_map(vector pathvec, char *params, struct multipath *mpp)FREE (word);for (j =0 ; j < num_paths; j++) { -char devname[FILE_NAME_SIZE ]; - pp =NULL ; p +=get_word (p, &word);if (!word) goto out; -if (devt2devname (devname,FILE_NAME_SIZE , word)) { -condlog (2 ,"%s: cannot find block device" , - word); - devname[0 ] ='\0' ; - } - pp =find_path_by_devt (pathvec, word);if (!pp) {
@@ -305,46 +303,12 @@ int disassemble_map(vector pathvec, char *params, struct multipath *mpp) goto out1;strlcpy (pp->dev_t, word,BLK_DEV_SIZE ); -strlcpy (pp->dev, devname,FILE_NAME_SIZE ); -if (strlen (mpp->wwid)) { -strlcpy (pp->wwid, mpp->wwid, -WWID_SIZE ); - } -if (store_path (pathvec, pp)) - goto out1; - }else { -if (!strlen (pp->wwid) && -strlen (mpp->wwid)) -strlcpy (pp->wwid, mpp->wwid, -WWID_SIZE ); }FREE (word);if (store_path (pgp->paths, pp)) goto out; -/* -* Update wwid for multipaths which are not setup -* in the get_dm_mpvec() code path -*/ -if (!strlen (mpp->wwid)) -strlcpy (mpp->wwid, pp->wwid,WWID_SIZE ); - -/* -* Update wwid for paths which may not have been -* active at the time the getuid callout was run -*/ -else if (!strlen (pp->wwid)) -strlcpy (pp->wwid, mpp->wwid,WWID_SIZE ); - -/* -* Do not allow in-use patch to change wwid -*/ -else if (strcmp (pp->wwid, mpp->wwid) !=0 ) { -condlog (0 ,"%s: path wwid appears to have changed. Using map wwid.\n" , pp->dev_t); -strlcpy (pp->wwid, mpp->wwid,WWID_SIZE ); - } - pgp->id ^= (long )pp; pp->pgindex = i +1 ;
libmultipath/structs_vec.c +3 -0
@@ -389,6 +389,9 @@ update_multipath_table (struct multipath *mpp, vector pathvec)return DMP_ERR ; } +/* FIXME: we should deal with the return value here */ +update_pathvec_from_dm (pathvec, mpp,0 ); +return DMP_OK ; }
Signature · 1 line
2.26.2
PATCH 71/74 multipath: use update_pathvec_from_dm()
In reply to [email protected]From: Martin Wilck <[email protected]>
The multipath-specific function update_paths() can now be replaced with a call to update_pathvec_from_dm().
--- multipath/main.c | 67 +++--------------------------------------------- 1 file changed, 3 insertions(+), 64 deletions(-)
multipath/main.c +3 -64
@@ -187,63 +187,6 @@ usage (char * progname) } -static int -update_paths (struct multipath * mpp,int quick) -{ -int i, j; -struct pathgroup * pgp; -struct path * pp; -struct config *conf; - -if (!mpp->pg) -return 0 ; - - vector_foreach_slot (mpp->pg, pgp, i) { -if (!pgp->paths) -continue ; - - vector_foreach_slot (pgp->paths, pp, j) { -if (!strlen (pp->dev)) { -if (devt2devname (pp->dev,FILE_NAME_SIZE , - pp->dev_t)) { -/* -* path is not in sysfs anymore -*/ - pp->chkrstate = pp->state =PATH_DOWN ; - pp->offline =1 ; -continue ; - } - pp->mpp = mpp; -if (quick) -continue ; - conf =get_multipath_config (); -if (pathinfo (pp, conf,DI_ALL )) - pp->state =PATH_UNCHECKED ; -put_multipath_config (conf); -continue ; - } - pp->mpp = mpp; -if (quick) -continue ; -if (pp->state ==PATH_UNCHECKED || - pp->state ==PATH_WILD ) { - conf =get_multipath_config (); -if (pathinfo (pp, conf,DI_CHECKER )) - pp->state =PATH_UNCHECKED ; -put_multipath_config (conf); - } - -if (pp->priority ==PRIO_UNDEF ) { - conf =get_multipath_config (); -if (pathinfo (pp, conf,DI_PRIO )) - pp->priority =PRIO_UNDEF ; -put_multipath_config (conf); - } - } - } -return 0 ; -} -static int get_dm_mpvec (enum mpath_cmds cmd,vector curmp,vector pathvec,char * refwwid) {
@@ -273,13 +216,8 @@ get_dm_mpvec (enum mpath_cmds cmd, vector curmp, vector pathvec, char * refwwid)condlog (3 ,"status = %s" , status);disassemble_map (pathvec, params, mpp); - -/* -* disassemble_map() can add new paths to pathvec. -* If not in "fast list mode", we need to fetch information -* about them -*/ -update_paths (mpp, (cmd ==CMD_LIST_SHORT )); +update_pathvec_from_dm (pathvec, mpp, + (cmd ==CMD_LIST_SHORT ?0 :DI_ALL ));if (cmd ==CMD_LIST_LONG ) mpp->bestpg =select_path_group (mpp);
@@ -353,6 +291,7 @@ static int check_usable_paths(struct config *conf,dm_get_map (mpp->alias, &mpp->size, params);dm_get_status (mpp->alias, status);disassemble_map (pathvec, params, mpp); +update_pathvec_from_dm (pathvec, mpp,0 );disassemble_status (status, mpp); vector_foreach_slot (mpp->pg, pg, i) {
Signature · 1 line
2.26.2
Benjamin Marzinski
maintainerRe: [PATCH 71/74] multipath: use update_pathvec_from_dm()
Quoted 81 lines · On Thu, Jul 09, 2020 at 12:51:42PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
The multipath-specific function
update_paths()can now be replaced with a call toupdate_pathvec_from_dm().Signed-off-by Martin Wilck <[email protected]>--- multipath/main.c | 67 +++--------------------------------------------- 1 file changed, 3 insertions(+), 64 deletions(-)multipath/main.c +0
-57@@ -187,63 +187,6 @@ usage (char * progname) } -static int -update_paths (struct multipath * mpp,int quick) -{ -int i, j; -struct pathgroup * pgp; -struct path * pp; -struct config *conf; - -if (!mpp->pg) -return 0 ; - - vector_foreach_slot (mpp->pg, pgp, i) { -if (!pgp->paths) -continue ; - - vector_foreach_slot (pgp->paths, pp, j) { -if (!strlen (pp->dev)) { -if (devt2devname (pp->dev,FILE_NAME_SIZE , - pp->dev_t)) { -/* -* path is not in sysfs anymore -*/ - pp->chkrstate = pp->state =PATH_DOWN ; - pp->offline =1 ; -continue ; - } - pp->mpp = mpp; -if (quick) -continue ; - conf =get_multipath_config (); -if (pathinfo (pp, conf,DI_ALL )) - pp->state =PATH_UNCHECKED ; -put_multipath_config (conf); -continue ; - } - pp->mpp = mpp; -if (quick) -continue ; -if (pp->state ==PATH_UNCHECKED || - pp->state ==PATH_WILD ) { - conf =get_multipath_config (); -if (pathinfo (pp, conf,DI_CHECKER )) - pp->state =PATH_UNCHECKED ; -put_multipath_config (conf); - } - -if (pp->priority ==PRIO_UNDEF ) { - conf =get_multipath_config (); -if (pathinfo (pp, conf,DI_PRIO )) - pp->priority =PRIO_UNDEF ; -put_multipath_config (conf); - } - } - } -return 0 ; -} -static int get_dm_mpvec (enum mpath_cmds cmd,vector curmp,vector pathvec,char * refwwid) {@@ -273,13 +216,8 @@ get_dm_mpvec (enum mpath_cmds cmd, vector curmp, vector pathvec, char * refwwid)condlog (3 ,"status = %s" , status);
multipath/main.c get_dm_mpvec (enum mpath_cmds cmd, vector curmp, vector pathvec, char * refwwid) · line 220
Quoted 10 lines
multipath/main.c +2
-7disassemble_map (pathvec, params, mpp); - -/* -* disassemble_map() can add new paths to pathvec. -* If not in "fast list mode", we need to fetch information -* about them -*/ -update_paths (mpp, (cmd ==CMD_LIST_SHORT )); +update_pathvec_from_dm (pathvec, mpp, + (cmd ==CMD_LIST_SHORT ?0 :DI_ALL ));
I personally don't think that "multipath -l" should be opening the path fd. Checking sysfs seems like the limit of what we want to do for fast listing. So, I would prefer
update_pathvec_from_dm(pathvec, mpp,
(cmd == CMD_LIST_SHORT ? DI_NOIO : DI_ALL));
That will make pathinfo exit early.
-Ben
Quoted 13 lines
multipath/main.c +1
-0if (cmd ==CMD_LIST_LONG ) mpp->bestpg =select_path_group (mpp);@@ -353,6 +291,7 @@ static int check_usable_paths(struct config *conf,dm_get_map (mpp->alias, &mpp->size, params);dm_get_status (mpp->alias, status);disassemble_map (pathvec, params, mpp); +update_pathvec_from_dm (pathvec, mpp,0 );disassemble_status (status, mpp); vector_foreach_slot (mpp->pg, pg, i) {Signature · 1 line
2.26.2
Re: [PATCH 71/74] multipath: use update_pathvec_from_dm()
Quoted 45 lines · Benjamin Marzinski
Quoted 36 lines · On Thu, Jul 09, 2020 at 12:51:42PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
The multipath-specific function
update_paths()can now be replaced with
a call toupdate_pathvec_from_dm().Signed-off-by Martin Wilck <[email protected]>--- multipath/main.c | 67 +++----------------------------------------- ---- 1 file changed, 3 insertions(+), 64 deletions(-)multipath/main.c +0
-0static int get_dm_mpvec (enum mpath_cmds cmd, vector curmp, vector pathvec,char * refwwid)
{multipath/main.c +0
-4@@ -273,13 +216,8 @@ get_dm_mpvec (enum mpath_cmds cmd, vector curmp, vector pathvec, char * refwwid)condlog (3 ,"status = %s" , status);disassemble_map (pathvec, params, mpp); - -/* -* disassemble_map() can add new paths to pathvec. -* If not in "fast list mode", we need to fetch information
multipath/main.c +2
-3-* about them -*/ -update_paths (mpp, (cmd ==CMD_LIST_SHORT )); + update_pathvec_from_dm(pathvec, mpp, + (cmd == CMD_LIST_SHORT ? 0 :Quoted 1 line
DI_ALL));I personally don't think that "multipath -l" should be opening the path
fd. Checking sysfs seems like the limit of what we want to do for fast
listing. So, I would prefer
Quoted 4 lines
update_pathvec_from_dm(pathvec, mpp,
(cmd == CMD_LIST_SHORT ? DI_NOIO : DI_ALL));That will make pathinfo exit early.
Good suggestion. But then I need to avoid rerunning pathinfo() for already scanned paths in update_pathvec_from_dm(). Will do.
Regards
Martin
PATCH 72/74 libmpathpersist: use update_pathvec_from_dm()
In reply to [email protected]From: Martin Wilck <[email protected]>
The libmpathpersist-specific function updatepaths() can be replaced with the generic function.
--- libmpathpersist/mpath_persist.c | 54 +-------------------------------- 1 file changed, 1 insertion(+), 53 deletions(-)
libmpathpersist/mpath_persist.c +1 -53
@@ -65,52 +65,6 @@ mpath_lib_exit (struct config *conf)return 0 ; } -static int -updatepaths (struct multipath * mpp) -{ -int i, j; -struct pathgroup * pgp; -struct path * pp; -struct config *conf; - -if (!mpp->pg) -return 0 ; - - vector_foreach_slot (mpp->pg, pgp, i){ -if (!pgp->paths) -continue ; - - vector_foreach_slot (pgp->paths, pp, j){ -if (!strlen (pp->dev)){ -/* -* path is not in sysfs anymore -*/ - pp->state =PATH_DOWN ; -continue ; - } - pp->mpp = mpp; -if (pp->udev ==NULL ) { - pp->udev =get_udev_device (pp->dev_t,DEV_DEVT ); -if (pp->udev ==NULL ) { - pp->state =PATH_DOWN ; -continue ; - } - conf =get_multipath_config (); -pathinfo (pp, conf,DI_SYSFS |DI_CHECKER ); -put_multipath_config (conf); -continue ; - } -if (pp->state ==PATH_UNCHECKED || - pp->state ==PATH_WILD ) { - conf =get_multipath_config (); -pathinfo (pp, conf,DI_CHECKER ); -put_multipath_config (conf); - } - } - } -return 0 ; -} -int mpath_prin_activepath (struct multipath *mpp,int rq_servact,struct prin_resp * resp,int noisy)
@@ -392,13 +346,7 @@ get_mpvec (vector curmp, vector pathvec, char * refwwid)dm_get_status (mpp->alias, status);condlog (3 ,"status = %s" , status);disassemble_map (pathvec, params, mpp); - -/* -* disassemble_map() can add new paths to pathvec. -* If not in "fast list mode", we need to fetch information -* about them -*/ -updatepaths (mpp); +update_pathvec_from_dm (pathvec, mpp,DI_CHECKER );disassemble_status (status, mpp); }
Signature · 1 line
2.26.2
PATCH 73/74 libmultipath: decrease loglevel in store_path()
In reply to [email protected]From: Martin Wilck <[email protected]>
"Empty device name" in store_path() can happen regularly and shouldn't be logged at -v2.
--- libmultipath/structs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
libmultipath/structs.c +1 -1
@@ -322,7 +322,7 @@ store_path (vector pathvec, struct path * pp) err++; }if (!strlen (pp->dev)) { -condlog (2 ,"%s: Empty device name" , pp->dev_t); +condlog (3 ,"%s: Empty device name" , pp->dev_t); err++; }
Signature · 1 line
2.26.2
PATCH 74/74 libmultipath: dmparser: constify function arguments
In reply to [email protected]From: Martin Wilck <[email protected]>
With the previous change that avoids additions to pathvec, the pathvec argument to disassemble_map() is const now. Also use const for the string arguments where possible.
--- libmultipath/dmparser.c | 11 ++++++----- libmultipath/dmparser.h | 4 ++-- 2 files changed, 8 insertions(+), 7 deletions(-)
libmultipath/dmparser.c +6 -5
@@ -128,10 +128,11 @@ err:* Call update_pathvec_from_dm() after this function to make sure * all data structures are in a sane state. */ -int disassemble_map (vector pathvec,char *params,struct multipath *mpp) +int disassemble_map (const struct _vector *pathvec, +const char *params,struct multipath *mpp) {char * word; -char * p; +const char *p;int i, j, k;int num_features =0 ;int num_hwhandler =0 ;
@@ -344,10 +345,10 @@ out:return 1 ; } -int disassemble_status (char *params,struct multipath *mpp) +int disassemble_status (const char *params,struct multipath *mpp) { -char * word; -char * p; +char *word; +const char *p;int i, j, k;int num_feature_args;int num_hwhandler_args;
libmultipath/dmparser.h +2 -2
@@ -1,3 +1,3 @@int assemble_map (struct multipath *,char *,int ); -int disassemble_map (vector ,char *,struct multipath *); -int disassemble_status (char *,struct multipath *); +int disassemble_map (const struct _vector *,const char *,struct multipath *); +int disassemble_status (const char *,struct multipath *);
Signature · 1 line
2.26.2
Quoted 4 lines · On Thu, Jul 09, 2020 at 12:51:24PM +0200, [email protected] wrote:
From: Martin Wilck <[email protected]>
Hi Christophe, hi Ben,
Quoted 8 lines
This is part V of a larger patch series for multipath-tools I've been preparing. It's based on the previously submitted part IV.
The full series will also be available here:
https://github.com/mwilck/multipath-tools/tree/ups/submit-2007There are tags in that repo for each part of the series. This part is tagged "submit-200709-5".
For the part, with the exception of patches 61,63,64,65,66 & 71
Quoted 66 lines
The series addresses several issues with missing, unintialized, and removed paths. The motivation was that I strongly dislike the side effects of
disassemble_map(), adding paths and setting WWIDs while parsing map parameters. IMO this has always been a layering violation. This patch set adds a new function dedicated to handling paths which have not been detected via udev but are present as members of dm maps. That makes it much easier to track and improve the logic applied when such devices are detected.I believe that the new logic will also fix the issue recently reported by Chongyun ("libmultipath/dmparser: add missing path with good status when sync state with dm kernel"). At least, the approach is very similar, although the call to
pathinfo()for new devices now happens inupdate_pathvec_from_dm()rather than indisassemble_map().The patch set also gets rid of the "
is_daemon" argument fordisassemble_map(), causing semantics in daemon and non-daemon mode. The reason for this parameter goes way back into history; it was avoiding to re-add manually removed paths because they were still showing up in maps. But OTOH we must add missing devices which we've failed to detect. The patch set handles this by tracking the state of "being removed" as a new init state, and only actually removing the paths from pathvec when they don't show up in maps any more.Regards
MartinMartin Wilck (21):
libmultipath: protect use ofpp->udev
libmultipath: adduninitialize_path()
multipath-tools: introduceINIT_REMOVEDstate
libmultipath:update_mpp_paths(): handleINIT_REMOVED
libmultipath:verify_paths(): don't delete paths from pathvec
libmultipath:sync_paths(): handleINIT_REMOVED
libmultipath:orphan_paths(): delete paths inINIT_REMOVEDstate
libmultipath:adopt_paths(): skip removed paths
multipathd:ev_remove_path(): useINIT_REMOVED
multipathd: deal withINIT_REMOVEDduring path addition
multipathd:check_path(): setretrigger_delayif necessary
libmultipath: addupdate_pathvec_from_dm()
libmultipath:update_pathvec_from_dm: handlepp->mppmismatch
libmultipath:disassemble_map(): always search paths bydev_t
libmultipath:disassemble_map(): require non-NULL pathvec
libmultipath:disassemble_map(): get rid of "is_daemon" argument
libmultipath:disassemble_map(): do not change pathvec and WWIDs
multipath: useupdate_pathvec_from_dm()
libmpathpersist: useupdate_pathvec_from_dm()
libmultipath: decrease loglevel instore_path()
libmultipath: dmparser: constify function argumentslibmpathpersist/mpath_persist.c | 56 +----- libmultipath/configure.c | 2 +- libmultipath/discovery.c | 15 +- libmultipath/dmparser.c | 70 ++----- libmultipath/dmparser.h | 4 +- libmultipath/prioritizers/alua_rtpg.c | 6 +- libmultipath/structs.c | 21 +- libmultipath/structs.h | 6 + libmultipath/structs_vec.c | 263 +++++++++++++++++++++++--- libmultipath/structs_vec.h | 11 +- multipath/main.c | 71 +------ multipathd/cli_handlers.c | 49 ++++- multipathd/main.c | 113 ++++++++--- 13 files changed, 445 insertions(+), 242 deletions(-)Signature · 1 line
2.26.2