change how multipathd deletes maps plus cleanups
Benjamin Marzinski
maintainerv2 cover change how multipathd deletes maps plus cleanups
The main purpose of this patchset is to make multipathd call dm_suspend_and_flush_map() for the "del map" and "del maps" interactive commands. I ran into an issue where if there was outstanding IO to a queueing multipath device, dm_flush_map() would hang removing the last kpartx device, since that would close the last opener of the multipath device, forcing it to wait for the IO to flush. This is avoided by dm_suspend_and_flush_map(), since it disables queueing first. The first three patches do this. The next three are cleanups around removing a device. The rest of the patches are make multipath return better errors when it delegates removes since it doesn't need to failback to doing its own removes anymore, make libdmmp handle failures from running multipathd commands better, and make multipath and multipathc return proper error codes when multipathd commands fail.
Differences from v1 (based on suggestions by Martin Wilck) 0003: simplify dm_flush_maps(), since all callers set need_suspend 0005: Minor change in dm_flush_maps() to accomodate change in 0003. I
retained the reviewed-by.
0007: Stripped the initial "fail\n" off messages with additional failure
information.
0009: Minor changes in dm_flush_maps() and its callers to accomodate
change in 0003. I retained the reviewed-by.
0010: changed the log level of the error message to 2.
0012: search for path by dev_t first, and change the log level of the
error message to 2.
0015: Minor change to calling dm_flush_maps() to accomodate change in
0003. I retained the reviewed-by
0016: multipath repeatedly calls delegate_to_multipathd for all
remove_retries, instead of failing back to locally calling remove.
0017: New patch based on patch 0016 discussion.
0018: New patch to deal with a issue caused by the change to track the
queue_if_no_path state in mpp->features, from my previous
patchset.Benjamin Marzinski (18):
multipathd: remove nopath flushing code from flush_map()
multipathd: make flush_map() delete maps like the multipath command
multipathd: disable queueing when removing unknown maps
multipathd: simplify cli_del_map()
libmultipath: make _dm_flush_map() return an enum
libmultipath: make dm_remove_partmaps() a static function
multipathd: always start failure replies with "fail\n"
multipathd: print extra default reply msg for busy devices
multipathd: handle busy devices in cli_del_maps()
libmultipath: print error when find_mp_by_str() fails.
multipathd: don't open code find_mp_by_str() in client handers
multipathd: make cli_handlers check for paths by dev and devt
multipathd: add cli_handler reply message for missing devs
libdmmp: handle failures in _process_cmd
multipath: get rid of unnecessary retries variable
multipath: Don't locally retry deletgated remove failures
multipath: if delegation times out mark as not delegated
multipathd: sync features on flush_map failure corner case
libdmmp/libdmmp.c | 71 +++++------- libmultipath/devmapper.c | 47 ++++---- libmultipath/devmapper.h | 13 ++- libmultipath/libmultipath.version | 3 +- libmultipath/structs.c | 9 +- multipath/main.c | 72 +++++++----- multipathd/cli_handlers.c | 182 ++++++++++++++---------------- multipathd/main.c | 70 ++++-------- multipathd/main.h | 3 +- multipathd/uxclnt.c | 9 +- multipathd/uxlsnr.c | 15 ++- 11 files changed, 241 insertions(+), 253 deletions(-)
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 1/18 multipathd: remove nopath flushing code from flush_map()
Instead of flush_map() handling both user requested flushes and automatic flushes when the last path has been deleted, make flush_map_nopaths() handle the automatic flushes itself, since a later patch will change the behavior of flush_map().
--- multipathd/cli_handlers.c | 2 +- multipathd/main.c | 45 +++++++++++++++++---------------------- multipathd/main.h | 2 +- 3 files changed, 21 insertions(+), 28 deletions(-)
multipathd/cli_handlers.c +1 -1
@@ -775,7 +775,7 @@ cli_del_maps (void *v, struct strbuf *reply, void *data)condlog (2 ,"remove maps (operator)" ); vector_foreach_slot(vecs->mpvec, mpp, i) { -if (flush_map (mpp, vecs,0 )) +if (flush_map (mpp, vecs)) ret++;else i--;
multipathd/main.c +19 -26
79 lines not on this page · raw message
multipathd/main.h +1 -1
@@ -40,7 +40,7 @@ int ev_add_path (struct path *, struct vectors *, int);int ev_remove_path (struct path *,struct vectors *,int );int ev_add_map (char *,const char *,struct vectors *);int ev_remove_map (char *,char *,int ,struct vectors *); -int flush_map (struct multipath *,struct vectors *,int ); +int flush_map (struct multipath *,struct vectors *);void handle_signals (bool );int refresh_multipath (struct vectors * vecs,struct multipath * mpp);
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 2/18 multipathd: make flush_map() delete maps like the multipath command
In reply to Benjamin MarzinskiWhen the multipath command tries to delete a multipath device, it first disables queueing and then suspends the device to force the IOs to get flushed. Then it attempts to delete the device and any kpartx partitions. multipathd, on the other hand, simply tries to delete the device and kpartx partitions, without disabling queueing or suspending. If there are no paths but there is outstanding IO, multipathd will hang trying to delete the last kpartx device. This is because it must be the last opener of the multipath device (multipath won't try to delete the device if it has any openers besides the kpartx devices) and the kernel will not allow the last opener of a block device to close until all the outstanding IO is flushed. This hang can be avoided if multipathd calls dm_suspend_and_flush_map() like the multipath command does, instead of dm_flush_map().
--- multipathd/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
multipathd/main.c +1 -1
@@ -788,7 +788,7 @@ sync_maps_state(vector mpvec)int flush_map (struct multipath * mpp,struct vectors * vecs) { -int r =dm_flush_map (mpp->alias); +int r =dm_suspend_and_flush_map (mpp->alias,0 );if (r) {condlog (0 ,"%s: can't flush" , mpp->alias);return r;
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 3/18 multipathd: disable queueing when removing unknown maps
In reply to Benjamin MarzinskiMake cli_del_maps() call dm_suspend_and_flush_map() for the unknown multipath devices as well.
After this change, all callers of cli_del_maps() set need_suspend, so simplify dm_flush_maps().
--- libmultipath/devmapper.c | 7 ++----- libmultipath/devmapper.h | 2 +- multipath/main.c | 2 +- multipathd/cli_handlers.c | 2 +- 4 files changed, 5 insertions(+), 8 deletions(-)
libmultipath/devmapper.c +2 -5
@@ -1159,7 +1159,7 @@ dm_flush_map_nopaths(const char * mapname,#endif -int dm_flush_maps (int need_suspend,int retries) +int dm_flush_maps (int retries) {int r =1 ;struct dm_task *dmt;
@@ -1184,10 +1184,7 @@ int dm_flush_maps (int need_suspend, int retries) goto out;do { -if (need_suspend) - r |=dm_suspend_and_flush_map (names->name, retries); -else - r |=dm_flush_map (names->name); + r |=dm_suspend_and_flush_map (names->name, retries); next = names->next; names = (void *) names + next; }while (next);
libmultipath/devmapper.h +1 -1
@@ -55,7 +55,7 @@ int dm_flush_map_nopaths(const char * mapname, int deferred_remove);#define dm_suspend_and_flush_map (mapname, retries) \ _dm_flush_map(mapname, 1, 0, 1, retries)int dm_cancel_deferred_remove (struct multipath *mpp); -int dm_flush_maps (int need_suspend,int retries); +int dm_flush_maps (int retries);int dm_fail_path (const char * mapname,char * path);int dm_reinstate_path (const char * mapname,char * path);int dm_queue_if_no_path (struct multipath *mpp,int enable);
multipath/main.c +1 -1
@@ -1075,7 +1075,7 @@ main (int argc, char *argv[]) goto out; }else if (cmd ==CMD_FLUSH_ALL ) { - r =dm_flush_maps (1 , retries) ?RTVL_FAIL :RTVL_OK ; + r =dm_flush_maps (retries) ?RTVL_FAIL :RTVL_OK ; goto out; }while ((r =configure (conf, cmd, dev_type, dev)) ==RTVL_RETRY )
multipathd/cli_handlers.c +1 -1
@@ -781,7 +781,7 @@ cli_del_maps (void *v, struct strbuf *reply, void *data) i--; }/* flush any multipath maps that aren't currently known by multipathd */ - ret |=dm_flush_maps (0 ,0 ); + ret |=dm_flush_maps (0 );return ret; }
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 4/18 multipathd: simplify cli_del_map()
In reply to Benjamin Marzinskicli_del_map() does a lot of unnecessary work to match the arguments of ev_remove_map(), of which it is the only caller. Then ev_remove_map() does more unnecessary work to verify the arguments passed in. remove ev_remove_map() and make cli_del_map() get the mpp like the rest of the client handlers do.
--- multipathd/cli_handlers.c | 23 ++++++----------------- multipathd/main.c | 21 --------------------- multipathd/main.h | 1 - 3 files changed, 6 insertions(+), 39 deletions(-)
multipathd/cli_handlers.c +6 -17
@@ -742,28 +742,17 @@ cli_del_map (void * v, struct strbuf *reply, void * data) {struct vectors * vecs = (struct vectors *)data;char * param =get_keyparam (v,KEY_MAP ); -int major, minor; -char *alias; -int rc; +struct multipath *mpp; param =convert_dev (param,0 );condlog (2 ,"%s: remove map (operator)" , param); -if (dm_get_major_minor (param, &major, &minor) <0 ) { -condlog (2 ,"%s: not a device mapper table" , param); -return 1 ; - } - alias =dm_mapname (major, minor); -if (!alias) { -condlog (2 ,"%s: mapname not found for %d:%d" , - param, major, minor); + + mpp =find_mp_by_str (vecs->mpvec, param); + +if (!mpp)return 1 ; - } - rc =ev_remove_map (param, alias, minor, vecs); -if (rc ==2 ) -append_strbuf_str (reply,"delayed\n" ); -free (alias); -return rc; +return flush_map (mpp, vecs); } static int
multipathd/main.c +0 -21
@@ -931,27 +931,6 @@ out:return 0 ; } -/* Called from CLI handler */ -int -ev_remove_map (char * devname,char * alias,int minor,struct vectors * vecs) -{ -struct multipath * mpp; - - mpp =find_mp_by_minor (vecs->mpvec, minor); - -if (!mpp) { -condlog (2 ,"%s: devmap not registered, can't remove" , - devname); -return 1 ; - } -if (strcmp (mpp->alias, alias)) { -condlog (2 ,"%s: minor number mismatch (map %d, event %d)" , - mpp->alias, mpp->dmi.minor, minor); -return 1 ; - } -return flush_map (mpp, vecs); -} -static void rescan_path (struct udev_device *ud) {
multipathd/main.h +0 -1
@@ -39,7 +39,6 @@ int need_to_delay_reconfig (struct vectors *);int ev_add_path (struct path *,struct vectors *,int );int ev_remove_path (struct path *,struct vectors *,int );int ev_add_map (char *,const char *,struct vectors *); -int ev_remove_map (char *,char *,int ,struct vectors *);int flush_map (struct multipath *,struct vectors *);void handle_signals (bool );
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 5/18 libmultipath: make _dm_flush_map() return an enum
In reply to Benjamin MarzinskiA future patch will add an additional return code, so make this an enum instead of just using numbers.
--- libmultipath/devmapper.c | 18 +++++++++--------- libmultipath/devmapper.h | 7 +++++++ multipath/main.c | 2 +- multipathd/main.c | 12 ++++++------ 4 files changed, 23 insertions(+), 16 deletions(-)
libmultipath/devmapper.c +9 -9
@@ -1071,7 +1071,7 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove,char *params =NULL ;if (dm_is_mpath (mapname) !=1 ) -return 0 ;/* nothing to do */ +return DM_FLUSH_OK ;/* nothing to do */ /* if the device currently has no partitions, do not run kpartx on it if you fail to delete it */
@@ -1081,7 +1081,7 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove,/* If you aren't doing a deferred remove, make sure that no * devices are in use */ if (!do_deferred (deferred_remove) &&partmap_in_use (mapname,NULL )) -return 1 ; +return DM_FLUSH_FAIL ; if (need_suspend && dm_get_map(mapname, &mapsize, ¶ms) == DMP_OK &&
@@ -1096,11 +1096,11 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove, params =NULL ;if (dm_remove_partmaps (mapname, need_sync, deferred_remove)) -return 1 ; +return DM_FLUSH_FAIL ;if (!do_deferred (deferred_remove) &&dm_get_opencount (mapname)) {condlog (2 ,"%s: map in use" , mapname); -return 1 ; +return DM_FLUSH_FAIL ; } do {
@@ -1114,14 +1114,14 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove, && dm_map_present(mapname)) {condlog (4 ,"multipath map %s remove deferred" , mapname); -return 2 ; +return DM_FLUSH_DEFERRED ; }condlog (4 ,"multipath map %s removed" , mapname); -return 0 ; +return DM_FLUSH_OK ; }else if (dm_is_mpath (mapname) !=1 ) {condlog (4 ,"multipath map %s removed externally" , mapname); -return 0 ;/*we raced with someone else removing it */ +return DM_FLUSH_OK ;/* raced. someone else removed it */ }else {condlog (2 ,"failed to remove multipath map %s" , mapname);
@@ -1137,7 +1137,7 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove,if (queue_if_no_path ==1 )_dm_queue_if_no_path (mapname,1 ); -return 1 ; +return DM_FLUSH_FAIL ; }#ifdef LIBDM_API_DEFERRED
@@ -1184,7 +1184,7 @@ int dm_flush_maps (int retries) goto out;do { - r |=dm_suspend_and_flush_map (names->name, retries); + r |=dm_suspend_and_flush_map (names->name, retries) !=DM_FLUSH_OK ; next = names->next; names = (void *) names + next; }while (next);
libmultipath/devmapper.h +7 -0
@@ -48,6 +48,13 @@ int dm_get_map(const char *, unsigned long long *, char **);int dm_get_status (const char *,char **);int dm_type (const char *,char *);int dm_is_mpath (const char *); + +enum { +DM_FLUSH_OK =0 , +DM_FLUSH_FAIL , +DM_FLUSH_DEFERRED , +}; +int _dm_flush_map (const char *,int ,int ,int ,int );int dm_flush_map_nopaths (const char * mapname,int deferred_remove);#define dm_flush_map (mapname) _dm_flush_map(mapname, 1, 0, 0, 0)
multipath/main.c +1 -1
@@ -1070,7 +1070,7 @@ main (int argc, char *argv[]) r =RTVL_FAIL ; goto out; } - r =dm_suspend_and_flush_map (dev, retries) ? + r = (dm_suspend_and_flush_map (dev, retries) !=DM_FLUSH_OK ) ?RTVL_FAIL :RTVL_OK ; goto out; }
multipathd/main.c +6 -6
@@ -598,13 +598,13 @@ flush_map_nopaths(struct multipath *mpp, struct vectors *vecs) {dm_queue_if_no_path (mpp,0 ); } r =dm_flush_map_nopaths (mpp->alias, mpp->deferred_remove); -if (r) { -if (r ==1 ) -condlog (0 ,"%s: can't flush" , mpp->alias); -else { +if (r !=DM_FLUSH_OK ) { +if (r ==DM_FLUSH_DEFERRED ) {condlog (2 ,"%s: devmap deferred remove" , mpp->alias); mpp->deferred_remove =DEFERRED_REMOVE_IN_PROGRESS ; } +else +condlog (0 ,"%s: can't flush" , mpp->alias);return false; }
@@ -746,7 +746,7 @@ coalesce_maps(struct vectors *vecs, vector nmpv)* remove all current maps not allowed by the * current configuration */ -if (dm_flush_map (ompp->alias)) { +if (dm_flush_map (ompp->alias) !=DM_FLUSH_OK ) {condlog (0 ,"%s: unable to flush devmap" , ompp->alias);/*
@@ -789,7 +789,7 @@ int flush_map(struct multipath * mpp, struct vectors * vecs) { int r = dm_suspend_and_flush_map(mpp->alias, 0); -if (r) { +if (r !=DM_FLUSH_OK ) {condlog (0 ,"%s: can't flush" , mpp->alias);return r; }
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 6/18 libmultipath: make dm_remove_partmaps() a static function
In reply to Benjamin Marzinskidm_remove_partmaps() is only used in devmapper.c, so make it static. It does need to be declared early, since remove_partmaps() and it call eachother.
--- libmultipath/devmapper.c | 4 +++- libmultipath/devmapper.h | 2 -- 2 files changed, 3 insertions(+), 3 deletions(-)
libmultipath/devmapper.c +3 -1
@@ -53,6 +53,8 @@ static int dm_cancel_remove_partmaps(const char * mapname);#define __DR_UNUSED__ __attribute__((unused))#endif +static int dm_remove_partmaps (const char * mapname,int need_sync, +int deferred_remove);static int do_foreach_partmaps (const char * mapname,int (*partmap_func)(const char *,void *),void *data);
@@ -1544,7 +1546,7 @@ remove_partmap(const char *name, void *data)return 0 ; } -int +static int dm_remove_partmaps (const char * mapname,int need_sync,int deferred_remove) {struct remove_data rd = { need_sync, deferred_remove };
libmultipath/devmapper.h +0 -2
@@ -74,8 +74,6 @@ int dm_geteventnr (const char *name);int dm_is_suspended (const char *name);int dm_get_major_minor (const char *name,int *major,int *minor);char *dm_mapname (int major,int minor); -int dm_remove_partmaps (const char * mapname,int need_sync, -int deferred_remove);int dm_get_uuid (const char *name,char *uuid,int uuid_len);bool has_dm_info (const struct multipath *mpp);int dm_get_info (const char * mapname,struct dm_info *dmi);
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 7/18 multipathd: always start failure replies with "fail\n"
In reply to Benjamin MarzinskiWhen multipathd interactive commands fail for certain reasons, like timing out or incorrect permissions, they do not reply with "fail\n". Currently, multipath and multipathc expect that a reply other than "fail\n" means success. Instead, prefix all failure replies with "fail\n", and adapt multipath and multipathc to return failure for any reply starting with "fail\n".
--- libdmmp/libdmmp.c | 6 +++--- multipath/main.c | 13 ++++++++++--- multipathd/uxclnt.c | 9 +++++++-- multipathd/uxlsnr.c | 12 +++++------- 4 files changed, 25 insertions(+), 15 deletions(-)
libdmmp/libdmmp.c +3 -3
@@ -321,7 +321,7 @@ invoke: } }if ((*output !=NULL ) && - (strncmp (*output,"timeout" ,strlen ("timeout" )) ==0 )) + (strncmp (*output,"fail\ntimeout" ,strlen ("fail\ntimeout" )) ==0 )) flag_check_tmo = true;if (flag_check_tmo == true) {
@@ -364,8 +364,8 @@ invoke: }if ((*output !=NULL ) && -strncmp (*output,"permission deny" , -strlen ("permission deny" )) ==0 ) { +strncmp (*output,"fail\npermission deny" , +strlen ("fail\npermission deny" )) ==0 ) {_error (ctx,"Permission deny, need to be root" ); rc =DMMP_ERR_PERMISSION_DENY ; goto out;
multipath/main.c +10 -3
@@ -804,10 +804,17 @@ int delegate_to_multipathd(enum mpath_cmds cmd, }if (reply !=NULL && *reply !='\0' ) { -if (strcmp (reply,"fail\n" )) +if (strncmp (reply,"fail\n" ,5 )) r =DELEGATE_OK ; -if (r !=NOT_DELEGATED &&strcmp (reply,"ok\n" )) -printf ("%s" , reply); +if (r !=NOT_DELEGATED &&strcmp (reply,"ok\n" )) { +/* If there is additional failure information, skip the +* initial 'fail' */ +if (strncmp (reply,"fail\n" ,5 ) ==0 && +strlen (reply) >5 ) +printf ("%s" , reply +5 ); +else +printf ("%s" , reply); + } } out:
multipathd/uxclnt.c +7 -2
@@ -30,8 +30,13 @@ static int process_req(int fd, char * inbuf, unsigned int timeout)printf ("error %d receiving packet\n" , ret);return 1 ; }else { -printf ("%s" , reply); - ret = (strcmp (reply,"fail\n" ) ==0 ); + ret = (strncmp (reply,"fail\n" ,5 ) ==0 ); +/* If there is additional failure information, skip the +* initial 'fail' */ +if (ret &&strlen (reply) >5 ) +printf ("%s" , reply +5 ); +else +printf ("%s" , reply);free (reply);return ret; }
multipathd/uxlsnr.c +5 -7
@@ -392,6 +392,11 @@ static void drain_idle_fd(int fd)void default_reply (struct client *c,int r) { +if (r ==0 ) { +append_strbuf_str (&c->reply,"ok\n" ); +return ; + } +append_strbuf_str (&c->reply,"fail\n" );switch (r) {case -EINVAL :case -ESRCH :
@@ -406,13 +411,6 @@ void default_reply(struct client *c, int r)case -ETIMEDOUT :append_strbuf_str (&c->reply,"timeout\n" );break ; -case 0 : -append_strbuf_str (&c->reply,"ok\n" ); -break ; -default : -/* cli_handler functions return 1 on unspecified error */ -append_strbuf_str (&c->reply,"fail\n" ); -break ; } }
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 8/18 multipathd: print extra default reply msg for busy devices
In reply to Benjamin MarzinskiIf removing a multipath device fails because the device is in use, return DM_FLUSH_BUSY from remove_functions, which causes cli_del_map() to return -EBUSY, which will now print extra information in default_reply().
--- libmultipath/devmapper.c | 14 +++++++------- libmultipath/devmapper.h | 1 + multipathd/cli_handlers.c | 9 ++++++++- multipathd/uxlsnr.c | 3 +++ 4 files changed, 19 insertions(+), 8 deletions(-)
libmultipath/devmapper.c +7 -7
@@ -1083,7 +1083,7 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove,/* If you aren't doing a deferred remove, make sure that no * devices are in use */ if (!do_deferred (deferred_remove) &&partmap_in_use (mapname,NULL )) -return DM_FLUSH_FAIL ; +return DM_FLUSH_BUSY ; if (need_suspend && dm_get_map(mapname, &mapsize, ¶ms) == DMP_OK &&
@@ -1097,12 +1097,12 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove,free (params); params =NULL ; -if (dm_remove_partmaps (mapname, need_sync, deferred_remove)) -return DM_FLUSH_FAIL ; +if ((r =dm_remove_partmaps (mapname, need_sync, deferred_remove))) +return r;if (!do_deferred (deferred_remove) &&dm_get_opencount (mapname)) {condlog (2 ,"%s: map in use" , mapname); -return DM_FLUSH_FAIL ; +return DM_FLUSH_BUSY ; } do {
@@ -1506,7 +1506,7 @@ do_foreach_partmaps (const char * mapname, (p = strstr(params, dev_t)) && !isdigit(*(p + strlen(dev_t))) ) { -if (partmap_func (names->name, data) !=0 ) +if ((r =partmap_func (names->name, data)) !=0 ) goto out; }
@@ -1538,12 +1538,12 @@ remove_partmap(const char *name, void *data)if (!do_deferred (rd->deferred_remove) &&dm_get_opencount (name)) {condlog (2 ,"%s: map in use" , name); -return 1 ; +return DM_FLUSH_BUSY ; } }condlog (4 ,"partition map %s removed" , name);dm_device_remove (name, rd->need_sync, rd->deferred_remove); -return 0 ; +return DM_FLUSH_OK ; } static int
libmultipath/devmapper.h +1 -0
@@ -53,6 +53,7 @@ enum {DM_FLUSH_OK =0 ,DM_FLUSH_FAIL ,DM_FLUSH_DEFERRED , + DM_FLUSH_BUSY, };int _dm_flush_map (const char *,int ,int ,int ,int );
multipathd/cli_handlers.c +8 -1
@@ -31,6 +31,7 @@#include "foreign.h" #include "strbuf.h" #include "cli_handlers.h" +#include "devmapper.h" static int show_paths (struct strbuf *reply,struct vectors *vecs,char *style,int pretty)
@@ -743,6 +744,7 @@ cli_del_map (void * v, struct strbuf *reply, void * data)struct vectors * vecs = (struct vectors *)data;char * param =get_keyparam (v,KEY_MAP );struct multipath *mpp; +int r; param =convert_dev (param,0 );condlog (2 ,"%s: remove map (operator)" , param);
@@ -752,7 +754,12 @@ cli_del_map (void * v, struct strbuf *reply, void * data)if (!mpp)return 1 ; -return flush_map (mpp, vecs); + r =flush_map (mpp, vecs); +if (r ==DM_FLUSH_OK ) +return 0 ; +else if (r ==DM_FLUSH_BUSY ) +return -EBUSY ; +return 1 ; } static int
multipathd/uxlsnr.c +3 -0
@@ -411,6 +411,9 @@ void default_reply(struct client *c, int r)case -ETIMEDOUT :append_strbuf_str (&c->reply,"timeout\n" );break ; +case -EBUSY : +append_strbuf_str (&c->reply,"map or partition in use\n" ); +break ; } }
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 9/18 multipathd: handle busy devices in cli_del_maps()
In reply to Benjamin MarzinskiMake cli_del_maps() return -EBUSY like cli_del_map() if it fails because a device is in use and it doesn't run into any other type of failures.
--- libmultipath/devmapper.c | 10 +++++++--- multipath/main.c | 3 ++- multipathd/cli_handlers.c | 19 +++++++++++++------ 3 files changed, 22 insertions(+), 10 deletions(-)
libmultipath/devmapper.c +7 -3
@@ -1163,7 +1163,7 @@ dm_flush_map_nopaths(const char * mapname,int dm_flush_maps (int retries) { -int r =1 ; +int r =DM_FLUSH_FAIL ;struct dm_task *dmt;struct dm_names *names;unsigned next =0 ;
@@ -1181,12 +1181,16 @@ int dm_flush_maps (int retries)if (!(names =dm_task_get_names (dmt))) goto out; - r =0 ; + r =DM_FLUSH_OK ;if (!names->dev) goto out;do { - r |=dm_suspend_and_flush_map (names->name, retries) !=DM_FLUSH_OK ; +int ret; + ret =dm_suspend_and_flush_map (names->name, retries); +if (ret ==DM_FLUSH_FAIL || + (r !=DM_FLUSH_FAIL && ret ==DM_FLUSH_BUSY )) + r = ret; next = names->next; names = (void *) names + next; }while (next);
multipath/main.c +2 -1
@@ -1082,7 +1082,8 @@ main (int argc, char *argv[]) goto out; }else if (cmd ==CMD_FLUSH_ALL ) { - r =dm_flush_maps (retries) ?RTVL_FAIL :RTVL_OK ; + r = (dm_flush_maps (retries) !=DM_FLUSH_OK ) ? +RTVL_FAIL :RTVL_OK ; goto out; }while ((r =configure (conf, cmd, dev_type, dev)) ==RTVL_RETRY )
multipathd/cli_handlers.c +13 -6
@@ -767,18 +767,25 @@ cli_del_maps (void *v, struct strbuf *reply, void *data) {struct vectors * vecs = (struct vectors *)data;struct multipath *mpp; -int i, ret =0 ; +int i, ret, r =0 ;condlog (2 ,"remove maps (operator)" ); vector_foreach_slot(vecs->mpvec, mpp, i) { -if (flush_map (mpp, vecs)) - ret++; -else + ret =flush_map (mpp, vecs); +if (ret ==DM_FLUSH_OK ) i--; +else if (ret ==DM_FLUSH_BUSY && r !=1 ) + r = -EBUSY ; +else + r =1 ; }/* flush any multipath maps that aren't currently known by multipathd */ - ret |=dm_flush_maps (0 ); -return ret; + ret =dm_flush_maps (0 ); +if (ret ==DM_FLUSH_BUSY && r !=1 ) + r = -EBUSY ; +else if (ret !=DM_FLUSH_OK ) + r =1 ; +return r; } static int
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 10/18 libmultipath: print error when find_mp_by_str() fails.
In reply to Benjamin MarzinskiNone of the callers of find_mp_by_str() print any message if they fail because the map name is invalid. Print one in find_mp_by_str() to save the effort of adding it to all the callers.
--- libmultipath/structs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
libmultipath/structs.c +7 -2
@@ -480,11 +480,16 @@ struct multipath * find_mp_by_str (const struct _vector *mpvec, const char * str) { int minor; +struct multipath *mpp;if (sscanf (str,"dm-%d" , &minor) ==1 ) -return find_mp_by_minor (mpvec, minor); + mpp =find_mp_by_minor (mpvec, minor);else -return find_mp_by_alias (mpvec, str); + mpp =find_mp_by_alias (mpvec, str); + +if (!mpp) +condlog (2 ,"%s: invalid map name." , str); +return mpp; } struct path *
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 11/18 multipathd: don't open code find_mp_by_str() in client handers
In reply to Benjamin Marzinskimultiple client handlers simply open coded find_mp_by_str(). Just use the function instead.
--- multipathd/cli_handlers.c | 53 +++++++++------------------------------ 1 file changed, 12 insertions(+), 41 deletions(-)
multipathd/cli_handlers.c +12 -41
@@ -794,19 +794,14 @@ cli_reload(void *v, struct strbuf *reply, void *data)struct vectors * vecs = (struct vectors *)data;char * mapname =get_keyparam (v,KEY_MAP );struct multipath *mpp; -int minor; mapname =convert_dev (mapname,0 );condlog (2 ,"%s: reload map (operator)" , mapname); -if (sscanf (mapname,"dm-%d" , &minor) ==1 ) - mpp =find_mp_by_minor (vecs->mpvec, minor); -else - mpp =find_mp_by_alias (vecs->mpvec, mapname); + mpp =find_mp_by_str (vecs->mpvec, mapname); -if (!mpp) { -condlog (0 ,"%s: invalid map name. cannot reload" , mapname); +if (!mpp)return 1 ; - } +if (mpp->wait_for_udev) {condlog (2 ,"%s: device not fully created, failing reload" , mpp->alias);
@@ -822,7 +817,6 @@ cli_resize(void *v, struct strbuf *reply, void *data)struct vectors * vecs = (struct vectors *)data;char * mapname =get_keyparam (v,KEY_MAP );struct multipath *mpp; -int minor;unsigned long long size =0 ;struct pathgroup *pgp;struct path *pp;
@@ -831,15 +825,10 @@ cli_resize(void *v, struct strbuf *reply, void *data) mapname =convert_dev (mapname,0 );condlog (2 ,"%s: resize map (operator)" , mapname); -if (sscanf (mapname,"dm-%d" , &minor) ==1 ) - mpp =find_mp_by_minor (vecs->mpvec, minor); -else - mpp =find_mp_by_alias (vecs->mpvec, mapname); + mpp =find_mp_by_str (vecs->mpvec, mapname); -if (!mpp) { -condlog (0 ,"%s: invalid map name. cannot resize" , mapname); +if (!mpp)return 1 ; - }if (mpp->wait_for_udev) { condlog(2, "%s: device not fully created, failing resize",
@@ -916,20 +905,14 @@ cli_restore_queueing(void *v, struct strbuf *reply, void *data)struct vectors * vecs = (struct vectors *)data;char * mapname =get_keyparam (v,KEY_MAP );struct multipath *mpp; -int minor;struct config *conf; mapname =convert_dev (mapname,0 );condlog (2 ,"%s: restore map queueing (operator)" , mapname); -if (sscanf (mapname,"dm-%d" , &minor) ==1 ) - mpp =find_mp_by_minor (vecs->mpvec, minor); -else - mpp =find_mp_by_alias (vecs->mpvec, mapname); + mpp =find_mp_by_str (vecs->mpvec, mapname); -if (!mpp) { -condlog (0 ,"%s: invalid map name, cannot restore queueing" , mapname); +if (!mpp)return 1 ; - }if (mpp->disable_queueing) { mpp->disable_queueing =0 ;
@@ -973,19 +956,13 @@ cli_disable_queueing(void *v, struct strbuf *reply, void *data)struct vectors * vecs = (struct vectors *)data;char * mapname =get_keyparam (v,KEY_MAP );struct multipath *mpp; -int minor; mapname =convert_dev (mapname,0 );condlog (2 ,"%s: disable map queueing (operator)" , mapname); -if (sscanf (mapname,"dm-%d" , &minor) ==1 ) - mpp =find_mp_by_minor (vecs->mpvec, minor); -else - mpp =find_mp_by_alias (vecs->mpvec, mapname); + mpp =find_mp_by_str (vecs->mpvec, mapname); -if (!mpp) { -condlog (0 ,"%s: invalid map name, cannot disable queueing" , mapname); +if (!mpp)return 1 ; - }if (count_active_paths (mpp) ==0 ) mpp->stat_map_failures++;
@@ -1462,22 +1439,16 @@ static int cli_unset_all_marginal(void * v, struct strbuf *reply, void * data)struct pathgroup * pgp;struct path * pp;unsigned int i, j; -int minor; mapname =convert_dev (mapname,0 );condlog (2 ,"%s: unset all marginal paths (operator)" , mapname); -if (sscanf (mapname,"dm-%d" , &minor) ==1 ) - mpp =find_mp_by_minor (vecs->mpvec, minor); -else - mpp =find_mp_by_alias (vecs->mpvec, mapname); + mpp =find_mp_by_str (vecs->mpvec, mapname); -if (!mpp) { -condlog (0 ,"%s: invalid map name. " -"cannot unset marginal paths" , mapname); +if (!mpp)return 1 ; - } +if (mpp->wait_for_udev) {condlog (2 ,"%s: device not fully created, " "failing unset all marginal" , mpp->alias);
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 12/18 multipathd: make cli_handlers check for paths by dev and devt
In reply to Benjamin MarzinskiSome of the client handlers checked for paths by both dev and devt, but not all. Also, many of the client handlers don't print anything if they failed to find a path. Make all the client handlers which work on path devices use a new function, find_path_by_str(), which will try both methods to find a path, and can print out an error message if none is found.
--- multipathd/cli_handlers.c | 40 ++++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 19 deletions(-)
multipathd/cli_handlers.c +21 -19
@@ -33,6 +33,20 @@#include "cli_handlers.h" #include "devmapper.h" +static struct path * +find_path_by_str (const struct _vector *pathvec,const char *str, +const char *action_str) +{ +struct path *pp; + +if (!(pp =find_path_by_devt (pathvec, str))) + pp =find_path_by_dev (pathvec, str); + +if (!pp && action_str) +condlog (2 ,"%s: invalid path name. cannot %s" , str, action_str); +return pp; +} +static int show_paths (struct strbuf *reply,struct vectors *vecs,char *style,int pretty) {
@@ -241,7 +255,7 @@ cli_list_path (void *v, struct strbuf *reply, void *data) param =convert_dev (param,1 );condlog (3 ,"%s: list path (operator)" , param); - pp =find_path_by_dev (vecs->pathvec, param); + pp =find_path_by_str (vecs->pathvec, param,"list path" );if (!pp)return 1 ;
@@ -556,7 +570,7 @@ cli_add_path (void *v, struct strbuf *reply, void *data)if (invalid) goto blacklisted; - pp =find_path_by_dev (vecs->pathvec, param); + pp =find_path_by_str (vecs->pathvec, param,NULL );if (pp && pp->initialized !=INIT_REMOVED ) {condlog (2 ,"%s: path already in pathvec" , param);
@@ -665,7 +679,7 @@ cli_del_path (void * v, struct strbuf *reply, void * data) param =convert_dev (param,1 );condlog (2 ,"%s: remove path (operator)" , param); - pp =find_path_by_dev (vecs->pathvec, param); + pp =find_path_by_str (vecs->pathvec, param,NULL );if (!pp) {condlog (0 ,"%s: path already removed" , param);return 1 ;
@@ -1092,10 +1106,7 @@ cli_reinstate(void * v, struct strbuf *reply, void * data)struct path * pp; param =convert_dev (param,1 ); - pp =find_path_by_dev (vecs->pathvec, param); - -if (!pp) - pp =find_path_by_devt (vecs->pathvec, param); + pp =find_path_by_str (vecs->pathvec, param,"reinstate path" );if (!pp || !pp->mpp || !pp->mpp->alias)return 1 ;
@@ -1140,10 +1151,7 @@ cli_fail(void * v, struct strbuf *reply, void * data)int r; param =convert_dev (param,1 ); - pp =find_path_by_dev (vecs->pathvec, param); - -if (!pp) - pp =find_path_by_devt (vecs->pathvec, param); + pp =find_path_by_str (vecs->pathvec, param,"fail path" );if (!pp || !pp->mpp || !pp->mpp->alias)return 1 ;
@@ -1384,10 +1392,7 @@ static int cli_set_marginal(void * v, struct strbuf *reply, void * data)struct path * pp; param =convert_dev (param,1 ); - pp =find_path_by_dev (vecs->pathvec, param); - -if (!pp) - pp =find_path_by_devt (vecs->pathvec, param); + pp =find_path_by_str (vecs->pathvec, param,"set marginal path" );if (!pp || !pp->mpp || !pp->mpp->alias)return 1 ;
@@ -1411,10 +1416,7 @@ static int cli_unset_marginal(void * v, struct strbuf *reply, void * data)struct path * pp; param =convert_dev (param,1 ); - pp =find_path_by_dev (vecs->pathvec, param); - -if (!pp) - pp =find_path_by_devt (vecs->pathvec, param); + pp =find_path_by_str (vecs->pathvec, param,"unset marginal path" );if (!pp || !pp->mpp || !pp->mpp->alias)return 1 ;
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 13/18 multipathd: add cli_handler reply message for missing devs
In reply to Benjamin MarzinskiWhen the cli_handlers cannot find the requested map or path, they will now return -ENODEV, which prints extra information in default_reply().
--- multipathd/cli_handlers.c | 56 ++++++++++++++++++++++++--------------- multipathd/uxlsnr.c | 2 ++ 2 files changed, 36 insertions(+), 22 deletions(-)
multipathd/cli_handlers.c +34 -22
@@ -257,7 +257,7 @@ cli_list_path (void *v, struct strbuf *reply, void *data) pp =find_path_by_str (vecs->pathvec, param,"list path" );if (!pp) -return 1 ; +return -ENODEV ;return show_path (reply, vecs, pp,"%o" ); }
@@ -277,7 +277,7 @@ cli_list_map_topology (void *v, struct strbuf *reply, void *data) mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ;condlog (3 ,"list multipath %s (operator)" , param);
@@ -305,7 +305,7 @@ cli_list_map_json (void *v, struct strbuf *reply, void *data) mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ;condlog (3 ,"list multipath json %s (operator)" , param);
@@ -434,7 +434,7 @@ cli_list_map_fmt (void *v, struct strbuf *reply, void *data) param =convert_dev (param,0 ); mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ;condlog (3 ,"list map %s fmt %s (operator)" , param, fmt);
@@ -515,7 +515,7 @@ cli_reset_map_stats (void *v, struct strbuf *reply, void *data) mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ;condlog (3 ,"reset multipath %s stats (operator)" , param);reset_stats (mpp);
@@ -682,7 +682,7 @@ cli_del_path (void * v, struct strbuf *reply, void * data) pp =find_path_by_str (vecs->pathvec, param,NULL );if (!pp) {condlog (0 ,"%s: path already removed" , param); -return 1 ; +return -ENODEV ; } ret =ev_remove_path (pp, vecs,1 );if (ret ==REMOVE_PATH_DELAY )
@@ -766,7 +766,7 @@ cli_del_map (void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ; r =flush_map (mpp, vecs);if (r ==DM_FLUSH_OK )
@@ -814,7 +814,7 @@ cli_reload(void *v, struct strbuf *reply, void *data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ;if (mpp->wait_for_udev) { condlog(2, "%s: device not fully created, failing reload",
@@ -842,7 +842,7 @@ cli_resize(void *v, struct strbuf *reply, void *data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ;if (mpp->wait_for_udev) { condlog(2, "%s: device not fully created, failing resize",
@@ -926,7 +926,7 @@ cli_restore_queueing(void *v, struct strbuf *reply, void *data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ;if (mpp->disable_queueing) { mpp->disable_queueing =0 ;
@@ -976,7 +976,7 @@ cli_disable_queueing(void *v, struct strbuf *reply, void *data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ;if (count_active_paths (mpp) ==0 ) mpp->stat_map_failures++;
@@ -1108,7 +1108,10 @@ cli_reinstate(void * v, struct strbuf *reply, void * data) param =convert_dev (param,1 ); pp =find_path_by_str (vecs->pathvec, param,"reinstate path" ); -if (!pp || !pp->mpp || !pp->mpp->alias) +if (!pp) +return -ENODEV ; + +if (!pp->mpp || !pp->mpp->alias)return 1 ; condlog(2, "%s: reinstate path %s (operator)",
@@ -1153,7 +1156,10 @@ cli_fail(void * v, struct strbuf *reply, void * data) param =convert_dev (param,1 ); pp =find_path_by_str (vecs->pathvec, param,"fail path" ); -if (!pp || !pp->mpp || !pp->mpp->alias) +if (!pp) +return -ENODEV ; + +if (!pp->mpp || !pp->mpp->alias)return 1 ; condlog(2, "%s: fail path %s (operator)",
@@ -1250,7 +1256,7 @@ cli_getprstatus (void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ;append_strbuf_str (reply, prflag_str[mpp->prflag]);
@@ -1270,7 +1276,7 @@ cli_setprstatus(void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ;if (mpp->prflag !=PRFLAG_SET ) { mpp->prflag =PRFLAG_SET ;
@@ -1292,7 +1298,7 @@ cli_unsetprstatus(void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, param);if (!mpp) -return 1 ; +return -ENODEV ;if (mpp->prflag !=PRFLAG_UNSET ) { mpp->prflag =PRFLAG_UNSET ;
@@ -1315,7 +1321,7 @@ cli_getprkey(void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ; key =get_be64 (mpp->reservation_key);if (!key) {
@@ -1343,7 +1349,7 @@ cli_unsetprkey(void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ; conf =get_multipath_config ();pthread_cleanup_push (put_multipath_config, conf);
@@ -1370,7 +1376,7 @@ cli_setprkey(void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ;if (parse_prkey_flags (keyparam, &prkey, &flags) !=0 ) {condlog (0 ,"%s: invalid prkey : '%s'" , mapname, keyparam);
@@ -1394,7 +1400,10 @@ static int cli_set_marginal(void * v, struct strbuf *reply, void * data) param =convert_dev (param,1 ); pp =find_path_by_str (vecs->pathvec, param,"set marginal path" ); -if (!pp || !pp->mpp || !pp->mpp->alias) +if (!pp) +return -ENODEV ; + +if (!pp->mpp || !pp->mpp->alias)return 1 ; condlog(2, "%s: set marginal path %s (operator)",
@@ -1418,7 +1427,10 @@ static int cli_unset_marginal(void * v, struct strbuf *reply, void * data) param =convert_dev (param,1 ); pp =find_path_by_str (vecs->pathvec, param,"unset marginal path" ); -if (!pp || !pp->mpp || !pp->mpp->alias) +if (!pp) +return -ENODEV ; + +if (!pp->mpp || !pp->mpp->alias)return 1 ; condlog(2, "%s: unset marginal path %s (operator)",
@@ -1449,7 +1461,7 @@ static int cli_unset_all_marginal(void * v, struct strbuf *reply, void * data) mpp =find_mp_by_str (vecs->mpvec, mapname);if (!mpp) -return 1 ; +return -ENODEV ;if (mpp->wait_for_udev) { condlog(2, "%s: device not fully created, "
multipathd/uxlsnr.c +2 -0
@@ -414,6 +414,8 @@ void default_reply(struct client *c, int r)case -EBUSY :append_strbuf_str (&c->reply,"map or partition in use\n" );break ; +case -ENODEV : +append_strbuf_str (&c->reply,"device not found\n" ); } }
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 14/18 libdmmp: handle failures in _process_cmd
In reply to Benjamin Marzinskicurrently, most of the callers of _process_cmd() do not gracefully handle the case where multipathd returns "fail\n". dmmp_flush_mpath() does, but it does extra work to try to figure out after the fact why the flush command failed. Instead, handle fail replies in _process_cmd() using the appropriate DMMP error codes.
--- libdmmp/libdmmp.c | 69 +++++++++++++++++++---------------------------- 1 file changed, 28 insertions(+), 41 deletions(-)
libdmmp/libdmmp.c +28 -41
@@ -354,22 +354,36 @@ invoke:_debug (ctx,"IPC timeout, but user requested timeout has not " "reached yet, still have %u milliseconds" , ipc_tmo); goto invoke; - }else { -if ((*output ==NULL ) || (strlen (*output) ==0 )) { -_error (ctx,"IPC return empty reply for command %s" , - cmd); - rc =DMMP_ERR_IPC_ERROR ; - goto out; - } } - -if ((*output !=NULL ) && -strncmp (*output,"fail\npermission deny" , -strlen ("fail\npermission deny" )) ==0 ) { -_error (ctx,"Permission deny, need to be root" ); - rc =DMMP_ERR_PERMISSION_DENY ; +if ((*output ==NULL ) || (strlen (*output) ==0 )) { +_error (ctx,"IPC return empty reply for command %s" , + cmd); + rc =DMMP_ERR_IPC_ERROR ; goto out; } +if (strncmp (*output,"fail\n" ,5 ) ==0 ) { +if (strncmp (*output,"fail\npermission deny" , +strlen ("fail\npermission deny" )) ==0 ) { +_error (ctx,"Permission deny, need to be root" ); + rc =DMMP_ERR_PERMISSION_DENY ; + goto out; + } +else if (strncmp (*output,"fail\nmap or partition in use" , +strlen ("fail\nmap or partition in use" )) ==0 ) { +_error (ctx,"Specified mpath is in use" ); + rc =DMMP_ERR_MPATH_BUSY ; + goto out; + } +else if (strncmp (*output,"fail\ndevice not found" , +strlen ("fail\ndevice not found" )) ==0 ) { +_error (ctx,"Specified mpath not found" ); + rc =DMMP_ERR_MPATH_NOT_FOUND ; + goto out; + } +_error (ctx,"Got unexpected error for cmd '%s': '%s'" , + cmd, *output); + rc =DMMP_ERR_BUG ; + } out:if (rc !=DMMP_OK ) {
@@ -411,10 +425,6 @@ static int _ipc_connect(struct dmmp_context *ctx, int *fd)int dmmp_flush_mpath (struct dmmp_context *ctx,const char *mpath_name) {int rc =DMMP_OK ; -struct dmmp_mpath **dmmp_mps =NULL ; -uint32_t dmmp_mp_count =0 ; -uint32_t i =0 ; -bool found = false;int ipc_fd =-1 ;char cmd[_IPC_MAX_CMD_LEN];char *output =NULL ;
@@ -433,29 +443,7 @@ int dmmp_flush_mpath(struct dmmp_context *ctx, const char *mpath_name)_good (_process_cmd (ctx, ipc_fd, cmd, &output), rc, out);/* _process_cmd() already make sure output is not NULL */ - -if (strncmp (output,"fail" ,strlen ("fail" )) ==0 ) { -/* Check whether specified mpath exits */ -_good (dmmp_mpath_array_get (ctx, &dmmp_mps, &dmmp_mp_count), - rc, out); - -for (i =0 ; i < dmmp_mp_count; ++i) { -if (strcmp (dmmp_mpath_name_get (dmmp_mps[i]), - mpath_name) ==0 ) { - found = true; -break ; - } - } - -if (found == false) { - rc =DMMP_ERR_MPATH_NOT_FOUND ; -_error (ctx,"Specified mpath %s not found" , mpath_name); - goto out; - } - - rc =DMMP_ERR_MPATH_BUSY ; -_error (ctx,"Specified mpath is in use" ); - }else if (strncmp (output,"ok" ,strlen ("ok" )) !=0 ) { +if (strncmp (output,"ok" ,strlen ("ok" )) !=0 ) { rc =DMMP_ERR_BUG ;_error (ctx,"Got unexpected output for cmd '%s': '%s'" , cmd, output);
@@ -464,7 +452,6 @@ int dmmp_flush_mpath(struct dmmp_context *ctx, const char *mpath_name) out:if (ipc_fd >=0 )mpath_disconnect (ipc_fd); -dmmp_mpath_array_free (dmmp_mps, dmmp_mp_count);free (output);return rc; }
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 15/18 multipath: get rid of unnecessary retries variable
In reply to Benjamin Marzinskithe -R option can just set conf->remove_retries directly. It has the same effect.
--- multipath/main.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
multipath/main.c +3 -6
@@ -834,7 +834,6 @@ main (int argc, char *argv[])enum devtypes dev_type =DEV_NONE ;char *dev =NULL ;struct config *conf; -int retries =-1 ;bool enable_foreign = false;libmultipath_init ();
@@ -944,7 +943,7 @@ main (int argc, char *argv[]) cmd =CMD_ADD_WWID ;break ;case 'R' : - retries =atoi (optarg); + conf->remove_retries =atoi (optarg);break ;case 'e' : enable_foreign = true;
@@ -1069,20 +1068,18 @@ main (int argc, char *argv[])vector_free (curmp); goto out; } -if (retries <0 ) - retries = conf->remove_retries;if (cmd ==CMD_FLUSH_ONE ) {if (dm_is_mpath (dev) !=1 ) {condlog (0 ,"%s is not a multipath device" , dev); r =RTVL_FAIL ; goto out; } - r = (dm_suspend_and_flush_map (dev, retries) !=DM_FLUSH_OK ) ? + r = (dm_suspend_and_flush_map (dev, conf->remove_retries) !=DM_FLUSH_OK ) ?RTVL_FAIL :RTVL_OK ; goto out; }else if (cmd ==CMD_FLUSH_ALL ) { - r = (dm_flush_maps (retries) !=DM_FLUSH_OK ) ? + r = (dm_flush_maps (conf->remove_retries) !=DM_FLUSH_OK ) ?RTVL_FAIL :RTVL_OK ; goto out; }
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 16/18 multipath: Don't locally retry deletgated remove failures
In reply to Benjamin MarzinskiNow that multipathd is running the same code to remove devices as multipath, multipath doesn't need to automatically retry the remove failures. Instead it now repeatedly calls delegate_to_multipathd() to handle any retries.
--- multipath/main.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
multipath/main.c +25 -20
@@ -741,15 +741,16 @@ get_dev_type(char *dev) {* It is safer to use equivalent multipathd client commands instead. */ enum { -DELEGATE_OK =0 , -DELEGATE_ERROR =-1 , -NOT_DELEGATED =1 , +DELEGATE_OK , +DELEGATE_ERROR , +DELEGATE_RETRY , +NOT_DELEGATED , };int delegate_to_multipathd (enum mpath_cmds cmd, __attribute__((unused))const char *dev, __attribute__((unused))enum devtypes dev_type, -const struct config *conf) +struct config *conf) {int fd;char command[1024 ], *p, *reply =NULL ;
@@ -767,17 +768,17 @@ int delegate_to_multipathd(enum mpath_cmds cmd, }else if (cmd ==CMD_FLUSH_ONE && dev && dev_type ==DEV_DEVMAP ) { p +=snprintf (p, n,"del map %s" , dev); -/* multipathd doesn't try as hard, to avoid potentially -* hanging. If it fails, retry with the regular multipath -* command */ - r =NOT_DELEGATED ; +if (conf->remove_retries >0 ) { + r =DELEGATE_RETRY ; + conf->remove_retries--; + } }else if (cmd ==CMD_FLUSH_ALL ) { p +=snprintf (p, n,"del maps" ); -/* multipathd doesn't try as hard, to avoid potentially -* hanging. If it fails, retry with the regular multipath -* command */ - r =NOT_DELEGATED ; +if (conf->remove_retries >0 ) { + r =DELEGATE_RETRY ; + conf->remove_retries--; + } }/* Add other translations here */
@@ -806,7 +807,7 @@ int delegate_to_multipathd(enum mpath_cmds cmd,if (reply !=NULL && *reply !='\0' ) {if (strncmp (reply,"fail\n" ,5 )) r =DELEGATE_OK ; -if (r !=NOT_DELEGATED &&strcmp (reply,"ok\n" )) { +if (r !=DELEGATE_RETRY &&strcmp (reply,"ok\n" )) {/* If there is additional failure information, skip the * initial 'fail' */ if (strncmp(reply, "fail\n", 5) == 0 &&
@@ -1021,13 +1022,17 @@ main (int argc, char *argv[]) goto out; } -switch (delegate_to_multipathd (cmd, dev, dev_type, conf)) { -case DELEGATE_OK : -exit (RTVL_OK ); -case DELEGATE_ERROR : -exit (RTVL_FAIL ); -case NOT_DELEGATED : -break ; +while (1 ) { +int ret =delegate_to_multipathd (cmd, dev, dev_type, conf); + +if (ret ==DELEGATE_OK ) +exit (RTVL_OK ); +if (ret ==DELEGATE_ERROR ) +exit (RTVL_FAIL ); +if (ret ==DELEGATE_RETRY ) +sleep (1 ); +else /* NOT_DELEGATED */ +break ; }if (check_alias_settings (conf)) {
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 17/18 multipath: if delegation times out mark as not delegated
In reply to Benjamin MarzinskiIf either mpath_process_cmd fails with ETIMEDOUT or the reply is "fail\ntimeout\n", failback to multipath local commands.
--- multipath/main.c | 6 ++++++ 1 file changed, 6 insertions(+)
multipath/main.c +6 -0
@@ -799,6 +799,8 @@ int delegate_to_multipathd(enum mpath_cmds cmd,if (mpath_process_cmd (fd, command, &reply, conf->uxsock_timeout) ==-1 ) { +if (errno ==ETIMEDOUT ) + r =NOT_DELEGATED ;condlog (1 ,"error in multipath command %s: %s" , command,strerror (errno)); goto out;
@@ -807,6 +809,10 @@ int delegate_to_multipathd(enum mpath_cmds cmd,if (reply !=NULL && *reply !='\0' ) {if (strncmp (reply,"fail\n" ,5 )) r =DELEGATE_OK ; +else if (strcmp (reply,"fail\ntimeout\n" ) ==0 ) { + r =NOT_DELEGATED ; + goto out; + }if (r !=DELEGATE_RETRY &&strcmp (reply,"ok\n" )) {/* If there is additional failure information, skip the * initial 'fail' */
Signature · 1 line
2.43.0
Benjamin Marzinski
maintainerPATCH v2 18/18 multipathd: sync features on flush_map failure corner case
In reply to Benjamin MarzinskiIf dm_suspend_and_flush_map() disables queueing on a device, and then fails both to flush the device and restore queueing, the device will still have queue_if_no_path set in mpp->features, but not in reality. Fix this.
--- libmultipath/devmapper.c | 4 ++-- libmultipath/devmapper.h | 1 + libmultipath/libmultipath.version | 3 ++- multipathd/main.c | 2 ++ 4 files changed, 7 insertions(+), 3 deletions(-)
libmultipath/devmapper.c +2 -2
@@ -1136,8 +1136,8 @@ int _dm_flush_map (const char * mapname, int need_sync, int deferred_remove,sleep (1 ); }while (retries-- >0 ); -if (queue_if_no_path ==1 ) -_dm_queue_if_no_path (mapname,1 ); +if (queue_if_no_path ==1 &&_dm_queue_if_no_path (mapname,1 ) !=0 ) +return DM_FLUSH_FAIL_CANT_RESTORE ;return DM_FLUSH_FAIL ; }
libmultipath/devmapper.h +1 -0
@@ -52,6 +52,7 @@ int dm_is_mpath(const char *);enum {DM_FLUSH_OK =0 ,DM_FLUSH_FAIL , +DM_FLUSH_FAIL_CANT_RESTORE ,DM_FLUSH_DEFERRED ,DM_FLUSH_BUSY , };
libmultipath/libmultipath.version +2 -1
@@ -43,7 +43,7 @@ LIBMPATHCOMMON_1.0.0 { put_multipath_config; }; -LIBMULTIPATH_22.0.0 { +LIBMULTIPATH_23.0.0 { global: /* symbols referenced by multipath and multipathd */ add_foreign;
@@ -151,6 +151,7 @@ global: _print_multipath_topology; reinstate_paths; remember_wwid; + remove_feature; remove_map; remove_map_by_alias; remove_map_callback;
multipathd/main.c +2 -0
@@ -790,6 +790,8 @@ flush_map(struct multipath * mpp, struct vectors * vecs) {int r =dm_suspend_and_flush_map (mpp->alias,0 );if (r !=DM_FLUSH_OK ) { +if (DM_FLUSH_FAIL_CANT_RESTORE ) +remove_feature (&mpp->features,"queue_if_no_path" );condlog (0 ,"%s: can't flush" , mpp->alias);return r; }
Signature · 1 line
2.43.0
Quoted 20 lines · Benjamin Marzinski
The main purpose of this patchset is to make multipathd call
dm_suspend_and_flush_map()for the "del map" and "del maps" interactive
commands. I ran into an issue where if there was outstanding IO to a queueing multipath device,dm_flush_map()would hang removing the last
kpartx device, since that would close the last opener of the multipath
device, forcing it to wait for the IO to flush. This is avoided bydm_suspend_and_flush_map(), since it disables queueing first. The first
three patches do this. The next three are cleanups around removing a device. The rest of the patches are make multipath return better errors
when it delegates removes since it doesn't need to failback to doing its
own removes anymore, make libdmmp handle failures from running multipathd commands better, and make multipath and multipathc return proper error codes when multipathd commands fail.
Quoted 12 lines
Differences from v1 (based on suggestions by Martin Wilck) 0003: simplify
dm_flush_maps(), since all callers setneed_suspend0005: Minor change indm_flush_maps()to accomodate change in 0003. I retained the reviewed-by.
0007: Stripped the initial "fail\n" off messages with additional failure
information.
0009: Minor changes indm_flush_maps()and its callers to accomodate change in 0003. I retained the reviewed-by.
0010: changed the log level of the error message to 2. 0012: search for path bydev_tfirst, and change the log level of the error message to 2.
The commit message still says "make cli_handlers check for paths by dev and devt". I'll take the freedom to change this to the actual ordering.
Quoted 2 lines
0015: Minor change to calling
dm_flush_maps()to accomodate change in 0003. I retained the reviewed-by
You didn't, actually. I'll re-add it.
Quoted 41 lines
0016: multipath repeatedly calls
delegate_to_multipathdfor allremove_retries, instead of failing back to locally calling remove.
0017: New patch based on patch 0016 discussion.
0018: New patch to deal with a issue caused by the change to track the
queue_if_no_pathstate inmpp->features, from my previous patchset.Benjamin Marzinski (18):
multipathd: remove nopath flushing code fromflush_map()
multipathd: makeflush_map()delete maps like the multipath command
multipathd: disable queueing when removing unknown maps
multipathd: simplifycli_del_map()
libmultipath: make_dm_flush_map()return an enum
libmultipath: makedm_remove_partmaps()a static function
multipathd: always start failure replies with "fail\n"
multipathd: print extra default reply msg for busy devices
multipathd: handle busy devices incli_del_maps()
libmultipath: print error whenfind_mp_by_str()fails.
multipathd: don't open codefind_mp_by_str()in client handers
multipathd: makecli_handlerscheck for paths by dev and devt
multipathd: addcli_handlerreply message for missing devs
libdmmp: handle failures in_process_cmd
multipath: get rid of unnecessary retries variable
multipath: Don't locally retry deletgated remove failures
multipath: if delegation times out mark as not delegated
multipathd: sync features onflush_mapfailure corner caselibdmmp/libdmmp.c | 71 +++++------- libmultipath/devmapper.c | 47 ++++---- libmultipath/devmapper.h | 13 ++- libmultipath/libmultipath.version | 3 +- libmultipath/structs.c | 9 +- multipath/main.c | 72 +++++++----- multipathd/cli_handlers.c | 182 ++++++++++++++-------------- --
multipathd/main.c | 70 ++++-------- multipathd/main.h | 3 +- multipathd/uxclnt.c | 9 +- multipathd/uxlsnr.c | 15 ++-
Quoted 1 line
11 files changed, 241 insertions(+), 253 deletions(-)
For the set: