Skip to contentSkip to conversation

mp-tools: exception list handling

PATCH In review Volker Sameske 2 participants 2 messages Dec 7, 2006 – Jan 7, 2007 Device mapper

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mp-tools: exception list handling

mp-tools: exception list handling

---

On IBM System z we often have the problem that we have attached several hundred devices. In case we want to use only a few of them with mp-tools, we have to blacklist all others. For that reason I have introduced a blacklist_exception keyword for the multipath.conf which fills an internal exception list.

Additionally this patch adds two commands to the multipathd cli to display blacklist rules and blacklisted devices.

A multipath.conf like:
------------------- snip --------------------

blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z][[0-9]*]"
        devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
        devnode "^dasd[a-z]+[0-9]*"
}
blacklist_exceptions {
        devnode "^dasd[c-d]+[0-9]*"
}

devices {
device {
vendor "IBM"
product "S/390 DASD ECKD"
path_grouping_policy multibus
getuid_callout "dasdinfo -u -l %n"
path_checker directio
path_selector "round-robin 0"
hardware_handler "0"
}
}
------------------- snap --------------------

would activate S/390 DASD ECKD devices. In the blacklist section all DASD devices are blacklisted except dasdc and dasdd and corresponding partitions, which are listed in the blacklist:exceptions section.
This example would produce the following output in the "multipathd -k" cli:

------------------- snip --------------------
multipathd> show blacklist
device node rules:
- blacklist:

        (config file rule) ^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*
        (config file rule) ^hd[a-z][[0-9]*]
        (config file rule) ^cciss!c[0-9]d[0-9]*[p[0-9]*]
        (config file rule) ^dasd[a-z]+[0-9]*
        (default rule)     ^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*
        (default rule)     ^hd[a-z]
        (default rule)     ^cciss!c[0-9]d[0-9]*
- exceptions:
        (config file rule) ^dasd[c-d]+[0-9]*
wwid rules:
- blacklist:
        <empty>
- exceptions:
        <empty>

device rules:
- blacklist:
(default rule) DGC:LUNZ
multipathd>

multipathd> show devices
available block devices:
dasda (blacklisted)
dasdb (blacklisted)
dasdc
dasdd
dm-0 (blacklisted)
dm-1 (blacklisted)
dm-2 (blacklisted)
dm-3 (blacklisted)
dm-4 (blacklisted)
dm-5 (blacklisted)
dm-6 (blacklisted)
loop0 (blacklisted)
loop1 (blacklisted)
loop2 (blacklisted)
loop3 (blacklisted)
loop4 (blacklisted)
loop5 (blacklisted)
loop6 (blacklisted)
loop7 (blacklisted)
ram0 (blacklisted)
ram10 (blacklisted)
ram11 (blacklisted)
ram12 (blacklisted)
ram13 (blacklisted)
ram14 (blacklisted)
ram15 (blacklisted)
ram1 (blacklisted)
ram2 (blacklisted)
ram3 (blacklisted)
ram4 (blacklisted)
ram5 (blacklisted)
ram6 (blacklisted)
ram7 (blacklisted)
ram8 (blacklisted)
ram9 (blacklisted)
sda
sdb
sdc
sdd
multipathd>
------------------- snap --------------------

# diffstat mp-tools-exception-list-handling.patch
 libmultipath/blacklist.c  |   40 +++++++---
 libmultipath/blacklist.h  |    8 +-
 libmultipath/config.c     |   19 ++++
 libmultipath/config.h     |    5 +
 libmultipath/dict.c       |   49 +++++++++++-
 libmultipath/discovery.c  |    2
 libmultipath/print.c      |  182 ++++++++++++++++++++++++++++++++--------------
 libmultipath/print.h      |    1
 multipath.conf.annotated  |   16 ++++
 multipath.conf.synthetic  |    6 +
 multipath/main.c          |    4 -
 multipathd/cli.c          |    2
 multipathd/cli.h          |    4 +
 multipathd/cli_handlers.c |   80 +++++++++++++++++++-
 multipathd/cli_handlers.h |    2
 multipathd/main.c         |    4 -
 multipathd/multipathd.8   |    6 +
 17 files changed, 353 insertions(+), 77 deletions(-)
#
libmultipath/blacklist.c +30 -9
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/libmultipath/blacklist.c multipath-tools-mp-bleed-20061130-work/libmultipath/blacklist.c
@@ -14,7 +13,7 @@
 #include "blacklist.h"
 
 extern int
-store_ble (vector blist, char * str)
+store_ble (vector blist, char * str, int origin)
 {
 	struct blentry * ble;
 	
@@ -36,6 +35,7 @@ store_ble (vector blist, char * str)
 		goto out1;
 
 	ble->str = str;
+	ble->origin = origin;
 	vector_set_slot(blist, ble);
 	return 0;
 out1:
@@ -63,7 +63,7 @@ alloc_ble_device (vector blist)
 }
 	
 extern int
-set_ble_device (vector blist, char * vendor, char * product)
+set_ble_device (vector blist, char * vendor, char * product, int origin)
 {
 	struct blentry_device * ble;
 	
@@ -91,6 +91,7 @@ set_ble_device (vector blist, char * ven
 		}
 		ble->product = product;
 	}
+	ble->origin = origin;
 	return 0;
 }
 
@@ -105,19 +106,19 @@ setup_default_blist (struct config * con
 	str = STRDUP("^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*");
 	if (!str)
 		return 1;
-	if (store_ble(conf->blist_devnode, str))
+	if (store_ble(conf->blist_devnode, str, ORIGIN_DEFAULT))
 		return 1;
 
 	str = STRDUP("^hd[a-z]");
 	if (!str)
 		return 1;
-	if (store_ble(conf->blist_devnode, str))
+	if (store_ble(conf->blist_devnode, str, ORIGIN_DEFAULT))
 		return 1;
 	
 	str = STRDUP("^cciss!c[0-9]d[0-9]*");
 	if (!str)
 		return 1;
-	if (store_ble(conf->blist_devnode, str))
+	if (store_ble(conf->blist_devnode, str, ORIGIN_DEFAULT))
 		return 1;
 
 	vector_foreach_slot (conf->hwtable, hwe, i) {
@@ -128,7 +129,8 @@ setup_default_blist (struct config * con
 					  VECTOR_SIZE(conf->blist_device) -1);
 			if (set_ble_device(conf->blist_device,
 					   STRDUP(hwe->vendor),
-					   STRDUP(hwe->bl_product))) {
+					   STRDUP(hwe->bl_product),
+					   ORIGIN_DEFAULT)) {
 				FREE(ble);
 				return 1;
 			}
@@ -139,11 +141,29 @@ setup_default_blist (struct config * con
 }
 
 int
-blacklist (vector blist, char * str)
+blacklist_exceptions (vector elist, char * str)
+{
+        int i;
+        struct blentry * ele;
+
+        vector_foreach_slot (elist, ele, i) {
+                if (!regexec(&ele->regex, str, 0, NULL, 0)) {
+			condlog(3, "%s: exception-listed", str);
+			return 1;
+		}
+	}
+        return 0;
+}
+
+int
+blacklist (vector blist, vector elist, char * str)
 {
 	int i;
 	struct blentry * ble;
 
+	if (blacklist_exceptions(elist, str))
+		return 0;
+
 	vector_foreach_slot (blist, ble, i) {
 		if (!regexec(&ble->regex, str, 0, NULL, 0)) {
 			condlog(3, "%s: blacklisted", str);
@@ -172,10 +192,10 @@ blacklist_device (vector blist, char * v
 int
 blacklist_path (struct config * conf, struct path * pp)
 {
-	if (blacklist(conf->blist_devnode, pp->dev))
+	if (blacklist(conf->blist_devnode, conf->elist_devnode, pp->dev))
 		return 1;
 
-	if (blacklist(conf->blist_wwid, pp->wwid))
+	if (blacklist(conf->blist_wwid, conf->elist_wwid, pp->wwid))
 		return 1;
 
 	if (pp->vendor_id && pp->product_id &&
libmultipath/blacklist.h +5 -3
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/libmultipath/blacklist.h multipath-tools-mp-bleed-20061130-work/libmultipath/blacklist.h
@@ -6,6 +6,7 @@
 struct blentry {
 	char * str;
 	regex_t regex;
+	int origin;
 };
 
 struct blentry_device {
@@ -13,15 +14,16 @@ struct blentry_device {
 	char * product;
 	regex_t vendor_reg;
 	regex_t product_reg;
+	int origin;
 };
 
 int setup_default_blist (struct config *);
 int alloc_ble_device (vector);
-int blacklist (vector, char *);
+int blacklist (vector, vector, char *);
 int blacklist_device (vector, char *, char *);
 int blacklist_path (struct config *, struct path *);
-int store_ble (vector, char *);
-int set_ble_device (vector, char *, char *);
+int store_ble (vector, char *, int);
+int set_ble_device (vector, char *, char *, int);
 void free_blacklist (vector);
 void free_blacklist_device (vector);
 
libmultipath/config.c +19 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/libmultipath/config.c multipath-tools-mp-bleed-20061130-work/libmultipath/config.c
@@ -333,6 +333,12 @@ free_config (struct config * conf)
 	free_blacklist(conf->blist_devnode);
 	free_blacklist(conf->blist_wwid);
 	free_blacklist_device(conf->blist_device);
+
+	if (conf->elist_devnode)
+		FREE(conf->elist_devnode);
+	if (conf->elist_wwid)
+		FREE(conf->elist_wwid);
+
 	free_mptable(conf->mptable);
 	free_hwtable(conf->hwtable);
 	free_keywords(conf->keywords);
@@ -403,6 +409,19 @@ load_config (char * file)
 	if (setup_default_blist(conf))
 		goto out;
 
+	if (conf->elist_devnode == NULL) {
+                conf->elist_devnode = vector_alloc();
+
+                if (!conf->elist_devnode)
+			goto out;
+	}
+	if (conf->elist_wwid == NULL) {
+		conf->elist_wwid = vector_alloc();
+
+                if (!conf->elist_wwid)
+			goto out;
+	}
+
 	if (conf->mptable == NULL) {
 		conf->mptable = vector_alloc();
 
libmultipath/config.h +5 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/libmultipath/config.h multipath-tools-mp-bleed-20061130-work/libmultipath/config.h
@@ -1,6 +1,9 @@
 #ifndef _CONFIG_H
 #define _CONFIG_H
 
+#define ORIGIN_DEFAULT 0
+#define ORIGIN_CONFIG  1
+
 enum devtypes {
 	DEV_NONE,
 	DEV_DEVT,
@@ -75,6 +78,8 @@ struct config {
 	vector blist_devnode;
 	vector blist_wwid;
 	vector blist_device;
+	vector elist_devnode;
+	vector elist_wwid;
 };
 
 struct config * conf;
libmultipath/dict.c +45 -4

85 lines not on this page · raw message

libmultipath/discovery.c +1 -1
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/libmultipath/discovery.c multipath-tools-mp-bleed-20061130-work/libmultipath/discovery.c
@@ -61,7 +61,7 @@ path_discover (vector pathvec, struct co
 	if (!devname)
 		return 0;
 
-	if (blacklist(conf->blist_devnode, devname))
+	if (blacklist(conf->blist_devnode, conf->elist_devnode, devname))
 		return 0;
 
 	if(safe_sprintf(path, "%s/block/%s/device", sysfs_path,
libmultipath/print.c +129 -53

215 lines not on this page · raw message

libmultipath/print.h +1 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/libmultipath/print.h multipath-tools-mp-bleed-20061130-work/libmultipath/print.h
@@ -42,6 +42,7 @@ int snprint_multipath_topology (char *, 
 				int verbosity);
 int snprint_defaults (char *, int);
 int snprint_blacklist (char *, int);
+int snprint_devices (char *, int, struct vectors *);
 int snprint_hwtable (char *, int, vector);
 int snprint_mptable (char *, int, vector);
 
multipath/main.c +2 -2
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipath/main.c multipath-tools-mp-bleed-20061130-work/multipath/main.c
@@ -233,7 +233,7 @@ configure (void)
 			dev = conf->dev;
 	}
 	
-	if (dev && blacklist(conf->blist_devnode, dev))
+	if (dev && blacklist(conf->blist_devnode, conf->elist_devnode, dev))
 		goto out;
 	
 	/*
@@ -249,7 +249,7 @@ configure (void)
 		}
 		condlog(3, "scope limited to %s", refwwid);
 
-		if (blacklist(conf->blist_wwid, refwwid))
+		if (blacklist(conf->blist_wwid, conf->elist_wwid, refwwid))
 			goto out;
 	}
 
multipath.conf.annotated +16 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipath.conf.annotated multipath-tools-mp-bleed-20061130-work/multipath.conf.annotated
@@ -142,6 +142,22 @@
 #               product MSA[15]00
 #       }
 #}
+##
+## name    : blacklist_exceptions
+## scope   : multipath & multipathd
+## desc    : list of device names to be treated as multipath candidates
+##           even if they are on the blacklist.
+##           Note: blacklist exceptions are only valid in the same class.
+##           It is not possible to blacklist devices using the devnode keyword
+##           and to exclude some devices of them using the wwid keyword.
+## default : -
+##
+#blacklist_exceptions {
+#       devnode "^dasd[c-d]+[0-9]*"
+#       wwid    "IBM.75000000092461.4d00.34"
+#       wwid    "IBM.75000000092461.4d00.35"
+#       wwid    "IBM.75000000092461.4d00.36"
+#}
 #
 ##
 ## name    : multipaths
multipath.conf.synthetic +5 -1
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipath.conf.synthetic multipath-tools-mp-bleed-20061130-work/multipath.conf.synthetic
@@ -16,7 +16,7 @@
 #	no_path_retry		fail
 #	user_friendly_names	no
 #}
-#devnode_blacklist {
+#blacklist {
 #       wwid 26353900f02796769
 #	devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
 #	devnode "^hd[a-z][[0-9]*]"
@@ -26,6 +26,10 @@
 #		product MSA[15]00
 #	}
 #}
+#blacklist_exceptions {
+#       devnode "^dasd[c-d]+[0-9]*"
+#       wwid    "IBM.75000000092461.4d00.34"
+#}
 #multipaths {
 #	multipath {
 #		wwid			3600508b4000156d700012000000b0000
multipathd/cli.c +2 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipathd/cli.c multipath-tools-mp-bleed-20061130-work/multipathd/cli.c
@@ -141,6 +141,8 @@ load_keys (void)
 	r += add_key(keys, "stats", STATS, 0);
 	r += add_key(keys, "topology", TOPOLOGY, 0);
 	r += add_key(keys, "config", CONFIG, 0);
+	r += add_key(keys, "blacklist", BLACKLIST, 0);
+	r += add_key(keys, "devices", DEVICES, 0);
 
 	if (r) {
 		free_keys(keys);
multipathd/cli.h +4 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipathd/cli.h multipath-tools-mp-bleed-20061130-work/multipathd/cli.h
@@ -17,6 +17,8 @@ enum {
 	__STATS,
 	__TOPOLOGY,
 	__CONFIG,
+	__BLACKLIST,
+	__DEVICES,
 };
 
 #define LIST		(1 << __LIST)
@@ -37,6 +39,8 @@ enum {
 #define STATS		(1 << __STATS)
 #define TOPOLOGY	(1 << __TOPOLOGY)
 #define CONFIG		(1 << __CONFIG)
+#define BLACKLIST	(1 << __BLACKLIST)
+#define DEVICES  	(1 << __DEVICES)
 
 #define INITIAL_REPLY_LEN 1000
 
multipathd/cli_handlers.c +78 -2

95 lines not on this page · raw message

multipathd/cli_handlers.h +2 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipathd/cli_handlers.h multipath-tools-mp-bleed-20061130-work/multipathd/cli_handlers.h
@@ -5,6 +5,8 @@ int cli_list_maps_stats (void * v, char 
 int cli_list_map_topology (void * v, char ** reply, int * len, void * data);
 int cli_list_maps_topology (void * v, char ** reply, int * len, void * data);
 int cli_list_config (void * v, char ** reply, int * len, void * data);
+int cli_list_blacklist (void * v, char ** reply, int * len, void * data);
+int cli_list_devices (void * v, char ** reply, int * len, void * data);
 int cli_add_path (void * v, char ** reply, int * len, void * data);
 int cli_del_path (void * v, char ** reply, int * len, void * data);
 int cli_add_map (void * v, char ** reply, int * len, void * data);
multipathd/main.c +3 -1
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipathd/main.c multipath-tools-mp-bleed-20061130-work/multipathd/main.c
@@ -667,7 +667,7 @@ uev_trigger (struct uevent * uev, void *
 	/*
 	 * path add/remove event
 	 */
-	if (blacklist(conf->blist_devnode, devname))
+	if (blacklist(conf->blist_devnode, conf->elist_devnode, devname))
 		goto out;
 
 	if (!strncmp(uev->action, "add", 3)) {
@@ -710,6 +710,8 @@ uxlsnrloop (void * ap)
 	add_handler(LIST+TOPOLOGY, cli_list_maps_topology);
 	add_handler(LIST+MAP+TOPOLOGY, cli_list_map_topology);
 	add_handler(LIST+CONFIG, cli_list_config);
+	add_handler(LIST+BLACKLIST, cli_list_blacklist);
+	add_handler(LIST+DEVICES, cli_list_devices);
 	add_handler(ADD+PATH, cli_add_path);
 	add_handler(DEL+PATH, cli_del_path);
 	add_handler(ADD+MAP, cli_add_map);
multipathd/multipathd.8 +6 -0
diff -Nurp --exclude='*~' multipath-tools-mp-bleed-20061130/multipathd/multipathd.8 multipath-tools-mp-bleed-20061130-work/multipathd/multipathd.8
@@ -57,6 +57,12 @@ This map could be obtained from "list ma
 .B list|show config
 Show the currently used configuration, derived from default values and values specified within the configuration file /etc/multipath.conf.
 .TP
+.B list|show blacklist
+Show the currently used blacklist rules, derived from default values and values specified within the configuration file /etc/multipath.conf.
+.TP
+.B list|show devices
+Show all available block devices by name including the information if they are blacklisted or not.
+.TP
 .B add path $path
 Add a path to the list of monitored paths. $path is as listed in /sys/block (e.g. sda).
 .TP 
#1 lore raw
Quoted 19 lines · Volker Sameske

On IBM System z we often have the problem that we have attached several hundred devices. In case we want to use only a few of them with mp-tools, we have to blacklist all others. For that reason I have introduced a blacklist_exception keyword for the multipath.conf which fills an internal exception list.

Additionally this patch adds two commands to the multipathd cli to display blacklist rules and blacklisted devices.

------------------- snip --------------------

blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z][[0-9]*]"
        devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
        devnode "^dasd[a-z]+[0-9]*"
}
blacklist_exceptions {
        devnode "^dasd[c-d]+[0-9]*"
}

...

Quoted 29 lines

multipathd> show blacklist
device node rules:
- blacklist:

        (config file rule) ^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*
        (config file rule) ^hd[a-z][[0-9]*]
        (config file rule) ^cciss!c[0-9]d[0-9]*[p[0-9]*]
        (config file rule) ^dasd[a-z]+[0-9]*
        (default rule)     ^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*
        (default rule)     ^hd[a-z]
        (default rule)     ^cciss!c[0-9]d[0-9]*
- exceptions:
        (config file rule) ^dasd[c-d]+[0-9]*
wwid rules:
- blacklist:
        <empty>
- exceptions:
        <empty>

device rules:
- blacklist:
(default rule) DGC:LUNZ
multipathd>

multipathd> show devices
available block devices:
dasda (blacklisted)
dasdb (blacklisted)
dasdc
dasdd
dm-0 (blacklisted)

...

Quoted 3 lines

------------------- snap --------------------

Hello,

I merged the latest version of your patch.

Minor mangling : I had to rename your snprint_blacklist() rewrite to snprint_blacklist_report() because the "show config" CLI command purpose is to dump a config file in sync with the daemon actual configuration. Your stealing the snprint_blacklist() fn name made for your blacklist "report" to be included in this dump in place of the correct "blacklist {}" block.

I also implemented the missing snprint_blacklist_exceptions() for config dump completeness.

Please verify everything is still working for you.

Thanks for this nicely crafted work.
Regards,
cvaroqui