Skip to contentSkip to conversation

Next try: systemd watchdog integration

PATCH In review Hannes Reinecke 2 participants 5 messages Dec 13, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Next try: systemd watchdog integration

Hi Christophe,

here is now a clean version of the systemd watchdog integration. It's now completely optional, and will only be enabled if the user / admin manually modifies the service file.
So that should cover all objections Ben had.

Hannes Reinecke (3):
multipathd: enable core dumps for systemd Use correct systemd unit directory
multipathd: Implement systemd watchdog integration

 Makefile.inc                  |  2 +-
 libmultipath/config.h         |  3 +++
 multipath/multipath.conf.5    |  7 +++++--
 multipathd/main.c             | 23 +++++++++++++++++++++-
 multipathd/multipathd.8       | 45 ++++++++++++++++++++++++++++++++++++++++++-
 multipathd/multipathd.service |  1 +
 6 files changed, 76 insertions(+), 5 deletions(-)
Signature · 1 line
1.8.1.4
#1 lore raw

PATCH 1/3 multipathd: enable core dumps for systemd

Add 'LimitCORE' definition to the service file to enable core dumps when running under systemd.

---
 multipathd/multipathd.service | 1 +
 1 file changed, 1 insertion(+)
multipathd/multipathd.service +1 -0
index 3874bcb..be3ba3f 100644
@@ -8,6 +8,7 @@ Conflicts=shutdown.target
 [Service]
 Type=notify
 NotifyAccess=main
+LimitCORE=infinity
 ExecStartPre=/sbin/modprobe dm-multipath
 ExecStart=/sbin/multipathd -d -s
 ExecReload=/sbin/multipathd reconfigure
Signature · 1 line
1.8.1.4
#2 lore raw

PATCH 2/3 Use correct systemd unit directory

In reply to Hannes Reinecke

The systemd unit directory has been moved to /usr/lib/systemd.

---
 Makefile.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile.inc +1 -1
index 7186484..f445160 100644
@@ -38,7 +38,7 @@ man3dir      = $(prefix)/usr/share/man/man3
 rcdir	    = $(prefix)/etc/init.d
 syslibdir   = $(prefix)/$(LIB)
 libdir	    = $(prefix)/$(LIB)/multipath
-unitdir     = $(prefix)/lib/systemd/system
+unitdir     = $(prefix)/usr/lib/systemd/system
 mpathpersistdir = $(TOPDIR)/libmpathpersist
 
 GZIP        = gzip -9 -c
Signature · 1 line
1.8.1.4
#3 lore raw

PATCH 3/3 multipathd: Implement systemd watchdog integration

In reply to Hannes Reinecke

In the past there have been several instances where multipathd would hang with the checkerloop as some path checker might not be able to return in time.
This patch now activates the watchdog feature from systemd to shutdown (and possibly restart) multipathd in these situations.
Due to a bug in systemd watchdog integration only works correctly with later version (> 206), so watchdog integration has been disabled per default.

---
 libmultipath/config.h      |  3 +++
 multipath/multipath.conf.5 |  7 +++++--
 multipathd/main.c          | 23 ++++++++++++++++++++++-
 multipathd/multipathd.8    | 45 ++++++++++++++++++++++++++++++++++++++++++++-
 4 files changed, 74 insertions(+), 4 deletions(-)
libmultipath/config.h +3 -0
index 9c467e8..445525b 100644
@@ -98,6 +98,9 @@ struct config {
 	int queue_without_daemon;
 	int checker_timeout;
 	int daemon;
+#ifdef USE_SYSTEMD
+	int watchdog;
+#endif
 	int flush_on_last_del;
 	int attribute_flags;
 	int fast_io_fail;
multipath/multipath.conf.5 +5 -2
index 0fd3035..cf5bec0 100644
@@ -71,8 +71,11 @@ section recognizes the following keywords:
 .B polling_interval
 interval between two path checks in seconds. For properly functioning paths,
 the interval between checks will gradually increase to
-.B max_polling_interval;
-default is
+.B max_polling_interval.
+This value will be overridden by the
+.B WatchdogSec
+setting in the multipathd.service definition if systemd is used.
+Default is
 .I 5
 .TP
 .B max_polling_interval
multipathd/main.c +22 -1
index 49e74a6..3219511 100644
@@ -1294,7 +1294,10 @@ checkerloop (void *ap)
 		lock(vecs->lock);
 		pthread_testcancel();
 		condlog(4, "tick");
-
+#ifdef USE_SYSTEMD
+		if (conf->watchdog)
+			sd_notify(0, "WATCHDOG=1");
+#endif
 		if (vecs->pathvec) {
 			vector_foreach_slot (vecs->pathvec, pp, i) {
 				num_paths += check_path(vecs, pp);
@@ -1616,6 +1619,9 @@ child (void * param)
 	struct vectors * vecs;
 	struct multipath * mpp;
 	int i;
+#ifdef USE_SYSTEMD
+	unsigned long checkint;
+#endif
 	int rc, pid_rc;
 	char *envp;
 
@@ -1696,6 +1702,21 @@ child (void * param)
 
 	conf->daemon = 1;
 	udev_set_sync_support(0);
+#ifdef USE_SYSTEMD
+	envp = getenv("WATCHDOG_USEC");
+	if (envp && sscanf(envp, "%lu", &checkint) == 1) {
+		/* Value is in microseconds */
+		conf->max_checkint = checkint / 1000000;
+		/* Rescale checkint */
+		if (conf->checkint > conf->max_checkint)
+			conf->checkint = conf->max_checkint;
+		else
+			conf->checkint = conf->max_checkint / 4;
+		condlog(3, "enabling watchdog, interval %d max %d",
+			conf->checkint, conf->max_checkint);
+		conf->watchdog = conf->checkint;
+	}
+#endif
 	/*
 	 * Start uevent listener early to catch events
 	 */
multipathd/multipathd.8 +44 -1

54 lines not on this page · raw message

Signature · 1 line
1.8.1.4
#4 lore raw
In reply to Hannes Reinecke
Quoted 6 lines · Hannes Reinecke

Hi Christophe,

here is now a clean version of the systemd watchdog integration. It's now completely optional, and will only be enabled if the user / admin manually modifies the service file.
So that should cover all objections Ben had.

Great

Thanks!

-Ben

Quoted 16 lines

Hannes Reinecke (3):
multipathd: enable core dumps for systemd Use correct systemd unit directory
multipathd: Implement systemd watchdog integration

 Makefile.inc                  |  2 +-
 libmultipath/config.h         |  3 +++
 multipath/multipath.conf.5    |  7 +++++--
 multipathd/main.c             | 23 +++++++++++++++++++++-
 multipathd/multipathd.8       | 45 ++++++++++++++++++++++++++++++++++++++++++-
 multipathd/multipathd.service |  1 +
 6 files changed, 76 insertions(+), 5 deletions(-)
Signature · 1 line
1.8.1.4