Skip to contentSkip to conversation

Clvmd -E fixes

PATCH New Zdenek Kabelac 2 participants 6 messages Sep 22, 2011

lore
Share & tools
lore
#0 lore raw

cover Clvmd -E fixes

Patchset fixed restart of clvmd with -E option.

1st. Adds support for easier restart of clvmd
(i.e. -Isinglenode is preserved).

2nd. Bug fixes support of -E arg.

3rd. Allows to build test suite script for checking clvmd restart.
(Any better idea here ?)

Zdenek Kabelac (3):
Add named cluster_ops
CLVMD bugfix support arg -E
CLVMD support for CLVMD_BINARY and LVM_BINARY

 configure.in                     |    9 ++++--
 daemons/clvmd/clvmd-cman.c       |    1 +
 daemons/clvmd/clvmd-command.c    |   15 ++++-----
 daemons/clvmd/clvmd-comms.h      |    3 +-
 daemons/clvmd/clvmd-corosync.c   |    1 +
 daemons/clvmd/clvmd-openais.c    |    1 +
 daemons/clvmd/clvmd-singlenode.c |    1 +
 daemons/clvmd/clvmd.c            |   19 ++++++++++--
 daemons/clvmd/lvm-functions.c    |   61 ++++++++++++++-----------------------
 daemons/clvmd/lvm-functions.h    |    2 +-
 man/Makefile.in                  |    2 +-
 man/clvmd.8.in                   |   14 +++++++++
 12 files changed, 74 insertions(+), 55 deletions(-)
Signature · 1 line
1.7.6.2
#1 lore raw

PATCH 1/3 Add named cluster_ops

To easily learn the name of the active cluster ops.

---
 daemons/clvmd/clvmd-cman.c       |    1 +
 daemons/clvmd/clvmd-command.c    |   10 ++++------
 daemons/clvmd/clvmd-comms.h      |    3 ++-
 daemons/clvmd/clvmd-corosync.c   |    1 +
 daemons/clvmd/clvmd-openais.c    |    1 +
 daemons/clvmd/clvmd-singlenode.c |    1 +
 6 files changed, 10 insertions(+), 7 deletions(-)
daemons/clvmd/clvmd-cman.c +1 -0
index c7a2b2d..7e76dc4 100644
@@ -479,6 +479,7 @@ static int _get_cluster_name(char *buf, int buflen)
 }
 
 static struct cluster_ops _cluster_cman_ops = {
+	.name                     = "cman",
 	.cluster_init_completed   = _cluster_init_completed,
 	.cluster_send_message     = _cluster_send_message,
 	.name_from_csid           = _name_from_csid,
daemons/clvmd/clvmd-command.c +4 -6
index 4ba7151..bd6e55e 100644
@@ -1,6 +1,6 @@
 /*
  * Copyright (C) 2002-2004 Sistina Software, Inc. All rights reserved.
- * Copyright (C) 2004 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2004-2011 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -380,7 +380,7 @@ static int restart_clvmd(void)
 	} while (hn);
 
 	/* clvmd + locks (-E uuid) + debug (-d X) + NULL */
-	if (!(argv = malloc((max_locks * 2 + 4) * sizeof(*argv))))
+	if (!(argv = malloc((max_locks * 2 + 5) * sizeof(*argv))))
 		goto_out;
 
 	/*
@@ -395,10 +395,8 @@ static int restart_clvmd(void)
 		argv[argc++] = debug_arg;
 	}
 
-	/*
-	 * FIXME: specify used cluster backend
-	 * argv[argc++] = strdup("-Isinglenode");
-	 */
+	argv[argc++] = "-I";
+	argv[argc++] = clops->name;
 
 	/* Now add the exclusively-open LVs */
 	hn = NULL;
daemons/clvmd/clvmd-comms.h +2 -1
index 500bd57..7207334 100644
@@ -1,6 +1,6 @@
 /*
  * Copyright (C) 2002-2004 Sistina Software, Inc. All rights reserved.
- * Copyright (C) 2004-2009 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2004-2011 Red Hat, Inc. All rights reserved.
  *
  * This file is part of LVM2.
  *
@@ -23,6 +23,7 @@
 struct local_client;
 
 struct cluster_ops {
+	const char *name;
 	void (*cluster_init_completed) (void);
 
 	int (*cluster_send_message) (const void *buf, int msglen,
daemons/clvmd/clvmd-corosync.c +1 -0
index 01ee09f..e929681 100644
@@ -598,6 +598,7 @@ out:
 }
 
 static struct cluster_ops _cluster_corosync_ops = {
+	.name                     = "corosync",
 	.cluster_init_completed   = NULL,
 	.cluster_send_message     = _cluster_send_message,
 	.name_from_csid           = _name_from_csid,
daemons/clvmd/clvmd-openais.c +1 -0
index 0cfb637..a2fe69e 100644
@@ -667,6 +667,7 @@ static int _get_cluster_name(char *buf, int buflen)
 }
 
 static struct cluster_ops _cluster_openais_ops = {
+	.name                     = "openais",
 	.cluster_init_completed   = NULL,
 	.cluster_send_message     = _cluster_send_message,
 	.name_from_csid           = _name_from_csid,
daemons/clvmd/clvmd-singlenode.c +1 -0
index aba4fa6..be04880 100644
@@ -267,6 +267,7 @@ static int _get_cluster_name(char *buf, int buflen)
 }
 
 static struct cluster_ops _cluster_singlenode_ops = {
+	.name                     = "singlenode",
 	.cluster_init_completed   = NULL,
 	.cluster_send_message     = _cluster_send_message,
 	.name_from_csid           = _name_from_csid,
Signature · 1 line
1.7.6.2
#2 lore raw

PATCH 2/3 CLVMD bugfix support arg -E

In reply to Zdenek Kabelac

Bugfix:
Add (most probably unfinished) support for -E arg with list of exclusive locks. (During clvmd restart all exclusive locks would have been lost and in fact, if there would have been an exclusive lock, usage text would be printed and clvmd exits.)

Instead of parsing list options multiple times every time some lock UUID is checked - put them straight into the hash table - make the code easier to understand as well.

Remove was_ex_lock() function (replaced with dm_hash_lookup()).

Swap return value for get_initial_state() (1 means success).

Update man pages and usage info for -E option.

---
 daemons/clvmd/clvmd.c         |   19 +++++++++++++++--
 daemons/clvmd/lvm-functions.c |   42 ++++++++--------------------------------
 daemons/clvmd/lvm-functions.h |    2 +-
 man/clvmd.8.in                |    4 +++
 4 files changed, 30 insertions(+), 37 deletions(-)
daemons/clvmd/clvmd.c +16 -3
index 2c9f8db..b0383dc 100644
@@ -78,7 +78,7 @@ struct lvm_thread_cmd {
 };
 
 struct lvm_startup_params {
-	char **argv;
+	struct dm_hash_table *excl_uuid;
 };
 
 static debug_t debug = DEBUG_OFF;
@@ -149,6 +149,7 @@ static void usage(const char *prog, FILE *file)
 		"   -h       Show this help information\n"
 		"   -d[n]    Set debug logging (0:none, 1:stderr (implies -f option), 2:syslog)\n"
 		"   -f       Don't fork, run in the foreground\n"
+		"   -E<lockuuid> Take this lock uuid as exclusively locked resource (for restart)\n"
 		"   -R       Tell all running clvmds in the cluster to reload their device cache\n"
 		"   -S       Restart clvmd, preserving exclusive locks\n"
 		"   -C       Sets debug level (from -d) on all clvmd instances clusterwide\n"
@@ -349,6 +350,11 @@ int main(int argc, char *argv[])
 		{ NULL, 0, 0, 0 }
 	};
 
+	if (!(lvm_params.excl_uuid = dm_hash_create(128))) {
+		fprintf(stderr, "Failed to allocate hash table\n");
+		return 1;
+	}
+
 	/* Deal with command-line arguments */
 	opterr = 0;
 	optind = 0;
@@ -392,6 +398,12 @@ int main(int argc, char *argv[])
 		case 'I':
 			cluster_iface = parse_cluster_interface(optarg);
 			break;
+		case 'E':
+			if (!dm_hash_insert(lvm_params.excl_uuid, optarg, optarg)) {
+				fprintf(stderr, "Failed to allocate hash entry\n");
+				return 1;
+			}
+			break;
 		case 'T':
 			start_timeout = atoi(optarg);
 			if (start_timeout <= 0) {
@@ -566,7 +578,6 @@ int main(int argc, char *argv[])
 
 	/* Don't let anyone else to do work until we are started */
 	pthread_mutex_lock(&lvm_start_mutex);
-	lvm_params.argv = argv;
 	pthread_create(&lvm_thread, NULL, lvm_thread_fn, &lvm_params);
 
 	/* Tell the rest of the cluster our version number */
@@ -604,6 +615,8 @@ int main(int argc, char *argv[])
 		free(delfd);
 	}
 
+	dm_hash_destroy(lvm_params.excl_uuid);
+
 	return 0;
 }
 
@@ -1981,7 +1994,7 @@ static void *lvm_thread_fn(void *arg)
 	pthread_sigmask(SIG_BLOCK, &ss, NULL);
 
 	/* Initialise the interface to liblvm */
-	init_clvm(lvm_params->argv);
+	init_clvm(lvm_params->excl_uuid);
 
 	/* Allow others to get moving */
 	pthread_mutex_unlock(&lvm_start_mutex);
daemons/clvmd/lvm-functions.c +9 -33

89 lines not on this page · raw message

daemons/clvmd/lvm-functions.h +1 -1
index f9c43b7..565f878 100644
@@ -27,7 +27,7 @@ extern int post_lock_lv(unsigned char lock_cmd, unsigned char lock_flags,
 			char *resource);
 extern int do_check_lvm1(const char *vgname);
 extern int do_refresh_cache(void);
-extern int init_clvm(char **argv);
+extern int init_clvm(struct dm_hash_table *excl_uuid);
 extern void destroy_lvm(void);
 extern void init_lvhash(void);
 extern void destroy_lvhash(void);
man/clvmd.8.in +4 -0
index a4b9cc9..22d94b2 100644
@@ -4,6 +4,7 @@ clvmd \- cluster LVM daemon
 .SH SYNOPSIS
 .B clvmd
 [\-d[<value>]] [\-C] [\-h]
+[\-E <lock uuid>]
 [\-R]
 [\-S]
 [\-t <timeout>]
@@ -69,6 +70,9 @@ sensible.
 .br
 This timeout will be ignored if you start clvmd with the -d switch.
 .TP
+.I \-E <lock uuid>
+Pass lock uuid to be reacquired exclusively when clvmd is restarted.
+.TP
 .I \-R
 Tells all the running clvmds in the cluster to reload their device cache and
 re-read the lvm configuration file. This command should be run whenever the
Signature · 1 line
1.7.6.2
#3 lore raw

PATCH 3/3 CLVMD support for CLVMD_BINARY and LVM_BINARY

In reply to Zdenek Kabelac

Read 2 environmental vars to learn about overide position for CLVMD and LVM binaries.

We support LVM_BINARY in other script - and this way we could easily test restart in our test-suite.

---
 configure.in                  |    9 ++++++---
 daemons/clvmd/clvmd-command.c |    5 +++--
 daemons/clvmd/lvm-functions.c |   19 ++++++++++++++-----
 man/Makefile.in               |    2 +-
 man/clvmd.8.in                |   10 ++++++++++
 5 files changed, 34 insertions(+), 11 deletions(-)
configure.in +6 -3
index 30dd765..244bfc8 100644
@@ -1180,13 +1180,14 @@ fi
 lvm_exec_prefix=$exec_prefix
 test "$lvm_exec_prefix" = NONE && lvm_exec_prefix=$prefix
 test "$lvm_exec_prefix" = NONE && lvm_exec_prefix=$ac_default_prefix
-AC_DEFINE_UNQUOTED(LVM_PATH, ["$lvm_exec_prefix/sbin/lvm"], [Path to lvm binary.])
+LVM_PATH="$lvm_exec_prefix/sbin/lvm"
+AC_DEFINE_UNQUOTED(LVM_PATH, ["$LVM_PATH"], [Path to lvm binary.])
 
 if  test "$CLVMD" != none; then
         clvmd_prefix=$ac_default_prefix
+        CLVMD_PATH="$clvmd_prefix/sbin/clvmd"
         test "$prefix" != NONE && clvmd_prefix=$prefix
-	AC_DEFINE_UNQUOTED(CLVMD_PATH, ["$clvmd_prefix/sbin/clvmd"],
-			   [Path to clvmd binary.])
+	AC_DEFINE_UNQUOTED(CLVMD_PATH, ["$CLVMD_PATH"], [Path to clvmd binary.])
 fi
 
 ################################################################################
@@ -1309,6 +1310,7 @@ AC_SUBST(CLDWHOLEARCHIVE)
 AC_SUBST(CLUSTER)
 AC_SUBST(CLVMD)
 AC_SUBST(CLVMD_CMANAGERS)
+AC_SUBST(CLVMD_PATH)
 AC_SUBST(CMAN_CFLAGS)
 AC_SUBST(CMAN_LIBS)
 AC_SUBST(CMDLIB)
@@ -1357,6 +1359,7 @@ AC_SUBST(LVM_LIBAPI)
 AC_SUBST(LVM_MAJOR)
 AC_SUBST(LVM_MINOR)
 AC_SUBST(LVM_PATCHLEVEL)
+AC_SUBST(LVM_PATH)
 AC_SUBST(LVM_RELEASE)
 AC_SUBST(LVM_RELEASE_DATE)
 AC_SUBST(MIRRORS)
daemons/clvmd/clvmd-command.c +3 -2
index bd6e55e..e1ded83 100644
@@ -366,6 +366,7 @@ static int restart_clvmd(void)
 	int argc = 0, max_locks = 0;
 	struct dm_hash_node *hn = NULL;
 	char debug_arg[16];
+	const char *clvmd_binary = getenv("CLVMD_BINARY") ? : CLVMD_PATH;
 
 	DEBUGLOG("clvmd restart requested\n");
 
@@ -413,11 +414,11 @@ static int restart_clvmd(void)
 	argv[argc] = NULL;
 
 	/* Exec new clvmd */
-	DEBUGLOG("--- Restarting %s ---\n", CLVMD_PATH);
+	DEBUGLOG("--- Restarting %s ---\n", clvmd_binary);
 	for (argc = 1; argv[argc]; argc++) DEBUGLOG("--- %d: %s\n", argc, argv[argc]);
 
 	/* NOTE: This will fail when downgrading! */
-	execve(CLVMD_PATH, (char **)argv, NULL);
+	execve(clvmd_binary, (char **)argv, NULL);
 out:
 	/* We failed */
 	DEBUGLOG("Restart of clvmd failed.\n");
daemons/clvmd/lvm-functions.c +14 -5
index ba786bb..f16d006 100644
@@ -704,14 +704,21 @@ static int get_initial_state(struct dm_hash_table *excl_uuid)
 	char lv[64], vg[64], flags[25], vg_flags[25];
 	char uuid[65];
 	char line[255];
-	FILE *lvs =
-	    popen
-	    (LVM_PATH " lvs  --config 'log{command_names=0 prefix=\"\"}' --nolocking --noheadings -o vg_uuid,lv_uuid,lv_attr,vg_attr",
-	     "r");
+	char *lvs_cmd;
+	const char *lvm_binary = getenv("LVM_BINARY") ? : LVM_PATH;
+	FILE *lvs;
 
-	if (!lvs)
+	if (dm_asprintf(&lvs_cmd, "%s lvs  --config 'log{command_names=0 prefix=\"\"}' "
+			"--nolocking --noheadings -o vg_uuid,lv_uuid,lv_attr,vg_attr",
+			lvm_binary) < 0)
 		return 0;
 
+	/* FIXME: Maybe link and use liblvm2cmd directly instead of fork */
+	if (!(lvs = popen(lvs_cmd, "r"))) {
+		dm_free(lvs_cmd);
+		return 0;
+	}
+
 	while (fgets(line, sizeof(line), lvs)) {
 	        if (sscanf(line, "%s %s %s %s\n", vg, lv, flags, vg_flags) == 4) {
 
@@ -749,6 +756,8 @@ static int get_initial_state(struct dm_hash_table *excl_uuid)
 	if (fclose(lvs))
 		DEBUGLOG("lvs fclose failed: %s\n", strerror(errno));
 
+	dm_free(lvs_cmd);
+
 	return 1;
 }
 
man/Makefile.in +1 -1
index 2506fbd..3fb39b7 100644
@@ -73,7 +73,7 @@ $(MAN5) $(MAN8) $(MAN8CLUSTER):	Makefile
 %: %.in
 	@case "$@" in \
 	  */*) ;; \
-	  *) echo "Creating $@" ; $(SED) -e "s+#VERSION#+$(LVM_VERSION)+;s+#DEFAULT_SYS_DIR#+$(DEFAULT_SYS_DIR)+;s+#DEFAULT_ARCHIVE_DIR#+$(DEFAULT_ARCHIVE_DIR)+;s+#DEFAULT_BACKUP_DIR#+$(DEFAULT_BACKUP_DIR)+;s+#DEFAULT_CACHE_DIR#+$(DEFAULT_CACHE_DIR)+;s+#DEFAULT_LOCK_DIR#+$(DEFAULT_LOCK_DIR)+" $< > $@ ;; \
+	  *) echo "Creating $@" ; $(SED) -e "s+#VERSION#+$(LVM_VERSION)+;s+#DEFAULT_SYS_DIR#+$(DEFAULT_SYS_DIR)+;s+#DEFAULT_ARCHIVE_DIR#+$(DEFAULT_ARCHIVE_DIR)+;s+#DEFAULT_BACKUP_DIR#+$(DEFAULT_BACKUP_DIR)+;s+#DEFAULT_CACHE_DIR#+$(DEFAULT_CACHE_DIR)+;s+#DEFAULT_LOCK_DIR#+$(DEFAULT_LOCK_DIR)+;s+#CLVMD_PATH#+ at CLVMD_PATH@+;s+#LVM_PATH#+ at LVM_PATH@+;" $< > $@ ;; \
 	esac
 
 install_man5: $(MAN5)
man/clvmd.8.in +10 -0
index 22d94b2..0d1ed56 100644
@@ -95,5 +95,15 @@ manually specify this option to override the search.
 .TP
 .I \-V
 Display the version of the cluster LVM daemon.
+
+.SH ENVIRONMENT VARIABLES
+.TP
+.B CLVMD_BINARY
+The CLVMD binary to use when clmvd restart is requested.
+Defaults to #CLVMD_PATH#.
+.TP
+.B LVM_BINARY
+The LVM2 binary to use. Defaults to #LVM_PATH#.
+
 .SH SEE ALSO
 .BR lvm (8)
Signature · 1 line
1.7.6.2
#4 lore raw

cover Clvmd -E fixes

In reply to Zdenek Kabelac
Quoted 6 lines · Zdenek Kabelac

Patchset fixed restart of clvmd with -E option.

1st. Adds support for easier restart of clvmd
(i.e. -Isinglenode is preserved).

2nd. Bug fixes support of -E arg.

ack these two.

(maybe cluster lock type is on of the attribute we can store in udev db? to read it later :-)

Quoted 2 lines

3rd. Allows to build test suite script for checking clvmd restart.
(Any better idea here ?)

We need something for testsuite but I would like to avoid any online reconfiguration of daemon paths, this is tricky and bug there is security problem (clvms -S must not start arbitrary program
- it run with root privileges.)

IMHO that popen() should go before this patch.

I am not keen for committing such change now...

Milan

#5 lore raw

cover Clvmd -E fixes

Dne 22.9.2011 18:00, Milan Broz napsal(a):

Quoted 15 lines · Zdenek Kabelac
Quoted 6 lines

Patchset fixed restart of clvmd with -E option.

1st. Adds support for easier restart of clvmd
(i.e. -Isinglenode is preserved).

2nd. Bug fixes support of -E arg.

ack these two.

(maybe cluster lock type is on of the attribute we can store in udev db? to read it later :-)

Quoted 7 lines
Quoted 2 lines

3rd. Allows to build test suite script for checking clvmd restart.
(Any better idea here ?)

We need something for testsuite but I would like to avoid any online reconfiguration of daemon paths, this is tricky and bug there is security problem (clvms -S must not start arbitrary program
- it run with root privileges.)

Note - setting (LVM_CLVMD_BINARY as was suggest by agk) must be done before executing the first clvmd.

You can't influence anything by any further clvmd -S call - since this environment is not passed to the original restarting daemons.

Thus it should not introduce security risk here - as you are working only with settings you've had with the first start.

(If someone could manipulate root's environment - he could surely replace disc content as well)

Zdenek