Skip to contentSkip to conversation

Valgrind fixes

PATCH New Zdenek Kabelac 2 participants 9 messages Mar 24, 2011

lore
Share & tools
lore
#0 lore raw

cover Valgrind fixes

Another small set mainly with fixes discover with valgrind. I assume patch2 is the really important one.

Zdenek Kabelac (4):
Reading of unitialized memory
Fix access to released memory
Another fix for garbage send in clvmd message
Better shutdown for clvmd

 daemons/clvmd/clvmd.c         |   30 ++++++++++++++++++++++--------
 daemons/clvmd/refresh_clvmd.c |    5 +++--
 lib/cache/lvmcache.c          |   24 ++++++++++++++----------
 lib/format1/import-extents.c  |    2 +-
 4 files changed, 40 insertions(+), 21 deletions(-)
Signature · 1 line
1.7.4.1
#1 lore raw

PATCH 1/4 Reading of unitialized memory

This is rather hint fix just for reading of unknown memory.

Could be reached via few our test cases:

==11501== Invalid read of size 8

==11501==    at 0x49B2E0: _area_length (import-extents.c:204)
==11501==    by 0x49B40C: _read_linear (import-extents.c:222)
==11501==    by 0x49B952: _build_segments (import-extents.c:323)
==11501==    by 0x49B9A0: _build_all_segments (import-extents.c:334)
==11501==    by 0x49BB4C: import_extents (import-extents.c:364)
==11501==    by 0x497655: _format1_vg_read (format1.c:217)
==11501==    by 0x47E43E: _vg_read (metadata.c:2901)

cut from t-vgcvgbackup-usage.sh
--
pvcreate -M1 $(cat DEVICES)
vgcreate -M1 -c n $vg $(cat DEVICES)
lvcreate -l1 -n $lv1 $vg $dev1
--

Idea of the fix is rather defensive - to allocate one extra element to 'map' array which is then used in _area_length().

lv->le_count == 1 - thus allocated 'map' array allows to access only element [0] - but the loop in _area_lenght start directly with:
len++ (== 1) - thus accessing element behind the allocated array.

As I'm not familiar with the LVM1 code so there is potentialy better fix. My patch is rather hint where is the bug - and avoids read of random memory - but does not fix algorithmic bug here - do we even care about format 1 ?

---
 lib/format1/import-extents.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
lib/format1/import-extents.c +1 -1
index 99723ee..dc12776 100644
@@ -64,7 +64,7 @@ static struct dm_hash_table *_create_lv_maps(struct dm_pool *mem,
 
 		lvm->lv = ll->lv;
 		if (!(lvm->map = dm_pool_zalloc(mem, sizeof(*lvm->map)
-					     * ll->lv->le_count)))
+					     * (ll->lv->le_count + 1))))
 			goto_bad;
 
 		if (!dm_hash_insert(maps, ll->lv->name, lvm))
Signature · 1 line
1.7.4.1
#2 lore raw

PATCH 1/4 Reading of unitialized memory

lib/format1/import-extents.c static struct dm_hash_table *_create_lv_maps(struct dm_pool *mem, · line 67

Quoted 3 lines · Zdenek Kabelac
lib/format1/import-extents.c +1 -1
 		if (!(lvm->map = dm_pool_zalloc(mem, sizeof(*lvm->map)
-					     * ll->lv->le_count)))
+					     * (ll->lv->le_count + 1))))

I do not like such fixes, but because this code is really not going to be used in future (and I know how complicated it is), let's "fix" it this way.

turn a blind eye to ... ACK :-)

Milan

#3 lore raw

PATCH 2/4 Fix access to released memory

In reply to Zdenek Kabelac

Somewhat tricky part of the code.

Invalid primary_vginfo was supposed to move all its lvmcache_info to orphan_vginfo - however it has called _drop_vginfo() inside the loop that remove primary_vginfo itself - thus made the loop using released memory.

Patch used _vginfo_detach_info() instead and call _drop_vginfo after this whole loop is finished.

Here is valgrind trace it should be fixed:

Invalid read of size 8
at 0x41E960: _lvmcache_update_vgname (lvmcache.c:1229) by 0x41EF86: lvmcache_update_vgname_and_id (lvmcache.c:1360) by 0x441393: _text_read (text_label.c:329)
by 0x442221: label_read (label.c:289)
by 0x41CF92: lvmcache_label_scan (lvmcache.c:635) by 0x45B303: _vg_read_by_vgid (metadata.c:3342)
by 0x45B4A6: lv_from_lvid (metadata.c:3381)
by 0x41B555: lv_activation_filter (activate.c:1346) by 0x415868: do_activate_lv (lvm-functions.c:343) by 0x415E8C: do_lock_lv (lvm-functions.c:532)
by 0x40FD5F: do_command (clvmd-command.c:120)
by 0x413D7B: process_local_command (clvmd.c:1686)
Address 0x63eba10 is 16 bytes inside a block of size 160 free'd
at 0x4C2756E: free (vg_replace_malloc.c:366)
by 0x41DE70: _free_vginfo (lvmcache.c:980)
by 0x41DEDA: _drop_vginfo (lvmcache.c:998)
by 0x41E854: _lvmcache_update_vgname (lvmcache.c:1238) by 0x41EF86: lvmcache_update_vgname_and_id (lvmcache.c:1360) by 0x441393: _text_read (text_label.c:329)
by 0x442221: label_read (label.c:289)
by 0x41CF92: lvmcache_label_scan (lvmcache.c:635) by 0x45B303: _vg_read_by_vgid (metadata.c:3342)
by 0x45B4A6: lv_from_lvid (metadata.c:3381)
by 0x41B555: lv_activation_filter (activate.c:1346) by 0x415868: do_activate_lv (lvm-functions.c:343)

problematic line:
dm_list_iterate_items_safe(info2, info3, &primary_vginfo->infos)

---
 lib/cache/lvmcache.c |   24 ++++++++++++++----------
 1 files changed, 14 insertions(+), 10 deletions(-)
lib/cache/lvmcache.c +14 -10
index 4e2e158..ee7d278 100644
@@ -1107,17 +1107,17 @@ static int _lvmcache_update_vgname(struct lvmcache_info *info,
 		 * Otherwise we risk bogus warnings of duplicate VGs.
 		 */
 		while ((primary_vginfo = vginfo_from_vgname(vgname, NULL)) &&
-		       _scanning_in_progress && _vginfo_is_invalid(primary_vginfo))
+		       _scanning_in_progress && _vginfo_is_invalid(primary_vginfo)) {
+			orphan_vginfo = vginfo_from_vgname(primary_vginfo->fmt->orphan_vg_name, NULL);
+			if (!orphan_vginfo) {
+				log_error(INTERNAL_ERROR "Orphan vginfo %s lost from cache.",
+					  primary_vginfo->fmt->orphan_vg_name);
+				dm_free(vginfo->vgname);
+				dm_free(vginfo);
+				return 0;
+			}
 			dm_list_iterate_items_safe(info2, info3, &primary_vginfo->infos) {
-				orphan_vginfo = vginfo_from_vgname(primary_vginfo->fmt->orphan_vg_name, NULL);
-				if (!orphan_vginfo) {
-					log_error(INTERNAL_ERROR "Orphan vginfo %s lost from cache.",
-						  primary_vginfo->fmt->orphan_vg_name);
-					dm_free(vginfo->vgname);
-					dm_free(vginfo);
-					return 0;
-				}
-				_drop_vginfo(info2, primary_vginfo);	
+				_vginfo_detach_info(info2);
 				_vginfo_attach_info(orphan_vginfo, info2);
 				if (info2->mdas.n)
 					sprintf(mdabuf, " with %u mdas",
@@ -1129,6 +1129,10 @@ static int _lvmcache_update_vgname(struct lvmcache_info *info,
 					  vgname, orphan_vginfo->vgid[0] ? " (" : "",
 					  orphan_vginfo->vgid[0] ? orphan_vginfo->vgid : "",
 					  orphan_vginfo->vgid[0] ? ")" : "", mdabuf);
+			}
+
+			if (!_drop_vginfo(NULL, primary_vginfo))
+				return_0;
 		}
 
 		if (!_insert_vginfo(vginfo, vgid, vgstatus, creation_host,
Signature · 1 line
1.7.4.1
#4 lore raw

PATCH 2/4 Fix access to released memory

Quoted 4 lines · Zdenek Kabelac

Invalid primary_vginfo was supposed to move all its lvmcache_info to orphan_vginfo - however it has called _drop_vginfo() inside the loop that remove primary_vginfo itself - thus made the loop using released memory.

ack (we talked about it yesterday)

Milan

#5 lore raw

PATCH 3/4 Another fix for garbage send in clvmd message

In reply to Zdenek Kabelac

(Similar to my 2 other previous already posted patches).

---
 daemons/clvmd/refresh_clvmd.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)
daemons/clvmd/refresh_clvmd.c +3 -2
index 1e88b5c..805fbe3 100644
@@ -154,6 +154,7 @@ static void _build_header(struct clvm_header *head, int cmd, const char *node,
 	head->cmd = cmd;
 	head->status = 0;
 	head->flags = 0;
+	head->xid = 0;
 	head->clientid = 0;
 	head->arglen = len;
 
@@ -197,11 +198,11 @@ static int _cluster_request(char cmd, const char *node, void *data, int len,
 	if (_clvmd_sock == -1)
 		return 0;
 
-	_build_header(head, cmd, node, len);
+	_build_header(head, cmd, node, len - 1);
 	memcpy(head->node + strlen(head->node) + 1, data, len);
 
 	status = _send_request(outbuf, sizeof(struct clvm_header) +
-			       strlen(head->node) + len, &retbuf, no_response);
+			       strlen(head->node) + len - 1, &retbuf, no_response);
 	if (!status || no_response)
 		goto out;
 
Signature · 1 line
1.7.4.1
#6 lore raw

PATCH 3/4 Another fix for garbage send in clvmd message

daemons/clvmd/refresh_clvmd.c static int _cluster_request(char cmd, const char *node, void *data, int len, · line 205

Quoted 7 lines · Zdenek Kabelac
daemons/clvmd/refresh_clvmd.c +2 -2
-	_build_header(head, cmd, node, len);
+	_build_header(head, cmd, node, len - 1);
 	memcpy(head->node + strlen(head->node) + 1, data, len);
 
 	status = _send_request(outbuf, sizeof(struct clvm_header) +
-			       strlen(head->node) + len, &retbuf, no_response);
+			       strlen(head->node) + len - 1, &retbuf, no_response);

ACK, but this protocol _MUST_ be rewritten soon completely.

really, no comment to use of this struct...
(note that if strlen(node) > 0, args points somewhere inside on node name. wonderful.

struct clvm_header {
uint8_t cmd; /* See below */
...
char node[1]; /* Actually a NUL-terminated string, node name.
If this is empty then the command is forwarded to all cluster nodes unless FLAG_LOCAL is also set. */
char args[1]; /* Arguments for the command follow the
node name, This member is only valid if the node name is empty */

Milan

#7 lore raw

PATCH 4/4 Better shutdown for clvmd

In reply to Zdenek Kabelac

Ok - this is 'a small step' towards cleaner shutdown sequence. It could be seens as unneeded - as normally clvmd doens't care about unreleased memory on exit - but for valgrind testing it's better to have them cleaned all.

So - few things are left on exit path - this patch starts to remove just some of them.

1. lvm_thread_fs is made as a thread which could be joined on exit()
2. memory allocated to local_clien_head list is released. (this part is somewhat more complex if the proper reaction is needed - and as it requires some heavier code moving - it will be resolved later.

---
 daemons/clvmd/clvmd.c |   30 ++++++++++++++++++++++--------
 1 files changed, 22 insertions(+), 8 deletions(-)
daemons/clvmd/clvmd.c +22 -8
index 361fc62..fc092cf 100644
@@ -103,8 +103,6 @@ static int child_pipe[2];
 
 typedef enum {IF_AUTO, IF_CMAN, IF_GULM, IF_OPENAIS, IF_COROSYNC, IF_SINGLENODE} if_type_t;
 
-typedef void *(lvm_pthread_fn_t)(void*);
-
 /* Prototypes for code further down */
 static void sigusr2_handler(int sig);
 static void sighup_handler(int sig);
@@ -134,7 +132,7 @@ static int check_all_clvmds_running(struct local_client *client);
 static int local_rendezvous_callback(struct local_client *thisfd, char *buf,
 				     int len, const char *csid,
 				     struct local_client **new_client);
-static void lvm_thread_fn(void *) __attribute__ ((noreturn));
+static void *lvm_thread_fn(void *);
 static int add_to_lvmqueue(struct local_client *client, struct clvm_header *msg,
 			   int msglen, const char *csid);
 static int distribute_command(struct local_client *thisfd);
@@ -333,7 +331,7 @@ static void check_permissions(void)
 int main(int argc, char *argv[])
 {
 	int local_sock;
-	struct local_client *newfd;
+	struct local_client *newfd, *delfd;
 	struct utsname nodeinfo;
 	struct lvm_startup_params lvm_params;
 	int opt;
@@ -581,8 +579,7 @@ int main(int argc, char *argv[])
 	pthread_mutex_lock(&lvm_start_mutex);
 	lvm_params.using_gulm = using_gulm;
 	lvm_params.argv = argv;
-	pthread_create(&lvm_thread, NULL, (lvm_pthread_fn_t*)lvm_thread_fn,
-			(void *)&lvm_params);
+	pthread_create(&lvm_thread, NULL, lvm_thread_fn, &lvm_params);
 
 	/* Tell the rest of the cluster our version number */
 	/* CMAN can do this immediately, gulm needs to wait until
@@ -601,9 +598,24 @@ int main(int argc, char *argv[])
 	/* Do some work */
 	main_loop(local_sock, cmd_timeout);
 
+	pthread_mutex_lock(&lvm_thread_mutex);
+	pthread_cond_signal(&lvm_thread_cond);
+	pthread_mutex_unlock(&lvm_thread_mutex);
+	if ((errno = pthread_join(lvm_thread, NULL)))
+		log_sys_error("pthread_join", "");
+
 	close_local_sock(local_sock);
 	destroy_lvm();
 
+	for (newfd = local_client_head.next; newfd != NULL;) {
+		delfd = newfd;
+		newfd = newfd->next;
+		/* FIXME: needs cleanup code from read_from_local_sock() */
+                /* for now break of CLVMD presents access to free memory here */
+		safe_close(&(delfd->fd));
+		free(delfd);
+	}
+
 	return 0;
 }
 
@@ -1931,7 +1943,7 @@ static int process_work_item(struct lvm_thread_cmd *cmd)
 /*
  * Routine that runs in the "LVM thread".
  */
-static void lvm_thread_fn(void *arg)
+static void *lvm_thread_fn(void *arg)
 {
 	struct dm_list *cmdl, *tmp;
 	sigset_t ss;
@@ -1952,7 +1964,7 @@ static void lvm_thread_fn(void *arg)
 	pthread_mutex_unlock(&lvm_start_mutex);
 
 	/* Now wait for some actual work */
-	for (;;) {
+	while (!quit) {
 		DEBUGLOG("LVM thread waiting for work\n");
 
 		pthread_mutex_lock(&lvm_thread_mutex);
@@ -1975,6 +1987,8 @@ static void lvm_thread_fn(void *arg)
 		}
 		pthread_mutex_unlock(&lvm_thread_mutex);
 	}
+
+	pthread_exit(NULL);
 }
 
 /* Pass down some work to the LVM thread */
Signature · 1 line
1.7.4.1
#8 lore raw

PATCH 4/4 Better shutdown for clvmd

Quoted 1 line · Zdenek Kabelac

Ok - this is 'a small step' towards cleaner shutdown sequence.

(Yes it is small step. I would like just mention that we are not walking on the Moon, but trying to move an elephant from porcelain storage room.)

Btw there are 3 problems in this patch
- unrelated code tidy
- thread join
- fix memleak and close of descriptors

daemons/clvmd/clvmd.c int main(int argc, char *argv[]) · line 582

Quoted 6 lines
daemons/clvmd/clvmd.c +2 -3
-typedef void *(lvm_pthread_fn_t)(void*);
+static void *lvm_thread_fn(void *);
 
-	pthread_create(&lvm_thread, NULL, (lvm_pthread_fn_t*)lvm_thread_fn,
-			(void *)&lvm_params);
+	pthread_create(&lvm_thread, NULL, lvm_thread_fn, &lvm_params);

code tidying, please use separate patch for these next time, so it is not mixed with real bugfixes... ;-)

daemons/clvmd/clvmd.c static void lvm_thread_fn(void *arg) · line 1967

Quoted 12 lines
daemons/clvmd/clvmd.c +6 -1
+	pthread_mutex_lock(&lvm_thread_mutex);
+	pthread_cond_signal(&lvm_thread_cond);
+	pthread_mutex_unlock(&lvm_thread_mutex);
+	if ((errno = pthread_join(lvm_thread, NULL)))
+		log_sys_error("pthread_join", "");
@@ -1952,7 +1964,7 @@ static void lvm_thread_fn(void *arg)
 	pthread_mutex_unlock(&lvm_start_mutex);
 
 	/* Now wait for some actual work */
-	for (;;) {
+	while (!quit) {

I hope I thought what can happen when there is still work in queue and I think it is correctly handled...

ACK

daemons/clvmd/clvmd.c int main(int argc, char *argv[]) · line 617

Quoted 8 lines
daemons/clvmd/clvmd.c +8 -0
+	for (newfd = local_client_head.next; newfd != NULL;) {
+		delfd = newfd;
+		newfd = newfd->next;
+		/* FIXME: needs cleanup code from read_from_local_sock() */
+                /* for now break of CLVMD presents access to free memory here */
+		safe_close(&(delfd->fd));
+		free(delfd);
+	}

Well, this looks safe. Everything should be stopped, so even there is FIXME still, it fixes part of problem.

ACK

Milan