Skip to contentSkip to conversation

Current state of Vals patches and the file map lookup patches

PATCH In review Ian Kent 3 participants 9 messages Jan 19, 2009 – Jan 20, 2009

Subsystems (2)
lore
Share & tools
lore

Ian Kent

author maintainer
#0 lore raw

cover Current state of Vals patches and the file map lookup patches

I've pulled in the patches that Val posted and made a couple of minor changes. I also have what I think is just about what we need to reduce the overhead of file map scanning.

Check'em out please folks.

I'm a little concerned about the "fix kernel includes" patch because there should be more to it and I've altered it in line with that. I'm not sure if we should post a kernel patch first, pull in the updated kernel header files to the tar and finally make the other couple of changes then. But then I may have it wrong so check it out.

Ian

---

Ian Kent (1):
autofs-5.0.4 - always read file maps

Valerie Aurora Henson (3):
autofs-5.0.4 - fix kernel includes
autofs-5.0.4 - easy alloca replacements
autofs-5.0.4 - make MAX_ERR_BUF and PARSE_MAX_BUF use easier to audit

 daemon/automount.c             |   29 ++++----
 daemon/direct.c                |   12 +--
 daemon/flag.c                  |   13 ++--
 daemon/indirect.c              |   12 +--
 daemon/lookup.c                |    9 ++-
 daemon/module.c                |   45 ++++---------
 include/automount.h            |    2 -
 include/linux/auto_dev-ioctl.h |   10 +++
 include/linux/auto_fs.h        |    5 +
 lib/cache.c                    |   31 +++------
 lib/cat_path.c                 |    1 
 modules/lookup_file.c          |  141 +++++++++++++---------------------------
 modules/lookup_ldap.c          |  142 +++++++++++++++++++++++-----------------
 modules/lookup_nisplus.c       |   71 ++++++++++++--------
 modules/mount_autofs.c         |    1 
 modules/mount_bind.c           |    7 +-
 modules/mount_changer.c        |    5 -
 modules/mount_ext2.c           |    5 -
 modules/mount_generic.c        |    5 -
 19 files changed, 248 insertions(+), 298 deletions(-)
Signature · 1 line
Signature
author maintainer
#1 lore raw

PATCH 1/4 autofs-5.0.4 - always read file maps

autofs tries to not load an entire map into the internal cache unless it has to. For maps that do get loaded into the cache it relies on checks to work out if a map is up to date in order to trigger a map read. This is fine for maps that can do direct key lookups but file maps need to do a linear search through the file when locating an entry for a key. For large maps this can be a huge overhead. This patch make autofs always load file based maps at start and makes use of the map file mtime to discover if the cache needs to be refreshed.
---

 daemon/lookup.c       |    9 +++++--
 modules/lookup_file.c |   59 +++++++++++++++++--------------------------------
 2 files changed, 27 insertions(+), 41 deletions(-)
daemon/lookup.c +6 -3
index 741d846..e034348 100644
@@ -283,10 +283,13 @@ static int do_read_map(struct autofs_point *ap, struct map_source *map, time_t a
 	 * for the fail cases to function correctly and to cache the
 	 * lookup handle.
 	 *
-	 * We always need to whole map for direct mounts in order to
-	 * mount the triggers.
+	 * We always need to read the whole map for direct mounts in
+	 * order to mount the triggers. We also want to read the whole
+	 * map if it's a file map to avoid potentially lengthy linear
+	 * file scanning.
 	 */
-	if (!(ap->flags & MOUNT_FLAG_GHOST) && ap->type != LKP_DIRECT)
+	if (strcmp(map->type, "file") &&
+	    !(ap->flags & MOUNT_FLAG_GHOST) && ap->type != LKP_DIRECT)
 		return NSS_STATUS_SUCCESS;
 
 	if (!map->stale)
modules/lookup_file.c +21 -38

135 lines not on this page · raw message

#2 lore raw

Re: [PATCH 1/4] autofs-5.0.4 - always read file maps

On Mon, Jan 19, 2009 at 11:24 PM, Ian Kent <[email protected]> wrote:

autofs tries to not load an entire map into the internal cache unless it

Quoted 76 lines

has to. For maps that do get loaded into the cache it relies on checks to work out if a map is up to date in order to trigger a map read. This is fine for maps that can do direct key lookups but file maps need to do a linear search through the file when locating an entry for a key. For large maps this can be a huge overhead. This patch make autofs always load file based maps at start and makes use of the map file mtime to discover if the cache needs to be refreshed.
---

 daemon/lookup.c       |    9 +++++--
 modules/lookup_file.c |   59
+++++++++++++++++--------------------------------
 2 files changed, 27 insertions(+), 41 deletions(-)
daemon/lookup.c +0 -0
index 741d846..e034348 100644
@@ -283,10 +283,13 @@ static int do_read_map(struct autofs_point *ap,

struct map_source *map, time_t a
* for the fail cases to function correctly and to cache the
* lookup handle.
*

daemon/lookup.c +6 -3
-        * We always need to whole map for direct mounts in order to
-        * mount the triggers.
+        * We always need to read the whole map for direct mounts in
+        * order to mount the triggers. We also want to read the whole
+        * map if it's a file map to avoid potentially lengthy linear
+        * file scanning.
        */
-       if (!(ap->flags & MOUNT_FLAG_GHOST) && ap->type != LKP_DIRECT)
+       if (strcmp(map->type, "file") &&
+           !(ap->flags & MOUNT_FLAG_GHOST) && ap->type != LKP_DIRECT)
               return NSS_STATUS_SUCCESS;
 
       if (!map->stale)
modules/lookup_file.c +0 -11
index 95b9f6f..7672725 100644
@@ -44,7 +44,6 @@ typedef enum { esc_none, esc_char, esc_val, esc_all }
 ESCAPES;
 
 struct lookup_context {
       const char *mapname;
-       time_t mtime;
       struct parse_mod *parse;
 };
@@ -54,7 +53,6 @@ int lookup_init(const char *mapfmt, int argc, const char
 *const *argv, void **co
 {
       struct lookup_context *ctxt;
       char buf[MAX_ERR_BUF];
-       struct stat st;
 
       *context = NULL;
@@ -87,15 +85,6 @@ int lookup_init(const char *mapfmt, int argc, const char
 *const *argv, void **co
               return 1;
       }
 
-       if (stat(ctxt->mapname, &st)) {
-               free(ctxt);
-               logmsg(MODPREFIX "file map %s, could not stat",
-                    argv[0]);
-               return 1;
-       }
-
-       ctxt->mtime = st.st_mtime;
-
       if (!mapfmt)
               mapfmt = MAPFMT_DEFAULT;

modules/lookup_file.c int lookup_read_master(struct master *master, time_t · line 385

Quoted 7 lines
modules/lookup_file.c +0 -1
@@ -391,7 +380,6 @@ int lookup_read_master(struct master *master, time_t
 age, void *context)
       int blen;
       char *path;
       char *ent;
-       struct stat st;
       FILE *f;
       int fd;

You should be able to get rid of `fd' and the fileno(3) call now.

       unsigned int path_len, ent_len;
Quoted 16 lines
modules/lookup_file.c +0 -7
@@ -504,13 +492,6 @@ int lookup_read_master(struct master *master, time_t
 age, void *context)
                       break;
       }
 
-       if (fstat(fd, &st)) {
-               crit(logopt, MODPREFIX "file map %s, could not stat",
-                      ctxt->mapname);
-               return NSS_STATUS_UNAVAIL;
-       }
-       ctxt->mtime = st.st_mtime;
-
       fclose(f);
 
       return NSS_STATUS_SUCCESS;

modules/lookup_file.c int lookup_read_map(struct autofs_point *ap, time_t · line 628

Quoted 7 lines
modules/lookup_file.c +0 -1
@@ -642,7 +623,6 @@ int lookup_read_map(struct autofs_point *ap, time_t
 age, void *context)
       struct mapent_cache *mc;
       char *key;
       char *mapent;
-       struct stat st;
       FILE *f;
       int fd;

You should be able to get rid of `fd' and the fileno(3) call now.

       unsigned int k_len, m_len;
Quoted 66 lines
modules/lookup_file.c +0 -7
@@ -748,13 +728,6 @@ int lookup_read_map(struct autofs_point *ap, time_t
 age, void *context)
                       break;
       }
 
-       if (fstat(fd, &st)) {
-               crit(ap->logopt,
-                    MODPREFIX "file map %s, could not stat",
-                    ctxt->mapname);
-               return NSS_STATUS_UNAVAIL;
-       }
-       ctxt->mtime = st.st_mtime;
       source->age = age;
 
       fclose(f);
modules/lookup_file.c +11 -11
@@ -951,9 +924,6 @@ static int check_map_indirect(struct autofs_point *ap,
       if (ret == CHE_FAIL)
               return NSS_STATUS_NOTFOUND;
 
-       if (ret & CHE_UPDATED)
-               source->stale = 1;
-
       pthread_cleanup_push(cache_lock_cleanup, mc);
       cache_writelock(mc);
       exists = cache_lookup_distinct(mc, key);
@@ -963,7 +933,6 @@ static int check_map_indirect(struct autofs_point *ap,
                       free(exists->mapent);
                       exists->mapent = NULL;
                       exists->status = 0;
-                       source->stale = 1;
               }
       }
       pthread_cleanup_pop(1);
@@ -985,14 +954,8 @@ static int check_map_indirect(struct autofs_point *ap,
               we = cache_lookup_distinct(mc, "*");
               if (we) {
                       /* Wildcard entry existed and is now gone */
-                       if (we->source == source && (wild & CHE_MISSING)) {
+                       if (we->source == source && (wild & CHE_MISSING))
                               cache_delete(mc, "*");
-                               source->stale = 1;
-                       }
-               } else {
-                       /* Wildcard not in map but now is */
-                       if (wild & (CHE_OK | CHE_UPDATED))
-                               source->stale = 1;
               }
               pthread_cleanup_pop(1);
 
@@ -1062,9 +1025,28 @@ int lookup_mount(struct autofs_point *ap, const char
 *name, int name_len, void *
        * we never know about it.
        */
       if (ap->type == LKP_INDIRECT && *key != '/') {
+               struct stat st;
               char *lkp_key;
 
+               /*
+                * We can skip the map lookup and cache update altogether
+                * if we know the map hasn't been modified since it was
+                * last read. If it has then we can mark the map stale
+                * so a re-read is triggered following the lookup.
+                */
+               if (stat(ctxt->mapname, &st)) {
+                       error(ap->logopt, MODPREFIX
+                             "file map %s, could not stat",

modules/lookup_file.c int lookup_mount(struct autofs_point *ap, const char · line 1050

Quoted 10 lines
modules/lookup_file.c +8 -0
 ctxt->mapname);
+                       return NSS_STATUS_UNAVAIL;
+               }
+
               cache_readlock(mc);
+               me = cache_lookup_first(mc);
+               if (me && st.st_mtime <= me->age)
+                       goto do_cache_lookup;
+               else
+                       source->stale = 1;

Perhaps some output for debugging purposes wouldn't go astray?

+

Quoted 12 lines
modules/lookup_file.c +1 -0
               me = cache_lookup_distinct(mc, key);
               if (me && me->multi)
                       lkp_key = strdup(me->multi->key);
@@ -1088,6 +1070,7 @@ int lookup_mount(struct autofs_point *ap, const char
 *name, int name_len, void *
       }
 
       cache_readlock(mc);
+do_cache_lookup:
       me = cache_lookup(mc, key);
       /* Stale mapent => check for entry in alternate source or wildcard

*/

Quoted 3 lines
       if (me && !me->mapent) {

_______________________________________________
autofs mailing list
[email protected]
http://linux.kernel.org/mailman/listinfo/autofs

author maintainer
#3 lore raw

Re: [PATCH 1/4] autofs-5.0.4 - always read file maps

Quoted 1 line · Paul Wankadia

On Mon, Jan 19, 2009 at 11:24 PM, Ian Kent <[email protected]> wrote:

snip ...

Quoted 38 lines
        @@ -391,7 +380,6 @@ int lookup_read_master(struct master
        *master, time_t age, void *context)
               int blen;
               char *path;
               char *ent;
        -       struct stat st;
               FILE *f;
               int fd;

You should be able to get rid of `fd' and the fileno(3) call now.

               unsigned int path_len, ent_len;
        @@ -504,13 +492,6 @@ int lookup_read_master(struct master
        *master, time_t age, void *context)
                               break;
               }
        
        -       if (fstat(fd, &st)) {
        -               crit(logopt, MODPREFIX "file map %s, could not
        stat",
        -                      ctxt->mapname);
        -               return NSS_STATUS_UNAVAIL;
        -       }
        -       ctxt->mtime = st.st_mtime;
        -
               fclose(f);
        
               return NSS_STATUS_SUCCESS;
        @@ -642,7 +623,6 @@ int lookup_read_map(struct autofs_point
        *ap, time_t age, void *context)
               struct mapent_cache *mc;
               char *key;
               char *mapent;
        -       struct stat st;
               FILE *f;
               int fd;
Quoted 1 line

You should be able to get rid of `fd' and the fileno(3) call now.

Yep, quite right.

Quoted 90 lines

unsigned int k_len, m_len;
@@ -748,13 +728,6 @@ int lookup_read_map(struct autofs_point *ap, time_t age, void *context)
break;
}

- if (fstat(fd, &st)) {
- crit(ap->logopt,
- MODPREFIX "file map %s, could not stat",
- ctxt->mapname);
- return NSS_STATUS_UNAVAIL;
- }
- ctxt->mtime = st.st_mtime;
source->age = age;

fclose(f);
@@ -951,9 +924,6 @@ static int check_map_indirect(struct autofs_point *ap,
if (ret == CHE_FAIL)
return NSS_STATUS_NOTFOUND;

- if (ret & CHE_UPDATED)
- source->stale = 1;
-
pthread_cleanup_push(cache_lock_cleanup, mc); cache_writelock(mc);
exists = cache_lookup_distinct(mc, key);
@@ -963,7 +933,6 @@ static int check_map_indirect(struct autofs_point *ap,
free(exists->mapent); exists->mapent = NULL; exists->status = 0;
- source->stale = 1;
}
}
pthread_cleanup_pop(1);
@@ -985,14 +954,8 @@ static int check_map_indirect(struct autofs_point *ap,
we = cache_lookup_distinct(mc, "*"); if (we) {
/* Wildcard entry existed and is now
gone */
- if (we->source == source && (wild &
CHE_MISSING)) {
+ if (we->source == source && (wild &
CHE_MISSING))
cache_delete(mc, "*");
- source->stale = 1;
- }
- } else {
- /* Wildcard not in map but now is */
- if (wild & (CHE_OK | CHE_UPDATED))
- source->stale = 1;
}
pthread_cleanup_pop(1);

@@ -1062,9 +1025,28 @@ int lookup_mount(struct autofs_point *ap, const char *name, int name_len, void *
* we never know about it.
*/
if (ap->type == LKP_INDIRECT && *key != '/') {
+ struct stat st;
char *lkp_key;

+ /*
+ * We can skip the map lookup and cache update
altogether
+ * if we know the map hasn't been modified
since it was
+ * last read. If it has then we can mark the
map stale
+ * so a re-read is triggered following the
lookup.
+ */
+ if (stat(ctxt->mapname, &st)) {
+ error(ap->logopt, MODPREFIX
+ "file map %s, could not stat",
ctxt->mapname);
+ return NSS_STATUS_UNAVAIL;
+ }
+
cache_readlock(mc);
+ me = cache_lookup_first(mc);
+ if (me && st.st_mtime <= me->age)
+ goto do_cache_lookup;
+ else
+ source->stale = 1;

Quoted 1 line

Perhaps some output for debugging purposes wouldn't go astray?

Mmm .. I'll see what I can do.

Quoted 18 lines

        +
                       me = cache_lookup_distinct(mc, key);
                       if (me && me->multi)
                               lkp_key = strdup(me->multi->key);
        @@ -1088,6 +1070,7 @@ int lookup_mount(struct autofs_point
        *ap, const char *name, int name_len, void *
               }
        
               cache_readlock(mc);
        +do_cache_lookup:
               me = cache_lookup(mc, key);
               /* Stale mapent => check for entry in alternate source
        or wildcard */
               if (me && !me->mapent) {
author maintainer
#4 lore raw

PATCH 2/4 autofs-5.0.4 - make MAX_ERR_BUF and PARSE_MAX_BUF use easier to audit

In reply to Ian Kent

From: Valerie Aurora Henson <[email protected]>

Non-critical changes to make auditing buffer lengths easier.

* Some buffers were the wrong (too big) size, some were used twice for different purposes.
* Use sizeof(buf) instead of repeating the *MAX* define in functions that need to know the size of a statically allocated buffer.
* Fix a compiler warning about discarding the const on a string.
---

 modules/lookup_ldap.c |   49 ++++++++++++++++++++++---------------------------
 1 files changed, 22 insertions(+), 27 deletions(-)
modules/lookup_ldap.c +22 -27
index 6ba80eb..c7aecf0 100644
@@ -274,7 +274,7 @@ LDAP *init_ldap_connection(unsigned logopt, const char *uri, struct lookup_conte
 
 static int get_query_dn(unsigned logopt, LDAP *ldap, struct lookup_context *ctxt, const char *class, const char *key)
 {
-	char buf[PARSE_MAX_BUF];
+	char buf[MAX_ERR_BUF];
 	char *query, *dn, *qdn;
 	LDAPMessage *result, *e;
 	struct ldap_searchdn *sdns = NULL;
@@ -298,7 +298,7 @@ static int get_query_dn(unsigned logopt, LDAP *ldap, struct lookup_context *ctxt
 
 	query = alloca(l);
 	if (query == NULL) {
-		char *estr = strerror_r(errno, buf, MAX_ERR_BUF);
+		char *estr = strerror_r(errno, buf, sizeof(buf));
 		crit(logopt, MODPREFIX "alloca: %s", estr);
 		return NSS_STATUS_UNAVAIL;
 	}
@@ -1073,7 +1073,7 @@ static int parse_server_string(unsigned logopt, const char *url, struct lookup_c
 			}
 			if (!tmp) {
 				char *estr;
-				estr = strerror_r(errno, buf, MAX_ERR_BUF);
+				estr = strerror_r(errno, buf, sizeof(buf));
 				logerr(MODPREFIX "malloc: %s", estr);
 				return 0;
 			}
@@ -1095,7 +1095,7 @@ static int parse_server_string(unsigned logopt, const char *url, struct lookup_c
 			tmp = malloc(l + 1);
 			if (!tmp) {
 				char *estr;
-				estr = strerror_r(errno, buf, MAX_ERR_BUF);
+				estr = strerror_r(errno, buf, sizeof(buf));
 				crit(logopt, MODPREFIX "malloc: %s", estr);
 				return 0;
 			}
@@ -1130,7 +1130,7 @@ static int parse_server_string(unsigned logopt, const char *url, struct lookup_c
 		/* Isolate the server's name. */
 		if (!tmp) {
 			char *estr;
-			estr = strerror_r(errno, buf, MAX_ERR_BUF);
+			estr = strerror_r(errno, buf, sizeof(buf));
 			logerr(MODPREFIX "malloc: %s", estr);
 			return 0;
 		}
@@ -1171,7 +1171,7 @@ static int parse_server_string(unsigned logopt, const char *url, struct lookup_c
 				ctxt->mapname = map;
 			else {
 				char *estr;
-				estr = strerror_r(errno, buf, MAX_ERR_BUF);
+				estr = strerror_r(errno, buf, sizeof(buf));
 				logerr(MODPREFIX "malloc: %s", estr);
 				if (ctxt->server)
 					free(ctxt->server);
@@ -1182,7 +1182,7 @@ static int parse_server_string(unsigned logopt, const char *url, struct lookup_c
 			base = malloc(l + 1);
 			if (!base) {
 				char *estr;
-				estr = strerror_r(errno, buf, MAX_ERR_BUF);
+				estr = strerror_r(errno, buf, sizeof(buf));
 				logerr(MODPREFIX "malloc: %s", estr);
 				if (ctxt->server)
 					free(ctxt->server);
@@ -1196,7 +1196,7 @@ static int parse_server_string(unsigned logopt, const char *url, struct lookup_c
 		char *map = malloc(l + 1);
 		if (!map) {
 			char *estr;
-			estr = strerror_r(errno, buf, MAX_ERR_BUF);
+			estr = strerror_r(errno, buf, sizeof(buf));
 			logerr(MODPREFIX "malloc: %s", estr);
 			if (ctxt->server)
 				free(ctxt->server);
@@ -1309,7 +1309,7 @@ int lookup_init(const char *mapfmt, int argc, const char *const *argv, void **co
 	/* If we can't build a context, bail. */
 	ctxt = malloc(sizeof(struct lookup_context));
 	if (!ctxt) {
-		char *estr = strerror_r(errno, buf, MAX_ERR_BUF);
+		char *estr = strerror_r(errno, buf, sizeof(buf));
 		logerr(MODPREFIX "malloc: %s", estr);
 		return 1;
 	}
@@ -1410,8 +1410,9 @@ int lookup_read_master(struct master *master, time_t age, void *context)
 	unsigned int timeout = master->default_timeout;
 	unsigned int logging = master->default_logging;
 	unsigned int logopt = master->logopt;
-	int rv, l, count, blen;
-	char buf[PARSE_MAX_BUF];
+	int rv, l, count;
+	char buf[MAX_ERR_BUF];
+	char parse_buf[PARSE_MAX_BUF];
 	char *query;
 	LDAPMessage *result, *e;
 	char *class, *info, *entry;
@@ -1433,7 +1434,7 @@ int lookup_read_master(struct master *master, time_t age, void *context)
 
 	query = alloca(l);
 	if (query == NULL) {
-		char *estr = strerror_r(errno, buf, MAX_ERR_BUF);
+		char *estr = strerror_r(errno, buf, sizeof(buf));
 		logerr(MODPREFIX "alloca: %s", estr);
 		return NSS_STATUS_UNAVAIL;
 	}
@@ -1523,18 +1524,12 @@ int lookup_read_master(struct master *master, time_t age, void *context)
 			goto next;
 		}
 
-		blen = strlen(*keyValue) + 1 + strlen(*values) + 2;
-		if (blen > PARSE_MAX_BUF) {
+		if (snprintf(parse_buf, sizeof(parse_buf), "%s %s",
+			     *keyValue, *values) >= sizeof(parse_buf)) {
 			error(logopt, MODPREFIX "map entry too long");
 			ldap_value_free(values);
 			goto next;
 		}
-		memset(buf, 0, PARSE_MAX_BUF);
-
-		strcpy(buf, *keyValue);
-		strcat(buf, " ");
-		strcat(buf, *values);
-
 		master_parse_entry(buf, timeout, logging, age);
 next:
 		ldap_value_free(keyValue);
@@ -1552,7 +1547,7 @@ static int get_percent_decoded_len(const char *name)
 {
 	int escapes = 0;
 	int escaped = 0;
-	char *tmp = name;
+	const char *tmp = name;
 	int look_for_close = 0;
 
 	while (*tmp) {
@@ -2051,7 +2046,7 @@ static int do_get_entries(struct ldap_search_params *sp, struct map_source *sour
 				mapent = malloc(v_len + 1);
 				if (!mapent) {
 					char *estr;
-					estr = strerror_r(errno, buf, MAX_ERR_BUF);
+					estr = strerror_r(errno, buf, sizeof(buf));
 					logerr(MODPREFIX "malloc: %s", estr);
 					ldap_value_free_len(bvValues);
 					goto next;
@@ -2071,7 +2066,7 @@ static int do_get_entries(struct ldap_search_params *sp, struct map_source *sour
 					mapent_len = new_size;
 				} else {
 					char *estr;
-					estr = strerror_r(errno, buf, MAX_ERR_BUF);
+					estr = strerror_r(errno, buf, sizeof(buf));
 					logerr(MODPREFIX "realloc: %s", estr);
 				}
 			}
@@ -2172,7 +2167,7 @@ static int read_one_map(struct autofs_point *ap,
 
 	sp.query = alloca(l);
 	if (sp.query == NULL) {
-		char *estr = strerror_r(errno, buf, MAX_ERR_BUF);
+		char *estr = strerror_r(errno, buf, sizeof(buf));
 		logerr(MODPREFIX "malloc: %s", estr);
 		return NSS_STATUS_UNAVAIL;
 	}
@@ -2326,7 +2321,7 @@ static int lookup_one(struct autofs_point *ap,
 
 	query = alloca(l);
 	if (query == NULL) {
-		char *estr = strerror_r(errno, buf, MAX_ERR_BUF);
+		char *estr = strerror_r(errno, buf, sizeof(buf));
 		crit(ap->logopt, MODPREFIX "malloc: %s", estr);
 		if (enc_len1) {
 			free(enc_key1);
@@ -2498,7 +2493,7 @@ static int lookup_one(struct autofs_point *ap,
 				mapent = malloc(v_len + 1);
 				if (!mapent) {
 					char *estr;
-					estr = strerror_r(errno, buf, MAX_ERR_BUF);
+					estr = strerror_r(errno, buf, sizeof(buf));
 					logerr(MODPREFIX "malloc: %s", estr);
 					ldap_value_free_len(bvValues);
 					goto next;
@@ -2518,7 +2513,7 @@ static int lookup_one(struct autofs_point *ap,
 					mapent_len = new_size;
 				} else {
 					char *estr;
-					estr = strerror_r(errno, buf, MAX_ERR_BUF);
+					estr = strerror_r(errno, buf, sizeof(buf));
 					logerr(MODPREFIX "realloc: %s", estr);
 				}
 			}
author maintainer
#5 lore raw

PATCH 3/4 autofs-5.0.4 - easy alloca replacements

In reply to Ian Kent

From: Valerie Aurora Henson <[email protected]>

alloca() is compiler-dependent, non-standard, and has undefined behavior when it fails (IOW, the program crashes). Replace with normal C stack variables where possible and malloc() where not.
---

 daemon/automount.c       |   29 ++++++--------
 daemon/direct.c          |   12 ++----
 daemon/flag.c            |   13 +++---
 daemon/indirect.c        |   12 ++----
 daemon/module.c          |   45 +++++++---------------
 lib/cache.c              |   31 +++++----------
 lib/cat_path.c           |    1 
 modules/lookup_file.c    |   82 +++++++++++++----------------------------
 modules/lookup_ldap.c    |   93 +++++++++++++++++++++++++++++-----------------
 modules/lookup_nisplus.c |   71 ++++++++++++++++++++---------------
 modules/mount_autofs.c   |    1 
 modules/mount_bind.c     |    7 +--
 modules/mount_changer.c  |    5 --
 modules/mount_ext2.c     |    5 --
 modules/mount_generic.c  |    5 --
 15 files changed, 184 insertions(+), 228 deletions(-)
daemon/automount.c +13 -16
index e120f50..c167da0 100644
@@ -125,8 +125,8 @@ static int do_mkdir(const char *parent, const char *path, mode_t mode)
 
 int mkdir_path(const char *path, mode_t mode)
 {
-	char *buf = alloca(strlen(path) + 1);
-	char *parent = alloca(strlen(path) + 1);
+	char buf[PATH_MAX];
+	char parent[PATH_MAX];
 	const char *cp = path, *lcp = path;
 	char *bp = buf, *pp = parent;
 
@@ -161,7 +161,7 @@ int mkdir_path(const char *path, mode_t mode)
 int rmdir_path(struct autofs_point *ap, const char *path, dev_t dev)
 {
 	int len = strlen(path);
-	char *buf = alloca(len + 1);
+	char buf[PATH_MAX];
 	char *cp;
 	int first = 1;
 	struct stat st;
@@ -466,20 +466,17 @@ static int umount_subtree_mounts(struct autofs_point *ap, const char *path, unsi
 	pthread_cleanup_push(cache_lock_cleanup, mc);
 
 	if (me->multi) {
-		char *root, *base;
-		size_t ap_len;
+		char root[PATH_MAX];
+		char *base;
 		int cur_state;
 
-		ap_len = strlen(ap->path);
-
-		if (!strchr(me->multi->key, '/')) {
+		if (!strchr(me->multi->key, '/'))
 			/* Indirect multi-mount root */
-			root = alloca(ap_len + strlen(me->multi->key) + 2);
-			strcpy(root, ap->path);
-			strcat(root, "/");
-			strcat(root, me->multi->key);
-		} else
-			root = me->multi->key;
+			/* sprintf okay - if it's mounted, it's
+			 * PATH_MAX or less bytes */
+			sprintf(root, "%s/%s", ap->path, me->multi->key);
+		else
+			strcpy(root, me->multi->key);
 
 		if (is_mm_root)
 			base = NULL;
@@ -927,14 +924,14 @@ static int get_pkt(struct autofs_point *ap, union autofs_v5_packet_union *pkt)
 
 int do_expire(struct autofs_point *ap, const char *name, int namelen)
 {
-	char buf[PATH_MAX + 1];
+	char buf[PATH_MAX];
 	int len, ret;
 
 	if (*name != '/') {
 		len = ncat_path(buf, sizeof(buf), ap->path, name, namelen);
 	} else {
 		len = snprintf(buf, PATH_MAX, "%s", name);
-		if (len > PATH_MAX)
+		if (len >= PATH_MAX)
 			len = 0;
 	}
 
daemon/direct.c +4 -8
index c0243c4..4c81d14 100644
@@ -636,7 +636,9 @@ int mount_autofs_offset(struct autofs_point *ap, struct mapent *me, const char *
 	time_t timeout = ap->exp_timeout;
 	struct stat st;
 	int ioctlfd, status, ret;
-	const char *type, *map_name = NULL;
+	const char *hosts_map_name = "-hosts";
+	const char *map_name = hosts_map_name;
+	const char *type;
 	char mountpoint[PATH_MAX];
 
 	if (ops->version && ap->flags & MOUNT_FLAG_REMOUNT) {
@@ -739,13 +741,7 @@ int mount_autofs_offset(struct autofs_point *ap, struct mapent *me, const char *
 	      mp->options, mountpoint);
 
 	type = ap->entry->maps->type;
-	if (type && !strcmp(ap->entry->maps->type, "hosts")) {
-		char *tmp = alloca(7);
-		if (tmp) {
-			strcpy(tmp, "-hosts");
-			map_name = (const char *) tmp;
-		}
-	} else
+	if (!type || strcmp(ap->entry->maps->type, "hosts"))
 		map_name = me->mc->map->argv[0];
 
 	ret = mount(map_name, mountpoint, "autofs", MS_MGC_VAL, mp->options);
daemon/flag.c +7 -6
index e43cece..f8fe163 100644
@@ -23,10 +23,10 @@
 #include <sys/stat.h>
 #include <time.h>
 #include <string.h>
-#include <alloca.h>
 #include <stdio.h>
 #include <signal.h>
 #include <errno.h>
+#include <limits.h>
 
 #include "automount.h"
 
@@ -113,12 +113,13 @@ void release_flag_file(void)
 /* * Try to create flag file */
 int aquire_flag_file(void)
 {
-	char *linkf;
-	int len;
+	char linkf[PATH_MAX];
+	size_t len;
 
-	len = strlen(FLAG_FILE) + MAX_PIDSIZE;
-	linkf = alloca(len + 1);
-	snprintf(linkf, len, "%s.%d", FLAG_FILE, getpid());
+	len = snprintf(linkf, sizeof(linkf), "%s.%d", FLAG_FILE, getpid());
+	if (len >= sizeof(linkf))
+		/* Didn't acquire it */
+		return 0;
 
 	/*
 	 * Repeat until it was us who made the link or we find the
daemon/indirect.c +4 -8
index 9d3745c..4b7124c 100644
@@ -89,7 +89,9 @@ static int do_mount_autofs_indirect(struct autofs_point *ap, const char *root)
 	struct ioctl_ops *ops = get_ioctl_ops();
 	time_t timeout = ap->exp_timeout;
 	char *options = NULL;
-	const char *type, *map_name = NULL;
+	const char *hosts_map_name = "-hosts";
+	const char *map_name = hosts_map_name;
+	const char *type;
 	struct stat st;
 	struct mnt_list *mnts;
 	int ret;
@@ -141,13 +143,7 @@ static int do_mount_autofs_indirect(struct autofs_point *ap, const char *root)
 	}
 
 	type = ap->entry->maps->type;
-	if (type && !strcmp(ap->entry->maps->type, "hosts")) {
-		char *tmp = alloca(7);
-		if (tmp) {
-			strcpy(tmp, "-hosts");
-			map_name = (const char *) tmp;
-		}
-	} else
+	if (!type || strcmp(ap->entry->maps->type, "hosts"))
 		map_name = ap->entry->maps->argv[0];
 
 	ret = mount(map_name, root, "autofs", MS_MGC_VAL, options);
daemon/module.c +15 -30

108 lines not on this page · raw message

lib/cache.c +11 -20
index edb3192..1dfc563 100644
@@ -482,27 +482,23 @@ struct mapent *cache_lookup_offset(const char *prefix, const char *offset, int s
 {
 	struct list_head *p;
 	struct mapent *this;
-	int plen = strlen(prefix);
-	char *o_key;
+	/* Keys for direct maps may be as long as a path name */
+	char o_key[PATH_MAX];
+	/* Avoid "//" at the beginning of paths */
+	const char *path_prefix = strlen(prefix) > 1 ? prefix : "";
+	size_t size;
 
 	/* root offset duplicates "/" */
-	if (plen > 1) {
-		o_key = alloca(plen + strlen(offset) + 1);
-		strcpy(o_key, prefix);
-		strcat(o_key, offset);
-	} else {
-		o_key = alloca(strlen(offset) + 1);
-		strcpy(o_key, offset);
-	}
+	size = snprintf(o_key, sizeof(o_key), "%s%s", path_prefix, offset);
+	if (size >= sizeof(o_key))
+		return NULL;
 
 	list_for_each(p, head) {
 		this = list_entry(p, struct mapent, multi_list);
 		if (!strcmp(&this->key[start], o_key))
-			goto done;
+			return this;
 	}
-	this = NULL;
-done:
-	return this;
+	return NULL;
 }
 
 /* cache must be read locked by caller */
@@ -759,13 +755,8 @@ int cache_delete(struct mapent_cache *mc, const char *key)
 	struct mapent *me = NULL, *pred;
 	u_int32_t hashval = hash(key, mc->size);
 	int status, ret = CHE_OK;
-	char *this;
+	char this[PATH_MAX];
 
-	this = alloca(strlen(key) + 1);
-	if (!this) {
-		ret = CHE_FAIL;
-		goto done;
-	}
 	strcpy(this, key);
 
 	me = mc->hash[hashval];
lib/cat_path.c +0 -1
index 576b424..60669db 100644
@@ -12,7 +12,6 @@
  *
  * ----------------------------------------------------------------------- */
 
-#include <alloca.h>
 #include <string.h>
 #include <limits.h>
 #include <ctype.h>
modules/lookup_file.c +26 -56

127 lines not on this page · raw message

modules/lookup_ldap.c +58 -35

279 lines not on this page · raw message

modules/lookup_nisplus.c +41 -30

178 lines not on this page · raw message

modules/mount_autofs.c +0 -1
index 82a5ef3..182da4e 100644
@@ -18,7 +18,6 @@
 #include <malloc.h>
 #include <string.h>
 #include <signal.h>
-#include <alloca.h>
 #include <sys/param.h>
 #include <sys/types.h>
 #include <sys/stat.h>
modules/mount_bind.c +2 -5
index 361f0c2..b8ef581 100644
@@ -69,7 +69,7 @@ out:
 int mount_mount(struct autofs_point *ap, const char *root, const char *name, int name_len,
 		const char *what, const char *fstype, const char *options, void *context)
 {
-	char *fullpath;
+	char fullpath[PATH_MAX];
 	char buf[MAX_ERR_BUF];
 	int err;
 	int i, len;
@@ -80,14 +80,11 @@ int mount_mount(struct autofs_point *ap, const char *root, const char *name, int
 	/* Root offset of multi-mount */
 	len = strlen(root);
 	if (root[len - 1] == '/') {
-		fullpath = alloca(len);
 		len = snprintf(fullpath, len, "%s", root);
 	/* Direct mount name is absolute path so don't use root */
 	} else if (*name == '/') {
-		fullpath = alloca(len + 1);
 		len = sprintf(fullpath, "%s", root);
 	} else {
-		fullpath = alloca(len + name_len + 2);
 		len = sprintf(fullpath, "%s/%s", root, name);
 	}
 	fullpath[len] = '\0';
@@ -141,7 +138,7 @@ int mount_mount(struct autofs_point *ap, const char *root, const char *name, int
 		}
 	} else {
 		char *cp;
-		char *basepath = alloca(strlen(fullpath) + 1);
+		char basepath[PATH_MAX];
 		int status;
 		struct stat st;
 
modules/mount_changer.c +1 -4
index 92bb72b..024d244 100644
@@ -44,7 +44,7 @@ int mount_init(void **context)
 int mount_mount(struct autofs_point *ap, const char *root, const char *name, int name_len,
 		const char *what, const char *fstype, const char *options, void *context)
 {
-	char *fullpath;
+	char fullpath[PATH_MAX];
 	char buf[MAX_ERR_BUF];
 	int err;
 	int len, status, existed = 1;
@@ -57,14 +57,11 @@ int mount_mount(struct autofs_point *ap, const char *root, const char *name, int
 	/* Root offset of multi-mount */
 	len = strlen(root);
 	if (root[len - 1] == '/') {
-		fullpath = alloca(len);
 		len = snprintf(fullpath, len, "%s", root);
 	/* Direct mount name is absolute path so don't use root */
 	} else if (*name == '/') {
-		fullpath = alloca(len + 1);
 		len = sprintf(fullpath, "%s", root);
 	} else {
-		fullpath = alloca(len + name_len + 2);
 		len = sprintf(fullpath, "%s/%s", root, name);
 	}
 	fullpath[len] = '\0';
modules/mount_ext2.c +1 -4
index 192ec04..85329ab 100644
@@ -36,7 +36,7 @@ int mount_init(void **context)
 int mount_mount(struct autofs_point *ap, const char *root, const char *name, int name_len,
 		const char *what, const char *fstype, const char *options, void *context)
 {
-	char *fullpath;
+	char fullpath[PATH_MAX];
 	char buf[MAX_ERR_BUF];
 	const char *p, *p1;
 	int err, ro = 0;
@@ -49,14 +49,11 @@ int mount_mount(struct autofs_point *ap, const char *root, const char *name, int
 	/* Root offset of multi-mount */
 	len = strlen(root);
 	if (root[len - 1] == '/') {
-		fullpath = alloca(len);
 		len = snprintf(fullpath, len, "%s", root);
 	/* Direct mount name is absolute path so don't use root */
 	} else if (*name == '/') {
-		fullpath = alloca(len + 1);
 		len = sprintf(fullpath, "%s", root);
 	} else {
-		fullpath = alloca(len + name_len + 2);
 		len = sprintf(fullpath, "%s/%s", root, name);
 	}
 	fullpath[len] = '\0';
modules/mount_generic.c +1 -4
index 6d7b4b3..1dc1bfd 100644
@@ -37,7 +37,7 @@ int mount_mount(struct autofs_point *ap, const char *root, const char *name, int
 		const char *what, const char *fstype, const char *options,
 		void *context)
 {
-	char *fullpath;
+	char fullpath[PATH_MAX];
 	char buf[MAX_ERR_BUF];
 	int err;
 	int len, status, existed = 1;
@@ -48,14 +48,11 @@ int mount_mount(struct autofs_point *ap, const char *root, const char *name, int
 	/* Root offset of multi-mount */
 	len = strlen(root);
 	if (root[len - 1] == '/') {
-		fullpath = alloca(len);
 		len = snprintf(fullpath, len, "%s", root);
 	/* Direct mount name is absolute path so don't use root */
 	} else if (*name == '/') {
-		fullpath = alloca(len + 1);
 		len = sprintf(fullpath, "%s", root);
 	} else {
-		fullpath = alloca(len + name_len + 2);
 		len = sprintf(fullpath, "%s/%s", root, name);
 	}
 	fullpath[len] = '\0';
author maintainer
#6 lore raw

PATCH 4/4 autofs-5.0.4 - fix kernel includes

In reply to Ian Kent

From: Valerie Aurora Henson <[email protected]>

autofs_dev-ioctl.h is included by both the kernel module and the userland tools, and it includes two kernel include files. The compile worked if you had kernel headers installed but failed otherwise.

imk: there are a couple of other instances were we include kernel-headers includes. I've tried to fix that up.
---

 include/automount.h            |    2 +-
 include/linux/auto_dev-ioctl.h |   10 ++++++++++
 include/linux/auto_fs.h        |    5 ++++-
 3 files changed, 15 insertions(+), 2 deletions(-)
include/automount.h +1 -1
index 005d209..8a83fb4 100644
@@ -13,7 +13,7 @@
 #include <limits.h>
 #include <time.h>
 #include <syslog.h>
-#include <linux/types.h>
+#include <sys/types.h>
 #include <pthread.h>
 #include <sched.h>
 #include <errno.h>
include/linux/auto_dev-ioctl.h +10 -0
index 91a7739..8c5b71a 100644
@@ -10,8 +10,18 @@
 #ifndef _LINUX_AUTO_DEV_IOCTL_H
 #define _LINUX_AUTO_DEV_IOCTL_H
 
+/* This file is shared by both user and kernel space */
+#ifdef __KERNEL__
 #include <linux/string.h>
 #include <linux/types.h>
+#else
+#include <string.h>
+#ifndef _LINUX_AUTO_FS_H
+#include <sys/types.h>
+typedef		u_int32_t	__u32;
+typedef		u_int64_t	__u64;
+#endif
+#endif /* __KERNEL__ */
 
 #define AUTOFS_DEVICE_NAME		"autofs"
 
include/linux/auto_fs.h +4 -1
index bd39f09..5340fc6 100644
@@ -18,9 +18,12 @@
 #include <linux/fs.h>
 #include <linux/limits.h>
 #include <asm/types.h>
+#else
+typedef		u_int32_t	__u32;
+typedef		u_int64_t	__u64;
 #endif /* __KERNEL__ */
 
-#include <linux/ioctl.h>
+#include <asm/ioctl.h>
 
 /* This file describes autofs v3 */
 #define AUTOFS_PROTO_VERSION	3
#7 lore raw
In reply to Ian Kent
Quoted 3 lines · Ian Kent

I've pulled in the patches that Val posted and made a couple of minor changes. I also have what I think is just about what we need to reduce the overhead of file map scanning.

Great, thank you!

Quoted 1 line

Check'em out please folks.

I have a crumpled post-it note on my desk saying "valgrind automount." How hard would this be to do while running the Connectathon suite? It would sure make me feel better about these patches.

Quoted 5 lines

I'm a little concerned about the "fix kernel includes" patch because there should be more to it and I've altered it in line with that. I'm not sure if we should post a kernel patch first, pull in the updated kernel header files to the tar and finally make the other couple of changes then. But then I may have it wrong so check it out.

Yeah, there's a trade-off there. I think the approach you've taken is the most sensible, given that nothing makes me more insane than out of sync user and kernel code.

-VAL

author maintainer
#8 lore raw
Quoted 5 lines · Valerie Aurora Henson
Quoted 3 lines · Ian Kent

I've pulled in the patches that Val posted and made a couple of minor changes. I also have what I think is just about what we need to reduce the overhead of file map scanning.

Quoted 7 lines

Great, thank you!

Quoted 1 line

Check'em out please folks.

I have a crumpled post-it note on my desk saying "valgrind automount." How hard would this be to do while running the Connectathon suite? It would sure make me feel better about these patches.

Oh yeah, I need to get onto cleaning that up and send it over so you can have a look at it and the license statement. Sorry, I will get to it.

There may be something to be gained from running valgrind against automount and I do so from time to time. Last time I did got a bunch of leak records but I've been unable to track them down. You may have more success.

As far as Connectathon goes that shouldn't be too hard to do but you might find it's a bit painful.

The way the autofs Connectathon works is you run a setup script after setting the values you want in the configuration. I found that running it as root and opening up permissions on the export directory of the two NFS servers needed and then closing them after works best. After that the test suite assumes autofs is running so you can run "valgrind automount -f" (I think that is the only option needed) to run in foreground. We might need to change the log output code a bit, I'm not sure.

The real problems start when you hit autofs hard then valgrind becomes quite challenged. The whole thing gets really slow.

Quoted 10 lines
Quoted 6 lines

I'm a little concerned about the "fix kernel includes" patch because there should be more to it and I've altered it in line with that. I'm not sure if we should post a kernel patch first, pull in the updated kernel header files to the tar and finally make the other couple of changes then. But then I may have it wrong so check it out.

Yeah, there's a trade-off there. I think the approach you've taken is the most sensible, given that nothing makes me more insane than out of sync user and kernel code.

So I think I'll defer that patch for a little while then.

Ian