Skip to contentSkip to conversation

Limit on return string from program map

DISCUSSION Keith Owens 5 participants 10 messages Dec 16, 2004 – Dec 20, 2004 1 message not in the archive Kernel automounter

lore
Share & tools
lore
#0 lore raw

Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3). I added "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a point. ps shows

/usr/sbin/automount --timeout=60 /net program /etc/auto.net

If the target server has a lot of mounted filesystems then automount will only mount the first few entries on the list. Then it tries to run mount with invalid input. It looks like there is a hard coded limit on the string that automount expects from a program map. My virtual CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is quite long.

"/etc/auto.net virtcd" returns a long string, over 20K. The start of this string is below, some of the names have been sanitized without changing their length. irix-6.5.21-overlays-4 mounts correctly, irix-6.5.22-applications-1 does not. Instead of issuing

/bin/mount -t nfs -s -o hard,intr,nodev,nosuid virtcd:/virtcd/mnt/irix-6.5.22-applications-1 /net/virtcd/virtcd/mnt/irix-6.5.22-applications-1

it issues

/bin/mount -t nfs -s -o hard,intr,nodev,nosuid virtcd:/virtcd/mnt /net/virtcd/virtcd/mnt/irix-6.5.22-applications-1

The NFS directory string has been truncated. Interesting that the failure point is around the 4K mark on the input string.

-fstype=nfs,hard,intr,nodev,nosuid \
	/ virtcd:/ \
	/virtcd virtcd:/virtcd \
	/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-009.iso virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-009.iso \
	/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-1162-xxx.iso virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-1162-xxx.iso \
	/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-004.iso virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-004.iso \
	/virtcd/mnt/xxxxxxxxxxxxxxxx-MR-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxx-MR-1 \
	/virtcd/mnt/xxxxxxxxxxxxxxxx-RC1-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxx-RC1-1 \
	/virtcd/mnt/xxxxxxxxxxxxxxxx-RC2-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxx-RC2-1 \
	/virtcd/mnt/xxxxxxxxxxxxxx_28_05_02-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxx_28_05_02-1 \
	/virtcd/mnt/xxxxx-xxxxx-1 virtcd:/virtcd/mnt/xxxxx-xxxxx-1 \
	/virtcd/mnt/FC2-i386-disc1.iso virtcd:/virtcd/mnt/FC2-i386-disc1.iso \
	/virtcd/mnt/FC2-i386-disc2.iso virtcd:/virtcd/mnt/FC2-i386-disc2.iso \
	/virtcd/mnt/FC2-i386-disc3.iso virtcd:/virtcd/mnt/FC2-i386-disc3.iso \
	/virtcd/mnt/FC2-i386-disc4.iso virtcd:/virtcd/mnt/FC2-i386-disc4.iso \
	/virtcd/mnt/FC2-i386-rescuecd.iso virtcd:/virtcd/mnt/FC2-i386-rescuecd.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc1.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc1.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc2.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc2.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc3.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc3.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc4.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc4.iso \
	/virtcd/mnt/FC3-i386-disc1.iso virtcd:/virtcd/mnt/FC3-i386-disc1.iso \
	/virtcd/mnt/FC3-i386-disc2.iso virtcd:/virtcd/mnt/FC3-i386-disc2.iso \
	/virtcd/mnt/FC3-i386-disc3.iso virtcd:/virtcd/mnt/FC3-i386-disc3.iso \
	/virtcd/mnt/FC3-i386-disc4.iso virtcd:/virtcd/mnt/FC3-i386-disc4.iso \
	/virtcd/mnt/FC3-i386-rescuecd.iso virtcd:/virtcd/mnt/FC3-i386-rescuecd.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc1.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc1.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc2.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc2.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc3.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc3.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc4.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc4.iso \
	/virtcd/mnt/xxxxxxxx-xxxxxx-1 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-1 \
	/virtcd/mnt/xxxxxxxx-xxxxxx-2 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-2 \
	/virtcd/mnt/xxxxxxxx-xxxxxx-3 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-3 \
	/virtcd/mnt/xxxxxxxx-xxxxxx-4 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-4 \
	/virtcd/mnt/irix-6.5.18-applications-1 virtcd:/virtcd/mnt/irix-6.5.18-applications-1 \
	/virtcd/mnt/irix-6.5.18-overlays-1 virtcd:/virtcd/mnt/irix-6.5.18-overlays-1 \
	/virtcd/mnt/irix-6.5.18-overlays-2 virtcd:/virtcd/mnt/irix-6.5.18-overlays-2 \
	/virtcd/mnt/irix-6.5.18-overlays-3 virtcd:/virtcd/mnt/irix-6.5.18-overlays-3 \
	/virtcd/mnt/irix-6.5.18-overlays-4 virtcd:/virtcd/mnt/irix-6.5.18-overlays-4 \
	/virtcd/mnt/irix-6.5.19-applications-1 virtcd:/virtcd/mnt/irix-6.5.19-applications-1 \
	/virtcd/mnt/irix-6.5.19-overlays-1 virtcd:/virtcd/mnt/irix-6.5.19-overlays-1 \
	/virtcd/mnt/irix-6.5.19-overlays-2 virtcd:/virtcd/mnt/irix-6.5.19-overlays-2 \
	/virtcd/mnt/irix-6.5.19-overlays-3 virtcd:/virtcd/mnt/irix-6.5.19-overlays-3 \
	/virtcd/mnt/irix-6.5.19-overlays-4 virtcd:/virtcd/mnt/irix-6.5.19-overlays-4 \
	/virtcd/mnt/irix-6.5.20-applications-1 virtcd:/virtcd/mnt/irix-6.5.20-applications-1 \
	/virtcd/mnt/irix-6.5.20-overlays-1 virtcd:/virtcd/mnt/irix-6.5.20-overlays-1 \
	/virtcd/mnt/irix-6.5.20-overlays-2 virtcd:/virtcd/mnt/irix-6.5.20-overlays-2 \
	/virtcd/mnt/irix-6.5.20-overlays-3 virtcd:/virtcd/mnt/irix-6.5.20-overlays-3 \
	/virtcd/mnt/irix-6.5.20-overlays-4 virtcd:/virtcd/mnt/irix-6.5.20-overlays-4 \
	/virtcd/mnt/irix-6.5.21-applications-1 virtcd:/virtcd/mnt/irix-6.5.21-applications-1 \
	/virtcd/mnt/irix-6.5.21-overlays-1 virtcd:/virtcd/mnt/irix-6.5.21-overlays-1 \
	/virtcd/mnt/irix-6.5.21-overlays-2 virtcd:/virtcd/mnt/irix-6.5.21-overlays-2 \
	/virtcd/mnt/irix-6.5.21-overlays-3 virtcd:/virtcd/mnt/irix-6.5.21-overlays-3 \
	/virtcd/mnt/irix-6.5.21-overlays-4 virtcd:/virtcd/mnt/irix-6.5.21-overlays-4 \
	/virtcd/mnt/irix-6.5.22-applications-1 virtcd:/virtcd/mnt/irix-6.5.22-applications-1 \
	/virtcd/mnt/irix-6.5.22-applications-2 virtcd:/virtcd/mnt/irix-6.5.22-applications-2 \
	/virtcd/mnt/irix-6.5.22-overlays-1 virtcd:/virtcd/mnt/irix-6.5.22-overlays-1 \
	/virtcd/mnt/irix-6.5.22-overlays-2 virtcd:/virtcd/mnt/irix-6.5.22-overlays-2 \
	/virtcd/mnt/irix-6.5.22-overlays-3 virtcd:/virtcd/mnt/irix-6.5.22-overlays-3 \

	Another 16K of entries suppressed.
maintainer
#1 lore raw
Quoted 6 lines · Keith Owens

Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3). I added "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a point. ps shows

/usr/sbin/automount --timeout=60 /net program /etc/auto.net

Quoted 7 lines

If the target server has a lot of mounted filesystems then automount will only mount the first few entries on the list. Then it tries to run mount with invalid input. It looks like there is a hard coded limit on the string that automount expects from a program map. My virtual CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is quite long.

The buffer size for map entries is 4096.

Ian

#2 lore raw

==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <[email protected]> adds:

Quoted 4 lines · Keith Owens
Quoted 4 lines

Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3). I added "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a point. ps shows

Quoted 8 lines
Quoted 8 lines

/usr/sbin/automount --timeout=60 /net program /etc/auto.net

If the target server has a lot of mounted filesystems then automount will only mount the first few entries on the list. Then it tries to run mount with invalid input. It looks like there is a hard coded limit on the string that automount expects from a program map. My virtual CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is quite long.

raven> The buffer size for map entries is 4096.

I think I posted a patch here that would extend that? I can't seem to find it in the archives, so I'll repost. This is the version of the patch that is in our tree, so no guarantees about it applying to Ian's tree. *sigh* (patch originally done by from Neil Horman)

Ian, when is 4.1.4 coming out. ;-)

-Jeff

modules/lookup_program.c renamed from modules/lookup_program.c~ +49 -20
@@ -84,7 +84,7 @@ int lookup_ghost(const char *root, int g
 int lookup_mount(const char *root, const char *name, int name_len, void *context)
 {
 	struct lookup_context *ctxt = (struct lookup_context *) context;
-	char mapent[MAPENT_MAX_LEN + 1], *mapp;
+	char *mapent, *mapp, *tmp;
 	char errbuf[1024], *errp;
 	char ch;
 	int pipefd[2], epipefd[2];
@@ -94,11 +94,19 @@ int lookup_mount(const char *root, const
 	fd_set readfds, ourfds;
 	enum state { st_space, st_map, st_done } state;
 	int quoted = 0;
-	int ret;
+	int ret = 1;
 	int max_fd;
+	int distance;
+	int alloci = 1;
 
 	debug(MODPREFIX "looking up %s", name);
 
+	mapent = (char *)malloc(MAPENT_MAX_LEN + 1);
+	if (!mapent) {
+		error(MODPREFIX "malloc: %s\n", strerror(errno));
+		return 1;
+	}
+
 	/*
 	 * We don't use popen because we don't want to run /bin/sh plus we
 	 * want to send stderr to the syslog, and we don't use spawnl()
@@ -107,12 +115,12 @@ int lookup_mount(const char *root, const
 
 	if (pipe(pipefd)) {
 		error(MODPREFIX "pipe: %m");
-		return 1;
+		goto out_free;
 	}
 	if (pipe(epipefd)) {
 		close(pipefd[0]);
 		close(pipefd[1]);
-		return 1;
+		goto out_free;
 	}
 
 	f = fork();
@@ -122,7 +130,7 @@ int lookup_mount(const char *root, const
 		close(epipefd[0]);
 		close(epipefd[1]);
 		error(MODPREFIX "fork: %m");
-		return 1;
+		goto out_free;
 	} else if (f == 0) {
 		reset_signals();
 		close(pipefd[0]);
@@ -177,21 +185,40 @@ int lookup_mount(const char *root, const
 				if (!quoted && ch == '\n') {
 					*mapp = '\0';
 					state = st_done;
-				} else if (mapp - mapent < MAPENT_MAX_LEN - 1) {
-					/* 
-					 * Eat \ quoting \n, otherwise pass it
-					 * through for the parser
+				/* We overwrite up to 3 characters, so we
+				 * need to make sure we have enough room
+				 * in the buffer for this. */
+				} else if (mapp - mapent > 
+					   ((MAPENT_MAX_LEN+1) * alloci) - 3) {
+					/*
+					 * Alloc another page for map entries.
 					 */
-					if (quoted) {
-						if (ch == '\n')
-							*mapp++ = ' ';
-						else {
-							*mapp++ = '\\';
-							*mapp++ = ch;
-						}
-					} else
-						*mapp++ = ch;
+					distance = mapp - mapent;
+					tmp = realloc(mapent,
+						      ((MAPENT_MAX_LEN + 1) * 
+						       ++alloci));
+					if (!tmp) {
+						alloci--;
+						error(MODPREFIX "realloc: %s\n",
+						      strerror(errno));
+						break;
+					}
+					mapent = tmp;
+					mapp = tmp + distance;
 				}
+				/* 
+				 * Eat \ quoting \n, otherwise pass it
+				 * through for the parser
+				 */
+				if (quoted) {
+					if (ch == '\n')
+						*mapp++ = ' ';
+					else {
+						*mapp++ = '\\';
+						*mapp++ = ch;
+					}
+				} else
+					*mapp++ = ch;
 				break;
 			case st_done:
 				/* Eat characters till there's no more output */
@@ -233,18 +260,20 @@ int lookup_mount(const char *root, const
 
 	if (waitpid(f, &status, 0) != f) {
 		error(MODPREFIX "waitpid: %m");
-		return 1;
+		goto out_free;
 	}
 
 	if (mapp == mapent || !WIFEXITED(status) || WEXITSTATUS(status) != 0) {
 		error(MODPREFIX "lookup for %s failed", name);
-		return 1;
+		goto out_free;
 	}
 
 	debug(MODPREFIX "%s -> %s", name, mapent);
 
 	ret = ctxt->parse->parse_mount(root, name, name_len,
 				       mapent, ctxt->parse->context);
+out_free:
+	free(mapent);
 	return ret;
 }
#3 lore raw
Quoted 4 lines · Jeff Moyer
Quoted 2 lines
Quoted 2 lines

the string that automount expects from a program map. My virtual
CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is quite long.

raven> The buffer size for map entries is 4096.

not sure it's related: wasn't there also a very small limit on the number of loop-devices?

Thomas Jahns

Signature · 3 lines
"Computers are good at following instructions,
but not at reading your mind."
D. E. Knuth, The TeXbook, Addison-Wesley 1984, 1986, 1996, p. 9
#4 lore raw

==> Regarding Re: [autofs] Limit on return string from program map; Thomas Jahns <[email protected]> adds:

Quoted 4 lines · Jeff Moyer
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

the string that automount expects from a program map. My virtual CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is quite long.

raven> The buffer size for map entries is 4096.

Thomas.Jahns> not sure it's related: wasn't there also a very small limit Thomas.Jahns> on the number of loop-devices?

MODULE_PARM(max_loop, "i");
MODULE_PARM_DESC(max_loop, "Maximum number of loop devices (1-256)");

-Jeff

#5 lore raw
In reply to Jeff Moyer

==> Regarding Re: [autofs] Limit on return string from program map; Jeff Moyer <[email protected]> adds:

==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <[email protected]> adds:

Quoted 6 lines · Keith Owens
Quoted 6 lines
Quoted 6 lines

Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3). I added "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a point. ps shows

/usr/sbin/automount --timeout=60 /net program /etc/auto.net

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

If the target server has a lot of mounted filesystems then automount will only mount the first few entries on the list. Then it tries to run mount with invalid input. It looks like there is a hard coded limit on the string that automount expects from a program map. My virtual CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is quite long.

raven> The buffer size for map entries is 4096.

jmoyer> I think I posted a patch here that would extend that? I can't seem jmoyer> to find it in the archives, so I'll repost. This is the version of jmoyer> the patch that is in our tree, so no guarantees about it applying jmoyer> to Ian's tree. *sigh* (patch originally done by from Neil Horman)

jmoyer> Ian, when is 4.1.4 coming out. ;-)

Bah, and it was just pointed out to me that this patch adds a bug. I'll repost the patch when the bug is fixed. (see red hat bz #138994).

No ETA just yet.

-Jeff

maintainer
#6 lore raw
Quoted 25 lines · Jeff Moyer

==> Regarding Re: [autofs] Limit on return string from program map; Jeff Moyer <[email protected]> adds:

==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <[email protected]> adds:

Quoted 6 lines · Keith Owens
Quoted 6 lines
Quoted 6 lines

Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3). I added "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a point. ps shows

/usr/sbin/automount --timeout=60 /net program /etc/auto.net

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

If the target server has a lot of mounted filesystems then automount will only mount the first few entries on the list. Then it tries to run mount with invalid input. It looks like there is a hard coded limit on the string that automount expects from a program map. My virtual CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is quite long.

raven> The buffer size for map entries is 4096.

jmoyer> I think I posted a patch here that would extend that? I can't seem jmoyer> to find it in the archives, so I'll repost. This is the version of jmoyer> the patch that is in our tree, so no guarantees about it applying jmoyer> to Ian's tree. *sigh* (patch originally done by from Neil Horman)

Quoted 6 lines

jmoyer> Ian, when is 4.1.4 coming out. ;-)

Bah, and it was just pointed out to me that this patch adds a bug. I'll repost the patch when the bug is fixed. (see red hat bz #138994).

No ETA just yet.

OK.

I think I've already merge the above patch.
I'll look forward to the fix.

Ian

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw
Quoted 36 lines · Jeff Moyer

==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <[email protected]> adds:

Quoted 5 lines · Jeff Moyer
Quoted 5 lines

==> Regarding Re: [autofs] Limit on return string from program map; Jeff Moyer <[email protected]> adds:

==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <[email protected]> adds:

Quoted 6 lines · Keith Owens
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3). I added "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a point. ps shows

/usr/sbin/automount --timeout=60 /net program /etc/auto.net

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

If the target server has a lot of mounted filesystems then automount will only mount the first few entries on the list. Then it tries to run mount with invalid input. It looks like there is a hard coded limit on the string that automount expects from a program map. My virtual CD server has lots of CD images mounted over loopback, each with its own export entry, so the return string from auto.net is

quite >>> long.

raven> The buffer size for map entries is 4096.

Quoted 1 line
Quoted 1 line

jmoyer> I think I posted a patch here that would extend that? I can't seem jmoyer> to find it in the archives, so I'll repost. This is the version of jmoyer> the patch that is in our tree, so no guarantees about it applying jmoyer> to Ian's tree. *sigh* (patch originally done by from Neil Horman)

Quoted 1 line
Quoted 1 line

jmoyer> Ian, when is 4.1.4 coming out. ;-)

Quoted 4 lines
Quoted 4 lines

Bah, and it was just pointed out to me that this patch adds a bug. I'll repost the patch when the bug is fixed. (see red hat bz #138994).

No ETA just yet.

Quoted 7 lines

raven> OK.

raven> I think I've already merge the above patch. I'll look forward to raven> the fix.

New patch attached. Let me know if you need an incremental. This is as of yet untested.

Incremental would probably be better if it's not to much trouble.

There will have to be quite a bit of testing when I'm finished patching. I expect it won't even compile first off.

Ian

#9 lore raw

4.1.4?

In reply to Jeff Moyer

I'm curious too.

Quoted 3 lines

Ian, when is 4.1.4 coming out. ;-)

-Jeff

Signature · 7 lines
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rhythm & Hues
5404 Jandy Place
Los Angeles, CA 90066
Voice: 310 448-7763
Fax:   310 448-7600
[email protected]
maintainer
#10 lore raw

Re: 4.1.4?

Quoted 5 lines · Greg Bradner

I'm curious too.

Quoted 3 lines

Ian, when is 4.1.4 coming out. ;-)

-Jeff

I'm working kernel patch updates now.

I have a couple of difficult daemon patches to merge. In particular init script patches.

A couple of patches that have been submitted have not been applied, largely because of the large number of changes already added. In particular the debug info to standard out. I must point out that this is not because of problems with the patch but because of the number of changes. I don't want to add additional function until this lot has settled.

So when?

I should be able to finish over Xmass and get something out in the first or second week of Jan.

Ian