Skip to contentSkip to conversation

nfs-utils: multithreaded mountd fixes

PATCH In review Ben Myers 2 participants 5 messages Jan 28, 2010 – Feb 12, 2010 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

Ben Myers

author
#0 lore raw

cover nfs-utils: multithreaded mountd fixes

Hey Steve,

The following fixes a problem with mountd where you get backed up behind the rmtab lock. It may have happened that an xflock timed out and stat failed so the filedescriptor was leaked in mountlist_list and the lock never released. Skipping the fdatasync has also helped performance-wise.

Tested very lightly on TOT and more heavily on 1.0.7 where this bug was discovered.

Thanks,
Ben

---

Ben Myers (3):
nfs-utils: remove xflock timeout
nfs-utils: dont leak fd in mountlist_list
nfs-utils: don't fdatasync the rmtab

 support/nfs/cacheio.c |    5 ++---
 support/nfs/rmtab.c   |   23 ++++++++++++++++++-----
 support/nfs/xio.c     |   18 +-----------------
 utils/mountd/rmtab.c  |    5 ++++-
 4 files changed, 25 insertions(+), 26 deletions(-)

A message that is not in the archive (sent off-list or not yet received): 20100128211454.29681.24752.stgit-PhfrMOq4MEUPybYDWDrblq0bRtRcJeJQ@public.gmane.org

author
#2 lore raw

PATCH 1/3 nfs-utils: remove xflock timeout

Remove this 10 second timeout which can cause unexpected behavior and corruption in the rmtab when hit.

---
 support/nfs/xio.c |   18 +-----------------
 1 files changed, 1 insertions(+), 17 deletions(-)
support/nfs/xio.c +1 -17
index 5e2e1e9..e3d27d2 100644
@@ -44,16 +44,9 @@ xfclose(XFILE *xfp)
 	xfree(xfp);
 }
 
-static void
-doalarm(int sig)
-{
-	return;
-}
-
 int
 xflock(char *fname, char *type)
 {
-	struct sigaction sa, oldsa;
 	int		readonly = !strcmp(type, "r");
 	struct flock	fl = { readonly? F_RDLCK : F_WRLCK, SEEK_SET, 0, 0, 0 };
 	int		fd;
@@ -68,21 +61,12 @@ xflock(char *fname, char *type)
 		return -1;
 	}
 
-	sa.sa_handler = doalarm;
-	sa.sa_flags = 0;
-	sigemptyset(&sa.sa_mask);
-	sigaction(SIGALRM, &sa, &oldsa);
-	alarm(10);
 	if (fcntl(fd, F_SETLKW, &fl) < 0) {
-		alarm(0);
 		xlog(L_WARNING, "failed to lock %s: errno %d (%s)",
 				fname, errno, strerror(errno));
 		close(fd);
-		fd = 0;
-	} else {
-		alarm(0);
+		fd = -1;
 	}
-	sigaction(SIGALRM, &oldsa, NULL);
 
 	return fd;
 }
author
#3 lore raw

PATCH 2/3 nfs-utils: dont leak fd in mountlist_list

In reply to a message not in the archive

Don't leak this file descriptor if stat should fail.

---
 utils/mountd/rmtab.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
utils/mountd/rmtab.c +4 -1
index b028529..19b22ee 100644
@@ -24,6 +24,7 @@
 #include "ha-callout.h"
 
 #include <limits.h> /* PATH_MAX */
+#include <errno.h>
 
 extern int reverse_resolve;
 
@@ -187,7 +188,9 @@ mountlist_list(void)
 	if ((lockid = xflock(_PATH_RMTABLCK, "r")) < 0)
 		return NULL;
 	if (stat(_PATH_RMTAB, &stb) < 0) {
-		xlog(L_ERROR, "can't stat %s", _PATH_RMTAB);
+		xlog(L_ERROR, "can't stat %s: %s",
+				_PATH_RMTAB, strerror(errno));
+		xfunlock(lockid);
 		return NULL;
 	}
 	if (stb.st_mtime != last_mtime) {
author
#4 lore raw

PATCH 3/3 nfs-utils: don't fdatasync the rmtab

In reply to a message not in the archive

If we're using the new caching interface the rmtab will be ignored by exportfs so there is no need to fdatasync. This improves mountd performance.

---
 support/nfs/cacheio.c |    5 ++---
 support/nfs/rmtab.c   |   23 ++++++++++++++++++-----
 2 files changed, 20 insertions(+), 8 deletions(-)
support/nfs/cacheio.c +2 -3
index 6a6ed5a..bdf5d84 100644
@@ -285,9 +285,8 @@ int readline(int fd, char **buf, int *lenp)
 int
 check_new_cache(void)
 {
-	struct stat stb;
-	return	(stat("/proc/fs/nfs/filehandle", &stb) == 0) ||
-		(stat("/proc/fs/nfsd/filehandle", &stb) == 0);
+	return	(access("/proc/fs/nfs/filehandle", F_OK) == 0) ||
+		(access("/proc/fs/nfsd/filehandle", F_OK) == 0);
 }	
 
 
support/nfs/rmtab.c +18 -5
index 4cbd285..a28abf3 100644
@@ -117,11 +117,24 @@ void
 fendrmtabent(FILE *fp)
 {
 	if (fp) {
-		/* If it was written to, we really want
-		 * to flush to disk before returning
-		 */
-		fflush(fp);
-		fdatasync(fileno(fp));
+		static int have_new_cache = -1;
+		if (have_new_cache == -1) /* check only once */
+			have_new_cache = check_new_cache();
+
+		if (!have_new_cache) {
+			/*
+			 * If we are using the old caching interface: exportfs
+			 * uses the rmtab to determine what should be exported,
+			 * so it is important that it be up-to-date.
+			 *
+			 * If we are using the new caching interface: the rmtab
+			 * is ignored by exportfs and the fdatasync only serves
+			 * to slow us down.
+			 */
+			fflush(fp);
+			fdatasync(fileno(fp));
+		}
+
 		fclose(fp);
 	}
 }
#5 lore raw
In reply to a message not in the archive
Quoted 9 lines · Ben Myers

Hey Steve,

The following fixes a problem with mountd where you get backed up behind the rmtab lock. It may have happened that an xflock timed out and stat failed so the filedescriptor was leaked in mountlist_list and the lock never released. Skipping the fdatasync has also helped performance-wise.

Tested very lightly on TOT and more heavily on 1.0.7 where this bug was discovered.

Committed...

steved.