Skip to contentSkip to conversation

Issue in nfs-utils 1.2.3

DISCUSSION sdrb 3 participants 5 messages Jan 12, 2011 – Jan 13, 2011

Subsystems (2)
lore
Share & tools
lore

sdrb

author
#0 lore raw

Hello,

Recently I tried to upgrade nfs-utils to the newest nfs-utils 1.2.3. During tests I noticed that in some circumstances rpc.mountd crashes with segmentation fault.
I'm testing it with 2.6.36 linux kernel.

Configuration of nfs-server:

server# cat /etc/exports
/export *(rw)
/tmp/nfs *(rw)

The scenario how to reproduce the issue:

server# rpc.mountd -F -d all
server# showmount -a 127.0.0.1
host# umount /mnt/nfs2 ; mount -t nfs server:/tmp/nfs /mnt/nfs2 -o
nfsvers=3,nolock
server# showmount -a 127.0.0.1

and after spawning showmount for the second time I got two segmentation faults: at showmount and at rpc.mountd.

Here is output from rpc.mountd:
rpc.mountd: Received DUMP request from 127.0.0.1 rpc.mountd: Received NULL request from host
rpc.mountd: Received UMNT(/tmp/nfs) request from host rpc.mountd: authenticated unmount request from host:844 for /tmp/nfs
(/tmp/nfs)
rpc.mountd: Received NULL request from host
rpc.mountd: Received NULL request from host
rpc.mountd: Received MNT3(/tmp/nfs) request from host rpc.mountd: authenticated mount request from host:729 for /tmp/nfs
(/tmp/nfs)
rpc.mountd: nfsd_fh: inbuf '* 7
\x0ab4100000000000dd2efb04e753f0980000000000000000'
rpc.mountd: nfsd_fh: found 0x1f13380 path /tmp/nfs rpc.mountd: Received DUMP request from 127.0.0.1 Segmentation fault
.

To gather more info I run rpc.mountd in gdb:

Starting program: /usr/sbin/rpc.mountd -F

Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7b6f7a2 in xdr_string_internal () from /lib64/libc.so.6

#0 0x00007ffff7b6f7a2 in xdr_string_internal () from /lib64/libc.so.6 #1 0x0000000000409eee in xdr_name (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:83
#2 0x0000000000409ff9 in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b530) at mount_xdr.c:103

  #3  0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6
  #4  0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6
  #5  0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>, 
objp=<value optimized out>) at mount_xdr.c:93
  #6  0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0, 
objp=0x63b590) at mount_xdr.c:107
  #7  0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6
  #8  0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6
  #9  0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>, 

objp=<value optimized out>) at mount_xdr.c:93
#10 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b470) at mount_xdr.c:107
#11 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 #12 0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6 #13 0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:93
#14 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b4d0) at mount_xdr.c:107
#15 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 #16 0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6 #17 0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:93
#18 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b530) at mount_xdr.c:107
#19 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 #20 0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6 #21 0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:93
#22 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b590) at mount_xdr.c:107
#23 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 #24 0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6 #25 0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:93
#26 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b470) at mount_xdr.c:107
#27 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 #28 0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6 #29 0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:93
#30 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b4d0) at mount_xdr.c:107
#31 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 #32 0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6 #33 0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:93
#34 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b530) at mount_xdr.c:107
#35 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 #36 0x00007ffff7b70de1 in xdr_pointer () from /lib64/libc.so.6 #37 0x0000000000409ed3 in xdr_mountlist (xdrs=<value optimized out>,
objp=<value optimized out>) at mount_xdr.c:93
#38 0x000000000040a02c in xdr_mountbody (xdrs=0x63afd0,
objp=0x63b590) at mount_xdr.c:107
#39 0x00007ffff7b70e20 in xdr_reference_internal () from /lib64/libc.so.6 (...)

Seems like two procedures (xdr_mountlist and xdr_mountbody) call one another infinitely until they fill the stack completely and then segfault occures.

Is it known problem?
Maybe I misconfigured or missed something?

author
#1 lore raw
Quoted 60 lines · sdrb

Hello,

Recently I tried to upgrade nfs-utils to the newest nfs-utils 1.2.3. During tests I noticed that in some circumstances rpc.mountd crashes with segmentation fault.
I'm testing it with 2.6.36 linux kernel.

Configuration of nfs-server:

server# cat /etc/exports
/export *(rw)
/tmp/nfs *(rw)

The scenario how to reproduce the issue:

server# rpc.mountd -F -d all
server# showmount -a 127.0.0.1
host# umount /mnt/nfs2 ; mount -t nfs server:/tmp/nfs /mnt/nfs2 -o nfsvers=3,nolock
server# showmount -a 127.0.0.1

and after spawning showmount for the second time I got two segmentation
faults: at showmount and at rpc.mountd.

Here is output from rpc.mountd:
rpc.mountd: Received DUMP request from 127.0.0.1 rpc.mountd: Received NULL request from host
rpc.mountd: Received UMNT(/tmp/nfs) request from host rpc.mountd: authenticated unmount request from host:844 for /tmp/nfs (/tmp/nfs)
rpc.mountd: Received NULL request from host
rpc.mountd: Received NULL request from host
rpc.mountd: Received MNT3(/tmp/nfs) request from host rpc.mountd: authenticated mount request from host:729 for /tmp/nfs (/tmp/nfs)
rpc.mountd: nfsd_fh: inbuf '* 7 \x0ab4100000000000dd2efb04e753f0980000000000000000' rpc.mountd: nfsd_fh: found 0x1f13380 path /tmp/nfs rpc.mountd: Received DUMP request from 127.0.0.1 Segmentation fault
.

To gather more info I run rpc.mountd in gdb:

Starting program: /usr/sbin/rpc.mountd -F

Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7b6f7a2 in xdr_string_internal () from /lib64/libc.so.6

#0 0x00007ffff7b6f7a2 in xdr_string_internal () from /lib64/libc.so.6 #1 0x0000000000409eee in xdr_name (xdrs=<value optimized out>, objp=<value optimized out>) at mount_xdr.c:83
(...)

Quoted 6 lines

Seems like two procedures (xdr_mountlist and xdr_mountbody) call one another infinitely until they fill the stack completely and then segfault occures.

Is it known problem?
Maybe I misconfigured or missed something?

I've investigated a little the sources and I noticed that probably there should be some pointer NULL-ed in mountlist_list() procedure like in patch I've attached.

Anyone can confirm that such a fix is ok?

utils/mountd/rmtab.c +1 -0
diff -rNup nfs-utils-1.2.3_orig/utils/mountd/rmtab.c nfs-utils-1.2.3/utils/mountd/rmtab.c
@@ -205,6 +205,7 @@ mountlist_list(void)
 	}
 	if (stb.st_mtime != last_mtime) {
 		mountlist_freeall(mlist);
+		mlist=NULL;
 		last_mtime = stb.st_mtime;
 
 		setrmtabent("r");
#2 lore raw
Quoted 5 lines · sdrb

I've investigated a little the sources and I noticed that probably there should be some pointer NULL-ed in mountlist_list() procedure like in patch I've attached.

Anyone can confirm that such a fix is ok?

Thanks for the report.

I haven't tried to verify that it could cause the backtrace you saw, but clearly mlist is used after that mountlist_freeall(mlist), so your patch is necessary.

Looks like this was introduced with a8348c2c4 "mountd: Add mountlist_freeall()".

--b.

Quoted 11 lines

diff -rNup nfs-utils-1.2.3_orig/utils/mountd/rmtab.c nfs-utils-1.2.3/utils/mountd/rmtab.c

utils/mountd/rmtab.c +1 -0
@@ -205,6 +205,7 @@ mountlist_list(void)
 	}
 	if (stb.st_mtime != last_mtime) {
 		mountlist_freeall(mlist);
+		mlist=NULL;
 		last_mtime = stb.st_mtime;
 
 		setrmtabent("r");
#3 lore raw
Quoted 7 lines · J. Bruce Fields
Quoted 5 lines · sdrb

I've investigated a little the sources and I noticed that probably there should be some pointer NULL-ed in mountlist_list() procedure like in patch I've attached.

Anyone can confirm that such a fix is ok?

Quoted 8 lines

Thanks for the report.

I haven't tried to verify that it could cause the backtrace you saw, but clearly mlist is used after that mountlist_freeall(mlist), so your patch is necessary.

Looks like this was introduced with a8348c2c4 "mountd: Add mountlist_freeall()".

Is your theory that the introduction of a function call ( mountlist_freeall() ) hides the side-effects of that while loop, leaving the mlist variable in the mountlist_list() scope pointing at freed memory?

Quoted 10 lines

--b.

utils/mountd/rmtab.c mountlist_list(void) · line 208

Quoted 8 lines

diff -rNup nfs-utils-1.2.3_orig/utils/mountd/rmtab.c nfs-utils-1.2.3/utils/mountd/rmtab.c

utils/mountd/rmtab.c +1 -0
@@ -205,6 +205,7 @@ mountlist_list(void)
 	}
 	if (stb.st_mtime != last_mtime) {
 		mountlist_freeall(mlist);
+		mlist=NULL;

Nit: Please use white space conventions which match the rest of the file (single blanks around "=").

Quoted 4 lines
Quoted 3 lines
utils/mountd/rmtab.c +0 -0
 		last_mtime = stb.st_mtime;
 
 		setrmtabent("r");
Signature · 2 lines
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
#4 lore raw
Quoted 12 lines · Chuck Lever
Quoted 9 lines · J. Bruce Fields
Quoted 5 lines · sdrb

I've investigated a little the sources and I noticed that probably there should be some pointer NULL-ed in mountlist_list() procedure like in patch I've attached.

Anyone can confirm that such a fix is ok?

Thanks for the report.

Quoted 8 lines
Quoted 6 lines

I haven't tried to verify that it could cause the backtrace you saw, but clearly mlist is used after that mountlist_freeall(mlist), so your patch is necessary.

Looks like this was introduced with a8348c2c4 "mountd: Add mountlist_freeall()".

Is your theory that the introduction of a function call ( mountlist_freeall() ) hides the side-effects of that while loop, leaving the mlist variable in the mountlist_list() scope pointing at freed memory?

Yup.--b.

Quoted 26 lines
Quoted 11 lines

--b.

utils/mountd/rmtab.c mountlist_list(void) · line 208

Quoted 8 lines

diff -rNup nfs-utils-1.2.3_orig/utils/mountd/rmtab.c nfs-utils-1.2.3/utils/mountd/rmtab.c

utils/mountd/rmtab.c +1 -0
@@ -205,6 +205,7 @@ mountlist_list(void)
 	}
 	if (stb.st_mtime != last_mtime) {
 		mountlist_freeall(mlist);
+		mlist=NULL;

Nit: Please use white space conventions which match the rest of the file (single blanks around "=").

Quoted 4 lines
Quoted 3 lines
utils/mountd/rmtab.c +0 -0
 		last_mtime = stb.st_mtime;
 
 		setrmtabent("r");
Signature · 2 lines
Chuck Lever
chuck[dot]lever[at]oracle[dot]com