Delays on "first" access to a NFS mount

Hi,

I get a good 10 second delay anytime I am accessing my NFS mounts from a client for the first time (or after a long time not accessing them -- I suppose whenever the cache is cleared or something similar).

I usually did not bother, even though it is very annoying, but this time I collected a network protocol capture, which is attached. Notice the big delay between packet #6 and #8, while #7 should show that it is not a network issue.

I would be very glad if somebody could explain these delays.

Thanks,
Simon

No.     Time        Source                Destination           Protocol Info
      3 2.814252    192.168.110.10        192.168.110.1         TCP      675 > nfs [SYN] Seq=0 Len=0 MSS=1460 TSV=1145045 TSER=0 WS=7

Frame 3 (74 bytes on wire, 74 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 0, Len: 0

No.     Time        Source                Destination           Protocol Info
      4 2.814382    192.168.110.1         192.168.110.10        TCP      nfs > 675 [SYN, ACK] Seq=0 Ack=1 Win=92672 Len=0 MSS=1460 TSV=474034265 TSER=1145045 WS=4

Frame 4 (74 bytes on wire, 74 bytes captured)
Ethernet II, Src: ZonetTec_8c:11:73 (00:50:22:8c:11:73), Dst: AsustekC_0a:06:28 (00:0c:6e:0a:06:28) Internet Protocol, Src: 192.168.110.1 (192.168.110.1), Dst: 192.168.110.10 (192.168.110.10) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 675 (675), Seq: 0, Ack: 1, Len: 0

No.     Time        Source                Destination           Protocol Info
      5 2.814410    192.168.110.10        192.168.110.1         TCP      675 > nfs [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=1145045 TSER=474034265

Frame 5 (66 bytes on wire, 66 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 1, Ack: 1, Len: 0

No.     Time        Source                Destination           Protocol Info
      6 2.822284    192.168.110.10        192.168.110.1         NFS      V3 GETATTR Call (Reply In 8), FH:0x43fe0000

Frame 6 (210 bytes on wire, 210 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 1, Ack: 1, Len: 144 Remote Procedure Call, Type:Call XID:0x8e4655b0
Network File System, GETATTR Call FH:0x43fe0000
[Program Version: 3]
[V3 Procedure: GETATTR (1)]
object

No.     Time        Source                Destination           Protocol Info
      7 2.822400    192.168.110.1         192.168.110.10        TCP      nfs > 675 [ACK] Seq=1 Ack=145 Win=6864 Len=0 TSV=474034267 TSER=1145047

Frame 7 (66 bytes on wire, 66 bytes captured)
Ethernet II, Src: ZonetTec_8c:11:73 (00:50:22:8c:11:73), Dst: AsustekC_0a:06:28 (00:0c:6e:0a:06:28) Internet Protocol, Src: 192.168.110.1 (192.168.110.1), Dst: 192.168.110.10 (192.168.110.10) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 675 (675), Seq: 1, Ack: 145, Len: 0

No.     Time        Source                Destination           Protocol Info
      8 13.749967   192.168.110.1         192.168.110.10        NFS      V3 GETATTR Reply (Call In 6)  Directory mode:2775 uid:102 gid:1000

Frame 8 (182 bytes on wire, 182 bytes captured)
Ethernet II, Src: ZonetTec_8c:11:73 (00:50:22:8c:11:73), Dst: AsustekC_0a:06:28 (00:0c:6e:0a:06:28) Internet Protocol, Src: 192.168.110.1 (192.168.110.1), Dst: 192.168.110.10 (192.168.110.10) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 675 (675), Seq: 1, Ack: 145, Len: 116 Remote Procedure Call, Type:Reply XID:0x8e4655b0
Network File System, GETATTR Reply Directory mode:2775 uid:102 gid:1000

    [Program Version: 3]
    [V3 Procedure: GETATTR (1)]
    Status: NFS3_OK (0)
    obj_attributes  Directory mode:2775 uid:102 gid:1000
        Type: Directory (2)
        mode: 042775
        nlink: 2
        uid: 102
        gid: 1000
        size: 4096
        used: 4096
        rdev: 0,0
        fsid: 0x000000000000fe00
        fileid: 15650
        atime: Mar  7, 2007 10:31:07.000000000
        mtime: Mar  7, 2007 10:31:07.000000000
        ctime: Mar  7, 2007 10:31:07.000000000

No.     Time        Source                Destination           Protocol Info
      9 13.750012   192.168.110.10        192.168.110.1         TCP      675 > nfs [ACK] Seq=145 Ack=117 Win=5888 Len=0 TSV=1147787 TSER=474037003

Frame 9 (66 bytes on wire, 66 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 145, Ack: 117, Len: 0

No.     Time        Source                Destination           Protocol Info
     12 14.695027   192.168.110.10        192.168.110.1         NFS      V3 ACCESS Call (Reply In 14), FH:0x43fe0000

Frame 12 (214 bytes on wire, 214 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 145, Ack: 117, Len: 148 Remote Procedure Call, Type:Call XID:0x8f4655b0
Network File System, ACCESS Call FH:0x43fe0000

    [Program Version: 3]
    [V3 Procedure: ACCESS (4)]
    object
    access: 0x1f

No.     Time        Source                Destination           Protocol Info
     13 14.695157   192.168.110.1         192.168.110.10        TCP      nfs > 675 [ACK] Seq=117 Ack=293 Win=7936 Len=0 TSV=474037239 TSER=1148027

Frame 13 (66 bytes on wire, 66 bytes captured)
Ethernet II, Src: ZonetTec_8c:11:73 (00:50:22:8c:11:73), Dst: AsustekC_0a:06:28 (00:0c:6e:0a:06:28) Internet Protocol, Src: 192.168.110.1 (192.168.110.1), Dst: 192.168.110.10 (192.168.110.10) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 675 (675), Seq: 117, Ack: 293, Len: 0

No.     Time        Source                Destination           Protocol Info
     14 14.695260   192.168.110.1         192.168.110.10        NFS      V3 ACCESS Reply (Call In 12)

Frame 14 (190 bytes on wire, 190 bytes captured)
Ethernet II, Src: ZonetTec_8c:11:73 (00:50:22:8c:11:73), Dst: AsustekC_0a:06:28 (00:0c:6e:0a:06:28) Internet Protocol, Src: 192.168.110.1 (192.168.110.1), Dst: 192.168.110.10 (192.168.110.10) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 675 (675), Seq: 117, Ack: 293, Len: 124 Remote Procedure Call, Type:Reply XID:0x8f4655b0
Network File System, ACCESS Reply

    [Program Version: 3]
    [V3 Procedure: ACCESS (4)]
    Status: NFS3_OK (0)
    obj_attributes  Directory mode:2775 uid:102 gid:1000
    access: 0x1f

No.     Time        Source                Destination           Protocol Info
     15 14.695269   192.168.110.10        192.168.110.1         TCP      675 > nfs [ACK] Seq=293 Ack=241 Win=5888 Len=0 TSV=1148027 TSER=474037239

Frame 15 (66 bytes on wire, 66 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 293, Ack: 241, Len: 0

No.     Time        Source                Destination           Protocol Info
     16 14.695344   192.168.110.10        192.168.110.1         NFS      V3 READDIRPLUS Call (Reply In 17), FH:0x43fe0000

Frame 16 (234 bytes on wire, 234 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 293, Ack: 241, Len: 168 Remote Procedure Call, Type:Call XID:0x904655b0
Network File System, READDIRPLUS Call FH:0x43fe0000
[Program Version: 3]
[V3 Procedure: READDIRPLUS (17)]
dir
cookie: 0
Verifier: Opaque Data
dircount: 512
maxcount: 4096

No.     Time        Source                Destination           Protocol Info
     17 14.695604   192.168.110.1         192.168.110.10        NFS      V3 READDIRPLUS Reply (Call In 16) . ..

Frame 17 (482 bytes on wire, 482 bytes captured)
Ethernet II, Src: ZonetTec_8c:11:73 (00:50:22:8c:11:73), Dst: AsustekC_0a:06:28 (00:0c:6e:0a:06:28) Internet Protocol, Src: 192.168.110.1 (192.168.110.1), Dst: 192.168.110.10 (192.168.110.10) Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 675 (675), Seq: 241, Ack: 461, Len: 416 Remote Procedure Call, Type:Reply XID:0x904655b0
Network File System, READDIRPLUS Reply
[Program Version: 3]
[V3 Procedure: READDIRPLUS (17)]
Status: NFS3_OK (0)
dir_attributes Directory mode:2775 uid:102 gid:1000
Verifier: Opaque Data
Value Follows: Yes
Entry: name .
Value Follows: Yes
Entry: name ..
Value Follows: No
EOF: 1

No.     Time        Source                Destination           Protocol Info
     18 14.733282   192.168.110.10        192.168.110.1         TCP      675 > nfs [ACK] Seq=461 Ack=657 Win=6912 Len=0 TSV=1148037 TSER=474037239

Frame 18 (66 bytes on wire, 66 bytes captured)
Ethernet II, Src: AsustekC_0a:06:28 (00:0c:6e:0a:06:28), Dst: ZonetTec_8c:11:73 (00:50:22:8c:11:73) Internet Protocol, Src: 192.168.110.10 (192.168.110.10), Dst: 192.168.110.1 (192.168.110.1) Transmission Control Protocol, Src Port: 675 (675), Dst Port: nfs (2049), Seq: 461, Ack: 657, Len: 0

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

The delay seems to be on the server side. What is the server running? Does it have any nameservice issues? Delays on the first response can often be due to server exports checking, which usually requires reverse name lookups.

Tom.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

The server runs Debian unstable:
Linux server 2.6.18-3-k7 #1 SMP Sun Dec 10 20:17:39 UTC 2006 i686 GNU/Linux

And I use classic NFSv3, exporting only to that subnet with options (rw,sync). The client uses solely the intr option.

There shouldn't be any nameservice issues since the server is also running the nameserver and is the authority for the subnet that I am mounting from. I tried canonical and reverse lookups of the client's name/IP from that server, all without delay. How would I check for nameservice issues?

The server has some disks with exported directories that spin down after some idle time, but the disk of that particular mount point that I am using is always online. Maybe the server somehow checks all exports all the time and not just the particular requested one and thus spins up all the other disks?

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Hi again,

just verified that the server indeed spins up all disks before answering the request. I thus suspect it is somehow checking all exports whenever any one export is mounted. Is this correct behaviour?

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

That should have read "accessed", not "mounted". The export is already mounted. :)

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Hm. If you have the nfs-utils source, you can see there's a loop in

nfs-utils/utils/mountd/cache.c:nfsd_fh()

that stats the root of each export, in two places; the first it looks like you shouldn't hit if you don't have the mountpoint export option set:

	 if (exp->m_export.e_mountpoint &&
	     !is_mountpoint(exp->m_export.e_mountpoint[0]?
	                    exp->m_export.e_mountpoint:
	                    exp->m_export.e_path))
		dev_missing ++;

The second is to figure out which filesystem the filehandle that you passed in that getattr is for:

         if (stat(exp->m_export.e_path, &stb) != 0)
                 continue;
         if (fsidtype == 1 &&
             ((exp->m_export.e_flags & NFSEXP_FSID) == 0 ||
              exp->m_export.e_fsid != fsidnum))
                 continue;
         if (fsidtype != 1) {
                 if (stb.st_ino != inode)
                         continue;
                 if (major != major(stb.st_dev) ||
                     minor != minor(stb.st_dev))
                         continue;
         }
	/* It's a match !! */

You could stick a printf() in there somewhere or something to check whether this is really where it's waiting.

Could we cache the stat information in the export and then double-check it if necessary when there's a match? Or is there some way we could get the kernel to keep that cached for us?

It seems reasonable to want to export filesystems from a bunch of disks without necessarily keeping them all spun up all the time.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Correct. This one is never hit in my case.

This is where the wait for the respective disk to spin up occurs.

I could certainly cook up a patch for mountd to cache that information on its own. I don't have too much clue about how the kernel does its cacheing, though. If it's useful to do that directly in mountd, I could get my hands on it.

This is at least what I would like to see...

Thanks,
Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

OK, cool, so we understand the problem.

There's two caches involved:

- the filesystem caches attributes so that subsequent stat's of the exported directory can be answered without having to go to disk. I guess it's not suprising that that wouldn't be cached anymore if you hadn't touched the filesystem in a long time. Though there's one point I'm unclear on: are the directories you're exporting mountpoints? That's the normal configuration, and in that case I would've thought the inode for that directory would be pinned in memory so the stat wouldn't have to go to disk. I'm probably missing something.

- the filehandle->export mapping that this function tells the kernel about is cached by nfsd for a half-hour. That time is set a little later in nfsd_fh:

qword_printint(f, time(0)+30*60);

I don't think there would be any harm to just changing that time(0)+30*60 to 0x7FFFFFFF (== never expire)--nfs-utils should be invalidating that cache explicitly whenever it's needed. Maybe that should be the default.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Not all of my exported directories are mountpoints of the underlying VFS of the server. Some are, though.

I could try that for now.

Are you sure these are invalidated automatically, especially through nfs-utils? If the kernel cache never expires, it should consequently never ask for it, so nfs-utils would not be involved. Am I missing something?

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I'd be curious why. There's some hard-to-solve security problems there--people can guess filehandles of unexported files and access them directly without lookups. So some day I'd love to actually forbid (or at least strongly discourage) what you're doing.... But clearly we'd first need to understand why people do that and make sure there are adequate alternatives.

Are the spinning-up delays happening only on those drives that have exported directories that aren't mountpoints?

There's also a mechanism by which nfs-utils can ask for the whole cache to be flushed immediately on its own. So re-running exportfs to change the exports, for example, should result in the cache being flushed. I haven't checked whether that's done in all the places it should be, but it probably is.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Well, I've actually done it for security (not knowing what you just said about it). There are some directories on those disks that I don't want people to poke around in, so I don't export the whole filesystem of a disk. For some other directories, I have different access constraints.

For example, there's one subdirectory that I export to two subnets and one that is only exported to one of them. I do that because I have an "access granting" security philosophy: At first, any access is denied and then I grant access only to those people who can make use of their granted resources. Since one of those directories is only useful to the users of that one subnet, I only export it for that one.

I just notice that I was wrong. No exports are on mountpoints. I'm sorry.

Okay. So if we really only need major, minor and inode information, like Neil said, then that would work. Because otherwise the data on disk could change without the kernel noticing.

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I picked 30 minutes because it seems like a good number at the time and there where plenty of other more important things to think about. On reflection, I agree that never-expire is appropriate for the fsid->exportpoint cache (which is relevant here) and the client+exportpoint -> export options cache.
The IP->clientname cache should have an expiry time based on the TTL from the DNS (assuming that the DNS was used to do part of the mapping), but that information is not available at all easily... so 30minutes is probably as good as anything else.

Patches welcome.... :-)

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I've done so and it seems to work! Been using the changed version the whole day.

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Is there any way that we could see this change incorporated into nfs-utils? I certainly would like to have it.

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Make a diff showing that change (and nothing else), stick an explanation of the change and why it's correct at the top, and mail it to Neil cc'd to this list?....

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

This sounds like a job for inotify. The mountd could stat the export root and use inotify_add_watch(2) to keep an eye on it to see if the stat contents changed. Since the export already has a reference, it doesn't seem offhand like it would change things much, operationally. Of course, making mountd depend on an optional facility might be an issue, but it could always fall back to the current behavior.

You probably don't want to sign up for enhancing the in-kernel export cache. :-) Let's just say it's a bit mysterious, especially its interaction with mountd.

Tom.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Hm. Would it be enough just to hold an open file descriptor for the directory? Is it safe to assume that for any filesystem (uh, any disk filesystem anyway) that if you have something open then stat() on it won't have to go to the disk?

I sympathize, though this is actually one of the few mysteries of our nfs implementation that I feel like I understand, at least on alternate Thursdays.... What's bugging me a lot these days is I don't understand well enough why it's the way it is and what might need to be done to make it better.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I was actually trying to talk Simon out of trying, in case that wasn't obvious.

But I'm really glad it bugs you! Keep thinking that way, maybe you can untangle it someday. :-)

While you're thinking about it, what's the actual timeout on a given in-kernel export cache entry? There's a 120-second deadline on an unresolved cache miss being populated, but when, exactly, does an existing (resolved) entry go stale? I admit to having tried to figure it out once and wound up going in circles.

Tom.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Probably good advice.

There's an expiry time that's passed down with each cache entry. In this particular case it's 30 minutes. There's also a "flush" file you can write to to ask that the whole cache be flushed. I don't remember how this works in detail, though.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Aha - so the time comes from mountd. There's some sort of refresh timer that the kernel triggers though. So it's not a deadline of this time (I think). Or is it.

The "flush" file lives in /proc/net/rpc/nfsd.export, and you write an integer value to it. I *think* it then flushes any entries which are more than that many seconds old.

The whole thing makes my head hurt and I try not to look at it.

Tom.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

The kernel sweeps through the cache every now and then and cleans out expired entries. I think it also takes note of the earliest future expiry it runs across in the process, and uses that to decide when to check next. This is all in linux/net/sunrpc/cache.c:cache_clean().

Right.

Well, non-head-hurty ideas always welcomed. I've got two export-related problems to fix:

- Our current NFSv4 pseudofs fsid=0 hack is a pain to administer and results in inconsistent paths across different NFS versions.

- The trick of using the pseudoflavor as a client name (so doing

/export gss/krb5(rw)

instead of

/export *(sec=krb5,rw)

), is inconsistent with what other os's do, and makes it impossible to specify restrictions based both on flavor and on ip network/dns name/netgroup.

While I'm at it Trond and Christoph and others seem to be asking whether we can't make some more fundamental changes, such as:

- Maintaining a static in-kernel exports table instead of loading it on demand from mountd, and

- divorcing the exports namespace completely from any local process namespace, to the extent that you could even just say "I want to export /dev/sda7 as /usr/local/bin" without first mounting /dev/sda7 someplace.

But I really need a better idea of the requirements on the exports system. And some other examples to look at wouldn't hurt either. (Take pity on me, Linux is all I know....)

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

You've got to put that v4 pseudo root somewhere...
It just needs cleverness in nfs-utils to auto-bind-mount things into the pseudoroot... but I guess people cannot magically unmount things then.

How about this. We add an export option "follow-symlinks" so that when nfsd is asked to stat a symlink it does a 'stat' instead of an 'lstat' (effectively).
Then we get mountd to make a tmpfs in /var/lib/nfs/pseudoroot which contains directories and symlinks to the various export points names in etab. This tmpfs is exported as fsid=0,follow-symlinks.

Problem solved?

Ofcourse if different clients get to see different exports, then we might need multiple tmpfs's in /var/lib/nfs/pseudoroot/$CLIENT/ ....

Yeeesssss. If you are using crypto-security, then the source-ip-address (which is a terribly weak form of security) should be irrelevant. But I think you've convinced me that some people have valid cases for the combined tested. Grumble Grumble ;-)

Don't like that idea at all. Demand-loading is a good thing.

I like that even less. Much much less. Way way way less. Yuck.

Having a private name-space for nfsd and co might be OK, but that is the closest I could come to the above suggestion, and even then I'm not convinced. I think private name spaces are a very powerful tool that should be used very very carefully. There is plenty of room for confusion of the poor sysadmin if you start doing too much with private name spaces.

If you built a system where every daemon has a private namespace, then having one for nfsd would be ok, but it really should be a system wide approach to management, not an approach only used by nfsd.

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Depending on why you need it, this may be inconsistent with the nfsv4 pseudofilesystem construction; we need to be able to service readdir on the pseudofilesystem root, for example, which means knowing at least the list of paths.

So could you remind me what the uses cases are here? Who is it that requires demand loading, and why?

I'll promise to write it all down someplace and then hopefully we won't have to re-ask the same questions too many times....

This is Christoph's suggestion, so I'm cc:'ing him.

I can believe that it would be convenient for sysadmin's to be able to export filesystems at paths other than the paths they're locally mounted at. And once you're going to allow that, it seems cleanest just to let the exports tree be a purely in-kernel thing. But this also doesn't seem like a hard requirement to me.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Partly it is the principle that demand-based configuration is more flexible. Witness the various efforts to replace rc.d scripts with something event/demand based.

The IP->clientname table must be demand loaded because you obviously cannot know all needed IP addresses in advance. (The rmtab experience proves that)

The clientname+path->export-options table must be demand loaded because - depending a bit of how you choose client names and how complicated /etc/exports is - you either don't know all client names in advance, or computing them all is complex and wasteful.

The fsid->path table could possible be made 'static', but I think demand-loading is still best. There are multiple possible fsids for some filesystems, and telling the kernel about all of them when only one will be used seems wasteful. And the filesystems may not all be available when you try to create the static table. You could update the table at every mount, but with demand-loading, you don't have to.

Imagine having hundreds of filesystems on some sort of library (a CD library?) where each can be identified by a UUID which gets stored in the fsid in the filehandle.
Imagine a simple extension to mountd so that a call-out were made when an unknown filehandle arrived. This callout could mount the required filesystem and export it. Maybe the library only allows 3 filesystems to be mounted at a time, so it would unmount the lease-recently-used one.

How are you going to handle that system except with demand-loading of the fsid->path table?

Sounds like a fine idea.
I have often wanted to write a 'Linux commentary' that explains all the hows and whys of things. I even started some bits once (to help me understand the VFS layer). But Linux changes so fast that any entry in such a commentary would be out-of-date before it was written....

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Maybe. Is this practical? Do we know of any cases of users doing this? Do you block forever if you try to access 4 filesystems at once? I dunno....

And there's a lot to document. I mean, look:

http://www.oreilly.com/catalog/understandlni/

Just over a thousand pages, just covering the kernel networking code (and only some of it at that). Maybe they just lost the forest for the trees, but still, yipes.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Someone once mentioned doing something vaguely like this... I don't know the exact details. It was mentioned in the context of the fsid being too small to do a really good job.

I wonder if you can export autofs to get them mounted in the first place.....

round-robin? Sending lots of EINPROGRESS for one filesystem while the others have a turn.

Is there really a market for that sort of book? Maybe we should write "Understanding Linux NFS" in our spare time .... oh wait, we don't have any spare time. Maybe in a couple of decades when I retire:-)

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

IIRC SGI had a storage appliance a while back that included a tape robot, but it was hiding the details somewhere deep inside XFS. I remember seeing patches involving nfsd and dmapi (I can see you cringe, Christoph :-)

Note that in real-life scenarios, we're sometimes talking about literally thousands of exported file systems. My previous employer has a customer with such a setup, using NetApp filers. We had some trouble getting the Linux client to survive in this environment, as it ran out of privileged ports way too quickly. Absurd as it may sound, this kind of setup seems to be the trend.

Now think about handling a system with several thousand exported file systems on the server side - if you need to look at each file system before nfsd is ready to service requests, we're talking of a considerable delay in boot time. In the worst case we're talking about several thousand *disks* that need to be spun up, and fuses going pop-pop-pop.

Short summary - if you want to scale beyond small work group servers, you need something that scales well. Demand loading the exports table does.

Olaf

Olaf Kirch  |  --- o --- Nous sommes du soleil we love when we play
[email protected] |    / | \   sol.dhoop.naytheet.ah kin.ir.samse.qurax

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

There's some confusion here--the reason that this was happening was that we're mapping filehandles to exports by stat()ing the root of every exported filesystem. That may be an obstacle to handling large numbers of exports, but it's not really related to the demand-loading question.

So why does demand-loading scale better? Is the worry just the kernel memory required to store the export table for thousands of mostly inactive exports?

So you need the mountpoints for the exported filesystem, the export options, and the name of the client(s). If that adds up to a few K, thousands would add up to a few Megs.

Are there other problems? (E.g. does the VFS handle thousands of mounts well?)

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Sorry if I was expressing myself poorly. What I was driving at was that it makes sense to not mount all file systems prior to starting the NFS server.

It means you can start serving files without having to wait for all file systems to be mounted (and having their journals replayed, etc). All you need is a way for mountd to figure out whether a file system is there already (so we can push the rootfh into the kernel) or whether it's not (so nfsd can return EJUKEBOX or defer the request)

Olaf

Olaf Kirch  |  --- o --- Nous sommes du soleil we love when we play
[email protected] |    / | \   sol.dhoop.naytheet.ah kin.ir.samse.qurax

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

First I'd like to apologize for jumping in a little late, but I have been very busy at work the last weeks.

Then I'll try to put my repsones to most concerns here in one length mail because there's a lot of different things talked about here that are are all interwinded.

There's a lot of things about exporting handling that can be static vs ondemand, kernel vs userspace etc.

- with NFSv4 there's a clear pseudo-filesystem structure where that really should be represented using the kernel mount tables, or to speak be a namespace reusing the infrastructure we have
- for NFSv2/3 that's not as important, but when we have to have this namespace magic for NFSv4 anyway, it would be nice to reuse it for NFSv3 as far as possible, e.g. serving older NFS client from the same namespace as NFSv4 clients, even if only the actually defined mountpoints without the pseudo root instead of the full hiearchy that is visible
- once we're talking about mounting filesystems into specific places we really should reuse all the existing excellent kernel code to deal with this. Given that for NFSv4 we most likely want to represent a different tree that the normal local filesystem views that means a separate namespace. Note that we should have a way for the administrator to easily see and modify this namespace to not cause all too much trouble.
- using namespace and the kernel mount code means we need to have at least enough information on where these mount points are in kernel space.
- it does however not mean that we need to set up all these mount points at boot time! We have a nice scheme to creates mounts when a special follow_link operation that is for example used in the nfs client for submount, or we could have some sort of automounter that does userspace upcalls.

Does this give a better view of where I want to go?

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Note that clients may access filesystems by traversing mountpoints, but they may also just jump in with a filehandle they got on a previous boot. So we still need to be able to look up exports by filehandle as well as by path.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Yeah, but an explicit kern_mount is easy. For old dev_t based filehandles anyway, and for uuid or fsid based onces we'd need an upcall for a uuid/fsid to dev_t mapping.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I've got make-mountd-clever patches here; if I can get them working in the next couple days then I'll pass them along so people can see what they're doing.

Maybe. Getting those symlink/mountpoints right sounds tricky.

How different is this from the in-kernel automounting that the NFS client is using for fsid traversal, for example?

Ugh. Is this something a lot of people do?

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Cool.

Is it?
mount -t tmpfs tmpfs /var/lib/nfs/pseudoroot
grep '^/' /etc/exports | while read a b
do
d=dirname $a
mkdir -p /var/lib/nfs/pseudoroot/$d
ln -s $a /var/lib/nfs/pseudoroot/$a
done
(untested, and probably has some corner cases but the essence is
there).

Well, if you export a different root filesystem to each diskless client, or if you export /home to some places and /backup to others, then you already have the potential for a different pseudo filesystem for each client. My little hacky shell script above essentially merges them all which might be OK, or might not.

Suppose you wanted to allow every diskless client to see it's root as '/'? Is that a dumb thing to do, or just a difficult thing to do?

I was just looking at the RFC again and saw this in section 7.3:

Based on the construction of the server's name space, it is possible that multiple pseudo filesystems may exist. For example,

   /a         pseudo filesystem
   /a/b       real filesystem
   /a/b/c     pseudo filesystem
   /a/b/c/d   real filesystem

Each of the pseudo filesystems are considered separate entities and therefore will have a unique fsid.

That adds a whole new dimension of complexity..... do we really want to go there?

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I think we want a single export tree, parts of which clients may or may not have permission to see, rather than try to support per-client namespaces. If nothing else, secinfo becomes a bit weird otherwise (do you tell the client that it can use auth_unix security for this filesystem, if actually using auth_unix might result in the client seeing a *different* filesystem?).

Hah.

We certainly aren't required to support every weird configuration that the RFC allows.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Well, the original implementation did just that, and people kept forgetting to re-run exportfs after changing the exports table, and whatnot. Lots of gross inconsistencies. The addition of a dynamic exports table was considered a sliced bread kind of innovation... so it does feel like time warp when we talk about a static export table now.

Is that really a desirable goal? From an admin's point of view, file names are usually more "natural" than using fs uuids or retro stuff such as device file names (the udev people would actually hit you with their "device numbers are smoke and mirrors" bat now).

Users actually want things like "I export /mnt and then clients can see the contents of the CD mounted on /mnt/cdrom"

Olaf

Olaf Kirch  |  --- o --- Nous sommes du soleil we love when we play
[email protected] |    / | \   sol.dhoop.naytheet.ah kin.ir.samse.qurax

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

What exactly changed? It's not the case today that you can expect modifications to /etc/exports to be noticed automatically.

--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

You write a timestamp in seconds-since-epoch and any entries older that time are treated as expired. We typically write out the mtime of the etab file. Or '1' to force a full flush.

Arguably that sort of fine control isn't needed, but it seemed like a good idea at the time (it cause problems if your system clock goes backwards).

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I think what Tom had in mind was to stat all directories once, remember their values, have inotify keep an eye on 'em and whenever they change, update the remembered values. This way, disk access would only have to be done whenever something changes, which is when the disk is spun up anyway.

Simon

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

There is certainly some sense in that approach. I don't think inotify is needed though. The only part of the stat information we are interested in is major/minor/inode numbers, and they don't change.

We could possibly stat everything that seems to be interesting once and store the details.
When a request comes in, match against the records to find a path, then double-check the path still matches. If it does, good. If not, take the long way around again.

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Actually I thought it would important to also look at the permission bits and/or any acls, in case access were revoked e.g mode 0. The export needs to be invalidated, basically as mountd would have found if exporting from scratch. Wouldn't it?

Come to think of it, maybe not. If the export goes bad then the client gets ESTALE, that's different from revoking access or never exporting. In that case I think mountd just needs to be sure the export point doesn't go away due to unmount, or is covered by a new one.

Tom.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

No. The mode/acl on anything isn't interesting to mountd. If the client has access to a file, it gets access, if not: not. That is all handled by nfsd.
And remember that if you "chmod 0" a directory, that doesn't remove access from people with files in the directory already open.

Mounting on top of an export point is an odd case that is not explicitly handled. Currently if you do that, the clients won't notice until after 32 minutes of inactivity.
I'm not sure it is a case that is worth any effort to deal with 'sensibly'.

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Well, if mountd isn't running as root and the export point is mode 0, then it can't be exported because the daemon can't stat it, right?

Corner case, I guess, maybe mountd can't be non-root. Agreed on the revoke-when-open, of course.

Tom.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Trouble with holding an open file descriptor is that sometimes people want to unmount their export filesystems, and an open file descriptor will stop that.
Currently you can "exportfs -f" and unmount/remount (as long as no client is actively using it.

NeilBrown

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Yeah, OK. I was thinking of this as an advantage rather than a disadvantage, to be perfectly honest. But no doubt they have some good reason....

Right, OK.--b.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs