Questions and problems with NFS4

Hi all...

I'm beginning to move my boxen to use nfs4, and have some questions...

First of all, versions of my software: kernel 2.6.33.5, nfs-utils 1.2.2 (perhaps some of my doubts are solved in newer versions)

Questions:

- Is it mandatory to use a setup with a common root for exported filesystems ? In my setup I first did not use it, no fsid=0 root, and everything worked fine, just change the mount type in the clients. In fact, new nfs-utils just try first version 4 and you dont have to do anything. You loose the ability to use simple paths inside /export, for example, and have to use full paths like in V3, but that is not so bad. Am I loosing something else wrt functionality ?

- If you have a common root, is it mandatory to mount it in the client ?

- If I setup a common root (say /export), and aoutomount some folders:

/scratch/local
/scratch/node/00
/scratch/node/01
......
/export/scratch -> /scratch/local (bind mount).

/scratch/node/?? is automounted from node ??, in which it is its own /scratch/local. When I acess scratch for local node, it backs to a local mount and mounts the REAL /scratch, not /export/scratch.
Any ideas ?

- AFAIK, with NFS4 the only needed daemons are nfsd and idmapd. And the only accesible port from the outside is 2049, for nfsd.
I have tried to strip down my nfs server (-N 2 -N 3 -U), but rpcinfo still gives me:

annwn:~# rpcinfo -p localhost
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  48461  status
    100024    1   tcp  37515  status
    100021    1   udp  38583  nlockmgr
    100021    3   udp  38583  nlockmgr
    100021    4   udp  38583  nlockmgr
    100021    1   tcp  37873  nlockmgr
    100021    3   tcp  37873  nlockmgr
    100021    4   tcp  37873  nlockmgr
    100003    4   tcp   2049  nfs
    100005    1   udp  45341  mountd
    100005    1   tcp  58639  mountd

disabling portampper and mountd is just a matter of initscripts requirements, but how can I disable nlockmgr ? It isn't needed for NFS4, isn't it ? Nor portmapper nor mountd...

TIA

--
J.A. Magallon <jamagallon()ono!com> \ Software is like sex:
\ It's better when it's free

I'm not sure i have understand your means, is it like this?

# cat /etc/exports
/nfsroot *(fsid=0,insecure,no_root_squash,rw)

# mkdir -p /nfsroot/nfsroot
# touch /nfsroot/tstfile
# service nfs restart
# mount -t nfs 127.0.0.1:/nfsroot /mnt

# ls /mnt <== Here, we access real /nfsroot/nfsroot, not /nfsroot,
so we can not find "tstfile", right?

Actually, when we use "-t nfs", nfs-utils try to mount NFSv4 first, if you want use NFSv3, you should use "-t nfs -o vers=3" in the nfs-utils-1.2.2.

    ... snip ...

As i see, there's no way to disabe nlockmgr, it's start by lockd, and lockd will be up when we start nfsd. It's always work until the last nfsd thread stop.

Best Regards
 Bian Naimeng

Some day we should probably modify the server to turn off lockd in the v4-only case. For now we don't do that.

It should be fine to firewall off lockd if you only care about NFSv4.

--b.

Strictly speaking, portmapper is not required for NFSv4 service. However, the NFS infrastructure on Linux is still designed for NFSv2 and v3. There remains some work needed to make portmapper optional for a v4-only server. For now, continue to run it in order to handle kernel upcalls.

rpc.mountd is, however, still required on Linux NFSv4 servers. Although NFSv4 clients do not contact the server's mountd, the kernel's NFS server performs upcalls to rpc.mountd to manage export information. You can firewall off the mountd service on the server without affecting NFSv4 clients. Recent versions of rpc.mountd accept command line options that disable the mountd network service while still handling kernel upcalls.

And, as long as lockd is running, you will need to keep rpc.statd around. Again, you can firewall this service so that it is not exposed on the network, but it must continue to be available to handle kernel upcalls. This is something we hope to address eventually as part of the lockd work Bruce mentioned.

Err...

First of all, plz correct me if this is in some kind of wiki, web page or the like, but I have not been able to find it. All this things should be on a document somewhere, perhaps even in the kernel itself... These are the things I have found out (thanks to your answers...). I'm a long time unix admin, not new to NFS, but all this things were not obvious to me, so perhaps they deserve to be in a document, something like "NFS 3 to 4 migration for dummy admins":

- Everything just works if you mount shares as nfs4. Even, if you have a bunch of linux boxen with recent kernel/nfs-utils, probably you are already doing nfs4... modern mount tries nfs4 first.

- Using nfsroot with fsid=0 is not mandatory, nor bind-mounting everything under some /export (like many documents say), that only forces you to use the old way of specifying paths in the server (absolute, not relative to /export).

- Even if you use nfsroot(fsid=0), and you mount it on the client at /somepoint, you are not forced to mount everything else under /somepoint (plz, correct me if I'm wrong).

- You can strip your server for NFS4, but not too much... Old daemons are still needed locally. For example, you can get rid of NFS2 and UDP for nfsd (-N 2 -U, -any os still uses NFS2??-), and old mount protocols (-N 1 -N 2 for mountd)...

- ... but they can be firewalled, use is just local

- portmap/rpcbind is not needed, but still used because nfsd is not yet proper clean for only-nfs4-behavior.

There are also some things I have not been able to discover, like using the interesting things of NFS4:

- How do you activate delegations ? Is this an automatic thing, or do I have to add any option somewhere ?

- How do you use cache ? Many docs talk about fsc option, but man does not mention it (nfs-utils 1.2.2).
I have tried to use cachefiles module, and cachefilesd, but when I try to run it, I get:

bran:~/c/cachefilesd-0.10# ./cachefilesd -dddns -f $PWD/cachefilesd.conf
About to bind cache
CacheFiles bind failed: errno 95 (Operation not supported)

In dmesg:
[cachef] ==> cachefiles_get_security_ID({system_u:system_r:cachefiles_kernel_t:s0})
CacheFiles: Security denies permission to nominate security context: error -95

I can not run cachefiles without SELinux or something like that ?

Thanks, perhaps this notes help someone in the future.

TIA

J.A. Magallon <jamagallon()ono!com>     \               Software is like sex:
                                         \         It's better when it's free

Agreed.

We should update that documentation.

It's automatic. You can turn them off on the server side with /proc/sys/fs/leases-enable. They'll be turned off automatically if the server can't open a tcp connection back to the client (for example, if the client is behind a NAT).

I haven't used that. I don't believe there's anything NFSv4-specific about it.

--b.