NFSv4 and nfsroot

I've set up nfsroot based systems before but I am now attempting to do the same thing using nfsv4 using the _in-kernel_ nfsroot support (ie: no initrd with script that does the mounting). I have been unable to find "authoritative" references to how to accomplish this and wanted to ask for solid references before I start polluting the mailing list with my probably completely erroneous approach. Things that might be worth mentioning:

Some basic information, here is a sequence where I mount the nfsv4 export onto the same machine exporting the tree:

/etc/exports, using FSID=0/root:

livecd ~ # exportfs -v
/tftproot/nfsroot/x86_64

10.0.0.0/255.255.255.0(ro,async,wdelay,no_root_squash,no_subtree_check,fsid=0)

Simple call to mount won't work, I'm assuming mount defaults to nfsv3:

livecd ~ # mount 10.0.0.1:/ /mnt/gentoo/ mount.nfs: access denied by server while mounting 10.0.0.1:/

Specifying nfs4 works:

livecd ~ # mount -t nfs4 10.0.0.1:/ /mnt/gentoo/

Attempts to use the nfsv3 syntax fails (as expected since I put --no-nfs-version 3 --no-nfs-version 2 as a startup arg for nfsd):

livecd ~ # mount -t nfs4 10.0.0.1:/tftproot/nfsroot/x86_64 /mnt/cdrom/ mount.nfs4: mounting 10.0.0.1:/tftproot/nfsroot/x86_64 failed, reason given by server:
No such file or directory

Booting the diskless node, I get the following:

1- If the args are (v3 syntax, in pxelinux.cfg/default):

     append ip=dhcp nfsroot=10.0.0.1:/tftproot/nfsroot/x86_64,hard,intr

init=/boot/stateless.sh

The node boots up to the point of calling init and panics because it can't find "/boot/stateless.sh" (screen shot: http://wiki.neuralbs.com/~kyron/soc2008/screenshoots/NFSv4WithV3CommandLine_no_init.png ) But stateless is there and accessible (from the earlyer local mounting of the root):

livecd ~ # ls -l /mnt/gentoo/boot/stateless.sh -rwxr-xr-x 1 4294967294 4294967294 3964 Jul 11 04:22 /mnt/gentoo/boot/stateless.sh

2- If the args are (v4 syntax, in pxelinux.cfg/default):

     append ip=dhcp nfsroot=10.0.0.1:/,hard,intr init=/boot/stateless.sh

The node boots up but panics on mounting root (screen shot : http://wiki.neuralbs.com/~kyron/soc2008/screenshoots/NFSv4PanicOnMount.png )

What can be seen in the logs:

livecd ~ # egrep -i '(nfs|mount)' /var/log/messages

   Jul 16 01:37:23 livecd Mount-cache hash table entries: 256
   Jul 16 01:37:36 livecd rc-scripts: ERROR:  cannot start nfsmount as
   rpc.statd could not start
   Jul 16 01:52:00 livecd aufs test_add:377:mount[27642]: uid/gid/perm
   /mnt/livecd/tftproot 0/0/0755, 0/0/01777
   Jul 16 01:52:11 livecd Installing knfsd (copyright (C) 1996
   [email protected]).
   Jul 16 01:52:12 livecd NFSD: Using /var/lib/nfs/v4recovery as the
   NFSv4 state recovery directory
   Jul 16 01:52:12 livecd NFSD: starting 90-second grace period
   Jul 16 02:05:17 livecd mountd[28146]: Caught signal 15,
   un-registering and exiting.
   Jul 16 02:05:17 livecd nfsd: last server has exited
   Jul 16 02:05:17 livecd nfsd: unexporting all filesystems
   Jul 16 02:05:17 livecd NFSD: Using /var/lib/nfs/v4recovery as the
   NFSv4 state recovery directory
   Jul 16 02:05:17 livecd NFSD: starting 90-second grace period
   Jul 16 02:05:40 livecd mountd[28441]: refused mount request from
   10.0.0.1 for / (/): not exported
   Jul 16 02:05:49 livecd nfsd: nfsv4 idmapping failing: has idmapd not
   been started?
   Jul 16 02:12:52 livecd dnsmasq[27954]: TFTP sent
   /tftproot/nfsroot/x86_64/boot/kernel to 10.0.0.55
   Jul 16 02:13:10 livecd mountd[28441]: authenticated mount request
   from 10.0.0.55:955 for /tftproot/nfsroot/x86_64
   (/tftproot/nfsroot/x86_64)
   Jul 16 02:13:10 livecd aufs aufs_encode_fh:529:nfsd[28447]: NFSv2
   client (max_len 6)?
   Jul 16 02:14:44 livecd dnsmasq[27954]: TFTP sent
   /tftproot/nfsroot/x86_64/boot/kernel to 10.0.0.55
   Jul 16 02:15:01 livecd mountd[28441]: authenticated mount request
   from 10.0.0.55:682 for /tftproot/nfsroot/x86_64
   (/tftproot/nfsroot/x86_64)
   Jul 16 02:24:55 livecd dnsmasq[28619]: TFTP sent
   /tftproot/nfsroot/x86_64/boot/kernel to 10.0.0.55
   Jul 16 02:25:12 livecd mountd[28441]: refused mount request from
   10.0.0.55 for / (/): not exported

NFS-booting-node's kernel config (nfs stuff), notice v3 client is disabled:

kyron clustering-livecd # grep -i nfs /tmp/nfsboot_kernel/gentoo/usr/src/linux-2.6.26-rc5-mm3/.config CONFIG_NFS_FS=y
# CONFIG_NFS_V3 is not set
CONFIG_NFS_V4=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not set
CONFIG_NFSD_V4=y
CONFIG_ROOT_NFS=y
CONFIG_NFS_COMMON=y

The server's kernel config (nfs):

kyron clustering-livecd # grep -i nfs /var/tmp/catalyst/kerncache/default/livecd-stage2-amd64-2008.0/gentoo/usr/src/linux-2.6.24-gentoo-r8/.config CONFIG_NFS_FS=y
# CONFIG_NFS_V3 is not set
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not set
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_ROOT_NFS=y
CONFIG_NFS_COMMON=y

...so, it seems kernel boot time nfs is not NFSv4 compatible (only v2/3 is supported _in kernel_).

Eric

nfsroot for NFSv4 is not supported. You can still do it by using an initrd file in order to bootstrap the necessary user-space utilities like the idmapper (and possibly rpc.gssd) before mounting the partition and pivoting.

Cheers
Trond