NFS clientaddr, kerberos
Subsystems (2)
Dear NFS experts, I have a few questions:
1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.
Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?
2) In /etc/exports, for an NFSv4 export, you often see docs suggesting:
sec=sys,krb5,krb5i,krb5p OR the same but without the 'sys' part. If you instead do 'sec=krb5p' will this *force* clients to use full encryption and authentication, and deny those who try to mount without sec=krb5p ? In particular, if a client tries to mount with sec=krb5i, what should happen? For some reason I haven't seen anyone just use 'sec=krb5p' and I wanted to know what was up.
Thank you in advance,
James
Quoted 6 lines · James
Dear NFS experts, I have a few questions:
1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.
If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.
Quoted 2 lines
Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?
Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.
The server tests the provided callback arguments with a CB_NULL request (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successful CB_NULL, the server simply disables delegation for that client.
You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:
nfs.callback_tcpport=
[NFS] set the TCP port on which the NFSv4 callback channel should listen.
Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.
Quoted 7 lines
2) In
/etc/exports, for an NFSv4 export, you often see docs suggesting:
sec=sys,krb5,krb5i,krb5p OR the same but without the 'sys' part. If you instead do 'sec=krb5p' will this *force* clients to use full encryption and authentication, and deny those who try to mount without sec=krb5p ? In particular, if a client tries to mount with sec=krb5i, what should happen? For some reason I haven't seen anyone just use 'sec=krb5p' and I wanted to know what was up.
If you specify "sec=krb5p" for an export, clients can mount that export only using sec=krb5p. Thus a mount command that specifies sec=krb5i should fail with EACCES.
Signature · 2 lines
Chuck Lever chuck[dot]lever[at]oracle[dot]com
Quoted 18 lines · Chuck Lever
Quoted 6 lines · James
Dear NFS experts, I have a few questions:
1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.
If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.
Quoted 2 lines
Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?
Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.
The server tests the provided callback arguments with a
CB_NULLrequest (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successfulCB_NULL, the server simply disables delegation for that client.
Quoted 7 lines
You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:
nfs.callback_tcpport=
[NFS] set the TCP port on which the NFSv4 callback channel should listen.Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.
Kernel parameters cannot be per-namespace; containers don't boot a separate kernel.
Note that if you have compiled nfs as a module, you will want to do something along the lines of:
echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
Also note that this requirement is for NFSv4 only. NFSv4.1 callbacks use the same connection as the outgoing RPC calls, and so support callbacks through NAT without requiring you to open for incoming connections.
Signature · 6 lines
Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com
Quoted 19 lines · Trond
Quoted 18 lines · Chuck Lever
Quoted 6 lines · James
Dear NFS experts, I have a few questions:
1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.
If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.
Quoted 2 lines
Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?
Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.
The server tests the provided callback arguments with a
CB_NULLrequest (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successfulCB_NULL, the server simply disables delegation for that client.
Quoted 7 lines
Quoted 7 lines
You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:
nfs.callback_tcpport=
[NFS] set the TCP port on which the NFSv4 callback channel should listen.Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.
Thank you Chuck, Trond for your prompt replies, it really helped me have a productive weekend of hacking.
Quoted 8 lines
Kernel parameters cannot be per-namespace; containers don't boot a separate kernel.
Note that if you have compiled nfs as a module, you will want to do something along the lines of:
echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
This is very useful to know.
Is there any way to specify this as a mount option?
Will NFSv4.1 silently ignore this and use the same RPC connection if I include it for both 4.0, and 4.1 machines?
Cheers,
James
Quoted 5 lines
Also note that this requirement is for NFSv4 only. NFSv4.1 callbacks use the same connection as the outgoing RPC calls, and so support callbacks through NAT without requiring you to open for incoming connections.
Quoted 11 lines · James
Quoted 8 lines · Trond
Kernel parameters cannot be per-namespace; containers don't boot a separate kernel.
Note that if you have compiled nfs as a module, you will want to do something along the lines of:
echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
This is very useful to know.
Is there any way to specify this as a mount option?
No. The NFSv4 callback channel port number is a global property. It does not want to be a per-mount thing.
Quoted 2 lines
Will NFSv4.1 silently ignore this and use the same RPC connection if I include it for both 4.0, and 4.1 machines?
Yes.
Signature · 6 lines
Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com
Quoted 3 lines · Trond
Quoted 1 line
echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf
This is very useful to know.
Is there any way to specify this as a mount option?
No. The NFSv4 callback channel port number is a global property. It does not want to be a per-mount thing.
Would it make sense as a sysctl? I can sort of imagine cases where this might be useful. Not that I'm volunteering to implement it.
Quoted 12 lines
-----Original Message-----
From: [email protected] [mailto:linux-nfs- [email protected]] On Behalf Of Jim Rees
Sent: Sunday, June 23, 2013 3:43 PM
To: Myklebust, Trond
Cc: James; Chuck Lever; [email protected]
Subject: Re: NFS clientaddr, kerberosQuoted 2 lines · Trond
Quoted 2 lines
echo "options nfs callback_tcpport=<port>"
/etc/modprobe.d/options-local.conf
Quoted 8 lines
Quoted 2 lines
This is very useful to know.
Is there any way to specify this as a mount option?No. The NFSv4 callback channel port number is a global property. It does not want to be a per-mount thing.
Would it make sense as a sysctl? I can sort of imagine cases where this might be useful. Not that I'm volunteering to implement it.
Hi Jim,
With the sysfs interface to kernel parameters, there is little advantage in implementing a sysctl.
Since this is a writeable kernel parameter, it means that you can also change it via the pseudofile in /sys/module/nfs/parameters/callback_tcpport after the module has been loaded (but you still need to do so before mounting any NFSv4 volumes).
Cheers
Trond