Skip to contentSkip to conversation

NFS clientaddr, kerberos

DISCUSSION James 4 participants 7 messages Jun 22, 2013 – Jun 24, 2013

Subsystems (2)
lore
Share & tools
lore

James

author
#0 lore raw

Dear NFS experts, I have a few questions:

1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.

Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?

2) In /etc/exports, for an NFSv4 export, you often see docs suggesting:
sec=sys,krb5,krb5i,krb5p OR the same but without the 'sys' part. If you instead do 'sec=krb5p' will this *force* clients to use full encryption and authentication, and deny those who try to mount without sec=krb5p ? In particular, if a client tries to mount with sec=krb5i, what should happen? For some reason I haven't seen anyone just use 'sec=krb5p' and I wanted to know what was up.

Thank you in advance,
James

#1 lore raw
Quoted 6 lines · James

Dear NFS experts, I have a few questions:

1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.

If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.

Quoted 2 lines

Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?

Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.

The server tests the provided callback arguments with a CB_NULL request (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successful CB_NULL, the server simply disables delegation for that client.

You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:

nfs.callback_tcpport=
[NFS] set the TCP port on which the NFSv4 callback channel should listen.

Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.

Quoted 7 lines

2) In /etc/exports, for an NFSv4 export, you often see docs suggesting:
sec=sys,krb5,krb5i,krb5p OR the same but without the 'sys' part. If you instead do 'sec=krb5p' will this *force* clients to use full encryption and authentication, and deny those who try to mount without sec=krb5p ? In particular, if a client tries to mount with sec=krb5i, what should happen? For some reason I haven't seen anyone just use 'sec=krb5p' and I wanted to know what was up.

If you specify "sec=krb5p" for an export, clients can mount that export only using sec=krb5p. Thus a mount command that specifies sec=krb5i should fail with EACCES.

Signature · 2 lines
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
#2 lore raw
Quoted 18 lines · Chuck Lever
Quoted 6 lines · James

Dear NFS experts, I have a few questions:

1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.

If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.

Quoted 2 lines

Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?

Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.

The server tests the provided callback arguments with a CB_NULL request (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successful CB_NULL, the server simply disables delegation for that client.

Quoted 7 lines

You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:

nfs.callback_tcpport=
[NFS] set the TCP port on which the NFSv4 callback channel should listen.

Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.

Kernel parameters cannot be per-namespace; containers don't boot a separate kernel.

Note that if you have compiled nfs as a module, you will want to do something along the lines of:

echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf

Also note that this requirement is for NFSv4 only. NFSv4.1 callbacks use the same connection as the outgoing RPC calls, and so support callbacks through NAT without requiring you to open for incoming connections.

Signature · 6 lines
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
author
#3 lore raw
Quoted 19 lines · Trond
Quoted 18 lines · Chuck Lever
Quoted 6 lines · James

Dear NFS experts, I have a few questions:

1) Concerning the NFSv4 clientaddr option, I'm curious about the technical details of why the server needs a callback address, and what to do if the client isn't directly routable? (eg: behind NAT) I am thinking of the situation with *many* clients.

If a callback path is not available, the server will not grant delegations to the client. Delegation is simply a performance optimization. Normal operation can proceed.

Quoted 2 lines

Also, what ports need to be open on the client? Does it need to respond to "NEW" traffic, or only "ESTABLISHED" or ?

Typically the client will choose a port at random. The client's callback address and port are provided to the server by the NFSv4 SETCLIENTID operation.

The server tests the provided callback arguments with a CB_NULL request (and a new TCP connection) either at mount time or when a client application first opens a file on that server. If the arguments do not result in a successful CB_NULL, the server simply disables delegation for that client.

Quoted 7 lines
Quoted 7 lines

You can fix the port the client uses, if you have a firewall in place and want to leave an open port. A kernel command-line parameter is used on the client:

nfs.callback_tcpport=
[NFS] set the TCP port on which the NFSv4 callback channel should listen.

Although, these days, it may be a per-namespace thing. A quick browse of the documentation wasn't revealing.

Thank you Chuck, Trond for your prompt replies, it really helped me have a productive weekend of hacking.

Quoted 8 lines

Kernel parameters cannot be per-namespace; containers don't boot a separate kernel.

Note that if you have compiled nfs as a module, you will want to do something along the lines of:

echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf

This is very useful to know.
Is there any way to specify this as a mount option?

Will NFSv4.1 silently ignore this and use the same RPC connection if I include it for both 4.0, and 4.1 machines?

Cheers,
James

Quoted 5 lines

Also note that this requirement is for NFSv4 only. NFSv4.1 callbacks use the same connection as the outgoing RPC calls, and so support callbacks through NAT without requiring you to open for incoming connections.

#4 lore raw
Quoted 11 lines · James
Quoted 8 lines · Trond

Kernel parameters cannot be per-namespace; containers don't boot a separate kernel.

Note that if you have compiled nfs as a module, you will want to do something along the lines of:

echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf

This is very useful to know.
Is there any way to specify this as a mount option?

No. The NFSv4 callback channel port number is a global property. It does not want to be a per-mount thing.

Quoted 2 lines

Will NFSv4.1 silently ignore this and use the same RPC connection if I include it for both 4.0, and 4.1 machines?

Yes.

Signature · 6 lines
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
#5 lore raw
Quoted 3 lines · Trond
Quoted 1 line

echo "options nfs callback_tcpport=<port>" >>/etc/modprobe.d/options-local.conf

This is very useful to know.
Is there any way to specify this as a mount option?

No. The NFSv4 callback channel port number is a global property. It does not want to be a per-mount thing.

Would it make sense as a sysctl? I can sort of imagine cases where this might be useful. Not that I'm volunteering to implement it.

#6 lore raw
Quoted 12 lines

-----Original Message-----
From: [email protected] [mailto:linux-nfs- [email protected]] On Behalf Of Jim Rees
Sent: Sunday, June 23, 2013 3:43 PM
To: Myklebust, Trond
Cc: James; Chuck Lever; [email protected]
Subject: Re: NFS clientaddr, kerberos

Quoted 2 lines · Trond
Quoted 2 lines

echo "options nfs callback_tcpport=<port>"
/etc/modprobe.d/options-local.conf

Quoted 8 lines
Quoted 2 lines

This is very useful to know.
Is there any way to specify this as a mount option?

No. The NFSv4 callback channel port number is a global property. It does not want to be a per-mount thing.

Would it make sense as a sysctl? I can sort of imagine cases where this might be useful. Not that I'm volunteering to implement it.

Hi Jim,

With the sysfs interface to kernel parameters, there is little advantage in implementing a sysctl.
Since this is a writeable kernel parameter, it means that you can also change it via the pseudofile in /sys/module/nfs/parameters/callback_tcpport after the module has been loaded (but you still need to do so before mounting any NFSv4 volumes).

Cheers
Trond