Side effects of having NFSv4 mounted over udp
Subsystems (2)
Hi,
I know NFSv4 is definitely a connection oriented protocol, but noting prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead of UDP could be of great interest, especially in a HPC context where thousands of clients will operate at the same time. If a nfs mount point is made over udp, what would be the side effects ? I tried to run the connectathon on such a udp based mount point and I could find no errors. Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ?
Thanks in advance for your answer.
Regards
Philippe
Quoted 3 lines · DENIEL Philippe
Hi,
I know NFSv4 is definitely a connection oriented protocol, but noting prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead of UDP could be of great interest, especially in a HPC context where thousands of clients will operate at the same time. If a nfs mount point is made over udp, what would be the side effects ? I tried to run the connectathon on such a udp based mount point and I could find no errors. Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ?
You might find that some implementations have UDP disabled and it might then make it hard to triage what is going on.
I believe both NetApp's OnTap and Oracle's Solaris have it turned off.
Quoted 10 lines
Thanks in advance for your answer.
Regards
Philippe
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 5 lines · Thomas Haynes
Quoted 2 lines · DENIEL Philippe
Hi,
Quoted 7 lines
Quoted 1 line
I know NFSv4 is definitely a connection oriented protocol, but noting prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead of UDP could be of great interest, especially in a HPC context where thousands of clients will operate at the same time. If a nfs mount point is made over udp, what would be the side effects ? I tried to run the connectathon on such a udp based mount point and I could find no errors. Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ?
You might find that some implementations have UDP disabled and it might then make it hard to triage what is going on.
I believe both NetApp's OnTap and Oracle's Solaris have it turned off.
The Linux server allows it, but I've been considering that a (low-priority) bug, so it wouldn't be safe to assume it will continue working.
That aside, if you have a perfect network, NFSv4.0 at least will probably work. (Not 4.1 since backchannel setup will fail?)
Are you really sure that you can't make tcp scale to thousands of clients?
--b.
Quoted 17 lines
Quoted 5 lines
Thanks in advance for your answer.
Quoted 7 lines
Regards
Philippe
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 18 lines · J. Bruce Fields
Quoted 5 lines · Thomas Haynes
Quoted 2 lines · DENIEL Philippe
Hi,
Quoted 7 lines
Quoted 1 line
I know NFSv4 is definitely a connection oriented protocol, but noting prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead of UDP could be of great interest, especially in a HPC context where thousands of clients will operate at the same time. If a nfs mount point is made over udp, what would be the side effects ? I tried to run the connectathon on such a udp based mount point and I could find no errors. Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ?
You might find that some implementations have UDP disabled and it might then make it hard to triage what is going on.
I believe both NetApp's OnTap and Oracle's Solaris have it turned off.
The Linux server allows it, but I've been considering that a (low-priority) bug, so it wouldn't be safe to assume it will continue working.
Quoted 5 lines
That aside, if you have a perfect network, NFSv4.0 at least will probably work. (Not 4.1 since backchannel setup will fail?)
Are you really sure that you can't make tcp scale to thousands of clients?
Maybe SCTP would be the way to go then? We've often talked about it over the years, but nobody has yet (to my knowledge) written an ONC RPC transport for it.
Cheers
Trond
Signature · 6 lines
Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com
Hi
Quoted 4 lines
The Linux server allows it, but I've been considering that a (low-priority) bug, so it wouldn't be safe to assume it will continue working.
I am mostly thinking at the client side using as well udp and tcp. Would the "NFSv4 client over UDP" behaves differently then the same client over TCP ?
Quoted 1 line
That aside, if you have a perfect network,
I can trust my network. It's not a WAN, it's located into a very massive cluster (it's kind of "internal LAN"). It's a very high throughput network (IB based) so I believe there are less "hardware based reason" to loose packets.
Quoted 2 lines
NFSv4.0 at least will
probably work.
OK.
Quoted 2 lines
(Not 4.1 since backchannel setup will fail?)
What is erroneous in using UDP for NFSv4.1 backchannels ?
Quoted 3 lines
Are you really sure that you can't make tcp scale to thousands of clients?
I am a bit afraid of a "No more file descriptors" effect. If I have one TCP socket per client and thousands of clients, I have less remaining fds for other purposes. Another point : UDP is a "cheap" protocol. I can have bunches of clients without overloading the server (a new client will almost cost nothing to the server, just the cost of a new clientid negotiation) . I was wondering if it could be reliable to use it for NFSv4 inside a large cluster.
Philippe
Quoted 14 lines · DENIEL Philippe
Hi
Quoted 3 lines
The Linux server allows it, but I've been considering that a (low-priority) bug, so it wouldn't be safe to assume it will continue working.
I am mostly thinking at the client side using as well udp and tcp. Would the "NFSv4 client over UDP" behaves differently then the same client over TCP ?
Quoted 1 line
That aside, if you have a perfect network,
I can trust my network. It's not a WAN, it's located into a very massive cluster (it's kind of "internal LAN"). It's a very high throughput network (IB based) so I believe there are less "hardware based reason" to loose packets.
Quoted 5 lines
Quoted 2 lines
NFSv4.0 at least will
probably work.OK.
Quoted 1 line
(Not 4.1 since backchannel setup will fail?)
What is erroneous in using UDP for NFSv4.1 backchannels ?
I assumed it would require some stronger notion of a "connection" than UDP could provide. But maybe it could be made to work somehow.
Quoted 8 lines
Quoted 2 lines
Are you really sure that you can't make tcp scale to thousands of clients?
I am a bit afraid of a "No more file descriptors" effect. If I have one TCP socket per client and thousands of clients, I have less remaining fds for other purposes. Another point : UDP is a "cheap" protocol. I can have bunches of clients without overloading the server (a new client will almost cost nothing to the server, just the cost of a new clientid negotiation) .
Is the tcp state all that much more?
Quoted 2 lines
I was wondering if it
could be reliable to use it for NFSv4 inside a large cluster.
Well, it'd require some spec changes in any case.
--b.