Skip to contentSkip to conversation

Text based mount options ignoring the preferred rwsize?

DISCUSSION James Pearson 3 participants 7 messages Sep 8, 2009 – Sep 10, 2009 4 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I've noticed a difference in the rsize used when mounting a file system
between using text and binary mount options.

The client is running a CentOS5 based distro with a 2.6.32-rc8 kernel

The server has a preferred rsize of 128kb and maximum rsize of 512kb

When I use mount.nfs from CentOS5/RHEL5 nfs-utils (based on v1.0.9) and don't give any rsize option, it mounts the file system with a rsize of 128kb. This uses binary mount options

But, when using mount.nfs from nfs-utils 1.2.0, the file system is mounted with an rsize of 512kb

Looking at the nfs-utils and kernel source, it appears that for binary options, rsize is set to 0 if not given by mount.nfs, and the kernel eventually, in this case, increases this to preferred size.

But for text mount options, if not set by mount.nfs, the default size is
set to NFS_MAX_FILE_IO_SIZE in the kernel, which, in this case, gets
reduced to the server maximum size.

Should the kernel be setting rsize (and wsize) to 0 by default?

Thanks

James Pearson

#2 lore raw
In reply to a message not in the archive
Quoted 8 lines · James Pearson

I've noticed a difference in the rsize used when mounting a file system between using text and binary mount options.

The client is running a CentOS5 based distro with a 2.6.32-rc8 kernel

The server has a preferred rsize of 128kb and maximum rsize of 512kb

When I use mount.nfs from CentOS5/RHEL5 nfs-utils (based on v1.0.9) and don't give any rsize option, it mounts the file system with a rsize of 128kb. This uses binary mount options

Quoted 7 lines

But, when using mount.nfs from nfs-utils 1.2.0, the file system is mounted with an rsize of 512kb

Looking at the nfs-utils and kernel source, it appears that for binary options, rsize is set to 0 if not given by mount.nfs, and the kernel eventually, in this case, increases this to preferred size.

But for text mount options, if not set by mount.nfs, the default size is set to NFS_MAX_FILE_IO_SIZE in the kernel, which, in this case, gets reduced to the server maximum size.

Should the kernel be setting rsize (and wsize) to 0 by default?

nfs(5) says:

"If an [rw]size value is not specified, or if the specified [rw]size value is larger than the maximum that either client or server can support, the client and server negotiate the largest [rw]size value that they can both support."

So the text-based behavior is what is documented now.

Does anyone know of a reason to use the server's "preferred" transfer size rather than the largest size supported by both client and server? Usually those are the same.

Quoted 7 lines

Thanks

James Pearson
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

#3 lore raw
Quoted 11 lines · Chuck Lever
Quoted 2 lines

Should the kernel be setting rsize (and wsize) to 0 by default?

nfs(5) says:

"If an [rw]size value is not specified, or if the specified [rw]size value is larger than the maximum that either client or server can support, the client and server negotiate the largest [rw]size value that they can both support."

So the text-based behavior is what is documented now.

Does anyone know of a reason to use the server's "preferred" transfer size rather than the largest size supported by both client and server? Usually those are the same.

In this case, the manufacturer of the NFS server recommends using 128Kb for rsize and 512Kb for wsize - although the maximum rsize it supports is 512Kb. I assume in their testing, these values have given optimal performance figures.

The default behaviour with binary mount options when no [rw]size is to select these preferred values - which to me, makes sense - as by not giving a [rw]size, you are leaving it up the server to pick the 'best' values for you - which I guess in most (all other?) cases happen to be the maximum size.

James Pearson

#5 lore raw
In reply to a message not in the archive
Quoted 9 lines · James Pearson
Quoted 6 lines · Chuck Lever
Quoted 2 lines

Should the kernel be setting rsize (and wsize) to 0 by default?

nfs(5) says:
"If an [rw]size value is not specified, or if the specified [rw]size value is larger than the maximum that either client or server can support, the client and server negotiate the largest [rw]size value that they can both support." So the text-based behavior is what is documented now.
Does anyone know of a reason to use the server's "preferred" transfer size rather than the largest size supported by both client and server? Usually those are the same.

In this case, the manufacturer of the NFS server recommends using 128Kb for rsize and 512Kb for wsize - although the maximum rsize it supports is 512Kb. I assume in their testing, these values have given optimal performance figures.

In this case maybe you should set these explicitly.

Quoted 1 line

The default behaviour with binary mount options when no [rw]size is to select these preferred values - which to me, makes sense - as by not giving a [rw]size, you are leaving it up the server to pick the 'best' values for you - which I guess in most (all other?) cases happen to be the maximum size.

It might make sense to think of the server's preference as the value you should use. But for the client, the larger the better in nearly every case. So which one should you choose by default?

Someone could still make the case that the old behavior was better in some way (and provide benchmark evidence for this, of course, for multiple combinations of implementations across different networks).

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

#6 lore raw
In reply to a message not in the archive
Quoted 5 lines · James Pearson

The default behaviour with binary mount options when no [rw]size is to select these preferred values - which to me, makes sense - as by not giving a [rw]size, you are leaving it up the server to pick the 'best' values for you - which I guess in most (all other?) cases happen to be the maximum size.

Right. The above was indeed the guiding principle back when I did the rsize/wsize negotiation for NFSv3 and NFSv2 for the binary mount code.

The NFS protocol specifies that the maximum values are there to tell you that the server will do short read/writes if you exceed these. However, the preferred values may correspond to a different 'sweet spot' for the server read and write implementations.

Cheers
Trond

#8 lore raw
In reply to a message not in the archive
Quoted 5 lines · Trond Myklebust
Quoted 1 line · James Pearson

The default behaviour with binary mount options when no [rw]size is to select these preferred values - which to me, makes sense - as by not giving a [rw]size, you are leaving it up the server to pick the 'best' values for you - which I guess in most (all other?) cases happen to be the maximum size.

Quoted 7 lines

Right. The above was indeed the guiding principle back when I did the rsize/wsize negotiation for NFSv3 and NFSv2 for the binary mount code.

The NFS protocol specifies that the maximum values are there to tell you that the server will do short read/writes if you exceed these. However, the preferred values may correspond to a different 'sweet spot' for the server read and write implementations.

So does that mean that the binary mount options are doing the right thing, whereas the text mount options are not? Also, just to confirm, I'm using NFSv3.

Thanks

James Pearson

#10 lore raw
In reply to a message not in the archive
Quoted 11 lines · James Pearson
Quoted 8 lines · Trond Myklebust
Quoted 1 line · James Pearson

The default behaviour with binary mount options when no [rw]size is to select these preferred values - which to me, makes sense - as by not giving a [rw]size, you are leaving it up the server to pick the 'best' values for you - which I guess in most (all other?) cases happen to be the maximum size.

Right. The above was indeed the guiding principle back when I did the rsize/wsize negotiation for NFSv3 and NFSv2 for the binary mount code. The NFS protocol specifies that the maximum values are there to tell you that the server will do short read/writes if you exceed these. However, the preferred values may correspond to a different 'sweet spot' for the server read and write implementations.

So does that mean that the binary mount options are doing the right thing, whereas the text mount options are not? Also, just to confirm, I'm using NFSv3.

I don't think there is a clear right or wrong here. The binary behavior follows protocol guidelines, which is fine. But, there are plenty of cases where using the server's preferred size by default is not always optimal. For example, the server may advertise a preference for 32768, but using that setting with UDP on a congested network would result in poor performance.

In other words, the client may decide that using a larger or smaller setting than the server's preference would be better. Or, it may use the largest possible setting for r/wsize, but advertise the server's preference number in stat(2) results.

Fwiw, I don't have any problem switching the text-based defaults to zero.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com