why nfs server delay 10ms in nfsd_write()?
Hi Peter,
My envionment looks like that:
NFS Server:Suse9 Enterprise
NFS Client:Redhat AS3.0(kernel 2.4.21)
I made a ramdisk and export it with nfs
Server and Client are connected with 1000Mbps
mount the ramdisk on the client with parameters: -t nfs -o rw,noac
then test with iometer and the parameters are:
Outstanding IO is 32, transfer request size is 512, sequential write the result is about 300KB/s, iops is about 600
with dd test i find the delay most cost at the server.
i agree with Avi that "if the NFS client has no (or low) concurrency, then write gathering would reduce preformance"
Regards!
Steve
2005-05-20
======= 2005-05-19 10:10:00 =======
Quoted 50 lines
Quoted 17 lines · Lee Revell
Quoted 7 lines · Peter Staubach
There are certainly many others way to get gathering, without adding an artificial delay. There are already delay slots built into the code which could
be used to trigger the gathering, so with a little bit different architecture, the
performance increases could be achieved.Quoted 6 lines
Some implementations actually do write gathering with NFSv3, even. Is this interesting enough to play with? I suspect that just doing the work for
NFSv2 is not...Quoted 4 lines
Also, how do you explain the big performance hit that steve observed? Write gathering is supposed to help performance, but it's a big loss on his test...
Well, a little bit more information about what he is doing would be helpful. I'd
like to better understand the environment and what the traffic from the client
looks like.Write gathering is not a panacea for all of the ills caused by the NFSv2 WRITE
stable storage requirements. In fact, if done wrong, it can cause performance
regressions, such as those being noticed by Steve.--
I implemented the write gathering used in Solaris and experimented with several (many?) different approachs. Adding a delay in order to allow subsequent WRITE requests to arrive seems like a good thing, but can end up just adding latency to the entire process if not done right.
A suggestion might be to use the delay caused by the
nfsd_sync()call and synchronize other WRITE requests around that. The delay caused by doing real i/o to the storage subsystem should allow write gathering to take place, assuming that the client is generating concurrent WRITE requests.ps
Quoted 8 lines · steve
Hi Peter,
My envionment looks like that:NFS Server:Suse9 Enterprise NFS Client:Redhat AS3.0(kernel 2.4.21)
I made a ramdisk and export it with nfs
Server and Client are connected with 1000Mbps
Quoted 9 lines
mount the ramdisk on the client with parameters: -t nfs -o rw,noac
then test with iometer and the parameters are: Outstanding IO is 32, transfer request size is 512, sequential write the result is about 300KB/s, iops is about 600
with dd test i find the delay most cost at the server.
i agree with Avi that "if the NFS client has no (or low) concurrency, then write gathering would reduce preformance"
I would agree too with Avi, and especially in this configuration...
I think that we could construct a writer gathering implementation which did not show measurable performance impact in this sort of situation though still.
Thanx...
ps