Skip to contentSkip to conversation

why nfs server delay 10ms in nfsd_write()?

DISCUSSION steve 2 participants 2 messages May 20, 2005

lore
Share & tools
lore

steve

author
#0 lore raw

Hi Peter,
My envionment looks like that:

NFS Server:Suse9 Enterprise
NFS Client:Redhat AS3.0(kernel 2.4.21)

I made a ramdisk and export it with nfs
Server and Client are connected with 1000Mbps

mount the ramdisk on the client with parameters: -t nfs -o rw,noac

then test with iometer and the parameters are:
Outstanding IO is 32, transfer request size is 512, sequential write the result is about 300KB/s, iops is about 600

with dd test i find the delay most cost at the server.

i agree with Avi that "if the NFS client has no (or low) concurrency, then write gathering would reduce preformance"

Regards!
Steve
2005-05-20

======= 2005-05-19 10:10:00 =======

Quoted 50 lines
Quoted 17 lines · Lee Revell
Quoted 7 lines · Peter Staubach

There are certainly many others way to get gathering, without adding an artificial delay. There are already delay slots built into the code which could
be used to trigger the gathering, so with a little bit different architecture, the
performance increases could be achieved.

Quoted 6 lines

Some implementations actually do write gathering with NFSv3, even. Is this interesting enough to play with? I suspect that just doing the work for
NFSv2 is not...

Quoted 4 lines

Also, how do you explain the big performance hit that steve observed? Write gathering is supposed to help performance, but it's a big loss on his test...

Well, a little bit more information about what he is doing would be helpful. I'd
like to better understand the environment and what the traffic from the client
looks like.

Write gathering is not a panacea for all of the ills caused by the NFSv2 WRITE
stable storage requirements. In fact, if done wrong, it can cause performance
regressions, such as those being noticed by Steve.

--

I implemented the write gathering used in Solaris and experimented with several (many?) different approachs. Adding a delay in order to allow subsequent WRITE requests to arrive seems like a good thing, but can end up just adding latency to the entire process if not done right.

A suggestion might be to use the delay caused by the nfsd_sync() call and synchronize other WRITE requests around that. The delay caused by doing real i/o to the storage subsystem should allow write gathering to take place, assuming that the client is generating concurrent WRITE requests.

ps

#1 lore raw
Quoted 8 lines · steve

Hi Peter,
My envionment looks like that:

NFS Server:Suse9 Enterprise NFS Client:Redhat AS3.0(kernel 2.4.21)

I made a ramdisk and export it with nfs
Server and Client are connected with 1000Mbps

Quoted 9 lines

mount the ramdisk on the client with parameters: -t nfs -o rw,noac

then test with iometer and the parameters are: Outstanding IO is 32, transfer request size is 512, sequential write the result is about 300KB/s, iops is about 600

with dd test i find the delay most cost at the server.

i agree with Avi that "if the NFS client has no (or low) concurrency, then write gathering would reduce preformance"

I would agree too with Avi, and especially in this configuration...

I think that we could construct a writer gathering implementation which did not show measurable performance impact in this sort of situation though still.

Thanx...

ps