NFS Write Delays

Hi,

I'm NFS mounting a 100G ext3 partion over NFS. There's a gigabit link between the server and client. When I attempt to copy a file from the client to the server, nothing is transferred for between 5 and 10 seconds. If I run wireshark, I see some RPC requests and replys. It looks like every request gets a reply, so I don't think the server is blocking. As well, I have another client that is able to transfer files without delay to this server. So it appears to be a client issue. I'm not using any extra options when mounting (I did try with async and with udp, but this did not fix anything). This also isn't an issue with the client's disk, as I attempted to dd /dev/zero across and got the same delay.

I'm running Debian on all systems. Here's the versions I've tested this with (client and server):

nfs-utils: 1:1.0.10-6+etch.1 and 1.1.1~git-20070709-3
portmap: 5-26 and 6.0-4
kernel: client: 2.6.21-1-alpha-smp, server: 2.6.22-1-amd64

If you need any more information, let me know. If there's any flag I can turn on to see what's happening, that would be useful to test.

-- David

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

I was able to turn on debugging and I believe I see what's happening. Here's what I see in /var/log/messages:

01:01:31 kernel: nfs: write(/mterry2.avi(147458), 262144@0) 01:01:31 kernel: nfs: write(/mterry2.avi(147458), 262144@262144) 01:01:31 kernel: nfs: write(/mterry2.avi(147458), 262144@524288) 01:01:31 kernel: nfs: write(/mterry2.avi(147458), 262144@786432) .
.
.
01:01:38 kernel: nfs: write(/mterry2.avi(147458), 262144@417071104) 01:01:38 kernel: nfs: write(/mterry2.avi(147458), 201314@417333248) 01:01:38 kernel: nfs: flush(0:13/147458)
01:01:39 kernel: RPC: 62652 reserved req fffffc007b4fc000 xid 4e72324e 01:01:39 kernel: RPC: 62652 xprt_prepare_transmit
01:01:39 kernel: RPC: 62652 xprt_transmit(262332)
01:01:39 kernel: RPC: 62653 reserved req fffffc007b4fc128 xid 4f72324e

So it looks like it takes 7 seconds to read the entire file into memory (the client has 6G of RAM), and only after it is read in is the file pushed to the network. I'm wondering if there is some way to force the data to start transmitting before it's does reading the file in.

-- David

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

Sure. You can simply tell the kernel to keep fewer dirty pages by adjusting the values of /proc/sys/vm/dirty_background_ratio and /proc/sys/vm/dirty_ratio.

See Documentation/filesystems/proc.txt and/or
Documentation/sysctl/vm.txt

Cheers
Trond

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs