Differences between 2.6.5 und 2.6.16: getattr with actimeo=0 and O_DIRECT
Subsystems (2)
Hi,
analyzing performance issues with Oracle databases on an NFS client running on SLES9 SP3 and NetApp as NFS server, I found that in SLES9 SP3 each write call is followed by an getattr. This is not the case with SLES10 SP1.
Mount options in use are:
rw,v3,rsize=32768,wsize=32768,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,lock,proto=tcp,addr=172.18.131.134
I do
dd if=/dev/zero of=/mnt/qqq oflag=direct bs=8k count=100000
and using tcpdump (BTW, is there an easier way ?) I see with SLES9 SP3 (i.e. 2.6.5-7.244) each single 8k write followed by an getattr (which comes at some cost).
Using SLES10 SP1 (2.6.16.46-0.12) there is only one getattr when dd closes the file.
Is there anything I can do to avoid the getattr calls in SLES9 SP3 (no, sorry, can't update to SLES10 SP1) ?
Thanks
Gerd
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 5 lines · Gerd Bavendiek
analyzing performance issues with Oracle databases on an NFS client running on SLES9 SP3 and NetApp as NFS server, I found that in SLES9 SP3 each write call is followed by an getattr. This is not the case with SLES10 SP1.
The changes between 2.6.5 and 2.6.16 are too many that it is not easy to track down.
Quoted 7 lines
I do
dd if=/dev/zero of=/mnt/qqq oflag=direct bs=8k count=100000
and using tcpdump (BTW, is there an easier way ?) I see with SLES9 SP3 (i.e. 2.6.5-7.244) each single 8k write followed by an getattr (which comes at some cost).
I Would suggest trying the latest kernel update from Novell/SUSE.
If the issue is seen in the latest kernels, create a bug report in Novell/SUSE bugzilla - https://bugzilla.novell.com
Thanks,
Signature · 10 lines
Suresh Jayaraman ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 19 lines · Gerd Bavendiek
Hi,
analyzing performance issues with Oracle databases on an NFS client running on SLES9 SP3 and NetApp as NFS server, I found that in SLES9 SP3 each write call is followed by an getattr. This is not the case with SLES10 SP1.
Mount options in use are:
rw,v3,rsize=32768,wsize=32768,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,lock,proto=tcp,addr=172.18.131.134
I do
dd if=/dev/zero of=/mnt/qqq oflag=direct bs=8k count=100000
and using tcpdump (BTW, is there an easier way ?) I see with SLES9 SP3 (i.e. 2.6.5-7.244) each single 8k write followed by an getattr (which comes at some cost).
Quoted 5 lines
Using SLES10 SP1 (2.6.16.46-0.12) there is only one getattr when dd closes the file.
Is there anything I can do to avoid the getattr calls in SLES9 SP3 (no, sorry, can't update to SLES10 SP1) ?
For one thing, you could turn attribute caching back on. I don't know why SLES10 fails to GETATTR, but acregmin=0,acregmax=0 turns attribute caching off. Append writes need to know where the end-of-file is, and so they will force a GETATTR when there is no attribute caching.
Trond
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 26 lines · Trond Myklebust
Quoted 19 lines · Gerd Bavendiek
Hi,
analyzing performance issues with Oracle databases on an NFS client running on SLES9 SP3 and NetApp as NFS server, I found that in SLES9 SP3 each write call is followed by an getattr. This is not the case with SLES10 SP1.
Mount options in use are:
rw,v3,rsize=32768,wsize=32768,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,lock,proto=tcp,addr=172.18.131.134
I do
dd if=/dev/zero of=/mnt/qqq oflag=direct bs=8k count=100000
and using tcpdump (BTW, is there an easier way ?) I see with SLES9 SP3 (i.e. 2.6.5-7.244) each single 8k write followed by an getattr (which comes at some cost).
Quoted 5 lines
Using SLES10 SP1 (2.6.16.46-0.12) there is only one getattr when dd closes the file.
Is there anything I can do to avoid the getattr calls in SLES9 SP3 (no, sorry, can't update to SLES10 SP1) ?
Quoted 8 lines
For one thing, you could turn attribute caching back on. I don't know why SLES10 fails to GETATTR, but acregmin=0,acregmax=0 turns attribute caching off. Append writes need to know where the end-of-file is, and so they will force a GETATTR when there is no attribute caching.
Trond
Trond,
you say: SLES10 fails to GETATTR.
So with acregmin=0,acregmax=0 etc. we should always have one write AND one getattr ?
So SLES9 SP3 does it the right way ?
Gerd
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 30 lines · Gerd Bavendiek
Quoted 29 lines · Trond Myklebust
Quoted 27 lines · Gerd Bavendiek
Hi,
analyzing performance issues with Oracle databases on an NFS client running on SLES9 SP3 and NetApp as NFS server, I found that in SLES9 SP3 each write call is followed by an getattr. This is not the case with SLES10 SP1.
Mount options in use are:
rw,v3,rsize=32768,wsize=32768,acregmin=0,acregmax=0,acdirmin=0,acdir max=0,hard,lock,proto=tcp,addr=172.18.131.134
I do
dd if=/dev/zero of=/mnt/qqq oflag=direct bs=8k count=100000
and using tcpdump (BTW, is there an easier way ?) I see with SLES9 SP3
(i.e. 2.6.5-7.244) each single 8k write followed by an getattr (which
comes at some cost).Using SLES10 SP1 (2.6.16.46-0.12) there is only one getattr when dd closes the file.
Is there anything I can do to avoid the getattr calls in SLES9 SP3 (no, sorry, can't update to SLES10 SP1) ?
Quoted 6 lines
Quoted 6 lines
For one thing, you could turn attribute caching back on. I don't know why SLES10 fails to GETATTR, but acregmin=0,acregmax=0 turns attribute
caching off. Append writes need to know where the end-of-file is, and so
they will force a GETATTR when there is no attribute caching.
Note "oflag=direct".
Quoted 6 lines
Quoted 3 lines
Trond
Trond,
Quoted 8 lines
you say: SLES10 fails to GETATTR.
So with acregmin=0,acregmax=0 etc. we should always have one write AND one getattr ?
So SLES9 SP3 does it the right way ?
Gerd
SLES 9's direct I/O engine does the GETATTRs. There's no way to get rid of that behavior in that kernel version; it's an implementation limitation that was corrected in later versions.
SLES 10's direct I/O engine is more advanced, and doesn't do GETATTRs with direct I/O.
When performing direct I/O, the client doesn't need to know how large the file is.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs