bug in linux mount? (says NetApp)
Subsystems (3)
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)
page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Our defaults (assuming man pages are correct, RedHat Enterprise Linux 3) would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...
* What "bug" in the mount command do you believe NetApp is talking about?
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
* What is the reference standard behavior?
Thanks,
--Greg
Signature · 18 lines
---------------------------------------------------------------------- Greg Baker 512-602-3287 (work) [email protected] 512-602-6970 (fax) 5900 E. Ben White Blvd MS 626 512-555-1212 (info) Austin, TX 78741 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 22 lines · Gregory Baker
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Quoted 4 lines
Our defaults (assuming man pages are correct, RedHat Enterprise Linux 3) would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...
It's important to distinguish two different types of timeouts.
1. The mount operation has timed out.
2. After the mount operation succeeds, an NFS RPC operation has timed out.
TR-3183 discusses the proper settings for 2, but you are experiencing 1.
The automounter attempts to mount one of the filer's exports, but the mount request times out causing the mounted-on directory to be exposed. Your filer is heavily loaded, and the filer's mountd is single-threaded. The filer may also be experiencing delays when requesting information from external servers (like DNS or NIS), in which case the mount request is held up at the filer.
Both sides are at fault: the Linux mount command should retry (and I believe later releases of RHEL 3 were fixed to do this) and the filer configuration should be reviewed to make sure there are no avoidable delays while processing mount requests.
Quoted 1 line
* What "bug" in the mount command do you believe NetApp is talking about?
The bug is that the mount command overrides the proper default RPC timeout value with a timeout value of 0.7 seconds. This is *not* the timeout for mount operations, it is the timeout for the in-kernel NFS client to retransmit RPC requests.
Quoted 2 lines
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
If you are using NFS over TCP, the proper timeout value is 60 seconds.
Quoted 1 line
* What is the reference standard behavior?
Solaris, which is the NFSv3 reference implementation, uses effectively a 60 second timeout on TCP mounts.
Signature · 12 lines
"We who cut mere stones must always be envisioning cathedrals" -- Quarry worker's creed ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Ahh... I should have expanded "linux clients" to "linux clients running RHEL 3 U5".
[greg@apathy greg]$ rpm -qa util-linux
util-linux-2.11y-31.6
Red Hat support has this to say...
[...snip...]
"I have been looking into this issue and I have found other people are experiencing similar behavior. I also found a fix that was added to the util-linux package that I think addresses this issue...... I believe this is what Chuck refers to with his comment "and I believe later releases of RHEL 3 were fixed to do this"
Quoted 1 line
From the upstream package change log:
"RHEL3 util-linux >=2.11y-31.8 should make the default 70s (instead of 7s) for TCP mounts:
* Wed Jun 8 2005 Steve Dickson <[email protected]> 2.11y-31.8
- Changed nfsmount to retry calls to mountd in foreground as well as in background (bz# 138775)
- Increased TCP timeouts to 70 secs (bz# 151097)"
I am pretty sure this will fix the problem that you are seeing. The util-linux package in the Red Hat Enterprise Linux AS (v. 3 for x86) Beta channel on RHN is version util-linux-2.11y-31.16.i386.rpm, which shold have this fix in it."
[...snip...]
The bug/errata
http://rhn.redhat.com/errata/RHBA-2005-626.html
became available in RHEL3 U6. Sigh.
We skipped U3, U4 (autofs woes) U6 (just finished upgrading from U2->U5 and dealing with fallout) and recently began using U7 (to support Sun x4100 SAS drives).
Thanks,
--Greg
Quoted 66 lines · Chuck Lever
Quoted 22 lines · Gregory Baker
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Quoted 5 lines
Our defaults (assuming man pages are correct, RedHat Enterprise Linux 3) would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...
It's important to distinguish two different types of timeouts.
1. The mount operation has timed out.
2. After the mount operation succeeds, an NFS RPC operation has timed out.
TR-3183 discusses the proper settings for 2, but you are experiencing 1.
The automounter attempts to mount one of the filer's exports, but the mount request times out causing the mounted-on directory to be exposed. Your filer is heavily loaded, and the filer's mountd is single-threaded. The filer may also be experiencing delays when requesting information from external servers (like DNS or NIS), in which case the mount request is held up at the filer.
Both sides are at fault: the Linux mount command should retry (and I believe later releases of RHEL 3 were fixed to do this) and the filer configuration should be reviewed to make sure there are no avoidable delays while processing mount requests.
Quoted 1 line
* What "bug" in the mount command do you believe NetApp is talking about?
The bug is that the mount command overrides the proper default RPC timeout value with a timeout value of 0.7 seconds. This is *not* the timeout for mount operations, it is the timeout for the in-kernel NFS client to retransmit RPC requests.
Quoted 2 lines
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
If you are using NFS over TCP, the proper timeout value is 60 seconds.
Quoted 1 line
* What is the reference standard behavior?
Solaris, which is the NFSv3 reference implementation, uses effectively a 60 second timeout on TCP mounts.
Signature · 18 lines
---------------------------------------------------------------------- Greg Baker 512-602-3287 (work) [email protected] 512-602-6970 (fax) 5900 E. Ben White Blvd MS 626 512-555-1212 (info) Austin, TX 78741 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 22 lines · Gregory Baker
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Quoted 4 lines
Our defaults (assuming man pages are correct, RedHat Enterprise Linux 3) would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...
No they are not. See below.
Quoted 1 line
* What "bug" in the mount command do you believe NetApp is talking about?
It has nothing to do with the mount timeout: Chuck is talking about the retransmission timeout for TCP connections 'timeo' which should indeed be set to a high value since TCP guarantees message delivery (unlike UDP which requires a small timeo value). Setting it too low means that you end up spamming your server with a load of unnecessary retransmissions.
This was indeed the case for some older versions of 'mount' and also for older versions of the am-utils/amd automounters.
Quoted 2 lines
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
Something like
mount -t nfs -ohard,timeo=600,retrans=2,rsize=32768,wsize=32768,tcp foo:/ /bar
should be a fairly safe bet. You might want to add the 'intr' flag too, depending on how you feel about the behaviour w.r.t. pressing ^C.
Quoted 1 line
* What is the reference standard behavior?
To which reference are you referring?
Cheers,
Trond
Thanks Trond!
I was referring to the 'standard' comment from the netapp PDF:
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
And was wondering what the 'standard' was. Chuck politely pointed me to Solaris as the NFSv3 reference for 'standard'.
Thanks,
--Greg
Quoted 43 lines · Trond Myklebust
Quoted 4 lines · Gregory Baker
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
Quoted 8 lines
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Our defaults (assuming man pages are correct, RedHat Enterprise Linux 3) would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...
No they are not. See below.
Quoted 1 line
* What "bug" in the mount command do you believe NetApp is talking about?
It has nothing to do with the mount timeout: Chuck is talking about the retransmission timeout for TCP connections 'timeo' which should indeed be set to a high value since TCP guarantees message delivery (unlike UDP which requires a small timeo value). Setting it too low means that you end up spamming your server with a load of unnecessary retransmissions.
This was indeed the case for some older versions of 'mount' and also for older versions of the am-utils/amd automounters.
Quoted 1 line
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
Something like
mount -t nfs -ohard,timeo=600,retrans=2,rsize=32768,wsize=32768,tcp foo:/ /bar
should be a fairly safe bet. You might want to add the 'intr' flag too, depending on how you feel about the behaviour w.r.t. pressing ^C.
Quoted 1 line
* What is the reference standard behavior?
To which reference are you referring?
Cheers,
Trond
Signature · 5 lines
---------------------------------------------------------------------- Greg Baker 512-602-3287 (work) [email protected] 512-602-6970 (fax) 5900 E. Ben White Blvd MS 626 512-555-1212 (info) Austin, TX 78741
Ian Kent
maintainerQuoted 41 lines · Trond Myklebust
Quoted 22 lines · Gregory Baker
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Quoted 4 lines
Our defaults (assuming man pages are correct, RedHat Enterprise Linux 3) would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...
No they are not. See below.
Quoted 1 line
* What "bug" in the mount command do you believe NetApp is talking about?
It has nothing to do with the mount timeout: Chuck is talking about the retransmission timeout for TCP connections 'timeo' which should indeed be set to a high value since TCP guarantees message delivery (unlike UDP which requires a small timeo value). Setting it too low means that you end up spamming your server with a load of unnecessary retransmissions.
This was indeed the case for some older versions of 'mount' and also for older versions of the am-utils/amd automounters.
Quoted 6 lines
Quoted 2 lines
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
Something like
mount -t nfs -ohard,timeo=600,retrans=2,rsize=32768,wsize=32768,tcp foo:/ /bar
I thought that the default timeo had changed to 600 (60 secs) for TCP mounts in later versions of mount (it should be) and that the retrans shouldn't matter as 60 secs is the RPC major timeout. I thought the point of this default was to prevent RPC from retransmitting as the TCP layer would take care of it.
Trond what am I missing.
Quoted 5 lines
should be a fairly safe bet. You might want to add the 'intr' flag too, depending on how you feel about the behaviour w.r.t. pressing ^C.
Quoted 1 line
* What is the reference standard behavior?
I don't think it's a standard in as much as it's common sense. Certainly if the defaults are set to those that are sensible for UDP you are much more likely to bogus failures.
Ian
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 15 lines · Ian Kent
Quoted 3 lines · Trond Myklebust
Something like
mount -t nfs -ohard,timeo=600,retrans=2,rsize=32768,wsize=32768,tcp foo:/ /bar
I thought that the default timeo had changed to 600 (60 secs) for TCP mounts in later versions of mount (it should be) and that the retrans shouldn't matter as 60 secs is the RPC major timeout. I thought the point of this default was to prevent RPC from retransmitting as the TCP la
Quoted 4 lines
should be a fairly safe bet. You might want to add the 'intr' flag too, depending on how you feel about the behaviour w.r.t. pressing ^C.
Quoted 1 line
* What is the reference standard behavior?
Quoted 8 lines
I don't think it's a standard in as much as it's common sense. Certainly if the defaults are set to those that are sensible for UDP you are much more likely to bogus failures.
Ian
yer would take care of it.Trond what am I missing.
The fact that not everyone may be using a recent version of mount :-) The above line should work fine with all versions.
Actually, the "retrans" value is pretty irrelevant for hard mounts. It is more important for soft mounts for which it defines the number of resends before giving up the RPC call and returning an error to the application.
Cheers,
Trond
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 9 lines · Trond Myklebust
It has nothing to do with the mount timeout: Chuck is talking about the retransmission timeout for TCP connections 'timeo' which should indeed be set to a high value since TCP guarantees message delivery (unlike UDP which requires a small timeo value). Setting it too low means that you end up spamming your server with a load of unnecessary retransmissions.
This was indeed the case for some older versions of 'mount' and also for older versions of the am-utils/amd automounters.
Do you know how you tell what value of timeo is being used by default?
The source code for mount (nfsmount.c, part of util-linux v2.12) has the comment:
/* timeo is filled in after we know whether it'll be TCP or UDP */
Can I assume, in this case, the value of timeo will be a suitable value for tcp mounts?
Thanks
James Pearson
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
What version of OnTap are you running?
-Blake
Quoted 34 lines · Gregory Baker
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Our defaults (assuming man pages are correct, RedHat Enterprise Linux
3)
would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths
of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...* What "bug" in the mount command do you believe NetApp is talking about?
Quoted 35 lines
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
* What is the reference standard behavior?
Thanks,
--Greg
Signature · 23 lines
---------------------------------------------------------------------- Greg Baker 512-602-3287 (work) [email protected] 512-602-6970 (fax) 5900 E. Ben White Blvd MS 626 512-555-1212 (info) Austin, TX 78741 ----------------------------------------------------------------------- -- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel? cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
We have various versions; 3500 and R series are running 7.x, the older 840/880 are running 6.5.x
NetApp Release 7.0.2P4: Wed Nov 30 01:52:11 PST 2005 charity.amd.com NetApp Release 7.0.3P4: Sat Jan 28 06:03:18 PST 2006 chang.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 diego.amd.com NetApp Release 7.0.3P4: Sat Jan 28 06:03:18 PST 2006 eng.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 escher.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 faith.amd.com NetApp Release 6.5.6P10D3:Thu Apr 13 22:13:21 PDT 2006 fortitud.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 frida.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 hope.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 justice.amd.com NetApp Release 7.0.3P4: Sat Jan 28 06:03:18 PST 2006 kuching.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 okeeffe.amd.com NetApp Release 7.0.2P4: Wed Nov 30 01:52:11 PST 2005 prudence.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 raffael.amd.com NetApp Release 7.0.3P4: Sat Jan 28 06:03:18 PST 2006 siam.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 sobriety.amd.com NetApp Release 6.5.5P7: Thu Jul 21 02:10:50 PDT 2005 vangogh.amd.com NetApp Release 7.2RC1: Wed Feb 15 00:46:09 PST 2006 jacen.amd.com NetApp Release 7.2RC1: Wed Feb 15 00:46:09 PST 2006 jaina.amd.com
--Greg
Quoted 69 lines · Blake Golliher
What version of OnTap are you running?
-Blake
Quoted 62 lines · Gregory Baker
We have thousands of linux clients hitting netapp file servers (many 3500 series, clustered) on a local gigabit LAN. From time to time, applications return "file not found" when attempting to automount a directory and access a file. An example of this is a long running process, which reads in data, processes it for hours (in which time the filesystem is unmounted) then tries to read more data from that mount point (which causes a "file not found" error in the application). This occurs about 1/100th of the time.
Researching at Netapp turns up this bit by Chuck Lever (Linux NFS contributer)
"Using the Linux NFS Client with Network Appliance Filers"
http://www.netapp.com/libr ary/tr/3183.pdf (February 2006)page 10 says...
"Due to a bug in the mount command, the default retransmission timeout value on Linux for NFS over TCP is quite small...To obtain standard behavior, we strongly recommend using "timeo=600, retrans=2" explicitly when mounting via TCP."
Our defaults (assuming man pages are correct, RedHat Enterprise Linux 3) would be timeo=7, retrans=3, which translates to 7+14+28+56 = 105 tenths of a second (10 seconds). It appears netapp is suggesting waiting 600+600 = 1200 tenths (120 seconds) before giving up on the mount command...
* What "bug" in the mount command do you believe NetApp is talking about?
* What do you think proper options for NFS auto/mounts would be for extremely busy centralized NFS filers?
* What is the reference standard behavior?
Thanks,
--Greg
-- ----------------------------------------------------------------------
Greg Baker 512-602-3287 (work) [email protected] 512-602-6970 (fax) 5900 E. Ben White Blvd MS 626 512-555-1212 (info)Austin, TX 78741
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Signature · 18 lines
---------------------------------------------------------------------- Greg Baker 512-602-3287 (work) [email protected] 512-602-6970 (fax) 5900 E. Ben White Blvd MS 626 512-555-1212 (info) Austin, TX 78741 ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs