why do attempts to access a nfs v3 filesystem (ro,soft) block the process for minutes at a time? (when the nfs server is down)
Subsystems (2)
(apologies for the cross post from the deprecated list)
Hi all,
I have a web server which serves some content from an nfs filesystem mounted like so;
nfsserver1:/somemount /var/www/html/somefiles nfs rw,soft
0 0
# mount | grep nfs nfsserver1:/somemount on /var/www/html/somefiles type nfs (ro,soft,addr=xx.xx.xx.xx)
According to the documentation, an NFS operation on a soft mount should wait for a "major timeout" and then report "server not responding" to syslog and return an error. where a major timeout is after default retrans=3 retransmissions.
I understand the process to be like this;
call --->0.7 secs --->retransmission--->1.4 secs--->retransmission--->2.8 secs--->server not responding(major timeout)
However it is pretty clear that this is retrying indefinitely (or at least many more times that I would like), as the log files show loads of;
Jul 16 07:56:09 server1 kernel: nfs: server server2 not responding, timed out Jul 16 07:57:09 server1 last message repeated 4 times Jul 16 07:57:09 server1 last message repeated 6 times
and eventually this kills the apache server as all the available processes are blocked during "retrying indefinitely", until the apache server is restarted. (restarting the nfs server at this point does not seem to recover the apache child processes)
So what should my strategy be to stop the failed mount killing apache. I care more about the apache staying up, as I don't have that much control over the nfs server..
(also I noticed that it seems to timeout quicker with the mount options set like (soft, timeo=7, retrans=3) which is unexpected, because they are supposed to be the default)
Regards and thanks in advance,
T
Quoted 36 lines · Tom H
(apologies for the cross post from the deprecated list)
Hi all,
I have a web server which serves some content from an nfs filesystem mounted like so;
nfsserver1:/somemount/var/www/html/somefilesnfs rw,soft 0 0# mount | grep nfs nfsserver1:/somemount on /var/www/html/somefiles type nfs (ro,soft,addr=xx.xx.xx.xx)According to the documentation, an NFS operation on a soft mount should wait for a "major timeout" and then report "server not responding" to syslog and return an error. where a major timeout is after default retrans=3 retransmissions.
I understand the process to be like this;
call --->0.7 secs --->retransmission--->1.4 secs--->retransmission--->2.8 secs--->server not responding(major timeout)However it is pretty clear that this is retrying indefinitely (or at least many more times that I would like), as the
log files show loads of;Jul 16 07:56:09 server1 kernel: nfs: server server2 not responding, timed out Jul 16 07:57:09 server1 last message repeated 4 times Jul 16 07:57:09 server1 last message repeated 6 timesand eventually this kills the apache server as all the available processes are blocked during "retrying indefinitely", until the apache server is restarted. (restarting the nfs server at this point does not seem to recover the apache child processes)
Quoted 7 lines
So what should my strategy be to stop the failed mount killing apache. I care more about the apache staying up, as I don't have that much control over the nfs server..
(also I noticed that it seems to timeout quicker with the mount options set like (soft, timeo=7, retrans=3) which is unexpected, because they are supposed to be the default)
They are the default settings for UDP mounts, but you didn't specify UDP. TCP is the default transport protocol, and has been for some time.
TCP uses a long retransmit timeout. See nfs(5).
Signature · 1 line
Chuck Lever
Quoted 8 lines · Chuck Lever
Quoted 4 lines · Tom H
(also I noticed that it seems to timeout quicker with the mount options set like (soft, timeo=7, retrans=3) which is unexpected, because they are supposed to be the default)
They are the default settings for UDP mounts, but you didn't specify UDP. TCP is the default transport protocol, and has been for some time. TCP uses a long retransmit timeout. See nfs(5).
OK, I see that now. Thanks.!
However further experimentation with mount options (ro,soft,retrans=0,timeo=0,intr,proto=tcp) - requests to a failed nfs file-system still block the apache process for some apparently random time up to 3 minutes.
Cheers
T
Quoted 11 lines · Tom H
Quoted 10 lines · Chuck Lever
Quoted 4 lines · Tom H
(also I noticed that it seems to timeout quicker with the mount options set like (soft, timeo=7, retrans=3) which is unexpected, because they are supposed to be the default)
They are the default settings for UDP mounts, but you didn't specify UDP. TCP is the default transport protocol, and has been for some time. TCP uses a long retransmit timeout. See nfs(5).
Quoted 6 lines
OK, I see that now. Thanks.!
However further experimentation with mount options (ro,soft,retrans=0,timeo=0,intr,proto=tcp) - requests to a failed nfs file-system still block the apache process for some apparently random time up to 3 minutes.
I don't know exactly what retrans=0 and timeo=0 might do, but short timeouts over TCP are not recommended. If you want it to fail sooner (and your network is clean enough), use proto=udp.