Skip to contentSkip to conversation

Timeout issue (similar to bugs 11061 and 11154), bisected

DISCUSSION Arto Jantunen 2 participants 3 messages Feb 16, 2009 – Feb 17, 2009 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

(I'm not subscribed, so please CC me on any replies)

I seem to have hit a NFS bug while upgrading a machine from Debian Etch to Debian Lenny. I have a NFS server running FreeBSD 7.0 RC1 and a bunch of clients running Linux. The ones running kernel 2.6.18 work perfectly, as do the ones running 2.6.24. The one I upgraded to 2.6.26 fails. After 5-15 minutes of working normally the mount dies and I get the usual "nfs: server <server> not responding, still trying" in dmesg. The only way I have found to get the mount back is umount -f && mount, waiting does not bring it back.

I have tested quite a bunch of different kernel versions, and starting from 25 and ending at the git tree last week they all fail in the same way. Bisecting tracks the problem to commit e06799f958bf7f9f8fae15f0c6f519953fb0257c

I originally thought that it was the same as bug 11154, but the patches attached to that bug do not fix this issue.

Any thoughts, patches, ideas?

Signature · 1 line
Arto Jantunen
#1 lore raw
Quoted 16 lines · Arto Jantunen

(I'm not subscribed, so please CC me on any replies)

I seem to have hit a NFS bug while upgrading a machine from Debian Etch to Debian Lenny. I have a NFS server running FreeBSD 7.0 RC1 and a bunch of clients running Linux. The ones running kernel 2.6.18 work perfectly, as do the ones running 2.6.24. The one I upgraded to 2.6.26 fails. After 5-15 minutes of working normally the mount dies and I get the usual "nfs: server <server> not responding, still trying" in dmesg. The only way I have found to get the mount back is umount -f && mount, waiting does not bring it back.

I have tested quite a bunch of different kernel versions, and starting from 25 and ending at the git tree last week they all fail in the same way. Bisecting tracks the problem to commit e06799f958bf7f9f8fae15f0c6f519953fb0257c

Quoted 4 lines

I originally thought that it was the same as bug 11154, but the patches attached to that bug do not fix this issue.

Any thoughts, patches, ideas?

That looks like the known problem with the NFS server failing to close connections in a timely manner. There is a fix for this in

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git&a=commitdiff&h=69b6ba3712b796a66595cfaf0a5ab4dfe1cf964a

There is also a client side patch that increases the robustness of the client when it hits a buggy server, and that causes it to do the equivalent of a linger2 timeout. That patch is as of yet not merged into mainline, however I've attached it below together with a followup patch that makes the timeout configurable...

Cheers
Trond

#3 lore raw
In reply to a message not in the archive
Quoted 25 lines · Trond Myklebust
Quoted 16 lines · Arto Jantunen

(I'm not subscribed, so please CC me on any replies)

I seem to have hit a NFS bug while upgrading a machine from Debian Etch to Debian Lenny. I have a NFS server running FreeBSD 7.0 RC1 and a bunch of clients running Linux. The ones running kernel 2.6.18 work perfectly, as do the ones running 2.6.24. The one I upgraded to 2.6.26 fails. After 5-15 minutes of working normally the mount dies and I get the usual "nfs: server <server> not responding, still trying" in dmesg. The only way I have found to get the mount back is umount -f && mount, waiting does not bring it back.

I have tested quite a bunch of different kernel versions, and starting from 25 and ending at the git tree last week they all fail in the same way. Bisecting tracks the problem to commit e06799f958bf7f9f8fae15f0c6f519953fb0257c

Quoted 4 lines

I originally thought that it was the same as bug 11154, but the patches attached to that bug do not fix this issue.

Any thoughts, patches, ideas?

That looks like the known problem with the NFS server failing to close connections in a timely manner. There is a fix for this in

Quoted 7 lines

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git&a=commitdiff&h=69b6ba3712b796a66595cfaf0a5ab4dfe1cf964a

There is also a client side patch that increases the robustness of the client when it hits a buggy server, and that causes it to do the equivalent of a linger2 timeout. That patch is as of yet not merged into mainline, however I've attached it below together with a followup patch that makes the timeout configurable...

The client side patch you attached hides the problem on the server, after applying it the mount sticks around. As previously discussed, the server is running an apparently buggy version of FreeBSD and I'd rather not touch it right now since it is in production.

Thanks for your fast response.

Signature · 1 line
Arto Jantunen