Revisting mmap() and truncate() related corruption
Subsystems (2)
We're running a large farm of compile servers, recently upgraded to kernel 2.4.20 from 2.2.14. We use distcc for compilation. Shortly after the upgrade, we started noticing object files were occasionally corrupted, having the right length but full of nulls.
A little Googling turned up this thread
http://marc.theaimsgroup.com/?l=linux-nfs&m=104948555032343&w=2 that pretty much sums up what we're seeing. I built fsx and tried in on our network and it reveals the same type of corruption we're seeing during compilation.
(I also tested a 2.4.24 kernel, which exhibited the same problem, and a 2.6.0-testXX kernel which did not exhibit any problems. The original 2.2.14 kernel also survived several hours of fsx without incident.)
distcc writes object files received from the network to disk by mmap()'ing a file, using ftruncate() on it to get it to the right length and then writing the data into it.
I have also seen mention of this issue in the distcc FAQ, where they recommend using "no_subtree_check" when exporting NFS filesystems, but our servers are NetApps, not Linux. So I don't think this is applicable.
After reading the thread cited above, I could not find any further discussion of the problem or any evidence of the patch making it into a released kernel.
So, I'd like to find out what ever happened with this issue and if there ever were, or will be, any official fixes against the 2.4.x line...
Thanks.
-Chris Newbold
Laurel Networks, Inc.
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · Marcin Zieba
Chris Newbold wrote:
[...]Quoted 4 lines
I have also seen mention of this issue in the distcc FAQ, where they recommend using "
no_subtree_check" when exporting NFS filesystems, but our servers are NetApps, not Linux. So I don't think this is applicable.
Quoted 5 lines
You can try to change nfs.udp.xfersize to, say, 8192, and on linux, use that size, too. Or try tcp, instead of udp. I'm not saying, that it'll solve the problem, but maybe allow you to narrow the problem.
From what I've gathered, there are (at least) two issues with
mmap()and
NFS: one involving rename() and another with truncate(). It seems like the "no_subtree_check" recommendation for Linux servers was directed at the rename() problem.
We're having the truncate() problem.
Researching that original thread a little more, I see that Steve Dickson (who originally posted a candidate patch for the problem here) found that this problem persisted regardless of whether UDP or TCP was used. FWIW...
-Chris
Signature · 15 lines
====( Chris Newbold <[email protected]> )========================== Laurel Networks, Inc. voice: +1 412 809 4200 fax: +1 412 809 4201 "If you fool around with a thing for very long you will screw it up." --Murphy ------------------------------------------------------------------------------ ------------------------------------------------------- The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. http://www.eclipsecon.org/osdn _______________________________________________ NFS maillist - [email protected] https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 2 lines · Chris Newbold
So, I'd like to find out what ever happened with this issue and if there ever were, or will be, any official fixes against the 2.4.x line...
FWIW, the Fedora Core 1 kernels contain this patch.
[But the FC1 kernels have other problems, namely hanging in umount. :-/]
Regards,
Bill Rugolsky
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004 Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs