NFS corruption (duplicated data)
I really don't understand what could be causing this, but it happens on several machine and at least on kernels 2.4.22, 2.4.25, 2.4.26. NFS v3 : hard, udp, rsize=8192,wsize=8192
local filesystems are XFS
Trond, this is data corruption not dropped packets so the protocol being UDP is not the problem.
Here is what is happening :
Copying a file of offsets from machine A to machine B over NFS and then comparing the file on B with the file on A over NFS, the file on machine B is corrupted in the following ways.
Usually, data earlier in the file will show up again later. For example :
57344 bytes of data from 672190464-672247807 is also in positions 1449664512-1449721855
sometimes, data later in the file is dupped to a position before it should be
53248 bytes of data from 1197158400-1197211647 is also in positions 1036660736-1036713983
Any ideas
Andy
På ty , 08/06/2004 klokka 11:44, skreiv Andy:
Quoted 4 lines
I really don't understand what could be causing this, but it happens on several machine and at least on kernels 2.4.22, 2.4.25, 2.4.26. NFS v3 : hard, udp, rsize=8192,wsize=8192
local filesystems are XFS
Does it occur on non-XFS partitions?
Cheers,
Trond
Hi Andy,
Be good to try this with files served from ext2/3 as well, to try isolate it to XFS/NFS. We have a known issue thats possibly related to this in XFS - Russell, does this sound like that problem you've been looking at?
If you have a simple test case to reproduce it (we have an extremely complex test case to reproduce that other issue, but from your description I'm not sure its the same), that would be very helpful Andy.
thanks.
Quoted 34 lines · Andy
I really don't understand what could be causing this, but it happens on several machine and at least on kernels 2.4.22, 2.4.25, 2.4.26. NFS v3 : hard, udp, rsize=8192,wsize=8192
local filesystems are XFSTrond, this is data corruption not dropped packets so the protocol being UDP is not the problem.
Here is what is happening :
Copying a file of offsets from machine A to machine B over NFS and then comparing the file on B with the file on A over NFS, the file on machine B is corrupted in the following ways.
Usually, data earlier in the file will show up again later. For example :
57344 bytes of data from 672190464-672247807 is also in positions 1449664512-1449721855
sometimes, data later in the file is dupped to a position before it should be
53248 bytes of data from 1197158400-1197211647 is also in positions 1036660736-1036713983
Any ideas
Andy
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
Nathan
Ya it's probably the same problem.
To really understand what is going on in terms of corruption pattern up the disk with a known pattern and re-run the test.
If the corrupted areas show up as the known pattern you are dealing with stale disk data and not data from the wrong file/process.
And at that point I would definitely say it's bug:
http://oss.sgi.com/bugzilla/show_bug.cgi?id=198
Quoted 53 lines · Nathan Scott
Hi Andy,
Be good to try this with files served from ext2/3 as well, to try isolate it to XFS/NFS. We have a known issue thats possibly related to this in XFS - Russell, does this sound like that problem you've been looking at?
If you have a simple test case to reproduce it (we have an extremely complex test case to reproduce that other issue, but from your description I'm not sure its the same), that would be very helpful Andy.
thanks.
Quoted 34 lines · Andy
I really don't understand what could be causing this, but it happens on several machine and at least on kernels 2.4.22, 2.4.25, 2.4.26. NFS v3 : hard, udp, rsize=8192,wsize=8192
local filesystems are XFSTrond, this is data corruption not dropped packets so the protocol being UDP is not the problem.
Here is what is happening :
Copying a file of offsets from machine A to machine B over NFS and then comparing the file on B with the file on A over NFS, the file on machine B is corrupted in the following ways.
Usually, data earlier in the file will show up again later. For example :
57344 bytes of data from 672190464-672247807 is also in positions 1449664512-1449721855
sometimes, data later in the file is dupped to a position before it should be
53248 bytes of data from 1197158400-1197211647 is also in positions 1036660736-1036713983
Any ideas
Andy
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/Signature · 1 line
Nathan
Quoted 12 lines · Nathan Scott
Hi Andy,
Be good to try this with files served from ext2/3 as well, to try isolate it to XFS/NFS. We have a known issue thats possibly related to this in XFS - Russell, does this sound like that problem you've been looking at?
If you have a simple test case to reproduce it (we have an extremely complex test case to reproduce that other issue, but from your description I'm not sure its the same), that would be very helpful Andy.
Quoted 30 lines
thanks.
Quoted 26 lines · Andy
I really don't understand what could be causing this, but it happens on several machine and at least on kernels 2.4.22, 2.4.25, 2.4.26. NFS v3 : hard, udp, rsize=8192,wsize=8192
local filesystems are XFSTrond, this is data corruption not dropped packets so the protocol being UDP is not the problem.
Here is what is happening :
Copying a file of offsets from machine A to machine B over NFS and then comparing the file on B with the file on A over NFS, the file on machine B is corrupted in the following ways.
Usually, data earlier in the file will show up again later. For example :
57344 bytes of data from 672190464-672247807 is also in positions 1449664512-1449721855
sometimes, data later in the file is dupped to a position before it should be
53248 bytes of data from 1197158400-1197211647 is also in positions 1036660736-1036713983
Are you performing other IO on the NFS system while this copy is occuring? Are you copying the same file over and over to try and cause this problem?
Is it possible to zero all or as much disk as possible? If you are copying the same file over and over, you might be seeing old data on the disk and not necessarily seeing the filesystem putting the data in the wrong place. This might help isolate the problem to the one Russell is working on vs. a different problem.
Craig