Checksumming RAID?
Hi all
I see from an article at http://pages.cs.wisc.edu/~bpkroth/cs736/md-checksums/md-checksums-paper.pdf that an implementation has been made to allow for ZFS-like checksumming inside Linux MD. However, this code doesn't seem to exist in any kernel trees. Does anyone know the current status for data checksumming in MD?
Signature · 15 lines
Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 [email protected] http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 9 lines · Roy Sigurd Karlsbakk
Hi all
I see from an article at
http://pages.cs.wisc.edu/~bpkroth/cs736/md-checksums/md-checksums-paper.pdf
that an implementation has been made to allow for ZFS-like checksumming inside Linux MD. However, this code doesn't seem to exist in any kernel trees. Does anyone know the current status for data checksumming in MD?
See <http://neil.brown.name/blog/20110227114201> for a discussion on data checksums.
As far as I have seen on this mailing list, there has been no "official" work on checksums as described in that paper. I suspect it's just a matter of a student or two doing a project as part of their university degree. It's great that people can do that - they are free to take a copy of the kernel, and experiment with new ideas. If the ideas are good, then it is possible to work it back into the mainline kernel development.
However, in this case I think there is not much support for data checksumming amongst the "big boys" in this part of the Linux kernel - as explained by Neil in his blog post.
My first thought when reading the paper in question is that it doesn't really add much that is actually useful. md does not need checksums - it already has a more powerful system for error detection and correction through the parity blocks. If you want more checksumming than raid5 gives you, then use raid6.
What might be of interest for confirming the data integrity is so say that whenever a block is to be read, the stripe it is in should be scrubbed. This would enforce regular scrubbing of data that is regularly used, and give the same benefits as the article's data checksumming. It would lead to more disk reads when you have small reads, but the overhead would be small for larger reads or for RMW writes (since the whole stripe, minus the parity, is read in this case).
However, referring to another of Neil's blog posts at <http://neil.brown.name/blog/20100211050355>, you have to ask yourself how likely is it that data will be read from the drive with an error, but without the disk telling you of the error - and what can you sensibly do about it? You don't need checksums to tell you that there is a problem reading data from the disk - the disk already has very comprehensive checking of the data, and if that fails it will report an error and the md layer will re-construct the data from the parity and the rest of the stripe.
So before worrying about data checksums, please read Neil's posts, and try to think out scenarios where it really would help. And if you find you have a good argument, then post it here.
David
Quoted 41 lines · David Brown
Quoted 10 lines · Roy Sigurd Karlsbakk
Hi all
I see from an article at
http://pages.cs.wisc.edu/~bpkroth/cs736/md-checksums/md-checksums-paper.pdfthat an implementation has been made to allow for ZFS-like checksumming inside Linux MD. However, this code doesn't seem to exist in any kernel trees. Does anyone know the current status for data checksumming in MD?
See <http://neil.brown.name/blog/20110227114201> for a discussion on data checksums.
As far as I have seen on this mailing list, there has been no "official" work on checksums as described in that paper. I suspect it's just a matter of a student or two doing a project as part of their university degree. It's great that people can do that - they are free to take a copy of the kernel, and experiment with new ideas. If the ideas are good, then it is possible to work it back into the mainline kernel development.
However, in this case I think there is not much support for data checksumming amongst the "big boys" in this part of the Linux kernel - as explained by Neil in his blog post.
My first thought when reading the paper in question is that it doesn't really add much that is actually useful. md does not need checksums - it already has a more powerful system for error detection and correction through the parity blocks. If you want more checksumming than raid5 gives you, then use raid6.
What might be of interest for confirming the data integrity is so say that whenever a block is to be read, the stripe it is in should be scrubbed. This would enforce regular scrubbing of data that is regularly used, and give the same benefits as the article's data checksumming. It would lead to more disk reads when you have small reads, but the overhead would be small for larger reads or for RMW writes (since the whole stripe, minus the parity, is read in this case).
Quoted 9 lines
However, referring to another of Neil's blog posts at
<http://neil.brown.name/blog/20100211050355>, you have to ask yourself how likely is it that data will be read from the drive with an error, but without the disk telling you of the error - and what can you sensibly do about it? You don't need checksums to tell you that there is a problem reading data from the disk - the disk already has very comprehensive checking of the data, and if that fails it will report an error and the md layer will re-construct the data from the parity and the rest of the stripe.
Thats the theory, real live unfortunately teaches a different story. I just helped to recover as much data as possible from a troublesome infortrend raid system, which again is part of a software raid. The stupid hardware raid decided for unknown reasons to return different data on each read. And this is already the 4th or 5th time that happened (its a rather big installation and each time another hardware raid causes the trouble). And yes, I also aready have seen several hard disks to return wrong data. That is actually the reason why some hardware raid vendors such as DDN do parity reads all the time and then correct wrong data or entirely fail the disks.
I will sent patches to better handle parity mismatches during the next weeks (for performance reasons only for background checks).
Cheers,
Bernd
Quoted 65 lines · Bernd Schubert
Quoted 42 lines · David Brown
Quoted 11 lines · Roy Sigurd Karlsbakk
Hi all
I see from an article at
http://pages.cs.wisc.edu/~bpkroth/cs736/md-checksums/md-checksums-paper.pdfthat an implementation has been made to allow for ZFS-like checksumming inside Linux MD. However, this code doesn't seem to exist in any kernel trees. Does anyone know the current status for data checksumming in MD?
See <http://neil.brown.name/blog/20110227114201> for a discussion on data checksums.
As far as I have seen on this mailing list, there has been no "official" work on checksums as described in that paper. I suspect it's just a matter of a student or two doing a project as part of their university degree. It's great that people can do that - they are free to take a copy of the kernel, and experiment with new ideas. If the ideas are good, then it is possible to work it back into the mainline kernel development.
However, in this case I think there is not much support for data checksumming amongst the "big boys" in this part of the Linux kernel - as explained by Neil in his blog post.
My first thought when reading the paper in question is that it doesn't really add much that is actually useful. md does not need checksums - it already has a more powerful system for error detection and correction through the parity blocks. If you want more checksumming than raid5 gives you, then use raid6.
What might be of interest for confirming the data integrity is so say that whenever a block is to be read, the stripe it is in should be scrubbed. This would enforce regular scrubbing of data that is regularly used, and give the same benefits as the article's data checksumming. It would lead to more disk reads when you have small reads, but the overhead would be small for larger reads or for RMW writes (since the whole stripe, minus the parity, is read in this case).
Quoted 9 lines
However, referring to another of Neil's blog posts at
<http://neil.brown.name/blog/20100211050355>, you have to ask yourself how likely is it that data will be read from the drive with an error, but without the disk telling you of the error - and what can you sensibly do about it? You don't need checksums to tell you that there is a problem reading data from the disk - the disk already has very comprehensive checking of the data, and if that fails it will report an error and the md layer will re-construct the data from the parity and the rest of the stripe.Thats the theory, real live unfortunately teaches a different story. I just helped to recover as much data as possible from a troublesome infortrend raid system, which again is part of a software raid. The stupid hardware raid decided for unknown reasons to return different data on each read. And this is already the 4th or 5th time that happened (its a rather big installation and each time another hardware raid causes the trouble).
And yes, I also aready have seen several hard disks to return wrong data. That is actually the reason why some hardware raid vendors such as DDN do parity reads all the time and then correct wrong data or entirely fail the disks.
Quoted 7 lines
I will sent patches to better handle parity mismatches during the next weeks (for performance reasons only for background checks).
Cheers,
Bernd
I can certainly sympathise with you, but I am not sure that data checksumming would help here. If your hardware raid sends out nonsense, then it is going to be very difficult to get anything trustworthy. The obvious answer here is to throw out the broken hardware raid and use a system that works - but it is equally obvious that that is easier said than done! But I would find it hard to believe that this is a common issue with hardware raid systems - it goes against the whole point of data storage.
There is always a chance of undetected read errors - the question is if the chances of such read errors, and the consequences of them, justify the costs of extra checking. And if they /do/ justify extra checking, are data checksums the right way? I agree with Neil's post that end-to-end checksums (such as CRCs in a gzip file, or GPG integrity checks) are the best check when they are possible, but they are not always possible because they are not transparent.
mvh.,
David
Quoted 10 lines
I can certainly sympathise with you, but I am not sure that data checksumming would help here. If your hardware raid sends out nonsense,
then it is going to be very difficult to get anything trustworthy. The obvious answer here is to throw out the broken hardware raid and use a system that works - but it is equally obvious that that is easier said than done! But I would find it hard to believe that this is a common issue with hardware raid systems - it goes against the whole point of data storage.
Quoted 5 lines
There is always a chance of undetected read errors - the question is if
the chances of such read errors, and the consequences of them, justify the costs of extra checking. And if they /do/ justify extra checking, are data checksums the right way?
The chance of a silent corruption is rather small with your average 3TB home storage. On the other hand, if you had a petabyte or five, the chances would be very high indeed to get silent corruption (ref the CERN study done in 2007). In my last job, I worked with ZFS with ~350TiB storage, and there we saw errors happen rather frequently, but then, since ZFS checksums data and uses it to deal with errors, we never saw any data loss. That is, except on an older machine, running ZFS on a hardware RAID controlled storage unit (NexSAN SATABeast). We had error corruption on that one as well, after a disk failure, and had to resort to restoring from tape, since ZFS couldn't control the RAID.
Quoted 4 lines
I agree with Neil's post that
end-to-end checksums (such as CRCs in a gzip file, or GPG integrity checks) are the best check when they are possible, but they are not always possible because they are not transparent.
The problem with end-to-end-checksums at the application level, is it will only be able to detect the error, not fix it, similar to the issues I mentioned above.
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 14 lines · Roy Sigurd Karlsbakk
Quoted 9 lines
I can certainly sympathise with you, but I am not sure that data checksumming would help here. If your hardware raid sends out nonsense, then it is going to be very difficult to get anything trustworthy. The obvious answer here is to throw out the broken hardware raid and use a system that works - but it is equally obvious that that is easier said than done! But I would find it hard to believe that this is a common issue with hardware raid systems - it goes against the whole point of data storage.
Quoted 4 lines
There is always a chance of undetected read errors - the question is if the chances of such read errors, and the consequences of them, justify the costs of extra checking. And if they /do/ justify extra checking, are data checksums the right way?
Quoted 11 lines
The chance of a silent corruption is rather small with your average 3TB home storage. On the other hand, if you had a petabyte or five, the chances would be very high indeed to get silent corruption (ref the CERN study done in 2007). In my last job, I worked with ZFS with ~350TiB storage, and there we saw errors happen rather frequently, but then, since ZFS checksums data and uses it to deal with errors, we never saw any data loss. That is, except on an older machine, running ZFS on a hardware RAID controlled storage unit (NexSAN SATABeast). We had error corruption on that one as well, after a disk failure, and had to resort to restoring from tape, since ZFS couldn't control the RAID.
Of course even a small chance-per-bit turns into a significant total chance when you have enough bits! There is always a chance of undetected issues - your aim it to reduce that chance until it is no longer relevant (or until the chance is under 1 in 150 million per year - then you should worry more about being killed by lightning).
Quoted 10 lines
Quoted 5 lines
I agree with Neil's post that end-to-end checksums (such as CRCs in a gzip file, or GPG integrity checks) are the best check when they are possible, but they are not always possible because they are not transparent.
The problem with end-to-end-checksums at the application level, is it will only be able to detect the error, not fix it, similar to the issues I mentioned above.
Checksumming, as suggested by the originally mentioned paper, will not be able to correct anything either. At first glance, it might seem that it would tell you which block was wrong, and therefore let you re-build that block from the rest of the raid stripe. But that will not be the case if there are issues while writing, such as unexpected power failures - it could just as easily be the data blocks that are correctly written while the checksum block is wrong. And exactly as discussed in Neil's post on "smart" recovery, the principle of least surprise suggests giving the data blocks back unchanged is the least harmful.
To do checksumming (and in particular, recovery), requires higher level knowledge of the data. The filesystem can track when it writes a file, and update metadata (including, if desired, a data checksum) once it knows the file is correctly stored. But I don't think it can sensibly be done at the block device level - the recovery procedure doesn't know what is old data, what is new data, or which bit is important to the filesystem.
So I think it can make sense to use a filesystem like ZFS or BTRFS that can do checksumming - that is a reasonable level to add the checksum.
One way to handle this at md block level would be to have an option for raid arrays to always do a full stripe read and consistency check whenever a block is read. If the consistency check fails (without any errors being indicated from the drives), the array should simply return a read error - it should /not/ attempt to recover the data (since it can't tell which parts are the real problem). If arrays with this option are used as first-level arrays, with a "normal" md raid array (raid1, raid5, etc.) on top, then the normal raid recovery process will replace the bad data and initiate a new write to correct the undetected read error. I think this would perhaps give you the level of reliability you are looking for, and be suitable for big arrays (indeed, it would be unsuitable for small arrays as you need at least two levels).
mvh.,
David
Quoted 10 lines
One way to handle this at md block level would be to have an option for
raid arrays to always do a full stripe read and consistency check whenever a block is read. If the consistency check fails (without any errors being indicated from the drives), the array should simply return
a read error - it should /not/ attempt to recover the data (since it can't tell which parts are the real problem). If arrays with this option are used as first-level arrays, with a "normal" md raid array (raid1, raid5, etc.) on top, then the normal raid recovery process
Quoted 8 lines
will
replace the bad data and initiate a new write to correct the undetected
read error. I think this would perhaps give you the level of reliability you are looking for, and be suitable for big arrays (indeed,
it would be unsuitable for small arrays as you need at least two levels).
If this system is running RAID-6, recovery should be possible to check both parity chunks, right?
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 15 lines · Roy Sigurd Karlsbakk
Quoted 6 lines
One way to handle this at md block level would be to have an option for raid arrays to always do a full stripe read and consistency check whenever a block is read. If the consistency check fails (without any errors being indicated from the drives), the array should simply return a read error - it should /not/ attempt to recover the data (since it can't tell which parts are
Quoted 8 lines
the real problem). If arrays with this option are used as first-level arrays, with a "normal" md raid array (raid1, raid5, etc.) on top, then the normal raid recovery process will replace the bad data and initiate a new write to correct the undetected read error. I think this would perhaps give you the level of reliability you are looking for, and be suitable for big arrays (indeed, it would be unsuitable for small arrays as you need at least two levels).
Quoted 2 lines
If this system is running RAID-6, recovery should be possible to check both parity chunks, right?
Yes, of course. (And if anyone ever needs it, it is possible to extend raid6 to 3 parity chunks. I've done the maths, but it is not implemented - there doesn't seem to be a big need for it.) But - again referring back to Neil's blog - if the low-level raid spots a consistency error, it still cannot correct it reliably even with 2 parity chunks, and should pass on a read error to the higher level raid.
Using raid6 at the low level would let you do a good consistency check
even in the case of a failed drive (or a known read error on a drive) - or two simultaneous undetected read errors. And raid6 on the higher level raid would let you correct such errors, even when there are other errors around. You'd soon reach the point where it is more likely for your disks to spontaneously turn into a bowl of petunias than for read errors to be undetected or unrecoverable.
Quoted 12 lines
Vennlige hilsener / Best regards
roy -- Roy Sigurd Karlsbakk (+47) 98013356 [email protected]
http://blogg.karlsbakk.net/ GPG Public key:
http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 19 lines
Quoted 2 lines
If this system is running RAID-6, recovery should be possible to check both parity chunks, right?
Yes, of course. (And if anyone ever needs it, it is possible to extend raid6 to 3 parity chunks. I've done the maths, but it is not implemented - there doesn't seem to be a big need for it.) But - again referring back to Neil's blog - if the low-level raid spots a consistency error, it still cannot correct it reliably even with 2 parity chunks, and should pass on a read error to the higher level raid.
Using raid6 at the low level would let you do a good consistency check even in the case of a failed drive (or a known read error on a drive) -
or two simultaneous undetected read errors. And raid6 on the higher level raid would let you correct such errors, even when there are other
errors around. You'd soon reach the point where it is more likely for your disks to spontaneously turn into a bowl of petunias than for read errors to be undetected or unrecoverable.
That would be nice. So what should be done here in the first place, is to change the code to allow parity data to be read and calculated also on reads?
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 18 lines · Roy Sigurd Karlsbakk
Quoted 17 lines
Quoted 2 lines
If this system is running RAID-6, recovery should be possible to check both parity chunks, right?
Yes, of course. (And if anyone ever needs it, it is possible to extend raid6 to 3 parity chunks. I've done the maths, but it is not implemented - there doesn't seem to be a big need for it.) But
- again referring back to Neil's blog - if the low-level raid spots a consistency error, it still cannot correct it reliably even with 2 parity chunks, and should pass on a read error to the higher level raid. Using raid6 at the low level would let you do a good consistency check even in the case of a failed drive (or a known read error on a drive) - or two simultaneous undetected read errors. And raid6 on the higher level raid would let you correct such errors, even when there are other errors around. You'd soon reach the point where it is more likely for your disks to spontaneously turn into a bowl of petunias than for read errors to be undetected or unrecoverable.
Quoted 3 lines
That would be nice. So what should be done here in the first place, is to change the code to allow parity data to be read and calculated also on reads?
Well, what should be done /first/ is to hope that some of the more experienced md raid experts express an opinion on the idea - is it possible, is it useful, and is it practical to implement?
The main aim would be to add an option to a md arrays that will turn each read into an implicit scrub or check of the whole stripe, and that a consistency error there would return a read error to the next layer of md raid.
I can see plenty of scope for complications here, such as what to do on normal (detected) read errors, or how to ensure that the upper layer re-writes the whole stripe and not just part of it (or perhaps partial re-writes would be enough). I am fully aware that I'm just giving a rough idea here - it needs a lot more thought before anyone can start changing code. But if my theory here is correct, and if it is practical to implement, then it might be a useful tool for big data producers.
mvh.,
David
Quoted 4 lines
Vennlige hilsener / Best regards
roy
Quoted 10 lines · David Brown
To do checksumming (and in particular, recovery), requires higher level knowledge of the data. The filesystem can track when it writes a file, and update metadata (including, if desired, a data checksum) once it knows the file is correctly stored. But I don't think it can sensibly be done at the block device level - the recovery procedure doesn't know what is old data, what is new data, or which bit is important to the filesystem.
So I think it can make sense to use a filesystem like ZFS or BTRFS that can do checksumming - that is a reasonable level to add the checksum.
You'll see CRC in XFS in the future as well. Some of the foundation is already laid to allow it, but IIRC it requires an on disk format change for full implementation. On disk format changes are a big deal and are taken with great care. IIRC XFS has only seen one or two in 18 years.
Signature · 1 line
Stan
Quoted 17 lines
Quoted 12 lines
To do checksumming (and in particular, recovery), requires higher level
knowledge of the data. The filesystem can track when it writes a file,
and update metadata (including, if desired, a data checksum) once it knows the file is correctly stored. But I don't think it can sensibly
be done at the block device level - the recovery procedure doesn't know
what is old data, what is new data, or which bit is important to the filesystem.Quoted 4 lines
So I think it can make sense to use a filesystem like ZFS or BTRFS that
can do checksumming - that is a reasonable level to add the checksum.
Quoted 5 lines
You'll see CRC in XFS in the future as well. Some of the foundation is already laid to allow it, but IIRC it requires an on disk format change
for full implementation. On disk format changes are a big deal and are taken with great care. IIRC XFS has only seen one or two in 18 years.
I'm afraid that won't help much either, since it'll only allow for *detecting* the errors, and not *fixing* them (as with ZFS and Btrfs). Or perhaps if XFS can integrate with MD somehow?
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 3 lines · David Brown
I can certainly sympathise with you, but I am not sure that data checksumming would help here. If your hardware raid sends out nonsense, then it is going to be very difficult to get anything trustworthy. The
When a single hardware unit (any kind of block device) in a
raid-level > 0 decides to send wrong data, correct data always can be reconstructed. You only need to know which unit it is - checksums help to figure that out.
Quoted 5 lines
obvious answer here is to throw out the broken hardware raid and use a system that works - but it is equally obvious that that is easier said than done! But I would find it hard to believe that this is a common issue with hardware raid systems - it goes against the whole point of data storage.
With disks it is not that uncommon. But yes, hardware raid controllers usually do not scramble data.
Quoted 8 lines
There is always a chance of undetected read errors - the question is if the chances of such read errors, and the consequences of them, justify the costs of extra checking. And if they /do/ justify extra checking, are data checksums the right way? I agree with Neil's post that end-to-end checksums (such as CRCs in a gzip file, or GPG integrity checks) are the best check when they are possible, but they are not always possible because they are not transparent.
Everything below block or filesystem level is too late. Just remember, writing not a complete stripe implies reads in order to update the p and q parity blocks. So even if your application could later on detect that (Do your applications usually verify checksums? In HPC I don't know of a single application to do that...), file system meta data already would be broken.
Cheers,
Bernd
Quoted 9 lines · Bernd Schubert
Quoted 3 lines · David Brown
I can certainly sympathise with you, but I am not sure that data checksumming would help here. If your hardware raid sends out nonsense, then it is going to be very difficult to get anything trustworthy. The
When a single hardware unit (any kind of block device) in a raid-level > 0 decides to send wrong data, correct data always can be reconstructed. You only need to know which unit it is - checksums help to figure that out.
If checksums (as described in the paper) only "help" to figure that out, then they are not good enough - you can only do automatic on-the-fly correction if you are /sure/ you know which device is the problem (at least for a very high probability of "sure"). I think that adding an extra checksum block to the stripe only gives an indication of the problem disk (or lower-level raid) - without being sure of the order that data hits the different disks (or lower-level raids), I don't think it is reliable enough. (I could be wrong in all this - I'm just waving around ideas, and have no experience with big arrays.)
Quoted 9 lines
Quoted 6 lines
obvious answer here is to throw out the broken hardware raid and use a system that works - but it is equally obvious that that is easier said than done! But I would find it hard to believe that this is a common issue with hardware raid systems - it goes against the whole point of data storage.
With disks it is not that uncommon. But yes, hardware raid controllers usually do not scramble data.
With disks it /is/ uncommon. /Detected/ disk errors are not a problem - the disks's own ECC system finds it has an unrecoverable error, and returns a read error, and the raid system replaces the data using the rest of the stripe. It is /undetected/ disk errors that are a problem.
Typical figures I have seen are around 1 in 1e12 4KB blocks - or 1 in
3e16 bits. If you've got a 1 PB disk array, that's one error for every four full reads - which is certainly enough to be relevant, but I wouldn't say it is "not that uncommon".
Quoted 10 lines
Quoted 9 lines
There is always a chance of undetected read errors - the question is if the chances of such read errors, and the consequences of them, justify the costs of extra checking. And if they /do/ justify extra checking, are data checksums the right way? I agree with Neil's post that end-to-end checksums (such as CRCs in a gzip file, or GPG integrity checks) are the best check when they are possible, but they are not always possible because they are not transparent.
Quoted 7 lines
Everything below block or filesystem level is too late. Just remember, writing not a complete stripe implies reads in order to update the p and q parity blocks. So even if your application could later on detect that (Do your applications usually verify checksums? In HPC I don't know of a single application to do that...), file system meta data already would be broken.
When you say "below block or filesystem level", I presume you mean such as "application level"? I always think of that as above the filesystem, which is above the block level. I certainly agree that it is often not practical to verify checksums at the application level.
As I mentioned in another post, I think there are times when filesystem checksumming can make sense. I also described another idea at block level - I am curious as to what you think of that.
mvh.,
David
Quoted 2 lines · David Brown
It is /undetected/ disk errors that are a problem. Typical figures I have seen are around 1 in 1e12 4KB blocks - or 1 in 3e16 bits.
Is this with consumer SATA? Or other?
Chris Murphy
----- Opprinnelig melding -----
Quoted 7 lines
Quoted 3 lines · David Brown
It is /undetected/ disk errors that are a problem. Typical figures I have seen are around 1 in 1e12 4KB blocks - or 1 in 3e16 bits.
Is this with consumer SATA? Or other?
According to a few studies I've read, the number of silent errors are identical on both consumer SATA drives and SAS drives. It's the density that makes errors, and the platters and disk heads are produced alike.
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 10 lines · Roy Sigurd Karlsbakk
----- Opprinnelig melding -----
Quoted 7 lines
Quoted 3 lines · David Brown
It is /undetected/ disk errors that are a problem. Typical figures I have seen are around 1 in 1e12 4KB blocks - or 1 in 3e16 bits.
Is this with consumer SATA? Or other?
According to a few studies I've read, the number of silent errors are identical on both consumer SATA drives and SAS drives. It's the density that makes errors, and the platters and disk heads are produced alike.
That's not what I'm reading:
Corruption detected in 8.5% of nearline disks, and 1.9% in enterprise disks.
http://www.pdsi-scidac.org/events/PDSW09/resources/pdsw09_slides13.pdf
In this study it's 0.86% of nearline, and 0.065% of enterprise disks developing checksum mismatches, out of 1.53 million disks.
http://static.usenix.org/event/fast08/tech/full_papers/bairavasundaram/bairavasundaram.pdf
The ratios are the same.
Chris Murphy
Quoted 10 lines
Quoted 4 lines
According to a few studies I've read, the number of silent errors are identical on both consumer SATA drives and SAS drives. It's the density that makes errors, and the platters and disk heads are produced alike.
That's not what I'm reading:
Corruption detected in 8.5% of nearline disks, and 1.9% in enterprise disks.
http://www.pdsi-scidac.org/events/PDSW09/resources/pdsw09_slides13.pdf
That doesn't make sense. Nearline and enterprise drives are large 7k2 drives that come in consumer and enterprise models, such as Hitachi Deskstar and Ultrastar. The latter is said to have better bearings etc, although I don't know the difference for sure. With WDs large drives, most of the difference is said to be in the firmware (such as TLER in the enterprise/RAID drives). Comparing "nearline" and "enterprise" like done here, is merely a comparison of small and large drives, where it's well known that the smaller (less dense) drives have less errors.
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 3 lines
That doesn't make sense. Nearline and enterprise drives are large 7k2 drives that come in consumer and enterprise models, such as Hitachi Deskstar and Ultrastar.
Small typo here. Change to "Nearline drives are large 7k2 drives that come in consumer and enterprise models…"
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 2 lines
That doesn't make sense. Nearline and enterprise drives are large 7k2 drives that come in consumer and enterprise models, such as Hitachi Deskstar and Ultrastar. The latter is said to have better bearings etc, although I don't know the difference for sure. With WDs large drives, most of the difference is said to be in the firmware (such as TLER in the enterprise/RAID drives). Comparing "nearline" and "enterprise" like done here, is merely a comparison of small and large drives, where it's well known that the smaller (less dense) drives have less errors.
Less dense but also the drives are probably the 10k & 15k RPM drives which have to be built to better tolerances given the expectation they'll be spinning for most of their five year life.
Signature · 7 lines
Drew "Nothing in life is to be feared. It is only to be understood." --Marie Curie "This started out as a hobby and spun horribly out of control." -Unknown
Quoted 11 lines
Quoted 10 lines
That doesn't make sense. Nearline and enterprise drives are large 7k2 drives that come in consumer and enterprise models, such as Hitachi Deskstar and Ultrastar. The latter is said to have better bearings etc, although I don't know the difference for sure. With WDs large drives, most of the difference is said to be in the firmware (such as TLER in the enterprise/RAID drives). Comparing "nearline" and "enterprise" like done here, is merely a comparison of small and large drives, where it's well known that the smaller (less dense) drives have less errors.
Quoted 3 lines
Less dense but also the drives are probably the 10k & 15k RPM drives which have to be built to better tolerances given the expectation they'll be spinning for most of their five year life.
So what you're saying is that the "enterprise" drives that spin on 7k2 are just overpriced desktop drives? That really doesn't make sense. In my vocabolary, "nearline" means "slower", not worse. It'd be really interesting to get some opinions from someone that actually knows this, or perhaps some real-world statistics. Google had a study some years ago, which at that point showed desktop drives and enterprise drives had about the same error rate. That may be slightly outdated now, but still…
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 2 lines · Roy Sigurd Karlsbakk
Google had a study some years ago, which at that point showed desktop drives and enterprise drives had about the same error rate. That may be slightly outdated now, but still…
The only Google drive study I'm aware of is this one:
http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/disk_failures.pdf
That study is about drive failures, not SDC. They aren't looking at corruption, but rather SMART errors relating to disk failure. And the disks are consumer grade, none were enterprise.
The main finding of the study is the correlation (or lack thereof) of SMART prediction of drive failure (the health status of the drive) to reality. It turns out SMART is not so great at doing this if you only trust the health status. You can get better prediction of drive failure by taking specific attributes into account, but even this isn't completely reliable. New attributes are needed to better predict drive failures is the take away.
Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · Roy Sigurd Karlsbakk
That doesn't make sense. Nearline and enterprise drives are large 7k2 drives that come in consumer and enterprise models,
Nearline drives are in between consumer and enterprise.
Quoted 1 line
Comparing "nearline" and "enterprise" like done here, is merely a comparison of small and large drives, where it's well known that the smaller (less dense) drives have less errors.
The paper I cited doesn't support this. The nearline SATA with hardware adapters to convert the interface to fibre channel. The enterprise drives were already fibre channel. The paper discusses the contribution of the adapter to checksum mismatches, but this alone doesn't account for all of the higher rate of error in nearline SATA.
Further, the paper says "There is no clear indication that disk size affects the probability of developing checksum mis- matches."
At least when it comes to URE (which is not SDC), we note that the manufacturer specs are the same for a model regardless of disk size.
Chris Murphy
Quoted 4 lines
Quoted 2 lines
That doesn't make sense. Nearline and enterprise drives are large 7k2 drives that come in consumer and enterprise models,
Nearline drives are in between consumer and enterprise.
Well, they are named enterprise…
Quoted 9 lines
Quoted 3 lines
Comparing "nearline" and "enterprise" like done here, is merely a comparison of small and large drives, where it's well known that the smaller (less dense) drives have less errors.
The paper I cited doesn't support this. The nearline SATA with hardware adapters to convert the interface to fibre channel. The enterprise drives were already fibre channel. The paper discusses the contribution of the adapter to checksum mismatches, but this alone doesn't account for all of the higher rate of error in nearline SATA.
7k2 "nearline" SAS drives exist from several vendors, with SAS interfaces without the need for adapters. Also, the interface, wheather SAS or SATA or FC, doesn't really mean much when the drive is the same. SAS (and FC?) has better timing and better controllers, but a SAS drive is a SAS drive nonetheless if it has a SAS interface.
Quoted 5 lines
Further, the paper says "There is no clear indication that disk size affects the probability of developing checksum mis- matches."
At least when it comes to URE (which is not SDC), we note that the manufacturer specs are the same for a model regardless of disk size.
Then, if this is true, perhaps they should name their 4TB drives something else than "enterprise", which they do. Do you have more references on this part?
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · Roy Sigurd Karlsbakk
Quoted 4 lines
Quoted 2 lines
That doesn't make sense. Nearline and enterprise drives are large 7k2 drives that come in consumer and enterprise models,
Nearline drives are in between consumer and enterprise.
Well, they are named enterprise…
Yep it's misleading. SNIA has a slide show describing the differences between consumer nearline and enterprise.
Quoted 1 line
7k2 "nearline" SAS drives exist from several vendors, with SAS interfaces without the need for adapters. Also, the interface, wheather SAS or SATA or FC, doesn't really mean much when the drive is the same. SAS (and FC?) has better timing and better controllers, but a SAS drive is a SAS drive nonetheless if it has a SAS interface.
Nearline SAS is effectively a consumer SATA drive (you don't get the better mechanics), with a SAS interface.
Quoted 1 line
Then, if this is true, perhaps they should name their 4TB drives something else than "enterprise", which they do. Do you have more references on this part?
Basically you have to look at the specs. Maybe a reasonable rule of thumb is the URE, but even that gets perverted sometimes. < 1 bit in 1E14 is consumer, 1E15 is nearline, 1E16 is enterprise.
For example:
http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-771386.pdf
<10 in 10^16 = <1 in 10^15, means this is nearline. It's clearly the same as the RE SATA in every way (all mechanical specs) except that it's a SAS interface.
Whereas this:
http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-771429.pdf
<10 in 10^17 = <1 in 10^16 is enterprise.
Notice the size and speed difference, which is where "nearline" is being used. More capacity and slower than enterprise makes it nearline, more so than the URE. But URE makes it easier to distinguish between consumer and nearline.
Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · David Brown
On 27/11/2012 11:17, Bernd Schubert wrote:
[...]
Quoted 5 lines
Quoted 5 lines
I will sent patches to better handle parity mismatches during the next weeks (for performance reasons only for background checks).
Cheers,
Bernd
Give me a heads up when they are ready, and I can get some testing in for you.
Quoted 5 lines
Quoted 2 lines
I can certainly sympathise with you, but I am not sure that data checksumming would help here. If your hardware raid sends out nonsense,
Well, unfortunately, Bernd (and DDN et al) are right, it is helpful. It has to be engineered correctly to be of use. T10-DIF and PI are efforts in this direction. This can be implemented in software.
Quoted 3 lines
then it is going to be very difficult to get anything trustworthy. The obvious answer here is to throw out the broken hardware raid and use a system that works - but it is equally obvious that that is easier said
... which is not so obvious when you start dealing with hundreds to TB and PB of data, and you have hardware which works perfectly most of the time (apart from a random cosmic ray, power fluctuation/surge, ...)
Quoted 3 lines
than done! But I would find it hard to believe that this is a common issue with hardware raid systems - it goes against the whole point of data storage.
http://storagemojo.com/2007/09/19/cerns-data-corruption-research/
(and no, I don't work for Robin, he largely ignores us :( )
Quoted 3 lines
There is always a chance of undetected read errors - the question is if the chances of such read errors, and the consequences of them, justify the costs of extra checking. And if they /do/ justify extra checking,
I guess the real question is, how valuable is your data ... if you took the trouble to store it, I am guessing that you'd like to know a) its stored correctly, b) it is retrievable, and c) what you retrieve is correct.
Hardware (and software) RAID help with b, and sometimes a. C is what T10-DIF/PI and related are trying to solve.
Quoted 4 lines
are data checksums the right way? I agree with Neil's post that end-to-end checksums (such as CRCs in a gzip file, or GPG integrity checks) are the best check when they are possible, but they are not always possible because they are not transparent.
I personally would like to push the checking more into the file system layers than the disk block layers. Though I expect strong resistance to that as well. File systems assume perfectly operating underlying storage blocks in most cases, and breaking that model (or breaking it any more than we are doing now) would be troubling to many.
Adding in crc verify on read (and crc generation and storage on write) shouldn't be too painful at the block layer. Much of the infrastructure is in place. I've been thinking of building a pluggable connection into MD, so we could experiment with different mechanisms (w/o rebuilding the kernel or MD each time). Though this is (unfortunately) pretty far down our list of priorities at the moment.
Quoted 10 lines
mvh.,
David
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · David Brown
My first thought when reading the paper in question is that it doesn't really add much that is actually useful. md does not need checksums - it already has a more powerful system for error detection and correction through the parity blocks. If you want more checksumming than raid5 gives you, then use raid6.
parity != checksum
Reading a data block in, computing a checksum and comparing it to a previously written checksum for that same data block, is much less expensive than the choices you have with parity. Reading a data block in, computing parity, reading previously written parity, comparing the two; or, reading data block, and using parity to reconstruct data, and comparing the two data streams.
Further in the case of raid5 single parity, it's ambiguous whether a bit error is attributable to parity or the data. Which is wrong? You don't actually know. With raid6 dual parity, there is no ambiguity if both sets of parity are used to reconstruct data and are compared to each other rather than one being blindly trusted, or more cheaply is to reconstruct the data once from either set of parity, compute a checksum and compare to the written checksum.
So there is still a role for checksums, the question is whether this is better or equally managed at a file system level vs an md level. The checksumming with RAID applications today are file system level in ZFS and btrfs (and maybe ReFS).
Chris Murphy
Quoted 1 line · Chris Murphy
Further in the case of raid5 single parity, it's ambiguous whether a bit error is attributable to parity or the data. Which is wrong? You don't actually know. With raid6 dual parity, there is no ambiguity if both sets of parity are used to reconstruct data and are compared to each other rather than one being blindly trusted, or more cheaply is to reconstruct the data once from either set of parity, compute a checksum and compare to the written checksum.
Also a parity only solution, is in effect a RAID 6+ (dual+ parity) solution. RAID5 and lower are left out. And since RAID 10 and RAID 1+linear are scalable and parity solutions aren't, I think that a solution that works for them from the outset needs consideration.
Chris Murphy
Quoted 4 lines · Roy Sigurd Karlsbakk
Hi all
I see from an article at http://pages.cs.wisc.edu/~bpkroth/cs736/md-checksums/md-checksums-paper.pdf that an implementation has been made to allow for ZFS-like checksumming inside Linux MD. However, this code doesn't seem to exist in any kernel trees. Does anyone know the current status for data checksumming in MD?
Afaik Linux md-raid raid0 and raid1 already supports T10 PI (Protection Information) and DIF (Data Integrity Fields) / DIX (Data Integrity Extensions).
I wonder if it'd be possible to utilize this mdadm code with normal disks that don't have T10 PI support? ie. implement custom checksums in the "backend" with normal non-PI disks..
T10 PI and DIF/DIX is a SCSI/SAS feature, so it's mostly available in new(ish) enterprise SAS disks and it's not available in SATA.. but I think there are plans to implement something similar in SATA aswell in the future.
A couple of SAS HBAs (mpt2sas, for example) and some FC HBAs support DIF/DIX in Linux.
The purpose of DIF/DIX is to allow passing and verifying end-to-end checksums from all the way from the applications to the disks.. and probably in more common case that'd be checksums all the way from filesystems to the disks.
Some more links about T10 PI and DIF/DIX:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob_plain;f=Documentation/block/data-integrity.txt;hb=HEAD
https://oss.oracle.com/~mkp/docs/dix.pdf
https://oss.oracle.com/~mkp/docs/osd2008-data-integrity.pdf
https://oss.oracle.com/~mkp/docs/ols2008-petersen.pdf
https://oss.oracle.com/~mkp/docs/ppdc.pdf
-- Pasi
Re: Checksumming RAID? / SCSI SAS T10 PI and DIF/DIX / T13 SATA EPP
Quoted 12 lines · Pasi Kärkkäinen
Quoted 4 lines · Roy Sigurd Karlsbakk
Hi all
I see from an article at http://pages.cs.wisc.edu/~bpkroth/cs736/md-checksums/md-checksums-paper.pdf that an implementation has been made to allow for ZFS-like checksumming inside Linux MD. However, this code doesn't seem to exist in any kernel trees. Does anyone know the current status for data checksumming in MD?
Afaik Linux md-raid raid0 and raid1 already supports T10 PI (Protection Information) and DIF (Data Integrity Fields) / DIX (Data Integrity Extensions).
I wonder if it'd be possible to utilize this mdadm code with normal disks that don't have T10 PI support? ie. implement custom checksums in the "backend" with normal non-PI disks..
T10 PI and DIF/DIX is a SCSI/SAS feature, so it's mostly available in new(ish) enterprise SAS disks and it's not available in SATA.. but I think there are plans to implement something similar in SATA aswell in the future.
Now I remember.. it's the SATA T13 "External Path Protection" (EPP).
(Added CC to Martin in the case he has some thoughts about generic Linux checksumming RAID without T10 PI disks..)
-- Pasi
Quoted 12 lines
A couple of SAS HBAs (mpt2sas, for example) and some FC HBAs support DIF/DIX in Linux.
The purpose of DIF/DIX is to allow passing and verifying end-to-end checksums from all the way from the applications to the disks.. and probably in more common case that'd be checksums all the way from filesystems to the disks.
Some more links about T10 PI and DIF/DIX:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob_plain;f=Documentation/block/data-integrity.txt;hb=HEAD
https://oss.oracle.com/~mkp/docs/dix.pdf
https://oss.oracle.com/~mkp/docs/osd2008-data-integrity.pdf
https://oss.oracle.com/~mkp/docs/ols2008-petersen.pdf
https://oss.oracle.com/~mkp/docs/ppdc.pdf
Quoted 2 lines
-- Pasi
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Checksumming RAID? / SCSI SAS T10 PI and DIF/DIX / T13 SATA EPP
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Pasi" == Pasi Kärkkäinen <[email protected]> writes:
Pasi> Now I remember.. it's the SATA T13 "External Path Protection" Pasi> (EPP).
EPP is dead. So is SATA. Not going to happen.
Pasi> (Added CC to Martin in the case he has some thoughts about generic Pasi> Linux checksumming RAID without T10 PI disks..)
There have been a few attempts at a checksumming DM target. However, I think btrfs is a much better solution for this stuff.
Signature · 5 lines
Martin K. Petersen Oracle Linux Engineering -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Checksumming RAID? / SCSI SAS T10 PI and DIF/DIX / T13 SATA EPP
Quoted 6 lines · Martin K. Petersen
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Pasi" == Pasi Kärkkäinen <[email protected]> writes:
Pasi> (Added CC to Martin in the case he has some thoughts about generic Pasi> Linux checksumming RAID without T10 PI disks..)
There have been a few attempts at a checksumming DM target. However, I think btrfs is a much better solution for this stuff.
I think there's room for both. Checksumming at the block level, below md RAID so presumably in a DM target, could help avoid silent data corruption in such a way that the md layer could reconstruct valid data. Checksumming at the filesystem level doesn't give you reconstruction - unless you add redundancy/RAID functions into the filesystem, which is a whole other discussion.
Cheers,
John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html