Data integrity and RAID
I'm looking for a software solution to help replace my existing raid hardware setup. The hardware mimics RAID-1 (in that it has 3 mirrors), except for the fact that on read all 3 mirrors are compared and possibly error corrected before data is returned. I don't neccessarily need a RAID-1 solution, it just seems closer to what I already have and also that it also can recover from 2 simulateous disk failures.
So far I've played with a software RAID-1 array of USB flash drives. My issue is that software RAID-1 does not check for or recover from data corruption unless a read or write to disk actually fails. Integrity is a major concern for me, I need to know that all data going to and from disk is correct at all times.
All advice and comments are welcomed.
Signature · 2 lines
View this message in context: http://www.nabble.com/Data-integrity-and-RAID-tp20841008p20841008.html Sent from the linux-raid mailing list archive at Nabble.com.
Quoted 8 lines · peter.stevens
I'm looking for a software solution to help replace my existing raid hardware setup. The hardware mimics RAID-1 (in that it has 3 mirrors), except for the fact that on read all 3 mirrors are compared and possibly error corrected before data is returned. I don't neccessarily need a RAID-1 solution, it just seems closer to what I already have and also that it also can recover from 2 simulateous disk failures.
Quoted 7 lines
So far I've played with a software RAID-1 array of USB flash drives. My issue is that software RAID-1 does not check for or recover from data corruption unless a read or write to disk actually fails. Integrity is a major concern for me, I need to know that all data going to and from disk is correct at all times.
All advice and comments are welcomed.
Well USB flash isn't a traditional disk that remaps bad sectors etc.. AFAIK md/raid will try and correct read-errors, but for write errors you have to trust the disks to do this for you, otherwise the bad disk will be kicked out of the array. If you need the bad write to be re-mapped elsewhere in the array you need a hw raid controller, 3ware, etc..
Justin.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · Ryan Wagoner
The issue I've had with RAID 1 using mdadm is the disk's aren't consistent. I've noticed that unlike RAID 5 when a write is aborted mdadm doesn't completely write the data.
Quoted 8 lines
When I run a check on the RAID 1 I always have inconsistencies where I have never had one on a RAID 5. The issue here is you never know if the inconsistencies can be ignored or are signs of a failing disk, etc.
I sent an email to the list previously on this with no response.
I have asked about this before as well, do you have your swap on raid1? I recall someone replying that it was normal if that is the case.
Justin.
A message that is not in the archive (sent off-list or not yet received): [email protected]
What kind of disks? Does it go away after a 'repair'? Show smartctl -a of both disks in the raid?
Quoted 31 lines · Ryan Wagoner
My swap is on a different RAID 1, but my other RAID 1 with the root partition was showing inconsistencies as well. It does it on both my servers.
Ryan
Quoted 23 lines · Justin Piszcz
Quoted 4 lines · Ryan Wagoner
The issue I've had with RAID 1 using mdadm is the disk's aren't consistent. I've noticed that unlike RAID 5 when a write is aborted mdadm doesn't completely write the data.
Quoted 8 lines
When I run a check on the RAID 1 I always have inconsistencies where I have never had one on a RAID 5. The issue here is you never know if the inconsistencies can be ignored or are signs of a failing disk, etc.
I sent an email to the list previously on this with no response.
I have asked about this before as well, do you have your swap on raid1? I recall someone replying that it was normal if that is the case.
Justin.
Quoted 8 lines · peter.stevens
I'm looking for a software solution to help replace my existing raid hardware setup. The hardware mimics RAID-1 (in that it has 3 mirrors), except for the fact that on read all 3 mirrors are compared and possibly error corrected before data is returned. I don't neccessarily need a RAID-1 solution, it just seems closer to what I already have and also that it also can recover from 2 simulateous disk failures.
Quoted 8 lines
So far I've played with a software RAID-1 array of USB flash drives. My issue is that software RAID-1 does not check for or recover from data corruption unless a read or write to disk actually fails. Integrity is a major concern for me, I need to know that all data going to and from disk is correct at all times.
All advice and comments are welcomed.
I'm not aware of anything, no. Software RAID-1 will do some of what you're after but doesn't verify reads against all drives (this doesn't guarantee integrity anyway, only reduce the chance of an error). You can run regular checks of the drives to detect (and repair) any data mismatches though.
Mind you, to do this properly, you'd need to disable write caching on all drives; do a read of all drives after every write (to verify that the write was successful); and compare all drives on every read (to verify that the read was successful). You probably also want checksums on the drives to help detect which drive(s) are incorrect on read mismatches as well (I believe there's now a standard for doing this, but I don't think it's supported end-to-end yet). Doing all this will really limit performance though - generally you'll be better off building error recovery into the file format instead.
Cheers,
Robin
Signature · 4 lines
___
( ' } | Robin Hill <[email protected]> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |Quoted 24 lines · Robin Hill
Quoted 8 lines · peter.stevens
I'm looking for a software solution to help replace my existing raid hardware setup. The hardware mimics RAID-1 (in that it has 3 mirrors), except for the fact that on read all 3 mirrors are compared and possibly error corrected before data is returned. I don't neccessarily need a RAID-1 solution, it just seems closer to what I already have and also that it also can recover from 2 simulateous disk failures.
Quoted 8 lines
So far I've played with a software RAID-1 array of USB flash drives. My issue is that software RAID-1 does not check for or recover from data corruption unless a read or write to disk actually fails. Integrity is a major concern for me, I need to know that all data going to and from disk is correct at all times.
All advice and comments are welcomed.
I'm not aware of anything, no. Software RAID-1 will do some of what you're after but doesn't verify reads against all drives (this doesn't guarantee integrity anyway, only reduce the chance of an error). You can run regular checks of the drives to detect (and repair) any data mismatches though.
Quoted 9 lines
Mind you, to do this properly, you'd need to disable write caching on all drives; do a read of all drives after every write (to verify that the write was successful); and compare all drives on every read (to verify that the read was successful). You probably also want checksums on the drives to help detect which drive(s) are incorrect on read mismatches as well (I believe there's now a standard for doing this, but I don't think it's supported end-to-end yet). Doing all this will really limit performance though - generally you'll be better off building error recovery into the file format instead.
Sounds like he should use ZFS.
Quoted 20 lines
-----Original Message-----
From: [email protected] [mailto:linux-raid- [email protected]] On Behalf Of peter.stevens
Sent: Tuesday, December 09, 2008 8:03 AM
To: [email protected]
Subject: Data integrity and RAIDI'm looking for a software solution to help replace my existing raid hardware
setup. The hardware mimics RAID-1 (in that it has 3 mirrors), except for the
fact that on read all 3 mirrors are compared and possibly error corrected
before data is returned. I don't neccessarily need a RAID-1 solution, it
just seems closer to what I already have and also that it also can recover
from 2 simulateous disk failures.
Quoted 1 line
So far I've played with a software RAID-1 array of USB flash drives.
My
Quoted 10 lines
issue is that software RAID-1 does not check for or recover from data corruption unless a read or write to disk actually fails. Integrity is a
major concern for me, I need to know that all data going to and from disk is
correct at all times.All advice and comments are welcomed.
--
View this message in context:
http://www.nabble.com/Data-integrity-and-
Quoted 8 lines
RAID-tp20841008p20841008.html
Sent from the linux-raid mailing list archive at Nabble.com.--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
A few things come to mind ...
1) Use Solaris w/ZFS instead of LINUX. You can even put on target-mode
drivers and make this a SAN appliance or NAS box. Obviously this won't work for you if you have application software you want to run on the same machine, but just throwing it out. ZFS supports n-Way mirroring.
2) Use disks that support BGMS. This means late-generation Seagate SAS disks (or SCSI or FC, just not SATA). Then you can get software (shameless plug here, but we have such a product for $90.) that runs background media scans and repairs bad blocks 24x7 with near zero host overhead. Blocks that can't auto-repair will be reported, so you can kick off a consistency repair once they are detected. Many of the SATA disks do have error detection/correction firmware, but their ability to report specifics of problem and auto repair are extremely limited due to fewer ECC bits and less sophisticated algorithms in the firmware).
Regards
David
[email protected]