Simulating Drive Failure on Mirrored OS drive
Various documents on Mirrored OS Drives suggest you can simulate the failure of one of your mirrored pairs by marking one or more partitions as "failed" with mdadm.
Indeed when I mark a drive as "failed", if I have the mdadm monitoring daemon running, I get an email warning me that my RAIDS are degraded.
Also, I can reboot with only one drive plugged in and the OS comes up fine (albeit degraded).
This gives me a REASONABLE degree of confidence that the Mirrored Partitions will continue working okay if one drive should fail. However, I would like to run a more definitive test.
I tried simply unplugging one drive from its power and from its SATA connector. The OS didn't like that at all. My KDE session kept running, but I could no longer open any new terminals. I couldn't become root in an existing terminal that was already running. And I couldn't SSH into the machine.
It was like I had an OS running on only a 1/4th of a cylinder. I couldn't even make the OS cleanly shutdown or reboot.
I know that simply unplugging a drive is not the same as a drive failing or timing out. But is there a more realistic way to simulate a failure so that I can know that the mirror will work when it's needed?
Andy Liebman
Quoted 5 lines · andy liebman
I tried simply unplugging one drive from its power and from its SATA connector. The OS didn't like that at all. My KDE session kept running, but I could no longer open any new terminals. I couldn't become root in an existing terminal that was already running. And I couldn't SSH into the machine.
That's likely to be because sata hotswap isn't supported (yet). dmesg should give you more info.
Quoted 3 lines
I know that simply unplugging a drive is not the same as a drive failing or timing out. But is there a more realistic way to simulate a failure so that I can know that the mirror will work when it's needed?
Read up on the md-faulty device.
Also, FWIW, md works just fine :)
(Lots of other things can go wrong so testing your setup is a food idea though)
David
Quoted 3 lines
Read up on the md-faulty device.
Got any links to this? As I said, we know how to set the device as faulty, but I'm not convinced this is a good simulation of a drive that fails (times out, becomes unresponsive, etc.)
Andy
Quoted 7 lines · andy liebman
Quoted 3 lines
Read up on the md-faulty device.
Got any links to this? As I said, we know how to set the device as faulty, but I'm not convinced this is a good simulation of a drive that fails (times out, becomes unresponsive, etc.)
Note that 'set device as faulty' is NOT the same as md-faulty device'. Read mdamd(8) manpage, and see options -l' (level) and `-p' (parity) for create mode.
/mjt