Skip to contentSkip to conversation

Do I understand my RAID6 correctly?

DISCUSSION Stefan G. Weichinger 2 participants 4 messages Apr 12, 2011 MD RAID

lore
Share & tools
lore
#0 lore raw

Today I had a drive fail in a customers server.

It was part of a RAID6 which seems to have rebuilt onto a spare drive now.

Right now it looks like:

# mdadm -D /dev/md3
/dev/md3:
        Version : 00.90.03
  Creation Time : Thu Dec 20 17:47:07 2007
     Raid Level : raid6
     Array Size : 4391334912 (4187.90 GiB 4496.73 GB)
  Used Dev Size : 731889152 (697.98 GiB 749.45 GB)
   Raid Devices : 8
  Total Devices : 9
Preferred Minor : 3
    Persistence : Superblock is persistent

Update Time : Tue Apr 12 10:27:45 2011
State : clean
Active Devices : 8
Working Devices : 8
Failed Devices : 1
Spare Devices : 0

Chunk Size : 64K

UUID : e848b637:ca2bde73:9f92f3cc:128cdbad
Events : 0.47127534

Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 177 1 active sync /dev/sdl1
2 8 65 2 active sync /dev/sde1
3 8 81 3 active sync /dev/sdf1
4 8 97 4 active sync /dev/sdg1
5 8 113 5 active sync /dev/sdh1
6 8 129 6 active sync /dev/sdi1
7 8 145 7 active sync /dev/sdj1

8 8 161 - faulty spare /dev/sdk1

My question (just to be sure):

Do I understand it correctly that the system has substituted the failed /dev/sdk1 by a former spare drive (dunno the device name now) and that I now I have a valid RAID6-device with 8 drives in it?

So out of the 8 drives there could fail another 2 now without losing data ...

correct?

I have to tell the customer what to do and the grade of redundancy available also relates to how urgent it is to get a new drive into the system.

I assume I would remove /dev/sdk1 from md3, swap the drive, fdisk it and re-add sdk1 to md3 (it is failed already now, so the fail-step isn't necessary anymore). It would the be the new spare drive ... ?

Thanks for refreshing my RAID-knowledge ;-)
Stefan

#1 lore raw

Subject: Do I understand my RAID6 correctly?
From: Stefan G. Weichinger <[email protected]>
To: Yann Ormanns <[email protected]>
Date: 2011-04-12 11:11 (+0200)

Quoted 10 lines
     Array Size : 4391334912 (4187.90 GiB 4496.73 GB)

Used Dev Size : 731889152 (697.98 GiB 749.45 GB)
Raid Devices : 8
Total Devices : 9
Preferred Minor : 3
Persistence : Superblock is persistent

Active Devices : 8
Working Devices : 8
Failed Devices : 1

You now have 9 devices in this Array (750GB*8 = 6 TB, 6 TB - (2*750GB) = 4.5 TB). One of them is the failed spare disk. That means, that this array can "lose" two disks without losing data, as you already wrote.

Of course you can re-use /dev/sdk as a spare disk, but before, you should check, why it failed (SMART data for example).

You should also have a look at the used drive models. E.g. if this array uses 9x model XYZ from manucaturer ABC, perhaps more drives will fail in the next time.
If the array uses mixed models, it should not be THAT urgent - but that depends on the importance of the data...
I've read several times of people losing their RAID6, because they did not mix the hard drive models. Then, a manufacturing fault have very bad consequences.

Best regards,
Yann

#2 lore raw

Am 12.04.2011 11:23, schrieb Yann Ormanns:

Quoted 3 lines

You now have 9 devices in this Array (750GB*8 = 6 TB, 6 TB - (2*750GB) = 4.5 TB). One of them is the failed spare disk. That means, that this array can "lose" two disks without losing data, as you already wrote.

Yep, fine.

Quoted 2 lines

Of course you can re-use /dev/sdk as a spare disk, but before, you should check, why it failed (SMART data for example).

I already exported the controller-logs and will look through for SMART info. Unfortunately the controller does not allow the use of smartmontools, I can only use the specific ICP Storage Manager.

Quoted 3 lines

You should also have a look at the used drive models. E.g. if this array uses 9x model XYZ from manucaturer ABC, perhaps more drives will fail in the next time.

uuuh

Quoted 5 lines

If the array uses mixed models, it should not be THAT urgent - but that depends on the importance of the data...
I've read several times of people losing their RAID6, because they did not mix the hard drive models. Then, a manufacturing fault have very bad consequences.

Scary. Yes, the server uses the same model for all 9 devices. From your domain I see that you seem to be located in germany, so you might know the manufacturer of the server: transtec.

I already opened a support ticket there, we still have a valid support contract. Last time they sent a new drive, we'll see.

Thanks, Stefan

#3 lore raw

Subject: Re: Do I understand my RAID6 correctly?
From: Stefan G. Weichinger <[email protected]>
To: Yann Ormanns <[email protected]>
Date: 2011-04-12 19:03 (+0200)

Quoted 5 lines

I already exported the controller-logs and will look through for SMART info. Unfortunately the controller does not allow the use of smartmontools, I can only use the specific ICP Storage Manager.

I recommend to compare the active hours and the serial numbers of the disks. So you can _perhaps_ predict the next disk with problems. Unfortunately, SMART-data is no credible basis for any hard disk failure predictions.
For further information, you may want to take a look at this german link:
http://www.heise.de/newsticker/meldung/Google-Studie-zur-Ausfallursache-von-Festplatten-147178.html
and / or at this document http://labs.google.com/papers/disk_failures.pdf

Quoted 2 lines

Scary. Yes, the server uses the same model for all 9 devices.

Yeah, that's really scary - it shows, that even with a RAID6 your data is not absolutely safe. But I have to admit that this is "only" the worst case scenario - the chance that this situation occurs, is really small (but not impossible).
I suppose, that your customer keeps his backups up to date although he uses a RAID6?

Quoted 3 lines

From your domain I see that you seem to be located in germany, so you might know the manufacturer of the server: transtec.

No, I do not know this manucaturer - but I'm just a private user, so I don't really have any experiences with "real" servers :)

Best regards,
Yann