Skip to contentSkip to conversation

what superblock to use

DISCUSSION Farkas Levente 7 participants 9 messages Apr 20, 2009 – Apr 22, 2009 MD RAID

lore
Share & tools
lore
#0 lore raw

hi,
what's the current recommended superblock to use for a newly created raid5-6 array with 6 pieces of 1tb disk? by default mdamd use 0.90. is it worth to change it to any 1.x format?
anyway is there any advantage of a raid6 over raid5+1spare disk? afaik raid5 will be faster and use less cpu and both case 2 disk can failed. thanks in advance.

Signature · 1 line
  Levente                               "Si vis pacem para bellum!"
#1 lore raw
Quoted 1 line · Farkas Levente

anyway is there any advantage of a raid6 over raid5+1spare disk?

The raid6 owner sleeps soundly after a single disk failure.

#2 lore raw
In reply to Farkas Levente
Quoted 4 lines · Farkas Levente

hi,
what's the current recommended superblock to use for a newly created raid5-6 array with 6 pieces of 1tb disk? by default mdamd use 0.90. is it worth to change it to any 1.x format?

0.9 has some limitations that 1.x do not have, none apply in your case max_components = 28
max_size of component=2TB

1.x lifts those limitation
my stance would be going to 1.2
1.1 puts the superblock at the start of the device, this will prevent the kernel from finding a partition table or fs superblock on the first component device
1.2 puts the superblock 4k from start, thus preventing anything that tries to touch the mbr from damaging it.
if you put the sb at the start of the device lilo and grub wont be able to boot from it, but then they won't be able to boot of a raid5/6 anyway.

Quoted 3 lines

anyway is there any advantage of a raid6 over raid5+1spare disk? afaik raid5 will be faster and use less cpu and both case 2 disk can failed. thanks in advance.

raid6 prevents two possible risks in raid5+spare
first:
when using a spare, you are never certain it is in perfect working order, smart may help, or it might not.
second: when a device fails a rebuild process starts using the spare drive if during the rebuild a second device fails or has any error you risk losing data.

the first problem could be solved by using raid5e, which md does not support yet

the second can be minimized by 'scrubbing' via
/sys/block/md*/md/sync_action

regards,
L.

Signature · 6 lines
Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \
#5 lore raw
In reply to Farkas Levente
Quoted 2 lines · Farkas Levente

anyway is there any advantage of a raid6 over raid5+1spare disk? afaik raid5 will be faster and use less cpu and both case 2 disk can failed.

No, raid6 survives the simultaneous failure of two disks. Raid5 survives the simultaneous failure of one disk only. Even with a hot-spare, after this failure you have a time frame where the spare is synching and your array has no redundancy left. Thus, nearly every other disk failure (except a failure on the synching spare) within this time frame kills your array.

regards
Mario

Signature · 2 lines
There are two major products that come from Berkeley: LSD and UNIX.
We don't believe this to be a coincidence.    -- Jeremy S. Anderson
#3 lore raw
In reply to Luca Berra
Quoted 7 lines
Quoted 4 lines

what's the current recommended superblock to use for a newly created raid5-6 array with 6 pieces of 1tb disk? by default mdamd use 0.90. is it worth to change it to any 1.x format?

0.9 has some limitations that 1.x do not have, none apply in your case max_components = 28
max_size of component=2TB

Oops! I did not realize this when I built my array. I'm going to need to grow the array to 3T components after the 3T drives are released. Can the superblock be changed on an existing array, or does it have to be rebuilt from scratch? If the latter, I'm glad I learned this now, because I may have to rebuild the array, anyway, to try to alleviate the halt issue I am experiencing.

#4 lore raw
Quoted 8 lines · Leslie Rhorer
Quoted 7 lines
Quoted 4 lines

what's the current recommended superblock to use for a newly created raid5-6 array with 6 pieces of 1tb disk? by default mdamd use 0.90. is it worth to change it to any 1.x format?

0.9 has some limitations that 1.x do not have, none apply in your case max_components = 28
max_size of component=2TB

Quoted 6 lines

Oops! I did not realize this when I built my array. I'm going to need to grow the array to 3T components after the 3T drives are released. Can the superblock be changed on an existing array, or does it have to be rebuilt from scratch? If the latter, I'm glad I learned this now, because I may have to rebuild the array, anyway, to try to alleviate the halt issue I am experiencing.

afaik it cannot be changed after creation
if you have free space you could play tricks reducing the fs, recreating the array with the exact same parameters except metadata and --assume-clean (you are limited to using 1.0 in this case) anyway do a backup before trying this.

L.

Signature · 6 lines
Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \
#8 lore raw
In reply to Farkas Levente
Quoted 9 lines · Farkas Levente

hi,
what's the current recommended superblock to use for a newly created raid5-6 array with 6 pieces of 1tb disk? by default mdamd use 0.90. is it worth to change it to any 1.x format?
anyway is there any advantage of a raid6 over raid5+1spare disk? afaik raid5 will be faster and use less cpu and both case 2 disk can failed. thanks in advance.

Let me be Devil's Advocate. The advantage of raid6 is that it will survive the failure of two drives at the same time, while a spare must be rebuilt (a good argument for fast rebuild and let response go to blazes).

The advantage of raid5+S is that with a failure of a single drive you run your io in recovery mode, and it is slow, while after rebuild on the spare raid5+S is as fast as ever.

I have been trying some things with raid5e, and as soon as I find a good primer on using events to kick the recovery off I will be able to report some sucess with this. My POC uses a script, and works fine if I poll to detect the disk failure.

Signature · 6 lines
bill davidsen <[email protected]>
 CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
   - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.
#6 lore raw
In reply to Mario 'BitKoenig' Holbe
Quoted 11 lines · Mario 'BitKoenig' Holbe
Quoted 2 lines · Farkas Levente

anyway is there any advantage of a raid6 over raid5+1spare disk? afaik raid5 will be faster and use less cpu and both case 2 disk can failed.

No, raid6 survives the simultaneous failure of two disks. Raid5 survives the simultaneous failure of one disk only. Even with a hot-spare, after this failure you have a time frame where the spare is synching and your array has no redundancy left. Thus, nearly every other disk failure (except a failure on the synching spare) within this time frame kills your array.

It's worth noting that a failure mode that is getting increasingly frequently reported is the failure of a drive *during sync*. I suspect that the cause is that synchronization puts different stresses on the drives than normal operation.

-hpa

Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.
#7 lore raw
Quoted 9 lines
Quoted 8 lines

No, raid6 survives the simultaneous failure of two disks. Raid5 survives the simultaneous failure of one disk only. Even with a hot-spare, after this failure you have a time frame where the spare is synching and your array has no redundancy left. Thus, nearly every other disk failure (except a failure on the synching spare) within this time frame kills your array.

Quoted 4 lines

It's worth noting that a failure mode that is getting increasingly frequently reported is the failure of a drive *during sync*. I suspect that the cause is that synchronization puts different stresses on the drives than normal operation.

Yes, that, and with both drives and RAID arrays growing larger, it's taking longer for arrays to re-sync, meaning a failure during any random time period is going to be more likely to fall during a sync. When I first installed a RAID system on my server, it only took a few hours to sync the 320G array. Now it can take up to 3 days to synch the 8T array.