Skip to contentSkip to conversation

raid 5 and different device size

DISCUSSION Hendrik Friedel 4 participants 4 messages May 11, 2015 – May 12, 2015 Btrfs file system

lore
Share & tools
lore

Hendrik Friedel

author
#0 lore raw

Hello,

I need your advice for the following scenario:
I do have three -identical- 3TB HDD. On these I store:
-Backups (Images of my Families Laptop; Backup of the Machines (the one with the HDD) running System). -Videos (recordings, not essential)
-HomeVideos (Familiy, essential)
-Photos
-Documents

Now I intend to:
* increase the available disc space
* stop worrying how much space is available on which drive
* create redundancy
* have low operating cost (power-consumption)

So, I need to buy one additional HDD.
And I want to combine the drives to one big volume.

I am aware that raid is no backup.
Thus, I backup the HomeVideos and the Photos on an external drive not connected to the power supply and USB.

I think, that I do not need raid5, depending on the definition of 'not essential'. Nevertheless, raid5 would efficiently (at low cost) decrease the likelyhood of loss of the 'non-essential' data. Furthermore, raid5 would fill the gap that I have between backups (the fact, that I do these backup manually means that I do not do them every day) and at least safe me from hardware failures.

So, yes, I think I do want raid5 and I want to combine.
But what size of HDD do I want to buy? The best value (cost per TB) I get for 3TB drives. So I could by two 3TB drives, one for additional space, the other for the parity information. Or I could by one bigger drive, with potentially lower power consumption. But how would the raid5 look in this case? I fear, only 3TB of the 6 would be redundant and the other 3 would be not redundant, right?

Now the last point: Power consumption: Under which conditions can the drives spin down in case of raid5? I assume that all drives have to run as in case the data is written on any one of the drives, right? Is that also true during reading of data, i.e. is the parity also checked for read operations?

Thanks for your advice in advance,
Hendrik

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
http://www.avast.com

#1 lore raw
Quoted 32 lines · Hendrik Friedel

Hello,

I need your advice for the following scenario:
I do have three -identical- 3TB HDD. On these I store:
-Backups (Images of my Families Laptop; Backup of the Machines (the one with the HDD) running System).

(quoted diff) +0 -4
-Videos (recordings, not essential)
-HomeVideos (Familiy, essential)
-Photos
-Documents

Now I intend to:
* increase the available disc space
* stop worrying how much space is available on which drive
* create redundancy
* have low operating cost (power-consumption)

So, I need to buy one additional HDD.
And I want to combine the drives to one big volume.

I am aware that raid is no backup.
Thus, I backup the HomeVideos and the Photos on an external drive not connected to the power supply and USB.

I think, that I do not need raid5, depending on the definition of 'not essential'. Nevertheless, raid5 would efficiently (at low cost) decrease the likelyhood of loss of the 'non-essential' data. Furthermore, raid5 would fill the gap that I have between backups (the fact, that I do these backup manually means that I do not do them every day) and at least safe me from hardware failures.

Quoted 8 lines

So, yes, I think I do want raid5 and I want to combine. But what size of HDD do I want to buy? The best value (cost per TB) I get for 3TB drives. So I could by two 3TB drives, one for additional space, the other for the parity information. Or I could by one bigger drive, with potentially lower power consumption. But how would the raid5 look in this case? I fear, only 3TB of the 6 would be redundant and the other 3 would be not redundant, right?

No, 3 TB of the new drive would be usable, giving you a grand total
of (4-1) * 3TB = 9 TB of usable space on the array, with 3 TB of raw space unusable (the remainder of the 6 TB device).

http://carfax.org.uk/btrfs-usage may be helpful to you here.

Quoted 6 lines

Now the last point: Power consumption: Under which conditions can the drives spin down in case of raid5? I assume that all drives have to run as in case the data is written on any one of the drives, right?
Is that also true during reading of data, i.e. is the parity also checked for read operations?

Parity is spread evenly across all devices, so a read of more than
a trivial quantity of data would almost certainly spin up all devices anyway.

Hugo.

Signature · 4 lines
Hugo Mills             | A gentleman doesn't do damage unless he's paid for
hugo@... carfax.org.uk | it.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                            Juri Papay
#2 lore raw
In reply to Hendrik Friedel

For that kind of data where you mainly store lots of (big?) files but rarely modify the files once stored,
I would suggest using snapraid it is mor stable and reliable than current btrfs raid5 state.
And it allows you to to go to raid6 and beyond as your data and redundancy factors grow.

Quoted 62 lines · Hendrik Friedel

Hello,

I need your advice for the following scenario:
I do have three -identical- 3TB HDD. On these I store:
-Backups (Images of my Families Laptop; Backup of the Machines (the one with the HDD) running System).

(quoted diff) +0 -4
-Videos (recordings, not essential)
-HomeVideos (Familiy, essential)
-Photos
-Documents

Now I intend to:
* increase the available disc space
* stop worrying how much space is available on which drive
* create redundancy
* have low operating cost (power-consumption)

So, I need to buy one additional HDD.
And I want to combine the drives to one big volume.

I am aware that raid is no backup.
Thus, I backup the HomeVideos and the Photos on an external drive not connected to the power supply and USB.

I think, that I do not need raid5, depending on the definition of 'not essential'. Nevertheless, raid5 would efficiently (at low cost) decrease the likelyhood of loss of the 'non-essential' data.
Furthermore, raid5 would fill the gap that I have between backups (the fact, that I do these backup manually means that I do not do them every day) and at least safe me from hardware failures.

So, yes, I think I do want raid5 and I want to combine.
But what size of HDD do I want to buy? The best value (cost per TB) I get for 3TB drives. So I could by two 3TB drives, one for additional space, the other for the parity information.
Or I could by one bigger drive, with potentially lower power consumption. But how would the raid5 look in this case? I fear, only 3TB of the 6 would be redundant and the other 3 would be not redundant, right?

Now the last point: Power consumption: Under which conditions can the drives spin down in case of raid5? I assume that all drives have to run as in case the data is written on any one of the drives, right?
Is that also true during reading of data, i.e. is the parity also checked for read operations?

Thanks for your advice in advance,
Hendrik

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
http://www.avast.com

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
In reply to Hendrik Friedel
Quoted 8 lines · Hendrik Friedel

I do have three -identical- 3TB HDD.

Now I intend to:
* increase the available disc space
* stop worrying how much space is available on which drive
* create redundancy
* have low operating cost (power-consumption)

Quoted 6 lines

So, I need to buy one additional HDD.
And I want to combine the drives to one big volume.

I am aware that raid is no backup.
Thus, I backup the HomeVideos and the Photos on an external drive not connected to the power supply and USB.

In my experience, people tend to GREATLY undervalue their data, until it is lost!

Your proposed 12 TB (4x3TB) of storage is > 10^14 bits, so you are very likely to to encounter an unrecoverable sector on a full-read with most typically error-rated drives.

Also consider drive wear (average lifespan of 3 years), spares, and how you will deal with a single or multi-drive failure.

You may want to rethink your plan, and the value of your data.

Quoted 5 lines

Now the last point: Power consumption: Under which conditions can the drives spin down in case of raid5? I assume that all drives have to run as in case the data is written on any one of the drives, right? Is that also true during reading of data, i.e. is the parity also checked for read operations?

In RAID-5/6, parity is only checked when an error or failure is reported by the drive. This can lead to undetected errors from "bit rot". Hence regular scrubbing (reading of all data) is recommended.

Note that BTRFS checksums data, and verifies data read, so even if the drive doesn't detect an error BTRFS is likely to, but BTRFS can't recover without additional redundancy.

Personally, I use BTRFS formatted with "-mdup -dsingle" on top of either hardware RAID-6 (primary array) or software RAID-5 (backup array), so I don't really rely upon BTRFS for recovery.

Most external consumer-grade enclosures are best used with "Green" drives, which tend to be 5400 RPM max, as these enclosures often have very limited cooling. Make sure your drives are compatible with the enclosure.

I have even seen server-grade rack enclosures with reduced temperature ratings when using 7200 RPM or faster drives. Heat and vibration are important considerations with arrays.

I prefer Seagate enterprise drives and WD consumer drives.

Note that WD "Red Label" drives are identical to their "Green Label" drives, except "Red Label" drives have time-limited error recovery (TLER) firmware for use in arrays. However, it is possible to increase the error timeout in Linux, so even non-TLER drives can be used in arrays, but it isn't as convenient nor recommended. Both are error-rated 1 unrecoverable sector in 10^14 bits read.

~~Andrew E. Mileski