Software based SATA RAID-5 expandable arrays?

I am researching my option to build a Media NAS server. Sorry for the long message, but I wanted to provide as much details as possible to my problem, for the best solution. I have Bolded sections as to save people who don't have the time to read all of this.

Option 1: Expand My current Dream Machine!
I could buying a RAID-5 Hardware card for my current system (vista ultimate 64 with a extreme 6800 and 1066mb 2 gig RAM). The Adaptec RAID controller (model "3805", you can search NewEgg for the infomation) will cost me near $500 (consume 23w) and support 8 drives (I have 6). This controller contains a 800mhz processor with a large cache of memory. It will support expandable RAID-5 array! I would also buy a 750w+ PSU (for the additional safety and security). The drives in this machine would be placed in shock absorbing (noise reduction) 3 slot 4 drive bay containers with fans ( I have 2 of these) and I will be removing a IDE based Pioneer DVD Burner (1 of 3) because of its flaky performance given the p965 intel chip set lack of native IDE support and thus the Motherboards Micron SATA to IDE device. Ive already installed 4 drives in this machine (on the native MB SATA controller) only to find a fan fail on me within days of the installation. One of the drives went bad (may or may not have to do with the heat). There are 5mm between these drives, and I would now replace both fans with higher RPM ball baring fans for added reliability (more noise). I would also need to find a Freeware SMART monitor software which at this time I can not find for Vista, to warn me of increased temps due to failure of fan, increased environmental heat, etc. The only option is commercial SMART monitoring software (which may not work with the Adaptec RAID adapter.

Option 2: Build a server.
I have a copy of Windows 2003 server, which I have yet to find out if it supports native software expandable RAID-5 arrays. I can also use Linux (which I have very little experience with) but have always wanted to use and learn.

To do either of the last two options, I would still need to buy a new power supply for my current VISTA machine (for added reliability). The current PSU is 550w and with a power hungry RADEON, 3 DVD Drives and a X-Fi sound card... My nerves are getting frayed.

I would buy a cheap motherboard, processor and 1gig or less of RAM. Lastly I would want a VERY large Case. I have a 7300 NVidia PCI card that was replaced with a X1950GT on my Home Theater PC so that I may play back HD/Blue Ray DVD's.

The server option may cost a bit more then the $500 for the Adaptec Raid controller. This will only work if Linux or Windows 2003 supports my much needed requirements. My Linux OS will be installed on a 40mb IDE Drive (not part of the Array).

The options I seek are to be able to start with a 6 Drive array RAID-5 array, then as my demand for more space increases in the future I want to be able to plug in more drives and incorporate them into the Array without the need to backup the data. Basically I need the software to add the drive/drives to the Array, then Rebuild the array incorporating the new drives while preserving the data on the original array.

QUESTIONS
Since this is a media server, and would only be used to serve Movies and Video to my two machines It wouldn't have to be powered up full time (My Music consumes less space and will be contained on two seperate machines). Is there a way to considerably lower the power consumption of this server the 90% of time its not in use?

Can Linux support Drive Arrays of Significant Sizes (4-8 terabytes)?

Can Linux Software support RAID-5 expandability, allowing me to increase the number of disks in the array, without the need to backup the media, recreate the array from scratch and then copy the backup to the machine (something I will be unable to do)?

I know this is a Linux forum, but I figure many of you guys work with Windows Server. If so does Windows 2003 provide the same support for the requested requirements above?

Thanks
GreenJelly

View this message in context: http://www.nabble.com/Software-based-SATA-RAID-5-expandable-arrays--tf3937421.html#a11167521
Sent from the linux-raid mailing list archive at Nabble.com.

Yes, some reports of over 40-60TB.

With a recent kernel, yes.

I hear Windows 2003 SW raid is extremely slow.

$ /usr/bin/time dd if=/dev/zero of=file bs=1M
dd: writing `file': No space left on device
1070704+0 records in
1070703+0 records out
1122713473024 bytes (1.1 TB) copied, 2565.89 seconds, 438 MB/s

$ time dd if=100gb of=/dev/null bs=1M count=102400
102400+0 records in
102400+0 records out
107374182400 bytes (107 GB) copied, 172.588 seconds, 622 MB/s

Sure its cached, and I could use the direct flag, but I do not use my disks with the cache turned off.

Most important thing is cooling IMO, make sure you get a good case.

^^ That is with 10 raptors and software raid5. I'd use a separate Linux box with MD SW raid and have your dream machine separate. If you use XFS make sure you buy yourself a UPS and then you'll have yourself a nice stable box.

Justin.

Why dontcha just cut all the "look how big my ePenis is" chatter and tell us what you wanna do?
Nobody gives a rat if your ultra1337 sound cards needs a 10 megawatt power supply.

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS d--(+)@ s-:+ a- C++++ UL++ P+>++ L+++>++++ E-- W++ N o? K-
w--(---) !O M+ V- PS+ PE Y++ PGP t++(---)@ 5 X+(++) R+(++) tv--(+)@ 
b++(+++) DI+++ D- G++ e* h>++ r* y?
------END GEEK CODE BLOCK------

http://www.stop1984.com
http://www.againsttcpa.com

Chill Dexter.

How many faults have you seen on this list attributed to poor PSUs?
How many people whinge about the performance of their controllers/setups 'cos they find out _after_ they bought them just how naff they are?

Sure he went a bit OTT in the description - but if you'd rather see "Hey dudez, what do I need for a really quick server" then #linux is good :)

He's clearly new to linux, (and granted, maybe a bit over-excited by hardware!) but give the guy a break.

He very clearly told us what he wanted to do in the QUESTIONs bit.

greenjelly (btw most people use real names in the linux kernel part of the world), feel free to search the archives, read the FAQs and continue to ask questions :)

And don't think too badly of Dexter, he's usually OK.

David

PS, Dex, I wonder who posted these noobie sounding question in April last year:

I'm currently planning my first raid array.
I intend to go for softraid (budget's the limiting factor), not sure about 5 or 6 yet.

Plan so far: build a raid5 from 3 disks, later add a disk and reconf to raid6.
Question: is that possible at all? Can a raid5 be reconfed to a raid6 with raidreconf?
Next Question: how stable is it? Will I likely get away without making backups or is there like a 10% chance of failure?
Other precautions advised?

A 750W supply doesn't increase server stability - that's what redundant PSUs are for.
Plus: there are sh!tty 750W-PSUs otu there as well - numbers mean jack.

Could have done right from the start instead of immersing into "Vista", "X-Fi" "8800" and "Radeon".

Guess I had a newbie overdose since migrating the desktop box to Kubuntu.

Yes? What about that? All tech questions regarding file servers and raid setups on Linux. Don't see how I go about how my Radeon makes the 10k barrier in 3Dmark.

Nuff said.

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS d--(+)@ s-:+ a- C++++ UL++ P+>++ L+++>++++ E-- W++ N o? K-
w--(---) !O M+ V- PS+ PE Y++ PGP t++(---)@ 5 X+(++) R+(++) tv--(+)@ 
b++(+++) DI+++ D- G++ e* h>++ r* y?
------END GEEK CODE BLOCK------

http://www.stop1984.com
http://www.againsttcpa.com

I've got 2 boxes. One has 14 drives and a 480W PSU and the other has 15 drives and a 600W PSU. It's not rocket science. Put a lot of drives in a box, make sure you have enough sata ports and power to go around (watch your peak 12V consumption on spin up really) and use linux md. Easy.. Oh, but make sure the drives stay cool!

For a cheap-o home server (which is what I have) I'd certainly not bother with a dedicated RAID card. You are not even going to need GB ethernet really.. I've got 15 drives on a single PCI bus, it's as slow as a wet week in may (in the southern hemisphere), but I'm streaming to 3 head units which total a combined 5MB/s if I'm lucky.. Rebuilds can take up to 10 hours though.

Yes, don't poll for SMART and spin down the drives when idle (man hdparm). Use S3 sleep and WOL if you are really clever. (I'm not, my boxes live in a dedicated server room with its own AC, but that's because I'm nuts). I also have over 25k hours on the drives because I don't spin them down. I figure the extra power is a trade off for drive life. They've got less than 50 spin cycles on them in over 25k hours..

Yes, easily (6TB here)

Yes but get a cheap UPS at least (it's cheap insurance)

Why would you even _ask_ ??

Read the man page for mdadm, then read it again (and a third time). Then google for "Raid-5 two drive failure linux" just to familiarise yourself with the background.

What you are doing has been done before many, many times. There are some well written sites out there relating to building exactly what you want to build with great detail.

If you are serious about using windows, I pity you.. Linux (actually a combination of the kernel md layer and mdadm) makes it so easy you'd be nuts to beat your head against the wall with the alternative.

Brad

"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams