raid5/lvm setup questions
Quoted 9 lines · Shane
Hello all,
I'm building a new server which will use a number of disks and am not sure of the best way to go about the setup. There will be 4 320gb SATA drives installed at first. I'm just wondering how to set the system up for upgradability. I'll be using raid5 but not sure whether to use lvm over the raid array.
Quoted 4 lines
By upgradability, I'd like to do several things. Adding another drive of the same size to the array. I understand reshape can be used here to expand the underlying block device.
Yes, it can.
If the block device is the pv of an lvm array,
Quoted 3 lines
would that also automatically expand in which I would create additional lvs in the new space. If this isn't automatic, are there ways to do it manually?
Not automatic AFAIK - but doable.
Quoted 4 lines
What about replacing all four drives with larger units. Say going from 300gbx4 to 500gbx4. Can one replace them one at a time, going through fail/rebuild as appropriate and then expand the array into the unused space
Yes.
or would
Quoted 1 line
one have to reinstall at that point.
No
None of the requirements above drive you to layering lvm over the top.
That's not to say don't do it - but you certainly don't *need* to do it.
Pros:
* allows snapshots (for consistent backups)
* allows various lvm block movements etc...
* Can later grow vg to use discrete additional block devices without raid5 grow limitations (eg same-ish size disks etc)
Cons:
* extra complexity -> risk of bugs/admin errors...
* performance impact
As an example of the cons: I've just set up lvm2 over my raid5 and whilst testing snapshots, the first thing that happened was a kernel BUG and an oops...
David
2006/8/5, Shane <[email protected]>:
Quoted 3 lines
Well, the reason I was looking at LVM is because since this is a fairly big array, I didn't want to lose a bunch of space with ext3 inodes. For example, the PostGreSQL
Then forget about ext{2|3} and use xfs or reiserfs. ext3 is limited to 4TB anyway.
Best
Martin
On 5 Aug 2006, David Greaves prattled cheerily:
Quoted 2 lines
As an example of the cons: I've just set up lvm2 over my raid5 and whilst testing snapshots, the first thing that happened was a kernel BUG and an oops...
I've been backing up using writable snapshots on LVM2 over RAID-5 for some time. No BUGs.
I think the blame here is likely to be layable at the snapshots' door,
anyway: they're still a little wobbly and the implementation is pretty
complex: bugs surface on a regular basis.
Signature · 2 lines
`We're sysadmins. We deal with the inconceivable so often I can clearly see the need to define levels of inconceivability.' --- Rik Steenwinkel
Quoted 3 lines
I seem to recall patches to md floating around a couple years back for partitioning of md devices. Are those still available somewhere?
I believe the patches that you are referring to are now included into the standard 2.6+ kernel.
-Chet
Quoted 4 lines · Shane
Quoted 3 lines · Nix
On 5 Aug 2006, David Greaves prattled cheerily:
Quoted 2 lines
As an example of the cons: I've just set up lvm2 over my raid5 and whilst testing snapshots, the first thing that happened was a kernel BUG and an oops...
Quoted 8 lines
Quoted 2 lines
I've been backing up using writable snapshots on LVM2 over RAID-5 for some time. No BUGs.
Just performed some basic throughput tests using 4 SATA disks in a raid5 array. The read performance on the
/dev/mdxdevice runs around 180mbps but if lvm is layered over that, reads on the lv are around 130mbps. Not an unsubstantial reduction.
Check the readahead at various block levels
blockdev --setra xxx
I think I found the best throughput (for me) was with 0 readahead for /dev/hdX, 0 for /dev/mdX and lots for /dev/vg/lv
Quoted 4 lines
I seem to recall patches to md floating around a couple years back for partitioning of md devices. Are those still available somewhere?
man mdadm and see --auto...
David
Quoted 1 line · Nix
On 5 Aug 2006, David Greaves prattled cheerily:
that's me :)
Quoted 5 lines
Quoted 2 lines
As an example of the cons: I've just set up lvm2 over my raid5 and whilst testing snapshots, the first thing that happened was a kernel BUG and an oops...
I've been backing up using writable snapshots on LVM2 over RAID-5 for some time. No BUGs.
I tried but it didn't recurr.
I sent a report to lkml.
Quoted 3 lines
I think the blame here is likely to be layable at the snapshots' door,
anyway: they're still a little wobbly and the implementation is pretty
complex: bugs surface on a regular basis.
Hmmm. Bugs in a backup strategy. Hmmm.
I think I can live with a nightly shutdown of the daemons whilst rsync does it's stuff across the LAN.
David
Quoted 10 lines · On Saturday August 5, [email protected] wrote:
Quoted 4 lines · David Greaves
Quoted 3 lines
Say going from 300gbx4 to 500gbx4. Can one replace them one at a time, going through fail/rebuild as appropriate and then expand the array into the unused space
Yes.
I didn't see anything in the mdadm manual on this. Would one just do a --grow
/dev/md0once the disks were changed out? It looks like --grow is used to change the number of devices in the array but not the device size itself.
It does both (and more).
mdadm --grow /dev/md0 --raid-disk=5
changes the number of drives to 5.
mdadm --grow /dev/md0 --size=max
changes the used-size of each drive to the maximum available.
mdadm --grow /dev/md0 --bitmap=internal
adds an internal write-intent bitmap
mdadm --grow /dev/md0 --chunksize=128
might change the chunksize to 128k.. but doesn't yet. Maybe one day :-)
NeilBrown