LVM questions

Hi guys,

I'm putting together a storage server for NAS and iSCSI purposes and am looking to use LVM for some or all of this. I've read up on LVM but I still have some basic newbie questions for this.

I've got a bunch of Raptor 150 ADFD drives and a 3Ware 9650SE board providing the basic back-end storage (currently RAID-5 but will move to RAID-6 soon, with estimated capacity of approx 800-900 GB, although the entire array may shrink or grow as projects demand). The opsys root partition will be about 40 GB XFS, with another 4GB swap partition, and the NAS/storage pool will be about 300 GB XFS, with the remainder of the space being used by the iSCSI volumes (about another 300 GB, divvied up among different iSCSI LUNs). I want to be able to shrink and expand these by about 10 GB per (although swap may only change by +/-4GB) so I figure I'll use LVM for all of the partitions. A couple of other considerations
here: I need to align the partitions on page boundaries for iSCSI performance reasons, and I am also thinking about managing blockdev read-ahead differently for each of the final volumes.

I assume that a single physical extent across /dev/sda will work as well as anything else here. Will having more partitions make it easier to shrink and expand the whole RAID if that ever becomes necessary? Anything else I should take into consideration?

Should I create a single volume group or should I create multiple volume groups, or does it even matter? What are the issues?

I have seen some people talk about creating hundreds of small partitions and using those for moving and resizing. Is this needed, desirable, or just a stupid human trick?

Anything else I should take into consideration here would be appreciated

Thanks!

That is a work around for lack of functionality in LVM. Having used LVM on AIX, that is the gold standard for me. The LVM divides into little partitions and keeps track of them for you. So why do it manually? Using md for mirroring, each partition synchronizes separately. Manually making several smaller partitions lets you move them around and resync
- all while the system is running. This happens automatically when LVM supports mirroring, but LVM mirroring is still a new experimental feature on linux.

Multiple volume groups just get in the way when you are resizing and migrating between physical volumes. You need a separate volume group when you need to physically pick up the entire volume group and take it somewhere else. For instance, scientific datasets are often much too large (terabytes) for DVD, tape, or internet. So, just make a volume group on a handful of 500G drives, build and process your data there, then deactivate the VG and send the drives to your colleague (make a copy first, of course). Your colleague plugs the drives into his USB/SATA/iSCSI/whatever controller, activates the volume group, and now has a copy of your work.

For anything that will never be physically moved to another machine, keep it all on one volume group. But, for instance, if you think you might want to move the iSCSI data to another box at some point, keep it in its own VG (with its own set of drives) to make that easy.

	      Stuart D. Gathman <[email protected]>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

I also use VGs when I have several classes of PV with radically different properties (especially failure properties) and want to be sure that the failure of one PV can't possibly affect the LVs on the devices with other failure properties.

At one point I had three VGs on one of my systems. One spanned PVs on top of a RAID-5 md array. That array was composed of disks of radically different sizes, so I had another VG spanning PVs consuming all that slack space. Then, for testing purposes, I had another VG sitting entirely atop the network block device.

The NBDed VG could be expected to go down fairly frequently (as NBD connections don't outlast server reboots), but I didn't want that to affect either of the local VGs. The non-RAIDed VG might fail if a single disk dies, but that only had news spool and caches on it so I didn't want it to affect the important stuff on the RAIDed VGs.

Hence, three VGs made considerable sense.