lvm and md

Should we build lvm volumns on top of soft raid(md) or build md on top of lvm volumns?
I prefer to build md on top of lvm volumns because lvm can hide the hardware details from md. Just want to know how do you think about this. Or any URLs? Thanks.

Peter Yao

Peter,
both ways are fine depending on your priorities.

I'ld prefer to group disks together with MD for resilience (i.e RAID1) and then use those as LVM physical volumes to have the flexibility to allocate space to logical volumes online.

More convenient this way rather than changing raidtabs which cause interruptions to your volumes on size changes.

Because a core feature of LVM is to resize logical volumes online, the given resizing capabilities of filesystems (i.e. reiser, xfs) allow you to grow the filesystem online as well.

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
[email protected]                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

why would you need to hide the hardware details from md ? The way i do it is to md 2 disks together into a pair, and then pvcreate it and add it to my VG. That way i can get more and more disks into the system. I have thought about creating 2 VG's, and then run md ontop of that, but it's more work, as you would have to resize 2 LV's, and besides, a harddisk failure might trash one of your VG's. Meaning more repair work.

JonB

I am using some hot-plugging disks and their device number may change after reboot. For example, if i build md with /dev/sda and /dev/sdb, after reboot md may cannot start because the two disks' device name change to /dev/sdc and /dev/sdd. If i build md on top of lvm this problem is avoided. Can you give me some advice for this issue?

Peter Yao

it depends on what you use to start your md devices, using raidstart with an /etc/raidtab file shows this behaviour, using auto-detect does not but is only available at boot. best thing would be to use mdadm instead of raidstart

http://www.cse.unsw.edu.au/~neilb/source/mdadm

with this you can define your raid arrays based on UUID instead.

to create mdadm conf file use:

echo "DEVICE partitions" >> /etc/mdadm.conf
mdadm -E -s |grep ARRAY >> /etc/mdadm.conf

Luca Berra -- [email protected]
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

I don't think it would be a good idea to put MD on top of LVM for reasons of performance. Ideally you want as few layers as possible on top of the disk.

As an alternative to Luca Berra's suggestion to use mdadm, it might be possible to do what you want using devfs.

I'm not saying it's a better approach, just a possible alternative. The best approach will depend on your particular situation.

(The mdadm approach looks like a good technical solution to your situation. However, for example, you may already have devfs installed, in which case it might be easier to set up and "good enough" to be going on with.)

I have never used devfs myself, by from what I can tell it gives logical names to SCSI devices, which are invariant of devices being added to or removed from a SCSI bus.

So your HDDs would appear as something like:

/dev/scsi/host0/bus0/target5/lun0/disc
/dev/scsi/host0/bus0/target6/lun0/disc

You mention that /dev/sda and /dev/sdb are sometimes displaced to /dev/sdc and /dev/sdd.

What are the two devices which are displacing /dev/sdc and /dev/sdd ?

Regards,

Bill

The contents of this e-mail and any attachments are confidential and may be legally privileged. If you have received this e-mail and you are not a named addressee, please inform us as soon as possible on +44 118 901 2999 and then delete the e-mail from your system. If you are not a named addressee you must not copy, use, disclose, distribute, print or rely on this e-mail. Any views expressed in this e-mail or any attachments may not necessarily reflect those of Tao's management. Although we routinely screen for viruses, addressees should scan this e-mail and any attachments for viruses. Tao makes no representation or warranty as to the absence of viruses in this e-mail or any attachments. Please note that for the protection of our business, we may monitor and read e-mails sent to and from our server(s).

Putting MD on top of LVM puts in one extra layer.

This layer will have to be traversed for every single access to the disk.

Actually having a layer is unnecessary.

All that is necessary is to connect the MD to the right disk device at the beginning when the MD is being started.

The MD is then connected to the disk with no intervening layers.

Both of these suggestions address the problem of getting the MD connected to the right disk at the beginning, without adding an intervening layer.

Bill

The contents of this e-mail and any attachments are confidential and may be legally privileged. If you have received this e-mail and you are not a named addressee, please inform us as soon as possible on +44 118 901 2999 and then delete the e-mail from your system. If you are not a named addressee you must not copy, use, disclose, distribute, print or rely on this e-mail. Any views expressed in this e-mail or any attachments may not necessarily reflect those of Tao's management. Although we routinely screen for viruses, addressees should scan this e-mail and any attachments for viruses. Tao makes no representation or warranty as to the absence of viruses in this e-mail or any attachments. Please note that for the protection of our business, we may monitor and read e-mails sent to and from our server(s).

Hello,

Am I making some stupid mistake here?
I want to extend the space on /dev/vg1/mysql on physical volume /dev/sdb4

The command I tried is:

[root@cms-db root]# lvextend -L +50G /dev/vg1/mysql /dev/sdb4
lvextend -- extending logical volume "/dev/vg1/mysql" to 100 GB
lvextend -- not enough free/allocatable physical extents to extend logical volume "/dev/vg1/mysql"

But pvdisplay says there should be enough PE there:

[root@cms-db root]# pvdisplay /dev/sdb4
--- Physical volume ---

PV Name               /dev/sdb4
VG Name               vg1
PV Size               108.92 GB [228428235 secs] / NOT usable 4.25 MB [LVM: 236 KB]
PV#                   1
PV Status             available
Allocatable           yes
Cur LV                2
PE Size (KByte)       4096
Total PE              27883
Free PE               18358
Allocated PE          9525
PV UUID               C7C5NM-PpaJ-hSbj-jPqn-f1TF-mvuQ-M7a70T

lvdisplay says the following:

[root@cms-db root]# lvdisplay /dev/vg1/mysql
--- Logical volume ---

LV Name                /dev/vg1/mysql
VG Name                vg1
LV Write Access        read/write
LV Status              available
LV #                   1
# open                 1
LV Size                50 GB
Current LE             12800
Allocated LE           12800
Stripes                2
Stripe size (KByte)    16
Allocation             next free
Read ahead sectors     10000
Block device           58:0

According to the FAQ I should not specify the physical volume, and when I tried that it worked, but now O don't know where my space is allocated.

Ron Arts

Netland Internet Services
bedrijfsmatige internetoplossingen

http://www.netland.nl   Kruislaan 419              1098 VA Amsterdam
info: 020-5628282       servicedesk: 020-5628280   fax: 020-5628281

Let's split up, we can do more damage that way.

sure. Dont change device names ;-D

Perhaps you can use labels ? and label the partitions ?

Another thing is that i dont think the kernel uses /etc/raidtab i think it looks at the partitions and then starts the md. You can set the partition type to raid autodetect. Then it might work even if you change the partition names.

JonB

I propose to use LVM on top of md because that way you have to deal with one disk mirror only instead of several mirrors per LV.

Stephan