Skip to contentSkip to conversation

Adding a disk to expand an existing logical volume

linux-lvm kevin 11 participants 26 messages May 19, 2010 – May 28, 2010

lore
Share & tools
lore

kevin

author
#0 lore raw

Howdy,

I have a live production system w/ logical volume that i want to add a drive to. I just wanted to confirm that adding a drive (fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding a disk to logical volumes? I want to ensure that the existing data on the logical volume isn't compromised and have never done this before.

If theres something I should be aware of or if my process is incorrect, please let me know. Any help is extremely appreciated J

Thanks,

Kevin

#1 lore raw
Quoted 3 lines · kevin

Howdy,

(fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org
author
#2 lore raw
Quoted 3 lines

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

Thanks for the quick response. So to recap, for a 144gb drive (/dev/sda1) :

1. fdisk /dev/sda1 type 8e

2. pvcreate /dev/sda1

3. vgextend existing_group_name /dev/sda1

4. lvextend -L+144G /dev/VolGroup00/existing_group_name

Does that look correct?

Thanks so much once again,

Kevin
http://blog.stardothosting.com

#3 lore raw
Quoted 10 lines · kevin
Quoted 2 lines

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

Thanks for the quick response. So to recap, for a 144gb drive (/dev/sda1) :

1. fdisk /dev/sda1 type 8e
2. pvcreate /dev/sda1
3. vgextend existing_group_name /dev/sda1
4. lvextend -L+144G /dev/VolGroup00/existing_group_name

Quoted 5 lines

Does that look correct?

Thanks so much once again,

Kevin

Hi Kevin,

I've been meaning to add this process to my wiki for a while now.
This is as good a time as any, so take a gander here:

http://wiki.alteeve.com/index.php/Adding_Space_to_an_LVM

Note that it's a copy-pasted from some of my other docs, so some of
it will reference non-existing things, but the steps should all be there.

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org
#4 lore raw
In reply to Digimer
Quoted 7 lines · Digimer [[email protected]] wrote:
Quoted 3 lines · kevin

Howdy,

(fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

author
#5 lore raw
Quoted 3 lines

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

I intend for the entire disk to be used -- so I should just skip the fdisk step?

Thanks,

Kevin
http://blog.stardothosting.com

#6 lore raw
Quoted 10 lines · kevin
Quoted 3 lines

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

I intend for the entire disk to be used -- so I should just skip the fdisk step?

Quoted 5 lines

Thanks,

Kevin
http://blog.stardothosting.com

Unless Malahal knows something I don't (which is always possible), then you still need to create a single partition that consumes the whole disk and then assign that to you VG.

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org
#7 lore raw
Quoted 14 lines · Digimer
Quoted 13 lines · kevin
Quoted 3 lines

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

I intend for the entire disk to be used -- so I should just skip the fdisk step?

Thanks,

Quoted 7 lines
Quoted 2 lines

Kevin
http://blog.stardothosting.com

Unless Malahal knows something I don't (which is always possible), then you still need to create a single partition that consumes the whole disk and then assign that to you VG.

Running pvcreate on a whole disk works just fine.

Regards,
Bryn.

#8 lore raw
In reply to kevin
Quoted 5 lines · kevin
Quoted 2 lines

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

I intend for the entire disk to be used -- so I should just skip the fdisk step?

Yes, just do the pvcreate directly on /dev/sdc (instead of /dev/sdc1).

I would only recommend this if the drive is permanently attached to your server. If there is any possibility of it getting moved to another system, especially another OS, it is likely to appear "unformatted" and the other OS will offer to create a partition table, etc, destroying the data. The partition table, although otherwise useless for LVM (and a pain when trying to align PPs to some lower lever boundary), does serve the purpose of telling other PC operating systems that there is something there.

Signature · 4 lines
	      Stuart D. Gathman <[email protected]>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
#9 lore raw
Quoted 10 lines · Stuart D. Gathman
Quoted 5 lines · kevin
Quoted 2 lines

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

I intend for the entire disk to be used -- so I should just skip the fdisk step?

Yes, just do the pvcreate directly on /dev/sdc (instead of /dev/sdc1).

Quoted 7 lines

I would only recommend this if the drive is permanently attached to your server. If there is any possibility of it getting moved to another system, especially another OS, it is likely to appear "unformatted" and the other OS will offer to create a partition table, etc, destroying the data. The partition table, although otherwise useless for LVM (and a pain when trying to align PPs to some lower lever boundary), does serve the purpose of telling other PC operating systems that there is something there.

There ya go, I learn something new every day. :)

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org
author
#10 lore raw
Quoted 2 lines

I would only recommend this if the drive is permanently attached to your server. If there is any possibility of it getting moved to another

system,

Quoted 5 lines

especially another OS, it is likely to appear "unformatted" and the other OS will offer to create a partition table, etc, destroying the data. The partition table, although otherwise useless for LVM (and a pain when trying to align PPs to some lower lever boundary), does serve the purpose of telling other PC operating systems that there is something there.

This drive is to be permanently attached to the system so I don't think I need to worry about marking it as 8e in fdisk. Thanks all for your help. This clarifies a lot of things for me.

~k

author
#11 lore raw
In reply to Digimer

One last thing I want to confirm is the lvextend command to 'grow' the logical volume :

lvextend -L+144G /dev/VolGroup00/my_volume

After all said steps in previous messages on this thread are complete, is the above command correct for my particular scenario? Do I want to 'extend' the volume size or 'grow' it?

Not sure what the difference is.

~k

#12 lore raw
Quoted 1 line

lvextend -L+144G /dev/VolGroup00/my_volume

If you want to use all available space:

lvextend -L+100%FREE /dev/VolGroup00/my_volume

Quoted 4 lines

I should note that this is a logical volume for a collection of Xen virtualized environments so its not an actual EXT3 filesystem. I wouldn’t need to run the last step I imagine.

Typically Xen images are files on the filesystem
of the host, so yes, you'd need to resize the host filesystem, if you are creating Xen image files. Howver, as long as you are making changes to your LVM, there is another way which we use, because we think it's better. Rather than having one huge LV which contains files which are accessed as though they were block devices, we create a LV for each guest. If you have only one big LV, that's kind of pointless indirection, overhead that gains you nothing. It's also problematic for recovery because you have multiple overlapping ext3 filesystems - one inside the other, meaning that recovery utilities will find bogus superblocks belonging to the wrong filesystem. --
Ray Morris
[email protected]

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php

Quoted 23 lines · kevin

One last thing I want to confirm is the lvextend command to 'grow' the logical volume :

lvextend -L+144G /dev/VolGroup00/my_volume

After all said steps in previous messages on this thread are complete, is the above command correct for my particular scenario? Do I want to 'extend' the volume size or 'grow' it?

Not sure what the difference is.

~k

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#13 lore raw
Quoted 5 lines · Ray Morris
Quoted 1 line

lvextend -L+144G /dev/VolGroup00/my_volume

If you want to use all available space:

lvextend -L+100%FREE /dev/VolGroup00/my_volume

As Richard pointed out, using '+100%FREE' calls for the small 'l', not L. The capital L is used when specifying a fixed size.

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org
#14 lore raw
In reply to kevin
Quoted 11 lines · kevin

One last thing I want to confirm is the lvextend command to 'grow' the logical volume :

lvextend -L+144G /dev/VolGroup00/my_volume

After all said steps in previous messages on this thread are complete, is the above command correct for my particular scenario? Do I want to 'extend' the volume size or 'grow' it?

Not sure what the difference is.

Chiming in a little late here but I would suggest the following (although I'm certainly not a LVM expert):

lvextend -l +100%FREE /dev/VolGroup00/my_volume (notice the lower case l in -l, not upper)

This should make sure you add 100% of the extents of the new drive to your logical volume. As I've said, I'm not an expert so I don't know the innards of "lvextend" but I would be concerned that if you specify the size of the expansion in gigabytes instead of as a percentage that you might get some unallocated extents due to rounding.

Richard

#15 lore raw
In reply to Digimer
Quoted 3 lines
Quoted 3 lines

The partition table, although otherwise useless for LVM (and a pain when trying to align PPs to some lower lever boundary), does serve the purpose of telling other PC operating systems that there is something there.

It's also useful during the boot process. If, for example, you
later add another disk, this disk might become sdd, where it used to be sdc. That can confuse initrd and friends and result in an unbootable system. On almost every machine where I've skipped creating partitions for RAID or LVM, this kind of thing has come back to bite me later - maybe years later. Now, I always create "Linux LVM" partitions for LVM and "Linux raid autodetect" partitions for RAID. It makes booting an recovering much more reliable in case of changes or problems.

Quoted 1 line

a pain when trying to align PPs to some lower lever boundary

A trick to avoid all of the math of alignment is to just
start the first partition on sector 512 every time, rather than sector 63. Sometimes you can save a few KB by doing the math and finding a lower offset, but it's just a few KB - who cares. That works because 512 sectors is 256KB - a perfect multiple of any RAID size you might use.
--
Ray Morris
[email protected]

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php

Quoted 33 lines · Digimer
Quoted 10 lines · Stuart D. Gathman
Quoted 5 lines · kevin
Quoted 2 lines

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

I intend for the entire disk to be used -- so I should just skip the fdisk step?

Yes, just do the pvcreate directly on /dev/sdc (instead of /dev/sdc1).

Quoted 7 lines

I would only recommend this if the drive is permanently attached to your server. If there is any possibility of it getting moved to another system, especially another OS, it is likely to appear "unformatted" and the other OS will offer to create a partition table, etc, destroying the data. The partition table, although otherwise useless for LVM (and a pain when trying to align PPs to some lower lever boundary), does serve the purpose of telling other PC operating systems that there is something there.

There ya go, I learn something new every day. :)

Signature · 10 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
#16 lore raw
Quoted 9 lines · Ray Morris

It's also useful during the boot process. If, for example, you
later add another disk, this disk might become sdd, where it used to be sdc. That can confuse initrd and friends and result in an unbootable system. On almost every machine where I've skipped creating partitions for RAID or LVM, this kind of thing has come back to bite me later - maybe years later. Now, I always create "Linux LVM" partitions for LVM and "Linux raid autodetect" partitions for RAID. It makes booting an recovering much more reliable in case of changes or problems.

PVs are identified by UUID. It doesn't matter whether they change names. And changing names from sdc1 to sdd1 is just as bad as changing names from sdc to sdd.

However, you are correct that the default device searching rules used by lvm might work correctly with a Linux LVM labeled partition where they might otherwise skip a device. If you aren't familiar with lvm.conf, then using labeled partitions is a safer course. (initrd uses a copy of lvm.conf to find PVs so it can activate the volume group of the root filesystem if necessary.)

Signature · 4 lines
	      Stuart D. Gathman <[email protected]>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
#17 lore raw
Quoted 3 lines

If you aren't familiar with lvm.conf, then using labeled partitions
is a safer course. (initrd uses a copy of lvm.conf to find PVs so it can activate the volume group of the root filesystem if necessary.)

Perhaps my point was obscured by my other text, so I'll highlight it:

Quoted 2 lines
Quoted 2 lines

It makes booting an recovering much more reliable in case of changes or problems.

Agreed, if all the devices are listed in lvm.conf, mdadm.conf,
etc. and nothing changes or has any problems then whole devices work fine. As long as all of the configuration files list everything and initrd knows about any changes. It's when something's not exactly how it was originally configured that it's really handy for raid devices to be marked as such by using partition type "fd", LVM devices to be marked as such by using partition type "8e", etc. That's been my experience. Your experience may differ, but that's my experience from dealing with a lot of "won't boot" situations.

Having said that, if you use partitions it's important that
the partition types not be WRONG. I once spent far too long trying to figure out why my devices were undetected. It turned out a RAID or LVM partition was marked with the default "83". --
Ray Morris
[email protected]

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php

Quoted 38 lines · Stuart D. Gathman
Quoted 7 lines · Ray Morris

It's also useful during the boot process. If, for example, you
later add another disk, this disk might become sdd, where it used to be sdc. That can confuse initrd and friends and result in an unbootable system. On almost every machine where I've skipped creating partitions for RAID or LVM, this kind of thing has come back to bite me later - maybe years later. Now, I always create "Linux LVM" partitions for LVM and "Linux raid autodetect"

partitions

Quoted 2 lines

for RAID. It makes booting an recovering much more reliable in case of changes or problems.

PVs are identified by UUID. It doesn't matter whether they change names. And changing names from sdc1 to sdd1 is just as bad as changing names from sdc to sdd.

However, you are correct that the default device searching rules used by lvm might work correctly with a Linux LVM labeled partition where they might otherwise skip a device. If you aren't familiar with lvm.conf, then using labeled partitions is a safer course. (initrd uses a copy of lvm.conf to find PVs so it can activate the volume group of the root filesystem if necessary.)

--

	      Stuart D. Gathman <[email protected]>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703  

591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#18 lore raw
In reply to kevin
Quoted 7 lines · kevin
Quoted 3 lines

If I were you, I would not bother with partition(s). I will add the whole disk to LVM.

I intend for the entire disk to be used -- so I should just skip the fdisk step?

You can either create a partition that spans the entire disk or create the PV directly on the whole-disk device.

Both approaches have their advantages; with a partition table you are less likely to run into problems with (buggy) software that assumes the disks are "unformatted" because they do not contain an MBR-format partition table and handily initialises them for you. On the other hand it's somewhat easier (at least with older tools) to get alignment constraints right when creating things on the bare disk (although you still need to make sure that the start of the data area of the PV is aligned properly if you want to get the best performance).

Regards,
Bryn.

#19 lore raw
In reply to Digimer
Quoted 7 lines · Digimer
Quoted 3 lines · kevin

Howdy,

(fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

Or use the full device without any partition at all.

/Michael

#20 lore raw
Quoted 10 lines · Michael Guntsche
Quoted 7 lines · Digimer
Quoted 3 lines · kevin

Howdy,

(fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

Or use the full device without any partition at all.

I wonder if this will have any adverse affects on drives with 4KB sectors?

Signature · 1 line
Dissent is patriotic, remember?
#21 lore raw
Quoted 5 lines · Ron Johnson
Quoted 4 lines · Michael Guntsche
Quoted 3 lines · Digimer
Quoted 2 lines · kevin

Howdy,

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line

(fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

Or use the full device without any partition at all.

I wonder if this will have any adverse affects on drives with 4KB sectors?

it depends
on a disk with 4k sector, using the whole device is more likely to have a correct alignment (0 is a multiple of 4k)

on a disk with 4k sector and the first sector shifted (some drives have a jumper to achieve this, it might be called something like "XP compatibility") using the whole device, or anything not aligned to 63x512byte will result in unaligned data.

afair recent kernels should expose the alignment and sector size under /sys/block/..., (if the drive exposes that info and does not lie, that is.) and fdisk/parted/lvm/md should correctly align data start.

On a personal note, i really despise partition tables, it is a broken design from the early eighties, we should be able to move on. If you must use a GPT partition, at least it has a backup block.

L.

Signature · 6 lines
Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \
#22 lore raw
In reply to Ron Johnson
Quoted 7 lines · Ron Johnson
Quoted 6 lines · Michael Guntsche
Quoted 5 lines · Digimer
Quoted 3 lines · kevin

Howdy,

(fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

The mkfs isn't supposed to be there, just set the new drive's partition to type '8e' (Linux LVM).

Or use the full device without any partition at all.

I wonder if this will have any adverse affects on drives with 4KB sectors?

Not really, if anything it simplifies getting correct alignment for the data region of the PV.

Regards,
Bryn.

#23 lore raw
In reply to kevin
Quoted 29 lines · kevin

Howdy,

I have a live production system w/ logical volume that i want to add a drive to. I just wanted to confirm that adding a drive (fdisk+mkfs+pvcreate+vgextend+lvextend) is the right way to go about adding a disk to logical volumes? I want to ensure that the existing data on the logical volume isn’t compromised and have never done this before.

If theres something I should be aware of or if my process is incorrect, please let me know. Any help is extremely appreciated J

Thanks,

Kevin

Quoted 6 lines

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

The "mkfs" step isnt needed if you are expanding an existing logical volume. A revised procedure would be the following:

- Add disk to system
- Partition it via fdisk. Set partition type to 8e (Linux LVM)
- run pvcreate initialize the partition for LVM
- run vgextend to expand your existing volume group with the new device.
- run lvextend to grow your logical volume.
- finally, run resize2fs (assuming you are using ext3/4) to grow the file system.

Rich

author
#24 lore raw
Quoted 9 lines

- Add disk to system
- Partition it via fdisk. Set partition type to 8e (Linux LVM)
- run pvcreate initialize the partition for LVM
- run vgextend to expand your existing volume group with the new device.
- run lvextend to grow your logical volume.
- finally, run resize2fs (assuming you are using ext3/4) to grow the file system.

Rich

I should note that this is a logical volume for a collection of Xen virtualized environments so its not an actual EXT3 filesystem. I wouldn’t need to run the last step I imagine.

#25 lore raw
In reply to kevin
Quoted 5 lines · kevin

a disk to logical volumes? I want to ensure that the existing data on the logical volume isn't compromised and have never done this before.

If theres something I should be aware of or if my process is incorrect, please let me know. Any help is extremely appreciated J

While others have given you the procedure, I'll be the killjoy and point out that the reliability of the VG as a whole goes down with additional drives - unless you incorporate some kind of RAID. For instance, if the uptime ratio of a single drive is .99999 (5 sigmas), the uptime of the system with two such drives is .99998.

This is not so bad with just 2 drives (even 2 sigmas gives you .99^2 == .98),
but is something to keep in mind before adding the 3rd. I prefer software
RAID1, it provides a lot of flexibility including "cloning" a drive by simply removing it and is simple. Other configurations will provide better performance.

Signature · 4 lines
	      Stuart D. Gathman <[email protected]>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.