Skip to contentSkip to conversation

Migrating LVM

linux-lvm M.Lewis 5 participants 8 messages Apr 21, 2010

lore
Share & tools
lore

M.Lewis

author
#0 lore raw

I'm building a new machine with RAID1+LVM2. I plan to migrate the data from the old machine HD to the new machine.

Is it better to make the new VG the same name as the existing VG on the old box? Or should I make it different.

Is there an easy way to do this that I've not yet found? I was thinking to make the new VG the same name as the old, add the old drives to the machine then pvmove the data.

Thanks for any pointers!
Mike

#1 lore raw
Quoted 6 lines · M.Lewis

I'm building a new machine with RAID1+LVM2. I plan to migrate the data from the old machine HD to the new machine.

Is it better to make the new VG the same name as the existing VG on the old box? Or should I make it different.

Is there an easy way to do this that I've not yet found? I was thinking to make the new VG the same name as the old, add the old drives to the machine then pvmove the data.

You cannot pvmove between two different VGs
You cannot have two VGs with the same name

You could skip creating the new vg, vgextend the old vg on the new raid1, then pvmove, but i don't know if you need to change any parameter in the new vg.
Another option is file level copy or dd-ing the logical volume.

L.

Signature · 6 lines
Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \
#2 lore raw
In reply to M.Lewis

pvmove is good for when you have to keep the machine
live during the copy. dd is about 10 times as fast if you can be down during the copy. This specific dd invocation is the fastest I've found for the purpose, running 2-3 times faster than a simple dd without arguments:

dd if=/dev/old_vg/$1 bs=64M iflag=direct | dd of=/dev/new_vg/$1 bs=64M oflag=direct --
Ray Morris
[email protected]

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php

Quoted 16 lines · M.Lewis

I'm building a new machine with RAID1+LVM2. I plan to migrate the data from the old machine HD to the new machine.

Is it better to make the new VG the same name as the existing VG on the old box? Or should I make it different.

Is there an easy way to do this that I've not yet found? I was thinking to make the new VG the same name as the old, add the old drives to the machine then pvmove the data.

Thanks for any pointers!
Mike

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#3 lore raw
Quoted 8 lines · Ray Morris [[email protected]] wrote:

pvmove is good for when you have to keep the machine
live during the copy. dd is about 10 times as fast if you can be down during the copy. This specific dd invocation is the fastest I've found for the purpose, running 2-3 times faster than a simple dd without arguments:

dd if=/dev/old_vg/$1 bs=64M iflag=direct | dd of=/dev/new_vg/$1 bs=64M oflag=direct

Interesting! You are doing direct I/O to avoid copying from cache to user buffer for read and vice-versa for write, but you are losing the ability to do them parallel! You are doing the next best, that is creating two "dd" threads -- one for reading and another for writing. Since the pipe is really implemented in memory, why should this be faster than normal "dd" that uses page cache? Likely that kswapd is not kicking early enough?

Enhancing "dd" to create a reader and a writer thread would really help, I believe.

Thanks, Malahal.

#4 lore raw
Quoted 7 lines · On 4/21/2010 11:36 AM, [email protected] wrote:

Interesting! You are doing direct I/O to avoid copying from cache to user buffer for read and vice-versa for write, but you are losing the ability to do them parallel! You are doing the next best, that is creating two "dd" threads -- one for reading and another for writing. Since the pipe is really implemented in memory, why should this be faster than normal "dd" that uses page cache? Likely that kswapd is not kicking early enough?

Oops, my eyes missed the pipe and second dd when I made my previous comments. That is pretty good for different disks then yes... not so good for same physical disk.

Quoted 2 lines

Enhancing "dd" to create a reader and a writer thread would really help, I believe.

I actually have some old hacked up dd code I made once to use 16 concurrent aio requests with O_DIRECT. I need to clean it up a bit but it showed great promise.

#5 lore raw
Quoted 3 lines

Oops, my eyes missed the pipe and second dd when I made my previous comments. That is pretty good for different disks then yes... not so good for same physical disk.

Indeed my tests were done copying from the "old" disk
to the "new" disk, as the OP is doing, I believe.

Quoted 3 lines

I actually have some old hacked up dd code I made once to use 16 concurrent aio requests with O_DIRECT. I need to clean it up a bit but it showed great promise.

Considering how often "dd" is used for copying large amounts
of data, even a modest improvement could save many thousands of hours of admin time. I would like to encourage you to do any needed cleanup and make it available, preferably integrated with GNU dd - it could save hundreds of thousands of dollars worth of time.
--
Ray Morris
[email protected]

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php

Quoted 32 lines · Phillip Susi
Quoted 1 line · On 4/21/2010 11:36 AM, [email protected] wrote:

Interesting! You are doing direct I/O to avoid copying from cache

to user

Quoted 1 line

buffer for read and vice-versa for write, but you are losing the

ability

Quoted 1 line

to do them parallel! You are doing the next best, that is creating

two

Quoted 1 line

"dd" threads -- one for reading and another for writing. Since the

pipe

Quoted 1 line

is really implemented in memory, why should this be faster than

normal

Quoted 2 lines

"dd" that uses page cache? Likely that kswapd is not kicking early enough?

Oops, my eyes missed the pipe and second dd when I made my previous comments. That is pretty good for different disks then yes... not so good for same physical disk.

Quoted 2 lines

Enhancing "dd" to create a reader and a writer thread would really help, I believe.

I actually have some old hacked up dd code I made once to use 16 concurrent aio requests with O_DIRECT. I need to clean it up a bit but it showed great promise.

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

author
#6 lore raw
Quoted 11 lines · Ray Morris
Quoted 3 lines

Oops, my eyes missed the pipe and second dd when I made my previous comments. That is pretty good for different disks then yes... not so good for same physical disk.

Indeed my tests were done copying from the "old" disk
to the "new" disk, as the OP is doing, I believe.

Quoted 3 lines

I actually have some old hacked up dd code I made once to use 16 concurrent aio requests with O_DIRECT. I need to clean it up a bit but it showed great promise.

Quoted 9 lines

Considering how often "dd" is used for copying large amounts
of data, even a modest improvement could save many thousands of hours of admin time. I would like to encourage you to do any needed cleanup and make it available, preferably integrated with GNU dd - it could save hundreds of thousands of dollars worth of time.

Signature · 2 lines
Ray Morris
[email protected]

Thanks to all for the suggestions. I believe I will go the dd route. That seems like the best (simplest) option.

Thanks,
Mike

#7 lore raw
In reply to Ray Morris
Quoted 8 lines · Ray Morris

pvmove is good for when you have to keep the machine
live during the copy. dd is about 10 times as fast if you can be down during the copy. This specific dd invocation is the fastest I've found for the purpose, running 2-3 times faster than a simple dd without arguments:

dd if=/dev/old_vg/$1 bs=64M iflag=direct | dd of=/dev/new_vg/$1 bs=64M oflag=direct

What arguments are best depend on many factors. These particular ones should be rather good if the src and and dest volumes are on the same physical disk. It would not be very good if they are on different disks, since while dd is reading from the source, the output disk is idle, and vice versa. If src and dest are different disks, then you want to not use direct, and stick with a more reasonable block size, say 64k.

Also if the fs has much free space in it then dd will waste a lot of time on that. You probably would save a lot of time doing something like dump | restore. This also does not require that the source and destination be the same size.