Migrating to an LVM system (boot/root) disk

I would like to migrate my system disk to using LVM, completely -- boot and root filesystems (as well as others). I have 0.9.1beta7 as well as the lilo patch posted previous to this list.

Fortunately I have a new ide disk that is larger than my system disk currently so I can migrate to it. My current system disk is on /dev/hda and the new disk is on /dev/hdc. I have created one large /dev/hdc1 partition on the new disk.

I have also created a PV, a VG and two LVs -- root and boot. I have mounted the lvs and copied my boot and root filesytems to the PVs. I also create the lvm initrd. So far so good.

I edited my /etc/lilo.conf on the new root filesystem to change the following:

...
-boot=/dev/hda
+boot=/dev/hdc
...
image=/boot/vmlinuz
label=linux

-       root=/dev/hda5
+       root=/dev/system/boot
+       initrd=/boot/initrd-lvm-2.4.5-1.1mdk.gz

read-only

However when I chroot to the root filesystem on the new disk and run the lilo command I get the following:

# lilo
Warning: /dev/hdc is not on the first disk
Fatal: This version of LVM does not support boot LVs

The warning I understand but it would seem there is some problem with what I have cobbled together here as my lilo does not like my LVM for booting.

One other thing that has occurred to me is that the PV that underlies the VG was named as /dev/hdc1. I have a suspicion that I will not be able to simply move that drive from the secondary IDE bus to the primary IDE bus as it will then be known as /dev/sda1. Is this correct?

Is there a better way to do this migration if my desire is really to have one large parition/PV for my VG rather than using a bunch of partitions/PV's?

Any comments most welcome.

Thanx much, in advance,
b.

P.S. I also have horrible nightsweats of trying to boot an LVM box from "emergency" boot media (diskettes, CDROM, etc.). :-)

Brian J. Murrell

1. In the LFS howto, I saw a warning to the effect of: NEVER run lilo from a chrooted environment. You can destroy your MBR making your system bootable only from floppy... Dunno if it applied here.

2. Having a /boot on LVM *is* possible, BUT you will need to boot from floppy. This is because the info on /boot is needed at boot time, and without it you won't be able to even get an init ramdrive. (unless there is a new feature of LVM that allows this?)

3. to get over your night horrors, make a boot disk that includes the LVM tools. A good starting point is miniroot. I posted a URL to it on this list a little while ago. I don't have it to hand but it'll be in the archive!
:)

If you do not tell it which disk to write the MBR to, yes, that would be true.

No you don't. There are several in depth discussions in the archive about doing this including a posting (several times in fact) of a patch to Lilo to help it locate the kernel and initrd on an LVM /boot. Please see the archives for details.

disks! My initrd is 1.3MB alone!

Yeah, I realize that making boot diskettes is what I have to do, but the "automated" mkbootdisk that some distros ship as well as the various root/boot kits out there sure are not going to inclue the LMV tools. Having to manually craft the pair is a pain. :-)

Perhaps just like LVM has a nice little script for creating the initrd, they can ship with a nice little script for creating root/boot diskettes.

Of course a lot of this headache goes away if the kernel had the "auto-detection" code like it does for MDs.

b.

Brian J. Murrell

Cool, I'll check it out!

compressed? Two disks (1 for kernel 1 for root) is not unusual.

No need.
Given the miniroot.gz file, one uncompresses it, mounts it (eg)
mount -o loop miniroot /mnt/loop/

then simply copy the lvm tools over to /mnt/loop/sbin You can change the init script too.
Then just unmount it and regzip it and rerun lilo.

The lvm initramdrive can be mounted in the same way. It should give some clues.

Yes and I hope the mkinitrd (whatever its called...) is fixed so that it copies sed to the ramdrive...

ahhhh that would be cool!
:)
Shoudn't be too far off?

Steve Wray wrote:
[]

Hey, sed should NOT be here. Initrd exeists for exactly ONE version of kernel, so you know what IOP that kernel uses and what usertools should be placed to initrd BEFORE doing initrd, and sed can/should be used only at initrd creation stage. Idea: why not put to initrd ALL modules for ALL kernels your system have, and update it each time you add a new kernel to your system? ;)

Regards,
Michael.

uh?
I really don't understand this part -- the lvm tools on my system work for more than one kernel, why not the tools on the init ramdrive?

How hard can it be?

Whats IOP anyhow?
(I'm no newbie -- I've been messing with Linux since '93 and I never came across "IOP" before. Someone want to de-ignorance me? Please?)

I'd ask the opposite: if your initrd is for exactly ONE kernel, why it should contain stuff for many kernels? See below.

[]

Don't remember exactly, or maybe even misspelled. I mean an lvm protocol version between kernel and userland tools here. In short, protocol in kernel and userland should match or else things will not work. Due to many protocol changes, some lvmtools wan't work with some kernels (or again the opposite: only a few lvm tools will work with any given kernel). This is a headache, and someone (Andreas Dilger?) introduced wrapper for every lvm program, that first asks kernel for a protocol it implements, and then calls appropriate lvm tools from /lib/lvm-$VERSION or something like that. When you're in initrd, you already know (before even that initrd was created) what version will be used in a kernel for that you make this initrd, and the above wrappers not needed at all, you can jjust put appropriate lvmtools there. Yes, you can put tools for all versions of a protocol and use a wrapper in initrd, but *why*?? ;) Sed is needed to find that proto version based on output of an utility, and used inside a wrapper.

[Please someone correct me if I'm wrong here about sed's usage.]

Regards,
Michael.

Yes, you are correct that you know in advance which IOP version you need. However, it is much easier for lvmcreate_initrd to just copy all of the programs (including the wrapper) to the initrd, instead of trying to find which ones are needed and which are not needed. In the end, the wrapper is one disk block + a few directory entries (no more space needed).

The problem is that "sed" is used in the wrapper script. AFAIK, I do not have "sed" in my wrapper script, so I don't know where it came from. Can you tell me where the wrapper script comes from that you are using and/or post a copy to this list so it can be fixed?

Cheers, Andreas

Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

[]

The wrapper is small enouth, but not all the tools. If using only one version of lvmtools in initrd, then wrapper is just unnecessary complicates things (may be not an issue as you demonstrated). If not, then size is in trouble. BTW, I see no reasons on not using small wrapper's piece directly in mkinitrd, the only thing it should support is to determine lvm version from on-disk kernel file (is it possible?), not from a running kernel (or else all the tool versions should be in initrd).

I leave that to Steve Wray -- I personally not use any wrappers at all, and ever never saw mkinitrd script(s)... ;)

Regards,
Michael.

[snip]

Can

and/or

phew I have no idea where the wrapper script came from, if not from LVM? Lets see. lvscan --version yields;
Logical Volume Manager 0.9.1_beta7
Heinz Mauelshagen, Sistina Software 10/04/2001 (IOP 10)

Also,

$ file `which lvscan`
yields;
Bourne shell script...

lvmcreate_initrd is also a script.

Need any more info?

Hmm...I put 'sed' in the debian package wrapper scripts since Debian put's basename in /usr/bin and if you don't use an initrd script and you have your /usr partition on LVM, you can't use the wrapper...

Is there an easy way we can work around this? Maybe hack up the create_initrd script a bit more? I can just adjust it in the debian packages too i guess.

Regards,

AJ Lewis
Sistina Software Inc.                  Voice:  612-638-0500
1313 5th St SE, Suite 111              Fax:    612-638-0500
Minneapolis, MN 55414                  E-Mail: [email protected]
http://www.sistina.com

Current GPG fingerprint = 3B5F 6011 5216 76A5 2F6B  52A0 941E 1261 0029 2648
Get my key at: http://www.sistina.com/~lewis/gpgkey
 (Unfortunately, the PKS-type keyservers do not work with multiple sub-keys)

-----Begin Obligatory Humorous Quote----------------------------------------
FATAL ERROR! SYSTEM HALTED! - Press any key to do nothing...
-----End Obligatory Humorous Quote------------------------------------------

AJ Lewis wrote:
[]

So basename also should be placed to initrd? ;) I think that problem should be resolved at a first place (in mkinitrd), not by putting numerous workarounds all other the place. BTW, bash have nice construct ${var##*/} to be used instead of basename, and it works *much* faster (also ${var%/*} for dirname etc).

Regards,
Michael.

naw it ain't bash, luckily thass a posix construct please use parameter expansion instead of basename/dirname in any script.

L.

Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Hmm...that's pretty nice. I'll put that in the next revision of the packages i put together for LVM. If I don't, someone smack me please.

Regards,

AJ Lewis
Sistina Software Inc.                  Voice:  612-638-0500
1313 5th St SE, Suite 111              Fax:    612-638-0500
Minneapolis, MN 55414                  E-Mail: [email protected]
http://www.sistina.com

Current GPG fingerprint = 3B5F 6011 5216 76A5 2F6B  52A0 941E 1261 0029 2648
Get my key at: http://www.sistina.com/~lewis/gpgkey
 (Unfortunately, the PKS-type keyservers do not work with multiple sub-keys)

-----Begin Obligatory Humorous Quote----------------------------------------
To know recursion, you must first know recursion.
-----End Obligatory Humorous Quote------------------------------------------

I tried this in my lvm-wrapper and it works fine, but on my system /bin/sh is /bin/bash. This also works with /bin/zsh, but doesn't work with /bin/ash.

One issue is how to get this fix out. Since the lvm-wrapper is not part of the LVM tools itself, it is up to vendors to read this email and fix it...

Cheers, Andreas

Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

I wrote a wrapper in c and posted it to this list some months ago,

Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

That's because LVM 0.9.1b7 is broken w.r.t. LILO. It worked for a while between b4 and b6. I've been meaning to supply a patch for this (I can even check it into CVS directly now ;-) but have not had a chance to do so. For now, use the below patch.

Basically, the LILO patch does a "test" mapping on the given device, with two blocks within a single PE a fixed distance apart. If the mapped blocks do not return the same difference, then LVM is broken (as you see in your case).

Initially, this was done because LV_BMAP would take blocks as input values and return sectors as output values, which somewhat screwed up LILO. At the time, the LV_BMAP ioctl was totally broken anyways (i.e. it would crash you machine if you tried to use it), so I didn't have any quandries about changing the interface to be more sane (i.e. block numbers for both input and output).

You must have a very strange system if your IDE drive will be known as /dev/sda1 (maybe IDE-SCSI)? In any case, yes I think that this will screw up your boot sector. What you can try is the "disk=" parameter in your lilo.conf, saying /dev/hdc has BIOS number 0x80, so that when you shuffle your disks the boot sector will be correct.

I put a (non-LVM) rescue boot partition on my disk which has basically everything from /bin, /sbin, and some /lib/lib*, /lib/modules/<kernel>. This is basically self-contained, and after I got it to work I have left it totally alone to avoid breaking it just in time to have a boot problem.

Cheers, Andreas
========================================================================

diff -u -u -r1.18 lvm.c
@@ -1174,6 +1181,7 @@
 	bh.b_blocknr = block;
 	bh.b_dev = bh.b_rdev = inode->i_rdev;
 	bh.b_size = lvm_get_blksize(bh.b_dev);
+	bh.b_rsector = block * (bh.b_size >> 9);
 	if ((err=lvm_map(&bh, READ)) < 0)  {
 		printk("lvm map failed: %d\n", err);
 		return -EINVAL;

Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Ahhh. Now that makes sense. :-)

Oops. I am a SCSI-head. I don't normally use IDE but I am giving ATA66 a whirl on my workstation. Because of my predominant use of SCSI /dev/sd* is what I am used to dealing with. Of course I meant /dev/hda1 rather than /dev/sda1

It's not so much the boot sector I am worried about. What concerns me is that when I set up my new disk (on /dev/hdc1) with LVM, LVM is associating the device /dev/hdc1 with the PV. What happens to that association when I move the disk to the primary ide bus and it becomes /dev/hda1?

Everything that is needed for LVM you mean or do you mean you copied everything from your root filesystem? If the latter, would that not be overkill?

But I like the idea. I was thinking it was time to move on from emergency boot diskette(s) to a boot CD-ROM.

Thanx Andreas,
b.

Brian J. Murrell

Brian, you write:

If you run a vgscan within initrd (and also after you have moved to the new disk), then LVM will know that the disk has moved. Not a problem.

Not necessarily everything. It is about 50MB or so, but it is a fully usable environment which allows me to fix problems. I need this on occasion because I do a lot of development on that system and run into bad kernels, unmountable root filesystems, etc. on occasion.

The nice thing about an emergency boot partition is that you always have it handy - no chance of losing this one.

Cheers, Andreas

Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert