Skip to contentSkip to conversation

vgchange doesn't enable found VGs

linux-lvm Brian J. Murrell 3 participants 9 messages Feb 27, 2009

lore
Share & tools
lore
#0 lore raw

Strange:

+ /usr/sbin/vgscan --ignorelockingfailure Reading all physical volumes. This may take a while... Found volume group "foobar" using metadata type lvm2
+ /usr/sbin/vgchange -a y --ignorelockingfailure No volume groups found

It seems I have to tell vgchange to enable the foobar VG explicitly.

Why is this?

b.

#1 lore raw
Quoted 11 lines · Brian J. Murrell

Strange:

+ /usr/sbin/vgscan --ignorelockingfailure Reading all physical volumes. This may take a while... Found volume group "foobar" using metadata type lvm2 + /usr/sbin/vgchange -a y --ignorelockingfailure No volume groups found

It seems I have to tell vgchange to enable the foobar VG explicitly.

Why is this?

It's difficult to say from the output above. Try running the tools with "-vvv" and looking at the detailed output. They'll produce a lot of output when run this way - you may want to use something like pastebin & send a link to the list rather than posting the entire output.

Why do you need the --ignorelockingfailure?

Bryn.

#2 lore raw
Quoted 4 lines · Bryn M. Reeves

It's difficult to say from the output above. Try running the tools with "-vvv" and looking at the detailed output. They'll produce a lot of output when run this way

Not so much in fact.

Quoted 1 line

Why do you need the --ignorelockingfailure?

Because this is being done during system boot.

Here's "vgchange -v -v -v -a n" output:

Processing: vgchange -v -v -v -a n
O_DIRECT will be used
Setting global/locking_type to 1
File-based locking selected.
Setting global/locking_dir to /var/lock/lvm
Finding all volume groups
/dev/ramdisk: Not using O_DIRECT
Opened /dev/ramdisk RO
/dev/ramdisk: block size is 4096 bytes
/dev/ramdisk: No label detected
Closed /dev/ramdisk
/dev/ram: Not using O_DIRECT
Opened /dev/ram RO
/dev/ram: block size is 4096 bytes
/dev/ram: No label detected
Closed /dev/ram
/dev/ram2: Not using O_DIRECT
Opened /dev/ram2 RO
/dev/ram2: block size is 4096 bytes
/dev/ram2: No label detected
Closed /dev/ram2
...
/dev/ram14: Not using O_DIRECT
Opened /dev/ram14 RO
/dev/ram14: block size is 4096 bytes
/dev/ram14: No label detected
Closed /dev/ram14
/dev/ram15: Not using O_DIRECT
Opened /dev/ram15 RO
/dev/ram15: block size is 4096 bytes
/dev/ram15: No label detected
Closed /dev/ram15
No volume groups found

The eclipses represents ram3, ram4, ram5, to ram 13.

When I specify the volume group, I see this:

Processing: vgchange -v -v -v -a n lustre
O_DIRECT will be used
Setting global/locking_type to 1 File-based locking selected.
Setting global/locking_dir to /var/lock/lvm
Using volume group(s) on command line
Locking /var/lock/lvm/V_lustre RB
Finding volume group "lustre"
/dev/ramdisk: Not using O_DIRECT Opened /dev/ramdisk RO /dev/ramdisk: block size is 4096 bytes
/dev/ramdisk: No label detected
Closed /dev/ramdisk
/dev/ram: Not using O_DIRECT Opened /dev/ram RO
/dev/ram: block size is 4096 bytes
/dev/ram: No label detected
Closed /dev/ram
/dev/ram2: Not using O_DIRECT Opened /dev/ram2 RO
/dev/ram2: block size is 4096 bytes
/dev/ram2: No label detected
Closed /dev/ram2
...
Wiping cache of LVM-capable devices
/dev/console: Not a block device /dev/core: Not a block device /dev/evtchn: Not a block device /dev/fd: Symbolic link to directory
...

and it considers a whole boatload more devices, including /dev/sda, where my volume groups is. Now the one thing that just occurred to me as relevant is that my root filesystem (/) is read-only.

b.

#3 lore raw
Quoted 3 lines · Brian J. Murrell

/dev/ram14: Not using O_DIRECT Opened /dev/ram14 RO /dev/ram14:
block size is 4096 bytes /dev/ram14: No label detected Closed
/dev/ram14 /dev/ram15: Not using O_DIRECT Opened /dev/ram15 RO /dev/ram15: block size is 4096 bytes /dev/ram15: No label detected Closed /dev/ram15 No volume groups found

So it's not scanning /dev/sda at all (only ramdisks?). Do you have any filter settings in lvm.conf and are you sure that the SD device nodes will have been created at this point in the boot process?

You don't mention the distribution you're using but most 2.6 based setups will use udev to manage driver loading and device node creation during boot. It's possible for these to get out of sync in some cases.

Is there any way you can grab a listing of /dev at the moment the vgchange runs during boot?

Quoted 6 lines

When I specify the volume group, I see this:

Processing: vgchange -v -v -v -a n lustre O_DIRECT will be used Setting global/locking_type to 1 File-based locking selected. Setting global/locking_dir to /var/lock/lvm Using volume group(s) on command line Locking /var/lock/lvm/V_lustre RB Finding volume
group "lustre" ...

Did it scan /dev/sda this time?

Quoted 7 lines

Wiping cache of LVM-capable devices /dev/console: Not a block device /dev/core: Not a block device /dev/evtchn: Not a block device /dev/fd: Symbolic link to directory ...

and it considers a whole boatload more devices, including /dev/sda,
where my volume groups is. Now the one thing that just occurred
to me as relevant is that my root filesystem (/) is read-only.

Shouldn't matter.

Just to be clear, are you only seeing this problem during boot or are you able to reproduce once the system has completed booting (i.e. vgchange -ay fails but vgchange -ay <vg name> succeeds)?

Your earlier messages didn't make that clear.

Regards,
Bryn.

#4 lore raw
Quoted 2 lines · Bryn M. Reeves

So it's not scanning /dev/sda at all (only ramdisks?).

So it seems, yes.

Quoted 2 lines

Do you have any
filter settings in lvm.conf

devices {
dir = "/dev"
scan = [ "/dev" ]
preferred_names = [ ]
filter = [ "a/.*/" ]
cache_dir = "/etc/lvm/cache"
cache_file_prefix = ""
write_cache_state = 1
sysfs_scan = 1
md_component_detection = 1
ignore_suspended_devices = 0
}

Quoted 2 lines

and are you sure that the SD device nodes
will have been created at this point in the boot process?

Well, the data from the previous message was done on a fully booted machine, complete with /dev/sda being present for many, many minutes if not hours.

Quoted 1 line

You don't mention the distribution you're using

RHEL5.2.

Quoted 3 lines

but most 2.6 based
setups will use udev to manage driver loading and device node creation during boot. It's possible for these to get out of sync in some cases.

Yes, but again, the data I am giving you is being taken from a completely up and running system.

Quoted 2 lines

Is there any way you can grab a listing of /dev at the moment the vgchange runs during boot?

Not relevant per above.

Quoted 1 line

Did it scan /dev/sda this time?

Yes.

Quoted 3 lines

Just to be clear, are you only seeing this problem during boot or are you able to reproduce once the system has completed booting (i.e. vgchange -ay fails but vgchange -ay <vg name> succeeds)?

As per the above, I can reproduce on a completed booted system.

b.

#5 lore raw
Quoted 1 line · Brian J. Murrell

cache_dir = "/etc/lvm/cache"

The cache_dir must be *writeable*.
If you're attempting to use read-only root, then move it somewhere else (is /var writeable? or make a ramfs) or delete that directory *and* change

Quoted 1 line

write_cache_state = 1

to 0.

(This will slow commands down.)

Or if all devices on your system are the same on every boot, place the correct version of that file into your read only image. This won't work if devices can move about unless you include all the possible locations in there.

Quoted 3 lines
Quoted 2 lines

Is there any way you can grab a listing of /dev at the moment the vgchange runs during boot?

Not relevant per above.

-vvvv output provides this information already.

Alasdair

Signature · 1 line
[email protected]
#6 lore raw
Quoted 4 lines · Alasdair G Kergon
Quoted 1 line · Brian J. Murrell

cache_dir = "/etc/lvm/cache"

The cache_dir must be *writeable*.

And persistent or just writable?

Interestingly enough, RHEL5.2's /etc/rwtab (the files and directories that need to be made "magically" writable -- through bind mounting to either persistent storage or tmpfs) has:

files /etc/lvm/.cache

That's starting to look like a type methinks.

Quoted 3 lines

If you're attempting to use read-only root, then move it somewhere else (is /var writeable? or make a ramfs) or delete that directory *and* change

Ahhh. "a ramfs". So persistence is not a requirement it seems.

b.

#7 lore raw
Quoted 8 lines · Brian J. Murrell
Quoted 4 lines · Alasdair G Kergon
Quoted 1 line · Brian J. Murrell

cache_dir = "/etc/lvm/cache"

The cache_dir must be *writeable*.

And persistent or just writable?

Quoted 7 lines

Interestingly enough, RHEL5.2's /etc/rwtab (the files and directories that need to be made "magically" writable -- through bind mounting to either persistent storage or tmpfs) has:

files /etc/lvm/.cache

That's starting to look like a type methinks.

But this is moot as the /etc/rwtab bind mounts are not configured until after lvm in the rc.sysinit and crappity crap crap: there is nothing mounted at that point in the boot that is writable.

But changing (only) "write_cache_state = 0" in the lvm.conf made no difference in vgchange's inability to find the volume group.

b.

#8 lore raw

You wrote:
my root filesystem (/) is read-only.

That's what I was reacting to.
With a read-only root, if /etc/lvm/cache/ is being used and is wrong it will be unable to correct itself.
You must either make the cache file writeable or delete it.

vgchange -ay
and
vgchange -ay XXX

behave differently.
The first activates all VGs it sees, respecting the cache file if it exists.
But with the second, if after doing that (and unless trustcache is set) it can't find XXX, it considers the cache file might be wrong and tries again without it, updating it afterwards iff write_cache_file is set.

Whether or not to preserve that file across boot depends on your system, but typically vgscan will be run each time to regenerate it.

Alasdair

Signature · 1 line
[email protected]