Exporting VG
I have a cluster with 4 nodes. 3 of them has several disks which I need to see as an only one disk. After a really strong work I got LVM working on all the 3 machines !!!! i created three Volume Groups, one for machine. Now, I would like to export them to get only one virtual group. I am trying to export them to a machine and merge the volume groups but I do not success trying to export them.I have not find many documentation about this ?!?
Thats my 3 machines: N1, N2 and N3. From N2 I try to export my VG to N1.
N2> vgchange -a n 'vg_name'
N2> vgexport 'vg_name'
From N1 I type,
N1>vgimport 'vg_name' /dev/sda3 (this vg is made from /dev/sda3 )
but I only get error messages !!!!
pv_create_name_from_kdev_t -- LEAVING with dev_name: /dev/sda3 pv_read -- LEAVING with ret: 0
vgimport -- ERROR: physical volume "/dev/sda3" doesn't belong to an exported volume group
Does anybody try this ?!!!!
Thanks in advance,
Xavi !
--
Xavi Fustero Benavent
Institut Municipal d'Investigacio Medica (IMIM)
Departament Genome Informatics
Tel.: +34-93-221 10 09 Ext: 2016
Quoted 4 lines · Xavi Benavent
I have a cluster with 4 nodes. 3 of them has several disks which I need to see as an only one disk. After a really strong work I got LVM working on all the 3 machines !!!! i created three Volume Groups, one for machine. Now, I would like to export them to get only one virtual group.
When you do vgexport, this does not do anything like NFS export (does NOT allow network access to the VG). It allows you to physically move the disks to another machine.
Quoted 6 lines
Thats my 3 machines: N1, N2 and N3. From N2 I try to export my VG to N1. N2> vgchange -a n '
vg_name'
N2> vgexport 'vg_name'From N1 I type,
N1>vgimport 'vg_name'/dev/sda3(this vg is made from/dev/sda3)
Are the disks connected via shared SCSI or Fibre Channel? Is /dev/sda3 the disk on the local machine?
Cheers, Andreas
Signature · 3 lines
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- DogbertQuoted 3 lines
When you do vgexport, this does not do anything like NFS export (does NOT allow network access to the VG). It allows you to physically move the disks to another machine.
Is vgexport/vgimport doing something important? I just moved a harddisk with a vg to another computer without vgexport/vgimport and didnot notice any problems. Or is it only needed if I add a vg to an already running system?
Jan
Quoted 7 lines · Jan
Quoted 3 lines
When you do vgexport, this does not do anything like NFS export (does NOT allow network access to the VG). It allows you to physically move the disks to another machine.
Is vgexport/vgimport doing something important? I just moved a harddisk with a vg to another computer without vgexport/vgimport and didnot notice any problems. Or is it only needed if I add a vg to an already running system?
I think the only thing that vgexport does is mark on disk that the VG is "exported" and not shown on vgscan or imported when a "vgchange -a" is done. On AIX, there is no difference on disk for imported and exported VGs, the VG state is stored in a config file.
IMHO, we should not store this state in the VG, nor advocate running "vgscan" on each boot. From what I have read in the past on this mailing list, running vgscan on boot is likely to delete important info from /etc/lvmtab if there is a minor problem, that has to later be restored (if possible).
It would be a lot safer to keep the state of LVM in /etc/lvmtab, and only add or remove VGs with administrator actions (explicit vgimport, vgexport, vgscan (for a new system or major problems). This at least lets you know at boot time that a VG has a problem, rather than vgscan simply deleting it from lvmtab.
What will make this a lot more likely is the work that Heinz is doing on PVIDs, so that you only store the VGID + vgname in /etc/lvmtab and then only the PVIDs in the VGDA. You don't have to worry about device names changing if disks are added/moved/removed, because you never depend on the device name.
PVIDs will also make vgimport much easier (like AIX) so you only specify a single device and the VGDA knows all of the PVIDs that belong to that volume group. Using VG names is dangerous if you are moving VGs between systems in case the two systems have identical VG names.
Cheers, Andreas
Signature · 3 lines
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- DogbertQuoted 5 lines · Andreas Dilger
Quoted 4 lines · Jan
Quoted 3 lines
When you do vgexport, this does not do anything like NFS export (does NOT allow network access to the VG). It allows you to physically move the disks to another machine.
Quoted 7 lines
Quoted 3 lines
Is vgexport/vgimport doing something important? I just moved a harddisk with a vg to another computer without vgexport/vgimport and didnot notice any problems. Or is it only needed if I add a vg to an already running system?
I think the only thing that vgexport does is mark on disk that the VG is "exported" and not shown on vgscan or imported when a "vgchange -a" is done.
Correct.
Quoted 5 lines
On AIX, there is no difference on disk for imported and exported VGs, the VG state is stored in a config file.
IMHO, we should not store this state in the VG, nor advocate running "vgscan" on each boot.
You are right that there is no need to run vgscan on each boot. Nevertheless vgscan takes care of any changes in the i/o configuration of the system in order to make physical volumes accessable using different device nodes.
Quoted 4 lines
From what I have read in the past on this mailing
list, running vgscan on boot is likely to delete important info from/etc/lvmtabif there is a minor problem, that has to later be restored (if possible).
Assumed that it is really that likely it s rather easy to get /etc/lvmtab.d/ files containing the metadata copies back, because they are format identical to /etc/lvmconf/ files./etc/lvmtab contains the 0 delimited volume group names.
Quoted 6 lines
It would be a lot safer to keep the state of LVM in
/etc/lvmtab, and only add or remove VGs with administrator actions (explicit vgimport, vgexport, vgscan (for a new system or major problems). This at least lets you know at boot time that a VG has a problem, rather than vgscan simply deleting it from lvmtab.
If we had a database keeping track of changes in the i/o configuration of the system, we could run vgscan conditionally if needed. Because there's none I recommended to run it at every boot to avoid i/o path inconsistencies.
Quoted 4 lines
What will make this a lot more likely is the work that Heinz is doing on PVIDs, so that you only store the VGID + vgname in
/etc/lvmtaband then only the PVIDs in the VGDA.
/etc/lvmtab only contains the 0 delimited volume group names.
/etc/lvmtab.d/VolumeGroupName contains a metadata copy of the current state of a volume group. In case of i/o configuration changes it is up to vgscan to update this file's contents.
In order to do i/o to the physical volumes we need to know the actual device node to access the device anyway. This could well happen at vgchange time wich wouldn't be a valuable different solution. We could change the concept though to deal with pv_uuids to access a device in the kernel as well but we needed a mapping function between uuids and device addresses within the kernel in order to achieve this.
Quoted 7 lines
You don't have to worry about
device names changing if disks are added/moved/removed, because you never depend on the device name.PVIDs will also make vgimport much easier (like AIX) so you only specify a single device and the VGDA knows all of the PVIDs that belong to that volume group.
All right.
Need to implement vgimport support for this though....
Quoted 2 lines
Using VG names is dangerous if you are moving VGs between systems in case the two systems have identical VG names.
That's why you should use vgexport/vgimport which avoids problems with regard to identical volume group names.
Signature · 12 lines
Regards,
Heinz -- The LVM guy --
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Heinz Mauelshagen Sistina Software Inc.
Senior Consultant/Developer Bartningstr. 12
64289 Darmstadt
Germany
[email protected] +49 6151 7103 86
FAX 7103 96
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-Quoted 10 lines · Heinz J. Mauelshagen
Quoted 5 lines
From what I have read in the past on this mailing list, running vgscan on boot is likely to delete important info from
/etc/lvmtabif there is a minor problem, that has to later be restored (if possible).Assumed that it is really that likely it s rather easy to get
/etc/lvmtab.d/files containing the metadata copies back, because they are format identical to/etc/lvmconf/files./etc/lvmtabcontains the 0 delimited volume group names.
I actually had this problem and it scared the willies out of me.
I was quite surprised to discover I had any left, given my antipathy towards Microsft and William Gates III, but I digress.
I didn't know this about /etc/lvmconf and /etc/lvmtab.d. I had
to very carefully restore them using vcfgrestore.
Quoted 4 lines
If we had a database keeping track of changes in the i/o configuration of the system, we could run vgscan conditionally if needed. Because there's none I recommended to run it at every boot to avoid i/o path inconsistencies.
You kinda do. It's called /etc/lvmtab.d.
Quoted 3 lines
/etc/lvmtab.d/VolumeGroupNamecontains a metadata copy of the current state of a volume group. In case of i/o configuration changes it is up to vgscan to update this file's contents.
Why not have vgscan proceed on the initial opinion that the
files contents are correct, and scan the devices mentioned in the file to see if they still contain exactly the VG data the file says they ought to?
This would save a LOT of vgscan time because the most common
case is that everything is the way it was before the machine went down. If it wasn't, vgscan could revert to its normal behavior.
It wouldn't help the problem I had though, because apparently
one of the PVs somehow had corrupt VG information, though I have no clue how it happened. I should've saved the headers, but I was in too much of a panic, and too relieved when I got everything working again.
Have fun (if at all possible),
Signature · 3 lines
The best we can hope for concerning the people at large is that they be properly armed. -- Alexander Hamilton -- Eric Hopper ([email protected] http://www.omnifarious.org/~hopper) --
Quoted 4 lines · Heinz
You are right that there is no need to run vgscan on each boot. Nevertheless vgscan takes care of any changes in the i/o configuration of the system in order to make physical volumes accessable using different device nodes.
If we did something like a pvscan on each boot, which identified the PVID->device mappings, we would not need to change the VG configuration information each time. In the future, devfs will automatically do PVID->device mappings dynamically for us, from what Richard said.
Quoted 3 lines
Assumed that it is really that likely it's rather easy to get
/etc/lvmtab.d/files containing the metadata copies back, because they are format identical to/etc/lvmconf/files.
But people have problems nonetheless. It may not be because of vgscan, or because it's hard to do restores, but that's an extra step that people have to make when trying to fix a broken system.
Quoted 1 line
/etc/lvmtabcontains the 0 delimited volume group names.
Yes, I was just referring to the whole LVM configuration information.
Quoted 3 lines
/etc/lvmtab.d/VolumeGroupNamecontains a metadata copy of the current state of a volume group. In case of i/o configuration changes it is up to vgscan to update this file's contents.
Looking on my development system, /etc/lvmtab.d/vgtest is not quite the same data as /etc/lvmconf/vgtest.conf. Is it possible that the backups in /etc/lvmconf are 1 version behind what's on disk?
For example, the /etc/lvmconf/vgtest.conf file says V_EXP, while /etc/lvmtab.d/vgtest does not have this (I was testing vgexport and vgimport last). The issue is that this is fine for the case where you made a mistake changing a VG, but if the disk is corrupted, and you run vgscan it will overwrite the last good copy of the current config (/etc/lvmtab.d/vgtest). Any action you do to restore the VGDA will only return the previous configuration, not the current one.
I think that once we have on-disk backups of the VGDA, and we only update one at a time, we will be OK. However, maybe for now we should also backup the _current_ configuration to /etc/lvmconf/vgtest.conf, and have the _previous_ configuration in /etc/lvmconf/vgtest.conf.1, etc.
Cheers, Andreas
Signature · 3 lines
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert