Two questions about cluster support (2 problems, actually)
Hi all,
Last year, when I last setup an LVM for cluster mode (running on a
DRBD partition in Primary/Primary mode), I needed to recompile the RPM for CentOS 5.4 in order to get the liblvm2clusterlock.so to install. This was in reference to an initial error report I saw here: http://www.centos.org/modules/newbb/viewtopic.php?forum=37&post_id=28736&topic_id=8449 in 2007.
Is this still a problem? I don't see that file still (fully updated
CentOS 5.4 x86_64). Without that file, I can't try 'locking_type = 2'.
Further, when I try to use 'locking_type = 3', restart 'cman' and
then try to run 'pvdisplay', I *only* see the DRBD LVM PV. The underlying PV used by Xen's dom0 no longer shows:
# df -hP Filesystem Size Used Avail Use% Mounted on /dev/mapper/san02-lv01 19G 3.0G 15G 17% / /dev/sda1 244M 35M 196M 16% /boot tmpfs 845M 0 845M 0% /dev/shm none 845M 104K 845M 1% /var/lib/xenstored
# pvdisplay connect() failed on local socket: Connection refused WARNING: Falling back to local file-based locking. Volume Groups with the clustered attribute will be inaccessible. "/dev/drbd0" is a new physical volume of "399.99 GB" --- NEW Physical volume --- PV Name /dev/drbd0 VG Name PV Size 399.99 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID kklTST-D2ta-NRxL-Zaa2-Fa0T-ebkB-KhSq1b
Resetting 'locking_type=1' and even after a reboot, I still can't see
the original LVM PV.
Signature · 4 lines
Digimer E-Mail: [email protected] AN!Whitepapers: http://alteeve.com Node Assassin: http://nodeassassin.org
Quoted 3 lines · Digimer
Last year, when I last setup an LVM for cluster mode (running on a
DRBD partition in Primary/Primary mode), I needed to recompile the RPM for CentOS 5.4 in order to get the liblvm2clusterlock.so to install.
Don't do this. Use "locking_type = 3", or better, use
lvmconf --enable-cluster /--disable-cluster to set proper cluster locking type.
(locking_type = 2 was external locking which need that *.so,
3 is internal cluster locking - basically the same, just compiled in base package. CentOS 5 should always use 3 for cluster locking.)
(but configuring DRBD is separate thing to clustered lvm)
Quoted 5 lines
# pvdisplay connect() failed on local socket: Connection refused WARNING: Falling back to local file-based locking. Volume Groups with the clustered attribute will be inaccessible. "/dev/drbd0" is a new physical volume of "399.99 GB"
apparently clvmd is not running, you cannot see clustered PVs.
Anyway, the DRBD need to setup filters to work properly with lvm2, (filter out underlying drbd device)
(or update to new lvm2 version, RHEL5/CentOS5 build will be available in next minor update).
For more info see https://bugzilla.redhat.com/show_bug.cgi?id=530881
(not sure it is related, but for archive and other users...)
Milan
--
[email protected]
Quoted 8 lines · Milan Broz
Quoted 5 lines
# pvdisplay connect() failed on local socket: Connection refused WARNING: Falling back to local file-based locking. Volume Groups with the clustered attribute will be inaccessible. "/dev/drbd0" is a new physical volume of "399.99 GB"apparently clvmd is not running, you cannot see clustered PVs.
Quoted 8 lines
Anyway, the DRBD need to setup filters to work properly with lvm2, (filter out underlying drbd device)
(or update to new lvm2 version, RHEL5/CentOS5 build will be available in next minor update).
For more info see https://bugzilla.redhat.com/show_bug.cgi?id=530881
(not sure it is related, but for archive and other users...)Milan
Thanks for the reply, Milan!
I had made the change to the 'filter' argument, but I was using 'filter = [ "a|drbd.*|", "r|.*|" ]'. I switched to 'filter = [ "a|^/dev/drbd|", "a/sd*/", "r/.*/" ]' and started 'clvmd'. This seems to have solved the problem, so thank you.
If I may ask another question though;
My DRBD device is '/dev/drbd0' which sits on either node's '/dev/san0x/lv02'. When I ran 'pvcreate /dev/drbd0' the PV was created successfully and 'pvdisplay' shows the new PV on both nodes. However, I see this:
# pvdisplay --- Physical volume --- PV Name /dev/sda2 VG Name san02 PV Size 465.51 GB / not usable 14.52 MB Allocatable yes PE Size (KByte) 32768 Total PE 14896 Free PE 1407 Allocated PE 13489 PV UUID ru7tmz-Qd4d-aatg-4ReY-TPPq-gMll-Z5ZOkL
"/dev/san02/lv02" is a new physical volume of "399.99 GB" --- NEW Physical volume ---
PV Name /dev/san02/lv02
VG Name
PV Size 399.99 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID MjwXhY-T3dw-IUkI-ZdSI-FKEB-KJdq-nl9GUo
Note that the 'PV Name' is '/dev/san02/lv02' but the device passed was 'pvcreate /dev/drbd0'. Is this correct? I'd expect it to show the DRBD device, not the DRBD's underlying logical volume.
Signature · 4 lines
Digimer E-Mail: [email protected] AN!Whitepapers: http://alteeve.com Node Assassin: http://nodeassassin.org
[linux-lvm] Fixed! (was: Re: Two questions about cluster support (2 problems, actually))
Quoted 38 lines · Digimer
Thanks for the reply, Milan!
I had made the change to the 'filter' argument, but I was using 'filter = [ "a|drbd.*|", "r|.*|" ]'. I switched to 'filter = [ "a|^/dev/drbd|", "a/sd*/", "r/.*/" ]' and started 'clvmd'. This seems to have solved the problem, so thank you.
If I may ask another question though;
My DRBD device is '
/dev/drbd0' which sits on either node's '/dev/san0x/lv02'. When I ran 'pvcreate/dev/drbd0' the PV was created successfully and 'pvdisplay' shows the new PV on both nodes. However, I see this:# pvdisplay --- Physical volume --- PV Name /dev/sda2 VG Name san02 PV Size 465.51 GB / not usable 14.52 MB Allocatable yes PE Size (KByte) 32768 Total PE 14896 Free PE 1407 Allocated PE 13489 PV UUID ru7tmz-Qd4d-aatg-4ReY-TPPq-gMll-Z5ZOkL"
/dev/san02/lv02" is a new physical volume of "399.99 GB" --- NEW Physical volume ---
PV Name/dev/san02/lv02
VG Name
PV Size 399.99 GB
Allocatable NO
PE Size (KByte) 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID MjwXhY-T3dw-IUkI-ZdSI-FKEB-KJdq-nl9GUo
Quoted 4 lines
Note that the 'PV Name' is '
/dev/san02/lv02' but the device passed was 'pvcreate/dev/drbd0'. Is this correct? I'd expect it to show the DRBD device, not the DRBD's underlying logical volume.
I switched back to my old syntax and added an entry for 'sdX' devices, restarted clvmd on both nodes and every seems to be fine now! Here is my latest, working, filter line:
filter = [ "a|drbd.*|", "a|sd.*|", "r|.*|" ]
Thanks for the help! I'll be looking forward to testing out the fix in CentOS 5.5. :)
Signature · 4 lines
Digimer E-Mail: [email protected] AN!Whitepapers: http://alteeve.com Node Assassin: http://nodeassassin.org