Skip to contentSkip to conversation

Two questions about cluster support (2 problems, actually)

linux-lvm Digimer 2 participants 4 messages Mar 16, 2010 – Mar 17, 2010

lore
Share & tools
lore

Digimer

author
#0 lore raw

Hi all,

Last year, when I last setup an LVM for cluster mode (running on a
DRBD partition in Primary/Primary mode), I needed to recompile the RPM for CentOS 5.4 in order to get the liblvm2clusterlock.so to install. This was in reference to an initial error report I saw here: http://www.centos.org/modules/newbb/viewtopic.php?forum=37&post_id=28736&topic_id=8449 in 2007.

Is this still a problem? I don't see that file still (fully updated
CentOS 5.4 x86_64). Without that file, I can't try 'locking_type = 2'.

Further, when I try to use 'locking_type = 3', restart 'cman' and
then try to run 'pvdisplay', I *only* see the DRBD LVM PV. The underlying PV used by Xen's dom0 no longer shows:

# df -hP
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/san02-lv01   19G  3.0G   15G  17% /
/dev/sda1             244M   35M  196M  16% /boot
tmpfs                 845M     0  845M   0% /dev/shm
none                  845M  104K  845M   1% /var/lib/xenstored
# pvdisplay
  connect() failed on local socket: Connection refused
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  "/dev/drbd0" is a new physical volume of "399.99 GB"
  --- NEW Physical volume ---
  PV Name               /dev/drbd0
  VG Name
  PV Size               399.99 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               kklTST-D2ta-NRxL-Zaa2-Fa0T-ebkB-KhSq1b

Resetting 'locking_type=1' and even after a reboot, I still can't see
the original LVM PV.

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org
#1 lore raw
Quoted 3 lines · Digimer

Last year, when I last setup an LVM for cluster mode (running on a
DRBD partition in Primary/Primary mode), I needed to recompile the RPM for CentOS 5.4 in order to get the liblvm2clusterlock.so to install.

Don't do this. Use "locking_type = 3", or better, use
lvmconf --enable-cluster /--disable-cluster to set proper cluster locking type.

(locking_type = 2 was external locking which need that *.so,
3 is internal cluster locking - basically the same, just compiled in base package. CentOS 5 should always use 3 for cluster locking.)

(but configuring DRBD is separate thing to clustered lvm)

Quoted 5 lines
# pvdisplay
   connect() failed on local socket: Connection refused
   WARNING: Falling back to local file-based locking.
   Volume Groups with the clustered attribute will be inaccessible.
   "/dev/drbd0" is a new physical volume of "399.99 GB"

apparently clvmd is not running, you cannot see clustered PVs.

Anyway, the DRBD need to setup filters to work properly with lvm2, (filter out underlying drbd device)
(or update to new lvm2 version, RHEL5/CentOS5 build will be available in next minor update).
For more info see https://bugzilla.redhat.com/show_bug.cgi?id=530881
(not sure it is related, but for archive and other users...)

Milan
--
[email protected]

author
#2 lore raw
Quoted 8 lines · Milan Broz
Quoted 5 lines
# pvdisplay
    connect() failed on local socket: Connection refused
    WARNING: Falling back to local file-based locking.
    Volume Groups with the clustered attribute will be inaccessible.
    "/dev/drbd0" is a new physical volume of "399.99 GB"

apparently clvmd is not running, you cannot see clustered PVs.

Quoted 8 lines

Anyway, the DRBD need to setup filters to work properly with lvm2, (filter out underlying drbd device)
(or update to new lvm2 version, RHEL5/CentOS5 build will be available in next minor update).
For more info see https://bugzilla.redhat.com/show_bug.cgi?id=530881
(not sure it is related, but for archive and other users...)

Milan

Thanks for the reply, Milan!

I had made the change to the 'filter' argument, but I was using 'filter = [ "a|drbd.*|", "r|.*|" ]'. I switched to 'filter = [ "a|^/dev/drbd|", "a/sd*/", "r/.*/" ]' and started 'clvmd'. This seems to have solved the problem, so thank you.

If I may ask another question though;

My DRBD device is '/dev/drbd0' which sits on either node's '/dev/san0x/lv02'. When I ran 'pvcreate /dev/drbd0' the PV was created successfully and 'pvdisplay' shows the new PV on both nodes. However, I see this:

# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               san02
  PV Size               465.51 GB / not usable 14.52 MB
  Allocatable           yes
  PE Size (KByte)       32768
  Total PE              14896
  Free PE               1407
  Allocated PE          13489
  PV UUID               ru7tmz-Qd4d-aatg-4ReY-TPPq-gMll-Z5ZOkL

"/dev/san02/lv02" is a new physical volume of "399.99 GB" --- NEW Physical volume ---
PV Name /dev/san02/lv02
VG Name

  PV Size               399.99 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               MjwXhY-T3dw-IUkI-ZdSI-FKEB-KJdq-nl9GUo

Note that the 'PV Name' is '/dev/san02/lv02' but the device passed was 'pvcreate /dev/drbd0'. Is this correct? I'd expect it to show the DRBD device, not the DRBD's underlying logical volume.

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org
author
#3 lore raw

[linux-lvm] Fixed! (was: Re: Two questions about cluster support (2 problems, actually))

Quoted 38 lines · Digimer

Thanks for the reply, Milan!

I had made the change to the 'filter' argument, but I was using 'filter = [ "a|drbd.*|", "r|.*|" ]'. I switched to 'filter = [ "a|^/dev/drbd|", "a/sd*/", "r/.*/" ]' and started 'clvmd'. This seems to have solved the problem, so thank you.

If I may ask another question though;

My DRBD device is '/dev/drbd0' which sits on either node's '/dev/san0x/lv02'. When I ran 'pvcreate /dev/drbd0' the PV was created successfully and 'pvdisplay' shows the new PV on both nodes. However, I see this:

# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name san02
PV Size 465.51 GB / not usable 14.52 MB
Allocatable yes
PE Size (KByte) 32768
Total PE 14896
Free PE 1407
Allocated PE 13489
PV UUID ru7tmz-Qd4d-aatg-4ReY-TPPq-gMll-Z5ZOkL

"/dev/san02/lv02" is a new physical volume of "399.99 GB" --- NEW Physical volume ---
PV Name /dev/san02/lv02
VG Name
PV Size 399.99 GB
Allocatable NO
PE Size (KByte) 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID MjwXhY-T3dw-IUkI-ZdSI-FKEB-KJdq-nl9GUo

Quoted 4 lines

Note that the 'PV Name' is '/dev/san02/lv02' but the device passed was 'pvcreate /dev/drbd0'. Is this correct? I'd expect it to show the DRBD device, not the DRBD's underlying logical volume.

I switched back to my old syntax and added an entry for 'sdX' devices, restarted clvmd on both nodes and every seems to be fine now! Here is my latest, working, filter line:

filter = [ "a|drbd.*|", "a|sd.*|", "r|.*|" ]

Thanks for the help! I'll be looking forward to testing out the fix in CentOS 5.5. :)

Signature · 4 lines
Digimer
E-Mail:         [email protected]
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org