metadata too large for circular buffer - any way to increase PV metadatasize?

Hi,

I'm getting the dreaded "metadata too large for circular buffer" message trying to create a new LV in a volume group of a few hundred other LVs.

As I understand, this is because the PVs don't have enough space for the metadata. This size can be set to a larger value when pvcreating with --metadatasize but I have obviously missed that boat :)

Is there a way to increase this with live PVs, rather than shifting all the data off them and recreating and shifting back on?

I have a multinode RedHat Cluster with CLVMD and two PVs. I can add additional PVs if that helps (though as I understand *every* PV needs enough metadata space).

Thanks in advance!

John.

http://johnleach.co.uk

Unfortunately there is no way to increase this retrospectively. Nor have we written tools to manipulate metadata areas yet.

You might be able to take steps to make the metadata smaller if you have long names you can shorten or fragmented LVs that can be defragmented with a sequence of pvmoves.

You're right that the best option is to add more PVs.

Roughly:

Create a new PV with a much larger metadata area, add it to the VG, then stop using the too-small metadata areas on the old PVs.

For extra resiliance it might be wise to add two PVs so you have two copies of the metadata, or use the pvcreate option to have 2 metadata copies on the single PV you're adding, one at the front and ane at the end.

To remove the metadata areas you need to:

get an up-to-date metadata backup (vgcfgbackup)

pvcreate --restorefile pointing at a copy of that backup file requesting 0 metadata areas
and specifying the same uuid as it had before

vgcfgrestore from the backup file

Alasdair

[email protected]

And if you have problems making space in the metadata to add the PV to it, a workaround is to temporarily use in-filesystem metadata:

create a metadata dir in a local filesystem (NOT an LV), and place an up-to-date backup copy of the metadata there (from vgcfgbackup)

update lvm.conf to start using this metadata (metadata/dirs)
- use verbose display options to check it's doing this

pvcreate as in my last message to remove the too-small metadata area
- you're now relying entirely on the in-filesystem metadata

add the new PVs to the VG as before

revert the lvm.conf change

Alasdair

[email protected]

I should also point out that although using in-filesystem metadata is a fully-supported configuration, it does not receive much testing because few people use it, so if you do need to use that method and hit any problems, certainly let us know.

Alasdair

[email protected]

Will this work with clustered LVM? If so, I take it all nodes will have to be configured to use in-filesystem metadata?

Thanks,

John.

http://johnleach.co.uk

Hi Alasdair,

thanks for the advice - very much appreciated. I'm going to test this out on a test cluster. It will buy some time whilst we consider what to do next (probably pvmove things around and recreate PVs with more metadata space).

Thanks again,

John.

http://johnleach.co.uk

This doesn't look like something you can do with the volume group active (with cluster lvm anyway):

[root@testnode0 ~]# pvcreate --restorefile san-metadata -u fTLglk-j1C1-02Z7-8k6l-DTAm-2WNj-9ZGT19 --metadatacopies 0 /dev/hdb -ff Really INITIALIZE physical volume "/dev/hdb" of volume group "san" [y/n]? y
Can't open /dev/hdb exclusively. Mounted filesystem?

Is that right, or am I missing something?

Thanks,

John.