Add infrastructure for metadata balancing.
cover Add infrastructure for metadata balancing.
This patchset is a third round of the metadata balancing support, and builds on the second round of patches already submitted. It adds the infrastructure to set a bit inside the mda header on a per-PV basis, control it from pvchange, display the number of mdas with this bit set on a particular PV or VG, and adds VG 'min' / 'max' parameters to control the per-PV bits.
Changes from the second round include:
1) Define the vg-level parameters which control metadata balancing:
a) vg_mda_balance_max: a number indicating the maximum number of PVs to contain enabled metadata areas; range is vg_mda_balance_min:# of PVs in VG
b) vg_mda_balance_min: a number indicating the minimum number of PVs to contain enabled metadata areas; range 1:vg_mda_balance_max
2) Add metadata_balance_{min|max} to struct volume_group, and store these parameters on disk in vg metadata.
3) Update vgchange to take metadatabalance{min|max} parameters.
4) Various small bugs fixed in the second round of patches.
Still TODO:
1) Implement the disable of mdas properly by disabling certain metadata_area_ops functions. I began this work, then ran into some lvmcache issues that need to be resolved. Code is isolated to one patch (not included in this set).
2) Implement the setting of the disable bit on various PVs when the metadatabalance{min|max} parameters are given to vgchange. Should be fairly straightforward - setting a lower 'max' value might cause metadata disable bits set on PVs, and setting a higher 'min' value may cause metadata disable bits cleared on PVs.
3) Consider renaming the 'disable' to 'enable' as this is more consistent with the 'min' and 'max' parameters
PATCH 1/21 Change 'filler' to 'flags' in on-disk 'raw_locn' structure.
Future patches will make use of a specific flag in the on-disk 'raw_locn' structure to enable/disable metadata areas, and facilitate metadata balancing.
Note that 'filler' is always set to '0' (see add_mda() - memset), so use of this area as a non-zero flags field is a safe way to provide future code features.
--- lib/format_text/layout.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
lib/format_text/layout.h +1 -1
@@ -51,7 +51,7 @@ struct raw_locn {uint64_t offset;/* Offset in bytes to start sector */ uint64_t size;/* Bytes */ uint32_t checksum; -uint32_t filler; +uint32_t flags; } __attribute__ ((packed));/* On disk */
Signature · 1 line
1.6.0.6
PATCH 2/21 Add text format specific 'rlocn' disable flag and access functions.
Adding a flag to the 'rlocn' structure in the mda header of the text format allows us to flip a bit to disable an mda via the text format specific mda_header and mda_context structures. This patch defines the flag and access functions to manage the flag. Other patches will manage the disable on a format-independent basis.
--- include/.symlinks.in | 1 + lib/format_text/format-text.c | 13 +++++++++++++ lib/format_text/format-text.h | 1 + lib/format_text/layout.h | 10 ++++++++++ 4 files changed, 25 insertions(+), 0 deletions(-)
include/.symlinks.in +1 -0
@@ -26,6 +26,7 @@ @top_srcdir@/lib/format_text/format-text.h @top_srcdir@/lib/format_text/text_export.h @top_srcdir@/lib/format_text/text_import.h + at top_srcdir@/lib/format_text/layout.h @top_srcdir@/lib/label/label.h @top_srcdir@/lib/locking/locking.h @top_srcdir@/lib/log/log.h
lib/format_text/format-text.c +13 -0
@@ -66,6 +66,19 @@ struct text_context {char *desc;/* Description placed inside file */ }; +int rlocn_is_disabled (const struct raw_locn *rlocn) +{ +return (rlocn->flags &RAW_LOCN_FLAG_DISABLE ?1 :0 ); +} + +void rlocn_set_disabled (struct raw_locn *rlocn,int value) +{ +if (value) + rlocn->flags |=RAW_LOCN_FLAG_DISABLE ; +else + rlocn->flags &= ~RAW_LOCN_FLAG_DISABLE ; +} +/* * NOTE: Currently there can be only one vg per text file. */
lib/format_text/format-text.h +1 -0
@@ -17,6 +17,7 @@#define _LVM_FORMAT_TEXT_H#include "lvm-types.h" +#include "layout.h" #include "metadata.h" #define FMT_TEXT_NAME "lvm2"
lib/format_text/layout.h +10 -0
@@ -46,6 +46,13 @@ struct pv_header {struct disk_locn disk_areas_xl[0 ];/* Two lists */ } __attribute__ ((packed)); +/* +* Disable this raw location. This allows us to enable/disable +* metadata easily, and thus balance metadata across VGs with +* many PVs. +*/ +#define RAW_LOCN_FLAG_DISABLE 0x00000001 +/* On disk */ struct raw_locn {uint64_t offset;/* Offset in bytes to start sector */
@@ -54,6 +61,9 @@ struct raw_locn {uint32_t flags; } __attribute__ ((packed)); +int rlocn_is_disabled (const struct raw_locn *rlocn); +void rlocn_set_disabled (struct raw_locn *rlocn,int value); +/* On disk */ /* Structure size limited to one sector */ struct mda_header {
Signature · 1 line
1.6.0.6
PATCH 3/21 Add format independent flag and functions to disable mdas.
First we add a 'flags' field to the format independent metadata_area structure, and a MDA_DISABLE flag. The mda_is_disabled and mda_set_disabled functions are added to manage the MDA_DISABLE flag. Adding the flag and functions gives a library interface to disable / enable metadata areas independent of the underlying format. The format-specific read/write functions must then handle any reading or writing of this flag to disk.
--- lib/format_text/text_label.c | 1 + lib/metadata/metadata.c | 13 +++++++++++++ lib/metadata/metadata.h | 6 ++++++ 3 files changed, 20 insertions(+), 0 deletions(-)
lib/format_text/text_label.c +1 -0
@@ -215,6 +215,7 @@ int add_mda(const struct format_type *fmt, struct dm_pool *mem, struct dm_list * mdal->ops = mda_lists->raw_ops; mdal->metadata_locn = mdac; + mdal->flags =0 ; mdac->area.dev = dev; mdac->area.start = start;
lib/metadata/metadata.c +13 -0
@@ -3628,6 +3628,19 @@ uint32_t pv_pe_alloc_count(const struct physical_volume *pv)return pv_field (pv, pe_alloc_count); } +int mda_is_disabled (struct metadata_area *mda) +{ +return (mda->flags &MDA_DISABLED ); +} + +void mda_set_disabled (struct metadata_area *mda,int value) +{ +if (value) + mda->flags |=MDA_DISABLED ; +else + mda->flags &= ~MDA_DISABLED ; +} +uint32_t pv_mda_count (const struct physical_volume *pv) {struct lvmcache_info *info;
lib/metadata/metadata.h +6 -0
@@ -158,12 +158,18 @@ struct metadata_area_ops { }; +#define MDA_DISABLED 0x00000001 +struct metadata_area {struct dm_list list;struct metadata_area_ops *ops;void *metadata_locn; +uint32_t flags; }; +int mda_is_disabled (struct metadata_area *mda); +void mda_set_disabled (struct metadata_area *mda,int value); +#define seg_pvseg (seg, s) (seg)->areas[(s)].u.pv.pvseg#define seg_dev (seg, s) (seg)->areas[(s)].u.pv.pvseg->pv->dev#define seg_pe (seg, s) (seg)->areas[(s)].u.pv.pvseg->pe
Signature · 1 line
1.6.0.6
PATCH 4/21 Move dev_open / dev_close outside _vg_read_raw_area().
This refactoring moves the device open/close up one level to the caller of _vg_read_raw_area(). Should be no functional change and facilitate future changes related to metadata balancing.
--- lib/format_text/format-text.c | 40 ++++++++++++++++++++++++++++++---------- 1 files changed, 30 insertions(+), 10 deletions(-)
lib/format_text/format-text.c +30 -10
@@ -464,9 +464,6 @@ static struct volume_group *_vg_read_raw_area(struct format_instance *fid,char *desc;uint32_t wrap =0 ; -if (!dev_open (area->dev)) - return_NULL; -if (!(mdah =_raw_read_mda_header (fid->fmt, area))) goto_out;
@@ -501,9 +498,6 @@ static struct volume_group *_vg_read_raw_area(struct format_instance *fid, vg->status |=PRECOMMITTED ; out: -if (!dev_close (area->dev)) - stack; -return vg; }
@@ -512,8 +506,17 @@ static struct volume_group *_vg_read_raw(struct format_instance *fid, struct metadata_area *mda) {struct mda_context *mdac = (struct mda_context *) mda->metadata_locn; +struct volume_group *vg; -return _vg_read_raw_area (fid, vgname, &mdac->area,0 ); +if (!dev_open (mdac->area.dev)) + return_NULL; + + vg =_vg_read_raw_area (fid, vgname, &mdac->area,0 ); + +if (!dev_close (mdac->area.dev)) + stack; + +return vg; } static struct volume_group *_vg_read_precommit_raw(struct format_instance *fid,
@@ -521,8 +524,17 @@ static struct volume_group *_vg_read_precommit_raw(struct format_instance *fid, struct metadata_area *mda) {struct mda_context *mdac = (struct mda_context *) mda->metadata_locn; +struct volume_group *vg; + +if (!dev_open (mdac->area.dev)) + return_NULL; + + vg =_vg_read_raw_area (fid, vgname, &mdac->area,1 ); + +if (!dev_close (mdac->area.dev)) + stack; -return _vg_read_raw_area (fid, vgname, &mdac->area,1 ); +return vg; } static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg,
@@ -1170,9 +1182,17 @@ static int _scan_raw(const struct format_type *fmt)/* FIXME We're reading mdah twice here... */ if ((vgname =vgname_from_mda (fmt, &rl->dev_area, &vgid, &vgstatus,NULL ,NULL ))) { -if ((vg =_vg_read_raw_area (&fid, vgname, - &rl->dev_area,0 ))) +if (!dev_open (rl->dev_area.dev)) { + stack; +continue ; + } + + vg =_vg_read_raw_area (&fid, vgname, &rl->dev_area,0 ); +if (vg)lvmcache_update_vg (vg,0 ); + +if (!dev_close (rl->dev_area.dev)) + stack; } }
Signature · 1 line
1.6.0.6
PATCH 5/21 Move dev_open/dev_close outside vgname_from_mda().
Refactor vgname_from_mda() so caller must open/close the device. Should be no functional change.
--- lib/format_text/format-text.c | 20 +++++++------------- lib/format_text/text_label.c | 6 ++++++ 2 files changed, 13 insertions(+), 13 deletions(-)
lib/format_text/format-text.c +7 -13
@@ -1077,9 +1077,6 @@ const char *vgname_from_mda(const struct format_type *fmt,if (mda_free_sectors) *mda_free_sectors = ((dev_area->size -MDA_HEADER_SIZE ) /2 ) >>SECTOR_SHIFT ; -if (!dev_open (dev_area->dev)) - return_NULL; -if (!(mdah =_raw_read_mda_header (fmt, dev_area))) goto_out;
@@ -1157,9 +1154,6 @@ const char *vgname_from_mda(const struct format_type *fmt, } out: -if (!dev_close (dev_area->dev)) - stack; -return vgname; }
@@ -1180,20 +1174,20 @@ static int _scan_raw(const struct format_type *fmt) dm_list_iterate_items(rl, raw_list) {/* FIXME We're reading mdah twice here... */ +if (!dev_open (rl->dev_area.dev)) { + stack; +continue ; + } +if ((vgname =vgname_from_mda (fmt, &rl->dev_area, &vgid, &vgstatus,NULL ,NULL ))) { -if (!dev_open (rl->dev_area.dev)) { - stack; -continue ; - } - vg =_vg_read_raw_area (&fid, vgname, &rl->dev_area,0 );if (vg)lvmcache_update_vg (vg,0 ); -if (!dev_close (rl->dev_area.dev)) - stack; } +if (!dev_close (rl->dev_area.dev)) + stack; }return 1 ;
lib/format_text/text_label.c +6 -0
@@ -299,6 +299,10 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, dm_list_iterate_items(mda, &info->mdas) { mdac = (struct mda_context *) mda->metadata_locn; +if (!dev_open (mdac->area.dev)) { + stack; +continue ; + } if ((vgname = vgname_from_mda(info->fmt, &mdac->area, &vgid, &vgstatus, &creation_host, &mdac->free_sectors)) &&
@@ -306,6 +310,8 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, (char *) &vgid, vgstatus, creation_host)) return_0; +if (!dev_close (mdac->area.dev)) + stack; } info->status &= ~CACHE_INVALID ;
Signature · 1 line
1.6.0.6
PATCH 6/21 Allow raw_read_mda_header to be called from text_label.c.
We'd like to pass in mda_header to vgname_from_mda(). In order to do this, we need to call raw_read_mda_header() from text_label.c, _text_read(), which gets called from the label_read() path, and peers into the metadata and update vginfo cache. We should check the disable bit here, and if set, not peer into the vg metadata, thus reducing the I/O to disk.
--- lib/format_text/format-text.c | 33 +++++++++++++++++++-------------- lib/format_text/format-text.h | 1 + lib/format_text/layout.h | 4 ++++ lib/format_text/text_label.c | 15 +++++++++++++-- 4 files changed, 37 insertions(+), 16 deletions(-)
lib/format_text/format-text.c +19 -14
@@ -37,9 +37,6 @@#include <dirent.h> #include <ctype.h> -static struct mda_header *_raw_read_mda_header (const struct format_type *fmt, -struct device_area *dev_area); - static struct format_instance *_text_create_text_instance(const struct format_type *fmt, const char *vgname, const char *vgid,
@@ -181,7 +178,7 @@ static int _pv_analyze_mda_raw (const struct format_type * fmt,if (!dev_open (area->dev)) return_0; -if (!(mdah =_raw_read_mda_header (fmt, area))) +if (!(mdah =raw_read_mda_header (fmt, area))) goto_out; rlocn = mdah->raw_locns;
@@ -308,8 +305,8 @@ static void _xlate_mdah(struct mda_header *mdah) } } -static struct mda_header *_raw_read_mda_header (const struct format_type *fmt, -struct device_area *dev_area) +struct mda_header *raw_read_mda_header (const struct format_type *fmt, +struct device_area *dev_area) {struct mda_header *mdah;
@@ -440,7 +437,7 @@ static int _raw_holds_vgname(struct format_instance *fid,if (!dev_open (dev_area->dev)) return_0; -if (!(mdah =_raw_read_mda_header (fid->fmt, dev_area))) +if (!(mdah =raw_read_mda_header (fid->fmt, dev_area))) return_0;if (_find_vg_rlocn (dev_area, mdah, vgname, &noprecommit))
@@ -464,7 +461,7 @@ static struct volume_group *_vg_read_raw_area(struct format_instance *fid,char *desc;uint32_t wrap =0 ; -if (!(mdah =_raw_read_mda_header (fid->fmt, area))) +if (!(mdah =raw_read_mda_header (fid->fmt, area))) goto_out;if (!(rlocn =_find_vg_rlocn (area, mdah, vgname, &precommitted))) {
@@ -564,7 +561,7 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg,if (!dev_open (mdac->area.dev)) return_0; -if (!(mdah =_raw_read_mda_header (fid->fmt, &mdac->area))) +if (!(mdah =raw_read_mda_header (fid->fmt, &mdac->area))) goto_out; rlocn =_find_vg_rlocn (&mdac->area, mdah, vg->name, &noprecommit);
@@ -669,7 +666,7 @@ static int _vg_commit_raw_rlocn(struct format_instance *fid,if (!found)return 1 ; -if (!(mdah =_raw_read_mda_header (fid->fmt, &mdac->area))) +if (!(mdah =raw_read_mda_header (fid->fmt, &mdac->area))) goto_out;if (!(rlocn =_find_vg_rlocn (&mdac->area, mdah, vg->name, &noprecommit))) {
@@ -778,7 +775,7 @@ static int _vg_remove_raw(struct format_instance *fid, struct volume_group *vg,if (!dev_open (mdac->area.dev)) return_0; -if (!(mdah =_raw_read_mda_header (fid->fmt, &mdac->area))) +if (!(mdah =raw_read_mda_header (fid->fmt, &mdac->area))) goto_out;if (!(rlocn =_find_vg_rlocn (&mdac->area, mdah, vg->name, &noprecommit))) {
@@ -1061,12 +1058,12 @@ static int _scan_file(const struct format_type *fmt) }const char *vgname_from_mda (const struct format_type *fmt, +struct mda_header *mdah,struct device_area *dev_area,struct id *vgid,uint64_t *vgstatus,char **creation_host,uint64_t *mda_free_sectors) {struct raw_locn *rlocn; -struct mda_header *mdah;uint32_t wrap =0 ;const char *vgname =NULL ;unsigned int len =0 ;
@@ -1077,7 +1074,7 @@ const char *vgname_from_mda(const struct format_type *fmt,if (mda_free_sectors) *mda_free_sectors = ((dev_area->size -MDA_HEADER_SIZE ) /2 ) >>SECTOR_SHIFT ; -if (!(mdah =_raw_read_mda_header (fmt, dev_area))) +if (!mdah) goto_out;/* FIXME Cope with returning a list */
@@ -1166,6 +1163,7 @@ static int _scan_raw(const struct format_type *fmt)struct format_instance fid;struct id vgid;uint64_t vgstatus; +struct mda_header *mdah; raw_list = &((struct mda_lists *) fmt->private)->raws;
@@ -1179,13 +1177,20 @@ static int _scan_raw(const struct format_type *fmt)continue ; } -if ((vgname =vgname_from_mda (fmt, &rl->dev_area, &vgid, &vgstatus, +if (!(mdah =raw_read_mda_header (fmt, &rl->dev_area))) { + stack; + goto close_dev; + } + +if ((vgname =vgname_from_mda (fmt, mdah, + &rl->dev_area, &vgid, &vgstatus,NULL ,NULL ))) { vg =_vg_read_raw_area (&fid, vgname, &rl->dev_area,0 );if (vg)lvmcache_update_vg (vg,0 ); } + close_dev:if (!dev_close (rl->dev_area.dev)) stack; }
lib/format_text/format-text.h +1 -0
@@ -61,6 +61,7 @@ int add_mda(const struct format_type *fmt, struct dm_pool *mem, struct dm_list *void del_mdas (struct dm_list *mdas);const char *vgname_from_mda (const struct format_type *fmt, +struct mda_header *mdah,struct device_area *dev_area,struct id *vgid,uint64_t *vgstatus,char **creation_host,uint64_t *mda_free_sectors);
lib/format_text/layout.h +4 -0
@@ -76,6 +76,9 @@ struct mda_header {struct raw_locn raw_locns[0 ];/* NULL-terminated list */ } __attribute__ ((packed)); +struct mda_header *raw_read_mda_header (const struct format_type *fmt, +struct device_area *dev_area); +struct mda_lists {struct dm_list dirs;struct dm_list raws;
@@ -86,6 +89,7 @@ struct mda_lists {struct mda_context {struct device_area area;uint64_t free_sectors; +struct mda_header *mdah;/* copy of the on-disk mdah */ struct raw_locn rlocn;/* Store inbetween write and commit */ };
lib/format_text/text_label.c +13 -2
@@ -262,6 +262,7 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf,const char *vgname;uint64_t vgstatus;char *creation_host; +struct mda_header *mdah; pvhdr = (struct pv_header *) ((void *) buf +xlate32 (lh->offset_xl));
@@ -303,13 +304,23 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, stack;continue ; } -if ((vgname =vgname_from_mda (info->fmt, &mdac->area, +if (!(mdah =raw_read_mda_header (info->fmt, &mdac->area))) { + stack; + goto close_dev; + } + +if ((vgname =vgname_from_mda (info->fmt, mdah, + &mdac->area, &vgid, &vgstatus, &creation_host, &mdac->free_sectors)) && !lvmcache_update_vgname_and_id (info, vgname, (char *) &vgid, vgstatus, - creation_host)) + creation_host)) { +if (!dev_close (mdac->area.dev)) + stack; return_0; + } + close_dev:if (!dev_close (mdac->area.dev)) stack; }
Signature · 1 line
1.6.0.6
PATCH 7/21 Ensure in-memory state matches on-disk state of mda disable bit.
--- lib/format_text/format-text.c | 3 +++ lib/format_text/text_label.c | 2 ++ 2 files changed, 5 insertions(+), 0 deletions(-)
lib/format_text/format-text.c +3 -0
@@ -704,6 +704,7 @@ static int _vg_commit_raw_rlocn(struct format_instance *fid,"header at %" PRIu64, vg->name, dev_name(mdac->area.dev), mdac->area.start); +rlocn_set_disabled (mdah->raw_locns,mda_is_disabled (mda));if (!_raw_write_mda_header (fid->fmt, mdac->area.dev, mdac->area.start, mdah)) {dm_pool_free (fid->fmt->cmd->mem, mdah);
@@ -786,6 +787,7 @@ static int _vg_remove_raw(struct format_instance *fid, struct volume_group *vg, rlocn->offset =0 ; rlocn->size =0 ; rlocn->checksum =0 ; +rlocn_set_disabled (mdah->raw_locns,mda_is_disabled (mda));if (!_raw_write_mda_header (fid->fmt, mdac->area.dev, mdac->area.start, mdah)) {
@@ -1489,6 +1491,7 @@ static int _text_pv_write(const struct format_type *fmt, struct physical_volume mdac = mda->metadata_locn;memset (&buf,0 ,sizeof (buf)); mdah->size = mdac->area.size; +rlocn_set_disabled (mdah->raw_locns,mda_is_disabled (mda));if (!_raw_write_mda_header (fmt, mdac->area.dev, mdac->area.start, mdah)) {if (!dev_close (pv->dev))
lib/format_text/text_label.c +2 -0
@@ -301,6 +301,7 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, dm_list_iterate_items(mda, &info->mdas) { mdac = (struct mda_context *) mda->metadata_locn;if (!dev_open (mdac->area.dev)) { +mda_set_disabled (mda,1 ); stack;continue ; }
@@ -308,6 +309,7 @@ static int _text_read(struct labeller *l, struct device *dev, void *buf, stack; goto close_dev; } +mda_set_disabled (mda,rlocn_is_disabled (mdah->raw_locns)); if ((vgname = vgname_from_mda(info->fmt, mdah, &mdac->area,
Signature · 1 line
1.6.0.6
PATCH 8/21 Add vg field to struct physical_volume and find_vg_by_pv().
Until now, the physical_volume struct contained only the vgname and vgid. However, this provides some difficulty when designing a proper API around pv handles and their properties. In particular, this patch is required if we want to enable/disable mdas for a particular PV with a nice PV-based interface.
For tools and operations such as pvchange, a clean API might look something like this:
pv = pv_open(pvname, mode);
value = pv_get_property(pv, propertyname);pv_set_property(pv, propertyname, newvalue);
The problem is today we have no way of really "opening" a PV for reading or writing, other than starting with a VG. This seems unduly restrictive to an application, since at times they may be starting with a device (pv) name and they may not know the vgname, nor do they really care if they are changing a device/pv property.
The possible pv_open() API above would implicitly call vg_read(), which would obtain the proper vg lock, and then return the pv handle. But then we'd have the problem of what happens to the vg handle obtained in the vg_read() call - do we leak it or store it somewhere? Thus, we introduce this small but important addition to struct physical_volume. Future patches will make use of this API to rework related areas of the code as well as enhance liblvm.
--- lib/format_text/import_vsn1.c | 1 + lib/metadata/metadata-exported.h | 2 ++ lib/metadata/metadata.c | 5 +++++ 3 files changed, 8 insertions(+), 0 deletions(-)
lib/format_text/import_vsn1.c +1 -0
@@ -268,6 +268,7 @@ static int _read_pv(struct format_instance *fid, struct dm_pool *mem, vg->pv_count++;dm_list_add (&vg->pvs, &pvl->list); + pv->vg = vg;return 1 ; }
lib/metadata/metadata-exported.h +2 -0
@@ -182,6 +182,7 @@ struct physical_volume {const struct format_type *fmt;const char *vg_name;struct id vgid; +struct volume_group *vg;uint64_t status;uint64_t size;
@@ -594,6 +595,7 @@ struct pv_list *find_pv_in_vg(const struct volume_group *vg, const char *pv_name);struct physical_volume *find_pv_in_vg_by_uuid (const struct volume_group *vg,const struct id *id); +struct volume_group *find_vg_by_pv (const struct physical_volume *pv);/* Find an LV within a given VG */ struct lv_list *find_lv_in_vg(const struct volume_group *vg,
lib/metadata/metadata.c +5 -0
@@ -1573,6 +1573,11 @@ static struct pv_list *_find_pv_in_vg(const struct volume_group *vg,return NULL ; } +struct volume_group *find_vg_by_pv (const struct physical_volume *pv) +{ +return pv->vg; +} +struct pv_list *find_pv_in_pv_list (const struct dm_list *pl,const struct physical_volume *pv) {
Signature · 1 line
1.6.0.6
PATCH 9/21 Add mda_locn_match() internal library function.
A metadata_area is defined independent of the location. One downside is that there is no obvious mapping from a pv to an mda. For a PV in a VG, we need a way to start with a PV and end up with an MDA, if we are to manage mdas. This function provides us a way to go down the list of PVs on a VG, and identify which ones match a particular PV.
I'm not entirely happy with this approach, but it does fit into the existing structures in a reasonable way.
A better solution might be to refactor the VG - PV interface such that mdas are a list tied to a PV. However, this seemed a bit tricky since a PV does not come into existence until after the list of mdas is constructed (see _vg_read() - we create a 'fid' and attach mdas to it, then we go through them and attach pvs).
--- lib/format_text/format-text.c | 15 +++++++++++++++ lib/metadata/metadata.c | 30 ++++++++++++++++++++++++++++++ lib/metadata/metadata.h | 7 +++++++ 3 files changed, 52 insertions(+), 0 deletions(-)
lib/format_text/format-text.c +15 -0
@@ -121,6 +121,20 @@ static int _mda_in_vg_raw(struct format_instance *fid __attribute((unused)),return 0 ; } +static int _mda_locn_match_raw (struct metadata_area *mda1, +struct metadata_area *mda2) +{ +struct mda_context *mda1c = (struct mda_context *) mda1->metadata_locn; +struct mda_context *mda2c = (struct mda_context *) mda2->metadata_locn; + +if ((mda1c->area.dev == mda2c->area.dev) && + (mda1c->area.start == mda2c->area.start) && + (mda1c->area.size == mda2c->area.size)) +return 1 ; + +return 0 ; +} +/* * For circular region between region_start and region_start + region_size, * back up one SECTOR_SIZE from 'region_ptr' and return the value.
@@ -1705,6 +1719,7 @@ static struct metadata_area_ops _metadata_text_raw_ops = { .mda_total_sectors = _mda_total_sectors_raw, .mda_in_vg = _mda_in_vg_raw, .pv_analyze_mda = _pv_analyze_mda_raw, + .mda_locn_match = _mda_locn_match_raw };/* pvmetadatasize in sectors */
lib/metadata/metadata.c +30 -0
@@ -3633,6 +3633,36 @@ uint32_t pv_pe_alloc_count(const struct physical_volume *pv)return pv_field (pv, pe_alloc_count); } +/* +* This function provides a way to answer the question on a format specific +* basis - does the format specfic context of these two metadata areas +* match? +* +* A metatdata_area is defined to be independent of the underlying context. +* This has the benefit that we can use the same abstraction to read disks +* (see _metadata_text_raw_ops) or files (see _metadata_text_file_ops). +* However, one downside is there is no format-independent way to determine +* whether a given metadata_area is attached to a specific device - in fact, +* it may not be attached to a device at all. +* +* Thus, LVM is structured such that an mda is not a member of struct +* physical_volume. The location of the mda depends on whether +* the PV is in a volume group. A PV not in a VG has an mda on the +* 'info->mda' list in lvmcache, while a PV in a VG has an mda on +* the vg->fid->metadata_areas list. For further details, see _vg_read(), +* and the sequence of creating the format_instance with fid->metadata_areas +* list, as well as the construction of the VG, with list of PVs (comes +* after the construction of the fid and list of mdas). +*/ +int mda_locn_match (struct metadata_area *mda1,struct metadata_area *mda2) +{ +if (!mda1->ops->mda_locn_match || !mda2->ops->mda_locn_match || + mda1->ops->mda_locn_match != mda2->ops->mda_locn_match) +return 0 ; + +return mda1->ops->mda_locn_match (mda1, mda2); +} +int mda_is_disabled (struct metadata_area *mda) {return (mda->flags &MDA_DISABLED );
lib/metadata/metadata.h +7 -0
@@ -156,6 +156,12 @@ struct metadata_area_ops {int (*pv_analyze_mda) (const struct format_type * fmt,struct metadata_area *mda); +/* +* Do these two metadata_areas match with respect to their underlying +* location? +*/ +int (*mda_locn_match)(struct metadata_area *mda1, +struct metadata_area *mda2); };#define MDA_DISABLED 0x00000001
@@ -169,6 +175,7 @@ struct metadata_area {int mda_is_disabled (struct metadata_area *mda);void mda_set_disabled (struct metadata_area *mda,int value); +int mda_locn_match (struct metadata_area *mda1,struct metadata_area *mda2);#define seg_pvseg (seg, s) (seg)->areas[(s)].u.pv.pvseg#define seg_dev (seg, s) (seg)->areas[(s)].u.pv.pvseg->pv->dev
Signature · 1 line
1.6.0.6
PATCH 10/21 Define new functions and vgs/pvs fields related to mda disable/enable.
Define a new pvs field, pv_mda_count_disabled, and a new vgs field, vg_mda_count_disabled to match the existing pv_mda_count and vg_mda_count. Also define various supporting functions to implement the counting as well as setting the disabled flag and determining if an mda is disabled. These high level functions call into the lower level format independent mda disable functions defined by earlier patches.
--- lib/metadata/metadata-exported.h | 4 ++ lib/metadata/metadata.c | 89 ++++++++++++++++++++++++++++++++++++++ lib/report/columns.h | 2 + lib/report/report.c | 25 +++++++++++ 4 files changed, 120 insertions(+), 0 deletions(-)
lib/metadata/metadata-exported.h +4 -0
@@ -733,6 +733,9 @@ uint64_t pv_pe_start(const struct physical_volume *pv);uint32_t pv_pe_count (const struct physical_volume *pv);uint32_t pv_pe_alloc_count (const struct physical_volume *pv);uint32_t pv_mda_count (const struct physical_volume *pv); +uint32_t pv_mda_count_disabled (const struct physical_volume *pv); +uint32_t pv_mda_is_disabled (const struct physical_volume *pv); +int pv_mda_set_disabled (const struct physical_volume *pv,int value);uint64_t lv_size (const struct logical_volume *lv);
@@ -748,6 +751,7 @@ uint64_t vg_pv_count(const struct volume_group *vg);uint64_t vg_max_pv (const struct volume_group *vg);uint64_t vg_max_lv (const struct volume_group *vg);uint32_t vg_mda_count (const struct volume_group *vg); +uint32_t vg_mda_count_disabled (const struct volume_group *vg);int vg_check_write_mode (struct volume_group *vg);#define vg_is_clustered (vg) (vg_status((vg)) & CLUSTERED)#define vg_is_exported (vg) (vg_status((vg)) & EXPORTED_VG)
lib/metadata/metadata.c +89 -0
101 lines not on this page · raw message
lib/report/columns.h +2 -0
@@ -92,6 +92,7 @@ FIELD(PVS, pv, NUM, "PE", pe_count, 3, uint32, "pv_pe_count", "Total number of P FIELD(PVS, pv, NUM, "Alloc", pe_alloc_count, 5, uint32, "pv_pe_alloc_count", "Total number of allocated Physical Extents.") FIELD(PVS, pv, STR, "PV Tags", tags, 7, tags, "pv_tags", "Tags, if any.") FIELD(PVS, pv, NUM, "#PMda", id, 5, pvmdas, "pv_mda_count", "Number of metadata areas on this device.") +FIELD(PVS, pv, NUM, "#PMdaDis", id, 8, pvmdas_disabled, "pv_mda_count_disabled", "Number of metadata areas disabled on this device.") FIELD(VGS, vg, STR, "Fmt", cmd, 3, vgfmt, "vg_fmt", "Type of metadata.")FIELD (VGS , vg,STR ,"VG UUID" , id,38 , uuid,"vg_uuid" ,"Unique identifier." )
@@ -111,6 +112,7 @@ FIELD(VGS, vg, NUM, "#SN", cmd, 3, snapcount, "snap_count", "Number of snapshots FIELD(VGS, vg, NUM, "Seq", seqno, 3, uint32, "vg_seqno", "Revision number of internal metadata. Incremented whenever it changes.") FIELD(VGS, vg, STR, "VG Tags", tags, 7, tags, "vg_tags", "Tags, if any.") FIELD(VGS, vg, NUM, "#VMda", cmd, 5, vgmdas, "vg_mda_count", "Number of metadata areas in use by this VG.") +FIELD(VGS, vg, NUM, "#VMdaDis", cmd, 8, vgmdas_disabled, "vg_mda_count_disabled", "Number of metadata areas disabled in this VG.") FIELD(VGS, vg, NUM, "VMdaFree", cmd, 9, vgmdafree, "vg_mda_free", "Free metadata area space for this VG in current units.")FIELD (VGS , vg,NUM ,"VMdaSize" , cmd,9 , vgmdasize,"vg_mda_size" ,"Size of smallest metadata area for this VG in current units." )
lib/report/report.c +25 -0
@@ -861,6 +861,19 @@ static int _pvmdas_disp(struct dm_report *rh, struct dm_pool *mem,return _uint32_disp (rh, mem, field, &count, private); } +static int _pvmdas_disabled_disp (struct dm_report *rh,struct dm_pool *mem, +struct dm_report_field *field, +const void *data,void *private) +{ +uint32_t count; +const struct physical_volume *pv = + (const struct physical_volume *) data; + + count =pv_mda_count_disabled (pv); + +return _uint32_disp (rh, mem, field, &count, private); +} +static int _vgmdas_disp (struct dm_report *rh,struct dm_pool *mem,struct dm_report_field *field,const void *data,void *private)
@@ -873,6 +886,18 @@ static int _vgmdas_disp(struct dm_report *rh, struct dm_pool *mem,return _uint32_disp (rh, mem, field, &count, private); } +static int _vgmdas_disabled_disp (struct dm_report *rh,struct dm_pool *mem, +struct dm_report_field *field, +const void *data,void *private) +{ +const struct volume_group *vg = (const struct volume_group *) data; +uint32_t count; + + count =vg_mda_count_disabled (vg); + +return _uint32_disp (rh, mem, field, &count, private); +} +static int _pvmdafree_disp (struct dm_report *rh,struct dm_pool *mem,struct dm_report_field *field,const void *data,void *private)
Signature · 1 line
1.6.0.6
PATCH 11/21 Add --metadatadisable to pvchange, allowing for disabling of metadata areas.
This patch just modifies pvchange to call the underlying enable/disable functions for mdas.
--- tools/args.h | 1 + tools/commands.h | 3 ++- tools/pvchange.c | 30 +++++++++++++++++++++++++++++- 3 files changed, 32 insertions(+), 2 deletions(-)
tools/args.h +1 -0
@@ -26,6 +26,7 @@ arg(nolocking_ARG, '\0', "nolocking", NULL, 0) arg(pvmetadatacopies_ARG, '\0', "pvmetadatacopies", int_arg, 0) arg(metadatacopies_ARG, '\0', "metadatacopies", int_arg, 0) arg(metadatasize_ARG, '\0', "metadatasize", size_mb_arg, 0) +arg(metadatadisable_ARG, '\0', "metadatadisable", yes_no_arg, 0) arg(restorefile_ARG, '\0', "restorefile", string_arg, 0) arg(labelsector_ARG, '\0', "labelsector", int_arg, 0)arg (driverloaded_ARG,'\0' ,"driverloaded" , yes_no_arg,0 )
tools/commands.h +2 -1
@@ -448,6 +448,7 @@ xx(pvchange,"\t[-t|--test]\n" "\t[-u|--uuid]\n" "\t[-x|--allocatable y|n]\n" +"\t[--metadatadisable y|n]\n" "\t[-v|--verbose]\n" "\t[--addtag Tag]\n" "\t[--deltag Tag]\n"
@@ -455,7 +456,7 @@ xx(pvchange, "\t[PhysicalVolumePath...]\n", all_ARG, allocatable_ARG, allocation_ARG, autobackup_ARG, deltag_ARG, - addtag_ARG, test_ARG, uuid_ARG) + addtag_ARG, metadatadisable_ARG, test_ARG, uuid_ARG) xx(pvresize, "Resize physical volume(s)",
tools/pvchange.c +29 -1
@@ -37,6 +37,7 @@ static int _pvchange_single(struct cmd_context *cmd, struct physical_volume *pv,int allocatable =0 ;int tagarg =0 ;int r =0 ; +int mda_disable =0 ;if (arg_count (cmd, addtag_ARG)) tagarg = addtag_ARG;
@@ -46,6 +47,9 @@ static int _pvchange_single(struct cmd_context *cmd, struct physical_volume *pv,if (arg_count (cmd, allocatable_ARG)) allocatable = !strcmp (arg_str_value (cmd, allocatable_ARG,"n" ),"y" ); +if (arg_count (cmd, metadatadisable_ARG)) + mda_disable = !strcmp (arg_str_value (cmd, metadatadisable_ARG,"n" ), +"y" );else if (tagarg && !(tag =arg_str_value (cmd, tagarg,NULL ))) {log_error ("Failed to get tag" );return 0 ;
@@ -149,6 +153,29 @@ static int _pvchange_single(struct cmd_context *cmd, struct physical_volume *pv, goto out; } } + }else if (arg_count (cmd, metadatadisable_ARG)) { +if (mda_disable &&pv_mda_is_disabled (pv)) { +log_error ("Physical volume \"%s\" mdas already " +"disabled" , pv_name); + r =1 ; + goto out; + } +if (!mda_disable && !pv_mda_is_disabled (pv)) { +log_error ("Physical volume \"%s\" mdas already " +"enabled" , pv_name); + r =1 ; + goto out; + } +if (mda_disable) { +log_verbose ("Setting physical volume \"%s\" " +"metadata disable" , pv_name); + }else { +log_verbose ("Setting physical volume \"%s\" " +"metadata enable" , pv_name); + } +if (!pv_mda_set_disabled (pv, mda_disable)) { + goto out; + } }else {/* --uuid: Change PV ID randomly */ if (!id_create (&pv->id)) {
@@ -220,7 +247,8 @@ int pvchange(struct cmd_context *cmd, int argc, char **argv)struct dm_list mdas;if (arg_count (cmd, allocatable_ARG) +arg_count (cmd, addtag_ARG) + -arg_count (cmd, deltag_ARG) +arg_count (cmd, uuid_ARG) !=1 ) { +arg_count (cmd, deltag_ARG) +arg_count (cmd, uuid_ARG) + +arg_count (cmd, metadatadisable_ARG) !=1 ) {log_error ("Please give exactly one option of -x, -uuid, " "--addtag or --deltag" );return EINVALID_CMD_LINE ;
Signature · 1 line
1.6.0.6
PATCH 12/21 Update pvchange, pvs and vgs man pages for metadata disable.
Explain --metadatadisable argument to pvchange and new fields to pvs / vgs.
--- man/pvchange.8.in | 6 +++++- man/pvs.8.in | 3 ++- man/vgs.8.in | 2 +- 3 files changed, 8 insertions(+), 3 deletions(-)
man/pvchange.8.in +5 -1
@@ -6,6 +6,7 @@ pvchange \- change attributes of a physical volume [\-\-addtag Tag] [\-A|\-\-autobackup y|n] [\-d|\-\-debug] [\-\-deltag Tag] +[\-\-metadatadisable y|n] [\-h|\-?|\-\-help] [\-t|\-\-test] [\-v|\-\-verbose] [\-a|\-\-all] [\-x|\-\-allocatable y|n]
@@ -25,10 +26,13 @@ Generate new random UUID for specified physical volumes. .TP .I \-x, \-\-allocatable y|n Enable or disable allocation of physical extents on this physical volume. +.TP +.I \-\-metadatadisable y|n +Disable or enable metadata areas on this physical volume. .SH Example "pvchange -x n /dev/sdk1" disallows the allocation of physical extents on this physical volume (possibly because of disk errors, or because it will be removed after freeing it. .SH SEE ALSO -.BR lvm (8), +.BR lvm (8), .BR pvcreate (8)
man/pvs.8.in +2 -1
@@ -54,7 +54,8 @@ Use \fb-o help\fP to view the full list of columns available. .IP Column names include: pv_fmt, pv_uuid, dev_size, pv_name, pv_mda_free, pv_mda_size, pe_start, pv_size, pv_free, pv_used, pv_attr, pv_pe_count, -pv_pe_alloc_count, pv_tags, pv_mda_count, pvseg_start, and pvseg_size. +pv_pe_alloc_count, pv_tags, pv_mda_count, pv_mda_count_disabled, +pvseg_start, and pvseg_size. .IP With --segments, any "pvseg_" prefixes are optional; otherwise any "pv_" prefixes are optional. Columns mentioned in \fBvgs (8)\fP can also
man/vgs.8.in +1 -1
@@ -49,7 +49,7 @@ Use \fb-o help\fP to view the full list of columns available. Column names include: vg_fmt, vg_uuid, vg_name, vg_attr, vg_size, vg_free, vg_sysid, vg_extent_size, vg_extent_count, vg_free_count, max_lv, max_pv, pv_count, lv_count, snap_count, vg_seqno, vg_tags, vg_mda_count, vg_mda_free, -and vg_mda_size. +and vg_mda_size, vg_mda_count_disabled. .IP Any "vg_" prefixes are optional. Columns mentioned in either \fBpvs (8)\fP or \fBlvs (8)\fP can also be chosen, but columns cannot be taken from both
Signature · 1 line
1.6.0.6
PATCH 13/21 Add tests for metadata balance - set/clear/display per-PV disable bit.
--- test/t-metadata-balance.sh | 41 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 41 insertions(+), 0 deletions(-) create mode 100755 test/t-metadata-balance.sh
test/t-metadata-balance.sh new file +41 -0
@@ -0,0 +1,41 @@ +# Copyright (C) 2008 Red Hat, Inc. All rights reserved. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions +# of the GNU General Public License v.2. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +. ./test-utils.sh + +aux prepare_devs 5 + +# Make sure we can disable/enable mdas on a per-PV basis +for pv_in_vg in 1 0; do +for mdacp in 1 2; do + pvcreate --metadatacopies $mdacp $dev1 $dev2 + pvcreate --metadatacopies 0 $dev3 + if [ $pv_in_vg = 1 ]; then + vgcreate -c n "$vg" $dev1 $dev2 $dev3 + fi + pvchange --metadatadisable y $dev1 + check_pv_field_ $dev1 pv_mda_count $mdacp + check_pv_field_ $dev1 pv_mda_count_disabled $mdacp + check_pv_field_ $dev2 pv_mda_count $mdacp + check_pv_field_ $dev2 pv_mda_count_disabled 0 + if [ $pv_in_vg = 1 ]; then + check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) + check_vg_field_ $vg vg_mda_count_disabled $mdacp + fi + pvchange --metadatadisable n $dev1 + check_pv_field_ $dev1 pv_mda_count $mdacp + check_pv_field_ $dev1 pv_mda_count_disabled 0 + if [ $pv_in_vg = 1 ]; then + check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) + check_vg_field_ $vg vg_mda_count_disabled 0 + vgremove -f $vg + fi +done +done
Signature · 1 line
1.6.0.6
PATCH 14/21 Add metadata balance arguments, update vgcreate man page and help.
The 3 parameters for metadata balancing are added here. We really only need the 'min' and 'max' values, since the defaults are really what LVM does today (you must have metadata on at least on PV, and by default we read/write from all PVs that have metadatacopies != 0). The only question is how to explain this properly, so for now I've left the main 'enable' parameter.
--- man/vgcreate.8.in | 24 ++++++++++++++++++++++++ tools/args.h | 3 +++ tools/commands.h | 8 ++++++-- 3 files changed, 33 insertions(+), 2 deletions(-)
man/vgcreate.8.in +24 -0
@@ -14,6 +14,11 @@ vgcreate \- create a volume group .RB [ \-l | \-\-maxlogicalvolumes .IR MaxLogicalVolumes ] .RB [ -M | \-\-metadatatype type] +.RB [ \-\-metadatabalance " {" y | n } ] +.RB [ \-\-metadatabalancemin +.IR NumberOfPhysicalVolumes ] +.RB [ \-\-metadatabalancemax +.IR NumberOfPhysicalVolumes ] .RB [ -p | \-\-maxphysicalvolumes .IR MaxPhysicalVolumes ] .RB [ \-s | \-\-physicalextentsize
@@ -94,6 +99,25 @@ but having a large number of extents will slow down the tools but have no impact on I/O performance to the logical volume. The smallest PE is 1KB. The 2.4 kernel has a limitation of 2TB per block device. +.TP +.BR \-\-metadatabalance " " { y | n } +Enables automatic metadata balancing on physical volumes in the volume group. +If set to \fBy\fP, LVM will use the values of \fBmetadatabalancmin\fP and +\fBmetadatabalancemax\fP to determine the set of physical volumes to read and +write metadata. Only physical volumes created with a non-zero value of +\fBmetadatacopies\fP are used. This option is useful for volume groups +containing large numbers of physical volumes. The option is set to \fBn\fP +by default, and all physical volumes created with non-zero \fBmetadatacopies\fP +will contain metadata. +.TP +.BR \-\-metadatabalancemin " " \fIMinPhysicalVolumes\fR +Sets the minimum number of physical volumes that should contain metadata. +Default is 1. +.TP +.BR \-\-metadatabalancemax " " \fIMaxPhysicalVolumes\fR +Sets the maximum number of physical volumes that should contain metadata. +Default is all physical volumes. +\fB .SH PHYSICAL DEVICE OPTIONS The following options are available for initializing physical devices in the volume group. These options are further described in the pvcreate man page.
tools/args.h +3 -0
@@ -24,6 +24,9 @@ arg(physicalvolumesize_ARG, '\0', "setphysicalvolumesize", size_mb_arg, 0) arg(ignorelockingfailure_ARG, '\0', "ignorelockingfailure", NULL, 0) arg(nolocking_ARG, '\0', "nolocking", NULL, 0) arg(pvmetadatacopies_ARG, '\0', "pvmetadatacopies", int_arg, 0) +arg(metadatabalance_ARG, '\0', "metadatabalance", yes_no_arg, 0) +arg(metadatabalancemin_ARG, '\0', "metadatabalancemin", int_arg, 0) +arg(metadatabalancemax_ARG, '\0', "metadatabalancemax", int_arg, 0) arg(metadatacopies_ARG, '\0', "metadatacopies", int_arg, 0) arg(metadatasize_ARG, '\0', "metadatasize", size_mb_arg, 0)arg (metadatadisable_ARG,'\0' ,"metadatadisable" , yes_no_arg,0 )
tools/commands.h +6 -2
@@ -766,6 +766,9 @@ xx(vgcreate,"\t[-d|--debug]" "\n" "\t[-h|--help]" "\n" "\t[-l|--maxlogicalvolumes MaxLogicalVolumes]" "\n" +"\t[--metadatabalance {y|n}] " "\n" +"\t[--metadatabalancemin NumberOfPhysicalVolumes] " "\n" +"\t[--metadatabalancemax NumberOfPhysicalVolumes] " "\n" "\t[-M|--metadatatype 1|2] " "\n" "\t[-p|--maxphysicalvolumes MaxPhysicalVolumes] " "\n" "\t[-s|--physicalextentsize PhysicalExtentSize[bBsSkKmMgGtTpPeE]] " "\n"
@@ -778,8 +781,9 @@ xx(vgcreate, addtag_ARG, alloc_ARG, autobackup_ARG, clustered_ARG, maxlogicalvolumes_ARG, maxphysicalvolumes_ARG, metadatatype_ARG, physicalextentsize_ARG, test_ARG, force_ARG, yes_ARG, zero_ARG, labelsector_ARG, metadatasize_ARG, - pvmetadatacopies_ARG, metadatacopies_ARG, dataalignment_ARG, - dataalignmentoffset_ARG) + pvmetadatacopies_ARG, metadatacopies_ARG, + metadatabalance_ARG, metadatabalancemin_ARG, metadatabalancemax_ARG, + dataalignment_ARG, dataalignmentoffset_ARG) xx(vgdisplay, "Display volume group information",
Signature · 1 line
1.6.0.6
PATCH 15/21 Add vg metadata balance parameters with defaults.
Add 3 parameters that control vg metadata balancing.
1) metadata_balance: enable metadata balancing; default 0 (disable)
2) metadata_balance_min: min # of PVs to contain metadata; default 1
3) metadata_balance_max: max # of PVs to contain metadata; default UINT_MAX (all PVs contain metadata)
This patch just adds the parameters to the structures with the default values but does not modify any commands. Should be no functional change.
--- lib/config/defaults.h | 3 +++ lib/metadata/metadata-exported.h | 9 +++++++++ lib/metadata/metadata.c | 3 +++ tools/toollib.c | 6 ++++++ 4 files changed, 21 insertions(+), 0 deletions(-)
lib/config/defaults.h +3 -0
@@ -74,6 +74,9 @@#define DEFAULT_MAX_LV 0#define DEFAULT_ALLOC_POLICY ALLOC_NORMAL#define DEFAULT_CLUSTERED 0 +#define DEFAULT_METADATA_BALANCE 0 +#define DEFAULT_METADATA_BALANCE_MIN 1 +#define DEFAULT_METADATA_BALANCE_MAX UINT32_MAX#define DEFAULT_MSG_PREFIX " "#define DEFAULT_CMD_NAME 0
lib/metadata/metadata-exported.h +9 -0
@@ -263,6 +263,12 @@ struct volume_group {* 0 for success else appropriate FAILURE_* bits set. */ uint32_t read_status; +/* +* Metadata balance parameters +*/ +uint32_t metadata_balance;/* 0 - disable (default); 1 - enable */ +uint32_t metadata_balance_min;/* min # of mdas in this VG */ +uint32_t metadata_balance_max;/* max # of mdas in this VG */ };/* There will be one area for each stripe */
@@ -767,6 +773,9 @@ struct vgcreate_params {size_t max_lv;alloc_policy_t alloc;int clustered;/* FIXME: put this into a 'status' variable instead? */ +int metadata_balance; +uint32_t metadata_balance_max; +uint32_t metadata_balance_min; }; int vgcreate_params_validate(struct cmd_context *cmd,
lib/metadata/metadata.c +3 -0
@@ -832,6 +832,9 @@ struct volume_group *vg_create(struct cmd_context *cmd, const char *vg_name) vg->max_pv =DEFAULT_MAX_PV ; vg->alloc =DEFAULT_ALLOC_POLICY ; + vg->metadata_balance =DEFAULT_METADATA_BALANCE ; + vg->metadata_balance_min =DEFAULT_METADATA_BALANCE_MIN ; + vg->metadata_balance_max =DEFAULT_METADATA_BALANCE_MAX ; vg->pv_count =0 ;dm_list_init (&vg->pvs);
tools/toollib.c +6 -0
@@ -1189,6 +1189,9 @@ void vgcreate_params_set_defaults(struct vgcreate_params *vp_def, vp_def->max_lv = vg->max_lv; vp_def->alloc = vg->alloc; vp_def->clustered =vg_is_clustered (vg); + vp_def->metadata_balance = vg->metadata_balance; + vp_def->metadata_balance_min = vg->metadata_balance_min; + vp_def->metadata_balance_max = vg->metadata_balance_max; }else { vp_def->vg_name =NULL ; vp_def->extent_size =DEFAULT_EXTENT_SIZE *2 ;
@@ -1196,6 +1199,9 @@ void vgcreate_params_set_defaults(struct vgcreate_params *vp_def, vp_def->max_lv =DEFAULT_MAX_LV ; vp_def->alloc =DEFAULT_ALLOC_POLICY ; vp_def->clustered =DEFAULT_CLUSTERED ; + vp_def->metadata_balance =DEFAULT_METADATA_BALANCE ; + vp_def->metadata_balance_min =DEFAULT_METADATA_BALANCE_MIN ; + vp_def->metadata_balance_max =DEFAULT_METADATA_BALANCE_MAX ; } }
Signature · 1 line
1.6.0.6
PATCH 16/21 Add vg get/set methods for metadata balance min/max.
This patch adds the get and partially implemented set functions. The 'set' functions should probably enable or disable metadata areas based on new values.
--- lib/metadata/metadata-exported.h | 4 +++ lib/metadata/metadata.c | 55 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+), 0 deletions(-)
lib/metadata/metadata-exported.h +4 -0
@@ -758,6 +758,10 @@ uint64_t vg_max_pv(const struct volume_group *vg);uint64_t vg_max_lv (const struct volume_group *vg);uint32_t vg_mda_count (const struct volume_group *vg);uint32_t vg_mda_count_disabled (const struct volume_group *vg); +uint32_t vg_mda_balance_min (const struct volume_group *vg); +uint32_t vg_mda_balance_max (const struct volume_group *vg); +int vg_set_mda_balance_min (struct volume_group *vg,uint32_t value); +int vg_set_mda_balance_max (struct volume_group *vg,uint32_t value);int vg_check_write_mode (struct volume_group *vg);#define vg_is_clustered (vg) (vg_status((vg)) & CLUSTERED)#define vg_is_exported (vg) (vg_status((vg)) & EXPORTED_VG)
lib/metadata/metadata.c +55 -0
61 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 17/21 Make metadata_balance_{min|max} persistent on disk.
This patch adds the ability to read/write the metadata balance min/max values to the vg metadata.
For existing metdata that does not have a value, we set to the default values for min/max. In the case of the metadata balance max default value, set the max to the # of PVs. This should preserve existing LVM behavior and ensure backward compatibility.
--- lib/format_text/export.c | 2 ++ lib/format_text/import_vsn1.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+), 0 deletions(-)
lib/format_text/export.c +2 -0
@@ -397,6 +397,8 @@ static int _print_vg(struct formatter *f, struct volume_group *vg)outf (f,"allocation_policy = \"%s\"" ,get_alloc_string (vg->alloc)); } +outf (f,"metadata_balance_min = %u" , vg->metadata_balance_min); +outf (f,"metadata_balance_max = %u" , vg->metadata_balance_max);return 1 ; }
lib/format_text/import_vsn1.c +14 -0
@@ -23,6 +23,7 @@#include "pv_alloc.h" #include "segtype.h" #include "text_import.h" +#include "defaults.h" typedef int (*section_fn) (struct format_instance * fid, struct dm_pool * mem, struct volume_group * vg, struct config_node * pvn,
@@ -729,6 +730,16 @@ static struct volume_group *_read_vg(struct format_instance *fid, return_0; } +if (!_read_uint32 (vgn,"metadata_balance_min" , + &vg->metadata_balance_min)) { + vg->metadata_balance_min =DEFAULT_METADATA_BALANCE_MIN ; + } + +if (!_read_uint32 (vgn,"metadata_balance_max" , + &vg->metadata_balance_max)) { + vg->metadata_balance_max =DEFAULT_METADATA_BALANCE_MAX ; + } +/* * The pv hash memoises the pv section names -> pv * structures.
@@ -746,6 +757,9 @@ static struct volume_group *_read_vg(struct format_instance *fid, goto bad; } +if (vg->metadata_balance_max ==DEFAULT_METADATA_BALANCE_MAX ) + vg->metadata_balance_max =dm_list_size (&vg->pvs); +dm_list_init (&vg->lvs);dm_list_init (&vg->tags);dm_list_init (&vg->removed_pvs);
Signature · 1 line
1.6.0.6
PATCH 18/21 Add vg_mda_balance_{min|max} display fields to 'vgs' command.
--- lib/report/columns.h | 2 ++ lib/report/report.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 0 deletions(-)
lib/report/columns.h +2 -0
@@ -115,6 +115,8 @@ FIELD(VGS, vg, NUM, "#VMda", cmd, 5, vgmdas, "vg_mda_count", "Number of metadata FIELD(VGS, vg, NUM, "#VMdaDis", cmd, 8, vgmdas_disabled, "vg_mda_count_disabled", "Number of metadata areas disabled in this VG.") FIELD(VGS, vg, NUM, "VMdaFree", cmd, 9, vgmdafree, "vg_mda_free", "Free metadata area space for this VG in current units.") FIELD(VGS, vg, NUM, "VMdaSize", cmd, 9, vgmdasize, "vg_mda_size", "Size of smallest metadata area for this VG in current units.") +FIELD(VGS, vg, NUM, "#VMdaMin", cmd, 8, vgmda_balance_min, "vg_mda_balance_min", "Minimum number of PVs on which to read/write metadata.") +FIELD(VGS, vg, NUM, "#VMdaMax", cmd, 8, vgmda_balance_max, "vg_mda_balance_max", "Maximum number of PVs on which to read/write metadata.") FIELD(SEGS, seg, STR, "Type", list, 4, segtype, "segtype", "Type of LV segment.")FIELD (SEGS , seg,NUM ,"#Str" , area_count,4 , uint32,"stripes" ,"Number of stripes or mirror legs." )
lib/report/report.c +24 -0
@@ -898,6 +898,30 @@ static int _vgmdas_disabled_disp(struct dm_report *rh, struct dm_pool *mem,return _uint32_disp (rh, mem, field, &count, private); } +static int _vgmda_balance_min_disp (struct dm_report *rh,struct dm_pool *mem, +struct dm_report_field *field, +const void *data,void *private) +{ +const struct volume_group *vg = (const struct volume_group *) data; +uint32_t count; + + count =vg_mda_balance_min (vg); + +return _uint32_disp (rh, mem, field, &count, private); +} + +static int _vgmda_balance_max_disp (struct dm_report *rh,struct dm_pool *mem, +struct dm_report_field *field, +const void *data,void *private) +{ +const struct volume_group *vg = (const struct volume_group *) data; +uint32_t count; + + count =vg_mda_balance_max (vg); + +return _uint32_disp (rh, mem, field, &count, private); +} +static int _pvmdafree_disp (struct dm_report *rh,struct dm_pool *mem,struct dm_report_field *field,const void *data,void *private)
Signature · 1 line
1.6.0.6
PATCH 19/21 Add vg metadata balance parameters to vgchange man page and help.
--- man/vgchange.8.in | 23 +++++++++++++++++++++++ tools/commands.h | 4 ++++ 2 files changed, 27 insertions(+), 0 deletions(-)
man/vgchange.8.in +23 -0
@@ -24,6 +24,11 @@ vgchange \- change attributes of a volume group .IR MaxLogicalVolumes ] .RB [ -p | \-\-maxphysicalvolumes .IR MaxPhysicalVolumes ] +.RB [ \-\-metadatabalance " {" y | n } ] +.RB [ \-\-metadatabalancemin +.IR NumberOfPhysicalVolumes ] +.RB [ \-\-metadatabalancemax +.IR NumberOfPhysicalVolumes ] .RB [ \-P | \-\-partial] .RB [ \-s | \-\-physicalextentsize .IR PhysicalExtentSize [ \fBbBsSkKmMgGtTpPeE\fR ]]
@@ -155,6 +160,24 @@ manually without a clustered lock manager. .BR \-x ", " \-\-resizeable " " { y | n } Enables or disables the extension/reduction of this volume group with/by physical volumes. +.TP +.BR \-\-metadatabalance " " { y | n } +Enables automatic metadata balancing on physical volumes in the volume group. +If set to \fBy\fP, LVM will use the values of \fBmetadatabalancmin\fP and +\fBmetadatabalancemax\fP to determine the set of physical volumes to read and +write metadata. Only physical volumes created with a non-zero value of +\fBmetadatacopies\fP are used. This option is useful for volume groups +containing large numbers of physical volumes. The option is set to \fBn\fP +by default, and all physical volumes created with non-zero \fBmetadatacopies\fP +will contain metadata. +.TP +.BR \-\-metadatabalancemin " " \fIMinPhysicalVolumes\fR +Sets the minimum number of physical volumes that should contain metadata. +Default is 1. +.TP +.BR \-\-metadatabalancemax " " \fIMaxPhysicalVolumes\fR +Sets the maximum number of physical volumes that should contain metadata. +Default is all physical volumes. .SH EXAMPLES To activate all known volume groups in the system: .nf
tools/commands.h +4 -0
@@ -717,6 +717,9 @@ xx(vgchange,"\t -l|--logicalvolume MaxLogicalVolumes |" "\n" "\t -p|--maxphysicalvolumes MaxPhysicalVolumes |" "\n" "\t -s|--physicalextentsize PhysicalExtentSize[bBsSkKmMgGtTpPeE] |" "\n" +"\t[--metadatabalance {y|n}] " "\n" +"\t[--metadatabalancemin NumberOfPhysicalVolumes] " "\n" +"\t[--metadatabalancemax NumberOfPhysicalVolumes] " "\n" "\t --addtag Tag |\n" "\t --deltag Tag}\n" "\t[VolumeGroupName...]\n",
@@ -724,6 +727,7 @@ xx(vgchange, addtag_ARG, alloc_ARG, allocation_ARG, autobackup_ARG, available_ARG, clustered_ARG, deltag_ARG, ignorelockingfailure_ARG, ignoremonitoring_ARG, logicalvolume_ARG, maxphysicalvolumes_ARG, monitor_ARG, noudevsync_ARG, + metadatabalance_ARG, metadatabalancemin_ARG, metadatabalancemax_ARG, partial_ARG, physicalextentsize_ARG, poll_ARG, refresh_ARG, resizeable_ARG, resizable_ARG, test_ARG, uuid_ARG)
Signature · 1 line
1.6.0.6
PATCH 20/21 Update vgchange tool to handle metadata balance min/max.
--- tools/vgchange.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 76 insertions(+), 1 deletions(-)
tools/vgchange.c +76 -1
@@ -532,6 +532,74 @@ static int _vgchange_refresh(struct cmd_context *cmd, struct volume_group *vg)return ECMD_PROCESSED ; } +static int _vgchange_metadata_balance_min (struct cmd_context *cmd, +struct volume_group *vg) +{ +uint32_t value; + + value =arg_uint_value (cmd, metadatabalancemin_ARG,0 ); +if (value ==vg_mda_balance_min (vg)) { +log_error ("Metadata balance min of VG %s is already %u" , + vg->name, value); +return ECMD_PROCESSED ; + } + +if (!archive (vg)) { + stack; +return ECMD_FAILED ; + } + +if (!vg_set_mda_balance_min (vg, value)) { + stack; +return EINVALID_CMD_LINE ; + } + +if (!vg_write (vg) || !vg_commit (vg)) { + stack; +return ECMD_FAILED ; + } + +backup (vg); + +log_print ("Volume group \"%s\" successfully changed" , vg->name); + +return ECMD_PROCESSED ; +} + +static int _vgchange_metadata_balance_max (struct cmd_context *cmd, +struct volume_group *vg) +{ +uint32_t value; + + value =arg_uint_value (cmd, metadatabalancemax_ARG,0 ); +if (value ==vg_mda_balance_max (vg)) { +log_error ("Metadata balance max of VG %s is already %u" , + vg->name, value); +return ECMD_PROCESSED ; + } + +if (!archive (vg)) { + stack; +return ECMD_FAILED ; + } + +if (!vg_set_mda_balance_max (vg, value)) { + stack; +return EINVALID_CMD_LINE ; + } + +if (!vg_write (vg) || !vg_commit (vg)) { + stack; +return ECMD_FAILED ; + } + +backup (vg); + +log_print ("Volume group \"%s\" successfully changed" , vg->name); + +return ECMD_PROCESSED ; +} +static int vgchange_single (struct cmd_context *cmd,const char *vg_name,struct volume_group *vg,void *handle __attribute((unused)))
@@ -596,6 +664,12 @@ static int vgchange_single(struct cmd_context *cmd, const char *vg_name,else if (arg_count (cmd, refresh_ARG)) r =_vgchange_refresh (cmd, vg); +else if (arg_count (cmd, metadatabalancemin_ARG)) + r =_vgchange_metadata_balance_min (cmd, vg); + +else if (arg_count (cmd, metadatabalancemax_ARG)) + r =_vgchange_metadata_balance_max (cmd, vg); +return r; }
@@ -609,7 +683,8 @@ int vgchange(struct cmd_context *cmd, int argc, char **argv) arg_count(cmd, physicalextentsize_ARG) + arg_count(cmd, clustered_ARG) + arg_count(cmd, alloc_ARG) + arg_count(cmd, monitor_ARG) + arg_count(cmd, poll_ARG) + - arg_count(cmd, refresh_ARG))) { + arg_count(cmd, refresh_ARG) + arg_count(cmd, metadatabalancemin_ARG) + + arg_count(cmd, metadatabalancemax_ARG))) {log_error ("Need 1 or more of -a, -c, -l, -p, -s, -x, " "--refresh, --uuid, --alloc, --addtag, --deltag, " "--monitor or --poll" );
Signature · 1 line
1.6.0.6
PATCH 21/21 Update metadatabalance tests to include sanity checks for vgchange.
--- test/t-metadata-balance.sh | 25 +++++++++++++++++++++++++ 1 files changed, 25 insertions(+), 0 deletions(-)
test/t-metadata-balance.sh +25 -0
@@ -28,6 +28,8 @@ for mdacp in 1 2; do if [ $pv_in_vg = 1 ]; then check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) check_vg_field_ $vg vg_mda_count_disabled $mdacp + check_vg_field_ $vg vg_mda_balance_min 1 + check_vg_field_ $vg vg_mda_balance_max 3 fi pvchange --metadatadisable n $dev1 check_pv_field_ $dev1 pv_mda_count $mdacp
@@ -35,7 +37,30 @@ for mdacp in 1 2; do if [ $pv_in_vg = 1 ]; then check_vg_field_ $vg vg_mda_count $(($mdacp * 2)) check_vg_field_ $vg vg_mda_count_disabled 0 + check_vg_field_ $vg vg_mda_balance_min 1 + check_vg_field_ $vg vg_mda_balance_max 3 + # Sanity test of metadata balance params + # 'min' threshold (1 < min <= max) + # 'max' threshold (min <= max <= #pvs) + not vgchange --metadatabalancemin 0 $vg + not vgchange --metadatabalancemin 4 $vg + vgchange --metadatabalancemin 2 $vg + check_vg_field_ $vg vg_mda_balance_min 2 + not vgchange --metadatabalancemax 4 $vg + vgchange --metadatabalancemax 2 $vg + check_vg_field_ $vg vg_mda_balance_max 2 + vgchange --metadatabalancemax 3 $vg + check_vg_field_ $vg vg_mda_balance_max 3 + vgchange --metadatabalancemin 3 $vg + check_vg_field_ $vg vg_mda_balance_min 3 vgremove -f $vg fi done done + +# Test behavior of metadata balance min / max +# ensure setting min/max affects underlying pv enable/disable + +# Setting a larger min value should trigger enabling mdas +# Setting a smaller max value should trigger disabling mdas +# vg_mda_count_disabled should always be consistent with min/max
Signature · 1 line
1.6.0.6